Latest in Gear

Image credit: Microsoft

Microsoft app helps blind people ‘see’ the world with AI

The app recognizes currency, speaks text and even identifies visual emotion.
400 Shares
Share
Tweet
Share
Save

Sponsored Links

Microsoft

Microsoft has launched an iPhone app designed to help blind and partially-sighted people better navigate the world. The app, Seeing AI, uses 'computer vision' to narrate the user's surroundings, read text, describe scenes and even identify friends' facial cues.

The project has been in the works since September 2016; in March this year, Microsoft demonstrated a prototype of the app for the first time. It uses neural networks, similar to the technology found in self-driving cars, to identify its environment and speak its observations out loud.

Point your phone camera at a friend and it'll tell you who they are. Aim it toward a short piece of text such as a name badge or room number and it'll speak it instantly -- a marked step up from the optical character recognition (OCR) technology of yore. Plus, it guides the user into capturing the object in question correctly, telling them to move the camera left or right to get the target in shot.

The app also recognizes currency, identifies products via their barcodes and, through an experimental feature, can describe entire scenes, such as a man walking a dog or food cooking on a stove. Basic tasks can be carried out directly within the app, without the need for an internet connection. It's currently available to download for free in the US on iOS, but there's no indication when it'll come to other platforms or countries.

In a blog post by Harry Shum, executive vice president of Microsoft's AI and research group, the company explains that Seeing AI is "just the beginning" for this kind of AI application. Machine learning, perception and natural language processing have evolved over time into separate fields of research, it says, but "we believe AI will be even more helpful when we can create tools that combine those functions."

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
400 Shares
Share
Tweet
Share
Save

Popular on Engadget

Engadget's Guide to Privacy

Engadget's Guide to Privacy

View
Neo Geo retro stick console includes 'King of Fighters,' 'Samurai Shodown'

Neo Geo retro stick console includes 'King of Fighters,' 'Samurai Shodown'

View
Watch the 'Android' Nokia phone that never had a chance to exist

Watch the 'Android' Nokia phone that never had a chance to exist

View
TiVo tries running pre-roll ads before your recorded shows

TiVo tries running pre-roll ads before your recorded shows

View
YouTube CEO apologizes for channel verification mess (updated)

YouTube CEO apologizes for channel verification mess (updated)

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr