Latest in Gear

Image credit: Microsoft

Microsoft app helps blind people ‘see’ the world with AI

The app recognizes currency, speaks text and even identifies visual emotion.
400 Shares
Share
Tweet
Share
Save

Sponsored Links

Microsoft

Microsoft has launched an iPhone app designed to help blind and partially-sighted people better navigate the world. The app, Seeing AI, uses 'computer vision' to narrate the user's surroundings, read text, describe scenes and even identify friends' facial cues.

The project has been in the works since September 2016; in March this year, Microsoft demonstrated a prototype of the app for the first time. It uses neural networks, similar to the technology found in self-driving cars, to identify its environment and speak its observations out loud.

Point your phone camera at a friend and it'll tell you who they are. Aim it toward a short piece of text such as a name badge or room number and it'll speak it instantly -- a marked step up from the optical character recognition (OCR) technology of yore. Plus, it guides the user into capturing the object in question correctly, telling them to move the camera left or right to get the target in shot.

The app also recognizes currency, identifies products via their barcodes and, through an experimental feature, can describe entire scenes, such as a man walking a dog or food cooking on a stove. Basic tasks can be carried out directly within the app, without the need for an internet connection. It's currently available to download for free in the US on iOS, but there's no indication when it'll come to other platforms or countries.

In a blog post by Harry Shum, executive vice president of Microsoft's AI and research group, the company explains that Seeing AI is "just the beginning" for this kind of AI application. Machine learning, perception and natural language processing have evolved over time into separate fields of research, it says, but "we believe AI will be even more helpful when we can create tools that combine those functions."

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
400 Shares
Share
Tweet
Share
Save

Popular on Engadget

Engadget's 2019 Back-to-School Guide

Engadget's 2019 Back-to-School Guide

View
Google and Mozilla to block web surveillance in Kazakhstan

Google and Mozilla to block web surveillance in Kazakhstan

View
'Saturday Night Live' is coming to UK screens in early 2020

'Saturday Night Live' is coming to UK screens in early 2020

View
Watch and listen to THX's new Deep Note trailer with spatial 3D audio

Watch and listen to THX's new Deep Note trailer with spatial 3D audio

View
Facebook loses Oculus executive who led its mobile VR efforts

Facebook loses Oculus executive who led its mobile VR efforts

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr