Latest in Gear

Image credit: Microsoft

Microsoft app helps blind people ‘see’ the world with AI

The app recognizes currency, speaks text and even identifies visual emotion.
400 Shares
Share
Tweet
Share

Sponsored Links

Microsoft

Microsoft has launched an iPhone app designed to help blind and partially-sighted people better navigate the world. The app, Seeing AI, uses 'computer vision' to narrate the user's surroundings, read text, describe scenes and even identify friends' facial cues.

The project has been in the works since September 2016; in March this year, Microsoft demonstrated a prototype of the app for the first time. It uses neural networks, similar to the technology found in self-driving cars, to identify its environment and speak its observations out loud.

Point your phone camera at a friend and it'll tell you who they are. Aim it toward a short piece of text such as a name badge or room number and it'll speak it instantly -- a marked step up from the optical character recognition (OCR) technology of yore. Plus, it guides the user into capturing the object in question correctly, telling them to move the camera left or right to get the target in shot.

The app also recognizes currency, identifies products via their barcodes and, through an experimental feature, can describe entire scenes, such as a man walking a dog or food cooking on a stove. Basic tasks can be carried out directly within the app, without the need for an internet connection. It's currently available to download for free in the US on iOS, but there's no indication when it'll come to other platforms or countries.

In a blog post by Harry Shum, executive vice president of Microsoft's AI and research group, the company explains that Seeing AI is "just the beginning" for this kind of AI application. Machine learning, perception and natural language processing have evolved over time into separate fields of research, it says, but "we believe AI will be even more helpful when we can create tools that combine those functions."

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
400 Shares
Share
Tweet
Share

Popular on Engadget

Fossil's latest Wear OS watches now make calls using iPhones

Fossil's latest Wear OS watches now make calls using iPhones

View
Toyota will debut its tiny city EV at the 2019 Tokyo Motor Show

Toyota will debut its tiny city EV at the 2019 Tokyo Motor Show

View
Adidas readies an entire collection of Star Wars basketball shoes

Adidas readies an entire collection of Star Wars basketball shoes

View
Mercedes app was leaking car owners' data to other users

Mercedes app was leaking car owners' data to other users

View
AT&T hikes TV Now prices by as much as $15 per month

AT&T hikes TV Now prices by as much as $15 per month

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr