Latest in Science

Image credit:

Google's AI is getting really good at captioning photos

It has open-sourced an algorithm that describes images with 94 percent accuracy.
Steve Dent, @stevetdent
September 23, 2016
Share
Tweet
Share

Sponsored Links

It's great to be an AI developer right now, but maybe not a good time to have a job that can be done by a machine. Take image captioning -- Google has released its "Show and Tell" algorithm to developers, who can train it recognize objects in photos with up to 93.9 percent accuracy. That's a significant improvement from just two years ago, when it could correctly classify 89.6 percent of images. Better photo descriptions can be used in numerous ways to help historians, visually impaired folks, and of course, other AI researchers, to name a few examples.

Google's open-source code release uses its third-gen "Inception" model and a new vision system that's better at picking out individual objects in a shot. The researchers also fine-tuned it for better accuracy. "For example, an image classification model will tell you that a dog, grass and a frisbee are in the image, but a natural description should also tell you the color of the grass and how the dog relates to the frisbee," the team wrote.

After it was trained using human captions, Google's system was able to describe images it hasn't seen before. "Excitingly, our model does indeed develop the ability to generate accurate new captions when presented with completely new scenes, indicating a deeper understanding of the objects and context in the images," the researchers say. Using several photos of dogs on beach (above), for instance, it was able to generate a caption for a similar, but slightly different scene.

Google has released the source code on its TensorFlow system to any interested parties. To use it, though, you'll have to train it yourself -- a process that could take a couple of weeks, assuming you have an NVIDIA Telsa GPU. So, if you were hoping to have it caption your Instagram collection, you'll need to wait for someone to release an already-trained model.

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
Tweet
Share

Popular on Engadget

Sony's PS5 DualSense controller has a built-in mic and adaptive triggers

Sony's PS5 DualSense controller has a built-in mic and adaptive triggers

View
Scientists visualize a black hole plasma jet in unprecedented detail

Scientists visualize a black hole plasma jet in unprecedented detail

View
Dell's XPS 15 and 17 leak with sleek new designs

Dell's XPS 15 and 17 leak with sleek new designs

View
See every square foot of asteroid Bennu, Earth's little frenemy

See every square foot of asteroid Bennu, Earth's little frenemy

View
'Call of Duty: Warzone' is introducing four-player squads

'Call of Duty: Warzone' is introducing four-player squads

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr