Latest in Gear

Image credit: Jason Dorfman/MIT CSAIL

MIT's AI knows what's in your cookies just by looking at them

It could eventually help you track your daily nutrition.
284 Shares
Share
Tweet
Share
Save

Sponsored Links

Jason Dorfman/MIT CSAIL

Imagine an app that can help you figure out how to replicate what you're eating in a restaurant and help track your calorie intake just by taking a picture of your plate. A team of MIT CSAIL researchers have developed an artificial intelligence system that has the potential to evolve into that kind of application. They call the AI Pic2Recipe, because it can predict the ingredients and recipe used to make a dish from a single snapshot. If that sounds familia, it's probably because of "See Food," the fictional app that made an appearance in HBO's Silicon Valley, and a new-ish Pinterest feature that recognizes the most prominent ingredients in a picture of food.

Other teams and institutions are also working on similar projects, and a common issue is the lack of samples that leads to limited accuracy. The team had to scour various social networks and websites, such as All Recipes and Food, to collect 1 million recipes. They then annotated the collection with more information about their ingredients and used the database to train a neural network.

The result? Their system can identify ingredients like like flour, eggs and butter and can spit out the correct recipe for the dish in an image 65 percent of the time. CSAIL graduate student and lead author Nick Hynes explains:

"This could potentially help people figure out what's in their food when they don't have explicit nutritional information. For example, if you know what ingredients went into a dish but not the amount, you can take a photo, enter the ingredients, and run the model to find a similar recipe with known quantities, and then use that information to approximate your own meal."

Before you can use it as part of your fitness regimen, though, the team still has to work on making it more accurate. They're also developing its ability to infer how a dish is prepared and to distinguish one ingredient from another. You can put the AI to the test yourself if you want -- just upload a pic on the team's demo website or click one of the available images.

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
284 Shares
Share
Tweet
Share
Save

Popular on Engadget

Engadget's Guide to Privacy

Engadget's Guide to Privacy

View
iFixit's iPhone 11 Pro Max teardown investigates charging rumors

iFixit's iPhone 11 Pro Max teardown investigates charging rumors

View
TiVo wants to make a comeback with $50 Android TV dongle

TiVo wants to make a comeback with $50 Android TV dongle

View
Neo Geo retro stick console includes 'King of Fighters,' 'Samurai Shodown'

Neo Geo retro stick console includes 'King of Fighters,' 'Samurai Shodown'

View
Watch the 'Android' Nokia phone that never had a chance to exist

Watch the 'Android' Nokia phone that never had a chance to exist

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr