Latest in Tomorrow

Image credit: gonin via Getty Images

MIT finds smaller neural networks that are easier to train

MIT’s "Lottery Ticket" mini-neural networks train only the nodes they need.
402 Shares
Share
Tweet
Share
Save

Sponsored Links

gonin via Getty Images

Despite all the advancements in artificial intelligence, most AI-based products still rely on "deep neural networks," which are often extremely large and prohibitively expensive to train. Researchers at MIT are hoping to change that. In a paper presented today, the researchers reveal that neural networks contain "subnetworks" that are up to 10 times smaller and could be cheaper and faster to teach.

To train most neural networks, engineers feed them massive datasets, but that can take days and expensive GPUs. The researchers from MIT's Computer Science and Artificial Intelligence Lab (CSAIL) found that within those trained networks are smaller, subnetworks that can make equally accurate predictions. CSAIL's so-called 'lottery-ticket hypothesis' is based on the idea that training most neural networks is something like buying all the tickets in a lottery to guarantee a win. By comparison, training the subnetworks would be like buying just the winning tickets.

The catch is that the researchers haven't figured out how to find those subnetworks without building a full neural network and then pruning out the unnecessary bits. If they can find a way to skip that step and go straight to the subnetworks, this process could save hours of work and make training neural networks accessible to individual programmers -- not just huge companies. But determining how to efficiently find subnetworks and understanding why some are better than others at learning will likely keep researchers busy for years.

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
402 Shares
Share
Tweet
Share
Save

Popular on Engadget

Engadget’s guide to Home Entertainment

Engadget’s guide to Home Entertainment

View
Crowdfunded case will give your Windows PC that Mac Pro look

Crowdfunded case will give your Windows PC that Mac Pro look

View
Original 'Fortnite' map vanishes in a cataclysmic event

Original 'Fortnite' map vanishes in a cataclysmic event

View
Andy Rubin quietly left the venture company he founded

Andy Rubin quietly left the venture company he founded

View
The best outdoor security camera

The best outdoor security camera

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr