Latest in Gear

Image credit: NickNick_ko via Getty Images

MIT's music AI can identify instruments and isolate their sounds

PixelPlayer can mute the piano if you just want to listen to the violin.
629 Shares
Share
Tweet
Share
Save

Sponsored Links

NickNick_ko via Getty Images

If you've ever played a YouTube video for what it seems like the thousandth time to listen to your instrument's part of a composition, you'll love MIT's new AI. PixelPlayer, which hails from the institution's Computer Science and Artificial Intelligence Laboratory (CSAIL), can recognize instruments in a video, identify specific ones at pixel level and isolate the sounds they produce. If there are several instruments playing in a video, for instance, PixelPlayer will allow you to pick the one you want to listen to -- it will play the sounds coming out of that instrument the loudest and will lower the volume or everything else.

CSAIL trained PixelPlayer by using a self-supervised deep learning technique and feeding it over 60 hours of videos to learn from. It's still far from perfect, though: it can only identify the sounds of 20 instruments at the moment and is still having trouble telling similar ones apart. With further development, it could become an effective audio editing tool, giving engineers a way to improve or restore the quality of old concert footage. It could also be used to train robots on how to identify various environmental sounds, such as those made by animals, vehicles and appliances.

Source: MIT
In this article: art, csail, gear, mit, music, pixelplayer
All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
629 Shares
Share
Tweet
Share
Save
Comments

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr