Latest in Gear

Image credit: NickNick_ko via Getty Images

MIT's music AI can identify instruments and isolate their sounds

PixelPlayer can mute the piano if you just want to listen to the violin.
630 Shares
Share
Tweet
Share
Save

Sponsored Links

NickNick_ko via Getty Images

If you've ever played a YouTube video for what it seems like the thousandth time to listen to your instrument's part of a composition, you'll love MIT's new AI. PixelPlayer, which hails from the institution's Computer Science and Artificial Intelligence Laboratory (CSAIL), can recognize instruments in a video, identify specific ones at pixel level and isolate the sounds they produce. If there are several instruments playing in a video, for instance, PixelPlayer will allow you to pick the one you want to listen to -- it will play the sounds coming out of that instrument the loudest and will lower the volume or everything else.

CSAIL trained PixelPlayer by using a self-supervised deep learning technique and feeding it over 60 hours of videos to learn from. It's still far from perfect, though: it can only identify the sounds of 20 instruments at the moment and is still having trouble telling similar ones apart. With further development, it could become an effective audio editing tool, giving engineers a way to improve or restore the quality of old concert footage. It could also be used to train robots on how to identify various environmental sounds, such as those made by animals, vehicles and appliances.

Source: MIT
In this article: art, csail, gear, mit, music, pixelplayer
All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
630 Shares
Share
Tweet
Share
Save

Popular on Engadget

Engadget's Guide to Privacy

Engadget's Guide to Privacy

View
iFixit's iPhone 11 Pro Max teardown investigates charging rumors

iFixit's iPhone 11 Pro Max teardown investigates charging rumors

View
TiVo wants to make a comeback with $50 Android TV dongle

TiVo wants to make a comeback with $50 Android TV dongle

View
Neo Geo retro stick console includes 'King of Fighters,' 'Samurai Shodown'

Neo Geo retro stick console includes 'King of Fighters,' 'Samurai Shodown'

View
Watch the 'Android' Nokia phone that never had a chance to exist

Watch the 'Android' Nokia phone that never had a chance to exist

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr