Latest in Gadgetry

Image credit:

Facebook can predict a musician’s movement using only an audio clip

The AR avatars could use an upgrade, though.
Share
Tweet
Share

Sponsored Links

Facebook

We're used to seeing the likes of Wii avatars crudely banging at pianos or sawing away at violins, but Facebook is now working on an application that uses music alone to govern the movements of an augmented reality avatar. In other words, input some music, and the avatar will move just like a human pianist or violinist would.

The research, completed in partnership with the University of Washington and Stanford, has just been presented by Facebook at the CVPR (Computer Vision and Pattern Recognition) conference. The team wanted to see if natural body dynamics could be predicted from music, and while the base animation is pretty basic and shaky, they're certainly on their way to achieving that. Next on the agenda will be detailed arm and finger motions.

This is the first time anyone has tried to predict body movement from music alone. It's an interesting piece of AR research, and it could also have wide-ranging applications for music not only in gaming, but in the real world, too. If you wanted to learn a piece of piano music, for example, you could simply input the audio and then mimic the key movements of the avatar pianist -- much more fun than those seventh grade lessons you were forced into.

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
Tweet
Share

Popular on Engadget

Engadget's 2020 Back-to-School Guide

Engadget's 2020 Back-to-School Guide

View
Space Force official logo and motto unveiled

Space Force official logo and motto unveiled

View
Our readers get real about their issues with the AirPods Pro

Our readers get real about their issues with the AirPods Pro

View
Microsoft's Surface Duo may launch with AT&T

Microsoft's Surface Duo may launch with AT&T

View
Nintendo 'gigaleak' reveals the classic games that never were

Nintendo 'gigaleak' reveals the classic games that never were

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr