Latest in Gear

Image credit:

New MIT tech lets you mess with objects in pre-recorded video

Yes, they’ve already successfully tested it in ‘Pokémon Go.’ 
David Lumb, @OutOnALumb
August 3, 2016
Share
Tweet
Share

Sponsored Links

Despite everything going on within the frame, videos are still a passive experience to observe. You can't reach in and mess with the objects you're watching — until now. An MIT researcher has pioneered new technology that lets you "touch" recorded things, which are simulated to respond like you'd fiddled with them in the real world.

So how do you predict which ways an already-recorded object will move when tweaked? The system, called "Interactive Dynamic Video" (IDV), needs five less than a minute of footage to track movement possibilities. It does so by analyzing how it shifts when intentionally jostled: In the video example below, a researcher slams the table on which a humanoid figure is resting, which lets the system see how it vibrates across different frequencies. Then it extrapolates how the item should behave when viewers reach in with their cursors and jostle the object in video.

Obviously this could have many applications in entertainment and education, but the researchers wanted a more topical example — so they inserted the system into Pokémon Go's augmented reality. With IDV, the environment automatically reacts to the pocket monsters' movements, making it seem like the wee beasts are actually bending leaves and grass as they bounce around.

The simulation isn't perfect, admits MIT PhD student Abe Davis in the first video, but more information will boost the movement accuracy of the IDV-produced models. With sufficient precision, it could be used to evaluate the structural integrity of buildings. But it also has uses in movie special effects, tweaking real-world objects that a computer-generated character interacts with on screen.

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
Tweet
Share

Popular on Engadget

The 2020 Engadget Holiday Gift Guide

The 2020 Engadget Holiday Gift Guide

View
The Arecibo Observatory's telescope has collapsed

The Arecibo Observatory's telescope has collapsed

View
The second-gen Eve V may take on the Surface Pro again in 2021

The second-gen Eve V may take on the Surface Pro again in 2021

View
Watch the trailer for Studio Ghibli's first fully CG movie

Watch the trailer for Studio Ghibli's first fully CG movie

View
The Snapdragon 888 is Qualcomm's latest premium CPU for smartphones

The Snapdragon 888 is Qualcomm's latest premium CPU for smartphones

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr