Latest in Gear

Image credit: Google

Google's mixed reality tech shows faces behind VR headsets

It uses dynamic 3D models of the users' faces.
545 Shares
Share
Tweet
Share
Save
Google

Sponsored Links

YouTube has already perfected the art of mixed reality videos that show the VR world and the real world at the same time. A great example is the video of Conan O'Brien's virtual trip to outer space taken when he visited YouTube's VR Lab. The video platform's technique still can't capture the whole picture, however, since VR headsets get in the way. Google Research and Daydream Labs had to team up to solve the problem, and they did just that with the help of machine learning, 3D computer vision and advanced rendering techniques.

In order to "remove" the VR headset in a video, the user's face has to be scanned with a camera and reconstructed as a dynamic 3D model that blinks and looks in different directions. Next, the VR headset -- Google used an HTC Vive for its tests -- has to be outfitted with eye-tracking tech in order to detect where the user is looking. Google's and Daydream's technique superimposes the 3D model on the face of the real person. It doesn't completely remove the VR headset, though: it just replaces it with a transparent version to prevent the uncanny valley effect.

As you can guess, the technique will benefit YouTube creators the most at this point in time. In fact, Google is exploring ways to make it accessible to YouTubers. It could become a lot more useful when VR becomes more widespread, though. For instance, it could be adapted to show people's faces in VR video conferences or the faces of friends playing VR games together.

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
545 Shares
Share
Tweet
Share
Save

Popular on Engadget

Engadget's Guide to Privacy

Engadget's Guide to Privacy

View
Batman comes to 'Fortnite' along with Catwoman and Gotham City

Batman comes to 'Fortnite' along with Catwoman and Gotham City

View
Hitting the Books: Searching for ancient cities from space

Hitting the Books: Searching for ancient cities from space

View
The Morning After: Taking a look inside the iPhone 11 Pro Max

The Morning After: Taking a look inside the iPhone 11 Pro Max

View
Ask Engadget: How can I improve my rural internet?

Ask Engadget: How can I improve my rural internet?

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr