Latest in Science

Image credit:

Microsoft's hand-tracking magic may lead to gamepad-free VR

The algorithms also gives users "a virtual sense of touch."
Steve Dent, @stevetdent
June 28, 2016
396 Shares
Share
Tweet
Share

Sponsored Links

The next Xbox One S console doesn't even have a Kinect port, showing that even Microsoft doesn't care about the depth sensor for gaming. However, it may yet make a comeback in VR. Microsoft researchers have made hand- and finger-tracking much more accurate using the Kinect, which may eventually make it a more attractive option than a controller for VR games. "We're getting to the point that the accuracy is such that the user can start to feel like the avatar hand is their real hand," says Microsoft UK researcher Jamie Shotton.

Finger tracking currently works on the Kinect (using Kinect Studio and other apps), but its accuracy is very poor. The problem, says Microsoft, is that your hand is relatively small and complex, so when you rotate it or make a fist, your fingers disappear from view and fool the software.

To solve the problem, Microsoft creates a smooth surface model of the hand from the data points tracked by the Kinect. It then uses an algorithm that was developed way back in the 1940s to calculate where the tracking points should go next in order to conserve the most energy. The system also uses a "reinitializer" to quickly reset a hand pose in case of tracking errors.

The smoother surface model does require more computations, but the software can resolve hand positions more quickly, within four iterations. In addition, it requires fewer tracking points than other solutions. As a result, it "can track hands smoothly, quickly and accurately –- in real time –- but can run on a regular consumer gadget," says the team.

With the app, users can smoothly interact with objects nearly in real time, something that could be highly beneficial for gaming, design and other VR applications. "The system lets you see what your hands are doing, fixing a common and befuddling disconnect that happens when people are interacting with virtual reality but can't see their own hands," according to the blog.

What's more, with proper interface design, users aren't thrown off by the lack of haptics, or simulated touch. "Thin" virtual controls make you touch your own fingers together for a fake sense of feel, for instance. And with a realistic-looking interface that works in real time and includes sound, users can be convinced they're actually touching something.

The finger-tracking algorithms are purely research for now, and problems like collisions and tracking errors still need to be resolved. However, it holds a lot of promise for gaming, user interface design and other applications. Using an off-the-shelf Kinect, Microsoft showed off virtual gamepads, painting tools that let you "grab" a color with your fingers and a game where the player is a "god" that literally gives a helping hand to his characters.

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
396 Shares
Share
Tweet
Share

Popular on Engadget

Facebook envisions using holographics for super-slim VR glasses

Facebook envisions using holographics for super-slim VR glasses

View
100 million people watch YouTube on TVs each month

100 million people watch YouTube on TVs each month

View
PlayStation is the latest to join the Facebook ad boycott

PlayStation is the latest to join the Facebook ad boycott

View
Amazon-owned Ring is preparing its first smart light bulb

Amazon-owned Ring is preparing its first smart light bulb

View
House approves $100 billion in broadband funding for rural areas

House approves $100 billion in broadband funding for rural areas

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr