Latest in Science

Image credit:

Microsoft's hand-tracking magic may lead to gamepad-free VR

The algorithms also gives users "a virtual sense of touch."
396 Shares
Share
Tweet
Share
Save

Sponsored Links

The next Xbox One S console doesn't even have a Kinect port, showing that even Microsoft doesn't care about the depth sensor for gaming. However, it may yet make a comeback in VR. Microsoft researchers have made hand- and finger-tracking much more accurate using the Kinect, which may eventually make it a more attractive option than a controller for VR games. "We're getting to the point that the accuracy is such that the user can start to feel like the avatar hand is their real hand," says Microsoft UK researcher Jamie Shotton.

Finger tracking currently works on the Kinect (using Kinect Studio and other apps), but its accuracy is very poor. The problem, says Microsoft, is that your hand is relatively small and complex, so when you rotate it or make a fist, your fingers disappear from view and fool the software.

To solve the problem, Microsoft creates a smooth surface model of the hand from the data points tracked by the Kinect. It then uses an algorithm that was developed way back in the 1940s to calculate where the tracking points should go next in order to conserve the most energy. The system also uses a "reinitializer" to quickly reset a hand pose in case of tracking errors.

The smoother surface model does require more computations, but the software can resolve hand positions more quickly, within four iterations. In addition, it requires fewer tracking points than other solutions. As a result, it "can track hands smoothly, quickly and accurately –- in real time –- but can run on a regular consumer gadget," says the team.

With the app, users can smoothly interact with objects nearly in real time, something that could be highly beneficial for gaming, design and other VR applications. "The system lets you see what your hands are doing, fixing a common and befuddling disconnect that happens when people are interacting with virtual reality but can't see their own hands," according to the blog.

What's more, with proper interface design, users aren't thrown off by the lack of haptics, or simulated touch. "Thin" virtual controls make you touch your own fingers together for a fake sense of feel, for instance. And with a realistic-looking interface that works in real time and includes sound, users can be convinced they're actually touching something.

The finger-tracking algorithms are purely research for now, and problems like collisions and tracking errors still need to be resolved. However, it holds a lot of promise for gaming, user interface design and other applications. Using an off-the-shelf Kinect, Microsoft showed off virtual gamepads, painting tools that let you "grab" a color with your fingers and a game where the player is a "god" that literally gives a helping hand to his characters.

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
396 Shares
Share
Tweet
Share
Save

Popular on Engadget

Hackers make jailbreaking iPhones a thing again

Hackers make jailbreaking iPhones a thing again

View
 Apple’s game subscription service will reportedly cost $5 per month

Apple’s game subscription service will reportedly cost $5 per month

View
7-Eleven's cashier-free mobile checkouts are available in NYC

7-Eleven's cashier-free mobile checkouts are available in NYC

View
'Gears Pop' brings cute combat to your phone on August 22nd

'Gears Pop' brings cute combat to your phone on August 22nd

View
Stranger Things' Demogorgon will be a playable killer in 'Dead by Daylight'

Stranger Things' Demogorgon will be a playable killer in 'Dead by Daylight'

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr