MotionCapture
Latest
How I turned my Xbox's Kinect into a wondrous motion-capture device
When Microsoft started selling a basic Xbox One package without a Kinect V2 for $100 less, the result was unequivocal: Sales took off. Most gamers can take or leave the ubiquitous depth camera, because it just isn't as useful for gaming as, say, the Wii controller. It is indispensable for certain titles, like Just Dance 2014, Xbox Fitness and Fighter Within. Others, such as Madden NFL 25 and Battlefield 4, can make use of the Kinect 2, but absolutely don't need it. In other words, it's a big bag of meh for gamers and casual users. But recently, my ears perked up when Microsoft released a $50 cable that lets you use the Xbox One's Kinect on a PC.
Motion capture suits can detect lies better by tracking your fidgeting
Stone-cold bad guys don't worry about lie detectors, because they can only suss out fibs 60 percent of the time -- not much better than someone without one. But there's a new hope for cops. Scientists at the University of Cambridge have used full-body motion capture suits to cajole the truth at a much better clip. They put 90 volunteers in $12,000 Xsens mocap suits, and had them lie to other volunteers. By tracking joint displacements, their algorithms could pick three out of four liars -- a much more useful result for law enforcement. One researcher said "put simply, guilty people fidget more... independent of cultural background, cognitive load and anxiety." The team thinks it can fine tune it for even better accuracy, so police may one day be squeezing perps into skin-tight suits rather than finger sensors.
Crowdfunded project uses 3D motion capture to catalog ancient Kung Fu styles
The influence of Hong Kong action cinema stars like Bruce Lee lives on in today's cinema, but the ancient styles they based their techniques on are slowly dying out. There's now a crowdfuding project aimed at preserving the heritage of different Kung Fu fighting styles, called the Hong Kong Martial Arts Living Archive. It's a collaboration between a martial arts society called the International Guoshu Association (IGU) and the City University of Hong Kong. The goal is to use photos, high-speed video, panoptic video and motion capture to record and quantize the different techniques.
Watch a dome full of cameras capture 3D motion in extreme detail
Conventional 3D motion capture is awkward; even if you don't mind people covered with dots or ping pong balls, you often get just a handful of data points that miss out on subtle movements. Researchers at Carnegie Mellon University might just have a more elegant solution in store, though. They've built a geodesic dome whose 480 cameras work in concert to track a whopping 100,000 points without the need for markers. The system uses its abundance of video footage to estimate trajectories based on changes in light, motion and shape, rather than looking for arbitrary cues or interpolating image frames. As you can see in the clips below, the resulting data is both vividly detailed and natural-looking -- you can see individual confetti flakes falling to the ground, and it's easy to follow every nuance of a batter's swing.
What you need to know about 3D motion capture
Close your eyes and go back... back in time. Picture Jar Jar Binks or Polar Express, movies that put the "Uncanny Valley" on the map. I know these aren't pleasant memories, but new technology like motion capture (mocap) can be... awkward in its youth. Now, let's forget all that and move forward to a time when the tech started hitting its stride -- from Lord of the Rings' Gollum to Avatar to The Avengers' Hulk. And let's not forget games -- The Last of Us has some of the best mocap done in any medium and Electronic Arts has used the technique since Madden NFL '94. But what is mocap, exactly, and how is it done? Will it ever replace live actors or put 3D animators out of business? To answer all that, let's head back in time 100 years.
Watch filmmakers render realistic CG on the fly using $14k of graphics cards
A new short film teaser has taken digital character rendering to a new level, making real time motion capture a lot easier for animators. While working on "Construct" (see the stunning video after the break) filmmakers captured the movements of real actors in a studio, similar to how James Cameron did for Avatar. Instead of seeing the performer, however, the director saw a ray-traced version of the animated character on his screen. Though heavily pixelated, freezing the scene instantly gave animators a clear idea of the final result, something that can normally take hours in post-production. The system used custom software from ray-tracing outfit V-Ray powered by three top-of-the-line NVIDIA K6000 GPUs -- not exactly a home setup. Still, it's not hard to see how such tech could eventually power ultra-realistic gaming, though at $4,500 a pop or so for the graphics cards, we're not there yet.
Microsoft shows off next-gen Kinect motion and voice capture in Xbox One's Project Spark
Microsoft just flaunted some of Project Spark's gaming powers at Gamescom 2013 in Cologne: motion and voice capture. If you'll recall, the platform enables gamers to create their own digital spaces for games using the Xbox One's integrated Kinect sensor and the upcoming SmartGlass. Microsoft's Team Dakota group showed how to use facial capture, body motion capture and voice and sound to create animations, dialogue, cut scenes and more. You'll be able to try it yourself on Windows 8 at the end of October 2013, or by January 2014 for the Xbox One. Check the video after the jump to see the fruits of their labor, but maybe turn the volume down a hair. Just sayin'.
OptiTrack debuts $3,700 PRIME 17W mocap cam for small spaces
Independent creators keen on motion capture have had affordable solutions like cheaper sensors and Kinect-based implementations for awhile now, but a large space for moving around has usually been required. OptiTrack has come up with an answer to that problem, however, in the form of the PRIME 17W mocap camera that it introduced at the Game Developers Conference in San Francisco. The 1.7-megapixel lens has a 70-degree by 51-degree field of view that promises to capture motion in a relatively small space, which also means you need fewer cameras to get a full 360-degree shot. Other features include a global shutter, high-speed 360 FPS capture and low distortion, enabling UAV and sports tracking. At $3,700, it's still not exactly cheap, but it's certainly affordable enough for indie engineers and animators with space constraints to get started in the mocap biz.%Gallery-184366%
Xsens teases wearable 3D body sensors that won't cost, will track an arm and a leg (video)
When we think of full-body motion capture, we most often associate it with movie-grade equipment that demands a dedicated room, odd-looking suits and a corporate bank account to finance it all. Xsens hints that we may not have to rent a professional studio (or stand in front of a Kinect) to get complete body tracking for personal use. It's planning to show a wearable, 3D-capable tracking system at CES that uses "consumer grade" MEMS sensors to monitor joint positions and movement -- in other words, the kind of technology that might go into a phone's accelerometer, just strapped to our arms and legs. Further details are scarce, although Xsens is pressing for uses in everything from fitness to gaming. We'd like to see partners line up so that there's a product we can buy in a store. Until then, we'll have to make do with the company's skateboard-dominated teaser clip, which you can find after the break.
Microsoft patent applications take Kinect into mobile cameras, movie-making
Microsoft has never been shy about its ambitions for Kinect's depth sensing abilities. A pair of patent applications, however, show that its hopes and dreams are taking a more Hollywood turn. One patent has the depth camera going portable: a "mobile environment sensor" determines its trajectory through a room and generates a depth map as it goes, whether it's using a Kinect-style infrared sensor or stereoscopic cameras. If the visual mapping isn't enough, the would-be camera relies on a motion sensor like an accelerometer to better judge its position as it's jostled around. Microsoft doesn't want to suggest what kind of device (if any) might use the patent for its camera, but it's not ruling out anything from smartphones through to traditional PCs. The second patent filing uses the Kinect already in the house for that directorial debut you've always been putting off. Hand gestures control the movie editing, but the depth camera both generates a model of the environment and creates 3D props out of real objects. Motion capture, naturally, lets the humans in the scene pursue their own short-lived acting careers. We haven't seen any immediate signs that Microsoft is planning to use this or the mobile sensor patent filing in the real world, although both are closer to reality than some of the flights of fancy that pass by the USPTO -- the movie editor has all the hallmarks of a potential Dashboard update or Kinect Fun Labs project.
Heavy Rain creators produce 'Kara' PS3 tech-demo (video)
Heavy Rain creator David Cage was showing off Quantic Dream's new game engine at GDC, which includes an innovative new performance-capture technology the company's developed. He's directed a seven-minute original short called Kara, which is the story of a female android as she becomes self-aware. Unlike traditional game production methods, this technology is able to record face and body movements at the same time as recording the actors voice -- ensuring natural and consistent performances from the characters. Actress Valorie Curry wore 90 sensors on her face, unlike in, say, Avatar, where the performers wore head-mounted cameras. Cage promises that the short is nothing more than a demo (it was rendered in real-time on a PlayStation 3) and none of these elements will appear in his next game. You can catch the impressive-looking footage after the break with one disclaimer: there's nudity throughout and a reference to adult themes, okay?
Hey yo, T(ether) technology tracks rap gestures in 3D
T(ether) is a bit like that amazing Reality Fighters game we saw on the PS Vita, except it's more tranquil and it works slightly differently. The tablet-based technology uses motion capture cameras to track users' heads and the orientation of the device itself, building a 1:1 spatial connection between real and virtual coordinates and allowing "intuitive interaction with volumetric data." As you'll see in the clip after the break, this equates to some pretty neat tricks. And, if you throw in multiple users wearing motion-tracked gloves, things get even more Eastside.
Researchers demo 3D face scanning breakthroughs at SIGGRAPH, Kinect crowd squarely targeted
Lookin' to get your Grown Nerd on? Look no further. We just sat through 1.5 hours of high-brow technobabble here at SIGGRAPH 2011, where a gaggle of gurus with IQs far, far higher than ours explained in detail what the future of 3D face scanning would hold. Scientists from ETH Zürich, Texas A&M, Technion-Israel Institute of Technology, Carnegie Mellon University as well as a variety of folks from Microsoft Research and Disney Research labs were on hand, with each subset revealing a slightly different technique to solving an all-too-similar problem: painfully accurate 3D face tracking. Haoda Huang et al. revealed a highly technical new method that involved the combination of marker-based motion capture with 3D scanning in an effort to overcome drift, while Thabo Beeler et al. took a drastically different approach. Those folks relied on a markerless system that used a well-lit, multi-camera system to overcome occlusion, with anchor frames acting as staples in the success of its capture abilities. J. Rafael Tena et al. developed "a method that not only translates the motions of actors into a three-dimensional face model, but also subdivides it into facial regions that enable animators to intuitively create the poses they need." Naturally, this one's most useful for animators and designers, but the first system detailed is obviously gunning to work on lower-cost devices -- Microsoft's Kinect was specifically mentioned, and it doesn't take a seasoned imagination to see how in-home facial scanning could lead to far more interactive games and augmented reality sessions. The full shebang can be grokked by diving into the links below, but we'd advise you to set aside a few hours (and rest up beforehand). %Gallery-130390%
Organic Motion's OpenStage motion capture system grabs 200FPS, no backdrop required (video)
At just under $40,000 for an eight camera setup, we're hardly in hobbyist territory here, but Organic Motion's new OpenStage 2.0 motion capture system could certainly make do in the average basement. Unlike a few competing solutions shown here at SIGGRAPH, this one actually has no backdrop mandate, and better still, doesn't require you to latch a single sensor onto your subject. The magic lies within the cameras hung above -- kits are sold that contain between eight and 24 cameras, and even the latter can be handled with a single workstation. Multi-person tracking ain't no thang, and while you aren't capturing HD footage here, the high-speed VGA capability enables up to 200 frames per second to be logged. Not surprisingly, the company's aiming this squarely at the animation and medical realms, and should start shipping bundles as early as next month. Looking to take down Pixar? You'll need a lot more than 40 large, but perhaps the video after the break will give you a bit of inspiration. %Gallery-130288%
SoftEther's sensor-laden QUMA robot demonstrates poses, intimidates your acting coach (video)
A solution in search of a problem, or a solution to a problem that you were too proud to cop to? SoftEther has just revealed what might be the final blow to Barbie's distinguished career: the sensor-splashed QUMA. So far as we can tell, the human-shaped puppet contains a myriad sensors to pick up precise bends and flexes, and then pipes that information to a screen. Aside from showing your team of ballerinas exactly how their routine should look, we're guessing that the real future here is in far more sophisticated tasks -- things like artificial intelligence, major motion pictures and scientific research. As the saying goes, a video's worth a zillion words, so have a peek for yourself just after the break.
Aiken Labs brings 9-axis modular motion sensing to Android, we go hands-on (video)
We already had a chance to try out Immersive Motion from Aiken Labs at CES, but now the nine-axis modular sensing system is making its way to Android and other mobile platforms, including iOS and Windows Phone. The more compact battery-powered server brings motion-controlled gaming to mobile environments, capturing position data from matchbox-size modular sensors that you can tape to a wooden sword or Viking helmet for live-action outdoor role-playing, or on you paws and dome during a virtual jam session with friends, for example. The mobile kit includes a pair of wireless sensors with a 50-foot range that you can attach to literally any accessory or appendage, and is expected to sell for about $300 when it ships later this year. You'll also be able to connect up to two smaller wired sensors to each wireless sensor, for about $50 a pop. The kit's price tag makes it cost-prohibitive for all but the most hardcore gamers and devs (there's an SDK available as well), but Aiken hopes to make its flagship product more affordable if its able to sell the kits in high volume. The tool has applications in other industries as well, including research and Hollywood, where it could be used as a (relatively) low-cost outdoor motion-capture suit. The early version we saw at E3 today is definitely not ready to head to production, but we're still months away from an actual release, giving Aiken some time to improve accuracy, and perhaps find a way to reduce that price. Jump past the break to see how it works.%Gallery-125868% Tim Stevens contributed to this report.
Kinect keeps surgeons on task, Nintendo 3DS might assist optometrists with diagnoses
The latest generation of gaming gadgets do some nifty tricks, and one of the niftiest they might perform is assisting the realm of medicine. Microsoft's Kinect sounded like a candidate for surgery, and this month real-life surgeons have actually put it to use -- Sunnybrook Hospital in Toronto, Canada rigged the Xbox 360 depth camera to its medical imaging computer. Now, doctors don't have to scrub out to manipulate an MRI scan, or even appoint a peon to the task -- rather, they simply raise their bloodied glove, and dive into the digital imagery with a wave of a dextrous hand. Meanwhile, the American Optometric Association has expanded upon its initial praise of Nintendo's 3DS, saying the autostereoscopic 3D handheld "could be a godsend for identifying kids under 6 who need vision therapy." Though Nintendo's warning labels had originally incited a bit of fear among parents, the organization says that kids who can't experience the 3DS to its full potential may have amblyopia (or other vision disorders) that can be more easily treated the earlier it's caught, though one doctor interviewed by the Associated Press contends that kids with amblyopia may not know what they're missing to begin with -- so don't necessarily expect a panacea, folks.
Kinect hacked for home automation, does your mood lighting for you (video)
Microsoft's Kinect has become quite the hacking hotbed -- the fields of medicine, music, and even shadow puppeteering have all benefitted from the peripheral's incredible versatility. And now, to the delight of home automation nerds everywhere, an enterprising young hacker has rigged a Kinect to automate the lighting in his home. By positioning the camera bar in a corner to track his movements, connecting it to the automation controller, and coding on / off commands, he's able to control the lights throughout his geektastic domicile. The automation logic then turns on the lights when entering the room, localizes them according to location, and turns them off upon leaving. One less thing to worry about -- here's hoping a method for spotting our perpetually misplaced keys is in version 2.0. Vid's after the break.
Lockheed Martin's CHIL blends motion capture with VR, creates zombie engineers (video)
Computer-aided design is a great way to build products, but does it let you bust a funky move while wearing some crazy glasses and gloves? Heck no. You need Lockheed Martin's CHIL for that. It's the Collaborative Human Immersive Laboratory, virtual reality goggles and gloves combined with motion capture enabling teams of engineers to work together in a virtual space. You can see it in action below, used first for installing polygonal munitions into a rendered version of one of the company's F-35 Joint Strike Fighters, then for doing a little VR tai chi. A Lockheed rep promises that this enables the team to ensure the plane can be more easily and affordably maintained, but we just see this as high-tech training tool for the company's world-renowned synchronized dance teams.
L.A. Noire's amazing MotionScan facial capture system demonstrated (video)
In gaming, 3D graphics get more powerful, environments get more expansive, enemies get more intelligent, but still facial animations haven't progressed much since Pac-Man chomped his first power pellet in 1980. Finally, a major breakthrough courtesy of Australian company Depth Analysis. It has developed technology called MotionScan, which enables a high-res 3D recreation of a person's face -- not just capturing bits and pieces of facial animation but their entire head, right down to the hairstyle. It's getting its first use in next year's L.A. Noire, a 1940s PS3 and Xbox 360 murder mystery game from Rockstar, and while we don't know enough about the title to be able to say whether it's worth plunking down a pre-order now, after watching the video embedded below it's clear that the bar has been raised.
