facetracking
Latest
Cornell researchers created an earphone that can track facial expressions
The team says C-Face works even when the subject is wearing a mask.
Google's AR tools make it easier for apps to apply face filters
Augmented reality experiences are still in their relative infancy, but because Android runs on so many devices, it can't always assume they'll have dedicated hardware to create extra effects. While Apple is already pushing ahead with AR and packed an entire Kinect-like camera into the front of its iPhone X family to support it, Google is enhancing its software to work even without capabilities like the LG G8's depth-sensing hardware.
Intel AI helped create a music video
AI is increasingly finding its way into music videos, and not necessarily in obvious ways. Intel has revealed that the promo clip for Chinese pop star Chris Lee's "Rainy Day, But We Are Together" is the first music video to lean on its AI technology. Director Timothy Saccenti and Intel's producers created dramatic special effects on the songstress' face (such as trickles of water and twinkling stars) by training a machine learning system to instantly reconstruct a face in 3D and track its movements in real time, including facial expressions. Instead of asking Lee (aka Li Yuchun) to wear tracking markers or the camera crew to shoot a specific way, the creative team could focus on capturing scenes that lined up with their artistic goals.
The next iPhone creates animated emoji from your facial expressions
You may already know that the next iPhone will use face detection for all kinds of clever tricks, but here's one you probably weren't expecting: customized emoji. The 9to5Mac crew has discovered that leaked "gold master" iOS 11 firmware includes references to 'Animoji,' or 3D emoji that you create using your facial expressions and voice. Pick one of the familiar non-human faces in the emoji library and it'll map your eye, mouth and cheek expressions to that character -- you can make a robot smile or have a dog raise its eyebrows. Even the poo emoji can be animated. This comes across as a gimmick (we can see many people dropping this once the novelty wears off), but it shows what's possible now that Apple has face tracking at its disposal. And there's more to the leak than just emoji.
The Passport foldable drone makes for a fun travel companion
My first drone flight experience was with the DJI Phantom 2 Vision, and as much as I appreciated its advanced capabilities at the time, I longed for something more compact -- a device so small that I wouldn't need to carry a separate bag or case for it, preferably without sacrificing performance. Eventually, a Chinese startup called Zero Zero Robotics released the $599 Hover Camera Passport, which comes in the unique form of a foldable cage while packing cool features like body tracking, face tracking and orbiting. I got to spend some time with the Passport over the past few weeks, and eventually it got to the point where I rarely leave home without it, lest I find time to take it for a quick spin.
Eye-tracking app lets you make music hands-free
Eye- and face-tracking systems already help you communicate when your hands aren't an option, but what if you want to have some fun? You're covered there, too. Andreas Refsgaard has built Eye Conductor, an app designed to help the paralyzed (and virtually anyone else) make music. All you need is a computer, an off-the-shelf eye tracker and a willingness to look slightly goofy as you twitch your eyes and open your mouth. Unlike some previous attempts, it's all about having an accessible, flexible interface -- you know exactly what sounds you'll make, whether it's an experimental piece or a slick dance track.
Maybe it's not Maybelline: Omote paints your face with digital makeup
Enhancing the world around us with projectors and 3D scanners is nothing new. But what about stylizing yourself? With Omote, you can use face-tracking tech to paint on digital makeup -- or anything imaginable, really. In the demo below, it renders a variety of makeup styles and visual effects (like changing eye color) on a model's face. What's most impressive, though, is that the CGI-like projection reflects her movements in real-time. This doesn't necessarily mean your face-painting days are coming to an end though, ladies. Omote's visual magic is equipment-heavy and limited; the woman in the video below never leaves her seat and doesn't crack so much as a smile. But if you someday find yourself using the technology to virtually try a new shade of foundation, we wouldn't be surprised.
Touchless music game lets you create jittery drum and bass with your face
If there was one ubiquitous item at NYU's ITP Winter Show, it was the Kinect. Countless projects were built around the Microsoft-made sensor. Max Ma's Touchless, which he built with a ton of help from Tony Lim, originally featured one, but the version that made it to the floor went with an OEM equivalent instead. But the effect is the same: a set of cameras and sensors track various parts of your face, turning your muscle twitches and eyebrow raises into raw data. While Max says this data can be used for a host of different applications, such as unlocking your door with a series of blinks and winks, he focused on bringing joy to people's lives through music creation. The sensor tracks between 16 and 64 points (under ideal conditions) on your face, and uses your movements to trigger and manipulate samples. Truth is, it's hard not to smile while making ridiculous faces, though, I was a little disappointed to find out that the tracker did not play well with my winter beard. The main method of interacting is by tilting your head, opening your mouth and raising your eyebrows, but Max added some depth by turning a Leap Motion sensor into a controller for a software synthesizer. So samples and beats are all above the neck, but you can wave your hands through the air to play a lovely lead melody. Really, the whole thing is pretty self-explanatory and quite fun, as you can see in the video after the break.
Alt-week 6.29.13: DARPA's robot finalists, the IRIS solar mission and empathetic computers
Alt-week takes a look at the best science and alternative tech stories from the last seven days. Sure, DARPA is slightly sinister, but it's so into robots that we're willing to let that slide. In fact, last year it launched the DARPA Robotics Challenge, and it just announced the top six nine seven teams to advance. But if just the idea of figuring out robotics frustrates you, NC State's face tracking program literally gets that, and NASA just launched the IRIS solar probe from the belly of a transport jet. It's Alt-week, baby.
Galaxy S 4, future Samsung devices to use DigitalOptics tech for face tracking (updated)
When Samsung unveiled the Galaxy S 4 in March, there was a near-inescapable emphasis on face detection features. What we didn't know is just whose technology was making them possible. As it happens, it's not entirely Samsung's -- DigitalOptics has stepped forward to claim some of the responsibility. The California firm recently struck a multi-year licensing deal with Samsung to supply its Face Detection and Face Tracking software, which can detect pupils for interface features (think Smart Stay or Smart Pause) and keep tabs on photo subjects. DigitalOptics hasn't provided the exact details of its involvement in the GS4, let alone a roadmap, but it's safe to presume that Samsung isn't dropping its emphasis on camera-driven software anytime soon. Update: DigitalOptics says the release wasn't clear on just what was involved in the deal: while the face detection and tracking are present, Samsung didn't pick up the pupil component. As such, you're mostly seeing DigitalOptics' influence in regular camera features and other software that doesn't involve eye tracking.
Microsoft job posting hints at Connected Car strategy: Azure, Kinect and WP8
Redmond seems to have more grandiose ideas for Connected Car than it's let on before, judging from a recent help wanted ad on its site. Reading more like PR for its car-based plans, the job notice waxes poetically about using "the full power of the Microsoft ecosystem" in an upcoming auto platform with tech such as Kinect, Azure, Windows 8 and Windows Phone. Those products would use face-tracking, speech and gestures to learn your driving habits and safely guide or entertain you on the road, according to the software engineer listing. It also hints that everything would be tied together using Azure's cloud platform, so that your favorite music or shortcuts would follow you around, even if you're not piloting your own rig. All that makes its original Connected Car plans from 2009 seem a bit laughable -- check the original video for yourself after the break.
SOEmote tracks our face, transplants our expressions onto an Everquest II persona (faces-on)
Sony's PlayStation division may have been the main focus this week at E3, but Sony Online Entertainment has something unique of its own to show off for MMORG players. Specifically, folks who are into the PC title Everquest II will soon be able to access a facial recognition feature called SOEmote (S-O Emote). It's no secret that these types of games require massive amounts of communication between players, and SOEmote is an attempt to make in-game conversations more personal than ever. Utilizing any webcam, the software is able map and track your noggin at 64 points, allowing your onscreen character to replicate any facial movements you make. We got a chance to mess around with a beta version of the software, and the results are impressive to say the least -- pretty much all of our facial expressions were accurately recreated. Join us past the break for more details and a hands-on faces-on video demo of it in action. %Gallery-157578%
Perifoveal Display tracks head positioning, highlights changing data on secondary LCDs (hands-on)
If there's a large display as part of your workstation, you know how difficult it can be to keep track of all of your windows simultaneously, without missing a single update. Now imagine surrounding yourself with three, or four, or five jumbo LCDs, each littered with dozens of windows tracking realtime data -- be it RSS feeds, an inbox or chat. Financial analysts, security guards and transit dispatchers are but a few of the professionals tasked with monitoring such arrays, constantly scanning each monitor to keep abreast of updates. One project from the MIT Media Lab offers a solution, pairing Microsoft Kinect cameras with detection software, then highlighting changes with a new graphical user interface. Perifoveal Display presents data at normal brightness on the monitor that you're facing directly. Then, as you move your head to a different LCD, that panel becomes brighter, while changes on any of the displays that you're not facing directly (but still remain within your peripheral vision) -- a rising stock price, or motion on a security camera -- are highlighted with a white square, which slowly fades once you turn to face the new information. During our hands-on demo, everything worked as described, albeit without the instant response times you may expect from such a platform. As with most Media Lab projects, there's no release date in sight, but you can gawk at the prototype in our video just after the break.
Toshiba's 4K, glasses-free 3DTV announced in Japan with more specs this time
We first saw the retail edition of Toshiba's 3840x2160 resolution autostereoscopic (no glasses) 3DTV when it was announced in Europe at IFA last month, and now it has debuted in Japan during CEATEC. This time the company dropped a few more details, revealing that in lenticular 3D mode it's limited to an effective resolution of 1280x720, and showed off the face tracking that automatically optimizes the experience for up to nine simultaneous viewers depending on where they're sitting. Also mentioned were an optional THD-MBA1 input adapter due in 2012 and that 4K-res streaming IPTV is currently being tested. The Regza 55X3 will be priced comparably to its $10K~ Euro-spec counterpart when it arrives in December, but there's still no word on when it will ship in the US. Check out a few pictures of the presentation in the gallery below. %Gallery-135528%
Microsoft's OneVision Video Recognizer can detect, identify, and track your face on video... so smile!
Here's your classic case of "just because you can, doesn't mean you should." Microsoft's Innovation Labs have just demonstrated a OneVision Video Recognizer algorithm that's powerful enough to perform face detection duties on a running video feed. It can recognize and track humanoid visages even while they're moving, accept tags that allow auto-identification of people as they enter the frame, and can ultimately lead to some highly sophisticated video editing and indexing via its automated information gathering. Of course, it's that very ease with which it can keep a watchful eye on everyone that has us feeling uneasy right now, but what are you gonna do? Watch the video after the break, that's what.
Evigroup Paddle tablet goes Pro, gets cursor-controlling, head-tracking webcam
Some day, in the distant future, we'll be activating windows, clicking buttons, and playing Farmville with our minds. Our minds. There have been attempts to get us there, none fully comprehensive, though the Evigroup Paddle Pro tablet is taking an interesting alternative approach: using head tracking to control the cursor. Apparently its front-facing webcam detects your front-facing mug and as you look about the screen it moves the cursor appropriately. Staring rudely at any button or control for a half-second equates to a click and, while we don't yet know how you'll double-click, we'd like to think a spasmodic twitch will be required. Evigroup is also launching a curvy keyboard to go with the Paddle Pro and is promising the ability to play video and audio wirelessly courtesy of a "small station" that connects to your TV. The internals, meanwhile, are perfectly predictable: a netbook spec Atom N450 struggling with Windows 7 Home Premium. No word on price or availability.
Nintendo DSi game lets your face do the flying (video)
When it comes to video games that make you look absolutely ridiculous, nothing beats Microsoft's Kinect, but Nintendo's got a substitute if you just can't wait. The aptly-named Face Pilot: Fly With Your Nintendo DSi Camera! does exactly that, letting you literally direct a virtual hang glider with your face, by using the Nintendo DSi's camera to track your head in 2D space. Sure, Sony's EyeToy did similar things earlier this decade and you could program Windows-based FreeTrack software to do the same, but neither provide the portable hilarity (or challenge) of Face Pilot played on commuter rail. The downloadable title will set you back 500 Nintendo Points ($5) at the DSiWare store; watch a quick video demo after the break.
Microsoft hints at touchless Surface combining camera and transparent OLED (video)
We've always wondered whether Microsoft's multitouch table would actually ever arrive, dreaming of Minority Report hijinx all the while, but after seeing what the company's Applied Sciences Group is currently cooking up -- a touchless telepresence display -- we'd rather drop that antiquated pinch-to-zoom stuff in favor of what might be Surface's next generation. Starting with one of Samsung's prototype transparent OLED panels, Microsoft dropped a sub-two-inch camera behind the glass, creating a 3D gesture control interface that tracks your every move by literally seeing through the display. Combined with that proprietary wedge-shaped lens we saw earlier this month and some good ol' Johnny Chung Lee headtracking by the man himself, we're looking at one hell of a screen. Don't you dare read another word without seeing the prototype in a trifecta of videos after the break.
Video Kinect: video chat and stream sharing over Xbox Live and Live Messenger
Xbox Live and Windows Live Messenger are about to meet up in a very personal way -- Microsoft has just announced a new video chat service called Video Kinect, which serves as a logical extension of its brand new Kinect voice- and motion-sensing control system. But it's not just video calling, no sir, you'll be able to watch movies, news, sports, and the like together with whoever you're chatting with. Additionally, thanks to a motorized base and a new skeleton-tracking feature, the Kinect unit will also follow users as they move around the room. Yup, not creepy at all.
Microsoft's new lens tracks your face, steers 3D images to your eyes (video)
Glasses-free 3D has taken several forms, but most have a critical flaw -- viewers have to stand in predefined locations to get the effect. That just won't do, so Microsoft's prototyped a new approach, and it's one of the wildest we've seen. Taking a cue from Project N... we mean Kinect, cameras track the face while a special wedge-shaped lens traps bouncing light, and after the beams have reached a "critical angle," it exits towards the viewers eyes, aimed by programmable LEDs at the bottom of the screen. Since the system can beam a pair of simultaneous images to two different places, the obvious use is stereoscopic 3D, but researchers found they could also send different images to different viewers, as a sort of privacy screen. If that sounds far fetched, you're not alone -- but you'll find a video proof-of-concept at the more coverage link.
