Latest in Gear

Image credit: Cornell University

Cornell researchers created an earphone that can track facial expressions

The team says C-Face works even when the subject is wearing a mask.
Kris Holt, @krisholt
October 12, 2020
147 Shares
Share
Tweet
Share

Sponsored Links

Facial expression tracking system C-Face
Cornell University

Researchers from Cornell University have created an earphone system that can track a wearer's facial expressions even when they're wearing a mask. C-Face can monitor cheek contours and convert the wearer's expression into an emoji. That could allow people to, for instance, convey their emotions during group calls without having to turn on their webcam.

“This device is simpler, less obtrusive and more capable than any existing ear-mounted wearable technologies for tracking facial expressions,” Cheng Zhang, director of Cornell’s SciFi Lab and senior author of a paper on C-Face, said in a statement. “In previous wearable technology aiming to recognize facial expressions, most solutions needed to attach sensors on the face and even with so much instrumentation, they could only recognize a limited set of discrete facial expressions.”

The earphone uses two RGB cameras that are positioned below each ear. They can record changes in cheek contours when the wearer's facial muscles move. 

Once the images have been reconstructed using computer vision and a deep learning model, a convolutional neural network analyzes the 2D images. The tech can translate those to 42 facial feature points representing the position and shape of the wearer's mouth, eyes and eyebrows. 

C-Face can translate those expressions into eight emoji, including ones representing neutral or angry faces. The system can also use facial cues to control playback options on a music app. Other possible uses include having avatars in games or other virtual settings express a person’s actual emotions. Teachers might be able to track how engaged their students are during remote classes too.

Due to the impact of COVID-19, the researchers could only test C-Face with nine participants. Still, the emoji recognition was more than 88 percent accurate and the facial cues more than 85 percent accurate. The researchers found the earphones' battery capacity limited the system. As such, they plan to develop less power-intensive sensing tech.

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
147 Shares
Share
Tweet
Share

Popular on Engadget

Chevy will start selling EV retrofit kits in 2021

Chevy will start selling EV retrofit kits in 2021

View
Nest thermostats in the US and Canada can now monitor your HVAC system

Nest thermostats in the US and Canada can now monitor your HVAC system

View
Disney robot with human-like gaze is equal parts uncanny and horrifying

Disney robot with human-like gaze is equal parts uncanny and horrifying

View
$149 Playdate handheld is 'ready to go,' orders start in early 2021

$149 Playdate handheld is 'ready to go,' orders start in early 2021

View
AI can detect COVID-19 by listening to your coughs

AI can detect COVID-19 by listening to your coughs

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr