Behind the quest to control a wheelchair with a smile

Facial recognition software has earned a difficult reputation over the past few years, what with its massive privacy implications and ease of being misused by governments and retailers alike, but the technology has just as many beneficial applications. Take the Wheelie 7, for example.

While travelling a few years ago, Dr. Paulo Pinheiro was struck by a scene unfolding before him. "I saw a girl at the airport, she was in a wheelchair," he explained to Engadget. "She couldn't move her arms or her legs, her father was helping her with her wheelchair but she had a great smile... so I thought it would be a good idea to try to translate that smile into commands that could move a wheelchair."

That experience proved to be the impetus in his decision to found HooBox Robotics. For the past two years he and his team have been developing the Wheelie 7, an AI-driven face-reading accessibility accessory that enables users to control their motorized wheelchair with nearly a dozen facial expressions. And for the past 18 months, the HooBox team has been working with Intel's Software Innovators Program to accelerate the system's development. Specifically, Intel helped integrate its 3D RealSense camera into the Wheelie 7 kit.

"The problem was that to control a wheelchair you need not just facial recognition, you need high precision facial recognition," Pinheiro explained. "So that if people are using a smile to stop their chair, you want to make sure that it works one hundred percent of the time, right?"

Intel's camera helps do just that. As such, HooBox's system can accurately detect one of 10 facial expressions in a variety of lighting conditions. What's more, the camera doesn't need to be directly facing the user -- it can be offset from center by up to 60 degrees and still catch the user's subtle facial movements.

Despite its impressive capabilities, the Wheelie 7 does not require a large processing footprint. In fact, the system doesn't even need an internet connection. "Everything is running locally," Pinheiro said. "We can embed this technology in almost any platform, so you can use your laptop if it's running a Core i5 or better. You don't have to use GPUs or the cloud, you don't have to use any extra processor power."

Interestingly, for all that the kit can do, there's one feature conspicuously missing. "At the beginning, we started with an algorithm that can detect when you're having a conversation with someone so that it will disable the interface," Pinheiro explained. In fact, the first prototype iteration of the system included that capability but was subsequently removed from later versions due to negative user feedback.

"When they are going to start a conversation they prefer to enable and disable the interface by themselves," he continued. Now users can use a combination of expressions -- raising one eyebrow, then the other for three seconds apiece -- to turn the interface on and off.

Currently, the Wheelie system only requires five of the available 10 facial expressions to control the chair, which means that future iterations will be able to control other functions and even other accessories. And, to a degree, it already does. The system is compatible with Amazon's Alexa, enabling Wheelie 7 users to control various aspects of their smart homes, such as turning the lights on and off and adjusting the thermostat.

The company is also working to develop a "vocabulary of facial expressions" which will allow users to write full sentences through various facial gesticulations, rather than simply select them from a pre-programmed list as many do today.

Eventually, Pinheiro hopes to advance the technology to the point that it doesn't just understand facial expressions but human behavior as well. To that end, HooBox is currently working with one of the largest medical facilities in Brazil, where the company is located, to develop a system for measuring patient's pain levels via a camera mounted on the ICU bed. There's no word yet on when the system will be ready for widespread adoption but, should it work out, it could revolutionize the ways in which modern medicine grapples with pain management and opioid addiction.