Smart homes are a broken mess and Nest wants to fix it

Rishi Chandra has a vision for what a smart home should look like in five years. As the lead for Google Nest products, it's his job to think ahead. He doesn't have all the answers, he admits, but he has an idea of what he'd like to see. Given his role at one of the most influential companies in the world, his ideas about how to push the smart home forward could affect the rest of the industry. Based on what Chandra shared with me at a recent interview in New York, the future of the smart home involves a lot of playing nice, even with would-be rivals.

Chandra believes we are entering the fourth stage of computing. "We went from PC to web, web to mobile, and now we're going from mobile to ambient," he said. Ambient computing is the idea that devices like sensors, speakers and displays are embedded in the environment all around you -- thus, "ambient." Chandra also calls this distributed computing as opposed to integrated. In the latter, all the parts of the computer are in one central hub or device. Distributed computing, meanwhile, has its components scattered across a space. In Chandra's mind, these pieces fall into three categories: sensors, inputs and outputs. An input is something like a microphone in a remote or smart speaker or a touchscreen on a connected display while an output could be an Android TV, an Echo speaker or even a Philips Hue light.

There are a variety of devices that make up this ecosystem, and therein lies the problem: It's not convenient for the consumer. "There's no one thing I can put in my home and just put in the wall and be like, 'Oh, now it's all smart,'" Chandra said. Individual smart home companies sell bits and pieces of a puzzle, and to be fair, they're not wrong to focus on making the things they're good at making. Philips Hue, for example, has focused on lights, and part of its strategy has been "to ensure that partner brands work seamlessly with (its products)," according to the company's head of technology, George Yianni.

But fragmentation creates friction.

"As an industry, we're giving all the building blocks for this notion of a smart home, right? We sell you a thermostat, we sell you a camera, we sell you smart speakers, smart display, whatever it might be," Chandra said. "But then we ask the consumer to actually build the house."

This process of building the house is made even more challenging by the fact that there are so many different standards and protocols. Each company has its own app that you need to install. Some of them connect to your WiFi, others use Bluetooth. It used to be that you had to buy a separate hub to connect different brands of products to your home network. Thankfully, the industry has wised up and we're seeing signs of improvement -- brand-specific hubs are pretty much gone.

A spokesperson for Canary told Engadget "We wouldn't say the industry right now is immensely fragmented — if anything, the industry is now more consolidated than it has ever been."

But there's still no unified platform, and it's one of the biggest hurdles to the "smart home" becoming ubiquitous.

Niccolo de Masi, chief innovation officer at Honeywell spinoff Resideo, agrees. He told Engadget, "The home is becoming more complex faster than the average homeowner can keep up with." He added that consumers are still confused, frustrated and distrustful of their smart home devices. "This is the opposite of how they're supposed to work," he said.

Not only are there too many disparate products, but also they neither communicate with each other nor do they understand context. Your devices should know what you like and where you are so they can respond to you in the appropriate place. That's not quite the case right now.

"If you have multiple Google Homes today in your house, sometimes it'll answer in the wrong place," Chandra pointed out. That's been one of my biggest complaints about the Assistant, and I vehemently agreed when he brought that up. Thankfully, Chandra is aware of the issue and said, "We're working on that."

"But that, in my mind, summarizes the problem," he added.

Chandra believes there are two key pieces that are missing. "One is a connectivity layer that actually connects all these devices to share context in a trustworthy secure way," he said. This layer might be WiFi, Bluetooth Low Energy or Thread radios, a Nest-initiated low-power network protocol.

The second missing piece, according to Chandra, is an intelligence system. "It has to be able to process: 'Oh, here's all the signals I'm understanding, here's the right output,'" he explained. So the next time you're in your kitchen and you ask your Google Home for a cookie recipe, for example, it should display the steps on the Nest Hub rather than, say, your phone.

Ideally, for Chandra, the processing of these decisions would happen locally within the home, rather than relying on the Google Cloud. On-device processing is a core focus of the Face Match tool that debuted on the Nest Hub Max that Google announced in May. The smart display's camera will look out for faces of people in front of it to show personalized information and notifications. During the launch, Google stressed that the data involved in that recognition process stays on the Hub Max.

This means your data (like pictures of your face, in this case) won't be sent to the cloud for processing and the verification will happen on your device itself. This minimizes the chances for people to intercept and obtain your info. The idea doesn't just have privacy benefits either. Google's already planning to make its next-generation Assistant eventually run completely on your phone, without relying on an internet connection. "We're not stopping at the phone," Chandra said. "We want to continue to see how we can bring a lot of elements of the Assistant onto as many devices as possible to unlock this value of intelligence that sits inside the home as well."

With a powerful and secure contextual engine in place, smart home companies can focus on creating experiences that are helpful and intuitive. One of the improvements or shifts that Chandra would like to see is a switch from social personal computing -- that is, your phone existing solely to serve you -- to communal computing. That's when you have a device in your home that's meant to be used by several people who share your space.

"We have to understand the context of multiple people in the house and react to it," Chandra said. If he's listening to music by himself, for example, he wants the device to play his favorite songs. When one of his kids walks in the room, though, he wants it to be smart enough to not play anything explicit.

"That's kind of the next thing we have to go build," Chandra said. And a key component is making the transitions between personal and communal seamless. But how can Google go about doing this? "We're not going to ask users to log in and out of their homes," Chandra said. He points to some Google technologies as some of those building blocks -- Voice Match on Home speakers and Face Match on the Nest Hub Max.

There also needs to be a unified platform to help reduce the friction and frustration facing consumers who buy devices from different companies. "That's where I see the biggest innovation happening," Chandra said. He recalled a time when web developers had to code four different versions of their sites for the many different browsers and rendering engines that people were using. The industry aligned then with the W3C standardized protocols for web design. Chandra believes something similar must happen for the smart home.

Other companies generally agree. Yianni said that in future, "We expect more consolidation to happen." De Masi said that the future of IoT cannot be different products with a separate app controlling each one. The good news is, it sounds like we're closer to a simple unified smart home than we've ever been.