Facebook hopes to prove AR is more than selfie filters and games
But is its vision of augmented reality ads any less gimmicky?
As I sit surrounded by software engineers in a conference room with no natural light, playing augmented reality games on an iPhone, I forget for a second that I'm in Seattle visiting Facebook. Not Amazon or Microsoft. Facebook, a company that's evolved from a simple social network to a full-on technology behemoth. Here, inside the company's largest engineering hub besides its Menlo Park headquarters, Facebook says people are working on many of the projects that will impact its 10-year road map and mission of "bringing the world closer together," including Games, Groups, Messenger and, of course, ads. But I'm here to talk about one particular emerging technology that the company believes will be key to its future: augmented reality.
Over the past couple of years, Facebook has been investing heavily in AR and trying to prove that it can work not just for games but for ads in the News Feed or to help brands sell shoes and phones in Messenger. Facebook sees combining digital objects with the physical world as the perfect way to keep people tied to its services. On Messenger, for instance, not only can you video-chat with your friends, but now you can also play AR games with them when the conversation needs a little pick-me-up. And the more time you spend using a Facebook product, be it Messenger or Instagram, the more money the company makes. That's why AR isn't simply an experiment for Facebook -- it's a potential gold mine.
Aside from its acquisition of Oculus in 2014, there's evidence that suggests the company is working on its own augmented reality hardware too, based on a patent application it filed in 2017 for a pair of camera-centric spectacles. This concept, which Facebook co-founder and CEO Mark Zuckerberg first touched on at the 2016 F8 developer conference, indicates that the glasses could use a waveguide system to project images and light into the wearer's eyes. It's similar to the technology powering Microsoft's HoloLens and Magic Leap's $2,300 AR headset. Whether Facebook ever actually launches an AR headset, it's still working to create an augmented reality ecosystem that may be rivaled only by Apple, which has the benefit of having its ARKit baked right into millions of iPhones and iPads.
In 2017, Facebook went all in on AR with the introduction of its Camera Effects Platform, a developer system for augmented reality experiences. Companies could use for everything from displaying nutritional facts on food you're looking at to showing your latest running stats on selfies. AR Studio was born at the same time, which makes it easy to build animated frames, masks and other effects to be used during videos, including Facebook Live broadcasts -- all without needing to know how to code.
As with Apple, Facebook's AR hopes are helped by the fact that its products are being used by billions of people, and that creates an enticing environment for both developers and brands. What Facebook wants with its Camera Effects Platform and AR Studio is to shape the whole augmented reality medium, not just take it on as a hobby.
With Facebook having 2.23 billion monthly active users on its site, as well as 1.3 billion and 1 billion on Messenger and Instagram respectively, its AR projects enjoy a reach that its Silicon Valley rivals can only dream of. While these numbers clearly show the company's digital prowess, they also highlight a potential land mine. Facebook controls virtually unmeasurable stores of data about its users and controls the flow of that information between itself and countless apps and services.
But Facebook isn't the only social media giant that's embracing augmented reality. Snapchat, which has 188 million daily active users, has been building a rather extensive ecosystem that includes letting people create their own AR effects. With Lens Studio, introduced in 2017, the company made it easier for users, brands and advertisers to design and bring 3D objects to life in its app -- like that dancing hot dog no one can ever forget about. Snapchat's goal is to use AR to let users put animations everywhere, including their face and the environment around them.
While the Camera Effects Platform and AR Studio are undoubtedly essential to Facebook's overall AR strategy, since they're going to be the birthplace of its third-party experiences, the company has its own projects to work on as well. It's now making a big push into multiplayer AR games, starting with Messenger. During our visit to its Seattle office, Facebook gave us a look at some of the titles it'll be launching, such as Kitten Craze, which lets up to six different users play a game where they have to move their head to avoid falling cats. It's like Galaga, except your enemies are kittens, not aliens.
In Just Beachy, another AR game arriving soon on Messenger, the premise is slightly different than in Kitten Craze. Instead of trying to reach a high score, you use your head to "bounce" a beach ball back and forth with people you're video-chatting. And you can add AR stickers like sunglasses and hats to your face. Multiplayer titles such as these are being developed in-house by Facebook's Real Time Communication team in Seattle. But beyond making things fun for users, these engineers also have to ensure that their games aren't draining your battery and there's no lag as you play them with your friends.
Jason Clark, engineering director of Real Time Communication at Facebook, told Engadget that those are two of the biggest challenges the company faces when building AR projects. Still, he added, Facebook is only scratching the surface in terms of what it can do with augmented reality -- and it needs to figure out how users respond to it. "The technology is only half the story," Clark said. The other half, he said, is Facebook's "continued focus" on making AR experiences that are going to resonate with people, whether that be a game like Kitten Craze or something else entirely. Clark said that augmented reality has the potential to go beyond gimmicks and provide "real, tangible value," since it can mix the virtual and real worlds together. That's what sets it apart from other technologies like VR, he said.
Another area where Facebook is experimenting with AR is the movie industry, in partnership with studios such as Universal. Just last week, the companies demoed an AR experience that brought Jurassic World's beloved velociraptor, Blue, to life through the Messenger camera simply by scanning a code on a physical copy of the film. Sure, it's a marketing gimmick, but it shows that Facebook is exploring multiple avenues to make the most out of augmented reality. "AR is very much an enabling technology," said Matthew Simari, Facebook's product manager of Camera AR. "What you do with it and how that helps you connect and build relationships with people around you, I think that is the really interesting nugget."
"The [augmented reality] technology is only half the story."
To Simari's point, Facebook believes it can use AR to "connect people" in different ways, beyond games, movie tie-ins and silly selfie filters. And not just with one another but also with the brands advertising on the site. That's why the company is also bringing augmented reality ads to its News Feed. The new feature, which will launch later this year, will let Facebook users "try on" items as they browse, including things like sunglasses and cosmetics. And brands such as Michael Kors are already on board. While AR ads in the News Feed will work only in the US at launch, Facebook does have plans to expand its availability in the near future.
And then there's Instagram, which recently hit 1 billion monthly active users and has quickly become one of Facebook's most popular products since it acquired it in 2012. You can see traces of augmented reality in the dozens of photo filters people use every day -- from a cannon that shoots out cute red hearts to the bunny ears people love putting on their face so much. There aren't any multiplayer AR games on Instagram right now, but Clark said Facebook "only [knows] by trying" different things and that it is "reasonable to generalize that playfulness is universally appealing."
In other words, don't be surprised when games like Kitten Craze and Just Beachy start to make their way to Instagram. If that does happen, real-time gaming on the app could work in a similar way to how it does on Messenger, especially now that Facebook has started to roll out video group chats for Instagram. That said, even though it's obvious that the company has big ambitions for augmented reality, it's going to need to show a lot more than just gimmicky selfie filters, games and bonus film content if it wants to take the medium to the next step.
Facebook clearly hopes that AR will play a crucial role across its portfolio, even if it's still trying to figure out where it makes the most sense for people to use. And the company will have to do some serious work to assuage the fears of those users. Facebook's trove of personal data has been turned against us and compromised before: the Cambridge Analytica scandal has already proven that a lackluster privacy policy can easily be weaponized. And things only seem more dystopian when you imagine having information about your political inclinations and personal relationships floating over your head in AR.
But Facebook isn't backing off. Its augmented reality push is serious, and it seems as if it's only going to get more so as time goes on.
