Latest in Gear

Image credit: Facebook

Facebook’s AI-created virtual memories are magically haunting

Wandering through pointillistic art felt like being in a dream.
595 Shares
Share
Tweet
Share
Save
Facebook

Sponsored Links

Ever since Facebook bought Oculus, it's been investing in virtual reality in a big way. Beyond just hardware like the Rift and the Go, it's also putting a lot of money and research into software. It delved into the realm of social last year with Oculus Rooms and Facebook Spaces, but that's not where Facebook's VR ambitions stop. At F8, the company unveiled a new way for users to experience VR: by recreating their memories.

At the F8 keynote this week, head of social VR Rachel Franklin showed off an experimental feature where Facebook's machine learning algorithms could convert a 2D photo or video into a virtual three-dimensional environment through point cloud reconstruction. According to Facebook, the tech uses "photogrammetry" to create 3D space from flat photos or videos. This means that you could conceivably transform a photo of your 10th birthday party into a virtual recreation of your childhood living room.

I went through a demo of this feature at F8, and it felt a lot like being in a dream. Thanks in part to the point cloud reconstruction mentioned earlier, the virtual room that I was in was not lifelike in the slightest. Instead, it was as if I stepped into a world of pointillistic art, which lent the environment an ethereal, magical quality. When I waved my cursor hand around, the dots that make up the room shifted and moved, emphasizing the unreal nature of the world around me.

Scattered throughout the place -- which seems to be a recreation of a real Oakland home -- were cut-out squares floating in the air. When I hovered over them with a cursor, I saw thumbnails of photos and videos, all of which were supposedly taken in the room that I was in. When I clicked on the thumbnails, I teleported over to them so that I could see the photos and videos up close. One was a photo of a family, while another was a short video clip of a young couple getting ready for prom.

The next environment I was in was yet another living room, and this time all of the photos and videos were taken during a birthday party. The thumbnail near the couch revealed a posed photo of several attendees, while the one by the dinner table showed the crowd around the cake. In the very last scenario, I was transported over to a pointillist version of the Palace of Fine Arts in San Francisco. The photos taken at this location were lined up in a path surrounding the lake, inviting you to follow along.

Facebook told me that this pointillism effect is used to help fill in the gaps, and to "create a sense of presence in the space." But more than that, the fuzzy nature of the environment also evokes a sense of memory. After all, most people don't dream in HD. Instead, dreams and memories are misty and hazy, and a pointillistic world where objects are blurry and ill-defined contributes to that feeling immensely. The fact that the environment was peppered with real-life photos was pretty surreal as well. Teleporting from one photo to another felt like I was hopping in and out of reality. I sometimes felt as if I was going through forgotten memories like in Eternal Sunshine of the Spotless Mind.

According to Facebook, the demo I experienced is just a prototype, and it has no plans to share specific implementations. In the future, the company hopes to help people share moments in "new and rich ways." "We think in the future, people may be able to use technology like this to revisit moments and places that are important to them, with people they care about," said a Facebook spokesperson in a statement.

In short, think of this as a Facebook album come to life. I can imagine a future where instead of forcing your relatives to sit through a boring vacation slideshow on the TV, you'll be strapping them to headsets so that they can relive your trip to Italy with you. They still might not appreciate it, but at least it won't be quite as boring.

Click here to catch up on the latest news from F8 2018!

In this article: f8, f82018, facebook, gear, internet
All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
595 Shares
Share
Tweet
Share
Save

Popular on Engadget

'The Outer Worlds' DLC is coming next year

'The Outer Worlds' DLC is coming next year

View
TCL's 2018 65-inch 6-Series 4K TV drops to $500

TCL's 2018 65-inch 6-Series 4K TV drops to $500

View
The $1,399 Pixelbook Go with 4K display is now available

The $1,399 Pixelbook Go with 4K display is now available

View
US cancels plans for new penalty tariffs on Chinese-made products

US cancels plans for new penalty tariffs on Chinese-made products

View
Tesla Cybertruck will likely get medium-duty truck classification like Ford Super Duty and others

Tesla Cybertruck will likely get medium-duty truck classification like Ford Super Duty and others

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr