Last week, I sat around a table with fellow journalists as Greg Sullivan, Microsoft's head of Mixed Reality, detailed the company's vision for the future of virtual collaboration. Nobody was wearing masks or standing apart. We weren't worried about getting sick. Instead, we were all wearing HoloLens 2 headsets and sitting in different parts of the world. The holographic table was right beside my actual desk, and my media pals were floating around my office as we chatted with our cartoonish avatars. For a second, it felt like mingling in real life during the Before Times.
Gallery: Microsoft Mesh | 7 Photos
Gallery: Microsoft Mesh | 7 Photos
We were experiencing one of the first apps powered by Microsoft Mesh, the company's ambitious new attempt at unifying holographic virtual collaboration across multiple devices, be they VR headsets, AR (like HoloLens), laptops or smartphones. Powered by Microsoft's Azure cloud, Mesh isn't just an app, it's a platform that other developers can use to bring remote collaboration to their own software. If remote work is here to stay — and by most accounts, it is — Microsoft wants to be the company taking us beyond Zoom video chats, and towards holographic experiences that everyone can join.
"Not only are we going to be able to share holograms, but we'll be able to do so in a way that gives us agency and presence," Sullivan said during our virtual meeting. "We can create these experiences, where even though we're physically separated, it feels like we're in the same room, sharing in an experience and collaborating on a project."
While we've seen a solid stab at virtual collaboration from Spatial, Microsoft is attempting something even more complex. Sullivan likens Mesh to the launch of Xbox Live in 2002, a service that dramatically simplified online multiplayer gaming for consoles. It made it easier for developers to connect their games to the internet, and led to a boom in online multiplayer titles for the Xbox and Xbox 360. That gave Microsoft a strong leg up on Sony and Nintendo, both of which took years to catch up.
Microsoft is using today's Ignite conference keynote to show off the capabilities of Mesh. Alex Kipman, the company's Technical Fellow behind the HoloLens and Kinect, will hit the stage as a real-time hologram (something Microsoft calls "holoportation"). Think of it a bit like the holographic messages we've seen in Star Wars and other science fiction stories. It's not photo realistic, but if you're wearing a VR headset, it's almost as if he's in the room with you. On a standard monitor or phone screen, it may just come off as a hokey special effect. But it's not hard to imagine eventually slipping on an AR headset like the HoloLens 2 and watching a hologram presentation right in your living room, as if you were sitting in the front row during a show.
My Microsoft Mesh demo, to be clear, was nowhere near as impressive. Our avatars were simplistic, with detached arms and limited facial movement. It was like being surrounded by a bunch of Nintendo Miis. But there was still a decent sense of immersion: I could tell exactly where everyone was even when I had my eyes closed, thanks to realistic audio processing. And we were able to collaborate with 3D models, passing them around the table and resizing them to our heart's content.
While we were looking at fairly basic 3D figures, Sullivan pointed out that Mesh can also stream high-quality models from the cloud (it's powered by Azure, after all). That would allow designers and engineers to collaborate with the same assets they're using on their workstations from anywhere in the world. That's what film director and producer James Cameron is aiming to do with his upcoming series, OceanXplorers. The non-profit behind that show, OceanX, plans to create a Mesh-enabled "holographic laboratory" on its advanced ship, allowing scientists on-board and remotely to collaborate around 3D models.
“The idea is to take all this amazing scientific data we’re collecting and bring it into a holographic setting and use it as a way to guide scientific missions in real time,” Vincent Pieribone, vice chairman of OceanX, said in a statement. It would let researchers huddle around data and chat as they would in real life, no matter how far apart they actually are.
