Microsoft Mesh uses AR to bring your co-workers, or the ocean, into your home

Microsoft today unveiled Mesh, a new mixed reality platform for Microsoft Teams meetings that might one day take your coworkers off the screen and into your home. Using a headset like the company's HoloLens 2, digital representations of participants appear in the space around you, creating a feeling that they're right there with you even when they're geographically distributed. Participants can walk around and interact with shared 3D objects in order to, for instance, collaborate on product designs.

Mesh was demoed today at Microsoft's Ignite conference using the HoloLens 2, but Microsoft says the product will work with other company’s virtual reality headsets, tablets, smartphones, and PCs. Mixed reality glasses like HoloLens will provide the best feeling of actual presence as wearers see and can interact with digital objects overlaid in their field of view. A preview of the Mesh app is available for HoloLens 2 starting today.

Beam me up — Initially, Mesh will present digital representations of people using virtual avatars from social VR platform AltspaceVR. Although virtual avatars that communicate body movements do provide some level of intimacy, it's still a simulacrum of actually being there with another person. For that, Microsoft says Mesh will eventually use "holoportation" technology to beam people into others' spaces and allow them to appear as photorealistic versions of themselves. We're going from Zoom meetings in 2020 to Star Wars in 2021.

Here's how The Verge described its experience testing Mesh:

During my hour-long meeting in Microsoft Mesh, I constantly felt like this could be a far-future version of Microsoft Teams. Kipman appeared next to me as an avatar and started handing me virtual jellyfish and sharks. I could reshape the animals, pass them back, or just place them down in front of me. Although we weren’t working on some grand design or 3D model, it felt far more immersive than the Zoom video calls I have to attend on a near-daily basis.

The preview video of Mesh looks a lot better than the demo, but if Mesh works anything like it's actually supposed to, it'll no doubt be a pretty incredible feat. Virtual reality meeting spaces like AltspaceVR already provide an experience that tricks the brain into thinking it's inside a digital world.

But practically speaking, for those among us who are too self-conscious to leave our cameras on during Zoom calls, having our entire bodies transported into a coworker's home sounds kind of terrible.

Business applications — Microsoft's Alex Kipman, the inventor of HoloLens (and Kinect before that), envisions Mesh as a way to enable hybrid work as corporation's rethink their office footprints. That corporate focus will probably be important at first, as a HoloLens 2 headset costs $3,500. Companies might be willing to drop down that kind of money if Mesh can cut down on unnecessary executive travel when a virtual meeting could suffice.

Even then, Mesh is going to need to offer use cases that are a real improvement over the teleconferencing that we're all used to. Just because you can teleport into someone else's space, doesn't mean you should. To that end, Microsoft recruited major names like filmmaker James Cameron and Pokémon Go developer Niantic to demonstrate its potential at the Ignite keynote today.

Games and concerts — A proof of concept version of Pokémon Go and virtual concerts are two ideas that Microsoft thinks demonstrate how Mesh could help people connect virtually. If corporations continue to embrace remote work to the degree they are today, AR could potentially help maintain team cohesion and empathy that some leaders worry will be lost with remote work.

Anyone who's tried AR products like Snapchat's clothing Lenses that let users "try on" 3D shoes will know that overlaying digital objects in the real world can communicate ideas better than looking at pictures on screens. Many tech companies including Facebook and Snap are hoping to get in on the AR/VR movement as it promises to be the next major platform for personal computing after the smartphone.