Related Articles
VR headsets take you places and AR headsets show you things in the real world that aren’t really there. The next step is likely to be a wave of camera-filled VR headsets that are going to blend the real world with the virtual. They’ll also be able to track your room and your face at the same time. Call it mixed reality, XR or anything you want. Whatever you call it, VR’s going to get weirder. Qualcomm’s new AR/VR tech will be a big part of that. It’s coming as soon as next year, from a lot of different companies.
At the end of 2019, we’re still not living in a world where people are wearing magic virtual glasses everywhere. But we’re getting closer. VR headsets such as the Oculus Quest and AR headsets, which include the Hololens 2, are phone-free, totally wireless and both use chips made by Qualcomm. In the past, my peeks at early Qualcomm hardware were good indicators of where the VR and AR landscape were heading, for that very reason.
Qualcomm’s new 5G-compatible Snapdragon XR2 chip, announced today, could be the processor that drives all future headsets like Oculus Quest, Hololens and others in the next year or more. Qualcomm is readying a VR-based reference design with XR2 early next year and is separately working on a project with Niantic Labs to develop a reference design for smart glasses.
I got to take a quick peek at what’s coming. It seems promising, but it also raises a lot of questions.
A wave of next-gen 5G VR/AR headsets, without phones
Qualcomm’s new XR2 chip sounds powerful: It’s a VR/AR-specialized variation of the company’s newest Snapdragon 865 chip. Graphics should be twice as good as what’s on the current (and already excellent) Oculus Quest, and much higher display resolution should mean crisper visuals. Don’t expect retina-level displays, but 3K resolution per eye (that’s 2,880×2,880 pixels at 69 pixels per degree and at 90Hz) will be a big leap over current VR technology. Qualcomm says 8K video will play on these processors at 60fps, or 4K at 120fps.
5G is also possible on VR and AR headsets using this chip, using both sub-6 GHz and millimeter wave (mmWave) parts of the spectrum, but it’s more of an optional piece than an essential one. Qualcomm’s head of XR, Hugo Swart, says that 5G will be used for enterprise-based (read: more expensive) headsets for now. 5G should be a big deal for VR and AR eventually, especially with the promise to remotely render graphics in the cloud and reduce the load on the headset itself, making standalone headsets do more than they currently can. Microsoft has been promising remote rendering with Hololens 2, but it’s not been used yet in mainstream devices. “There is a strong interest and momentum with carriers to have the 5G version, so we may see an accelerated path there as well,” Swart says.
Qualcomm’s vision for XR2-powered smartglasses are a lot sleeker than VR headsets, and also studded with inner and outer cameras.
Qualcomm
Cameras to track your inner and outer worlds
The wildest and weirdest part of the new chipset, though, is its support for seven cameras at once. Cameras are crucial in VR and AR: VR cameras track movement in the world, such as hand gestures and eye tracking. Qualcomm’s Swart sees more simultaneous cameras allowing for more face tracking, with cameras able to track facial movements, lips and microexpressions. Enterprise headsets could use cameras that sense other things. (Could that mean thermal cameras? Improved depth sensors?)
Up to 7 cameras could run at once, but Swart said there could be as many as 12 cameras added to a headset with XR2, with the headset cycling between which cameras are used at any time. Qualcomm’s early prototype headset also has hand-tracking sensors made by Ultra Haptics, which didn’t work yet on the demo I tried. But the idea is that this next generation of headsets could be studded with sensors which handle better motion tracking, hand tracking, eye tracking, live collision avoidance of in-room obstacles and more.
All those cameras also raise questions of privacy. Where is all this information going? Swart says Qualcomm has a “strong security system” in place, but didn’t elaborate. How will companies like Facebook and others use more camera-studded, always-tracking headsets like these? It’s a big unknown.
I saw an extremely early pre-reference design prototype using the Snapdragon XR2 chipset.
Nic Henry/CNET
A hands-on peek at the hardware’s promises
Qualcomm’s Swart says that companies are already working with the chip hardware, and headsets using Snapdragon XR2 will be coming by the end of 2020. But there isn’t even a final reference design for an XR2-based VR headset yet. What I got to demo was an early, early prototype, running an app from avatar-based workplace collaboration app Spatial at their New York headquarters. (The hardware you see here isn’t at all what future hardware would look like.)
Spatial’s CEO and co-founder, Anand Agarawala, guided me through a multidevice collaborative demo where I used Qualcomm’s XR2 hardware, while others worked on the Oculus Quest, the Hololens 2 and an Android phone. We all appeared as hovering avatars together. The XR2’s face-tracking and hand-tracking weren’t in place yet — the demo was clearly in an early state. But I got to see that the display resolution was a lot sharper than what I’m used to on Oculus Quest. A virtual sea turtle that floated in front of me looked vivid. Documents in a virtual conference room had far more readable text, even with smaller fonts.
Snapdragon XR2 VR will do hand tracking and face tracking simultaneously.
Nic Henry/CNET
I didn’t get to see Snapdragon XR2’s biggest killer app: It’s able to pull things from the outside world into VR, creating a pass-through augmented reality in VR. I’ve seen VR that does things like this, and even the Oculus Quest dabbles in it a bit to paint room boundaries in your actual room with the headset on. But future XR2-powered VR headsets should blend reality far more seamlessly, and Qualcomm’s Swart promises VR headsets will start offering AR capabilities at a lower price while smaller, sleeker smart glasses are still being developed. It reminds me of what Mark Zuckerberg was telling me earlier this year at Oculus Connect.
“I was skeptical at first, but when you try it… ‘oh, I can write on a whiteboard, I can move around, I can totally use this but it’s not an order of magnitude more cost,'” says Spatial’s Agarawala. “It’s going to be what the next wave of devices, Quest 2, et cetera, are probably going to feel like.”
Agarawala also feels bullish on where face tracking could take Spatial’s avatars next. “The ability to put a camera to face your lips … our most realistic avatars could get more realistic. They can also get facial expressions, that sort of stuff, non-verbal.”
It reminds me of Facebook’s research into next-generation avatars, ones that can mirror us, read our expressions and allow possible subtle gestures that start to become impossibly real.
And based on all indicators from Qualcomm, we’re headed there soon enough.
