Exclusive: Inside the new display tech that makes virtual objects seem totally real
The thing about interactive head-mounted displays is that they hold great promise, but so far, have resulted in unnatural optics, awkward social interactions, and an overall sense of discomfort, whether it’s about the products themselves or the future that we seem to be barreling toward. Or, you could cite an example like Microsoft’s HoloLens, which, as a standalone PC on your face, is an impressive technical feat. But so far it has found a niche firmly in the enterprise market.
That’s what a company called Avegant thinks it can change. Avegant is most well-known for the Glyph, first introduced in 2013. The Glyph (now just called the Personal Video Headset) uses a patented technology called virtual retinal display, which is a technical way of saying the company came up with a way to shine videos directly into your eyeballs. But about two years ago, technologists at Avegant began looking into transparent display technology, and eventually landed on the thing it believes will someday replace our smartphones: light field displays.
For this episode of Next Level, The Verge video team and I visited Avegant’s headquarters in Belmont, California, and I experienced the latest version of Avegant’s light field display headset prototype. My co-worker Nick Statt also tried this back in March, but Avegant says the prototype has been significantly refined since then, and that no one has captured it on video before this.
From the exterior, Avegant’s headset prototype doesn’t look any more refined than most other “mixed reality” headsets we’ve tried. And unlike Microsoft’s HoloLens, it’s tethered (attached via cable) to a desktop PC running Unity’s game engine. But inside the headset’s optics engine is where things get interesting. Avegant is using what’s known as a “multi-focal plane approach” to send digital imagery directly to your eyes in a way that’s supposed to replicate the way our eyes naturally perceive things and shift focus.
It’s a highly technical concept, one that we explain in more depth in the episode, but one way to imagine it is this: normally, when we’re wearing “smart glasses” or another kind of AR / MR headset, you’re often seeing a flat, 2D image on the display, one with a fixed focal point. And when that display is mashed up against your face, it just doesn’t feel natural.
By creating a digital approximation of how our eyes normally perceive light fields, Avegant has made a headset that allows your eyes to shift focus from digital object to digital object and feel more… normal. So, as I shifted my eyes from planet to planet in a digital Solar System that floated around me, my eyes didn’t feel strained. Objects had volume, and although they were digital representations, they looked realistic. The same went for when I was holding digital objects in my hand, and moving my arm from a near position to an outstretched one.
All of this might have anyone who has followed the mixed reality space asking: But how does it compare to Magic Leap? Magic Leap is the super-secretive, Florida-based company that to date has raised more than $1 billion to finance development of a light field headset. The short answer is: we don’t really know yet. Relatively few people have seen or tried Magic Leap’s tech, and not surprisingly the company declined to share more specific details by phone. But Magic Leap has shared that it’s working on what you might call a “full stack” solution: it’s building the computing system, the processors, the headset, the software, and they’re even said to be working directly with content creators.
Avegant’s plan, on the other hand, is to license its technology to other hardware makers who either already are or looking to get into head-mounted displays. Chief executive Joerg Tewes says that Avegant’s technology will start showing up in other headsets by 2018, largely in the commercial space at first, but with consumer versions to follow.
But Avegant’s grander vision is that it’s this kind of technology that will make head-mounted displays so good, that they will eventually replace our smartphones. Both Tewes and chief technology officer Ed Tang told me that they think it’s only a matter of time before people are carrying around lightweight, light field glasses that can show us the information we need right in front of our eyes, which would supplant the glass rectangles we all stare at now.
“I think from the very get-go, one of the most exciting that things we saw happening in the space was this trend toward wearable computing,” Tang said. “We always knew that, as things were moving to mobile devices like phones and tablets, that the next evolution of that was this idea of the ambient computer. Something that I can put on like glasses, and wear, and suddenly I’m surrounded by these experiences, and not limited to the screens that we can fit in our pockets and in our bags… so first we set out to get rid of the screen.”
For more on how Avegant thinks it can make this kind of mixed reality our new reality, check out the Next Level episode above.