AR and Oculus Quest Passthrough
So as we all now know augmented reality (AR) is about placing virtual objects (graphics or data) on top of real world vision. There are two main approaches to achieving this.
The first involves the more hyped smart glasses, whereby images are projected onto the lenses of the glasses (e.g. Microsoft Hololens 2 and Magic Leap 1), or directly onto the eye.
The second uses external cameras. These cameras “pass through” the real world view, and add the virtual elements on the way. Think of VR headsets that are fully enclosed, but you can still see the real world through its external cameras. These cameras are common on VR headsets and mostly used for safety reasons.
At Headfelt we believe that it is the second, VR headset passthrough, that will offer the more immediate and functional delivery of consumer accessible Augmented Reality. There are a number of reasons for this.
Firstly, the failings of AR Smart Glasses/Headset:
- AR Smart glasses, well the cool looking ones, are not yet a reality
- The current AR Hololens and Magic Leap headsets are enterprise (Business) units. They are expensive and not overly robust. Not for the consumer!
- Smart Glasses/Headset image quality and Field of View (FOV) still require significant improvement
In favour of the VR headset:
- They are common readily available consumer devices
- They are cost effective, $400 - $1500
- The camera passthrough method enables higher quality, control, and more immersive augmented content
The reason VR passthrough provides a better quality AR experience is that it is not trying to map virtual assets onto real world elements that you are seeing directly with your eye. Passthrough ‘pre-renders’ the virtual assets onto the relevent real world elements in the video stream, before it is displayed to you. This is far more efficient, more robust, and produces a much more immersive result.
Now consider Oculus’ recent release of the Quest 2 API to developers. In short, they are starting to open up access to the camera for developers to start creating this augmented content and applications. Although the current camera is B&W, it is expected that the new Quest (Project Cambria), due for release in early 2022, will have a colour feed.
Once this happens we expect a massive growth in the creation and use of functional consumer focused AR applications. Just imagine the VR headset as more than only an entertainment device, such that it is specifically worn at times when functional AR needs arise e.g. While cooking dinner and following a recipe, or when repairing the lawnmower, learning how to knit, or redesigning the backyard.
Below are some recent, ‘early’ videos that show the foundations of what we will be building upon.