A Brief History of AR & VR: Merging Realities and Technologies

    The time has come when we can finally say that high-end consumer virtual reality (VR) is here. Both the Oculus Rift and HTC Vive have begun shipping to consumer audiences across the globe, bringing with them brand new worlds of possibility. Augmented reality (AR) however, is a different story. There’s still a lot of ground left to cover before AR can become a viable household technology, but where once both technologies were born they both shall converge in due course.

    Though the roots of AR and VR can be traced far back beyond the birth of modern computing technology, it’s Ivan Sutherland who is often credited with the development of a design which is now most familiar: the head-mounted display (HMD). Back in 1968, while working as the Associate Professor of Electrical Engineering at Harvard University, Sutherland created The Sword of Damocles. A display suspended above the user, the system was capable of displaying simple wireframe drawings with the illusion of stereoscopic 3D.

    However, Philco, a US firm based in Philadelphia, contests that it was actually they who created the first HMD. In 1961 a system was developed in which a camera was placed in one room, while the user sat in another with the display. Magnets were used to adjust the position of the camera in accordance with the user’s head movement: the first example of the potential for telepresence.

    The HMD is a construct used for almost all modern AR and VR. Aside from entertainment properties such as the PlayStation’s Wonderbook and Eye Pet franchises or mobile applications including the likes of Nokia’s HERE City Lens, the HMD forms the backbone of AR in the same way that VR is more commonly associated with an enclosed device opposed to Caves.

    AR has already been successfully implemented in many forms, however it’s the adaptation of the technology to improve or assist daily life that is considered the proving ground for the technology. Louis Rosenberg’s Virtual Fixtures, developed in 1991 at the U.S. Air Force Armstrong Labs, was designed to allow for the manipulation of virtually guided machinery from a remote location. Rosenberg’s subsequent research into AR, included virtual overlays atop of the user’s real-world view: the most commonly discussed implementation of VR in the current era.

    Successfully adapting this technology for real-time broadcast was first achieved in 1998, when Sportvision developed the 1st & Ten system, which generated the now widely used yellow ‘first down’ line that a television viewer sees during live football broadcasts. This technology was actually based upon an earlier design for Ice Hockey, in which a blue halo was used to demonstrate where the hockey puck was located at all times, however this implementation was not well received by the sport’s general viewing audience.

    A Brief History of AR & VR: Merging Realities and Technologies

    Just one year later, ARToolKit saw its first release. An open-source tracking library that allowed for the creation of overlay information, first developed by Hirokazu Kato of Nara Institute of Science and Technology. ARToolKit is arguably the first instance of consumer-grade AR, with a mobile SDK having been developed and incorporated into early smartphone handsets such as Symbian in 2005. The SDK allowed developers access to video tracking capabilities that can calculate position and orientation of a camera relative to physical markers or natural feature markers in real-time. The technology was seen as a revolution in AR, and is still used in Android and iOS devices under the guise of ARToolWorks.

    The next part of the story will be familiar to anyone who has been following the development of modern VR. A young part-time engineer from the Mixed Reality Lab (MxR) at the Institute for Creative Technologies (ICT), while studying at the University of Southern California, began work on his own HMD. Disappointed with the low-grade technology of VR in the 1990s and the resulting disappearance of the medium from the public consciousness, Palmer Luckey, founder of Oculus VR, had developed several prototypes before going public with the Rift. Attracting a great deal of attention from the videogames development community, including John Carmack of id Software and former CEO of Scaleform, Brendan Iribe, the Oculus Rift was about to become the leading light in consumer perception of the medium.

    With the aid of a number of videogame luminaries, a Kickstarter campaign was launched in 2012. Raising $2,437,429 USD – 974% of its original target – the Rift was transformed from its original goal of a DIY HMD package into a consumer product. In March 2014, it was announced that Facebook had purchased Oculus VR, LLC, for a total value of $2billion. This in turn gave Oculus VR the resources required to establish a team of high profile developers, internal hardware and software development studios that would create a much higher grade of VR experience than was imagined possible at the time of the Kickstarter launch.

    A Brief History of AR & VR: Merging Realities and Technologies

    While Oculus VR was gaining traction with the Rift, Google was pushing in a different direction. April 2013 saw the launch of the first prototype of Google Glass, an AR HMD that was able to overlay small amounts of smartphone-style information as well as communicate directly with local devices. Google Glass was far from becoming the groundbreaking AR device technophiles had been begging for, but it did reignite interest in the potential of the medium.

    The next few years would see numerous other HMDs clawing for their own space in the upcoming market for both AR and VR, including impressive entries from Google, Sony Interactive Entertainment (formerly Sony Computer Entertainment) and the HTC/Valve collaborative effort: the HTC Vive. In 2015 however, Microsoft announced their foray into HMD technology with the HoloLens.

    A Brief History of AR & VR: Merging Realities and Technologies

    Clearly demonstrating the distance between VR and AR as consumer technologies, the HoloLens began shipping to developers in the same month the Oculus Rift began arriving with end users. Microsoft’s HMD arguably acts as the first point at which the two technologies could converge. What once started as a display to deliver experiences that the human eye could not conceive on its own became segregated by the gap between the ability to deliver on consumer expectations; within the HoloLens, there is the opportunity to bring AR much closer to the quality of what is now consumer VR. It’s still years away from a retail launch, but if AR has now reached the point of Luckey’s Kickstarter campaign four years ago the progress soon to be made will be rapid and very public. Facebook has already expressed interest in a HMD capable of doing both AR and VR within a single device, and with the HoloLens providing a jumping-off point it’s not hard to believe that the next decade will look very different for modern computing both in industry and within the home.