The MIT Press Essential Knowledge Series, Cambridge, MA, 2019
264 pp., illus. 9 b&w. Trade, $15.95
Gazing down at me mutely, perhaps needing dusting, is a one-foot shelf of virtual reality books from the era now called VR 1.0: Benedikt’s Cyberspace: First Steps, Jacobson’s Garage Virtual Reality, Anders’ Envisioning Cyberspace, and other worthies of the 1990s. There was much excitement at the nascent field’s potential, serious philosophical and technical issues mapped out, bluesky hand-waving inspired by creaky proof-of-concept systems. Pay no attention to the refrigerator-sized bank of computers beneath the conference tablecloths!
Yet in the past half-decade, the same progress that drove the mobile computers in our pockets has re-invigorated this arena, and I sought a book on it to assign to my contemporary Communication Media grad students. In this compact compendium of its “essential knowledge” Samuel Greengard examines virtual reality (VR) and augmented reality (AR) in the context of what he calls XR, or Extended Reality. Extended realities best supplement physical experiences not replace them. XR blends real-world physics with virtual or augmented worlds. An example might be a smartphone app that lets you see a virtual dog in a real room. Or an app that positions your real dog inside a virtual room.
The chapter “Why AR & VR Matter” gives a history, touching on optical illusions, 19th century stereograms, panoramas, dioramas, science fiction, my beloved home View Master of the 1950s, Morton Heilig’s Sensorama, and a 1957 patent of a head mounted display for telepresence. The virtual becomes more tangible in military-funded work of Tom Furness, by DARPA and Naval Research, MIT’s Wearable Media Lab, and the 1990s CAVE (Computer-Aided Visual Environment) at the University of Illinois Chicago campus. They built the cathedral that was VR 1.0. Yet early pioneer Jaron Lanier became dissatisfied with the public reception of VR, saying it had been sold as an illusion, though real for “what it is”.
Cut to the last few years. In 2014 Google Cardboard offered AR on a high-end Samsung smartphone with a snap-together inexpensive viewer, yet Google Glass ignited controversy, as bar owners banning “glassholes” whose augmented gaze made female patrons especially nervous. Oculus offered VR experiences, Microsoft with its Hololens, and SketchUp viewers appeared. The 2016 Pokémon Go treasure hunt thoroughly engrossed teenagers finding virtual pocket monsters inserted into their real world, while the company harvested and monetized all useful location, transactional, and social media data from their phones.
A contemporary phone app offers real time IKEA furniture placement.
Developers’ tools have been introduced for Apple’s ARKit and Google’s ARCore. Greengard predicts more convincing immersions, better miniaturizations, smoother body integrations as students internationally attend and graduate from Master of Art programs in XR design, then enter and influence the field.
XRs needn’t be individual experiences only, isolation in an opaque headmount or a lonely study of one’s phone. The Cave at Louisiana State University was created as a collective immersive experience to contrast with an individual in a headset or viewing real-time interactive 3D objects on a display on a desk. It was used for data visualizations, sometimes with active shutter glasses like in a 3D movie theater, an ancestor of the multisensory VOID in Las Vegas. There have also been some experiments with VR in law enforcement and the military, including the BRAVEMIND project to aid troops in overcoming combat-related PTSD. A student of mine was forced to drop out of his studies during the 2020 Coronavirus quarantine from the stress of online classes while protecting his child and pregnant wife; I wonder if an XR, wider-bandwidth university experience now would have been more reassuring.
Greengard unpacks the “technology behind the real”—OK, the illusion of it—and directions it needs to develop in order to reach the nearly flawless experience in XR that mission-critical applications will demand. Polygon count for rendering quickly and correctly under constant refresh with each movement through space needs to be complex and robust enough to feel real. We’re shown the process in creating product from concept model to actual model to AR experience, citing SLAM simultaneous motion and tracking software. San Francisco State University’s Kinesiology Department pioneered VR in exercise training. The addition of haptics brings greater responsiveness and precision to VR, and the HaptX AxonVR glove can pick up a grain of rice. Greengard maintains there needs to be development in this realm before VR can be used for surgical practice or flight simulation with the mission-critical accuracy necessary; the goal might be a full body suit to perform these.
A key consideration in any XR experience is the field of view (FOV), which really should be 180 degrees or more, not the 50 degrees of the past, or the 90 to 110 degrees prevalent today. Wearable AR technology makes use of liquid crystal displays (LCD), light emitting diode (LED) and, increasingly, organic light emitting diode (OLED) or liquid crystal on silicon (LCoS) eyewear. Will it be paired with head-mounted device (HMD) optical or wave-guide carriers? Employing polarized or high-resolution displays? These and other experimental optical methods are being developed and tested in various labs. Yet all are grateful HMDs are no longer tethered by anaconda-sized cables, with cathode ray tubes pressed to the eyes at the same time their deleterious ergonomic toll in the workplace was being realized, back in the glorious days of VR 1.0.
Is this essential knowledge of VR/AR/XR in 2020? This little book passes the test and is recommended to my students. In Samuel Greengard’s clear, concise and succinct Virtual Reality, the graduate in communications or technologies has a useful overview of the field. One that I hope she or he enters to innovate.