What Do We Mean by "Reality" in 2022

DOROTHY, THIS IS NOT KANSAS ANYMORE

The thinking behind this post started with a discussion about "Virtual Reality" (In Oculus / BigScreen) and widened almost out of control. There are so many issues it's hard to know where to start.

As far as I know, the term "Virtual Reality" was coined by Jared Lanier - a very smart guy who is still very much around. However, once it's out there, the word entered the public domain and tended to change meaning very rapidly, especially when commercial entities get ahold of it. There is now very little similarity between "VR" and what Lanier was thinking about. So, goodbye, for now, Jared.

I will surrender to the common usage, more or less co-opted by the makers of "VR" headsets, so, when I talk about "VR" I am regrettably talking about what you experience with a headset, such as Oculus (the one I use).

Sadly, this leaves me without a word for whatever is happening in Second Life - a "world" I have explored for over 10 years. In my view, Second Life offers huge advantages over "VR". In fact, the VR-Goggle headset approach has been tried in Second Life and it's awful. This is very hard to understand for people who are VR-heads from the start, enjoying the admittedly wonderful advantages of VR when it comes to games. In passing, it's worth noting that a "game" is not a "world". To me, at least, the truly interesting thing is how we experience the world, "virtual or not". I propose to show that this is changing so fast that our idea of "reality" is shifting beneath our feet.

WHAT IS "REALITY"?

What is ordinary "reality" in the first place? The dictionary definition of "reality", like the definition of "Virtual Reality," seems totally inadequate.

I don't want to wander off into philosophy and "ontology". I hope you will agree, at least for the moment, that there is something "out there" we call the "real world'. What's more, we are part of that world - something we can be reminded of easily by a heart attack or an X-ray of our bodies. That's real.

On the other hand, it's not too hard to arrive in the territory where very large segments of the population regard things as "real" that we regard as total fantasy. Trump's "big lie" and "Q-Anon" come to mind. Most deeply religious people experience some intrusion of spooky influences that they regard as real. For the moment this kind of "reality" can be excluded from my discussion. For me, at least, these experiences are neither available on-demand nor shareable. It turns out we don't lose much by simply putting such "alternate realities" to one side. 

I'd like to zero in on experiences that are shareable, verifiable, and repeatable. By doing so, I am admitting quite a few experiences that don't seem to fit squarely into the dictionary definition of "real".

A simple example is the tiny little world inside your smartphone that attempts to "mirror" the real world as much as possible - the GPS navigation "app". Technically, although we experience this "world" through a tiny smartphone window or experience of it is real enough to allow it to guide us through the "real" world.

GPS applications present such a stable, reliable view of the "real" world that it makes sense to think of them as part of the real world. Not just the hardware, but the "mirror world" shown by the application. Perhaps we will admit to some degree of "reality" - since what the GPS application shows us is, perhaps, less "real" than the view out the windshield. But it's close, as when the view out the windshield is obscured by weather or totally incomprehensible when we are lost and need the GPS application to tell us where we "really" are.

In fact, our smartphones are packed with tiny alternate universes. Some, like the phone application itself, are so familiar that we forget that we are talking to the phone "as if" a real person is with us. Some seem "sophisticated", as when we point the phone to a world to learn the name of a mountain or constellation in the sky.

I think all that kind of thing shades into what we are now calling "augmented reality". But here again (as with VR) the term slides into a product description rather than an experience:

his experience is seamlessly interwoven with the physical world such that it is perceived as an immersive aspect of the real environment.^[4] In this way, augmented reality alters one's ongoing perception of a real-world environment, whereas virtual reality completely replaces the user's real-world environment with a simulated one

To meet this description a product must be "immersive" and blend "seamlessly" with the "real-world" environment. This seems to be more like marketing boilerplate than a fundamental difference, but we need to acknowledge that "Augmented Reality" (AR), in general usage, means more than a GPS application on your dashboard. But for my purposes, it's a matter of degree and sophistication rather than some fundamentally different relationship between the "real" experience and the "augmented" experience.

The above definition brings up a useful term: "immersive", which actually means something, but here again, it tends to be used to describe hardware rather than experience.

You may guess that I am trying to distinguish experienced reality rather than the hardware that may or may not be used to produce that experience.

Which brings me back to fundamentals ...

THE BRAIN AND THE BASIC "IMMERSIVE" EXPERIENCE

The Visual Cortex

Our visual cortex does a lot to make our experience of the "real world" "immersive". We are surrounded by objects that seem familiar (doors, trees, dogs ...) simply because our brains have done an amazing job separating these shadows from the background and "labeling" them as something real.

Something similar happens with our auditory cortex ...

Auditory Cortex

This helps us identify the dog's bark, the wind in the trees and so many other things that turn out to be almost impossible for a modern computer program to identify. In "VR" the sense of immersion is remarkably enhanced by "realistic" and identifiable sound. The human voice is an important example,  Not shown above are the areas of the brain specifically dedicated to processing speech. Someone talking to you in "VR" is almost shockingly real, even though what you see may be little more than a cartoon.

What is often ignored is that the brain itself is our main arbitrator of what "seems" real. All the gadgets discussed in the following essays are built on that simple fact.

Eventually, I will get around to diving deep into how imaginary worlds that trick our brains using computer technology. For now, I give a tiny overview of how they all work. But the secret to how they all work is between our ears. That's where it all begins.

I will start my tour with "imaginary worlds", created by nothing more than a voice around a campfire.