I’d like you to imagine it’s the year 2019. You are wearing a set of extremely lightweight wraparound lenses and have just gotten off the train in an unfamiliar part of the city to meet your friend at a new club. As you look around, you see a floating icon over a board against the wall. You point at it, and before your eyes, a transparent map of the city opens, floating in midair before you. You tell it your destination, and the map zooms into where you are, highlights a path to where you want to go, and then zooms in even more as it tilts and merges with the scenery around you, the path now an illuminated line on the floor.

You follow the path out to the stairs, and up to the street, but as you come to street level, a warning pops up advising you that it’s started to rain and that its previous path will result in you getting drenched, so would you like to reroute along a longer path that will keep you dry? You nod in acknowledgement and follow the line into a shopping district across the street. As you enter the mall, a small sign pops up in front of you asking if you’d like to see a list of current sales. You shake your head, and the sign vanishes. You continue to follow the line through the mall, looking around at the various people standing in front of store windows, making waving motions as they browse through the inventories. For you, all you see is a blank screen with the store’s logo and a button saying “touch here for catalog”.

You do happen to notice a logo for a store you frequent, and pause to hit the catalog button. The window clears and an attractive lady with horns and a spaded tail appears. “Hiya, and Welcome to the Succubus’s Den! We’re having a special today on horns and halos, all models are 50% off. Would you like to browse our selection?”

“Sure” you say, as a three way mirror pops up in front of you showing your present appearance. You frown as you take in your business attire, and decide it’s just way to boring for a night at a club. A request sign pops up asking if you’d like to deactivate “professional mode” and you think “yes” at it.

Suddenly the mall around you transforms from a rather dull set of storefronts to a sylvan glade, with elves, and centaurs, even a couple of fairies mixed in with trolls, Klingons, and what appears to be a storm trooper shopping behind you. Your suit and tie have also vanished and you look at the brawny barbarian warrior you chose to wear when you were playing an MMO last night. As you think about changing it, a menu pops up and you decide to go with your goth avatar, the image in the mirror changing. The sales clerk smiles. “Ohhh, I have a nice pair of wings and a black light halo that would match that avatar so well!”

You tell her to let you see it, and in an instant, you are admiring the smoky black wings and the shimmering purple halo. You nod in approval, and tell the clerk you’ll take them. A small icon shows the price and you think “yes” at it. A note comes up showing that “Goth Angel” has been added to your inventory. You thank the clerk and start following your guideline again.  As you walk down the mall, you note a couple of vampire girls licking their fangs as you pass. At the far end of the mall a shimmering portal opens onto the city street, where you can see the rain is still falling. Your guideline leads under an awning along the sidewalk, and down the street. You can see a glowing arrow pointing down at the club you are heading for. You head down the sidewalk, then stop when a warning sign pops up pointing at an alley entrance just ahead of you, and you wait as the delivery truck pulls out onto the road.

As you enter the club, you look around and note that there’s a nice mix of reals and virtuals, only a small icon over the heads of those visiting entirely in VR enabling you to tell who’s physically there, and who isn’t. A flashing icon calls your attention to where your friend is waiting, and as you head towards him, a small fairy flutters up and asks you what you’d like from the bar. You order, and by the time you get to the table, the waitress, who the fairy was a small sized copy of, has your drink waiting.  You smile as you anticipate a nice evening and settle down to have some fun.

That’s just a taste of a world in which VR and reality are intermixed, and I’m sure it’s pretty simplistic compared to what we will actually experience, but nonetheless, it’s sufficient to make this article’s real point, which is  actually not the uses of VR. Instead, I’m hoping I can get you thinking about what’s going to make this sort of VR possible, and the implications of that technology.

So let’s start with our map, shall we? How, exactly, did we call it up in front of us?  It should be obvious that we’re wearing a pair of video lenses capable of overlaying graphics on our view of the world, but how did the map know we wanted to access it? How did the sign realize we clicked on it from who knows how far away?

The short answer is that there’s communication between our glasses and the sign, but the reality is that it’s not quite that simple. In order to position a button on the map, our glasses had to know where in our field of vision the sign was, which means our glasses had to be aware of the environment around us. It had to be aware of the 3 dimensional space surrounding us; be aware of the physical objects in that environment, and on top of everything else, know that the map was a map. It could do this many ways — by scanning our environment with a lidar, or THz wave scanner; it could link to a local system which has a 3d map of the station already created; it could communicate with a set of lidars or other scanners in the environment, and their are quite a few other methods it could employ. The common factor in all of them is still the same. You are being watched continuously by an untold number of sensors and cameras.

Got it? Every person in that station has a device just like yours, watching you and your every action, recording every twitch of every muscle. For that map to provide the “guideline,” it has to know to a millimeter where you are. Same for the “weather warning.” The “mall” knew when you entered. The store knew when you were standing in front of it, and who you were, what your avatar looked like, and how to access your payment info. To allow others to see your avatar, they had to be enabled to know what that avatar was, and overlay it over your physical position, again, requiring sensors able to map you to millimeter precision. And what’s more, to enable such things as the warning about the truck, your lenses had to know more about the environment than you did. It had to “see around corners” by connecting to sensors in the alleyway. In both the mall and club, it had to access not only the “real” environment, it had to know the “virtual” one as well, and be able to distinguish which one you desired to see at any given moment, as well as create the appearance of virtual objects overlaying the real world. In other words, our environment was “self aware.”

Now, think about that for a second — think about how many cameras and sensors it’s going to take to make our environments “aware” of itself and us, so it can enable such VR interactivity.

Then think about being able to walk onto an airplane without having to pass security, because no one with explosives would be able to get within ten miles of the airport. Think about being able to walk down the darkest alleyway in NYC in perfect safety, because there are no more muggers, because everyone knows that it’s impossible to escape arrest it you try. Imagine your car speeding down the road at 200 mph while you are surfing the web without the slightest fear of being arrested for speeding or crashing because you’re distracted, because your car knows where every other car is on the road, and is driving itself. Imagine working in space, while living in Iowa, telecommuting to a remote teleprescence unit building a new and much larger space station. Imagine a classroom filled with students from nations all around the world, learning about ancient Rome by visiting it. Think about a million other uses for VR that we will demand, and the endless other potentials made possible by a self aware environment.

Think about it, and maybe you’ll understand why I laugh at those who continue to believe that we will never become a “Transparent Society.”