A Journey into VR, AR, and Spatial Computing
So there I was, standing in a prison cell, looking directly at a somewhat menacing bunkmate. I looked around the small room, the small window, the uninviting bunk bed. This is where I’d have to live for months, years. It felt inhumane, no matter what someone had done. I looked at the other person in the room. This was no ‘Orange is the New Black’ character in a prison set. There weren’t actors in the shot who, once someone yelled, “Cut!” would go back to their posh LA homes to have their personal chef whip up some sushi. This was real – well, it felt real. Some part of my brain told me it was real, but only a few minutes prior I was walking around a large dimly lit room filled with people waiting — some not so patiently — to put on a headset and escape reality.
I had arrived about 30 minutes into my allotted time at the Tribeca Film Festival’s Virtual Arcade. Partially due to my general habit of doing ‘one more thing’ before I go. Partially as I didn’t want to seem overeager to geek out on technology. This was a mistake. Even waiting to check in at the shiny white desk with some amazingly hip check-out women and men, I could sense I had chosen wrong. She seemed to say, “Really?” when I told her the time I signed up for. Like the reaction of a movie theater attendant when someone decides to buy a ticket 15 minutes after the film has begun.
When I took the elevator up to the 6th floor, I felt a little like I was in one of those fake Disney elevators where you really weren’t moving anywhere, but then the door opened and I walked into a hallway with ceilings that felt twice as high as they needed to be, which make the purple hue of the room feel a little cooler. Not just someone had swapped out the regular light bulbs to create some neat ambiance.
1. How to use subtle AR filters to survive your Zoom meetings?
2. The First No-Headset Virtual Monitor
As I was looking straight at a big white wall and a logo, it wasn’t clear immediately whether I turn right or left. Left? No, right? Right. I walked down the hallway like I knew where I was going and then almost missed the door to my left.
“Virtual Arcade? Immersive Cinema?” said an overeager intern in a black t-shirt. I shrugged an “I think so” and then went through the doors behind her. It was a rush of cinema-types, tech-geeks, much older retired and adventurous folks, with a few teenagers mixed in. It looked almost like a small carnival in a somewhat dark room. There were booths, each decorated differently, some by clear artisans that could impress any design fanatic (like myself) and others that felt a little stark and casually homemade. Dim lights of different colors emanated at each booth. People in black wearing Tribeca branded t-shirts helped to check in the next guest into each booth, where then they were outfitted with a black shiny large headset, adjusted around their head and sometimes given semi-circular matching black handsets. The attendant would then quickly look around the guest seemingly to check for any loose wires or bags on the floor. “Have fun,” “Enjoy,” or “Good luck” was said, and then they walked away to leave a somewhat anxious guest behind them reaching out towards the back wall, feet firmly planted on the ground until they tentatively took small steps forward like a baby toddling for the first time.
Surrounding me was a frenzy of people talking about the experiences they just had, using grand arm and hand gestures like someone who was sharing stories of their exotic travels. Other people were standing around with arms crossed waiting for their experience to begin.
There seemed to be endless small lines. “Where to start first?” I thought. I had, on a whim, decided to get the tickets to just try something new. I didn’t do the heavy research as I normally do before The New York Film Festival (mainly to avoid the way too heady indie films that seem to exist for the pleasures of film study professors and festival juries). I just wanted to ‘try VR’ and that was it. I was intimidated by Tribeca and its crowd, but I was determined to have a memorable experience.
“Do you have any room?” I ask several people standing outside of the booths, like bouncers outside of mini-clubs. Each had clearly been asked that question hundreds of times, although they did their best to be friendly. I was getting more and more frustrated as the time passed. I finally went over to what seemed like a very nice man who knew about the festival. If nothing else, I thought, I would make friends with someone new— as I like to do almost anywhere — but also, I needed to find someone to complain to about my experience thus far.
“Is it always like this?” I asked casually. “Yes,” he said. He seemed particularly nice and friendly, and I hadn’t spoken to anyone in hours — something that isn’t typical of me. So I proceeded to vent and vent. “Why can’t everyone experience everything in the room?” “Why does there have to be so many lines?” “Couldn’t the space be even bigger?” “Couldn’t you schedule longer visits?” I said impatiently. He had answers to almost every question I had. After about 5 minutes of talking, I said, “I’m sorry. I forgot to introduce myself, I’m Sam. I work in TV, and this is my first time trying VR.” He then introduced himself, “I’m Loren. I curate the VR festival.”
Yes, if you can believe it, I had just unloaded all of my frustrations on the person RUNNING THE FESTIVAL. Ha! I turned into a mess of apologies. He was very nice about it and politely introduced me to someone else working at the festival as a gesture to say everything was fine. He liked getting the feedback and agreed that adjustments needed to be made. He also suggested that, if I hadn’t seen anything yet, that I wait in line for the 360 videos at the series of spinning chairs at the other end of the room.
“Thank you, thank you,” I continued to apologize with my proverbial tail between my legs. I quickly sped quickly passed many people, hoping to hide some of my embarrassment among the crowd. I then got in line for the 360 videos, somewhat defeated by that experience. I began watching about 20 people with headsets on, spinning in unpredictable circles on chairs set in 5 rows. Everything was pretty quiet without a brief “wow” or “whoa” by the individual members of the group as they reached out into nothingness in no particular pattern. One after another, at unpredictable intervals, people slowly came out of their headsets, like skiers entering into a lodge from a snowy day on the mountain. They had a look of exhaustion that didn’t quite match the amount of exertion they spent spinning on the chair.
Eventually I was quietly ushered over and sat down by another attendant in a Tribeca shirt.
“Glasses or no glasses?” I whispered.
“Either way,” the busy woman responded.
“You ready?” she asked.
“Sure,” I said as she lowered the headset.
I took a deep breath as she helped me reorient myself in a digital world.
“It looks sorta like a TV screen, but it’s all around me,” I thought as I looked up, down, left, right, slowly spinning in all directions. I caught myself off guard when I felt my foot bang lightly on my backpack on the ground.
“It’s like I’m IN the TV,” I thought. I had always immersed myself in my TV shows, feeling like I knew the characters, the director, the writers. I obsessed over behind-the-scenes footage and stories in Entertainment Weekly. I loved TV shows and movies so much, that I chose to work in the entertainment industry as a career.
But now it just wasn’t my passion for good stories that lowered my suspension of disbelief. My brain was doing it without any effort at all. I was just THERE. IN the story.
I watched a documentary about Antarctica scientists that easily could have been an accompanying piece to a 10 part series on Discovery Channel or National Geographic. Then, I overheard someone nearby saying, “That was so powerful,” and I was intrigued. I asked the same host of the 360-video space what they might have been talking about.
“Probably ‘Step to the Line’,” she answered.
“Can I try that one?” I asked, wondering if I had to go back to stand in line and potentially run into Loren again and give him a somewhat embarrassed half-smile and wave.
“Sure,” she answered.
And then a few minutes later I was standing in the jail, in a prison cell. ‘Step to the Line’ was powerful. It made you get over your bias of prisoners quite effectively in just a few minutes. You learned about your bunkmate’s life through joining him and your fellow prisoners in an exercise with people from outside of the prison system. People were crying about their lives in what seemed like a foot or two away from you. What I felt was true empathy. Not the fake, manufactured one built in the minds of the writers of the TV series ‘This is Us,’ but I came to it myself, by almost literally putting my feet in the shoes of a prisoner in the prison system.
So when I finally finished watching that short experience and maybe one other that I have since forgotten. I took off my headset by myself, somewhat emotionally and physically exhausted by just about 20 minutes in the chair.
“What IS this?” I thought.
A video player strapped to my head had in 20 minutes changed my mind about prisoners and the prison system.
“Whoa,” I said.
After those few 360 videos, I knew I needed to learn more. I knew already that I needed to arrive much, much earlier to Tribeca the following year. Whether people thought I was geeky or not for doing so. I also knew I wanted to get into those booths, where likely there were much more powerful experiences.
So the next day, and the day after that, and after that, for the past 3 years, I have been learning about virtual reality and all the related technologies. I have tried countless experiences. I have written about the topic, including co-authoring a book and contributing to another. I have created a social group focused on marketing news around the industry that has over 3000 people in it. I have spoken and hosted industry events. I’ve judged at hackathons and festivals (although not Tribeca yet, but there’s always hope). I continue meeting everyone I can in the industry (and was recently unofficially named XR’s Networker-In-Chief) including some of the most some of the talented developers and creative people in the world. I’ve filled these past few years not just learning about the tech, but focusing on how to communicate its value within and outside of the emerging tech ecosystem.
And this fall, I am going to be teaching 50 NYU students all about XR and spatial computing (aka virtual reality and augmented reality and the related topics). It will be a particular challenge as it will all be remote as almost all the LBE (location based entertainment) centers are currently closed. I am inviting some of the most inspiring people I’ve met along my journey to talk during the class, hopefully including Loren Hammonds (who, luckily since my first day in a headset, has become someone I get really excited to chat with as we have found a camaraderie over our passion for great VR and AR).
I’m teaching in the Media Studies school, which is very different than the Engineering or Film Studies or Art programs that have begun to teach about these technologies. In my recent exposure to the school, I’ve found that media studies faculty and students often takes a highly critical eye, drawn immediately to the negative ethical implications of emerging tech. I, quite unsurprisingly, am enamored with XR. I hope to break through a little of their resistance to understand how jaw-dropping this technology can be and how it can be used for good as well. I also want them to understand, whether they realize it or not, that this tech is changing slowly the reality around us both at work and at play. (Don’t think so, well, that SnapChat filter is AR, that Zoom background is AR.) Most of all, I want everyone in the class to eventually let out at least one sigh of “Whoa” (just like the recent Oculus Quest ad here), just like I did at Tribeca 3 years ago.
