A VR Simulation of an Augmented Reality Future

A Virtual Reality experience created for the CSUEB Multimedia Masters Program (2018)

Made in Unity for the Oculus Rift

In my first year of the Multimedia Master’s program at Cal State East Bay, we had to work individually on a project to craft and deliver a multimedia presentation that showcased our skillset as designers, developers, and story tellers. We also needed to display a certain amount of knowledge of the technology that we would be using in order to present this work.

I have always been a sucker for the portrayal of futuristic technologies in science fiction movies. One of my favorite sci-fi flicks, “Minority Report”, is one of these such portrayals because it displayed a harrowing vision of a dystopian surveillance state where facial recognition cameras and drones dominate your every move in the name of safety in order to prevent “future crime.” One scene from the movie depicts this same facial recognition technology to display Augmented Reality (AR) advertisements to the main character as he is walking through a mall:

Minority Report. Directed by Stephen Spielberg,
performance by Tom Cruise, 20th Century Fox, 2002

As you could imagine, being bombarded with basically floating ad commercials everywhere you go would not be a pleasant experience at all. When I first watched this scene, I didn’t know what AR was but I knew that I wanted to capture this same type of dystopian futuristic technology and be immersed in it. This inspired me to want to create this somehow and would serve as the basis for my individual project.

Because I had not really experienced much AR before this aside from Google Glass and AR mobile phone apps like “IKEA Place”, I had to decide how I was going to present the AR to those wanting to experience my project. When I started the masters program, my wife had surprised me with a new Oculus Rift Virtual Reality (VR) headset. I suddenly knew how I was going to portray this dysptopian AR future…in VR, as a simulation. I started researching the tools needed to develop for VR and ended up choosing Unity, a cross-platform game engine with many, many tutorials on YouTube.

I learned Unity in one weekend and started developing this dystopian AR future. The first asset that was utilized was 3D 360 degree video footage for the VR setting. I used an Insta360 Pro 8K Spherical VR camera to capture the video footage shot around Oakland and San Francisco. The second asset used were GIFs that would serve as the AR images floating in 3D space. I used Giphy.com to acquire most GIFs however I did create a few GIFs through a Photoshop batch script.

The third asset were 3D head models from the Unity asset store that I used as the narrators of the sci-fi story that I was going to be telling. I had to animate their mouths moving so it could sync up with the audio that had been generated through an online text-to-speech service from Oddcast. The final piece that brought everything together was the sci-fi story that was told. I didn’t want to go with a very serious dystopian future so I kept it light and wanted it to be a little funny.

It was set in the not-to-distant future where humans felt comfortable enough with Artificial Intelligence (A.I.) to allow it to suggest ways to enhance outside surroundings with AR objects. The A.I. called itself “A.R.I.E.L.” which stood for (Augmented Reality Intelligent Electronic Legend) and it would inform the humans that it is there to serve them and to make their lives a pleasant experience. As the years go by, the A.I. is creating more of these AR objects because it believes that they make the lives of its human counterparts easier. The A.I. starts to make itself sound like a religious belief and tells the humans to call its name and to believe in it. In the end, the A.I. has created so many AR objects to the point that they are everywhere and surround oneself.

Believe in me, and I will give you unlimited knowledge to help you through whatever situation that you might be in. I will always know where you are…

A.R.I.E.L – A VR Simulation of an Augmented Reality Future

In order to go through the different scenes in the VR simulation, I had to create some interactivity inside the VR space with the Oculus controllers. In order to do this, I used some Interaction scripts that were in C#. The script itself was pretty straightforward but needed a little modifying to work with the VR objects. In order to do this, I learned how to use VSCode and basic C# syntax in a weekend. The end result was simple object interaction in 3DoF (three degrees of freedom) which allowed for some fun 3D AR object throwing. The beginning of the script is below:

//INTERACTION SCRIPT IN C# FOR UNITY

using UnityEngine;
using System.Collections;
 
public class PlayerInteraction : MonoBehaviour {
    public GameObject mainCameraObject;
    private Camera cam;
    public Interactable focus = null;
    Transform player;
 
    void Start() {
        cam = mainCameraObject.GetComponent<Camera>();
    }
 
    void Update() {
        if (Input.GetMouseButtonDown(0)) {
            Ray ray = cam.ScreenPointToRay(Input.mousePosition);
            RaycastHit hit;
 
            if (Physics.Raycast(ray, out hit)) {
                Interactable interactable = hit.collider.GetComponent<Interactable>();

I finally had the VR interactive experience finished and I packaged it as a stand-alone game that could be played through the Oculus without Unity. I presented this at Cal State East Bay’s Art Gallery in the Spring of ’18 and it was a hit with some of my former Art professors! Since the Unity project folder in which I created this VR experience is too big to upload, please check out the video below of how the VR experience came out instead!

Let’s hope that this dystopian AR future stays a VR simulation in our timeline…