How do you make Virtual Reality? Working with Unity 3D

Tron, Total Recall, The Matrix – all these sci-fi classics have one thing in common. They imagine a world where Virtual Reality is the norm. The concept of VR has captivated the human race as far back as the invention of the kaleidoscope. Now for the first time in history, the dream of achieving seamless VR is a reality. How hard is it to make Virtual Reality experiences? Creating a VR experience requires more than just a hacker punching a bunch of green-colored code into a black monitor. Truly immersive Virtual Reality takes close collaboration between both artists and developers.

From design to Virtual Reality development

I started my career as a designer and 3D artist, but when modeling 3D characters and working on indie game projects introduced me to VR development, and I knew that I wanted to be a part of this new medium. I ordered an Oculus Rift when it was first announced on Kickstarter, even though I couldn’t create experiences for it yet. My interest in Virtual Reality made me more and more interested in learning code. It wasn’t just for the excitement of a new technology; I felt that bringing my knowledge and experience as an artist into the world of Virtual Reality could be a gateway to creating a whole new kind of user experience. I wanted to help turn VR development into an art.

As part of the Marxent team, I had the opportunity to achieve that goal of learning how to make Virtual Reality as a developer. With support from other VR developers here that have made the same jump to becoming Unity 3D developers, I started learning Unity 3D. If you want to get into Virtual Reality Development, Unity 3D is the first tool you should learn. I focused on building my new Unity 3D Virtual Reality knowledge and boosting my C++ experience, and I was soon developing full-time, creating a game in Unity in less than a month. Working in Unity, I was marrying both art and development; using UI design and programming to create all new UX effects that allowed our team to better communicate an idea to the end user. My experience as a 3D artist gives me a different perspective on Virtual Reality development than VR developers that have come from a computer science background. I’m able to incorporate visual details that might seem small but can make a huge difference in the way a user perceives their environment.

How to make a compelling Virtual Reality experience

Making a seamless and believable Virtual Reality experience is no easy feat; even the rules of user experience are different for VR. This technology has unique qualities that must be taken into account to a realistic experience. The way Virtual Reality works is by essentially tricking the viewer’s mind into thinking that what they’re seeing is a real 3D space in front of them. If the illusion is not realistic enough, the environment can fall into the “uncanny valley,” ruining believability. Because Virtual Reality is immersive (feeling like you’re inside the environment) vs. observational (screen at a distance), traditional rules that designers have followed for years when creating content for traditional screens are different for VR. Issues like peripheral vision, proprioception, and other display changes are important to take into account for VR.


Proprioception is the sense of one’s relative position in a 3D space. When designing assets for Virtual Reality, the scale of 3D models should feel consistent with the scale people see in real life to make the environment feel natural. Adding a reference to a user’s hands or body with motion detection further gives viewers a sense of immersion and control in the experience. Devices like the Microsoft HoloLens go a step further, providing an immersive Augmented Reality experience that still allows the user to see their arms in front of them.

Peripheral vision

Developers can use the way peripheral vision works as an advantage for both enhancing realism and a performance by using foveated rendering. This technique uses eye tracking to render the most detail in the center of the line of sight, while filling the peripheral area with lower resolution imagery and lower-poly models. Creating this takes planning in the design phase, when a 3D artist creates both high and low resolution graphics.

User interface

Information in the Virtual Reality environment can be designed to function in several ways. Does it scale and rotate with the 3D objects? Is it part of the screen or part of the scenery? Information can be incorporated with Diegetic (in-scene, scaling with objects) UI hotspots plus more detailed Non-Diegetic (overlaid, separate) windows of text or flat graphics. It’s important to these in a combination that feels natural to the user.

Bridging art and science with Virtual Reality

Virtual Reality is not just a fancy new way to watch a movie or play a video game; it’s an entirely new medium with the potential for artists and developers alike to express entirely unique experiences. Developers that have an understanding of art and design are better able to use Virtual Reality technology to its full potential because VR depends so heavily on the fundamental natural tool artists use: their eyes.

We finally have the technology and the platforms to allow developers to take Virtual Reality to the next level; beyond just cool experiments and prototypes. We just have to start thinking creatively.