You may have seen the words “Mixed Reality” used in relation to Augmented and Virtual Reality — the term has been popping up recently in articles posted on Forbes and Entrepreneur, among others. Google around enough and you’ll soon realize that there’s some difference of opinion on what Mixed Reality even is. Is the term largely interchangeable with Augmented Reality? Or is it some other technology, similar to but distinctly different from AR and VR?

For an explanation, I went to Marxent Software Engineer Ken Moser, who received his PhD in Computer Science at Mississippi State University. Ken’s main area of study was Augmented Reality, and he spent two years teaching at the university level, and two summers, 2014 and 2015, conducting projects at the Nara Institute of Science and Technology in Nara Japan, under the supervision of Christian Sandor, a leading researcher in the field of Augmented Reality.


Q: Ok, Ken, what is Mixed Reality?

Ken Moser: At the highest level, there are actually 3 types of “Reality” — Virtual, Actual, and Mixed. Mixed Reality (MR) refers to any combination of Virtual and Actual reality, and that includes what’s known as Augmented Reality (AR). In the Markerless AR Q&A we did in February, I briefly alluded to a taxonomy published by Paul Milgrim et al. on what he/they denoted as the Virtuality Continuum. It’s worth reviewing when talking about the different realities.

It’s hard sometimes to wrap my brain around new concepts.

“Newness” is a subjective measure for any technology — or any thing, really. Augmented and Virtual Reality are not, in fact, new concepts. The theories and basis for them have been developing over the last 60 to 70 years. Kids these days (and I’m sure this is true for every generation) think that they are the ones responsible for the latest tech and therefore are free to fling around nomenclature all willy nilly. The reality is that far greater minds have spent the last half-century or more developing the things we’re using today and there are already well-established terms for all of the “breakthroughs” we’re seeing now.

You already alluded to this, but please elaborate: What’s the difference
between MR and AR?

Because MR and AR are still fresh concepts for most people, and because information and misinformation can be so quickly disseminated to the masses, it’s not surprising that confusion has arisen around these concepts. The answer, though, is actually as simple as the answer to this question: “What is the difference between a Honda and an Accord?“ On the one hand, there is no difference — Honda and Accord can be used interchangeably to describe a specific car. On the other hand, Honda is a broader term that can be used to describe a whole host of ofter car models — Civic, CRV, etc. This is paralleled by MR and AR. Mixed Reality is used to describe the general technique of mixing sensory information from a Virtual and Real environment, while Augmented Reality refers to a specific type of sensory mixing.

Can you explain that last bit about AR and sensory mixing?

I’ll give a brief synopsis on where AR falls in MR, but in fact, this distinction is actually due to the limitations of Virtual Reality. If you interviewed a random dude or dudette on the street and asked them to give a definition of VR, they’d probably say something like, “It’s goggles you put on to see inside a video game.” They wouldn’t be wrong, of course. The experience they’re describing does indeed accurately depict one modality of VR: A visually immersive “reality simulation.” And that is, in the barest sense, what VR is: a simulation of reality.

I sense a “but” coming.

But … current VR is a purely synthetic reality, since it doesn’t have to fully simulate the way we experience the real world. As such, VR can be used to simulate any reality, including realities in which the physical laws we are accustomed to no longer apply. We are still only capable of experiencing these realities through our natural five senses, meaning that all stimuli we receive from a VE (Virtual Environment) must be turned into an equivalent stimuli in this reality that when applied to our beings attempts to induce the sensation of the parallel stimuli from the VE.

This seems confusing to me.

That’s because it is! More succinctly stated, if one was to experience true Virtual Reality, none of the physical sensations they would perceive as being real would actually be real. For example, if they were to pick up a red ball off of the floor and hold it in there hands, the proprioceptive sensation of moving their bodies down to the ball and extending their grasp, as well as the visual imagery of the ball and its surroundings, and also the stereoscopic input used to measure the location of the ball relative to their hand, all of that information would have to be synthetically generated and fed into the user’s brain. For lack of a better analogy, and because we’re always using references to the Construct and Morpheus here at Marxent, a true Virtual Reality would be the equivalent of what is shown in The Matrix. “Do you believe that my being stronger or faster has anything to do with my muscles in this place? You think that’s air you’re breathing now?”

So, what does any of that have to do with MR and AR?

Well, the current sophistication of our modern technology does not allow us to create anything even remotely close to what a Matrix experience would be like. We are forced to experience VR in our actual reality. Meaning, if we want to make someone see something in a VE, we have to make an actual image in our real world and put that image in front of the person’s eyes and they have to see it and process that image as they would anything in the real world. Same for sound and touch. Because of the limitations of VR tech, the user will always experience some mixing of the real world and the virtual through their senses. And that is the essence of Mixed Reality.