Markerless Augmented Reality: The basics
Markerless Augmented Reality (Markerless AR) is a term used to denote an Augmented Reality application that does not need prior knowledge of a user’s environment to overlay 3D content into a scene and hold it to a fixed point in space. Until a few years ago, most AR fell under the category of “marker-based AR,” which required the user to place a “tracker” — an image encoded with information that was translated by complex software to produce a 3D object that maintained orientation within a scene — in order to achieve the desired effect. Markerless AR solutions started as hardware packages like Google Tango, but have evolved into software-based SDKs that produce the same effect without the need for specialized equipment.
Markerless AR is everywhere
With the recent emergence of advanced camera systems and more precise sensors in mainstream devices like Apple’s iPhone, Augmented Reality has completed the transition from image- or QR code-based activations to markerless Augmented Reality experiences. Originally referred to as “dead reckoning,” Markerless AR uses a combination of camera systems, dedicated sensors, and complex math to accurately detect and map the real-world environment — such as the locations of walls and points of intersection. With a map of the area, an AR-enabled application makes it possible to place virtual objects into a real context and have them stay in place without the need for an QR code or image.
AR SDKs: Apple’s ARKit and Google’s ARCore
Unveiled in June of 2017, Apple’s ARKit SDK brought core AR functionality (plane tracking, anchoring, etc.) to the iOS operating system. Over night, 100 million people had an AR-capable device in their pocket. Google followed suit in early 2018 with ARCore, essentially replicating the functionality of Apple’s SDK. While ARKit and ARCore started life with feature parity, Google’s ARCore has suffered from the device fragmentation that hampers the Android platform in general.
Marxent has also developed a proprietary AR tracking SDK called MxT Tracking. MxT Tracking offers “instant initialization” when compared to ARKit and ARCore — both of which are slow to establish a tracking plane and render a 3D object into the scene. A second advantage to Marxent’s MxT Tracking is that it can be paired with ARKit or ARCore, allowing developers to leverage the best parts of each SDK and deliver the best AR experience to consumers.
Current uses of Markerless Augmented Reality:
Markerless Augmented Reality has seen its biggest impact in gaming, with the AR-enabled Pokémon Go becoming a smash hit back in 2016. There’s been a steady stream of AR titles since, though none has made the impact Pikachu and company did. (The imminent release of Harry Potter: Wizards Unite may change that.) These games are still just scratching the surface of what AR can do, and ARKit/ARCore make it possible to create multi-user real world games with 3D content that remains fixed in place for users to find and interact with. We’re still at the beginning of realizing how AR technology will change the gaming world.
2) Retail product visualization
Have you ever wanted to see how that new appliance might look in your kitchen? Or wondered whether you had room for said appliance? Markerless AR makes it possible to use a mobile device to scan a real world environment such as a kitchen, and virtually place a product there to see how it would look and fit. Shoppers can “furnish” an entire room virtually, trading out styles of cabinets, flooring and appliances to see how they work before buying. Macy’s, Amazon, IKea and Ashley Furniture are just a few of the Fortune 100 companies giving consumers the freedom to shop in their own space, a trend that is accelerating with each new generation of hardware and update to the ARKit/ARCore SDKs.
AR Advertising started life as tracker-based print advertisements, where the user would hold their device over the ad and have a 3D character of some kind emerge. Markerless AR has freed AR marketing from its print prison, putting it into the real world. Companies already using AR advertising include WayFair, the NBA, Sephora and Gucci, with the experience providing a differentiating factor that these big brands can’t live without. The possibilities with AR advertising will become even greater when wearables like the Hololens become more common. Users could freely cycle between multiple virtual layers and experiences in their real world environment.
Markerless Augmented Reality’s ability to overlay digital information on the real world is a boon to educators, who are using the tools to better illustrate complex concepts for students. Apps like Anatomy 4D (for students learning about the human body) or Elements 4D (for those studying chemistry) are part of this early wave. Museums have also started using AR to bolster their physical exhibits with digital content that provide context or additional information for the pieces. In the corporate world, companies and the US Military are using Microsoft’s Hololens to create training exercises and experiences on everything from repairing airplane engines to surviving in combat.
Updated April 2019.