What are the Best Augmented Reality SDKs?
Augmented Reality SDKs are not a one-size-fits-all proposition. The only way to answer which is the “best” is to examine the most important factors related to a given use-case. Some developers shop for AR SDKs based on speed, price, or platform; others look at more nuanced details, such as tracking and SLAM capabilities.
There are open source, free or free trial options for developers just dipping a toe into the water. There are also major platforms — such as Apple iOS and Google Android — that offer free AR SDKs to developers. Don’t let that lack of a subscription fee fool you: AR SDK solutions are challenging to develop, design and maintain. If you’re looking for speed, stability and scale, it’s likely that you’ll have to pay for your AR SDK eventually.
Augmented Reality SDKs and Tracking
Most Augmented Reality SDKs share a group of similar features, but there are key differences. Each SDK has a specialty. For some SDKs, it’s their tracking ability, for others it’s recognizing objects. One key feature of all AR SDKs is their tracking system. Augmented Reality tracking can be accomplished in multiple ways, with each solution bringing its own pluses and minuses to the experience.
Tracking Specialties
There are four main tracking specialties — plane tracking, SLAM/Markerless tracking, face tracking, and object recognition.
Plane tracking
Plane tracking, in the technical sense, is the localization of one (or both) of the translational offset (X, Y, Z) and the rotational orientation (roll, pitch, yaw) of an object, with respect to an origin point. In terms of Augmented Reality, the origin point is usually a mobile device with embedded camera. All Augmented Reality SDKs include some kind of plane tracking, though individual results vary between solutions.
SLAM aka markerless tracking
SLAM, aka markerless tracking, originally required special hardware (such as an infrared camera systems, tracking lasers, etc.) like that found on Google’s Tango platform. Today, the technologies and techniques for performing AR tracking do most of the heavy lifting via software. Apple ARKIT and Google ARCore are the two most popular Augmented Reality SDKs that feature “markerless tracking.” This means the software uses a standard camera system to survey a scene and applies complex math that allows the SDK to place a 3D object into the scene and hold it in a point in space no matter how much the camera moves.
Face tracking
Face tracking technology involves identifying and verifying a human face from a digital image or a live video frame. A common example of face tracking is Snapchat’s AR lenses, which use an algorithm to accurate track a users face and add digital masks or makeup that track with the person’s facial movement. Beauty apps that use AR to let the user test eye shadow or lipstick — Sephora Virtual Artist, for example — are other good examples. Game creators use face tracking to allow players to become avatars. Face tracking is also important for security technology. Augmented Reality SDKs that have face tracking include: Vuforia, DeepAR and Aurasma.
Object recognition
Still in its infancy, Object recognition promises AR systems that can identify objects in the real world, allowing developers to then overlay digital information about the object into the field of view. Object recognition has applications in education, health care, manufacturing, construction and design.
Augmented Reality SDKs Reviewed
There are dozens of Augmented Reality SDKs, but that doesn’t mean they all solve every problem. To help you find the one that is right for you, we have put together an overview of the best, most mature AR SDKs currently on the market. They are listed in alphabetical order.
Amazon Sumerian
Amazon Sumerian, which debuted in 2017, claims it allows developers to create and run VR, AR, and 3D applications without needing any expertise in programming or 3D graphics. The SDK is a part of Amazon Web Services (AWS), and works with Android and iOS mobile devices as well as Oculus Go, Oculus Rift, HTC Vive, HTC Vive Pro, Google Daydream, and Lenovo Mirage. Sumerian includes a free tier. Pricing for other services depends on usage rather than a subscription.
Hosts are the key feature of Amazon Sumerian, which guide users through the AR experience. The same AI technology that powers Amazon Alexa is present in Sumerian. Users can ask the host questions and the hosts can react based on programming from developers.
Key Details:
• Built on open web standards and is based on WebGL JavaScript API
• Supports ARKit and ARCore
• Supports open standards and frameworks like WebGL, WebAR, WebVR, WebXR, and Azure
Links:
https://aws.amazon.com/sumerian/pricing/
https://aws.amazon.com/sumerian/
ARCore
Google launched the free, open source SDK ARCore in March 2018 for both the Android and iOS platforms. ARCore uses existing features of the phone and is free and open source. Key features include motion tracking, environmental understanding, light estimation, user interaction, oriented points, anchors and trackables, augmented images, and sharing.
Key Details:
• Development Plugins: Unity for Android, Android NDK, Unity for iOS, Unreal
• Version 1.6 released December 2018
• Update included upgraded Sceneform which optimizes ambient light and allows apps to feature more natural scenes
• Works for Android 7 or later
ARKit
First released in June 2017, ARKit is designed exclusively for iOS devices like the iPhone and iPad. Using ARKit is free for developers.
The key features of ARKit are space recognition using SLAM, objection detection, light estimation, and a continuing AR experience through several sessions.
Key Details:
• Runs on older model phones, starting with iPhone 6 and iPad Pro devices
• Apple announced ARKit 2 in June 2018
• Latest version includes multiplayer support which allows sharing the AR experience with multiple users on multiple Apple devices
Links:
https://developer.apple.com/videos/play/wwdc2018/602/
https://developer.apple.com/arkit/
ARToolKit and artoolkitX
ARToolKit is a free and open-source SDK and was one of the first Augmented Reality SDKs for mobile devices. It was originally released in 1999 by the University of Washington’s HIT lab. ARToolworks continued releasing it as open source until 2015, when the company was acquired by Daqri.
In the fall of 2017, two former executives at ARToolworks created artoolkitX so they could develop, support, and maintain it. It is still open-source and multi-platform.
ARTookKit was up to version 5.4 in March 2016 when support for it stopped, but it is still available and artoolkitX is up to version 1.0.5.1.
Key features include camera position orientation and tracking, tracking of planar images, camera calibration, and square marker tracking.
Key Details:
• Works for Linux, Windows, OS X, iOS, Android
• Plugins for Unity, OpenSceneGraph, some smart glasses
• Currently maintained as an open-source project and is available on GitHub
DeepAR
The main purpose of DeepAR is to create face lenses and manipulate them. This technology has been used for advertising, and for try-on apps in the beauty category. DeepAR can detect facial features in real time and runs on Android, iOS, HTML5, and Unity. Pricing is available by contacting the company. Clients include Absolut, Armani, Burger King, Krispy Kreme, Ray-Ban, and Sephora.
Key features include: face recognition and the ability to detect facial features, face tracking to follow the movements of the face in real time, and emotion detection.
Key Details:
• Can remove video backgrounds in real time
• Detects up to 68 facial features at 60 frames per second
• Can detect multiple faces in real time
• Works with existing software like Maya and Blender
EasyAR
EasyAR is available in both free and pro versions and works on Android, iOS, UWP, Windows, Mac, and Unity Editor. Each version has different features.
Created by VisionStar Information Technology out of Shanghai, the basic version of EasyAR includes planar image tracking for 2D images, smooth loading and recognition for more than 1000 local targets, video playback based on NW codes, recognition of QR codes, and multi-target tracking. Additional features in the pro version 3D object tracking, SLAM, screen recording, and cloud recognition.
Version 2.3.0 Basic came out in October 2018 and is free for commercial use without watermarks. The pro version is in version 2.2.1. KFC, Pepsi, Vogue, and other apps use EasyAR.
Key Details:
• Free version only supports up to 100 uses per day
• Development is available in different languages including C API, C++11 API, traditional C++ API, Java API for Android, Swift API for iOS, Objective-C API for iOS, Unity3D API
• Video playback using H.264 Hardware Decoding
Lumin SDK (for Magic Leap)
Florida-based Magic Leap developed the Lumin SDK primarily for its Magic Leap One ($2,000), a lightweight mixed-reality headset that mixes 3D digital imagery and physical worlds. Lumin, a derivative of Linux and the Android Open Source Project (ASOP), is Magic Leap One’s operating system. Key features of Lumin SDK include the ability to track eye movements, head pose, images, and hand gestures.
Key Details:
• Version 0.20 supports Unity 2019 which provides performance and editor workflow improvements
• There is a cost for the Magic Leap Professional Developer Package
• Supports animations with Lumin Runtime and Lumin Animation Studio
MAXST AR SDK
Developed in South Korea, MAXST is an Augmented Reality SDK that supports Android, iOS, Windows, Mac OS, and Unity as well as many HUDs and smart eyewear products like the Epson MOVERIO BT-300, 350, and ODG R-7.
Key features include a 2D image tracker that can track of up to three flat images simultaneously, instant tracking, visual SLAM, 3D object tracker, and cloud recognition.
A free version is available for non-commercial use with a watermark paid versions depend on the number of targets involved.
Key Details:
• Version 4.1 allows the creation of a 3D map using a bounding box and pin
• Uses native smartphone phone camera
• QR code tracker and barcode reader
• Optimized for Smart Glasses
• Version 3.0 came out in 2017
Marxent AR SDK with MxT Tracking
The Marxent AR SDK with MxT Tracking provides a near instantaneous markerless tracking experience for placing objects on the floor without the need for an initial scanning of ground anchors. The user points their device and tracking starts — as simple and frustration-free as it gets.
Best suited for short, fast AR experiences, such as visualizing home furnishings in context, the Marxent Augmented Reality SDK works well for retailers who want to provide shoppers with fast, realistic Augmented Reality visualizations of products in context. Since it requires no action from the user to start placing objects on the ground, the experience affords for rapid viewing of augmentations.
Key Details:
• The Marxent AR SDK is subscription based. Contact Marxent for pricing
• MxT uses a relative tracking approach, meaning that the scale of the tracking space is based on an estimation about how the user is standing or sitting with the application
• MxT tracking pairs with Apple ARKit or Google ARCore for a fast, stable AR experience
• MxT allows for the placement and interaction of multiple objects within a scene
Visual Studio
Visual Studio is an integrated development environment for Android, iOS, Windows, web, and cloud. It can also be used with the HoloLens, Microsoft’s untethered mixed reality headset. Visual Studio is available either by monthly or annual subscription. The subscription includes access to developer tools, cloud services, support, training, and support.
Other Features:
• Popular developer platforms like Unity, Unreal, and Vuforia also work with HoloLens 2Pricing | Visual Studio
Vuforia
Vuforia claims to be one of the most popular AR development platforms with more than 600,000 registered developers. It supports Android, iOS, UWP, and Unity, and also works with Microsoft HoloLens 2. The technology allows developers to orient and track objects in relation to real-world images. Vuforia has robust tracking capabilities and allows developers to orient and place virtual objects into the real world.
The first prototype came out in March 2015 and the company PTC purchased Vuforia from Qualcomm in November 2015. Version 8.1 came out in March 2019 and another update is expected in May 2019. The biggest update was the option to train model targets for instant and automatic object recognition. Developers can use multiple models in one scene and switch between each experience.
There are free trials with watermarks and paid subscription plans. Vuforia has several products, Engine, Expert Capture, Engine, and Chalk.
Key Details:
• Markerless object recognition
• Flat surface recognition using Vuforia Ground Plane, triggers AR content by recognizing images
• APIs for Java, C++, Objective C++, and .NET via an extension of the Unity game engine
Wikitude
Wikitude AR SDK is for smartphones, tablets, and smart glasses. It supports Windows, Android, and iOS devices as well as Epson Moverio and Vuzix smart glasses.
The first version of Wikitude was released back in 2008 and has been growing quickly ever since. The latest version’s key features include object and scene recognition using SLAM, instant tracking allowing placement of a virtual object in real space, the ability to identify about 1,000 images and more than one of them at the same time, cloud recognition, ad 3D model rendering.
Wikitude is free for a trial version with a watermark for non-commercial use. Other versions are available for a fee.
Key Details:
• Development Plugins: Unity, Cordova, Xamarin, Titanium, React Native, Ionic, Adobe Air, Qt by V-Pay
• Works with ARKit and ARCore to use native features on the device
• Ability to import and render 3D models developers created in other tools
• Version 8 came out in July 2018 with the most recent update being version 8.3 which came out in February 2019
AR SDK Glossary of Terms
Application Programming Interfaces – Abbreviated API, Application Programming Interfaces are tools that allow developers to create applications for a given operating system.
Environmental Understanding – Used to detect the size and location of all kinds of surfaces, horizontal, vertical, or angled.
Head Mounted Display – Abbreviated HMD, and also known as a Heads Up Display (HUD), the Head Mounted Display is a device worn on the head or face that puts a screen in front of the wearer’s eyes that displays 3D Content. All VR and AR “glasses” are HMDs.
Integrated Development Environment – Abbreviated IDE, a core component of an SDK. The IDE contains a debugger tool, a console for writing code, and a visual editor for the UI.
Light Estimation – Looks at real-life lighting conditions.
Motion Tracking – Tracks the phone’s position relative to the real-world surroundings.
Object Recognition – The ability of an AR system to identify objects in a scene, allowing the application to then overlay information about the object into the field of view.
Plane Tracking – The localization of one (or both) of the translational offset (X, Y, Z) and the Rotational orientation (Roll, Pitch, Yaw) of an object, with respect to some origin point. In terms of AR, the object in question is taken to be a camera.
Simultaneous Localization and Mapping – Better known by the abbreviation SLAM, this technology understands the physical world by locking onto a series of feature points.
Software Development Kit – Abbreviated SDK, and also known as a devkit. SDKs are a set of development tools that allow the developer to create apps.
Image Tracker – A physical image (usually a piece of paper, but potentially anything an image can be attached to) that’s “read” by an AR system and tells it where and how to display 3D content in a real world scene.
User Experience – Abbreviated UX, the experience of using a specific piece of technology or software.
User Interface – Abbreviated UI, a User Interface is the way in which a human and computer system interact. This includes both electronic devices and the software they run.
Next Step: Picking the right AR SDK
There are many AR SDKs on the market, each with their own strengths and weaknesses. While it’s tempting to compare and rank the solutions, it’s also a fools errand. Which AR SDK is the “best” is dependent on what you want to use it for. When picking an AR SDK for your app, it’s important to assemble your criteria ahead of time. What’s are the most important features? Do you have a preferred platform? What is the use case for your AR app? Is a simple markerless tracking system enough, or does your implementation require bells and whistles like face tracking or object recognition? The answers to these questions will help guide you to the right mix of features and price, and ultimately to the right AR SDK for your project.
Suggest an Augmented Reality SDK
Did we miss an important AR SDK or miss important details about an Augmented Reality SDK with which you are experienced? Please contact us with the details. For more Augmented Reality insights, check out the Marxent AR Blog.
Joe Bardi is Marxent’s tech-obsessed former Communications Director.
