Learn more about careers and jobs at Marxent, the Marxent team and our company culture

Apple ARKit Update: New demos, rumors and more |
The In Reality Podcast

Apple ARKit Update

Welcome back for “lucky” Episode 13 of The In Reality Podcast, covering all things Augmented & Virtual Reality. In Reality features industry news, commentary, and perspective from AR/VR veterans and experts.

In Reality is co-hosted by Marxent’s Joe Johnson and Joe Bardi. Johnson is the Creative Director at Marxent, and has been in the AR/VR industry for four and a half years following a stint on Microsoft’s Office UX team. Bardi is Marxent’s Senior Content Strategist, and has been in the industry for a year, after having spent more than a decade in print and TV media.

This week we’re checking in on Apple ARKit — which we last discussed back in June, when Marxent Ph.D. Ken Moser walked us through the ins and outs of the SDK. Developers have been kicking the tires on ARKit since its June reveal, in the process creating some amazing and eye-catching demos. Now rumors are hinting at AR-specific hardware being included in the next iPhone, to go along with the snazzy software. Could the future be any more exciting? You’ll just have to listen to find out …

Thanks for listening. We really do appreciate it! Enjoyed the show? Please check out previous episodes at the In Reality Soundcloud page, or subscribe at the iTunes store, Stitcher or the Google Play Music.

Questions, comments, concerns? Email us at

Check back soon for another episode of In Reality.


00:01 Intro: Welcome to the In Reality Podcast. Now starting in three, two, one.


00:08 Joe Bardi: Covering all things augmented and virtual reality, the In Reality Podcast is hosted by Joe Bardi and Joe Johnson and features news, commentary, and perspective from industry veterans and experts. First up, introductions. I’m Joe Bardi, Senior Content Strategist here at Marxent, and I’ve been in the industry for a little more than a year.

00:24 Joe Johnson: And I’m Joe Johnson. I’ve been with Marxent for five years in the AR and VR industry.

00:27 JB: This week we’re checking back in on Apple’s AR kit. Developers have been kicking the tires on the SDK since its June reveal, in the process creating some amazing and eye-catching demos. There’s a lot to unpack there, so let’s get started.


00:45 JB: Apple’s AR kit software has been a major hit with the developer community since its unveiling at the Worldwide Developers Conference in June. The SDK has proved to be a powerful tool for developing augmented reality apps that run natively on Apple’s iOS platform and leverage the hardware to produce high quality results. After an early spate of proof of concept demos, mostly featuring 3D cubes being dropped on floors and from balconies, developers got creative doing everything from porting Minecraft and Super Mario Brothers into the real world to remaking the classic music video for a-ha’s “Take On Me.” There’s a measurement app, rocket demos, and even a beer pong simulator. Alcohol is sadly not included. As the devs were figuring out the software, an Apple employee was making a career ending mistake that revealed much of what Apple had up its sleeve for the hardware of the next iPhone.

01:28 JJ: Did they fire him?

01:29 JB: No, I’m just assuming. You accidentally release the… You accidentally upload to some public server the raw firmware for the HomePod that includes every reference to the iPhone 8, also called D22, also called Ferrari. That’s probably not gonna go over well with your bosses.

01:45 JJ: Good job, random Apple employee.

01:47 JB: Thank you. The big news from that leak is that the next iPhone looks to include a bevy of additional sensors that will further bolster its AR capabilities. So, Joe, with Apple’s big iPhone event and the release of iOS 11 a little more than a month away, let’s talk expectations. From everything you’ve seen so far, is there a killer app waiting in the wings to take the world by storm? Or is this going to be more of a slow burn as developers continue to figure out how to best make AR useful in everyday life?

02:10 JJ: So I know that we like to talk about the past not being an indicator necessarily of what the present’s going to be, or what future’s gonna be. So let’s look back at some stuff. So iTunes, just as an example, right?

02:22 JB: Okay.

02:23 JJ: People were making MP3s, Napster was definitely doing well, letting people swap MP3s, but it wasn’t like a mainstream thing until the iPod showed up and people were like, “Oh, hey there’s this great device that I can get all of my stuff on that is… It’s a nice package, the software is usable and it’s reliable.” That is honestly what I would compare this to the most in my head. They’re going to create an ecosystem where anybody, and it’s important that it’s everybody, can make their own AR app that’s reliable with tracking, is capable of a lot more things than just some guy screwing around in Unity with a webcam can do. I think this is the start of not an avalanche, more like a sea change.

03:11 JB: A sea change. Yeah, I think that’s a good way to put it. I would also [03:13] ____ like the original iPod, what was Jobs’ pitch, was like a thousand songs in your pocket?

03:19 JJ: Yeah, and we’re way past that now.

03:21 JB: Where, I think, if you have Spotify or Apple Music, you’re at 30 million songs in your pocket [03:24] ____. [chuckle]

03:25 JJ: And honestly, if you think about it, the iPhone actually killed the iPod to a certain extent.

03:28 JB: Yes, it did. It did. I think that this is Apple’s defense against AR killing the iPhone in the future by making it sort of inextricably linked with that sort of process. I generally agree [03:41] ____.

03:41 JJ: Is this how they’re killing Google Tango?

03:43 JB: They are.

03:44 JJ: They’re going to. I don’t think there’s any doubt about that.

03:47 JB: Once the hardware element really became involved, like there was a leak this week about a company that makes depth-sensing lasers that had had… That was obviously shipping in volume to Apple. So that there’s most likely a depth-sensing laser attached to the iPhone 8, or whatever we’re gonna call it.

04:02 JJ: I’m sure that’ll be good for its battery life.

04:04 JB: That’s the other interesting thing. Knowing Apple, they have probably figured out how to maximize that. The battery life on whatever the phone is will be the same as the last generation or five minutes longer, but it will not tank the device. So if you were unhappy with Apple battery life up until now, whatever.

04:21 JJ: Surprise, surprise, it’s not getting any better.

04:23 JB: I would compare the early AR kit apps to the first iPhone apps that Apple made. Stocks, weather…

04:32 JJ: Let’s not talk about maps.

04:33 JB: Maps. Very basic stuff, proof of concepts. Sort of like, this is what apps will look like. This is what AR will look like to some extent.

04:42 JJ: I think Apple knows that getting other third-party developers hands on their SDKs is probably the most important way to build their ecosystem.

04:49 JB: Absolutely, absolutely. And the buy-in has been incredible. Every day, it seems, there is a new demo where I’m just like, “Whoa, that’s crazy.”

04:58 JJ: Well, it is a crazy good tool. The fact of the matter is, they’ve got enterprise class tech in their consumer iPhone or their consumer iPad, and it looks and feels stable and good.

05:12 JB: Yeah. And some of the things that the developers have been doing with it, I’ve seen there’s the tape measure type stuff.

05:19 JJ: That’s actually a really big deal. I know that seems like a little thing but…

05:23 JB: It does. [chuckle]

05:24 JJ: But I think that… Okay, so think about why the iPhone killed other things. Like, it killed GPSs, like standalone GPSs, for a lot of people because it did it roughly as good and was in your pocket and was convenient.

05:35 JB: Right.

05:36 JJ: And the fact of the matter is, if you can create a set of tools and a measuring tape is just one of many tools that are possible with a AR kit, you will obviate the need to carry a lot of things, because you have one multi-tool, or god forbid, personal assistant, personal digital assistant.


05:52 JJ: You’re welcome palm.

05:53 JB: I would love a real personal assistant on my phone.

05:56 JJ: Yeah, you’re never gonna get that.

05:56 JB: I know. It’s sad.

05:57 JJ: I mean you might get that, but not now.

06:00 JB: One of these days. Anyway, sorry.

06:00 JJ: But I mean that’s a fair point. Like once it turns into like a little concierge, like a little valet in your pocket it becomes harder and harder to justify spending money on other things, to carry them around, when the tool that you have is already as good as… Almost as good as the real thing.

06:16 JB: Yeah. For me, the sort of the Trojan Horse that this represents is, I really view this as a full frontal assault on Google in a number of ways. Not just hardware-wise, where you had the Tango platform which has now been obsoleted pretty much overnight, but also their search monopoly. Because some of these proof of concepts have shown hold your phone up and look at a building and get all kinds of information, and it definitely seems like one of the major applications of AR going forward is going to be “look at object, phone tells you, or device, tells you what object is, history of object, things you need to know.” Perhaps because of its knowledge of you, maybe it can guess why you’re even asking and is presenting information that’s tailored.

07:02 JJ: Let me interrupt you briefly. Okay. So if it’s an assault on YouTube… Sorry if it’s an assault on Google’s search monopoly, they’re going to have… That’s an uphill battle. Google owns the number one and the number two most popular search engines, with YouTube being number two. And the fact of the matter is, I don’t know that Apple is prepared to attack data the same way that Google does. It may end up being a symbiotic thing.

07:25 JB: And that’s sort of my hope. That’s how I would like it to turn out.

07:28 JJ: Did you just say that you hope that Apple and Google turn into some nightmare-ish monopoly on search and image recognition?

07:34 JB: No. That’s one way to put it.

07:37 JJ: That’s how I typically put it.

07:38 JB: The way I would put it is, I hope that rather than it become one giant dystopian company running the future that there’s…

07:44 JJ: One utopian company.

07:46 JB: Four dystopian companies running the future. I feel like four is better than one. No, Google does a lot of good. I don’t want to see Google go the way of so many tech companies before it. I don’t want to see it turn into, like, Sony in the early 2000s, where it’s trying to figure out what it’s there for.

08:03 JJ: You don’t want to see its employees randomly release anti-diversity memos.

08:08 JB: That’s not helping.

08:09 JJ: I’m just saying.

08:10 JB: That’s not helping. [laughter]

08:11 JJ: Look, the fact of the matter is, that all of these companies are… You know, they employ human beings.

08:15 JB: They do.

08:15 JJ: As far as I know. And we’re all whatever we are. I’m just saying.

08:20 JB: Yeah. No, that’s all true. But… All that said, the point of… The idea of AR becoming a search platform, I think is a real thing. And if people get… People right now, we don’t even realize that it’s an invisible training that we all have. We need to know something, we type it into Google immediately. And if something gets in the way of that little exchange, which if it’s suddenly easier for me to just hold up my handset in front of something and I’m gonna get the search done, I’m gonna go that way.

08:51 JJ: Yeah. So, I have my own experience with typing things into Google, or deciding not to type things into Google. I switched over to DuckDuckGo a little while ago…

08:58 JB: Oh, right.

08:58 JJ: Because they don’t track your searches. I have noticed that the quality of results is not the same that I’m expecting from Google. And the fact of the matter is, you’re probably right. Everybody just uses Google because it feels familiar and normal and you get the type of results that you’re looking for.

09:14 JB: Yeah, and it works great. They know what they’re doing as far as search goes, and they’re constantly refining the results and the results are constantly getting better or sharper. I’m sure you remember back in the early days of search, you would search for stuff and get this just really disparate and wide selection of items that may or may not have even been related to what you were asking for. Now it’s really hyper-specific.

09:37 JJ: In some ways people use Google to navigate. You might not remember the URL for a place you’ve been 20 times, but you know if you type a certain search term in to Google it’ll be the first result. Which is a really interesting behavior.

09:48 JB: I do it all the time for insanely common websites.

09:51 JJ: You’re officially 40 or something.

09:52 JB: Yeah, yeah. And as I’m doing it too, I’m like, “You should not be Google searching CNN.” [laughter] It’s like, “This is stupid.” Hopefully I don’t do that one. But the actual examples would not be any better than that.

10:05 JJ: Do you type Google into the Chrome bar?


10:08 JB: No.

10:08 JJ: You know you don’t have to.

10:09 JB: I don’t do that. I’m one level above that. I know that the address bar doubles as a search bar. I’m glad I learned that.

10:17 JJ: Well, getting away from that. You and I predicted this back when we did our PayPal episode. When PayPal created a patent for image recognition combined with object identification and a whole bunch of e-commerce related behaviors in AR. There’s definitely a land race for the head space that AR represents.

10:39 JB: Right, absolutely.

10:44 JJ: They’re clearly planning for their devices to become an ever present pane of glass that we look through at the world for everything, and the statistics are already showing that. You know, you look at research around how millennials use their phones, that includes me. We know that we use them, basically, constantly. Whether or not it makes us happier people, or whether or not it actually does anything for us.

11:04 JB: No.

11:05 JJ: We are glued to them, and it looks like there’s really no way that we’re gonna unglue ourselves from them. Our mental health be damned.

11:14 JB: I basically agree with you. The only caveat I would put is that it feels to me like some of what they hope… I don’t know if any of this is actually gonna happen, because I think think that there’s a real leap to get people to wear their computers. Right? And that’s the whole argument about wearable computers.

11:28 JJ: So, I have to chime in here.

11:29 JB: Go ahead.

11:29 JJ: Neal Stephenson predicted people would be wearing their computers at some point. He calls them gargoyles. They’re outfitted with laser sensors, and they look inhuman and weird. Google Glass definitely hit the mark for gargoyles, when it first came out…

11:40 JB: Yes.

11:42 JJ: But I think that you’re gonna be seeing unobtrusive wearable tech soon enough.

11:46 JB: Yeah. And I mean, looking just at what’s out there right now and what’s coming, if I have an iPhone… Like just staying in the Apple ecosystem, right? If I have an iPhone, an Apple Watch, a pair of AirPods…

11:56 JJ: Technically, you’re wearing your computer already.

11:58 JB: I mean, I’ve broken the sound card out. I’ve broken the… And if there’s some kind of glasses that are coming, which allegedly they’re developing for enterprise first, whatever, now I have the screen is on my face, the sound card’s in my ears, and the processor is just basically in my pocket. And it’s interesting, ’cause in addition to miniaturizing everything, now we’ve separated it out and you carry it in little [12:20] ____.

12:20 JJ: Distributed computing, I believe is the phrase?

12:21 JB: Yeah. Yes, there you go. That’s gonna happen, like it’s…

12:27 JJ: Yeah. So here’s the real question, and this a question I always land on, when are you gonna have it implanted inside of you?

12:33 JB: I’m not, but my kids are gonna love it.

12:34 JJ: I’m gonna give it 10 years. I’m not even kidding. I’ll give it 10 years until people are seriously implanting.

12:39 JB: Let me ask it a different way. Because we know that implanting is possible now, right? They…

12:45 JJ: There are already people who are, say, implanting magnets in their hands so that they can feel magnetic fields, and there’s all kinds of bio-implants that are becoming popular.

12:50 JB: Sure. There’s always a cutting edge group who’s willing to try anything, right?

12:55 JJ: Next up on transhumanism.

12:56 JB: Right. Exactly, exactly. The question is, what’s gonna be the thing? What is the incentive that’s gonna make the average person do that? I think I [13:07] ____.

13:08 JJ: Well, it’s harder to lose your phone if it’s part of your skin but…

13:13 JB: Like I don’t think that that’s gonna convince grandma to do it.

13:16 JJ: It won’t convince grandma to do it, but if it’s something that makes it easier for people to do the things that they already want to do… Like I don’t need a tape measure, my contact lenses have access to measuring software and all I have to do is take my hands, pinch in one space, in imaginary space, and then pinch the other side, and it’ll tell me how far my hands are apart. They’re gonna do it.

13:36 JB: Yes. My answer to this question by the way would be medical technology is the Trojan Horse.

13:41 JJ: Do you think so?

13:42 JB: We are going to start implanting all kinds stuff into people in the name of medicine.

13:45 JJ: By the way, we are getting far afield on this episode.

13:47 JB: I agree. We’re way out there. And then based on that, it will eventually become totally okay to implant your phone into your body. Because it’s like, “Well, I’ve had the free roaming immune system chips for years, so… ”

14:00 JJ: Yeah. What do you need the screen for when it’s displayed in front of your eyes?

14:04 JB: Yes. Talk about augmented reality.

14:06 JJ: Well, no. I mean, we joke, we laugh about it, but that’s where it’s gonna end up if it continues to develop in the ways that we anticipate it will. Which it has so far. Like any number of science fiction genre writers have written various versions of a world where people have super computers in their pocket, and they have all kinds of distributive computing and they use it to do completely banal things.

14:31 JB: Yes.

14:32 JJ: And I think that that future has been realized.

14:34 JB: Yes. And what we’re saying is, it all began with a AR kit demo of a-ha’s “Take on Me.”

14:39 JJ: It kind of does, I think. You know how I am, I don’t like to give Apple all the credit in the world. And I’m still not…

14:47 JB: Rightfully so, by the way.

14:47 JJ: Because Metaio did it, right? But…

14:50 JB: Yes. They probably improved it a little.

14:52 JJ: They probably improved it a little bit, all that money helped. But Apple has certainly seen something coming. Apple has certainly leveraged an asset that they have, I think better than any other company in the space right now. I will give them that credit.

15:06 JB: Yes. This is the best… People always talk about the thing Apple does, they go into the market after it’s already starting to mature, and they let other people make the mistakes, and all of this stuff that sort of become this…

15:18 JJ: Late but great.

15:19 JB: Myth and legend that’s built around this company, that hasn’t actually been true as of late, right?

15:24 JJ: I would say 90% of the time it’s not true. But the fact of the matter is, that along with any other, say, human being, you’re remembered with the three or four awesome things you did.

15:32 JB: Right, of course.

15:34 JJ: And nobody remembers all of the misses.

15:36 JB: Yes.

15:36 JJ: I remember the misses, Apple.

15:37 JB: But this feels like not… This feels like one of the hits.

15:40 JJ: Yeah, it absolutely does.

15:40 JB: Where they triangulated it right. They got in at the right time. Google had Tango a year and a half ago, right? And they didn’t move.

15:49 JJ: I would say this, so the conversation around AR has been shaky from time to time, nothing is shaky about the Apple conversation.

15:55 JB: No.

15:56 JJ: Everybody is like, “Holy crap.”

15:57 JB: Yeah. And just knowing that on launch day, it’s gonna be in the hands of tens of millions of people…

16:04 JJ: Already.

16:04 JB: Who don’t have to buy a thing and are gonna get it for free. It’s incredibly powerful.


16:18 JJ: So thanks for listening to Joe Bardi do the intro this week.


16:22 JJ: It was weird. Was it weird for you guys? Email us.


16:27 JB: Or was it actually great? Email us. And in reality, I thought it was kind of awesome. And I’m Joe Bardi.


16:33 JJ: And I’m Joe Johnson, talk to you guys next week.


Start your project