Password Required

Please enter the password to view this page

Apple Releases ARKit and Everything Changes

In this week’s episode, Joe B. squeals like a fangirl over the latest Apple News — the unveiling of an Augmented Reality Software Development Kit for native AR apps — while Joe J. tries to calm him down.


00:01 Speaker 1: Welcome to the In Reality Podcast. Now starting in three, two, one.


00:08 S1: Covering all things augmented in virtual reality, the In Reality Podcast is hosted by Joe Bardi and Joe Johnson, and features news, commentary, and perspective from industry veterans and experts. First up, introductions. I’m Joe Johnson, creative director at Marxent Labs, and I’ve been in the AR and VR industry for four and a half years now since coming from Microsoft’s Office UX team.

00:28 Speaker 2: And I’m Joe Bardi. I’m senior content strategist here at Marxent. I’ve been here for a little over a year after spending more than decade in print and TV media.

00:35 S1: This week, Bardi wants to go full fanboy with an in-depth look at the augmented and virtual reality announcements made by Apple at this year’s Worldwide Developers Conference. I’d really rather not but since he wrote it in the intro, hey, Siri.


00:54 S1: Apple’s annual Worldwide Developers Conference kicked off on Monday with a product-packed keynote. CEO Tim Cook, SVPs Craig Federighi and Phil Schiller and other members of Apple leadership took to the stage to unveil new iMacs, MacBooks, iPads and a home speaker that’s totally not designed to go head-to-head with Alexa. In the middle of all that hardware, Apple dropped the biggest VR news of the year so far. First, Federighi took the wraps off of Metal for VR, Apple’s graphic processing API, which is coming soon for macOS. Insert weed joke here. Craig sure did. Did he really?

01:29 S2: Yes. No. You cut out the Hi-C error shit.

01:32 S1: Oh, whatever. Then I’m not gonna worry about it.

01:33 S2: Well, you can’t make the weed joke because you cut out the weed joke. [chuckle]

01:33 S1: Yeah that’s fine. I know. Oh my God, I cut out the weed joke, I didn’t like it.

01:38 S2: He called it like half-baked or fully-baked.

01:41 S1: Fully baked. Oh, Jesus Christ. Okay. There’s an update to Final Cut that adds support for professional 360-degree workflows with the ability to import, edit and export 360 degree video natively. And Apple announced corporate partnerships with Valve, Unity and Unreal who are bringing VR SDKs to the refreshed line of iMacs. The VR announcements were a mere appetizer, however, to an even bigger reveal. Apple’s ARKit SDK, which the company touted as enabling fast stable motion tracking, plain estimation with basic boundaries, ambient lighting estimation, scale estimation and support for Unity, Unreal and SceneKit. Alasdair Coull, creative director at Wingnut AR, Peter Jackson’s AR company, then took the stage to narrate a sweet, sweet tabletop AR demo involving an airship battle over a dusty outpost. Apple boasted that ARKit would make iOS the “Largest AR platform in the world” because of how many devices already exist in the wild. And then closed out the AR section of the presentation with the invitation/warning to its competitors, “Look out for an AR experience from us in the app store later this year.” So Joe, there’s a lot to unpack here, what is your first impression of this deluge?

02:57 S2: So, my first impression watching the keynote was, “Wow! That’s really cool. That’s incredibly impressive. The demo played wonderfully on the stage.” Apple is nothing, if not fantastic, at showcasing their products.

03:11 S1: Yeah, they’re good marketers.

03:12 S2: They did another great job here capturing, I think, probably the imagination of that room.

03:18 S1: Is it appreciably different from AR apps that have existed a year ago?

03:23 S2: And so that’s the rub. No, not really.

03:25 S1: Yeah.

03:28 S2: It’s a great first step for Apple. It seems like a powerful group of tools that will get used. The thing about this announcement is it’s not so much the technology that is the major part of it, it’s the…

03:41 S1: It’s the broadening of the platform.

03:42 S2: It’s the broadening of the platform. It’s the idea that AR experiences now are gonna go even further beyond Pokémon Go, although they…

03:51 S1: Which they made a big deal about on stage which I thought was really weird.

03:54 S2: Well, it was funny. We spent a year talking about how Pokémon Go is not real augmented reality.

04:00 S1: Yeah. [chuckle] And then they made it augmented reality.

04:00 S2: They were like, “Sorry. Here it is. Well, it’s real.”

04:03 S1: I’ll be honest. I’m actually a little more interested in it, not a ton, but it is slightly better. It’s true that it’s slightly better as an actual…

04:08 S2: Could I bounce my Poké ball off a tree and then…

04:10 S1: Did you just call it a Poké ball?

04:11 S2: Isn’t that what it’s called?

04:12 S1: We’re gonna move on.

04:13 S2: Okay. So the demo began with a tea cup and a lamp and a little flower, a little vase. And they were able to place them on the table that Craig was standing at. It was a very cool little demo, it was done live. You can see the audience in the background.

04:28 S1: But they didn’t have anything in front of the experience in between the camera and the object. So they clearly haven’t marked out for occlusion yet, which is the next holy grail on AR, having things be able to intercede in between the camera and the experience itself.

04:41 S2: That’s right, yeah. The other big question I had watching it the entire time, and I’ve gone back now and I’ve watched it three different times to try to figure out, and I still don’t really have an answer. How much of this was gonna be camera-based and how much of it was actually… How much was it two camera-based versus a single camera solution? Because that’s one of big Markerless AR deal.

05:01 S1: Yeah. It’s the difference between a Tango and the iPad.

05:03 S2: Correct. And so if you watch the video, they highlight an iPhone 7 plus with the two cameras and it’s circled, and Craig actually says “cameras.” Our developers had now downloaded the ARKit and they’ve been looking at it. And from what they’re saying is it’s actually really… It’s the camera combined with the motion chip that really makes it work, which is why it’s apparently only gonna be usable on any Apple device that’s an A9 processor or better, so…

05:34 S1: What is that, like 6 plus?

05:35 S2: 6 plus or better.

05:37 S1: So, yeah. A reasonable sampling of devices.

05:39 S2: Yeah. And a good way to obsolete a whole lotta iPads as you’ve got a whole new line of iPads coming up.

05:48 S1: Yeah. I’ve heard that Apple’s been known to do that from time to time…


05:51 S1: To accidentally, on purpose, make a bunch of their equipment obsolete.

05:54 S2: Yeah, it’s a rumor. [chuckle] No.

05:57 S1: So the interesting thing to me is that they didn’t really highlight any new use cases for augmented reality. Everything that we saw was basically a rehash of what augmented reality’s already been doing up to this point. They didn’t showcase anything like buying objects in augmented reality. They didn’t showcase any productivity tools in augmented reality. Mostly what I saw was toys. Now they’re cool experiences, obviously, but I don’t know how useful that nomadic desert outpost getting attacked by ships demo is, considering we can already do that.

06:24 S2: Yeah, no. I think that that’s a great point. One of the most interesting things about WWDC is that it’s developer-focused, obviously, and they program it as if it’s for developers, which makes perfect sense except that the whole world consumes it as content, not just developers. So I saw a lot of reviews of the show after the fact that were like, they were positive reviews and they were like, “Well, they took a more no-frills approach and it didn’t end with a big musical production or whatever.” And I think some of that was by design based upon previous events but some of that was just that it was WWDC and it’s not the product announcement from the fall where they bring out the iPhone. So the minute the announcement comes, you start thinking about the competitive landscape and how these things work and who does this really affect and those kinds of things…

07:11 S1: You know, ironically, I don’t think it really affects us all that much.

07:14 S2: I don’t think it affects us either. The system that Marxent has build for Markerless AR is not similar in the underpinnings as the Apple but they do the same stuff.

07:24 S1: Yeah, you get the same output, maybe not the same input.

07:26 S2: Correct. And the Marxent solution will work on Android as well.

07:30 S1: Yeah, that’s convenient for us.

07:31 S2: There’s some bonuses. It will definitely…

07:35 S1: Well, actually, it isn’t just that it works for Android, it’ll work for any device that has a camera. Any device that has a camera and a computer attached to it can do the Markerless that we do. And I don’t say that to boast or make a competitive differentiation, I say it because it just means that we do different things.

07:51 S2: Yeah, from those perspective of what we do and how it’s gonna affect us and whatever, I was excited because I just see it broadening AR as a platform.

08:01 S1: Yeah, I mean that promise/warning where they say look for an AR app for us in the app store, coming soon, I’m like, “Great.” Maybe it means that other people are gonna be taking a look at what we’re doing and saying, “Oh, this is in the mainstream, this is something that’s going to be a real thing.”

08:13 S2: Yeah. And, so if you look at web design or app design as a history, right? Apple is now providing tools that will make it easier to do these things. You could have built in a Markerless AR application for the last five years.

08:26 S1: Yeah. So we’re gonna get glut of terrible, crappy AR applications, and probably, you’re gonna see three or four really useful killer apps that help adoption.

08:37 S2: And the people who will make those are the ones who have been playing in this space for a long time and who didn’t just get ARKit yesterday and decided to make an AR app and now, they have a breakthrough.

08:46 S1: Well, there’s an old story right? The people who’ve already been doing it are already ahead of other people who are just now getting new tools.

08:52 S2: A great sort of corollary is web design in the ’90s. Web design in the ’90s was impossible.

08:57 S1: Please don’t remind of web design in the ’90s.

09:00 S2: But then there were things like GeoCities or Dreamweaver that made it much easier for…

09:04 S1: Angelfire.

09:06 S2: Angelfire for a larger group of people to do it. It didn’t mean that suddenly there where a million great websites, it meant that there were 999,000 terrible websites and a few good ones.

09:15 S1: We don’t need to get into my brother’s…


09:17 S1: Dragon Ball Z, RPG GeoCities site. Although, hey, James. What’s up?

09:22 S2: That was exactly the site I was referencing.

09:24 S1: Yeah, I know.


09:25 S1: Yes, you’re right and accordingly, web design in the ’90s is atrocious.

09:30 S2: Yes, we look back and laugh.

09:31 S1: Yeah. And there’s a reason that the web as it stands right now is a more beautiful, more functional place.

09:39 S2: And one other aspect of this I’d like throw out, which I haven’t seen written about a lot but I think it’s gonna be really important, is we keep talking about and we keep hearing that retail is changing and that augmented reality, virtual reality are gonna be drivers of this change or they’re gonna really lead to these new experiences. One of the things holding that back is lack of technology, lack of applications, lack of devices. And Apple’s announcement goes a long way towards putting an AR device in everyone’s pocket and at that point…

10:10 S1: I think I actually talked about this on our PayPal episode. I said that the next step for AR commerce really is having it in your phone and you just pull it out and you look at the camera, there’s that functionality built into it to determine what you’re trying to do.

10:22 S2: Yeah, I think you’re gonna see major… Is Target gonna have some kind of AR implementation in the store in another year or two? Probably.

10:31 S1: Judging from the stuff at WWDC, I don’t think that’s the case.

10:35 S2: Really?

10:36 S1: They didn’t showcase anything that speaks to me or, hang on. They didn’t showcase anything that says to me, “We are thinking about how to integrate this into commerce.” Now I know that that’s not their job, it’s the job of the developers. However, usually there needs to be some sort of inspiration chain for figuring out how to use these things and I don’t think anybody’s there yet.

10:57 S2: Yeah, I think that’s fair.

11:00 S1: I’m gonna toot our own horn a little bit but as far as I can tell, the only AR and VR developers out there who are thinking about how to help people buy things using AR and VR might be us.

11:07 S2: Yeah, I don’t disagree. Again, that’s why I thought yesterday’s news was so exciting because A, I don’t feel like there’s any sort of competition involved here, it’s more like Apple is broadening the pie of what augmented reality is going to be.

11:24 S1: Did you just say broadening the pie?

11:25 S2: I did, is that not a proven metaphor?

11:28 S1: They’re selling the discords of love.

11:30 S2: They’re broadening the base there.


11:34 S1: So we’ve established that they’re not really like a competitor of ours. Is this going to be something we can use? I don’t see any reason not to necessarily. Obviously, developer’s gonna have to get on and tell us why we can’t use this. Leaving out for the moment whether or not we can just use Apple’s tool kit to do what we already do. I mean where we go from here? What’s the next step for us? I think it’s the same.

11:53 S2: So A, I don’t think it changes anything that we’re doing or we’re developing or whatever I think that what happens now is the developer community that’s around Apple is going to go out and make a whole bunch of AR apps and 99…

12:03 S1: I cannot wait.

12:04 S2: 99% of them are gonna be garbage.

12:06 S1: The Schadenfreude is gonna be real.

12:08 S2: But that 1% that isn’t, is gonna be really cool when it’s gonna show us a bunch of new things and new ways to use the phone and new ways to use the iPad. I, obviously, being who I am and where I work, I believe in this stuff but it’s impossible for me to see a future four or five years down the line where augmented reality, virtual reality are not deeply ingrained into technology experiences across a range of devices.

12:34 S1: Are we going to be looking at somebody like Samsung or Google developing their own SDK, or do you think that they are going to pick up SDKs from somewhere else? Because the fact of the matter is that Apple didn’t develop that SDK either. Okay, that was Metaio and in addition to that, Snap didn’t developed their AR SDK, Cimagine developed theirs. So, is there going to be a future where a bunch of AR companies get snapped up by say Samsung or Google or whatever?

12:58 S2: Amazon.

13:00 S1: Oh yeah, that’s a good point.

13:00 S2: You know, Facebook. If you are, we talk about the big five or the big six. At the top level of Apple’s announcement, that’s what you’re thinking, how does this affect Google, Facebook, Amazon, Microsoft?

13:13 S1: Well, this is a good opportunity to talk about that. So who’s ahead and who’s behind? It looks like Apple’s ahead.

13:17 S2: It does.

13:18 S1: And it looks like Snapchat is nearby in terms of AR functionality, maybe not necessarily the market that they’re in or what they are doing but they’re certainly close in terms of tech.

13:27 S2: Snapchat has probably brought AR tech to the most people.

13:30 S1: Oh, I would agree with that.

13:32 S2: They’ve got the largest base of people who, whether they know it or not, are using AR technology.

13:38 S1: Didn’t Apple just talk some shit about how they have the biggest AR network now?

13:41 S2: They did. They did, but they would count everybody who’s using Snapchat probably as on their AR network ’cause they’re using iPhones.

13:47 S1: Dirty birds.

13:48 S2: I think Microsoft is the leader as far as some kind of headset or glasses go for augmented reality.

13:53 S1: I would agree with that. I think the HoloLens is clearly the winner in that category.

13:58 S2: Yeah. It’s a good piece of technology, it works. You can clearly see how it’s pointing the way to the future.

14:02 S1: It’s the piece of technology that I wanna get my hands on the most, personally.

14:05 S2: Yeah, exactly. So you have those two.

14:09 S1: Those three?

14:10 S2: Well, I did Snap and…

14:12 S1: Snap, Apple, Microsoft.

14:13 S2: I was going to say, so Apple now has AR tools that’s gonna allow their developer base to build AR across all kinds of different apps for iOS. I think that puts Google on notice. I think Google was a loser.

14:28 S1: Yeah, I think Google might be the loser in this case.

14:30 S2: And I’ll be honest, I watched both Google’s keynote and Apple’s keynote all the way through and both have like… You’re sitting there watching the future, it’s hard to be…

14:39 S1: So that’s an interesting point, Google’s play was clearly for VR.

14:42 S2: Yes.

14:42 S1: Yeah, the Daydream is clearly a VR play. I don’t see them getting involved in augmented reality a ton, at least not as far as they’re talking about.

14:48 S2: Yeah, and to me, the VR announcements from Apple were a way to, A, end a talking point, which is VR isn’t for Mac. No VR Mac, no VR in Mac and that’s what you always hear.

14:58 S1: Well, that’s not true. There was some conversation… Sorry, what did you just say?

15:02 S2: Well, I was saying Apple’s VR announcement was a way to end a talking point that’s used very regularly, which is, “Well, if you wanna do VR, you have to use a PC.” If you are a developer, you will, going forward, be able to develop on a Mac if you get one. Now, there’s dongles and whatever else.

15:18 S1: They are also in a bad spot because VR is mostly about gaming, and let’s be honest.

15:23 S2: Not a gaming platform.

15:24 S1: That’s not a thing, it’s not a gaming platform so it may well be that VR doesn’t really penetrate well in the Mac landscape.

15:29 S2: Yes. And I think Apple is very smart about themselves and you see in their strategy in what they are doing. They know.

15:37 S1: Are you saying they’ve self-actualized? Did they hit the fifth realm?

15:39 S2: I think they are more than a lot of people realize and then that’s why the subtle changes between these events are always so important. The fact that Tim Cook seems to be de-emphasizing himself. He’s still there. He’s still the master of ceremonies but…

15:53 S1: Yeah, but there were a bunch of other people talking.

15:54 S2: The majority, majority is other people like, that’s evolution there. They realize what they do well and what they don’t.

16:01 S1: So where does Amazon fit into the big six?

16:03 S2: Amazon…

16:04 S1: We’ve established that Google’s probably the loser in this case despite the fact that Daydream looks impressive.

16:10 S2: Another reason that Google is probably a loser is… Amazon Prime Video now coming to the Apple TV. I don’t think it was explicitly said but it’s generally assumed that the Apple TV will now be sold on Amazon, which up until now, they haven’t been selling Chromecast, they haven’t been selling Apple TV.

16:24 S1: That means that my Amazon Fire Stick is a waste of time.

16:26 S2: And so you have the enemy of the enemy is my friend kinda thing here. It definitely feels to me like you have Amazon and Apple playing nice as sort of a play against Google, gobbling everything up. Google is trying to invent everything. So it’s a real interesting dynamic.

16:45 S1: Maybe they’re just not really a serious player in the space then.

16:48 S2: I don’t think you could ever say they’re not in anything they think about.

16:51 S1: Well, I just did.

16:52 S2: You did. [chuckle] You did but they have billions of reasons…

16:55 S1: That’s true, you’re not wrong.

16:57 S2: That mean you should take them seriously.

17:00 S1: Well, speaking of taking things seriously or maybe not taking things seriously, it was funny to me how much they focused on the Pokémon stuff in the presentation. Like, I don’t think it’s great to take what could be considered a fad after the fact and then stick it into your giant presentation and crow over how real the Pokémon are when nobody… I don’t think anybody really cares right now.

17:21 S2: No, no.

17:23 S1: It’s already past that.

17:24 S2: It’s kind of a year after the hula hoop craze being like, “Check out this hula hoop, man, we really souped it up.”

17:29 S1: Is it like an oval?

17:32 S2: You know for kids.

17:32 S1: For kids. [chuckle] Is that a Hudsucker Proxy reference?

17:35 S2: That is.

17:35 S1: O man, you’re my hero.


17:37 S1: Hey, everybody, watch all the Coen Brothers movies as you can.

17:39 S2: Yes, yes, in AR.

17:41 S1: I am serious though, you can fanboy out all you want, I wanna talk about missteps.

17:47 S2: Yeah, let’s do that.

17:49 S1: Okay, so if we’re talking about missteps, you’ve got your Pokémon Go focus. I don’t think it was a good move. Them referring to Ikea and what Ikea is doing as far as AR goes, is a little past prime too.

18:02 S2: Well, the thing that was interesting about that is IKEA is attempting but they’re not doing a great job of it up till now.

18:07 S1: Yeah, we need to talk to those guys.

18:09 S2: I don’t know how much they’ve dedicated to it or whatever. There’s definitely an attempt being made but compared to some of the other stuff that’s on the market, it’s not as good.

18:18 S1: I’ll boil it down. I think that Apple’s missteps is hitting AR stories that were popular a year ago.

18:24 S2: I got you.

18:25 S1: That’s the misstep that I see. Them not showcasing something that looked like an actual tool for people to use AR profitably, productively is a big miss. And that’s something that HoloLens nailed a lot better. Their entertainment pitch is better. They’re talking about “Do your fantasy football in the headset while you’re watching the game,” and all that stuff. That stuff actually looks interesting and useful rather than just a toy.

18:47 S2: It does. Right. I think they were just trying to be attention getting. That’s what they’re doing. It’s not a defense but…

18:57 S1: They missed my attention.

19:00 S2: Apple always sort of leads from behind. That’s their thing, right?

19:03 S1: Jesus, you’re just like Patrice.

19:06 S2: But that’s what they do. Because now, the history of the iPod is such this ingrained story like, “In the annals of history… ”

19:14 S1: You can’t see this right this right now but I’m rolling my eyes.

19:17 S2: They’re rolled so hard right now.

19:19 S1: I think they might get stuck.

19:20 S2: “The Rhapsody music player that was out and then Apple came along.” That’s the story, that’s the mythology of it.

19:28 S1: I still miss my Zune.

19:30 S2: When I looked at that home speaker, right? I was like, “Oh, that looks nice.”

19:35 S1: Yeah, it’s Mac’s Alexa.

19:37 S2: Exactly. And I thought they were really smart about how they positioned it with the music and whatever. Do I think that that’s gonna dethrone Alexa? No, not one bit.

19:44 S1: No. It just means that the people that are already in the Mac ecosystem…

19:48 S2: Will now have an option.

19:49 S1: Who wanted a home speaker can now get something.

19:51 S2: My point was it still gets taken incredibly seriously because even though it’s late to the party by over a year and there’s already multiple competitors, everybody knows that iPod story. So they’re like, “Well, maybe Apple can do it again.” I don’t think that’s the case in this.

20:10 S1: It reminds me of the era post-Apple 2 computer when the Mac was not doing anything interesting and PCs were doing their thing and then eventually, Mac came out with iMacs. And then they started become relevant again from a business productivity standpoint. I think they’re doing that again. They had a big hit. They’ve had a bunch of big hits but they’re still just sort of coasting. It’s hard to say that one of the biggest companies in the world is sort of coasting but I feel like they’re sort of coasting.

20:35 S2: It is, for sure. I also don’t discount how much effort went into that spaceship building they just built, which we could discuss whether or not that was a good idea or a bad idea.

20:45 S1: I’m sorry, what?

20:46 S2: They just build a brand new headquarters that looks like a spaceship that cost $6 billion that took multiple years.

20:48 S1: Oh did they? I do like spaceships.

20:51 S2: It looks like the Pentagon as a circle. I think that since Jobs died, they’ve been finding their way a lot and I think that process, that will go on for forever but you can clearly see how we’re now in the post-Jobs era. And whatever that means and for better for worse but that’s where they are and it’s still exciting.


21:18 S1: On a personal note, watching somebody produce video for an AR application live on stage with an iPad. Personally, vindicating. It’s not easy.

21:32 S2: Yeah, oh no it’s not, knowing what we know about how let’s say AR demo videos are produced, right? The whole time I’m watching that, it’s just a high wire act to me. And I don’t know if you noticed and actually, Dr. Ken pointed this out to me. Right as that demo starts, there’s some kind of orange grid on the iPad that the guy pushes the button, it goes away. That was Peter Jackson’s company, right? We laugh about this because there’s that famous magically demo…

22:02 S1: Magically.

22:03 S2: Of the people shooting aliens in their office and it’s awesome.

22:06 S1: And totally not real.

22:07 S2: I want to play that game and right at the bottom, it says like, “Shot through Magic Leap’s technology.” Not so much, not so much.

22:13 S1: I’ll be honest. I used that bug on a video to spite them at some point.

22:17 S2: [laughter] It’s a funny joke.

22:18 S1: It is a funny joke.

22:18 S2: For the nine people who were really that inside.

22:21 S1: Anyway, what I really meant to say was, “Good job, Apple. It’s really not easy to shoot an AR event live.”


22:27 S1: In reality, I’m Joe Johnson.

22:29 S2: And I’m Joe Bardi.

22:31 S1: And we’ll see you next week.


Most Recent Podcasts

3D Content Lightning Round

Welcome to the In Reality podcast, where we cover all things Augmented & Virtual Reality. The In Reality Podcast is… continue reading

Catching up with the VR/AR Hype Cycle

Welcome (back) to the In Reality podcast, where we cover all things Augmented & Virtual Reality. The In Reality Podcast… continue reading

The future of retail with Sterling Hawkins

This week, we’re wrapping up Season 2 of the podcast with another very special guest: Sterling Hawkins is a global… continue reading

Request Demo