Beginning ARKit

An overview of the Beginning ARKit course, as well as what you’ll be learning in this part of the course.

Find out what augmented reality is, from the century-old perspective of a book written by the author of The Wizard of Oz.

Learn about ARKit, its history to date, the hardware and software you’ll need to develop and use ARKit apps, and what options ARKit provides for rendering virtual objects.

It’s time to dive right into ARKit programming by starting with the ARKit app template that Xcode provides.

A look at SceneKit and some key concepts, including 3D coordinates, aligning those coordinates with the real world, SceneKit geometries, and SceneKit’s node graph.

It may be a simple one, but it’s the first step to drawing more complex SceneKit scenes.

You just drew a dull object. Now you’ll make it shiny, and learn about diffuse and specular reflection while doing so.

You know how to draw a sphere, and make it shiny and blue. Can you draw a shiny green pyramid?

Let's turn that shiny blue sphere into our home planet by covering it with a photo of the Earth’s surface.

Take what you’ve learned from the previous lesson and draw a flat surface with a cat picture on it.

They’re pronounced “oiler”, not “yoo-ler”, and they’re how you can tilt objects at jaunty angles.

Now that you know about Euler angles, apply them to that cat picture.

Actions are objects that make changes to SceneKit objects. They can move objects, rotate them, grow and shrink them, change their appearance and more. Learn how to use them to put a little spin on the Earth!

All this time, you’ve been drawing simple shapes using SceneKit geometries. Let’s draw something fancier.

You’ve just completed a crash course in SceneKit and written some basic AR app. Let's review where you are and see what comes next.

Let's review what you'll be learning in this part of the course, and review the history of its inspiration, the legendary Bob Ross.

Before you begin with this exercise, you’ll need to learn a couple of matrix math concepts, as well as an AR SceneKit property called pointOfView.

So far, you’ve been writing apps that simply draw images over the real world. It’s now time to write apps that respond the to real world. Your first step: finding out where in the real world the device is, and which way its camera is facing.

What’s with the other tab in this app? You’ll find out here.

Now that you can determine the device’s position and orientation in the real world, you can use that information to leave a trail of virtual objects, effectively “painting” in 3D space.

Your “paint” trails are SceneKit objects, and SceneKit objects can have SceneKit actions applied to them. Let’s liven them up by animating them.

You’ve just written a 3D painting app! Let's review where you are and see what comes next.

In this part of the course, you’ll build an app similar to one made by everyone’s favorite Swedish semi-disposable furniture store.

A look at the “two worlds” theory of the real and virtual world, as well as the important concept of AR anchors.

In this episode, you’ll configure the app to detect horizontal and vertical planar surfaces in the real world, and get it to tell you where they are.

It often helps to understand the theory behind how something works. Let’s take a look at how ARKit detects horizontal planes (relatively easy) and vertical ones (a little trickier).

Now that we now how to detect real-world planes, let’s draw AR planes over them. We’ll draw a grid that says “place furniture here” over detected horizontal surfaces, and a poster of Ray over detected vertical surfaces.

We return to the “two worlds” theory to talk about the fact that as an AR session goes on, ARKit learns more about the real world, and updates its virtual world to match.

Now that you know that ARKit revises its internal map of the world as it learns more about it, you’ll take advantage of that learning to revise the planes you draw in AR space.

In Raykea, the user taps on the “place furniture here” grid on the screen, and that tap specifies where the furniture goes. But how do we translate those taps into a 3-dimensional location for the virtual furniture? With hit tests.

Let’s use hit tests to see if a detected horizontal surface is onscreen at the moment.

Putting it all together, you'll take everything we’ve covered and finish building the app.

Congratulations — you’ve written a scaled-down version of one of the most popular AR apps! Let's review where you are and see what comes next.

In this section, you’ll be building an AR guidebook — a Baedeker — for paintings and photos in galleries, museums, or any place that displays 2D art.

The first step in building BaedekAR is providing it with the set of images that it should detect and recognize. You’ll do that in this step.

With the reference images imported into the app, it’s time to take the first step and detect those images in the real world, making note of their location and size.

Now that the app can detect the reference images in the real world, let’s make it so that the user can tap them to find out more about them.

Let’s improve the app by displaying the name of the detected image using AR text. We’ll also make use of billboard constraints to make sure that the text is always facing the user, so that it’s easy to read.

What if there were images that we wanted to block from the user?

Let’s protect the user from scary clown images by blocking them with a soothing picture of Ray.

Congratulations — you’ve written an AR app that can detect known real-world images and block scary ones! Let's review where you are and see what comes next.

In this part of the course, you’ll take BaedekAR into the third dimension by using the same concepts to detect known 3D objects instead of 2D ones.

Before you can detect 3D objects in the real world, you’ll need to provide reference objects, which means that you’ll have to scan them first. You’ll download Apple’s app for scanning 3D objects and put it to use.

Now that you’ve scanned your reference objects, it’s time to import them into the app.

With the reference images imported into the app, now you can detect their real-world counterparts.

Congratulations — you’ve written an AR app that can detect known real-world objects! You’ve also completed the course. Let's review where you are and see where to go from here.