1. AR Quick Start

Written by Chris Language

Welcome to your first step into Apple’s wonderful world of augmented reality (AR)!

This chapter has been designed from the ground up to satisfy the need to create an Augmented Reality App with Xcode as quickly — and impatiently — as possible. Who wants to read pages upon pages of information before getting to the fun part? Well, you’re in luck, because the fun begins right now!

In this chapter, you’ll create a fun little augmented reality app using Xcode. By the end of this chapter, you’ll have a fully remote-controllable Tiny Toy Tank. You’ll be able to drive it around and make it aim and shoot at stuff. Sound good? Great! It’s time to get to it.

Creating an AR project with Xcode

In the following section, you’ll create a small augmented reality app project using Xcode.

Start Xcode, then create a new project by selecting File ▸ New ▸ Project…. When asked to choose a new template for your project, select iOS ▸ Application ▸ Augmented Reality App, then click Next to continue.

Enter TinyToyTank for the Product Name and choose RealityKit as your Content Technology of choice.

Make sure to set the Interface to Storyboard before clicking Next to continue.

Select a secure location to save your project. To keep things simple, the Desktop is always a good place. Once that’s done, click Create to finalize the project creation process.

Excellent, you’ve just created an augmented reality app with Xcode. Congratulations, chapter done! :]

Wait, didn’t someone mention something about a Tiny Toy Tank? Oh, of course, yes, continue on, then.

Note: Feel free to do a quick build and run at this point. You’ll see a glorious cube sitting on the nearest flat surface in front of you. How boring! Now, if that was a pretty-looking tank, it would be much more interesting.

Exploring the project

Not to spend too much time on this, but take a look at some of the key components Xcode generated for your augmented reality app:

• AppDelegate.swift: Here’s the starting point of your app.

• ViewController.swift: All of the code behind the entire AR experience resides here. You’ll add a lot more code to this file, too.

• Experience.rcproject: This is a Reality Composer project. It contains all of the 3D assets, animations and sounds you’ll use for the AR experience. You’ll replace it with a custom project containing all of the graphics, animations and sound effects for the Tiny Toy Tank AR experience.

• Assets.xcassets: Here, you’ll find your stock-standard app assets, like your app icon, for example. You’ll add some images for the buttons here.

• Main.storyboard: Contains all of the UI for your app. You’ll add some basic control buttons for the Tiny Toy Tank here.

• Info.plist: This one is important when dealing with AR apps. It’s where your app requests access to various hardware components, like the camera, for example, to deliver the AR experience.

Great, a lot of stuff is ready for you out of the box. Now that you’re aware of what’s contained within the default project template, you can start adding some custom content.

Asking for camera permission

To present an AR experience, the user must grant your app access to the device camera. You do this by adding the Privacy — Camera Usage Description property to Info.plist. Your Info.plist already has this property, but it still needs a proper description.

With Info.plist open, set the value of Privacy — Camera Usage Description to AR experience requires access to Camera.

Now, when your app launches, it’ll request access to the camera, explaining to the user that it needs that access to deliver the AR experience.

Adding custom assets

All of the 3D assets that the Tiny Toy Tank experience requires are contained within a single Reality Composer project file. You’re going to add it to your project now.

Using Finder, locate starter/resources/TinyToyTank/TinyToyTank.rcproject. Drag and drop this file into your project, placing it immediately below Experience.rcproject.

When prompted to choose an option for adding the file, select Copy items if needed to add a copy of the file, not a reference, to your project. Select Finish to complete the process.

Select TinyToyTank.rcproject in the project hierarchy and meet the hero of your AR app — the glorious Tiny Toy Tank.

Exploring the Reality Composer project

You might wonder what’s inside this Reality Composer project file. To take a closer look, select the Open in Reality Composer button at the top-right of the view. The project will open up in Reality Composer, a companion app that’s installed with Xcode.

In the top bar, select Scenes to view the Scenes panel and select Properties to see the current scene’s properties.

Note that this project only contains one scene named TinyToyTank and that it uses a Horizontal anchor type.

Select the Behaviors option to reveal the Behaviors panel at the bottom.

Here, you’ll find a collection of Behaviors, which animate the tank and play sound effects. Select the TurretLeft behavior, then select the little Play button next to the Action Sequence.

The turret on top of the tank will turn left while making a ratchet sound. When it’s done, the turret resets to its original position. You’ll use all of these behaviors to move and turn the tank.

Enough about that for now, you’ll learn more about Reality Composer and its features in another chapter. For now, close Reality Composer and jump back into Xcode.

Presenting the custom scene

Currently, the AR experience is only showing a box. You need to change it to show the tank instead.

Open ViewController.swift so you can make some code changes.

Declare the following variable at the top of ViewController:

var tankAnchor: TinyToyTank._TinyToyTank?

This declares tankAnchor as type _TinyToyTank, which you’ll initialize by loading the TinyToyTank scene from the Reality Composer project.

Replace the contents of viewDidLoad() with:

// 1
super.viewDidLoad()
// 2
tankAnchor = try! TinyToyTank.load_TinyToyTank()
// 3
arView.scene.anchors.append(tankAnchor!)

Look at what’s happening here:

1. This ensures that the super class viewDidLoad() is called.
2. Xcode has generated access methods to the TinyToyTank.rcproject. Here, you’re using load_TinyToyTank() to load the scene and store it in tankAnchor.

Excellent, now you’re presenting the newly-added TinyTankScene instead of the old Box scene from Experience.rcproject.

To keep your project nicely organized and clean, now would be a great time to remove Experience.rcproject from your project. Right-click it then select Delete. When asked, select Move to Trash to completely remove it.

Now that you’ve removed the file, your project is once again clean and organized. It’s time to test things out. Connect your device, then build and run.

Your app starts and requests permission to access the camera. Once you grant that permission, point the camera toward a flat surface. A tiny tank will spawn out of thin air, placing itself right on top of that surface. Amazing!

It’s a pretty little tank indeed, but it ain’t moving? That’s no fun! You’ll take care of that next.

Setting up the UI

You’ll need to add some on-screen controls for the user to let them control the tank. There are a several button images for you to use under starter/resources/Buttons.

Select Assets.scnassets then drag and drop all of the images into it.

Setting up ARView

Now that you’ve added the images, you need to configure the UI. Keep in mind, this chapter is not about building UIs, it’s about creating the AR experience. With that in mind, and to keep things as simple as possible, you’re going to take some shortcuts when creating the UI.

Open Main.storyboard, then select ARView Scene ▸ ARView ▸ Ar View.

Press Backspace to delete it. You can’t use the current ARView alone as a container for other UI elements; you have to use an UIView instead.

Open Object Library by selecting the + sign at the top, then search for the UIView. Drag and drop it onto the ARView view controller.

Open the Object Library again, then search for ARView. Drag and drop a RealityKit AR View into the design view.

Manually adjust the control points so that it fits over the entire screen. Then, under the Size Inspector, turn on both horizontal and vertical Autoresizing. This will keep the ARView full screen at all times.

Because you recreated the ARView from scratch, you need to reconnect it to the arView outlet in ViewController.swift.

Create a side-by-side view so that you can see Main.storyboard on one side and ViewController.swift on the other.

From the view controller side, click, drag and connect the arView outlet to RealityKit ARView.

The view will rename itself: Ar View.

Finally, you’re ready to add the buttons.

Adding buttons

Open the Object Library and search for Button. Drag and drop a Button into the design view.

Rename the button TankRight, then under the Attributes Inspector, select the TankRight image for it.

Under the Size inspector, set the Width and Height to 80. Under Autoresizing, anchor the button to the Bottom-Right.

Finally, move the button to the bottom-right of the design view.

Go into a side-by-side view and open ViewController.swift next to Main.storyboard. While holding down Control, click and drag from the TankRight button into the view controller.

Insert an Action named tankRightPressed. Select Connect to complete the new connection.

Once you’re done, you’ll see the newly-created @IBAction.

Now, when the user presses the TankRight button, it will trigger tankRightPressed() within the ViewController.

Follow the same steps as above and add the remaining buttons. Here’s a list of the remaining buttons.

The final result should look like this:

All six buttons are connected to their outlets in the ViewController. Congratulations, you’ve finished the UI.

Adding tank animations

Now that the UI is set, you need to make the tank move on command. To do that, you only need to invoke those behaviors you saw earlier, within the Reality Composer project.

Start with the tank movement. Add the following line to tankRightPressed(_:):

tankAnchor!.notifications.tankRight.post()

You can access any of the available behaviors through notifications. You invoke them by calling post() on the behavior.

Continue to add the other behaviors.

Add the following line of code to tankForwardPressed(_:):

tankAnchor!.notifications.tankForward.post()

And this code to tankLeftPressed(_:):

tankAnchor!.notifications.tankLeft.post()

Build and run to test it out.

Great, the tank moves when you press forward, left and right. But hang on, something is horribly wrong. The turret on top of the tank isn’t moving with it. What’s going on here?

Parenting entities

The problem you’re experiencing is the fact that the turret is currently a separate entity — it’s not connected to the tank. To connect the turret to the tank, you need to set the tank as the turret’s parent. That way, when the tank entity moves and rotates, its child entities will move and rotate with it.

Add the following line of code to viewDidLoad(), right after tankAnchor has been initialized:

tankAnchor!.turret?.setParent(
  tankAnchor!.tank, preservingWorldTransform: true)

This sets the tank entity as the turret entity’s parent. Preserving the World Transform while parenting the tank to the turret keeps the current position of the turret relative to the tank’s current position, as you see it in the scene.

Build and run again, and you’ll see you’ve attached the turret nicely to the tank.

Adding turret animations

Finally, you can now continue to add the turret behaviors to rotate it left and right — and don’t forget the all-important behavior to fire the gun.

Add the following line of code to turretRightPressed(_:):

tankAnchor!.notifications.turretRight.post()

Then, add the following to turretLeftPressed(_:):

tankAnchor!.notifications.turretLeft.post()

And to make the cannon fire, add the following line of code to cannonFirePressed(_:):

tankAnchor!.notifications.cannonFire.post()

That’s it, you now have full control over your tank. Build and run again to test it.

The tank moves on command, the turret stays connected to the tank and moves with it. The turret moves independently from the tank and fires off its cannon like a champ. But hang on, there’s yet another issue.

Final touches

When the user’s a bit trigger happy and presses the button too enthusiastically, the animation goes pear-shaped. When pressing a button while an animation is playing, the next animation is compounded on top of the currently-playing animation, which has some undesired side effects. This could lead to the turret being stuck in mid-air, for example.

The best way to solve this issue is to prevent any other animations from playing while an animation is currently active.

Add the following declaration to the top of ViewController:

var isActionPlaying: Bool = false

When an animation triggers, you’ll set isActionPlaying to true. Then, while it’s true, you can prevent any other animations from triggering.

To do this, add the following code to the top of all of the “button pressed event” handlers:

if self.isActionPlaying { return }
else { self.isActionPlaying = true }

Now, when the user presses a button to kick off an animation, it will first check to see if there’s an animation already in progress. If there is, it will ignore the button press. If not, it will set the flag to indicate that there is now an animation in progress, right before kicking of the actual animation.

You still need to reset isActionPlaying once the animation completes. To solve that, the behaviors have already been configured to send back a notification once the animation sequence completes. You just need to react to it.

Add the following code to viewDidLoad(), right after setting the tank as the turret’s parent:

tankAnchor?.actions.actionComplete.onAction = { _ in
  self.isActionPlaying = false
}

Each behavior will send an actionComplete notification once its animation sequence ends. Here, you’re creating a little handler that makes sure isActionPlaying resets to false, which allows another animation to play.

That’s it, you’re done. Build and run one last time and test out the glorious Tiny Toy Tank.

Now would be a good time to look away, because the little bear is about to get it! :]

Note: For completeness’ sake, there are a bunch of app icons you can use under starter/resources/AppIcons. Just drag and drop them into the AppIcon component of Assets.xcassets.

You can find a copy of the final project under the final/TinyToyTank folder.

Key points

Congratulations on finishing this chapter. You now know how quick and easy it is to create an AR experience with Xcode.

Take a look at some of the key points you picked up along the way:

• Xcode AR Templates: It’s super easy to create an AR experience with Xcode. It comes with a collection of AR templates out of the box, giving you a quick starting point for your AR projects.

• User Interface: You learned how to create a very basic user interface allowing the user to interact with the AR content. You can easily extend this with touch-based gestures or even voice activation for more intuitive interaction. Later, you’ll also learn how to create a SwiftUI-based UI.

• Reality Composer: In this project, you got a quick introduction to Reality Composer. You’ll learn more about using this extremely powerful tool in the coming chapters.

• RealityKit: You got a real quick introduction to RealityKit. Now, you shouldn’t be intimidated by the coding component of creating AR experiences. In the coming chapters, you’ll learn more about it with a few more project examples.

• 3D Virtual Content: In this chapter, you used ready-made 3D virtual content, but that won’t be the case when you’re working on your own projects from scratch. In the coming chapters, you’ll learn much more about content creation, which is an extremely important part of creating AR experiences.

Go take your Tiny Toy Tank for a spin and shoot at a few things. See you soon in the next chapter.