Augmented Reality and ARKit Tutorial

Respond to Session Events

ARSKView’s session has delegate methods for certain events. For example, you’ll want to know if the session failed. Perhaps the user denied access to the camera, or she could be running the game on a device that doesn’t support AR. You need to address these issues.

In GameViewController.swift, add an extension for the delegate methods with placeholder error messages:

extension GameViewController: ARSKViewDelegate {
  func session(_ session: ARSession,
               didFailWithError error: Error) {
  print("Session Failed - probably due to lack of camera access")
}
  
func sessionWasInterrupted(_ session: ARSession) {
  print("Session interrupted")
}
  
func sessionInterruptionEnded(_ session: ARSession) {
  print("Session resumed")
  sceneView.session.run(session.configuration!,
                        options: [.resetTracking, 
                                  .removeExistingAnchors])
 }                                  
}

• session(_:didFailWithError:): will execute when the view can’t create a session. This generally means that to be able to use the game, the user will have to allow access to the camera through the Settings app. This is a good spot to display an appropriate dialog.
• sessionWasInterrupted(_:): means that the app is now in the background. The user may have pressed the home button or received a phone call.
• sessionInterruptionEnded(_:): means that play is back on again. The camera won’t be in exactly the same orientation or position so you reset tracking and anchors. In the challenge at the end of the tutorial, you’ll restart the game.

Next, replace viewDidLoad() with this code:

override func viewDidLoad() {
  super.viewDidLoad()
  if let view = self.view as? ARSKView {
    sceneView = view
    sceneView!.delegate = self
    let scene = GameScene(size: view.bounds.size)
    scene.scaleMode = .resizeFill
    scene.anchorPoint = CGPoint(x: 0.5, y: 0.5)
    view.presentScene(scene)
    view.showsFPS = true
    view.showsNodeCount = true
  }
}

Here you set the view’s delegate and initialize GameScene directly, instead of through the scene file.

The Current Frame, Camera and Anchors

You’ve now set up your ARSKView with a session so the camera information will render into the view.

For every frame, the session will capture the image and tracking information into an ARFrame object, named currentFrame. This ARFrame object has a camera which holds positional information about the frame, along with a list of anchors.

These anchors are stationary tracked positions within the scene. Whenever you add an anchor, the scene view will execute a delegate method view(_:nodeFor:) and attach an SKNode to the anchor. When you add the game’s bug nodes to the scene, you’ll attach the bug nodes to these anchors.

Adding Bugs to the Scene

Now you’re ready to add bugs to the game scene.

First, you’ll remove all the template code. Delete GameScene.sks and Actions.sks since you won’t need them anymore.

In GameScene.swift, remove all the code in GameScene, leaving you with an empty class.

class GameScene: SKScene {
}

Replace the imports at the top with:

import ARKit

To get acquainted with ARKit, and possibly help with your entomophobia, you’re going to place a bug just in front of you.

Create a convenience property to return the scene’s view as an ARSKView:

var sceneView: ARSKView {
  return view as! ARSKView
}

Before you add the bug to your AR world, you need to make sure the AR session is ready. An AR session takes time to set up and configure itself.

First, create a property so you can check whether you added your AR nodes to the game world:

var isWorldSetUp = false

You’ll load the bug once — only if isWorldSetUp is false.

Add the following method:

private func setUpWorld() {
  guard let currentFrame = sceneView.session.currentFrame
    else { return }
  
  isWorldSetUp = true
}

Here you check whether the session has an initialized currentFrame. If the session doesn’t have a currentFrame, then you’ll have to try again later.

update(_:) is called every frame, so you can attempt to call the method inside there.

Override update(_:):

override func update(_ currentTime: TimeInterval) {
  if !isWorldSetUp {
    setUpWorld()
  }
}

Doing it this way, you only run the set up code once, and only when the session is ready.

Next you’ll create an anchor 0.3 meters in front of the camera, and you’ll attach a bug to this anchor in the view’s delegate.

But first there’s some math-y stuff to explain. Don’t worry — I’ll be gentle!

A Brief Introduction to 3D Math

Even though your game is a 2D SpriteKit game, you’re looking through the camera into a 3D world. Instead of setting position and rotation properties of an object, you set values in a 3D transformation matrix.

You may be a Neo-phyte to 3D (no more Matrix jokes, I promise!), so I’ll briefly explain.

A matrix is made up of rows and columns. You’ll be using four-dimensional matrices, which have four rows and four columns.

Both ARCamera and ARAnchor have a transform property, which is a four-dimensional matrix holding rotation, scaling and translation information. The current frame calculates ARCamera’s transform property, but you’ll adjust the translation element of the ARAnchor matrix directly.

The magic of matrices — and why they are used everywhere in 3D — is that you can create a transform matrix with rotation, scaling and translation information and multiply it by another matrix with different information. The result is then a new position in 3D space relative to an origin position.

Add this after the guard statement in setUpWorld():

var translation = matrix_identity_float4x4

Here you create a four-dimensional identity matrix. When you multiply any matrix by an identity matrix, the result is the same matrix. For example, when you multiply any number by 1, the result is the same number. 1 is actually a one-dimensional identity matrix. The origin’s transform matrix is an identity matrix. So you always set a matrix to identity before adding positional information to it.

This is what the identity matrix looks like:

The last column consists of (x, y, z, 1) and is where you set translation values.

Add this to setUpWorld(), right after the previous line:

translation.columns.3.z = -0.3

This is what the translation matrix looks like now:

Rotation and scaling use the first three columns and are more complex. They’re beyond the scope of this tutorial, and in ARniegeddon you won’t need them.

Continue adding to your code in setUpWorld():

let transform = currentFrame.camera.transform * translation

Here you multiply the transform matrix of the current frame’s camera by your translation matrix. This results in a new transform matrix. When you create an anchor using this new matrix, ARKit will place the anchor at the correct position in 3D space relative to the camera.

Now, add the anchor to the scene using the new transformation matrix. Continue adding this code to setUpWorld():

let anchor = ARAnchor(transform: transform)
sceneView.session.add(anchor: anchor)

Here you add an anchor to the session. The anchor is now a permanent feature in your 3D world (until you remove it). Each frame tracks this anchor and recalculates the transformation matrices of the anchors and the camera using the device’s new position and orientation.

When you add an anchor, the session calls sceneView’s delegate method view(_:nodeFor:) to find out what sort of SKNode you want to attach to this anchor.

Next, you’re going to attach a bug.

In GameViewController.swift, add this delegate method to GameViewController’s ARSKViewDelegate extension:

func view(_ view: ARSKView,
          nodeFor anchor: ARAnchor) -> SKNode? {
  let bug = SKSpriteNode(imageNamed: "bug")
  bug.name = "bug"
  return bug
}

Hold up your phone and build and run. And I mean… RUN! Your space has been invaded!

Move your phone around, and the bug will stay where it’s anchored. The tracking becomes more effective the more information you give the phone, so move your phone around a bit to give it some position and orientation updates.

Notice that whichever way you turn the camera, the bug faces you. This is called a billboard, which is a technique used in many 3D games as a cheap way of adding elements such as trees and grass to a scene. Simply add a 2D object to a 3D scene and make sure that it’s always facing the viewer.

If you want to be able to walk around the bug and see what it looks like from behind, you’ll have to model the bug in a 3D app such as Blender, and use ARKit with either SceneKit or Metal.