Building a Portal App in ARKit: Getting Started

This is an excerpt taken from Chapter 7, “Creating Your Portal”, of our book ARKit by Tutorials. This book show you how to build five immersive, great-looking AR apps in ARKit, Apple’s augmented reality framework. Enjoy!

Over this series of tutorials, you’ll implement a portal app using ARKit and SceneKit. Portal apps can be used for educational purposes, like a virtual tour of the solar system from space, or for more leisurely activities, like enjoying a virtual beach vacation.

The Portal App

The portal app you’ll be building lets you place a virtual doorway to a futuristic room, somewhere on a horizontal plane in the real world. You can walk in and out of this virtual room, and you can explore what’s inside.

In this tutorial, you’ll set up the basics for your portal app. By the end of the tutorial, you’ll know how to:

• Set up an ARSession
• Detect and render horizontal planes using ARKit

Are you ready to build a gateway into another world? Perfect!

Getting Started

In Xcode, open the starter project, Portal.xcodeproj. Build and run the project, and you’ll see a blank white screen.

Ah, yes, the blank canvas of opportunity!

Open Main.storyboard and expand the Portal View Controller Scene.

The PortalViewController is presented to the user when the app is launched. The PortalViewController contains an ARSCNView that displays the camera preview. It also contains two UILabels that provide instructions and feedback to the user.

Now, open PortalViewController.swift. In this file, you’ll see the following variables, which represent the elements in the storyboard:

// 1
@IBOutlet var sceneView: ARSCNView?
// 2
@IBOutlet weak var messageLabel: UILabel?
// 3
@IBOutlet weak var sessionStateLabel: UILabel?

Let’s take a look at what each one does:

1. sceneView is used to augment the camera view with 3D SceneKit objects.
2. messageLabel, which is a UILabel, will display instructional messages to the user. For example, telling them how to interact with your app.
3. sessionStateLabel, another UILabel, will inform the user about session interruptions, such as when the app goes into the background or if the ambient lighting is insufficient.

ARSCNView is a framework provided by Apple which you can use to easily integrate ARKit data with SceneKit. There are many benefits to using ARSCNView, which is why you’ll use it in this tutorial’s project.

In the starter project, you’ll also find a few utility classes in the Helpers group. You’ll be using these as you develop the app further.

Setting Up ARKit

The first step to setting things up is to capture the device’s video stream using the camera. For that, you’ll be using an ARSCNView object.

Open PortalViewController.swift and add the following method:

func runSession() {
  // 1  
  let configuration = ARWorldTrackingConfiguration.init()
  // 2
  configuration.planeDetection = .horizontal
  // 3
  configuration.isLightEstimationEnabled = true
  // 4
  sceneView?.session.run(configuration)

  // 5
  #if DEBUG
    sceneView?.debugOptions = [ARSCNDebugOptions.showFeaturePoints]
  #endif
}

Let’s take a look at what’s happening with this code:

Using ARSessionConfiguration is not recommended because it only accounts for the rotation of the device, not its position. For devices that use an A9 processor, ARWorldTrackingSessionConfiguration gives the best results, as it tracks all degrees of movement of the device.

1. You first instantiate an ARWorldTrackingConfiguration object. This defines the configuration for your ARSession. There are two types of configurations available for an ARSession: ARSessionConfiguration and ARWorldTrackingConfiguration.

   Using ARSessionConfiguration is not recommended because it only accounts for the rotation of the device, not its position. For devices that use an A9 processor, ARWorldTrackingSessionConfiguration gives the best results, as it tracks all degrees of movement of the device.

2. configuration.planeDetection is set to detect horizontal planes. The extent of the plane can change, and multiple planes can merge into one as the camera moves. It can find planes on any horizontal surface such as a floor, table or couch.
3. This enables light estimation calculations, which can be used by the rendering framework to make the virtual content look more realistic.
4. Start the session’s AR processing with the specified session configuration. This will start the ARKit session and video capturing from the camera, which is displayed in the sceneView.
5. For debug builds, this adds visible feature points; these are overlaid on the camera view.

Now it’s time to set up the defaults for the labels. Replace resetLabels() with the following:

func resetLabels() {
  messageLabel?.alpha = 1.0
  messageLabel?.text =
    "Move the phone around and allow the app to find a plane." +
    "You will see a yellow horizontal plane."
  sessionStateLabel?.alpha = 0.0
  sessionStateLabel?.text = ""    
}

This resets the opacity and text of messageLabel and sessionStateLabel. Remember, messageLabel is used to display instructions to the user, while sessionStateLabel is used to display any error messages, in the case something goes wrong.

Now, add runSession() to viewDidLoad() of PortalViewController:

override func viewDidLoad() {
  super.viewDidLoad()    
  resetLabels()
  runSession()
}

This will run the ARKit session when the app launches and loads the view.

Next, build and run the app. Don’t forget — you’ll need to grant camera permissions to the app.

ARSCNView does the heavy lifting of displaying the camera video capture. Because you’re in debug mode, you can also see the rendered feature points, which form a point cloud showing the intermediate results of scene analysis.

Plane Detection and Rendering

Previously, in runSession(), you set planeDetection to .horizontal, which means your app can detect horizontal planes. You can obtain the captured plane information in the delegate callback methods of the ARSCNViewDelegate protocol.

Start by extending PortalViewController so it implements the ARSCNViewDelegate protocol:

extension PortalViewController: ARSCNViewDelegate {

}

Add the following line to the very end of runSession():

sceneView?.delegate = self

This sets the ARSCNViewDelegate delegate property of the sceneView as the PortalViewController.

ARPlaneAnchors are added automatically to the ARSession anchors array, and ARSCNView automatically converts ARPlaneAnchor objects to SCNNode nodes.

Now, to render the planes, all you need to do is implement the delegate method in the ARSCNViewDelegate extension of PortalViewController:

// 1
func renderer(_ renderer: SCNSceneRenderer,
              didAdd node: SCNNode,
              for anchor: ARAnchor) {
  // 2
  DispatchQueue.main.async {
    // 3
    if let planeAnchor = anchor as? ARPlaneAnchor {
        // 4
      #if DEBUG
        // 5
        let debugPlaneNode = createPlaneNode(
          center: planeAnchor.center,
          extent: planeAnchor.extent)
        // 6  
        node.addChildNode(debugPlaneNode)
      #endif
      // 7
      self.messageLabel?.text =
      "Tap on the detected horizontal plane to place the portal"
    }
  }
}

Here’s what’s happening:

1. The delegate method, renderer(_:didAdd:for:), is called when ARSession detects a new plane, and the ARSCNView automatically adds an ARPlaneAnchor for the plane.
2. The callbacks occur on a background thread. Here, you dispatch the block to the main queue because any operations updating the UI should be done on the main UI thread.
3. You check to see if the ARAnchor that was added is an ARPlaneAnchor.
4. This checks to see if you’re in debug mode.
5. If so, create the plane SCNNode object by passing in the center and extent coordinates of the planeAnchor detected by ARKit. The createPlaneNode() is a helper method which you’ll implement shortly.
6. The node object is an empty SCNNode that’s automatically added to the scene by ARSCNView; its coordinates correspond to the ARAnchor’s position. Here, you add the debugPlaneNode as a child node, so that it gets placed in the same position as the node.
7. Finally, regardless of whether or not you’re in debug mode, you update the instructional message to the user to indicate that the app is now ready to place the portal into the scene.

Now it’s time to set up the helper methods.

Create a new Swift file named SCNNodeHelpers.swift. This file will contain all of the utility methods related to rendering SCNNode objects.

Import SceneKit into this file by adding the following line:

import SceneKit

Now, add the following helper method:

// 1
func createPlaneNode(center: vector_float3,
                     extent: vector_float3) -> SCNNode {
  // 2
  let plane = SCNPlane(width: CGFloat(extent.x),
                      height: CGFloat(extent.z))
  // 3
  let planeMaterial = SCNMaterial()
  planeMaterial.diffuse.contents = UIColor.yellow.withAlphaComponent(0.4)
  // 4
  plane.materials = [planeMaterial]
  // 5
  let planeNode = SCNNode(geometry: plane)
  // 6
  planeNode.position = SCNVector3Make(center.x, 0, center.z)
  // 7
  planeNode.transform = SCNMatrix4MakeRotation(-Float.pi / 2, 1, 0, 0)
  // 8
  return planeNode
}

Let’s go through this step-by-step:

1. The createPlaneNode method has two arguments: the center and extent of the plane to be rendered, both of type vector_float3. This type denotes the coordinates of the points. The function returns the SCNNode object created for the plane.
2. You instantiate the SCNPlane by specifying the width and height of the plane. You get the width from the x coordinate of the extent and the height from its z coordinate.
3. You initialize and assign the diffuse content for the SCNMaterial object. The diffuse layer color is set to a translucent yellow.
4. The SCNMaterial object is then added to the materials array of the plane. This defines the texture and color of the plane.
5. This creates an SCNNode with the geometry of the plane. The SCNPlane inherits from the SCNGeometry class, which only provides the form of a visible object rendered by SceneKit. You specify the position and orientation of the geometry by attaching it to an SCNNode object. Multiple nodes can reference the same geometry object, allowing it to appear at different positions in a scene.
6. You set the position of the planeNode. Note that the node is translated to coordinates (center.x, 0, center.z) reported by ARKit via the ARPlaneAnchor instance.
7. Planes in SceneKit are vertical by default, so you need to rotate the plane by 90 degrees in order to make it horizontal.
8. This returns the planeNode object created in the previous steps.

Build and run the app. If ARKit is able to detect a suitable surface in your camera view, you’ll see a yellow horizontal plane.

Move the device around and you’ll notice the app sometimes shows multiple planes. As it finds more planes, it adds them to the view. Existing planes, however, do not update or change size as ARKit analyzes more features in the scene.

ARKit constantly updates the plane’s position and extents based on new feature points it finds. To receive these updates in your app, add the following renderer(_:didUpdate:for:) delegate method to PortalViewController.swift:

// 1
func renderer(_ renderer: SCNSceneRenderer,
              didUpdate node: SCNNode,
              for anchor: ARAnchor) {
  // 2              
  DispatchQueue.main.async {
    // 3
    if let planeAnchor = anchor as? ARPlaneAnchor,
      node.childNodes.count > 0 {
      // 4  
      updatePlaneNode(node.childNodes[0],
                      center: planeAnchor.center,
                      extent: planeAnchor.extent)
    }
  }
}

Here’s what’s happening:

1. renderer(_:didUpdate:for:) is called when the corresponding ARAnchor updates.
2. Operations that update the UI should be executed on the main UI thread.
3. Check that the ARAnchor is an ARPlaneAnchor and make sure it has at least one child node that corresponds to the plane’s SCNNode.
4. updatePlaneNode(_:center:extent:) is a method that you’ll implement shortly. It updates the coordinates and size of the plane to the updated values contained in the ARPlaneAnchor.

Open SCNNodeHelpers.swift and add the following code:

func updatePlaneNode(_ node: SCNNode,
                     center: vector_float3,
                     extent: vector_float3) {
  // 1                    
  let geometry = node.geometry as? SCNPlane
  // 2
  geometry?.width = CGFloat(extent.x)
  geometry?.height = CGFloat(extent.z)
  // 3
  node.position = SCNVector3Make(center.x, 0, center.z)
}

Going through this code step-by-step:

1. Check if the node has SCNPlane geometry.
2. Update the node geometry using the new values that are passed in. Use the extent or size of the ARPlaneAnchor to update the width and height of the plane.
3. Update the position of the plane node with the new position.

Now that you can successfully update the position of the plane, build and run the app. You’ll see that the plane’s size and position shifts as it detects new feature points.

There’s still one problem that needs to be solved. Once the app detects the plane, if you exit the app and come back in, you’ll see that the previously detected plane is now on top of other objects within the camera view; it no longer matches the plane surface it previously detected.

To fix this, you need to remove the plane node whenever the ARSession is interrupted. You’ll handle that in the next tutorial.