How To Make An App Like Pokemon Go

Adding the Camera Preview

Open ViewController.swift, and import AVFoundation after the import of SceneKit

import UIKit
import SceneKit
import AVFoundation
    
class ViewController: UIViewController {
...

and add the following properties to store an AVCaptureSession and an AVCaptureVideoPreviewLayer:

var cameraSession: AVCaptureSession?
var cameraLayer: AVCaptureVideoPreviewLayer?

You use a capture session to connect a video input, such as the camera, and an output, such as the preview layer.

Now add the following method:

func createCaptureSession() -> (session: AVCaptureSession?, error: NSError?) {
  //1
  var error: NSError?
  var captureSession: AVCaptureSession?
    
  //2
  let backVideoDevice = AVCaptureDevice.defaultDevice(withDeviceType: .builtInWideAngleCamera, mediaType: AVMediaTypeVideo, position: .back)
    
  //3
  if backVideoDevice != nil {
    var videoInput: AVCaptureDeviceInput!
    do {
      videoInput = try AVCaptureDeviceInput(device: backVideoDevice)
    } catch let error1 as NSError {
      error = error1
      videoInput = nil
    }
      
    //4
    if error == nil {
      captureSession = AVCaptureSession()
        
      //5
      if captureSession!.canAddInput(videoInput) {
        captureSession!.addInput(videoInput)
      } else {
        error = NSError(domain: "", code: 0, userInfo: ["description": "Error adding video input."])
      }
    } else {
      error = NSError(domain: "", code: 1, userInfo: ["description": "Error creating capture device input."])
    }
  } else {
    error = NSError(domain: "", code: 2, userInfo: ["description": "Back video device not found."])
  }
    
  //6
  return (session: captureSession, error: error)
}

Here’s what the method above does:

1. Create some variables for the return value of the method.
2. Get the rear camera of the device.
3. If the camera exists, get it's input.
4. Create an instance of AVCaptureSession.
5. Add the video device as an input.
6. Return a tuple that contains either the captureSession or an error.

Now that you have the input from the camera, you can load it into your view:

func loadCamera() {
  //1
  let captureSessionResult = createCaptureSession()
   
  //2  
  guard captureSessionResult.error == nil, let session = captureSessionResult.session else {
    print("Error creating capture session.")
    return
  }
    
  //3
  self.cameraSession = session
    
  //4
  if let cameraLayer = AVCaptureVideoPreviewLayer(session: self.cameraSession) {
    cameraLayer.videoGravity = AVLayerVideoGravityResizeAspectFill
    cameraLayer.frame = self.view.bounds
    //5
    self.view.layer.insertSublayer(cameraLayer, at: 0)
    self.cameraLayer = cameraLayer
  }
}

Taking the above method step-by-step:

1. First, you call the method you created above to get a capture session.
2. If there was an error, or captureSession is nil, you return. Bye-bye augmented reality.
3. If everything was fine, you store the capture session in cameraSession.
4. This line tries to create a video preview layer; if successful, it sets videoGravity and sets the frame of the layer to the views bounds. This gives you a fullscreen preview.
5. Finally, you add the layer as a sublayer and store it in cameraLayer.

Now add the following to viewDidLoad():

  loadCamera()
  self.cameraSession?.startRunning()

Really just two things going on here: first you call all the glorious code you just wrote, then start grabbing frames from the camera. The frames are displayed automatically on the preview layer.

Build and run your project, then tap a location near you and enjoy the new camera preview:

Adding a Cube

A preview is nice, but it’s not really augmented reality — yet. In this section, you’ll add a simple cube for an enemy and move it depending on the user’s location and heading.

This small game has two kind of enemies: wolves and dragons. Therefore, you need to know what kind of enemy you’re facing and where to display it.

Add the following property to ViewController (this will help you store information about the enemies in a bit):

var target: ARItem!

Now open MapViewController.swift, find mapView(_:, didSelect:) and change the last if statement to look like the following:

if let mapAnnotation = view.annotation as? MapAnnotation {
  //1
  viewController.target = mapAnnotation.item

  self.present(viewController, animated: true, completion: nil)
}

• Before you present viewController you store a reference to the ARItem of the tapped annotation. So viewController knows what kind of enemy you're facing.

Now ViewController has everything it needs to know about the target.

Open ARItem.swift and import SceneKit.

import Foundation
import SceneKit

struct ARItem {
...
}

Next, add the following property to store a SCNNode for an item:

var itemNode: SCNNode?

Be sure to define this property after the ARItem structure’s existing properties, since you will be relying on the implicit initializer to define arguments in the same order.

Now Xcode displays an error in MapViewController.swift. To fix that, open the file and scroll to setupLocations().

Change the lines Xcode marked with a red dot on the left of the editor pane.

In each line, you’ll add the missing itemNode argument as a nil value.

As an example, change the line below:

let firstTarget = ARItem(itemDescription: "wolf", location: CLLocation(latitude: 50.5184, longitude: 8.3902))

...to the following:

let firstTarget = ARItem(itemDescription: "wolf", location: CLLocation(latitude: 50.5184, longitude: 8.3902), itemNode: nil)

You know the type of enemy to display, and what it’s position is, but you don't yet know the direction of the device.

Open ViewController.swift and import CoreLocation, your imports should look like this now.

import UIKit
import SceneKit
import AVFoundation
import CoreLocation

Next, add the following properties:

//1
var locationManager = CLLocationManager()
var heading: Double = 0
var userLocation = CLLocation()
//2
let scene = SCNScene()
let cameraNode = SCNNode()
let targetNode = SCNNode(geometry: SCNBox(width: 1, height: 1, length: 1, chamferRadius: 0))

Here’s the play-by-play:

1. You use a CLLocationManager to receive the heading the device is looking. Heading is measured in degrees from either true north or the magnetic north pole.
2. This creates an empty SCNScene and SCNNode. targetNode is a SCNNode containing a cube.

Add the following to the bottom of viewDidLoad():

//1
self.locationManager.delegate = self
//2
self.locationManager.startUpdatingHeading()
    
//3
sceneView.scene = scene  
cameraNode.camera = SCNCamera()
cameraNode.position = SCNVector3(x: 0, y: 0, z: 10)
scene.rootNode.addChildNode(cameraNode)

This is fairly straightforward code:

1. This sets ViewController as the delegate for the CLLocationManager.
2. After this call, you’ll have the heading information. By default, the delegate is informed when the heading changes more than 1 degree.
3. This is some setup code for the SCNView. It creates an empty scene and adds a camera.

To adopt the CLLocationManagerDelegate protocol, add the following extension to ViewController

extension ViewController: CLLocationManagerDelegate {
  func locationManager(_ manager: CLLocationManager, didUpdateHeading newHeading: CLHeading) {
    //1
    self.heading = fmod(newHeading.trueHeading, 360.0)
    repositionTarget()
  }
}

CLLocationManager calls this delegate method each time new heading information is available. fmod is the modulo function for double values, and assures that heading is in the range of 0 to 359.

Now add repostionTarget() to ViewController.swift, but inside the normal implementation and not inside the CLLocationManagerDelegate extension:

func repositionTarget() {
  //1
  let heading = getHeadingForDirectionFromCoordinate(from: userLocation, to: target.location)
    
  //2
  let delta = heading - self.heading
    
  if delta < -15.0 {
    leftIndicator.isHidden = false
    rightIndicator.isHidden = true
  } else if delta > 15 {
    leftIndicator.isHidden = true
    rightIndicator.isHidden = false
  } else {
    leftIndicator.isHidden = true
    rightIndicator.isHidden = true
  }
    
  //3
  let distance = userLocation.distance(from: target.location)
    
  //4
  if let node = target.itemNode {
    
    //5
    if node.parent == nil {
      node.position = SCNVector3(x: Float(delta), y: 0, z: Float(-distance))
      scene.rootNode.addChildNode(node)
    } else {
      //6
      node.removeAllActions()
      node.runAction(SCNAction.move(to: SCNVector3(x: Float(delta), y: 0, z: Float(-distance)), duration: 0.2))
    }
  }
}

Here’s what each commented section does:

1. You will implement this method in the next step, but this basically calculates the heading from the current location to the target.
2. Then you calculate a delta value of the device’s current heading and the location’s heading. If the delta is less than -15, display the left indicator label. If it is greater than 15, display the right indicator label. If it’s between -15 and 15, hide both as the the enemy should be onscreen.
3. Here you get the distance from the device’s position to the enemy.
4. If the item has a node assigned...
5. and the node has no parent, you set the position using the distance and add the node to the scene.
6. Otherwise, you remove all actions and create a new action.

If you are familiar with SceneKit or SpriteKit the last line should be no problem. If not, here is a more detailed explanation.

SCNAction.move(to:, duration:) creates an action that moves a node to the given position in the given duration. runAction(_:) is a method of SCNNode and executes an action. You can also create groups and/or sequences of actions. Our book 3D Apple Games by Tutorials is a good resource for learning more.

Now to implement the missing method. Add the following methods to ViewController.swift:

func radiansToDegrees(_ radians: Double) -> Double {
  return (radians) * (180.0 / M_PI)
}
  
func degreesToRadians(_ degrees: Double) -> Double {
  return (degrees) * (M_PI / 180.0)
}
  
func getHeadingForDirectionFromCoordinate(from: CLLocation, to: CLLocation) -> Double {
  //1
  let fLat = degreesToRadians(from.coordinate.latitude)
  let fLng = degreesToRadians(from.coordinate.longitude)
  let tLat = degreesToRadians(to.coordinate.latitude)
  let tLng = degreesToRadians(to.coordinate.longitude)
    
  //2
  let degree = radiansToDegrees(atan2(sin(tLng-fLng)*cos(tLat), cos(fLat)*sin(tLat)-sin(fLat)*cos(tLat)*cos(tLng-fLng)))
    
  //3
  if degree >= 0 {
    return degree
  } else {
    return degree + 360
  }
}

radiansToDegrees(_:) and degreesToRadians(_:) are simply two helper methods to convert values between radians and degrees.

Here’s what’s going on in getHeadingForDirectionFromCoordinate(from:to:):

1. First, you convert all values for latitude and longitude to radians.
2. With these values, you calculate the heading and convert it back to degrees.
3. If the value is negative, normalize it by adding 360 degrees. This is no problem, since -90 degrees is the same as 270 degree.

There are two small steps left before you can see your work in action.

First, you'll need to pass the user's location along to viewController. Open MapViewController.swift and find the last if statement inside mapView(_:, didSelect:) and add the following line right before you present the view controller;

viewController.userLocation = mapView.userLocation.location!

Now add the following method to ViewController.swift:

func setupTarget() {
  targetNode.name = "enemy"
  self.target.itemNode = targetNode    
}

Here you simply give targetNode a name and assign it to the target. Now you can call this method at the end of viewDidLoad(), just after you add the camera node:

scene.rootNode.addChildNode(cameraNode)
setupTarget()

Build and run your project; watch your not-exactly-menacing cube move around: