Augmented Reality’s RoomPlan for iOS: Getting Started

Taking a Look at a Scan Result

Before exporting the model, look at what the result of a scan looks like.

Scanning a room with RoomCaptureView creates a CapturedRoom. This object encapsulates various information about the room. It contains two different types of room-defining elements: Surface and Object.

Surface is a 2D area recognized in the scanned room. A surface can be:

• A wall
• An opening
• A window
• An opened or closed door

An Object is a 3D area. There are a lot of object categories:

• Storage area
• Refrigerator
• Stove
• Bed
• Sink
• Washer or dryer
• Toilet
• Bathtube
• Oven
• Dishwasher
• Table
• Sofa
• Chair
• Fireplace
• Television
• Stairs

That’s a pretty extensive list, right? Additionally, both surfaces and objects have a confidence value, which can either be low, medium or high. They also have a bounding box called dimensions. Another common property is a matrix that defines position and orientation called transform.

How Can We Access Room Data?

You may wonder what you can do with the resulting room data! RoomPlan makes it easy to export the depth and complex scanning result as a USDZ file.

USDZ is an addition to Pixars Universal Scene Description file format, USD in short. This file format describes 3D scenes and allows users to collaboratively work on them across different 3D programs. USDZ is a package file combining USD files, images, textures and audio files.

To learn more about USD and USDZ, check out Pixars Introduction to USD and Apple’s documentation about USDZ.

Once you export your room model as a USDZ file, you’ll be able to open, view and edit the file in other 3D applications like Apple’s AR Quick Look.

Exporting your Room Data

Now it’s time for you to export your room model. All you need to do is call export(to:exportOptions:) on the captured room.

Still in RoomCaptureViewController.swift replace the empty body of export with:

do {
  // 1
  try capturedRoom?.export(to: viewModel.exportUrl)
} catch {
  // 2
  print("Error exporting usdz scan: \(error)")
  return
}
// 3
viewModel.showShareSheet = true

Here’s what’s happening:

1. Exporting the model is as easy as calling export(to:exportOptions:) on the captured room. You can export the model either as polygons or as a mesh. You don’t define custom export options here, so it’s exported as a mesh by default.
2. Like any other file operation, exporting the model can fail. In a real app, you would try to handle the error more gracefully and show some information to the user. But in this example, printing the error to the console is fine.
3. Finally, you inform the view model that the app needs to show a share sheet to allow the user to select where to send the exported USDZ file.

Build and run. Scan your room, and you’ll see the export button again. Tap it, and this time you’ll see a share sheet allowing you to export the 3D model of your room.

Now that you’re an expert in using the scanning experience API in the form of RoomCaptureView, it’s time to look at the more advanced data API.

Advanced Scanning With the Data API

RoomCaptureView is pretty impressive. But unfortunately, it doesn’t solve your problem of potentially dangerous boxes lying around on the floor. :] For that, you need more customization options. That’s where the second way of using RoomPlan comes into play: the data API.

Open CustomCaptureView.swift. Like RoomCaptureViewController.swift, this file already contains a bunch of code. CustomCaptureView is a custom ARView, different than CustomARView that you saw earlier. You’ll use RoomPlan to add context to the scene. Important parts are missing, and you’ll create the missing pieces in this section of the tutorial.

Again, the first step is to start the room capture session.

Start by adding these two properties below viewModel:

private let captureSession = RoomCaptureSession()
private var capturedRoom: CapturedRoom?

captureSession is the session used for scanning the room and capturedRoom stores the result.

Next, add this line to the body of startSession:

captureSession.run(configuration: RoomCaptureSession.Configuration())

Just like before, this starts the session with a default configuration.

Setting up Delegate Callbacks

The next step is to prepare placing blocks whenever an updated room model is available. To do so, add these two lines of code at the beginning of setup:

captureSession.delegate = self
self.session = captureSession.arSession

This informs the captureSession that CustomCaptureView acts as its delegate. Now it needs to conform to that delegate protocol. Add the following code above CustomCaptureViewRepresentable:

extension CustomCaptureView: RoomCaptureSessionDelegate {
  // 1
  func captureSession(_ session: RoomCaptureSession, didUpdate: CapturedRoom) {
    // 2
    capturedRoom = didUpdate
    // 3
    DispatchQueue.main.async {
      self.viewModel.canPlaceBlock = didUpdate.objects.contains { 
        $0.category == .table 
      }
    }
  }
}

This is what’s going on:

1. You implement the delegate method to get updates on the scanned room just like earlier.
2. You store the new room in the property capturedRoom.
3. If there are tables in the list of objects of the updated room, you change the view model’s property canPlaceBlock. This makes the place block button appear.

Build and run. This time tap the navigation option Custom Capture Session at the bottom of the list. Once you start scanning a room and the session recognizes a table, the place block button appears. It doesn’t do anything yet, that’s what you’ll change next.

Other Capture Session Delegate Methods

Again, you’re only using the delegate method captureSession(_:didUpdate:) of RoomCaptureSessionDelegate. That’s because it informs you of all updates to the captured room. But there are more methods available that provide a more fine-granular control.

For updates on surfaces and objects, you can implement three different methods:

1. captureSession(_:didAdd:): This notifies the delegate about newly added surfaces and objects.
2. captureSession(_:didChange:): Informs about changes to dimension, position or orientation.
3. captureSession(_:didRemove:): Notifies when the session removes a surface or object.

The next delegate method is captureSession(_:didProvide:). RoomCaptureSession calls this one whenever new instructions and recommendations are available to show the user. These instructions are part of the enum RoomCaptureSession.Instruction and contain hints like moveCloseToWall and turnOnLight. You can implement this method to show your own instruction view, similar to the one RoomCaptureView shows.

Finally, there are captureSession(_:didStartWith:) and captureSession(_:didEndWith:error:) delegate methods. They notify you about the start and end of a scan.

All of these delegate methods have an empty default implementation, so they are optional.