3. Reality Composer & Reality Files

Written by Chris Language

Now that you know all about AR Quick Look, it’s time to dip your toes a little deeper into the shallow end of Augmented Reality (AR). In this chapter, you’ll learn about Reality Composer and Reality Files.

Until recently, creating immersive AR experiences was a somewhat difficult task. AR developers required a vast amount of skill, as well as knowledge of a wide variety of technologies, just to make a little cube appear in AR. That all changed when Apple announced Reality Composer.

What is Reality Composer?

Reality Composer is an AR authoring tool that lets you create interactive AR-based experiences with an intuitive WYSIWYG (What You See Is What You Get) design. It’s self-explanatory and super easy to use. You don’t need any prior coding or 3D development experience to use it.

Any AR Quick Look-compatible app, including iMessage, Safari, Files and even apps you build yourself, can then view the AR experiences you create.

Reality Composer is fully integrated into Xcode, which allows you to easily extend and customize your app’s AR experiences with the power of Xcode and Swift.

If you’re using an iPhone or iPad, you can download and install Reality Composer directly from the App Store using this link: https://apple.co/2RfDvt3

Reality Composer on an iPhone:

Although the interface is extremely compact, the app itself is fully functional, keeping you productive while you’re on the go. The best part of using an iPhone is that it allows you to edit your 3D scenes directly in AR space. You experience the end result first hand – no more playing in the dark!

Reality Composer on an iPad:

The iPad’s interface is similar to the iPhone’s interface, but with a little more breathing room.

There’s also a macOS version that installs automatically with the latest versions of Xcode.

Reality Composer on macOS:

You’ll notice subtle interface differences between the iPhone, iPad and macOS versions, but rest assured that the underlying functionality is exactly the same. However, the macOS version doesn’t allow editing in AR space, mainly due to the lack of a rear-facing camera on most Macs.

Overall, Apple did a fantastic job at delivering an experience across multiple platforms that is consistent and pleasant.

Note: At the time of writing, you can only install Reality Composer for macOS through Xcode. There’s no App Store version available for download yet.

Reality Composer features

Out of the box, Reality Composer is quite impressive and includes the following features:

• Basic Transform Editing: With the built-in scene editor, you can easily drag, drop, rotate and scale virtual objects to construct AR scenes.

• Edit in Augmented Space: You can transfer scenes between macOS and iOS. This allows you to test and edit virtual scenes either on-screen or directly in augmented space.

• Built-in Content Library: Reality Composer comes with its own content library of virtual objects that should cover most of your AR prototyping needs. There’s even a collection of primitive objects, like cubes, spheres, springs and spirals, that you can use to build your own creations.

• USDZ File Support: If you want a form that’s not available in the built-in content library, you can create your own content and import it via a USDZ file.

• Animations: Bring your virtual content to life with animations that can make them wiggle and spin or put emphasis on them to attract the user’s attention.

• Spatial Audio: You can easily add ambient sounds or special effects to your scene and virtual content. Spatial audio accurately simulates sounds emitted from objects based on their position in space.

• Exporting: You can either export directly to AR Quick Look or create an integrated project with Xcode where you can use code to control your virtual content.

• Recording Sensor and Camera Data: Pre-record sounds in an actual location, then play the recording on a simulated iOS device to test and debug the experience in Xcode. This is fantastic if you’re creating an AR experience for a fixed location.

Reality Composer’s limitations

Reality Composer is not perfect, and there are some limitations you need to understand before you make it your first choice for creating AR experiences. Here are a few important things to keep in mind:

• Cross-Platform Support: The biggest limitation you’ll face is that Reality Composer restricts your AR experiences to Apple devices only. Other platforms, like Android or Windows, do not support Reality Composer.

• iOS Support: The latest features of AR Quick Look are only available on iOS 13 or newer. You might run into serious constraints when using a device with an older iOS version on it.

• Scripting Support: There’s a lack of custom scripting support within AR Quick Look. If you want to create a complex AR experience, you’ll have to create an Xcode app that integrates with Reality Composer instead.

Note: Before you continue, make sure you’ve installed the latest version of Xcode. This will ensure that Reality Composer is ready to go on your Mac.

You should also install Reality Composer on your iPhone or iPad, so you can test on a device later.

Also, ensure that all of your devices are on the same network, which will make them discoverable.

Creating Reality Composer projects

With the basics out of the way, it’s time to get those hands dirty and create your first AR experience with Reality Composer. To create a new project, all you need to do is start Reality Composer.

Use Spotlight Search to find and start the Reality Composer app that Xcode installed for you.

On start-up, Reality Composer creates a default scene for your project. Each scene has a default anchor, which forms the root position of the scene in AR space. You can anchor your AR content onto a floor, a table, a wall, an image or even a face. You’ll learn more about anchors a little later.

For now, select Horizontal and make sure you’ve checked Use template content content. Click Choose to continue.

This creates the project and adds some default content. But at this point, you still haven’t saved the project.

Press Command-S to open the Save Project dialog. You can also use the File ▸ Save… menu option to accomplish the same result.

Note: It’s good practice to save your project periodically, so you don’t lose any progress.

Set the project name to HelloRealityKit, then select a more suitable destination of your choice. Select Save to finish.

Excellent, you’ve successfully created and saved your first Reality Composer project.

Note: The project is available for you as a starter project under the starter folder. Just double-click HelloRealityKit.rcproject to open it in Reality Composer.

Exploring the UI

With the HelloRealityKit project open, you’re ready to explore the User Interface (UI).

The UI consists of a few main components. Here’s what each component does:

1. Main menu: Here, you’ll find a typical menu with access to all available options.

2. Toolbar: The toolbar at the top of the screen contains the most commonly-used options. The shortcuts are accessible through the Main menu, but the toolbar makes them more convenient. You can customize the toolbar, so it matches your liking. Toggle its visibility by pressing Option-Command-T.

3. Scenes: The Scenes panel is on the left and provides quick access to the available scenes in the project. Simply click on the desired scene to switch to it. You can toggle its visibility by pressing Option-Command-S.

4. Properties: The Properties panel on the right shows the available attributes of the selected object. It’s context-sensitive, so the properties apply to only the active selected object. Toggle its visibility by pressing Option-Command-P.

5. Scene view: The Scene view forms the center point of the UI. It’s the main 3D view, where you add and manipulate virtual content and can see a live preview of your changes. You can also use the visual guides to directly transform and manipulate selected elements, changing their position, scale and rotation.

Adding scenes

Your project can contain more than one scene. Adding a new scene is as easy as clicking the + button in the top-right corner of the Scenes panel on the left. You could also use the Main menu by going to Scene ▸ Add Scene….

Remember that a scene is associated with an Anchor, so you need to choose the anchor type of the new scene. Choose Horizontal again and uncheck Use template content. Click Choose to continue.

Now that you’ve created a new empty scene, the project contains two scenes: Scene and Scene 1. You can easily switch between the two by clicking on the scene that you want active in the Scenes panel on the right. The contents of the Scene View will change accordingly.

Select Scene 1, then continue.

Navigating scenes

To navigate around the scene, you can manipulate the view in three ways: Zoom, Pan and Rotate.

1. Zoom: To zoom in and out, hold down the Option key, then slide your finger backward and forward.

2. Pan: To pan around, slide your finger in any direction; the scene view will follow.

3. Rotate: To rotate the scene, press and hold the left mouse button while moving the mouse around.

Note: The methods of zooming, panning and rotating mentioned above assume that you’re using an Apple Magic Mouse or touchpad.

Adding objects

An empty scene isn’t that exciting, so your next step is to spruce things up a bit by adding some objects to it.

Open the Content Library by clicking the Objects button with a + sign in it from the toolbar at the top of the screen.

The Content Library contains a library of various shapes and 3D objects that you can use to prototype your AR scenes.

The basic shapes section is special. These are all parametric shapes, meaning that you can alter them by adjusting their parameters — more on that a little later.

Double-click the Cube from Basic ▸ Shapes to add it to the active scene.

Left-click the cube to select it.

You’ll notice a circle appears around the box with three colored arrows.

Each color represents a specific axis in 3D space:

• Red: This is the X-axis, which moves from left to right in 3D space.
• Green: This is the Y-axis, which goes from bottom to top in 3D space.
• Blue: This is the Z-axis, running from front to back in 3D space.

Right-click the cube to show contextual options.

You can easily replace, modify, cut, copy, paste, duplicate and delete objects from here. You can also use shortcut keys like backspace to delete, Command-Z to undo, Command-C to copy and Command-P to paste objects work.

Adjusting object transforms

With the 3D cube still selected, adjust its transform, which consists of three components: position, rotation and scale.

Clicking and dragging any of the arrows will move the object’s position in that direction, restricting it to that selected axis.

When you rotate around the scene view, the circle color will change depending on the direction you’re viewing it from.

Click and drag the circle to rotate the object around the specified axis.

For more precise control over the object transform, you can adjust the settings from the Properties panel.

Adjusting object properties

With the cube still selected, open the Properties panel by clicking the Properties button in the toolbar.

This area shows all of the available properties associated with the cube.

Change Object Name to Cube.

Expand the Transform section on the Properties panel, then adjust the properties to match the following settings:

• Set Position to (X: 0, Y: 15, Z: 0)
• Set Rotation to (X: 30°, Y: 0°, Z: 30°)
• Set Scale to 50%

Using these properties, you can adjust where and how the object appears within the scene.

Adjusting parametric properties

The cube you’ve added to the scene is a special object known as a parametric-shaped object. This means you can change the look of the object by adjusting its parameters.

Reality Composer offers quite a few parametric shapes that you can use to create weird and wonderful AR experiences.

Feel free to experiment! :]

With the Properties panel still visible, expand the Look section to see what the cube shape has to offer.

Here, you can change the material and shape of the cube. Adjust the cube, like so:

• Set the Material to Aluminium.
• Set the Width to 10cm.
• Set the Height to 10cm.
• Set the Depth to 10cm.
• Set the Bevel Radius to 1cm.

Your next step is to add some physics to your scene to bring it to life with some quick and dirty animation.

Enabling physics

Reality Composer comes with an easy-to-use but powerful built-in physics simulator. Adding basic animation to your virtual objects elevates the believability of any AR scene.

With the cube still selected, expand the Physics section on the Properties panel.

Check the Participates option to enable physics for the selected object. When the scene plays, the object will come to life and fall to the ground, just like a real object in the real world would due to gravity.

Motion Type

There are two options for Motion Type, which determines how the object will participate in the physics simulation:

• Static: When set to Static, the object will participate in the physics simulation, but the physics engine won’t dynamically move it. You can animate the object by other means, and it will collide with other physics-enabled objects – like a baseball bat, for example.

• Dynamic: When set to Dynamic, the object will participate in the physics simulation, and the physics engine will dynamically move it. This could be a baseball, for example.

Select Dynamic to make the object participate dynamically in the physics simulation.

Physics Material

Your next step is to set up the Physics Material of the object. Reality Composer provides a few pre-configured material types.

These options control the object mass, material density, surface drag coefficient and restitution, or bounciness, of the object. So setting the material to concrete would make the object heavier and less bouncy than setting it to rubber, for example.

For this example, set the Material to Concrete.

Physics Collision Shape

Up next is setting the object’s Physics Collision Shape, which defines a rough shape for the object using the least amount of geometry possible. This reduces the overhead the physics engine has to deal with when computing collisions between multiple objects.

Reality Composer offers four basic shapes:

• Automatic: When you choose Automatic, Reality Composer automatically generates a low-polygon collision mesh based on the actual 3D model mesh. This should be your last resort if none of the other basic shapes work for you. This is the least performant collision shape.

• Box: Sets the collision mesh to a box shape. Useful for boxes, walls and flat objects, for example.

• Capsule: Setting the collision shape to capsule is useful for character bodies, for example.

• Sphere: Spheres are useful for round objects like balls. This is also the most performant collision shape.

Set the Collision Shape to Box.

Your basic AR scene is done now. Yay! Now wouldn’t it be great if there was a way to test it? Well, actually, there is… and you’ll do that next.

Playing scenes

Reality Composer has the ability to play the AR scene so that you can test it to find and debug any possible issues.

Click the Play button in the toolbar — but keep a watchful eye, because what you’re about to see happens quickly.

Did you see it? The cube comes alive, falls to the floor and bounces. Excellent!

Now wouldn’t it be nice to experience the scene in augmented reality? That brings us to Reality files.

What are Reality files?

Reality Files contain an AR experience created with Reality Composer. They’re archives that contain all of the required graphics, animations, textures and sounds that the AR scene requires. You can share the file and play it back on any app that supports AR Quick Look.

Exporting and sharing Reality files

To share your AR experience, you first have to export it.

Make sure you still have Scene 1 selected and select File ▸ Export ▸ Export Current Scene… to export the scene as a Reality file.

Name the file HelloRealityComposer.reality, then click Export.

That’s it; you’re done. You’ve successfully created your first AR experience with Reality Composer.

To test it out on your device, find the file in Finder, then right-click it and select Share ▸ AirDrop. Finally, select your device from the list.

On your device, accept and save the file using Files, then open it.

A shiny metal cube magically drops on your desk. How cool is that?

Note: You can find the final project and the exported Reality file inside the final folder.

Key points

Congratulations! You’ve reached the end of this chapter. Consider yourself now one full toe deep into the fantastic world of augmented reality. :]

Here are some of the key points covered in this chapter:

• Reality Composer: You now have a basic understanding of what Reality Composer is and what you can do with it.

• Features and Limitations: You have a good understanding of the feature set Reality Composer has to offer, along with its limitations.

• Projects: You know how to create your own Reality Composer projects.

• UI: You know your way around the user interface.

• Creating Multiple Scenes: It’s super easy to give your project multiple scenes.

• Content Library: It’s also easy to add virtual content to the scenes, especially with the wide range of objects to choose from the built-in content library.

• Parametric Shapes: You learned about parametric shapes and how to manipulate the parameters under the Properties panel.

• Physics: You got a quick introduction to basic physics and how easy it is to set up in Reality Composer.

• Play and Debug: You learned how to playtest your AR scenes with the Play option.

• Export and Share: Finally, you saw that once you’ve finished creating your experience, exporting it and sharing it is a breeze!

That’s it for now. See you in the next chapter, where you’ll learn about USDZ files and how they operate.