AR Foundation in Unity: Getting Started

Augmented reality (AR) is one of the fastest growing technologies in today’s software industry. Many applications use AR technology to simulate makeup, camera filters and stage effects.

Developing an AR application from scratch isn’t an easy task. You need to know advanced algorithms for imaging processing, motion tracking, spatial analysis and even machine learning. Luckily, Apple and Android have developed their own AR software development kits (SDKs), which combine the necessary algorithms into tidy packages to make the task easier.

Unfortunately, if you want to build an AR application for both iOS and Android devices, you need to use both SDKs, which doubles your development efforts. To solve this, Unity has a handy library called AR Foundation. This library can help you build your AR application for both iOS and Android with a single codebase!

In this tutorial, you’ll learn how to build an AR doodling application. After you’ve completed this tutorial, you’ll be familiar with:

• How AR Foundation works
• Installing and setting up AR Foundation
• Detecting surfaces in the AR environment
• Interacting with the AR environment with raycasting
• Improving visuals with the post-processing module

The materials for this tutorial were built in Unity version 2020.2. You can get this version from the Unity website or install it via Unity Hub.

Getting Started

Download the starter project by clicking the Download Materials button at the top or bottom of the tutorial.

Unzip its contents and open the ARDoodle-Starter project in Unity.

After the project loads, you’ll see the RW folder in the Project window and the folder structure broken down as follows:

• Materials: Materials for the scene
• Prefabs: Pre-built components composed of scripts and models
• Scenes: The AR scene and the doodle testing scene
• Scripts: Scripts of Doodle components and the UI
• Settings: The settings file for the rendering
• Sprites: The graphics used by the UI

Open the DoodleTest demo scene from the Scenes folder and click Play to try it for yourself!

Your task is to bring this doodling application into the realm of AR! (Bonus: If you have kids, this will let them doodle freely without upsetting mom and dad. :])

But first you’ll learn more about AR Foundation, what it is and what it does.

Understanding AR Foundation

Take a look at the most popular AR devices and their SDKs:

Before AR Foundation, developers had to write different sets of code using device-specific SDKs to communicate with the AR devices. If you wanted to support all devices in a single application, you had to write several branching logics to switch between platforms. This made the development more time-consuming and the codebase more complicated.

With AR Foundation, you can use a unified interface to control device-specific SDKs!

To do this, you’ll dive down a little bit more into the architecture of AR Foundation.

AR Foundation Architecture

There are two core parts of the AR Foundation architecture:

• AR Subsystem
• AR Components

The AR subsystem is the bridge between Unity and the Native AR SDK modules. Every subsystem provides different features, e.g. ImageTrackingSubSystem detects images and RaycastSubSystem casts rays.

The subsystems only define method interfaces; they don’t have any implementation details. The actual implementation details are written in different device-specific XR plugins, such as the ARKIT XR Plugin for iOS and the ARCore XR Plugin for Android. These plugins implement the methods defined in the subsystem using the native AR SDK functions.

Although you can use the subsystems to control the native AR SDK, they’re not always easy to use. AR components are much more developer friendly.

Here are some components you’ll encounter:

• ARSession: for the lifecycle of the AR
• ARPlaneManager: for surface detection
• ARRaycastManager: for detecting user touch in the AR environment

Now that you have an overview of the architecture, you’ll move on to the next step of the tutorial.

Building Your Application to the Device

AR Foundation requires either an ARKit- or ARCore-powered phone to run. So first things first, you need to have your device ready and connected to your computer.

If you haven’t set up iOS or Android projects before, please read the instructions:

• For iOS, please click here.
• For Android, please click here.

Now, try to build and run the starter project on your device.

Open the Building Setting window by selecting File ▸ Build Setting from the menu. Set RW/Scene/ARScene as the first open scene, and choose iOS or Android in the platform panel.

Next, choose Switch Platform. It may take a few minutes to process, so grab some coffee or tea. :]

When the process is done, click Build and Run. When the file dialog appears, select a build folder, enter ArDoodle in the Save As field, and Save.

Unity will build and deploy the application to your device.

Installing AR Foundation Packages

Next, you’ll install the AR Foundation packages you need for this tutorial:

• AR Foundation: The core module for all AR Foundation applications
• ARCore XR Plugin: The plugin for Google AR SDK — ARCore
• ARKit XR Plugin: The plugin for iOS AR SDK — ARKit

Here’s how you do it:

1. Open the package manager from Window ▸ Package Manager.
2. Select Unity Registry at the top of the menu.
3. Find AR Foundation, ARCore XR Plugin and ARKit XR Plugin in the package list, select version 4.0.10, and click Install.

Here’s what your Package Manager should look like with the AR Foundation packages installed:

To verify the installation, check that the following items exist:

1. XR option in GameObject Menu
2. XR option in Assets Menu
3. XR Plugin Management setting in Project Setting

Setting Up the AR Environment

Before you start developing, you need to modify some settings to get AR Foundation up and running.

First, configure the Project Setting:

1. Select Edit ▸ Project Setting to open the Project Setting window.
2. Select the XR Plug-in Management ▸ iOS tab.
3. Check ARKit in Plug-in Providers.
4. Open the XR Plug-in Management ▸ Android Setting tab.
5. Check ARCore in Plug-in Providers.

A new folder named XR is automatically created in the Assets folder. This folder contains scripts and settings needed by AR Foundation.

Since this project uses the Universal Render Pipeline (URP), you also need to configure the URP renderer setting. Open Assets/RW/Settings/ForwardRenderer, click Add Renderer Features in the Inspector, and select AR Background Renderer Feature.

One of the awesome things about URP is its post-processing feature, which enhances the visuals of the application. You’ll near about this later.

Next, you’ll modify the AR scene. Every AR Foundation scene needs two core components: ARSession and ARSessionOrigin. So you’ll add them to the scene next.

• Open Assets/RW/Scene/ARScene, right-click an empty space in the Hierarchy, and select XR ▸ AR Session to create the ARSession object.
• Right-click an empty space in the Hierarchy again and select XR ▸ ARSessionOrigin to create the ARSessionOrigin object.

The ARSession and ARSessionOrigin objects are ready now, but they don’t have any AR features to help verify whether they’re working.