Why I Love ARKit for Apple iOS 11

With every new release of iOS, Apple seems to introduce at least one new library or technology that generates extra excitement among developers. A new technology like GPS or a gyroscope can make new types of apps possible. And to be honest, sometimes it’s because the new tech just looks like a whole lot of fun!

ARKit is one of those technologies that’s fun, but also opens up many new avenues for iOS app development. Since ARKit was announced, the buzz about augmented reality has only continued to build.

Maybe you’ve heard of augmented reality, but aren’t sure what it means. Or maybe you’re interested in augmented reality but not sure how ARKit can help you as a developer. In this article, we’ll examine what ARKit brings to the table and what you can do with it.

What is Augmented Reality?

Augmented reality, at its core, is the process of combining virtual objects with the real world. It’s not the same as virtual reality, which completely replaces the real world with a simulated one. Where virtual reality blocks out the real world, augmented reality seeks to enhance reality.

An augmented reality app superimposes 2D or 3D objects on the live camera view of the device. The result is an illusion that places these virtual objects in the real world, and gives the user extra information about the world that is not normally present, or invisible to the unaided eye.

Augmented Reality Before ARKit for Apple iOS 11

Augmented reality is a not a new technology; in fact, it’s existed in many forms before Apple came along and provided an SDK for iOS app development.

As an example, the heads up display (HUD) in some aircraft aids pilots by displaying important flight information on the cockpit window. That way, the pilot can see the information without looking away from the view. Similar technologies exist in vehicles to show speed and other pieces of data on the windshield.

Scenic look-offs usually have signs that identify key points of interest visible from the overlook. These signs provide extra context and information or metadata about the view, such as the height of peaks in the distance.

Even your Monday night football games uses a form of augmented reality: the current status, score, and time remaining in the game display on top of the video feed, along with stats about the star players in the game. Announcers will often use virtual pens to draw on top of replays when describing what worked, or went wrong, during a play.

The Current State of AR Apps

ARKit is an exciting new library for iOS app development, but augmented reality apps have been on Apple devices for years. Early iOS devices lacked some of the sensors needed to make augmented reality work well, such as a compass and gyroscope. As these sensors arrived, augmented reality apps began to appear on the App Store.

Pokémon Go

You would have had to try quite hard to avoid hearing about Pokémon Go during the summer of 2016, as it seemed to be the game that everyone was playing. Pokémon Go combines Nintendo’s second most popular video game franchise with an augmented reality interface. For many, it served as an introduction to augmented reality gaming on their phone.

PeakFinder Earth

PeakFinder Earth shows the names of nearby mountains as you view them through your phone. It also overlays a panoramic drawing — along with the paths of the sun and moon — on top of the view. It’s basically an augmented reality version of the scenic overlook sign I mentioned earlier.

Sky View

The first augmented reality app I remember using on the iPhone was the stargazing app Sky View. Identifying what’s in the sky above from a paper map while standing outside at night can be difficult. When looking through your phone and seeing the night sky labeled, identifying becomes a simple process.

Gymaholic

I use Gymaholic regularly to track my weight training routines. When adapting a new exercise, learning the proper form is important to reduce chances of injury and get the most from the workout. While many apps will show an exercise, Gymaholic can work with your Apple Watch or a sheet of paper printed with a target to display the exercise in question in augmented reality. Unlike a standard video or diagram, I can move and shift around the 3D-animated exerciser and view the exercise from any angle to ensure I understand the proper technique.

Ink Hunter

Getting a tattoo is a personal and somewhat permanent decision. Before committing to a tattoo, why not use your phone to see how it will look? Ink Hunter lets you do that. The app will allow you to search for tattoo designs and then see what it would look like on your skin.

PhotoPills

I enjoy photography, and I especially enjoy taking photos of landscapes. The time of day, season of the year and even your location on the planet often makes the difference between a nice photo and a great one. PhotoPills provides several tools to help you plan your photo shoot: it will overlay the path and location of the Sun, Moon, and Milky Way at any time, past or future, based on your location. Seeing how the sun will move through the sky for a given season and location makes planning the perfect photo easier.

Augmented Reality in iOS Before ARKit

To no one’s surprise, building augmented reality libraries is a complex problem. I once wrote an iPhone app to overlay a text description of nearby points of interest onto the view from the device’s camera. This simple augmented reality app required finding the current location of the device. It then needed to determine where the device is pointing in space. From the location, the app could determine nearby points of interest. From the orientation it could determine which points are actually in view. The app then had to combine these elements to display text at the correct point on the display.

It took a lot of math to combine all these items together to a the result for the screen. In the case of that app, most of the calculations were 3D calculations that involved matrix manipulations and linear algebra. More complex augmented reality apps may also need computer vision algorithms to detect surfaces, map points on the display back to virtual objects, and calculate lighting effects.

So make no mistake, although augmented reality was possible before ARKit, it wasn’t always easy.