AR Face Tracking Tutorial for iOS: Getting Started

Right Eye Blink

This will be very similar to the code for the left eye. Add the following case to the same switch statement:

case "rightEye":
  let scaleX = child?.scale.x ?? 1.0
  let eyeBlinkValue = anchor.blendShapes[.eyeBlinkRight]?.floatValue ?? 0.0
  child?.scale = SCNVector3(scaleX, 1.0 - eyeBlinkValue, 1.0)

Build and run your app again, and you should be able to blink with both eyes!

Open Jaw

Currently, in the app, if you open your mouth, the mouth emoji stays between the lips, but no longer covers the mouth. It’s a bit odd, wouldn’t you say?

You are going to fix that problem now. Add the following case to the same switch statement:

case "mouth":
  let jawOpenValue = anchor.blendShapes[.jawOpen]?.floatValue ?? 0.2
  child?.scale = SCNVector3(1.0, 0.8 + jawOpenValue, 1.0)

Here you are using the jawOpen blend shape, which is 0.0 for a closed jaw and 1.0 for an open jaw. Wait a second… can’t you have your jaw open but still close you mouth? True; however, the other option, mouthClose, doesn’t seem to work as reliably. That’s why you’re using .jawOpen.

Go ahead and build and run your app one last time, and marvel at your creation.

Where to Go From Here?

Wow, that was a lot of work! Congratulations are in order!

You’ve essentially learned how to turn facial expressions into input controls for an app. Put aside playing around with emoji for a second. How wild would it be to create an app in which facial expressions became shortcuts to productivity? Or how about game where blinking left and right causes the character to move and puffing out your cheeks causes the character to jump? No more tapping the screen like an animal!

If you want, you can download the final project using the Download Materials link at the top or bottom of this tutorial.

We hope you enjoyed this face-tracking tutorial. Feel free to tweet out screenshots of your amazing emoji bling creations!

Want to go even deeper into ARKit? You’re in luck. There’s a book for that!™ Check out ARKit by Tutorials, brought to you by your friendly neighborhood raywenderlich.com team.

If you have any questions or comments, please join the forum discussion below!