Instruction 02

When AI models work with data, they use lots of computing power and fast code execution. As the technologies were first appearing, mobile devices weren’t powerful enough to handle the computations needed. The general strategy was:

1. Upload an image to the AI prompts.
2. Do the processing on a server.
3. Return answers.

The remote server processing made many security-minded professionals very nervous. Also, the added network calls limited the speed of any responses.

One of the most significant advantages in using the Vision Framework is that all the analysis is on device. This helps ensure data privacy and security. With their Mx chips, Apple have started incorporating processors specifically for AI work. This helps keep the CPU from getting bogged down with the AI computations and also doesn’t require a GPU. As you’ll learn in later lessons, you can tell the Vision Framework where to process your requests. This lets you control how all the different processors work together to make your app run most efficiently.

Asking Permission

One of the first steps in working with the Vision Framework in your applications is getting an image to process. Images can come from many sources. You can download them from the web or include them in the app bundle. You can let the users choose them from the Files app. The most common way is to let users choose from the Photo library or use their device’s camera.

Except for the first two locations, your app is importing user data. This means your app usually needs to request their permission. Also, when a user denies permission, your app must handle it gracefully.

The PhotosUI framework introduced in iOS 16 allows for a limited, read-only access to the user’s library. You can also configure the document picker to have read-only access, which is also limited. Neither of these requires asking the user for permission. But if your app needs write access or you need to support iOS photo libraries before iOS 16, you must ask for permission. For each type of permission, you’ll add a key to the app’s info.plist.

Here’s a handy list of the different info.plist keys you might need to add to your app.

• Photo Library Access (NSPhotoLibraryUsageDescription): Describes why the app needs access to the photo library if you can’t use PhotosUI.
• Photo Library Add-Only Access (NSPhotoLibraryAddUsageDescription): Describes why the app needs to add photos to the user’s photo library.
• Camera Access (NSCameraUsageDescription): Describes why the app needs access to the camera.
• Files App Access (NSDocumentsFolderUsageDescription): Describes why the app needs access to add to the user’s files.

If you forget to ask for permissions, the app will crash when you attempt to access the user’s data or camera. Xcode will display a message like the one below in the console:

This app has crashed because it attempted to access privacy-sensitive data
without a usage description. The app's Info.plist must contain an
NSCameraUsageDescription key with a string value explaining to the
user how the app uses this data.

After you’ve added a permissions request, the first time your app tries to use something like the camera, the user sees a dialog box like this one.

In the previous image, the text in the rectangle is provided by Apple and you can’t modify it. The text the arrow is pointing to is the text you enter into the info.plist. Notice that the viewfinder of the camera is black. Once the user grants permissions, the viewfinder will show what the camera sees.

The permissions modal displays only once. If the user denies permission, you’ll need to direct them to grant permissions by going to the device “Settings”. The next section is a video demo of this process.