17. Interfacing With UIKit

Written by Caroline Begbie

Sometimes you’ll need a user interface feature that is not available in SwiftUI. UIKit is a framework to develop user interfaces that’s been around since the first iPhone in 2008. Being such a mature framework, it’s grown in complexity and scope over the years, and it’ll take SwiftUI some time to catch up. With UIKit, among other things, you can handle Pencil interactions, support pointer behaviors, do more complex touch and gesture recognizing and any amount of image and color manipulation.

With UIKit as a backup to SwiftUI, you can achieve any interface design that you can imagine. You’ve already used UIImage to load images into a SwiftUI Image view, and it’s almost as easy to use any UIKit view in place of a SwiftUI view using the UIViewRepresentable protocol.

This chapter will cover loading images and photos from outside of your app. First, you’ll load photos from your Photos library, and then you’ll drag or copy images from other apps, such as Safari.

UIKit

UIKit has a completely different data flow paradigm from SwiftUI. With SwiftUI, you define Views and a source of truth. When that source of truth changes, the views automatically update. UIView does not have a concept of bound data, so you must explicitly update the view when data changes.

Whenever you can for new apps, you should stick with SwiftUI. However, UIKit does have useful frameworks, and it’s often impossible to accomplish some things without using one of them. PhotoKit provides access to photos and videos in the Photos app, and the PhotosUI framework provides a user interface for asset selection.

Using the Representable protocols

Representable protocols are how you insert UIKit views into your SwiftUI apps. Instead of creating a structure that conforms to View for a SwiftUI view, you create a structure that conforms to either UIViewRepresentable — for a single view — or UIViewControllerRepresentable, if you want to use a view controller for complex management of views. To receive information from the UIKit view, you create a Coordinator.

To get started with UIViewRepresentable, you’ll first show a basic, colored UIView on top of a simple SwiftUI View.

➤ In the group Card Modal Views, create a new Swift file named PhotoPicker.swift.

➤ Replace the code with:

import SwiftUI

struct PhotoPicker: UIViewRepresentable {
}

Here you create a structure that will conform to the protocol UIViewRepresentable.

➤ There are two required methods, so add these:

func makeUIView(context: Context) -> UILabel {
  let label = UILabel()
  label.text = "Hello UIKit!"
  return label
}

func updateUIView(_ uiView: UILabel, context: Context) {
}

makeUIView(context:) is where you create and return a UIView — in this case, a UILabel, which simply shows some text.

updateUIView(_:context:) is where the parent view can communicate with the child view. In this case, the child view is a read-only label, and there’s no communication.

➤ Create a preview that shows the SwiftUI Text view:

struct PhotoPicker_Previews: PreviewProvider {
  static var previews: some View {
    Text("Hello SwiftUI!")
      .background(Color.yellow)
  }
}

➤ Preview this and then replace Text("Hello SwiftUI!") with:

PhotoPicker()

➤ Preview the view.

Whereas the SwiftUI text view takes up just enough space to show itself, a UIKit view will take up the whole of the available space.

UIKit delegate pattern

Many of the UIKit classes use protocols that ask for a delegate object to deal with events. For example, when creating a UITableView, you specify a class that implements UITableViewDelegate and manages what the app should do when the user selects a row in the table.

Using delegation, you can change the behavior of the table without having to subclass UITableView.

In PhotoPicker, your code will use the system photo picker: PHPickerViewController. This class shows the user’s photos in a UIView and allows users to select photos from their library. An event occurs when the user taps Add or Cancel. When this happens, PHPickerViewController will inform the delegate object, and the delegate can take any action. This delegate must be a class that is a subclass of NSObject and conforms to PHPickerViewControllerDelegate.

PhotoPicker is a structure, so it can’t conform to PHPickerViewControllerDelegate. Inside PhotoPicker, you’ll create a coordinator class that will interface with the system photo picker and return all the selected images to CardDetailView via PhotoPicker.

Picking photos

As the system photo picker is a subclass of UIViewController, your Representable structure will be a UIViewControllerRepresentable.

➤ At the top of PhotoPicker.swift import the photos framework:

import PhotosUI

➤ Replace PhotoPicker with:

struct PhotoPicker: UIViewControllerRepresentable {
  func makeUIViewController(context: Context) 
    -> some UIViewController {
  }
  
  func updateUIViewController(
    _ uiViewController: UIViewControllerType,
    context: Context
  ) {
  }
}

These are the two methods required to conform to UIViewControllerRepresentable.

➤ In makeUIViewController(context:), add the photo picker:

// 1
var configuration = PHPickerConfiguration()
configuration.filter = .images
// 2
configuration.selectionLimit = 0
// 3
let picker = 
  PHPickerViewController(configuration: configuration)
return picker

Going through the code:

1. You create a PHPickerConfiguration. This configuration has a filter where you can specify the types of photos for the user to select. As well as images, you can select livePhotos and videos.
2. Specify the number of photos a user is allowed to pick. Use 0 to allow selection of multiple photos.
3. Create a PHPickerViewController using the previously created configuration.

➤ Create a new class inside PhotoPicker:

class PhotosCoordinator: NSObject,
  PHPickerViewControllerDelegate {
}

The class doesn’t need to be internal to PhotoPicker, but you probably won’t want to use it on its own and making it internal means that it can only be in scope inside PhotoPicker.

PhotosCoordinator is a subclass of NSObject. This is the class that most Objective-C objects inherit from, so when you’re using class delegates from UIKit objects, you’ll usually inherit from NSObject.

➤ Add the required method for PHPickerViewControllerDelegate:

func picker(
  _ picker: PHPickerViewController,
  didFinishPicking results: [PHPickerResult]
) {
}

When the user taps Add after selecting photos, PHPickerViewController will call this delegate method and pass the selected photos as UIImages in an array of PHPickerResult objects. You’ll process each of these images.

➤ Add a new property to PhotoPicker (not PhotosCoordinator):

@Binding var images: [UIImage]

CardDetailView will pass an empty array to PhotoPicker, and the delegate method will fill this array with the picked results.

➤ Update the preview:

struct PhotoPicker_Previews: PreviewProvider {
  static var previews: some View {
    PhotoPicker(images: .constant([UIImage]()))
  }
}

The preview won’t be acting on the images array, so pass a constant array to the binding.

➤ Add a new property and initializer to PhotosCoordinator:

var parent: PhotoPicker

init(parent: PhotoPicker) {
  self.parent = parent
}

When you initialize PhotosCoordinator from PhotoPicker, you’ll pass the PhotoPicker instance, so that you can access the images array.

➤ Add this method to PhotoPicker:

func makeCoordinator() -> PhotosCoordinator {
  PhotosCoordinator(parent: self)
}

makeCoordinator() saves the coordinator class in the Representable context. If you need to access the coordinator class in updateUIViewController(_:context:), you can do so with context.coordinator.

UIViewControllerRepresentable calls makeCoordinator() before it calls makeUIViewController(context:). This method exists solely to instantiate the coordinator class that coordinates with UIKit classes. If you don’t need data returning from UIKit, then you don’t need to create a coordinator.

Setting the delegate

➤ Add this to makeUIViewController(context:) at the end of the method, before returning picker:

picker.delegate = context.coordinator

picker now knows that PhotosCoordinator is its delegate. PhotosCoordinator implements delegate?.picker(_:didFinishPicking:) when the user taps the Add button on the system photo picker modal.

NSItemProvider

You’ve now set up the interface between the SwiftUI PhotoPicker and the UIKit PHPickerViewController. All that’s left is to load the images array from the modal results.

➤ In picker(_:didFinishPicking:), add this code:

let itemProviders = results.map(\.itemProvider)
for item in itemProviders {
  // load the image from the item here 
}

Each PHPickerResult holds an NSItemProvider. Using the key path \.itemProvider, you extract the item providers from all the results into an array and then iterate through that array.

Swift Tip: Using map with the key path \.itemProvider is syntactic sugar for let itemProviders = results.map { $0.itemProvider }

Any class with the NS prefix is an Objective-C class which inherits from NSObject. So NSItemProvider ultimately inherits from NSObject. When you want to transfer data around your app, or between apps, you use item providers. You can ask the item provider whether it can load a particular type and then asynchronously load it. Later in this chapter, you’ll be dragging images from Safari into your app — again using item providers.

➤ Inside the for loop, add the code to load the image:

// 1
if item.canLoadObject(ofClass: UIImage.self) {
  // 2
  item.loadObject(ofClass: UIImage.self) { image, error in
    // 3
    if let error = error {
      print("Error!", error.localizedDescription)
    } else {
      // 4
      DispatchQueue.main.async {
        if let image = image as? UIImage {
          self.parent.images.append(image)
        }
      }
    }
  }
}

Going through the code:

1. Check whether the item can load a UIImage.
2. Load the UIImage. The closure parameters provide an object that conforms to NSItemProviderReading and an error object.
3. If the error is not nil, print out the description. In a full app, you would provide error handling.
4. Ensure that the passed object is a UIImage and add the image to PhotoPicker’s image array asynchronously. You must do so on the main queue because it will cause an update to the UI. All NSItemProvider completion closures execute on an internal system queue in the background.

With the images loading, you should dismiss the system modal.

➤ Add a property to PhotoPicker:

@Environment(\.presentationMode) var presentationMode

➤ At the end of picker(_:didFinishPicking:), add this:

parent.presentationMode.wrappedValue.dismiss()

This will tell the environment to close the modal. PhotoPicker is now all ready for use in SwiftUI.

➤ Live preview PhotoPicker to see how the system photo picker works. You can select multiple photos and also select from your photo albums. You can also show all the selected images.

Adding PhotoPicker to your app

To use PhotoPicker, you’ll need hook it up to your Photos toolbar button and save the loaded photos as ImageElements.

➤ Open CardDetailView.swift and add this new property to CardDetailView:

@State private var images: [UIImage] = []

This is the array you’ll hand over to PhotoPicker.

➤ Locate .sheet(item:). Inside switch item, add the new modal:

case .photoPicker:
  PhotoPicker(images: $images)
  .onDisappear {
    for image in images {
      card.addElement(uiImage: image)
    }
    images = []
  }

Here you show the modal, passing in the array to hold the photos the user will select. When the modal disappears, you process the array and add the photos to the card elements. Finally, you clear the images array to make it ready for the next time.

➤ Build and run, and choose the second orange card. Add a couple of photos using the system photo picker. The app adds the photos to the card elements so that you can resize and reposition them.

Note: At the time of writing, the simulator pink flowers photo causes an error. This appears to be an Apple bug, but it does give you the chance to make sure that your PhotoPicker error checking works. In the console, you should see Error! Cannot load representation of type public.jpeg.

Adding photos to the simulator

If you want more photos than the ones Apple supplies, you can simply drag and drop your photos from Finder on to the simulator. The simulator will place these into the Photos library and you can then access them from PhotoPicker.

Drag and drop from other apps

As well as adding photos from the ones on your device, you’ll add drag and drop of images from any app. Similar to the photos system modal, you do this using an item provider.

First set up Simulator so that you’ll be able to do the drag and drop.

➤ Build and run your app on an iPad simulator and turn the iPad to landscape mode. You can use the icon on the top bar, or use Command-Right Arrow. With your mouse cursor just touching the black bevel at the bottom of the app, drag upward slowly to show the dock. One of the apps on the dock should be Safari.

➤ Hold down the Safari icon and drag it off the dock to the right of the Cards app. A space will open up for you to drop the icon.

➤ Use the bar in the middle to resize each app to take up half the iPad screen area.

➤ In the Cards app, tap the orange card. In Safari, Google your favorite animal and tap Images. Long press an image until it gets slightly larger and drag it onto your orange card.

Cards is not ready to receive a drop yet, so nothing happens. If the drop area were able to receive an item, you would get a plus sign next to the image.

Uniform Type Identifiers

Your app needs to distinguish between dropping an image and dropping another format, such as text. Most apps have associated data formats. For example, when you right-click a macOS file and choose Open With, the menu presents you with all the apps associated with that file’s data format. When you right-click a .png file, you might see a list like this:

These are the apps that are able to open .png files.

Uniform Type Identifiers, or UTIs, identify file types. For example, PNG is a standard UTI, with the identifier public.png. It’s a subtype of the image data base type public.image.

Note: There are many standard system UTIs which you can find at https://apple.co/3xASdxD.

If you have a custom data format, you can create your own UTI and include it in Info.plist. For this app, however, you only need to use public.image to receive any image format.

Adding the drop view modifier

In Xcode, open CardDetailView.swift. Add a new modifier to content above the toolbar modifier:

// 1
.onDrop(of: [.image], isTargeted: nil) { 
  // 2
  itemProviders, _ in
  // 3  
  return true
}

Going through the code:

1. This is where you specify the identifier of the file type you wish to process; in your case .image. There are several onDrop... modifiers. This one takes an array of UTTypes and a Boolean binding to indicate whether there is a drag and drop operation currently happening.
2. The closure presents the dropped items in an array of NSItemProviders and the drop location. For the moment you won’t use the location, so you replace the parameter with _.
3. Returning true indicates to the system that the drop was successful.

➤ Build and run and repeat dragging an image on to the card.

This time, even though the drop does nothing, you get the plus sign.

➤ Inside onDrop(of:isTargeted:perform:), before return true, add this code:

for item in itemProviders {
  if item.canLoadObject(ofClass: UIImage.self) {
    item.loadObject(ofClass: UIImage.self) { image, _ in
      if let image = image as? UIImage {
        DispatchQueue.main.async {
          card.addElement(uiImage: image)
        }
      }
    }
  }
}

This code is almost exactly the same as the code you wrote earlier for the photo picker. Iterate through the items and load them as a UIImage. Here, you add the image directly to the card’s elements.

➤ Build and run. Repeat dragging an image on to the orange card and this time the drop action saves the image to the card elements.

In Simulator, to select multiple images in Safari at the same time, pick up an image and start dragging it. That small drag is important — you won’t be able to multiple select without it. Then hold down Control. Release the click and then Control. A gray dot appears on the image representing your finger on a device. Click other images to add them to the drag pile. When you’ve collected all the images, drag them to Cards.

Currently, no matter where you drop images, the card adds the new elements at the center. You can use the drop location to place the element where you dropped it. However, to calculate the offset for the element’s transform, you’ll need to convert the location point on the card to an offset from the center of the card. This involves knowing the screen size of the card. You’ll revisit this problem in Chapter 20, “Delightful UX — Layout”.

Refactoring the code

CardDetailView is getting quite large and complex now, and you should start to think about how you can refactor it and split out as much code as you can. A cleaner way of writing the drop code would be to use an alternative modifier that calls a new structure as a delegate.

➤ Create a new Swift file called CardDrop.swift to contain this delegate.

➤ Replace the code with the following:

import SwiftUI

struct CardDrop: DropDelegate {
  @Binding var card: Card
}

You create a new structure that will conform to DropDelegate and receive the card that the drop delegate should update. You need to implement one required method to conform to DropDelegate.

➤ Add this method to CardDrop:

func performDrop(info: DropInfo) -> Bool {
  let itemProviders = info.itemProviders(for: [.image])

  for item in itemProviders {
    if item.canLoadObject(ofClass: UIImage.self) {
      item.loadObject(ofClass: UIImage.self) { image, _ in
        if let image = image as? UIImage {
          DispatchQueue.main.async {
            card.addElement(uiImage: image)
          }
        }
      }
    }
  }
  return true
}

At the start of the method, you extract the item providers from the drop info, and then the rest of the code is the same as you have in CardDetailView.

DropDelegate has several other required methods that have default implementations, so you don’t need to define them in your app.

• dropEntered(info:): A potential drop has entered the view.
• dropExited(info:): A potential drop has exited the view.
• dropUpdated(info:): A potential drop has moved inside the view.

If you need to have complete control of where in the screen your user is dragging items, then implement these methods.

➤ Open CardDetailView.swift. Replace onDrop(of:isTargeted:perform:) and all its code with:

.onDrop(of: [.image], delegate: CardDrop(card: $card))

Here you still use the image UTI, but you offload the code into CardDrop.

With this one line of code, you have reduced the apparent complexity. The CardDrop code is difficult to read, and you don’t need to be viewing it every time you’re updating your card detail code. It’s a good idea to reduce brain overload whenever you can. :]

➤ Build and run and your app works the same as it did before.

Challenge

Challenge: Leverage PencilKit

Now that you know how to host UIKit views in SwiftUI, you have access to a wide range of Apple frameworks. One fun framework is PencilKit where you can draw into a canvas.

Your challenge is to write a few lines of code and run a live preview in which you can scribble.

• Create a new View and import PencilKit. Create a PKCanvasView state property. Pass this property to a UIViewRepresentable object.
• Create the two required methods in the UIViewRepresentable object.
• There’s only two extra lines of code needed. In makeUIView(context:), set the canvas drawingPolicy to anyInput to allow input from both finger and Pencil and return the canvas.

You won’t integrate this view in your current version of Cards, but this could be a feature in a later version where you can extract an image from the scribble.

If you have any difficulty, you’ll find the solution to this challenge in the challenge folder for this chapter in the file PencilView.swift.

Key points

• SwiftUI and UIKit can go hand in hand. Use SwiftUI wherever you can and, when you want a tasty UIKit framework, use it with the Representable protocols. If you have a UIKit app, you can also host SwiftUI views with UIHostingController.
• The delegate pattern is common throughout UIKit. Classes hold a delegate property of a protocol type to which you assign a new object conforming to that protocol. The UIKit object performs methods on its delegate.
• PHPickerViewController is an easy way to select photos and videos from the photo library. Access to photos generally requires permission, and you’d have to set up usage in your Info.plist. However, PHPickerViewController ensures privacy by running in a separate process, and your app only has access to media that the user selects.
• Item providers enable passing data more easily between apps.
• Using Uniform Type Identifiers and the onDrop modifier, you can support drag and drop in your app.