17. Adding Photos to Your App

Written by Caroline Begbie

In the previous chapter, you learned how to add stickers to your card. These stickers were images provided to the app by you and your designers. Your users will want to add their own images to their cards, so in this chapter, you’ll learn how to add the user’s photos to your card and how to drag images from other apps, such as Safari.

The PhotosUI Framework

With the stickers, you load the sticker images lazily, and when the user selects one, you use that one image. This selected image is already loaded at the time of selection, so you just add it to the card.

Loading photos is not as simple as loading stickers, because the user’s media library might number in the tens of thousand of assets. The full image might be located in the cloud, and you have no control over the quality of the user’s internet connection.

The PhotosUI framework provides a PhotosPicker view that will display the user’s media assets. The user then selects photos, and each selected item goes into an array. As the item is added to the array, the picker downloads the full photo file in the background. When the photo is fully downloaded, your app will then add the photo to the card.

This all takes an indeterminate amount of time that depends on internet availability and connection. Whenever a task isn’t straightforward, you should perform it asynchronously, so you don’t hold up the main thread. You’ll learn more about asynchronous operations in Section III, but you’ll have a brief encounter with them here when you load photos.

The PhotosPicker View

Skills you’ll learn in this section: PhotosPicker

Instead of using your own modal view, you’ll use PhotosUI’s PhotosPicker.

➤ Open the starter project for this chapter, which is the same as the previous chapter’s challenge project.

➤ In the Card Modal Views group, create a new SwiftUI View file called PhotosModal.swift and import the framework:

import PhotosUI

➤ Replace PhotosModal with:

struct PhotosModal: View {
  @Binding var card: Card
  // 1
  @State private var selectedItems: [PhotosPickerItem] = []

  var body: some View {
    // 2
    PhotosPicker(
      // 3
      selection: $selectedItems,
      // 4
      matching: .images) {
      // 5
        ToolbarButton(modal: .photoModal)
    }
  }
}

Going through the code:

1. Create an array to hold the selected images. The type PhotosPickerItem doesn’t contain the actual image data. Instead, it contains only an identifier and the type of content, such as jpeg, that the item supports.
2. Display the photos picker view.
3. As the user taps and selects media assets, the photos picker adds them to selectedItems.
4. You can filter the photo library in various ways, such as screenshots or videos. For Cards, you filter images. You can see the other available filters here.
5. PhotosPicker requires a label to start it, so you include the image and text you already set up in ToolbarButton.

Note: Be careful with your file names. If you create a structure called PhotosPicker, that will override the one used by PhotosUI without any warning. You can still use PhotosUI.PhotosPicker, but you have to specifically reference PhotosUI when you do.

➤ In PhotosModal_Previews, change PhotosModal() to:

PhotosModal(card: .constant(Card()))

In Live Preview, you’ll see the button with the label you provided:

➤ Tap the button to see how the system photos picker works. You can select multiple photos and also select from your photo albums. You can also show larger versions of all the images you’ve selected.

Adding the Photos Picker to Your App

➤ Open CardToolbar.swift, and locate .sheet(item: $currentModal).

This is where you display the modal views when the user taps a button on the bottom toolbar.

➤ Add a new case to switch item:

case .photoModal:
  PhotosModal(card: $card)

Here you set up the photo button on the toolbar to display your photos modal view.

➤ Open SingleCardView.swift and pin the preview. In Live Preview, tap the Photos button on the bottom toolbar.

You may have anticipated this. Your PhotosModal view pops up from your button. This view contains the system PhotosPicker view which you defined with its own label. You obviously don’t want to compel your user to press two buttons.

➤ Open BottomToolbar.swift and, in BottomToolbar, locate Button {...} inside ForEach.... Command-Click Button, and select Embed….

➤ Change Container { to:

switch selection {
default:

➤ Add a new case to switch selection before the default case:

case .photoModal:
  Button {
  } label: {
    PhotosModal(card: $card)
  }

The resulting view from PhotosModal is the button you supply to PhotosPicker, so this replaces the previous ToolbarButton.

➤ BottomToolbar needs to contain the binding, so add the new property to BottomToolbar:

@Binding var card: Card

➤ Replace BottomToolbar in the preview with:

BottomToolbar(
  card: .constant(Card()),
  modal: .constant(.stickerModal))

Remember to add the parameters in the order they appear in BottomToolbar.

➤ Open CardToolbar.swift and remove:

case .photoModal:
  PhotosModal(card: $card)

BottomToolbar loads the photos view now, so this is no longer needed.

➤ Inside ToolbarItem(placement: .bottomBar) {, locate BottomToolbar(modal: $currentModal). Replace it with:

BottomToolbar(
  card: $card,
  modal: $currentModal)

You now pass the binding and your app compiles.

➤ Resume Live Preview on Single Card View, and select the Photos button on the bottom toolbar.

This time you see the system photos picker. When you tap Cancel, the Photos modal disappears. So far, when you select photos and tap Add, nothing happens. The system retains the selection, however, as you’ll see if you return to the photos picker.

The Transferable Protocol

Skills you’ll learn in this section: Transferable; Uniform Type Identifiers; add photos to Simulator

It’s not only photos that you might want to add to your app. You might want to be able to copy and paste text, or even custom types, such as files created by another app. You’ll also want to share your card with your friends, which means exporting your card from your app. Transferable is a flexible protocol that allows you to describe how to import and export any types.

Some existing data types, such as Data, which is a string of bytes, already conform to Transferable. When you add photos, these will be of type UIImage, which unfortunately does not conform.

It’s easy to add conformance, and you’ll do that later in the chapter. For the moment, though, to get you quickly adding photos, you’ll transfer the photos as Data.

➤ Open PhotosModal.swift and add this modifier to PhotosPicker:

.onChange(of: selectedItems) { items in
  for item in items {
    print(item)
  }
  selectedItems = []
}

Whenever selectedItems changes, you’ll print out each element in the array. After you’ve processed each item, clear the array.

➤ Build and run the app in Simulator, choose a card, tap the Photos button on the bottom toolbar and select the pink flowers photo and one other. Tap Add.

The details of each item selected print out in the debug console. Notice the _supportedContentTypes. These are the supported content types for the item. The pink flowers photo has two types: public.jpeg and public.heic. The other photo has just one type: public.jpeg.

Uniform Type Identifiers

Uniform Type Identifiers, or UTIs, identify file types. For example, JPG is a standard UTI, with the identifier public.jpeg. It’s a subtype of the base image data type public.image.

public.text encompasses all text data, including public.plainText and public.rtf.

Most apps have associated data types. For example, when you right-click a macOS file and choose Open With, the menu presents you with all the apps associated with that file’s data format. When you right-click a .png file, you might see a list like this:

These are the apps that are able to open .png files.

There are many standard system UTIs which you can find at https://apple.co/3xASdxD.

If you have a custom data format, you can create your own type in a UTType extension:

extension UTType {
  static var myType: UTType =
    { UTType(exportedAs: "com.kodeco.myType") }
}

public.data is a base type representing a stream of bytes. Using this type, you can load the photos as a data stream and then convert the data to a UIImage.

Adding Photos to Your App

➤ Still in PhotosModal.swift, in the for loop, replace print(item) with:

item.loadTransferable(type: Data.self) { result in
  Task {
  // create a UIImage
  }
}

You load the item as a Data type. result is of type Result<Success, Failure>. Success contains the image data, and Failure contains a failure value.

For each item, you load the image on a background thread using Task {}.

➤ Replace // create a UIImage with:

switch result {
case .success(let data):
  if let data,
    let uiImage = UIImage(data: data) {
    card.addElement(uiImage: uiImage)
  }
case .failure(let failure):
  fatalError("Image transfer failed: \(failure)")
}

If the result succeeds, use the data to create a UIImage and add that image to the card’s element array. If the result fails, produce a fatal error.

Note: At the time of writing, the simulator pink flowers photo in HEIC format causes an error. This appears to be an Apple bug, but it does give you the chance to make sure that your PhotosPicker error checking works. When you run your app in Simulator and choose that photo, in the console, you should see Fatal error: Image transfer failed: with information about the failure. HEIC format files will work on a device with your own photos.

➤ Live Preview Single Card View and add some photos (not the pink flowers) to the card.

Adding Photos to Simulator

If you want more photos than the ones Apple supplies, you can simply drag and drop your photos from Finder into Simulator. Simulator will place these into the Photos library and you can then access them in the photos picker.

Drag and Drop From Other Apps

Skills you’ll learn in this section: Split view; drag and drop; data representation

The photos library is not the only place you can access photos. Modern apps should accept photos and images that you drag from any other app.

First set up Simulator so that you’ll be able to do the drag and drop.

➤ Build and run your app on an iPad simulator and turn the iPad to landscape mode. You can use the icon on the top bar, or use Command-Right Arrow.

➤ Tap the three dots at the top of Simulator’s screen and choose Split View.

➤ Locate the Safari icon and tap it.

Safari will load using half of the iPad screen.

➤ In the Cards app, tap a card. In Safari, Google your favorite animal and tap Images. Long press an image until it gets slightly larger and drag it onto your card.

Cards is not ready to receive a drop yet, so nothing happens. If the drop area were able to receive an item, you would get a plus sign next to the image.

Adding the Dropped Item to Your App

➤ Open CardDetailView.swift and, in body, add this modifier to ZStack:

.dropDestination(for: Data.self) { receivedData, location in
  print(location)
  for data in receivedData {
    if let image = UIImage(data: data) {
      card.addElement(uiImage: image)
    }
  }
  return !receivedData.isEmpty
}

Just as you did with your photos, you receive the dragged image or images as an array of data streams. You create a UIImage from the data and add the image to the card’s array of elements. You return whether the operation was successful.

Currently you don’t use location, so any dropped items are added to the center of the card. To calculate the offset for the element’s transform, you’ll need to convert the location point on the card to an offset from the center of the card. This involves knowing the screen size of the card. You’ll revisit this problem in Chapter 20, “Delightful UX — Layout”.

➤ Build and run again, and drag in photos from Safari.

As you drag over the drop area, a plus sign will appear on the drop pile, indicating that the drop destination is allowable for this data type. When you drop the photo, it’s added to the card at the center.

In Simulator, to select multiple images in Safari at the same time, pick up an image and start dragging it. That small drag is important — you won’t be able to multiple select without it. Then hold down Control. Release the click and then Control. A gray dot appears on the image representing your finger on a device. Click other images to add them to the drag pile.

When you’ve collected all the images, drag them to Cards.

Conforming Types to Transferable

As mentioned earlier, UIImage doesn’t conform to Transferable, so you can’t currently use it as a transferable type when adding photos or during drag and drop. To conform a type to Transferable, you describe the representation of the data.

➤ Create a new Swift file in the Extensions group, called UIImageExtensions.swift and replace the code with this:

import SwiftUI
// 1
extension UIImage: Transferable {
  // 2
  public static var transferRepresentation: some TransferRepresentation {
    // 3
    DataRepresentation(importedContentType: .image) { image in
      // 4
      UIImage(data: image) ?? errorImage
    }
  }

  public static var errorImage: UIImage {
    UIImage(named: "error-image") ?? UIImage()
  }
}

This code needs some explanation:

1. Add a new extension to UIImage to conform to Transferable.
2. transferRepresentation is a required property. TransferRepresentation describes how to transfer an item. You can describe how to import and export the item.
3. When the imported UTType is an image, you’ll import the image as data and construct a UIImage from that data. DataRepresentation expects the return type to be the same type as Self, in this case, a UIImage.
4. Create the UIImage. If the operation fails, create an error image using the image in the asset catalog.

If you want to transfer a custom object that conforms to Codable, instead of DataRepresentation, you can use CodableRepresentation in the same way.

Updating the Drag and Drop

You can now import dropped photos using UIImage instead of Data. This makes your code correspond more closely to your intent. When you see the word “data”, it’s not always obvious what type that data is.

➤ Open CardDetailView.swift and replace the dropDestination modifier with:

.dropDestination(for: UIImage.self) { images, location in
  print(location)
  for image in images {
    card.addElement(uiImage: image)
  }
  return !images.isEmpty
}

You can now use UIImage.self as a Transferable type. The data representation in the UIImage extension reads in the data and converts it to a UIImage, which you can add to the the card directly.

Drag and Drop of Custom Types

You can now drag in images from another app, but you also want to drag in text. You can try this naively and see how it will work.

➤ First, open Card.swift in the Model group and add a new method to Card that adds text to the card elements.

mutating func addElement(text: TextElement) {
  elements.append(text)
}

You receive in a TextElement and add it to the elements array.

➤ Open CardDetailView.swift and locate the UIImage dropDestination modifier.

Unfortunately, having two drop destination modifiers doesn’t work.

➤ Replace the UIImage drop destination modifier with this code:

.dropDestination(for: String.self) { strings, _ in
  for text in strings {
    card.addElement(text: TextElement(text: text))
  }
  return !strings.isEmpty
}

You add a new drop destination modifier that will take in a String and add a text element to the card.

➤ Build and run the app in an iPad simulator with Safari open in Split View. In Safari, highlight a small amount of text and drag it on to your card.

Cards will add the text element to the center of the card.

So far, so good. You can drag an image to your card, and you can drag some text to your card. The only problem is that you have to change the code each time.

You can overcome this problem with a custom transfer type that conforms to Transferable.

➤ In the Model group, create a new Swift file named CustomTransfer.swift and replace the code with:

import SwiftUI

struct CustomTransfer: Transferable {
  var image: UIImage?
  var text: String?

  public static var transferRepresentation: some TransferRepresentation {
    DataRepresentation(importedContentType: .image) { data in
      let image = UIImage(data: data)
        ?? UIImage(named: "error-image")
      return CustomTransfer(image: image)
    }
    DataRepresentation(importedContentType: .text) { data in
      let text = String(decoding: data, as: UTF8.self)
      return CustomTransfer(text: text)
    }
  }
}

CustomTransfer contains two properties, one for text and one for image. The transfer representation takes into account the image type and the text type and fills in the relevant property. For images, the data representation is the same as you did for UIImage, and for text, you create a String from the data. UTF8 is the most common Unicode encoding system.

Once CustomTransfer has created either an image or some text from the transferred data, you’ll add an element to the card.

➤ Open Card.swift and add this new method:

mutating func addElements(from transfer: [CustomTransfer]) {
  for element in transfer {
    if let text = element.text {
      addElement(text: TextElement(text: text))
    } else if let image = element.image {
      addElement(uiImage: image)
    }
  }
}

Using your custom Transferable structure, you can add both text and image elements to the card appropriately.

➤ Open CardDetailView.swift, and in body, replace the drop destination modifier with:

.dropDestination(for: CustomTransfer.self) { items, location in
  print(location)
  Task {
    card.addElements(from: items)
  }
  return !items.isEmpty
}

➤ Build and run the app in Simulator.

When you drop the transferred items, the view will print the location to the debug console for later use. A new task will start that adds the elements to the card.

Pasting From Another App

Skills you’ll learn in this section: Cut and paste

Once you’ve set up your CustomTransfer, as well as dragging photos from another app, you can instead copy them and paste them on your card.

SwiftUI provides PasteButton for this. It doesn’t allow a lot of customization, and you can’t add it to a Menu, but it is easy to implement.

➤ Open CardToolbar.swift.

This is where you place your toolbar items.

➤ Add a new item inside toolbar(content:):

ToolbarItem(placement: .navigationBarLeading) {
  PasteButton(payloadType: CustomTransfer.self) { items in
    Task {
      card.addElements(from: items)
    }
  }
}

You’ve now implemented paste in your app. I told you it was easy! PasteButton will be disabled unless it detects a CustomTransfer item. Then when you tap Paste, the items will be added to your card in the same way as the drop.

➤ Build and run the app on an iPad simulator with Safari in split screen and choose a card. Long press an image in Safari, and choose Copy.

The image is now in the pasteboard (also known as clipboard) ready to paste. Copy-and-paste will also work with text.

➤ Tap Paste a few times to add copies of the image to your card.

➤ Add these modifiers to PasteButton(payloadType:):

.labelStyle(.iconOnly)
.buttonBorderShape(.capsule)

This removes the word “Paste” leaving only the icon and gives the button a capsule shape. The paste button is now a little less obtrusive, but the design still doesn’t fit well.

Adding a Pop-up Menu

Skills you’ll learn in this section: Pop-up menu; context menu; UIPasteBoard; remove from array

As you build up your app, you’ll probably want to add a few extra buttons for operations that don’t really need to be always on screen. You can add a pop-up menu for all these operations. Unfortunately, PasteButton won’t work on this menu, so you’ll use a Button which updates UIKit’s UIPasteboard.

➤ Replace the PasteButton ToolbarItem with:

ToolbarItem(placement: .navigationBarTrailing) {
  menu
}

➤ This toolbar item will be more complicated than the previous one, so add a new property to CardToolbar:

var menu: some View {
  // 1
  Menu {
    Button {
      // add action here
    } label: {
      Label("Paste", systemImage: "doc.on.clipboard")
    }
    // 2
    .disabled(!UIPasteboard.general.hasImages
      && !UIPasteboard.general.hasStrings)
  } label: {
    Label("Add", systemImage: "ellipsis.circle")
  }
}

There are a couple of things to note here:

1. You add a Menu to the top toolbar just to the left of the Done button. A Menu is a list of buttons. For this app, you’ll only have one button, but you can very easily add more under the Paste button.
2. You only want the paste button to be enabled when there is something to paste, so you check hasImages and hasStrings. If both are false, you disable the button.

➤ Build and run the app and tap the ellipsis.

Your paste button shows up on the menu.

➤ Back in CardToolbar.swift, in menu, replace // add action here with:

if UIPasteboard.general.hasImages {
  if let images = UIPasteboard.general.images {
    for image in images {
      card.addElement(uiImage: image)
    }
  }
} else if UIPasteboard.general.hasStrings {
  if let strings = UIPasteboard.general.strings {
    for text in strings {
      card.addElement(text: TextElement(text: text))
    }
  }
}

You can check whether the pasteboard contains images or strings. Apple’s documentation states not to test images or strings to see whether they contain data, but to check hasImages and hasStrings.

➤ Build and run your app on iPad with Safari in split screen. Then, try copying and pasting text and images.

When pasting from another app, iOS will ask permission whether to paste.

Note: Apple’s Universal Clipboard is very powerful. For example, if you run Cards on a device, you can select and copy photos in the macOS Photos app and paste them into Cards on the device.

Copying Elements

You can copy from other apps, so it makes sense to implement copying elements within your own app.

You do this with contextMenu(menuItems:) modifiers on card elements. You activate the context menu with a long press, just as you did when you copied from Safari. When you choose Copy from the context menu, the system will add the element — text or image — to the pasteboard. You can then paste the text or image in your app, or even in another app.

➤ Open CardDetailView.swift.

If you add a context menu with several buttons to CardElementView(element:), the view will get over-complicated. Instead of adding the context menu here, you’ll create it in a new view modifier file.

➤ In the SingleCardViews group, create a new Swift file called ElementContextMenu.swift and replace the code with:

import SwiftUI

struct ElementContextMenu: ViewModifier {
  @Binding var card: Card
  @Binding var element: CardElement

  func body(content: Content) -> some View {
    content
  }
}

The context menu will need access to the current card and current element. Creating a view modifier should be familiar to you from Chapter 14, “Gestures”, when you created resizableView().

➤ Add a new modifier to content:

.contextMenu {
  Button {
    if let element = element as? TextElement {
      UIPasteboard.general.string = element.text
    } else if let element = element as? ImageElement,
      let image = element.uiImage {
        UIPasteboard.general.image = image
    }
  } label: {
    Label("Copy", systemImage: "doc.on.doc")
  }
}

The context menu will pop up when you perform a long press on a card element. When you tap Copy, the pasteboard will record the text or image element details ready for pasting elsewhere.

Your modifier is ready to use, but, as you did with ResizableView, you should make it easier to use.

➤ Add this to the end of ElementContextMenu.swift.

extension View {
  func elementContextMenu(
    card: Binding<Card>,
    element: Binding<CardElement>
  ) -> some View {
    modifier(ElementContextMenu(
      card: card,
      element: element))
  }
}

This extension to View simply calls your new modifier with a card and an element value.

➤ Open CardDetailView.swift, and, in body, add this code to CardElementView(element: element) as the first modifier:

.elementContextMenu(
  card: $card,
  element: $element)

You have now added a new context menu to each element that you can access with a long press on that element. You must place the modifier before the following ones so that the context menu appears in the correct place on the screen.

➤ Build and run the app and experiment with copying elements and pasting them in other cards, or even in other apps. Even when copying the element in Simulator, you can paste it into another macOS app.

Deletion

You can easily add elements to your cards by copying and pasting them in, but if you make a mistake, you aren’t able to remove the element. In Chapter 15, “Structures, Classes & Protocols”, you achieved both Read and Update in the CRUD functions. Next, you’ll take on Deletion.

You’ll add an entry to the context menu. When you tap the menu item, your app will remove the selected card element from the card’s array.

➤ Open Card.swift and add this code to Card:

mutating func remove(_ element: CardElement) {
  if let index = element.index(in: elements) {
    elements.remove(at: index)
  }
}

Here you retrieve the index of the card element. You then remove the element from the array using the index.

➤ Open ElementContextMenu.swift and add a new button to the context menu:

Button(role: .destructive) {
  card.remove(element)
} label: {
  Label("Delete", systemImage: "trash")
}

Your delete button should be highlighted as dangerous, and that’s what the destructive role does for you. The menu item will be in red.

➤ Live Preview Single Card View, add a photo to the card, and then, long press the photo.

You’ll see the context menu pop up.

➤ Tap Delete to delete the element, or tap away from the menu if you decide not to delete it.

In summary, when you delete the element, you delete it from card.elements. card is bound to cards in the data store, and cards is a published property. When cards changes, all views containing cards will redisplay their content.

Challenge

Challenge: Delete a Card

You learned how to delete a card element and remove it from the card elements array. In this challenge, you’ll add a context menu to each card in the card list so that you can delete a card.

1. In CardStore, create a similar remove method as the one in Card to remove a card from the cards array.

2. In CardsListView, add a new context menu to a card with a delete option that calls your new method to remove the card.

You’ll find the solution to this challenge in the challenge folder for this chapter.

Key Points

• Instead of having to implement your own photos picker view, Apple provides the PhotosUI framework with a PhotosPicker view. It’s an easy way to select photos and videos from the photo library.
• Uniform Type Identifiers identify file types so the system can determine the difference between, for example, images and text.
• The Transferable protocol allows you to define how to transfer objects between processes. You use Transferable for drag and drop, pasting and sharing. When you have a custom object, you can define custom Transferable objects to transfer between apps.
• A Menu is a list of Buttons. Each Button can have a role. By making the role destructive, the menu item will appear in red.
• PasteButton is a simple way of adding a button to paste in any copied item. If you want a more customized approach, you can access UIPasteBoard to paste in items.
• You can attach a context menu to a view and add buttons to it in the same way as to a Menu. You access the context menu by a long press. SwiftUI brings the view to the foreground and darkens the other views. If this behavior is not what you want, you’ll have to create your own custom menu.