Vision Framework Tutorial for iOS: Contour Detection

Processing Images in the Detector

Now you just need to tie these two private methods together somewhere that is callable from outside the class. Still in ContourDetector.swift, add the following method:

func process(image: CGImage?) throws -> [Contour] {
  guard let image = image else {
    return []
  }
    
  let contourRequest = try perform(request: request, on: image)
  return postProcess(request: contourRequest)
}

Here you're checking if there's an image, then using perform(request:on:) to create a request, and finally returning the result using postProcess(request:). This will be the method your view model will call to detect contours for an image, which is exactly what you'll do next.

Open ContentViewModel.swift and add the following method to the end of the class:

func asyncUpdateContours() async -> [Contour] {
  let detector = ContourDetector.shared
  return (try? detector.process(image: self.image)) ?? []
}

In this code, you're creating an asynchronous method to detect contours. Why asynchronous? Although detecting contours is generally relatively quick, you still don't want to tie up the UI while waiting for the API call results. The asynchronous method returns an empty array if the detector doesn't find any contours. Also, spoiler alert, you'll add a lot more logic here later, which will tax your device's processor. :]

However, you still need to call this method from somewhere. Find the method stub for updateContours, and fill it in with the following code:

func updateContours() {
  // 1
  guard !calculating else { return }
  calculating = true
  
  // 2
  Task {
    // 3
    let contours = await asyncUpdateContours()
    
    // 4
    DispatchQueue.main.async {
      self.contours = contours
      self.calculating = false
    }
  }
}

With this code, you:

1. Do nothing if we're already calculating contours. Otherwise set a flag to indicate that you're calculating contours. The UI will then be able to inform the user, so they remain patient.
2. Create an asynchronous context, from which to run the contour detector. This is necessary for asynchronous work.
3. Kick off the contour detection method and await its results.
4. Set the results back on the main thread and clear the calculating flag. Since both contours and calculating are published properties, they should only be assigned on the main thread.

This update method needs to be called from somewhere and the bottom of init is as good a place as any! Find init and add the following line to the bottom:

updateContours()

It's now time to build and run your app. After the app loads and you see the image, tap the screen to show its detected contours using the default settings.

Great job!

VNContoursObservation and VNContour

At the time of writing, a VNDetectContoursObservation never seems to return more than one VNContoursObservation in the results array. Instead, all of the contours you see, which is a total of 43 in the previous screenshot, are referenced by the single VNContoursObservation.

Note: The code you wrote handles multiple VNContoursObservation results, just in case Apple ever decides to change how this works.

Each individual contour is described by a VNContour and is organized hierarchically. A VNContour can have child contours. To access them, you have two options:

1. Index the childContours property, which is an array of VNContours.
2. Use the childContourCount integer property in conjunction with the childContour(at: Int) method to loop through and access each child contour.

As any VNContour can have a child VNContour, you'll have to recursively access them if you need to preserve the hierarchal information.

If you don't care about the hierarchy, VNContoursObservation gives you an easy way to access all contours in simple manner. A VNContoursObservation has a contourCount integer property and a contour(at: Int) method to access all contours as if they were a flat data structure.

However, if hierarchy is important to you, you need to access the topLevelContours property, which is an array of VNContours. From there, you can access each contour's child contours.

If you were to write some simple code to count top-level and child contours, you'd find that the sample image, with default settings, has four top-level contours and 39 child contours, for a total of 43.

VNDetectContoursRequest Settings

So far, you've created a VNDetectContoursRequest without experimenting with the various settings available to you. Currently, there are four properties you can change to achieve different results-

1. contrastAdjustment: The algorithm has a built-in way to adjust the contrast of the image prior to performing contour detection. Adjusting the contrast tries to darken the dark parts of the image and lighten the light parts to exaggerate their differences. This float property ranges from 0.0 to 3.0, with a default value of 2.0. The higher the value, the more contrast will be applied to the image, making it easier to detect some contours.
2. contrastPivot: How does the algorithm know what part of the image should be considered dark vs. light? That's where the contrast pivot comes in. It's an optional NSNumber property ranging from 0.0 to 1.0, with a default of 0.5. Any pixels below this value will be darkened, and any pixels above will be lightened. You can also set this property to nil to have the Vision framework automatically detect what this value "should" be.
3. detectsDarkOnLight: This boolean property is a hint to the contour detection algorithm. The default is set to true, which means it should look for dark objects on a light background.
4. maximumImageDimension: Since you can pass in any size image to the request handler, this integer property lets you set the maximum image dimension to use. If your image has a dimension larger than this value, the API scales the image such that the larger of the two-dimensions will be equal to maximumImageDimension. The default value for this property is 512. Why would you want to change this? Contour detection requires quite a bit of processing power - the larger the image, the more it needs. However, the larger the image, the more accurate it can be. This property allows you to fine-tune this trade-off for your needs.