Chapters
Chapters

Chapters

Hide chapters

Metal by Tutorials

Second Edition · iOS 13 · Swift 5.1 · Xcode 11

Before You Begin

Section 0: 3 chapters
Show chapters Hide chapters

Section I: The Player

Section 1: 8 chapters
Show chapters Hide chapters

Section II: The Scene

Section 2: 7 chapters
Show chapters Hide chapters

Section III: The Effects

Section 3: 10 chapters
Show chapters Hide chapters

15. GPU-Driven Rendering
Written by Caroline Begbie

So far, you’ve created an engine where you can load complex models with textures and materials, animate or update them per frame and render them. Your scenes will start to get more and more complicated as you develop your game, and you’ll want to find more performant ways of doing things.

In this chapter, you’ll take a simple scene and instead of organizing the render command codes on the CPU on each frame, you’ll set up a list of all the commands before you even start the rendering loop. Along the way, you’ll learn about argument buffers, resource heaps and indirect command buffers. Finally you’ll move the command list creation to the GPU, and have a fully GPU-driven pipeline.

As you progress through the chapter, you may not see the immediate gains. However, once you’ve centralized your rendering at the end of the chapter, your app will have more complex initial loading, but much simpler rendering.

If you feel that you haven’t spent enough time so far digging around inside buffers and examining memory, then this is the chapter for you.

Note: Indirect command buffers are supported by: iOS - Apple A9 devices and up; iMacs - models from 2015; and MacBook and MacBook Pro - models from 2016. As you’re going to be delving deep into using the hardware directly, much of this chapter won’t work on the iOS simulator.

Argument buffers

In previous chapters, you sent up to five textures to the fragment shader - the base color, normals, roughness, metalness and ambient occlusion textures. During the frame render loop, each of these incurs a renderEncoder.setFragmentTexture(texture:at:) command. Using argument buffers, you can group five pointers to these five textures into one buffer, and set this buffer on the render command encoder with just one command. This argument buffer doesn’t only have to contain textures, it can contain any other data necessary to render the frame.

When you come to draw time, instead of setting the five fragment textures on the render command encoder, you set the single argument buffer. You then perform renderEncoder.useResource(_:usage:) for each texture, which places all five textures onto the GPU as indirect resources.

Once you’ve set up an argument buffer, you can refer to it in a shader, using a struct as a parameter to the shader function.

The starter project

With the concepts under your belt, open up the starter project for this chapter and examine the project. The project is minimal and it’s not the most exciting scene, but the two models have very few vertices, and you can examine the loaded model buffers more easily. The barn has a color texture, and the grass has color and normal textures.

Model loading takes place, as usual, in Model. For simplicity, all rendering takes place in Renderer’s draw(in:).

  1. Model only loads one mesh and one submesh, with two textures - the color and normal. You hold the minimum objects needed to render each model:
  • a pipeline state object
  • a vertex buffer mesh
  • an index buffer
  • a color texture and optionally a normal texture
  • model parameters - a model matrix and tiling for the fragment textures

  1. Uniforms are split up between per-frame constants, being the projection and view matrix, and per-model constants, being the model matrix and tiling. You’ll see the reason for this later in the chapter.

  2. The focus of this chapter will be shader function parameters and data passed to the GPU. As a result, the bodies of the two shader functions in Shaders.metal are extremely simple. The normal texture is not used in the fragment shader - it’s included simply as illustration of the concept of passing multiple textures.

In Shaders.metal, the fragment function has two parameters for textures. You’re going to combine both of these into one struct, and use the struct as the parameter.

Create the struct

Combine both textures into a new struct. Add this before fragment_main:

struct Textures {
  texture2d<float> baseColorTexture;
  texture2d<float> normalTexture;
};

This struct currently holds only two textures, but could be a list of all the textures you need for rendering the model, including roughness, metallic, etc. It could also include other data buffers such as materials.

You can now replace the two textures in the fragment function header with a parameter that points to this struct. Replace:

texture2d<float> baseColorTexture [[texture(BaseColorTexture)]],
texture2d<float> normalTexture [[texture(NormalTexture)]],

With:

constant Textures &textures [[buffer(BufferIndexTextures)]],

Instead of receiving the textures directly, you’ll just receive one struct containing all the textures needed for the model.

Change the assignment to baseColor to:

float3 baseColor = 
  textures.baseColorTexture.sample(textureSampler,
                            in.uv * modelParams.tiling).rgb;

Create the argument buffer

To pass these textures, you create an argument buffer that matches the struct.

Open Model.swift and add a new argument buffer property:

var texturesBuffer: MTLBuffer!

This is a simple MTLBuffer which will contain pointers to the textures. In this chapter, for brevity’s sake, you’ll use implicitly unwrapped optionals on all of your buffers. In a real world app, you should include error checking.

Create a new method to create the argument buffer. You’re creating it in a method so that it’s easier to change the code later in the chapter.

func initializeTextures() {
  // 1
  let textureEncoder = fragmentFunction.makeArgumentEncoder(
    bufferIndex: Int(BufferIndexTextures.rawValue))
  // 2
  texturesBuffer =
    Renderer.device.makeBuffer(
            length: textureEncoder.encodedLength,
            options: [])!
  texturesBuffer.label = "Textures"
  //3
  textureEncoder.setArgumentBuffer(texturesBuffer, offset: 0)
  textureEncoder.setTexture(colorTexture, index: 0)
  if let normalTexture = normalTexture {
    textureEncoder.setTexture(normalTexture, index: 1)
  }
}

Going through the code:

  1. init(name:) initializes fragmentFunction with the Metal library function fragment_main. You create a texture encoder using this fragment function and use the same buffer index that you used for your textures struct in fragment_main. The encoder refers to this argument and can work out how many elements there are in the struct, and therefore how long the length of the new argument buffer should be.
  2. Create an MTLBuffer of the texture encoder’s length. With the added complexity that you’ll have with many buffers, it’s sensible to label each buffer as you create it.
  3. Set the argument buffer and the textures on the texture encoder. During the render loop, setting the textures on the render command encoder incurs some internal verifying of the textures. This verifying will now take place here, when the textures are initially set into the texture encoder. Anything you can move outside of the render loop is a gain.

In init(name:), call this method after super.init() to initialize the argument buffer:

initializeTextures()

You’ve now set up two textures into an argument buffer. Instead of sending the textures to the fragment shader during the render loop, you’ll send this single argument buffer.

The draw call

In Renderer.swift, in draw(in:), change:

renderEncoder.setFragmentTexture(model.colorTexture,
                        index: Int(BaseColorTexture.rawValue))
renderEncoder.setFragmentTexture(model.normalTexture,
                        index: Int(NormalTexture.rawValue))

To:

renderEncoder.setFragmentBuffer(model.texturesBuffer, 
                       offset: 0, 
                       index: Int(BufferIndexTextures.rawValue))

Here you send the textures argument buffer to the GPU. BufferIndexTextures is the index that links the argument buffer between the render encoder and the fragment function.

Build and run. The render may look exactly the same as before, although on the iPhone, you will probably receive many errors. Click the Capture GPU frame icon, and take a look behind the scenes. The result of the capture will resemble this:

It may not be an exact replica, because when you have GPU memory errors, weird things can happen on the display. Debugging GPU errors can be frustrating when your display locks up because you have accessed memory that you’re not supposed to.

With the second draw call under plane.obj selected, choose Automatic ▸ Attachments from the top left navigator icon. You’ll see that the textures aren’t rendered correctly. Choose Bound Resources to show the list of resources currently on the GPU.

The following image shows Bound Resources on the left and Attachments on the right.

Under Fragment, double click the Textures buffer. Textures is the name of the label that you set on your model’s argument buffer. You’ll see that neither of these textures is a valid texture.

You’ve set up a level of indirection with the argument buffer pointing to the textures, but you still have to tell the GPU to load these textures. When dealing with indirection and buffer data, it’s often easy to omit this vital step, so if you have errors at any time, check in the GPU debugger that the resource is available in the indirect resource list, but also check that you are using the resource in the render command encoder command list.

Back in Renderer.swift, under the previous code where you set the textures argument buffer, add this:

if let colorTexture = model.colorTexture {
  renderEncoder.useResource(colorTexture, usage: .read)
}
if let normalTexture = model.normalTexture {
  renderEncoder.useResource(normalTexture, usage: .read)
}

This tells the GPU that you’re going to read from these textures. Build and run again, and in the GPU debugger, check that the Textures buffer now points to the textures:

You’ll also see that your textures are listed under Indirect Resources, and are available for any shader function to use:

You’ve now set up your app to use argument buffers for the textures instead of sending them individually. This may not feel like a win yet, and you’ve increased overhead by adding a new buffer. But you’ve reduced overhead on the render command encoder. Instead of having to validate the textures each frame, the textures are validated when they are first placed into the argument buffer, while you’re still initializing your app data. In addition to this, you’re grouping your textures together into the one struct, and the one parameter to the fragment function. If you have many parameters that you can group together, this will save time too.

Resource heaps

You’ve grouped textures into an argument buffer, but you can also combine all your app’s textures into a resource heap.

A resource heap is simply an area of memory where you bundle resources. These can be textures or data buffers. To make your textures available on the GPU, instead of having to perform renderEncoder.useResource(_:usage:) for every single texture, you can perform renderEncoder.useHeap(_:) once per frame instead. That’s one step further in the quest for reducing render commands.

You’re going to create a TextureController which takes care of all your app’s textures and the texture heap.

Note: For simplicity, you’ll create TextureController as a singleton, but you could easily hold it as an instance per scene in a larger app. As an added feature of centralizing textures into the texture controller, there’s a performance improvement. You may have previously loaded some of Apple’s usdz sample models from https://developer.apple.com/augmented-reality/quick-look/. These are split up into many submeshes, and so far, the engine code loads one texture per submesh. As a result, each of the sample models takes up a huge amount of memory. You could create a method that checks whether Texture Controller already holds a texture, and so load a texture only once, no matter how many submeshes refer to it.

Create a new Swift file called TextureController.swift. Don’t forget to add it to both the iOS and macOS targets. Replace the code with:

import MetalKit

class TextureController {
  static var textures: [MTLTexture] = []
}

TextureController will round up all the textures used by your models and hold them in an array. Model, instead of holding a reference to a texture, will hold an index into this texture array.

Add a new method to TextureController:

static func addTexture(texture: MTLTexture?) -> Int? {
  guard let texture = texture else { return nil }
  TextureController.textures.append(texture)
  return TextureController.textures.count - 1
}

Here, you receive a texture, add it to the central texture array and return an index to the texture.

In Model.swift, change:

let colorTexture: MTLTexture?
let normalTexture: MTLTexture?

To:

let colorTexture: Int?
let normalTexture: Int?

Instead of holding the texture on Model, you’ll hold the index to the texture held in TextureController’s textures array. Your code won’t compile until you’ve changed all the places where you refer to these textures.

In init(name:), replace:

colorTexture = textures.baseColor
normalTexture = textures.normal

With:

colorTexture = 
    TextureController.addTexture(texture: textures.baseColor)
normalTexture = 
    TextureController.addTexture(texture: textures.normal)

Also, in initializeTextures(), update the argument buffer code that doesn’t compile, to reference the correct textures:

if let index = colorTexture {
  textureEncoder.setTexture(TextureController.textures[index], 
                            index: 0)
}
if let index = normalTexture {
  textureEncoder.setTexture(TextureController.textures[index], 
                            index: 1)
}

In Renderer.swift, in draw(in:), change:

if let colorTexture = model.colorTexture {
  renderEncoder.useResource(colorTexture, usage: .read)
}
if let normalTexture = model.normalTexture {
  renderEncoder.useResource(normalTexture, usage: .read)
}

To:

if let index = model.colorTexture {
  renderEncoder.useResource(TextureController.textures[index], 
                            usage: .read)
}
if let index = model.normalTexture {
  renderEncoder.useResource(TextureController.textures[index], 
                            usage: .read)
}

Your code should compile again, so build and run to make sure everything still works. Your textures should render, only now they are all held centrally in TextureController. In this app, there’s no performance gain here, but if you use this technique on models that have many submeshes accessing the same texture, it will be a huge memory usage reduction.

This modification means that instead of sending a texture to the GPU per model, you’re ready to gather up all the textures into a heap and move the whole heap at one time to the GPU.

In TextureController, create a new property:

static var heap: MTLHeap?

Create a new type method to build the heap:

static func buildHeap() -> MTLHeap?  {
  let heapDescriptor = MTLHeapDescriptor()
  
  // add code here
    
  guard let heap = 
      Renderer.device.makeHeap(descriptor: heapDescriptor)
    else { fatalError() }
  return heap
}

You build a heap from a heap descriptor. This descriptor will need to know the size of all the textures. Unfortunately MTLTexture doesn’t hold that information, but you can retrieve the size of a texture from a texture descriptor.

In Extensions.swift, there’s an extension on MTLTextureDescriptor that will provide a descriptor based on an MTLTexture.

In TextureController.swift, in buildHeap(), add this under // add code here

let descriptors = textures.map { texture in
  MTLTextureDescriptor.descriptor(from: texture)
}

Here, you create an array of texture descriptors to match the array of textures. Now you can add up the size of all these descriptors. Following on from the previous code, add this:

let sizeAndAligns = descriptors.map { 
  Renderer.device.heapTextureSizeAndAlign(descriptor: $0)
}
heapDescriptor.size = sizeAndAligns.reduce(0) { 
  $0 + $1.size - ($1.size & ($1.align - 1)) + $1.align
}
if heapDescriptor.size == 0 {
  return nil
}

Here you calculate the size of the heap using size and correct alignment within the heap. As long as align is a power of two, (size & (align - 1)) will give you the remainder when size is divided by alignment. For example, if you have a size of 129 bytes, and you want to align it to memory blocks of 128 bytes, this is the result of $1.size - ($1.size & ($1.align - 1)) + $1.align:

129 - (129 & (128 - 1)) + 128 = 256

This result shows that if you want to align blocks to 128, you’ll need a 256 byte block to fit 129 bytes.

You have an empty heap, but you need to populate it with the textures. You’ll iterate through the texture array and create a new array of textures that you’ll store in the heap. For each texture you’ll create a new texture resource and then copy the original texture contents to the new resource using a blit command encoder.

At the end of the method, but before the return, add this:

let heapTextures = descriptors.map { descriptor -> MTLTexture in
  descriptor.storageMode = heapDescriptor.storageMode
  return heap.makeTexture(descriptor: descriptor)!
}

Here you create an array of new texture resources using the texture descriptors. These are empty texture resources, so you need to copy the model texture information to the heap texture resources.

Add this to create a blit command encoder:

guard 
  let commandBuffer = Renderer.commandQueue.makeCommandBuffer(),
  let blitEncoder = commandBuffer.makeBlitCommandEncoder() 
  else {
    fatalError()
  }

You’ve used the blit command encoder to do fast copies of memory several times previously. You’ll copy each texture to the heap texture. Add this code to do the copy:

zip(textures, heapTextures).forEach { (texture, heapTexture) in
  var region = MTLRegionMake2D(0, 0, texture.width, 
                               texture.height)
  for level in 0..<texture.mipmapLevelCount {
    for slice in 0..<texture.arrayLength {
      blitEncoder.copy(from: texture,
                       sourceSlice: slice,
                       sourceLevel: level,
                       sourceOrigin: region.origin,
                       sourceSize: region.size,
                       to: heapTexture,
                       destinationSlice: slice,
                       destinationLevel: level,
                       destinationOrigin: region.origin)
    }
    region.size.width /= 2
    region.size.height /= 2
  }
}

Here you copy each texture to a heap texture. Within each texture, you copy each level and slice. Levels contain the texture mipmaps, which is why you halve the region each loop, and slices will contain texture arrays, if there are any.

Before returning the heap from the method, add the following:

blitEncoder.endEncoding()
commandBuffer.commit()
TextureController.textures = heapTextures

This ends the encoding, commits the command buffer and replaces the original textures with the heap textures. Models will now point to a heap texture with their texture index, instead of to the original texture. The models’ texture argument buffers, however, point to textures that no longer exist. You’ll fix that up in a moment.

You’ve created a class method to create the heap. To use this method, open Renderer.swift. Create a new method to initialize the heap. You’ll add other initializations to this method shortly.

func initialize() {
  TextureController.heap = TextureController.buildHeap()
}

Call this new method at the end of init(metalView:)

initialize()

Now you’ve created a texture controller, and centralized the texture allocations. Before rendering any models, you can send the textures to the GPU at the start of the frame to be all ready and waiting for processing.

In draw(in:), remove:

if let index = model.colorTexture {
  renderEncoder.useResource(TextureController.textures[index], 
                            usage: .read)
}
if let index = model.normalTexture {
  renderEncoder.useResource(TextureController.textures[index], 
                            usage: .read)
}

At the top of draw(in:), after setting the depth stencil state on the render command encoder, add this:

if let heap = TextureController.heap {
  renderEncoder.useHeap(heap)
}

Instead of having a useResource command for every texture, you perform one useHeap every frame. This could be a huge saving on the number of commands in a render command encoder, and so a reduction of the number of commands that a GPU has to process each frame.

The models’ argument buffers are still pointing to the old texture, and not the new heap texture, so you need to re-initialize the argument buffers.

Still in Renderer.swift, at the end of initialize(), add this:

models.forEach { model in
  model.initializeTextures()
}

This will call the method you wrote earlier to create the argument buffer and set the textures. In Model.swift, remove initializeTextures() from the end of init(name:).

Build and run, and your render should be exactly the same as it was.

Take a look at the GPU debugger via the Capture GPU frame button.

Under CommandBuffer \ RenderCommandEncoder, select the useHeap command that you set at the start of the frame. Select Bound Resources using the navigator icon at the top left of the pane. The three textures in the heap are at this point available in Indirect Resources for any draw call to use. There’s one color texture for the barn, and a color and normal texture for the grass.

Locate the drawIndexedPrimitives command under plane.obj, and, in the Bound Resources, under Fragment, locate Textures. Double click Textures, and you’ll see a pointer to the grass texture. Click on the arrow, and it will show you the texture. As before, you can check the mipmaps at the bottom left of that pane.

You’ve now separated out your textures from your rendering code, with a level of indirection via the argument buffer. But have you seen any performance improvement? In this example, probably not. But the more textures you load, the better the improvement, as there will be fewer render commands.

In addition, your app is more flexible and you can schedule more complicated events, which will have an overall impact on performance.

Indirect Command Buffers

You’ve created several levels of indirection with your textures by using an argument buffer and a heap, but you can also create indirection with commands on command encoders.

At the start of this chapter, your rendering process was this:

You loaded all the model data, materials and pipeline states at the start of the app. Each frame, you created a render command encoder and issued commands to that encoder, ending with a draw call.

But instead of creating these commands per frame, you can create them all at the start of the app using an indirect command buffer with a list of commands. You’ll set up each command with pointers to the relevant uniform, texture and vertex buffers, and specify how to do the draw.

With that one initialization process, at the start of the app, to set up the per-frame command list, during the render loop, you can just issue one execute command to the render command encoder, and the encoder will send the list of commands, all at once, off to the GPU .

Your process will then look like this:

Remember that your aim is to do as much as you can when your app first loads, and as little as you have to per frame. To achieve this, you’ll:

  1. Place all your uniform data in buffers. As the indirect commands need to point to buffers at the start of the app, you can’t send ad hoc bytes to the GPU. You can still update the buffers each frame. For the model constants containing the model matrix for the model, you’ll hold an array of model constants and update the model matrices for all models at the start of each frame.
  2. Set up an Indirect Command Buffer. This buffer will hold all the draw commands.
  3. Loop through the models and set up the indirect commands.
  4. Clean up the render loop and use the resources you referred to in the indirect commands to send them to the GPU.
  5. Change the shader functions to use the array of model constants.
  6. Execute the command list.

1. Uniform buffers

In Renderer.swift, create three new properties to hold the uniforms and model constants in buffers:

var uniformsBuffer: MTLBuffer!
var fragmentUniformsBuffer: MTLBuffer!
var modelParamsBuffer: MTLBuffer!

Add this at the end of initialize():

var bufferLength = MemoryLayout<Uniforms>.stride
uniformsBuffer = 
  Renderer.device.makeBuffer(length: bufferLength, options: [])
uniformsBuffer.label = "Uniforms"
bufferLength = MemoryLayout<FragmentUniforms>.stride
fragmentUniformsBuffer = 
  Renderer.device.makeBuffer(length: bufferLength, options: [])
fragmentUniformsBuffer.label = "Fragment Uniforms"
bufferLength = models.count * MemoryLayout<ModelParams>.stride
modelParamsBuffer = 
  Renderer.device.makeBuffer(length: bufferLength, options: [])
modelParamsBuffer.label = "Model Parameters"

Here, you set up three empty buffers ready to take the uniform data. draw(in:) calls updateUniforms() at the start of each frame, and that’s where you’ll update the contents of these buffers.

Add this code to the end of updateUniforms().

// 1
var bufferLength = MemoryLayout<Uniforms>.stride
uniformsBuffer.contents().copyMemory(from: &uniforms,
                                     byteCount: bufferLength)
bufferLength = MemoryLayout<FragmentUniforms>.stride
fragmentUniformsBuffer.contents().copyMemory(
             from: &fragmentUniforms,
             byteCount: bufferLength)

// 2
var pointer = 
  modelParamsBuffer.contents().bindMemory(to: ModelParams.self,
                                  capacity: models.count)
// 3
for model in models {
  pointer.pointee.modelMatrix = model.modelMatrix
  pointer.pointee.tiling = model.tiling
  pointer = pointer.advanced(by: 1)
}

Going through this code:

  1. You copy the uniforms and fragment uniforms data from the struct to the MTLBuffers.
  2. For the model data, you’ll need to iterate through the models to get each model matrix, so you bind a pointer to the buffer.
  3. Iterate through the models and fill the buffer.

2. Indirect command buffer

You’re now ready to create some indirect commands. Take a look at draw(in:) to refresh your memory on all the render commands that you set in the rendering for loop. You’re going to move all these commands to an indirect command list. You’ll set up this command list at the start of the app, and simply call executeCommandsInBuffer on the render command encoder each frame. This will execute the entire command list with just that one command.

At the top of Renderer, create a property for the Indirect Command Buffer (ICB) .

var icb: MTLIndirectCommandBuffer!

Create a new method where you’ll build up the command list in an indirect command buffer:

func initializeCommands() {
  let icbDescriptor = MTLIndirectCommandBufferDescriptor()
  icbDescriptor.commandTypes = [.drawIndexed]
  icbDescriptor.inheritBuffers = false
  icbDescriptor.maxVertexBufferBindCount = 25
  icbDescriptor.maxFragmentBufferBindCount = 25
  icbDescriptor.inheritPipelineState = false
}

Here you create an Indirect Command Buffer descriptor. You specify that (eventually) the GPU should expect an indexed draw call. That’s a draw call that uses an index buffer for indexing into the vertices. You set the maximum number of buffers that the ICB can bind to in the vertex and fragment shader parameters to 25. This is far too many, but it’s somewhere to start.

You set inheritPipelineState to false. In simple apps, like this one, that only use one vertex shader and one fragment shader, you could set one pipeline state at the start of the frame. In that case you’d set inheritPipelineState to true. In this app, though, you’ll find out how to set a different pipeline state for each draw call, so you won’t be inheriting a pipeline state.

Following on from that code, create the indirect command buffer:

guard let icb =
  Renderer.device.makeIndirectCommandBuffer(
    descriptor: icbDescriptor,
    maxCommandCount: models.count,
    options: []) 
  else { fatalError() }
self.icb = icb

The ICB will need one command per draw call. In this app, you’re only performing one draw call per model, but in a more complex app where you’re doing a draw call for every submesh, you’d have to iterate through the models prior to setting up the ICB to find out how many draw calls you’ll do.

3. Indirect commands

Now that you’ve set up an indirect command buffer, you’ll add the list of commands to it. Add this to initializeCommands()

for (modelIndex, model) in models.enumerated() {
  let icbCommand = icb.indirectRenderCommandAt(modelIndex)
  icbCommand.setRenderPipelineState(model.pipelineState)
  icbCommand.setVertexBuffer(uniformsBuffer, offset: 0,
    at: Int(BufferIndexUniforms.rawValue))
  icbCommand.setFragmentBuffer(fragmentUniformsBuffer, 
    offset: 0,
    at: Int(BufferIndexFragmentUniforms.rawValue))
  icbCommand.setVertexBuffer(modelParamsBuffer, offset: 0,
    at: Int(BufferIndexModelParams.rawValue))
  icbCommand.setFragmentBuffer(modelParamsBuffer, offset: 0,
    at: Int(BufferIndexModelParams.rawValue))
  
  icbCommand.setVertexBuffer(model.vertexBuffer, offset: 0,
    at: Int(BufferIndexVertices.rawValue))
  icbCommand.setFragmentBuffer(model.texturesBuffer, offset: 0,
    at: Int(BufferIndexTextures.rawValue))
}

This may look familiar to you from the render loop in draw(in:). You use the model index to keep track of the command list, and you set all the necessary data for each draw call.

The one thing missing is the actual draw call. Add this to the end of the for loop:

icbCommand.drawIndexedPrimitives(.triangle,
  indexCount: model.submesh.indexCount,
  indexType: model.submesh.indexType,
  indexBuffer: model.submesh.indexBuffer.buffer,
  indexBufferOffset: model.submesh.indexBuffer.offset,
  instanceCount: 1,
  baseVertex: 0,
  baseInstance: modelIndex)

This draw command is very similar to the one that you’ve already been using. There are a couple of extra arguments:

  • baseVertex - the vertex in the vertex buffer to start rendering from.
  • baseInstance - the instance to start rendering from. You have set up an array of modelParams, one for each model. Using baseInstance, in the shader, you can index into the array to get the correct element.

The command list is now complete. Call this method at the end of init(metalView:):

initializeCommands()

4. Update the render loop

You can now remove most of the render encoder commands from draw(in:). Remove all the code after setting the heap down to, but not including, renderEncoder.endEncoding().

draw(in:) should look like this:

func draw(in view: MTKView) {
  guard
    let descriptor = view.currentRenderPassDescriptor,
    let commandBuffer = 
       Renderer.commandQueue.makeCommandBuffer() else {
      return
  }

  updateUniforms()
  guard let renderEncoder =
  commandBuffer.makeRenderCommandEncoder(descriptor: descriptor) 
      else { return }
  renderEncoder.setDepthStencilState(depthStencilState)
  if let heap = TextureController.heap {
    renderEncoder.useHeap(heap)
  }

  renderEncoder.endEncoding()
  guard let drawable = view.currentDrawable else {
    return
  }
  commandBuffer.present(drawable)
  commandBuffer.commit()
}

Build and run to make sure that your app still works after all this time. As you only create the command list but don’t execute it, you should expect to see just a clear blue sky. However, there’s an error:

failed assertion `Setting a pipeline that does not have supportIndirectCommandBuffers = YES is invalid’

When you use a pipeline state in an indirect command list, you have to tell it that it should support indirect command buffers. Open Model.swift, and, in buildPipelineState(vertexFunction:fragmentFunction:), add this before do:

pipelineDescriptor.supportIndirectCommandBuffers = true

Build and run again, and you should get a plain clear background.

Currently none of your resources are making their way to the GPU. To fix this, open Renderer.swift. In draw(in:), before renderEncoder.endEncoding(), add this:

renderEncoder.useResource(uniformsBuffer, usage: .read)
renderEncoder.useResource(fragmentUniformsBuffer, usage: .read)
renderEncoder.useResource(modelParamsBuffer, usage: .read)
for model in models {
  renderEncoder.useResource(model.vertexBuffer, usage: .read)
  renderEncoder.useResource(model.submesh.indexBuffer.buffer, 
                            usage: .read)
  renderEncoder.useResource(model.texturesBuffer, usage: .read)
}

These will move each model’s buffers to the GPU as indirect resources ready for the GPU to access.

5. Update the shader functions

Both vertex_main and fragment_main use modelParams, which holds each model’s matrix and the tiling of textures for the model. You’ve changed the single instance of modelParams to be an array, so now you’ll change the shader functions to match the incoming buffers and access the correct element in the model parameters array.

Open Shaders.metal. In vertex_main, change:

constant ModelParams &modelParams [[buffer(BufferIndexModelParams)]]

To:

constant ModelParams *modelParamsArray 
                  [[buffer(BufferIndexModelParams)]],
uint baseInstance [[base_instance]]

Here you change the parameter to receive an array in the buffer, and also receive the model index using the base_instance attribute. This matches the baseInstance parameter in the draw call.

At the start of vertex_main, add this to get the parameters for the current model:

ModelParams modelParams = modelParamsArray[baseInstance];

To pass the model index to the fragment function, add this to VertexOut:

uint modelIndex [[flat]];

The [[flat]] attribute ensures that the value won’t be interpolated between the vertex and fragment function.

In vertex_main, add the model index to the VertexOut out assignment:

.modelIndex = baseInstance

That’s dealt with the vertex function, now for the fragment function. As you did before, replace:

constant ModelParams &modelParams [[buffer(BufferIndexModelParams)]]

With:

constant ModelParams *modelParamsArray [[buffer(BufferIndexModelParams)]]

Add this to the start of the fragment_main:

ModelParams modelParams = modelParamsArray[in.modelIndex];

That’s fixed up the shader functions so that they’ll read an array of parameters instead of a single instance.

6. Execute the command list

All the code you have written in this chapter so far has been building up to one command. Drum roll….

In Renderer.swift, in draw(in:), add this before renderEncoder.endEncoding:

renderEncoder.executeCommandsInBuffer(icb, 
                                      range: 0..<models.count)

This will execute all the commands in the indirect command buffer’s list within the range specified here. If you specify a range of 0..<1, then only the first draw call would be performed.

Build and run, and you should get the same render as at the start of the chapter. Very little is happening in your render loop, and all the heavy lifting is done at the very start of the app. Success :].

As before, examine everything in the GPU Debugger, and make sure you understand what’s happening. At the bottom of the Bound Resources, you’ll see the Indirect Command Buffer. Double click this to see the two commands listed, one for each model.

Double click each of the resources to see their contents. Textures will only show floats. It’s not until it’s received into fragment_main that the debugger formats it into the Textures struct.

Note: to see the debugger, you may have to run the app on macOS. It doesn’t always work on devices.

GPU driven rendering

You’ve achieved indirect CPU rendering, by setting up a command list and rendering it. However, you can go one better, and get the GPU to create this command list. Open Renderer.swift and take a look at the for loop in initializeCommands().

This for loop executes serially on the CPU, but is one that you can easily parallelize. Each command executes one after another, but by moving this loop to the GPU, you can create each command at the same time over multiple GPU cores.

When you come to write real-world apps, setting up the render loop at the very start of the app is impractical. In each frame you’ll be determining which models to render. Are the models in front of the camera? Is the model occluded by another model? Should you render a model with lower level of detail? By creating the command list every frame, you have complete flexibility in which models you should render, and which you should ignore. As you’ll see, the GPU is amazingly fast at creating these render command lists, so you can include this process each frame.

In Chapter 16, “Particle Systems”, you’re going to learn all about compute programming, or GPGPU (general purpose GPU programming). But you’re going to have a gentle preview in this chapter. Compute is very good at performing multi-threaded tasks in parallel. You may not understand yet how it all works, but all you need to know in this chapter is that you’ll perform a compute shader function using one thread for each model.

This compute shader function is very similar to the vertex and fragment function in that you set up a pipeline state pointing to the compute function. You set buffers on a compute command encoder, just as you do on a render command encoder, to pass data to the compute function. The compute function will need all the data that you used during the initializeCommands() for loop:

  • uniform and model parameter buffers
  • the indirect command buffer

For the models’ vertex buffers and textures, you’ll create a Model struct for the compute function in Metal, and an argument buffer in Swift, containing, for each model:

  • the vertex buffer
  • the index buffer
  • the texture argument buffer
  • the pipeline state

You’ll set up an array of Models in one single buffer and pass this to the compute function.

There’s one more array you’ll need to send. This is the draw arguments for each model. Each model’s draw call is different from every other. You have to specify, for example, what the index buffer is and what is the index count. Fortunately Apple have created a format that you can use for this, called MTLDrawIndexedPrimitivesIndirectArguments. That’s some mouthful!

Compute shader function

You’ll start by creating the compute shader, so that you can see what data you have to pass. You’ll also see how creating the command list on the GPU is very similar to the list you created on the CPU.

Create a new Metal file called ICB.metal. Remember to add it to both the iOS and macOS targets.

Add the following code:

#import "Common.h"

struct ICBContainer {
  command_buffer icb [[id(0)]];
};

struct Model {
  constant float *vertexBuffer;
  constant uint *indexBuffer;
  constant float *texturesBuffer;
  render_pipeline_state pipelineState;
};

Here you set up a struct to hold the indirect command buffer. On the Swift side, you’ll create an argument buffer to hold the ICB. You also create a struct for the model. This holds all the necessary data for the model’s draw call. Note that you can use [[id(n)]] to assign a custom index number. In Model, the index numbers will run from 0 to 3.

Create the compute shader function:

kernel void encodeCommands(
  uint modelIndex [[thread_position_in_grid]],
  constant Uniforms &uniforms [[buffer(BufferIndexUniforms)]],
  constant FragmentUniforms &fragmentUniforms 
    [[buffer(BufferIndexFragmentUniforms)]],
  constant MTLDrawIndexedPrimitivesIndirectArguments 
    *drawArgumentsBuffer [[buffer(BufferIndexDrawArguments)]],
  constant ModelParams *modelParamsArray 
    [[buffer(BufferIndexModelParams)]],
  constant Model *modelsArray [[buffer(BufferIndexModels)]],
  device ICBContainer *icbContainer [[buffer(BufferIndexICB)]]) {
}

Notice the keyword kernel. This sets it apart from vertex and fragment shader functions. Each GPU thread will process one model, and you get the model’s index from the thread position. You receive the uniform buffers just as you did in a vertex function. You also set an array for the draw arguments, the model constants and the models. Lastly, you receive the ICB in device space, which allows you to write to it within the shader function.

In encodeCommands, first extract the elements from the arrays using the model index:

Model model = modelsArray[modelIndex];
MTLDrawIndexedPrimitivesIndirectArguments drawArguments
  = drawArgumentsBuffer[modelIndex];
render_command cmd(icbContainer->icb, modelIndex);

Here you extract the model and the model’s draw arguments for its draw call. You also get an indirect render command object from the ICB, using the model’s index.

Set all the buffers for the command:

cmd.set_render_pipeline_state(model.pipelineState);
cmd.set_vertex_buffer(&uniforms, BufferIndexUniforms);
cmd.set_fragment_buffer(&fragmentUniforms, 
                        BufferIndexFragmentUniforms);
cmd.set_vertex_buffer(modelParamsArray, BufferIndexModelParams);
cmd.set_fragment_buffer(modelParamsArray, 
                        BufferIndexModelParams);
cmd.set_vertex_buffer(model.vertexBuffer, 0);
cmd.set_fragment_buffer(model.texturesBuffer, 
                        BufferIndexTextures);

This looks very similar to your original render loop. You set the model’s pipeline state, the uniforms, the vertex buffer and the textures buffer on the render command object.

Next you encode the draw call using the draw arguments:

cmd.draw_indexed_primitives(
  primitive_type::triangle,
  drawArguments.indexCount,
  model.indexBuffer + drawArguments.indexStart,
  drawArguments.instanceCount,
  drawArguments.baseVertex,
  drawArguments.baseInstance);

This draw command is almost exactly the same as the one on the Swift side, minus the argument labels.

You’ve now encoded a complete draw call, and that’s all that’s required for the compute function. Your next task is to set up the compute function on the CPU side, with a compute pipeline state and pass all the data to the compute function.

Note: You’re not performing any extra logic here to see whether the model should be rendered this frame. But if you determine that the model shouldn’t be rendered, instead of doing a draw call, you’d create an empty command with cmd.reset().

The compute pipeline state

In Renderer.swift, create these new properties:

let icbPipelineState: MTLComputePipelineState
let icbComputeFunction: MTLFunction

You’ll need a new compute pipeline state which uses the compute function you just created.

Create a new type method to build the pipeline state:

static func buildComputePipelineState(function: MTLFunction) -> 
  MTLComputePipelineState {
  let computePipelineState: MTLComputePipelineState
  do {
    computePipelineState = try 
      Renderer.device.makeComputePipelineState(
                 function: function)
  } catch {
    fatalError(error.localizedDescription)
  }
  return computePipelineState
}

In init(metalView:), before calling super.init(), add this:

icbComputeFunction = 
  Renderer.library.makeFunction(name: "encodeCommands")!
icbPipelineState = 
  Renderer.buildComputePipelineState(function: icbComputeFunction)

This creates the compute function in the Metal library, and also the compute pipeline state.

The argument buffers

In the compute shader, you created two structs — one for the ICB, and one for the model. In Renderer, create two buffer properties for the argument buffers to match these structs:

var icbBuffer: MTLBuffer!
var modelsBuffer: MTLBuffer!

In initializeCommands(), remove the for loop. You’re going to be creating the commands on the GPU each frame now.

Add this code at the end of initializeCommands():

let icbEncoder = icbComputeFunction.makeArgumentEncoder(
                   bufferIndex: Int(BufferIndexICB.rawValue))
icbBuffer = Renderer.device.makeBuffer(
              length: icbEncoder.encodedLength,
              options: [])
icbEncoder.setArgumentBuffer(icbBuffer, offset: 0)
icbEncoder.setIndirectCommandBuffer(icb, index: 0)

Just as you did in the first part of this chapter, you create an argument encoder for the compute function, and assign an argument buffer, that will contain the command list, to the encoder. You also set the indirect command buffer.

You’ll now create an argument buffer for each model and hold it in an array. Add this, following on from the previous code:

var mBuffers: [MTLBuffer] = []
var mBuffersLength = 0
for model in models {
  let encoder = icbComputeFunction.makeArgumentEncoder(
                  bufferIndex: Int(BufferIndexModels.rawValue))
  let mBuffer = Renderer.device.makeBuffer(
                  length: encoder.encodedLength, options: [])!
  encoder.setArgumentBuffer(mBuffer, offset: 0)
  encoder.setBuffer(model.vertexBuffer, offset: 0, index: 0)
  encoder.setBuffer(model.submesh.indexBuffer.buffer,
                    offset: 0, index: 1)
  encoder.setBuffer(model.texturesBuffer!, offset: 0, index: 2)
  encoder.setRenderPipelineState(model.pipelineState, index: 3)
  mBuffers.append(mBuffer)
  mBuffersLength += mBuffer.length
}

Here you create an array of argument buffers to match the Model struct you created in ICB.metal. You also keep track of the total buffer length.

Now that you’ve created an array of argument buffers, you’ll create one large buffer to hold all these buffers. Add this code:

modelsBuffer = Renderer.device.makeBuffer(length: mBuffersLength, 
                                          options: [])
modelsBuffer.label = "Models Array Buffer"

This is an empty buffer. You can’t directly assign the array to the Metal buffer, but you can copy the bytes. You’ll iterate through mBuffers and copy each MTLBuffer element into modelsBuffer.

Add this after the previous code:

var offset = 0
for mBuffer in mBuffers {
  var pointer = modelsBuffer.contents()
  pointer = pointer.advanced(by: offset)
  pointer.copyMemory(from: mBuffer.contents(), byteCount: mBuffer.length)
  offset += mBuffer.length
}

This copies each MTLBuffer into modelsBuffer. You keep track of the offset in the buffer to copy to.

You’ve created the compute pipeline state and the argument buffers. The last item to create is the draw arguments for the model.

Draw arguments

At the top of Renderer, create a new buffer property for the draw arguments:

var drawArgumentsBuffer: MTLBuffer!

At the end of initializeCommands(), create this buffer:

let drawLength = models.count * 
 MemoryLayout<MTLDrawIndexedPrimitivesIndirectArguments>.stride
drawArgumentsBuffer = 
     Renderer.device.makeBuffer(length: drawLength,
                                options: [])!
drawArgumentsBuffer.label = "Draw Arguments"

To fill the draw arguments buffer, add this:

// 1
var drawPointer = 
  drawArgumentsBuffer.contents().bindMemory(
    to: MTLDrawIndexedPrimitivesIndirectArguments.self,
    capacity: models.count)
// 2
for (modelIndex, model) in models.enumerated() {
  var drawArgument = MTLDrawIndexedPrimitivesIndirectArguments()
  drawArgument.indexCount = UInt32(model.submesh.indexCount)
  drawArgument.instanceCount = 1
  drawArgument.indexStart = 
      UInt32(model.submesh.indexBuffer.offset)
  drawArgument.baseVertex = 0
  drawArgument.baseInstance = UInt32(modelIndex)
  // 3
  drawPointer.pointee = drawArgument
  drawPointer = drawPointer.advanced(by: 1)
}

Going through this code:

  1. You bind a pointer to the buffer, of the type MTLDrawIndexedPrimitivesIndirectArguments.
  2. You iterate through the models and provide the draw arguments. These are exactly the same as you set for the draw call in the command list that you commented out.
  3. Set the draw argument into the buffer and advance the pointer.

The compute command encoder

You’ve done all the preamble and setup code. All that’s left to do now is create a compute command encoder to run the compute shader function. This will create a render command to render every model.

In draw(in:), after updateUniforms(), add this:

guard
  let computeEncoder = commandBuffer.makeComputeCommandEncoder()
  else { return }
computeEncoder.setComputePipelineState(icbPipelineState)
computeEncoder.setBuffer(uniformsBuffer, offset: 0, 
  index: Int(BufferIndexUniforms.rawValue))
computeEncoder.setBuffer(fragmentUniformsBuffer, offset: 0, 
  index: Int(BufferIndexFragmentUniforms.rawValue))
computeEncoder.setBuffer(drawArgumentsBuffer, offset: 0, 
  index: Int(BufferIndexDrawArguments.rawValue))
computeEncoder.setBuffer(modelParamsBuffer, offset: 0, 
  index: Int(BufferIndexModelParams.rawValue))
computeEncoder.setBuffer(modelsBuffer, offset: 0, 
  index: Int(BufferIndexModels.rawValue))
computeEncoder.setBuffer(icbBuffer, offset: 0, 
  index: Int(BufferIndexICB.rawValue))

Here you create a compute command encoder, and, just as you have previously with render command encoders, you set a pipeline state and then pass all the buffers.

Add this code:

computeEncoder.useResource(icb, usage: .write)
computeEncoder.useResource(modelsBuffer, usage: .read)

if let heap = TextureController.heap {
  computeEncoder.useHeap(heap)
}

for model in models {
  computeEncoder.useResource(model.vertexBuffer, usage: .read)
  computeEncoder.useResource(model.submesh.indexBuffer.buffer, 
                             usage: .read)
  computeEncoder.useResource(model.texturesBuffer!, 
                             usage: .read)
}

Just as before, when you use argument buffers, you have to use the resources.

Complete the command encoder. Add this:

let threadExecutionWidth = icbPipelineState.threadExecutionWidth
let threads = MTLSize(width: models.count, height: 1, depth: 1)
let threadsPerThreadgroup = MTLSize(width: threadExecutionWidth, 
                                    height: 1, depth: 1)
computeEncoder.dispatchThreads(threads, 
  threadsPerThreadgroup: threadsPerThreadgroup)
computeEncoder.endEncoding()

The next chapter will go into compute threads in depth. Just notice for now, that there’s models.count number of threads in the threads per threadgroup width. That means that the compute shader function will execute models.count number of times.

Finally, optimize your ICB commands with a blit command encoder:

let blitEncoder = commandBuffer.makeBlitCommandEncoder()!
blitEncoder.optimizeIndirectCommandBuffer(icb, 
                                 range: 0..<models.count)
blitEncoder.endEncoding()

This will remove any empty commands, if you have them.

You’re using all the resources and sending them to the GPU for the compute function, so remove all the use commands off renderEncoder. The only commands on the render encoder should be:

renderEncoder.setDepthStencilState(depthStencilState)
renderEncoder.executeCommandsInBuffer(icb, 
                                      range: 0..<models.count)
renderEncoder.endEncoding()

Before you build and run, save all the documents you have open. When you’re programming GPUs and moving around blocks of memory, sometimes you can accidentally set memory blocks in areas where you’re not supposed to. When this happens, your display may go crazy with flickering and drawing weirdness, and you’ll have to restart your computer. Hopefully, you have followed this chapter correctly, and this won’t happen to you. Not until you start experimenting, anyway :].

Build and run, capture the frame using the Capture GPU frame button, and examine both the compute command list and the render command list and take note where exactly the resources load. Notice how few commands there are on the render command coder.

Congratulations! You’ve learned a number of new techniques in this chapter.

  • You started out collecting resources into argument buffers, which means passing fewer parameters to shaders.
  • You gathered your textures into a resource heap. The heap you created is static, but you can reuse space on the heap where you use different textures at different times.
  • You explored indirect commands on the CPU. For simple static rendering work, this works fine.
  • Finally you took creating your indirect commands from the CPU to a compute function on the GPU. You created indirect render commands in this chapter, but WWDC 2019 introduced indirect compute commands also.

Where to go from here?

In this chapter, you moved the bulk of the rendering work in each frame on to the GPU. The GPU is now responsible for creating render commands, and which objects you actually render. Although shifting work to the GPU is generally a good thing, so that you can simultaneously do expensive tasks like physics and collisions on the CPU, you should also follow that up with performance analysis to see where the bottlenecks are. You can read more about this at the end of the next section.

GPU-driven rendering is a recent concept, and the best resources are Apple’s WWDC sessions:

Metal for Game Developers has a wide range of sample code using argument buffers, heaps and GPU encoding.

14. Multipass & Deferred Rendering
16. Particle Systems
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2026 Kodeco Inc.