2. 3D Models

Written by Marius Horga & Caroline Begbie

Do you know what makes a good game even better? Gorgeous graphics!

Creating amazing graphics — like those in Divinity: Original Sin 2, Diablo 3 and The Witcher 3 — generally requires a team of programmers and 3D artists who are fairly skilled at what they do. The graphics you see onscreen are created using 3D models that are rendered with custom renderers, similar to the one you wrote in the previous chapter, only more advanced. Nevertheless, the principle of rendering 3D models is still the same.

In this chapter, you’ll learn all about 3D models, including how to render them onscreen, and how to work with them in Blender.

What Are 3D Models?

3D models are made up of vertices. Each vertex refers to a point in 3D space using x, y and z values.

As you saw in the previous chapter, you send these vertex points to the GPU for rendering. You need three vertices to create a triangle, and GPUs are able to render triangles efficiently. To show smaller details, a 3D model may also use textures. You’ll learn more about textures in Chapter 8, “Textures”.

➤ Open the starter playground for this chapter.

This playground contains the train model in two formats (.obj and .usd), as well as two pages (Render and Export 3D Model and Import Train). If you don’t see these items, you may need to hide/show the Project navigator using the icon at the top-left.

To show the file extensions, open Xcode Preferences, and on the General tab, choose File Extensions: Show All.

➤ From the Project navigator, select Render and Export 3D Model.

This page contains the code from Chapter 1, “Hello, Metal!”. Examine the rendered sphere in the playground’s live view. Notice how the sphere renders as a solid red shape and appears flat.

To see the edges of each individual triangle, you can render the model in wireframe.

➤ To render in wireframe, add the following line of code just before the draw call:

renderEncoder.setTriangleFillMode(.lines)

This code tells the GPU to render lines instead of solid triangles.

➤ Run the playground:

There’s a bit of an optical illusion happening here. It may not look like it, but the GPU is rendering straight lines. The reason the sphere edges look curved is because of the number of triangles the GPU is rendering. If you render fewer triangles, curved models tend to look “blocky”.

You can really see the 3D nature of the sphere now. The model’s triangles are evenly spaced horizontally, but because you’re viewing on a two dimensional screen, they appear smaller at the edges of the sphere than the triangles in the middle.

In 3D apps such as Blender or Maya, you generally manipulate points, lines and faces. Points are the vertices; lines, also called edges, are the lines between the vertices; and faces are the triangular flat areas.

The vertices are generally ordered into triangles because GPU hardware is specialized to process them. The GPU’s core instructions are expecting to see a triangle. Of all possible shapes, why a triangle?

• A triangle has the least number of points of any polygon that can be drawn in two dimensions.
• No matter which way you move the points of a triangle, the three points will always be on the same plane.
• When you divide a triangle starting from any vertex, it always becomes two triangles.

When you’re modeling in a 3D app, you generally work with quads (four point polygons). Quads work well with subdivision or smoothing algorithms.

Creating Models With Blender

To create 3D models, you need a 3D modeling app. These apps range from free to hugely expensive. The best of the free apps — and the one used throughout this book — is Blender (v. 3.0). A lot of professionals use Blender, but if you’re more familiar with another 3D app — such as Cheetah3D, Maya or Houdini — then you’re welcome to use it since the concepts are the same.

➤ Download and install Blender from https://www.blender.org.

➤ Launch Blender. Click outside of the splash screen to close it, and you’ll see an interface similar to this one:

Your interface may look different. However, if you want your Blender interface to look like the image shown here, choose Edit ▸ Preferences…. Click the hamburger menu at the bottom left, choose Load Factory Preferences, and then click the pop-up Load Factory Preferences, which will appear under the cursor. Click Save Preferences to retain these preferences for future sessions.

Note: If you want to create your own models, the best place to start is with our Blender tutorial. This tutorial teaches you how to make a mushroom. You can then render your mushroom in your playground at the end of this chapter.

3D File Formats

There are several standard 3D file formats. Here’s an overview of what each one offers:

• .obj: This format, developed by Wavefront Technologies, has been around for awhile, and almost every 3D app supports importing and exporting .obj files. You can specify materials (textures and surface properties) using an accompanying .mtl file, however, this format does not support animation or vertex colors.

• .glTF: Developed by Khronos — who oversee Vulkan and OpenGL — this format is relatively new and is still under active development. It has strong community support because of its flexibility. It supports animated models.

• .blend: This is the native Blender file format.

• .fbx: A proprietary format owned by Autodesk. This is a commonly used format that supports animation but is losing favor because it’s proprietary and doesn’t have a single standard.

• .usd: A scalable open source format introduced by Pixar. USD can reference many models and files, which is not ideal for sharing assets. .usdz is a USD archive file that contains everything needed for the model or scene. Apple uses the USDZ format for their AR models.

An .obj file contains only a single model, whereas .glTF and .usd files are containers for entire scenes, complete with models, animation, cameras and lights.

In this book, you’ll use Wavefront OBJ (.obj), USD, USDZ and Blender format (.blend).

Note: You can use Apple’s Reality Converter to convert 3D files to USDZ. Apple also provides tools for validating and inspecting USDZ files, as well as a gallery of sample USDZ files.

Exporting to Blender

Now that you have Blender all set up, it’s time to export a model from your playground into Blender.

➤ Still in Render and Export 3D Model, near the top of the playground where you create the mesh, change:

let mdlMesh = MDLMesh(
  sphereWithExtent: [0.75, 0.75, 0.75],
  segments: [100, 100],
  inwardNormals: false,
  geometryType: .triangles,
  allocator: allocator)

To:

let mdlMesh = MDLMesh(
  coneWithExtent: [1,1,1],
  segments: [10, 10],
  inwardNormals: false,
  cap: true,
  geometryType: .triangles,
  allocator: allocator)

This code will generate a primitive cone mesh in place of the sphere. Run the playground, and you’ll see the wireframe cone.

This is the model you’ll export using Model I/O.

➤ Open Finder, and in the Documents folder, create a new directory named Shared Playground Data. All of your saved files from the Playground will end up here, so make sure you name it correctly.

Note: The global constant playgroundSharedDataDirectory holds this folder name.

➤ To export the cone, add this code just after creating the mesh:

// begin export code
// 1
let asset = MDLAsset()
asset.add(mdlMesh)
// 2
let fileExtension = "obj"
guard MDLAsset.canExportFileExtension(fileExtension) else {
  fatalError("Can't export a .\(fileExtension) format")
}
// 3
do {
  let url = playgroundSharedDataDirectory
    .appendingPathComponent("primitive.\(fileExtension)")
  try asset.export(to: url)
} catch {
  fatalError("Error \(error.localizedDescription)")
}
// end export code

Let’s have a closer look at the code:

1. The top level of a scene in Model I/O is an MDLAsset. You can build a complete scene hierarchy by adding child objects such as meshes, cameras and lights to the asset.
2. Check that Model I/O can export an .obj file type.
3. Export the cone to the directory stored in Shared Playground Data.

➤ Run the playground to export the cone object.

Note: If your playground crashes, it’s probably because you haven’t created the Shared Playground Data directory in Documents.

The .obj File Format

➤ In Finder, navigate to Documents ▸ Shared Playground Data.

Here, you’ll find the two exported files, primitive.obj and primitive.mtl.

➤ Using a plain text editor, open primitive.obj.

The following is an example .obj file. It describes a plane primitive with four corner vertices. The cone .obj file looks similar, except it has more data.

# Apple ModelIO OBJ File: plane
mtllib plane.mtl
g submesh
v 0 0.5 -0.5
v 0 -0.5 -0.5
v 0 -0.5 0.5
v 0 0.5 0.5
vn -1 0 0
vt 1 0
vt 0 0
vt 0 1
vt 1 1
usemtl material_1
f 1/1/1 2/2/1 3/3/1
f 1/1/1 3/3/1 4/4/1
s off

Here’s the breakdown:

• mtllib: This is the name of the accompanying .mtl file. This file holds the material details and texture file names for the model.
• g: Starts a group of vertices.
• v: Vertex. For the cone, you’ll have 102 of these.
• vn: Surface normal. This is a vector that points orthogonally — that’s directly outwards. You’ll read more about normals later.
• vt: uv coordinate that determines the vertex’s position on a 2D texture. Textures use uv coordinates rather than xy coordinates.
• usemtl: The name of a material providing the surface information — such as color — for the following faces. This material is defined in the accompanying .mtl file.
• f: Defines faces. You can see here that the plane has two faces, and each face has three elements consisting of a vertex/texture/normal index. In this example, the last face listed: 4/4/1 would be the fourth vertex element / the fourth texture element / the first normal element: 0 0.5 0.5 / 1 1 / -1 0 0.
• s: Smoothing, currently off, means there are no groups that will form a smooth surface.

The .mtl File Format

The second file you exported contains the model’s materials. Materials describe how the 3D renderer should color the vertex. For example, should the vertex be smooth and shiny? Pink? Reflective? The .mtl file contains values for these properties.

➤ Using a plain text editor, open primitive.mtl:

# Apple ModelI/O MTL File: primitive.mtl

newmtl material_1
	Kd 1 1 1
	Ka 0 0 0
	Ks 0
	ao 0
	subsurface 0
	metallic 0
	specularTint 0
	roughness 0.9
	anisotropicRotation 0
	sheen 0.05
	sheenTint 0
	clearCoat 0
	clearCoatGloss 0

Here’s the breakdown:

• newmtl material_1: This is the group that contains all of the cone’s vertices.
• Kd: The diffuse color of the surface. In this case, 1 1 1 will color the object white.
• Ka: The ambient color. This models the ambient lighting in the room.
• Ks: The specular color. The specular color is the color reflected from a highlight.

You’ll read more about these and the other material properties later.

Importing the Cone

It’s time to import the cone into Blender.

➤ To start with a clean and empty Blender file:

1. Open Blender.
2. Choose File ▸ New ▸ General.
3. Left-click the cube that appears in the start-up file to select it.
4. Press X to delete the cube.
5. Left-click Delete in the menu under the cursor to confirm the deletion.

You now have a clear and ready-for-importing Blender file, so let’s get to it.

➤ Choose File ▸ Import ▸ Wavefront (.obj), and select primitive.obj from the Documents ▸ Shared Playground Data Playground directory.

The cone imports into Blender.

➤ Left-click the cone to select it, and press Tab to put Blender into Edit Mode. Edit Mode allows you to see the vertices and triangles that make up the cone.

While you’re in Edit Mode, you can move the vertices around and add new vertices to create any 3D model you can imagine.

Note: In the resources directory for this chapter, there’s a file with links to some excellent Blender tutorials.

Using only a playground, you now have the ability to create, render and export a primitive. In the next part of this chapter, you’ll review and render a more complex model with separate material groups.

Material Groups

➤ In Blender, open train.blend, which you’ll find in the resources directory for this chapter.

This file is the original Blender file of the .obj train in your playground.

➤ Left-click the model to select it, and press Tab to go into Edit Mode.

Unlike the cone, the train model has several material groups — one for each color. On the right-hand side of the Blender screen, you’ll see the Properties panel, with the Material context already selected (that’s the icon at the bottom of the vertical list of icons), and the list of materials within this model at the top.

➤ Select Body, and then click Select underneath the material list.

The vertices assigned to this material are now colored orange.

Notice how the vertices are separated into different groups or materials. This separation makes it easier to select the various parts within Blender and also gives you the ability to assign different colors.

Note: When you first import this model into your playground, the renderer will render each of the material groups, but it may not pick up the correct colors. One way to verify a model’s appearance is to view it in Blender.

➤ Go back to Xcode, and from the Project navigator, open the Import Train playground page. This playground renders — but does not export — the wireframe cone.

In the playground’s Resources folder, you’ll see three files: train.mtl, train.obj and train.usd.

Note: Files in the Playground Resources folder are available to all playground pages. Files in each page’s Resources folder are only available to that page.

➤ In Import Train, remove the line where you create the MDLMesh cone:

let mdlMesh = MDLMesh(
  coneWithExtent: [1, 1, 1],
  segments: [10, 10],
  inwardNormals: false,
  cap: true,
  geometryType: .triangles,
  allocator: allocator)

Don’t worry about that compile error. You’ve still got some work to do.

➤ Replacing the code you just removed, add this code in its place:

guard let assetURL = Bundle.main.url(
  forResource: "train",
  withExtension: "obj") else {
  fatalError()
}

This code sets up the file URL for the .obj format of the model. Later, you can try the .usd format, and you should get the same result.

Vertex Descriptors

Metal uses descriptors as a common pattern to create objects. You saw this pattern in the previous chapter when you set up a pipeline descriptor to describe a pipeline state. Before loading the model, you’ll tell Metal how to lay out the vertices and other data by creating a vertex descriptor.

The following diagram describes an incoming buffer of model vertex data. It has two vertices with position, normal and texture coordinate attributes. The vertex descriptor informs Metal how you want to view this data.

➤ Add this code below the code you just added:

// 1
let vertexDescriptor = MTLVertexDescriptor()
// 2
vertexDescriptor.attributes[0].format = .float3
// 3
vertexDescriptor.attributes[0].offset = 0
// 4
vertexDescriptor.attributes[0].bufferIndex = 0

Looking closer:

1. You create a vertex descriptor that you’ll use to configure all of the properties that an object will need to know about.

Note: You can reuse this vertex descriptor with either the same values or reconfigured values to instantiate a different object.

2. The .obj file holds normal and texture coordinate data as well as vertex position data. For the moment, you don’t need the surface normals or texture coordinates; you only need the position. You tell the descriptor that the xyz position data should load as a float3, which is a simd data type consisting of three Float values. An MTLVertexDescriptor has an array of 31 attributes where you can configure the data format — and in future chapters, you’ll load the normal and texture coordinate attributes.

3. The offset specifies where in the buffer this particular data will start.

4. When you send your vertex data to the GPU via the render encoder, you send it in an MTLBuffer and identify the buffer by an index. There are 31 buffers available and Metal keeps track of them in a buffer argument table. You use buffer 0 here so that the vertex shader function will be able to match the incoming vertex data in buffer 0 with this vertex layout.

➤ Now add this code below the previous lines:

// 1
vertexDescriptor.layouts[0].stride =
  MemoryLayout<SIMD3<Float>>.stride
// 2
let meshDescriptor =
  MTKModelIOVertexDescriptorFromMetal(vertexDescriptor)
// 3
(meshDescriptor.attributes[0] as! MDLVertexAttribute).name =
  MDLVertexAttributePosition

Going through everything:

1. Here, you specify the stride for buffer 0. The stride is the number of bytes between each set of vertex information. Referring back to the previous diagram which described position, normal and texture coordinate information, the stride between each vertex would be float3 + float3 + float2. However, here you’re only loading position data, so to get to the next position, you jump by a stride of float3.

Using the buffer layout index and stride format, you can set up complex vertex descriptors referencing multiple MTLBuffers with different layouts. You have the option of interleaving position, normal and texture coordinates; or you can lay out a buffer containing all of the position data first, followed by other data.

Note: The SIMD3<Float> type is Swift’s equivalent to float3. Later, you’ll use a typealias for float3.

2. Model I/O needs a slightly different format vertex descriptor, so you create a new Model I/O descriptor from the Metal vertex descriptor. If you have a Model I/O descriptor and need a Metal one, MTKMetalVertexDescriptorFromModelIO() provides a solution.

3. Assign a string name “position” to the attribute. This tells Model I/O that this is positional data. The normal and texture coordinate data is also available, but with this vertex descriptor, you told Model I/O that you’re not interested in those attributes.

➤ Continue by adding this code:

let asset = MDLAsset(
  url: assetURL,
  vertexDescriptor: meshDescriptor,
  bufferAllocator: allocator)
let mdlMesh =
  asset.childObjects(of: MDLMesh.self).first as! MDLMesh

This code reads the asset using the URL, vertex descriptor and memory allocator. You then read in the first Model I/O mesh buffer in the asset. Some more complex objects will have multiple meshes, but you’ll deal with that later.

Now that you’ve loaded the model vertex information, the rest of the code will be the same, and your playground will load mesh from the new mdlMesh variable.

➤ Run the playground to see your train in wireframe.

Well, that’s not good. The train is missing some wheels, and the ones that are there are way too high off the ground. Plus, the rest of the train is missing! Time to fix these problems, starting with the train’s wheels.

Metal Coordinate System

All models have an origin. The origin is the location of the mesh. The train’s origin is at [0, 0, 0]. In Blender, this places the train right at the center of the scene.

The Metal NDC (Normalized Device Coordinate) system is a 2-unit wide by 2-unit high by 1-unit deep box where X is right / left, Y is up / down and Z is in / out of the screen.

To normalize means to adjust to a standard scale. On a screen, you might address a location in screen coordinates of width 0 to 375, whereas the Metal normalized coordinate system doesn’t care what the physical width of a screen is — its coordinates along the X axis are -1.0 to 1.0. In Chapter 6, “Coordinate Spaces”, you’ll learn about various coordinate systems and spaces. Because the origin of the train is at [0,0,0], the train appears halfway up the screen, which is where [0,0,0] is in the Metal coordinate system.

➤ Select train.obj in the Project navigator. The SceneKit editor opens and shows you the train model. Currently, the editor doesn’t apply the materials, so the train appears white on a white background. You can still select the train by clicking somewhere around the center of the window. Do that now, and you’ll see three arrows appear when you select the train. Now, open the Node inspector on the right, and change y Position to -1.

Note: Typically, you’d change the position of the model in code. However, the purpose of this example is to illustrate how you can affect the model.

➤ Go back to Import Train, and run the playground. The wheels now appear at the bottom of the screen.

Now that the wheels are fixed, you’re ready to solve the case of the missing train!

Submeshes

So far, your primitive models included only one material group, and thus one submesh. Here’s a plane with four vertices and two material groups.

When Model I/O loads this plane, it places the four vertices in an MTLBuffer. The following image shows the vertex position data and also how two submesh buffers index into the vertex data.

The first submesh buffer holds the vertex indices of the light-colored triangle ACD. These indices point to vertices 0, 2 and 3. The second submesh buffer holds the indices of the dark triangle ADB. The submesh also has an offset where the submesh buffer starts. The index can be held in either a uint16 or a uint32. The offset of this second submesh buffer would be three times the size of the uint type.

Winding Order

The vertex order, also known as the winding order, is important here. The vertex order of this plane is counter-clockwise, as is the default .obj winding order. With a counter-clockwise winding order, triangles that are defined in counter-clockwise order are facing toward you. Whereas triangles that are in clockwise order are facing away from you. In the next chapter, you’ll go down the graphics pipeline and you’ll see that the GPU can cull triangles that are not facing toward you, saving valuable processing time.

Render Submeshes

Currently, you’re only rendering the first submesh, but because the train has several material groups, you’ll need to loop through the submeshes to render them all.

➤ Toward the end of the playground, change:

guard let submesh = mesh.submeshes.first else {
 fatalError()
}
renderEncoder.drawIndexedPrimitives(
  type: .triangle,
  indexCount: submesh.indexCount,
  indexType: submesh.indexType,
  indexBuffer: submesh.indexBuffer.buffer,
  indexBufferOffset: 0)

To:

for submesh in mesh.submeshes {
  renderEncoder.drawIndexedPrimitives(
    type: .triangle,
    indexCount: submesh.indexCount,
    indexType: submesh.indexType,
    indexBuffer: submesh.indexBuffer.buffer,
    indexBufferOffset: submesh.indexBuffer.offset
  )
}

This code loops through the submeshes and issues a draw call for each one. The mesh and submeshes are in MTLBuffers, and the submesh holds the index listing of the vertices in the mesh.

➤ Run the playground, and your train renders completely — minus the material colors, which you’ll take care of in Chapter 11, “Maps & Materials”.

Congratulations! You’re now rendering 3D models. For now, don’t worry that you’re only rendering them in two dimensions or that the colors aren’t correct. After the next chapter, you’ll know more about the internals of rendering. Following on from that, you’ll learn how to move those vertices into the third dimension.

Challenge

If you’re in for a fun challenge, complete the Blender tutorial to make a mushroom, and then export what you make in Blender to an .obj file. If you want to skip the modeling, you’ll find the mushroom.obj file in the resources directory for this chapter.

➤ Import mushroom.obj into the playground and render it.

If you use the mushroom from the resources directory, you’ll first have to scale and reposition the mushroom in the SceneKit editor to view it correctly.

If you have difficulty, the completed playground is in the Projects ▸ Challenge directory for this chapter.

Key Points

• 3D models consist of vertices. Each vertex has a position in 3D space.
• In 3D modeling apps, you create models using quads, or polygons with four vertices. On import, Model I/O converts these quads to triangles.
• Triangles are the GPU’s native format.
• Blender is a fully-featured professional free 3D modeling, animation and rendering app available from https://www.blender.org.
• There are many 3D file formats. Apple has standardized its AR models on Pixar’s USD format in a compressed USDZ format.
• Vertex descriptors describe the buffer format for the model’s vertices. You set the GPU pipeline state with the vertex descriptor, so that the GPU knows what the vertex buffer format is.
• A model is made up of at least one submesh. This submesh corresponds to a material group where you can define the color and other surface attributes of the group.
• Metal Normalized Device Coordinates are -1 to 1 on the X and Y axes, and 0 to 1 on the Z axis. X is left / right, Y is down / up and Z is front / back.
• The GPU will render only vertices positioned in Metal NDC.