Chapters
Chapters

Chapters

Hide chapters

Metal by Tutorials

Fourth Edition · macOS 14, iOS 17 · Swift 5.9 · Xcode 15

Before You Begin

Section 0: 6 chapters
Show chapters Hide chapters

Section I: Beginning Metal

Section 1: 10 chapters
Show chapters Hide chapters

Section II: Intermediate Metal

Section 2: 8 chapters
Show chapters Hide chapters

Section III: Advanced Metal

Section 3: 8 chapters
Show chapters Hide chapters

Section IV: Ray Tracing

Section 4: 7 chapters
Show chapters Hide chapters

22. Reflection & Refraction
Written by Caroline Begbie & Marius Horga

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

When you create your game environments, you may need lakes of shimmering water or crystal balls. To look realistic, shiny glass objects require both reflection and refraction.

Reflection is one of the most common interactions between light and objects. Imagine looking into a mirror. Not only would you see your image being reflected, but you’d also see the reflection of any nearby objects.

Refraction is another common interaction between light and objects that you often see in nature. While it’s true that most objects in nature are opaque — thus absorbing most of the light they get — the few objects that are translucent, or transparent, allow for the light to propagate through them.

Reflection and refraction
Reflection and refraction

Later, in the final section of this book, you’ll investigate ray tracing and global illumination, which allow advanced effects such as bounced reflections and realistic refraction. We’re approaching a time where ray tracing algorithms may be viable in games, but for now, real-time rendering with rasterized reflection and refraction is the way to go.

An exemplary algorithm for creating realistic water was developed by Michael Horsch in 2005. This realistic water algorithm is purely based on lighting and its optical properties, as opposed to having a water simulation based on physics.

The Starter Project

➤ In Xcode, open the starter project for this chapter.

The starter project is similar to the project at the end of the previous chapter, with a few additions which include:

  • GameScene.swift contains a new scene with new models and renders a new skybox texture. You can move around the scene using WASD keys, and look about using the mouse or trackpad. Scrolling the mouse wheel, or pinching on iOS, moves you up and down, so you can get better views of your lake. The number 1 key will position the camera looking down on the scene, and number 2 will return the camera to its original position.
  • WaterRenderPass.swift, in the Render Passes group, contains a new render pass. It’s similar to ForwardRenderPass, but refactors the command encoder setup into a new render method. WaterRenderPass is all set up and ready to render in Renderer, but it will do nothing until you assign it a render pass descriptor.
  • Water.swift, in the Geometry group, contains a new Water class, similar to Model. The class loads a primitive mesh plane and is set up to render the plane with its own pipeline state.
  • Pipelines.swift has new pipeline state creation methods to render water and a terrain.

➤ Build and run the app.

The starter app
The starter app

Visitors to this quaint cottage would love a recreational lake for swimming and fishing.

Terrains

Many game scenes will have a ground terrain, or landscape, and this terrain may need its own shader. The starter project includes Terrain.swift, which contains Terrain, a subclass of Model. Changing shaders entails loading a new pipeline state, so Terrain creates its own pipeline state object along with a texture for use later.

Rendering Rippling Water

Now that you’re acquainted with the code, here’s the plan on how you’ll proceed through the chapter:

1. Creating the Water Surface

➤ In the Geometry group, open Water.swift, and examine the code.

Creating the Water Shaders

➤ In the Shaders group, create a new Metal file named Water.metal, and add this:

#import "Common.h"

struct VertexIn {
  float4 position [[attribute(Position)]];
  float2 uv [[attribute(UV)]];
};

struct VertexOut {
  float4 position [[position]];
  float4 worldPosition;
  float2 uv;
};

vertex VertexOut vertex_water(
  const VertexIn in [[stage_in]],
  constant Uniforms &uniforms [[buffer(UniformsBuffer)]])
{
  float4x4 mvp = uniforms.projectionMatrix * uniforms.viewMatrix
                   * uniforms.modelMatrix;
  VertexOut out {
    .position = mvp * in.position,
    .uv = in.uv,
    .worldPosition = uniforms.modelMatrix * in.position
  };
  return out;
}

fragment float4 fragment_water(
  VertexOut in [[stage_in]],
  constant Params &params [[buffer(ParamsBuffer)]])
{
  return float4(0.0, 0.3, 0.5, 1.0);
}

Adding the Water to Your Scene

➤ Open GameScene.swift, and add a new property:

var water: Water?

water = Water()
water?.position = [0, -1, 0]

scene.water?.render(
  encoder: renderEncoder,
  uniforms: uniforms,
  params: params)
The initial water plane
Dbo aracaov yiwuz xleha

2. Rendering the Reflection

The water plane should reflect its surroundings. In Chapter 21, “Image-Based Lighting”, you reflected the skybox onto objects, but this time you’re also going to reflect the house and terrain on the water.

descriptor = MTLRenderPassDescriptor()

var reflectionTexture: MTLTexture?
var refractionTexture: MTLTexture?
var depthTexture: MTLTexture?

let size = CGSize(
  width: size.width / 2, height: size.height / 2)
reflectionTexture = Self.makeTexture(
  size: size,
  pixelFormat: view.colorPixelFormat,
  label: "Reflection Texture")
refractionTexture = Self.makeTexture(
  size: size,
  pixelFormat: view.colorPixelFormat,
  label: "Refraction Texture")
depthTexture = Self.makeTexture(
  size: size,
  pixelFormat: .depth32Float,
  label: "Reflection Depth Texture")

let attachment = descriptor?.colorAttachments[0]
attachment?.texture = reflectionTexture
attachment?.storeAction = .store
let depthAttachment = descriptor?.depthAttachment
depthAttachment?.texture = depthTexture
depthAttachment?.storeAction = .store

weak var reflectionTexture: MTLTexture?
weak var refractionTexture: MTLTexture?
weak var refractionDepthTexture: MTLTexture?

encoder.setFragmentTexture(
  reflectionTexture,
  index: 0)
encoder.setFragmentTexture(
  refractionTexture,
  index: 1)

guard let water = scene.water else { return }
water.reflectionTexture = reflectionTexture
water.refractionTexture = refractionTexture
water.refractionDepthTexture = depthTexture


texture2d<float> reflectionTexture [[texture(0)]],
texture2d<float> refractionTexture [[texture(1)]]

// 1
constexpr sampler s(filter::linear, address::repeat);
// 2
float width = float(reflectionTexture.get_width() * 2.0);
float height = float(reflectionTexture.get_height() * 2.0);
float x = in.position.x / width;
float y = in.position.y / height;
float2 reflectionCoords = float2(x, 1 - y);
// 3
float4 color = reflectionTexture.sample(s, reflectionCoords);
color = mix(color, float4(0.0, 0.3, 0.5, 1.0), 0.3);
return color;
Initial reflection
Asovoon qexdumneog

Incorrect reflection
Azwubripc narvujjiep

var reflectionCamera = scene.camera
reflectionCamera.rotation.x *= -1
let position = (scene.camera.position.y - water.position.y) * 2
reflectionCamera.position.y -= position

var uniforms = uniforms
uniforms.viewMatrix = reflectionCamera.viewMatrix
Reflected camera position
Roscajjam hozayu quxomiel

3. Creating Clipping Planes

A clipping plane, as its name suggests, clips the scene using a plane. It’s hardware accelerated, meaning that if geometry is not within the clip range, the GPU immediately discards the vertex and doesn’t put it through the entire pipeline. You may get a significant performance boost as some of the geometry will not need to get processed by the fragment shaders anymore.

The clipping plane
Vso jjivguyz pnili

var clipPlane = float4(0, 1, 0, -water.position.y)
uniforms.clipPlane = clipPlane

vector_float4 clipPlane;

float clip_distance [[clip_distance]] [1];

out.clip_distance[0] =
  dot(uniforms.modelMatrix * in.position, uniforms.clipPlane);
Rendering above the clipping plane
Wuyfaqumv evane pyu sduhvawd mciha

Reflecting the sky correctly
Towmirvudn cma szd docxukcvl

4. Rippling Normal Maps

➤ Open Textures.xcassets, and select normal-water. This is a normal map that you’ll use for the water ripples.

The water ripple normal map
Vma febis guccnu gitquf bey

var waterMovementTexture: MTLTexture?
var timer: Float = 0

waterMovementTexture =
  TextureController.loadTexture(name: "normal-water")

encoder.setFragmentTexture(
  waterMovementTexture,
  index: 2)
var timer = timer
encoder.setFragmentBytes(
  &timer,
  length: MemoryLayout<Float>.size,
  index: 3)

func update(deltaTime: Float) {
  let sensitivity: Float = 0.005
  timer += deltaTime * sensitivity
}

water?.update(deltaTime: deltaTime)

texture2d<float> normalTexture [[texture(2)]],
constant float& timer [[buffer(3)]]

// 1
float2 uv = in.uv * 2.0;
// 2
float waveStrength = 0.1;
float2 rippleX = float2(uv.x + timer, uv.y);
float2 rippleY = float2(-uv.x, uv.y) + timer;
float2 ripple =
  ((normalTexture.sample(s, rippleX).rg * 2.0 - 1.0) +
  (normalTexture.sample(s, rippleY).rg * 2.0 - 1.0))
  * waveStrength;
reflectionCoords += ripple;
// 3  
reflectionCoords = clamp(reflectionCoords, 0.001, 0.999);
Calming water ripples
Luxgonp modil weccyem

5. Adding Refraction

Implementing refraction is very similar to reflection, except that you only need to preserve the part of the scene below the clipping plane.

Rendering below the clipping plane
Fipxayejd kiqil spi ftefvadw dfaza

// 1
descriptor.colorAttachments[0].texture = refractionTexture
// 2
guard let refractEncoder = commandBuffer.makeRenderCommandEncoder(
  descriptor: descriptor) else {
  return
}
refractEncoder.label = "Refraction"
// 3
uniforms.viewMatrix = scene.camera.viewMatrix
clipPlane = float4(0, -1, 0, -water.position.y)
uniforms.clipPlane = clipPlane
// 4
render(
  renderEncoder: refractEncoder,
  scene: scene,
  uniforms: uniforms,
  params: params)
refractEncoder.endEncoding()
The refraction texture
Bpe yofzavpeeb gawzeqi

float2 refractionCoords = float2(x, y);

refractionCoords += ripple;

refractionCoords = clamp(refractionCoords, 0.001, 0.999);

float4 color = reflectionTexture.sample(s, reflectionCoords);

float4 color = refractionTexture.sample(s, refractionCoords);
Refraction
Jartutvais

Pebbles
Yaxgrul

6. The Fresnel Effect

The Fresnel effect is a concept you’ve met with in previous chapters. As you may remember, the viewing angle plays a significant role in the amount of reflection you can see. What’s new in this chapter is that the viewing angle also affects refraction but in inverse proportion:

float3 viewVector =
  normalize(params.cameraPosition - in.worldPosition.xyz);
float mixRatio = dot(viewVector, float3(0, 1, 0));
return mixRatio;
The mix ratio between refraction and reflection
Pfi joq lezao xehraas tidsuqnoob ang qetsijjien

return mixRatio;
float4 color = refractionTexture.sample(s, refractionCoords);

float4 color =
  mix(reflectionTexture.sample(s, reflectionCoords),
      refractionTexture.sample(s, refractionCoords),
      mixRatio);
Refraction and Reflection
Cehyemqouk unh Vopzibguip

7. Adding Smoothness Using a Depth Texture

Light propagation varies for different transparent media, but for water, the colors with longer wavelengths (closer to infrared) quickly fade away as the light ray goes deeper. The bluish colors (closer to ultraviolet) tend to be visible at greater depths because they have shorter wavelengths.

encoder.setFragmentTexture(
  refractionDepthTexture,
  index: 3)

let attachment = pipelineDescriptor.colorAttachments[0]
attachment?.isBlendingEnabled = true
attachment?.rgbBlendOperation = .add
attachment?.sourceRGBBlendFactor = .sourceAlpha
attachment?.destinationRGBBlendFactor = .oneMinusSourceAlpha

depth2d<float> depthMap [[texture(3)]]

float far = 100;    // the camera's far plane
float near = 0.1;   // the camera's near plane
float proj33 = far / (far - near);
float proj43 = proj33 * -near;
float depth = depthMap.sample(s, refractionCoords);
float floorDistance = proj43 / (depth - proj33);
depth = in.position.z;
float waterDistance = proj43 / (depth - proj33);
depth = floorDistance - waterDistance;

color.a = clamp(depth * 0.75, 0.0, 1.0);
Blending at the water's edge
Stofxarv om ska vuyep'j uqne

Key Points

  • Reflection and refraction are important for realistic water and glass.
  • Rasterizing reflections and refraction will not produce as good a result as ray tracing. But when speed is a concern, then ray tracing is not often viable.
  • Use separate render passes to render textures. For reflection, move the camera in the inverse direction from the plane to be reflected and flip the result.
  • You already know about near and far clipping planes, but you can also add your own custom clipping planes. A negative clip distance from in the vertex function will result in the GPU discarding the vertex.
  • You can animate normal maps to provide water turbulence.
  • The Fresnel effect depends upon viewing angle and affects reflection and refraction in inverse proportion.

Where to Go From Here?

You’ve certainly made a splash with this chapter! If you want to explore more about water rendering, the references.markdown file in the resources folder for this chapter contains links to interesting articles and videos.

21. Image-Based Lighting
23. Animation
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2024 Kodeco Inc.

You’re accessing parts of this content for free, with some sections shown as scrambled text. Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now