Unreal Engine 4 Toon Outlines Tutorial

What is Convolution?

In image processing, convolution is an operation on two groups of numbers to produce a single number. First, you take a grid of numbers (known as a kernel) and place the center over each pixel. Below is an example of a 3×3 kernel moving over the top two rows of an image:

For every pixel, multiply each kernel entry by its corresponding pixel. Let’s take the pixel from the top-left corner of the mouth for demonstration. We’ll also convert the image to grayscale to simplify the calculations.

First, place the kernel (we’ll use the same one from before) so that the target pixel is in the center. Afterwards, multiply each kernel element with the pixel it overlaps.

Finally, add all the results together. This will be the new value for the center pixel. In this case, the new value is 0.5 + 0.5 or 1. Here is the image after performing convolution on every pixel:

The kernel you use determines what effect you get. The kernel from the examples is used for edge detection. Here are a few examples of other kernels:

To detect edges in an image, you can use Laplacian edge detection.

Laplacian Edge Detection

First, what is the kernel for Laplacian edge detection? It’s actually the one you saw in the examples from the last section!

This kernel works for edge detection because the Laplacian measures the change in slope. Areas with greater change diverge from zero, indicating it is an edge.

To help you understand it, let’s look at the Laplacian in one dimension. The kernel for this would be:

First, place the kernel over an edge pixel and then perform convolution.

This will give you a value of 1 which indicates there was a large change. This means the target pixel is likely to be an edge.

Next, let’s convolve an area with less variance.

Even though the pixels have different values, the gradient is linear. This means there is no change in slope and indicates the target pixel is not an edge.

Below is the image after convolution and a graph with each value plotted. You can see that pixels on an edge are further away from zero.

Phew! That was a lot of theory but don’t worry — now comes the fun part. In the next section, you will build a post process material that performs Laplacian edge detection on the depth buffer.

Building the Laplacian Edge Detector

Navigate to the Maps folder and open PostProcess. You will see a black screen. This is because the map contains a Post Process Volume using an empty post process material.

This is the material you will edit to build the edge detector. The first step is to figure out how to sample neighboring pixels.

To get the position of the current pixel, you can use a TextureCoordinate. For example, if the current pixel is in the middle, it will return (0.5, 0.5). This two-component vector is called a UV.

To sample a different pixel, you just need to add an offset to the TextureCoordinate. In a 100×100 image, each pixel has a size of 0.01 in UV space. To sample a pixel to the right, you add 0.01 on the X-axis.

However, there is a problem with this. As the image resolution changes, the pixel size also changes. If you use the same offset (0.01, 0) in a 200×200 image, it will sample two pixels to the right.

To fix this, you can use the SceneTexelSize node which returns the pixel size. To use it, you do something like this:

Since you are going to be sampling multiple pixels, you would have to create this multiple times.

Obviously, this will quickly become messy. Fortunately, you can use material functions to keep your graph clean.

In the next section, you will put the duplicate nodes into the function and create an input for the offset.

Creating the Sample Pixel Function

First, navigate to the Materials\PostProcess folder. To create a material function, click Add New and select Materials & Textures\Material Function.

Rename it to MF_GetPixelDepth and then open it. The graph will have a single FunctionOutput. This is where you will connect the value of the sampled pixel.

First, you need to create an input that will accept an offset. To do this, create a FunctionInput.

This will show up as an input pin when you use the function later.

Now you need to specify a few settings for the input. Make sure you have the FunctionInput selected and then go to the Details panel. Adjust the following settings:

• InputName: Offset
• InputType: Function Input Vector 2. Since the depth buffer is a 2D image, the offset needs to be a Vector 2.
• Use Preview Value as Default: Enabled. If you don’t provide an input value, the function will use the value from Preview Value.

Next, you need to multiply the offset by the pixel size. Then, you need to add the result to the TextureCoordinate. To do this, add the highlighted nodes:

Finally, you need to sample the depth buffer using the provided UVs. Add a SceneDepth and connect everything like so:

Summary:

1. Offset will take in a Vector 2 and multiply it by SceneTexelSize. This will give you an offset in UV space.
2. Add the offset to TextureCoordinate to get a pixel that is (x, y) pixels away from the current pixel
3. SceneDepth will use the provided UVs to sample the appropriate pixel and then output it

That’s it for the material function. Click Apply and then close MF_GetPixelDepth.

Next, you need to use the function to perform convolution on the depth buffer.