GLSL Programming/Unity/Toon Shading
This tutorial covers toon shading (also known as cel shading) as an example of non-photorealistic rendering techniques.
It is one of several tutorials about lighting that go beyond the Phong reflection model. However, it is based on per-pixel lighting with the Phong reflection model as described in Section “Smooth Specular Highlights”. If you haven't read that tutorial yet, you should read it first.
Non-photorealistic rendering is a very broad term in computer graphics that covers all rendering techniques and visual styles that are obviously and deliberately different from the appearance of photographs of physical objects. Examples include hatching, outlining, distortions of linear perspective, coarse dithering, coarse color quantization, etc.
Toon shading (or cel shading) is any subset of non-photorealistic rendering techniques that is used to achieve a cartoonish or hand-drawn appearance of three-dimensional models.
Shaders for a Specific Visual Style
editJohn Lasseter from Pixar once said in an interview: “Art challenges technology, and technology inspires the art.” Many visual styles and drawing techniques that are traditionally used to depict three-dimensional objects are in fact very difficult to implement in shaders. However, there is no fundamental reason not to try it.
When implementing one or more shaders for any specific visual style, one should first determine which features of the style have to be implemented. This is mainly a task of precise analysis of examples of the visual style. Without such examples, it is usually unlikely that the characteristic features of a style can be determined. Even artists who master a certain style are unlikely to be able to describe these features appropriately; for example, because they are no longer aware of certain features or might consider some characteristic features as unnecessary imperfections that are not worth mentioning.
For each of the features it should then be determined whether and how accurately to implement them. Some features are rather easy to implement, others are very difficult to implement by a programmer or to compute by a GPU. Therefore, a discussion between shader programmers and (technical) artists in the spirit of John Lasseter's quote above is often extremely worthwhile to decide which features to include and how accurately to reproduce them.
Stylized Specular Highlights
editIn comparison to the Phong reflection model that was implemented in Section “Smooth Specular Highlights”, the specular highlights in the images in this section are plainly white without any addition of other colors. Furthermore, they have a very sharp boundary.
We can implement this kind of stylized specular highlights by computing the specular reflection term of the Phong shading model and setting the fragment color to the specular reflection color times the (unattenuated) color of the light source if the specular reflection term is greater than a certain threshold, e.g. half the maximum intensity.
But what if there shouldn't been any highlights? Usually, the user would specify a black specular reflection color for this case; however, with our method this results in black highlights. One way to solve this problem is to take the opacity of the specular reflection color into account and “blend” the color of the highlight over other colors by compositing them based on the opacity of the specular color. Alpha blending as a per-fragment operation was described in Section “Transparency”. However, if all colors are known in a fragment shader, it can also be computed within a fragment shader.
In the following code snippet, fragmentColor is assumed to have already a color assigned, e.g. based on diffuse illumination. The specular color _SpecColor times the light source color _LightColor0 is then blended over fragment_Color based on the opacity of the specular color _SpecColor.a:
if (dot(normalDirection, lightDirection) > 0.0
// light source on the right side?
&& attenuation * pow(max(0.0, dot(
reflect(-lightDirection, normalDirection),
viewDirection)), _Shininess) > 0.5)
// more than half highlight intensity?
{
fragmentColor = _SpecColor.a
* vec3(_LightColor0) * vec3(_SpecColor)
+ (1.0 - _SpecColor.a) * fragmentColor;
}
Is this sufficient? If you look closely at the eyes of the bull in the image to the left, you will see two pairs of specular highlights, i.e. there is more than one light source that causes specular highlights. In most tutorials, we have taken additional light sources into account by a second render pass with additive blending. However, if the color of specular highlights should not be added to other colors then additive blending should not be used. Instead, alpha blending with a (usually) opaque color for the specular highlights and transparent fragments for other fragments would be a feasible solution. (See Section “Transparency” for a description of alpha blending.)
Stylized Diffuse Illumination
editThe diffuse illumination in the image of the bull to the left consists of just two colors: a light brown for lit fur and a dark brown for unlit fur. The color of other parts of the bull is independent of the lighting.
One way to implement this, is to use the full diffuse reflection color whenever the diffuse reflection term of the Phong reflection model reaches a certain threshold, e.g. greater than 0, and a second color otherwise. For the fur of the bull, these two colors would be different; for the other parts, they would be the same such that there is no visual difference between lit and unlit areas. An implementation for a threshold _DiffuseThreshold to switch from the darker color _UnlitColor to the lighter color _Color (multiplied with the light source color _LightColor0) could look like this:
vec3 fragmentColor = vec3(_UnlitColor);
if (attenuation
* max(0.0, dot(normalDirection, lightDirection))
>= _DiffuseThreshold)
{
fragmentColor = vec3(_LightColor0) * vec3(_Color);
}
Is this all there is to say about the stylized diffuse illumination in the image to the left? A really close look reveals that there is a light, irregular line between the dark brown and the light brown. In fact, the situation is even more complicated and the dark brown sometimes doesn't cover all areas that would be covered by the technique described above, and sometimes it covers more than that and even goes beyond the black outline. This adds rich detail to the visual style and creates a hand-drawn appearance. On the other hand, it is very difficult to reproduce this convincingly in a shader.
Outlines
editOne of the characteristic features of many toon shaders are outlines in a specific color along the silhouettes of the model (usually black, but also other colors, see the cow above for an example).
There are various techniques to achieve this effect in a shader. Unity 3.3 is shipped with a toon shader in the standard assets that renders these outlines by rendering the back faces of an enlarged model in the color of the outlines (enlarged by moving the vertex positions in the direction of the surface normal vectors) and then rendering the front faces on top of them. Here we use another technique based on Section “Silhouette Enhancement”: if a fragment is determined to be close enough to a silhouette, it is set to the color of the outline. This works only for smooth surfaces, and it will generate outlines of varying thickness (which is a plus or a minus depending on the visual style). However, at least the overall thickness of the outlines should be controllable by a shader property.
Are we done yet? If you have a close look at the donkey, you will see that the outlines at its belly and in the ears are considerably thicker than other outlines. This conveys unlit areas; however, the change in thickness is continuous. One way to simulate this effect would be to let the user specify two overall outline thicknesses: one for fully lit areas and one for unlit areas (according to the diffuse reflection term of the Phong reflection model). In between these extremes, the thickness parameter could be interpolated (again according to the diffuse reflection term). This, however, makes the outlines dependent on a specific light source; therefore, the shader below renders outlines and diffuse illumination only for the first light source, which should usually be the most important one. All other light sources only render specular highlights.
The following implementation uses the mix instruction to interpolate between the _UnlitOutlineThickness (if the dot product of the diffuse reflection term is less or equal 0) and _LitOutlineThickness (if the dot product is 1). For a linear interpolation from a value a to another value b with a parameter x between 0 and 1, GLSL offers the built-in function mix(a, b, x). This interpolated value is then used as a threshold to determine whether a point is close enough to the silhouette. If it is, the fragment color is set to the color of the outline _OutlineColor:
if (dot(viewDirection, normalDirection)
< mix(_UnlitOutlineThickness, _LitOutlineThickness,
max(0.0, dot(normalDirection, lightDirection))))
{
fragmentColor =
vec3(_LightColor0) * vec3(_OutlineColor);
}
Complete Shader Code
editIt should be clear by now that even the few images above pose some really difficult challenges for a faithful implementation. Thus, the shader below only implements a few characteristics as described above and ignores many others. Note that the different color contributions (diffuse illumination, outlines, highlights) are given different priorities according to which should occlude which. You could also think of these priorities as different layers that are put on top of each other.
Shader "GLSL shader for toon shading" {
Properties {
_Color ("Diffuse Color", Color) = (1,1,1,1)
_UnlitColor ("Unlit Diffuse Color", Color) = (0.5,0.5,0.5,1)
_DiffuseThreshold ("Threshold for Diffuse Colors", Range(0,1))
= 0.1
_OutlineColor ("Outline Color", Color) = (0,0,0,1)
_LitOutlineThickness ("Lit Outline Thickness", Range(0,1)) = 0.1
_UnlitOutlineThickness ("Unlit Outline Thickness", Range(0,1))
= 0.4
_SpecColor ("Specular Color", Color) = (1,1,1,1)
_Shininess ("Shininess", Float) = 10
}
SubShader {
Pass {
Tags { "LightMode" = "ForwardBase" }
// pass for ambient light and first light source
GLSLPROGRAM
// User-specified properties
uniform vec4 _Color;
uniform vec4 _UnlitColor;
uniform float _DiffuseThreshold;
uniform vec4 _OutlineColor;
uniform float _LitOutlineThickness;
uniform float _UnlitOutlineThickness;
uniform vec4 _SpecColor;
uniform float _Shininess;
// The following built-in uniforms (except _LightColor0)
// are also defined in "UnityCG.glslinc",
// i.e. one could #include "UnityCG.glslinc"
uniform vec3 _WorldSpaceCameraPos;
// camera position in world space
uniform mat4 _Object2World; // model matrix
uniform mat4 _World2Object; // inverse model matrix
uniform vec4 _WorldSpaceLightPos0;
// direction to or position of light source
uniform vec4 _LightColor0;
// color of light source (from "Lighting.cginc")
varying vec4 position;
// position of the vertex (and fragment) in world space
varying vec3 varyingNormalDirection;
// surface normal vector in world space
#ifdef VERTEX
void main()
{
mat4 modelMatrix = _Object2World;
mat4 modelMatrixInverse = _World2Object; // unity_Scale.w
// is unnecessary because we normalize vectors
position = modelMatrix * gl_Vertex;
varyingNormalDirection = normalize(vec3(
vec4(gl_Normal, 0.0) * modelMatrixInverse));
gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
}
#endif
#ifdef FRAGMENT
void main()
{
vec3 normalDirection = normalize(varyingNormalDirection);
vec3 viewDirection =
normalize(_WorldSpaceCameraPos - vec3(position));
vec3 lightDirection;
float attenuation;
if (0.0 == _WorldSpaceLightPos0.w) // directional light?
{
attenuation = 1.0; // no attenuation
lightDirection = normalize(vec3(_WorldSpaceLightPos0));
}
else // point or spot light
{
vec3 vertexToLightSource =
vec3(_WorldSpaceLightPos0 - position);
float distance = length(vertexToLightSource);
attenuation = 1.0 / distance; // linear attenuation
lightDirection = normalize(vertexToLightSource);
}
// default: unlit
vec3 fragmentColor = vec3(_UnlitColor);
// low priority: diffuse illumination
if (attenuation
* max(0.0, dot(normalDirection, lightDirection))
>= _DiffuseThreshold)
{
fragmentColor = vec3(_LightColor0) * vec3(_Color);
}
// higher priority: outline
if (dot(viewDirection, normalDirection)
< mix(_UnlitOutlineThickness, _LitOutlineThickness,
max(0.0, dot(normalDirection, lightDirection))))
{
fragmentColor =
vec3(_LightColor0) * vec3(_OutlineColor);
}
// highest priority: highlights
if (dot(normalDirection, lightDirection) > 0.0
// light source on the right side?
&& attenuation * pow(max(0.0, dot(
reflect(-lightDirection, normalDirection),
viewDirection)), _Shininess) > 0.5)
// more than half highlight intensity?
{
fragmentColor = _SpecColor.a
* vec3(_LightColor0) * vec3(_SpecColor)
+ (1.0 - _SpecColor.a) * fragmentColor;
}
gl_FragColor = vec4(fragmentColor, 1.0);
}
#endif
ENDGLSL
}
Pass {
Tags { "LightMode" = "ForwardAdd" }
// pass for additional light sources
Blend SrcAlpha OneMinusSrcAlpha
// blend specular highlights over framebuffer
GLSLPROGRAM
// User-specified properties
uniform vec4 _Color;
uniform vec4 _UnlitColor;
uniform float _DiffuseThreshold;
uniform vec4 _OutlineColor;
uniform float _LitOutlineThickness;
uniform float _UnlitOutlineThickness;
uniform vec4 _SpecColor;
uniform float _Shininess;
// The following built-in uniforms (except _LightColor0)
// are also defined in "UnityCG.glslinc",
// i.e. one could #include "UnityCG.glslinc"
uniform vec3 _WorldSpaceCameraPos;
// camera position in world space
uniform mat4 _Object2World; // model matrix
uniform mat4 _World2Object; // inverse model matrix
uniform vec4 _WorldSpaceLightPos0;
// direction to or position of light source
uniform vec4 _LightColor0;
// color of light source (from "Lighting.cginc")
varying vec4 position;
// position of the vertex (and fragment) in world space
varying vec3 varyingNormalDirection;
// surface normal vector in world space
#ifdef VERTEX
void main()
{
mat4 modelMatrix = _Object2World;
mat4 modelMatrixInverse = _World2Object; // unity_Scale.w
// is unnecessary because we normalize vectors
position = modelMatrix * gl_Vertex;
varyingNormalDirection = normalize(vec3(
vec4(gl_Normal, 0.0) * modelMatrixInverse));
gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
}
#endif
#ifdef FRAGMENT
void main()
{
vec3 normalDirection = normalize(varyingNormalDirection);
vec3 viewDirection =
normalize(_WorldSpaceCameraPos - vec3(position));
vec3 lightDirection;
float attenuation;
if (0.0 == _WorldSpaceLightPos0.w) // directional light?
{
attenuation = 1.0; // no attenuation
lightDirection = normalize(vec3(_WorldSpaceLightPos0));
}
else // point or spot light
{
vec3 vertexToLightSource =
vec3(_WorldSpaceLightPos0 - position);
float distance = length(vertexToLightSource);
attenuation = 1.0 / distance; // linear attenuation
lightDirection = normalize(vertexToLightSource);
}
vec4 fragmentColor = vec4(0.0, 0.0, 0.0, 0.0);
if (dot(normalDirection, lightDirection) > 0.0
// light source on the right side?
&& attenuation * pow(max(0.0, dot(
reflect(-lightDirection, normalDirection),
viewDirection)), _Shininess) > 0.5)
// more than half highlight intensity?
{
fragmentColor =
vec4(_LightColor0.rgb, 1.0) * _SpecColor;
}
gl_FragColor = fragmentColor;
}
#endif
ENDGLSL
}
}
// The definition of a fallback shader should be commented out
// during development:
// Fallback "Specular"
}
One problem with this shader are the hard edges between colors, which often result in noticeable aliasing, in particular at the outlines. This could be alleviated by using the smoothstep function to provide a smoother transition.
Summary
editCongratulations, you have reached the end of this tutorial. We have seen:
- What toon shading, cel shading, and non-photorealistic rendering are.
- How some of the non-photorealistic rendering techniques are used in toon shading.
- How to implement these techniques in a shader.
Further Reading
editIf you still want to know more
- about the Phong reflection model and the per-pixel lighting, you should read Section “Smooth Specular Highlights”.
- about the computation of silhouettes, you should read Section “Silhouette Enhancement”.
- about blending, you should read Section “Transparency”.
- about non-photorealistic rendering techniques, you could read Chapter 18 of the book “OpenGL Shading Language” (3rd edition) by Randi Rost et al., published 2009 by Addison-Wesley.