Cg Programming/Unity/Portals

This tutorial covers the rendering of portals and magic lenses.

It requires quite some knowledge about shader programming in general, and specifically Section “Programmable Graphics Pipeline” and Section “Vertex Transformations”.

What is a "portal" and a "magic lens"?

The term "portal" is used with multiple meanings in computer graphics. Usually, it refers to objects that accelerate the computation of surface visibility by "portal rendering". Here, however, we use the term "portal" to describe the concept of a 3D object (often a planar object) in a 3D scene that allows us to look into another 3D scene; i.e., a different 3D scene than the scene around the portal appears at the position of the portal. A "magic lens" is technically very similar but in this case the other scene is usually very closely related to the scene around the magic lens.

Some use cases for portals: rendering a 3D scene that includes a (virtual) display that shows another 3D scene; rendering a portal into another time and/or location; etc. Some use cases for magic lenses: rendering a simulated augmented-reality display; rendering a lens/filter that shows the same scene but in a different style; etc.

How to render a portal with one camera?

This section presents a method using a single camera. It follows these steps:

Render the scene where the portal is located (but without the portal).
Render the portal into the scene and mark all pixels where the portal is visible in the stencil buffer.
Clear the depth buffer where the portal is visible (as specified in the stencil buffer).
Render the other scene where the portal is visible (as specified in the stencil buffer) but only parts of the scene that are behind the portal.

Let's look at these steps one by one.

Rendering the scene where the portal is located

This can use any rendering techniques for opaque objects as long as the depth buffer is set correctly such that the portal can be inserted with correct occlusions. (Transparent objects can result in artifacts with this method if they are behind the portal.)

Rendering the portal into the scene

This step has to be performed after the rest of the opaque scene is rendered such that the portal is only rasterized into those pixels where it is actually visible. We make sure that the portal is rendered after the opaque scene by using this line in the shader for the portal:

      Tags { "RenderType"="Opaque" "Queue"="Geometry+200"}

Geometry+200 specifies that this object should be rendered after opaque objects. (Instead of 200, any other positive number is also fine.)

Furthermore, this pass has to mark the visible pixels of the portal in the stencil buffer by setting the values of the pixels in the stencil buffer to a particular value, let's say 1. (The default stencil value of all pixels is 0.) The code for setting the stencil value for all rasterized pixels to 1 looks like this:

         Stencil {
            Ref 1
            Comp Always // always pass stencil test
            Fail Keep // do not change stencil value if stencil test fails
            ZFail Keep // do not change stencil value if stencil test passes but depth test fails
            Pass Replace // set stencil value to 1 if stencil test and depth test pass
         }

This code specifies the stencil test but since we don't want to test anything, we set the comparison (Comp) to Always, i.e., the "test" always passes. Therefore, our stencil test should never fail, but just to be safe, we set the operation for a failing stencil test (Fail) to Keep, i.e., we don't change the value of the stencil buffer in this case. In case the depth test fails (ZFail), i.e., the portal is occluded by something, we don't want to mark the pixel (since the portal is not visible in this pixel). Therefore, we set the operation for a failing depth test (and passing stencil test) to Keep, i.e., no change of the stencil value for this pixel. Lastly, for all pixels where the depth test does not fail and our always-pass stencil test passes (Pass), we set the operation to Replace, which means that the reference value (the number after Ref) is written into the stencil value of a pixel.

The complete shader code for the portal could then look like this:

Shader "Custom/PortalShader" {
   SubShader
   {
      Tags { "RenderType"="Opaque" "Queue"="Geometry+200"}
      
      Pass
      {
         Stencil {
            Ref 1
            Comp Always // always pass stencil test
            Fail Keep // do not change stencil value if stencil test fails
            ZFail Keep // do not change stencil value if stencil test passes but depth test fails
            Pass Replace // set stencil value to 1 if stencil test and depth test pass
         }

         Cull Off // turn off backface culling
         
         CGPROGRAM
         #pragma vertex vert
         #pragma fragment frag
         
         #include "UnityCG.cginc"

         float4 vert (float4 vertex: POSITION) : SV_POSITION
         {
            return UnityObjectToClipPos(vertex);
         }
         
         fixed4 frag () : SV_Target
         {
            return float4(0.0, 1.0, 0.0, 1.0);
         }
         ENDCG
      }
   }
}

We set the color to an arbitrary color (green in this case), which is useful for debugging but doesn't matter since it will be overwritten by later objects.

Clearing the depth buffer where the portal is visible

Before we can render the other scene into the pixels where the portal is visible, we have to clear the depth buffer. There are multiple ways of clearing the depth buffer only in those pixels that are marked in the stencil buffer. The easiest way is to simply render a sphere that is large enough to include the other scene (the scene that is seen through the portal). You can think of this sphere as a skydome for the other scene. If the sphere is large enough, the resulting depth values are larger than any depth values of the scene. This is not quite the same as clearing the depth buffer but good enough.

To make sure that this sphere is rendered after the portal, we use Geometry+210 (instead of 210, any value larger than the value that we used for the portal would also work):

      Tags { "RenderType"="Opaque" "Queue"="Geometry+210"}

To make sure that we render the sphere even if we look at it from the inside, we use Cull Off.

To make sure that we render the sphere even if it is geometrically occluded by other objects, we use ZTest Always.

Since we want to clear the depth buffer only for those pixels where the stencil buffer has been set to 1, we use the following stencil test:

         Stencil {
            Ref 1
            Comp Equal // only pass stencil test if stencil value equals 1
            Fail Keep // do not change stencil value if stencil test fails
            ZFail Keep // do not change stencil value if stencil test passes but depth test fails
            Pass Keep // keep stencil value if stencil test passes
         }

The comparison (Comp) is set to Equal such that only pixels with a stencil value equal to the reference value 1 (Ref 1) pass the stencil test and all other pixels are discarded. The operations are all set to Keep since we don't want to change the stencil value in any case.

The complete shader might look like this:

Shader "Custom/ClearDepthShader" {
   SubShader
   {
      Tags { "RenderType"="Opaque" "Queue"="Geometry+210"}
      
      Pass
      {
         ZTest Always // always pass depth test (nothing occludes this material)        
         Cull Off // turn off backface culling

         Stencil {
            Ref 1
            Comp Equal // only pass stencil test if stencil value equals 1
            Fail Keep // do not change stencil value if stencil test fails
            ZFail Keep // do not change stencil value if stencil test passes but depth test fails
            Pass Keep // keep stencil value if stencil test passes
         }
         
         CGPROGRAM
         #pragma vertex vert
         #pragma fragment frag
         
         float4 vert (float4 vertex: POSITION) : SV_POSITION
         {
            return UnityObjectToClipPos(vertex);
         }
         
         fixed4 frag () : SV_Target
         {
            return float4(0.0, 0.0, 0.0, 0.0);
         }
         ENDCG
      }
   }
}

We set the fragments' color to black, which will be the background of the other scene. See Section “Skyboxes” for shader code to render skyboxes, which could be extended with the stencil test.

Rendering the other scene where the portal is visible

Once the depth buffer is cleared (and a background is rendered), we can render the opaque geometry of the other scene. To make sure that we render it only after clearing the depth buffer, we use Geometry+220 (instead of 220, any value larger than the value that we used for clearing would also work)::

      Tags { "RenderType"="Opaque" "Queue"="Geometry+220"}

We use the same stencil test as for clearing the depth buffer (see the previous section) since we want to rasterize the other world in exactly the same pixels.

There are three more issues. One is that it might be necessary to avoid shadows that are cast from the scene around the portal into the other scene, i.e., the shader for the other scene should not receive any shadows. This is particularly important if objects cast shadows that are not visible in the other scene. In a surface shader, we can avoid any shadow computations with the noshadow keyword in this line:

#pragma surface surf Standard noshadow

The second issue is that the objects of the other scene should not cast shadows into the scene around the portal. Again, the problem is that potentially invisible objects should not cast shadows. When we use a surface shader, we can avoid shadow casting by using Fallback Off in order not(!) to specify a fallback shader, since fallback shaders usually include a shadow-casting pass with "LightMode" = "ShadowCaster".

The third issue is that objects of the other scene that are in front of the portal usually should be clipped. This can be achieved by transforming the fragment position in world coordinates (3D vector IN.worldPos) into the local coordinate system of the portal. In the shader code below, the 4x4 transformation matrix is _WorldToPortal. We assume that the portal is a standard Unity "quad" where the surface normal vector is in the direction of the local z axis. Then the sign of the local z coordinate tells us whether the fragment is in front or behind the portal. We can do the same transformation with the world position of the camera (3D vector _WorldSpaceCameraPos); and the sign of the local z coordinate of the camera position tells us whether the camera is in front or behind the portal. We want to clip (i.e. discard) a fragment if its local z coordinate has the same sign as the local z coordinate of the camera position. In code:

         if (mul(_WorldToPortal, float4(_WorldSpaceCameraPos, 1.0)).z > 0.0) {
            if (mul(_WorldToPortal, float4(IN.worldPos, 1.0)).z + _ClipDistanceOffset > 0.0) { 
               // position on same side of portal as camera?
               discard; // discard fragment
            }
         }
         else {
            if (-mul(_WorldToPortal, float4(IN.worldPos, 1.0)).z + _ClipDistanceOffset > 0.0) { 
               // position on same side of portal as camera?
               discard; // discard fragment
            }
         }

The variable _ClipDistanceOffset is 0 by default. Positive values move the clipping plane into the other scene, negative values move the clipping plane outwards. This is useful to avoid rendering artifacts.

The complete shader for diffuse materials in the other scene could then look like this:

Shader "Custom/OtherworldShader" {
   Properties {
      _Color ("Color", Color) = (1,1,1,1)
      _ClipDistanceOffset ("Clip Offset", Float) = 0.0
   }
   SubShader {
      Tags { "RenderType"="Opaque" "Queue" = "Geometry+220" }
      
      Stencil {
         Ref 1
         Comp Equal // only pass stencil test if stencil value equals 1
         Fail Keep // do not change stencil value if stencil test fails
         ZFail Keep // do not change stencil value if stencil test passes but depth test fails
         Pass Keep // do not change stencil value if stencil test passes
      }
            
      CGPROGRAM
      #pragma surface surf Standard noshadow // don't receive shadows

      fixed4 _Color;
      float _ClipDistanceOffset;
      float4x4 _WorldToPortal;
      
      struct Input {
         float3 worldPos;
      };

      void surf (Input IN, inout SurfaceOutputStandard o) {
         o.Albedo = _Color;
         if (mul(_WorldToPortal, float4(_WorldSpaceCameraPos, 1.0)).z > 0.0) {
            if (mul(_WorldToPortal, float4(IN.worldPos, 1.0)).z + _ClipDistanceOffset > 0.0) { 
               // position on same side of portal as camera?
               discard; // discard fragment
            }
         }
         else {
            if (-mul(_WorldToPortal, float4(IN.worldPos, 1.0)).z + _ClipDistanceOffset > 0.0) { 
               // position on same side of portal as camera?
               discard; // discard fragment
            }
         }
      }
      ENDCG      
   }
   Fallback Off 
      // no fallback shader, thus, no pass with tag "LightMode" = "ShadowCaster" 
      // and therefore not casting shadows
}

To set the shader variable _WorldToPortal, the following C# script sets the variable in the material of a specific object, thus, for each material using the shader, the script should be attached to one of the objects using that material:

using UnityEngine;

[ExecuteInEditMode]
public class SetMatrixProperty : MonoBehaviour {
   public GameObject portal;
   public Material otherWorldMaterial;
   
   void Update () {
      Renderer portalRenderer = portal.GetComponent<Renderer>();
      otherWorldMaterial = GetComponent<Renderer>().sharedMaterial;
      otherWorldMaterial.SetMatrix("_WorldToPortal", portalRenderer.worldToLocalMatrix);
   }	
}

This code also runs in the Unity editor. If the script does not need to run in the editor, it can be optimized by assigning portalRenderer and otherWorldMaterial in a Start function.

Limitations

There are three main limitations of this approach:

It's not possible to go through the portal.
Transparent objects may result in incorrect occlusions.
Shadow casting and receiving in the other scene might have to be disabled.

Going through the portal requires switching the roles of the scene with the other scene. One solution is to have two copies of all objects: one for the case that the object is in the scene of the portal, and one for the case that it is in the other scene. By activating the appropriate copy of each object, the correct material can be used.

Some of the problems with transparent objects could be solved by rendering the depth of the portal again, i.e., without rasterizing colors (using ColorMask 0). This should follow the rendering of transparent objects in the other scene (behind the portal) but precede the rendering of transparent objects of the scene around the portal. This is left as an exercise for the reader.

Another way to address the problems with transparency and shadows is to use a second camera. This would work by setting up a second camera and synchronizing its position and rotation with the main camera (but in the scene behind the portal). This second camera renders the scene behind the portal into a Render Texture, which is then used to texture the portal in the main scene. This would work very similar to the flat mirror in Section “Mirrors”. The disadvantage is mainly the performance cost of using a render texture and potential problems with using an additional camera that has to be synchronized with the main camera.

Summary

This tutorial discussed a method to render a portal or magic lens with a single camera. This is particularly useful if you want to avoid using multiple cameras. However, it also results in problems with transparent objects and shadows, which might not be acceptable in many applications.