How to cut holes through Unity 5 terrain

Because Unity's terrain is literally just a flat plane where the vertices are raised or lowered you can't actually modify the mesh directly to cut out a physic hole so the only option left is to cheat. If you have Googled this question before you will likely have seen that using a depth masking shader is your best bet to "cut" a hole into the terrain. Unfortunately all those shaders or forum posts are pre-5.0 before Unity switched to nice physical-based lighting shaders for it's terrain material. It took me quite a while of digging and playing around but I found a solution without having to pay for a custom package. 

The idea here is we need to put a custom material on the terrain with a modified shader on it and then we need another material with a depthmask shader on any object we want to be used as the masker. What this will achieve is that if the masking object is in front of the terrain then your camera will not render the terrain that is behind the object. So it appears invisible even though the terrain is still actually there.

Here is how you do it.

Create two shaders anywhere in your assets folder, name them whatever you like. Then paste the code below into each one respectively. One is for the terrain material and the other is for the depth masking material. The terrain one is from the standard terrain shaders I just modified it, and the other is from the Unity Wiki post about depthmasking.

       
Shader "Custom/Terrain" {
 Properties {
  // set by terrain engine
  [HideInInspector] _Control ("Control (RGBA)", 2D) = "red" {}
  [HideInInspector] _Splat3 ("Layer 3 (A)", 2D) = "white" {}
  [HideInInspector] _Splat2 ("Layer 2 (B)", 2D) = "white" {}
  [HideInInspector] _Splat1 ("Layer 1 (G)", 2D) = "white" {}
  [HideInInspector] _Splat0 ("Layer 0 (R)", 2D) = "white" {}
  [HideInInspector] _Normal3 ("Normal 3 (A)", 2D) = "bump" {}
  [HideInInspector] _Normal2 ("Normal 2 (B)", 2D) = "bump" {}
  [HideInInspector] _Normal1 ("Normal 1 (G)", 2D) = "bump" {}
  [HideInInspector] _Normal0 ("Normal 0 (R)", 2D) = "bump" {}
  [HideInInspector] [Gamma] _Metallic0 ("Metallic 0", Range(0.0, 1.0)) = 0.0 
  [HideInInspector] [Gamma] _Metallic1 ("Metallic 1", Range(0.0, 1.0)) = 0.0 
  [HideInInspector] [Gamma] _Metallic2 ("Metallic 2", Range(0.0, 1.0)) = 0.0 
  [HideInInspector] [Gamma] _Metallic3 ("Metallic 3", Range(0.0, 1.0)) = 0.0
  [HideInInspector] _Smoothness0 ("Smoothness 0", Range(0.0, 1.0)) = 1.0 
  [HideInInspector] _Smoothness1 ("Smoothness 1", Range(0.0, 1.0)) = 1.0 
  [HideInInspector] _Smoothness2 ("Smoothness 2", Range(0.0, 1.0)) = 1.0 
  [HideInInspector] _Smoothness3 ("Smoothness 3", Range(0.0, 1.0)) = 1.0

  // used in fallback on old cards & base map
  [HideInInspector] _MainTex ("BaseMap (RGB)", 2D) = "white" {}
  [HideInInspector] _Color ("Main Color", Color) = (1,1,1,1)
 }

 SubShader {
  Tags {
   "Queue" = "AlphaTest"//"Geometry-100"
   "RenderType" = "Transparent" //"Opaque"
  }

  CGPROGRAM
  //surf Standard
  #pragma surface surf Standard vertex:SplatmapVert finalcolor:SplatmapFinalColor finalgbuffer:SplatmapFinalGBuffer fullforwardshadows 

  #pragma multi_compile_fog
  #pragma target 3.0
  // needs more than 8 texcoords
  #pragma exclude_renderers gles
  #include "UnityPBSLighting.cginc"

  #pragma multi_compile __ _TERRAIN_NORMAL_MAP

  #define TERRAIN_STANDARD_SHADER
  #define TERRAIN_SURFACE_OUTPUT SurfaceOutputStandard
  #include "TerrainSplatmapCommon.cginc"

  half _Metallic0;
  half _Metallic1;
  half _Metallic2;
  half _Metallic3;
  
  half _Smoothness0;
  half _Smoothness1;
  half _Smoothness2;
  half _Smoothness3;
  
  //surface function
  void surf (Input IN, inout SurfaceOutputStandard o) {
   half4 splat_control;
   half weight;
   fixed4 mixedDiffuse;


   half4 defaultSmoothness = half4(_Smoothness0, _Smoothness1, _Smoothness2, _Smoothness3);
   SplatmapMix(IN, defaultSmoothness, splat_control, weight, mixedDiffuse, o.Normal);
   o.Albedo = mixedDiffuse.rgb;

   o.Alpha = 1 - splat_control.a;//weight;
   
   o.Smoothness = mixedDiffuse.a;
   o.Metallic = dot(splat_control, half4(_Metallic0, _Metallic1, _Metallic2, _Metallic3));

  }
  ENDCG
 }

 Dependency "AddPassShader" = "Hidden/TerrainEngine/Splatmap/Standard-AddPass"
 Dependency "BaseMapShader" = "Hidden/TerrainEngine/Splatmap/Standard-Base"

 Fallback "Nature/Terrain/Diffuse"
}

 

       
Shader "Custom/Mask" {

 SubShader{
  // Render the mask after regular geometry, but before masked geometry and
  // transparent things.

  Tags{ "Queue" = "Geometry+10" }

  // Don't draw in the RGBA channels; just the depth buffer

  ColorMask 0
  ZWrite On

  // Do nothing specific in the pass:

  Pass{}
 }
}
       
 

Next step you must create two materials anywhere in your assets folder. You can name them whatever you like, but I like to call them "_InvisibleTerrain" and "_InvisibleObject" just so they appear at the top of my list of materials.

Now just assign the shaders to each one.
-_InvisibleTerrain should be using the shader "Custom/Terrain"
-_InvisibleObject should be using the shader "MaskingObject"

Now just apply the material to your terrain by changing the Material tab to Custom, then you will have the option to pick a material from your project. Use the _InvisibleTerrain material and you are good. Next make any object you want in my case I just used a Cube. Now just apply the _InvisibleObject material to the Cubes MeshRenderer and you are done. Now anywhere you see the Cube if it is in front of terrain that is using that material then the terrain will appear transparent.

Here is the end result.

Now there are a couple of catches.

First is that there is no real hole, just that where ever the cube is the terrain behind it won't render. So there will still be collision. However, because you are using a cube with a physics collider you can just set it's collision to be a trigger and when the player is passing through it just turn off the players collider and turn it back on when the player leaves the trigger volume. Not the most ideal way but it should work.

The other catch is that because there is backface culling on objects if the masking object is big and the player enters it then they will not see that masking material and the terrain will be visible again. So you have two options.
1. Lazy option is scaling your object down so the player doesn't really have a chance to be inside the object. This is what I did.
2. Modify the Masking shader to allow for 2 sided polygons, I'm not sure how to do this so option 1 should be good enough.

Hope this helps you guys as much as it did for me.