Creating your own ShadingModel in UE4

As the title says, this post will go through quickly how to add in your own custom shading model into UE4 (version used in this post was 4.21)
Adding custom shading models is fairly straightforward and simple.

The easiest way to do this is actually by just following the code for other shading models, as UE's code tend to change a lot from version to version (last time I added a shading model was in 4.15 and the code has changed a lot since then so I had to re-learn how to do it which took quite a bit of time)

and also since the code changes so much making these posts is only truly helpful if you are integrating it to the version that the post was created for.

Anyway, here's how to do it in 4.21 at least. I won't go into explaining much but rather dump a bunch of screenshots, sorry :)

•  Start by setting r.ShaderDevelopmentMode=1 in Engine/Config/ConsoleVariables.ini

• This will save you the hazzle of crashing every time you miss something in the shader

• In Source/Runtime/Engine/Classes/Engine/EngineTypes.h declare your shading model

• In Source/Runtime/Engine/Private/Materials/MaterialShader.cpp add the following code in these functions

• Then in Source/Runtime/Engine/Private/Materials/MaterialShared.cpp add the following code into the SetupMaterialEnvironment function

  

• Then in Source/Editor/UnrealEd/Private/Lightmass/LightmassRender.cpp

  

• Now that that's done we can go ahead and choose which output nodes to enable and what to name them we do that in Source/Runtime/Engine/Private/Materials/Material.cpp in the IsPropertyActive function (in this example I've enabled CustomData0 & CustomData1)

 Note that a lot of parameters are always enabled as long as the shading model is not Unlit

• Then in Source/Editor/UnrealEd/Private/MaterialGraph.cpp you can rename your pins as you'd like
  (Since I enabled CustomData0 & CustomData1 and I did not feel like calling them that I chose to change their names)

 

• Now onto the shaders itself, first in Engine/Shaders/Private/DeferredShadingCommon.ush

• Then in Definitions.usf

• And in ShadingModelMaterial.ush in the SetGBufferForShadingModel you fill whatever parameters you need into the GBuffers

• If you plan on using the CustomData GBuffer you need to also add this into BasePassCommon.ush

If it's hard to read, all you need to do is add another OR ( || MATERIAL_MY_SHADINGMODEL) to the long or case for WRITES_CUSTOMDATA_TO_GBUFFER

• Then finally in ShadingModels.ush you can write your own BRDF

And that's it at least for a standard simple shadingModel, if you plan on adding a subsurface type shadingModel there are a few more places here and there that you need to adjust (just follow one of the subsurface shading models already in the engine to find those places)

and also if you plan on repurposing certain things in the GBuffer such as using GBuffer.metallic for something else than metallic then you need to add a lot of if cases in a lot of places but it should be fairly simple to find all of it.

I also chose to skip adding in the debugging for PixelInspector but that also is pretty straightforward if you follow other shadingModels in the code