The power of byte code (Shader Parameter System)

After reading through the implementation of the shader system used in Destiny , in which they use bytecode for shader parameter setting, I thought to myself, why don't I expand upon this idea to handle descriptor sets? In this blog I will go through a few of the steps I took to make this into a reality. 

Resource Sets

The Black Osiris Engine has a concept of "resource sets". A resource set is a combination of shader parameters , textures, and buffer resources. An example of a resource set is a grouping of albedo and normal textures for a mesh, or a lighting buffer for a tiled shading pass. For the purposes of efficiency, Resource Sets are split based on update frequency. Currently there are four resource slots; EResourceLayoutSlot_PerInstance, EResourceLayoutSlot_PerInstanceCustomData, EResourceLayoutSlot_PerMaterial, and EResourceLayoutSlot_PerPass. 

To make resource set creation and management easier, each resource set is accompanied by a C++ class.

Example of a Resource set used for the tiled deferred shading stage.

Function that registers the resource set struct with the global system, and states that it is used for compute shader stages.

This macro definition generates meta information that will be used to generate bytecode explained later. The user provides the name of the resource set and the parameters (typename,name,shader visibility,arraysize). Below is an example of the creation and setting of this resource set.

Example of creation and setting of a resource set

In this example the user creates a resource set with the usage of "SingleFrame". In the DirectX12 implementation, resource sets with the usage of "SingleFrame" are allocated from the per frame descriptor heaps , while DirectX11 resource sets allocate data from a per frame stack allocator. After creating the resource set , the user sets the shader parameters and then proceeds to "compile" the resource set through the function "UploadContents".  The user then sets this resource set in the commandlist at the "EResourceLayoutSlot_PerPass" binding location.

API Abstraction : D3D11

As said previously, I've chosen to use a bytecode implementation for resource sets in the api backends. The Directx 11 implementation has a concept known as a "CScriptLayout" which has a 1:1 relation to resource sets and a 1:1 relation to pipeline state objects. The CScriptLayout contains a packed buffer of bytecode data that dictates where each shader parameter should be bound to. Prior to issuing a draw call, the directx backend checks to see if any of the currently bound resource sets are dirty , i.e have been updated since the last draw call, and calls the function "CommitResourceTablesCommon". 

This function traverses the compiled bytecode for each resource set and gets the proper resources from the "ResourceStream" , the opaque byte data that holds all of the resources set prior to calling "UploadContents".