Custom materials in Qt 6

Custom materials are not very usable in practice.

• Change the frontend (the QML API) to be user friendly. Multi-pass support (which is not implemented in practice) should be removed from the API. Writing a CustomMaterial element should be as simple as writing a ShaderEffect. As an example, a revised version of the customshader example:

CustomMaterial {
    property real time: 0.0
    property real amplitude: 5.0
    property real alpha: 1.0

    shadingMode: CustomMaterial.Unshaded
    hasTransparency: alpha < 1.0
    sourceBlend: CustomMaterial.SrcAlpha
    destinationBlend: CustomMaterial.OneMinusSrcAlpha
    cullMode: CustomMaterial.BackFaceCulling

    vertexShader: "example.vert"
    fragmentShader: "example.frag"
}

• We need to start differentiating between unshaded (unlit) custom materials, where the shader code is almost completely user provided and does not care about lighting, light probes, shadows, anything in the environment. This is done by the shadingMode property as shown above. The metadata handling needs to change: writing the ugly JSON blocks in a comment in an ifdef is not quite acceptable.

From the customshader example:

#ifdef QQ3D_SHADER_META
/*{
  "inputs": [
    { "stage": "vertex", "type": "vec3", "name": "attr_pos" }
  ],
  "outputs": [
    { "stage": "vertex", "type": "vec3", "name": "pos" }
  ]
}*/
#endif

void main()
{
    pos = attr_pos;
    pos.x += sin(time * 4.0 + pos.y) * amplitude;
    gl_Position = modelViewProjection * vec4(pos, 1.0);
}

this really should have been something like:

VARYING vec3 pos;
void main()
{
    pos = VERTEX;
    pos.x += sin(time * 4.0 + pos.y) * amplitude;
    POSITION = MODELVIEWPROJECTION * vec4(pos, 1.0);
}

This needs a set of smart lookups and substitutions (varying, uppercase built-ins)

• Custom materials in "shaded" mode. These should be completely changed, possibly removing the existing custom material generators completely. The existing approach (https://doc.qt.io/qt-5/custom-material-reference.html) is not usable in practice.

Rather, rely on the default material generator, so that empty shader snippets in a CustomMaterial shader snippet should effectively behave almost as if it was a DefaultMaterial.

Have functions in the snippets plug in to the generated shader code, e.g. a void light() { } function could, if present, control the lighting calculations. Inject inout and in arguments.

• Custom materials (and effects) should support referencing a QSGTexture in textures. Currently this is completely missing.

• Depth and shadow passes do not use the material's vertex shader at all at the moment. This is a problem: a custom vertex shader can displace vertices for example, which is then not reflected in the
  • shadows.
  • or in the depth texture used by SSAO or effects. (or accessed by the custom material itself)
  • or in the depth prepass, when enabled

• Needs Lancelot tests. For all the various cases. Lots of combinations...

• Depth and SSAO textures should be accessible in custom material shaders. Esp. in Unshaded, since Shaded gets built-in SSAO support. More importantly: depth texture is mandatory for screen reading (see below) anyway. Could be that these will need to be opted in via CustomMaterial properties, which then leads to magic sampler uniforms e.g. DEPTH_TEXTURE or AO_TEXTURE becoming usable.

• Screen reading shaders, both color and depth. (getting a copy of the main framebuffer, be it the backbuffer for a swapchain or the texture we are rendering to, containing the objects that got rendered up to the model with the custom material in question) (i.e. like SCREEN_TEXTURE and DEPTH_TEXTURE in Godot)
  Required for glass-like materials for example, to do the refraction.
  The sane solution here is to generate the screen texture once per scene, with the opaque objects only, not including transparent objects. Generating individual textures for each and every object is not feasible. The depth texture already works like this, including all (but only) opaque objects.

• Old custom material doc page to be removed. Document CustomMaterial properly. (like ShaderEffect)

• Existing custom materials go away