Lightmap support

The 2022 exercise provides a whole new implementation from scratch: see QTBUG-103340 for details.

Below is the original description for the spring 2021 exercise. That revolved more around the basics of lightmapping and the performance aspect, with not much focus on global illumination or the quality gains raytraced baked lighting can provide. The results from that are mostly
documented in https://intranet.qt.io/display/QTRD/Qt+Quick+3D%3A+Lightmap+baking

While not necessarily obvious to many users (often due to postponed/inappropriate focus on performance in the real production environment), the realtime lighting and shadow mapping that is performed by the Quick3D pipeline and the materials' generated shaders, is not an approach that is suitable as the primary choice for complex real world applications and systems deployed on mobile and embedded class hardware, even when considering the ever-increasing power and performance budgets of present and upcoming SoCs/GPUs.

For this task we need to research and provide the foundations of a solution that is able to facilitate Qt Design Studio and standalone tools to (pre-)generate lightmap textures which can then be used as alternative to real-time lighting for materials in a scene, at least for "static" objects (where both the geometry and the object's inherited model transform is static; this comes in addition to all lights in the scene being static).

Lightmapping is a complex topic, with two main problem domains from our prespective:

• The actual lightmap baker implementation. For Quick3D purposes the primary goal is to do something with the relatively simple UV space rasterization approach. State of the art approaches are different from this, and those are not in our scope at this time. Ray/Path tracing based bakers providing full global illumination support, functioning either through GPU compute or some accelerated raytracing approach are not in scope here. Baking shadow maps is going to be essential too, but it is to be considered in a later phase of this work, once the approach for handling maps with the directional/point/spot light contribution has crystalized.

• How exactly baking and then the usage of the lightmaps at runtime happens, what is the API and tooling? (is it manually controlled via the QML API? or does it involve parsing QML source code, not unlike the shader pregeneration tool? how does it plug into Design Studio? what about dynamic scenes where lights or some model does change eventually, invalidating the pre-baked lightmaps? etc.)

This is one of the most complicated tasks we have currently, and requires significant effort due to its overarching and somewhat intrusive nature, with a number of unclear and not yet seen aspects.

It is expected that, especially the #2 point above is going to be a long-term development item, iterating and evolving for a long time after Qt 6.2 as well.

Plan for a first phase:

The QML API currently inherits from Qt 3D Studio / NDD three "lightmap" Texture properties in the Material base class: Indirect, Radiosity, Shadow

While the generated shader for the default/principled material may support using (sampling) these maps, it has no facilities for generating them, nor any documentation of how to generate them, or what to do regarding lightmap-compatible UV coordinate sets, using external tooling (Maya, 3DS-Max, Blender).

In the first iteration most of this will need to be replaced with a more functional solution, that gives us the possibility to:

• having an additional, lightmap-compatible UV set for meshes (UV2?)
  • there must be a solution for automatically generating a suitable UV set, this will involve pulling in some 3rd party solution, https://github.com/jpcy/xatlas most likely
  • when exactly this UV generation happens, is not yet decided (at asset import? at lightmap baking time?)
  • optionally, we can in addition figure out how one can generate a suitable UV set in tools like Blender, and have appropriate instructions in the docs
• the existing properties need to be revamped, most likely replaced by a single lightMap Texture for now
• devise a standalone tool, that can load and render a Quick 3D scene (preferably offscreen), while switching the rendering of certain Model objects into a special lightmap-baking mode, where the model's submeshes get rendered into a texture in UV2 space)
  • the generated textures are then read back and saved to files (or packed together in a single container of some sort?)
• the application's QML code can then reference these generated lightmaps in the Material's lightMap property
  • there are open questions around this since having the property on the material is questionable for multiple reasons (mainly due to the lightmap being mesh/transform dependent, and cannot be reused for another Model, not even if it's the same mesh) It could also be that the long-term solution is something else, without manual lightmap texture management, but that is out of scope for the initial implementation.,
  • demonstrate that setting the lightMap on a Model gives similar results to realtime lighting, without performing the lighting calculations in the material's fragment shader
  • prove the benefits on real embedded devices

Second phase

Shadow maps? "Automatic" baking for all models of a scene? How does the Designer workflow look like? Many open questions.