Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-90398

QRhi support for retrieving and loading the blobs for the Vulkan pipeline cache + a fake version with OpenGL program binaries



    • Task
    • Resolution: Done
    • P2: Important
    • 6.1.0 Alpha
    • None
    • Qt RHI
    • None
    • df0e98d4080f50de7ecacdc4cae079ab31280481 (qt/qtbase/dev)


      In Qt 5 the ability to hack in direct OpenGL calls to collect, export, and import shader program binaries generated during the lifetime of a scene was available by design in Qt Quick 3D or Qt Quick.

      In Qt 6 this needs a more centralized solution, also taking into account what's available with some other APIs, particularly with Vulkan in form of https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkPipelineCache.html

      • This has nothing to do with Qt Quick 3D's material shader pre-generation feature (genshaders tool). They are independent features, covering two distinct phases of the holistic graphics/asset pipelines.
        • The material shader generation covers the generation of plain text Vulkan-style GLSL code for a given material, compiling the resulting shader source code into SPIR-V, and then translating back into source form for other shading languages (GLSL, HLSL, MSL).
        • While this avoids the time consuming shader source code generation at run time, it cannot prevent spending time on shader compilation or graphics pipeline object creation (even with SPIR-V, compilation from the intermediate format to the vendor-specific binary format will likely happen, invisible to the application or to Qt).
          • That's where the Vulkan pipeline cache, or OpenGL program binaries come in.
      • It also does not replace or interact with Qt's traditional OpenGL program binary disk cache, although the implementation may be eventually somewhat intertwined in the QRhi's OpenGL backend.

      We are not interested in an individual shader or "shader program" level solution since this is not applicable to anything but OpenGL. Rather, the QRhi API is expected to expose simple QByteArray-based functions to export and import the blob for the pipeline cache (which is not a concept visible on the QRhi API).

      • With Vulkan this maps naturally to vkGetPipelineCacheData and co. (see e.g. https://zeux.io/2019/07/17/serializing-pipeline-cache/ )
      • With OpenGL this would all be faked via gl(Get)ProgramBinary, likely utilizing the cache key calculation we have in place for the disk cache. This means that the OpenGL-based of QRhi would, when enabled, maintain a list of the program binaries from all QRhiGraphicsPipelines that were created (perhaps only when a flag was set on the pipeline object), associated with appropriate keys, also allowing serialization into blob form. Then the opposite for importing. The lookup in QRhiGraphicsPipeline::create() would be very similar what we already do with the disk cache.
      • Other backends are not interesting for the time being.

      The QRhi API can be as simple as the following. The opt-in flag is required in order not to introduce unacceptable slowdowns with OpenGL due to retrieving program binaries. (but this would be ignored with Vulkan; it also does not need to affect the setter)

      The QRhi flag should likely be mapped to an environment variable as well with Qt Quick, so one can opt-in for the QRhi created by Qt Quick (also used by Quick 3D) by setting some QSG_RHI_nnn env.var.

      diff --git a/src/gui/rhi/qrhi_p.h b/src/gui/rhi/qrhi_p.h
      index 3c7f9b6bc4..11f532f0a7 100644
      --- a/src/gui/rhi/qrhi_p.h
      +++ b/src/gui/rhi/qrhi_p.h
      @@ -1488,7 +1488,8 @@ public:
           enum Flag {
               EnableProfiling = 1 << 0,
               EnableDebugMarkers = 1 << 1,
      -        PreferSoftwareRenderer = 1 << 2
      +        PreferSoftwareRenderer = 1 << 2,
      +        EnablePipelineCacheRetrieval = 1 << 3
           Q_DECLARE_FLAGS(Flags, Flag)
      @@ -1637,6 +1638,9 @@ public:
           bool isDeviceLost() const;
      +    QByteArray pipelineCacheData();
      +    void setPipelineCacheData(const QByteArray &data);


        For Gerrit Dashboard: QTBUG-90398
        # Subject Branch Project Status CR V



            lagocs Laszlo Agocs
            lagocs Laszlo Agocs
            0 Vote for this issue
            1 Start watching this issue



              Gerrit Reviews

                There are no open Gerrit changes