Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-114769

Multiview and related in the Qt graphics stack and Quick 3D XR

    XMLWordPrintable

Details

    • User Story
    • Resolution: Unresolved
    • P2: Important
    • None
    • 6.7, 6.8
    • Qt RHI, Quick: 3D
    • None

    Description

      Enable multiview in the graphics stack.

      Use it in Qt Quick 3D XR, and by extension, in every layer below in stack; unfortunately multiview affects everything down to the QRhi backend (with implications for the shaders and qsb/shadertools.

      This is not merely a performance optimization for VR/AR: the naive separate left/right eye rendering of Quick3D XR breaks API contracts in Qt Quick for instance (because one "frame" suddenly consists of two (left/right eye) frames; and so QQuickWindow signals suddenly and unexpectedly get emitted twice per frame etc.). For proper compatibility with the non-XR world multiview must be used.

      On the QRhi level it is known that this can be implemented for OpenGL (ES), Vulkan, and D3D12. Perhaps Metal as well.

      To prove the low-level integration works (QRhi, QRhi backends), have a manual test that renders a triangle or other geometry with two different transforms into two layers of a texture array using a single draw call, and then draw two textured quads to show what got rendered in that single draw call.

      To enable this for D3D12, it involves a bunch of related work for pipeline and shader management as well since we need HLSL 6.1 and fxc/D3DCompile will not support this. To do HLSL->DXIL compilation with dxc/IDxcCompiler needs some work both in qsb (simple) and in the runtime HLSL compilation support in the D3D12 backend (not so simple). Interesting build and deployment problems as well (ancient headers in MinGW, missing DLLs on regular PCs, etc.) Perhaps will be limited to MSVC? Also update windeployqt.
      On top, view instancing in D3D12 relies on the extensible PSO creation method (CreatePipelineState()), not the old CreateGraphicsPipelineState(). Have to switch.

      The multiview feature is not going to be supported on D3D11.

      Test MSAA as well. (must work with a texture array, so QRhiRenderBuffer is out; we never even exercised MSAA and auto-resolve with things other than 2D textures)

      Layers:

      • qtbase, QRhi, QRhi backends
      • qtshadertools Infrastructure for specifying view count, storing metadata, new HLSL versions, etc.
      • qtdeclarative, Qt Quick scenegraph (internal), built-in materials, public API additions and implications for QSGMaterial and ShaderEffect (plus build system consequences for qt_add_shaders, thinking of suitable HLSL versions for example, etc.)
      • qtquick3d Internal support, not sure if there are public API implications.
      • qtquick3dxr Changes from the XrSwapChain up to how cameras and the rendering logic work. May need to deal with various implications.

      Related work and features to investigate:

      • Multisampling. Even non-multiview MSAA is impossible to achieve for XrView without further enablers in Quick/Gui, whereas multiview support needs extra care when using in combination with MSAA. (investigate both the XrSwapchains MSAA support (which is usually not available) and also doing MSAA ourselves via the 3D API)
      • Issues due to VR compositor / OpenXR implementations insistence of handing out textures with a _SRGB format. Quick 3D XR does not want extra linear->sRGB conversions on already sRGB fragment colors. Needs further enablers in qtbase.
      • If we want to "submit the depth buffer", to help improving reprojections (XR_KHR_composition_layer_depth), that presents further interesting questions and complications, both with and without multiview. (one does not submit the depth buffer, but has to create a "depth swapchain", i.e. an XrSwapchain with a format like D32 or D24S8 and get the subimages from that, and render into that, instead of Qt creating its own texture/renderbuffer to depth-stencil)

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              lagocs Laszlo Agocs
              lagocs Laszlo Agocs
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes