Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-114769 Multiview and related in the Qt graphics stack and Quick 3D XR
  3. QTBUG-122292

Option to submit depth buffer to compositor (XR_KHR_composition_layer_depth)

    XMLWordPrintable

Details

    • Sub-task
    • Resolution: Done
    • P2: Important
    • None
    • None
    • None
    • e36c1a8d8 (dev), 8e45b822d (dev), 8e0ae9599 (dev), ffe4d06c8 (dev)

    Description

      When multiview and MSAA (and their combination) is tackled, we need to look into the XR_KHR_composition_layer_depth extension, because support for this, if it is something we want supported at any point, ties in to the exact same places in QRhiTextureRenderTarget, QQuickRenderTarget, and all the XrSwapchain management, it just revolves around the depth(stencil) texture we render into, not the color buffer images.

      This is riddled with challenges as well. Consider what happens with MSAA enabled: we now need a resolved depth texture (meaning that the resolved rendering should end up in the depth xrswapchain-provided texture). This isn't something we support atm, there is no use case for resolving a multisample depth texture right now. For instance, there is no way to pass in a "depthResolveTexture" in addition to the resolveTexture in a QRhiColorAttachment.

      An outline:

      • Check for XR_KHR_composition_layer_depth. Enable it when supported. But perhaps have a dedicated flag to opt-in to actual usage for now? (e.g. env.var.) Later maybe swap it around so that the env.var is for opting out, not in? But not entirely sure if that's the way to go, so perhaps check what others do.
        • The expectation is that submitting the depth buffer improves reprojection quality (so presumably this is more interesting for heavy content that is expensive to render, so frame rate drops occur).
        • However it may have performance tradeoffs. (would guess that when using this, the depth buffer contents has to be written out, and with MSAA it needs to be resolved, which is not necessarily something that happens normally with tiled architecture GPUs, so the load will increase)
        • https://learn.microsoft.com/en-us/windows/mixed-reality/develop/native/openxr-best-practices ("Always use XR_KHR_composition_layer_depth ...")
        • The Unity docs say: "Depth submission is only supported on selected VR hardware. In Unity, depth layers can be enabled by opening Project Settings > XR Plug-in Management > OpenXR and selecting a Depth Submission Mode of Depth 16 Bit or Depth 24 Bit. Depth images corresponding to the default projection layer are then automatically generated and submitted with no further action required by the developer." This would imply it is an opt-in feature in Unity. The equivalent for Quick3DXR would be to default to Off and then have a property on the XrView for application control.
      • When enabled, pick a depth(stencil) swapchain format. Prefer the native equivalents of D24S8 or similar , followed by D32(F), and D16.
      • Create a depth XrSwapchain in addition to the color one.
      • Manage it similarly to the color swapchain, so that we have not just a color buffer (texture/texture array) to render into, but also a depth(stencil) texture (or texture array if multiview).
      • The goal is then to pass that in to Qt Quick (QQuickRenderTarget) somehow. Perhaps with new private APIs. Not sure if we need more public APIs in QQuickRenderTarget.
        • In QQuickRt support needs to be added, so that an externally provided QRhiTexture can be used as depth(stencil) in the QRhiTextureRenderTarget, if provided, in addition to the default case of automatically creating/managing a renderbuffer or texture not exposed to the higher layer.
      • The MSAA case is a problem.
        • Do we want to introduce a depthResolveTexture in QRhiColorAttachment? And so myriads of Qt Quick and QRhi changes follow.
      • Submit a XrCompositionLayerDepthInfoKHR in the XrCompositionLayerProjectionView chain.
        (min/maxDepth should be 0 and 1? Also needs near and far Z? Are there 3D API specifics, e.g. OpenGL versus others? unclear)

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            lagocs Laszlo Agocs
            lagocs Laszlo Agocs
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes