Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-87272

Vulkan QSGBatchRenderer::Renderer crash on pipelineCache access (with fix)

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Not Evaluated
    • Resolution: Done
    • Affects Version/s: 5.15.1
    • Fix Version/s: 6.0.0
    • Labels:
      None
    • Environment:
      Win10 x64
    • Platform/s:
      Windows

      Description

      The crash is easily reproducible once you hit the right circumstances. You need to be running Qt 5.15.1 (or 5.15.0) QML with VulkanRhi backend. The issue is not reproducible with OpenGL or DirectX backends.

      Symptoms:

      The general situation is this: construct multiple (30+) objects in QML, these objects must use some kind of effect, like OpacityMask. Then destroy the objects, and create them again. Repeat a few times, and the application will crash.

      The crash always produces the same callstack:

      Qt5Guid.dll!QVarLengthArray<VkAttachmentDescription,8>::count() Line 120
      Qt5Guid.dll!QVkRenderPassDescriptor::isCompatible(const QRhiRenderPassDescriptor * other) Line 5742
      Qt5Quickd.dll!QSGBatchRenderer::operator==(const QSGBatchRenderer::GraphicsPipelineStateKey & a, const QSGBatchRenderer::GraphicsPipelineStateKey & b) Line 4693
      Qt5Quickd.dll!QHashNode<QSGBatchRenderer::GraphicsPipelineStateKey,QRhiGraphicsPipeline *>::same_key(unsigned int h0, const QSGBatchRenderer::GraphicsPipelineStateKey & key0) Line 156
      Qt5Quickd.dll!QHash<QSGBatchRenderer::GraphicsPipelineStateKey,QRhiGraphicsPipeline *>::findNode(const QSGBatchRenderer::GraphicsPipelineStateKey & akey, unsigned int h) Line 931
      Qt5Quickd.dll!QHash<QSGBatchRenderer::GraphicsPipelineStateKey,QRhiGraphicsPipeline *>::findNode(const QSGBatchRenderer::GraphicsPipelineStateKey & akey, unsigned int * ahp) Line 951
      Qt5Quickd.dll!QHash<QSGBatchRenderer::GraphicsPipelineStateKey,QRhiGraphicsPipeline *>::constFind(const QSGBatchRenderer::GraphicsPipelineStateKey & akey) Line 907
      Qt5Quickd.dll!QSGBatchRenderer::Renderer::ensurePipelineState(QSGBatchRenderer::Element * e, const QSGBatchRenderer::ShaderManagerShader * sms) Line 3291
      Qt5Quickd.dll!QSGBatchRenderer::Renderer::prepareRenderMergedBatch(QSGBatchRenderer::Batch * batch, QSGBatchRenderer::Renderer::PreparedRenderBatch * renderBatch) Line 3696
      Qt5Quickd.dll!QSGBatchRenderer::Renderer::renderBatches() Line 4139
      Qt5Quickd.dll!QSGBatchRenderer::Renderer::render() Line 4365
      Qt5Quickd.dll!QSGRenderer::renderScene(const QSGBindable & bindable) Line 265
      Qt5Quickd.dll!QSGRenderer::renderScene(unsigned int fboId) Line 201
      Qt5Quickd.dll!QSGDefaultRenderContext::renderNextRhiFrame(QSGRenderer * renderer) Line 257
      Qt5Quickd.dll!QSGRhiLayer::grab() Line 416
      Qt5Quickd.dll!QSGRhiLayer::updateTexture() Line 122
      Qt5Quickd.dll!QSGRhiShaderEffectNode::preprocess() Line 772
      Qt5Quickd.dll!QSGRenderer::preprocess() Line 339
      Qt5Quickd.dll!QSGRenderer::renderScene(const QSGBindable & bindable) Line 241
      Qt5Quickd.dll!QSGRenderer::renderScene(unsigned int fboId) Line 201
      Qt5Quickd.dll!QSGDefaultRenderContext::renderNextRhiFrame(QSGRenderer * renderer) Line 257
      Qt5Quickd.dll!QQuickWindowPrivate::renderSceneGraph(const QSize & size, const QSize & surfaceSize) Line 615
      Qt5Quickd.dll!QSGRenderThread::syncAndRender(QImage * grabImage) Line 839
      Qt5Quickd.dll!QSGRenderThread::run() Line 1028
      Qt5Cored.dll!QThreadPrivate::start(void * arg) Line 405
      kernel32.dll!00007ffd41ba7bd4()
      ntdll.dll!00007ffd42dcce51()
      

      At frame 2, `const QRhiRenderPassDescriptor * other` pointer is dangling - the address is fine, but the content is invalid.

      Analysis:

      The problem is in \qtdeclarative\src\quick\scenegraph\qsgrhilayer.cpp, and \qtdeclarative\src\quick\scenegraph\coreapi\qsgbatchrenderer.cpp. The following sequence of events causes the crash:

      1. QSGRhiLayer::grab() method is called
        1. Creates newCompatibleRenderPassDescriptor m_rtRp
        2. Creates QSGBatchRenderer::Renderer
        3. Calls renderer setRenderPassDescriptor with m_rtRp
      2. In QSGBatchRenderer::Renderer constructor
        1. Creates ShaderManager object m_shaderManager
        2. Assigns m_shaderManager as a child to QSGRhiLayer::m_context with name "__qt_ShaderManager"
      3. QSGBatchRenderer::Renderer::ensurePipelineState() method is called
        1. Constructs a key using the current render pass descriptor (set with m_rtRp)
        2. Looks up in m_shaderManager->pipelineCache by this key
        3. If not found, creates QRhiGraphicsPipeline object and saves in the hash using m_rtRp pointer as a part of the key
      4. All good and well, until we start removing objects
      5. QSGRhiLayer::releaseResources() method is called
        1. Deletes m_rtRp
      6. QSGRhiLayer::grab() method is called
        1. Creates new or reuses objects
        2. Calls renderer setRenderPassDescriptor with a new m_rtRp
      7. QSGBatchRenderer::Renderer::ensurePipelineState() method is called
        1. Constructs a key using the current render pass descriptor (set with m_rtRp)
        2. Looks up in m_shaderManager->pipelineCache by this key
        3. Old entry with old m_rtRp is still in the cache, at some point tries to call a.compatibleRenderPassDescriptor->isCompatible(b.compatibleRenderPassDescriptor)
      8. Crash

      Solution:

      Perhaps it is more of a workaround rather than proper solution, but nevertheless.
      Modify qtdeclarative\src\quick\scenegraph\qsgrhilayer.cpp:
      Add include:

      #include "coreapi/qsgbatchrenderer_p.h"
      

      Modify QSGRhiLayer::releaseResources:

      void QSGRhiLayer::releaseResources()
      {
          if (auto shaderManager = m_context->findChild<QSGBatchRenderer::ShaderManager *>(QStringLiteral("__qt_ShaderManager"), Qt::FindDirectChildrenOnly)) {
              for (auto it = shaderManager->pipelineCache.begin(); it != shaderManager->pipelineCache.end();) {
                  if (it.key().compatibleRenderPassDescriptor == m_rtRp) {
                      delete it.value();
                      it = shaderManager->pipelineCache.erase(it);
                  } else {
                      ++it;
                  }
              }
          }
      ...
      

      releaseResources() deletes m_rtRp used previously in shaderManager->pipelineCache, resulting in a dangling pointer. So here before we delete m_rtRp, clean up the cache.

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

            Assignee:
            lagocs Laszlo Agocs
            Reporter:
            dzmitry.mazouka.1 Dzmitry Mazouka
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Gerrit Reviews

                There are no open Gerrit changes