Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-120565

Bottlenecks when compositing widgets with RHI

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • P2: Important
    • None
    • 6.5, 6.6, 6.7, 6.8
    • Qt RHI
    • None
    • aa12713c1 (dev), f8786a867 (dev), a1371783a (dev), 690bb2e0c (dev), aa6d59dee (6.7), 191459da4 (6.7)

    Description

      I've found multiple bottlenecks when compositing widgets with RHI. I tested only on Wayland but looking at the code some of the problems seem to be cross-platform.

      I noticed that memcpy is done three times with QT_WIDGETS_RHI=1 on my system: QImage::detach inside QBackingStoreDefaultCompositor::toTexture, QImage::copy inside QRhiGles2::enqueueSubresUpload and the glTexSubImage2D itself.

      As a first step to fix it, I tried to do the following:

      diff --git a/src/client/qwaylandshmbackingstore.cpp b/src/client/qwaylandshmbackingstore.cpp
      index 02d830ad..01b1a569 100644
      --- a/src/client/qwaylandshmbackingstore.cpp
      +++ b/src/client/qwaylandshmbackingstore.cpp
      @@ -370,7 +370,12 @@ QImage QWaylandShmBackingStore::toImage() const
           // instead of flush() for widgets that have renderToTexture children
           // (QOpenGLWidget, QQuickWidget).
      
      -    return *contentSurface();
      +    QImage &image = *contentSurface();
      +
      +    // Return an image that does not share QImageData with the original image,
      +    // even if they both point to the same data, otherwise painting to
      +    // the image would detach it from the data.
      +    return QImage(image.bits(), image.width(), image.height(), image.bytesPerLine(), image.format());
       }
       #endif  // opengl
      
      
      

      That alone didn't help. Further looking at the code I noticed that QBackingStoreDefaultCompositor::toTexture accepts a const reference and then does a copy so detach will always happen whatever one will pass to toTexture.

      I did the following:

      diff --git a/src/gui/painting/qbackingstoredefaultcompositor.cpp b/src/gui/painting/qbackingstoredefaultcompositor.cpp
      index 13a8bdb157..af5d88fe66 100644
      --- a/src/gui/painting/qbackingstoredefaultcompositor.cpp
      +++ b/src/gui/painting/qbackingstoredefaultcompositor.cpp
      @@ -46,7 +46,7 @@ QRhiTexture *QBackingStoreDefaultCompositor::toTexture(const QPlatformBackingSto
           return toTexture(backingStore->toImage(), rhi, resourceUpdates, dirtyRegion, flags);
       }
      
      -QRhiTexture *QBackingStoreDefaultCompositor::toTexture(const QImage &sourceImage,
      +QRhiTexture *QBackingStoreDefaultCompositor::toTexture(QImage image,
                                                              QRhi *rhi,
                                                              QRhiResourceUpdateBatch *resourceUpdates,
                                                              const QRegion &dirtyRegion,
      @@ -63,8 +63,6 @@ QRhiTexture *QBackingStoreDefaultCompositor::toTexture(const QImage &sourceImage
               return nullptr;
           }
      
      -    QImage image = sourceImage;
      -
           bool needsConversion = false;
           *flags = {};
      
      @@ -515,7 +513,7 @@ QPlatformBackingStore::FlushResult QBackingStoreDefaultCompositor::flush(QPlatfo
                   const QImage::Format format = QImage::toImageFormat(graphicsBuffer->format());
                   const QSize size = graphicsBuffer->size();
                   QImage wrapperImage(graphicsBuffer->data(), size.width(), size.height(), graphicsBuffer->bytesPerLine(), format);
      -            toTexture(wrapperImage, rhi, resourceUpdates, scaledRegion(region, sourceDevicePixelRatio, offset), &flags);
      +            toTexture(std::move(wrapperImage), rhi, resourceUpdates, scaledRegion(region, sourceDevicePixelRatio, offset), &flags);
                   gotTextureFromGraphicsBuffer = true;
                   graphicsBuffer->unlock();
                   if (graphicsBuffer->origin() == QPlatformGraphicsBuffer::OriginBottomLeft)
      diff --git a/src/gui/painting/qbackingstoredefaultcompositor_p.h b/src/gui/painting/qbackingstoredefaultcompositor_p.h
      index 2f835e5d35..1ced8e4cdc 100644
      --- a/src/gui/painting/qbackingstoredefaultcompositor_p.h
      +++ b/src/gui/painting/qbackingstoredefaultcompositor_p.h
      @@ -55,7 +55,7 @@ private:
           Q_DECLARE_FLAGS(UpdateQuadDataOptions, UpdateQuadDataOption)
      
           void ensureResources(QRhiSwapChain *swapchain, QRhiResourceUpdateBatch *resourceUpdates);
      -    QRhiTexture *toTexture(const QImage &image,
      +    QRhiTexture *toTexture(QImage image,
                                  QRhi *rhi,
                                  QRhiResourceUpdateBatch *resourceUpdates,
                                  const QRegion &dirtyRegion,
      

      This has helped! The number of memcpy calls has decreased down to two.

      During inspecting QPAs, I noticed this xcb commit: https://github.com/qt/qtbase/commit/db780c4ca529a0d8d6896a08e0dc59c33b703adc

      But it doesn't seem to do what it's meant to cause it will detach anyway due to constBits(). It should likely be fixed like that:

      diff --git a/src/plugins/platforms/xcb/qxcbbackingstore.cpp b/src/plugins/platforms/xcb/qxcbbackingstore.cpp
      index 0659cf9fc4..9e4442af45 100644
      --- a/src/plugins/platforms/xcb/qxcbbackingstore.cpp
      +++ b/src/plugins/platforms/xcb/qxcbbackingstore.cpp
      @@ -815,12 +815,12 @@ QImage QXcbBackingStore::toImage() const
      
           m_image->flushScrolledRegion(true);
      
      -    QImage image = *m_image->image();
      +    QImage &image = *m_image->image();
      
           // Return an image that does not share QImageData with the original image,
           // even if they both point to the same data of the m_xcb_image, otherwise
           // painting to m_qimage would detach it from the m_xcb_image data.
      -    return QImage(image.constBits(), image.width(), image.height(), image.format());
      +    return QImage(image.bits(), image.width(), image.height(), image.format());
       }
      
       QPlatformGraphicsBuffer *QXcbBackingStore::graphicsBuffer() const
      

      But I guess it's not noticeable because QXcbBackingStore implements QPlatformBackingStore::graphicsBuffer so toImage() is not called when uploading the backing store.

      Next, I tried to copy some code from the pre-RHI times:

      diff --git a/src/gui/rhi/qrhigles2.cpp b/src/gui/rhi/qrhigles2.cpp
      index 02138d7fb7..6d89b9d228 100644
      --- a/src/gui/rhi/qrhigles2.cpp
      +++ b/src/gui/rhi/qrhigles2.cpp
      @@ -2224,13 +2224,21 @@ void QRhiGles2::enqueueSubresUpload(QGles2Texture *texD, QGles2CommandBuffer *cb
           if (!subresDesc.image().isNull()) {
               QImage img = subresDesc.image();
               QSize size = img.size();
      +        static const int bytesPerPixel = 4;
      +        const int strideInPixels = img.bytesPerLine() / bytesPerPixel;
      +        const bool hasUnpackRowLength = !ctx->isOpenGLES() || ctx->format().majorVersion() >= 3;
               QGles2CommandBuffer::Command &cmd(cbD->commands.get());
               cmd.cmd = QGles2CommandBuffer::Command::SubImage;
               if (!subresDesc.sourceSize().isEmpty() || !subresDesc.sourceTopLeft().isNull()) {
                   const QPoint sp = subresDesc.sourceTopLeft();
                   if (!subresDesc.sourceSize().isEmpty())
                       size = subresDesc.sourceSize();
      -            img = img.copy(sp.x(), sp.y(), size.width(), size.height());
      +            if (hasUnpackRowLength || (strideInPixels == img.width() && size.width() == img.width())) {
      +                const uchar *data = img.constBits() + ((sp.x() * img.depth()) >> 3) + sp.y() * img.bytesPerLine();
      +                img = QImage(data, size.width(), size.height(), img.bytesPerLine(), img.format());
      +            } else {
      +                img = img.copy(sp.x(), sp.y(), size.width(), size.height());
      +            }
               }
               cmd.args.subImage.target = texD->target;
               cmd.args.subImage.texture = texD->texture;
      @@ -2244,7 +2252,7 @@ void QRhiGles2::enqueueSubresUpload(QGles2Texture *texD, QGles2CommandBuffer *cb
               cmd.args.subImage.glformat = texD->glformat;
               cmd.args.subImage.gltype = texD->gltype;
               cmd.args.subImage.rowStartAlign = 4;
      -        cmd.args.subImage.rowLength = 0;
      +        cmd.args.subImage.rowLength = hasUnpackRowLength ? strideInPixels : 0;
               cmd.args.subImage.data = cbD->retainImage(img);
           } else if (!rawData.isEmpty() && isCompressed) {
               const int depth = qMax(1, texD->m_depth);
      

      Voila, I have only one memcpy from the GL driver!

      But I don't understand, what's the point of image.detach() inside QBackingStoreDefaultCompositor::toTexture? It says

      if it was just wrapping data, that's no good, we need ownership, so detach

      but detach() doesn't change the ownership of data unless you initialize QImage with constBits() which seems to be an unlikely case (and the existing usage in xcb is wanting the other way around behavior). If QImage is initialized with non-const bits(), detach() won't get the ownership and if QImage is owning the data then it won't be destroyed unless last reference is destroyed anyway, detach() doesn't make an improvement here, it only makes things worse given that most backing stores don't seem to be adapted (only xcb is attempted to be adapted but cocoa is likely to avoid the problem after fixing the argument part as it has graphics buffer implemented) to this and get memcpy on detach() without any profit.

      Attachments

        For Gerrit Dashboard: QTBUG-120565
        # Subject Branch Project Status CR V

        Activity

          People

            lagocs Laszlo Agocs
            ilya-fedin Ilya Fedin
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: