Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-131918

Refactor usage of rhi in QVideoFrames

    XMLWordPrintable

Details

    • Task
    • Resolution: Fixed
    • P2: Important
    • None
    • None
    • Multimedia
    • None
    • All
    • 6222a36d7 (dev), 4dafa2439 (dev), bfad11c91 (dev), 6427a741d (dev), 5261e6c1a (6.9), 43d051d61 (6.9), 5b27693c0 (6.9), a91775f78 (6.9), 057dbe448 (dev), 37c2ba0df (dev), db955418a (dev), 8e5ff7053 (dev), 3b0763936 (dev), d5bec721c (dev), 559bd6326 (dev)
    • Multimedia wk 49-50, Multimedia wk 51-2

    Description

      A few bottlenecks exist in using rhi in video frames during hardware texture conversion.

      Let's go through the existing problems.

      1. QVideoFrame references to QRhi, but it doesn't track the lifetime of QRhi, and the frame contains a broken pointer after removing the rhi.
      2. QVideoFrame contains converted/mapped textures to be rendered in QRhi. The conversion is based on QRhi and the source backend (FFmpeg frames context).
      3. QVideoFrame can be deleted in any thread by a user or QtMM machinery, and QRhi or any of its belongings can be used upon deleting of QVideoFrame. Some textures are required to be deleted in QRhi's thread. The ticket https://bugreports.qt.io/browse/QTBUG-129994 describes a related crash on VAAPI+opengl.
      4. As long as QVideoFrame contains a texture created for the specified QRhi, the same texture will be taken for rendering in another rhi, which can cause UB, in theory.

      In other words, QVideoFrame holds the converted texture, but the texture also belongs to QRhi; QVideoFrame and QVideoSink hold shared QTextureConverter, but QTextureConverter references to QRhi and FFmpeg frames context.

      The picture briefly explains processes, dependencies, and user actions that can lead to UB.

      The proposal is to implement a rhi objects mapper, that will remove all rhi belongings when the rhi is cleaned up, and allow the holder (e.g. QVideoFrame) to associate many textures to itself and different rhi, and remove all of them at the frame's destruction.

      The picture below describes the design of the proposal:

      Thus, we ensure that all textures, converters, etc, use a valid rhi, and that we render the texture for the rhi it has been created for.

      Attachments

        1. screenshot-2.png
          screenshot-2.png
          343 kB
        2. screenshot-1.png
          screenshot-1.png
          362 kB
        3. Refactoring rhi in QtMM.pdf
          561 kB

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              artemiy Artem Dyomin
              artemiy Artem Dyomin
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes