Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-104133

QML animation overlaid on video becomes unusably slow & with increasing memory consumption over time

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • P1: Critical
    • 6.2.6, 6.4.0 RC1, 6.5.0 Beta1
    • 6.3.0, 6.3.1, 6.4.0 Beta1
    • Multimedia
    • None
    • LG G8X smartphone running Android 11 on arm64-v8a architecture. Android Emulator on Linux desktop running Android 12 on x86_64 architecture.
    • Android
    • e8248b7a95 (qt/qtmultimedia/dev) dc92e5b592 (qt/qtmultimedia/6.4) dc92e5b592 (qt/tqtc-qtmultimedia/6.4) e8248b7a95 (qt/tqtc-qtmultimedia/dev) bd57cdf241 (qt/qtmultimedia/6.3) bd57cdf241 (qt/tqtc-qtmultimedia/6.3) faa100d592 (qt/tqtc-qtmultimedia/6.2)
    • 2022wk26FOQtforAndroid, 2022wk28FOQtforAndroid

    Description

      Although I'm seeing this in an unreleased app, I was able to duplicate and document the significant frame rate drop with an existing example application org.qtproject.example.qmlvideo found in Examples/Qt-6.4.0/multimedia/video/qmlvideo.

      Although the lack of Qt6 android permissions support prevents loading video files with this app, the "camera" part of this app does work on android to display video in a QML item with various overlays and animations occuring on top of the video. Without the video in the background, and just animations, the slowdown does not occur. Therefore this is an issue with Multimedia, and in particular with video displayed in a QML application that is overlaid by other graphics and animated.

      The particular test in 'qmlvideo' to demonstrate the problem is under the "Camera modes" column – activated by clicking the "Overlay" button, then watching the FPS count meter slowly decrease over time, alongside very laggy and jittery animations and overdrawing on top of the video (which doesn't show lagging or obvious performance problems like the overdrawn animated rotating rectangle over the video).

      This bug shows is a sigificant level of slowdown that is accompanied by a growth in size of the app as seen by the following successive lines of top(1) output. Note growth of the "RES" (resident set) column over time, alongside the level of FPS slowdown (see attached screenshots showing the clock and the FPS rate at the time to get an idea for how long it takes to manifest this problem).

      The slowdown problem manifests over minutes; for example the following would happen over the course of a half hour's video playback:

      • starts out at 60 FPS
        • 15G 156M  97M S 31.3   2.8   0:01.91 org.qtproject.example.qmlvideo
      • after running a few minutes but still at 60 FPS
        • 15G 185M 104M S  109   3.3   1:40.03 org.qtproject.example.qmlvideo
      • 48 FPS
        • 15G 187M 101M S  150   3.3   4:05.01 org.qtproject.example.qmlvideo
      • 36 FPS
        • 15G 188M  99M S  152   3.4   5:57.45 org.qtproject.example.qmlvideo
      • 28 FPS
        • 15G 189M  99M S  149   3.4   7:59.50 org.qtproject.example.qmlvideo
      • 16 FPS
        • 15G 195M  99M S  159   3.5  16:21.25 org.qtproject.example.qmlvideo
      • 9FPS
        • 15G 203M  98M S  155   3.6  35:51.84 org.qtproject.example.qmlvideo

      As evidenced by the above, running this app for five or ten minutes sees a framerate slowdown from 60FPS to under 10FPS, while the application resident set size grows from 156M to 203M and continues growing.

      Although the size of the app doesn't decrease, the FPS slowdown will go away by hitting "Back" and then re-clicking the "Camera Modes" -> "Overlay" button again. However, over time it will slowdown again and continue to grow in size.

      I'm seeing this issue both on Android 11 arm64-v8a with an LG G8X mobile phone, as well as for Android 12 x86_64 android emulator run from QtCreator.

      In my opinion, the last version of Qt6 that didn't suffer from this video animation overlay slowdown issue was 6.2.3 (6.2.4 having a bug that prevents video playback and crashes on android). It is reproducible for Qt 6.3.0 as well as the 6.4.0-0-202206020504 "dev" release.

      Attachments

        Issue Links

          For Gerrit Dashboard: QTBUG-104133
          # Subject Branch Project Status CR V

          Activity

            People

              samuelmira Samuel Mira
              nielsmayer Niels Mayer
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews