Uploaded image for project: 'Qt Quality Assurance Infrastructure'
  1. Qt Quality Assurance Infrastructure
  2. QTQAINFRA-4681

All tests are super flaky on Android



    • Bug
    • Resolution: Done
    • P1: Critical
    • None
    • master
    • Test scripts
    • Android
    • 0aaf514042 (qt/qtbase/dev) 8d1ec1449d (qt/qtbase/6.2) cdc60890c5 (qt/qtbase/6.3) c8e0a742fb (qt/qtbase/dev) 8d1ec1449d (qt/tqtc-qtbase/6.2) cdc60890c5 (qt/tqtc-qtbase/6.3) 0aaf514042 (qt/tqtc-qtbase/dev) bd3f787623 (qt/qtbase/6.2)


       Execute /opt/android/sdk/platform-tools/adb install -r -g /home/qt/work/qt/qtbase_standalone_tests/tests/auto/testlib/qabstractitemmodeltester/android-build/tst_qabstractitemmodeltester.apk.
       Performing Push Install
       /home/qt/work/qt/qtbase_standalone_tests/tests/auto/testlib/qabstractitemmodeltester/android-build/tst_qabstractitemmodeltester.apk: 1 file pushed, 0 skipped. 55.3 MB/s (11934463 bytes in 0.206s)
       Error: Could not access the Package Manager.  Is the system running?
       Execute /opt/android/sdk/platform-tools/adb shell am start -e applicationArguments \"'-o output.txt,txt'\" -n org.qtproject.example.tst_qabstractitemmodeltester/org.qtproject.qt.android.bindings.QtActivity.
       Error type 2
       android.util.AndroidException: Can't connect to activity manager; is the system running?
       	at com.android.commands.am.Am.onRun(Am.java:353)
       	at com.android.internal.os.BaseCommand.run(BaseCommand.java:47)
       	at com.android.commands.am.Am.main(Am.java:100)
       	at com.android.internal.os.RuntimeInit.nativeFinishInit(Native Method)
       	at com.android.internal.os.RuntimeInit.main(RuntimeInit.java:251)
       Execute /opt/android/sdk/platform-tools/adb uninstall org.qtproject.example.tst_qabstractitemmodeltester.
       CMake Error at tst_qabstractitemmodeltesterWrapperRelWithDebInfo.cmake:17 (message):
         /home/qt/work/install/bin/androidtestrunner --path
         --adb /opt/android/sdk/platform-tools/adb --skip-install-root --make
         /opt/cmake-3.21.1/bin/cmake --build
         /home/qt/work/qt/qtbase_standalone_tests --target
         tst_qabstractitemmodeltester_make_apk --apk
         --verbose execution failed.

      This error happens on every single qtbase integration on dev, on either of these 2 platforms on the testing phase:

      • Android_ANY (clang-x86) on Linux RHEL_8_2 (gcc-x86_64)
      • Android_ANY (clang-x86) on Linux RHEL_8_4 (gcc-x86_64)

      (BTW do we need both of these platforms? The minor revisions of RHEL are binary compatible AFAICT, if we only test the oldest, we are guaranteed to run on the newest.)

      The error happens, then CTest retries the test and the error does not happen. But it is there on every single log I have checked, which makes it a different story from a flaky test, since it's 100% reproducible.

      BTW, I found this while resolving QTBUG-96353, which results in removing the repeat flags --rerun-failed --force-new-ctest-process --repeat until-pass:5 from CTest. Then absolutely no status check of mine would pass.


        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.



              jimis Dimitrios Apostolou
              jimis Dimitrios Apostolou
              1 Vote for this issue
              6 Start watching this issue