Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-95756

tst_QQmlImport::removeDynamicPlugin() is flaky on macOS

    XMLWordPrintable

Details

    Description

      I ran a pre-check on https://codereview.qt-project.org/c/qt/qtdeclarative/+/363532/17#message-21078822ec5972ec100aaa32d292fe4904f2930d and got a failure:

      agent:2021/08/11 17:52:17 build.go:391: 92: FAIL!  : tst_QQmlImport::removeDynamicPlugin() '!plugins.isEmpty()' returned FALSE. ()
      agent:2021/08/11 17:52:17 build.go:391: 92:    Loc: [/Users/qt/work/qt/qtdeclarative/tests/auto/qml/qqmlimport/tst_qqmlimport.cpp(374)]
      

      I built Qt on macOS without the patch applied (qtdeclarative at 32b27f24a94a5b1a3d125b5b9f97109c4ca9c5c6 and dependent modules synced according to dependencies.yaml) and got a different failure:

      Mitchs-MacBook-Pro:qtdeclarative mitch$ ninja tst_qqmlimport_check
      [8/8] Running /usr/local/Cellar/cmake/3.21.1/bin/ctest -V -R "^tst_qqmlimport$"
      FAILED: tests/auto/qml/qqmlimport/CMakeFiles/tst_qqmlimport_check /Users/mitch/dev/qt-dev2-debug-non-fw/qtdeclarative/tests/auto/qml/qqmlimport/CMakeFiles/tst_qqmlimport_check 
      cd /Users/mitch/dev/qt-dev2-debug-non-fw/qtdeclarative/tests/auto/qml/qqmlimport && /usr/local/Cellar/cmake/3.21.1/bin/ctest -V -R "^tst_qqmlimport\$"
      UpdateCTestConfiguration  from :/Users/mitch/dev/qt-dev2-debug-non-fw/qtdeclarative/tests/auto/qml/qqmlimport/DartConfiguration.tcl
      UpdateCTestConfiguration  from :/Users/mitch/dev/qt-dev2-debug-non-fw/qtdeclarative/tests/auto/qml/qqmlimport/DartConfiguration.tcl
      Test project /Users/mitch/dev/qt-dev2-debug-non-fw/qtdeclarative/tests/auto/qml/qqmlimport
      Constructing a list of tests
      Done constructing a list of tests
      Updating test list for fixtures
      Added 0 tests to meet fixture requirements
      Checking test dependency graph...
      Checking test dependency graph end
      test 1
          Start 1: tst_qqmlimport
      
      1: Test command: /usr/local/Cellar/cmake/3.21.1/bin/cmake "-P" "/Users/mitch/dev/qt-dev2-debug-non-fw/qtdeclarative/tests/auto/qml/qqmlimport/tst_qqmlimportWrapperDebug.cmake"
      1: Test timeout computed to be: 10000000
      1: ********* Start testing of tst_QQmlImport *********
      1: Config: Using QtTest library 6.2.0, Qt 6.2.0 (x86_64-little_endian-lp64 shared (dynamic) debug build; by Clang 12.0.5 (clang-1205.0.22.11) (Apple)), macos 11.4
      1: PASS   : tst_QQmlImport::initTestCase()
      1: PASS   : tst_QQmlImport::importPathOrder()
      1: PASS   : tst_QQmlImport::testDesignerSupported()
      1: PASS   : tst_QQmlImport::uiFormatLoading()
      1: PASS   : tst_QQmlImport::completeQmldirPaths(QtQml)
      1: PASS   : tst_QQmlImport::completeQmldirPaths(QtQml.Models)
      1: PASS   : tst_QQmlImport::completeQmldirPaths(org.qt-project.foo.bar 0.1)
      1: PASS   : tst_QQmlImport::completeQmldirPaths(org.qt-project.foo.bar 4)
      1: PASS   : tst_QQmlImport::completeQmldirPaths(org.qt-project.foo.bar)
      1: PASS   : tst_QQmlImport::interceptQmldir()
      1: PASS   : tst_QQmlImport::singletonVersionResolution()
      1: FAIL!  : tst_QQmlImport::removeDynamicPlugin() 'component.isReady()' returned FALSE. ()
      1:    Loc: [/Users/mitch/dev/qt-dev2/qtdeclarative/tests/auto/qml/qqmlimport/tst_qqmlimport.cpp(362)]
      1: PASS   : tst_QQmlImport::partialImportVersions(empty)
      1: PASS   : tst_QQmlImport::partialImportVersions(2)
      1: PASS   : tst_QQmlImport::partialImportVersions(6)
      1: PASS   : tst_QQmlImport::partialImportVersions(2.0)
      1: PASS   : tst_QQmlImport::partialImportVersions(2.3)
      1: PASS   : tst_QQmlImport::partialImportVersions(2.15)
      1: PASS   : tst_QQmlImport::partialImportVersions(6.0)
      1: PASS   : tst_QQmlImport::registerModuleImport()
      1: PASS   : tst_QQmlImport::importDependenciesPrecedence()
      1: PASS   : tst_QQmlImport::envResourceImportPath()
      1: PASS   : tst_QQmlImport::preferResourcePath()
      1: PASS   : tst_QQmlImport::cleanupTestCase()
      1: Totals: 23 passed, 1 failed, 0 skipped, 0 blacklisted, 2501ms
      1: ********* Finished testing of tst_QQmlImport *********
      1: CMake Error at tst_qqmlimportWrapperDebug.cmake:17 (message):
      1:   
      1:   /Users/mitch/dev/qt-dev2-debug-non-fw/qtdeclarative/tests/auto/qml/qqmlimport/tst_qqmlimport
      1:   execution failed.
      1: 
      1: 
      1/1 Test #1: tst_qqmlimport ...................***Failed    5.38 sec
      
      0% tests passed, 1 tests failed out of 1
      
      Label Time Summary:
      tests/auto/qml/qqmlimport/tst_qqmlimport    =   5.38 sec*proc (1 test)
      
      Total Test time (real) =   5.38 sec
      
      The following tests FAILED:
      	  1 - tst_qqmlimport (Failed)
      Errors while running CTest
      Output from these tests are in: /Users/mitch/dev/qt-dev2-debug-non-fw/qtdeclarative/tests/auto/qml/qqmlimport/Testing/Temporary/LastTest.log
      Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
      ninja: build stopped: subcommand failed.
      

      I then tried improving the failure message:

      @@ -359,7 +359,7 @@ void tst_QQmlImport::removeDynamicPlugin()
               // Load something that adds a dynamic plugin
               QQmlComponent component(&engine);
               component.setData(QByteArray("import QtTest 1.0; TestResult{}"), QUrl());
      -        QVERIFY(component.isReady());
      +        QVERIFY2(component.isReady(), qPrintable(component.errorString()));
           }
           QQmlImportDatabase *imports = &QQmlEnginePrivate::get(&engine)->importDatabase;
           const QStringList &plugins = imports->dynamicPlugins();
      

      but then the test started passing.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              qtqmlteam Qt Qml Team User
              mitch_curtis Mitch Curtis
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes