Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-67803

builtin_testdata alternative that doesn't imply lack of TEST_HELPER_INSTALLS

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • P2: Important
    • None
    • 5.12
    • None

    Description

      I'd like to run tests from a shadow-build directory, mounted to a VM, so that I can isolate test running from my host machine, as well as run tests on many different macOS versions from the same host build.

      This doesn't work out of the box, as a lot of tests assume access to the source dir to pick up the test data.

      Some tests unconditionally include their test data as resources, these tests work fine. 

      Some tests add the testdata to TESTDATA, but this will by default result in a source dir lookup, which is not what we want.

      There's a feature flag for making TESTDATA result in being embedded as qrc resources, and extracted on the target machine, namely builtin_testdata. 

      But enabling this feature also means you can't have any TEST_HELPER_INSTALLS:

       

      builtin_testdata {
       ALL_TESTDATA = $$TESTDATA $$GENERATED_TESTDATA
      # BLACKLIST needs to be added to the testdata
       BLACKLISTPATH = $$_PRO_FILE_PWD_/BLACKLIST
       exists($$BLACKLISTPATH): ALL_TESTDATA *= $$BLACKLISTPATH
      # RESOURCES does not support wildcards (for good reasons)
       for(td, ALL_TESTDATA): \
       testdata.files += $$files($$absolute_path($$td, $$_PRO_FILE_PWD_))
       !isEmpty(testdata.files) {
       testdata.base = $$_PRO_FILE_PWD_
       RESOURCES += testdata
       }
      !isEmpty(TEST_HELPER_INSTALLS): \
       error("This platform does not support tests which require helpers.")
      }
      

       

      This coupling does not make sense for the mounted build dir usecase.

      We should fix builtin_testdata to not couple to TEST_HELPER_INSTALLS, or add a new mechanism for this usecase.

      Adding a new mechanism is of course adding to the mess of how we deploy data and dependencies with our applications, so fixing builtin_testdata seems preferable.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              qtbuildsystem Qt Build System Team
              vestbo Tor Arne Vestbø
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes