Details
-
Bug
-
Resolution: Unresolved
-
P2: Important
-
None
-
5.12
-
None
Description
I'd like to run tests from a shadow-build directory, mounted to a VM, so that I can isolate test running from my host machine, as well as run tests on many different macOS versions from the same host build.
This doesn't work out of the box, as a lot of tests assume access to the source dir to pick up the test data.
Some tests unconditionally include their test data as resources, these tests work fine.
Some tests add the testdata to TESTDATA, but this will by default result in a source dir lookup, which is not what we want.
There's a feature flag for making TESTDATA result in being embedded as qrc resources, and extracted on the target machine, namely builtin_testdata.
But enabling this feature also means you can't have any TEST_HELPER_INSTALLS:
builtin_testdata { ALL_TESTDATA = $$TESTDATA $$GENERATED_TESTDATA # BLACKLIST needs to be added to the testdata BLACKLISTPATH = $$_PRO_FILE_PWD_/BLACKLIST exists($$BLACKLISTPATH): ALL_TESTDATA *= $$BLACKLISTPATH # RESOURCES does not support wildcards (for good reasons) for(td, ALL_TESTDATA): \ testdata.files += $$files($$absolute_path($$td, $$_PRO_FILE_PWD_)) !isEmpty(testdata.files) { testdata.base = $$_PRO_FILE_PWD_ RESOURCES += testdata } !isEmpty(TEST_HELPER_INSTALLS): \ error("This platform does not support tests which require helpers.") }
This coupling does not make sense for the mounted build dir usecase.
We should fix builtin_testdata to not couple to TEST_HELPER_INSTALLS, or add a new mechanism for this usecase.
Adding a new mechanism is of course adding to the mess of how we deploy data and dependencies with our applications, so fixing builtin_testdata seems preferable.
Attachments
Issue Links
- relates to
-
QTBUG-117098 qt_internal_add_test's non builtin_testdata mode can't be relied upon for QFINDTESTDATA
- Reported