Details
-
Bug
-
Resolution: Unresolved
-
P2: Important
-
None
-
6.9, 6.10
-
None
-
-
25
-
Android
Description
QtTest's search for the BLACKLIST file is done in qtestblacklist.cpp by calling QTest::qFindTestData(u"BLACKLIST"_s). (Note that this is a direct call, not using the QFINDTESTDATA() helper, which supplies CMake-derived defaults for the builddir and sourcedir parameters; but the direct call, by omitting them, gets nulls, skipping two candidates from qFindTestData()'s search. This is necessary as it'd get QtTest's values for those CMake-derived defaults, instead of those actually pertinent to the test being run.)
As for QTest::qFindTestData(), it's a cascade of heuristics (and some of its comments are Just Plain Wrong, e.g. 3. relative to test source. is in fact relative to builddir, the build directory, not the source directory; but that's irrelevant to our present case as we get null builddir). Filtering out the cases omitted due to null parameters and a relative path, this searches in:
- QCoreApplication::applicationDirPath(), where the comments claim the test binary is.
- In a sudbir, based on lower-casing the test-class name, of QLibraryInfo::path(QLibraryInfo::TestsPath), described by a comment as the installed path.
- (In the builddir, but that's null.)
- In resources (as ":/BLACKLIST"); this is how it's provided on Android.
- In QDir::currentPath(); it may be there, in an in-source build, if run from the source directory.
- In QTest::mainSourcePath, set via QTest::setMainSourcePath(), usually called via QTEST_SET_MAIN_SOURCE_PATH, which uses __FILE__ and expects to be exercised in main(), typically generated via QTEST_MAIN_WRAPPER() by a QTEST_*MAIN() variant (but some tests hand-code their own main() functions). This is where testlib normally finds it, at least in a developer build. (There's a problem with that for non-local test source, see below.)
- (In the sourcedir, but that's null.)
It's possible we should reconsider some of the choices there, but that's the status quo.
I'm not sure where in that chain a prefix-build would install it; I guess packaging, when it includes tests, includes the BLACKLIST file (if found) in one of the first two, but I don't know where it takes that file from to put it in the package.
In the case of Android, it emerges that CMake is supplying it as a resource; but the place it looks for the BLACKLIST file is the directory containing the CMakeLists.txt file that invoked qt_internal_add_test(). When a test needs a helper binary (as, for example, qtbase/tests/auto/widgets/kernel/qapplication/ does), we commonly put the source for the test in the test-dir, with each helper program in a sub-dir (with the CMakeLists.txt to build it) and one more sub-dir, e.g. qapplication/test/, to contain the CMakeLists.txt that actually builds the test binary. This means that Android gets (at choice 4) qapplication/test/BLACKLIST, for example, while developer builds get (either from choice 5, if in-source, or from choice 6) qapplication/BLACKLIST.
Life is further complicated by a few pairs of tests sharing common source files:
- qtbase/tests/auto/gui/kernel/qshortcut/tst_qshortcut.cpp shared with qtbase/tests/auto/widgets/kernel/qshortcut/
- qtbase/tests/auto/corelib/kernel/qapplicationstatic/tst_qapplicationstatic.cpp shared with qtbase/tests/auto/widgets/kernel/qapplication/ – which has a helper and uses a test/ sub-dir for the test itself; and has BLACKLIST files in both qapplication/ and qapplication/test/
- qtbase/tests/auto/corelib/kernel/qeventdispatcher/tst_qeventdispatcher.cpp shared with qtbase/tests/auto/gui/kernel/qguieventdispatcher/
- qtbase/tests/auto/corelib/kernel/qeventloop/tst_qeventloop.cpp shard with qtbase/tests/auto/gui/kernel/qguieventloop/
- qtbase/tests/auto/corelib/kernel/qtimer/tst_qtimer.cpp shared with qtbase/tests/auto/gui/kernel/qguitimer/
- qtbase/tests/auto/corelib/kernel/qchronotimer/tst_qchronotimer.cpp shared with qtbase/tests/auto/gui/kernel/qguichronotimer/
- qtbase/tests/auto/corelib/kernel/qcoreapplication/tst_qcoreapplication.{cpp,h} shared with qtbase/tests/auto/gui/kernel/qguiapplication/, which has a BLACKLIST file, and
- qtbase/tests/auto/gui/text/qfontengine/tst_qfontengine.cpp shared with its benchmark qtbase/tests/benchmarks/gui/text/qfontengine/.
For these, the source .cpp containing main() isn't necessarily where CMake should look for the BLACKLIST file.
This leads to some confusion and potential problems with different builds seeing different BLACKLIST files as applying to the same test; and may make it difficult for those adding blacklistings to find the right file to put them in or understand where to create the blacklist file.
Attachments
Issue Links
- relates to
-
QTBUG-87420 tst_QMdiArea fails on Android
-
- Reported
-