Details
-
Bug
-
Resolution: Unresolved
-
P3: Somewhat important
-
dev
-
None
Description
Can't tell apart tests running under qmltestrunner.
Example copied from this log but this is not the only one:
agent:2023/08/27 23:26:26 build.go:405: 107: PASS : qmltestrunner::A::test_do() agent:2023/08/27 23:26:26 build.go:405: 107: PASS : qmltestrunner::A::cleanupTestCase() agent:2023/08/27 23:26:26 build.go:405: 107: PASS : qmltestrunner::A::initTestCase() agent:2023/08/27 23:26:26 build.go:405: 107: PASS : qmltestrunner::A::test_do() agent:2023/08/27 23:26:26 build.go:405: 107: PASS : qmltestrunner::A::cleanupTestCase() agent:2023/08/27 23:26:26 build.go:405: 107: PASS : qmltestrunner::A::initTestCase() agent:2023/08/27 23:26:26 build.go:405: 107: PASS : qmltestrunner::A::test_do() agent:2023/08/27 23:26:26 build.go:405: 107: PASS : qmltestrunner::A::cleanupTestCase() [ ... seemingly same test runs 100 times ... ]
Here is part of the XML file generated by qtestlib:
<?xml version="1.0" encoding="UTF-8"?> <TestCase name="qmltestrunner"> <Environment> <QtVersion>6.7.0</QtVersion> <QtBuild>Qt 6.7.0 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 11.4.0)</QtBuild> <QTestVersion>6.7.0</QTestVersion> </Environment> [...] <TestFunction name="A::initTestCase"> <Incident type="pass" file="" line="0" /> <Duration msecs="0.16281"/> </TestFunction> <TestFunction name="A::test_do"> <Incident type="pass" file="" line="0" /> <Duration msecs="1.27274"/> </TestFunction> <TestFunction name="A::cleanupTestCase"> <Incident type="pass" file="" line="0" /> <Duration msecs="0.060903"/> </TestFunction> <TestFunction name="A::initTestCase"> <Incident type="pass" file="" line="0" /> <Duration msecs="0.1256"/> </TestFunction> <TestFunction name="A::test_do"> <Incident type="pass" file="" line="0" /> <Duration msecs="1.65724"/> </TestFunction> <TestFunction name="A::cleanupTestCase"> <Incident type="pass" file="" line="0" /> <Duration msecs="0.053843"/> </TestFunction> <TestFunction name="A::initTestCase"> <Incident type="pass" file="" line="0" /> <Duration msecs="0.064591"/> </TestFunction> [...] <Duration msecs="3253.55"/> </TestCase>
- The executable is always qmltestrunner without any parameters. AFAICT all the QML files from the CWD (current working directory) are executed. For the example snippets I attach here, the CWD is qtdeclarative/tests/auto/qmltest/selftests but the file executed is tst_multiTestCase.qml.
- It would be better to explicitly pass only one QML file as argument to qmltestrunner, and execute that one.
- The testcase is always qmltestrunner (in common tests this is tst_whatever) when qmltestrunner is involved. I believe this is a bug. In the snippet example here, what is printed in the log is qmltestrunner::A::test_do(). Either "tst_multiTestCase.qml" from the filename, or this "A" from the testfunction should be in the testCase.
- Many A::test_do "functions" are being called, and I can't tell them apart. Is it the same test executing again and again? IMO qtestlib should not allow this, but I'll file a separate bug for it. (EDIT: filed QTBUG-116772)
Attaching the file tst_multiTestCase.qml as an example, but I believe the problem is more generic.
Attachments
Issue Links
- relates to
-
-
- Reported
-