Description
qt3d/tests/auto/qml3d/qml3d.pro is marked as an insignificant_test.
Examining the test results from the CI system, it appears that there are two problems with the test:
1. On platforms other than Windows, the test fails in a stable fashion:
Testing qml3d
FAIL! : qml3d::Picking Test::test_picking() 'timed out' returned FALSE. ()
Loc: [/home/qt/.pulse2-agent/data/recipes/179082385/base/qt/qt3d/tests/auto/qml3d/picking/tst_picking.qml(177)]
FAIL! : qml3d::Sphere::test_defaults() radius
Actual (): 0.5
Expected (): 1
Loc: [/home/qt/.pulse2-agent/data/recipes/179082385/base/qt/qt3d/tests/auto/qml3d/sphere/tst_sphere.qml(82)]
Totals: 59 passed, 2 failed, 15 skipped
The above output means that on platforms other than Windows we are effectively ignoring 59 good test cases that could otherwise block regressions. Those test cases could be re-enabled by adding QEXPECT_FAIL calls to the two failing test cases (or fixing the failures), which would allow the insignificant_test marker to be removed for platforms other than Windows.
2. On Windows (at least on the "win32-msvc2010 Windows 7" configuration, which is the only Windows config in CI at present), the test appears to crash consistently with exit code 0xC0000005 (STATUS_ACCESS_VIOLATION).
For this platform, the test could be re-enabled by adding a QSKIP to the beginning of the crashing test case, e.g.:
#ifdef Q_OS_WIN
QSKIP("QTBUG-?????: This test crashes on Windows");
#endif
Running the test with the -v1 command-line option should be enough to reveal which test case is crashing (CI currently runs the tests with -silent which hides the output from passing test cases before the crash).
Attachments
For Gerrit Dashboard: QTBUG-25276 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
24974,1 | Try to prevent crash on windows qml3d. | master | qt/qt3d | Status: ABANDONED | +2 | 0 |
24975,1 | See if tests pass now after last change. | master | qt/qt3d | Status: ABANDONED | +2 | 0 |
25141,3 | Fix qml3d tests incl windows. | master | qt/qt3d | Status: MERGED | +2 | 0 |