Details
-
Task
-
Resolution: Fixed
-
P2: Important
-
None
-
None
-
None
-
655bed656 (dev)
Description
Given a test, Test, with the tests 'a', 'b', 'c' and 'd'.
If 'c' times out then it is labelled FAIL and the test returns.
The test-runner sees this and reruns 'c'. If 'c' passes then all is good.
Except 'd' never ran.
Real example below:
ctrl-f "FAIL! : tst_QTimer::singleShotToFunctors"
agent:2022/10/14 08:08:03 build.go:394: Start 19: tst_qtimer agent:2022/10/14 08:08:03 build.go:394: 19: Test command: /opt/cmake-3.21.3/bin/cmake "-P" "/home/qt/work/qt/qtbase_standalone_tests/tests/auto/corelib/kernel/qtimer/tst_qtimerWrapperRelWithDebInfo.cmake" agent:2022/10/14 08:08:03 build.go:394: 19: Test timeout computed to be: 1500 agent:2022/10/14 08:08:03 build.go:394: 19: qt-testrunner.py INFO: Will write XML test logs to directory COIN_CTEST_RESULTSDIR=/home/qt/work/testresults agent:2022/10/14 08:08:03 build.go:394: 19: qt-testrunner.py DEBUG: Running test command line: ['/home/qt/work/qt/qtbase_standalone_tests/tests/auto/corelib/kernel/qtimer/tst_qtimer', '-o', '/home/qt/work/testresults/tst_qtimer-1665734883650.xml,xml', '-o', '/home/qt/work/testresults/tst_qtimer-1665734883650.junit.xml,junitxml', '-o', '-,txt'] agent:2022/10/14 08:08:03 build.go:394: 19: ********* Start testing of tst_QTimer ********* agent:2022/10/14 08:08:03 build.go:394: 19: Config: Using QtTest library 6.5.0, Qt 6.5.0 (arm64-little_endian-lp64 shared (dynamic) release build; by GCC 10.2.0), b2qt 3.2 agent:2022/10/14 08:08:03 build.go:394: 19: PASS : tst_QTimer::initTestCase() agent:2022/10/14 08:08:03 build.go:394: 19: PASS : tst_QTimer::zeroTimer() agent:2022/10/14 08:08:04 build.go:394: 19: PASS : tst_QTimer::singleShotTimeout() agent:2022/10/14 08:08:04 build.go:394: 19: PASS : tst_QTimer::timeout() agent:2022/10/14 08:08:05 build.go:394: 19: PASS : tst_QTimer::remainingTime() agent:2022/10/14 08:08:05 build.go:394: 19: PASS : tst_QTimer::remainingTimeInitial(precise time 0ms) agent:2022/10/14 08:08:05 build.go:394: 19: PASS : tst_QTimer::remainingTimeInitial(precise time 1ms) agent:2022/10/14 08:08:05 build.go:394: 19: PASS : tst_QTimer::remainingTimeInitial(precise time 10ms) agent:2022/10/14 08:08:05 build.go:394: 19: PASS : tst_QTimer::remainingTimeInitial(coarse time 0ms) agent:2022/10/14 08:08:05 build.go:394: 19: PASS : tst_QTimer::remainingTimeInitial(coarse time 1ms) agent:2022/10/14 08:08:05 build.go:394: 19: PASS : tst_QTimer::remainingTimeInitial(coarse time 10ms) agent:2022/10/14 08:08:05 build.go:394: 19: PASS : tst_QTimer::remainingTimeDuringActivation(repeating) agent:2022/10/14 08:08:05 build.go:394: 19: PASS : tst_QTimer::remainingTimeDuringActivation(single-shot) agent:2022/10/14 08:08:05 build.go:394: 19: PASS : tst_QTimer::basic_chrono() agent:2022/10/14 08:08:06 build.go:394: 19: PASS : tst_QTimer::livelock(zero timer) agent:2022/10/14 08:08:06 build.go:394: 19: PASS : tst_QTimer::livelock(non-zero timer) agent:2022/10/14 08:08:06 build.go:394: 19: PASS : tst_QTimer::livelock(longer than sleep) agent:2022/10/14 08:08:06 build.go:394: 19: PASS : tst_QTimer::timerInfiniteRecursion(zero timer) agent:2022/10/14 08:08:06 build.go:394: 19: PASS : tst_QTimer::timerInfiniteRecursion(non-zero timer) agent:2022/10/14 08:08:06 build.go:394: 19: PASS : tst_QTimer::timerInfiniteRecursion(10ms timer) agent:2022/10/14 08:08:06 build.go:394: 19: PASS : tst_QTimer::timerInfiniteRecursion(11ms timer) agent:2022/10/14 08:08:07 build.go:394: 19: PASS : tst_QTimer::timerInfiniteRecursion(100ms timer) agent:2022/10/14 08:08:10 build.go:394: 19: PASS : tst_QTimer::timerInfiniteRecursion(1s timer) agent:2022/10/14 08:08:10 build.go:394: 19: PASS : tst_QTimer::recurringTimer(zero timer, don't recurse) agent:2022/10/14 08:08:10 build.go:394: 19: PASS : tst_QTimer::recurringTimer(zero timer, recurse) agent:2022/10/14 08:08:10 build.go:394: 19: PASS : tst_QTimer::recurringTimer(non-zero timer, don't recurse) agent:2022/10/14 08:08:11 build.go:394: 19: PASS : tst_QTimer::recurringTimer(non-zero timer, recurse) agent:2022/10/14 08:08:11 build.go:394: 19: PASS : tst_QTimer::deleteLaterOnQTimer() agent:2022/10/14 08:08:11 build.go:394: 19: QWARN : tst_QTimer::moveToThread() QObject::killTimer: Timers cannot be stopped from another thread agent:2022/10/14 08:08:11 build.go:394: 19: QWARN : tst_QTimer::moveToThread() QObject::~QObject: Timers cannot be stopped from another thread agent:2022/10/14 08:08:11 build.go:394: 19: PASS : tst_QTimer::moveToThread() agent:2022/10/14 08:08:22 build.go:394: 19: PASS : tst_QTimer::restartedTimerFiresTooSoon() agent:2022/10/14 08:08:22 build.go:394: 19: PASS : tst_QTimer::timerFiresOnlyOncePerProcessEvents(zero timer) agent:2022/10/14 08:08:23 build.go:394: 19: PASS : tst_QTimer::timerFiresOnlyOncePerProcessEvents(non-zero timer) agent:2022/10/14 08:08:23 build.go:394: 19: QWARN : tst_QTimer::timerIdPersistsAfterThreadExit() QObject::killTimer: Timers cannot be stopped from another thread agent:2022/10/14 08:08:23 build.go:394: 19: QWARN : tst_QTimer::timerIdPersistsAfterThreadExit() QObject::~QObject: Timers cannot be stopped from another thread agent:2022/10/14 08:08:23 build.go:394: 19: PASS : tst_QTimer::timerIdPersistsAfterThreadExit() agent:2022/10/14 08:08:23 build.go:394: 19: PASS : tst_QTimer::cancelLongTimer() agent:2022/10/14 08:08:24 build.go:394: 19: PASS : tst_QTimer::singleShotStaticFunctionZeroTimeout() agent:2022/10/14 08:08:25 build.go:394: 19: PASS : tst_QTimer::recurseOnTimeoutAndStopTimer() agent:2022/10/14 08:13:25 build.go:394: 19: agent:2022/10/14 08:13:25 build.go:394: 19: singleShotToFunctors function time: 300002ms, total time: 321365ms agent:2022/10/14 08:13:25 build.go:394: 19: agent:2022/10/14 08:13:25 build.go:394: 19: === Stack trace === agent:2022/10/14 08:13:25 build.go:394: 19: Could not attach to process. If your uid matches the uid of the target agent:2022/10/14 08:13:25 build.go:394: 19: process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try agent:2022/10/14 08:13:25 build.go:394: 19: again as the root user. For more details, see /etc/sysctl.d/10-ptrace.conf agent:2022/10/14 08:13:25 build.go:394: 19: ptrace: Inappropriate ioctl for device. agent:2022/10/14 08:13:25 build.go:394: 19: === End of stack trace === agent:2022/10/14 08:13:25 build.go:394: 19: QFATAL : tst_QTimer::singleShotToFunctors() Test function timed out agent:2022/10/14 08:13:25 build.go:394: 19: QObject: Cannot create children for a parent that is in a different thread. agent:2022/10/14 08:13:25 build.go:394: 19: (Parent is QCoreApplication(0x5501833ad8), parent's thread is QThread(0x550004d9a0), current thread is QThread(0x5500055390) agent:2022/10/14 08:13:25 build.go:394: 19: FAIL! : tst_QTimer::singleShotToFunctors() Received a fatal error. agent:2022/10/14 08:13:25 build.go:394: 19: Totals: 36 passed, 1 failed, 0 skipped, 0 blacklisted, 322019ms agent:2022/10/14 08:13:25 build.go:394: 19: ********* Finished testing of tst_QTimer ********* agent:2022/10/14 08:13:25 build.go:394: 19: Received signal 6 (SIGABRT) agent:2022/10/14 08:13:25 build.go:394: 19: singleShotToFunctors function time: 300659ms, total time: 322022ms agent:2022/10/14 08:13:25 build.go:394: 19: agent:2022/10/14 08:13:25 build.go:394: 19: === Stack trace === agent:2022/10/14 08:13:25 build.go:394: 19: Could not attach to process. If your uid matches the uid of the target agent:2022/10/14 08:13:25 build.go:394: 19: process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try agent:2022/10/14 08:13:25 build.go:394: 19: again as the root user. For more details, see /etc/sysctl.d/10-ptrace.conf agent:2022/10/14 08:13:25 build.go:394: 19: ptrace: Inappropriate ioctl for device. agent:2022/10/14 08:13:25 build.go:394: 19: === End of stack trace === agent:2022/10/14 08:13:25 build.go:394: 19: qemu: uncaught target signal 6 (Aborted) - core dumped agent:2022/10/14 08:13:25 build.go:394: 19: qt-testrunner.py INFO: Test process exited with code: -6 agent:2022/10/14 08:13:25 build.go:394: 19: qt-testrunner.py INFO: Parsed XML file /home/qt/work/testresults/tst_qtimer-1665734883650.xml in 0.001 seconds agent:2022/10/14 08:13:25 build.go:394: 19: qt-testrunner.py INFO: Found 36 passes and 1 failures agent:2022/10/14 08:13:25 build.go:394: 19: qt-testrunner.py INFO: Some tests failed, will re-run at most 5 times. agent:2022/10/14 08:13:25 build.go:394: 19: agent:2022/10/14 08:13:25 build.go:394: 19: qt-testrunner.py INFO: Re-running testcase: singleShotToFunctors agent:2022/10/14 08:13:25 build.go:394: 19: qt-testrunner.py DEBUG: Running test command line: ['/home/qt/work/qt/qtbase_standalone_tests/tests/auto/corelib/kernel/qtimer/tst_qtimer', '-o', '/home/qt/work/testresults/tst_qtimer-1665735205929.xml,xml', '-o', '/home/qt/work/testresults/tst_qtimer-1665735205929.junit.xml,junitxml', '-o', '-,txt', 'singleShotToFunctors'] agent:2022/10/14 08:13:26 build.go:394: 19: ********* Start testing of tst_QTimer ********* agent:2022/10/14 08:13:26 build.go:394: 19: Config: Using QtTest library 6.5.0, Qt 6.5.0 (arm64-little_endian-lp64 shared (dynamic) release build; by GCC 10.2.0), b2qt 3.2 agent:2022/10/14 08:13:26 build.go:394: 19: PASS : tst_QTimer::initTestCase() agent:2022/10/14 08:13:26 build.go:394: 19: PASS : tst_QTimer::singleShotToFunctors() agent:2022/10/14 08:13:26 build.go:394: 19: PASS : tst_QTimer::cleanupTestCase() agent:2022/10/14 08:13:26 build.go:394: 19: Totals: 3 passed, 0 failed, 0 skipped, 0 blacklisted, 937ms agent:2022/10/14 08:13:26 build.go:394: 19: ********* Finished testing of tst_QTimer ********* agent:2022/10/14 08:13:26 build.go:394: 19: qt-testrunner.py INFO: Test process exited with code: 0 agent:2022/10/14 08:13:26 build.go:394: 19: qt-testrunner.py INFO: Test has PASSed as FLAKY after re-runs:1, passes:1, failures:0 agent:2022/10/14 08:13:27 build.go:394: 19/591 Test #19: tst_qtimer .............................................. Passed 323.47 sec
Attachments
For Gerrit Dashboard: QTQAINFRA-5226 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
589927,2 | qt-testrunner: add testcase that verifies a crashed test re-runs fully | dev | qt/qtbase | Status: NEW | 0 | +1 |
588147,3 | qt-testrunner: re-run the full test if it crashed | dev | qt/qtbase | Status: MERGED | +2 | 0 |