Details
-
Bug
-
Resolution: Done
-
P1: Critical
-
None
-
production
-
None
Description
When auto test fails in CI, the CI will re-run the failed steps n times to see if the test is really failing, or if it is just flaky. Now the re-running got broken while it seems that the testlib doesn't understand the given argument: "-maxwarning".
See for example:
https://testresults.qt.io/coin/integration/qt/qt5/tasks/1549632670
agent:2019/01/16 16:00:54 build.go:193: Totals: 1362 passed, 25 failed, 14 skipped, 0 blacklisted, 2869ms agent:2019/01/16 16:00:54 build.go:193: ********* Finished testing of tst_QString ********* agent:2019/01/16 16:00:54 build.go:193: Makefile:320: recipe for target 'check' failed agent:2019/01/16 16:00:54 build.go:193: make: *** [check] Error 25 agent:2019/01/16 16:00:54 build.go:237: Process finished with error: exit status 2 agent:2019/01/16 16:00:54 build.go:196: Error reading from stdout/err: exit status 2 agent:2019/01/16 16:01:14 runner.go:151: Re-running tests/auto/corelib/tools/qstring with TESTARGS=localeAwareCompare agent:2019/01/16 16:01:14 build.go:313: [taskset -c 0 make check -j1] 900 900 agent:2019/01/16 16:01:14 build.go:193: /home/qt/work/qt/qtbase/tests/auto/corelib/tools/qstring/target_wrapper.sh ./tst_qstring localeAwareCompare agent:2019/01/16 16:01:14 build.go:193: ********* Start testing of tst_QString *********
vs.
https://testresults.qt.io/coin/integration/qt/qtbase/tasks/1549674376
https://testresults.qt.io/logs/qt/qtbase/b07b7b2e7d2dbfceb10743cab2b6c30a3de641d4/LinuxUbuntu_18_04x86_64LinuxUbuntu_18_04x86_64GCCqtci-linux-Ubuntu-18.04-x86_64-9a8851NoWidgets_ForceDebugInfo/2a48a3b7dcb9652b1532a3ad14de7580b5b9b1ee/test_1549674399/log.txt.gz
agent:2019/01/17 13:57:33 build.go:192: Totals: 1360 passed, 25 failed, 14 skipped, 0 blacklisted, 166ms agent:2019/01/17 13:57:33 build.go:192: ********* Finished testing of tst_QString ********* agent:2019/01/17 13:57:33 build.go:192: Makefile:298: recipe for target 'check' failed agent:2019/01/17 13:57:33 build.go:192: make: *** [check] Error 25 agent:2019/01/17 13:57:33 build.go:236: Process finished with error: exit status 2 agent:2019/01/17 13:57:33 build.go:195: Error reading from stdout/err: exit status 2 agent:2019/01/17 13:57:53 runner.go:151: Re-running tests/auto/corelib/tools/qstring with TESTARGS=localeAwareCompare agent:2019/01/17 13:57:53 build.go:314: [make check -j1] 900 900 agent:2019/01/17 13:57:53 build.go:192: /home/qt/work/qt/qtbase/tests/auto/corelib/tools/qstring/target_wrapper.sh ./tst_qstring -maxwarnings agent:2019/01/17 13:57:53 build.go:192: -maxwarnings needs an extra parameter with the amount of warnings agent:2019/01/17 13:57:53 build.go:192: Makefile:298: recipe for target 'check' failed agent:2019/01/17 13:57:53 build.go:192: make: *** [check] Error 1
Attachments
Issue Links
- resulted from
-
-
- Closed
-
| For Gerrit Dashboard: QTQAINFRA-2584 | ||||||
|---|---|---|---|---|---|---|
| # | Subject | Branch | Project | Status | CR | V |
| 250472,2 | Fix flaky test re-run | master | qtqa/tqtc-coin-ci | Status: MERGED | +2 | 0 |