Details
-
User Story
-
Resolution: Fixed
-
P3: Somewhat important
-
6.4.1, 6.5.0 Beta1
-
None
-
2cb78aaf6 (dev)
Description
Currently our test logs are very big, they contain the output of running all tests.
This makes it hard to find the exact location of the failure in the log when something fails.
One idea would be to buffer the output of each test, and only print it if a failure happened.
This could be implemented either at ctest level using it's --output-on-failure option, or at qt-testrunner.py level.
Doing at ctest level is simple to understand, but it will be harder to know which tests flaky passed, because we don't see the reruns that qt-testrunner is doing.
The advantage of doing it at qt-testrunner level, is that we could detect flaky passes and print a short one liner to the ci log mentioning that a test not only passed, but it actually flaky passed with x out of y tries.
Attachments
Issue Links
- depends on
-
COIN-978 Add a verbose test logs checkbox to integration scheduler
- Closed
- relates to
-
QTQAINFRA-5146 qt-testrunner.py (or ctest) should write a status file
- Closed
-
QTQAINFRA-6592 Disable test output unless test fails
- Closed