Details
-
Bug
-
Resolution: Fixed
-
Not Evaluated
-
None
-
master
-
None
-
-
2cb78aaf6 (dev)
Description
At the moment we run ctest -V (verbose), which results in logging all of the stdout/stderr of all tests to the resulting test log, even for passing tests.
This makes it harder investigate test failures, as you have to load 25MB (qtbase) of logs into your browser or editor to look for a test failure.
ctest supports a --output-on-failure option that only enables -V for any failing tests.
I propose that we enable this option in our CI to make it easier to investigate failures.
The reduced (focused) text content of a failing log would presumably also help our new AI overlords in analyzing the failure.
Note that we would still collect all tests results, as both Qt XML and JUnit XML, via the -o options to testlib. These are used for our grafana metrics.
If the full log is needed , we could add a -o foo.txt,txt option to redirect the plain text logging to a file on disk, in addition to the stdout/stderr that we currently do (that will be hidden unless the test fails).
Attachments
Issue Links
- relates to
-
QTBUG-109441 When running tests in CI, only show output when a test fails
- Closed
For Gerrit Dashboard: QTQAINFRA-6592 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
592470,13 | ctest: Only output test output on failure | dev | qt/qtbase | Status: MERGED | +2 | +1 |