Uploaded image for project: 'Qt Quality Assurance Infrastructure'
  1. Qt Quality Assurance Infrastructure
  2. QTQAINFRA-6592

Disable test output unless test fails

    XMLWordPrintable

Details

    • All
    • 2cb78aaf6 (dev)

    Description

      At the moment we run ctest -V (verbose), which results in logging all of the stdout/stderr of all tests to the resulting test log, even for passing tests.

      This makes it harder investigate test failures, as you have to load 25MB (qtbase) of logs into your browser or editor to look for a test failure.

      ctest supports a --output-on-failure option that only enables -V for any failing tests.

      I propose that we enable this option in our CI to make it easier to investigate failures.

      The reduced (focused) text content of a failing log would presumably also help our new AI overlords in analyzing the failure.

      Note that we would still collect all tests results, as both Qt XML and JUnit XML, via the -o options to testlib. These are used for our grafana metrics.

      If the full log is needed , we could add a -o foo.txt,txt option to redirect the plain text logging to a file on disk, in addition to the stdout/stderr that we currently do (that will be hidden unless the test fails).

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              vestbo Tor Arne Vestbø
              vestbo Tor Arne Vestbø
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes