Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-109441

When running tests in CI, only show output when a test fails

XMLWordPrintable

    • 2cb78aaf6 (dev)

      Currently our test logs are very big, they contain the output of running all tests.
      This makes it hard to find the exact location of the failure in the log when something fails.
      One idea would be to buffer the output of each test, and only print it if a failure happened.

      This could be implemented either at ctest level using it's --output-on-failure option, or at qt-testrunner.py level.

      Doing at ctest level is simple to understand, but it will be harder to know which tests flaky passed, because we don't see the reruns that qt-testrunner is doing.
      The advantage of doing it at qt-testrunner level, is that we could detect flaky passes and print a short one liner to the ci log mentioning that a test not only passed, but it actually flaky passed with x out of y tries.

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            vestbo Tor Arne Vestbø
            alexandru.croitor Alexandru Croitor
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved:

                There are no open Gerrit changes