Uploaded image for project: 'Qt for Python'
  1. Qt for Python
  2. PYSIDE-578

Improve testrunner to handle flaky tests

    XMLWordPrintable

Details

    • Task
    • Resolution: Done
    • P2: Important
    • 5.6, 5.9, 6.0
    • 5.6, 5.9
    • PySide, Shiboken
    • None

    Description

      The testrunner was in most cases sufficient for our cases. Sometimes a test behaved flaky, and we hat to repeat the COIN procedure.

      End of November 2017, we suddenly observed strange behavior of the RHEL 6.6 tests: It was almost impossible to check something in. The problem was circumvented by analysis of some specific errors and by black-listing others.

      This change shall improve our situation:
      Tests will be run multiple times in the COIN system, and failing once will not be reported as an error, but the behavior of such failing tests will be monitored.

      The correctness of this change is crucial, since it modifies the whole test system.

      Attachments

        Issue Links

          relates to

          Task - A task that needs to be done. PYSIDE-2557 Improve the Testing Framework

          • P4: Low - Would be nice to fix, but it's not very important.
          • Reported
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              ctismer Christian Tismer
              ctismer Christian Tismer
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes