Uploaded image for project: 'Qt for Python'
  1. Qt for Python
  2. PYSIDE-578

Improve testrunner to handle flaky tests

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Closed
    • Priority: P2: Important
    • Resolution: Done
    • Affects Version/s: 5.6, 5.9, dev
    • Fix Version/s: 5.6, 5.9, dev
    • Component/s: PySide, Shiboken
    • Labels:
      None
    • Technical Risk:
      High

      Description

      The testrunner was in most cases sufficient for our cases. Sometimes a test behaved flaky, and we hat to repeat the COIN procedure.

      End of November 2017, we suddenly observed strange behavior of the RHEL 6.6 tests: It was almost impossible to check something in. The problem was circumvented by analysis of some specific errors and by black-listing others.

      This change shall improve our situation:
      Tests will be run multiple times in the COIN system, and failing once will not be reported as an error, but the behavior of such failing tests will be monitored.

      The correctness of this change is crucial, since it modifies the whole test system.

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

            Assignee:
            ctismer Christian Tismer
            Reporter:
            ctismer Christian Tismer
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Gerrit Reviews

                There are no open Gerrit changes