Uploaded image for project: 'Qt Quality Assurance Infrastructure'
  1. Qt Quality Assurance Infrastructure
  2. QTQAINFRA-6519

Track BFAIL/BPASS histories of tests so we notice if either outcome stops happening

    XMLWordPrintable

Details

    • Suggestion
    • Resolution: Unresolved
    • P2: Important
    • None
    • 6.8
    • Metrics / Test Results
    • None
    • All

    Description

      When a test is blacklisted, we don't notice if it either switches to always failing – which would indicate that some change has fully broken it, where it was previously only flaky – or to always passing – which would indicate that we could remove its blacklisting.
      To the extend that we track the behaviour of flaky tests over time, it would be constructive to notice whether either of these transitions happens to a test, so that we can take appropriate action (albeit belatedly).

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            anwojcie Anna Wojciechowska
            Eddy Edward Welbourne
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:

              Gerrit Reviews

                There are no open Gerrit changes