XMLWordPrintable

Details

    • Task
    • Resolution: Fixed
    • P2: Important
    • None
    • None
    • Metrics / Test Results
    • None

    Description

      Create a script that, given a specific workitem ID (aka buildKey):

      • it fetches the testresults tarball
      • it reads the XML test files
      • It prints data points about whether tests were passing/failing or being flaky
      • (Later) it inserts the points to a database

      Example workitem ID (buildKey):

      qt/qtlanguageserver/19b874f185b768bd902a23e40e9a305998bf28ff/MacOSMacOS_12x86_64AndroidAndroid_ANYx86_64Clangqtci-macos-12-x86_64-102-09b1e8DisableTests_Sccache_UseConfigure/f96bee4fbb6540effbede89dc0816a3fc1bf6bc9/build_1661782709
      

      NOTES

      • A test_name consists of test_executable+test_function+test_datatag
      • A test_name may re-run up to 5 times in order to combat flakiness. The final run is what sets the test_outcome.
        • Possible test outcomes: pass, fail, crash
      • Every single run of a test_name can have a test_result.
        • These are printed listed in the XML file: FAIL,XPASS, PASS,XFAIL,BFAIL,BPASS,BXFAIL,BXPASS,SKIP...
      • The number of re-runs should also be recorded.

      See suggested schema for storing all that in QTQAINFRA-4563.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              teheikki Tero Heikkinen
              jimis Dimitrios Apostolou
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes