Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-21848 Improve reporting of autotest run-rates and pass-rates
  3. QTBUG-21850

Generate accurate list of valid tests for each supported Qt5 platform

    XMLWordPrintable

Details

    • Sub-task
    • Resolution: Done
    • P1: Critical
    • 5.0.0
    • Testing: qtestlib
    • None
    • b68bae1132b5dc5c8f55435c72cd48e24aa52ec9

    Description

      For each platform, a list of valid autotests that should be executed on that platform by the CI gate is needed so that the test run-rate can be calculated by the metrics system. Previous methods of generating such a list have been flawed, usually counting tests that were not valid for some platforms.

      Ideally, a user should be able to run something like "make test-list" to generate a list of all test programs and test functions. This would recurse through the tests/auto tree, just like "make check", but it would run each autotest binary with the -functions switch to list the available test functions. The CI gate would run this step before running "make check" and the results would be used to determine which of the available tests did/didn't report a usable result.

      For embedded platforms this will be a little more complicated, because the CI gate usually uses some substitute for "make check" and will need to do something similar for "make test-list".

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            jasmcdon Jason McDonald (Closed Nokia Identity. Please assign to "macadder" instead) (Inactive)
            jasmcdon Jason McDonald (Closed Nokia Identity. Please assign to "macadder" instead) (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes