Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-21848 Improve reporting of autotest run-rates and pass-rates
  3. QTBUG-21849

Generate lists of autotests that are expected to run for Qt 4.7 and 4.8

    XMLWordPrintable

Details

    • Sub-task
    • Resolution: Out of scope
    • P1: Critical
    • None
    • None
    • None

    Description

      In order to calculate each platform's run-rate for autotests, the list of tests executed in a CI run must be compared with a list of all tests that are expected to run for that platform. The current technique for generating the expected list is flawed and may generate a list that contains tests that are not valid for a particular platform.

      Given the limited development lifetime of Qt4 and its low rate of autotest creation, it is acceptable to provide a "quick-and-dirty" solution for those branches and developer a more elegant solution for Qt5.

      The issue could be solved either by hard-coding a list of exclusions (things we know aren't valid autotests) into the metrics script and/or by including a flag in the .pro file of each non-test project in the tests/auto tree. The list of non-tests can be determined by examining current test results to determine which tests don't currently run for each platform, inspecting those tests to decide whether they should run.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            sbradley Shane Bradley (closed Nokia identity) (Inactive)
            jasmcdon Jason McDonald (Closed Nokia Identity. Please assign to "macadder" instead) (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes