Uploaded image for project: 'Qt Quality Assurance Infrastructure'
  1. Qt Quality Assurance Infrastructure
  2. QTQAINFRA-6210

Report crashes that are discoverable because of the information in ctest_results table

    XMLWordPrintable

Details

    • Task
    • Resolution: Unresolved
    • P2: Important
    • None
    • None
    • Metrics / Test Results
    • None

    Description

      With the ctest_results table we can see find the integrations that have zero entries in it. These are CI-flaky since the test run never completed.

      • has a "fail" ctest result recorded but
        no JUnit XML result OR a pass-outcome result?
      • ctest_message is Timeout
      • the workitem result is fail/error AND type=Test AND ctest-junit-xml file was expected (i.e. not the legacy platform) but missing

      TODO a crashed_tests_2 TABLE that contain crashed tests without XML files, known only to ctest_results

      0. if a workitem has no XML results at all, then it's OK (possibly ASAN build or a non-qt project)
      1. test/emulator crash: failure in ctest_results, missing test XML file, not ASAN testrun or other type that does not generate XML (i.e. workitem must have produced other XMLs)
      2. timeout/other crash: existing XML results for workitem but missing ctest_result
      3. failed workitem without individually failed test in neither ctest or QTest -> CI flakiness

      For the association between qtest and ctest results to happen, we need to match the TestCase as reported by qtest, with the testcase as reported by ctest (is that the test executable?)

      • TODO might need to INDEX ctest_results based on ctest_testcase_n

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            jimis Dimitrios Apostolou
            jimis Dimitrios Apostolou
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Gerrit Reviews

                There are no open Gerrit changes