Uploaded image for project: 'Qt Quality Assurance Infrastructure'
  1. Qt Quality Assurance Infrastructure
  2. QTQAINFRA-5093

double storing of some tests as both crashed and failed

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • P2: Important
    • None
    • master
    • None
    • 3e71573355 (qtqa/tqtc-coin-ci/master) 3e71573355 (qtqa/tqtc-coin-ci/1.5)

    Description

      In last 30 days 33 test functions from dev branch were stored as both crashed and failed at same time. That makes our statistics unreliable - since we distinguish between failed (test function returned Fail result) and crashed (function did not return the result, because ended prematurely or the result could not be parsed).
      In the attachment you will find the list as well as notebook - how the data was obtained.
      I have added links:
      coin_log_link - is link to coin with logs for a givine integration
      qt5_integration_link - is link to integration if test was part of qt5 integration
      normal_integration_link - is link to integratin if test was part of regular module integration

      Here is one example using grafana:
      https://testresults.qt.io/grafana/d/3dhio4K7k/fastcheck-ci-test-info?orgId=1&var-testCase=tst_qabstractitemview&var-testFunction=QTBUG31411_noSelection&from=1657404000000&to=1657576799000

      Attachments

        For Gerrit Dashboard: QTQAINFRA-5093
        # Subject Branch Project Status CR V

        Activity

          People

            falt Simo Fält
            anwojcie Anna Wojciechowska
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes