Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-95661

Test result counts are incorrect

    XMLWordPrintable

Details

    • All
    • 8
    • 13d2e13290 (qt/qtbase/dev) 13d2e13290 (qt/tqtc-qtbase/dev)
    • Qt6_Foundation_Sprint 40, Foundation_Sprint 41 (Team 1), Team 1 Foundation_Sprint 42, Team 1 Foundation_Sprint 43, Team 1 Foundation_Sprint 44, Team 1 Foundation_Sprint 45, Team 1 Foundation_Sprint 46, Team 1 Foundation_Sprint 47, Team 1 Foundation_Sprint 48, Team One Foundation Sprint 49

    Description

      https://codereview.qt-project.org/c/qt/qtbase/+/253709 introduced incorrect behavior for counting tests. A blacklisted test will be counted twice now.

      There are also issues with expect-fail tests (both blacklisted and not), where an expected pass will count as a failure, but with Continue will go on testing, possibly introducing more expected passes, each one increasing the failure count, and perhaps finally resulting in a failed QVERIFY, furthermore increasing the count. The real count should increase by 1, since these are all failures in the same test. (we count tests, not verifications).

      Attachments

        Issue Links

          For Gerrit Dashboard: QTBUG-95661
          # Subject Branch Project Status CR V

          Activity

            People

              Eddy Edward Welbourne
              vestbo Tor Arne Vestbø
              Vladimir Minenko Vladimir Minenko
              Alex Blasche Alex Blasche
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: