Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-95661

Test result counts are incorrect

XMLWordPrintable

    • All
    • 8
    • 13d2e13290 (qt/qtbase/dev) 13d2e13290 (qt/tqtc-qtbase/dev)
    • Qt6_Foundation_Sprint 40, Foundation_Sprint 41 (Team 1), Team 1 Foundation_Sprint 42, Team 1 Foundation_Sprint 43, Team 1 Foundation_Sprint 44, Team 1 Foundation_Sprint 45, Team 1 Foundation_Sprint 46, Team 1 Foundation_Sprint 47, Team 1 Foundation_Sprint 48, Team One Foundation Sprint 49

      https://codereview.qt-project.org/c/qt/qtbase/+/253709 introduced incorrect behavior for counting tests. A blacklisted test will be counted twice now.

      There are also issues with expect-fail tests (both blacklisted and not), where an expected pass will count as a failure, but with Continue will go on testing, possibly introducing more expected passes, each one increasing the failure count, and perhaps finally resulting in a failed QVERIFY, furthermore increasing the count. The real count should increase by 1, since these are all failures in the same test. (we count tests, not verifications).

        For Gerrit Dashboard: QTBUG-95661
        # Subject Branch Project Status CR V

            Eddy Edward Welbourne
            vestbo Tor Arne Vestbø
            Vladimir Minenko Vladimir Minenko
            Alex Blasche Alex Blasche
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: