Details
-
Type:
Bug
-
Status: Closed
-
Priority:
P1: Critical
-
Resolution: Done
-
Affects Version/s: 6.0, 6.1, 6.2
-
Fix Version/s: 6.4.0 Beta1
-
Component/s: Testing: qtestlib
-
Labels:
-
Platform/s:
-
Story Points:8
-
Commits:13d2e13290 (qt/qtbase/dev) 13d2e13290 (qt/tqtc-qtbase/dev)
-
Sprint:Qt6_Foundation_Sprint 40, Foundation_Sprint 41 (Team 1), Team 1 Foundation_Sprint 42, Team 1 Foundation_Sprint 43, Team 1 Foundation_Sprint 44, Team 1 Foundation_Sprint 45, Team 1 Foundation_Sprint 46, Team 1 Foundation_Sprint 47, Team 1 Foundation_Sprint 48, Team One Foundation Sprint 49
Description
https://codereview.qt-project.org/c/qt/qtbase/+/253709Â introduced incorrect behavior for counting tests. A blacklisted test will be counted twice now.
There are also issues with expect-fail tests (both blacklisted and not), where an expected pass will count as a failure, but with Continue will go on testing, possibly introducing more expected passes, each one increasing the failure count, and perhaps finally resulting in a failed QVERIFY, furthermore increasing the count. The real count should increase by 1, since these are all failures in the same test. (we count tests, not verifications).
Attachments
Issue Links
- is duplicated by
-
QTBUG-98454 QtTestlib JUnitXML reporter generates a failure and a skip in the same test case
-
- Reported
-
- is required for
-
QTBUG-96844 Rationalize logging in QtTestLib
-
- Closed
-