Details
-
Bug
-
Resolution: Incomplete
-
P2: Important
-
None
-
production
-
None
-
e4fe162f0 (dev), 741981ff3 (dev), 5fafd262a (dev), 48aedc5c9 (dev), 51f283f00 (dev), 8b17904d6 (dev), 2d9daf0b9 (dev), bc9c99021 (dev), cb9589860 (dev), 074e3a90a (dev), d08f85f4a (dev), 75400d7c5 (dev)
Description
In preparation for Bugfix week, H2 2024, we want to blacklist the worst offending flaky tests in an effort to improve overall CI smoothness. Since a release is not immediately pending the bugfix week, this should be a relatively safe operation, though measures will be taken immedately after bugfix week to verify that the blacklisted tests are not hard-failing before un-blacklisting them again.
Axel Sporel has taken a cursory look at the proposed blacklist list below, and found no obvious customer-reported bugs which would relate to the tests being blacklisted.
| test_case | test_function | flaky_instances | fail_instances |
| inputpanel | tst_plugin::test_hangulInputMethod | 1 | 28 |
| tst_QWidget_window | mouseMoveWithPopup | 5 | 18 |
| tst_lupdate | good | 16 | 17 |
| Tst_QMqttSubscription | qtbug_106203 | 16 | 12 |
| tst_QHostInfo | lookupConnectToLambda | 56 | 9 |
| tst_QQuickMultiPointTouchArea | inFlickable | 51 | 9 |
| tst_Http2 | duplicateRequestsWithAborts | 31 | 8 |
| tst_QNetworkReply | authenticationCacheAfterCancel | 101 | 5 |
| tst_clientSSL | testRun | 61 | 5 |
| tst_QWidget | multipleToplevelFocusCheck | 9 | 5 |
| tst_QNetworkReply | ioGetFromHttpWithProxyAuth | 1 | 5 |
| tst_QWidget | hoverPosition | 60 | 4 |
| tst_QNetworkReply | httpReUsingConnectionSequential | 54 | 4 |
| tst_Printing | printFromPdfViewer | 35 | 4 |
| tst_QTcpSocket | waitForConnectedInHostLookupSlot | 24 | 4 |
| tst_QNetworkReply | ioHttpRedirectPostPut | 11 | 4 |
| tst_QNetworkReply | ioHttpRedirectFromLocalToRemote | 7 | 4 |
| tst_QHostInfo | reverseLookup | 3 | 4 |
| tst_qml | itemAndWindowGeometry | 1 | 4 |
| declarative_ui | MapItems::test_no_items_on_map | 238 | 3 |