Details
-
Bug
-
Resolution: Done
-
P2: Important
-
5.0.0 Beta 1
-
None
Description
Analysis of the historical CI logs for the qtquick1 module (builds 00054 to 00129) shows that the qdeclarativedebug test is unstable.
Two of the test functions in the test fail almost 50% of the time on Mac, Windows, Ubuntu 10.04 and Ubuntu 11.10. It appears that when one fails they both fail – I could not find any instances in the logs where one of the functions passed and the other failed.
Example of failure on Windows 7
5/14/12 3:24:38 PM EST: Testing tst_QDeclarativeDebug^M 5/14/12 3:24:40 PM EST: FAIL! : tst_QDeclarativeDebug::queryObjectTree() Compared values are not the same^M 5/14/12 3:24:40 PM EST: Actual (state.className()): Transition^M 5/14/12 3:24:40 PM EST: Expected (QString("State")): State^M 5/14/12 3:24:40 PM EST: .\tst_qdeclarativedebug.cpp(1171) : failure location^M 5/14/12 3:24:40 PM EST: FAIL! : tst_QDeclarativeDebug::setBindingInStates() Compared values are not the same^M 5/14/12 3:24:40 PM EST: Actual (state.className()): Transition^M 5/14/12 3:24:40 PM EST: Expected (QString("State")): State^M 5/14/12 3:24:40 PM EST: .\tst_qdeclarativedebug.cpp(1087) : failure location^M 5/14/12 3:24:40 PM EST: Totals: 24 passed, 2 failed, 0 skipped^M 5/14/12 3:24:40 PM EST: QtQA::App::TestRunner: test failed, running again to see if it is flaky...^M 5/14/12 3:24:40 PM EST: Qml debugging is enabled. Only use this in a safe environment!^M 5/14/12 3:24:40 PM EST: Testing tst_QDeclarativeDebug^M 5/14/12 3:24:40 PM EST: Totals: 26 passed, 0 failed, 0 skipped^M 5/14/12 3:24:40 PM EST: QtQA::App::TestRunner: test failed on first attempt and passed on second attempt!^M 5/14/12 3:24:40 PM EST: QtQA::App::TestRunner: first attempt: exited with exit code 2^M 5/14/12 3:24:40 PM EST: QtQA::App::TestRunner: the test seems to be flaky, please fix this^M 5/14/12 3:24:40 PM EST: QtQA::App::TestRunner: this flaky test is being treated as a PASS^M
Attachments
Issue Links
- is required for
-
QTQAINFRA-1949
Remove insignificant tests in Qt
-
- Withdrawn
-
| For Gerrit Dashboard: QTBUG-26702 | ||||||
|---|---|---|---|---|---|---|
| # | Subject | Branch | Project | Status | CR | V |
| 31739,1 | Refer to specific bug report for qdeclarativedebug test. | master | qt/qtquick1 | Status: MERGED | +2 | 0 |
| 32007,1 | Ensure children of the default property are correctly ordered | master | qt/qtquick1 | Status: ABANDONED | 0 | 0 |
| 32015,2 | Ensure children of the default property are correctly ordered | master | qt/qtquick1 | Status: ABANDONED | 0 | 0 |
| 32018,1 | Do not assume ordering of resources declared for the default property | master | qt/qtquick1 | Status: MERGED | +2 | 0 |