The wanted action
- Fix the autotest flakiness in all affected versions, not just one.
- Please delegate this task to a suitable developer
Guidelines for debugging
Please see the internal page https://intranet.qt.io/display/QTRD/Metrics+Guidelines+for+Grafana+Tool+for+Autotests+with+Issues
How to locate the failed flaky test:
Please open the link to the Grafana dashboard showing the failure history of the test:
Please see the panel List of the recent flaky tests (both passed and failed). Sort it based on the results to see the failed instances.
Please open the links to the Coin tasks containing the logs. Please see the test section containing state "Error" and open corresponding logs. There are two versions of the links. The first one is for the Qt company internal use allowing you to re-schedule build for debugging. The second one is public, allowing only to browse the logs.
List of the incidents:
- Incident 1: Qt 5.14: Host Ubuntu_18_04, target QEMU, GCC compiler
- Incident 2: Qt 5.15: Host Ubuntu_18_04, target QEMU, GCC compiler
- Incident 3: Qt 5.15: Host Windows_10, target Windows_10, MSVC2017 compiler
- Incident 4: Qt 5.15: Host Windows_10, target Windows_10, MSVC2017 compiler
- Incident 5: Qt 5.15: Host Ubuntu_18_04, target QEMU, GCC compiler
- Incident 6: Qt 5.15: Host Ubuntu_18_04, target QEMU, GCC compiler
The Coin log shows details and the location of the error message in it. At need, the raw log can be opened to see the entire log at once.
Here is an example of the log from the incident 6:
Public link to the log: https://testresults.qt.io/coin/logview/qt/qtmultimedia/1dcfa6ed65a8a52662c9ee9fd37dd7e3570653a0/LinuxUbuntu_18_04x86_64LinuxQEMUarmv7GCCqtci-linux-Ubuntu-18.04-x86_64-22d77f/112209028f84c1dba0d04aef767fff68fcad6fa1/test_1582753558
The keywords for searching the logs are "FAIL!", "re-running" and the test function name itself.