We should add a QUIP that instructs developers how to handle failing and flaky tests.

Things that should be covered:

• When and how to use QSKIP
• When and how to use QEXPECT_FAIL
• When and how to use BLACKLIST

Quoting https://lists.qt-project.org/pipermail/development/2017-February/028715.html:

Anybody who identifies a flaky test (ie. a test that is randomly failing in CI), can blacklist that test; under one condition. He needs to at the same time create a P0 bug report about it. Please also add the labels ‘autotest’ and ‘flaky’ to the bug report, so that we can follow up on those.

Flaky tests might be badly written tests, but they can also hide real problems in Qt. So we need to have a very high priority for looking at them and keeping our blacklist minimal. Making them a P0 ensures that the issue will be looked quickly (as they will block the next release).

It could also link to https://wiki.qt.io/How_to_reproduce_autotest_fails as additional reading.