Details
-
Epic
-
Resolution: Unresolved
-
P2: Important
-
None
-
None
-
None
-
Test metrics from nightly builds
Description
Goals:
- Get statistics of most flaky tests by re-running already passed integrations nightly
- Get statistics of which tests are being run and which are being skipped per branch (from the same nightly runs)
- Notice broken provisioning earlier (parallel builds)
Desired architecture
- nightly runs in coin on already integrated branch heads, results available publicly
- test result parser & db importer running outside of COIN
- new database with optimized data model
- Grafana (testresults) for viewing
Phases
- Enable just for a QtBase branch
- Enable for a whole Qt5 branch integration
- Enable for multiple (2 or 3) branches
- Separate nightly builds for provisioning