Details
-
Suggestion
-
Resolution: Fixed
-
P2: Important
-
None
-
6.8
-
None
Description
When a test is blacklisted, we don't notice if it either switches to always failing – which would indicate that some change has fully broken it, where it was previously only flaky – or to always passing – which would indicate that we could remove its blacklisting.
To the extend that we track the behaviour of flaky tests over time, it would be constructive to notice whether either of these transitions happens to a test, so that we can take appropriate action (albeit belatedly).