Details
-
Sub-task
-
Resolution: Out of scope
-
P2: Important
-
None
Description
Currently we track only a subset of the existing benchmarks under tests/benchmarks. Many of the benchmarks are filtered out (i.e. not even built) by the 'make check-trusted' mechanism (see also QTQAINFRA-256).
There has been at least one report of a performance regression that could have been detected in the BM tool if a certain benchmark had been tracked (tests/benchmarks/gui/image/blendbench - regression from 4.6 to 4.7).
Note: The reason not all of the benchmarks are currently being tracked is that each team was asked to select just a small number of "their" benchmarks for initial tracking. Now the time is right to revise this list.