-
Bug
-
Resolution: Unresolved
-
P2: Important
-
None
-
6.11
-
None
-
-
13
The QtTest documentation of benchmarking doesn't seem to tell the reader about the technical details of how QtTest decides how many times to rerun each benchmark or what statistics it produces. (Related, non-doc: … and perhaps should actually report standard deviation and maybe other details, at least in its verbose mode. It currently does not.) I may have missed a page elsewhere that does, but in that case it's not findable enough. One way or another, we should improve this.
It's not the primary material for the "how to write a benchmark" page (it's more of an appendix-style doc), but should be covered. Sadly, this shall almost certainly involve reading the code and working out what's going on, which shall be tricky thanks to the abstractions to let the system support several different types of measurement (time, clock ticks, …) in a common framework. Topics we should cover:
- How the system decides when to stop repeating the test to gather more data
- How the various command-line options reconfigure that
and we should take the opportunity to evaluate whether, in fact, we need to provide more information, most obviously standard deviation.
- relates to
-
QTBUG-138735 Add support for standard deviation data in benchmarks
-
- Reported
-