At present, QtTest only provides the average of a benchmark (with supporting report of total and number of runs). When examining data from routine running of a benchmark, over time, to see whether any regressions or improvements happen, it is necessary to distinguish noise from signal. To do that, those analysing the results need some way to assess the variability within each run, so as to compare changes between runs to it, as this is what can reveal whether those changes are material or noise. Sadly this information is not presently supplied (not even in verbose benchmark mode), making it hard for our QA&QE folk to interpret the data.

Determining standard deviation would also provide a way for the benchmarking subsystem to assess when it's run the test enough times to get a sensible result.