I would like to have the possibility of testing selected groups of tests. When I would run ctest I would "activate" the group I want (by using an environment variable or another way the CI can set).

For example, lets say that I run ctest in qtdeclarative. Here is a log that shows the runtimes of each test in ASAN configuration.We see that tst_qmlformat is so slow (1500s) that it's worthwhile running it in its own workitem.

Ideas

• Use the "label" functionality of CTest and instruct Coin to execute ctest -L group1 and ctest -LE group1 in two parallel workitems.
  • Apparently the tests have to be labelled accordingly, not sure where this happens.
• Use the "regex" matching functionality of CTest and hardcode the test groups in Coin's instructions.
  • For example for the ASAN testrun we would want on workitem running ctest -R tst_qmlformat and another workitem running ctest -E tst_qmlformat.
  • This would complicate instructions quite a bit if used a lot, and we know that the Coin's yaml files are not the most readable.

More ideas and feedback is appreciated.