Details
-
Sub-task
-
Resolution: Done
-
P1: Critical
-
None
-
b68bae1132b5dc5c8f55435c72cd48e24aa52ec9
Description
For each platform, a list of valid autotests that should be executed on that platform by the CI gate is needed so that the test run-rate can be calculated by the metrics system. Previous methods of generating such a list have been flawed, usually counting tests that were not valid for some platforms.
Ideally, a user should be able to run something like "make test-list" to generate a list of all test programs and test functions. This would recurse through the tests/auto tree, just like "make check", but it would run each autotest binary with the -functions switch to list the available test functions. The CI gate would run this step before running "make check" and the results would be used to determine which of the available tests did/didn't report a usable result.
For embedded platforms this will be a little more complicated, because the CI gate usually uses some substitute for "make check" and will need to do something similar for "make test-list".