The need to write scripts to determine test plans and execute them (flaky handling) should be minimized (for example, COIN currently runs qmake -r to create a test plan). However, it should be considered what ctest provides, first.
- Add functionality create test plan and present results
- Move flaky rerun policies into testlib (note ctest also has some re-run feature)
- Provide more Gui utilities (High DPI screen/window sizes, High DPI fuzz comparison, image comparison); potentially as a separate module? This would allow for removing a lot of code duplicated in the tests.
- Re-evaluate reasons why we limit executables to single test classes and if the reasons do no more apply add functionality to fully support this feature
(QTest::qExec() can be used multiple times and results could be added to return the final result.. but this loses e.g. functionality of explicitly running single test functions)
- Bring code up to scratch (however carefully considering performance)
- Check macros:
- Consider using exceptions instead of void returns - this would remove the need to always use void for helper functions
- Performance: Restructure QCOMPARE to only format the operands to strings in case of failure (currently, operands are always formatted, which is costly)
- Provide other operators (QLESS_THAN,...) which allows for removing a lot of QVERIFY and log the values in case of fails
- Data driven tests
- Performance: Use coroutines to yield test data via some tuple (would remove the need to store a matrix of QVariants which is a performance hit, currently)
- Provide something for resources during shared over the live time of data driven tests. This is currently not possible.
|For Gerrit Dashboard: QTBUG-69413|
|265652,3||WIP: QTestLib: support for test macros in sub-functions||dev||qt/qtbase||Status: NEW||-2||0|
|264951,6||QTestLib: Use member initialization||dev||qt/qtbase||Status: MERGED||+2||0|
|264952,6||QTestLib: Fix class declarations/structure||dev||qt/qtbase||Status: MERGED||+2||0|
|264953,6||QTestLib: Introduce nullptr||dev||qt/qtbase||Status: MERGED||+2||0|
|264954,8||QTestLib: Replace typedef by 'using'||dev||qt/qtbase||Status: MERGED||+2||0|
|264955,5||QTestLib: Fix various clang warnings||dev||qt/qtbase||Status: MERGED||+2||0|