Details
-
Suggestion
-
Resolution: Unresolved
-
P3: Somewhat important
-
None
-
None
-
None
Description
In https://codereview.qt-project.org/c/qt/qtdeclarative/+/575359/110/tests/auto/quickshapes/designhelpers/qquickrectangleshape/tst_qquickrectangleshape.cpp we have a performance optimisation:
// If we load the QML from scratch every test, each row takes ~350ms. // However, we still want it to be data-driven (so that e.g. you can // run individual rows), so we store it here and create it once at startup // rather than loop through a list in changeSignals.
Currently, since the init and cleanup callbacks are run for every data tag, we have no way of cleaning up the QML engine immediately after all of the data rows are run, and must leave all of the items it has created hanging around until the test case finishes. This is fine for smaller tests, but could cause issues in larger ones.
This minimal example demonstrates the issue:
#include <QtTest> class foo : public QObject { Q_OBJECT public: foo(); ~foo(); private slots: void initTestCase(); void cleanupTestCase(); void init(); void cleanup(); void test_changeSignals_data(); void test_changeSignals(); private: // Some pointer that is new'd in initTestCase and should be destroyed after // all data rows have run for the function }; foo::foo() {} foo::~foo() {} void foo::initTestCase() {} void foo::cleanupTestCase() {} void foo::init() { qDebug() << "init" << QTest::currentTestFunction() << QTest::currentDataTag(); } void foo::cleanup() { qDebug() << "cleanup" << QTest::currentTestFunction() << QTest::currentDataTag(); } void foo::test_changeSignals_data() { QTest::addColumn<QVariant>("propertyValue"); QTest::newRow("a") << QVariant::fromValue(1); QTest::newRow("b") << QVariant::fromValue(1); } void foo::test_changeSignals() { QFETCH(const QVariant, propertyValue); } QTEST_APPLESS_MAIN(foo) #include "tst_foo.moc"
The output is:
17:23:33: Starting /home/mitch/dev/temp/untitled6-qt_dev2_debug_no_asan-Debug/foo... ********* Start testing of foo ********* Config: Using QtTest library 6.10.0, Qt 6.10.0 (x86_64-little_endian-lp64 shared (dynamic) debug build; by GCC 13.3.0), ubuntu 24.04 PASS : foo::initTestCase() QDEBUG : foo::test_changeSignals(a) init test_changeSignals a QDEBUG : foo::test_changeSignals(a) cleanup test_changeSignals a PASS : foo::test_changeSignals(a) QDEBUG : foo::test_changeSignals(b) init test_changeSignals b QDEBUG : foo::test_changeSignals(b) cleanup test_changeSignals b PASS : foo::test_changeSignals(b) PASS : foo::cleanupTestCase() Totals: 4 passed, 0 failed, 0 skipped, 0 blacklisted, 1ms ********* Finished testing of foo ********* 17:23:33: /home/mitch/dev/temp/untitled6-qt_dev2_debug_no_asan-Debug/foo exited with code 0
https://doc.qt.io/qt-6/qtest-overview.html#creating-a-test says:
- init() will be called before each test function is executed.
- cleanup() will be called after every test function.
We should add e.g. initTestFunction(), cleanupTestFunction(), initTestFunctionRow() and cleanupTestFunctionRow().