Details
-
Suggestion
-
Resolution: Fixed
-
P3: Somewhat important
-
None
-
None
-
e769cf026 (dev), fde57300a (dev), 45e9f5f2e (dev), 2b8cf058f (6.8), ad568b859 (dev)
-
Foundations Sprint 99, Foundation Sprint 100
Description
I have a helper class from which my test case derives from. The helper class sets up a bunch of stuff via non-test helper functions. I was using QVERIFY, QCOMPARE, etc. in those helper functions, until I realised that the test function keeps executing.
In the docs, each test function has this note:
Note: This macro can only be used in a test function that is invoked by the test framework.
This all makes sense, but it leaves a huge gap in implementing and using helper functions in tests.
My current workaround is to duplicate some of the macros:
// https://bugreports.qt.io/browse/QTBUG-66320 // Taken from qtestcase.h and modified to return bool and store a failureMessage // so that they can be used in helper functions (non-test functions). // We don't have COMPARE, because it would require us to get the failure message // into a string, the logic for which is hidden in testlib. #define VERIFY(statement) \ do { \ if (!static_cast<bool>(statement)) { \ failureMessage = #statement; \ return false; \ } \ } while (false) #define FAIL(message) \ do { \ failureMessage = message; \ return false; \ } while (false) #define VERIFY2(statement, description) \ do { \ if (!static_cast<bool>(statement)) { \ failureMessage = description; \ return false; \ } \ } while (false) #define TRY_LOOP_IMPL(expr, timeoutValue, step) \ if (!(expr)) { \ QTest::qWait(0); \ } \ int qt_test_i = 0; \ for (; qt_test_i < timeoutValue && !(expr); qt_test_i += step) { \ QTest::qWait(step); \ } #define TRY_TIMEOUT_DEBUG_IMPL(expr, timeoutValue, step)\ if (!(expr)) { \ TRY_LOOP_IMPL((expr), (2 * timeoutValue), step);\ if (expr) { \ QString msg = QString::fromUtf8("QTestLib: This test case check (\"%1\") failed because the requested timeout (%2 ms) was too short, %3 ms would have been sufficient this time."); \ msg = msg.arg(QString::fromUtf8(#expr)).arg(timeoutValue).arg(timeoutValue + qt_test_i); \ FAIL(qPrintable(msg)); \ } \ } // Ideally we'd use qWaitFor instead of QTRY_LOOP_IMPL, but due // to a compiler bug on MSVC < 2017 we can't (see QTBUG-59096) #define TRY_IMPL(expr, timeout)\ const int qt_test_step = 50; \ const int qt_test_timeoutValue = timeout; \ TRY_LOOP_IMPL((expr), qt_test_timeoutValue, qt_test_step); \ TRY_TIMEOUT_DEBUG_IMPL((expr), qt_test_timeoutValue, qt_test_step)\ // Will try to wait for the expression to become true while allowing event processing #define TRY_VERIFY_WITH_TIMEOUT(expr, timeout) \ do { \ TRY_IMPL((expr), timeout);\ VERIFY(expr); \ } while (false) #define TRY_VERIFY(expr) TRY_VERIFY_WITH_TIMEOUT((expr), 5000) // Will try to wait for the expression to become true while allowing event processing #define TRY_VERIFY2_WITH_TIMEOUT(expr, messageExpression, timeout) \ do { \ TRY_IMPL((expr), timeout);\ VERIFY2(expr, messageExpression); \ } while (false) #define TRY_VERIFY2(expr, messageExpression) TRY_VERIFY2_WITH_TIMEOUT((expr), (messageExpression), 5000)
I store a failureMessage string in the helper class:
const char *failureMessage;
I ensure that the result of the helper is used (they return bool so that the actual test function can verify that they succeed), and I give them a goofy prefix so that it's obvious that they're a helper:
Q_REQUIRED_RESULT bool helpDeleteSavedTestGames();
bool GameTestCase::helpDeleteSavedTestGames() { // ... VERIFY(game->view()->hasActiveFocus()); // ...
For example, the VERIFY macro returns false if it fails, and stores the failure message. The client code in the actual test functions uses the helper like so:
QVERIFY2(helpDeleteSavedTestGames(), failureMessage);
This has worked fine so far, but it uses private API, so it would be good if there was an official way to do this.
Attachments
Issue Links
- relates to
-
QTBUG-62354 QVERIFY / QCOMPARE macros could use an upgrade
- Reported
-
QTBUG-38890 Speed up autotests
- Closed
-
QTBUG-123009 Print a "backtrace" from the exception-throwing QTest macros
- Reported
-
QTBUG-94476 Add varargs variants of QFAIL(), QSKIP(), QVERIFY2() that take printf-type format strings and parameters for them
- Open