Details
-
Suggestion
-
Resolution: Unresolved
-
Not Evaluated
-
None
-
None
-
None
Description
I briefly tried the new exception-throwing behaviour of the QTest macros (https://codereview.qt-project.org/c/qt/qtbase/+/529437) out in one of my projects:
https://github.com/mitchcurtis/slate/commit/60d385db24bdfa74f5e8150332df369e41d2703d
They work well, but there is one thing I'm missing. Before, I had custom macros that would tell me the call sites of each helper so that I could see where they were actually called from in the test. An example of output from a failing helper function:
FAIL! : tst_App::rulersAndGuides(TilesetType) 'addNewGuide(10, Qt::Horizontal)' returned FALSE. (bool TestHelper::addNewGuide(int, Qt::Orientation):1581 Compared values are not the same Actual: (grabWithGuide.pixelColor(releasePos.x(), releasePos.y()).name(QColor::HexArgb)): "#ffcccccc" Expected: (QColor(Qt::cyan).name(QColor::HexArgb)): "#ff00ffff") Loc: [/Users/mitch/dev/slate/tests/auto/tst_app.cpp(3384)]
With the throwing macros, you only get the location of the failure:
FAIL! : tst_App::rulersAndGuides(TilesetType) Compared values are not the same Actual (grabWithGuide.pixelColor(releasePos.x(), releasePos.y()).name(QColor::HexArgb)): "#ffcccccc" Expected (QColor(Qt::cyan).name(QColor::HexArgb)) : "#ff00ffff" Loc: [/Users/mitch/dev/slate/tests/shared/testhelper.cpp(1581)]
This makes it very hard to locate the execution path that led to the failure. In the output above, for example, you can't tell where in the actual test function the failure happened if there is more than one call to that helper. It becomes even harder if you have helpers calling other helpers.
I'm proposing an opt-in API that prints a backtrace of sorts in the failure message. It could do this by having the macros check whatever new flag we add to control this, and if it's set, print the location of the failure and re-throw the exception. Then, at each call site of a helper:
// Has no return value, so can't use QVERIFY
QCATCH(verifyWithSomeHelperFunction());
Or it could be named QCATCH_AND_RETHROW, etc.
Attachments
Issue Links
- is covered by
-
QTBUG-121556 As a user of QTestLib, I'd like to use Catch2-style hierarchical SCENARIO/GIVEN/WHEN/THEN tests
- Open
- relates to
-
QTBUG-66320 Provide a way to conveniently use test helper functions with qtestlib
- Closed
Gerrit Reviews
For Gerrit Dashboard: QTBUG-123009 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
547718,1 | WIP: Add QCATCH | dev | qt/qtbase | Status: NEW | -2 | 0 |