Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-123009

Print a "backtrace" from the exception-throwing QTest macros

    XMLWordPrintable

Details

    • Suggestion
    • Resolution: Unresolved
    • Not Evaluated
    • None
    • None
    • Testing: qtestlib
    • None

    Description

      I briefly tried the new exception-throwing behaviour of the QTest macros (https://codereview.qt-project.org/c/qt/qtbase/+/529437) out in one of my projects:

      https://github.com/mitchcurtis/slate/commit/60d385db24bdfa74f5e8150332df369e41d2703d

      They work well, but there is one thing I'm missing. Before, I had custom macros that would tell me the call sites of each helper so that I could see where they were actually called from in the test. An example of output from a failing helper function:

      FAIL!  : tst_App::rulersAndGuides(TilesetType) 'addNewGuide(10, Qt::Horizontal)' returned FALSE. (bool TestHelper::addNewGuide(int, Qt::Orientation):1581 Compared values are not the same
         Actual:   (grabWithGuide.pixelColor(releasePos.x(), releasePos.y()).name(QColor::HexArgb)): "#ffcccccc"
         Expected: (QColor(Qt::cyan).name(QColor::HexArgb)): "#ff00ffff")
         Loc: [/Users/mitch/dev/slate/tests/auto/tst_app.cpp(3384)]
      

      With the throwing macros, you only get the location of the failure:

      FAIL!  : tst_App::rulersAndGuides(TilesetType) Compared values are not the same
         Actual   (grabWithGuide.pixelColor(releasePos.x(), releasePos.y()).name(QColor::HexArgb)): "#ffcccccc"
         Expected (QColor(Qt::cyan).name(QColor::HexArgb))                                        : "#ff00ffff"
         Loc: [/Users/mitch/dev/slate/tests/shared/testhelper.cpp(1581)]
      

      This makes it very hard to locate the execution path that led to the failure. In the output above, for example, you can't tell where in the actual test function the failure happened if there is more than one call to that helper. It becomes even harder if you have helpers calling other helpers.

      I'm proposing an opt-in API that prints a backtrace of sorts in the failure message. It could do this by having the macros check whatever new flag we add to control this, and if it's set, print the location of the failure and re-throw the exception. Then, at each call site of a helper:

      // Has no return value, so can't use QVERIFY
      QCATCH(verifyWithSomeHelperFunction());
      

      Or it could be named QCATCH_AND_RETHROW, etc.

      Attachments

        Issue Links

          For Gerrit Dashboard: QTBUG-123009
          # Subject Branch Project Status CR V

          Activity

            People

              macadder Jason McDonald
              mitch_curtis Mitch Curtis
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There is 1 open Gerrit change