Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-46175

The output of a failed verify with a message is confusing



    • Type: Bug
    • Status: Open
    • Priority: P5: Not important
    • Resolution: Unresolved
    • Affects Version/s: 5.5.1
    • Fix Version/s: None
    • Component/s: Testing: QuickTest
    • Labels:


      The documentation for QVERIFY2 says:

      The QVERIFY2() macro behaves exactly like QVERIFY(), except that it outputs a verbose message when condition is false. The message is a plain C string.

      It then uses this code example:

      QVERIFY2(1 + 1 == 2, "A breach in basic arithmetic occurred.");

      This would imply that the message should be used to state that something went wrong. The documentation for the verify() method of TestCase says that it should be used like its C++ counterpart. So, following that advice:

      var newImage = grabImage(control);
      verify(hasContents(control, newImage), "RulerLabels has no content when its from-to range is reversed");

      We get this failure message:

      FAIL!  : tst_extras::RulerLabels::test_reversed() 'RulerLabels has no content when its from-to range is reversed' returned FALSE. ()
         Loc: [/home/micurtis/dev/qtquickcontrols2/tests/auto/extras/data/tst_rulerlabels.qml(162)]

      This is confusing, as it reads as if the statement "RulerLabels has no content when its from-to range is reversed" is false for this test. In C++, you'd get something like:

      FAIL!  : RulerLabels::test_reversed() 'hasContents(control, newImage)' returned FALSE. (RulerLabels has no content when its from-to range is reversed)

      This makes sense. You see the expression and that it returned false, and then you have a detailed explanation of what the test was actually testing. We can't do this in QML, since we don't have macros, but we could change the output for failures to be something like:

      FAIL!  : tst_extras::RulerLabels::test_reversed() verify() returned FALSE (RulerLabels has no content when its from-to range is reversed)
         Loc: [/home/micurtis/dev/qtquickcontrols2/tests/auto/extras/data/tst_rulerlabels.qml(162)]


        No reviews matched the request. Check your Options in the drop-down menu of this sections header.



            • Assignee:
              qt.team.quick.subscriptions Qt Quick and Widgets Team
              mitch_curtis Mitch Curtis
            • Votes:
              0 Vote for this issue
              1 Start watching this issue


              • Created:

                Gerrit Reviews

                There are no open Gerrit changes