Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-5903

When a QCOMPARE of two lengthy values fails, the debugging output is incomplete

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • P2: Important
    • 6.6.1, 6.7.0 FF
    • 4.5.2, 6.5
    • Testing: qtestlib
    • None
    • I expect this to work in many environments, but my particular one is a MacBook Pro from two years ago running Mac OS 10.6 and Qt 4.5.2 with gcc 4.2.1 (Apple Inc. build 5646).
    • All
    • 3602eac18 (dev), 18152699e (dev), a63e0b853 (6.6)

    Description

      When QCOMPARE fails, it writes its output using qDebug(), which caps the output at a certain number of characters (1024, I think). But that output includes both sides of the comparison, and thus if one or both are lengthy, essential info may be missing, including most importantly the line number of the failed test case.

      For example, if you build the attached .cpp file with the attached .pro file and run it, it produces the output shown here:

      ********* Start testing of test_long_qtest_debug *********
      Config: Using QTest library 4.5.2, Qt 4.5.2
      PASS   : test_long_qtest_debug::initTestCase()
      FAIL!  : test_long_qtest_debug::test_one() Compared strings are not the same
         Actual ("this"): this
         Expected ("that"): that
         Loc: [test_long_qtest_debug.cpp(12)]
      FAIL!  : test_long_qtest_debug::test_two() Compared values are not the same
         Actual (oneThing): some blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah
       blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah
       blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah
       blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah
       blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah
       blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah
       blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah
       blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah
       blahsome blah blahsome blah blahsome blah blahsPASS   : test_long_qtest_debug::cleanupTestCase()
      Totals: 2 passed, 2 failed, 0 skipped
      ********* Finished testing of test_long_qtest_debug *********
      

      The first failed test does what you'd hope: gives you both sides of the comparison, together with the line number of the failure.
      The second failed test only gives you part of the left side of the comparison, and thus neither the right side (not even part of it) nor the line number of the failure are visible. This is particularly difficult if you have a lengthy unit test and something fails; you're reduced to the long and silly process of experimenting with debugging statements to narrow down the problem location, which the test suite aims to provide, but in this case fails.
      This could be solved by using three or four different qDebug() calls rather than just one, together with tests for length of operands' string representations. Alternately, the format of the output could be changed to put the line number first, but this solution is less elegant and less backwards-compatible for those who have scripts that automate testing and parse its output.

      Attachments

        Issue Links

          For Gerrit Dashboard: QTBUG-5903
          # Subject Branch Project Status CR V

          Activity

            People

              mitch_curtis Mitch Curtis
              nathan Nathan Carter
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There is 1 open Gerrit change