Loading...

XML

Word

Printable

Details

Type: Bug
Resolution: Fixed
Priority: P2: Important
Fix Version/s: 6.6.1, 6.7.0 FF
Affects Version/s: 4.5.2, 6.5
Component/s: Testing: qtestlib
Labels:
None
Environment:
I expect this to work in many environments, but my particular one is a MacBook Pro from two years ago running Mac OS 10.6 and Qt 4.5.2 with gcc 4.2.1 (Apple Inc. build 5646).

Platform/s:

All
Commits:
3602eac18 (dev), 18152699e (dev), a63e0b853 (6.6)

Description

When QCOMPARE fails, it writes its output using qDebug(), which caps the output at a certain number of characters (1024, I think). But that output includes both sides of the comparison, and thus if one or both are lengthy, essential info may be missing, including most importantly the line number of the failed test case.

For example, if you build the attached .cpp file with the attached .pro file and run it, it produces the output shown here:

********* Start testing of test_long_qtest_debug *********
Config: Using QTest library 4.5.2, Qt 4.5.2
PASS   : test_long_qtest_debug::initTestCase()
FAIL!  : test_long_qtest_debug::test_one() Compared strings are not the same
   Actual ("this"): this
   Expected ("that"): that
   Loc: [test_long_qtest_debug.cpp(12)]
FAIL!  : test_long_qtest_debug::test_two() Compared values are not the same
   Actual (oneThing): some blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah
 blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah
 blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah
 blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah
 blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah
 blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah
 blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah
 blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah blahsome blah
 blahsome blah blahsome blah blahsome blah blahsPASS   : test_long_qtest_debug::cleanupTestCase()
Totals: 2 passed, 2 failed, 0 skipped
********* Finished testing of test_long_qtest_debug *********

The first failed test does what you'd hope: gives you both sides of the comparison, together with the line number of the failure.
The second failed test only gives you part of the left side of the comparison, and thus neither the right side (not even part of it) nor the line number of the failure are visible. This is particularly difficult if you have a lengthy unit test and something fails; you're reduced to the long and silly process of experimenting with debugging statements to narrow down the problem location, which the test suite aims to provide, but in this case fails.
This could be solved by using three or four different qDebug() calls rather than just one, together with tests for length of operands' string representations. Alternately, the format of the output could be changed to put the line number first, but this solution is less elegant and less backwards-compatible for those who have scripts that automate testing and parse its output.