Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-117005

Review problematic usages of QEXPECT_FAIL()

    XMLWordPrintable

Details

    • Suggestion
    • Resolution: Unresolved
    • P4: Low
    • None
    • None
    • Testing: qtestlib
    • None

    Description

      QEXPECT_FAIL() is sometimes abused to cover against flaky failures. It is usually done like this:

      #if defined(Q_OS_WHATEVER)
          if (testedValue != expectedValue)
              QEXPECT_FAIL("", "test might fail on platform whatever because ...", Continue)
      #endif
          QCOMPARE(testedValue, expectedValue);
      

      This is the wrong way to do it. Mostly because we don't get notified when the test starts passing. The correct way would be to mark the whole testcase in the BLACKLIST file.

      Suggestion: QEXPECT_FAIL() should only be used unconditionally.

      All the existing such uses should be changed to BLACKLIST. It's easy to find them by doing git grep -C9 QEXPECT_FAIL() but they are quite many, unfortunately.

      Can we enforce somehow via qtestlib that QEXPECT_FAIL should never be conditioned?

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            Eddy Edward Welbourne
            jimis Dimitrios Apostolou
            Vladimir Minenko Vladimir Minenko
            Alex Blasche Test Alex Blasche Test
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:

              Gerrit Reviews

                There are no open Gerrit changes