Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-100312

androidtestrunner: A test with XPASS does not return error

    XMLWordPrintable

Details

    • bc71496a34 (qt/qtbase/dev) bc71496a34 (qt/tqtc-qtbase/dev) 388f959080 (qt/qtbase/6.2) 388f959080 (qt/tqtc-qtbase/6.2) e4b439c8de (qt/qtbase/6.3) e4b439c8de (qt/tqtc-qtbase/6.3)
    • 2022wk14FOAndroid&Automotive

    Description

      Example log:

      XPASS  : tst_QFontDatabase::condensedFontMatching() QCOMPARE(QFontMetrics(tfcByStretch).horizontalAdvance(testString()), QFontMetrics(tfcByStyleName).horizontalAdvance(testString())) returned TRUE unexpectedly.
   Loc: [/home/qt/work/qt/qtbase/tests/auto/gui/text/qfontdatabase/tst_qfontdatabase.cpp(449)]
SKIP   : tst_QFontDatabase::condensedFontMatching() This platform doesn't support font sub-family names (QTBUG-55625)

[..]

Totals: 12 passed, 1 failed, 13 skipped, 1 blacklisted, 82ms
********* Finished testing of tst_QFontDatabase *********
Execute /opt/android/sdk/platform-tools/adb uninstall org.qtproject.example.tst_qfontdatabase.
Success

qt-testrunner.py     INFO: Test process exited with code: 0

      The process returns 0, while it should have returned 1. Additionally the XML log file contains one "xpass" entry correctly.

      EDIT: Even for crashed tests androidtestrunner can return success, see comment-644144.

      Attachments

        Issue Links

          is covered by

          Bug - A problem which impairs or prevents the functions of the product. QTBUG-97695 correctly report Android test failures on crashes

          • P1: Critical - Urgent and Important, will STOP the release if matched with set FixVersion field. This includes regressions from the last version that are not edge cases; Data loss; Build issues; All but the most unlikely crashes/lockups/hanging; Serious usability issues and embarrassing bugs & Bugs critical to a deliverable. This is the highest possible priority for requirements.
          • Closed
          is required for

          User Story - Created by Jira Software - do not edit or delete. Issue type for a user story. QTBUG-98120 Fix failing qtbase CI tests for Android

          • P2: Important - Urgent, should be fixed, but will not stop the release.
          • In Progress
          relates to

          Bug - A problem which impairs or prevents the functions of the product. QTBUG-100470 Undetected test crashes on Android

          • P1: Critical - Urgent and Important, will STOP the release if matched with set FixVersion field. This includes regressions from the last version that are not edge cases; Data loss; Build issues; All but the most unlikely crashes/lockups/hanging; Serious usability issues and embarrassing bugs & Bugs critical to a deliverable. This is the highest possible priority for requirements.
          • Closed

          Suggestion - Public suggestions QTBUG-97652 Improve test to show crashes and Jni env in Android

          • P2: Important - Urgent, should be fixed, but will not stop the release.
          • Closed
          mentioned in

          Page Loading...

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              assam Assam Boudjelthia
              jimis Dimitrios Apostolou
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes