Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-25298

Fix insignificant QMdiArea test

    XMLWordPrintable

Details

    • c171a21ae4913d4e6cd8d88a8b23474895336829

    Description

      qtbase/tests/auto/widgets/widgets/qmdiarea/qmdiarea.pro is marked as an insignificant_test, meaning that the results are discarded and the test is not used by the CI system to block regressions.

      Examining recent CI logs shows that the test appears to fail in a stable fashion on Mac, while on Linux 3 failures appear stable and one appears unstable.

      Unable to find source-code formatter for language: mac failure. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml
      4/13/12 10:25:11 AM EST: Testing tst_QMdiArea
      4/13/12 10:25:21 AM EST: FAIL!  : tst_QMdiArea::subWindowActivated2() Compared values are not the same
      4/13/12 10:25:21 AM EST:    Actual   (spy.count()): 2
      4/13/12 10:25:21 AM EST:    Expected (1): 1
      4/13/12 10:25:21 AM EST:    Loc: [tst_qmdiarea.cpp(533)]
      4/13/12 10:25:31 AM EST: FAIL!  : tst_QMdiArea::tileSubWindows() Compared values are not the same
      4/13/12 10:25:31 AM EST:    Actual   (workspace.viewport()->rect().size()): QSize(917x317)
      4/13/12 10:25:31 AM EST:    Expected (expectedViewportSize): QSize(902x302)
      4/13/12 10:25:31 AM EST:    Loc: [tst_qmdiarea.cpp(1759)]
      4/13/12 10:25:31 AM EST: FAIL!  : tst_QMdiArea::cascadeAndTileSubWindows() Compared values are not the same
      4/13/12 10:25:31 AM EST:    Actual   (windows.at(2)->geometry().top() - windows.at(1)->geometry().top()): 21
      4/13/12 10:25:31 AM EST:    Expected (dy): 19
      4/13/12 10:25:31 AM EST:    Loc: [tst_qmdiarea.cpp(1828)]
      4/13/12 10:25:36 AM EST: FAIL!  : tst_QMdiArea::setActivationOrder(CreationOrder) 'verifyArrangement(&mdiArea, Cascaded, expectedCascadeIndices)' returned FALSE. ()
      4/13/12 10:25:36 AM EST:    Loc: [tst_qmdiarea.cpp(2289)]
      4/13/12 10:25:36 AM EST: FAIL!  : tst_QMdiArea::setActivationOrder(StackingOrder) 'verifyArrangement(&mdiArea, Cascaded, expectedCascadeIndices)' returned FALSE. ()
      4/13/12 10:25:36 AM EST:    Loc: [tst_qmdiarea.cpp(2289)]
      4/13/12 10:25:36 AM EST: FAIL!  : tst_QMdiArea::setActivationOrder(ActivationHistoryOrder) 'verifyArrangement(&mdiArea, Cascaded, expectedCascadeIndices)' returned FALSE. ()
      4/13/12 10:25:36 AM EST:    Loc: [tst_qmdiarea.cpp(2289)]
      4/13/12 10:25:36 AM EST: FAIL!  : tst_QMdiArea::tabBetweenSubWindows() 'tabBetweenSubWindowsIn(&mdiArea)' returned FALSE. ()
      4/13/12 10:25:36 AM EST:    Loc: [tst_qmdiarea.cpp(2336)]
      4/13/12 10:25:36 AM EST: Totals: 43 passed, 7 failed, 0 skipped
      
      linux failure
      4/13/12 10:27:10 AM EST: Testing tst_QMdiArea
      4/13/12 10:27:15 AM EST: FAIL!  : tst_QMdiArea::subWindowActivated2() Compared values are not the same
      4/13/12 10:27:15 AM EST:    Actual   (spy.count()): 0
      4/13/12 10:27:15 AM EST:    Expected (1): 1
      4/13/12 10:27:15 AM EST:    Loc: [tst_qmdiarea.cpp(533)]
      4/13/12 10:27:15 AM EST: FAIL!  : tst_QMdiArea::activeSubWindow() '!mdiArea->activeSubWindow()' returned FALSE. ()
      4/13/12 10:27:15 AM EST:    Loc: [tst_qmdiarea.cpp(1034)]
      4/13/12 10:27:15 AM EST: FAIL!  : tst_QMdiArea::currentSubWindow() '!mdiArea.activeSubWindow()' returned FALSE. ()
      4/13/12 10:27:15 AM EST:    Loc: [tst_qmdiarea.cpp(1119)]
      4/13/12 10:27:20 AM EST: FAIL!  : tst_QMdiArea::tileSubWindows() '!hBar->isVisible()' returned FALSE. ()
      4/13/12 10:27:20 AM EST:    Loc: [tst_qmdiarea.cpp(1788)]
      4/13/12 10:27:30 AM EST: Totals: 46 passed, 4 failed, 0 skipped
      

      The last of the linux failures only seems to appear in the log around 1/3 of the time.

      Test coverage could be improved by using QEXPECT_FAIL to document these known failures. That would allow the insignificant_test marker to be removed so that the good part of the test can be used by CI to block future regressions.

      Attachments

        Issue Links

          For Gerrit Dashboard: QTBUG-25298
          # Subject Branch Project Status CR V

          Activity

            People

              tvete Paul Olav Tvete
              jasmcdon Jason McDonald (Closed Nokia Identity. Please assign to "macadder" instead) (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes