Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-25298

Fix insignificant QMdiArea test

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Closed
    • Priority: P2: Important
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 5.4.0
    • Labels:
      None
    • Technical Risk:
      Normal
    • Commits:
      c171a21ae4913d4e6cd8d88a8b23474895336829

      Description

      qtbase/tests/auto/widgets/widgets/qmdiarea/qmdiarea.pro is marked as an insignificant_test, meaning that the results are discarded and the test is not used by the CI system to block regressions.

      Examining recent CI logs shows that the test appears to fail in a stable fashion on Mac, while on Linux 3 failures appear stable and one appears unstable.

      Unable to find source-code formatter for language: mac failure. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml
      4/13/12 10:25:11 AM EST: Testing tst_QMdiArea
      4/13/12 10:25:21 AM EST: FAIL!  : tst_QMdiArea::subWindowActivated2() Compared values are not the same
      4/13/12 10:25:21 AM EST:    Actual   (spy.count()): 2
      4/13/12 10:25:21 AM EST:    Expected (1): 1
      4/13/12 10:25:21 AM EST:    Loc: [tst_qmdiarea.cpp(533)]
      4/13/12 10:25:31 AM EST: FAIL!  : tst_QMdiArea::tileSubWindows() Compared values are not the same
      4/13/12 10:25:31 AM EST:    Actual   (workspace.viewport()->rect().size()): QSize(917x317)
      4/13/12 10:25:31 AM EST:    Expected (expectedViewportSize): QSize(902x302)
      4/13/12 10:25:31 AM EST:    Loc: [tst_qmdiarea.cpp(1759)]
      4/13/12 10:25:31 AM EST: FAIL!  : tst_QMdiArea::cascadeAndTileSubWindows() Compared values are not the same
      4/13/12 10:25:31 AM EST:    Actual   (windows.at(2)->geometry().top() - windows.at(1)->geometry().top()): 21
      4/13/12 10:25:31 AM EST:    Expected (dy): 19
      4/13/12 10:25:31 AM EST:    Loc: [tst_qmdiarea.cpp(1828)]
      4/13/12 10:25:36 AM EST: FAIL!  : tst_QMdiArea::setActivationOrder(CreationOrder) 'verifyArrangement(&mdiArea, Cascaded, expectedCascadeIndices)' returned FALSE. ()
      4/13/12 10:25:36 AM EST:    Loc: [tst_qmdiarea.cpp(2289)]
      4/13/12 10:25:36 AM EST: FAIL!  : tst_QMdiArea::setActivationOrder(StackingOrder) 'verifyArrangement(&mdiArea, Cascaded, expectedCascadeIndices)' returned FALSE. ()
      4/13/12 10:25:36 AM EST:    Loc: [tst_qmdiarea.cpp(2289)]
      4/13/12 10:25:36 AM EST: FAIL!  : tst_QMdiArea::setActivationOrder(ActivationHistoryOrder) 'verifyArrangement(&mdiArea, Cascaded, expectedCascadeIndices)' returned FALSE. ()
      4/13/12 10:25:36 AM EST:    Loc: [tst_qmdiarea.cpp(2289)]
      4/13/12 10:25:36 AM EST: FAIL!  : tst_QMdiArea::tabBetweenSubWindows() 'tabBetweenSubWindowsIn(&mdiArea)' returned FALSE. ()
      4/13/12 10:25:36 AM EST:    Loc: [tst_qmdiarea.cpp(2336)]
      4/13/12 10:25:36 AM EST: Totals: 43 passed, 7 failed, 0 skipped
      
      linux failure
      4/13/12 10:27:10 AM EST: Testing tst_QMdiArea
      4/13/12 10:27:15 AM EST: FAIL!  : tst_QMdiArea::subWindowActivated2() Compared values are not the same
      4/13/12 10:27:15 AM EST:    Actual   (spy.count()): 0
      4/13/12 10:27:15 AM EST:    Expected (1): 1
      4/13/12 10:27:15 AM EST:    Loc: [tst_qmdiarea.cpp(533)]
      4/13/12 10:27:15 AM EST: FAIL!  : tst_QMdiArea::activeSubWindow() '!mdiArea->activeSubWindow()' returned FALSE. ()
      4/13/12 10:27:15 AM EST:    Loc: [tst_qmdiarea.cpp(1034)]
      4/13/12 10:27:15 AM EST: FAIL!  : tst_QMdiArea::currentSubWindow() '!mdiArea.activeSubWindow()' returned FALSE. ()
      4/13/12 10:27:15 AM EST:    Loc: [tst_qmdiarea.cpp(1119)]
      4/13/12 10:27:20 AM EST: FAIL!  : tst_QMdiArea::tileSubWindows() '!hBar->isVisible()' returned FALSE. ()
      4/13/12 10:27:20 AM EST:    Loc: [tst_qmdiarea.cpp(1788)]
      4/13/12 10:27:30 AM EST: Totals: 46 passed, 4 failed, 0 skipped
      

      The last of the linux failures only seems to appear in the log around 1/3 of the time.

      Test coverage could be improved by using QEXPECT_FAIL to document these known failures. That would allow the insignificant_test marker to be removed so that the good part of the test can be used by CI to block future regressions.

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

              People

              Assignee:
              tvete Paul Olav Tvete
              Reporter:
              jasmcdon Jason McDonald (Closed Nokia Identity. Please assign to "macadder" instead) (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Gerrit Reviews

                  There are no open Gerrit changes