Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-83419

Q2-2020 Flaky failing autotest function remainingTime in qtimer

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • P1: Critical
    • None
    • 6.0, 5.15
    • Core: Date/Time
    • None
    • Host OS: macOS_10_14, target OS: macOS_10_14, Clang compiler
      Host OS: Windows_10, target OS: Windows_10, MSCV2019 compiler
    • macOS, Windows
    • 21
    • Qt6_Foundation_Sprint 17

    Description

      I made a quick check if there is already a bug report or a bug fix for this test case. No obvious explicit information was found. Therefore a new bug report is added.

      Here is the log from the incident

      Coin task #1: http://coin/coin/integration/qt/qtbase/tasks/1585598810

      Log #1: http://coin/coin/logview/qt/qtbase/f1fb4857bb31a4ed6acc8bfa9879462d59839550/MacOSMacOS_10_14x86_64MacOSMacOS_10_14x86_64Clangqtci-macos-10.14-x86_64-cf5cdfUseLegacyInstructions/50b35b081c15befa3297d6abc832d32c0b384250/test_1585598891

      Externally accessible log: https://testresults.qt.io/coin/integration/qt/qtbase/tasks/1585598810

       

      ********* Start testing of tst_QTimer *********
      Config: Using QtTest library 6.0.0, Qt 6.0.0 (x86_64-little_endian-lp64 shared (dynamic) release build; by Clang 11.0.0 (clang-1100.0.33.16) (Apple))
      PASS : tst_QTimer::initTestCase()
      FAIL! : tst_QTimer::remainingTime() 'remainingTime >= 50 && remainingTime <= 200' returned FALSE. (0)
      Loc: [tst_qtimer.cpp(148)]
      PASS : tst_QTimer::cleanupTestCase()
      Totals: 2 passed, 1 failed, 0 skipped, 0 blacklisted, 356ms
      ********* Finished testing of tst_QTimer *********
      

      The wanted action

      • Please delegate this task to a suitable developer
      • Fix the autotest flakiness in all affected versions, not just one. 
      • Cherry-pick the fix to all relevant branches
      • Mark the fix versions to Jira
      • Try to fix the actual issue instead of just blacklisting the test function
      • Find any possible directly or indirectly related bug reports and mark it as a resolved duplicate.

       

      Guidelines for debugging

      Sometimes the test is flaky because of the load in the virtual machine. This Qt wiki page tells you how to reproduce similar circumstances.

      https://wiki.qt.io/How_to_reproduce_autotest_fails

      The TQtC members can use internal CI-system virtual machines for debugging:

      https://intranet.qt.io/display/QTRD/Metrics+Guidelines+for+Grafana+Tool+for+Autotests+with+Issues

      Anyone can use the MiniCoin system at the need for handling the virtual machines:

      https://git.qt.io/vohilshe/minicoin

      How to locate the failed flaky test:

      Please open the link to the Grafana dashboard showing the failure history of the test:

      https://testresults.qt.io/grafana/d/5GKDJSPZz/02-qt-framework-please-fix-these-flaky-tests-random-fails?orgId=1&from=now%2Fy&to=now&var-project=qt%2Fqtbase&var-testcase=tests%2Fauto%2Fcorelib%2Fkernel%2Fqtimer&var-testfunction=remainingTime&var-branch=5.12&var-branch=5.13&var-branch=5.14&var-branch=5.15&var-branch=dev&var-inter=24h

      Please see the panel List of the recent flaky tests (both passed and failed). Sort it based on the results to see the failed instances. 

      Please open the links to the Coin tasks containing the logs. Please see the test section containing state "Error" and open corresponding logs. 

      Need to know more about the test binary execution history?

      Please see this Grafana dashboard: 

      https://testresults.qt.io/grafana/d/C-1oxSEWz/04-qt-framework-please-see-the-flakiness-history-of-tests?orgId=1&from=now%2Fy&to=now&var-project=qt%2Fqtbase&var-testcase=tests%2Fauto%2Fcorelib%2Fkernel%2Fqtimer&var-testfunction=All&var-branch=5.12&var-branch=5.13&var-branch=5.14&var-branch=5.15&var-branch=dev&var-inter=24h

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              jkobus Jarek Kobus
              assaarel Asmo Saarela (Inactive)
              Maurice Kalinowski Maurice Kalinowski
              Alex Blasche Alex Blasche
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes