Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-83080

Q12020 Flaky failing autotest function testRun in /external_IODevice/tst_client

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Reported
    • Priority: P1: Critical
    • Resolution: Unresolved
    • Affects Version/s: 5.15
    • Fix Version/s: None
    • Component/s: Remote Objects
    • Labels:
      None
    • Platform/s:
      Linux/X11, macOS

      Description

      I made a quick check if there is already a bug report or a bug fix for this test case. No obvious explicit information was found. Therefore a new bug report is added.

      Here is the log from the incident

      Coin task #1: http://coin/coin/integration/qt/qtremoteobjects/tasks/1580906237

      Host OS version #1: Ubuntu_18_04, target OS version QEMU, GCC compiler

      Host OS version #2: macOS_10_13, target OS macOS_10_13, Clang compiler

      Log #1:http://coin/coin/logview/qt/qtremoteobjects/6df31445814ec205c1dd5344e9b2b80f2de9c715/LinuxUbuntu_18_04x86_64LinuxQEMUarmv7GCCqtci-linux-Ubuntu-18.04-x86_64-ca0cfe/2f0e6c893be3e2af718e9857f7edb7a178ebfbd8/test_1580908050

      ********* Start testing of tst_clientSSL *********
      Config: Using QtTest library 5.15.0, Qt 5.15.0 (arm-little_endian-ilp32-eabi-hardfloat shared (dynamic) release build; by GCC 8.2.0)
      PASS : tst_clientSSL::initTestCase()
      getsockopt level=41 optname=26 not yet supported
      FAIL! : tst_clientSSL::testRun() 'socketClient->waitForEncrypted(-1)' returned FALSE. ()
      Loc: [tst_client.cpp(77)]
      QWARN : tst_clientSSL::testRun() QProcess: Destroyed while process ("/home/qt/work/qt/qtremoteobjects/tests/auto/external_IODevice/sslTestServer/sslTestServer") is still running.
      PASS : tst_clientSSL::cleanupTestCase()
      Totals: 2 passed, 1 failed, 0 skipped, 0 blacklisted, 726ms
      ********* Finished testing of tst_clientSSL *********
      ...
      ********* Start testing of tst_clientSSL *********
      Config: Using QtTest library 5.15.0, Qt 5.15.0 (arm-little_endian-ilp32-eabi-hardfloat shared (dynamic) release build; by GCC 8.2.0)
      PASS : tst_clientSSL::initTestCase()
      FAIL! : tst_clientSSL::testRun() 'socketClient->waitForEncrypted(-1)' returned FALSE. ()
      Loc: [tst_client.cpp(77)]
      getsockopt level=41 optname=26 not yet supported
      QWARN : tst_clientSSL::testRun() [ 0.010 W] default unknown - QProcess: Destroyed while process ("/home/qt/work/qt/qtremoteobjects/tests/auto/external_IODevice/sslTestServer/sslTestServer") is still running.
      PASS : tst_clientSSL::cleanupTestCase()
      Totals: 2 passed, 1 failed, 0 skipped, 0 blacklisted, 702ms
      ********* Finished testing of tst_clientSSL *********
      

      Externally accessible log: https://testresults.qt.io/coin/integration/qt/qtremoteobjects/tasks/1580906237

      The wanted action

      • Please delegate this task to a suitable developer
      • Fix the autotest flakiness in all affected versions, not just one. 
      • Cherry-pick the fix to all relevant branches
      • Mark the fix versions to Jira
      • Try to fix the actual issue instead of just blacklisting the test function
      • Find any possible directly or indirectly related bug reports and mark it as a resolved duplicate.

      Guidelines for debugging

      Please see the internal page https://intranet.qt.io/display/QTRD/Metrics+Guidelines+for+Grafana+Tool+for+Autotests+with+Issues

      How to locate the failed flaky test:

      Please open the link to the Grafana dashboard showing the failure history of the test:

      https://testresults.qt.io/grafana/d/5GKDJSPZz/02-qt-framework-please-fix-these-flaky-tests-random-fails?orgId=1&from=now%2Fy&to=now&var-project=qt%2Fqtremoteobjects&var-testcase=tests%2Fauto%2Fexternal_IODevice%2Ftst_client&var-testfunction=testRun&var-branch=5.12&var-branch=5.13&var-branch=5.14&var-branch=5.15&var-branch=dev&var-inter=24h

      Please see the panel List of the recent flaky tests (both passed and failed). Sort it based on the results to see the failed instances. 

      Please open the links to the Coin tasks containing the logs. Please see the test section containing state "Error" and open corresponding logs. 

      Need to know more about the test binary execution history?

      Please see this Grafana dashboard: 

      https://testresults.qt.io/grafana/d/C-1oxSEWz/04-qt-framework-please-see-the-flakiness-history-of-tests?orgId=1&from=now%2Fy&to=now&var-project=qt%2Fqtremoteobjects&var-testcase=tests%2Fauto%2Fexternal_IODevice%2Ftst_client&var-testfunction=All&var-branch=5.12&var-branch=5.13&var-branch=5.14&var-branch=5.15&var-branch=dev&var-inter=24h

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

            Assignee:
            bstottle Brett Stottlemyer
            Reporter:
            assaarel Asmo Saarela
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:

                Gerrit Reviews

                There are no open Gerrit changes