Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-82906

Q12020 Flaky failing autotest function getErrors in qnetworkreply/test

    XMLWordPrintable

Details

    • Bug
    • Resolution: Cannot Reproduce
    • P1: Critical
    • None
    • 5.14
    • Network
    • None
    • Linux/X11
    • Qt6_Foundation_ Sprint 10

    Description

      Here is the log from the incident

      Coin task #1: http://coin/coin/integration/qt/qtbase/tasks/1581162938

      Host OS versions: Ubuntu_18_04. openSUSE_15_0, RHEL_7_6. Corresponding target OS versions. GCC compiler

      Please check if this was an error in the infrastructure around 2020-02-08...10 or real flaky failure

      Log #1: http://coin/coin/logview/qt/qtbase/08cd2ea3b138afbcbdafbf399b0268bc2a5b6423/LinuxUbuntu_18_04x86_64LinuxUbuntu_18_04x86_64GCCqtci-linux-Ubuntu-18.04-x86_64-5a750d/479c74d222449998da42b966abf3109b0ac22a11/test_1581163031

      ********* Start testing of tst_QNetworkReply *********
Config: Using QtTest library 5.14.2, Qt 5.14.2 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 9.1.0)
PASS : tst_QNetworkReply::initTestCase()
...
FAIL! : tst_QNetworkReply::getErrors(http-exist) Compared values are not the same
Loc: [../tst_qnetworkreply.cpp(2145)]
FAIL! : tst_QNetworkReply::getErrors(http-authentication) Compared values are not the same
Loc: [../tst_qnetworkreply.cpp(2145)]
PASS : tst_QNetworkReply::headFromHttp(rfc)
PASS : tst_QNetworkReply::headFromHttp(bigfile)
FAIL! : tst_QNetworkReply::headFromHttp(index) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2020)]
FAIL! : tst_QNetworkReply::headFromHttp(with-authentication) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2020)]
FAIL! : tst_QNetworkReply::headFromHttp(cgi) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2020)]
PASS : tst_QNetworkReply::headFromHttp(rfc+proxy)
PASS : tst_QNetworkReply::headFromHttp(bigfile+proxy)
FAIL! : tst_QNetworkReply::headFromHttp(index+proxy) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2020)]
FAIL! : tst_QNetworkReply::headFromHttp(with-authentication+proxy) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2020)]
FAIL! : tst_QNetworkReply::headFromHttp(cgi+proxy) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2020)]
FAIL! : tst_QNetworkReply::headFromHttp(rfc+proxyauth) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2020)]
FAIL! : tst_QNetworkReply::headFromHttp(bigfile+proxyauth) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2020)]
FAIL! : tst_QNetworkReply::headFromHttp(index+proxyauth) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2020)]
FAIL! : tst_QNetworkReply::headFromHttp(with-authentication+proxyauth) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2020)]
FAIL! : tst_QNetworkReply::headFromHttp(cgi+proxyauth) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2020)]
PASS : tst_QNetworkReply::headFromHttp(rfc+socks)
FAIL! : tst_QNetworkReply::headFromHttp(bigfile+socks) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2020)]
PASS : tst_QNetworkReply::headFromHttp(index+socks)
PASS : tst_QNetworkReply::headFromHttp(with-authentication+socks)
PASS : tst_QNetworkReply::headFromHttp(cgi+socks)
FAIL! : tst_QNetworkReply::headFromHttp(rfc+socksauth) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2020)]
PASS : tst_QNetworkReply::headFromHttp(bigfile+socksauth)
FAIL! : tst_QNetworkReply::headFromHttp(index+socksauth) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2020)]
FAIL! : tst_QNetworkReply::headFromHttp(with-authentication+socksauth) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2020)]
...
FAIL! : tst_QNetworkReply::putToHttp(128k+1) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2304)]
FAIL! : tst_QNetworkReply::putToHttpSynchronous(generic) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2345)]
FAIL! : tst_QNetworkReply::putToHttpSynchronous(with-nul) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2345)]
...
FAIL! : tst_QNetworkReply::postToHttp(empty) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2379)]
FAIL! : tst_QNetworkReply::postToHttp(4k+1) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2379)]
FAIL! : tst_QNetworkReply::postToHttp(128k+1) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2379)]
FAIL! : tst_QNetworkReply::postToHttp(2MB+1) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2379)]
FAIL! : tst_QNetworkReply::postToHttpSynchronous(small) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2411)]
FAIL! : tst_QNetworkReply::postToHttpSynchronous(with-nul) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2411)]
FAIL! : tst_QNetworkReply::postToHttpSynchronous(4k+1) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2411)]
FAIL! : tst_QNetworkReply::postToHttpMultipart(empty) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2673)]
FAIL! : tst_QNetworkReply::postToHttpMultipart(empty-related) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2673)]
FAIL! : tst_QNetworkReply::postToHttpMultipart(text) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2673)]
FAIL! : tst_QNetworkReply::postToHttpMultipart(text-custom) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2673)]
FAIL! : tst_QNetworkReply::postToHttpMultipart(text-text) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2673)]
FAIL! : tst_QNetworkReply::postToHttpMultipart(image) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2673)]
FAIL! : tst_QNetworkReply::postToHttpMultipart(image-as-content) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2673)]
FAIL! : tst_QNetworkReply::multipartSkipIndices() Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2717)]
FAIL! : tst_QNetworkReply::putToHttps(empty) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2801)]
FAIL! : tst_QNetworkReply::putToHttps(generic) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2801)]
FAIL! : tst_QNetworkReply::putToHttps(with-nul) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2801)]
FAIL! : tst_QNetworkReply::putToHttps(2MB+1) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2801)]
FAIL! : tst_QNetworkReply::putToHttpsSynchronous(empty) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2846)]
FAIL! : tst_QNetworkReply::putToHttpsSynchronous(generic) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2846)]
FAIL! : tst_QNetworkReply::putToHttpsSynchronous(small) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2846)]
FAIL! : tst_QNetworkReply::putToHttpsSynchronous(4k+1) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2846)]
FAIL! : tst_QNetworkReply::postToHttps(empty) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2884)]
FAIL! : tst_QNetworkReply::postToHttps(generic) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2884)]
FAIL! : tst_QNetworkReply::postToHttps(with-nul) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2884)]
FAIL! : tst_QNetworkReply::postToHttps(4k+1) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2884)]
FAIL! : tst_QNetworkReply::postToHttps(2MB+1) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2884)]
FAIL! : tst_QNetworkReply::postToHttpsSynchronous(empty) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2920)]
FAIL! : tst_QNetworkReply::postToHttpsSynchronous(generic) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2920)]
FAIL! : tst_QNetworkReply::postToHttpsSynchronous(with-nul) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2920)]
FAIL! : tst_QNetworkReply::postToHttpsSynchronous(4k+1) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2920)]
FAIL! : tst_QNetworkReply::postToHttpsSynchronous(128k+1) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2920)]
FAIL! : tst_QNetworkReply::postToHttpsMultipart(empty) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2964)]
FAIL! : tst_QNetworkReply::postToHttpsMultipart(text) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2964)]
FAIL! : tst_QNetworkReply::postToHttpsMultipart(text-custom) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2964)]
FAIL! : tst_QNetworkReply::postToHttpsMultipart(text-text-text) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2964)]
FAIL! : tst_QNetworkReply::postToHttpsMultipart(image) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2964)]
FAIL! : tst_QNetworkReply::postToHttpsMultipart(image-as-content) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(2964)]
FAIL! : tst_QNetworkReply::deleteFromHttp(405-method-not-allowed) Compared values are not the same
Loc: [../tst_qnetworkreply.cpp(3021)]
FAIL! : tst_QNetworkReply::deleteFromHttp(202-accepted) Compared values are not the same
Loc: [../tst_qnetworkreply.cpp(3021)]
FAIL! : tst_QNetworkReply::deleteFromHttp(204-no-content) Compared values are not the same
Loc: [../tst_qnetworkreply.cpp(3021)]
FAIL! : tst_QNetworkReply::deleteFromHttp(404-not-found) Compared values are not the same
Loc: [../tst_qnetworkreply.cpp(3021)]
FAIL! : tst_QNetworkReply::putGetDeleteGetFromHttp(success) Request failed: Unknown error
Loc: [../tst_qnetworkreply.cpp(3070)]
FAIL! : tst_QNetworkReply::sendCustomRequestToHttp(nonsense) Compared values are not the same
Actual (reply->error()): InsecureRedirectError
Expected (error) : OperationNotImplementedError
Loc: [../tst_qnetworkreply.cpp(3174)]
FAIL! : tst_QNetworkReply::sendCustomRequestToHttp(post) Compared values are not the same
Loc: [../tst_qnetworkreply.cpp(3174)]
FAIL! : tst_QNetworkReply::ioGetFromBuiltinHttp(http+unlimited) 'result == Success' returned FALSE. (QUrl("http://127.0.0.1:32879/qtest/rfc3252.txt") failed: # QNetworkReply::NetworkError(325) "Unknown error" )
Loc: [../tst_qnetworkreply.cpp(5163)]
QFATAL : tst_QNetworkReply::ioGetFromBuiltinHttp(http+unlimited) QThread: Destroyed while thread is still running
FAIL! : tst_QNetworkReply::ioGetFromBuiltinHttp(http+unlimited) Received a fatal error.
Loc: [Unknown file(0)]
Totals: 137 passed, 68 failed, 11 skipped, 4 blacklisted, 88063ms
********* Finished testing of tst_QNetworkReply *********
...
********* Start testing of tst_QNetworkReply *********
Config: Using QtTest library 5.14.2, Qt 5.14.2 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 9.1.0)
PASS : tst_QNetworkReply::initTestCase()
...
FAIL! : tst_QNetworkReply::getErrors(http-authentication) Compared values are not the same
Actual (reply->error()) : OperationCanceledError
Expected (QNetworkReply::NetworkError(error)): AuthenticationRequiredError
Loc: [../tst_qnetworkreply.cpp(2145)]
PASS : tst_QNetworkReply::cleanupTestCase()
Totals: 18 passed, 1 failed, 0 skipped, 1 blacklisted, 4614ms
********* Finished testing of tst_QNetworkReply *********

      Externally accessible log: https://testresults.qt.io/coin/integration/qt/qtbase/tasks/1581162938

      The wanted action

      • Please delegate this task to a suitable developer
      • Fix the autotest flakiness in all affected versions, not just one. 
      • Cherry-pick the fix to all relevant branches
      • Mark the fix versions to Jira
      • Try to fix the actual issue instead of just blacklisting the test function

      Guidelines for debugging

      Please see the internal page https://intranet.qt.io/display/QTRD/Metrics+Guidelines+for+Grafana+Tool+for+Autotests+with+Issues

      How to locate the failed flaky test:

      Please open the link to the Grafana dashboard showing the failure history of the test:

      https://testresults.qt.io/grafana/d/5GKDJSPZz/02-qt-framework-please-fix-these-flaky-tests-random-fails?orgId=1&from=now%2Fy&to=now&var-project=qt%2Fqtbase&var-testcase=tests%2Fauto%2Fnetwork%2Faccess%2Fqnetworkreply%2Ftest&var-testfunction=getErrors&var-branch=5.12&var-branch=5.13&var-branch=5.14&var-branch=5.15&var-branch=dev&var-inter=24h

      Please see the panel List of the recent flaky tests (both passed and failed). Sort it based on the results to see the failed instances. 

      Please open the links to the Coin tasks containing the logs. Please see the test section containing state "Error" and open corresponding logs. 

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            tpochep Timur Pocheptsov
            assaarel Asmo Saarela (Inactive)
            Maurice Kalinowski Maurice Kalinowski
            Alex Blasche Alex Blasche
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes