Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-83083

Q12020 Flaky failing autotest function clear in qsequentialanimationgroup"

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: P1: Critical
    • Resolution: Fixed
    • Affects Version/s: 5.12, 6.0 (Next Major Release)
    • Fix Version/s: None
    • Component/s: Network
    • Labels:
      None
    • Platform/s:
      macOS
    • Commits:
      19aa8de06cf43aa375aeb2609003beb71ad4cac0 c668fd940d0c610324254d5aa5aab6e0769f78a6
    • Story Points:
      21

      Description

      I made a quick check if there is already a bug report or a bug fix for this test case. No obvious explicit information was found. Therefore a new bug report is added.

      Here is the log from the incident

      Coin task #1: http://coin/coin/integration/qt/qtbase/tasks/1582563857

      Host OS: macOS_10_12, target macOS_10_12, Clang compiler

      Log #1: http://coin/coin/logview/qt/qtbase/76b1e8300e7a46642e9a097c902af3f0ca299071/WindowsWindows_10x86_64WindowsWindows_10x86_64MSVC2017qtci-windows-10-x86_64-10-2c54d8/f8930b3a2d099dccbf3965271755acd446c51007/test_1582564900

      ********* Start testing of tst_QUdpSocket *********
      Config: Using QtTest library 5.12.8, Qt 5.12.8 (x86_64-little_endian-llp64 shared (dynamic) release build; by MSVC 2017)
      QDEBUG : tst_QUdpSocket::initTestCase() Will use multicast groups QHostAddress("239.177.64.17") QHostAddress("ff14::1140:b1e7:76a:66be") QVector(QHostAddress("ff12::1140:b1e7:76a:66be%ethernet_32773"), QHostAddress("ff12::1140:b1e7:76a:66be%ethernet_32774"), QHostAddress("ff12::1140:b1e7:76a:66be%tunnel_32772"))
      PASS : tst_QUdpSocket::initTestCase()
      QWARN : tst_QUdpSocket::constructing(WithoutProxy) QIODevice::read (QUdpSocket): device not open
      PASS : tst_QUdpSocket::constructing(WithoutProxy)
      PASS : tst_QUdpSocket::unconnectedServerAndClientTest(WithoutProxy)
      PASS : tst_QUdpSocket::broadcasting(WithoutProxy)
      PASS : tst_QUdpSocket::loop(WithoutProxy:"Almond!" | "Joy!")
      PASS : tst_QUdpSocket::loop(WithoutProxy:"A" | "B")
      PASS : tst_QUdpSocket::loop(WithoutProxy:"AB" | "B")
      PASS : tst_QUdpSocket::loop(WithoutProxy:"AB" | "BB")
      PASS : tst_QUdpSocket::loop(WithoutProxy:"A\0B" | "B\0B")
      PASS : tst_QUdpSocket::loop(WithoutProxy:"(nil)" | "(nil)")
      PASS : tst_QUdpSocket::loop(WithoutProxy:Bigmessage)
      PASS : tst_QUdpSocket::ipv6Loop(WithoutProxy:"Almond!" | "Joy!")
      PASS : tst_QUdpSocket::ipv6Loop(WithoutProxy:"A" | "B")
      PASS : tst_QUdpSocket::ipv6Loop(WithoutProxy:"AB" | "B")
      PASS : tst_QUdpSocket::ipv6Loop(WithoutProxy:"AB" | "BB")
      PASS : tst_QUdpSocket::ipv6Loop(WithoutProxy:"A\0B" | "B\0B")
      PASS : tst_QUdpSocket::ipv6Loop(WithoutProxy:"(nil)" | "(nil)")
      PASS : tst_QUdpSocket::ipv6Loop(WithoutProxy:Bigmessage)
      QINFO : tst_QUdpSocket::dualStack(WithoutProxy) Getting IPv4 destination address failed.
      PASS : tst_QUdpSocket::dualStack(WithoutProxy)
      PASS : tst_QUdpSocket::dualStackAutoBinding(WithoutProxy)
      PASS : tst_QUdpSocket::dualStackNoIPv4onV6only(WithoutProxy)
      PASS : tst_QUdpSocket::connectToHost(WithoutProxy)
      PASS : tst_QUdpSocket::bindAndConnectToHost(WithoutProxy)
      PASS : tst_QUdpSocket::pendingDatagramSize(WithoutProxy)
      PASS : tst_QUdpSocket::writeDatagram(WithoutProxy)
      QDEBUG : tst_QUdpSocket::performance(WithoutProxy) 1524.21MB/5.00s: 304.84MB/s
      PASS : tst_QUdpSocket::performance(WithoutProxy)
      QWARN : tst_QUdpSocket::bindMode(WithoutProxy) Failed to bind with QUdpSocket::ReuseAddressHint(The bound address is already in use), user isn't an administrator?
      PASS : tst_QUdpSocket::bindMode(WithoutProxy)
      BFAIL : tst_QUdpSocket::writeDatagramToNonExistingPeer(WithoutProxy:localhost-unbound) Compared values are not the same
      Actual (sReadyReadSpy.count()): 1
      Expected (0) : 0
      ..\tst_qudpsocket.cpp(1011) : failure location
      BFAIL : tst_QUdpSocket::writeDatagramToNonExistingPeer(WithoutProxy:localhost-bound) Compared values are not the same
      Actual (sReadyReadSpy.count()): 1
      Expected (0) : 0
      ..\tst_qudpsocket.cpp(1011) : failure location
      BFAIL : tst_QUdpSocket::writeDatagramToNonExistingPeer(WithoutProxy:remote-unbound) Compared values are not the same
      Actual (sReadyReadSpy.count()): 1
      Expected (0) : 0
      ..\tst_qudpsocket.cpp(1011) : failure location
      BFAIL : tst_QUdpSocket::writeDatagramToNonExistingPeer(WithoutProxy:remote-bound) Compared values are not the same
      Actual (sReadyReadSpy.count()): 1
      Expected (0) : 0
      ..\tst_qudpsocket.cpp(1011) : failure location
      SKIP : tst_QUdpSocket::writeToNonExistingPeer(WithoutProxy:localhost) Connected-mode UDP sockets and their behaviour are erratic
      ..\tst_qudpsocket.cpp(1030) : failure location
      SKIP : tst_QUdpSocket::writeToNonExistingPeer(WithoutProxy:remote) Connected-mode UDP sockets and their behaviour are erratic
      ..\tst_qudpsocket.cpp(1030) : failure location
      FAIL! : tst_QUdpSocket::outOfProcessConnectedClientServerTest(WithoutProxy) 'serverProcess.waitForReadyRead(3000)' returned FALSE. ()
      ..\tst_qudpsocket.cpp(1090) : failure location
      QWARN : tst_QUdpSocket::outOfProcessConnectedClientServerTest(WithoutProxy) QProcess: Destroyed while process ("clientserver\\clientserver") is still running.
      FAIL! : tst_QUdpSocket::outOfProcessUnconnectedClientServerTest(WithoutProxy) 'serverProcess.waitForReadyRead(3000)' returned FALSE. ()
      ..\tst_qudpsocket.cpp(1151) : failure location
      QWARN : tst_QUdpSocket::outOfProcessUnconnectedClientServerTest(WithoutProxy) QProcess: Destroyed while process ("clientserver\\clientserver") is still running.
      PASS : tst_QUdpSocket::zeroLengthDatagram(WithoutProxy)
      PASS : tst_QUdpSocket::multicastTtlOption(WithoutProxy:0.0.0.0 0)
      PASS : tst_QUdpSocket::multicastTtlOption(WithoutProxy:0.0.0.0 1)
      PASS : tst_QUdpSocket::multicastTtlOption(WithoutProxy:0.0.0.0 2)
      PASS : tst_QUdpSocket::multicastTtlOption(WithoutProxy:0.0.0.0 128)
      PASS : tst_QUdpSocket::multicastTtlOption(WithoutProxy:0.0.0.0 255)
      PASS : tst_QUdpSocket::multicastTtlOption(WithoutProxy:0.0.0.0 1024)
      PASS : tst_QUdpSocket::multicastTtlOption(WithoutProxy::: 0)
      PASS : tst_QUdpSocket::multicastTtlOption(WithoutProxy::: 1)
      PASS : tst_QUdpSocket::multicastTtlOption(WithoutProxy::: 2)
      PASS : tst_QUdpSocket::multicastTtlOption(WithoutProxy::: 128)
      PASS : tst_QUdpSocket::multicastTtlOption(WithoutProxy::: 255)
      PASS : tst_QUdpSocket::multicastTtlOption(WithoutProxy::: 1024)
      PASS : tst_QUdpSocket::multicastLoopbackOption(WithoutProxy:0.0.0.0 0)
      PASS : tst_QUdpSocket::multicastLoopbackOption(WithoutProxy:0.0.0.0 1)
      PASS : tst_QUdpSocket::multicastLoopbackOption(WithoutProxy:0.0.0.0 2)
      PASS : tst_QUdpSocket::multicastLoopbackOption(WithoutProxy:0.0.0.0 0 again)
      PASS : tst_QUdpSocket::multicastLoopbackOption(WithoutProxy:0.0.0.0 2 again)
      PASS : tst_QUdpSocket::multicastLoopbackOption(WithoutProxy:0.0.0.0 0 last time)
      PASS : tst_QUdpSocket::multicastLoopbackOption(WithoutProxy:0.0.0.0 1 again)
      PASS : tst_QUdpSocket::multicastLoopbackOption(WithoutProxy::: 0)
      PASS : tst_QUdpSocket::multicastLoopbackOption(WithoutProxy::: 1)
      PASS : tst_QUdpSocket::multicastLoopbackOption(WithoutProxy::: 2)
      PASS : tst_QUdpSocket::multicastLoopbackOption(WithoutProxy::: 0 again)
      PASS : tst_QUdpSocket::multicastLoopbackOption(WithoutProxy::: 2 again)
      PASS : tst_QUdpSocket::multicastLoopbackOption(WithoutProxy::: 0 last time)
      PASS : tst_QUdpSocket::multicastLoopbackOption(WithoutProxy::: 1 again)
      PASS : tst_QUdpSocket::multicastJoinBeforeBind(WithoutProxy:valid ipv4 group address)
      PASS : tst_QUdpSocket::multicastJoinBeforeBind(WithoutProxy:invalid ipv4 group address)
      PASS : tst_QUdpSocket::multicastJoinBeforeBind(WithoutProxy:valid ipv6 group address)
      PASS : tst_QUdpSocket::multicastJoinBeforeBind(WithoutProxy:valid ipv6 ethernet_32773-link group address)
      PASS : tst_QUdpSocket::multicastJoinBeforeBind(WithoutProxy:valid ipv6 ethernet_32774-link group address)
      PASS : tst_QUdpSocket::multicastJoinBeforeBind(WithoutProxy:valid ipv6 tunnel_32772-link group address)
      PASS : tst_QUdpSocket::multicastJoinBeforeBind(WithoutProxy:invalid ipv6 group address)
      PASS : tst_QUdpSocket::multicastLeaveAfterClose(WithoutProxy:ipv4)
      QDEBUG : tst_QUdpSocket::multicastLeaveAfterClose(WithoutProxy:ipv6) Will bind IPv6 sockets to QNetworkInterface(name = "ethernet_32773", hardware address = "02:00:3D:24:E1:C0", flags = IsUp IsRunning CanBroadcast CanMulticast , entries = ((address = QHostAddress("fe80::9dc9:f44:a7b9:4e1c%ethernet_32773"), netmask = QHostAddress("ffff:ffff:ffff:ffff::")), (address = QHostAddress("10.225.234.202"), netmask = QHostAddress("255.255.0.0"), broadcast = QHostAddress("10.225.255.255"))))
      PASS : tst_QUdpSocket::multicastLeaveAfterClose(WithoutProxy:ipv6)
      PASS : tst_QUdpSocket::multicastLeaveAfterClose(WithoutProxy:ipv6-link-ethernet_32773)
      PASS : tst_QUdpSocket::multicastLeaveAfterClose(WithoutProxy:ipv6-link-ethernet_32774)
      PASS : tst_QUdpSocket::multicastLeaveAfterClose(WithoutProxy:ipv6-link-tunnel_32772)
      PASS : tst_QUdpSocket::setMulticastInterface(WithoutProxy:ethernet_32773:fe80::9dc9:f44:a7b9:4e1c%ethernet_32773)
      PASS : tst_QUdpSocket::setMulticastInterface(WithoutProxy:ethernet_32773:10.225.234.202)
      PASS : tst_QUdpSocket::setMulticastInterface(WithoutProxy:ethernet_32774:fe80::91f:d52c:1682:dd2f%ethernet_32774)
      PASS : tst_QUdpSocket::setMulticastInterface(WithoutProxy:ethernet_32774:192.168.56.1)
      PASS : tst_QUdpSocket::setMulticastInterface(WithoutProxy:loopback_0:::1)
      PASS : tst_QUdpSocket::setMulticastInterface(WithoutProxy:loopback_0:127.0.0.1)
      PASS : tst_QUdpSocket::setMulticastInterface(WithoutProxy:tunnel_32772:2001:0:2851:782c:30df:61d0:3fa6:87c1)
      PASS : tst_QUdpSocket::setMulticastInterface(WithoutProxy:tunnel_32772:fe80::30df:61d0:3fa6:87c1%tunnel_32772)
      PASS : tst_QUdpSocket::multicast(WithoutProxy:valid bind, group ipv4 address)
      PASS : tst_QUdpSocket::multicast(WithoutProxy:valid bind, invalid group ipv4 address)
      PASS : tst_QUdpSocket::multicast(WithoutProxy:valid bind, group ipv6 address)
      PASS : tst_QUdpSocket::multicast(WithoutProxy:valid bind, ethernet_32773-link group ipv6 address)
      PASS : tst_QUdpSocket::multicast(WithoutProxy:valid bind, ethernet_32774-link group ipv6 address)
      PASS : tst_QUdpSocket::multicast(WithoutProxy:valid bind, tunnel_32772-link group ipv6 address)
      PASS : tst_QUdpSocket::multicast(WithoutProxy:valid bind, invalid group ipv6 address)
      PASS : tst_QUdpSocket::multicast(WithoutProxy:dual bind, group ipv4 address)
      PASS : tst_QUdpSocket::multicast(WithoutProxy:dual bind, group ipv6 address)
      PASS : tst_QUdpSocket::multicast(WithoutProxy:dual bind, ethernet_32773-link group ipv6 address)
      PASS : tst_QUdpSocket::multicast(WithoutProxy:dual bind, ethernet_32774-link group ipv6 address)
      PASS : tst_QUdpSocket::multicast(WithoutProxy:dual bind, tunnel_32772-link group ipv6 address)
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:writeDatagram) packets in 1 out 0
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:writeDatagram) packets in 2 out 1
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:writeDatagram) packets in 3 out 2
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:writeDatagram) packets in 4 out 3
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:writeDatagram) packets in 5 out 4
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:writeDatagram) packets in 6 out 5
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:writeDatagram) packets in 7 out 6
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:writeDatagram) packets in 8 out 7
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:writeDatagram) packets in 9 out 8
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:writeDatagram) packets in 10 out 9
      PASS : tst_QUdpSocket::echo(WithoutProxy:writeDatagram)
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:write) packets in 1 out 0
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:write) packets in 2 out 1
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:write) packets in 3 out 2
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:write) packets in 4 out 3
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:write) packets in 5 out 4
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:write) packets in 6 out 5
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:write) packets in 7 out 6
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:write) packets in 8 out 7
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:write) packets in 9 out 8
      QDEBUG : tst_QUdpSocket::echo(WithoutProxy:write) packets in 10 out 9
      PASS : tst_QUdpSocket::echo(WithoutProxy:write)
      QDEBUG : tst_QUdpSocket::linkLocalIPv6(WithoutProxy) QHostAddress("fe80::9dc9:f44:a7b9:4e1c%ethernet_32773")
      QDEBUG : tst_QUdpSocket::linkLocalIPv6(WithoutProxy) QHostAddress("fe80::91f:d52c:1682:dd2f%ethernet_32774")
      PASS : tst_QUdpSocket::linkLocalIPv6(WithoutProxy)
      SKIP : tst_QUdpSocket::linkLocalIPv4(WithoutProxy) No IPv4 link local addresses
      ..\tst_qudpsocket.cpp(1714) : failure location
      PASS : tst_QUdpSocket::readyRead(WithoutProxy)
      PASS : tst_QUdpSocket::readyReadForEmptyDatagram(WithoutProxy)
      PASS : tst_QUdpSocket::asyncReadDatagram(WithoutProxy)
      PASS : tst_QUdpSocket::writeInHostLookupState(WithoutProxy)
      PASS : tst_QUdpSocket::cleanupTestCase()
      Totals: 94 passed, 2 failed, 3 skipped, 4 blacklisted, 23839ms
      ********* Finished testing of tst_QUdpSocket *********
      

      Externally accessible log: https://testresults.qt.io/coin/integration/qt/qtbase/tasks/1582563857

      The wanted action

      • Please delegate this task to a suitable developer
      • Fix the autotest flakiness in all affected versions, not just one. 
      • Cherry-pick the fix to all relevant branches
      • Mark the fix versions to Jira
      • Try to fix the actual issue instead of just blacklisting the test function
      • Find any possible directly or indirectly related bug reports and mark it as a resolved duplicate.

      Guidelines for debugging

      Please see the internal page https://intranet.qt.io/display/QTRD/Metrics+Guidelines+for+Grafana+Tool+for+Autotests+with+Issues

      How to locate the failed flaky test:

      Please open the link to the Grafana dashboard showing the failure history of the test:

      https://testresults.qt.io/grafana/d/5GKDJSPZz/02-qt-framework-please-fix-these-flaky-tests-random-fails?orgId=1&from=now%2Fy&to=now&var-project=qt%2Fqtbase&var-testcase=tests%2Fauto%2Fcorelib%2Fanimation%2Fqsequentialanimationgroup&var-testfunction=clear&var-branch=5.12&var-branch=5.13&var-branch=5.14&var-branch=5.15&var-branch=dev&var-inter=24h

      Please see the panel List of the recent flaky tests (both passed and failed). Sort it based on the results to see the failed instances. 

      Please open the links to the Coin tasks containing the logs. Please see the test section containing state "Error" and open corresponding logs. 

      Need to know more about the test binary execution history?

      Please see this Grafana dashboard: 

      https://testresults.qt.io/grafana/d/C-1oxSEWz/04-qt-framework-please-see-the-flakiness-history-of-tests?orgId=1&from=now%2Fy&to=now&var-project=qt%2Fqtbase&var-testcase=tests%2Fauto%2Fcorelib%2Fanimation%2Fqsequentialanimationgroup&var-testfunction=All&var-branch=5.12&var-branch=5.13&var-branch=5.14&var-branch=5.15&var-branch=dev&var-inter=24h

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

            Assignee:
            sonakur Sona Kurazyan
            Reporter:
            assaarel Asmo Saarela
            PM Owner:
            Maurice Kalinowski
            RnD Owner:
            Alex Blasche
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Gerrit Reviews

                There are no open Gerrit changes