Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-74611

Flaky tests/auto/webchannel crashes on multiple platforms in CI

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • P1: Critical
    • Resolution: Done
    • 5.12, 5.13
    • 5.14.1
    • WebChannel
    • None

    Description

      Crashing TestWebChannel is causing almost all failures of qtwebchannel component in CI:

      https://testresults.qt.io/grafana/d/000000007/coin-flaky-tests?orgId=1&var-project=qt%2Fqtwebchannel&var-branch=All&var-inter=1d

       

      Windows10 crash:

      http://testresults.qt.io/coin/integration/qt/qtwebchannel/tasks/1553172265

       

      agent:2019/03/21 12:53:24 build.go:449: Executing instruction 31 of 36 - RunQtUnitTest
      agent:2019/03/21 12:53:24 runner.go:72: Generating test results with TESTARGS=-o C:\Users\qt\work\testresults\tests\auto\webchannel251589919,xml -o -,txt
      agent:2019/03/21 12:53:24 build.go:314: [c:\users\qt\MSVC2017.bat jom check] 900 900
      agent:2019/03/21 12:53:24 build.go:192: 
      agent:2019/03/21 12:53:24 build.go:192: C:\Users\qt\work\qt\qtwebchannel\tests\auto\webchannel>call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\Build\vcvarsall.bat" amd64_x86  
      agent:2019/03/21 12:53:25 build.go:192: **********************************************************************
      agent:2019/03/21 12:53:25 build.go:192: ** Visual Studio 2017 Developer Command Prompt v15.4.0
      agent:2019/03/21 12:53:25 build.go:192: ** Copyright (c) 2017 Microsoft Corporation
      agent:2019/03/21 12:53:25 build.go:192: **********************************************************************
      agent:2019/03/21 12:53:26 build.go:192: [vcvarsall.bat] Environment initialized for: 'x64_x86'
      agent:2019/03/21 12:53:26 build.go:192: 
      agent:2019/03/21 12:53:26 build.go:192: jom 1.1.2 - empower your cores
      agent:2019/03/21 12:53:26 build.go:192: 
      agent:2019/03/21 12:53:26 build.go:192: 	C:\Utils\Jom\jom.exe -f Makefile.Release check
      agent:2019/03/21 12:53:27 build.go:192: 	call C:\Users\qt\work\qt\qtwebchannel\tests\auto\webchannel\target_wrapper.bat  release\tst_webchannel.exe -o C:\Users\qt\work\testresults\tests\auto\webchannel251589919,xml -o -,txt
      agent:2019/03/21 12:53:27 build.go:192: ********* Start testing of TestWebChannel *********
      agent:2019/03/21 12:53:27 build.go:192: Config: Using QtTest library 5.14.0, Qt 5.14.0 (i386-little_endian-ilp32 shared (dynamic) release build; by MSVC 2017)
      agent:2019/03/21 12:53:27 build.go:192: PASS   : TestWebChannel::initTestCase()
      agent:2019/03/21 12:53:27 build.go:192: PASS   : TestWebChannel::testRegisterObjects()
      agent:2019/03/21 12:53:27 build.go:192: PASS   : TestWebChannel::testDeregisterObjects()
      agent:2019/03/21 12:53:27 build.go:192: PASS   : TestWebChannel::testDeregisterObjectAtStart()
      agent:2019/03/21 12:53:27 build.go:192: PASS   : TestWebChannel::testInfoForObject()
      agent:2019/03/21 12:53:27 build.go:192: PASS   : TestWebChannel::testInvokeMethodConversion()
      agent:2019/03/21 12:53:27 build.go:192: PASS   : TestWebChannel::testSetPropertyConversion()
      agent:2019/03/21 12:53:27 build.go:192: PASS   : TestWebChannel::testDisconnect()
      agent:2019/03/21 12:53:27 build.go:192: PASS   : TestWebChannel::testWrapRegisteredObject()
      agent:2019/03/21 12:53:27 build.go:192: PASS   : TestWebChannel::testUnwrapObject()
      agent:2019/03/21 12:53:27 build.go:192: PASS   : TestWebChannel::testRemoveUnusedTransports()
      agent:2019/03/21 12:53:27 build.go:192: PASS   : TestWebChannel::testPassWrappedObjectBack()
      agent:2019/03/21 12:53:27 build.go:192: PASS   : TestWebChannel::testWrapValues()
      agent:2019/03/21 12:53:27 build.go:192: PASS   : TestWebChannel::testWrapObjectWithMultipleTransports()
      agent:2019/03/21 12:53:27 build.go:192: PASS   : TestWebChannel::testJsonToVariant()
      agent:2019/03/21 12:53:27 build.go:192: PASS   : TestWebChannel::testInfiniteRecursion()
      agent:2019/03/21 12:53:28 build.go:192: A crash occurred in C:\Users\qt\work\qt\qtwebchannel\tests\auto\webchannel\release\tst_webchannel.exe.
      agent:2019/03/21 12:53:28 build.go:192: Function time: 76ms Total time: 96ms
      agent:2019/03/21 12:53:28 build.go:192: 
      agent:2019/03/21 12:53:28 build.go:192: Exception address: 0x696953BD
      agent:2019/03/21 12:53:28 build.go:192: Exception code   : 0xc0000005
      agent:2019/03/21 12:53:28 build.go:192: Nearby symbol    : QMetaObjectPublisher::deleteWrappedObject
      agent:2019/03/21 12:53:28 build.go:192: 
      agent:2019/03/21 12:53:28 build.go:192: Stack:
      agent:2019/03/21 12:53:28 build.go:192: #  1: QTest::toString() - 0x69659B40
      agent:2019/03/21 12:53:28 build.go:192: 
      agent:2019/03/21 12:53:28 build.go:192: jom: C:\Users\qt\work\qt\qtwebchannel\tests\auto\webchannel\Makefile.Release [check] Error 2
      agent:2019/03/21 12:53:28 build.go:192: jom: C:\Users\qt\work\qt\qtwebchannel\tests\auto\webchannel\Makefile [release-check] Error 2
      agent:2019/03/21 12:53:28 build.go:236: Process finished with error: exit status 2
      

       

      MacOS crash:

      https://testresults.qt.io/coin/integration/qt/qtwebchannel/tasks/1552645324

      agent:2019/03/15 11:01:16 runner.go:153: Re-running tests/auto/webchannel with TESTARGS=testAsyncObject
      agent:2019/03/15 11:01:16 build.go:319: [make check -j1] 900 900
      agent:2019/03/15 11:01:16 build.go:197: /Users/qt/work/qt/qtwebchannel/tests/auto/webchannel/target_wrapper.sh  ./tst_webchannel testAsyncObject -maxwarnings 0
      agent:2019/03/15 11:01:16 build.go:197: ********* Start testing of TestWebChannel *********
      agent:2019/03/15 11:01:16 build.go:197: Config: Using QtTest library 5.12.2, Qt 5.12.2 (x86_64-little_endian-lp64 shared (dynamic) release build; by Clang 9.0.0 (clang-900.0.39.2) (Apple))
      agent:2019/03/15 11:01:16 build.go:197: PASS   : TestWebChannel::initTestCase()
      agent:2019/03/15 11:01:16 build.go:197: 
      agent:2019/03/15 11:01:16 build.go:197: === Received signal at function time: 54ms, total time: 56ms, dumping stack ===
      agent:2019/03/15 11:01:17 build.go:197: (lldb) process attach --pid 950
      agent:2019/03/15 11:01:32 build.go:197: Process 950 stopped
      agent:2019/03/15 11:01:32 build.go:197: * thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
      agent:2019/03/15 11:01:32 build.go:197:     frame #0: 0x00007fffb339a406 libsystem_kernel.dylib`__wait4_nocancel + 10
      agent:2019/03/15 11:01:32 build.go:197: libsystem_kernel.dylib`__wait4_nocancel:
      agent:2019/03/15 11:01:32 build.go:197: ->  0x7fffb339a406 <+10>: jae    0x7fffb339a410            ; <+20>
      agent:2019/03/15 11:01:32 build.go:197:     0x7fffb339a408 <+12>: movq   %rax, %rdi
      agent:2019/03/15 11:01:32 build.go:197:     0x7fffb339a40b <+15>: jmp    0x7fffb3392caf            ; cerror_nocancel
      agent:2019/03/15 11:01:32 build.go:197:     0x7fffb339a410 <+20>: retq   
      agent:2019/03/15 11:01:32 build.go:197: Target 0: (tst_webchannel) stopped.
      agent:2019/03/15 11:01:32 build.go:197: 
      agent:2019/03/15 11:01:32 build.go:197: Executable module set to "/Users/qt/work/qt/qtwebchannel/tests/auto/webchannel/tst_webchannel".
      agent:2019/03/15 11:01:32 build.go:197: Architecture set to: x86_64-apple-macosx.
      agent:2019/03/15 11:01:32 build.go:197: (lldb) bt all
      agent:2019/03/15 11:01:32 build.go:197: * thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
      agent:2019/03/15 11:01:32 build.go:197:   * frame #0: 0x00007fffb339a406 libsystem_kernel.dylib`__wait4_nocancel + 10
      agent:2019/03/15 11:01:32 build.go:197:     frame #1: 0x00007fffb3320e3d libsystem_c.dylib`system + 463
      agent:2019/03/15 11:01:32 build.go:197:     frame #2: 0x000000010daf3ba5 libQt5Test.5.dylib`___lldb_unnamed_symbol20$$libQt5Test.5.dylib + 325
      agent:2019/03/15 11:01:32 build.go:197:     frame #3: 0x000000010daf39cf libQt5Test.5.dylib`___lldb_unnamed_symbol19$$libQt5Test.5.dylib + 207
      agent:2019/03/15 11:01:32 build.go:197:     frame #4: 0x00007fffb347ab3a libsystem_platform.dylib`_sigtramp + 26
      agent:2019/03/15 11:01:32 build.go:197:     frame #5: 0x000000010d514744 libQt5WebChannel.5.dylib`___lldb_unnamed_symbol48$$libQt5WebChannel.5.dylib + 36
      agent:2019/03/15 11:01:32 build.go:197:     frame #6: 0x000000010d5145ad libQt5WebChannel.5.dylib`___lldb_unnamed_symbol42$$libQt5WebChannel.5.dylib + 61
      agent:2019/03/15 11:01:32 build.go:197:     frame #7: 0x000000010dd4a761 libQt5Core.5.dylib`TestNamespace::QObject::event(TestNamespace::QEvent*) + 769
      agent:2019/03/15 11:01:32 build.go:197:     frame #8: 0x000000010dd21008 libQt5Core.5.dylib`TestNamespace::QCoreApplicationPrivate::notify_helper(TestNamespace::QObject*, TestNamespace::QEvent*) + 88
      agent:2019/03/15 11:01:32 build.go:197:     frame #9: 0x000000010dd20c58 libQt5Core.5.dylib`TestNamespace::QCoreApplication::notifyInternal2(TestNamespace::QObject*, TestNamespace::QEvent*) + 168
      agent:2019/03/15 11:01:32 build.go:197:     frame #10: 0x000000010dd21e38 libQt5Core.5.dylib`TestNamespace::QCoreApplicationPrivate::sendPostedEvents(TestNamespace::QObject*, int, TestNamespace::QThreadData*) + 840
      agent:2019/03/15 11:01:32 build.go:197:     frame #11: 0x000000010dd792b8 libQt5Core.5.dylib`TestNamespace::QEventDispatcherUNIX::processEvents(TestNamespace::QFlags<TestNamespace::QEventLoop::ProcessEventsFlag>) + 72
      agent:2019/03/15 11:01:32 build.go:197:     frame #12: 0x000000010dd211bd libQt5Core.5.dylib`TestNamespace::QCoreApplication::processEvents(TestNamespace::QFlags<TestNamespace::QEventLoop::ProcessEventsFlag>, int) + 93
      agent:2019/03/15 11:01:32 build.go:197:     frame #13: 0x000000010dd74ae9 libQt5Core.5.dylib`TestNamespace::QTest::qWait(int) + 57
      agent:2019/03/15 11:01:32 build.go:197:     frame #14: 0x000000010d4dd966 tst_webchannel`TestNamespace::TestWebChannel::testAsyncObject() + 3846
      agent:2019/03/15 11:01:32 build.go:197:     frame #15: 0x000000010dd2adfc libQt5Core.5.dylib`TestNamespace::QMetaMethod::invoke(TestNamespace::QObject*, TestNamespace::Qt::ConnectionType, TestNamespace::QGenericReturnArgument, TestNamespace::QGenericArgument, TestNamespace::QGenericArgument, TestNamespace::QGenericArgument, TestNamespace::QGenericArgument, TestNamespace::QGenericArgument, TestNamespace::QGenericArgument, TestNamespace::QGenericArgument, TestNamespace::QGenericArgument, TestNamespace::QGenericArgument, TestNamespace::QGenericArgument) const + 1308
      agent:2019/03/15 11:01:32 build.go:197:     frame #16: 0x000000010daf1751 libQt5Test.5.dylib`___lldb_unnamed_symbol11$$libQt5Test.5.dylib + 1169
      agent:2019/03/15 11:01:32 build.go:197:     frame #17: 0x000000010daf2433 libQt5Test.5.dylib`___lldb_unnamed_symbol13$$libQt5Test.5.dylib + 787
      agent:2019/03/15 11:01:32 build.go:197:     frame #18: 0x000000010daf3557 libQt5Test.5.dylib`___lldb_unnamed_symbol17$$libQt5Test.5.dylib + 1031
      agent:2019/03/15 11:01:32 build.go:197:     frame #19: 0x000000010daf4157 libQt5Test.5.dylib`TestNamespace::QTest::qRun() + 247
      agent:2019/03/15 11:01:32 build.go:197:     frame #20: 0x000000010daf3e10 libQt5Test.5.dylib`TestNamespace::QTest::qExec(TestNamespace::QObject*, int, char**) + 16
      agent:2019/03/15 11:01:32 build.go:197:     frame #21: 0x000000010d4e1389 tst_webchannel`main + 105
      agent:2019/03/15 11:01:32 build.go:197:     frame #22: 0x00007fffb326b235 libdyld.dylib`start + 1
      agent:2019/03/15 11:01:32 build.go:197:     frame #23: 0x00007fffb326b235 libdyld.dylib`start + 1
      agent:2019/03/15 11:01:32 build.go:197:   thread #2
      agent:2019/03/15 11:01:32 build.go:197:     frame #0: 0x00007fffb339a44e libsystem_kernel.dylib`__workq_kernreturn + 10
      agent:2019/03/15 11:01:32 build.go:197:     frame #1: 0x00007fffb3484621 libsystem_pthread.dylib`_pthread_wqthread + 1426
      agent:2019/03/15 11:01:32 build.go:197:     frame #2: 0x00007fffb348407d libsystem_pthread.dylib`start_wqthread + 13
      agent:2019/03/15 11:01:32 build.go:197:   thread #3
      agent:2019/03/15 11:01:32 build.go:197:     frame #0: 0x00007fffb339a44e libsystem_kernel.dylib`__workq_kernreturn + 10
      agent:2019/03/15 11:01:32 build.go:197:     frame #1: 0x00007fffb348448e libsystem_pthread.dylib`_pthread_wqthread + 1023
      agent:2019/03/15 11:01:32 build.go:197:     frame #2: 0x00007fffb348407d libsystem_pthread.dylib`start_wqthread + 13
      agent:2019/03/15 11:01:32 build.go:197:   thread #4
      agent:2019/03/15 11:01:32 build.go:197:     frame #0: 0x00007fffb339a44e libsystem_kernel.dylib`__workq_kernreturn + 10
      agent:2019/03/15 11:01:32 build.go:197:     frame #1: 0x00007fffb3484621 libsystem_pthread.dylib`_pthread_wqthread + 1426
      agent:2019/03/15 11:01:32 build.go:197:     frame #2: 0x00007fffb348407d libsystem_pthread.dylib`start_wqthread + 13
      agent:2019/03/15 11:01:32 build.go:197:   thread #5, name = 'QThread'
      agent:2019/03/15 11:01:32 build.go:197:     frame #0: 0x00007fffb3399bf2 libsystem_kernel.dylib`__psynch_cvwait + 10
      agent:2019/03/15 11:01:32 build.go:197:     frame #1: 0x00007fffb34857fa libsystem_pthread.dylib`_pthread_cond_wait + 712
      agent:2019/03/15 11:01:32 build.go:197:     frame #2: 0x000000010db6930d libQt5Core.5.dylib`___lldb_unnamed_symbol286$$libQt5Core.5.dylib + 301
      agent:2019/03/15 11:01:32 build.go:197:     frame #3: 0x000000010db68ffe libQt5Core.5.dylib`___lldb_unnamed_symbol285$$libQt5Core.5.dylib + 94
      agent:2019/03/15 11:01:32 build.go:197:     frame #4: 0x000000010db68f77 libQt5Core.5.dylib`TestNamespace::QWaitCondition::wait(TestNamespace::QMutex*, TestNamespace::QDeadlineTimer) + 167
      agent:2019/03/15 11:01:32 build.go:197:     frame #5: 0x000000010db68e9e libQt5Core.5.dylib`TestNamespace::QWaitCondition::wait(TestNamespace::QMutex*, unsigned long) + 62
      agent:2019/03/15 11:01:32 build.go:197:     frame #6: 0x000000010daf91d6 libQt5Test.5.dylib`___lldb_unnamed_symbol34$$libQt5Test.5.dylib + 70
      agent:2019/03/15 11:01:32 build.go:197:     frame #7: 0x000000010db607dc libQt5Core.5.dylib`___lldb_unnamed_symbol227$$libQt5Core.5.dylib + 332
      agent:2019/03/15 11:01:32 build.go:197:     frame #8: 0x00007fffb348493b libsystem_pthread.dylib`_pthread_body + 180
      agent:2019/03/15 11:01:32 build.go:197:     frame #9: 0x00007fffb3484887 libsystem_pthread.dylib`_pthread_start + 286
      agent:2019/03/15 11:01:32 build.go:197:     frame #10: 0x00007fffb348408d libsystem_pthread.dylib`thread_start + 13
      agent:2019/03/15 11:01:32 build.go:197:   thread #6, name = 'QThread'
      agent:2019/03/15 11:01:32 build.go:197:     frame #0: 0x00007fffb339b19e libsystem_kernel.dylib`poll + 10
      agent:2019/03/15 11:01:32 build.go:197:     frame #1: 0x000000010dd77f80 libQt5Core.5.dylib`TestNamespace::qt_safe_poll(pollfd*, unsigned int, timespec const*) + 528
      agent:2019/03/15 11:01:32 build.go:197:     frame #2: 0x000000010dd795c9 libQt5Core.5.dylib`TestNamespace::QEventDispatcherUNIX::processEvents(TestNamespace::QFlags<TestNamespace::QEventLoop::ProcessEventsFlag>) + 857
      agent:2019/03/15 11:01:32 build.go:197:     frame #3: 0x000000010dd1c83e libQt5Core.5.dylib`TestNamespace::QEventLoop::exec(TestNamespace::QFlags<TestNamespace::QEventLoop::ProcessEventsFlag>) + 398
      agent:2019/03/15 11:01:32 build.go:197:     frame #4: 0x000000010db5f862 libQt5Core.5.dylib`TestNamespace::QThread::exec() + 114
      agent:2019/03/15 11:01:32 build.go:197:     frame #5: 0x000000010db607dc libQt5Core.5.dylib`___lldb_unnamed_symbol227$$libQt5Core.5.dylib + 332
      agent:2019/03/15 11:01:32 build.go:197:     frame #6: 0x00007fffb348493b libsystem_pthread.dylib`_pthread_body + 180
      agent:2019/03/15 11:01:32 build.go:197:     frame #7: 0x00007fffb3484887 libsystem_pthread.dylib`_pthread_start + 286
      agent:2019/03/15 11:01:32 build.go:197:     frame #8: 0x00007fffb348408d libsystem_pthread.dylib`thread_start + 13
      agent:2019/03/15 11:01:32 build.go:197: (lldb) quit
      agent:2019/03/15 11:01:32 build.go:197: === End of stack trace ===
      agent:2019/03/15 11:01:32 build.go:197: QFATAL : TestWebChannel::testAsyncObject() [     0.000 F] default unknown - Received signal 11
      agent:2019/03/15 11:01:32 build.go:197:          Function time: 54ms Total time: 56ms
      agent:2019/03/15 11:01:32 build.go:197: FAIL!  : TestWebChannel::testAsyncObject() Received a fatal error.
      agent:2019/03/15 11:01:32 build.go:197:    Loc: [Unknown file(0)]
      agent:2019/03/15 11:01:32 build.go:197: Totals: 1 passed, 1 failed, 0 skipped, 0 blacklisted, 15912ms
      agent:2019/03/15 11:01:32 build.go:197: ********* Finished testing of TestWebChannel *********
      agent:2019/03/15 11:01:32 build.go:197: make: *** [check] Abort trap: 6
      agent:2019/03/15 11:01:32 build.go:241: Process finished with error: exit status 2
      agent:2019/03/15 11:01:32 build.go:200: Error reading from stdout/err: exit status 2

       

      Flaky tests are slowing down Qt development so it is better to either fix or blacklist them.

       

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              kiburtse Kirill Burtsev
              aakeskimo Aapo Keskimölö
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes