Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-103386

A test can FAIL then PASS and no failure will be recorded in the log

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: P2: Important P2: Important
    • None
    • 6.4
    • Testing: qtestlib
    • None
    • macOS

      In this nightly build and this test log of qtpositioning:

      test 4
    Start 4: tst_opcua
4: Test command: /Applications/CMake.app/Contents/bin/cmake "-P" "/Users/qt/work/qt/qtopcua_standalone_tests/tests/auto/declarative/tst_opcuaWrapperRelWithDebInfo.cmake"
4: Test timeout computed to be: 1500
4: qt-testrunner.py     INFO: Will write XML test logs to directory COIN_CTEST_RESULTSDIR=/Users/qt/work/testresults
4: qt-testrunner.py    DEBUG: Running test command line: ['/Users/qt/work/qt/qtopcua_standalone_tests/tests/auto/declarative/tst_opcua', '-import', '/Users/qt/work/qt/qtopcua/tests/auto/declarative/../../../src/plugins/declarative', '-o', '/Users/qt/work/testresults/tst_opcua-1651870032738.xml,xml', '-o', '-,txt']
4: Server Path: "/Users/qt/work/qt/qtopcua_standalone_tests/tests/auto/declarative/../../open62541-testserver/open62541-testserver.app/Contents/MacOS/open62541-testserver"
4: Starting test server
4: qml: Setting up AbsoluteNodeTest for open62541
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.qml: Client instance is invalid
4: qt.opcua.plugins.open62541.sdk.userland: "AcceptAll Certificate Verification. Any remote certificate will be accepted."
4: qml: Running AbsoluteNodeTest with open62541
4: qt.opcua.plugins.open62541.sdk.securechannel: "Connection 19 | SecureChannel 1 | Opened SecureChannel with SecurityPolicy http://opcfoundation.org/UA/SecurityPolicy#None"
4: qt.opcua.plugins.open62541.sdk.client: "Client Status: ChannelState: Open, SessionState: Closed, ConnectStatus: Good"
4: qt.opcua.plugins.open62541.sdk.client: "Client Status: ChannelState: Closed, SessionState: Closed, ConnectStatus: Good"
4: qt.opcua.plugins.open62541.sdk.userland: "AcceptAll Certificate Verification. Any remote certificate will be accepted."
4: qt.opcua.plugins.open62541.sdk.securechannel: "Connection 24 | SecureChannel 2 | Opened SecureChannel with SecurityPolicy http://opcfoundation.org/UA/SecurityPolicy#None"
4: qt.opcua.plugins.open62541.sdk.client: "Client Status: ChannelState: Open, SessionState: Closed, ConnectStatus: Good"
4: qt.opcua.plugins.open62541.sdk.client: "Client Status: ChannelState: Closed, SessionState: Closed, ConnectStatus: Good"
4: qt.opcua.plugins.open62541.sdk.userland: "AcceptAll Certificate Verification. Any remote certificate will be accepted."
4: qt.opcua.plugins.open62541.sdk.securechannel: "Connection 24 | SecureChannel 3 | Opened SecureChannel with SecurityPolicy http://opcfoundation.org/UA/SecurityPolicy#None"
4: qt.opcua.plugins.open62541.sdk.client: "Client Status: ChannelState: Open, SessionState: Closed, ConnectStatus: Good"
4: qt.opcua.plugins.open62541.sdk.client: "Selected Endpoint opc.tcp://localhost:43344/ with SecurityMode None and SecurityPolicy http://opcfoundation.org/UA/SecurityPolicy#None"
4: qt.opcua.plugins.open62541.sdk.client: "Selected UserTokenPolicy open62541-anonymous-policy with UserTokenType Anonymous and SecurityPolicy http://opcfoundation.org/UA/SecurityPolicy#None"
4: qt.opcua.plugins.open62541.sdk.client: "Client Status: ChannelState: Open, SessionState: Created, ConnectStatus: Good"
4: qt.opcua.plugins.open62541.sdk.client: "Client Status: ChannelState: Open, SessionState: Activated, ConnectStatus: Good"
4: ********* Start testing of opcua *********
4: Config: Using QtTest library 6.4.0, Qt 6.4.0 (arm64-little_endian-lp64 shared (dynamic) release build; by Clang 13.0.0 (clang-1300.0.29.30) (Apple)), macos 11.6
4: PASS   : opcua::AbsoluteNodeTest: open62541: Create String Node Id::initTestCase()
4: PASS   : opcua::AbsoluteNodeTest: open62541: Create GUID Node Id::initTestCase()
4: PASS   : opcua::AbsoluteNodeTest: open62541: Create GUID Node Id::test_nodeTest()
4: PASS   : opcua::AbsoluteNodeTest: open62541: Create Opaque Node Id::initTestCase()
4: PASS   : opcua::AbsoluteNodeTest: open62541: Create Opaque Node Id::test_nodeTest()
4: PASS   : opcua::AbsoluteNodeTest: open62541: Create Integer Node Id::initTestCase()
4: PASS   : opcua::AbsoluteNodeTest: open62541: Create Integer Node Id::test_nodeTest()
4: PASS   : opcua::AbsoluteNodeTest: open62541: Write Value Node with Multiple Listeners::initTestCase()
4: PASS   : opcua::AbsoluteNodeTest: open62541: Emitting signals on node changes::initTestCase()
4: PASS   : opcua::AbsoluteNodeTest: open62541: Node with namespace in Id::initTestCase()
4: PASS   : opcua::AbsoluteNodeTest: open62541: Node with namespace in Id::test_nodeTest()
4: PASS   : opcua::AbsoluteNodeTest: open62541: Standard attributes on variable node::initTestCase()
4: PASS   : opcua::AbsoluteNodeTest: open62541: Node timestamps::initTestCase()
4: PASS   : opcua::AbsoluteNodeTest: open62541: Node timestamps::test_nodeTest()
4: PASS   : opcua::AbsoluteNodeTest: open62541: Node timestamps::cleanupTestCase()
4: PASS   : opcua::AbsoluteNodeTest: open62541: Assign int value to double node::initTestCase()
4: FAIL!  : opcua::AbsoluteNodeTest: open62541: Assign int value to double node::test_nodeTest() Compared values are not the same
4:    Actual   (): 2
4:    Expected (): 1
4:    Loc: [/Users/qt/work/qt/qtopcua/tests/auto/declarative/AbsoluteNodeTest.qml(440)]
4: PASS   : opcua::AbsoluteNodeTest: open62541: Assign int value to double node::cleanupTestCase()
4: PASS   : opcua::UnknownTestFunc()
[...]

      This subtest prints FAIL then PASS. At the end execution, qt-testrunner reports that exit code was 1 (correctly) but the XML did not contain any failure:

      4: qt-testrunner.py     INFO: Test process exited with code: 1
4: qt-testrunner.py     INFO: Parsed XML file /Users/qt/work/testresults/tst_opcua-1651870032738.xml in 0.003 seconds
4: qt-testrunner.py     INFO: Found 36 passes and 0 failures
4: qt-testrunner.py  WARNING: No failures listed in the XML test log! Did the test CRASH right after all its testcases PASSed?
4: qt-testrunner.py     INFO: Will re-run the full test executable

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            macadder Jason McDonald
            jimis Dimitrios Apostolou
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:

                There are no open Gerrit changes