Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-116772

qtestlib shouldn't allow the same test+testdata to be invoked multiple times in one run

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: P3: Somewhat important P3: Somewhat important
    • None
    • dev
    • Testing: qtestlib
    • None
    • 8
    • Foundation PM Staging

      From QTBUG-116771 I notice that qmltestrunner invokes the same test function again and again. By reading the XML logs one can't see a difference. QTest should exit with error, or at least warn.

      Attaching tst_multiTestCase.qml. Example logs copied from this log:

      agent:2023/08/27 23:26:26 build.go:405: 107: PASS   : qmltestrunner::A::test_do()
agent:2023/08/27 23:26:26 build.go:405: 107: PASS   : qmltestrunner::A::cleanupTestCase()
agent:2023/08/27 23:26:26 build.go:405: 107: PASS   : qmltestrunner::A::initTestCase()
agent:2023/08/27 23:26:26 build.go:405: 107: PASS   : qmltestrunner::A::test_do()
agent:2023/08/27 23:26:26 build.go:405: 107: PASS   : qmltestrunner::A::cleanupTestCase()
agent:2023/08/27 23:26:26 build.go:405: 107: PASS   : qmltestrunner::A::initTestCase()
agent:2023/08/27 23:26:26 build.go:405: 107: PASS   : qmltestrunner::A::test_do()
agent:2023/08/27 23:26:26 build.go:405: 107: PASS   : qmltestrunner::A::cleanupTestCase() 
[ ... seemingly same test runs 100 times ... ]

      Here is part of the XML file generated by qtestlib:

      <?xml version="1.0" encoding="UTF-8"?>
<TestCase name="qmltestrunner">
  <Environment>
    <QtVersion>6.7.0</QtVersion>
    <QtBuild>Qt 6.7.0 (x86_64&#x002D;little_endian&#x002D;lp64 shared (dynamic) release build; by GCC 11.4.0)</QtBuild>
    <QTestVersion>6.7.0</QTestVersion>
  </Environment>
[...]
  <TestFunction name="A::initTestCase">
    <Incident type="pass" file="" line="0" />
    <Duration msecs="0.16281"/>
  </TestFunction>
  <TestFunction name="A::test_do">
    <Incident type="pass" file="" line="0" />
    <Duration msecs="1.27274"/>
  </TestFunction>
  <TestFunction name="A::cleanupTestCase">
    <Incident type="pass" file="" line="0" />
    <Duration msecs="0.060903"/>
  </TestFunction>
  <TestFunction name="A::initTestCase">
    <Incident type="pass" file="" line="0" />
    <Duration msecs="0.1256"/>
  </TestFunction>
  <TestFunction name="A::test_do">
    <Incident type="pass" file="" line="0" />
    <Duration msecs="1.65724"/>
  </TestFunction>
  <TestFunction name="A::cleanupTestCase">
    <Incident type="pass" file="" line="0" />
    <Duration msecs="0.053843"/>
  </TestFunction>
  <TestFunction name="A::initTestCase">
    <Incident type="pass" file="" line="0" />
    <Duration msecs="0.064591"/>
  </TestFunction>
[...]
  <Duration msecs="3253.55"/>
</TestCase>

        1. tst_multiTestCase.qml
          3 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Eddy Edward Welbourne
            jimis Dimitrios Apostolou
            Vladimir Minenko Vladimir Minenko
            Alex Blasche Alex Blasche
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:

                There are no open Gerrit changes