Uploaded image for project: 'Qt Quality Assurance Infrastructure'
  1. Qt Quality Assurance Infrastructure
  2. QTQAINFRA-5138 Implement testresult parser
  3. QTQAINFRA-5187

Some tests having multiple PASS results even over 300 times - Should they have more unique test_names

    XMLWordPrintable

Details

    • Technical task
    • Resolution: Fixed
    • P2: Important
    • None
    • None
    • None

    Description

      Needs to verify what makes Testcase names unique

      • testname = test_executable + test_function + test_datatag
      • sometimes there isn't test datatag because it's only in use if there's many variations of the same test

       

      Fond out there's same test with same DataTag tested twice or even over 300 times - even though test is PASS in first run.

      This causes some obstacles for XML parsing and sending results to DB

      Obstacles:

      • Retesting
        • Not sure which of those are retested?
        • Or how many times retested?
      • Testname uniqueness
        • Might need new name addition to make testnames unique
        • Not yet sure which solution works the best - tests not yet that familiar

       

       

      Some tests are tested 56, 361 or 625 times - Could need more unique testnames to separate them

       

      This 625 is having same DataTag content in every test run (XML file attached)

      grep "[CDATA[exhaustive]]" ./temp_folder/tst_qquaternion-1664035244811.xml | wc -l
      625

       

      Possible fix:

      • If "exhaustive" would be made more unique like in other tests, could fix this issue
      • Test to verify this in future: New test verification to check that DataTags are unique

      Attachments

        1. image-2022-09-26-14-21-49-603.png
          161 kB
          Tero Heikkinen
        2. image-2022-09-26-14-22-26-909.png
          134 kB
          Tero Heikkinen
        3. image-2022-09-26-15-31-32-936.png
          107 kB
          Tero Heikkinen
        4. tst_qquaternion-1664035244811.xml
          95 kB
          Tero Heikkinen

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              teheikki Tero Heikkinen
              teheikki Tero Heikkinen
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes