Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-15111

Un-blacklist a big number of network CI auto test

    XMLWordPrintable

Details

    Description

      We have a huge number of tests in tst_qnetworkreply and tst_q*socket.
      However I know that a lot of them is blacklisted, in case of QNetworkReply the whole class auto test is blacklisted.

      The network auto tests depend on the qt-test-server to be available which is often a problem (network connection flackiness for some reason, disk full, high load, ?, ...).
      Some of the network auto test also depend on external servers to be available.

      However a lot of those tests actually use their own integrated 127.0.0.1 server.
      Those then should be un-blacklisted.

      This required some work, e.g. un-blacklist QNetworkReply auto test and re-blacklist all the tests inside it that use the qt-test-server or another local resource.
      Then get this tested via Pulse/CI so we have the results for all OSes. Then update the blacklist (coordinate this with QA).

      Apparantly there is a script called blacklist-hitman to show all the blacklisted tests and how many times they've failed and when the most recent failure was.

      You can view the blacklisted auto tests and their current number of failures on:
      http://gitestr.nokia.troll.no/blacklist_qt_47.html
      http://gitestr.nokia.troll.no/blacklist_qt_master.html

      A good idea for the first steps:

      1. Make a test list of all QNetworkReply test: (cd tests/auto/qnetworkreply/ ; make && ./tst_qnetworkreply -functions | cut -d '(' -f 1)
      2. Delete the ones from the list that don't depend on external server (internet or qt-test-server) but use an included webserver
      3. Get QA to un-blacklist the whole QNetworkReply and re-blacklist the ones from your list

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              jasmcdon Jason McDonald (Closed Nokia Identity. Please assign to "macadder" instead) (Inactive)
              mgoetz Markus Goetz (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes