Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-81336

QNetworkAccessManager::_q_replyFinished is very slow with huge amount of network requests

    XMLWordPrintable

Details

    • 6faaef5a665f86465d616f20b5effd2ba650cee7 (qt/qtbase/5.15)

    Description

      _q_replyFinished() calls QObject::sender(), which locks a mutex and scans a linked list of connections. This takes a very long time with tens of thousands of network requests. Attached project can demonstrate the issue along with Nginx server on running on localhost with multiple ports open. For example this configuration:

      server {
      	listen 8080;
      	listen 8081;
      	listen 8082;
      	listen 8083;
      	listen 8084;
      	listen 8085;
      	listen 8086;
      	listen 8087;
      	listen 8088;
      	listen 8089;
      	listen 8090;
      	listen 8091;
      	listen 8092;
      	listen 8093;
      	listen 8094;
      	listen 8095;
      	root /var/www/html;
      	server_name _;
      	location / {
      		try_files $uri $uri/ =404;
      	}
      }
      

      With 20k request and Qt 5.14.0 on my laptop I see this kind of output:

      requests made
      0.875657 per sec. 19999 left. 1142 ms
      325.349 per sec. 19673 left. 2145 ms
      392.216 per sec. 19280 left. 3147 ms
      418.164 per sec. 18861 left. 4149 ms
      423.729 per sec. 18436 left. 5153 ms
      354.645 per sec. 18081 left. 6155 ms
      506.494 per sec. 17574 left. 7156 ms
      274.725 per sec. 17299 left. 8158 ms
      558.442 per sec. 16740 left. 9160 ms
      343.656 per sec. 16396 left. 10161 ms
      351.648 per sec. 16044 left. 11162 ms
      639.361 per sec. 15404 left. 12164 ms
      265.672 per sec. 15137 left. 13169 ms
      740.26 per sec. 14396 left. 14170 ms
      483.146 per sec. 13880 left. 15239 ms
      788.212 per sec. 13091 left. 16241 ms
      514.486 per sec. 12576 left. 17242 ms
      1000 per sec. 11574 left. 18244 ms
      824.176 per sec. 10749 left. 19246 ms
      754.246 per sec. 9994 left. 20247 ms
      1143.48 per sec. 8679 left. 21398 ms
      1137.75 per sec. 7374 left. 22546 ms
      1942.06 per sec. 5430 left. 23547 ms
      2945.05 per sec. 2482 left. 24548 ms
      done in 24979 ms
      

      Speed clearly increases towards the end when the replies get destroyed. And in Callgrind sender() function is the highest, followed by calls related to old style connect().

      Attachments

        1. callgrind.png
          callgrind.png
          487 kB
        2. fix_qnam_many_connections.diff
          3 kB
        3. qtbug81336.tar.gz
          0.9 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            manordheim MÃ¥rten Nordheim
            poikelin Joni Poikelin
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes