Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-27522

Downloading a big file from QNetworkDiskCache can be very slow and CPU-intensive

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: P2: Important
    • Resolution: Done
    • Affects Version/s: 5.0.0
    • Fix Version/s: 5.0.0
    • Component/s: Network: Cache
    • Labels:
      None
    • Commits:
      8fb379dc8a4d7069a99c3181283e581b86e99ceb

      Description

      When developing for a QtWebKit2 -based browser I noticed that a file download can take 100% CPU for many seconds. This happens when the file is downloaded from QNetworkDiskCache and isn't fully read in a directly connected slot to QNetworkReply::readyRead() signal. I attached a test app demonstrating this code path (that is hit in QtWebKit2) as "slowcachedownload.tar.gz".

      The problem comes down to this: the file is read into a QByteArray which is appended into QNetworkReplyHttpImplPrivate::downloadMultiBuffer. From there it is read via QIODevice::readAll() 16kB at a time, and the remaining data is copied to the start of the array each time.

      I made a fix that eliminates unnecessary copying by adding a read position variable to QByteDataBuffer. A fix and a unit test that includes benchmarks is attached as "qbytedatabuffer.tar.gz". The fix adds very little overhead to the best case scenario and improves the worst case scenario I encountered with a ~30MB file from 10s user time to practically 0s.

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

            • Assignee:
              aha Antti Harju
              Reporter:
              aha Antti Harju
            • Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Gerrit Reviews

                There are no open Gerrit changes