Priority: Not Evaluated
Affects Version/s: 5.12.2
Fix Version/s: None
Component/s: Extras: X11
Here you can find minimal reproducible example. It tries to download just one file - http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-all-3gram-20120701-an.gz.
I measure memory consumption via two tools: standard KDE's System Monitor, and google's tcmalloc heapprofile. More precisely, I link against tcmalloc, and then run the binary from the example as follows: $ HEAPPROFILE=mybin.hprof ./google_ngram_qt_bug.
I also used strace to figure out what is going on.
So, heapprofile reports two things:
- Lines in logs like "Dumping heap profile to mybin.hprof.0361.heap (36215 MB currently in use)".
- and, more importantly, mybin.hprof.0361.heap files that can later be visualised (see the attached screenshot). In particular, on the screenshot you can see how some mysterious (non-main) thread (QTcpServer -> QAbstractSocket -> <signal-slot> -> QHttpPart) in the background downloaded 36GB.
Apart from the tool, I have a very useful cout in my app itself:
which gives the following output (slightly redacted):
So, code in the main
- request 16K,
- but in response to get 16K, Qt got 500MB.
strace shows huge amount of read traffic from socket 8 in thread id 4418:
Now if we grep for the thread id we will get:
$ ps -e -T | grep 4418
4415 4418 pts/1 00:00:20 QNetworkAccessM
strace reported that file descriptor #8 was used for reading. let's have a look at what it is associated with:
8u IPv4 fb4c8169a8dd:38282->lhr35s07-in-f16.1e100.net:http
I presume that someone decided that it would be perfectly fine to implicitly download a page in the background in a different thread. It also means that:
- QIODevice (from the point of view of end-user) does not behave like a stream - it eagerly reads everything in memory. (while I understand that it is not QIODevice that does it, but rather a different component in a different thread).
- setReadBufferSize() is completely broken.
So, since QIOStream/setReadBufferSize are not working, is there a way to disable eager and implicit downloading in the background?