Details
-
Suggestion
-
Resolution: Unresolved
-
P2: Important
-
None
-
None
Description
The API is designed to provide zero-copy access to the data of a QIODevice.
The current read() and readAll() calls all do a memory copy instead.
For implementing:
virtual QByteArray QIODevice::readAnyData();
For using:
QByteArray QIODevice::readAny();
A readAny() call would avoid a memory copy: It could return a QByteArray of an arbitrary size
from the data it has stored anyway "inside".
Depending on the specific QIODevice implementation different things could be done:
QTcpSocket:
This class is getting notifications from the OS for new data. The new data is roughly in ~MTU
sized chunks. Instead of the user getting a copy of the data, he would get the
data itself, so only one copy from OS to user.
QSslSocket:
Similar to QTcpSocket, although the decryption process is obviously a copy.
QNetworkReply:
There are two cases here. For the case where the server sends a content-length,
the zero-copy QNetworkReply would kick in. This allocates a big buffer where
all the network reply data is stored in. A readAny() there returns a QByteArray
created with QByteArray::fromRawData() which represents a part of the network reply
that is already downloaded.
For servers not sending a content-length (e.g. that stream data of unknown length)
the QNetworkReply behaves very similar to the QTcpSocket/QSslSocket.
With clever implementation only one copy from OS to user happens.
QIODevice returned by a QAbstractNetworkCache:
Depending on the network cache implementation various things are imaginable,
e.g. the usage of memory mapped files or for an in-memory cache.
QIODevice given to a QNetworkAccessManager for POST/PUT:
Let's take QtWebKit with the POSTing of an HTML form as an example.
QNetworkAccessManager reads from QtWebKit's QIODevice (=copy) and then
at some point writes that data to the OS. This copy could be avoided by
QtWebKit's QIODevice returning QByteArrays for its form elements.
QFile:
Who knows, on some OSes or with some file systems it might make sense to
have QFile always memory-map stuff and then readAny provides the access to that
buffer similar to the QNetworkReply zero-copy.
If memory-mapping of the file doesn't make sense, we could read an "optimal" buffer amount to read from the OS.
QBuffer:
If at the start position, readAny() is basically readAll().
If in the middle of the buffer, a slice of the QByteArray is returned via QByteArray::fromRawData().
Any buffered QIODevice:
If there is data in the QIODevice's internal buffer, return that.
Default implementation:
We should find something nice for that.
Attachments
Issue Links
- relates to
-
QTBUG-76492 [Spike]: Research options for a better QIODevice abstraction
- Open
- resulted from
-
QTBUG-6705 Bring "zero-copy" to QIODevice::read
- Closed
Gerrit Reviews
For Gerrit Dashboard: QTBUG-19169 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
149930,3 | QIODevice: allow zero-copy in read() | dev | qt/qtbase | Status: MERGED | +2 | 0 |