Details
-
Bug
-
Resolution: Unresolved
-
P3: Somewhat important
-
None
-
5.9.4
-
None
-
Windows 1O Pro (1709)
QT 5.9.4
Description
We have an application that uses the QNetworkAccessManager to submit requests to a SOAP webservice, ie does an HTTP post of an XML request. We have occasionally seen that multiple submissions of these requests from the QT HTTP networking stack, even though only one submission was made from the application. These repeated submissions cause application problems for us, if the initial error was propagated to the client, we could decide whether or not to resubmit the request ( for example querying the backend before deciding to resubmit ).
This can be fairy easily reproduced, I will attach a test case that does 1000 POST's to a simple CGI script, which returns a fixed payload.
The setup to reproduce this is as follows:
QT Client Prog(HTTP POST * 1000) --> HAProxy Load Balancer --> Apache with CGI Page
By simply restarting the HAProxy LB every few seconds during the test, this causes the re-submissions to occur. Another method is to simply introduce a network with a small degree of packet loss between the client and server.
The requests are setup to POST urls of the form:
POST /cgi-bin/qt-test.cgi?a=FIXEDTIMESTAMP-REQUESTINDEX
This allows easy identification of duplicate requests.
The apache log looks like this:
127.0.0.1 - - [19/Mar/2018:13:20:19 +0000] "POST /cgi-bin/qt-test.cgi?a=1521465603-996 HTTP/1.1" 200 24768 "-" "Mozilla/5.0"
127.0.0.1 - - [19/Mar/2018:13:20:19 +0000] "POST /cgi-bin/qt-test.cgi?a=1521465603-997 HTTP/1.1" 200 24768 "-" "Mozilla/5.0"
127.0.0.1 - - [19/Mar/2018:13:20:19 +0000] "POST /cgi-bin/qt-test.cgi?a=1521465603-998 HTTP/1.1" 200 24768 "-" "Mozilla/5.0"
127.0.0.1 - - [19/Mar/2018:13:20:19 +0000] "POST /cgi-bin/qt-test.cgi?a=1521465603-999 HTTP/1.1" 200 24768 "-" "Mozilla/5.0"
127.0.0.1 - - [19/Mar/2018:13:20:19 +0000] "POST /cgi-bin/qt-test.cgi?a=1521465603-1000 HTTP/1.1" 200 24768 "-" "Mozilla/5.0"
Duplicates can be found like this:
[root@qt-http-test httpd]# grep 1521465603 access_log | awk -F= '{print $2}' | sort | wc -l
1005
[root@qt-http-test httpd]# grep 1521465603 access_log | awk -F= '{print $2}' | sort | uniq -c | grep -v " 1"
2 1521465603-250 HTTP/1.1" 200 24768 "-" "Mozilla/5.0"
2 1521465603-438 HTTP/1.1" 200 24768 "-" "Mozilla/5.0"
2 1521465603-44 HTTP/1.1" 200 24768 "-" "Mozilla/5.0"
2 1521465603-639 HTTP/1.1" 200 24768 "-" "Mozilla/5.0"
2 1521465603-849 HTTP/1.1" 200 24768 "-" "Mozilla/5.0"
[root@qt-http-test httpd]# grep 1521465603-849 access_log
127.0.0.1 - - [19/Mar/2018:13:20:16 +0000] "POST /cgi-bin/qt-test.cgi?a=1521465603-849 HTTP/1.1" 200 24768 "-" "Mozilla/5.0"
127.0.0.1 - - [19/Mar/2018:13:20:17 +0000] "POST /cgi-bin/qt-test.cgi?a=1521465603-849 HTTP/1.1" 200 24768 "-" "Mozilla/5.0"
I believe these re-submissions are coming from logic in qhttpnetworkconnectionchannel.cpp
https://github.com/qt/qtbase/blob/5.9/src/network/access/qhttpnetworkconnectionchannel.cpp#LC72
It looks to me as if this behavior has been added by design, however I can't find it documented in the Public APIs anywhere.
From my reading of the code, no attempt is made to distinguish between "safe" or "idempotent" methods as defined in the HTTP RFC : https://tools.ietf.org/html/rfc7231#section-4.2 or other methods, so this re-submission could have un-intended consequences, as in our case ( think of double booking or double payment scenarios )
1) Is my understanding correct?
2) Could this be documented somewhere - we spent a long time identifying this behaviour?
3) Could this behavior be made configurable in QNAM, with the current behavior as the default, allowing clients to programatically decide if they wanted silent re-submissions?
Our current workaround is patch the value of reconnectAttemptsDefault to 0 in qhttpnetworkconnectionchannel.cpp, but we would prefer not to have to patch QT.