Details
-
Bug
-
Resolution: Done
-
P1: Critical
-
None
-
master, production
-
None
Description
Agent logs all messages in parallel to a file and to a remote TCP connection. If everything goes fine then it works nicely, but logging, conceptually, is very sensitive to transport layer problems. For example, it is hard to write a message to a file if the disk is full and there is no way to inform a user about that problem if the file is the only interface the user is listening to. That same applies to the remote logger. If there is no network, then we can not send any messages. In addition network usually is slower and more susceptible to failures. By handling file log and remote log in parallel we may suffer from all issues in the same time. That is why we need crazy log buffering, re-try and post network error log messages mechanizes. I think it could be simplified if we agree to send logs from agent always only to a file, we can ensure the disk space so it is quite safe option. Then on top of it we would span a goroutine that would read that file and use remote logger to upload it to coin.
Alternative:
Increase log buffer.
Attachments
Issue Links
- relates to
-
COIN-45 Creation of qmake fails for qt/qtbase integrations during running configure
- Closed
-
QTQAINFRA-1974 Compilation times are up to 100% slower with Coin agents than manually run builds
- Closed
-
QTQAINFRA-2382 VM ignores Coin being unresponsive and thinks it finishes successfully
- Closed
-
QTQAINFRA-2050 Compressed file ended before the end-of-stream marker was reached
- Closed
-
COIN-160 Logging improvements
- Closed
- replaces
-
QTQAINFRA-1909 Invalid log links in builds restarted with heartbeat
- Closed
For Gerrit Dashboard: QTQAINFRA-2192 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
246718,13 | Improve reliability of agent log streaming | master | qtqa/tqtc-coin-ci | Status: ABANDONED | -2 | 0 |
247156,3 | Buffer the log writing | master | qtqa/tqtc-coin-ci | Status: ABANDONED | -2 | 0 |
247553,6 | Fix scheduler connection error handling | master | qtqa/tqtc-coin-ci | Status: MERGED | +2 | 0 |