Implement centralised log aggregation and indexing, most likely using the ELK stack (Elasticsearch, Logstash, Kibana).
- collect logs from journald logs (journalbeat)
- collect logs from specified log files (filebeat)
- Windows: C:\Windows\Logs\DISM\dism.log
- collect event logs on windows (winlogbeat)
When a test crashes for unexplained reasons, or a VM misbehaves, we need to investigate what could have possibly gone wrong in the environment. To do this we often need access to kernel and system logs and all other traces we can get, but the VM is already dead (TM), and the host machines are not accessible to all developers (for good reasons). Things like I/O errors, segfaults, memory corruption, kernel stacktraces etc.