Implement centralised log aggregation for all hosts/VMs in Coin and OpenNebula

Implement centralised log aggregation and indexing, most likely using the ELK stack (Elasticsearch, Logstash, Kibana).

Requirements

• collect logs from journald logs (journalbeat)
• collect logs from specified log files (filebeat)
  • Windows: C:\Windows\Logs\DISM\dism.log
• collect event logs on windows (winlogbeat)

Why do I need access to system logs as a developer

When a test crashes for unexplained reasons, or a VM misbehaves, we need to investigate what could have possibly gone wrong in the environment. To do this we often need access to kernel and system logs and all other traces we can get, but the VM is already dead (TM), and the host machines are not accessible to all developers (for good reasons). Things like I/O errors, segfaults, memory corruption, kernel stacktraces etc.