Details
-
Task
-
Resolution: Fixed
-
P1: Critical
-
None
-
None
-
None
Description
Right now we have too much of old (older than 2 years) data on influxdb on test results server. This causes slowdown and increases memory usage. We need to archive old data and remove it from influx
affects coin database on influx.
measurement: blacklisted_test, 2022 data start: 2022-01-01T00:32:24.593909Z
measurement: crashed_test, 2022 data start: 2022-01-01T17:04:11.003882Z
measurement: failed_test, 2022 data start: 2022-01-01T01:13:38.188538Z
measurement: flaky_test, 2022 data start: 2022-01-01T00:32:24.593909Z
measurement: integration, 2022 data start: 2022-01-01T14:32:05Z
measurement: workitem, 2022 data start: 2022-01-01T00:12:43Z
Right now we do not have retention policies:
> show retention policies on coin
name duration shardGroupDuration replicaN default
---- -------- ------------------ -------- -------
autogen 0s 168h0m0s 1 true
>
see python notebook (displayed only in firefox at my machine):
https://git.qt.io/qtqa/notebooks/-/blob/main/old_data/2022_old_data.ipynb?ref_type=heads
Should we just drop older data? Afik nobody was ever checking the archive data we store.