perf: bound lake scans and add platform Grafana dashboard
CI & Release / Verify simulator (push) Successful in 11s
CI & Release / Trivy scan (push) Successful in 9s
CI & Release / Semantic Release (push) Successful in 5s

Explorer and exporter were scanning the full Parquet lake every 1–5s
(~2000 files, 200MB+), driving ~2.2 CPU cores. Limit metrics to the
last N flights, cache DuckDB views and the partition tree, slow live
polls to 2s, and keep drones alive after seal to stop restart churn.

Add node-exporter, scan-duration metrics, and a Swarm Platform Grafana
dashboard for node CPU/memory and scan health.
This commit is contained in:
2026-07-08 21:19:53 +01:00
parent bcd956d11a
commit ea062af86a
11 changed files with 478 additions and 94 deletions
+5 -1
View File
@@ -24,7 +24,8 @@ services:
command: ["python", "monitoring/exporter.py"]
environment:
DATA_DIR: /data
SCAN_INTERVAL_S: "5"
SCAN_INTERVAL_S: "30"
METRIC_FLIGHT_WINDOW: "5"
volumes:
- ${SWARM_T1_DIR:-../var/t1}:/data:ro
- ./monitoring:/app/monitoring:ro
@@ -37,6 +38,9 @@ services:
command: ["python", "explorer/server.py"]
environment:
DATA_DIR: /data
VIEW_REFRESH_S: "30"
TREE_CACHE_S: "15"
METRIC_FLIGHT_WINDOW: "5"
volumes:
- ${SWARM_T1_DIR:-../var/t1}:/data:ro
- ./explorer:/app/explorer:ro