perf: bound lake scans and add platform Grafana dashboard
CI & Release / Verify simulator (push) Successful in 11s
CI & Release / Trivy scan (push) Successful in 9s
CI & Release / Semantic Release (push) Successful in 5s

Explorer and exporter were scanning the full Parquet lake every 1–5s
(~2000 files, 200MB+), driving ~2.2 CPU cores. Limit metrics to the
last N flights, cache DuckDB views and the partition tree, slow live
polls to 2s, and keep drones alive after seal to stop restart churn.

Add node-exporter, scan-duration metrics, and a Swarm Platform Grafana
dashboard for node CPU/memory and scan health.
This commit is contained in:
2026-07-08 21:19:53 +01:00
parent bcd956d11a
commit ea062af86a
11 changed files with 478 additions and 94 deletions
+5
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import os
import random
import select
import time
@@ -108,6 +109,10 @@ def run(cfg: Config) -> None:
writer.seal()
print(f"[{cfg.drone_id}] done: {frames_sent} state frames sent, "
f"{len(peers_seen)} peers seen {sorted(peers_seen)}; sealed to {root}")
if os.environ.get("KEEP_ALIVE", "0") == "1":
print(f"[{cfg.drone_id}] KEEP_ALIVE=1 — idle after seal (no pod restart churn)")
while True:
time.sleep(3600)
if __name__ == "__main__":