eSlider
ea062af86a
Explorer and exporter were scanning the full Parquet lake every 1–5s (~2000 files, 200MB+), driving ~2.2 CPU cores. Limit metrics to the last N flights, cache DuckDB views and the partition tree, slow live polls to 2s, and keep drones alive after seal to stop restart churn. Add node-exporter, scan-duration metrics, and a Swarm Platform Grafana dashboard for node CPU/memory and scan health.
Terraform — simulation environment (T4)
Declares the simulated fleet and its observability stack on the k3d cluster
created by ../../ansible/sim-cluster.yml.
This module is the executable miniature of the dev/sim environment from
06 — Environments.
| Resource | Purpose |
|---|---|
StatefulSet drone × drone_count |
Virtual drones; stable pod names become DRONE_IDs; all write Hive-partitioned Parquet into the shared /data lake |
Deployment exporter |
Prometheus exporter reading the lake with DuckDB |
Deployment explorer + NodePort 30088 |
Partition tree + read-only SQL console (also feeds the prototype's live mode) |
Deployment prometheus + NodePort 30990 |
Scrapes the exporter |
Deployment grafana + NodePort 30300 |
Same dashboard JSON as the Compose profile, provisioned from a ConfigMap |
Usage
ansible-playbook ../../ansible/sim-cluster.yml # once: cluster + image
terraform init
terraform apply
terraform output # URLs
Scale the fleet without touching YAML:
terraform apply -var drone_count=9 -var speedup=4
Notes
image_pull_policy = "Never"— the simulator image is imported by the Ansible playbook (k3d image import); the cluster never pulls from a registry, mirroring the air-gap doctrine.- Pods discover their identity from the pod name (StatefulSet ordinal), so
a flight survives
kubectl delete podthe same way a drone survives a power cycle: new process, same identity, new flight ID. - State is local (
terraform.tfstatein this directory) — the simulation cluster is disposable; nothing here is shared infrastructure.