docs: audit links, glossary T0-T4, readability pass
CI & Release / Verify simulator (push) Skipped
CI & Release / Trivy scan (push) Skipped
CI & Release / Semantic Release (push) Skipped
CI & Release / Verify simulator (pull_request) Failing after 4s
CI & Release / Trivy scan (pull_request) Skipped
CI & Release / Semantic Release (pull_request) Skipped

Extend glossary with storage floors, source of truth, and linkable anchors.
Fix broken links, pose frame drift, rsync/offload wording, and README tone.
Add docs/improvement audit records and live demo links in design journey.
This commit is contained in:
2026-07-10 10:46:59 +01:00
parent ed531f9fb6
commit 5f6fd8462c
27 changed files with 376 additions and 193 deletions
+14 -15
View File
@@ -1,8 +1,8 @@
# Virtual drone fleet
A data generator that exercises the **exact on-board pipeline** described in [03 — Data platform](../docs/03-data-platform.md): seeded kinematics along a patrol route, sensor streams at realistic rates, detection events, the hot `current/` → sealed Parquet write path, and the compact UDP state broadcast between drones.
Data generator. Same on-board pipeline as [03 — Data platform](../docs/03-data-platform.md): seeded kinematics, sensor streams, detection events, hot `current/` → sealed Parquet, compact UDP state broadcast.
One process = one drone. Scaling the swarm is a Compose flag.
One process = one drone. Scale swarm with Compose flag.
## Run a swarm
@@ -14,7 +14,7 @@ FLIGHT_ID=$(date -u +%Y%m%dT%H%MZ)-sim docker compose up --build --scale drone=5
DRONE_COUNT=10 DURATION_S=300 SPEEDUP=4 docker compose up --build --scale drone=10
```
Output lands in [`../var/t1/`](../var/t1/) (the shared T1 lake — same path the k3d fleet and warehouse offload use). Override with `SWARM_T1_DIR` if needed:
Output lands in [`../var/t1/`](../var/t1/) ([T1](../docs/00-glossary.md#t1--warm) live lake — same path k3d fleet and warehouse offload use). Override with `SWARM_T1_DIR` if needed:
```
var/t1/dataset=telemetry/flight=…/drone=…/sensor=imu/year=…/…/hour=…/data.parquet
@@ -22,7 +22,7 @@ var/t1/dataset=detections/flight=…/drone=…/…
var/t1/dataset=state/flight=…/drone=…/… ← sent + received broadcasts
```
Historical flights after offload live under [`../var/t3/`](../var/t3/) (T3 warehouse).
Historical flights after offload: [`../var/t3/`](../var/t3/) ([T3](../docs/00-glossary.md#t3--warehouse) warehouse).
## Run a single drone without Docker
@@ -52,33 +52,32 @@ print(con.sql("""
EOF
```
## Monitoring: Grafana over the generated data
## Monitoring
The `monitoring` profile spins up a small metrics chain — a DuckDB-based exporter that scans the generated Parquet every few seconds, Prometheus, and a pre-provisioned Grafana dashboard:
`monitoring` profile: DuckDB exporter → Prometheus → Grafana ([07 — Observability](../docs/07-observability.md)):
```bash
docker compose --profile monitoring up -d # exporter + prometheus + grafana
# generate some flights in parallel or beforehand:
docker compose --profile monitoring up -d
FLIGHT_ID=$(date -u +%Y%m%dT%H%MZ)-sim docker compose up --scale drone=5
```
- Grafana: `http://localhost:3000` (anonymous admin — demo only) → dashboard **Swarm Fleet — generated data**
- Grafana: `http://localhost:3000` **Swarm Fleet — generated data**
- Prometheus: `http://localhost:9090` · exporter: `http://localhost:9105/metrics`
Panels: telemetry rows by drone/sensor, detections by class, pose frames sent/received, battery per drone, RSSI and estimated distance per link, Parquet bytes/files on disk. The exporter is deliberately a demonstration of the observability doctrine from [07 — Observability](../docs/07-observability.md): fleet statistics are *derived from the data platform itself* — no agent on the drone, just SQL over the same Parquet everyone else reads.
Fleet stats derived from Parquet via SQL — no on-drone agent.
## Data-plane explorer
The `monitoring` profile also starts a **data-plane explorer** at `http://localhost:8088` — a single-page web view over the lake itself, complementing Grafana (which shows aggregates, not the structure):
`monitoring` profile also starts explorer at `http://localhost:8088` (k3d: `:30088`):
- **Partition tree**, live: `dataset → flight → drone → sensor → year/…/hour → file`, with file counts and bytes rolled up at every level. You can watch `current/` files appear and get sealed while a swarm is flying.
- **Read-only SQL console** with the datasets pre-registered as views (`telemetry`, `detections`, `state`) and one-click sample queries (fleet overview, battery timeline, detections by class, peer link quality, schema).
- Partition tree live: `dataset → flight → drone → sensor → hour → file`
- Read-only SQL console — same [statement gate](../docs/04-swarm-sync.md) as peer query channel: `SELECT`/`WITH`/`DESCRIBE`/`SUMMARIZE`/`SHOW` only
The console enforces the **same statement gate as the peer query channel** from [04 — Swarm sync](../docs/04-swarm-sync.md): a single statement, `SELECT`/`WITH`/`DESCRIBE`/`SUMMARIZE`/`SHOW` only, write/config/extension keywords rejected. That is the point: exploring the data plane on the bench uses the same read-only SQL contract a peer drone uses in flight — the explorer is the "no debug API" principle made visible.
Point: bench exploration uses same read-only SQL contract as flight peers.
## Reproducibility
Every run is deterministic per `(SEED, DRONE_ID)`: same route jitter, same sensor noise, same detection sequence. A bug report is a seed and a config, not a description.
Deterministic per `(SEED, DRONE_ID)`. Bug report = seed + config.
## Knobs