eSlider
5f6fd8462c
CI & Release / Verify simulator (push) Skipped
CI & Release / Trivy scan (push) Skipped
CI & Release / Semantic Release (push) Skipped
CI & Release / Verify simulator (pull_request) Failing after 4s
CI & Release / Trivy scan (pull_request) Skipped
CI & Release / Semantic Release (pull_request) Skipped
Extend glossary with storage floors, source of truth, and linkable anchors. Fix broken links, pose frame drift, rsync/offload wording, and README tone. Add docs/improvement audit records and live demo links in design journey.
94 lines
3.4 KiB
Markdown
94 lines
3.4 KiB
Markdown
# Virtual drone fleet
|
|
|
|
Data generator. Same on-board pipeline as [03 — Data platform](../docs/03-data-platform.md): seeded kinematics, sensor streams, detection events, hot `current/` → sealed Parquet, compact UDP state broadcast.
|
|
|
|
One process = one drone. Scale swarm with Compose flag.
|
|
|
|
## Run a swarm
|
|
|
|
```bash
|
|
# 5 drones (default), 2-minute flight, shared flight id
|
|
FLIGHT_ID=$(date -u +%Y%m%dT%H%MZ)-sim docker compose up --build --scale drone=5
|
|
|
|
# 10 drones, 5-minute flight, 4x accelerated
|
|
DRONE_COUNT=10 DURATION_S=300 SPEEDUP=4 docker compose up --build --scale drone=10
|
|
```
|
|
|
|
Output lands in [`../var/t1/`](../var/t1/) ([T1](../docs/00-glossary.md#t1--warm) live lake — same path k3d fleet and warehouse offload use). Override with `SWARM_T1_DIR` if needed:
|
|
|
|
```
|
|
var/t1/dataset=telemetry/flight=…/drone=…/sensor=imu/year=…/…/hour=…/data.parquet
|
|
var/t1/dataset=detections/flight=…/drone=…/…
|
|
var/t1/dataset=state/flight=…/drone=…/… ← sent + received broadcasts
|
|
```
|
|
|
|
Historical flights after offload: [`../var/t3/`](../var/t3/) ([T3](../docs/00-glossary.md#t3--warehouse) warehouse).
|
|
|
|
## Run a single drone without Docker
|
|
|
|
```bash
|
|
pip install -r requirements.txt
|
|
DRONE_ID=dr-01 DURATION_S=30 SPEEDUP=10 DATA_DIR=./data python -m virtual_drone.main
|
|
```
|
|
|
|
## Query the results
|
|
|
|
```bash
|
|
python - <<'EOF'
|
|
import duckdb
|
|
con = duckdb.connect()
|
|
print(con.sql("""
|
|
SELECT drone, sensor, count(*) AS rows,
|
|
to_timestamp(min(ts_ns)/1e9) AS first_row,
|
|
to_timestamp(max(ts_ns)/1e9) AS last_row
|
|
FROM read_parquet('data/dataset=telemetry/**/*.parquet', hive_partitioning=true)
|
|
GROUP BY drone, sensor ORDER BY drone, sensor
|
|
"""))
|
|
print(con.sql("""
|
|
SELECT drone, direction, count(*) AS frames, count(DISTINCT peer_id) AS peers
|
|
FROM read_parquet('data/dataset=state/**/*.parquet', hive_partitioning=true)
|
|
GROUP BY drone, direction ORDER BY drone, direction
|
|
"""))
|
|
EOF
|
|
```
|
|
|
|
## Monitoring
|
|
|
|
`monitoring` profile: DuckDB exporter → Prometheus → Grafana ([07 — Observability](../docs/07-observability.md)):
|
|
|
|
```bash
|
|
docker compose --profile monitoring up -d
|
|
FLIGHT_ID=$(date -u +%Y%m%dT%H%MZ)-sim docker compose up --scale drone=5
|
|
```
|
|
|
|
- Grafana: `http://localhost:3000` → **Swarm Fleet — generated data**
|
|
- Prometheus: `http://localhost:9090` · exporter: `http://localhost:9105/metrics`
|
|
|
|
Fleet stats derived from Parquet via SQL — no on-drone agent.
|
|
|
|
## Data-plane explorer
|
|
|
|
`monitoring` profile also starts explorer at `http://localhost:8088` (k3d: `:30088`):
|
|
|
|
- Partition tree live: `dataset → flight → drone → sensor → hour → file`
|
|
- Read-only SQL console — same [statement gate](../docs/04-swarm-sync.md) as peer query channel: `SELECT`/`WITH`/`DESCRIBE`/`SUMMARIZE`/`SHOW` only
|
|
|
|
Point: bench exploration uses same read-only SQL contract as flight peers.
|
|
|
|
## Reproducibility
|
|
|
|
Deterministic per `(SEED, DRONE_ID)`. Bug report = seed + config.
|
|
|
|
## Knobs
|
|
|
|
| Env | Default | Meaning |
|
|
| --- | --- | --- |
|
|
| `DRONE_ID` | derived from hostname | Unique per container automatically under `--scale` |
|
|
| `FLIGHT_ID` | generated | Set explicitly so all drones share one flight partition |
|
|
| `SEED` | `42` | Determinism root |
|
|
| `DURATION_S` | `120` | Simulated flight length |
|
|
| `SPEEDUP` | `1.0` | Wall-clock acceleration |
|
|
| `IMU_HZ` | `50` | High-rate sensor load |
|
|
| `STATE_HZ` | `5` | Broadcast frequency |
|
|
| `AREA_SIZE_M` | `400` | Patrol square side |
|