swarm-house/simulator

Virtual drone fleet

Data generator. Same on-board pipeline as 03 — Data platform: seeded kinematics, sensor streams, detection events, hot current/ → sealed Parquet, compact UDP state broadcast.

One process = one drone. Scale swarm with Compose flag.

Run a swarm

# 5 drones (default), 2-minute flight, shared flight id
FLIGHT_ID=$(date -u +%Y%m%dT%H%MZ)-sim docker compose up --build --scale drone=5

# 10 drones, 5-minute flight, 4x accelerated
DRONE_COUNT=10 DURATION_S=300 SPEEDUP=4 docker compose up --build --scale drone=10

Output lands in ../var/t1/ (T1 live lake — same path k3d fleet and warehouse offload use). Override with SWARM_T1_DIR if needed:

var/t1/dataset=telemetry/flight=…/drone=…/sensor=imu/year=…/…/hour=…/data.parquet
var/t1/dataset=detections/flight=…/drone=…/…
var/t1/dataset=state/flight=…/drone=…/…        ← sent + received broadcasts

Historical flights after offload: ../var/t3/ (T3 warehouse).

Run a single drone without Docker

pip install -r requirements.txt
DRONE_ID=dr-01 DURATION_S=30 SPEEDUP=10 DATA_DIR=./data python -m virtual_drone.main

Query the results

python - <<'EOF'
import duckdb
con = duckdb.connect()
print(con.sql("""
    SELECT drone, sensor, count(*) AS rows,
           to_timestamp(min(ts_ns)/1e9) AS first_row,
           to_timestamp(max(ts_ns)/1e9) AS last_row
    FROM read_parquet('data/dataset=telemetry/**/*.parquet', hive_partitioning=true)
    GROUP BY drone, sensor ORDER BY drone, sensor
"""))
print(con.sql("""
    SELECT drone, direction, count(*) AS frames, count(DISTINCT peer_id) AS peers
    FROM read_parquet('data/dataset=state/**/*.parquet', hive_partitioning=true)
    GROUP BY drone, direction ORDER BY drone, direction
"""))
EOF

Monitoring

monitoring profile: DuckDB exporter → Prometheus → Grafana (07 — Observability):

docker compose --profile monitoring up -d
FLIGHT_ID=$(date -u +%Y%m%dT%H%MZ)-sim docker compose up --scale drone=5

• Grafana: http://localhost:3000 → Swarm Fleet — generated data
• Prometheus: http://localhost:9090 · exporter: http://localhost:9105/metrics

Fleet stats derived from Parquet via SQL — no on-drone agent.

Data-plane explorer

monitoring profile also starts explorer at http://localhost:8088 (k3d: :30088):

• Partition tree live: dataset → flight → drone → sensor → hour → file
• Read-only SQL console — same statement gate as peer query channel: SELECT/WITH/DESCRIBE/SUMMARIZE/SHOW only

Point: bench exploration uses same read-only SQL contract as flight peers.

Reproducibility

Deterministic per (SEED, DRONE_ID). Bug report = seed + config.

Knobs

Env	Default	Meaning
DRONE_ID	derived from hostname	Unique per container automatically under --scale
FLIGHT_ID	generated	Set explicitly so all drones share one flight partition
SEED	42	Determinism root
DURATION_S	120	Simulated flight length
SPEEDUP	1.0	Wall-clock acceleration
IMU_HZ	50	High-rate sensor load
STATE_HZ	5	Broadcast frequency
AREA_SIZE_M	400	Patrol square side