docs: put MinIO first-class on ground, add executive overview
CI & Release / Verify simulator (push) Skipped
CI & Release / Trivy scan (push) Skipped
CI & Release / Semantic Release (push) Skipped
CI & Release / Verify simulator (push) Skipped
CI & Release / Trivy scan (push) Skipped
CI & Release / Semantic Release (push) Skipped
Treat MinIO as existing stack infrastructure (default T3 warehouse, not on the drone), add a one-page executive map, and record placement as an open team question rather than a rip-and-replace.
This commit is contained in:
@@ -0,0 +1,61 @@
|
||||
# 00 — Executive overview
|
||||
|
||||
A one-page map of the proposal. Details and trade-offs live in the linked docs.
|
||||
Nothing here is a mandate: it is a from-scratch platform sketch the team can
|
||||
reshape once real constraints are on the table.
|
||||
|
||||
## Goal
|
||||
|
||||
Build an **on-prem, air-gapped data platform** for an autonomous drone swarm:
|
||||
accumulate sensor and detection data on each unit, share only what peers need in
|
||||
flight, and offload complete flights to a ground warehouse for replay and
|
||||
training.
|
||||
|
||||
## Constraints (given)
|
||||
|
||||
- No internet / no external access in flight; data stays inside the system.
|
||||
- Each drone is autonomous; mesh links are intermittent — **no swarm-wide orchestrator**.
|
||||
- On-board today: **Docker Compose**, **DuckDB**, YOLO-like video analytics; **Parquet** under evaluation.
|
||||
- **MinIO is already in the stack** — exact placement not fully known yet (see below).
|
||||
- Ground side can run lightweight Kubernetes; this proposal keeps k3s **on the ground only**.
|
||||
|
||||
## Data contract (proposed)
|
||||
|
||||
| What | Where | Crosses the air? |
|
||||
| --- | --- | --- |
|
||||
| Raw telemetry | Local NVMe (Parquet / Hive layout) | **No** — offloads after landing |
|
||||
| Pose / detections (derived) | Local store + peer pub/sub | **Yes** — small, budgeted |
|
||||
| Full flight archive | Ground warehouse (Parquet + object store) | After landing only |
|
||||
|
||||
One layout on every floor so offload is a **mirror**, not a migration
|
||||
([03 — Data platform](03-data-platform.md)).
|
||||
|
||||
## Sync rule
|
||||
|
||||
1. **Fast path:** compact pose frames (~45 B @ 5 Hz) + event-shaped detections.
|
||||
2. **Bulk path (peers):** sealed derived partitions when the link allows
|
||||
(rsync/SSH proposed; alternatives open).
|
||||
3. **Never** push raw high-rate telemetry peer-to-peer in flight.
|
||||
|
||||
## MinIO (first-class, placement TBD)
|
||||
|
||||
MinIO is treated as **already chosen infrastructure**, not something to rip out.
|
||||
|
||||
| Role | Stance in this sketch |
|
||||
| --- | --- |
|
||||
| **Ground warehouse (T3)** | **Primary home** — S3 API under the Parquet lake / offload target |
|
||||
| **On-board (drone)** | **Default: no** — competes with flight-critical CPU/RAM; local Parquet + DuckDB is enough for the hot path |
|
||||
| **On-board exception** | Not excluded if a team already relies on an S3 API in Compose; then derived-only, never the 5 Hz pose path |
|
||||
| **5 Hz pose / collision state** | **Not MinIO** — pub/sub (UDP in the PoC; Zenoh proposed) |
|
||||
| **Inter-drone history catch-up** | Prefer pull of sealed partitions (SSH/rsync); MinIO replication only if ops already standardised on it |
|
||||
|
||||
Open item for the team: document how MinIO is used today, then lock T2/T3 roles
|
||||
([09 — Open questions](09-open-questions.md)).
|
||||
|
||||
## What to read next
|
||||
|
||||
1. [12 — Design journey](12-design-journey.md) — narrative + deep links into code
|
||||
2. [Implemented vs proposed](../README.md#implemented-now-vs-proposed-next) — PoC vs production intent
|
||||
3. [09 — Open questions](09-open-questions.md) — including MinIO placement
|
||||
|
||||
Live visual PoC: [swarm.produktor.io](https://swarm.produktor.io/) (same access as this repository).
|
||||
Reference in New Issue
Block a user