eSlider
87551fc1c1
CI & Release / Verify simulator (push) Skipped
CI & Release / Trivy scan (push) Skipped
CI & Release / Semantic Release (push) Skipped
Treat MinIO as existing stack infrastructure (default T3 warehouse, not on the drone), add a one-page executive map, and record placement as an open team question rather than a rip-and-replace.
62 lines
3.0 KiB
Markdown
62 lines
3.0 KiB
Markdown
# 00 — Executive overview
|
|
|
|
A one-page map of the proposal. Details and trade-offs live in the linked docs.
|
|
Nothing here is a mandate: it is a from-scratch platform sketch the team can
|
|
reshape once real constraints are on the table.
|
|
|
|
## Goal
|
|
|
|
Build an **on-prem, air-gapped data platform** for an autonomous drone swarm:
|
|
accumulate sensor and detection data on each unit, share only what peers need in
|
|
flight, and offload complete flights to a ground warehouse for replay and
|
|
training.
|
|
|
|
## Constraints (given)
|
|
|
|
- No internet / no external access in flight; data stays inside the system.
|
|
- Each drone is autonomous; mesh links are intermittent — **no swarm-wide orchestrator**.
|
|
- On-board today: **Docker Compose**, **DuckDB**, YOLO-like video analytics; **Parquet** under evaluation.
|
|
- **MinIO is already in the stack** — exact placement not fully known yet (see below).
|
|
- Ground side can run lightweight Kubernetes; this proposal keeps k3s **on the ground only**.
|
|
|
|
## Data contract (proposed)
|
|
|
|
| What | Where | Crosses the air? |
|
|
| --- | --- | --- |
|
|
| Raw telemetry | Local NVMe (Parquet / Hive layout) | **No** — offloads after landing |
|
|
| Pose / detections (derived) | Local store + peer pub/sub | **Yes** — small, budgeted |
|
|
| Full flight archive | Ground warehouse (Parquet + object store) | After landing only |
|
|
|
|
One layout on every floor so offload is a **mirror**, not a migration
|
|
([03 — Data platform](03-data-platform.md)).
|
|
|
|
## Sync rule
|
|
|
|
1. **Fast path:** compact pose frames (~45 B @ 5 Hz) + event-shaped detections.
|
|
2. **Bulk path (peers):** sealed derived partitions when the link allows
|
|
(rsync/SSH proposed; alternatives open).
|
|
3. **Never** push raw high-rate telemetry peer-to-peer in flight.
|
|
|
|
## MinIO (first-class, placement TBD)
|
|
|
|
MinIO is treated as **already chosen infrastructure**, not something to rip out.
|
|
|
|
| Role | Stance in this sketch |
|
|
| --- | --- |
|
|
| **Ground warehouse (T3)** | **Primary home** — S3 API under the Parquet lake / offload target |
|
|
| **On-board (drone)** | **Default: no** — competes with flight-critical CPU/RAM; local Parquet + DuckDB is enough for the hot path |
|
|
| **On-board exception** | Not excluded if a team already relies on an S3 API in Compose; then derived-only, never the 5 Hz pose path |
|
|
| **5 Hz pose / collision state** | **Not MinIO** — pub/sub (UDP in the PoC; Zenoh proposed) |
|
|
| **Inter-drone history catch-up** | Prefer pull of sealed partitions (SSH/rsync); MinIO replication only if ops already standardised on it |
|
|
|
|
Open item for the team: document how MinIO is used today, then lock T2/T3 roles
|
|
([09 — Open questions](09-open-questions.md)).
|
|
|
|
## What to read next
|
|
|
|
1. [12 — Design journey](12-design-journey.md) — narrative + deep links into code
|
|
2. [Implemented vs proposed](../README.md#implemented-now-vs-proposed-next) — PoC vs production intent
|
|
3. [09 — Open questions](09-open-questions.md) — including MinIO placement
|
|
|
|
Live visual PoC: [swarm.produktor.io](https://swarm.produktor.io/) (same access as this repository).
|