swarm-house/docs/02-architecture.md

# 02 — Architecture

## On-board: three layers, one Compose file

Every drone runs the same three-layer data plane. **Docker Compose under systemd is the only on-board orchestrator.** Cluster orchestrators (Kubernetes, Swarm mode) assume a stable control plane and continuous connectivity — in an ad-hoc mesh with intermittent links that is an anti-pattern. Each drone is fully autonomous; coordination happens through data exchange, not through a shared control plane. Lightweight Kubernetes (k3s) is used on the ground only (warehouse, CI runners, simulation farm — see [06 — Environments](06-environments.md)).

```mermaid
graph TB
    subgraph drone [Single drone — Docker Compose under systemd]
        subgraph ingestion [Layer 1 — Ingestion]
            SENSORS["sensor-ingest<br/>ROS 2 topics / drivers → stream"]
            VIDEO["video-analytics<br/>YOLO-like detector, GPU"]
        end
        subgraph storage [Layer 2 — Storage and transform]
            WRITER["parquet-writer<br/>stream → current/ blocks"]
            SEALER["sealer<br/>compact closed windows"]
            DUCK["DuckDB<br/>in-process SQL over Parquet"]
            NVME[("NVMe<br/>Hive-partitioned Parquet")]
        end
        subgraph serving [Layer 3 — Serving and sync]
            HOOK["event hook<br/>fires on new derived data"]
            PUB["state publisher<br/>pub/sub broadcast"]
            BULK["bulk sync<br/>rsync over SSH"]
            MINIO["MinIO<br/>optional derived bucket"]
            QAPI["query API<br/>SQL-over-SSH"]
        end
    end

    SENSORS --> WRITER
    VIDEO -->|detections| WRITER
    WRITER --> NVME
    SEALER --> NVME
    DUCK --> NVME
    WRITER -->|derived rows| HOOK
    HOOK --> PUB
    HOOK --> MINIO
    HOOK --> BULK
    QAPI --> DUCK
    PUB -.->|mesh pose| PEERS["peer drones"]
    BULK -.->|sealed partitions| PEERS
    QAPI -.->|on demand| PEERS
```

MinIO stays available where an S3 API helps (on-board derived datasets, ground warehouse). **In-flight peer bulk sync is SSH/rsync**, not object-store replication — see [04 — Swarm sync](04-swarm-sync.md).
### Layer 1 — Ingestion

- `sensor-ingest` subscribes to sensor sources (ROS 2 topics where available, raw drivers otherwise) and normalizes them into typed streams: IMU, barometer, temperature, LiDAR, RSSI, power, and so on.
- `video-analytics` runs a **YOLO-like object detector** on the camera stream. The model is deliberately treated as a swappable, versioned artifact (weights differ per mission and improve over time — see [11 — CI/CD & delivery](11-cicd-delivery.md)). GPU access via nvidia-container-toolkit; ARM64 builds use the vendor's L4T-class base images.
- Both services emit rows, not files. Timestamps are `int64` epoch **nanoseconds** end to end (native for ROS 2; IMU-class rates make milliseconds insufficient).

### Layer 2 — Storage and transform

- `parquet-writer` appends incoming rows into small per-minute Parquet blocks under the partition's `current/` directory (details in [03 — Data platform](03-data-platform.md)).
- `sealer` compacts each closed window into one ZSTD-compressed file and enforces retention quotas on the NVMe.
- DuckDB runs **in-process** inside whichever service needs SQL — there is no database server to babysit.

### Layer 3 — Serving and sync

- The **event hook** is the on-board "lambda": when the writer lands new *derived* rows (state, detections), it triggers registered actions — broadcast, optional MinIO put, bulk-sync hint, or a local mission-logic callback. Nothing polls.
- The **state publisher** broadcasts compact position/attitude/detection payloads over the mesh pub/sub (UDP in the PoC; Zenoh proposed — [04](04-swarm-sync.md)).
- **Bulk sync** pulls sealed derived partitions from peers over persistent SSH (rsync delta transfer). MinIO is optional where an S3 API is wanted; it is **not** the in-flight peer replication path.
- **Peer queries** are read-only DuckDB SQL — HTTP explorer gate in the PoC; **SSH forced commands** proposed for flight (SELECT-only gate, read-only OS user, columnar responses — [04](04-swarm-sync.md)).

## Communication planes

Three isolated planes with different lifecycles:

```mermaid
graph LR
    subgraph planes [Communication planes]
        C2["C2 control plane<br/>mission intent, narrow, in production"]
        DATA["swarm data plane<br/>state broadcast + sync, in production"]
        DEV["dev/debug plane<br/>live telemetry, bench only"]
    end
    OPERATOR["supervising operator<br/>(man-in-the-loop)"] --> C2
    C2 --> SWARM["swarm"]
    SWARM <--> DATA
    DEV -.->|absent from production builds| SWARM
```

| Plane | Purpose | Bandwidth | In production |
| --- | --- | --- | --- |
| **C2 control plane** | Deliver declarative mission goals and boundaries; receive high-level status. Human supervises intent, not actions | Very low | Yes |
| **Swarm data plane** | State broadcast, derived-data replication, peer queries | The scarce resource; budget per message class | Yes |
| **Dev/debug plane** | Full live telemetry for bench development and HIL tests | High | **No — physically absent from production builds** |

The dev plane is not "disabled by config": production images and radio profiles simply do not contain it. That removes an entire attack surface instead of guarding it.

## Out of scope (consumed as services)

Flight control, trajectory planning, mission logic, and model training sit **on top of** this platform: they subscribe to the event hook, query DuckDB, and read the warehouse. Their internals do not affect the platform design beyond the data contracts defined here.