eSlider
df78a7c5db
Replace object-store replication with rsync over persistent multiplexed SSH (ed25519, pinned known_hosts, forced commands as the only remote API); demote GraphQL to the ground integration standard. Add DuckDB spatial extension for separation audits, perimeter predicates and trajectory analysis in the local planar frame.
89 lines
5.1 KiB
Markdown
89 lines
5.1 KiB
Markdown
# 02 — Architecture
|
|
|
|
## On-board: three layers, one Compose file
|
|
|
|
Every drone runs the same three-layer data plane. **Docker Compose under systemd is the only on-board orchestrator.** Cluster orchestrators (Kubernetes, Swarm mode) assume a stable control plane and continuous connectivity — in an ad-hoc mesh with intermittent links that is an anti-pattern. Each drone is fully autonomous; coordination happens through data exchange, not through a shared control plane. Lightweight Kubernetes (k3s) is used on the ground only (warehouse, CI runners, simulation farm — see [06 — Environments](06-environments.md)).
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph drone [Single drone — Docker Compose under systemd]
|
|
subgraph ingestion [Layer 1 — Ingestion]
|
|
SENSORS["sensor-ingest<br/>ROS 2 topics / drivers → stream"]
|
|
VIDEO["video-analytics<br/>YOLO-like detector, GPU"]
|
|
end
|
|
subgraph storage [Layer 2 — Storage and transform]
|
|
WRITER["parquet-writer<br/>stream → current/ blocks"]
|
|
SEALER["sealer<br/>compact closed windows"]
|
|
DUCK["DuckDB<br/>in-process SQL over Parquet"]
|
|
NVME[("NVMe<br/>Hive-partitioned Parquet")]
|
|
end
|
|
subgraph serving [Layer 3 — Serving and sync]
|
|
HOOK["event hook<br/>fires on new derived data"]
|
|
PUB["state publisher<br/>pub/sub broadcast"]
|
|
MINIO["MinIO<br/>derived datasets bucket"]
|
|
QAPI["query API<br/>peer data requests"]
|
|
end
|
|
end
|
|
|
|
SENSORS --> WRITER
|
|
VIDEO -->|detections| WRITER
|
|
WRITER --> NVME
|
|
SEALER --> NVME
|
|
DUCK --> NVME
|
|
WRITER -->|derived rows| HOOK
|
|
HOOK --> PUB
|
|
HOOK --> MINIO
|
|
QAPI --> DUCK
|
|
PUB -.->|mesh| PEERS["peer drones"]
|
|
MINIO -.->|replication| PEERS
|
|
QAPI -.->|on demand| PEERS
|
|
```
|
|
|
|
### Layer 1 — Ingestion
|
|
|
|
- `sensor-ingest` subscribes to sensor sources (ROS 2 topics where available, raw drivers otherwise) and normalizes them into typed streams: IMU, barometer, temperature, LiDAR, RSSI, power, and so on.
|
|
- `video-analytics` runs a **YOLO-like object detector** on the camera stream. The model is deliberately treated as a swappable, versioned artifact (weights differ per mission and improve over time — see [11 — CI/CD & delivery](11-cicd-delivery.md)). GPU access via nvidia-container-toolkit; ARM64 builds use the vendor's L4T-class base images.
|
|
- Both services emit rows, not files. Timestamps are `int64` epoch **nanoseconds** end to end (native for ROS 2; IMU-class rates make milliseconds insufficient).
|
|
|
|
### Layer 2 — Storage and transform
|
|
|
|
- `parquet-writer` appends incoming rows into small per-minute Parquet blocks under the partition's `current/` directory (details in [03 — Data platform](03-data-platform.md)).
|
|
- `sealer` compacts each closed window into one ZSTD-compressed file and enforces retention quotas on the NVMe.
|
|
- DuckDB runs **in-process** inside whichever service needs SQL — there is no database server to babysit.
|
|
|
|
### Layer 3 — Serving and sync
|
|
|
|
- The **event hook** is the on-board "lambda": when the writer lands new *derived* rows (state, detections), it triggers registered actions — broadcast, MinIO upload, or a local mission-logic callback. Nothing polls.
|
|
- The **state publisher** broadcasts compact position/attitude/detection payloads over the mesh pub/sub (transport analysis in [04 — Swarm sync](04-swarm-sync.md)).
|
|
- **Bulk sync** pulls sealed derived partitions from peers over persistent SSH (rsync delta transfer); MinIO remains optional where an S3 API is wanted.
|
|
- **Peer queries** run against local DuckDB via SSH forced commands — allow-listed query templates, columnar responses (the standard is in [04](04-swarm-sync.md)).
|
|
|
|
## Communication planes
|
|
|
|
Three isolated planes with different lifecycles:
|
|
|
|
```mermaid
|
|
graph LR
|
|
subgraph planes [Communication planes]
|
|
C2["C2 control plane<br/>mission intent, narrow, in production"]
|
|
DATA["swarm data plane<br/>state broadcast + sync, in production"]
|
|
DEV["dev/debug plane<br/>live telemetry, bench only"]
|
|
end
|
|
OPERATOR["supervising operator<br/>(man-in-the-loop)"] --> C2
|
|
C2 --> SWARM["swarm"]
|
|
SWARM <--> DATA
|
|
DEV -.->|absent from production builds| SWARM
|
|
```
|
|
|
|
| Plane | Purpose | Bandwidth | In production |
|
|
| --- | --- | --- | --- |
|
|
| **C2 control plane** | Deliver declarative mission goals and boundaries; receive high-level status. Human supervises intent, not actions | Very low | Yes |
|
|
| **Swarm data plane** | State broadcast, derived-data replication, peer queries | The scarce resource; budget per message class | Yes |
|
|
| **Dev/debug plane** | Full live telemetry for bench development and HIL tests | High | **No — physically absent from production builds** |
|
|
|
|
The dev plane is not "disabled by config": production images and radio profiles simply do not contain it. That removes an entire attack surface instead of guarding it.
|
|
|
|
## Out of scope (consumed as services)
|
|
|
|
Flight control, trajectory planning, mission logic, and model training sit **on top of** this platform: they subscribe to the event hook, query DuckDB, and read the warehouse. Their internals do not affect the platform design beyond the data contracts defined here.
|