eSlider
57e48e0b55
Problem statement, on-board architecture, Parquet/DuckDB storage design, swarm sync strategy, zero-trust networking, environments, observability, CI/CD delivery with fleet release manifests, roadmap and open questions.
72 lines
4.1 KiB
Markdown
72 lines
4.1 KiB
Markdown
# 06 — Environments
|
||
|
||
Three infrastructures, one data contract. The layout, schemas, and pipelines are identical everywhere; only scale and lifetime differ.
|
||
|
||
```mermaid
|
||
graph LR
|
||
subgraph dev [3 — Dev and simulation]
|
||
SIM["virtual drone fleet<br/>(simulator, Compose)"]
|
||
LAP["engineer laptop<br/>Dev Container"]
|
||
SLICE["T4: data slices"]
|
||
end
|
||
subgraph fleet [1 — Fleet, in flight]
|
||
D1["drone: Compose data plane<br/>T0 hot + T1 NVMe + T2 MinIO"]
|
||
D2["drone …"]
|
||
end
|
||
subgraph ground [2 — Ground, on-prem]
|
||
DOCK["base station docks"]
|
||
DWH["T3 warehouse<br/>object store + Parquet/DuckDB"]
|
||
K3S["k3s: CI runners, registry mirror,<br/>sim farm, dashboards"]
|
||
TRAIN["model training (out of scope)<br/>reads T3, ships weights"]
|
||
end
|
||
|
||
D1 <-.->|mesh| D2
|
||
D1 -->|"offload: mirror + audit"| DOCK
|
||
DOCK --> DWH
|
||
DWH -->|"slices for development"| SLICE
|
||
SIM -->|"same pipeline, synthetic flights"| SLICE
|
||
DWH --> TRAIN
|
||
TRAIN -->|"versioned weights"| K3S
|
||
```
|
||
|
||
## 1 — Fleet infrastructure (on the drones)
|
||
|
||
- Docker Compose under systemd; the full stack from [02 — Architecture](02-architecture.md).
|
||
- Storage floors T0–T2. Nothing here requires ground contact during a mission.
|
||
- Receives new software only while docked, from the registry mirror, pinned by the fleet release manifest.
|
||
|
||
## 2 — Ground infrastructure (on-prem)
|
||
|
||
The permanent installation. This **is** allowed to be a cluster — links are wired and stable here, so k3s earns its keep:
|
||
|
||
| Component | Runs on | Role |
|
||
| --- | --- | --- |
|
||
| **Warehouse (T3)** | Object store (MinIO or equivalent) + Parquet | Every flight of every drone; the system's long-term memory |
|
||
| **Base station docks** | Bare metal | Wired offload + integrity audit + drone provisioning ([03](03-data-platform.md), [05](05-network-security.md)) |
|
||
| **Registry mirror** | k3s | In-air-gap container registry + artifact storage; the only software source drones ever see |
|
||
| **CI runners** | k3s | Multi-arch builds, tests, scans ([11](11-cicd-delivery.md)) |
|
||
| **Simulation farm** | k3s | Scaled-up virtual swarm runs for regression and pre-mission validation |
|
||
| **Dashboards** | k3s | Observability stack ([07](07-observability.md)) + flight replay tooling |
|
||
|
||
Replay is the warehouse's flagship feature: because every broadcast, detection, and re-tasking trigger is preserved with nanosecond timestamps in one layout, any mission can be reconstructed step by step — for engineering analysis, for audit/traceability, and as training input.
|
||
|
||
## 3 — Dev & simulation infrastructure
|
||
|
||
The environment engineers live in daily — and deliberately the first thing to build (see [08 — Roadmap](08-roadmap.md)), because everything else is validated inside it.
|
||
|
||
- **Dev Containers** give every engineer the same toolchain in minutes; no snowflake laptops.
|
||
- **The virtual drone fleet** ([`simulator/`](../simulator/)) runs the *identical* data plane: same writer, same partitioning, same sealing, same broadcast schema. `DRONE_COUNT=10 docker compose up` is a swarm on a laptop.
|
||
- **Scenarios are reproducible by seed:** route shapes, sensor noise, drone dropouts, link losses are all parameterized — a bug report is a seed + config, not a war story.
|
||
- **T4 data slices**: a thin tool pulls partition subsets from T3 (`flight=X, drone=Y, hour=Z`) for local work — layout-identical, so every query and pipeline runs unmodified.
|
||
- The same simulator images scale out on the ground k3s farm for CI regression runs: every merge request replays canonical scenarios and asserts on the resulting Parquet output (row counts, coverage, staleness budgets).
|
||
|
||
## Why identical layouts matter (the payoff)
|
||
|
||
| Operation | What it costs with one layout everywhere |
|
||
| --- | --- |
|
||
| Flight offload | `mc mirror` / `rsync` + audit — no transform step |
|
||
| Debugging a field issue | Pull the flight slice, replay locally, same queries |
|
||
| Validating a pipeline change | Run simulator, diff Parquet outputs |
|
||
| Training data prep | Read T3 directly, no export pipeline |
|
||
| Onboarding an engineer | One Dev Container, one `docker compose up` |
|