eSlider
57e48e0b55
Problem statement, on-board architecture, Parquet/DuckDB storage design, swarm sync strategy, zero-trust networking, environments, observability, CI/CD delivery with fleet release manifests, roadmap and open questions.
60 lines
4.0 KiB
Markdown
60 lines
4.0 KiB
Markdown
# 08 — Roadmap
|
||
|
||
From simple to complex, ordered by dependency — each stage is independently useful and validates the next. The guiding principle: **build the environment that makes everything else testable first.**
|
||
|
||
## Dependency graph
|
||
|
||
```mermaid
|
||
graph TD
|
||
S1["Stage 1<br/>Dev environment + CI skeleton<br/>Dev Containers, GitLab templates, multi-arch builds"]
|
||
S2["Stage 2<br/>Registry and artifacts<br/>images, model weights, schemas, cleanup policies"]
|
||
S3["Stage 3<br/>Single-drone data pipeline<br/>ingest → current/ → sealed Parquet → DuckDB"]
|
||
S4["Stage 4<br/>Serving layer<br/>event hook, query API, local MinIO"]
|
||
S5["Stage 5<br/>Virtual swarm simulation<br/>N drones in Compose, seeded scenarios, CI regression"]
|
||
S6["Stage 6<br/>Mesh network and security<br/>WireGuard overlay, PKI provisioning, mTLS"]
|
||
S7["Stage 7<br/>Swarm sync<br/>pose broadcast, detections pub/sub, MinIO replication"]
|
||
S8["Stage 8<br/>Ground warehouse<br/>offload + audit, T3 store, replay tooling"]
|
||
S9["Stage 9<br/>Observability<br/>platform metrics as sensor stream, ground dashboards"]
|
||
S10["Stage 10<br/>Fleet releases<br/>manifest, mirror, dock delivery, atomic rollback"]
|
||
S11["Stage 11<br/>Query standard<br/>GraphQL contract, schema registry, multi-team surface"]
|
||
|
||
S1 --> S2
|
||
S1 --> S3
|
||
S2 --> S5
|
||
S3 --> S4
|
||
S4 --> S5
|
||
S5 --> S7
|
||
S6 --> S7
|
||
S3 --> S8
|
||
S7 --> S9
|
||
S8 --> S9
|
||
S2 --> S10
|
||
S5 --> S10
|
||
S8 --> S11
|
||
S7 --> S11
|
||
```
|
||
|
||
## Stage detail
|
||
|
||
| Stage | Deliverable | Proves | Depends on |
|
||
| --- | --- | --- | --- |
|
||
| **1 — Dev environment + CI skeleton** | Dev Containers, GitLab shared templates, multi-arch runner fleet | An engineer builds and tests on day one; ARM64/AMD64 both green | — |
|
||
| **2 — Registry & artifacts** | GitLab registry live, model weights + schemas as versioned packages | One artifact store, scanning wired in (Trivy, SonarQube) | 1 |
|
||
| **3 — Single-drone data pipeline** | `sensor-ingest` → writer → `current/` → sealed → DuckDB queries on one node | The storage core: rates sustained, compression measured, quotas enforced | 1 |
|
||
| **4 — Serving layer** | Event hook + query API + on-board MinIO | Event-driven consumption; nothing polls | 3 |
|
||
| **5 — Virtual swarm** | Simulator, N drones via Compose, seeded scenarios, CI regression gate | Everything after this ships with a test bench | 2, 4 |
|
||
| **6 — Mesh & security** | WireGuard overlay, offline CA, provisioning flow, mTLS | Zero-trust fabric exists before any real sync traffic | 1 |
|
||
| **7 — Swarm sync** | Pose broadcast + detections pub/sub + bulk replication over the mesh | Staleness budgets met under simulated loss/partitions | 5, 6 |
|
||
| **8 — Ground warehouse** | Dock offload (mirror + audit), T3 store, replay queries | A full flight round-trips: fly (simulated) → offload → replay | 3 |
|
||
| **9 — Observability** | Platform metrics as sensor stream; ground Prometheus/Grafana/Loki; post-mortem dashboards | Every health question from [07](07-observability.md) answerable | 7, 8 |
|
||
| **10 — Fleet releases** | Release manifest, registry mirror, dock delivery, atomic rollback drill | The whole fleet moves as one version, rollback rehearsed | 2, 5 |
|
||
| **11 — Query standard** | GraphQL contract over versioned schemas; documentation for other teams | The multi-team integration surface | 7, 8 |
|
||
|
||
## Sequencing rationale
|
||
|
||
- **Stages 1–2 before everything:** unglamorous, but every later stage is only as fast as its build/test loop.
|
||
- **Stage 3 is the heart** — and it needs no network, no security, no swarm: a single machine and real (or simulated) sensor rates. Risk is retired early where iteration is cheapest.
|
||
- **Stage 5 before sync (7):** building a distributed protocol without a reproducible multi-node bench means debugging it in the field. The virtual swarm makes stage 7 a CI problem instead.
|
||
- **Stage 6 in parallel:** the trust fabric has no dependency on the data path and can mature alongside it.
|
||
- **Stages 8–11 stack on a proven core** — by then, every layer under them is regression-tested.
|