docs: on-prem data platform proposal for autonomous drone swarms

Problem statement, on-board architecture, Parquet/DuckDB storage design, swarm sync strategy, zero-trust networking, environments, observability, CI/CD delivery with fleet release manifests, roadmap and open questions.
2026-07-08 13:03:23 +01:00
commit 57e48e0b55
15 changed files with 958 additions and 0 deletions
@@ -0,0 +1,104 @@
+# 11 — CI/CD & delivery
+
+How software, models, and configuration reach the fleet — reproducibly, scanned, versioned, and atomically rollback-able. Everything lives inside the air gap.
+
+## Source and pipelines: GitLab
+
+GitLab (self-managed, on-prem) is the backbone: repositories, CI/CD, and the container registry in one system.
+
+- **Shared pipeline templates** — one template library (`ci-templates` repo) defines the standard stages; service repos include and parameterize them instead of copy-pasting YAML.
+- **Multi-arch by default** — every image builds for `linux/amd64` and `linux/arm64` via buildx on dedicated runners; GPU-inference images additionally build against the vendor's L4T-class base for the ARM targets.
+- **Runner fleet on ground k3s** — build runners (amd64 + arm64), a GPU runner for inference smoke tests, and simulation runners that execute virtual-swarm regression scenarios ([06](06-environments.md)).
+
+```mermaid
+graph LR
+    subgraph gitlab [GitLab, on-prem]
+        SRC["service repos<br/>+ ci-templates"]
+        CI["CI pipelines<br/>build · test · scan"]
+        REG["GitLab Container Registry<br/>images + generic packages"]
+    end
+    subgraph quality [Quality gates]
+        SQ["SonarQube<br/>code quality"]
+        TR["Trivy<br/>image + dependency scan"]
+        SIMT["virtual swarm<br/>regression scenarios"]
+    end
+    subgraph delivery [Delivery]
+        MAN["fleet release manifest<br/>semver, digests pinned"]
+        MIR["registry mirror<br/>base station"]
+        DOCK["dock: verify + apply"]
+    end
+    SRC --> CI
+    CI --> SQ
+    CI --> TR
+    CI --> SIMT
+    CI --> REG
+    REG --> MAN
+    MAN --> MIR
+    MIR --> DOCK
+```
+
+## Artifacts: GitLab Registry as the single store
+
+One registry for everything, next to the pipelines that produce it:
+
+| Artifact | Stored as |
+| --- | --- |
+| Service images (multi-arch) | Container registry, immutable tags + digests |
+| **Detection model weights** (YOLO-like family — architectures and weights vary per mission and improve over iterations) | Generic package registry, semver-versioned, checksummed |
+| Dataset schemas | Generic packages, semver ([03](03-data-platform.md)) |
+| Compose bundles, radio profiles, provisioning configs | Generic packages, semver |
+
+Registry hygiene is part of the design: cleanup policies per repository, immutable release tags, access split between CI (write) and mirrors (read).
+
+> Consolidating on the GitLab registry removes a separate artifact platform from the stack — one fewer system to run inside the air gap, one auth domain, artifacts adjacent to the pipelines that build them. Scanning moves to Trivy (below).
+
+## Quality gates
+
+| Gate | Tool | Blocks merge when |
+| --- | --- | --- |
+| Code quality, coverage, static analysis | **SonarQube** | Quality gate red |
+| Image and dependency vulnerabilities | **Trivy** (in CI + scheduled re-scan of released images) | Critical findings without an accepted waiver |
+| Behavioral regression | **Virtual swarm scenarios** — canonical seeds replayed, Parquet outputs asserted (row counts, coverage, staleness budgets) | Any budget exceeded |
+
+Scheduled Trivy re-scans matter in an air gap: a released image that was clean in March may carry a known CVE by June; the scan flags it for the next fleet release even though the image never changed.
+
+## The fleet release manifest
+
+The central delivery idea: **the fleet has exactly one version.**
+
+```yaml
+# fleet-release: 3.4.1
+schema_version: 1
+images:
+  sensor-ingest:    registry.internal/fleet/sensor-ingest@sha256:9f2c…   # 2.1.0
+  video-analytics:  registry.internal/fleet/video-analytics@sha256:5e11… # 3.0.2
+  parquet-writer:   registry.internal/fleet/parquet-writer@sha256:aa04…  # 1.8.0
+  state-publisher:  registry.internal/fleet/state-publisher@sha256:c7d9… # 1.4.3
+models:
+  detector: { package: detector-weights, version: 5.2.0, sha256: "e3b0…" }
+schemas:
+  telemetry: 2.1.0
+  detections: 1.3.0
+  state: 1.2.0
+config:
+  compose-bundle: 3.4.0
+  radio-profile: production-2
+  peer-registry: fleet-42-r7      # provisioning + revocations, see 05
+```
+
+- Everything is **semver-versioned individually**, and the manifest itself carries the fleet version — a lockfile for the whole swarm.
+- Images are referenced **by digest**; docks verify signatures and checksums before applying.
+- **Rollback is atomic:** re-apply the previous manifest. No per-service drift, ever — a drone either runs release 3.4.1 in full or 3.4.0 in full.
+- The manifest is what the simulation farm certifies: regression scenarios run against the exact manifest that will ship.
+
+## Delivery to the drones
+
+1. CI publishes a release manifest; the base-station **registry mirror** pulls all referenced artifacts inside the air gap.
+2. Docked drones fetch the manifest, verify digests/signatures ([05](05-network-security.md)), stage the new Compose bundle, and switch on the next boot cycle.
+3. **Never mid-flight.** Updates are a dock-only operation by construction — the update endpoint does not exist in the flight radio profile.
+
+## Developer experience
+
+- **Dev Containers** define the full toolchain (Python data tooling, DuckDB, compose, linters) — identical on any machine, onboarding in minutes.
+- **Linux dev VMs** are provisioned from the same configuration standard (cloud-init + the provisioning role), so "my VM" and "the CI runner" cannot diverge.
+- The virtual swarm ([`simulator/`](../simulator/)) is the daily inner loop: change the writer, `docker compose up`, query the output Parquet, done.