eSlider
5f6fd8462c
Extend glossary with storage floors, source of truth, and linkable anchors. Fix broken links, pose frame drift, rsync/offload wording, and README tone. Add docs/improvement audit records and live demo links in design journey.
9.0 KiB
00 — Glossary
Terms and abbreviations used throughout this proposal.
Vocabulary guide: architecture layers (1–3) = on-board software stack; storage floors (T0–T4) = retention/latency ladder; pipeline stages = narrative capture → reduction → sync only (12 — Design journey §4). Do not mix these three.
Project vocabulary
T0 — hot
RAM / DuckDB in-process sliding window of the last minutes; what mission logic queries in flight. Retention: minutes. Maps to hot in the classic three-tier model.
T1 — warm
Drone NVMe: full raw telemetry + detections + state of the current flight; never leaves the drone in flight. Retention: current flight plus quota-based headroom. Maps to warm. Runtime path: var/t1/ (ADR-0008).
T2 — shared
On-board MinIO bucket: derived data only (detections, state), replicated opportunistically across the swarm. Retention: current mission.
T3 — warehouse
Ground on-prem object store + Parquet lakehouse: every flight of every drone, forever; replay, analytics, model training. Retention: years. Maps to cold / DWH. Runtime path: var/t3/ (ADR-0008). After offload and integrity audit, T3 is the source of truth for historical flight data.
T4 — dev
Engineer laptop or sim farm: slices pulled from T3, or synthetic data from the simulator. Retention: ephemeral.
Source of truth
The authoritative copy of flight data after offload and integrity audit: the T3 warehouse. Each drone holds its own partial record in flight (T1); peers exchange derived state only (04 — Swarm sync). Replay, post-flight fusion, training, and audit read from T3 — not from any single drone's local store. See 06 — Environments, ADR-0003.
Architecture layer 1 — ingestion
On-board services that capture sensor streams and video analytics (02 — Architecture): sensor-ingest, video-analytics.
Architecture layer 2 — storage and transform
On-board Parquet write path, sealer, DuckDB, NVMe (02 — Architecture).
Architecture layer 3 — serving and sync
On-board event hook, pub/sub broadcast, MinIO derived bucket, peer query API (02 — Architecture).
Pipeline stage
Narrative inside each drone: (1) raw capture, (2) ETL/reduction, (3) bidirectional sync interface. Not the same as storage floors T0–T4 (12 — Design journey §4).
Bulk sync
In-flight pull of sealed detections/state Parquet partitions between peers via rsync delta over persistent SSH forced commands (04 — Swarm sync).
Offload
Post-flight mirror of sealed T1 partitions into T3 at the dock: rsync or MinIO mc mirror, then integrity audit before local prune (03 — Data platform).
rclone
Optional rsync-class tool when the far end is object storage and rsync is awkward; not the default in-flight mesh path (09 — Open questions §12).
ADR
Architecture Decision Record — a point-in-time decision for this repository (adr/README.md).
ASR
Architecture Standard Record — a standing cross-repository policy (lives in the ecosystem inventar repo, not here). See adr/README.md.
Data & storage
| Term | Meaning |
|---|---|
| Parquet | Columnar file format; compresses time series extremely well (ZSTD + delta encoding), supports predicate pushdown |
| Hive partitioning | Directory layout key=value/… understood by most engines; enables partition pruning (only relevant directories are read) |
| DuckDB | In-process analytical SQL engine; queries Parquet globs directly, no server to operate |
| ZSTD | Zstandard compression; the default codec for all Parquet files here |
| DWH | Data warehouse — here: the on-prem ground store holding every flight ever flown; same role as source of truth |
| Hot / warm / cold tiers | Classic three-tier model: in-memory window → local NVMe → ground warehouse. Here mapped to T0, T1, and T3; T2 and T4 are named explicitly |
| Sealing | Compacting many small streamed Parquet files into one final file per partition once a time window closes |
| UUIDv7 | Time-ordered UUID; sortable by creation time, used as the canonical flight identifier |
| DuckDB spatial | Extension adding a GEOMETRY type and ST_* functions; used for separation audits, perimeter predicates, and trajectory analysis in the local planar frame |
| Schema registry | Versioned catalog of dataset schemas so producers and consumers can evolve independently |
| ETL | Extract, transform, load — the pipelines that move and reshape data between tiers |
Swarm & robotics
| Term | Meaning |
|---|---|
| UAV / UGV / USV / UUV | Unmanned aerial / ground / surface / underwater vehicle |
| Swarm | Group of autonomous vehicles coordinating without a central controller |
| Sustainable pulsing | Swarm ability to repeatedly engage a target area from multiple directions and self-reorganize as conditions evolve |
| Ubiquitous sensing | The swarm acting as one distributed sensor, fusing local observations into shared situational awareness ("top sight") |
| Man-in-the-loop | A human supervises intent and boundaries but does not steer individual vehicles |
| GPS-denied | Operating without satellite positioning; position comes from visual odometry and relative localization |
| Visual odometry | Estimating a vehicle's motion from its own camera stream |
| ROS 2 | Robot Operating System 2 — the de-facto middleware for robotics software; nodes exchange typed messages over topics |
| DDS | Data Distribution Service — the default ROS 2 transport middleware |
| Zenoh | Pub/sub + query protocol designed for constrained, dynamic networks; official ROS 2 RMW implementation exists |
| YOLO-like detector | Family of single-pass object-detection models; the on-board video service runs one of them, weights are swappable |
| IMU | Inertial measurement unit — accelerometer + gyroscope (+ magnetometer) |
| LiDAR | Laser-based distance sensing |
| RSSI | Received signal strength indicator; Wi-Fi signal strength, usable as a coarse relative-positioning signal |
| C2 | Command and control — the narrow channel that delivers mission intent to the swarm |
Infrastructure & delivery
| Term | Meaning |
|---|---|
| Air-gapped | No route to the internet, physically or logically |
| On-prem | Runs on owned hardware; no public cloud involvement |
| Zero trust | No implicit trust from network position; every peer authenticates with pre-provisioned credentials |
| mTLS | Mutual TLS — both sides of a connection present certificates |
| ed25519 | Modern elliptic-curve signature scheme; the SSH key type provisioned per device and per operation |
| Forced command | command="…" restriction in authorized_keys: the key can execute exactly one predefined operation — the key is the API |
| ControlMaster | SSH connection multiplexing: one persistent authenticated session carries many transfers/commands |
| PKI | Public key infrastructure — the certificate authority hierarchy behind mTLS |
| WireGuard | Lightweight encrypted tunnel protocol; basis for the swarm mesh overlay |
| Ad-hoc mesh | Wireless network formed directly between peers, no fixed infrastructure |
| Docker Compose | Declarative multi-container runtime; the only orchestrator on board a drone |
| k3s | Lightweight Kubernetes distribution; used on the ground only |
| Dev Container | Reproducible containerized development environment definition |
| MinIO | S3-compatible object store; runs on-board for derived data and on the ground as the warehouse backend |
| Registry mirror | Local copy of a container registry inside the air gap; drones pull images from it |
| OTA | Over-the-air update — delivered before a mission while docked, never mid-flight |
| Fleet release manifest | A versioned lockfile pinning every artifact (image digests, model weights, schemas, configs) that defines one fleet version |
| SemVer | Semantic versioning (MAJOR.MINOR.PATCH) applied to every artifact and to the fleet manifest itself |
| Trivy | Open-source scanner for container images and dependencies |
| SonarQube | Static code-quality and security analysis platform |
| nvidia-container-toolkit | Runtime hook exposing GPUs to containers; needed for on-board inference |
| HIL | Hardware-in-the-loop testing — real flight hardware wired into a simulated environment |
| CRDT | Conflict-free replicated data type; merges concurrent updates without coordination |