eSlider
5f6fd8462c
CI & Release / Verify simulator (push) Skipped
CI & Release / Trivy scan (push) Skipped
CI & Release / Semantic Release (push) Skipped
CI & Release / Verify simulator (pull_request) Failing after 4s
CI & Release / Trivy scan (pull_request) Skipped
CI & Release / Semantic Release (pull_request) Skipped
Extend glossary with storage floors, source of truth, and linkable anchors. Fix broken links, pose frame drift, rsync/offload wording, and README tone. Add docs/improvement audit records and live demo links in design journey.
132 lines
9.0 KiB
Markdown
132 lines
9.0 KiB
Markdown
# 00 — Glossary
|
||
|
||
Terms and abbreviations used throughout this proposal.
|
||
|
||
**Vocabulary guide:** **architecture layers** (1–3) = on-board software stack; **storage floors** (T0–T4) = retention/latency ladder; **pipeline stages** = narrative capture → reduction → sync only ([12 — Design journey](12-design-journey.md) §4). Do not mix these three.
|
||
|
||
## Project vocabulary
|
||
|
||
### T0 — hot
|
||
|
||
RAM / DuckDB in-process sliding window of the last minutes; what mission logic queries in flight. Retention: minutes. Maps to **hot** in the classic three-tier model.
|
||
|
||
### T1 — warm
|
||
|
||
Drone NVMe: full raw `telemetry` + `detections` + `state` of the current flight; never leaves the drone in flight. Retention: current flight plus quota-based headroom. Maps to **warm**. Runtime path: [`var/t1/`](../var/t1/) ([ADR-0008](adr/ADR-0008-runtime-data-paths.md)).
|
||
|
||
### T2 — shared
|
||
|
||
On-board MinIO bucket: derived data only (`detections`, `state`), replicated opportunistically across the swarm. Retention: current mission.
|
||
|
||
### T3 — warehouse
|
||
|
||
Ground on-prem object store + Parquet lakehouse: every flight of every drone, forever; replay, analytics, model training. Retention: years. Maps to **cold** / **DWH**. Runtime path: [`var/t3/`](../var/t3/) ([ADR-0008](adr/ADR-0008-runtime-data-paths.md)). After offload and integrity audit, T3 is the [source of truth](#source-of-truth) for historical flight data.
|
||
|
||
### T4 — dev
|
||
|
||
Engineer laptop or sim farm: slices pulled from T3, or synthetic data from the simulator. Retention: ephemeral.
|
||
|
||
### Source of truth
|
||
|
||
The authoritative copy of flight data after [offload](#offload) and integrity audit: the [T3](#t3--warehouse) warehouse. Each drone holds its own partial record in flight ([T1](#t1--warm)); peers exchange derived state only ([04 — Swarm sync](04-swarm-sync.md)). Replay, post-flight fusion, training, and audit read from T3 — not from any single drone's local store. See [06 — Environments](06-environments.md), [ADR-0003](adr/ADR-0003-sync-derived-state-only.md).
|
||
|
||
### Architecture layer 1 — ingestion
|
||
|
||
On-board services that capture sensor streams and video analytics ([02 — Architecture](02-architecture.md)): `sensor-ingest`, `video-analytics`.
|
||
|
||
### Architecture layer 2 — storage and transform
|
||
|
||
On-board Parquet write path, sealer, DuckDB, NVMe ([02 — Architecture](02-architecture.md)).
|
||
|
||
### Architecture layer 3 — serving and sync
|
||
|
||
On-board event hook, pub/sub broadcast, MinIO derived bucket, peer query API ([02 — Architecture](02-architecture.md)).
|
||
|
||
### Pipeline stage
|
||
|
||
Narrative inside each drone: (1) raw capture, (2) ETL/reduction, (3) bidirectional sync interface. Not the same as storage floors T0–T4 ([12 — Design journey](12-design-journey.md) §4).
|
||
|
||
### Bulk sync
|
||
|
||
In-flight pull of sealed `detections`/`state` Parquet partitions between peers via **rsync** delta over persistent SSH forced commands ([04 — Swarm sync](04-swarm-sync.md)).
|
||
|
||
### Offload
|
||
|
||
Post-flight mirror of sealed T1 partitions into T3 at the dock: **rsync** or MinIO **`mc mirror`**, then integrity audit before local prune ([03 — Data platform](03-data-platform.md)).
|
||
|
||
### rclone
|
||
|
||
Optional rsync-class tool when the far end is object storage and rsync is awkward; not the default in-flight mesh path ([09 — Open questions](09-open-questions.md) §12).
|
||
|
||
### ADR
|
||
|
||
Architecture Decision Record — a point-in-time decision for this repository ([adr/README.md](adr/README.md)).
|
||
|
||
### ASR
|
||
|
||
Architecture Standard Record — a standing cross-repository policy (lives in the ecosystem `inventar` repo, not here). See [adr/README.md](adr/README.md).
|
||
|
||
## Data & storage
|
||
|
||
| Term | Meaning |
|
||
| --- | --- |
|
||
| **Parquet** | Columnar file format; compresses time series extremely well (ZSTD + delta encoding), supports predicate pushdown |
|
||
| **Hive partitioning** | Directory layout `key=value/…` understood by most engines; enables partition pruning (only relevant directories are read) |
|
||
| **DuckDB** | In-process analytical SQL engine; queries Parquet globs directly, no server to operate |
|
||
| **ZSTD** | Zstandard compression; the default codec for all Parquet files here |
|
||
| **DWH** | Data warehouse — here: the on-prem ground store holding every flight ever flown; same role as [source of truth](#source-of-truth) |
|
||
| **Hot / warm / cold tiers** | Classic three-tier model: in-memory window → local NVMe → ground warehouse. Here mapped to [T0](#t0--hot), [T1](#t1--warm), and [T3](#t3--warehouse); [T2](#t2--shared) and [T4](#t4--dev) are named explicitly |
|
||
| **Sealing** | Compacting many small streamed Parquet files into one final file per partition once a time window closes |
|
||
| **UUIDv7** | Time-ordered UUID; sortable by creation time, used as the canonical flight identifier |
|
||
| **DuckDB spatial** | Extension adding a GEOMETRY type and `ST_*` functions; used for separation audits, perimeter predicates, and trajectory analysis in the local planar frame |
|
||
| **Schema registry** | Versioned catalog of dataset schemas so producers and consumers can evolve independently |
|
||
| **ETL** | Extract, transform, load — the pipelines that move and reshape data between tiers |
|
||
|
||
## Swarm & robotics
|
||
|
||
| Term | Meaning |
|
||
| --- | --- |
|
||
| **UAV / UGV / USV / UUV** | Unmanned aerial / ground / surface / underwater vehicle |
|
||
| **Swarm** | Group of autonomous vehicles coordinating without a central controller |
|
||
| **Sustainable pulsing** | Swarm ability to repeatedly engage a target area from multiple directions and self-reorganize as conditions evolve |
|
||
| **Ubiquitous sensing** | The swarm acting as one distributed sensor, fusing local observations into shared situational awareness ("top sight") |
|
||
| **Man-in-the-loop** | A human supervises intent and boundaries but does not steer individual vehicles |
|
||
| **GPS-denied** | Operating without satellite positioning; position comes from visual odometry and relative localization |
|
||
| **Visual odometry** | Estimating a vehicle's motion from its own camera stream |
|
||
| **ROS 2** | Robot Operating System 2 — the de-facto middleware for robotics software; nodes exchange typed messages over topics |
|
||
| **DDS** | Data Distribution Service — the default ROS 2 transport middleware |
|
||
| **Zenoh** | Pub/sub + query protocol designed for constrained, dynamic networks; official ROS 2 RMW implementation exists |
|
||
| **YOLO-like detector** | Family of single-pass object-detection models; the on-board video service runs one of them, weights are swappable |
|
||
| **IMU** | Inertial measurement unit — accelerometer + gyroscope (+ magnetometer) |
|
||
| **LiDAR** | Laser-based distance sensing |
|
||
| **RSSI** | Received signal strength indicator; Wi-Fi signal strength, usable as a coarse relative-positioning signal |
|
||
| **C2** | Command and control — the narrow channel that delivers mission intent to the swarm |
|
||
|
||
## Infrastructure & delivery
|
||
|
||
| Term | Meaning |
|
||
| --- | --- |
|
||
| **Air-gapped** | No route to the internet, physically or logically |
|
||
| **On-prem** | Runs on owned hardware; no public cloud involvement |
|
||
| **Zero trust** | No implicit trust from network position; every peer authenticates with pre-provisioned credentials |
|
||
| **mTLS** | Mutual TLS — both sides of a connection present certificates |
|
||
| **ed25519** | Modern elliptic-curve signature scheme; the SSH key type provisioned per device and per operation |
|
||
| **Forced command** | `command="…"` restriction in `authorized_keys`: the key can execute exactly one predefined operation — the key *is* the API |
|
||
| **ControlMaster** | SSH connection multiplexing: one persistent authenticated session carries many transfers/commands |
|
||
| **PKI** | Public key infrastructure — the certificate authority hierarchy behind mTLS |
|
||
| **WireGuard** | Lightweight encrypted tunnel protocol; basis for the swarm mesh overlay |
|
||
| **Ad-hoc mesh** | Wireless network formed directly between peers, no fixed infrastructure |
|
||
| **Docker Compose** | Declarative multi-container runtime; the only orchestrator on board a drone |
|
||
| **k3s** | Lightweight Kubernetes distribution; used on the ground only |
|
||
| **Dev Container** | Reproducible containerized development environment definition |
|
||
| **MinIO** | S3-compatible object store; runs on-board for derived data and on the ground as the warehouse backend |
|
||
| **Registry mirror** | Local copy of a container registry inside the air gap; drones pull images from it |
|
||
| **OTA** | Over-the-air update — delivered before a mission while docked, never mid-flight |
|
||
| **Fleet release manifest** | A versioned lockfile pinning every artifact (image digests, model weights, schemas, configs) that defines one fleet version |
|
||
| **SemVer** | Semantic versioning (`MAJOR.MINOR.PATCH`) applied to every artifact and to the fleet manifest itself |
|
||
| **Trivy** | Open-source scanner for container images and dependencies |
|
||
| **SonarQube** | Static code-quality and security analysis platform |
|
||
| **nvidia-container-toolkit** | Runtime hook exposing GPUs to containers; needed for on-board inference |
|
||
| **HIL** | Hardware-in-the-loop testing — real flight hardware wired into a simulated environment |
|
||
| **CRDT** | Conflict-free replicated data type; merges concurrent updates without coordination |
|