docs: audit links, glossary T0-T4, readability pass
CI & Release / Verify simulator (push) Skipped
CI & Release / Trivy scan (push) Skipped
CI & Release / Semantic Release (push) Skipped
CI & Release / Verify simulator (pull_request) Failing after 4s
CI & Release / Trivy scan (pull_request) Skipped
CI & Release / Semantic Release (pull_request) Skipped
CI & Release / Verify simulator (push) Skipped
CI & Release / Trivy scan (push) Skipped
CI & Release / Semantic Release (push) Skipped
CI & Release / Verify simulator (pull_request) Failing after 4s
CI & Release / Trivy scan (pull_request) Skipped
CI & Release / Semantic Release (pull_request) Skipped
Extend glossary with storage floors, source of truth, and linkable anchors. Fix broken links, pose frame drift, rsync/offload wording, and README tone. Add docs/improvement audit records and live demo links in design journey.
This commit is contained in:
+66
-2
@@ -2,6 +2,70 @@
|
||||
|
||||
Terms and abbreviations used throughout this proposal.
|
||||
|
||||
**Vocabulary guide:** **architecture layers** (1–3) = on-board software stack; **storage floors** (T0–T4) = retention/latency ladder; **pipeline stages** = narrative capture → reduction → sync only ([12 — Design journey](12-design-journey.md) §4). Do not mix these three.
|
||||
|
||||
## Project vocabulary
|
||||
|
||||
### T0 — hot
|
||||
|
||||
RAM / DuckDB in-process sliding window of the last minutes; what mission logic queries in flight. Retention: minutes. Maps to **hot** in the classic three-tier model.
|
||||
|
||||
### T1 — warm
|
||||
|
||||
Drone NVMe: full raw `telemetry` + `detections` + `state` of the current flight; never leaves the drone in flight. Retention: current flight plus quota-based headroom. Maps to **warm**. Runtime path: [`var/t1/`](../var/t1/) ([ADR-0008](adr/ADR-0008-runtime-data-paths.md)).
|
||||
|
||||
### T2 — shared
|
||||
|
||||
On-board MinIO bucket: derived data only (`detections`, `state`), replicated opportunistically across the swarm. Retention: current mission.
|
||||
|
||||
### T3 — warehouse
|
||||
|
||||
Ground on-prem object store + Parquet lakehouse: every flight of every drone, forever; replay, analytics, model training. Retention: years. Maps to **cold** / **DWH**. Runtime path: [`var/t3/`](../var/t3/) ([ADR-0008](adr/ADR-0008-runtime-data-paths.md)). After offload and integrity audit, T3 is the [source of truth](#source-of-truth) for historical flight data.
|
||||
|
||||
### T4 — dev
|
||||
|
||||
Engineer laptop or sim farm: slices pulled from T3, or synthetic data from the simulator. Retention: ephemeral.
|
||||
|
||||
### Source of truth
|
||||
|
||||
The authoritative copy of flight data after [offload](#offload) and integrity audit: the [T3](#t3--warehouse) warehouse. Each drone holds its own partial record in flight ([T1](#t1--warm)); peers exchange derived state only ([04 — Swarm sync](04-swarm-sync.md)). Replay, post-flight fusion, training, and audit read from T3 — not from any single drone's local store. See [06 — Environments](06-environments.md), [ADR-0003](adr/ADR-0003-sync-derived-state-only.md).
|
||||
|
||||
### Architecture layer 1 — ingestion
|
||||
|
||||
On-board services that capture sensor streams and video analytics ([02 — Architecture](02-architecture.md)): `sensor-ingest`, `video-analytics`.
|
||||
|
||||
### Architecture layer 2 — storage and transform
|
||||
|
||||
On-board Parquet write path, sealer, DuckDB, NVMe ([02 — Architecture](02-architecture.md)).
|
||||
|
||||
### Architecture layer 3 — serving and sync
|
||||
|
||||
On-board event hook, pub/sub broadcast, MinIO derived bucket, peer query API ([02 — Architecture](02-architecture.md)).
|
||||
|
||||
### Pipeline stage
|
||||
|
||||
Narrative inside each drone: (1) raw capture, (2) ETL/reduction, (3) bidirectional sync interface. Not the same as storage floors T0–T4 ([12 — Design journey](12-design-journey.md) §4).
|
||||
|
||||
### Bulk sync
|
||||
|
||||
In-flight pull of sealed `detections`/`state` Parquet partitions between peers via **rsync** delta over persistent SSH forced commands ([04 — Swarm sync](04-swarm-sync.md)).
|
||||
|
||||
### Offload
|
||||
|
||||
Post-flight mirror of sealed T1 partitions into T3 at the dock: **rsync** or MinIO **`mc mirror`**, then integrity audit before local prune ([03 — Data platform](03-data-platform.md)).
|
||||
|
||||
### rclone
|
||||
|
||||
Optional rsync-class tool when the far end is object storage and rsync is awkward; not the default in-flight mesh path ([09 — Open questions](09-open-questions.md) §12).
|
||||
|
||||
### ADR
|
||||
|
||||
Architecture Decision Record — a point-in-time decision for this repository ([adr/README.md](adr/README.md)).
|
||||
|
||||
### ASR
|
||||
|
||||
Architecture Standard Record — a standing cross-repository policy (lives in the ecosystem `inventar` repo, not here). See [adr/README.md](adr/README.md).
|
||||
|
||||
## Data & storage
|
||||
|
||||
| Term | Meaning |
|
||||
@@ -10,8 +74,8 @@ Terms and abbreviations used throughout this proposal.
|
||||
| **Hive partitioning** | Directory layout `key=value/…` understood by most engines; enables partition pruning (only relevant directories are read) |
|
||||
| **DuckDB** | In-process analytical SQL engine; queries Parquet globs directly, no server to operate |
|
||||
| **ZSTD** | Zstandard compression; the default codec for all Parquet files here |
|
||||
| **DWH** | Data warehouse — here: the on-prem ground store holding every flight ever flown |
|
||||
| **Hot / warm / cold tiers** | Storage floors ordered by access latency and retention: in-memory window → local NVMe → ground warehouse |
|
||||
| **DWH** | Data warehouse — here: the on-prem ground store holding every flight ever flown; same role as [source of truth](#source-of-truth) |
|
||||
| **Hot / warm / cold tiers** | Classic three-tier model: in-memory window → local NVMe → ground warehouse. Here mapped to [T0](#t0--hot), [T1](#t1--warm), and [T3](#t3--warehouse); [T2](#t2--shared) and [T4](#t4--dev) are named explicitly |
|
||||
| **Sealing** | Compacting many small streamed Parquet files into one final file per partition once a time window closes |
|
||||
| **UUIDv7** | Time-ordered UUID; sortable by creation time, used as the canonical flight identifier |
|
||||
| **DuckDB spatial** | Extension adding a GEOMETRY type and `ST_*` functions; used for separation audits, perimeter predicates, and trajectory analysis in the local planar frame |
|
||||
|
||||
Reference in New Issue
Block a user