From 523e15f4901f66e3875b8b5ce6f2aa1e206dfb17 Mon Sep 17 00:00:00 2001 From: Andriy Oblivantsev Date: Wed, 8 Jul 2026 13:29:41 +0100 Subject: [PATCH] docs: adopt read-only SQL-over-SSH as the peer query decision Record the full reasoning path (GraphQL service, Arrow Flight, raw SSH, forced-command synthesis) and specify read-only enforcement as five fail-safe layers: key-per-operation, SELECT-only statement gate, read-only OS user, engine hardening, resource caps. Promote 'SQL is the contract' to a top-level design principle. --- README.md | 1 + docs/02-architecture.md | 2 +- docs/04-swarm-sync.md | 36 ++++++++++++++++++++++++++++++------ 3 files changed, 32 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 4a2c797..23df4b5 100644 --- a/README.md +++ b/README.md @@ -14,6 +14,7 @@ This repository describes how to build, deliver, test, and operate that platform 4. **Event-driven, not polled.** New derived data triggers downstream actions through hooks; nothing burns cycles asking "anything new yet?". 5. **Zero trust, provisioned ahead.** Keys are issued per device before deployment; nothing joins the swarm at runtime. Dev channels are physically absent from production builds. 6. **The whole fleet has one version.** A release manifest pins every image digest, model weight, schema, and config. Rollback is atomic. +7. **SQL is the contract.** Peer data access is read-only DuckDB SQL over SSH forced commands — the query language already lives on both ends, so no service, port, or protocol is invented for it. ## Documentation diff --git a/docs/02-architecture.md b/docs/02-architecture.md index cd9e219..c2aff81 100644 --- a/docs/02-architecture.md +++ b/docs/02-architecture.md @@ -56,7 +56,7 @@ graph TB - The **event hook** is the on-board "lambda": when the writer lands new *derived* rows (state, detections), it triggers registered actions — broadcast, MinIO upload, or a local mission-logic callback. Nothing polls. - The **state publisher** broadcasts compact position/attitude/detection payloads over the mesh pub/sub (transport analysis in [04 — Swarm sync](04-swarm-sync.md)). - **Bulk sync** pulls sealed derived partitions from peers over persistent SSH (rsync delta transfer); MinIO remains optional where an S3 API is wanted. -- **Peer queries** run against local DuckDB via SSH forced commands — allow-listed query templates, columnar responses (the standard is in [04](04-swarm-sync.md)). +- **Peer queries** are read-only DuckDB SQL over SSH forced commands — SELECT-only gate, read-only OS user, columnar responses (the design decision and its reasoning are in [04](04-swarm-sync.md)). ## Communication planes diff --git a/docs/04-swarm-sync.md b/docs/04-swarm-sync.md index e8e8423..c7c4f38 100644 --- a/docs/04-swarm-sync.md +++ b/docs/04-swarm-sync.md @@ -85,14 +85,38 @@ Two deliberate non-features of the SSH channel: - **No remote filesystem mounts in flight.** FUSE/sshfs over a lossy mesh inherits NFS hang semantics — processes block uninterruptibly when the link drops. Mounts are fine on the bench and at the dock; in flight, data moves by pull, never by mount. - **No free-form remote execution.** Arbitrary drone-to-drone exec would mean one compromised unit owns the fleet. Instead, every permitted operation is an **SSH forced command** (`command="…"` in `authorized_keys`, one key pair per operation): the key *is* the API. Flexible like a CLI, auditable like an RPC, least-privilege by construction. -## Peer query standard +## Design decision: SQL-over-SSH as the peer query channel -For everything that is not broadcast, drones (and mission services) query each other explicitly. The contract: +**Decision: the in-flight peer query API is read-only DuckDB SQL executed through a dedicated SSH forced command.** This section records how the design got there, because the reasoning is as important as the result. -- **Schema-first:** all datasets share the versioned schemas from [03](03-data-platform.md); a query addresses `dataset + partition predicates + column projection + time range`. -- **In flight — query over forced command:** a peer query is a DuckDB statement (from an allow-listed template set) executed via its dedicated SSH forced command, streaming the result back as Parquet/Arrow over the same multiplexed session. No server process, no new port, no new protocol — and the receiving side lands data straight into its own store. -- **On the ground — GraphQL as the integration standard:** the warehouse exposes the same schemas through GraphQL for the multi-team surface, where flexibility and introspection matter more than grams and milliwatts. **Arrow Flight** is the upgrade path for heavy columnar serving from T3 if GraphQL result sizes become the bottleneck. -- Responses are always columnar batches (Parquet/Arrow), never JSON blobs. +### The path to the decision + +1. **A GraphQL service on every drone** was the first candidate: one flexible, introspectable, schema-first contract. Rejected for the flight side — it means a resolver service, an HTTP stack, and a port on every flight-critical node, all to re-express what is already expressible in SQL against the local store. (GraphQL survives as the *ground* integration standard, where those costs are irrelevant.) +2. **Arrow Flight** — efficient columnar RPC, but again a server process and a protocol surface on board; its sweet spot is heavy serving from the ground warehouse, not grams-and-milliwatts peer queries. +3. **"Just give drones SSH to each other"** — the counter-idea: the transport already exists (ed25519, pinned known_hosts, one multiplexed session for bulk sync), so why not direct CLI access? Objection: free-form shell means one compromised drone owns the fleet; and remote mounts hang on lossy links. So — not raw SSH. +4. **The synthesis: SSH forced commands.** Keep the zero-extra-infrastructure transport, remove the general-purpose shell. Each `authorized_keys` entry binds a key to exactly one command; the query key runs exactly one thing: a read-only query wrapper. The peer's "API surface" is the set of keys it has provisioned — versioned, auditable, revocable per operation ([05](05-network-security.md), [11](11-cicd-delivery.md)). + +What made the final option win: **the query language already lives on both ends.** Every drone runs DuckDB over the same versioned schemas; SQL *is* the contract. No IDL, no resolver layer, no serialization format to invent — the response is a Parquet/Arrow stream that the requesting side lands directly into its own store. + +### Read-only, enforced in layers + +SQL access must not become a write channel. A single "read-only connection" flag is not enough — DuckDB can materialize files via `COPY TO`, attach databases, or read outside the data root. Defense in depth, any single layer failing safe: + +| Layer | Mechanism | What it stops | +| --- | --- | --- | +| 1. Key = operation | Forced command: the query key can only invoke the query wrapper, nothing else | Arbitrary exec, lateral movement | +| 2. Statement gate | Wrapper accepts a single statement, parses it, rejects anything but `SELECT` (no `COPY`, `ATTACH`, `INSTALL`, `SET`, multi-statements); parameters bound, not interpolated | SQL-as-a-write-channel, config tampering | +| 3. OS permissions | Wrapper runs as a dedicated user with **read-only filesystem access** to the data root and write access to nothing | Any write that slips past layer 2 | +| 4. Engine hardening | `:memory:` database, external access disabled except the data-root glob, extension loading off | Reaching outside the store | +| 5. Resource caps | Timeout, memory cap, niced CPU (flight software always wins), response size budget | Denial of service via expensive queries | + +### The contract + +- **Schema-first:** all datasets share the versioned schemas from [03](03-data-platform.md); a query addresses `dataset + partition predicates + column projection + time range`. Schema version is negotiated from the fleet release manifest — peers on one release speak one schema by construction. +- **Read-only SELECT over the local Parquet store**, streamed back as Parquet/Arrow batches over the multiplexed SSH session. Never JSON blobs. +- **On the ground — GraphQL as the integration standard:** the warehouse exposes the same schemas through GraphQL for the multi-team surface, where flexibility and introspection matter more than grams and milliwatts. **Arrow Flight** is the upgrade path for heavy columnar serving from T3 if result sizes outgrow it. + +What this buys operationally: the dev/bench experience and the flight protocol are the same thing. An engineer debugging on the bench runs the identical query a peer drone would run — same wrapper, same permissions, same output format. There is no "debug API" that behaves differently from the real one. ## Event-driven end to end