eSlider
cb4664f5ee
Ansible provisions the k3d cluster and Flux controllers; Terraform modules deploy the simulated fleet and ground warehouse. Compose and k3d share var/t1 (live lake) and var/t3 (warehouse). The prototype gains a live mode fed by the explorer read-only SQL API.
39 lines
1.7 KiB
Markdown
39 lines
1.7 KiB
Markdown
# Terraform — ground warehouse (T3)
|
|
|
|
The stationary half of the system: the historical warehouse that receives
|
|
sealed partitions after every flight ([03 — Storage tiers](../../../docs/03-storage-design.md)).
|
|
It is the longest-lived, most conventional infrastructure in the design —
|
|
and therefore the most natural Terraform territory.
|
|
|
|
Runs in the same k3d cluster as [`../sim-env`](../sim-env/) but in its own
|
|
namespace with its own state: the fleet is disposable, the warehouse is not,
|
|
and the two lifecycles must never share a `terraform destroy`.
|
|
|
|
| Resource | Purpose |
|
|
| --- | --- |
|
|
| Deployment `minio` + PVC + NodePort 30901 | Object-store facade of the warehouse for downstream consumers (training pipelines, replay) |
|
|
| CronJob `offload` | Post-flight offload: copies Hive partitions verbatim from the fleet lake (T1) into the warehouse (T3), then mirrors into MinIO |
|
|
| Deployment `warehouse-explorer` + NodePort 30089 | Second explorer instance over the warehouse path — historical read-only SQL across **all** flights |
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
terraform init
|
|
terraform apply
|
|
terraform output
|
|
```
|
|
|
|
After a couple of offload runs, the warehouse explorer shows the same
|
|
partition tree as the live lake but accumulated across flights — the T1
|
|
layout **is** the T3 layout, which is the whole point: no transform step,
|
|
no schema drift, DuckDB queries work identically on both ends.
|
|
|
|
## Simulation vs production
|
|
|
|
- The CronJob compresses "drone lands, docks, offloads, prunes" into a
|
|
periodic rsync-style copy. Production offload is event-driven per docking
|
|
and verifies checksums from the flight's partition manifest before pruning
|
|
the on-board lake.
|
|
- MinIO credentials here are throwaway defaults; production credentials are
|
|
sealed per site and never in state or VCS.
|