eSlider
5f6fd8462c
CI & Release / Verify simulator (push) Skipped
CI & Release / Trivy scan (push) Skipped
CI & Release / Semantic Release (push) Skipped
CI & Release / Verify simulator (pull_request) Failing after 4s
CI & Release / Trivy scan (pull_request) Skipped
CI & Release / Semantic Release (pull_request) Skipped
Extend glossary with storage floors, source of truth, and linkable anchors. Fix broken links, pose frame drift, rsync/offload wording, and README tone. Add docs/improvement audit records and live demo links in design journey.
27 lines
1.5 KiB
Markdown
27 lines
1.5 KiB
Markdown
# Terraform — ground warehouse ([T3](../../../docs/00-glossary.md#t3--warehouse))
|
|
|
|
Ground warehouse. Receives sealed partitions post-flight ([storage floors](../../../docs/03-data-platform.md#storage-floors-tiers)). Long-lived infra → Terraform.
|
|
|
|
Runs in same k3d cluster as [`../sim-env`](../sim-env/) but separate namespace and state: fleet disposable, warehouse not — never share `terraform destroy`.
|
|
|
|
| Resource | Purpose |
|
|
| --- | --- |
|
|
| Deployment `minio` + PVC + NodePort 30901 | Object-store facade for downstream consumers |
|
|
| CronJob `offload` | [T1](../../../docs/00-glossary.md#t1--warm) → [T3](../../../docs/00-glossary.md#t3--warehouse) offload: Hive partitions verbatim, then `mc mirror` into MinIO |
|
|
| Deployment `warehouse-explorer` + NodePort 30089 | Read-only SQL across **all** offloaded flights |
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
terraform init
|
|
terraform apply
|
|
terraform output
|
|
```
|
|
|
|
After offload runs, warehouse explorer shows same partition tree as live lake, accumulated across flights. [T1](../../../docs/00-glossary.md#t1--warm) layout **is** [T3](../../../docs/00-glossary.md#t3--warehouse) layout. No transform. No schema drift. Same DuckDB queries both ends.
|
|
|
|
## Simulation vs production
|
|
|
|
- Sim CronJob: `cp -ru` + `mc mirror` on a schedule — shortcut for "lands, offloads, prunes". Production: event-driven per dock, rsync or `mc mirror`, checksum audit before prune ([03 — Data platform](../../../docs/03-data-platform.md)).
|
|
- MinIO credentials here are throwaway defaults; production credentials sealed per site, never in state or VCS.
|