eSliderandGitHub
a331042488
OQ3: internal/duckstats + bin/qa/stats.go. Zig stays Ladybug-only. mikefarah/yq for small structured slices. Gitea #30.
1.3 KiB
1.3 KiB
name, description
| name | description |
|---|---|
| duckdb | Use https://github.com/duckdb/duckdb-go in-process for columnar analytics (quantiles, GROUP BY, JSON/CSV/Parquet/JSONL scans) when that is faster than nested Go loops. Not Ladybug. Not the web-search sqlite cache. Use when aggregating samples, counting JSONL, or SQL over tabular files. |
duckdb-go
Use https://github.com/duckdb/duckdb-go where it makes sense to get better performance in code.
In-process DuckDB (internal/duckstats, database/sql driver duckdb).
Vectorized SQL over tables, JSONL, CSV, Parquet. CGO with bundled libs
(linux/darwin amd64/arm64). Links with gcc/g++ (libstdc++), not Zig.
D21 Zig (bin/cgo/zcc) is Ladybug/tokenizers only. After
eval "$(bin/cgo/zig env)":
CC=gcc CXX=g++ CGO_CFLAGS= CGO_LDFLAGS= ./bin/qa/stats.go <<< '[1,2,3,4,5]'
CC=gcc CXX=g++ CGO_CFLAGS= CGO_LDFLAGS= go test ./internal/duckstats
| Store | Job |
|---|---|
| Ladybug | graph + FTS + HNSW (facts/info) |
| modernc sqlite | web-search KV cache + throttle |
| duckdb-go | OLAP: quantiles, counts, scans of many rows/files |
| mikefarah/yq | small YAML/JSON/XML/CSV/TOML/HCL slice, not bulk |
./bin/qa/stats.go <<< '[1,2,3,4,5]'
./bin/qa/stats.go --jsonl path/to/rows.jsonl
Do not open Ladybug through DuckDB. Do not put secrets or client PII into DuckDB files under the repo.