feat: DuckDB quantiles in-process (gcc CGO), not Ladybug.
Tests / Test (push) Skipped
Tests / OCR (tesseract fixture) (push) Skipped
Tests / Release (semver) (push) Skipped
Tests / Test (pull_request) Failing after 5s
Tests / OCR (tesseract fixture) (pull_request) Failing after 4s
Tests / Release (semver) (pull_request) Skipped

OQ3: internal/duckstats + bin/qa/stats.go. Zig stays Ladybug-only.
mikefarah/yq for small structured slices. Gitea #30.
This commit is contained in:
2026-08-14 11:29:22 +01:00
parent f99dfea104
commit 168a0a3b53
28 changed files with 745 additions and 46 deletions
+38
View File
@@ -0,0 +1,38 @@
---
name: duckdb
description: >-
Use https://github.com/duckdb/duckdb-go in-process for columnar analytics
(quantiles, GROUP BY, JSON/CSV/Parquet/JSONL scans) when that is faster than
nested Go loops. Not Ladybug. Not the web-search sqlite cache. Use when
aggregating samples, counting JSONL, or SQL over tabular files.
---
# duckdb-go
Use https://github.com/duckdb/duckdb-go where it makes sense to get better performance in code.
In-process DuckDB (`internal/duckstats`, `database/sql` driver `duckdb`).
Vectorized SQL over tables, JSONL, CSV, Parquet. CGO with bundled libs
(linux/darwin amd64/arm64). Links with **gcc/g++** (libstdc++), not Zig.
D21 Zig (`bin/cgo/zcc`) is Ladybug/tokenizers only. After
`eval "$(bin/cgo/zig env)"`:
```bash
CC=gcc CXX=g++ CGO_CFLAGS= CGO_LDFLAGS= ./bin/qa/stats.go <<< '[1,2,3,4,5]'
CC=gcc CXX=g++ CGO_CFLAGS= CGO_LDFLAGS= go test ./internal/duckstats
```
| Store | Job |
|-------|-----|
| Ladybug | graph + FTS + HNSW (facts/info) |
| modernc sqlite | web-search KV cache + throttle |
| duckdb-go | OLAP: quantiles, counts, scans of many rows/files |
| mikefarah/yq | small YAML/JSON/XML/CSV/TOML/HCL slice, not bulk |
```bash
./bin/qa/stats.go <<< '[1,2,3,4,5]'
./bin/qa/stats.go --jsonl path/to/rows.jsonl
```
Do not open Ladybug through DuckDB. Do not put secrets or client PII into
DuckDB files under the repo.
+5 -5
View File
@@ -1,15 +1,15 @@
---
name: picoclaw
description: >-
2dph is the memory/fact gate, not the agent loop. Use when wiring PicoClaw
or any MCP client: call brain search/get/audit before a factual reply.
throttled is not a negative finding.
2dph is the memory/fact gate. Compose runs the official PicoClaw gateway.
Use when wiring PicoClaw or any MCP client: call brain search/get/audit
before a factual reply. throttled is not a negative finding.
---
# PicoClaw — fact-check before assert
PicoClaw (or any agent) speaks MCP at `POST /mcp` on `bin/brain/serve.go`.
2dph does not run the agent loop. Compose: `docker compose --profile picoclaw up brain-mcp`
PicoClaw speaks MCP at `POST /mcp` on `bin/brain/serve.go`. Compose profile
`picoclaw` runs the official `sipeed/picoclaw` gateway plus `brain-mcp`
(see [docs/picoclaw.md](../../docs/picoclaw.md)).
## Tool order (before a factual reply)
+1 -1
View File
@@ -9,7 +9,7 @@ description: >-
# postgres
`bin/postgres/query.go` wraps vendored `bin/db/psql-yq`. Output is YAML
(cheaper than psql ASCII, easy to slice with `yq`).
(cheaper than psql ASCII, easy to slice with mikefarah/yq).
```bash
bin/postgres/query.go --profile onlyoffice -s document_asset # column list
+1 -1
View File
@@ -11,7 +11,7 @@ description: >-
```bash
bin/web/search.go "LadybugDB vector index"
bin/web/search.go "model2vec multilingual" --category it
bin/web/search.go "hypervisor" --site example.com --json | jq -r '.results[].url'
bin/web/search.go "hypervisor" --site example.com --json | yq -r '.results[].url'
bin/web/search.go "postgres partial index" --lang en --fresh year
```
@@ -43,7 +43,7 @@ found" unless the client refuses to call it absence.
```bash
for i in $(seq 10); do
bin/web/search.go "test $i" -n 1 --refresh --json | jq -r .status
bin/web/search.go "test $i" -n 1 --refresh --json | yq -r '.status'
done
```
+33
View File
@@ -0,0 +1,33 @@
---
name: yq
description: >-
Use https://github.com/mikefarah/yq to work with YAML, JSON, XML, CSV,
TOML, HCL where it's efficient and less code. Use when slicing compose,
config, --json tool output, CSV/TOML/HCL/XML, or converting between those
formats. Not kislyuk Python yq. Not jq when yq already does the job.
---
# yq (mikefarah)
Use https://github.com/mikefarah/yq to work with YAML, JSON, XML, CSV, TOML, HCL where it's efficient and less code.
This is the Go `yq` (`yq --version` contains `mikefarah`). It is not
kislyuk/yq (Python, jq-syntax, YAML-only wrapper). `bin/db/psql-yq` already
calls this binary.
Prefer `yq` over `python3 -c`, `jq`, or ad-hoc parsers when one expression
reads or converts the file. Keep Python/Go for HTTP, binary protocols, and
in-process tests.
```bash
yq '.services.picoclaw.image' compose.yaml
yq -P . deploy/picoclaw/config.json # JSON → YAML
yq -o=json '.gates' # JSON stdin (qa/system_perf.py --json)
yq -p=csv -o=json .
yq -p=xml -o=json .
yq -p=toml '.package.name' file.toml
bin/brain/search.go "LadybugDB" --json | yq '.[].ref'
bin/web/search.go "hypervisor" --json | yq -r '.results[].url'
```
Do not print secrets, PII, or `$HOME/.config/brain/` through `yq`.