ci: recall@5 SoT is Zig bin/brain/eval.go (#19) (#32)
Tests / Test (push) Failing after 6s
Tests / Release (semver) (push) Skipped

* ci: recall@5 SoT is Zig bin/brain/eval.go.

GitHub Actions rebuilds the repo corpus and runs the Go eval gate
instead of Python bin/kb/eval. Audit self is a hard fail (Gitea #19).

* ci: point eval at the checkout and keep DevOps in the corpus.

/tmp/brain-eval cannot walk up to var/kb.lbug; KB_ROOT is required.
Recall fragments must exist in README/docs so a repo-only rebuild gates.
This commit is contained in:
2026-08-14 11:03:39 +01:00
committed by GitHub
co-authored by GitHub
parent c5be3f19be
commit bae1494258
5 changed files with 50 additions and 12 deletions
+13 -3
View File
@@ -54,13 +54,23 @@ jobs:
run: go test ./internal/brain/rank -count=1 run: go test ./internal/brain/rank -count=1
- name: facts/audit self (lexicon consistency, no network) - name: facts/audit self (lexicon consistency, no network)
run: | run: ./bin/facts/audit self
./bin/facts/audit self 2>/dev/null || echo "audit: not yet implemented; gate skipped"
- name: CGO via Zig (compile brain/search) - name: CGO via Zig (compile brain/search + eval)
run: | run: |
chmod +x bin/cgo/zig bin/cgo/zcc bin/cgo/zc++ chmod +x bin/cgo/zig bin/cgo/zcc bin/cgo/zc++
bin/cgo/zig go build -tags system_ladybug -o /tmp/brain-search ./bin/brain/search.go bin/cgo/zig go build -tags system_ladybug -o /tmp/brain-search ./bin/brain/search.go
bin/cgo/zig go build -tags 'system_ladybug,brain_eval' -o /tmp/brain-eval ./bin/brain/eval.go
- uses: actions/cache@v4
with:
path: ~/.cache/huggingface
key: ${{ runner.os }}-hf-potion-multilingual-128M
- name: recall@5 SoT (Zig bin/brain/eval.go)
run: |
uv run python bin/kb/index --rebuild --json
KB_ROOT="$PWD" /tmp/brain-eval --json
release: release:
name: Release (semver) name: Release (semver)
+4 -4
View File
@@ -144,8 +144,8 @@ Common props on every node/edge: `root`, `confidence`, `evidence[]`, `how`,
2. `go test ./internal/brain/rank` (cgo-free ranking + flag parser) 2. `go test ./internal/brain/rank` (cgo-free ranking + flag parser)
3. python -m unittest discover -s bin/tools (includes published-docs SoT) 3. python -m unittest discover -s bin/tools (includes published-docs SoT)
4. `bin/facts/audit self` (lexicon internal consistency; `bin/facts/audit.go` is the D14 wrapper) 4. `bin/facts/audit self` (lexicon internal consistency; `bin/facts/audit.go` is the D14 wrapper)
5. `bin/kb/eval` (recall@5 ≥ 0.95). Local SoT is `bin/brain/eval.go` via Zig CGO. 5. `bin/brain/eval.go` via Zig (recall@5 ≥ 0.95). Python `bin/kb/eval` is an
CI SoT switch: [#19](https://git.produktor.io/eSlider/2dph/issues/19). explicit fallback, not the CI SoT.
6. `bin/cgo/zig go build -tags system_ladybug` (compile search with zig cc; fetches pinned zig+libs). 6. `bin/cgo/zig go build -tags system_ladybug` (compile search with zig cc; fetches pinned zig+libs).
Feedback loop: every commit → PR → CI → green/gate → merge. Same discipline as Feedback loop: every commit → PR → CI → green/gate → merge. Same discipline as
@@ -164,7 +164,7 @@ Feedback loop: every commit → PR → CI → green/gate → merge. Same discipl
## Gap to v1 (epic #16) ## Gap to v1 (epic #16)
Remaining: CI eval SoT. Board: Remaining: none for epic #16 (v1). Board:
[epic #16](https://git.produktor.io/eSlider/2dph/issues/16), [epic #16](https://git.produktor.io/eSlider/2dph/issues/16),
milestone [v1 detective brain](https://git.produktor.io/eSlider/2dph/milestone/12). milestone [v1 detective brain](https://git.produktor.io/eSlider/2dph/milestone/12).
Narrative: [docs/roadmap.md](docs/roadmap.md). Narrative: [docs/roadmap.md](docs/roadmap.md).
@@ -175,6 +175,6 @@ Narrative: [docs/roadmap.md](docs/roadmap.md).
| 2 | [#17](https://git.produktor.io/eSlider/2dph/issues/17) | **in**`--hop N` walks `FROM_FILE``HAS_VERSION``AUTHORED` (max 3). | | 2 | [#17](https://git.produktor.io/eSlider/2dph/issues/17) | **in**`--hop N` walks `FROM_FILE``HAS_VERSION``AUTHORED` (max 3). |
| 3 | [#18](https://git.produktor.io/eSlider/2dph/issues/18) | **in**`--with-facts` / `--facts-json` land `root=facts`; `--with-chats` indexes `var/chats/md`. WhatsApp sync is out of v1. | | 3 | [#18](https://git.produktor.io/eSlider/2dph/issues/18) | **in**`--with-facts` / `--facts-json` land `root=facts`; `--with-chats` indexes `var/chats/md`. WhatsApp sync is out of v1. |
| 4 | [#15](https://git.produktor.io/eSlider/2dph/issues/15) | **in** — lever/loop documented (`search``get``audit`). | | 4 | [#15](https://git.produktor.io/eSlider/2dph/issues/15) | **in** — lever/loop documented (`search``get``audit`). |
| 5 | [#19](https://git.produktor.io/eSlider/2dph/issues/19) | GitHub CI recall still runs Python `bin/kb/eval`. | | 5 | [#19](https://git.produktor.io/eSlider/2dph/issues/19) | **in** — CI recall SoT is `bin/brain/eval.go` via Zig. Python `bin/kb/eval` stays as an explicit fallback. |
Does **not** block epic close: [#6](https://git.produktor.io/eSlider/2dph/issues/6) OCR, OQ1, OQ3, OQ4. Does **not** block epic close: [#6](https://git.produktor.io/eSlider/2dph/issues/6) OCR, OQ1, OQ3, OQ4.
+3
View File
@@ -169,6 +169,9 @@ docker compose up brain-watch # auto re-index on change
## Related ## Related
eSlider DevOps engineer practice: ops, OnlyOffice, and mail feed the facts
root through `bin/facts/extract` (two-source pairing).
- [go-second-brain](https://github.com/eSlider/go-second-brain) — the earlier - [go-second-brain](https://github.com/eSlider/go-second-brain) — the earlier
Neo4j + Qdrant + Matrix RAG brain Neo4j + Qdrant + Matrix RAG brain
- [agent-skills](https://github.com/eSlider/agent-skills) — upstream - [agent-skills](https://github.com/eSlider/agent-skills) — upstream
+22
View File
@@ -207,3 +207,25 @@ class BinLayoutTest(unittest.TestCase):
search = (ROOT / "bin" / "kb" / "search").read_text() search = (ROOT / "bin" / "kb" / "search").read_text()
self.assertIn("bin/cgo/zig", search) self.assertIn("bin/cgo/zig", search)
self.assertNotIn("command -v gcc", search) self.assertNotIn("command -v gcc", search)
def test_ci_recall_sot_is_zig_brain_eval(self) -> None:
ci = (ROOT / ".github" / "workflows" / "ci.yml").read_text()
self.assertIn("bin/brain/eval.go", ci)
self.assertIn("system_ladybug,brain_eval", ci)
self.assertIn("/tmp/brain-eval", ci)
self.assertIn("KB_ROOT", ci)
self.assertNotIn("bin/kb/eval", ci)
self.assertNotIn("gate skipped", ci)
self.assertIn("./bin/facts/audit self", ci)
def test_eval_fragments_live_in_default_corpus(self) -> None:
"""CI --rebuild indexes README/PLAN/docs/skills; fragments must be there."""
corpus = []
for rel in ("README.md", "PLAN.md", "AGENTS.md"):
corpus.append((ROOT / rel).read_text())
for d in ("docs", "skills"):
for p in (ROOT / d).rglob("*.md"):
corpus.append(p.read_text())
blob = "\n".join(corpus)
for frag in ("BM25", "DevOps", "LadybugDB"):
self.assertIn(frag, blob, f"{frag} must appear in default index corpus")
+8 -5
View File
@@ -26,11 +26,18 @@ Compose `api` (no CPython) / `index` (Python write). Issues #1#5, #7#13.
[#15](https://git.produktor.io/eSlider/2dph/issues/15) lever/loop. [#15](https://git.produktor.io/eSlider/2dph/issues/15) lever/loop.
[#14](https://git.produktor.io/eSlider/2dph/issues/14) `bin/brain/add.go` / [#14](https://git.produktor.io/eSlider/2dph/issues/14) `bin/brain/add.go` /
`POST /ingest` (Python `kblib.add_leafs`; no Go upsert port). `POST /ingest` (Python `kblib.add_leafs`; no Go upsert port).
[#17](https://git.produktor.io/eSlider/2dph/issues/17) `--hop N` walks
FROM_FILE / HAS_VERSION / AUTHORED.
[#18](https://git.produktor.io/eSlider/2dph/issues/18) `--with-facts` / [#18](https://git.produktor.io/eSlider/2dph/issues/18) `--with-facts` /
`--with-chats` on rebuild (WhatsApp out of v1). `--with-chats` on rebuild (WhatsApp out of v1).
[#19](https://git.produktor.io/eSlider/2dph/issues/19) CI recall SoT =
`bin/brain/eval.go` via Zig.
## Blockers ## Blockers
None for epic #16. v2: [#6](https://git.produktor.io/eSlider/2dph/issues/6) OCR,
OQ1 contradiction resolution, OQ3 duckdb-md, OQ4 YAML-first leafs.
``` ```
question question
@@ -42,9 +49,6 @@ question
└─ facts+chats corpus ← in └─ facts+chats corpus ← in
``` ```
1. **[#19](https://git.produktor.io/eSlider/2dph/issues/19) CI eval** —
recall SoT should be `bin/brain/eval.go` via Zig, not Python `bin/kb/eval`.
## Not v1 ## Not v1
[#6](https://git.produktor.io/eSlider/2dph/issues/6) OCR (OQ2), OQ1 [#6](https://git.produktor.io/eSlider/2dph/issues/6) OCR (OQ2), OQ1
@@ -52,5 +56,4 @@ contradiction resolution, OQ3 duckdb-md export, OQ4 YAML-first leafs.
## Close epic #16 when ## Close epic #16 when
- MCP tool order is documented and still gated by tests Children #14, #15, #17, #18, #19 are closed. MCP tool order stays gated by tests.
- CI recall SoT is `bin/brain/eval.go` via Zig