Compare commits
2
Commits
main
...
feat/ci-eval
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2332613c43 | ||
|
|
8feb93da78 |
@@ -54,13 +54,23 @@ jobs:
|
|||||||
run: go test ./internal/brain/rank -count=1
|
run: go test ./internal/brain/rank -count=1
|
||||||
|
|
||||||
- name: facts/audit self (lexicon consistency, no network)
|
- name: facts/audit self (lexicon consistency, no network)
|
||||||
run: |
|
run: ./bin/facts/audit self
|
||||||
./bin/facts/audit self 2>/dev/null || echo "audit: not yet implemented; gate skipped"
|
|
||||||
|
|
||||||
- name: CGO via Zig (compile brain/search)
|
- name: CGO via Zig (compile brain/search + eval)
|
||||||
run: |
|
run: |
|
||||||
chmod +x bin/cgo/zig bin/cgo/zcc bin/cgo/zc++
|
chmod +x bin/cgo/zig bin/cgo/zcc bin/cgo/zc++
|
||||||
bin/cgo/zig go build -tags system_ladybug -o /tmp/brain-search ./bin/brain/search.go
|
bin/cgo/zig go build -tags system_ladybug -o /tmp/brain-search ./bin/brain/search.go
|
||||||
|
bin/cgo/zig go build -tags 'system_ladybug,brain_eval' -o /tmp/brain-eval ./bin/brain/eval.go
|
||||||
|
|
||||||
|
- uses: actions/cache@v4
|
||||||
|
with:
|
||||||
|
path: ~/.cache/huggingface
|
||||||
|
key: ${{ runner.os }}-hf-potion-multilingual-128M
|
||||||
|
|
||||||
|
- name: recall@5 SoT (Zig bin/brain/eval.go)
|
||||||
|
run: |
|
||||||
|
uv run python bin/kb/index --rebuild --json
|
||||||
|
KB_ROOT="$PWD" /tmp/brain-eval --json
|
||||||
|
|
||||||
release:
|
release:
|
||||||
name: Release (semver)
|
name: Release (semver)
|
||||||
|
|||||||
@@ -144,8 +144,8 @@ Common props on every node/edge: `root`, `confidence`, `evidence[]`, `how`,
|
|||||||
2. `go test ./internal/brain/rank` (cgo-free ranking + flag parser)
|
2. `go test ./internal/brain/rank` (cgo-free ranking + flag parser)
|
||||||
3. python -m unittest discover -s bin/tools (includes published-docs SoT)
|
3. python -m unittest discover -s bin/tools (includes published-docs SoT)
|
||||||
4. `bin/facts/audit self` (lexicon internal consistency; `bin/facts/audit.go` is the D14 wrapper)
|
4. `bin/facts/audit self` (lexicon internal consistency; `bin/facts/audit.go` is the D14 wrapper)
|
||||||
5. `bin/kb/eval` (recall@5 ≥ 0.95). Local SoT is `bin/brain/eval.go` via Zig CGO.
|
5. `bin/brain/eval.go` via Zig (recall@5 ≥ 0.95). Python `bin/kb/eval` is an
|
||||||
CI SoT switch: [#19](https://git.produktor.io/eSlider/2dph/issues/19).
|
explicit fallback, not the CI SoT.
|
||||||
6. `bin/cgo/zig go build -tags system_ladybug` (compile search with zig cc; fetches pinned zig+libs).
|
6. `bin/cgo/zig go build -tags system_ladybug` (compile search with zig cc; fetches pinned zig+libs).
|
||||||
|
|
||||||
Feedback loop: every commit → PR → CI → green/gate → merge. Same discipline as
|
Feedback loop: every commit → PR → CI → green/gate → merge. Same discipline as
|
||||||
@@ -164,7 +164,7 @@ Feedback loop: every commit → PR → CI → green/gate → merge. Same discipl
|
|||||||
|
|
||||||
## Gap to v1 (epic #16)
|
## Gap to v1 (epic #16)
|
||||||
|
|
||||||
Remaining: CI eval SoT. Board:
|
Remaining: none for epic #16 (v1). Board:
|
||||||
[epic #16](https://git.produktor.io/eSlider/2dph/issues/16),
|
[epic #16](https://git.produktor.io/eSlider/2dph/issues/16),
|
||||||
milestone [v1 detective brain](https://git.produktor.io/eSlider/2dph/milestone/12).
|
milestone [v1 detective brain](https://git.produktor.io/eSlider/2dph/milestone/12).
|
||||||
Narrative: [docs/roadmap.md](docs/roadmap.md).
|
Narrative: [docs/roadmap.md](docs/roadmap.md).
|
||||||
@@ -175,6 +175,6 @@ Narrative: [docs/roadmap.md](docs/roadmap.md).
|
|||||||
| 2 | [#17](https://git.produktor.io/eSlider/2dph/issues/17) | **in** — `--hop N` walks `FROM_FILE` → `HAS_VERSION` → `AUTHORED` (max 3). |
|
| 2 | [#17](https://git.produktor.io/eSlider/2dph/issues/17) | **in** — `--hop N` walks `FROM_FILE` → `HAS_VERSION` → `AUTHORED` (max 3). |
|
||||||
| 3 | [#18](https://git.produktor.io/eSlider/2dph/issues/18) | **in** — `--with-facts` / `--facts-json` land `root=facts`; `--with-chats` indexes `var/chats/md`. WhatsApp sync is out of v1. |
|
| 3 | [#18](https://git.produktor.io/eSlider/2dph/issues/18) | **in** — `--with-facts` / `--facts-json` land `root=facts`; `--with-chats` indexes `var/chats/md`. WhatsApp sync is out of v1. |
|
||||||
| 4 | [#15](https://git.produktor.io/eSlider/2dph/issues/15) | **in** — lever/loop documented (`search` → `get` → `audit`). |
|
| 4 | [#15](https://git.produktor.io/eSlider/2dph/issues/15) | **in** — lever/loop documented (`search` → `get` → `audit`). |
|
||||||
| 5 | [#19](https://git.produktor.io/eSlider/2dph/issues/19) | GitHub CI recall still runs Python `bin/kb/eval`. |
|
| 5 | [#19](https://git.produktor.io/eSlider/2dph/issues/19) | **in** — CI recall SoT is `bin/brain/eval.go` via Zig. Python `bin/kb/eval` stays as an explicit fallback. |
|
||||||
|
|
||||||
Does **not** block epic close: [#6](https://git.produktor.io/eSlider/2dph/issues/6) OCR, OQ1, OQ3, OQ4.
|
Does **not** block epic close: [#6](https://git.produktor.io/eSlider/2dph/issues/6) OCR, OQ1, OQ3, OQ4.
|
||||||
@@ -169,6 +169,9 @@ docker compose up brain-watch # auto re-index on change
|
|||||||
|
|
||||||
## Related
|
## Related
|
||||||
|
|
||||||
|
eSlider DevOps engineer practice: ops, OnlyOffice, and mail feed the facts
|
||||||
|
root through `bin/facts/extract` (two-source pairing).
|
||||||
|
|
||||||
- [go-second-brain](https://github.com/eSlider/go-second-brain) — the earlier
|
- [go-second-brain](https://github.com/eSlider/go-second-brain) — the earlier
|
||||||
Neo4j + Qdrant + Matrix RAG brain
|
Neo4j + Qdrant + Matrix RAG brain
|
||||||
- [agent-skills](https://github.com/eSlider/agent-skills) — upstream
|
- [agent-skills](https://github.com/eSlider/agent-skills) — upstream
|
||||||
|
|||||||
@@ -207,3 +207,25 @@ class BinLayoutTest(unittest.TestCase):
|
|||||||
search = (ROOT / "bin" / "kb" / "search").read_text()
|
search = (ROOT / "bin" / "kb" / "search").read_text()
|
||||||
self.assertIn("bin/cgo/zig", search)
|
self.assertIn("bin/cgo/zig", search)
|
||||||
self.assertNotIn("command -v gcc", search)
|
self.assertNotIn("command -v gcc", search)
|
||||||
|
|
||||||
|
def test_ci_recall_sot_is_zig_brain_eval(self) -> None:
|
||||||
|
ci = (ROOT / ".github" / "workflows" / "ci.yml").read_text()
|
||||||
|
self.assertIn("bin/brain/eval.go", ci)
|
||||||
|
self.assertIn("system_ladybug,brain_eval", ci)
|
||||||
|
self.assertIn("/tmp/brain-eval", ci)
|
||||||
|
self.assertIn("KB_ROOT", ci)
|
||||||
|
self.assertNotIn("bin/kb/eval", ci)
|
||||||
|
self.assertNotIn("gate skipped", ci)
|
||||||
|
self.assertIn("./bin/facts/audit self", ci)
|
||||||
|
|
||||||
|
def test_eval_fragments_live_in_default_corpus(self) -> None:
|
||||||
|
"""CI --rebuild indexes README/PLAN/docs/skills; fragments must be there."""
|
||||||
|
corpus = []
|
||||||
|
for rel in ("README.md", "PLAN.md", "AGENTS.md"):
|
||||||
|
corpus.append((ROOT / rel).read_text())
|
||||||
|
for d in ("docs", "skills"):
|
||||||
|
for p in (ROOT / d).rglob("*.md"):
|
||||||
|
corpus.append(p.read_text())
|
||||||
|
blob = "\n".join(corpus)
|
||||||
|
for frag in ("BM25", "DevOps", "LadybugDB"):
|
||||||
|
self.assertIn(frag, blob, f"{frag} must appear in default index corpus")
|
||||||
|
|||||||
+8
-5
@@ -26,11 +26,18 @@ Compose `api` (no CPython) / `index` (Python write). Issues #1–#5, #7–#13.
|
|||||||
[#15](https://git.produktor.io/eSlider/2dph/issues/15) lever/loop.
|
[#15](https://git.produktor.io/eSlider/2dph/issues/15) lever/loop.
|
||||||
[#14](https://git.produktor.io/eSlider/2dph/issues/14) `bin/brain/add.go` /
|
[#14](https://git.produktor.io/eSlider/2dph/issues/14) `bin/brain/add.go` /
|
||||||
`POST /ingest` (Python `kblib.add_leafs`; no Go upsert port).
|
`POST /ingest` (Python `kblib.add_leafs`; no Go upsert port).
|
||||||
|
[#17](https://git.produktor.io/eSlider/2dph/issues/17) `--hop N` walks
|
||||||
|
FROM_FILE / HAS_VERSION / AUTHORED.
|
||||||
[#18](https://git.produktor.io/eSlider/2dph/issues/18) `--with-facts` /
|
[#18](https://git.produktor.io/eSlider/2dph/issues/18) `--with-facts` /
|
||||||
`--with-chats` on rebuild (WhatsApp out of v1).
|
`--with-chats` on rebuild (WhatsApp out of v1).
|
||||||
|
[#19](https://git.produktor.io/eSlider/2dph/issues/19) CI recall SoT =
|
||||||
|
`bin/brain/eval.go` via Zig.
|
||||||
|
|
||||||
## Blockers
|
## Blockers
|
||||||
|
|
||||||
|
None for epic #16. v2: [#6](https://git.produktor.io/eSlider/2dph/issues/6) OCR,
|
||||||
|
OQ1 contradiction resolution, OQ3 duckdb-md, OQ4 YAML-first leafs.
|
||||||
|
|
||||||
```
|
```
|
||||||
question
|
question
|
||||||
│
|
│
|
||||||
@@ -42,9 +49,6 @@ question
|
|||||||
└─ facts+chats corpus ← in
|
└─ facts+chats corpus ← in
|
||||||
```
|
```
|
||||||
|
|
||||||
1. **[#19](https://git.produktor.io/eSlider/2dph/issues/19) CI eval** —
|
|
||||||
recall SoT should be `bin/brain/eval.go` via Zig, not Python `bin/kb/eval`.
|
|
||||||
|
|
||||||
## Not v1
|
## Not v1
|
||||||
|
|
||||||
[#6](https://git.produktor.io/eSlider/2dph/issues/6) OCR (OQ2), OQ1
|
[#6](https://git.produktor.io/eSlider/2dph/issues/6) OCR (OQ2), OQ1
|
||||||
@@ -52,5 +56,4 @@ contradiction resolution, OQ3 duckdb-md export, OQ4 YAML-first leafs.
|
|||||||
|
|
||||||
## Close epic #16 when
|
## Close epic #16 when
|
||||||
|
|
||||||
- MCP tool order is documented and still gated by tests
|
Children #14, #15, #17, #18, #19 are closed. MCP tool order stays gated by tests.
|
||||||
- CI recall SoT is `bin/brain/eval.go` via Zig
|
|
||||||
|
|||||||
Reference in New Issue
Block a user