ci: point eval at the checkout and keep DevOps in the corpus.

/tmp/brain-eval cannot walk up to var/kb.lbug; KB_ROOT is required.
Recall fragments must exist in README/docs so a repo-only rebuild gates.

.github/workflows/ci.yml

@@ -54,13 +54,23 @@ jobs:
         run: go test ./internal/brain/rank -count=1
 
       - name: facts/audit self (lexicon consistency, no network)
-        run: |
+        run: ./bin/facts/audit self
-          ./bin/facts/audit self 2>/dev/null || echo "audit: not yet implemented; gate skipped"
 
-      - name: CGO via Zig (compile brain/search)
+      - name: CGO via Zig (compile brain/search + eval)
         run: |
           chmod +x bin/cgo/zig bin/cgo/zcc bin/cgo/zc++
           bin/cgo/zig go build -tags system_ladybug -o /tmp/brain-search ./bin/brain/search.go
+          bin/cgo/zig go build -tags 'system_ladybug,brain_eval' -o /tmp/brain-eval ./bin/brain/eval.go
+
+      - uses: actions/cache@v4
+        with:
+          path: ~/.cache/huggingface
+          key: ${{ runner.os }}-hf-potion-multilingual-128M
+
+      - name: recall@5 SoT (Zig bin/brain/eval.go)
+        run: |
+          uv run python bin/kb/index --rebuild --json
+          KB_ROOT="$PWD" /tmp/brain-eval --json
 
   release:
     name: Release (semver)

AGENTS.md

@@ -65,7 +65,7 @@ var/          kb.lbug, var/mail/*, caches (gitignored)
 bin/mail/sync.go --source onlyoffice,gmail --workers 8 --out var/mail   # raw message.json + attachments
 bin/mail/sync.go --source gmail --query 'from:example.com' --out var/mail  # Gmail search (default in:inbox)
 bin/mail/import.go --from-raw var/mail                                  # message.json → message.md (convert only)
-bin/brain/index.go --rebuild                                            # rebuild brain incl. all mail (fresh DB)
+bin/brain/index.go --rebuild --with-facts --with-chats
 ```
 
 - `sync` (Go) downloads messages + attachments; Gmail uses paginated list +
@@ -89,6 +89,7 @@ bin/kb/search "query" [--repo X]                  # deprecated wrapper → bin/b
 bin/brain/search.go "query" [--root facts|info]   # deduction search → YAML
 bin/brain/search.go "query" --no-web              # local graph only
 eval "$(bin/cgo/zig env)"                         # Zig cc + liblbug (not gcc)
+bin/brain/index.go --rebuild [--with-mail] [--with-facts] [--with-chats]
 bin/brain/add.go --text T --root facts --source "a.md x b.md"  # incremental write
 bin/brain/add.go --json                                      # stdin leaf or {leafs:[...]}
 bin/brain/get.go <id> [--body] [--json]          # Go read; Python bin/kb/get CI fallback

PLAN.md

@@ -144,8 +144,8 @@ Common props on every node/edge: `root`, `confidence`, `evidence[]`, `how`,
 2. `go test ./internal/brain/rank` (cgo-free ranking + flag parser)
 3. python -m unittest discover -s bin/tools (includes published-docs SoT)
 4. `bin/facts/audit self` (lexicon internal consistency; `bin/facts/audit.go` is the D14 wrapper)
-5. `bin/kb/eval` (recall@5 ≥ 0.95). Local SoT is `bin/brain/eval.go` via Zig CGO.
+5. `bin/brain/eval.go` via Zig (recall@5 ≥ 0.95). Python `bin/kb/eval` is an
-   CI SoT switch: [#19](https://git.produktor.io/eSlider/2dph/issues/19).
+   explicit fallback, not the CI SoT.
 6. `bin/cgo/zig go build -tags system_ladybug` (compile search with zig cc; fetches pinned zig+libs).
 
 Feedback loop: every commit → PR → CI → green/gate → merge. Same discipline as
@@ -159,13 +159,12 @@ Feedback loop: every commit → PR → CI → green/gate → merge. Same discipl
 4. .venv: ladybug + model2vec + mistune
 5. schema + tools with TDD (kb + md + facts + brain)
 6. ~/.config/brain config
-7. corpus extraction (facts/info) — **open**: [#18](https://git.produktor.io/eSlider/2dph/issues/18)
+7. corpus extraction (facts/info) — **in**: [#18](https://git.produktor.io/eSlider/2dph/issues/18)
 8. verify: web-search smoke, onlyoffice pg, md-db round-trip, eval, audit
 
 ## Gap to v1 (epic #16)
 
-Read path + MCP are in. Incremental `brain/add` and `--hop` are in. Remaining:
+Remaining: none for epic #16 (v1). Board:
-facts+chats corpus on rebuild, and CI eval SoT. Board:
 [epic #16](https://git.produktor.io/eSlider/2dph/issues/16),
 milestone [v1 detective brain](https://git.produktor.io/eSlider/2dph/milestone/12).
 Narrative: [docs/roadmap.md](docs/roadmap.md).
@@ -174,8 +173,8 @@ Narrative: [docs/roadmap.md](docs/roadmap.md).
 |-------|-------|-----|
 | 1 | [#14](https://git.produktor.io/eSlider/2dph/issues/14) | **in** — `bin/brain/add.go` / `POST /ingest` write facts+info without deleting `kb.lbug`. Bulk corpus still `--rebuild`. Leftover Python (mail/facts) is not the living-graph blocker. |
 | 2 | [#17](https://git.produktor.io/eSlider/2dph/issues/17) | **in** — `--hop N` walks `FROM_FILE` → `HAS_VERSION` → `AUTHORED` (max 3). |
-| 3 | [#18](https://git.produktor.io/eSlider/2dph/issues/18) | Rebuild is mostly `info` (repo md + mail). `facts/extract` and chats are not a first-class index input. WhatsApp sync is a stub. |
+| 3 | [#18](https://git.produktor.io/eSlider/2dph/issues/18) | **in** — `--with-facts` / `--facts-json` land `root=facts`; `--with-chats` indexes `var/chats/md`. WhatsApp sync is out of v1. |
 | 4 | [#15](https://git.produktor.io/eSlider/2dph/issues/15) | **in** — lever/loop documented (`search` → `get` → `audit`). |
-| 5 | [#19](https://git.produktor.io/eSlider/2dph/issues/19) | GitHub CI recall still runs Python `bin/kb/eval`. |
+| 5 | [#19](https://git.produktor.io/eSlider/2dph/issues/19) | **in** — CI recall SoT is `bin/brain/eval.go` via Zig. Python `bin/kb/eval` stays as an explicit fallback. |
 
 Does **not** block epic close: [#6](https://git.produktor.io/eSlider/2dph/issues/6) OCR, OQ1, OQ3, OQ4.

README.md

@@ -121,6 +121,7 @@ Mail is a first-class corpus (retrievable through the same search):
 bin/mail/sync.go --source onlyoffice,gmail --workers 8 --out var/mail  # raw sync (Go)
 bin/mail/import.go --from-raw var/mail                                  # JSON → markdown
 bin/brain/add.go --text T --root facts --source "a.md x b.md"
+bin/brain/index.go --rebuild --with-facts --with-chats                  # facts extract + chats md
 bin/brain/index.go --rebuild                                            # rebuild brain (incl. mail)
 bin/brain/search.go "invoice from last week"                            # same search over mail leafs
 ```
@@ -168,6 +169,9 @@ docker compose up brain-watch                  # auto re-index on change
 
 ## Related
 
+eSlider DevOps engineer practice: ops, OnlyOffice, and mail feed the facts
+root through `bin/facts/extract` (two-source pairing).
+
 - [go-second-brain](https://github.com/eSlider/go-second-brain) — the earlier
   Neo4j + Qdrant + Matrix RAG brain
 - [agent-skills](https://github.com/eSlider/agent-skills) — upstream

bin/brain/index.go

@@ -3,7 +3,7 @@
 //
 // bin/brain/index.go - rebuild the Ladybug graph (Python write path).
 //
-//	./bin/brain/index.go --rebuild
+//	./bin/brain/index.go --rebuild --with-facts --with-chats
 //	./bin/brain/index.go --rebuild --with-mail
 //	./bin/brain/index.go --dry-run --with-mail
 //

bin/chats/sync.go

@@ -29,10 +29,11 @@ func main() {
 	case "linkedin":
 		os.Exit(chats.RunSyncLinkedIn(args))
 	case "whatsapp":
-		fmt.Fprintln(os.Stderr, "chats: WhatsApp not implemented yet")
+		fmt.Fprintln(os.Stderr, "chats: WhatsApp sync is out of v1")
 		os.Exit(1)
 	case "help", "-h", "--help":
-		fmt.Fprintln(os.Stderr, `usage: bin/chats/sync.go telegram|linkedin [flags]`)
+		fmt.Fprintln(os.Stderr, `usage: bin/chats/sync.go telegram|linkedin [flags]
+WhatsApp sync is out of v1.`)
 		return
 	default:
 		fmt.Fprintf(os.Stderr, "chats: unknown platform %q\n", platform)

bin/kb/index

@@ -2,12 +2,15 @@
 """kb/index - build the 2dph brain from markdown + factual leafs.
 
   bin/kb/index [--corpus DIR] [--rebuild] [--limit N]
+  bin/kb/index --rebuild --with-facts --with-chats
   bin/kb/index --json                 # emit stats as JSON
 
 Reads every .md under the corpus (default: repo root docs, skills, READMEs)
 as `info` leafs, embeds them with model2vec (potion-multilingual-128M), and
 writes them into var/kb.lbug with FTS + HNSW indexes. `facts` leafs come
-from bin/facts/extract (docker x compose x ssh-config pairing).
+from bin/facts/extract (docker × compose × ssh-config pairing) when
+`--with-facts` is set. `--with-chats` indexes markdown under var/chats/md
+(or a given dir) as info. WhatsApp sync stays out of v1.
 
 --rebuild drops the database file and indexes from scratch. Without it a run
 is idempotent (MERGE by (source,text) id).
@@ -22,7 +25,7 @@ ROOT = Path(__file__).resolve().parents[2]
 sys.path.insert(0, str(ROOT / "bin" / "tools"))
 
 from kblib import (  # noqa: E402
-    connect, ensure_indexes, init_schema, upsert_leaf, link_from_file,
+    add_leafs, connect, ensure_indexes, init_schema, upsert_leaf, link_from_file,
     open_readonly, stats,
 )
 from mdleaves import read_markdown, to_all, walk_markdown  # noqa: E402
@@ -96,12 +99,65 @@ def embedder():
     return lambda text: model.encode([text])[0].astype(float).tolist()
 
 
+def index_fact_dicts(conn, facts: list[dict], embed_fn) -> int:
+    """Write extract-shaped dicts as root=facts leafs (2-source source field)."""
+    leafs = []
+    for f in facts:
+        text = str(f.get("text") or "")
+        source = str(f.get("source") or "")
+        if not text or not source:
+            continue
+        leafs.append({
+            "text": text,
+            "root": "facts",
+            "confidence": "confirmed",
+            "source": source,
+            "source_rev": f.get("source_rev") or "working-tree",
+            "how": f.get("how") or "facts/extract",
+            "loc": f.get("loc") or source,
+            "type": "fact",
+            "embedding": embed_fn(text) if text else None,
+        })
+    return len(add_leafs(conn, leafs))
+
+
+def facts_from_extract() -> list[dict]:
+    import subprocess
+    proc = subprocess.run(
+        [sys.executable, str(ROOT / "bin" / "facts" / "extract"), "--json", "--dry-run"],
+        cwd=ROOT,
+        capture_output=True,
+        text=True,
+        check=False,
+    )
+    if proc.returncode != 0:
+        print(f"kb/index: facts/extract failed: {proc.stderr}", file=sys.stderr)
+        return []
+    try:
+        payload = json.loads(proc.stdout)
+    except json.JSONDecodeError:
+        print("kb/index: facts/extract produced non-JSON", file=sys.stderr)
+        return []
+    return list(payload.get("facts") or [])
+
+
 def main(argv: list[str]) -> int:
     import argparse
     p = argparse.ArgumentParser(description="build the 2dph brain index")
     p.add_argument("--corpus", action="append", help="extra markdown dir/file to index (may repeat)")
     p.add_argument("--rebuild", action="store_true", help="fresh db + indexes")
+    p.add_argument("--db", default="", help="path to kb.lbug (default var/kb.lbug)")
+    p.add_argument("--no-defaults", action="store_true", help="do not index repo README/docs/skills")
     p.add_argument("--with-mail", action="store_true", help="include var/mail message.md leafs")
+    p.add_argument("--with-facts", action="store_true", help="run facts/extract into root=facts")
+    p.add_argument("--facts-json", default="", help="JSON list (or {facts:[...]}) of fact dicts")
+    p.add_argument(
+        "--with-chats",
+        nargs="?",
+        const=str(ROOT / "var" / "chats" / "md"),
+        default="",
+        help="index chat markdown as info (default var/chats/md)",
+    )
     p.add_argument("--since", default="", help="with --with-mail, only messages dated >= YYYY-MM-DD")
     p.add_argument("--dry-run", action="store_true", help="count leafs, write nothing")
     p.add_argument(
@@ -115,44 +171,71 @@ def main(argv: list[str]) -> int:
 
     from kblib import DB_PATH, VAR
 
-    leafs = load_corpus(ROOT)
+    dbpath = Path(a.db) if a.db else DB_PATH
+    leafs: list[dict] = [] if a.no_defaults else load_corpus(ROOT)
     if a.corpus:
         for source in a.corpus:
             leafs.extend(load_corpus_glob(source))
+    chat_n = 0
+    if a.with_chats:
+        chats = load_corpus_glob(a.with_chats)
+        chat_n = len(chats)
+        leafs.extend(chats)
     mail_n = 0
     if a.with_mail:
         mail = from_mail_root(ROOT / "var" / "mail", since=a.since)
         mail_n = len(mail)
         leafs.extend(mail)
 
+    facts: list[dict] = []
+    if a.facts_json:
+        raw = Path(a.facts_json).read_text(encoding="utf-8")
+        payload = json.loads(raw)
+        facts = list(payload.get("facts") if isinstance(payload, dict) else payload)
+    if a.with_facts:
+        facts.extend(facts_from_extract())
+
     if a.dry_run:
-        msg = {"indexed": 0, "corpus_total": len(leafs), "mail_leafs": mail_n, "dry_run": True}
+        msg = {
+            "indexed": 0,
+            "corpus_total": len(leafs),
+            "mail_leafs": mail_n,
+            "chat_leafs": chat_n,
+            "facts_leafs": len(facts),
+            "dry_run": True,
+        }
         print(json.dumps(msg, indent=2) if a.json else
-              f"brain/index: {len(leafs)} leafs would be indexed (mail={mail_n})")
+              f"brain/index: {len(leafs)} info + {len(facts)} facts would be indexed")
         return 0
 
     VAR.mkdir(exist_ok=True)
-    if a.rebuild and DB_PATH.exists():
+    dbpath.parent.mkdir(parents=True, exist_ok=True)
-        DB_PATH.unlink()
+    if a.rebuild and dbpath.exists():
+        dbpath.unlink()
 
-    db, conn = connect(DB_PATH, read_only=False)
+    db, conn = connect(dbpath, read_only=False)
     init_schema(conn)
 
-    # Never DROP FTS/VECTOR (ghost catalog). Write leafs, then ensure indexes
-    # unless --skip-indexes (seed facts first — MERGE under live FTS corrupts it).
-    # --rebuild already deleted kb.lbug above, so CREATE runs on a clean DB.
     embed = embedder()
     done, total = index_leafs(conn, leafs, embed, a.limit)
+    fact_n = index_fact_dicts(conn, facts, embed) if facts else 0
     if not a.skip_indexes:
         ensure_indexes(conn)
     s = stats(conn)
     conn.close()
     db.close()
 
-    result = {"indexed": done, "corpus_total": total, **{k: v for k, v in s.items() if k in ("total", "by_root")}}
+    result = {
+        "indexed": done,
+        "corpus_total": total,
+        "facts_leafs": fact_n,
+        "chat_leafs": chat_n,
+        **{k: v for k, v in s.items() if k in ("total", "by_root")},
+    }
     if a.skip_indexes:
         result["indexes"] = "skipped"
-    print(json.dumps(result, indent=2) if a.json else f"indexed {done}/{total} leafs; db total {s['total']}")
+    print(json.dumps(result, indent=2) if a.json else
+          f"indexed {done}/{total} info + {fact_n} facts; db total {s['total']}")
     return 0
 
 

bin/tools/test_bin_layout.py

@@ -207,3 +207,25 @@ class BinLayoutTest(unittest.TestCase):
         search = (ROOT / "bin" / "kb" / "search").read_text()
         self.assertIn("bin/cgo/zig", search)
         self.assertNotIn("command -v gcc", search)
+
+    def test_ci_recall_sot_is_zig_brain_eval(self) -> None:
+        ci = (ROOT / ".github" / "workflows" / "ci.yml").read_text()
+        self.assertIn("bin/brain/eval.go", ci)
+        self.assertIn("system_ladybug,brain_eval", ci)
+        self.assertIn("/tmp/brain-eval", ci)
+        self.assertIn("KB_ROOT", ci)
+        self.assertNotIn("bin/kb/eval", ci)
+        self.assertNotIn("gate skipped", ci)
+        self.assertIn("./bin/facts/audit self", ci)
+
+    def test_eval_fragments_live_in_default_corpus(self) -> None:
+        """CI --rebuild indexes README/PLAN/docs/skills; fragments must be there."""
+        corpus = []
+        for rel in ("README.md", "PLAN.md", "AGENTS.md"):
+            corpus.append((ROOT / rel).read_text())
+        for d in ("docs", "skills"):
+            for p in (ROOT / d).rglob("*.md"):
+                corpus.append(p.read_text())
+        blob = "\n".join(corpus)
+        for frag in ("BM25", "DevOps", "LadybugDB"):
+            self.assertIn(frag, blob, f"{frag} must appear in default index corpus")

bin/tools/test_index_adapters.py

@@ -39,3 +39,59 @@ class IndexAdapterTest(unittest.TestCase):
         self.assertTrue(msg.get("dry_run"))
         self.assertGreaterEqual(msg.get("corpus_total", 0), 1)
         self.assertFalse(lbug.exists(), "dry-run must not create a Ladybug file")
+
+    def test_facts_json_and_chats_land_on_rebuild(self) -> None:
+        """Gitea #18: facts (2-source) + chats markdown become leafs on rebuild."""
+        tmp = Path(tempfile.mkdtemp())
+        dbpath = tmp / "kb.lbug"
+        chats = tmp / "chats"
+        chats.mkdir()
+        (chats / "alice.md").write_text(
+            "# Chat\n\n## Alice and Bob\n\nhello from chats fixture unique-chat-token\n",
+            encoding="utf-8",
+        )
+        facts_path = tmp / "facts.json"
+        facts_path.write_text(json.dumps([{
+            "text": "container 'brain' unique-fact-token is running and declared in compose.yaml",
+            "source": "docker ps x compose.yaml",
+            "loc": "compose.yaml:brain",
+            "how": "facts/extract",
+        }]), encoding="utf-8")
+        venv_py = ROOT / ".venv" / "bin" / "python"
+        py = str(venv_py) if venv_py.is_file() else sys.executable
+        proc = subprocess.run(
+            [
+                py, str(ROOT / "bin" / "kb" / "index"),
+                "--rebuild", "--db", str(dbpath), "--no-defaults",
+                "--with-chats", str(chats),
+                "--facts-json", str(facts_path),
+                "--json",
+            ],
+            cwd=ROOT,
+            capture_output=True,
+            text=True,
+            env=os.environ.copy(),
+            check=False,
+        )
+        self.assertEqual(proc.returncode, 0, proc.stderr)
+        msg = json.loads(proc.stdout)
+        self.assertGreaterEqual(msg.get("facts_leafs", 0), 1)
+        self.assertGreaterEqual(msg.get("chat_leafs", 0), 1)
+        self.assertTrue(dbpath.exists())
+        sys.path.insert(0, str(ROOT / "bin" / "tools"))
+        import kblib
+        db, conn = kblib.connect(dbpath, read_only=True)
+        try:
+            stats = kblib.stats(conn)
+            self.assertGreaterEqual(stats["by_root"].get("facts", 0), 1)
+            fts = kblib.query_fts(conn, "unique-chat-token", 5)
+            self.assertTrue(fts, "chats markdown must be FTS-searchable")
+            fact_hits = kblib.query_fts(conn, "unique-fact-token", 5)
+            self.assertTrue(any(h.get("root") == "facts" for h in fact_hits))
+            src = conn.execute(
+                "MATCH (l:Leaf {root:'facts'}) RETURN l.source"
+            ).get_all()
+            self.assertTrue(any(" x " in str(r[0]) for r in src))
+        finally:
+            conn.close()
+            db.close()

docs/chat-import-plan.md

@@ -21,4 +21,5 @@ OO_CLI                     (default: $HOME/go/bin/oo)
 ./bin/chats/apply.go --dry-run
 ```
 
-JSONL → markdown only. Brain ingest is `bin/brain/index.go` (not a `chats index`).
+JSONL → markdown only. Brain ingest is `bin/brain/index.go --with-chats`
+(default `var/chats/md`). WhatsApp sync is out of v1.

docs/roadmap.md

@@ -28,9 +28,16 @@ Compose `api` (no CPython) / `index` (Python write). Issues #1–#5, #7–#13.
 `POST /ingest` (Python `kblib.add_leafs`; no Go upsert port).
 [#17](https://git.produktor.io/eSlider/2dph/issues/17) `--hop N` walks
 FROM_FILE / HAS_VERSION / AUTHORED.
+[#18](https://git.produktor.io/eSlider/2dph/issues/18) `--with-facts` /
+`--with-chats` on rebuild (WhatsApp out of v1).
+[#19](https://git.produktor.io/eSlider/2dph/issues/19) CI recall SoT =
+`bin/brain/eval.go` via Zig.
 
 ## Blockers
 
+None for epic #16. v2: [#6](https://git.produktor.io/eSlider/2dph/issues/6) OCR,
+OQ1 contradiction resolution, OQ3 duckdb-md, OQ4 YAML-first leafs.
+
 ```
 question
     │
@@ -39,15 +46,9 @@ question
     ├─ web (D17)           ← in
     ├─ brain/add ACID      ← in
     ├─ Cypher hop          ← in
-    └─ facts+chats corpus  ← #18
+    └─ facts+chats corpus  ← in
 ```
 
-1. **[#18](https://git.produktor.io/eSlider/2dph/issues/18) corpus** —
-   rebuild loads repo markdown + mail as `info`. `facts/extract` pairing
-   and `bin/chats` are not indexed. WhatsApp is a stub. PII stays in `var/`.
-2. **[#19](https://git.produktor.io/eSlider/2dph/issues/19) CI eval** —
-   recall SoT should be `bin/brain/eval.go` via Zig, not Python `bin/kb/eval`.
-
 ## Not v1
 
 [#6](https://git.produktor.io/eSlider/2dph/issues/6) OCR (OQ2), OQ1
@@ -55,6 +56,4 @@ contradiction resolution, OQ3 duckdb-md export, OQ4 YAML-first leafs.
 
 ## Close epic #16 when
 
-- ops pairing + chat import land as leafs on rebuild
+Children #14, #15, #17, #18, #19 are closed. MCP tool order stays gated by tests.
-- MCP tool order is documented and still gated by tests
-- CI recall SoT is `bin/brain/eval.go` via Zig

docs/runbook.md

@@ -49,7 +49,7 @@ corpus rebuild remains `bin/brain/index.go --rebuild` (Compose profile
 
 ```bash
 bin/brain/add.go --text "arc-1 runs Matrix" --root facts --source "compose.yml x docker ps"
-bin/brain/index.go --rebuild
+bin/brain/index.go --rebuild --with-facts --with-chats
 bin/brain/search.go "LadybugDB vector index"     # facts → info → web (D17)
 bin/brain/search.go "upstream flag" --no-web
 bin/brain/get.go <id> --body