feat: markdown import splits H2 leafs in Go

Drop the Python ExecFile wrapper. Conversion still does not write Ladybug; add a dry-run fixture check for the index adapter split.
2026-08-13 23:24:47 +01:00
5 changed files with 31 additions and 158 deletions
@@ -7,16 +7,14 @@
 [![Latest Release](https://img.shields.io/github/v/tag/eSlider/2dph?sort=semver&label=release)](https://github.com/eSlider/2dph/releases)
 [![GitHub Stars](https://img.shields.io/github/stars/eSlider/2dph?style=social)](https://github.com/eSlider/2dph/stargazers)
-An evidence-first brain. **Facts need two independent sources, or they are
+An evidence-first brain over the operational eSlider stack. **Facts need two
-`(not confirmed)`.** Cursor is not the runtime.
+independent sources, or they are `(not confirmed)`.**
-`2dph` is a single embedded knowledge graph (LadybugDB) with native **HNSW
+`2dph` is a single embedded knowledge graph (LadybugDB = Kuzu successor) with
-vector** + **BM25 full-text** indexes. Search is *deduction*: confirmed facts
+native **HNSW vector** + **BM25 full-text** indexes, built from markdown,
-first, supporting info second, `web-search` as the independent second source
+compose files, ssh config, docker state, and git history. Search is
-when the local graph cannot confirm.
+*deduction*: confirmed facts first, supporting info second, `web-search` as
-
+the independent second source when the local graph cannot confirm.
 Run it: [docs/runbook.md](docs/runbook.md). Design: [docs/design.md](docs/design.md).
 Docs index: [docs/README.md](docs/README.md).
 ## Architecture
@@ -76,7 +74,7 @@ graph TB
 ## The method
 Every assertion is `Who / What / How / Where / When + evidence + confidence`,
-mirroring the detective method: **≥2 independent sources confirm a
+mirroring the detective detective skill: **≥2 independent sources confirm a
 fact; conflicting sources or a single source → `hypothesis` → `(not confirmed)`.**
 | root | meaning | used for answers |
@@ -126,15 +124,17 @@ bin/brain/search.go "invoice from last week"                            # same s
 ## Storage
- **LadybugDB** — single `var/kb.lbug`, Cypher + HNSW + BM25, embedded.
+- **LadybugDB** — single `var/kb.lbug`, Cypher property graph, HNSW + BM25
-  Read tools (`get` / `stats` / `eval`) are Go + Zig CGO (`bin/cgo/zcc`).
+  in one engine, embedded (no server), ACID, read-only-safe for concurrent
-  Python fallbacks stay for CI until the runner fetches Zig. Write is
+  readers. Read tools (`get` / `stats` / `eval`) are Go + Zig CGO (`bin/cgo/zcc`); Python
-  Compose profile `index` (`bin/brain/index.go`).
+  `bin/kb/{get,stats,eval}` is the CI fallback. **Never `DROP INDEX` FTS/VECTOR** on Ladybug 0.19: DROP leaves
- **model2vec** — `potion-multilingual-128M` (256-dim), CPU, no Ollama
+  ghost catalog tables (`_0_Leaf_vec_UPPER`) so recreate fails while
-  runtime dependency.
+  `SHOW_INDEXES` omits HNSW. Fresh indexes = delete `var/kb.lbug` +
- facts and info split by `root` but written in the same transaction.
+  `bin/brain/index.go --rebuild`. Use `ensure_indexes()` after upserts.
-
+- **model2vec** — `potion-multilingual-128M` static embeddings (256-dim),
-Ladybug 0.19 DROP INDEX warning: [docs/runbook.md](docs/runbook.md).
+  CPU-fast, deterministic, no Ollama runtime dependency.
 - facts and info split semantically by `root` column but written inside the
  same transaction.
 ## Tooling conventions
@@ -145,13 +145,11 @@ machines. Tests gate every commit. HTTP: `bin/brain/serve.go` calls
 ## Development
 See the portable runbook: [docs/runbook.md](docs/runbook.md).
 ```bash
-uv venv .venv
+uv venv .venv                                  # Python 3.12, uv-managed
-uv pip install -r requirements.lock.txt
+uv pip install -r requirements.lock.txt        # pinned toolchain
-bin/facts/audit.go self
+bin/facts/audit.go self                        # lexicon consistency gate
-go test ./... && uv run python -m unittest discover -s bin/tools -t .
+go test ./... && python -m unittest discover -s bin/tools -t .
 ```
 Docker (optional, cached model + var volumes):
@@ -145,8 +145,6 @@ class PublishedDocsTest(unittest.TestCase):
            ROOT / "docs" / "design.md",
            ROOT / "skills" / "brain" / "SKILL.md",
            ROOT / "skills" / "diataxis-docs" / "SKILL.md",
            ROOT / "docs" / "runbook.md",
            ROOT / "docs" / "README.md",
        ]
        # Command-style `--hop 1` / `--hop N` plus follow/walk = the old lie.
        # Honest "not implemented" notes must not match.
@@ -157,19 +155,3 @@ class PublishedDocsTest(unittest.TestCase):
                lie.search(text),
                f"{path.relative_to(ROOT)} still claims --hop walks the graph",
            )
    def test_docs_are_portable_diataxis(self) -> None:
        index = (ROOT / "docs" / "README.md").read_text()
        self.assertIn("type: reference", index)
        for d in ("D3", "D6", "D14", "D15", "D17", "D18"):
            self.assertIn(d, index)
        runbook = (ROOT / "docs" / "runbook.md").read_text()
        self.assertIn("type: howto", runbook)
        self.assertIn("bin/brain/search.go", runbook)
        self.assertIn("bin/brain/index.go", runbook)
        self.assertNotIn("search.ops.io", runbook)
        self.assertNotIn("/mnt/", runbook)
        self.assertNotIn("/home/", runbook)
        readme = (ROOT / "README.md").read_text()
        self.assertIn("docs/runbook.md", readme)
        self.assertNotIn("search.ops.io", readme)
@@ -1,34 +1,16 @@
---
+# 2dph (deductionphile)
 type: reference
 status: current
 related:
  - docs/runbook.md
  - docs/design.md
  - PLAN.md
 ---
-# 2dph docs (Diataxis)
+Evidence-first knowledge graph + hybrid RAG over the operational
-
+Brain/ops/eSlider stack. Facts need proof or they are
 Evidence-first knowledge graph. Facts need proof or they are
 `(not confirmed)`.
-| Type | Doc |
+- [PLAN.md](../PLAN.md) — decisions, execution order, open questions (v2)
-|------|-----|
+- [design](design.md) — schema, deduction model, sources
-| tutorial / howto | [runbook](runbook.md) — run anywhere (uv, Go, Docker) |
+- [reasoner](reasoner.md) — D18 CPU bake-off (Qwen3.5-9B vs Bonsai / Qwen3.6-27B)
-| explanation | [design](design.md) — two roots, deduction, D17/D20/D18 |
+- [Gitea issues](https://git.produktor.io/eSlider/2dph/issues) — work board (origin)
 | howto | [picoclaw](picoclaw.md) — MCP agent profile |
 | howto | [reasoner](reasoner.md) — CPU bake-off (D18) |
 | reference | [PLAN.md](../PLAN.md) — decisions D1–D21 |
 Decisions the public face must name: **D3** SearXNG compose, **D6** Go service /
 Python write sidecar, **D14** `bin/{subject}/{method}.go`, **D15** Gitea origin,
 **D17** assertion gate (facts → info → web), **D18** pluggable reasoner.
 Search: `bin/brain/search.go "query"` (HTTP: `bin/brain/serve.go` —
 `/health` `/search` `/get` `/stats` `/audit` `/ingest`). `--hop` is
 not a walk; the flag errors until File/FROM_FILE edges exist.
-Work board: [Gitea issues](https://git.produktor.io/eSlider/2dph/issues).
+Published docs live here and mirror the project state.
 PRs and CI: GitHub [`eSlider/2dph`](https://github.com/eSlider/2dph).
 Published docs live here and match live commands.
@@ -1,11 +1,3 @@
 ---
 type: explanation
 status: current
 related:
  - docs/README.md
  - docs/runbook.md
 ---
 # Design — facts, info, deduction
 ## Two roots, one transaction
@@ -1,81 +0,0 @@
 ---
 type: howto
 status: current
 related:
  - docs/README.md
  - PLAN.md
 ---
 # Run 2dph (portable)
 No laptop-absolute paths. Config lives in env files under `$HOME/.config/brain/`
 (mode 0600), not in git.
 ## Toolchain
 - Go (see `go.mod`)
 - Python 3.12 + [uv](https://docs.astral.sh/uv)
 - Optional: Docker, Zig CGO via `bin/cgo/zig` (not gcc)
 ```bash
 uv venv .venv
 uv pip install -r requirements.lock.txt
 eval "$(bin/cgo/zig env)"   # when compiling Ladybug read tools
 go test ./...
 uv run python -m unittest discover -s bin/tools -t .
 ```
 ## Config
 | File / env | Purpose |
 |------------|---------|
 | `$BRAIN_SEARCH_ENV` (default `$HOME/.config/brain/search.env`) | `BRAIN_SEARCH_URL` (SearXNG). Optional Basic Auth. |
 | `$HOME/.config/brain/db-profiles.yml` | read-only Postgres profiles (OnlyOffice via tunnel) |
 If the host already runs SearXNG, point `BRAIN_SEARCH_URL` at it. Do not start
 a second copy (D3). Optional Compose instance:
 ```bash
 SEARXNG_SECRET=$(openssl rand -hex 32) docker compose --profile searxng up -d
 ```
 That binds `127.0.0.1:8888`. JSON format must stay enabled.
 ## Index then search
 Write path is Compose profile `index` (Python Ladybug rebuild) until
 `brain/add` is v2. The operator command is `bin/brain/index.go`.
 ```bash
 bin/brain/index.go --rebuild
 bin/brain/search.go "LadybugDB vector index"     # facts → info → web (D17)
 bin/brain/search.go "upstream flag" --no-web
 bin/brain/get.go <id> --body
 bin/brain/stats.go
 ```
 `--hop` is not implemented. Empty web results are `throttled`, not absence.
 Ladybug 0.19: never `DROP INDEX` FTS/VECTOR (ghost catalog). Fresh indexes =
 delete `var/kb.lbug` then `--rebuild`.
 ## HTTP / MCP
 ```bash
 docker compose up -d brain                       # :8630 Zig CGO serve
 docker compose --profile index run --rm index    # rebuild
 docker compose --profile picoclaw up brain-mcp   # MCP 127.0.0.1:8630
 ```
 `GET /openapi.json`, `POST /mcp`. Agent tool order: `search` → `get` → `audit`.
 ## Reasoner (optional, D18)
 CPU sidecar on `127.0.0.1:11435`. Weights are not in the 2dph image.
 ```bash
 docker compose --profile reasoner up -d reasoner
 REASONER_BASE_URL=http://127.0.0.1:11435/v1 ./bin/reasoner/bakeoff.go --json
 ```
 See [reasoner.md](reasoner.md).