Compare commits

...
Author SHA1 Message Date
eSlider 7db9e6b836 docs: record v1 detective epic and remaining gaps.
Tests / Test (push) Skipped
Tests / Release (semver) (push) Skipped
Tests / Test (pull_request) Failing after 6s
Tests / Release (semver) (pull_request) Skipped
Read path and MCP are in; write, hops, corpus, and CI eval are still open. Board is Gitea epic #16.
2026-08-14 10:22:36 +01:00
9 changed files with 138 additions and 18 deletions
+2 -1
View File
@@ -3,7 +3,8 @@
Evidence-first brain over the ops/eSlider stack. Facts need proof or they are Evidence-first brain over the ops/eSlider stack. Facts need proof or they are
`(not confirmed)`. `(not confirmed)`.
Read first: [PLAN](PLAN.md) → [docs](docs/). Read first: [PLAN](PLAN.md) → [docs](docs/) → [roadmap](docs/roadmap.md)
(epic [#16](https://git.produktor.io/eSlider/2dph/issues/16)).
## Method (detective, no fork) ## Method (detective, no fork)
+30 -8
View File
@@ -4,7 +4,9 @@ A brain that loves facts and deduction. Evidence-first knowledge graph + hybrid
RAG over the operational Brain/ops/eSlider stack. Built like Sherlock RAG over the operational Brain/ops/eSlider stack. Built like Sherlock
Holmes: nothing is asserted unless it has proof. Holmes: nothing is asserted unless it has proof.
Status: **in progress**this file is the plan and the record of decisions. Status: **in progress**read path + MCP work; v1 goal is [epic #16](https://git.produktor.io/eSlider/2dph/issues/16)
(milestone [v1 detective brain](https://git.produktor.io/eSlider/2dph/milestone/12)).
Gap: [docs/roadmap.md](docs/roadmap.md).
## What ## What
@@ -41,7 +43,7 @@ detective method: **a fact needs ≥2 independent sources or it is
| D15 | repo | Gitea [`eSlider/2dph`](https://git.produktor.io/eSlider/2dph) is origin + [issues](https://git.produktor.io/eSlider/2dph/issues). GitHub `eSlider/2dph` is the public clone (PRs + Actions CI). No direct `main` pushes. TDD → PR → CI green → merge. | | D15 | repo | Gitea [`eSlider/2dph`](https://git.produktor.io/eSlider/2dph) is origin + [issues](https://git.produktor.io/eSlider/2dph/issues). GitHub `eSlider/2dph` is the public clone (PRs + Actions CI). No direct `main` pushes. TDD → PR → CI green → merge. |
| D16 | contradictions | ≥2 yes vs ≥2 no → unrelated sources conflict → hypothesis → `(not confirmed)`. Resolution (authority, staleness adjudication) = **v2**, tracked as open question. | | D16 | contradictions | ≥2 yes vs ≥2 no → unrelated sources conflict → hypothesis → `(not confirmed)`. Resolution (authority, staleness adjudication) = **v2**, tracked as open question. |
| D17 | assertion gate | Fact-check every *claim* (facts → info → live → web), not every edit. `bin/brain/search.go` adds a `web` block when there is no facts hit (`throttled`/`skipped`/`refused` ≠ absence). `--root` and `--no-web` stay local. Missing graph ≠ “does not exist”. | | D17 | assertion gate | Fact-check every *claim* (facts → info → live → web), not every edit. `bin/brain/search.go` adds a `web` block when there is no facts hit (`throttled`/`skipped`/`refused` ≠ absence). `--root` and `--no-web` stay local. Missing graph ≠ “does not exist”. |
| D18 | reasoner | Pluggable OpenAI-compatible URL (`REASONER_BASE_URL`). RAM: `Qwen/Qwen3.5-9B`. Quality: `prism-ml/Bonsai-27B-gguf` or `Qwen/Qwen3.6-27B`. No official Qwen3.6-9B. CPU bake-off: `bin/reasoner/bakeoff.go` + compose profile `reasoner` (`OLLAMA_NUM_GPU=0`, `:11435`). PicoClaw is not shipped; tools are `search`/`get`/`audit`. Weights are not copied into the 2dph image. | | D18 | reasoner | Pluggable OpenAI-compatible URL (`REASONER_BASE_URL`). RAM: `Qwen/Qwen3.5-9B`. Quality: `prism-ml/Bonsai-27B-gguf` or `Qwen/Qwen3.6-27B`. No official Qwen3.6-9B. CPU bake-off: `bin/reasoner/bakeoff.go` + compose profile `reasoner` (`OLLAMA_NUM_GPU=0`, `:11435`). PicoClaw is compose profile `picoclaw`; tools are `search`/`get`/`audit`. Weights are not copied into the 2dph image. Agent lever/loop: [#15](https://git.produktor.io/eSlider/2dph/issues/15). |
| D19 | git history | [go-git](https://github.com/go-git/go-git) via `bin/git/import.go`. No subprocess of the git binary. Conversion prints commit leafs; brain write is `bin/brain/index.go`. | | D19 | git history | [go-git](https://github.com/go-git/go-git) via `bin/git/import.go`. No subprocess of the git binary. Conversion prints commit leafs; brain write is `bin/brain/index.go`. |
| D20 | agent API | OpenAPI + MCP are generated from the same `internal/httpapi.Ops` table as `bin/brain/serve.go` handlers. `GET /openapi.json`, `POST /mcp` (JSON-RPC tools/list + tools/call). Tool names match OpenAPI paths (`search`/`get`/`stats`/`audit`). | | D20 | agent API | OpenAPI + MCP are generated from the same `internal/httpapi.Ops` table as `bin/brain/serve.go` handlers. `GET /openapi.json`, `POST /mcp` (JSON-RPC tools/list + tools/call). Tool names match OpenAPI paths (`search`/`get`/`stats`/`audit`). |
| D21 | CGO | Ladybug/tokenizers CGO is compiled with **Zig** (`bin/cgo/zcc``zig cc -target …-linux-gnu`), not gcc. `bin/cgo/zig` pins Zig 0.14.1 + liblbug 0.19.1 + libtokenizers 1.27.0. Compose `target: api` has no CPython; write/rebuild is profile `index`. | | D21 | CGO | Ladybug/tokenizers CGO is compiled with **Zig** (`bin/cgo/zcc``zig cc -target …-linux-gnu`), not gcc. `bin/cgo/zig` pins Zig 0.14.1 + liblbug 0.19.1 + libtokenizers 1.27.0. Compose `target: api` has no CPython; write/rebuild is profile `index`. |
@@ -83,7 +85,8 @@ detective method: **a fact needs ≥2 independent sources or it is
Node tables: `Person, Service, Host, Container, Repo, File, Commit, Leaf`. Node tables: `Person, Service, Host, Container, Repo, File, Commit, Leaf`.
`Leaf(embedding FLOAT[N])` — FTS on `text`, HNSW vector index on `embedding`. `Leaf(embedding FLOAT[N])` — FTS on `text`, HNSW vector index on `embedding`.
Edges: `RUNS / USES / HAS_VERSION / AUTHORED / ABOUT / ASSOCIATED / SIMILAR_0.85`. Edges: `RUNS / USES / FROM_FILE / HAS_VERSION / AUTHORED / ABOUT / ASSOCIATED / SIMILAR_0.85`.
`FROM_FILE` / `HAS_VERSION` exist in schema; search `--hop` does not walk them yet ([#17](https://git.produktor.io/eSlider/2dph/issues/17)).
Common props on every node/edge: `root`, `confidence`, `evidence[]`, `how`, Common props on every node/edge: `root`, `confidence`, `evidence[]`, `how`,
`where`, `when`, `source_rev`. `where`, `when`, `source_rev`.
@@ -108,9 +111,9 @@ Common props on every node/edge: `root`, `confidence`, `evidence[]`, `how`,
## Open questions (v2) ## Open questions (v2)
- OQ1: mutually-contradicting evidence — how to resolve (authority weighting, - OQ1: mutually-contradicting evidence — how to resolve (authority weighting,
temporal freshness, audit adjudication). temporal freshness, audit adjudication). **v2**; does not block epic #16.
- OQ2: OCR pipeline for pdfs/images/docs — mostly solved: poppler pdftotext - OQ2: OCR — poppler `pdftotext` fast-path exists; scans still docling.
fast-path for born-digital PDFs, docling fallback for the ~5% textless ones. [#6](https://git.produktor.io/eSlider/2dph/issues/6) (v2, does not block #16).
- OQ3: optional duckdb-md layer for `SELECT … FORMAT MARKDOWN` export/write-back. - OQ3: optional duckdb-md layer for `SELECT … FORMAT MARKDOWN` export/write-back.
- OQ4: YAML-first storage for leafs — deferred: JSON is ~10x faster to - OQ4: YAML-first storage for leafs — deferred: JSON is ~10x faster to
serialize and unambiguous; YAML only where humans edit files. serialize and unambiguous; YAML only where humans edit files.
@@ -138,6 +141,7 @@ Common props on every node/edge: `root`, `confidence`, `evidence[]`, `how`,
3. python -m unittest discover -s bin/tools (includes published-docs SoT) 3. python -m unittest discover -s bin/tools (includes published-docs SoT)
4. `bin/facts/audit self` (lexicon internal consistency; `bin/facts/audit.go` is the D14 wrapper) 4. `bin/facts/audit self` (lexicon internal consistency; `bin/facts/audit.go` is the D14 wrapper)
5. `bin/kb/eval` (recall@5 ≥ 0.95). Local SoT is `bin/brain/eval.go` via Zig CGO. 5. `bin/kb/eval` (recall@5 ≥ 0.95). Local SoT is `bin/brain/eval.go` via Zig CGO.
CI SoT switch: [#19](https://git.produktor.io/eSlider/2dph/issues/19).
6. `bin/cgo/zig go build -tags system_ladybug` (compile search with zig cc; fetches pinned zig+libs). 6. `bin/cgo/zig go build -tags system_ladybug` (compile search with zig cc; fetches pinned zig+libs).
Feedback loop: every commit → PR → CI → green/gate → merge. Same discipline as Feedback loop: every commit → PR → CI → green/gate → merge. Same discipline as
@@ -151,5 +155,23 @@ Feedback loop: every commit → PR → CI → green/gate → merge. Same discipl
4. .venv: ladybug + model2vec + mistune 4. .venv: ladybug + model2vec + mistune
5. schema + tools with TDD (kb + md + facts + brain) 5. schema + tools with TDD (kb + md + facts + brain)
6. ~/.config/brain config 6. ~/.config/brain config
7. corpus extraction (facts/info) 7. corpus extraction (facts/info)**open**: [#18](https://git.produktor.io/eSlider/2dph/issues/18)
8. verify: web-search smoke, onlyoffice pg, md-db round-trip, eval, audit 8. verify: web-search smoke, onlyoffice pg, md-db round-trip, eval, audit
## Gap to v1 (epic #16)
Read path + MCP are in. The detective brain is not closed until the graph is
**writable incrementally** and search can **walk** it. Board:
[epic #16](https://git.produktor.io/eSlider/2dph/issues/16),
milestone [v1 detective brain](https://git.produktor.io/eSlider/2dph/milestone/12).
Narrative: [docs/roadmap.md](docs/roadmap.md).
| Order | Issue | Gap |
|-------|-------|-----|
| 1 | [#14](https://git.produktor.io/eSlider/2dph/issues/14) | Write stays Python rebuild; `brain/add` / `POST /ingest` are hints. Ladybug 0.19 WAL corrupts if new leafs land while FTS/HNSW exist. |
| 2 | [#17](https://git.produktor.io/eSlider/2dph/issues/17) | `--hop` errors. `FROM_FILE` / `HAS_VERSION` are in schema; search does not walk them. |
| 3 | [#18](https://git.produktor.io/eSlider/2dph/issues/18) | Rebuild is mostly `info` (repo md + mail). `facts/extract` and chats are not a first-class index input. WhatsApp sync is a stub. |
| 4 | [#15](https://git.produktor.io/eSlider/2dph/issues/15) | Lever = 2dph fact-check. Loop = PicoClaw/MCP `search``get``audit`. Specify in-repo, not only live config. |
| 5 | [#19](https://git.produktor.io/eSlider/2dph/issues/19) | GitHub CI recall still runs Python `bin/kb/eval`. |
Does **not** block epic close: [#6](https://git.produktor.io/eSlider/2dph/issues/6) OCR, OQ1, OQ3, OQ4.
+4 -2
View File
@@ -172,7 +172,9 @@ docker compose up brain-watch # auto re-index on change
skills (`web-search`, `postgres`, …) that 2dph integrates skills (`web-search`, `postgres`, …) that 2dph integrates
- detective method — the two-source method - detective method — the two-source method
Work board (issues): [git.produktor.io/eSlider/2dph/issues](https://git.produktor.io/eSlider/2dph/issues). Work board (issues): [epic #16](https://git.produktor.io/eSlider/2dph/issues/16)
on [git.produktor.io/eSlider/2dph/issues](https://git.produktor.io/eSlider/2dph/issues).
PRs and CI: GitHub [`eSlider/2dph`](https://github.com/eSlider/2dph). PRs and CI: GitHub [`eSlider/2dph`](https://github.com/eSlider/2dph).
See [PLAN.md](PLAN.md) for decisions, execution status, and v2 open questions. See [PLAN.md](PLAN.md) for decisions, [docs/roadmap.md](docs/roadmap.md) for
the gap to v1, and v2 open questions.
+17
View File
@@ -147,6 +147,7 @@ class PublishedDocsTest(unittest.TestCase):
ROOT / "skills" / "diataxis-docs" / "SKILL.md", ROOT / "skills" / "diataxis-docs" / "SKILL.md",
ROOT / "docs" / "runbook.md", ROOT / "docs" / "runbook.md",
ROOT / "docs" / "README.md", ROOT / "docs" / "README.md",
ROOT / "docs" / "roadmap.md",
] ]
# Command-style `--hop 1` / `--hop N` plus follow/walk = the old lie. # Command-style `--hop 1` / `--hop N` plus follow/walk = the old lie.
# Honest "not implemented" notes must not match. # Honest "not implemented" notes must not match.
@@ -173,3 +174,19 @@ class PublishedDocsTest(unittest.TestCase):
readme = (ROOT / "README.md").read_text() readme = (ROOT / "README.md").read_text()
self.assertIn("docs/runbook.md", readme) self.assertIn("docs/runbook.md", readme)
self.assertNotIn("search.ops.io", readme) self.assertNotIn("search.ops.io", readme)
def test_v1_epic_is_named_in_docs(self) -> None:
plan = (ROOT / "PLAN.md").read_text()
self.assertIn("Gap to v1", plan)
self.assertIn("eSlider/2dph/issues/16", plan)
self.assertIn("eSlider/2dph/issues/17", plan)
self.assertIn("eSlider/2dph/milestone/12", plan)
road = (ROOT / "docs" / "roadmap.md").read_text()
self.assertIn("type: explanation", road)
self.assertIn("issues/16", road)
self.assertIn("issues/14", road)
index = (ROOT / "docs" / "README.md").read_text()
self.assertIn("roadmap.md", index)
self.assertIn("epic #16", index)
agents = (ROOT / "AGENTS.md").read_text()
self.assertIn("roadmap.md", agents)
+6 -2
View File
@@ -5,6 +5,7 @@ related:
- docs/runbook.md - docs/runbook.md
- docs/design.md - docs/design.md
- PLAN.md - PLAN.md
- docs/roadmap.md
--- ---
# 2dph docs (Diataxis) # 2dph docs (Diataxis)
@@ -16,6 +17,7 @@ Evidence-first knowledge graph. Facts need proof or they are
|------|-----| |------|-----|
| tutorial / howto | [runbook](runbook.md) — run anywhere (uv, Go, Docker) | | tutorial / howto | [runbook](runbook.md) — run anywhere (uv, Go, Docker) |
| explanation | [design](design.md) — two roots, deduction, D17/D20/D18 | | explanation | [design](design.md) — two roots, deduction, D17/D20/D18 |
| explanation | [roadmap](roadmap.md) — gap to v1 (epic #16) |
| howto | [picoclaw](picoclaw.md) — MCP agent profile | | howto | [picoclaw](picoclaw.md) — MCP agent profile |
| howto | [reasoner](reasoner.md) — CPU bake-off (D18) | | howto | [reasoner](reasoner.md) — CPU bake-off (D18) |
| reference | [PLAN.md](../PLAN.md) — decisions D1D21 | | reference | [PLAN.md](../PLAN.md) — decisions D1D21 |
@@ -26,9 +28,11 @@ Python write sidecar, **D14** `bin/{subject}/{method}.go`, **D15** Gitea origin,
Search: `bin/brain/search.go "query"` (HTTP: `bin/brain/serve.go` Search: `bin/brain/search.go "query"` (HTTP: `bin/brain/serve.go`
`/health` `/search` `/get` `/stats` `/audit` `/ingest`). `--hop` is `/health` `/search` `/get` `/stats` `/audit` `/ingest`). `--hop` is
not a walk; the flag errors until File/FROM_FILE edges exist. not a walk; the flag errors. Schema has `FROM_FILE`; search does not
use it ([#17](https://git.produktor.io/eSlider/2dph/issues/17)).
Work board: [Gitea issues](https://git.produktor.io/eSlider/2dph/issues). Work board: [Gitea issues](https://git.produktor.io/eSlider/2dph/issues)
([epic #16](https://git.produktor.io/eSlider/2dph/issues/16)).
PRs and CI: GitHub [`eSlider/2dph`](https://github.com/eSlider/2dph). PRs and CI: GitHub [`eSlider/2dph`](https://github.com/eSlider/2dph).
Published docs live here and match live commands. Published docs live here and match live commands.
+8 -3
View File
@@ -4,6 +4,7 @@ status: current
related: related:
- docs/README.md - docs/README.md
- docs/runbook.md - docs/runbook.md
- docs/roadmap.md
--- ---
# Design — facts, info, deduction # Design — facts, info, deduction
@@ -33,8 +34,9 @@ bin/brain/search.go "question"
is not evidence of absence; `--no-web` / `--root` skip it) is not evidence of absence; `--no-web` / `--root` skip it)
``` ```
`--hop` is not implemented yet (needs File/FROM_FILE edges). The flag is an `--hop` is not implemented. `FROM_FILE` / `HAS_VERSION` exist in schema;
error; it is not a graph walk. search does not walk them ([#17](https://git.produktor.io/eSlider/2dph/issues/17)).
The flag is an error; it is not a graph walk.
## Who / What / How / Where / When + evidence ## Who / What / How / Where / When + evidence
@@ -94,4 +96,7 @@ Pluggable OpenAI-compatible URL. RAM: `Qwen/Qwen3.5-9B`. Quality:
`prism-ml/Bonsai-27B-gguf` or `Qwen/Qwen3.6-27B`. No official Qwen3.6-9B. `prism-ml/Bonsai-27B-gguf` or `Qwen/Qwen3.6-27B`. No official Qwen3.6-9B.
CPU sidecar: compose profile `reasoner` (`OLLAMA_NUM_GPU=0`, CPU sidecar: compose profile `reasoner` (`OLLAMA_NUM_GPU=0`,
`127.0.0.1:11435`). Bake-off: `bin/reasoner/bakeoff.go`. Weights stay out `127.0.0.1:11435`). Bake-off: `bin/reasoner/bakeoff.go`. Weights stay out
of the 2dph image. See [docs/reasoner.md](reasoner.md). of the 2dph image. See [docs/reasoner.md](reasoner.md).
Gap to v1 (write, hops, corpus, agent loop): [roadmap](roadmap.md),
[epic #16](https://git.produktor.io/eSlider/2dph/issues/16).
+67
View File
@@ -0,0 +1,67 @@
---
type: explanation
status: current
related:
- PLAN.md
- docs/design.md
- docs/runbook.md
---
# Gap to v1 — detective brain
Goal: a brain that does not assert without proof. Search is deduction
(`facts` ≥2 sources → `info``web`). `confirmed` only from the facts root.
**v1 is a living graph the agent can write and walk**, not “more RAG”.
Epic: [Gitea #16](https://git.produktor.io/eSlider/2dph/issues/16).
Milestone: [v1 detective brain](https://git.produktor.io/eSlider/2dph/milestone/12).
Decisions: [PLAN.md](../PLAN.md).
## In (do not reopen)
Read path Go + Zig CGO (D21). HTTP + OpenAPI + MCP (D20). PicoClaw compose
profile + CPU reasoner (D18). Mail sync → import → rebuild. D14 shebangs.
Compose `api` (no CPython) / `index` (Python write). Issues #1#5, #7#13.
`POST /ingest` is a rebuild **hint**. `add` is not implemented.
## Blockers
```
question
├─ FTS + HNSW ← in
├─ facts / info roots ← in
├─ web (D17) ← in
├─ Cypher hop ← #17 schema yes, search no
├─ brain/add ACID ← #14 rebuild only
└─ facts+chats corpus ← #18
```
1. **[#14](https://git.produktor.io/eSlider/2dph/issues/14) write** —
`bin/brain/index.go --rebuild` (Python `kblib`). No incremental
`brain/add`. Watch/mail/git cannot become facts “now”.
2. **[#17](https://git.produktor.io/eSlider/2dph/issues/17) hops** —
`Leaf-[:FROM_FILE]->File-[:HAS_VERSION]->Commit-[:AUTHORED]->Person`
exists; `--hop` still errors. Without a walk, D9/D10 are paper.
3. **[#18](https://git.produktor.io/eSlider/2dph/issues/18) corpus** —
rebuild loads repo markdown + mail as `info`. `facts/extract` pairing
and `bin/chats` are not indexed. WhatsApp is a stub. PII stays in `var/`.
4. **[#15](https://git.produktor.io/eSlider/2dph/issues/15) lever/loop** —
2dph is the lever (`search``get``audit`). PicoClaw is the loop.
Document the contour in-repo; CPU turns need a large context window.
5. **[#19](https://git.produktor.io/eSlider/2dph/issues/19) CI eval** —
recall SoT should be `bin/brain/eval.go` via Zig, not Python `bin/kb/eval`.
## Not v1
[#6](https://git.produktor.io/eSlider/2dph/issues/6) OCR (OQ2), OQ1
contradiction resolution, OQ3 duckdb-md export, OQ4 YAML-first leafs.
## Close epic #16 when
- facts+info can be written without a full rebuild for every leaf
- `--hop` stops erroring and runs a Cypher path from search hits
- ops pairing + chat import land as leafs on rebuild
- MCP tool order is documented and still gated by tests
+1
View File
@@ -55,6 +55,7 @@ bin/brain/stats.go
``` ```
`--hop` is not implemented. Empty web results are `throttled`, not absence. `--hop` is not implemented. Empty web results are `throttled`, not absence.
Gap to v1: [roadmap](roadmap.md) / [epic #16](https://git.produktor.io/eSlider/2dph/issues/16).
Ladybug 0.19: never `DROP INDEX` FTS/VECTOR (ghost catalog). Fresh indexes = Ladybug 0.19: never `DROP INDEX` FTS/VECTOR (ghost catalog). Fresh indexes =
delete `var/kb.lbug` then `--rebuild`. delete `var/kb.lbug` then `--rebuild`.
+3 -2
View File
@@ -31,8 +31,9 @@ bin/brain/stats.go # index health
bin/brain/eval.go # recall@5 >= 0.95 gate (Go; Python bin/kb/eval is CI fallback) bin/brain/eval.go # recall@5 >= 0.95 gate (Go; Python bin/kb/eval is CI fallback)
``` ```
`bin/kb/search` is a deprecated wrapper. `--hop` errors (File/FROM_FILE edges `bin/kb/search` is a deprecated wrapper. `--hop` errors (schema has
are not wired yet); do not treat it as a graph walk. `FROM_FILE`; search does not walk it yet, [#17](https://git.produktor.io/eSlider/2dph/issues/17));
do not treat it as a graph walk.
## Rules ## Rules