feat: CPU reasoner bake-off for Qwen3.5-9B vs Bonsai (#25)

Measure OpenAI tool_calls (search/get/audit) on a CPU Ollama sidecar instead of gating D18 on GPU or PicoClaw. Weights stay out of the image.
2026-08-13 23:19:51 +01:00
parent e393cc6a99
commit bb272d7416
13 changed files with 717 additions and 3 deletions
@@ -6,6 +6,7 @@ Brain/ops/eSlider stack. Facts need proof or they are

 - [PLAN.md](../PLAN.md) — decisions, execution order, open questions (v2)
 - [design](design.md) — schema, deduction model, sources
+- [reasoner](reasoner.md) — D18 CPU bake-off (Qwen3.5-9B vs Bonsai / Qwen3.6-27B)
 - [Gitea issues](https://git.produktor.io/eSlider/2dph/issues) — work board (origin)

 Search: `bin/brain/search.go "query"` (HTTP: `bin/brain/serve.go` —
@@ -78,4 +78,12 @@ fetches Zig + libs (`bin/cgo/zig`). Index/write is still `bin/kb/index`
 `bin/brain/serve.go` exposes the same `internal/httpapi.Ops` table as OpenAPI
 (`GET /openapi.json`) and MCP (`POST /mcp` JSON-RPC `tools/list` +
 `tools/call`). Tool names match paths: `search`, `get`, `stats`, `audit`.
-Agents should use these endpoints instead of shebang CLIs.
+Agents should use these endpoints instead of shebang CLIs.
+
+## Reasoner (D18)
+
+Pluggable OpenAI-compatible URL. RAM: `Qwen/Qwen3.5-9B`. Quality:
+`prism-ml/Bonsai-27B-gguf` or `Qwen/Qwen3.6-27B`. No official Qwen3.6-9B.
+CPU sidecar: compose profile `reasoner` (`OLLAMA_NUM_GPU=0`,
+`127.0.0.1:11435`). Bake-off: `bin/reasoner/bakeoff.go`. Weights stay out
+of the 2dph image. See [docs/reasoner.md](reasoner.md).
@@ -0,0 +1,69 @@
+# Reasoner bake-off (D18)
+
+Pluggable OpenAI-compatible URL. 2dph does not ship weights. PicoClaw is
+not in this repo; the bake-off hits the same tool names PicoClaw would
+(`search` → `get` → `audit` from `internal/httpapi.Ops`).
+
+```bash
+docker compose --profile reasoner up -d reasoner
+docker compose --profile reasoner exec reasoner ollama pull qwen3.5:9b
+REASONER_BASE_URL=http://127.0.0.1:11435/v1 REASONER_MODEL=qwen3.5:9b \
+  ./bin/reasoner/bakeoff.go --json
+```
+
+Host Ollama on `:11434` is left alone. This sidecar binds `127.0.0.1:11435`
+with `OLLAMA_NUM_GPU=0` (CPU). Measure RSS (`/api/ps` `size`), not VRAM.
+
+If Compose cannot allocate a project network (Docker IPAM pool exhausted),
+the same sidecar is:
+
+```bash
+docker run -d --name 2dph-reasoner \
+  -e OLLAMA_NUM_GPU=0 \
+  -p 127.0.0.1:11435:11434 \
+  -v 2dph-reasoner-ollama:/root/.ollama \
+  ollama/ollama:latest
+```
+
+## Real Hugging Face ids
+
+| Role | HF id | Ollama tag (this bake-off) |
+|------|-------|----------------------------|
+| RAM / 9B | `Qwen/Qwen3.5-9B` | `qwen3.5:9b` |
+| Quality 27B (CPU) | `prism-ml/Bonsai-27B-gguf` (derived from Qwen3.6-27B) | `MichelRosselli/bonsai-27b:Q1_0` |
+| Quality 27B (full) | `Qwen/Qwen3.6-27B` | not pulled on this CPU box |
+
+There is **no official Qwen3.6-9B**. Do not invent that id.
+
+Qwen3.5-9B has documented upstream tool-call XML bugs. A 9B win on tools
+is only claimed if this bake-off records OpenAI `tool_calls` (not
+`<tool_call>` XML in `content`).
+
+The 2dph API image does not `COPY` GGUF/safetensors. Pull at runtime into
+the `reasoner-ollama` volume.
+
+## Live CPU run
+
+Host sidecar: Ollama **0.32.9**, `OLLAMA_NUM_GPU=0`, `127.0.0.1:11435`,
+`device: cpu`, `vram_mb: 0`. Date: 2026-08-13. Same three prompts
+(`search` / `get` / `audit`). PicoClaw binary was not used; the OpenAI
+tools payload is the surface it would send.
+
+| Model | HF id | tool_call | xml_leak | rss_mb | latency_ms (search/get/audit) |
+|-------|-------|-----------|----------|--------|-------------------------------|
+| `qwen3.5:9b` | `Qwen/Qwen3.5-9B` | 3/3 | 0 | 5790 | 50059 / 51678 / 31980 |
+| `MichelRosselli/bonsai-27b:Q1_0` | `prism-ml/Bonsai-27B-gguf` | 3/3 | 0 | 21951 | 327702 / 166830 / 119808 |
+| `Qwen/Qwen3.6-27B` | `Qwen/Qwen3.6-27B` | not loaded | — | — | too heavy for this CPU box |
+
+Both loaded models emitted OpenAI `tool_calls` (not `<tool_call>` XML) on
+this runtime. **Do not claim 9B is better at tools** — the score is tied
+at 3/3. 9B is smaller and faster. Bonsai RSS includes weights + KV
+(`size` from `/api/ps`); first Bonsai prompt includes cold load.
+
+Re-run:
+
+```bash
+REASONER_BASE_URL=http://127.0.0.1:11435/v1 REASONER_MODEL=qwen3.5:9b \
+  ./bin/reasoner/bakeoff.go --json
+REASONER_MODEL=MichelRosselli/bonsai-27b:Q1_0 ./bin/reasoner/bakeoff.go --json
+```