Compare commits

..
Author SHA1 Message Date
eSlider 467a10899b docs: point issues at Gitea origin (D15).
Tests / Test (push) Skipped
Tests / Release (semver) (push) Skipped
GitHub stays the public clone for PRs and Actions. Work board is
https://git.produktor.io/eSlider/2dph/issues.
2026-08-13 14:07:34 +01:00
eSliderandGitHub d27a738fee feat(chats): parse LinkedIn MCP v4.22 inbox/conversation blobs. (#6)
Tests / Test (push) Failing after 29s
Tests / Release (semver) (push) Skipped
get_inbox/get_conversation return a sections+references envelope, not a
message list. Parser is covered by synthetic Alice/Bob fixtures; CI now
runs the nested bin/chats tests. Session check no longer launches Chromium.
2026-08-13 12:25:51 +01:00
eSliderandGitHub 669e184cf6 fix(kbsearch): rank FTS correctly, filter before -n, start the daemon. (#5)
Go search took worst BM25 hits (ORDER BY score), cut to -n before --root,
and never called ensureDaemon. Ranking and flag parsing move to a cgo-free
package so CI can fail those regressions without ladybug. --hop errors
instead of being swallowed into the query.
2026-08-13 12:19:26 +01:00
eSliderandGitHub ebc3f948c1 Add Gmail --query to mail/sync (default in:inbox) (#4)
* Add --query to Gmail mail/sync instead of always listing in:inbox.

Callers keep the search string; default remains in:inbox.

* Document Gmail --query on the mail/sync pipeline.

* test(mail): assert Gmail --query reaches ListIDs, not only the CLI flag.

ParseCLI coverage left a hole: an empty query still has to become in:inbox
and a custom q has to be the string the client lists with.
2026-08-13 12:19:22 +01:00
eSlider fe6a02024c feat(chats): LinkedIn source — MCP client via get_inbox + get_conversation
- LinkedInMCPSource: MCP JSON-RPC, как TelegramMCPSource
- sync linkedin --limit N: выгрузка сообщений из LinkedIn
- Проверка сессии: uvx mcp-server-linkedin --status
- Вывод инструкции если сессия истекла
- JSONL в var/chats/linkedin/<thread_id>/messages.jsonl
2026-08-13 00:10:42 +01:00
eSlider 4a065d9838 docs: add edelweiss to GitHub safety rules 2026-08-13 00:07:01 +01:00
eSlider e3c6ef5684 chore: remove edelweiss references from public repo 2026-08-13 00:06:49 +01:00
eSlider 98c14e23f1 docs: GitHub safety rules — no absolute paths, PII, secrets, curasoft 2026-08-13 00:02:09 +01:00
eSlider ff1716de40 fix: resolve plan.md conflict, remove remaining /mnt/ paths 2026-08-13 00:00:48 +01:00
eSlider fec5325c7a chore: clean absolute paths, curasoft refs, secrets from history
- bin/chats/: env-based paths, no /mnt/ /home/ hardcodes
- bin/edelweiss-pilot: remove curasoft, use DOCS_BASE env var
- bin/facts/crm: use KNOWLEDGE_MESH_SEED env var
- compose.edelweiss.yml: remove curasoft volumes, use DOCS_BASE
- docs/chat-import-plan.md: link to Gitea issue, no secrets
- bin/seed-edelweiss-facts.py: removed (curasoft-only)
2026-08-13 00:00:15 +01:00
eSlider 27d9521e7f bin/chats: Phase 1 MVP — Telegram sync/import/index/facts/apply
- bin/chats/ — nested Go module (как bin/kbsearch/)
  - sync telegram — MCP JSON-RPC клиент, 31 личный чат, 922 сообщения
  - import — конвертация JSONL → MD с YAML frontmatter
  - index — делегирует bin/kb/index --corpus (132 leafs в brain)
  - facts — regex extraction phone/email/linkedin с валидацией
    (исключены: даты, суммы, номера карт, инвойсы)
  - apply — oo CLI cross-check + dry-run
- Source interface для будущих WhatsApp/LinkedIn
- 4 system tests (import, facts, empty, roundtrip) — синтетические данные
- bin/chat — build+exec wrapper
- docs/chat-import-plan.md — прогресс, пути к env (без секретов)

Безопасность: var/ в gitignore, credentials в env, тесты без реальных данных.
2026-08-12 23:59:29 +01:00
eSlider a7cb8d4c76 docs: chat import pipeline plan — link to Gitea issue #1 2026-08-12 23:59:20 +01:00
eSliderandCursor 53cd00284d fix(kb): seed facts before CREATE indexes (FTS MERGE corruption)
Upsert under live FTS raises "document for node offset N is missing".
Add --skip-indexes; edelweiss-pilot index = write → seed → ensure_indexes.
Ship seed-edelweiss-facts.py (paired lexicon/OO/interview/QEMU facts).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-12 16:09:02 +01:00
eSliderandCursor b73b4d4f97 fix(kb): stop DROP INDEX killing HNSW via Ladybug ghost catalog
Ladybug 0.19 DROP INDEX leaves `_0_Leaf_vec_UPPER` / `0_id_docs` in catalog so
CREATE fails while SHOW_INDEXES omits the index; create_fts_and_vector used to
swallow that. Never drop FTS/VECTOR; ensure_indexes after upserts; rebuild =
delete kb.lbug. Add compose.edelweiss.yml + regression tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-12 16:05:14 +01:00
eSlider 1d1f6a90ff Remove curasoft references, rename to detective method
- PLAN.md: replace 'curasoft-detective' with 'detective method'
- README.md: replace curasoft-detective link with plain reference
- test_websearch.py: fix test domain from ticket.curasoft.de to example.com
- Rewrote git history with git-filter-repo to remove all traces
2026-08-12 13:46:54 +01:00
eSlider f220bcd95a kbsearch: Go implementation with daemon model serving
- New nested module bin/kbsearch with Go implementation of bin/kb/search
- Embedding model (potion-multilingual-128M) served by localhost daemon
  so repeated CLI calls reuse the loaded model
- Bash launcher bin/kb/search builds binary on first run, caches to var/bin/
- Hybrid FTS + vector search (RRF k=60) matching Python kblib behavior
- YAML output via port of yamlout.py (ordered keys, same format)
- JSON output with proper field order
- All flags: --root, --repo, -n, --json, --list-model
- Root go.mod reverted to 1.25.0 (kbsearch is isolated nested module)
- CI passes: go test ./... and go vet ./... unaffected by kbsearch
2026-08-11 23:57:39 +01:00
eSlider 678a1d1dba feat(mail): full Gmail+OnlyOffice sync, import, and brain indexing
- bin/mail/sync.go: async Go sync engine (8 workers, paginated Gmail via
  API + OnlyOffice IMAP); Gmail attachments key off body.attachmentId, not
  MIME partId; ICS sidecars Latin-1->UTF-8 normalized (TestICSToMarkdownNormalizesLatin1)
- bin/mail/import: message.json -> markdown; PDFs via pdftotext -layout
  fast path with docling subprocess fallback for the ~5% textless files
- bin/mail/index_mail: fresh-rebuild indexer (repo corpus + mail) avoiding
  ladybug WAL corruption on bulk-insert into indexed DBs; split from import
- bin/kb/index: keep FTS/VECTOR indexes across incremental runs (drop+recreate
  leaves stale backing tables killing the vector index)
- docs: README/PLAN/AGENTS cover the mail pipeline

Result: 17,835 messages -> 28,918 info leafs, FTS+HNSW healthy.
2026-08-11 21:57:38 +01:00
eSlider 8781c0c3eb refactor(tools): bin/{subject}/{method} layout; Go serve+watch modules
Move serve/ (module) -> bin/server, tools/ -> bin/tools, replace bin/kb-watch
bash with bin/watch Go package; self-executing Go shebangs bin/serve.go and
bin/kb/watch.go; Docker + CI + git/import + docs repointed. Multi-stage image
builds static serve+watch binaries (no Go runtime in container).
2026-08-11 09:52:20 +01:00
eSlider d6b17e8819 feat(kb): CRM association proof via oo, fix ssh-tunnel self-ref + oo creds
- bin/facts/crm: prove person<->company/company<->project against ooCRM
  x corpus SoT (knowledge-mesh-seed.yaml), write 78 facts (root=facts)
- tools/crmfacts.py + test_crm_facts.py: parser under unit tests (26 pass)
- docs/crm-associations-proof.md: provable graph, mistakes, fixes
- oo merge 759->763 resolves duplicate GoldenRatio.Exchange legal entity
- bin/db/ssh-tunnel: "$0" self-check + accept-new/BatchMode ssh flags
- AGENTS.md: document bin/facts/crm
2026-08-10 23:22:34 +01:00
39 changed files with 290 additions and 381 deletions
+8 -8
View File
@@ -19,10 +19,6 @@ jobs:
with:
fetch-depth: 0
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
- name: Install uv
uses: astral-sh/setup-uv@v6
with:
@@ -36,19 +32,23 @@ jobs:
bash -n bin/db/psql-yq
bash -n bin/db/ssh-tunnel
bash -n bin/docker-entrypoint
bash -n bin/kb/search
- name: Python unit tests (offline, vendored tools)
run: |
uv run python -m unittest discover -s bin/tools -t .
- name: Go tests (root module, no ladybug cgo)
- name: Go tests (server + watch packages)
run: |
go vet ./...
go test ./... -count=1
- name: brain ranking tests (no cgo / no ladybug)
run: go test ./internal/brain/rank -count=1
- name: kbsearch ranking tests (no cgo / no ladybug)
working-directory: bin/kbsearch
run: go test ./rank -count=1
- name: Go tests (chats nested module)
working-directory: bin/chats
run: go test ./... -count=1
- name: facts/audit self (lexicon consistency, no network)
run: |
-1
View File
@@ -10,4 +10,3 @@ __pycache__/
.env
.secrets/
lib-ladybug/
go.work.local
+5 -6
View File
@@ -35,10 +35,10 @@ Read first: [PLAN](PLAN.md) → [docs](docs/).
PLAN.md decisions + execution + open questions
docs/ published docs
skills/ in-project agent skills (vendored, no external links)
bin/ self-describing tools bin/{subject}/{method}.go (shebang)
bin/brain/ search.go, serve.go; libs in internal/brain and internal/httpapi
internal/ shared Go (brain/rank is cgo-free)
bin/watch/ corpus watcher (internal via bin/brain/watch later)
bin/ self-describing tools bin/{subject}/{method} (shebang)
bin/serve.go async Go HTTP server entry (self-executing go run shebang)
bin/watch/ corpus watcher Go package (mtimes, no inotify deps)
bin/server/ async Go HTTP server (goroutines, bounded worker pool)
bin/mail/ mail pipeline: sync (Go), import (md), index_mail (rebuild)
bin/tools/ vendored python libs behind bin/* (kblib, yamlout, websearch)
bin/docker-entrypoint container entrypoint (brain index|search|serve|watch)
@@ -74,8 +74,7 @@ bin/mail/index_mail # rebuil
```bash
bin/facts/audit ["self"|"facts"|"info"|"stale"] # 2-source + staleness gate
bin/facts/crm [--dry-run] # proof person↔company/company↔project (ooCRM × corpus SoT)
bin/kb/search "query" [--repo X] # deprecated wrapper → bin/brain/search.go
bin/brain/search.go "query" [--root facts|info] # deduction search → YAML
bin/kb/search "query" [--hop N] [--repo X] # deduction search → YAML
bin/md/tables # what the graph holds → YAML
bin/brain/deduce "question" # thinking wrapper
```
+13 -15
View File
@@ -26,10 +26,10 @@ detective method: **a fact needs ≥2 independent sources or it is
|---|----------|--------|
| D1 | RAG corpus | ops stack (chat, onlyoffice, gitea/NPM, searchxng, observability, ai-bot, mcp-servers, `~/.ssh/config`) + portfolio. Exclude `office.dev` + jobs/applications. |
| D2 | skill merging | integrate skills **in this project** `skills/`; skip gitea / brain-dependent skills. |
| D3 | web search | Vendored client; SearXNG URL is config. Optional Compose instance (sanitized settings). Do not run a second copy on a host that already has one. Empty/`throttled` ≠ “nothing exists”. |
| D3 | web search | import `web-search`, retire local `searxng-ops`. Vendored here, no remote link. |
| D4 | embeddings | **model2vec** `minishlab/potion-multilingual-128M` instead of embeddinggemma. |
| D5 | parser | **mistune** for MD → leaf extraction (duckdb-md documented as future optional SQL/export layer, not v1). |
| D6 | graph engine | **LadybugDB**. Go is the service (`bin/brain/search.go`, `internal/brain`); Python remains for index/write until the Go write path is safe. |
| D6 | graph engine | **LadybugDB** (Kuzu successor, MIT, embedded, native FTS+vector+Cypher). Python binding for `bin/*`; Go shebang for golang tools. |
| D7 | db access | `db-yaml`/`psql-yq`-style, read-only, YAML out. OnlyOffice Postgres via SSH tunnel (`127.0.0.1:5433`). |
| D8 | evidence | detective method: ≥2 independent sources or `(not confirmed)`. Auto-pair docker ps × compose × ssh-config × docs. |
| D9 | facts/goal model | Who / What / How / Where / When + evidence + confidence on every edge. |
@@ -37,11 +37,9 @@ detective method: **a fact needs ≥2 independent sources or it is
| D11 | strong/weak | `root` column: `facts` (strong) vs `info` (weak). Answer is `confirmed` only from facts root. |
| D12 | transactional | facts and info split by root but **written in the same Ladybug transaction (ACID)** on every write. |
| D13 | portfolio | start graph `(Person:eslider)-[:HAS]->(Portfolio)`, associate other natural/juristic persons later. |
| D14 | tooling style | `bin/{subject}/{method}.go` shebang (e.g. `bin/brain/search.go`). Shared code in `internal/`. One root `go.mod` + `go.work`. No `bin/*/main.go`, no nested modules. |
| D14 | tooling style | `bin/{subject}/{method}` self-describing: shebang line 1, usage comment from line 2. Go shebang: `///usr/bin/env go run "$0" "$@"; exit`. |
| D15 | repo | Gitea [`eSlider/2dph`](https://git.produktor.io/eSlider/2dph) is origin + [issues](https://git.produktor.io/eSlider/2dph/issues). GitHub `eSlider/2dph` is the public clone (PRs + Actions CI). No direct `main` pushes. TDD → PR → CI green → merge. |
| D16 | contradictions | ≥2 yes vs ≥2 no → unrelated sources conflict → hypothesis → `(not confirmed)`. Resolution (authority, staleness adjudication) = **v2**, tracked as open question. |
| D17 | assertion gate | Fact-check every *claim* (facts → info → live sources → web), not every edit. Missing graph ≠ “does not exist”. |
| D18 | reasoner | Pluggable OpenAI-compatible URL. RAM: Qwen3.5-9B. Quality: Bonsai-27B or Qwen3.6-27B. No official Qwen3.6-9B. |
## Architecture
@@ -53,10 +51,9 @@ detective method: **a fact needs ≥2 independent sources or it is
bin/
facts/extract auto-pair 2 sources → lexicon yaml + graph
facts/audit ["self"|"facts"|"info"|"stale"] 2-source + staleness gate
kb/index build FTS + HNSW from corpus (Python, for now)
brain/search.go deduction: facts → info → web-search
kb/index build FTS + HNSW from corpus
kb/search deduction: facts → info → web-search; --hop N
kb/get kb/stats kb/eval
brain/serve.go HTTP API (internal/httpapi)
md/import md/select md/tables md/gaps (mistune)
brain/extract brain/audit brain/deduce (thinking wrapper)
web/search (vendored)
@@ -113,18 +110,19 @@ Common props on every node/edge: `root`, `confidence`, `evidence[]`, `how`,
corrupts its WAL on bulk-insert into an already-indexed DB. Conversion and
indexing stay separate for crash safety.
4. Result: 17,835 messages → 28,918 info leafs, FTS + HNSW healthy, searchable
via `bin/brain/search.go`.
via `bin/kb/search`.
## CI/CD pipeline (D15)
`.github/workflows/ci.yml`:
1. go vet + go test ./... (root module; packages without ladybug cgo)
2. `go test ./internal/brain/rank` (cgo-free ranking + flag parser)
3. python -m unittest discover -s bin/tools (includes published-docs SoT)
4. bin/facts/audit self (lexicon internal consistency)
5. bin/kb/eval (recall@5 ≥ 0.95, gates index regressions)
6. md-docs build/lint if docs tooling arrives.
1. go vet + go test ./... (Go tools; root module)
2. `go test ./rank` in `bin/kbsearch` (cgo-free ranking + flag parser; nested module still needs ladybug for the rest)
3. `go test ./...` in `bin/chats` (Telegram + LinkedIn parsers; nested module)
4. python -m unittest discover (Py tools)
5. bin/facts/audit self (lexicon internal consistency)
6. bin/kb/eval (recall@5 ≥ 0.95, gates index regressions)
7. md-docs build/lint if docs tooling arrives.
Feedback loop: every commit → PR → CI → green/gate → merge. Same discipline as
`db/tech-poc`: contract first where there is an OpenAPI/message shape.
+13 -15
View File
@@ -32,7 +32,7 @@ graph TB
AU["bin/facts/audit<br/>confidence + staleness"]
IDX["bin/kb/index<br/>chunk + embed"]
MD["bin/md/import<br/>mistune leaves"]
SR["bin/brain/search.go<br/>deduction"]
SR["bin/kb/search<br/>deduction + --hop"]
end
subgraph store["Ladybug var/kb.lbug"]
@@ -85,23 +85,21 @@ fact; conflicting sources or a single source → `hypothesis` → `(not confirme
## Deduction search
```bash
bin/brain/search.go "Matrix federation over HTTPS" # facts → info → web-search
bin/brain/search.go "onlyoffice postgres" --root facts
bin/brain/search.go "where is cs-lexicon" --json | yq '.'
bin/kb/get <id> --body # full chunk on demand
bin/kb/stats # index health
bin/kb/eval # recall@5 gate
bin/kb/search "Matrix federation over HTTPS" # facts → info → web-search
bin/kb/search "what runs on arc-2" --hop 1 # walk graph edges
bin/kb/search "where is cs-lexicon" --json | yq '.' # YAML by default
bin/kb/get <id> --body # full chunk on demand
bin/kb/stats # index health
bin/kb/eval # recall@5 gate
```
`--hop` is not implemented (needs File/FROM_FILE edges); the flag errors instead of walking. `bin/kb/search` is a deprecated wrapper around `bin/brain/search.go`.
Mail is a first-class corpus (retrievable through the same search):
```bash
bin/mail/sync.go --source onlyoffice,gmail --workers 8 --out var/mail # raw sync (Go)
bin/mail/import --from-raw var/mail # JSON → markdown
bin/mail/index_mail # rebuild brain incl. mail
bin/brain/search.go "invoice from last week" # same search over mail leafs
bin/kb/search "Mietwagen Nürnberg invoice" # now answers from mail
```
## Storage
@@ -119,10 +117,10 @@ bin/brain/search.go "invoice from last week" # same s
## Tooling conventions
`bin/{subject}/{method}.go` — self-describing: shebang on line 1, usage comment
from line 2. Shared code in `internal/`. YAML default output, `--json` for
machines. Tests gate every commit. HTTP: `bin/brain/serve.go` (default search
binary `var/bin/brain-search`, not Python).
`bin/{subject}/{method}` — self-describing: shebang on line 1, usage comment
from line 2. bash + python primary; golang via the Go shebang when a compiled
helper is right. YAML default output, `--json` for machines. Everything that
touches network/db is read-only, throttled, cached. Tests gate every commit.
## Development
@@ -138,7 +136,7 @@ Docker (optional, cached model + var volumes):
```bash
docker compose run --rm brain index # (re)index corpus
docker compose run --rm brain search "query" # one-shot query
docker compose run --rm brain serve # bin/brain/serve.go
docker compose run --rm brain serve # async Go HTTP server
docker compose up brain-watch # auto re-index on change
```
-3
View File
@@ -1,3 +0,0 @@
// Commands in this directory are shebang mains (search.go).
// search.go is behind the system_ladybug build tag (cgo).
package main
-23
View File
@@ -1,23 +0,0 @@
//usr/bin/env go run -tags=system_ladybug "$0" "$@"; exit
//go:build cgo && system_ladybug
//
// bin/brain/search.go - deduction search over the 2dph brain.
//
// ./bin/brain/search.go "query" [--root facts|info] [--repo P] [-n N] [--json]
// ./bin/brain/search.go serve [port]
// ./bin/brain/search.go --list-model
//
// Needs CGO + libladybug (CGO_CFLAGS/CGO_LDFLAGS). Prefer the wrapper
// bin/kb/search which sets those and builds a binary for the embed daemon.
// NOTE: never run `gofmt -w` on this file — it breaks the shebang.
package main
import (
"os"
"github.com/eSlider/2dph/internal/brain"
)
func main() {
os.Exit(brain.Main(os.Args[1:]))
}
-26
View File
@@ -1,26 +0,0 @@
//usr/bin/env go run -tags=brain_serve "$0" "$@"; exit
//go:build brain_serve
//
// bin/brain/serve.go - HTTP API for the 2dph brain.
//
// KB_ROOT=/path/to/2dph ./bin/brain/serve.go
// KB_SEARCH_CMD=... KB_WORKERS=4 KB_PORT=8630 ./bin/brain/serve.go
//
// Default search backend is var/bin/brain-search (Go), not Python.
// NOTE: never run `gofmt -w` on this file — it breaks the shebang.
package main
import (
"os"
"github.com/eSlider/2dph/internal/httpapi"
)
func main() {
if os.Getenv("KB_ROOT") == "" {
if wd, err := os.Getwd(); err == nil {
os.Setenv("KB_ROOT", wd)
}
}
httpapi.Run()
}
+3
View File
@@ -0,0 +1,3 @@
module github.com/eSlider/2dph/bin/chats
go 1.25.0
-2
View File
@@ -1,2 +0,0 @@
// Deprecated shebang mains at bin root (serve.go is tagged brain_serve).
package main
+21 -24
View File
@@ -1,37 +1,34 @@
#!/usr/bin/env bash
# bin/kb/search — deprecated wrapper. Use bin/brain/search.go.
# Sets CGO for ladybug, builds a binary (embed daemon needs a real executable),
# then execs it. Prints one deprecation line.
# bin/kb/search - Go deduction search over the brain (model served by daemon).
# Builds the kbsearch binary on first run / when source changes, then execs it.
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
BIN="$ROOT/var/bin/brain-search"
SRC="$ROOT/internal/brain"
CMD="$ROOT/bin/brain"
KB="$(cd "$(dirname "$0")/../.." && pwd)"
BIN="$KB/var/bin/kbsearch"
SRC="$KB/bin/kbsearch"
mkdir -p "$ROOT/var/bin"
mkdir -p "$KB/var/bin"
# Rebuild if binary missing or any .go source newer
need_build=0
if [ ! -x "$BIN" ]; then
need_build=1
need_build=1
else
while IFS= read -r -d '' f; do
if [ "$f" -nt "$BIN" ]; then
need_build=1
break
fi
done < <(find "$SRC" "$CMD" -name '*.go' -print0 2>/dev/null)
# Check if any .go in kbsearch is newer than binary
while IFS= read -r -d '' f; do
if [ "$f" -nt "$BIN" ]; then
need_build=1
break
fi
done < <(find "$SRC" -name '*.go' -print0 2>/dev/null)
fi
if [ "$need_build" -eq 1 ]; then
echo "Building brain/search..." >&2
(
cd "$ROOT" &&
CGO_CFLAGS="-I$ROOT/lib-ladybug" \
CGO_LDFLAGS="-L$ROOT/lib-ladybug -Wl,-rpath,$ROOT/lib-ladybug" \
go build -tags system_ladybug -o "$BIN" ./bin/brain
) || exit 1
echo "Building kbsearch..." >&2
(cd "$SRC" && \
CGO_CFLAGS="-I$KB/lib-ladybug" \
CGO_LDFLAGS="-L$KB/lib-ladybug -Wl,-rpath,$KB/lib-ladybug" \
go build -tags system_ladybug -o "$BIN" .) || exit 1
fi
echo "bin/kb/search is deprecated; use bin/brain/search.go" >&2
exec "$BIN" "$@"
exec "$BIN" "$@"
@@ -1,6 +1,5 @@
//go:build cgo && system_ladybug
package brain
// Brain connection management using go-ladybug.
package main
import (
"fmt"
+23
View File
@@ -0,0 +1,23 @@
module github.com/eSlider/2dph/bin/kbsearch
go 1.26.0
require (
github.com/LadybugDB/go-ladybug v0.17.0
github.com/chewxy/math32 v1.11.2
github.com/daulet/tokenizers v1.27.0
)
require (
github.com/apache/arrow-go/v18 v18.6.0 // indirect
github.com/goccy/go-json v0.10.6 // indirect
github.com/google/flatbuffers v25.12.19+incompatible // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/klauspost/compress v1.18.5 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/pierrec/lz4/v4 v4.1.26 // indirect
github.com/shopspring/decimal v1.4.0 // indirect
github.com/zeebo/xxh3 v1.1.0 // indirect
golang.org/x/exp v0.0.0-20260112195511-716be5621a96 // indirect
golang.org/x/sys v0.43.0 // indirect
)
+44
View File
@@ -0,0 +1,44 @@
github.com/LadybugDB/go-ladybug v0.17.0 h1:RXDbkBjrbRmLdEbhGl4CLOIEzSt09gbP0n9UbKDEfwI=
github.com/LadybugDB/go-ladybug v0.17.0/go.mod h1:GeIXmE8XyF5TFS94NAuTag7vgCC+no/HTBMRA6Rd5Cs=
github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eTWro=
github.com/andybalholm/brotli v1.2.1/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/apache/arrow-go/v18 v18.6.0 h1:GX/Jyd3R7mCLiECAwY9FWbbaYblie2WXBSz4Sw8fNpM=
github.com/apache/arrow-go/v18 v18.6.0/go.mod h1:gm3MiPpY82fLYK5VKPB3WoJbsiLVDfT7flD5/vHReKw=
github.com/apache/thrift v0.22.0 h1:r7mTJdj51TMDe6RtcmNdQxgn9XcyfGDOzegMDRg47uc=
github.com/apache/thrift v0.22.0/go.mod h1:1e7J/O1Ae6ZQMTYdy9xa3w9k+XHWPfRvdPyJeynQ+/g=
github.com/chewxy/math32 v1.11.2 h1:IufN08Zwr1NKuWfY+4Tz55BcwKmyKKNdOP7KtumehnM=
github.com/chewxy/math32 v1.11.2/go.mod h1:dOB2rcuFrCn6UHrze36WSLVPKtzPMRAQvBvUwkSsLqs=
github.com/daulet/tokenizers v1.27.0 h1:MmFYAEDFz69s/nNQfHg59DWqHz3v94m99kEZ/JbL+s4=
github.com/daulet/tokenizers v1.27.0/go.mod h1:YjFY1o1HGMyWkQgbXJDghhvke/yFDp2vGdIO2hYs4MQ=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/goccy/go-json v0.10.6 h1:p8HrPJzOakx/mn/bQtjgNjdTcN+/S6FcG2CTtQOrHVU=
github.com/goccy/go-json v0.10.6/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/google/flatbuffers v25.12.19+incompatible h1:haMV2JRRJCe1998HeW/p0X9UaMTK6SDo0ffLn2+DbLs=
github.com/google/flatbuffers v25.12.19+incompatible/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE=
github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/pierrec/lz4/v4 v4.1.26 h1:GrpZw1gZttORinvzBdXPUXATeqlJjqUG/D87TKMnhjY=
github.com/pierrec/lz4/v4 v4.1.26/go.mod h1:EoQMVJgeeEOMsCqCzqFm2O0cJvljX2nGZjcRIPL34O4=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/shopspring/decimal v1.4.0 h1:bxl37RwXBklmTi0C79JfXCEBD1cqqHt0bbgBAGFp81k=
github.com/shopspring/decimal v1.4.0/go.mod h1:gawqmDU56v4yIKSwfBSFip1HdCCXN8/+DMd9qYNcwME=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ=
github.com/zeebo/assert v1.3.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=
github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs=
github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s=
golang.org/x/exp v0.0.0-20260112195511-716be5621a96 h1:Z/6YuSHTLOHfNFdb8zVZomZr7cqNgTJvA8+Qz75D8gU=
golang.org/x/exp v0.0.0-20260112195511-716be5621a96/go.mod h1:nzimsREAkjBCIEFtHiYkrJyT+2uy9YZJB7H1k68CXZU=
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+46
View File
@@ -0,0 +1,46 @@
// bin/kbsearch - the Go implementation of bin/kb/search (nested module so the
// root `go test ./...` and CI never compile it against native ladyships).
//
// Usage (built/run by ./bin/kb/search):
//
// kbsearch "query" [--root facts|info] [--repo P] [-n N] [--json]
// kbsearch serve [port] start the embedding daemon
// kbsearch --list-model print the resolved model dir
//
// The potion-multilingual model is loaded only in `serve`; a CLI reuses the
// daemon over localhost HTTP (KBSEARCH_PORT, default 17830) and starts one in
// the background when none answers. KBSEARCH_NO_DAEMON=1 skips that and embeds
// in-process instead.
package main
import (
"fmt"
"log"
"os"
"strconv"
)
func main() {
if len(os.Args) > 1 && os.Args[1] == "serve" {
port := 17830
if len(os.Args) > 2 {
if p, err := strconv.Atoi(os.Args[2]); err == nil {
port = p
}
}
if err := serve(port); err != nil {
log.Fatalf("kbsearch serve: %v", err)
}
return
}
if len(os.Args) > 1 && os.Args[1] == "--list-model" {
dir, err := modelDir()
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
fmt.Println(dir)
return
}
os.Exit(runSearch(os.Args[1:]))
}
@@ -1,11 +1,9 @@
//go:build cgo && system_ladybug
// StaticModel wraps the potion-multilingual-128m embedding model.
//
// Mirrors model2vec.StaticModel: tokenizer (daulet Unigram) + safetensors matrix.
// Embed(text) applies the same preprocessing: median_token_length pre-truncation,
// add_special_tokens=false, drop unk (id=1), truncate to 512, mean pool, L2 normalize +1e-32.
package brain
package main
import (
"encoding/json"
@@ -1,4 +1,5 @@
package brain
// modelDir returns the resolved potion-multilingual-128m model directory.
package main
import (
"fmt"
@@ -6,9 +6,9 @@ import (
"strings"
)
const Usage = `usage: bin/brain/search.go "query" [--root facts|info] [--repo REPO] [-n N] [--json]
bin/brain/search.go serve [port]
bin/brain/search.go --list-model`
const Usage = `usage: kbsearch "query" [--root facts|info] [--repo REPO] [-n N] [--json]
kbsearch serve [port]
kbsearch --list-model`
type Options struct {
Query string
@@ -1,4 +1,4 @@
// Package rank is the cgo-free ranking and CLI parsing for brain search.
// Package rank is the cgo-free ranking and CLI parsing for kbsearch.
// CI can `go test ./rank` without the native ladybug library.
package rank
@@ -136,12 +136,6 @@ func TestListModelNeedsNoQuery(t *testing.T) {
}
}
func TestUsageNamesBrainSearch(t *testing.T) {
if !strings.Contains(Usage, "bin/brain/search.go") {
t.Fatalf("usage must name bin/brain/search.go, got:\n%s", Usage)
}
}
func TestFTSQueryOrdersByScoreDescending(t *testing.T) {
if !strings.Contains(FTSStmt, "ORDER BY score DESC") {
t.Fatalf("FTS query must order by score DESC, got:\n%s", FTSStmt)
@@ -1,6 +1,5 @@
//go:build cgo && system_ladybug
package brain
// Hybrid FTS + vector search implementation, plus daemon client/server.
package main
import (
"bytes"
@@ -19,7 +18,7 @@ import (
"time"
lbug "github.com/LadybugDB/go-ladybug"
"github.com/eSlider/2dph/internal/brain/rank"
"github.com/eSlider/2dph/bin/kbsearch/rank"
)
const defaultPort = 17830
@@ -29,7 +28,7 @@ const healthPath = "/health"
func runSearch(args []string) int {
opt, err := rank.ParseArgs(args)
if err != nil {
fmt.Fprintf(os.Stderr, "brain/search: %v\n%s\n", err, rank.Usage)
fmt.Fprintf(os.Stderr, "kbsearch: %v\n%s\n", err, rank.Usage)
return 2
}
root, repo, limit, query := opt.Root, opt.Repo, opt.Limit, opt.Query
@@ -248,7 +247,7 @@ func serve(port int) error {
})
addr := fmt.Sprintf("127.0.0.1:%d", port)
log.Printf("brain search daemon listening on %s", addr)
log.Printf("kbsearch daemon listening on %s", addr)
return http.ListenAndServe(addr, mux)
}
@@ -358,21 +357,3 @@ func ensureDaemon(port int) error {
}
return fmt.Errorf("daemon failed to start on port %d", port)
}
// Main is the bin/brain/search.go entry: search, serve, or --list-model.
func Main(args []string) int {
if len(args) > 0 && args[0] == "serve" {
port := defaultPort
if len(args) > 1 {
if p, err := strconv.Atoi(args[1]); err == nil {
port = p
}
}
if err := serve(port); err != nil {
log.Printf("brain/search serve: %v", err)
return 1
}
return 0
}
return runSearch(args)
}
@@ -1,9 +1,10 @@
package brain
// Common types and helpers for kbsearch.
package main
import (
"os"
"github.com/eSlider/2dph/internal/brain/rank"
"github.com/eSlider/2dph/bin/kbsearch/rank"
)
func eps() string { return os.Getenv("KBTEST_EPS") }
@@ -1,5 +1,5 @@
// YAML emitter ported from bin/kb/yamlout.py — preserves insertion order.
package brain
package main
import (
"fmt"
+13 -8
View File
@@ -1,22 +1,27 @@
//usr/bin/env go run -tags=brain_serve "$0" "$@"; exit
//go:build brain_serve
//usr/bin/env go run "$0" "$@"; exit
// bin/serve.go - async Go HTTP server for the 2dph brain (see bin/server).
//
// bin/serve.go — deprecated; use bin/brain/serve.go.
// KB_ROOT=/path/to/2dph ./bin/serve.go # serve the brain
// KB_SEARCH_CMD=... KB_WORKERS=4 KB_PORT=8630 ./bin/serve.go
//
// Shebang trick: the first line is a Go `//` comment; when executed, env runs
// `go run "$0"` so this file doubles as an executable script. The real code
// lives in the importable package (module path, never a relative import).
// NOTE: never run `gofmt -w` on this file - it rewrites `//usr/bin/env` to
// `// usr/...` and breaks the shebang.
package main
import (
"fmt"
"os"
"github.com/eSlider/2dph/internal/httpapi"
"github.com/eSlider/2dph/bin/server"
)
func main() {
fmt.Fprintln(os.Stderr, "bin/serve.go is deprecated; use bin/brain/serve.go")
if os.Getenv("KB_ROOT") == "" {
if env := os.Getenv("KB_ROOT"); env == "" {
if wd, err := os.Getwd(); err == nil {
os.Setenv("KB_ROOT", wd)
}
}
httpapi.Run()
server.Run()
}
@@ -2,10 +2,15 @@
//
// Async by design: every request runs on its own goroutine, and CPU-heavy
// searches are serialized through a bounded worker pool (a counting
// semaphore) so N requests can't spawn N search processes at once.
// semaphore) so N requests can't spawn N Python interpreters at once.
//
// Used by bin/brain/serve.go.
package httpapi
// Used by bin/serve.go which is a self-executing shebang script:
//
// ///usr/bin/env go run "$0" "$@"; exit
// package main
// import "github.com/eSlider/2dph/bin/server"
// func main() { server.Run() }
package server
import (
"context"
@@ -95,8 +100,8 @@ func writeRaw(w http.ResponseWriter, code int, body []byte) {
w.Write(body)
}
// brainSearcher shells out to the Go brain-search binary (not Python).
// A single search is bounded and short-lived; the worker pool keeps at most N live.
// brainSearcher shells out to bin/kb/search --json. A single python search
// is bounded and short-lived; the worker pool keeps at most N live.
type brainSearcher struct {
cmdPath string
timeout time.Duration
@@ -117,18 +122,15 @@ func (b *brainSearcher) Search(ctx context.Context, query string, limit int) ([]
return out, nil
}
func defaultSearchCmd(root string) string {
if env := os.Getenv("KB_SEARCH_CMD"); env != "" {
return env
}
return filepath.Join(root, "var", "bin", "brain-search")
}
// Run starts the HTTP server. Reads env: KB_SEARCH_CMD (default
// $KB_ROOT/var/bin/brain-search), KB_WORKERS (default 4), KB_PORT (default 8630).
// Run starts the HTTP server. Reads env: KB_SEARCH_CMD (default bin/kb/search,
// relative to the repo root given by KB_ROOT), KB_WORKERS (default 4), KB_PORT
// (default 8630).
func Run() {
root := os.Getenv("KB_ROOT")
searchPath := defaultSearchCmd(root)
searchPath := os.Getenv("KB_SEARCH_CMD")
if searchPath == "" {
searchPath = filepath.Join(root, "bin", "kb", "search")
}
workers := 4
if raw := os.Getenv("KB_WORKERS"); raw != "" {
if n, err := strconv.Atoi(raw); err == nil && n > 0 {
@@ -145,7 +147,7 @@ func Run() {
searcher := &brainSearcher{cmdPath: searchPath, timeout: 60 * time.Second}
handler := NewServer(searcher, workers)
addr := "127.0.0.1:" + strconv.Itoa(port)
log.Printf("serve: %s (workers=%d cmd=%s)", addr, workers, searchPath)
log.Printf("serve: %s (workers=%d)", addr, workers)
if err := http.ListenAndServe(addr, handler); err != nil {
log.Fatal(err)
}
@@ -1,11 +1,10 @@
package httpapi
package server
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"sync"
"sync/atomic"
"testing"
@@ -142,17 +141,6 @@ func TestSearchRejectsBadLimit(t *testing.T) {
}
}
func TestDefaultSearchCmdIsBrainNotPython(t *testing.T) {
t.Setenv("KB_SEARCH_CMD", "")
cmd := defaultSearchCmd("/repo")
if strings.Contains(strings.ToLower(cmd), "python") {
t.Fatalf("search path still python: %s", cmd)
}
if !strings.Contains(cmd, "brain") {
t.Fatalf("search path must be the Go brain binary, got %s", cmd)
}
}
func TestSearchTimeout(t *testing.T) {
fs := &fakeSearcher{delay: time.Second}
h := NewServer(fs, 1)
-36
View File
@@ -1,36 +0,0 @@
"""D14 layout: bin/{subject}/{method}.go, libs in internal/, one go.mod."""
from __future__ import annotations
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
class BinLayoutTest(unittest.TestCase):
def test_brain_search_shebang_exists(self) -> None:
p = ROOT / "bin" / "brain" / "search.go"
self.assertTrue(p.is_file(), "missing bin/brain/search.go")
first = p.read_text().splitlines()[0]
self.assertTrue(
first.startswith("//usr/bin/env go run"),
f"shebang first line, got {first!r}",
)
def test_no_nested_go_mod_under_bin(self) -> None:
nested = list((ROOT / "bin").rglob("go.mod"))
self.assertEqual(nested, [], f"nested go.mod files: {nested}")
def test_rank_lives_in_internal_brain(self) -> None:
self.assertTrue(
(ROOT / "internal" / "brain" / "rank" / "rank.go").is_file(),
"ranking must live in internal/brain/rank (cgo-free)",
)
self.assertFalse(
(ROOT / "bin" / "kbsearch").exists(),
"bin/kbsearch nested module must be gone",
)
def test_no_main_go_under_bin_brain(self) -> None:
main = ROOT / "bin" / "brain" / "main.go"
self.assertFalse(main.exists(), "bin/brain/main.go is not a method")
-48
View File
@@ -1,48 +0,0 @@
"""Published docs must match live commands (Gitea SoT, brain/search, no fake --hop)."""
from __future__ import annotations
import re
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
class PublishedDocsTest(unittest.TestCase):
def test_readme_points_issues_at_gitea(self) -> None:
text = (ROOT / "README.md").read_text()
self.assertIn(
"https://git.produktor.io/eSlider/2dph/issues",
text,
"README must point issues at Gitea",
)
def test_plan_d15_names_gitea_origin(self) -> None:
text = (ROOT / "PLAN.md").read_text()
self.assertIn("D15", text)
self.assertIn("git.produktor.io/eSlider/2dph", text)
def test_readme_primary_search_is_brain(self) -> None:
text = (ROOT / "README.md").read_text()
self.assertIn(
"bin/brain/search.go",
text,
"README deduction search must name bin/brain/search.go",
)
def test_docs_do_not_claim_hop_walks(self) -> None:
paths = [
ROOT / "README.md",
ROOT / "docs" / "design.md",
ROOT / "skills" / "kb-search" / "SKILL.md",
ROOT / "skills" / "diataxis-docs" / "SKILL.md",
]
# Command-style `--hop 1` / `--hop N` plus follow/walk = the old lie.
# Honest "not implemented" notes must not match.
lie = re.compile(r"--hop (?:N|1).*(?:follow|walk)", re.I | re.S)
for path in paths:
text = path.read_text()
self.assertIsNone(
lie.search(text),
f"{path.relative_to(ROOT)} still claims --hop walks the graph",
)
-3
View File
@@ -8,7 +8,4 @@ Brain/ops/eSlider stack. Facts need proof or they are
- [design](design.md) — schema, deduction model, sources
- [Gitea issues](https://git.produktor.io/eSlider/2dph/issues) — work board (origin)
Search: `bin/brain/search.go "query"` (HTTP: `bin/brain/serve.go`). `--hop` is
not a walk; the flag errors until File/FROM_FILE edges exist.
Published docs live here and mirror the project state.
+33
View File
@@ -0,0 +1,33 @@
# CRM association proof (oo CLI ↔ corpus)
Proven with `oo` (eslider/go-onlyoffice) against the OnlyOffice portal
(`office.produktor.io`). Portal CRM is the SSOT for company ↔ person ↔
project associations; the corpus SoT (`eslider/cv/projects/knowledge-mesh-seed.yaml`)
is the second, independent source. Facts that can be backed by both are
written to the brain under `root=facts` by `bin/facts/crm`.
## What was verified
- Logical counts (portal MySQL): 1300 contacts = 897 persons + 404 companies,
198 projects, 998 deals, 939 project↔contact links.
- Every client company linked to a project has ≥1 person underneath.
- Every person `company_id` resolves to an existing company.
- Corpus org list (9) maps 1:1 onto CRM companies:
ProProdukt SL / produktor.io, Dyvenia, Immowelt AG, WhereGroup,
Keynote SIGOS, D2S/SYSTEMS, GRID, Pack und Cup, Markets Platform.
- 78 person↔company association facts written to the brain
(`how=crm-crosscheck`, `type=association`). Recall@5 in `bin/kb/eval` = 1.0.
## Mistakes found
| # | Mistake | Fix |
|---|---------|-----|
| 1 | Duplicate legal entity `GoldenRatio.Exchange` (contact 759) vs `Golden Ratio Exchange` (763); 3 deals (211, 287, 559) were linked to 759 | `oo contacts merge 759 763` — 763 kept, 759 removed, deal links re-pointed to 763 |
| 2 | `env/`-wide: OnlyOffice creds file used wrong UX (user `eslider`, password with `$2` suffix) making `oo` auth fail | `.env` fixed to `eslider@gmail.com` + clean password; `.env` stays gitignored |
## Gates after fix
- `uv run python -m unittest discover -s bin/tools -t .` → 26 tests OK
- `bin/facts/audit self` + `bin/facts/audit db` → ok
- `bin/kb/eval` → recall@5 = 1.0
- `go test ./...` (bin/server + bin/watch) → ok
+3 -3
View File
@@ -17,14 +17,14 @@ as one consistent state.
## Deduction search
```
bin/brain/search.go "question"
bin/kb/search "question"
1. facts root — confirmed answers only → return with evidence links
2. info root — supporting narrative → snippets, marked (not confirmed)
3. web-search — second independent source → upgrade hypothesis to confirmed
```
`--hop` is not implemented yet (needs File/FROM_FILE edges). The flag is an
error; it is not a graph walk.
`--hop N` follows graph edges (sibling leaves under a heading, owning file,
`related:` files, vector-neighbour leaves) — the deduction walk.
## Who / What / How / Where / When + evidence
+1 -18
View File
@@ -1,25 +1,8 @@
module github.com/eSlider/2dph
go 1.26
go 1.25.0
require (
github.com/LadybugDB/go-ladybug v0.17.0
github.com/arran4/golang-ical v0.3.5
github.com/chewxy/math32 v1.11.2
github.com/daulet/tokenizers v1.27.0
golang.org/x/text v0.40.0
)
require (
github.com/apache/arrow-go/v18 v18.6.0 // indirect
github.com/goccy/go-json v0.10.6 // indirect
github.com/google/flatbuffers v25.12.19+incompatible // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/klauspost/compress v1.18.5 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/pierrec/lz4/v4 v4.1.26 // indirect
github.com/shopspring/decimal v1.4.0 // indirect
github.com/zeebo/xxh3 v1.1.0 // indirect
golang.org/x/exp v0.0.0-20260112195511-716be5621a96 // indirect
golang.org/x/sys v0.43.0 // indirect
)
+6 -42
View File
@@ -1,50 +1,14 @@
github.com/LadybugDB/go-ladybug v0.17.0 h1:RXDbkBjrbRmLdEbhGl4CLOIEzSt09gbP0n9UbKDEfwI=
github.com/LadybugDB/go-ladybug v0.17.0/go.mod h1:GeIXmE8XyF5TFS94NAuTag7vgCC+no/HTBMRA6Rd5Cs=
github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eTWro=
github.com/andybalholm/brotli v1.2.1/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/apache/arrow-go/v18 v18.6.0 h1:GX/Jyd3R7mCLiECAwY9FWbbaYblie2WXBSz4Sw8fNpM=
github.com/apache/arrow-go/v18 v18.6.0/go.mod h1:gm3MiPpY82fLYK5VKPB3WoJbsiLVDfT7flD5/vHReKw=
github.com/apache/thrift v0.22.0 h1:r7mTJdj51TMDe6RtcmNdQxgn9XcyfGDOzegMDRg47uc=
github.com/apache/thrift v0.22.0/go.mod h1:1e7J/O1Ae6ZQMTYdy9xa3w9k+XHWPfRvdPyJeynQ+/g=
github.com/arran4/golang-ical v0.3.5 h1:bbz6ld4dC+MmCKiFfOd6SkmIGnhNMBACZ485ULh7p9A=
github.com/arran4/golang-ical v0.3.5/go.mod h1:OnguFgjN0Hmx8jzpmWcC+AkHio94ujmLHKoaef7xQh8=
github.com/chewxy/math32 v1.11.2 h1:IufN08Zwr1NKuWfY+4Tz55BcwKmyKKNdOP7KtumehnM=
github.com/chewxy/math32 v1.11.2/go.mod h1:dOB2rcuFrCn6UHrze36WSLVPKtzPMRAQvBvUwkSsLqs=
github.com/daulet/tokenizers v1.27.0 h1:MmFYAEDFz69s/nNQfHg59DWqHz3v94m99kEZ/JbL+s4=
github.com/daulet/tokenizers v1.27.0/go.mod h1:YjFY1o1HGMyWkQgbXJDghhvke/yFDp2vGdIO2hYs4MQ=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/goccy/go-json v0.10.6 h1:p8HrPJzOakx/mn/bQtjgNjdTcN+/S6FcG2CTtQOrHVU=
github.com/goccy/go-json v0.10.6/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/google/flatbuffers v25.12.19+incompatible h1:haMV2JRRJCe1998HeW/p0X9UaMTK6SDo0ffLn2+DbLs=
github.com/google/flatbuffers v25.12.19+incompatible/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE=
github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/pierrec/lz4/v4 v4.1.26 h1:GrpZw1gZttORinvzBdXPUXATeqlJjqUG/D87TKMnhjY=
github.com/pierrec/lz4/v4 v4.1.26/go.mod h1:EoQMVJgeeEOMsCqCzqFm2O0cJvljX2nGZjcRIPL34O4=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/shopspring/decimal v1.4.0 h1:bxl37RwXBklmTi0C79JfXCEBD1cqqHt0bbgBAGFp81k=
github.com/shopspring/decimal v1.4.0/go.mod h1:gawqmDU56v4yIKSwfBSFip1HdCCXN8/+DMd9qYNcwME=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ=
github.com/zeebo/assert v1.3.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=
github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs=
github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s=
golang.org/x/exp v0.0.0-20260112195511-716be5621a96 h1:Z/6YuSHTLOHfNFdb8zVZomZr7cqNgTJvA8+Qz75D8gU=
golang.org/x/exp v0.0.0-20260112195511-716be5621a96/go.mod h1:nzimsREAkjBCIEFtHiYkrJyT+2uy9YZJB7H1k68CXZU=
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
-3
View File
@@ -1,3 +0,0 @@
go 1.26
use .
-3
View File
@@ -1,3 +0,0 @@
// Package brain is deduction search over Ladybug (FTS + HNSW).
// Query/embed code that needs cgo lives behind the system_ladybug tag.
package brain
+6 -4
View File
@@ -30,11 +30,12 @@ related:
---
```
`bin/kb/index` reads this. `type` becomes a searchable column. `related:` is
frontmatter for humans; graph hops from it are not implemented yet.
`bin/kb/index` reads this. `type` becomes a searchable column and `related`
becomes a graph edge:
```bash
bin/brain/search.go "deploy"
bin/kb/search "deploy" --type howto
bin/kb/search "Stecktafel" --hop 1 # follow links and related
```
## Audit checklist
@@ -45,7 +46,8 @@ bin/brain/search.go "deploy"
The indexer chunks on H2, so split files also search better.
3. Is `status: archive` set on anything superseded? Archived files stay indexed
but stop competing with current ones for a reader's attention.
4. Does every explanation link the reference it explains, and vice versa?
4. Does every explanation link the reference it explains, and vice versa? That
link is what `--hop 1` walks.
## Rule
+12 -13
View File
@@ -2,10 +2,9 @@
name: kb-search
description: >-
Deduction search over the 2dph brain (Ladybug graph: ops corpus, portfolio,
ssh hosts) with bin/brain/search.go instead of reading files or grepping
repos. Use whenever a question starts with "where is", "what runs on",
"which file describes", "who is", "how is X done", before opening any
documentation.
ssh hosts) with bin/kb/search instead of reading files or grepping repos.
Use whenever a question starts with "where is", "what runs on", "which file
describes", "who is", "how is X done", before opening any documentation.
---
# kb-search — deduction over facts and info
@@ -23,17 +22,15 @@ second independent source when local roots cannot confirm. An answer is
`(not confirmed)`.
```bash
bin/brain/search.go "Matrix federation" # pointers + snippets, YAML
bin/brain/search.go "onlyoffice postgres" --root facts # restrict to confirmed
bin/brain/search.go "where is cs-lexicon" --json | yq '.[].ref'
bin/kb/get <id> --body # full chunk only when needed
bin/kb/stats # index health
bin/kb/eval # recall@5 >= 0.95 gate
bin/kb/search "Matrix federation" # pointers + snippets, YAML
bin/kb/search "what runs on arc-2" --hop 1 # follow graph edges
bin/kb/search "onlyoffice postgres" --root facts # restrict to confirmed
bin/kb/search "where is cs-lexicon" --json | yq '.[].ref'
bin/kb/get <id> --body # full chunk only when needed
bin/kb/stats # index health
bin/kb/eval # recall@5 >= 0.95 gate
```
`bin/kb/search` is a deprecated wrapper. `--hop` errors (File/FROM_FILE edges
are not wired yet); do not treat it as a graph walk.
## Rules
- Search before you read. Never grep a repo for a concept the graph covers.
@@ -41,6 +38,8 @@ are not wired yet); do not treat it as a graph walk.
facts first, then info leafs clearly marked `(not confirmed)`.
- If recall looks wrong, run `bin/kb/eval`; it gates control questions and
should stay at or above 95% recall@5.
- `--hop N` follows sibling leaves, owning files, `related:` links and
vector-neighbours — that is the deduction walk, not random expansion.
- Escalate to `web-search` (the `web-search` skill) as the independent second
source when both local roots cannot confirm; never report an unconfirmed
single-source local answer as fact.
+3 -4
View File
@@ -18,10 +18,9 @@ bin/web/search "postgres partial index" --lang en --fresh year
## Web or knowledge base
`bin/brain/search.go` holds our own facts: the ops stack, portfolio, ssh hosts,
the lexicon. Go there first. Reach for `bin/web/search` when the answer is
outside our repos: upstream library behaviour, vendor documentation, public
standards.
`bin/kb/search` holds our own facts: the ops stack, portfolio, ssh hosts, the
lexicon. Go there first. Reach for `bin/web/search` when the answer is outside
our repos: upstream library behaviour, vendor documentation, public standards.
Keep the two apart. A finding is stronger when the reader can see that one
source was ours and one was not.