diff --git a/AGENTS.md b/AGENTS.md index d04c932..67ad421 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -48,7 +48,7 @@ bin/postgres/ query.go (read-only YAML) bin/git/ import.go (go-git history; Python shim execs it) bin/web/ search.go (SearXNG; Python shim execs it) bin/reasoner/ bakeoff.go (D18 CPU OpenAI tool-call bake-off) -internal/ shared Go (brain/rank is cgo-free; chats parsers; gitlog; websearch; reasoner; duckstats) +internal/ shared Go (brain/rank is cgo-free; facts D16; chats; gitlog; websearch; reasoner; duckstats) bin/qa/ stats.go (DuckDB quantiles / JSONL count; gcc CGO, not Zig) bin/watch/ corpus watcher (used by bin/brain/watch.go) bin/tools/ vendored python libs behind bin/* (kblib, yamlout, websearch) @@ -84,7 +84,7 @@ bin/brain/index.go --rebuild --with-facts --with-chats ## Tools ```bash -bin/facts/audit.go ["self"|"facts"|"info"|"stale"] # 2-source + staleness gate +bin/facts/audit.go ["self"|"db"|"contradict"] # 2-source + D16 adjudication bin/facts/crm.go [--dry-run] # proof person↔company/company↔project (ooCRM × corpus SoT) bin/kb/search "query" [--repo X] # deprecated wrapper → bin/brain/search.go bin/brain/search.go "query" [--root facts|info] # deduction search → YAML diff --git a/PLAN.md b/PLAN.md index ca62089..b8fb4a6 100644 --- a/PLAN.md +++ b/PLAN.md @@ -5,8 +5,10 @@ RAG over the operational Brain/ops/eSlider stack. Built like Sherlock Holmes: nothing is asserted unless it has proof. Status: **v1 in** (epic [#16](https://git.produktor.io/eSlider/2dph/issues/16) closed). -v2 board: milestone [v2](https://git.produktor.io/eSlider/2dph/milestone/13) — OCR [#6](https://git.produktor.io/eSlider/2dph/issues/6), -[#29](https://git.produktor.io/eSlider/2dph/issues/29) OQ1, [#30](https://git.produktor.io/eSlider/2dph/issues/30) OQ3. +v2 board: milestone [v2](https://git.produktor.io/eSlider/2dph/milestone/13) — +OCR [#6](https://git.produktor.io/eSlider/2dph/issues/6) in, +[#29](https://git.produktor.io/eSlider/2dph/issues/29) OQ1 in, +[#30](https://git.produktor.io/eSlider/2dph/issues/30) OQ3 in. Gap: [docs/roadmap.md](docs/roadmap.md). ## What @@ -42,7 +44,7 @@ detective method: **a fact needs ≥2 independent sources or it is | D13 | portfolio | start graph `(Person:eslider)-[:HAS]->(Portfolio)`, associate other natural/juristic persons later. | | D14 | tooling style | `bin/{subject}/{method}.go` shebang (e.g. `bin/brain/search.go`). Shared code in `internal/`. One root `go.mod` + `go.work`. No `bin/*/main.go`, no nested modules. | | D15 | repo | Gitea [`eSlider/2dph`](https://git.produktor.io/eSlider/2dph) is origin + [issues](https://git.produktor.io/eSlider/2dph/issues). GitHub `eSlider/2dph` is the public clone (PRs + Actions CI). No direct `main` pushes. TDD → PR → CI green → merge. | -| D16 | contradictions | ≥2 yes vs ≥2 no → unrelated sources conflict → hypothesis → `(not confirmed)`. Resolution (authority, staleness adjudication) = **v2**, tracked as open question. | +| D16 | contradictions | ≥2 yes vs ≥2 no → hypothesis → `(not confirmed)` until a rule fires. Order: **temporal_freshness** (fresh ≥2 vs stale minority), then **authority_pairing** (runtime/config A×B beats narrative C). Store as `a x b vs c x d` on hypothesis leafs. `bin/facts/audit contradict`. [#29](https://git.produktor.io/eSlider/2dph/issues/29). | | D17 | assertion gate | Fact-check every *claim* (facts → info → live → web), not every edit. `bin/brain/search.go` adds a `web` block when there is no facts hit (`throttled`/`skipped`/`refused` ≠ absence). `--root` and `--no-web` stay local. Missing graph ≠ “does not exist”. | | D18 | reasoner | Pluggable OpenAI-compatible URL (`REASONER_BASE_URL`). RAM: `Qwen/Qwen3.5-9B`. Quality: `prism-ml/Bonsai-27B-gguf` or `Qwen/Qwen3.6-27B`. No official Qwen3.6-9B. CPU bake-off: `bin/reasoner/bakeoff.go` + compose profile `reasoner` (`OLLAMA_NUM_GPU=0`, `:11435`). PicoClaw is compose profile `picoclaw`; tools are `search`/`get`/`audit`. Weights are not copied into the 2dph image. Agent lever/loop: [#15](https://git.produktor.io/eSlider/2dph/issues/15). | | D19 | git history | [go-git](https://github.com/go-git/go-git) via `bin/git/import.go`. No subprocess of the git binary. Conversion prints commit leafs; brain write is `bin/brain/index.go`. | @@ -117,8 +119,8 @@ Common props on every node/edge: `root`, `confidence`, `evidence[]`, `how`, ## Open questions (v2) -- OQ1: mutually-contradicting evidence — how to resolve (authority weighting, - temporal freshness, audit adjudication). **v2**; [#29](https://git.produktor.io/eSlider/2dph/issues/29). +- OQ1: **in** — D16 adjudication: `temporal_freshness` then `authority_pairing`. + Unresolved 2v2 stays hypothesis. [#29](https://git.produktor.io/eSlider/2dph/issues/29). - OQ2: OCR — **in**. `pdftotext -layout` first; scans `pdftoppm` + tesseract `eng+deu` (`bin/mail/ocr.go`, `internal/ocr`). No gocv, no gosseract CGO (D21 Zig owns Ladybug CGO). Optional `OCR_ENGINE=paddle` / compose profile @@ -184,4 +186,4 @@ Narrative: [docs/roadmap.md](docs/roadmap.md). | 4 | [#15](https://git.produktor.io/eSlider/2dph/issues/15) | **in** — lever/loop documented (`search` → `get` → `audit`). | | 5 | [#19](https://git.produktor.io/eSlider/2dph/issues/19) | **in** — CI recall SoT is `bin/brain/eval.go` via Zig. Python `bin/kb/eval` stays as an explicit fallback. | -Does **not** block epic close: OQ1 [#29](https://git.produktor.io/eSlider/2dph/issues/29), OQ3 [#30](https://git.produktor.io/eSlider/2dph/issues/30), OQ4. OCR [#6](https://git.produktor.io/eSlider/2dph/issues/6) is **in**. \ No newline at end of file +Does **not** block epic close: OQ4. OCR [#6](https://git.produktor.io/eSlider/2dph/issues/6), OQ1 [#29](https://git.produktor.io/eSlider/2dph/issues/29), OQ3 [#30](https://git.produktor.io/eSlider/2dph/issues/30) are **in**. \ No newline at end of file diff --git a/bin/facts/audit b/bin/facts/audit index a377bfa..0a04d55 100755 --- a/bin/facts/audit +++ b/bin/facts/audit @@ -1,14 +1,15 @@ #!/usr/bin/env python3 """facts/audit - evidence & lexicon checks for the 2dph brain. - bin/facts/audit self # lexicon: every fact in db has >=2 sources - bin/facts/audit db # evidence gate: run against var/kb.lbug + bin/facts/audit self # lexicon: docs + two-source rule + bin/facts/audit db # evidence gate against var/kb.lbug + bin/facts/audit contradict # D16 adjudication (JSON claim(s) on stdin) -`self` mode checks the repo itself (no network, no runtime deps). It greps -for known-good two-source pairings and confirms the docs are consistent. -`db` mode loads every Leaf with root=facts and asserts each has source_rev -and a non-empty `loc` (the "where did you see it" evidence pointer) and that -'confirmed' facts carry a two-source `source` field. +`self` mode checks the repo itself (no network, no runtime deps). +`db` mode loads every Leaf with root=facts. Confirmed facts need ` x `; +hypothesis contradictions need `a x b vs c x d` (both sides ≥2). +`contradict` applies temporal_freshness then authority_pairing; ≥2 vs ≥2 +with no rule stays hypothesis / `(not confirmed)`. Exit 0 = all checks pass, 1 = audit failures, 2 = could not evaluate. """ @@ -22,6 +23,8 @@ from pathlib import Path ROOT = Path(__file__).resolve().parents[2] sys.path.insert(0, str(ROOT / "bin" / "tools")) +from contradict import adjudicate, check_fact_row # noqa: E402 + def audit_db() -> list[str]: from kblib import connect @@ -33,14 +36,8 @@ def audit_db() -> list[str]: r = conn.execute("MATCH (l:Leaf {root:'facts'}) RETURN l.id, l.source, l.loc, l.how, l.confidence") problems: list[str] = [] for lid, source, loc, how, conf in r.get_all(): - if conf != "confirmed": - problems.append(f"{lid}: facts require confidence='confirmed', got '{conf}'") - if not source or " x " not in source: - problems.append(f"{lid}: needs 2-source evidence in source, got '{source}'") - if not loc: - problems.append(f"{lid}: missing loc (evidence pointer)") - if not how: - problems.append(f"{lid}: missing how") + problems.extend(check_fact_row(str(lid), str(source or ""), str(loc or ""), + str(how or ""), str(conf or ""))) conn.close() db.close() return problems @@ -54,20 +51,50 @@ def audit_self() -> list[str]: problems.append("PLAN.md missing recall@5 gate") if re.search(r"(?i)facts must have.*2 sources|2.source", plan) is None: problems.append("PLAN.md missing the two-source evidence rule for facts") + if "temporal_freshness" not in plan or "authority_pairing" not in plan: + problems.append("PLAN.md missing D16 adjudication rules") if re.search(r"(?i)HNSW|BM25|deduction", (ROOT / "README.md").read_text()) is None: problems.append("README.md missing search/retrieval description") return problems +def audit_contradict(raw: str) -> tuple[list[str], list[dict]]: + raw = raw.strip() + if not raw: + return ["contradict: empty stdin (JSON claim or {claims:[...]})"], [] + try: + payload = json.loads(raw) + except json.JSONDecodeError as e: + return [f"contradict: invalid JSON: {e}"], [] + if isinstance(payload, dict) and "claims" in payload: + claims = list(payload.get("claims") or []) + elif isinstance(payload, dict): + claims = [payload] + elif isinstance(payload, list): + claims = payload + else: + return ["contradict: expected object or list"], [] + details = [adjudicate(c) for c in claims] + return [], details + + def main(argv: list[str]) -> int: import argparse p = argparse.ArgumentParser(description="evidence & lexicon audit") - p.add_argument("mode", choices=("self", "db")) + p.add_argument("mode", choices=("self", "db", "contradict")) p.add_argument("--json", action="store_true") a = p.parse_args(argv) - problems = audit_self() if a.mode == "self" else audit_db() - out = {"mode": a.mode, "ok": not problems, "problems": problems} + details: list[dict] = [] + if a.mode == "self": + problems = audit_self() + elif a.mode == "db": + problems = audit_db() + else: + problems, details = audit_contradict(sys.stdin.read()) + out: dict = {"mode": a.mode, "ok": not problems, "problems": problems} + if details: + out["contradictions"] = details if a.json: print(json.dumps(out, indent=2)) else: @@ -77,4 +104,4 @@ def main(argv: list[str]) -> int: if __name__ == "__main__": - sys.exit(main(sys.argv[1:])) \ No newline at end of file + sys.exit(main(sys.argv[1:])) diff --git a/bin/facts/audit.go b/bin/facts/audit.go index e36756f..7ee23ab 100755 --- a/bin/facts/audit.go +++ b/bin/facts/audit.go @@ -5,6 +5,7 @@ // // ./bin/facts/audit.go self // ./bin/facts/audit.go db +// ./bin/facts/audit.go contradict --json < claim.json // // Python bin/facts/audit is the implementation (CI runs it directly). // NOTE: never run `gofmt -w` on this file — it breaks the shebang. diff --git a/bin/tools/contradict.py b/bin/tools/contradict.py new file mode 100644 index 0000000..2132d3a --- /dev/null +++ b/bin/tools/contradict.py @@ -0,0 +1,103 @@ +"""D16 contradiction adjudication (same rules as internal/facts).""" +from __future__ import annotations + +from typing import Any + +CONF_CONFIRMED = "confirmed" +CONF_HYPOTHESIS = "hypothesis" + +RULE_UNRESOLVED = "unresolved" +RULE_TEMPORAL = "temporal_freshness" +RULE_AUTHORITY = "authority_pairing" +RULE_TWO_SOURCE = "two_source" +RULE_SINGLE = "single_source" + +KIND_RUNTIME = "runtime" +KIND_CONFIG = "config" +KIND_NARRATIVE = "narrative" + + +def _independent(sources: list[dict]) -> int: + seen: set[str] = set() + for i, s in enumerate(sources): + sid = str(s.get("id") or "") or f"{s.get('kind', '')}#{i}" + seen.add(sid) + return len(seen) + + +def _fresh_n(sources: list[dict]) -> int: + return sum(1 for s in sources if not s.get("stale")) + + +def _strong_n(sources: list[dict]) -> int: + return sum(1 for s in sources if s.get("kind") in (KIND_RUNTIME, KIND_CONFIG)) + + +def adjudicate(claim: dict[str, Any]) -> dict[str, Any]: + yes = list(claim.get("yes") or []) + no = list(claim.get("no") or []) + yes_n, no_n = _independent(yes), _independent(no) + text = str(claim.get("text") or "") + + def out(conf: str, rule: str, winner: str = "") -> dict[str, Any]: + return { + "text": text, + "confidence": conf, + "confirmed": conf == CONF_CONFIRMED, + "rule": rule, + "winner": winner, + "yes": yes_n, + "no": no_n, + } + + if yes_n < 2 or no_n < 2: + if yes_n >= 2: + return out(CONF_CONFIRMED, RULE_TWO_SOURCE, "yes") + if no_n >= 2: + return out(CONF_CONFIRMED, RULE_TWO_SOURCE, "no") + return out(CONF_HYPOTHESIS, RULE_SINGLE) + yf, nf = _fresh_n(yes), _fresh_n(no) + if yf >= 2 and nf < 2: + return out(CONF_CONFIRMED, RULE_TEMPORAL, "yes") + if nf >= 2 and yf < 2: + return out(CONF_CONFIRMED, RULE_TEMPORAL, "no") + ys, ns = _strong_n(yes), _strong_n(no) + if ys >= 2 and ns < 2: + return out(CONF_CONFIRMED, RULE_AUTHORITY, "yes") + if ns >= 2 and ys < 2: + return out(CONF_CONFIRMED, RULE_AUTHORITY, "no") + return out(CONF_HYPOTHESIS, RULE_UNRESOLVED) + + +def parse_source_field(source: str) -> tuple[str, str]: + """Split `a x b vs c x d` into (yes, no). Empty no if no ` vs `.""" + if " vs " not in source: + return source, "" + yes, _, no = source.partition(" vs ") + return yes.strip(), no.strip() + + +def check_fact_row(lid: str, source: str, loc: str, how: str, conf: str) -> list[str]: + """Lexicon checks for one facts leaf (no Ladybug).""" + problems: list[str] = [] + src = source or "" + if conf == CONF_CONFIRMED: + if " vs " in src: + problems.append(f"{lid}: confirmed fact cannot keep a vs-contradiction") + if " x " not in src: + problems.append(f"{lid}: needs 2-source evidence in source, got '{source}'") + elif conf == CONF_HYPOTHESIS: + yes, no = parse_source_field(src) + if not no or " x " not in yes or " x " not in no: + problems.append( + f"{lid}: hypothesis contradiction needs 'a x b vs c x d', got '{source}'" + ) + elif conf == "partial": + pass + else: + problems.append(f"{lid}: unknown confidence '{conf}'") + if not loc: + problems.append(f"{lid}: missing loc (evidence pointer)") + if not how: + problems.append(f"{lid}: missing how") + return problems diff --git a/bin/tools/test_bin_layout.py b/bin/tools/test_bin_layout.py index 07b7a51..169d667 100644 --- a/bin/tools/test_bin_layout.py +++ b/bin/tools/test_bin_layout.py @@ -127,6 +127,22 @@ class BinLayoutTest(unittest.TestCase): self.assertIn("cmdbin.ExecFile", text) self.assertIn(f"bin/facts/{method.removesuffix('.go')}", text) + def test_d16_adjudication_is_cgo_free(self) -> None: + self.assertTrue((ROOT / "internal" / "facts" / "contradict.go").is_file()) + go = (ROOT / "internal" / "facts" / "contradict.go").read_text() + py = (ROOT / "bin" / "tools" / "contradict.py").read_text() + audit = (ROOT / "bin" / "facts" / "audit").read_text() + for token in ("temporal_freshness", "authority_pairing", "unresolved"): + self.assertIn(token, go) + self.assertIn(token, py) + self.assertIn("contradict", audit) + self.assertIn(" vs ", py) + plan = (ROOT / "PLAN.md").read_text() + self.assertIn("temporal_freshness", plan) + self.assertIn("authority_pairing", plan) + shebang = (ROOT / "bin" / "facts" / "audit.go").read_text() + self.assertIn("contradict", shebang) + def test_mail_import_is_shebang_not_brain_write(self) -> None: self._assert_shebang("bin/mail/import.go") index_mail = (ROOT / "bin" / "mail" / "index_mail").read_text() diff --git a/bin/tools/test_contradict.py b/bin/tools/test_contradict.py new file mode 100644 index 0000000..68d1856 --- /dev/null +++ b/bin/tools/test_contradict.py @@ -0,0 +1,104 @@ +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(__file__)) + +from contradict import ( # noqa: E402 + RULE_AUTHORITY, + RULE_SINGLE, + RULE_TEMPORAL, + RULE_TWO_SOURCE, + RULE_UNRESOLVED, + adjudicate, + check_fact_row, + parse_source_field, +) + + +def src(i, kind, stale=False): + return {"id": i, "kind": kind, "stale": stale} + + +class TestContradict(unittest.TestCase): + def test_two_vs_two_stays_hypothesis(self): + r = adjudicate({ + "text": "svc listens on 443", + "yes": [src("docker-ps", "runtime"), src("compose", "config")], + "no": [src("docker-old", "runtime"), src("compose-old", "config")], + }) + self.assertFalse(r["confirmed"]) + self.assertEqual(r["rule"], RULE_UNRESOLVED) + self.assertEqual(r["winner"], "") + + def test_temporal_freshness(self): + r = adjudicate({ + "text": "svc listens on 443", + "yes": [src("docker-ps", "runtime"), src("compose", "config")], + "no": [src("old-readme", "narrative", True), src("old-wiki", "narrative", True)], + }) + self.assertTrue(r["confirmed"]) + self.assertEqual(r["rule"], RULE_TEMPORAL) + self.assertEqual(r["winner"], "yes") + + def test_authority_pairing(self): + r = adjudicate({ + "text": "svc listens on 443", + "yes": [src("docker-ps", "runtime"), src("compose", "config")], + "no": [src("readme", "narrative"), src("wiki", "narrative")], + }) + self.assertTrue(r["confirmed"]) + self.assertEqual(r["rule"], RULE_AUTHORITY) + self.assertEqual(r["winner"], "yes") + + def test_two_source_and_single(self): + two = adjudicate({ + "text": "arc-1 runs Matrix", + "yes": [src("compose", "config"), src("docker-ps", "runtime")], + }) + self.assertTrue(two["confirmed"]) + self.assertEqual(two["rule"], RULE_TWO_SOURCE) + one = adjudicate({"text": "maybe", "yes": [src("readme", "narrative")]}) + self.assertFalse(one["confirmed"]) + self.assertEqual(one["rule"], RULE_SINGLE) + + def test_parse_source_field(self): + yes, no = parse_source_field("docker ps x compose.yml vs old.md x wiki.md") + self.assertIn(" x ", yes) + self.assertIn(" x ", no) + + def test_check_fact_row_allows_hypothesis_vs(self): + p = check_fact_row( + "L1", "a.md x b.md vs c.md x d.md", "var/", "audit", "hypothesis", + ) + self.assertEqual(p, []) + p = check_fact_row("L2", "a.md x b.md", "var/", "audit", "confirmed") + self.assertEqual(p, []) + p = check_fact_row("L3", "a.md x b.md vs c.md x d.md", "var/", "audit", "confirmed") + self.assertTrue(any("vs-contradiction" in x for x in p)) + p = check_fact_row("L4", "only-one.md", "var/", "audit", "hypothesis") + self.assertTrue(any("a x b vs" in x for x in p)) + + def test_audit_contradict_cli_unresolved(self): + import json + import subprocess + from pathlib import Path + root = Path(__file__).resolve().parents[2] + payload = json.dumps({ + "text": "svc 443", + "yes": [src("a", "runtime"), src("b", "config")], + "no": [src("c", "runtime"), src("d", "config")], + }) + proc = subprocess.run( + [sys.executable, str(root / "bin" / "facts" / "audit"), "contradict", "--json"], + input=payload, capture_output=True, text=True, check=False, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + out = json.loads(proc.stdout) + self.assertTrue(out["ok"]) + self.assertEqual(out["contradictions"][0]["rule"], RULE_UNRESOLVED) + self.assertFalse(out["contradictions"][0]["confirmed"]) + + +if __name__ == "__main__": + unittest.main() diff --git a/docs/design.md b/docs/design.md index 3f15a14..27e223d 100644 --- a/docs/design.md +++ b/docs/design.md @@ -71,7 +71,9 @@ corpus HEAD. - C: narrative — READMEs, AGENTS.md, docs Confirmed = A×B or B×C agreement. Single source = hypothesis + `(not confirmed)`. -Conflicting pairings (≥2 yes vs ≥2 no) = hypothesis (OQ1 → v2 resolution). +Conflicting pairings (≥2 yes vs ≥2 no) stay hypothesis until +`temporal_freshness` or `authority_pairing` fires (`bin/facts/audit contradict`, +[#29](https://git.produktor.io/eSlider/2dph/issues/29)). ## Read path diff --git a/docs/roadmap.md b/docs/roadmap.md index b899705..3e79fe0 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -39,11 +39,11 @@ Epic [#16](https://git.produktor.io/eSlider/2dph/issues/16) closed. [#6](https://git.produktor.io/eSlider/2dph/issues/6) OCR — **in**. [#30](https://git.produktor.io/eSlider/2dph/issues/30) OQ3 duckdb-go — **in**. [#29](https://git.produktor.io/eSlider/2dph/issues/29) OQ1 contradiction -resolution. +resolution — **in** (`temporal_freshness`, `authority_pairing`). ## Blockers -None for epic #16 (closed). Remaining v2: OQ1, OQ4. +None for epic #16 (closed). Remaining v2: OQ4. ``` question @@ -58,8 +58,8 @@ question ## Not v1 -OQ1 contradiction resolution, OQ4 YAML-first leafs. -OCR (OQ2) and duckdb-go (OQ3/D22) are in. +OQ4 YAML-first leafs. +OCR (OQ2), duckdb-go (OQ3/D22), and D16 adjudication (OQ1) are in. ## Close epic #16 when diff --git a/internal/brain/rank/escalate.go b/internal/brain/rank/escalate.go index 62b5b11..d3e6e04 100644 --- a/internal/brain/rank/escalate.go +++ b/internal/brain/rank/escalate.go @@ -19,20 +19,35 @@ type SecondSourceHit struct { type WebFn func(query string) SecondSource -// ShouldEscalate is true when the default deduction path has no facts hit. +// ShouldEscalate is true when the default deduction path has no confirmed +// facts hit. Hypothesis/partial facts are `(not confirmed)` (D16). // `--root facts|info` is a single-root ask: do not mix in the web. func ShouldEscalate(hits []Hit, rootFilter string) bool { if rootFilter != "" { return false } for _, h := range hits { - if h.Root == "facts" { + if ConfirmedFact(h) { return false } } return true } +// ConfirmedFact is a facts-root hit that is not hypothesis/partial. +// Empty confidence is treated as confirmed (legacy leafs). +func ConfirmedFact(h Hit) bool { + if h.Root != "facts" { + return false + } + switch h.Confidence { + case "hypothesis", "partial": + return false + default: + return true + } +} + // Deduce returns the second-source block, or nil when web must not run. func Deduce(hits []Hit, query, rootFilter string, noWeb bool, web WebFn) *SecondSource { if noWeb || web == nil || !ShouldEscalate(hits, rootFilter) { diff --git a/internal/brain/rank/escalate_test.go b/internal/brain/rank/escalate_test.go index c5dd168..c1e3ae8 100644 --- a/internal/brain/rank/escalate_test.go +++ b/internal/brain/rank/escalate_test.go @@ -14,6 +14,16 @@ func TestShouldEscalateWhenNoFacts(t *testing.T) { } } +func TestShouldEscalateWhenHypothesisFacts(t *testing.T) { + hyp := Hit{ID: "c", Root: "facts", Confidence: "hypothesis", Source: "a x b vs c x d"} + if !ShouldEscalate([]Hit{hyp}, "") { + t.Fatal("hypothesis facts are (not confirmed); escalate") + } + if ConfirmedFact(hyp) { + t.Fatal("hypothesis is not confirmed") + } +} + func TestShouldNotEscalateWhenFactsConfirm(t *testing.T) { hits := []Hit{h("f", "facts", "docker ps x compose"), h("i", "info", "docs/a.md")} if ShouldEscalate(hits, "") { diff --git a/internal/brain/rank/query.go b/internal/brain/rank/query.go index ff3d023..26f30a4 100644 --- a/internal/brain/rank/query.go +++ b/internal/brain/rank/query.go @@ -3,10 +3,10 @@ package rank // BM25 ranks best-first, so the top hits are the *highest* scores; cosine // distance ranks best-first ascending. Both mirror kblib.py. const FTSStmt = "CALL QUERY_FTS_INDEX('Leaf', 'id', $q) " + - "RETURN node.id, node.text, node.root, node.source, score ORDER BY score DESC LIMIT $n" + "RETURN node.id, node.text, node.root, node.source, score, node.confidence ORDER BY score DESC LIMIT $n" const VecStmt = "CALL QUERY_VECTOR_INDEX('Leaf', 'Leaf_vec', $q, $n) " + - "RETURN node.id, node.text, node.root, node.source, distance ORDER BY distance LIMIT $n" + "RETURN node.id, node.text, node.root, node.source, distance, node.confidence ORDER BY distance LIMIT $n" // HopStmt is the Cypher walk from a search hit. Depth 1 = File, 2 = Commit, 3 = Person. func HopStmt(depth int) string { diff --git a/internal/brain/rank/rank.go b/internal/brain/rank/rank.go index 06f6f50..6aed929 100644 --- a/internal/brain/rank/rank.go +++ b/internal/brain/rank/rank.go @@ -16,13 +16,14 @@ type HopNode struct { // Hit is one search result, mirroring the python script's dict shape. type Hit struct { - ID string `json:"id"` - Text string `json:"text"` - Root string `json:"root"` - Source string `json:"-"` - Score float64 `json:"score"` - Snippet string `json:"snippet,omitempty"` - Hops []HopNode `json:"hops,omitempty"` + ID string `json:"id"` + Text string `json:"text"` + Root string `json:"root"` + Confidence string `json:"confidence,omitempty"` + Source string `json:"-"` + Score float64 `json:"score"` + Snippet string `json:"snippet,omitempty"` + Hops []HopNode `json:"hops,omitempty"` } // rrfK dampens the contribution of low ranks; same constant as kblib.py. diff --git a/internal/brain/rank/rank_test.go b/internal/brain/rank/rank_test.go index 6d07e5c..9c3c181 100644 --- a/internal/brain/rank/rank_test.go +++ b/internal/brain/rank/rank_test.go @@ -172,4 +172,7 @@ func TestFTSQueryOrdersByScoreDescending(t *testing.T) { if !strings.Contains(FTSStmt, "ORDER BY score DESC") { t.Fatalf("FTS query must order by score DESC, got:\n%s", FTSStmt) } + if !strings.Contains(FTSStmt, "node.confidence") { + t.Fatal("FTS must return confidence for D16") + } } diff --git a/internal/brain/search.go b/internal/brain/search.go index 263ab5a..d275c6e 100644 --- a/internal/brain/search.go +++ b/internal/brain/search.go @@ -212,7 +212,11 @@ func rowsToHits(res *lbug.QueryResult) ([]Hit, error) { root := fmt.Sprint(vals[2]) source := fmt.Sprint(vals[3]) score := float64(vals[4].(float64)) - hits = append(hits, Hit{ID: id, Text: text, Root: root, Source: source, Score: score}) + conf := "" + if len(vals) >= 6 { + conf = fmt.Sprint(vals[5]) + } + hits = append(hits, Hit{ID: id, Text: text, Root: root, Source: source, Score: score, Confidence: conf}) } return hits, nil } @@ -227,24 +231,26 @@ type jsonOut struct { } type jsonHit struct { - ID string `json:"id"` - Text string `json:"text"` - Root string `json:"root"` - Score float64 `json:"score"` - Snippet string `json:"snippet,omitempty"` - Hops []rank.HopNode `json:"hops,omitempty"` + ID string `json:"id"` + Text string `json:"text"` + Root string `json:"root"` + Confidence string `json:"confidence,omitempty"` + Score float64 `json:"score"` + Snippet string `json:"snippet,omitempty"` + Hops []rank.HopNode `json:"hops,omitempty"` } func toJSONOut(hits []Hit, query, rootFilter string, web *rank.SecondSource) *jsonOut { out := make([]jsonHit, len(hits)) for i, h := range hits { out[i] = jsonHit{ - ID: h.ID, - Text: h.Text, - Root: h.Root, - Score: h.Score, - Snippet: h.Snippet, - Hops: h.Hops, + ID: h.ID, + Text: h.Text, + Root: h.Root, + Confidence: h.Confidence, + Score: h.Score, + Snippet: h.Snippet, + Hops: h.Hops, } } return &jsonOut{ @@ -265,6 +271,9 @@ func resultsToDicts(hits []Hit) []any { {"root", h.Root}, {"score", h.Score}, } + if h.Confidence != "" { + d = append(d, KV{"confidence", h.Confidence}) + } if h.Snippet != "" { d = append(d, KV{"snippet", h.Snippet}) } diff --git a/internal/facts/contradict.go b/internal/facts/contradict.go new file mode 100644 index 0000000..54eb6c3 --- /dev/null +++ b/internal/facts/contradict.go @@ -0,0 +1,120 @@ +// Package facts is cgo-free evidence rules (D16 contradictions). +package facts + +import "strconv" + +const ( + ConfConfirmed = "confirmed" + ConfHypothesis = "hypothesis" + + RuleUnresolved = "unresolved" + RuleTemporalFreshness = "temporal_freshness" + RuleAuthorityPairing = "authority_pairing" + RuleTwoSource = "two_source" + RuleSingleSource = "single_source" + + KindRuntime = "runtime" + KindConfig = "config" + KindNarrative = "narrative" +) + +// Source is one independent pointer on a yes or no side. +type Source struct { + ID string `json:"id"` + Kind string `json:"kind"` + When string `json:"when,omitempty"` + Stale bool `json:"stale,omitempty"` +} + +// Claim is one assertion with yes/no evidence lists. +type Claim struct { + Text string `json:"text"` + Yes []Source `json:"yes"` + No []Source `json:"no"` +} + +// Result is audit output. Confirmed=false means `(not confirmed)`. +type Result struct { + Text string `json:"text"` + Confidence string `json:"confidence"` + Confirmed bool `json:"confirmed"` + Rule string `json:"rule"` + Winner string `json:"winner,omitempty"` + YesN int `json:"yes"` + NoN int `json:"no"` +} + +func independent(ss []Source) int { + seen := map[string]struct{}{} + for i, s := range ss { + id := s.ID + if id == "" { + id = s.Kind + "#" + strconv.Itoa(i) + } + seen[id] = struct{}{} + } + return len(seen) +} + +func freshN(ss []Source) int { + n := 0 + for _, s := range ss { + if !s.Stale { + n++ + } + } + return n +} + +func strongN(ss []Source) int { + n := 0 + for _, s := range ss { + if s.Kind == KindRuntime || s.Kind == KindConfig { + n++ + } + } + return n +} + +func out(c Claim, conf, rule, winner string) Result { + return Result{ + Text: c.Text, + Confidence: conf, + Confirmed: conf == ConfConfirmed, + Rule: rule, + Winner: winner, + YesN: independent(c.Yes), + NoN: independent(c.No), + } +} + +// Adjudicate applies D16: ≥2 yes vs ≥2 no stays hypothesis until a rule fires. +// Order: temporal_freshness, then authority_pairing (A/B beats narrative C). +func Adjudicate(c Claim) Result { + yesN := independent(c.Yes) + noN := independent(c.No) + if yesN < 2 || noN < 2 { + if yesN >= 2 { + return out(c, ConfConfirmed, RuleTwoSource, "yes") + } + if noN >= 2 { + return out(c, ConfConfirmed, RuleTwoSource, "no") + } + return out(c, ConfHypothesis, RuleSingleSource, "") + } + yf, nf := freshN(c.Yes), freshN(c.No) + if yf >= 2 && nf < 2 { + return out(c, ConfConfirmed, RuleTemporalFreshness, "yes") + } + if nf >= 2 && yf < 2 { + return out(c, ConfConfirmed, RuleTemporalFreshness, "no") + } + ys, ns := strongN(c.Yes), strongN(c.No) + if ys >= 2 && ns < 2 { + return out(c, ConfConfirmed, RuleAuthorityPairing, "yes") + } + if ns >= 2 && ys < 2 { + return out(c, ConfConfirmed, RuleAuthorityPairing, "no") + } + return out(c, ConfHypothesis, RuleUnresolved, "") +} diff --git a/internal/facts/contradict_test.go b/internal/facts/contradict_test.go new file mode 100644 index 0000000..702b6da --- /dev/null +++ b/internal/facts/contradict_test.go @@ -0,0 +1,86 @@ +package facts + +import "testing" + +func src(id, kind string, stale bool) Source { + return Source{ID: id, Kind: kind, Stale: stale} +} + +func TestTwoVsTwoStaysHypothesis(t *testing.T) { + c := Claim{ + Text: "svc listens on 443", + Yes: []Source{ + src("docker-ps", KindRuntime, false), + src("compose", KindConfig, false), + }, + No: []Source{ + src("docker-ps-old", KindRuntime, false), + src("compose-old", KindConfig, false), + }, + } + r := Adjudicate(c) + if r.Confirmed || r.Confidence != ConfHypothesis || r.Rule != RuleUnresolved { + t.Fatalf("2v2 must stay (not confirmed): %+v", r) + } + if r.Winner != "" { + t.Fatalf("unresolved must not name a winner: %+v", r) + } +} + +func TestTemporalFreshnessResolvesStaleSide(t *testing.T) { + c := Claim{ + Text: "svc listens on 443", + Yes: []Source{ + src("docker-ps", KindRuntime, false), + src("compose", KindConfig, false), + }, + No: []Source{ + src("old-readme", KindNarrative, true), + src("old-wiki", KindNarrative, true), + }, + } + r := Adjudicate(c) + if !r.Confirmed || r.Rule != RuleTemporalFreshness || r.Winner != "yes" { + t.Fatalf("fresh yes vs stale no: %+v", r) + } +} + +func TestAuthorityPairingBeatsNarrative(t *testing.T) { + c := Claim{ + Text: "svc listens on 443", + Yes: []Source{ + src("docker-ps", KindRuntime, false), + src("compose", KindConfig, false), + }, + No: []Source{ + src("readme", KindNarrative, false), + src("wiki", KindNarrative, false), + }, + } + r := Adjudicate(c) + if !r.Confirmed || r.Rule != RuleAuthorityPairing || r.Winner != "yes" { + t.Fatalf("A×B vs C×C: %+v", r) + } +} + +func TestTwoSourceYesIsConfirmed(t *testing.T) { + c := Claim{ + Text: "arc-1 runs Matrix", + Yes: []Source{ + src("compose", KindConfig, false), + src("docker-ps", KindRuntime, false), + }, + } + r := Adjudicate(c) + if !r.Confirmed || r.Rule != RuleTwoSource || r.Winner != "yes" { + t.Fatalf("%+v", r) + } +} + +func TestSingleSourceIsHypothesis(t *testing.T) { + c := Claim{Text: "maybe", Yes: []Source{src("readme", KindNarrative, false)}} + r := Adjudicate(c) + if r.Confirmed || r.Rule != RuleSingleSource { + t.Fatalf("%+v", r) + } +} diff --git a/skills/brain/SKILL.md b/skills/brain/SKILL.md index 8476c10..d6c2eb6 100644 --- a/skills/brain/SKILL.md +++ b/skills/brain/SKILL.md @@ -45,6 +45,8 @@ bin/brain/eval.go # recall@5 >= 0.95 gate ( are not evidence of absence. `--root facts|info` and `--no-web` skip the web. - If recall looks wrong, run `bin/brain/eval.go`; it gates control questions and should stay at or above 95% recall@5. +- Contradictions (≥2 yes vs ≥2 no) stay `(not confirmed)` until + `bin/facts/audit contradict` fires `temporal_freshness` or `authority_pairing`. - Agents: `GET /openapi.json` and `POST /mcp` on `bin/brain/serve.go` (same handlers; tool names match paths `search`/`get`/`stats`/`audit`). Generated list: [tools.md](tools.md).