feat: add CI documentation rules checker and docs-verify job

Enforce link resolution, glossary anchors, and readability rules on every
push and PR; docs-only pushes to main still run docs-verify while skipping
the simulator gate.

.github/workflows/release.yaml

@@ -1,4 +1,5 @@
 # Swarm House - Gitea Actions
+# docs-verify: markdown links, glossary anchors, readability rules (every push/PR)
 # verify: compile check, virtual-drone smoke flight, SQL gate tests (every push/PR)
 # scan:   Trivy vulnerability + misconfiguration scan
 # release: semantic version bump (conventional commits), tag, Gitea release
@@ -17,6 +18,24 @@ on:
     branches: [main]
 
 jobs:
+  docs-verify:
+    name: Verify documentation
+    runs-on: ubuntu-latest
+    container: python:3.12-bookworm
+    steps:
+      - name: Checkout
+        run: |
+          git init .
+          git remote add origin https://${{ gitea.actor }}:${{ gitea.token }}@git.produktor.io/${{ gitea.repository }}.git
+          git fetch --depth=50 origin "${{ gitea.ref }}"
+          git checkout FETCH_HEAD
+
+      - name: Install checker dependencies
+        run: pip install --quiet pyyaml
+
+      - name: Documentation rules (links, glossary, readability)
+        run: python scripts/check_docs.py
+
   verify:
     name: Verify simulator
     runs-on: ubuntu-latest
@@ -85,7 +104,7 @@ jobs:
     name: Semantic Release
     runs-on: ubuntu-latest
     container: python:3.12-bookworm
-    needs: [verify, scan]
+    needs: [docs-verify, verify, scan]
     if: gitea.event_name == 'push'
     permissions:
       contents: write

CONTRIBUTING.md

@@ -17,6 +17,7 @@ The CI pipeline enforces this:
 
 | Gate | When it runs |
 | --- | --- |
+| `python scripts/check_docs.py` | Every push/PR (`docs-verify` job) — links, glossary anchors, readability rules ([ci-rules](docs/improvement/ci-rules.md)) |
 | `pytest tests/` | Every push/PR (`verify` job) |
 | `RUN pytest tests/` in the simulator Docker build | Every `docker build` — a red test blocks the image |
 | Smoke flight + DuckDB assertion | Every push/PR |

docs/improvement/2026-07-10-audit.md

@@ -16,7 +16,7 @@ Manual pass over `docs/`, all READMEs, and glossary. Goal: correct links, unify
 ### Links
 
 - `infra/terraform/ground/README.md` pointed at non-existent `03-storage-design.md` → `03-data-platform.md`.
-- `11-cicd-delivery.md` bare `see 05` → proper markdown link.
+- `11-cicd-delivery.md` bare cross-reference to doc 05 → proper markdown link.
 
 ### Factual drift (code vs docs)
 
@@ -63,11 +63,7 @@ Manual pass over `docs/`, all READMEs, and glossary. Goal: correct links, unify
 
 ## Suggested CI checks (for issue #1)
 
-```bash
-# Relative .md link resolver (Python one-liner in audit script)
-# Glossary anchor: every 00-glossary.md#... from docs/ must resolve
-# Optional: markdown-spellcheck with US locale
-```
+Implemented in [`scripts/check_docs.py`](../../scripts/check_docs.py) — see [ci-rules.md](ci-rules.md).
 
 ## Files touched
 

docs/improvement/README.md

@@ -6,9 +6,11 @@ Point-in-time audits and follow-up work for links, terminology, readability, and
 | --- | --- | --- |
 | [2026-07-10 audit](2026-07-10-audit.md) | 2026-07-10 | Full `docs/` + README pass; glossary T0–T4; live demo links |
 
-## Future automation
+## CI automation
 
-Tracked in [issue #1](https://git.produktor.io/eSlider/swarm-house/issues/1): CI link checker, glossary anchor validation, terminology lint.
+Enforced in Gitea Actions (`docs-verify` job). Rules live in [`scripts/docs_rules.yaml`](../../scripts/docs_rules.yaml); see [ci-rules.md](ci-rules.md).
+
+Originally tracked in [issue #1](https://git.produktor.io/eSlider/swarm-house/issues/1).
 
 ## How to add a new report
 

docs/improvement/ci-rules.md

@@ -0,0 +1,56 @@
+# CI documentation rules
+
+Automated checks for markdown in this repository. Enforced on every **pull request** and on **push to `main`** (including docs-only pushes that skip the simulator gate).
+
+Implementation: [`scripts/check_docs.py`](../../scripts/check_docs.py) reads [`scripts/docs_rules.yaml`](../../scripts/docs_rules.yaml).
+
+Workflow: [`.github/workflows/release.yaml`](../../.github/workflows/release.yaml) job **`Verify documentation`**.
+
+Tracked from [issue #1](https://git.produktor.io/eSlider/swarm-house/issues/1) and the [2026-07-10 audit](2026-07-10-audit.md).
+
+## Gates
+
+| Rule | Severity | What it checks |
+| --- | --- | --- |
+| Relative links | error | Every `[text](path)` and `[text](path#anchor)` to a local `.md` file resolves to an existing file |
+| Fragment anchors | error | `#anchor` fragments match a `##` or `###` heading in the target file (GitHub-style slug) |
+| Banned targets | error | Known-bad paths (e.g. deleted `03-storage-design.md`) are not linked |
+| Glossary fragments | error | All `00-glossary.md#…` links use anchors that exist in the glossary |
+| Section vs term links | warning | Links to glossary *section* headers (`#swarm--robotics`, …) are listed; prefer `###` term anchors where they exist |
+| Bare cross-refs | error | Prose cross-references to numbered docs without a proper markdown link |
+
+Warnings are printed in the job log but do not fail CI. Errors exit non-zero and block merge.
+
+## PR workflow
+
+```mermaid
+flowchart LR
+  PR[Pull request] --> DV[docs-verify]
+  PR --> V[verify simulator]
+  V --> S[Trivy scan]
+  DV --> OK{All required green?}
+  V --> OK
+  S --> OK
+  OK -->|yes| Merge[Mergeable]
+```
+
+| Event | `docs-verify` | `verify` + `scan` |
+| --- | --- | --- |
+| PR → `main` | always | always |
+| Push → `main` (code paths) | always | runs |
+| Push → `main` (docs/README/AGENTS only) | always | skipped (`paths-ignore`) |
+
+Docs-only changes therefore still get link and glossary validation on `main`; they do not trigger a semantic release.
+
+## Local run
+
+```bash
+python scripts/check_docs.py
+python scripts/check_docs.py --warnings-as-errors   # treat section-link warnings as failures
+```
+
+## Changing rules
+
+1. Edit `scripts/docs_rules.yaml` (paths, banned targets, preferred anchors).
+2. Adjust `scripts/check_docs.py` only when adding a new rule *type*.
+3. Add a row to the table above and mention it in the audit/improvement record if the change is significant.

scripts/check_docs.py

@@ -0,0 +1,219 @@
+#!/usr/bin/env python3
+"""Validate markdown links, glossary anchors, and readability rules."""
+
+from __future__ import annotations
+
+import argparse
+import re
+import sys
+from pathlib import Path
+
+try:
+    import yaml
+except ImportError:  # pragma: no cover - stdlib fallback for minimal envs
+    yaml = None  # type: ignore[assignment]
+
+ROOT = Path(__file__).resolve().parents[1]
+RULES_PATH = Path(__file__).with_name("docs_rules.yaml")
+
+LINK_RE = re.compile(r"!?\[[^\]]*\]\(([^)]+)\)")
+BARE_SEE_RE = re.compile(r"(?<![\[(/])see\s+(\d{2})(?![\w-])", re.IGNORECASE)
+SKIP_SCHEMES = ("http://", "https://", "mailto:", "tel:", "#")
+
+
+def load_rules(path: Path = RULES_PATH) -> dict:
+    text = path.read_text(encoding="utf-8")
+    if yaml is not None:
+        return yaml.safe_load(text)
+    raise RuntimeError("PyYAML is required to load docs_rules.yaml")
+
+
+def slug_heading(text: str) -> str:
+    """Anchor slug used in this repository's glossary links."""
+    text = text.strip()
+    text = re.sub(r"\s&\s", " — ", text)
+    if " — " in text:
+        left, right = text.split(" — ", 1)
+        return f"{_slug_part(left)}--{_slug_part(right)}"
+    return _slug_part(text)
+
+
+def _slug_part(text: str) -> str:
+    slug = text.strip().lower()
+    slug = re.sub(r"[^\w\s-]", "", slug)
+    slug = re.sub(r"\s+", "-", slug)
+    return slug
+
+
+def collect_anchors(md_path: Path) -> set[str]:
+    anchors: set[str] = set()
+    for line in md_path.read_text(encoding="utf-8").splitlines():
+        if line.startswith("##"):
+            anchors.add(slug_heading(line.lstrip("#").strip()))
+    return anchors
+
+
+def expand_markdown_paths(patterns: list[str]) -> list[Path]:
+    files: set[Path] = set()
+    for pattern in patterns:
+        for path in ROOT.glob(pattern):
+            if path.is_file() and path.suffix == ".md":
+                files.add(path)
+    return sorted(files)
+
+
+def split_link_target(raw: str) -> tuple[str, str | None]:
+    target = raw.strip()
+    if target.startswith("<") and target.endswith(">"):
+        target = target[1:-1]
+    if "#" in target:
+        path_part, fragment = target.split("#", 1)
+        return path_part, fragment or None
+    return target, None
+
+
+def resolve_path(source: Path, link_path: str) -> Path | None:
+    if not link_path or link_path.startswith(SKIP_SCHEMES):
+        return None
+    candidate = (source.parent / link_path).resolve()
+    try:
+        candidate.relative_to(ROOT.resolve())
+    except ValueError:
+        return None
+    return candidate
+
+
+def is_glossary_link(link_path: str, rules: dict) -> bool:
+    glossary = rules["glossary"]["file"].replace("\\", "/")
+    normalized = link_path.replace("\\", "/")
+    return (
+        normalized == glossary
+        or normalized.endswith("/" + glossary)
+        or normalized.endswith(glossary.split("/")[-1])
+    )
+
+
+def check_file(
+    md_path: Path,
+    rules: dict,
+    anchor_cache: dict[Path, set[str]],
+) -> tuple[list[str], list[str]]:
+    errors: list[str] = []
+    warnings: list[str] = []
+    rel = md_path.relative_to(ROOT)
+    text = md_path.read_text(encoding="utf-8")
+
+    if rules.get("readability", {}).get("forbid_bare_see_refs"):
+        prose = re.sub(r"`[^`]*`", "", text)
+        for match in BARE_SEE_RE.finditer(prose):
+            errors.append(
+                f"{rel}:{_line_no(text, match.start())}: bare cross-ref "
+                f"'see {match.group(1)}' — use a markdown link"
+            )
+
+    for match in LINK_RE.finditer(text):
+        raw = match.group(1)
+        if any(raw.startswith(s) for s in SKIP_SCHEMES):
+            continue
+
+        link_path, fragment = split_link_target(raw)
+        if not link_path.endswith(".md"):
+            continue
+
+        for banned in rules.get("banned_link_targets", []):
+            if banned in link_path.replace("\\", "/"):
+                errors.append(
+                    f"{rel}:{_line_no(text, match.start())}: banned link target "
+                    f"'{link_path}'"
+                )
+
+        target = resolve_path(md_path, link_path)
+        if target is None:
+            errors.append(
+                f"{rel}:{_line_no(text, match.start())}: link escapes repo or "
+                f"is invalid: {raw}"
+            )
+            continue
+        if not target.is_file():
+            errors.append(
+                f"{rel}:{_line_no(text, match.start())}: broken link "
+                f"{link_path} -> {target.relative_to(ROOT)}"
+            )
+            continue
+
+        if fragment:
+            if target not in anchor_cache:
+                anchor_cache[target] = collect_anchors(target)
+            if fragment not in anchor_cache[target]:
+                errors.append(
+                    f"{rel}:{_line_no(text, match.start())}: unknown anchor "
+                    f"#{fragment} in {target.relative_to(ROOT)}"
+                )
+
+        if fragment and is_glossary_link(link_path, rules):
+            section_anchors = set(rules.get("glossary_section_anchors", []))
+            if fragment in section_anchors:
+                warnings.append(
+                    f"{rel}:{_line_no(text, match.start())}: glossary section "
+                    f"link #{fragment} — prefer a ### term anchor where one exists"
+                )
+
+    return errors, warnings
+
+
+def _line_no(text: str, index: int) -> int:
+    return text.count("\n", 0, index) + 1
+
+
+def run_checks(warnings_as_errors: bool = False) -> int:
+    rules = load_rules(ROOT / "scripts/docs_rules.yaml")
+    anchor_cache: dict[Path, set[str]] = {}
+    all_errors: list[str] = []
+    all_warnings: list[str] = []
+
+    glossary_path = ROOT / rules["glossary"]["file"]
+    anchor_cache[glossary_path] = collect_anchors(glossary_path)
+
+    for md_path in expand_markdown_paths(rules["markdown_paths"]):
+        errors, warnings = check_file(md_path, rules, anchor_cache)
+        all_errors.extend(errors)
+        all_warnings.extend(warnings)
+
+    if all_warnings:
+        print("Documentation warnings:")
+        for msg in sorted(all_warnings):
+            print(f"  warning: {msg}")
+        print()
+
+    if all_errors:
+        print("Documentation errors:")
+        for msg in sorted(all_errors):
+            print(f"  error: {msg}")
+        print(f"\n{len(all_errors)} error(s), {len(all_warnings)} warning(s)")
+        return 1
+
+    if warnings_as_errors and all_warnings:
+        print(f"\n{len(all_warnings)} warning(s) treated as errors")
+        return 1
+
+    print(
+        f"Documentation checks passed "
+        f"({len(expand_markdown_paths(rules['markdown_paths']))} files, "
+        f"{len(all_warnings)} warning(s))"
+    )
+    return 0
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument(
+        "--warnings-as-errors",
+        action="store_true",
+        help="Exit non-zero when section-link warnings are present",
+    )
+    args = parser.parse_args()
+    sys.exit(run_checks(warnings_as_errors=args.warnings_as_errors))
+
+
+if __name__ == "__main__":
+    main()

scripts/docs_rules.yaml

@@ -0,0 +1,47 @@
+# Documentation rules enforced by CI (scripts/check_docs.py).
+# See docs/improvement/ci-rules.md for rationale and PR workflow.
+
+markdown_paths:
+  - docs/**/*.md
+  - README.md
+  - CONTRIBUTING.md
+  - AGENTS.md
+  - simulator/README.md
+  - prototype/README.md
+  - infra/**/README.md
+
+glossary:
+  file: docs/00-glossary.md
+
+# Relative link targets that must never reappear (known regressions).
+banned_link_targets:
+  - 03-storage-design.md
+
+# Glossary links to these section headers are allowed but reported as warnings
+# when a term-specific ### anchor exists (see preferred_term_anchors).
+glossary_section_anchors:
+  - data--storage
+  - swarm--robotics
+  - infrastructure--delivery
+
+# First-use vocabulary should link to these ### anchors, not section headers.
+preferred_term_anchors:
+  - t0--hot
+  - t1--warm
+  - t2--shared
+  - t3--warehouse
+  - t4--dev
+  - source-of-truth
+  - architecture-layer-1--ingestion
+  - architecture-layer-2--storage-and-transform
+  - architecture-layer-3--serving-and-sync
+  - pipeline-stage
+  - bulk-sync
+  - offload
+  - rclone
+  - adr
+  - asr
+
+readability:
+  # Fail when prose uses bare "see 05" instead of a markdown link.
+  forbid_bare_see_refs: true

simulator/tests/test_check_docs.py

@@ -0,0 +1,73 @@
+"""Tests for scripts/check_docs.py — documentation CI rules."""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+
+import pytest
+
+SCRIPTS = Path(__file__).resolve().parents[2] / "scripts"
+sys.path.insert(0, str(SCRIPTS))
+
+from check_docs import (  # noqa: E402
+    collect_anchors,
+    run_checks,
+    slug_heading,
+    split_link_target,
+)
+
+ROOT = Path(__file__).resolve().parents[2]
+GLOSSARY = ROOT / "docs/00-glossary.md"
+
+
+def test_slug_heading_matches_glossary_anchors() -> None:
+    assert slug_heading("T0 — hot") == "t0--hot"
+    assert slug_heading("T3 — warehouse") == "t3--warehouse"
+    assert slug_heading("Source of truth") == "source-of-truth"
+    assert slug_heading("Data & storage") == "data--storage"
+
+
+def test_glossary_term_anchors_exist() -> None:
+    anchors = collect_anchors(GLOSSARY)
+    for term in (
+        "t0--hot",
+        "t1--warm",
+        "t2--shared",
+        "t3--warehouse",
+        "t4--dev",
+        "source-of-truth",
+        "bulk-sync",
+        "offload",
+    ):
+        assert term in anchors, f"missing glossary anchor #{term}"
+
+
+def test_split_link_target() -> None:
+    assert split_link_target("foo.md#bar") == ("foo.md", "bar")
+    assert split_link_target("foo.md") == ("foo.md", None)
+
+
+def test_repo_docs_pass() -> None:
+    assert run_checks() == 0
+
+
+def test_broken_link_detected(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+    docs = tmp_path / "docs"
+    docs.mkdir()
+    (docs / "00-glossary.md").write_text("# Glossary\n\n## Data & storage\n")
+    (docs / "bad.md").write_text("See [missing](missing.md).\n")
+    rules = tmp_path / "scripts"
+    rules.mkdir()
+    (rules / "docs_rules.yaml").write_text(
+        "markdown_paths:\n  - docs/**/*.md\n"
+        "glossary:\n  file: docs/00-glossary.md\n"
+        "banned_link_targets: []\n"
+        "glossary_section_anchors: []\n"
+        "preferred_term_anchors: []\n"
+        "readability:\n  forbid_bare_see_refs: false\n"
+    )
+    checker = SCRIPTS / "check_docs.py"
+    monkeypatch.setattr("check_docs.ROOT", tmp_path)
+    monkeypatch.setattr("check_docs.RULES_PATH", rules / "docs_rules.yaml")
+    assert run_checks() == 1