2dph/qa/load_test_summary.md

Brain Load Test Summary

Date: 2026-08-11 Project: 2dph (deductionphile) Target: LadybugDB-embedded knowledge graph brain

Test Suite

Four independent load tests were written and executed in qa/:

Test	Purpose	Key Finding
load_test_search.py	FTS, vector, hybrid search latency	FTS: 2.8ms, Vector: 1.9ms, Hybrid: 3.3ms
load_test_graph.py	Cypher hop traversal (1-hop, 2-hop, 3-hop)	1-hop: 2.8ms, 2-hop: 4.7ms, 3-hop: 6.6ms
load_test_queries.py	Query pattern diversity (9 patterns)	All patterns under 25ms
load_test_bulk.py	Bulk insert throughput (leafs/sec)	251 leafs/sec (with index drop/recreate)

Results

1. Search Performance (load_test_search.py, 10 iterations)

Mode	Avg Latency (ms)	Description
FTS (BM25)	2.8 ms	Pure keyword search
Vector (HNSW cosine)	1.9 ms	Embedding similarity search
Hybrid (RRF merge)	3.3 ms	FTS + vector fusion

Observation: All modes under 5ms. Hybrid is ~1.8x slower than individual modes due to RRF overhead, but still well under 25ms per query.

2. Graph Traversal (load_test_graph.py, 10 iterations)

Pattern	Avg Latency (ms)	Description
1-hop (Leaf -FROM_FILE-> File)	2.8 ms	Simple edge traversal
2-hop (Leaf -> File -> Commit)	4.7 ms	Two-hop path with mix node types
3-hop (facts -from_file-> File -> HAS_VERSION-> Commit -AUTHORED-> Person)	6.6 ms	Three-hop path with root filter
Degree centrality (avg children per file)	4.2 ms	Aggregation query

Observation: Graph queries are very fast (<10ms even for 3 hops) on the knowledge graph.

3. Query Pattern Diversity (load_test_queries.py, 10 iterations)

Query Pattern	Avg Latency (ms)
fact_source (docker, root=facts)	5.3
info_docker (docker, root=info)	3.4
info_k8s (kubernetes, root=info)	3.2
repo_2dph (search term, repo=eSlider/2dph)	3.6
facts_no_root (search, root=facts)	4.1
hybrid_container (container, hybrid search)	3.9
hybrid_service (service, hybrid search)	3.7
multi_obs (observability, multi-word)	4.3
multi_container (container orchestration, multi-word)	4.7

Observation: All 9 query patterns complete in under 25ms. The system correctly handles root-filtered and repo-filtered searches.

4. Bulk Insert (load_test_bulk.py, 30 leafs, indexes dropped before insert)

Metric	Value
Total time for 30 leafs	0.12s
Throughput	251 leafs/sec

Critical observation (corrected 2026-08-12): LadybugDB 0.19 must not DROP INDEX for FTS/VECTOR and recreate. DROP leaves ghost catalog tables (_0_Leaf_vec_UPPER, 0_id_docs); CREATE then fails with "already exists in catalog" while SHOW_INDEXES omits the index — HNSW looks dead until var/kb.lbug is deleted. Upsert while indexes exist keeps HNSW queryable. Fresh indexes: delete the DB file and bin/kb/index --rebuild. See kblib.create_fts_and_vector / ensure_indexes.

Critical Assessment - Evidence Rule Working

The most important finding: the evidence-based audit correctly enforces the two-source rule for facts.

• bin/facts/audit db runs against var/kb.lbug and asserts each root=facts leaf has:

  • A source field containing " x " (indicating two independent sources, e.g., "docker ps x compose:docker-compose.yml")
  • A non-empty loc (evidence pointer)
  • confidence='confirmed'

• Before cleanup: Database had 50 test facts with source="load-test" (single source) → audit correctly flagged all as failing the 2-source rule

• After cleanup (12 facts from extract): Audit passes (ok: true, problems: []) because the 12 facts have proper 2-source evidence:

  • 11 facts: source="docker ps x compose:..." or source="docker ps x compose:..."
  • 1 fact: source="ssh config x docs(README.md, PLAN.md, AGENTS.md)"

This validates the core design principle from PLAN.md (D8/D11): a fact needs ≥2 independent sources or it is (not confirmed).

Database State (After Cleanup)

Metric	Value
Total leaves	89 (47 info + 12 facts)
Facts (root=facts)	12, all with 2-source evidence
Info (root=info)	47 (from markdown corpus)
Audit result	ok: true, problems: []

Files in qa/

• load_test_search.py - Search latency test (FT/Vector/Hybrid)
• load_test_graph.py - Graph traversal test (1-hop, 2-hop, 3-hop)
• load_test_queries.py - Query pattern diversity test (9 patterns)
• load_test_bulk.py - Bulk insert throughput test
• load_test_summary.md - This summary

Verdict

The brain performs well within design parameters:

• Search/retrieval latency: sub-25ms across all modes
• Graph traversal: under 10ms even for 3-hop paths
• Bulk insertion: ~250 leafs/sec (with proper index management)
• Evidence enforcement: The two-source audit correctly validates facts, confirming the detective method works as designed (facts root = strong assertions, info root = weak claims)

The system is ready for production use with the understanding that:

1. Bulk inserts must drop/recreate indexes to avoid corruption
2. Facts are only stored when backed by >=2 independent sources (enforced by audit)
3. The info root holds the narrative corpus (28K+ markdown-derived leafs)
4. Facts root holds confirmed assertions with evidence links