feat(mail): full Gmail+OnlyOffice sync, import, and brain indexing

- bin/mail/sync.go: async Go sync engine (8 workers, paginated Gmail via
  API + OnlyOffice IMAP); Gmail attachments key off body.attachmentId, not
  MIME partId; ICS sidecars Latin-1->UTF-8 normalized (TestICSToMarkdownNormalizesLatin1)
- bin/mail/import: message.json -> markdown; PDFs via pdftotext -layout
  fast path with docling subprocess fallback for the ~5% textless files
- bin/mail/index_mail: fresh-rebuild indexer (repo corpus + mail) avoiding
  ladybug WAL corruption on bulk-insert into indexed DBs; split from import
- bin/kb/index: keep FTS/VECTOR indexes across incremental runs (drop+recreate
  leaves stale backing tables killing the vector index)
- docs: README/PLAN/AGENTS cover the mail pipeline

Result: 17,835 messages -> 28,918 info leafs, FTS+HNSW healthy.

This commit is contained in:

eSlider

2026-08-11 21:57:38 +01:00

parent 8781c0c3eb

commit 678a1d1dba

20 changed files with 5000 additions and 26 deletions

									
										go.mod
									
		+6
		-1
	
												View File
												
				@@ -1,3 +1,8 @@

				module github.com/eSlider/2dph

				go 1.25

				go 1.25.0

				require (

					github.com/arran4/golang-ical v0.3.5

					golang.org/x/text v0.40.0

				)