feat(mail): full Gmail+OnlyOffice sync, import, and brain indexing
- bin/mail/sync.go: async Go sync engine (8 workers, paginated Gmail via API + OnlyOffice IMAP); Gmail attachments key off body.attachmentId, not MIME partId; ICS sidecars Latin-1->UTF-8 normalized (TestICSToMarkdownNormalizesLatin1) - bin/mail/import: message.json -> markdown; PDFs via pdftotext -layout fast path with docling subprocess fallback for the ~5% textless files - bin/mail/index_mail: fresh-rebuild indexer (repo corpus + mail) avoiding ladybug WAL corruption on bulk-insert into indexed DBs; split from import - bin/kb/index: keep FTS/VECTOR indexes across incremental runs (drop+recreate leaves stale backing tables killing the vector index) - docs: README/PLAN/AGENTS cover the mail pipeline Result: 17,835 messages -> 28,918 info leafs, FTS+HNSW healthy.
This commit is contained in:
Executable
+25
@@ -0,0 +1,25 @@
|
||||
//usr/bin/env go run "$0" "$@"; exit
|
||||
// bin/mail/sync.go - async download of OnlyOffice and Gmail mail to var/mail/.
|
||||
//
|
||||
// ./bin/mail/sync.go --source onlyoffice,gmail --limit 50 --workers 8
|
||||
// ./bin/mail/sync.go --source gmail --force
|
||||
// ./bin/mail/sync.go --dry-run
|
||||
//
|
||||
// Writes raw message.json + attachments under var/mail/<folder>/<id>/; run
|
||||
// bin/mail/import --from-raw afterwards to convert everything to markdown.
|
||||
//
|
||||
// Shebang trick: first line is a Go `//` comment; the real code lives in the
|
||||
// importable package (module path, never a relative import).
|
||||
// NOTE: never run `gofmt -w` on this file - it rewrites `//usr/bin/env` to
|
||||
// `// usr/...` and breaks the shebang.
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
|
||||
"github.com/eSlider/2dph/bin/mail/sync"
|
||||
)
|
||||
|
||||
func main() {
|
||||
os.Exit(sync.Main(os.Args[1:]))
|
||||
}
|
||||
Reference in New Issue
Block a user