Compare commits

..
173 Commits
Author SHA1 Message Date
eSlider aa069bea56 Merge pull request 'chore(release): v0.21.0 (#78)' (#79) from chore/release-v0.21.0#78 into main
Release / GoReleaser (push) Skipped
Tests / Test (Go 1.25) (push) Successful in 1m23s
Tests / Secret scan (gitleaks) (push) Successful in 3s
Tests / Test (Go stable) (push) Successful in 1m28s
2026-09-23 18:25:08 +01:00
eSlider a8fa202496 chore(release): v0.21.0 (#78)
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 5s
Tests / Test (Go 1.25) (pull_request) Successful in 1m29s
Tests / Test (Go stable) (pull_request) Successful in 1m33s
2026-09-23 18:25:00 +01:00
eSlider 87ff317166 Merge pull request 'chore: move business tooling out of the public library; tidy filestore naming' (#77) from chore/split-business-tools into main
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go 1.25) (push) Successful in 1m25s
Tests / Test (Go stable) (push) Successful in 1m28s
2026-09-23 09:05:48 +01:00
eSlider dc62570a5c chore: move business tooling out of the public library; tidy filestore naming
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 1m19s
Tests / Test (Go 1.25) (pull_request) Successful in 1m24s
Keep the public tree project-generic. Business/one-off tools, deployment
and business docs move to the private oo-workspace repo.

Moved to oo-workspace:
- cmd/ooscan, cmd/pdfamount, cmd/kontoblatt, cmd/kontolink
- internal/xlspipe (cutover-portugal workbook) -> oow workbook build
  (drops the --template/--title flags from oo docs put-xlsx)
- deploy/docker-compose.rclone-webdav.yml + docs/rclone-webdav.md
- docs/crm-associations.md

Removed GitHub-era leftovers:
- .github/workflows/release-please.yml, release-please-config.json,
  .release-please-manifest.json (tags are created on Gitea per SemVer)

Naming: the FileStore subsystem is now filestore_*.go (was file_*.go) to
match files_*.go (project Documents). Docs/AGENTS/README updated.
2026-09-23 09:05:40 +01:00
eSlider a9a77e110d Merge pull request 'feat: upstream generic workspace tooling (board-sync, crm audit, catalog names)' (#76) from feat/upstream-oow-workspace into main
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go stable) (push) Successful in 1m23s
Tests / Test (Go 1.25) (push) Successful in 1m23s
2026-09-22 23:13:49 +01:00
eSlider 82da45dc4d feat: upstream generic workspace tooling (board-sync, crm audit, catalog names)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 5s
Tests / Test (Go 1.25) (pull_request) Successful in 1m16s
Tests / Test (Go stable) (pull_request) Successful in 1m36s
Promote the generic, reusable parts of the private oo-workspace into the
public library/CLI, so oo-workspace can shrink to business glue.

- board.go: Board types + (c *Client) SyncBoard — upsert project
  milestones/tasks by exact title from a YAML board (dry-run when
  apply=false). CLI: oo projects board-sync.
- crm_audit.go: OpportunityAudit + (c *Client) AuditOpportunities —
  file/task/member counts with a generic class (ok|dup|empty|junk-title).
  CLI: oo crm audit [--out].
- catalog/names.go: CleanPersonNames / GuessNameFromEmail /
  FormatProjectTitle (ported from oo-workspace).
- catalog: adopt the newer apply/match logic — name repair (never encode
  company in lastName), company grouping key, contact-info type
  normalization, preserve an already-applied oo_id. Keeps the local
  Address/OOProjects fields and the config-driven classifier.

Business-only oo-workspace commands (crm clean/migrate, folders, search)
deliberately stay private.
2026-09-22 23:13:40 +01:00
eSlider 8848ee00f1 Merge pull request 'feat(oo): native conversion, deep links, sheet export (re-land)' (#75) from feat/oo-conversion-links into main
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go 1.25) (push) Successful in 1m29s
Tests / Test (Go stable) (push) Successful in 1m30s
2026-09-22 22:42:07 +01:00
eSlider ff46aad615 chore: keep the re-landed feature files showroom-safe
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 1m21s
Tests / Test (Go 1.25) (pull_request) Successful in 1m27s
The conversion / link / sheet commits predate the tree cleanup, so their
comments and README/test examples still carried internal refs. Re-apply
the genericization (no internal hosts, personal names or client file ids).
2026-09-22 22:41:58 +01:00
eSlider 5e73f6c335 feat(oo): sheet-aware spreadsheet export (docs csv/json)
- lib: WorkbookSheetNames/CSV/JSON (excelize) — sheet selection + JSON rows
- oo docs csv  SRC... [--sheet N] [--delimiter ,|;|||tab] [--out]
- oo docs json SRC... [--sheet N] [--out]
- SRC = OO file id or local path; --sheet is 1-based (default first)
- DS csv output is first-sheet-only (verified) -> local reader for selection/JSON
- test: TestWorkbookSheetExport
2026-09-22 22:41:14 +01:00
eSlider 7c089a6439 feat(oo): docs pdf --stream/--pipe (converted bytes to stdout) 2026-09-22 22:41:14 +01:00
eSlider 4515a88fe8 docs(readme): examples for links, conversion, team/users CRUD 2026-09-22 22:41:14 +01:00
eSlider 58b905cc49 feat(oo): native document conversion (docs pdf/presigned)
- lib: PresignedURI, SignJWT (HS256, stdlib), ConvertDocument (DocumentServer
  /converter; legacy /ConvertService.ashx), DownloadURLTo
- oo docs pdf FILE_ID|PATH...: OO file ids or local files (temp upload to
  --folder, convert, download, cleanup) -> PDF/other
- oo docs presigned FILE_ID
- docs base from $ONLYOFFICE_DOCS_URL else $ONLYOFFICE_URL/ds-vpath; JWT secret
  from $ONLYOFFICE_DS_SECRET (DocumentServer CoAuthoring secret)
- test: SignJWT
2026-09-22 22:41:14 +01:00
eSlider 520d6d05c8 docs(oo): comment where the new helpers are used
- link.go/links.go: third-party doc packs (fileid links), id-stability caveat
- auth.go AuthenticateAs / users check: email-vs-userName login finding
- projects files replace-in: version-history rationale
2026-09-22 22:41:14 +01:00
eSlider df38b75011 feat(oo): file deep links, login check, replace-in (fresh upload)
- link: print Products/Files/DocEditor.aspx?fileid=… deep links (title+url)
- lib: FileEditorURL/FolderURL helpers (+tests)
- users check: verify credentials (userName vs email) via authentication.json
- projects files replace-in FOLDER_ID FILE...: hard delete same stem|ext in a
  folder + fresh upload → single clean version (no version history)
2026-09-22 22:41:14 +01:00
eSlider ee8e88fb9b feat(oo): projects files update (overwrite existing file content) 2026-09-22 22:41:14 +01:00
eSlider e323a43317 feat(oo): projects milestone-delete 2026-09-22 22:41:14 +01:00
eSlider 71d421fb41 Merge pull request 'chore: keep host/client specifics out of the tree (env & config)' (#74) from chore/showroom-generic into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go 1.25) (push) Successful in 55s
Tests / Test (Go stable) (push) Successful in 1m6s
2026-09-22 22:40:49 +01:00
eSlider afb93feae5 chore: keep host/client specifics out of the tree (env & config)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 59s
Tests / Test (Go stable) (pull_request) Successful in 1m7s
Showroom-safe: the tree no longer carries internal hosts, IPs, ports,
personal names, client domains or client file names. Behaviour is
unchanged and now supplied per deployment.

- catalog: hardcoded mail-org / project classifiers become a YAML-driven
  Classifier (OO_CATALOG_CONFIG or --config); neutral default classifies
  nothing as work. New catalog/classify.go + example + tests.
- storage_fallback: drop the baked-in MinIO endpoint IP; require
  MINIO_ENDPOINT (+ keys) from the env.
- kontolink: build DocEditor links from $ONLYOFFICE_URL instead of a
  hardcoded portal host; kontoblatt: no client file id in the output name.
- oo: load .env CLI-wide (bootstrap.LoadEnv in execute) so
  non-authenticating commands (catalog scans) also see config.
- genericize comments/docs/fixtures (AGENTS, README, .env.example,
  crm-associations, catalog tests, ES/pdfattach tests, mails).
2026-09-22 22:03:36 +01:00
eSlider b493472f2e Merge pull request 'feat(oo): project team CRUD and user lifecycle' (#73) from feat/oo-team-users-crud into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go stable) (push) Successful in 1m1s
Tests / Test (Go 1.25) (push) Successful in 1m2s
Reviewed-on: #73
2026-09-22 14:29:47 +01:00
eSlider 1dc99d6997 feat(oo): project team CRUD and user lifecycle (block/unblock/password/delete)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 1m2s
Tests / Test (Go stable) (pull_request) Successful in 1m40s
- project team: oo projects team list|add|remove|set (portal users)
- users: get|create|update|delete|block|unblock|password
- lib: CreateUser, DeleteUser (auto-suspend before delete), Add/Remove/
  SetProjectTeam, ListProjectTeam; deleteJSON reused; jsonBodyReader helper
- tests: unmarshalResponseArray
2026-09-22 14:26:23 +01:00
eSlider 9b21d7e52a Merge pull request 'fix(crm): адреса контактов через ContactInfo типа Address (#288)' (#72) from feat/catalog-addresses#288 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go stable) (push) Successful in 58s
Tests / Test (Go 1.25) (push) Successful in 59s
2026-09-22 08:32:25 +01:00
eSlider c4beba5173 fix(crm): store contact addresses via ContactInfo Address type (#288)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 1m1s
Tests / Test (Go 1.25) (pull_request) Successful in 1m1s
2026-09-22 07:08:01 +00:00
eSlider 2a187ae132 Merge pull request 'fix(retry): глобальный rate-limit + Retry-After + cooldown (#70)' (#71) from fix/oo-backoff#70 into main
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go stable) (push) Successful in 21s
Tests / Test (Go 1.25) (push) Successful in 22s
2026-09-17 13:39:56 +01:00
eSlider c9e16c7169 fix(retry): глобальный rate-limit + Retry-After + cooldown (#70)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 5s
Tests / Test (Go 1.25) (pull_request) Successful in 25s
Tests / Test (Go stable) (pull_request) Successful in 27s
2026-09-17 12:39:25 +00:00
eSlider 194ae62720 Merge pull request 'docs: убрать потребительские детали match из index-and-search (#34)' (#69) from docs/index-and-search-scope into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go stable) (push) Successful in 26s
Tests / Test (Go 1.25) (push) Successful in 26s
2026-09-17 08:51:45 +01:00
eSlider 0d73146ebc docs: убрать потребительские детали match из index-and-search (#34)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 3s
Tests / Test (Go 1.25) (pull_request) Successful in 19s
Tests / Test (Go stable) (pull_request) Successful in 29s
2026-09-17 07:51:02 +00:00
eSlider 46661f51f8 Merge pull request 'docs: карта поиска и обновления индексов (#34)' (#68) from docs/index-and-search into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go stable) (push) Successful in 25s
Tests / Test (Go 1.25) (push) Successful in 29s
2026-09-17 08:49:31 +01:00
eSlider f22d714be7 docs: карта поиска и обновления индексов (#34)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 19s
Tests / Test (Go stable) (pull_request) Successful in 21s
2026-09-17 07:48:40 +00:00
eSlider 6f85c206cf Merge pull request 'test(files): live-тест файлового dedup (#63)' (#65) from test/w6-file-dedup#63 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Test (Go stable) (push) Failing after 3s
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go 1.25) (push) Successful in 28s
2026-09-16 23:03:23 +01:00
eSlider 1c00edbd0b Merge pull request 'test(files): live CRUD папок + фасадный CRUD (#62)' (#66) from test/w5-folder-crud#62 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Test (Go stable) (push) Failing after 4s
Tests / Secret scan (gitleaks) (push) Successful in 6s
Tests / Test (Go 1.25) (push) Successful in 23s
2026-09-16 23:03:17 +01:00
eSlider 9cd157c59f test(files): live CRUD папок + фасадный CRUD (#62)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Test (Go stable) (pull_request) Failing after 4s
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 19s
2026-09-16 21:49:10 +00:00
eSlider 932cd2895a fix(files): REST FileStore resolves folders for stat/rename/move/delete (#62) 2026-09-16 21:49:08 +00:00
eSlider 5588c2d3f7 fix(files): ретраить transient-ответы при удалении файлов (#63)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 20s
Tests / Test (Go 1.25) (pull_request) Successful in 20s
DeleteDavItems (через ApplyDedupGroups/DeleteFilesByDedupKey) ходил в
deleteJSON напрямую и падал на 429/502/503/504. Теперь каждый delete идёт
через DoRetry, как остальные bulk-пути.
2026-09-16 21:47:37 +00:00
eSlider f8783b86a8 fix(files): пропускать пустой dedup-ключ (dotfiles) (#63)
Файлы без stem (.env, .gitignore, .npmrc) давали FileDedupKey="", и
findWithinFolderDuplicates/findCrossFolderDuplicates считали их одной
группой дублей и удаляли лишние. Пустой ключ больше не образует группу.
2026-09-16 21:47:36 +00:00
eSlider d9db1c5e56 test(files): live-тест файлового dedup (#63)
TestIntegrationFileDedup: throwaway-проект, две папки + root, дубли по
stem|ext, FindProjectDuplicates/mergeProjectRootForDedupe,
ApplyDedupGroups/DeleteFilesByDedupKey, survivor и удаления подтверждены
polling-ом. Папки/аплоады ретраят временный post-create 500 портала.
2026-09-16 21:47:33 +00:00
eSlider c551634e55 Merge pull request 'docs: Testing + rclone/SQL + docs index (#53)' (#64) from docs/w1-w3#53 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go stable) (push) Successful in 19s
Tests / Test (Go 1.25) (push) Successful in 24s
2026-09-16 22:41:12 +01:00
eSlider a88fd5b40c docs: Testing section, rclone/SQL refs, docs index (#53)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 19s
Tests / Test (Go stable) (pull_request) Successful in 22s
2026-09-16 21:39:41 +00:00
eSlider 9b149281a8 Merge pull request 'fix(retry): починка красного, полный прогон тестов (#57)' (#61) from chore/test-green#57 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go 1.25) (push) Successful in 23s
Tests / Test (Go stable) (push) Successful in 24s
2026-09-16 22:37:05 +01:00
eSlider d6b8eb7777 fix(retry): retry transient edge answers centrally, fix stale task test (#57)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 21s
Tests / Test (Go stable) (pull_request) Successful in 21s
- Route HTTP helpers (getJSON/formRequest/deleteReq/postJSON/putJSON/
  multipart upload), Query and AuthenticateContext/ensureToken through the
  deterministic 429/502/503/504 retry (retryRaw), so the integration suite
  no longer fails on the shared openresty rate limit under parallel runs.
- Fix stale cmd/office/fetch integration test: loader.TaskFields was
  replaced by loader.DetailForm (broke go vet -tags=integration).
- Add unit tests for Transient/DoRetry.
2026-09-16 21:32:14 +00:00
eSlider 8a17a32269 Merge pull request 'feat(deploy): rclone WebDAV mount (docker compose) + smoke + docs (#54)' (#59) from feat/rclone-webdav#54 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 8s
Tests / Test (Go 1.25) (push) Successful in 25s
Tests / Test (Go stable) (push) Successful in 27s
2026-09-16 22:30:16 +01:00
eSlider 43a1a47faa Merge pull request 'test(es): доказать, что oo search идёт в Elasticsearch (#56)' (#58) from feat/es-integration#56 into main
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go 1.25) (push) Successful in 26s
Tests / Test (Go stable) (push) Successful in 28s
2026-09-16 22:30:14 +01:00
eSlider 620505cb24 Merge pull request 'feat(files): SQL backend via Client.FileStore + live MySQL facade test (#55)' (#60) from feat/db-integration#55 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go stable) (push) Successful in 30s
Tests / Test (Go 1.25) (push) Successful in 37s
2026-09-16 22:30:05 +01:00
eSlider a6bba30438 feat(files): SQL file backend via Client.FileStore/SQLFileStore + live MySQL integration (#55)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 24s
Tests / Test (Go 1.25) (pull_request) Successful in 26s
2026-09-16 21:26:15 +00:00
SE 6549fbf7e2 feat(deploy): rclone WebDAV mount compose + smoke + docs (#54)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 20s
Tests / Test (Go stable) (pull_request) Successful in 22s
2026-09-16 21:25:38 +00:00
eSlider 1a8bf0f83c test(es): prove oo search uses the Elasticsearch backend (#56)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 3s
Tests / Test (Go stable) (pull_request) Successful in 24s
Tests / Test (Go 1.25) (pull_request) Successful in 25s
2026-09-16 21:22:43 +00:00
eSlider 046882b8ef Merge pull request 'feat(search): полный уникальный path первым + ближайшая папка в результатах (#51)' (#52) from feat/search-path#51 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go stable) (push) Successful in 22s
Tests / Test (Go 1.25) (push) Successful in 22s
2026-09-16 22:19:46 +01:00
Yftyr a3ef83a961 feat(search): full unique path first + immediate folder in results (#51)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 21s
Tests / Test (Go stable) (pull_request) Successful in 26s
2026-09-16 21:19:32 +00:00
eSlider 54f825f8e8 Merge pull request 'feat(search): подстрока/AND, scope по поддереву, лимит 1000 (#49)' (#50) from fix/es-substring#49 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go stable) (push) Successful in 24s
Tests / Test (Go 1.25) (push) Successful in 26s
2026-09-16 22:16:01 +01:00
Yftyr 6a4940ba82 feat(search): substring/AND terms, nested folder scope, limit 1000 (#49)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 26s
Tests / Test (Go 1.25) (pull_request) Successful in 39s
2026-09-16 21:15:44 +00:00
eSlider 1da368fd5d Merge pull request 'fix(docpipe): индекс вложений .yaml и без расширения (#47)' (#48) from fix/attachments-yaml#47 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go stable) (push) Successful in 26s
Tests / Test (Go 1.25) (push) Successful in 27s
2026-09-16 19:54:20 +01:00
Yftyr 66a6b57401 fix(docpipe): index .yaml and extensionless PDF attachments (#47)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 21s
Tests / Test (Go 1.25) (pull_request) Successful in 22s
2026-09-16 18:54:09 +00:00
eSlider 075c66d4d7 Merge pull request 'docs: единый контракт файлового клиента (#39)' (#46) from docs/file-client#39 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 6s
Tests / Test (Go stable) (push) Successful in 21s
Tests / Test (Go 1.25) (push) Successful in 1m20s
2026-09-16 18:37:36 +01:00
eSlider c59ad645ea docs(files): add unified file client contract (#39)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 17s
Tests / Test (Go stable) (pull_request) Successful in 1m20s
2026-09-16 17:37:15 +00:00
eSlider c76ab7267d Merge pull request 'feat(search): PDF-контент в поиске — свой индекс oo_docs_text (#42)' (#44) from feat/pdf-content#42 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go stable) (push) Successful in 1m21s
Tests / Test (Go 1.25) (push) Successful in 1m23s
2026-09-16 18:34:54 +01:00
eSlider 3cc288d281 feat(search): index embedded PDF attachment text (#42)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 25s
Tests / Test (Go 1.25) (pull_request) Successful in 28s
2026-09-16 17:34:04 +00:00
eSlider ce4778bdf1 fix(files): dedupe ProviderPG after facade/SQL merge, rename test fake 2026-09-16 17:29:01 +00:00
eSlider 3fe43ee82c feat(search): PDF content via own ES index and oo index (#42) 2026-09-16 17:28:05 +00:00
eSlider 6d93ab5b1f Merge pull request 'feat(files): read-only SQL file store (Community Server DB) (#36)' (#45) from feat/pg-store#36 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 3s
Tests / Test (Go stable) (push) Failing after 26s
Tests / Test (Go 1.25) (push) Failing after 28s
2026-09-16 18:27:21 +01:00
eSlider 95d79925ea Merge pull request 'refactor(files): единый файловый фасад + миграция CLI/TUI (#38)' (#43) from feat/file-facade#38 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go 1.25) (push) Failing after 37s
Tests / Test (Go stable) (push) Failing after 38s
2026-09-16 18:27:21 +01:00
eSlider bf3ef025aa feat(files): read-only SQL file store over Community Server DB (#36)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 1m56s
Tests / Test (Go stable) (pull_request) Successful in 1m57s
2026-09-16 17:24:22 +00:00
eSlider ba29738482 refactor(files): single file client facade + CLI/TUI migration (#38)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 19s
Tests / Test (Go 1.25) (pull_request) Successful in 22s
- FileClient composes FileStore/Searcher backends; Read()/Write()/Search()
  select REST/DAV/PG/ES with transient read fallback. Client.Files() returns
  it and *FileClient implements FileStore, so existing callers keep working.
- Entry gains backend-native Updated + folder FilesCount/FoldersCount so
  dav ls output round-trips.
- cmd/oo dav/projects files and cmd/office/fetch download/preview/delete go
  through FileStore; oo search goes through the facade.
- Mark transport methods that FileStore now abstracts as deprecated.
2026-09-16 17:20:41 +00:00
eSlider 72c7cd5173 Merge pull request 'feat(search): Elasticsearch-клиент (имя + контент) + oo search (#37)' (#41) from feat/es-search#37 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 3s
Tests / Test (Go stable) (push) Successful in 17s
Tests / Test (Go 1.25) (push) Successful in 24s
2026-09-16 17:43:21 +01:00
Yftyr b3dfff1a4c refactor(search): use canonical model from file_core.go (#37) 2026-09-16 16:43:20 +00:00
eSlider b5a61ad420 feat(search): Elasticsearch searcher (name+content) and oo search (#37) 2026-09-16 16:42:57 +00:00
eSlider 1aba545e1d Merge pull request 'feat(files): канонический Entry + FileStore (REST/DAV) (#35)' (#40) from feat/file-store#35 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 3s
Tests / Test (Go stable) (push) Successful in 18s
Tests / Test (Go 1.25) (push) Successful in 18s
2026-09-16 17:42:49 +01:00
eSlider 94951fc6c5 feat(files): canonical Entry + FileStore REST/DAV adapters (#35)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 18s
Tests / Test (Go stable) (pull_request) Successful in 18s
2026-09-16 16:30:07 +00:00
eSlider d95985e6d0 Merge pull request 'fix(pdfamount): ставка НДС не сумма; итог DKV (#29)' (#33) from fix/pdfamount-tax#29 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go 1.25) (push) Successful in 19s
Tests / Test (Go stable) (push) Successful in 20s
2026-09-15 12:58:30 +01:00
eSlider 57b383d579 fix(pdfamount): не считать ставку НДС суммой; итог DKV (#29)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 20s
Tests / Test (Go stable) (pull_request) Successful in 23s
- Строки с %/MwSt/USt/Prozent/Steuer больше не дают сумму (было: Storchen
  19,00 % -> amount 19.00 и ложная отбраковка кандидатов в матчере).
- Приоритет меток: zu zahlender betrag > rechnungsbetrag > rechnungsendbetrag
  > gesamtbetrag > gesamtsumme (inkl. Steuern); low-priority не перебивает.
- DKV: секция Gesamtsummenaufstellung (значение после «»») важнее повторяющихся
  TOTAL-строк по машинам.
- Тесты: 19,00 % не сумма; fallback на usable-метку; DKV-итог.
2026-09-15 11:57:31 +00:00
eSlider a73f6c172d Merge pull request 'feat(oo): dav rm — удаление папок/файлов Documents (#30)' (#31) from feat/dav-rm#30 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 7s
Tests / Test (Go stable) (push) Successful in 29s
Tests / Test (Go 1.25) (push) Successful in 42s
2026-09-15 09:44:02 +01:00
eSlider e49693ad2c feat(oo): dav rm — удаление папок/файлов Documents (#30)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 6s
Tests / Test (Go stable) (pull_request) Successful in 22s
Tests / Test (Go 1.25) (pull_request) Successful in 25s
DeleteDavItems был только в клиенте; добавлена команда
`oo dav rm [FILE_ID...] --folders F1,F2`.
2026-09-15 08:31:47 +00:00
eSlider b86d71211f merge: GitHub release-please v0.18+ into Gitea main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go stable) (push) Successful in 23s
Tests / Test (Go 1.25) (push) Successful in 30s
2026-09-15 09:29:13 +01:00
eSlider a2b919478e Merge pull request 'fix(pdfamount): суммы DKV/Diashop и др. (метки и форматы) (#27)' (#28) from fix/pdfamount-amounts#27 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go 1.25) (push) Successful in 18s
Tests / Test (Go stable) (push) Successful in 19s
2026-09-15 08:55:40 +01:00
eSlider ccabc11153 fix(pdfamount): widen amount labels and formats (#27)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 19s
Tests / Test (Go stable) (pull_request) Successful in 18s
Match more payable-amount labels by priority and accept DE/EN/plain
number formats, normalising to a dot-separated 2-decimal string. Adds
unit tests with synthetic DKV-style and Diashop-style lines.
2026-09-15 07:51:23 +00:00
eSlider 9a6da1a4f7 Merge pull request 'fix(files): UpdateFile uses PUT (#25)' (#26) from fix/update-file-put#25 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go stable) (push) Successful in 18s
Tests / Test (Go 1.25) (push) Successful in 23s
2026-09-15 08:22:50 +01:00
eSlider 504d13ed08 fix(files): UpdateFile uses PUT /api/2.0/files/{id}/update (#25)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 18s
Tests / Test (Go 1.25) (pull_request) Successful in 20s
2026-09-15 07:19:42 +00:00
eSlider efc0864d42 Merge pull request 'docs(oo): dav, documents files api, bulk tools, fix verbs (#22)' (#24) from docs/oo-reference#22 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go stable) (push) Successful in 19s
Tests / Test (Go 1.25) (push) Successful in 19s
Reviewed-on: #24
2026-09-14 22:44:25 +01:00
eSlider 59caf5e560 docs(oo): dav, documents files api, bulk tools, fix verbs (#22)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 17s
Tests / Test (Go stable) (pull_request) Successful in 18s
2026-09-14 21:43:37 +00:00
eSlider e7803c5269 Merge pull request 'feat(files): Documents Dav ops + UpdateFile, fileops errors (#152)' (#20) from feat/oo-automation#152 into main
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go stable) (push) Successful in 14s
Tests / Test (Go 1.25) (push) Successful in 1m6s
2026-09-14 22:35:09 +01:00
eSlider a08e7c49ad Merge branch 'main' into feat/oo-automation#152
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 1m49s
Tests / Test (Go 1.25) (pull_request) Successful in 2m20s
2026-09-14 22:31:44 +01:00
eSlider 4310aa7002 Merge pull request 'feat(files): MinIO fallback for stale S3 downloads (#152)' (#21) from feat/oo-minio-fallback#152 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 12s
Tests / Test (Go 1.25) (push) Successful in 23s
Tests / Test (Go stable) (push) Successful in 2m19s
Reviewed-on: #21
2026-09-14 22:31:32 +01:00
eSlider 1a7a962183 Merge pull request 'feat(oo): kontoblatt bulk tools, oo dav, update, retry (#22)' (#23) from feat/kontoblatt-tools#22 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go 1.25) (push) Successful in 16s
Tests / Test (Go stable) (push) Successful in 24s
Reviewed-on: #23
2026-09-14 22:31:22 +01:00
eSlider af8e3b053a feat(oo): kontoblatt bulk tools, oo dav, update, retry (#22)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 1m22s
Tests / Test (Go stable) (pull_request) Successful in 1m21s
2026-09-14 21:03:39 +00:00
eSlider 8ac777c031 feat(files): MinIO fallback for stale S3 downloads (#152)
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 1m15s
Tests / Test (Go 1.25) (pull_request) Successful in 1m22s
2026-09-14 15:41:39 +00:00
eSlider c576bfe2ee feat(files): Documents Dav ops + UpdateFile, fileops errors (#152)
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 1m17s
Tests / Test (Go 1.25) (pull_request) Successful in 1m20s
2026-09-14 12:37:57 +00:00
eSliderandGitHub 47a2c256f3 Merge pull request #48 from eSlider/release-please--branches--main--components--go-onlyoffice
chore(main): release 0.18.0
2026-09-14 09:18:11 +01:00
github-actions[bot]andGitHub c1881e1e61 chore(main): release 0.18.0 2026-09-04 14:47:27 +00:00
eSliderandGitHub 7f56332285 Merge pull request #49 from eSlider/sync/gitea-main-20260904
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go 1.25) (push) Successful in 2m4s
Tests / Test (Go stable) (push) Successful in 2m4s
sync: gitea main (sortBy fix #18 + funding)
2026-09-04 15:47:06 +01:00
eSlider a5807b9c31 Merge pull request 'docs(funding): eSlider support links (reverse-import GitHub)' (#19) from docs/funding-github into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 13s
Tests / Test (Go stable) (push) Successful in 30s
Tests / Test (Go 1.25) (push) Successful in 32s
2026-09-04 12:31:47 +01:00
eSlider d650a16a36 docs(funding): eSlider support links (reverse-import GitHub e9c969a)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 5s
Tests / Test (Go stable) (pull_request) Successful in 34s
Tests / Test (Go 1.25) (pull_request) Successful in 37s
2026-09-04 12:29:45 +01:00
eSlider 3da11586f9 Merge pull request 'fix(crm): детерминированный sortBy=id в постраничных списках контактов' (#18) from fix/crm-filter-sortby into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 10s
Tests / Test (Go 1.25) (push) Successful in 34s
Tests / Test (Go stable) (push) Successful in 1m38s
2026-09-03 13:14:36 +01:00
eSlider e9c969a89c funding: eSlider support links (sponsors, ko-fi, liberapay, patreon, polar) 2026-09-03 05:20:18 +01:00
eSlider 4d91726179 fix(crm): deterministic sortBy=id in contact paged lists
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 26s
Tests / Test (Go stable) (pull_request) Successful in 1m21s
2026-09-03 04:50:15 +01:00
eSlider 4d8af7a2fe feat(crm): add UpdateContactName and CloseCRMTask helpers
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 6s
Tests / Test (Go stable) (push) Successful in 28s
Tests / Test (Go 1.25) (push) Successful in 38s
- UpdateContactName renames a company contact via PUT /crm/contact/company/{id}
- CloseCRMTask closes a CRM task via PUT /crm/task/{id}/close.json
2026-08-31 11:38:50 +01:00
eSlider 34745e349e chore(release): trigger CI (#16)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 26s
Tests / Test (Go stable) (pull_request) Successful in 28s
2026-08-31 11:38:48 +01:00
eSliderandGitHub dfea57a57b Merge pull request #46 from eSlider/release-please--branches--main--components--go-onlyoffice
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 23s
Tests / Test (Go stable) (pull_request) Successful in 26s
Release / GoReleaser (push) Skipped
chore(main): release 0.17.0
2026-08-31 11:34:41 +01:00
github-actions[bot]andGitHub 8595c17f25 chore(main): release 0.17.0 2026-08-31 10:34:21 +00:00
eSlider 61da2fb88b Merge pull request 'fix(files): upsert uploads by default + dedupe project root (#45)' (#15) from fix/docs-upsert-dedupe into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go stable) (push) Successful in 24s
Tests / Test (Go 1.25) (push) Successful in 31s
2026-08-31 11:33:51 +01:00
eSliderandCursor 24ca144b22 fix(files): upsert uploads by default and dedupe project root
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 25s
Tests / Test (Go 1.25) (pull_request) Successful in 27s
OnlyOffice allows duplicate stem|ext in the same folder; agents hit this
via projects files upload and put-md without --folder. Default all upload
paths to replace-by-stem, add no-clobber via --no-replace, and scan
projectFolder in dedupe.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-31 11:18:54 +01:00
eSlider 93828ee19d Merge pull request 'feat(mailsync): FetchMailFolder — integration-layer mail walk for ETL consumers (2dph)' (#4) from feat/2dph-mail-integration-layer into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go 1.25) (push) Successful in 29s
Tests / Test (Go stable) (push) Successful in 35s
2026-08-31 10:08:19 +01:00
mdx-1andeSlider 35f0cb8d20 feat(mailsync): FetchMailFolder — integration-layer walk for ETL consumers
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 5s
Tests / Test (Go 1.25) (pull_request) Successful in 24s
Tests / Test (Go stable) (pull_request) Successful in 40s
Adds the high-level mail folder walk that sync pipelines need on top of
the raw mail API (list -> get -> download-attachment), so consumers stop
re-implementing it against private client copies.

  type MailSyncMessage struct { ID, Folder, Subject, From, Date, IsNew,
                                HasAttachments, Attachments }
  type MailSyncAttachment struct { ID, Name, Size, Body }
  func (c *Client) FetchMailFolder(ctx, folderID, MailSyncOptions)
                                   ([]MailSyncMessage, error)

Options: Limit / StartIndex for checkpointed walks, FetchBodies to
eagerly download attachment bytes via download.ashx (session-cookie path).

Hydration details:
- list items may omit the attachment array; when hasAttachments is set
  the full record is fetched and its attachments merged
- attachment ids accepted from id/fileId/attachmentId variants
- timestamps parsed from RFC3339 (any fractional digits) and
  second-precision forms

This is the first step of the 2dph integration layer (#1): the brain's
mail-ingest pipeline can now drop its private OOClient copy and consume
this canonical walk directly.

Tests: httptest-backed coverage for pagination, hydration with
full-record fallback, body download incl. auth-cookie requirement,
Limit/StartIndex windows, timestamp parsing.
2026-08-31 10:05:47 +01:00
eSlider ca5c85a4e2 Merge pull request 'chore(release): v0.16.0 — put-xlsx (excelize) (#13)' (#14) from chore/release-v0.16.0#13 into main
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go stable) (push) Successful in 23s
Tests / Test (Go 1.25) (push) Successful in 24s
2026-08-30 13:44:47 +01:00
eSlider 220c7e8265 chore(release): v0.16.0 — put-xlsx (excelize) (#13)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 8s
Tests / Test (Go 1.25) (pull_request) Successful in 2m3s
Tests / Test (Go stable) (pull_request) Successful in 2m5s
2026-08-30 13:41:51 +01:00
eSlider ff4c0481ba Merge pull request 'feat(docs): put-xlsx с excelize — multi-sheet бюджеты (cutover-portugal)' (#12) from feat/docs-put-xlsx into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go 1.25) (push) Successful in 2m10s
Tests / Test (Go stable) (push) Successful in 2m12s
2026-08-30 13:41:25 +01:00
eSliderandCursor 68445b0dfd feat(docs): put-xlsx with excelize cutover workbook
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 5s
Tests / Test (Go 1.25) (pull_request) Successful in 1m59s
Tests / Test (Go stable) (pull_request) Successful in 2m5s
Add internal/xlspipe (github.com/xuri/excelize/v2) and oo docs put-xlsx
to generate multi-sheet budgets with named inputs, SUM/AVG/MIN formulas,
cross-sheet links, Russian labels, and cell comments for OnlyOffice.

Template cutover-portugal: sheets Ввод, Вс 6.09, Чт 3.09, Ср гип, Сводка.
Upload via UploadProjectFileReplacing (--replace default).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-30 13:38:27 +01:00
eSlider 8739d45b1a Merge pull request 'chore(release): v0.15.0 — files folder-ops + docpipe (#8)' (#9) from chore/release-v0.15.0#8 into main
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Test (Go 1.25) (push) Successful in 23s
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go stable) (push) Successful in 26s
2026-08-29 16:19:41 +01:00
eSlider f3150436fd chore(release): rerun CI после gitleaks fix (#8)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 23s
Tests / Test (Go stable) (pull_request) Successful in 25s
2026-08-29 16:18:42 +01:00
eSlider a30ae20f32 Merge pull request 'fix(ci): gitleaks через бинарник (Gitea runner) (#10)' (#11) from fix/gitleaks-gitea#10 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 6s
Tests / Test (Go 1.25) (push) Successful in 21s
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go stable) (push) Successful in 27s
Tests / Test (Go 1.25) (pull_request) Successful in 27s
Tests / Test (Go stable) (pull_request) Successful in 31s
2026-08-29 16:14:51 +01:00
eSlider 5a0b2ab412 fix(ci): gitleaks через бинарник на $GITHUB_WORKSPACE (Gitea runner) (#10)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 5s
Tests / Test (Go stable) (pull_request) Successful in 21s
Tests / Test (Go 1.25) (pull_request) Successful in 30s
2026-08-29 16:14:00 +01:00
eSlider 69844c9122 chore(release): v0.15.0 — files folder-ops + docpipe (#8)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Test (Go 1.25) (pull_request) Successful in 25s
Tests / Test (Go stable) (pull_request) Successful in 34s
Tests / Secret scan (gitleaks) (pull_request) Failing after 3s
2026-08-29 16:06:47 +01:00
eSliderandGitHub b04d610275 Merge pull request #42 from eSlider/feat/docs-optimize-gs
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Failing after 4s
Tests / Test (Go 1.25) (push) Successful in 20s
Tests / Test (Go stable) (push) Successful in 27s
feat(docs): Ghostscript PDF optimize for OO
2026-08-28 10:03:04 +01:00
eSliderandCursor 6b3f40e1a1 feat(docs): optimize PDF via Ghostscript pdfwrite
Add oo docs optimize for native-text PDFs (OO-friendly rewrite without
ocrmypdf overlay). Default ocr --force to false when text layer exists.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-28 10:02:58 +01:00
eSliderandGitHub ba8148a9e1 Merge pull request #41 from eSlider/fix/put-txt-fixed-width
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Failing after 4s
Tests / Test (Go 1.25) (push) Successful in 20s
Tests / Test (Go stable) (push) Successful in 27s
fix(docs): fixed-width txt as monospace code block
2026-08-28 09:59:59 +01:00
eSliderandCursor f1739dc9dc fix(docs): put-txt uses code block for fixed-width extracts
INE/pdftotext -layout txt (e.g. loan-ine) opens reliably in OO as
monospace; prose txt still uses hard line breaks.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-28 09:59:34 +01:00
eSliderandGitHub 5b812ce346 Merge pull request #40 from eSlider/feat/put-txt-line-breaks
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Failing after 4s
Tests / Test (Go 1.25) (push) Successful in 26s
Tests / Test (Go stable) (push) Successful in 27s
feat(docs): put-txt with preserved line breaks
2026-08-28 09:58:04 +01:00
eSliderandCursor 309ae44ee4 feat(docs): put-txt preserves line breaks in DOCX
TxtToMarkdown uses markdown hard breaks so pandoc keeps each source
line on its own row. Adds oo docs put-txt and projects files alias.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-28 09:57:55 +01:00
eSliderandGitHub ff2bc539f5 Merge pull request #39 from eSlider/fix/delete-immediately
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Failing after 5s
Tests / Test (Go 1.25) (push) Successful in 28s
Tests / Test (Go stable) (push) Successful in 29s
fix(files): permanent delete via DeleteDavItems
2026-08-28 09:56:21 +01:00
eSliderandCursor d9a7adcd94 fix(files): DeleteDavItems with Immediately true
Soft-delete (Immediately false) failed on some project files with
"You don't have enough permission to create" when moving to Trash.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-28 09:56:10 +01:00
eSliderandGitHub 3713ed6c51 Merge pull request #37 from eSlider/feat/files-dedupe
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Failing after 3s
Tests / Test (Go 1.25) (push) Successful in 26s
Tests / Test (Go stable) (push) Successful in 27s
feat(files): dedupe command + stem|ext matching
2026-08-27 23:40:16 +01:00
eSliderandCursor ac06d86363 feat(files): dedupe by stem|ext + oo projects files dedupe
- FileDedupKey groups true duplicates (same name+extension)
- put-md --replace deletes matching stem|ext only (not other formats)
- oo projects files dedupe PROJECT_ID [--apply] [--cross]
- Cross-folder mode prefers non-_trash folders, keeps newest file

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-27 23:40:07 +01:00
eSliderandGitHub 7d397ac680 Merge pull request #35 from eSlider/release-please--branches--main--components--go-onlyoffice
chore(main): release 0.14.0
2026-08-27 23:36:54 +01:00
github-actions[bot]andGitHub e52cb31bb7 chore(main): release 0.14.0 2026-08-27 22:36:44 +00:00
eSliderandGitHub a8cb4e9780 Merge pull request #36 from eSlider/fix/delete-files-dav
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Failing after 4s
Tests / Test (Go 1.25) (push) Successful in 1m29s
Tests / Test (Go stable) (push) Successful in 1m31s
fix(files): DeleteFiles actually removes files on produktor OO
2026-08-27 23:36:19 +01:00
eSliderandCursor 622dcdb7bf fix(files): DeleteFiles via per-file DELETE API
fileops/delete returns 200 on produktor.io OO without removing files;
route DeleteFiles through DeleteDavItems so put-md --replace and oo rm work.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-27 23:36:10 +01:00
eSliderandGitHub 2b267d36a8 Merge pull request #34 from eSlider/fix/put-md-upsert
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Failing after 4s
Tests / Test (Go stable) (push) Successful in 1m23s
Tests / Test (Go 1.25) (push) Successful in 1m31s
fix(docs): put-md upsert — no duplicate folder files
2026-08-27 23:29:09 +01:00
eSliderandCursor 50bd47570b fix(docs): put-md upsert by stem to avoid duplicate folder files
UploadToFolderReplacing deletes same-stem files before upload (--replace,
default true). OO may still display title+fileExst as double extension.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-27 23:28:11 +01:00
eSliderandGitHub 096a996726 Merge pull request #33 from eSlider/release-please--branches--main--components--go-onlyoffice
chore(main): release 0.13.0
2026-08-27 17:57:02 +01:00
github-actions[bot]andGitHub 625dd0a60a chore(main): release 0.13.0 2026-08-27 16:40:41 +00:00
eSliderandGitHub e531de280f Merge pull request #32 from eSlider/feat/docs-hocr
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Failing after 3s
Tests / Test (Go stable) (push) Successful in 1m43s
Tests / Test (Go 1.25) (push) Successful in 1m48s
feat(docs): hOCR → Markdown via go-hocr
2026-08-27 17:40:15 +01:00
eSliderandCursor 129e3a58cc chore: tidy go.sum after go-hocr bump
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-27 17:37:39 +01:00
eSliderandCursor 70e605b4ca docs(readme): document oo docs hocr
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-27 17:36:37 +01:00
eSliderandCursor a264cbd61c feat(docs): oo docs hocr — tesseract hOCR → go-hocr Markdown/YAML
Structured OCR path for agents alongside ocrmypdf searchable PDF.
as-md --hocr selects the same pipeline for OO downloads.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-27 17:36:20 +01:00
eSliderandGitHub a97a160c44 Merge pull request #29 from eSlider/release-please--branches--main--components--go-onlyoffice
chore(main): release 0.12.0
2026-08-27 17:05:08 +01:00
github-actions[bot]andGitHub df447f2673 chore(main): release 0.12.0 2026-08-27 15:59:02 +00:00
eSliderandGitHub db9ef12ff4 Merge pull request #31 from eSlider/feat/docs-md-ocr
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Failing after 4s
Tests / Test (Go 1.25) (push) Successful in 1m57s
Tests / Test (Go stable) (push) Successful in 2m1s
feat(docs): md↔docx, OCR→PDF, as-md/put-md
2026-08-27 16:58:36 +01:00
eSliderandCursor 2c0df0d55f fix(ci): point gitleaks at /github/workspace in docker action
The docker:// runner mounts the checkout at /github/workspace; using
${{ github.workspace }} (host path) made gitleaks fail with ENOENT.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-27 16:56:40 +01:00
eSliderandCursor 239c5ea672 docs(readme): document oo docs md↔docx / OCR agent workflow
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-27 16:49:18 +01:00
eSliderandCursor 6ea2fbadf7 feat(docs): md↔docx convert, OCR→PDF, as-md/put-md for agents
Add oo docs pipeline (pandoc/ocrmypdf) so agents keep Markdown locally while OnlyOffice stores versioned DOCX; OCR weak PDFs/images before returning MD. Also expose ListFolder/CreateFolder/MoveFiles/UploadToFolder.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-27 16:49:06 +01:00
eSlider 50d8974154 Merge pull request 'feat(security): secret-scan в CI + pre-push хуки (#142)' (#5) from feat/secret-scan#142 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 3s
Tests / Test (Go 1.25) (push) Successful in 44s
Tests / Test (Go stable) (push) Successful in 45s
2026-08-23 22:46:45 +01:00
eSlider 3fd3f797dc Merge pull request 'reconcile(github): sync GitHub-only commits into Gitea main (#144)' (#6) from reconcile/github-sync#144 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Test (Go 1.25) (push) Successful in 50s
Tests / Test (Go stable) (push) Successful in 44s
merge(github): sync GitHub-only commits into Gitea main (#144)
2026-08-23 19:55:48 +01:00
eSlider 0e299d0692 merge(github): bring GitHub-only commits into Gitea main (#144)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Test (Go 1.25) (pull_request) Successful in 48s
Tests / Test (Go stable) (pull_request) Successful in 50s
Merges GitHub main content (mail-send SendMail, files ops/WebDAV,
contact-indexes CRM, history whitelist) onto Gitea main so Gitea (source
of truth) contains every commit from both sides.
2026-08-23 19:55:39 +01:00
Andriy Oblivantsev 9ead554f5c feat(security): secret-scan via gitleaks in CI + pre-push/pre-commit hooks (#142)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 48s
Tests / Test (Go stable) (pull_request) Successful in 51s
2026-08-23 19:51:06 +01:00
eSlider 77c188c640 Merge origin/main (mail-send PRs, files ops) into local main
Release Please / Release Please (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Release / GoReleaser (push) Skipped
2026-08-21 22:03:04 +01:00
eSlider 2a59817f51 Merge feat/contact-indexes: crm indexes + history whitelist 2026-08-21 22:02:48 +01:00
eSlider 9c450a0352 feat(crm): contact email & person-opportunity indexes, history entity whitelist
- BuildContactEmailIndex: one-pass email→contact id map over all persons
  (O(N) instead of per-address FindPersonByEmail O(N×M))
- BuildPersonOpportunityIndex: person→lowest-id opportunity for history notes
  (OO has no person-level history; verified live)
- HistoryEntity{Opportunity,Case} whitelist constants
- IsCompany/ContactID/NumericIDLess/SortIDs helpers with unit tests
2026-08-21 22:02:47 +01:00
eSliderandGitHub ab7dfbd859 Merge pull request #28 from eSlider/feat/mail-send
feat(mail): oo mails send — SendMail client + guard empty-by-id
2026-08-21 17:21:18 +01:00
eSlider dda2bd3ca5 feat(mail): oo mails send — SendMail + guard empty-by-id send
- Client.SendMail: PUT /api/2.0/mail/messages/send.json, omits empty cc/bcc
- oo mails send subcommand with --id/--to/--subject/--body/--cc/--bcc
- guard: sending by --id without --body errors (send.json does not copy
  subject/body from the draft; prevents empty emails)
- unit test: empty cc/bcc omitted, correct path
2026-08-21 16:43:00 +01:00
eSliderandGitHub ceea6d909e Merge pull request #27 from eSlider/feat/mails-download-attachment-cli
oo CLI: mail attachment downloads
2026-08-21 15:25:08 +01:00
eSlider ecf34ba51a fix(files): rewrite viewUrl host to API base on download
resolveAPIURL now rewrites absolute viewUrl hosts to the configured portal
base so downloads stay on the internal network and keep the Authorization
header (a cross-host HTTP->HTTPS redirect would otherwise strip it, failing
large-file reads).
2026-08-20 10:48:38 +01:00
eSlider ebbd5d5373 feat(files): add WebDAV-oriented Files operations
Add value-oriented Files module operations for filesystem backends:
ListDavFolder, ListDavSections (@root), CreateDavFolder, RenameDavFolder/
RenameDavFile, MoveDavItems, CopyDavItems, DeleteDavItems, UploadDavFile
(streaming), DownloadDavFile. Dependency-free (net/http only).
2026-08-20 10:37:07 +01:00
eSlider 20a09530cd Merge pull request 'Add oo CLI support for mail attachment downloads' (#3) from feat/mails-download-attachment-cli into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Reviewed-on: #3
2026-08-19 12:17:10 +01:00
eSlider 666be883dc Add oo CLI support for mail attachment downloads.
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Test (Go 1.25) (pull_request) Successful in 24s
Tests / Test (Go stable) (pull_request) Successful in 26s
This exposes the canonical attachment download path via oo mails so downstream tools like 2dph can migrate off local OnlyOffice mail adapters without taking a direct module dependency first.
2026-08-19 12:10:21 +01:00
eSlider 65cd3f5c74 Merge pull request #2 from feat/mail-download-for-2dph
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Test (Go 1.25) (push) Successful in 41s
Tests / Test (Go stable) (push) Successful in 41s
Add mail attachment download support for 2dph integration.
2026-08-19 12:08:04 +01:00
eSlider 1ef0670624 Add mail attachment download support for 2dph integration.
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Test (Go 1.25) (pull_request) Successful in 50s
Tests / Test (Go stable) (pull_request) Successful in 51s
This gives go-onlyoffice a cookie-backed attachment download path so 2dph can reuse the canonical OnlyOffice client instead of keeping a local mail adapter.
2026-08-19 12:05:25 +01:00
eSliderandGitHub 10c1cfc6a7 ci: node16-safe actions + GitHub-only gates for Gitea mirror (#25)
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Test (Go 1.25) (push) Successful in 1m28s
Tests / Test (Go stable) (push) Successful in 1m23s
The Gitea act_runner runs action JS in node:16-bullseye; setup-go@v5 crashes
with 'ReferenceError: ReadableStream is not defined'. Gitea discovers
.github/workflows too, so:

- setup-go@v5 -> @v4 (node16-safe, fine on GitHub)
- gate GoReleaser (release.yml) + release-please jobs to
  github.server_url == https://github.com so the GitHub CD never runs on the
  Gitea mirror (GoReleaser/release-please are GitHub-release-centric)
2026-08-19 11:12:58 +01:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
8ced804a6f chore(main): release 0.11.0 (#24)
Release Please / Release Please (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Release / GoReleaser (push) Failing after 47s
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-08-19 08:52:13 +01:00
eSlider 256864f613 feat(catalog): port oo catalog (scan/merge/match/apply) from legacy branch
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Failing after 2s
Tests / Test (Go 1.25) (push) Successful in 1m22s
Tests / Test (Go stable) (push) Successful in 1m23s
The catalog feature (approved ASR-0014) was lost during the GitHub history
rewrite: the merge commit on main is empty and the code only survived on the
old gitea-history branches (no merge base with main). Port catalog/* and the
oo catalog CLI verbatim; drop the library-only applications package whose
Sync() created the 'Write application'/'Apply application'/'Send CV' CRM
task spam (23k+ tasks deleted 2026-08-18).
2026-08-18 12:41:58 +01:00
eSliderandGitHub a3a6ad3543 Merge pull request #23 from eSlider/release-please--branches--main--components--go-onlyoffice
Release Please / Release Please (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Release / GoReleaser (push) Failing after 46s
chore(main): release 0.10.0
2026-08-13 13:13:52 +01:00
github-actions[bot]andGitHub 0161a92c52 chore(main): release 0.10.0 2026-08-13 09:21:46 +00:00
eSlider 0e80a1a058 Add milestone create and task start dates so Gantt bars can exist.
CLI: oo projects milestone-create, oo tasks create --start --milestone.
2026-08-13 10:21:15 +01:00
eSlider 8b27539145 chore: restore LICENSE copyright holder
History rewrite anonymized author strings; keep accurate MIT copyright on tip.
2026-08-10 00:14:08 +01:00
eSlider a0add3b771 chore: anonymize remaining CLI examples and hostnames
Keep public docs/help free of client names and fixed portal URLs.
2026-08-10 00:13:52 +01:00
eSlider b58940fcf2 chore: redact personal ops from public tree
Move catalog, dossier sync, and SearXNG to private oo-workspace; keep
generic CRM name/title helpers and anonymized docs/examples.
2026-08-10 00:12:54 +01:00
eSlider 11aa23f1a1 docs(crm): Team vs Contacts and persons update note 2026-08-09 14:28:17 +01:00
eSlider 42c31d4b48 fix(crm): JSON person update and oo persons update CLI
Form-encoded PUT ignores companyId/about and can unlink the employer;
use JSON and re-fetch. Expose update plus contact-tag helpers on oo.
2026-08-09 14:28:09 +01:00
eSliderandGitHub f95830961d Merge pull request #20 from eSlider/feat/crm-invoices
feat(crm): invoices CLI and opportunity update
2026-08-06 14:58:30 +01:00
eSlider 6b5a82bef8 docs(crm): clarify mail API send vs signature for chat links 2026-08-06 14:52:35 +01:00
eSlider 490426c277 docs(crm): capture association graph and invoice/mail quirks
Document Medex-learned OO rules (one company, entityId at create, PDF
cache/cleanup, Matrix # truncation) and add invoices pdf/status helpers.
2026-08-06 14:26:43 +01:00
eSlider 31c556d34c feat(crm): allow updating invoice terms via oo
Needed to put issuer NIE in Bedingungen without a full recreate.
2026-08-06 13:35:08 +01:00
eSlider 7f35b0e910 fix(mail): German spacing for invoice draft template
Separate paragraphs with blank lines; mailbox signature uses Synapse MXID.
2026-08-06 13:25:53 +01:00
eSlider 73b1050a83 feat(mail): draft, attach, and draft-invoice CLI
Save OO Mail drafts via body field, attach Files documents, and one-shot
invoice PDF regenerate + draft for client billing emails.
2026-08-06 13:18:02 +01:00
eSlider d81e7de911 feat(crm): invoice update notes and PO fields
Allow oo invoices update --description/--po so Notizen can be
multiline refs without portal URLs; keep opportunity link support.
2026-08-06 12:47:06 +01:00
eSlider 3201a58df7 fix(crm): link invoices to opportunities
Create/update accept entityId (opportunity); oo invoices create
--opportunity and invoices update --opportunity so deal #invoices
tabs show the draft.
2026-08-06 12:45:09 +01:00
eSlider 19a7e9cbf8 feat(crm): add invoices CLI and opportunity update
Expose CRM invoice list/get/create/delete and catalog items via oo,
plus contact address/about helpers and opportunity bid/stage update.
Needed for Medex VM billing and other draft invoices.
2026-08-06 12:32:20 +01:00
139 changed files with 20047 additions and 350 deletions
+47 -1
View File
@@ -11,7 +11,7 @@ ONLYOFFICE_PASS=
# ONLYOFFICE_NAME=
# ONLYOFFICE_PASSWORD=
#
# produktor.io operator aliases (CLI-only):
# Optional CLI-only aliases:
# OO_URL=
# OO_USER=
# OO_PASS=
@@ -22,6 +22,52 @@ ONLYOFFICE_PROJECT_ID=33
# oo mails uses ONLYOFFICE_URL/USER/PASS above (Workspace Mail addon).
# deploy/docker-compose.rclone-webdav.yml — rclone FUSE mount of the oo-webdav
# sidecar. Reuses ONLYOFFICE_USER and accepts ONLYOFFICE_PASSWORD (alias
# ONLYOFFICE_PASS). Set ONLYOFFICE_WEBDAV_URL only if the sidecar is not on
# the default docker bridge address. See docs/rclone-webdav.md.
# ONLYOFFICE_WEBDAV_URL=http://172.17.0.1:8098/webdav
# cmd/office TUI — optional Document Server for DOCX→HTML preview:
# ONLYOFFICE_DOCS_URL=https://docs.example.com
# ONLYOFFICE_DOCS_SECRET=
# MinIO download fallback for the portal's stale AWS S3 consumer (older
# Documents folders). When the portal redirects to amazonaws.com with access
# key "minio" (403 InvalidAccessKeyId), files are fetched from the configured
# MinIO store instead. Without endpoint + key/secret the fallback is disabled.
# MINIO_ENDPOINT=http://minio.example.com:9000
# MINIO_BUCKET=office
# MINIO_ACCESS_KEY=
# MINIO_SECRET_KEY=
# Catalog scan classification rules (client mail domains, project remotes/names).
# Copy catalog/classify.example.yaml and point this at it; no rules means
# nothing is classified as work.
# OO_CATALOG_CONFIG=~/.config/oo/catalog-classify.yaml
# oo search — direct Elasticsearch access for name + content search. ES lives
# inside the OnlyOffice VM on localhost:9200; expose it with an SSH tunnel
# (see docs/elasticsearch.md). ONLYOFFICE_ES_INDEX defaults to files_file.
# ONLYOFFICE_ES_URL=http://127.0.0.1:9200
# ONLYOFFICE_ES_INDEX=files_file
# ONLYOFFICE_TENANT=
# Read-only SQL file store over the Community Server database (see
# docs/community-server-db.md). The live portal runs MySQL; ONLYOFFICE_DSN is
# `user:pass@tcp(host:port)/onlyoffice?parseTime=true`, or a `postgres://` URL.
# ONLYOFFICE_DSN=
# ONLYOFFICE_PG_DRIVER= # postgres | mysql (auto-detected from DSN)
# ONLYOFFICE_PG_TENANT=1 # falls back to ONLYOFFICE_TENANT
# Alternatively build a PostgreSQL DSN from parts:
# ONLYOFFICE_PG_HOST=
# ONLYOFFICE_PG_PORT=5432
# ONLYOFFICE_PG_USER=
# ONLYOFFICE_PG_PASSWORD=
# ONLYOFFICE_PG_DBNAME=onlyoffice
# ONLYOFFICE_PG_SSLMODE=disable
# oo index / oo search --backend own — own full-text index for PDF/scans,
# filled by `oo index` from internal/docpipe (pdftotext + OCR). Defaults to
# oo_docs_text. Uses the same ONLYOFFICE_ES_URL.
# ONLYOFFICE_ES_TEXT_INDEX=oo_docs_text
+6
View File
@@ -0,0 +1,6 @@
github: eSlider
ko_fi: eslider
liberapay: eslider
patreon: eslider
custom:
- https://polar.sh/eslider
-41
View File
@@ -1,41 +0,0 @@
name: Release Please
on:
push:
branches: [main]
workflow_dispatch:
permissions:
contents: write
pull-requests: write
issues: write
actions: write
jobs:
release-please:
name: Release Please
runs-on: ubuntu-latest
steps:
- name: Run release-please
uses: googleapis/release-please-action@v4
id: release
with:
config-file: release-please-config.json
manifest-file: .release-please-manifest.json
token: ${{ secrets.RELEASE_PLEASE_TOKEN != '' && secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
# Tags/releases created with GITHUB_TOKEN do not trigger other workflows
# (push:tags on Release.yml never fires). Dispatch GoReleaser explicitly.
- name: Trigger GoReleaser
if: ${{ steps.release.outputs.release_created == 'true' }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG: ${{ steps.release.outputs.tag_name }}
run: |
set -euo pipefail
echo "Dispatching Release workflow for $TAG"
gh workflow run release.yml --repo "${{ github.repository }}" -f "tag=${TAG}"
outputs:
release_created: ${{ steps.release.outputs.release_created }}
tag_name: ${{ steps.release.outputs.tag_name }}
+3 -2
View File
@@ -16,11 +16,12 @@ permissions:
jobs:
goreleaser:
name: GoReleaser
if: github.server_url == 'https://github.com'
runs-on: ubuntu-latest
steps:
# Always clone default branch first. workflow_dispatch often races with
# release-please creating the tag; fetching refs/tags/X before it exists
# fails checkout. Retry fetch, then check out the tag.
# tag creation; fetching refs/tags/X before it exists fails checkout.
# Retry fetch, then check out the tag.
- uses: actions/checkout@v4
with:
fetch-depth: 0
+46 -1
View File
@@ -10,6 +10,51 @@ permissions:
contents: read
jobs:
secret-scan:
name: Secret scan (gitleaks)
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Compute scan range (diff of new commits only)
id: range
run: |
if [ "$GITHUB_EVENT_NAME" = "pull_request" ]; then
RANGE="${{ github.event.pull_request.base.sha }}...${{ github.event.pull_request.head.sha }}"
else
BEFORE="${{ github.event.before }}"
if [ "$BEFORE" = "0000000000000000000000000000000000000000" ]; then
RANGE="$(git rev-list --max-parents=0 HEAD | tail -1)..$GITHUB_SHA"
else
RANGE="$BEFORE..$GITHUB_SHA"
fi
fi
echo "RANGE=$RANGE" >> "$GITHUB_ENV"
echo "Scanning range: $RANGE"
# Install the gitleaks binary instead of a docker action: the
# docker://zricethezav/gitleaks action hardcodes /github/workspace,
# which does not exist on the Gitea (act) runner. $GITHUB_WORKSPACE is
# the checkout dir on BOTH runners (GitHub and Gitea act). Mirrors the
# fix applied to 2dph (issue #142).
- name: Gitleaks (diff-only, fail on leak)
env:
GITLEAKS_RANGE: ${{ env.RANGE }}
run: |
set -euo pipefail
curl -fsSLo /tmp/gitleaks.tar.gz \
https://github.com/gitleaks/gitleaks/releases/download/v8.30.1/gitleaks_8.30.1_linux_x64.tar.gz
tar -xzf /tmp/gitleaks.tar.gz -C /tmp gitleaks
chmod +x /tmp/gitleaks
/tmp/gitleaks detect \
--source "$GITHUB_WORKSPACE" \
--log-opts="$GITLEAKS_RANGE" \
--redact \
--verbose
test:
name: Test (Go ${{ matrix.go }})
runs-on: ubuntu-latest
@@ -22,7 +67,7 @@ jobs:
- uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
uses: actions/setup-go@v4
with:
go-version: ${{ matrix.go }}
cache: true
-3
View File
@@ -1,3 +0,0 @@
{
".": "0.9.0"
}
+16 -8
View File
@@ -9,18 +9,19 @@ Canonical Go client for OnlyOffice Workspace (Projects + Calendar + CRM) and the
- `request.go` — `Request`, `Query`, `Time`, `Token`, `MetaResponse`, `Permissions`.
- `auth.go` — `Authenticate`, `AuthenticateContext`, `InvalidateToken`, `Auth`, token lifecycle.
- `http.go` — transport + DRY response decoders (`ResponseArray`/`ResponseObject`/`postFormObject`/`putFormObject`/`deleteObject`).
- `projects.go`, `tasks.go`, `users.go`, `calendar.go`, `crm.go`, `files.go`, `mails.go` — typed / untyped domain methods. **`files.go`** — CRM opportunity upload plus **project/task Documents** (`GetProjectFiles`, `UploadProjectFile`, `GetTaskFiles`, `AttachFilesToTask`, `UploadTaskFile`, `DetachTaskFile`, `GetFile`, `RenameFile`, `DeleteFiles`, `DownloadFile`). **`mails.go`** — OnlyOffice Workspace Mail addon (`ListMailAccounts`, `ListMailFolders`, `ListMailMessages`, `GetMailMessage`, `RemoveMailMessages`).
- `projects.go`, `tasks.go`, `users.go`, `calendar.go`, `crm.go`, `files.go`, `files_webdav.go`, `files_stem.go`, `retry.go`, `mails.go`, `invoices.go` — typed / untyped domain methods. **`files.go`** — CRM opportunity upload plus **project/task Documents** (`UpdateFile`, `UploadToFolderReplacing`). **`files_webdav.go`** — Documents module by id (`ListDavFolder`, `MoveDavItems`/`CopyDavItems` with per-operation error surfacing, `ListFileOps`). **`retry.go`** — `DoRetry`: deterministic exponential backoff (no jitter) on 429/502/503/504; every bulk tool routes API calls through it, and the HTTP transport + auth (`retryRaw`, `AuthenticateContext`) retry transient answers centrally. `ratelimit.go` adds a process-wide token bucket (`OO_RATE_LIMIT`/`OO_BURST`) and a shared 429 cooldown gate, installed via `pacedTransport` in `NewClient`; `Retry-After` is parsed into `*TransientError` and honoured. See [`docs/rate-limiting.md`](docs/rate-limiting.md). **`mails.go`** — OnlyOffice Workspace Mail. **`invoices.go`** — CRM invoices, PDF regen/cleanup, status. Association rules live with the private `oo-workspace` tooling.
- **Unified file client (epic #34) — `filestore_core.go`, `filestore_rest.go`, `filestore_dav.go`, `filestore_pg.go`, `filestore_es.go`, `filestore_es_text.go`, `filestore_text_index.go`, `filestore_facade.go`.** `filestore_core.go` — model (`Entry`, `Kind`) + `FileStore`/`Searcher`; `filestore_rest.go`/`filestore_dav.go` — REST/WebDAV adapters; `filestore_pg.go` — **read-only** SQL store (PostgreSQL/MySQL, `ErrReadOnly` on writes); `filestore_es.go` — OnlyOffice Elasticsearch searcher; `filestore_es_text.go`/`filestore_text_index.go` — own PDF/scan index (`oo_docs_text`, PDF attachments via pdfdetach); `filestore_facade.go` — `FileClient` with read/write/search order and transient fallback. Use `c.Files()` (facade), `c.FileStore("rest"|"dav"|"pg"|"sql")` or `c.SQLFileStore()`; contract and how to add a backend: [`docs/unified-file-client.md`](docs/unified-file-client.md).
- Pure stdlib + `google/go-querystring`; no UI, no dotenv.
- **CLI — `cmd/oo/` as `package main`.** Cobra wrapper that loads `.env` via `godotenv` at startup. **Subject-based command tree** mirroring [`tea`](https://gitea.com/gitea/tea):
- `main.go` — entry point (docstring lists the command tree).
- `common.go` — `rootCmd`, `newOO`, `printTable`/`printObject`, `--output table|json` flag.
- `calendar.go`, `projects.go`, `projects_files.go`, `tasks.go`, `tasks_files.go`, `users.go`, `contacts.go`, `opportunities.go`, `cases.go`, `crm_tasks.go`, `apps.go`, `catalog.go` — one file per subject (or per subject facet), each registers in `init()`.
- `calendar.go`, `projects.go`, `projects_files.go`, `tasks.go`, `tasks_files.go`, `users.go`, `contacts.go`, `opportunities.go`, `cases.go`, `crm.go`, `crm_tasks.go`, `catalog.go`, `docs.go`, `dav.go`, `search.go`, `index.go`, `mails.go`, `invoices.go` — one file per subject (or per subject facet), each registers in `init()`. `dav.go` exposes the Documents module by id (`oo dav ls|move|copy|mkdir|rename-file|rename-folder|download|fileops`); `search.go` runs `oo search QUERY` (name/content, `--backend oo|own`); `index.go` fills the own full-text index (`oo index folder|files`, see [`docs/unified-file-client.md`](docs/unified-file-client.md)).
- CLI-only deps (`spf13/cobra`, `joho/godotenv`) stay out of the library.
- **Catalog inventory — `catalog/` package.** Filesystem VCF/project scan → YAML → match/apply against CRM. Used by `oo catalog`; not part of the flat `*Client` surface (uses Client as a dependency).
- **TUI — `cmd/office/` as `package main`.** Bubble Tea three-pane browser (module tree, selectable list, markdown preview). Reuses `cmd/internal/bootstrap` for env/auth and the root `onlyoffice` library for all API calls. UI logic in `cmd/office/ui/`; preview/formatting in `cmd/office/preview/`; list loaders in `cmd/office/fetch/`.
- **List table (`DataTable`)** — `cmd/office/ui/table*.go`. Column layout policies live in `cmd/office/model/table_layout.go` (`TableFlexLayoutFor`); cell rendering uses the bubbles/table inline pattern in `table_render.go` (`renderTableCell`, `padANSIWidth`). See `.cursor/skills/office-tui-table/SKILL.md` before changing center-pane tables.
- **Shared bootstrap — `cmd/internal/bootstrap/`.** `LoadEnv()` + `NewClient(ctx)` extracted from `oo`; both binaries import it.
- **Applications sync — `cmd/oo/applications/`.** README→CRM bridge, CV-specific; kept under `cmd/oo/` so it's clear it's internal to the binary, not a library feature.
- **Bulk Documents tools live in the private `oo-workspace` repo** (`ooscan`, `pdfamount`, `kontoblatt`, `kontolink`), not in this public tree. They use the public client and its `DoRetry` pacing.
- **Personal ops tooling** (disk inventory, dossier→CRM sync, SearXNG) lives in a private companion repo `eSlider/oo-workspace` (the `oow` CLI), not in this public tree.
## Rules
@@ -28,9 +29,15 @@ Canonical Go client for OnlyOffice Workspace (Projects + Calendar + CRM) and the
- New endpoints go into the library first; CLI commands are thin wrappers.
- Prefer `ResponseObject` / `postFormObject` / `putFormObject` / `deleteObject` over hand-rolled `json.Unmarshal(responseField(...))` blocks — they exist for DRY, use them.
- Domain split is by file, **not** by subpackage. Don't introduce `internal/` or `pkg/*` subpackages inside the library — it flattens the `*Client` call surface for a reason.
- CLI commands follow **subject → verb** structure (`oo <subject> <verb>`), never `oo <verb>-<subject>`. Add new commands to the existing subject file if one fits; create a new `cmd/oo/<subject>.go` for a genuinely new domain.
- CLI commands follow **subject → verb** structure (`oo <subject> <verb>`), never `oo <verb>-<subject>`. Add new commands to the existing subject file if one fits; create a new `cmd/oo/<subject>.go` for a genuinely new domain. The subject→verb tree in `cmd/oo/main.go` and the README table are documentation — update them with the code.
- **Documents for agents:** prefer Markdown in git; OnlyOffice UI is weak for `.md`/`.txt`. Use `oo docs put-md` (md→docx) and `oo docs put-txt` (txt→docx, preserves line breaks). All upload paths default to **upsert** by `stem|ext` (`--replace`, default true); `--no-replace` fails on conflict; `--allow-duplicate` opts into raw OO append. `oo projects files dedupe PROJECT_ID` reports/removes duplicate stem|ext copies (`--apply`, `--cross`; includes project root folder).
- Every table output goes through `printTable(headers, rows)`; every single-object through `printObject(v)`. Do not `fmt.Println` rows ad-hoc or the `--output json` flag breaks for that command.
- No secrets in the repo; use `.env` (gitignored). Commit `.env.example` only.
- **No host/client specifics in the tree.** Endpoints, IPs/ports, client mail
domains, project remotes/names and personal names stay out of source and
fixtures — they come from env/config (`MINIO_*`, `OO_CATALOG_CONFIG`, see
[`catalog/classify.example.yaml`](catalog/classify.example.yaml)). This repo is
mirrored to GitHub as a public showroom, so the tree must stay project-generic.
- Follow SemVer on tags; this repo is tagged at GitHub under `git@github.com:eSlider/go-onlyoffice.git`.
### Testing policy (2026-04-24)
@@ -54,6 +61,7 @@ write `mux.HandleFunc("/api/2.0/...")` to emulate OnlyOffice, we write an
## Related
- [`eSlider/inventar`](https://git.produktor.io/eSlider/inventar) — ASR/ADR (see ASR-0008 Go library module conventions).
- [`eSlider/inventar-sync`](https://git.produktor.io/eSlider/inventar-sync) — OnlyOffice → Gitea issue sync, consumes this library.
- [`produktor.io/vidarr`](https://git.produktor.io/produktor.io/vidarr) — legacy consumer being migrated from `pkg/onlyoffice` to this module.
- [`docs/README.md`](docs/README.md) — reference index (file client, ES, SQL, rclone).
- `eSlider/inventar` — ASR/ADR (see ASR-0008 Go library module conventions).
- `eSlider/inventar-sync` — OnlyOffice → Gitea issue sync, consumes this library.
- `vidarr` — legacy consumer being migrated from `pkg/onlyoffice` to this module.
+197
View File
@@ -4,6 +4,203 @@ All notable changes to this project are documented here. The format is based on
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project
adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.21.0](https://git.produktor.io/eSlider/go-onlyoffice/compare/v0.20.0...v0.21.0) (2026-09-23)
### Features
* **retry:** global token-bucket rate limit (`OO_RATE_LIMIT`/`OO_BURST`), typed
`TransientError` with `Retry-After`, exponential backoff
(`OO_RETRY_ATTEMPTS`/`_BASE`/`_MAX`) and a process-wide 429 cooldown gate.
All HTTP paths are paced via `pacedTransport` (#70).
* **oo:** native document conversion (`oo docs pdf`), deep links, sheet-aware
export (`oo docs csv`/`json`), `oo docs presigned`, plus project team and
user lifecycle, milestone-delete and file update/replace (#73, #75).
* **catalog:** upstream generic workspace tooling (board-sync, CRM audit,
catalog names) (#76).
### Chores
* move business tooling (`ooscan`, `pdfamount`, `kontoblatt`, `kontolink`) out
of the public library into the private `oo-workspace` repo; tidy filestore
naming (#77).
* keep host/client specifics out of the tree (#74).
## [0.18.0](https://github.com/eSlider/go-onlyoffice/compare/v0.17.0...v0.18.0) (2026-09-04)
### Features
* **crm:** add UpdateContactName and CloseCRMTask helpers ([4d8af7a](https://github.com/eSlider/go-onlyoffice/commit/4d8af7a2fef1fcfd96ad913cefc6497e3389963b))
### Bug Fixes
* **crm:** deterministic sortBy=id in contact paged lists ([4d91726](https://github.com/eSlider/go-onlyoffice/commit/4d917261792203732ac739644c9c82124b2166eb))
### Documentation
* **funding:** eSlider support links (reverse-import GitHub e9c969a) ([d650a16](https://github.com/eSlider/go-onlyoffice/commit/d650a16a36037949505eb017c92bd3283e6e2f0f))
## [0.17.0](https://github.com/eSlider/go-onlyoffice/compare/v0.16.0...v0.17.0) (2026-08-31)
### Features
* **mailsync:** FetchMailFolder — integration-layer walk for ETL consumers ([35f0cb8](https://github.com/eSlider/go-onlyoffice/commit/35f0cb8d20076244141065c07e203e633bc3612a))
### Bug Fixes
* **files:** upsert uploads by default and dedupe project root ([24ca144](https://github.com/eSlider/go-onlyoffice/commit/24ca144b22abc5d056a5fd1ed9a1887f26a79d15))
## [0.16.0](https://github.com/eSlider/go-onlyoffice/compare/v0.15.0...v0.16.0) (2026-08-30)
### Features
* **docs:** `put-xlsx` — multi-sheet бюджеты с named inputs, SUM/AVG/MIN
формулами, cross-sheet ссылками и cell comments (`internal/xlspipe`,
excelize) ([68445b0](https://github.com/eSlider/go-onlyoffice/commit/68445b0))
### Added
* **docs:** CRM association graph and OO quirks (`docs/crm-associations.md`)
* **crm:** `ForceRegenerateInvoicePDF`, `SetInvoiceStatus`, `PurgeStaleInvoicePDFs`, contact/opportunity file list helpers
* **oo:** `invoices pdf`, `pdf-cleanup`, `status`; create `--consignee`; draft-invoice force-regens PDF
### Fixed
* Document that invoice→deal must be set at create (`update --opportunity` often HTTP 400)
## [0.15.0](https://github.com/eSlider/go-onlyoffice/compare/v0.14.0...v0.15.0) (2026-08-29)
### Features
* **files:** `ListFolder`, `CreateFolder`, `MoveFiles`, `UploadToFolder` — Documents folder helpers for OO Documents ingestion ([6ea2fba](https://github.com/eSlider/go-onlyoffice/commit/6ea2fba))
* **files:** dedupe by stem|ext (`files_stem.go`) + `oo projects files dedupe` ([ac06d86](https://github.com/eSlider/go-onlyoffice/commit/ac06d86))
* **docs:** `internal/docpipe` — md↔docx convert, OCR→PDF, hOCR→Markdown via go-hocr, as-md/put-md for agents ([6ea2fba](https://github.com/eSlider/go-onlyoffice/commit/6ea2fba), [a264cbd](https://github.com/eSlider/go-onlyoffice/commit/a264cbd))
* **docs:** optimize PDF via Ghostscript pdfwrite ([6b3f40e](https://github.com/eSlider/go-onlyoffice/commit/6b3f40e))
* **security:** gitleaks secret-scan in CI + pre-push/pre-commit hooks ([9ead554](https://github.com/eSlider/go-onlyoffice/commit/9ead554))
### Fixes
* **files:** `DeleteFiles` via per-file DELETE API; `DeleteDavItems` with `Immediately` true ([622dcdb](https://github.com/eSlider/go-onlyoffice/commit/622dcdb), [d9a7adc](https://github.com/eSlider/go-onlyoffice/commit/d9a7adc))
* **docs:** put-txt preserves line breaks in DOCX; fixed-width extracts in code block; put-md upsert by stem ([309ae44](https://github.com/eSlider/go-onlyoffice/commit/309ae44), [f1739dc](https://github.com/eSlider/go-onlyoffice/commit/f1739dc), [50bd475](https://github.com/eSlider/go-onlyoffice/commit/50bd475))
* **ci:** point gitleaks at `/github/workspace` in docker action ([2c0df0d](https://github.com/eSlider/go-onlyoffice/commit/2c0df0d))
## [0.14.0](https://github.com/eSlider/go-onlyoffice/compare/v0.13.0...v0.14.0) (2026-08-27)
### Features
* **crm:** contact email & person-opportunity indexes, history entity whitelist ([9c450a0](https://github.com/eSlider/go-onlyoffice/commit/9c450a0352c25f12ac16d91062744ab026ffe660))
* **docs:** hOCR → Markdown via go-hocr ([e531de2](https://github.com/eSlider/go-onlyoffice/commit/e531de280f2c521a7411f75212803649b177db58))
* **docs:** md↔docx convert, OCR→PDF, as-md/put-md for agents ([6ea2fba](https://github.com/eSlider/go-onlyoffice/commit/6ea2fbadf768d7d85f6c7a49a9bc1050e642bc56))
* **docs:** md↔docx, OCR→PDF, as-md/put-md ([db9ef12](https://github.com/eSlider/go-onlyoffice/commit/db9ef12ff4096f8aadb557ca128b6edcb503028c))
* **docs:** oo docs hocr — tesseract hOCR → go-hocr Markdown/YAML ([a264cbd](https://github.com/eSlider/go-onlyoffice/commit/a264cbd61c12be9098c8d16e7c1c1253eb460816))
* **mail:** oo mails send — SendMail + guard empty-by-id send ([dda2bd3](https://github.com/eSlider/go-onlyoffice/commit/dda2bd3ca5cea6ebb89e1ac02b785b9838727bda))
* **mail:** oo mails send — SendMail client + guard empty-by-id ([ab7dfbd](https://github.com/eSlider/go-onlyoffice/commit/ab7dfbd8594de5724286e2b460e789f4acb114bc))
* **security:** secret-scan via gitleaks in CI + pre-push/pre-commit hooks ([#142](https://github.com/eSlider/go-onlyoffice/issues/142)) ([9ead554](https://github.com/eSlider/go-onlyoffice/commit/9ead554f5cc229687d3a27934a023fb1dddaef15))
### Bug Fixes
* **ci:** point gitleaks at /github/workspace in docker action ([2c0df0d](https://github.com/eSlider/go-onlyoffice/commit/2c0df0d55fb03555b60481695554d75dceffce48))
* **docs:** put-md upsert — no duplicate folder files ([2b267d3](https://github.com/eSlider/go-onlyoffice/commit/2b267d36a8086b14a94706468f7e92e9179e62d4))
* **docs:** put-md upsert by stem to avoid duplicate folder files ([50bd475](https://github.com/eSlider/go-onlyoffice/commit/50bd47570b6430cebde43641a7b1d2f8d54d32f2))
* **files:** DeleteFiles actually removes files on produktor OO ([a8cb4e9](https://github.com/eSlider/go-onlyoffice/commit/a8cb4e97805226e8af694ec6a64ee3ac6a5e9fdd))
* **files:** DeleteFiles via per-file DELETE API ([622dcdb](https://github.com/eSlider/go-onlyoffice/commit/622dcdb7bffd49acbea5ef07f5d5d009f44b1358))
### Documentation
* **readme:** document oo docs hocr ([70e605b](https://github.com/eSlider/go-onlyoffice/commit/70e605b4ca8ee005e4351b2fc2d2eec081604998))
* **readme:** document oo docs md↔docx / OCR agent workflow ([239c5ea](https://github.com/eSlider/go-onlyoffice/commit/239c5ea67201e62de6ba067f0e7963c7c9bda188))
## [0.13.0](https://github.com/eSlider/go-onlyoffice/compare/v0.12.0...v0.13.0) (2026-08-27)
### Features
* **crm:** contact email & person-opportunity indexes, history entity whitelist ([9c450a0](https://github.com/eSlider/go-onlyoffice/commit/9c450a0352c25f12ac16d91062744ab026ffe660))
* **docs:** hOCR → Markdown via go-hocr ([e531de2](https://github.com/eSlider/go-onlyoffice/commit/e531de280f2c521a7411f75212803649b177db58))
* **docs:** md↔docx convert, OCR→PDF, as-md/put-md for agents ([6ea2fba](https://github.com/eSlider/go-onlyoffice/commit/6ea2fbadf768d7d85f6c7a49a9bc1050e642bc56))
* **docs:** md↔docx, OCR→PDF, as-md/put-md ([db9ef12](https://github.com/eSlider/go-onlyoffice/commit/db9ef12ff4096f8aadb557ca128b6edcb503028c))
* **docs:** oo docs hocr — tesseract hOCR → go-hocr Markdown/YAML ([a264cbd](https://github.com/eSlider/go-onlyoffice/commit/a264cbd61c12be9098c8d16e7c1c1253eb460816))
* **mail:** oo mails send — SendMail + guard empty-by-id send ([dda2bd3](https://github.com/eSlider/go-onlyoffice/commit/dda2bd3ca5cea6ebb89e1ac02b785b9838727bda))
* **mail:** oo mails send — SendMail client + guard empty-by-id ([ab7dfbd](https://github.com/eSlider/go-onlyoffice/commit/ab7dfbd8594de5724286e2b460e789f4acb114bc))
* **security:** secret-scan via gitleaks in CI + pre-push/pre-commit hooks ([#142](https://github.com/eSlider/go-onlyoffice/issues/142)) ([9ead554](https://github.com/eSlider/go-onlyoffice/commit/9ead554f5cc229687d3a27934a023fb1dddaef15))
### Bug Fixes
* **ci:** point gitleaks at /github/workspace in docker action ([2c0df0d](https://github.com/eSlider/go-onlyoffice/commit/2c0df0d55fb03555b60481695554d75dceffce48))
* **files:** rewrite viewUrl host to API base on download ([ecf34ba](https://github.com/eSlider/go-onlyoffice/commit/ecf34ba51aae77f1626a60795f7329f25d9344e5))
### Documentation
* **readme:** document oo docs hocr ([70e605b](https://github.com/eSlider/go-onlyoffice/commit/70e605b4ca8ee005e4351b2fc2d2eec081604998))
* **readme:** document oo docs md↔docx / OCR agent workflow ([239c5ea](https://github.com/eSlider/go-onlyoffice/commit/239c5ea67201e62de6ba067f0e7963c7c9bda188))
## [0.12.0](https://github.com/eSlider/go-onlyoffice/compare/v0.11.0...v0.12.0) (2026-08-27)
### Features
* **crm:** contact email & person-opportunity indexes, history entity whitelist ([9c450a0](https://github.com/eSlider/go-onlyoffice/commit/9c450a0352c25f12ac16d91062744ab026ffe660))
* **docs:** md↔docx convert, OCR→PDF, as-md/put-md for agents ([6ea2fba](https://github.com/eSlider/go-onlyoffice/commit/6ea2fbadf768d7d85f6c7a49a9bc1050e642bc56))
* **docs:** md↔docx, OCR→PDF, as-md/put-md ([db9ef12](https://github.com/eSlider/go-onlyoffice/commit/db9ef12ff4096f8aadb557ca128b6edcb503028c))
* **files:** add WebDAV-oriented Files operations ([ebbd5d5](https://github.com/eSlider/go-onlyoffice/commit/ebbd5d5373abfeca5f160312f201cbd683f55e42))
* **mail:** oo mails send — SendMail + guard empty-by-id send ([dda2bd3](https://github.com/eSlider/go-onlyoffice/commit/dda2bd3ca5cea6ebb89e1ac02b785b9838727bda))
* **mail:** oo mails send — SendMail client + guard empty-by-id ([ab7dfbd](https://github.com/eSlider/go-onlyoffice/commit/ab7dfbd8594de5724286e2b460e789f4acb114bc))
* **security:** secret-scan via gitleaks in CI + pre-push/pre-commit hooks ([#142](https://github.com/eSlider/go-onlyoffice/issues/142)) ([9ead554](https://github.com/eSlider/go-onlyoffice/commit/9ead554f5cc229687d3a27934a023fb1dddaef15))
### Bug Fixes
* **ci:** point gitleaks at /github/workspace in docker action ([2c0df0d](https://github.com/eSlider/go-onlyoffice/commit/2c0df0d55fb03555b60481695554d75dceffce48))
* **files:** rewrite viewUrl host to API base on download ([ecf34ba](https://github.com/eSlider/go-onlyoffice/commit/ecf34ba51aae77f1626a60795f7329f25d9344e5))
### Documentation
* **readme:** document oo docs md↔docx / OCR agent workflow ([239c5ea](https://github.com/eSlider/go-onlyoffice/commit/239c5ea67201e62de6ba067f0e7963c7c9bda188))
## [0.11.0](https://github.com/eSlider/go-onlyoffice/compare/v0.10.0...v0.11.0) (2026-08-18)
### Features
* **catalog:** port oo catalog (scan/merge/match/apply) from legacy branch ([256864f](https://github.com/eSlider/go-onlyoffice/commit/256864f613f31570cf995dddc51255a9a91a38f8))
## [0.10.0](https://github.com/eSlider/go-onlyoffice/compare/v0.9.0...v0.10.0) (2026-08-13)
### Features
* **crm:** add invoices CLI and opportunity update ([19a7e9c](https://github.com/eSlider/go-onlyoffice/commit/19a7e9cbf8f7bd335c4e4cfd4d3b8a84a350cc41))
* **crm:** allow updating invoice terms via oo ([31c556d](https://github.com/eSlider/go-onlyoffice/commit/31c556d34cedd65d6957441f448520af9a7e380b))
* **crm:** invoice update notes and PO fields ([d81e7de](https://github.com/eSlider/go-onlyoffice/commit/d81e7de9114a29439142efba67f205653f562730))
* **crm:** invoices CLI and opportunity update ([f958309](https://github.com/eSlider/go-onlyoffice/commit/f95830961d3aecba65f3cb074d33c9c98f6c6cf2))
* **mail:** draft, attach, and draft-invoice CLI ([73b1050](https://github.com/eSlider/go-onlyoffice/commit/73b1050a83e9e49dcf99d8a966a84746376d7dd9))
### Bug Fixes
* **crm:** JSON person update and oo persons update CLI ([42c31d4](https://github.com/eSlider/go-onlyoffice/commit/42c31d4b485bfab6903aeed0dd6fb2c43f61b4f2))
* **crm:** link invoices to opportunities ([3201a58](https://github.com/eSlider/go-onlyoffice/commit/3201a58df75bceb7d41034970cf384a9169163ed))
* **mail:** German spacing for invoice draft template ([7f35b0e](https://github.com/eSlider/go-onlyoffice/commit/7f35b0e91075d871f87de95b6c84b2353e02f9cf))
### Documentation
* **crm:** capture association graph and invoice/mail quirks ([490426c](https://github.com/eSlider/go-onlyoffice/commit/490426c2779160f1a9b168f6ccae9dd8dccf89e9))
* **crm:** clarify mail API send vs signature for chat links ([6b5a82b](https://github.com/eSlider/go-onlyoffice/commit/6b5a82bef831341aee16b0b0ea1fe6db2c24567d))
* **crm:** Team vs Contacts and persons update note ([11aa23f](https://github.com/eSlider/go-onlyoffice/commit/11aa23f1a17471bad2bd95fddb769a94afbcf97a))
## [0.9.0](https://github.com/eSlider/go-onlyoffice/compare/v0.8.3...v0.9.0) (2026-07-26)
+1 -1
View File
@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2026 Example Author
Copyright (c) 2026 Andriy Oblivantsev
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
+318 -96
View File
@@ -476,6 +476,11 @@ type Task struct {
| `UpdateProject(req)` | Update project details |
| `DeleteProject(id)` | Delete a project |
| `GetProjectMilestones(project)` | Get milestones with task counts |
| `DeleteMilestone(id)` | Remove a milestone |
| `ListProjectTeam(ctx, id)` | Portal users on the project team |
| `AddProjectTeamUser(ctx, id, userID)` | Add a portal user to the team |
| `RemoveProjectTeamUser(ctx, id, userID)` | Remove a portal user from the team |
| `SetProjectTeam(ctx, id, participants, notify)` | Replace the team with the given user ids |
### Tasks
@@ -502,6 +507,61 @@ type Task struct {
| Method | Description |
|---|---|
| `GetUsers()` | List all users with profiles |
| `GetUser(ctx, id)` | One user profile by id |
| `CreateUser(ctx, NewUserRequest)` | Add a portal user |
| `UpdateUser(ctx, id, body)` | Update profile fields (JSON PUT) |
| `DeleteUser(ctx, id)` | Delete permanently (auto-terminates first — OO refuses active users) |
| `BlockUser(ctx, id)` / `UnblockUser(ctx, id)` | Terminate / reactivate (login kept/denied) |
| `ChangeUserPassword(ctx, id, pw)` | Set a new password |
| `ChangeUserStatus(ctx, id, active)` | Activate / Terminate via `people/status` |
| `AuthenticateAs(ctx, login, pw)` | Verify a login without mutating the cached token |
### Documents Files
| Method | Description |
|---|---|
| `ListDavFolder(ctx, id)` | List a Documents folder (`@root` for virtual sections) |
| `ListDavSections(ctx)` | Virtual sections (Documents, Projects, …) |
| `CreateDavFolder(ctx, parentID, title)` | Create a subfolder |
| `RenameDavFolder(ctx, id, title)` / `RenameDavFile(ctx, id, title)` | Rename folder / file |
| `DownloadFile(ctx, id, dst)` / `DownloadDavFile(ctx, id, w)` | Download file bytes |
| `UploadDavFile(ctx, folderID, fileName, src)` | Upload from a reader |
| `UploadToFolder(ctx, folderID, localPath)` | Upload a local file into a folder |
| `UploadToFolderReplacing(ctx, folderID, localPath)` | Upsert by `stem\|ext`; returns replaced ids |
| `UpdateFile(ctx, fileID, localPath)` | New version of an existing file (same id, no copy) |
| `MoveDavItems(ctx, folderIDs, fileIDs, dest)` | Move (`resolveType=Skip`); per-operation errors surfaced, not silent nil |
| `CopyDavItems(ctx, folderIDs, fileIDs, dest)` | Copy (`conflictResolveType=Skip`); errors surfaced |
| `MoveFiles(ctx, destFolderID, fileIDs)` | Move with `resolveType=Skip` + `holdResult`; errors surfaced |
| `ListFileOps(ctx)` | Active file operations (move/copy status polling) |
| `FolderFiles(ctx, folderID)` | Flat file list of a folder (stem helpers) |
| `DeleteFilesByStem(ctx, folderID, stem)` | Remove `stem\|ext` copies |
| `DoRetry(ctx, policy, fn)` | Deterministic exponential backoff (`Base·2^(N-1)`, no jitter) on 429/502/503/504; honours `Retry-After` and the process-wide cooldown gate |
| `DefaultRetryPolicy()` | From env: 7 attempts, 2s base, 2m cap (`OO_RETRY_ATTEMPTS/_BASE/_MAX`) |
| `Transient(err)` | True for retriable OnlyOffice answers (`*TransientError` or HTTP 429/502/503/504 text) |
### Deep links & conversion
| Method | Description |
|---|---|
| `FileEditorURL(portalBase, id)` / `(c *Client).FileEditorURL(id)` | DocEditor deep link `/Products/Files/DocEditor.aspx?fileid=` |
| `FolderURL(portalBase, id)` / `(c *Client).FolderURL(id)` | Documents folder link |
| `PresignedURI(ctx, fileID)` | Short-lived fetchable URL of a portal file (`presigneduri`) |
| `SignJWT(secret, payload)` | HS256 JWT, stdlib only |
| `ConvertDocument(ctx, docsBase, secret, req)` | OnlyOffice DocumentServer conversion (`/converter`; legacy `/ConvertService.ashx`) |
| `DownloadURLTo(ctx, url, w)` | Stream an absolute URL into a writer |
| `SyncBoard(ctx, board, apply)` | Upsert project milestones/tasks from a YAML board (dry-run when apply=false) |
| `AuditOpportunities(ctx)` | Opportunities with file/task/member counts and a coarse class |
| `WorkbookSheetNames(data)` | Worksheet names of an XLS/XLSX/ODS workbook |
| `WorkbookSheetCSV(data, sheet, delim)` | One worksheet → CSV (sheet-aware, excelize) |
| `WorkbookSheetJSON(data, sheet)` | One worksheet → rows as objects (first row = header) |
Example — convert a portal file (or a local file) to PDF with the native engine:
```bash
export ONLYOFFICE_DS_SECRET=<DocumentServer CoAuthoring secret>
oo docs pdf 1234 --out out.pdf # OO file id → PDF
oo docs pdf ./report.docx --to pdf # local file → PDF (temp upload, auto-cleanup)
```
### Helper Types
@@ -533,7 +593,7 @@ deals, total, _ := client.ListOpportunities(ctx, 50, 0)
company, _ := client.FindCompany(ctx, "ACME")
// Subtasks (form-encoded)
client.AddSubtask(ctx, "4242", "Prepare CV")
client.AddSubtask(ctx, "4242", "Prepare notes")
```
### oo (bundled CLI)
@@ -552,14 +612,48 @@ oo calendar events --start 2026-04-24 --end 2026-05-01
oo projects list
oo projects get 33
oo tasks list --all --verbose
oo tasks subtask add 4242 "Prepare CV"
oo tasks subtask add 4242 "Prepare notes"
oo persons create --first Jane --last Doe --email jane@example.com
oo companies create --name "Acme GmbH" --website https://acme.com
oo opportunities list
oo opportunities stages
oo cases list
oo crm-tasks categories
oo applications sync --path ./applications/2026 --apply
# Deep links & native document conversion
oo link 1234 2345 # DocEditor URL for file ids (title + url)
oo docs presigned 1234 # short-lived fetchable URL of an OO file
oo docs pdf 1234 --out out.pdf # OO file → PDF (via DocumentServer)
oo docs pdf ./report.docx --to pdf # local file → PDF on the fly (temp upload+cleanup)
oo docs pdf 1234 --stream > out.pdf # pipe: bytes to stdout (alias --pipe)
# Spreadsheet export (sheet-aware; local reader — the DS csv output is first-sheet-only)
oo docs csv 1234 --sheet 2 # XLS/XLSX/ODS worksheet → CSV (--sheet N, 1-based)
oo docs csv 1234 --delimiter ';' # ; | | \t via --delimiter
oo docs json 1234 --sheet 2 # worksheet → JSON rows (first row = header)
oo docs csv ./book.xlsx --sheet 1 --out sheet1.csv
# export ONLYOFFICE_DS_SECRET=<DocumentServer CoAuthoring secret>
# docs base: $ONLYOFFICE_DOCS_URL, else $ONLYOFFICE_URL + /ds-vpath
# Project team (portal users) CRUD
oo projects team list 42
oo projects team add 42 <user_id> [<user_id>...]
oo projects team remove 42 <user_id>
oo projects team set 42 <user_id> [...] # replace team (may lag; verify with list)
oo projects milestone-delete 7
# Project documents: fresh re-upload (single clean version) / new version
oo projects files replace-in 1 ./contract.pdf # hard delete same stem|ext in folder + upload
oo projects files update 1 ./contract.docx # overwrite content, same file id
# Users lifecycle
oo users list ; oo users get <user_id>
oo users create --first Jane --last Doe --email jane.doe@example.com --password '…'
oo users check --login jane.doe@example.com # verify login (email works when userName 500s)
oo users update <user_id> --title "…" --location "…"
oo users block <user_id> ; oo users unblock <user_id>
oo users password <user_id> # reads the new password from stdin
oo users delete <user_id> [<user_id>...]
```
### office (TUI)
@@ -608,49 +702,150 @@ go test -tags=integration ./cmd/office/fetch/... ./cmd/office/preview/...
```bash
# Project Documents (files module)
oo projects files list 33
oo projects files upload 33 ./notes.md
oo projects files download 12345 --to ./copy.md
oo projects files rename 12345 notes-v2.md
oo projects files upload 33 ./notes.docx
oo projects files download 12345 --to ./copy.docx
oo projects files rename 12345 notes-v2.docx
oo projects files delete 12345
oo projects files dedupe 7 # dry-run duplicate report
oo projects files dedupe 7 --apply # remove older stem|ext copies per folder
oo projects files dedupe 7 --cross --apply # cross-folder; keep non-_trash
# Agent document pipeline (md in git ↔ docx in OO; OCR scans)
oo docs tools
oo docs convert ./note.md # → note.docx
oo docs convert ./note.docx # → note.md
oo docs ocr ./scan.jpg --md ./scan.md # searchable PDF + markdown
oo docs hocr ./scan.jpg --lang spa --md ./scan.hocr.md --yaml ./scan.yml
oo docs put-md 7 ./OO-HONDA-7-INDEX.md --folder 490
oo docs put-txt 7 ./notes.txt --folder 490
oo docs put-xlsx 7 ./table.xlsx --folder 490
oo docs as-md 2815 --to ./parte.md # download OO file as MD (OCR if needed)
oo docs as-md 307 --hocr --lang spa # OO download via go-hocr structure
oo projects files put-md 7 ./note.md # alias
oo projects files as-md 2815 # alias
oo tasks files list 208
oo tasks files upload 208 ./cv.pdf
oo tasks files upload 208 ./notes.pdf
oo tasks files detach 208 12345
```
### Documents module (`oo dav`)
Direct access to the Documents module by folder/file id — the same calls that
back `oo-webdav` and the project/task file commands. `move` sends
`resolveType=Skip` + `holdResult=true`: without those params the legacy
`fileops/move` endpoint answers 200 without moving anything, and the library
surfaces such per-operation errors instead of a silent nil
(`MoveDavItems` / `CopyDavItems` / `MoveFiles`).
```bash
oo dav ls 659
oo dav ls @root # virtual sections (Documents, Projects, …)
oo dav mkdir 659 "2026 inbox"
oo dav move 659 22881 22882 # DEST_FOLDER_ID FILE_ID…
oo dav move 659 22881 --folders 670 # move folders along with files
oo dav copy 659 22881
oo dav rename-file 22881 invoice-v2.pdf
oo dav rename-folder 671 o2-archive
oo dav download 22881 --to ./copy.pdf # default path: ./<server title>
oo dav fileops # active move/copy operations (status polling)
```
### Search and index (`oo search`, `oo index`)
Full-text search over the Documents index. The REST endpoint
`/api/2.0/files/@search/{query}` only searches file names in the database, so
`oo search` talks to the OnlyOffice **Elasticsearch** directly (index
`files_file`). Name search is default; `--content` also matches extracted
document text (`document.attachment.content`, Office formats only).
See [`docs/elasticsearch.md`](docs/elasticsearch.md) for the tunnel setup.
```bash
oo search "Rechnung" # names only
oo search "Mahngebühr" --content # names + document text
oo search "Rechnung" --folder 649 --limit 50
oo search "Rechnung" --json # shorthand for -o json
```
Requires `ONLYOFFICE_ES_URL` (plus optional `ONLYOFFICE_ES_INDEX`,
`ONLYOFFICE_TENANT`).
#### PDF/scans: own index (`oo index` + `--backend own`)
The OnlyOffice index covers Office formats only, so PDFs (`S1019`-style invoice
numbers) are not searchable by content. `oo index` extracts PDF text with
`internal/docpipe` (pdftotext, OCR for scans) — including the text of embedded
PDF attachments (`pdfdetach`: `<doc>.md`, `.xml`, covers the original/scan and
ZUGFeRD e-invoice XML) — into a separate index (`ONLYOFFICE_ES_TEXT_INDEX`,
default `oo_docs_text`); the OnlyOffice server and its index are **not**
modified. Then search it with `--backend own`.
```bash
oo index folder 634 --recursive --exts pdf # populate (idempotent upsert)
oo index files 3576 3578 # specific files
oo index folder 634 --dry-run # plan only
oo search "S1021" --content --backend own # finds the PDF
oo search "Rechnung" --backend own --folder 634 --json
```
See [`docs/elasticsearch.md`](docs/elasticsearch.md) for the decision and
trade-offs.
### Unified file client
All file backends (REST, WebDAV, read-only SQL, Elasticsearch) sit behind one
facade: `c.Files()` returns a `*FileClient` that also implements `FileStore`,
so old call sites keep working. Pick a transport per call with
`c.FileStore("rest"|"dav"|"pg"|"sql")` (SQL is read-only), open the SQL store
with `c.SQLFileStore()`, or register a backend on the facade
(`RegisterStore`/`RegisterSearcher`). Contract, model (`Entry`/`Kind`),
fallback rules, env names and how to add a backend:
[`docs/unified-file-client.md`](docs/unified-file-client.md).
### Bulk tools
Business / one-off bulk tools (`ooscan`, `pdfamount`, `kontoblatt`,
`kontolink`) live in the private `oo-workspace` repo, not in this public
library. They build on the public client and the same `DoRetry` pacing.
| Subject | Verbs |
|---|---|
| `calendar` | `list`, `events`, `add`, `delete` |
| `projects` | `list`, `get`, `milestones`, `create`, `update`, `delete`, **`files`** (`list`, `upload`, `download`, `rename`, `delete`) |
| `projects` | `list`, `get`, `milestones`, `milestone-create`, `board-sync`, `create`, `update`, `delete`, `contacts` (`add`, `remove`), `link-authors`, `link-git`, **`files`** (`list`, `upload`, `download`, `rename`, `delete`, `dedupe`, `as-md`, `put-md`, `put-txt`, `put-xlsx`) |
| `tasks` | `list`, `get`, `create`, `update`, `delete`, `subtask add`, **`files`** (`list`, `upload`, `detach`) |
| `users` | `list`, `self` (alias: `oo whoami`) |
| `contacts` | `list`, `get`, `delete`, `info-add` |
| `persons` | `list` (filtered), `create`, `delete` |
| `companies` | `list` (filtered), `create`, `delete` |
| `contacts` | `list`, `get`, `delete`, `info-add`, `dedupe-info` |
| `contacts` | `list`, `get`, `delete`, `info-add`, `merge`, `dedupe-info`, `tags`, `tag-add`, `tag-create`, `tag-remove` |
| `persons` | `list`, `create`, `delete`, `dedupe` |
| `companies` | `list`, `create`, `delete`, `dedupe`, `dedupe-persons` |
| `opportunities` | `list`, `get`, `create`, `delete`, `stages`, `member-add`, `dedupe`, `dedupe-members`, `fix-titles` |
| `crm` | `cleanup` |
| `mails` | `accounts`, `folders`, `list`, `get`, `delete` |
| `opportunities` | `list`, `get`, `create`, `update`, `delete`, `stages`, `member-add`, `dedupe`, `dedupe-members`, `fix-titles` |
| `invoices` | `list`, `get`, `create`, `update`, `pdf`, `pdf-cleanup`, `status`, `delete`, `items …` |
| `crm` | `audit`, `cleanup` |
| `mails` | `accounts`, `folders`, `list`, `get`, `download-attachment`, `draft`, `attach`, `draft-invoice`, `send`, `delete` |
| `cases` | `list`, `create`, `delete`, `member-add` |
| `crm-tasks` | `list`, `create`, `delete`, `categories` |
| `applications` | `sync` |
| `catalog` | `scan-contacts`, `scan-projects`, `scan-thunderbird`, `merge`, `match`, `apply` |
| `crm-tasks` | `list`, `create`, `delete`, `categories`, `reassign-self` |
| `docs` | `tools`, `convert`, `pdf`, `presigned`, `csv`, `json`, `optimize`, `ocr`, `hocr`, `as-md`, `put-md`, `put-txt`, `put-xlsx` |
| `catalog` | `match`, `merge`, `apply`, `scan-contacts`, `scan-projects`, `scan-thunderbird` |
| `dav` | `ls`, `move`, `copy`, `mkdir`, `rename-file`, `rename-folder`, `download`, `fileops` |
| `search` | `QUERY` (`--content`, `--folder ID`, `--limit N`, `--backend oo\|own`, `--json`) |
| `index` | `folder FOLDER_ID`, `files FILE_ID...` (`--recursive`, `--exts pdf`, `--limit N`, `--dry-run`) |
The CLI reads only `.env` from the current working directory (godotenv is a
CLI-only concern — the library itself never loads dotfiles).
Canonical `ONLYOFFICE_*` variables win over aliases. For produktor.io operator
files, `OO_URL` / `OO_USER` / `OO_PASS` are accepted as CLI-only aliases for
`ONLYOFFICE_URL` / `ONLYOFFICE_USER` / `ONLYOFFICE_PASS`.
Canonical `ONLYOFFICE_*` variables win over aliases. Optional CLI-only aliases:
`OO_URL` / `OO_USER` / `OO_PASS` → `ONLYOFFICE_URL` / `ONLYOFFICE_USER` / `ONLYOFFICE_PASS`.
Catalog scanning (`oo catalog scan-projects` / `scan-thunderbird`) classifies
clients from rules in `$OO_CATALOG_CONFIG` (or `--config`); see
[`catalog/classify.example.yaml`](catalog/classify.example.yaml). Without rules
nothing is classified as work. The MinIO download fallback is off unless
`MINIO_ENDPOINT` + `MINIO_ACCESS_KEY` + `MINIO_SECRET_KEY` are set.
Run `oo --help` or `oo <subject> --help` for the full command reference.
> **0.5.0 migration note:** the command tree was flattened per-subject. Old
> flat names (`oo cal-events`, `oo task-list`, `oo crm-contacts`,
> `oo applications-sync`, …) were replaced by subject-based equivalents
> (`oo calendar events`, `oo tasks list`, `oo contacts list`,
> `oo applications sync`). Flags on leaf commands are unchanged.
> flat names (`oo cal-events`, `oo task-list`, `oo crm-contacts`, …) were
> replaced by subject-based equivalents (`oo calendar events`, `oo tasks list`,
> `oo contacts list`). Flags on leaf commands are unchanged.
## oo CLI use cases
@@ -672,7 +867,7 @@ Every list command accepts `-o table` (default) or `-o json` for scripting.
### CRM cleanup after imports or sync drift
**Problem:** Duplicate companies (`Acme` / `ACME GmbH`), persons created twice,
the same email on a contact three times, deals titled ` @ contoso`, or the same
the same email on a contact three times, deals titled ` @ Acme`, or the same
HR contact linked to a deal twice.
**One-shot fix** — runs every dedupe pass in order:
@@ -711,13 +906,39 @@ oo contacts dedupe-info
# Two deals with the same title
oo opportunities dedupe
# Same contact attached twice to one deal (common after applications sync)
# Same contact attached twice to one deal
oo opportunities dedupe-members
# Titles like " @ contoso" or extra whitespace
# Titles like " @ Acme" or extra whitespace
oo opportunities fix-titles
```
Merge two known company ids (keeps `INTO`):
```bash
oo contacts merge FROM_ID INTO_ID
```
Company ↔ person ↔ deal ↔ project ↔ invoice ↔ mail rules and OO quirks live
with the private `oo-workspace` tooling.
### Invoices (`oo invoices`)
**Problem:** Bill a client deal as Draft, regenerate PDF, prune duplicate PDF
attachments, prepare OnlyOffice Mail — without inventing a second bill-to company.
```bash
# Always pass --opportunity at create (update --opportunity often HTTP 400)
oo invoices create --number INV-2026-01 --contact COMPANY_ID --item ITEM_ID \
--price 300 --opportunity DEAL_ID --language de-DE \
--line-description "…" --terms $'…' --description $'…' --po "Deal #DEAL_ID"
oo invoices pdf INVOICE_ID --force
oo invoices pdf-cleanup INVOICE_ID
oo invoices status INVOICE_ID --status draft
oo mails draft-invoice --invoice INVOICE_ID --to billing@example.com
```
**Deal grouping flag** — when the same role at the same company created
separate deals (`Engineer @ Acme` vs `Engineer`):
@@ -729,66 +950,11 @@ oo crm cleanup --ignore-company-suffix
**Inspect before/after:**
```bash
oo opportunities list --count 200 | grep -i contoso
oo contacts get 857 -o json
oo opportunities list --count 200
oo contacts get CONTACT_ID -o json
oo crm cleanup -o json
```
### Clients/contacts inventory (`catalog`)
**Problem:** Contacts live as VCF/folders on disk and project trees under
`~/work` / ops-host; OnlyOffice CRM is the SSOT for persons/companies but
starts sparse. You need a reviewable inventory before writes.
```bash
oo catalog scan-contacts --root /path/to/contacts -O /tmp/contacts.yaml
oo catalog scan-projects --root ~/work -O /tmp/projects.yaml
oo catalog scan-thunderbird --root /path/to/thunderbird-profile -O /tmp/thunderbird.yaml
oo catalog merge -i /tmp/contacts.yaml -i /tmp/projects.yaml -i /tmp/thunderbird.yaml -O clients-contacts.yaml
oo catalog match -i clients-contacts.yaml
# set approve: true on pilot rows in the YAML
oo catalog apply --dry-run -i clients-contacts.yaml
oo catalog apply --apply -i clients-contacts.yaml
oo crm cleanup
```
YAML schema and ops runbook live in the inventar repo under `docs/catalog/` and
`docs/ops/oo-clients-contacts-sync.md`. Library helper: `FindPersonByEmail`.
### Job applications → CRM (`applications sync`)
**Problem:** You keep CVs in a folder tree (`applications/2026/Acme/README.md`)
and want companies, persons, deals, and history notes in OnlyOffice without
re-typing.
**Dry-run first** (default — prints what would happen, writes nothing):
```bash
oo applications sync --path ./applications/2026 --verbose
```
**Apply** when the preview looks right:
```bash
oo applications sync --path ./applications/2026 --apply --verbose
```
Each dossier `README.md` is parsed for company, role, email, phone, LinkedIn, etc.
Discovery walks `--path` and **skips** junk trees (`node_modules`, `tools`, `.venv`,
`pdfs`, …). Only folders that look like CV application slugs (`source-NNN-…`, or
long hyphenated dossiers) are synced — not npm package READMEs.
The sync creates or finds contacts, opens a deal, adds members, and appends a
history note. Re-running is safe: duplicate members and duplicate deal titles
are skipped when already present.
**After a large sync**, run CRM cleanup to collapse duplicates introduced by
repeated runs or manual edits:
```bash
oo applications sync --path ./applications/2026 --apply
oo crm cleanup -o json
```
### Workspace mail (`oo mails`)
@@ -821,8 +987,21 @@ oo mails get 5664 -o json | jq '{subject, from, to, date}'
# Remove one or more messages (server moves to trash or deletes per Mail rules)
oo mails delete 5664
oo mails delete 5664 5663 5661
# Create / update a draft (HTML body field is API "body"; plain text is wrapped)
oo mails draft --to client@example.com --subject "Rechnung INV-2026-01" \
--body "Guten Tag,\n\nanbei die Rechnung.\n\nMit freundlichen Grüßen"
# Attach an OnlyOffice Files document (e.g. invoice PDF file id) to a draft
oo mails attach 7301 --file-id 12345
# Regenerate invoice PDF (force) + draft + attach (does not send)
oo mails draft-invoice --invoice 16 --to info@example.com
```
Matrix / chat URLs in signatures: use plain text
`chat: https://matrix.to/#/@user:server` — HTML `<a href="…#…">` truncates at `#`.
**Table output** splits the `from` header into `fromName` and `fromAddress`
(e.g. `Bitfinex` + `no-reply@bitfinex.com`). **JSON output** returns the raw
API payload.
@@ -862,8 +1041,8 @@ oo calendar events --start 2026-06-24 --end 2026-07-01
# Schedule interview block
oo calendar add "Technical interview" 2026-06-26T10:00:00Z 2026-06-26T11:00:00Z
# Attach CV to a hiring task
oo tasks files upload 208 ./cv.pdf
# Attach a file to a task
oo tasks files upload 208 ./notes.pdf
oo projects files list 33
```
@@ -871,7 +1050,7 @@ oo projects files list 33
| When | Command |
|------|---------|
| After `applications sync --apply` | `oo crm cleanup` |
| After bulk CRM imports | `oo crm cleanup` |
| After bulk CSV import into CRM | `oo crm cleanup` |
| Weekly inbox triage | `oo mails list --folder inbox --limit 100` |
| Before exec reporting | `oo opportunities list` + `oo projects list` |
@@ -885,23 +1064,66 @@ oo projects files list 33
| `ONLYOFFICE_PASS` (or `ONLYOFFICE_PASSWORD`) | Password |
| `ONLYOFFICE_CALENDAR_ID` | Default calendar id used when omitted (default `1`) |
| `ONLYOFFICE_PROJECT_ID` | Default project id used when omitted (default `33`) |
| `OO_URL`, `OO_USER`, `OO_PASS` | CLI-only produktor.io aliases mapped to `ONLYOFFICE_URL`, `ONLYOFFICE_USER`, `ONLYOFFICE_PASS` |
| `OO_URL`, `OO_USER`, `OO_PASS` | Optional CLI-only aliases for `ONLYOFFICE_*` |
| `OO_RATE_LIMIT` | Process-wide request pacing, req/s (default `4`; `0` disables) |
| `OO_BURST` | Token-bucket burst (default `1`) |
| `OO_RETRY_ATTEMPTS` | Transient retries, total attempts (default `7`) |
| `OO_RETRY_BASE` | Exponential backoff base (default `2s`) |
| `OO_RETRY_MAX` | Backoff cap (default `2m`) |
| `ONLYOFFICE_ES_URL` | Elasticsearch base URL (`oo search`, own index); see [`docs/elasticsearch.md`](docs/elasticsearch.md) |
| `ONLYOFFICE_ES_INDEX` | OnlyOffice index (default `files_file`) |
| `ONLYOFFICE_ES_TEXT_INDEX` | Own PDF/scan index (default `oo_docs_text`) |
| `ONLYOFFICE_TENANT` | `tenantId` filter for ES/SQL (empty = all) |
| `ONLYOFFICE_DSN` | Read-only SQL DSN (MySQL or `postgres://`); see [`docs/community-server-db.md`](docs/community-server-db.md) |
| `ONLYOFFICE_PG_DRIVER`, `ONLYOFFICE_PG_TENANT`, `ONLYOFFICE_PG_HOST/_PORT/_USER/_PASSWORD/_DBNAME/_SSLMODE` | SQL store override / DSN by parts (PostgreSQL) |
| `MINIO_ENDPOINT`, `MINIO_BUCKET`, `MINIO_ACCESS_KEY`, `MINIO_SECRET_KEY` | Object-store layout for SQL `Download` |
| `ONLYOFFICE_WEBDAV_URL` | rclone WebDAV sidecar URL (default `http://172.17.0.1:8098/webdav`) |
Mail, CRM cleanup, and applications sync are documented in [oo CLI use cases](#oo-cli-use-cases) above.
Mail and CRM cleanup are documented in [oo CLI use cases](#oo-cli-use-cases) above. Personal disk inventory / dossier sync lives in the private `oo-workspace` (`oow`) tooling.
## Testing
```bash
go build ./... && go vet ./...
go test ./... # unit — no network, no vendor mocks
go test -race ./...
go test -tags=integration ./... # live OnlyOffice (skip without creds)
```
Unit tests are pure Go (parsers, encoders, conversions). Integration tests
(`//go:build integration`) hit a live instance and **skip** cleanly when the
env is missing, so `go test ./...` stays green offline. New endpoints ship with
an integration test before merge (policy in [`AGENTS.md`](AGENTS.md)).
Live runs need credentials (`ONLYOFFICE_URL`, `ONLYOFFICE_USER`,
`ONLYOFFICE_PASS`) and, per backend:
- **Elasticsearch** (`oo search`, own index) — ES lives on `127.0.0.1:9200`
inside the OnlyOffice VM; expose it over SSH
(`-L 9200:127.0.0.1:9200`) and set `ONLYOFFICE_ES_URL`
(see [`docs/elasticsearch.md`](docs/elasticsearch.md)).
- **SQL backend** (`FileClient`, `SQLFileStore`) — MySQL on `127.0.0.1:3306`
in the same VM; tunnel `-L 3306:127.0.0.1:3306`, then set `ONLYOFFICE_DSN`
(see [`docs/community-server-db.md`](docs/community-server-db.md)).
`ONLYOFFICE_PG_TEST_FILE_ID` / `ONLYOFFICE_PG_TEST_FOLDER_ID` select a real
file for the REST cross-check; `MINIO_*` enable the download check.
### rclone WebDAV mount
The rclone WebDAV mount (compose + `oo-webdav` sidecar) is deployment
tooling and lives in the private `oo-workspace` repo. Set
`ONLYOFFICE_WEBDAV_URL` to point the client at it.
### CI / releases
GitHub Actions (pattern from [`eSlider/go-config`](https://github.com/eSlider/go-config)):
| Workflow | Trigger | Purpose |
|---|---|---|
| `test.yml` | push / PR | `go vet`, unit tests, build `oo` + `office` |
| `release-please.yml` | push to `main` | semver PR from conventional commits |
| `release.yml` | tag `v*` | GoReleaser cross-platform `oo` + `office` binaries |
Repo setting required once: **Settings → Actions → General → Allow GitHub Actions to create and approve pull requests**.
Merge the release-please PR to tag a version; GoReleaser publishes assets to [GitHub Releases](https://github.com/eSlider/go-onlyoffice/releases).
Gitea is canonical; tags are created there per SemVer (`fix:` → patch,
`feat:` → minor, `!` → major). GoReleaser publishes assets to
[GitHub Releases](https://github.com/eSlider/go-onlyoffice/releases).
## Examples
+61 -8
View File
@@ -43,6 +43,11 @@ func (c *Client) Authenticate() error { return c.ensureToken() }
// cached token is still valid it returns immediately; otherwise it performs
// a POST to /api/2.0/authentication.json that is cancellable via ctx.
//
// Transient answers from the edge (openresty 429/502/503/504) are retried with
// the same deterministic policy as every other request (see retry.go), because
// the server rate-limits authentication and the integration suite otherwise
// fails with a raw HTML 429 page.
//
// This is the recommended entry point for long-running syncs (cron,
// watchers) because it guarantees that a stalled auth call will not block
// the caller past its deadline.
@@ -50,6 +55,14 @@ func (c *Client) AuthenticateContext(ctx context.Context) error {
if c.tokenValid() {
return nil
}
return DoRetry(ctx, DefaultRetryPolicy(), func() error {
return c.authenticateOnce(ctx)
})
}
// authenticateOnce performs a single authentication POST. Callers must handle
// retries; use AuthenticateContext.
func (c *Client) authenticateOnce(ctx context.Context) error {
body, err := json.Marshal(c.credentials)
if err != nil {
return fmt.Errorf("marshal credentials: %w", err)
@@ -69,6 +82,51 @@ func (c *Client) AuthenticateContext(ctx context.Context) error {
if err != nil {
return err
}
if resp.StatusCode >= 400 {
return statusError(resp.StatusCode, retryAfterOf(resp), "auth: %d %s", resp.StatusCode, truncate(string(raw), 400))
}
var env struct {
Response *Token `json:"response"`
}
if err := json.Unmarshal(raw, &env); err != nil {
return fmt.Errorf("auth decode: %w", err)
}
if env.Response == nil || env.Response.Value == "" {
return fmt.Errorf("auth: empty token in response")
}
c.token = env.Response
return nil
}
// AuthenticateAs verifies a login/password pair against the portal WITHOUT
// mutating the client's cached token. It returns nil when the portal issues a
// token, and the portal error otherwise.
//
// OnlyOffice accepts either the userName or the account email as the login. On
// some portals the account userName login returns HTTP 500 "User authentication
// failed" while the account email succeeds — confirmed for a freshly created
// guest user. Use this probe before sharing credentials (see `oo users check`),
// and prefer the email as the login.
func (c *Client) AuthenticateAs(ctx context.Context, login, password string) error {
body, err := json.Marshal(Credentials{User: login, Password: password})
if err != nil {
return fmt.Errorf("marshal credentials: %w", err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.baseURL()+"/api/2.0/authentication.json", bytes.NewReader(body))
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
resp, err := c.client.Do(req)
if err != nil {
return fmt.Errorf("auth request: %w", err)
}
defer resp.Body.Close()
raw, err := io.ReadAll(resp.Body)
if err != nil {
return err
}
if resp.StatusCode >= 400 {
return fmt.Errorf("auth: %d %s", resp.StatusCode, truncate(string(raw), 400))
}
@@ -81,7 +139,6 @@ func (c *Client) AuthenticateContext(ctx context.Context) error {
if env.Response == nil || env.Response.Value == "" {
return fmt.Errorf("auth: empty token in response")
}
c.token = env.Response
return nil
}
@@ -99,17 +156,13 @@ func (c *Client) tokenValid() bool {
// ensureToken refreshes the authentication token when missing or expired.
// Mirrors the logic inline in Query() but is safe to call from helpers that
// bypass the typed Request abstraction.
// bypass the typed Request abstraction. It shares AuthenticateContext so the
// transient-retry policy applies to every code path.
func (c *Client) ensureToken() error {
if c.tokenValid() {
return nil
}
tok, err := c.Auth(c.credentials)
if err != nil {
return err
}
c.token = tok
return nil
return c.AuthenticateContext(context.Background())
}
// authHeader returns the value for the Authorization header, ensuring a token.
+205
View File
@@ -0,0 +1,205 @@
package onlyoffice
// Project board (Gantt) upsert from a YAML board file.
//
// A board describes projects, their milestones and tasks by exact title. Sync
// creates only what is missing: existing milestones/tasks (matched by title)
// are left untouched, so the file can be the source of truth for a project
// plan and re-applied safely. Dry-run (apply=false) reports counts without
// writing.
//
// The file format is deliberately small and presentation-free:
//
// projects:
// - id: 42
// name: "Example"
// milestones:
// - title: "Kickoff"
// deadline: "2026-01-15"
// key: true
// tasks:
// - title: "Draft"
// start: "2026-01-02"
// deadline: "2026-01-10"
// description: "…"
import (
"context"
"fmt"
"os"
"strconv"
"time"
"gopkg.in/yaml.v3"
)
// Board is a YAML mapping from a project plan onto OnlyOffice milestones/tasks.
type Board struct {
Projects []BoardProject `yaml:"projects"`
}
// BoardProject is one project with its milestones.
type BoardProject struct {
ID int `yaml:"id"`
Name string `yaml:"name,omitempty"`
Milestones []BoardMilestone `yaml:"milestones"`
}
// BoardMilestone is a milestone ("key" marks it as a key milestone).
type BoardMilestone struct {
Title string `yaml:"title"`
Deadline string `yaml:"deadline"`
Key bool `yaml:"key,omitempty"`
Tasks []BoardTask `yaml:"tasks"`
}
// BoardTask is a task inside a milestone. Start falls back to Deadline.
type BoardTask struct {
Title string `yaml:"title"`
Start string `yaml:"start,omitempty"`
Deadline string `yaml:"deadline"`
Description string `yaml:"description,omitempty"`
}
// BoardSyncResult counts what SyncBoard created or skipped.
type BoardSyncResult struct {
CreatedMilestones int
SkippedMilestones int
CreatedTasks int
SkippedTasks int
DryRun bool
}
// LoadBoard reads a board YAML file.
func LoadBoard(path string) (*Board, error) {
b, err := os.ReadFile(path)
if err != nil {
return nil, err
}
return ParseBoard(b)
}
// ParseBoard decodes a board from YAML bytes.
func ParseBoard(data []byte) (*Board, error) {
var board Board
if err := yaml.Unmarshal(data, &board); err != nil {
return nil, fmt.Errorf("board: %w", err)
}
if len(board.Projects) == 0 {
return nil, fmt.Errorf("board: no projects")
}
return &board, nil
}
func boardDay(s string) (Time, error) {
t, err := time.Parse("2006-01-02", s)
if err != nil {
return Time{}, err
}
return Time(t), nil
}
func boardMilestoneIDs(ms []*Milestone) map[string]int64 {
out := map[string]int64{}
for _, m := range ms {
if m == nil || m.Title == nil || m.ID == nil {
continue
}
out[*m.Title] = *m.ID
}
return out
}
func boardTaskTitles(rows []map[string]any) map[string]struct{} {
out := map[string]struct{}{}
for _, r := range rows {
if t, _ := r["title"].(string); t != "" {
out[t] = struct{}{}
}
}
return out
}
// SyncBoard upserts milestones and tasks by exact title. With apply=false it
// only counts what would be created.
func (c *Client) SyncBoard(ctx context.Context, board *Board, apply bool) (*BoardSyncResult, error) {
if board == nil || len(board.Projects) == 0 {
return nil, fmt.Errorf("board: no projects")
}
res := &BoardSyncResult{DryRun: !apply}
for _, p := range board.Projects {
pid := p.ID
existing, err := c.GetProjectMilestones(&Project{ID: &pid})
if err != nil {
return res, fmt.Errorf("project %d milestones: %w", pid, err)
}
haveMS := boardMilestoneIDs(existing)
tasks, err := c.ListTasks(ctx, strconv.Itoa(pid), "")
if err != nil {
return res, fmt.Errorf("project %d tasks: %w", pid, err)
}
haveTask := boardTaskTitles(tasks)
for _, m := range p.Milestones {
msID, ok := haveMS[m.Title]
if !ok {
res.CreatedMilestones++
if apply {
dl, err := boardDay(m.Deadline)
if err != nil {
return res, fmt.Errorf("milestone %q deadline: %w", m.Title, err)
}
created, err := c.CreateMilestone(NewMilestoneRequest{
ProjectID: pid,
Title: m.Title,
Deadline: dl,
IsKey: m.Key,
})
if err != nil {
return res, fmt.Errorf("create milestone %q: %w", m.Title, err)
}
if created.ID != nil {
msID = *created.ID
}
haveMS[m.Title] = msID
}
} else {
res.SkippedMilestones++
}
for _, t := range m.Tasks {
if _, exists := haveTask[t.Title]; exists {
res.SkippedTasks++
continue
}
res.CreatedTasks++
if !apply {
continue
}
start := t.Start
if start == "" {
start = t.Deadline
}
st, err := boardDay(start)
if err != nil {
return res, fmt.Errorf("task %q start: %w", t.Title, err)
}
dl, err := boardDay(t.Deadline)
if err != nil {
return res, fmt.Errorf("task %q deadline: %w", t.Title, err)
}
if _, err := c.CreateProjectTask(NewProjectTaskRequest{
ProjectId: pid,
Title: t.Title,
Description: t.Description,
StartDate: st,
Deadline: dl,
MilestoneId: int(msID),
}); err != nil {
return res, fmt.Errorf("create task %q: %w", t.Title, err)
}
haveTask[t.Title] = struct{}{}
}
}
}
return res, nil
}
+70
View File
@@ -0,0 +1,70 @@
package onlyoffice
import "testing"
func TestParseBoard(t *testing.T) {
b, err := ParseBoard([]byte(`
projects:
- id: 13
name: Example
milestones:
- title: "[lq] Test"
deadline: "2026-01-15"
key: true
tasks:
- title: Draft
start: "2026-01-02"
deadline: "2026-01-10"
description: "…"
`))
if err != nil {
t.Fatal(err)
}
if len(b.Projects) != 1 || b.Projects[0].ID != 13 {
t.Fatalf("projects: %+v", b.Projects)
}
ms := b.Projects[0].Milestones[0]
if ms.Title != "[lq] Test" || !ms.Key || ms.Deadline != "2026-01-15" {
t.Fatalf("milestone: %+v", ms)
}
if len(ms.Tasks) != 1 || ms.Tasks[0].Title != "Draft" || ms.Tasks[0].Start != "2026-01-02" {
t.Fatalf("task: %+v", ms.Tasks)
}
}
func TestParseBoardEmpty(t *testing.T) {
if _, err := ParseBoard([]byte("projects: []")); err == nil {
t.Fatal("expected error for empty board")
}
}
func TestBoardMilestoneIDs(t *testing.T) {
title := "[lq] Test"
id := int64(9)
got := boardMilestoneIDs([]*Milestone{{Title: &title, ID: &id}, nil})
if got[title] != 9 {
t.Fatalf("%v", got)
}
}
func TestBoardTaskTitles(t *testing.T) {
got := boardTaskTitles([]map[string]any{{"title": "a"}, {"title": "b"}, {"nope": 1}})
if _, ok := got["a"]; !ok {
t.Fatal("missing a")
}
if _, ok := got["b"]; !ok {
t.Fatal("missing b")
}
if len(got) != 2 {
t.Fatalf("%v", got)
}
}
func TestBoardDay(t *testing.T) {
if _, err := boardDay("2026-01-15"); err != nil {
t.Fatal(err)
}
if _, err := boardDay("15.01.2026"); err == nil {
t.Fatal("expected error for non-ISO date")
}
}
+28
View File
@@ -0,0 +1,28 @@
package catalog
import "testing"
func TestMergeAddressesDedup(t *testing.T) {
dst := []Address{{Street: "Weg 1", City: "Stadt", Zip: "1"}}
got := mergeAddresses(dst, []Address{
{Street: "weg 1", City: "stadt", Zip: "1"}, // duplicate (case-insensitive)
{Street: "Weg 2", City: "Stadt", Zip: "2"}, // new
})
if len(got) != 2 {
t.Fatalf("got %d addresses, want 2: %+v", len(got), got)
}
if got[1].Street != "Weg 2" {
t.Errorf("second = %+v", got[1])
}
}
func TestEntryAddressesYAML(t *testing.T) {
doc := &Document{Entries: []Entry{{
ID: "person:max maier", Kind: "person", Name: "Max Maier",
Addresses: []Address{{Street: "Weg 1", City: "Stadt", Zip: "12345", Category: "Billing", Primary: true}},
}}}
merged := MergeDocs(doc)
if len(merged.Entries) != 1 || len(merged.Entries[0].Addresses) != 1 {
t.Fatalf("merged = %+v", merged.Entries)
}
}
+209
View File
@@ -0,0 +1,209 @@
package catalog
import (
"context"
"fmt"
"strconv"
"strings"
onlyoffice "github.com/eslider/go-onlyoffice"
)
// ApplyResult summarizes one apply pass.
type ApplyResult struct {
Created int `json:"created"`
Updated int `json:"updated"`
Skipped int `json:"skipped"`
Errors []string `json:"errors,omitempty"`
DryRun bool `json:"dry_run"`
}
// ApplyApproved creates/updates OO contacts for entries with approve=true.
// Companies are applied before persons so Org links resolve in the same run.
func ApplyApproved(ctx context.Context, client *onlyoffice.Client, doc *Document, dryRun bool) (*ApplyResult, error) {
res := &ApplyResult{DryRun: dryRun}
order := make([]int, 0, len(doc.Entries))
for i, e := range doc.Entries {
if e.Kind == "company" {
order = append(order, i)
}
}
for i, e := range doc.Entries {
if e.Kind != "company" {
order = append(order, i)
}
}
for _, i := range order {
e := &doc.Entries[i]
if !e.Approve {
res.Skipped++
continue
}
if e.Status == "conflict" {
res.Skipped++
res.Errors = append(res.Errors, e.ID+": conflict — resolve manually")
continue
}
if dryRun {
if e.OOID != "" || e.Status == "exists" {
res.Updated++
} else {
res.Created++
}
continue
}
created, err := applyOne(ctx, client, e)
if err != nil {
res.Errors = append(res.Errors, e.ID+": "+err.Error())
continue
}
if created {
res.Created++
} else {
res.Updated++
}
}
return res, nil
}
func applyOne(ctx context.Context, client *onlyoffice.Client, e *Entry) (created bool, err error) {
switch e.Kind {
case "company":
return applyCompany(ctx, client, e)
case "person":
return applyPerson(ctx, client, e)
default:
return false, fmt.Errorf("unknown kind %q", e.Kind)
}
}
func applyCompany(ctx context.Context, client *onlyoffice.Client, e *Entry) (bool, error) {
name := strings.TrimSpace(e.Name)
if name == "" {
return false, fmt.Errorf("company missing name")
}
var co map[string]any
var err error
if e.OOID != "" {
co, err = client.GetContact(ctx, e.OOID)
} else {
co, err = client.FindCompany(ctx, name)
}
if err != nil {
return false, err
}
created := false
if co == nil {
co, err = client.CreateCompany(ctx, name)
if err != nil {
return false, err
}
created = true
}
id := contactIDString(co)
e.OOID = id
if err := ensureContactInfos(ctx, client, id, e); err != nil {
return created, err
}
e.Status = "applied"
if created {
e.Notes = "created"
} else {
e.Notes = "updated"
}
return created, nil
}
func applyPerson(ctx context.Context, client *onlyoffice.Client, e *Entry) (bool, error) {
org := strings.TrimSpace(e.Org)
first, last := CleanPersonNames(e.First, e.Last, e.Name, org, e.Emails)
e.First, e.Last = first, last
if first == "" {
return false, fmt.Errorf("person missing name")
}
e.Name = strings.TrimSpace(first + " " + strings.Trim(last, "-"))
var p map[string]any
var err error
if e.OOID != "" {
p, err = client.GetContact(ctx, e.OOID)
} else if len(e.Emails) > 0 {
p, err = client.FindPersonByEmail(ctx, e.Emails[0])
} else {
p, err = client.FindPerson(ctx, first, last)
}
if err != nil {
return false, err
}
created := false
companyID := 0
if org != "" {
if co, ferr := client.FindCompany(ctx, org); ferr == nil && co != nil {
companyID, _ = strconv.Atoi(contactIDString(co))
}
}
if p == nil {
about := ""
if org != "" {
about = "org: " + org
}
p, err = client.CreatePerson(ctx, first, last, companyID, "", about)
if err != nil {
return false, err
}
created = true
} else {
// Repair names + ensure company link (never encode company in lastName).
id := contactIDString(p)
if _, err := client.UpdatePerson(ctx, id, first, last, companyID, "", ""); err != nil {
return false, fmt.Errorf("update person %s: %w", id, err)
}
}
id := contactIDString(p)
e.OOID = id
if err := ensureContactInfos(ctx, client, id, e); err != nil {
return created, err
}
e.Status = "applied"
if created {
e.Notes = "created"
} else {
e.Notes = "updated"
}
return created, nil
}
func ensureContactInfos(ctx context.Context, client *onlyoffice.Client, contactID string, e *Entry) error {
existing, err := client.GetContact(ctx, contactID)
if err != nil {
return err
}
for i, em := range e.Emails {
if onlyoffice.HasContactInfo(existing, "Email", em) {
continue
}
if _, err := client.AddContactInfo(ctx, contactID, "Email", em, "Work", i == 0); err != nil {
return fmt.Errorf("add email %s: %w", em, err)
}
}
for _, ph := range e.Phones {
if onlyoffice.HasContactInfo(existing, "Phone", ph) {
continue
}
if _, err := client.AddContactInfo(ctx, contactID, "Phone", ph, "Work", false); err != nil {
return fmt.Errorf("add phone %s: %w", ph, err)
}
}
for _, a := range e.Addresses {
if strings.TrimSpace(a.Street) == "" && strings.TrimSpace(a.City) == "" && strings.TrimSpace(a.Zip) == "" {
continue
}
if onlyoffice.HasContactAddress(existing, a.Street, a.City, a.Zip, a.Category) {
continue
}
if _, err := client.AddContactAddress(ctx, contactID, a.Street, a.City, a.State, a.Zip, a.Country, a.Category, a.Primary); err != nil {
return fmt.Errorf("add address %s: %w", strings.TrimSpace(a.Street), err)
}
}
return nil
}
+144
View File
@@ -0,0 +1,144 @@
package catalog
import (
"os"
"path/filepath"
"testing"
)
func TestParseVCFFile(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "sample.vcf")
body := `BEGIN:VCARD
VERSION:3.0
FN:Ada Lovelace
N:Lovelace;Ada;;;
EMAIL;TYPE=INTERNET:ada@example.com
TEL;TYPE=CELL:+1-555-0100
ORG:Analytical Engines
END:VCARD
`
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
entries, err := ParseVCFFile(path)
if err != nil {
t.Fatal(err)
}
if len(entries) != 1 {
t.Fatalf("got %d entries", len(entries))
}
e := entries[0]
if e.Kind != "person" || e.First != "Ada" || e.Last != "Lovelace" {
t.Fatalf("unexpected entry: %+v", e)
}
if len(e.Emails) != 1 || e.Emails[0] != "ada@example.com" {
t.Fatalf("emails: %v", e.Emails)
}
if e.Org != "Analytical Engines" {
t.Fatalf("org: %q", e.Org)
}
if e.Approve {
t.Fatal("approve should default false")
}
}
func TestMergeDocs(t *testing.T) {
a := &Document{Entries: []Entry{{
ID: "person:ada@example.com", Kind: "person", First: "Ada", Emails: []string{"ada@example.com"},
Sources: []string{"a.vcf"}, Zone: "private", Role: "unknown",
}}}
b := &Document{Entries: []Entry{{
ID: "person:ada@example.com", Kind: "person", Last: "Lovelace", Phones: []string{"+1"},
Sources: []string{"folder/Ada"}, Zone: "warm", Role: "work", Approve: true,
}}}
m := MergeDocs(a, b)
if len(m.Entries) != 1 {
t.Fatalf("len=%d", len(m.Entries))
}
e := m.Entries[0]
if e.First != "Ada" || e.Last != "Lovelace" {
t.Fatalf("%+v", e)
}
if len(e.Sources) != 2 || len(e.Phones) != 1 || !e.Approve {
t.Fatalf("%+v", e)
}
if e.Zone != "warm" || e.Role != "work" {
t.Fatalf("zone/role: %s/%s", e.Zone, e.Role)
}
}
func TestScanContactsRoot(t *testing.T) {
root := t.TempDir()
vcfDir := filepath.Join(root, "Contacts VCF's")
if err := os.MkdirAll(vcfDir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(vcfDir, "bob.vcf"), []byte(`BEGIN:VCARD
VERSION:3.0
FN:Bob Builder
EMAIL:bob@work.test
END:VCARD
`), 0o644); err != nil {
t.Fatal(err)
}
personDir := filepath.Join(root, "Carol Smith")
if err := os.MkdirAll(personDir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(personDir, "carol@example.com.txt"), []byte(""), 0o644); err != nil {
t.Fatal(err)
}
doc, err := ScanContactsRoot(root)
if err != nil {
t.Fatal(err)
}
if len(doc.Entries) < 2 {
t.Fatalf("expected >=2 entries, got %d: %+v", len(doc.Entries), doc.Entries)
}
}
func TestScanProjectsRoot(t *testing.T) {
root := t.TempDir()
repo := filepath.Join(root, "acme-demo")
if err := os.MkdirAll(filepath.Join(repo, ".git"), 0o755); err != nil {
t.Fatal(err)
}
cl := &Classifier{WorkNames: []string{"acme"}}
doc, err := ScanProjectsRootOpts(root, 3, ScanOptions{Classifier: cl})
if err != nil {
t.Fatal(err)
}
found := false
for _, e := range doc.Entries {
if e.Kind == "company" && e.Name == "acme-demo" {
found = true
if e.Role != "work" || e.Zone != "warm" {
t.Fatalf("role=%q zone=%q", e.Role, e.Zone)
}
}
}
if !found {
t.Fatalf("missing company: %+v", doc.Entries)
}
// Neutral default leaves it unclassified.
doc, err = ScanProjectsRoot(root, 3)
if err != nil {
t.Fatal(err)
}
for _, e := range doc.Entries {
if e.Kind == "company" && e.Name == "acme-demo" && e.Role != "unknown" {
t.Fatalf("neutral role=%q", e.Role)
}
}
}
func TestEntryID(t *testing.T) {
if got := EntryID("person", "A@B.COM", ""); got != "person:a@b.com" {
t.Fatal(got)
}
if got := EntryID("company", "", "Acme Corp"); got != "company:acme corp" {
t.Fatal(got)
}
}
+26
View File
@@ -0,0 +1,26 @@
# Deployment classification rules for `oo catalog scan-projects` /
# `oo catalog scan-thunderbird`. Point OO_CATALOG_CONFIG (or --config) at a copy
# of this file. Keep your real rules out of the repository — they name your
# clients and hosts. The library defaults to no rules (nothing is "work").
#
# work_remotes: a git remote containing any of these substrings → work/hot.
work_remotes:
- git.internal.example
- github.com/acme
#
# work_names: a project directory name containing any of these substrings → work/warm.
work_names:
- acme
#
# mail_orgs: ordered rules for Thunderbird/mbox identities; the first match wins.
# Match by exact `domain`, `suffix` (e.g. ".example.com") and/or display `name`.
# `zone` defaults to hot, `role` to work.
mail_orgs:
- domain: acme.example
org: Acme GmbH
zone: hot
role: work
- suffix: .gov.example
org: Public Sector
zone: warm
role: work
+127
View File
@@ -0,0 +1,127 @@
package catalog
import (
"fmt"
"os"
"strings"
"gopkg.in/yaml.v3"
)
// Classifier maps project trees and mail identities to catalog org/zone/role.
//
// The library ships with neutral defaults: nothing is classified as work unless
// the deployment supplies rules. Those rules are deployment-specific, so they
// live in a YAML config file (path from OO_CATALOG_CONFIG or the --config flag),
// not in the code. See catalog/classify.example.yaml.
type Classifier struct {
// WorkRemotes: a git remote containing any of these substrings → work/hot.
WorkRemotes []string `yaml:"work_remotes,omitempty"`
// WorkNames: a project name containing any of these substrings → work/warm.
WorkNames []string `yaml:"work_names,omitempty"`
// MailOrgs: ordered mail-identity rules; the first match wins.
MailOrgs []MailRule `yaml:"mail_orgs,omitempty"`
}
// MailRule maps an email domain and/or a display-name substring to an org with
// a zone/role. At least one of Domain, Suffix or Name must be set.
type MailRule struct {
Domain string `yaml:"domain,omitempty"` // exact domain, case-insensitive
Suffix string `yaml:"suffix,omitempty"` // domain suffix, e.g. ".example.com"
Name string `yaml:"name,omitempty"` // substring of the display name
Org string `yaml:"org"`
Zone string `yaml:"zone,omitempty"` // default "hot"
Role string `yaml:"role,omitempty"` // default "work"
}
// DefaultClassifier returns the neutral classifier (no deployment rules).
func DefaultClassifier() *Classifier { return &Classifier{} }
// LoadClassifier reads a classifier config from a YAML file.
func LoadClassifier(path string) (*Classifier, error) {
b, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var c Classifier
if err := yaml.Unmarshal(b, &c); err != nil {
return nil, fmt.Errorf("parse classifier config %s: %w", path, err)
}
return &c, nil
}
// LoadClassifierFromEnv loads the classifier named by OO_CATALOG_CONFIG. An
// empty variable yields the neutral classifier.
func LoadClassifierFromEnv() (*Classifier, error) {
path := strings.TrimSpace(os.Getenv("OO_CATALOG_CONFIG"))
if path == "" {
return DefaultClassifier(), nil
}
return LoadClassifier(path)
}
// ClassifyProject returns (role, zone) for a project name and git remote.
// Generic name heuristics come first; deployment rules supply the work cases.
func (c *Classifier) ClassifyProject(name, remote string) (role, zone string) {
lower := strings.ToLower(name)
remoteL := strings.ToLower(remote)
switch {
case strings.Contains(lower, "experiment") || strings.HasPrefix(lower, "test"):
return "experiment", "cold"
case lower == "mama" || lower == "personal" || strings.Contains(lower, "private"):
return "personal", "private"
}
if c != nil {
for _, r := range c.WorkRemotes {
if r != "" && strings.Contains(remoteL, strings.ToLower(r)) {
return "work", "hot"
}
}
for _, n := range c.WorkNames {
if n != "" && strings.Contains(lower, strings.ToLower(n)) {
return "work", "warm"
}
}
}
return "unknown", "warm"
}
// ClassifyMail returns (org, zone, role) for a mail identity. name is the
// display name (may be empty); email is the address.
func (c *Classifier) ClassifyMail(name, email string) (org, zone, role string) {
em := NormalizeEmail(email)
_, domain, _ := strings.Cut(em, "@")
nameL := strings.ToLower(strings.TrimSpace(name))
if c != nil {
for _, r := range c.MailOrgs {
if !mailRuleMatches(r, domain, nameL) {
continue
}
z, ro := r.Zone, r.Role
if z == "" {
z = "hot"
}
if ro == "" {
ro = "work"
}
return r.Org, z, ro
}
}
if strings.HasSuffix(domain, ".de") && looksPublicSector(domain) {
return domain, "warm", "work"
}
return "", "private", "unknown"
}
func mailRuleMatches(r MailRule, domain, nameL string) bool {
if r.Domain != "" && domain == strings.ToLower(strings.TrimSpace(r.Domain)) {
return true
}
if r.Suffix != "" && strings.HasSuffix(domain, strings.ToLower(strings.TrimSpace(r.Suffix))) {
return true
}
if r.Name != "" && nameL != "" && strings.Contains(nameL, strings.ToLower(strings.TrimSpace(r.Name))) {
return true
}
return false
}
+64
View File
@@ -0,0 +1,64 @@
package catalog
import (
"os"
"path/filepath"
"testing"
)
func TestClassifierRules(t *testing.T) {
cl := &Classifier{
WorkRemotes: []string{"git.internal.example"},
WorkNames: []string{"acme"},
MailOrgs: []MailRule{
{Domain: "acme.example", Org: "Acme", Zone: "hot", Role: "work"},
{Suffix: ".gov.example", Org: "Public", Zone: "warm", Role: "work"},
},
}
if role, zone := cl.ClassifyProject("acme-app", "git@git.internal.example:team/acme-app.git"); role != "work" || zone != "hot" {
t.Fatalf("remote: %s/%s", role, zone)
}
if role, zone := cl.ClassifyProject("acme-demo", ""); role != "work" || zone != "warm" {
t.Fatalf("name: %s/%s", role, zone)
}
if role, zone := cl.ClassifyProject("experiment-x", ""); role != "experiment" || zone != "cold" {
t.Fatalf("experiment: %s/%s", role, zone)
}
if org, zone, role := cl.ClassifyMail("", "bob@acme.example"); org != "Acme" || zone != "hot" || role != "work" {
t.Fatalf("mail: %s/%s/%s", org, zone, role)
}
if org, _, _ := cl.ClassifyMail("X", "x@team.gov.example"); org != "Public" {
t.Fatalf("suffix: %s", org)
}
if org, zone, role := cl.ClassifyMail("", "someone@unknown.example"); org != "" || zone != "private" || role != "unknown" {
t.Fatalf("neutral: %s/%s/%s", org, zone, role)
}
}
func TestLoadClassifier(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "classify.yaml")
if err := os.WriteFile(path, []byte("work_names:\n - acme\nmail_orgs:\n - domain: acme.example\n org: Acme\n"), 0o644); err != nil {
t.Fatal(err)
}
cl, err := LoadClassifier(path)
if err != nil {
t.Fatal(err)
}
if role, _ := cl.ClassifyProject("acme-app", ""); role != "work" {
t.Fatalf("role=%s", role)
}
if org, zone, _ := cl.ClassifyMail("", "a@acme.example"); org != "Acme" || zone != "hot" {
t.Fatalf("org=%s zone=%s", org, zone)
}
t.Setenv("OO_CATALOG_CONFIG", path)
if envCl, err := LoadClassifierFromEnv(); err != nil || envCl == nil || len(envCl.WorkNames) == 0 {
t.Fatalf("env: %+v %v", envCl, err)
}
t.Setenv("OO_CATALOG_CONFIG", "")
neutral, err := LoadClassifierFromEnv()
if err != nil || len(neutral.WorkNames) != 0 || len(neutral.MailOrgs) != 0 {
t.Fatalf("neutral: %+v %v", neutral, err)
}
}
+138
View File
@@ -0,0 +1,138 @@
package catalog
import (
"context"
"fmt"
"strconv"
"strings"
onlyoffice "github.com/eslider/go-onlyoffice"
)
// MatchAgainstOO sets status/oo_id by email then normalized name.
func MatchAgainstOO(ctx context.Context, client *onlyoffice.Client, doc *Document) error {
all, err := client.ListAllContacts(ctx)
if err != nil {
return err
}
byEmail := map[string]map[string]any{}
byPersonName := map[string]map[string]any{}
byCompanyName := map[string]map[string]any{}
for _, c := range all {
id := contactIDString(c)
isCo, _ := c["isCompany"].(bool)
for _, em := range contactEmails(c) {
byEmail[NormalizeEmail(em)] = c
}
if isCo {
key := onlyoffice.CompanyGroupingKey(fmt.Sprint(c["displayName"]))
if key != "" {
byCompanyName[key] = c
}
continue
}
first := strings.TrimSpace(fmt.Sprint(c["firstName"]))
last := strings.TrimSpace(fmt.Sprint(c["lastName"]))
key := NormalizeName(first + " " + last)
if key == "" {
key = NormalizeName(fmt.Sprint(c["displayName"]))
}
if key != "" {
byPersonName[key] = c
}
_ = id
}
for i := range doc.Entries {
e := &doc.Entries[i]
// preserve approve
matched := false
conflict := false
var oo map[string]any
for _, em := range e.Emails {
if c, ok := byEmail[NormalizeEmail(em)]; ok {
if oo != nil && contactIDString(oo) != contactIDString(c) {
conflict = true
}
oo = c
matched = true
}
}
if !matched {
if e.Kind == "company" {
if c, ok := byCompanyName[onlyoffice.CompanyGroupingKey(e.Name)]; ok {
oo = c
matched = true
}
} else {
key := NormalizeName(strings.TrimSpace(e.First + " " + e.Last))
if key == "" {
key = NormalizeName(e.Name)
}
if c, ok := byPersonName[key]; ok {
oo = c
matched = true
}
}
}
if conflict {
e.Status = "conflict"
e.Notes = strings.TrimSpace(e.Notes + " email_matches_multiple_oo")
if oo != nil {
e.OOID = contactIDString(oo)
}
continue
}
if matched && oo != nil {
e.Status = "exists"
e.OOID = contactIDString(oo)
continue
}
// Keep a previously applied oo_id (list payloads often omit emails, so
// email match can miss persons that already exist in CRM).
if strings.TrimSpace(e.OOID) != "" {
e.Status = "exists"
continue
}
e.Status = "new"
e.OOID = ""
}
return nil
}
func contactIDString(c map[string]any) string {
switch v := c["id"].(type) {
case float64:
return strconv.FormatInt(int64(v), 10)
case int:
return strconv.Itoa(v)
case string:
return v
default:
return fmt.Sprint(v)
}
}
func contactEmails(c map[string]any) []string {
var out []string
if em := strings.TrimSpace(fmt.Sprint(c["email"])); em != "" && em != "<nil>" {
out = append(out, em)
}
if em := strings.TrimSpace(fmt.Sprint(c["primaryEmail"])); em != "" && em != "<nil>" {
out = append(out, em)
}
for _, row := range onlyoffice.ContactInfoRows(c) {
if onlyoffice.NormalizeContactInfoType(fmt.Sprint(row["infoType"])) != "email" {
continue
}
data := strings.TrimSpace(fmt.Sprint(row["data"]))
if data != "" {
out = append(out, data)
}
}
return uniqueEmails(out)
}
+26
View File
@@ -0,0 +1,26 @@
package catalog
import (
"testing"
)
func TestMatchAgainstOO_emailThenName(t *testing.T) {
// MatchAgainstOO needs a live client; unit-test the helpers used by matching.
c := map[string]any{
"id": float64(42),
"isCompany": false,
"firstName": "Ada",
"lastName": "Lovelace",
"email": "ada@example.com",
"commonData": []any{
map[string]any{"infoType": "Email", "data": "ada.alt@example.com"},
},
}
if contactIDString(c) != "42" {
t.Fatal(contactIDString(c))
}
emails := contactEmails(c)
if len(emails) < 2 {
t.Fatalf("%v", emails)
}
}
+244
View File
@@ -0,0 +1,244 @@
package catalog
import (
"bufio"
"fmt"
"io"
"os"
"path/filepath"
"strings"
)
// ScanOptions controls Thunderbird / mbox extraction depth.
type ScanOptions struct {
// MboxHeaders, when true, also walks mbox-like files under the root and
// extracts From/To/Cc/Reply-To addresses (headers only, no bodies).
MboxHeaders bool
// MboxMaxBytes skips individual mbox files larger than this (0 = 256 MiB default).
MboxMaxBytes int64
// Classifier supplies deployment classification rules; nil → neutral default.
Classifier *Classifier
}
// ScanThunderbirdRoot finds Thunderbird profiles under root and emits person rows
// from address books (*.mab) and Gloda global-messages-db.sqlite.
func ScanThunderbirdRoot(root string) (*Document, error) {
return ScanThunderbirdRootOpts(root, ScanOptions{})
}
// ScanThunderbirdRootOpts is ScanThunderbirdRoot with optional mbox header pass.
func ScanThunderbirdRootOpts(root string, opts ScanOptions) (*Document, error) {
root = filepath.Clean(root)
st, err := os.Stat(root)
if err != nil {
return nil, err
}
if !st.IsDir() {
return nil, fmt.Errorf("not a directory: %s", root)
}
if opts.MboxMaxBytes <= 0 {
opts.MboxMaxBytes = 256 << 20
}
cl := opts.Classifier
if cl == nil {
cl = DefaultClassifier()
}
var entries []Entry
seenDB := map[string]struct{}{}
seenMAB := map[string]struct{}{}
seenMbox := map[string]struct{}{}
err = filepath.WalkDir(root, func(path string, d os.DirEntry, err error) error {
if err != nil {
return nil
}
name := d.Name()
if d.IsDir() {
switch name {
case "Cache", "cache2", "startupCache", "OfflineCache", "minidumps",
"crashes", "safebrowsing", "thumbnails", "chrome", "extensions",
"node_modules", ".git":
return filepath.SkipDir
}
return nil
}
lower := strings.ToLower(name)
switch {
case lower == "global-messages-db.sqlite":
if _, ok := seenDB[path]; ok {
return nil
}
seenDB[path] = struct{}{}
parsed, perr := parseGlodaContacts(path, cl)
if perr != nil {
entries = append(entries, Entry{
ID: EntryID("person", "", filepath.Base(path)),
Kind: "person",
Name: "gloda",
Sources: []string{path},
Zone: "private",
Role: "unknown",
Notes: "gloda_parse_error: " + perr.Error(),
Status: "new",
})
return nil
}
entries = append(entries, parsed...)
case strings.HasSuffix(lower, ".mab"):
if _, ok := seenMAB[path]; ok {
return nil
}
seenMAB[path] = struct{}{}
parsed, perr := parseMABEmails(path, cl)
if perr != nil {
return nil
}
entries = append(entries, parsed...)
case opts.MboxHeaders && isLikelyMboxFile(name, path):
if _, ok := seenMbox[path]; ok {
return nil
}
seenMbox[path] = struct{}{}
info, ierr := d.Info()
if ierr != nil {
return nil
}
if info.Size() > opts.MboxMaxBytes {
return nil
}
parsed, perr := parseMboxHeaderEmails(path, cl)
if perr != nil {
return nil
}
entries = append(entries, parsed...)
}
return nil
})
if err != nil {
return nil, err
}
return MergeDocs(&Document{Entries: entries}), nil
}
func isLikelyMboxFile(name, path string) bool {
lower := strings.ToLower(name)
if strings.HasSuffix(lower, ".msf") || strings.HasSuffix(lower, ".sqlite") ||
strings.HasSuffix(lower, ".mab") || strings.HasSuffix(lower, ".json") ||
strings.HasSuffix(lower, ".dat") || strings.HasSuffix(lower, ".ini") ||
strings.HasSuffix(lower, ".log") {
return false
}
if strings.HasSuffix(lower, ".mbox") || strings.HasSuffix(lower, ".mbx") {
return true
}
// Thunderbird: …/ImapMail/<server>/INBOX or …/Mail/Local Folders/Inbox
// or …/INBOX.sbd/<folder>
sep := string(filepath.Separator)
norm := filepath.ToSlash(path)
if strings.Contains(norm, "/ImapMail/") || strings.Contains(norm, "/Mail/") {
if !strings.Contains(name, ".") {
return true
}
known := map[string]struct{}{
"inbox": {}, "sent": {}, "drafts": {}, "trash": {}, "archives": {}, "junk": {},
}
if _, ok := known[lower]; ok {
return true
}
}
if strings.Contains(filepath.Base(filepath.Dir(path)), ".sbd") && !strings.Contains(name, ".") {
return true
}
_ = sep
return false
}
// parseMboxHeaderEmails extracts addresses from From/To/Cc/Reply-To headers only.
func parseMboxHeaderEmails(path string, cl *Classifier) ([]Entry, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
emails := map[string]struct{}{}
br := bufio.NewReaderSize(f, 1<<20)
inHeaders := false
var headerBuf strings.Builder
flushHeaders := func() {
if headerBuf.Len() == 0 {
return
}
block := headerBuf.String()
headerBuf.Reset()
for _, line := range strings.Split(block, "\n") {
lower := strings.ToLower(strings.TrimSpace(line))
if strings.HasPrefix(lower, "from:") || strings.HasPrefix(lower, "to:") ||
strings.HasPrefix(lower, "cc:") || strings.HasPrefix(lower, "reply-to:") ||
strings.HasPrefix(lower, "sender:") {
for _, m := range emailRE.FindAllString(line, -1) {
em := NormalizeEmail(m)
if !noisyEmail(em) {
emails[em] = struct{}{}
}
}
}
}
}
for {
line, err := br.ReadString('\n')
if len(line) > 0 {
// Classic mbox separator
if strings.HasPrefix(line, "From ") && (len(line) == 5 || line[5] != ':') {
flushHeaders()
inHeaders = true
continue
}
if inHeaders {
trimmed := strings.TrimRight(line, "\r\n")
if trimmed == "" {
flushHeaders()
inHeaders = false
continue
}
headerBuf.WriteString(trimmed)
headerBuf.WriteByte('\n')
// cap pathological headers
if headerBuf.Len() > 64<<10 {
flushHeaders()
inHeaders = false
}
}
}
if err == io.EOF {
flushHeaders()
break
}
if err != nil {
flushHeaders()
return nil, err
}
}
var out []Entry
for em := range emails {
org, zone, role := cl.ClassifyMail("", em)
out = append(out, Entry{
ID: EntryID("person", em, ""),
Kind: "person",
Emails: []string{em},
Org: org,
Sources: []string{path},
Zone: zone,
Role: role,
Approve: false,
Status: "new",
Notes: "thunderbird_mbox_header",
})
}
return out, nil
}
+67
View File
@@ -0,0 +1,67 @@
package catalog
import (
"os"
"path/filepath"
"testing"
)
func TestParseMboxHeaderEmails(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "INBOX")
body := `From - Mon Jul 1 00:00:00 2016
From: Alice Smith <alice.smith@acme.example>
To: Bob Jones <bob.jones@acme.example>
Cc: noreply@example.com, client@stadt-example.de
Subject: test
Body line ignored
From - Mon Jul 2 00:00:00 2016
From: Someone <paul.schmidt@acme.example>
To: list@acme.example
more body
`
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
ents, err := parseMboxHeaderEmails(path, DefaultClassifier())
if err != nil {
t.Fatal(err)
}
got := map[string]bool{}
for _, e := range ents {
if len(e.Emails) > 0 {
got[e.Emails[0]] = true
}
}
if !got["alice.smith@acme.example"] || !got["bob.jones@acme.example"] {
t.Fatalf("%v", got)
}
if got["noreply@example.com"] {
t.Fatal("noreply should be filtered")
}
if !got["client@stadt-example.de"] {
t.Fatal("missing client")
}
}
func TestScanThunderbirdRootOptsMbox(t *testing.T) {
root := t.TempDir()
imap := filepath.Join(root, "ImapMail", "mail.example.com")
if err := os.MkdirAll(imap, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(imap, "INBOX"), []byte(
"From - x\nFrom: a@acme.example\nTo: b@acme.example\n\nbody\n",
), 0o644); err != nil {
t.Fatal(err)
}
doc, err := ScanThunderbirdRootOpts(root, ScanOptions{MboxHeaders: true})
if err != nil {
t.Fatal(err)
}
if len(doc.Entries) < 2 {
t.Fatalf("%+v", doc.Entries)
}
}
+152
View File
@@ -0,0 +1,152 @@
package catalog
import (
"regexp"
"strings"
"unicode"
)
var (
parenSuffixRE = regexp.MustCompile(`(?i)\s*[\(\[\{][^)\]\}]*[\)\]\}]\s*$`)
dashCompanyRE = regexp.MustCompile(`(?i)\s+[-–—]\s+[A-Za-z0-9].*$`)
emailLocalRE = regexp.MustCompile(`(?i)^[a-z0-9._%+\-]+@[a-z0-9.\-]+\.[a-z]{2,}$`)
nonNameTokenRE = regexp.MustCompile(`[^a-zA-ZÀ-öø-ÿĀ-ž0-9'’.\-]+`)
)
// CleanPersonNames strips company annotations from display names and fills
// first/last from the email local-part when the source used an address as the
// name. Company affiliation belongs on Org / the CRM companyId — never in LastName.
func CleanPersonNames(first, last, display, org string, emails []string) (cleanFirst, cleanLast string) {
first = strings.TrimSpace(first)
last = strings.TrimSpace(last)
display = strings.TrimSpace(display)
org = strings.TrimSpace(org)
if looksLikeEmail(first) {
ef, el := GuessNameFromEmail(first)
first, last = ef, el
}
if looksLikeEmail(display) && first == "" && last == "" {
display = ""
}
if first == "" && last == "" && display != "" {
first, last = SplitDisplayName(display)
}
first = stripCompanyAnnotation(first, org)
last = stripCompanyAnnotation(last, org)
// "Smith - Acme" / "Jones (Acme)" landed in last.
last = stripCompanyAnnotation(last, org)
if i := strings.IndexAny(first, "(["); i > 0 {
first = strings.TrimSpace(first[:i])
}
// Entire last name is just the company (e.g. last="Acme").
if org != "" && personLastIsOrg(last, org) {
last = ""
}
if (first == "" || looksLikeEmail(first)) && len(emails) > 0 {
ef, el := GuessNameFromEmail(emails[0])
if first == "" || looksLikeEmail(first) {
first = ef
}
if last == "" || last == "-" {
last = el
}
}
first = strings.TrimSpace(first)
last = strings.TrimSpace(last)
if last == "" {
last = "-"
}
return first, last
}
func stripCompanyAnnotation(s, org string) string {
s = strings.TrimSpace(s)
if s == "" {
return ""
}
s = parenSuffixRE.ReplaceAllString(s, "")
s = strings.TrimSpace(s)
s = dashCompanyRE.ReplaceAllString(s, "")
s = strings.TrimSpace(s)
if org != "" {
for _, sep := range []string{" - ", " – ", " — ", " / "} {
if i := strings.LastIndex(strings.ToLower(s), strings.ToLower(sep+org)); i >= 0 {
s = strings.TrimSpace(s[:i])
}
}
suf := " (" + org + ")"
if strings.HasSuffix(strings.ToLower(s), strings.ToLower(suf)) {
s = strings.TrimSpace(s[:len(s)-len(suf)])
}
}
return strings.TrimSpace(s)
}
func personLastIsOrg(last, org string) bool {
last = NormalizeName(last)
org = NormalizeName(org)
if last == "" || org == "" {
return false
}
if last == org {
return true
}
// "Acme" vs "Acme GmbH & Co. KG"
return strings.HasPrefix(org, last+" ") || strings.HasPrefix(org, last+",")
}
func looksLikeEmail(s string) bool {
return emailLocalRE.MatchString(strings.TrimSpace(s))
}
// GuessNameFromEmail turns local@domain into Title-Case first/last when the
// local part looks like first.last / first_last / first-last.
func GuessNameFromEmail(email string) (first, last string) {
email = NormalizeEmail(email)
local, _, ok := strings.Cut(email, "@")
if !ok || local == "" {
return "", ""
}
local = strings.Split(local, "+")[0]
parts := strings.FieldsFunc(local, func(r rune) bool {
return r == '.' || r == '_' || r == '-'
})
if len(parts) == 0 {
return titleToken(local), ""
}
if len(parts) == 1 {
return titleToken(parts[0]), ""
}
return titleToken(parts[0]), titleToken(strings.Join(parts[1:], " "))
}
func titleToken(s string) string {
s = nonNameTokenRE.ReplaceAllString(s, " ")
s = strings.TrimSpace(s)
if s == "" {
return ""
}
runes := []rune(strings.ToLower(s))
runes[0] = unicode.ToTitle(runes[0])
return string(runes)
}
// FormatProjectTitle builds "CC | Company | Title" (spaces around |).
// Country should be a short code (DE, TF, UA, …). Empty segments are dropped.
func FormatProjectTitle(country, company, title string) string {
parts := make([]string, 0, 3)
for _, p := range []string{country, company, title} {
p = strings.TrimSpace(p)
p = strings.ReplaceAll(p, "|", "/")
if p != "" {
parts = append(parts, p)
}
}
return strings.Join(parts, " | ")
}
+41
View File
@@ -0,0 +1,41 @@
package catalog
import "testing"
func TestCleanPersonNamesStripsCompanyParen(t *testing.T) {
f, l := CleanPersonNames("John", "Smith (Acme)", "John Smith (Acme)", "Acme", nil)
if f != "John" || l != "Smith" {
t.Fatalf("got %q %q", f, l)
}
}
func TestCleanPersonNamesStripsDashCompany(t *testing.T) {
f, l := CleanPersonNames("Jens", "Meyer - Acme", "", "Acme", nil)
if f != "Jens" || l != "Meyer" {
t.Fatalf("got %q %q", f, l)
}
}
func TestCleanPersonNamesFromEmail(t *testing.T) {
f, l := CleanPersonNames("david.patzke@acme.example", "-", "", "Acme",
[]string{"david.patzke@acme.example"})
if f != "David" || l != "Patzke" {
t.Fatalf("got %q %q", f, l)
}
}
func TestCleanPersonNamesLastIsCompany(t *testing.T) {
f, l := CleanPersonNames("Thorsten", "Acme", "", "Acme GmbH & Co. KG", nil)
if f != "Thorsten" || l != "-" {
t.Fatalf("got %q %q", f, l)
}
}
func TestFormatProjectTitle(t *testing.T) {
if got := FormatProjectTitle("DE", "Acme", "Golang"); got != "DE | Acme | Golang" {
t.Fatalf("got %q", got)
}
if got := FormatProjectTitle("", "Acme", ""); got != "Acme" {
t.Fatalf("got %q", got)
}
}
+117
View File
@@ -0,0 +1,117 @@
package catalog
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
)
// ScanProjectsRoot finds git roots under root (max depth) and emits company rows.
func ScanProjectsRoot(root string, maxDepth int) (*Document, error) {
return ScanProjectsRootOpts(root, maxDepth, ScanOptions{})
}
// ScanProjectsRootOpts is ScanProjectsRoot with deployment classification rules.
func ScanProjectsRootOpts(root string, maxDepth int, opts ScanOptions) (*Document, error) {
root = filepath.Clean(root)
if maxDepth <= 0 {
maxDepth = 4
}
cl := opts.Classifier
if cl == nil {
cl = DefaultClassifier()
}
st, err := os.Stat(root)
if err != nil {
return nil, err
}
if !st.IsDir() {
return nil, fmt.Errorf("not a directory: %s", root)
}
var entries []Entry
err = walkGitRoots(root, root, 0, maxDepth, cl, &entries)
if err != nil {
return nil, err
}
// Also top-level dirs as company stubs (even without git) — coverage 1B.
dents, _ := os.ReadDir(root)
for _, d := range dents {
if !d.IsDir() || strings.HasPrefix(d.Name(), ".") {
continue
}
name := d.Name()
skip := map[string]struct{}{
"node_modules": {}, "vendor": {}, ".cache": {},
}
if _, ok := skip[name]; ok {
continue
}
path := filepath.Join(root, name)
id := EntryID("company", "", name)
role, zone := cl.ClassifyProject(name, "")
entries = append(entries, Entry{
ID: id,
Kind: "company",
Name: name,
Sources: []string{path},
Zone: zone,
Role: role,
Status: "new",
Notes: "top_level_dir",
})
}
return MergeDocs(&Document{Entries: entries}), nil
}
func walkGitRoots(root, dir string, depth, maxDepth int, cl *Classifier, out *[]Entry) error {
if depth > maxDepth {
return nil
}
// is this a git root?
if st, err := os.Stat(filepath.Join(dir, ".git")); err == nil && (st.IsDir() || st.Mode().IsRegular()) {
name := filepath.Base(dir)
remote := gitRemoteOrigin(dir)
role, zone := cl.ClassifyProject(name, remote)
*out = append(*out, Entry{
ID: EntryID("company", "", name),
Kind: "company",
Name: name,
Sources: []string{dir},
Remote: remote,
GitRoot: dir,
Zone: zone,
Role: role,
Status: "new",
})
return nil // do not descend into nested repos from a git root
}
dents, err := os.ReadDir(dir)
if err != nil {
return nil
}
for _, d := range dents {
if !d.IsDir() {
continue
}
name := d.Name()
if name == ".git" || name == "node_modules" || name == "vendor" || name == ".venv" || name == "dist" {
continue
}
_ = walkGitRoots(root, filepath.Join(dir, name), depth+1, maxDepth, cl, out)
}
return nil
}
func gitRemoteOrigin(dir string) string {
cmd := exec.Command("git", "-C", dir, "remote", "get-url", "origin")
b, err := cmd.Output()
if err != nil {
return ""
}
return strings.TrimSpace(string(b))
}
+172
View File
@@ -0,0 +1,172 @@
package catalog
import (
"database/sql"
"os"
"regexp"
"strings"
_ "modernc.org/sqlite"
)
var emailRE = regexp.MustCompile(`(?i)[a-z0-9._%+\-]+@[a-z0-9.\-]+\.[a-z]{2,}`)
// noisyEmail reports addresses that should not become CRM catalog persons.
func noisyEmail(email string) bool {
e := NormalizeEmail(email)
if e == "" || !strings.Contains(e, "@") {
return true
}
local, domain, ok := strings.Cut(e, "@")
if !ok {
return true
}
noiseLocal := []string{
"noreply", "no-reply", "donotreply", "mailer-daemon", "postmaster",
"bounce", "notifications", "newsletter", "unsubscribe",
"root", "admin", "abuse", "webmaster", "hostmaster",
}
for _, p := range noiseLocal {
if local == p || strings.Contains(local, p) {
return true
}
}
// role / shared mailboxes (keep human dotted names)
roleExact := map[string]struct{}{
"info": {}, "alle": {}, "all": {}, "office": {}, "office@": {},
"wartung": {}, "starface": {}, "gitlab": {}, "pl": {}, "gf": {},
"umsetzung": {}, "support": {}, "sales": {}, "billing": {},
}
if _, ok := roleExact[local]; ok {
return true
}
if strings.HasSuffix(local, "-request") || strings.HasSuffix(local, "-owner") {
return true
}
noiseDomain := []string{
"marketplace.amazon.", "reply.github.com", "users.noreply.github.com",
"groups.facebook.com", "googlegroups.com", "yahoogroups.",
"email.apple.com", "amazonses.com", "mailchimp.com",
"sendgrid.net", "mailgun.org", "postmarkapp.com",
"property.booking.com", "transvascular.com",
}
for _, p := range noiseDomain {
if strings.Contains(domain, p) {
return true
}
}
// random marketplace / tracking locals
if len(local) >= 16 && !strings.Contains(local, ".") && !strings.Contains(local, "-") {
if strings.Contains(domain, "amazon.") {
return true
}
}
return false
}
func parseMABEmails(path string, cl *Classifier) ([]Entry, error) {
b, err := os.ReadFile(path)
if err != nil {
return nil, err
}
found := emailRE.FindAllString(string(b), -1)
var out []Entry
for _, raw := range found {
em := NormalizeEmail(raw)
if noisyEmail(em) {
continue
}
org, zone, role := cl.ClassifyMail("", em)
out = append(out, Entry{
ID: EntryID("person", em, ""),
Kind: "person",
Emails: []string{em},
Org: org,
Sources: []string{path},
Zone: zone,
Role: role,
Approve: false,
Status: "new",
Notes: "thunderbird_mab",
})
}
return out, nil
}
func parseGlodaContacts(dbPath string, cl *Classifier) ([]Entry, error) {
// read-only URI; immutable=1 helps when WAL/shm are missing
dsn := "file:" + dbPath + "?mode=ro&_pragma=query_only(1)"
db, err := sql.Open("sqlite", dsn)
if err != nil {
return nil, err
}
defer db.Close()
rows, err := db.Query(`
SELECT COALESCE(c.name, ''), i.value
FROM identities i
LEFT JOIN contacts c ON c.id = i.contactID
WHERE lower(i.kind) = 'email' AND i.value IS NOT NULL AND i.value != ''
`)
if err != nil {
return nil, err
}
defer rows.Close()
var out []Entry
for rows.Next() {
var name, value string
if err := rows.Scan(&name, &value); err != nil {
return out, err
}
em := NormalizeEmail(value)
// Gloda sometimes stores "Name <email>" in value
if m := emailRE.FindString(em); m != "" {
em = NormalizeEmail(m)
}
if noisyEmail(em) {
continue
}
display := strings.TrimSpace(name)
// strip wrapping quotes / email leftovers
display = strings.Trim(display, `"'`)
if idx := strings.Index(display, "<"); idx > 0 {
display = strings.TrimSpace(display[:idx])
}
first, last := "", ""
if display != "" {
first, last = SplitDisplayName(display)
}
org, zone, role := cl.ClassifyMail(display, em)
out = append(out, Entry{
ID: EntryID("person", em, display),
Kind: "person",
Name: display,
First: first,
Last: last,
Emails: []string{em},
Org: org,
Sources: []string{dbPath},
Zone: zone,
Role: role,
Approve: false,
Status: "new",
Notes: "thunderbird_gloda",
})
}
return out, rows.Err()
}
func looksPublicSector(domain string) bool {
d := strings.ToLower(domain)
hints := []string{
"stadt-", "stadt.", "kreis-", "gemeinde", "landkreis",
"bund.", "bundes", "lvermgeo", "rvr.", "eba.",
}
for _, h := range hints {
if strings.Contains(d, h) {
return true
}
}
return false
}
+109
View File
@@ -0,0 +1,109 @@
package catalog
import (
"database/sql"
"os"
"path/filepath"
"testing"
_ "modernc.org/sqlite"
)
func TestNoisyEmail(t *testing.T) {
if !noisyEmail("noreply@example.com") {
t.Fatal("expected noisy")
}
if !noisyEmail("x@marketplace.amazon.de") {
t.Fatal("amazon marketplace")
}
if noisyEmail("alice.smith@acme.example") {
t.Fatal("should keep a human work address")
}
}
func TestParseMABEmails(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "abook.mab")
body := `// mork junk
PrimaryEmail=alice.smith@acme.example
noreply@github.com
bob.jones@acme.example
`
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
// Neutral default: no deployment rules → unclassified.
ents, err := parseMABEmails(path, DefaultClassifier())
if err != nil {
t.Fatal(err)
}
if len(ents) != 2 {
t.Fatalf("got %d: %+v", len(ents), ents)
}
for _, e := range ents {
if e.Org != "" || e.Role != "unknown" {
t.Fatalf("%+v", e)
}
}
// A deployment rule classifies the domain as work.
cl := &Classifier{MailOrgs: []MailRule{{Domain: "acme.example", Org: "Acme", Zone: "warm", Role: "work"}}}
ents, err = parseMABEmails(path, cl)
if err != nil {
t.Fatal(err)
}
for _, e := range ents {
if e.Org != "Acme" || e.Role != "work" || e.Zone != "warm" {
t.Fatalf("%+v", e)
}
}
}
func TestParseGlodaContacts(t *testing.T) {
dir := t.TempDir()
dbPath := filepath.Join(dir, "global-messages-db.sqlite")
db, err := sql.Open("sqlite", dbPath)
if err != nil {
t.Fatal(err)
}
_, err = db.Exec(`
CREATE TABLE contacts (id INTEGER PRIMARY KEY, name TEXT);
CREATE TABLE identities (id INTEGER PRIMARY KEY, contactID INTEGER, kind TEXT, value TEXT);
INSERT INTO contacts VALUES (1, 'Alice Smith');
INSERT INTO identities VALUES (1, 1, 'email', 'alice.smith@acme.example');
INSERT INTO contacts VALUES (2, 'Noise Bot');
INSERT INTO identities VALUES (2, 2, 'email', 'noreply@example.com');
`)
if err != nil {
t.Fatal(err)
}
_ = db.Close()
ents, err := parseGlodaContacts(dbPath, DefaultClassifier())
if err != nil {
t.Fatal(err)
}
if len(ents) != 1 {
t.Fatalf("got %d %+v", len(ents), ents)
}
if ents[0].First != "Alice" || ents[0].Emails[0] != "alice.smith@acme.example" {
t.Fatalf("%+v", ents[0])
}
}
func TestScanThunderbirdRoot(t *testing.T) {
root := t.TempDir()
prof := filepath.Join(root, "profile")
if err := os.MkdirAll(prof, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(prof, "abook.mab"), []byte("mail=paul.schmidt@acme.example\n"), 0o644); err != nil {
t.Fatal(err)
}
doc, err := ScanThunderbirdRoot(root)
if err != nil {
t.Fatal(err)
}
if len(doc.Entries) < 1 {
t.Fatal(doc.Entries)
}
}
+244
View File
@@ -0,0 +1,244 @@
// Package catalog builds a YAML inventory of CRM persons/companies from
// filesystem contacts (VCF, folders) and project trees, then matches/applies
// against OnlyOffice CRM.
package catalog
import (
"fmt"
"os"
"strings"
"time"
"gopkg.in/yaml.v3"
)
// Address is one postal address of a catalog row. Category is the OO
// AddressCategory label (Home|Postal|Office|Billing|Other|Work).
type Address struct {
Street string `yaml:"street,omitempty"`
City string `yaml:"city,omitempty"`
State string `yaml:"state,omitempty"`
Zip string `yaml:"zip,omitempty"`
Country string `yaml:"country,omitempty"`
Category string `yaml:"category,omitempty"`
Primary bool `yaml:"primary,omitempty"`
}
// Entry is one catalog row (person or company).
type Entry struct {
ID string `yaml:"id"`
Kind string `yaml:"kind"` // person | company
Name string `yaml:"name,omitempty"`
First string `yaml:"first,omitempty"`
Last string `yaml:"last,omitempty"`
Emails []string `yaml:"emails,omitempty"`
Phones []string `yaml:"phones,omitempty"`
Addresses []Address `yaml:"addresses,omitempty"`
Org string `yaml:"org,omitempty"`
Sources []string `yaml:"sources,omitempty"`
Zone string `yaml:"zone"`
Role string `yaml:"role"`
OOID string `yaml:"oo_id,omitempty"`
Approve bool `yaml:"approve"`
Status string `yaml:"status,omitempty"` // new | exists | conflict | applied | skipped
Notes string `yaml:"notes,omitempty"`
Remote string `yaml:"remote,omitempty"`
GitRoot string `yaml:"git_root,omitempty"`
}
// Document is the on-disk catalog file.
type Document struct {
GeneratedAt string `yaml:"generated_at"`
Entries []Entry `yaml:"entries"`
}
// LoadYAML reads a catalog document from path.
func LoadYAML(path string) (*Document, error) {
b, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var doc Document
if err := yaml.Unmarshal(b, &doc); err != nil {
return nil, err
}
return &doc, nil
}
// SaveYAML writes the catalog document.
func SaveYAML(path string, doc *Document) error {
doc.GeneratedAt = time.Now().UTC().Format(time.RFC3339)
b, err := yaml.Marshal(doc)
if err != nil {
return err
}
return os.WriteFile(path, b, 0o644)
}
// NormalizeEmail lowercases and trims.
func NormalizeEmail(s string) string {
return strings.ToLower(strings.TrimSpace(s))
}
// NormalizeName collapses whitespace and lowercases for matching.
func NormalizeName(s string) string {
fields := strings.Fields(strings.ToLower(strings.TrimSpace(s)))
return strings.Join(fields, " ")
}
// SplitDisplayName splits "First Last …" into first/last (last = remainder).
func SplitDisplayName(name string) (first, last string) {
parts := strings.Fields(strings.TrimSpace(name))
if len(parts) == 0 {
return "", ""
}
if len(parts) == 1 {
return parts[0], ""
}
return parts[0], strings.Join(parts[1:], " ")
}
// EntryID builds a stable-ish id from kind + email or name.
func EntryID(kind, email, name string) string {
if e := NormalizeEmail(email); e != "" {
return fmt.Sprintf("%s:%s", kind, e)
}
return fmt.Sprintf("%s:%s", kind, NormalizeName(name))
}
// MergeDocs unions entries by id; later sources append sources[] and fill empties.
func MergeDocs(docs ...*Document) *Document {
byID := map[string]*Entry{}
order := []string{}
for _, d := range docs {
if d == nil {
continue
}
for i := range d.Entries {
e := d.Entries[i]
id := e.ID
if id == "" {
primary := ""
if len(e.Emails) > 0 {
primary = e.Emails[0]
}
name := e.Name
if name == "" {
name = strings.TrimSpace(e.First + " " + e.Last)
}
id = EntryID(e.Kind, primary, name)
e.ID = id
}
if prev, ok := byID[id]; ok {
mergeEntry(prev, &e)
} else {
cp := e
byID[id] = &cp
order = append(order, id)
}
}
}
out := &Document{Entries: make([]Entry, 0, len(order))}
for _, id := range order {
out.Entries = append(out.Entries, *byID[id])
}
return out
}
func mergeEntry(dst, src *Entry) {
dst.Sources = uniqueStrings(append(dst.Sources, src.Sources...))
dst.Emails = uniqueEmails(append(dst.Emails, src.Emails...))
dst.Phones = uniqueStrings(append(dst.Phones, src.Phones...))
dst.Addresses = mergeAddresses(dst.Addresses, src.Addresses)
if dst.First == "" {
dst.First = src.First
}
if dst.Last == "" {
dst.Last = src.Last
}
if dst.Name == "" {
dst.Name = src.Name
}
if dst.Org == "" {
dst.Org = src.Org
}
if dst.Remote == "" {
dst.Remote = src.Remote
}
if dst.GitRoot == "" {
dst.GitRoot = src.GitRoot
}
if dst.Notes == "" {
dst.Notes = src.Notes
}
// Prefer more specific zone/role from project scans over default private.
if dst.Zone == "private" && src.Zone != "" && src.Zone != "private" {
dst.Zone = src.Zone
}
if dst.Role == "" || (dst.Role == "unknown" && src.Role != "" && src.Role != "unknown") {
dst.Role = src.Role
}
// Preserve approve/oo_id/status from whichever already set.
if !dst.Approve && src.Approve {
dst.Approve = true
}
if dst.OOID == "" {
dst.OOID = src.OOID
}
if dst.Status == "" {
dst.Status = src.Status
}
}
// mergeAddresses appends src addresses not already present (by street/city/zip).
func mergeAddresses(dst, src []Address) []Address {
for _, a := range src {
exists := false
for _, b := range dst {
if strings.EqualFold(strings.TrimSpace(a.Street), strings.TrimSpace(b.Street)) &&
strings.EqualFold(strings.TrimSpace(a.City), strings.TrimSpace(b.City)) &&
strings.EqualFold(strings.TrimSpace(a.Zip), strings.TrimSpace(b.Zip)) {
exists = true
break
}
}
if !exists {
dst = append(dst, a)
}
}
return dst
}
func uniqueEmails(in []string) []string {
seen := map[string]struct{}{}
var out []string
for _, e := range in {
e = NormalizeEmail(e)
if e == "" {
continue
}
if _, ok := seen[e]; ok {
continue
}
seen[e] = struct{}{}
out = append(out, e)
}
return out
}
func uniqueStrings(in []string) []string {
seen := map[string]struct{}{}
var out []string
for _, s := range in {
s = strings.TrimSpace(s)
if s == "" {
continue
}
if _, ok := seen[s]; ok {
continue
}
seen[s] = struct{}{}
out = append(out, s)
}
return out
}
+218
View File
@@ -0,0 +1,218 @@
package catalog
import (
"bufio"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"github.com/emersion/go-vcard"
)
// ParseVCFFile extracts person candidates from a vCard file (may contain many cards).
func ParseVCFFile(path string) ([]Entry, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
dec := vcard.NewDecoder(f)
var out []Entry
for {
card, err := dec.Decode()
if err != nil {
if errors.Is(err, io.EOF) {
break
}
return out, fmt.Errorf("%s: %w", path, err)
}
e := entryFromCard(card, path)
if e.Name == "" && e.First == "" && len(e.Emails) == 0 {
continue
}
out = append(out, e)
}
return out, nil
}
func entryFromCard(card vcard.Card, source string) Entry {
fn := strings.TrimSpace(card.PreferredValue(vcard.FieldFormattedName))
n := card.Name()
first, last := "", ""
if n != nil {
first = strings.TrimSpace(n.GivenName)
last = strings.TrimSpace(n.FamilyName)
}
if fn == "" {
fn = strings.TrimSpace(first + " " + last)
}
if first == "" && last == "" && fn != "" {
first, last = SplitDisplayName(fn)
}
var emails, phones []string
for _, v := range card.Values(vcard.FieldEmail) {
if e := NormalizeEmail(v); e != "" {
emails = append(emails, e)
}
}
for _, v := range card.Values(vcard.FieldTelephone) {
p := strings.TrimSpace(v)
if p != "" {
phones = append(phones, p)
}
}
org := strings.TrimSpace(card.PreferredValue(vcard.FieldOrganization))
primary := ""
if len(emails) > 0 {
primary = emails[0]
}
return Entry{
ID: EntryID("person", primary, fn),
Kind: "person",
Name: fn,
First: first,
Last: last,
Emails: uniqueEmails(emails),
Phones: uniqueStrings(phones),
Org: org,
Sources: []string{source},
Zone: "private",
Role: "unknown",
Approve: false,
Status: "new",
}
}
// ScanContactsRoot walks a contacts directory: *.vcf anywhere, person-named
// subdirs, and *@*.txt email hint files.
func ScanContactsRoot(root string) (*Document, error) {
root = filepath.Clean(root)
st, err := os.Stat(root)
if err != nil {
return nil, err
}
if !st.IsDir() {
return nil, fmt.Errorf("not a directory: %s", root)
}
var entries []Entry
seenVCF := map[string]struct{}{}
err = filepath.WalkDir(root, func(path string, d os.DirEntry, err error) error {
if err != nil {
return nil // skip unreadable
}
if d.IsDir() {
base := d.Name()
if base == ".git" || base == "node_modules" {
return filepath.SkipDir
}
return nil
}
name := d.Name()
lower := strings.ToLower(name)
if strings.HasSuffix(lower, ".vcf") {
if _, ok := seenVCF[path]; ok {
return nil
}
seenVCF[path] = struct{}{}
parsed, perr := ParseVCFFile(path)
if perr != nil {
entries = append(entries, Entry{
ID: EntryID("person", "", filepath.Base(path)),
Kind: "person",
Name: strings.TrimSuffix(filepath.Base(path), filepath.Ext(path)),
Sources: []string{path},
Zone: "private",
Role: "unknown",
Notes: "vcf_parse_error: " + perr.Error(),
Status: "new",
})
return nil
}
entries = append(entries, parsed...)
return nil
}
// email hint files: name contains @ and ends with .txt
if strings.Contains(name, "@") && strings.HasSuffix(lower, ".txt") {
email := NormalizeEmail(strings.TrimSuffix(name, filepath.Ext(name)))
if !strings.Contains(email, "@") {
return nil
}
parent := filepath.Base(filepath.Dir(path))
first, last := SplitDisplayName(parent)
entries = append(entries, Entry{
ID: EntryID("person", email, parent),
Kind: "person",
Name: parent,
First: first,
Last: last,
Emails: []string{email},
Sources: []string{path},
Zone: "private",
Role: "unknown",
Status: "new",
})
}
return nil
})
if err != nil {
return nil, err
}
// Top-level person folders without VCF still become stub persons.
dents, _ := os.ReadDir(root)
for _, d := range dents {
if !d.IsDir() {
continue
}
name := d.Name()
if name == "Contacts VCF's" || strings.HasPrefix(name, ".") {
continue
}
// skip obvious non-person dumps
lower := strings.ToLower(name)
if strings.Contains(lower, "vcf") {
continue
}
first, last := SplitDisplayName(name)
id := EntryID("person", "", name)
entries = append(entries, Entry{
ID: id,
Kind: "person",
Name: name,
First: first,
Last: last,
Sources: []string{filepath.Join(root, name)},
Zone: "private",
Role: "unknown",
Status: "new",
Notes: "folder_stub",
})
}
doc := MergeDocs(&Document{Entries: entries})
return doc, nil
}
// ReadEmailsFromTxt reads loose emails from a text file (one per line or free text).
func ReadEmailsFromTxt(path string) ([]string, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
var out []string
sc := bufio.NewScanner(f)
for sc.Scan() {
line := strings.TrimSpace(sc.Text())
if strings.Contains(line, "@") && !strings.Contains(line, " ") {
out = append(out, NormalizeEmail(line))
}
}
return uniqueEmails(out), sc.Err()
}
+12 -2
View File
@@ -11,8 +11,10 @@ package onlyoffice
import (
"net/http"
"net/http/cookiejar"
"os"
"strings"
"sync"
)
// Client of OnlyOffice API uses credentials to get a token and query the API
@@ -31,12 +33,20 @@ type Client struct {
defaults Defaults // optional fallbacks for calendar/project IDs
selfID string // cached /api/2.0/people/@self id
noteCatID int // cached CRM history category id for "note"
folderTitles map[string]string // cached Documents folder id -> title (F9)
folderTitlesMu sync.Mutex
}
// NewClient returns a new Client backed by http.DefaultClient.
// NewClient returns a new Client whose transport is paced by the process-wide
// rate limiter and 429 cooldown gate (OO_RATE_LIMIT/OO_BURST; see ratelimit.go).
func NewClient(c Credentials) *Client {
jar, _ := cookiejar.New(nil)
return &Client{
client: http.DefaultClient,
client: &http.Client{
Jar: jar,
Transport: &pacedTransport{base: http.DefaultTransport},
},
credentials: &c,
}
}
+18
View File
@@ -139,6 +139,23 @@ func TestIntegrationProjectAndTaskLifecycle(t *testing.T) {
start := Time(time.Now().AddDate(0, 0, -2))
deadline := Time(time.Now().AddDate(0, 0, 2))
ms, err := c.CreateMilestone(NewMilestoneRequest{
ProjectID: *project.ID,
Title: "integration milestone",
Deadline: deadline,
})
if err != nil {
t.Fatalf("CreateMilestone: %v", err)
}
if ms == nil || ms.ID == nil {
t.Fatal("created milestone without id")
}
t.Cleanup(func() {
if err := c.DeleteMilestone(*ms.ID); err != nil {
t.Logf("cleanup DeleteMilestone: %v", err)
}
})
task, err := c.CreateProjectTask(NewProjectTaskRequest{
ProjectId: *project.ID,
Title: "integration parent task",
@@ -146,6 +163,7 @@ func TestIntegrationProjectAndTaskLifecycle(t *testing.T) {
StartDate: start,
Deadline: deadline,
Priority: int(TaskPriorityNormal),
MilestoneId: int(*ms.ID),
})
if err != nil {
t.Fatalf("CreateProjectTask: %v", err)
+14 -6
View File
@@ -17,6 +17,18 @@ const MailListPageSize = 25
// Loader fetches list items for a menu subject using the OnlyOffice client.
type Loader struct {
Client *onlyoffice.Client
// Files is the backend-agnostic file store used for file download, preview
// and delete. When nil it falls back to Client.FileStore(ProviderREST).
Files onlyoffice.FileStore
}
// fileStore returns the configured file store, defaulting to REST.
func (l *Loader) fileStore() onlyoffice.FileStore {
if l.Files != nil {
return l.Files
}
return l.Client.FileStore(onlyoffice.ProviderREST)
}
// List returns items for the given list spec (nav leaf).
@@ -171,11 +183,7 @@ func (l *Loader) executeDelete(ctx context.Context, item model.Item) (string, er
}
return fmt.Sprintf("Deleted message %s", item.Title), nil
case model.KindFile:
id, err := strconv.Atoi(item.ID)
if err != nil {
return "", err
}
if err := l.Client.DeleteFiles(ctx, []int{id}); err != nil {
if err := l.fileStore().Delete(ctx, []string{item.ID}); err != nil {
return "", err
}
return fmt.Sprintf("Deleted file %s", item.Title), nil
@@ -199,7 +207,7 @@ func (l *Loader) executeDownload(ctx context.Context, item model.Item, destPath
return "", err
}
defer f.Close()
if _, err := l.Client.DownloadFile(ctx, item.ID, f); err != nil {
if _, err := l.fileStore().Download(ctx, item.ID, f); err != nil {
return "", err
}
return fmt.Sprintf("Downloaded to %s", destPath), nil
+3 -6
View File
@@ -5,7 +5,6 @@ import (
"context"
"fmt"
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/eslider/go-onlyoffice/cmd/office/model"
"github.com/eslider/go-onlyoffice/cmd/office/preview"
)
@@ -30,13 +29,11 @@ func (l *Loader) filePreviewMarkdown(ctx context.Context, item model.Item) (stri
return "", fmt.Errorf("file id missing")
}
name := item.Title
if meta, err := l.Client.GetFile(ctx, item.ID); err == nil && meta != nil {
if t := onlyoffice.FileEntryTitle(meta); t != "" {
name = t
}
if e, err := l.fileStore().Stat(ctx, item.ID); err == nil && e.Title != "" {
name = e.Title
}
var buf bytes.Buffer
if _, err := l.Client.DownloadFile(ctx, item.ID, &buf); err != nil {
if _, err := l.fileStore().Download(ctx, item.ID, &buf); err != nil {
return "", err
}
return preview.FileBytesToMarkdown(name, buf.Bytes())
+3 -7
View File
@@ -4,7 +4,6 @@ package fetch_test
import (
"context"
"os"
"testing"
"github.com/eslider/go-onlyoffice/cmd/office/model"
@@ -46,9 +45,6 @@ func TestIntegrationUpdateTaskTitleDescription(t *testing.T) {
}
func TestIntegrationTaskFieldsFromLiveAPI(t *testing.T) {
if os.Getenv("ONLYOFFICE_URL") == "" && os.Getenv("ONLYOFFICE_HOST") == "" {
t.Skip("ONLYOFFICE_URL not set")
}
loader, ctx := liveLoader(t)
items, err := loader.List(ctx, model.ListSpec{Subject: model.SubjectTasks})
if err != nil {
@@ -57,12 +53,12 @@ func TestIntegrationTaskFieldsFromLiveAPI(t *testing.T) {
if len(items) == 0 {
t.Skip("no tasks")
}
title, desc, err := loader.TaskFields(ctx, items[0])
fields, err := loader.DetailForm(ctx, items[0])
if err != nil {
t.Fatal(err)
}
if title == "" {
if fields.Primary == "" {
t.Fatal("empty title")
}
_ = desc
_ = fields.Secondary
}
+320
View File
@@ -0,0 +1,320 @@
package main
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"github.com/eslider/go-onlyoffice/catalog"
"github.com/spf13/cobra"
)
var catalogCmd = &cobra.Command{
Use: "catalog",
Short: "Inventory clients/contacts from disk; match and apply to OO CRM",
Long: `Build a YAML catalog of persons/companies from contacts folders and project
trees, match against live OnlyOffice CRM, then create/update only rows with
approve: true.
Typical flow:
oo catalog scan-contacts --root PATH -O /tmp/contacts.yaml
oo catalog scan-projects --root ~/projects -O /tmp/projects.yaml
oo catalog scan-thunderbird --root PATH -O /tmp/thunderbird.yaml
oo catalog merge -i /tmp/contacts.yaml -i /tmp/projects.yaml -i /tmp/thunderbird.yaml -O docs/catalog/clients-contacts.yaml
oo catalog match -i docs/catalog/clients-contacts.yaml
# edit approve: true on pilot rows
oo catalog apply --dry-run -i docs/catalog/clients-contacts.yaml
oo catalog apply --apply -i docs/catalog/clients-contacts.yaml
`}
func init() {
rootCmd.AddCommand(catalogCmd)
catalogCmd.AddCommand(catalogScanContactsCmd())
catalogCmd.AddCommand(catalogScanProjectsCmd())
catalogCmd.AddCommand(catalogScanThunderbirdCmd())
catalogCmd.AddCommand(catalogMergeCmd())
catalogCmd.AddCommand(catalogMatchCmd())
catalogCmd.AddCommand(catalogApplyCmd())
}
func catalogScanContactsCmd() *cobra.Command {
var outPath string
cmd := &cobra.Command{
Use: "scan-contacts",
Short: "Parse VCF + folder stubs + email txt under a contacts root",
RunE: func(cmd *cobra.Command, args []string) error {
root, _ := cmd.Flags().GetString("root")
if root == "" {
return fmt.Errorf("--root is required")
}
doc, err := catalog.ScanContactsRoot(root)
if err != nil {
return err
}
return writeCatalog(cmd, doc, outPath)
},
}
cmd.Flags().String("root", "", "contacts directory (local path)")
cmd.Flags().StringVarP(&outPath, "out", "O", "", "write YAML to this path (default: stdout summary)")
_ = cmd.MarkFlagRequired("root")
return cmd
}
func catalogScanProjectsCmd() *cobra.Command {
var outPath string
var maxDepth int
var configPath string
cmd := &cobra.Command{
Use: "scan-projects",
Short: "Git roots / remotes / top-level dirs → company rows",
RunE: func(cmd *cobra.Command, args []string) error {
root, _ := cmd.Flags().GetString("root")
if root == "" {
return fmt.Errorf("--root is required")
}
cl, err := catalogClassifier(configPath)
if err != nil {
return err
}
doc, err := catalog.ScanProjectsRootOpts(root, maxDepth, catalog.ScanOptions{Classifier: cl})
if err != nil {
return err
}
return writeCatalog(cmd, doc, outPath)
},
}
cmd.Flags().String("root", "", "projects directory (local path)")
cmd.Flags().IntVar(&maxDepth, "max-depth", 4, "max directory depth for git roots")
cmd.Flags().StringVar(&configPath, "config", "", "classification rules YAML (default $OO_CATALOG_CONFIG)")
cmd.Flags().StringVarP(&outPath, "out", "O", "", "write YAML to this path")
_ = cmd.MarkFlagRequired("root")
return cmd
}
func catalogScanThunderbirdCmd() *cobra.Command {
var outPath string
var mboxHeaders bool
var configPath string
cmd := &cobra.Command{
Use: "scan-thunderbird",
Short: "Thunderbird profiles: abook/history.mab + Gloda SQLite contacts",
Long: `Walk a directory tree for Thunderbird profiles and extract person rows from:
- *.mab address books (email regex)
- global-messages-db.sqlite Gloda contacts/identities
- optional --mbox-headers: From/To/Cc/Reply-To from mbox folder files (no bodies)
Noisy senders (noreply, Amazon marketplace, GitHub reply, …) are skipped.
Default zone is private; work domains/names are classified from the rules in
$OO_CATALOG_CONFIG (or --config) — none are hardcoded.`,
RunE: func(cmd *cobra.Command, args []string) error {
root, _ := cmd.Flags().GetString("root")
if root == "" {
return fmt.Errorf("--root is required")
}
cl, err := catalogClassifier(configPath)
if err != nil {
return err
}
doc, err := catalog.ScanThunderbirdRootOpts(root, catalog.ScanOptions{MboxHeaders: mboxHeaders, Classifier: cl})
if err != nil {
return err
}
return writeCatalog(cmd, doc, outPath)
},
}
cmd.Flags().String("root", "", "Thunderbird profile or parent directory (local path)")
cmd.Flags().BoolVar(&mboxHeaders, "mbox-headers", false, "also extract emails from mbox From/To/Cc headers")
cmd.Flags().StringVar(&configPath, "config", "", "classification rules YAML (default $OO_CATALOG_CONFIG)")
cmd.Flags().StringVarP(&outPath, "out", "O", "", "write YAML to this path")
_ = cmd.MarkFlagRequired("root")
return cmd
}
// catalogClassifier loads classification rules from --config, else
// $OO_CATALOG_CONFIG, else the neutral default (no rules).
func catalogClassifier(configPath string) (*catalog.Classifier, error) {
if configPath == "" {
return catalog.LoadClassifierFromEnv()
}
return catalog.LoadClassifier(configPath)
}
func catalogMergeCmd() *cobra.Command {
var inputs []string
var outPath string
cmd := &cobra.Command{
Use: "merge",
Short: "Union + normalize catalog YAML files by entry id",
RunE: func(cmd *cobra.Command, args []string) error {
if len(inputs) == 0 {
return fmt.Errorf("at least one -i/--input required")
}
var docs []*catalog.Document
for _, p := range inputs {
d, err := catalog.LoadYAML(p)
if err != nil {
return err
}
docs = append(docs, d)
}
merged := catalog.MergeDocs(docs...)
if outPath == "" {
return fmt.Errorf("--out is required for merge")
}
if err := ensureParent(outPath); err != nil {
return err
}
if err := catalog.SaveYAML(outPath, merged); err != nil {
return err
}
fmt.Fprintf(cmd.OutOrStdout(), "merged %d entries → %s\n", len(merged.Entries), outPath)
return nil
},
}
cmd.Flags().StringArrayVarP(&inputs, "input", "i", nil, "input catalog YAML (repeatable)")
cmd.Flags().StringVarP(&outPath, "out", "O", "", "output catalog YAML")
_ = cmd.MarkFlagRequired("out")
return cmd
}
func catalogMatchCmd() *cobra.Command {
var inPath, outPath string
cmd := &cobra.Command{
Use: "match",
Short: "Diff catalog vs live OO; set status new|exists|conflict and oo_id",
RunE: func(cmd *cobra.Command, args []string) error {
if inPath == "" {
return fmt.Errorf("--input is required")
}
doc, err := catalog.LoadYAML(inPath)
if err != nil {
return err
}
client, err := newOO(cmd)
if err != nil {
return err
}
if err := catalog.MatchAgainstOO(cmd.Context(), client, doc); err != nil {
return err
}
if outPath == "" {
outPath = inPath
}
if err := catalog.SaveYAML(outPath, doc); err != nil {
return err
}
counts := map[string]int{}
for _, e := range doc.Entries {
counts[e.Status]++
}
fmt.Fprintf(cmd.OutOrStdout(), "matched → %s new=%d exists=%d conflict=%d\n",
outPath, counts["new"], counts["exists"], counts["conflict"])
return nil
},
}
cmd.Flags().StringVarP(&inPath, "input", "i", "", "catalog YAML")
cmd.Flags().StringVarP(&outPath, "out", "O", "", "output path (default: overwrite input)")
_ = cmd.MarkFlagRequired("input")
return cmd
}
func catalogApplyCmd() *cobra.Command {
var inPath, outPath string
var dryRun, doApply bool
cmd := &cobra.Command{
Use: "apply",
Short: "Create/update OO contacts for approve:true rows",
RunE: func(cmd *cobra.Command, args []string) error {
if !dryRun && !doApply {
return fmt.Errorf("pass --dry-run or --apply")
}
if dryRun && doApply {
return fmt.Errorf("use either --dry-run or --apply, not both")
}
if inPath == "" {
return fmt.Errorf("--input is required")
}
doc, err := catalog.LoadYAML(inPath)
if err != nil {
return err
}
client, err := newOO(cmd)
if err != nil {
return err
}
res, err := catalog.ApplyApproved(cmd.Context(), client, doc, dryRun)
if err != nil {
return err
}
if !dryRun {
if outPath == "" {
outPath = inPath
}
if err := catalog.SaveYAML(outPath, doc); err != nil {
return err
}
}
if outputFormat == "json" {
printJSON(res)
return nil
}
mode := "apply"
if dryRun {
mode = "dry-run"
}
fmt.Fprintf(cmd.OutOrStdout(), "%s: created=%d updated=%d skipped=%d errors=%d\n",
mode, res.Created, res.Updated, res.Skipped, len(res.Errors))
for _, e := range res.Errors {
fmt.Fprintf(cmd.ErrOrStderr(), " error: %s\n", e)
}
return nil
},
}
cmd.Flags().StringVarP(&inPath, "input", "i", "", "catalog YAML")
cmd.Flags().StringVarP(&outPath, "out", "O", "", "write updated catalog after apply (default: overwrite input)")
cmd.Flags().BoolVar(&dryRun, "dry-run", false, "count actions without writing to OO")
cmd.Flags().BoolVar(&doApply, "apply", false, "perform OO creates/updates")
_ = cmd.MarkFlagRequired("input")
return cmd
}
func writeCatalog(cmd *cobra.Command, doc *catalog.Document, outPath string) error {
if outPath != "" {
if err := ensureParent(outPath); err != nil {
return err
}
if err := catalog.SaveYAML(outPath, doc); err != nil {
return err
}
fmt.Fprintf(cmd.OutOrStdout(), "wrote %d entries → %s\n", len(doc.Entries), outPath)
return nil
}
if outputFormat == "json" {
b, err := json.MarshalIndent(doc, "", " ")
if err != nil {
return err
}
fmt.Fprintln(cmd.OutOrStdout(), string(b))
return nil
}
persons, companies := 0, 0
for _, e := range doc.Entries {
if e.Kind == "company" {
companies++
} else {
persons++
}
}
fmt.Fprintf(cmd.OutOrStdout(), "entries=%d persons=%d companies=%d (pass -O PATH to write YAML)\n",
len(doc.Entries), persons, companies)
return nil
}
func ensureParent(path string) error {
dir := filepath.Dir(path)
if dir == "" || dir == "." {
return nil
}
return os.MkdirAll(dir, 0o755)
}
+9 -2
View File
@@ -11,7 +11,7 @@ func TestRootRegistersSubjects(t *testing.T) {
want := []string{
"calendar", "projects", "tasks", "users", "whoami",
"contacts", "persons", "companies",
"opportunities", "cases", "crm-tasks", "applications", "crm", "mails", "catalog",
"opportunities", "cases", "crm-tasks", "crm", "mails", "invoices",
}
got := make(map[string]bool, len(rootCmd.Commands()))
for _, c := range rootCmd.Commands() {
@@ -38,7 +38,7 @@ func TestRootHelpListsSubjects(t *testing.T) {
t.Fatal(err)
}
help := out.String()
for _, snippet := range []string{"calendar", "projects", "tasks", "users", "opportunities"} {
for _, snippet := range []string{"calendar", "projects", "tasks", "users", "opportunities", "invoices"} {
if !strings.Contains(help, snippet) {
t.Fatalf("help missing %q", snippet)
}
@@ -53,6 +53,13 @@ func TestProjectsAlias(t *testing.T) {
if cmd.Name() != "projects" {
t.Fatalf("prj alias resolved to %q", cmd.Name())
}
ms, _, err := rootCmd.Find([]string{"projects", "milestone-create"})
if err != nil {
t.Fatal(err)
}
if ms.Name() != "milestone-create" {
t.Fatalf("milestone-create resolved to %q", ms.Name())
}
}
func TestNewOOReturnsErrorWithoutCredentials(t *testing.T) {
+4 -2
View File
@@ -20,7 +20,7 @@ var outputFormat = "table"
var rootCmd = &cobra.Command{
Use: "oo",
Short: "OnlyOffice Workspace CLI — subject-based command tree",
Long: "oo is a thin CLI over github.com/eslider/go-onlyoffice.\nCommands are grouped by OnlyOffice subject (calendar, projects, tasks, users, persons, companies, opportunities, cases, crm-tasks, applications, catalog).",
Long: "oo is a thin CLI over github.com/eslider/go-onlyoffice.\nCommands are grouped by OnlyOffice subject (calendar, projects, tasks, users, persons, companies, opportunities, cases, crm-tasks, mails, invoices).",
Version: version,
SilenceUsage: true,
SilenceErrors: false,
@@ -31,7 +31,9 @@ func init() {
}
// execute runs the root command. Exported only to main.go in the same package.
func execute() error { return rootCmd.Execute() }
// .env is loaded CLI-wide so non-authenticating commands (e.g. catalog scans)
// also see configuration such as OO_CATALOG_CONFIG.
func execute() error { bootstrap.LoadEnv(); return rootCmd.Execute() }
// newOO loads env (only .env in CWD) and returns an authenticated client.
// godotenv is a CLI-only concern; the library itself never loads dotfiles.
+197 -5
View File
@@ -6,7 +6,6 @@ import (
"strings"
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/eslider/go-onlyoffice/catalog"
"github.com/spf13/cobra"
)
@@ -44,10 +43,16 @@ func init() {
contactsCmd.AddCommand(contactsMergeCmd())
contactsCmd.AddCommand(contactsInfoAddCmd())
contactsCmd.AddCommand(contactsDedupeInfoCmd())
contactsCmd.AddCommand(contactsTagsCmd())
contactsCmd.AddCommand(contactsTagCreateCmd())
contactsCmd.AddCommand(contactsTagAddCmd())
contactsCmd.AddCommand(contactsTagRemoveCmd())
contactsCmd.AddCommand(contactsByTagCmd())
only := true
personsCmd.AddCommand(contactsListCmd(&only)) // persons only
personsCmd.AddCommand(personsCreateCmd())
personsCmd.AddCommand(personsUpdateCmd())
personsCmd.AddCommand(personsFixNamesCmd())
personsCmd.AddCommand(contactsDeleteCmd())
personsCmd.AddCommand(personsDedupeCmd())
@@ -201,7 +206,7 @@ func contactsInfoAddCmd() *cobra.Command {
}
func personsCreateCmd() *cobra.Command {
var first, last, email, linkedin string
var first, last, email, linkedin, phone string
var companyID int
var jobTitle, about string
cmd := &cobra.Command{
@@ -224,6 +229,9 @@ func personsCreateCmd() *cobra.Command {
if email != "" {
_, _ = c.AddContactInfo(cmd.Context(), pid, "Email", email, "Work", true)
}
if phone != "" {
_, _ = c.AddContactInfo(cmd.Context(), pid, "Phone", phone, "Work", true)
}
if linkedin != "" {
_, _ = c.AddContactInfo(cmd.Context(), pid, "LinkedIn", linkedin, "Work", false)
}
@@ -237,10 +245,62 @@ func personsCreateCmd() *cobra.Command {
cmd.Flags().StringVar(&jobTitle, "job-title", "", "")
cmd.Flags().StringVar(&about, "about", "", "about / bio")
cmd.Flags().StringVar(&email, "email", "", "primary email (adds ContactInfo)")
cmd.Flags().StringVar(&phone, "phone", "", "primary phone (adds ContactInfo)")
cmd.Flags().StringVar(&linkedin, "linkedin", "", "linkedin url (adds ContactInfo)")
return cmd
}
func personsUpdateCmd() *cobra.Command {
var first, last string
var companyID int
var jobTitle, about string
cmd := &cobra.Command{
Use: "update PERSON_ID",
Short: "Update a person (name, company, job title, about)",
Long: `Updates CRM person fields via PUT /crm/contact/person/{id}.
--company-id 0 leaves the employer association unchanged.
Omit --first/--last to keep current names (fetched first).`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
pid := args[0]
cur, err := c.GetContact(cmd.Context(), pid)
if err != nil {
return err
}
if first == "" {
if v, ok := cur["firstName"].(string); ok {
first = v
}
}
if last == "" {
if v, ok := cur["lastName"].(string); ok {
last = v
}
}
if first == "" || last == "" {
return fmt.Errorf("first and last name required (pass --first/--last or ensure contact has them)")
}
out, err := c.UpdatePerson(cmd.Context(), pid, first, last, companyID, jobTitle, about)
if err != nil {
return err
}
printObject(out)
return nil
},
}
cmd.Flags().StringVar(&first, "first", "", "first name (default: current)")
cmd.Flags().StringVar(&last, "last", "", "last name (default: current)")
cmd.Flags().IntVar(&companyID, "company-id", 0, "employer company id (0 = leave unchanged)")
cmd.Flags().StringVar(&jobTitle, "job-title", "", "job title")
cmd.Flags().StringVar(&about, "about", "", "about / bio")
return cmd
}
func personsFixNamesCmd() *cobra.Command {
var companyID int
var dryRun bool
@@ -248,7 +308,7 @@ func personsFixNamesCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "fix-names",
Short: "Strip company annotations from last names; keep companyId link",
Long: `Repairs CRM persons whose lastName embeds a company ("Thomsen (Acme)")
Long: `Repairs CRM persons whose lastName embeds a company ("Doe (Acme)")
or whose firstName is an email. Company belongs on companyId, not in the name.
With --company-id, only persons linked to that company are scanned.`,
@@ -300,7 +360,7 @@ With --company-id, only persons linked to that company are scanned.`,
}
}
}
cf, cl := catalog.CleanPersonNames(fn, ln, dn, org, emails)
cf, cl := onlyoffice.CleanPersonNames(fn, ln, dn, org, emails)
needName := cf != fn || cl != ln
linkID := companyID
if linkID == 0 {
@@ -347,7 +407,7 @@ func contactEmailsFromMap(p map[string]any) []string {
}
func companiesCreateCmd() *cobra.Command {
var name, email, website string
var name, email, website, phone, about, street, city, state, zip, country string
cmd := &cobra.Command{
Use: "create",
Aliases: []string{"add"},
@@ -365,12 +425,34 @@ func companiesCreateCmd() *cobra.Command {
return err
}
cid := strconv.Itoa(int(flexIDFloat(out["id"])))
if about != "" || street != "" {
aboutText := about
if street != "" {
addr := strings.TrimSpace(strings.Join([]string{street, zip, city, state, country}, ", "))
if aboutText != "" {
aboutText = aboutText + "\n" + addr
} else {
aboutText = "Billing address: " + addr
}
}
if updated, err := c.UpdateCompany(cmd.Context(), cid, name, aboutText); err == nil {
out = updated
}
}
if email != "" {
_, _ = c.AddContactInfo(cmd.Context(), cid, "Email", email, "Work", true)
}
if website != "" {
_, _ = c.AddContactInfo(cmd.Context(), cid, "Website", website, "Work", false)
}
if phone != "" {
_, _ = c.AddContactInfo(cmd.Context(), cid, "Phone", phone, "Work", true)
}
if street != "" {
if _, err := c.AddContactAddress(cmd.Context(), cid, street, city, state, zip, country, "Billing", true); err != nil {
fmt.Fprintf(cmd.ErrOrStderr(), "warning: address API failed (stored in about): %v\n", err)
}
}
printObject(out)
return nil
},
@@ -378,6 +460,13 @@ func companiesCreateCmd() *cobra.Command {
cmd.Flags().StringVar(&name, "name", "", "company name")
cmd.Flags().StringVar(&email, "email", "", "primary email (adds ContactInfo)")
cmd.Flags().StringVar(&website, "website", "", "website url (adds ContactInfo)")
cmd.Flags().StringVar(&phone, "phone", "", "primary phone (adds ContactInfo)")
cmd.Flags().StringVar(&about, "about", "", "about / notes")
cmd.Flags().StringVar(&street, "street", "", "billing street")
cmd.Flags().StringVar(&city, "city", "", "billing city")
cmd.Flags().StringVar(&state, "state", "", "billing state")
cmd.Flags().StringVar(&zip, "zip", "", "billing zip")
cmd.Flags().StringVar(&country, "country", "", "billing country")
return cmd
}
@@ -440,3 +529,106 @@ func contactsDedupeInfoCmd() *cobra.Command {
}),
}
}
func contactsTagsCmd() *cobra.Command {
return &cobra.Command{
Use: "tags",
Short: "List CRM contact tags",
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
list, err := c.ListContactTags(cmd.Context())
if err != nil {
return err
}
printTable([]string{"title", "relativeItemsCount"}, list)
return nil
},
}
}
func contactsTagCreateCmd() *cobra.Command {
return &cobra.Command{
Use: "tag-create TAG",
Short: "Create a CRM contact tag",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
if err := c.CreateContactTag(cmd.Context(), args[0]); err != nil {
return err
}
fmt.Println(args[0])
return nil
},
}
}
func contactsTagAddCmd() *cobra.Command {
return &cobra.Command{
Use: "tag-add CONTACT_ID TAG",
Short: "Attach a tag to a contact",
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
if err := c.AddContactTag(cmd.Context(), args[0], args[1]); err != nil {
return err
}
fmt.Printf("%s ← %s\n", args[0], args[1])
return nil
},
}
}
func contactsTagRemoveCmd() *cobra.Command {
return &cobra.Command{
Use: "tag-remove CONTACT_ID TAG",
Short: "Remove a tag from a contact",
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
if err := c.RemoveContactTag(cmd.Context(), args[0], args[1]); err != nil {
return err
}
fmt.Printf("%s ✕ %s\n", args[0], args[1])
return nil
},
}
}
func contactsByTagCmd() *cobra.Command {
var count, offset int
cmd := &cobra.Command{
Use: "by-tag TAG",
Short: "List contacts with a given CRM tag (ignore-list = tag ignore)",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
list, total, err := c.ListContactsByTag(cmd.Context(), args[0], count, offset)
if err != nil {
return err
}
if outputFormat == "table" {
fmt.Printf("tag=%s total: %d (shown: %d)\n", args[0], total, len(list))
}
printTable([]string{"id", "displayName", "isCompany", "title", "about"}, list)
return nil
},
}
cmd.Flags().IntVar(&count, "count", 50, "")
cmd.Flags().IntVar(&offset, "offset", 0, "")
return cmd
}
+1 -1
View File
@@ -7,7 +7,7 @@ import (
var crmCmd = &cobra.Command{
Use: "crm",
Short: "CRM maintenance (dedupe, cleanup)",
Short: "CRM maintenance (audit, dedupe, cleanup)",
}
func init() {
+63
View File
@@ -0,0 +1,63 @@
package main
import (
"encoding/json"
"fmt"
"os"
"sort"
"github.com/spf13/cobra"
)
func init() {
crmCmd.AddCommand(crmAuditCmd())
}
func crmAuditCmd() *cobra.Command {
var outPath string
cmd := &cobra.Command{
Use: "audit",
Short: "Audit opportunities (files/tasks/members per deal) and classify",
Long: `Lists every opportunity with its file/task/member counts and a coarse class
(ok | dup | empty | junk-title). --out writes the full JSON audit for later use.`,
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
audits, err := c.AuditOpportunities(cmd.Context())
if err != nil {
return err
}
if outPath != "" {
b, merr := json.MarshalIndent(audits, "", " ")
if merr != nil {
return merr
}
if werr := os.WriteFile(outPath, b, 0o644); werr != nil {
return werr
}
}
byClass := map[string]int{}
for _, a := range audits {
byClass[a.Class]++
}
classes := make([]string, 0, len(byClass))
for k := range byClass {
classes = append(classes, k)
}
sort.Strings(classes)
rows := make([]map[string]any, 0, len(classes))
for _, k := range classes {
rows = append(rows, map[string]any{"class": k, "count": byClass[k]})
}
printTable([]string{"class", "count"}, rows)
if outPath != "" {
fmt.Fprintf(cmd.OutOrStdout(), "wrote %d audits → %s\n", len(audits), outPath)
}
return nil
},
}
cmd.Flags().StringVar(&outPath, "out", "", "write audit JSON to this file")
return cmd
}
+322
View File
@@ -0,0 +1,322 @@
package main
import (
"fmt"
"os"
"time"
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/spf13/cobra"
)
func init() {
rootCmd.AddCommand(davCmd())
}
// davCmd exposes the Documents module through the backend-agnostic FileStore
// (DAV backend). The underlying Dav calls are the oo-webdav proven path:
// MoveDavItems sends resolveType=Skip + holdResult=true, which the legacy
// fileops/move call without those params silently ignores (200 without move).
func davCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "dav",
Short: "Documents module by folder/file id (oo-webdav proven path)",
}
cmd.AddCommand(davLsCmd())
cmd.AddCommand(davMoveCmd())
cmd.AddCommand(davCopyCmd())
cmd.AddCommand(davMkdirCmd())
cmd.AddCommand(davRemoveCmd())
cmd.AddCommand(davRenameFileCmd())
cmd.AddCommand(davRenameFolderCmd())
cmd.AddCommand(davDownloadCmd())
cmd.AddCommand(davFileOpsCmd())
return cmd
}
func davLsCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "ls FOLDER_ID",
Short: "List a Documents folder (@root for virtual sections)",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
ctx := cmd.Context()
if args[0] == "@root" {
sections, err := c.ListDavSections(ctx)
if err != nil {
return err
}
rows := make([]map[string]any, 0, len(sections))
for _, s := range sections {
rows = append(rows, map[string]any{
"id": s.ID,
"title": s.Title,
"filesCount": s.FilesCount,
"foldersCount": s.FoldersCount,
})
}
printTable([]string{"id", "title", "filesCount", "foldersCount"}, rows)
return nil
}
entries, err := c.FileStore(onlyoffice.ProviderDAV).List(ctx, args[0])
if err != nil {
return err
}
folders := make([]onlyoffice.Entry, 0, len(entries))
files := make([]onlyoffice.Entry, 0, len(entries))
for _, e := range entries {
if e.Kind == onlyoffice.Folder {
folders = append(folders, e)
} else {
files = append(files, e)
}
}
frows := make([]map[string]any, 0, len(folders))
for _, f := range folders {
frows = append(frows, map[string]any{
"id": f.ID,
"title": f.Title,
"filesCount": f.FilesCount,
"foldersCount": f.FoldersCount,
})
}
rows := make([]map[string]any, 0, len(files))
for _, f := range files {
rows = append(rows, map[string]any{
"id": f.ID,
"title": f.Title,
"size": f.Size,
"updated": entryUpdated(f),
})
}
if outputFormat == "json" {
printObject(map[string]any{"folders": frows, "files": rows})
return nil
}
if len(frows) > 0 {
fmt.Println("folders:")
printTable([]string{"id", "title", "filesCount", "foldersCount"}, frows)
}
fmt.Println("files:")
printTable([]string{"id", "title", "size", "updated"}, rows)
return nil
},
}
return cmd
}
// entryUpdated prefers the backend-native timestamp string so table/JSON output
// round-trips what the API returned.
func entryUpdated(e onlyoffice.Entry) string {
if e.Updated != "" {
return e.Updated
}
if e.Modified.IsZero() {
return ""
}
return e.Modified.Format(time.RFC3339)
}
func davMoveCmd() *cobra.Command {
var folderIDs []string
cmd := &cobra.Command{
Use: "move DEST_FOLDER_ID FILE_ID [FILE_ID...]",
Short: "Move file(s) into a Documents folder (resolveType=Skip, holdResult)",
Args: cobra.MinimumNArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
ids := append(append([]string{}, folderIDs...), args[1:]...)
if err := c.FileStore(onlyoffice.ProviderDAV).Move(cmd.Context(), ids, args[0]); err != nil {
return err
}
printObject(map[string]any{"moved_files": args[1:], "moved_folders": folderIDs, "dest": args[0]})
return nil
},
}
cmd.Flags().StringSliceVar(&folderIDs, "folders", nil, "folder ids to move along with the files")
return cmd
}
func davCopyCmd() *cobra.Command {
var folderIDs []string
cmd := &cobra.Command{
Use: "copy DEST_FOLDER_ID FILE_ID [FILE_ID...]",
Short: "Copy file(s) into a Documents folder (conflictResolveType=Skip)",
Args: cobra.MinimumNArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
ids := append(append([]string{}, folderIDs...), args[1:]...)
if err := c.FileStore(onlyoffice.ProviderDAV).Copy(cmd.Context(), ids, args[0]); err != nil {
return err
}
printObject(map[string]any{"copied_files": args[1:], "copied_folders": folderIDs, "dest": args[0]})
return nil
},
}
cmd.Flags().StringSliceVar(&folderIDs, "folders", nil, "folder ids to copy along with the files")
return cmd
}
func davMkdirCmd() *cobra.Command {
return &cobra.Command{
Use: "mkdir PARENT_FOLDER_ID TITLE",
Short: "Create a subfolder in Documents",
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
f, err := c.FileStore(onlyoffice.ProviderDAV).CreateFolder(cmd.Context(), args[0], args[1])
if err != nil {
return err
}
printObject(map[string]any{"id": f.ID, "title": f.Title, "parent": args[0]})
return nil
},
}
}
func davRemoveCmd() *cobra.Command {
var folderIDs []string
cmd := &cobra.Command{
Use: "rm [FILE_ID...]",
Aliases: []string{"delete"},
Short: "Permanently delete file(s) and/or folder(s) from Documents",
Args: cobra.ArbitraryArgs,
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) == 0 && len(folderIDs) == 0 {
return fmt.Errorf("dav rm: give at least one FILE_ID or --folders")
}
c, err := newOO(cmd)
if err != nil {
return err
}
ids := append(append([]string{}, args...), folderIDs...)
if err := c.FileStore(onlyoffice.ProviderDAV).Delete(cmd.Context(), ids); err != nil {
return err
}
printObject(map[string]any{"deleted_files": args, "deleted_folders": folderIDs})
return nil
},
}
cmd.Flags().StringSliceVar(&folderIDs, "folders", nil, "folder ids to delete")
return cmd
}
func davRenameFileCmd() *cobra.Command {
return &cobra.Command{
Use: "rename-file FILE_ID NEW_TITLE",
Short: "Rename a Documents file (include extension in NEW_TITLE)",
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
if err := c.FileStore(onlyoffice.ProviderDAV).Rename(cmd.Context(), args[0], args[1]); err != nil {
return err
}
printObject(map[string]any{"id": args[0], "title": args[1]})
return nil
},
}
}
func davRenameFolderCmd() *cobra.Command {
return &cobra.Command{
Use: "rename-folder FOLDER_ID NEW_TITLE",
Short: "Rename a Documents folder",
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
if err := c.FileStore(onlyoffice.ProviderDAV).Rename(cmd.Context(), args[0], args[1]); err != nil {
return err
}
printObject(map[string]any{"id": args[0], "title": args[1]})
return nil
},
}
}
func davDownloadCmd() *cobra.Command {
var to string
cmd := &cobra.Command{
Use: "download FILE_ID",
Short: "Download Documents file bytes (default path: ./<title>)",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
ctx := cmd.Context()
store := c.FileStore(onlyoffice.ProviderDAV)
e, err := store.Stat(ctx, args[0])
if err != nil {
return err
}
path := to
if path == "" {
path = onlyoffice.SafeLocalFileName(e.Title)
}
out, err := os.Create(path)
if err != nil {
return err
}
defer out.Close()
n, err := store.Download(ctx, args[0], out)
if err != nil {
_ = os.Remove(path)
return err
}
printObject(map[string]any{"path": path, "bytes": n})
return nil
},
}
cmd.Flags().StringVar(&to, "to", "", "output path (default: ./<server title>)")
return cmd
}
func davFileOpsCmd() *cobra.Command {
return &cobra.Command{
Use: "fileops",
Short: "List active file operations (move/copy status polling)",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
ops, err := c.ListFileOps(cmd.Context())
if err != nil {
return err
}
rows := make([]map[string]any, 0, len(ops))
for _, op := range ops {
rows = append(rows, map[string]any{
"id": fmt.Sprint(op["id"]),
"operation": fmt.Sprint(op["operation"]),
"progress": fmt.Sprint(op["progress"]),
"finished": fmt.Sprint(op["finished"]),
"error": fmt.Sprint(op["error"]),
})
}
printTable([]string{"id", "operation", "progress", "finished", "error"}, rows)
return nil
},
}
}
+890
View File
@@ -0,0 +1,890 @@
package main
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"strconv"
"strings"
"time"
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/eslider/go-onlyoffice/internal/docpipe"
"github.com/spf13/cobra"
)
func init() {
rootCmd.AddCommand(docsCmd())
}
func docsCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "docs",
Short: "Local document pipeline: md↔docx, OCR→PDF, extract Markdown",
Long: `Agent-friendly conversions (requires pandoc / ocrmypdf / pdftotext on PATH).
OnlyOffice Documents UI is poor for .md/.txt — keep sources in git, store .docx in OO.
Upload Markdown as DOCX: oo docs put-md PROJECT_ID file.md
Upload plain text: oo docs put-txt PROJECT_ID file.txt (preserves line breaks)
Upload XLSX: oo docs put-xlsx PROJECT_ID FILE.xlsx
Read an OO file as MD: oo docs as-md FILE_ID
OCR a scan locally: oo docs ocr scan.pdf --md out.md
Structured OCR (hOCR→MD): oo docs hocr scan.jpg --md out.md --yaml out.yml`,
}
cmd.AddCommand(docsConvertCmd())
cmd.AddCommand(docsPDFCmd())
cmd.AddCommand(docsPresignedCmd())
cmd.AddCommand(docsCSVCmd())
cmd.AddCommand(docsJSONCmd())
cmd.AddCommand(docsOptimizeCmd())
cmd.AddCommand(docsOCRCmd())
cmd.AddCommand(docsHOCRCmd())
cmd.AddCommand(docsAsMDCmd())
cmd.AddCommand(docsPutMDCmd())
cmd.AddCommand(docsPutTxtCmd())
cmd.AddCommand(docsPutXlsxCmd())
cmd.AddCommand(docsToolsCmd())
return cmd
}
func docsToolsCmd() *cobra.Command {
return &cobra.Command{
Use: "tools",
Short: "Show which converter binaries are on PATH",
RunE: func(cmd *cobra.Command, args []string) error {
t := docpipe.LookPath()
printObject(map[string]any{
"pandoc": strOrNil(t.Pandoc),
"ocrmypdf": strOrNil(t.OCRMyPDF),
"pdftotext": strOrNil(t.PDFToText),
"tesseract": strOrNil(t.Tesseract),
"ghostscript": strOrNil(t.Ghostscript),
})
return nil
},
}
}
func docsPDFCmd() *cobra.Command {
var out, docsURL, secret, output, folder string
var stream, pipe bool
cmd := &cobra.Command{
Use: "pdf FILE_ID | PATH [ARG...]",
Short: "Convert files to PDF via the DocumentServer converter (OO file ids or local paths)",
Long: `Native OnlyOffice conversion (the engine behind the portal's "Download as PDF"):
1. GET /api/2.0/files/file/{id}/presigneduri → fetchable source URL
2. POST <docs>/converter with a JWT → converted file URL
3. download the result
Arguments may be OnlyOffice file ids OR local file paths. A local path is
uploaded to the scratch folder (--folder, default 2 = "My documents"), converted,
downloaded and then removed — so any local document yields a PDF on the fly.
Docs base defaults to $ONLYOFFICE_DOCS_URL, else $ONLYOFFICE_URL + "/ds-vpath"
(/ds-vpath is the usual reverse-proxy mount for the DocumentServer). The JWT secret is
$ONLYOFFICE_DS_SECRET (DocumentServer services.CoAuthoring.secret).`,
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
base := docsBaseURL(docsURL)
if base == "" {
return fmt.Errorf("docs base url unknown; set --docs-url or ONLYOFFICE_DOCS_URL")
}
sec := secret
if sec == "" {
sec = firstEnv("ONLYOFFICE_DS_SECRET", "OO_DS_SECRET")
}
if sec == "" {
return fmt.Errorf("JWT secret required: --secret or ONLYOFFICE_DS_SECRET")
}
ot := output
if ot == "" {
ot = "pdf"
}
toStdout := stream || pipe
if toStdout && len(args) > 1 {
return fmt.Errorf("--stream/--pipe writes one file to stdout; pass a single input")
}
for _, arg := range args {
id, local := arg, false
title := ""
if fi, statErr := os.Stat(arg); statErr == nil && !fi.IsDir() {
// Local file → temporary upload into the scratch folder.
ent, uerr := c.UploadToFolder(cmd.Context(), folder, arg)
if uerr != nil {
return fmt.Errorf("upload %s: %w", arg, uerr)
}
id = strconv.FormatInt(onlyoffice.FileEntryNumericID(ent), 10)
local = true
title = filepath.Base(arg)
} else {
if f, ferr := c.GetFile(cmd.Context(), id); ferr == nil && f != nil && f.Title != nil {
title = *f.Title
}
}
src, err := c.PresignedURI(cmd.Context(), id)
if err != nil {
return fmt.Errorf("presigneduri %s: %w", id, err)
}
res, err := c.ConvertDocument(cmd.Context(), base, sec, onlyoffice.ConvertRequest{
URL: src,
OutputType: ot,
FileType: strings.TrimPrefix(filepath.Ext(title), "."),
Title: title,
Key: fmt.Sprintf("oo-%s-%d", id, time.Now().UnixNano()),
})
if err != nil {
return fmt.Errorf("convert %s: %w", id, err)
}
var w io.Writer
dst := out
if toStdout {
w = os.Stdout
} else {
if dst == "" {
stem := strings.TrimSuffix(title, filepath.Ext(title))
if stem == "" {
stem = "file-" + id
}
dst = stem + "." + ot
}
fh, err := os.Create(dst)
if err != nil {
return err
}
w = fh
defer fh.Close()
}
n, derr := c.DownloadURLTo(cmd.Context(), res.FileURL, w)
if local {
// Best-effort cleanup of the temporary upload.
if nid, e := strconv.Atoi(id); e == nil {
_ = c.DeleteFiles(cmd.Context(), []int{nid})
}
}
if derr != nil {
return fmt.Errorf("download: %w", derr)
}
if toStdout {
// Keep stdout byte-clean for pipes; status goes to stderr.
fmt.Fprintf(os.Stderr, "converted %s -> stdout (%d bytes, %s)\n", arg, n, ot)
} else {
printObject(map[string]any{"source": arg, "fileid": id, "title": title, "output": dst, "bytes": n, "type": ot})
}
}
return nil
},
}
cmd.Flags().StringVar(&out, "out", "", "output path (default: ./<title>.<format>)")
cmd.Flags().StringVar(&output, "to", "pdf", "output format (pdf, docx, xlsx, …)")
cmd.Flags().StringVar(&docsURL, "docs-url", "", "DocumentServer base (default $ONLYOFFICE_DOCS_URL or $ONLYOFFICE_URL/ds-vpath)")
cmd.Flags().StringVar(&secret, "secret", "", "JWT secret (default $ONLYOFFICE_DS_SECRET)")
cmd.Flags().StringVar(&folder, "folder", "2", "scratch folder id for local-file uploads")
cmd.Flags().BoolVar(&stream, "stream", false, "write the converted bytes to stdout (pipe-friendly)")
cmd.Flags().BoolVar(&pipe, "pipe", false, "alias for --stream")
return cmd
}
func docsPresignedCmd() *cobra.Command {
return &cobra.Command{
Use: "presigned FILE_ID",
Short: "Print a short-lived fetchable URI for a portal file",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
u, err := c.PresignedURI(cmd.Context(), args[0])
if err != nil {
return err
}
printObject(map[string]any{"fileid": args[0], "uri": u})
return nil
},
}
}
// loadWorkbookBytes reads an argument that is either an OnlyOffice file id
// (downloaded via the client) or a local path.
func loadWorkbookBytes(cmd *cobra.Command, c *onlyoffice.Client, arg string) ([]byte, error) {
if fi, err := os.Stat(arg); err == nil && !fi.IsDir() {
return os.ReadFile(arg)
}
var buf bytes.Buffer
if _, err := c.DownloadFile(cmd.Context(), arg, &buf); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
// parseDelimiter maps a flag value to a CSV delimiter rune ("," default).
func parseDelimiter(s string) rune {
switch s {
case "", ",":
return ','
case "\\t", "tab", "\t":
return '\t'
case ";":
return ';'
case "|":
return '|'
default:
r := []rune(s)
return r[0]
}
}
func docsCSVCmd() *cobra.Command {
var sheet int
var delim, out string
cmd := &cobra.Command{
Use: "csv SRC [SRC...]",
Short: "Export a worksheet (XLS/XLSX/ODS) to CSV (sheet-aware)",
Long: `SRC is an OnlyOffice file id or a local path. --sheet is 1-based (default 1 = first).
Why not the DocumentServer: its csv output covers only the first worksheet and
ignores a sheet selector (verified). Sheet selection and JSON use a local reader.`,
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
d := parseDelimiter(delim)
for _, arg := range args {
data, err := loadWorkbookBytes(cmd, c, arg)
if err != nil {
return fmt.Errorf("read %s: %w", arg, err)
}
text, err := onlyoffice.WorkbookSheetCSV(data, sheet-1, d)
if err != nil {
return fmt.Errorf("%s: %w", arg, err)
}
if out != "" {
if err := os.WriteFile(out, []byte(text), 0o644); err != nil {
return err
}
printObject(map[string]any{"source": arg, "sheet": sheet, "output": out, "bytes": len(text)})
continue
}
fmt.Print(text)
}
return nil
},
}
cmd.Flags().IntVar(&sheet, "sheet", 1, "worksheet number (1-based, default first)")
cmd.Flags().StringVar(&delim, "delimiter", ",", "CSV delimiter (',', ';', '|', 'tab')")
cmd.Flags().StringVar(&out, "out", "", "write to this file instead of stdout (single input)")
return cmd
}
func docsJSONCmd() *cobra.Command {
var sheet int
var out string
cmd := &cobra.Command{
Use: "json SRC [SRC...]",
Short: "Export a worksheet (XLS/XLSX/ODS) to JSON rows (first row = header)",
Long: `SRC is an OnlyOffice file id or a local path. --sheet is 1-based (default 1 = first).
Each data row becomes an object keyed by the header cells of that sheet.`,
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
for _, arg := range args {
data, err := loadWorkbookBytes(cmd, c, arg)
if err != nil {
return fmt.Errorf("read %s: %w", arg, err)
}
rows, err := onlyoffice.WorkbookSheetJSON(data, sheet-1)
if err != nil {
return fmt.Errorf("%s: %w", arg, err)
}
b, err := json.MarshalIndent(rows, "", " ")
if err != nil {
return err
}
b = append(b, '\n')
if out != "" {
if err := os.WriteFile(out, b, 0o644); err != nil {
return err
}
printObject(map[string]any{"source": arg, "sheet": sheet, "rows": len(rows), "output": out})
continue
}
os.Stdout.Write(b)
}
return nil
},
}
cmd.Flags().IntVar(&sheet, "sheet", 1, "worksheet number (1-based, default first)")
cmd.Flags().StringVar(&out, "out", "", "write to this file instead of stdout (single input)")
return cmd
}
// docsBaseURL resolves the DocumentServer base: --docs-url, $ONLYOFFICE_DOCS_URL,
// else the standard /ds-vpath reverse-proxy mount.
func docsBaseURL(flag string) string {
if flag != "" {
return flag
}
if v := os.Getenv("ONLYOFFICE_DOCS_URL"); v != "" {
return v
}
if v := firstEnv("ONLYOFFICE_URL", "ONLYOFFICE_HOST", "OO_URL"); v != "" {
return strings.TrimRight(v, "/") + "/ds-vpath"
}
return ""
}
func firstEnv(keys ...string) string {
for _, k := range keys {
if v := os.Getenv(k); v != "" {
return v
}
}
return ""
}
func strOrNil(s string) any {
if s == "" {
return nil
}
return s
}
func docsConvertCmd() *cobra.Command {
var to string
cmd := &cobra.Command{
Use: "convert PATH",
Short: "Convert a local file with pandoc (md↔docx by default)",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
in := args[0]
t := docpipe.LookPath()
out := to
if out == "" {
switch docpipe.Ext(in) {
case ".md", ".markdown":
out = docpipe.SiblingDOCX(in)
case ".docx":
out = strings.TrimSuffix(in, docpipe.Ext(in)) + ".md"
default:
return fmt.Errorf("--to required for input type %s", docpipe.Ext(in))
}
}
if err := docpipe.EnsureDir(out); err != nil {
return err
}
if err := t.ConvertFile(in, out); err != nil {
return err
}
printObject(map[string]any{"in": in, "out": out})
return nil
},
}
cmd.Flags().StringVar(&to, "to", "", "output path (default: sibling .docx or .md)")
return cmd
}
func docsOptimizeCmd() *cobra.Command {
var out string
cmd := &cobra.Command{
Use: "optimize PDF_PATH",
Short: "Rewrite PDF via Ghostscript (PostScript pdfwrite) for OO-friendly size/text",
Long: `Use for InDesign/iText PDFs with a good text layer — avoids ocrmypdf invisible
text overlays that break OnlyOffice DocEditor. Skips OCR; rewrites via gs pdfwrite.`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
in := args[0]
t := docpipe.LookPath()
if out == "" {
base := strings.TrimSuffix(filepath.Base(in), filepath.Ext(in))
out = filepath.Join(filepath.Dir(in), base+".optimized.pdf")
}
if err := docpipe.EnsureDir(out); err != nil {
return err
}
chars, _ := t.PDFTextLayerChars(in)
if err := t.OptimizePDF(in, out); err != nil {
return err
}
outChars, _ := t.PDFTextLayerChars(out)
printObject(map[string]any{
"in": in,
"pdf": out,
"text_chars_in": chars,
"text_chars_out": outChars,
"note": "native text layer preserved; no OCR overlay",
})
return nil
},
}
cmd.Flags().StringVar(&out, "out", "", "output PDF (default: <name>.optimized.pdf)")
return cmd
}
func docsOCRCmd() *cobra.Command {
var out, mdOut, lang string
var force bool
var writeMD bool
cmd := &cobra.Command{
Use: "ocr PATH",
Short: "OCR image/PDF → searchable PDF (and optional Markdown)",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
in := args[0]
t := docpipe.LookPath()
if out == "" {
base := strings.TrimSuffix(filepath.Base(in), filepath.Ext(in))
out = filepath.Join(filepath.Dir(in), base+".ocr.pdf")
}
if err := docpipe.EnsureDir(out); err != nil {
return err
}
if err := t.OCRToPDF(in, out, force, lang); err != nil {
return err
}
res := map[string]any{"in": in, "pdf": out}
if writeMD || mdOut != "" {
if mdOut == "" {
mdOut = strings.TrimSuffix(out, filepath.Ext(out)) + ".md"
}
text, err := t.ExtractPDFText(out)
if err != nil {
return err
}
body := "# " + filepath.Base(in) + "\n\n" + strings.TrimSpace(text) + "\n"
if err := os.WriteFile(mdOut, []byte(body), 0o644); err != nil {
return err
}
res["md"] = mdOut
}
printObject(res)
return nil
},
}
cmd.Flags().StringVar(&out, "out", "", "output searchable PDF (default: <name>.ocr.pdf)")
cmd.Flags().StringVar(&mdOut, "md", "", "write Markdown extraction to this path")
cmd.Flags().BoolVar(&writeMD, "markdown", false, "also write sibling .md next to OCR PDF")
cmd.Flags().StringVar(&lang, "lang", "eng", "OCR language(s) for tesseract/ocrmypdf")
cmd.Flags().BoolVar(&force, "force", false, "force OCR even if a text layer exists (default: skip when pdftotext finds enough text)")
return cmd
}
func docsHOCRCmd() *cobra.Command {
var mdOut, yamlOut, hocrOut, lang string
var dpi int
var minConf float32
cmd := &cobra.Command{
Use: "hocr PATH",
Short: "Tesseract hOCR → structured Markdown/YAML (via go-hocr)",
Long: `Runs tesseract with hOCR output, parses with go-hocr, writes Markdown
(and optional YAML). Better reading order / confidence than plain pdftotext.
For Spanish scans: --lang spa or spa+eng (needs tesseract-ocr-spa / TESSDATA_PREFIX).
Phone photos: --dpi 200..300.`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
in := args[0]
t := docpipe.LookPath()
dir, err := os.MkdirTemp("", "oo-docs-hocr-*")
if err != nil {
return err
}
defer os.RemoveAll(dir)
res, err := t.ToHOCRMarkdown(in, dir, lang, dpi, minConf)
if err != nil {
return err
}
if mdOut == "" {
base := strings.TrimSuffix(filepath.Base(in), filepath.Ext(in))
mdOut = filepath.Join(filepath.Dir(in), base+".hocr.md")
}
if err := docpipe.EnsureDir(mdOut); err != nil {
return err
}
if err := os.WriteFile(mdOut, []byte(res.Markdown), 0o644); err != nil {
return err
}
obj := map[string]any{
"in": in,
"md": mdOut,
"did_ocr": res.DidOCR,
}
if hocrOut != "" {
if err := docpipe.EnsureDir(hocrOut); err != nil {
return err
}
b, err := os.ReadFile(res.HOCRPath)
if err != nil {
return err
}
if err := os.WriteFile(hocrOut, b, 0o644); err != nil {
return err
}
obj["hocr"] = hocrOut
} else {
obj["hocr_tmp"] = res.HOCRPath
}
if yamlOut != "" {
if err := docpipe.EnsureDir(yamlOut); err != nil {
return err
}
if err := os.WriteFile(yamlOut, []byte(res.YAML), 0o644); err != nil {
return err
}
obj["yaml"] = yamlOut
}
printObject(obj)
return nil
},
}
cmd.Flags().StringVar(&mdOut, "md", "", "Markdown output (default: <name>.hocr.md)")
cmd.Flags().StringVar(&yamlOut, "yaml", "", "also write structured YAML from go-hocr")
cmd.Flags().StringVar(&hocrOut, "hocr", "", "also keep raw .hocr file at this path")
cmd.Flags().StringVar(&lang, "lang", "eng", "tesseract language(s), e.g. spa+eng")
cmd.Flags().IntVar(&dpi, "dpi", 220, "hint DPI for phone photos / scans (0 = tesseract default)")
cmd.Flags().Float32Var(&minConf, "min-conf", 0, "drop words with OCR confidence below this (0 = keep all)")
return cmd
}
func docsAsMDCmd() *cobra.Command {
var to, lang string
var minChars int
var uploadOCR bool
var useHOCR bool
var dpi int
var minConf float32
cmd := &cobra.Command{
Use: "as-md FILE_ID",
Short: "Download an OO Documents file and emit Markdown (OCR PDF/image if needed)",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
ctx := cmd.Context()
meta, err := c.GetFile(ctx, args[0])
if err != nil {
return err
}
title := onlyoffice.FileEntryTitle(meta)
dir, err := os.MkdirTemp("", "oo-docs-as-md-*")
if err != nil {
return err
}
defer os.RemoveAll(dir)
local := filepath.Join(dir, onlyoffice.SafeLocalFileName(title))
f, err := os.Create(local)
if err != nil {
return err
}
if _, err := c.DownloadFile(ctx, args[0], f); err != nil {
_ = f.Close()
return err
}
_ = f.Close()
tools := docpipe.LookPath()
var res docpipe.Result
if useHOCR {
hr, err := tools.ToHOCRMarkdown(local, dir, lang, dpi, minConf)
if err != nil {
return err
}
res = docpipe.Result{Markdown: hr.Markdown, DidOCR: hr.DidOCR, Source: hr.Source}
} else {
res, err = tools.ToMarkdown(local, dir, lang, minChars)
if err != nil {
return err
}
}
outPath := to
if outPath == "" {
base := strings.TrimSuffix(onlyoffice.SafeLocalFileName(title), filepath.Ext(onlyoffice.SafeLocalFileName(title)))
if base == "" || base == "download" {
base = "file-" + args[0]
}
outPath = base + ".md"
}
if err := docpipe.EnsureDir(outPath); err != nil {
return err
}
if err := os.WriteFile(outPath, []byte(res.Markdown), 0o644); err != nil {
return err
}
obj := map[string]any{
"file_id": args[0],
"title": title,
"md": outPath,
"did_ocr": res.DidOCR,
}
if res.OCRPDFPath != "" && uploadOCR {
folderID := onlyoffice.FileFolderID(meta)
upName := strings.TrimSuffix(onlyoffice.SafeLocalFileName(title), filepath.Ext(onlyoffice.SafeLocalFileName(title))) + ".ocr.pdf"
tmpUp := filepath.Join(dir, upName)
data, err := os.ReadFile(res.OCRPDFPath)
if err != nil {
return err
}
if err := os.WriteFile(tmpUp, data, 0o644); err != nil {
return err
}
if folderID == "" {
obj["ocr_pdf_local"] = res.OCRPDFPath
obj["note"] = "file has no folderId; OCR PDF left local — pass after moving into a folder"
} else {
ent, err := c.UploadToFolder(ctx, folderID, tmpUp)
if err != nil {
return err
}
obj["ocr_pdf_file_id"] = fileIDStr(ent)
obj["ocr_pdf_title"] = onlyoffice.FileEntryTitle(ent)
}
} else if res.OCRPDFPath != "" {
// Keep OCR PDF outside temp by copying beside md if requested via env-less default:
kept := strings.TrimSuffix(outPath, filepath.Ext(outPath)) + ".ocr.pdf"
if b, err := os.ReadFile(res.OCRPDFPath); err == nil {
_ = os.WriteFile(kept, b, 0o644)
obj["ocr_pdf_local"] = kept
} else {
obj["ocr_pdf_local"] = res.OCRPDFPath
}
}
printObject(obj)
return nil
},
}
cmd.Flags().StringVar(&to, "to", "", "write Markdown to this path (default: ./<title>.md)")
cmd.Flags().StringVar(&lang, "lang", "eng", "OCR language")
cmd.Flags().IntVar(&minChars, "min-chars", docpipe.DefaultMinTextChars, "OCR PDF if text layer shorter than this")
cmd.Flags().BoolVar(&uploadOCR, "upload-ocr", false, "upload searchable OCR PDF back into the same OO folder")
cmd.Flags().BoolVar(&useHOCR, "hocr", false, "use tesseract hOCR + go-hocr instead of ocrmypdf/pdftotext")
cmd.Flags().IntVar(&dpi, "dpi", 220, "DPI hint when --hocr (phone photos)")
cmd.Flags().Float32Var(&minConf, "min-conf", 0, "drop low-confidence words when --hocr")
return cmd
}
func docsPutMDCmd() *cobra.Command {
var folderID string
var keepLocalDOCX string
var replace bool
cmd := &cobra.Command{
Use: "put-md PROJECT_ID MARKDOWN_PATH",
Short: "Convert Markdown→DOCX and upload DOCX into a project (OO-friendly)",
Long: `Agents edit .md locally; this uploads .docx so OnlyOffice can open/version it.`,
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
pid, mdPath := args[0], args[1]
c, err := newOO(cmd)
if err != nil {
return err
}
tools := docpipe.LookPath()
dir, err := os.MkdirTemp("", "oo-docs-put-md-*")
if err != nil {
return err
}
defer os.RemoveAll(dir)
docxName := strings.TrimSuffix(filepath.Base(mdPath), filepath.Ext(mdPath)) + ".docx"
docxPath := filepath.Join(dir, docxName)
if err := tools.MDToDOCX(mdPath, docxPath); err != nil {
return err
}
if keepLocalDOCX != "" {
if err := docpipe.EnsureDir(keepLocalDOCX); err != nil {
return err
}
b, err := os.ReadFile(docxPath)
if err != nil {
return err
}
if err := os.WriteFile(keepLocalDOCX, b, 0o644); err != nil {
return err
}
}
ctx := cmd.Context()
ent, deleted, err := uploadProjectDoc(ctx, c, pid, docxPath, folderID, replace)
if err != nil {
return err
}
obj := map[string]any{
"project_id": pid,
"md": mdPath,
"uploaded": fileEntryToMap(ent),
}
if folderID != "" {
obj["folder_id"] = folderID
}
if len(deleted) > 0 {
obj["replaced_file_ids"] = deleted
}
printObject(obj)
return nil
},
}
cmd.Flags().StringVar(&folderID, "folder", "", "Documents folder id (default: project root)")
cmd.Flags().StringVar(&keepLocalDOCX, "keep-docx", "", "also write the generated DOCX to this local path")
cmd.Flags().BoolVar(&replace, "replace", true, "replace same stem|ext before upload (default); false = fail if name taken")
return cmd
}
func docsPutTxtCmd() *cobra.Command {
var folderID string
var keepLocalDOCX string
var replace bool
cmd := &cobra.Command{
Use: "put-txt PROJECT_ID TEXT_PATH",
Short: "Convert plain text→DOCX (preserve line breaks) and upload into a project",
Long: `OnlyOffice cannot render .txt well. This keeps each source line on its own DOCX line.`,
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
pid, txtPath := args[0], args[1]
c, err := newOO(cmd)
if err != nil {
return err
}
tools := docpipe.LookPath()
dir, err := os.MkdirTemp("", "oo-docs-put-txt-*")
if err != nil {
return err
}
defer os.RemoveAll(dir)
docxName := strings.TrimSuffix(filepath.Base(txtPath), filepath.Ext(txtPath)) + ".docx"
docxPath := filepath.Join(dir, docxName)
if err := tools.TXTToDOCX(txtPath, docxPath); err != nil {
return err
}
if keepLocalDOCX != "" {
if err := docpipe.EnsureDir(keepLocalDOCX); err != nil {
return err
}
b, err := os.ReadFile(docxPath)
if err != nil {
return err
}
if err := os.WriteFile(keepLocalDOCX, b, 0o644); err != nil {
return err
}
}
ctx := cmd.Context()
ent, deleted, err := uploadProjectDoc(ctx, c, pid, docxPath, folderID, replace)
if err != nil {
return err
}
obj := map[string]any{
"project_id": pid,
"txt": txtPath,
"uploaded": fileEntryToMap(ent),
}
if folderID != "" {
obj["folder_id"] = folderID
}
if len(deleted) > 0 {
obj["replaced_file_ids"] = deleted
}
printObject(obj)
return nil
},
}
cmd.Flags().StringVar(&folderID, "folder", "", "Documents folder id (default: project root)")
cmd.Flags().StringVar(&keepLocalDOCX, "keep-docx", "", "also write the generated DOCX to this local path")
cmd.Flags().BoolVar(&replace, "replace", true, "replace same stem|ext before upload (default); false = fail if name taken")
return cmd
}
func docsPutXlsxCmd() *cobra.Command {
var folderID, keepLocal string
var replace bool
cmd := &cobra.Command{
Use: "put-xlsx PROJECT_ID LOCAL_XLSX",
Short: "Upload an XLSX workbook into a project (upsert by stem|ext)",
Long: `Spreadsheets live in OnlyOffice — not in git. Uploads an existing .xlsx into the
project Documents (or --folder), replacing the same stem|ext by default.
oo docs put-xlsx 218 ./my.xlsx`,
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
pid := args[0]
c, err := newOO(cmd)
if err != nil {
return err
}
xlsxPath := args[1]
if docpipe.Ext(xlsxPath) != ".xlsx" {
return fmt.Errorf("expected .xlsx, got %s", docpipe.Ext(xlsxPath))
}
srcLabel := xlsxPath
if keepLocal != "" {
b, err := os.ReadFile(xlsxPath)
if err != nil {
return err
}
if err := docpipe.EnsureDir(keepLocal); err != nil {
return err
}
if err := os.WriteFile(keepLocal, b, 0o644); err != nil {
return err
}
}
ctx := cmd.Context()
ent, deleted, err := uploadProjectDoc(ctx, c, pid, xlsxPath, folderID, replace)
if err != nil {
return err
}
obj := map[string]any{
"project_id": pid,
"source": srcLabel,
"uploaded": fileEntryToMap(ent),
}
if folderID != "" {
obj["folder_id"] = folderID
}
if len(deleted) > 0 {
obj["replaced_file_ids"] = deleted
}
printObject(obj)
return nil
},
}
cmd.Flags().StringVar(&folderID, "folder", "", "Documents folder id (default: project root)")
cmd.Flags().StringVar(&keepLocal, "keep-xlsx", "", "also copy the uploaded bytes to this local path")
cmd.Flags().BoolVar(&replace, "replace", true, "replace same stem|ext before upload (default); false = fail if name taken")
return cmd
}
// uploadProjectDoc upserts (--replace, default) or no-clobbers into project/folder Documents.
func uploadProjectDoc(ctx context.Context, c *onlyoffice.Client, pid, localPath, folderID string, replace bool) (*onlyoffice.FileEntry, []int, error) {
if folderID != "" {
if replace {
return c.UploadToFolderReplacing(ctx, folderID, localPath)
}
if err := c.AssertNoFileConflict(ctx, folderID, localPath); err != nil {
return nil, nil, err
}
ent, err := c.UploadToFolder(ctx, folderID, localPath)
return ent, nil, err
}
if replace {
return c.UploadProjectFileReplacing(ctx, pid, localPath)
}
ent, err := c.UploadProjectFileNoClobber(ctx, pid, localPath)
return ent, nil, err
}
+158
View File
@@ -0,0 +1,158 @@
package main
import (
"strings"
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/spf13/cobra"
)
func init() {
rootCmd.AddCommand(indexCmd())
}
// indexFlags are shared by the `oo index folder` and `oo index files` verbs.
type indexFlags struct {
recursive bool
exts string
limit int
workers int
lang string
minChars int
workDir string
backend string
dryRun bool
asJSON bool
}
// indexCmd populates the own full-text index (oo_docs_text) that makes PDF and
// scanned content searchable. The OnlyOffice index is left untouched.
func indexCmd() *cobra.Command {
f := &indexFlags{}
cmd := &cobra.Command{
Use: "index",
Short: "Populate the own full-text index for PDF/scan content",
Long: "Index document text that the OnlyOffice Elasticsearch index does not\n" +
"cover (PDFs and scans) into a separate index (ONLYOFFICE_ES_TEXT_INDEX,\n" +
"default oo_docs_text). Text is extracted with docpipe (pdftotext, OCR)\n" +
"and the OnlyOffice server is never modified.\n\n" +
"Requires ONLYOFFICE_URL/USER/PASS (to download files) and ONLYOFFICE_ES_URL\n" +
"(to write the index). See docs/elasticsearch.md.",
}
cmd.PersistentFlags().BoolVar(&f.recursive, "recursive", false, "folder: descend into subfolders")
cmd.PersistentFlags().StringVar(&f.exts, "exts", "pdf", "comma-separated extensions to index")
cmd.PersistentFlags().IntVar(&f.limit, "limit", 0, "maximum number of files to index (0 = all)")
cmd.PersistentFlags().IntVar(&f.workers, "workers", 3, "parallel downloads/extractions")
cmd.PersistentFlags().StringVar(&f.lang, "lang", "deu+eng", "OCR language(s)")
cmd.PersistentFlags().IntVar(&f.minChars, "min-chars", 0, "text-layer threshold below which OCR runs")
cmd.PersistentFlags().StringVar(&f.workDir, "work-dir", "", "temp dir for downloads (default: system temp)")
cmd.PersistentFlags().StringVar(&f.backend, "backend", "rest", "file backend: rest|dav")
cmd.PersistentFlags().BoolVar(&f.dryRun, "dry-run", false, "list what would be indexed, without changes")
cmd.PersistentFlags().BoolVar(&f.asJSON, "json", false, "shorthand for --output json")
cmd.AddCommand(indexFolderCmd(f), indexFilesCmd(f))
return cmd
}
func indexFolderCmd(f *indexFlags) *cobra.Command {
return &cobra.Command{
Use: "folder FOLDER_ID",
Short: "Index every matching file in a Documents folder",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
return runIndex(cmd, f, args[0], nil)
},
}
}
func indexFilesCmd(f *indexFlags) *cobra.Command {
return &cobra.Command{
Use: "files FILE_ID...",
Short: "Index specific Documents files",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
return runIndex(cmd, f, "", args)
},
}
}
func runIndex(cmd *cobra.Command, f *indexFlags, folderID string, ids []string) error {
if f.asJSON {
outputFormat = "json"
}
c, err := newOO(cmd)
if err != nil {
return err
}
idx, err := onlyoffice.NewESTextIndex(onlyoffice.ESTextConfigFromEnv())
if err != nil {
return err
}
ti := onlyoffice.NewTextIndexer(c.FileStore(f.backend), idx)
ti.WorkDir = f.workDir
opts := onlyoffice.IndexOptions{
Recursive: f.recursive,
Extensions: splitList(f.exts),
Limit: f.limit,
Lang: f.lang,
MinChars: f.minChars,
Workers: f.workers,
}
ctx := cmd.Context()
if f.dryRun {
var entries []onlyoffice.Entry
if folderID != "" {
entries, err = ti.PlanFolder(ctx, folderID, opts)
} else {
entries, err = ti.PlanFiles(ctx, ids, opts)
}
if err != nil {
return err
}
rows := make([]map[string]any, 0, len(entries))
for _, e := range entries {
rows = append(rows, map[string]any{
"id": e.ID,
"title": e.Title,
"folder": e.ParentID,
})
}
printTable([]string{"id", "title", "folder"}, rows)
return nil
}
if err := ti.Ensure(ctx); err != nil {
return err
}
var res onlyoffice.IndexResult
if folderID != "" {
res, err = ti.IndexFolder(ctx, folderID, opts)
} else {
res, err = ti.IndexFiles(ctx, ids, opts)
}
if err != nil {
return err
}
printObject(map[string]any{
"index": idx.Index(),
"scanned": res.Scanned,
"indexed": res.Indexed,
"skipped": res.Skipped,
"failed": res.Failed,
"errors": res.Errors,
})
return nil
}
// splitList parses a comma-separated flag value, dropping blanks.
func splitList(s string) []string {
parts := strings.Split(s, ",")
out := make([]string, 0, len(parts))
for _, p := range parts {
if p = strings.TrimSpace(p); p != "" {
out = append(out, p)
}
}
return out
}
+55
View File
@@ -0,0 +1,55 @@
package main
import (
"reflect"
"strings"
"testing"
)
func TestIndexCommandRegistered(t *testing.T) {
cmd, _, err := rootCmd.Find([]string{"index"})
if err != nil {
t.Fatal(err)
}
if cmd.Name() != "index" {
t.Fatalf("index resolved to %q", cmd.Name())
}
for _, name := range []string{"exts", "limit", "workers", "lang", "min-chars", "work-dir", "backend", "dry-run", "recursive", "json"} {
if cmd.PersistentFlags().Lookup(name) == nil {
t.Errorf("index: missing --%s flag", name)
}
}
if cmd.PersistentFlags().Lookup("exts").DefValue != "pdf" {
t.Errorf("--exts default = %q, want pdf", cmd.PersistentFlags().Lookup("exts").DefValue)
}
for _, verb := range []string{"index folder", "index files"} {
sub, _, err := rootCmd.Find(strings.Fields(verb))
if err != nil {
t.Fatalf("%s: %v", verb, err)
}
if sub.Name() != strings.Fields(verb)[1] {
t.Errorf("%s resolved to %q", verb, sub.Name())
}
}
}
func TestIndexSearchBackendFlag(t *testing.T) {
cmd, _, err := rootCmd.Find([]string{"search"})
if err != nil {
t.Fatal(err)
}
if cmd.Flags().Lookup("backend") == nil {
t.Fatal("search: missing --backend flag")
}
if cmd.Flags().Lookup("backend").DefValue != "oo" {
t.Errorf("--backend default = %q, want oo", cmd.Flags().Lookup("backend").DefValue)
}
}
func TestSplitList(t *testing.T) {
got := splitList(" pdf , .PDF, docx ,, ")
want := []string{"pdf", ".PDF", "docx"}
if !reflect.DeepEqual(got, want) {
t.Errorf("splitList = %v, want %v", got, want)
}
}
+441
View File
@@ -0,0 +1,441 @@
package main
import (
"fmt"
"strings"
"time"
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/spf13/cobra"
)
var invoicesCmd = &cobra.Command{
Use: "invoices",
Aliases: []string{"invoice"},
Short: "CRM invoices (billing)",
}
func init() {
rootCmd.AddCommand(invoicesCmd)
invoicesCmd.AddCommand(invoiceListCmd())
invoicesCmd.AddCommand(invoiceGetCmd())
invoicesCmd.AddCommand(invoiceCreateCmd())
invoicesCmd.AddCommand(invoiceUpdateCmd())
invoicesCmd.AddCommand(invoicePDFCmd())
invoicesCmd.AddCommand(invoicePDFCleanupCmd())
invoicesCmd.AddCommand(invoiceStatusCmd())
invoicesCmd.AddCommand(invoiceDeleteCmd())
invoicesCmd.AddCommand(invoiceItemsCmd())
}
func invoiceListCmd() *cobra.Command {
var count, offset int
cmd := &cobra.Command{
Use: "list",
Short: "List CRM invoices",
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
list, total, err := c.ListInvoices(cmd.Context(), count, offset)
if err != nil {
return err
}
rows := make([]map[string]any, 0, len(list))
for _, inv := range list {
rows = append(rows, onlyoffice.FlattenInvoiceRow(inv))
}
if outputFormat == "table" {
fmt.Printf("total: %d (shown: %d)\n", total, len(rows))
}
printTable([]string{"id", "number", "cost", "statusTitle", "contactName", "issueDate", "dueDate"}, rows)
return nil
},
}
cmd.Flags().IntVar(&count, "count", 50, "")
cmd.Flags().IntVar(&offset, "offset", 0, "")
return cmd
}
func invoiceGetCmd() *cobra.Command {
return &cobra.Command{
Use: "get INVOICE_ID",
Short: "Show an invoice by id (incl. lines)",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
out, err := c.GetInvoice(cmd.Context(), args[0])
if err != nil {
return err
}
printObject(out)
return nil
},
}
}
func invoiceCreateCmd() *cobra.Command {
var (
number, issueDate, dueDate, language, currency, terms, description, po string
contactID, consigneeID, itemID, opportunityID int64
price, qty float64
lineDesc string
)
cmd := &cobra.Command{
Use: "create",
Short: "Create a draft invoice with one line",
Long: `Create a CRM invoice (Draft) with a single line.
Always pass --opportunity when a deal exists (entity link at create). Updating
--opportunity later often fails with HTTP 400 — see the CRM association rules.
Example:
oo invoices create --number INV-2026-01 --contact CONTACT_ID --item ITEM_ID \
--price 300 --opportunity OPPORTUNITY_ID --line-description "Service package" \
--terms "…payment terms…"
`,
RunE: func(cmd *cobra.Command, args []string) error {
if number == "" || contactID == 0 || itemID == 0 {
return fmt.Errorf("--number, --contact and --item are required")
}
if issueDate == "" {
issueDate = time.Now().Format("2006-01-02") + "T00:00:00.0000000+01:00"
} else if !strings.Contains(issueDate, "T") {
issueDate = issueDate + "T00:00:00.0000000+01:00"
}
if dueDate == "" {
dueDate = time.Now().Add(14*24*time.Hour).Format("2006-01-02") + "T00:00:00.0000000+01:00"
} else if !strings.Contains(dueDate, "T") {
dueDate = dueDate + "T00:00:00.0000000+01:00"
}
if qty == 0 {
qty = 1
}
c, err := newOO(cmd)
if err != nil {
return err
}
out, err := c.CreateInvoice(cmd.Context(), onlyoffice.CreateInvoiceParams{
Number: number,
IssueDate: issueDate,
DueDate: dueDate,
ContactID: contactID,
ConsigneeID: consigneeID,
EntityID: opportunityID,
EntityType: 0, // Opportunity
Language: language,
Currency: currency,
ExchangeRate: 1,
PurchaseOrderNumber: po,
Terms: terms,
Description: description,
Lines: []onlyoffice.InvoiceLine{{
InvoiceItemID: itemID,
Description: lineDesc,
Quantity: qty,
Price: price,
SortOrder: 0,
}},
})
if err != nil {
return err
}
printObject(out)
return nil
},
}
cmd.Flags().StringVar(&number, "number", "", "invoice number (e.g. INV-2026-01)")
cmd.Flags().Int64Var(&contactID, "contact", 0, "bill-to company contact id (canonical company, not a PDF-only clone)")
cmd.Flags().Int64Var(&consigneeID, "consignee", 0, "optional Empfänger person/contact id")
cmd.Flags().Int64Var(&opportunityID, "opportunity", 0, "link to CRM opportunity/deal id (set at create)")
cmd.Flags().Int64Var(&itemID, "item", 0, "catalog invoice item id")
cmd.Flags().Float64Var(&price, "price", 0, "line price")
cmd.Flags().Float64Var(&qty, "qty", 1, "line quantity")
cmd.Flags().StringVar(&lineDesc, "line-description", "", "line description override")
cmd.Flags().StringVar(&issueDate, "issue-date", "", "YYYY-MM-DD (default today)")
cmd.Flags().StringVar(&dueDate, "due-date", "", "YYYY-MM-DD (default +14d)")
cmd.Flags().StringVar(&language, "language", "de-DE", "invoice language")
cmd.Flags().StringVar(&currency, "currency", "EUR", "currency abbreviation")
cmd.Flags().StringVar(&terms, "terms", "", "payment terms / footer")
cmd.Flags().StringVar(&description, "description", "", "invoice description")
cmd.Flags().StringVar(&po, "po", "", "purchase order number")
return cmd
}
func invoiceUpdateCmd() *cobra.Command {
var opportunityID int64
var description, po, terms string
cmd := &cobra.Command{
Use: "update INVOICE_ID",
Short: "Update invoice (notes, PO, terms; opportunity link unreliable)",
Long: `Update Draft invoice fields.
--opportunity often returns HTTP 400 on existing invoices. Prefer
oo invoices create … --opportunity, or delete+recreate. See the CRM association rules.
`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if opportunityID == 0 && !cmd.Flags().Changed("description") && !cmd.Flags().Changed("po") && !cmd.Flags().Changed("terms") {
return fmt.Errorf("set --opportunity and/or --description and/or --po and/or --terms")
}
c, err := newOO(cmd)
if err != nil {
return err
}
out, err := c.UpdateInvoice(cmd.Context(), args[0], onlyoffice.UpdateInvoiceParams{
EntityID: opportunityID,
EntityType: 0,
Description: description,
DescriptionSet: cmd.Flags().Changed("description"),
PurchaseOrder: po,
PurchaseOrderSet: cmd.Flags().Changed("po"),
Terms: terms,
TermsSet: cmd.Flags().Changed("terms"),
})
if err != nil {
return err
}
printObject(out)
return nil
},
}
cmd.Flags().Int64Var(&opportunityID, "opportunity", 0, "CRM opportunity/deal id (may 400 — prefer create)")
cmd.Flags().StringVar(&description, "description", "", "invoice notes (Notizen); use \\n for line breaks")
cmd.Flags().StringVar(&po, "po", "", "purchase order number")
cmd.Flags().StringVar(&terms, "terms", "", "payment terms / footer (Bedingungen)")
return cmd
}
func invoicePDFCmd() *cobra.Command {
var force bool
cmd := &cobra.Command{
Use: "pdf INVOICE_ID",
Short: "Return / regenerate invoice PDF file metadata",
Long: `GET /api/2.0/crm/invoice/{id}/pdf.
With --force, touches the Draft to clear cached fileID first (layout changes).
`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
var out map[string]any
if force {
out, err = c.ForceRegenerateInvoicePDF(cmd.Context(), args[0])
} else {
out, err = c.InvoicePDFFile(cmd.Context(), args[0])
}
if err != nil {
return err
}
printObject(out)
return nil
},
}
cmd.Flags().BoolVar(&force, "force", false, "clear PDF cache then regenerate")
return cmd
}
func invoicePDFCleanupCmd() *cobra.Command {
return &cobra.Command{
Use: "pdf-cleanup INVOICE_ID",
Short: "Delete older P-*.pdf copies on company/deal; keep invoice.fileID",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
deleted, err := c.PurgeStaleInvoicePDFs(cmd.Context(), args[0])
if err != nil {
return err
}
if outputFormat == "json" {
printObject(map[string]any{"deleted": deleted})
return nil
}
fmt.Printf("deleted %d stale PDF file(s): %v\n", len(deleted), deleted)
return nil
},
}
}
func invoiceStatusCmd() *cobra.Command {
var status string
cmd := &cobra.Command{
Use: "status INVOICE_ID [INVOICE_ID...]",
Short: "Set invoice status (draft|billed|rejected|paid)",
Long: `PUT /api/2.0/crm/invoice/status/{id}.
Billed invoices are not content-editable. Billed→Draft often does not work —
recreate as Draft instead (the CRM association rules).
`,
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
statusID, err := parseInvoiceStatus(status)
if err != nil {
return err
}
ids := make([]int64, 0, len(args))
for _, a := range args {
var id int64
if _, err := fmt.Sscan(a, &id); err != nil || id <= 0 {
return fmt.Errorf("invalid invoice id %q", a)
}
ids = append(ids, id)
}
c, err := newOO(cmd)
if err != nil {
return err
}
out, err := c.SetInvoiceStatus(cmd.Context(), statusID, ids...)
if err != nil {
return err
}
printObject(out)
return nil
},
}
cmd.Flags().StringVar(&status, "status", "draft", "draft|billed|rejected|paid or numeric id")
return cmd
}
func parseInvoiceStatus(s string) (int, error) {
s = strings.TrimSpace(strings.ToLower(s))
switch s {
case "", "draft", "1":
return onlyoffice.InvoiceStatusDraft, nil
case "billed", "2":
return onlyoffice.InvoiceStatusBilled, nil
case "rejected", "3":
return onlyoffice.InvoiceStatusRejected, nil
case "paid", "4":
return onlyoffice.InvoiceStatusPaid, nil
default:
var n int
if _, err := fmt.Sscan(s, &n); err != nil || n <= 0 {
return 0, fmt.Errorf("unknown status %q (draft|billed|rejected|paid)", s)
}
return n, nil
}
}
func invoiceDeleteCmd() *cobra.Command {
return &cobra.Command{
Use: "delete INVOICE_ID [INVOICE_ID...]",
Short: "Delete one or more invoices",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
for _, id := range args {
if _, err := c.DeleteInvoice(cmd.Context(), id); err != nil {
return err
}
fmt.Println("deleted", id)
}
return nil
},
}
}
func invoiceItemsCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "items",
Aliases: []string{"item"},
Short: "Invoice catalog items",
}
cmd.AddCommand(invoiceItemsListCmd())
cmd.AddCommand(invoiceItemsCreateCmd())
cmd.AddCommand(invoiceItemsDeleteCmd())
return cmd
}
func invoiceItemsListCmd() *cobra.Command {
var count, offset int
cmd := &cobra.Command{
Use: "list",
Short: "List catalog invoice items",
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
list, total, err := c.ListInvoiceItems(cmd.Context(), count, offset)
if err != nil {
return err
}
if outputFormat == "table" {
fmt.Printf("total: %d (shown: %d)\n", total, len(list))
for _, row := range list {
if cur, ok := row["currency"].(map[string]any); ok {
row["currency"] = cur["abbreviation"]
}
}
}
printTable([]string{"id", "title", "price", "currency", "description"}, list)
return nil
},
}
cmd.Flags().IntVar(&count, "count", 50, "")
cmd.Flags().IntVar(&offset, "offset", 0, "")
return cmd
}
func invoiceItemsCreateCmd() *cobra.Command {
var title, description, currency string
var price float64
cmd := &cobra.Command{
Use: "create",
Short: "Create a catalog invoice item",
RunE: func(cmd *cobra.Command, args []string) error {
if title == "" {
return fmt.Errorf("--title is required")
}
c, err := newOO(cmd)
if err != nil {
return err
}
out, err := c.CreateInvoiceItem(cmd.Context(), title, description, price, currency)
if err != nil {
return err
}
printObject(out)
return nil
},
}
cmd.Flags().StringVar(&title, "title", "", "item title")
cmd.Flags().StringVar(&description, "description", "", "item description")
cmd.Flags().Float64Var(&price, "price", 0, "unit price")
cmd.Flags().StringVar(&currency, "currency", "EUR", "currency")
return cmd
}
func invoiceItemsDeleteCmd() *cobra.Command {
return &cobra.Command{
Use: "delete ITEM_ID [ITEM_ID...]",
Short: "Delete catalog invoice items",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
for _, id := range args {
if _, err := c.DeleteInvoiceItem(cmd.Context(), id); err != nil {
return err
}
fmt.Println("deleted", id)
}
return nil
},
}
}
+45
View File
@@ -0,0 +1,45 @@
package main
import (
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/spf13/cobra"
)
func init() {
rootCmd.AddCommand(linkCmd())
}
// linkCmd prints deep links for file ids. Used to build third-party document
// packs whose cover embeds links to contracts and supporting files. File ids
// come from `oo projects files list` / `oo dav ls`; `oo projects files
// replace-in` keeps them clean when a document is re-uploaded.
func linkCmd() *cobra.Command {
return &cobra.Command{
Use: "link FILE_ID [FILE_ID...]",
Short: "Print OnlyOffice DocEditor deep links (Products/Files/DocEditor.aspx?fileid=…)",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
for _, id := range args {
printObject(map[string]any{
"fileid": id,
"title": fileTitle(cmd, c, id),
"url": c.FileEditorURL(id),
})
}
return nil
},
}
}
// fileTitle best-effort resolves a file title; never fails the command.
func fileTitle(cmd *cobra.Command, c *onlyoffice.Client, id string) string {
f, err := c.GetFile(cmd.Context(), id)
if err != nil || f == nil || f.Title == nil {
return ""
}
return *f.Title
}
+288 -1
View File
@@ -1,6 +1,8 @@
package main
import (
"fmt"
"os"
"strconv"
"strings"
@@ -11,7 +13,7 @@ import (
var mailsCmd = &cobra.Command{
Use: "mails",
Aliases: []string{"mail"},
Short: "OnlyOffice Workspace mail — list, read, delete",
Short: "OnlyOffice Workspace mail — list, read, draft, delete",
}
func init() {
@@ -20,6 +22,11 @@ func init() {
mailsCmd.AddCommand(mailsFoldersCmd())
mailsCmd.AddCommand(mailsListCmd())
mailsCmd.AddCommand(mailsGetCmd())
mailsCmd.AddCommand(mailsDownloadAttachmentCmd())
mailsCmd.AddCommand(mailsDraftCmd())
mailsCmd.AddCommand(mailsAttachCmd())
mailsCmd.AddCommand(mailsDraftInvoiceCmd())
mailsCmd.AddCommand(mailsSendCmd())
mailsCmd.AddCommand(mailsDeleteCmd())
}
@@ -126,6 +133,286 @@ func mailsGetCmd() *cobra.Command {
}
}
func mailsDownloadAttachmentCmd() *cobra.Command {
var outPath string
cmd := &cobra.Command{
Use: "download-attachment ATTACHMENT_ID",
Short: "Download a mail attachment by attachment id",
Long: `Download a raw attachment from OnlyOffice Mail's download.ashx handler.
Example:
oo mails download-attachment 12345 --out /tmp/attach.bin
`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if strings.TrimSpace(outPath) == "" {
return fmt.Errorf("--out is required")
}
c, err := newOO(cmd)
if err != nil {
return err
}
body, err := c.DownloadMailAttachment(cmd.Context(), args[0])
if err != nil {
return err
}
if err := writeMailAttachment(outPath, body); err != nil {
return err
}
if outputFormat == "json" {
printObject(map[string]any{
"attachmentId": args[0],
"bytes": len(body),
"path": outPath,
})
return nil
}
fmt.Printf("saved %d bytes to %s\n", len(body), outPath)
return nil
},
}
cmd.Flags().StringVar(&outPath, "out", "", "output file path")
return cmd
}
func mailsDraftCmd() *cobra.Command {
var from, to, cc, bcc, subject, body, html string
var id int64
cmd := &cobra.Command{
Use: "draft",
Short: "Create or update a mail draft (OnlyOffice Mail)",
Long: `Save a draft in /addons/mail (PUT /api/2.0/mail/drafts/save).
oo mails draft --to a@b.com --subject "…" --body "<p>…</p>"
oo mails draft --id 123 --to a@b.com --subject "…" --html "<p>…</p>"
`,
RunE: func(cmd *cobra.Command, args []string) error {
if to == "" {
return fmt.Errorf("--to is required")
}
htmlBody := body
if html != "" {
htmlBody = html
}
if htmlBody == "" {
return fmt.Errorf("--body or --html is required")
}
c, err := newOO(cmd)
if err != nil {
return err
}
out, err := c.SaveMailDraft(cmd.Context(), onlyoffice.SaveMailDraftParams{
ID: id,
From: from,
To: to,
Cc: cc,
Bcc: bcc,
Subject: subject,
Body: onlyoffice.PlainTextToMailHTML(htmlBody),
})
if err != nil {
return err
}
printObject(out)
return nil
},
}
cmd.Flags().Int64Var(&id, "id", 0, "existing draft id (0 = create)")
cmd.Flags().StringVar(&from, "from", "", "from address (default: first enabled mailbox)")
cmd.Flags().StringVar(&to, "to", "", "recipient (required)")
cmd.Flags().StringVar(&cc, "cc", "", "cc")
cmd.Flags().StringVar(&bcc, "bcc", "", "bcc")
cmd.Flags().StringVar(&subject, "subject", "", "subject")
cmd.Flags().StringVar(&body, "body", "", "plain text or HTML body")
cmd.Flags().StringVar(&html, "html", "", "HTML body (alias of --body when set)")
return cmd
}
func mailsAttachCmd() *cobra.Command {
var fileID int64
cmd := &cobra.Command{
Use: "attach MESSAGE_ID",
Short: "Attach an OnlyOffice Files document to a draft/message",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if fileID <= 0 {
return fmt.Errorf("--file-id is required")
}
c, err := newOO(cmd)
if err != nil {
return err
}
out, err := c.AttachMailDocument(cmd.Context(), args[0], fileID)
if err != nil {
return err
}
printObject(out)
return nil
},
}
cmd.Flags().Int64Var(&fileID, "file-id", 0, "OnlyOffice file id (e.g. invoice PDF)")
return cmd
}
func mailsDraftInvoiceCmd() *cobra.Command {
var to, from, subject, body string
var invoiceID int
cmd := &cobra.Command{
Use: "draft-invoice",
Short: "Create a mail draft with regenerated invoice PDF attached",
Long: `Regenerate the CRM invoice PDF, save an OnlyOffice Mail draft, and attach the PDF.
Does not send. Open /addons/mail/#drafts to review.
oo mails draft-invoice --invoice 16 --to info@example.com
`,
RunE: func(cmd *cobra.Command, args []string) error {
if invoiceID <= 0 || to == "" {
return fmt.Errorf("--invoice and --to are required")
}
c, err := newOO(cmd)
if err != nil {
return err
}
pdf, err := c.ForceRegenerateInvoicePDF(cmd.Context(), strconv.Itoa(invoiceID))
if err != nil {
return err
}
fileID := onlyoffice.Int64FromMap(pdf, "id")
if fileID <= 0 {
return fmt.Errorf("invoice PDF has no file id: %+v", pdf)
}
inv, err := c.GetInvoice(cmd.Context(), strconv.Itoa(invoiceID))
if err != nil {
return err
}
number := strings.TrimSpace(fmt.Sprint(inv["number"]))
cost := formatInvoiceCostEUR(inv["cost"])
if subject == "" {
subject = fmt.Sprintf("Invoice %s (%s EUR)", number, cost)
}
if body == "" {
body = fmt.Sprintf(`Hello,
please find attached invoice %s for %s EUR.
Payment terms: see invoice notes.
Best regards`, number, cost)
}
draft, err := c.SaveMailDraft(cmd.Context(), onlyoffice.SaveMailDraftParams{
From: from,
To: to,
Subject: subject,
Body: onlyoffice.MailHTMLWithBlankParagraphs(onlyoffice.PlainTextToMailHTML(body)),
})
if err != nil {
return err
}
mid := fmt.Sprint(draft["id"])
att, err := c.AttachMailDocument(cmd.Context(), mid, fileID)
if err != nil {
return fmt.Errorf("draft %s created but attach failed: %w", mid, err)
}
if outputFormat == "json" {
printObject(map[string]any{"draft": draft, "attachment": att, "pdfFileId": fileID})
return nil
}
fmt.Printf("draft %s to=%s subject=%q pdfFileId=%d\n", mid, to, subject, fileID)
fmt.Printf("open: %s/addons/mail/#drafts\n", strings.TrimRight(os.Getenv("ONLYOFFICE_URL"), "/"))
return nil
},
}
cmd.Flags().IntVar(&invoiceID, "invoice", 0, "CRM invoice id")
cmd.Flags().StringVar(&to, "to", "", "recipient")
cmd.Flags().StringVar(&from, "from", "", "from address (default mailbox)")
cmd.Flags().StringVar(&subject, "subject", "", "override subject")
cmd.Flags().StringVar(&body, "body", "", "override plain-text body")
return cmd
}
func formatInvoiceCostEUR(v any) string {
s := strings.TrimSpace(fmt.Sprint(v))
s = strings.TrimSuffix(s, ".00")
s = strings.TrimSuffix(s, ".0")
if s == "" || s == "<nil>" {
return "?"
}
return s
}
func writeMailAttachment(path string, body []byte) error {
if strings.TrimSpace(path) == "" {
return fmt.Errorf("attachment output path is required")
}
return os.WriteFile(path, body, 0o644)
}
func mailsSendCmd() *cobra.Command {
var from, to, cc, bcc, subject, body, html string
var id int64
cmd := &cobra.Command{
Use: "send",
Short: "Send a mail message (OnlyOffice Mail)",
Long: `Send via PUT /api/2.0/mail/messages/send.json.
oo mails send --id 7803 --body "…" # send referencing a draft id
oo mails send --to a@b.com --subject "…" --body "…"
oo mails send --id 7803 --to a@b.com --subject "…" --body "…" --cc x@y.com
IMPORTANT: send.json does NOT copy subject/body from the referenced draft — the
content must be in this request (--subject/--body). Cc/Bcc are omitted when empty
(the API 400s on empty strings). The API send does not append the UI signature —
put the chat line in --body if needed.
`,
RunE: func(cmd *cobra.Command, args []string) error {
if to == "" && id == 0 {
return fmt.Errorf("--to is required (or --id of an existing draft)")
}
htmlBody := body
if html != "" {
htmlBody = html
}
if htmlBody == "" && id != 0 {
// The send.json endpoint does NOT copy subject/body from the
// referenced draft — an empty body here sends an empty message.
// Warn instead of silently mailing an empty email.
return fmt.Errorf("--body/--html is required when sending by --id (send.json needs the content in the request)")
}
if htmlBody == "" && to == "" {
return fmt.Errorf("--body is required for a fresh message")
}
c, err := newOO(cmd)
if err != nil {
return err
}
raw, err := c.SendMail(cmd.Context(), onlyoffice.SendMailParams{
ID: id,
From: from,
To: to,
Cc: cc,
Bcc: bcc,
Subject: subject,
Body: htmlBody,
})
if err != nil {
return err
}
fmt.Println(string(raw))
return nil
},
}
cmd.Flags().Int64Var(&id, "id", 0, "existing draft id to send (0 = fresh message)")
cmd.Flags().StringVar(&from, "from", "", "from address (default: first enabled mailbox)")
cmd.Flags().StringVar(&to, "to", "", "recipient (required unless --id)")
cmd.Flags().StringVar(&cc, "cc", "", "cc")
cmd.Flags().StringVar(&bcc, "bcc", "", "bcc")
cmd.Flags().StringVar(&subject, "subject", "", "subject")
cmd.Flags().StringVar(&body, "body", "", "plain text or HTML body")
cmd.Flags().StringVar(&html, "html", "", "HTML body (alias of --body when set)")
return cmd
}
func mailsDeleteCmd() *cobra.Command {
return &cobra.Command{
Use: "delete ID [ID...]",
+28
View File
@@ -0,0 +1,28 @@
package main
import (
"os"
"path/filepath"
"testing"
)
func TestWriteMailAttachment(t *testing.T) {
path := filepath.Join(t.TempDir(), "attach.bin")
body := []byte("payload")
if err := writeMailAttachment(path, body); err != nil {
t.Fatalf("writeMailAttachment: %v", err)
}
got, err := os.ReadFile(path)
if err != nil {
t.Fatalf("ReadFile: %v", err)
}
if string(got) != string(body) {
t.Fatalf("body = %q", got)
}
}
func TestWriteMailAttachmentRequiresPath(t *testing.T) {
if err := writeMailAttachment("", []byte("x")); err == nil {
t.Fatal("expected error for empty path")
}
}
+16 -9
View File
@@ -3,19 +3,26 @@
// Command tree is subject-based (mirrors the library split and the `tea` CLI):
//
// oo calendar list | events | add | delete
// oo projects list | get | milestones | create | update | delete | files (list|upload|download|rename|delete)
// oo projects list | get | milestones | milestone-create | milestone-delete | board-sync | create | update | delete | contacts (add|remove) | team (list|add|remove|set) | link-authors | link-git | files (list|upload|replace-in|update|download|rename|delete|dedupe|as-md|put-md|put-txt|put-xlsx)
// oo tasks list | get | create | update | delete | subtask add | files (list|upload|detach)
// oo users list | self (alias: oo whoami)
// oo contacts list | get | delete | info-add | dedupe-info
// oo users list | self | get | create | update | delete | block | unblock | password | check (alias: oo whoami)
// oo link FILE_ID [FILE_ID...] DocEditor deep links (Products/Files/DocEditor.aspx?fileid=…)
// oo contacts list | get | delete | info-add | merge | dedupe-info | tags | tag-add | tag-create | tag-remove
// oo persons list | create | delete | dedupe
// oo companies list | create | delete | dedupe | dedupe-persons
// oo opportunities list | get | create | delete | stages | member-add | dedupe | dedupe-members | fix-titles
// oo opportunities list | get | create | update | delete | stages | member-add | dedupe | dedupe-members | fix-titles
// oo cases list | create | delete | member-add
// oo crm-tasks list | create | delete | categories
// oo crm cleanup
// oo mails accounts | folders | list | get | delete
// oo applications sync
// oo catalog scan-contacts | scan-projects | scan-thunderbird | merge | match | apply
// oo crm-tasks list | create | delete | categories | reassign-self
// oo crm audit | cleanup
// oo mails accounts | folders | list | get | download-attachment | draft | attach | draft-invoice | send | delete
// oo invoices list | get | create | update | pdf | pdf-cleanup | status | delete | items …
// oo docs tools | convert | pdf | presigned | csv | json | optimize | ocr | hocr | as-md | put-md | put-txt | put-xlsx
// oo catalog match | merge | apply | scan-contacts | scan-projects | scan-thunderbird
// oo dav ls | move | copy | mkdir | rename-file | rename-folder | download | fileops
// oo search QUERY [--content] [--folder ID] [--limit N] [--backend oo|own] [--json]
// oo index folder FOLDER_ID | files FILE_ID... [--recursive] [--exts pdf] [--dry-run]
//
// CRM association rules live with the private oo-workspace tooling.
//
// Every list supports `--output/-o json|table` (table is the default).
//
+35
View File
@@ -19,6 +19,7 @@ func init() {
opportunitiesCmd.AddCommand(oppListCmd())
opportunitiesCmd.AddCommand(oppGetCmd())
opportunitiesCmd.AddCommand(oppCreateCmd())
opportunitiesCmd.AddCommand(oppUpdateCmd())
opportunitiesCmd.AddCommand(oppDeleteCmd())
opportunitiesCmd.AddCommand(oppStagesCmd())
opportunitiesCmd.AddCommand(oppMemberAddCmd())
@@ -118,6 +119,40 @@ func oppCreateCmd() *cobra.Command {
return cmd
}
func oppUpdateCmd() *cobra.Command {
var title, desc string
var stage int64
var bid float64
cmd := &cobra.Command{
Use: "update OPPORTUNITY_ID",
Short: "Update opportunity title, description, stage, or bid",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
out, err := c.UpdateOpportunity(cmd.Context(), args[0], onlyoffice.UpdateOpportunityParams{
Title: title,
Description: desc,
StageID: stage,
BidValue: bid,
BidValueSet: cmd.Flags().Changed("bid"),
})
if err != nil {
return err
}
printObject(out)
return nil
},
}
cmd.Flags().StringVar(&title, "title", "", "new title")
cmd.Flags().StringVar(&desc, "description", "", "new description")
cmd.Flags().Int64Var(&stage, "stage", 0, "pipeline stage id")
cmd.Flags().Float64Var(&bid, "bid", 0, "bid value")
return cmd
}
func oppDeleteCmd() *cobra.Command {
return &cobra.Command{
Use: "delete OPPORTUNITY_ID [OPPORTUNITY_ID...]",
+210 -7
View File
@@ -6,9 +6,9 @@ import (
"os/exec"
"strconv"
"strings"
"time"
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/eslider/go-onlyoffice/catalog"
"github.com/spf13/cobra"
)
@@ -23,10 +23,143 @@ func init() {
projectsCmd.AddCommand(prjListCmd())
projectsCmd.AddCommand(prjGetCmd())
projectsCmd.AddCommand(prjMilestonesCmd())
projectsCmd.AddCommand(prjMilestoneCreateCmd())
projectsCmd.AddCommand(prjMilestoneDeleteCmd())
projectsCmd.AddCommand(prjCreateCmd())
projectsCmd.AddCommand(prjUpdateCmd())
projectsCmd.AddCommand(prjDeleteCmd())
projectsCmd.AddCommand(prjContactsCmd())
projectsCmd.AddCommand(prjTeamCmd())
}
func prjTeamCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "team",
Short: "Project team (portal users) — CRUD",
Long: `Project team members are portal users (People), not CRM contacts.
CRM companies/persons linked to a project live under 'oo projects contacts'.`,
}
cmd.AddCommand(prjTeamListCmd())
cmd.AddCommand(prjTeamAddCmd())
cmd.AddCommand(prjTeamRemoveCmd())
cmd.AddCommand(prjTeamSetCmd())
return cmd
}
func prjTeamListCmd() *cobra.Command {
return &cobra.Command{
Use: "list PROJECT_ID",
Short: "List portal users on the project team",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
pid, err := strconv.Atoi(args[0])
if err != nil {
return fmt.Errorf("project id: %w", err)
}
c, err := newOO(cmd)
if err != nil {
return err
}
list, err := c.ListProjectTeam(cmd.Context(), pid)
if err != nil {
return err
}
printTable([]string{"id", "displayName", "userName", "email", "isAdmin"}, teamRows(list))
return nil
},
}
}
func prjTeamAddCmd() *cobra.Command {
return &cobra.Command{
Use: "add PROJECT_ID USER_ID [USER_ID...]",
Short: "Add portal user(s) to the project team",
Args: cobra.MinimumNArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
pid, err := strconv.Atoi(args[0])
if err != nil {
return fmt.Errorf("project id: %w", err)
}
c, err := newOO(cmd)
if err != nil {
return err
}
for _, uid := range args[1:] {
if _, err := c.AddProjectTeamUser(cmd.Context(), pid, uid); err != nil {
return fmt.Errorf("add %s: %w", uid, err)
}
printObject(map[string]any{"project_id": pid, "user_id": uid, "added": true})
}
return nil
},
}
}
func prjTeamRemoveCmd() *cobra.Command {
return &cobra.Command{
Use: "remove PROJECT_ID USER_ID [USER_ID...]",
Short: "Remove portal user(s) from the project team",
Args: cobra.MinimumNArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
pid, err := strconv.Atoi(args[0])
if err != nil {
return fmt.Errorf("project id: %w", err)
}
c, err := newOO(cmd)
if err != nil {
return err
}
for _, uid := range args[1:] {
if _, err := c.RemoveProjectTeamUser(cmd.Context(), pid, uid); err != nil {
return fmt.Errorf("remove %s: %w", uid, err)
}
printObject(map[string]any{"project_id": pid, "user_id": uid, "removed": true})
}
return nil
},
}
}
func prjTeamSetCmd() *cobra.Command {
var notify bool
cmd := &cobra.Command{
Use: "set PROJECT_ID USER_ID [USER_ID...]",
Short: "Replace the project team with the given users (register several at once)",
Args: cobra.MinimumNArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
pid, err := strconv.Atoi(args[0])
if err != nil {
return fmt.Errorf("project id: %w", err)
}
c, err := newOO(cmd)
if err != nil {
return err
}
team, err := c.SetProjectTeam(cmd.Context(), pid, args[1:], notify)
if err != nil {
return err
}
printTable([]string{"id", "displayName", "userName", "email", "isAdmin"}, teamRows(team))
return nil
},
}
cmd.Flags().BoolVar(&notify, "notify", false, "notify added members")
return cmd
}
// teamRows maps raw team member maps into table rows.
func teamRows(list []map[string]any) []map[string]any {
rows := make([]map[string]any, 0, len(list))
for _, m := range list {
rows = append(rows, map[string]any{
"id": idString(m, "id"),
"displayName": m["displayName"],
"userName": m["userName"],
"email": m["email"],
"isAdmin": m["isAdmin"],
})
}
return rows
}
func prjListCmd() *cobra.Command {
@@ -119,6 +252,78 @@ func prjMilestonesCmd() *cobra.Command {
}
}
func prjMilestoneCreateCmd() *cobra.Command {
var deadline, desc string
var key bool
cmd := &cobra.Command{
Use: "milestone-create PROJECT_ID TITLE",
Short: "Create a project milestone (Gantt row)",
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
pid, err := strconv.Atoi(args[0])
if err != nil {
return fmt.Errorf("project id must be integer: %w", err)
}
if deadline == "" {
return fmt.Errorf("--deadline YYYY-MM-DD is required")
}
day, err := time.Parse("2006-01-02", deadline)
if err != nil {
return fmt.Errorf("deadline: %w", err)
}
ms, err := c.CreateMilestone(onlyoffice.NewMilestoneRequest{
ProjectID: pid,
Title: args[1],
Deadline: onlyoffice.Time(day),
Description: desc,
IsKey: key,
})
if err != nil {
return err
}
printObject(map[string]any{
"id": derefInt64(ms.ID),
"title": derefString(ms.Title),
})
return nil
},
}
cmd.Flags().StringVar(&deadline, "deadline", "", "deadline YYYY-MM-DD")
cmd.Flags().StringVar(&desc, "description", "", "description")
cmd.Flags().BoolVar(&key, "key", false, "mark as key milestone")
return cmd
}
func prjMilestoneDeleteCmd() *cobra.Command {
return &cobra.Command{
Use: "milestone-delete MILESTONE_ID [MILESTONE_ID...]",
Aliases: []string{"milestone-rm"},
Short: "Delete project milestone(s) by id",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
for _, raw := range args {
id, err := strconv.ParseInt(raw, 10, 64)
if err != nil {
return fmt.Errorf("milestone id %q must be integer: %w", raw, err)
}
if err := c.DeleteMilestone(id); err != nil {
return fmt.Errorf("delete milestone %d: %w", id, err)
}
printObject(map[string]any{"milestone_id": id, "deleted": true})
}
return nil
},
}
}
func prjCreateCmd() *cobra.Command {
var desc, resp string
var country, company string
@@ -143,7 +348,7 @@ full title is composed as "CC | Company | Title".`,
}
title := args[0]
if country != "" || company != "" {
title = catalog.FormatProjectTitle(country, company, args[0])
title = onlyoffice.FormatProjectTitle(country, company, args[0])
}
p, err := c.CreateProject(onlyoffice.NewProjectRequest{
Title: title,
@@ -381,9 +586,9 @@ func prjContactsLinkGitCmd() *cobra.Command {
Short: "Link CRM persons matched from git shortlog (+ optional companies)",
Long: `Runs git shortlog -sne --all in --git-root, matches author emails to CRM
persons (FindPersonByEmail), and links found contacts to the project.
Does not create new persons — approve/apply them via catalog first.
Does not create new persons — create/link them in CRM first.
Also links --company-id contacts (e.g. Acme + end-client company).`,
Also links --company-id contacts (employer and/or client company).`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if gitRoot == "" && len(companyIDs) == 0 {
@@ -425,10 +630,8 @@ func prjContactsLinkAuthorsCmd() *cobra.Command {
Long: `Same as link-git, but reads authors from a file produced by:
git shortlog -sne --all > authors.txt
# or via ssh:
ssh ops-host 'git -C /path shortlog -sne --all' > authors.txt
Then: oo projects contacts link-authors 59 --from authors.txt --company-id 9`,
Then: oo projects contacts link-authors 59 --from authors.txt --company-id COMPANY_ID`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if authorsFile == "" && len(companyIDs) == 0 {
+54
View File
@@ -0,0 +1,54 @@
package main
import (
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/spf13/cobra"
)
func init() {
projectsCmd.AddCommand(prjBoardSyncCmd())
}
// prjBoardSyncCmd upserts project milestones/tasks from a board YAML.
func prjBoardSyncCmd() *cobra.Command {
var apply bool
cmd := &cobra.Command{
Use: "board-sync BOARD.yaml",
Short: "Upsert project milestones/tasks from a board YAML (dry-run by default)",
Long: `Reads a board YAML (projects → milestones → tasks) and creates only the
milestones/tasks that are missing, matching by exact title. Existing entries are
left untouched, so the same file can be re-applied safely.
Dry-run by default; pass --apply to write to OnlyOffice.`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
board, err := onlyoffice.LoadBoard(args[0])
if err != nil {
return err
}
c, err := newOO(cmd)
if err != nil {
return err
}
res, err := c.SyncBoard(cmd.Context(), board, apply)
if err != nil {
return err
}
mode := "dry-run"
if apply {
mode = "apply"
}
printObject(map[string]any{
"mode": mode,
"projects": len(board.Projects),
"created_milestones": res.CreatedMilestones,
"skipped_milestones": res.SkippedMilestones,
"created_tasks": res.CreatedTasks,
"skipped_tasks": res.SkippedTasks,
})
return nil
},
}
cmd.Flags().BoolVar(&apply, "apply", false, "write to OnlyOffice (default: dry-run)")
return cmd
}
+217 -13
View File
@@ -3,6 +3,7 @@ package main
import (
"fmt"
"os"
"path/filepath"
"strconv"
"time"
@@ -21,12 +22,48 @@ func projectFilesCmd() *cobra.Command {
}
cmd.AddCommand(prjFilesListCmd())
cmd.AddCommand(prjFilesUploadCmd())
cmd.AddCommand(prjFilesReplaceInCmd())
cmd.AddCommand(prjFilesUpdateCmd())
cmd.AddCommand(prjFilesDownloadCmd())
cmd.AddCommand(prjFilesRenameCmd())
cmd.AddCommand(prjFilesDeleteCmd())
cmd.AddCommand(prjFilesDedupeCmd())
// Convenience aliases into oo docs (md↔docx / OCR pipeline).
cmd.AddCommand(aliasDocsAsMD())
cmd.AddCommand(aliasDocsPutMD())
cmd.AddCommand(aliasDocsPutTxt())
cmd.AddCommand(aliasDocsPutXlsx())
return cmd
}
func aliasDocsAsMD() *cobra.Command {
c := docsAsMDCmd()
c.Use = "as-md FILE_ID"
c.Short = "Alias of `oo docs as-md` — download OO file as Markdown (OCR if needed)"
return c
}
func aliasDocsPutMD() *cobra.Command {
c := docsPutMDCmd()
c.Use = "put-md PROJECT_ID MARKDOWN_PATH"
c.Short = "Alias of `oo docs put-md` — Markdown→DOCX upload into project"
return c
}
func aliasDocsPutTxt() *cobra.Command {
c := docsPutTxtCmd()
c.Use = "put-txt PROJECT_ID TEXT_PATH"
c.Short = "Alias of `oo docs put-txt` — plain text→DOCX upload into project"
return c
}
func aliasDocsPutXlsx() *cobra.Command {
c := docsPutXlsxCmd()
c.Use = "put-xlsx PROJECT_ID [LOCAL_XLSX]"
c.Short = "Alias of `oo docs put-xlsx` — generate/upload XLSX with formulas"
return c
}
func prjFilesListCmd() *cobra.Command {
var showFolders bool
cmd := &cobra.Command{
@@ -49,8 +86,8 @@ func prjFilesListCmd() *cobra.Command {
continue
}
frows = append(frows, map[string]any{
"id": folderIDStr(f),
"title": derefString(f.Title),
"id": folderIDStr(f),
"title": derefString(f.Title),
"filesCount": derefInt(f.FilesCount),
"foldersCount": derefInt(f.FoldersCount),
})
@@ -73,10 +110,13 @@ func prjFilesListCmd() *cobra.Command {
}
func prjFilesUploadCmd() *cobra.Command {
return &cobra.Command{
var replace, allowDuplicate bool
cmd := &cobra.Command{
Use: "upload PROJECT_ID LOCAL_PATH [LOCAL_PATH...]",
Short: "Upload file(s) into the project's Documents folder",
Args: cobra.MinimumNArgs(2),
Short: "Upload file(s) into the project's Documents folder (upsert by stem|ext)",
Long: `Default: replace an existing file with the same logical name (stem|ext), like cp overwrite.
Pass --no-replace to fail when the name is taken; --allow-duplicate to always create a new file id.`,
Args: cobra.MinimumNArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
@@ -84,15 +124,93 @@ func prjFilesUploadCmd() *cobra.Command {
}
pid := args[0]
for _, p := range args[1:] {
entry, err := c.UploadProjectFile(cmd.Context(), pid, p)
var entry *onlyoffice.FileEntry
var deleted []int
switch {
case allowDuplicate:
entry, err = c.UploadProjectFile(cmd.Context(), pid, p)
case replace:
entry, deleted, err = c.UploadProjectFileReplacing(cmd.Context(), pid, p)
default:
entry, err = c.UploadProjectFileNoClobber(cmd.Context(), pid, p)
}
if err != nil {
return err
}
printObject(fileEntryToMap(entry))
obj := fileEntryToMap(entry)
if len(deleted) > 0 {
obj["replaced_file_ids"] = deleted
}
printObject(obj)
}
return nil
},
}
cmd.Flags().BoolVar(&replace, "replace", true, "replace same stem|ext in project folder (default)")
cmd.Flags().BoolVar(&allowDuplicate, "allow-duplicate", false, "always create a new file even when the name exists")
return cmd
}
func prjFilesReplaceInCmd() *cobra.Command {
return &cobra.Command{
Use: "replace-in FOLDER_ID LOCAL_PATH [LOCAL_PATH...]",
Short: "Replace same-named file(s) in a folder: hard delete + fresh upload (no version history)",
Long: `Deletes any file in FOLDER_ID with the same stem|ext (hard delete — the CLI
delete is permanent) and uploads the local file fresh. Unlike 'update' this
leaves a single clean version.
Why it exists: repeated 'update' of a shared document accumulated a visible
version history and left a stale id. replace-in yields one clean revision; then
point links at the returned id (or keep an nginx alias for the legacy fileid).
Note: file ids are server-assigned; a fresh upload gets a new id.`,
Args: cobra.MinimumNArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
folderID := args[0]
for _, p := range args[1:] {
stem := onlyoffice.UploadStemFromLocal(p)
ext := onlyoffice.UploadExtFromLocal(p)
deleted, derr := c.DeleteFilesByDedupKey(cmd.Context(), folderID, stem, ext)
if derr != nil {
return derr
}
ent, uerr := c.UploadToFolder(cmd.Context(), folderID, p)
if uerr != nil {
return uerr
}
obj := fileEntryToMap(ent)
if len(deleted) > 0 {
obj["replaced_file_ids"] = deleted
}
printObject(obj)
}
return nil
},
}
}
func prjFilesUpdateCmd() *cobra.Command {
return &cobra.Command{
Use: "update FILE_ID LOCAL_PATH",
Short: "Overwrite an existing Documents file with new content (new version)",
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
entry, err := c.UpdateFile(cmd.Context(), args[0], args[1])
if err != nil {
return err
}
printObject(fileEntryToMap(entry))
return nil
},
}
}
func prjFilesDownloadCmd() *cobra.Command {
@@ -106,20 +224,22 @@ func prjFilesDownloadCmd() *cobra.Command {
if err != nil {
return err
}
f, err := c.GetFile(cmd.Context(), args[0])
ctx := cmd.Context()
store := c.Files()
e, err := store.Stat(ctx, args[0])
if err != nil {
return err
}
path := to
if path == "" {
path = onlyoffice.SafeLocalFileName(onlyoffice.FileEntryTitle(f))
path = onlyoffice.SafeLocalFileName(e.Title)
}
out, err := os.Create(path)
if err != nil {
return err
}
defer out.Close()
n, err := c.DownloadFile(cmd.Context(), args[0], out)
n, err := store.Download(ctx, args[0], out)
if err != nil {
_ = os.Remove(path)
return err
@@ -146,11 +266,16 @@ func prjFilesRenameCmd() *cobra.Command {
if err != nil {
return err
}
entry, err := c.RenameFile(cmd.Context(), args[0], args[1])
store := c.Files()
if err := store.Rename(cmd.Context(), args[0], args[1]); err != nil {
return err
}
entry, err := store.Stat(cmd.Context(), args[0])
if err != nil {
return err
}
printObject(fileEntryToMap(entry))
entry.Title = args[1]
printObject(entryToMap(entry))
return nil
},
}
@@ -175,7 +300,7 @@ func prjFilesDeleteCmd() *cobra.Command {
}
ids = append(ids, id)
}
if err := c.DeleteFiles(cmd.Context(), ids); err != nil {
if err := c.Files().Delete(cmd.Context(), args); err != nil {
return err
}
printObject(map[string]any{"deleted": ids})
@@ -184,6 +309,62 @@ func prjFilesDeleteCmd() *cobra.Command {
}
}
func prjFilesDedupeCmd() *cobra.Command {
var apply, cross bool
cmd := &cobra.Command{
Use: "dedupe PROJECT_ID",
Short: "Find (and optionally remove) duplicate files in project Documents folders",
Long: `Duplicates share the same logical name: stem|ext (OO title+fileExst).
Default: dry-run report. Pass --apply to delete older copies (keeps newest; --cross prefers non-trash folders).`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
groups, deleted, err := c.DedupeProject(cmd.Context(), args[0], onlyoffice.DedupOptions{
CrossFolder: cross,
}, apply)
if err != nil {
return err
}
rows := make([]map[string]any, 0, len(groups))
for _, g := range groups {
row := map[string]any{
"key": g.Key,
"folder_id": g.FolderID,
"folder_title": g.FolderTitle,
"keep_id": fileIDStr(g.Keep),
"keep_title": onlyoffice.FileEntryTitle(g.Keep),
"remove_count": len(g.Remove),
}
removeIDs := make([]string, 0, len(g.Remove))
for _, f := range g.Remove {
removeIDs = append(removeIDs, fileIDStr(f))
}
row["remove_ids"] = removeIDs
rows = append(rows, row)
}
out := map[string]any{
"project_id": args[0],
"dry_run": !apply,
"cross": cross,
"groups": len(groups),
"duplicates": rows,
}
if apply {
out["deleted_ids"] = deleted
}
printObject(out)
return nil
},
}
cmd.Flags().BoolVar(&apply, "apply", false, "delete duplicate files (default: report only)")
cmd.Flags().BoolVar(&cross, "cross", false, "also dedupe same stem|ext across folders (prefers non-_trash)")
return cmd
}
func fileEntryRows(files []*onlyoffice.FileEntry) []map[string]any {
rows := make([]map[string]any, 0, len(files))
for _, f := range files {
@@ -208,6 +389,29 @@ func fileEntryToMap(f *onlyoffice.FileEntry) map[string]any {
return m
}
// entryToMap renders a canonical Entry with the same keys as fileEntryToMap.
func entryToMap(e onlyoffice.Entry) map[string]any {
m := map[string]any{
"id": e.ID,
"title": e.Title,
"fileExst": filepath.Ext(e.Title),
"contentLength": contentLengthString(e.Size),
}
if e.Updated != "" {
m["updated"] = e.Updated
} else if !e.Modified.IsZero() {
m["updated"] = e.Modified.Format(time.RFC3339)
}
return m
}
func contentLengthString(n int64) string {
if n <= 0 {
return ""
}
return strconv.FormatInt(n, 10)
}
func fileIDStr(f *onlyoffice.FileEntry) string {
if f == nil || f.ID == nil {
return ""
+102
View File
@@ -0,0 +1,102 @@
package main
import (
"fmt"
"strings"
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/eslider/go-onlyoffice/cmd/internal/bootstrap"
"github.com/spf13/cobra"
)
func init() {
rootCmd.AddCommand(searchCmd())
}
// searchCmd queries the OnlyOffice document index through the file facade. The
// REST /api/2.0/files/@search endpoint only searches file names in the database;
// content search needs Elasticsearch (see docs/elasticsearch.md).
func searchCmd() *cobra.Command {
var (
content bool
folder string
limit int
backend string
asJSON bool
substring bool
)
cmd := &cobra.Command{
Use: "search QUERY...",
Short: "Full-text search over documents by name, optionally by content (Elasticsearch)",
Long: "Search the OnlyOffice Documents index.\n\n" +
"By default only file names are matched. With --content the query also\n" +
"matches extracted document text (document.attachment.content); this covers\n" +
"Office formats (docx/xlsx/pptx) and is slower.\n\n" +
"--backend own queries the separate index populated by `oo index`\n" +
"(ONLYOFFICE_ES_TEXT_INDEX, default oo_docs_text) instead, which also holds\n" +
"PDFs and scans (see docs/elasticsearch.md).\n\n" +
"Requires ONLYOFFICE_ES_URL (and optionally ONLYOFFICE_ES_INDEX,\n" +
"ONLYOFFICE_TENANT). See docs/elasticsearch.md for the tunnel setup.",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if asJSON {
outputFormat = "json"
}
bootstrap.LoadEnv()
c := onlyoffice.NewClient(onlyoffice.GetEnvironmentCredentials())
var (
searcher onlyoffice.Searcher
err error
)
switch strings.ToLower(strings.TrimSpace(backend)) {
case "", "oo", "elasticsearch":
searcher, err = c.Files().Search()
case "own", "es-text":
searcher, err = onlyoffice.NewESTextIndex(onlyoffice.ESTextConfigFromEnv())
default:
return fmt.Errorf("unknown search backend %q (want oo|own)", backend)
}
if err != nil {
return err
}
hits, err := searcher.Search(cmd.Context(), onlyoffice.SearchQuery{
Text: strings.Join(args, " "),
InContent: content,
FolderID: folder,
Limit: limit,
Substring: substring,
})
if err != nil {
return err
}
rows := make([]map[string]any, 0, len(hits))
for _, h := range hits {
folderPath := h.Path
if len(folderPath) == 0 && h.ParentID != "" {
folderPath = []string{h.ParentID}
}
rows = append(rows, map[string]any{
"path": c.UniquePath(cmd.Context(), folderPath, h.Title),
"id": h.ID,
"title": h.Title,
"folder": h.ParentID,
"score": h.Score,
"highlight": h.Highlight,
})
}
if outputFormat == "json" {
printJSON(rows)
return nil
}
printTable([]string{"path", "id", "title", "folder", "score", "highlight"}, rows)
return nil
},
}
cmd.Flags().BoolVar(&content, "content", false, "also match extracted document content")
cmd.Flags().BoolVar(&substring, "substring", false, "case-insensitive *term* title match; multiple QUERY args are ANDed")
cmd.Flags().StringVar(&folder, "folder", "", "limit to a Documents folder id (matches the folder subtree)")
cmd.Flags().IntVar(&limit, "limit", 20, "maximum number of results")
cmd.Flags().StringVar(&backend, "backend", "oo", "index to query: oo (OnlyOffice) | own (oo index)")
cmd.Flags().BoolVar(&asJSON, "json", false, "shorthand for --output json")
return cmd
}
+46
View File
@@ -0,0 +1,46 @@
package main
import (
"bytes"
"strings"
"testing"
)
func TestSearchCommandRegisteredWithFlags(t *testing.T) {
cmd, _, err := rootCmd.Find([]string{"search"})
if err != nil {
t.Fatal(err)
}
if cmd.Name() != "search" {
t.Fatalf("search resolved to %q", cmd.Name())
}
for _, name := range []string{"content", "folder", "limit", "json"} {
if cmd.Flags().Lookup(name) == nil {
t.Errorf("search: missing --%s flag", name)
}
}
if got := cmd.Flags().Lookup("limit").DefValue; got != "20" {
t.Errorf("--limit default = %q, want 20", got)
}
}
func TestSearchWithoutESURLIsClearError(t *testing.T) {
clearEnv(t, "ONLYOFFICE_ES_URL", "ONLYOFFICE_ES_INDEX", "ONLYOFFICE_TENANT")
errBuf := &bytes.Buffer{}
rootCmd.SetErr(errBuf)
rootCmd.SetOut(&bytes.Buffer{})
rootCmd.SetArgs([]string{"search", "Rechnung"})
t.Cleanup(func() {
rootCmd.SetArgs(nil)
rootCmd.SetOut(nil)
rootCmd.SetErr(nil)
})
err := rootCmd.Execute()
if err == nil {
t.Fatal("expected error without ONLYOFFICE_ES_URL")
}
if !strings.Contains(err.Error(), "ONLYOFFICE_ES_URL") {
t.Fatalf("error %q missing ONLYOFFICE_ES_URL", err.Error())
}
}
+55 -3
View File
@@ -2,7 +2,11 @@ package main
import (
"fmt"
"os"
"strconv"
"time"
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/spf13/cobra"
)
@@ -82,11 +86,12 @@ func taskGetCmd() *cobra.Command {
}
func taskCreateCmd() *cobra.Command {
var project, desc, deadline, prio string
var project, desc, deadline, start, prio string
var milestone int64
cmd := &cobra.Command{
Use: "create TITLE",
Aliases: []string{"add"},
Short: "Create a project task",
Short: "Create a project task (Gantt bar when --start/--deadline/--milestone set)",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
@@ -100,6 +105,51 @@ func taskCreateCmd() *cobra.Command {
case "low":
p = -1
}
if start != "" || milestone != 0 {
pid, err := strconv.Atoi(project)
if err != nil && project != "" {
return fmt.Errorf("project id: %w", err)
}
if project == "" {
pid, err = strconv.Atoi(os.Getenv("OO_PROJECT_ID"))
if err != nil {
return fmt.Errorf("--project or OO_PROJECT_ID required")
}
}
req := onlyoffice.NewProjectTaskRequest{
ProjectId: pid,
Title: args[0],
Description: desc,
Priority: p,
MilestoneId: int(milestone),
}
if deadline == "" {
deadline = time.Now().Add(14 * 24 * time.Hour).Format("2006-01-02")
}
d, err := time.Parse("2006-01-02", deadline)
if err != nil {
return fmt.Errorf("deadline: %w", err)
}
req.Deadline = onlyoffice.Time(d)
if start != "" {
s, err := time.Parse("2006-01-02", start)
if err != nil {
return fmt.Errorf("start: %w", err)
}
req.StartDate = onlyoffice.Time(s)
} else {
req.StartDate = onlyoffice.Time(time.Now())
}
task, err := c.CreateProjectTask(req)
if err != nil {
return err
}
printObject(map[string]any{
"id": derefInt(task.ID),
"title": derefString(task.Title),
})
return nil
}
out, err := c.AddTask(cmd.Context(), project, args[0], desc, p, deadline)
if err != nil {
return err
@@ -110,7 +160,9 @@ func taskCreateCmd() *cobra.Command {
}
cmd.Flags().StringVarP(&project, "project", "p", "", "project id (default $OO_PROJECT_ID)")
cmd.Flags().StringVar(&desc, "description", "", "description")
cmd.Flags().StringVar(&deadline, "deadline", "", "deadline YYYY-MM-DD (default now+14d; always assigned to you)")
cmd.Flags().StringVar(&deadline, "deadline", "", "deadline YYYY-MM-DD (default now+14d)")
cmd.Flags().StringVar(&start, "start", "", "start YYYY-MM-DD (uses CreateProjectTask / Gantt)")
cmd.Flags().Int64Var(&milestone, "milestone", 0, "milestone id (Gantt row)")
cmd.Flags().StringVar(&prio, "priority", "normal", "high|normal|low")
return cmd
}
+308
View File
@@ -1,6 +1,12 @@
package main
import (
"bufio"
"fmt"
"os"
"strings"
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/spf13/cobra"
)
@@ -13,6 +19,14 @@ func init() {
rootCmd.AddCommand(usersCmd)
usersCmd.AddCommand(usersListCmd())
usersCmd.AddCommand(usersSelfCmd())
usersCmd.AddCommand(usersGetCmd())
usersCmd.AddCommand(usersCreateCmd())
usersCmd.AddCommand(usersUpdateCmd())
usersCmd.AddCommand(usersDeleteCmd())
usersCmd.AddCommand(usersBlockCmd())
usersCmd.AddCommand(usersUnblockCmd())
usersCmd.AddCommand(usersPasswordCmd())
usersCmd.AddCommand(usersCheckCmd())
rootCmd.AddCommand(whoamiCmd())
}
@@ -65,6 +79,300 @@ func usersSelfCmd() *cobra.Command {
}
}
func usersGetCmd() *cobra.Command {
return &cobra.Command{
Use: "get USER_ID",
Short: "Show one portal user profile",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
u, err := c.GetUser(cmd.Context(), args[0])
if err != nil {
return err
}
printObject(u)
return nil
},
}
}
func usersCreateCmd() *cobra.Command {
var first, last, email, password, title, location, sex, comment string
var visitor bool
cmd := &cobra.Command{
Use: "create",
Short: "Create a portal user",
Long: `Create a portal user (POST /api/2.0/people).
Without --password the portal generates one and the account stays NotActivated
until the user follows the activation link. With --password the account is
Active immediately. Use --visitor for a guest account.`,
RunE: func(cmd *cobra.Command, args []string) error {
if email == "" || first == "" || last == "" {
return fmt.Errorf("--email, --first and --last are required")
}
c, err := newOO(cmd)
if err != nil {
return err
}
req := onlyoffice.NewUserRequest{
FirstName: first,
LastName: last,
Email: email,
Password: password,
Title: title,
Location: location,
Sex: sex,
Comment: comment,
}
if cmd.Flags().Changed("visitor") {
req.IsVisitor = &visitor
}
u, err := c.CreateUser(cmd.Context(), req)
if err != nil {
return err
}
printObject(map[string]any{
"id": idString(u, "id"),
"displayName": idString(u, "displayName"),
"email": idString(u, "email"),
"status": u["status"],
})
return nil
},
}
cmd.Flags().StringVar(&first, "first", "", "first name (required)")
cmd.Flags().StringVar(&last, "last", "", "last name (required)")
cmd.Flags().StringVar(&email, "email", "", "email (required)")
cmd.Flags().StringVar(&password, "password", "", "initial password (default: portal-generated)")
cmd.Flags().StringVar(&title, "title", "", "job title")
cmd.Flags().StringVar(&location, "location", "", "location")
cmd.Flags().StringVar(&sex, "sex", "", "sex: male|female")
cmd.Flags().StringVar(&comment, "comment", "", "comment")
cmd.Flags().BoolVar(&visitor, "visitor", false, "create as guest (isVisitor=true)")
return cmd
}
func usersUpdateCmd() *cobra.Command {
var first, last, email, title, location, sex, comment string
cmd := &cobra.Command{
Use: "update USER_ID",
Short: "Update portal user profile fields (only flags passed)",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
body := map[string]any{}
if cmd.Flags().Changed("first") {
body["firstname"] = first
}
if cmd.Flags().Changed("last") {
body["lastname"] = last
}
if cmd.Flags().Changed("email") {
body["email"] = email
}
if cmd.Flags().Changed("title") {
body["title"] = title
}
if cmd.Flags().Changed("location") {
body["location"] = location
}
if cmd.Flags().Changed("sex") {
body["sex"] = sex
}
if cmd.Flags().Changed("comment") {
body["comment"] = comment
}
if len(body) == 0 {
return fmt.Errorf("nothing to update: pass at least one of --first/--last/--email/--title/--location/--sex/--comment")
}
c, err := newOO(cmd)
if err != nil {
return err
}
u, err := c.UpdateUser(cmd.Context(), args[0], body)
if err != nil {
return err
}
printObject(map[string]any{
"id": idString(u, "id"),
"displayName": idString(u, "displayName"),
})
return nil
},
}
cmd.Flags().StringVar(&first, "first", "", "first name")
cmd.Flags().StringVar(&last, "last", "", "last name")
cmd.Flags().StringVar(&email, "email", "", "email")
cmd.Flags().StringVar(&title, "title", "", "job title")
cmd.Flags().StringVar(&location, "location", "", "location")
cmd.Flags().StringVar(&sex, "sex", "", "sex: male|female")
cmd.Flags().StringVar(&comment, "comment", "", "comment")
return cmd
}
func usersDeleteCmd() *cobra.Command {
return &cobra.Command{
Use: "delete USER_ID [USER_ID...]",
Aliases: []string{"rm"},
Short: "Delete portal user(s) permanently",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
for _, id := range args {
u, err := c.DeleteUser(cmd.Context(), id)
if err != nil {
return fmt.Errorf("delete %s: %w", id, err)
}
printObject(map[string]any{"id": id, "deleted": true, "displayName": idString(u, "displayName")})
}
return nil
},
}
}
func usersBlockCmd() *cobra.Command {
return &cobra.Command{
Use: "block USER_ID [USER_ID...]",
Aliases: []string{"disable"},
Short: "Block (terminate) user(s): login denied, profile kept",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
for _, id := range args {
if err := c.BlockUser(cmd.Context(), id); err != nil {
return fmt.Errorf("block %s: %w", id, err)
}
printObject(map[string]any{"id": id, "blocked": true})
}
return nil
},
}
}
func usersUnblockCmd() *cobra.Command {
return &cobra.Command{
Use: "unblock USER_ID [USER_ID...]",
Aliases: []string{"enable", "activate"},
Short: "Unblock (activate) user(s)",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
for _, id := range args {
if err := c.UnblockUser(cmd.Context(), id); err != nil {
return fmt.Errorf("unblock %s: %w", id, err)
}
printObject(map[string]any{"id": id, "unblocked": true})
}
return nil
},
}
}
func usersPasswordCmd() *cobra.Command {
var password string
cmd := &cobra.Command{
Use: "password USER_ID",
Short: "Set a user password (reads stdin when --password is empty)",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
pwd := password
if pwd == "" {
b, err := readLine(os.Stdin)
if err != nil {
return fmt.Errorf("read password: %w", err)
}
pwd = b
}
if pwd == "" {
return fmt.Errorf("password is empty")
}
c, err := newOO(cmd)
if err != nil {
return err
}
if err := c.ChangeUserPassword(cmd.Context(), args[0], pwd); err != nil {
return err
}
printObject(map[string]any{"id": args[0], "password_changed": true})
return nil
},
}
cmd.Flags().StringVar(&password, "password", "", "new password (omit to read one line from stdin)")
return cmd
}
func usersCheckCmd() *cobra.Command {
var login, password string
cmd := &cobra.Command{
Use: "check",
Short: "Check that a login can authenticate (userName or email)",
Long: `Probes POST /api/2.0/authentication.json with the given credentials and
discards the token.
Where this is used: before handing portal credentials to an external party
(e.g. a guest given read access to a document pack), verify the login actually
works. On some portals the account email is the reliable login identifier — the
userName login fails with 500 for a freshly created user — so share the email,
not the userName.`,
RunE: func(cmd *cobra.Command, args []string) error {
if login == "" {
return fmt.Errorf("--login is required (userName or email)")
}
if password == "" {
if b, err := readLine(os.Stdin); err == nil {
password = b
}
}
c, err := newOO(cmd)
if err != nil {
return err
}
if err := c.AuthenticateAs(cmd.Context(), login, password); err != nil {
printObject(map[string]any{"login": login, "ok": false, "error": trimAuthErr(err)})
return fmt.Errorf("login failed for %s", login)
}
printObject(map[string]any{"login": login, "ok": true})
return nil
},
}
cmd.Flags().StringVar(&login, "login", "", "userName or email")
cmd.Flags().StringVar(&password, "password", "", "password (omit to read one line from stdin)")
return cmd
}
// trimAuthErr keeps the error short for table output.
func trimAuthErr(err error) string {
s := err.Error()
if len(s) > 160 {
s = s[:160] + "…"
}
return s
}
// readLine reads a single trimmed line from r.
func readLine(r *os.File) (string, error) {
sc := bufio.NewScanner(r)
if !sc.Scan() {
if err := sc.Err(); err != nil {
return "", err
}
return "", nil
}
return strings.TrimSpace(sc.Text()), nil
}
// whoamiCmd is a convenience shortcut at the root level.
func whoamiCmd() *cobra.Command {
return &cobra.Command{
+96
View File
@@ -0,0 +1,96 @@
package onlyoffice
import (
"context"
"fmt"
"sort"
"strconv"
"strings"
)
// History entities that OnlyOffice CRM actually accepts for history notes.
// There is NO person/contact history in this API version: POST /api/2.0/crm/history.json
// returns 400 "Value does not fall within the expected range." for entityType
// contact/person/people/client/member. Verified against a live instance (#74).
const (
HistoryEntityOpportunity = "opportunity"
HistoryEntityCase = "case"
)
// IsCompany reports whether a CRM contact row is a company (vs a person).
// The field arrives as JSON bool; be liberal about what we accept.
func IsCompany(person map[string]any) bool {
b, _ := person["isCompany"].(bool)
return b
}
// ContactID returns the CRM id of a contact row as a plain string.
func ContactID(row map[string]any) string {
return fmt.Sprint(row["id"])
}
// BuildContactEmailIndex scans all persons once and maps lowercase email →
// contact id. Use this instead of calling FindPersonByEmail per address:
// the index is O(N) over the whole CRM, the per-address lookup is O(N×M).
func (c *Client) BuildContactEmailIndex(ctx context.Context) (map[string]string, error) {
all, err := c.ListAllContacts(ctx)
if err != nil {
return nil, err
}
index := make(map[string]string, len(all)*2)
for _, person := range all {
if IsCompany(person) {
continue
}
id := ContactID(person)
for _, row := range ContactInfoRows(person) {
if NormalizeContactInfoType(fmt.Sprint(row["infoType"])) != "email" {
continue
}
email := strings.ToLower(strings.TrimSpace(fmt.Sprint(row["data"])))
if email != "" && email != "<nil>" {
index[email] = id
}
}
}
return index, nil
}
// BuildPersonOpportunityIndex maps every opportunity member's contact id to a
// deterministic representative opportunity: the one with the lowest numeric id.
// OnlyOffice has no person-level history, so notes for a person go on their
// deal — this index answers "which deal" in one pass.
func (c *Client) BuildPersonOpportunityIndex(ctx context.Context) (map[string]string, error) {
opps, err := c.ListAllOpportunities(ctx)
if err != nil {
return nil, err
}
index := map[string]string{}
for _, opp := range opps {
oppID := ContactID(opp)
for _, member := range OpportunityMembers(opp) {
pid := ContactID(member)
if cur, ok := index[pid]; !ok || NumericIDLess(oppID, cur) {
index[pid] = oppID
}
}
}
return index, nil
}
// NumericIDLess compares two string ids numerically when possible, falling
// back to lexicographic order so results stay deterministic either way.
func NumericIDLess(a, b string) bool {
na, errA := strconv.Atoi(strings.TrimSpace(a))
nb, errB := strconv.Atoi(strings.TrimSpace(b))
if errA == nil && errB == nil && na != nb {
return na < nb
}
return a < b
}
// SortIDs orders id strings deterministically (numeric first, then lexical).
func SortIDs(ids []string) {
sort.Strings(ids)
sort.SliceStable(ids, func(i, j int) bool { return NumericIDLess(ids[i], ids[j]) })
}
+65
View File
@@ -0,0 +1,65 @@
package onlyoffice
import "testing"
func TestIsCompany(t *testing.T) {
if IsCompany(map[string]any{"isCompany": true}) != true {
t.Fatal("true row not detected")
}
if IsCompany(map[string]any{"isCompany": false}) {
t.Fatal("false row detected as company")
}
if IsCompany(map[string]any{}) {
t.Fatal("missing field detected as company")
}
if IsCompany(nil) {
t.Fatal("nil row detected as company")
}
}
func TestNumericIDLess(t *testing.T) {
cases := []struct {
a, b string
want bool
}{
{"9", "10", true},
{"1747", "1748", true},
{"abc", "abd", true},
{"10", "9", false},
{" 12 ", "13", true},
{"x1", "2", false}, // non-numeric falls back lexical: "x1" > "2"
}
for _, c := range cases {
if got := NumericIDLess(c.a, c.b); got != c.want {
t.Errorf("NumericIDLess(%q,%q)=%v want %v", c.a, c.b, got, c.want)
}
}
}
func TestSortIDs(t *testing.T) {
ids := []string{"20", "3", "100", "1"}
SortIDs(ids)
want := "1 3 20 100"
got := ""
for i, id := range ids {
if i > 0 {
got += " "
}
got += id
}
if got != want {
t.Fatalf("SortIDs=%q want %q", got, want)
}
}
func TestContactID(t *testing.T) {
if ContactID(map[string]any{"id": float64(42)}) != "42" {
t.Fatal("numeric id formatting broken")
}
}
func TestHistoryEntityConstants(t *testing.T) {
if HistoryEntityOpportunity != "opportunity" || HistoryEntityCase != "case" {
t.Fatal("history entity whitelist drifted from live-verified values")
}
}
+170
View File
@@ -0,0 +1,170 @@
package onlyoffice
// Document conversion via the OnlyOffice DocumentServer converter.
//
// The DocumentServer (the same engine behind the portal's "Download as PDF")
// converts any office format. From a portal-reachable host the converter is
// exposed at "<portal>/ds-vpath/converter" (reverse proxy) or directly at
// "http://<docs-server>:8083/converter" (legacy path: /ConvertService.ashx).
//
// Flow: PresignedURI(fileId) → Convert(docsBase, secret, req) → download
// result.FileURL. The JWT is HS256 signed with the DocumentServer's
// services.CoAuthoring.secret (NOT storage.fs.secretString).
import (
"bytes"
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"strings"
)
// PresignedURI returns a short-lived, fetchable download URI for a portal file
// (GET /api/2.0/files/file/{fileId}/presigneduri). The DocumentServer can fetch
// it without the caller's session, so it is the input for Convert.
func (c *Client) PresignedURI(ctx context.Context, fileID string) (string, error) {
if fileID == "" {
return "", fmt.Errorf("file id is required")
}
raw, err := c.getJSON(ctx, fmt.Sprintf("/api/2.0/files/file/%s/presigneduri", url.PathEscape(fileID)))
if err != nil {
return "", err
}
resp, err := responseField(raw, "response")
if err != nil {
return "", err
}
var s string
if err := json.Unmarshal(resp, &s); err == nil && s != "" {
return s, nil
}
// Some builds return an object instead of a bare string.
var o map[string]any
if err := json.Unmarshal(resp, &o); err == nil {
for _, k := range []string{"uri", "url", "Uri", "Url"} {
if v, ok := o[k].(string); ok && v != "" {
return v, nil
}
}
}
return "", fmt.Errorf("presigneduri: unexpected response %s", truncate(string(resp), 200))
}
// ConvertRequest is the DocumentServer converter body.
type ConvertRequest struct {
URL string `json:"url"`
OutputType string `json:"outputtype"`
FileType string `json:"filetype,omitempty"`
Key string `json:"key"`
Title string `json:"title,omitempty"`
}
// ConvertResult is the DocumentServer converter reply.
type ConvertResult struct {
FileURL string `json:"fileUrl"`
FileType string `json:"fileType"`
Percent int `json:"percent"`
EndConvert bool `json:"endConvert"`
Error *int `json:"error,omitempty"`
}
// SignJWT builds an HS256 JWT with the given payload (stdlib only).
func SignJWT(secret string, payload any) (string, error) {
if secret == "" {
return "", fmt.Errorf("jwt secret is empty")
}
hb, err := json.Marshal(map[string]string{"alg": "HS256", "typ": "JWT"})
if err != nil {
return "", err
}
pb, err := json.Marshal(payload)
if err != nil {
return "", err
}
enc := base64.RawURLEncoding.EncodeToString
signing := enc(hb) + "." + enc(pb)
mac := hmac.New(sha256.New, []byte(secret))
mac.Write([]byte(signing))
return signing + "." + enc(mac.Sum(nil)), nil
}
// ConvertDocument asks a DocumentServer to convert req.URL into req.OutputType.
// docsBase is e.g. "https://portal/ds-vpath" or "http://localhost:8083";
// secret is the DocumentServer CoAuthoring JWT secret. Passes the JWT both as
// the AuthorizationJwt header and as a body token.
func (c *Client) ConvertDocument(ctx context.Context, docsBase, secret string, req ConvertRequest) (*ConvertResult, error) {
if strings.TrimSpace(docsBase) == "" {
return nil, fmt.Errorf("docs base url is required")
}
if req.URL == "" {
return nil, fmt.Errorf("source url is required")
}
if req.OutputType == "" {
return nil, fmt.Errorf("outputtype is required")
}
if req.Key == "" {
return nil, fmt.Errorf("conversion key is required")
}
jwt, err := SignJWT(secret, req)
if err != nil {
return nil, err
}
body, err := json.Marshal(req)
if err != nil {
return nil, err
}
endpoint := strings.TrimRight(docsBase, "/") + "/converter"
httpReq, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(body))
if err != nil {
return nil, err
}
httpReq.Header.Set("Content-Type", "application/json")
httpReq.Header.Set("Accept", "application/json")
httpReq.Header.Set("AuthorizationJwt", "Bearer "+jwt)
resp, err := c.client.Do(httpReq)
if err != nil {
return nil, fmt.Errorf("converter request: %w", err)
}
defer resp.Body.Close()
raw, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
if resp.StatusCode >= 400 {
return nil, fmt.Errorf("converter: %d %s", resp.StatusCode, truncate(string(raw), 300))
}
var out ConvertResult
if err := json.Unmarshal(raw, &out); err != nil {
return nil, fmt.Errorf("converter decode: %w (%s)", err, truncate(string(raw), 200))
}
if out.Error != nil {
return &out, fmt.Errorf("converter error %d", *out.Error)
}
if out.FileURL == "" {
return &out, fmt.Errorf("converter returned no fileUrl")
}
return &out, nil
}
// DownloadURLTo streams an absolute URL (no portal auth) into dst.
func (c *Client) DownloadURLTo(ctx context.Context, rawurl string, dst io.Writer) (int64, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, rawurl, nil)
if err != nil {
return 0, err
}
resp, err := c.client.Do(req)
if err != nil {
return 0, err
}
defer resp.Body.Close()
if resp.StatusCode >= 400 {
return 0, fmt.Errorf("download: %d", resp.StatusCode)
}
return io.Copy(dst, resp.Body)
}
+24
View File
@@ -0,0 +1,24 @@
package onlyoffice
import (
"strings"
"testing"
)
func TestSignJWT(t *testing.T) {
payload := map[string]any{"url": "u", "outputtype": "pdf"}
tok, err := SignJWT("secret", payload)
if err != nil {
t.Fatal(err)
}
if n := len(strings.Split(tok, ".")); n != 3 {
t.Fatalf("JWT must have 3 parts, got %d", n)
}
tok2, _ := SignJWT("secret", payload)
if tok != tok2 {
t.Fatal("SignJWT must be deterministic for identical input")
}
if _, err := SignJWT("", payload); err == nil {
t.Fatal("expected error for empty secret")
}
}
+122 -10
View File
@@ -2,7 +2,7 @@ package onlyoffice
// Minimal CRM helpers: contacts, opportunities, cases, tasks, and history notes.
// These expose untyped maps for flexibility — they are primarily consumed by
// cmd/oo and the applications-sync workflow.
// cmd/oo and CRM sync tooling.
import (
"context"
@@ -15,10 +15,15 @@ import (
)
// ListContacts returns a page of CRM contacts and the total count.
// sortBy=id is always set: OnlyOffice filter.json without an explicit sort
// order is non-deterministic on large contact sets, so a paged walk
// (ListAllContacts, ListContactsByTag, FindCompany, FindPerson) can skip or
// duplicate contacts across page boundaries.
func (c *Client) ListContacts(ctx context.Context, count, startIndex int, search string) ([]map[string]any, int, error) {
q := url.Values{}
q.Set("count", strconv.Itoa(count))
q.Set("startIndex", strconv.Itoa(startIndex))
q.Set("sortBy", "id")
if search != "" {
q.Set("filterValue", search)
}
@@ -188,21 +193,41 @@ func (c *Client) CreatePerson(ctx context.Context, first, last string, companyID
}
// UpdatePerson updates first/last name and optional company link on a person.
// companyID == 0 leaves the company association unchanged.
// companyID == 0 leaves the company association unchanged (re-sends current
// company id when present). OnlyOffice ignores companyId/about on form-encoded
// PUT and may unlink the company when companyId is omitted — use JSON body.
func (c *Client) UpdatePerson(ctx context.Context, personID, first, last string, companyID int, jobTitle, about string) (map[string]any, error) {
fields := url.Values{}
fields.Set("firstName", first)
fields.Set("lastName", last)
body := map[string]any{
"firstName": first,
"lastName": last,
}
if companyID != 0 {
fields.Set("companyId", strconv.Itoa(companyID))
body["companyId"] = companyID
} else {
// Preserve existing employer: omitted companyId unlinks on this API.
if cur, err := c.GetContact(ctx, personID); err == nil {
if co, ok := cur["company"].(map[string]any); ok {
if id := flexInt(co["id"]); id != 0 {
body["companyId"] = id
}
}
}
}
if jobTitle != "" {
fields.Set("jobTitle", jobTitle)
body["jobTitle"] = jobTitle
}
if about != "" {
fields.Set("about", about)
body["about"] = about
}
return c.putFormObject(ctx, fmt.Sprintf("/api/2.0/crm/contact/person/%s.json", url.PathEscape(personID)), fields)
out, err := c.putJSONObject(ctx, fmt.Sprintf("/api/2.0/crm/contact/person/%s.json", url.PathEscape(personID)), body)
if err != nil {
return out, err
}
// PUT response body is often stale; re-fetch for authoritative fields.
if fresh, gerr := c.GetContact(ctx, personID); gerr == nil && fresh != nil {
out = fresh
}
return out, nil
}
// AddContactInfo attaches an email/website/phone/etc. to a contact.
@@ -223,6 +248,88 @@ func (c *Client) DeleteContact(ctx context.Context, contactID string) (map[strin
return c.deleteObject(ctx, fmt.Sprintf("/api/2.0/crm/contact/%s.json", url.PathEscape(contactID)))
}
// UpdateContactName renames a CRM company contact displayName.
// Uses the company endpoint (person names go through /crm/contact/person/{id}).
func (c *Client) UpdateContactName(ctx context.Context, contactID, newName string) (map[string]any, error) {
body := map[string]any{
"displayName": newName,
"companyName": newName,
"isCompany": true,
}
out, err := c.putJSONObject(ctx, fmt.Sprintf("/api/2.0/crm/contact/company/%s.json", url.PathEscape(contactID)), body)
if err != nil {
return out, err
}
if fresh, gerr := c.GetContact(ctx, contactID); gerr == nil && fresh != nil {
out = fresh
}
return out, nil
}
// ListContactTags returns all CRM contact tags (title + relativeItemsCount).
func (c *Client) ListContactTags(ctx context.Context) ([]map[string]any, error) {
return c.ResponseArray(ctx, "/api/2.0/crm/contact/tag.json")
}
// CreateContactTag creates a contact tag by name. Idempotent: "already exists" is OK.
func (c *Client) CreateContactTag(ctx context.Context, tagName string) error {
fields := url.Values{}
fields.Set("tagName", tagName)
_, err := c.postFormObject(ctx, "/api/2.0/crm/contact/tag.json", fields)
if err != nil && strings.Contains(err.Error(), "already exists") {
return nil
}
return err
}
// AddContactTag attaches tagName to a contact.
func (c *Client) AddContactTag(ctx context.Context, contactID, tagName string) error {
fields := url.Values{}
fields.Set("tagName", tagName)
_, err := c.postFormObject(ctx, fmt.Sprintf("/api/2.0/crm/contact/%s/tag.json", url.PathEscape(contactID)), fields)
return err
}
// RemoveContactTag removes tagName from a contact.
func (c *Client) RemoveContactTag(ctx context.Context, contactID, tagName string) error {
fields := url.Values{}
fields.Set("tagName", tagName)
raw, err := c.deleteForm(ctx, fmt.Sprintf("/api/2.0/crm/contact/%s/tag.json", url.PathEscape(contactID)), fields)
if err != nil {
return err
}
_, err = unmarshalResponseObject(raw)
return err
}
// ListContactsByTag returns contacts filtered by a single CRM tag name.
func (c *Client) ListContactsByTag(ctx context.Context, tagName string, count, startIndex int) ([]map[string]any, int, error) {
if count <= 0 {
count = 50
}
q := url.Values{}
q.Set("count", strconv.Itoa(count))
q.Set("startIndex", strconv.Itoa(startIndex))
q.Set("tags", tagName)
q.Set("sortBy", "id")
raw, err := c.getJSON(ctx, "/api/2.0/crm/contact/filter.json?"+q.Encode())
if err != nil {
return nil, 0, err
}
var env struct {
Response []map[string]any `json:"response"`
Total int `json:"total"`
}
if err := json.Unmarshal(raw, &env); err != nil {
return nil, 0, err
}
total := env.Total
if total == 0 && len(env.Response) > 0 {
total = len(env.Response)
}
return env.Response, total, nil
}
// ListAllContacts paginates through every CRM contact.
func (c *Client) ListAllContacts(ctx context.Context) ([]map[string]any, error) {
const page = 100
@@ -580,7 +687,7 @@ func (c *Client) CreateCRMTask(ctx context.Context, title, deadline string, cate
deadline = time.Now().Add(14 * 24 * time.Hour).Format("2006-01-02T15:04:05")
}
if categoryID == 0 {
categoryID = 2 // Opportunity — matches applications sync
categoryID = 2 // Opportunity (default CRM task category)
}
uid, err := c.SelfUserID(ctx)
if err != nil {
@@ -634,6 +741,11 @@ func (c *Client) DeleteCRMTask(ctx context.Context, id string) (map[string]any,
return c.deleteObject(ctx, fmt.Sprintf("/api/2.0/crm/task/%s.json", url.PathEscape(id)))
}
// CloseCRMTask closes (completes) a CRM task via the task close endpoint.
func (c *Client) CloseCRMTask(ctx context.Context, id string) (map[string]any, error) {
return c.putFormObject(ctx, fmt.Sprintf("/api/2.0/crm/task/%s/close.json", url.PathEscape(id)), url.Values{})
}
// ListTaskCategories returns CRM task categories.
func (c *Client) ListTaskCategories(ctx context.Context) ([]map[string]any, error) {
return c.ResponseArray(ctx, "/api/2.0/crm/task/category.json")
+91
View File
@@ -0,0 +1,91 @@
package onlyoffice
import (
"fmt"
"strconv"
"strings"
)
// addressCategoryCodes maps ASC.CRM.Core.AddressCategory names to their numeric
// codes. The OO API expects the code, the UI/docs use the label.
var addressCategoryCodes = map[string]int{
"home": 0,
"postal": 1,
"office": 2,
"billing": 3,
"other": 4,
"work": 5,
}
// AddressCategoryCode returns the numeric code for an AddressCategory label
// (Home|Postal|Office|Billing|Other|Work) or a numeric string. Unknown/empty
// labels fall back to Billing, the category `oo companies create` used.
func AddressCategoryCode(category string) int {
s := strings.ToLower(strings.TrimSpace(category))
if s == "" {
return addressCategoryCodes["billing"]
}
if n, err := strconv.Atoi(s); err == nil {
if n >= 0 && n <= 5 {
return n
}
return addressCategoryCodes["billing"]
}
if n, ok := addressCategoryCodes[s]; ok {
return n
}
return addressCategoryCodes["billing"]
}
// ContactAddresses returns the postal address rows of a contact map.
func ContactAddresses(contact map[string]any) []map[string]any {
if rows, ok := contact["addresses"].([]any); ok {
return mapsFromAnySlice(rows)
}
if rows, ok := contact["addresses"].([]map[string]any); ok {
return rows
}
return nil
}
// HasContactAddress reports whether a contact already has the given postal
// address. street+city+zip+category identify it; comparison is normalized.
func HasContactAddress(contact map[string]any, street, city, zip, category string) bool {
wantStreet, wantCity, wantZip := normalizeAddressPart(street), normalizeAddressPart(city), normalizeAddressPart(zip)
wantCat := AddressCategoryCode(category)
for _, row := range ContactAddresses(contact) {
if normalizeAddressPart(fmt.Sprint(row["street"])) != wantStreet {
continue
}
if normalizeAddressPart(fmt.Sprint(row["city"])) != wantCity {
continue
}
if normalizeAddressPart(fmt.Sprint(row["zip"])) != wantZip {
continue
}
if int(anyToFloat(row["category"])) != wantCat {
continue
}
return true
}
return false
}
func normalizeAddressPart(s string) string {
s = strings.ToLower(strings.TrimSpace(s))
return strings.Join(strings.Fields(s), " ")
}
func anyToFloat(v any) float64 {
switch n := v.(type) {
case float64:
return n
case int:
return float64(n)
case string:
f, _ := strconv.ParseFloat(strings.TrimSpace(n), 64)
return f
default:
return 0
}
}
+42
View File
@@ -0,0 +1,42 @@
package onlyoffice
import "testing"
func TestAddressCategoryCode(t *testing.T) {
cases := map[string]int{
"Home": 0, "Postal": 1, "Office": 2, "Billing": 3, "Other": 4, "Work": 5,
"billing": 3, " work ": 5, "3": 3, "5": 5,
"": 3, "nonsense": 3, "99": 3,
}
for in, want := range cases {
if got := AddressCategoryCode(in); got != want {
t.Errorf("AddressCategoryCode(%q) = %d, want %d", in, got, want)
}
}
}
func TestHasContactAddress(t *testing.T) {
contact := map[string]any{
"addresses": []any{
map[string]any{
"street": "Lubanas st. 125a-25", "city": "Riga",
"zip": "LV-1021", "country": "Latvia", "category": float64(3),
},
},
}
if !HasContactAddress(contact, " Lubanas St. 125a-25 ", "riga", "lv-1021", "Billing") {
t.Error("want match (normalized, case-insensitive)")
}
if HasContactAddress(contact, "Lubanas st. 125a-25", "Riga", "LV-1021", "Work") {
t.Error("different category must not match")
}
if HasContactAddress(contact, "Lubanas st. 125a-25", "Riga", "00000", "Billing") {
t.Error("different zip must not match")
}
if HasContactAddress(map[string]any{}, "x", "y", "z", "Billing") {
t.Error("empty contact must not match")
}
if got := ContactAddresses(contact); len(got) != 1 {
t.Fatalf("ContactAddresses = %d rows", len(got))
}
}
+126
View File
@@ -0,0 +1,126 @@
package onlyoffice
import (
"context"
"fmt"
"strconv"
"strings"
)
// OpportunityAudit is one CRM opportunity with its resource counts and a coarse
// class, for hygiene reporting (see `oo crm audit`).
type OpportunityAudit struct {
ID int64 `json:"id"`
Title string `json:"title"`
Created string `json:"created,omitempty"`
Files int `json:"files"`
OpenTasks int `json:"open_tasks"`
ClosedTasks int `json:"closed_tasks"`
Members int `json:"members"`
GroupKey string `json:"group_key,omitempty"`
Class string `json:"class"` // ok | dup | empty | junk-title
}
// AuditOpportunities lists every opportunity with file/task/member counts and a
// coarse class. The classification is generic and rule-free:
//
// dup — another opportunity shares the same title key
// empty — no files, tasks or members
// junk-title — title is not of the "Role @ Company" shape
// ok — everything else
//
// Callers that need stricter business rules can post-process the result.
func (c *Client) AuditOpportunities(ctx context.Context) ([]OpportunityAudit, error) {
deals, err := c.ListAllOpportunities(ctx)
if err != nil {
return nil, err
}
tasks, _, err := c.ListCRMTasks(ctx, 5000, 0)
if err != nil {
return nil, err
}
open, closed := taskCountsByOpportunity(tasks)
out := make([]OpportunityAudit, 0, len(deals))
for _, row := range deals {
id := auditID(row["id"])
if id == 0 {
continue
}
title := auditStr(row["title"])
files := 0
if fl, ferr := c.ListOpportunityFiles(ctx, strconv.FormatInt(id, 10)); ferr == nil {
files = len(fl)
}
key := strconv.FormatInt(id, 10)
out = append(out, OpportunityAudit{
ID: id,
Title: title,
Created: auditStr(row["created"]),
Files: files,
OpenTasks: open[key],
ClosedTasks: closed[key],
Members: len(OpportunityMembers(row)),
GroupKey: DealTitleKey(title, false),
})
}
groupCount := map[string]int{}
for _, a := range out {
groupCount[a.GroupKey]++
}
for i := range out {
a := &out[i]
switch {
case groupCount[a.GroupKey] > 1:
a.Class = "dup"
case a.Files == 0 && a.OpenTasks == 0 && a.ClosedTasks == 0 && a.Members == 0:
a.Class = "empty"
case !strings.Contains(a.Title, "@") || strings.HasPrefix(strings.TrimSpace(a.Title), "@"):
a.Class = "junk-title"
default:
a.Class = "ok"
}
}
return out, nil
}
// taskCountsByOpportunity buckets CRM task statuses per opportunity id.
func taskCountsByOpportunity(tasks []map[string]any) (open, closed map[string]int) {
open, closed = map[string]int{}, map[string]int{}
for _, t := range tasks {
ent, ok := t["entity"].(map[string]any)
if !ok || auditStr(ent["entityType"]) != "opportunity" {
continue
}
eid := auditStr(ent["entityId"])
status := strings.ToLower(auditStr(t["status"]))
if status == "2" || status == "closed" {
closed[eid]++
} else {
open[eid]++
}
}
return open, closed
}
func auditID(v any) int64 {
switch x := v.(type) {
case float64:
return int64(x)
case int:
return int64(x)
case int64:
return x
default:
n, _ := strconv.ParseInt(strings.TrimSpace(fmt.Sprint(x)), 10, 64)
return n
}
}
func auditStr(v any) string {
if v == nil {
return ""
}
return fmt.Sprint(v)
}
+36
View File
@@ -0,0 +1,36 @@
package onlyoffice
import "testing"
func TestAuditID(t *testing.T) {
cases := []struct {
in any
want int64
}{
{float64(12), 12},
{7, 7},
{int64(9), 9},
{"42", 42},
{nil, 0},
}
for _, c := range cases {
if got := auditID(c.in); got != c.want {
t.Fatalf("auditID(%v) = %d, want %d", c.in, got, c.want)
}
}
}
func TestTaskCountsByOpportunity(t *testing.T) {
open, closed := taskCountsByOpportunity([]map[string]any{
{"id": 1, "status": 1, "entity": map[string]any{"entityType": "opportunity", "entityId": 10}},
{"id": 2, "status": "2", "entity": map[string]any{"entityType": "opportunity", "entityId": 10}},
{"id": 3, "status": 1, "entity": map[string]any{"entityType": "contact", "entityId": 10}},
{"id": 4, "entity": "not-a-map"},
})
if open["10"] != 1 || closed["10"] != 1 {
t.Fatalf("open=%v closed=%v", open, closed)
}
if len(open) != 1 || len(closed) != 1 {
t.Fatalf("unexpected buckets: open=%v closed=%v", open, closed)
}
}
+172
View File
@@ -0,0 +1,172 @@
package onlyoffice
import (
"regexp"
"strings"
"unicode"
)
var (
parenSuffixRE = regexp.MustCompile(`(?i)\s*[\(\[\{][^)\]\}]*[\)\]\}]\s*$`)
dashCompanyRE = regexp.MustCompile(`(?i)\s+[-–—]\s+[A-Za-z0-9].*$`)
emailLocalRE = regexp.MustCompile(`(?i)^[a-z0-9._%+\-]+@[a-z0-9.\-]+\.[a-z]{2,}$`)
nonNameTokenRE = regexp.MustCompile(`[^a-zA-ZÀ-öø-ÿĀ-ž0-9'’.\-]+`)
)
// CleanPersonNames strips company annotations from display names and fills
// first/last from email local-part when the source used an address as the name.
// Company affiliation belongs on CRM companyId — never in LastName.
func CleanPersonNames(first, last, display, org string, emails []string) (cleanFirst, cleanLast string) {
first = strings.TrimSpace(first)
last = strings.TrimSpace(last)
display = strings.TrimSpace(display)
org = strings.TrimSpace(org)
if looksLikeEmail(first) {
ef, el := GuessNameFromEmail(first)
first, last = ef, el
}
if looksLikeEmail(display) && first == "" && last == "" {
display = ""
}
if first == "" && last == "" && display != "" {
first, last = SplitDisplayName(display)
}
first = stripCompanyAnnotation(first, org)
last = stripCompanyAnnotation(last, org)
last = stripCompanyAnnotation(last, org)
if i := strings.IndexAny(first, "(["); i > 0 {
first = strings.TrimSpace(first[:i])
}
if org != "" && personLastIsOrg(last, org) {
last = ""
}
if (first == "" || looksLikeEmail(first)) && len(emails) > 0 {
ef, el := GuessNameFromEmail(emails[0])
if first == "" || looksLikeEmail(first) {
first = ef
}
if last == "" || last == "-" {
last = el
}
}
first = strings.TrimSpace(first)
last = strings.TrimSpace(last)
if last == "" {
last = "-"
}
return first, last
}
func stripCompanyAnnotation(s, org string) string {
s = strings.TrimSpace(s)
if s == "" {
return ""
}
s = parenSuffixRE.ReplaceAllString(s, "")
s = strings.TrimSpace(s)
s = dashCompanyRE.ReplaceAllString(s, "")
s = strings.TrimSpace(s)
if org != "" {
for _, sep := range []string{" - ", " – ", " — ", " / "} {
if i := strings.LastIndex(strings.ToLower(s), strings.ToLower(sep+org)); i >= 0 {
s = strings.TrimSpace(s[:i])
}
}
suf := " (" + org + ")"
if strings.HasSuffix(strings.ToLower(s), strings.ToLower(suf)) {
s = strings.TrimSpace(s[:len(s)-len(suf)])
}
}
return strings.TrimSpace(s)
}
func personLastIsOrg(last, org string) bool {
last = NormalizePersonNameKey(last)
org = NormalizePersonNameKey(org)
if last == "" || org == "" {
return false
}
if last == org {
return true
}
return strings.HasPrefix(org, last+" ") || strings.HasPrefix(org, last+",")
}
func looksLikeEmail(s string) bool {
return emailLocalRE.MatchString(strings.TrimSpace(s))
}
// GuessNameFromEmail turns local@domain into Title-Case first/last when the
// local part looks like first.last / first_last / first-last.
func GuessNameFromEmail(email string) (first, last string) {
email = NormalizeContactEmail(email)
local, _, ok := strings.Cut(email, "@")
if !ok || local == "" {
return "", ""
}
local = strings.Split(local, "+")[0]
parts := strings.FieldsFunc(local, func(r rune) bool {
return r == '.' || r == '_' || r == '-'
})
if len(parts) == 0 {
return titleNameToken(local), ""
}
if len(parts) == 1 {
return titleNameToken(parts[0]), ""
}
return titleNameToken(parts[0]), titleNameToken(strings.Join(parts[1:], " "))
}
func titleNameToken(s string) string {
s = nonNameTokenRE.ReplaceAllString(s, " ")
s = strings.TrimSpace(s)
if s == "" {
return ""
}
runes := []rune(strings.ToLower(s))
runes[0] = unicode.ToTitle(runes[0])
return string(runes)
}
// FormatProjectTitle builds "CC | Company | Title" (spaces around |).
// Country should be a short code (DE, TF, UA, …). Empty segments are dropped.
func FormatProjectTitle(country, company, title string) string {
parts := make([]string, 0, 3)
for _, p := range []string{country, company, title} {
p = strings.TrimSpace(p)
p = strings.ReplaceAll(p, "|", "/")
if p != "" {
parts = append(parts, p)
}
}
return strings.Join(parts, " | ")
}
// NormalizeContactEmail lowercases and trims.
func NormalizeContactEmail(s string) string {
return strings.ToLower(strings.TrimSpace(s))
}
// NormalizePersonNameKey collapses whitespace and lowercases for matching.
func NormalizePersonNameKey(s string) string {
fields := strings.Fields(strings.ToLower(strings.TrimSpace(s)))
return strings.Join(fields, " ")
}
// SplitDisplayName splits "First Last …" into first/last (last = remainder).
func SplitDisplayName(name string) (first, last string) {
parts := strings.Fields(strings.TrimSpace(name))
if len(parts) == 0 {
return "", ""
}
if len(parts) == 1 {
return parts[0], ""
}
return parts[0], strings.Join(parts[1:], " ")
}
+39
View File
@@ -0,0 +1,39 @@
package onlyoffice
import "testing"
func TestCleanPersonNames_stripsCompanyParen(t *testing.T) {
f, l := CleanPersonNames("Jane", "Doe (Acme)", "Jane Doe (Acme)", "Acme", nil)
if f != "Jane" || l != "Doe" {
t.Fatalf("got %q %q", f, l)
}
}
func TestCleanPersonNames_stripsDashCompany(t *testing.T) {
f, l := CleanPersonNames("Jane", "Doe - Acme", "", "Acme", nil)
if f != "Jane" || l != "Doe" {
t.Fatalf("got %q %q", f, l)
}
}
func TestCleanPersonNames_fromEmail(t *testing.T) {
f, l := CleanPersonNames("jane.doe@example.com", "-", "", "Acme",
[]string{"jane.doe@example.com"})
if f != "Jane" || l != "Doe" {
t.Fatalf("got %q %q", f, l)
}
}
func TestCleanPersonNames_lastIsCompany(t *testing.T) {
f, l := CleanPersonNames("Jane", "Acme", "", "Acme GmbH & Co. KG", nil)
if f != "Jane" || l != "-" {
t.Fatalf("got %q %q", f, l)
}
}
func TestFormatProjectTitle(t *testing.T) {
got := FormatProjectTitle("DE", "Acme", "Map App")
if got != "DE | Acme | Map App" {
t.Fatalf("got %q", got)
}
}
+1 -1
View File
@@ -69,7 +69,7 @@ func StripCompanySuffix(title string) string {
}
// FixDealTitle strips a leading @, normalizes separator spacing, and collapses
// empty-position titles like " @ contoso" to "contoso".
// empty-position titles like " @ Acme" to "Acme".
func FixDealTitle(s string) string {
s = strings.TrimSpace(s)
for strings.HasPrefix(s, "@") {
+27
View File
@@ -0,0 +1,27 @@
---
type: reference
status: current
related:
- README.md
---
# go-onlyoffice — docs
Индекс справочников. Общее — [README.md](../README.md), правила — [AGENTS.md](../AGENTS.md).
## Файлы
- [unified-file-client.md](unified-file-client.md) — единый файловый клиент:
`Entry`/`FileStore`/`FileClient`, бэкенды REST/DAV/SQL/ES, env, как добавить
бэкенд.
- [community-server-db.md](community-server-db.md) — read-only SQL-бэкенд
(MySQL/PostgreSQL): схема, SSH-туннель, DSN, MinIO download.
- [elasticsearch.md](elasticsearch.md) — поиск: индекс OnlyOffice `files_file`
и свой `oo_docs_text` (PDF/сканы), туннель.
- [index-and-search.md](index-and-search.md) — карта контуров поиска и как
обновлять индексы (`oo index`, `oo search`).
- [rate-limiting.md](rate-limiting.md) — rate limit, exponential backoff,
`Retry-After`, общий cooldown против 429; env `OO_RATE_LIMIT`/`OO_BURST`/
`OO_RETRY_*`.
## Тесты
Команды и туннели — раздел Testing в [README.md](../README.md#testing).
+179
View File
@@ -0,0 +1,179 @@
---
type: reference
status: current
related:
- README.md
- filestore_pg.go
- docs/elasticsearch.md
---
# Community Server DB — прямой SQL-доступ (read-only)
## Что это
Бэкенд `pgStore` (`filestore_pg.go`) читает файлы и папки **напрямую из БД
Community Server**, без HTTP-слоя. Реализует `FileStore` (`List`/`Stat`/
`Download`) и `Searcher` по имени. Запись запрещена: все write-методы
возвращают `ErrReadOnly`.
## Что за БД (research, live)
Проверено на VM `onlyoffice-v2` (SSH `127.0.0.1:32`):
- Community Server работает на **MySQL 8.0**, не на PostgreSQL.
- Хост: `127.0.0.1:3306` внутри VM, база `onlyoffice`.
- Конфиг: `/etc/onlyoffice/communityserver/appsettings.production.json`,
`providerName: MySql.Data.MySqlClient`.
- Таблицы: `files_file`, `files_folder`, `files_folder_tree`,
`files_security`, тенанты — `tenants_tenants` (не `tenants`).
- PostgreSQL 16 в той же VM — **наш** контур (`edw_docs`, роли `edw`/`edw_ro`,
office-assistant), к OnlyOffice отношения не имеет. `files_file` в PG нет.
- Портал хранит файлы в **S3/MinIO** (DiscStorage только для мелочи).
Бакет `office`, объект — по ключу (см. ниже).
Вывод: бэкенд назван по issue «PostgreSQL», но живой источник — MySQL.
`database/sql` + драйвер по DSN: `mysql` для MySQL, `pgx` для PostgreSQL.
`Name()` возвращает фактический движок (`mysql` или `postgres`).
## Схема
`files_file` — одна строка **на версию** (PK `tenant_id, id, version`):
| поле | смысл |
|------|-------|
| `id` | id файла (тот же, что в REST/ES) |
| `version` | номер версии этой строки |
| `version_group` | номер версии |
| `current_version` | `1` = текущая версия, `0` = старая |
| `folder_id` | id родительской папки |
| `title` | имя файла с расширением |
| `content_length` | размер в байтах |
| `create_on`, `modified_on` | даты (UTC, без зоны) |
| `tenant_id` | тенант (портал) |
`files_folder`: `id`, `parent_id`, `title`, `create_on`, `modified_on`,
`tenant_id`. `files_folder_tree`: `folder_id`, `parent_id`, `level` — готовое
дерево, пока не используется.
Текущую строку файла берём по `current_version = 1`.
## Доступ (SSH-туннель)
MySQL слушает только `127.0.0.1:3306` внутри VM. Снаружи — SSH-туннель
(SSH в VM открыт как `127.0.0.1:32`):
```bash
ssh -f -N -o ControlMaster=no -o ControlPath=none \
-p 32 -i ~/.ssh/id_ed25519 \
-L 3306:127.0.0.1:3306 root@127.0.0.1
# MySQL DSN затем:
# root:<pw>@tcp(127.0.0.1:3306)/onlyoffice?parseTime=true
```
Любой свободный локальный порт подойдёт (напр. `13306`); тогда тот же порт —
в DSN. `-o ControlMaster=no -o ControlPath=none` обязательны: иначе forward
уходит в persistent master из `~/.ssh/config`.
Креды MySQL — в конфиге Community Server внутри VM:
`/etc/onlyoffice/communityserver/appsettings.production.json` →
`ConnectionStrings.connectionString` (поля `User ID`, `Password`), база
`onlyoffice`. В самом MySQL-контейнере (`onlyoffice-mysql-server`) база пустая;
рабочий сервер — host-mysqld на `127.0.0.1:3306` (207 таблиц). Не печатать
пароль.
## Переменные
| env | default | смысл |
|-----|---------|-------|
| `ONLYOFFICE_DSN` | — | DSN драйвера (MySQL `...@tcp(...)/...` или `postgres://...`) |
| `ONLYOFFICE_PG_DRIVER` | авто | `postgres` или `mysql`; иначе по форме DSN |
| `ONLYOFFICE_PG_TENANT` | `ONLYOFFICE_TENANT` | фильтр `tenant_id` (пусто = все) |
| `ONLYOFFICE_PG_HOST/PORT/USER/PASSWORD/DBNAME/SSLMODE` | — | собрать PG DSN, если `ONLYOFFICE_DSN` пуст |
Имена — в [`.env.example`](../.env.example). Секретов нет.
## Использование
Напрямую: `NewPGStore(PGConfigFromEnv())`.
Через фасад (эпик #34): SQL-стор регистрируется на `FileClient`. После этого
`Read()` и все чтения (`Stat`/`List`) идут в БД, `Write()` остаётся REST/DAV.
```go
c := onlyoffice.NewClient(onlyoffice.GetEnvironmentCredentials())
sql, err := c.SQLFileStore() // открыть из env; caller закрывает
if err != nil { /* нет DSN / нет связи */ }
if closer, ok := sql.(interface{ Close() error }); ok { defer closer.Close() }
f := c.Files()
f.RegisterStore(onlyoffice.ProviderPG, sql)
e, _ := f.Stat(ctx, "19423") // e.Provider == "mysql" — ответил SQL
```
`Client.FileStore("pg"|"sql"|"postgres"|"mysql")` тоже отдаёт SQL-стор
(открывает из env). Если DSN нет/битый — возвращается не `nil`, а заглушка,
чей метод отдаёт ошибку открытия; ошибку как таковую даёт `SQLFileStore()`.
Различить бэкенд в ответе можно по `Entry.Provider` (`mysql` у SQL, `rest` у
REST).
## Download (MinIO)
`Download` не ходит в REST. Ключ объекта собирается из строки `files_file`:
```
00/00/<tenant>/files/folder_<shard>/file_<id>/v<version>/content.<ext>
shard = (id/1000 + 1) * 1000
```
`shard` — не `folder_id`, а следующая тысяча над `id` (файл 3727 →
`folder_4000`). Проверено live по бакету `office`.
Стриминг переиспользует `downloadMinioObject` из `storage_fallback.go`
(та же подпись SigV4 и `MINIO_*`), без дублирования.
Ограничение: схема валидна только для файлов, лежащих в **MinIO/S3** (старые
папки). Файлы в **Disc**-хранилище портала (`Data/Products/Files/...`, новые
папки) по этому ключу недоступны — `Download` вернёт `404`. Если
`MINIO_ACCESS_KEY`/`MINIO_SECRET_KEY` не заданы, `Download` вернёт явную
ошибку; `Stat`/`List`/`Search` работают и без них.
## Тесты
```bash
go test ./... # unit: rebind, csObjectKey, маппинг
go test -race ./...
# integration (нужен DSN; skip без него)
ONLYOFFICE_DSN='root:<pw>@tcp(127.0.0.1:3306)/onlyoffice?parseTime=true' \
ONLYOFFICE_PG_TENANT=1 \
ONLYOFFICE_PG_TEST_FILE_ID=19423 \
ONLYOFFICE_PG_TEST_FOLDER_ID=676 \
go test -tags=integration -run 'TestIntegrationPGStore|TestIntegrationSQLFacade' -v ./...
# плюс MINIO_* для сверки Download с REST (иначе этот шаг skip)
MINIO_ENDPOINT=http://127.0.0.1:9000 MINIO_BUCKET=office \
MINIO_ACCESS_KEY=... MINIO_SECRET_KEY=... \
go test -tags=integration -run TestIntegrationPGStore -v ./...
```
- `TestIntegrationPGStore` — `Stat`/`List`/`Download` SQL против REST и
`ErrReadOnly` у write-методов.
- `TestIntegrationSQLFacade` — SQL-стор, зарегистрированный на фасаде, реально
обслуживает чтения: `Read().Name()` = SQL-бэкенд, `Entry.Provider == "mysql"`
(у REST — `"rest"`), сверка `Stat`/`List` с REST, и прямой
`Client.FileStore("pg")`.
Без `ONLYOFFICE_DSN` оба теста делают чистый `skip`.
## Грабли
- MySQL хранит `datetime` без зоны; `parseTime=true` (ставится автоматически)
читает их как UTC. REST отдаёт `+02:00` — сравнивать моменты, не строки.
- `GetFile` (REST) не отдаёт `contentLength` — размер сверять с `Stat` SQL.
- Один файл = много строк `files_file` (по версиям). Без `current_version = 1`
получите дубликаты.
- `folder_id` не входит в ключ MinIO; ключ считает `shard` от `id`.
- Searcher SQL ищет только по имени (`LIKE`). Контент — Elasticsearch
([elasticsearch.md](elasticsearch.md)).
+260
View File
@@ -0,0 +1,260 @@
---
type: reference
status: current
related:
- README.md
- filestore_es.go
---
# Elasticsearch — полнотекстовый поиск OnlyOffice
## Что это
Полнотекстовый поиск OnlyOffice Workspace работает на **Elasticsearch**.
Клиент на сервере — NEST. Индекс — имя таблицы.
Для файлов индекс `files_file`:
| поле | тип | смысл |
|------|-----|-------|
| `id` | integer | id файла (тот же, что в REST/Documents) |
| `title` | text (`whitespacecustom`) | имя файла |
| `tenantId` | integer | тенант (портал) |
| `folders` | nested | список папок: `folderId` (строка), `id`, `tenantId` |
| `document.attachment.content` | text (`document`) | извлеченный текст (ingest-attachment) |
| `document.attachment.content_type` | text | MIME |
Важно:
- Живой сервер — **Elasticsearch 7.16.3**, кластер `elasticsearch`.
- REST `GET /api/2.0/files/@search/{query}` ищет **только по имени в БД**
(`fileDao.Search`), ES не задействует. Для поиска по содержимому нужен
прямой ES — это и делает `oo search`.
- `title` analyzer `whitespacecustom` режет по пробелам и lower-case. Полное
имя файла — один токен (`Rechnung-4711.pdf`), поэтому поиск по имени ищет
слово целиком, а не подстроку.
- `document.attachment.content` заполняется **только для Office-форматов**
(docx / xlsx / pptx). У PDF/txt, залитых через API, контент не извлекается.
- Индексация асинхронная (TeamLabSvc) — файл появляется в ES не мгновенно.
## Доступ
ES слушает `127.0.0.1:9200` **внутри** VM OnlyOffice. Снаружи порт закрыт,
SSH в VM открыт на хосте как `127.0.0.1:32` (контейнер `onlyoffice-v2`,
QEMU). Схема — SSH-туннель.
```bash
# из корня go-onlyoffice (ключ и хост — как в infra-доках)
ssh -f -N -o ControlMaster=no -o ControlPath=none \
-p 32 -i ~/.ssh/id_ed25519 \
-L 9200:127.0.0.1:9200 root@127.0.0.1
curl -s http://127.0.0.1:9200/ | head # tagline + version
curl -s 'http://127.0.0.1:9200/_cat/indices?h=index,docs.count'
```
`-o ControlMaster=no -o ControlPath=none` обязательны: иначе forward уходит
в persistent master-соединение из `~/.ssh/config` и порт остаётся занят.
Проверить, что туннель жив:
```bash
curl -s http://127.0.0.1:9200/files_file/_count
```
## Переменные
| env | default | смысл |
|-----|---------|-------|
| `ONLYOFFICE_ES_URL` | — (обязателен) | `scheme://host:port` ES |
| `ONLYOFFICE_ES_INDEX` | `files_file` | индекс |
| `ONLYOFFICE_TENANT` | пусто (все) | фильтр `tenantId` |
Имена — в [`.env.example`](../.env.example). Секретов нет: ES без пароля.
## CLI
```bash
ONLYOFFICE_ES_URL=http://127.0.0.1:9200 oo search "Rechnung"
ONLYOFFICE_ES_URL=http://127.0.0.1:9200 oo search "Mahngebühr" --content
oo search "Rechnung" --folder 649 --limit 50 --json
```
Флаги: `--content` (искать и по тексту), `--folder ID` (папка
`folders.folderId`), `--limit N` (по умолчанию 20, максимум 200),
`--json` = `-o json`.
## Обновление индекса и карта поиска
Обзор всех контуров поиска и как обновлять индексы (`oo index`) —
[index-and-search.md](index-and-search.md).
## Библиотека
`filestore_es.go` — `ESSearcher` (`Name() = "elasticsearch"`), прямой ES REST на
stdlib `net/http`:
```go
es, _ := onlyoffice.NewESSearcher(onlyoffice.ESConfigFromEnv())
hits, _ := es.Search(ctx, onlyoffice.SearchQuery{
Text: "Rechnung", InContent: true, Limit: 20,
})
```
Запрос: `multi_match` по `title^2` (+ `document.attachment.content` при
`InContent`), фильтры `tenantId` и `folders.folderId`, `_source`
id/title/folders, `highlight` для фрагмента. Ответ → `[]SearchHit` (модель из
эпика #34; пока объявлена в `filestore_es.go`, переедет в `filestore_core.go` с F1 #35).
## Тесты
```bash
# unit — чистые builders/парсеры, без сети
go test ./ -run ES
# integration — нужен ONLYOFFICE_ES_URL (+ креды REST для залива)
set -a; . .env; set +a
ONLYOFFICE_ES_URL=http://127.0.0.1:9200 ONLYOFFICE_TENANT=1 \
go test -tags=integration -run TestIntegrationESSearch -v .
```
Интеграционный тест заливает временный xlsx (в имени и в ячейке — уникальные
токены), ждёт индексации, проверяет поиск по имени и по содержимому, затем
удаляет проект.
## Грабли
- `locale`/версия ES: 7.16.3, `_search` совместим с REST 7.x.
- ES без auth и слушает только localhost — туннель обязателен.
- Фильтр `tenantId` сузит выдачу; без него видны документы всех тенантов.
- Поиск по содержимому PDF в индексе OnlyOffice не работает (для PDF нет
`attachment.content`) — только Office-форматы. Решение для PDF — свой индекс
`oo_docs_text` (F6 #42), см. ниже.
# PDF и сканы — свой индекс (F6 #42)
## Проблема
`oo search --content "S1019"` не находил номер внутри PDF-счёта: в индексе
OnlyOffice PDF лежит только по имени.
## Почему PDF исключён (исходники CommunityServer)
Разобрано в `ONLYOFFICE/CommunityServer`:
- `web/core/ASC.Web.Core/Files/FileUtility.cs` — `CanIndex(fileName)` читает
серверную настройку `files.index.formats` (в `web/studio/ASC.Web.Studio/web.appsettings.config`
значение по умолчанию `".pptx|.xlsx|.docx"`).
- `web/studio/ASC.Web.Studio/Products/Files/Core/Search/FilesWrapper.cs` —
`GetDocumentStream*` возвращает `null`, если `!FileUtility.CanIndex(Title)`,
файл зашифрован или больше `MaxFileSize`.
- `module/ASC.ElasticSearch/Core/WrapperWithDoc.cs` + mapping в `Wrapper.cs` —
маппинг `document.attachment.content` и ingest-pipeline `attachments`
формат-агностичны: они распарсят любой поток.
Вывод: PDF исключён **только настройкой** `files.index.formats`; жёсткого
ограничения на формат в коде нет.
## Варианты и решение
| # | Вариант | Оценка |
|---|---------|--------|
| a | Включить `.pdf` в `files.index.formats` + reindex | Правка сервера OO; настройка может потеряться при обновлении; полный reindex 39k док-в; Tika **не OCR** — сканы без текстового слоя дадут пустой контент. Отклонён без решения PO. |
| b | Server-side ingest/attachment для PDF | По факту то же, что (a): сервер кормит поток только для `CanIndex`. |
| c | **Свой индекс** `oo_docs_text`, наполняемый `internal/docpipe` | **Выбран.** Сервер OO не трогаем; детерминированно; работает OCR для сканов; независимо от обновлений OO; любые форматы; фильтры папка/тип. |
| d | Локальный поиск без индекса | Отклонён как основной: качаем и извлекаем на каждый запрос, нет выдачи/ранжирования/highlight. |
Итог: **вариант c**. Индекс OnlyOffice (`files_file`) не изменяется; наш
индекс живёт рядом.
## Устройство
- `filestore_es_text.go` — `ESTextIndex` (`Name() = "es-text"`):
`Ensure` (создаёт индекс с явным маппингом), `Put` (bulk, `refresh`),
`Delete` (по `id`), `Search` (`multi_match` по `title^2` + `content`,
фильтры `folder`/`ext`, highlight).
- `filestore_text_index.go` — `TextIndexer`: листает папки (`FileStore.List`),
качает файлы (`FileStore.Download`), извлекает текст через
`internal/docpipe` (`pdftotext`, для сканов — `ocrmypdf`/`tesseract`),
пишет в `TextIndex`. Пул воркеров (по умолчанию 3).
- CLI: `oo index folder|files` наполняет индекс; `oo search --backend own`
ищет по нему.
### Встроенные вложения PDF
Оцифрованные PDF несут вложения (`<doc>.md` — текст/таблицы скана,
`<doc>.yaml`/`.json` — метаданные, `.xml` — EN 16931 CII eRechnung,
`factur-x.xml` у ZUGFeRD; см. `office-assistant/docs/reference/document-metadata.md`).
`TextIndexer` обходит их: `pdfdetach -list` перечисляет, `-save` сохраняет,
каждое вложение проходит штатный `docpipe.ToMarkdown` (PDF/картинки → OCR,
`.md`/`.txt` — как есть). Форматы, которые docpipe не конвертирует
(`.xml`/`.html` — снимаются теги; `.json`/`.csv` — как текст), извлекаются
текстом; нечитаемые — пропускаются.
Текст склеивается: тело, затем по секции на вложение с маркером
`[attachment: <имя>]` (функция `docpipe.JoinWithAttachments`). Индекс — тот же
`file_id`, upsert идемпотентен. Нет вложений или pdfdetach/формат нечитаем —
индексируется тело (без падения).
Поля `oo_docs_text`:
| поле | тип | смысл |
|------|-----|-------|
| `id` | keyword | id файла Documents |
| `title` | text (+`.keyword`) | имя файла |
| `folder` | keyword | id папки |
| `ext` | keyword | расширение |
| `content` | text | извлечённый текст (pdftotext/OCR) |
## CLI
```bash
set -a; . .env; set +a # ONLYOFFICE_URL/USER/PASS + ONLYOFFICE_ES_URL
oo index folder 634 # PDF в папке 634
oo index folder 634 --recursive --exts pdf,png --limit 100
oo index files 3576 3578 # точечно
oo index folder 634 --dry-run # показать план, ничего не менять
oo search "S1021" --content --backend own
oo search "S1021" --backend own --folder 634 --json
```
`--backend` у `oo search`: `oo` (по умолчанию, индекс OnlyOffice) или `own`
(наш `ONLYOFFICE_ES_TEXT_INDEX`).
## Переменные (дополнение)
| env | default | смысл |
|-----|---------|-------|
| `ONLYOFFICE_ES_TEXT_INDEX` | `oo_docs_text` | индекс своего конвейера |
`ONLYOFFICE_ES_URL` — общий для обоих индексов.
## Тесты
```bash
go test -run 'ESText|TextIndexer|Index' ./ ./cmd/oo/ # unit, без сети
ONLYOFFICE_ES_URL=http://127.0.0.1:9200 \
go test -tags=integration -run TestIntegrationESTextIndex -v .
```
Интеграционный тест создаёт временный индекс, наполняет, ищет по контенту,
проверяет фильтры и удаление, затем удаляет индекс;
`TestIntegrationESTextIndexPDFAttachment` индексирует
`testdata/pdf-with-attachment.pdf` реальным конвейером (pdfdetach + pdftotext)
и ищет токен, лежащий только во вложении. Unit-тесты используют
fake-store/fake-extractor и не требуют pdftotext/OCR (парсер списка, склейка
`JoinWithAttachments`, снятие тегов `xmlToText` — чистые).
## Грабли
- Наполнение — ручное (`oo index`); после изменения/добавления PDF повтори.
Повтор идемпотентен (upsert по id файла).
- В индексе ищется только то, что проиндексировано; `oo index` качает каждый
файл и (для сканов) гоняет OCR — это медленно, отсюда `--limit`/`--exts`.
- `folder` фильтруется как id папки, а не как путь.
- Дубликаты (напр. `S1055.pdf` и `2026-08-20-S1055-…`) дадут несколько строк —
это ожидаемо, дедуп — на стороне потребителя.
- Вложения: нужен `pdfdetach` (poppler); если его нет — индексируется только
тело. Вложенный PDF/картинка с плохим текстовым слоем проходит OCR, это
медленно. `.json`-метаданные (CuraSoft) индексируются как текст и могут
добавить шумовых токенов.
+96
View File
@@ -0,0 +1,96 @@
---
type: reference
status: current
related:
- docs/elasticsearch.md
- docs/unified-file-client.md
- docs/community-server-db.md
---
# Поиск и индексация
Четыре разных контура поиска. Не путать: у каждого свой индекс, свои входы и
свой способ обновления.
| Контур | Что ищет | Индекс | Обновление | Вход |
|--------|----------|--------|------------|------|
| REST `@search` | только имена в БД | нет | — (живой запрос) | `oo search` (по умолчанию `--backend oo`) |
| ES `files_file` | имя + текст Office | Elasticsearch портала | сервер, асинхронно | `oo search --content` |
| ES `oo_docs_text` | PDF/сканы (свой) | Elasticsearch портала | `oo index` | `oo search --backend own` |
## Карта кода
- `filestore_core.go` — интерфейсы `Searcher`, модели `SearchQuery`/`SearchHit`.
- `filestore_es.go` — `ESSearcher` (индекс OnlyOffice `files_file`).
- `filestore_es_text.go` — `ESTextIndex` (`oo_docs_text`): `Ensure`, `Put`, `Delete`,
`Search`.
- `filestore_text_index.go` — `TextIndexer`: обход папок (`FileStore.List`), download
(`FileStore.Download`), извлечение текста (`internal/docpipe`), запись в
`TextIndex`; пул воркеров.
- `filestore_facade.go` — связка бэкендов (`Files().Search()`, порядок и fallback).
- CLI: `cmd/oo/search.go`, `cmd/oo/index.go`.
- Разовые бинари для match (`ooscan`, `pdfamount`) живут в приватном
`oo-workspace`.
- `internal/docpipe` — текст из PDF (pdftotext), для сканов OCR
(ocrmypdf/tesseract), вложения PDF (pdfdetach).
## Поиск
```bash
# имя, индекс портала
oo search "Rechnung" --limit 50 --json
# имя + текст Office (docx/xlsx/pptx)
oo search "Mahngebühr" --content
# свой индекс: PDF и сканы
oo search "S1019" --backend own --folder 649
```
Флаги `oo search`: `--content`, `--folder ID`, `--limit N`, `--backend oo|own`,
`--substring`, `--json`. Требует `ONLYOFFICE_ES_URL` (см.
[elasticsearch.md](elasticsearch.md)); `--backend own` дополнительно ничего не
требует от сервера — читает `oo_docs_text`.
Почему не REST: `GET /api/2.0/files/@search/{query}` ищет только имя в БД
(`fileDao.Search`), ES не трогает. Почему PDF не в `files_file`: сервер индексит
контент только для форматов из `files.index.formats` (по умолчанию
`.pptx|.xlsx|.docx`) — отсюда свой `oo_docs_text`.
## Обновление своего индекса (`oo index`)
```bash
# одна папка
oo index folder 649 --recursive --exts pdf
# точечно по id
oo index files 3576 3578
# без записи: что было бы проиндексировано
oo index folder 649 --recursive --dry-run
```
Флаги: `--recursive`, `--exts pdf` (по умолчанию), `--limit N`,
`--workers 3`, `--lang deu+eng`, `--min-chars N` (порог текстового слоя, ниже
которого включается OCR), `--work-dir`, `--backend rest|dav`, `--dry-run`,
`--json`.
Свойства:
- Идемпотентно: upsert по `id` файла; повтор не двоит.
- Сервер OnlyOffice не меняется: индекс живёт рядом (`ONLYOFFICE_ES_TEXT_INDEX`,
по умолчанию `oo_docs_text`).
- Медленно на сканах (OCR на каждый файл). Ограничивай `--folder`/`--limit`,
не индексируй корень целиком.
- Индексация PDF в `files_file` не делается — только `oo_docs_text`.
## Bulk-инструменты
Плоские TSV-инструменты (`ooscan`, `pdfamount`) и сверка Excel живут в
приватном `oo-workspace`, не в публичной библиотеке.
## Грабли
- ES слушает только `127.0.0.1:9200` внутри VM — SSH-туннель обязателен
(`-o ControlMaster=no -o ControlPath=none`, см. [elasticsearch.md](elasticsearch.md)).
- Портальные листинги/скачивание упираются в 429; все bulk-пути идут через
`DoRetry` (линейный бэкофф), `ooscan` дополнительно спит 350 мс на папку.
- `files_file` обновляется сервером асинхронно — свежий файл виден не сразу.
- `title` analyzer `whitespacecustom`: имя — один токен, подстрока только через
`--substring` (или wildcard).
+55
View File
@@ -0,0 +1,55 @@
---
type: reference
status: current
related:
- README.md
- ../AGENTS.md
---
# Rate limit, backoff и cooldown
Устойчивость к 429 (openresty). Всё встроено в библиотеку — отдельный пакет не
нужен. Реализация: `ratelimit.go`, `retry.go`.
## Что происходит с каждым запросом
1. **Cooldown-гейт** — общий на процесс. Если недавно пришёл 429, все запросы
ждут конца окна.
2. **Rate limiter** — token bucket на процесс. Пейсит все HTTP-пути: листинг,
создание папок, загрузку, `get project`, auth.
3. Запрос уходит.
4. Ответ ≥400 → `*TransientError` (для 429/502/503/504) с `Retry-After`.
5. `DoRetry` — экспоненциальный backoff, без jitter.
6. `Retry-After` длиннее backoff → ждём его; окно уходит в общий cooldown.
Установлено в `NewClient` через `pacedTransport`; отдельный код трогать не надо.
## Env
| Переменная | Default | Смысл |
|---|---|---|
| `OO_RATE_LIMIT` | `4` | запросов/с на процесс; `0` — лимитер выключен |
| `OO_BURST` | `1` | запас токенов token bucket |
| `OO_RETRY_ATTEMPTS` | `7` | всего попыток, включая первую |
| `OO_RETRY_BASE` | `2s` | база экспоненты: ждать перед попыткой N = `Base*2^(N-1)` |
| `OO_RETRY_MAX` | `2m` | потолок ожидания |
Битые значения → default. `OO_RETRY_*` — формат `time.ParseDuration`
(`2s`, `30s`, `2m`).
## Правила
- Детерминированно, без jitter — повторный прогон ждёт столько же.
- `Retry-After` — секунды (`120`) или HTTP-date.
- Cooldown общий: параллельные и последовательные вызовы не бьют в стену.
- Backoff cap не ограничивает `Retry-After` — серверу верим больше.
- Только stdlib.
## Когда руками снять нагрузку
`OO_RATE_LIMIT` ниже (`2`), `OO_BURST=1`; при массовом apply — батчами.
## Тесты
`retry_test.go` — `Retry-After`, экспонента, cap; `ratelimit_test.go` — burst,
`OO_RATE_LIMIT=0`, cooldown. Фейковый сервер отдаёт 429 с заголовком.
+204
View File
@@ -0,0 +1,204 @@
---
type: reference
status: current
related:
- README.md
- filestore_core.go
- filestore_facade.go
- docs/elasticsearch.md
- docs/community-server-db.md
---
# Unified file client — контракт файловых бэкендов
## Что это
Один файловый клиент на все бэкенды (эпик #34). Модель и интерфейсы —
`filestore_core.go`. Фасад `FileClient` — `filestore_facade.go`. Бэкенды:
REST, WebDAV, SQL (PostgreSQL/MySQL), Elasticsearch. Правило одно:
код зовёт `c.Files()` и не знает про транспорт.
## Модель
- `Kind` — `File` (0) или `Folder` (1).
- `Entry` — бэкенд-независимая строка: `ID`, `ParentID`, `Title`, `Kind`,
`Size`, `MIME`, `Created`, `Modified`, `Updated` (сырая строка API),
`Version`, `Provider`, `FilesCount`/`FoldersCount` (папки).
Чего бэкенд не даёт — остаётся в нуле.
- `SearchQuery` — `Text`, `InContent`, `FolderID`, `Extensions`, `Limit`.
- `SearchHit` — `Entry` + `Score`, `Highlight`, `Path`.
## Интерфейсы
`FileStore` — операции с файлами:
```go
type FileStore interface {
Name() string
List(ctx, parentID) ([]Entry, error)
Stat(ctx, id) (Entry, error)
CreateFolder(ctx, parentID, title) (Entry, error)
Upload(ctx, parentID, title, r) (Entry, error)
Download(ctx, id, w) (int64, error)
Move(ctx, ids, parentID) error
Copy(ctx, ids, parentID) error
Rename(ctx, id, title) error
Delete(ctx, ids) error
}
```
`Searcher` — поиск (необязательный):
```go
type Searcher interface {
Search(ctx, q SearchQuery) ([]SearchHit, error)
Name() string
}
```
`TextIndex` (`filestore_es_text.go`) — свой индекс: `Put`, `Delete`, `Search`,
`Name`. `ESTextIndex` реализует и `Searcher`, и `TextIndex`.
## Бэкенды
| бэкенд | провайдер | файл | что умеет |
|--------|-----------|------|-----------|
| REST | `rest` | `filestore_rest.go` | read + write, Documents API |
| WebDAV | `dav` | `filestore_dav.go` | read + write, Documents fileops |
| SQL | `postgres` / `mysql` | `filestore_pg.go` | **read-only** |
| OnlyOffice ES | `elasticsearch` | `filestore_es.go` | поиск (имя + контент Office) |
| свой ES-индекс | `es-text` | `filestore_es_text.go` | поиск + запись (PDF/сканы) |
- REST: `Stat` знает только файлы; папки — через `List`.
- WebDAV: `Move`/`Copy`/`Delete` сперва `Stat`-ят id (папка/файл), потом зовут
fileops.
- SQL: `List`/`Stat`/`Download`/`Search` (по имени). Все write-методы →
`ErrReadOnly`. `Download` идёт в S3/MinIO по layout портала.
- OnlyOffice ES: индекс `files_file`, контент только для docx/xlsx/pptx.
- Свой ES: индекс `oo_docs_text`, контент из `internal/docpipe`, в т.ч.
встроенные PDF-вложения.
## Фасад `FileClient`
`c.Files()` → `*FileClient`. Он же реализует `FileStore`, старый код
компилируется.
- `Read()` — первый зарегистрированный из `readOrder`:
`postgres` → `mysql` → `rest` → `dav`.
- `Write()` — первый из `writeOrder`: `rest` → `dav`. SQL не пишет.
- `Search()` — первый из `searchOrder`: `elasticsearch`. Нет бэкенда →
ошибка (`ONLYOFFICE_ES_URL`).
- `RegisterStore(name, s)` / `RegisterSearcher(name, s)` — добавить бэкенд.
Fallback:
- `List`/`Stat` идут по `readOrder`; переходят к следующему только на
transient-ошибке (429/502/503/504). Иначе ошибка финальная.
- `Download` **без** fallback: часть байтов уже в `w`, второй бэкенд допишет.
- Запись (`CreateFolder`/`Upload`/`Move`/`Copy`/`Rename`/`Delete`) — только
`Write()`, без fallback.
`newFileClient` сам кладёт `rest` и `dav`; ES-поиск — если задан
`ONLYOFFICE_ES_URL`. SQL-стор регистрирует вызывающий: фасад создаётся на
каждый `c.Files()`, регистрируй на том же экземпляре.
```go
sql, err := c.SQLFileStore() // открыть из env (ONLYOFFICE_DSN)
if err != nil { /* нет DSN */ }
if closer, ok := sql.(interface{ Close() error }); ok { defer closer.Close() }
f := c.Files()
f.RegisterStore(onlyoffice.ProviderPG, sql) // или sql.Name() == "mysql"
e, _ := f.Stat(ctx, "19423") // e.Provider == "mysql"
entries, _ := f.List(ctx, "676") // пойдёт в SQL
```
`Client.FileStore("pg"|"sql"|"postgres"|"mysql")` — одноразовый доступ к
SQL-стору без фасада: открывает из env; при ошибке возвращает заглушку,
которая отдаёт ошибку открытия на каждом вызове (не `nil`). `SQLFileStore()`
— тот же открыватель, но с ошибкой. Отвечавший бэкенд видно по
`Entry.Provider` (`mysql` / `postgres` у SQL, `rest` у REST).
## CLI
```bash
# поиск: --backend oo (индекс OnlyOffice) | own (свой oo_docs_text)
oo search "Rechnung"
oo search "Mahngebühr" --content
oo search "S1021" --content --backend own --folder 634 --limit 50 --json
# наполнение своего индекса (PDF/сканы, idempotent upsert по file id)
oo index folder 634
oo index folder 634 --recursive --exts pdf,png --limit 100
oo index files 3576 3578
oo index folder 634 --dry-run
```
`oo index` флаги: `--recursive`, `--exts` (default `pdf`), `--limit`,
`--workers` (3), `--lang` (`deu+eng`), `--min-chars`, `--work-dir`,
`--backend rest|dav`, `--dry-run`, `--json`.
Библиотека:
```go
idx, _ := onlyoffice.NewESTextIndex(onlyoffice.ESTextConfigFromEnv())
ti := onlyoffice.NewTextIndexer(store, idx) // store = FileStore
res, _ := ti.IndexFolder(ctx, "634", onlyoffice.IndexOptions{Recursive: true})
```
## Env (только имена)
| env | default | кто читает |
|-----|---------|------------|
| `ONLYOFFICE_ES_URL` | — | ES (оба индекса), обязателен |
| `ONLYOFFICE_ES_INDEX` | `files_file` | индекс OnlyOffice |
| `ONLYOFFICE_ES_TEXT_INDEX` | `oo_docs_text` | свой индекс |
| `ONLYOFFICE_TENANT` | пусто | фильтр `tenantId` |
| `ONLYOFFICE_DSN` | — | SQL DSN (MySQL/PostgreSQL) |
| `ONLYOFFICE_PG_DRIVER` | auto | `postgres` / `mysql` |
| `ONLYOFFICE_PG_TENANT` | `ONLYOFFICE_TENANT` | SQL tenant |
| `ONLYOFFICE_PG_HOST` `_PORT` `_USER` `_PASSWORD` `_DBNAME` `_SSLMODE` | — | DSN по частям |
| `MINIO_ENDPOINT` `MINIO_BUCKET` `MINIO_ACCESS_KEY` `MINIO_SECRET_KEY` | — | download SQL-стора |
| `OO_URL` `OO_USER` `OO_PASS` | — | CLI-алиасы |
Имена — в [`.env.example`](../.env.example). Секретов в репо нет.
## Ограничения
- OnlyOffice ES: контент только Office-форматов. PDF — только по имени.
Встроенные вложения PDF сервер не индексирует.
- Свой индекс `oo_docs_text`: покрывает PDF/сканы и вложения (pdfdetach), но
наполняется вручную (`oo index`) и идемпотентен. Фильтр `folder` — id папки,
не путь. Дубли дают несколько строк — дедуп на потребителе.
- SQL: read-only. `InContent` игнорируется (только имя). Download — через
MinIO-схему, не HTTP.
- ES: без auth, слушает localhost внутри VM — нужен SSH-туннель
(см. [elasticsearch.md](elasticsearch.md)).
- `oo index` качает каждый файл и для сканов гоняет OCR — медленно; отсюда
`--limit` и `--exts`. Нужен `pdfdetach` (poppler); без него — только тело PDF.
## Как добавить бэкенд
1. Файл `file_<name>.go`. Реализуй `FileStore` (`Name` + 9 методов). Нужен
поиск — добавь `Searcher`; нужна запись своего индекса — `TextIndex`.
2. Добавь const провайдера рядом с `ProviderREST`/`ProviderDAV`.
3. Зарегистрируй: в `newFileClient` или снаружи через
`RegisterStore`/`RegisterSearcher`.
4. Внеси имя в `readOrder` / `writeOrder` / `searchOrder`.
5. Есть CLI-команда — добавь значение в `--backend`.
6. Тесты: unit (чистые builders/парсеры, без сети) + интеграционный
(`//go:build integration`, skip без кред).
## Тесты
```bash
go test ./... # unit, без сети
go test -tags=integration ./... # live (креды в .env)
go test ./ -run 'FileStore|Facade|ESText|PG'
```
## См. также
- [README.md](README.md) — индекс справочников.
- [elasticsearch.md](elasticsearch.md) — индекс OnlyOffice и свой `oo_docs_text`.
- [community-server-db.md](community-server-db.md) — SQL-стор и схема БД.
+155 -37
View File
@@ -237,7 +237,18 @@ func (c *Client) UploadProjectFile(ctx context.Context, projectID, localPath str
return decodeResponseFileEntry(raw)
}
// UploadProjectFileReplacing upserts by stem|ext in the project Documents folder.
func (c *Client) UploadProjectFileReplacing(ctx context.Context, projectID, localPath string) (*FileEntry, []int, error) {
folderID, err := c.projectFolderID(ctx, projectID)
if err != nil {
return nil, nil, err
}
return c.UploadToFolderReplacing(ctx, folderID, localPath)
}
// GetFile returns file metadata including viewUrl for download.
//
// Deprecated: use FileStore.Stat via Client.Files()/Client.FileStore.
func (c *Client) GetFile(ctx context.Context, fileID string) (*FileEntry, error) {
if fileID == "" {
return nil, fmt.Errorf("file id is required")
@@ -251,6 +262,8 @@ func (c *Client) GetFile(ctx context.Context, fileID string) (*FileEntry, error)
}
// RenameFile sets a new title (including extension) for the file.
//
// Deprecated: use FileStore.Rename via Client.Files()/Client.FileStore.
func (c *Client) RenameFile(ctx context.Context, fileID, newTitle string) (*FileEntry, error) {
if fileID == "" || newTitle == "" {
return nil, fmt.Errorf("file id and new title are required")
@@ -263,55 +276,150 @@ func (c *Client) RenameFile(ctx context.Context, fileID, newTitle string) (*File
return decodeResponseFileEntry(raw)
}
type deleteFilesBody struct {
FileIDs []int `json:"fileIds"`
FolderIDs []int `json:"folderIds"`
}
// DeleteFiles permanently deletes files by numeric id (Documents module).
// Uses per-file DELETE (DeleteDavItems); fileops/delete returns 200 on some
// portals without actually removing the file.
//
// Deprecated: use FileStore.Delete via Client.Files()/Client.FileStore.
func (c *Client) DeleteFiles(ctx context.Context, fileIDs []int) error {
if len(fileIDs) == 0 {
return fmt.Errorf("no file ids to delete")
}
body := deleteFilesBody{FileIDs: fileIDs, FolderIDs: nil}
_, err := c.putJSON(ctx, "/api/2.0/files/fileops/delete.json", body)
if err != nil {
_, err = c.putJSON(ctx, "/api/2.0/files/fileops/delete", body)
strIDs := make([]string, len(fileIDs))
for i, id := range fileIDs {
strIDs[i] = strconv.Itoa(id)
}
return err
return c.DeleteDavItems(ctx, nil, strIDs)
}
// ListFolder returns the Documents module listing for a folder id
// (GET /api/2.0/files/{folderId}).
//
// Deprecated: use FileStore.List via Client.Files()/Client.FileStore.
func (c *Client) ListFolder(ctx context.Context, folderID string) (map[string]any, error) {
if folderID == "" {
return nil, fmt.Errorf("folder id is required")
}
out, err := c.ResponseObject(ctx, "/api/2.0/files/"+url.PathEscape(folderID)+".json")
if err != nil {
out, err = c.ResponseObject(ctx, "/api/2.0/files/"+url.PathEscape(folderID))
}
return out, err
}
// CreateFolder creates a subfolder under parentFolderID.
func (c *Client) CreateFolder(ctx context.Context, parentFolderID, title string) (map[string]any, error) {
if parentFolderID == "" || title == "" {
return nil, fmt.Errorf("parent folder id and title are required")
}
body := map[string]any{"title": title}
out, err := c.postJSONObject(ctx, "/api/2.0/files/folder/"+url.PathEscape(parentFolderID)+".json", body)
if err != nil {
out, err = c.postJSONObject(ctx, "/api/2.0/files/folder/"+url.PathEscape(parentFolderID), body)
}
return out, err
}
// MoveFiles moves file ids into destFolderID (Documents fileops/move).
//
// Deprecated: use FileStore.Move via Client.Files()/Client.FileStore.
func (c *Client) MoveFiles(ctx context.Context, destFolderID int, fileIDs []int) (map[string]any, error) {
if destFolderID == 0 || len(fileIDs) == 0 {
return nil, fmt.Errorf("dest folder and file ids are required")
}
body := map[string]any{
"folderIds": []int{},
"fileIds": fileIDs,
"destFolderId": destFolderID,
"resolveType": "Skip",
"holdResult": true,
}
// fileops/move answers an operations envelope (like MoveDavItems), not a
// single object, so parse the raw body before unwrapping and surface any
// per-operation error. Unwrapping first (putJSONObject) made fileopsError
// look for a "response" key that was already stripped.
raw, err := c.putJSON(ctx, "/api/2.0/files/fileops/move", body)
if err != nil {
raw, err = c.putJSON(ctx, "/api/2.0/files/fileops/move.json", body)
if err != nil {
return nil, err
}
}
if ferr := fileopsError(raw); ferr != nil {
return nil, ferr
}
out, _ := unmarshalResponseObject(raw)
return out, nil
}
// UploadToFolder uploads a local file into an arbitrary Documents folder id.
//
// Deprecated: use FileStore.Upload via Client.Files()/Client.FileStore.
func (c *Client) UploadToFolder(ctx context.Context, folderID, localPath string) (*FileEntry, error) {
if folderID == "" || localPath == "" {
return nil, fmt.Errorf("folder id and local path are required")
}
uploadPath := fmt.Sprintf("/api/2.0/files/%s/upload.json", url.PathEscape(folderID))
raw, err := c.uploadMultipart(ctx, uploadPath, "file", localPath)
if err != nil {
uploadPath = fmt.Sprintf("/api/2.0/files/%s/upload", url.PathEscape(folderID))
raw, err = c.uploadMultipart(ctx, uploadPath, "file", localPath)
if err != nil {
return nil, err
}
}
return decodeResponseFileEntry(raw)
}
// UpdateFile uploads a new version of an existing file (same id, name and
// folder). It does not delete and does not create a second file.
//
// The Documents API method is PUT /api/2.0/files/{id}/update; POST is kept as
// a fallback for older servers. The path is tried with and without .json.
func (c *Client) UpdateFile(ctx context.Context, fileID, localPath string) (*FileEntry, error) {
if fileID == "" || localPath == "" {
return nil, fmt.Errorf("file id and local path are required")
}
base := fmt.Sprintf("/api/2.0/files/%s/update", url.PathEscape(fileID))
attempts := []struct {
method, path string
}{
{http.MethodPut, base},
{http.MethodPut, base + ".json"},
{http.MethodPost, base},
{http.MethodPost, base + ".json"},
}
var lastErr error
for _, a := range attempts {
raw, err := c.uploadMultipartMethod(ctx, a.method, a.path, "file", localPath)
if err == nil {
return decodeResponseFileEntry(raw)
}
lastErr = err
}
return nil, lastErr
}
// FileFolderID returns the parent folder id string for a file entry, if known.
func FileFolderID(f *FileEntry) string {
if f == nil || f.FolderID == nil {
return ""
}
return f.FolderID.String()
}
// DownloadFile streams file bytes from the file's viewUrl using the same auth
// as API calls. Writes into dst.
// as API calls. Writes into dst. When the portal serves the file from its stale
// AWS S3 consumer, the bytes are fetched from the local MinIO store instead
// (see storage_fallback.go).
//
// Deprecated: use FileStore.Download via Client.Files()/Client.FileStore.
func (c *Client) DownloadFile(ctx context.Context, fileID string, dst io.Writer) (int64, error) {
f, err := c.GetFile(ctx, fileID)
if err != nil {
return 0, err
}
if f.ViewURL == nil || *f.ViewURL == "" {
return 0, fmt.Errorf("file %s has no viewUrl", fileID)
}
downloadURL := c.resolveAPIURL(*f.ViewURL)
auth, err := c.authHeader()
if err != nil {
return 0, err
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, downloadURL, nil)
if err != nil {
return 0, err
}
req.Header.Set("Authorization", auth)
resp, err := c.client.Do(req)
if err != nil {
return 0, err
}
defer resp.Body.Close()
if resp.StatusCode >= 400 {
b, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
return 0, fmt.Errorf("GET viewUrl: %d %s", resp.StatusCode, truncate(string(b), 400))
}
n, err := io.Copy(dst, resp.Body)
return n, err
return c.downloadFileEntry(ctx, f, dst)
}
func (c *Client) resolveAPIURL(ref string) string {
@@ -319,8 +427,18 @@ func (c *Client) resolveAPIURL(ref string) string {
if ref == "" {
return ref
}
if strings.HasPrefix(ref, "http://") || strings.HasPrefix(ref, "https://") {
return ref
// Rewrite any host to the configured API base so downloads stay on the
// internal network and keep the Authorization header (no cross-host
// redirect that would strip it). Scheme-relative URLs are handled too.
if strings.HasPrefix(ref, "//") {
ref = "http:" + ref
}
if u, err := url.Parse(ref); err == nil && u.IsAbs() {
if base, err2 := url.Parse(c.baseURL()); err2 == nil {
u.Scheme = base.Scheme
u.Host = base.Host
return u.String()
}
}
base := c.baseURL()
if strings.HasPrefix(ref, "/") {
+389
View File
@@ -0,0 +1,389 @@
package onlyoffice
import (
"context"
"encoding/json"
"path/filepath"
"sort"
"strings"
)
// FileEntryExt returns a normalized extension (lowercase, with leading dot).
func FileEntryExt(f *FileEntry) string {
if f == nil {
return ""
}
exst := ""
if f.FileExst != nil {
exst = strings.TrimSpace(*f.FileExst)
}
if exst != "" {
if !strings.HasPrefix(exst, ".") {
exst = "." + exst
}
return strings.ToLower(exst)
}
if f.Title != nil {
if ext := filepath.Ext(*f.Title); ext != "" {
return strings.ToLower(ext)
}
}
return ""
}
// FileDedupKey is stem|ext — two files with the same key are duplicates.
func FileDedupKey(f *FileEntry) string {
st := FileEntryStem(f)
ext := FileEntryExt(f)
if st == "" {
return ""
}
if ext == "" {
return st
}
return st + "|" + strings.TrimPrefix(ext, ".")
}
// FindFilesByDedupKey returns folder files matching stem and extension.
func FindFilesByDedupKey(files []*FileEntry, stem, ext string) []*FileEntry {
key := dedupKeyFromParts(stem, ext)
if key == "" {
return nil
}
var out []*FileEntry
for _, f := range files {
if FileDedupKey(f) == key {
out = append(out, f)
}
}
return out
}
func dedupKeyFromParts(stem, ext string) string {
stem = strings.TrimSpace(stem)
if stem == "" {
return ""
}
ext = strings.ToLower(strings.TrimSpace(ext))
if ext != "" && !strings.HasPrefix(ext, ".") {
ext = "." + ext
}
if ext == "" {
return stem
}
return stem + "|" + strings.TrimPrefix(ext, ".")
}
// UploadExtFromLocal returns the lowercase extension from a local path.
func UploadExtFromLocal(localPath string) string {
ext := filepath.Ext(localPath)
if ext == "" {
return ""
}
return strings.ToLower(ext)
}
// IsTrashFolderTitle reports staging/trash folders (e.g. _trash-md).
func IsTrashFolderTitle(title string) bool {
t := strings.ToLower(strings.TrimSpace(title))
return strings.HasPrefix(t, "_") || strings.Contains(t, "trash")
}
// ProjectFolderFile ties a file to its project Documents subfolder.
type ProjectFolderFile struct {
FolderID string
FolderTitle string
File *FileEntry
}
// DedupGroup is one duplicate set: keep the newest (or non-trash) file.
type DedupGroup struct {
Key string
FolderID string
FolderTitle string
Keep *FileEntry
Remove []*FileEntry
}
// DedupOptions controls project-wide duplicate scans.
type DedupOptions struct {
CrossFolder bool
}
// FindProjectDuplicates scans project folders for duplicate files.
func FindProjectDuplicates(folders []*FolderEntry, filesByFolder map[string][]*FileEntry, opts DedupOptions) []DedupGroup {
var indexed []ProjectFolderFile
for _, folder := range folders {
if folder == nil || folder.ID == nil {
continue
}
fid := folder.ID.String()
title := ""
if folder.Title != nil {
title = *folder.Title
}
for _, f := range filesByFolder[fid] {
if f == nil {
continue
}
indexed = append(indexed, ProjectFolderFile{
FolderID: fid, FolderTitle: title, File: f,
})
}
}
if opts.CrossFolder {
return findCrossFolderDuplicates(indexed)
}
return findWithinFolderDuplicates(indexed)
}
func findWithinFolderDuplicates(indexed []ProjectFolderFile) []DedupGroup {
byFolder := map[string][]ProjectFolderFile{}
for _, it := range indexed {
byFolder[it.FolderID] = append(byFolder[it.FolderID], it)
}
var out []DedupGroup
for fid, items := range byFolder {
title := ""
if len(items) > 0 {
title = items[0].FolderTitle
}
byKey := map[string][]*FileEntry{}
for _, it := range items {
k := FileDedupKey(it.File)
byKey[k] = append(byKey[k], it.File)
}
for k, group := range byKey {
if k == "" {
continue // dotfiles etc. have no stem: never treat as duplicates
}
if len(group) < 2 {
continue
}
keep, remove := pickDuplicateKeeper(group, false)
if keep == nil || len(remove) == 0 {
continue
}
out = append(out, DedupGroup{
Key: k, FolderID: fid, FolderTitle: title, Keep: keep, Remove: remove,
})
}
}
sortDedupGroups(out)
return out
}
func findCrossFolderDuplicates(indexed []ProjectFolderFile) []DedupGroup {
byKey := map[string][]ProjectFolderFile{}
for _, it := range indexed {
k := FileDedupKey(it.File)
byKey[k] = append(byKey[k], it)
}
var out []DedupGroup
for k, items := range byKey {
if k == "" {
continue // dotfiles etc. have no stem: never treat as duplicates
}
if len(items) < 2 {
continue
}
files := make([]*FileEntry, len(items))
folders := make([]string, len(items))
folderTitles := make([]string, len(items))
for i, it := range items {
files[i] = it.File
folders[i] = it.FolderID
folderTitles[i] = it.FolderTitle
}
keep, remove := pickDuplicateKeeperWithFolders(files, folders, folderTitles)
if keep == nil || len(remove) == 0 {
continue
}
fid, ftitle := "", ""
for _, it := range items {
if it.File == keep {
fid, ftitle = it.FolderID, it.FolderTitle
break
}
}
out = append(out, DedupGroup{
Key: k, FolderID: fid, FolderTitle: ftitle, Keep: keep, Remove: remove,
})
}
sortDedupGroups(out)
return out
}
func pickDuplicateKeeper(files []*FileEntry, _ bool) (*FileEntry, []*FileEntry) {
return pickDuplicateKeeperWithFolders(files, nil, nil)
}
func pickDuplicateKeeperWithFolders(files []*FileEntry, folderIDs, folderTitles []string) (*FileEntry, []*FileEntry) {
if len(files) == 0 {
return nil, nil
}
type ranked struct {
file *FileEntry
trash bool
}
rankedFiles := make([]ranked, len(files))
for i, f := range files {
trash := false
if folderTitles != nil && i < len(folderTitles) {
trash = IsTrashFolderTitle(folderTitles[i])
}
rankedFiles[i] = ranked{file: f, trash: trash}
}
sort.SliceStable(rankedFiles, func(i, j int) bool {
ri, rj := rankedFiles[i], rankedFiles[j]
if ri.trash != rj.trash {
return !ri.trash // non-trash first
}
ti, tj := rankedFiles[i].file.Updated, rankedFiles[j].file.Updated
if ti == nil {
return false
}
if tj == nil {
return true
}
return ti.After(*tj) // newest first
})
keep := rankedFiles[0].file
var remove []*FileEntry
for _, r := range rankedFiles[1:] {
remove = append(remove, r.file)
}
return keep, remove
}
func sortDedupGroups(groups []DedupGroup) {
sort.Slice(groups, func(i, j int) bool {
if groups[i].FolderTitle != groups[j].FolderTitle {
return groups[i].FolderTitle < groups[j].FolderTitle
}
return groups[i].Key < groups[j].Key
})
}
// ApplyDedupGroups deletes Remove files from each group.
func (c *Client) ApplyDedupGroups(ctx context.Context, groups []DedupGroup) ([]int, error) {
seen := map[int]struct{}{}
var ids []int
for _, g := range groups {
for _, f := range g.Remove {
n := int(FileEntryNumericID(f))
if n == 0 {
continue
}
if _, ok := seen[n]; ok {
continue
}
seen[n] = struct{}{}
ids = append(ids, n)
}
}
if len(ids) == 0 {
return nil, nil
}
if err := c.DeleteFiles(ctx, ids); err != nil {
return ids, err
}
return ids, nil
}
// DeleteFilesByDedupKey removes all files in folderID matching stem+ext.
func (c *Client) DeleteFilesByDedupKey(ctx context.Context, folderID, stem, ext string) ([]int, error) {
files, err := c.FolderFiles(ctx, folderID)
if err != nil {
return nil, err
}
matches := FindFilesByDedupKey(files, stem, ext)
if len(matches) == 0 {
return nil, nil
}
ids := make([]int, 0, len(matches))
for _, f := range matches {
n := int(FileEntryNumericID(f))
if n != 0 {
ids = append(ids, n)
}
}
if len(ids) == 0 {
return nil, nil
}
if err := c.DeleteFiles(ctx, ids); err != nil {
return nil, err
}
return ids, nil
}
// mergeProjectRootForDedupe includes projectFolder files in dedupe scans. OO often lists
// root documents only in pf.Files while pf.Folders is empty.
func mergeProjectRootForDedupe(rootID string, folders []*FolderEntry, filesByFolder map[string][]*FileEntry, rootFiles []*FileEntry) ([]*FolderEntry, map[string][]*FileEntry) {
if rootID == "" {
return folders, filesByFolder
}
if filesByFolder == nil {
filesByFolder = map[string][]*FileEntry{}
}
for _, folder := range folders {
if folder != nil && folder.ID != nil && folder.ID.String() == rootID {
if len(rootFiles) > 0 {
filesByFolder[rootID] = rootFiles
}
return folders, filesByFolder
}
}
if len(rootFiles) == 0 {
return folders, filesByFolder
}
id := json.Number(rootID)
title := "(project root)"
folders = append(folders, &FolderEntry{ID: &id, Title: &title})
filesByFolder[rootID] = rootFiles
return folders, filesByFolder
}
// DedupeProject scans project folders and optionally deletes duplicates.
func (c *Client) DedupeProject(ctx context.Context, projectID string, opts DedupOptions, apply bool) ([]DedupGroup, []int, error) {
pf, err := c.GetProjectFiles(ctx, projectID)
if err != nil {
return nil, nil, err
}
rootID, err := c.projectFolderID(ctx, projectID)
if err != nil {
return nil, nil, err
}
var rootFiles []*FileEntry
if rootID != "" {
rootFiles, err = c.FolderFiles(ctx, rootID)
if err != nil {
return nil, nil, err
}
}
filesByFolder := make(map[string][]*FileEntry, len(pf.Folders)+1)
folders := make([]*FolderEntry, 0, len(pf.Folders)+1)
for _, folder := range pf.Folders {
if folder == nil || folder.ID == nil {
continue
}
fid := folder.ID.String()
if fid == rootID {
filesByFolder[fid] = rootFiles
} else {
files, err := c.FolderFiles(ctx, fid)
if err != nil {
return nil, nil, err
}
filesByFolder[fid] = files
}
folders = append(folders, folder)
}
folders, filesByFolder = mergeProjectRootForDedupe(rootID, folders, filesByFolder, rootFiles)
groups := FindProjectDuplicates(folders, filesByFolder, opts)
if !apply || len(groups) == 0 {
return groups, nil, nil
}
deleted, err := c.ApplyDedupGroups(ctx, groups)
return groups, deleted, err
}
+416
View File
@@ -0,0 +1,416 @@
//go:build integration
package onlyoffice
import (
"context"
"encoding/json"
"os"
"path/filepath"
"strconv"
"testing"
"time"
)
// TestIntegrationFileDedup proves on a live OnlyOffice portal that the file
// dedup helpers find real duplicates and delete only the redundant copies.
// It creates a throwaway "go-onlyoffice-test-" project (removed by cleanup),
// places same stem|ext files in two subfolders and in the project root, then
// exercises FindProjectDuplicates, mergeProjectRootForDedupe,
// ApplyDedupGroups and DeleteFilesByDedupKey. Destructive — run only against
// an instance you own.
func TestIntegrationFileDedup(t *testing.T) {
c := liveClient(t)
t.Cleanup(func() { cleanupTestProjects(t, c) })
ctx := context.Background()
suffix := time.Now().UTC().Format("20060102-150405")
project, err := c.CreateProject(NewProjectRequest{
Title: testProjectPrefix + "dedup-" + suffix,
Description: "go-onlyoffice file dedup integration",
})
if err != nil {
t.Fatalf("CreateProject: %v", err)
}
if project.ID == nil {
t.Fatal("created project without id")
}
pid := strconv.Itoa(*project.ID)
root := projectFolderEventually(t, ctx, c, pid)
aID := createFolderLive(t, ctx, c, root, "A-"+suffix)
bID := createFolderLive(t, ctx, c, root, "B-"+suffix)
createFolderLive(t, ctx, c, root, "_trash-"+suffix)
// Check IsTrashFolderTitle against a real live folder title.
if !IsTrashFolderTitle("_trash-" + suffix) {
t.Fatalf("IsTrashFolderTitle(%q) = false for a live _trash folder", "_trash-"+suffix)
}
stem := "dedup-" + suffix
local := writeLocalFile(t, stem+".txt", []byte("dedup integration "+suffix+"\n"))
a1 := uploadFolderLive(t, ctx, c, aID, local)
a2 := uploadFolderLive(t, ctx, c, aID, local)
b1 := uploadFolderLive(t, ctx, c, bID, local)
r1 := uploadFolderLive(t, ctx, c, root, local)
r2 := uploadFolderLive(t, ctx, c, root, local)
t.Logf("uploaded a1=%d a2=%d b1=%d r1=%d r2=%d",
FileEntryNumericID(a1), FileEntryNumericID(a2), FileEntryNumericID(b1),
FileEntryNumericID(r1), FileEntryNumericID(r2))
if n := dedupWaitCount(t, ctx, c, aID, stem, ".txt", 2, 30*time.Second); n != 2 {
t.Fatalf("folder A has %d copies after upload, want 2", n)
}
if n := dedupWaitCount(t, ctx, c, bID, stem, ".txt", 1, 30*time.Second); n != 1 {
t.Fatalf("folder B has %d copies after upload, want 1", n)
}
if n := dedupWaitCount(t, ctx, c, root, stem, ".txt", 2, 30*time.Second); n != 2 {
t.Fatalf("project root has %d copies after upload, want 2", n)
}
// #2 mergeProjectRootForDedupe on the live tree: the project root that
// carries documents must be part of the scan exactly once.
folders, byFolder, rootFiles := liveProjectIndex(t, ctx, c, pid, root)
merged, mergedBy := mergeProjectRootForDedupe(root, folders, byFolder, rootFiles)
rootCount := 0
for _, folder := range merged {
if folder != nil && folder.ID != nil && folder.ID.String() == root {
rootCount++
}
}
if rootCount != 1 {
t.Fatalf("mergeProjectRootForDedupe: project root appears %d times in live tree, want 1", rootCount)
}
if got := len(FindFilesByDedupKey(mergedBy[root], stem, ".txt")); got != 2 {
t.Fatalf("mergeProjectRootForDedupe: root carries %d matching files, want 2", got)
}
// Within-folder scan: a duplicate pair in A and in the project root.
within := FindProjectDuplicates(merged, mergedBy, DedupOptions{})
removesByFolder := map[string]int{}
for _, g := range within {
removesByFolder[g.FolderID] = len(g.Remove)
}
if len(within) != 2 || removesByFolder[aID] != 1 || removesByFolder[root] != 1 {
t.Fatalf("within-folder groups = %d (%v), want exactly A:1 root:1", len(within), removesByFolder)
}
// DeleteFilesByDedupKey removes every stem|ext copy in one folder.
removed, err := c.DeleteFilesByDedupKey(ctx, aID, stem, ".txt")
if err != nil {
t.Fatalf("DeleteFilesByDedupKey(A): %v", err)
}
if len(removed) != 2 {
t.Fatalf("DeleteFilesByDedupKey(A) removed %v, want 2 ids", removed)
}
if n := dedupWaitCount(t, ctx, c, aID, stem, ".txt", 0, 30*time.Second); n != 0 {
t.Fatalf("folder A still has %d copies after DeleteFilesByDedupKey", n)
}
// Re-create the A duplicates so the cross-folder project scan can be
// applied and a single survivor proven by polling.
uploadFolderLive(t, ctx, c, aID, local)
uploadFolderLive(t, ctx, c, aID, local)
if n := dedupWaitCount(t, ctx, c, aID, stem, ".txt", 2, 30*time.Second); n != 2 {
t.Fatalf("folder A has %d re-uploaded copies, want 2", n)
}
// Cross-folder dry-run over the live project: one key, five copies, and
// the project-root copies must be part of the group (root merge live).
groups, deleted, err := dedupeProjectEventually(t, ctx, c, pid, DedupOptions{CrossFolder: true}, false)
if err != nil {
t.Fatalf("DedupeProject: %v", err)
}
if len(deleted) != 0 {
t.Fatalf("dry-run DedupeProject deleted %v", deleted)
}
if len(groups) != 1 {
t.Fatalf("DedupeProject cross-folder groups = %d, want 1 (%+v)", len(groups), groups)
}
if len(groups[0].Remove) != 4 {
t.Fatalf("cross-folder group removes %d files, want 4", len(groups[0].Remove))
}
if !dedupGroupHasID(groups[0], FileEntryNumericID(r1)) && !dedupGroupHasID(groups[0], FileEntryNumericID(r2)) {
t.Fatalf("cross-folder group does not include a project-root copy (merge not applied)")
}
keepID := FileEntryNumericID(groups[0].Keep)
deleted, err = c.ApplyDedupGroups(ctx, groups)
if err != nil {
t.Fatalf("ApplyDedupGroups: %v", err)
}
if len(deleted) != 4 {
t.Fatalf("ApplyDedupGroups deleted %v, want 4 ids", deleted)
}
survivorID, total := dedupWaitTotal(t, ctx, c, []string{aID, bID, root}, stem, ".txt", 1, 40*time.Second)
if total != 1 {
t.Fatalf("after ApplyDedupGroups %d copies survive, want 1", total)
}
if survivorID != keepID {
t.Fatalf("remaining copy id = %d, want kept id %d", survivorID, keepID)
}
// The removed ids must really be gone from every folder.
gone := map[int64]bool{
FileEntryNumericID(a2): true,
FileEntryNumericID(b1): true,
FileEntryNumericID(r1): true,
FileEntryNumericID(r2): true,
}
for _, fid := range []string{aID, bID, root} {
files, err := c.FolderFiles(ctx, fid)
if err != nil {
t.Fatalf("FolderFiles %s: %v", fid, err)
}
for _, f := range FindFilesByDedupKey(files, stem, ".txt") {
id := FileEntryNumericID(f)
if gone[id] {
t.Fatalf("removed id %d still present in folder %s", id, fid)
}
}
}
t.Logf("survivor id=%d keep id=%d, deleted=%v", survivorID, keepID, deleted)
}
// createFolderLive creates a subfolder (retrying transient 5xx) and returns
// its Documents folder id.
func createFolderLive(t *testing.T, ctx context.Context, c *Client, parentID, title string) string {
t.Helper()
deadline := time.Now().Add(30 * time.Second)
var (
m map[string]any
err error
)
for {
m, err = c.CreateFolder(ctx, parentID, title)
if err == nil || time.Now().After(deadline) {
break
}
time.Sleep(time.Second)
}
if err != nil {
t.Fatalf("CreateFolder %q: %v", title, err)
}
if m == nil {
t.Fatalf("CreateFolder %q: empty response", title)
}
switch v := m["id"].(type) {
case float64:
return strconv.FormatInt(int64(v), 10)
case json.Number:
return v.String()
case string:
if v != "" {
return v
}
}
t.Fatalf("CreateFolder %q: no id in response %#v", title, m)
return ""
}
// writeLocalFile writes content to a temp file and returns its path.
func writeLocalFile(t *testing.T, name string, content []byte) string {
t.Helper()
p := filepath.Join(t.TempDir(), name)
if err := os.WriteFile(p, content, 0o600); err != nil {
t.Fatal(err)
}
return p
}
// uploadFolderLive uploads localPath into folderID (retrying the portal's
// transient post-create 500) and returns the file entry.
func uploadFolderLive(t *testing.T, ctx context.Context, c *Client, folderID, localPath string) *FileEntry {
t.Helper()
deadline := time.Now().Add(30 * time.Second)
var (
e *FileEntry
err error
)
for {
e, err = c.UploadToFolder(ctx, folderID, localPath)
if err == nil || time.Now().After(deadline) {
break
}
time.Sleep(time.Second)
}
if err != nil {
t.Fatalf("UploadToFolder %s: %v", folderID, err)
}
if e == nil || e.ID == nil {
t.Fatalf("UploadToFolder %s: no file entry (%+v)", folderID, e)
}
return e
}
// liveProjectIndex rebuilds the project folder/file index the same way
// DedupeProject does, for direct mergeProjectRootForDedupe assertions.
func liveProjectIndex(t *testing.T, ctx context.Context, c *Client, projectID, rootID string) ([]*FolderEntry, map[string][]*FileEntry, []*FileEntry) {
t.Helper()
pf := getProjectFilesEventually(t, ctx, c, projectID)
rootFiles := folderFilesEventually(t, ctx, c, rootID)
folders := make([]*FolderEntry, 0, len(pf.Folders)+1)
byFolder := make(map[string][]*FileEntry, len(pf.Folders)+1)
for _, folder := range pf.Folders {
if folder == nil || folder.ID == nil {
continue
}
fid := folder.ID.String()
if fid == rootID {
byFolder[fid] = rootFiles
} else {
byFolder[fid] = folderFilesEventually(t, ctx, c, fid)
}
folders = append(folders, folder)
}
return folders, byFolder, rootFiles
}
// projectFolderEventually resolves the project Documents root id, retrying on
// a transient portal answer.
func projectFolderEventually(t *testing.T, ctx context.Context, c *Client, projectID string) string {
t.Helper()
deadline := time.Now().Add(30 * time.Second)
var (
root string
err error
)
for {
root, err = c.projectFolderID(ctx, projectID)
if err == nil || time.Now().After(deadline) {
break
}
time.Sleep(time.Second)
}
if err != nil {
t.Fatalf("projectFolderID: %v", err)
}
if root == "" {
t.Fatal("projectFolderID returned empty id")
}
return root
}
// getProjectFilesEventually lists a project's files/folders, retrying on a
// transient portal answer.
func getProjectFilesEventually(t *testing.T, ctx context.Context, c *Client, projectID string) *ProjectFilesResponse {
t.Helper()
deadline := time.Now().Add(30 * time.Second)
var last error
for {
pf, err := c.GetProjectFiles(ctx, projectID)
if err == nil {
return pf
}
last = err
if time.Now().After(deadline) {
t.Fatalf("GetProjectFiles %s: %v", projectID, last)
}
time.Sleep(500 * time.Millisecond)
}
}
// folderFilesEventually lists a folder, retrying while the portal answers
// transiently (a freshly created folder can 500 until its parent map settles).
func folderFilesEventually(t *testing.T, ctx context.Context, c *Client, folderID string) []*FileEntry {
t.Helper()
deadline := time.Now().Add(30 * time.Second)
var last error
for {
files, err := c.FolderFiles(ctx, folderID)
if err == nil {
return files
}
last = err
if time.Now().After(deadline) {
t.Fatalf("FolderFiles %s: %v", folderID, last)
}
time.Sleep(500 * time.Millisecond)
}
}
// dedupeProjectEventually runs a project dedup scan, retrying the whole scan on
// a transient portal error. It is used for dry-runs only (apply must stay a
// single deliberate call).
func dedupeProjectEventually(t *testing.T, ctx context.Context, c *Client, projectID string, opts DedupOptions, apply bool) ([]DedupGroup, []int, error) {
t.Helper()
deadline := time.Now().Add(30 * time.Second)
var (
groups []DedupGroup
deleted []int
err error
)
for {
groups, deleted, err = c.DedupeProject(ctx, projectID, opts, apply)
if err == nil || time.Now().After(deadline) {
return groups, deleted, err
}
time.Sleep(time.Second)
}
}
// dedupWaitCount polls folderID until want stem|ext copies are visible.
func dedupWaitCount(t *testing.T, ctx context.Context, c *Client, folderID, stem, ext string, want int, d time.Duration) int {
t.Helper()
deadline := time.Now().Add(d)
got := -1
for {
if files, err := c.FolderFiles(ctx, folderID); err == nil {
got = len(FindFilesByDedupKey(files, stem, ext))
if got == want {
return got
}
}
if time.Now().After(deadline) {
return got
}
time.Sleep(500 * time.Millisecond)
}
}
// dedupWaitTotal polls the given folders until the total number of stem|ext
// copies reaches want, returning the last seen file id and count.
func dedupWaitTotal(t *testing.T, ctx context.Context, c *Client, folderIDs []string, stem, ext string, want int, d time.Duration) (int64, int) {
t.Helper()
deadline := time.Now().Add(d)
var survivor int64
total := -1
for {
survivor, total = 0, 0
for _, fid := range folderIDs {
files, err := c.FolderFiles(ctx, fid)
if err != nil {
continue
}
for _, f := range FindFilesByDedupKey(files, stem, ext) {
total++
survivor = FileEntryNumericID(f)
}
}
if total == want {
return survivor, total
}
if time.Now().After(deadline) {
return survivor, total
}
time.Sleep(500 * time.Millisecond)
}
}
func dedupGroupHasID(g DedupGroup, id int64) bool {
if id == 0 {
return false
}
if FileEntryNumericID(g.Keep) == id {
return true
}
for _, f := range g.Remove {
if FileEntryNumericID(f) == id {
return true
}
}
return false
}
+127
View File
@@ -0,0 +1,127 @@
package onlyoffice
import (
"encoding/json"
"testing"
"time"
)
func TestFileDedupKey(t *testing.T) {
title := "OO-HONDA-7-INDEX.docx"
exst := ".docx"
f := &FileEntry{Title: &title, FileExst: &exst}
if got := FileDedupKey(f); got != "OO-HONDA-7-INDEX|docx" {
t.Fatalf("got %q", got)
}
}
func TestFindFilesByDedupKey(t *testing.T) {
a := &FileEntry{Title: strPtr("foo.docx"), FileExst: strPtr(".docx")}
b := &FileEntry{Title: strPtr("foo.md"), FileExst: strPtr(".md")}
files := []*FileEntry{a, b}
got := FindFilesByDedupKey(files, "foo", ".docx")
if len(got) != 1 || got[0] != a {
t.Fatalf("got %+v", got)
}
}
func TestFindWithinFolderDuplicates(t *testing.T) {
t1 := time.Date(2026, 8, 27, 16, 0, 0, 0, time.UTC)
t2 := t1.Add(time.Hour)
old := &FileEntry{ID: jsonNum("1"), Title: strPtr("idx.docx"), FileExst: strPtr(".docx"), Updated: &t1}
new := &FileEntry{ID: jsonNum("2"), Title: strPtr("idx.docx"), FileExst: strPtr(".docx"), Updated: &t2}
indexed := []ProjectFolderFile{
{FolderID: "490", FolderTitle: "00-Index", File: old},
{FolderID: "490", FolderTitle: "00-Index", File: new},
}
groups := findWithinFolderDuplicates(indexed)
if len(groups) != 1 {
t.Fatalf("groups=%d", len(groups))
}
if FileEntryNumericID(groups[0].Keep) != 2 {
t.Fatalf("keep id=%d", FileEntryNumericID(groups[0].Keep))
}
if len(groups[0].Remove) != 1 || FileEntryNumericID(groups[0].Remove[0]) != 1 {
t.Fatalf("remove=%v", groups[0].Remove)
}
}
func TestCrossFolderPrefersNonTrash(t *testing.T) {
t1 := time.Date(2026, 8, 27, 18, 0, 0, 0, time.UTC)
t2 := t1.Add(-time.Hour)
trash := &FileEntry{ID: jsonNum("10"), Title: strPtr("INDEX.md"), FileExst: strPtr(".md"), Updated: &t1}
good := &FileEntry{ID: jsonNum("20"), Title: strPtr("INDEX.md"), FileExst: strPtr(".md"), Updated: &t2}
indexed := []ProjectFolderFile{
{FolderID: "493", FolderTitle: "_trash-md", File: trash},
{FolderID: "492", FolderTitle: "OCR", File: good},
}
groups := findCrossFolderDuplicates(indexed)
if len(groups) != 1 {
t.Fatalf("groups=%d", len(groups))
}
if FileEntryNumericID(groups[0].Keep) != 20 {
t.Fatalf("keep id=%d", FileEntryNumericID(groups[0].Keep))
}
}
func TestMergeProjectRootForDedupe(t *testing.T) {
old := &FileEntry{ID: jsonNum("1"), Title: strPtr("a.docx"), FileExst: strPtr(".docx")}
newer := &FileEntry{ID: jsonNum("2"), Title: strPtr("a.docx"), FileExst: strPtr(".docx")}
rootFiles := []*FileEntry{old, newer}
folders, byFolder := mergeProjectRootForDedupe("489", nil, nil, rootFiles)
if len(folders) != 1 || folders[0].ID.String() != "489" {
t.Fatalf("folders=%+v", folders)
}
if len(byFolder["489"]) != 2 {
t.Fatalf("root files=%d", len(byFolder["489"]))
}
groups := findWithinFolderDuplicates([]ProjectFolderFile{
{FolderID: "489", FolderTitle: "(project root)", File: old},
{FolderID: "489", FolderTitle: "(project root)", File: newer},
})
if len(groups) != 1 {
t.Fatalf("groups=%d", len(groups))
}
}
func TestFindDuplicatesSkipsEmptyKey(t *testing.T) {
// Dotfiles (".env", ".gitignore", ".npmrc") normalize to an empty stem, so
// FileDedupKey is "". They are not duplicates of each other and must never
// form a dedup group that would delete one of them.
env := &FileEntry{ID: jsonNum("1"), Title: strPtr(".env")}
gitignore := &FileEntry{ID: jsonNum("2"), Title: strPtr(".gitignore")}
npmrc := &FileEntry{ID: jsonNum("3"), Title: strPtr(".npmrc")}
if FileDedupKey(env) != "" || FileDedupKey(gitignore) != "" || FileDedupKey(npmrc) != "" {
t.Fatalf("dotfiles should have empty dedup key")
}
within := []ProjectFolderFile{
{FolderID: "500", FolderTitle: "Cfg", File: env},
{FolderID: "500", FolderTitle: "Cfg", File: gitignore},
}
if groups := findWithinFolderDuplicates(within); len(groups) != 0 {
t.Fatalf("within-folder empty-key groups = %d, want 0 (%+v)", len(groups), groups)
}
cross := []ProjectFolderFile{
{FolderID: "500", FolderTitle: "Cfg", File: env},
{FolderID: "501", FolderTitle: "Other", File: npmrc},
}
if groups := findCrossFolderDuplicates(cross); len(groups) != 0 {
t.Fatalf("cross-folder empty-key groups = %d, want 0 (%+v)", len(groups), groups)
}
}
func TestIsTrashFolderTitle(t *testing.T) {
if !IsTrashFolderTitle("_trash-md") {
t.Fatal("expected trash")
}
if IsTrashFolderTitle("00-Index") {
t.Fatal("expected not trash")
}
}
func strPtr(s string) *string { return &s }
func jsonNum(s string) *json.Number {
n := json.Number(s)
return &n
}
+79
View File
@@ -0,0 +1,79 @@
package onlyoffice
// Human-readable folder paths for search results (F9). The OnlyOffice ES
// index stores only ancestor folder ids; titles live in the Documents tree, so
// resolving a path costs one GET /api/2.0/files/{id} per distinct folder,
// cached on the client. Folders that cannot be listed (e.g. a section root)
// fall back to their id, so a path is always produced.
import (
"context"
"strings"
)
// FolderTitle returns the title of a Documents folder id, cached on the client.
// An empty id yields an empty title. Unknown/unlistable ids (section roots)
// return ("", nil) so callers can fall back to the id.
func (c *Client) FolderTitle(ctx context.Context, folderID string) (string, error) {
folderID = strings.TrimSpace(folderID)
if folderID == "" {
return "", nil
}
c.folderTitlesMu.Lock()
if c.folderTitles != nil {
if t, ok := c.folderTitles[folderID]; ok {
c.folderTitlesMu.Unlock()
return t, nil
}
}
c.folderTitlesMu.Unlock()
title := ""
out, err := c.ListFolder(ctx, folderID)
if err == nil {
if cur, ok := out["current"].(map[string]any); ok {
if s, ok := cur["title"].(string); ok {
title = strings.TrimSpace(s)
}
}
}
c.folderTitlesMu.Lock()
if c.folderTitles == nil {
c.folderTitles = map[string]string{}
}
c.folderTitles[folderID] = title
c.folderTitlesMu.Unlock()
return title, nil
}
// FolderPath resolves an ancestor folder id chain (root → leaf, as the ES
// backend reports it) into folder titles, falling back to the id when a title
// cannot be read. The result never fails on a single lookup: only the whole
// call honours ctx cancellation.
func (c *Client) FolderPath(ctx context.Context, ids []string) []string {
out := make([]string, 0, len(ids))
for _, id := range ids {
if err := ctx.Err(); err != nil {
break
}
title, err := c.FolderTitle(ctx, id)
if err != nil || title == "" {
title = id
}
out = append(out, title)
}
return out
}
// UniquePath builds a stable, human-readable, unique path for a result: the
// resolved folder chain plus the file title. "." separates nothing — the
// segments are joined with "/", matching the Documents breadcrumb the web UI
// shows.
func (c *Client) UniquePath(ctx context.Context, folderPath []string, title string) string {
parts := c.FolderPath(ctx, folderPath)
if t := strings.TrimSpace(title); t != "" {
parts = append(parts, t)
}
return strings.Join(parts, "/")
}
+35
View File
@@ -0,0 +1,35 @@
//go:build integration
package onlyoffice
import (
"context"
"strings"
"testing"
)
// TestIntegrationFolderPath resolves the real Fibu EDL folder chain
// (project root 522 → Eingangsrechnungen 647 → 2025 649) to titles.
func TestIntegrationFolderPath(t *testing.T) {
creds := GetEnvironmentCredentials()
if strings.TrimSpace(creds.Url) == "" || strings.TrimSpace(creds.User) == "" {
t.Skip("no ONLYOFFICE_URL/USER credentials")
}
c := NewClient(creds)
ctx := context.Background()
path := c.FolderPath(ctx, []string{"522", "647", "649"})
if len(path) != 3 {
t.Fatalf("FolderPath returned %v, want 3 segments", path)
}
for i, seg := range path {
if strings.TrimSpace(seg) == "" {
t.Errorf("segment %d empty: %v", i, path)
}
}
full := c.UniquePath(ctx, []string{"522", "647", "649"}, "Rechnung-x.pdf")
if !strings.HasSuffix(full, "Rechnung-x.pdf") || !strings.Contains(full, "/") {
t.Errorf("UniquePath = %q, want a slash-joined path ending in the file", full)
}
t.Logf("path=%v full=%q", path, full)
}
+166
View File
@@ -0,0 +1,166 @@
package onlyoffice
import (
"context"
"encoding/json"
"errors"
"fmt"
"path/filepath"
"strings"
)
// ErrFileExists is returned when --no-replace / no-clobber upload hits an existing stem|ext.
var ErrFileExists = errors.New("onlyoffice: file already exists in folder (use replace or delete first)")
// FileEntryStem returns the logical basename without duplicated extensions.
// OO often stores title="foo.docx" and fileExst=".docx" (UI shows foo.docx.docx).
func FileEntryStem(f *FileEntry) string {
if f == nil || f.Title == nil {
return ""
}
exst := ""
if f.FileExst != nil {
exst = *f.FileExst
}
return NormalizeUploadStem(*f.Title, exst)
}
// NormalizeUploadStem derives a stable stem for matching uploads.
func NormalizeUploadStem(title, exst string) string {
t := strings.TrimSpace(title)
t = strings.TrimSuffix(t, ".")
if exst != "" && strings.HasSuffix(t, exst) {
t = strings.TrimSuffix(t, exst)
}
if ext := filepath.Ext(t); ext != "" {
t = strings.TrimSuffix(t, ext)
}
return strings.TrimSpace(t)
}
// UploadStemFromLocal returns the stem used to match/replace folder files.
func UploadStemFromLocal(localPath string) string {
base := filepath.Base(localPath)
ext := filepath.Ext(base)
if ext != "" {
base = strings.TrimSuffix(base, ext)
}
return base
}
// FolderFiles returns file entries in a Documents folder.
func (c *Client) FolderFiles(ctx context.Context, folderID string) ([]*FileEntry, error) {
raw, err := c.ListFolder(ctx, folderID)
if err != nil {
return nil, err
}
return ParseFolderFileEntries(raw), nil
}
// ParseFolderFileEntries extracts []*FileEntry from ListFolder JSON.
func ParseFolderFileEntries(raw map[string]any) []*FileEntry {
items, _ := raw["files"].([]any)
out := make([]*FileEntry, 0, len(items))
for _, it := range items {
m, ok := it.(map[string]any)
if !ok {
continue
}
b, err := json.Marshal(m)
if err != nil {
continue
}
var f FileEntry
if err := json.Unmarshal(b, &f); err != nil {
continue
}
out = append(out, &f)
}
return out
}
// FindFilesByStem returns folder files whose logical stem matches.
func FindFilesByStem(files []*FileEntry, stem string) []*FileEntry {
stem = strings.TrimSpace(stem)
if stem == "" {
return nil
}
var out []*FileEntry
for _, f := range files {
if FileEntryStem(f) == stem {
out = append(out, f)
}
}
return out
}
// DeleteFilesByStem removes all files in folderID matching stem (any extension).
// Prefer DeleteFilesByDedupKey when the upload extension is known.
func (c *Client) DeleteFilesByStem(ctx context.Context, folderID, stem string) ([]int, error) {
files, err := c.FolderFiles(ctx, folderID)
if err != nil {
return nil, err
}
matches := FindFilesByStem(files, stem)
if len(matches) == 0 {
return nil, nil
}
ids := make([]int, 0, len(matches))
for _, f := range matches {
n := int(FileEntryNumericID(f))
if n != 0 {
ids = append(ids, n)
}
}
if len(ids) == 0 {
return nil, nil
}
if err := c.DeleteFiles(ctx, ids); err != nil {
return nil, err
}
return ids, nil
}
// AssertNoFileConflict reports ErrFileExists when localPath stem|ext is already in folderID.
func (c *Client) AssertNoFileConflict(ctx context.Context, folderID, localPath string) error {
files, err := c.FolderFiles(ctx, folderID)
if err != nil {
return err
}
stem := UploadStemFromLocal(localPath)
ext := UploadExtFromLocal(localPath)
matches := FindFilesByDedupKey(files, stem, ext)
if len(matches) == 0 {
return nil
}
ids := make([]string, 0, len(matches))
for _, f := range matches {
ids = append(ids, fmt.Sprintf("%d", FileEntryNumericID(f)))
}
return fmt.Errorf("%w: %s%s in folder %s (existing file ids: %s)",
ErrFileExists, stem, ext, folderID, strings.Join(ids, ", "))
}
// UploadProjectFileNoClobber uploads only when stem|ext is not already in the project folder.
func (c *Client) UploadProjectFileNoClobber(ctx context.Context, projectID, localPath string) (*FileEntry, error) {
folderID, err := c.projectFolderID(ctx, projectID)
if err != nil {
return nil, err
}
if err := c.AssertNoFileConflict(ctx, folderID, localPath); err != nil {
return nil, err
}
return c.UploadProjectFile(ctx, projectID, localPath)
}
// UploadToFolderReplacing deletes same stem+ext files then uploads localPath.
func (c *Client) UploadToFolderReplacing(ctx context.Context, folderID, localPath string) (*FileEntry, []int, error) {
stem := UploadStemFromLocal(localPath)
ext := UploadExtFromLocal(localPath)
deleted, err := c.DeleteFilesByDedupKey(ctx, folderID, stem, ext)
if err != nil {
return nil, deleted, err
}
ent, err := c.UploadToFolder(ctx, folderID, localPath)
return ent, deleted, err
}
+35
View File
@@ -0,0 +1,35 @@
package onlyoffice
import "testing"
func TestNormalizeUploadStem(t *testing.T) {
tests := []struct {
title, exst, want string
}{
{"OO-HONDA-7-INDEX.docx", ".docx", "OO-HONDA-7-INDEX"},
{"README.txt", ".txt", "README"},
{"car-docs-print.docx", ".docx", "car-docs-print"},
{"plain.", "", "plain"},
{"foo", ".docx", "foo"},
}
for _, tc := range tests {
if got := NormalizeUploadStem(tc.title, tc.exst); got != tc.want {
t.Fatalf("NormalizeUploadStem(%q,%q)=%q want %q", tc.title, tc.exst, got, tc.want)
}
}
}
func TestFileEntryStem(t *testing.T) {
title := "00-INDEX.docx"
exst := ".docx"
f := &FileEntry{Title: &title, FileExst: &exst}
if got := FileEntryStem(f); got != "00-INDEX" {
t.Fatalf("got %q", got)
}
}
func TestUploadStemFromLocal(t *testing.T) {
if got := UploadStemFromLocal("/tmp/OO-HONDA-7-INDEX.docx"); got != "OO-HONDA-7-INDEX" {
t.Fatalf("got %q", got)
}
}
+482
View File
@@ -0,0 +1,482 @@
package onlyoffice
// WebDAV-oriented Files operations. These expose the Documents module through
// value types and cover everything needed to back a filesystem mapping:
// listing (including the virtual @root sections), folder/file CRUD, move/copy,
// and streaming upload/download. They are intentionally small and dependency
// free (only net/http), so callers are not forced to import heavier parts of
// the library.
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"mime/multipart"
"net/http"
"net/url"
"strconv"
"strings"
"time"
)
// DavFolder is a folder row from the Files module.
type DavFolder struct {
ID string
Title string
ParentID string
RootType int // 1=Common, 3=Trash, 5=My, 6=Share, 8=Projects, ...
FilesCount int
FoldersCount int
Access int
Shared bool
Updated string
}
// DavFile is a file row from the Files module.
type DavFile struct {
ID string
Title string
Size int64
Updated string
ViewURL string
}
// DavListing is the contents of one folder.
type DavListing struct {
Current DavFolder
Files []DavFile
Folders []DavFolder
}
// ListDavFolder returns the contents of a folder by id, which may be a
// symbolic root such as "@my". For "@root" use ListDavSections.
//
// Deprecated: use FileStore.List via Client.Files()/Client.FileStore.
func (c *Client) ListDavFolder(ctx context.Context, id string) (*DavListing, error) {
raw, err := c.getJSON(ctx, "/api/2.0/files/"+url.PathEscape(id))
if err != nil {
return nil, err
}
resp, err := responseField(raw, "response")
if err != nil {
return nil, err
}
// @root returns an array with a single blob; a normal folder returns an
// object. Normalize both.
if len(resp) > 0 && resp[0] == '[' {
var arr []*DavListing
if err := json.Unmarshal(resp, &arr); err != nil {
return nil, err
}
if len(arr) == 0 {
return &DavListing{}, nil
}
return arr[0], nil
}
var l DavListing
if err := json.Unmarshal(resp, &l); err != nil {
return nil, err
}
return &l, nil
}
// ListDavSections returns the virtual top-level sections shown by @root
// ("In projects", "My documents", "Shared with me", "Common", "Favorites",
// "Recent", "Trash"). Each is the `current` folder of one @root element.
func (c *Client) ListDavSections(ctx context.Context) ([]DavFolder, error) {
raw, err := c.getJSON(ctx, "/api/2.0/files/@root")
if err != nil {
return nil, err
}
resp, err := responseField(raw, "response")
if err != nil {
return nil, err
}
var arr []struct {
Current DavFolder `json:"current"`
}
if err := json.Unmarshal(resp, &arr); err != nil {
// Tolerate a non-array (single listing) response.
var single DavListing
if err2 := json.Unmarshal(resp, &single); err2 != nil {
return nil, err
}
return []DavFolder{single.Current}, nil
}
sections := make([]DavFolder, 0, len(arr))
for i := range arr {
sections = append(sections, arr[i].Current)
}
return sections, nil
}
// CreateDavFolder creates a folder titled title inside parentID.
//
// Deprecated: use FileStore.CreateFolder via Client.Files()/Client.FileStore.
func (c *Client) CreateDavFolder(ctx context.Context, parentID, title string) (*DavFolder, error) {
raw, err := c.postJSON(ctx, "/api/2.0/files/folder/"+url.PathEscape(parentID),
map[string]string{"title": title})
if err != nil {
return nil, err
}
var env struct {
Response *DavFolder `json:"response"`
}
if err := json.Unmarshal(raw, &env); err != nil {
return nil, err
}
if env.Response == nil {
return nil, fmt.Errorf("onlyoffice: empty create-folder response")
}
return env.Response, nil
}
// RenameDavFolder renames a folder.
//
// Deprecated: use FileStore.Rename via Client.Files()/Client.FileStore.
func (c *Client) RenameDavFolder(ctx context.Context, id, title string) error {
_, err := c.putJSON(ctx, "/api/2.0/files/folder/"+url.PathEscape(id),
map[string]string{"title": title})
return err
}
// RenameDavFile renames a file (title includes the extension).
//
// Deprecated: use FileStore.Rename via Client.Files()/Client.FileStore.
func (c *Client) RenameDavFile(ctx context.Context, id, title string) error {
_, err := c.putJSON(ctx, "/api/2.0/files/file/"+url.PathEscape(id),
map[string]string{"title": title})
return err
}
// MoveDavItems moves the given folders and/or files into destFolderID.
// The fileops API answers 200 with per-operation error strings even when
// nothing moves (e.g. missing permission), so the response is parsed and the
// first operation error is returned instead of a silent nil.
//
// Deprecated: use FileStore.Move via Client.Files()/Client.FileStore.
func (c *Client) MoveDavItems(ctx context.Context, folderIDs, fileIDs []string, destFolderID string) error {
raw, err := c.putJSON(ctx, "/api/2.0/files/fileops/move", map[string]any{
"folderIds": nums(folderIDs),
"fileIds": nums(fileIDs),
"destFolderId": num(destFolderID),
"resolveType": "Skip",
"holdResult": true,
})
if err != nil {
return err
}
return fileopsError(raw)
}
// CopyDavItems copies the given folders and/or files into destFolderID.
// Per-operation errors are surfaced like in MoveDavItems.
//
// Deprecated: use FileStore.Copy via Client.Files()/Client.FileStore.
func (c *Client) CopyDavItems(ctx context.Context, folderIDs, fileIDs []string, destFolderID string) error {
raw, err := c.putJSON(ctx, "/api/2.0/files/fileops/copy", map[string]any{
"folderIds": nums(folderIDs),
"fileIds": nums(fileIDs),
"destFolderId": num(destFolderID),
"conflictResolveType": "Skip",
"deleteAfter": true,
})
if err != nil {
return err
}
return fileopsError(raw)
}
// ListFileOps returns the currently active file operations
// (GET /api/2.0/files/fileops) for status polling.
func (c *Client) ListFileOps(ctx context.Context) ([]map[string]any, error) {
raw, err := c.getJSON(ctx, "/api/2.0/files/fileops")
if err != nil {
return nil, err
}
resp, err := responseField(raw, "response")
if err != nil {
return nil, err
}
if len(resp) == 0 || string(resp) == "null" {
return nil, nil
}
var ops []map[string]any
if err := json.Unmarshal(resp, &ops); err != nil {
return nil, err
}
return ops, nil
}
// fileopsError extracts per-operation "error" strings from a fileops/move or
// fileops/copy envelope. A 200 with error entries means nothing moved.
func fileopsError(raw json.RawMessage) error {
resp, err := responseField(raw, "response")
if err != nil {
return err
}
var ops []struct {
Error *string `json:"error"`
Finished *bool `json:"finished"`
Progress *int `json:"progress"`
}
if err := json.Unmarshal(resp, &ops); err != nil {
return nil // not an operations envelope — nothing to report
}
var errs []string
for _, op := range ops {
if op.Error != nil && *op.Error != "" {
errs = append(errs, *op.Error)
}
}
if len(errs) > 0 {
return fmt.Errorf("onlyoffice: fileops: %s", strings.Join(errs, "; "))
}
return nil
}
// DeleteDavItems deletes the given folders and/or files.
//
// Deprecated: use FileStore.Delete via Client.Files()/Client.FileStore.
func (c *Client) DeleteDavItems(ctx context.Context, folderIDs, fileIDs []string) error {
body := map[string]any{"DeleteAfter": true, "Immediately": true}
for _, id := range folderIDs {
if err := c.deleteDavItem(ctx, "/api/2.0/files/folder/"+url.PathEscape(id), body); err != nil {
return err
}
}
for _, id := range fileIDs {
if err := c.deleteDavItem(ctx, "/api/2.0/files/file/"+url.PathEscape(id), body); err != nil {
return err
}
}
return nil
}
// deleteDavItem deletes one item, retrying transient 429/502/503/504 answers
// through DoRetry like every other bulk path (deletes are idempotent).
func (c *Client) deleteDavItem(ctx context.Context, path string, body any) error {
return DoRetry(ctx, DefaultRetryPolicy(), func() error {
_, err := c.deleteJSON(ctx, path, body)
return err
})
}
// UploadDavFile uploads src (fileName) into folderID, streaming from src.
//
// Deprecated: use FileStore.Upload via Client.Files()/Client.FileStore.
func (c *Client) UploadDavFile(ctx context.Context, folderID, fileName string, src io.Reader) (*DavFile, error) {
raw, err := c.uploadReader(ctx, "/api/2.0/files/"+url.PathEscape(folderID)+"/upload", "file", fileName, src)
if err != nil {
return nil, err
}
var env struct {
Response *DavFile `json:"response"`
}
if err := json.Unmarshal(raw, &env); err != nil {
return nil, err
}
if env.Response == nil {
return nil, fmt.Errorf("onlyoffice: empty upload response")
}
return env.Response, nil
}
// DownloadDavFile streams the file identified by id to w, returning bytes
// copied. It shares the MinIO stale-S3 fallback with DownloadFile.
//
// Deprecated: use FileStore.Download via Client.Files()/Client.FileStore.
func (c *Client) DownloadDavFile(ctx context.Context, id string, w io.Writer) (int64, error) {
return c.DownloadFile(ctx, id, w)
}
// --- internal helpers -------------------------------------------------------
// deleteJSON performs an authenticated DELETE with an optional JSON body.
func (c *Client) deleteJSON(ctx context.Context, path string, body any) (json.RawMessage, error) {
var reader io.Reader
if body != nil {
buf, err := json.Marshal(body)
if err != nil {
return nil, err
}
reader = bytes.NewReader(buf)
}
auth, err := c.authHeader()
if err != nil {
return nil, err
}
req, err := http.NewRequestWithContext(ctx, http.MethodDelete, c.baseURL()+path, reader)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", auth)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
resp, err := c.client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
raw, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
if resp.StatusCode >= 400 {
return nil, statusError(resp.StatusCode, retryAfterOf(resp), "DELETE %s: %d %s", path, resp.StatusCode, truncate(string(raw), 400))
}
return raw, nil
}
// uploadReader uploads a stream to path under the given form field name.
func (c *Client) uploadReader(ctx context.Context, path, fieldName, fileName string, src io.Reader) (json.RawMessage, error) {
var buf bytes.Buffer
mw := multipart.NewWriter(&buf)
part, err := mw.CreateFormFile(fieldName, fileName)
if err != nil {
return nil, err
}
if _, err := io.Copy(part, src); err != nil {
return nil, err
}
if err := mw.Close(); err != nil {
return nil, err
}
auth, err := c.authHeader()
if err != nil {
return nil, err
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.baseURL()+path, &buf)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", auth)
req.Header.Set("Content-Type", mw.FormDataContentType())
req.Header.Set("Accept", "application/json")
resp, err := c.client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
raw, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
if resp.StatusCode >= 400 {
return nil, statusError(resp.StatusCode, retryAfterOf(resp), "upload %s: %d %s", path, resp.StatusCode, truncate(string(raw), 400))
}
return raw, nil
}
func nums(ids []string) []json.Number {
out := make([]json.Number, 0, len(ids))
for _, id := range ids {
if _, err := strconv.Atoi(id); err == nil {
out = append(out, json.Number(id))
}
}
return out
}
func num(id string) any {
if _, err := strconv.Atoi(id); err == nil {
return json.Number(id)
}
return id
}
// UnmarshalJSON decodes a folder from the portal envelope, including fields
// that the base FolderEntry omits (parentId, rootFolderType, access, ...).
func (f *DavFolder) UnmarshalJSON(b []byte) error {
var raw struct {
ID *json.Number `json:"id"`
Title *string `json:"title"`
ParentID *json.Number `json:"parentId"`
RootType *int `json:"rootFolderType"`
FilesCount *int `json:"filesCount"`
FoldersCount *int `json:"foldersCount"`
Access *int `json:"access"`
Shared *bool `json:"shared"`
Updated *string `json:"updated"`
}
if err := json.Unmarshal(b, &raw); err != nil {
return err
}
if raw.ID != nil {
f.ID = raw.ID.String()
}
if raw.Title != nil {
f.Title = *raw.Title
}
if raw.ParentID != nil {
f.ParentID = raw.ParentID.String()
}
if raw.RootType != nil {
f.RootType = *raw.RootType
}
if raw.FilesCount != nil {
f.FilesCount = *raw.FilesCount
}
if raw.FoldersCount != nil {
f.FoldersCount = *raw.FoldersCount
}
if raw.Access != nil {
f.Access = *raw.Access
}
if raw.Shared != nil {
f.Shared = *raw.Shared
}
if raw.Updated != nil {
f.Updated = *raw.Updated
}
return nil
}
// UnmarshalJSON decodes a file row, capturing size and timestamps.
func (f *DavFile) UnmarshalJSON(b []byte) error {
var raw struct {
ID *json.Number `json:"id"`
Title *string `json:"title"`
PureSize *int64 `json:"pureContentLength"`
SizeStr *string `json:"contentLength"`
Updated *string `json:"updated"`
ViewURL *string `json:"viewUrl"`
}
if err := json.Unmarshal(b, &raw); err != nil {
return err
}
if raw.ID != nil {
f.ID = raw.ID.String()
}
if raw.Title != nil {
f.Title = *raw.Title
}
if raw.PureSize != nil {
f.Size = *raw.PureSize
} else if raw.SizeStr != nil {
if n, err := strconv.ParseInt(strings.Fields(*raw.SizeStr)[0], 10, 64); err == nil {
f.Size = n
}
}
if raw.Updated != nil {
f.Updated = *raw.Updated
}
if raw.ViewURL != nil {
f.ViewURL = *raw.ViewURL
}
return nil
}
// ModTime parses the folder's updated timestamp.
func (f *DavFolder) ModTime() time.Time {
t, _ := time.Parse("2006-01-02T15:04:05.0000000-07:00", f.Updated)
return t
}
// ModTime parses the file's updated timestamp.
func (f *DavFile) ModTime() time.Time {
t, _ := time.Parse("2006-01-02T15:04:05.0000000-07:00", f.Updated)
return t
}
+285
View File
@@ -0,0 +1,285 @@
//go:build integration
package onlyoffice
import (
"bytes"
"context"
"strconv"
"testing"
"time"
)
// TestIntegrationFileStores runs the same operation set through the REST and
// DAV FileStore adapters against a throwaway project Documents folder: file
// create/upload/list/stat/download/move/copy/rename/delete and folder
// create/stat/list/rename/move/delete. Destructive — only run against
// instances you own.
//
// The Documents fileops API is asynchronous: a move/copy/delete is accepted
// immediately and becomes visible a moment later, so effects are polled.
func TestIntegrationFileStores(t *testing.T) {
c := liveClient(t)
t.Cleanup(func() { cleanupTestProjects(t, c) })
ctx := context.Background()
suffix := time.Now().UTC().Format("20060102-150405")
project, err := c.CreateProject(NewProjectRequest{
Title: testProjectPrefix + "store-" + suffix,
Description: "go-onlyoffice file store integration",
})
if err != nil {
t.Fatalf("CreateProject: %v", err)
}
if project.ID == nil {
t.Fatal("created project without id")
}
root, err := c.projectFolderID(ctx, strconv.Itoa(*project.ID))
if err != nil {
t.Fatalf("projectFolderID: %v", err)
}
for _, backend := range []string{ProviderREST, ProviderDAV} {
t.Run(backend, func(t *testing.T) {
testFileStoreOps(t, ctx, c, c.FileStore(backend), root, suffix)
})
}
}
func testFileStoreOps(t *testing.T, ctx context.Context, c *Client, store FileStore, root, suffix string) {
t.Helper()
content := []byte("file store " + store.Name() + " " + suffix + "\n")
src, err := store.CreateFolder(ctx, root, "fs-src-"+suffix)
if err != nil {
t.Fatalf("CreateFolder src: %v", err)
}
if src.Kind != Folder || src.ID == "" {
t.Fatalf("created src folder: %+v", src)
}
dst, err := store.CreateFolder(ctx, root, "fs-dst-"+suffix)
if err != nil {
t.Fatalf("CreateFolder dst: %v", err)
}
if dst.Kind != Folder || dst.ID == "" {
t.Fatalf("created dst folder: %+v", dst)
}
t.Cleanup(func() {
if err := c.DeleteDavItems(ctx, []string{src.ID, dst.ID}, nil); err != nil {
t.Logf("cleanup folders: %v", err)
}
})
up, err := store.Upload(ctx, src.ID, "doc-"+suffix+".txt", bytes.NewReader(content))
if err != nil {
t.Fatalf("Upload: %v", err)
}
if up.Kind != File || up.ID == "" {
t.Fatalf("uploaded entry: %+v", up)
}
if !waitEntry(ctx, store, src.ID, up.ID, 15*time.Second) {
t.Fatalf("uploaded %s not listed in src", up.ID)
}
st, err := store.Stat(ctx, up.ID)
if err != nil {
t.Fatalf("Stat: %v", err)
}
if st.ID != up.ID || st.Kind != File {
t.Fatalf("stat = %+v", st)
}
var buf bytes.Buffer
n, err := store.Download(ctx, up.ID, &buf)
if err != nil {
t.Fatalf("Download: %v", err)
}
if n != int64(len(content)) || !bytes.Equal(buf.Bytes(), content) {
t.Fatalf("download mismatch: got %d bytes %q want %d", n, buf.String(), len(content))
}
moveEventually(t, ctx, store, up.ID, dst.ID)
if !waitEntry(ctx, store, dst.ID, up.ID, 20*time.Second) {
t.Fatalf("moved file %s not in dst", up.ID)
}
copied := copyEventually(t, ctx, store, up.ID, src.ID, 20*time.Second)
if copied == nil {
t.Fatalf("no copy found in src after Copy")
}
newTitle := "renamed-" + suffix + ".txt"
renameEventually(t, ctx, store, up.ID, newTitle)
if err := store.Delete(ctx, []string{up.ID, copied.ID}); err != nil {
t.Fatalf("Delete: %v", err)
}
if !waitNoEntry(ctx, store, dst.ID, up.ID, 20*time.Second) {
t.Fatalf("file %s still present in dst after delete", up.ID)
}
if !waitNoEntry(ctx, store, src.ID, copied.ID, 20*time.Second) {
t.Fatalf("copy %s still present in src after delete", copied.ID)
}
// --- CRUD on the folders themselves, reusing the throwaway src/dst ---
// A child file lets us prove it survives the folder rename and move.
child, err := store.Upload(ctx, src.ID, "child-"+suffix+".txt", bytes.NewReader(content))
if err != nil {
t.Fatalf("Upload child: %v", err)
}
if !waitEntry(ctx, store, src.ID, child.ID, 15*time.Second) {
t.Fatalf("child %s not listed in src", child.ID)
}
fst, err := store.Stat(ctx, src.ID)
if err != nil {
t.Fatalf("Stat(folder): %v", err)
}
if fst.ID != src.ID || fst.Kind != Folder {
t.Fatalf("Stat(folder) = %+v", fst)
}
flist, err := store.List(ctx, src.ID)
if err != nil {
t.Fatalf("List(folder): %v", err)
}
if entryByID(flist, child.ID) == nil {
t.Fatalf("child %s not in List(src)", child.ID)
}
folderTitle := "renamed-folder-" + suffix
renameEventually(t, ctx, store, src.ID, folderTitle)
if e, err := store.Stat(ctx, src.ID); err != nil {
t.Fatalf("Stat(folder) after rename: %v", err)
} else if e.Kind != Folder || e.Title != folderTitle {
t.Fatalf("folder after rename = %+v, want title %q", e, folderTitle)
}
moveEventually(t, ctx, store, src.ID, dst.ID)
if !waitEntry(ctx, store, dst.ID, src.ID, 20*time.Second) {
t.Fatalf("moved folder %s not in dst %s", src.ID, dst.ID)
}
if !waitNoEntry(ctx, store, root, src.ID, 20*time.Second) {
t.Fatalf("folder %s still in root after move", src.ID)
}
if !waitEntry(ctx, store, src.ID, child.ID, 20*time.Second) {
t.Fatalf("child file %s lost after moving folder %s", child.ID, src.ID)
}
if err := store.Delete(ctx, []string{child.ID}); err != nil {
t.Fatalf("Delete(child): %v", err)
}
if err := store.Delete(ctx, []string{src.ID}); err != nil {
t.Fatalf("Delete(folder): %v", err)
}
if !waitNoEntry(ctx, store, dst.ID, src.ID, 20*time.Second) {
t.Fatalf("folder %s still present in dst after delete", src.ID)
}
if err := store.Delete(ctx, []string{dst.ID}); err != nil {
t.Fatalf("Delete(dst folder): %v", err)
}
if !waitNoEntry(ctx, store, root, dst.ID, 20*time.Second) {
t.Fatalf("dst folder %s still present in root after delete", dst.ID)
}
}
// moveEventually issues Move and retries while the operation is not visible yet
// (the fileops API accepts asynchronously and occasionally rejects a move that
// raced the just-finished upload).
func moveEventually(t *testing.T, ctx context.Context, store FileStore, id, dstID string) {
t.Helper()
var lastErr error
for attempt := 0; attempt < 5; attempt++ {
if lastErr = store.Move(ctx, []string{id}, dstID); lastErr == nil {
if waitEntry(ctx, store, dstID, id, 6*time.Second) {
return
}
}
time.Sleep(time.Second)
}
t.Fatalf("Move %s -> %s: %v", id, dstID, lastErr)
}
// copyEventually issues Copy and retries while the new copy is not visible yet
// (copy is accepted asynchronously, like move).
func copyEventually(t *testing.T, ctx context.Context, store FileStore, id, dstID string, d time.Duration) *Entry {
t.Helper()
var lastErr error
for attempt := 0; attempt < 5; attempt++ {
if lastErr = store.Copy(ctx, []string{id}, dstID); lastErr == nil {
if e := waitOtherFile(ctx, store, dstID, id, d); e != nil {
return e
}
}
time.Sleep(time.Second)
}
t.Fatalf("Copy %s -> %s: %v", id, dstID, lastErr)
return nil
}
func renameEventually(t *testing.T, ctx context.Context, store FileStore, id, title string) {
t.Helper()
var lastErr error
for attempt := 0; attempt < 5; attempt++ {
if lastErr = store.Rename(ctx, id, title); lastErr == nil {
if e, err := store.Stat(ctx, id); err == nil && e.Title == title {
return
}
}
time.Sleep(time.Second)
}
t.Fatalf("Rename %s -> %q: %v", id, title, lastErr)
}
func waitEntry(ctx context.Context, store FileStore, parentID, id string, d time.Duration) bool {
deadline := time.Now().Add(d)
for time.Now().Before(deadline) {
if list, err := store.List(ctx, parentID); err == nil && entryByID(list, id) != nil {
return true
}
time.Sleep(500 * time.Millisecond)
}
return false
}
func waitNoEntry(ctx context.Context, store FileStore, parentID, id string, d time.Duration) bool {
deadline := time.Now().Add(d)
for time.Now().Before(deadline) {
if list, err := store.List(ctx, parentID); err == nil && entryByID(list, id) == nil {
return true
}
time.Sleep(500 * time.Millisecond)
}
return false
}
func waitOtherFile(ctx context.Context, store FileStore, parentID, id string, d time.Duration) *Entry {
deadline := time.Now().Add(d)
for time.Now().Before(deadline) {
if list, err := store.List(ctx, parentID); err == nil {
if e := firstFileOtherThan(list, id); e != nil {
return e
}
}
time.Sleep(500 * time.Millisecond)
}
return nil
}
func entryByID(entries []Entry, id string) *Entry {
for i := range entries {
if entries[i].ID == id {
return &entries[i]
}
}
return nil
}
func firstFileOtherThan(entries []Entry, id string) *Entry {
for i := range entries {
if entries[i].Kind == File && entries[i].ID != id {
return &entries[i]
}
}
return nil
}
+262
View File
@@ -0,0 +1,262 @@
package onlyoffice
// Canonical file model and the backend-agnostic store interface. REST
// (files.go), WebDAV (files_webdav.go) and future backends (PostgreSQL,
// Elasticsearch) implement FileStore/Searcher so callers stop depending on a
// concrete transport. This file holds only types and pure conversions — no IO.
import (
"context"
"io"
"mime"
"path/filepath"
"strconv"
"strings"
"time"
)
// Kind distinguishes files from folders in the canonical model.
type Kind int
const (
File Kind = iota
Folder
)
// String renders the kind for logs and table output.
func (k Kind) String() string {
switch k {
case File:
return "file"
case Folder:
return "folder"
default:
return "unknown"
}
}
// Provider names for the FileStore adapters.
const (
ProviderREST = "rest"
ProviderDAV = "dav"
)
// Entry is the backend-independent representation of a document or folder.
// Fields that a backend cannot supply stay at their zero value.
type Entry struct {
ID string
ParentID string
Title string
Kind Kind
Size int64
MIME string
Created time.Time
Modified time.Time
// Updated is the backend-native timestamp string, when the backend exposes
// one. It lets list output round-trip the API value; Modified is the
// parsed form for logic.
Updated string
Version int
Provider string
// Folder-only counters. Zero for files and for backends that do not
// report them.
FilesCount int
FoldersCount int
}
// FileStore is the operation surface every file backend implements.
type FileStore interface {
Name() string
List(ctx context.Context, parentID string) ([]Entry, error)
Stat(ctx context.Context, id string) (Entry, error)
CreateFolder(ctx context.Context, parentID, title string) (Entry, error)
Upload(ctx context.Context, parentID, title string, r io.Reader) (Entry, error)
Download(ctx context.Context, id string, w io.Writer) (int64, error)
Move(ctx context.Context, ids []string, parentID string) error
Copy(ctx context.Context, ids []string, parentID string) error
Rename(ctx context.Context, id, title string) error
Delete(ctx context.Context, ids []string) error
}
// SearchQuery narrows a Searcher request. InContent asks the backend to match
// document bodies, not just titles. Substring switches title matching from the
// analyzer's whole-token match to a case-insensitive "*term*" wildcard and ANDs
// every whitespace-separated term (e.g. "rechnung 2025").
type SearchQuery struct {
Text string
InContent bool
FolderID string
Extensions []string
Limit int
Substring bool
}
// SearchHit is one Searcher result: the matching entry plus backend-specific
// ranking metadata.
type SearchHit struct {
Entry
Score float64
Highlight string
Path []string
}
// Searcher is the optional content/name search surface. Only some backends
// (for example Elasticsearch) provide it.
type Searcher interface {
Search(ctx context.Context, q SearchQuery) ([]SearchHit, error)
Name() string
}
// FileStore returns the adapter for a backend name: ProviderREST (default),
// ProviderDAV (alias "webdav") or the read-only SQL store (ProviderPG,
// ProviderMySQL and the aliases "pg"/"sql"). The SQL store is opened from the
// environment (ONLYOFFICE_DSN / ONLYOFFICE_PG_*); when it cannot be opened the
// returned store surfaces that error on every operation instead of returning
// nil. Use SQLFileStore when the open error itself is needed. Unknown or empty
// names select the REST backend. The composed facade (backend
// selection/fallback) lives on FileClient in filestore_facade.go.
func (c *Client) FileStore(backend string) FileStore {
switch strings.ToLower(strings.TrimSpace(backend)) {
case ProviderDAV, "webdav":
return &davStore{c: c}
case ProviderPG, ProviderMySQL, "pg", "sql":
s, err := c.SQLFileStore()
if err != nil {
return &errStore{name: strings.ToLower(strings.TrimSpace(backend)), err: err}
}
return s
default:
return &restStore{c: c}
}
}
// errStore is the FileStore placeholder returned when a backend cannot be
// opened (for example SQL without a DSN). Every operation returns the recorded
// error instead of panicking on a nil interface.
type errStore struct {
name string
err error
}
func (s *errStore) Name() string { return s.name }
func (s *errStore) List(context.Context, string) ([]Entry, error) { return nil, s.err }
func (s *errStore) Stat(context.Context, string) (Entry, error) { return Entry{}, s.err }
func (s *errStore) CreateFolder(context.Context, string, string) (Entry, error) {
return Entry{}, s.err
}
func (s *errStore) Upload(context.Context, string, string, io.Reader) (Entry, error) {
return Entry{}, s.err
}
func (s *errStore) Download(context.Context, string, io.Writer) (int64, error) {
return 0, s.err
}
func (s *errStore) Move(context.Context, []string, string) error { return s.err }
func (s *errStore) Copy(context.Context, []string, string) error { return s.err }
func (s *errStore) Rename(context.Context, string, string) error { return s.err }
func (s *errStore) Delete(context.Context, []string) error { return s.err }
// Files returns the composed file facade. The returned *FileClient implements
// FileStore, so callers that used Files() as the plain REST store keep working.
func (c *Client) Files() *FileClient { return c.newFileClient() }
// retryStoreOp runs one store operation under the shared deterministic
// transient-error policy (429/502/503/504).
func retryStoreOp(ctx context.Context, fn func() error) error {
return DoRetry(ctx, DefaultRetryPolicy(), fn)
}
// FileEntryToEntry converts a Files-module file row to the canonical model.
func FileEntryToEntry(f *FileEntry, provider string) Entry {
e := Entry{Kind: File, Provider: provider}
if f == nil {
return e
}
if f.ID != nil {
e.ID = f.ID.String()
}
e.ParentID = FileFolderID(f)
if f.Title != nil {
e.Title = *f.Title
}
if f.ContentLength != nil {
e.Size = parseContentLength(*f.ContentLength)
}
exst := ""
if f.FileExst != nil {
exst = *f.FileExst
}
e.MIME = mimeForTitle(e.Title, exst)
if f.Updated != nil {
e.Modified = *f.Updated
e.Updated = f.Updated.Format(time.RFC3339)
}
return e
}
// DavFileToEntry converts a WebDAV file row to the canonical model.
func DavFileToEntry(f DavFile, provider string) Entry {
return Entry{
ID: f.ID,
Title: f.Title,
Kind: File,
Size: f.Size,
MIME: mimeForTitle(f.Title, ""),
Modified: f.ModTime(),
Updated: f.Updated,
Provider: provider,
}
}
// DavFolderToEntry converts a WebDAV folder row to the canonical model.
func DavFolderToEntry(f DavFolder, provider string) Entry {
return Entry{
ID: f.ID,
ParentID: f.ParentID,
Title: f.Title,
Kind: Folder,
Modified: f.ModTime(),
Updated: f.Updated,
Provider: provider,
FilesCount: f.FilesCount,
FoldersCount: f.FoldersCount,
}
}
// parseContentLength reads the leading integer of an OnlyOffice contentLength
// string (the API sometimes appends a unit, e.g. "12345 b").
func parseContentLength(s string) int64 {
fields := strings.Fields(s)
if len(fields) == 0 {
return 0
}
n, err := strconv.ParseInt(fields[0], 10, 64)
if err != nil {
return 0
}
return n
}
// mimeForTitle derives a MIME type from an explicit extension or the title.
func mimeForTitle(title, exst string) string {
ext := strings.TrimSpace(exst)
if ext == "" {
ext = filepath.Ext(title)
}
if ext == "" {
return ""
}
if !strings.HasPrefix(ext, ".") {
ext = "." + ext
}
return mime.TypeByExtension(strings.ToLower(ext))
}
+193
View File
@@ -0,0 +1,193 @@
package onlyoffice
import (
"encoding/json"
"strings"
"testing"
"time"
)
func TestFileEntryToEntry(t *testing.T) {
id := json.Number("42")
title := "invoice.pdf"
exst := ".pdf"
size := "12345"
parent := json.Number("7")
updated := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
f := &FileEntry{
ID: &id,
Title: &title,
FileExst: &exst,
ContentLength: &size,
FolderID: &parent,
Updated: &updated,
}
e := FileEntryToEntry(f, ProviderREST)
if e.ID != "42" {
t.Errorf("ID = %q, want 42", e.ID)
}
if e.ParentID != "7" {
t.Errorf("ParentID = %q, want 7", e.ParentID)
}
if e.Title != title {
t.Errorf("Title = %q, want %q", e.Title, title)
}
if e.Kind != File {
t.Errorf("Kind = %v, want file", e.Kind)
}
if e.Size != 12345 {
t.Errorf("Size = %d, want 12345", e.Size)
}
if e.MIME != "application/pdf" {
t.Errorf("MIME = %q, want application/pdf", e.MIME)
}
if !e.Modified.Equal(updated) {
t.Errorf("Modified = %v, want %v", e.Modified, updated)
}
if e.Provider != ProviderREST {
t.Errorf("Provider = %q, want %q", e.Provider, ProviderREST)
}
}
func TestFileEntryToEntryNil(t *testing.T) {
e := FileEntryToEntry(nil, ProviderDAV)
if e.Kind != File {
t.Errorf("Kind = %v, want file", e.Kind)
}
if e.ID != "" || e.Title != "" {
t.Errorf("nil entry should be empty: %+v", e)
}
if e.Provider != ProviderDAV {
t.Errorf("Provider = %q, want %q", e.Provider, ProviderDAV)
}
}
func TestFileEntryToEntrySizeFormats(t *testing.T) {
cases := map[string]int64{
"12345": 12345,
"12345 b": 12345,
"0": 0,
"": 0,
"notanum": 0,
}
for in, want := range cases {
got := parseContentLength(in)
if got != want {
t.Errorf("parseContentLength(%q) = %d, want %d", in, got, want)
}
}
}
func TestDavFileToEntry(t *testing.T) {
f := DavFile{
ID: "9",
Title: "note.txt",
Size: 10,
Updated: "2026-01-02T03:04:05.0000000+01:00",
}
e := DavFileToEntry(f, ProviderDAV)
if e.ID != "9" || e.Title != "note.txt" {
t.Errorf("identity mismatch: %+v", e)
}
if e.Kind != File {
t.Errorf("Kind = %v, want file", e.Kind)
}
if e.Size != 10 {
t.Errorf("Size = %d, want 10", e.Size)
}
if !strings.HasPrefix(e.MIME, "text/plain") {
t.Errorf("MIME = %q, want text/plain*", e.MIME)
}
if e.Modified.IsZero() {
t.Error("Modified not parsed")
}
if e.Provider != ProviderDAV {
t.Errorf("Provider = %q, want %q", e.Provider, ProviderDAV)
}
}
func TestDavFolderToEntry(t *testing.T) {
f := DavFolder{
ID: "5",
Title: "inbox",
ParentID: "1",
Updated: "2026-01-02T03:04:05.0000000+01:00",
}
e := DavFolderToEntry(f, ProviderDAV)
if e.ID != "5" || e.Title != "inbox" || e.ParentID != "1" {
t.Errorf("identity mismatch: %+v", e)
}
if e.Kind != Folder {
t.Errorf("Kind = %v, want folder", e.Kind)
}
if e.MIME != "" {
t.Errorf("folder MIME = %q, want empty", e.MIME)
}
if e.Modified.IsZero() {
t.Error("Modified not parsed")
}
}
func TestEntriesFromFolderMap(t *testing.T) {
m := map[string]any{
"files": []any{
map[string]any{"id": float64(42), "title": "a.pdf", "pureContentLength": float64(7)},
},
"folders": []any{
map[string]any{"id": float64(7), "title": "sub", "parentId": float64(1)},
},
}
entries, err := entriesFromFolderMap(m, ProviderREST)
if err != nil {
t.Fatalf("entriesFromFolderMap: %v", err)
}
if len(entries) != 2 {
t.Fatalf("got %d entries, want 2: %+v", len(entries), entries)
}
byID := map[string]Entry{}
for _, e := range entries {
byID[e.ID] = e
}
if got := byID["42"]; got.Kind != File || got.Size != 7 || got.Title != "a.pdf" {
t.Errorf("file entry = %+v", got)
}
if got := byID["7"]; got.Kind != Folder || got.ParentID != "1" || got.Title != "sub" {
t.Errorf("folder entry = %+v", got)
}
}
func TestEntriesFromFolderMapNil(t *testing.T) {
entries, err := entriesFromFolderMap(nil, ProviderREST)
if err != nil || entries != nil {
t.Fatalf("got %v, %v; want nil, nil", entries, err)
}
}
func TestKindString(t *testing.T) {
if File.String() != "file" || Folder.String() != "folder" {
t.Errorf("kind strings: %q %q", File.String(), Folder.String())
}
if Kind(9).String() != "unknown" {
t.Errorf("unknown kind = %q", Kind(9).String())
}
}
func TestClientFileStoreSelection(t *testing.T) {
c := NewClient(Credentials{})
if got := c.FileStore(ProviderDAV).Name(); got != ProviderDAV {
t.Errorf("FileStore(dav).Name() = %q", got)
}
if got := c.FileStore("webdav").Name(); got != ProviderDAV {
t.Errorf("FileStore(webdav).Name() = %q", got)
}
if got := c.FileStore(ProviderREST).Name(); got != ProviderREST {
t.Errorf("FileStore(rest).Name() = %q", got)
}
if got := c.FileStore("").Name(); got != ProviderREST {
t.Errorf("FileStore(\"\").Name() = %q", got)
}
if got := c.Files().Name(); got != ProviderREST {
t.Errorf("Files().Name() = %q", got)
}
}
+189
View File
@@ -0,0 +1,189 @@
package onlyoffice
// davStore implements FileStore on top of the Documents/WebDAV methods in
// files_webdav.go. The Documents fileops calls need folder and file ids
// separated, so ids are classified through Stat before move/copy/rename/delete.
import (
"bytes"
"context"
"fmt"
"io"
)
// davStore is a FileStore over the WebDAV-oriented Documents API.
type davStore struct{ c *Client }
// Name reports the backend name.
func (s *davStore) Name() string { return ProviderDAV }
// List returns the files and folders directly below parentID.
func (s *davStore) List(ctx context.Context, parentID string) ([]Entry, error) {
var out []Entry
err := retryStoreOp(ctx, func() error {
l, err := s.c.ListDavFolder(ctx, parentID)
if err != nil {
return err
}
entries := make([]Entry, 0, len(l.Folders)+len(l.Files))
for _, f := range l.Folders {
entries = append(entries, DavFolderToEntry(f, ProviderDAV))
}
for _, f := range l.Files {
entries = append(entries, DavFileToEntry(f, ProviderDAV))
}
out = entries
return nil
})
return out, err
}
// Stat resolves a folder or file entry by id. A folder answers ListDavFolder
// with its own metadata in Current; otherwise the file metadata API is used.
func (s *davStore) Stat(ctx context.Context, id string) (Entry, error) {
return s.stat(ctx, id)
}
// CreateFolder creates a subfolder under parentID.
func (s *davStore) CreateFolder(ctx context.Context, parentID, title string) (Entry, error) {
var out Entry
err := retryStoreOp(ctx, func() error {
f, err := s.c.CreateDavFolder(ctx, parentID, title)
if err != nil {
return err
}
if f == nil {
return fmt.Errorf("onlyoffice: dav store: empty create-folder response")
}
out = DavFolderToEntry(*f, ProviderDAV)
return nil
})
return out, err
}
// Upload streams r into parentID as title. The reader is buffered once so a
// retry re-sends the same bytes instead of an exhausted stream.
func (s *davStore) Upload(ctx context.Context, parentID, title string, r io.Reader) (Entry, error) {
data, err := io.ReadAll(r)
if err != nil {
return Entry{}, err
}
var out Entry
err = retryStoreOp(ctx, func() error {
f, err := s.c.UploadDavFile(ctx, parentID, title, bytes.NewReader(data))
if err != nil {
return err
}
if f == nil {
return fmt.Errorf("onlyoffice: dav store: empty upload response")
}
out = DavFileToEntry(*f, ProviderDAV)
return nil
})
return out, err
}
// Download streams the file bytes into w.
func (s *davStore) Download(ctx context.Context, id string, w io.Writer) (int64, error) {
var n int64
err := retryStoreOp(ctx, func() error {
var e error
n, e = s.c.DownloadDavFile(ctx, id, w)
return e
})
return n, err
}
// Move moves ids into parentID, splitting folders from files.
func (s *davStore) Move(ctx context.Context, ids []string, parentID string) error {
folders, files, err := s.split(ctx, ids)
if err != nil {
return err
}
if len(folders) == 0 && len(files) == 0 {
return nil
}
return retryStoreOp(ctx, func() error {
return s.c.MoveDavItems(ctx, folders, files, parentID)
})
}
// Copy copies ids into parentID, splitting folders from files.
func (s *davStore) Copy(ctx context.Context, ids []string, parentID string) error {
folders, files, err := s.split(ctx, ids)
if err != nil {
return err
}
if len(folders) == 0 && len(files) == 0 {
return nil
}
return retryStoreOp(ctx, func() error {
return s.c.CopyDavItems(ctx, folders, files, parentID)
})
}
// Rename renames a folder or file.
func (s *davStore) Rename(ctx context.Context, id, title string) error {
e, err := s.stat(ctx, id)
if err != nil {
return err
}
return retryStoreOp(ctx, func() error {
if e.Kind == Folder {
return s.c.RenameDavFolder(ctx, id, title)
}
return s.c.RenameDavFile(ctx, id, title)
})
}
// Delete removes ids, splitting folders from files.
func (s *davStore) Delete(ctx context.Context, ids []string) error {
folders, files, err := s.split(ctx, ids)
if err != nil {
return err
}
if len(folders) == 0 && len(files) == 0 {
return nil
}
return retryStoreOp(ctx, func() error {
return s.c.DeleteDavItems(ctx, folders, files)
})
}
// stat resolves a single id to a folder or file Entry.
func (s *davStore) stat(ctx context.Context, id string) (Entry, error) {
var out Entry
err := retryStoreOp(ctx, func() error {
if l, err := s.c.ListDavFolder(ctx, id); err == nil {
if l != nil && l.Current.ID != "" && l.Current.ID == id {
out = DavFolderToEntry(l.Current, ProviderDAV)
return nil
}
} else if Transient(err) {
return err
}
f, err := s.c.GetFile(ctx, id)
if err != nil {
return err
}
out = FileEntryToEntry(f, ProviderDAV)
return nil
})
return out, err
}
// split classifies ids into folder and file id lists.
func (s *davStore) split(ctx context.Context, ids []string) (folders, files []string, err error) {
for _, id := range ids {
e, err := s.stat(ctx, id)
if err != nil {
return nil, nil, err
}
if e.Kind == Folder {
folders = append(folders, id)
} else {
files = append(files, id)
}
}
return folders, files, nil
}
+325
View File
@@ -0,0 +1,325 @@
package onlyoffice
// Elasticsearch backend of the unified file client (epic #34, F3 #37).
//
// OnlyOffice full-text search runs on Elasticsearch (index `files_file`, NEST
// client on the server). The REST endpoint GET /api/2.0/files/@search/{query}
// only searches file names in the database, so content search needs a direct
// ES query. The live server is Elasticsearch 7.16.3; the request shape below
// is plain REST and stays stdlib-only, matching the repo's no-extra-deps rule.
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"regexp"
"strconv"
"strings"
"time"
)
// The canonical model (Kind, Entry, SearchQuery, SearchHit, Searcher) lives in
// filestore_core.go (F1 #35).
const (
defaultESIndex = "files_file"
defaultESLimit = 20
maxESLimit = 1000
maxESResponseSize = 8 << 20
)
// ESConfig configures the direct Elasticsearch searcher.
type ESConfig struct {
URL string // scheme://host:port of the ES HTTP endpoint
Index string // index name, default files_file
Tenant string // tenantId filter, empty means all tenants
}
// ESConfigFromEnv reads ONLYOFFICE_ES_URL, ONLYOFFICE_ES_INDEX (default
// files_file) and ONLYOFFICE_TENANT. The library never loads dotfiles — the
// CLI does that.
func ESConfigFromEnv() ESConfig {
return ESConfig{
URL: strings.TrimRight(strings.TrimSpace(os.Getenv("ONLYOFFICE_ES_URL")), "/"),
Index: firstNonEmpty(os.Getenv("ONLYOFFICE_ES_INDEX"), defaultESIndex),
Tenant: strings.TrimSpace(os.Getenv("ONLYOFFICE_TENANT")),
}
}
// ESSearcher queries OnlyOffice's Elasticsearch index directly for file name
// and document content.
type ESSearcher struct {
cfg ESConfig
http *http.Client
}
// NewESSearcher returns a searcher for the OnlyOffice Elasticsearch index.
// The URL is required; an empty index falls back to files_file.
func NewESSearcher(cfg ESConfig) (*ESSearcher, error) {
if strings.TrimSpace(cfg.URL) == "" {
return nil, fmt.Errorf("onlyoffice: elasticsearch URL is empty (set ONLYOFFICE_ES_URL)")
}
cfg.URL = strings.TrimRight(cfg.URL, "/")
if cfg.Index == "" {
cfg.Index = defaultESIndex
}
return &ESSearcher{cfg: cfg, http: &http.Client{Timeout: 30 * time.Second}}, nil
}
// Name implements Searcher.
func (s *ESSearcher) Name() string { return "elasticsearch" }
// Search runs a multi_match over title (and, when q.InContent is set,
// document.attachment.content), filtered by tenant and optional folder.
func (s *ESSearcher) Search(ctx context.Context, q SearchQuery) ([]SearchHit, error) {
q.Text = strings.TrimSpace(q.Text)
if q.Text == "" {
return nil, fmt.Errorf("onlyoffice: empty search query")
}
body, err := json.Marshal(esSearchRequest(q, s.cfg.Tenant))
if err != nil {
return nil, fmt.Errorf("onlyoffice: build elasticsearch query: %w", err)
}
endpoint := s.cfg.URL + "/" + s.cfg.Index + "/_search"
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(body))
if err != nil {
return nil, err
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
resp, err := s.http.Do(req)
if err != nil {
return nil, fmt.Errorf("onlyoffice: elasticsearch search: %w", err)
}
defer resp.Body.Close()
raw, err := io.ReadAll(io.LimitReader(resp.Body, maxESResponseSize))
if err != nil {
return nil, err
}
if resp.StatusCode >= 400 {
return nil, fmt.Errorf("onlyoffice: elasticsearch search: %d %s", resp.StatusCode, truncate(string(raw), 400))
}
return parseESSearchResponse(raw)
}
// esSearchRequest builds the ES query body. Pure, so it is unit-tested.
func esSearchRequest(q SearchQuery, tenant string) esRequest {
limit := q.Limit
if limit <= 0 {
limit = defaultESLimit
}
if limit > maxESLimit {
limit = maxESLimit
}
fields := []string{"title^2"}
if q.InContent {
fields = append(fields, "document.attachment.content")
}
var must []esClause
if q.Substring {
for _, term := range strings.Fields(strings.ToLower(q.Text)) {
if term = escapeWildcard(term); term != "" {
must = append(must, esClause{Wildcard: map[string]any{"title": "*" + term + "*"}})
}
}
}
if len(must) == 0 {
must = []esClause{{MultiMatch: &esMultiMatch{Query: q.Text, Fields: fields}}}
}
var filter []esClause
if t := strings.TrimSpace(tenant); t != "" {
filter = append(filter, esClause{Term: map[string]any{"tenantId": numericOrString(t)}})
}
if f := strings.TrimSpace(q.FolderID); f != "" {
// folders is an ES nested field; a plain term on folders.folderId would
// not match. The stored Folders list holds every ancestor id, so
// filtering by a project root id scopes to its whole subtree.
filter = append(filter, esClause{Nested: &esNested{
Path: "folders",
Query: esNestedTerm{Term: map[string]any{"folders.folderId": f}},
}})
}
for _, ext := range normalizeExtensions(q.Extensions) {
filter = append(filter, esClause{Wildcard: map[string]any{"title": "*." + ext}})
}
highlightFields := map[string]struct{}{"title": {}}
if q.InContent {
highlightFields["document.attachment.content"] = struct{}{}
}
return esRequest{
Size: limit,
Source: []string{"id", "title", "folders"},
Query: esQuery{Bool: esBool{Must: must, Filter: filter}},
Highlight: esHighlight{PreTags: []string{"<em>"}, PostTags: []string{"</em>"}, Fields: highlightFields},
}
}
// normalizeExtensions lowercases, trims leading dots and drops empties.
func normalizeExtensions(exts []string) []string {
out := make([]string, 0, len(exts))
seen := map[string]bool{}
for _, e := range exts {
e = strings.ToLower(strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(e), ".")))
if e == "" || seen[e] {
continue
}
seen[e] = true
out = append(out, e)
}
return out
}
// numericOrString keeps an integer-looking filter value numeric (tenantId is
// a long) and leaves anything else as a string (folderId is a text token).
func numericOrString(s string) any {
if n, err := strconv.ParseInt(s, 10, 64); err == nil {
return n
}
return s
}
// esRequest is the subset of the ES query DSL this client emits.
type esRequest struct {
Size int `json:"size"`
Source []string `json:"_source"`
Query esQuery `json:"query"`
Highlight esHighlight `json:"highlight"`
}
type esQuery struct {
Bool esBool `json:"bool"`
}
type esBool struct {
Must []esClause `json:"must,omitempty"`
Filter []esClause `json:"filter,omitempty"`
}
type esClause struct {
MultiMatch *esMultiMatch `json:"multi_match,omitempty"`
Term map[string]any `json:"term,omitempty"`
Terms map[string]any `json:"terms,omitempty"`
Wildcard map[string]any `json:"wildcard,omitempty"`
Nested *esNested `json:"nested,omitempty"`
}
type esNested struct {
Path string `json:"path"`
Query esNestedTerm `json:"query"`
}
type esNestedTerm struct {
Term map[string]any `json:"term,omitempty"`
}
// escapeWildcard strips ES wildcard metacharacters from a user term so a query
// cannot inject wildcard syntax. Pure, so it is unit-tested.
func escapeWildcard(s string) string {
return strings.NewReplacer("*", "", "?", "", `\`, "").Replace(s)
}
type esMultiMatch struct {
Query string `json:"query"`
Fields []string `json:"fields"`
}
type esHighlight struct {
PreTags []string `json:"pre_tags,omitempty"`
PostTags []string `json:"post_tags,omitempty"`
Fields map[string]struct{} `json:"fields"`
}
// esResponse is the subset of an ES search response we consume.
type esResponse struct {
Took int `json:"took"`
Hits struct {
Total struct {
Value int `json:"value"`
Relation string `json:"relation"`
} `json:"total"`
Hits []esResponseHit `json:"hits"`
} `json:"hits"`
}
type esResponseHit struct {
ID string `json:"_id"`
Score float64 `json:"_score"`
Source struct {
ID int `json:"id"`
Title string `json:"title"`
Folders []struct {
FolderID string `json:"folderId"`
ID int `json:"id"`
} `json:"folders"`
} `json:"_source"`
Highlight map[string][]string `json:"highlight"`
}
// parseESSearchResponse converts an ES search response into SearchHit values.
// Pure, so it is unit-tested.
func parseESSearchResponse(raw []byte) ([]SearchHit, error) {
var r esResponse
if err := json.Unmarshal(raw, &r); err != nil {
return nil, fmt.Errorf("onlyoffice: decode elasticsearch response: %w", err)
}
hits := make([]SearchHit, 0, len(r.Hits.Hits))
for _, h := range r.Hits.Hits {
id := strconv.Itoa(h.Source.ID)
if h.Source.ID == 0 {
id = h.ID
}
// Folders is the ancestor breadcrumb in root → leaf order, so the last
// entry is the immediate parent (the previous "first" value was the
// project root, which made every result look like it lived in #522).
var parent string
path := make([]string, 0, len(h.Source.Folders))
for _, f := range h.Source.Folders {
if strings.TrimSpace(f.FolderID) == "" {
continue
}
path = append(path, f.FolderID)
}
if len(path) > 0 {
parent = path[len(path)-1]
}
hits = append(hits, SearchHit{
Entry: Entry{
ID: id,
ParentID: parent,
Title: h.Source.Title,
Kind: File,
Provider: "elasticsearch",
},
Score: h.Score,
Highlight: esHighlightText(h.Highlight),
Path: path,
})
}
return hits, nil
}
var esHighlightTag = regexp.MustCompile(`</?em[^>]*>`)
// esHighlightText flattens a highlight map into one plain-text snippet,
// preferring the content fragment over the title. It covers both the
// OnlyOffice content field and the own-index "content" field.
func esHighlightText(hl map[string][]string) string {
for _, key := range []string{"document.attachment.content", "content", "title"} {
frags := hl[key]
if len(frags) == 0 {
continue
}
clean := make([]string, 0, len(frags))
for _, f := range frags {
clean = append(clean, esHighlightTag.ReplaceAllString(f, ""))
}
return strings.Join(clean, " … ")
}
return ""
}
+226
View File
@@ -0,0 +1,226 @@
//go:build integration
package onlyoffice
import (
"context"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
"time"
"github.com/xuri/excelize/v2"
)
// Default known fixtures for TestIntegrationESFacadeUsesES on the live index.
const (
defaultESTestQuery = "Rechnung_986-2025.pdf"
defaultESTestSubstring = "rechnung 2025"
defaultESTestFolder = "522"
)
// TestIntegrationESFacadeUsesES proves that the public search path — `oo search`
// and Client.Files().Search() — really runs against the OnlyOffice
// Elasticsearch backend and not the REST @search endpoint, which only looks at
// file names in the database and is not a Searcher at all (see
// docs/elasticsearch.md). It pins the concrete backend and checks that a known
// document comes back with a non-empty id and folder path.
//
// Requires ONLYOFFICE_ES_URL only — the query never touches the REST API, so no
// OnlyOffice credentials are needed. Skips when it is missing. The fixture is
// overridable with ONLYOFFICE_ES_TEST_QUERY, ONLYOFFICE_ES_TEST_TITLE,
// ONLYOFFICE_ES_TEST_SUBSTRING and ONLYOFFICE_ES_TEST_FOLDER.
func TestIntegrationESFacadeUsesES(t *testing.T) {
esURL := strings.TrimSpace(os.Getenv("ONLYOFFICE_ES_URL"))
if esURL == "" {
t.Skip("ONLYOFFICE_ES_URL not set — skipping Elasticsearch integration test")
}
query := firstNonEmpty(strings.TrimSpace(os.Getenv("ONLYOFFICE_ES_TEST_QUERY")), defaultESTestQuery)
wantTitle := firstNonEmpty(strings.TrimSpace(os.Getenv("ONLYOFFICE_ES_TEST_TITLE")), query)
substring := firstNonEmpty(strings.TrimSpace(os.Getenv("ONLYOFFICE_ES_TEST_SUBSTRING")), defaultESTestSubstring)
folder := firstNonEmpty(strings.TrimSpace(os.Getenv("ONLYOFFICE_ES_TEST_FOLDER")), defaultESTestFolder)
c := NewClient(Credentials{})
searcher, err := c.Files().Search()
if err != nil {
t.Fatalf("Files().Search(): %v", err)
}
if got := searcher.Name(); got != ProviderES {
t.Fatalf("searcher.Name() = %q, want %q (REST @search is not a Searcher)", got, ProviderES)
}
if _, ok := searcher.(*ESSearcher); !ok {
t.Fatalf("searcher = %T, want *ESSearcher (ES backend, not REST)", searcher)
}
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
// Known file name: multi_match over title, as `oo search <file>` does.
start := time.Now()
hits, err := searcher.Search(ctx, SearchQuery{Text: query, Limit: 20})
if err != nil {
t.Fatalf("Search(%q): %v", query, err)
}
t.Logf("ES facade query %q: %d hits in %s", query, len(hits), time.Since(start))
known := findHitByTitle(hits, wantTitle)
if known == nil {
t.Fatalf("query %q returned %d hits, none titled %q", query, len(hits), wantTitle)
}
if strings.TrimSpace(known.ID) == "" {
t.Errorf("hit %q has empty id", known.Title)
}
if len(known.Path) == 0 {
t.Errorf("hit %q has empty path", known.Title)
}
if known.Provider != ProviderES {
t.Errorf("hit provider = %q, want %q", known.Provider, ProviderES)
}
// Substring + folder subtree, as `oo search <terms> --substring --folder N`
// does: wildcard terms ANDed together, scoped to the folder's subtree.
start = time.Now()
subHits, err := searcher.Search(ctx, SearchQuery{Text: substring, Substring: true, FolderID: folder, Limit: 200})
if err != nil {
t.Fatalf("substring Search(%q, folder %s): %v", substring, folder, err)
}
t.Logf("ES facade substring %q folder %s: %d hits in %s", substring, folder, len(subHits), time.Since(start))
if len(subHits) == 0 {
t.Fatalf("substring query %q in folder %s returned no hits", substring, folder)
}
if findHitByTitle(subHits, wantTitle) == nil {
t.Errorf("substring query %q in folder %s did not return %q", substring, folder, wantTitle)
}
}
// findHitByTitle returns the first hit whose title matches, case-insensitively.
func findHitByTitle(hits []SearchHit, title string) *SearchHit {
for i := range hits {
if strings.EqualFold(strings.TrimSpace(hits[i].Title), title) {
return &hits[i]
}
}
return nil
}
// TestIntegrationESSearch uploads a throwaway workbook and verifies that the
// direct Elasticsearch search finds it by file name and by content.
//
// The content index (document.attachment.content) is only populated for Office
// formats (docx/xlsx/pptx), so the fixture is an xlsx whose cell carries a
// unique token. Requires ONLYOFFICE_ES_URL (a reachable ES endpoint — in the
// current setup a tunnel to the ES inside the OnlyOffice VM, see
// docs/elasticsearch.md) plus the regular REST credentials for the upload.
// Skips when either is missing.
func TestIntegrationESSearch(t *testing.T) {
esURL := strings.TrimSpace(os.Getenv("ONLYOFFICE_ES_URL"))
if esURL == "" {
t.Skip("ONLYOFFICE_ES_URL not set — skipping Elasticsearch integration test")
}
c := liveClient(t)
t.Cleanup(func() { cleanupTestProjects(t, c) })
stamp := time.Now().UTC().Format("20060102-150405")
nameToken := "goesname" + stamp
contentToken := "goescontent" + stamp
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
defer cancel()
project, err := c.CreateProject(NewProjectRequest{
Title: testProjectPrefix + "es-" + stamp,
Description: "go-onlyoffice elasticsearch integration",
})
if err != nil {
t.Fatalf("CreateProject: %v", err)
}
if project.ID == nil {
t.Fatal("created project without id")
}
pid := strconv.Itoa(*project.ID)
title := nameToken + ".xlsx"
localPath := filepath.Join(t.TempDir(), title)
book := excelize.NewFile()
if err := book.SetCellValue("Sheet1", "A1", "OnlyOffice Elasticsearch content fixture "+contentToken); err != nil {
t.Fatalf("SetCellValue: %v", err)
}
if err := book.SaveAs(localPath); err != nil {
t.Fatalf("SaveAs: %v", err)
}
entry, err := c.UploadProjectFile(ctx, pid, localPath)
if err != nil {
t.Fatalf("UploadProjectFile: %v", err)
}
fileID := strconv.Itoa(int(FileEntryNumericID(entry)))
if fileID == "0" {
t.Fatalf("upload returned no file id: %+v", entry)
}
es, err := NewESSearcher(ESConfig{
URL: esURL,
Index: os.Getenv("ONLYOFFICE_ES_INDEX"),
Tenant: os.Getenv("ONLYOFFICE_TENANT"),
})
if err != nil {
t.Fatalf("NewESSearcher: %v", err)
}
// Indexing is asynchronous on the server; poll until the file shows up.
// The server's title analyzer splits on whitespace, so the name query is
// the full file name token (including extension), as a user would type it.
nameHit := waitForHit(t, ctx, es, SearchQuery{Text: title}, fileID)
if nameHit.Title != title {
t.Errorf("name hit title = %q, want %q", nameHit.Title, title)
}
contentHit := waitForHit(t, ctx, es, SearchQuery{Text: contentToken, InContent: true}, fileID)
if contentHit.Highlight == "" {
t.Error("content hit has no highlight fragment")
}
if !strings.Contains(contentHit.Title, nameToken) {
t.Errorf("content hit title = %q, want the uploaded workbook", contentHit.Title)
}
// The content token is absent from the title, so a name-only search must
// not return the file — this proves the content field is really queried.
if hits := searchQuiet(t, es, SearchQuery{Text: contentToken}); len(hits) != 0 {
t.Errorf("name-only search for content token returned %d hits, want 0", len(hits))
}
}
// waitForHit polls ES until the file with fileID appears and returns that hit.
func waitForHit(t *testing.T, ctx context.Context, s *ESSearcher, q SearchQuery, fileID string) SearchHit {
t.Helper()
var lastErr error
for {
hits, err := s.Search(ctx, q)
if err != nil {
lastErr = err
} else {
for _, h := range hits {
if h.ID == fileID {
return h
}
}
}
select {
case <-ctx.Done():
t.Fatalf("search %q: file %s not indexed in time (last err: %v)", q.Text, fileID, lastErr)
case <-time.After(3 * time.Second):
}
}
}
func searchQuiet(t *testing.T, s *ESSearcher, q SearchQuery) []SearchHit {
t.Helper()
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
hits, err := s.Search(ctx, q)
if err != nil {
t.Fatalf("Search(%q): %v", q.Text, err)
}
return hits
}
+217
View File
@@ -0,0 +1,217 @@
package onlyoffice
import (
"encoding/json"
"reflect"
"testing"
)
func TestESSearchRequestNameOnly(t *testing.T) {
got := esSearchRequest(SearchQuery{Text: "Rechnung"}, "1")
if got.Size != defaultESLimit {
t.Errorf("size = %d, want %d", got.Size, defaultESLimit)
}
if !reflect.DeepEqual(got.Source, []string{"id", "title", "folders"}) {
t.Errorf("_source = %v", got.Source)
}
if len(got.Query.Bool.Must) != 1 || got.Query.Bool.Must[0].MultiMatch == nil {
t.Fatalf("must = %+v, want one multi_match", got.Query.Bool.Must)
}
mm := got.Query.Bool.Must[0].MultiMatch
if mm.Query != "Rechnung" {
t.Errorf("query = %q", mm.Query)
}
if !reflect.DeepEqual(mm.Fields, []string{"title^2"}) {
t.Errorf("fields = %v, want title only", mm.Fields)
}
if _, ok := got.Highlight.Fields["document.attachment.content"]; ok {
t.Error("content highlight present without InContent")
}
if _, ok := got.Highlight.Fields["title"]; !ok {
t.Error("title highlight missing")
}
if len(got.Query.Bool.Filter) != 1 || got.Query.Bool.Filter[0].Term["tenantId"] != int64(1) {
t.Errorf("tenant filter = %+v, want numeric tenantId=1", got.Query.Bool.Filter)
}
}
func TestESSearchRequestContentFields(t *testing.T) {
got := esSearchRequest(SearchQuery{Text: "Mahnung", InContent: true}, "")
mm := got.Query.Bool.Must[0].MultiMatch
want := []string{"title^2", "document.attachment.content"}
if !reflect.DeepEqual(mm.Fields, want) {
t.Errorf("fields = %v, want %v", mm.Fields, want)
}
if _, ok := got.Highlight.Fields["document.attachment.content"]; !ok {
t.Error("content highlight missing with InContent")
}
if len(got.Query.Bool.Filter) != 0 {
t.Errorf("filter = %+v, want none without tenant/folder", got.Query.Bool.Filter)
}
}
func TestESSearchRequestFiltersAndLimit(t *testing.T) {
got := esSearchRequest(SearchQuery{
Text: "Storchen",
FolderID: "649",
Extensions: []string{".PDF", "pdf", "docx"},
Limit: 5000,
}, "42")
if got.Size != maxESLimit {
t.Errorf("size = %d, want cap %d", got.Size, maxESLimit)
}
var tenant, folder, wildcards int
for _, f := range got.Query.Bool.Filter {
switch {
case f.Term != nil && f.Term["tenantId"] != nil:
tenant++
case f.Nested != nil:
folder++
if f.Nested.Path != "folders" || f.Nested.Query.Term["folders.folderId"] != "649" {
t.Errorf("folder filter = %+v, want nested folders term 649", f.Nested)
}
case f.Wildcard != nil:
wildcards++
}
}
if tenant != 1 || folder != 1 {
t.Errorf("term filters tenant=%d folder=%d, want 1 each", tenant, folder)
}
if wildcards != 2 {
t.Errorf("wildcard filters = %d, want deduped PDF+docx", wildcards)
}
}
func TestESSearchRequestSubstringAndsTerms(t *testing.T) {
got := esSearchRequest(SearchQuery{Text: "Rechnung 2025", Substring: true, FolderID: "522"}, "")
if got.Query.Bool.Must[0].MultiMatch != nil {
t.Fatalf("substring must not use multi_match: %+v", got.Query.Bool.Must)
}
if len(got.Query.Bool.Must) != 2 {
t.Fatalf("must = %+v, want two ANDed wildcard terms", got.Query.Bool.Must)
}
want := []string{"*rechnung*", "*2025*"}
for i, m := range got.Query.Bool.Must {
if m.Wildcard == nil || m.Wildcard["title"] != want[i] {
t.Errorf("must[%d] = %+v, want title wildcard %q", i, m, want[i])
}
}
if len(got.Query.Bool.Filter) != 1 || got.Query.Bool.Filter[0].Nested == nil {
t.Errorf("folder filter = %+v, want nested", got.Query.Bool.Filter)
}
}
func TestEscapeWildcard(t *testing.T) {
cases := map[string]string{"*rechnung*": "rechnung", "a?b\\c": "abc", "plain": "plain"}
for in, want := range cases {
if got := escapeWildcard(in); got != want {
t.Errorf("escapeWildcard(%q) = %q, want %q", in, got, want)
}
}
}
func TestESSearchRequestRejectsEmptyTextAtSearch(t *testing.T) {
s, err := NewESSearcher(ESConfig{URL: "http://localhost:9200"})
if err != nil {
t.Fatalf("NewESSearcher: %v", err)
}
if _, err := s.Search(t.Context(), SearchQuery{Text: " "}); err == nil {
t.Error("empty query: want error")
}
}
func TestNewESSearcherRequiresURL(t *testing.T) {
if _, err := NewESSearcher(ESConfig{}); err == nil {
t.Error("empty URL: want error")
}
s, err := NewESSearcher(ESConfig{URL: "http://es:9200/"})
if err != nil {
t.Fatalf("NewESSearcher: %v", err)
}
if s.cfg.Index != defaultESIndex {
t.Errorf("index = %q, want %q", s.cfg.Index, defaultESIndex)
}
if s.cfg.URL != "http://es:9200" {
t.Errorf("url = %q, want trimmed", s.cfg.URL)
}
if s.Name() != "elasticsearch" {
t.Errorf("Name() = %q", s.Name())
}
}
func TestNormalizeExtensions(t *testing.T) {
got := normalizeExtensions([]string{" .PDF ", "pdf", "", "xlsx"})
want := []string{"pdf", "xlsx"}
if !reflect.DeepEqual(got, want) {
t.Errorf("normalizeExtensions = %v, want %v", got, want)
}
}
func TestParseESSearchResponse(t *testing.T) {
raw := []byte(`{
"took": 12,
"hits": {
"total": {"value": 2, "relation": "eq"},
"hits": [
{
"_id": "2395",
"_score": 7.31,
"_source": {"id": 2395, "title": "Rechnung-4711.pdf",
"folders": [{"folderId": "438", "id": 0}, {"folderId": "11", "id": 0}]},
"highlight": {
"title": ["<em>Rechnung</em>-4711.pdf"],
"document.attachment.content": ["… Zahlung der <em>Rechnung</em> …"]
}
},
{
"_id": "2318",
"_score": 6.02,
"_source": {"id": 2318, "title": "Mahnung.pdf", "folders": []},
"highlight": {"title": ["<em>Mahnung</em>.pdf"]}
}
]
}
}`)
hits, err := parseESSearchResponse(raw)
if err != nil {
t.Fatalf("parseESSearchResponse: %v", err)
}
if len(hits) != 2 {
t.Fatalf("hits = %d, want 2", len(hits))
}
h0 := hits[0]
if h0.ID != "2395" || h0.Title != "Rechnung-4711.pdf" || h0.Kind != File {
t.Errorf("hit0 entry = %+v", h0.Entry)
}
// folders is root → leaf; the immediate parent is the last entry.
if h0.ParentID != "11" || !reflect.DeepEqual(h0.Path, []string{"438", "11"}) {
t.Errorf("hit0 path = %v parent = %q, want parent 11", h0.Path, h0.ParentID)
}
if h0.Score != 7.31 {
t.Errorf("hit0 score = %v", h0.Score)
}
if h0.Highlight != "… Zahlung der Rechnung …" {
t.Errorf("hit0 highlight = %q, want content fragment", h0.Highlight)
}
if hits[1].Highlight != "Mahnung.pdf" {
t.Errorf("hit1 highlight = %q, want title without tags", hits[1].Highlight)
}
if hits[1].ParentID != "" || len(hits[1].Path) != 0 {
t.Errorf("hit1 path = %v", hits[1].Path)
}
}
func TestESSearchRequestJSONShape(t *testing.T) {
got := esSearchRequest(SearchQuery{Text: "Rechnung", InContent: true}, "1")
b, err := json.Marshal(got)
if err != nil {
t.Fatalf("marshal: %v", err)
}
var back map[string]any
if err := json.Unmarshal(b, &back); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if _, ok := back["query"].(map[string]any)["bool"]; !ok {
t.Errorf("query.bool missing: %s", b)
}
}
+336
View File
@@ -0,0 +1,336 @@
package onlyoffice
// Own full-text index (epic #34, F6 #42).
//
// The OnlyOffice Elasticsearch index (files_file) only holds extracted content
// for Office formats. FileUtility.CanIndex gates extraction by the server
// setting files.index.formats, whose default is ".pptx|.xlsx|.docx", so PDFs
// are indexed by name only. Instead of patching the server (risky: lost on
// upgrade, forces a full reindex) this file implements a second, independent
// index (default oo_docs_text) that our own pipeline fills from
// internal/docpipe (pdftotext + OCR). The OnlyOffice index is never touched.
//
// See docs/elasticsearch.md for the decision and the trade-offs.
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strings"
"time"
)
const defaultESTextIndex = "oo_docs_text"
// ESTextConfig configures the own full-text index.
type ESTextConfig struct {
URL string // scheme://host:port of the ES HTTP endpoint
Index string // index name, default oo_docs_text
Tenant string // reserved for future multi-tenant data; unused for now
}
// ESTextConfigFromEnv reads ONLYOFFICE_ES_URL and ONLYOFFICE_ES_TEXT_INDEX
// (default oo_docs_text). The library never loads dotfiles — the CLI does that.
func ESTextConfigFromEnv() ESTextConfig {
return ESTextConfig{
URL: strings.TrimRight(strings.TrimSpace(os.Getenv("ONLYOFFICE_ES_URL")), "/"),
Index: firstNonEmpty(os.Getenv("ONLYOFFICE_ES_TEXT_INDEX"), defaultESTextIndex),
Tenant: strings.TrimSpace(os.Getenv("ONLYOFFICE_TENANT")),
}
}
// TextDoc is one document in the own full-text index. It is keyed by the
// OnlyOffice file id so hits map straight back to Documents entries.
type TextDoc struct {
ID string `json:"id"`
Title string `json:"title"`
FolderID string `json:"folder,omitempty"`
Ext string `json:"ext,omitempty"`
Content string `json:"content"`
}
// TextIndex is the storage/search surface for locally extracted document text.
// It complements Searcher: ESSearcher reads OnlyOffice's index, ESTextIndex
// reads ours.
type TextIndex interface {
Put(ctx context.Context, docs []TextDoc) error
Delete(ctx context.Context, ids []string) error
Search(ctx context.Context, q SearchQuery) ([]SearchHit, error)
Name() string
}
// ESTextIndex is a TextIndex (and Searcher) over a dedicated Elasticsearch
// index filled by TextIndexer.
type ESTextIndex struct {
cfg ESTextConfig
http *http.Client
}
var (
_ TextIndex = (*ESTextIndex)(nil)
_ Searcher = (*ESTextIndex)(nil)
)
// NewESTextIndex returns a searcher/writer for the own full-text index. The URL
// is required; an empty index falls back to oo_docs_text.
func NewESTextIndex(cfg ESTextConfig) (*ESTextIndex, error) {
if strings.TrimSpace(cfg.URL) == "" {
return nil, fmt.Errorf("onlyoffice: elasticsearch URL is empty (set ONLYOFFICE_ES_URL)")
}
cfg.URL = strings.TrimRight(cfg.URL, "/")
if cfg.Index == "" {
cfg.Index = defaultESTextIndex
}
return &ESTextIndex{cfg: cfg, http: &http.Client{Timeout: 120 * time.Second}}, nil
}
// Name implements Searcher and TextIndex.
func (x *ESTextIndex) Name() string { return "es-text" }
// Index returns the configured index name.
func (x *ESTextIndex) Index() string { return x.cfg.Index }
// esTextMapping pins explicit types: content must stay a plain text field (no
// keyword subfield) and folder/ext stay exact keywords for filters.
const esTextMapping = `{
"mappings": {
"properties": {
"id": {"type": "keyword"},
"title": {"type": "text", "fields": {"keyword": {"type": "keyword", "ignore_above": 512}}},
"folder": {"type": "keyword"},
"ext": {"type": "keyword"},
"content": {"type": "text"}
}
}
}`
// Ensure creates the index with the explicit mapping. A missing index is
// created; an already existing one is left untouched.
func (x *ESTextIndex) Ensure(ctx context.Context) error {
status, raw, err := x.do(ctx, http.MethodPut, "/"+x.cfg.Index, []byte(esTextMapping), "application/json")
if err != nil {
return err
}
if status == http.StatusOK {
return nil
}
if status == http.StatusBadRequest && bytes.Contains(raw, []byte("resource_already_exists_exception")) {
return nil
}
return fmt.Errorf("onlyoffice: create text index %s: %d %s", x.cfg.Index, status, truncate(string(raw), 300))
}
// Put upserts documents via the bulk API and refreshes so they are immediately
// searchable.
func (x *ESTextIndex) Put(ctx context.Context, docs []TextDoc) error {
if len(docs) == 0 {
return nil
}
status, raw, err := x.do(ctx, http.MethodPost, "/"+x.cfg.Index+"/_bulk?refresh=true", esTextBulkBody(docs), "application/x-ndjson")
if err != nil {
return err
}
if status >= 400 {
return fmt.Errorf("onlyoffice: bulk index %s: %d %s", x.cfg.Index, status, truncate(string(raw), 400))
}
var res esBulkResponse
if err := json.Unmarshal(raw, &res); err != nil {
return fmt.Errorf("onlyoffice: decode bulk response: %w", err)
}
if !res.Errors {
return nil
}
return fmt.Errorf("onlyoffice: bulk index %s: %s", x.cfg.Index, res.firstError())
}
// Delete removes documents by OnlyOffice file id. A missing index means there
// is nothing to delete.
func (x *ESTextIndex) Delete(ctx context.Context, ids []string) error {
if len(ids) == 0 {
return nil
}
body, err := json.Marshal(map[string]any{"query": map[string]any{"terms": map[string]any{"id": ids}}})
if err != nil {
return err
}
status, raw, err := x.do(ctx, http.MethodPost, "/"+x.cfg.Index+"/_delete_by_query?refresh=true", body, "application/json")
if err != nil {
return err
}
if status == http.StatusNotFound {
return nil
}
if status >= 400 {
return fmt.Errorf("onlyoffice: delete from %s: %d %s", x.cfg.Index, status, truncate(string(raw), 400))
}
return nil
}
// Search runs a multi_match over title (boosted) and content, with optional
// folder and extension filters. A missing index yields no hits, not an error.
func (x *ESTextIndex) Search(ctx context.Context, q SearchQuery) ([]SearchHit, error) {
q.Text = strings.TrimSpace(q.Text)
if q.Text == "" {
return nil, fmt.Errorf("onlyoffice: empty search query")
}
body, err := json.Marshal(esTextSearchRequest(q))
if err != nil {
return nil, fmt.Errorf("onlyoffice: build elasticsearch query: %w", err)
}
status, raw, err := x.do(ctx, http.MethodPost, "/"+x.cfg.Index+"/_search", body, "application/json")
if err != nil {
return nil, err
}
if status == http.StatusNotFound {
return nil, nil
}
if status >= 400 {
return nil, fmt.Errorf("onlyoffice: search %s: %d %s", x.cfg.Index, status, truncate(string(raw), 400))
}
return parseESTextResponse(raw)
}
// do sends one request and returns the status and body (bounded). The caller
// decides which statuses are errors.
func (x *ESTextIndex) do(ctx context.Context, method, path string, body []byte, contentType string) (int, []byte, error) {
var r io.Reader
if body != nil {
r = bytes.NewReader(body)
}
req, err := http.NewRequestWithContext(ctx, method, x.cfg.URL+path, r)
if err != nil {
return 0, nil, err
}
req.Header.Set("Accept", "application/json")
if contentType != "" {
req.Header.Set("Content-Type", contentType)
}
resp, err := x.http.Do(req)
if err != nil {
return 0, nil, fmt.Errorf("onlyoffice: elasticsearch %s: %w", method, err)
}
defer resp.Body.Close()
raw, err := io.ReadAll(io.LimitReader(resp.Body, maxESResponseSize))
if err != nil {
return resp.StatusCode, nil, err
}
return resp.StatusCode, raw, nil
}
// esTextBulkBody renders the NDJSON bulk payload. Pure, so it is unit-tested.
func esTextBulkBody(docs []TextDoc) []byte {
var b bytes.Buffer
enc := json.NewEncoder(&b)
enc.SetEscapeHTML(false)
for _, d := range docs {
_ = enc.Encode(map[string]any{"index": map[string]any{"_id": d.ID}})
_ = enc.Encode(d)
}
return b.Bytes()
}
// esTextSearchRequest builds the own-index query. Pure, so it is unit-tested.
func esTextSearchRequest(q SearchQuery) esRequest {
limit := q.Limit
if limit <= 0 {
limit = defaultESLimit
}
if limit > maxESLimit {
limit = maxESLimit
}
fields := []string{"title^2", "content"}
must := []esClause{{MultiMatch: &esMultiMatch{Query: q.Text, Fields: fields}}}
var filter []esClause
if f := strings.TrimSpace(q.FolderID); f != "" {
filter = append(filter, esClause{Term: map[string]any{"folder": f}})
}
if exts := normalizeExtensions(q.Extensions); len(exts) > 0 {
filter = append(filter, esClause{Terms: map[string]any{"ext": exts}})
}
return esRequest{
Size: limit,
Source: []string{"id", "title", "folder", "ext"},
Query: esQuery{Bool: esBool{Must: must, Filter: filter}},
Highlight: esHighlight{PreTags: []string{"<em>"}, PostTags: []string{"</em>"}, Fields: map[string]struct{}{"title": {}, "content": {}}},
}
}
// esBulkResponse is the subset of an ES bulk response we consume.
type esBulkResponse struct {
Errors bool `json:"errors"`
Items []map[string]struct {
ID string `json:"_id"`
Status int `json:"status"`
Error *struct {
Type string `json:"type"`
Reason string `json:"reason"`
} `json:"error"`
} `json:"items"`
}
// firstError returns a compact description of the first failed bulk item.
func (r esBulkResponse) firstError() string {
for _, item := range r.Items {
for op, res := range item {
if res.Error != nil {
return fmt.Sprintf("%s %s: %s %s", op, res.ID, res.Error.Type, res.Error.Reason)
}
}
}
return "unknown bulk error"
}
// esTextResponse is the subset of an own-index search response we consume.
type esTextResponse struct {
Hits struct {
Total struct {
Value int `json:"value"`
} `json:"total"`
Hits []struct {
ID string `json:"_id"`
Score float64 `json:"_score"`
Source TextDoc `json:"_source"`
HL map[string][]string `json:"highlight"`
} `json:"hits"`
} `json:"hits"`
}
// parseESTextResponse converts an own-index search response into SearchHit
// values. Pure, so it is unit-tested.
func parseESTextResponse(raw []byte) ([]SearchHit, error) {
var r esTextResponse
if err := json.Unmarshal(raw, &r); err != nil {
return nil, fmt.Errorf("onlyoffice: decode elasticsearch response: %w", err)
}
hits := make([]SearchHit, 0, len(r.Hits.Hits))
for _, h := range r.Hits.Hits {
id := h.Source.ID
if id == "" {
id = h.ID
}
parent := h.Source.FolderID
var path []string
if parent != "" {
path = []string{parent}
}
hits = append(hits, SearchHit{
Entry: Entry{
ID: id,
ParentID: parent,
Title: h.Source.Title,
Kind: File,
Provider: "es-text",
},
Score: h.Score,
Highlight: esHighlightText(h.HL),
Path: path,
})
}
return hits, nil
}
+158
View File
@@ -0,0 +1,158 @@
//go:build integration
package onlyoffice
import (
"context"
"net/http"
"os"
"strings"
"testing"
"time"
"github.com/eslider/go-onlyoffice/internal/docpipe"
)
// TestIntegrationESTextIndex verifies the own full-text index end to end
// against a live Elasticsearch: create the index with its mapping, index a
// document, find it by content (and reject it via a folder filter and after
// deletion), then drop the throwaway index.
//
// Requires ONLYOFFICE_ES_URL (a reachable ES endpoint — in the current setup a
// tunnel to the ES inside the OnlyOffice VM, see docs/elasticsearch.md). It
// does not need OnlyOffice credentials because no file is downloaded: the
// TextIndexer write path is covered by unit tests with a fake extractor.
func TestIntegrationESTextIndex(t *testing.T) {
esURL := strings.TrimSpace(os.Getenv("ONLYOFFICE_ES_URL"))
if esURL == "" {
t.Skip("ONLYOFFICE_ES_URL not set — skipping Elasticsearch integration test")
}
stamp := time.Now().UTC().Format("20060102150405")
idx, err := NewESTextIndex(ESTextConfig{URL: esURL, Index: "oo_docs_text_it_" + stamp})
if err != nil {
t.Fatalf("NewESTextIndex: %v", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
t.Cleanup(func() {
cleanupCtx, done := context.WithTimeout(context.Background(), 30*time.Second)
defer done()
_, _, _ = idx.do(cleanupCtx, http.MethodDelete, "/"+idx.Index(), nil, "")
})
if err := idx.Ensure(ctx); err != nil {
t.Fatalf("Ensure: %v", err)
}
// Ensure is idempotent.
if err := idx.Ensure(ctx); err != nil {
t.Fatalf("Ensure (second): %v", err)
}
token := "gotes" + stamp
doc := TextDoc{
ID: "3578",
Title: "2026-07-28-S1021-acme-rechnung.pdf",
FolderID: "634",
Ext: "pdf",
Content: "Begleitzettel SGB XI — Rechnung " + token,
}
if err := idx.Put(ctx, []TextDoc{doc}); err != nil {
t.Fatalf("Put: %v", err)
}
hits, err := idx.Search(ctx, SearchQuery{Text: token})
if err != nil {
t.Fatalf("Search: %v", err)
}
if len(hits) != 1 || hits[0].ID != "3578" {
t.Fatalf("content search hits = %+v, want doc 3578", hits)
}
if !strings.Contains(hits[0].Highlight, token) {
t.Errorf("highlight = %q, want token", hits[0].Highlight)
}
if hits, err := idx.Search(ctx, SearchQuery{Text: token, FolderID: "999"}); err != nil {
t.Fatalf("Search with folder filter: %v", err)
} else if len(hits) != 0 {
t.Errorf("folder filter returned %d hits, want 0", len(hits))
}
if hits, err := idx.Search(ctx, SearchQuery{Text: token, Extensions: []string{"docx"}}); err != nil {
t.Fatalf("Search with ext filter: %v", err)
} else if len(hits) != 0 {
t.Errorf("ext filter returned %d hits, want 0", len(hits))
}
if err := idx.Delete(ctx, []string{"3578"}); err != nil {
t.Fatalf("Delete: %v", err)
}
if hits, err := idx.Search(ctx, SearchQuery{Text: token}); err != nil {
t.Fatalf("Search after delete: %v", err)
} else if len(hits) != 0 {
t.Errorf("after delete search returned %d hits, want 0", len(hits))
}
}
// TestIntegrationESTextIndexPDFAttachment indexes testdata/pdf-with-attachment.pdf
// through the real pipeline (TextIndexer + docpipe: pdfdetach + pdftotext) and
// verifies that text living only in the embedded attachment is searchable.
//
// Requires ONLYOFFICE_ES_URL plus poppler (pdfdetach/pdftotext). No OnlyOffice
// credentials are needed: a fixture FileStore serves the PDF bytes.
func TestIntegrationESTextIndexPDFAttachment(t *testing.T) {
esURL := strings.TrimSpace(os.Getenv("ONLYOFFICE_ES_URL"))
if esURL == "" {
t.Skip("ONLYOFFICE_ES_URL not set — skipping Elasticsearch integration test")
}
if docpipe.LookPath().PDFDetach == "" {
t.Skip("pdfdetach not on PATH — skipping PDF attachment integration test")
}
pdf, err := os.ReadFile("testdata/pdf-with-attachment.pdf")
if err != nil {
t.Fatalf("read fixture: %v", err)
}
stamp := time.Now().UTC().Format("20060102150405")
idx, err := NewESTextIndex(ESTextConfig{URL: esURL, Index: "oo_docs_text_it_att_" + stamp})
if err != nil {
t.Fatalf("NewESTextIndex: %v", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
t.Cleanup(func() {
cleanupCtx, done := context.WithTimeout(context.Background(), 30*time.Second)
defer done()
_, _, _ = idx.do(cleanupCtx, http.MethodDelete, "/"+idx.Index(), nil, "")
})
if err := idx.Ensure(ctx); err != nil {
t.Fatalf("Ensure: %v", err)
}
store := &textFakeStore{files: map[string][]byte{"9001": pdf}}
ix := NewTextIndexer(store, idx)
res, err := ix.IndexEntries(ctx, []Entry{{ID: "9001", Title: "scan.pdf", ParentID: "777", Kind: File}}, IndexOptions{MinChars: 1})
if err != nil {
t.Fatalf("IndexEntries: %v", err)
}
if res.Indexed != 1 || res.Failed != 0 {
t.Fatalf("result = %+v, want one indexed doc", res)
}
// Token appears only inside the embedded goo-note.txt attachment.
hits, err := idx.Search(ctx, SearchQuery{Text: "gooattachmenttoken"})
if err != nil {
t.Fatalf("Search attachment token: %v", err)
}
if len(hits) != 1 || hits[0].ID != "9001" {
t.Fatalf("attachment-token hits = %+v, want doc 9001", hits)
}
if !strings.Contains(hits[0].Highlight, "gooattachmenttoken") {
t.Errorf("highlight = %q, want attachment token", hits[0].Highlight)
}
// Body text is indexed as before.
if hits, err := idx.Search(ctx, SearchQuery{Text: "goobodytoken"}); err != nil {
t.Fatalf("Search body token: %v", err)
} else if len(hits) != 1 {
t.Errorf("body-token hits = %d, want 1", len(hits))
}
}
+275
View File
@@ -0,0 +1,275 @@
package onlyoffice
import (
"context"
"fmt"
"io"
"os"
"reflect"
"strings"
"testing"
)
func TestESTextSearchRequestShape(t *testing.T) {
got := esTextSearchRequest(SearchQuery{
Text: "S1021",
FolderID: "634",
Extensions: []string{".PDF", "pdf"},
Limit: 5,
})
if got.Size != 5 {
t.Errorf("size = %d, want 5", got.Size)
}
mm := got.Query.Bool.Must[0].MultiMatch
if mm == nil || !reflect.DeepEqual(mm.Fields, []string{"title^2", "content"}) {
t.Fatalf("multi_match = %+v, want title^2 + content", mm)
}
if _, ok := got.Highlight.Fields["content"]; !ok {
t.Error("content highlight missing")
}
if _, ok := got.Highlight.Fields["title"]; !ok {
t.Error("title highlight missing")
}
var folder, exts int
for _, f := range got.Query.Bool.Filter {
switch {
case f.Term != nil && f.Term["folder"] != nil:
folder++
if f.Term["folder"] != "634" {
t.Errorf("folder term = %+v", f.Term)
}
case f.Terms != nil:
exts++
if !reflect.DeepEqual(f.Terms["ext"], []string{"pdf"}) {
t.Errorf("ext terms = %+v, want deduped pdf", f.Terms)
}
}
}
if folder != 1 || exts != 1 {
t.Errorf("filters folder=%d exts=%d, want 1 each", folder, exts)
}
}
func TestESTextBulkBody(t *testing.T) {
body := esTextBulkBody([]TextDoc{
{ID: "3578", Title: "S1021.pdf", FolderID: "634", Ext: "pdf", Content: "Begleitzettel <S1021> & mehr"},
{ID: "3579", Title: "S1023.pdf", Ext: "pdf", Content: "x"},
})
lines := strings.Split(strings.TrimRight(string(body), "\n"), "\n")
if len(lines) != 4 {
t.Fatalf("bulk body has %d lines, want 4:\n%s", len(lines), body)
}
if !strings.Contains(lines[0], `"index"`) || !strings.Contains(lines[0], `"_id":"3578"`) {
t.Errorf("action line = %q", lines[0])
}
if !strings.Contains(lines[1], `"content":"Begleitzettel <S1021> & mehr"`) {
t.Errorf("source line should keep HTML unescaped, got %q", lines[1])
}
if !strings.Contains(lines[2], `"_id":"3579"`) {
t.Errorf("second action line = %q", lines[2])
}
}
func TestParseESTextResponse(t *testing.T) {
raw := []byte(`{
"hits": {
"total": {"value": 1, "relation": "eq"},
"hits": [
{
"_id": "3578",
"_score": 3.21,
"_source": {"id": "3578", "title": "2026-07-28-S1021-acme-rechnung.pdf", "folder": "634", "ext": "pdf"},
"highlight": {"content": ["Begleitzettel … <em>S1021</em> …"]}
}
]
}
}`)
hits, err := parseESTextResponse(raw)
if err != nil {
t.Fatalf("parseESTextResponse: %v", err)
}
if len(hits) != 1 {
t.Fatalf("hits = %d, want 1", len(hits))
}
h := hits[0]
if h.ID != "3578" || h.Title != "2026-07-28-S1021-acme-rechnung.pdf" || h.Kind != File {
t.Errorf("entry = %+v", h.Entry)
}
if h.ParentID != "634" || !reflect.DeepEqual(h.Path, []string{"634"}) {
t.Errorf("path = %v parent = %q", h.Path, h.ParentID)
}
if h.Provider != "es-text" {
t.Errorf("provider = %q", h.Provider)
}
if h.Highlight != "Begleitzettel … S1021 …" {
t.Errorf("highlight = %q, want tags stripped", h.Highlight)
}
}
func TestNewESTextIndexDefaults(t *testing.T) {
if _, err := NewESTextIndex(ESTextConfig{}); err == nil {
t.Error("empty URL: want error")
}
x, err := NewESTextIndex(ESTextConfig{URL: "http://es:9200/"})
if err != nil {
t.Fatalf("NewESTextIndex: %v", err)
}
if x.Index() != defaultESTextIndex {
t.Errorf("index = %q, want %q", x.Index(), defaultESTextIndex)
}
if x.cfg.URL != "http://es:9200" {
t.Errorf("url = %q, want trimmed", x.cfg.URL)
}
if x.Name() != "es-text" {
t.Errorf("Name() = %q", x.Name())
}
}
func TestESTextConfigFromEnvIndexDefault(t *testing.T) {
t.Setenv("ONLYOFFICE_ES_URL", "http://es:9200/")
t.Setenv("ONLYOFFICE_ES_TEXT_INDEX", "")
cfg := ESTextConfigFromEnv()
if cfg.Index != defaultESTextIndex {
t.Errorf("index = %q, want %q", cfg.Index, defaultESTextIndex)
}
}
func TestTextIndexerIndexEntries(t *testing.T) {
store := &textFakeStore{
files: map[string][]byte{"1": []byte("PDFBYTES")},
}
idx := &textFakeIndex{}
ix := NewTextIndexer(store, idx)
ix.Extractor = textFakeExtractor{prefix: "TEXT "}
res, err := ix.IndexEntries(context.Background(), []Entry{
{ID: "1", Title: "Rechnung.PDF", ParentID: "649", Kind: File},
{ID: "2", Title: "Tabelle.xlsx", ParentID: "649", Kind: File},
{ID: "3", Title: "Unterordner", Kind: Folder},
}, IndexOptions{})
if err != nil {
t.Fatalf("IndexEntries: %v", err)
}
if res.Scanned != 3 || res.Indexed != 1 || res.Skipped != 2 || res.Failed != 0 {
t.Errorf("result = %+v, want scanned=3 indexed=1 skipped=2 failed=0", res)
}
if len(idx.docs) != 1 {
t.Fatalf("indexed docs = %d, want 1", len(idx.docs))
}
got := idx.docs[0]
want := TextDoc{ID: "1", Title: "Rechnung.PDF", FolderID: "649", Ext: "pdf", Content: "TEXT PDFBYTES"}
if !reflect.DeepEqual(got, want) {
t.Errorf("doc = %+v, want %+v", got, want)
}
}
func TestTextIndexerRecordsExtractionFailure(t *testing.T) {
store := &textFakeStore{files: map[string][]byte{"1": []byte("x")}}
idx := &textFakeIndex{}
ix := NewTextIndexer(store, idx)
ix.Extractor = textFailingExtractor{}
res, err := ix.IndexEntries(context.Background(), []Entry{{ID: "1", Title: "a.pdf", Kind: File}}, IndexOptions{})
if err != nil {
t.Fatalf("IndexEntries: %v", err)
}
if res.Indexed != 0 || res.Failed != 1 || len(res.Errors) != 1 {
t.Errorf("result = %+v, want one failure recorded", res)
}
}
func TestTextIndexerPlanFolder(t *testing.T) {
store := &textFakeStore{dirs: map[string][]Entry{
"root": {
{ID: "10", Title: "a.pdf", Kind: File},
{ID: "11", Title: "sub", Kind: Folder},
},
"11": {
{ID: "12", Title: "b.PDF", Kind: File},
{ID: "13", Title: "c.xlsx", Kind: File},
},
}}
ix := NewTextIndexer(store, &textFakeIndex{})
flat, err := ix.PlanFolder(context.Background(), "root", IndexOptions{})
if err != nil {
t.Fatalf("PlanFolder: %v", err)
}
if len(flat) != 1 || flat[0].ID != "10" {
t.Errorf("flat plan = %+v, want only a.pdf", flat)
}
deep, err := ix.PlanFolder(context.Background(), "root", IndexOptions{Recursive: true, Limit: 10})
if err != nil {
t.Fatalf("PlanFolder recursive: %v", err)
}
if len(deep) != 2 {
t.Errorf("recursive plan = %d entries, want 2", len(deep))
}
}
// --- fakes -----------------------------------------------------------------
type textFakeStore struct {
dirs map[string][]Entry
files map[string][]byte
stat map[string]Entry
}
func (f *textFakeStore) Name() string { return "fake" }
func (f *textFakeStore) List(_ context.Context, parentID string) ([]Entry, error) {
return f.dirs[parentID], nil
}
func (f *textFakeStore) Stat(_ context.Context, id string) (Entry, error) {
if e, ok := f.stat[id]; ok {
return e, nil
}
return Entry{}, fmt.Errorf("not found: %s", id)
}
func (f *textFakeStore) Download(_ context.Context, id string, w io.Writer) (int64, error) {
b, ok := f.files[id]
if !ok {
return 0, fmt.Errorf("no bytes for %s", id)
}
n, err := w.Write(b)
return int64(n), err
}
func (f *textFakeStore) CreateFolder(context.Context, string, string) (Entry, error) {
return Entry{}, nil
}
func (f *textFakeStore) Upload(context.Context, string, string, io.Reader) (Entry, error) {
return Entry{}, nil
}
func (f *textFakeStore) Move(context.Context, []string, string) error { return nil }
func (f *textFakeStore) Copy(context.Context, []string, string) error { return nil }
func (f *textFakeStore) Rename(context.Context, string, string) error { return nil }
func (f *textFakeStore) Delete(context.Context, []string) error { return nil }
type textFakeIndex struct{ docs []TextDoc }
func (f *textFakeIndex) Put(_ context.Context, docs []TextDoc) error {
f.docs = append(f.docs, docs...)
return nil
}
func (f *textFakeIndex) Delete(context.Context, []string) error { return nil }
func (f *textFakeIndex) Search(context.Context, SearchQuery) ([]SearchHit, error) { return nil, nil }
func (f *textFakeIndex) Name() string { return "fake" }
type textFakeExtractor struct{ prefix string }
func (f textFakeExtractor) Extract(path, _, _ string, _ int) (string, error) {
b, err := os.ReadFile(path)
if err != nil {
return "", err
}
return f.prefix + string(b), nil
}
type textFailingExtractor struct{}
func (textFailingExtractor) Extract(string, string, string, int) (string, error) {
return "", fmt.Errorf("boom")
}
+249
View File
@@ -0,0 +1,249 @@
package onlyoffice
// Single file client (epic #34, F4 #38). FileClient composes the registered
// FileStore and Searcher backends and picks one per operation: REST/DAV for
// writes, PostgreSQL (when registered) for fast reads, Elasticsearch for name
// and content search. Client.Files returns the facade; it also implements
// FileStore, so existing callers keep compiling.
import (
"context"
"errors"
"io"
"strings"
)
// ProviderES is the composed Elasticsearch searcher. The SQL store owns
// ProviderPG/ProviderMySQL (filestore_pg.go); the facade references ProviderPG in
// readOrder.
const ProviderES = "elasticsearch"
var (
errNoReadBackend = errors.New("onlyoffice: no file backend registered for reads")
errNoWriteBackend = errors.New("onlyoffice: no file backend registered for writes")
errNoSearcher = errors.New("onlyoffice: no search backend registered (set ONLYOFFICE_ES_URL)")
)
// FileClient is the single entry point for file operations. It holds the
// registered backends and the order in which each operation tries them.
type FileClient struct {
stores map[string]FileStore
searchers map[string]Searcher
readOrder []string
writeOrder []string
searchOrder []string
}
// newFileClient builds the facade over the built-in REST and DAV stores. The
// Elasticsearch searcher is registered when ONLYOFFICE_ES_URL is set; the
// missing-credential case is left to Search so read-only commands still work.
func (c *Client) newFileClient() *FileClient {
f := &FileClient{
stores: map[string]FileStore{
ProviderREST: &restStore{c: c},
ProviderDAV: &davStore{c: c},
},
searchers: map[string]Searcher{},
readOrder: []string{ProviderPG, ProviderMySQL, ProviderREST, ProviderDAV},
writeOrder: []string{ProviderREST, ProviderDAV},
searchOrder: []string{ProviderES},
}
if cfg := ESConfigFromEnv(); cfg.URL != "" {
if es, err := NewESSearcher(cfg); err == nil {
f.searchers[ProviderES] = es
}
}
return f
}
// RegisterStore adds or replaces a named backend (for example the PostgreSQL
// read store). The name is matched case-insensitively.
func (f *FileClient) RegisterStore(name string, s FileStore) {
if f == nil || s == nil {
return
}
name = normalizeProvider(name)
if name == "" {
return
}
if f.stores == nil {
f.stores = map[string]FileStore{}
}
f.stores[name] = s
}
// RegisterSearcher adds or replaces a named search backend.
func (f *FileClient) RegisterSearcher(name string, s Searcher) {
if f == nil || s == nil {
return
}
name = normalizeProvider(name)
if name == "" {
return
}
if f.searchers == nil {
f.searchers = map[string]Searcher{}
}
f.searchers[name] = s
}
// Read returns the preferred backend for reads: the SQL store (PostgreSQL or
// MySQL) when registered, then REST, then WebDAV.
func (f *FileClient) Read() FileStore { return f.firstStore(f.readOrder) }
// Write returns the preferred backend for writes: REST, then WebDAV.
func (f *FileClient) Write() FileStore { return f.firstStore(f.writeOrder) }
// Search returns the preferred name/content searcher (Elasticsearch), or an
// error when no search backend is configured.
func (f *FileClient) Search() (Searcher, error) {
if f == nil {
return nil, errNoSearcher
}
for _, name := range f.searchOrder {
if s := f.searchers[normalizeProvider(name)]; s != nil {
return s, nil
}
}
return nil, errNoSearcher
}
// firstStore returns the first registered store in the order.
func (f *FileClient) firstStore(order []string) FileStore {
if f == nil {
return nil
}
for _, name := range order {
if s := f.stores[normalizeProvider(name)]; s != nil {
return s
}
}
return nil
}
// orderedStores returns the registered stores in the order.
func (f *FileClient) orderedStores(order []string) []FileStore {
if f == nil {
return nil
}
out := make([]FileStore, 0, len(order))
for _, name := range order {
if s := f.stores[normalizeProvider(name)]; s != nil {
out = append(out, s)
}
}
return out
}
func normalizeProvider(name string) string {
return strings.ToLower(strings.TrimSpace(name))
}
// Name implements FileStore and reports the preferred read backend.
func (f *FileClient) Name() string {
if s := f.Read(); s != nil {
return s.Name()
}
return ""
}
// List reads from the preferred backend, falling back to the next read backend
// only on a transient error (429/502/503/504).
func (f *FileClient) List(ctx context.Context, parentID string) ([]Entry, error) {
return fallbackRead(ctx, f.orderedStores(f.readOrder), func(s FileStore) ([]Entry, error) {
return s.List(ctx, parentID)
})
}
// Stat reads from the preferred backend, with the same transient fallback.
func (f *FileClient) Stat(ctx context.Context, id string) (Entry, error) {
return fallbackRead(ctx, f.orderedStores(f.readOrder), func(s FileStore) (Entry, error) {
return s.Stat(ctx, id)
})
}
// Download streams file bytes. It does not fall back: a failed attempt may have
// already written partial bytes into w, so a second backend would append.
func (f *FileClient) Download(ctx context.Context, id string, w io.Writer) (int64, error) {
s := f.Read()
if s == nil {
return 0, errNoReadBackend
}
return s.Download(ctx, id, w)
}
// CreateFolder writes to the preferred write backend.
func (f *FileClient) CreateFolder(ctx context.Context, parentID, title string) (Entry, error) {
s := f.Write()
if s == nil {
return Entry{}, errNoWriteBackend
}
return s.CreateFolder(ctx, parentID, title)
}
// Upload writes to the preferred write backend.
func (f *FileClient) Upload(ctx context.Context, parentID, title string, r io.Reader) (Entry, error) {
s := f.Write()
if s == nil {
return Entry{}, errNoWriteBackend
}
return s.Upload(ctx, parentID, title, r)
}
// Move writes to the preferred write backend.
func (f *FileClient) Move(ctx context.Context, ids []string, parentID string) error {
s := f.Write()
if s == nil {
return errNoWriteBackend
}
return s.Move(ctx, ids, parentID)
}
// Copy writes to the preferred write backend.
func (f *FileClient) Copy(ctx context.Context, ids []string, parentID string) error {
s := f.Write()
if s == nil {
return errNoWriteBackend
}
return s.Copy(ctx, ids, parentID)
}
// Rename writes to the preferred write backend.
func (f *FileClient) Rename(ctx context.Context, id, title string) error {
s := f.Write()
if s == nil {
return errNoWriteBackend
}
return s.Rename(ctx, id, title)
}
// Delete writes to the preferred write backend.
func (f *FileClient) Delete(ctx context.Context, ids []string) error {
s := f.Write()
if s == nil {
return errNoWriteBackend
}
return s.Delete(ctx, ids)
}
// fallbackRead runs op against each store in order, moving on only when the
// error is transient. Non-transient errors (not found, forbidden) are final.
func fallbackRead[T any](ctx context.Context, stores []FileStore, op func(FileStore) (T, error)) (T, error) {
var zero T
if len(stores) == 0 {
return zero, errNoReadBackend
}
var err error
for i, s := range stores {
var v T
v, err = op(s)
if err == nil {
return v, nil
}
if i == len(stores)-1 || !Transient(err) {
return zero, err
}
}
return zero, err
}
+131
View File
@@ -0,0 +1,131 @@
//go:build integration
package onlyoffice
import (
"bytes"
"context"
"strconv"
"testing"
"time"
)
// TestIntegrationFacadeCRUD drives the whole operation set through the composed
// facade c.Files(): folder create, upload, stat, list, rename, move, copy,
// delete. Writes must go to REST (the default writeOrder), reads follow
// readOrder (REST when no SQL backend is registered) and every returned Entry
// must report its provider. Destructive — throwaway project, cleaned up.
func TestIntegrationFacadeCRUD(t *testing.T) {
c := liveClient(t)
t.Cleanup(func() { cleanupTestProjects(t, c) })
ctx := context.Background()
suffix := time.Now().UTC().Format("20060102-150405")
project, err := c.CreateProject(NewProjectRequest{
Title: testProjectPrefix + "facade-" + suffix,
Description: "go-onlyoffice facade CRUD integration",
})
if err != nil {
t.Fatalf("CreateProject: %v", err)
}
if project.ID == nil {
t.Fatal("created project without id")
}
root, err := c.projectFolderID(ctx, strconv.Itoa(*project.ID))
if err != nil {
t.Fatalf("projectFolderID: %v", err)
}
f := c.Files()
if got := f.Write().Name(); got != ProviderREST {
t.Fatalf("Write().Name() = %q, want %q", got, ProviderREST)
}
if got := f.Read().Name(); got != ProviderREST {
t.Fatalf("Read().Name() = %q, want %q (no SQL backend registered)", got, ProviderREST)
}
src, err := f.CreateFolder(ctx, root, "facade-src-"+suffix)
if err != nil {
t.Fatalf("CreateFolder src: %v", err)
}
if src.Kind != Folder || src.ID == "" {
t.Fatalf("created src folder: %+v", src)
}
if src.Provider != ProviderREST {
t.Fatalf("CreateFolder provider = %q, want %q", src.Provider, ProviderREST)
}
dst, err := f.CreateFolder(ctx, root, "facade-dst-"+suffix)
if err != nil {
t.Fatalf("CreateFolder dst: %v", err)
}
if dst.Provider != ProviderREST {
t.Fatalf("CreateFolder dst provider = %q, want %q", dst.Provider, ProviderREST)
}
t.Cleanup(func() {
if err := c.DeleteDavItems(ctx, []string{src.ID, dst.ID}, nil); err != nil {
t.Logf("cleanup folders: %v", err)
}
})
content := []byte("facade crud " + suffix + "\n")
up, err := f.Upload(ctx, src.ID, "facade-doc-"+suffix+".txt", bytes.NewReader(content))
if err != nil {
t.Fatalf("Upload: %v", err)
}
if up.Kind != File || up.ID == "" {
t.Fatalf("uploaded entry: %+v", up)
}
if up.Provider != ProviderREST {
t.Fatalf("Upload provider = %q, want %q (write order REST first)", up.Provider, ProviderREST)
}
if !waitEntry(ctx, f, src.ID, up.ID, 15*time.Second) {
t.Fatalf("uploaded %s not listed in src", up.ID)
}
st, err := f.Stat(ctx, up.ID)
if err != nil {
t.Fatalf("Stat: %v", err)
}
if st.ID != up.ID || st.Kind != File {
t.Fatalf("Stat = %+v", st)
}
if st.Provider != ProviderREST {
t.Fatalf("Stat provider = %q, want %q (read order REST)", st.Provider, ProviderREST)
}
list, err := f.List(ctx, src.ID)
if err != nil {
t.Fatalf("List: %v", err)
}
if e := entryByID(list, up.ID); e == nil {
t.Fatalf("uploaded %s not in List(src)", up.ID)
} else if e.Provider != ProviderREST {
t.Fatalf("List provider = %q, want %q", e.Provider, ProviderREST)
}
renamed := "facade-renamed-" + suffix + ".txt"
renameEventually(t, ctx, f, up.ID, renamed)
moveEventually(t, ctx, f, up.ID, dst.ID)
if !waitEntry(ctx, f, dst.ID, up.ID, 20*time.Second) {
t.Fatalf("moved file %s not in dst", up.ID)
}
copied := copyEventually(t, ctx, f, up.ID, src.ID, 20*time.Second)
if copied == nil {
t.Fatalf("no copy found in src after Copy")
}
if copied.Provider != ProviderREST {
t.Fatalf("Copy provider = %q, want %q", copied.Provider, ProviderREST)
}
if err := f.Delete(ctx, []string{up.ID, copied.ID}); err != nil {
t.Fatalf("Delete: %v", err)
}
if !waitNoEntry(ctx, f, dst.ID, up.ID, 20*time.Second) {
t.Fatalf("file %s still present in dst after delete", up.ID)
}
if !waitNoEntry(ctx, f, src.ID, copied.ID, 20*time.Second) {
t.Fatalf("copy %s still present in src after delete", copied.ID)
}
}
+298
View File
@@ -0,0 +1,298 @@
package onlyoffice
import (
"context"
"errors"
"fmt"
"io"
"strings"
"testing"
)
// fakeStore is a FileStore test double; it records which backend served a call
// and returns a canned result or error.
type fakeStore struct {
name string
entries []Entry
err error
calls *[]string
}
func (f *fakeStore) record(op string) {
if f.calls != nil {
*f.calls = append(*f.calls, op+":"+f.name)
}
}
func (f *fakeStore) Name() string { return f.name }
func (f *fakeStore) List(_ context.Context, _ string) ([]Entry, error) {
f.record("list")
if f.err != nil {
return nil, f.err
}
return f.entries, nil
}
func (f *fakeStore) Stat(_ context.Context, id string) (Entry, error) {
f.record("stat")
if f.err != nil {
return Entry{}, f.err
}
return Entry{ID: id, Title: "t-" + f.name, Provider: f.name}, nil
}
func (f *fakeStore) CreateFolder(_ context.Context, _, title string) (Entry, error) {
f.record("mkdir")
if f.err != nil {
return Entry{}, f.err
}
return Entry{ID: "new", Title: title, Provider: f.name}, nil
}
func (f *fakeStore) Upload(_ context.Context, _, title string, _ io.Reader) (Entry, error) {
f.record("upload")
return Entry{ID: "up", Title: title, Provider: f.name}, f.err
}
func (f *fakeStore) Download(_ context.Context, _ string, _ io.Writer) (int64, error) {
f.record("download")
return 0, f.err
}
func (f *fakeStore) Move(_ context.Context, _ []string, _ string) error {
f.record("move")
return f.err
}
func (f *fakeStore) Copy(_ context.Context, _ []string, _ string) error {
f.record("copy")
return f.err
}
func (f *fakeStore) Rename(_ context.Context, _, _ string) error {
f.record("rename")
return f.err
}
func (f *fakeStore) Delete(_ context.Context, _ []string) error {
f.record("delete")
return f.err
}
type fakeSearcher struct{ name string }
func (s *fakeSearcher) Name() string { return s.name }
func (s *fakeSearcher) Search(_ context.Context, _ SearchQuery) ([]SearchHit, error) {
return []SearchHit{{Entry: Entry{Title: s.name}}}, nil
}
func newFacadeTestClient(stores map[string]FileStore, read, write []string) *FileClient {
return &FileClient{
stores: stores,
searchers: map[string]Searcher{},
readOrder: read,
writeOrder: write,
}
}
// TestFileClientIsFileStore guarantees the facade can stand in for the
// interface anywhere a plain FileStore is expected.
func TestFileClientIsFileStore(t *testing.T) {
var _ FileStore = (*FileClient)(nil)
}
func TestClientFilesPrefersRESTForReadsAndWrites(t *testing.T) {
c := NewClient(Credentials{})
f := c.Files()
if got := f.Read().Name(); got != ProviderREST {
t.Errorf("Read().Name() = %q, want %q", got, ProviderREST)
}
if got := f.Write().Name(); got != ProviderREST {
t.Errorf("Write().Name() = %q, want %q", got, ProviderREST)
}
if got := f.Name(); got != ProviderREST {
t.Errorf("Name() = %q, want %q", got, ProviderREST)
}
}
func TestFileClientPostgresTakesReadPriority(t *testing.T) {
pg := &fakeStore{name: ProviderPG}
f := newFacadeTestClient(
map[string]FileStore{ProviderREST: &fakeStore{name: ProviderREST}, ProviderPG: pg},
[]string{ProviderPG, ProviderREST},
[]string{ProviderREST},
)
if got := f.Read().Name(); got != ProviderPG {
t.Errorf("Read().Name() = %q, want %q", got, ProviderPG)
}
if got := f.Write().Name(); got != ProviderREST {
t.Errorf("Write().Name() = %q, want %q (PG is read-only)", got, ProviderREST)
}
}
func TestFileClientRegisterStoreNormalizesName(t *testing.T) {
pg := &fakeStore{name: "pg"}
f := newFacadeTestClient(map[string]FileStore{}, []string{ProviderPG}, nil)
f.RegisterStore(" POSTGRES ", pg)
if got := f.Read(); got != pg {
t.Fatalf("Read() = %v, want registered postgres store", got)
}
f.RegisterStore("", pg)
f.RegisterStore("pg", nil)
}
func TestFileClientReadFallsBackOnlyOnTransient(t *testing.T) {
var calls []string
primary := &fakeStore{name: "primary", err: fmt.Errorf("onlyoffice: list: 503 unavailable"), calls: &calls}
secondary := &fakeStore{name: "secondary", entries: []Entry{{ID: "1"}}, calls: &calls}
f := newFacadeTestClient(
map[string]FileStore{"primary": primary, "secondary": secondary},
[]string{"primary", "secondary"},
nil,
)
got, err := f.List(context.Background(), "root")
if err != nil {
t.Fatalf("List: %v", err)
}
if len(got) != 1 || got[0].ID != "1" {
t.Fatalf("List() = %+v, want secondary entry", got)
}
want := []string{"list:primary", "list:secondary"}
if fmt.Sprint(calls) != fmt.Sprint(want) {
t.Fatalf("call order = %v, want %v", calls, want)
}
}
func TestFileClientReadStopsOnPermanentError(t *testing.T) {
var calls []string
primary := &fakeStore{name: "primary", err: errors.New("onlyoffice: not found"), calls: &calls}
secondary := &fakeStore{name: "secondary", entries: []Entry{{ID: "1"}}, calls: &calls}
f := newFacadeTestClient(
map[string]FileStore{"primary": primary, "secondary": secondary},
[]string{"primary", "secondary"},
nil,
)
if _, err := f.List(context.Background(), "root"); err == nil {
t.Fatal("expected permanent error to be returned")
}
if len(calls) != 1 || calls[0] != "list:primary" {
t.Fatalf("secondary backend must not run on a permanent error: %v", calls)
}
}
func TestFileClientWriteUsesWriteBackend(t *testing.T) {
var calls []string
rest := &fakeStore{name: ProviderREST, calls: &calls}
dav := &fakeStore{name: ProviderDAV, calls: &calls}
f := newFacadeTestClient(
map[string]FileStore{ProviderREST: rest, ProviderDAV: dav},
[]string{ProviderREST},
[]string{ProviderREST, ProviderDAV},
)
if _, err := f.CreateFolder(context.Background(), "p", "t"); err != nil {
t.Fatalf("CreateFolder: %v", err)
}
if _, err := f.Upload(context.Background(), "p", "t", nil); err != nil {
t.Fatalf("Upload: %v", err)
}
if len(calls) != 2 || calls[0] != "mkdir:rest" || calls[1] != "upload:rest" {
t.Fatalf("write calls = %v, want REST", calls)
}
}
func TestFileClientWriteWithoutBackend(t *testing.T) {
f := newFacadeTestClient(map[string]FileStore{}, nil, nil)
if err := f.Delete(context.Background(), []string{"1"}); !errors.Is(err, errNoWriteBackend) {
t.Fatalf("Delete err = %v, want errNoWriteBackend", err)
}
if _, err := f.List(context.Background(), "root"); !errors.Is(err, errNoReadBackend) {
t.Fatalf("List err = %v, want errNoReadBackend", err)
}
}
func TestNewFileClientReadOrderIncludesSQL(t *testing.T) {
f := NewClient(Credentials{}).newFileClient()
want := []string{ProviderPG, ProviderMySQL, ProviderREST, ProviderDAV}
if fmt.Sprint(f.readOrder) != fmt.Sprint(want) {
t.Fatalf("readOrder = %v, want %v", f.readOrder, want)
}
}
// TestClientFileStoreSQLRoutingWithoutDSN checks that the SQL backend names are
// recognised and never yield nil: without a DSN the returned store surfaces the
// open error on use.
func TestClientFileStoreSQLRoutingWithoutDSN(t *testing.T) {
t.Setenv("ONLYOFFICE_DSN", "")
t.Setenv("ONLYOFFICE_PG_HOST", "")
c := NewClient(Credentials{})
for _, name := range []string{"pg", "sql", ProviderPG, ProviderMySQL} {
s := c.FileStore(name)
if s == nil {
t.Fatalf("FileStore(%q) = nil", name)
}
if _, err := s.Stat(context.Background(), "1"); err == nil {
t.Errorf("FileStore(%q).Stat without DSN: want error", name)
}
}
}
func TestFileClientMySQLStoreIsPreferredForReads(t *testing.T) {
mysql := &fakeStore{name: ProviderMySQL}
f := newFacadeTestClient(
map[string]FileStore{ProviderREST: &fakeStore{name: ProviderREST}, ProviderMySQL: mysql},
[]string{ProviderPG, ProviderMySQL, ProviderREST},
[]string{ProviderREST},
)
if got := f.Read().Name(); got != ProviderMySQL {
t.Errorf("Read().Name() = %q, want %q", got, ProviderMySQL)
}
}
// TestFileClientWriteToReadOnlyStore guarantees the facade surfaces ErrReadOnly
// when the configured write backend is the read-only SQL store.
func TestFileClientWriteToReadOnlyStore(t *testing.T) {
pg := &pgStore{driver: ProviderPG}
f := newFacadeTestClient(
map[string]FileStore{ProviderREST: &fakeStore{name: ProviderREST}, ProviderPG: pg},
[]string{ProviderPG, ProviderREST},
[]string{ProviderPG, ProviderREST},
)
ctx := context.Background()
if _, err := f.CreateFolder(ctx, "1", "x"); !errors.Is(err, ErrReadOnly) {
t.Errorf("CreateFolder err = %v", err)
}
if _, err := f.Upload(ctx, "1", "x", strings.NewReader("x")); !errors.Is(err, ErrReadOnly) {
t.Errorf("Upload err = %v", err)
}
if err := f.Move(ctx, []string{"1"}, "2"); !errors.Is(err, ErrReadOnly) {
t.Errorf("Move err = %v", err)
}
if err := f.Copy(ctx, []string{"1"}, "2"); !errors.Is(err, ErrReadOnly) {
t.Errorf("Copy err = %v", err)
}
if err := f.Rename(ctx, "1", "x"); !errors.Is(err, ErrReadOnly) {
t.Errorf("Rename err = %v", err)
}
if err := f.Delete(ctx, []string{"1"}); !errors.Is(err, ErrReadOnly) {
t.Errorf("Delete err = %v", err)
}
}
func TestFileClientSearchSelection(t *testing.T) {
f := &FileClient{searchers: map[string]Searcher{}, searchOrder: []string{ProviderES}}
_, err := f.Search()
if err == nil || !strings.Contains(err.Error(), "ONLYOFFICE_ES_URL") {
t.Fatalf("Search without backend = %v, want ONLYOFFICE_ES_URL hint", err)
}
es := &fakeSearcher{name: "fake-es"}
f.RegisterSearcher(ProviderES, es)
got, err := f.Search()
if err != nil {
t.Fatalf("Search: %v", err)
}
if got.Name() != "fake-es" {
t.Fatalf("searcher = %q, want fake-es", got.Name())
}
}
+525
View File
@@ -0,0 +1,525 @@
package onlyoffice
// Read-only SQL backend of the unified file client (epic #34, F2 #36).
//
// The goal is to read files and folders straight from the Community Server
// database, without the REST layer. Research on the live portal (VM
// `onlyoffice-v2`) showed the server runs on **MySQL 8.0** (`files_file`,
// `files_folder`, `files_folder_tree`, tenant `tenants_tenants`), not
// PostgreSQL — see docs/community-server-db.md. The store below therefore
// speaks `database/sql` and selects its driver from the DSN, so it works
// against the live MySQL today and against PostgreSQL if the portal is ever
// migrated. Every query is a SELECT; the write methods of FileStore return
// ErrReadOnly.
//
// Downloads follow the portal's S3/MinIO object layout through the shared
// MinIO helper in storage_fallback.go — no HTTP file endpoint is used.
import (
"context"
"database/sql"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"os"
"path/filepath"
"strconv"
"strings"
"time"
"github.com/go-sql-driver/mysql"
_ "github.com/jackc/pgx/v5/stdlib"
)
// Provider names for the SQL backend. ProviderPG is the value Name reports for
// a PostgreSQL connection and ProviderMySQL for MySQL.
const (
ProviderPG = "postgres"
ProviderMySQL = "mysql"
)
// ErrReadOnly is returned by every FileStore write method of the SQL backend.
var ErrReadOnly = errors.New("onlyoffice: sql file store is read-only")
const (
pgConnectTimeout = 10 * time.Second
pgSearchLimit = 50
pgSearchMaxLimit = 500
)
// PGConfig configures the read-only SQL store. DSN is a driver DSN:
// `user:pass@tcp(host:port)/onlyoffice?parseTime=true` for MySQL or a
// `postgres://` / libpq keyword string for PostgreSQL. Driver, when set,
// forces the engine ("postgres" or "mysql"); otherwise it is detected from the
// DSN. Tenant filters rows (empty means all tenants).
type PGConfig struct {
DSN string
Driver string
Tenant string
}
// PGConfigFromEnv reads ONLYOFFICE_DSN (or the ONLYOFFICE_PG_* parts),
// ONLYOFFICE_PG_DRIVER and the tenant from ONLYOFFICE_PG_TENANT /
// ONLYOFFICE_TENANT. The library never loads dotfiles — the CLI does that.
func PGConfigFromEnv() PGConfig {
dsn := strings.TrimSpace(os.Getenv("ONLYOFFICE_DSN"))
if dsn == "" {
dsn = pgDSNFromParts()
}
return PGConfig{
DSN: dsn,
Driver: strings.TrimSpace(os.Getenv("ONLYOFFICE_PG_DRIVER")),
Tenant: firstNonEmpty(os.Getenv("ONLYOFFICE_PG_TENANT"), os.Getenv("ONLYOFFICE_TENANT")),
}
}
// pgDSNFromParts builds a libpq keyword DSN from ONLYOFFICE_PG_* variables.
// It returns "" unless a host is set, which keeps the MySQL path (ONLYOFFICE_DSN)
// the default.
func pgDSNFromParts() string {
host := strings.TrimSpace(os.Getenv("ONLYOFFICE_PG_HOST"))
if host == "" {
return ""
}
port := firstNonEmpty(os.Getenv("ONLYOFFICE_PG_PORT"), "5432")
dbname := firstNonEmpty(os.Getenv("ONLYOFFICE_PG_DBNAME"), "onlyoffice")
sslmode := firstNonEmpty(os.Getenv("ONLYOFFICE_PG_SSLMODE"), "disable")
return fmt.Sprintf("host=%s port=%s user=%s password=%s dbname=%s sslmode=%s",
host, port, os.Getenv("ONLYOFFICE_PG_USER"), os.Getenv("ONLYOFFICE_PG_PASSWORD"), dbname, sslmode)
}
// SQLFileStore opens the read-only SQL store from the environment
// (PGConfigFromEnv: ONLYOFFICE_DSN or the ONLYOFFICE_PG_* parts). It is the
// error-aware counterpart of Client.FileStore("pg"/"sql"), which returns an
// errStore when the open fails. The caller owns the returned store and should
// close it (the concrete type has a Close method).
func (c *Client) SQLFileStore() (FileStore, error) {
return NewPGStore(PGConfigFromEnv())
}
// pgStore is a read-only FileStore/Searcher over the Community Server database.
type pgStore struct {
db *sql.DB
driver string
tenantID int64
hasTenant bool
http *http.Client
}
var (
_ FileStore = (*pgStore)(nil)
_ Searcher = (*pgStore)(nil)
)
// NewPGStore opens the database and verifies connectivity. It never writes.
func NewPGStore(cfg PGConfig) (*pgStore, error) {
dsn := strings.TrimSpace(cfg.DSN)
if dsn == "" {
return nil, fmt.Errorf("onlyoffice: sql file store: empty DSN (set ONLYOFFICE_DSN)")
}
driver := pgDriver(dsn, cfg.Driver)
dsn, err := normalizeSQLDSN(driver, dsn)
if err != nil {
return nil, err
}
db, err := sql.Open(sqlDriverName(driver), dsn)
if err != nil {
return nil, fmt.Errorf("onlyoffice: sql file store: open %s: %w", driver, err)
}
ctx, cancel := context.WithTimeout(context.Background(), pgConnectTimeout)
defer cancel()
if err := db.PingContext(ctx); err != nil {
db.Close()
return nil, fmt.Errorf("onlyoffice: sql file store: ping %s: %w", driver, err)
}
s := &pgStore{db: db, driver: driver, http: &http.Client{}}
if t := strings.TrimSpace(cfg.Tenant); t != "" {
n, err := strconv.ParseInt(t, 10, 64)
if err != nil {
db.Close()
return nil, fmt.Errorf("onlyoffice: sql file store: non-numeric tenant %q", t)
}
s.tenantID, s.hasTenant = n, true
}
return s, nil
}
// Close releases the database handle.
func (s *pgStore) Close() error { return s.db.Close() }
// Name implements FileStore and Searcher.
func (s *pgStore) Name() string { return s.driver }
// pgDriver resolves the engine: the explicit value wins, otherwise the DSN
// shape decides. A leading postgres:// scheme or a libpq keyword DSN (which
// always carries '=') selects PostgreSQL; anything else is MySQL.
func pgDriver(dsn, explicit string) string {
switch strings.ToLower(strings.TrimSpace(explicit)) {
case ProviderPG, "pg", "postgresql", "pgx":
return ProviderPG
case ProviderMySQL, "mariadb":
return ProviderMySQL
}
l := strings.ToLower(strings.TrimSpace(dsn))
switch {
case strings.HasPrefix(l, "postgres://"), strings.HasPrefix(l, "postgresql://"):
return ProviderPG
case strings.HasPrefix(l, "mysql://"), strings.Contains(l, "@tcp("), strings.Contains(l, "@unix("):
return ProviderMySQL
case strings.Contains(l, "="):
return ProviderPG
default:
return ProviderMySQL
}
}
// sqlDriverName maps the engine to its registered database/sql driver.
func sqlDriverName(driver string) string {
if driver == ProviderPG {
return "pgx"
}
return "mysql"
}
// normalizeSQLDSN converts a mysql:// URL to the go-sql-driver form and forces
// parseTime so datetime columns scan into time.Time. PostgreSQL DSNs pass
// through untouched.
func normalizeSQLDSN(driver, dsn string) (string, error) {
if driver != ProviderMySQL {
return dsn, nil
}
if strings.HasPrefix(strings.ToLower(dsn), "mysql://") {
converted, err := mysqlDSNFromURL(dsn)
if err != nil {
return "", err
}
dsn = converted
}
cfg, err := mysql.ParseDSN(dsn)
if err != nil {
return "", fmt.Errorf("onlyoffice: sql file store: parse mysql DSN: %w", err)
}
cfg.ParseTime = true
return cfg.FormatDSN(), nil
}
// mysqlDSNFromURL turns mysql://user:pass@host:port/db into the driver DSN.
func mysqlDSNFromURL(raw string) (string, error) {
u, err := url.Parse(raw)
if err != nil || u.Host == "" {
return "", fmt.Errorf("onlyoffice: sql file store: bad mysql URL %q", raw)
}
user := ""
if u.User != nil {
user = u.User.Username()
if p, ok := u.User.Password(); ok {
user += ":" + p
}
}
q := u.Query()
q.Set("parseTime", "true")
return fmt.Sprintf("%s@tcp(%s)/%s?%s", user, u.Host, strings.TrimPrefix(u.Path, "/"), q.Encode()), nil
}
// rebind rewrites '?' placeholders to PostgreSQL's $1..$n. MySQL keeps them.
func rebind(query, driver string) string {
if driver != ProviderPG {
return query
}
var b strings.Builder
b.Grow(len(query) + 8)
n := 0
for _, r := range query {
if r == '?' {
n++
b.WriteByte('$')
b.WriteString(strconv.Itoa(n))
continue
}
b.WriteRune(r)
}
return b.String()
}
// List returns the folders and files directly below parentID, folders first.
func (s *pgStore) List(ctx context.Context, parentID string) ([]Entry, error) {
pid, err := parseEntryID(parentID)
if err != nil {
return nil, err
}
folders, err := s.queryFolders(ctx, "parent_id = ?", pid)
if err != nil {
return nil, err
}
files, err := s.queryFiles(ctx, "folder_id = ? AND current_version = 1", pid)
if err != nil {
return nil, err
}
out := make([]Entry, 0, len(folders)+len(files))
for _, f := range folders {
out = append(out, folderRowToEntry(f, s.Name()))
}
for _, f := range files {
out = append(out, fileRowToEntry(f, s.Name()))
}
return out, nil
}
// Stat resolves a folder or file id to an Entry. Folders win when both id
// spaces overlap (they never do on a real portal, but the lookup is cheap).
func (s *pgStore) Stat(ctx context.Context, id string) (Entry, error) {
n, err := parseEntryID(id)
if err != nil {
return Entry{}, err
}
folders, err := s.queryFolders(ctx, "id = ?", n)
if err != nil {
return Entry{}, err
}
if len(folders) > 0 {
return folderRowToEntry(folders[0], s.Name()), nil
}
files, err := s.queryFiles(ctx, "id = ? AND current_version = 1", n)
if err != nil {
return Entry{}, err
}
if len(files) == 0 {
return Entry{}, fmt.Errorf("onlyoffice: sql file store: id %s not found", id)
}
return fileRowToEntry(files[0], s.Name()), nil
}
// Download streams the file's current version from the portal's S3/MinIO store.
// The object key is reconstructed from the file id and version; the parent
// folder id is not part of the key.
func (s *pgStore) Download(ctx context.Context, id string, w io.Writer) (int64, error) {
n, err := parseEntryID(id)
if err != nil {
return 0, err
}
files, err := s.queryFiles(ctx, "id = ? AND current_version = 1", n)
if err != nil {
return 0, err
}
if len(files) == 0 {
return 0, fmt.Errorf("onlyoffice: sql file store: file %s not found", id)
}
f := files[0]
key := csObjectKey(s.tenantID, f.id, f.version, filepath.Ext(f.title))
return downloadMinioObject(ctx, s.http, key, w)
}
// CreateFolder is unavailable: the SQL backend is read-only.
func (s *pgStore) CreateFolder(context.Context, string, string) (Entry, error) {
return Entry{}, ErrReadOnly
}
// Upload is unavailable: the SQL backend is read-only.
func (s *pgStore) Upload(context.Context, string, string, io.Reader) (Entry, error) {
return Entry{}, ErrReadOnly
}
// Move is unavailable: the SQL backend is read-only.
func (s *pgStore) Move(context.Context, []string, string) error { return ErrReadOnly }
// Copy is unavailable: the SQL backend is read-only.
func (s *pgStore) Copy(context.Context, []string, string) error { return ErrReadOnly }
// Rename is unavailable: the SQL backend is read-only.
func (s *pgStore) Rename(context.Context, string, string) error { return ErrReadOnly }
// Delete is unavailable: the SQL backend is read-only.
func (s *pgStore) Delete(context.Context, []string) error { return ErrReadOnly }
// Search matches file titles by substring. Content search lives in the
// Elasticsearch backend; q.InContent is ignored here.
func (s *pgStore) Search(ctx context.Context, q SearchQuery) ([]SearchHit, error) {
text := strings.TrimSpace(q.Text)
if text == "" {
return nil, fmt.Errorf("onlyoffice: empty search query")
}
limit := q.Limit
if limit <= 0 {
limit = pgSearchLimit
}
if limit > pgSearchMaxLimit {
limit = pgSearchMaxLimit
}
where := "title LIKE ? AND current_version = 1"
args := []any{"%" + text + "%"}
if s.hasTenant {
where += " AND tenant_id = ?"
args = append(args, s.tenantID)
}
if fid := strings.TrimSpace(q.FolderID); fid != "" {
n, err := parseEntryID(fid)
if err != nil {
return nil, err
}
where += " AND folder_id = ?"
args = append(args, n)
}
for _, ext := range normalizeExtensions(q.Extensions) {
where += " AND LOWER(title) LIKE ?"
args = append(args, "%."+ext)
}
query := rebind(`SELECT id, folder_id, title, content_length, version, create_on, modified_on
FROM files_file WHERE `+where+` ORDER BY modified_on DESC, id DESC LIMIT ?`, s.driver)
args = append(args, limit)
rows, err := s.db.QueryContext(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("onlyoffice: sql search: %w", err)
}
defer rows.Close()
var hits []SearchHit
for rows.Next() {
f, err := scanFileRow(rows)
if err != nil {
return nil, err
}
hits = append(hits, SearchHit{Entry: fileRowToEntry(f, s.Name())})
}
return hits, rows.Err()
}
// queryFolders runs a folder SELECT with the tenant filter applied.
func (s *pgStore) queryFolders(ctx context.Context, where string, arg any) ([]pgFolderRow, error) {
args := []any{arg}
if s.hasTenant {
where += " AND tenant_id = ?"
args = append(args, s.tenantID)
}
query := rebind(`SELECT id, parent_id, title, create_on, modified_on
FROM files_folder WHERE `+where+` ORDER BY title, id`, s.driver)
rows, err := s.db.QueryContext(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("onlyoffice: sql list folders: %w", err)
}
defer rows.Close()
var out []pgFolderRow
for rows.Next() {
var r pgFolderRow
if err := rows.Scan(&r.id, &r.parentID, &r.title, &r.created, &r.modified); err != nil {
return nil, fmt.Errorf("onlyoffice: sql folder row: %w", err)
}
out = append(out, r)
}
return out, rows.Err()
}
// queryFiles runs a file SELECT for the current version with the tenant filter.
func (s *pgStore) queryFiles(ctx context.Context, where string, arg any) ([]pgFileRow, error) {
args := []any{arg}
if s.hasTenant {
where += " AND tenant_id = ?"
args = append(args, s.tenantID)
}
query := rebind(`SELECT id, folder_id, title, content_length, version, create_on, modified_on
FROM files_file WHERE `+where+` ORDER BY title, id`, s.driver)
rows, err := s.db.QueryContext(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("onlyoffice: sql list files: %w", err)
}
defer rows.Close()
var out []pgFileRow
for rows.Next() {
f, err := scanFileRow(rows)
if err != nil {
return nil, err
}
out = append(out, f)
}
return out, rows.Err()
}
// pgFileRow is one current files_file row.
type pgFileRow struct {
id int64
folderID int64
title string
size int64
version int
created time.Time
modified time.Time
}
// pgFolderRow is one files_folder row.
type pgFolderRow struct {
id int64
parentID int64
title string
created time.Time
modified time.Time
}
// scanFileRow reads the canonical file column order.
func scanFileRow(rows *sql.Rows) (pgFileRow, error) {
var f pgFileRow
if err := rows.Scan(&f.id, &f.folderID, &f.title, &f.size, &f.version, &f.created, &f.modified); err != nil {
return f, fmt.Errorf("onlyoffice: sql file row: %w", err)
}
return f, nil
}
// fileRowToEntry maps a files_file row to the canonical model.
func fileRowToEntry(f pgFileRow, provider string) Entry {
return Entry{
ID: strconv.FormatInt(f.id, 10),
ParentID: strconv.FormatInt(f.folderID, 10),
Title: f.title,
Kind: File,
Size: f.size,
MIME: mimeForTitle(f.title, ""),
Created: f.created.UTC(),
Modified: f.modified.UTC(),
Version: f.version,
Provider: provider,
}
}
// folderRowToEntry maps a files_folder row to the canonical model.
func folderRowToEntry(f pgFolderRow, provider string) Entry {
return Entry{
ID: strconv.FormatInt(f.id, 10),
ParentID: strconv.FormatInt(f.parentID, 10),
Title: f.title,
Kind: Folder,
Created: f.created.UTC(),
Modified: f.modified.UTC(),
Provider: provider,
}
}
// csObjectKey reconstructs the object key the portal's S3 consumer uses:
//
// 00/00/<tenant>/files/folder_<shard>/file_<id>/v<version>/content.<ext>
//
// The shard is the next thousand above the file id (file 3727 -> folder_4000),
// NOT the parent folder id — verified live against the MinIO bucket.
func csObjectKey(tenant int64, fileID int64, version int, ext string) string {
shard := (fileID/1000 + 1) * 1000
ext = strings.TrimPrefix(strings.ToLower(strings.TrimSpace(ext)), ".")
if ext == "" {
ext = "bin"
}
if version < 1 {
version = 1
}
if tenant <= 0 {
tenant = 1
}
return fmt.Sprintf("00/00/%02d/files/folder_%d/file_%d/v%d/content.%s", tenant, shard, fileID, version, ext)
}
// parseEntryID parses a numeric OnlyOffice id or returns a store error.
func parseEntryID(id string) (int64, error) {
n, err := strconv.ParseInt(strings.TrimSpace(id), 10, 64)
if err != nil {
return 0, fmt.Errorf("onlyoffice: sql file store: non-numeric id %q", id)
}
return n, nil
}
+224
View File
@@ -0,0 +1,224 @@
//go:build integration
package onlyoffice
import (
"bytes"
"context"
"errors"
"os"
"strings"
"testing"
"time"
)
// TestIntegrationPGStore exercises the read-only SQL backend against the live
// Community Server database and cross-checks list/stat/download with the REST
// FileStore. It needs ONLYOFFICE_DSN plus the usual ONLYOFFICE_URL/USER/PASS;
// ONLYOFFICE_PG_TEST_FILE_ID / ONLYOFFICE_PG_TEST_FOLDER_ID pick a real file
// (a file reachable over REST too). Download streams from MinIO, so it also
// needs MINIO_ACCESS_KEY/MINIO_SECRET_KEY.
//
// The live Community Server runs on MySQL; PostgreSQL is supported by the same
// code path when the DSN says so.
func TestIntegrationPGStore(t *testing.T) {
cfg := PGConfigFromEnv()
if strings.TrimSpace(cfg.DSN) == "" {
t.Skip("ONLYOFFICE_DSN not set — skipping SQL store integration test")
}
store, err := NewPGStore(cfg)
if err != nil {
t.Fatalf("NewPGStore: %v", err)
}
t.Cleanup(func() { _ = store.Close() })
t.Logf("sql store backend: %s", store.Name())
ctx := context.Background()
if err := testPGStoreReadOnly(ctx, store); err != nil {
t.Fatal(err)
}
fileID := strings.TrimSpace(os.Getenv("ONLYOFFICE_PG_TEST_FILE_ID"))
folderID := strings.TrimSpace(os.Getenv("ONLYOFFICE_PG_TEST_FOLDER_ID"))
if fileID == "" || folderID == "" {
t.Skip("ONLYOFFICE_PG_TEST_FILE_ID / ONLYOFFICE_PG_TEST_FOLDER_ID not set — skipping live comparison")
}
c := liveClient(t)
rest := c.Files()
dbFile, err := store.Stat(ctx, fileID)
if err != nil {
t.Fatalf("sql Stat(%s): %v", fileID, err)
}
restFile, err := rest.Stat(ctx, fileID)
if err != nil {
t.Fatalf("rest Stat(%s): %v", fileID, err)
}
if dbFile.Kind != File {
t.Errorf("sql kind = %v, want file", dbFile.Kind)
}
if dbFile.ID != restFile.ID || dbFile.Title != restFile.Title || dbFile.ParentID != restFile.ParentID {
t.Errorf("stat mismatch sql=%+v rest=%+v", dbFile, restFile)
}
// GetFile omits contentLength, so size is only comparable when REST has it.
if restFile.Size > 0 && dbFile.Size != restFile.Size {
t.Errorf("size sql=%d rest=%d", dbFile.Size, restFile.Size)
}
if d := dbFile.Modified.Sub(restFile.Modified); d > 2*time.Minute || d < -2*time.Minute {
t.Errorf("modified sql=%v rest=%v", dbFile.Modified, restFile.Modified)
}
list, err := store.List(ctx, folderID)
if err != nil {
t.Fatalf("sql List(%s): %v", folderID, err)
}
if entryByID(list, fileID) == nil {
t.Errorf("file %s not in sql List(%s)", fileID, folderID)
}
// Every file the REST layer can see in the folder must be in the SQL list
// (the SQL store sees more, so only assert this direction).
restList, err := rest.List(ctx, folderID)
if err != nil {
t.Fatalf("rest List(%s): %v", folderID, err)
}
dbIDs := make(map[string]bool, len(list))
for _, e := range list {
dbIDs[e.ID] = true
}
for _, e := range restList {
if e.Kind == File && !dbIDs[e.ID] {
t.Errorf("rest file %s (%q) missing from sql list", e.ID, e.Title)
}
}
// Download reads the object store, not the database, so it only runs with
// the MinIO credentials configured (the portal's S3 layout). Without them
// the DSN-only assertions above still prove the SQL reads.
if os.Getenv("MINIO_ACCESS_KEY") == "" || os.Getenv("MINIO_SECRET_KEY") == "" {
t.Log("MINIO_ACCESS_KEY/MINIO_SECRET_KEY not set — SQL download cross-check skipped")
return
}
var buf bytes.Buffer
n, err := store.Download(ctx, fileID, &buf)
if err != nil {
t.Fatalf("sql Download(%s): %v", fileID, err)
}
if n == 0 || n != dbFile.Size {
t.Errorf("sql Download = %d bytes, stat says %d", n, dbFile.Size)
}
var restBuf bytes.Buffer
rn, err := rest.Download(ctx, fileID, &restBuf)
if err != nil {
t.Fatalf("rest Download(%s): %v", fileID, err)
}
if rn != n || !bytes.Equal(restBuf.Bytes(), buf.Bytes()) {
t.Errorf("download mismatch sql=%d rest=%d bytes", n, rn)
}
}
// TestIntegrationSQLFacade proves that reads are served by the SQL store when
// it is part of the file client, not by REST. It needs ONLYOFFICE_DSN plus
// ONLYOFFICE_PG_TEST_FILE_ID / ONLYOFFICE_PG_TEST_FOLDER_ID and the usual REST
// credentials (for the cross-check). Every Entry served by SQL carries
// Provider "mysql"; REST entries carry "rest", so the provider is the proof of
// which backend answered.
func TestIntegrationSQLFacade(t *testing.T) {
cfg := PGConfigFromEnv()
if strings.TrimSpace(cfg.DSN) == "" {
t.Skip("ONLYOFFICE_DSN not set — skipping SQL facade integration test")
}
fileID := strings.TrimSpace(os.Getenv("ONLYOFFICE_PG_TEST_FILE_ID"))
folderID := strings.TrimSpace(os.Getenv("ONLYOFFICE_PG_TEST_FOLDER_ID"))
if fileID == "" || folderID == "" {
t.Skip("ONLYOFFICE_PG_TEST_FILE_ID / ONLYOFFICE_PG_TEST_FOLDER_ID not set — skipping SQL facade integration test")
}
c := liveClient(t)
ctx := context.Background()
sqlStore, err := c.SQLFileStore()
if err != nil {
t.Fatalf("SQLFileStore: %v", err)
}
if closer, ok := sqlStore.(interface{ Close() error }); ok {
t.Cleanup(func() { _ = closer.Close() })
}
if sqlStore.Name() == ProviderREST {
t.Fatalf("SQLFileStore returned REST")
}
// Direct constructor: Client.FileStore("pg") must not be REST either.
direct := c.FileStore("pg")
if direct == nil || direct.Name() == ProviderREST {
t.Fatalf("FileStore(\"pg\") = %v, want SQL backend", direct)
}
if closer, ok := direct.(interface{ Close() error }); ok {
t.Cleanup(func() { _ = closer.Close() })
}
f := c.Files()
f.RegisterStore(ProviderPG, sqlStore)
if got := f.Read().Name(); got != sqlStore.Name() {
t.Fatalf("facade read backend = %q, want %q (SQL)", got, sqlStore.Name())
}
got, err := f.Stat(ctx, fileID)
if err != nil {
t.Fatalf("facade Stat(%s): %v", fileID, err)
}
if got.Provider != ProviderMySQL {
t.Errorf("facade Stat provider = %q, want %q (SQL, not REST)", got.Provider, ProviderMySQL)
}
want, err := c.FileStore(ProviderREST).Stat(ctx, fileID)
if err != nil {
t.Fatalf("rest Stat(%s): %v", fileID, err)
}
if got.ID != want.ID || got.Title != want.Title || got.ParentID != want.ParentID {
t.Errorf("facade SQL stat %+v != REST %+v", got, want)
}
list, err := f.List(ctx, folderID)
if err != nil {
t.Fatalf("facade List(%s): %v", folderID, err)
}
entry := entryByID(list, fileID)
if entry == nil {
t.Fatalf("file %s not in facade List(%s)", fileID, folderID)
}
if entry.Provider != ProviderMySQL {
t.Errorf("facade List provider = %q, want %q", entry.Provider, ProviderMySQL)
}
d, err := direct.Stat(ctx, fileID)
if err != nil {
t.Fatalf("FileStore(\"pg\").Stat(%s): %v", fileID, err)
}
if d.Provider != ProviderMySQL {
t.Errorf("FileStore(\"pg\") provider = %q, want %q", d.Provider, ProviderMySQL)
}
}
// testPGStoreReadOnly asserts that every write method returns ErrReadOnly.
func testPGStoreReadOnly(ctx context.Context, s *pgStore) error {
if _, err := s.CreateFolder(ctx, "1", "x"); !errors.Is(err, ErrReadOnly) {
return errors.New("CreateFolder did not return ErrReadOnly")
}
if _, err := s.Upload(ctx, "1", "x", strings.NewReader("x")); !errors.Is(err, ErrReadOnly) {
return errors.New("Upload did not return ErrReadOnly")
}
if err := s.Move(ctx, []string{"1"}, "2"); !errors.Is(err, ErrReadOnly) {
return errors.New("Move did not return ErrReadOnly")
}
if err := s.Copy(ctx, []string{"1"}, "2"); !errors.Is(err, ErrReadOnly) {
return errors.New("Copy did not return ErrReadOnly")
}
if err := s.Rename(ctx, "1", "x"); !errors.Is(err, ErrReadOnly) {
return errors.New("Rename did not return ErrReadOnly")
}
if err := s.Delete(ctx, []string{"1"}); !errors.Is(err, ErrReadOnly) {
return errors.New("Delete did not return ErrReadOnly")
}
return nil
}

Some files were not shown because too many files have changed in this diff Show More