Compare commits

..
38 Commits
Author SHA1 Message Date
eSlider 13a4b9f846 Merge pull request 'fix(dav): re-upload converted ext instead of UpdateFile (#84)' (#85) from fix/dav-updatefile-convert#84 into main
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go 1.25) (push) Successful in 1m33s
Tests / Test (Go stable) (push) Successful in 1m34s
2026-09-27 15:12:36 +01:00
eSlider aad6315731 fix(dav): re-upload converted ext instead of UpdateFile (#84)
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 1m30s
Tests / Test (Go 1.25) (pull_request) Successful in 1m31s
UploadToFolderReplacing matched legacy->OOXML counterparts (f.xls vs the
saved f.xlsx) and updated them in place. UpdateFile replaces the body without
re-running OnlyOffice's server-side conversion, so the raw OLE2 .xls landed
under the stored .xlsx name and the file would not open (#84).

UpdateFile now runs only when the stored and local extensions are equal.
A conversion-equivalent match is deleted (all duplicates collapsed) and
uploaded afresh, so the server converts again. AssertNoFileConflict stays
conversion-aware for --no-replace and never had an UpdateFile path.
2026-09-27 15:12:13 +01:00
eSlider 329c0b9561 Merge pull request 'fix(dav): upload --replace matches server-converted ext (xls→xlsx) (#82)' (#83) from fix/dav-replace-convert#82 into main
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 6s
Tests / Test (Go 1.25) (push) Successful in 1m5s
Tests / Test (Go stable) (push) Successful in 1m13s
2026-09-27 15:06:16 +01:00
eSlider 74e72d0e06 fix(dav): upsert upload matches server-converted ext (xls→xlsx) (#82)
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 5s
Tests / Test (Go 1.25) (pull_request) Successful in 1m3s
Tests / Test (Go stable) (pull_request) Successful in 1m11s
OnlyOffice converts legacy binary Office uploads (.xls/.doc/.ppt) into
OOXML (.xlsx/.docx/.pptx) on the server. UploadToFolderReplacing matched
by the exact stem|ext via FindFilesByDedupKey, so a repeated
`oo dav upload FOLDER f.xls --replace` never found the stored f.xlsx and
appended a second file (live: ids 3799+3887, 3800+3888).

- EquivalentUploadExt / FindFilesByStemExt: match by stem with a
  legacy↔OOXML extension equivalence, so .xls finds the saved .xlsx.
- planUploadReplacement: pick the surviving file and the redundant
  duplicate ids for a replacing upload.
- UploadToFolderReplacing updates the existing file in place (UpdateFile,
  stable id, no delete window), removes extra duplicates, and falls back
  to conversion-aware delete + upload when the portal rejects the update.
- AssertNoFileConflict (--no-replace) uses the same conversion-aware
  matching so a .xls upload conflicts with an existing .xlsx.
- Offline tests cover the matcher, the plan and the repeated-.xls
  regression; pdf/xlsx behaviour unchanged.
2026-09-27 15:05:13 +01:00
eSlider 534661dec7 Merge pull request 'feat(dav): oo dav upload (локальный файл) + ensure-path (mkdir -p) (#80)' (#81) from feat/dav-upload#80 into main
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 6s
Tests / Test (Go stable) (push) Successful in 1m10s
Tests / Test (Go 1.25) (push) Successful in 1m15s
2026-09-25 15:08:36 +01:00
eSlider 55356558a1 feat(dav): oo dav upload (local file) + ensure-path (mkdir -p) in Documents (#80)
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 5s
Tests / Test (Go stable) (pull_request) Successful in 1m9s
Tests / Test (Go 1.25) (pull_request) Successful in 1m11s
2026-09-25 15:07:57 +01:00
eSlider aa069bea56 Merge pull request 'chore(release): v0.21.0 (#78)' (#79) from chore/release-v0.21.0#78 into main
Release / GoReleaser (push) Skipped
Tests / Test (Go 1.25) (push) Successful in 1m23s
Tests / Secret scan (gitleaks) (push) Successful in 3s
Tests / Test (Go stable) (push) Successful in 1m28s
2026-09-23 18:25:08 +01:00
eSlider a8fa202496 chore(release): v0.21.0 (#78)
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 5s
Tests / Test (Go 1.25) (pull_request) Successful in 1m29s
Tests / Test (Go stable) (pull_request) Successful in 1m33s
2026-09-23 18:25:00 +01:00
eSlider 87ff317166 Merge pull request 'chore: move business tooling out of the public library; tidy filestore naming' (#77) from chore/split-business-tools into main
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go 1.25) (push) Successful in 1m25s
Tests / Test (Go stable) (push) Successful in 1m28s
2026-09-23 09:05:48 +01:00
eSlider dc62570a5c chore: move business tooling out of the public library; tidy filestore naming
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 1m19s
Tests / Test (Go 1.25) (pull_request) Successful in 1m24s
Keep the public tree project-generic. Business/one-off tools, deployment
and business docs move to the private oo-workspace repo.

Moved to oo-workspace:
- cmd/ooscan, cmd/pdfamount, cmd/kontoblatt, cmd/kontolink
- internal/xlspipe (cutover-portugal workbook) -> oow workbook build
  (drops the --template/--title flags from oo docs put-xlsx)
- deploy/docker-compose.rclone-webdav.yml + docs/rclone-webdav.md
- docs/crm-associations.md

Removed GitHub-era leftovers:
- .github/workflows/release-please.yml, release-please-config.json,
  .release-please-manifest.json (tags are created on Gitea per SemVer)

Naming: the FileStore subsystem is now filestore_*.go (was file_*.go) to
match files_*.go (project Documents). Docs/AGENTS/README updated.
2026-09-23 09:05:40 +01:00
eSlider a9a77e110d Merge pull request 'feat: upstream generic workspace tooling (board-sync, crm audit, catalog names)' (#76) from feat/upstream-oow-workspace into main
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go stable) (push) Successful in 1m23s
Tests / Test (Go 1.25) (push) Successful in 1m23s
2026-09-22 23:13:49 +01:00
eSlider 82da45dc4d feat: upstream generic workspace tooling (board-sync, crm audit, catalog names)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 5s
Tests / Test (Go 1.25) (pull_request) Successful in 1m16s
Tests / Test (Go stable) (pull_request) Successful in 1m36s
Promote the generic, reusable parts of the private oo-workspace into the
public library/CLI, so oo-workspace can shrink to business glue.

- board.go: Board types + (c *Client) SyncBoard — upsert project
  milestones/tasks by exact title from a YAML board (dry-run when
  apply=false). CLI: oo projects board-sync.
- crm_audit.go: OpportunityAudit + (c *Client) AuditOpportunities —
  file/task/member counts with a generic class (ok|dup|empty|junk-title).
  CLI: oo crm audit [--out].
- catalog/names.go: CleanPersonNames / GuessNameFromEmail /
  FormatProjectTitle (ported from oo-workspace).
- catalog: adopt the newer apply/match logic — name repair (never encode
  company in lastName), company grouping key, contact-info type
  normalization, preserve an already-applied oo_id. Keeps the local
  Address/OOProjects fields and the config-driven classifier.

Business-only oo-workspace commands (crm clean/migrate, folders, search)
deliberately stay private.
2026-09-22 23:13:40 +01:00
eSlider 8848ee00f1 Merge pull request 'feat(oo): native conversion, deep links, sheet export (re-land)' (#75) from feat/oo-conversion-links into main
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go 1.25) (push) Successful in 1m29s
Tests / Test (Go stable) (push) Successful in 1m30s
2026-09-22 22:42:07 +01:00
eSlider ff46aad615 chore: keep the re-landed feature files showroom-safe
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 1m21s
Tests / Test (Go 1.25) (pull_request) Successful in 1m27s
The conversion / link / sheet commits predate the tree cleanup, so their
comments and README/test examples still carried internal refs. Re-apply
the genericization (no internal hosts, personal names or client file ids).
2026-09-22 22:41:58 +01:00
eSlider 5e73f6c335 feat(oo): sheet-aware spreadsheet export (docs csv/json)
- lib: WorkbookSheetNames/CSV/JSON (excelize) — sheet selection + JSON rows
- oo docs csv  SRC... [--sheet N] [--delimiter ,|;|||tab] [--out]
- oo docs json SRC... [--sheet N] [--out]
- SRC = OO file id or local path; --sheet is 1-based (default first)
- DS csv output is first-sheet-only (verified) -> local reader for selection/JSON
- test: TestWorkbookSheetExport
2026-09-22 22:41:14 +01:00
eSlider 7c089a6439 feat(oo): docs pdf --stream/--pipe (converted bytes to stdout) 2026-09-22 22:41:14 +01:00
eSlider 4515a88fe8 docs(readme): examples for links, conversion, team/users CRUD 2026-09-22 22:41:14 +01:00
eSlider 58b905cc49 feat(oo): native document conversion (docs pdf/presigned)
- lib: PresignedURI, SignJWT (HS256, stdlib), ConvertDocument (DocumentServer
  /converter; legacy /ConvertService.ashx), DownloadURLTo
- oo docs pdf FILE_ID|PATH...: OO file ids or local files (temp upload to
  --folder, convert, download, cleanup) -> PDF/other
- oo docs presigned FILE_ID
- docs base from $ONLYOFFICE_DOCS_URL else $ONLYOFFICE_URL/ds-vpath; JWT secret
  from $ONLYOFFICE_DS_SECRET (DocumentServer CoAuthoring secret)
- test: SignJWT
2026-09-22 22:41:14 +01:00
eSlider 520d6d05c8 docs(oo): comment where the new helpers are used
- link.go/links.go: third-party doc packs (fileid links), id-stability caveat
- auth.go AuthenticateAs / users check: email-vs-userName login finding
- projects files replace-in: version-history rationale
2026-09-22 22:41:14 +01:00
eSlider df38b75011 feat(oo): file deep links, login check, replace-in (fresh upload)
- link: print Products/Files/DocEditor.aspx?fileid=… deep links (title+url)
- lib: FileEditorURL/FolderURL helpers (+tests)
- users check: verify credentials (userName vs email) via authentication.json
- projects files replace-in FOLDER_ID FILE...: hard delete same stem|ext in a
  folder + fresh upload → single clean version (no version history)
2026-09-22 22:41:14 +01:00
eSlider ee8e88fb9b feat(oo): projects files update (overwrite existing file content) 2026-09-22 22:41:14 +01:00
eSlider e323a43317 feat(oo): projects milestone-delete 2026-09-22 22:41:14 +01:00
eSlider 71d421fb41 Merge pull request 'chore: keep host/client specifics out of the tree (env & config)' (#74) from chore/showroom-generic into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go 1.25) (push) Successful in 55s
Tests / Test (Go stable) (push) Successful in 1m6s
2026-09-22 22:40:49 +01:00
eSlider afb93feae5 chore: keep host/client specifics out of the tree (env & config)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 59s
Tests / Test (Go stable) (pull_request) Successful in 1m7s
Showroom-safe: the tree no longer carries internal hosts, IPs, ports,
personal names, client domains or client file names. Behaviour is
unchanged and now supplied per deployment.

- catalog: hardcoded mail-org / project classifiers become a YAML-driven
  Classifier (OO_CATALOG_CONFIG or --config); neutral default classifies
  nothing as work. New catalog/classify.go + example + tests.
- storage_fallback: drop the baked-in MinIO endpoint IP; require
  MINIO_ENDPOINT (+ keys) from the env.
- kontolink: build DocEditor links from $ONLYOFFICE_URL instead of a
  hardcoded portal host; kontoblatt: no client file id in the output name.
- oo: load .env CLI-wide (bootstrap.LoadEnv in execute) so
  non-authenticating commands (catalog scans) also see config.
- genericize comments/docs/fixtures (AGENTS, README, .env.example,
  crm-associations, catalog tests, ES/pdfattach tests, mails).
2026-09-22 22:03:36 +01:00
eSlider b493472f2e Merge pull request 'feat(oo): project team CRUD and user lifecycle' (#73) from feat/oo-team-users-crud into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go stable) (push) Successful in 1m1s
Tests / Test (Go 1.25) (push) Successful in 1m2s
Reviewed-on: #73
2026-09-22 14:29:47 +01:00
eSlider 1dc99d6997 feat(oo): project team CRUD and user lifecycle (block/unblock/password/delete)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 1m2s
Tests / Test (Go stable) (pull_request) Successful in 1m40s
- project team: oo projects team list|add|remove|set (portal users)
- users: get|create|update|delete|block|unblock|password
- lib: CreateUser, DeleteUser (auto-suspend before delete), Add/Remove/
  SetProjectTeam, ListProjectTeam; deleteJSON reused; jsonBodyReader helper
- tests: unmarshalResponseArray
2026-09-22 14:26:23 +01:00
eSlider 9b21d7e52a Merge pull request 'fix(crm): адреса контактов через ContactInfo типа Address (#288)' (#72) from feat/catalog-addresses#288 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go stable) (push) Successful in 58s
Tests / Test (Go 1.25) (push) Successful in 59s
2026-09-22 08:32:25 +01:00
eSlider c4beba5173 fix(crm): store contact addresses via ContactInfo Address type (#288)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go stable) (pull_request) Successful in 1m1s
Tests / Test (Go 1.25) (pull_request) Successful in 1m1s
2026-09-22 07:08:01 +00:00
eSlider 2a187ae132 Merge pull request 'fix(retry): глобальный rate-limit + Retry-After + cooldown (#70)' (#71) from fix/oo-backoff#70 into main
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 5s
Tests / Test (Go stable) (push) Successful in 21s
Tests / Test (Go 1.25) (push) Successful in 22s
2026-09-17 13:39:56 +01:00
eSlider c9e16c7169 fix(retry): глобальный rate-limit + Retry-After + cooldown (#70)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 5s
Tests / Test (Go 1.25) (pull_request) Successful in 25s
Tests / Test (Go stable) (pull_request) Successful in 27s
2026-09-17 12:39:25 +00:00
eSlider 194ae62720 Merge pull request 'docs: убрать потребительские детали match из index-and-search (#34)' (#69) from docs/index-and-search-scope into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go stable) (push) Successful in 26s
Tests / Test (Go 1.25) (push) Successful in 26s
2026-09-17 08:51:45 +01:00
eSlider 0d73146ebc docs: убрать потребительские детали match из index-and-search (#34)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 3s
Tests / Test (Go 1.25) (pull_request) Successful in 19s
Tests / Test (Go stable) (pull_request) Successful in 29s
2026-09-17 07:51:02 +00:00
eSlider 46661f51f8 Merge pull request 'docs: карта поиска и обновления индексов (#34)' (#68) from docs/index-and-search into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go stable) (push) Successful in 25s
Tests / Test (Go 1.25) (push) Successful in 29s
2026-09-17 08:49:31 +01:00
eSlider f22d714be7 docs: карта поиска и обновления индексов (#34)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 19s
Tests / Test (Go stable) (pull_request) Successful in 21s
2026-09-17 07:48:40 +00:00
eSlider 6f85c206cf Merge pull request 'test(files): live-тест файлового dedup (#63)' (#65) from test/w6-file-dedup#63 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Test (Go stable) (push) Failing after 3s
Tests / Secret scan (gitleaks) (push) Successful in 4s
Tests / Test (Go 1.25) (push) Successful in 28s
2026-09-16 23:03:23 +01:00
eSlider 1c00edbd0b Merge pull request 'test(files): live CRUD папок + фасадный CRUD (#62)' (#66) from test/w5-folder-crud#62 into main
Release / GoReleaser (push) Skipped
Release Please / Release Please (push) Skipped
Tests / Test (Go stable) (push) Failing after 4s
Tests / Secret scan (gitleaks) (push) Successful in 6s
Tests / Test (Go 1.25) (push) Successful in 23s
2026-09-16 23:03:17 +01:00
eSlider 9cd157c59f test(files): live CRUD папок + фасадный CRUD (#62)
Release Please / Release Please (push) Skipped
Release / GoReleaser (push) Skipped
Tests / Secret scan (gitleaks) (push) Skipped
Tests / Test (Go 1.25) (push) Skipped
Tests / Test (Go stable) (push) Skipped
Tests / Test (Go stable) (pull_request) Failing after 4s
Tests / Secret scan (gitleaks) (pull_request) Successful in 4s
Tests / Test (Go 1.25) (pull_request) Successful in 19s
2026-09-16 21:49:10 +00:00
eSlider 932cd2895a fix(files): REST FileStore resolves folders for stat/rename/move/delete (#62) 2026-09-16 21:49:08 +00:00
107 changed files with 5146 additions and 2625 deletions
+8 -3
View File
@@ -34,13 +34,18 @@ ONLYOFFICE_PROJECT_ID=33
# MinIO download fallback for the portal's stale AWS S3 consumer (older
# Documents folders). When the portal redirects to amazonaws.com with access
# key "minio" (403 InvalidAccessKeyId), files are fetched from the local MinIO
# store instead. Without a key/secret the fallback is disabled.
# MINIO_ENDPOINT=http://192.168.188.10:9000
# key "minio" (403 InvalidAccessKeyId), files are fetched from the configured
# MinIO store instead. Without endpoint + key/secret the fallback is disabled.
# MINIO_ENDPOINT=http://minio.example.com:9000
# MINIO_BUCKET=office
# MINIO_ACCESS_KEY=
# MINIO_SECRET_KEY=
# Catalog scan classification rules (client mail domains, project remotes/names).
# Copy catalog/classify.example.yaml and point this at it; no rules means
# nothing is classified as work.
# OO_CATALOG_CONFIG=~/.config/oo/catalog-classify.yaml
# oo search — direct Elasticsearch access for name + content search. ES lives
# inside the OnlyOffice VM on localhost:9200; expose it with an SSH tunnel
# (see docs/elasticsearch.md). ONLYOFFICE_ES_INDEX defaults to files_file.
-42
View File
@@ -1,42 +0,0 @@
name: Release Please
on:
push:
branches: [main]
workflow_dispatch:
permissions:
contents: write
pull-requests: write
issues: write
actions: write
jobs:
release-please:
name: Release Please
if: github.server_url == 'https://github.com'
runs-on: ubuntu-latest
steps:
- name: Run release-please
uses: googleapis/release-please-action@v4
id: release
with:
config-file: release-please-config.json
manifest-file: .release-please-manifest.json
token: ${{ secrets.RELEASE_PLEASE_TOKEN != '' && secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
# Tags/releases created with GITHUB_TOKEN do not trigger other workflows
# (push:tags on Release.yml never fires). Dispatch GoReleaser explicitly.
- name: Trigger GoReleaser
if: ${{ steps.release.outputs.release_created == 'true' }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG: ${{ steps.release.outputs.tag_name }}
run: |
set -euo pipefail
echo "Dispatching Release workflow for $TAG"
gh workflow run release.yml --repo "${{ github.repository }}" -f "tag=${TAG}"
outputs:
release_created: ${{ steps.release.outputs.release_created }}
tag_name: ${{ steps.release.outputs.tag_name }}
+2 -2
View File
@@ -20,8 +20,8 @@ jobs:
runs-on: ubuntu-latest
steps:
# Always clone default branch first. workflow_dispatch often races with
# release-please creating the tag; fetching refs/tags/X before it exists
# fails checkout. Retry fetch, then check out the tag.
# tag creation; fetching refs/tags/X before it exists fails checkout.
# Retry fetch, then check out the tag.
- uses: actions/checkout@v4
with:
fetch-depth: 0
-3
View File
@@ -1,3 +0,0 @@
{
".": "0.18.0"
}
+14 -9
View File
@@ -9,19 +9,19 @@ Canonical Go client for OnlyOffice Workspace (Projects + Calendar + CRM) and the
- `request.go` — `Request`, `Query`, `Time`, `Token`, `MetaResponse`, `Permissions`.
- `auth.go` — `Authenticate`, `AuthenticateContext`, `InvalidateToken`, `Auth`, token lifecycle.
- `http.go` — transport + DRY response decoders (`ResponseArray`/`ResponseObject`/`postFormObject`/`putFormObject`/`deleteObject`).
- `projects.go`, `tasks.go`, `users.go`, `calendar.go`, `crm.go`, `files.go`, `files_webdav.go`, `files_stem.go`, `retry.go`, `mails.go`, `invoices.go` — typed / untyped domain methods. **`files.go`** — CRM opportunity upload plus **project/task Documents** (`UpdateFile`, `UploadToFolderReplacing`). **`files_webdav.go`** — Documents module by id (`ListDavFolder`, `MoveDavItems`/`CopyDavItems` with per-operation error surfacing, `ListFileOps`). **`retry.go`** — `DoRetry`: deterministic linear backoff (no jitter) on 429/502/503/504; every bulk tool routes API calls through it, and the HTTP transport + auth (`retryRaw`, `AuthenticateContext`) retry transient answers centrally. **`mails.go`** — OnlyOffice Workspace Mail. **`invoices.go`** — CRM invoices, PDF regen/cleanup, status. Association rules: [`docs/crm-associations.md`](docs/crm-associations.md).
- **Unified file client (epic #34) — `file_core.go`, `file_rest.go`, `file_dav.go`, `file_pg.go`, `file_es.go`, `file_es_text.go`, `file_text_index.go`, `file_facade.go`.** `file_core.go` — model (`Entry`, `Kind`) + `FileStore`/`Searcher`; `file_rest.go`/`file_dav.go` — REST/WebDAV adapters; `file_pg.go` — **read-only** SQL store (PostgreSQL/MySQL, `ErrReadOnly` on writes); `file_es.go` — OnlyOffice Elasticsearch searcher; `file_es_text.go`/`file_text_index.go` — own PDF/scan index (`oo_docs_text`, PDF attachments via pdfdetach); `file_facade.go` — `FileClient` with read/write/search order and transient fallback. Use `c.Files()` (facade), `c.FileStore("rest"|"dav"|"pg"|"sql")` or `c.SQLFileStore()`; contract and how to add a backend: [`docs/unified-file-client.md`](docs/unified-file-client.md).
- `projects.go`, `tasks.go`, `users.go`, `calendar.go`, `crm.go`, `files.go`, `files_webdav.go`, `files_stem.go`, `retry.go`, `mails.go`, `invoices.go` — typed / untyped domain methods. **`files.go`** — CRM opportunity upload plus **project/task Documents** (`UpdateFile`, `UploadToFolderReplacing`). **`files_webdav.go`** — Documents module by id (`ListDavFolder`, `MoveDavItems`/`CopyDavItems` with per-operation error surfacing, `ListFileOps`). **`retry.go`** — `DoRetry`: deterministic exponential backoff (no jitter) on 429/502/503/504; every bulk tool routes API calls through it, and the HTTP transport + auth (`retryRaw`, `AuthenticateContext`) retry transient answers centrally. `ratelimit.go` adds a process-wide token bucket (`OO_RATE_LIMIT`/`OO_BURST`) and a shared 429 cooldown gate, installed via `pacedTransport` in `NewClient`; `Retry-After` is parsed into `*TransientError` and honoured. See [`docs/rate-limiting.md`](docs/rate-limiting.md). **`mails.go`** — OnlyOffice Workspace Mail. **`invoices.go`** — CRM invoices, PDF regen/cleanup, status. Association rules live with the private `oo-workspace` tooling.
- **Unified file client (epic #34) — `filestore_core.go`, `filestore_rest.go`, `filestore_dav.go`, `filestore_pg.go`, `filestore_es.go`, `filestore_es_text.go`, `filestore_text_index.go`, `filestore_facade.go`.** `filestore_core.go` — model (`Entry`, `Kind`) + `FileStore`/`Searcher`; `filestore_rest.go`/`filestore_dav.go` — REST/WebDAV adapters; `filestore_pg.go` — **read-only** SQL store (PostgreSQL/MySQL, `ErrReadOnly` on writes); `filestore_es.go` — OnlyOffice Elasticsearch searcher; `filestore_es_text.go`/`filestore_text_index.go` — own PDF/scan index (`oo_docs_text`, PDF attachments via pdfdetach); `filestore_facade.go` — `FileClient` with read/write/search order and transient fallback. Use `c.Files()` (facade), `c.FileStore("rest"|"dav"|"pg"|"sql")` or `c.SQLFileStore()`; contract and how to add a backend: [`docs/unified-file-client.md`](docs/unified-file-client.md).
- Pure stdlib + `google/go-querystring`; no UI, no dotenv.
- **CLI — `cmd/oo/` as `package main`.** Cobra wrapper that loads `.env` via `godotenv` at startup. **Subject-based command tree** mirroring [`tea`](https://gitea.com/gitea/tea):
- `main.go` — entry point (docstring lists the command tree).
- `common.go` — `rootCmd`, `newOO`, `printTable`/`printObject`, `--output table|json` flag.
- `calendar.go`, `projects.go`, `projects_files.go`, `tasks.go`, `tasks_files.go`, `users.go`, `contacts.go`, `opportunities.go`, `cases.go`, `crm.go`, `crm_tasks.go`, `catalog.go`, `docs.go`, `dav.go`, `search.go`, `index.go`, `mails.go`, `invoices.go` — one file per subject (or per subject facet), each registers in `init()`. `dav.go` exposes the Documents module by id (`oo dav ls|move|copy|mkdir|rename-file|rename-folder|download|fileops`); `search.go` runs `oo search QUERY` (name/content, `--backend oo|own`); `index.go` fills the own full-text index (`oo index folder|files`, see [`docs/unified-file-client.md`](docs/unified-file-client.md)).
- `calendar.go`, `projects.go`, `projects_files.go`, `tasks.go`, `tasks_files.go`, `users.go`, `contacts.go`, `opportunities.go`, `cases.go`, `crm.go`, `crm_tasks.go`, `catalog.go`, `docs.go`, `dav.go`, `search.go`, `index.go`, `mails.go`, `invoices.go` — one file per subject (or per subject facet), each registers in `init()`. `dav.go` exposes the Documents module by id (`oo dav ls|move|copy|mkdir|ensure-path|upload|rename-file|rename-folder|download|fileops`); `search.go` runs `oo search QUERY` (name/content, `--backend oo|own`); `index.go` fills the own full-text index (`oo index folder|files`, see [`docs/unified-file-client.md`](docs/unified-file-client.md)).
- CLI-only deps (`spf13/cobra`, `joho/godotenv`) stay out of the library.
- **TUI — `cmd/office/` as `package main`.** Bubble Tea three-pane browser (module tree, selectable list, markdown preview). Reuses `cmd/internal/bootstrap` for env/auth and the root `onlyoffice` library for all API calls. UI logic in `cmd/office/ui/`; preview/formatting in `cmd/office/preview/`; list loaders in `cmd/office/fetch/`.
- **List table (`DataTable`)** — `cmd/office/ui/table*.go`. Column layout policies live in `cmd/office/model/table_layout.go` (`TableFlexLayoutFor`); cell rendering uses the bubbles/table inline pattern in `table_render.go` (`renderTableCell`, `padANSIWidth`). See `.cursor/skills/office-tui-table/SKILL.md` before changing center-pane tables.
- **Shared bootstrap — `cmd/internal/bootstrap/`.** `LoadEnv()` + `NewClient(ctx)` extracted from `oo`; both binaries import it.
- **Bulk Documents tools — `cmd/ooscan/`, `cmd/pdfamount/`, `cmd/kontoblatt/`, `cmd/kontolink/`.** Single-purpose binaries (folder index, PDF amounts, Kontoblatt summary/linking). Pace requests, route API calls through `DoRetry`; usage in README.
- **Personal ops tooling** (disk inventory, dossier→CRM sync, SearXNG) lives in private [`eSlider/oo-workspace`](https://git.produktor.io/eSlider/oo-workspace) (`oow`), not in this public tree.
- **Bulk Documents tools live in the private `oo-workspace` repo** (`ooscan`, `pdfamount`, `kontoblatt`, `kontolink`), not in this public tree. They use the public client and its `DoRetry` pacing.
- **Personal ops tooling** (disk inventory, dossier→CRM sync, SearXNG) lives in a private companion repo `eSlider/oo-workspace` (the `oow` CLI), not in this public tree.
## Rules
@@ -30,9 +30,14 @@ Canonical Go client for OnlyOffice Workspace (Projects + Calendar + CRM) and the
- Prefer `ResponseObject` / `postFormObject` / `putFormObject` / `deleteObject` over hand-rolled `json.Unmarshal(responseField(...))` blocks — they exist for DRY, use them.
- Domain split is by file, **not** by subpackage. Don't introduce `internal/` or `pkg/*` subpackages inside the library — it flattens the `*Client` call surface for a reason.
- CLI commands follow **subject → verb** structure (`oo <subject> <verb>`), never `oo <verb>-<subject>`. Add new commands to the existing subject file if one fits; create a new `cmd/oo/<subject>.go` for a genuinely new domain. The subject→verb tree in `cmd/oo/main.go` and the README table are documentation — update them with the code.
- **Documents for agents:** prefer Markdown in git; OnlyOffice UI is weak for `.md`/`.txt`. Use `oo docs put-md` (md→docx) and `oo docs put-txt` (txt→docx, preserves line breaks). All upload paths default to **upsert** by `stem|ext` (`--replace`, default true); `--no-replace` fails on conflict; `--allow-duplicate` opts into raw OO append. `oo projects files dedupe PROJECT_ID` reports/removes duplicate stem|ext copies (`--apply`, `--cross`; includes project root folder).
- **Documents for agents:** prefer Markdown in git; OnlyOffice UI is weak for `.md`/`.txt`. Use `oo docs put-md` (md→docx) and `oo docs put-txt` (txt→docx, preserves line breaks). All upload paths default to **upsert** by `stem` with the extension matched conversion-aware (legacy `.xls/.doc/.ppt` ↔ OOXML `.xlsx/.docx/.pptx`, since OnlyOffice converts them on upload), so a repeated `.xls` upload updates the saved `.xlsx` instead of appending a duplicate (`--replace`, default true); `--no-replace` fails on conflict; `--allow-duplicate` opts into raw OO append. `oo projects files dedupe PROJECT_ID` reports/removes duplicate stem|ext copies (`--apply`, `--cross`; includes project root folder).
- Every table output goes through `printTable(headers, rows)`; every single-object through `printObject(v)`. Do not `fmt.Println` rows ad-hoc or the `--output json` flag breaks for that command.
- No secrets in the repo; use `.env` (gitignored). Commit `.env.example` only.
- **No host/client specifics in the tree.** Endpoints, IPs/ports, client mail
domains, project remotes/names and personal names stay out of source and
fixtures — they come from env/config (`MINIO_*`, `OO_CATALOG_CONFIG`, see
[`catalog/classify.example.yaml`](catalog/classify.example.yaml)). This repo is
mirrored to GitHub as a public showroom, so the tree must stay project-generic.
- Follow SemVer on tags; this repo is tagged at GitHub under `git@github.com:eSlider/go-onlyoffice.git`.
### Testing policy (2026-04-24)
@@ -57,6 +62,6 @@ write `mux.HandleFunc("/api/2.0/...")` to emulate OnlyOffice, we write an
## Related
- [`docs/README.md`](docs/README.md) — reference index (file client, ES, SQL, rclone).
- [`eSlider/inventar`](https://git.produktor.io/eSlider/inventar) — ASR/ADR (see ASR-0008 Go library module conventions).
- [`eSlider/inventar-sync`](https://git.produktor.io/eSlider/inventar-sync) — OnlyOffice → Gitea issue sync, consumes this library.
- [`produktor.io/vidarr`](https://git.produktor.io/produktor.io/vidarr) — legacy consumer being migrated from `pkg/onlyoffice` to this module.
- `eSlider/inventar` — ASR/ADR (see ASR-0008 Go library module conventions).
- `eSlider/inventar-sync` — OnlyOffice → Gitea issue sync, consumes this library.
- `vidarr` — legacy consumer being migrated from `pkg/onlyoffice` to this module.
+22 -1
View File
@@ -4,7 +4,28 @@ All notable changes to this project are documented here. The format is based on
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project
adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## Unreleased
## [0.21.0](https://git.produktor.io/eSlider/go-onlyoffice/compare/v0.20.0...v0.21.0) (2026-09-23)
### Features
* **retry:** global token-bucket rate limit (`OO_RATE_LIMIT`/`OO_BURST`), typed
`TransientError` with `Retry-After`, exponential backoff
(`OO_RETRY_ATTEMPTS`/`_BASE`/`_MAX`) and a process-wide 429 cooldown gate.
All HTTP paths are paced via `pacedTransport` (#70).
* **oo:** native document conversion (`oo docs pdf`), deep links, sheet-aware
export (`oo docs csv`/`json`), `oo docs presigned`, plus project team and
user lifecycle, milestone-delete and file update/replace (#73, #75).
* **catalog:** upstream generic workspace tooling (board-sync, CRM audit,
catalog names) (#76).
### Chores
* move business tooling (`ooscan`, `pdfamount`, `kontoblatt`, `kontolink`) out
of the public library into the private `oo-workspace` repo; tidy filestore
naming (#77).
* keep host/client specifics out of the tree (#74).
## [0.18.0](https://github.com/eSlider/go-onlyoffice/compare/v0.17.0...v0.18.0) (2026-09-04)
+111 -35
View File
@@ -476,6 +476,11 @@ type Task struct {
| `UpdateProject(req)` | Update project details |
| `DeleteProject(id)` | Delete a project |
| `GetProjectMilestones(project)` | Get milestones with task counts |
| `DeleteMilestone(id)` | Remove a milestone |
| `ListProjectTeam(ctx, id)` | Portal users on the project team |
| `AddProjectTeamUser(ctx, id, userID)` | Add a portal user to the team |
| `RemoveProjectTeamUser(ctx, id, userID)` | Remove a portal user from the team |
| `SetProjectTeam(ctx, id, participants, notify)` | Replace the team with the given user ids |
### Tasks
@@ -502,6 +507,14 @@ type Task struct {
| Method | Description |
|---|---|
| `GetUsers()` | List all users with profiles |
| `GetUser(ctx, id)` | One user profile by id |
| `CreateUser(ctx, NewUserRequest)` | Add a portal user |
| `UpdateUser(ctx, id, body)` | Update profile fields (JSON PUT) |
| `DeleteUser(ctx, id)` | Delete permanently (auto-terminates first — OO refuses active users) |
| `BlockUser(ctx, id)` / `UnblockUser(ctx, id)` | Terminate / reactivate (login kept/denied) |
| `ChangeUserPassword(ctx, id, pw)` | Set a new password |
| `ChangeUserStatus(ctx, id, active)` | Activate / Terminate via `people/status` |
| `AuthenticateAs(ctx, login, pw)` | Verify a login without mutating the cached token |
### Documents Files
@@ -522,9 +535,33 @@ type Task struct {
| `ListFileOps(ctx)` | Active file operations (move/copy status polling) |
| `FolderFiles(ctx, folderID)` | Flat file list of a folder (stem helpers) |
| `DeleteFilesByStem(ctx, folderID, stem)` | Remove `stem\|ext` copies |
| `DoRetry(ctx, policy, fn)` | Deterministic linear backoff (N·Base, no jitter) on 429/502/503/504 |
| `DefaultRetryPolicy()` | 5 attempts, 1s·2s·3s·4s waits, 30s cap |
| `Transient(err)` | True for retriable OnlyOffice answers |
| `DoRetry(ctx, policy, fn)` | Deterministic exponential backoff (`Base·2^(N-1)`, no jitter) on 429/502/503/504; honours `Retry-After` and the process-wide cooldown gate |
| `DefaultRetryPolicy()` | From env: 7 attempts, 2s base, 2m cap (`OO_RETRY_ATTEMPTS/_BASE/_MAX`) |
| `Transient(err)` | True for retriable OnlyOffice answers (`*TransientError` or HTTP 429/502/503/504 text) |
### Deep links & conversion
| Method | Description |
|---|---|
| `FileEditorURL(portalBase, id)` / `(c *Client).FileEditorURL(id)` | DocEditor deep link `/Products/Files/DocEditor.aspx?fileid=` |
| `FolderURL(portalBase, id)` / `(c *Client).FolderURL(id)` | Documents folder link |
| `PresignedURI(ctx, fileID)` | Short-lived fetchable URL of a portal file (`presigneduri`) |
| `SignJWT(secret, payload)` | HS256 JWT, stdlib only |
| `ConvertDocument(ctx, docsBase, secret, req)` | OnlyOffice DocumentServer conversion (`/converter`; legacy `/ConvertService.ashx`) |
| `DownloadURLTo(ctx, url, w)` | Stream an absolute URL into a writer |
| `SyncBoard(ctx, board, apply)` | Upsert project milestones/tasks from a YAML board (dry-run when apply=false) |
| `AuditOpportunities(ctx)` | Opportunities with file/task/member counts and a coarse class |
| `WorkbookSheetNames(data)` | Worksheet names of an XLS/XLSX/ODS workbook |
| `WorkbookSheetCSV(data, sheet, delim)` | One worksheet → CSV (sheet-aware, excelize) |
| `WorkbookSheetJSON(data, sheet)` | One worksheet → rows as objects (first row = header) |
Example — convert a portal file (or a local file) to PDF with the native engine:
```bash
export ONLYOFFICE_DS_SECRET=<DocumentServer CoAuthoring secret>
oo docs pdf 1234 --out out.pdf # OO file id → PDF
oo docs pdf ./report.docx --to pdf # local file → PDF (temp upload, auto-cleanup)
```
### Helper Types
@@ -582,6 +619,41 @@ oo opportunities list
oo opportunities stages
oo cases list
oo crm-tasks categories
# Deep links & native document conversion
oo link 1234 2345 # DocEditor URL for file ids (title + url)
oo docs presigned 1234 # short-lived fetchable URL of an OO file
oo docs pdf 1234 --out out.pdf # OO file → PDF (via DocumentServer)
oo docs pdf ./report.docx --to pdf # local file → PDF on the fly (temp upload+cleanup)
oo docs pdf 1234 --stream > out.pdf # pipe: bytes to stdout (alias --pipe)
# Spreadsheet export (sheet-aware; local reader — the DS csv output is first-sheet-only)
oo docs csv 1234 --sheet 2 # XLS/XLSX/ODS worksheet → CSV (--sheet N, 1-based)
oo docs csv 1234 --delimiter ';' # ; | | \t via --delimiter
oo docs json 1234 --sheet 2 # worksheet → JSON rows (first row = header)
oo docs csv ./book.xlsx --sheet 1 --out sheet1.csv
# export ONLYOFFICE_DS_SECRET=<DocumentServer CoAuthoring secret>
# docs base: $ONLYOFFICE_DOCS_URL, else $ONLYOFFICE_URL + /ds-vpath
# Project team (portal users) CRUD
oo projects team list 42
oo projects team add 42 <user_id> [<user_id>...]
oo projects team remove 42 <user_id>
oo projects team set 42 <user_id> [...] # replace team (may lag; verify with list)
oo projects milestone-delete 7
# Project documents: fresh re-upload (single clean version) / new version
oo projects files replace-in 1 ./contract.pdf # hard delete same stem|ext in folder + upload
oo projects files update 1 ./contract.docx # overwrite content, same file id
# Users lifecycle
oo users list ; oo users get <user_id>
oo users create --first Jane --last Doe --email jane.doe@example.com --password '…'
oo users check --login jane.doe@example.com # verify login (email works when userName 500s)
oo users update <user_id> --title "…" --location "…"
oo users block <user_id> ; oo users unblock <user_id>
oo users password <user_id> # reads the new password from stdin
oo users delete <user_id> [<user_id>...]
```
### office (TUI)
@@ -669,6 +741,10 @@ surfaces such per-operation errors instead of a silent nil
oo dav ls 659
oo dav ls @root # virtual sections (Documents, Projects, …)
oo dav mkdir 659 "2026 inbox"
oo dav ensure-path "Banks/Caixa" # resolve-or-create; idempotent, prints the folder id
oo dav ensure-path "Banks/Caixa" --under 659 # start under an explicit folder id
oo dav upload 659 ./historico.xlsx # multipart upload, upsert by stem (legacy↔OOXML ext aware)
oo dav upload 659 ./a.xlsx ./b.pdf --replace=false # fail on name conflict instead of replacing
oo dav move 659 22881 22882 # DEST_FOLDER_ID FILE_ID…
oo dav move 659 22881 --folders 670 # move folders along with files
oo dav copy 659 22881
@@ -678,6 +754,11 @@ oo dav download 22881 --to ./copy.pdf # default path: ./<server title>
oo dav fileops # active move/copy operations (status polling)
```
`ensure-path` defaults to the concrete **My documents** section id (resolved
from `@root`, `rootFolderType=5`); `--under FOLDER_ID` overrides it. `upload`
reuses `UploadToFolderReplacing` (`--replace`, default) or `UploadToFolder`
after an `AssertNoFileConflict` check (`--no-replace`), so no new HTTP paths.
### Search and index (`oo search`, `oo index`)
Full-text search over the Documents index. The REST endpoint
@@ -729,28 +810,16 @@ with `c.SQLFileStore()`, or register a backend on the facade
fallback rules, env names and how to add a backend:
[`docs/unified-file-client.md`](docs/unified-file-client.md).
### Bulk tools (`cmd/`)
### Bulk tools
Small single-purpose binaries for bulk Documents work. All of them pace
requests and retry transient OnlyOffice answers (429/502/503/504) with a
deterministic linear backoff — no jitter, same waits on every run
(see `DoRetry` below). Build with `go build ./cmd/<tool>`.
```bash
ooscan 659 # recursive index → TSV: file_id, folder_id, path, title
ooscan 659 666 > oo-index.tsv # several roots into one index
pdfamount 671 # "Zu zahlender Betrag" per PDF → TSV: file_id, title, amount
kontoblatt 3906 ./kontoblatt.xlsx # summary (Gegenkonto/Monat) uploaded next to source
kontolink IN.xlsx oo-index.tsv OUT.xlsx [FILE_ID] [AMOUNTS_TSV]
# kontolink writes DocEditor links into the Link column: Beleg → supplier+month
# → amount+date (5th arg = pdfamount output); with FILE_ID it updates the
# source file in place, else uploads an "(links)" copy next to it.
```
Business / one-off bulk tools (`ooscan`, `pdfamount`, `kontoblatt`,
`kontolink`) live in the private `oo-workspace` repo, not in this public
library. They build on the public client and the same `DoRetry` pacing.
| Subject | Verbs |
|---|---|
| `calendar` | `list`, `events`, `add`, `delete` |
| `projects` | `list`, `get`, `milestones`, `milestone-create`, `create`, `update`, `delete`, `contacts` (`add`, `remove`), `link-authors`, `link-git`, **`files`** (`list`, `upload`, `download`, `rename`, `delete`, `dedupe`, `as-md`, `put-md`, `put-txt`, `put-xlsx`) |
| `projects` | `list`, `get`, `milestones`, `milestone-create`, `board-sync`, `create`, `update`, `delete`, `contacts` (`add`, `remove`), `link-authors`, `link-git`, **`files`** (`list`, `upload`, `download`, `rename`, `delete`, `dedupe`, `as-md`, `put-md`, `put-txt`, `put-xlsx`) |
| `tasks` | `list`, `get`, `create`, `update`, `delete`, `subtask add`, **`files`** (`list`, `upload`, `detach`) |
| `users` | `list`, `self` (alias: `oo whoami`) |
| `contacts` | `list`, `get`, `delete`, `info-add`, `merge`, `dedupe-info`, `tags`, `tag-add`, `tag-create`, `tag-remove` |
@@ -758,13 +827,13 @@ kontolink IN.xlsx oo-index.tsv OUT.xlsx [FILE_ID] [AMOUNTS_TSV]
| `companies` | `list`, `create`, `delete`, `dedupe`, `dedupe-persons` |
| `opportunities` | `list`, `get`, `create`, `update`, `delete`, `stages`, `member-add`, `dedupe`, `dedupe-members`, `fix-titles` |
| `invoices` | `list`, `get`, `create`, `update`, `pdf`, `pdf-cleanup`, `status`, `delete`, `items …` |
| `crm` | `cleanup` |
| `crm` | `audit`, `cleanup` |
| `mails` | `accounts`, `folders`, `list`, `get`, `download-attachment`, `draft`, `attach`, `draft-invoice`, `send`, `delete` |
| `cases` | `list`, `create`, `delete`, `member-add` |
| `crm-tasks` | `list`, `create`, `delete`, `categories`, `reassign-self` |
| `docs` | `tools`, `convert`, `optimize`, `ocr`, `hocr`, `as-md`, `put-md`, `put-txt`, `put-xlsx` |
| `docs` | `tools`, `convert`, `pdf`, `presigned`, `csv`, `json`, `optimize`, `ocr`, `hocr`, `as-md`, `put-md`, `put-txt`, `put-xlsx` |
| `catalog` | `match`, `merge`, `apply`, `scan-contacts`, `scan-projects`, `scan-thunderbird` |
| `dav` | `ls`, `move`, `copy`, `mkdir`, `rename-file`, `rename-folder`, `download`, `fileops` |
| `dav` | `ls`, `move`, `copy`, `mkdir`, `ensure-path`, `upload`, `rename-file`, `rename-folder`, `download`, `fileops` |
| `search` | `QUERY` (`--content`, `--folder ID`, `--limit N`, `--backend oo\|own`, `--json`) |
| `index` | `folder FOLDER_ID`, `files FILE_ID...` (`--recursive`, `--exts pdf`, `--limit N`, `--dry-run`) |
@@ -774,6 +843,12 @@ CLI-only concern — the library itself never loads dotfiles).
Canonical `ONLYOFFICE_*` variables win over aliases. Optional CLI-only aliases:
`OO_URL` / `OO_USER` / `OO_PASS` → `ONLYOFFICE_URL` / `ONLYOFFICE_USER` / `ONLYOFFICE_PASS`.
Catalog scanning (`oo catalog scan-projects` / `scan-thunderbird`) classifies
clients from rules in `$OO_CATALOG_CONFIG` (or `--config`); see
[`catalog/classify.example.yaml`](catalog/classify.example.yaml). Without rules
nothing is classified as work. The MinIO download fallback is off unless
`MINIO_ENDPOINT` + `MINIO_ACCESS_KEY` + `MINIO_SECRET_KEY` are set.
Run `oo --help` or `oo <subject> --help` for the full command reference.
> **0.5.0 migration note:** the command tree was flattened per-subject. Old
@@ -853,8 +928,8 @@ Merge two known company ids (keeps `INTO`):
oo contacts merge FROM_ID INTO_ID
```
Company ↔ person ↔ deal ↔ project ↔ invoice ↔ mail rules and OO quirks:
[docs/crm-associations.md](docs/crm-associations.md).
Company ↔ person ↔ deal ↔ project ↔ invoice ↔ mail rules and OO quirks live
with the private `oo-workspace` tooling.
### Invoices (`oo invoices`)
@@ -999,6 +1074,11 @@ oo projects files list 33
| `ONLYOFFICE_CALENDAR_ID` | Default calendar id used when omitted (default `1`) |
| `ONLYOFFICE_PROJECT_ID` | Default project id used when omitted (default `33`) |
| `OO_URL`, `OO_USER`, `OO_PASS` | Optional CLI-only aliases for `ONLYOFFICE_*` |
| `OO_RATE_LIMIT` | Process-wide request pacing, req/s (default `4`; `0` disables) |
| `OO_BURST` | Token-bucket burst (default `1`) |
| `OO_RETRY_ATTEMPTS` | Transient retries, total attempts (default `7`) |
| `OO_RETRY_BASE` | Exponential backoff base (default `2s`) |
| `OO_RETRY_MAX` | Backoff cap (default `2m`) |
| `ONLYOFFICE_ES_URL` | Elasticsearch base URL (`oo search`, own index); see [`docs/elasticsearch.md`](docs/elasticsearch.md) |
| `ONLYOFFICE_ES_INDEX` | OnlyOffice index (default `files_file`) |
| `ONLYOFFICE_ES_TEXT_INDEX` | Own PDF/scan index (default `oo_docs_text`) |
@@ -1039,24 +1119,20 @@ Live runs need credentials (`ONLYOFFICE_URL`, `ONLYOFFICE_USER`,
### rclone WebDAV mount
`deploy/docker-compose.rclone-webdav.yml` mounts the Documents tree as a
filesystem (compose, not systemd; container `rclone-webdav`) — read/write like
a normal FS over the `oo-webdav` sidecar. Setup, smoke log and limitations:
[`docs/rclone-webdav.md`](docs/rclone-webdav.md).
The rclone WebDAV mount (compose + `oo-webdav` sidecar) is deployment
tooling and lives in the private `oo-workspace` repo. Set
`ONLYOFFICE_WEBDAV_URL` to point the client at it.
### CI / releases
GitHub Actions (pattern from [`eSlider/go-config`](https://github.com/eSlider/go-config)):
| Workflow | Trigger | Purpose |
|---|---|---|
| `test.yml` | push / PR | `go vet`, unit tests, build `oo` + `office` |
| `release-please.yml` | push to `main` | semver PR from conventional commits |
| `release.yml` | tag `v*` | GoReleaser cross-platform `oo` + `office` binaries |
Repo setting required once: **Settings → Actions → General → Allow GitHub Actions to create and approve pull requests**.
Merge the release-please PR to tag a version; GoReleaser publishes assets to [GitHub Releases](https://github.com/eSlider/go-onlyoffice/releases).
Gitea is canonical; tags are created there per SemVer (`fix:` → patch,
`feat:` → minor, `!` → major). GoReleaser publishes assets to
[GitHub Releases](https://github.com/eSlider/go-onlyoffice/releases).
## Examples
+45 -1
View File
@@ -83,7 +83,7 @@ func (c *Client) authenticateOnce(ctx context.Context) error {
return err
}
if resp.StatusCode >= 400 {
return fmt.Errorf("auth: %d %s", resp.StatusCode, truncate(string(raw), 400))
return statusError(resp.StatusCode, retryAfterOf(resp), "auth: %d %s", resp.StatusCode, truncate(string(raw), 400))
}
var env struct {
Response *Token `json:"response"`
@@ -98,6 +98,50 @@ func (c *Client) authenticateOnce(ctx context.Context) error {
return nil
}
// AuthenticateAs verifies a login/password pair against the portal WITHOUT
// mutating the client's cached token. It returns nil when the portal issues a
// token, and the portal error otherwise.
//
// OnlyOffice accepts either the userName or the account email as the login. On
// some portals the account userName login returns HTTP 500 "User authentication
// failed" while the account email succeeds — confirmed for a freshly created
// guest user. Use this probe before sharing credentials (see `oo users check`),
// and prefer the email as the login.
func (c *Client) AuthenticateAs(ctx context.Context, login, password string) error {
body, err := json.Marshal(Credentials{User: login, Password: password})
if err != nil {
return fmt.Errorf("marshal credentials: %w", err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.baseURL()+"/api/2.0/authentication.json", bytes.NewReader(body))
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
resp, err := c.client.Do(req)
if err != nil {
return fmt.Errorf("auth request: %w", err)
}
defer resp.Body.Close()
raw, err := io.ReadAll(resp.Body)
if err != nil {
return err
}
if resp.StatusCode >= 400 {
return fmt.Errorf("auth: %d %s", resp.StatusCode, truncate(string(raw), 400))
}
var env struct {
Response *Token `json:"response"`
}
if err := json.Unmarshal(raw, &env); err != nil {
return fmt.Errorf("auth decode: %w", err)
}
if env.Response == nil || env.Response.Value == "" {
return fmt.Errorf("auth: empty token in response")
}
return nil
}
// InvalidateToken clears the cached authentication token. The next request
// (or call to Authenticate / AuthenticateContext) will re-authenticate.
//
+205
View File
@@ -0,0 +1,205 @@
package onlyoffice
// Project board (Gantt) upsert from a YAML board file.
//
// A board describes projects, their milestones and tasks by exact title. Sync
// creates only what is missing: existing milestones/tasks (matched by title)
// are left untouched, so the file can be the source of truth for a project
// plan and re-applied safely. Dry-run (apply=false) reports counts without
// writing.
//
// The file format is deliberately small and presentation-free:
//
// projects:
// - id: 42
// name: "Example"
// milestones:
// - title: "Kickoff"
// deadline: "2026-01-15"
// key: true
// tasks:
// - title: "Draft"
// start: "2026-01-02"
// deadline: "2026-01-10"
// description: "…"
import (
"context"
"fmt"
"os"
"strconv"
"time"
"gopkg.in/yaml.v3"
)
// Board is a YAML mapping from a project plan onto OnlyOffice milestones/tasks.
type Board struct {
Projects []BoardProject `yaml:"projects"`
}
// BoardProject is one project with its milestones.
type BoardProject struct {
ID int `yaml:"id"`
Name string `yaml:"name,omitempty"`
Milestones []BoardMilestone `yaml:"milestones"`
}
// BoardMilestone is a milestone ("key" marks it as a key milestone).
type BoardMilestone struct {
Title string `yaml:"title"`
Deadline string `yaml:"deadline"`
Key bool `yaml:"key,omitempty"`
Tasks []BoardTask `yaml:"tasks"`
}
// BoardTask is a task inside a milestone. Start falls back to Deadline.
type BoardTask struct {
Title string `yaml:"title"`
Start string `yaml:"start,omitempty"`
Deadline string `yaml:"deadline"`
Description string `yaml:"description,omitempty"`
}
// BoardSyncResult counts what SyncBoard created or skipped.
type BoardSyncResult struct {
CreatedMilestones int
SkippedMilestones int
CreatedTasks int
SkippedTasks int
DryRun bool
}
// LoadBoard reads a board YAML file.
func LoadBoard(path string) (*Board, error) {
b, err := os.ReadFile(path)
if err != nil {
return nil, err
}
return ParseBoard(b)
}
// ParseBoard decodes a board from YAML bytes.
func ParseBoard(data []byte) (*Board, error) {
var board Board
if err := yaml.Unmarshal(data, &board); err != nil {
return nil, fmt.Errorf("board: %w", err)
}
if len(board.Projects) == 0 {
return nil, fmt.Errorf("board: no projects")
}
return &board, nil
}
func boardDay(s string) (Time, error) {
t, err := time.Parse("2006-01-02", s)
if err != nil {
return Time{}, err
}
return Time(t), nil
}
func boardMilestoneIDs(ms []*Milestone) map[string]int64 {
out := map[string]int64{}
for _, m := range ms {
if m == nil || m.Title == nil || m.ID == nil {
continue
}
out[*m.Title] = *m.ID
}
return out
}
func boardTaskTitles(rows []map[string]any) map[string]struct{} {
out := map[string]struct{}{}
for _, r := range rows {
if t, _ := r["title"].(string); t != "" {
out[t] = struct{}{}
}
}
return out
}
// SyncBoard upserts milestones and tasks by exact title. With apply=false it
// only counts what would be created.
func (c *Client) SyncBoard(ctx context.Context, board *Board, apply bool) (*BoardSyncResult, error) {
if board == nil || len(board.Projects) == 0 {
return nil, fmt.Errorf("board: no projects")
}
res := &BoardSyncResult{DryRun: !apply}
for _, p := range board.Projects {
pid := p.ID
existing, err := c.GetProjectMilestones(&Project{ID: &pid})
if err != nil {
return res, fmt.Errorf("project %d milestones: %w", pid, err)
}
haveMS := boardMilestoneIDs(existing)
tasks, err := c.ListTasks(ctx, strconv.Itoa(pid), "")
if err != nil {
return res, fmt.Errorf("project %d tasks: %w", pid, err)
}
haveTask := boardTaskTitles(tasks)
for _, m := range p.Milestones {
msID, ok := haveMS[m.Title]
if !ok {
res.CreatedMilestones++
if apply {
dl, err := boardDay(m.Deadline)
if err != nil {
return res, fmt.Errorf("milestone %q deadline: %w", m.Title, err)
}
created, err := c.CreateMilestone(NewMilestoneRequest{
ProjectID: pid,
Title: m.Title,
Deadline: dl,
IsKey: m.Key,
})
if err != nil {
return res, fmt.Errorf("create milestone %q: %w", m.Title, err)
}
if created.ID != nil {
msID = *created.ID
}
haveMS[m.Title] = msID
}
} else {
res.SkippedMilestones++
}
for _, t := range m.Tasks {
if _, exists := haveTask[t.Title]; exists {
res.SkippedTasks++
continue
}
res.CreatedTasks++
if !apply {
continue
}
start := t.Start
if start == "" {
start = t.Deadline
}
st, err := boardDay(start)
if err != nil {
return res, fmt.Errorf("task %q start: %w", t.Title, err)
}
dl, err := boardDay(t.Deadline)
if err != nil {
return res, fmt.Errorf("task %q deadline: %w", t.Title, err)
}
if _, err := c.CreateProjectTask(NewProjectTaskRequest{
ProjectId: pid,
Title: t.Title,
Description: t.Description,
StartDate: st,
Deadline: dl,
MilestoneId: int(msID),
}); err != nil {
return res, fmt.Errorf("create task %q: %w", t.Title, err)
}
haveTask[t.Title] = struct{}{}
}
}
}
return res, nil
}
+70
View File
@@ -0,0 +1,70 @@
package onlyoffice
import "testing"
func TestParseBoard(t *testing.T) {
b, err := ParseBoard([]byte(`
projects:
- id: 13
name: Example
milestones:
- title: "[lq] Test"
deadline: "2026-01-15"
key: true
tasks:
- title: Draft
start: "2026-01-02"
deadline: "2026-01-10"
description: "…"
`))
if err != nil {
t.Fatal(err)
}
if len(b.Projects) != 1 || b.Projects[0].ID != 13 {
t.Fatalf("projects: %+v", b.Projects)
}
ms := b.Projects[0].Milestones[0]
if ms.Title != "[lq] Test" || !ms.Key || ms.Deadline != "2026-01-15" {
t.Fatalf("milestone: %+v", ms)
}
if len(ms.Tasks) != 1 || ms.Tasks[0].Title != "Draft" || ms.Tasks[0].Start != "2026-01-02" {
t.Fatalf("task: %+v", ms.Tasks)
}
}
func TestParseBoardEmpty(t *testing.T) {
if _, err := ParseBoard([]byte("projects: []")); err == nil {
t.Fatal("expected error for empty board")
}
}
func TestBoardMilestoneIDs(t *testing.T) {
title := "[lq] Test"
id := int64(9)
got := boardMilestoneIDs([]*Milestone{{Title: &title, ID: &id}, nil})
if got[title] != 9 {
t.Fatalf("%v", got)
}
}
func TestBoardTaskTitles(t *testing.T) {
got := boardTaskTitles([]map[string]any{{"title": "a"}, {"title": "b"}, {"nope": 1}})
if _, ok := got["a"]; !ok {
t.Fatal("missing a")
}
if _, ok := got["b"]; !ok {
t.Fatal("missing b")
}
if len(got) != 2 {
t.Fatalf("%v", got)
}
}
func TestBoardDay(t *testing.T) {
if _, err := boardDay("2026-01-15"); err != nil {
t.Fatal(err)
}
if _, err := boardDay("15.01.2026"); err == nil {
t.Fatal("expected error for non-ISO date")
}
}
+28
View File
@@ -0,0 +1,28 @@
package catalog
import "testing"
func TestMergeAddressesDedup(t *testing.T) {
dst := []Address{{Street: "Weg 1", City: "Stadt", Zip: "1"}}
got := mergeAddresses(dst, []Address{
{Street: "weg 1", City: "stadt", Zip: "1"}, // duplicate (case-insensitive)
{Street: "Weg 2", City: "Stadt", Zip: "2"}, // new
})
if len(got) != 2 {
t.Fatalf("got %d addresses, want 2: %+v", len(got), got)
}
if got[1].Street != "Weg 2" {
t.Errorf("second = %+v", got[1])
}
}
func TestEntryAddressesYAML(t *testing.T) {
doc := &Document{Entries: []Entry{{
ID: "person:max maier", Kind: "person", Name: "Max Maier",
Addresses: []Address{{Street: "Weg 1", City: "Stadt", Zip: "12345", Category: "Billing", Primary: true}},
}}}
merged := MergeDocs(doc)
if len(merged.Entries) != 1 || len(merged.Entries[0].Addresses) != 1 {
t.Fatalf("merged = %+v", merged.Entries)
}
}
+25 -15
View File
@@ -115,20 +115,13 @@ func applyCompany(ctx context.Context, client *onlyoffice.Client, e *Entry) (boo
}
func applyPerson(ctx context.Context, client *onlyoffice.Client, e *Entry) (bool, error) {
first := strings.TrimSpace(e.First)
last := strings.TrimSpace(e.Last)
if first == "" && last == "" {
first, last = SplitDisplayName(e.Name)
}
if first == "" {
first = strings.TrimSpace(e.Name)
}
org := strings.TrimSpace(e.Org)
first, last := CleanPersonNames(e.First, e.Last, e.Name, org, e.Emails)
e.First, e.Last = first, last
if first == "" {
return false, fmt.Errorf("person missing name")
}
if last == "" {
last = "-"
}
e.Name = strings.TrimSpace(first + " " + strings.Trim(last, "-"))
var p map[string]any
var err error
@@ -144,21 +137,27 @@ func applyPerson(ctx context.Context, client *onlyoffice.Client, e *Entry) (bool
}
created := false
companyID := 0
if e.Org != "" {
if co, ferr := client.FindCompany(ctx, e.Org); ferr == nil && co != nil {
if org != "" {
if co, ferr := client.FindCompany(ctx, org); ferr == nil && co != nil {
companyID, _ = strconv.Atoi(contactIDString(co))
}
}
if p == nil {
about := ""
if e.Org != "" {
about = "org: " + e.Org
if org != "" {
about = "org: " + org
}
p, err = client.CreatePerson(ctx, first, last, companyID, "", about)
if err != nil {
return false, err
}
created = true
} else {
// Repair names + ensure company link (never encode company in lastName).
id := contactIDString(p)
if _, err := client.UpdatePerson(ctx, id, first, last, companyID, "", ""); err != nil {
return false, fmt.Errorf("update person %s: %w", id, err)
}
}
id := contactIDString(p)
e.OOID = id
@@ -195,5 +194,16 @@ func ensureContactInfos(ctx context.Context, client *onlyoffice.Client, contactI
return fmt.Errorf("add phone %s: %w", ph, err)
}
}
for _, a := range e.Addresses {
if strings.TrimSpace(a.Street) == "" && strings.TrimSpace(a.City) == "" && strings.TrimSpace(a.Zip) == "" {
continue
}
if onlyoffice.HasContactAddress(existing, a.Street, a.City, a.Zip, a.Category) {
continue
}
if _, err := client.AddContactAddress(ctx, contactID, a.Street, a.City, a.State, a.Zip, a.Country, a.Category, a.Primary); err != nil {
return fmt.Errorf("add address %s: %w", strings.TrimSpace(a.Street), err)
}
}
return nil
}
+17 -5
View File
@@ -100,26 +100,38 @@ END:VCARD
func TestScanProjectsRoot(t *testing.T) {
root := t.TempDir()
repo := filepath.Join(root, "produktor-demo")
repo := filepath.Join(root, "acme-demo")
if err := os.MkdirAll(filepath.Join(repo, ".git"), 0o755); err != nil {
t.Fatal(err)
}
doc, err := ScanProjectsRoot(root, 3)
cl := &Classifier{WorkNames: []string{"acme"}}
doc, err := ScanProjectsRootOpts(root, 3, ScanOptions{Classifier: cl})
if err != nil {
t.Fatal(err)
}
found := false
for _, e := range doc.Entries {
if e.Kind == "company" && e.Name == "produktor-demo" {
if e.Kind == "company" && e.Name == "acme-demo" {
found = true
if e.Role != "work" {
t.Fatalf("role=%q", e.Role)
if e.Role != "work" || e.Zone != "warm" {
t.Fatalf("role=%q zone=%q", e.Role, e.Zone)
}
}
}
if !found {
t.Fatalf("missing company: %+v", doc.Entries)
}
// Neutral default leaves it unclassified.
doc, err = ScanProjectsRoot(root, 3)
if err != nil {
t.Fatal(err)
}
for _, e := range doc.Entries {
if e.Kind == "company" && e.Name == "acme-demo" && e.Role != "unknown" {
t.Fatalf("neutral role=%q", e.Role)
}
}
}
func TestEntryID(t *testing.T) {
+26
View File
@@ -0,0 +1,26 @@
# Deployment classification rules for `oo catalog scan-projects` /
# `oo catalog scan-thunderbird`. Point OO_CATALOG_CONFIG (or --config) at a copy
# of this file. Keep your real rules out of the repository — they name your
# clients and hosts. The library defaults to no rules (nothing is "work").
#
# work_remotes: a git remote containing any of these substrings → work/hot.
work_remotes:
- git.internal.example
- github.com/acme
#
# work_names: a project directory name containing any of these substrings → work/warm.
work_names:
- acme
#
# mail_orgs: ordered rules for Thunderbird/mbox identities; the first match wins.
# Match by exact `domain`, `suffix` (e.g. ".example.com") and/or display `name`.
# `zone` defaults to hot, `role` to work.
mail_orgs:
- domain: acme.example
org: Acme GmbH
zone: hot
role: work
- suffix: .gov.example
org: Public Sector
zone: warm
role: work
+127
View File
@@ -0,0 +1,127 @@
package catalog
import (
"fmt"
"os"
"strings"
"gopkg.in/yaml.v3"
)
// Classifier maps project trees and mail identities to catalog org/zone/role.
//
// The library ships with neutral defaults: nothing is classified as work unless
// the deployment supplies rules. Those rules are deployment-specific, so they
// live in a YAML config file (path from OO_CATALOG_CONFIG or the --config flag),
// not in the code. See catalog/classify.example.yaml.
type Classifier struct {
// WorkRemotes: a git remote containing any of these substrings → work/hot.
WorkRemotes []string `yaml:"work_remotes,omitempty"`
// WorkNames: a project name containing any of these substrings → work/warm.
WorkNames []string `yaml:"work_names,omitempty"`
// MailOrgs: ordered mail-identity rules; the first match wins.
MailOrgs []MailRule `yaml:"mail_orgs,omitempty"`
}
// MailRule maps an email domain and/or a display-name substring to an org with
// a zone/role. At least one of Domain, Suffix or Name must be set.
type MailRule struct {
Domain string `yaml:"domain,omitempty"` // exact domain, case-insensitive
Suffix string `yaml:"suffix,omitempty"` // domain suffix, e.g. ".example.com"
Name string `yaml:"name,omitempty"` // substring of the display name
Org string `yaml:"org"`
Zone string `yaml:"zone,omitempty"` // default "hot"
Role string `yaml:"role,omitempty"` // default "work"
}
// DefaultClassifier returns the neutral classifier (no deployment rules).
func DefaultClassifier() *Classifier { return &Classifier{} }
// LoadClassifier reads a classifier config from a YAML file.
func LoadClassifier(path string) (*Classifier, error) {
b, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var c Classifier
if err := yaml.Unmarshal(b, &c); err != nil {
return nil, fmt.Errorf("parse classifier config %s: %w", path, err)
}
return &c, nil
}
// LoadClassifierFromEnv loads the classifier named by OO_CATALOG_CONFIG. An
// empty variable yields the neutral classifier.
func LoadClassifierFromEnv() (*Classifier, error) {
path := strings.TrimSpace(os.Getenv("OO_CATALOG_CONFIG"))
if path == "" {
return DefaultClassifier(), nil
}
return LoadClassifier(path)
}
// ClassifyProject returns (role, zone) for a project name and git remote.
// Generic name heuristics come first; deployment rules supply the work cases.
func (c *Classifier) ClassifyProject(name, remote string) (role, zone string) {
lower := strings.ToLower(name)
remoteL := strings.ToLower(remote)
switch {
case strings.Contains(lower, "experiment") || strings.HasPrefix(lower, "test"):
return "experiment", "cold"
case lower == "mama" || lower == "personal" || strings.Contains(lower, "private"):
return "personal", "private"
}
if c != nil {
for _, r := range c.WorkRemotes {
if r != "" && strings.Contains(remoteL, strings.ToLower(r)) {
return "work", "hot"
}
}
for _, n := range c.WorkNames {
if n != "" && strings.Contains(lower, strings.ToLower(n)) {
return "work", "warm"
}
}
}
return "unknown", "warm"
}
// ClassifyMail returns (org, zone, role) for a mail identity. name is the
// display name (may be empty); email is the address.
func (c *Classifier) ClassifyMail(name, email string) (org, zone, role string) {
em := NormalizeEmail(email)
_, domain, _ := strings.Cut(em, "@")
nameL := strings.ToLower(strings.TrimSpace(name))
if c != nil {
for _, r := range c.MailOrgs {
if !mailRuleMatches(r, domain, nameL) {
continue
}
z, ro := r.Zone, r.Role
if z == "" {
z = "hot"
}
if ro == "" {
ro = "work"
}
return r.Org, z, ro
}
}
if strings.HasSuffix(domain, ".de") && looksPublicSector(domain) {
return domain, "warm", "work"
}
return "", "private", "unknown"
}
func mailRuleMatches(r MailRule, domain, nameL string) bool {
if r.Domain != "" && domain == strings.ToLower(strings.TrimSpace(r.Domain)) {
return true
}
if r.Suffix != "" && strings.HasSuffix(domain, strings.ToLower(strings.TrimSpace(r.Suffix))) {
return true
}
if r.Name != "" && nameL != "" && strings.Contains(nameL, strings.ToLower(strings.TrimSpace(r.Name))) {
return true
}
return false
}
+64
View File
@@ -0,0 +1,64 @@
package catalog
import (
"os"
"path/filepath"
"testing"
)
func TestClassifierRules(t *testing.T) {
cl := &Classifier{
WorkRemotes: []string{"git.internal.example"},
WorkNames: []string{"acme"},
MailOrgs: []MailRule{
{Domain: "acme.example", Org: "Acme", Zone: "hot", Role: "work"},
{Suffix: ".gov.example", Org: "Public", Zone: "warm", Role: "work"},
},
}
if role, zone := cl.ClassifyProject("acme-app", "git@git.internal.example:team/acme-app.git"); role != "work" || zone != "hot" {
t.Fatalf("remote: %s/%s", role, zone)
}
if role, zone := cl.ClassifyProject("acme-demo", ""); role != "work" || zone != "warm" {
t.Fatalf("name: %s/%s", role, zone)
}
if role, zone := cl.ClassifyProject("experiment-x", ""); role != "experiment" || zone != "cold" {
t.Fatalf("experiment: %s/%s", role, zone)
}
if org, zone, role := cl.ClassifyMail("", "bob@acme.example"); org != "Acme" || zone != "hot" || role != "work" {
t.Fatalf("mail: %s/%s/%s", org, zone, role)
}
if org, _, _ := cl.ClassifyMail("X", "x@team.gov.example"); org != "Public" {
t.Fatalf("suffix: %s", org)
}
if org, zone, role := cl.ClassifyMail("", "someone@unknown.example"); org != "" || zone != "private" || role != "unknown" {
t.Fatalf("neutral: %s/%s/%s", org, zone, role)
}
}
func TestLoadClassifier(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "classify.yaml")
if err := os.WriteFile(path, []byte("work_names:\n - acme\nmail_orgs:\n - domain: acme.example\n org: Acme\n"), 0o644); err != nil {
t.Fatal(err)
}
cl, err := LoadClassifier(path)
if err != nil {
t.Fatal(err)
}
if role, _ := cl.ClassifyProject("acme-app", ""); role != "work" {
t.Fatalf("role=%s", role)
}
if org, zone, _ := cl.ClassifyMail("", "a@acme.example"); org != "Acme" || zone != "hot" {
t.Fatalf("org=%s zone=%s", org, zone)
}
t.Setenv("OO_CATALOG_CONFIG", path)
if envCl, err := LoadClassifierFromEnv(); err != nil || envCl == nil || len(envCl.WorkNames) == 0 {
t.Fatalf("env: %+v %v", envCl, err)
}
t.Setenv("OO_CATALOG_CONFIG", "")
neutral, err := LoadClassifierFromEnv()
if err != nil || len(neutral.WorkNames) != 0 || len(neutral.MailOrgs) != 0 {
t.Fatalf("neutral: %+v %v", neutral, err)
}
}
+9 -4
View File
@@ -27,7 +27,7 @@ func MatchAgainstOO(ctx context.Context, client *onlyoffice.Client, doc *Documen
byEmail[NormalizeEmail(em)] = c
}
if isCo {
key := NormalizeName(fmt.Sprint(c["displayName"]))
key := onlyoffice.CompanyGroupingKey(fmt.Sprint(c["displayName"]))
if key != "" {
byCompanyName[key] = c
}
@@ -63,7 +63,7 @@ func MatchAgainstOO(ctx context.Context, client *onlyoffice.Client, doc *Documen
}
if !matched {
if e.Kind == "company" {
if c, ok := byCompanyName[NormalizeName(e.Name)]; ok {
if c, ok := byCompanyName[onlyoffice.CompanyGroupingKey(e.Name)]; ok {
oo = c
matched = true
}
@@ -92,6 +92,12 @@ func MatchAgainstOO(ctx context.Context, client *onlyoffice.Client, doc *Documen
e.OOID = contactIDString(oo)
continue
}
// Keep a previously applied oo_id (list payloads often omit emails, so
// email match can miss persons that already exist in CRM).
if strings.TrimSpace(e.OOID) != "" {
e.Status = "exists"
continue
}
e.Status = "new"
e.OOID = ""
}
@@ -120,8 +126,7 @@ func contactEmails(c map[string]any) []string {
out = append(out, em)
}
for _, row := range onlyoffice.ContactInfoRows(c) {
t := strings.ToLower(fmt.Sprint(row["infoType"]))
if t != "email" {
if onlyoffice.NormalizeContactInfoType(fmt.Sprint(row["infoType"])) != "email" {
continue
}
data := strings.TrimSpace(fmt.Sprint(row["data"]))
+11 -5
View File
@@ -16,6 +16,8 @@ type ScanOptions struct {
MboxHeaders bool
// MboxMaxBytes skips individual mbox files larger than this (0 = 256 MiB default).
MboxMaxBytes int64
// Classifier supplies deployment classification rules; nil → neutral default.
Classifier *Classifier
}
// ScanThunderbirdRoot finds Thunderbird profiles under root and emits person rows
@@ -37,6 +39,10 @@ func ScanThunderbirdRootOpts(root string, opts ScanOptions) (*Document, error) {
if opts.MboxMaxBytes <= 0 {
opts.MboxMaxBytes = 256 << 20
}
cl := opts.Classifier
if cl == nil {
cl = DefaultClassifier()
}
var entries []Entry
seenDB := map[string]struct{}{}
@@ -64,7 +70,7 @@ func ScanThunderbirdRootOpts(root string, opts ScanOptions) (*Document, error) {
return nil
}
seenDB[path] = struct{}{}
parsed, perr := parseGlodaContacts(path)
parsed, perr := parseGlodaContacts(path, cl)
if perr != nil {
entries = append(entries, Entry{
ID: EntryID("person", "", filepath.Base(path)),
@@ -84,7 +90,7 @@ func ScanThunderbirdRootOpts(root string, opts ScanOptions) (*Document, error) {
return nil
}
seenMAB[path] = struct{}{}
parsed, perr := parseMABEmails(path)
parsed, perr := parseMABEmails(path, cl)
if perr != nil {
return nil
}
@@ -101,7 +107,7 @@ func ScanThunderbirdRootOpts(root string, opts ScanOptions) (*Document, error) {
if info.Size() > opts.MboxMaxBytes {
return nil
}
parsed, perr := parseMboxHeaderEmails(path)
parsed, perr := parseMboxHeaderEmails(path, cl)
if perr != nil {
return nil
}
@@ -150,7 +156,7 @@ func isLikelyMboxFile(name, path string) bool {
}
// parseMboxHeaderEmails extracts addresses from From/To/Cc/Reply-To headers only.
func parseMboxHeaderEmails(path string) ([]Entry, error) {
func parseMboxHeaderEmails(path string, cl *Classifier) ([]Entry, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
@@ -220,7 +226,7 @@ func parseMboxHeaderEmails(path string) ([]Entry, error) {
var out []Entry
for em := range emails {
org, zone, role := classifyMailIdentity("", em)
org, zone, role := cl.ClassifyMail("", em)
out = append(out, Entry{
ID: EntryID("person", em, ""),
Kind: "person",
+7 -7
View File
@@ -10,22 +10,22 @@ func TestParseMboxHeaderEmails(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "INBOX")
body := `From - Mon Jul 1 00:00:00 2016
From: Axel Schaefer <axel.schaefer@wheregroup.com>
To: Andriy Oblivantsev <andriy.oblivantsev@wheregroup.com>
From: Alice Smith <alice.smith@acme.example>
To: Bob Jones <bob.jones@acme.example>
Cc: noreply@example.com, client@stadt-example.de
Subject: test
Body line ignored
From - Mon Jul 2 00:00:00 2016
From: Someone <paul.schmidt@wheregroup.com>
To: list@wheregroup.com
From: Someone <paul.schmidt@acme.example>
To: list@acme.example
more body
`
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
ents, err := parseMboxHeaderEmails(path)
ents, err := parseMboxHeaderEmails(path, DefaultClassifier())
if err != nil {
t.Fatal(err)
}
@@ -35,7 +35,7 @@ more body
got[e.Emails[0]] = true
}
}
if !got["axel.schaefer@wheregroup.com"] || !got["andriy.oblivantsev@wheregroup.com"] {
if !got["alice.smith@acme.example"] || !got["bob.jones@acme.example"] {
t.Fatalf("%v", got)
}
if got["noreply@example.com"] {
@@ -53,7 +53,7 @@ func TestScanThunderbirdRootOptsMbox(t *testing.T) {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(imap, "INBOX"), []byte(
"From - x\nFrom: a@wheregroup.com\nTo: b@wheregroup.com\n\nbody\n",
"From - x\nFrom: a@acme.example\nTo: b@acme.example\n\nbody\n",
), 0o644); err != nil {
t.Fatal(err)
}
+152
View File
@@ -0,0 +1,152 @@
package catalog
import (
"regexp"
"strings"
"unicode"
)
var (
parenSuffixRE = regexp.MustCompile(`(?i)\s*[\(\[\{][^)\]\}]*[\)\]\}]\s*$`)
dashCompanyRE = regexp.MustCompile(`(?i)\s+[-–—]\s+[A-Za-z0-9].*$`)
emailLocalRE = regexp.MustCompile(`(?i)^[a-z0-9._%+\-]+@[a-z0-9.\-]+\.[a-z]{2,}$`)
nonNameTokenRE = regexp.MustCompile(`[^a-zA-ZÀ-öø-ÿĀ-ž0-9'’.\-]+`)
)
// CleanPersonNames strips company annotations from display names and fills
// first/last from the email local-part when the source used an address as the
// name. Company affiliation belongs on Org / the CRM companyId — never in LastName.
func CleanPersonNames(first, last, display, org string, emails []string) (cleanFirst, cleanLast string) {
first = strings.TrimSpace(first)
last = strings.TrimSpace(last)
display = strings.TrimSpace(display)
org = strings.TrimSpace(org)
if looksLikeEmail(first) {
ef, el := GuessNameFromEmail(first)
first, last = ef, el
}
if looksLikeEmail(display) && first == "" && last == "" {
display = ""
}
if first == "" && last == "" && display != "" {
first, last = SplitDisplayName(display)
}
first = stripCompanyAnnotation(first, org)
last = stripCompanyAnnotation(last, org)
// "Smith - Acme" / "Jones (Acme)" landed in last.
last = stripCompanyAnnotation(last, org)
if i := strings.IndexAny(first, "(["); i > 0 {
first = strings.TrimSpace(first[:i])
}
// Entire last name is just the company (e.g. last="Acme").
if org != "" && personLastIsOrg(last, org) {
last = ""
}
if (first == "" || looksLikeEmail(first)) && len(emails) > 0 {
ef, el := GuessNameFromEmail(emails[0])
if first == "" || looksLikeEmail(first) {
first = ef
}
if last == "" || last == "-" {
last = el
}
}
first = strings.TrimSpace(first)
last = strings.TrimSpace(last)
if last == "" {
last = "-"
}
return first, last
}
func stripCompanyAnnotation(s, org string) string {
s = strings.TrimSpace(s)
if s == "" {
return ""
}
s = parenSuffixRE.ReplaceAllString(s, "")
s = strings.TrimSpace(s)
s = dashCompanyRE.ReplaceAllString(s, "")
s = strings.TrimSpace(s)
if org != "" {
for _, sep := range []string{" - ", " – ", " — ", " / "} {
if i := strings.LastIndex(strings.ToLower(s), strings.ToLower(sep+org)); i >= 0 {
s = strings.TrimSpace(s[:i])
}
}
suf := " (" + org + ")"
if strings.HasSuffix(strings.ToLower(s), strings.ToLower(suf)) {
s = strings.TrimSpace(s[:len(s)-len(suf)])
}
}
return strings.TrimSpace(s)
}
func personLastIsOrg(last, org string) bool {
last = NormalizeName(last)
org = NormalizeName(org)
if last == "" || org == "" {
return false
}
if last == org {
return true
}
// "Acme" vs "Acme GmbH & Co. KG"
return strings.HasPrefix(org, last+" ") || strings.HasPrefix(org, last+",")
}
func looksLikeEmail(s string) bool {
return emailLocalRE.MatchString(strings.TrimSpace(s))
}
// GuessNameFromEmail turns local@domain into Title-Case first/last when the
// local part looks like first.last / first_last / first-last.
func GuessNameFromEmail(email string) (first, last string) {
email = NormalizeEmail(email)
local, _, ok := strings.Cut(email, "@")
if !ok || local == "" {
return "", ""
}
local = strings.Split(local, "+")[0]
parts := strings.FieldsFunc(local, func(r rune) bool {
return r == '.' || r == '_' || r == '-'
})
if len(parts) == 0 {
return titleToken(local), ""
}
if len(parts) == 1 {
return titleToken(parts[0]), ""
}
return titleToken(parts[0]), titleToken(strings.Join(parts[1:], " "))
}
func titleToken(s string) string {
s = nonNameTokenRE.ReplaceAllString(s, " ")
s = strings.TrimSpace(s)
if s == "" {
return ""
}
runes := []rune(strings.ToLower(s))
runes[0] = unicode.ToTitle(runes[0])
return string(runes)
}
// FormatProjectTitle builds "CC | Company | Title" (spaces around |).
// Country should be a short code (DE, TF, UA, …). Empty segments are dropped.
func FormatProjectTitle(country, company, title string) string {
parts := make([]string, 0, 3)
for _, p := range []string{country, company, title} {
p = strings.TrimSpace(p)
p = strings.ReplaceAll(p, "|", "/")
if p != "" {
parts = append(parts, p)
}
}
return strings.Join(parts, " | ")
}
+41
View File
@@ -0,0 +1,41 @@
package catalog
import "testing"
func TestCleanPersonNamesStripsCompanyParen(t *testing.T) {
f, l := CleanPersonNames("John", "Smith (Acme)", "John Smith (Acme)", "Acme", nil)
if f != "John" || l != "Smith" {
t.Fatalf("got %q %q", f, l)
}
}
func TestCleanPersonNamesStripsDashCompany(t *testing.T) {
f, l := CleanPersonNames("Jens", "Meyer - Acme", "", "Acme", nil)
if f != "Jens" || l != "Meyer" {
t.Fatalf("got %q %q", f, l)
}
}
func TestCleanPersonNamesFromEmail(t *testing.T) {
f, l := CleanPersonNames("david.patzke@acme.example", "-", "", "Acme",
[]string{"david.patzke@acme.example"})
if f != "David" || l != "Patzke" {
t.Fatalf("got %q %q", f, l)
}
}
func TestCleanPersonNamesLastIsCompany(t *testing.T) {
f, l := CleanPersonNames("Thorsten", "Acme", "", "Acme GmbH & Co. KG", nil)
if f != "Thorsten" || l != "-" {
t.Fatalf("got %q %q", f, l)
}
}
func TestFormatProjectTitle(t *testing.T) {
if got := FormatProjectTitle("DE", "Acme", "Golang"); got != "DE | Acme | Golang" {
t.Fatalf("got %q", got)
}
if got := FormatProjectTitle("", "Acme", ""); got != "Acme" {
t.Fatalf("got %q", got)
}
}
+14 -24
View File
@@ -10,10 +10,19 @@ import (
// ScanProjectsRoot finds git roots under root (max depth) and emits company rows.
func ScanProjectsRoot(root string, maxDepth int) (*Document, error) {
return ScanProjectsRootOpts(root, maxDepth, ScanOptions{})
}
// ScanProjectsRootOpts is ScanProjectsRoot with deployment classification rules.
func ScanProjectsRootOpts(root string, maxDepth int, opts ScanOptions) (*Document, error) {
root = filepath.Clean(root)
if maxDepth <= 0 {
maxDepth = 4
}
cl := opts.Classifier
if cl == nil {
cl = DefaultClassifier()
}
st, err := os.Stat(root)
if err != nil {
return nil, err
@@ -23,7 +32,7 @@ func ScanProjectsRoot(root string, maxDepth int) (*Document, error) {
}
var entries []Entry
err = walkGitRoots(root, root, 0, maxDepth, &entries)
err = walkGitRoots(root, root, 0, maxDepth, cl, &entries)
if err != nil {
return nil, err
}
@@ -43,7 +52,7 @@ func ScanProjectsRoot(root string, maxDepth int) (*Document, error) {
}
path := filepath.Join(root, name)
id := EntryID("company", "", name)
role, zone := classifyProjectName(name, "")
role, zone := cl.ClassifyProject(name, "")
entries = append(entries, Entry{
ID: id,
Kind: "company",
@@ -59,7 +68,7 @@ func ScanProjectsRoot(root string, maxDepth int) (*Document, error) {
return MergeDocs(&Document{Entries: entries}), nil
}
func walkGitRoots(root, dir string, depth, maxDepth int, out *[]Entry) error {
func walkGitRoots(root, dir string, depth, maxDepth int, cl *Classifier, out *[]Entry) error {
if depth > maxDepth {
return nil
}
@@ -67,7 +76,7 @@ func walkGitRoots(root, dir string, depth, maxDepth int, out *[]Entry) error {
if st, err := os.Stat(filepath.Join(dir, ".git")); err == nil && (st.IsDir() || st.Mode().IsRegular()) {
name := filepath.Base(dir)
remote := gitRemoteOrigin(dir)
role, zone := classifyProjectName(name, remote)
role, zone := cl.ClassifyProject(name, remote)
*out = append(*out, Entry{
ID: EntryID("company", "", name),
Kind: "company",
@@ -93,7 +102,7 @@ func walkGitRoots(root, dir string, depth, maxDepth int, out *[]Entry) error {
if name == ".git" || name == "node_modules" || name == "vendor" || name == ".venv" || name == "dist" {
continue
}
_ = walkGitRoots(root, filepath.Join(dir, name), depth+1, maxDepth, out)
_ = walkGitRoots(root, filepath.Join(dir, name), depth+1, maxDepth, cl, out)
}
return nil
}
@@ -106,22 +115,3 @@ func gitRemoteOrigin(dir string) string {
}
return strings.TrimSpace(string(b))
}
func classifyProjectName(name, remote string) (role, zone string) {
lower := strings.ToLower(name)
remoteL := strings.ToLower(remote)
switch {
case strings.Contains(lower, "experiment") || strings.HasPrefix(lower, "test"):
return "experiment", "cold"
case lower == "mama" || lower == "personal" || strings.Contains(lower, "private"):
return "personal", "private"
case strings.Contains(remoteL, "git.produktor.io") || strings.Contains(remoteL, "github.com/eslider"):
return "work", "hot"
case strings.Contains(lower, "produktor") || strings.Contains(lower, "eslider") ||
strings.Contains(lower, "asesoria") || strings.Contains(lower, "dyvenia") ||
strings.Contains(lower, "onlyoffice"):
return "work", "warm"
default:
return "unknown", "warm"
}
}
+4 -23
View File
@@ -64,7 +64,7 @@ func noisyEmail(email string) bool {
return false
}
func parseMABEmails(path string) ([]Entry, error) {
func parseMABEmails(path string, cl *Classifier) ([]Entry, error) {
b, err := os.ReadFile(path)
if err != nil {
return nil, err
@@ -76,7 +76,7 @@ func parseMABEmails(path string) ([]Entry, error) {
if noisyEmail(em) {
continue
}
org, zone, role := classifyMailIdentity("", em)
org, zone, role := cl.ClassifyMail("", em)
out = append(out, Entry{
ID: EntryID("person", em, ""),
Kind: "person",
@@ -93,7 +93,7 @@ func parseMABEmails(path string) ([]Entry, error) {
return out, nil
}
func parseGlodaContacts(dbPath string) ([]Entry, error) {
func parseGlodaContacts(dbPath string, cl *Classifier) ([]Entry, error) {
// read-only URI; immutable=1 helps when WAL/shm are missing
dsn := "file:" + dbPath + "?mode=ro&_pragma=query_only(1)"
db, err := sql.Open("sqlite", dsn)
@@ -137,7 +137,7 @@ func parseGlodaContacts(dbPath string) ([]Entry, error) {
if display != "" {
first, last = SplitDisplayName(display)
}
org, zone, role := classifyMailIdentity(display, em)
org, zone, role := cl.ClassifyMail(display, em)
out = append(out, Entry{
ID: EntryID("person", em, display),
Kind: "person",
@@ -157,25 +157,6 @@ func parseGlodaContacts(dbPath string) ([]Entry, error) {
return out, rows.Err()
}
func classifyMailIdentity(name, email string) (org, zone, role string) {
em := NormalizeEmail(email)
_, domain, _ := strings.Cut(em, "@")
switch {
case domain == "wheregroup.com" || strings.Contains(strings.ToLower(name), "wheregroup"):
return "WhereGroup", "warm", "work"
case domain == "produktor.io" || domain == "eslider.de" || strings.HasSuffix(domain, ".produktor.io"):
return "produktor.io", "hot", "work"
case domain == "dyvenia.com":
return "Dyvenia", "warm", "work"
case domain == "immowelt.de" || domain == "immowelt.com":
return "Immowelt", "warm", "work"
case strings.HasSuffix(domain, ".de") && looksPublicSector(domain):
return domain, "warm", "work"
default:
return "", "private", "unknown"
}
}
func looksPublicSector(domain string) bool {
d := strings.ToLower(domain)
hints := []string{
+23 -11
View File
@@ -16,8 +16,8 @@ func TestNoisyEmail(t *testing.T) {
if !noisyEmail("x@marketplace.amazon.de") {
t.Fatal("amazon marketplace")
}
if noisyEmail("andriy.oblivantsev@wheregroup.com") {
t.Fatal("should keep wheregroup")
if noisyEmail("alice.smith@acme.example") {
t.Fatal("should keep a human work address")
}
}
@@ -25,14 +25,15 @@ func TestParseMABEmails(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "abook.mab")
body := `// mork junk
PrimaryEmail=andriy.oblivantsev@wheregroup.com
PrimaryEmail=alice.smith@acme.example
noreply@github.com
axel.schaefer@wheregroup.com
bob.jones@acme.example
`
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
ents, err := parseMABEmails(path)
// Neutral default: no deployment rules → unclassified.
ents, err := parseMABEmails(path, DefaultClassifier())
if err != nil {
t.Fatal(err)
}
@@ -40,7 +41,18 @@ func TestParseMABEmails(t *testing.T) {
t.Fatalf("got %d: %+v", len(ents), ents)
}
for _, e := range ents {
if e.Org != "WhereGroup" || e.Role != "work" {
if e.Org != "" || e.Role != "unknown" {
t.Fatalf("%+v", e)
}
}
// A deployment rule classifies the domain as work.
cl := &Classifier{MailOrgs: []MailRule{{Domain: "acme.example", Org: "Acme", Zone: "warm", Role: "work"}}}
ents, err = parseMABEmails(path, cl)
if err != nil {
t.Fatal(err)
}
for _, e := range ents {
if e.Org != "Acme" || e.Role != "work" || e.Zone != "warm" {
t.Fatalf("%+v", e)
}
}
@@ -56,8 +68,8 @@ func TestParseGlodaContacts(t *testing.T) {
_, err = db.Exec(`
CREATE TABLE contacts (id INTEGER PRIMARY KEY, name TEXT);
CREATE TABLE identities (id INTEGER PRIMARY KEY, contactID INTEGER, kind TEXT, value TEXT);
INSERT INTO contacts VALUES (1, 'Axel Schaefer');
INSERT INTO identities VALUES (1, 1, 'email', 'axel.schaefer@wheregroup.com');
INSERT INTO contacts VALUES (1, 'Alice Smith');
INSERT INTO identities VALUES (1, 1, 'email', 'alice.smith@acme.example');
INSERT INTO contacts VALUES (2, 'Noise Bot');
INSERT INTO identities VALUES (2, 2, 'email', 'noreply@example.com');
`)
@@ -66,14 +78,14 @@ func TestParseGlodaContacts(t *testing.T) {
}
_ = db.Close()
ents, err := parseGlodaContacts(dbPath)
ents, err := parseGlodaContacts(dbPath, DefaultClassifier())
if err != nil {
t.Fatal(err)
}
if len(ents) != 1 {
t.Fatalf("got %d %+v", len(ents), ents)
}
if ents[0].First != "Axel" || ents[0].Emails[0] != "axel.schaefer@wheregroup.com" {
if ents[0].First != "Alice" || ents[0].Emails[0] != "alice.smith@acme.example" {
t.Fatalf("%+v", ents[0])
}
}
@@ -84,7 +96,7 @@ func TestScanThunderbirdRoot(t *testing.T) {
if err := os.MkdirAll(prof, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(prof, "abook.mab"), []byte("mail=paul.schmidt@wheregroup.com\n"), 0o644); err != nil {
if err := os.WriteFile(filepath.Join(prof, "abook.mab"), []byte("mail=paul.schmidt@acme.example\n"), 0o644); err != nil {
t.Fatal(err)
}
doc, err := ScanThunderbirdRoot(root)
+33
View File
@@ -12,6 +12,18 @@ import (
"gopkg.in/yaml.v3"
)
// Address is one postal address of a catalog row. Category is the OO
// AddressCategory label (Home|Postal|Office|Billing|Other|Work).
type Address struct {
Street string `yaml:"street,omitempty"`
City string `yaml:"city,omitempty"`
State string `yaml:"state,omitempty"`
Zip string `yaml:"zip,omitempty"`
Country string `yaml:"country,omitempty"`
Category string `yaml:"category,omitempty"`
Primary bool `yaml:"primary,omitempty"`
}
// Entry is one catalog row (person or company).
type Entry struct {
ID string `yaml:"id"`
@@ -21,6 +33,7 @@ type Entry struct {
Last string `yaml:"last,omitempty"`
Emails []string `yaml:"emails,omitempty"`
Phones []string `yaml:"phones,omitempty"`
Addresses []Address `yaml:"addresses,omitempty"`
Org string `yaml:"org,omitempty"`
Sources []string `yaml:"sources,omitempty"`
Zone string `yaml:"zone"`
@@ -136,6 +149,7 @@ func mergeEntry(dst, src *Entry) {
dst.Sources = uniqueStrings(append(dst.Sources, src.Sources...))
dst.Emails = uniqueEmails(append(dst.Emails, src.Emails...))
dst.Phones = uniqueStrings(append(dst.Phones, src.Phones...))
dst.Addresses = mergeAddresses(dst.Addresses, src.Addresses)
if dst.First == "" {
dst.First = src.First
}
@@ -176,6 +190,25 @@ func mergeEntry(dst, src *Entry) {
}
}
// mergeAddresses appends src addresses not already present (by street/city/zip).
func mergeAddresses(dst, src []Address) []Address {
for _, a := range src {
exists := false
for _, b := range dst {
if strings.EqualFold(strings.TrimSpace(a.Street), strings.TrimSpace(b.Street)) &&
strings.EqualFold(strings.TrimSpace(a.City), strings.TrimSpace(b.City)) &&
strings.EqualFold(strings.TrimSpace(a.Zip), strings.TrimSpace(b.Zip)) {
exists = true
break
}
}
if !exists {
dst = append(dst, a)
}
}
return dst
}
func uniqueEmails(in []string) []string {
seen := map[string]struct{}{}
var out []string
+6 -2
View File
@@ -38,11 +38,15 @@ type Client struct {
folderTitlesMu sync.Mutex
}
// NewClient returns a new Client backed by http.DefaultClient.
// NewClient returns a new Client whose transport is paced by the process-wide
// rate limiter and 429 cooldown gate (OO_RATE_LIMIT/OO_BURST; see ratelimit.go).
func NewClient(c Credentials) *Client {
jar, _ := cookiejar.New(nil)
return &Client{
client: &http.Client{Jar: jar},
client: &http.Client{
Jar: jar,
Transport: &pacedTransport{base: http.DefaultTransport},
},
credentials: &c,
}
}
-220
View File
@@ -1,220 +0,0 @@
// Command kontoblatt builds a summary ("сводная таблица") of a Kontoblatt XLSX
// (Datum, Gegenkonto, Buchungstext, Beleg, Soll, Haben, Bemerkung) and uploads
// it back to the same OnlyOffice folder as the source file.
//
// Usage: kontoblatt <FILE_ID> <LOCAL_XLSX>
package main
import (
"context"
"fmt"
"os"
"regexp"
"sort"
"strconv"
"strings"
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/xuri/excelize/v2"
)
type agg struct {
count int
soll float64
haben float64
reFehlt int
}
type rec struct {
date, month, konto, text string
soll, haben float64
reFehlt bool
}
var dateRe = regexp.MustCompile(`^\d{2}\.\d{2}\.\d{4}$`)
func parseAmount(s string) float64 {
s = strings.TrimSpace(s)
s = strings.ReplaceAll(s, "€", "")
s = strings.ReplaceAll(s, " ", "")
s = strings.ReplaceAll(s, ",", "") // German thousands separator
s = strings.TrimSpace(s)
if s == "" {
return 0
}
v, err := strconv.ParseFloat(s, 64)
if err != nil {
return 0
}
return v
}
func cell(row []string, i int) string {
if i < len(row) {
return strings.TrimSpace(row[i])
}
return ""
}
func main() {
if len(os.Args) < 3 {
fmt.Fprintln(os.Stderr, "usage: kontoblatt <FILE_ID> <LOCAL_XLSX>")
os.Exit(2)
}
fileID, path := os.Args[1], os.Args[2]
ctx := context.Background()
f, err := excelize.OpenFile(path)
if err != nil {
panic(err)
}
defer f.Close()
var recs []rec
for _, sh := range f.GetSheetList() {
rows, err := f.GetRows(sh)
if err != nil {
continue
}
for _, r := range rows {
d := cell(r, 0)
if !dateRe.MatchString(d) {
continue
}
text := cell(r, 2)
recs = append(recs, rec{
date: d,
month: d[3:10],
konto: cell(r, 1),
text: text,
soll: parseAmount(cell(r, 4)),
haben: parseAmount(cell(r, 5)),
reFehlt: strings.Contains(strings.ToUpper(text), "FEHLT"),
})
}
}
byKonto := map[string]*agg{}
byMonth := map[string]*agg{}
getK := func(k string) *agg {
if byKonto[k] == nil {
byKonto[k] = &agg{}
}
return byKonto[k]
}
getM := func(k string) *agg {
if byMonth[k] == nil {
byMonth[k] = &agg{}
}
return byMonth[k]
}
var tot agg
for _, r := range recs {
k := getK(r.konto)
k.count++
k.soll += r.soll
k.haben += r.haben
if r.reFehlt {
k.reFehlt++
}
m := getM(r.month)
m.count++
m.soll += r.soll
m.haben += r.haben
if r.reFehlt {
m.reFehlt++
}
tot.count++
tot.soll += r.soll
tot.haben += r.haben
if r.reFehlt {
tot.reFehlt++
}
}
out := excelize.NewFile()
defer out.Close()
writeSheet(out, "Nach Gegenkonto", "Gegenkonto", byKonto, tot)
writeSheet(out, "Nach Monat", "Monat", byMonth, tot)
outPath := "/tmp/opencode/kontoblatt-zusammenfassung.xlsx"
if err := out.SaveAs(outPath); err != nil {
panic(err)
}
// upload next to the source file
creds := onlyoffice.GetEnvironmentCredentials()
c := onlyoffice.NewClient(creds)
var src *onlyoffice.FileEntry
if derr := onlyoffice.DoRetry(ctx, onlyoffice.DefaultRetryPolicy(), func() error {
var err error
src, err = c.GetFile(ctx, fileID)
return err
}); derr != nil {
panic(derr)
}
folder := ""
if src.FolderID != nil {
folder = src.FolderID.String()
}
title := ""
if src.Title != nil {
title = *src.Title
}
fmt.Printf("source: id=%s title=%q folder=%s\n", fileID, title, folder)
name := "Kontoblatt-1591-2025-Zusammenfassung.xlsx"
tmp := "/tmp/opencode/" + name
data, _ := os.ReadFile(outPath)
if err := os.WriteFile(tmp, data, 0o600); err != nil {
panic(err)
}
var entry *onlyoffice.FileEntry
if derr := onlyoffice.DoRetry(ctx, onlyoffice.DefaultRetryPolicy(), func() error {
var err error
entry, _, err = c.UploadToFolderReplacing(ctx, folder, tmp)
return err
}); derr != nil {
panic(derr)
}
fmt.Printf("uploaded: %s -> folder %s (id %v)\n", name, folder, entry.ID)
// print the summary
printAgg("Nach Gegenkonto", byKonto, tot)
printAgg("Nach Monat", byMonth, tot)
}
func writeSheet(f *excelize.File, sheet, key string, m map[string]*agg, tot agg) {
f.NewSheet(sheet)
rows := [][]any{{key, "Anzahl", "Soll", "Haben", "Saldo", `davon "fehlt"`}}
keys := make([]string, 0, len(m))
for k := range m {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
a := m[k]
rows = append(rows, []any{k, a.count, a.soll, a.haben, a.soll - a.haben, a.reFehlt})
}
rows = append(rows, []any{"GESAMT", tot.count, tot.soll, tot.haben, tot.soll - tot.haben, tot.reFehlt})
for i, row := range rows {
for j, v := range row {
cellRef, _ := excelize.CoordinatesToCellName(j+1, i+1)
_ = f.SetCellValue(sheet, cellRef, v)
}
}
}
func printAgg(title string, m map[string]*agg, tot agg) {
fmt.Printf("\n== %s ==\n", title)
keys := make([]string, 0, len(m))
for k := range m {
keys = append(keys, k)
}
sort.Strings(keys)
fmt.Printf("%-12s %6s %12s %12s %12s %7s\n", "key", "count", "soll", "haben", "saldo", "fehlt")
for _, k := range keys {
a := m[k]
fmt.Printf("%-12s %6d %12.2f %12.2f %12.2f %7d\n", k, a.count, a.soll, a.haben, a.soll-a.haben, a.reFehlt)
}
fmt.Printf("%-12s %6d %12.2f %12.2f %12.2f %7d\n", "GESAMT", tot.count, tot.soll, tot.haben, tot.soll-tot.haben, tot.reFehlt)
}
-478
View File
@@ -1,478 +0,0 @@
// Command kontolink fills the "Link" column of a Kontoblatt ("ungeklärte
// Posten") XLSX by matching each row to an OnlyOffice document.
//
// Strategy (deterministic, conservative — no LLM):
// 1. Beleg token (letters/digits from the "Beleg" column) appears in the file
// title; among candidates prefer (a) the row's month, (b) real invoices over
// copies/dupes, and require the result to be unique;
// 2. else supplier + row month + "rechnung", again unique.
//
// A file is linked at most once (rows already carrying a link are kept and their
// file counts as used). Ambiguous rows are left UNLINKED for manual review.
//
// Usage: kontolink <IN_XLSX> <INDEX_TSV> <OUT_XLSX>
package main
import (
"context"
"fmt"
"os"
"path/filepath"
"regexp"
"strconv"
"strings"
"time"
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/xuri/excelize/v2"
)
var (
dateRe = regexp.MustCompile(`^\d{2}\.\d{2}\.\d{4}$`)
nonAln = regexp.MustCompile(`[^0-9a-z]+`)
fileID = regexp.MustCompile(`fileid=(\d+)`)
)
func parseDay(s string) (time.Time, bool) {
t, err := time.Parse("02.01.2006", strings.TrimSpace(s))
return t, err == nil
}
func titleDay(title string) (time.Time, bool) {
if len(title) >= 10 {
if t, err := time.Parse("2006-01-02", title[:10]); err == nil {
return t, true
}
}
return time.Time{}, false
}
// nearest picks the candidate whose title date is closest to rd. Ties and
// undated candidates (when >1) are rejected.
func nearest(cands []entry, rd time.Time) (entry, bool) {
if len(cands) == 1 {
return cands[0], true
}
best, bestD, tie := -1, 0.0, false
for i, e := range cands {
td, ok := titleDay(e.title)
if !ok {
continue
}
d := td.Sub(rd).Hours() / 24
if d < 0 {
d = -d
}
if best < 0 || d < bestD {
best, bestD, tie = i, d, false
} else if d == bestD {
tie = true
}
}
if best < 0 || tie {
return entry{}, false
}
return cands[best], true
}
const linkPrefix = "https://office.pro-dukt.de/Products/Files/DocEditor.aspx?fileid="
type entry struct {
id, path, title, norm string
}
func norm(s string) string { return nonAln.ReplaceAllString(strings.ToLower(s), "") }
func main() {
if len(os.Args) < 4 {
fmt.Fprintln(os.Stderr, "usage: kontolink <IN_XLSX> <INDEX_TSV> <OUT_XLSX>")
os.Exit(2)
}
in, idxPath, out := os.Args[1], os.Args[2], os.Args[3]
idxRaw, err := os.ReadFile(idxPath)
if err != nil {
panic(err)
}
var entries []entry
for _, line := range strings.Split(string(idxRaw), "\n") {
parts := strings.Split(line, "\t")
if len(parts) < 4 || parts[0] == "" {
continue
}
entries = append(entries, entry{id: parts[0], path: parts[2], title: parts[3], norm: norm(parts[3])})
}
f, err := excelize.OpenFile(in)
if err != nil {
panic(err)
}
defer f.Close()
sheet := f.GetSheetList()[0]
rows, err := f.GetRows(sheet)
if err != nil {
panic(err)
}
// optional 5th arg: amounts TSV "file_id\ttitle\tamount" (see cmd/pdfamount)
var amts []amtEntry
if len(os.Args) >= 6 && os.Args[5] != "" {
amts = loadAmounts(os.Args[5])
}
used := map[string]bool{}
for _, r := range rows {
if m := fileID.FindStringSubmatch(cell(r, 7)); m != nil {
used[m[1]] = true
}
}
var linked, byBeleg, bySupplier, byAmount, unmatched, ambiguous int
for i, r := range rows {
if i == 0 || !dateRe.MatchString(cell(r, 0)) || strings.TrimSpace(cell(r, 7)) != "" {
continue
}
beleg := norm(cell(r, 3))
supplier := supplierNorm(cell(r, 2))
month := monthYear(cell(r, 0))
rd, _ := parseDay(cell(r, 0))
e, kind, ok := pick(entries, used, beleg, supplier, month, rd)
if !ok {
if ae, aok := amountPick(amts, used, supplier, rowAmount(r), rd); aok {
e, kind, ok = entry{id: ae.id, title: ae.title}, "amount", true
}
}
if !ok {
if beleg != "" {
ambiguous++
} else {
unmatched++
}
continue
}
ref, _ := excelize.CoordinatesToCellName(8, i+1)
if err := f.SetCellValue(sheet, ref, linkPrefix+e.id); err != nil {
panic(err)
}
used[e.id] = true
linked++
switch kind {
case "beleg":
byBeleg++
case "supplier":
bySupplier++
case "amount":
byAmount++
}
fmt.Printf("row %3d %-30s -> %s [%s]\n", i+1, cell(r, 2), e.title, kind)
}
if err := f.SaveAs(out); err != nil {
panic(err)
}
fmt.Printf("\nlinked=%d (beleg=%d, supplier=%d, amount=%d), ambiguous=%d, no-candidate=%d\n",
linked, byBeleg, bySupplier, byAmount, ambiguous, unmatched)
// Optional 4th arg: source OnlyOffice file id. Try to update it in place;
// if it is locked (OnlyOffice 500), upload a "(links)" copy next to it.
if len(os.Args) >= 5 && os.Args[4] != "" {
c := onlyoffice.NewClient(onlyoffice.GetEnvironmentCredentials())
ctx, cancel := context.WithTimeout(context.Background(), 120*time.Second)
defer cancel()
var src *onlyoffice.FileEntry
if derr := onlyoffice.DoRetry(ctx, onlyoffice.DefaultRetryPolicy(), func() error {
var err error
src, err = c.GetFile(ctx, os.Args[4])
return err
}); derr != nil {
panic(derr)
}
folder, title := "", ""
if src.FolderID != nil {
folder = src.FolderID.String()
}
if src.Title != nil {
title = *src.Title
}
uderr := onlyoffice.DoRetry(ctx, onlyoffice.DefaultRetryPolicy(), func() error {
_, err := c.UpdateFile(ctx, os.Args[4], out)
return err
})
if uderr == nil {
fmt.Printf("updated file %s in place\n", os.Args[4])
return
}
fmt.Printf("in-place update failed (locked?); uploading a copy to folder %s\n", folder)
ext := filepath.Ext(title)
name := strings.TrimSuffix(title, ext) + " (links)" + ext
tmp := filepath.Join(os.TempDir(), name)
data, _ := os.ReadFile(out)
if err := os.WriteFile(tmp, data, 0o600); err != nil {
panic(err)
}
if derr := onlyoffice.DoRetry(ctx, onlyoffice.DefaultRetryPolicy(), func() error {
_, _, err := c.UploadToFolderReplacing(ctx, folder, tmp)
return err
}); derr != nil {
panic(derr)
}
fmt.Printf("uploaded copy: %s -> folder %s\n", name, folder)
}
}
func cell(r []string, i int) string {
if i < len(r) {
return strings.TrimSpace(r[i])
}
return ""
}
func supplierNorm(s string) string {
s = strings.ToUpper(s)
if i := strings.Index(s, ","); i >= 0 {
s = s[:i]
}
for _, w := range []string{"RE FEHLT", "GS FEHLT", "WOFR", "WOFÜR"} {
s = strings.ReplaceAll(s, w, "")
}
return norm(s)
}
func monthYear(date string) string {
if len(date) == 10 {
return date[6:10] + "-" + date[3:5]
}
return ""
}
// pick returns an unused candidate. Beleg match wins; supplier+month is a
// fallback. When several candidates qualify, the one closest in time to the row
// date wins; a tie is rejected (ambiguous) rather than guessed.
func pick(entries []entry, used map[string]bool, beleg, supplier, month string, rd time.Time) (entry, string, bool) {
free := func(e entry) bool { return !used[e.id] }
if len(beleg) >= 5 {
var inMonth []entry
for _, e := range entries {
if free(e) && belegMatches(e.norm, beleg) &&
(month == "" || strings.Contains(e.title, month)) {
inMonth = append(inMonth, e)
}
}
if supplier != "" {
var s []entry
for _, e := range inMonth {
if strings.Contains(e.norm, supplier) {
s = append(s, e)
}
}
if len(s) > 0 {
inMonth = s
}
}
inMonth = topRank(inMonth)
if e, ok := nearest(inMonth, rd); ok {
return e, "beleg", true
}
// A Beleg is present but no file carries it: do NOT fall back to a
// supplier guess (that links the wrong invoice).
return entry{}, "", false
}
if supplier != "" && month != "" {
var c []entry
for _, e := range entries {
if free(e) && strings.Contains(e.norm, supplier) &&
strings.Contains(e.title, month) && strings.Contains(e.norm, "rechnung") {
c = append(c, e)
}
}
c = topRank(c)
if e, ok := nearest(c, rd); ok {
return e, "supplier", true
}
}
return entry{}, "", false
}
// topRank keeps only the highest-ranked candidates (real invoice over copy /
// dupe / op), so a tie with a duplicate does not mask the real file.
func topRank(cands []entry) []entry {
if len(cands) < 2 {
return cands
}
best := 0
for _, e := range cands {
if rank(e) > best {
best = rank(e)
}
}
out := cands[:0]
for _, e := range cands {
if rank(e) == best {
out = append(out, e)
}
}
return out
}
func rank(e entry) int {
s := 0
if strings.Contains(e.path, "/2025") || strings.Contains(e.path, "/2024") {
s += 4
}
if strings.Contains(e.norm, "rechnung") {
s += 2
}
if strings.Contains(e.norm, "dupe") || strings.Contains(e.norm, "copy") ||
strings.Contains(e.norm, "op") {
s--
}
return s
}
// belegMatches reports whether a Beleg identifies the file: the whole normalized
// Beleg appears, or (for long numeric Belege, e.g. "24/641393110") an 8-digit
// window of its longest digit run appears.
func belegMatches(titleNorm, beleg string) bool {
if strings.Contains(titleNorm, beleg) {
return true
}
run := longestDigitRun(beleg)
for i := 0; i+8 <= len(run); i++ {
if strings.Contains(titleNorm, run[i:i+8]) {
return true
}
}
return false
}
func longestDigitRun(s string) string {
var best, cur strings.Builder
for _, r := range s {
if r >= '0' && r <= '9' {
cur.WriteRune(r)
if cur.Len() > best.Len() {
best.Reset()
best.WriteString(cur.String())
}
} else {
cur.Reset()
}
}
return best.String()
}
type amtEntry struct {
id string
title string
norm string
amount float64
date time.Time
hasDate bool
}
func loadAmounts(path string) []amtEntry {
raw, err := os.ReadFile(path)
if err != nil {
return nil
}
var out []amtEntry
for _, line := range strings.Split(string(raw), "\n") {
p := strings.Split(line, "\t")
if len(p) < 3 {
continue
}
v, err := strconv.ParseFloat(strings.TrimSpace(p[2]), 64)
if err != nil {
continue
}
e := amtEntry{id: p[0], title: p[1], norm: norm(p[1]), amount: v}
if len(p[1]) >= 10 {
if t, err := time.Parse("2006-01-02", p[1][:10]); err == nil {
e.date, e.hasDate = t, true
}
}
out = append(out, e)
}
return out
}
func rowAmount(r []string) float64 {
if v := parseAmount(cell(r, 4)); v != 0 {
return v
}
return parseAmount(cell(r, 5))
}
func parseAmount(s string) float64 {
s = strings.ReplaceAll(s, "€", "")
s = strings.ReplaceAll(s, " ", "")
s = strings.ReplaceAll(s, ",", ".")
if s == "" {
return 0
}
v, err := strconv.ParseFloat(s, 64)
if err != nil {
return 0
}
return v
}
// amountPick matches a row to an O2 invoice by amount + nearest date. Scoped to
// Telefonica/O2 rows and O2 files, so it cannot cross-link other suppliers.
func amountPick(amts []amtEntry, used map[string]bool, supplier string, amt float64, rd time.Time) (amtEntry, bool) {
if amt <= 0 || len(amts) == 0 {
return amtEntry{}, false
}
if !strings.Contains(supplier, "telefonica") && !strings.Contains(supplier, "o2") {
return amtEntry{}, false
}
var cands []amtEntry
for _, a := range amts {
if used[a.id] || !strings.Contains(a.norm, "o2") {
continue
}
d := a.amount - amt
if d < 0 {
d = -d
}
if d > 0.005 {
continue
}
if a.hasDate && !rd.IsZero() {
days := a.date.Sub(rd).Hours() / 24
if days < 0 {
days = -days
}
if days > 75 {
continue
}
}
cands = append(cands, a)
}
if len(cands) == 1 {
return cands[0], true
}
best, bestD, tie := -1, 0.0, false
for i, a := range cands {
if !a.hasDate {
continue
}
d := a.date.Sub(rd).Hours() / 24
if d < 0 {
d = -d
}
if best < 0 || d < bestD {
best, bestD, tie = i, d, false
} else if d == bestD {
tie = true
}
}
if best < 0 || tie {
return amtEntry{}, false
}
return cands[best], true
}
+25 -3
View File
@@ -64,6 +64,7 @@ func catalogScanContactsCmd() *cobra.Command {
func catalogScanProjectsCmd() *cobra.Command {
var outPath string
var maxDepth int
var configPath string
cmd := &cobra.Command{
Use: "scan-projects",
Short: "Git roots / remotes / top-level dirs → company rows",
@@ -72,7 +73,11 @@ func catalogScanProjectsCmd() *cobra.Command {
if root == "" {
return fmt.Errorf("--root is required")
}
doc, err := catalog.ScanProjectsRoot(root, maxDepth)
cl, err := catalogClassifier(configPath)
if err != nil {
return err
}
doc, err := catalog.ScanProjectsRootOpts(root, maxDepth, catalog.ScanOptions{Classifier: cl})
if err != nil {
return err
}
@@ -81,6 +86,7 @@ func catalogScanProjectsCmd() *cobra.Command {
}
cmd.Flags().String("root", "", "projects directory (local path)")
cmd.Flags().IntVar(&maxDepth, "max-depth", 4, "max directory depth for git roots")
cmd.Flags().StringVar(&configPath, "config", "", "classification rules YAML (default $OO_CATALOG_CONFIG)")
cmd.Flags().StringVarP(&outPath, "out", "O", "", "write YAML to this path")
_ = cmd.MarkFlagRequired("root")
return cmd
@@ -89,6 +95,7 @@ func catalogScanProjectsCmd() *cobra.Command {
func catalogScanThunderbirdCmd() *cobra.Command {
var outPath string
var mboxHeaders bool
var configPath string
cmd := &cobra.Command{
Use: "scan-thunderbird",
Short: "Thunderbird profiles: abook/history.mab + Gloda SQLite contacts",
@@ -98,13 +105,18 @@ func catalogScanThunderbirdCmd() *cobra.Command {
- optional --mbox-headers: From/To/Cc/Reply-To from mbox folder files (no bodies)
Noisy senders (noreply, Amazon marketplace, GitHub reply, …) are skipped.
Default zone is private; known work domains (e.g. wheregroup.com) get zone=warm role=work.`,
Default zone is private; work domains/names are classified from the rules in
$OO_CATALOG_CONFIG (or --config) — none are hardcoded.`,
RunE: func(cmd *cobra.Command, args []string) error {
root, _ := cmd.Flags().GetString("root")
if root == "" {
return fmt.Errorf("--root is required")
}
doc, err := catalog.ScanThunderbirdRootOpts(root, catalog.ScanOptions{MboxHeaders: mboxHeaders})
cl, err := catalogClassifier(configPath)
if err != nil {
return err
}
doc, err := catalog.ScanThunderbirdRootOpts(root, catalog.ScanOptions{MboxHeaders: mboxHeaders, Classifier: cl})
if err != nil {
return err
}
@@ -113,11 +125,21 @@ Default zone is private; known work domains (e.g. wheregroup.com) get zone=warm
}
cmd.Flags().String("root", "", "Thunderbird profile or parent directory (local path)")
cmd.Flags().BoolVar(&mboxHeaders, "mbox-headers", false, "also extract emails from mbox From/To/Cc headers")
cmd.Flags().StringVar(&configPath, "config", "", "classification rules YAML (default $OO_CATALOG_CONFIG)")
cmd.Flags().StringVarP(&outPath, "out", "O", "", "write YAML to this path")
_ = cmd.MarkFlagRequired("root")
return cmd
}
// catalogClassifier loads classification rules from --config, else
// $OO_CATALOG_CONFIG, else the neutral default (no rules).
func catalogClassifier(configPath string) (*catalog.Classifier, error) {
if configPath == "" {
return catalog.LoadClassifierFromEnv()
}
return catalog.LoadClassifier(configPath)
}
func catalogMergeCmd() *cobra.Command {
var inputs []string
var outPath string
+3 -1
View File
@@ -31,7 +31,9 @@ func init() {
}
// execute runs the root command. Exported only to main.go in the same package.
func execute() error { return rootCmd.Execute() }
// .env is loaded CLI-wide so non-authenticating commands (e.g. catalog scans)
// also see configuration such as OO_CATALOG_CONFIG.
func execute() error { bootstrap.LoadEnv(); return rootCmd.Execute() }
// newOO loads env (only .env in CWD) and returns an authenticated client.
// godotenv is a CLI-only concern; the library itself never loads dotfiles.
+1 -1
View File
@@ -7,7 +7,7 @@ import (
var crmCmd = &cobra.Command{
Use: "crm",
Short: "CRM maintenance (dedupe, cleanup)",
Short: "CRM maintenance (audit, dedupe, cleanup)",
}
func init() {
+63
View File
@@ -0,0 +1,63 @@
package main
import (
"encoding/json"
"fmt"
"os"
"sort"
"github.com/spf13/cobra"
)
func init() {
crmCmd.AddCommand(crmAuditCmd())
}
func crmAuditCmd() *cobra.Command {
var outPath string
cmd := &cobra.Command{
Use: "audit",
Short: "Audit opportunities (files/tasks/members per deal) and classify",
Long: `Lists every opportunity with its file/task/member counts and a coarse class
(ok | dup | empty | junk-title). --out writes the full JSON audit for later use.`,
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
audits, err := c.AuditOpportunities(cmd.Context())
if err != nil {
return err
}
if outPath != "" {
b, merr := json.MarshalIndent(audits, "", " ")
if merr != nil {
return merr
}
if werr := os.WriteFile(outPath, b, 0o644); werr != nil {
return werr
}
}
byClass := map[string]int{}
for _, a := range audits {
byClass[a.Class]++
}
classes := make([]string, 0, len(byClass))
for k := range byClass {
classes = append(classes, k)
}
sort.Strings(classes)
rows := make([]map[string]any, 0, len(classes))
for _, k := range classes {
rows = append(rows, map[string]any{"class": k, "count": byClass[k]})
}
printTable([]string{"class", "count"}, rows)
if outPath != "" {
fmt.Fprintf(cmd.OutOrStdout(), "wrote %d audits → %s\n", len(audits), outPath)
}
return nil
},
}
cmd.Flags().StringVar(&outPath, "out", "", "write audit JSON to this file")
return cmd
}
+190
View File
@@ -1,8 +1,10 @@
package main
import (
"context"
"fmt"
"os"
"strings"
"time"
onlyoffice "github.com/eslider/go-onlyoffice"
@@ -26,6 +28,8 @@ func davCmd() *cobra.Command {
cmd.AddCommand(davMoveCmd())
cmd.AddCommand(davCopyCmd())
cmd.AddCommand(davMkdirCmd())
cmd.AddCommand(davEnsurePathCmd())
cmd.AddCommand(davUploadCmd())
cmd.AddCommand(davRemoveCmd())
cmd.AddCommand(davRenameFileCmd())
cmd.AddCommand(davRenameFolderCmd())
@@ -187,6 +191,192 @@ func davMkdirCmd() *cobra.Command {
}
}
func davEnsurePathCmd() *cobra.Command {
var under string
cmd := &cobra.Command{
Use: "ensure-path PATH",
Short: "Resolve or create nested Documents folders (mkdir -p), print the final folder id",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
root := under
if root == "" {
root, err = myDocumentsID(cmd.Context(), c)
if err != nil {
return err
}
}
f, err := ensurePath(cmd.Context(), c.FileStore(onlyoffice.ProviderDAV), root, args[0])
if err != nil {
return err
}
printObject(map[string]any{"id": f.ID, "title": f.Title, "under": root})
return nil
},
}
cmd.Flags().StringVar(&under, "under", "", "parent folder id (default: My documents)")
return cmd
}
func davUploadCmd() *cobra.Command {
var replace bool
cmd := &cobra.Command{
Use: "upload DEST_FOLDER_ID LOCAL_FILE [LOCAL_FILE...]",
Short: "Upload local file(s) into a Documents folder (multipart, upsert by name)",
Args: cobra.MinimumNArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
results, err := uploadLocal(cmd.Context(), c, args[0], args[1:], replace)
if err != nil {
return err
}
rows := make([]map[string]any, 0, len(results))
for _, r := range results {
replaced := make([]string, 0, len(r.Replaced))
for _, id := range r.Replaced {
replaced = append(replaced, fmt.Sprint(id))
}
rows = append(rows, map[string]any{
"id": r.Entry.ID,
"title": r.Entry.Title,
"size": r.Entry.Size,
"replaced": strings.Join(replaced, ", "),
})
}
printTable([]string{"id", "title", "size", "replaced"}, rows)
return nil
},
}
cmd.Flags().BoolVar(&replace, "replace", true, "replace the same logical file (stem, legacy↔OOXML ext) before upload (default); false = fail if name taken")
return cmd
}
// uploadedFile pairs an upload result with the ids deleted to make room for it.
type uploadedFile struct {
Entry onlyoffice.Entry
Replaced []int
}
// uploader is the slice of *onlyoffice.Client that dav upload needs, so the
// command logic is unit-testable with a fake.
type uploader interface {
UploadToFolderReplacing(ctx context.Context, folderID, localPath string) (*onlyoffice.FileEntry, []int, error)
AssertNoFileConflict(ctx context.Context, folderID, localPath string) error
UploadToFolder(ctx context.Context, folderID, localPath string) (*onlyoffice.FileEntry, error)
}
// uploadLocal uploads each local file into folderID. With replace the existing
// same logical file is overwritten in place (the UploadToFolderReplacing
// upsert, which matches the server-converted extension too); without replace a
// name clash fails with ErrFileExists before anything is sent.
func uploadLocal(ctx context.Context, up uploader, folderID string, paths []string, replace bool) ([]uploadedFile, error) {
results := make([]uploadedFile, 0, len(paths))
for _, path := range paths {
var (
fe *onlyoffice.FileEntry
deleted []int
err error
)
if replace {
fe, deleted, err = up.UploadToFolderReplacing(ctx, folderID, path)
} else {
if err := up.AssertNoFileConflict(ctx, folderID, path); err != nil {
return nil, err
}
fe, err = up.UploadToFolder(ctx, folderID, path)
}
if err != nil {
return nil, fmt.Errorf("dav upload: %s: %w", path, err)
}
results = append(results, uploadedFile{
Entry: onlyoffice.FileEntryToEntry(fe, onlyoffice.ProviderDAV),
Replaced: deleted,
})
}
return results, nil
}
// ensurePath resolves-or-creates every "/"-separated segment under rootID
// through store. Existing folders are reused by title (case-insensitive), so
// repeated calls return the same id without creating duplicates.
func ensurePath(ctx context.Context, store onlyoffice.FileStore, rootID, path string) (onlyoffice.Entry, error) {
segments, err := splitDavPath(path)
if err != nil {
return onlyoffice.Entry{}, err
}
parent := rootID
var current onlyoffice.Entry
for _, name := range segments {
entries, err := store.List(ctx, parent)
if err != nil {
return onlyoffice.Entry{}, fmt.Errorf("dav ensure-path: list %s: %w", parent, err)
}
found := false
for _, e := range entries {
if e.Kind == onlyoffice.Folder && strings.EqualFold(strings.TrimSpace(e.Title), name) {
current, found = e, true
break
}
}
if !found {
current, err = store.CreateFolder(ctx, parent, name)
if err != nil {
return onlyoffice.Entry{}, fmt.Errorf("dav ensure-path: mkdir %s/%s: %w", parent, name, err)
}
}
parent = current.ID
}
return current, nil
}
// splitDavPath normalizes a Documents path into non-empty segments. "." is
// ignored; ".." is rejected rather than creating a literal folder named "..".
func splitDavPath(path string) ([]string, error) {
var segments []string
for _, s := range strings.Split(path, "/") {
s = strings.TrimSpace(s)
switch s {
case "", ".":
continue
case "..":
return nil, fmt.Errorf("dav ensure-path: %q not allowed", s)
}
segments = append(segments, s)
}
if len(segments) == 0 {
return nil, fmt.Errorf("dav ensure-path: empty path")
}
return segments, nil
}
// myDocumentsID resolves the concrete id of the "My documents" section so
// ensure-path can create folders under it (the symbolic "@my" is list-only on
// some portals). rootFolderType 5 = My; the title match covers portals that
// omit rootFolderType.
func myDocumentsID(ctx context.Context, c *onlyoffice.Client) (string, error) {
sections, err := c.ListDavSections(ctx)
if err != nil {
return "", err
}
for _, s := range sections {
if s.RootType == 5 && s.ID != "" {
return s.ID, nil
}
}
for _, s := range sections {
if strings.EqualFold(strings.TrimSpace(s.Title), "My documents") && s.ID != "" {
return s.ID, nil
}
}
return "", fmt.Errorf("dav: no My documents section in @root; pass --under FOLDER_ID")
}
func davRemoveCmd() *cobra.Command {
var folderIDs []string
cmd := &cobra.Command{
+121
View File
@@ -0,0 +1,121 @@
//go:build integration
package main
import (
"context"
"errors"
"os"
"path/filepath"
"strings"
"testing"
"time"
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/eslider/go-onlyoffice/cmd/internal/bootstrap"
)
// TestIntegrationDavEnsurePathUpload exercises the dav ensure-path/upload
// helpers against a live instance: idempotent folder creation, upload, replace
// and no-replace conflict. Destructive (creates and removes a throwaway
// folder); skips cleanly without credentials. Only run against instances you
// own: go test -tags=integration ./cmd/oo -run TestIntegrationDavEnsurePathUpload
func TestIntegrationDavEnsurePathUpload(t *testing.T) {
ctx := context.Background()
c, err := bootstrap.NewClient(ctx)
if err != nil {
t.Skipf("no live OnlyOffice credentials: %v", err)
}
root, err := myDocumentsID(ctx, c)
if err != nil {
t.Skipf("My documents root unavailable: %v", err)
}
store := c.FileStore(onlyoffice.ProviderDAV)
suffix := time.Now().UTC().Format("20060102-150405")
base := "oo-it-" + suffix
first, err := ensurePath(ctx, store, root, base+"/nested")
if err != nil {
t.Fatalf("ensurePath: %v", err)
}
baseID := findFolderID(t, ctx, store, root, base)
t.Cleanup(func() { _ = store.Delete(context.Background(), []string{baseID}) })
second, err := ensurePath(ctx, store, root, base+"/nested")
if err != nil {
t.Fatalf("ensurePath (second): %v", err)
}
if second.ID != first.ID {
t.Fatalf("ensure-path not idempotent: %s != %s", second.ID, first.ID)
}
if n := countFolders(t, ctx, store, baseID, "nested"); n != 1 {
t.Fatalf("nested folder duplicated: %d children named nested, want 1", n)
}
local := filepath.Join(t.TempDir(), "oo-it-"+suffix+".xlsx")
if err := os.WriteFile(local, []byte("integration dav upload "+suffix+"\n"), 0o600); err != nil {
t.Fatal(err)
}
name := filepath.Base(local)
if _, err := uploadLocal(ctx, c, first.ID, []string{local}, true); err != nil {
t.Fatalf("uploadLocal: %v", err)
}
if n := countFiles(t, ctx, store, first.ID, name); n != 1 {
t.Fatalf("after upload: %d files named %s, want 1", n, name)
}
if _, err := uploadLocal(ctx, c, first.ID, []string{local}, true); err != nil {
t.Fatalf("uploadLocal (replace): %v", err)
}
if n := countFiles(t, ctx, store, first.ID, name); n != 1 {
t.Fatalf("after replace: %d files named %s, want 1", n, name)
}
if _, err := uploadLocal(ctx, c, first.ID, []string{local}, false); !errors.Is(err, onlyoffice.ErrFileExists) {
t.Fatalf("no-replace conflict err = %v, want ErrFileExists", err)
}
}
func findFolderID(t *testing.T, ctx context.Context, store onlyoffice.FileStore, parent, title string) string {
t.Helper()
for _, e := range mustList(t, ctx, store, parent) {
if e.Kind == onlyoffice.Folder && strings.EqualFold(e.Title, title) {
return e.ID
}
}
t.Fatalf("folder %q not found under %s", title, parent)
return ""
}
func countFolders(t *testing.T, ctx context.Context, store onlyoffice.FileStore, parent, title string) int {
t.Helper()
n := 0
for _, e := range mustList(t, ctx, store, parent) {
if e.Kind == onlyoffice.Folder && strings.EqualFold(e.Title, title) {
n++
}
}
return n
}
func countFiles(t *testing.T, ctx context.Context, store onlyoffice.FileStore, parent, title string) int {
t.Helper()
n := 0
for _, e := range mustList(t, ctx, store, parent) {
if e.Kind == onlyoffice.File && e.Title == title {
n++
}
}
return n
}
func mustList(t *testing.T, ctx context.Context, store onlyoffice.FileStore, parent string) []onlyoffice.Entry {
t.Helper()
entries, err := store.List(ctx, parent)
if err != nil {
t.Fatalf("List(%s): %v", parent, err)
}
return entries
}
+260
View File
@@ -0,0 +1,260 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"strings"
"testing"
onlyoffice "github.com/eslider/go-onlyoffice"
)
// fakeFolderStore is an in-memory onlyoffice.FileStore for the ensure-path
// logic. It records created folders so idempotency can be asserted.
type fakeFolderStore struct {
nextID int
entries map[string][]onlyoffice.Entry
created []string
}
func newFakeFolderStore() *fakeFolderStore {
return &fakeFolderStore{entries: map[string][]onlyoffice.Entry{}}
}
func (f *fakeFolderStore) Name() string { return "fake" }
func (f *fakeFolderStore) List(_ context.Context, parent string) ([]onlyoffice.Entry, error) {
return append([]onlyoffice.Entry(nil), f.entries[parent]...), nil
}
func (f *fakeFolderStore) CreateFolder(_ context.Context, parent, title string) (onlyoffice.Entry, error) {
f.nextID++
e := onlyoffice.Entry{
ID: fmt.Sprintf("id-%d", f.nextID),
ParentID: parent,
Title: title,
Kind: onlyoffice.Folder,
}
f.entries[parent] = append(f.entries[parent], e)
f.created = append(f.created, parent+"/"+title)
return e, nil
}
func (f *fakeFolderStore) Stat(context.Context, string) (onlyoffice.Entry, error) {
return onlyoffice.Entry{}, errors.New("not implemented")
}
func (f *fakeFolderStore) Upload(context.Context, string, string, io.Reader) (onlyoffice.Entry, error) {
return onlyoffice.Entry{}, errors.New("not implemented")
}
func (f *fakeFolderStore) Download(context.Context, string, io.Writer) (int64, error) {
return 0, errors.New("not implemented")
}
func (f *fakeFolderStore) Move(context.Context, []string, string) error {
return errors.New("not implemented")
}
func (f *fakeFolderStore) Copy(context.Context, []string, string) error {
return errors.New("not implemented")
}
func (f *fakeFolderStore) Rename(context.Context, string, string) error {
return errors.New("not implemented")
}
func (f *fakeFolderStore) Delete(context.Context, []string) error {
return errors.New("not implemented")
}
// fakeUploader is the *onlyoffice.Client slice dav upload depends on.
type fakeUploader struct {
conflict bool
uploads []string
replaced []string
}
func (f *fakeUploader) UploadToFolderReplacing(_ context.Context, folderID, localPath string) (*onlyoffice.FileEntry, []int, error) {
f.uploads = append(f.uploads, localPath)
ids := []int(nil)
if f.conflict {
ids = []int{7}
f.replaced = append(f.replaced, folderID+"#7")
}
return fakeFileEntry(99, localPath, folderID), ids, nil
}
func (f *fakeUploader) AssertNoFileConflict(_ context.Context, folderID, localPath string) error {
if f.conflict {
return fmt.Errorf("%w: conflict in folder %s for %s", onlyoffice.ErrFileExists, folderID, localPath)
}
return nil
}
func (f *fakeUploader) UploadToFolder(_ context.Context, folderID, localPath string) (*onlyoffice.FileEntry, error) {
f.uploads = append(f.uploads, localPath)
return fakeFileEntry(99, localPath, folderID), nil
}
func fakeFileEntry(id int, localPath, folderID string) *onlyoffice.FileEntry {
num := json.Number(fmt.Sprintf("%d", id))
title := localPath[strings.LastIndex(localPath, "/")+1:]
return &onlyoffice.FileEntry{
ID: &num,
Title: &title,
FolderID: &num,
}
}
func TestEnsurePathCreatesNestedFolders(t *testing.T) {
store := newFakeFolderStore()
ctx := context.Background()
got, err := ensurePath(ctx, store, "root", "Banks/Caixa")
if err != nil {
t.Fatalf("ensurePath: %v", err)
}
if got.Kind != onlyoffice.Folder || got.ID == "" {
t.Fatalf("ensurePath returned %+v, want a folder with an id", got)
}
if want := []string{"root/Banks", "id-1/Caixa"}; !equalStrings(store.created, want) {
t.Fatalf("created %v, want %v", store.created, want)
}
}
func TestEnsurePathIsIdempotent(t *testing.T) {
store := newFakeFolderStore()
ctx := context.Background()
first, err := ensurePath(ctx, store, "root", "Banks/Caixa")
if err != nil {
t.Fatalf("ensurePath first: %v", err)
}
created := len(store.created)
second, err := ensurePath(ctx, store, "root", "Banks/Caixa")
if err != nil {
t.Fatalf("ensurePath second: %v", err)
}
if second.ID != first.ID {
t.Fatalf("second run id = %q, want %q (no duplicate)", second.ID, first.ID)
}
if len(store.created) != created {
t.Fatalf("second run created folders: %v", store.created)
}
}
func TestEnsurePathReusesExistingFolder(t *testing.T) {
store := newFakeFolderStore()
store.entries["root"] = []onlyoffice.Entry{
{ID: "banks-id", Title: "Banks", Kind: onlyoffice.Folder},
}
store.entries["banks-id"] = []onlyoffice.Entry{
{ID: "caixa-id", Title: "Caixa", Kind: onlyoffice.Folder},
}
got, err := ensurePath(context.Background(), store, "root", "Banks/Caixa")
if err != nil {
t.Fatalf("ensurePath: %v", err)
}
if got.ID != "caixa-id" {
t.Fatalf("id = %q, want caixa-id", got.ID)
}
if len(store.created) != 0 {
t.Fatalf("created %v, want none", store.created)
}
}
func TestEnsurePathRejectsEmptyAndDotDot(t *testing.T) {
store := newFakeFolderStore()
for _, path := range []string{"", "/", "Banks/../Caixa"} {
if _, err := ensurePath(context.Background(), store, "root", path); err == nil {
t.Fatalf("ensurePath(%q) = nil error, want failure", path)
}
}
}
func TestUploadLocalReplacesByDefault(t *testing.T) {
up := &fakeUploader{conflict: true}
results, err := uploadLocal(context.Background(), up, "folder-1", []string{"a/f.xlsx"}, true)
if err != nil {
t.Fatalf("uploadLocal: %v", err)
}
if len(results) != 1 || results[0].Entry.Title != "f.xlsx" {
t.Fatalf("results = %+v", results)
}
if len(results[0].Replaced) != 1 || results[0].Replaced[0] != 7 {
t.Fatalf("replaced = %v, want [7]", results[0].Replaced)
}
if len(up.replaced) != 1 {
t.Fatalf("UploadToFolderReplacing not used for replace: %v", up.replaced)
}
}
func TestUploadLocalNoReplaceFailsOnConflict(t *testing.T) {
up := &fakeUploader{conflict: true}
_, err := uploadLocal(context.Background(), up, "folder-1", []string{"a/f.xlsx"}, false)
if !errors.Is(err, onlyoffice.ErrFileExists) {
t.Fatalf("err = %v, want ErrFileExists", err)
}
if len(up.uploads) != 0 {
t.Fatalf("uploaded despite conflict: %v", up.uploads)
}
}
func TestUploadLocalNoReplaceUploadsWhenFree(t *testing.T) {
up := &fakeUploader{}
results, err := uploadLocal(context.Background(), up, "folder-1", []string{"a/f.xlsx", "a/g.pdf"}, false)
if err != nil {
t.Fatalf("uploadLocal: %v", err)
}
if len(results) != 2 {
t.Fatalf("results = %+v", results)
}
if len(up.uploads) != 2 {
t.Fatalf("uploads = %v", up.uploads)
}
}
func TestDavRegistersUploadAndEnsurePath(t *testing.T) {
for _, name := range []string{"ensure-path", "upload"} {
cmd, _, err := rootCmd.Find([]string{"dav", name})
if err != nil {
t.Fatalf("dav %s not registered: %v", name, err)
}
if cmd.Name() != name {
t.Fatalf("resolved %q, want %q", cmd.Name(), name)
}
}
upload, _, err := rootCmd.Find([]string{"dav", "upload"})
if err != nil {
t.Fatal(err)
}
flag := upload.Flags().Lookup("replace")
if flag == nil || flag.DefValue != "true" {
t.Fatalf("upload --replace flag = %+v, want default true", flag)
}
ensure, _, err := rootCmd.Find([]string{"dav", "ensure-path"})
if err != nil {
t.Fatal(err)
}
if ensure.Flags().Lookup("under") == nil {
t.Fatal("ensure-path missing --under flag")
}
}
func equalStrings(a, b []string) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}
+306 -50
View File
@@ -1,15 +1,19 @@
package main
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"strconv"
"strings"
"time"
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/eslider/go-onlyoffice/internal/docpipe"
"github.com/eslider/go-onlyoffice/internal/xlspipe"
"github.com/spf13/cobra"
)
@@ -26,12 +30,16 @@ func docsCmd() *cobra.Command {
OnlyOffice Documents UI is poor for .md/.txt — keep sources in git, store .docx in OO.
Upload Markdown as DOCX: oo docs put-md PROJECT_ID file.md
Upload plain text: oo docs put-txt PROJECT_ID file.txt (preserves line breaks)
Upload/generate XLSX: oo docs put-xlsx PROJECT_ID [--template cutover-portugal | FILE.xlsx]
Upload XLSX: oo docs put-xlsx PROJECT_ID FILE.xlsx
Read an OO file as MD: oo docs as-md FILE_ID
OCR a scan locally: oo docs ocr scan.pdf --md out.md
Structured OCR (hOCR→MD): oo docs hocr scan.jpg --md out.md --yaml out.yml`,
}
cmd.AddCommand(docsConvertCmd())
cmd.AddCommand(docsPDFCmd())
cmd.AddCommand(docsPresignedCmd())
cmd.AddCommand(docsCSVCmd())
cmd.AddCommand(docsJSONCmd())
cmd.AddCommand(docsOptimizeCmd())
cmd.AddCommand(docsOCRCmd())
cmd.AddCommand(docsHOCRCmd())
@@ -61,6 +69,293 @@ func docsToolsCmd() *cobra.Command {
}
}
func docsPDFCmd() *cobra.Command {
var out, docsURL, secret, output, folder string
var stream, pipe bool
cmd := &cobra.Command{
Use: "pdf FILE_ID | PATH [ARG...]",
Short: "Convert files to PDF via the DocumentServer converter (OO file ids or local paths)",
Long: `Native OnlyOffice conversion (the engine behind the portal's "Download as PDF"):
1. GET /api/2.0/files/file/{id}/presigneduri → fetchable source URL
2. POST <docs>/converter with a JWT → converted file URL
3. download the result
Arguments may be OnlyOffice file ids OR local file paths. A local path is
uploaded to the scratch folder (--folder, default 2 = "My documents"), converted,
downloaded and then removed — so any local document yields a PDF on the fly.
Docs base defaults to $ONLYOFFICE_DOCS_URL, else $ONLYOFFICE_URL + "/ds-vpath"
(/ds-vpath is the usual reverse-proxy mount for the DocumentServer). The JWT secret is
$ONLYOFFICE_DS_SECRET (DocumentServer services.CoAuthoring.secret).`,
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
base := docsBaseURL(docsURL)
if base == "" {
return fmt.Errorf("docs base url unknown; set --docs-url or ONLYOFFICE_DOCS_URL")
}
sec := secret
if sec == "" {
sec = firstEnv("ONLYOFFICE_DS_SECRET", "OO_DS_SECRET")
}
if sec == "" {
return fmt.Errorf("JWT secret required: --secret or ONLYOFFICE_DS_SECRET")
}
ot := output
if ot == "" {
ot = "pdf"
}
toStdout := stream || pipe
if toStdout && len(args) > 1 {
return fmt.Errorf("--stream/--pipe writes one file to stdout; pass a single input")
}
for _, arg := range args {
id, local := arg, false
title := ""
if fi, statErr := os.Stat(arg); statErr == nil && !fi.IsDir() {
// Local file → temporary upload into the scratch folder.
ent, uerr := c.UploadToFolder(cmd.Context(), folder, arg)
if uerr != nil {
return fmt.Errorf("upload %s: %w", arg, uerr)
}
id = strconv.FormatInt(onlyoffice.FileEntryNumericID(ent), 10)
local = true
title = filepath.Base(arg)
} else {
if f, ferr := c.GetFile(cmd.Context(), id); ferr == nil && f != nil && f.Title != nil {
title = *f.Title
}
}
src, err := c.PresignedURI(cmd.Context(), id)
if err != nil {
return fmt.Errorf("presigneduri %s: %w", id, err)
}
res, err := c.ConvertDocument(cmd.Context(), base, sec, onlyoffice.ConvertRequest{
URL: src,
OutputType: ot,
FileType: strings.TrimPrefix(filepath.Ext(title), "."),
Title: title,
Key: fmt.Sprintf("oo-%s-%d", id, time.Now().UnixNano()),
})
if err != nil {
return fmt.Errorf("convert %s: %w", id, err)
}
var w io.Writer
dst := out
if toStdout {
w = os.Stdout
} else {
if dst == "" {
stem := strings.TrimSuffix(title, filepath.Ext(title))
if stem == "" {
stem = "file-" + id
}
dst = stem + "." + ot
}
fh, err := os.Create(dst)
if err != nil {
return err
}
w = fh
defer fh.Close()
}
n, derr := c.DownloadURLTo(cmd.Context(), res.FileURL, w)
if local {
// Best-effort cleanup of the temporary upload.
if nid, e := strconv.Atoi(id); e == nil {
_ = c.DeleteFiles(cmd.Context(), []int{nid})
}
}
if derr != nil {
return fmt.Errorf("download: %w", derr)
}
if toStdout {
// Keep stdout byte-clean for pipes; status goes to stderr.
fmt.Fprintf(os.Stderr, "converted %s -> stdout (%d bytes, %s)\n", arg, n, ot)
} else {
printObject(map[string]any{"source": arg, "fileid": id, "title": title, "output": dst, "bytes": n, "type": ot})
}
}
return nil
},
}
cmd.Flags().StringVar(&out, "out", "", "output path (default: ./<title>.<format>)")
cmd.Flags().StringVar(&output, "to", "pdf", "output format (pdf, docx, xlsx, …)")
cmd.Flags().StringVar(&docsURL, "docs-url", "", "DocumentServer base (default $ONLYOFFICE_DOCS_URL or $ONLYOFFICE_URL/ds-vpath)")
cmd.Flags().StringVar(&secret, "secret", "", "JWT secret (default $ONLYOFFICE_DS_SECRET)")
cmd.Flags().StringVar(&folder, "folder", "2", "scratch folder id for local-file uploads")
cmd.Flags().BoolVar(&stream, "stream", false, "write the converted bytes to stdout (pipe-friendly)")
cmd.Flags().BoolVar(&pipe, "pipe", false, "alias for --stream")
return cmd
}
func docsPresignedCmd() *cobra.Command {
return &cobra.Command{
Use: "presigned FILE_ID",
Short: "Print a short-lived fetchable URI for a portal file",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
u, err := c.PresignedURI(cmd.Context(), args[0])
if err != nil {
return err
}
printObject(map[string]any{"fileid": args[0], "uri": u})
return nil
},
}
}
// loadWorkbookBytes reads an argument that is either an OnlyOffice file id
// (downloaded via the client) or a local path.
func loadWorkbookBytes(cmd *cobra.Command, c *onlyoffice.Client, arg string) ([]byte, error) {
if fi, err := os.Stat(arg); err == nil && !fi.IsDir() {
return os.ReadFile(arg)
}
var buf bytes.Buffer
if _, err := c.DownloadFile(cmd.Context(), arg, &buf); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
// parseDelimiter maps a flag value to a CSV delimiter rune ("," default).
func parseDelimiter(s string) rune {
switch s {
case "", ",":
return ','
case "\\t", "tab", "\t":
return '\t'
case ";":
return ';'
case "|":
return '|'
default:
r := []rune(s)
return r[0]
}
}
func docsCSVCmd() *cobra.Command {
var sheet int
var delim, out string
cmd := &cobra.Command{
Use: "csv SRC [SRC...]",
Short: "Export a worksheet (XLS/XLSX/ODS) to CSV (sheet-aware)",
Long: `SRC is an OnlyOffice file id or a local path. --sheet is 1-based (default 1 = first).
Why not the DocumentServer: its csv output covers only the first worksheet and
ignores a sheet selector (verified). Sheet selection and JSON use a local reader.`,
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
d := parseDelimiter(delim)
for _, arg := range args {
data, err := loadWorkbookBytes(cmd, c, arg)
if err != nil {
return fmt.Errorf("read %s: %w", arg, err)
}
text, err := onlyoffice.WorkbookSheetCSV(data, sheet-1, d)
if err != nil {
return fmt.Errorf("%s: %w", arg, err)
}
if out != "" {
if err := os.WriteFile(out, []byte(text), 0o644); err != nil {
return err
}
printObject(map[string]any{"source": arg, "sheet": sheet, "output": out, "bytes": len(text)})
continue
}
fmt.Print(text)
}
return nil
},
}
cmd.Flags().IntVar(&sheet, "sheet", 1, "worksheet number (1-based, default first)")
cmd.Flags().StringVar(&delim, "delimiter", ",", "CSV delimiter (',', ';', '|', 'tab')")
cmd.Flags().StringVar(&out, "out", "", "write to this file instead of stdout (single input)")
return cmd
}
func docsJSONCmd() *cobra.Command {
var sheet int
var out string
cmd := &cobra.Command{
Use: "json SRC [SRC...]",
Short: "Export a worksheet (XLS/XLSX/ODS) to JSON rows (first row = header)",
Long: `SRC is an OnlyOffice file id or a local path. --sheet is 1-based (default 1 = first).
Each data row becomes an object keyed by the header cells of that sheet.`,
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
for _, arg := range args {
data, err := loadWorkbookBytes(cmd, c, arg)
if err != nil {
return fmt.Errorf("read %s: %w", arg, err)
}
rows, err := onlyoffice.WorkbookSheetJSON(data, sheet-1)
if err != nil {
return fmt.Errorf("%s: %w", arg, err)
}
b, err := json.MarshalIndent(rows, "", " ")
if err != nil {
return err
}
b = append(b, '\n')
if out != "" {
if err := os.WriteFile(out, b, 0o644); err != nil {
return err
}
printObject(map[string]any{"source": arg, "sheet": sheet, "rows": len(rows), "output": out})
continue
}
os.Stdout.Write(b)
}
return nil
},
}
cmd.Flags().IntVar(&sheet, "sheet", 1, "worksheet number (1-based, default first)")
cmd.Flags().StringVar(&out, "out", "", "write to this file instead of stdout (single input)")
return cmd
}
// docsBaseURL resolves the DocumentServer base: --docs-url, $ONLYOFFICE_DOCS_URL,
// else the standard /ds-vpath reverse-proxy mount.
func docsBaseURL(flag string) string {
if flag != "" {
return flag
}
if v := os.Getenv("ONLYOFFICE_DOCS_URL"); v != "" {
return v
}
if v := firstEnv("ONLYOFFICE_URL", "ONLYOFFICE_HOST", "OO_URL"); v != "" {
return strings.TrimRight(v, "/") + "/ds-vpath"
}
return ""
}
func firstEnv(keys ...string) string {
for _, k := range keys {
if v := os.Getenv(k); v != "" {
return v
}
}
return ""
}
func strOrNil(s string) any {
if s == "" {
return nil
@@ -514,64 +809,27 @@ func docsPutTxtCmd() *cobra.Command {
}
func docsPutXlsxCmd() *cobra.Command {
var folderID, template, title, keepLocal string
var folderID, keepLocal string
var replace bool
cmd := &cobra.Command{
Use: "put-xlsx PROJECT_ID [LOCAL_XLSX]",
Short: "Upload or generate an XLSX workbook into a project (excelize templates with formulas)",
Long: `Spreadsheets live in OnlyOffice — not in git. Generate multi-sheet workbooks with
formulas (SUM/AVG, cross-sheet refs, named inputs) via --template, or upload an existing .xlsx.
Use: "put-xlsx PROJECT_ID LOCAL_XLSX",
Short: "Upload an XLSX workbook into a project (upsert by stem|ext)",
Long: `Spreadsheets live in OnlyOffice — not in git. Uploads an existing .xlsx into the
project Documents (or --folder), replacing the same stem|ext by default.
oo docs put-xlsx 218 --template cutover-portugal
oo docs put-xlsx 218 --template cutover-portugal --title 2026-08-28-cutover-budget.xlsx
oo docs put-xlsx 218 ./my.xlsx`,
Args: cobra.RangeArgs(1, 2),
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
pid := args[0]
c, err := newOO(cmd)
if err != nil {
return err
}
dir, err := os.MkdirTemp("", "oo-docs-put-xlsx-*")
if err != nil {
return err
}
defer os.RemoveAll(dir)
var xlsxPath string
var srcLabel string
switch {
case template != "":
wb, err := xlspipe.BuildTemplate(template)
if err != nil {
return err
}
name := title
if name == "" {
name = "cutover-budget.xlsx"
if template == xlspipe.TemplateCutoverPortugal {
name = "2026-08-28-cutover-budget.xlsx"
}
}
if !strings.HasSuffix(strings.ToLower(name), ".xlsx") {
name += ".xlsx"
}
xlsxPath = filepath.Join(dir, name)
if err := xlspipe.Save(wb, xlsxPath); err != nil {
wb.Close()
return err
}
wb.Close()
srcLabel = "template:" + template
case len(args) == 2:
xlsxPath = args[1]
xlsxPath := args[1]
if docpipe.Ext(xlsxPath) != ".xlsx" {
return fmt.Errorf("expected .xlsx, got %s", docpipe.Ext(xlsxPath))
}
srcLabel = xlsxPath
default:
return fmt.Errorf("pass LOCAL_XLSX or --template")
}
srcLabel := xlsxPath
if keepLocal != "" {
b, err := os.ReadFile(xlsxPath)
@@ -607,9 +865,7 @@ formulas (SUM/AVG, cross-sheet refs, named inputs) via --template, or upload an
},
}
cmd.Flags().StringVar(&folderID, "folder", "", "Documents folder id (default: project root)")
cmd.Flags().StringVar(&template, "template", "", "built-in workbook template (cutover-portugal)")
cmd.Flags().StringVar(&title, "title", "", "upload file name when using --template")
cmd.Flags().StringVar(&keepLocal, "keep-xlsx", "", "also write generated/uploaded bytes to this local path")
cmd.Flags().StringVar(&keepLocal, "keep-xlsx", "", "also copy the uploaded bytes to this local path")
cmd.Flags().BoolVar(&replace, "replace", true, "replace same stem|ext before upload (default); false = fail if name taken")
return cmd
}
+4 -4
View File
@@ -91,7 +91,7 @@ func invoiceCreateCmd() *cobra.Command {
Long: `Create a CRM invoice (Draft) with a single line.
Always pass --opportunity when a deal exists (entity link at create). Updating
--opportunity later often fails with HTTP 400 — see docs/crm-associations.md.
--opportunity later often fails with HTTP 400 — see the CRM association rules.
Example:
oo invoices create --number INV-2026-01 --contact CONTACT_ID --item ITEM_ID \
@@ -108,7 +108,7 @@ Example:
issueDate = issueDate + "T00:00:00.0000000+01:00"
}
if dueDate == "" {
dueDate = time.Now().Add(14 * 24 * time.Hour).Format("2006-01-02") + "T00:00:00.0000000+01:00"
dueDate = time.Now().Add(14*24*time.Hour).Format("2006-01-02") + "T00:00:00.0000000+01:00"
} else if !strings.Contains(dueDate, "T") {
dueDate = dueDate + "T00:00:00.0000000+01:00"
}
@@ -175,7 +175,7 @@ func invoiceUpdateCmd() *cobra.Command {
Long: `Update Draft invoice fields.
--opportunity often returns HTTP 400 on existing invoices. Prefer
oo invoices create … --opportunity, or delete+recreate. See docs/crm-associations.md.
oo invoices create … --opportunity, or delete+recreate. See the CRM association rules.
`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
@@ -274,7 +274,7 @@ func invoiceStatusCmd() *cobra.Command {
Long: `PUT /api/2.0/crm/invoice/status/{id}.
Billed invoices are not content-editable. Billed→Draft often does not work —
recreate as Draft instead (docs/crm-associations.md).
recreate as Draft instead (the CRM association rules).
`,
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
+45
View File
@@ -0,0 +1,45 @@
package main
import (
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/spf13/cobra"
)
func init() {
rootCmd.AddCommand(linkCmd())
}
// linkCmd prints deep links for file ids. Used to build third-party document
// packs whose cover embeds links to contracts and supporting files. File ids
// come from `oo projects files list` / `oo dav ls`; `oo projects files
// replace-in` keeps them clean when a document is re-uploaded.
func linkCmd() *cobra.Command {
return &cobra.Command{
Use: "link FILE_ID [FILE_ID...]",
Short: "Print OnlyOffice DocEditor deep links (Products/Files/DocEditor.aspx?fileid=…)",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
for _, id := range args {
printObject(map[string]any{
"fileid": id,
"title": fileTitle(cmd, c, id),
"url": c.FileEditorURL(id),
})
}
return nil
},
}
}
// fileTitle best-effort resolves a file title; never fails the command.
func fileTitle(cmd *cobra.Command, c *onlyoffice.Client, id string) string {
f, err := c.GetFile(cmd.Context(), id)
if err != nil || f == nil || f.Title == nil {
return ""
}
return *f.Title
}
+7 -6
View File
@@ -3,25 +3,26 @@
// Command tree is subject-based (mirrors the library split and the `tea` CLI):
//
// oo calendar list | events | add | delete
// oo projects list | get | milestones | milestone-create | create | update | delete | contacts (add|remove) | link-authors | link-git | files (list|upload|download|rename|delete|dedupe|as-md|put-md|put-txt|put-xlsx)
// oo projects list | get | milestones | milestone-create | milestone-delete | board-sync | create | update | delete | contacts (add|remove) | team (list|add|remove|set) | link-authors | link-git | files (list|upload|replace-in|update|download|rename|delete|dedupe|as-md|put-md|put-txt|put-xlsx)
// oo tasks list | get | create | update | delete | subtask add | files (list|upload|detach)
// oo users list | self (alias: oo whoami)
// oo users list | self | get | create | update | delete | block | unblock | password | check (alias: oo whoami)
// oo link FILE_ID [FILE_ID...] DocEditor deep links (Products/Files/DocEditor.aspx?fileid=…)
// oo contacts list | get | delete | info-add | merge | dedupe-info | tags | tag-add | tag-create | tag-remove
// oo persons list | create | delete | dedupe
// oo companies list | create | delete | dedupe | dedupe-persons
// oo opportunities list | get | create | update | delete | stages | member-add | dedupe | dedupe-members | fix-titles
// oo cases list | create | delete | member-add
// oo crm-tasks list | create | delete | categories | reassign-self
// oo crm cleanup
// oo crm audit | cleanup
// oo mails accounts | folders | list | get | download-attachment | draft | attach | draft-invoice | send | delete
// oo invoices list | get | create | update | pdf | pdf-cleanup | status | delete | items …
// oo docs tools | convert | optimize | ocr | hocr | as-md | put-md | put-txt | put-xlsx
// oo docs tools | convert | pdf | presigned | csv | json | optimize | ocr | hocr | as-md | put-md | put-txt | put-xlsx
// oo catalog match | merge | apply | scan-contacts | scan-projects | scan-thunderbird
// oo dav ls | move | copy | mkdir | rename-file | rename-folder | download | fileops
// oo dav ls | move | copy | mkdir | ensure-path | upload | rename-file | rename-folder | download | fileops
// oo search QUERY [--content] [--folder ID] [--limit N] [--backend oo|own] [--json]
// oo index folder FOLDER_ID | files FILE_ID... [--recursive] [--exts pdf] [--dry-run]
//
// CRM association rules: docs/crm-associations.md
// CRM association rules live with the private oo-workspace tooling.
//
// Every list supports `--output/-o json|table` (table is the default).
//
+158
View File
@@ -24,10 +24,142 @@ func init() {
projectsCmd.AddCommand(prjGetCmd())
projectsCmd.AddCommand(prjMilestonesCmd())
projectsCmd.AddCommand(prjMilestoneCreateCmd())
projectsCmd.AddCommand(prjMilestoneDeleteCmd())
projectsCmd.AddCommand(prjCreateCmd())
projectsCmd.AddCommand(prjUpdateCmd())
projectsCmd.AddCommand(prjDeleteCmd())
projectsCmd.AddCommand(prjContactsCmd())
projectsCmd.AddCommand(prjTeamCmd())
}
func prjTeamCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "team",
Short: "Project team (portal users) — CRUD",
Long: `Project team members are portal users (People), not CRM contacts.
CRM companies/persons linked to a project live under 'oo projects contacts'.`,
}
cmd.AddCommand(prjTeamListCmd())
cmd.AddCommand(prjTeamAddCmd())
cmd.AddCommand(prjTeamRemoveCmd())
cmd.AddCommand(prjTeamSetCmd())
return cmd
}
func prjTeamListCmd() *cobra.Command {
return &cobra.Command{
Use: "list PROJECT_ID",
Short: "List portal users on the project team",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
pid, err := strconv.Atoi(args[0])
if err != nil {
return fmt.Errorf("project id: %w", err)
}
c, err := newOO(cmd)
if err != nil {
return err
}
list, err := c.ListProjectTeam(cmd.Context(), pid)
if err != nil {
return err
}
printTable([]string{"id", "displayName", "userName", "email", "isAdmin"}, teamRows(list))
return nil
},
}
}
func prjTeamAddCmd() *cobra.Command {
return &cobra.Command{
Use: "add PROJECT_ID USER_ID [USER_ID...]",
Short: "Add portal user(s) to the project team",
Args: cobra.MinimumNArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
pid, err := strconv.Atoi(args[0])
if err != nil {
return fmt.Errorf("project id: %w", err)
}
c, err := newOO(cmd)
if err != nil {
return err
}
for _, uid := range args[1:] {
if _, err := c.AddProjectTeamUser(cmd.Context(), pid, uid); err != nil {
return fmt.Errorf("add %s: %w", uid, err)
}
printObject(map[string]any{"project_id": pid, "user_id": uid, "added": true})
}
return nil
},
}
}
func prjTeamRemoveCmd() *cobra.Command {
return &cobra.Command{
Use: "remove PROJECT_ID USER_ID [USER_ID...]",
Short: "Remove portal user(s) from the project team",
Args: cobra.MinimumNArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
pid, err := strconv.Atoi(args[0])
if err != nil {
return fmt.Errorf("project id: %w", err)
}
c, err := newOO(cmd)
if err != nil {
return err
}
for _, uid := range args[1:] {
if _, err := c.RemoveProjectTeamUser(cmd.Context(), pid, uid); err != nil {
return fmt.Errorf("remove %s: %w", uid, err)
}
printObject(map[string]any{"project_id": pid, "user_id": uid, "removed": true})
}
return nil
},
}
}
func prjTeamSetCmd() *cobra.Command {
var notify bool
cmd := &cobra.Command{
Use: "set PROJECT_ID USER_ID [USER_ID...]",
Short: "Replace the project team with the given users (register several at once)",
Args: cobra.MinimumNArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
pid, err := strconv.Atoi(args[0])
if err != nil {
return fmt.Errorf("project id: %w", err)
}
c, err := newOO(cmd)
if err != nil {
return err
}
team, err := c.SetProjectTeam(cmd.Context(), pid, args[1:], notify)
if err != nil {
return err
}
printTable([]string{"id", "displayName", "userName", "email", "isAdmin"}, teamRows(team))
return nil
},
}
cmd.Flags().BoolVar(&notify, "notify", false, "notify added members")
return cmd
}
// teamRows maps raw team member maps into table rows.
func teamRows(list []map[string]any) []map[string]any {
rows := make([]map[string]any, 0, len(list))
for _, m := range list {
rows = append(rows, map[string]any{
"id": idString(m, "id"),
"displayName": m["displayName"],
"userName": m["userName"],
"email": m["email"],
"isAdmin": m["isAdmin"],
})
}
return rows
}
func prjListCmd() *cobra.Command {
@@ -166,6 +298,32 @@ func prjMilestoneCreateCmd() *cobra.Command {
return cmd
}
func prjMilestoneDeleteCmd() *cobra.Command {
return &cobra.Command{
Use: "milestone-delete MILESTONE_ID [MILESTONE_ID...]",
Aliases: []string{"milestone-rm"},
Short: "Delete project milestone(s) by id",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
for _, raw := range args {
id, err := strconv.ParseInt(raw, 10, 64)
if err != nil {
return fmt.Errorf("milestone id %q must be integer: %w", raw, err)
}
if err := c.DeleteMilestone(id); err != nil {
return fmt.Errorf("delete milestone %d: %w", id, err)
}
printObject(map[string]any{"milestone_id": id, "deleted": true})
}
return nil
},
}
}
func prjCreateCmd() *cobra.Command {
var desc, resp string
var country, company string
+54
View File
@@ -0,0 +1,54 @@
package main
import (
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/spf13/cobra"
)
func init() {
projectsCmd.AddCommand(prjBoardSyncCmd())
}
// prjBoardSyncCmd upserts project milestones/tasks from a board YAML.
func prjBoardSyncCmd() *cobra.Command {
var apply bool
cmd := &cobra.Command{
Use: "board-sync BOARD.yaml",
Short: "Upsert project milestones/tasks from a board YAML (dry-run by default)",
Long: `Reads a board YAML (projects → milestones → tasks) and creates only the
milestones/tasks that are missing, matching by exact title. Existing entries are
left untouched, so the same file can be re-applied safely.
Dry-run by default; pass --apply to write to OnlyOffice.`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
board, err := onlyoffice.LoadBoard(args[0])
if err != nil {
return err
}
c, err := newOO(cmd)
if err != nil {
return err
}
res, err := c.SyncBoard(cmd.Context(), board, apply)
if err != nil {
return err
}
mode := "dry-run"
if apply {
mode = "apply"
}
printObject(map[string]any{
"mode": mode,
"projects": len(board.Projects),
"created_milestones": res.CreatedMilestones,
"skipped_milestones": res.SkippedMilestones,
"created_tasks": res.CreatedTasks,
"skipped_tasks": res.SkippedTasks,
})
return nil
},
}
cmd.Flags().BoolVar(&apply, "apply", false, "write to OnlyOffice (default: dry-run)")
return cmd
}
+69 -3
View File
@@ -22,6 +22,8 @@ func projectFilesCmd() *cobra.Command {
}
cmd.AddCommand(prjFilesListCmd())
cmd.AddCommand(prjFilesUploadCmd())
cmd.AddCommand(prjFilesReplaceInCmd())
cmd.AddCommand(prjFilesUpdateCmd())
cmd.AddCommand(prjFilesDownloadCmd())
cmd.AddCommand(prjFilesRenameCmd())
cmd.AddCommand(prjFilesDeleteCmd())
@@ -111,9 +113,11 @@ func prjFilesUploadCmd() *cobra.Command {
var replace, allowDuplicate bool
cmd := &cobra.Command{
Use: "upload PROJECT_ID LOCAL_PATH [LOCAL_PATH...]",
Short: "Upload file(s) into the project's Documents folder (upsert by stem|ext)",
Long: `Default: replace an existing file with the same logical name (stem|ext), like cp overwrite.
Pass --no-replace to fail when the name is taken; --allow-duplicate to always create a new file id.`,
Short: "Upload file(s) into the project's Documents folder (upsert by stem, legacy↔OOXML ext)",
Long: `Default: replace an existing file with the same logical name (stem, treating
legacy .xls/.doc/.ppt and their OOXML .xlsx/.docx/.pptx as one file), like cp
overwrite. Pass --no-replace to fail when the name is taken; --allow-duplicate
to always create a new file id.`,
Args: cobra.MinimumNArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
@@ -149,6 +153,68 @@ Pass --no-replace to fail when the name is taken; --allow-duplicate to always cr
return cmd
}
func prjFilesReplaceInCmd() *cobra.Command {
return &cobra.Command{
Use: "replace-in FOLDER_ID LOCAL_PATH [LOCAL_PATH...]",
Short: "Replace same-named file(s) in a folder: hard delete + fresh upload (no version history)",
Long: `Deletes any file in FOLDER_ID with the same stem|ext (hard delete — the CLI
delete is permanent) and uploads the local file fresh. Unlike 'update' this
leaves a single clean version.
Why it exists: repeated 'update' of a shared document accumulated a visible
version history and left a stale id. replace-in yields one clean revision; then
point links at the returned id (or keep an nginx alias for the legacy fileid).
Note: file ids are server-assigned; a fresh upload gets a new id.`,
Args: cobra.MinimumNArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
folderID := args[0]
for _, p := range args[1:] {
stem := onlyoffice.UploadStemFromLocal(p)
ext := onlyoffice.UploadExtFromLocal(p)
deleted, derr := c.DeleteFilesByDedupKey(cmd.Context(), folderID, stem, ext)
if derr != nil {
return derr
}
ent, uerr := c.UploadToFolder(cmd.Context(), folderID, p)
if uerr != nil {
return uerr
}
obj := fileEntryToMap(ent)
if len(deleted) > 0 {
obj["replaced_file_ids"] = deleted
}
printObject(obj)
}
return nil
},
}
}
func prjFilesUpdateCmd() *cobra.Command {
return &cobra.Command{
Use: "update FILE_ID LOCAL_PATH",
Short: "Overwrite an existing Documents file with new content (new version)",
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
entry, err := c.UpdateFile(cmd.Context(), args[0], args[1])
if err != nil {
return err
}
printObject(fileEntryToMap(entry))
return nil
},
}
}
func prjFilesDownloadCmd() *cobra.Command {
var to string
cmd := &cobra.Command{
+308
View File
@@ -1,6 +1,12 @@
package main
import (
"bufio"
"fmt"
"os"
"strings"
onlyoffice "github.com/eslider/go-onlyoffice"
"github.com/spf13/cobra"
)
@@ -13,6 +19,14 @@ func init() {
rootCmd.AddCommand(usersCmd)
usersCmd.AddCommand(usersListCmd())
usersCmd.AddCommand(usersSelfCmd())
usersCmd.AddCommand(usersGetCmd())
usersCmd.AddCommand(usersCreateCmd())
usersCmd.AddCommand(usersUpdateCmd())
usersCmd.AddCommand(usersDeleteCmd())
usersCmd.AddCommand(usersBlockCmd())
usersCmd.AddCommand(usersUnblockCmd())
usersCmd.AddCommand(usersPasswordCmd())
usersCmd.AddCommand(usersCheckCmd())
rootCmd.AddCommand(whoamiCmd())
}
@@ -65,6 +79,300 @@ func usersSelfCmd() *cobra.Command {
}
}
func usersGetCmd() *cobra.Command {
return &cobra.Command{
Use: "get USER_ID",
Short: "Show one portal user profile",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
u, err := c.GetUser(cmd.Context(), args[0])
if err != nil {
return err
}
printObject(u)
return nil
},
}
}
func usersCreateCmd() *cobra.Command {
var first, last, email, password, title, location, sex, comment string
var visitor bool
cmd := &cobra.Command{
Use: "create",
Short: "Create a portal user",
Long: `Create a portal user (POST /api/2.0/people).
Without --password the portal generates one and the account stays NotActivated
until the user follows the activation link. With --password the account is
Active immediately. Use --visitor for a guest account.`,
RunE: func(cmd *cobra.Command, args []string) error {
if email == "" || first == "" || last == "" {
return fmt.Errorf("--email, --first and --last are required")
}
c, err := newOO(cmd)
if err != nil {
return err
}
req := onlyoffice.NewUserRequest{
FirstName: first,
LastName: last,
Email: email,
Password: password,
Title: title,
Location: location,
Sex: sex,
Comment: comment,
}
if cmd.Flags().Changed("visitor") {
req.IsVisitor = &visitor
}
u, err := c.CreateUser(cmd.Context(), req)
if err != nil {
return err
}
printObject(map[string]any{
"id": idString(u, "id"),
"displayName": idString(u, "displayName"),
"email": idString(u, "email"),
"status": u["status"],
})
return nil
},
}
cmd.Flags().StringVar(&first, "first", "", "first name (required)")
cmd.Flags().StringVar(&last, "last", "", "last name (required)")
cmd.Flags().StringVar(&email, "email", "", "email (required)")
cmd.Flags().StringVar(&password, "password", "", "initial password (default: portal-generated)")
cmd.Flags().StringVar(&title, "title", "", "job title")
cmd.Flags().StringVar(&location, "location", "", "location")
cmd.Flags().StringVar(&sex, "sex", "", "sex: male|female")
cmd.Flags().StringVar(&comment, "comment", "", "comment")
cmd.Flags().BoolVar(&visitor, "visitor", false, "create as guest (isVisitor=true)")
return cmd
}
func usersUpdateCmd() *cobra.Command {
var first, last, email, title, location, sex, comment string
cmd := &cobra.Command{
Use: "update USER_ID",
Short: "Update portal user profile fields (only flags passed)",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
body := map[string]any{}
if cmd.Flags().Changed("first") {
body["firstname"] = first
}
if cmd.Flags().Changed("last") {
body["lastname"] = last
}
if cmd.Flags().Changed("email") {
body["email"] = email
}
if cmd.Flags().Changed("title") {
body["title"] = title
}
if cmd.Flags().Changed("location") {
body["location"] = location
}
if cmd.Flags().Changed("sex") {
body["sex"] = sex
}
if cmd.Flags().Changed("comment") {
body["comment"] = comment
}
if len(body) == 0 {
return fmt.Errorf("nothing to update: pass at least one of --first/--last/--email/--title/--location/--sex/--comment")
}
c, err := newOO(cmd)
if err != nil {
return err
}
u, err := c.UpdateUser(cmd.Context(), args[0], body)
if err != nil {
return err
}
printObject(map[string]any{
"id": idString(u, "id"),
"displayName": idString(u, "displayName"),
})
return nil
},
}
cmd.Flags().StringVar(&first, "first", "", "first name")
cmd.Flags().StringVar(&last, "last", "", "last name")
cmd.Flags().StringVar(&email, "email", "", "email")
cmd.Flags().StringVar(&title, "title", "", "job title")
cmd.Flags().StringVar(&location, "location", "", "location")
cmd.Flags().StringVar(&sex, "sex", "", "sex: male|female")
cmd.Flags().StringVar(&comment, "comment", "", "comment")
return cmd
}
func usersDeleteCmd() *cobra.Command {
return &cobra.Command{
Use: "delete USER_ID [USER_ID...]",
Aliases: []string{"rm"},
Short: "Delete portal user(s) permanently",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
for _, id := range args {
u, err := c.DeleteUser(cmd.Context(), id)
if err != nil {
return fmt.Errorf("delete %s: %w", id, err)
}
printObject(map[string]any{"id": id, "deleted": true, "displayName": idString(u, "displayName")})
}
return nil
},
}
}
func usersBlockCmd() *cobra.Command {
return &cobra.Command{
Use: "block USER_ID [USER_ID...]",
Aliases: []string{"disable"},
Short: "Block (terminate) user(s): login denied, profile kept",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
for _, id := range args {
if err := c.BlockUser(cmd.Context(), id); err != nil {
return fmt.Errorf("block %s: %w", id, err)
}
printObject(map[string]any{"id": id, "blocked": true})
}
return nil
},
}
}
func usersUnblockCmd() *cobra.Command {
return &cobra.Command{
Use: "unblock USER_ID [USER_ID...]",
Aliases: []string{"enable", "activate"},
Short: "Unblock (activate) user(s)",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newOO(cmd)
if err != nil {
return err
}
for _, id := range args {
if err := c.UnblockUser(cmd.Context(), id); err != nil {
return fmt.Errorf("unblock %s: %w", id, err)
}
printObject(map[string]any{"id": id, "unblocked": true})
}
return nil
},
}
}
func usersPasswordCmd() *cobra.Command {
var password string
cmd := &cobra.Command{
Use: "password USER_ID",
Short: "Set a user password (reads stdin when --password is empty)",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
pwd := password
if pwd == "" {
b, err := readLine(os.Stdin)
if err != nil {
return fmt.Errorf("read password: %w", err)
}
pwd = b
}
if pwd == "" {
return fmt.Errorf("password is empty")
}
c, err := newOO(cmd)
if err != nil {
return err
}
if err := c.ChangeUserPassword(cmd.Context(), args[0], pwd); err != nil {
return err
}
printObject(map[string]any{"id": args[0], "password_changed": true})
return nil
},
}
cmd.Flags().StringVar(&password, "password", "", "new password (omit to read one line from stdin)")
return cmd
}
func usersCheckCmd() *cobra.Command {
var login, password string
cmd := &cobra.Command{
Use: "check",
Short: "Check that a login can authenticate (userName or email)",
Long: `Probes POST /api/2.0/authentication.json with the given credentials and
discards the token.
Where this is used: before handing portal credentials to an external party
(e.g. a guest given read access to a document pack), verify the login actually
works. On some portals the account email is the reliable login identifier — the
userName login fails with 500 for a freshly created user — so share the email,
not the userName.`,
RunE: func(cmd *cobra.Command, args []string) error {
if login == "" {
return fmt.Errorf("--login is required (userName or email)")
}
if password == "" {
if b, err := readLine(os.Stdin); err == nil {
password = b
}
}
c, err := newOO(cmd)
if err != nil {
return err
}
if err := c.AuthenticateAs(cmd.Context(), login, password); err != nil {
printObject(map[string]any{"login": login, "ok": false, "error": trimAuthErr(err)})
return fmt.Errorf("login failed for %s", login)
}
printObject(map[string]any{"login": login, "ok": true})
return nil
},
}
cmd.Flags().StringVar(&login, "login", "", "userName or email")
cmd.Flags().StringVar(&password, "password", "", "password (omit to read one line from stdin)")
return cmd
}
// trimAuthErr keeps the error short for table output.
func trimAuthErr(err error) string {
s := err.Error()
if len(s) > 160 {
s = s[:160] + "…"
}
return s
}
// readLine reads a single trimmed line from r.
func readLine(r *os.File) (string, error) {
sc := bufio.NewScanner(r)
if !sc.Scan() {
if err := sc.Err(); err != nil {
return "", err
}
return "", nil
}
return strings.TrimSpace(sc.Text()), nil
}
// whoamiCmd is a convenience shortcut at the root level.
func whoamiCmd() *cobra.Command {
return &cobra.Command{
-50
View File
@@ -1,50 +0,0 @@
// Command ooscan recursively lists OnlyOffice Documents folders into a TSV
// index: file_id, folder_id, path, title.
//
// Usage: ooscan <FOLDER_ID> [<FOLDER_ID>...]
package main
import (
"context"
"fmt"
"os"
"time"
onlyoffice "github.com/eslider/go-onlyoffice"
)
func main() {
ctx := context.Background()
c := onlyoffice.NewClient(onlyoffice.GetEnvironmentCredentials())
seen := map[string]bool{}
for _, root := range os.Args[1:] {
walk(ctx, c, root, "", 0, seen)
}
}
func walk(ctx context.Context, c *onlyoffice.Client, folderID, path string, depth int, seen map[string]bool) {
if depth > 8 || seen[folderID] {
return
}
seen[folderID] = true
// Throttle: OnlyOffice rate-limits (429) and the host must not be flooded.
time.Sleep(350 * time.Millisecond)
ctx, cancel := context.WithTimeout(ctx, 60*time.Second)
defer cancel()
var l *onlyoffice.DavListing
derr := onlyoffice.DoRetry(ctx, onlyoffice.DefaultRetryPolicy(), func() error {
var err error
l, err = c.ListDavFolder(ctx, folderID)
return err
})
if derr != nil {
fmt.Fprintf(os.Stderr, "list %s (%s): %v\n", path, folderID, derr)
return
}
for _, f := range l.Files {
fmt.Printf("%s\t%s\t%s\t%s\n", f.ID, folderID, path, f.Title)
}
for _, sub := range l.Folders {
walk(ctx, c, sub.ID, path+"/"+sub.Title, depth+1, seen)
}
}
-308
View File
@@ -1,308 +0,0 @@
// Command pdfamount walks a Documents folder, downloads matching PDFs and
// extracts the payable amount, printing "file_id\ttitle\tamount".
//
// Usage: pdfamount <FOLDER_ID> [TITLE_FILTER_REGEX]
package main
import (
"bytes"
"context"
"fmt"
"os"
"os/exec"
"regexp"
"strconv"
"strings"
"time"
onlyoffice "github.com/eslider/go-onlyoffice"
)
// amountPat is the amount capture shared by every amount regex.
const amountPat = `([0-9]+(?:[.,][0-9]+)*)`
// amountRE builds "<label> [optional (comment)] [: -] <number>".
func amountRE(label string) *regexp.Regexp {
return regexp.MustCompile(
`(?i)\b` + regexp.QuoteMeta(label) + `\b\s*(?:\([^)]*\))?\s*[:\-]?\s*` + amountPat)
}
// amountRes lists the payable-amount patterns in strict priority order: the
// first pattern with a usable amount wins, and a lower-priority label can never
// override a higher-priority one ("zu zahlender betrag" > "rechnungsbetrag" >
// "rechnungsendbetrag" > "gesamtbetrag" > "gesamtsumme (inkl. steuern)").
//
// "gesamtbetrag" and "gesamtsumme" are not in the original set but are the real
// labels on Diashop invoices ("Gesamtsumme (inkl. Steuern)"). The inclusive
// variant is matched before a plain "gesamtsumme". Everything after those
// primary labels is the broader fallback set, consulted only when no primary
// label yields an amount. Within one pattern the last usable amount is taken,
// because totals usually come last.
var amountRes = []*regexp.Regexp{
amountRE("zu zahlender betrag"),
amountRE("rechnungsbetrag"),
amountRE("rechnungsendbetrag"),
amountRE("gesamtbetrag"),
regexp.MustCompile(`(?i)\bgesamtsumme\b\s*\(\s*inkl\.?\s*steuern\s*\)\s*[:\-]?\s*` + amountPat),
amountRE("gesamtsumme"),
amountRE("endbetrag"),
amountRE("zahlbetrag"),
amountRE("bruttobetrag"),
amountRE("betrag"),
amountRE("total"),
amountRE("summe"),
}
// taxLineRe marks a line whose number is a tax rate/percentage: an explicit
// percent sign or a VAT/tax keyword. "Steuern" (plural, as in "inkl. Steuern")
// is handled separately so the inclusive total stays usable.
var taxLineRe = regexp.MustCompile(`(?i)%|\bMwSt\b|\bUSt\b|\bProzent\b`)
// steuerRe finds "Steuer"/"Umsatzsteuer" etc. RE2 has no lookahead, so the
// plural "Steuern" is excluded in isTaxLine.
var steuerRe = regexp.MustCompile(`(?i)steuer`)
// percentAfterRe detects a percent sign directly after a number (spaces ok).
var percentAfterRe = regexp.MustCompile(`^\s*%`)
func main() {
if len(os.Args) < 2 {
fmt.Fprintln(os.Stderr, "usage: pdfamount <FOLDER_ID> [TITLE_FILTER_REGEX]")
os.Exit(2)
}
folder := os.Args[1]
filter := regexp.MustCompile(`(?i)rechnung`)
if len(os.Args) >= 3 {
filter = regexp.MustCompile(os.Args[2])
}
ctx := context.Background()
c := onlyoffice.NewClient(onlyoffice.GetEnvironmentCredentials())
files := listAll(ctx, c, folder)
for _, f := range files {
if !filter.MatchString(f.title) {
continue
}
if !strings.HasSuffix(strings.ToLower(f.title), ".pdf") {
continue
}
amount, err := pdfAmount(ctx, c, f.id)
if err != nil {
fmt.Fprintf(os.Stderr, "%s: %v\n", f.title, err)
continue
}
if amount == "" {
continue
}
fmt.Printf("%s\t%s\t%s\n", f.id, f.title, amount)
}
}
type file struct{ id, title string }
func listAll(ctx context.Context, c *onlyoffice.Client, folder string) []file {
seen := map[string]bool{}
var out []file
var walk func(string)
walk = func(id string) {
if seen[id] {
return
}
seen[id] = true
time.Sleep(300 * time.Millisecond)
l, err := c.ListDavFolder(ctx, id)
if err != nil {
fmt.Fprintf(os.Stderr, "list %s: %v\n", id, err)
return
}
for _, f := range l.Files {
out = append(out, file{f.ID, f.Title})
}
for _, sub := range l.Folders {
walk(sub.ID)
}
}
walk(folder)
return out
}
func pdfAmount(ctx context.Context, c *onlyoffice.Client, id string) (string, error) {
time.Sleep(time.Second)
tmp, err := os.CreateTemp("", "pdf-*.pdf")
if err != nil {
return "", err
}
defer os.Remove(tmp.Name())
derr := onlyoffice.DoRetry(ctx, onlyoffice.DefaultRetryPolicy(), func() error {
_ = tmp.Truncate(0)
_, _ = tmp.Seek(0, 0)
_, err := c.DownloadFile(ctx, id, tmp)
return err
})
if derr != nil {
tmp.Close()
return "", derr
}
tmp.Close()
var buf bytes.Buffer
cmd := exec.CommandContext(ctx, "pdftotext", "-layout", tmp.Name(), "-")
cmd.Stdout = &buf
if err := cmd.Run(); err != nil {
return "", err
}
return extractAmount(buf.String()), nil
}
// extractAmount returns the normalised ("1234.56") payable amount found in
// text, or "" if no usable amount matches.
//
// DKV invoices are special-cased first: they repeat a per-vehicle "TOTAL:" line
// and carry the real total only in the "Gesamtsummenaufstellung" section.
func extractAmount(text string) string {
if v, ok := dkvGrandTotal(text); ok {
return v
}
for _, re := range amountRes {
if v, ok := lastUsableAmount(text, re); ok {
return v
}
}
return ""
}
// dkvGrandTotal extracts the total of a DKV "Gesamtsummenaufstellung" section.
//
// Rule: DKV invoices repeat a per-vehicle "TOTAL:" line, so the last TOTAL is
// not the invoice total. When a "Gesamtsummenaufstellung" section exists, its
// total wins over every "TOTAL:" line: the first amount after the "»" marker,
// or, if there is none, the last amount in the section. The section ends at the
// page break (form feed) or end of text.
func dkvGrandTotal(text string) (string, bool) {
idx := strings.Index(strings.ToLower(text), "gesamtsummenaufstellung")
if idx < 0 {
return "", false
}
section := text[idx:]
if ff := strings.IndexByte(section, '\f'); ff >= 0 {
section = section[:ff]
}
if m := strings.Index(section, "»"); m >= 0 {
if v, ok := firstAmount(section[m:]); ok {
return v, true
}
}
return lastAmount(section)
}
// lastUsableAmount returns the last amount matched by re that is not a tax rate
// or percentage. Within one label the last usable amount wins.
func lastUsableAmount(text string, re *regexp.Regexp) (string, bool) {
ms := re.FindAllStringSubmatchIndex(text, -1)
for i := len(ms) - 1; i >= 0; i-- {
m := ms[i]
if isTaxRate(text, m[2], m[3]) {
continue
}
if v, ok := normalizeAmount(text[m[2]:m[3]]); ok {
return v, true
}
}
return "", false
}
// isTaxRate reports whether the number at text[start:end] is a tax rate or a
// percentage instead of a payable amount. A candidate is rejected when the
// token right after the number is "%" or the number's line carries a percent
// sign or a tax keyword. Rejecting is deliberate: office matching treats a
// known-but-different amount as a hard disqualifier, so an empty result is
// safer than the VAT rate.
func isTaxRate(text string, start, end int) bool {
if percentAfterRe.MatchString(text[end:]) {
return true
}
lineStart := strings.LastIndexByte(text[:start], '\n') + 1
line := text[lineStart:]
if n := strings.IndexByte(text[end:], '\n'); n >= 0 {
line = text[lineStart : end+n]
}
return isTaxLine(line)
}
// isTaxLine reports whether a line looks like a tax rate rather than a payable
// amount. "Steuern" is treated as a qualifier ("inkl. Steuern"), not a rate.
func isTaxLine(line string) bool {
if taxLineRe.MatchString(line) {
return true
}
for _, loc := range steuerRe.FindAllStringIndex(line, -1) {
if loc[1] >= len(line) || (line[loc[1]] != 'n' && line[loc[1]] != 'N') {
return true
}
}
return false
}
// numberRe finds bare numbers (with optional thousands/decimal separators).
var numberRe = regexp.MustCompile(`[0-9]+(?:[.,][0-9]+)*`)
func firstAmount(s string) (string, bool) {
for _, m := range numberRe.FindAllString(s, -1) {
if v, ok := normalizeAmount(m); ok {
return v, true
}
}
return "", false
}
func lastAmount(s string) (string, bool) {
ms := numberRe.FindAllString(s, -1)
for i := len(ms) - 1; i >= 0; i-- {
if v, ok := normalizeAmount(ms[i]); ok {
return v, true
}
}
return "", false
}
// normalizeAmount turns "1.234,56" (DE), "1,234.56" (EN) or "1234.56" into
// "1234.56". The rightmost separator is decimal only when followed by one or
// two digits; otherwise every separator is a thousands separator.
func normalizeAmount(s string) (string, bool) {
last := -1
for i := 0; i < len(s); i++ {
if s[i] == '.' || s[i] == ',' {
last = i
}
}
var dec byte
if last >= 0 {
digits := 0
for i := last + 1; i < len(s); i++ {
if s[i] < '0' || s[i] > '9' {
return "", false
}
digits++
}
if digits == 1 || digits == 2 {
dec = s[last]
}
}
var b strings.Builder
for i := 0; i < len(s); i++ {
switch c := s[i]; {
case c >= '0' && c <= '9':
b.WriteByte(c)
case (c == '.' || c == ',') && c == dec:
b.WriteByte('.')
case c == '.' || c == ',':
// thousands separator
default:
return "", false
}
}
v, err := strconv.ParseFloat(b.String(), 64)
if err != nil {
return "", false
}
return strconv.FormatFloat(v, 'f', 2, 64), true
}
-175
View File
@@ -1,175 +0,0 @@
package main
import "testing"
func TestExtractAmount(t *testing.T) {
tests := []struct {
name, text, want string
}{
{
name: "rechnungsbetrag de format",
text: "Rechnungsbetrag: 1.234,56 €",
want: "1234.56",
},
{
name: "rechnungsbetrag en thousands and dot",
text: "Rechnungsbetrag: 1,234.56",
want: "1234.56",
},
{
name: "rechnungsbetrag plain dot",
text: "Rechnungsbetrag: 1234.56",
want: "1234.56",
},
{
name: "rechnungsbetrag de comma only",
text: "Rechnungsbetrag: 1234,56",
want: "1234.56",
},
{
name: "currency suffix eur",
text: "Rechnungsbetrag: 1.234,56 EUR",
want: "1234.56",
},
{
name: "zu zahlender betrag wins over rechnungsbetrag",
text: "Zu zahlender Betrag: 10,00\nRechnungsbetrag: 99,00",
want: "10.00",
},
{
name: "rechnungsbetrag wins over endbetrag",
text: "Endbetrag: 20,00\nRechnungsbetrag: 30,00",
want: "30.00",
},
{
name: "bruttobetrag wins over bare betrag",
text: "Bruttobetrag: 50,00\nBetrag: 10,00",
want: "50.00",
},
{
name: "gesamtbetrag wins over bare betrag",
text: "Gesamtbetrag: 80,00\nBetrag: 10,00",
want: "80.00",
},
{
name: "last occurrence of same label wins",
text: "Rechnungsbetrag: 10,00\nRechnungsbetrag: 20,00",
want: "20.00",
},
{
name: "endbetrag fallback",
text: "Endbetrag: 42,00",
want: "42.00",
},
{
name: "zahlbetrag fallback without colon",
text: "Zahlbetrag 7,50 €",
want: "7.50",
},
{
name: "rechnungsendbetrag beats endbetrag",
text: "Rechnungsendbetrag: 12,00\nEndbetrag: 13,00",
want: "12.00",
},
{
name: "dkv style total line",
text: "Kundenbezogene Daten\n» TOTAL: 123,45 100,00 23,45 123,45\n",
want: "123.45",
},
{
name: "dkv gesamtsummenaufstellung grand total after marker",
text: "» TOTAL: 111,11 100,00 11,11 111,11\n" +
"» TOTAL: 222,22 200,00 22,22 222,22\n" +
"Gesamtsummenaufstellung\n" +
"Netto 240,00\n" +
"MwSt 47,25\n" +
"» 287,25\n",
want: "287.25",
},
{
name: "dkv gesamtsummenaufstellung total on next line",
text: "» TOTAL: 111,11\nGesamtsummenaufstellung\n»\n287,25\n",
want: "287.25",
},
{
name: "tax rate with percent sign is not an amount",
text: "Betrag: 19,00 % MwSt",
want: "",
},
{
name: "mehrwertsteuer rate is not an amount",
text: "Gesamtsumme: 19,00% MwSt",
want: "",
},
{
name: "steuer word on the number line rejects it",
text: "Betrag: 2,83 Steuer",
want: "",
},
{
name: "rejected primary falls back to a usable label",
text: "Gesamtsumme: 19,00 % MwSt\nEndbetrag: 42,00",
want: "42.00",
},
{
name: "labeled zu zahlender betrag beats unlabeled larger number",
text: "unlabeled 999,99\nZu zahlender Betrag: 10,00",
want: "10.00",
},
{
name: "labeled zu zahlender betrag beats lower label larger number",
text: "Endbetrag: 999,99\nZu zahlender Betrag: 10,00",
want: "10.00",
},
{
name: "diashop style gesamtsumme with comment",
text: "Zwischensumme\n12,34 €\nZwischensumme\n12,34 €\nVersand & Bearbeitung\n4,95 €\nGesamtsumme (inkl. Steuern)\n17,29 €\n",
want: "17.29",
},
{
name: "diashop picks inclusive total last",
text: "Gesamtsumme (exkl. Steuern)\n12,34 €\nGesamtsumme (inkl. Steuern)\n17,29 €",
want: "17.29",
},
{
name: "no label",
text: "some text without any amount label 12,34",
want: "",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
if got := extractAmount(tc.text); got != tc.want {
t.Fatalf("extractAmount()=%q want %q", got, tc.want)
}
})
}
}
func TestNormalizeAmount(t *testing.T) {
tests := []struct {
in string
want string
ok bool
}{
{"1.234,56", "1234.56", true},
{"1,234.56", "1234.56", true},
{"1234.56", "1234.56", true},
{"1234,56", "1234.56", true},
{"1.234.567,89", "1234567.89", true},
{"1,234,567.89", "1234567.89", true},
{"1.234", "1234.00", true},
{"12,5", "12.50", true},
{"12", "12.00", true},
{"", "0.00", false},
}
for _, tc := range tests {
got, ok := normalizeAmount(tc.in)
if ok != tc.ok {
t.Fatalf("normalizeAmount(%q) ok=%v want %v", tc.in, ok, tc.ok)
}
if ok && got != tc.want {
t.Fatalf("normalizeAmount(%q)=%q want %q", tc.in, got, tc.want)
}
}
}
+170
View File
@@ -0,0 +1,170 @@
package onlyoffice
// Document conversion via the OnlyOffice DocumentServer converter.
//
// The DocumentServer (the same engine behind the portal's "Download as PDF")
// converts any office format. From a portal-reachable host the converter is
// exposed at "<portal>/ds-vpath/converter" (reverse proxy) or directly at
// "http://<docs-server>:8083/converter" (legacy path: /ConvertService.ashx).
//
// Flow: PresignedURI(fileId) → Convert(docsBase, secret, req) → download
// result.FileURL. The JWT is HS256 signed with the DocumentServer's
// services.CoAuthoring.secret (NOT storage.fs.secretString).
import (
"bytes"
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"strings"
)
// PresignedURI returns a short-lived, fetchable download URI for a portal file
// (GET /api/2.0/files/file/{fileId}/presigneduri). The DocumentServer can fetch
// it without the caller's session, so it is the input for Convert.
func (c *Client) PresignedURI(ctx context.Context, fileID string) (string, error) {
if fileID == "" {
return "", fmt.Errorf("file id is required")
}
raw, err := c.getJSON(ctx, fmt.Sprintf("/api/2.0/files/file/%s/presigneduri", url.PathEscape(fileID)))
if err != nil {
return "", err
}
resp, err := responseField(raw, "response")
if err != nil {
return "", err
}
var s string
if err := json.Unmarshal(resp, &s); err == nil && s != "" {
return s, nil
}
// Some builds return an object instead of a bare string.
var o map[string]any
if err := json.Unmarshal(resp, &o); err == nil {
for _, k := range []string{"uri", "url", "Uri", "Url"} {
if v, ok := o[k].(string); ok && v != "" {
return v, nil
}
}
}
return "", fmt.Errorf("presigneduri: unexpected response %s", truncate(string(resp), 200))
}
// ConvertRequest is the DocumentServer converter body.
type ConvertRequest struct {
URL string `json:"url"`
OutputType string `json:"outputtype"`
FileType string `json:"filetype,omitempty"`
Key string `json:"key"`
Title string `json:"title,omitempty"`
}
// ConvertResult is the DocumentServer converter reply.
type ConvertResult struct {
FileURL string `json:"fileUrl"`
FileType string `json:"fileType"`
Percent int `json:"percent"`
EndConvert bool `json:"endConvert"`
Error *int `json:"error,omitempty"`
}
// SignJWT builds an HS256 JWT with the given payload (stdlib only).
func SignJWT(secret string, payload any) (string, error) {
if secret == "" {
return "", fmt.Errorf("jwt secret is empty")
}
hb, err := json.Marshal(map[string]string{"alg": "HS256", "typ": "JWT"})
if err != nil {
return "", err
}
pb, err := json.Marshal(payload)
if err != nil {
return "", err
}
enc := base64.RawURLEncoding.EncodeToString
signing := enc(hb) + "." + enc(pb)
mac := hmac.New(sha256.New, []byte(secret))
mac.Write([]byte(signing))
return signing + "." + enc(mac.Sum(nil)), nil
}
// ConvertDocument asks a DocumentServer to convert req.URL into req.OutputType.
// docsBase is e.g. "https://portal/ds-vpath" or "http://localhost:8083";
// secret is the DocumentServer CoAuthoring JWT secret. Passes the JWT both as
// the AuthorizationJwt header and as a body token.
func (c *Client) ConvertDocument(ctx context.Context, docsBase, secret string, req ConvertRequest) (*ConvertResult, error) {
if strings.TrimSpace(docsBase) == "" {
return nil, fmt.Errorf("docs base url is required")
}
if req.URL == "" {
return nil, fmt.Errorf("source url is required")
}
if req.OutputType == "" {
return nil, fmt.Errorf("outputtype is required")
}
if req.Key == "" {
return nil, fmt.Errorf("conversion key is required")
}
jwt, err := SignJWT(secret, req)
if err != nil {
return nil, err
}
body, err := json.Marshal(req)
if err != nil {
return nil, err
}
endpoint := strings.TrimRight(docsBase, "/") + "/converter"
httpReq, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(body))
if err != nil {
return nil, err
}
httpReq.Header.Set("Content-Type", "application/json")
httpReq.Header.Set("Accept", "application/json")
httpReq.Header.Set("AuthorizationJwt", "Bearer "+jwt)
resp, err := c.client.Do(httpReq)
if err != nil {
return nil, fmt.Errorf("converter request: %w", err)
}
defer resp.Body.Close()
raw, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
if resp.StatusCode >= 400 {
return nil, fmt.Errorf("converter: %d %s", resp.StatusCode, truncate(string(raw), 300))
}
var out ConvertResult
if err := json.Unmarshal(raw, &out); err != nil {
return nil, fmt.Errorf("converter decode: %w (%s)", err, truncate(string(raw), 200))
}
if out.Error != nil {
return &out, fmt.Errorf("converter error %d", *out.Error)
}
if out.FileURL == "" {
return &out, fmt.Errorf("converter returned no fileUrl")
}
return &out, nil
}
// DownloadURLTo streams an absolute URL (no portal auth) into dst.
func (c *Client) DownloadURLTo(ctx context.Context, rawurl string, dst io.Writer) (int64, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, rawurl, nil)
if err != nil {
return 0, err
}
resp, err := c.client.Do(req)
if err != nil {
return 0, err
}
defer resp.Body.Close()
if resp.StatusCode >= 400 {
return 0, fmt.Errorf("download: %d", resp.StatusCode)
}
return io.Copy(dst, resp.Body)
}
+24
View File
@@ -0,0 +1,24 @@
package onlyoffice
import (
"strings"
"testing"
)
func TestSignJWT(t *testing.T) {
payload := map[string]any{"url": "u", "outputtype": "pdf"}
tok, err := SignJWT("secret", payload)
if err != nil {
t.Fatal(err)
}
if n := len(strings.Split(tok, ".")); n != 3 {
t.Fatalf("JWT must have 3 parts, got %d", n)
}
tok2, _ := SignJWT("secret", payload)
if tok != tok2 {
t.Fatal("SignJWT must be deterministic for identical input")
}
if _, err := SignJWT("", payload); err == nil {
t.Fatal("expected error for empty secret")
}
}
+91
View File
@@ -0,0 +1,91 @@
package onlyoffice
import (
"fmt"
"strconv"
"strings"
)
// addressCategoryCodes maps ASC.CRM.Core.AddressCategory names to their numeric
// codes. The OO API expects the code, the UI/docs use the label.
var addressCategoryCodes = map[string]int{
"home": 0,
"postal": 1,
"office": 2,
"billing": 3,
"other": 4,
"work": 5,
}
// AddressCategoryCode returns the numeric code for an AddressCategory label
// (Home|Postal|Office|Billing|Other|Work) or a numeric string. Unknown/empty
// labels fall back to Billing, the category `oo companies create` used.
func AddressCategoryCode(category string) int {
s := strings.ToLower(strings.TrimSpace(category))
if s == "" {
return addressCategoryCodes["billing"]
}
if n, err := strconv.Atoi(s); err == nil {
if n >= 0 && n <= 5 {
return n
}
return addressCategoryCodes["billing"]
}
if n, ok := addressCategoryCodes[s]; ok {
return n
}
return addressCategoryCodes["billing"]
}
// ContactAddresses returns the postal address rows of a contact map.
func ContactAddresses(contact map[string]any) []map[string]any {
if rows, ok := contact["addresses"].([]any); ok {
return mapsFromAnySlice(rows)
}
if rows, ok := contact["addresses"].([]map[string]any); ok {
return rows
}
return nil
}
// HasContactAddress reports whether a contact already has the given postal
// address. street+city+zip+category identify it; comparison is normalized.
func HasContactAddress(contact map[string]any, street, city, zip, category string) bool {
wantStreet, wantCity, wantZip := normalizeAddressPart(street), normalizeAddressPart(city), normalizeAddressPart(zip)
wantCat := AddressCategoryCode(category)
for _, row := range ContactAddresses(contact) {
if normalizeAddressPart(fmt.Sprint(row["street"])) != wantStreet {
continue
}
if normalizeAddressPart(fmt.Sprint(row["city"])) != wantCity {
continue
}
if normalizeAddressPart(fmt.Sprint(row["zip"])) != wantZip {
continue
}
if int(anyToFloat(row["category"])) != wantCat {
continue
}
return true
}
return false
}
func normalizeAddressPart(s string) string {
s = strings.ToLower(strings.TrimSpace(s))
return strings.Join(strings.Fields(s), " ")
}
func anyToFloat(v any) float64 {
switch n := v.(type) {
case float64:
return n
case int:
return float64(n)
case string:
f, _ := strconv.ParseFloat(strings.TrimSpace(n), 64)
return f
default:
return 0
}
}
+42
View File
@@ -0,0 +1,42 @@
package onlyoffice
import "testing"
func TestAddressCategoryCode(t *testing.T) {
cases := map[string]int{
"Home": 0, "Postal": 1, "Office": 2, "Billing": 3, "Other": 4, "Work": 5,
"billing": 3, " work ": 5, "3": 3, "5": 5,
"": 3, "nonsense": 3, "99": 3,
}
for in, want := range cases {
if got := AddressCategoryCode(in); got != want {
t.Errorf("AddressCategoryCode(%q) = %d, want %d", in, got, want)
}
}
}
func TestHasContactAddress(t *testing.T) {
contact := map[string]any{
"addresses": []any{
map[string]any{
"street": "Lubanas st. 125a-25", "city": "Riga",
"zip": "LV-1021", "country": "Latvia", "category": float64(3),
},
},
}
if !HasContactAddress(contact, " Lubanas St. 125a-25 ", "riga", "lv-1021", "Billing") {
t.Error("want match (normalized, case-insensitive)")
}
if HasContactAddress(contact, "Lubanas st. 125a-25", "Riga", "LV-1021", "Work") {
t.Error("different category must not match")
}
if HasContactAddress(contact, "Lubanas st. 125a-25", "Riga", "00000", "Billing") {
t.Error("different zip must not match")
}
if HasContactAddress(map[string]any{}, "x", "y", "z", "Billing") {
t.Error("empty contact must not match")
}
if got := ContactAddresses(contact); len(got) != 1 {
t.Fatalf("ContactAddresses = %d rows", len(got))
}
}
+126
View File
@@ -0,0 +1,126 @@
package onlyoffice
import (
"context"
"fmt"
"strconv"
"strings"
)
// OpportunityAudit is one CRM opportunity with its resource counts and a coarse
// class, for hygiene reporting (see `oo crm audit`).
type OpportunityAudit struct {
ID int64 `json:"id"`
Title string `json:"title"`
Created string `json:"created,omitempty"`
Files int `json:"files"`
OpenTasks int `json:"open_tasks"`
ClosedTasks int `json:"closed_tasks"`
Members int `json:"members"`
GroupKey string `json:"group_key,omitempty"`
Class string `json:"class"` // ok | dup | empty | junk-title
}
// AuditOpportunities lists every opportunity with file/task/member counts and a
// coarse class. The classification is generic and rule-free:
//
// dup — another opportunity shares the same title key
// empty — no files, tasks or members
// junk-title — title is not of the "Role @ Company" shape
// ok — everything else
//
// Callers that need stricter business rules can post-process the result.
func (c *Client) AuditOpportunities(ctx context.Context) ([]OpportunityAudit, error) {
deals, err := c.ListAllOpportunities(ctx)
if err != nil {
return nil, err
}
tasks, _, err := c.ListCRMTasks(ctx, 5000, 0)
if err != nil {
return nil, err
}
open, closed := taskCountsByOpportunity(tasks)
out := make([]OpportunityAudit, 0, len(deals))
for _, row := range deals {
id := auditID(row["id"])
if id == 0 {
continue
}
title := auditStr(row["title"])
files := 0
if fl, ferr := c.ListOpportunityFiles(ctx, strconv.FormatInt(id, 10)); ferr == nil {
files = len(fl)
}
key := strconv.FormatInt(id, 10)
out = append(out, OpportunityAudit{
ID: id,
Title: title,
Created: auditStr(row["created"]),
Files: files,
OpenTasks: open[key],
ClosedTasks: closed[key],
Members: len(OpportunityMembers(row)),
GroupKey: DealTitleKey(title, false),
})
}
groupCount := map[string]int{}
for _, a := range out {
groupCount[a.GroupKey]++
}
for i := range out {
a := &out[i]
switch {
case groupCount[a.GroupKey] > 1:
a.Class = "dup"
case a.Files == 0 && a.OpenTasks == 0 && a.ClosedTasks == 0 && a.Members == 0:
a.Class = "empty"
case !strings.Contains(a.Title, "@") || strings.HasPrefix(strings.TrimSpace(a.Title), "@"):
a.Class = "junk-title"
default:
a.Class = "ok"
}
}
return out, nil
}
// taskCountsByOpportunity buckets CRM task statuses per opportunity id.
func taskCountsByOpportunity(tasks []map[string]any) (open, closed map[string]int) {
open, closed = map[string]int{}, map[string]int{}
for _, t := range tasks {
ent, ok := t["entity"].(map[string]any)
if !ok || auditStr(ent["entityType"]) != "opportunity" {
continue
}
eid := auditStr(ent["entityId"])
status := strings.ToLower(auditStr(t["status"]))
if status == "2" || status == "closed" {
closed[eid]++
} else {
open[eid]++
}
}
return open, closed
}
func auditID(v any) int64 {
switch x := v.(type) {
case float64:
return int64(x)
case int:
return int64(x)
case int64:
return x
default:
n, _ := strconv.ParseInt(strings.TrimSpace(fmt.Sprint(x)), 10, 64)
return n
}
}
func auditStr(v any) string {
if v == nil {
return ""
}
return fmt.Sprint(v)
}
+36
View File
@@ -0,0 +1,36 @@
package onlyoffice
import "testing"
func TestAuditID(t *testing.T) {
cases := []struct {
in any
want int64
}{
{float64(12), 12},
{7, 7},
{int64(9), 9},
{"42", 42},
{nil, 0},
}
for _, c := range cases {
if got := auditID(c.in); got != c.want {
t.Fatalf("auditID(%v) = %d, want %d", c.in, got, c.want)
}
}
}
func TestTaskCountsByOpportunity(t *testing.T) {
open, closed := taskCountsByOpportunity([]map[string]any{
{"id": 1, "status": 1, "entity": map[string]any{"entityType": "opportunity", "entityId": 10}},
{"id": 2, "status": "2", "entity": map[string]any{"entityType": "opportunity", "entityId": 10}},
{"id": 3, "status": 1, "entity": map[string]any{"entityType": "contact", "entityId": 10}},
{"id": 4, "entity": "not-a-map"},
})
if open["10"] != 1 || closed["10"] != 1 {
t.Fatalf("open=%v closed=%v", open, closed)
}
if len(open) != 1 || len(closed) != 1 {
t.Fatalf("unexpected buckets: open=%v closed=%v", open, closed)
}
}
-70
View File
@@ -1,70 +0,0 @@
# rclone WebDAV mount of the oo-webdav sidecar (OnlyOffice Documents).
#
# The sidecar publishes the portal's WebDAV tree on the docker bridge
# (http://172.17.0.1:8098, prefix /webdav). rclone mounts it inside this
# container as a normal filesystem.
#
# Credentials are OnlyOffice portal credentials (the same the `oo` CLI uses).
# Export them before `docker compose up`, e.g.
# set -a; . .secrets/oo.env; set +a
# or copy .env.example to .env and fill it. Only names live in git.
#
# docker compose -f deploy/docker-compose.rclone-webdav.yml up -d
# docker exec rclone-webdav ls /mnt/onlyoffice
# docker compose -f deploy/docker-compose.rclone-webdav.yml down
#
# The mount point is the named volume `onlyoffice-mnt`; join it from another
# container with `external_volumes` to read the tree without the API.
name: rclone-webdav
services:
rclone-webdav:
image: rclone/rclone:latest
container_name: rclone-webdav
restart: unless-stopped
# rclone mount needs FUSE. The remote is defined on the fly via flags for
# the mount; the same credentials are written to an rclone config so that
# `docker exec rclone-webdav rclone ls webdav:` works without flags.
entrypoint: ["/bin/sh", "-c"]
command:
- |
set -eu
: "$${ONLYOFFICE_USER:?ONLYOFFICE_USER is required}"
: "$${ONLYOFFICE_PASSWORD:?ONLYOFFICE_PASSWORD is required}"
obscured=$$(rclone obscure "$${ONLYOFFICE_PASSWORD}")
mkdir -p /config/rclone
cat > /config/rclone/rclone.conf <<EOF
[webdav]
type = webdav
url = $${ONLYOFFICE_WEBDAV_URL}
vendor = other
user = $${ONLYOFFICE_USER}
pass = $${obscured}
EOF
exec rclone mount :webdav: /mnt/onlyoffice \
--webdav-url "$${ONLYOFFICE_WEBDAV_URL}" \
--webdav-user "$${ONLYOFFICE_USER}" \
--webdav-pass "$${obscured}" \
--vfs-cache-mode writes \
--cache-dir /cache \
--dir-cache-time 1m \
--allow-other \
--allow-non-empty
environment:
ONLYOFFICE_USER: ${ONLYOFFICE_USER:?ONLYOFFICE_USER is required}
ONLYOFFICE_PASSWORD: ${ONLYOFFICE_PASSWORD:-${ONLYOFFICE_PASS:?ONLYOFFICE_PASSWORD is required}}
ONLYOFFICE_WEBDAV_URL: ${ONLYOFFICE_WEBDAV_URL:-http://172.17.0.1:8098/webdav}
cap_add:
- SYS_ADMIN
devices:
- /dev/fuse
# Ubuntu hosts run AppArmor; FUSE mounts need it unconfined.
security_opt:
- apparmor:unconfined
volumes:
- onlyoffice-mnt:/mnt/onlyoffice
- rclone-cache:/cache
volumes:
onlyoffice-mnt:
rclone-cache:
+5 -3
View File
@@ -17,9 +17,11 @@ related:
(MySQL/PostgreSQL): схема, SSH-туннель, DSN, MinIO download.
- [elasticsearch.md](elasticsearch.md) — поиск: индекс OnlyOffice `files_file`
и свой `oo_docs_text` (PDF/сканы), туннель.
- [rclone-webdav.md](rclone-webdav.md) — rclone-монтирование Documents
(`deploy/docker-compose.rclone-webdav.yml`), smoke, ограничения.
- [crm-associations.md](crm-associations.md) — правила ассоциаций CRM.
- [index-and-search.md](index-and-search.md) — карта контуров поиска и как
обновлять индексы (`oo index`, `oo search`).
- [rate-limiting.md](rate-limiting.md) — rate limit, exponential backoff,
`Retry-After`, общий cooldown против 429; env `OO_RATE_LIMIT`/`OO_BURST`/
`OO_RETRY_*`.
## Тесты
Команды и туннели — раздел Testing в [README.md](../README.md#testing).
+2 -2
View File
@@ -3,7 +3,7 @@ type: reference
status: current
related:
- README.md
- file_pg.go
- filestore_pg.go
- docs/elasticsearch.md
---
@@ -11,7 +11,7 @@ related:
## Что это
Бэкенд `pgStore` (`file_pg.go`) читает файлы и папки **напрямую из БД
Бэкенд `pgStore` (`filestore_pg.go`) читает файлы и папки **напрямую из БД
Community Server**, без HTTP-слоя. Реализует `FileStore` (`List`/`Stat`/
`Download`) и `Searcher` по имени. Запись запрещена: все write-методы
возвращают `ErrReadOnly`.
-124
View File
@@ -1,124 +0,0 @@
# CRM associations (company ↔ person ↔ deal ↔ project ↔ invoice ↔ mail)
Operational rules for the `oo` CLI and this library. Business SSOT remains
OnlyOffice Workspace CRM + Projects.
## Canonical graph
One **legal company** owns the relationship. Do not invent a second “bill-to”
company just for PDF layout.
```text
Company
├── Person (buyer contact) oo persons create --company-id
├── Opportunity / Deal oo opportunities … ; member-add company + person
├── Project (hub) oo projects … ; contacts add company + person
│ └── Epic + subtasks
└── Invoice (Draft → …) oo invoices create --contact COMPANY --opportunity DEAL
└── PDF file oo invoices pdf ID
└── Mail draft oo mails draft-invoice --invoice ID --to …
```
| Layer | CLI | Must link |
|-------|-----|-----------|
| Company | `oo companies create` | website, email, phone, **one** Billing address |
| Person | `oo persons create --company-id` / `oo persons update ID` | job title; never encode employer in `lastName`; **update uses JSON** (form PUT ignores `companyId`/`about`) |
| Deal | `oo opportunities create` + `member-add` | company **and** person as members |
| Project | `oo projects create` + `contacts add` | same company + person |
| Invoice | `oo invoices create --contact COMPANY --opportunity DEAL` | `entityId` at **create** |
| Mail | `oo mails draft-invoice` | attach current PDF; **do not send** until confirmed |
UI checks (same company card):
- `#contacts` → person
- `#deals` → opportunity
- `#projects` → hub project
- `#invoices` on the **deal** → invoice (needs `entity`)
- `#files` → preferably **one** current invoice PDF
**Project Team ≠ Project Contacts.** Team = portal users. CRM people/companies
show under the project **Contacts** tab (`oo projects contacts list`).
## Hard rules
1. **One company per legal entity.** Duplicate “bill-to” contacts empty Deals /
Projects / Contacts tabs and break merge. Prefer
`oo contacts merge FROM INTO` (keeps `INTO`) or `oo companies dedupe`.
2. **Link invoice → deal at create.**
`POST /crm/invoice` with `entityId` + `entityType: 0` (Opportunity).
`oo invoices update … --opportunity` often returns **400**
(“Value does not fall within the expected range”). If the link is missing,
delete the Draft and recreate with `--opportunity`.
3. **Bill To = company id**, not a throwaway contact. Person stays under the
company (`companyId`). Optional `consigneeId` for Empfänger when the portal
template prints it.
4. **Stay Draft until mail is ready.** Billed (`status id=2`) is **not editable**
via content PUT. Going Billed → Draft via `…/crm/invoice/status/1` usually
**does not work** — delete + recreate Draft instead.
5. **Do not regenerate PDF in a loop** without cleanup. Each
`GET …/crm/invoice/{id}/pdf` attaches a new file to the company (and often
the deal). Keep `invoice.fileID`; delete older PDFs with
`oo invoices pdf-cleanup ID` / Documents `fileops/delete`.
## Invoice PDF quirks
| Symptom | Workaround |
|---------|------------|
| Cached / stale PDF | Touch invoice (Draft PUT that clears `fileID`), then `GET …/pdf` — `oo invoices pdf ID --force` |
| Billing address missing on **new** PDFs | Temporary multiline `companyName` (`Line1\nLine2\n…`) on the **canonical** company → force PDF → restore clean name. Cached `fileID` keeps the multiline Bill To. |
| Separate bill-to company for newlines | **Forbidden** — merge back to the real company |
| Invoice **number** won’t change on PUT | Delete Draft and recreate with the desired number |
| Notizen / Bedingungen spacing | Leading `\n` and blank lines only — no HTML (tags print literally) |
| Issuer street lines | Organisation profile address (`street` with `\n`), not only terms |
Status ids commonly used: `1` Draft, `2` Billed, `3` Rejected, `4` Paid.
## Mail quirks
| Symptom | Workaround |
|---------|------------|
| Signature / body doubles chat URL | Put chat in **one** place only. UI drafts: signature. API send: body (API **does not** append signature). |
| Signature / body cuts URL at `#` | Plain text URLs — avoid `<a href="…#…">` (or encode `#` as `%23` in href) |
| German letter spacing | Blank `<p>&nbsp;</p>` between blocks (`MailHTMLWithBlankParagraphs`) |
| Send | `PUT /api/2.0/mail/messages/send.json` with `id/from/to/subject/body`; omit empty `cc`/`bcc`. Never auto-send; draft only until the human confirms |
Prefer OnlyOffice Mail (`/addons/mail/#drafts`) for invoice delivery until confirmed.
## Project / task quirks
- Hub title: `CC | Company` (e.g. `DE | Acme GmbH`).
- Streams = epics/tasks under the hub, not a third title segment (unless the
project itself is a named delivery stream).
- Closing a **subtask**:
`PUT /api/2.0/project/task/{epicId}/{subtaskId}/status` with `status=2`.
`oo tasks update SUBTASK -s closed` returns **404** for subtasks.
- After deleting a CRM contact, `GET /project/contact/{deletedId}` may still
return projects (ghost). Official project contact list should only show live
ids; unlink may 400 if the contact is gone.
## Merge / cleanup cheat sheet
```bash
# Keep the preferred company (INTO), drop the duplicate (FROM)
oo contacts merge FROM_ID INTO_ID
# Or by normalized name (careful — whole CRM)
oo companies dedupe
# Invoice ↔ deal must exist at create
oo invoices create --number P-YYYY-NN --contact COMPANY_ID --item ITEM_ID \
--price 300 --opportunity DEAL_ID --language de-DE …
# Fresh PDF + prune older PDFs on company/deal
oo invoices pdf INVOICE_ID --force
oo invoices pdf-cleanup INVOICE_ID
# Mail draft (no send)
oo mails draft-invoice --invoice INVOICE_ID --to billing@example.com
```
## Related
- README § invoices / mail / CRM cleanup
- Personal workspace tooling (disk inventory, dossier sync): private
`git.produktor.io/eSlider/oo-workspace` (`oow` CLI)
+10 -5
View File
@@ -3,7 +3,7 @@ type: reference
status: current
related:
- README.md
- file_es.go
- filestore_es.go
---
# Elasticsearch — полнотекстовый поиск OnlyOffice
@@ -83,9 +83,14 @@ oo search "Rechnung" --folder 649 --limit 50 --json
`folders.folderId`), `--limit N` (по умолчанию 20, максимум 200),
`--json` = `-o json`.
## Обновление индекса и карта поиска
Обзор всех контуров поиска и как обновлять индексы (`oo index`) —
[index-and-search.md](index-and-search.md).
## Библиотека
`file_es.go` — `ESSearcher` (`Name() = "elasticsearch"`), прямой ES REST на
`filestore_es.go` — `ESSearcher` (`Name() = "elasticsearch"`), прямой ES REST на
stdlib `net/http`:
```go
@@ -98,7 +103,7 @@ hits, _ := es.Search(ctx, onlyoffice.SearchQuery{
Запрос: `multi_match` по `title^2` (+ `document.attachment.content` при
`InContent`), фильтры `tenantId` и `folders.folderId`, `_source`
id/title/folders, `highlight` для фрагмента. Ответ → `[]SearchHit` (модель из
эпика #34; пока объявлена в `file_es.go`, переедет в `file_core.go` с F1 #35).
эпика #34; пока объявлена в `filestore_es.go`, переедет в `filestore_core.go` с F1 #35).
## Тесты
@@ -163,11 +168,11 @@ OnlyOffice PDF лежит только по имени.
## Устройство
- `file_es_text.go` — `ESTextIndex` (`Name() = "es-text"`):
- `filestore_es_text.go` — `ESTextIndex` (`Name() = "es-text"`):
`Ensure` (создаёт индекс с явным маппингом), `Put` (bulk, `refresh`),
`Delete` (по `id`), `Search` (`multi_match` по `title^2` + `content`,
фильтры `folder`/`ext`, highlight).
- `file_text_index.go` — `TextIndexer`: листает папки (`FileStore.List`),
- `filestore_text_index.go` — `TextIndexer`: листает папки (`FileStore.List`),
качает файлы (`FileStore.Download`), извлекает текст через
`internal/docpipe` (`pdftotext`, для сканов — `ocrmypdf`/`tesseract`),
пишет в `TextIndex`. Пул воркеров (по умолчанию 3).
+96
View File
@@ -0,0 +1,96 @@
---
type: reference
status: current
related:
- docs/elasticsearch.md
- docs/unified-file-client.md
- docs/community-server-db.md
---
# Поиск и индексация
Четыре разных контура поиска. Не путать: у каждого свой индекс, свои входы и
свой способ обновления.
| Контур | Что ищет | Индекс | Обновление | Вход |
|--------|----------|--------|------------|------|
| REST `@search` | только имена в БД | нет | — (живой запрос) | `oo search` (по умолчанию `--backend oo`) |
| ES `files_file` | имя + текст Office | Elasticsearch портала | сервер, асинхронно | `oo search --content` |
| ES `oo_docs_text` | PDF/сканы (свой) | Elasticsearch портала | `oo index` | `oo search --backend own` |
## Карта кода
- `filestore_core.go` — интерфейсы `Searcher`, модели `SearchQuery`/`SearchHit`.
- `filestore_es.go` — `ESSearcher` (индекс OnlyOffice `files_file`).
- `filestore_es_text.go` — `ESTextIndex` (`oo_docs_text`): `Ensure`, `Put`, `Delete`,
`Search`.
- `filestore_text_index.go` — `TextIndexer`: обход папок (`FileStore.List`), download
(`FileStore.Download`), извлечение текста (`internal/docpipe`), запись в
`TextIndex`; пул воркеров.
- `filestore_facade.go` — связка бэкендов (`Files().Search()`, порядок и fallback).
- CLI: `cmd/oo/search.go`, `cmd/oo/index.go`.
- Разовые бинари для match (`ooscan`, `pdfamount`) живут в приватном
`oo-workspace`.
- `internal/docpipe` — текст из PDF (pdftotext), для сканов OCR
(ocrmypdf/tesseract), вложения PDF (pdfdetach).
## Поиск
```bash
# имя, индекс портала
oo search "Rechnung" --limit 50 --json
# имя + текст Office (docx/xlsx/pptx)
oo search "Mahngebühr" --content
# свой индекс: PDF и сканы
oo search "S1019" --backend own --folder 649
```
Флаги `oo search`: `--content`, `--folder ID`, `--limit N`, `--backend oo|own`,
`--substring`, `--json`. Требует `ONLYOFFICE_ES_URL` (см.
[elasticsearch.md](elasticsearch.md)); `--backend own` дополнительно ничего не
требует от сервера — читает `oo_docs_text`.
Почему не REST: `GET /api/2.0/files/@search/{query}` ищет только имя в БД
(`fileDao.Search`), ES не трогает. Почему PDF не в `files_file`: сервер индексит
контент только для форматов из `files.index.formats` (по умолчанию
`.pptx|.xlsx|.docx`) — отсюда свой `oo_docs_text`.
## Обновление своего индекса (`oo index`)
```bash
# одна папка
oo index folder 649 --recursive --exts pdf
# точечно по id
oo index files 3576 3578
# без записи: что было бы проиндексировано
oo index folder 649 --recursive --dry-run
```
Флаги: `--recursive`, `--exts pdf` (по умолчанию), `--limit N`,
`--workers 3`, `--lang deu+eng`, `--min-chars N` (порог текстового слоя, ниже
которого включается OCR), `--work-dir`, `--backend rest|dav`, `--dry-run`,
`--json`.
Свойства:
- Идемпотентно: upsert по `id` файла; повтор не двоит.
- Сервер OnlyOffice не меняется: индекс живёт рядом (`ONLYOFFICE_ES_TEXT_INDEX`,
по умолчанию `oo_docs_text`).
- Медленно на сканах (OCR на каждый файл). Ограничивай `--folder`/`--limit`,
не индексируй корень целиком.
- Индексация PDF в `files_file` не делается — только `oo_docs_text`.
## Bulk-инструменты
Плоские TSV-инструменты (`ooscan`, `pdfamount`) и сверка Excel живут в
приватном `oo-workspace`, не в публичной библиотеке.
## Грабли
- ES слушает только `127.0.0.1:9200` внутри VM — SSH-туннель обязателен
(`-o ControlMaster=no -o ControlPath=none`, см. [elasticsearch.md](elasticsearch.md)).
- Портальные листинги/скачивание упираются в 429; все bulk-пути идут через
`DoRetry` (линейный бэкофф), `ooscan` дополнительно спит 350 мс на папку.
- `files_file` обновляется сервером асинхронно — свежий файл виден не сразу.
- `title` analyzer `whitespacecustom`: имя — один токен, подстрока только через
`--substring` (или wildcard).
+55
View File
@@ -0,0 +1,55 @@
---
type: reference
status: current
related:
- README.md
- ../AGENTS.md
---
# Rate limit, backoff и cooldown
Устойчивость к 429 (openresty). Всё встроено в библиотеку — отдельный пакет не
нужен. Реализация: `ratelimit.go`, `retry.go`.
## Что происходит с каждым запросом
1. **Cooldown-гейт** — общий на процесс. Если недавно пришёл 429, все запросы
ждут конца окна.
2. **Rate limiter** — token bucket на процесс. Пейсит все HTTP-пути: листинг,
создание папок, загрузку, `get project`, auth.
3. Запрос уходит.
4. Ответ ≥400 → `*TransientError` (для 429/502/503/504) с `Retry-After`.
5. `DoRetry` — экспоненциальный backoff, без jitter.
6. `Retry-After` длиннее backoff → ждём его; окно уходит в общий cooldown.
Установлено в `NewClient` через `pacedTransport`; отдельный код трогать не надо.
## Env
| Переменная | Default | Смысл |
|---|---|---|
| `OO_RATE_LIMIT` | `4` | запросов/с на процесс; `0` — лимитер выключен |
| `OO_BURST` | `1` | запас токенов token bucket |
| `OO_RETRY_ATTEMPTS` | `7` | всего попыток, включая первую |
| `OO_RETRY_BASE` | `2s` | база экспоненты: ждать перед попыткой N = `Base*2^(N-1)` |
| `OO_RETRY_MAX` | `2m` | потолок ожидания |
Битые значения → default. `OO_RETRY_*` — формат `time.ParseDuration`
(`2s`, `30s`, `2m`).
## Правила
- Детерминированно, без jitter — повторный прогон ждёт столько же.
- `Retry-After` — секунды (`120`) или HTTP-date.
- Cooldown общий: параллельные и последовательные вызовы не бьют в стену.
- Backoff cap не ограничивает `Retry-After` — серверу верим больше.
- Только stdlib.
## Когда руками снять нагрузку
`OO_RATE_LIMIT` ниже (`2`), `OO_BURST=1`; при массовом apply — батчами.
## Тесты
`retry_test.go` — `Retry-After`, экспонента, cap; `ratelimit_test.go` — burst,
`OO_RATE_LIMIT=0`, cooldown. Фейковый сервер отдаёт 429 с заголовком.
-109
View File
@@ -1,109 +0,0 @@
---
type: reference
status: current
related:
- deploy/docker-compose.rclone-webdav.yml
- docs/unified-file-client.md
---
# rclone WebDAV mount — OnlyOffice Documents как файловая ФС
`deploy/docker-compose.rclone-webdav.yml` монтирует WebDAV-дерево OnlyOffice
через `rclone`. Не systemd, только compose. Контейнер — `rclone-webdav`.
## Что монтируется
- Источник — сайдкар `oo-webdav` (`ghcr.io/eslider/oo-webdav`).
- Адрес — `http://172.17.0.1:8098`, префикс `/webdav`.
- Basic-auth — креды портала OnlyOffice (те же, что у `oo`).
- Корень — `Dokumente der Projekte/…` (папка `Fibu EDL` внутри).
## Конфиг
- Образ — `rclone/rclone`.
- Remote — on-the-fly `:webdav:` плюс флаги `--webdav-url`, `--webdav-user`,
`--webdav-pass`.
- На старте контейнер пишет `/config/rclone/rclone.conf` (`[webdav]`), чтобы
`rclone ls webdav:` внутри контейнера работал без флагов.
- `ONLYOFFICE_PASSWORD` — plain; rclone сам зовёт `rclone obscure`.
- Флаги mount: `--vfs-cache-mode writes`, `--cache-dir /cache`,
`--dir-cache-time 1m`, `--allow-other`, `--allow-non-empty`.
- FUSE: `cap_add: [SYS_ADMIN]`, `devices: [/dev/fuse]`,
`security_opt: apparmor:unconfined`.
- Тома: `onlyoffice-mnt` → `/mnt/onlyoffice`, `rclone-cache` → `/cache`.
### env (только имена)
| Переменная | Значение |
|------------|----------|
| `ONLYOFFICE_USER` | пользователь портала |
| `ONLYOFFICE_PASSWORD` | пароль портала (алиас `ONLYOFFICE_PASS`) |
| `ONLYOFFICE_WEBDAV_URL` | по умолчанию `http://172.17.0.1:8098/webdav` |
## Команды
```bash
# креды (или .env рядом с compose)
set -a; . .secrets/oo.env; set +a
docker compose -f deploy/docker-compose.rclone-webdav.yml up -d
docker compose -f deploy/docker-compose.rclone-webdav.yml ps
docker compose -f deploy/docker-compose.rclone-webdav.yml down
# дерево
docker exec rclone-webdav rclone ls webdav:
docker exec rclone-webdav rclone lsf webdav:
# содержимое точки монтирования
docker exec rclone-webdav ls /mnt/onlyoffice
# чтение/запись как обычная ФС
docker exec rclone-webdav cat "/mnt/onlyoffice/Meine Dokumente/x.txt"
```
## Smoke (проверено 2026-09-16)
```text
$ docker exec rclone-webdav rclone ls webdav:
5357109 Meine Dokumente/ONLYOFFICE-Audiobeispiel.mp3
44805 Meine Dokumente/ONLYOFFICE-Beispiel-Tabellenblatt.xlsx
58988 Meine Dokumente/ONLYOFFICE-Beispieldokument.docx
...
$ TS=20260916-212459
$ docker exec rclone-webdav sh -c "printf 'rclone-webdav round-trip $TS\n' \
> '/mnt/onlyoffice/Meine Dokumente/rclone-smoke-$TS/hello.txt'"
# ждём появления на сервере (host HTTP, мимо монтирования):
GET /webdav/Meine%20Dokumente/rclone-smoke-$TS/hello.txt -> 200 (через 7s)
server bytes: rclone-webdav round-trip 20260916-212459
# читаем обратно через монтирование:
mount read: rclone-webdav round-trip 20260916-212459
MATCH: yes
$ docker exec rclone-webdav rm -f \
"/mnt/onlyoffice/Meine Dokumente/rclone-smoke-$TS/hello.txt"
GET /webdav/Meine%20Dokumente/rclone-smoke-$TS/hello.txt -> 404 (через 1s)
# пустой каталог: rmdir через монтирование на сервер не доходит,
# удаляем напрямую:
$ docker exec rclone-webdav rclone rmdir "webdav:Meine Dokumente/rclone-smoke-$TS"
PROPFIND rclone-smoke-20260916-212459/ -> 404
no rclone-smoke leftovers
```
Вывод: запись → чтение → удаление файла подтверждены. Файл удалён.
## Ограничения
- Нужен FUSE: `SYS_ADMIN` + `/dev/fuse`. На хосте без FUSE не поедет.
- Сайдкар слушает docker-bridge `172.17.0.1:8098`, публично не выставлен.
Только localhost/локальные контейнеры.
- Запись идёт через VFS write-back (по умолчанию ~5s). Перед проверкой
«файл на сервере» опрашивать сервер, не доверять сразу после `write`.
- `rmdir` через монтирование НЕ доходит до `oo-webdav` (пустой каталог
остаётся на сервере). Удалять каталоги напрямую:
`rclone rmdir webdav:<path>` или `rclone purge webdav:<path>`.
- VFS-кэш растёт в томе `rclone-cache`; ограничить `--vfs-cache-max-size`.
- Блокировок между редактором OnlyOffice и монтированием нет. Не редактировать
один и тот же файл одновременно.
- Данные не шифруются на диске хоста в `rclone-cache` (том Docker).
+9 -10
View File
@@ -3,8 +3,8 @@ type: reference
status: current
related:
- README.md
- file_core.go
- file_facade.go
- filestore_core.go
- filestore_facade.go
- docs/elasticsearch.md
- docs/community-server-db.md
---
@@ -14,7 +14,7 @@ related:
## Что это
Один файловый клиент на все бэкенды (эпик #34). Модель и интерфейсы —
`file_core.go`. Фасад `FileClient` — `file_facade.go`. Бэкенды:
`filestore_core.go`. Фасад `FileClient` — `filestore_facade.go`. Бэкенды:
REST, WebDAV, SQL (PostgreSQL/MySQL), Elasticsearch. Правило одно:
код зовёт `c.Files()` и не знает про транспорт.
@@ -56,18 +56,18 @@ type Searcher interface {
}
```
`TextIndex` (`file_es_text.go`) — свой индекс: `Put`, `Delete`, `Search`,
`TextIndex` (`filestore_es_text.go`) — свой индекс: `Put`, `Delete`, `Search`,
`Name`. `ESTextIndex` реализует и `Searcher`, и `TextIndex`.
## Бэкенды
| бэкенд | провайдер | файл | что умеет |
|--------|-----------|------|-----------|
| REST | `rest` | `file_rest.go` | read + write, Documents API |
| WebDAV | `dav` | `file_dav.go` | read + write, Documents fileops |
| SQL | `postgres` / `mysql` | `file_pg.go` | **read-only** |
| OnlyOffice ES | `elasticsearch` | `file_es.go` | поиск (имя + контент Office) |
| свой ES-индекс | `es-text` | `file_es_text.go` | поиск + запись (PDF/сканы) |
| REST | `rest` | `filestore_rest.go` | read + write, Documents API |
| WebDAV | `dav` | `filestore_dav.go` | read + write, Documents fileops |
| SQL | `postgres` / `mysql` | `filestore_pg.go` | **read-only** |
| OnlyOffice ES | `elasticsearch` | `filestore_es.go` | поиск (имя + контент Office) |
| свой ES-индекс | `es-text` | `filestore_es_text.go` | поиск + запись (PDF/сканы) |
- REST: `Stat` знает только файлы; папки — через `List`.
- WebDAV: `Move`/`Copy`/`Delete` сперва `Stat`-ят id (папка/файл), потом зовут
@@ -202,4 +202,3 @@ go test ./ -run 'FileStore|Facade|ESText|PG'
- [README.md](README.md) — индекс справочников.
- [elasticsearch.md](elasticsearch.md) — индекс OnlyOffice и свой `oo_docs_text`.
- [community-server-db.md](community-server-db.md) — SQL-стор и схема БД.
- [rclone-webdav.md](rclone-webdav.md) — монтирование Documents как ФС.
+3 -2
View File
@@ -237,7 +237,8 @@ func (c *Client) UploadProjectFile(ctx context.Context, projectID, localPath str
return decodeResponseFileEntry(raw)
}
// UploadProjectFileReplacing upserts by stem|ext in the project Documents folder.
// UploadProjectFileReplacing upserts by stem and (server-converted) extension
// in the project Documents folder.
func (c *Client) UploadProjectFileReplacing(ctx context.Context, projectID, localPath string) (*FileEntry, []int, error) {
folderID, err := c.projectFolderID(ctx, projectID)
if err != nil {
@@ -278,7 +279,7 @@ func (c *Client) RenameFile(ctx context.Context, fileID, newTitle string) (*File
// DeleteFiles permanently deletes files by numeric id (Documents module).
// Uses per-file DELETE (DeleteDavItems); fileops/delete returns 200 on some
// portals (e.g. produktor.io) without removing the file.
// portals without actually removing the file.
//
// Deprecated: use FileStore.Delete via Client.Files()/Client.FileStore.
func (c *Client) DeleteFiles(ctx context.Context, fileIDs []int) error {
+79
View File
@@ -83,6 +83,85 @@ func UploadExtFromLocal(localPath string) string {
return strings.ToLower(ext)
}
// legacyToOOXMLExt maps the legacy binary Office extensions OnlyOffice accepts
// on upload to the OOXML extension the server converts them into.
var legacyToOOXMLExt = map[string]string{
".xls": ".xlsx",
".doc": ".docx",
".ppt": ".pptx",
}
// normalizeExt lowercases an extension and ensures a leading dot.
func normalizeExt(ext string) string {
ext = strings.ToLower(strings.TrimSpace(ext))
if ext == "" {
return ""
}
if !strings.HasPrefix(ext, ".") {
ext = "." + ext
}
return ext
}
// EquivalentUploadExt reports whether two extensions designate the same
// document once the server-side conversion is taken into account: equal
// extensions, or a legacy binary Office format and its OOXML equivalent
// (.xls/.xlsx, .doc/.docx, .ppt/.pptx). Empty extensions only match each other.
func EquivalentUploadExt(a, b string) bool {
na, nb := normalizeExt(a), normalizeExt(b)
if na == nb {
return true
}
if na == "" || nb == "" {
return false
}
return legacyToOOXMLExt[na] == nb || legacyToOOXMLExt[nb] == na
}
// FindFilesByStemExt returns folder files matching stem and a same-or-converted
// extension (see EquivalentUploadExt). Unlike FindFilesByDedupKey, foo.xls and
// foo.xlsx are one logical file, so a replacing upload after OnlyOffice's
// legacy→OOXML conversion finds the saved file instead of appending a copy.
func FindFilesByStemExt(files []*FileEntry, stem, ext string) []*FileEntry {
stem = strings.TrimSpace(stem)
if stem == "" {
return nil
}
var out []*FileEntry
for _, f := range files {
if FileEntryStem(f) != stem {
continue
}
if EquivalentUploadExt(FileEntryExt(f), ext) {
out = append(out, f)
}
}
return out
}
// DeleteFilesByStemExt removes every folder file matching stem and a
// same-or-converted extension (legacy ↔ OOXML).
func (c *Client) DeleteFilesByStemExt(ctx context.Context, folderID, stem, ext string) ([]int, error) {
files, err := c.FolderFiles(ctx, folderID)
if err != nil {
return nil, err
}
matches := FindFilesByStemExt(files, stem, ext)
ids := make([]int, 0, len(matches))
for _, f := range matches {
if n := int(FileEntryNumericID(f)); n != 0 {
ids = append(ids, n)
}
}
if len(ids) == 0 {
return nil, nil
}
if err := c.DeleteFiles(ctx, ids); err != nil {
return nil, err
}
return ids, nil
}
// IsTrashFolderTitle reports staging/trash folders (e.g. _trash-md).
func IsTrashFolderTitle(title string) bool {
t := strings.ToLower(strings.TrimSpace(title))
+180
View File
@@ -0,0 +1,180 @@
package onlyoffice
import (
"testing"
"time"
)
func TestEquivalentUploadExt(t *testing.T) {
tests := []struct {
a, b string
want bool
}{
{".xls", ".xlsx", true},
{".XLS", ".xlsx", true},
{"xls", "xlsx", true},
{".doc", ".docx", true},
{".ppt", ".pptx", true},
{".pdf", ".pdf", true},
{"", "", true},
{".xls", ".docx", false},
{".xlsx", "", false},
{".csv", ".xlsx", false},
}
for _, tc := range tests {
if got := EquivalentUploadExt(tc.a, tc.b); got != tc.want {
t.Errorf("EquivalentUploadExt(%q,%q)=%v want %v", tc.a, tc.b, got, tc.want)
}
if got := EquivalentUploadExt(tc.b, tc.a); got != tc.want {
t.Errorf("EquivalentUploadExt(%q,%q)=%v want %v (symmetric)", tc.b, tc.a, got, tc.want)
}
}
}
func TestFindFilesByStemExtMatchesConvertedXLS(t *testing.T) {
xlsx := &FileEntry{ID: jsonNum("3799"), Title: strPtr("ES29-extracto.xlsx"), FileExst: strPtr(".xlsx")}
pdf := &FileEntry{ID: jsonNum("5"), Title: strPtr("ES29-extracto.pdf"), FileExst: strPtr(".pdf")}
other := &FileEntry{ID: jsonNum("3887"), Title: strPtr("ES87-extracto.xlsx"), FileExst: strPtr(".xlsx")}
files := []*FileEntry{xlsx, pdf, other}
got := FindFilesByStemExt(files, "ES29-extracto", ".xls")
if len(got) != 1 || got[0] != xlsx {
t.Fatalf("converted .xls match = %+v, want the saved .xlsx only", got)
}
}
func TestFindFilesByStemExtKeepsExactMatch(t *testing.T) {
xlsx := &FileEntry{ID: jsonNum("3799"), Title: strPtr("foo.xlsx"), FileExst: strPtr(".xlsx")}
pdf := &FileEntry{ID: jsonNum("5"), Title: strPtr("foo.pdf"), FileExst: strPtr(".pdf")}
files := []*FileEntry{xlsx, pdf}
if got := FindFilesByStemExt(files, "foo", ".xlsx"); len(got) != 1 || got[0] != xlsx {
t.Fatalf("exact .xlsx match = %+v, want only xlsx", got)
}
if got := FindFilesByStemExt(files, "foo", ".pdf"); len(got) != 1 || got[0] != pdf {
t.Fatalf("exact .pdf match = %+v, want only pdf", got)
}
if got := FindFilesByStemExt(files, "foo", ".docx"); len(got) != 0 {
t.Fatalf("unrelated ext matched %+v, want none", got)
}
}
func TestFindFilesByStemExtEmptyStem(t *testing.T) {
f := &FileEntry{ID: jsonNum("1"), Title: strPtr("foo.xlsx"), FileExst: strPtr(".xlsx")}
if got := FindFilesByStemExt([]*FileEntry{f}, "", ".xlsx"); len(got) != 0 {
t.Fatalf("empty stem matched %+v, want none", got)
}
}
func TestPlanUploadReplacementFreshUpload(t *testing.T) {
plan := planUploadReplacement(nil, "ES29-extracto", ".xls")
if plan.UpdateID != "" || len(plan.DeleteIDs) != 0 {
t.Fatalf("empty folder plan = %+v, want a fresh upload", plan)
}
}
// TestPlanUploadReplacementUpdatesExactExt covers the in-place path: only a
// stored file with the same extension is updated via UpdateFile. PDF over PDF
// and OOXML over OOXML keep the id and rewrite the body.
func TestPlanUploadReplacementUpdatesExactExt(t *testing.T) {
tests := []struct {
name string
file *FileEntry
stem string
ext string
id string
}{
{
name: "xlsx over xlsx",
file: &FileEntry{ID: jsonNum("3799"), Title: strPtr("ES29-extracto.xlsx"), FileExst: strPtr(".xlsx")},
stem: "ES29-extracto", ext: ".xlsx", id: "3799",
},
{
name: "pdf over pdf",
file: &FileEntry{ID: jsonNum("42"), Title: strPtr("extracto.pdf"), FileExst: strPtr(".pdf")},
stem: "extracto", ext: ".pdf", id: "42",
},
{
name: "legacy xls over xls",
file: &FileEntry{ID: jsonNum("11"), Title: strPtr("legacy.xls"), FileExst: strPtr(".xls")},
stem: "legacy", ext: ".xls", id: "11",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
plan := planUploadReplacement([]*FileEntry{tc.file}, tc.stem, tc.ext)
if plan.UpdateID != tc.id {
t.Fatalf("UpdateID = %q, want %q (same ext updates in place)", plan.UpdateID, tc.id)
}
if len(plan.DeleteIDs) != 0 {
t.Fatalf("DeleteIDs = %v, want none", plan.DeleteIDs)
}
})
}
}
// TestPlanUploadReplacementConvertedExtDeletesThenUploads is the regression for
// #84: UpdateFile does not re-run the server-side legacy→OOXML conversion, so a
// .xls upload must not overwrite a stored .xlsx in place (raw OLE2 under an
// .xlsx name). The stale counterpart is deleted and the file uploaded afresh.
func TestPlanUploadReplacementConvertedExtDeletesThenUploads(t *testing.T) {
xlsx := &FileEntry{ID: jsonNum("3799"), Title: strPtr("ES29-extracto.xlsx"), FileExst: strPtr(".xlsx")}
plan := planUploadReplacement([]*FileEntry{xlsx}, "ES29-extracto", ".xls")
if plan.UpdateID != "" {
t.Fatalf("UpdateID = %q, want empty (do not update across conversion)", plan.UpdateID)
}
if len(plan.DeleteIDs) != 1 || plan.DeleteIDs[0] != 3799 {
t.Fatalf("DeleteIDs = %v, want [3799] (delete the stale .xlsx before upload)", plan.DeleteIDs)
}
}
func TestPlanUploadReplacementCollapsesDuplicates(t *testing.T) {
older := time.Date(2026, 9, 1, 10, 0, 0, 0, time.UTC)
newer := older.Add(time.Hour)
first := &FileEntry{ID: jsonNum("3799"), Title: strPtr("ES29-extracto.xlsx"), FileExst: strPtr(".xlsx"), Updated: &older}
second := &FileEntry{ID: jsonNum("3887"), Title: strPtr("ES29-extracto.xlsx"), FileExst: strPtr(".xlsx"), Updated: &newer}
plan := planUploadReplacement([]*FileEntry{first, second}, "ES29-extracto", ".xlsx")
if plan.UpdateID != "3887" {
t.Fatalf("UpdateID = %q, want the newest duplicate 3887", plan.UpdateID)
}
if len(plan.DeleteIDs) != 1 || plan.DeleteIDs[0] != 3799 {
t.Fatalf("DeleteIDs = %v, want [3799]", plan.DeleteIDs)
}
}
// TestPlanUploadReplacementDeletesConvertedDuplicates: with a converted match
// every stale copy is deleted (there is no keeper — the fresh upload replaces
// them all).
func TestPlanUploadReplacementDeletesConvertedDuplicates(t *testing.T) {
first := &FileEntry{ID: jsonNum("3799"), Title: strPtr("ES29-extracto.xlsx"), FileExst: strPtr(".xlsx")}
second := &FileEntry{ID: jsonNum("3887"), Title: strPtr("ES29-extracto.xlsx"), FileExst: strPtr(".xlsx")}
plan := planUploadReplacement([]*FileEntry{first, second}, "ES29-extracto", ".xls")
if plan.UpdateID != "" {
t.Fatalf("UpdateID = %q, want empty", plan.UpdateID)
}
if len(plan.DeleteIDs) != 2 {
t.Fatalf("DeleteIDs = %v, want both stale .xlsx ids", plan.DeleteIDs)
}
}
// TestPlanUploadReplacementRepeatedXLS is the regression for #84: with the
// server-converted .xlsx already present, the second .xls upload deletes it and
// uploads anew so OnlyOffice converts again — UpdateFile would corrupt it.
func TestPlanUploadReplacementRepeatedXLS(t *testing.T) {
const stem = "ES29-extracto"
// First upload: nothing in the folder.
if plan := planUploadReplacement(nil, stem, ".xls"); plan.UpdateID != "" || len(plan.DeleteIDs) != 0 {
t.Fatalf("first upload plan = %+v, want create", plan)
}
// OnlyOffice converts .xls -> .xlsx on upload; the second upload must
// delete it and upload fresh, never UpdateFile it.
saved := &FileEntry{ID: jsonNum("3799"), Title: strPtr(stem + ".xlsx"), FileExst: strPtr(".xlsx")}
plan := planUploadReplacement([]*FileEntry{saved}, stem, ".xls")
if plan.UpdateID != "" {
t.Fatalf("second upload plan = %+v, want delete+upload (not UpdateFile)", plan)
}
if len(plan.DeleteIDs) != 1 || plan.DeleteIDs[0] != 3799 {
t.Fatalf("second upload DeleteIDs = %v, want [3799]", plan.DeleteIDs)
}
}
+95 -7
View File
@@ -6,6 +6,7 @@ import (
"errors"
"fmt"
"path/filepath"
"strconv"
"strings"
)
@@ -121,7 +122,9 @@ func (c *Client) DeleteFilesByStem(ctx context.Context, folderID, stem string) (
return ids, nil
}
// AssertNoFileConflict reports ErrFileExists when localPath stem|ext is already in folderID.
// AssertNoFileConflict reports ErrFileExists when localPath logical name is
// already in folderID. Matching is conversion-aware, so a .xls upload also
// conflicts with an existing .xlsx and does not create a hidden duplicate.
func (c *Client) AssertNoFileConflict(ctx context.Context, folderID, localPath string) error {
files, err := c.FolderFiles(ctx, folderID)
if err != nil {
@@ -129,7 +132,7 @@ func (c *Client) AssertNoFileConflict(ctx context.Context, folderID, localPath s
}
stem := UploadStemFromLocal(localPath)
ext := UploadExtFromLocal(localPath)
matches := FindFilesByDedupKey(files, stem, ext)
matches := FindFilesByStemExt(files, stem, ext)
if len(matches) == 0 {
return nil
}
@@ -153,14 +156,99 @@ func (c *Client) UploadProjectFileNoClobber(ctx context.Context, projectID, loca
return c.UploadProjectFile(ctx, projectID, localPath)
}
// UploadToFolderReplacing deletes same stem+ext files then uploads localPath.
// uploadReplacementPlan is how a replacing upload reconciles with a folder.
type uploadReplacementPlan struct {
// UpdateID is the existing file to overwrite in place; set only when the
// stored extension equals the local one. UpdateFile keeps the stored name,
// so overwriting across extensions would leave an unconverted body under a
// mismatched name.
UpdateID string
// DeleteIDs are the ids to remove before uploading. They are the redundant
// duplicates of an in-place update, or every conversion-equivalent
// counterpart when the body must be converted again by a fresh upload.
DeleteIDs []int
}
// planUploadReplacement matches an incoming local upload (stem + ext) against
// the files already in a folder and decides between a fresh upload, an in-place
// update and delete + reupload. Matching tolerates the legacy→OOXML conversion
// OnlyOffice performs on upload, so a repeated upload of f.xls finds the saved
// f.xlsx instead of creating a second file. Because UpdateFile replaces the body
// without re-running that conversion, an equivalent-but-different extension is
// deleted and re-uploaded rather than updated in place (#84).
func planUploadReplacement(files []*FileEntry, stem, ext string) uploadReplacementPlan {
matches := FindFilesByStemExt(files, stem, ext)
if len(matches) == 0 {
return uploadReplacementPlan{}
}
keep, remove := pickDuplicateKeeper(matches, false)
plan := uploadReplacementPlan{}
if FileEntryExt(keep) == normalizeExt(ext) {
if id := FileEntryNumericID(keep); id != 0 {
plan.UpdateID = strconv.FormatInt(id, 10)
}
} else if id := int(FileEntryNumericID(keep)); id != 0 {
plan.DeleteIDs = append(plan.DeleteIDs, id)
}
for _, f := range remove {
if id := int(FileEntryNumericID(f)); id != 0 {
plan.DeleteIDs = append(plan.DeleteIDs, id)
}
}
return plan
}
// UploadToFolderReplacing upserts localPath into folderID by logical name.
// Matching is stem + extension with OnlyOffice's server-side conversion
// accounted for: a local .xls is stored as .xlsx, so a repeated upload replaces
// the saved document instead of appending a duplicate. An exact-extension
// counterpart is updated in place (stable id, no window without the file); a
// conversion-equivalent one is deleted and uploaded afresh so the server
// converts the body again. Extra duplicates are collapsed either way.
func (c *Client) UploadToFolderReplacing(ctx context.Context, folderID, localPath string) (*FileEntry, []int, error) {
stem := UploadStemFromLocal(localPath)
ext := UploadExtFromLocal(localPath)
deleted, err := c.DeleteFilesByDedupKey(ctx, folderID, stem, ext)
files, err := c.FolderFiles(ctx, folderID)
if err != nil {
return nil, deleted, err
return nil, nil, err
}
ent, err := c.UploadToFolder(ctx, folderID, localPath)
return ent, deleted, err
plan := planUploadReplacement(files, stem, ext)
if plan.UpdateID != "" {
ent, err := c.UpdateFile(ctx, plan.UpdateID, localPath)
if err == nil {
deleted, derr := c.deleteReplacementStale(ctx, plan.DeleteIDs)
return ent, deleted, derr
}
// Portal rejected the in-place update: delete + fresh upload still
// leaves one file (server-converted).
deleted, derr := c.DeleteFilesByStemExt(ctx, folderID, stem, ext)
if derr != nil {
return nil, deleted, derr
}
ent, uerr := c.UploadToFolder(ctx, folderID, localPath)
return ent, deleted, uerr
}
// Conversion-equivalent (or no counterpart): remove the stale files first so
// the fresh upload is converted and exactly one file remains.
deleted, derr := c.deleteReplacementStale(ctx, plan.DeleteIDs)
if derr != nil {
return nil, deleted, derr
}
ent, uerr := c.UploadToFolder(ctx, folderID, localPath)
return ent, deleted, uerr
}
// deleteReplacementStale removes the ids collected by planUploadReplacement:
// duplicate files after an in-place update, or every stale counterpart before a
// converted reupload.
func (c *Client) deleteReplacementStale(ctx context.Context, ids []int) ([]int, error) {
if len(ids) == 0 {
return nil, nil
}
if err := c.DeleteFiles(ctx, ids); err != nil {
return ids, err
}
return ids, nil
}
+2 -2
View File
@@ -325,7 +325,7 @@ func (c *Client) deleteJSON(ctx context.Context, path string, body any) (json.Ra
return nil, err
}
if resp.StatusCode >= 400 {
return nil, fmt.Errorf("DELETE %s: %d %s", path, resp.StatusCode, truncate(string(raw), 400))
return nil, statusError(resp.StatusCode, retryAfterOf(resp), "DELETE %s: %d %s", path, resp.StatusCode, truncate(string(raw), 400))
}
return raw, nil
}
@@ -365,7 +365,7 @@ func (c *Client) uploadReader(ctx context.Context, path, fieldName, fileName str
return nil, err
}
if resp.StatusCode >= 400 {
return nil, fmt.Errorf("upload %s: %d %s", path, resp.StatusCode, truncate(string(raw), 400))
return nil, statusError(resp.StatusCode, retryAfterOf(resp), "upload %s: %d %s", path, resp.StatusCode, truncate(string(raw), 400))
}
return raw, nil
}
@@ -10,10 +10,11 @@ import (
"time"
)
// TestIntegrationFileStores runs the same operation set (create folder, upload,
// list, stat, download, move, copy, rename, delete) through the REST and DAV
// FileStore adapters against a throwaway project Documents folder. Destructive
// — only run against instances you own.
// TestIntegrationFileStores runs the same operation set through the REST and
// DAV FileStore adapters against a throwaway project Documents folder: file
// create/upload/list/stat/download/move/copy/rename/delete and folder
// create/stat/list/rename/move/delete. Destructive — only run against
// instances you own.
//
// The Documents fileops API is asynchronous: a move/copy/delete is accepted
// immediately and becomes visible a moment later, so effects are polled.
@@ -102,10 +103,7 @@ func testFileStoreOps(t *testing.T, ctx context.Context, c *Client, store FileSt
t.Fatalf("moved file %s not in dst", up.ID)
}
if err := store.Copy(ctx, []string{up.ID}, src.ID); err != nil {
t.Fatalf("Copy: %v", err)
}
copied := waitOtherFile(ctx, store, src.ID, up.ID, 20*time.Second)
copied := copyEventually(t, ctx, store, up.ID, src.ID, 20*time.Second)
if copied == nil {
t.Fatalf("no copy found in src after Copy")
}
@@ -122,6 +120,67 @@ func testFileStoreOps(t *testing.T, ctx context.Context, c *Client, store FileSt
if !waitNoEntry(ctx, store, src.ID, copied.ID, 20*time.Second) {
t.Fatalf("copy %s still present in src after delete", copied.ID)
}
// --- CRUD on the folders themselves, reusing the throwaway src/dst ---
// A child file lets us prove it survives the folder rename and move.
child, err := store.Upload(ctx, src.ID, "child-"+suffix+".txt", bytes.NewReader(content))
if err != nil {
t.Fatalf("Upload child: %v", err)
}
if !waitEntry(ctx, store, src.ID, child.ID, 15*time.Second) {
t.Fatalf("child %s not listed in src", child.ID)
}
fst, err := store.Stat(ctx, src.ID)
if err != nil {
t.Fatalf("Stat(folder): %v", err)
}
if fst.ID != src.ID || fst.Kind != Folder {
t.Fatalf("Stat(folder) = %+v", fst)
}
flist, err := store.List(ctx, src.ID)
if err != nil {
t.Fatalf("List(folder): %v", err)
}
if entryByID(flist, child.ID) == nil {
t.Fatalf("child %s not in List(src)", child.ID)
}
folderTitle := "renamed-folder-" + suffix
renameEventually(t, ctx, store, src.ID, folderTitle)
if e, err := store.Stat(ctx, src.ID); err != nil {
t.Fatalf("Stat(folder) after rename: %v", err)
} else if e.Kind != Folder || e.Title != folderTitle {
t.Fatalf("folder after rename = %+v, want title %q", e, folderTitle)
}
moveEventually(t, ctx, store, src.ID, dst.ID)
if !waitEntry(ctx, store, dst.ID, src.ID, 20*time.Second) {
t.Fatalf("moved folder %s not in dst %s", src.ID, dst.ID)
}
if !waitNoEntry(ctx, store, root, src.ID, 20*time.Second) {
t.Fatalf("folder %s still in root after move", src.ID)
}
if !waitEntry(ctx, store, src.ID, child.ID, 20*time.Second) {
t.Fatalf("child file %s lost after moving folder %s", child.ID, src.ID)
}
if err := store.Delete(ctx, []string{child.ID}); err != nil {
t.Fatalf("Delete(child): %v", err)
}
if err := store.Delete(ctx, []string{src.ID}); err != nil {
t.Fatalf("Delete(folder): %v", err)
}
if !waitNoEntry(ctx, store, dst.ID, src.ID, 20*time.Second) {
t.Fatalf("folder %s still present in dst after delete", src.ID)
}
if err := store.Delete(ctx, []string{dst.ID}); err != nil {
t.Fatalf("Delete(dst folder): %v", err)
}
if !waitNoEntry(ctx, store, root, dst.ID, 20*time.Second) {
t.Fatalf("dst folder %s still present in root after delete", dst.ID)
}
}
// moveEventually issues Move and retries while the operation is not visible yet
@@ -141,6 +200,23 @@ func moveEventually(t *testing.T, ctx context.Context, store FileStore, id, dstI
t.Fatalf("Move %s -> %s: %v", id, dstID, lastErr)
}
// copyEventually issues Copy and retries while the new copy is not visible yet
// (copy is accepted asynchronously, like move).
func copyEventually(t *testing.T, ctx context.Context, store FileStore, id, dstID string, d time.Duration) *Entry {
t.Helper()
var lastErr error
for attempt := 0; attempt < 5; attempt++ {
if lastErr = store.Copy(ctx, []string{id}, dstID); lastErr == nil {
if e := waitOtherFile(ctx, store, dstID, id, d); e != nil {
return e
}
}
time.Sleep(time.Second)
}
t.Fatalf("Copy %s -> %s: %v", id, dstID, lastErr)
return nil
}
func renameEventually(t *testing.T, ctx context.Context, store FileStore, id, title string) {
t.Helper()
var lastErr error
+1 -1
View File
@@ -115,7 +115,7 @@ type Searcher interface {
// returned store surfaces that error on every operation instead of returning
// nil. Use SQLFileStore when the open error itself is needed. Unknown or empty
// names select the REST backend. The composed facade (backend
// selection/fallback) lives on FileClient in file_facade.go.
// selection/fallback) lives on FileClient in filestore_facade.go.
func (c *Client) FileStore(backend string) FileStore {
switch strings.ToLower(strings.TrimSpace(backend)) {
case ProviderDAV, "webdav":
View File
+1 -1
View File
@@ -23,7 +23,7 @@ import (
)
// The canonical model (Kind, Entry, SearchQuery, SearchHit, Searcher) lives in
// file_core.go (F1 #35).
// filestore_core.go (F1 #35).
const (
defaultESIndex = "files_file"
@@ -52,7 +52,7 @@ func TestIntegrationESTextIndex(t *testing.T) {
token := "gotes" + stamp
doc := TextDoc{
ID: "3578",
Title: "2026-07-28-S1021-Edelweiss-rechnung.pdf",
Title: "2026-07-28-S1021-acme-rechnung.pdf",
FolderID: "634",
Ext: "pdf",
Content: "Begleitzettel SGB XI — Rechnung " + token,
@@ -78,7 +78,7 @@ func TestParseESTextResponse(t *testing.T) {
{
"_id": "3578",
"_score": 3.21,
"_source": {"id": "3578", "title": "2026-07-28-S1021-Edelweiss-rechnung.pdf", "folder": "634", "ext": "pdf"},
"_source": {"id": "3578", "title": "2026-07-28-S1021-acme-rechnung.pdf", "folder": "634", "ext": "pdf"},
"highlight": {"content": ["Begleitzettel … <em>S1021</em> …"]}
}
]
@@ -92,7 +92,7 @@ func TestParseESTextResponse(t *testing.T) {
t.Fatalf("hits = %d, want 1", len(hits))
}
h := hits[0]
if h.ID != "3578" || h.Title != "2026-07-28-S1021-Edelweiss-rechnung.pdf" || h.Kind != File {
if h.ID != "3578" || h.Title != "2026-07-28-S1021-acme-rechnung.pdf" || h.Kind != File {
t.Errorf("entry = %+v", h.Entry)
}
if h.ParentID != "634" || !reflect.DeepEqual(h.Path, []string{"634"}) {
+1 -1
View File
@@ -14,7 +14,7 @@ import (
)
// ProviderES is the composed Elasticsearch searcher. The SQL store owns
// ProviderPG/ProviderMySQL (file_pg.go); the facade references ProviderPG in
// ProviderPG/ProviderMySQL (filestore_pg.go); the facade references ProviderPG in
// readOrder.
const ProviderES = "elasticsearch"
+131
View File
@@ -0,0 +1,131 @@
//go:build integration
package onlyoffice
import (
"bytes"
"context"
"strconv"
"testing"
"time"
)
// TestIntegrationFacadeCRUD drives the whole operation set through the composed
// facade c.Files(): folder create, upload, stat, list, rename, move, copy,
// delete. Writes must go to REST (the default writeOrder), reads follow
// readOrder (REST when no SQL backend is registered) and every returned Entry
// must report its provider. Destructive — throwaway project, cleaned up.
func TestIntegrationFacadeCRUD(t *testing.T) {
c := liveClient(t)
t.Cleanup(func() { cleanupTestProjects(t, c) })
ctx := context.Background()
suffix := time.Now().UTC().Format("20060102-150405")
project, err := c.CreateProject(NewProjectRequest{
Title: testProjectPrefix + "facade-" + suffix,
Description: "go-onlyoffice facade CRUD integration",
})
if err != nil {
t.Fatalf("CreateProject: %v", err)
}
if project.ID == nil {
t.Fatal("created project without id")
}
root, err := c.projectFolderID(ctx, strconv.Itoa(*project.ID))
if err != nil {
t.Fatalf("projectFolderID: %v", err)
}
f := c.Files()
if got := f.Write().Name(); got != ProviderREST {
t.Fatalf("Write().Name() = %q, want %q", got, ProviderREST)
}
if got := f.Read().Name(); got != ProviderREST {
t.Fatalf("Read().Name() = %q, want %q (no SQL backend registered)", got, ProviderREST)
}
src, err := f.CreateFolder(ctx, root, "facade-src-"+suffix)
if err != nil {
t.Fatalf("CreateFolder src: %v", err)
}
if src.Kind != Folder || src.ID == "" {
t.Fatalf("created src folder: %+v", src)
}
if src.Provider != ProviderREST {
t.Fatalf("CreateFolder provider = %q, want %q", src.Provider, ProviderREST)
}
dst, err := f.CreateFolder(ctx, root, "facade-dst-"+suffix)
if err != nil {
t.Fatalf("CreateFolder dst: %v", err)
}
if dst.Provider != ProviderREST {
t.Fatalf("CreateFolder dst provider = %q, want %q", dst.Provider, ProviderREST)
}
t.Cleanup(func() {
if err := c.DeleteDavItems(ctx, []string{src.ID, dst.ID}, nil); err != nil {
t.Logf("cleanup folders: %v", err)
}
})
content := []byte("facade crud " + suffix + "\n")
up, err := f.Upload(ctx, src.ID, "facade-doc-"+suffix+".txt", bytes.NewReader(content))
if err != nil {
t.Fatalf("Upload: %v", err)
}
if up.Kind != File || up.ID == "" {
t.Fatalf("uploaded entry: %+v", up)
}
if up.Provider != ProviderREST {
t.Fatalf("Upload provider = %q, want %q (write order REST first)", up.Provider, ProviderREST)
}
if !waitEntry(ctx, f, src.ID, up.ID, 15*time.Second) {
t.Fatalf("uploaded %s not listed in src", up.ID)
}
st, err := f.Stat(ctx, up.ID)
if err != nil {
t.Fatalf("Stat: %v", err)
}
if st.ID != up.ID || st.Kind != File {
t.Fatalf("Stat = %+v", st)
}
if st.Provider != ProviderREST {
t.Fatalf("Stat provider = %q, want %q (read order REST)", st.Provider, ProviderREST)
}
list, err := f.List(ctx, src.ID)
if err != nil {
t.Fatalf("List: %v", err)
}
if e := entryByID(list, up.ID); e == nil {
t.Fatalf("uploaded %s not in List(src)", up.ID)
} else if e.Provider != ProviderREST {
t.Fatalf("List provider = %q, want %q", e.Provider, ProviderREST)
}
renamed := "facade-renamed-" + suffix + ".txt"
renameEventually(t, ctx, f, up.ID, renamed)
moveEventually(t, ctx, f, up.ID, dst.ID)
if !waitEntry(ctx, f, dst.ID, up.ID, 20*time.Second) {
t.Fatalf("moved file %s not in dst", up.ID)
}
copied := copyEventually(t, ctx, f, up.ID, src.ID, 20*time.Second)
if copied == nil {
t.Fatalf("no copy found in src after Copy")
}
if copied.Provider != ProviderREST {
t.Fatalf("Copy provider = %q, want %q", copied.Provider, ProviderREST)
}
if err := f.Delete(ctx, []string{up.ID, copied.ID}); err != nil {
t.Fatalf("Delete: %v", err)
}
if !waitNoEntry(ctx, f, dst.ID, up.ID, 20*time.Second) {
t.Fatalf("file %s still present in dst after delete", up.ID)
}
if !waitNoEntry(ctx, f, src.ID, copied.ID, 20*time.Second) {
t.Fatalf("copy %s still present in src after delete", copied.ID)
}
}
@@ -250,6 +250,36 @@ func TestFileClientMySQLStoreIsPreferredForReads(t *testing.T) {
}
}
// TestFileClientWriteToReadOnlyStore guarantees the facade surfaces ErrReadOnly
// when the configured write backend is the read-only SQL store.
func TestFileClientWriteToReadOnlyStore(t *testing.T) {
pg := &pgStore{driver: ProviderPG}
f := newFacadeTestClient(
map[string]FileStore{ProviderREST: &fakeStore{name: ProviderREST}, ProviderPG: pg},
[]string{ProviderPG, ProviderREST},
[]string{ProviderPG, ProviderREST},
)
ctx := context.Background()
if _, err := f.CreateFolder(ctx, "1", "x"); !errors.Is(err, ErrReadOnly) {
t.Errorf("CreateFolder err = %v", err)
}
if _, err := f.Upload(ctx, "1", "x", strings.NewReader("x")); !errors.Is(err, ErrReadOnly) {
t.Errorf("Upload err = %v", err)
}
if err := f.Move(ctx, []string{"1"}, "2"); !errors.Is(err, ErrReadOnly) {
t.Errorf("Move err = %v", err)
}
if err := f.Copy(ctx, []string{"1"}, "2"); !errors.Is(err, ErrReadOnly) {
t.Errorf("Copy err = %v", err)
}
if err := f.Rename(ctx, "1", "x"); !errors.Is(err, ErrReadOnly) {
t.Errorf("Rename err = %v", err)
}
if err := f.Delete(ctx, []string{"1"}); !errors.Is(err, ErrReadOnly) {
t.Errorf("Delete err = %v", err)
}
}
func TestFileClientSearchSelection(t *testing.T) {
f := &FileClient{searchers: map[string]Searcher{}, searchOrder: []string{ProviderES}}
_, err := f.Search()
View File
+67 -41
View File
@@ -7,12 +7,9 @@ package onlyoffice
import (
"context"
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"strconv"
"strings"
)
// restStore is a FileStore over the REST Documents API.
@@ -39,11 +36,25 @@ func (s *restStore) List(ctx context.Context, parentID string) ([]Entry, error)
return out, err
}
// Stat returns file metadata. The REST adapter resolves files only; folders
// are listed by their parent (use List).
// Stat returns file or folder metadata. Folders are resolved through the
// listing endpoint (their own id appears as the listing's Current); other ids
// fall back to the file metadata API.
func (s *restStore) Stat(ctx context.Context, id string) (Entry, error) {
return s.stat(ctx, id)
}
// stat resolves a single id to a folder or file Entry.
func (s *restStore) stat(ctx context.Context, id string) (Entry, error) {
var out Entry
err := retryStoreOp(ctx, func() error {
if l, err := s.c.ListDavFolder(ctx, id); err == nil {
if l != nil && l.Current.ID != "" && l.Current.ID == id {
out = DavFolderToEntry(l.Current, ProviderREST)
return nil
}
} else if Transient(err) {
return err
}
f, err := s.c.GetFile(ctx, id)
if err != nil {
return err
@@ -124,56 +135,84 @@ func (s *restStore) Download(ctx context.Context, id string, w io.Writer) (int64
return n, err
}
// Move moves file ids into parentID. The REST MoveFiles endpoint handles files
// only; folder moves are not exposed by this adapter.
// Move moves folders and/or files into parentID. Ids are classified through
// stat so folder moves use folderIds and file moves use fileIds on the shared
// fileops/move endpoint.
func (s *restStore) Move(ctx context.Context, ids []string, parentID string) error {
dest, err := strconv.Atoi(strings.TrimSpace(parentID))
if err != nil {
return fmt.Errorf("onlyoffice: rest store: move: non-numeric destination folder id %q", parentID)
}
fileIDs, err := numericIDs(ids)
folders, files, err := s.split(ctx, ids)
if err != nil {
return err
}
return retryStoreOp(ctx, func() error {
_, err := s.c.MoveFiles(ctx, dest, fileIDs)
return err
})
}
// Copy copies file ids into parentID. files.go has no copy method, so the
// shared REST fileops copy endpoint (CopyDavItems) is used.
func (s *restStore) Copy(ctx context.Context, ids []string, parentID string) error {
if len(ids) == 0 {
if len(folders) == 0 && len(files) == 0 {
return nil
}
return retryStoreOp(ctx, func() error {
return s.c.CopyDavItems(ctx, nil, ids, parentID)
return s.c.MoveDavItems(ctx, folders, files, parentID)
})
}
// Rename sets a new title (including extension) for a file.
// Copy copies folders and/or files into parentID. files.go has no copy method,
// so the shared REST fileops copy endpoint (CopyDavItems) is used.
func (s *restStore) Copy(ctx context.Context, ids []string, parentID string) error {
folders, files, err := s.split(ctx, ids)
if err != nil {
return err
}
if len(folders) == 0 && len(files) == 0 {
return nil
}
return retryStoreOp(ctx, func() error {
return s.c.CopyDavItems(ctx, folders, files, parentID)
})
}
// Rename sets a new title (including extension) for a file or folder.
func (s *restStore) Rename(ctx context.Context, id, title string) error {
e, err := s.stat(ctx, id)
if err != nil {
return err
}
if e.Kind == Folder {
return retryStoreOp(ctx, func() error {
return s.c.RenameDavFolder(ctx, id, title)
})
}
return retryStoreOp(ctx, func() error {
_, err := s.c.RenameFile(ctx, id, title)
return err
})
}
// Delete permanently deletes file ids.
// Delete permanently deletes folders and/or files.
func (s *restStore) Delete(ctx context.Context, ids []string) error {
fileIDs, err := numericIDs(ids)
folders, files, err := s.split(ctx, ids)
if err != nil {
return err
}
if len(fileIDs) == 0 {
if len(folders) == 0 && len(files) == 0 {
return nil
}
return retryStoreOp(ctx, func() error {
return s.c.DeleteFiles(ctx, fileIDs)
return s.c.DeleteDavItems(ctx, folders, files)
})
}
// split classifies ids into folder and file id lists.
func (s *restStore) split(ctx context.Context, ids []string) (folders, files []string, err error) {
for _, id := range ids {
e, err := s.stat(ctx, id)
if err != nil {
return nil, nil, err
}
if e.Kind == Folder {
folders = append(folders, id)
} else {
files = append(files, id)
}
}
return folders, files, nil
}
// entriesFromFolderMap converts a ListFolder response map into canonical
// entries, reusing the DavFile/DavFolder decoders for robust size handling.
func entriesFromFolderMap(m map[string]any, provider string) ([]Entry, error) {
@@ -215,16 +254,3 @@ func folderEntryFromMap(m map[string]any, parentID, provider string) (Entry, err
e = DavFolderToEntry(f, provider)
return e, nil
}
// numericIDs parses Documents numeric ids from strings.
func numericIDs(ids []string) ([]int, error) {
out := make([]int, 0, len(ids))
for _, id := range ids {
n, err := strconv.Atoi(strings.TrimSpace(id))
if err != nil {
return nil, fmt.Errorf("onlyoffice: rest store: non-numeric id %q", id)
}
out = append(out, n)
}
return out, nil
}
+73 -30
View File
@@ -111,6 +111,25 @@ func (c *Client) deleteObject(ctx context.Context, path string) (map[string]any,
return unmarshalResponseObject(raw)
}
// unmarshalResponseArray extracts the "response" field from a raw OnlyOffice
// envelope and decodes it into a list of maps. Returns (nil, nil) for a null,
// empty or scalar payload. Companion to unmarshalResponseObject for endpoints
// whose response is a list (project team, people/status, …).
func unmarshalResponseArray(raw json.RawMessage) ([]map[string]any, error) {
resp, err := responseField(raw, "response")
if err != nil {
return nil, err
}
if len(resp) == 0 || string(resp) == "null" || resp[0] != '[' {
return nil, nil
}
var list []map[string]any
if err := json.Unmarshal(resp, &list); err != nil {
return nil, err
}
return list, nil
}
// unmarshalResponseObject extracts the "response" field from a raw OnlyOffice
// envelope and decodes it into map[string]any. Returns (nil, nil) for a null
// response, an empty array, or scalar payloads. When the API returns a list
@@ -171,7 +190,7 @@ func (c *Client) getJSONOnce(ctx context.Context, path string) (json.RawMessage,
return nil, err
}
if resp.StatusCode >= 400 {
return nil, fmt.Errorf("GET %s: %d %s", path, resp.StatusCode, truncate(string(raw), 400))
return nil, statusError(resp.StatusCode, retryAfterOf(resp), "GET %s: %d %s", path, resp.StatusCode, truncate(string(raw), 400))
}
return raw, nil
}
@@ -219,7 +238,7 @@ func (c *Client) formRequestOnce(ctx context.Context, method, path string, field
return nil, err
}
if resp.StatusCode >= 400 {
return nil, fmt.Errorf("%s form %s: %d %s", method, path, resp.StatusCode, truncate(string(raw), 400))
return nil, statusError(resp.StatusCode, retryAfterOf(resp), "%s form %s: %d %s", method, path, resp.StatusCode, truncate(string(raw), 400))
}
return raw, nil
}
@@ -250,7 +269,7 @@ func (c *Client) deleteReqOnce(ctx context.Context, path string) (json.RawMessag
return nil, err
}
if resp.StatusCode >= 400 {
return nil, fmt.Errorf("DELETE %s: %d %s", path, resp.StatusCode, truncate(string(raw), 400))
return nil, statusError(resp.StatusCode, retryAfterOf(resp), "DELETE %s: %d %s", path, resp.StatusCode, truncate(string(raw), 400))
}
return raw, nil
}
@@ -273,6 +292,52 @@ func (c *Client) postJSONObject(ctx context.Context, path string, body any) (map
return unmarshalResponseObject(raw)
}
// jsonBodyReader turns a request body value into an io.Reader. nil becomes
// "{}", []byte/string pass through, anything else is JSON-marshalled.
func jsonBodyReader(body any) (io.Reader, error) {
switch b := body.(type) {
case nil:
return strings.NewReader("{}"), nil
case []byte:
return bytes.NewReader(b), nil
case string:
return strings.NewReader(b), nil
default:
buf, err := json.Marshal(b)
if err != nil {
return nil, err
}
return bytes.NewReader(buf), nil
}
}
// postJSONArray is postJSON + unmarshalResponseArray.
func (c *Client) postJSONArray(ctx context.Context, path string, body any) ([]map[string]any, error) {
raw, err := c.postJSON(ctx, path, body)
if err != nil {
return nil, err
}
return unmarshalResponseArray(raw)
}
// putJSONArray is putJSON + unmarshalResponseArray.
func (c *Client) putJSONArray(ctx context.Context, path string, body any) ([]map[string]any, error) {
raw, err := c.putJSON(ctx, path, body)
if err != nil {
return nil, err
}
return unmarshalResponseArray(raw)
}
// deleteJSONArray is deleteJSON + unmarshalResponseArray.
func (c *Client) deleteJSONArray(ctx context.Context, path string, body any) ([]map[string]any, error) {
raw, err := c.deleteJSON(ctx, path, body)
if err != nil {
return nil, err
}
return unmarshalResponseArray(raw)
}
// postJSON issues an authenticated POST with application/json body.
func (c *Client) postJSON(ctx context.Context, path string, body any) (json.RawMessage, error) {
return retryRaw(ctx, func() (json.RawMessage, error) { return c.postJSONOnce(ctx, path, body) })
@@ -283,21 +348,10 @@ func (c *Client) postJSONOnce(ctx context.Context, path string, body any) (json.
if err != nil {
return nil, err
}
var rdr io.Reader
switch b := body.(type) {
case nil:
rdr = strings.NewReader("{}")
case []byte:
rdr = bytes.NewReader(b)
case string:
rdr = strings.NewReader(b)
default:
buf, err := json.Marshal(b)
rdr, err := jsonBodyReader(body)
if err != nil {
return nil, err
}
rdr = bytes.NewReader(buf)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.baseURL()+path, rdr)
if err != nil {
return nil, err
@@ -315,7 +369,7 @@ func (c *Client) postJSONOnce(ctx context.Context, path string, body any) (json.
return nil, err
}
if resp.StatusCode >= 400 {
return nil, fmt.Errorf("POST JSON %s: %d %s", path, resp.StatusCode, truncate(string(raw), 400))
return nil, statusError(resp.StatusCode, retryAfterOf(resp), "POST JSON %s: %d %s", path, resp.StatusCode, truncate(string(raw), 400))
}
return raw, nil
}
@@ -330,21 +384,10 @@ func (c *Client) putJSONOnce(ctx context.Context, path string, body any) (json.R
if err != nil {
return nil, err
}
var rdr io.Reader
switch b := body.(type) {
case nil:
rdr = strings.NewReader("{}")
case []byte:
rdr = bytes.NewReader(b)
case string:
rdr = strings.NewReader(b)
default:
buf, err := json.Marshal(b)
rdr, err := jsonBodyReader(body)
if err != nil {
return nil, err
}
rdr = bytes.NewReader(buf)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPut, c.baseURL()+path, rdr)
if err != nil {
return nil, err
@@ -362,7 +405,7 @@ func (c *Client) putJSONOnce(ctx context.Context, path string, body any) (json.R
return nil, err
}
if resp.StatusCode >= 400 {
return nil, fmt.Errorf("PUT JSON %s: %d %s", path, resp.StatusCode, truncate(string(raw), 400))
return nil, statusError(resp.StatusCode, retryAfterOf(resp), "PUT JSON %s: %d %s", path, resp.StatusCode, truncate(string(raw), 400))
}
return raw, nil
}
@@ -423,7 +466,7 @@ func (c *Client) uploadMultipartOnce(ctx context.Context, method, path, fieldNam
return nil, err
}
if resp.StatusCode >= 400 {
return nil, fmt.Errorf("upload %s: %d %s", path, resp.StatusCode, truncate(string(raw), 400))
return nil, statusError(resp.StatusCode, retryAfterOf(resp), "upload %s: %d %s", path, resp.StatusCode, truncate(string(raw), 400))
}
return raw, nil
}
+23
View File
@@ -48,3 +48,26 @@ func TestUnmarshalResponseObjectNull(t *testing.T) {
t.Fatalf("expected nil, got %#v", out)
}
}
func TestUnmarshalResponseArrayList(t *testing.T) {
raw := json.RawMessage(`{"response":[{"id":"a","displayName":"A"},{"id":"b","displayName":"B"}]}`)
out, err := unmarshalResponseArray(raw)
if err != nil {
t.Fatal(err)
}
if len(out) != 2 || out[0]["id"] != "a" || out[1]["displayName"] != "B" {
t.Fatalf("unexpected list: %#v", out)
}
}
func TestUnmarshalResponseArrayNullAndScalar(t *testing.T) {
for _, raw := range []string{`{"response":null}`, `{"response":{}}`, `{"response":"x"}`} {
out, err := unmarshalResponseArray(json.RawMessage(raw))
if err != nil {
t.Fatalf("%s: %v", raw, err)
}
if out != nil {
t.Fatalf("%s: expected nil, got %#v", raw, out)
}
}
}
+2 -2
View File
@@ -105,10 +105,10 @@ func TestXMLToText(t *testing.T) {
raw := []byte(`<?xml version="1.0" encoding="UTF-8"?>
<rsm:CrossIndustryInvoice><rsm:ExchangedDocument>
<ram:ID>S1063</ram:ID></rsm:ExchangedDocument>
<ram:Name>Edelweiss &amp; Co</ram:Name><ram:GrandTotalAmount>42.00</ram:GrandTotalAmount>
<ram:Name>Acme &amp; Co</ram:Name><ram:GrandTotalAmount>42.00</ram:GrandTotalAmount>
</rsm:CrossIndustryInvoice>`)
got := xmlToText(raw)
for _, want := range []string{"S1063", "Edelweiss & Co", "42.00"} {
for _, want := range []string{"S1063", "Acme & Co", "42.00"} {
if !strings.Contains(got, want) {
t.Errorf("xmlToText missing %q:\n%s", want, got)
}
-393
View File
@@ -1,393 +0,0 @@
package xlspipe
import (
"fmt"
"github.com/xuri/excelize/v2"
)
// Sheet names (Russian tabs) for cutover Portugal workbook.
const (
SheetInputs = "Ввод"
SheetDom6 = "Вс 6.09"
SheetJue3 = "Чт 3.09"
SheetWedHyp = "Ср гип"
SheetSummary = "Сводка"
)
// CutoverPortugalDefaults holds live FO values (2026-08-28).
type CutoverPortugalDefaults struct {
FerryDom6 float64
FerryJue3 float64
FerrySuperiorDelta float64
HousingLow float64
HousingMid float64
HousingHigh float64
DriveLow float64
DriveMid float64
DriveHigh float64
BoardLow float64
BoardMid float64
BoardHigh float64
FoodLow float64
FoodMid float64
FoodHigh float64
SimLow float64
SimMid float64
SimHigh float64
MonthCap float64
ExtraNightsDom6 float64
ExtraNightsJue3 float64
ExtraNightsWed float64
}
// DefaultCutoverPortugal returns FO live snapshot from portugal track (28.08.2026).
func DefaultCutoverPortugal() CutoverPortugalDefaults {
return CutoverPortugalDefaults{
FerryDom6: 457.89,
FerryJue3: 484.09,
FerrySuperiorDelta: 19.64,
HousingLow: 509,
HousingMid: 600,
HousingHigh: 650,
DriveLow: 70,
DriveMid: 85,
DriveHigh: 100,
BoardLow: 40,
BoardMid: 60,
BoardHigh: 80,
FoodLow: 150,
FoodMid: 220,
FoodHigh: 300,
SimLow: 50,
SimMid: 100,
SimHigh: 150,
MonthCap: 2500,
ExtraNightsDom6: 0,
ExtraNightsJue3: 0,
ExtraNightsWed: 4,
}
}
type inputField struct {
name string // defined name (ASCII, for formulas)
label string
value float64
note string
comment string
}
// BuildCutoverPortugalWorkbook creates a multi-sheet cutover budget with Russian labels,
// cell comments on non-obvious inputs, named ranges, and cross-sheet formulas.
func BuildCutoverPortugalWorkbook(d CutoverPortugalDefaults) (*excelize.File, error) {
f := excelize.NewFile()
defaultSheet := f.GetSheetName(0)
if err := f.SetSheetName(defaultSheet, SheetInputs); err != nil {
f.Close()
return nil, err
}
for _, name := range []string{SheetDom6, SheetJue3, SheetWedHyp, SheetSummary} {
if _, err := f.NewSheet(name); err != nil {
f.Close()
return nil, err
}
}
if err := writeInputsSheet(f, d); err != nil {
f.Close()
return nil, err
}
scenarios := []struct {
sheet, ferry, extra, note string
}{
{SheetDom6, "ferry_dom6", "extra_nights_dom6", "Живой слот вс 6.09 20:30; заезд в квартиру вт 8.09"},
{SheetJue3, "ferry_jue3", "extra_nights_jue3", "Живой чт 3.09; T1a 05–12 если TF-крыша кончается раньше вс"},
{SheetWedHyp, "ferry_jue3", "extra_nights_wed", "Гипотеза: ср 2.09 20:00 по тарифу Jue3; заезд пт 4.09"},
}
for _, sc := range scenarios {
if err := writeScenarioSheet(f, sc.sheet, sc.ferry, sc.extra, sc.note); err != nil {
f.Close()
return nil, err
}
}
if err := writeSummarySheet(f); err != nil {
f.Close()
return nil, err
}
f.SetActiveSheet(0)
if err := finalizeWorkbook(f); err != nil {
f.Close()
return nil, err
}
return f, nil
}
func writeInputsSheet(f *excelize.File, d CutoverPortugalDefaults) error {
if err := setHeaders(f, SheetInputs, "Параметр", "Значение €", "Кратко"); err != nil {
return err
}
fields := []inputField{
{
name: "ferry_dom6", label: "Паром вс 6.09 (Básica, без residencia)",
value: d.FerryDom6, note: "FO live: 2взр+младенец+авто",
comment: "Fred Olsen Dom 6.09 20:30 SC→Huelva, прибытие Mar 8 09:00. Butaca Normal/Básica без субсидии канарского residencia (−183€). Меняйте после нового live FO.",
},
{
name: "ferry_jue3", label: "Паром чт 3.09 (Básica, без residencia)",
value: d.FerryJue3, note: "FO live Jue 3 20:00",
comment: "Прямой рейс 35 ч. Дороже вс на ~26€. Используется также для листа «Ср гип», если своего рейса ср 2.09 нет в продаже.",
},
{
name: "ferry_superior_uplift", label: "Доплата VIP / Butaca Superior",
value: d.FerrySuperiorDelta, note: "Superior − Normal (Jue3 live)",
comment: "Разница между Superior и Normal на live Jue3 ≈19,64€. На Dom6 Superior в сессии не перевыбирали — оценка по этой дельте. VIP = salón, не каюта.",
},
{
name: "housing_7n_low", label: "Крыша 7 ночей Setúbal — минимум",
value: d.HousingLow, note: "Airbnb low band",
comment: "Короткая аренда T1a (#11): 08–15.09, 1–2BR с парковкой. Низкая граница live Airbnb Setúbal.",
},
{
name: "housing_7n_mid", label: "Крыша 7 ночей Setúbal — целевой mid",
value: d.HousingMid, note: "Цель #11: 500–650€/нед",
comment: "Рабочая оценка для брони. Основной столбец mid на листах сценариев.",
},
{
name: "housing_7n_high", label: "Крыша 7 ночей Setúbal — максимум",
value: d.HousingHigh, note: "Airbnb high band",
},
{
name: "drive_low", label: "Проезд Huelva → Setúbal — мин",
value: d.DriveLow, note: "OSRM ~3,8 ч",
comment: "≈342 км: топливо + платные дороги. Низкая/средняя/высокая оценка.",
},
{name: "drive_mid", label: "Проезд Huelva → Setúbal — mid", value: d.DriveMid},
{name: "drive_high", label: "Проезд Huelva → Setúbal — макс", value: d.DriveHigh},
{
name: "board_low", label: "Еда на пароме — мин",
value: d.BoardLow, note: "Меню FO",
comment: "Питание на борту (Fred Olsen). George 0–3 обычно бесплатно как пассажир — еда отдельно.",
},
{name: "board_mid", label: "Еда на пароме — mid", value: d.BoardMid},
{name: "board_high", label: "Еда на пароме — макс", value: d.BoardHigh},
{
name: "food_7d_low", label: "Еда 7 дней в PT — мин",
value: d.FoodLow, note: "Готовим в apt",
comment: "Первая неделя в Setúbal после парома — продукты, не рестораны.",
},
{name: "food_7d_mid", label: "Еда 7 дней в PT — mid", value: d.FoodMid},
{name: "food_7d_high", label: "Еда 7 дней в PT — макс", value: d.FoodHigh},
{
name: "sim_low", label: "SIM + документы — мин",
value: d.SimLow, note: "Разовые cutover",
comment: "eSIM, копии, мелкие госпошлины при cutover. Не включает депозит аренды.",
},
{name: "sim_mid", label: "SIM + документы — mid", value: d.SimMid},
{name: "sim_high", label: "SIM + документы — макс", value: d.SimHigh},
{
name: "month_cap", label: "Потолок бюджета на месяц (€)",
value: d.MonthCap, note: "SoT: 2500€",
comment: "Жёсткий потолок Sep из source-of-truth. «Остаток» = потолок − итого mid сценария.",
},
{
name: "extra_nights_dom6", label: "Лишние ночи в PT (вс 6.09)",
value: d.ExtraNightsDom6, note: "0 = TF до вс",
comment: "Платные ночи в PT до начала 7-дневной крыши. Для Dom6 обычно 0: остаёмся на Тенерифе до вс, заезд вт 8.09.",
},
{
name: "extra_nights_jue3", label: "Лишние ночи в PT (чт 3.09)",
value: d.ExtraNightsJue3, note: "0 если TF до вс",
comment: "Если крыша TF кончается раньше вс — нужны ночи 05–07.09 до Airbnb. По умолчанию 0.",
},
{
name: "extra_nights_wed", label: "Лишние ночи в PT (ср гип)",
value: d.ExtraNightsWed, note: "Гип: заезд пт 4.09",
comment: "Гипотетический слот ср 2.09 → заезд пт 4.09 = 4 лишних ночи до типичного 7н блока. Рейса ср в FO нет — тариф как Jue3.",
},
}
for i, fld := range fields {
row := i + 2
if err := f.SetCellStr(SheetInputs, fmt.Sprintf("A%d", row), fld.label); err != nil {
return err
}
if err := defineInput(f, SheetInputs, fld.name, row, fld.value, fld.note, fld.comment); err != nil {
return err
}
}
if err := f.SetCellStr(SheetInputs, "A25", "—"); err != nil {
return err
}
if err := f.SetCellStr(SheetInputs, "B25", "Редактируйте жёлтые ячейки"); err != nil {
return err
}
if err := f.SetCellStr(SheetInputs, "C25", "Формулы на листах сценариев и «Сводка» пересчитаются в OnlyOffice"); err != nil {
return err
}
return styleSheet(f, SheetInputs, SheetInputs)
}
func writeScenarioSheet(f *excelize.File, sheet, ferryName, extraNightsName, scenarioNote string) error {
if err := setHeaders(f, sheet, "Статья", "Мин €", "Mid €", "Макс €", "Пояснение"); err != nil {
return err
}
if err := f.SetCellStr(sheet, "A2", "Паром Básica"); err != nil {
return err
}
for col, ref := range []string{ferryName, ferryName, ferryName} {
cell, _ := excelize.CoordinatesToCellName(col+2, 2)
if err := f.SetCellFormula(sheet, cell, "="+ref); err != nil {
return err
}
}
if err := addCellComment(f, sheet, "A2", "Тариф парома для этого сценария. Берётся с листа «Ввод»."); err != nil {
return err
}
lines := []struct {
label string
low, mid, high, note string
comment string
}{
{"Крыша 7 н Setúbal", "housing_7n_low", "housing_7n_mid", "housing_7n_high", "Airbnb T1a #11", ""},
{"Huelva → Setúbal", "drive_low", "drive_mid", "drive_high", "OSRM ~3,8 ч", ""},
{"Еда на борту", "board_low", "board_mid", "board_high", "Меню FO", ""},
{"Еда 7 д в PT", "food_7d_low", "food_7d_mid", "food_7d_high", "Готовим дома", ""},
{"SIM / документы", "sim_low", "sim_mid", "sim_high", "Cutover", ""},
}
for i, ln := range lines {
row := i + 3
if err := setFormulaRow(f, sheet, row, ln.label,
"="+ln.low, "="+ln.mid, "="+ln.high, ln.note); err != nil {
return err
}
}
if err := f.SetCellStr(sheet, "A8", "Итого Básica"); err != nil {
return err
}
for col := 2; col <= 4; col++ {
cell, _ := excelize.CoordinatesToCellName(col, 8)
colL, _ := excelize.CoordinatesToCellName(col, 2)
colH, _ := excelize.CoordinatesToCellName(col, 7)
if err := f.SetCellFormula(sheet, cell, fmt.Sprintf("=SUM(%s:%s)", colL, colH)); err != nil {
return err
}
}
if err := f.SetCellStr(sheet, "E8", "SUM строк 2–7"); err != nil {
return err
}
if err := setFormulaRow(f, sheet, 9, "Итого Superior",
"=B8+ferry_superior_uplift", "=C8+ferry_superior_uplift", "=D8+ferry_superior_uplift",
"Básica + VIP"); err != nil {
return err
}
if err := addCellComment(f, sheet, "A9", "Butaca Superior / VIP salón. Доплата с листа «Ввод»."); err != nil {
return err
}
if err := f.SetCellStr(sheet, "A10", "Лишние ночи PT"); err != nil {
return err
}
for col, housing := range []string{"housing_7n_low", "housing_7n_mid", "housing_7n_high"} {
cell, _ := excelize.CoordinatesToCellName(col+2, 10)
formula := fmt.Sprintf("=%s*%s/7", extraNightsName, housing)
if err := f.SetCellFormula(sheet, cell, formula); err != nil {
return err
}
}
if err := f.SetCellStr(sheet, "E10", "ночей × (крыша/7)"); err != nil {
return err
}
if err := addCellComment(f, sheet, "A10", "Платное жильё до начала 7-дневной брони. Число ночей — на листе «Ввод» для этого сценария."); err != nil {
return err
}
if err := setFormulaRow(f, sheet, 11, "Итого с ночами",
"=B8+B10", "=C8+C10", "=D8+D10", scenarioNote); err != nil {
return err
}
if err := f.SetCellStr(sheet, "A12", "Остаток от потолка"); err != nil {
return err
}
for _, col := range []string{"B", "C", "D"} {
if err := f.SetCellFormula(sheet, col+"12", "=month_cap-C11"); err != nil {
return err
}
}
if err := f.SetCellStr(sheet, "E12", "потолок − mid итого"); err != nil {
return err
}
if err := addCellComment(f, sheet, "C12", "Сколько остаётся от месячного потолка 2500€ после cutover (mid)."); err != nil {
return err
}
if err := f.SetCellStr(sheet, "A13", "Среднее по строкам"); err != nil {
return err
}
for col := 2; col <= 4; col++ {
cell, _ := excelize.CoordinatesToCellName(col, 13)
colL, _ := excelize.CoordinatesToCellName(col, 2)
colH, _ := excelize.CoordinatesToCellName(col, 7)
if err := f.SetCellFormula(sheet, cell, fmt.Sprintf("=AVERAGE(%s:%s)", colL, colH)); err != nil {
return err
}
}
if err := f.SetCellStr(sheet, "E13", "AVG статей 2–7"); err != nil {
return err
}
return styleSheet(f, sheet, SheetInputs)
}
func writeSummarySheet(f *excelize.File) error {
if err := setHeaders(f, SheetSummary,
"Сценарий", "Básica mid", "Superior mid", "Итого mid", "Остаток", "Δ vs вс"); err != nil {
return err
}
rows := []struct {
label, sheet string
}{
{"Вс 6.09 (live)", SheetDom6},
{"Чт 3.09 (live)", SheetJue3},
{"Ср 2.09 (гипотеза)", SheetWedHyp},
}
qs := quoteSheet
for i, r := range rows {
row := i + 2
if err := f.SetCellStr(SheetSummary, fmt.Sprintf("A%d", row), r.label); err != nil {
return err
}
pairs := []struct {
col int
ref string
}{
{2, fmt.Sprintf("%s!C8", qs(r.sheet))},
{3, fmt.Sprintf("%s!C9", qs(r.sheet))},
{4, fmt.Sprintf("%s!C11", qs(r.sheet))},
{5, fmt.Sprintf("%s!C12", qs(r.sheet))},
}
for _, p := range pairs {
cell, _ := excelize.CoordinatesToCellName(p.col, row)
if err := f.SetCellFormula(SheetSummary, cell, "="+p.ref); err != nil {
return err
}
}
cell, _ := excelize.CoordinatesToCellName(6, row)
if err := f.SetCellFormula(SheetSummary, cell, fmt.Sprintf("=D%d-$D$2", row)); err != nil {
return err
}
}
if err := f.SetCellStr(SheetSummary, "A6", "Потолок месяца"); err != nil {
return err
}
if err := f.SetCellFormula(SheetSummary, "D6", "=month_cap"); err != nil {
return err
}
if err := f.SetCellStr(SheetSummary, "A7", "Лучший итого mid"); err != nil {
return err
}
if err := f.SetCellFormula(SheetSummary, "D7", "=MIN(D2:D4)"); err != nil {
return err
}
if err := f.SetCellStr(SheetSummary, "E7", "MIN по сценариям"); err != nil {
return err
}
if err := addCellComment(f, SheetSummary, "D7", "Минимальный mid «Итого с ночами» среди трёх сценариев. Сейчас обычно вс 6.09."); err != nil {
return err
}
return styleSheet(f, SheetSummary, SheetInputs)
}
-68
View File
@@ -1,68 +0,0 @@
package xlspipe
import (
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"github.com/xuri/excelize/v2"
)
func parseCalcFloat(s string) (float64, error) {
s = strings.ReplaceAll(s, ",", "")
s = strings.TrimSpace(s)
var val float64
_, err := fmt.Sscan(s, &val)
return val, err
}
func TestBuildCutoverPortugalWorkbook_Formulas(t *testing.T) {
f, err := BuildCutoverPortugalWorkbook(DefaultCutoverPortugal())
if err != nil {
t.Fatal(err)
}
defer f.Close()
dir := t.TempDir()
path := filepath.Join(dir, "cutover.xlsx")
if err := Save(f, path); err != nil {
t.Fatal(err)
}
st, err := os.Stat(path)
if err != nil || st.Size() < 4096 {
t.Fatalf("xlsx too small: %v", err)
}
opened, err := excelize.OpenFile(path)
if err != nil {
t.Fatal(err)
}
defer opened.Close()
cases := []struct {
sheet, cell string
want float64
tol float64
}{
{SheetDom6, "C8", 1522.89, 0.02},
{SheetJue3, "C8", 1549.09, 0.02},
{SheetWedHyp, "C11", 1891.95, 0.05},
{SheetSummary, "D2", 1522.89, 0.02},
{SheetSummary, "D7", 1522.89, 0.02},
}
for _, tc := range cases {
got, err := opened.CalcCellValue(tc.sheet, tc.cell)
if err != nil {
t.Fatalf("%s!%s calc: %v", tc.sheet, tc.cell, err)
}
val, err := parseCalcFloat(got)
if err != nil {
t.Fatalf("%s!%s parse %q: %v", tc.sheet, tc.cell, got, err)
}
if diff := val - tc.want; diff < -tc.tol || diff > tc.tol {
t.Fatalf("%s!%s = %v want ~%v", tc.sheet, tc.cell, val, tc.want)
}
}
}
-26
View File
@@ -1,26 +0,0 @@
package xlspipe
import (
"fmt"
"strings"
"github.com/xuri/excelize/v2"
)
// Template names for oo docs put-xlsx --template.
const (
TemplateCutoverPortugal = "cutover-portugal"
)
// BuildTemplate returns an xlsx workbook for a known template name.
func BuildTemplate(name string) (*File, error) {
switch strings.ToLower(strings.TrimSpace(name)) {
case TemplateCutoverPortugal, "portugal-cutover", "cutover":
return BuildCutoverPortugalWorkbook(DefaultCutoverPortugal())
default:
return nil, fmt.Errorf("unknown xlsx template %q (try: %s)", name, TemplateCutoverPortugal)
}
}
// File is an alias so cmd/oo can refer to excelize.File without importing excelize in every handler.
type File = excelize.File
-135
View File
@@ -1,135 +0,0 @@
package xlspipe
import (
"fmt"
"os"
"path/filepath"
"strings"
"github.com/xuri/excelize/v2"
)
const commentAuthor = "oo"
// Save writes the workbook to path (creates parent dirs).
func Save(f *excelize.File, path string) error {
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return err
}
return f.SaveAs(path)
}
// quoteSheet returns an Excel sheet reference safe for formulas ('Name'!A1).
func quoteSheet(name string) string {
return "'" + strings.ReplaceAll(name, "'", "''") + "'"
}
// defineInput registers a named range on sheet!B{row}, sets value, note, optional comment.
func defineInput(f *excelize.File, sheet, name string, row int, value float64, note, comment string) error {
cell := fmt.Sprintf("B%d", row)
if err := f.SetCellFloat(sheet, cell, value, 2, 64); err != nil {
return err
}
if note != "" {
if err := f.SetCellStr(sheet, fmt.Sprintf("C%d", row), note); err != nil {
return err
}
}
ref := fmt.Sprintf("%s!$B$%d", quoteSheet(sheet), row)
if err := f.SetDefinedName(&excelize.DefinedName{Name: name, RefersTo: ref}); err != nil {
return err
}
if comment != "" {
return addCellComment(f, sheet, cell, comment)
}
return nil
}
func addCellComment(f *excelize.File, sheet, cell, text string) error {
return f.AddComment(sheet, excelize.Comment{
Cell: cell,
Author: commentAuthor,
Text: text,
Width: 280,
Height: 120,
})
}
func setHeaders(f *excelize.File, sheet string, headers ...string) error {
for i, h := range headers {
cell, err := excelize.CoordinatesToCellName(i+1, 1)
if err != nil {
return err
}
if err := f.SetCellStr(sheet, cell, h); err != nil {
return err
}
}
return nil
}
func setFormulaRow(f *excelize.File, sheet string, row int, label string, low, mid, high, note string) error {
if err := f.SetCellStr(sheet, fmt.Sprintf("A%d", row), label); err != nil {
return err
}
for col, formula := range []string{low, mid, high} {
cell, err := excelize.CoordinatesToCellName(col+2, row)
if err != nil {
return err
}
if err := f.SetCellFormula(sheet, cell, formula); err != nil {
return err
}
}
if note != "" {
return f.SetCellStr(sheet, fmt.Sprintf("E%d", row), note)
}
return nil
}
func styleSheet(f *excelize.File, sheet, inputsSheet string) error {
widths := map[string]float64{"A": 34, "B": 12, "C": 12, "D": 12, "E": 40}
for col, w := range widths {
if err := f.SetColWidth(sheet, col, col, w); err != nil {
return err
}
}
headerStyle, err := f.NewStyle(&excelize.Style{
Font: &excelize.Font{Bold: true},
Fill: excelize.Fill{Type: "pattern", Color: []string{"#E8F0FE"}, Pattern: 1},
Alignment: &excelize.Alignment{Horizontal: "center", WrapText: true},
})
if err != nil {
return err
}
euroStyle, err := f.NewStyle(&excelize.Style{NumFmt: 4})
if err != nil {
return err
}
inputStyle, err := f.NewStyle(&excelize.Style{
NumFmt: 4,
Fill: excelize.Fill{Type: "pattern", Color: []string{"#FFF9E6"}, Pattern: 1},
})
if err != nil {
return err
}
if err := f.SetCellStyle(sheet, "A1", "E1", headerStyle); err != nil {
return err
}
if sheet == inputsSheet {
return f.SetCellStyle(sheet, "B2", "B30", inputStyle)
}
if err := f.SetCellStyle(sheet, "B2", "D20", euroStyle); err != nil {
return err
}
return nil
}
func finalizeWorkbook(f *excelize.File) error {
if err := f.SetCalcProps(&excelize.CalcPropsOptions{FullCalcOnLoad: boolPtr(true)}); err != nil {
return err
}
return f.UpdateLinkedValue()
}
func boolPtr(v bool) *bool { return &v }
+23 -12
View File
@@ -131,7 +131,7 @@ type UpdateInvoiceParams struct {
// UpdateInvoice PUTs a full invoice body (OnlyOffice requires complete payload).
//
// Linking an opportunity via EntityID on an existing invoice often returns HTTP 400
// on this portal — prefer CreateInvoice with EntityID set. See docs/crm-associations.md.
// on this portal — prefer CreateInvoice with EntityID set. See the CRM association rules.
func (c *Client) UpdateInvoice(ctx context.Context, id string, p UpdateInvoiceParams) (map[string]any, error) {
inv, err := c.GetInvoice(ctx, id)
if err != nil {
@@ -249,7 +249,7 @@ const (
// SetInvoiceStatus sets CRM invoice status for one or more invoice ids
// (PUT /api/2.0/crm/invoice/status/{statusId} with invoiceids).
// Note: Billed→Draft often does not stick; recreate Draft instead (see docs/crm-associations.md).
// Note: Billed→Draft often does not stick; recreate Draft instead (see the CRM association rules).
func (c *Client) SetInvoiceStatus(ctx context.Context, statusID int, invoiceIDs ...int64) (map[string]any, error) {
if statusID <= 0 {
return nil, fmt.Errorf("status id is required")
@@ -443,21 +443,32 @@ func (c *Client) DeleteInvoiceItem(ctx context.Context, id string) (map[string]a
return c.deleteObject(ctx, fmt.Sprintf("/api/2.0/crm/invoiceitem/%s.json", url.PathEscape(id)))
}
// contactAddress is the JSON payload OO stores in a ContactInfo row of type
// Address (ASC.Api.CRM.Wrappers.Address).
type contactAddress struct {
Street string `json:"street"`
City string `json:"city"`
State string `json:"state"`
Zip string `json:"zip"`
Country string `json:"country"`
}
// AddContactAddress attaches a postal address to a contact.
// category: Home|Postal|Office|Billing|Other|Work (or numeric string).
// category: Home|Postal|Office|Billing|Other|Work.
//
// OO stores addresses as ContactInfo rows of infoType Address whose `data` is
// the Address object as JSON (the dedicated /addressdata endpoint binds the
// model from the body and is not accepted by all builds). The generic
// /contact/{id}/data endpoint is the one `oo contacts info-add` uses.
func (c *Client) AddContactAddress(ctx context.Context, contactID, street, city, state, zip, country, category string, isPrimary bool) (map[string]any, error) {
if category == "" {
category = "Billing"
}
fields := url.Values{}
fields.Set("street", street)
fields.Set("city", city)
fields.Set("state", state)
fields.Set("zip", zip)
fields.Set("country", country)
fields.Set("category", category)
fields.Set("isPrimary", strconv.FormatBool(isPrimary))
return c.postFormObject(ctx, fmt.Sprintf("/api/2.0/crm/contact/%s/address", url.PathEscape(contactID)), fields)
payload, err := json.Marshal(contactAddress{Street: street, City: city, State: state, Zip: zip, Country: country})
if err != nil {
return nil, err
}
return c.AddContactInfo(ctx, contactID, "Address", string(payload), category, isPrimary)
}
// UpdateCompany updates company name and optional about text.
+47
View File
@@ -0,0 +1,47 @@
package onlyoffice
// Deep links to OnlyOffice portal objects.
//
// Used by tools that embed per-file links of the form
// /Products/Files/DocEditor.aspx?fileid=<id> in a cover document or a chat
// message. Those links must be consistent so they match the file ids returned
// by `oo projects files list` / `oo link`.
//
// Caveat: file ids are server-assigned and a re-upload/delete yields a NEW id,
// so an already-shared link can go stale. Use `oo projects files replace-in`
// (keeps the id stable) and, if a legacy link must keep working, an nginx alias
// can 302 the old fileid to the new one.
import (
"fmt"
"net/url"
"strings"
)
// FileEditorURL builds the OnlyOffice DocEditor deep link for a portal file id:
//
// https://<portal>/Products/Files/DocEditor.aspx?fileid=<id>
//
// portalBase may include a trailing slash; fileID is trimmed and URL-escaped.
func FileEditorURL(portalBase, fileID string) string {
base := strings.TrimRight(strings.TrimSpace(portalBase), "/")
return fmt.Sprintf("%s/Products/Files/DocEditor.aspx?fileid=%s",
base, url.QueryEscape(strings.TrimSpace(fileID)))
}
// FileEditorURL is the client-bound convenience wrapper (uses the portal URL
// the client was built with).
func (c *Client) FileEditorURL(fileID string) string {
return FileEditorURL(c.baseURL(), fileID)
}
// FolderURL builds a Documents-folder deep link for a folder id.
func FolderURL(portalBase, folderID string) string {
base := strings.TrimRight(strings.TrimSpace(portalBase), "/")
return fmt.Sprintf("%s/Products/Files/Default.aspx#folder=%s", base, url.QueryEscape(strings.TrimSpace(folderID)))
}
// FolderURL is the client-bound convenience wrapper.
func (c *Client) FolderURL(folderID string) string {
return FolderURL(c.baseURL(), folderID)
}
+24
View File
@@ -0,0 +1,24 @@
package onlyoffice
import "testing"
func TestFileEditorURL(t *testing.T) {
cases := []struct{ base, id, want string }{
{"https://office.example.com", "3651", "https://office.example.com/Products/Files/DocEditor.aspx?fileid=3651"},
{"https://office.example.com/", " 1234 ", "https://office.example.com/Products/Files/DocEditor.aspx?fileid=1234"},
{"http://localhost:8087", "a/b", "http://localhost:8087/Products/Files/DocEditor.aspx?fileid=a%2Fb"},
}
for _, c := range cases {
if got := FileEditorURL(c.base, c.id); got != c.want {
t.Fatalf("FileEditorURL(%q,%q) = %q, want %q", c.base, c.id, got, c.want)
}
}
}
func TestFolderURL(t *testing.T) {
got := FolderURL("https://office.example.com/", "495")
want := "https://office.example.com/Products/Files/Default.aspx#folder=495"
if got != want {
t.Fatalf("FolderURL = %q, want %q", got, want)
}
}
+3 -3
View File
@@ -125,7 +125,7 @@ func (c *Client) DownloadMailAttachment(ctx context.Context, attachmentID string
return nil, err
}
if resp.StatusCode >= 400 {
return nil, fmt.Errorf("DownloadMailAttachment %s: %d %s", id, resp.StatusCode, truncate(string(raw), 400))
return nil, statusError(resp.StatusCode, retryAfterOf(resp), "DownloadMailAttachment %s: %d %s", id, resp.StatusCode, truncate(string(raw), 400))
}
return raw, nil
}
@@ -161,7 +161,7 @@ func (c *Client) RemoveMailMessages(ctx context.Context, ids ...int) (map[string
// Id 0 creates a new draft. Body is HTML; the API field name is "body" (not htmlBody).
type SaveMailDraftParams struct {
ID int64 // 0 = create
From string // mailbox address, e.g. eslider@gmail.com
From string // mailbox address, e.g. user@example.com
To string // comma-separated or single address
Cc string
Bcc string
@@ -372,7 +372,7 @@ func mailMessagesPath(f MailMessagesFilter, page, count int) string {
// ParseMailAddress splits a RFC 5322 mailbox string into display name and email.
// Examples:
// - `"LinkedIn" <a@b.com>` → name LinkedIn, address a@b.com
// - `eslider@gmail.com` → address only
// - `user@example.com` → address only
func ParseMailAddress(raw string) (name, address string) {
raw = strings.TrimSpace(raw)
if raw == "" {
+1 -1
View File
@@ -56,7 +56,7 @@ func TestParseMailAddress(t *testing.T) {
}{
{`"LinkedIn Jobbenachrichtigungen" <jobalerts-noreply@linkedin.com>`, "LinkedIn Jobbenachrichtigungen", "jobalerts-noreply@linkedin.com"},
{`"Bitfinex" <no-reply@bitfinex.com>`, "Bitfinex", "no-reply@bitfinex.com"},
{"eslider@gmail.com", "", "eslider@gmail.com"},
{"user@example.com", "", "user@example.com"},
{`"Glassdoor-Jobs" <noreply@glassdoor.com>`, "Glassdoor-Jobs", "noreply@glassdoor.com"},
{"", "", ""},
}
+39
View File
@@ -0,0 +1,39 @@
package onlyoffice
// Project team (portal users) CRUD. Team is distinct from CRM contacts,
// which are linked through project_contacts.go.
import (
"context"
"fmt"
)
// ListProjectTeam returns portal users on the project team.
// GET /api/2.0/project/{projectid}/team
func (c *Client) ListProjectTeam(ctx context.Context, projectID int) ([]map[string]any, error) {
return c.GetProjectTeam(ctx, projectID)
}
// AddProjectTeamUser adds a portal user to the project team.
// POST /api/2.0/project/{projectid}/team with {"userId": "..."}.
// Returns the resulting team list.
func (c *Client) AddProjectTeamUser(ctx context.Context, projectID int, userID string) ([]map[string]any, error) {
return c.postJSONArray(ctx, fmt.Sprintf("/api/2.0/project/%d/team", projectID),
map[string]any{"userId": userID})
}
// RemoveProjectTeamUser removes a portal user from the project team.
// DELETE /api/2.0/project/{projectid}/team with {"userId": "..."}.
// Returns the resulting team list.
func (c *Client) RemoveProjectTeamUser(ctx context.Context, projectID int, userID string) ([]map[string]any, error) {
return c.deleteJSONArray(ctx, fmt.Sprintf("/api/2.0/project/%d/team", projectID),
map[string]any{"userId": userID})
}
// SetProjectTeam replaces the project team with the given user IDs.
// PUT /api/2.0/project/{projectid}/team with participants + notify.
// This is the canonical way to register several members at once.
func (c *Client) SetProjectTeam(ctx context.Context, projectID int, participants []string, notify bool) ([]map[string]any, error) {
return c.putJSONArray(ctx, fmt.Sprintf("/api/2.0/project/%d/team", projectID),
map[string]any{"projectId": projectID, "participants": participants, "notify": notify})
}
+210
View File
@@ -0,0 +1,210 @@
package onlyoffice
// Client-side pacing and the process-wide 429 gate. A token bucket applies to
// every HTTP request a *Client makes (via pacedTransport), so listing, folder
// creation, uploads and even `get project` share one budget and cannot burst
// past openresty. A typed transient answer carrying Retry-After opens the
// cooldown gate for the whole process.
import (
"context"
"net/http"
"os"
"strconv"
"strings"
"sync"
"time"
)
// Default pacing: ~4 requests/second with a single-request burst. This keeps
// bulk syncs under openresty's threshold without noticeably slowing one-shot
// tools.
const (
defaultRateLimit = 4.0
defaultBurst = 1
)
// rateLimiter is a deterministic token bucket. A nil *rateLimiter is a no-op,
// which is how OO_RATE_LIMIT=0 disables pacing.
type rateLimiter struct {
mu sync.Mutex
rate float64
burst float64
tokens float64
last time.Time
}
// newRateLimiter returns a limiter at rate requests/second with the given
// burst. rate <= 0 disables pacing and returns nil.
func newRateLimiter(rate float64, burst int) *rateLimiter {
if rate <= 0 {
return nil
}
if burst < 1 {
burst = 1
}
return &rateLimiter{
rate: rate,
burst: float64(burst),
tokens: float64(burst),
last: time.Now(),
}
}
// wait consumes one token, blocking until one is available or ctx is done.
func (l *rateLimiter) wait(ctx context.Context) error {
if l == nil {
return nil
}
for {
l.mu.Lock()
now := time.Now()
l.tokens += now.Sub(l.last).Seconds() * l.rate
if l.tokens > l.burst {
l.tokens = l.burst
}
l.last = now
if l.tokens >= 1 {
l.tokens--
l.mu.Unlock()
return nil
}
need := time.Duration((1 - l.tokens) / l.rate * float64(time.Second))
l.mu.Unlock()
if need < time.Millisecond {
need = time.Millisecond
}
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(need):
}
}
}
// cooldownGate is the process-wide 429 gate: after a throttled answer every
// request waits until the gate opens, so concurrent and sequential callers do
// not hammer the server.
type cooldownGate struct {
mu sync.Mutex
until time.Time
}
// note extends the gate to at least now+d.
func (g *cooldownGate) note(d time.Duration) {
if d <= 0 {
return
}
g.mu.Lock()
if u := time.Now().Add(d); u.After(g.until) {
g.until = u
}
g.mu.Unlock()
}
// wait blocks until the gate opens or ctx is done.
func (g *cooldownGate) wait(ctx context.Context) error {
g.mu.Lock()
d := time.Until(g.until)
g.mu.Unlock()
if d <= 0 {
return nil
}
t := time.NewTimer(d)
defer t.Stop()
select {
case <-ctx.Done():
return ctx.Err()
case <-t.C:
return nil
}
}
var (
globalLimiter *rateLimiter
globalLimiterOn bool
globalLimiterMu sync.Mutex
globalCooldown = &cooldownGate{}
)
// limiter returns the process-wide rate limiter, initialising it from the
// environment on first use. OO_RATE_LIMIT=0 yields a nil (disabled) limiter.
func limiter() *rateLimiter {
globalLimiterMu.Lock()
defer globalLimiterMu.Unlock()
if !globalLimiterOn {
globalLimiter = newRateLimiterFromEnv()
globalLimiterOn = true
}
return globalLimiter
}
// newRateLimiterFromEnv builds a limiter from OO_RATE_LIMIT (requests/second,
// default 4; 0 disables) and OO_BURST (default 1). Malformed values fall back
// to the defaults.
func newRateLimiterFromEnv() *rateLimiter {
rate := defaultRateLimit
if v, ok := os.LookupEnv("OO_RATE_LIMIT"); ok {
if f, err := strconv.ParseFloat(strings.TrimSpace(v), 64); err == nil {
rate = f
}
}
burst := defaultBurst
if v, ok := os.LookupEnv("OO_BURST"); ok {
if n, err := strconv.Atoi(strings.TrimSpace(v)); err == nil {
burst = n
}
}
return newRateLimiter(rate, burst)
}
// pacedTransport applies the process-wide cooldown gate and rate limiter to
// every request, and arms the gate on a 429. Installed by NewClient, so all
// HTTP paths — typed Query, the http.go helpers, auth and WebDAV — are paced.
type pacedTransport struct {
base http.RoundTripper
}
func (t *pacedTransport) RoundTrip(req *http.Request) (*http.Response, error) {
if err := globalCooldown.wait(req.Context()); err != nil {
return nil, err
}
if err := limiter().wait(req.Context()); err != nil {
return nil, err
}
resp, err := t.base.RoundTrip(req)
if err != nil {
return nil, err
}
if resp.StatusCode == http.StatusTooManyRequests {
globalCooldown.note(retryAfterOf(resp))
}
return resp, nil
}
// envInt reads a positive integer env var, falling back to def.
func envInt(name string, def int) int {
v, ok := os.LookupEnv(name)
if !ok {
return def
}
n, err := strconv.Atoi(strings.TrimSpace(v))
if err != nil || n <= 0 {
return def
}
return n
}
// envDuration reads a duration env var (Go syntax, e.g. "2s", "2m"),
// falling back to def.
func envDuration(name string, def time.Duration) time.Duration {
v, ok := os.LookupEnv(name)
if !ok {
return def
}
d, err := time.ParseDuration(strings.TrimSpace(v))
if err != nil || d <= 0 {
return def
}
return d
}
+160
View File
@@ -0,0 +1,160 @@
package onlyoffice
import (
"context"
"net/http"
"net/http/httptest"
"os"
"testing"
"time"
)
// TestMain disables the process-wide rate limiter for the suite so existing
// fake-server tests stay fast; the pacing tests configure their own limiters.
func TestMain(m *testing.M) {
_ = os.Setenv("OO_RATE_LIMIT", "0")
os.Exit(m.Run())
}
// resetPacingForTest restores the package-level limiter and cooldown gate to a
// clean state. White-box helper for tests that arm the global 429 gate.
func resetPacingForTest() {
globalLimiterMu.Lock()
globalLimiter = nil
globalLimiterOn = false
globalLimiterMu.Unlock()
globalCooldown = &cooldownGate{}
}
func TestNewClientUsesPacedTransport(t *testing.T) {
c := NewClient(Credentials{Url: "https://example.test"})
if _, ok := c.client.Transport.(*pacedTransport); !ok {
t.Fatalf("transport = %T, want *pacedTransport", c.client.Transport)
}
}
func TestNewRateLimiterFromEnv(t *testing.T) {
t.Setenv("OO_RATE_LIMIT", "")
t.Setenv("OO_BURST", "")
if l := newRateLimiterFromEnv(); l == nil || l.rate != defaultRateLimit || l.burst != defaultBurst {
t.Fatalf("defaults: got %+v, want rate=%v burst=%d", l, defaultRateLimit, defaultBurst)
}
t.Setenv("OO_RATE_LIMIT", "0")
if l := newRateLimiterFromEnv(); l != nil {
t.Fatalf("OO_RATE_LIMIT=0: got %+v, want nil (off)", l)
}
t.Setenv("OO_RATE_LIMIT", "10")
t.Setenv("OO_BURST", "3")
if l := newRateLimiterFromEnv(); l == nil || l.rate != 10 || l.burst != 3 {
t.Fatalf("explicit: got %+v, want rate=10 burst=3", l)
}
}
func TestNilRateLimiterIsNoOp(t *testing.T) {
var l *rateLimiter
start := time.Now()
if err := l.wait(context.Background()); err != nil {
t.Fatalf("nil limiter wait: %v", err)
}
if elapsed := time.Since(start); elapsed > 20*time.Millisecond {
t.Fatalf("nil limiter blocked for %v", elapsed)
}
}
func TestRateLimiterPacesBurst(t *testing.T) {
l := newRateLimiter(50, 1) // 20ms per token, no burst headroom
start := time.Now()
for i := 0; i < 5; i++ {
if err := l.wait(context.Background()); err != nil {
t.Fatalf("wait %d: %v", i, err)
}
}
// 5 calls with burst 1 -> 4 gaps * 20ms = 80ms.
if elapsed := time.Since(start); elapsed < 60*time.Millisecond {
t.Fatalf("elapsed %v, want >= 60ms (not paced)", elapsed)
}
}
func TestRateLimiterHonoursContext(t *testing.T) {
l := newRateLimiter(1, 1)
if err := l.wait(context.Background()); err != nil {
t.Fatalf("first wait: %v", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Millisecond)
defer cancel()
if err := l.wait(ctx); err == nil {
t.Fatal("want context error while waiting for a token")
}
}
func TestCooldownGateBlocksAndExpires(t *testing.T) {
g := &cooldownGate{}
g.note(50 * time.Millisecond)
start := time.Now()
if err := g.wait(context.Background()); err != nil {
t.Fatalf("wait: %v", err)
}
if elapsed := time.Since(start); elapsed < 45*time.Millisecond {
t.Fatalf("gate did not block: %v", elapsed)
}
start = time.Now()
if err := g.wait(context.Background()); err != nil {
t.Fatalf("second wait: %v", err)
}
if elapsed := time.Since(start); elapsed > 20*time.Millisecond {
t.Fatalf("expired gate still blocked: %v", elapsed)
}
}
func TestCooldownGateNoteExtendsNotShortens(t *testing.T) {
g := &cooldownGate{}
g.note(80 * time.Millisecond)
g.note(10 * time.Millisecond) // must not shorten the window
if d := time.Until(g.until); d < 70*time.Millisecond {
t.Fatalf("note shortened gate: %v left", d)
}
}
// TestPacedTransportArmsCooldownOn429 verifies that a throttled answer opens the
// process-wide gate and that the next request is blocked for the Retry-After
// window even though it targets a healthy endpoint.
func TestPacedTransportArmsCooldownOn429(t *testing.T) {
resetPacingForTest()
defer resetPacingForTest()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/throttle" {
w.Header().Set("Retry-After", "1")
w.WriteHeader(http.StatusTooManyRequests)
return
}
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte("{}"))
}))
defer srv.Close()
tr := &pacedTransport{base: http.DefaultTransport}
req1, _ := http.NewRequestWithContext(context.Background(), http.MethodGet, srv.URL+"/throttle", nil)
resp1, err := tr.RoundTrip(req1)
if err != nil {
t.Fatalf("throttled round trip: %v", err)
}
resp1.Body.Close()
if resp1.StatusCode != http.StatusTooManyRequests {
t.Fatalf("status = %d, want 429", resp1.StatusCode)
}
start := time.Now()
req2, _ := http.NewRequestWithContext(context.Background(), http.MethodGet, srv.URL+"/ok", nil)
resp2, err := tr.RoundTrip(req2)
if err != nil {
t.Fatalf("second round trip: %v", err)
}
resp2.Body.Close()
if elapsed := time.Since(start); elapsed < 900*time.Millisecond {
t.Fatalf("cooldown did not block next call: %v", elapsed)
}
}
-28
View File
@@ -1,28 +0,0 @@
{
"$schema": "https://raw.githubusercontent.com/googleapis/release-please/main/schemas/config.json",
"release-type": "go",
"bump-minor-pre-major": true,
"bump-patch-for-minor-pre-major": false,
"include-v-in-tag": true,
"include-component-in-tag": false,
"pull-request-title-pattern": "chore${scope}: release ${version}",
"changelog-path": "CHANGELOG.md",
"changelog-sections": [
{ "type": "feat", "section": "Features" },
{ "type": "fix", "section": "Bug Fixes" },
{ "type": "perf", "section": "Performance Improvements" },
{ "type": "revert", "section": "Reverts" },
{ "type": "refactor", "section": "Code Refactoring" },
{ "type": "docs", "section": "Documentation" },
{ "type": "test", "section": "Tests", "hidden": true },
{ "type": "build", "section": "Build System", "hidden": true },
{ "type": "ci", "section": "Continuous Integration", "hidden": true },
{ "type": "chore", "section": "Miscellaneous", "hidden": true },
{ "type": "style", "section": "Styles", "hidden": true }
],
"packages": {
".": {
"package-name": "go-onlyoffice"
}
}
}

Some files were not shown because too many files have changed in this diff Show More