feat(sync): add PII-safe AppLog breadcrumbs to the sync engine (#324) #329

Closed
opened 2026-07-05 00:49:27 +00:00 by JMR-dev · 0 comments
JMR-dev commented 2026-07-05 00:49:27 +00:00 (Migrated from github.com)

Part of #324 (strangler-migrate debug logging to AppLog).

Sequencing: PARALLEL with the other migration areas, after the Seam ticket merges
(uses accountLogRef(...) + AppLog.*). This area is net-new breadcrumbs — these files
have no logging today, so there are no raw Log.* sites to migrate. Touches data/sync
(+ optionally data/repository), no collision with siblings.

Why

The diagnostically most valuable subsystem is currently silent: a submitted report shows
nothing about whether sync ran, how much it fetched, or why it was skipped.

Scope (files)

  • app/src/main/kotlin/org/libremail/data/sync/MailSyncer.kt
  • app/src/main/kotlin/org/libremail/data/sync/MailBackfiller.kt
  • app/src/main/kotlin/org/libremail/data/sync/MailPruner.kt
  • app/src/main/kotlin/org/libremail/data/sync/SyncWorker.kt, BackfillWorker.kt, PruneWorker.kt
  • (optional, light) data/repository/MailRepositoryImpl.kt, AccountRepositoryImpl.kt
  • Tests: MailSyncerTest, MailBackfillerTest, MailPrunerTest (GreenMail-backed JVM tests).

Breadcrumbs to add (sync start/end/counts)

  • MailSyncer.syncAll: start AppLog.i(TAG, "sync all: ${accounts.size} accounts"); end
    AppLog.i(TAG, "sync all done: fetched=$total"), or on failure AppLog.w(TAG, "sync all failed", firstError).
  • MailSyncer.syncAccount / syncFolder: AppLog.d(TAG, "sync ${accountLogRef(accountId)} folder=$folderLabel fetched=$n").
  • MailBackfiller.runBackfill: start AppLog.i(TAG, "backfill slice: maxBatches=$maxBatches");
    per folder AppLog.d(TAG, "backfill ${accountLogRef(account.id)} folder=$folderLabel pages=$batches complete=$complete");
    end AppLog.i(TAG, "backfill slice done: moreWork=$moreWork").
  • MailPruner.prune: AppLog.i(TAG, "prune done: removed=$removed").
  • Workers (SyncWorker/BackfillWorker/PruneWorker): on the cache-locked early return,
    AppLog.i(TAG, "<job> deferred: cache locked") (explains a report with no sync activity);
    and log the final Result (success vs retry) at i/w.

PII (hard rule)

  • Account attribution = accountLogRef(account.id) ONLY (never account.email; note
    Account.id embeds the email, so don't log the id either).
  • Counts / page numbers / booleans are safe.
  • Folder names caveat: system folders (INBOX, Sent, Drafts, Trash, Spam/Junk, Archive) are
    safe; a user-created folder/label name could be PII-ish, so log $folderLabel = the folder
    name only for the known system set and a placeholder (e.g. "<folder>") otherwise. Never log
    subjects, senders, addresses, or message bodies.

Test expectation (unit)

  • In MailSyncerTest / MailBackfillerTest / MailPrunerTest, install a real RingLogBuffer
    via AppLog.install(...) and assert the start/end/count breadcrumbs are captured at the right
    level with the expected counts. These suites use GreenMail with a known test address — assert
    that address (and any host) never appears in buffer.snapshot() (no-PII guarantee, incl. the
    folder-name caveat).
  • Per repo DoD, extend a sync E2E to assert a report captured a sync breadcrumb PII-free.

Parallelism: parallel with auth/lock, DB/keystore, connectivity/send, stragglers — after Seam.

Part of #324 (strangler-migrate debug logging to AppLog). **Sequencing: PARALLEL** with the other migration areas, **after the Seam ticket merges** (uses `accountLogRef(...)` + `AppLog.*`). This area is **net-new breadcrumbs** — these files have **no logging today**, so there are no raw `Log.*` sites to migrate. Touches `data/sync` (+ optionally `data/repository`), no collision with siblings. ## Why The diagnostically most valuable subsystem is currently silent: a submitted report shows nothing about whether sync ran, how much it fetched, or why it was skipped. ## Scope (files) - `app/src/main/kotlin/org/libremail/data/sync/MailSyncer.kt` - `app/src/main/kotlin/org/libremail/data/sync/MailBackfiller.kt` - `app/src/main/kotlin/org/libremail/data/sync/MailPruner.kt` - `app/src/main/kotlin/org/libremail/data/sync/SyncWorker.kt`, `BackfillWorker.kt`, `PruneWorker.kt` - (optional, light) `data/repository/MailRepositoryImpl.kt`, `AccountRepositoryImpl.kt` - Tests: `MailSyncerTest`, `MailBackfillerTest`, `MailPrunerTest` (GreenMail-backed JVM tests). ## Breadcrumbs to add (sync start/end/counts) - `MailSyncer.syncAll`: start `AppLog.i(TAG, "sync all: ${accounts.size} accounts")`; end `AppLog.i(TAG, "sync all done: fetched=$total")`, or on failure `AppLog.w(TAG, "sync all failed", firstError)`. - `MailSyncer.syncAccount` / `syncFolder`: `AppLog.d(TAG, "sync ${accountLogRef(accountId)} folder=$folderLabel fetched=$n")`. - `MailBackfiller.runBackfill`: start `AppLog.i(TAG, "backfill slice: maxBatches=$maxBatches")`; per folder `AppLog.d(TAG, "backfill ${accountLogRef(account.id)} folder=$folderLabel pages=$batches complete=$complete")`; end `AppLog.i(TAG, "backfill slice done: moreWork=$moreWork")`. - `MailPruner.prune`: `AppLog.i(TAG, "prune done: removed=$removed")`. - Workers (`SyncWorker`/`BackfillWorker`/`PruneWorker`): on the cache-locked early return, `AppLog.i(TAG, "<job> deferred: cache locked")` (explains a report with no sync activity); and log the final `Result` (success vs retry) at `i`/`w`. ## PII (hard rule) - Account attribution = `accountLogRef(account.id)` ONLY (never `account.email`; note `Account.id` embeds the email, so don't log the id either). - Counts / page numbers / booleans are safe. - **Folder names caveat:** system folders (INBOX, Sent, Drafts, Trash, Spam/Junk, Archive) are safe; a user-created folder/label name could be PII-ish, so log `$folderLabel` = the folder name only for the known system set and a placeholder (e.g. `"<folder>"`) otherwise. Never log subjects, senders, addresses, or message bodies. ## Test expectation (unit) - In `MailSyncerTest` / `MailBackfillerTest` / `MailPrunerTest`, install a real `RingLogBuffer` via `AppLog.install(...)` and assert the start/end/count breadcrumbs are captured at the right level with the expected counts. These suites use GreenMail with a known test address — assert that address (and any host) never appears in `buffer.snapshot()` (no-PII guarantee, incl. the folder-name caveat). - Per repo DoD, extend a sync E2E to assert a report captured a sync breadcrumb PII-free. **Parallelism:** parallel with auth/lock, DB/keystore, connectivity/send, stragglers — after Seam.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: JMR-dev/LibreMail#329