perf(sync): pause background backfill while a message is opening #459

Merged
JMR-dev merged 1 commits from feat-355-pause-backfill-on-open into main 2026-07-08 21:21:04 +00:00
JMR-dev commented 2026-07-08 20:18:28 +00:00 (Migrated from github.com)

Summary

Opening an uncached message stalled ~35-74s (avg 48s) behind the reader spinner because the on-demand IMAP body fetch has no priority over the continuous full-history backfill (#12) and loses the race for the account's IMAP throughput (ImapClient is connect-per-operation, and the interactive fetch shares no in-process lock with backfill). This makes an interactive open pre-empt the background backfill so the body the user is waiting on is fetched first.

Closes #355

What changed

  • New InteractiveImapGate (@Singleton) — a process-wide priority signal mirroring MailMaintenanceGate / AccountThrottleGate. A counter (not a mutex) so overlapping interactive fetches run concurrently and backfill waits for all to clear.
    • withInteractive { } raises an in-flight counter for the block and always lowers it in a finally (a failed fetch can never strand it).
    • awaitInteractiveIdle() suspends until the counter hits zero (StateFlow.first { it == 0 }, no lost-wakeup).
  • MailRepositoryImpl wraps the user-facing IMAP paths in withInteractive { }: openMessage, inlineImages, downloadAttachment, buildReplyDraft. Backfill's own content prefetch now calls ensureAttachmentFile directly so it bypasses the gate (it must not yield to itself; also removes a latent per-part routing re-read).
  • MailBackfiller parks at its natural per-page yield point (yieldToInteractive) while an interactive fetch is active, resuming the instant it clears. This is also the slice's first yield point, so a slice never begins a page while the user waits on a body.
  • PII-free AppLog breadcrumbs (accountLogRef) at the backfill park/resume points (see #358).

How this hooks into #360's framework

Orthogonal and complementary to AccountThrottleGate (#360, merged in #436): that gate makes background work back off after a provider rejects it; this gate makes background work yield to a foreground fetch pre-emptively. MailBackfiller now consults both — the throttle backoff per account, then the interactive gate per page.

Tests

  • InteractiveImapGateTest — counter raise/clear, park-until-release, release-on-throw (no deadlock), multi-fetch hold-until-last, Turbine on the count flow.
  • MailBackfillerTest — backfill parks before its next page while an interactive fetch holds the gate and resumes after (asserts the park/resume breadcrumbs); errored interactive fetch does not strand backfill.
  • MailRepositoryImplTest — openMessage holds the gate for the whole body fetch and releases it after.
  • InteractiveImapGateInstrumentedTest (androidTest) — on-device pause/resume + concurrent-hold + error-release, mock-free, for the CI API matrix.

Gate results (local, JDK 21)

assembleDebug, testDebugUnitTest, jacocoTestCoverageVerification, compileDebugAndroidTestKotlin, lintDebug, ktlintCheck, detekt — all green. Local emulator preflight skipped (flaky/wedges per repo guidance); CI's full matrix E2E is the authoritative gate.

Sequencing note

Touches the same sync surface as siblings #356 (bound backfill), #357 (fast first-open), #358 (observability). The gate is additive (new @Singleton + a per-page yield point), so it should merge cleanly alongside them; the per-page park combines naturally with #356's bounded backfill for the open-lands-mid-page case.

## Summary Opening an uncached message stalled ~35-74s (avg 48s) behind the reader spinner because the on-demand IMAP body fetch has **no priority** over the continuous full-history backfill (#12) and loses the race for the account's IMAP throughput (`ImapClient` is connect-per-operation, and the interactive fetch shares no in-process lock with backfill). This makes an interactive open **pre-empt** the background backfill so the body the user is waiting on is fetched first. `Closes #355` ## What changed - **New `InteractiveImapGate` (`@Singleton`)** — a process-wide priority signal mirroring `MailMaintenanceGate` / `AccountThrottleGate`. A counter (not a mutex) so overlapping interactive fetches run concurrently and backfill waits for *all* to clear. - `withInteractive { }` raises an in-flight counter for the block and **always** lowers it in a `finally` (a failed fetch can never strand it). - `awaitInteractiveIdle()` suspends until the counter hits zero (`StateFlow.first { it == 0 }`, no lost-wakeup). - **`MailRepositoryImpl`** wraps the user-facing IMAP paths in `withInteractive { }`: `openMessage`, `inlineImages`, `downloadAttachment`, `buildReplyDraft`. Backfill's own content prefetch now calls `ensureAttachmentFile` directly so it **bypasses** the gate (it must not yield to itself; also removes a latent per-part routing re-read). - **`MailBackfiller`** parks at its natural per-page yield point (`yieldToInteractive`) while an interactive fetch is active, resuming the instant it clears. This is also the slice's first yield point, so a slice never begins a page while the user waits on a body. - **PII-free `AppLog` breadcrumbs** (`accountLogRef`) at the backfill park/resume points (see #358). ## How this hooks into #360's framework Orthogonal and complementary to `AccountThrottleGate` (#360, merged in #436): that gate makes background work **back off after a provider rejects it**; this gate makes background work **yield to a foreground fetch pre-emptively**. `MailBackfiller` now consults both — the throttle backoff per account, then the interactive gate per page. ## Tests - `InteractiveImapGateTest` — counter raise/clear, park-until-release, **release-on-throw (no deadlock)**, multi-fetch hold-until-last, Turbine on the count flow. - `MailBackfillerTest` — backfill parks before its next page while an interactive fetch holds the gate and resumes after (asserts the park/resume breadcrumbs); errored interactive fetch does not strand backfill. - `MailRepositoryImplTest` — `openMessage` holds the gate for the whole body fetch and releases it after. - `InteractiveImapGateInstrumentedTest` (androidTest) — on-device pause/resume + concurrent-hold + error-release, mock-free, for the CI API matrix. ## Gate results (local, JDK 21) `assembleDebug`, `testDebugUnitTest`, `jacocoTestCoverageVerification`, `compileDebugAndroidTestKotlin`, `lintDebug`, `ktlintCheck`, `detekt` — **all green**. Local emulator preflight skipped (flaky/wedges per repo guidance); CI's full matrix E2E is the authoritative gate. ## Sequencing note Touches the same sync surface as siblings #356 (bound backfill), #357 (fast first-open), #358 (observability). The gate is additive (new `@Singleton` + a per-page yield point), so it should merge cleanly alongside them; the per-page park combines naturally with #356's bounded backfill for the open-lands-mid-page case.
mergify[bot] commented 2026-07-08 20:49:03 +00:00 (Migrated from github.com)

Merge Queue Status

This pull request spent 32 minutes 7 seconds in the queue, including 26 minutes 16 seconds running CI.

Required conditions to merge
<!--- DO NOT EDIT -*- Mergify Payload -*- {"version": 1, "state": "merged", "queue_rule_name": "default", "queued_at": "2026-07-08T20:49:01.555243+00:00", "estimated_time_of_merge": null, "speculative_check_pr": null, "required_conditions": []} -*- Mergify Payload End -*- --> # Merge Queue Status - ✅ **Entered queue** — `2026-07-08 20:49 UTC` · Rule: `default` · triggered by merge protections - ✅ **Checks passed** · on draft #461 - ✅ **Merged** — `2026-07-08 21:21 UTC` · at `52da77b6a0494d1e6f5dfa9f49a1b25ea7b6324d` · merge This pull request spent **32 minutes 7 seconds** in the queue, including **26 minutes 16 seconds** running CI. <details> <summary>Required conditions to merge</summary> - `-conflict` - [X] #459 - `-draft` - [X] #459 - [X] `base = main` - [X] `check-success = CI passed` - `github-review-approved` [🛡 GitHub repository ruleset rule `main`] - [X] #459 - `label != broken` - [X] #459 - [X] any of [🛡 GitHub branch protection]: - [X] `check-success = Debug build` - [ ] `check-neutral = Debug build` - [ ] `check-skipped = Debug build` - [X] any of [🛡 GitHub branch protection]: - [X] `check-success = Unit tests` - [ ] `check-neutral = Unit tests` - [ ] `check-skipped = Unit tests` - [X] any of [🛡 GitHub branch protection]: - [X] `check-success = CI passed` - [ ] `check-neutral = CI passed` - [ ] `check-skipped = CI passed` - [X] any of [🛡 GitHub repository ruleset rule `main`]: - [X] `check-success = @github-actions/CI passed` - [ ] `check-neutral = @github-actions/CI passed` - [ ] `check-skipped = @github-actions/CI passed` </details>
Sign in to join this conversation.