The full-history backfill (#12) runs flat-out for the entire session — BackfillWorker
chains bounded slices back-to-back with no cooldown as long as moreWork=true, and on a large
mailbox moreWorknever clears. This keeps the account's IMAP session continuously busy and
is the background load that starves interactive message-opens (see #355). Bound and back it off so
it fills history steadily without saturating the account.
Evidence (physical-device perf run, 2026-07-05)
Pixel 10 Pro XL, real Gmail (PASSWORD_IMAP) over LTE, origin/main 6118b6d.
From logcat_full_nav.txt, backfill ran continuously for the whole ~50-min session:
A new 20-page slice starts within the same second the previous finishes (~85 s/slice), from
14:55 through 15:40+, moreWork=true every time. The six message-opens measured at
15:22:44–15:27:38 (avg 48 s, up to 73.7 s to render the body) each overlapped one of these
slices. Default FetchPolicy.WIFI_ONLY on metered LTE meant this was headers-only paging
(no body prefetch) — yet it was still enough sustained IMAP load to stall interactive fetches.
Root cause (confirmed in code)
BackfillWorker.doWork(): while (mailBackfiller.runBackfill() && !isStopped) { /* next slice */ }
— no inter-slice delay; a single WorkManager run keeps paging until stopped or moreWork=false.
MailBackfiller.runBackfill() returns moreWork=true whenever any folder still has pages
(FolderResult.moreWork = !complete && !stalled), so a large mailbox loops indefinitely within
one worker run.
Inside a slice the only pacing is delay(BACKFILL_BATCH_DELAY_MS = 250 ms) between pages and BACKFILL_BATCH_SIZE = 50 headers/page, DEFAULT_MAX_BATCHES = 20 pages/slice. With
connect-per-operation ImapClient (reuse #125 OFF), that is ~20 fresh CONNECT+TLS+LOGIN
sequences per slice, back-to-back, indefinitely — a continuous per-account login+fetch load.
SyncScheduler.schedulePeriodicBackfill() sets a 30-min periodic cadence, but the
back-to-back inner loop means the work effectively never idles between periods on a big mailbox.
Proposed approach (any subset; each independently reduces the storm)
Inter-slice cooldown: delay() between slices in BackfillWorker (or return after N
slices and lean on the 30-min periodic cadence) so backfill idles instead of running flat-out.
Adaptive batch pacing: raise BACKFILL_BATCH_DELAY_MS, and/or lower DEFAULT_MAX_BATCHES,
when the app is foregrounded / recently interactive; keep it aggressive only when the app is
backgrounded and on unmetered power.
Cap per run: bound slices-per-worker-run so one run can't monopolize the account for the
whole session; the periodic schedule (and backfillNow()) resume later.
Complements #355: #355 pauses backfill during an open; this lowers the steady-state load so an
open that lands mid-page still competes with far less background traffic.
app/src/main/kotlin/org/libremail/data/sync/SyncScheduler.kt (cadence/constraints if tuned)
Definition of done (per repo DoD)
Unit tests: back-to-back slices are separated by the cooldown; per-run slice cap honored;
adaptive pacing picks the slower cadence when interactive/foregrounded.
Instrumented/E2E test that backfill still makes forward progress across runs (history keeps
filling) with the new pacing.
PII-free AppLog for slice cadence / cooldown decisions.
Relates to
Root cause #12 (closed/implemented). Sibling fixes: #355 (interactive priority), #357 (fast
first-open), #358 (observability). See also #322 (backfill persistBatch DB-write perf). Filed
from the 2026-07-05 Pixel 10 Pro XL perf run.
## Summary
The full-history backfill (#12) runs **flat-out for the entire session** — `BackfillWorker`
chains bounded slices back-to-back with no cooldown as long as `moreWork=true`, and on a large
mailbox `moreWork` **never clears**. This keeps the account's IMAP session continuously busy and
is the background load that starves interactive message-opens (see #355). Bound and back it off so
it fills history steadily without saturating the account.
## Evidence (physical-device perf run, 2026-07-05)
Pixel 10 Pro XL, real Gmail (`PASSWORD_IMAP`) over LTE, origin/main `6118b6d`.
From `logcat_full_nav.txt`, backfill ran continuously for the whole ~50-min session:
```
15:22:08 backfill slice done: moreWork=true / backfill slice: maxBatches=20
15:23:39 backfill slice done: moreWork=true / backfill slice: maxBatches=20
15:25:10 backfill slice done: moreWork=true / backfill slice: maxBatches=20
15:26:45 backfill slice done: moreWork=true / backfill slice: maxBatches=20
15:28:17 backfill slice done: moreWork=true / backfill slice: maxBatches=20
```
A new 20-page slice starts within the same second the previous finishes (~85 s/slice), from
14:55 through 15:40+, `moreWork=true` every time. The six message-opens measured at
15:22:44–15:27:38 (avg 48 s, up to 73.7 s to render the body) each overlapped one of these
slices. Default `FetchPolicy.WIFI_ONLY` on metered LTE meant this was **headers-only** paging
(no body prefetch) — yet it was still enough sustained IMAP load to stall interactive fetches.
## Root cause (confirmed in code)
- `BackfillWorker.doWork()`: `while (mailBackfiller.runBackfill() && !isStopped) { /* next slice */ }`
— no inter-slice delay; a single WorkManager run keeps paging until stopped or `moreWork=false`.
- `MailBackfiller.runBackfill()` returns `moreWork=true` whenever any folder still has pages
(`FolderResult.moreWork = !complete && !stalled`), so a large mailbox loops indefinitely within
one worker run.
- Inside a slice the only pacing is `delay(BACKFILL_BATCH_DELAY_MS = 250 ms)` between pages and
`BACKFILL_BATCH_SIZE = 50` headers/page, `DEFAULT_MAX_BATCHES = 20` pages/slice. With
connect-per-operation `ImapClient` (reuse #125 OFF), that is ~20 fresh CONNECT+TLS+LOGIN
sequences per slice, back-to-back, indefinitely — a continuous per-account login+fetch load.
- `SyncScheduler.schedulePeriodicBackfill()` sets a 30-min *periodic* cadence, but the
back-to-back inner loop means the work effectively never idles between periods on a big mailbox.
## Proposed approach (any subset; each independently reduces the storm)
- **Inter-slice cooldown**: `delay()` between slices in `BackfillWorker` (or return after N
slices and lean on the 30-min periodic cadence) so backfill idles instead of running flat-out.
- **Adaptive batch pacing**: raise `BACKFILL_BATCH_DELAY_MS`, and/or lower `DEFAULT_MAX_BATCHES`,
when the app is foregrounded / recently interactive; keep it aggressive only when the app is
backgrounded and on unmetered power.
- **Cap per run**: bound slices-per-worker-run so one run can't monopolize the account for the
whole session; the periodic schedule (and `backfillNow()`) resume later.
- Complements #355: #355 pauses backfill *during* an open; this lowers the steady-state load so an
open that lands mid-page still competes with far less background traffic.
## Affected files
- `app/src/main/kotlin/org/libremail/data/sync/BackfillWorker.kt` (inter-slice cooldown / per-run cap)
- `app/src/main/kotlin/org/libremail/data/sync/MailBackfiller.kt` (`BACKFILL_BATCH_DELAY_MS`, `DEFAULT_MAX_BATCHES`, adaptive pacing)
- `app/src/main/kotlin/org/libremail/data/sync/SyncScheduler.kt` (cadence/constraints if tuned)
## Definition of done (per repo DoD)
- Unit tests: back-to-back slices are separated by the cooldown; per-run slice cap honored;
adaptive pacing picks the slower cadence when interactive/foregrounded.
- Instrumented/E2E test that backfill still makes forward progress across runs (history keeps
filling) with the new pacing.
- PII-free `AppLog` for slice cadence / cooldown decisions.
## Relates to
Root cause #12 (closed/implemented). Sibling fixes: #355 (interactive priority), #357 (fast
first-open), #358 (observability). See also #322 (backfill `persistBatch` DB-write perf). Filed
from the 2026-07-05 Pixel 10 Pro XL perf run.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Summary
The full-history backfill (#12) runs flat-out for the entire session —
BackfillWorkerchains bounded slices back-to-back with no cooldown as long as
moreWork=true, and on a largemailbox
moreWorknever clears. This keeps the account's IMAP session continuously busy andis the background load that starves interactive message-opens (see #355). Bound and back it off so
it fills history steadily without saturating the account.
Evidence (physical-device perf run, 2026-07-05)
Pixel 10 Pro XL, real Gmail (
PASSWORD_IMAP) over LTE, origin/main6118b6d.From
logcat_full_nav.txt, backfill ran continuously for the whole ~50-min session:A new 20-page slice starts within the same second the previous finishes (~85 s/slice), from
14:55 through 15:40+,
moreWork=trueevery time. The six message-opens measured at15:22:44–15:27:38 (avg 48 s, up to 73.7 s to render the body) each overlapped one of these
slices. Default
FetchPolicy.WIFI_ONLYon metered LTE meant this was headers-only paging(no body prefetch) — yet it was still enough sustained IMAP load to stall interactive fetches.
Root cause (confirmed in code)
BackfillWorker.doWork():while (mailBackfiller.runBackfill() && !isStopped) { /* next slice */ }— no inter-slice delay; a single WorkManager run keeps paging until stopped or
moreWork=false.MailBackfiller.runBackfill()returnsmoreWork=truewhenever any folder still has pages(
FolderResult.moreWork = !complete && !stalled), so a large mailbox loops indefinitely withinone worker run.
delay(BACKFILL_BATCH_DELAY_MS = 250 ms)between pages andBACKFILL_BATCH_SIZE = 50headers/page,DEFAULT_MAX_BATCHES = 20pages/slice. Withconnect-per-operation
ImapClient(reuse #125 OFF), that is ~20 fresh CONNECT+TLS+LOGINsequences per slice, back-to-back, indefinitely — a continuous per-account login+fetch load.
SyncScheduler.schedulePeriodicBackfill()sets a 30-min periodic cadence, but theback-to-back inner loop means the work effectively never idles between periods on a big mailbox.
Proposed approach (any subset; each independently reduces the storm)
delay()between slices inBackfillWorker(or return after Nslices and lean on the 30-min periodic cadence) so backfill idles instead of running flat-out.
BACKFILL_BATCH_DELAY_MS, and/or lowerDEFAULT_MAX_BATCHES,when the app is foregrounded / recently interactive; keep it aggressive only when the app is
backgrounded and on unmetered power.
whole session; the periodic schedule (and
backfillNow()) resume later.open that lands mid-page still competes with far less background traffic.
Affected files
app/src/main/kotlin/org/libremail/data/sync/BackfillWorker.kt(inter-slice cooldown / per-run cap)app/src/main/kotlin/org/libremail/data/sync/MailBackfiller.kt(BACKFILL_BATCH_DELAY_MS,DEFAULT_MAX_BATCHES, adaptive pacing)app/src/main/kotlin/org/libremail/data/sync/SyncScheduler.kt(cadence/constraints if tuned)Definition of done (per repo DoD)
adaptive pacing picks the slower cadence when interactive/foregrounded.
filling) with the new pacing.
AppLogfor slice cadence / cooldown decisions.Relates to
Root cause #12 (closed/implemented). Sibling fixes: #355 (interactive priority), #357 (fast
first-open), #358 (observability). See also #322 (backfill
persistBatchDB-write perf). Filedfrom the 2026-07-05 Pixel 10 Pro XL perf run.