WIP: perf(yahoo): respect Yahoo/AOL IMAP limits & avoid the 1-hour auth lockout #472

Draft
JMR-dev wants to merge 5 commits from feat-362-yahoo-imap-limits into main
JMR-dev commented 2026-07-09 00:40:00 +00:00 (Migrated from github.com)

Closes #362.

What & why

Yahoo/AOL trip an automated ~1-hour service lockout after too many rapid or failed authentication attempts. LibreMail is connect-per-operation-heavy and, critically, the IDLE reconnect loop starts at a 5s backoff — so an unguarded auth failure could fire several failed LOGINs in the first minute and lock a real user out for an hour. This adds a proactive, Yahoo/AOL-scoped auth circuit-breaker that spaces out login attempts so we never reach the lockout. It builds on #360's reactive throttle framework (it does not reinvent it) and composes with #356's backfill pacer.

Design (all host-keyed — only Yahoo/AOL are gated; every other provider is a no-op)

  • ProviderAuthPolicy — per-host AuthCadencePolicy; Yahoo/AOL enabled, everything else DISABLED. Also exposes the documented 5-connection and 10k-folder ceilings.
  • AuthBackoff — pure schedule: exponential equal-jitter ramp up to a failure threshold, then a fixed open-circuit window ("back off long and stop").
  • AuthThrottleGate (@Singleton) — per-account state; onAuthFailure / onAuthSuccess / remainingAuthBlockMillis, PII-free logging.
  • Enforcement in ImapClient — guards every real LOGIN (connect-per-op, reuse connect/reconnect, and the IDLE connection): skips the login (AuthBackoffException) while backing off, arms the gate on an AuthenticationFailedException, clears it on success. A transient (non-auth) connect error never arms the backoff.
  • MailBackfiller skips an auth-blocked account exactly as it skips a reactively-throttled one (#360), so the pacer (#356) never spins a cooldown on it.

Auth-backoff policy (the load-bearing values)

knob value reasoning
base 60s (→ 30s floor after jitter) the 2nd login lands >=30s after the 1st; "rapid" retries (sub-second) are eliminated
ramp cap 15 min bounds a single wait; well under the ~1h lockout
circuit-open threshold 4 consecutive failures a wrong app-password does not fix itself; stop probing after ~3.5 min of spaced attempts
open-circuit window 30 min firm block once we give up; under the ~1h lockout (recovery beats the lockout) yet long enough that Yahoo's short failure window decays -> <=~2 failed logins/hr while open

Worst case (all failing): failed logins at ~0s, ~30s, ~90s, ~210s, then 30-min gaps — never rapid, and no single wait reaches the 1-hour lockout.

⚠️ Please review the lockout values closely

Yahoo's exact trigger (failure count + rolling window) is not publicly documented, so these are deliberately conservative choices, not tuned-to-spec numbers. If Yahoo's trigger were extremely sensitive (e.g. locks after 2 failures in a short window), the initial ramp could in theory still contribute — but the 4-failure circuit + 30-min open window hold us to ~2 failed logins/hr thereafter, which no rolling-window heuristic should read as rapid. I erred toward over-backing-off (recovery beats the lockout) per the "a wrong value locks a real user out for an hour" priority.

One intentional behavior note: for Yahoo/AOL, an open circuit also blocks interactive re-auth (a departure from "interactive is never blocked") — hammering "retry" on a bad Yahoo password is itself a lockout trigger. The circuit self-clears in <=30 min (or instantly on the next successful login).

Connection / folder caps

The 5-connection cap is already satisfied by connection reuse (#125/#357: ~1 warm socket + 1 IDLE per account = 2). The 10k folder truncation is respected for free (backfill stops when the server returns nothing older). Both are exposed as config + asserted in tests; a live per-host semaphore was deliberately deferred to avoid conflicting with #361's connection-cap work.

Parallel-safety (siblings #361/#363/#364 in flight)

Change is additive and Yahoo-scoped. New code is in new files. Shared files touched: ImapClient.kt (inject gate + guard 2 connect sites), MailBackfiller.kt (inject gate + one skip, extracted paramsForBackfill to stay under the complexity gate — behavior-preserving), and 2 test files that construct MailBackfiller (added the new authGate arg). No shared-config or domain-model files changed.

Tests

Pure-schedule, policy-resolution, gate (coroutines-test virtual time incl. composition with #360), GreenMail enforcement (auth-fail arms / transient does not / blocked skips even a correct credential / reuse path), the backfiller skip, and an on-device instrumented gate test.

Validation

Full static CI gate green locally (JDK 21): assembleDebug + testDebugUnitTest + jacocoTestCoverageVerification (coverage floor met) + compileDebugAndroidTestKotlin + lintDebug + ktlintCheck + detekt. Local emulator E2E left to the CI matrix.

Closes #362. ## What & why Yahoo/AOL trip an automated **~1-hour service lockout** after too many rapid or failed authentication attempts. LibreMail is connect-per-operation-heavy and, critically, the IDLE reconnect loop starts at a 5s backoff — so an unguarded auth failure could fire several failed `LOGIN`s in the first minute and lock a **real user** out for an hour. This adds a **proactive, Yahoo/AOL-scoped auth circuit-breaker** that spaces out login attempts so we never reach the lockout. It builds on #360's reactive throttle framework (it does not reinvent it) and composes with #356's backfill pacer. ## Design (all host-keyed — only Yahoo/AOL are gated; every other provider is a no-op) - **`ProviderAuthPolicy`** — per-host `AuthCadencePolicy`; Yahoo/AOL enabled, everything else `DISABLED`. Also exposes the documented **5-connection** and **10k-folder** ceilings. - **`AuthBackoff`** — pure schedule: exponential equal-jitter *ramp* up to a failure threshold, then a fixed *open-circuit* window ("back off long and stop"). - **`AuthThrottleGate`** (`@Singleton`) — per-account state; `onAuthFailure` / `onAuthSuccess` / `remainingAuthBlockMillis`, PII-free logging. - **Enforcement in `ImapClient`** — guards every real `LOGIN` (connect-per-op, reuse connect/reconnect, and the IDLE connection): skips the login (`AuthBackoffException`) while backing off, arms the gate on an `AuthenticationFailedException`, clears it on success. A **transient (non-auth) connect error never arms** the backoff. - **`MailBackfiller`** skips an auth-blocked account exactly as it skips a reactively-throttled one (#360), so the pacer (#356) never spins a cooldown on it. ## Auth-backoff policy (the load-bearing values) | knob | value | reasoning | |---|---|---| | base | **60s** (→ 30s floor after jitter) | the 2nd login lands >=30s after the 1st; "rapid" retries (sub-second) are eliminated | | ramp cap | **15 min** | bounds a single wait; well under the ~1h lockout | | circuit-open threshold | **4 consecutive failures** | a wrong app-password does not fix itself; stop probing after ~3.5 min of spaced attempts | | open-circuit window | **30 min** | firm block once we give up; under the ~1h lockout (recovery beats the lockout) yet long enough that Yahoo's short failure window decays -> <=~2 failed logins/hr while open | Worst case (all failing): failed logins at ~0s, ~30s, ~90s, ~210s, then 30-min gaps — never rapid, and **no single wait reaches the 1-hour lockout**. ## ⚠️ Please review the lockout values closely Yahoo's exact trigger (failure count + rolling window) is **not publicly documented**, so these are deliberately conservative choices, not tuned-to-spec numbers. If Yahoo's trigger were extremely sensitive (e.g. locks after 2 failures in a short window), the initial ramp could in theory still contribute — but the 4-failure circuit + 30-min open window hold us to ~2 failed logins/hr thereafter, which no rolling-window heuristic should read as rapid. I erred toward over-backing-off (recovery beats the lockout) per the "a wrong value locks a real user out for an hour" priority. One intentional behavior note: for Yahoo/AOL, an **open circuit also blocks interactive re-auth** (a departure from "interactive is never blocked") — hammering "retry" on a bad Yahoo password is itself a lockout trigger. The circuit self-clears in <=30 min (or instantly on the next successful login). ## Connection / folder caps The 5-connection cap is already satisfied by connection reuse (#125/#357: ~1 warm socket + 1 IDLE per account = 2). The 10k folder truncation is respected for free (backfill stops when the server returns nothing older). Both are exposed as config + asserted in tests; a live per-host semaphore was deliberately deferred to avoid conflicting with #361's connection-cap work. ## Parallel-safety (siblings #361/#363/#364 in flight) Change is additive and Yahoo-scoped. New code is in new files. **Shared files touched:** `ImapClient.kt` (inject gate + guard 2 connect sites), `MailBackfiller.kt` (inject gate + one skip, extracted `paramsForBackfill` to stay under the complexity gate — behavior-preserving), and 2 test files that construct `MailBackfiller` (added the new `authGate` arg). No shared-config or domain-model files changed. ## Tests Pure-schedule, policy-resolution, gate (coroutines-test virtual time incl. composition with #360), GreenMail enforcement (auth-fail arms / transient does not / blocked skips even a correct credential / reuse path), the backfiller skip, and an on-device instrumented gate test. ## Validation Full static CI gate green locally (JDK 21): `assembleDebug` + `testDebugUnitTest` + `jacocoTestCoverageVerification` (coverage floor met) + `compileDebugAndroidTestKotlin` + `lintDebug` + `ktlintCheck` + `detekt`. Local emulator E2E left to the CI matrix.
This pull request has changes conflicting with the target branch.
  • app/src/androidTest/kotlin/org/libremail/data/local/AccountDataMigratorTest.kt
This pull request is marked as a work in progress.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin feat-362-yahoo-imap-limits:feat-362-yahoo-imap-limits
git checkout feat-362-yahoo-imap-limits
Sign in to join this conversation.