feat(sync): graceful degradation + exponential backoff on provider throttling (429 / Too Many Requests / auth lockout) #360

Closed
opened 2026-07-05 22:23:58 +00:00 by JMR-dev · 0 comments
JMR-dev commented 2026-07-05 22:23:58 +00:00 (Migrated from github.com)

Context

LibreMail is an offline-first Android email client. IMAP is currently connect-per-operation (connection reuse, issue #125, is OFF by default); full-history backfill (issue #12) prefetches message bodies + attachments on unmetered networks (default FetchPolicy.WIFI_ONLY). A perf drilldown found this opens roughly 1 + K + attachments separate authenticated connections per backfill page, saturating a provider's connection/bandwidth budget and starving interactive message-opens. These tickets track making LibreMail's connection/request patterns and data usage respect each major provider's documented limits and degrade gracefully under throttling. (Outlook/Hotmail uses the Microsoft Graph REST API; Gmail, Yahoo/AOL, and iCloud use IMAP/SMTP.)

Problem

There is no shared, provider-aware throttle layer today. Each provider-specific sync/backfill path handles (or fails to handle) rate limiting, connection caps, and lockouts independently, so fixes get duplicated — or missed — per provider.

Optimization levers

  • Build a shared, provider-aware throttle layer used by all provider paths (IMAP and Graph alike).
  • Detect throttling:
    • HTTP 429 + Graph Retry-After header.
    • IMAP [THROTTLED] / "Too many requests" / NO responses.
    • Repeated auth failures.
  • Respond with exponential backoff + jitter, honoring Retry-After when present.
  • Circuit-breaker that stops retrying and backs off for a long window when a provider signals a lockout (e.g. Yahoo's 1-hour auth lockout).
  • Per-account connection/request concurrency caps keyed by provider (Gmail 15 / Yahoo 5 / iCloud 5 / Graph 4 concurrent), with interactive-request priority over backfill so opening a message is never starved behind backfill traffic.
  • PII-free AppLog breadcrumbs on backoff/lockout events (per repo Definition of Done).

Note

This is the umbrella issue referenced by the per-provider connection/throttling-limit issues (Gmail, Yahoo/AOL, iCloud, Outlook/Graph).

## Context LibreMail is an offline-first Android email client. IMAP is currently connect-per-operation (connection reuse, issue #125, is OFF by default); full-history backfill (issue #12) prefetches message bodies + attachments on unmetered networks (default `FetchPolicy.WIFI_ONLY`). A perf drilldown found this opens roughly `1 + K + attachments` separate authenticated connections per backfill page, saturating a provider's connection/bandwidth budget and starving interactive message-opens. These tickets track making LibreMail's connection/request patterns and data usage respect each major provider's documented limits and degrade gracefully under throttling. (Outlook/Hotmail uses the Microsoft Graph REST API; Gmail, Yahoo/AOL, and iCloud use IMAP/SMTP.) ## Problem There is no shared, provider-aware throttle layer today. Each provider-specific sync/backfill path handles (or fails to handle) rate limiting, connection caps, and lockouts independently, so fixes get duplicated — or missed — per provider. ## Optimization levers - Build a shared, provider-aware throttle layer used by all provider paths (IMAP and Graph alike). - Detect throttling: - HTTP 429 + Graph `Retry-After` header. - IMAP `[THROTTLED]` / "Too many requests" / NO responses. - Repeated auth failures. - Respond with exponential backoff + jitter, honoring `Retry-After` when present. - Circuit-breaker that stops retrying and backs off for a long window when a provider signals a lockout (e.g. Yahoo's 1-hour auth lockout). - Per-account connection/request concurrency caps keyed by provider (Gmail 15 / Yahoo 5 / iCloud 5 / Graph 4 concurrent), with **interactive-request priority over backfill** so opening a message is never starved behind backfill traffic. - PII-free `AppLog` breadcrumbs on backoff/lockout events (per repo Definition of Done). ## Note This is the umbrella issue referenced by the per-provider connection/throttling-limit issues (Gmail, Yahoo/AOL, iCloud, Outlook/Graph).
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: JMR-dev/LibreMail#360