Adds two iCloud-specific, provider-scoped policies that build on the
existing shared throttling framework (#360's AccountThrottleGate, #356's
BackfillPacer) without refactoring either:
- IcloudConnectionLimiter: a per-account permit gate capping an iCloud
account at 5 simultaneous connections (Apple documents 5-8; pinned to
the conservative low end). Wired into MailBackfiller around both
connection-opening call sites (the header-page fetch and the
body/attachment prefetch loop - the "1 + K + attachments" per-page
connection count issue #363 describes). A no-op passthrough for every
other provider, so it composes cleanly with #360's reactive backoff
(already consulted first, per account) and #356's slice pacing
(BackfillWorker composes BackfillPacer.runPaced around
MailBackfiller.runBackfill, so the cap sits one layer beneath the
pacer's cooldown/cap in the same call graph).
- IcloudSendLimits: enforces Apple's ~20 MB outgoing message-size cap
before SmtpSender ever opens a connection, estimating the actual
encoded wire size (base64 inflates binary attachment bytes by ~4/3)
rather than comparing raw file bytes, mirroring GraphSender's existing
pre-send attachment-size guard. An over-cap send throws
MessageTooLargeException, caught by SendWorker's existing runCatching
and turned into a clean, PII-free outbox error - no crash, no raw
provider rejection.
Both are new, self-contained files kept intentionally separate from a
shared cross-provider table, per the parallel-safety note on this ticket
(sibling issues #361/#362/#364 add their own provider's limits the same
way).
Touches two shared files: MailBackfiller.kt (new constructor dependency
+ two call sites wrapped in icloudConnectionLimiter.withPermit) and
SendWorker.kt (one guard call before smtpSender.send). Both are
minimal, additive edits — flagged for conflict-awareness with the
sibling provider tickets.
Also excludes MailBackfillerTest.kt from detekt's LargeClass rule
(config/detekt/detekt.yml), mirroring the existing MailRepositoryImplTest
exclusion: one cohesive single-SUT suite tipped over the LLOC boundary
by the new connection-cap wiring tests.
Closes#363
Add Gmail's documented IMAP caps (15 max simultaneous connections, 2,500 MB/day
download, 500 MB/day upload, 10,000 messages/labels per limit) as provider-scoped
config/policy that feeds the existing #360/#356 pacing machinery instead of
reinventing it:
- GmailSyncLimits: pure constants + `appliesTo(account)` provider detection via the
existing MailProvider.forImapHost lookup (no changes to MailProvider itself).
- GmailBandwidthTracker: a new, per-account/per-day download-byte tracker (mirrors
AccountThrottleGate's shape: ConcurrentHashMap state, injectable clock, PII-free
once-per-crossing AppLog breadcrumb). Proactive and orthogonal to the #360
AccountThrottleGate (which only reacts to a provider-issued throttle) and #356's
BackfillPacer (which paces slice cadence, not bytes) - same relationship
InteractiveImapGate already documents having to AccountThrottleGate.
Wiring: MailRepositoryImpl.prefetchMessage (the single funnel both MailBackfiller
and MailSyncer's background prefetch already share) records bytes actually pulled
over the network for Gmail accounts; MailBackfiller/MailSyncer's prefetchIfEnabled
consult isOverDailyBudget once per account before starting a batch and defer
body/attachment prefetch for the rest of the day once Gmail's budget is reached -
header paging/sync is never gated, and interactive fetches (open, attachment tap,
inline images) are never gated either, matching the existing interactive-priority
principle (#355/#360).
The 15-connection cap is already satisfied by the existing architecture
(ImapConnectionCache keeps one reused connection per account plus one dedicated
IDLE connection - 2 total, well under the cap); GmailSyncLimitsTest pins that
invariant against the documented ceiling so a future change that grows per-account
concurrency trips a test before it could approach Gmail's real limit. The
10k-messages-per-label figure is captured as a documented constant only - it is
deliberately NOT wired into a backfill stop condition, since issue #12's full
history backfill is intentional and a large real mailbox can exceed 10k messages.
AccountThrottleGate, BackfillPacer, ThrottleClassifier/ThrottleSignal/ThrottleBackoff,
InteractiveImapGate, and MailProvider are all untouched.
Closes#361
Backfill chained bounded slices back-to-back with no gap, and on a large
mailbox `moreWork` never clears, so a single BackfillWorker run paged
flat-out for its whole session and kept the account's IMAP connection
saturated -- the background load that starves interactive message-opens
(#355) and worsens provider throttling (#360).
Add BackfillPacer, a small in-process primitive that bounds one run:
- inter-slice cooldown: a fixed 30s idle between chained slices;
- per-run slice cap: at most 4 slices per run, then defer to the 30-min
periodic cadence so one run cannot monopolise the account.
Composes with the two sibling mechanisms instead of duplicating them:
- #355 (InteractiveImapGate): the cooldown is SKIPPED while an interactive
fetch is active -- the next slice already parks at its per-page yield
point, so a fixed delay on top would only double the idle (no pathological
double-delay);
- #360 (AccountThrottleGate): a slice whose only outstanding work is a
throttled account returns moreWork=false, so the loop stops and no
cooldown is spent spinning on a backed-off provider.
The cooldown is a cancellable delay and the loop rechecks !isStopped before
each slice, so a WorkManager stop / teardown ends a run promptly. All
breadcrumbs are PII-free (durations/counts only).
Tests: BackfillPacerTest (JVM, virtual time) covers cooldown timing, cap,
cancellation, and the #355/#360 composition; BackfillWorkerTest asserts the
worker caps a flat-out run; BackfillPacerInstrumentedTest proves forward
progress across paced runs, the interactive skip, and prompt cancellation on
the real Android runtime.
Closes#356
Addresses the below-cut data-core review nits from #313:
- SignatureRepository.delete: wrap delete + default-promotion in one SignatureDao
@Transaction (deletePromotingDefault) so a crash between them can't leave an
account with signatures but no default; log the promotion (PII-free).
- SignatureRepository.create: move the count-then-default check-then-act into a
SignatureDao @Transaction (insertMakingFirstDefault) so two concurrent
first-creates can't both become default.
- AccountSettingsRepository.update: route the read-modify-write through an
AccountSettingsDao @Transaction (readModifyWrite) so concurrent per-field
setters can't clobber each other.
- MailRepositoryImpl expunge/move/move-by-role: chunk the unbounded
getRoutingByIds/deleteByIds IN(:ids) queries (500/chunk) like MailPruner,
removing the latent SQLITE_MAX_VARIABLE_NUMBER crash.
- MessageDao.observeSummaries: remove the dead whole-table projection (superseded
by Paging #124/#214); migrate test/debug-probe callers to getById or the paged
query (which now guards the #51 CursorWindow regression).
- AccountDataMigrator: fix stale KDoc (schema is v2 with sortOrder, not v1).
- DatabaseEncryption.migrate: also sweep the stale -journal sidecar (journal_mode
= DELETE), matching AccountDataMigrator's sweep.
Unit tests updated for the repository delegations; instrumented DAO tests cover
the new @Transaction behaviour; MailRepositoryImplTest covers the chunk split;
DatabaseEncryptionTest covers the -journal sweep.
Closes#313
Six of the seven LOW findings collected in #308; the seventh is
deliberately skipped (see below).
- SettingsViewModel: run the app-lock disable-path Keystore/DataStore
reseal off the main dispatcher (withContext(Dispatchers.Default)),
matching AppLockViewModel's threading policy - no Keystore crypto on
Main.
- AppLockGateHost: clear the covered app content out of the semantics
tree while locked so TalkBack can't traverse the occluded mailbox/
compose nodes behind the opaque cover; content stays composed so its
state still survives a re-lock.
- ReaderViewModel.toggleStar: reconcile the optimistic star on a failed
persist - roll it back and surface a one-shot StarFailed event instead
of leaving the star stuck in a state the store rejected.
- OnboardingViewModel: persist firstAddedAccountId in SavedStateHandle so
a process kill mid-onboarding still finishes onto the first account's
inbox rather than the unfiltered mailbox (preserves #30).
- RichTextEditor: memoize the formatting toolbar's parse + selection
scans with remember(value) so the per-keystroke hot path isn't
re-derived on every recomposition.
- AccountSetupViewModel.onOutlookResult: treat a normal OAuth cancel
(null result) as a no-op instead of surfacing an error snackbar.
Skipped: MailboxViewModel per-keystroke search re-paging - the finding
is documented-intentional and only a "could". The local pager narrows
cached results instantly as you type while the expensive server search
is already debounced (400ms); debouncing the local pager would add lag
for no clear win, and correct scoping (query only, not account/folder)
adds risk to a hot, well-tested path.
Each behavioral change ships a JVM/Robolectric test; the toolbar
memoization (a pure refactor) adds a toolbarStateOf test. PII-free
AppLog breadcrumbs added on the new fallback/state-change paths.
Closes#308
Adds the reactive throttle layer for #360: when a provider rate-limits or
locks us (IMAP `[THROTTLED]`/"too many connections" NO, HTTP 429, auth
lockout), the app now backs off exponentially and pauses the offending
activity instead of hammering the server (which the perf drilldown proved
makes throttling worse — `docs/perf/issue-125-*`).
- ThrottleClassifier: message-text (IMAP/SMTP) + HTTP-status classification of
throttle vs lockout, distinct from ordinary transient errors; conservative
so a wrong password or the #390 "IMAP disabled" state is never misread.
- ThrottleBackoff: pure exponential schedule with equal jitter, a bounded cap,
and Retry-After honoring; a longer window for a lockout (sized to Yahoo's ~1h).
- AccountThrottleGate: per-account backoff state (@Singleton), so one throttled
account never stalls the others; PII-free AppLog breadcrumbs on throttle/clear.
- MailBackfiller: skips an account inside its backoff window and stops paging one
that throttles mid-slice (a degradation, not a moreWork spin) — resumes on a
later slice once the window elapses. Reset on the next successful page.
- MailSyncer: foreground sync feeds the gate but is never blocked by it, so
interactive work keeps priority over background backfill.
Integrates with the existing WorkManager retry (#403) and connection reuse
(#357) rather than duplicating them. Unit tests cover classification (positive
+ negative), the backoff schedule, and the degrade-not-tight-loop behaviour
(virtual time for the timing).
Closes#360
At account-add both LibreMailApplication's push collector and
IdleService.reconcileWatchers react to the accounts table. The account row was
inserted before its credential was saved, so a watcher could observe the new
account and call MailConnectionFactory.resolveSecret before the secret existed,
logging "No stored credentials" on the first IDLE attempt (it self-healed on
retry, but fired a failed IDLE + log noise on every add).
Primary fix: reorder the writes so the credential is committed before the account
row (the credentials table has no FK to accounts; account_settings does, so its
ensureDefaults still follows the insert). Any reactive observer of the account row
is then guaranteed to see the credential.
Defense-in-depth: resolveSecret now throws a typed MissingCredentialsException and
the IDLE watcher treats it as a transient miss, deferring quietly (short flat
re-check, PII-free info log) instead of the warn + exponential backoff a real
connection drop gets. Genuinely-absent credentials keep deferring without noise.
Tests: AccountRepositoryImplTest pins the credential-before-row order
(coVerifyOrder); MailConnectionFactoryTest asserts the typed exception; a new
instrumented test drives the real repository add path against a real
AccountDatabase + Keystore-backed CredentialStore and proves the secret is
resolvable the instant the account row becomes observable.
Adds AppLog breadcrumbs to the message-open path so a debug report can show
where the reader's spinner time goes:
- ImapClient.withStore: per-op connect vs. work timing plus a live
connect-per-op connection gauge (issue #125's provider-ceiling context).
- fetchBodyMarkingSeen: select/body/flag phase timings plus PII-free size
counts (RFC822 size, body chars, attachment count).
- MailRepositoryImpl.openMessage: end-to-end open latency plus the
cached-vs-fetched branch, keyed by accountLogRef and logSafeFolderLabel.
- ReaderViewModel: spinner-to-ready latency, split success vs. failure.
All breadcrumbs are PII-free: accounts are logged via the existing
accountLogRef hash, folders via the existing logSafeFolderLabel allowlist,
and everything else is sizes/durations/booleans only.
Fixes the 4 unit-test classes that exercise this code without mocking
android.util.Log (a throwing stub under plain JVM tests): mockkStatic(Log)
is now installed in MailRepositoryImplCoverageTest, ImapClientBackfillTest,
ImapFolderOpenLatencyTest, and ReaderViewModelActionsTest, following the
existing MailBackfillerTest/ImapClientTest conventions. detekt.yml gains two
more ForbiddenImport excludes for the newly Log-importing test files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a detekt style>ForbiddenImport rule that forbids `import android.util.Log`
so all logging flows through org.libremail.reporting.AppLog, which mirrors each
line into the debug-report RingLogBuffer. A raw android.util.Log import writes to
Logcat only and never reaches a user-reviewed DebugReport (epic #324, strangler
final step).
Excludes the AppLog facade itself (the one sanctioned wrapper) and the unit tests
that mockkStatic(Log) to verify forwarding — AppLog forwards to Log, a throwing
stub under plain JVM unit tests, so those tests must mock it; they do not bypass
the facade.
Closes#331
Part of #324
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire ktlint-gradle 14.2.0 and detekt 2.0.0-alpha.5 (the only detekt line
with Gradle 9 support) through the version catalog.
- .editorconfig: official Kotlin style, 120-col limit, @Composable exempt
from function-naming.
- config/detekt/detekt.yml: slim overrides on detekt's defaults —
@Composable exemptions for the OOP-era metrics, sane ReturnCount /
ThrowsCount / TooManyFunctions thresholds, and TooGenericExceptionCaught
off at the resilient network/push boundaries (which now log).
Findings fixed in code (behaviour-preserving; 81 unit tests still pass):
- SwallowedException: SendWorker / IdleService now log the caught exception.
- roleOf (Folder) and extractBody (ImapClient) split into named helpers.
- MailSyncer: hoisted a 4-condition `if` into a named val.
- TopDest extracted to its own file; ~14 magic numbers -> named constants.
The remainder is the ktlint auto-format across the module.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
JMR-dev
Jason Ross