Restructure isConnectionDrop as leading guard clauses (definite-drop
types, then a not-MessagingException early return) instead of a when
expression, per maintainer review feedback on PR #368. Behavior is
unchanged; verified by the existing ImapConnectionCacheTest suite
(all 8 cases still pass), including the FolderClosedException /
StoreClosedException cases that depend on the check running before
the MessagingException .cause guard.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve the IdleService.kt conflict as a union of both intents:
- #354 (already on main): foreground-service lifecycle rework —
onStartCommand delegates to the IdleForegroundStarter seam
(START_NOT_STICKY), cap-window skip/degrade.
- #357 Part 2 / #368: reused-connection idle-eviction sweep and
low-battery teardown of reused connections.
In startWatchingIfNeeded(), reconcileWatchers() stays inside the
cache-lock-guarded launch and evictIdleReuseConnectionsLoop() launches as
a sibling coroutine that runs while the service lives (its original #368
placement, independent of the cache-lock guard). No behavior change to
either side.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An on-device drilldown proved Gmail server-side throttles LibreMail's
connect-per-operation IMAP: every op was a fresh CONNECT+TLS+LOGIN, and
full-history backfill's body+attachment prefetch generated ~601 connections in
~22 min, tripping (and sustaining) Gmail's per-account rate/bandwidth clamp
(body download collapsed to ~4 KB/s). The `live` gauge peaked at only 5 (Gmail
allows ~15), so it is connection *volume*, not count. Outlook IMAP on the same
device opened in 2-3 s. Reusing one warm socket per account (~601 -> ~1) removes
the throttle's trigger. This wires the reuse path the #125 spike built and left
OFF (issue #357 Part 2 — connection reuse only; prefetch is a separate PR).
How it is enabled (with a safety switch):
- New `BuildConfig.IMAP_CONNECTION_REUSE` (default true) drives the production
`ImapClient` no-arg `@Inject` constructor. To disable if a server misbehaves,
flip it to "false" in app/build.gradle.kts — a build-config change, no Kotlin
edit. The internal `ImapClient(reuseConnections, reuseIdleTimeoutMillis)`
constructor stays the test/harness seam.
- Universal: applies to all providers (incl. Outlook). No per-provider caps or
throttling here — that is a separate effort (#356/#360-#364).
Hardening `ImapConnectionCache` for production (was a spike):
- Transparent stale recovery: broadened drop detection to Angus's own
`iap.ConnectionException` (and a MessagingException caused by one) — the real
signal `folder.open()` throws on a server-dropped idle socket, which the
IOException-only check missed, so the reconnect now actually fires. A dropped
reused socket is rebuilt once and the op retried, so callers see no spurious
error; a genuine app error (e.g. message-not-found) is never retried.
- Idle eviction: `evictIdle()` closes a connection unused past the reuse idle
timeout (default 5 min), swept every 2 min by `IdleService`; skips any
in-use connection.
- Teardown: `IdleService` also tears down reused connections on the low-battery
push-teardown path (#88/#89/#90), mirroring the IDLE connection teardown.
- Concurrency: one connection per account behind a per-account mutex; the
eviction sweep takes the lock non-blockingly so it never stalls or interrupts
an in-flight op. Coexists with IMAP IDLE (its own separate connection).
- PII-free AppLog on the lifecycle (open / reuse-hit / reconnect-stale / evict /
teardown) keyed by an opaque per-cache ordinal, plus the #358 ImapPerf
breadcrumb (connect~=0ms on a reuse hit).
Tests (all via the fast gate, no emulator):
- ImapConnectionCacheTest: reuse, retry-once stale recovery, narrow drop
detection, deterministic idle eviction (injected clock), teardown.
- ImapFolderOpenLatencyTest (GreenMail + counting proxy): N ops share one
connection/LOGIN; a force-dropped socket is transparently reconnected; an app
error does not reconnect; idle eviction LOGS-OUT and the next op reconnects.
- Correctness suites (ImapClientTest/ImapClientBackfillTest/MailBackfillerTest)
pinned to reuse-off to keep their connect-per-op assertions unchanged.
Fast gate green: assembleDebug, testDebugUnitTest, compileDebugAndroidTestKotlin,
lintDebug, ktlintCheck, detekt.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause: after #302's runtime-cap fallBackToPeriodicSync() stops the
dataSync foreground service, IdleService was restarted (START_STICKY
null-intent redelivery + explicit startForegroundService) and onStartCommand
unconditionally called startForeground(DATA_SYNC) while the rolling-24h budget
was still exhausted. The platform rejected the start with
ForegroundServiceStartNotAllowedException; it was uncaught, the process
crashed, and START_STICKY restarted straight back into the same rejection -- a
crash loop until the 24h window freed budget (#354).
Fix (IdleService.kt):
- onStartCommand now returns START_NOT_STICKY. Push is app-managed
(LibreMailApplication.ensurePushStarted deterministically restarts it), so the
sticky null-intent auto-restart was redundant and fired exactly when a dataSync
FGS start is illegal.
- Guard the foreground start via a new JVM-testable IdleForegroundStarter seam:
a ForegroundServiceStartNotAllowedException (caught via its IllegalStateException
supertype, so no minSdk-29 class load) degrades like the cap handler --
schedulePeriodicSync(), keep the degraded POLLING notification, stopSelf()
promptly (avoids the "did not call startForeground in time" ANR) -- instead of
propagating.
- Record the cap event (elapsedRealtime); while still inside the cap window,
onStartCommand skips the now-guaranteed-illegal foreground start entirely.
- onTimeout stop path kept fast so ForegroundServiceDidNotStopInTimeException
stays mitigated.
PII-free AppLog.w/i on the degrade paths.
Tests:
- Unit (IdleForegroundStarterTest): onStartCommand returns START_NOT_STICKY; a
rejected start is caught and routed to degrade without propagating; the cap
window skips the attempt; a non-ISE propagates.
- Instrumented (IdleServiceForegroundStartInstrumentedTest): the degrade path on
a real Context -- rejection caught, periodic-sync fallback scheduled, degraded
"instant delivery paused" notification built, watching skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds AppLog breadcrumbs to the message-open path so a debug report can show
where the reader's spinner time goes:
- ImapClient.withStore: per-op connect vs. work timing plus a live
connect-per-op connection gauge (issue #125's provider-ceiling context).
- fetchBodyMarkingSeen: select/body/flag phase timings plus PII-free size
counts (RFC822 size, body chars, attachment count).
- MailRepositoryImpl.openMessage: end-to-end open latency plus the
cached-vs-fetched branch, keyed by accountLogRef and logSafeFolderLabel.
- ReaderViewModel: spinner-to-ready latency, split success vs. failure.
All breadcrumbs are PII-free: accounts are logged via the existing
accountLogRef hash, folders via the existing logSafeFolderLabel allowlist,
and everything else is sizes/durations/booleans only.
Fixes the 4 unit-test classes that exercise this code without mocking
android.util.Log (a throwing stub under plain JVM tests): mockkStatic(Log)
is now installed in MailRepositoryImplCoverageTest, ImapClientBackfillTest,
ImapFolderOpenLatencyTest, and ReaderViewModelActionsTest, following the
existing MailBackfillerTest/ImapClientTest conventions. detekt.yml gains two
more ForbiddenImport excludes for the newly Log-importing test files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve DatabaseModule conflict from #320: main replaced the explicit .addMigrations(...) chain with .addMigrations(*ALL_MIGRATIONS) plus an introspectable ALL_MIGRATIONS list guarded by databaseModuleRegistersEveryDeclaredMigration (registered == declared). Add MIGRATION_19_20 to ALL_MIGRATIONS so the unified-inbox covering-index migration (cache schema v19->v20) is both registered on the Room builder and satisfies that safety-net test. Schema 20.json, the v20 @Database version, and DatabaseEncryptionTest's schema-version assertion (20) are unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The sync engine (MailSyncer, MailBackfiller, MailPruner, and their WorkManager
workers) was completely silent, so a submitted debug report showed nothing
about whether sync ran, how much it fetched, or why it was skipped. Add
net-new AppLog breadcrumbs at each class's lifecycle points per the #324
strangler-migration plan: sync start/done/failed and per-folder fetch counts,
backfill slice start/done and per-folder page counts, prune's removed count,
and each worker's cache-locked deferral and success/retry outcome (the retry
path now also carries the scrubbed failure throwable via AppLog's #325
overloads).
Every breadcrumb is PII-safe by construction: accounts are identified only via
accountLogRef(account.id) (never the id or email directly), and a new
logSafeFolderLabel() helper logs a folder's name only when it matches a fixed
allowlist of known system folders (INBOX, Sent, Drafts, Trash, Spam/Junk,
Archive, and their common provider variants) — every other folder, however
nested or named, logs as a fixed placeholder.
Adding logging to these previously-silent classes meant every existing test
exercising them now hits android.util.Log (a throwing stub under plain JVM
unit tests), so each affected suite gains the same static Log mock already
established by AppLogTest/SendWorkerTest/ImapClientTest.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Migrate AppLockViewModel (9 sites) and AccountSetupViewModel (1 site) off
raw android.util.Log onto the AppLog seam (#325), so their diagnostic
lines land in the RingLogBuffer and reach a submitted DebugReport instead
of only Logcat. Adds three new breadcrumbs that were previously silent:
auth-seal unlock success, the clear-cache-and-restart recovery trigger
(with the disableAppLock flag), and every onForeground LockAction
decision. AccountSetupViewModel's success path also now logs "Outlook
account added" (no email). None of these call sites carry PII; where a
throwable is attached, AppLog's StackTraceScrubber redacts it before it
reaches the buffer.
Both ViewModel test suites now install a real RingLogBuffer and assert
against it instead of `verify { Log... }`, including dedicated no-PII
assertions (a known test email never appears in a recorded line). A
minimal `mockkStatic(Log::class)` stub stays in both test files' shared
setUp — AppLog still forwards to the real android.util.Log internally,
which throws "not mocked" in JVM unit tests when uninvoked; the not-yet
-landed guard-rule ticket (#331) will need to reconcile that with a
repo-wide "no raw Log outside AppLog.kt" rule.
Closes#326
Part of #324🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Migrate ImapClient/SendWorker/IdleService off raw android.util.Log to AppLog,
per the debug-logging strangler epic (#324), so their diagnostics reach the
RingLogBuffer (and a submitted DebugReport) instead of logcat-only:
- ImapClient: IDLE connect + IDLE push (message count) breadcrumbs.
- SendWorker: outbox-drain count on entry, per-message sent/failed result,
and the existing Graph->SMTP fallback warning.
- IdleService: IDLE watch start, cache-locked defer, and the existing
IDLE-dropped/retrying warning.
Also closes#297: SendWorker and IdleService logged the raw account.email via
Log.w on the Graph->SMTP fallback and IDLE-drop paths. Both now log
accountLogRef(account.id) instead -- a short, stable, non-reversible
per-account reference -- so the account's email never reaches Logcat or a
report.
Rewrites ImapClientTest/SendWorkerTest to install a real RingLogBuffer via
AppLog.install(...) and assert on its contents (migrated calls + new
breadcrumbs), instead of verifying a mocked Log; every assertion also checks
no line carries the test account's email, regression-covering #297. Adds a
SendWorkerTest case that drives a real SmtpSender against an in-process
GreenMail SMTP server end to end. android.util.Log is still stubbed (by
fully-qualified name, without importing it) where AppLog's Logcat passthrough
would otherwise crash the unmocked Android stub in a JVM test.
Closes#328Closes#297
Part of #324
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Migrates the DB/keystore area's raw android.util.Log calls to AppLog so
key-invalidation and DB-conversion breadcrumbs land in the process
RingLogBuffer (and thus a user-reviewed debug report) even in release
builds, where Log.d is otherwise stripped from Logcat only.
- DatabaseKeyCipher: 4 auth-bound-key decision points (encrypt retry,
isInvalidated's three branches) now log via AppLog.d(tag, msg, e).
- DatabaseEncryption.migrate: adds an AppLog.i "converting local cache
database (targetEncrypted=...)" breadcrumb at the start, alongside the
existing "converted" completion line now routed through AppLog.d.
- AccountDataMigrator: the "moved account tables into the account
database: $present" breadcrumb (table names only) now routed through
AppLog.d.
No PII or key material is logged; table-name sets and boolean flags only.
Adds instrumented tests (DatabaseKeyCipher is device-only and
behavior-preserving, so no new test there) asserting the breadcrumbs
land in a RingLogBuffer and never contain the seeded email, secret, or
passphrase.
Part of #324.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Migrate RestartActivity's one raw Log.w site to AppLog, clearing the
final raw android.util.Log site outside the auth/lock, DB/keystore,
connectivity/send, and sync-engine migration areas so the codebase is
ready for the detekt android.util.Log guard (#331).
RestartActivity runs in the separate :restart trampoline process,
where LibreMailApplication.onCreate returns early and never calls
AppLog.install, so this breadcrumb reaches Logcat only, never a
DebugReport. The migration is guard-compliance + Logcat-consistency
only; behavior is unchanged since AppLog forwards to Logcat.
RestartActivity is DEVICE-ONLY (multi-process kill/relaunch), so a
JVM buffer-capture test doesn't apply here. Added
RestartActivityLoggingTest, which instead pins the null-buffer shape
this call runs under in the trampoline process: it forwards to
Logcat and no-ops the buffer cleanly.
Closes#330
Part of #324🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add throwable-recording overloads to AppLog.d/w and make AppLog.e record the
throwable it is given: the throwable's stack trace is scrubbed via the existing
StackTraceScrubber (exception class names + frames kept; host/email-bearing
exception messages stripped) and appended to the buffered log line, so a
throwable can reach a user-reviewed DebugReport without leaking PII. The
existing no-throwable overloads are unchanged.
Add accountLogRef(accountId): a short, stable, non-reversible reference
(scheme prefix + truncated SHA-256 of the id) so downstream logging can
identify an account without logging the raw Account.id, which embeds the email.
Foundation for the #324 debug-logging strangler epic; consumed by #326–#330.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Scope :app:jacocoTestReport's denominator to the JVM-testable surface and
add a :app:jacocoTestCoverageVerification no-regression gate that shares the
same classDirectories/executionData/sourceDirectories, wired into both the
`check` lifecycle task and CI's unit-test job (part of the `CI passed` gate).
Excluded from the denominator (structurally unreachable from a JVM unit
test): Compose screen/component render code, Android framework entry points
(*Activity/*Service/Application/*BackupAgent), Hilt DI (**/di/**), and the
src/debug cold-open probe. Kept in scope: ViewModels, repositories, mappers,
DAOs, utils, richtext, mail, reporting logic, and the six WorkManager Workers.
Corrects PR #292, which excluded **/*Worker*: SyncWorker, BackfillWorker,
PruneWorker, SendWorker, ReportPurgeWorker and ReportUploadWorker are all
directly unit-tested, so they stay counted in both numerator and denominator
(only their Hilt wiring, WorkManagerModule, is excluded, via **/di/**).
Baseline: 80.21% line (4838/6032). Floor: 0.79 (~1.2% headroom) so ordinary
noise doesn't red-flag it while a real drop fails. Manual ratchet for now:
bump the floor up in the same PR when coverage rises materially.
Closes#251Closes#292
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
MIGRATION_19_20 (issue #187) bumped the Room cache schema to version 20,
but DatabaseEncryptionTest.schemaVersionIsCarriedOntoTheEncryptedFile still
asserted the plaintext -> encrypted conversion carried version 19, so it
failed across all E2E levels after the rebase onto main.
DatabaseEncryption.migrate() carries PRAGMA user_version dynamically
(userVersion = source.version -> target.version = userVersion), and a fresh
Room open now stamps 20, so v20 genuinely survives the conversion. Update the
expected constant to 20; the assertion's intent (the version survives the
round-trip) is unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
JMR-dev
