CacheEncryptionGate.kt (the gate composable, blank cover, error screen, and ephemeral
report-review screen added for #359) is pure Compose render code, structurally
unreachable from a JVM unit test the same way every other Screen file in
jacocoNonJvmTestableSurface is. Left in scope, it dragged the whole-app line ratio to
0.78, just under the 0.79 no-regression floor.
Excluded it via "**/CacheEncryptionGateKt*" rather than the usual bare
"**/CacheEncryptionGate*" pattern this list otherwise uses, because
CacheEncryptionGateViewModel is named with "CacheEncryptionGate" as a literal
prefix - the bare wildcard would also have swallowed the already JVM-tested,
94%-covered ViewModel and its sealed CacheEncryptionGateState. CacheEncryptionGateViewModel
and CacheEncryptionUnavailableException stay in scope unchanged.
Verified locally: testDebugUnitTest + jacocoTestCoverageVerification now pass, with
the line ratio recovered to about 0.807 (5,044 covered / 6,249 total lines) - the
same 5,044 covered lines as before, just a smaller, honestly-JVM-testable denominator.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reworks #367. When SQLCipher native library loading fails while the opt-in
encrypted cache is enabled, the app previously degraded to a plaintext cache
(a silent fail-open that defeats the feature). Now it FAILS CLOSED.
DatabaseProvisioner raises a distinct CacheEncryptionUnavailableException
instead of degrading: it does NOT open plaintext, NOT wipe the on-disk
ciphertext, and NOT write the encryptCache setting. The throw is not
memoized, so a later launch re-attempts and recovers automatically if the
library loads.
A new CacheEncryptionGate wraps the app inside AppLockGateHost (so the
passphrase is already unlocked), probes prepareCache() before any DB-backed
screen composes, and on failure shows CacheEncryptionErrorScreen with the
exact message "Error - decryption could not proceed. Native decryption
library load failure." plus a "Report a problem" action. That action
generates an EPHEMERAL PII-free report via the existing DiagnosticsCollector
(never written to ReportStore, since encryption is unavailable in that
moment) for on-screen review and explicit Copy/Save; the copy says so.
The plaintext AccountDatabase tolerates the exception so accounts stay
readable for the error gate and the report. The encryptCache setting is now
written by exactly one caller: the user Settings toggle.
Tests: fail-closed raises the signal with no plaintext open / no wipe / no
setting write / not memoized; the gate VM resolves Ready vs Unavailable and
builds the ephemeral report; an instrumented error-screen UI test and an
AccountDatabase-resilience instrumented test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bind the non-exportable AES-256-GCM keys that seal the SQLCipher cache
passphrase to the hardware StrongBox secure element when the device has
one. Applied in the single shared place, AesGcmKeystoreCipher, so it
covers both the master (KeystoreCrypto) and auth-bound (DatabaseKeyCipher)
keys.
Devices without StrongBox throw StrongBoxUnavailableException at
KeyGenerator.generateKey(); a new generate-with-fallback path catches it
and regenerates a TEE-backed key so key creation still succeeds
everywhere. Guarded on API 28+ (minSdk is 29). Framing, seal/unseal, and
the missing-key policies are unchanged; the passphrase is still never
plaintext at rest and never logged.
Adds a JVM regression test for the StrongBox->TEE fallback via the
existing test seams (existingKey / a new generateKey seam).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a cross-platform, standard-library-only Python package under
scripts/device-testing/ that replicates LibreMail's on-device performance-test
scenarios and logging capture, codifying the methodology run by hand on
2026-07-05 (Pixel 10 Pro XL).
Modules:
- breadcrumbs.py: a pure, unit-tested parser for the ImapPerf / MailReader /
Reader / MailBackfiller breadcrumbs, plus open-correlation that reproduces the
manual timing-tables.md figures exactly.
- adb.py: a safety-guarded adb wrapper -- an allow-list of adb subcommands and a
deny-list + assertions on shell commands. The only sanctioned app-state
mutation is clearing LibreMail's own cache/ (exact-match); no pm clear /
uninstall / data wipe, and no touching databases/ files/ shared_prefs/
datastore/ can be constructed.
- uidump.py: uiautomator XML parser + screen recognition (mailbox rows with the
cached "Available offline" flag, reader, and the keyguard / foreign-app guards).
- scenarios.py: cold-open, message-open (uncached), back-nav, prefetch A/B
(fetch-policy toggle) and cross-provider, each keyguard-guarded and driven
through the guarded wrapper.
- report.py + perf_harness.py: aggregates, a timing-tables.md renderer mirroring
the manual write-up, and the CLI (timestamped run dir with the raw logcat, a
filtered breadcrumb extract, and the tables). --dry-run prints the exact
command plan without touching device state.
Tests (stdlib unittest, 68 cases) validate the parser, guardrails, UI
recognition and report against the manual run's real captures (fixtures include
a verbatim perf-extract slice and the reader/lockscreen/alarm dumps). Track the
*.log fixture past the gitignore *.log rule via a scoped negation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Restructure isConnectionDrop as leading guard clauses (definite-drop
types, then a not-MessagingException early return) instead of a when
expression, per maintainer review feedback on PR #368. Behavior is
unchanged; verified by the existing ImapConnectionCacheTest suite
(all 8 cases still pass), including the FolderClosedException /
StoreClosedException cases that depend on the check running before
the MessagingException .cause guard.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve the IdleService.kt conflict as a union of both intents:
- #354 (already on main): foreground-service lifecycle rework —
onStartCommand delegates to the IdleForegroundStarter seam
(START_NOT_STICKY), cap-window skip/degrade.
- #357 Part 2 / #368: reused-connection idle-eviction sweep and
low-battery teardown of reused connections.
In startWatchingIfNeeded(), reconcileWatchers() stays inside the
cache-lock-guarded launch and evictIdleReuseConnectionsLoop() launches as
a sibling coroutine that runs while the service lives (its original #368
placement, independent of the cache-lock guard). No behavior change to
either side.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An on-device drilldown proved Gmail server-side throttles LibreMail's
connect-per-operation IMAP: every op was a fresh CONNECT+TLS+LOGIN, and
full-history backfill's body+attachment prefetch generated ~601 connections in
~22 min, tripping (and sustaining) Gmail's per-account rate/bandwidth clamp
(body download collapsed to ~4 KB/s). The `live` gauge peaked at only 5 (Gmail
allows ~15), so it is connection *volume*, not count. Outlook IMAP on the same
device opened in 2-3 s. Reusing one warm socket per account (~601 -> ~1) removes
the throttle's trigger. This wires the reuse path the #125 spike built and left
OFF (issue #357 Part 2 — connection reuse only; prefetch is a separate PR).
How it is enabled (with a safety switch):
- New `BuildConfig.IMAP_CONNECTION_REUSE` (default true) drives the production
`ImapClient` no-arg `@Inject` constructor. To disable if a server misbehaves,
flip it to "false" in app/build.gradle.kts — a build-config change, no Kotlin
edit. The internal `ImapClient(reuseConnections, reuseIdleTimeoutMillis)`
constructor stays the test/harness seam.
- Universal: applies to all providers (incl. Outlook). No per-provider caps or
throttling here — that is a separate effort (#356/#360-#364).
Hardening `ImapConnectionCache` for production (was a spike):
- Transparent stale recovery: broadened drop detection to Angus's own
`iap.ConnectionException` (and a MessagingException caused by one) — the real
signal `folder.open()` throws on a server-dropped idle socket, which the
IOException-only check missed, so the reconnect now actually fires. A dropped
reused socket is rebuilt once and the op retried, so callers see no spurious
error; a genuine app error (e.g. message-not-found) is never retried.
- Idle eviction: `evictIdle()` closes a connection unused past the reuse idle
timeout (default 5 min), swept every 2 min by `IdleService`; skips any
in-use connection.
- Teardown: `IdleService` also tears down reused connections on the low-battery
push-teardown path (#88/#89/#90), mirroring the IDLE connection teardown.
- Concurrency: one connection per account behind a per-account mutex; the
eviction sweep takes the lock non-blockingly so it never stalls or interrupts
an in-flight op. Coexists with IMAP IDLE (its own separate connection).
- PII-free AppLog on the lifecycle (open / reuse-hit / reconnect-stale / evict /
teardown) keyed by an opaque per-cache ordinal, plus the #358 ImapPerf
breadcrumb (connect~=0ms on a reuse hit).
Tests (all via the fast gate, no emulator):
- ImapConnectionCacheTest: reuse, retry-once stale recovery, narrow drop
detection, deterministic idle eviction (injected clock), teardown.
- ImapFolderOpenLatencyTest (GreenMail + counting proxy): N ops share one
connection/LOGIN; a force-dropped socket is transparently reconnected; an app
error does not reconnect; idle eviction LOGS-OUT and the next op reconnects.
- Correctness suites (ImapClientTest/ImapClientBackfillTest/MailBackfillerTest)
pinned to reuse-off to keep their connect-per-op assertions unchanged.
Fast gate green: assembleDebug, testDebugUnitTest, compileDebugAndroidTestKotlin,
lintDebug, ktlintCheck, detekt.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On Android 15+ / SDK 37 devices with 16 KB memory pages (e.g. Pixel 10 Pro XL,
and the API-37 `google_apis_ps16k` emulator image), a native `.so` not aligned
for 16 KB pages fails to load with `UnsatisfiedLinkError` at
`SQLiteConnection.nativeOpen`. With the opt-in SQLCipher encrypted cache on, this
crashed the app on every cold start (issue #359, x4 on-device) instead of
degrading, and encryption silently never applied.
Fix: DatabaseProvisioner's encryption gate now catches `LinkageError`
(UnsatisfiedLinkError and related native-link failures) when opening/converting
the encrypted cache and degrades cleanly instead of propagating the crash — it
turns `encryptCache` off (so the next start does not re-attempt and re-wipe),
clears any on-disk ciphertext the plaintext framework opener cannot parse
(resetting its now-useless seals), and opens the cache unencrypted. The cache is
a re-syncable copy of server mail, so clearing it loses nothing that cannot be
re-fetched. PII-free AppLog.w breadcrumb on the degrade path.
Dependency: no bump needed or available. The repo already pins the newest
SQLCipher it references, `net.zetetic:sqlcipher-android:4.16.0`, which
docs/play-compliance.md certifies (ELF p_align = 0x4000) as 16 KB-aligned on
every ABI; SQLCipher has shipped 16 KB-aligned binaries since well before it, and
the other two bundled `.so` files (Compose graphics-path, DataStore
shared-counter) are already 16 KB-aligned per that doc. The graceful-degrade
catch is therefore the actionable fix.
Tests:
- Unit (DatabaseProvisionerTest): a simulated native-load failure degrades to a
plaintext open without crashing, turns encryptCache off, and wipes + reseals an
already-encrypted cache.
- Instrumented (DatabaseProvisionerInstrumentedTest): a fresh encrypt-on start
loads the real SQLCipher native library and opens the keyed cache — CI's API-37
`google_apis_ps16k` 16 KB job exercises the actual `.so` load, catching any
future 16 KB-alignment regression.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause: after #302's runtime-cap fallBackToPeriodicSync() stops the
dataSync foreground service, IdleService was restarted (START_STICKY
null-intent redelivery + explicit startForegroundService) and onStartCommand
unconditionally called startForeground(DATA_SYNC) while the rolling-24h budget
was still exhausted. The platform rejected the start with
ForegroundServiceStartNotAllowedException; it was uncaught, the process
crashed, and START_STICKY restarted straight back into the same rejection -- a
crash loop until the 24h window freed budget (#354).
Fix (IdleService.kt):
- onStartCommand now returns START_NOT_STICKY. Push is app-managed
(LibreMailApplication.ensurePushStarted deterministically restarts it), so the
sticky null-intent auto-restart was redundant and fired exactly when a dataSync
FGS start is illegal.
- Guard the foreground start via a new JVM-testable IdleForegroundStarter seam:
a ForegroundServiceStartNotAllowedException (caught via its IllegalStateException
supertype, so no minSdk-29 class load) degrades like the cap handler --
schedulePeriodicSync(), keep the degraded POLLING notification, stopSelf()
promptly (avoids the "did not call startForeground in time" ANR) -- instead of
propagating.
- Record the cap event (elapsedRealtime); while still inside the cap window,
onStartCommand skips the now-guaranteed-illegal foreground start entirely.
- onTimeout stop path kept fast so ForegroundServiceDidNotStopInTimeException
stays mitigated.
PII-free AppLog.w/i on the degrade paths.
Tests:
- Unit (IdleForegroundStarterTest): onStartCommand returns START_NOT_STICKY; a
rejected start is caught and routed to degrade without propagating; the cap
window skips the attempt; a non-ISE propagates.
- Instrumented (IdleServiceForegroundStartInstrumentedTest): the degrade path on
a real Context -- rejection caught, periodic-sync fallback scheduled, degraded
"instant delivery paused" notification built, watching skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds AppLog breadcrumbs to the message-open path so a debug report can show
where the reader's spinner time goes:
- ImapClient.withStore: per-op connect vs. work timing plus a live
connect-per-op connection gauge (issue #125's provider-ceiling context).
- fetchBodyMarkingSeen: select/body/flag phase timings plus PII-free size
counts (RFC822 size, body chars, attachment count).
- MailRepositoryImpl.openMessage: end-to-end open latency plus the
cached-vs-fetched branch, keyed by accountLogRef and logSafeFolderLabel.
- ReaderViewModel: spinner-to-ready latency, split success vs. failure.
All breadcrumbs are PII-free: accounts are logged via the existing
accountLogRef hash, folders via the existing logSafeFolderLabel allowlist,
and everything else is sizes/durations/booleans only.
Fixes the 4 unit-test classes that exercise this code without mocking
android.util.Log (a throwing stub under plain JVM tests): mockkStatic(Log)
is now installed in MailRepositoryImplCoverageTest, ImapClientBackfillTest,
ImapFolderOpenLatencyTest, and ReaderViewModelActionsTest, following the
existing MailBackfillerTest/ImapClientTest conventions. detekt.yml gains two
more ForbiddenImport excludes for the newly Log-importing test files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the traffic-control (runner-priority orchestration) job verbatim out of
.github/workflows/ci.yml into a new standalone workflow,
.github/workflows/traffic-control.yml, so the heavy CI jobs no longer depend
on it. The job's YAML (name, runs-on, timeout-minutes, permissions, env,
steps) and its documentation comment move unchanged; the decision core
.github/scripts/traffic_control.py is untouched and still unit-tested by the
traffic-control-tests job in ci.yml.
In ci.yml: removed the traffic-control job, dropped needs: traffic-control
from the five heavy jobs (static-analysis, debug-build, unit-tests, e2e,
e2e-preview) and from traffic-control-tests (its only needs, which would
otherwise dangle at a now-deleted job), and updated the now-stale header and
ci-passed comments to point at the extracted workflow.
The new workflow will be disabled pending a rebuild as a published GitHub
Action.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
#350 made ci-trigger.yml dispatch ci.yml with the built-in GITHUB_TOKEN, on the
claim that a workflow_dispatch is anti-recursion-exempt so no PAT is needed. In
practice a GITHUB_TOKEN-triggered run is held in `action_required` awaiting manual
approval and never runs un-attended, so auto-updated PRs' CI never ran (stalled
#285). The original #349 design was right: dispatch with a PAT so the run executes
as the authorized owner with no approval gate.
- ci-trigger.yml: the trigger step's GH_TOKEN is now
`${{ secrets.AUTOUPDATE_TOKEN || github.token }}` (was `${{ github.token }}`).
AUTOUPDATE_TOKEN (the PAT) is REQUIRED for the scheduler; the `|| github.token`
fallback stays fail-open but only starts CI if repo settings don't gate
GITHUB_TOKEN-triggered runs.
- autoupdate.yml: branch update stays on GITHUB_TOKEN (must NOT retrigger CI --
that would re-introduce the cascade). Clarified that AUTOUPDATE_TOKEN is still
required by the repo (by ci-trigger.yml) so the secret isn't deleted.
- Corrected the now-wrong "no PAT needed / workflow_dispatch anti-recursion-exempt"
comments in ci-trigger.yml and the traffic_control.py docstrings.
updates = GITHUB_TOKEN, triggering = PAT.
Validation: all three workflow YAMLs parse clean; traffic-control unit tests still
pass (59 tests) -- the change is workflow-env only, script logic unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve DatabaseModule conflict from #320: main replaced the explicit .addMigrations(...) chain with .addMigrations(*ALL_MIGRATIONS) plus an introspectable ALL_MIGRATIONS list guarded by databaseModuleRegistersEveryDeclaredMigration (registered == declared). Add MIGRATION_19_20 to ALL_MIGRATIONS so the unified-inbox covering-index migration (cache schema v19->v20) is both registered on the Room builder and satisfies that safety-net test. Schema 20.json, the v20 @Database version, and DatabaseEncryptionTest's schema-version assertion (20) are unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
End the merge cascade and give the traffic-controller ownership of CI *triggering*.
- autoupdate.yml updates PR branches with the built-in GITHUB_TOKEN instead of a PAT,
so an update push no longer auto-retriggers CI (GitHub's anti-recursion rule) — the
cascade (every merge re-runs every PR, cancel-in-progress thrashing them) is gone.
- New scheduler ci-trigger.yml -> traffic_control.py --mode trigger (re-)triggers CI
for the highest-priority PR(s) whose head SHA has absent/stale checks, a few at a
time (inflight cap), in the existing P0-P9 / broken-draft priority order — a
poor-man's merge queue reusing the priority core. It runs after autoupdate finishes
(workflow_run, race-free) plus a cron backstop plus manual dispatch.
- Triggering uses workflow_dispatch, which is EXEMPT from anti-recursion, so the
built-in GITHUB_TOKEN (actions: write) starts the run — NO PAT / secret change needed.
- ci.yml gains a workflow_dispatch trigger (pr/head_sha/reason inputs) and a per-PR
concurrency group unifying pull_request and dispatch runs; its on: pull_request path
is kept so brand-new PRs, human pushes, and fork PRs always get CI (fail-open).
Pure select_triggers / classify_sha_runs decision core added to traffic_control.py with
24 new unit tests (priority order, oldest-first fairness, inflight cap, fork skip, P0
bypass+preempt, head-SHA needy classification, and a liveness/anti-starvation simulation).
Closes#349
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a detekt style>ForbiddenImport rule that forbids `import android.util.Log`
so all logging flows through org.libremail.reporting.AppLog, which mirrors each
line into the debug-report RingLogBuffer. A raw android.util.Log import writes to
Logcat only and never reaches a user-reviewed DebugReport (epic #324, strangler
final step).
Excludes the AppLog facade itself (the one sanctioned wrapper) and the unit tests
that mockkStatic(Log) to verify forwarding — AppLog forwards to Log, a throwing
stub under plain JVM unit tests, so those tests must mock it; they do not bypass
the facade.
Closes#331
Part of #324
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
JMR-devandClaude Opus 4.8