Commit Graph
722 Commits
Author SHA1 Message Date
Jason Ross 8689885964 Merge branch 'main' into fix-359-sqlcipher-16kb 2026-07-06 19:23:24 -05:00
Jason Ross 6213ee8901 Merge pull request #391 from JMR-dev/ci-389-sdk-setup-hardening
ci: retry + cache Android SDK/emulator setup to survive corrupt-zip sdkmanager failures (#389)
2026-07-06 19:22:02 -05:00
Jason Ross 8ac1d28dc8 Merge branch 'main' into ci-389-sdk-setup-hardening 2026-07-06 19:03:26 -05:00
JMR-devandClaude Opus 4.8 aa628831be ci: retry + cache Android SDK/emulator setup to survive corrupt-zip sdkmanager failures (#389)
The dominant merge-blocking flake was the "Set up Android SDK" step
(android-actions/setup-android v4.0.1) dying BEFORE the emulator starts:

    Wrong version in preinstalled sdkmanager
    Warning: ... preparing SDK package Android Emulator: Error reading Zip
    content from a SeekableByteChannel.
    Error: The process '.../sdkmanager' failed with exit code 1

Root cause: the action's default cmdline-tools version (20.0) rarely matches the
runner image's preinstalled one, so it logs "Wrong version in preinstalled
sdkmanager" and re-fetches cmdline-tools with NO checksum; it then runs its
default `sdkmanager tools platform-tools` install. Any of those downloads can be
a corrupt/truncated zip, which sdkmanager turns into an un-retried exit 1. v4.0.1
is the latest release, so this is fixed by configuration + hardening, not a bump.

Harden with verify -> reject -> retry, never trusting sdkmanager's exit code
alone, via a new stdlib-only helper .github/scripts/setup_android_sdk.py:

- bootstrap: download the pinned cmdline-tools zip, verify size + SHA-256
  (authoritative pin, cross-checked against Google's published SHA-1), and
  install it to $ANDROID_SDK_ROOT/cmdline-tools/20.0 -- the exact path
  setup-android probes first, so the action reuses the verified tree and never
  does its own unverified "Wrong version" re-download. A mismatch (corrupt OR
  wrong version) deletes the bad zip + any half-extracted dir and re-downloads.
- install: sdkmanager --install with retry + backoff; on a corrupt package zip it
  purges the partial/corrupt package dir (and sdkmanager's temp dirs) before
  retrying, forcing a fresh download instead of a re-read.
- setup-android now runs with packages: "" (no flaky tools/platform-tools
  install) and cmdline-tools-version: "14742923" (reuse the verified bootstrap).
- actions/cache restore + success-gated save so only a verified SDK is ever
  cached (integrity gates the cache); shrinks the re-download/corruption surface.

Applied to every SDK-setup job (debug-build, unit-tests, static-analysis, e2e
matrix, e2e-preview). Emulator BOOT logic, #372 API-37 sharding, and #388
diagnostics are untouched. Pure-logic helpers are unit-tested
(test_setup_android_sdk.py, run by the traffic-control-tests job).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:59:05 -05:00
Jason Ross 1053fa80f0 Merge pull request #375 from JMR-dev/robolectric-compose-scope
test(compose): Robolectric JVM Compose testing infra + PoC (#373)
2026-07-06 18:36:36 -05:00
Jason Ross 35b6b35d16 Merge branch 'main' into robolectric-compose-scope 2026-07-06 17:44:37 -05:00
Jason Ross 5a8c4e386e Merge branch 'main' into fix-359-sqlcipher-16kb 2026-07-06 16:56:27 -05:00
Jason Ross c04198a158 Merge pull request #388 from JMR-dev/ci-387-e2e-diagnostics
ci(e2e): capture logcat + emulator/system diagnostics across the E2E matrix (#387)
2026-07-06 16:55:52 -05:00
Jason Ross 0b1fb05a90 Merge branch 'main' into ci-387-e2e-diagnostics 2026-07-06 16:42:18 -05:00
Jason Ross e062331e09 Merge pull request #374 from JMR-dev/fix-signatures-test-teardown-race
test(settings): cancel viewModelScope before db.close in SignaturesScreenTest to fix a Room teardown race
2026-07-06 16:21:56 -05:00
JMR-devandClaude Opus 4.8 187a8effb0 ci(e2e): capture logcat + emulator/system diagnostics across the E2E matrix (#387)
The API 29-36 `e2e` matrix uploaded only its test report, so an emulator
flake or a red leg (e.g. `E2E (31)` dying on a bare `sdkmanager` exit 1)
left nothing to diagnose. Bring the #334 API-37 diagnostics to the matrix,
inline (no changes to `e2e-preview`, which PR #372 is restructuring):

- Stream `adb logcat -v time` to `$RUNNER_TEMP/logcat-api<level>.txt` at the
  top of both the "Run E2E tests" and retry reactivecircus steps (emulator is
  booted there); backgrounded so gradle stays the exit-status-bearing command.
- New `if: failure()` step dumps device + runner state (adb devices, logcat
  tail, emulator -accel-check, /dev/kvm, free -h, df -h) to the step log and a
  diagnostics file; every probe guarded with `|| true`.
- New `if: always()` upload-artifact (same pinned v7 SHA) `e2e-diagnostics-api<level>`
  carries the logcat + diagnostics files, `if-no-files-found: warn`.
- Make "Install SDK platform and build-tools" diagnosable: bounded 3x retry with
  backoff for a transient sdkmanager failure, and print `--list_installed` on a
  hard failure instead of a bare exit 1.

Keeps reactivecircus/android-emulator-runner and the existing boot-race retry.
Additive/diagnostic only; no boot-affecting flags change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 15:53:54 -05:00
Jason Ross 0186fa5ed9 Merge branch 'main' into robolectric-compose-scope 2026-07-06 15:53:48 -05:00
JMR-devandClaude Opus 4.8 324f7c2c51 fix(test): resolve Robolectric android-all via Gradle so the JVM Compose test runs in CI (#373)
Robolectric resolved its android-all-instrumented runtime jar lazily at test
time via its own MavenDependencyResolver/MavenArtifactFetcher, and that
download is unreliable on CI runners: AddAnotherAccountScreenJvmTest failed
with `AssertionError at MavenArtifactFetcher ... IOException` ("Failed to
fetch maven artifact"), though it passed locally where ~/.m2 was warm.

Resolve the jar through Gradle instead (reliable, cached, persisted by the CI
Gradle cache) and hand it to Robolectric in offline mode so it never hits the
network at test time:
- Pin org.robolectric:android-all-instrumented:16-robolectric-13921718-i7
  (exactly what Robolectric 4.16.1 DefaultSdkProvider maps @Config(sdk=36) to)
  in the version catalog.
- Add it to a dedicated resolvable configuration (NOT testImplementation/
  testRuntimeOnly, which would flatten the ~200MB instrumented framework onto
  the JVM test classpath and collide with the stub android.jar).
- syncRobolectricAndroidAll stages the jar under its Maven filename, and
  robolectric.offline + robolectric.dependency.dir point Robolectric's
  LocalDependencyResolver at it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 15:49:07 -05:00
Jason Ross 62797bc8c3 Merge branch 'main' into fix-359-sqlcipher-16kb 2026-07-06 15:46:14 -05:00
Jason Ross c8290ee545 Merge branch 'main' into fix-signatures-test-teardown-race 2026-07-06 15:45:09 -05:00
Jason Ross 9aa83a458c Merge pull request #372 from JMR-dev/ci-api37-e2e-sharding-spike
ci: shard the API 37 preview E2E into 2 parallel shards + retry parity
2026-07-06 15:41:02 -05:00
JMR-devandClaude Opus 4.8 cf1a8f6b83 test(compose): Robolectric JVM Compose testing infra + PoC (#373)
Enables unit-testing Jetpack Compose UI on the JVM via Robolectric, so
render-only screens can leave the jacocoNonJvmTestableSurface exclusion
list and be counted by JaCoCo without an emulator.

- add Robolectric 4.16.1 (test scope) + Compose ui-test-junit4/-manifest
- testOptions.unitTests.isIncludeAndroidResources = true so resources
  (strings, Material3 theme) resolve on the JVM
- src/test/resources/robolectric.properties pins sdk=36 (targetSdk 37 is
  a preview level Robolectric 4.16 has no sandbox for)
- PoC: AddAnotherAccountScreenJvmTest drives the screen with the v2
  createComposeRule under RobolectricTestRunner (3 tests, green on the JVM)
- drop AddAnotherAccountScreen from jacocoNonJvmTestableSurface (now
  JVM-covered); floor stays 0.79 — re-ratchet deferred to end of #373

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 15:37:23 -05:00
Jason Ross bb412e263e Merge branch 'main' into fix-signatures-test-teardown-race 2026-07-06 15:29:59 -05:00
Jason Ross b02ed87313 Merge branch 'main' into ci-api37-e2e-sharding-spike 2026-07-06 15:23:07 -05:00
Jason Ross 32199ef037 Merge branch 'main' into fix-359-sqlcipher-16kb 2026-07-06 15:22:45 -05:00
Jason Ross 1423395454 Merge pull request #370 from JMR-dev/feat-device-testing-harness
feat(scripts): device-testing perf harness
2026-07-06 15:22:20 -05:00
Jason Ross 1a3393da53 Merge branch 'main' into ci-api37-e2e-sharding-spike 2026-07-06 15:12:47 -05:00
JMR-devandClaude Opus 4.8 08ee9e7abc ci: productionize API 37 preview E2E sharding (adopt N=2)
The spike commits already implemented the N=2 shard matrix (numShards/
shardIndex via -Pandroid.testInstrumentationRunnerArguments.*), per-shard
test-retry parity, adb start-server before the boot loop, and shard-suffixed
artifact names. This drops the SPIKE / DRAFT "do not merge as-is" framing from
the ci.yml comments and reframes docs/perf/api37-e2e-sharding-spike.md from a
feasibility spike into the adopted design, so the change is mergeable as-is.

Also fixes the doc's section 3a example, which showed 1-based shardIndex values
[1, 2]; shardIndex is 0-based (0..numShards-1) and the implementation correctly
uses matrix.shard: [0, 1] -- [1, 2] would run an empty bucket and silently drop
half the suite.

Fan-in unchanged and verified: ci-passed still lists e2e-preview once; GHA
matrix aggregation makes its result `failure` if either shard fails, so both
shards must pass for the gate to go green. Branch protection requires the
"CI passed" context (not the per-leg "E2E (API 37 preview) (N)" check names),
so no branch-protection change is needed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 14:52:48 -05:00
Jason Ross 5f77eb7d8b Merge branch 'main' into fix-359-sqlcipher-16kb 2026-07-06 14:51:12 -05:00
Jason Ross d6c694c7e0 Merge branch 'main' into feat-device-testing-harness 2026-07-06 14:47:16 -05:00
JMR-devandClaude Opus 4.8 ca6d90b602 test(settings): cancel viewModelScope before db.close in SignaturesScreenTest to fix a Room teardown race (flaky on API-37 CI)
SignaturesScreenTest built a real SignaturesViewModel by hand but tore down
with a bare `db.close()` that never cancelled viewModelScope. The ViewModel's
`signatures` StateFlow is a Room InvalidationTracker Flow kept alive by
stateIn(WhileSubscribed(5_000)), so the collector could stay live up to 5s
after the UI detached — a re-query then landed on the just-closed in-memory DB
and threw SQLITE_MISUSE ("connection is closed"). Timing-dependent, hence the
intermittent API-37 CI failure in tappingRadioOnNonDefault_makesItTheDefault.

Fix: hold the ViewModel in an androidx.lifecycle.ViewModelStore and, in @After,
call store.clear() (→ ViewModel.onCleared() → cancels viewModelScope) BEFORE
db.close(), so the collector is gone before the DB closes. Behaviour and
assertions are unchanged; the fix removes the race by construction.

Audited the androidTest tree for the same hazard and fixed two siblings the
same way:
- AccountSettingsScreenTest: had the same live-Room-Flow-vs-close race,
  previously worked around by never closing the in-memory DB at all; now
  clears the ViewModel then closes the DB.
- ComposeScreenTest: ComposeViewModel's init launches a viewModelScope
  coroutine that reads the real accountSettings/signature Room repos; clear
  the store before db.close() to avoid the same in-flight-read-vs-close race.

Verified locally: connectedDebugAndroidTest green for all three classes
(12/12) on a cold-booted emulator, plus the JVM fast gate (assembleDebug,
testDebugUnitTest, jacocoTestCoverageVerification, compileDebugAndroidTestKotlin,
lintDebug, ktlintCheck, detekt).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 14:46:40 -05:00
JMR-devandClaude Opus 4.8 0e9f54e686 ci(spike): retry parity + adb start-server for API 37 shard PoC
Folds the #370 root-cause finding into the spike. The stable `e2e` matrix
retries its test run once; `e2e-preview` runs connectedDebugAndroidTest exactly
once, so a flaky test self-heals on API 29-36 but wedges the required gate on
API 37 (e.g. #370's SignaturesScreenTest teardown race).

Doc: adds risk item 9 (retry-parity gap + its sharding interaction — per-test
flake is NOT amplified by sharding unlike boot flake, and a per-shard retry
costs only B + T/N; framed mitigation-not-fix) and two §6 recommendations
(retry parity, mirrored into api37_e2e.py; adb start-server before the boot
loop).

PoC (ci.yml): per-shard single test retry (::warning:: on retried-but-passed)
+ adb start-server before the boot loop. The api37_e2e.py retry mirror stays a
documented recommendation (local path needs a real-emulator validation this
spike did not boot). Still DRAFT, not auto-merged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 14:40:02 -05:00
Jason Ross a84b042af5 Merge pull request #371 from JMR-dev/chore-preflight-coverage-gate
chore(preflight): run jacocoTestCoverageVerification in the fast gate
2026-07-06 14:36:55 -05:00
JMR-devandClaude Opus 4.8 66da643249 ci(spike): PoC shard API 37 preview E2E + feasibility doc
Feasibility spike for sharding the e2e-preview job (the hand-provisioned
API 37 / google_apis_ps16k 16 KB-page emulator), CI's longest leg
(~16.4-17.6 min). docs/perf/api37-e2e-sharding-spike.md breaks the leg into
fixed overhead B ~8.3 min (setup + boot + Gradle daemon/config/compile/install)
vs parallelizable test execution T ~8.8 min, models B + T/N for N=2/3/4, and
recommends N=2 (~17.1 -> ~12.7 min, ~28% off the critical path) capped by the
API 30 matrix wall (~12.0 min) beyond N=3.

DRAFT PoC (do NOT merge as-is): converts e2e-preview to a strategy.matrix.shard
[0, 1] fan-out passing AndroidJUnitRunner numShards/shardIndex through the
existing -Pandroid.testInstrumentationRunnerArguments.* channel (no GMD, no
orchestrator, no Gradle change). Artifact names gain a shard suffix;
ci-passed still lists e2e-preview once (matrix fan-in keeps the single gate).
Local preflight stays single-emulator. Relates to #258.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 14:31:40 -05:00
JMR-devandClaude Opus 4.8 042b50116c build(jacoco): scope the fail-closed encryption UI out of the JVM coverage surface (#359)
CacheEncryptionGate.kt (the gate composable, blank cover, error screen, and ephemeral
report-review screen added for #359) is pure Compose render code, structurally
unreachable from a JVM unit test the same way every other Screen file in
jacocoNonJvmTestableSurface is. Left in scope, it dragged the whole-app line ratio to
0.78, just under the 0.79 no-regression floor.

Excluded it via "**/CacheEncryptionGateKt*" rather than the usual bare
"**/CacheEncryptionGate*" pattern this list otherwise uses, because
CacheEncryptionGateViewModel is named with "CacheEncryptionGate" as a literal
prefix - the bare wildcard would also have swallowed the already JVM-tested,
94%-covered ViewModel and its sealed CacheEncryptionGateState. CacheEncryptionGateViewModel
and CacheEncryptionUnavailableException stay in scope unchanged.

Verified locally: testDebugUnitTest + jacocoTestCoverageVerification now pass, with
the line ratio recovered to about 0.807 (5,044 covered / 6,249 total lines) - the
same 5,044 covered lines as before, just a smaller, honestly-JVM-testable denominator.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 14:19:22 -05:00
JMR-devandClaude Opus 4.8 b0ca5421b6 chore(preflight): run jacocoTestCoverageVerification in the fast gate
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 14:18:40 -05:00
JMR-devandClaude Opus 4.8 bfe5d46654 fix(security): fail closed on cache-encryption load failure with an error gate (#359)
Reworks #367. When SQLCipher native library loading fails while the opt-in
encrypted cache is enabled, the app previously degraded to a plaintext cache
(a silent fail-open that defeats the feature). Now it FAILS CLOSED.

DatabaseProvisioner raises a distinct CacheEncryptionUnavailableException
instead of degrading: it does NOT open plaintext, NOT wipe the on-disk
ciphertext, and NOT write the encryptCache setting. The throw is not
memoized, so a later launch re-attempts and recovers automatically if the
library loads.

A new CacheEncryptionGate wraps the app inside AppLockGateHost (so the
passphrase is already unlocked), probes prepareCache() before any DB-backed
screen composes, and on failure shows CacheEncryptionErrorScreen with the
exact message "Error - decryption could not proceed. Native decryption
library load failure." plus a "Report a problem" action. That action
generates an EPHEMERAL PII-free report via the existing DiagnosticsCollector
(never written to ReportStore, since encryption is unavailable in that
moment) for on-screen review and explicit Copy/Save; the copy says so.

The plaintext AccountDatabase tolerates the exception so accounts stay
readable for the error gate and the report. The encryptCache setting is now
written by exactly one caller: the user Settings toggle.

Tests: fail-closed raises the signal with no plaintext open / no wipe / no
setting write / not memoized; the gate VM resolves Ready vs Unavailable and
builds the ephemeral report; an instrumented error-screen UI test and an
AccountDatabase-resilience instrumented test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 13:48:35 -05:00
JMR-devandClaude Opus 4.8 551a2df66f feat(security): back cache-key Keystore keys with StrongBox, fall back to TEE (#359)
Bind the non-exportable AES-256-GCM keys that seal the SQLCipher cache
passphrase to the hardware StrongBox secure element when the device has
one. Applied in the single shared place, AesGcmKeystoreCipher, so it
covers both the master (KeystoreCrypto) and auth-bound (DatabaseKeyCipher)
keys.

Devices without StrongBox throw StrongBoxUnavailableException at
KeyGenerator.generateKey(); a new generate-with-fallback path catches it
and regenerates a TEE-backed key so key creation still succeeds
everywhere. Guarded on API 28+ (minSdk is 29). Framing, seal/unseal, and
the missing-key policies are unchanged; the passphrase is still never
plaintext at rest and never logged.

Adds a JVM regression test for the StrongBox->TEE fallback via the
existing test seams (existingKey / a new generateKey seam).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 13:46:03 -05:00
JMR-devandClaude Opus 4.8 e436503eaf feat(scripts): device-testing perf harness
Add a cross-platform, standard-library-only Python package under
scripts/device-testing/ that replicates LibreMail's on-device performance-test
scenarios and logging capture, codifying the methodology run by hand on
2026-07-05 (Pixel 10 Pro XL).

Modules:
- breadcrumbs.py: a pure, unit-tested parser for the ImapPerf / MailReader /
  Reader / MailBackfiller breadcrumbs, plus open-correlation that reproduces the
  manual timing-tables.md figures exactly.
- adb.py: a safety-guarded adb wrapper -- an allow-list of adb subcommands and a
  deny-list + assertions on shell commands. The only sanctioned app-state
  mutation is clearing LibreMail's own cache/ (exact-match); no pm clear /
  uninstall / data wipe, and no touching databases/ files/ shared_prefs/
  datastore/ can be constructed.
- uidump.py: uiautomator XML parser + screen recognition (mailbox rows with the
  cached "Available offline" flag, reader, and the keyguard / foreign-app guards).
- scenarios.py: cold-open, message-open (uncached), back-nav, prefetch A/B
  (fetch-policy toggle) and cross-provider, each keyguard-guarded and driven
  through the guarded wrapper.
- report.py + perf_harness.py: aggregates, a timing-tables.md renderer mirroring
  the manual write-up, and the CLI (timestamped run dir with the raw logcat, a
  filtered breadcrumb extract, and the tables). --dry-run prints the exact
  command plan without touching device state.

Tests (stdlib unittest, 68 cases) validate the parser, guardrails, UI
recognition and report against the manual run's real captures (fixtures include
a verbatim perf-extract slice and the reader/lockscreen/alarm dumps). Track the
*.log fixture past the gitignore *.log rule via a scoped negation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 13:21:34 -05:00
Jason Ross 5c715e1676 Merge branch 'main' into fix-359-sqlcipher-16kb 2026-07-06 12:50:29 -05:00
Jason Ross b8d67557a0 Merge pull request #368 from JMR-dev/feat-125-imap-connection-reuse
perf(mail): enable IMAP connection reuse by default with a hardened cache
2026-07-05 22:18:11 -05:00
JMR-dev e36afc8ade changed conditional style to easier to read/maintain when (like switch) statement 2026-07-05 21:18:49 -05:00
JMR-devandClaude Opus 4.8 81a3b7ea34 refactor(mail): early-return guard in ImapConnectionCache (#357 review)
Restructure isConnectionDrop as leading guard clauses (definite-drop
types, then a not-MessagingException early return) instead of a when
expression, per maintainer review feedback on PR #368. Behavior is
unchanged; verified by the existing ImapConnectionCacheTest suite
(all 8 cases still pass), including the FolderClosedException /
StoreClosedException cases that depend on the check running before
the MessagingException .cause guard.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 20:36:38 -05:00
JMR-devandClaude Opus 4.8 060b7b1a71 Merge origin/main into feat-125-imap-connection-reuse
Resolve the IdleService.kt conflict as a union of both intents:
- #354 (already on main): foreground-service lifecycle rework —
  onStartCommand delegates to the IdleForegroundStarter seam
  (START_NOT_STICKY), cap-window skip/degrade.
- #357 Part 2 / #368: reused-connection idle-eviction sweep and
  low-battery teardown of reused connections.

In startWatchingIfNeeded(), reconcileWatchers() stays inside the
cache-lock-guarded launch and evictIdleReuseConnectionsLoop() launches as
a sibling coroutine that runs while the service lives (its original #368
placement, independent of the cache-lock guard). No behavior change to
either side.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 20:15:53 -05:00
JMR-devandClaude Opus 4.8 cc067408b8 perf(mail): enable IMAP connection reuse by default with a hardened cache
An on-device drilldown proved Gmail server-side throttles LibreMail's
connect-per-operation IMAP: every op was a fresh CONNECT+TLS+LOGIN, and
full-history backfill's body+attachment prefetch generated ~601 connections in
~22 min, tripping (and sustaining) Gmail's per-account rate/bandwidth clamp
(body download collapsed to ~4 KB/s). The `live` gauge peaked at only 5 (Gmail
allows ~15), so it is connection *volume*, not count. Outlook IMAP on the same
device opened in 2-3 s. Reusing one warm socket per account (~601 -> ~1) removes
the throttle's trigger. This wires the reuse path the #125 spike built and left
OFF (issue #357 Part 2 — connection reuse only; prefetch is a separate PR).

How it is enabled (with a safety switch):
- New `BuildConfig.IMAP_CONNECTION_REUSE` (default true) drives the production
  `ImapClient` no-arg `@Inject` constructor. To disable if a server misbehaves,
  flip it to "false" in app/build.gradle.kts — a build-config change, no Kotlin
  edit. The internal `ImapClient(reuseConnections, reuseIdleTimeoutMillis)`
  constructor stays the test/harness seam.
- Universal: applies to all providers (incl. Outlook). No per-provider caps or
  throttling here — that is a separate effort (#356/#360-#364).

Hardening `ImapConnectionCache` for production (was a spike):
- Transparent stale recovery: broadened drop detection to Angus's own
  `iap.ConnectionException` (and a MessagingException caused by one) — the real
  signal `folder.open()` throws on a server-dropped idle socket, which the
  IOException-only check missed, so the reconnect now actually fires. A dropped
  reused socket is rebuilt once and the op retried, so callers see no spurious
  error; a genuine app error (e.g. message-not-found) is never retried.
- Idle eviction: `evictIdle()` closes a connection unused past the reuse idle
  timeout (default 5 min), swept every 2 min by `IdleService`; skips any
  in-use connection.
- Teardown: `IdleService` also tears down reused connections on the low-battery
  push-teardown path (#88/#89/#90), mirroring the IDLE connection teardown.
- Concurrency: one connection per account behind a per-account mutex; the
  eviction sweep takes the lock non-blockingly so it never stalls or interrupts
  an in-flight op. Coexists with IMAP IDLE (its own separate connection).
- PII-free AppLog on the lifecycle (open / reuse-hit / reconnect-stale / evict /
  teardown) keyed by an opaque per-cache ordinal, plus the #358 ImapPerf
  breadcrumb (connect~=0ms on a reuse hit).

Tests (all via the fast gate, no emulator):
- ImapConnectionCacheTest: reuse, retry-once stale recovery, narrow drop
  detection, deterministic idle eviction (injected clock), teardown.
- ImapFolderOpenLatencyTest (GreenMail + counting proxy): N ops share one
  connection/LOGIN; a force-dropped socket is transparently reconnected; an app
  error does not reconnect; idle eviction LOGS-OUT and the next op reconnects.
- Correctness suites (ImapClientTest/ImapClientBackfillTest/MailBackfillerTest)
  pinned to reuse-off to keep their connect-per-op assertions unchanged.

Fast gate green: assembleDebug, testDebugUnitTest, compileDebugAndroidTestKotlin,
lintDebug, ktlintCheck, detekt.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 19:47:01 -05:00
Jason Ross a069188b65 Merge pull request #366 from JMR-dev/fix-354-idleservice-fgs
fix(push): stop IdleService dataSync FGS crash-loop on exhausted 24h cap (#354)
2026-07-05 19:36:27 -05:00
JMR-devandClaude Opus 4.8 921681812c fix(data): degrade encrypted cache to plaintext on SQLCipher native-load failure
On Android 15+ / SDK 37 devices with 16 KB memory pages (e.g. Pixel 10 Pro XL,
and the API-37 `google_apis_ps16k` emulator image), a native `.so` not aligned
for 16 KB pages fails to load with `UnsatisfiedLinkError` at
`SQLiteConnection.nativeOpen`. With the opt-in SQLCipher encrypted cache on, this
crashed the app on every cold start (issue #359, x4 on-device) instead of
degrading, and encryption silently never applied.

Fix: DatabaseProvisioner's encryption gate now catches `LinkageError`
(UnsatisfiedLinkError and related native-link failures) when opening/converting
the encrypted cache and degrades cleanly instead of propagating the crash — it
turns `encryptCache` off (so the next start does not re-attempt and re-wipe),
clears any on-disk ciphertext the plaintext framework opener cannot parse
(resetting its now-useless seals), and opens the cache unencrypted. The cache is
a re-syncable copy of server mail, so clearing it loses nothing that cannot be
re-fetched. PII-free AppLog.w breadcrumb on the degrade path.

Dependency: no bump needed or available. The repo already pins the newest
SQLCipher it references, `net.zetetic:sqlcipher-android:4.16.0`, which
docs/play-compliance.md certifies (ELF p_align = 0x4000) as 16 KB-aligned on
every ABI; SQLCipher has shipped 16 KB-aligned binaries since well before it, and
the other two bundled `.so` files (Compose graphics-path, DataStore
shared-counter) are already 16 KB-aligned per that doc. The graceful-degrade
catch is therefore the actionable fix.

Tests:
- Unit (DatabaseProvisionerTest): a simulated native-load failure degrades to a
  plaintext open without crashing, turns encryptCache off, and wipes + reseals an
  already-encrypted cache.
- Instrumented (DatabaseProvisionerInstrumentedTest): a fresh encrypt-on start
  loads the real SQLCipher native library and opens the keyed cache — CI's API-37
  `google_apis_ps16k` 16 KB job exercises the actual `.so` load, catching any
  future 16 KB-alignment regression.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 19:26:05 -05:00
JMR-devandClaude Opus 4.8 52503c000c fix(push): stop IdleService dataSync FGS crash-loop on exhausted 24h cap
Root cause: after #302's runtime-cap fallBackToPeriodicSync() stops the
dataSync foreground service, IdleService was restarted (START_STICKY
null-intent redelivery + explicit startForegroundService) and onStartCommand
unconditionally called startForeground(DATA_SYNC) while the rolling-24h budget
was still exhausted. The platform rejected the start with
ForegroundServiceStartNotAllowedException; it was uncaught, the process
crashed, and START_STICKY restarted straight back into the same rejection -- a
crash loop until the 24h window freed budget (#354).

Fix (IdleService.kt):
- onStartCommand now returns START_NOT_STICKY. Push is app-managed
  (LibreMailApplication.ensurePushStarted deterministically restarts it), so the
  sticky null-intent auto-restart was redundant and fired exactly when a dataSync
  FGS start is illegal.
- Guard the foreground start via a new JVM-testable IdleForegroundStarter seam:
  a ForegroundServiceStartNotAllowedException (caught via its IllegalStateException
  supertype, so no minSdk-29 class load) degrades like the cap handler --
  schedulePeriodicSync(), keep the degraded POLLING notification, stopSelf()
  promptly (avoids the "did not call startForeground in time" ANR) -- instead of
  propagating.
- Record the cap event (elapsedRealtime); while still inside the cap window,
  onStartCommand skips the now-guaranteed-illegal foreground start entirely.
- onTimeout stop path kept fast so ForegroundServiceDidNotStopInTimeException
  stays mitigated.

PII-free AppLog.w/i on the degrade paths.

Tests:
- Unit (IdleForegroundStarterTest): onStartCommand returns START_NOT_STICKY; a
  rejected start is caught and routed to degrade without propagating; the cap
  window skips the attempt; a non-ISE propagates.
- Instrumented (IdleServiceForegroundStartInstrumentedTest): the degrade path on
  a real Context -- rejection caught, periodic-sync fallback scheduled, degraded
  "instant delivery paused" notification built, watching skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 19:17:19 -05:00
Jason Ross 6cb179bd8b Merge pull request #365 from JMR-dev/feat-358-reader-perf-logging
feat(reporting): PII-free latency breadcrumbs on the message-open path (#358)
2026-07-05 18:54:32 -05:00
JMR-devandClaude Opus 4.8 35b869944e feat(reporting): PII-free latency breadcrumbs on the message-open path (#358)
Adds AppLog breadcrumbs to the message-open path so a debug report can show
where the reader's spinner time goes:

- ImapClient.withStore: per-op connect vs. work timing plus a live
  connect-per-op connection gauge (issue #125's provider-ceiling context).
- fetchBodyMarkingSeen: select/body/flag phase timings plus PII-free size
  counts (RFC822 size, body chars, attachment count).
- MailRepositoryImpl.openMessage: end-to-end open latency plus the
  cached-vs-fetched branch, keyed by accountLogRef and logSafeFolderLabel.
- ReaderViewModel: spinner-to-ready latency, split success vs. failure.

All breadcrumbs are PII-free: accounts are logged via the existing
accountLogRef hash, folders via the existing logSafeFolderLabel allowlist,
and everything else is sizes/durations/booleans only.

Fixes the 4 unit-test classes that exercise this code without mocking
android.util.Log (a throwing stub under plain JVM tests): mockkStatic(Log)
is now installed in MailRepositoryImplCoverageTest, ImapClientBackfillTest,
ImapFolderOpenLatencyTest, and ReaderViewModelActionsTest, following the
existing MailBackfillerTest/ImapClientTest conventions. detekt.yml gains two
more ForbiddenImport excludes for the newly Log-importing test files.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 17:46:48 -05:00
Jason Ross f93e2dc2f0 Merge pull request #353 from JMR-dev/ci-extract-traffic-control
ci: extract traffic-control into its own workflow file
2026-07-05 16:17:10 -05:00
JMR-devandClaude Opus 4.8 939906986b ci: extract traffic-control into its own workflow file
Move the traffic-control (runner-priority orchestration) job verbatim out of
.github/workflows/ci.yml into a new standalone workflow,
.github/workflows/traffic-control.yml, so the heavy CI jobs no longer depend
on it. The job's YAML (name, runs-on, timeout-minutes, permissions, env,
steps) and its documentation comment move unchanged; the decision core
.github/scripts/traffic_control.py is untouched and still unit-tested by the
traffic-control-tests job in ci.yml.

In ci.yml: removed the traffic-control job, dropped needs: traffic-control
from the five heavy jobs (static-analysis, debug-build, unit-tests, e2e,
e2e-preview) and from traffic-control-tests (its only needs, which would
otherwise dangle at a now-deleted job), and updated the now-stale header and
ci-passed comments to point at the extracted workflow.

The new workflow will be disabled pending a rebuild as a published GitHub
Action.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 14:48:03 -05:00
Jason Ross 6118b6ded0 Merge pull request #285 from JMR-dev/perf-187-covering-index
perf(db): covering index for unified-inbox summary scans
2026-07-05 14:31:22 -05:00
github-actions[bot] 1cd0d66fa4 Merge main into perf-187-covering-index 2026-07-05 18:48:15 +00:00
Jason Ross c3933a3612 Merge pull request #352 from JMR-dev/ci-351-pat-triggering
ci: trigger CI with the PAT so dispatches don't need manual approval (#351)
2026-07-05 13:47:39 -05:00