Addresses the below-cut data-core review nits from #313:
- SignatureRepository.delete: wrap delete + default-promotion in one SignatureDao
@Transaction (deletePromotingDefault) so a crash between them can't leave an
account with signatures but no default; log the promotion (PII-free).
- SignatureRepository.create: move the count-then-default check-then-act into a
SignatureDao @Transaction (insertMakingFirstDefault) so two concurrent
first-creates can't both become default.
- AccountSettingsRepository.update: route the read-modify-write through an
AccountSettingsDao @Transaction (readModifyWrite) so concurrent per-field
setters can't clobber each other.
- MailRepositoryImpl expunge/move/move-by-role: chunk the unbounded
getRoutingByIds/deleteByIds IN(:ids) queries (500/chunk) like MailPruner,
removing the latent SQLITE_MAX_VARIABLE_NUMBER crash.
- MessageDao.observeSummaries: remove the dead whole-table projection (superseded
by Paging #124/#214); migrate test/debug-probe callers to getById or the paged
query (which now guards the #51 CursorWindow regression).
- AccountDataMigrator: fix stale KDoc (schema is v2 with sortOrder, not v1).
- DatabaseEncryption.migrate: also sweep the stale -journal sidecar (journal_mode
= DELETE), matching AccountDataMigrator's sweep.
Unit tests updated for the repository delegations; instrumented DAO tests cover
the new @Transaction behaviour; MailRepositoryImplTest covers the chunk split;
DatabaseEncryptionTest covers the -journal sweep.
Closes#313
Investigate Mergify (free-for-OSS merge queue + batching + speculative checks)
as the right-way replacement for the hand-rolled traffic-controller
(autoupdate.yml + ci-trigger.yml + mothballed traffic-control.yml) and the
manual serial-bump grind, now that GitHub's native merge queue is org-only and
unavailable to a user account.
Proposal only — NO live .mergify.yml, nothing activates:
- docs/ci/mergify-integration-spec.md: how the queue coexists with the single
`CI passed` gate; the require-up-to-date x merge-commits x batching trilemma
and its resolution (Phase 1 serial keeps the rule literally; Phase 2 merge-batch
moves the up-to-date GUARANTEE into the queue); P0-P9 -> priority_rules mapping;
what it replaces; interaction with path-filter/sharding/wedge-diag; risks;
phased adopt recommendation.
- docs/ci/mergify.yml.proposed: annotated, NOT-active proposed config.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The spike commits already implemented the N=2 shard matrix (numShards/
shardIndex via -Pandroid.testInstrumentationRunnerArguments.*), per-shard
test-retry parity, adb start-server before the boot loop, and shard-suffixed
artifact names. This drops the SPIKE / DRAFT "do not merge as-is" framing from
the ci.yml comments and reframes docs/perf/api37-e2e-sharding-spike.md from a
feasibility spike into the adopted design, so the change is mergeable as-is.
Also fixes the doc's section 3a example, which showed 1-based shardIndex values
[1, 2]; shardIndex is 0-based (0..numShards-1) and the implementation correctly
uses matrix.shard: [0, 1] -- [1, 2] would run an empty bucket and silently drop
half the suite.
Fan-in unchanged and verified: ci-passed still lists e2e-preview once; GHA
matrix aggregation makes its result `failure` if either shard fails, so both
shards must pass for the gate to go green. Branch protection requires the
"CI passed" context (not the per-leg "E2E (API 37 preview) (N)" check names),
so no branch-protection change is needed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Folds the #370 root-cause finding into the spike. The stable `e2e` matrix
retries its test run once; `e2e-preview` runs connectedDebugAndroidTest exactly
once, so a flaky test self-heals on API 29-36 but wedges the required gate on
API 37 (e.g. #370's SignaturesScreenTest teardown race).
Doc: adds risk item 9 (retry-parity gap + its sharding interaction — per-test
flake is NOT amplified by sharding unlike boot flake, and a per-shard retry
costs only B + T/N; framed mitigation-not-fix) and two §6 recommendations
(retry parity, mirrored into api37_e2e.py; adb start-server before the boot
loop).
PoC (ci.yml): per-shard single test retry (::warning:: on retried-but-passed)
+ adb start-server before the boot loop. The api37_e2e.py retry mirror stays a
documented recommendation (local path needs a real-emulator validation this
spike did not boot). Still DRAFT, not auto-merged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Feasibility spike for sharding the e2e-preview job (the hand-provisioned
API 37 / google_apis_ps16k 16 KB-page emulator), CI's longest leg
(~16.4-17.6 min). docs/perf/api37-e2e-sharding-spike.md breaks the leg into
fixed overhead B ~8.3 min (setup + boot + Gradle daemon/config/compile/install)
vs parallelizable test execution T ~8.8 min, models B + T/N for N=2/3/4, and
recommends N=2 (~17.1 -> ~12.7 min, ~28% off the critical path) capped by the
API 30 matrix wall (~12.0 min) beyond N=3.
DRAFT PoC (do NOT merge as-is): converts e2e-preview to a strategy.matrix.shard
[0, 1] fan-out passing AndroidJUnitRunner numShards/shardIndex through the
existing -Pandroid.testInstrumentationRunnerArguments.* channel (no GMD, no
orchestrator, no Gradle change). Artifact names gain a shard suffix;
ci-passed still lists e2e-preview once (matrix fan-in keeps the single gate).
Local preflight stays single-emulator. Relates to #258.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An on-device drilldown proved Gmail server-side throttles LibreMail's
connect-per-operation IMAP: every op was a fresh CONNECT+TLS+LOGIN, and
full-history backfill's body+attachment prefetch generated ~601 connections in
~22 min, tripping (and sustaining) Gmail's per-account rate/bandwidth clamp
(body download collapsed to ~4 KB/s). The `live` gauge peaked at only 5 (Gmail
allows ~15), so it is connection *volume*, not count. Outlook IMAP on the same
device opened in 2-3 s. Reusing one warm socket per account (~601 -> ~1) removes
the throttle's trigger. This wires the reuse path the #125 spike built and left
OFF (issue #357 Part 2 — connection reuse only; prefetch is a separate PR).
How it is enabled (with a safety switch):
- New `BuildConfig.IMAP_CONNECTION_REUSE` (default true) drives the production
`ImapClient` no-arg `@Inject` constructor. To disable if a server misbehaves,
flip it to "false" in app/build.gradle.kts — a build-config change, no Kotlin
edit. The internal `ImapClient(reuseConnections, reuseIdleTimeoutMillis)`
constructor stays the test/harness seam.
- Universal: applies to all providers (incl. Outlook). No per-provider caps or
throttling here — that is a separate effort (#356/#360-#364).
Hardening `ImapConnectionCache` for production (was a spike):
- Transparent stale recovery: broadened drop detection to Angus's own
`iap.ConnectionException` (and a MessagingException caused by one) — the real
signal `folder.open()` throws on a server-dropped idle socket, which the
IOException-only check missed, so the reconnect now actually fires. A dropped
reused socket is rebuilt once and the op retried, so callers see no spurious
error; a genuine app error (e.g. message-not-found) is never retried.
- Idle eviction: `evictIdle()` closes a connection unused past the reuse idle
timeout (default 5 min), swept every 2 min by `IdleService`; skips any
in-use connection.
- Teardown: `IdleService` also tears down reused connections on the low-battery
push-teardown path (#88/#89/#90), mirroring the IDLE connection teardown.
- Concurrency: one connection per account behind a per-account mutex; the
eviction sweep takes the lock non-blockingly so it never stalls or interrupts
an in-flight op. Coexists with IMAP IDLE (its own separate connection).
- PII-free AppLog on the lifecycle (open / reuse-hit / reconnect-stale / evict /
teardown) keyed by an opaque per-cache ordinal, plus the #358 ImapPerf
breadcrumb (connect~=0ms on a reuse hit).
Tests (all via the fast gate, no emulator):
- ImapConnectionCacheTest: reuse, retry-once stale recovery, narrow drop
detection, deterministic idle eviction (injected clock), teardown.
- ImapFolderOpenLatencyTest (GreenMail + counting proxy): N ops share one
connection/LOGIN; a force-dropped socket is transparently reconnected; an app
error does not reconnect; idle eviction LOGS-OUT and the next op reconnects.
- Correctness suites (ImapClientTest/ImapClientBackfillTest/MailBackfillerTest)
pinned to reuse-off to keep their connect-per-op assertions unchanged.
Fast gate green: assembleDebug, testDebugUnitTest, compileDebugAndroidTestKotlin,
lintDebug, ktlintCheck, detekt.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 19:47:01 -05:00
Jason RossClaude Opus 4.8github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(onboarding): request notification permission after welcome screen renders
The POST_NOTIFICATIONS request fired from a MainActivity-root
NotificationPermissionEffect whose LaunchedEffect(Unit) ran on the very
first composition, so the system dialog could pop the instant the icon
was tapped — overlapping cold start/splash before any onboarding context
was on screen.
Move the effect into OnboardingWelcomeScreen so it fires once that screen
(the onboarding start destination) is composed and visible, with the
welcome content behind the dialog. Already-onboarded users launch
straight into the mailbox and never compose the welcome screen, so they
are unaffected; the API 33+ gate and the already-granted no-op are
preserved unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(onboarding): grant POST_NOTIFICATIONS in onboarding E2E to fix API 33+ flow
Moving the notification-permission request into OnboardingWelcomeScreen
(#151) means the system POST_NOTIFICATIONS dialog now pops when that
screen composes. On API 33+ (where it became a runtime permission) the
dialog backgrounded the activity mid-flow, so OnboardingFlowTest failed
with "No compose hierarchies found" on API 33/34/35/36/37 while API
29–32 stayed green.
Pre-grant the permission via a GrantPermissionRule so the dialog never
appears during the flow, guarded for API 33+ (the permission does not
exist below TIRAMISU, so grant nothing there to avoid erroring on older
devices). Adds the androidx.test:rules dependency that provides the rule.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Recipient autocomplete's READ_CONTACTS permission was requested lazily on
every compose-screen open (a LaunchedEffect(Unit)), re-prompting users who
had declined. Move the request to a dedicated, skippable onboarding step and
add a Settings entry to turn it on later, each with an in-context rationale.
- #127: new skippable ONBOARDING_CONTACTS step (mirrors the battery step),
requested once. ComposeScreen no longer prompts; it only reads the current
grant on resume, so a grant made later (e.g. from Settings) still takes
effect the next time compose opens.
- #128: the onboarding step and the Settings request show a short rationale
(contacts are used only for on-device autocomplete, never uploaded) and
handle shouldShowRequestPermissionRationale so a re-request explains itself.
docs/play-permissions.md updated to match.
- #129: Settings -> Contacts -> Recipient autocomplete reflects on / off /
blocked-in-settings; requests in-app when grantable, deep-links to the app's
system settings when permanently denied.
Graceful degradation is preserved: ContactsRepository.search still runCatch-es,
ComposeViewModel.searchContacts() still guards on contactsAllowed, and the
suggestion list still renders only when non-empty.
Adds a pure ContactPermissionDecision (JVM unit-tested), extends the onboarding
view-model tests, and adds Compose UI tests for the onboarding step
(skip / grant / deny / rationale) and the Settings row states.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Prototype the per-account connection reuse the #125 investigation recommended
and deferred, behind an OFF-by-default flag so it cannot destabilize `main`.
- ImapConnectionCache: keeps one authenticated Store alive per account, guarded
by a per-account mutex, keyed by connection identity (not the rotating
secret), with lazy catch-and-retry-once stale handling. No eviction policy
yet beyond an explicit closeReusedConnections() hook.
- ImapClient gains a `reuseConnections` flag (default false via the @Inject
no-arg constructor). With it off, withStore is byte-for-byte the previous
connect + LOGOUT-per-call; with it on, calls borrow the kept-alive Store.
- ImapFolderOpenLatencyTest flips the flag on: the same real-IMAP operations
that cost N connections / N LOGINs collapse to 1 connection / 1 LOGIN, with
the necessary per-open EXAMINE unchanged (proven via CountingImapProxy +
GreenMail; localhost is ~0 RTT so this proves structure, not wall-clock).
- docs/perf/issue-125-connection-reuse-spike.md: prototype design, the
flag-off-vs-on proof, per-decision trade-offs, and the refined real-device
validation plan. References #125; does not close it (needs device validation).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The unified inbox query (WHERE folder = ?, no accountId) has no folder-leading
index, so it scans in timestamp order and materializes the whole unified inbox
(~4k rows at a 20k cache) into memory on every emission. Apply Paging 3 to the
unified browse path so query, mapping, and recomposition cost scale with the
visible window, not the total cache.
- MessageDao.pagingUnifiedFolderSummaries: a PagingSource over the folder's
synced rows (inInbox = 1); unified search keeps the whole-folder query so it
can still surface transient server-search hits.
- MailRepository.pagedUnifiedFolderMessages: a Pager (pageSize 40, initialLoad
120, no placeholders) mapping summaries to domain.
- MailboxViewModel.pagedMessages: paged while browsing the unified inbox, else
empty; the messages list flow stays empty in that state so the whole cache is
never materialized. Selection captures each row's accountId at tap time, so
"Move" still resolves the selection's account without an in-memory list.
- MailboxScreen renders the unified browse list via collectAsLazyPagingItems;
per-account and search views render the flat list unchanged (issue #86 stays
flat).
Profiling (docs/perf/issue-124-unified-inbox-paging.md) on an api29 emulator:
current whole-inbox first-emit ~24.6 ms at a 20k cache vs. the paged first page
~6.8 ms and flat regardless of cache size (~3.6x). EXPLAIN QUERY PLAN shows the
paged query still stops early on the existing timestamp index, so no
(folder, ...) index and no schema migration are added.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Investigate IMAP folder-open latency (follow-up to #86). Localhost GreenMail
has ~0 RTT, so real wall-clock latency can't be measured here; instead this
pins the folder-open round-trip STRUCTURE deterministically.
Finding: ImapClient.withStore wraps every operation in its own short-lived
Store, so each folder-open pays a full CONNECT + TLS + LOGIN + EXAMINE +
FETCH + LOGOUT. Only EXAMINE + FETCH is intrinsic to opening a folder; the
whole connection-setup group is avoidable on the 2nd+ operation if a
connection were reused. Optimistic render-from-cache already exists
(selectFolder renders cached rows; the network sync is a background refresh).
Adds:
- CountingImapProxy: a localhost TCP proxy that forwards a cleartext IMAP
session to GreenMail while counting TCP connections and parsing IMAP
command words.
- ImapFolderOpenLatencyTest: asserts the current no-reuse behaviour (N opens
=> N connections and N LOGINs; list+read => 2 connections) against a real
in-process IMAP server. Doubles as the harness to validate a future
connection-reuse fix (flip the counts to assert reuse).
- docs/perf/issue-125-imap-folder-open.md: the per-open round-trip sequence,
avoidable vs. necessary round-trips, and the recommended per-account
connection-reuse/keep-alive mitigation with its IDLE / thread-safety /
battery / stale-connection constraints.
Analysis + harness only; the connection-reuse fix is deferred pending
real-network + real-device measurement (see the doc's measurement plan), so
this references #125 without closing it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The mailbox list observed the entire `messages` table (observeSummaries, no
WHERE/LIMIT), mapped every cached row to a domain Message, and filtered down to
the visible account+folder in MailboxViewModel — so its cost scaled with the
whole cache and re-ran on every write to `messages` (IDLE delivery, a flag
toggle, a backfill page, any folder sync). On a 20k-row cache that is ~125 ms of
work per unrelated write.
Push the account/folder filter into SQL (observeFolderSummaries /
observeUnifiedFolderSummaries, exposed via observeFolderMessages /
observeUnifiedFolderMessages) and flatMapLatest the ViewModel over the selected
account+folder. The only remaining client-side pass separates the normal list
from an active search over the small folder-scoped set.
Validated on an emulator against 1k/5k/20k-row caches (docs/perf/issue-86-
profiling.md): the account-scoped query is ~1.5 ms flat (~80x faster at 20k) and
is already served by the existing (accountId, folder, uid) index — so NO
composite index and NO schema migration are added. The ticket's proposed
(accountId, folder, inInbox, timestampMillis) index changes timing only within
noise and isn't even preferred by SQLite's planner. The unified "All inboxes"
view stays an O(N) folder scan (still 5.6x better) and is a follow-up for paging;
IMAP latency on folder open is a separate, unmeasured concern.
Closes#86
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rewrite the manual-dispatch release.yml into the issue-#19 pipeline:
v* tag push (or dispatch with dry-run/re-release inputs) runs the fast
CI gate, builds bundleRelease + assembleRelease signed from base64
keystore secrets (falling back to *-unsigned artifacts when unset),
generates a Conventional-Commit changelog and SHA-256 checksums, then
creates the GitHub release and fans out to secret-gated Google Play
publish (staged rollout supported), a documented Galaxy Store manual
stub, and an S3-compatible archive under releases/<tag>/. Every
credentialed stage skips with a clear notice while the store accounts
(#16/#17/#18) don't exist yet; no secret lives in the repo and
app/build.gradle.kts is unchanged. docs/release.md documents the
secrets, flows, and per-store manual fallbacks.
Part of #19
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Repo-actionable deliverables for the Google Play compliance work (issue #17),
every claim verified against the code and the built release artifacts:
- PRIVACY.md: user-facing privacy policy (device-local mail cache, optional
SQLCipher encryption, traffic only to the user's own mail provider,
on-device-only contacts autocomplete, strictly local opt-in debug reports,
no ads/analytics/tracking SDKs).
- docs/play-data-safety.md: Play Data safety questionnaire mapping -- answer
'no data collected/shared' with per-category code evidence, the policy
exemptions relied on, a dependency audit, and a conservative fallback.
- docs/play-permissions.md: merged-manifest permission audit (incl. the
WorkManager-injected WAKE_LOCK / RECEIVE_BOOT_COMPLETED) with paste-ready
Console justifications for READ_CONTACTS, POST_NOTIFICATIONS, and the
FOREGROUND_SERVICE_DATA_SYNC declaration + demo-video script.
- docs/play-compliance.md: verified targetSdk 37 (requirement: 35+), 16 KB
page-size compliance (all packaged .so PT_LOAD p_align=0x4000, incl.
sqlcipher-android 4.16.0), bundleRelease AAB check, the Gmail-app-password /
no-CASA OAuth note, the console-steps checklist with drafted content-rating
and listing answers, and repo findings (push-mail default vs docs, README
minSdk/app-lock drift, debug-key release fallback).
- README.md: link PRIVACY.md and note the no-Google-OAuth/no-CASA status
(fuller README pass stays issue #20).
Part of #17.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CI runs ktlint/detekt and the E2E suites (not Android lintDebug), and the
KnownVuln rationale should not imply blanket TLS enforcement.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Prepare for F-Droid publication (issue #16):
- docs/fdroid-compliance.md: full dependency license audit (release
runtime classpath + buildscript classpath — all FOSS, no Play
Services/Firebase, no non-free Gradle plugins), an anti-feature
review of actual app behavior (none to declare: debug reporting is
opt-in/local-only with no endpoint by default, Android Backup is
gated off by default, Outlook OAuth is optional per-account with a
public client id), a complete network-surface inventory, and the
clean-room build verification (assembleRelease succeeds with no
secrets.properties).
- app/build.gradle.kts: stop embedding AGP's dependency-info block (a
Google-Play-encrypted dependency list in the APK signing block) in
APKs/bundles — a known F-Droid inclusion/reproducibility blocker.
- fastlane/metadata/android/en-US/: store listing (title, short/full
description, changelog for versionCode 1) that F-Droid reads from
the repo; listing .txt files deliberately carry no license headers.
- docs/fdroid/org.libremail.app.yml: commented template + instructions
for the eventual fdroiddata build recipe (submission out of scope).
- README.md: F-Droid section pointing at the above.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
JMR-dev
