The paged "All inboxes" query (MessageDao.pagingUnifiedFolderSummaries:
WHERE folder = ? AND inInbox = 1 ORDER BY timestampMillis DESC) had no
folder-leading index, so it SCANned the whole messages table via
index_messages_timestampMillis and filtered folder/inInbox per row.
Add a (folder, inInbox, timestampMillis) index so the two equality
predicates become an index seek and the ORDER BY is supplied by the
index. EXPLAIN QUERY PLAN for the query goes from
SCAN messages USING INDEX index_messages_timestampMillis
to
SEARCH messages USING INDEX index_messages_folder_inInbox_timestampMillis (folder=? AND inInbox=?)
with no temp B-tree sort.
Pure additive index (no column/table change): bump the Room DB to v20
with MIGRATION_19_20 (CREATE INDEX IF NOT EXISTS), register it in
DatabaseModule, export 20.json, and add a MigrationTest that runs the
migration and asserts the index shape plus the SEARCH plan on real
Android SQLite.
Closes#187
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Give accounts a user-controlled order (issue #164). Every surface that
lists accounts -- the Settings list, the drawer account switcher, and the
compose account picker -- reads the same `ORDER BY sortOrder` query, so a
reorder in Settings is honored app-wide. In Settings a row can be
long-pressed and dragged to a new position; the order persists and
survives restart.
Data layer:
- AccountEntity gains `sortOrder` (@ColumnInfo defaultValue "0"); AccountDao
orders by it and adds insertAtEnd / reorder / nextSortOrder / setSortOrder,
the mutations wrapped in transactions.
- New accounts are appended (current max + 1) via insertAtEnd.
Migration (AccountDatabase v1 -> v2):
- ACCOUNT_MIGRATION_1_2 adds the column and backfills existing accounts by
their previous alphabetical (email) rank, so the already-shown order does
not shuffle on upgrade. Registered in AccountDatabaseModule; 2.json is
exported and AccountMigrationTest replays and validates it.
- AccountDataMigrator (the pre-#111 cache->account-db move) creates the
v2-shaped table and applies the same email-rank backfill, since
ACCOUNT_MIGRATION_1_2 does not run for that path.
UI:
- AccountReorderList drives long-press drag over a plain Column (no nested
lazy list inside the scrolling settings column, no extra dependency); the
pure reorder-index maths (commitDrag) is unit-tested.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Restores Unicode-aware case-insensitive substring search (approach A of #227).
Paging (#223) moved search to a SQL LIKE scan whose case-folding is ASCII-only,
so non-ASCII terms stopped matching case-insensitively.
Adds per-field casefold columns to `messages` (senderFold/senderEmailFold/
subjectFold/snippetFold), each = Kotlin lowercase() of its source (Unicode-aware).
Per-field (not one concatenated column) because the fields are maintained by
partial UPDATEs that don't carry all four: toEntity sets all folds, updateBody
keeps snippetFold in sync, updateHeaderContent keeps the header folds -- via thin
DAO default-method wrappers so the five call sites are unchanged. Search matches
the fold columns with a pattern built from the lowercased query.
Schema v18->v19 (additive; ASCII lower() backfill, non-ASCII rows re-fold on next
write). Adds a MigrationTest v18->v19 case and a repo test asserting the query is
casefolded.
Closes#232
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire inline images through the whole send pipeline.
Compose UI: an image-picker (image/*, persistable URI grant, mirroring the
attachment picker) behind a new toolbar button appended at the END of the
toolbar — after the block/link buttons and the font/size/align controls — so it
never shifts the bullet button the compose E2E taps without scrolling. Picking
an image adds an inline OutgoingAttachment and hands the editor a
PendingInlineImage, which RichTextEditing.insertImage drops as a [image: name]
token + RichImage(contentId) at the caret. Deleting the token drops the image:
onBodyChange reconciles inline attachments against the body's surviving cid:
references. Inline images are tracked in ComposeUiState alongside regular
attachments but kept out of the attachment-chip row.
Domain/persistence: OutgoingAttachment gains contentId/isInline; the shared
draft/outbox attachment JSON carries them (drafts need no migration — an older
draft reads back as a plain attachment). The outbox stages files by index as
before but now also stores per-file {contentId,isInline} metadata in a new
OutboxEntity.attachments column (Room 17 -> 18, MIGRATION_17_18, DEFAULT '' per
the bccAddresses precedent so fresh-install == migrated; 18.json committed). The
send worker pairs each staged file with its metadata by index (with a positional
fallback for messages queued before the column existed).
SMTP (SmtpSender): inline images wrap the body in a multipart/related, each with
a Content-ID matching the HTML's cid: and inline disposition; regular
attachments keep today's multipart/mixed shape.
Graph (GraphSender): inline fileAttachments carry isInline + contentId.
Tests: GreenMail asserts multipart/related with a Content-ID matching the cid;
GraphSenderTest asserts the inline payload; a mapper test proves an inline
image's cid<->file pairing round-trips a draft save/reopen; RichTextEditing
tests cover insertImage (token/RichImage placement + offset shift + html
round-trip); MigrationTest gains migrate17To18. Reader-side cid: rendering stays
out of scope (follow-up).
Closes#77
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Inline images in rich HTML emails (embedded via Content-ID and
<img src="cid:...">, e.g. USPS Informed Delivery digests) were listed
under Attachments with a download button and never rendered in the body.
Two bugs combined; both are fixed here.
1. Misclassification: ImapClient classified any part with a filename as
an attachment, sweeping inline images (which carry a filename AND a
Content-ID under Content-Disposition: inline) into the list. A part is
now a downloadable attachment only when its disposition is attachment,
or it has a filename but no Content-ID; an inline image is collected
separately and excluded from the displayed list (AttachmentDao filters
contentId IS NULL). The Content-ID is read via MimePart.getContentID()
so it resolves from IMAP BODYSTRUCTURE rather than a per-part header
fetch that Angus leaves unpopulated.
2. No rendering path: HtmlBody's WebViewClient now overrides
shouldInterceptRequest to resolve cid:<id> to the matching part's
bytes (backing the CSP's existing cid: allowance). Content-ID is
threaded end-to-end through AttachmentPart, Attachment,
AttachmentEntity, and MailRepository.inlineImages(); ReaderViewModel
surfaces the cid->bytes map to the WebView.
Schema: adds attachments.contentId (v16 -> v17, MIGRATION_16_17).
Tests: MIME-part classification (inline+cid excluded, real/disposition/
filename-only kept), a GreenMail multipart/related round-trip, the
cid->bytes resolver, repository inlineImages(), the DAO display filter,
and the v16->v17 migration.
Closes#133
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Main advanced to @Database v15 (the #66 folder hierarchyDelimiter
migration). Renumbered the account-tables-drop migration 14->15 to
15->16 and bumped the cache DB to v16; this exports the v16 schema
(main's v15 delimiter schema minus the moved account tables). Main's
own 15.json is kept unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Accounts, credentials, per-account settings and signatures lived in the
same libremail.db that SQLCipher encrypts under the auth-bound passphrase
when app-lock + encrypted-cache are on. A genuine key invalidation
(biometric re-enrollment or lock removal/re-add) made that file
undecryptable, and the "clear + re-sync" recovery wiped the accounts and
stored credentials along with the mail cache, dropping the user into
onboarding (issue #111).
Move those four tables into a new plaintext AccountDatabase
(libremail-accounts.db) that is never bound to the auth key. Credentials
stay AES-GCM sealed at the column level by the surviving non-auth
KeystoreCrypto master key, so the only secret never touches disk in the
clear. A cache-key invalidation now wipes only libremail.db; the user
stays signed in.
- AccountDatabase (v1) + AccountDatabaseModule; the cache DB drops to v15
via MIGRATION_14_15. DAOs are unchanged and re-provided from the new DB,
so no injection site changes.
- AccountDataMigrator performs the one-time cross-DB copy at startup,
before Room opens either database. It attaches the cache (with its
resolved passphrase, so an encrypted source is handled) and copies with
INSERT OR IGNORE. It is crash-safe and idempotent: the source is dropped
only by MIGRATION_14_15 after the copy, a re-run never duplicates or
overwrites, and it runs after the clear-pending wipe so an unrecoverable
cache degrades to "nothing to move" instead of blocking.
- Exported schemas for both databases; MigrationTest asserts the account
rows/backfills survive to v14 then are dropped at v15, plus a dedicated
14->15 test. AccountDataMigratorTest covers the plaintext + encrypted
copy, idempotency, a DDL-vs-Room drift guard, and end-to-end survival of
a simulated cache wipe.
Closes#111
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
parentOf() re-inferred the IMAP hierarchy separator from each folder's name,
relying on an unenforced invariant (displayName == fullName.substringAfterLast(
separator)) established three layers from where ImapClient reads the
authoritative JavaMail folder.separator and then discards it.
Carry that separator through FetchedFolder -> FolderEntity -> Folder and split a
folder's parent on it. Fall back to the old name inference only for legacy rows
whose delimiter is null, until the next folder refresh (delete-then-insert)
backfills the real value.
Adds a nullable folders.hierarchyDelimiter column via a Room v14 -> v15 migration
with the exported v15 schema, and registers MIGRATION_14_15 in DatabaseModule so
existing v14 installs actually upgrade (provideDatabase configures no destructive
fallback, so an unregistered migration would crash every upgrading user).
Tests: JVM unit tests for parentOf() (persisted vs. null delimiter, incl. the
case where inference cannot locate the parent) and the FetchedFolder->entity
round-trip; an instrumented v14->v15 migration test plus the chain-replay
assertion.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
snippetOf() stripped only tag delimiters with a single regex on every
body, HTML or not: <style>/<script> text leaked into HTML snippets,
entities stayed encoded, and plain-text bodies had literal <...> text
eaten as if it were markup.
Replace it with Snippet.of(body, isHtml), which finally consults the
isHtml flag both call sites already had: HTML bodies go through
HtmlToText (script/style content dropped, tags stripped, entities
decoded), plain text gets no markup handling at all; both paths keep
the whitespace collapsing and the 140-char cap. HtmlToText's entity
decoding is now a single-pass decoder that also handles decimal/hex
numeric character references and never re-decodes produced characters.
Snippets are persisted when a body is first fetched and never
re-derived, so existing rows would keep their broken snippets forever;
a data-only v13->v14 migration re-derives every cached row's snippet
with the corrected logic (schema unchanged relative to v13, exported
14.json committed).
Closes#85
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Addresses the review of PR #46 (#12/#13):
- Age-retention backfill/prune loop: mark a folder complete at the
retention floor and resume from the persisted nextBeforeUid low-water
mark; loosening resumes via AccountRepository.resetBackfillProgress.
- Guard the windowed reconcile bound to the lowest positive UID so a
getUID==-1 message can't collapse it and wipe backfilled history.
- Order count-based retention by uid DESC to match the fetch window,
ending the re-fetch/re-prune churn for high-UID/old-Date messages.
- BackfillWorker chains slices while work remains.
- Extract shared effectiveRetention / isActiveNetworkUnmetered /
attachmentCacheDir helpers; remove dead deleteSyncedNotIn/getForAccount;
refresh only pre-existing rows in persistBatch; add composite index
(accountId, folder, uid) with migration + regenerated 13.json.
Adds an age-floor prune regression test. Fast gate + androidTest compile
green on JDK 21.
Follow-ups filed for below-the-cut findings: #93, #94, #95, #96.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve the Room schema-version collision: main's PR #54 added MIGRATION_11_12 (folders.specialUse), colliding with this branch's v11->v12 uid/retention/backfill migration. Renumbered ours to MIGRATION_12_13 — the two migrations touch disjoint tables, so ours stacks cleanly on top — bumped the DB to version 13, kept main's 12.json as the v12 schema and regenerated 13.json, and renamed Migration11To12Test -> Migration12To13Test.
Verified locally: assembleDebug, testDebugUnitTest, lintDebug, ktlint, detekt, compileDebugAndroidTestKotlin, and Migration12To13Test on an API 37 emulator all pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The drawer rendered every standard-role folder with a generic friendly
name (e.g. "Drafts") and discarded the server name, so a Gmail account
with both a provider built-in folder and a same-named user folder showed
two identical entries (Drafts, Archive, Spam).
De-duplicate labels provider-agnostically: when 2+ folders would render
the same name, the provider's built-in special folder (identified by RFC
6154 SPECIAL-USE flags, now persisted on the folder cache) gets the
provider name appended ("Archive - Gmail"), a nested user folder gets its
parent location ("Reports (Work)"), and a top-level user folder keeps its
plain name. Only triggers on a real collision, so stock accounts are
unchanged.
Adds a `specialUse` column to the folders table (Room v11 -> v12).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the fixed 50-message-per-folder header cap with a background,
resumable full-history backfill, and add a user-configurable device-only
retention limit that prunes local mail beyond it without ever deleting from
the server.
- ImapClient.fetchOlderThan pages a folder backwards in bounded batches,
locating the boundary by binary search over message numbers (O(log n) tiny
UID fetches, memory bounded to one batch).
- MailBackfiller + BackfillWorker page each synced folder newest→oldest,
persisting a per-folder boundary in a new backfill_progress table so a run
interrupted by process death / network loss resumes exactly where it stopped.
Runs off the sync mutex, so foreground sync / pull-to-refresh stay responsive.
- MailSyncer now reconciles server deletions only within the recent UID window
(deleteSyncedInWindowNotIn) instead of wiping everything outside the recent
50, so backfilled history survives each foreground sync. A materialized
messages.uid column powers the windowed reconcile and backfill boundary.
- Body/attachment prefetch still honours FetchPolicy (headers first).
- Per-account count/age overrides (nullable) with a global default; 0 = keep
everything (the default, matching #12).
- MailPruner + PruneWorker delete local rows beyond the limit (cascading
attachment rows + on-disk cache), never issuing a server delete. Deletes are
chunked under SQLite's 999-parameter limit.
- Precedence with backfill: backfill pauses (does not complete) at the
retention floor and both jobs share a maintenance mutex, so they never
contend; foreground fetch is also capped by the count so it can't re-download
what pruning just trimmed.
- Settings UI for the global default and per-account override, with copy making
clear it is device-only, not the server.
Room schema v9→v10 (migration + exported schema + MigrationTestHelper test).
GreenMail tests prove the backfill caches >50 and resumes after interruption;
pruning tests cover count/age limits and never touch the server.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring rich composition to LibreMail (issues #36, #37, #38, and #23).
#36 HTML editor + toolbar
- New pure, JVM-testable rich-text model (`richtext/`): RichTextContent with
inline styles + links and block markers ("• ", "N. ", "> "), serializing to a
narrow email-safe HTML subset and back (fromHtml is a faithful inverse).
- Rich editor in ComposeScreen with a bold/italic/underline, bulleted/numbered
list, block-quote, and link toolbar, backed by AnnotatedString. Unformatted
text stays plaintext-only (null HTML) so it feels unchanged and is accessible.
- #23: rounded corners on the compose fields/body via MaterialTheme.shapes.
#37 multipart/alternative + reply/forward quoting
- SmtpSender builds multipart/alternative (text/plain + text/html), nested in
multipart/mixed when there are attachments; GraphSender sends HTML content.
- HtmlToText produces a readable text/plain fallback; ReplyBuilder quotes HTML
originals as clean blockquotes (tags stripped) without corruption.
- HTML body persists/restores through drafts and the outbox (new nullable
bodyHtml columns; Room v9->v10 migration + schema).
#38 signatures
- New signatures table (multiple per account, one default) + repository/DAO;
migration backfills the existing per-account signature as the default.
- Rich signatures reuse the #36 editor; a Signatures management screen (list,
add/edit/delete, set default) is linked from per-account settings.
- The account's default signature auto-inserts on new compose / reply / forward
(honoring the enable toggle), placed above the quote, and stays editable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add intent filters so LibreMail handles mailto: (ACTION_VIEW / SENDTO)
and email SEND / SEND_MULTIPLE intents, opening a prefilled compose screen.
- MailtoParser: pure RFC 6068 parser (multiple recipients, to/cc/bcc/
subject/body, percent-encoding; preserves a literal '+'); JVM-tested.
- IntentComposeParser: builds a ComposePrefill from a mailto: URI or the
EXTRA_EMAIL/CC/BCC/SUBJECT/TEXT share extras.
- MainActivity parses the launch/new intent and hands a one-shot prefill to
the NavHost (guarded against config-change duplication).
- Compose form gains a Bcc field; Routes carry cc/bcc/body deep-link args.
- bcc wired end-to-end: OutgoingMessage, SMTP + Graph senders, and outbox +
drafts persistence via Room migration v9 -> v10.
- Multi-account send is served by the existing From picker on compose.
Default mail app: Android exposes no public RoleManager email role, so the
intent filters are what make LibreMail appear on the system "Open by default"
/ default-apps screen where the platform/OEM supports it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a per-account settings area (reached by tapping an account in
Settings), starting with signatures and notifications.
- Storage: new Room `account_settings` table (schema v9, MIGRATION_8_9)
with a cascading foreign key to `accounts`; AccountSettings model and
AccountSettingsRepository (a missing row resolves to defaults).
- Signatures: plain-text per-account signature (RFC 3676 "-- "
delimiter) auto-inserted below new messages and reply/forward drafts,
and swapped when the From-account changes.
- Notifications: one notification channel + channel group per account so
Android manages sound/vibration/importance (deep-linked from the app)
and the shade bundles per account, plus an in-app per-account on/off
gate. minSdk 29 >= API 26, so channels are always available (no
pre-channel fallback needed).
- UI: per-account settings screen (signature field/toggle, notification
toggle, system deep-link, remove account); shared settings components.
Verified: JVM unit tests, lintVitalRelease (NewApi), and the full
instrumented suite (30/30) on the API 37 emulator.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Folder view: a left navigation drawer lists each account's IMAP folders;
tapping one browses and caches that folder's mail. IMAP UIDs are unique only
within a folder, so message identity, the fetch/read/flag/delete paths, sync,
and the Room cache all became folder-aware (id = "accountId:folder:uid"; new
`folder` column; schema v7->v8). Standard folders (Inbox/Sent/Drafts/Spam/Trash/
Archive) surface with friendly names + icons via RFC 6154 SPECIAL-USE attributes
with a case-insensitive name fallback; the multi-account drawer adds an account
switcher and a unified "All Inboxes". INBOX stays the only auto-synced,
IDLE-watched, notifying folder; other folders sync on demand.
Lower minSdk 33 -> 29 for a rolling ~7-year Android support window; guard the
API-33 POST_NOTIFICATIONS runtime request accordingly.
Tests and CI:
- Bump espresso-core 3.6.1 -> 3.7.0 so Compose UI tests run on API 37
(3.6.1's InputManagerEventInjectionStrategy reflects a removed hidden method).
- New coverage across layers: FolderRoleTest, ImapClientTest folder cases,
MailboxViewModelTest, MailRepositoryImplTest folder routing, a FolderDrawer
Compose UI test, and LibreMailDatabaseTest folder DAO/reconcile tests.
- Gradle Managed Devices + a CI E2E matrix over every API 29-36; a single
"CI passed" gate job fans in all jobs and is required by branch protection.
- Non-blocking, custom-provisioned API 37 (preview) E2E job with image caching.
- Build + unit-test jobs run on arm64 (ubuntu-24.04-arm); emulators stay x86_64.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Addresses findings from a full-repo review across the mail, sync, persistence,
auth, and UI layers.
Send / outbox:
- Stop the Graph->SMTP fallback from duplicating a message when a Graph send may
already have been accepted; leave indeterminate sends queued for the user.
- Parse RFC822 display-name recipients on the Graph path.
- Preserve attachment order (staged in indexed subdirectories).
Data safety (schema v7):
- Disable cloud/device backup of the Keystore-encrypted credential DB.
- Add the missing v1->v2 migration and drop the destructive migration fallback.
- Normalize the messages.isHtml default and add an attachments->messages
ON DELETE CASCADE foreign key (no more orphaned attachment rows).
Concurrency:
- Serialize syncing and per-account OAuth token refresh; cache tokens by expiry.
- Synchronize Android Keystore key creation.
- Make a sync's persist+notify non-cancellable so an IDLE renewal can't drop it.
Notifications / UI:
- Per-message notifications under a group + summary instead of one overwriting id.
- Wire the "load remote images" and "allow STARTTLS" settings.
- Harden the reader WebView (scheme allowlist + user gesture; no reload on
recomposition); refresh headers without reverting optimistic read/star flags.
- Persist draft attachments; keep server-search hits out of the inbox; encode
the reader navigation argument.
Build / test:
- Export Room schemas; support a real release keystore; add unit/db tests.
- Remove dead Gmail account-setup code left after the sign-in removal.
Verified: debug + release (R8) + androidTest compile; unit tests pass;
MIGRATION_6_7 matches the generated v7 schema.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
JMR-devandClaude Opus 4.8