Author SHA1 Message Date
JMR-dev 700ce071ae feat(reporting): anonymize + encrypt debug reports before opt-in upload (#34)
Advances the client (app-repo) slice of the debug-report ingest pipeline: the
existing opt-in ReportUploadWorker now runs a best-effort PII anonymization pass
and seals each report with end-to-end (envelope) encryption to a maintainer
public key BEFORE it leaves the device, and fails closed if it cannot.

- ReportAnonymizer: pre-upload redaction of the free-text comment + log lines
  (emails, host:port, IPv4, JWTs, Bearer/Basic, key=value secrets). Re-scrubs the
  stack trace. Deliberately retains the user-supplied reply-to email (#159).
- ReportPayloadEncryptor / HybridReportPayloadEncryptor: AES-256-GCM content key
  wrapped with RSA-OAEP-SHA256 to a public key; JSON envelope. JCA only, no new
  dependency, no GMS -> F-Droid-safe. Public key from BuildConfig
  DEBUG_REPORT_PUBLIC_KEY (empty default); private key stays with the maintainer.
- ReportUploadWorker fails closed: uploads only when an endpoint AND a usable
  encryption key are configured; never transmits plaintext. PII-free AppLog at the
  anonymize/encrypt/upload lifecycle points.
- Does NOT put R2/S3 credentials or SigV4 signing in the app (would violate the
  F-Droid + secrets-never-in-app constraints); the app POSTs the encrypted envelope
  to the ingest Worker, which holds the R2 secrets server-side (#11/#34).
- Unit tests for anonymization, encryption round-trip, and the worker paths
  (success/retry/failure, fail-closed). docs/debug-report-privacy.md documents the
  data flow, anonymization, encryption scheme, and key custody (for #16/#20).

Closes #34
2026-07-08 10:24:52 -05:00
12 changed files with 801 additions and 25 deletions
+10 -3
View File
@@ -40,11 +40,17 @@ val outlookOAuthClientId: String = secrets.getProperty(
// OUTLOOK_OAUTH_REDIRECT_URI and the redirect URI registered in the Azure app registration.
val outlookRedirectScheme = "org.libremail.outlook"
// Debug-report ingest endpoint (issue #34, out of scope for this repo). Empty by default: the debug
// reporting client is strictly opt-in and never sends anything unless the user taps Submit AND an
// endpoint is configured here (overridable via git-ignored secrets.properties).
// Debug-report ingest endpoint (issue #34; the ingest server is separate infrastructure). Empty by
// default: the debug reporting client is strictly opt-in and never sends anything unless the user taps
// Submit AND an endpoint is configured here (overridable via git-ignored secrets.properties).
val debugReportEndpoint: String = secrets.getProperty("DEBUG_REPORT_ENDPOINT", "")
// Debug-report payload public key (issue #34): a single-line Base64 X.509/SPKI RSA public key. The app
// seals every report to this key before upload, so only the maintainer holding the matching PRIVATE
// key (never shipped in the app — F-Droid-safe) can read it. Empty by default; without it the client
// fails closed and uploads nothing. Overridable via git-ignored secrets.properties.
val debugReportPublicKey: String = secrets.getProperty("DEBUG_REPORT_PUBLIC_KEY", "")
// Optional release signing, configured via git-ignored secrets.properties. When absent, release
// builds fall back to the debug key (installable for testing, but not publishable).
val releaseStoreFile: String? = secrets.getProperty("RELEASE_STORE_FILE")
@@ -66,6 +72,7 @@ android {
buildConfigField("String", "OUTLOOK_OAUTH_CLIENT_ID", "\"$outlookOAuthClientId\"")
buildConfigField("String", "OUTLOOK_OAUTH_REDIRECT_URI", "\"$outlookRedirectScheme://oauth2redirect\"")
buildConfigField("String", "DEBUG_REPORT_ENDPOINT", "\"$debugReportEndpoint\"")
buildConfigField("String", "DEBUG_REPORT_PUBLIC_KEY", "\"$debugReportPublicKey\"")
// IMAP connection reuse (issue #357 Part 2, wiring the #125 spike): keep one authenticated
// IMAP connection warm per account instead of paying a cold CONNECT+TLS+LOGIN on every
// operation — the fix for Gmail throttling LibreMail's connect-per-operation traffic. ON by
@@ -9,7 +9,10 @@ import dagger.hilt.android.qualifiers.ApplicationContext
import dagger.hilt.components.SingletonComponent
import org.libremail.BuildConfig
import org.libremail.data.security.KeystoreReportEncryption
import org.libremail.reporting.AppLog
import org.libremail.reporting.DebugReportEndpoint
import org.libremail.reporting.HybridReportPayloadEncryptor
import org.libremail.reporting.ReportPayloadEncryptor
import org.libremail.reporting.ReportStore
import java.io.File
import javax.inject.Singleton
@@ -40,4 +43,25 @@ object ReportingModule {
@Provides
@DebugReportEndpoint
fun provideDebugReportEndpoint(): String = BuildConfig.DEBUG_REPORT_ENDPOINT
/**
* The seam that seals a report to the maintainer's public key before upload (issue #34). The key is
* `BuildConfig.DEBUG_REPORT_PUBLIC_KEY` (empty by default), so a stock/F-Droid build resolves to
* [ReportPayloadEncryptor.Disabled] and [org.libremail.reporting.ReportUploadWorker] fails closed —
* it never transmits an unencrypted report. A key that is set but unparseable is logged (PII-free)
* and likewise leaves the client disabled rather than silently sending in the clear.
*/
@Provides
@Singleton
fun provideReportPayloadEncryptor(): ReportPayloadEncryptor {
val publicKey = BuildConfig.DEBUG_REPORT_PUBLIC_KEY
if (publicKey.isBlank()) return ReportPayloadEncryptor.Disabled
return runCatching { HybridReportPayloadEncryptor(HybridReportPayloadEncryptor.parsePublicKey(publicKey)) }
.getOrElse { e ->
AppLog.w(TAG, "DEBUG_REPORT_PUBLIC_KEY is set but could not be parsed; upload stays disabled", e)
ReportPayloadEncryptor.Disabled
}
}
private const val TAG = "ReportingModule"
}
@@ -0,0 +1,95 @@
// SPDX-License-Identifier: GPL-3.0-or-later
package org.libremail.reporting
import javax.inject.Inject
import javax.inject.Singleton
/**
* Best-effort PII redaction applied to a [DebugReport] immediately before it is encrypted and uploaded
* (issue #34). It is the LAST line of defence, not the first: a report is already PII-free by
* construction — [DiagnosticsCollector] captures only coarse device/app/settings metadata and a
* bucketed provider list (no emails, hosts, or message content), and [StackTraceScrubber] strips the
* host/username-bearing message text out of every stack trace at capture time. This pass re-scrubs the
* two surfaces that can still carry PII the earlier stages never saw:
* - [DebugReport.userComment] — free text the user typed, which may paste an address, a server name,
* an error string with a token, etc.;
* - [DebugReport.logs] — already governed by the `AppLog` "no PII" contract, re-scrubbed here as
* defence-in-depth in case a log line slipped a raw value through.
*
* [DebugReport.userEmail] is deliberately NOT touched: it is the reply-to address the user chose to
* supply when submitting (issue #159), so it is consented, purposeful data rather than leaked PII —
* see `docs/debug-report-privacy.md`. Everything else in the report is non-PII by construction, so it
* is passed through unchanged.
*
* Redaction is intentionally conservative-but-lossy ("best-effort" per the ticket): it favours
* removing a real secret over preserving a false positive, so a `file.kt:42`-shaped token in free text
* may be redacted too. [hasResidualPii] lets the caller log (never block) when a PII shape survives.
*/
@Singleton
class ReportAnonymizer @Inject constructor() {
/** Returns a copy of [report] with free-text and log surfaces PII-redacted; other fields unchanged. */
fun anonymize(report: DebugReport): DebugReport = report.copy(
userComment = redact(report.userComment),
stackTrace = report.stackTrace?.let(StackTraceScrubber::scrub),
logs = report.logs.map(::redact),
)
/**
* True when a PII *shape* (email, host:port, IPv4, or a JWT-like token) still appears in the
* redactable surfaces of [report]. Used only to log a best-effort warning — [DebugReport.userEmail]
* is excluded because it is intentionally retained (see the class KDoc).
*/
fun hasResidualPii(report: DebugReport): Boolean {
val surfaces = report.logs + report.userComment + report.stackTrace.orEmpty()
return surfaces.any { line -> RESIDUAL_SHAPES.any { it.containsMatchIn(line) } }
}
private fun redact(text: String): String {
if (text.isEmpty()) return text
var out = text
out = SECRET_ASSIGNMENT.replace(out) { "${it.groupValues[1]}=$REDACTED" }
out = BEARER.replace(out, "Bearer $REDACTED")
out = BASIC.replace(out, "Basic $REDACTED")
out = JWT.replace(out, REDACTED)
out = EMAIL.replace(out, REDACTED)
out = HOST_PORT.replace(out, REDACTED)
out = IPV4.replace(out, REDACTED)
return out
}
private companion object {
const val REDACTED = "[redacted]"
/** `user@host.tld`. Mirrors [StackTraceScrubber]'s address pattern. */
val EMAIL = Regex("""[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}""")
/** A dotted host or IPv4 followed by `:port`, incl. the `host/1.2.3.4:port` rendering. */
val HOST_PORT = Regex("""[A-Za-z0-9.-]+(?:/[0-9.]+)?:\d{2,5}""")
/** A bare dotted-quad IPv4 address. */
val IPV4 = Regex("""\b(?:\d{1,3}\.){3}\d{1,3}\b""")
/** A three-segment `eyJ…`-prefixed JWT (access/id tokens the app might touch). */
val JWT = Regex("""\beyJ[A-Za-z0-9_-]{5,}\.[A-Za-z0-9_-]{5,}\.[A-Za-z0-9_-]{5,}""")
/**
* `key: value` / `key=value` where the key names a credential; keeps the key, drops the value.
* The `Authorization` header is deliberately NOT a key here — its value carries a scheme + a
* space (`Bearer …` / `Basic …`), so [BEARER] / [BASIC] redact the whole token instead.
*/
val SECRET_ASSIGNMENT = Regex(
"""(?i)\b(password|passwd|pwd|secret|client[_-]?secret|access[_-]?token|""" +
"""refresh[_-]?token|token|api[_-]?key)\b\s*[:=]\s*"?[^\s"]+""",
)
/** `Bearer <token>` authorization values. */
val BEARER = Regex("""(?i)\bbearer\s+[A-Za-z0-9._~+/-]+=*""")
/** `Basic <base64(user:pass)>` authorization values. */
val BASIC = Regex("""(?i)\bbasic\s+[A-Za-z0-9+/=]+""")
/** The residual PII *shapes* [hasResidualPii] flags after redaction (structural forms excluded). */
val RESIDUAL_SHAPES = listOf(EMAIL, HOST_PORT, IPV4, JWT)
}
}
@@ -0,0 +1,120 @@
// SPDX-License-Identifier: GPL-3.0-or-later
package org.libremail.reporting
import org.json.JSONObject
import java.security.KeyFactory
import java.security.PublicKey
import java.security.SecureRandom
import java.security.spec.MGF1ParameterSpec
import java.security.spec.X509EncodedKeySpec
import java.util.Base64
import javax.crypto.Cipher
import javax.crypto.KeyGenerator
import javax.crypto.spec.GCMParameterSpec
import javax.crypto.spec.OAEPParameterSpec
import javax.crypto.spec.PSource
/**
* Seals a debug-report payload so that only the maintainer can read it, BEFORE it leaves the device
* (issue #34). This is end-to-end encryption for the report: the ingest Worker (#34) and the R2 bucket
* it writes to only ever see opaque ciphertext, so "stored objects are not readable without the
* documented key" holds even against the transport and storage tiers, not just at rest.
*
* The device holds only a **public** key, so nothing secret ships in the (F-Droid-buildable) APK; the
* matching private key is held by the maintainer, off-device. Contrast the on-device [ReportEncryption]
* (at-rest, symmetric, Android-Keystore key that can never leave the phone): that key could never
* decrypt a report a remote reviewer must read, which is exactly why this path is asymmetric.
*/
interface ReportPayloadEncryptor {
/** Whether a usable recipient key is configured. When false, callers MUST NOT upload (fail closed). */
fun isConfigured(): Boolean
/** Seals UTF-8 [plaintext] into the self-describing JSON envelope. Only valid when [isConfigured]. */
fun encrypt(plaintext: String): String
/** The default when no `DEBUG_REPORT_PUBLIC_KEY` is configured: never encrypts, never uploads. */
object Disabled : ReportPayloadEncryptor {
override fun isConfigured(): Boolean = false
override fun encrypt(plaintext: String): String =
error("No debug-report public key configured; refusing to produce an unencrypted payload")
}
}
/**
* Hybrid (envelope) encryption to a recipient RSA public key: a fresh random AES-256-GCM content key
* encrypts the payload, and that content key is wrapped with RSA-OAEP (SHA-256 / MGF1-SHA-256). Only
* JCA primitives are used — no third-party crypto and no Google/GMS dependency, so it builds on pure
* FOSS toolchains. The output is a compact JSON envelope:
*
* ```json
* { "v": 1, "alg": "RSA-OAEP-SHA256+A256GCM", "ek": "<b64 wrapped key>",
* "iv": "<b64 12-byte GCM IV>", "ct": "<b64 ciphertext||tag>" }
* ```
*
* Uses [java.util.Base64] (not `android.util.Base64`) so the class is exercisable in JVM unit tests,
* where the `android.util` shim is a no-op stub.
*/
class HybridReportPayloadEncryptor(
private val recipient: PublicKey,
private val random: SecureRandom = SecureRandom(),
) : ReportPayloadEncryptor {
override fun isConfigured(): Boolean = true
override fun encrypt(plaintext: String): String {
val contentKey = KeyGenerator.getInstance("AES").apply { init(AES_KEY_BITS, random) }.generateKey()
val iv = ByteArray(GCM_IV_BYTES).also(random::nextBytes)
val gcm = Cipher.getInstance(AES_TRANSFORM).apply {
init(Cipher.ENCRYPT_MODE, contentKey, GCMParameterSpec(GCM_TAG_BITS, iv))
}
val ciphertext = gcm.doFinal(plaintext.toByteArray(Charsets.UTF_8))
val rsa = Cipher.getInstance(RSA_TRANSFORM).apply {
init(Cipher.ENCRYPT_MODE, recipient, oaepParams())
}
val wrappedKey = rsa.doFinal(contentKey.encoded)
return JSONObject()
.put("v", ENVELOPE_VERSION)
.put("alg", ALG)
.put("ek", b64(wrappedKey))
.put("iv", b64(iv))
.put("ct", b64(ciphertext))
.toString()
}
companion object {
const val ENVELOPE_VERSION = 1
const val ALG = "RSA-OAEP-SHA256+A256GCM"
private const val AES_TRANSFORM = "AES/GCM/NoPadding"
private const val RSA_TRANSFORM = "RSA/ECB/OAEPWithSHA-256AndMGF1Padding"
private const val AES_KEY_BITS = 256
private const val GCM_IV_BYTES = 12
private const val GCM_TAG_BITS = 128
private const val PEM_HEADER = "-----BEGIN PUBLIC KEY-----"
private const val PEM_FOOTER = "-----END PUBLIC KEY-----"
/**
* The OAEP parameters used for the RSA key wrap: SHA-256 for BOTH the digest and the MGF1 mask,
* pinned explicitly to avoid the classic provider default of a SHA-1 MGF1 under a SHA-256 digest.
* Decrypters must init with the identical spec.
*/
fun oaepParams(): OAEPParameterSpec =
OAEPParameterSpec("SHA-256", "MGF1", MGF1ParameterSpec.SHA256, PSource.PSpecified.DEFAULT)
/**
* Parses a Base64-encoded X.509 SubjectPublicKeyInfo (SPKI) RSA public key, tolerating PEM
* armor and embedded whitespace. Throws (`IllegalArgumentException` /
* `java.security.spec.InvalidKeySpecException`) on malformed input; the DI provider treats any
* such failure as "not configured".
*/
fun parsePublicKey(encoded: String): PublicKey {
val der = Base64.getDecoder().decode(normalize(encoded))
return KeyFactory.getInstance("RSA").generatePublic(X509EncodedKeySpec(der))
}
private fun normalize(encoded: String): String =
encoded.replace(PEM_HEADER, "").replace(PEM_FOOTER, "").filterNot(Char::isWhitespace)
private fun b64(bytes: ByteArray): String = Base64.getEncoder().encodeToString(bytes)
}
}
@@ -16,36 +16,80 @@ import java.net.URL
* Posts a single user-submitted report to the ingest endpoint. It runs ONLY when the user tapped
* Submit (enqueued by [ReportUploadScheduler]); nothing here runs automatically. The [endpoint] is
* injected (see [DebugReportEndpoint]) from `BuildConfig.DEBUG_REPORT_ENDPOINT` — empty by default,
* because the ingest server (issue #34) is out of scope for this repo, so submissions no-op with a
* clear failure until an endpoint is set.
* because the ingest server (issue #34) is separate infrastructure, so submissions no-op with a clear
* failure until an endpoint is set.
*
* Before anything leaves the device the report is (1) run through [ReportAnonymizer] — a best-effort
* final PII-redaction pass over the free-text/log surfaces — and (2) sealed by [ReportPayloadEncryptor]
* to the maintainer's public key, so the wire payload is an opaque encrypted envelope, not the
* plaintext JSON. The worker FAILS CLOSED (issue #34): if no encryption key is configured, or sealing
* throws, it returns a failure and sends nothing rather than transmit an unencrypted report.
*/
@HiltWorker
class ReportUploadWorker @AssistedInject constructor(
@Assisted appContext: Context,
@Assisted params: WorkerParameters,
private val store: ReportStore,
private val anonymizer: ReportAnonymizer,
private val encryptor: ReportPayloadEncryptor,
@DebugReportEndpoint private val endpoint: String,
) : CoroutineWorker(appContext, params) {
override suspend fun doWork(): Result {
val id = inputData.getString(KEY_REPORT_ID) ?: return Result.success()
val report = store.find(id) ?: return Result.success() // discarded before the job ran
if (endpoint.isBlank()) return Result.failure() // no ingest server configured in this build
val body = prepareBody(report) ?: return Result.failure()
return withContext(Dispatchers.IO) {
runCatching { post(endpoint, report.toSubmissionPayload()) }.fold(
runCatching { post(endpoint, body) }.fold(
onSuccess = { code -> onResponse(code, id) },
onFailure = { retryOrFail() }, // network error — retry with backoff
onFailure = { e ->
AppLog.w(TAG, "Debug-report upload hit a network error; retrying if attempts remain", e)
retryOrFail()
},
)
}
}
/**
* Anonymizes then seals [report] for upload, or returns null (with a logged reason) to abort the
* send. Null on: no endpoint configured, no encryption key configured, or a sealing failure — in
* every case the worker fails closed rather than transmit anything unencrypted.
*/
private fun prepareBody(report: DebugReport): String? {
if (endpoint.isBlank()) {
AppLog.w(TAG, "Report submit requested but no ingest endpoint is configured in this build; not sending")
return null
}
if (!encryptor.isConfigured()) {
AppLog.e(TAG, "Report submit aborted: no payload encryption key configured; refusing to send unencrypted")
return null
}
val anonymized = anonymizer.anonymize(report)
if (anonymizer.hasResidualPii(anonymized)) {
AppLog.w(TAG, "A PII-shaped token survived anonymization; it was redacted best-effort before upload")
}
return runCatching { encryptor.encrypt(anonymized.toSubmissionPayload()) }
.onSuccess { AppLog.i(TAG, "Sealed a debug report; uploading (attempt ${runAttemptCount + 1})") }
.getOrElse { e ->
AppLog.e(TAG, "Debug-report payload encryption failed; not sending", e)
null
}
}
private fun onResponse(code: Int, id: String): Result = when {
code in SUCCESS_CODES -> {
AppLog.i(TAG, "Debug report delivered (HTTP $code); dropping the local copy")
store.delete(id) // delivered — drop the local copy
Result.success()
}
code in SERVER_ERROR_CODES -> retryOrFail() // transient server-side failure
else -> Result.failure() // client error (4xx) — retrying won't help
code in SERVER_ERROR_CODES -> {
AppLog.w(TAG, "Debug-report ingest returned HTTP $code; retrying if attempts remain")
retryOrFail() // transient server-side failure
}
else -> {
AppLog.w(TAG, "Debug-report ingest rejected the report (HTTP $code); giving up")
Result.failure() // client error (4xx) — retrying won't help
}
}
private fun retryOrFail(): Result = if (runAttemptCount >= MAX_ATTEMPTS) Result.failure() else Result.retry()
@@ -68,6 +112,7 @@ class ReportUploadWorker @AssistedInject constructor(
companion object {
const val KEY_REPORT_ID = "report_id"
private const val TAG = "ReportUploadWorker"
private val SUCCESS_CODES = 200..299
private val SERVER_ERROR_CODES = 500..599
private const val TIMEOUT_MS = 15_000
@@ -0,0 +1,123 @@
// SPDX-License-Identifier: GPL-3.0-or-later
package org.libremail.reporting
import org.junit.Test
import kotlin.test.assertEquals
import kotlin.test.assertFalse
import kotlin.test.assertTrue
/**
* [ReportAnonymizer] is the pre-upload best-effort PII scrub. These feed representative PII through the
* two surfaces that can still carry it — the user's free-text comment and the log lines — and assert it
* is redacted, while the intentionally-retained reply-to [DebugReport.userEmail] and the non-PII
* metadata pass through untouched.
*/
class ReportAnonymizerTest {
private val anonymizer = ReportAnonymizer()
private fun report(
userComment: String = "",
userEmail: String = "",
logs: List<String> = emptyList(),
stackTrace: String? = null,
) = DebugReport(
id = "rid",
createdAtMillis = 1L,
kind = ReportKind.MANUAL,
appVersionName = "0.1.0",
appVersionCode = 1,
androidRelease = "14",
androidSdkInt = 34,
deviceManufacturer = "Google",
deviceModel = "Pixel",
stackTrace = stackTrace,
settings = mapOf("dynamicColor" to "true"),
logs = logs,
accounts = listOf("Gmail (PASSWORD_IMAP)"),
userComment = userComment,
userEmail = userEmail,
)
@Test
fun `redacts an email address in the user comment`() {
val out = anonymizer.anonymize(report(userComment = "reply to me at alice@example.com please"))
assertEquals("reply to me at [redacted] please", out.userComment)
}
@Test
fun `redacts a host and port in the user comment`() {
val out = anonymizer.anonymize(report(userComment = "fails talking to imap.example.com:993 always"))
assertEquals("fails talking to [redacted] always", out.userComment)
}
@Test
fun `redacts a bare IPv4 address in the user comment`() {
val out = anonymizer.anonymize(report(userComment = "server at 93.184.216.34 is down"))
assertEquals("server at [redacted] is down", out.userComment)
}
@Test
fun `redacts a JWT-shaped token in the user comment`() {
val jwt = "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.dozjgNryP4J3jVmNHl0w5N"
val out = anonymizer.anonymize(report(userComment = "token was $jwt when it broke"))
assertEquals("token was [redacted] when it broke", out.userComment)
}
@Test
fun `redacts a bearer authorization value in a log line`() {
val out = anonymizer.anonymize(report(logs = listOf("I/Auth sent header Authorization: Bearer abc123.DEF-456")))
assertTrue(out.logs.single().endsWith("Bearer [redacted]"), out.logs.single())
}
@Test
fun `keeps a credential key but redacts its value`() {
val out = anonymizer.anonymize(report(userComment = "config had password=hunter2 in it"))
assertEquals("config had password=[redacted] in it", out.userComment)
}
@Test
fun `re-scrubs the stack trace through StackTraceScrubber`() {
val trace = "java.net.ConnectException: Failed to connect to imap.example.com/93.184.216.34:993"
val out = anonymizer.anonymize(report(stackTrace = trace))
assertEquals("java.net.ConnectException", out.stackTrace)
}
@Test
fun `retains the user-supplied reply-to email untouched`() {
// userEmail is consented reply-to data (issue #159), not leaked PII: it must survive the pass.
val out = anonymizer.anonymize(report(userEmail = "bob@example.org"))
assertEquals("bob@example.org", out.userEmail)
}
@Test
fun `leaves non-PII metadata unchanged`() {
val input = report(userComment = "just a plain note")
val out = anonymizer.anonymize(input)
assertEquals(input.userComment, out.userComment)
assertEquals(input.deviceModel, out.deviceModel)
assertEquals(input.settings, out.settings)
assertEquals(input.accounts, out.accounts)
}
@Test
fun `an empty comment stays empty`() {
assertEquals("", anonymizer.anonymize(report(userComment = "")).userComment)
}
@Test
fun `hasResidualPii is false after anonymizing PII-laden text`() {
val dirty = report(
userComment = "alice@example.com via imap.example.com:993",
logs = listOf("saw 10.0.0.5 on the wire"),
)
assertTrue(anonymizer.hasResidualPii(dirty))
assertFalse(anonymizer.hasResidualPii(anonymizer.anonymize(dirty)))
}
@Test
fun `hasResidualPii ignores the intentional reply-to email`() {
// A clean report whose only address is the consented reply-to must NOT be flagged as residual PII.
assertFalse(anonymizer.hasResidualPii(report(userEmail = "bob@example.org")))
}
}
@@ -0,0 +1,129 @@
// SPDX-License-Identifier: GPL-3.0-or-later
package org.libremail.reporting
import org.json.JSONObject
import org.junit.Test
import java.security.KeyPair
import java.security.KeyPairGenerator
import java.security.PrivateKey
import java.util.Base64
import javax.crypto.AEADBadTagException
import javax.crypto.Cipher
import javax.crypto.spec.GCMParameterSpec
import javax.crypto.spec.SecretKeySpec
import kotlin.test.assertEquals
import kotlin.test.assertFailsWith
import kotlin.test.assertFalse
import kotlin.test.assertNotEquals
import kotlin.test.assertTrue
/**
* [HybridReportPayloadEncryptor] round-trips: the test generates an RSA key pair, encrypts with the
* PUBLIC half exactly as the app would, then decrypts with the PRIVATE half — the maintainer's role.
* It also pins the privacy-critical properties: the plaintext never appears in the envelope, each seal
* is unique (fresh content key + IV), and tampering fails the GCM tag.
*/
class ReportPayloadEncryptorTest {
private val keyPair: KeyPair = KeyPairGenerator.getInstance("RSA").apply { initialize(2048) }.generateKeyPair()
private val encryptor = HybridReportPayloadEncryptor(keyPair.public)
@Test
fun `encrypt then decrypt recovers the original payload`() {
val plaintext = """{"id":"rid","userComment":"the sync keeps failing"}"""
val recovered = decrypt(encryptor.encrypt(plaintext), keyPair.private)
assertEquals(plaintext, recovered)
}
@Test
fun `the envelope is well-formed and never contains the plaintext`() {
val plaintext = """{"marker":"TOP-SECRET-MARKER-42"}"""
val envelope = JSONObject(encryptor.encrypt(plaintext))
assertEquals(HybridReportPayloadEncryptor.ENVELOPE_VERSION, envelope.getInt("v"))
assertEquals(HybridReportPayloadEncryptor.ALG, envelope.getString("alg"))
assertTrue(envelope.getString("ek").isNotEmpty())
assertTrue(envelope.getString("iv").isNotEmpty())
assertTrue(envelope.getString("ct").isNotEmpty())
assertFalse(envelope.toString().contains("TOP-SECRET-MARKER-42"))
}
@Test
fun `each seal of the same plaintext is unique`() {
val plaintext = "identical input"
val first = JSONObject(encryptor.encrypt(plaintext))
val second = JSONObject(encryptor.encrypt(plaintext))
// Fresh random content key + IV each time -> different wrapped key, IV, and ciphertext.
assertNotEquals(first.getString("ct"), second.getString("ct"))
assertNotEquals(first.getString("iv"), second.getString("iv"))
assertNotEquals(first.getString("ek"), second.getString("ek"))
}
@Test
fun `a tampered ciphertext fails the GCM authentication tag`() {
val envelope = JSONObject(encryptor.encrypt("authentic payload"))
val ct = Base64.getDecoder().decode(envelope.getString("ct"))
ct[0] = (ct[0].toInt() xor 0x01).toByte() // flip one bit
envelope.put("ct", Base64.getEncoder().encodeToString(ct))
assertFailsWith<AEADBadTagException> { decrypt(envelope.toString(), keyPair.private) }
}
@Test
fun `isConfigured is true for a real key and false for Disabled`() {
assertTrue(encryptor.isConfigured())
assertFalse(ReportPayloadEncryptor.Disabled.isConfigured())
}
@Test
fun `Disabled refuses to encrypt`() {
assertFailsWith<IllegalStateException> { ReportPayloadEncryptor.Disabled.encrypt("anything") }
}
@Test
fun `parsePublicKey round-trips a Base64 SPKI key`() {
val spkiBase64 = Base64.getEncoder().encodeToString(keyPair.public.encoded)
val parsed = HybridReportPayloadEncryptor.parsePublicKey(spkiBase64)
assertEquals(keyPair.public, parsed)
}
@Test
fun `parsePublicKey tolerates PEM armor and whitespace`() {
val body = Base64.getEncoder().encodeToString(keyPair.public.encoded)
val pem = "-----BEGIN PUBLIC KEY-----\n" + body.chunked(64).joinToString("\n") + "\n-----END PUBLIC KEY-----\n"
assertEquals(keyPair.public, HybridReportPayloadEncryptor.parsePublicKey(pem))
}
@Test
fun `parsePublicKey rejects malformed input`() {
assertFailsWith<IllegalArgumentException> { HybridReportPayloadEncryptor.parsePublicKey("not base64 !!!") }
}
/** The maintainer-side decrypt: unwrap the content key with the RSA private key, then open the GCM box. */
private fun decrypt(envelopeJson: String, privateKey: PrivateKey): String {
val envelope = JSONObject(envelopeJson)
val wrappedKey = Base64.getDecoder().decode(envelope.getString("ek"))
val iv = Base64.getDecoder().decode(envelope.getString("iv"))
val ciphertext = Base64.getDecoder().decode(envelope.getString("ct"))
val rsa = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding").apply {
init(Cipher.DECRYPT_MODE, privateKey, HybridReportPayloadEncryptor.oaepParams())
}
val contentKey = SecretKeySpec(rsa.doFinal(wrappedKey), "AES")
val gcm = Cipher.getInstance("AES/GCM/NoPadding").apply {
init(Cipher.DECRYPT_MODE, contentKey, GCMParameterSpec(GCM_TAG_BITS, iv))
}
return String(gcm.doFinal(ciphertext), Charsets.UTF_8)
}
private companion object {
const val GCM_TAG_BITS = 128
}
}
@@ -1,41 +1,57 @@
// SPDX-License-Identifier: GPL-3.0-or-later
package org.libremail.reporting
import android.util.Log
import androidx.work.ListenableWorker.Result
import androidx.work.WorkerParameters
import androidx.work.workDataOf
import com.sun.net.httpserver.HttpServer
import io.mockk.every
import io.mockk.mockk
import io.mockk.mockkStatic
import io.mockk.unmockkAll
import io.mockk.verify
import kotlinx.coroutines.test.runTest
import org.json.JSONObject
import org.junit.After
import org.junit.Before
import org.junit.Test
import java.net.InetAddress
import java.net.InetSocketAddress
import java.net.ServerSocket
import java.security.KeyPair
import java.security.KeyPairGenerator
import java.security.PrivateKey
import java.util.Base64
import java.util.concurrent.atomic.AtomicInteger
import java.util.concurrent.atomic.AtomicReference
import javax.crypto.Cipher
import javax.crypto.spec.GCMParameterSpec
import javax.crypto.spec.SecretKeySpec
import kotlin.test.assertEquals
import kotlin.test.assertNotEquals
/**
* Covers [ReportUploadWorker]'s transmit path — unreachable while the default build ships an empty
* `DEBUG_REPORT_ENDPOINT` — by injecting a non-empty endpoint (issue #257). The endpoint points at an
* in-process [HttpServer] on loopback (JDK built-in, so no new dependency, and the same "real
* in-process server" approach the suite already uses with GreenMail for IMAP/SMTP), so the actual
* `HttpURLConnection` POST, response-code handling, retry/backoff decision, and on-success delete run
* end to end:
* `DEBUG_REPORT_ENDPOINT` — by injecting a non-empty endpoint AND a configured encryptor (issue #34;
* the worker fails closed without one). The endpoint points at an in-process [HttpServer] on loopback
* (JDK built-in, so no new dependency, and the same "real in-process server" approach the suite already
* uses with GreenMail for IMAP/SMTP), so the actual `HttpURLConnection` POST, response-code handling,
* retry/backoff decision, and on-success delete run end to end:
* - 2xx -> success, and the delivered report is dropped from the local store;
* - 4xx -> permanent failure (retrying a client error can't help), store untouched;
* - 5xx -> retry while attempts remain, then failure once the attempt cap is hit;
* - a network error (nothing listening) -> retry while attempts remain.
* It also pins the request the server receives: a JSON POST whose body is the report's submission
* payload — the exact wire format the (out-of-scope) ingest server would get.
* It also pins the wire format: the POST body is the ENCRYPTED envelope, not the plaintext payload —
* the test holds the matching RSA private key and decrypts what the server received to prove the
* plaintext is recoverable only by the maintainer, exactly as the ingest server would.
*/
class ReportUploadWorkerHttpTest {
private val store = mockk<ReportStore>(relaxed = true)
private val anonymizer = ReportAnonymizer()
private val keyPair: KeyPair = KeyPairGenerator.getInstance("RSA").apply { initialize(2048) }.generateKeyPair()
private val encryptor = HybridReportPayloadEncryptor(keyPair.public)
private lateinit var server: HttpServer
private val responseCode = AtomicInteger(HTTP_OK)
private val receivedBody = AtomicReference<String>()
@@ -44,6 +60,12 @@ class ReportUploadWorkerHttpTest {
@Before
fun startServer() {
mockkStatic(Log::class)
every { Log.d(any(), any()) } returns 0
every { Log.i(any(), any()) } returns 0
every { Log.w(any<String>(), any<String>()) } returns 0
every { Log.w(any<String>(), any<String>(), any()) } returns 0
every { Log.e(any(), any(), any()) } returns 0
server = HttpServer.create(InetSocketAddress(LOOPBACK, 0), 0)
server.createContext(PATH) { exchange ->
receivedContentType.set(exchange.requestHeaders.getFirst("Content-Type"))
@@ -58,10 +80,11 @@ class ReportUploadWorkerHttpTest {
@After
fun stopServer() {
server.stop(0)
unmockkAll()
}
@Test
fun `a 2xx response succeeds and drops the delivered report`() = runTest {
fun `a 2xx response succeeds, drops the report, and posts an encrypted envelope`() = runTest {
val report = report()
every { store.find(REPORT_ID) } returns report
responseCode.set(HTTP_OK)
@@ -70,8 +93,14 @@ class ReportUploadWorkerHttpTest {
assertEquals(Result.success(), result)
verify { store.delete(REPORT_ID) }
assertEquals(report.toSubmissionPayload(), receivedBody.get())
assertEquals("application/json; charset=utf-8", receivedContentType.get())
// The wire body is the sealed envelope, NOT the plaintext payload...
assertNotEquals(report.toSubmissionPayload(), receivedBody.get())
// ...but decrypting it with the private key yields exactly the (anonymized) submission payload.
assertEquals(
anonymizer.anonymize(report).toSubmissionPayload(),
decrypt(receivedBody.get(), keyPair.private),
)
}
@Test
@@ -123,6 +152,8 @@ class ReportUploadWorkerHttpTest {
every { runAttemptCount } returns attempt
},
store,
anonymizer,
encryptor,
endpoint = endpoint,
)
@@ -132,6 +163,23 @@ class ReportUploadWorkerHttpTest {
return "http://$LOOPBACK:$port$PATH"
}
/** The maintainer-side decrypt of a sealed envelope, using the RSA private key. */
private fun decrypt(envelopeJson: String, privateKey: PrivateKey): String {
val envelope = JSONObject(envelopeJson)
val rsa = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding").apply {
init(Cipher.DECRYPT_MODE, privateKey, HybridReportPayloadEncryptor.oaepParams())
}
val contentKey = SecretKeySpec(rsa.doFinal(Base64.getDecoder().decode(envelope.getString("ek"))), "AES")
val gcm = Cipher.getInstance("AES/GCM/NoPadding").apply {
init(
Cipher.DECRYPT_MODE,
contentKey,
GCMParameterSpec(GCM_TAG_BITS, Base64.getDecoder().decode(envelope.getString("iv"))),
)
}
return String(gcm.doFinal(Base64.getDecoder().decode(envelope.getString("ct"))), Charsets.UTF_8)
}
private fun report() = DebugReport(
id = REPORT_ID,
createdAtMillis = 1L,
@@ -155,6 +203,7 @@ class ReportUploadWorkerHttpTest {
const val HTTP_OK = 200
const val HTTP_BAD_REQUEST = 400
const val HTTP_SERVER_ERROR = 500
const val GCM_TAG_BITS = 128
// Mirrors ReportUploadWorker.MAX_ATTEMPTS (private): at/after this count a retry becomes a failure.
const val MAX_ATTEMPTS = 5
@@ -1,32 +1,60 @@
// SPDX-License-Identifier: GPL-3.0-or-later
package org.libremail.reporting
import android.util.Log
import androidx.work.ListenableWorker.Result
import androidx.work.workDataOf
import io.mockk.every
import io.mockk.mockk
import io.mockk.mockkStatic
import io.mockk.unmockkAll
import kotlinx.coroutines.test.runTest
import org.junit.After
import org.junit.Before
import org.junit.Test
import java.security.GeneralSecurityException
import kotlin.test.assertEquals
/**
* The transmit path only runs when a `DEBUG_REPORT_ENDPOINT` is configured — the default build ships
* an empty one (the ingest server is out of scope for this repo), so these cover the reachable control
* flow: a missing report id, a report deleted before the job ran, and the "no endpoint configured"
* short-circuit that returns a clear failure instead of a silent no-op.
* The reachable control flow of [ReportUploadWorker] that does NOT need a live HTTP endpoint (the
* transmit path is covered by [ReportUploadWorkerHttpTest]): a missing report id, a report deleted
* before the job ran, and the fail-closed guards — no endpoint configured, no encryption key
* configured, and a sealing failure — each of which returns a clear failure and sends nothing.
* `android.util.Log` is a no-op stub in JVM tests, so it is statically mocked (the worker logs its
* lifecycle through [AppLog]).
*/
class ReportUploadWorkerTest {
private val store = mockk<ReportStore>(relaxed = true)
private fun worker(inputId: String?) = ReportUploadWorker(
@Before
fun mockLog() {
mockkStatic(Log::class)
every { Log.d(any(), any()) } returns 0
every { Log.i(any(), any()) } returns 0
every { Log.w(any<String>(), any<String>()) } returns 0
every { Log.w(any<String>(), any<String>(), any()) } returns 0
every { Log.e(any(), any(), any()) } returns 0
}
@After
fun unmock() = unmockkAll()
private fun worker(
inputId: String?,
endpoint: String = "",
encryptor: ReportPayloadEncryptor = ReportPayloadEncryptor.Disabled,
anonymizer: ReportAnonymizer = ReportAnonymizer(),
) = ReportUploadWorker(
mockk(relaxed = true),
mockk(relaxed = true) {
every { inputData } returns
if (inputId == null) workDataOf() else workDataOf(ReportUploadWorker.KEY_REPORT_ID to inputId)
},
store,
endpoint = "", // default build ships no ingest endpoint; the transmit path is covered separately
anonymizer,
encryptor,
endpoint = endpoint,
)
private fun report(id: String) = DebugReport(
@@ -63,4 +91,36 @@ class ReportUploadWorkerTest {
// No endpoint configured -> a clear failure the UI can steer away from, never a silent success.
assertEquals(Result.failure(), worker(inputId = "rid").doWork())
}
@Test
fun `fails closed when an endpoint is set but no encryption key is configured`() = runTest {
every { store.find("rid") } returns report("rid")
// Endpoint present, encryptor Disabled -> refuse to send rather than transmit unencrypted.
val result = worker(inputId = "rid", endpoint = "https://ingest.invalid/report").doWork()
assertEquals(Result.failure(), result)
}
@Test
fun `fails closed when sealing the payload throws`() = runTest {
every { store.find("rid") } returns report("rid")
val anonymizer = mockk<ReportAnonymizer> {
every { anonymize(any()) } returns report("rid")
every { hasResidualPii(any()) } returns true // also exercises the residual-PII warning path
}
val encryptor = mockk<ReportPayloadEncryptor> {
every { isConfigured() } returns true
every { encrypt(any()) } throws GeneralSecurityException("no")
}
val result = worker(
inputId = "rid",
endpoint = "https://ingest.invalid/report",
encryptor = encryptor,
anonymizer = anonymizer,
).doWork()
assertEquals(Result.failure(), result)
}
}
+4
View File
@@ -79,6 +79,10 @@ style:
- '**/data/repository/AccountRepositoryImplTest.kt'
- '**/ui/reader/ReaderViewModelTest.kt'
- '**/ui/reader/ReaderViewModelActionsTest.kt'
# Debug-report ingest (issue #34): ReportUploadWorker logs its anonymize/encrypt/upload lifecycle
# via AppLog, so its unit tests mockkStatic(Log) to keep the throwing JVM stub quiet.
- '**/reporting/ReportUploadWorkerTest.kt'
- '**/reporting/ReportUploadWorkerHttpTest.kt'
MagicNumber:
# dp / sp / duration literals are idiomatic inline in Compose.
ignoreAnnotated: ['Composable']
+108
View File
@@ -0,0 +1,108 @@
<!-- SPDX-License-Identifier: GPL-3.0-or-later -->
# Debug report privacy posture
Data flow and privacy design for LibreMail's opt-in debug reporting, covering the **client (app) side**
that lives in this repository. It is referenced by the F-Droid audit (#16) and the README (#20), and is
the client half of the ingest pipeline epic (#11); the ingest server (#34) and the weekly publish job
(#35) are **separate infrastructure**, not part of this repo.
> TL;DR: nothing is ever sent unless the user taps **Submit** on a report they have read in full, and
> even then the payload is **anonymized** and **encrypted to the maintainer's public key on the device**
> before it leaves. A stock build (and every F-Droid build) is configured to send **nothing at all**.
## 1. What a report contains
A `DebugReport` is assembled by `DiagnosticsCollector` from **coarse, non-identifying** fields only:
- App version name/code; Android release + SDK int; device manufacturer + model.
- A small settings summary (feature toggles and enum names — e.g. `dynamicColor=true`, `fetchPolicy=ALL`).
- A **bucketed** account summary: one `"<provider> (<authType>)"` entry per account (e.g. `Gmail
(PASSWORD_IMAP)`) — never an email address, username, or server hostname (custom hosts collapse to
`Other`).
- For crashes, a stack trace **scrubbed at capture time** by `StackTraceScrubber`: exception class names
and stack frames are kept; the free-text exception *message* (the only place a host or username
appears) is dropped, and any residual `user@host` / `host:port` token is redacted.
- The recent in-app log buffer (`RingLogBuffer`), which is governed by the `AppLog` **"no PII" contract**
(a detekt guard forbids raw `android.util.Log`; throwables are auto-scrubbed).
- Two user-supplied fields: a free-text **comment**, and an optional **reply-to email** the user
typed when submitting (issue #159).
Message bodies, headers, attachments, credentials, and tokens are **never** collected.
## 2. Opt-in gating (strictly user-initiated)
Reporting is **off by default and cannot phone home** without deliberate action:
- A report is only transmitted when the user taps **Submit** on the review screen after reading the full
payload verbatim (issue #33). There is no background or automatic upload.
- The transmit path additionally requires build configuration that a stock/F-Droid build does **not**
ship: both `DEBUG_REPORT_ENDPOINT` and `DEBUG_REPORT_PUBLIC_KEY` (below) must be set. With either
empty, `ReportSubmitter.isEnabled` is false / the worker fails closed, and the UI steers the user to
**Copy** or **Save to file** instead.
## 3. Anonymization pass (`ReportAnonymizer`)
Immediately before upload, a best-effort redaction pass runs as defence-in-depth over the two surfaces
that can still carry PII the earlier stages never saw:
- **User comment** (free text) and **log lines** are scanned and redacted for: email addresses,
`host:port` tokens, bare IPv4 addresses, JWT-shaped tokens, `Bearer` authorization values, and
`key=value` credentials (`password=…`, `access_token=…`, …). The stack trace is re-run through
`StackTraceScrubber`.
- Redaction is intentionally **conservative-but-lossy** (the ticket's "best-effort" bar): it prefers to
over-redact a false positive rather than leak a real secret. If any PII *shape* survives, the worker
logs a PII-free warning (it never blocks the user's submission).
- **Exception — the reply-to email is retained by design.** `userEmail` is consented, purposeful data
the user chose to give so the maintainer can follow up (issue #159); it is the one identifier that
intentionally leaves the device, and only when the user fills it in. The ingest server (#34) is where
reply-to is separated from the report body if desired.
## 4. Encryption scheme and key custody
The report is **end-to-end encrypted on the device** before upload (`ReportPayloadEncryptor`), so the
ingest Worker and the R2 bucket only ever see opaque ciphertext — "not readable without the documented
key" holds against the transport and storage tiers, not just at rest.
**Scheme — hybrid (envelope) encryption, JCA only (no third-party or Google/GMS crypto):**
1. Generate a fresh random **AES-256-GCM** content key + 96-bit IV; encrypt the report payload with it.
2. Wrap the content key with the recipient's RSA public key using **RSA-OAEP (SHA-256, MGF1-SHA-256)**.
3. Emit a self-describing JSON envelope — the exact wire body of the POST:
```json
{ "v": 1, "alg": "RSA-OAEP-SHA256+A256GCM",
"ek": "<base64 RSA-OAEP-wrapped AES key>",
"iv": "<base64 12-byte GCM IV>",
"ct": "<base64 AES-GCM ciphertext||tag>" }
```
**Key custody:**
- The **public** key ships in the build via `BuildConfig.DEBUG_REPORT_PUBLIC_KEY` (a single-line Base64
X.509/SPKI RSA key). A public key is not a secret, so nothing sensitive is embedded in the APK — this
is F-Droid-safe.
- The matching **private** key is held by the maintainer, **off-device** (e.g. Cloudflare Secret Manager
per #11), and never appears in this repository or the app. Only that private key can decrypt a report.
- **Fail-closed:** if no public key is configured, or if sealing throws, `ReportUploadWorker` returns a
failure and sends nothing — it never transmits an unencrypted report. This is distinct from the
on-device at-rest `ReportEncryption` (#369), whose symmetric Android-Keystore key can never leave the
phone and so could never decrypt a report a remote reviewer must read; that is why the upload path is
asymmetric.
## 5. Why R2 credentials are NOT in the app
Uploading to Cloudflare R2 (S3-compatible) requires write credentials. Embedding those in a FOSS,
publicly-built APK would expose them to every user and violate both F-Droid's rules and #11/#34's
"secrets stored in Cloudflare, never in the app". So the app does **not** hold R2/S3 credentials or sign
requests: it performs a single authenticated-by-nothing **HTTPS POST of the encrypted envelope** to the
ingest Worker's endpoint, and the Worker (server-side infra, #34) holds the R2 credentials and writes
the object. The client's only configuration is a URL and a public key.
## 6. Configuration summary
| Build config (`secrets.properties`, git-ignored) | Default | Effect when empty |
|---|---|---|
| `DEBUG_REPORT_ENDPOINT` | empty | No submit path; UI offers Copy/Save only. |
| `DEBUG_REPORT_PUBLIC_KEY` | empty | Worker fails closed; nothing is ever uploaded. |
See `secrets.properties.example` for how to generate the key pair and populate these.
+12
View File
@@ -11,3 +11,15 @@
#RELEASE_STORE_PASSWORD=
#RELEASE_KEY_ALIAS=
#RELEASE_KEY_PASSWORD=
# Optional: debug-report ingest (issue #34). The debug reporter is strictly opt-in and transmits ONLY
# when the user taps Submit AND BOTH values below are set; leave them unset for a build that can never
# phone home. See docs/debug-report-privacy.md for the data flow, anonymization, and encryption scheme.
# DEBUG_REPORT_ENDPOINT HTTPS URL of the ingest Worker (separate server infrastructure, not this repo).
# DEBUG_REPORT_PUBLIC_KEY Single-line Base64 X.509/SPKI RSA public key. The app seals each report to
# it before upload; only the holder of the matching private key (the
# maintainer, never shipped in the app) can decrypt it. Generate with, e.g.:
# openssl genrsa -out report_key.pem 4096
# openssl rsa -in report_key.pem -pubout -outform DER | base64 -w0
#DEBUG_REPORT_ENDPOINT=
#DEBUG_REPORT_PUBLIC_KEY=