Files
LibreMail/.github/workflows/ci.yml
T
JMR-dev 58447d7d12 ci(e2e): gate the emulator on window focus to fix the RootViewPicker flake (#468)
Root cause: intermittently the launched activity window has has-window-focus=false
for the WHOLE instrumented run, so Espresso's RootViewPicker (onView().check(),
Intents.intended(), pressBack(), focus-dependent clipboard) times out after 10s and
fails EVERY focus-dependent test at once while the ~280 pure-Compose semantics tests
(which don't need window focus) pass. A failing E2E (35) leg's logcat (PR #470, run
28985259521) shows has-window-focus=true ZERO times across the whole session and both
the first attempt and the once-retry fail identically -- a persistent environmental
state, not a per-test transient. The prior mitigation, a single fire-and-forget
`adb shell input keyevent 82` (MENU) right after boot, is too weak: MENU no longer
dismisses the modern (API 30+) keyguard and, delivered before SystemUI/keyguard comes
up, is simply dropped -- so the insecure keyguard / non-interactive display persists
and no app window ever takes focus.

Fix: a single shared helper, .github/scripts/emulator_focus_gate.py, invoked
identically by BOTH E2E jobs (the e2e API 29-36 matrix AND e2e-preview API 37) and by
the local preflight runners (local_instrumented.py / api37_e2e.py), so it cannot drift.
It wakes the display (KEYCODE_WAKEUP), dismisses + disables the keyguard
(wm dismiss-keyguard, locksettings set-disabled true), keeps the screen on
(svc power stayon true + max screen_off_timeout), zeroes the animation scales, then
polls dumpsys power/window until the device is interactive AND a real window holds
input focus (mCurrentFocus non-null) -- re-nudging each iteration -- before the suite
runs. Applied uniformly, this also gives e2e-preview the animation-disable the matrix
already had. The gate is soft (bounded wait, then proceeds with a ::warning:: and the
final device state) and non-fatal (`|| true`), preserving #454's guarantee that the
unlock never aborts the boot; it leaves #454's manual boot, #460's path-filter and
#464's wedge-capture untouched.

The pure readiness parser is unit-tested by test_emulator_focus_gate.py (run by the
traffic-control-tests job). Determinism is validated by this PR's own matrix run.

Closes #468
2026-07-08 20:32:16 -05:00

1161 lines
67 KiB
YAML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# SPDX-License-Identifier: GPL-3.0-or-later
name: CI
on:
pull_request:
branches: [main]
# The traffic-controller SCHEDULER (ci-trigger.yml, issue #349) (re-)triggers CI for a
# specific PR via this workflow_dispatch after a GITHUB_TOKEN auto-update has left the PR's
# head SHA with absent/stale checks. Dispatched on the PR's head BRANCH, so the run's checks
# land on the PR head SHA and satisfy branch protection. `on: pull_request` above is KEPT so
# brand-new PRs, human pushes, and fork PRs still get CI directly — this is the fail-open
# guarantee: CI is always triggerable even if the scheduler is broken or absent.
workflow_dispatch:
inputs:
pr:
description: "PR number this run is for (set by the traffic-controller scheduler)."
required: false
type: string
head_sha:
description: "Expected head SHA (informational, for traceability in the run log)."
required: false
type: string
reason:
description: "Why this run was dispatched (informational)."
required: false
type: string
# A new trigger for a PR cancels that PR's own in-flight run (a newer head SHA supersedes).
# The group is keyed to the PR NUMBER so a `pull_request` run and a scheduler
# `workflow_dispatch` run for the SAME PR share one concurrency group (either supersedes a
# stale run of the other); it falls back to the ref when no PR number is in context.
concurrency:
group: ci-pr-${{ github.event.pull_request.number || inputs.pr || github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
# SDK packages this project builds against (compileSdk 37 / build-tools 37.0.0).
# Quote the package ids when passed to sdkmanager — the ';' is a shell separator.
ANDROID_PLATFORM: "platforms;android-37.0"
ANDROID_BUILD_TOOLS: "build-tools;37.0.0"
jobs:
# Path-filter gate (issue #399): a cheap first job that decides whether the heavy E2E matrix
# (`e2e` + `e2e-preview`, ~10 emulator jobs) needs to run for this change. Test-only / docs /
# dev-script PRs then skip E2E and merge on the fast gate (unit + static) instead of queuing
# behind the emulator matrix. The `ci-passed` gate below is rewritten to treat an INTENTIONAL
# E2E skip as a pass while still blocking a real E2E failure/cancel or a broken filter.
changes:
name: Detect changed paths
runs-on: ubuntu-latest
# dorny/paths-filter lists a pull request's changed files via the GitHub API (no checkout),
# which needs pull-requests: read on top of the workflow-default contents: read.
permissions:
contents: read
pull-requests: read
outputs:
# 'true' -> run the E2E matrix; 'false' -> skip it (only for test-only/docs/script PRs).
e2e_needed: ${{ steps.decide.outputs.e2e_needed }}
steps:
- name: Filter changed paths (pull requests only)
id: filter
if: github.event_name == 'pull_request'
uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2
with:
# HOW THIS WORKS -- and why it is written "inside-out" with NEGATED globs.
# dorny/paths-filter sets a filter's boolean output to true when AT LEAST ONE changed
# file matches that filter's per-file predicate. `predicate-quantifier: 'every'` makes
# the per-file predicate "this file matches EVERY pattern in the list". So to say
# "run E2E iff SOME changed file is E2E-relevant", `non_skippable` lists the safe
# allow-globs each NEGATED: a file is non-skippable when it matches NONE of the safe
# globs (it matches every '!'-pattern), and e2e_needed = non_skippable. Thus ANY file
# outside the allow-list -- app/src/main, app/src/androidTest, app/build.gradle.kts,
# root build.gradle*/settings.gradle*, gradle/** (wrapper + libs.versions.toml),
# gradle.properties, app/schemas, app/proguard-rules.pro, .github/workflows/**,
# .github/scripts/** -- forces the whole matrix. That is the conservative "err toward
# running E2E / default to true if unsure" rule: the skip set is an explicit allow-list
# of paths that provably cannot affect the app build or instrumented tests, never a
# guess about what is unsafe.
#
# This replaces #399's original form, which listed the same safe paths as POSITIVE
# globs under `predicate-quantifier: 'every'`. That could NEVER match: 'every' needs a
# single file to be under app/src/test AND docs AND scripts AND .claude AND be *.md at
# once (impossible), so `skippable` was always false and the matrix always ran -- even
# for the docs/scripts-only PRs it meant to skip (issue #420; e.g. scripts-only #419).
# Negated allow-globs are the form dorny documents for `predicate-quantifier: 'every'`.
predicate-quantifier: 'every'
filters: |
non_skippable:
- '!app/src/test/**'
- '!**/*.md'
- '!docs/**'
- '!scripts/**'
- '!.claude/**'
# Safety override (issue #420 HAZARD): the LOCAL E2E harness lives under .claude/,
# so '!.claude/**' above would let a preflight-only change skip E2E. Force the matrix
# when the instrumented-test harness itself changes. Everything else under .claude/
# (agents, hooks, settings, other skills) is dev-only config that CI's build/E2E
# never runs, so it stays skippable. Output is true iff a preflight file changed.
e2e_harness:
- '.claude/skills/preflight/**'
- name: Decide whether the E2E matrix is needed
id: decide
run: |
# e2e_needed is false ONLY for a pull_request where every changed file is in the safe
# allow-list (non_skippable == 'false') AND none touch the harness override
# (e2e_harness == 'false'). Any other event leaves both outputs empty, so we fall
# through to the conservative default of running the E2E matrix.
if [ "${{ github.event_name }}" = "pull_request" ] && [ "${{ steps.filter.outputs.non_skippable }}" = "false" ] && [ "${{ steps.filter.outputs.e2e_harness }}" = "false" ]; then
echo "e2e_needed=false" >> "$GITHUB_OUTPUT"
echo "E2E matrix SKIPPED: every changed file is a docs / unit-test / dev-script / .claude path that cannot affect the app build or instrumented tests."
else
echo "e2e_needed=true" >> "$GITHUB_OUTPUT"
echo "E2E matrix NEEDED: a build/runtime/harness path changed, or this is not a pull_request (conservative default)."
fi
# Runner-priority orchestration (traffic-control) has been EXTRACTED from this file.
# The `traffic-control` job that used to run here first (ordering the heavy jobs below, which
# each declared it as a `needs:` dependency) now lives VERBATIM in its own workflow at
# .github/workflows/traffic-control.yml, and is being mothballed: it will be disabled pending
# a rebuild as a published GitHub Action, so the heavy jobs below no longer depend on it. Its
# pure-Python decision core (.github/scripts/traffic_control.py) is unchanged and is still
# unit-tested by the `traffic-control-tests` job below.
# Fast, pure-stdlib-Python unit tests for the traffic-control decision core
# (.github/scripts/traffic_control.py / test_traffic_control.py — see the
# extracted `traffic-control` workflow). No emulator, no Gradle: this runs in seconds,
# independently of the Android jobs below, so a regression in the runner-priority
# logic fails fast and blocks merge via `ci-passed`.
traffic-control-tests:
name: Traffic-control unit tests
runs-on: ubuntu-latest
steps:
- name: Check out source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Set up Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with:
python-version: "3.x"
- name: Run traffic-controller unit tests
run: python -m unittest discover -s .github/scripts -p 'test_*.py' -v
debug-build:
name: Debug build
# x86_64: Linux-arm64 runners can't set up this SDK — android-actions/setup-android's sdkmanager
# fails (exit 1) on the android-37.0 preview platform, and the emulator package has no arm64-Linux
# build. Build/unit-test results are host-arch-independent anyway (R8/AGP/JVM); real arm64
# device-ABI coverage would need arm64 emulators, which require macOS hosts.
runs-on: ubuntu-latest
steps:
- name: Check out source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Set up JDK 21
uses: actions/setup-java@1bcf9fb12cf4aa7d266a90ae39939e61372fe520 # v5.4.0
with:
distribution: temurin
java-version: "21"
- name: Restore Android SDK cache
id: android-sdk-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
# Cache the SHA-256-verified command-line tools + platform/build-tools so a
# green run doesn't re-download (and risk re-corrupting) them. Restore-only
# here + the success-gated save below == "integrity gates the cache": a
# corrupt/failed SDK is never saved (#389). Shared key (identical contents).
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
# Bump the prefix if ANDROID_PLATFORM/ANDROID_BUILD_TOOLS or the pinned
# cmdline-tools build (14742923) change.
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
# Provision the SHA-256-verified command-line tools into the exact path
# setup-android probes first, so the action reuses it and never does its own
# unverified "Wrong version in preinstalled sdkmanager" re-download (#389).
- name: Bootstrap verified Android command-line tools
run: python3 .github/scripts/setup_android_sdk.py bootstrap
- name: Set up Android SDK
uses: android-actions/setup-android@40fd30fb8d7440372e1316f5d1809ec01dcd3699 # v4.0.1
with:
# Reuse the verified cmdline-tools bootstrapped above (matching version =>
# no unverified re-download) and pass '' packages so the action does NOT run
# the flaky `sdkmanager tools platform-tools` install — the corrupt-zip
# surface that failed the "Set up Android SDK" step (#389).
cmdline-tools-version: "14742923"
packages: ""
# verify -> reject -> retry: a corrupt package zip ("Error reading Zip
# content ...") is purged and re-downloaded instead of failing on sdkmanager's
# bare exit 1 (#389; supersedes the inline retry loop from #387/#388).
- name: Install SDK platform and build-tools
run: python3 .github/scripts/setup_android_sdk.py install "platform-tools" "$ANDROID_PLATFORM" "$ANDROID_BUILD_TOOLS"
# Save the verified SDK only on success (never cache a corrupt SDK) and only on
# a miss (avoid redundant re-saves).
- name: Save Android SDK cache
if: success() && steps.android-sdk-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
- name: Set up Gradle
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
- name: Assemble debug APK
run: ./gradlew assembleDebug --stacktrace
- name: Upload debug APK
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: debug-apk
path: app/build/outputs/apk/debug/*.apk
if-no-files-found: error
unit-tests:
name: Unit tests
runs-on: ubuntu-latest
steps:
- name: Check out source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Set up JDK 21
uses: actions/setup-java@1bcf9fb12cf4aa7d266a90ae39939e61372fe520 # v5.4.0
with:
distribution: temurin
java-version: "21"
- name: Restore Android SDK cache
id: android-sdk-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
# Cache the SHA-256-verified command-line tools + platform/build-tools so a
# green run doesn't re-download (and risk re-corrupting) them. Restore-only
# here + the success-gated save below == "integrity gates the cache": a
# corrupt/failed SDK is never saved (#389). Shared key (identical contents).
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
# Bump the prefix if ANDROID_PLATFORM/ANDROID_BUILD_TOOLS or the pinned
# cmdline-tools build (14742923) change.
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
# Provision the SHA-256-verified command-line tools into the exact path
# setup-android probes first, so the action reuses it and never does its own
# unverified "Wrong version in preinstalled sdkmanager" re-download (#389).
- name: Bootstrap verified Android command-line tools
run: python3 .github/scripts/setup_android_sdk.py bootstrap
- name: Set up Android SDK
uses: android-actions/setup-android@40fd30fb8d7440372e1316f5d1809ec01dcd3699 # v4.0.1
with:
# Reuse the verified cmdline-tools bootstrapped above (matching version =>
# no unverified re-download) and pass '' packages so the action does NOT run
# the flaky `sdkmanager tools platform-tools` install — the corrupt-zip
# surface that failed the "Set up Android SDK" step (#389).
cmdline-tools-version: "14742923"
packages: ""
# verify -> reject -> retry: a corrupt package zip ("Error reading Zip
# content ...") is purged and re-downloaded instead of failing on sdkmanager's
# bare exit 1 (#389; supersedes the inline retry loop from #387/#388).
- name: Install SDK platform and build-tools
run: python3 .github/scripts/setup_android_sdk.py install "platform-tools" "$ANDROID_PLATFORM" "$ANDROID_BUILD_TOOLS"
# Save the verified SDK only on success (never cache a corrupt SDK) and only on
# a miss (avoid redundant re-saves).
- name: Save Android SDK cache
if: success() && steps.android-sdk-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
- name: Set up Gradle
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
- name: Run unit tests
run: ./gradlew testDebugUnitTest --stacktrace
- name: Upload unit test report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: unit-test-report
path: app/build/reports/tests/testDebugUnitTest/
if-no-files-found: warn
# JaCoCo XML + HTML coverage for the JVM unit tests (issue #192), scoped to the JVM-testable
# surface (issues #290/#292). The report is generated and uploaded first, then a no-regression
# gate (issue #251) fails the job if overall LINE coverage drops below the floor pinned in
# app/build.gradle.kts. Kept in this unit-test job so it is part of the `CI passed` gate.
- name: Generate JaCoCo coverage report
if: ${{ !cancelled() }}
run: ./gradlew :app:jacocoTestReport --stacktrace
- name: Upload coverage report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: jacoco-coverage-report
path: app/build/reports/jacoco/jacocoTestReport/
if-no-files-found: warn
# No-regression coverage gate (issue #251): fails CI if scoped LINE coverage regresses below the
# floor pinned in app/build.gradle.kts. Runs after the upload so the HTML/XML report is always
# archived for triage even when this step goes red.
- name: Verify JaCoCo coverage (no-regression floor)
if: ${{ !cancelled() }}
run: ./gradlew :app:jacocoTestCoverageVerification --stacktrace
static-analysis:
name: Static analysis
runs-on: ubuntu-latest
steps:
- name: Check out source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Set up JDK 21
uses: actions/setup-java@1bcf9fb12cf4aa7d266a90ae39939e61372fe520 # v5.4.0
with:
distribution: temurin
java-version: "21"
# AGP configuration needs the SDK even for ktlint/detekt (they run on the :app module).
- name: Restore Android SDK cache
id: android-sdk-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
# Cache the SHA-256-verified command-line tools + platform/build-tools so a
# green run doesn't re-download (and risk re-corrupting) them. Restore-only
# here + the success-gated save below == "integrity gates the cache": a
# corrupt/failed SDK is never saved (#389). Shared key (identical contents).
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
# Bump the prefix if ANDROID_PLATFORM/ANDROID_BUILD_TOOLS or the pinned
# cmdline-tools build (14742923) change.
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
# Provision the SHA-256-verified command-line tools into the exact path
# setup-android probes first, so the action reuses it and never does its own
# unverified "Wrong version in preinstalled sdkmanager" re-download (#389).
- name: Bootstrap verified Android command-line tools
run: python3 .github/scripts/setup_android_sdk.py bootstrap
- name: Set up Android SDK
uses: android-actions/setup-android@40fd30fb8d7440372e1316f5d1809ec01dcd3699 # v4.0.1
with:
# Reuse the verified cmdline-tools bootstrapped above (matching version =>
# no unverified re-download) and pass '' packages so the action does NOT run
# the flaky `sdkmanager tools platform-tools` install — the corrupt-zip
# surface that failed the "Set up Android SDK" step (#389).
cmdline-tools-version: "14742923"
packages: ""
# verify -> reject -> retry: a corrupt package zip ("Error reading Zip
# content ...") is purged and re-downloaded instead of failing on sdkmanager's
# bare exit 1 (#389; supersedes the inline retry loop from #387/#388).
- name: Install SDK platform and build-tools
run: python3 .github/scripts/setup_android_sdk.py install "platform-tools" "$ANDROID_PLATFORM" "$ANDROID_BUILD_TOOLS"
# Save the verified SDK only on success (never cache a corrupt SDK) and only on
# a miss (avoid redundant re-saves).
- name: Save Android SDK cache
if: success() && steps.android-sdk-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
- name: Set up Gradle
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
# --continue so a ktlint failure still lets detekt report (and vice versa).
- name: Run ktlint and detekt
run: ./gradlew :app:ktlintCheck :app:detekt --continue --stacktrace
- name: Upload analysis reports
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: static-analysis-reports
path: |
app/build/reports/ktlint/
app/build/reports/detekt/
if-no-files-found: warn
e2e:
name: E2E
# Path-filter gate (issue #399): gating the whole job means every matrix leg runs — or is
# skipped — together. On an intentional skip the `ci-passed` gate treats e2e's 'skipped'
# result as OK (a real failure/cancel still blocks). `needs: changes` waits only on the
# seconds-long filter job.
needs: changes
if: needs.changes.outputs.e2e_needed == 'true'
runs-on: ubuntu-latest
timeout-minutes: 50
strategy:
fail-fast: false
matrix:
# Every Android API level across the rolling ~7-year support window: minSdk (29 / Android 10,
# 2019) through the latest stable. Each level boots its own emulator and runs the full
# instrumented + Compose UI (E2E) suite; all of them fan in to the "CI passed" gate. When a
# new Android ships, add it and drop the oldest level that has aged out of ~7 years. API 37
# (preview) is NOT in this matrix because emulator-runner can't provision its nonstandard
# android-37.0 / google_apis_ps16k image (it would wedge the gate) — it's covered separately
# by the custom-provisioned `e2e-preview` job below. Keep in sync with
# testOptions.managedDevices in app/build.gradle.kts.
api-level: [29, 30, 31, 32, 33, 34, 35, 36]
steps:
- name: Check out source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Set up JDK 21
uses: actions/setup-java@1bcf9fb12cf4aa7d266a90ae39939e61372fe520 # v5.4.0
with:
distribution: temurin
java-version: "21"
- name: Restore Android SDK cache
id: android-sdk-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
# Cache the SHA-256-verified command-line tools + platform/build-tools so a
# green run doesn't re-download (and risk re-corrupting) them. Restore-only
# here + the success-gated save below == "integrity gates the cache": a
# corrupt/failed SDK is never saved (#389). Shared key (identical contents).
# The emulator + this leg's system image are NOT cached here; they are
# pre-installed via the same #389-hardened installer in a dedicated step below
# (#443), so re-installing is a fast sdkmanager no-op when already present.
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
# Bump the prefix if ANDROID_PLATFORM/ANDROID_BUILD_TOOLS or the pinned
# cmdline-tools build (14742923) change.
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
# Provision the SHA-256-verified command-line tools into the exact path
# setup-android probes first, so the action reuses it and never does its own
# unverified "Wrong version in preinstalled sdkmanager" re-download (#389).
- name: Bootstrap verified Android command-line tools
run: python3 .github/scripts/setup_android_sdk.py bootstrap
- name: Set up Android SDK
uses: android-actions/setup-android@40fd30fb8d7440372e1316f5d1809ec01dcd3699 # v4.0.1
with:
# Reuse the verified cmdline-tools bootstrapped above (matching version =>
# no unverified re-download) and pass '' packages so the action does NOT run
# the flaky `sdkmanager tools platform-tools` install — the corrupt-zip
# surface that failed the "Set up Android SDK" step (#389).
cmdline-tools-version: "14742923"
packages: ""
# verify -> reject -> retry (#389, supersedes the #387/#388 inline loop): sdkmanager
# can exit 1 on a corrupt/truncated package zip ("Error reading Zip content ...") — an
# `E2E (31)` leg died this way. The helper purges each partial/corrupt package and
# re-downloads it clean, and on a hard failure prints the installed-package list so the
# cause is visible in the step log instead of a bare exit 1.
- name: Install SDK platform and build-tools
run: python3 .github/scripts/setup_android_sdk.py install "platform-tools" "$ANDROID_PLATFORM" "$ANDROID_BUILD_TOOLS"
# Save the verified SDK only on success (never cache a corrupt SDK) and only on
# a miss (avoid redundant re-saves).
- name: Save Android SDK cache
if: success() && steps.android-sdk-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
- name: Set up Gradle
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
# The hardware-accelerated emulator needs KVM, which is gated behind a udev rule.
- name: Enable KVM
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules
sudo udevadm trigger --name-match=kvm
# Pre-install the Android Emulator + this leg's google_apis/x86_64 system image through the
# #389-hardened installer (SHA-verify -> reject corrupt -> purge -> re-download) BEFORE the
# Create AVD + Boot steps below. Historically the android-emulator-runner's un-guarded "Create
# AVD and generate snapshot" fetch died on a corrupt emulator zip ("Error on ZipFile unknown
# archive") and wedged the merge queue for ~2h (#443); routing the install through this hardened
# path self-heals a corrupt zip here, and the manual boot below then finds both packages already
# installed. The emulator binary + system image live under the SDK root; they are intentionally
# NOT added to the android-sdk-v1 cache (kept small) — re-install is a fast sdkmanager no-op when
# already present. Keep target (google_apis) / arch (x86_64) in lockstep with the Create AVD +
# boot steps below and testOptions.managedDevices in app/build.gradle.kts.
- name: Pre-install emulator + system image (hardened)
run: python3 .github/scripts/setup_android_sdk.py install "emulator" "system-images;android-${{ matrix.api-level }};google_apis;x86_64"
# Hand-provision the AVD with avdmanager so the emulator is COLD-booted below (parity with the
# e2e-preview / api37_e2e.py manual boot) instead of resumed from a cached snapshot — snapshot
# resume is exactly what raced system_server's binder republish (#448). Pin ANDROID_AVD_HOME so
# avdmanager and the emulator agree on the AVD dir (they disagree when ANDROID_SDK_HOME is set on
# the runner, which else reports "Unknown AVD name") and carry it to the boot step via $GITHUB_ENV.
# Device profile (pixel_2) + image (google_apis / x86_64) stay in lockstep with
# testOptions.managedDevices in app/build.gradle.kts.
- name: Create AVD
run: |
export ANDROID_AVD_HOME="$HOME/.android/avd"
echo "ANDROID_AVD_HOME=$ANDROID_AVD_HOME" >> "$GITHUB_ENV"
mkdir -p "$ANDROID_AVD_HOME"
echo "no" | avdmanager create avd -n test -k "system-images;android-${{ matrix.api-level }};google_apis;x86_64" -d pixel_2 --force
echo "AVDs visible to the emulator:"; "$ANDROID_SDK_ROOT/emulator/emulator" -list-avds
# Boot the emulator with the e2e-preview job's proven hand-provisioned manual-boot pattern
# instead of reactivecircus/android-emulator-runner's default boot wait (#448). The action runs
# an un-guarded, fatal `adb shell input keyevent 82` right after sys.boot_completed=1; on a
# snapshot resume that raced the `input` binder service's republish and aborted the job with
# "No service published for: input" — an intermittent boot race that flaked the merge queue and
# hit BOTH the run and its retry once #446 finally let runs reach boot on API 33. The manual
# boot fixes it with: a 2-attempt COLD-boot loop; ONE bounded
# `timeout 300 adb wait-for-device shell 'wait sys.boot_completed'` per attempt (a stuck
# emulator fails fast instead of hanging to the 50-min job cap); a NON-FATAL
# `adb shell input keyevent 82 || true` unlock (killing the race); and a readiness gate
# (boot_completed==1) BEFORE connectedDebugAndroidTest runs. Emulator flags mirror e2e-preview
# (cold `-no-snapshot -accel on -gpu swiftshader_indirect`, plus `-verbose -debug ...` for boot
# diagnosability). The gradle run is wrapped in the #404 wedge-capture `timeout` — RESTORED to
# this matrix by #421. It was reverted in #406 (90dfb18) for hanging all 8 legs, but that
# wrapper wrapped reactivecircus/android-emulator-runner's emulator boot; #454 replaced that
# with this manual boot, so the wrapper now wraps ONLY the foreground gradle client — never the
# backgrounded emulator — exactly like e2e-preview's proven run_shard, and so cannot re-hang the
# leg. See capture_wedge / run_e2e below. The gradle run retries ONCE on a TEST failure (parity
# with the replaced retry step); boot flakes are covered by the 2-attempt boot loop above.
- name: Boot emulator and run E2E
run: |
set -euo pipefail
EMU_LOG="${RUNNER_TEMP:-/tmp}/emulator-api${{ matrix.api-level }}.log"
LOGCAT_LOG="${RUNNER_TEMP:-/tmp}/logcat-api${{ matrix.api-level }}.txt"
DIAG_LOG="${RUNNER_TEMP:-/tmp}/boot-diagnostics-api${{ matrix.api-level }}.txt"
# Wedge (hang) smoking-gun capture (#404, restored to the matrix by #421) — see
# capture_wedge / run_e2e below. WEDGE_TIMEOUT mirrors e2e-preview's 1200s: a healthy
# matrix leg has run the whole job in ~8.3–12.0 min historically and the wrapper only wraps
# the ./gradlew invocation (a subset of that), so 20 min is generous no-false-trip headroom
# yet well under this job's 50-min cap so a genuine wedge is caught and captured.
WEDGE_LOG="${RUNNER_TEMP:-/tmp}/wedge-diagnostics-api${{ matrix.api-level }}.txt"
WEDGE_TIMEOUT=1200
GPU_MODE="swiftshader_indirect"
# On a boot timeout, dump full system state into $DIAG_LOG for the artifact upload, then a
# concise summary to the step log so the cause is visible without downloading artifacts.
# Every probe is guarded (|| true) so a missing tool can't abort the retry under `set -e`.
dump_diagnostics() {
local attempt="$1" accel kvm
accel=$("$ANDROID_SDK_ROOT/emulator/emulator" -accel-check 2>&1) || true
kvm=$(ls -l /dev/kvm 2>&1) || true
{
echo "===== API ${{ matrix.api-level }} boot diagnostics (attempt $attempt) ====="
echo "--- adb devices ---"; adb devices 2>&1 || true
echo "--- emulator -accel-check ---"; echo "$accel"
echo "--- /dev/kvm ---"; echo "$kvm"
echo "--- GPU mode ---"; echo "$GPU_MODE"
echo "--- free memory ---"; free -h 2>&1 || true
echo "--- free disk ---"; df -h 2>&1 || true
echo "--- AVD config.ini ---"; cat "${ANDROID_AVD_HOME:-$HOME/.android/avd}/test.avd/config.ini" 2>&1 || true
echo "--- emulator.log (tail 200) ---"; tail -200 "$EMU_LOG" 2>&1 || true
} >> "$DIAG_LOG" 2>&1 || true
echo "----- BOOT FAILURE SUMMARY (attempt $attempt) -----"
echo "accel-check: $accel"
echo "/dev/kvm: $kvm"
echo "--- emulator.log (tail 50) ---"; tail -50 "$EMU_LOG" 2>&1 || true
}
boot_emulator() {
echo "::group::Start API ${{ matrix.api-level }} emulator (attempt $1)"
# Cold boot (-no-snapshot); capture the emulator's own output so a boot flake is
# diagnosable. -verbose -debug ... is DIAGNOSTICS ONLY — no boot-affecting flag changes.
"$ANDROID_SDK_ROOT/emulator/emulator" -avd test \
-no-window -no-audio -no-boot-anim -no-snapshot -accel on \
-gpu "$GPU_MODE" -camera-back none -camera-front none \
-verbose -debug init,avd_config,kernel > "$EMU_LOG" 2>&1 &
# Stream logcat from the moment the device registers into a file that survives to the
# artifact upload. Appended (with a header) per attempt.
echo "===== logcat (attempt $1) =====" >> "$LOGCAT_LOG"
adb wait-for-device logcat -v time >> "$LOGCAT_LOG" 2>&1 &
logcat_pid=$!
# ONE bounded wait covering both device registration and full boot, so a stuck emulator
# fails fast instead of hanging the whole job until the 50-min cap.
if timeout 300 adb wait-for-device shell \
'while [ "$(getprop sys.boot_completed | tr -d "\r")" != "1" ]; do sleep 2; done'; then
echo "::endgroup::"; return 0
fi
echo "::endgroup::"
echo "::warning::API ${{ matrix.api-level }} emulator did not boot within 300s (attempt $1)"
dump_diagnostics "$1"
kill "$logcat_pid" 2>/dev/null || true
adb emu kill 2>/dev/null || true
sleep 5
return 1
}
# Start the adb daemon up-front (mirrors api37_e2e.py) so attempt 1 can't lose the
# adb-server "Address already in use" bind race.
adb start-server || true
booted=0
for attempt in 1 2; do boot_emulator "$attempt" && { booted=1; break; }; done
[ "$booted" = "1" ] || { echo "::error::API ${{ matrix.api-level }} emulator failed to boot after 2 attempts"; exit 1; }
# Focus/readiness gate (issue #468): the durable fix for the intermittent app-window-
# focus flake. Occasionally the launched activity window has has-window-focus=false for
# the WHOLE run, so Espresso's RootViewPicker (onView().check(), Intents.intended(),
# pressBack(), focus-dependent clipboard) times out after 10s and fails EVERY
# focus-dependent test at once while the ~280 pure-Compose tests pass (evidence: a
# failing E2E leg's logcat had has-window-focus=true ZERO times, both attempt + retry).
# The single `input keyevent 82` here was too weak (MENU no longer dismisses the modern
# keyguard, and races SystemUI coming up). This shared helper WAKES the display, dismisses
# + disables the keyguard, keeps the screen on, disables animations (parity with the
# replaced `disable-animations: true`), then WAITS until a real window holds input focus
# before the suite runs. It is the IDENTICAL mechanism the e2e-preview job and the local
# preflight runners invoke, so the fix cannot drift between jobs. Non-fatal (`|| true`),
# preserving #454's guarantee that the unlock never aborts the boot.
python3 .github/scripts/emulator_focus_gate.py --label "api${{ matrix.api-level }}" || true
# WEDGE (hang) smoking-gun capture (#404, restored to the matrix by #421). On the wrapper
# `timeout` below (exit 124), grab the smoking gun WHILE this hand-provisioned emulator is
# still alive (it stays up for the rest of this step): which test was running, SIGQUIT
# (kill -3) thread dumps of the app + instrumentation processes (ART -> logcat + /data/anr),
# activity/window state, and — the boot-race crux — whether the binder services are
# published. Every probe guarded so a missing tool / dead device can't abort it; appended so
# both attempts survive. `|| true` keeps it from tripping this step's `set -e`. Identical in
# shape to e2e-preview's capture_wedge.
capture_wedge() {
{
echo "==================================================================="
echo "===== E2E WEDGE — API ${{ matrix.api-level }} — $1 ====="
echo "===== $(date -u +%FT%TZ) — after ${WEDGE_TIMEOUT}s wrapper timeout ====="
echo "==================================================================="
echo "--- snapshot: N/A — matrix cold-boots (-no-snapshot); no AVD snapshot cache ---"
echo "--- running/last instrumented test (logcat TestRunner) ---"
grep -a TestRunner "$LOGCAT_LOG" 2>/dev/null | tail -25 || true
echo "--- getprop sys.boot_completed ---"
adb shell getprop sys.boot_completed 2>&1 || true
echo "--- getprop init.svc.* (per-service init state) ---"
adb shell getprop 2>&1 | grep -a init.svc || true
echo "--- service list (are binder services published?) ---"
adb shell service list 2>&1 || true
for svc in input window activity; do
echo "--- service check $svc ---"
adb shell service check "$svc" 2>&1 || true
done
echo "--- pids ---"
APP_PID="$(adb shell pidof org.libremail.app 2>/dev/null | tr -d '\r')" || true
TEST_PID="$(adb shell pidof org.libremail.app.test 2>/dev/null | tr -d '\r')" || true
echo "app pid: ${APP_PID:-<none>}"
echo "test pid: ${TEST_PID:-<none>}"
echo "--- SIGQUIT (kill -3) thread dumps -> ART writes to logcat + /data/anr ---"
for pid in $APP_PID $TEST_PID; do
[ -n "$pid" ] && adb shell kill -3 "$pid" 2>&1 || true
done
sleep 5
echo "--- /data/anr/* (SIGQUIT + ANR traces) ---"
adb shell 'cat /data/anr/* 2>/dev/null' 2>&1 || true
echo "--- dumpsys activity ---"
adb shell dumpsys activity 2>&1 || true
echo "--- dumpsys window ---"
adb shell dumpsys window 2>&1 || true
echo "--- logcat -d (tail 400 — includes the SIGQUIT thread dump) ---"
adb logcat -d 2>&1 | tail -400 || true
echo "--- emulator accel / kvm / mem / disk ---"
"$ANDROID_SDK_ROOT/emulator/emulator" -accel-check 2>&1 || true
ls -l /dev/kvm 2>&1 || true
free -h 2>&1 || true
df -h 2>&1 || true
} >> "$WEDGE_LOG" 2>&1 || true
echo "::warning::E2E API ${{ matrix.api-level }} WEDGED ($1) — see the wedge-diagnostics-api${{ matrix.api-level }} artifact"
}
# Retry the test run ONCE on failure (parity with the replaced retry step). A retried-but-
# passed run is flagged as a ::warning:: because a blanket retry can mask a real regression.
# The gradle run is wrapped in the #404 wedge wrapper `timeout`: a wedge (hang) trips it
# (exit 124) -> capture_wedge runs while the emulator is still alive, then the run returns
# 124 so the retry / gate still see a failure. `|| status=$?` makes the exit code survive
# this step's `set -e`; -k 30s SIGKILLs a gradle client that ignores SIGTERM. The wrapper
# wraps ONLY the foreground gradle client (NOT the backgrounded emulator), identical in
# shape to e2e-preview's run_shard — the structural reason it cannot re-hang the leg.
run_e2e() {
local status=0
timeout -k 30s "$WEDGE_TIMEOUT" ./gradlew connectedDebugAndroidTest --stacktrace || status=$?
if [ "$status" -eq 124 ]; then capture_wedge "$1"; fi
return "$status"
}
run_e2e "attempt 1" || { echo "::warning::API ${{ matrix.api-level }} E2E run failed — retrying once"; run_e2e "attempt 2 (retry)"; }
# On any E2E failure (both boot attempts failed, a hung emulator, or an earlier setup/SDK
# step), snapshot device + runner state to the step log AND a file for the artifact upload —
# the `E2E (31)` sdkmanager death (#387) left no trail. Each probe is guarded (|| true) so a
# missing tool / offline device can't abort the step; accel-check, /dev/kvm, free -h and
# df -h characterise the runner even when the emulator never booted.
- name: Dump emulator + system diagnostics on failure
if: failure()
run: |
DIAG="${RUNNER_TEMP:-/tmp}/diagnostics-api${{ matrix.api-level }}.txt"
{
echo "===== E2E API ${{ matrix.api-level }} failure diagnostics ====="
echo "--- adb devices ---"; adb devices 2>&1 || true
echo "--- adb logcat -d (tail 200) ---"; adb logcat -d 2>&1 | tail -200 || true
echo "--- emulator -accel-check ---"; "$ANDROID_SDK_ROOT/emulator/emulator" -accel-check 2>&1 || true
echo "--- /dev/kvm ---"; ls -l /dev/kvm 2>&1 || true
echo "--- free memory ---"; free -h 2>&1 || true
echo "--- free disk ---"; df -h 2>&1 || true
} 2>&1 | tee "$DIAG"
- name: Upload E2E test report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-test-report-api${{ matrix.api-level }}
path: app/build/reports/androidTests/connected/
if-no-files-found: warn
# Always upload the streamed logcat + (on failure) the system-state dump so an emulator flake
# or a red matrix leg is diagnosable from artifacts without a re-run — parity with the
# e2e-preview boot-diagnostics artifact. Per-api-level name (upload-artifact@v7 rejects
# duplicate artifact names).
- name: Upload E2E diagnostics
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-diagnostics-api${{ matrix.api-level }}
path: |
${{ runner.temp }}/logcat-api${{ matrix.api-level }}.txt
${{ runner.temp }}/diagnostics-api${{ matrix.api-level }}.txt
${{ runner.temp }}/emulator-api${{ matrix.api-level }}.log
${{ runner.temp }}/boot-diagnostics-api${{ matrix.api-level }}.txt
if-no-files-found: warn
# Wedge-specific smoking gun (#404, restored to the matrix by #421): only present when the
# wrapper `timeout` tripped (a hang) on either attempt — separate from the E2E diagnostics
# artifact above. `if-no-files-found: ignore` keeps healthy runs quiet (no wedge => no file).
# Per-api-level name (upload-artifact@v7 rejects duplicate artifact names). Mirrors the
# e2e-preview job's wedge-diagnostics artifact.
- name: Upload wedge diagnostics
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: wedge-diagnostics-api${{ matrix.api-level }}
path: ${{ runner.temp }}/wedge-diagnostics-api${{ matrix.api-level }}.txt
if-no-files-found: ignore
# API 37 (Android 17, preview) E2E. Its only system image is the nonstandard
# android-37.0 / google_apis_ps16k (16 KB page size), which reactivecircus/android-emulator-runner
# can't provision (it builds android-37 / google_apis, neither of which exists), so this job
# CUSTOM-PROVISIONS the emulator with sdkmanager/avdmanager/emulator directly. It is REQUIRED:
# part of the "CI passed" gate's needs (the preview emulator has proven stable in practice), so a
# genuine failure blocks merges. When a stable, emulator-runner-friendly API 37 image ships, fold
# 37 into the main `e2e` matrix and delete this job.
e2e-preview:
name: E2E (API 37 preview)
# Path-filter gate (issue #399): runs or skips together with the `e2e` matrix. On an
# intentional skip the `ci-passed` gate treats e2e-preview's 'skipped' result as OK; a real
# failure/cancel still blocks. Both shard legs fan in under this one gated job.
needs: changes
if: needs.changes.outputs.e2e_needed == 'true'
runs-on: ubuntu-latest
timeout-minutes: 35
# Sharded N=2 (docs/perf/api37-e2e-sharding-spike.md). The instrumented suite is split across
# 2 parallel API 37 emulators — each a separate matrix leg that hand-provisions its OWN emulator
# and runs one shard via AndroidJUnitRunner's numShards/shardIndex. This leg is the pipeline's
# critical path, so sharding cuts it ~17.1 min -> ~12.7 min (~28% off total CI) for +1 emulator
# job. FAN-IN: `ci-passed` lists `e2e-preview` once and GHA rolls BOTH shard legs under that one
# entry — a matrix job's aggregate result is `failure` if ANY leg fails — so both shards must
# pass, and branch protection (which requires the "CI passed" context, not the per-leg
# "E2E (API 37 preview) (N)" names) needs no change. `fail-fast: false` lets one shard's failure
# NOT cancel the other, so both reports always upload. Keep API37_NUM_SHARDS in lockstep with the
# length of matrix.shard.
strategy:
fail-fast: false
matrix:
shard: [0, 1] # 0-based shardIndex values; length must equal API37_NUM_SHARDS below
env:
API37_IMAGE: "system-images;android-37.0;google_apis_ps16k;x86_64"
API37_NUM_SHARDS: "2"
steps:
- name: Check out source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Set up JDK 21
uses: actions/setup-java@1bcf9fb12cf4aa7d266a90ae39939e61372fe520 # v5.4.0
with:
distribution: temurin
java-version: "21"
- name: Restore Android SDK cache
id: android-sdk-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
# Cache the SHA-256-verified command-line tools + platform/build-tools so a
# green run doesn't re-download (and risk re-corrupting) them. Restore-only
# here + the success-gated save below == "integrity gates the cache": a
# corrupt/failed SDK is never saved (#389). The ~1 GB preview system image
# keeps its own cache below.
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
# Bump the prefix if ANDROID_PLATFORM/ANDROID_BUILD_TOOLS or the pinned
# cmdline-tools build (14742923) change.
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
# Provision the SHA-256-verified command-line tools into the exact path
# setup-android probes first, so the action reuses it and never does its own
# unverified "Wrong version in preinstalled sdkmanager" re-download (#389).
- name: Bootstrap verified Android command-line tools
run: python3 .github/scripts/setup_android_sdk.py bootstrap
- name: Set up Android SDK
uses: android-actions/setup-android@40fd30fb8d7440372e1316f5d1809ec01dcd3699 # v4.0.1
with:
# Reuse the verified cmdline-tools bootstrapped above (matching version =>
# no unverified re-download) and pass '' packages so the action does NOT run
# the flaky `sdkmanager tools platform-tools` install — the corrupt-zip
# surface that failed the "Set up Android SDK" step (#389).
cmdline-tools-version: "14742923"
packages: ""
- name: Set up Gradle
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
# The hardware-accelerated emulator needs KVM, which is gated behind a udev rule.
- name: Enable KVM
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules
sudo udevadm trigger --name-match=kvm
# Cache the ~1 GB preview system image so only the first run pays the download.
- name: Cache API 37 system image
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
# GitHub-hosted ubuntu runners install the SDK at /usr/local/lib/android/sdk; caching the
# image dir (with its package metadata) lets sdkmanager treat it as installed and skip the
# re-download on a cache hit.
path: /usr/local/lib/android/sdk/system-images/android-37.0
key: sysimg-android-37.0-google_apis_ps16k-x86_64
# verify -> reject -> retry (#389): the preview emulator + 16 KB-page system image
# download here; a corrupt/truncated package zip ("Error reading Zip content ...") is
# purged and re-downloaded clean instead of failing on sdkmanager's bare exit 1.
- name: Install SDK packages + preview system image
run: python3 .github/scripts/setup_android_sdk.py install "platform-tools" "emulator" "$ANDROID_PLATFORM" "$ANDROID_BUILD_TOOLS" "$API37_IMAGE"
# Save the verified command-line tools + platform/build-tools only on success and only
# on a miss. The ~1 GB system image keeps its own cache above; the emulator re-downloads
# (self-healing via the retry) to keep this shared key small.
- name: Save Android SDK cache
if: success() && steps.android-sdk-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
- name: Create API 37 AVD
run: |
# avdmanager and the emulator disagree on the default AVD dir when ANDROID_SDK_HOME is set
# on the runner (avdmanager writes $ANDROID_SDK_HOME/.android/avd; the emulator looks in
# $ANDROID_SDK_HOME/avd), which made the boot step report "Unknown AVD name [api37]". Pin
# ANDROID_AVD_HOME so both agree, and carry it to the boot step via $GITHUB_ENV.
export ANDROID_AVD_HOME="$HOME/.android/avd"
echo "ANDROID_AVD_HOME=$ANDROID_AVD_HOME" >> "$GITHUB_ENV"
mkdir -p "$ANDROID_AVD_HOME"
echo "no" | avdmanager create avd -n api37 -k "$API37_IMAGE" -d pixel_2 --force
echo "AVDs visible to the emulator:"; "$ANDROID_SDK_ROOT/emulator/emulator" -list-avds
- name: Boot emulator and run E2E
run: |
set -euo pipefail
EMU_LOG="${RUNNER_TEMP:-/tmp}/emulator.log"
LOGCAT_LOG="${RUNNER_TEMP:-/tmp}/logcat.txt"
DIAG_LOG="${RUNNER_TEMP:-/tmp}/boot-diagnostics.txt"
# Wedge (hang) smoking-gun capture (#404) — see capture_wedge / run_shard below.
WEDGE_LOG="${RUNNER_TEMP:-/tmp}/wedge-diagnostics-api37-shard${{ matrix.shard }}.txt"
WEDGE_TIMEOUT=1200
GPU_MODE="swiftshader_indirect"
# On a boot timeout, capture the full system state (accel/KVM/GPU/mem/disk/AVD config +
# emulator.log tail) into $DIAG_LOG for the artifact upload, then print a CONCISE summary
# (accel/KVM status + last 50 lines of emulator.log) to the step log so the cause is
# visible in the run output without downloading artifacts. Every probe is guarded (|| true)
# so a missing tool can't abort the retry under `set -e`.
dump_diagnostics() {
local attempt="$1" accel kvm
accel=$("$ANDROID_SDK_ROOT/emulator/emulator" -accel-check 2>&1) || true
kvm=$(ls -l /dev/kvm 2>&1) || true
{
echo "===== API 37 boot diagnostics (attempt $attempt) ====="
echo "--- adb devices ---"; adb devices 2>&1 || true
echo "--- emulator -accel-check ---"; echo "$accel"
echo "--- /dev/kvm ---"; echo "$kvm"
echo "--- GPU mode ---"; echo "$GPU_MODE"
echo "--- free memory ---"; free -h 2>&1 || true
echo "--- free disk ---"; df -h 2>&1 || true
echo "--- AVD config.ini ---"; cat "${ANDROID_AVD_HOME:-$HOME/.android/avd}/api37.avd/config.ini" 2>&1 || true
echo "--- emulator.log (tail 200) ---"; tail -200 "$EMU_LOG" 2>&1 || true
} >> "$DIAG_LOG" 2>&1 || true
echo "----- BOOT FAILURE SUMMARY (attempt $attempt) -----"
echo "accel-check: $accel"
echo "/dev/kvm: $kvm"
echo "--- emulator.log (tail 50) ---"; tail -50 "$EMU_LOG" 2>&1 || true
}
boot_emulator() {
echo "::group::Start API 37 emulator (attempt $1)"
# Capture the emulator's own output — without this a boot failure is invisible.
# -verbose -debug init,avd_config,kernel turns boot logging on by default so a boot flake
# is diagnosable from $EMU_LOG; DIAGNOSTICS ONLY — no boot-affecting flag is changed.
"$ANDROID_SDK_ROOT/emulator/emulator" -avd api37 \
-no-window -no-audio -no-boot-anim -no-snapshot -accel on \
-gpu "$GPU_MODE" -camera-back none -camera-front none \
-verbose -debug init,avd_config,kernel > "$EMU_LOG" 2>&1 &
# Stream logcat from the moment the device registers (wait-for-device blocks until then)
# into a file that survives to the artifact upload. Appended (with a header) per attempt.
echo "===== logcat (attempt $1) =====" >> "$LOGCAT_LOG"
adb wait-for-device logcat -v time >> "$LOGCAT_LOG" 2>&1 &
logcat_pid=$!
# ONE bounded wait covering both device registration and full boot, so a stuck emulator
# fails fast instead of hanging the whole job until the 35-min cap (the original bug).
if timeout 300 adb wait-for-device shell \
'while [ "$(getprop sys.boot_completed | tr -d "\r")" != "1" ]; do sleep 2; done'; then
echo "::endgroup::"; return 0
fi
echo "::endgroup::"
echo "::warning::API 37 emulator did not boot within 300s (attempt $1)"
dump_diagnostics "$1"
kill "$logcat_pid" 2>/dev/null || true
adb emu kill 2>/dev/null || true
sleep 5
return 1
}
# Start the adb daemon up-front (mirrors api37_e2e.py wait_for_boot) so attempt 1 can't
# lose the adb-server "Address already in use" bind race that flaked #370's boot.
adb start-server || true
booted=0
for attempt in 1 2; do boot_emulator "$attempt" && { booted=1; break; }; done
[ "$booted" = "1" ] || { echo "::error::API 37 preview emulator failed to boot after 2 attempts"; exit 1; }
# Focus/readiness gate (issue #468) -- the IDENTICAL shared mechanism the `e2e` matrix job
# (and the local preflight runners) invoke, so the fix can't drift: wake the display,
# dismiss + disable the keyguard, keep the screen on, disable animations, then WAIT until a
# real window holds input focus before the suite runs. This replaces the lone `input
# keyevent 82` and, applied UNIFORMLY, also gives this preview job the animation-disable
# the matrix already had. Non-fatal (`|| true`) -- the unlock must never abort the boot.
python3 .github/scripts/emulator_focus_gate.py --label "api37-shard${{ matrix.shard }}" || true
# WEDGE (hang) smoking-gun capture (#404). On the wrapper `timeout` below (exit 124), grab
# the smoking gun WHILE this hand-provisioned emulator is still alive (it stays up until the
# "Shut down emulator" step): which test was running, SIGQUIT (kill -3) thread dumps of the
# app + instrumentation processes (ART -> logcat + /data/anr), activity/window state, and —
# the boot-race crux — whether the binder services are published. Every probe guarded so a
# missing tool / dead device can't abort it; appended so both attempts survive. `|| true`
# keeps it from tripping this step's `set -e`.
capture_wedge() {
{
echo "==================================================================="
echo "===== E2E WEDGE — API 37 preview shard ${{ matrix.shard }} — $1 ====="
echo "===== $(date -u +%FT%TZ) — after ${WEDGE_TIMEOUT}s wrapper timeout ====="
echo "==================================================================="
echo "--- snapshot: N/A — preview cold-boots (-no-snapshot); no AVD snapshot cache ---"
echo "--- running/last instrumented test (logcat TestRunner) ---"
grep -a TestRunner "$LOGCAT_LOG" 2>/dev/null | tail -25 || true
echo "--- getprop sys.boot_completed ---"
adb shell getprop sys.boot_completed 2>&1 || true
echo "--- getprop init.svc.* (per-service init state) ---"
adb shell getprop 2>&1 | grep -a init.svc || true
echo "--- service list (are binder services published?) ---"
adb shell service list 2>&1 || true
for svc in input window activity; do
echo "--- service check $svc ---"
adb shell service check "$svc" 2>&1 || true
done
echo "--- pids ---"
APP_PID="$(adb shell pidof org.libremail.app 2>/dev/null | tr -d '\r')" || true
TEST_PID="$(adb shell pidof org.libremail.app.test 2>/dev/null | tr -d '\r')" || true
echo "app pid: ${APP_PID:-<none>}"
echo "test pid: ${TEST_PID:-<none>}"
echo "--- SIGQUIT (kill -3) thread dumps -> ART writes to logcat + /data/anr ---"
for pid in $APP_PID $TEST_PID; do
[ -n "$pid" ] && adb shell kill -3 "$pid" 2>&1 || true
done
sleep 5
echo "--- /data/anr/* (SIGQUIT + ANR traces) ---"
adb shell 'cat /data/anr/* 2>/dev/null' 2>&1 || true
echo "--- dumpsys activity ---"
adb shell dumpsys activity 2>&1 || true
echo "--- dumpsys window ---"
adb shell dumpsys window 2>&1 || true
echo "--- logcat -d (tail 400 — includes the SIGQUIT thread dump) ---"
adb logcat -d 2>&1 | tail -400 || true
echo "--- emulator accel / kvm / mem / disk ---"
"$ANDROID_SDK_ROOT/emulator/emulator" -accel-check 2>&1 || true
ls -l /dev/kvm 2>&1 || true
free -h 2>&1 || true
df -h 2>&1 || true
} >> "$WEDGE_LOG" 2>&1 || true
echo "::warning::E2E API 37 preview shard ${{ matrix.shard }} WEDGED ($1) — see the wedge-diagnostics-api37-preview-shard${{ matrix.shard }} artifact"
}
# Run only this matrix leg's shard. AndroidJUnitRunner hashes each test name into one of
# numShards buckets and runs only shardIndex's bucket. The args flow through AGP's
# -Pandroid.testInstrumentationRunnerArguments.* channel (the same one local_instrumented.py
# uses for .class=…) — no GMD, no orchestrator, no Gradle-side change. The gradle run is
# wrapped in the #404 wrapper `timeout`: a wedge trips it (exit 124) -> capture_wedge runs,
# then the shard returns 124 so the retry / gate still see a failure. `|| status=$?` makes
# the exit code survive this step's `set -e`; -k 30s SIGKILLs a gradle client that ignores
# SIGTERM.
run_shard() {
local status=0
timeout -k 30s "$WEDGE_TIMEOUT" ./gradlew connectedDebugAndroidTest --stacktrace \
"-Pandroid.testInstrumentationRunnerArguments.numShards=${API37_NUM_SHARDS}" \
"-Pandroid.testInstrumentationRunnerArguments.shardIndex=${{ matrix.shard }}" || status=$?
if [ "$status" -eq 124 ]; then capture_wedge "$1"; fi
return "$status"
}
# Retry this shard's test run ONCE on failure — retry parity with the stable `e2e` matrix
# (which retries once, so a flaky test self-heals on API 29–36 but would otherwise wedge
# the required gate on API 37, e.g. #370). Per-shard, so the retry re-runs only this shard's
# ~T/N tests, not the whole suite. MITIGATION, NOT A FIX: a blanket retry also masks genuine
# regressions, so the retried-but-passed case is flagged as a ::warning:: and the real fix
# stays test-level. See docs/perf/api37-e2e-sharding-spike.md. A wedge on EITHER attempt is
# captured (capture_wedge runs inside run_shard on exit 124).
run_shard "attempt 1" || { echo "::warning::API 37 shard ${{ matrix.shard }} test run failed — retrying once"; run_shard "attempt 2 (retry)"; }
- name: Dump emulator log on failure
if: failure()
run: |
echo "--- emulator.log ---"; tail -200 "${RUNNER_TEMP:-/tmp}/emulator.log" 2>/dev/null || echo "(none)"
echo "--- logcat ---"; adb logcat -d 2>/dev/null | tail -120 || echo "(device unavailable)"
# Always upload the emulator log + captured logcat + boot-diagnostics dump so a boot flake
# (which can time out or be cancelled) is diagnosable from artifacts without a re-run.
- name: Upload emulator boot diagnostics
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
# Shard-unique name — upload-artifact@v7 errors on duplicate artifact names.
name: e2e-api37-boot-diagnostics-shard${{ matrix.shard }}
path: |
${{ runner.temp }}/emulator.log
${{ runner.temp }}/logcat.txt
${{ runner.temp }}/boot-diagnostics.txt
if-no-files-found: warn
# Wedge-specific smoking gun (#404): only present when the wrapper `timeout` tripped (a hang)
# on either shard attempt — separate from the boot-diagnostics artifact above. `if-no-files-
# found: ignore` keeps healthy runs quiet (no wedge => no file). Shard-unique name.
- name: Upload wedge diagnostics
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: wedge-diagnostics-api37-preview-shard${{ matrix.shard }}
path: ${{ runner.temp }}/wedge-diagnostics-api37-shard${{ matrix.shard }}.txt
if-no-files-found: ignore
- name: Shut down emulator
if: always()
run: adb emu kill || true
- name: Upload E2E (API 37) report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
# Shard-unique name — upload-artifact@v7 errors on duplicate artifact names.
name: e2e-test-report-api37-preview-shard${{ matrix.shard }}
path: app/build/reports/androidTests/connected/
if-no-files-found: warn
# Single aggregating gate so branch protection can require ALL CI jobs with one stable status
# check. It depends on every job — including each api-level of the E2E matrix — so adding/removing
# a matrix level needs no change to branch protection (the per-"(api-level)" check names would
# otherwise have to be re-listed each time).
ci-passed:
name: CI passed
if: always()
# `traffic-control` is intentionally NOT listed here — it has been extracted to its own
# (mothballed/disabled) workflow, .github/workflows/traffic-control.yml, and the heavy jobs
# below no longer depend on it, so it plays no part in this gate. The `traffic-control-tests`
# job (its pure-Python decision-core unit tests) IS a required input below.
#
# E2E path-filter (issue #399): `e2e` / `e2e-preview` are SKIPPED for test-only/docs/script
# PRs (see the `changes` job). A skip there is INTENTIONAL and must PASS, so — unlike the
# naive `contains(needs.*.result, 'skipped')` gate this replaces — 'skipped' is TOLERATED for
# those two jobs only. The gate stays fail-safe by BLOCKING on:
# * changes != success (a broken/failed filter never waves a PR through)
# * traffic-control-tests / static-analysis / debug-build / unit-tests != success
# * e2e == failure OR cancelled (a real E2E failure/cancel still blocks)
# * e2e-preview == failure OR cancelled
# i.e. e2e/e2e-preview may be ONLY 'success' or 'skipped'; every other required job must be
# 'success'. (If `changes` itself fails, e2e/e2e-preview skip — but `changes != success`
# blocks the merge anyway, so a broken filter is never waved through.)
needs: [changes, traffic-control-tests, static-analysis, debug-build, unit-tests, e2e, e2e-preview]
runs-on: ubuntu-latest
steps:
# Always echo every required job's result (and the filter decision) for debuggability,
# before the gate step decides pass/fail.
- name: Echo required job results
run: |
echo "Required-job results (an E2E 'skipped' is allowed ONLY via the #399 path-filter):"
echo " changes: ${{ needs.changes.result }} (e2e_needed=${{ needs.changes.outputs.e2e_needed }})"
echo " traffic-control-tests: ${{ needs.traffic-control-tests.result }}"
echo " static-analysis: ${{ needs.static-analysis.result }}"
echo " debug-build: ${{ needs.debug-build.result }}"
echo " unit-tests: ${{ needs.unit-tests.result }}"
echo " e2e: ${{ needs.e2e.result }}"
echo " e2e-preview: ${{ needs.e2e-preview.result }}"
- name: Fail unless every required job passed (an intentionally path-filtered E2E skip is OK)
if: >-
needs.changes.result != 'success' ||
needs.traffic-control-tests.result != 'success' ||
needs.static-analysis.result != 'success' ||
needs.debug-build.result != 'success' ||
needs.unit-tests.result != 'success' ||
needs.e2e.result == 'failure' || needs.e2e.result == 'cancelled' ||
needs.e2e-preview.result == 'failure' || needs.e2e-preview.result == 'cancelled'
run: |
echo "::error::Required CI jobs did not all succeed (see the results above)."
echo "A path-filtered E2E 'skipped' is allowed; an E2E 'failure'/'cancelled', or any"
echo "non-success in changes/traffic-control-tests/static-analysis/debug-build/unit-tests, is not."
exit 1