Files
LibreMail/.github/workflows/ci.yml
T
JMR-devandClaude Opus 4.8 aa628831be ci: retry + cache Android SDK/emulator setup to survive corrupt-zip sdkmanager failures (#389)
The dominant merge-blocking flake was the "Set up Android SDK" step
(android-actions/setup-android v4.0.1) dying BEFORE the emulator starts:

    Wrong version in preinstalled sdkmanager
    Warning: ... preparing SDK package Android Emulator: Error reading Zip
    content from a SeekableByteChannel.
    Error: The process '.../sdkmanager' failed with exit code 1

Root cause: the action's default cmdline-tools version (20.0) rarely matches the
runner image's preinstalled one, so it logs "Wrong version in preinstalled
sdkmanager" and re-fetches cmdline-tools with NO checksum; it then runs its
default `sdkmanager tools platform-tools` install. Any of those downloads can be
a corrupt/truncated zip, which sdkmanager turns into an un-retried exit 1. v4.0.1
is the latest release, so this is fixed by configuration + hardening, not a bump.

Harden with verify -> reject -> retry, never trusting sdkmanager's exit code
alone, via a new stdlib-only helper .github/scripts/setup_android_sdk.py:

- bootstrap: download the pinned cmdline-tools zip, verify size + SHA-256
  (authoritative pin, cross-checked against Google's published SHA-1), and
  install it to $ANDROID_SDK_ROOT/cmdline-tools/20.0 -- the exact path
  setup-android probes first, so the action reuses the verified tree and never
  does its own unverified "Wrong version" re-download. A mismatch (corrupt OR
  wrong version) deletes the bad zip + any half-extracted dir and re-downloads.
- install: sdkmanager --install with retry + backoff; on a corrupt package zip it
  purges the partial/corrupt package dir (and sdkmanager's temp dirs) before
  retrying, forcing a fresh download instead of a re-read.
- setup-android now runs with packages: "" (no flaky tools/platform-tools
  install) and cmdline-tools-version: "14742923" (reuse the verified bootstrap).
- actions/cache restore + success-gated save so only a verified SDK is ever
  cached (integrity gates the cache); shrinks the re-download/corruption surface.

Applied to every SDK-setup job (debug-build, unit-tests, static-analysis, e2e
matrix, e2e-preview). Emulator BOOT logic, #372 API-37 sharding, and #388
diagnostics are untouched. Pure-logic helpers are unit-tested
(test_setup_android_sdk.py, run by the traffic-control-tests job).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:59:05 -05:00

811 lines
43 KiB
YAML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# SPDX-License-Identifier: GPL-3.0-or-later
name: CI
on:
pull_request:
branches: [main]
# The traffic-controller SCHEDULER (ci-trigger.yml, issue #349) (re-)triggers CI for a
# specific PR via this workflow_dispatch after a GITHUB_TOKEN auto-update has left the PR's
# head SHA with absent/stale checks. Dispatched on the PR's head BRANCH, so the run's checks
# land on the PR head SHA and satisfy branch protection. `on: pull_request` above is KEPT so
# brand-new PRs, human pushes, and fork PRs still get CI directly — this is the fail-open
# guarantee: CI is always triggerable even if the scheduler is broken or absent.
workflow_dispatch:
inputs:
pr:
description: "PR number this run is for (set by the traffic-controller scheduler)."
required: false
type: string
head_sha:
description: "Expected head SHA (informational, for traceability in the run log)."
required: false
type: string
reason:
description: "Why this run was dispatched (informational)."
required: false
type: string
# A new trigger for a PR cancels that PR's own in-flight run (a newer head SHA supersedes).
# The group is keyed to the PR NUMBER so a `pull_request` run and a scheduler
# `workflow_dispatch` run for the SAME PR share one concurrency group (either supersedes a
# stale run of the other); it falls back to the ref when no PR number is in context.
concurrency:
group: ci-pr-${{ github.event.pull_request.number || inputs.pr || github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
# SDK packages this project builds against (compileSdk 37 / build-tools 37.0.0).
# Quote the package ids when passed to sdkmanager — the ';' is a shell separator.
ANDROID_PLATFORM: "platforms;android-37.0"
ANDROID_BUILD_TOOLS: "build-tools;37.0.0"
jobs:
# Runner-priority orchestration (traffic-control) has been EXTRACTED from this file.
# The `traffic-control` job that used to run here first (ordering the heavy jobs below, which
# each declared it as a `needs:` dependency) now lives VERBATIM in its own workflow at
# .github/workflows/traffic-control.yml, and is being mothballed: it will be disabled pending
# a rebuild as a published GitHub Action, so the heavy jobs below no longer depend on it. Its
# pure-Python decision core (.github/scripts/traffic_control.py) is unchanged and is still
# unit-tested by the `traffic-control-tests` job below.
# Fast, pure-stdlib-Python unit tests for the traffic-control decision core
# (.github/scripts/traffic_control.py / test_traffic_control.py — see the
# extracted `traffic-control` workflow). No emulator, no Gradle: this runs in seconds,
# independently of the Android jobs below, so a regression in the runner-priority
# logic fails fast and blocks merge via `ci-passed`.
traffic-control-tests:
name: Traffic-control unit tests
runs-on: ubuntu-latest
steps:
- name: Check out source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Set up Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with:
python-version: "3.x"
- name: Run traffic-controller unit tests
run: python -m unittest discover -s .github/scripts -p 'test_*.py' -v
debug-build:
name: Debug build
# x86_64: Linux-arm64 runners can't set up this SDK — android-actions/setup-android's sdkmanager
# fails (exit 1) on the android-37.0 preview platform, and the emulator package has no arm64-Linux
# build. Build/unit-test results are host-arch-independent anyway (R8/AGP/JVM); real arm64
# device-ABI coverage would need arm64 emulators, which require macOS hosts.
runs-on: ubuntu-latest
steps:
- name: Check out source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Set up JDK 21
uses: actions/setup-java@1bcf9fb12cf4aa7d266a90ae39939e61372fe520 # v5.4.0
with:
distribution: temurin
java-version: "21"
- name: Restore Android SDK cache
id: android-sdk-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
# Cache the SHA-256-verified command-line tools + platform/build-tools so a
# green run doesn't re-download (and risk re-corrupting) them. Restore-only
# here + the success-gated save below == "integrity gates the cache": a
# corrupt/failed SDK is never saved (#389). Shared key (identical contents).
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
# Bump the prefix if ANDROID_PLATFORM/ANDROID_BUILD_TOOLS or the pinned
# cmdline-tools build (14742923) change.
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
# Provision the SHA-256-verified command-line tools into the exact path
# setup-android probes first, so the action reuses it and never does its own
# unverified "Wrong version in preinstalled sdkmanager" re-download (#389).
- name: Bootstrap verified Android command-line tools
run: python3 .github/scripts/setup_android_sdk.py bootstrap
- name: Set up Android SDK
uses: android-actions/setup-android@40fd30fb8d7440372e1316f5d1809ec01dcd3699 # v4.0.1
with:
# Reuse the verified cmdline-tools bootstrapped above (matching version =>
# no unverified re-download) and pass '' packages so the action does NOT run
# the flaky `sdkmanager tools platform-tools` install — the corrupt-zip
# surface that failed the "Set up Android SDK" step (#389).
cmdline-tools-version: "14742923"
packages: ""
# verify -> reject -> retry: a corrupt package zip ("Error reading Zip
# content ...") is purged and re-downloaded instead of failing on sdkmanager's
# bare exit 1 (#389; supersedes the inline retry loop from #387/#388).
- name: Install SDK platform and build-tools
run: python3 .github/scripts/setup_android_sdk.py install "platform-tools" "$ANDROID_PLATFORM" "$ANDROID_BUILD_TOOLS"
# Save the verified SDK only on success (never cache a corrupt SDK) and only on
# a miss (avoid redundant re-saves).
- name: Save Android SDK cache
if: success() && steps.android-sdk-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
- name: Set up Gradle
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
- name: Assemble debug APK
run: ./gradlew assembleDebug --stacktrace
- name: Upload debug APK
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: debug-apk
path: app/build/outputs/apk/debug/*.apk
if-no-files-found: error
unit-tests:
name: Unit tests
runs-on: ubuntu-latest
steps:
- name: Check out source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Set up JDK 21
uses: actions/setup-java@1bcf9fb12cf4aa7d266a90ae39939e61372fe520 # v5.4.0
with:
distribution: temurin
java-version: "21"
- name: Restore Android SDK cache
id: android-sdk-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
# Cache the SHA-256-verified command-line tools + platform/build-tools so a
# green run doesn't re-download (and risk re-corrupting) them. Restore-only
# here + the success-gated save below == "integrity gates the cache": a
# corrupt/failed SDK is never saved (#389). Shared key (identical contents).
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
# Bump the prefix if ANDROID_PLATFORM/ANDROID_BUILD_TOOLS or the pinned
# cmdline-tools build (14742923) change.
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
# Provision the SHA-256-verified command-line tools into the exact path
# setup-android probes first, so the action reuses it and never does its own
# unverified "Wrong version in preinstalled sdkmanager" re-download (#389).
- name: Bootstrap verified Android command-line tools
run: python3 .github/scripts/setup_android_sdk.py bootstrap
- name: Set up Android SDK
uses: android-actions/setup-android@40fd30fb8d7440372e1316f5d1809ec01dcd3699 # v4.0.1
with:
# Reuse the verified cmdline-tools bootstrapped above (matching version =>
# no unverified re-download) and pass '' packages so the action does NOT run
# the flaky `sdkmanager tools platform-tools` install — the corrupt-zip
# surface that failed the "Set up Android SDK" step (#389).
cmdline-tools-version: "14742923"
packages: ""
# verify -> reject -> retry: a corrupt package zip ("Error reading Zip
# content ...") is purged and re-downloaded instead of failing on sdkmanager's
# bare exit 1 (#389; supersedes the inline retry loop from #387/#388).
- name: Install SDK platform and build-tools
run: python3 .github/scripts/setup_android_sdk.py install "platform-tools" "$ANDROID_PLATFORM" "$ANDROID_BUILD_TOOLS"
# Save the verified SDK only on success (never cache a corrupt SDK) and only on
# a miss (avoid redundant re-saves).
- name: Save Android SDK cache
if: success() && steps.android-sdk-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
- name: Set up Gradle
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
- name: Run unit tests
run: ./gradlew testDebugUnitTest --stacktrace
- name: Upload unit test report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: unit-test-report
path: app/build/reports/tests/testDebugUnitTest/
if-no-files-found: warn
# JaCoCo XML + HTML coverage for the JVM unit tests (issue #192), scoped to the JVM-testable
# surface (issues #290/#292). The report is generated and uploaded first, then a no-regression
# gate (issue #251) fails the job if overall LINE coverage drops below the floor pinned in
# app/build.gradle.kts. Kept in this unit-test job so it is part of the `CI passed` gate.
- name: Generate JaCoCo coverage report
if: ${{ !cancelled() }}
run: ./gradlew :app:jacocoTestReport --stacktrace
- name: Upload coverage report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: jacoco-coverage-report
path: app/build/reports/jacoco/jacocoTestReport/
if-no-files-found: warn
# No-regression coverage gate (issue #251): fails CI if scoped LINE coverage regresses below the
# floor pinned in app/build.gradle.kts. Runs after the upload so the HTML/XML report is always
# archived for triage even when this step goes red.
- name: Verify JaCoCo coverage (no-regression floor)
if: ${{ !cancelled() }}
run: ./gradlew :app:jacocoTestCoverageVerification --stacktrace
static-analysis:
name: Static analysis
runs-on: ubuntu-latest
steps:
- name: Check out source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Set up JDK 21
uses: actions/setup-java@1bcf9fb12cf4aa7d266a90ae39939e61372fe520 # v5.4.0
with:
distribution: temurin
java-version: "21"
# AGP configuration needs the SDK even for ktlint/detekt (they run on the :app module).
- name: Restore Android SDK cache
id: android-sdk-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
# Cache the SHA-256-verified command-line tools + platform/build-tools so a
# green run doesn't re-download (and risk re-corrupting) them. Restore-only
# here + the success-gated save below == "integrity gates the cache": a
# corrupt/failed SDK is never saved (#389). Shared key (identical contents).
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
# Bump the prefix if ANDROID_PLATFORM/ANDROID_BUILD_TOOLS or the pinned
# cmdline-tools build (14742923) change.
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
# Provision the SHA-256-verified command-line tools into the exact path
# setup-android probes first, so the action reuses it and never does its own
# unverified "Wrong version in preinstalled sdkmanager" re-download (#389).
- name: Bootstrap verified Android command-line tools
run: python3 .github/scripts/setup_android_sdk.py bootstrap
- name: Set up Android SDK
uses: android-actions/setup-android@40fd30fb8d7440372e1316f5d1809ec01dcd3699 # v4.0.1
with:
# Reuse the verified cmdline-tools bootstrapped above (matching version =>
# no unverified re-download) and pass '' packages so the action does NOT run
# the flaky `sdkmanager tools platform-tools` install — the corrupt-zip
# surface that failed the "Set up Android SDK" step (#389).
cmdline-tools-version: "14742923"
packages: ""
# verify -> reject -> retry: a corrupt package zip ("Error reading Zip
# content ...") is purged and re-downloaded instead of failing on sdkmanager's
# bare exit 1 (#389; supersedes the inline retry loop from #387/#388).
- name: Install SDK platform and build-tools
run: python3 .github/scripts/setup_android_sdk.py install "platform-tools" "$ANDROID_PLATFORM" "$ANDROID_BUILD_TOOLS"
# Save the verified SDK only on success (never cache a corrupt SDK) and only on
# a miss (avoid redundant re-saves).
- name: Save Android SDK cache
if: success() && steps.android-sdk-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
- name: Set up Gradle
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
# --continue so a ktlint failure still lets detekt report (and vice versa).
- name: Run ktlint and detekt
run: ./gradlew :app:ktlintCheck :app:detekt --continue --stacktrace
- name: Upload analysis reports
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: static-analysis-reports
path: |
app/build/reports/ktlint/
app/build/reports/detekt/
if-no-files-found: warn
e2e:
name: E2E
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
# Every Android API level across the rolling ~7-year support window: minSdk (29 / Android 10,
# 2019) through the latest stable. Each level boots its own emulator and runs the full
# instrumented + Compose UI (E2E) suite; all of them fan in to the "CI passed" gate. When a
# new Android ships, add it and drop the oldest level that has aged out of ~7 years. API 37
# (preview) is NOT in this matrix because emulator-runner can't provision its nonstandard
# android-37.0 / google_apis_ps16k image (it would wedge the gate) — it's covered separately
# by the custom-provisioned `e2e-preview` job below. Keep in sync with
# testOptions.managedDevices in app/build.gradle.kts.
api-level: [29, 30, 31, 32, 33, 34, 35, 36]
steps:
- name: Check out source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Set up JDK 21
uses: actions/setup-java@1bcf9fb12cf4aa7d266a90ae39939e61372fe520 # v5.4.0
with:
distribution: temurin
java-version: "21"
- name: Restore Android SDK cache
id: android-sdk-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
# Cache the SHA-256-verified command-line tools + platform/build-tools so a
# green run doesn't re-download (and risk re-corrupting) them. Restore-only
# here + the success-gated save below == "integrity gates the cache": a
# corrupt/failed SDK is never saved (#389). Shared key (identical contents).
# The emulator + system image stay with android-emulator-runner (boot logic
# is out of scope for #389).
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
# Bump the prefix if ANDROID_PLATFORM/ANDROID_BUILD_TOOLS or the pinned
# cmdline-tools build (14742923) change.
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
# Provision the SHA-256-verified command-line tools into the exact path
# setup-android probes first, so the action reuses it and never does its own
# unverified "Wrong version in preinstalled sdkmanager" re-download (#389).
- name: Bootstrap verified Android command-line tools
run: python3 .github/scripts/setup_android_sdk.py bootstrap
- name: Set up Android SDK
uses: android-actions/setup-android@40fd30fb8d7440372e1316f5d1809ec01dcd3699 # v4.0.1
with:
# Reuse the verified cmdline-tools bootstrapped above (matching version =>
# no unverified re-download) and pass '' packages so the action does NOT run
# the flaky `sdkmanager tools platform-tools` install — the corrupt-zip
# surface that failed the "Set up Android SDK" step (#389).
cmdline-tools-version: "14742923"
packages: ""
# verify -> reject -> retry (#389, supersedes the #387/#388 inline loop): sdkmanager
# can exit 1 on a corrupt/truncated package zip ("Error reading Zip content ...") — an
# `E2E (31)` leg died this way. The helper purges each partial/corrupt package and
# re-downloads it clean, and on a hard failure prints the installed-package list so the
# cause is visible in the step log instead of a bare exit 1.
- name: Install SDK platform and build-tools
run: python3 .github/scripts/setup_android_sdk.py install "platform-tools" "$ANDROID_PLATFORM" "$ANDROID_BUILD_TOOLS"
# Save the verified SDK only on success (never cache a corrupt SDK) and only on
# a miss (avoid redundant re-saves).
- name: Save Android SDK cache
if: success() && steps.android-sdk-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
- name: Set up Gradle
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
# The hardware-accelerated emulator needs KVM, which is gated behind a udev rule.
- name: Enable KVM
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules
sudo udevadm trigger --name-match=kvm
- name: Cache AVD snapshot
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
id: avd-cache
with:
path: |
~/.android/avd/*
~/.android/adb*
key: avd-${{ matrix.api-level }}-google_apis-x86_64
# On a cache miss, cold-boot the emulator once so its snapshot can be cached,
# making subsequent runs start from a warm snapshot.
- name: Create AVD and generate snapshot for caching
if: steps.avd-cache.outputs.cache-hit != 'true'
uses: reactivecircus/android-emulator-runner@e89f39f1abbbd05b1113a29cf4db69e7540cae5a # v2.37.0
with:
api-level: ${{ matrix.api-level }}
target: google_apis
arch: x86_64
force-avd-creation: false
emulator-options: -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -camera-back none
disable-animations: false
script: echo "Generated AVD snapshot for caching."
# reactivecircus/android-emulator-runner runs an un-guarded, fatal `adb shell input keyevent 82`
# after boot. On snapshot resume that can race system_server (sys.boot_completed=1 before the
# `input` binder service is republished), aborting the job before Gradle runs with
# "No service published for: input" — an ~2%, API-29-only infra flake, not a test failure. Make
# the step non-fatal and retry once: two independent boots drop the race to ~0.04%. The definitive
# fix (adopt the e2e-preview job's manual-boot + `keyevent 82 || true`) is tracked separately.
- name: Run E2E tests
id: e2e
continue-on-error: true
uses: reactivecircus/android-emulator-runner@e89f39f1abbbd05b1113a29cf4db69e7540cae5a # v2.37.0
with:
api-level: ${{ matrix.api-level }}
target: google_apis
arch: x86_64
force-avd-creation: false
emulator-options: -no-snapshot-save -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -camera-back none
disable-animations: true
# Stream logcat to a per-api-level file (the emulator is booted here) before the tests,
# so a test failure or emulator flake is diagnosable from the uploaded artifact.
# Backgrounded; gradle stays the last foreground command so the step's exit status is
# still the test result (a real failure still trips continue-on-error -> the retry).
script: |
adb logcat -v time > "$RUNNER_TEMP/logcat-api${{ matrix.api-level }}.txt" 2>&1 &
./gradlew connectedDebugAndroidTest --stacktrace
- name: Run E2E tests (retry after emulator boot race)
if: steps.e2e.outcome == 'failure'
uses: reactivecircus/android-emulator-runner@e89f39f1abbbd05b1113a29cf4db69e7540cae5a # v2.37.0
with:
api-level: ${{ matrix.api-level }}
target: google_apis
arch: x86_64
force-avd-creation: false
emulator-options: -no-snapshot-save -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -camera-back none
disable-animations: true
# Retry runs a fresh emulator boot; stream its logcat the same way. `>` overwrites
# attempt 1's file so the artifact holds the FINAL attempt's logs, matching the
# failure-time dump below (which reflects this last attempt's state).
script: |
adb logcat -v time > "$RUNNER_TEMP/logcat-api${{ matrix.api-level }}.txt" 2>&1 &
./gradlew connectedDebugAndroidTest --stacktrace
# On any E2E failure (both boot attempts failed, a hung emulator, or an earlier setup/SDK
# step), snapshot device + runner state to the step log AND a file for the artifact upload —
# the `E2E (31)` sdkmanager death (#387) left no trail. Each probe is guarded (|| true) so a
# missing tool / offline device can't abort the step; accel-check, /dev/kvm, free -h and
# df -h characterise the runner even when the emulator never booted.
- name: Dump emulator + system diagnostics on failure
if: failure()
run: |
DIAG="${RUNNER_TEMP:-/tmp}/diagnostics-api${{ matrix.api-level }}.txt"
{
echo "===== E2E API ${{ matrix.api-level }} failure diagnostics ====="
echo "--- adb devices ---"; adb devices 2>&1 || true
echo "--- adb logcat -d (tail 200) ---"; adb logcat -d 2>&1 | tail -200 || true
echo "--- emulator -accel-check ---"; "$ANDROID_SDK_ROOT/emulator/emulator" -accel-check 2>&1 || true
echo "--- /dev/kvm ---"; ls -l /dev/kvm 2>&1 || true
echo "--- free memory ---"; free -h 2>&1 || true
echo "--- free disk ---"; df -h 2>&1 || true
} 2>&1 | tee "$DIAG"
- name: Upload E2E test report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-test-report-api${{ matrix.api-level }}
path: app/build/reports/androidTests/connected/
if-no-files-found: warn
# Always upload the streamed logcat + (on failure) the system-state dump so an emulator flake
# or a red matrix leg is diagnosable from artifacts without a re-run — parity with the
# e2e-preview boot-diagnostics artifact. Per-api-level name (upload-artifact@v7 rejects
# duplicate artifact names).
- name: Upload E2E diagnostics
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-diagnostics-api${{ matrix.api-level }}
path: |
${{ runner.temp }}/logcat-api${{ matrix.api-level }}.txt
${{ runner.temp }}/diagnostics-api${{ matrix.api-level }}.txt
if-no-files-found: warn
# API 37 (Android 17, preview) E2E. Its only system image is the nonstandard
# android-37.0 / google_apis_ps16k (16 KB page size), which reactivecircus/android-emulator-runner
# can't provision (it builds android-37 / google_apis, neither of which exists), so this job
# CUSTOM-PROVISIONS the emulator with sdkmanager/avdmanager/emulator directly. It is REQUIRED:
# part of the "CI passed" gate's needs (the preview emulator has proven stable in practice), so a
# genuine failure blocks merges. When a stable, emulator-runner-friendly API 37 image ships, fold
# 37 into the main `e2e` matrix and delete this job.
e2e-preview:
name: E2E (API 37 preview)
runs-on: ubuntu-latest
timeout-minutes: 35
# Sharded N=2 (docs/perf/api37-e2e-sharding-spike.md). The instrumented suite is split across
# 2 parallel API 37 emulators — each a separate matrix leg that hand-provisions its OWN emulator
# and runs one shard via AndroidJUnitRunner's numShards/shardIndex. This leg is the pipeline's
# critical path, so sharding cuts it ~17.1 min -> ~12.7 min (~28% off total CI) for +1 emulator
# job. FAN-IN: `ci-passed` lists `e2e-preview` once and GHA rolls BOTH shard legs under that one
# entry — a matrix job's aggregate result is `failure` if ANY leg fails — so both shards must
# pass, and branch protection (which requires the "CI passed" context, not the per-leg
# "E2E (API 37 preview) (N)" names) needs no change. `fail-fast: false` lets one shard's failure
# NOT cancel the other, so both reports always upload. Keep API37_NUM_SHARDS in lockstep with the
# length of matrix.shard.
strategy:
fail-fast: false
matrix:
shard: [0, 1] # 0-based shardIndex values; length must equal API37_NUM_SHARDS below
env:
API37_IMAGE: "system-images;android-37.0;google_apis_ps16k;x86_64"
API37_NUM_SHARDS: "2"
steps:
- name: Check out source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Set up JDK 21
uses: actions/setup-java@1bcf9fb12cf4aa7d266a90ae39939e61372fe520 # v5.4.0
with:
distribution: temurin
java-version: "21"
- name: Restore Android SDK cache
id: android-sdk-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
# Cache the SHA-256-verified command-line tools + platform/build-tools so a
# green run doesn't re-download (and risk re-corrupting) them. Restore-only
# here + the success-gated save below == "integrity gates the cache": a
# corrupt/failed SDK is never saved (#389). The ~1 GB preview system image
# keeps its own cache below.
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
# Bump the prefix if ANDROID_PLATFORM/ANDROID_BUILD_TOOLS or the pinned
# cmdline-tools build (14742923) change.
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
# Provision the SHA-256-verified command-line tools into the exact path
# setup-android probes first, so the action reuses it and never does its own
# unverified "Wrong version in preinstalled sdkmanager" re-download (#389).
- name: Bootstrap verified Android command-line tools
run: python3 .github/scripts/setup_android_sdk.py bootstrap
- name: Set up Android SDK
uses: android-actions/setup-android@40fd30fb8d7440372e1316f5d1809ec01dcd3699 # v4.0.1
with:
# Reuse the verified cmdline-tools bootstrapped above (matching version =>
# no unverified re-download) and pass '' packages so the action does NOT run
# the flaky `sdkmanager tools platform-tools` install — the corrupt-zip
# surface that failed the "Set up Android SDK" step (#389).
cmdline-tools-version: "14742923"
packages: ""
- name: Set up Gradle
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
# The hardware-accelerated emulator needs KVM, which is gated behind a udev rule.
- name: Enable KVM
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules
sudo udevadm trigger --name-match=kvm
# Cache the ~1 GB preview system image so only the first run pays the download.
- name: Cache API 37 system image
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
# GitHub-hosted ubuntu runners install the SDK at /usr/local/lib/android/sdk; caching the
# image dir (with its package metadata) lets sdkmanager treat it as installed and skip the
# re-download on a cache hit.
path: /usr/local/lib/android/sdk/system-images/android-37.0
key: sysimg-android-37.0-google_apis_ps16k-x86_64
# verify -> reject -> retry (#389): the preview emulator + 16 KB-page system image
# download here; a corrupt/truncated package zip ("Error reading Zip content ...") is
# purged and re-downloaded clean instead of failing on sdkmanager's bare exit 1.
- name: Install SDK packages + preview system image
run: python3 .github/scripts/setup_android_sdk.py install "platform-tools" "emulator" "$ANDROID_PLATFORM" "$ANDROID_BUILD_TOOLS" "$API37_IMAGE"
# Save the verified command-line tools + platform/build-tools only on success and only
# on a miss. The ~1 GB system image keeps its own cache above; the emulator re-downloads
# (self-healing via the retry) to keep this shared key small.
- name: Save Android SDK cache
if: success() && steps.android-sdk-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
/usr/local/lib/android/sdk/cmdline-tools/20.0
/usr/local/lib/android/sdk/platforms
/usr/local/lib/android/sdk/build-tools
/usr/local/lib/android/sdk/platform-tools
key: android-sdk-v1-${{ runner.os }}-clt14742923-plat37.0-bt37.0.0
- name: Create API 37 AVD
run: |
# avdmanager and the emulator disagree on the default AVD dir when ANDROID_SDK_HOME is set
# on the runner (avdmanager writes $ANDROID_SDK_HOME/.android/avd; the emulator looks in
# $ANDROID_SDK_HOME/avd), which made the boot step report "Unknown AVD name [api37]". Pin
# ANDROID_AVD_HOME so both agree, and carry it to the boot step via $GITHUB_ENV.
export ANDROID_AVD_HOME="$HOME/.android/avd"
echo "ANDROID_AVD_HOME=$ANDROID_AVD_HOME" >> "$GITHUB_ENV"
mkdir -p "$ANDROID_AVD_HOME"
echo "no" | avdmanager create avd -n api37 -k "$API37_IMAGE" -d pixel_2 --force
echo "AVDs visible to the emulator:"; "$ANDROID_SDK_ROOT/emulator/emulator" -list-avds
- name: Boot emulator and run E2E
run: |
set -euo pipefail
EMU_LOG="${RUNNER_TEMP:-/tmp}/emulator.log"
LOGCAT_LOG="${RUNNER_TEMP:-/tmp}/logcat.txt"
DIAG_LOG="${RUNNER_TEMP:-/tmp}/boot-diagnostics.txt"
GPU_MODE="swiftshader_indirect"
# On a boot timeout, capture the full system state (accel/KVM/GPU/mem/disk/AVD config +
# emulator.log tail) into $DIAG_LOG for the artifact upload, then print a CONCISE summary
# (accel/KVM status + last 50 lines of emulator.log) to the step log so the cause is
# visible in the run output without downloading artifacts. Every probe is guarded (|| true)
# so a missing tool can't abort the retry under `set -e`.
dump_diagnostics() {
local attempt="$1" accel kvm
accel=$("$ANDROID_SDK_ROOT/emulator/emulator" -accel-check 2>&1) || true
kvm=$(ls -l /dev/kvm 2>&1) || true
{
echo "===== API 37 boot diagnostics (attempt $attempt) ====="
echo "--- adb devices ---"; adb devices 2>&1 || true
echo "--- emulator -accel-check ---"; echo "$accel"
echo "--- /dev/kvm ---"; echo "$kvm"
echo "--- GPU mode ---"; echo "$GPU_MODE"
echo "--- free memory ---"; free -h 2>&1 || true
echo "--- free disk ---"; df -h 2>&1 || true
echo "--- AVD config.ini ---"; cat "${ANDROID_AVD_HOME:-$HOME/.android/avd}/api37.avd/config.ini" 2>&1 || true
echo "--- emulator.log (tail 200) ---"; tail -200 "$EMU_LOG" 2>&1 || true
} >> "$DIAG_LOG" 2>&1 || true
echo "----- BOOT FAILURE SUMMARY (attempt $attempt) -----"
echo "accel-check: $accel"
echo "/dev/kvm: $kvm"
echo "--- emulator.log (tail 50) ---"; tail -50 "$EMU_LOG" 2>&1 || true
}
boot_emulator() {
echo "::group::Start API 37 emulator (attempt $1)"
# Capture the emulator's own output — without this a boot failure is invisible.
# -verbose -debug init,avd_config,kernel turns boot logging on by default so a boot flake
# is diagnosable from $EMU_LOG; DIAGNOSTICS ONLY — no boot-affecting flag is changed.
"$ANDROID_SDK_ROOT/emulator/emulator" -avd api37 \
-no-window -no-audio -no-boot-anim -no-snapshot -accel on \
-gpu "$GPU_MODE" -camera-back none -camera-front none \
-verbose -debug init,avd_config,kernel > "$EMU_LOG" 2>&1 &
# Stream logcat from the moment the device registers (wait-for-device blocks until then)
# into a file that survives to the artifact upload. Appended (with a header) per attempt.
echo "===== logcat (attempt $1) =====" >> "$LOGCAT_LOG"
adb wait-for-device logcat -v time >> "$LOGCAT_LOG" 2>&1 &
logcat_pid=$!
# ONE bounded wait covering both device registration and full boot, so a stuck emulator
# fails fast instead of hanging the whole job until the 35-min cap (the original bug).
if timeout 300 adb wait-for-device shell \
'while [ "$(getprop sys.boot_completed | tr -d "\r")" != "1" ]; do sleep 2; done'; then
echo "::endgroup::"; return 0
fi
echo "::endgroup::"
echo "::warning::API 37 emulator did not boot within 300s (attempt $1)"
dump_diagnostics "$1"
kill "$logcat_pid" 2>/dev/null || true
adb emu kill 2>/dev/null || true
sleep 5
return 1
}
# Start the adb daemon up-front (mirrors api37_e2e.py wait_for_boot) so attempt 1 can't
# lose the adb-server "Address already in use" bind race that flaked #370's boot.
adb start-server || true
booted=0
for attempt in 1 2; do boot_emulator "$attempt" && { booted=1; break; }; done
[ "$booted" = "1" ] || { echo "::error::API 37 preview emulator failed to boot after 2 attempts"; exit 1; }
adb shell input keyevent 82 || true
# Run only this matrix leg's shard. AndroidJUnitRunner hashes each test name into one of
# numShards buckets and runs only shardIndex's bucket. The args flow through AGP's
# -Pandroid.testInstrumentationRunnerArguments.* channel (the same one local_instrumented.py
# uses for .class=…) — no GMD, no orchestrator, no Gradle-side change.
run_shard() {
./gradlew connectedDebugAndroidTest --stacktrace \
"-Pandroid.testInstrumentationRunnerArguments.numShards=${API37_NUM_SHARDS}" \
"-Pandroid.testInstrumentationRunnerArguments.shardIndex=${{ matrix.shard }}"
}
# Retry this shard's test run ONCE on failure — retry parity with the stable `e2e` matrix
# (which retries once, so a flaky test self-heals on API 29–36 but would otherwise wedge
# the required gate on API 37, e.g. #370). Per-shard, so the retry re-runs only this shard's
# ~T/N tests, not the whole suite. MITIGATION, NOT A FIX: a blanket retry also masks genuine
# regressions, so the retried-but-passed case is flagged as a ::warning:: and the real fix
# stays test-level. See docs/perf/api37-e2e-sharding-spike.md.
run_shard || { echo "::warning::API 37 shard ${{ matrix.shard }} test run failed — retrying once"; run_shard; }
- name: Dump emulator log on failure
if: failure()
run: |
echo "--- emulator.log ---"; tail -200 "${RUNNER_TEMP:-/tmp}/emulator.log" 2>/dev/null || echo "(none)"
echo "--- logcat ---"; adb logcat -d 2>/dev/null | tail -120 || echo "(device unavailable)"
# Always upload the emulator log + captured logcat + boot-diagnostics dump so a boot flake
# (which can time out or be cancelled) is diagnosable from artifacts without a re-run.
- name: Upload emulator boot diagnostics
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
# Shard-unique name — upload-artifact@v7 errors on duplicate artifact names.
name: e2e-api37-boot-diagnostics-shard${{ matrix.shard }}
path: |
${{ runner.temp }}/emulator.log
${{ runner.temp }}/logcat.txt
${{ runner.temp }}/boot-diagnostics.txt
if-no-files-found: warn
- name: Shut down emulator
if: always()
run: adb emu kill || true
- name: Upload E2E (API 37) report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
# Shard-unique name — upload-artifact@v7 errors on duplicate artifact names.
name: e2e-test-report-api37-preview-shard${{ matrix.shard }}
path: app/build/reports/androidTests/connected/
if-no-files-found: warn
# Single aggregating gate so branch protection can require ALL CI jobs with one stable status
# check. It depends on every job — including each api-level of the E2E matrix — so adding/removing
# a matrix level needs no change to branch protection (the per-"(api-level)" check names would
# otherwise have to be re-listed each time).
ci-passed:
name: CI passed
if: always()
# `traffic-control` is intentionally NOT listed here — it has been extracted to its own
# (mothballed/disabled) workflow, .github/workflows/traffic-control.yml, and the heavy jobs
# below no longer depend on it, so it plays no part in this gate. The `traffic-control-tests`
# job (its pure-Python decision-core unit tests) IS a required input below. Treating a
# 'skipped'/'cancelled' required job as a gate failure keeps this fail-safe (blocks the
# merge rather than letting an untested change through).
needs: [traffic-control-tests, static-analysis, debug-build, unit-tests, e2e, e2e-preview]
runs-on: ubuntu-latest
steps:
- name: Verify every required job succeeded
if: ${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') || contains(needs.*.result, 'skipped') }}
run: |
echo "Required CI jobs did not all succeed:"
echo " traffic-control-tests: ${{ needs.traffic-control-tests.result }}"
echo " static-analysis: ${{ needs.static-analysis.result }}"
echo " debug-build: ${{ needs.debug-build.result }}"
echo " unit-tests: ${{ needs.unit-tests.result }}"
echo " e2e: ${{ needs.e2e.result }}"
echo " e2e-preview: ${{ needs.e2e-preview.result }}"
exit 1