ci: retry + cache Android SDK/emulator setup to survive corrupt-zip sdkmanager failures (#389) #391

Merged
JMR-dev merged 2 commits from ci-389-sdk-setup-hardening into main 2026-07-07 00:22:02 +00:00
JMR-dev commented 2026-07-06 23:59:52 +00:00 (Migrated from github.com)

Problem (#389)

The dominant merge-blocking flake is the Set up Android SDK step
(android-actions/setup-android) dying before the emulator ever starts:

Wrong version in preinstalled sdkmanager
Warning: ... preparing SDK package Android Emulator: Error reading Zip content from a SeekableByteChannel.
Error: The process '.../sdkmanager' failed with exit code 1

Seen on E2E (31), E2E (35) (PR #375) and a #388 preview shard (the "SDK Tools" package).

Root cause

setup-android v4.0.1 is already the latest release — there is no newer version to
bump to, so this is a configuration + hardening fix. Reading the action source:

  • Its default cmdline-tools-version (build 14742923 = rev 20.0) rarely matches the
    runner image's preinstalled cmdline-tools, so it logs "Wrong version in preinstalled
    sdkmanager"
    and re-downloads cmdline-tools with no checksum.
  • It then runs its default packages: tools platform-tools install — the "SDK Tools"
    corrupt-zip surface (and where the emulator/platform package installs happen).

Any of those downloads can be a corrupt/truncated zip, which a bare sdkmanager turns into
an un-retried exit 1.

Fix — verify → reject → retry (never trust sdkmanager's exit code alone)

New stdlib-only helper .github/scripts/setup_android_sdk.py (per the repo's "Python for
CI helpers" rule), wired into every SDK-setup job:

  • bootstrap — download the pinned cmdline-tools zip, verify size + SHA-256
    (04453066…af7, an authoritative pin computed locally from bytes that matched Google's
    published SHA-1 48833c34… + size 172789259), and install it to
    $ANDROID_SDK_ROOT/cmdline-tools/20.0 — the exact path setup-android probes first, so
    the action reuses the verified tree and never does its own unverified "Wrong version"
    re-download
    . A mismatch (corrupt or wrong version) → delete the bad zip + any
    half-extracted dir → re-download clean.
  • install — sdkmanager --install with retry + backoff; on a corrupt package zip it
    purges the partial/corrupt package dir (and sdkmanager's temp dirs) before retrying,
    forcing a fresh re-download instead of a re-read. Package purge path = id.replace(';','/').
  • setup-android now runs with packages: "" (no flaky tools platform-tools
    install) and cmdline-tools-version: "14742923" (reuse the verified bootstrap → the
    "Wrong version" path never fires).
  • Cache via actions/cache/restore + a success-gated actions/cache/save, so only a
    verified SDK is ever cached (integrity gates the cache — a corrupt/failed SDK is
    never saved) and the re-download/corruption surface shrinks.

Integrity (checksum) + reject-and-retry, per the maintainer note

  1. SHA verified — cmdline-tools checked against a pinned SHA-256 after download; a
    mismatch = corrupt or wrong version, both caught here. (hashlib.sha256 is the
    portable superset of sha256sum/shasum -a 256.)
  2. Reject → retry — the corrupt/partial download (bad zip + half-extracted dir) is
    deleted before re-fetching, so the retry re-downloads rather than re-reading the bad zip.
  3. Integrity gates the cache — save is if: success(), so a corrupt/unverified SDK is
    never baked into the cache.
  4. For SDK packages (no pinnable published hash), a failed/corrupt extraction purges that
    package's dir before the retry so it re-downloads fresh.

Scope

Applied to debug-build, unit-tests, static-analysis, e2e (matrix), e2e-preview. Emulator
BOOT logic, #372 API-37 sharding, and #388 diagnostics are untouched. New cache
sub-actions reuse the file's already-pinned actions/cache SHA (55cc834…); no new external
action was introduced (Python replaces a nick-fields/retry dependency).

Validation

  • actionlint clean; YAML parses; both scripts py_compile.
  • Pure-logic helpers unit-tested in test_setup_android_sdk.py (package→purge-path
    mapping, size/SHA-256 gate, backoff, sdkmanager discovery, exec-bit-preserving extraction),
    auto-run by the existing traffic-control-tests job. 74 tests OK.
  • Real validation is this PR's own CI run — SDK setup must go green across the matrix.

Relates to #387/#388 (diagnostics — this is the "make it stop" complement).

🤖 Generated with Claude Code

## Problem (#389) The dominant merge-blocking flake is the **Set up Android SDK** step (`android-actions/setup-android`) dying **before** the emulator ever starts: ``` Wrong version in preinstalled sdkmanager Warning: ... preparing SDK package Android Emulator: Error reading Zip content from a SeekableByteChannel. Error: The process '.../sdkmanager' failed with exit code 1 ``` Seen on E2E (31), E2E (35) (PR #375) and a #388 preview shard (the "SDK Tools" package). ## Root cause `setup-android` **v4.0.1 is already the latest release** — there is no newer version to bump to, so this is a *configuration + hardening* fix. Reading the action source: - Its default `cmdline-tools-version` (build `14742923` = rev **20.0**) rarely matches the runner image's preinstalled cmdline-tools, so it logs **"Wrong version in preinstalled sdkmanager"** and re-downloads cmdline-tools **with no checksum**. - It then runs its default `packages: tools platform-tools` install — the **"SDK Tools"** corrupt-zip surface (and where the emulator/platform package installs happen). Any of those downloads can be a corrupt/truncated zip, which a bare `sdkmanager` turns into an un-retried `exit 1`. ## Fix — verify → reject → retry (never trust sdkmanager's exit code alone) New stdlib-only helper **`.github/scripts/setup_android_sdk.py`** (per the repo's "Python for CI helpers" rule), wired into every SDK-setup job: - **`bootstrap`** — download the *pinned* cmdline-tools zip, verify **size + SHA-256** (`04453066…af7`, an authoritative pin computed locally from bytes that matched Google's published SHA-1 `48833c34…` + size `172789259`), and install it to `$ANDROID_SDK_ROOT/cmdline-tools/20.0` — the **exact path setup-android probes first**, so the action reuses the verified tree and **never does its own unverified "Wrong version" re-download**. A mismatch (corrupt **or** wrong version) → delete the bad zip + any half-extracted dir → re-download clean. - **`install`** — `sdkmanager --install` with retry + backoff; on a corrupt package zip it **purges the partial/corrupt package dir** (and sdkmanager's temp dirs) before retrying, forcing a fresh re-download instead of a re-read. Package purge path = `id.replace(';','/')`. - **`setup-android`** now runs with **`packages: ""`** (no flaky `tools platform-tools` install) and **`cmdline-tools-version: "14742923"`** (reuse the verified bootstrap → the "Wrong version" path never fires). - **Cache** via `actions/cache/restore` + a **success-gated** `actions/cache/save`, so only a **verified** SDK is ever cached (**integrity gates the cache** — a corrupt/failed SDK is never saved) and the re-download/corruption surface shrinks. ### Integrity (checksum) + reject-and-retry, per the maintainer note 1. **SHA verified** — cmdline-tools checked against a pinned SHA-256 **after download**; a mismatch = corrupt **or** wrong version, both caught here. (`hashlib.sha256` is the portable superset of `sha256sum`/`shasum -a 256`.) 2. **Reject → retry** — the corrupt/partial download (bad zip + half-extracted dir) is deleted *before* re-fetching, so the retry re-downloads rather than re-reading the bad zip. 3. **Integrity gates the cache** — save is `if: success()`, so a corrupt/unverified SDK is never baked into the cache. 4. For SDK **packages** (no pinnable published hash), a failed/corrupt extraction purges that package's dir before the retry so it re-downloads fresh. ## Scope Applied to **debug-build, unit-tests, static-analysis, e2e (matrix), e2e-preview**. Emulator **BOOT logic**, #372 API-37 sharding, and #388 diagnostics are **untouched**. New cache sub-actions reuse the file's already-pinned `actions/cache` SHA (`55cc834…`); no new external action was introduced (Python replaces a `nick-fields/retry` dependency). ## Validation - `actionlint` clean; YAML parses; both scripts `py_compile`. - Pure-logic helpers unit-tested in **`test_setup_android_sdk.py`** (package→purge-path mapping, size/SHA-256 gate, backoff, sdkmanager discovery, exec-bit-preserving extraction), auto-run by the existing `traffic-control-tests` job. `74 tests OK`. - Real validation is **this PR's own CI run** — SDK setup must go green across the matrix. Relates to #387/#388 (diagnostics — this is the "make it stop" complement). 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign in to join this conversation.