ci(e2e): restore wedge-capture diagnostics on the matrix E2E legs (#421) #464

Merged
JMR-dev merged 1 commits from ci-421-matrix-wedge-capture into main 2026-07-08 22:50:02 +00:00
JMR-dev commented 2026-07-08 22:05:34 +00:00 (Migrated from github.com)

Closes #421.

Restores the in-flight wedge-capture diagnostics on the matrix e2e legs (API 29-36), mirroring the API-37 e2e-preview job. Unblocked + de-risked by #454, which replaced reactivecircus/android-emulator-runner on the matrix with the same hand-provisioned manual boot e2e-preview uses — so the capture_wedge mechanism that already runs on that manual-boot structure in e2e-preview (without hanging) can now be brought to the matrix.

What was mirrored from e2e-preview

Everything is copied identical in shape from e2e-preview (no new wrapper invented), adapted only for per-API-level naming (api${{ matrix.api-level }} instead of api37-...-shard${{ matrix.shard }}):

  1. WEDGE_LOG + WEDGE_TIMEOUT=1200 variables in the "Boot emulator and run E2E" step.
  2. capture_wedge() — byte-for-byte the same probe sequence as preview: running/last test (TestRunner logcat), sys.boot_completed, init.svc.*, service list + service check input/window/activity (the binder-publish crux), app/test pids, SIGQUIT (kill -3) thread dumps of the app + instrumentation processes, /data/anr/*, dumpsys activity, dumpsys window, logcat -d tail, and accel/kvm/mem/disk.
  3. run_e2e() wrapped in timeout -k 30s "$WEDGE_TIMEOUT" — mirrors preview's run_shard(): on exit 124 it calls capture_wedge then returns 124 so the retry/gate still see a failure. Retry-once parity preserved (attempt 1 / attempt 2 (retry)).
  4. "Upload wedge diagnostics" step — if: always(), if-no-files-found: ignore (healthy runs produce no file and stay quiet), per-API-level artifact name wedge-diagnostics-api${{ matrix.api-level }}.

The stale comment (NO #404 wedge-capture wrapper here ...) above the boot step was corrected to explain the restore.

Why it cannot re-hang the legs (the structural reason)

The #404/#406 revert (90dfb18) happened because that wrapper wrapped reactivecircus/android-emulator-runner's emulator boot — killing that via timeout orphaned the emulator and hung all 8 legs. Here the timeout wraps only the foreground ./gradlew connectedDebugAndroidTest client, never the backgrounded emulator (which stays alive so capture_wedge can query it via adb). This is exactly e2e-preview's proven run_shard structure. WEDGE_TIMEOUT=1200 (20 min) reuses preview's value: healthy matrix legs run the whole job in ~8.3-12.0 min (per docs/perf/api37-e2e-sharding-spike.md §4; API 30 is the ~12.0 min ceiling) and the wrapper covers only the gradle invocation (a subset of that), so 20 min is generous no-false-trip headroom and sits well under this job's 50-min cap — which itself backstops any pathological hang.

Self-validating: this PR's own CI run exercises the wrapper on all 8 matrix legs. A false trip or a hang would fail the run (won't merge, won't reach the queue).

Scope guarantee

  • #454's boot logic is untouched — boot_emulator(), adb start-server, the 2-attempt cold-boot loop, timeout 300 adb wait-for-device, the non-fatal keyevent 82 unlock, and the animation settings are all unchanged (diff hunks land only in the comment, the variable block, and after the boot loop).
  • #460's path-filter block (the changes job) is not touched at all.

Verification

  • python -c "yaml.safe_load(...)" → parses cleanly.
  • actionlint .github/workflows/ci.yml → exit 0.
  • Reviewed the full diff (+94/-6, confined to the e2e matrix job).
Closes #421. Restores the in-flight **wedge-capture diagnostics** on the matrix `e2e` legs (API 29-36), mirroring the API-37 `e2e-preview` job. Unblocked + de-risked by #454, which replaced `reactivecircus/android-emulator-runner` on the matrix with the same hand-provisioned manual boot `e2e-preview` uses — so the `capture_wedge` mechanism that already runs on that manual-boot structure in `e2e-preview` (without hanging) can now be brought to the matrix. ## What was mirrored from `e2e-preview` Everything is copied **identical in shape** from `e2e-preview` (no new wrapper invented), adapted only for per-API-level naming (`api${{ matrix.api-level }}` instead of `api37-...-shard${{ matrix.shard }}`): 1. **`WEDGE_LOG` + `WEDGE_TIMEOUT=1200`** variables in the "Boot emulator and run E2E" step. 2. **`capture_wedge()`** — byte-for-byte the same probe sequence as preview: running/last test (TestRunner logcat), `sys.boot_completed`, `init.svc.*`, `service list` + `service check input/window/activity` (the binder-publish crux), app/test pids, **SIGQUIT (`kill -3`) thread dumps** of the app + instrumentation processes, `/data/anr/*`, `dumpsys activity`, `dumpsys window`, `logcat -d` tail, and accel/kvm/mem/disk. 3. **`run_e2e()` wrapped in `timeout -k 30s "$WEDGE_TIMEOUT"`** — mirrors preview's `run_shard()`: on exit 124 it calls `capture_wedge` then returns 124 so the retry/gate still see a failure. Retry-once parity preserved (`attempt 1` / `attempt 2 (retry)`). 4. **"Upload wedge diagnostics" step** — `if: always()`, `if-no-files-found: ignore` (healthy runs produce no file and stay quiet), per-API-level artifact name `wedge-diagnostics-api${{ matrix.api-level }}`. The stale comment (`NO #404 wedge-capture wrapper here ...`) above the boot step was corrected to explain the restore. ## Why it cannot re-hang the legs (the structural reason) The #404/#406 revert (90dfb18) happened because that wrapper wrapped `reactivecircus/android-emulator-runner`'s **emulator boot** — killing that via `timeout` orphaned the emulator and hung all 8 legs. Here the `timeout` wraps **only the foreground `./gradlew connectedDebugAndroidTest` client**, never the backgrounded emulator (which stays alive so `capture_wedge` can query it via adb). This is exactly `e2e-preview`'s proven `run_shard` structure. `WEDGE_TIMEOUT=1200` (20 min) reuses preview's value: healthy matrix legs run the whole job in ~8.3-12.0 min (per `docs/perf/api37-e2e-sharding-spike.md` §4; API 30 is the ~12.0 min ceiling) and the wrapper covers only the gradle invocation (a subset of that), so 20 min is generous no-false-trip headroom and sits well under this job's 50-min cap — which itself backstops any pathological hang. **Self-validating:** this PR's own CI run exercises the wrapper on all 8 matrix legs. A false trip or a hang would fail the run (won't merge, won't reach the queue). ## Scope guarantee - **#454's boot logic is untouched** — `boot_emulator()`, `adb start-server`, the 2-attempt cold-boot loop, `timeout 300 adb wait-for-device`, the non-fatal `keyevent 82` unlock, and the animation settings are all unchanged (diff hunks land only in the comment, the variable block, and *after* the boot loop). - **#460's path-filter block (the `changes` job) is not touched at all.** ## Verification - `python -c "yaml.safe_load(...)"` → parses cleanly. - `actionlint .github/workflows/ci.yml` → exit 0. - Reviewed the full diff (+94/-6, confined to the `e2e` matrix job).
mergify[bot] commented 2026-07-08 22:27:12 +00:00 (Migrated from github.com)

Merge Queue Status

  • ✅ Entered queue — 2026-07-08 22:27 UTC · Rule: default · triggered by merge protections
  • ✅ Checks skipped · PR is already up-to-date
  • ✅ Merged — 2026-07-08 22:50 UTC · at 9407b20914119d4724983e2fe65f11d892c4b7ae · merge

This pull request spent 22 minutes 52 seconds in the queue, including 1 second running CI.

Required conditions to merge
<!--- DO NOT EDIT -*- Mergify Payload -*- {"version": 1, "state": "merged", "queue_rule_name": "default", "queued_at": "2026-07-08T22:27:11.412661+00:00", "estimated_time_of_merge": null, "speculative_check_pr": null, "required_conditions": []} -*- Mergify Payload End -*- --> # Merge Queue Status - ✅ **Entered queue** — `2026-07-08 22:27 UTC` · Rule: `default` · triggered by merge protections - ✅ **Checks skipped** · PR is already up-to-date - ✅ **Merged** — `2026-07-08 22:50 UTC` · at `9407b20914119d4724983e2fe65f11d892c4b7ae` · merge This pull request spent **22 minutes 52 seconds** in the queue, including **1 second** running CI. <details> <summary>Required conditions to merge</summary> - `-conflict` - [X] #464 - `-draft` - [X] #464 - [X] `base = main` - [X] `check-success = CI passed` - `github-review-approved` [🛡 GitHub repository ruleset rule `main`] - [X] #464 - `label != broken` - [X] #464 - [X] any of [🛡 GitHub branch protection]: - [X] `check-success = Debug build` - [ ] `check-neutral = Debug build` - [ ] `check-skipped = Debug build` - [X] any of [🛡 GitHub branch protection]: - [X] `check-success = Unit tests` - [ ] `check-neutral = Unit tests` - [ ] `check-skipped = Unit tests` - [X] any of [🛡 GitHub branch protection]: - [X] `check-success = CI passed` - [ ] `check-neutral = CI passed` - [ ] `check-skipped = CI passed` - [X] any of [🛡 GitHub repository ruleset rule `main`]: - [X] `check-success = @github-actions/CI passed` - [ ] `check-neutral = @github-actions/CI passed` - [ ] `check-skipped = @github-actions/CI passed` </details>
Sign in to join this conversation.