ci(e2e): restore matrix-leg wedge-capture without hanging reactivecircus (watchdog or manual-boot) #421

Closed
opened 2026-07-07 17:04:34 +00:00 by JMR-dev · 1 comment
JMR-dev commented 2026-07-07 17:04:34 +00:00 (Migrated from github.com)

Context

#404/#406 delivered E2E wedge diagnostics in scoped form: the matrix job now has a job-level timeout-minutes: 50 (bounds any wedge to 50 min vs the 6h default) and the API-37 preview leg keeps the full capture_wedge smoking-gun dump. The matrix legs (API 29-36) do NOT capture in-flight diagnostics.

Why

The original #406 approach — wrapping the matrix gradle step in timeout -k 30s 1200 inside reactivecircus/android-emulator-runner — reproducibly hung all 8 matrix legs (confirmed twice; the manual-boot preview with the same wrapper passed). Maintainer chose the safe revert (matrix back to main's working script + keep the timeout backstop + preview capture). See #406 for the full analysis.

This follow-up (deferred, if matrix-leg wedge-capture is wanted)

Restore in-flight wedge-capture on the matrix legs WITHOUT the reactivecircus-hanging timeout wrapper. Two candidate approaches to try in order (each with their own sub-issues):

  1. Background watchdog
  2. Manual-boot convergence
    Relates #404, #406, the ci-e2e-infra-flakes hung-wedge note.
## Context #404/#406 delivered E2E wedge diagnostics in **scoped** form: the matrix job now has a **job-level `timeout-minutes: 50`** (bounds any wedge to 50 min vs the 6h default) and the **API-37 preview** leg keeps the full `capture_wedge` smoking-gun dump. The **matrix legs (API 29-36) do NOT capture** in-flight diagnostics. ## Why The original #406 approach — wrapping the matrix gradle step in `timeout -k 30s 1200` inside `reactivecircus/android-emulator-runner` — **reproducibly hung all 8 matrix legs** (confirmed twice; the manual-boot preview with the same wrapper passed). Maintainer chose the safe revert (matrix back to main's working script + keep the timeout backstop + preview capture). See #406 for the full analysis. ## This follow-up (deferred, if matrix-leg wedge-capture is wanted) Restore in-flight wedge-capture on the **matrix** legs WITHOUT the reactivecircus-hanging `timeout` wrapper. Two candidate approaches to try in order (each with their own sub-issues): 1. **Background watchdog** 2. **Manual-boot convergence** Relates #404, #406, the [[ci-e2e-infra-flakes]] hung-wedge note.
JMR-dev commented 2026-07-08 19:46:27 +00:00 (Migrated from github.com)

Unblocked + de-risked by #454. The matrix e2e legs now hand-provision the emulator like e2e-preview (manual-boot convergence, #432 closed). The old blocker — the timeout wrapper hanging reactivecircus — no longer applies, since reactivecircus's boot is gone. Remaining scope narrows to: add the capture_wedge diagnostics dump to the matrix legs, mirroring the API-37 preview leg (which already captures on the same manual-boot structure). The watchdog alternative (#431) is likely unnecessary now that manual-boot is in.

**Unblocked + de-risked by #454.** The matrix `e2e` legs now hand-provision the emulator like `e2e-preview` (manual-boot convergence, #432 closed). The old blocker — the `timeout` wrapper hanging reactivecircus — no longer applies, since reactivecircus's boot is gone. Remaining scope narrows to: add the `capture_wedge` diagnostics dump to the matrix legs, mirroring the API-37 preview leg (which already captures on the same manual-boot structure). The watchdog alternative (#431) is likely unnecessary now that manual-boot is in.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: JMR-dev/LibreMail#421