A wedge already reports itself correctly, and then the leg spends half an hour
not saying so. From #151, run 33071570911:
12:31:29 > Task :app:connectedDebugAndroidTest
12:48:44 wedged: yes -- gradle was killed after 1200s and never returned
12:49:20 ::warning::E2E api37 WEDGED (api37)
... 36 minutes of nothing ...
13:25:05 ##[error]The operation was canceled. <- timeout-minutes: 60
13:25:09 Terminate orphan process: qemu-system-x86_64-headless, adb, java, java
WEDGE_TIMEOUT=1200 fired exactly as designed. What overran was everything after
it: `capture_wedge` and `dump_diagnostics` probe the device with adb, and the
device they are probing has just finished proving it stopped answering.
WHY `|| true` DID NOT COVER THIS. Every probe carries one, and the file header
says why:
# Every probe is guarded with `|| true`. A diagnostic must never be the thing
# that turns a run red.
`|| true` guards a probe that exits non-zero. It does nothing about one that
never exits at all. The discipline was enforced for exit codes and not for time,
and this is the other half of it.
WHAT IT COSTS BEYOND THE 36 MINUTES. The job ends `cancelled` rather than failing
with the wedge's own status, so `gh pr checks` renders a failure with no cause and
the carefully-built `wedged:` row sits above half an hour of silence -- the one
mechanism built to explain a wedge is the least likely to be read. `kill
"$LOGCAT_PID"` never runs either, which is the orphaned adb and qemu above. And a
cancelled required check blocks merges: this one stopped a seven-PR stack.
THE FIX. An `adbq` wrapper -- `timeout -k 5s 20 adb "$@"` -- applied to all 18
probes in `capture_wedge`, `dump_diagnostics`, and the pre-run memory snapshot.
20s is far more than any of them needs on a healthy device and far less than any
costs on a dead one; `-k` because adb itself can ignore the first signal when its
server is wedged.
WHAT IS DELIBERATELY LEFT UNBOUNDED, and it is not everything else by accident:
- The API 37 SystemUI disable machinery (`adb shell stop`/`start`, the
`service check` loop). Functional, not diagnostic, and it already carries its
own verify-and-retry -- see the comment block above it.
- The backgrounded `adb logcat -v time` stream. It is meant to run for the whole
leg; bounding it would truncate the log at 20 seconds.
Verified rather than assumed. Wrapper semantics, smoke-tested against a fake adb:
passthrough works, a non-zero exit is preserved (rc=7), a hang is killed at the
deadline (rc=124 after 2s), and `|| true` still swallows that -- so a bounded
probe cannot turn a run red either, which is the property the header demands.
shellcheck clean at the pinned digest over `git ls-files '*.sh'`, actionlint clean
at its pinned digest, `bash -n` clean.
One thing this does NOT do: it does not stop the wedge. #122 is still open for
that. It stops a wedge from being reported as a cancellation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
JMR-devandClaude Opus 5
cae050fcfb