Say in the run-shape table when the wedge timeout was what killed the leg #121

Merged
JMR-dev merged 3 commits from ci/wedged-leg-report into main 2026-08-26 04:11:39 +00:00
JMR-dev commented 2026-08-26 03:11:38 +00:00 (Migrated from github.com)

Closes #118.

The run-shape report from #111 runs on every path out of e2e-run.sh, the wedge
included, and there it answered a question it had not been asked. On job
98035980326 — E2E API 34, a docs-only PR — it printed this six seconds before
the wedge warning, for a leg WEDGE_TIMEOUT had killed 22 minutes in:

  expected:          59
  received:          59
  failed:            unknown
  completed cleanly: yes

completed cleanly means "instrumentation was not aborted", which was true. The
leg still died, and a reader scanning the table had to notice a separate
##[warning] line to find that out.

How the signal reaches the report

It is passed in, not sniffed: E2E_WEDGED_AFTER, set by e2e-run.sh at the
report's call site. A wedge is gradle never returning, so gradle printed no
verdict, no truncation line and no INSTRUMENTATION_ABORTED — the log of a
wedged leg is the log of a run that simply stops, and guessing from that would
call every cancelled run a wedge. Only e2e-run.sh saw timeout exit 124.

That fact is now derived once, and capture_wedge is driven from the same
variable, so the report and the diagnostics cannot disagree about what a wedge is.

What the table says now

  expected:          59
  received:          59
  failed:            unknown
  wedged:            yes -- gradle was killed after 1200s and never returned
  completed cleanly: no
  • failed: unknown is unchanged. Gradle printed no summary line, so the count
    genuinely is not knowable; that field is doing its job.
  • completed cleanly flips only where it would have said yes. An abort already
    says no and names the abort — which the wedge row does not — and a run that
    left no evidence still says unknown. A wedge on top of either prints both.
  • received's source told the same lie in the same table: "the run was not
    truncated, so every expected test reported" is only "gradle never got as far as
    saying so" when the leg was killed. Qualified on that path; the number is
    untouched.

Nothing here decides anything. No exit status, no pass/fail rule, no baseline
comparison, no ::notice:: behaviour. The leg already failed correctly.

Verification

Against captured CI output rather than a live emulator — how #111 was verified,
and for the same reason: this host cannot run API 37 and cannot wedge on demand.
Four real logs, both script versions (origin/main's and this branch's, run from
the same directory so the repo root resolves identically), both env states,
comparing stdout and the job summary:

log source
wedged API 34 job 98035980326 (#118's own evidence)
green API 34 job 98041335284
failing gating leg job 98034629977 (exit 1)
advisory API 37 + baseline job 98041335154, deviation and ::notice:: and all
  • The wedge, with the signal set → the table names it (above). Revert —
    the same fixture through origin/main's script, same env → completed cleanly: yes and no mention of the wedge.
  • Byte-identical elsewhere: green, failing and advisory logs, with the env
    var unset and set-empty (the state this script now passes on every non-wedge
    path), stdout and summary — 14 of 14 identical to main.
  • The advisory leg wedged: the only difference from main is the added
    wedged: row. The baseline DEVIATION line and the ::notice:: are
    byte-identical, and completed cleanly: no keeps the abort as its source.
  • End to end: the real e2e-run.sh driven with a stub timeout that replays
    the captured wedged log and exits 124 → the wedge row appears and the leg still
    exits 124 with its ##[warning]. Exits 0 and 1 through the same stub produce no
    wedge row and the same statuses as before.
  • Both mutations were checked for bite: dropping E2E_WEDGED_AFTER from the call
    site puts completed cleanly: yes back on the wedged run, and a stray echo in
    each emit block turns the byte-identity diffs red (stdout and summary
    separately — the first stray only reddened stdout, which is why both were run).

Pinned shellcheck and actionlint are clean; the gradle gate is unaffected — the
diff is two shell scripts and CLAUDE.md.

Closes #118. The run-shape report from #111 runs on every path out of `e2e-run.sh`, the wedge included, and there it answered a question it had not been asked. On job `98035980326` — `E2E API 34`, a docs-only PR — it printed this six seconds before the wedge warning, for a leg `WEDGE_TIMEOUT` had killed 22 minutes in: ``` expected: 59 received: 59 failed: unknown completed cleanly: yes ``` `completed cleanly` means "instrumentation was not aborted", which was true. The leg still died, and a reader scanning the table had to notice a separate `##[warning]` line to find that out. ## How the signal reaches the report It is passed in, not sniffed: `E2E_WEDGED_AFTER`, set by `e2e-run.sh` at the report's call site. A wedge is gradle never returning, so gradle printed no verdict, no truncation line and no `INSTRUMENTATION_ABORTED` — the log of a wedged leg is the log of a run that simply stops, and guessing from that would call every cancelled run a wedge. Only `e2e-run.sh` saw `timeout` exit 124. That fact is now derived once, and `capture_wedge` is driven from the same variable, so the report and the diagnostics cannot disagree about what a wedge is. ## What the table says now ``` expected: 59 received: 59 failed: unknown wedged: yes -- gradle was killed after 1200s and never returned completed cleanly: no ``` - `failed: unknown` is unchanged. Gradle printed no summary line, so the count genuinely is not knowable; that field is doing its job. - `completed cleanly` flips only where it would have said `yes`. An abort already says `no` and names the abort — which the wedge row does not — and a run that left no evidence still says `unknown`. A wedge on top of either prints both. - `received`'s **source** told the same lie in the same table: "the run was not truncated, so every expected test reported" is only "gradle never got as far as saying so" when the leg was killed. Qualified on that path; the number is untouched. **Nothing here decides anything.** No exit status, no pass/fail rule, no baseline comparison, no `::notice::` behaviour. The leg already failed correctly. ## Verification Against captured CI output rather than a live emulator — how #111 was verified, and for the same reason: this host cannot run API 37 and cannot wedge on demand. Four real logs, both script versions (`origin/main`'s and this branch's, run from the same directory so the repo root resolves identically), both env states, comparing stdout **and** the job summary: | log | source | | --- | --- | | wedged API 34 | job `98035980326` (#118's own evidence) | | green API 34 | job `98041335284` | | failing gating leg | job `98034629977` (exit 1) | | advisory API 37 + baseline | job `98041335154`, deviation and `::notice::` and all | - **The wedge, with the signal set** → the table names it (above). **Revert** — the same fixture through `origin/main`'s script, same env → `completed cleanly: yes` and no mention of the wedge. - **Byte-identical elsewhere**: green, failing and advisory logs, with the env var unset *and* set-empty (the state this script now passes on every non-wedge path), stdout and summary — 14 of 14 identical to `main`. - **The advisory leg wedged**: the only difference from `main` is the added `wedged:` row. The `baseline DEVIATION` line and the `::notice::` are byte-identical, and `completed cleanly: no` keeps the abort as its source. - **End to end**: the real `e2e-run.sh` driven with a stub `timeout` that replays the captured wedged log and exits 124 → the wedge row appears and the leg still exits 124 with its `##[warning]`. Exits 0 and 1 through the same stub produce no wedge row and the same statuses as before. - Both mutations were checked for bite: dropping `E2E_WEDGED_AFTER` from the call site puts `completed cleanly: yes` back on the wedged run, and a stray `echo` in each emit block turns the byte-identity diffs red (stdout and summary separately — the first stray only reddened stdout, which is why both were run). Pinned shellcheck and actionlint are clean; the gradle gate is unaffected — the diff is two shell scripts and CLAUDE.md.
Sign in to join this conversation.