Harden virtualized window against stale scroll; guard crash class

A defensive review (prompted by an external File Pilot NULL-write crash in
its per-directory stats roll-up) checked whether Librarian shares that bug
class: a derived index guarded by a stale count/offset rather than the
destination's bounds.

Found and fixed one latent instance: `window_for` clamped `end` to `count`
but not `start`, so a list that shrank under a stale scroll offset could
yield an inverted `start..end` that violates the documented `0..count`
contract and panics any caller that slices `rows[start..end]` (the grid
does). Clamp `start` to `end` so the window is always well-formed.

- Add `rapid_churn_never_leaves_a_stale_row_index`: drives a real
  create/delete burst (the `.lock` trigger) through the notify watcher and
  asserts reconciliation (`retain_below` + the window clamp) never leaves an
  out-of-bounds row index.
- Extend `visible_window_*` with a stale-scroll-past-end contract case.
- Document the analysis in docs/crash_class_review_filepilot.md.

Full librarian-app suite green (51); fmt + clippy clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-17 11:51:45 -05:00
co-authored by Claude Opus 4.8
parent de6499e09f
commit afd23ed2c1
2 changed files with 307 additions and 14 deletions
+175
View File
@@ -0,0 +1,175 @@
# Crash-class review: is Librarian prone to the File Pilot NULL-write bug?
**Date:** 2026-06-17
**Branch:** `feat-phase2`
**Scope:** A defensive review prompted by an *external* crash. A minidump from
File Pilot (Voidstar's file explorer, written in C) was analyzed separately; this
doc records whether Librarian — a different file explorer, in Rust — is exposed to
the **same class** of bug, what makes it (mostly) immune, and the one latent
instance the review found and fixed.
This was a **crash-class** review, not a general bug hunt. The conclusion: Librarian
is structurally resistant to that class, the worst realistic outcome is a bounded
panic rather than memory corruption, and one latent footgun in the exact class was
hardened (`window_for`) with a regression test added.
---
## The File Pilot bug (the thing we're checking for)
File Pilot crashed with an access violation — a **write through a NULL pointer** —
in its per-directory **statistics roll-up**:
- It maintains a long-lived per-name aggregate keyed by an **open-addressing hash
map**.
- On a **lookup miss** it sets the destination "aggregate record" pointer to NULL
as a sentinel.
- The accumulation loop that follows is guarded **only by a cached child-item
count** (`count == 32`), *not* by whether the destination is valid, so it runs
and executes `add qword ptr [rsi+30h], rax` with `rsi = 0` → write to `0x30` →
`0xC0000005`.
**Trigger:** a tab open on a directory containing `.claude.json.lock`, a lock file
an external tool creates and deletes very rapidly. The churn drove a stats update
for an entry whose name was momentarily absent from the aggregation map → miss →
NULL destination → unguarded count-driven loop → NULL write.
### The bug class, abstracted
Two ingredients are required:
1. **A long-lived aggregate that is mutated incrementally** as the directory
changes (rather than recomputed from scratch).
2. **A loop/index guarded by a separately-tracked count or stale offset**, instead
of by the validity/bounds of the destination it actually writes to.
When the two drift apart — count says "32 items", destination says "absent" — you
get a write to an invalid location.
---
## Why Librarian is structurally resistant
**1. It recomputes from scratch; it never mutates a persistent aggregate.**
On any external change the watcher emits `Message::DirChanged`, which re-enumerates
the whole directory off-thread (`crates/librarian-app/src/main.rs`, `DirChanged`
handler ~L1211); `Message::Reloaded` then rebuilds *all* derived state via
`recompute_rows()` (~L2003). There is no per-directory stats structure being
incrementally patched as files churn, so there is no aggregate to fall out of sync.
This is the single biggest reason ingredient (1) is absent.
**2. Reconciliation is unconditional.** `recompute_rows()` always calls
`self.selection.retain_below(self.rows.len())` (~L2034), which drops any
selection / lead / anchor index that no longer exists
(`selection.rs::retain_below`). The count and the data are rebuilt together from
the same source in the same step.
**3. Indexing is guarded by the destination, not a stale count.** Every raw index
into `self.rows` is clamped or guarded against the *current* length:
- list view `&self.rows[index]` — `index ∈ start..end` from `visible_window`,
whose `end` is `.min(count)`;
- grid thumbs `self.rows[start..end]` — both ends `.min(self.rows.len())`;
- grid tiles `&self.rows[i]` — explicit `if i >= total { break; }` first.
Everywhere else lookups use `self.rows.get(i)` → `Option`, and the one `HashMap`
lookup that matters degrades safely on miss:
`self.col_store.get(path).copied().unwrap_or_default()`.
**4. Language backstop.** Even if a stale index slipped through, safe Rust turns it
into a **bounded `index out of bounds` panic**, not a wild write into a struct
field. The data/enumeration/stats paths contain no `unsafe`, no raw-pointer
arithmetic (`model.rs` documents "no `unsafe`"). The C failure mode — `add [rsi+30h]`
with `rsi = 0` — is simply not expressible there. The `windows`-facing crate has
`unsafe`, but it is not on the watcher/enumeration path.
**5. The trigger is blunted, too.** The watcher is **debounced at 200 ms and
non-recursive** (`watch_stream`, ~L3566); each refresh supersedes the previous via
a load token (~L1223). A `.claude.json.lock`-style storm collapses into a single
re-enumeration, and a file that vanishes mid-scan is skipped, not fatal
(`enumerate.rs` — a failed `DirEntry::metadata()` → `continue`).
---
## The closest analog in Librarian
The Rust-shaped version of "count drifted from destination" is a **stale `usize`
index into `self.rows` that survives a listing shrink**: selection / lead / scroll
/ render-window indices computed against a *larger* prior listing, used after an
external change shrank it. Reasons (1)–(3) above keep all of these reconciled, so
there is no live panic on the current call sites.
### Latent instance found and fixed: `window_for`
While building the regression test, an extreme-stale-offset assertion failed and
surfaced a real latent footgun. `window_for` (which backs `visible_window`)
clamped `end` to `count` but **not** `start`:
```rust
// before
let start = first.saturating_sub(overscan);
let end = first.saturating_add(onscreen).saturating_add(overscan).min(count);
```
When the list shrinks under a **stale scroll offset** (an external change deletes
rows while you're scrolled down), `first` can sit past the new end, leaving `start`
above `count`. That:
- **violates the function's own documented `0..count` contract**, and
- produces an **inverted `start..end`** that panics the instant a caller *slices*
`rows[start..end]` — which the grid view does.
It was not a live panic *today*: the list view *iterates* `start..end` (harmlessly
empty when inverted), and the grid computes its own separately-clamped range. But
it is exactly the "derived index not reconciled to the current count" shape we were
reviewing for — a panic waiting for the next caller that slices. Fixed by clamping
`start` to `end`:
```rust
// after
let end = first.saturating_add(onscreen).saturating_add(overscan).min(count);
let start = first.saturating_sub(overscan).min(end); // start <= end <= count
```
Now the window is always a well-formed, in-bounds, possibly-empty range.
---
## Regression guard added
`crates/librarian-app/src/main.rs`, test module:
- **`rapid_churn_never_leaves_a_stale_row_index`** — arms the *real* notify watcher
(non-recursive, the same settings `watch_stream` uses) on a temp dir, hammers a
`.churn.lock` create/delete burst from a background thread (the exact File Pilot
trigger), and on every re-enumeration runs the app's reconciliation
(`Selection::retain_below` + the `visible_window`/`window_for` clamp), carrying
selection + scroll from a *larger* prior listing into the new one. It asserts no
produced index ever falls outside the current rows and nothing panics, however
the count moves between reads. It also exercises `read_dir_all` tolerating an
entry vanishing mid-scan.
- **`visible_window_renders_only_the_viewport_plus_overscan`** — extended with a
pure-arithmetic case proving the `0..count` contract holds for a stale scroll
offset past the end of a shrunk list (`start <= end <= count`, never inverted).
Both pass; full `librarian-app` suite green (51 tests), `cargo fmt --all --check`
and `cargo clippy -p librarian-app --all-targets` clean.
---
## Verdict
Librarian is **not susceptible** to File Pilot's bug class. It rebuilds derived
state wholesale instead of mutating a long-lived aggregate, reconciles indices
unconditionally, guards every raw index against the current length, and runs in
safe Rust where the worst outcome is a panic, not memory corruption. The one latent
footgun in the class (`window_for`'s un-clamped `start`) has been fixed and pinned
with tests.
### Future-risk watch
The immunity comes from the *recompute-from-scratch* design. The moment a feature
mutates a **persistent per-directory aggregate incrementally from watcher events** —
e.g. live folder-size totals, per-extension counts, or a running selection summary —
ingredient (1) reappears and this bug class is back in play. If that is ever added:
guard the accumulation on the **destination's existence/bounds**, not on a cached
item count, and re-run the churn test against it.