#10 Report lifecycle/status metadata (pending/removed/published) #35

Merged
JMR-dev merged 2 commits from ticket-10-lifecycle-metadata into main 2026-07-02 20:30:04 +00:00
JMR-dev commented 2026-07-02 20:26:28 +00:00 (Migrated from github.com)

What & why

Adds per-report lifecycle/status metadata — pending / removed / published — on top of the #9 ObjectStore, plus a query to list pending reports and transitions to removed/published. This unblocks the weekly job (#13), the maintainer removal path (#11), and the post-publish marking (#15).

Status model

Status is encoded in the object key, not inside the (encrypted) body. Every report lives at:

reports/<status>/<id>
  • New reports are born pending at ingest.
  • A report moves to exactly one terminal state: pending → removed (#11) or pending → published (#15).
  • <id> (<utc-ts>-<80-bit-rand>) is stable across transitions — only the status segment of the key changes.

No status lives in the ciphertext, so status can be read/changed without the decryption key.

Why list-pending is exact

Because status is the key prefix, "list pending" is a single List("reports/pending/") with no secondary index that could drift out of sync. The status prefixes are disjoint and trailing-slashed, so reports/pending/ can never match a reports/published/... or reports/removed/... key. Given a mix of reports in different states, ListPending returns exactly the pending ids by construction (verified by TestListPendingExactness).

Storage approach

  • Extended ObjectStore with List(ctx, prefix) ([]string, error) and Delete(ctx, key) error, implemented in both MemoryStore (host) and the js && wasm R2Store.
  • R2Store.List drives the R2 binding's list() directly with {prefix, cursor} and pages the whole result set (the syumai helper takes no options), so a status with >1000 objects is still enumerated exactly. Wasm impl stays behind the existing build tag; host correctness is covered by MemoryStore.
  • The #9 ingest Sink now writes new reports under reports/pending/<id> (default key func), so accepted reports enter the lifecycle as pending — wired in without changing the handler/ingest contract.

Transitions (ciphertext only, idempotent, retry-safe)

A transition copies the opaque AES-256-GCM frame to the destination status key and deletes the source key. The bytes are never decrypted, re-encrypted, or inspected.

Ordering is Put(dest) then Delete(src):

  • Never loses a report. If interrupted after the copy but before the delete, the object is briefly visible under both statuses; re-running converges (re-Put identical bytes, Delete the leftover source). Deleting an absent key is a no-op in every store, so retries don't spuriously fail.
  • A transition whose source is absent is idempotent success when the object is already at the destination, else ErrUnknownReport (id was never pending, or is in a different terminal state).

API (internal/lifecycle)

m := lifecycle.New(store)          // store: *storage.R2Store (prod) or *storage.MemoryStore (tests/devserver)
m.ListPending(ctx)      // []string ids, sorted oldest-first  — #13
m.GetPending(ctx, id)   // []byte ciphertext for the publish step — #14/#15
m.MarkRemoved(ctx, id)  // pending -> removed — #11
m.MarkPublished(ctx, id)// pending -> published — #15

Test evidence

Host tests only (go test ./..., no TinyGo); wasm-tagged files additionally type-checked with GOOS=js GOARCH=wasm go build.

$ go vet ./...
(no output)

$ go test ./...
?       github.com/JMR-dev/LibreMail-Bug-Report-Ingest/cmd/devserver    [no test files]
ok      github.com/JMR-dev/LibreMail-Bug-Report-Ingest/internal/crypto    0.643s
ok      github.com/JMR-dev/LibreMail-Bug-Report-Ingest/internal/handler   0.920s
ok      github.com/JMR-dev/LibreMail-Bug-Report-Ingest/internal/ingest    0.889s
ok      github.com/JMR-dev/LibreMail-Bug-Report-Ingest/internal/lifecycle 0.746s
ok      github.com/JMR-dev/LibreMail-Bug-Report-Ingest/internal/scrub     0.584s
ok      github.com/JMR-dev/LibreMail-Bug-Report-Ingest/internal/storage   0.690s

New lifecycle tests:

--- PASS: TestListPendingExactness
--- PASS: TestNewReportsAreListedAsPending
--- PASS: TestMarkRemoved
--- PASS: TestMarkPublished
--- PASS: TestTransitionIdempotentRetry
--- PASS: TestTransitionConvergesFromInterruptedState
--- PASS: TestUnknownIdErrors
--- PASS: TestGetPending

New storage tests: TestStatusKeyLayout, TestMemoryStoreListAndDelete (plus existing storage/ingest/handler tests still green). No new HTTP endpoint, so the api-tests/ Bruno suite and devserver are unchanged.

Closes #10

## What & why Adds per-report lifecycle/status metadata — **`pending` / `removed` / `published`** — on top of the `#9` `ObjectStore`, plus a query to list pending reports and transitions to removed/published. This unblocks the weekly job (#13), the maintainer removal path (#11), and the post-publish marking (#15). ## Status model Status is encoded in the **object key**, not inside the (encrypted) body. Every report lives at: ``` reports/<status>/<id> ``` - New reports are born **`pending`** at ingest. - A report moves to exactly one terminal state: `pending → removed` (#11) or `pending → published` (#15). - `<id>` (`<utc-ts>-<80-bit-rand>`) is **stable across transitions** — only the status segment of the key changes. No status lives in the ciphertext, so status can be read/changed **without the decryption key**. ## Why list-pending is exact Because status *is* the key prefix, "list pending" is a single `List("reports/pending/")` with **no secondary index that could drift out of sync**. The status prefixes are disjoint and trailing-slashed, so `reports/pending/` can never match a `reports/published/...` or `reports/removed/...` key. Given a mix of reports in different states, `ListPending` returns exactly the pending ids by construction (verified by `TestListPendingExactness`). ## Storage approach - Extended `ObjectStore` with `List(ctx, prefix) ([]string, error)` and `Delete(ctx, key) error`, implemented in **both** `MemoryStore` (host) and the `js && wasm` `R2Store`. - `R2Store.List` drives the R2 binding's `list()` directly with `{prefix, cursor}` and **pages the whole result set** (the syumai helper takes no options), so a status with >1000 objects is still enumerated exactly. Wasm impl stays behind the existing build tag; host correctness is covered by `MemoryStore`. - The `#9` ingest `Sink` now writes new reports under `reports/pending/<id>` (default key func), so **accepted reports enter the lifecycle as `pending`** — wired in without changing the handler/ingest contract. ## Transitions (ciphertext only, idempotent, retry-safe) A transition **copies the opaque AES-256-GCM frame** to the destination status key and **deletes the source key**. The bytes are never decrypted, re-encrypted, or inspected. Ordering is `Put(dest)` then `Delete(src)`: - Never loses a report. If interrupted after the copy but before the delete, the object is briefly visible under both statuses; re-running converges (re-`Put` identical bytes, `Delete` the leftover source). Deleting an absent key is a no-op in every store, so retries don't spuriously fail. - A transition whose source is absent is **idempotent success** when the object is already at the destination, else **`ErrUnknownReport`** (id was never pending, or is in a different terminal state). ## API (`internal/lifecycle`) ```go m := lifecycle.New(store) // store: *storage.R2Store (prod) or *storage.MemoryStore (tests/devserver) m.ListPending(ctx) // []string ids, sorted oldest-first — #13 m.GetPending(ctx, id) // []byte ciphertext for the publish step — #14/#15 m.MarkRemoved(ctx, id) // pending -> removed — #11 m.MarkPublished(ctx, id)// pending -> published — #15 ``` ## Test evidence Host tests only (`go test ./...`, no TinyGo); wasm-tagged files additionally type-checked with `GOOS=js GOARCH=wasm go build`. ``` $ go vet ./... (no output) $ go test ./... ? github.com/JMR-dev/LibreMail-Bug-Report-Ingest/cmd/devserver [no test files] ok github.com/JMR-dev/LibreMail-Bug-Report-Ingest/internal/crypto 0.643s ok github.com/JMR-dev/LibreMail-Bug-Report-Ingest/internal/handler 0.920s ok github.com/JMR-dev/LibreMail-Bug-Report-Ingest/internal/ingest 0.889s ok github.com/JMR-dev/LibreMail-Bug-Report-Ingest/internal/lifecycle 0.746s ok github.com/JMR-dev/LibreMail-Bug-Report-Ingest/internal/scrub 0.584s ok github.com/JMR-dev/LibreMail-Bug-Report-Ingest/internal/storage 0.690s ``` New `lifecycle` tests: ``` --- PASS: TestListPendingExactness --- PASS: TestNewReportsAreListedAsPending --- PASS: TestMarkRemoved --- PASS: TestMarkPublished --- PASS: TestTransitionIdempotentRetry --- PASS: TestTransitionConvergesFromInterruptedState --- PASS: TestUnknownIdErrors --- PASS: TestGetPending ``` New `storage` tests: `TestStatusKeyLayout`, `TestMemoryStoreListAndDelete` (plus existing storage/ingest/handler tests still green). No new HTTP endpoint, so the `api-tests/` Bruno suite and devserver are unchanged. Closes #10
Sign in to join this conversation.