Flaky test: internal/crypto TestSealOpenRoundtrip 1-byte case (~11% spurious CI failures) #41

Closed
opened 2026-07-02 21:28:50 +00:00 by JMR-dev · 0 comments
JMR-dev commented 2026-07-02 21:28:50 +00:00 (Migrated from github.com)

Context

internal/crypto TestSealOpenRoundtrip (merged in #9) asserts, for a 1-byte plaintext, that the plaintext byte is absent from the ~36-byte encrypted frame. With a random key/nonce the ciphertext byte can equal the plaintext byte purely by chance, so the assertion fails ~11% of the time.

Impact: this test is on main, so ~11% of every ci run (on every PR) can fail spuriously — causing random auto-merge failures and wasted re-runs across the whole pipeline. Surfaced during #14 development.

Fix

  • Make the assertion deterministic. Keep the round-trip (Open(Seal(x)) == x) and the "ciphertext differs from plaintext" intent, but stop asserting byte-absence on a 1-byte input. E.g. assert the ciphertext as a whole differs from the plaintext, and/or use a multi-byte/longer plaintext for any differs-from-plaintext check, and/or pin a deterministic key+nonce for a known-answer.
  • Touch only internal/crypto/crypto_test.go (or its helpers). No production-code change.

Acceptance criteria

  • The test is deterministic (0% spurious failure) and go test ./internal/crypto/... passes reliably across many reruns.
## Context `internal/crypto` `TestSealOpenRoundtrip` (merged in #9) asserts, for a **1-byte** plaintext, that the plaintext byte is absent from the ~36-byte encrypted frame. With a random key/nonce the ciphertext byte can equal the plaintext byte purely by chance, so the assertion fails **~11% of the time**. **Impact:** this test is on `main`, so ~11% of **every** `ci` run (on every PR) can fail spuriously — causing random auto-merge failures and wasted re-runs across the whole pipeline. Surfaced during #14 development. ## Fix - [ ] Make the assertion deterministic. Keep the round-trip (`Open(Seal(x)) == x`) and the "ciphertext differs from plaintext" intent, but stop asserting byte-absence on a 1-byte input. E.g. assert the ciphertext **as a whole** differs from the plaintext, and/or use a multi-byte/longer plaintext for any differs-from-plaintext check, and/or pin a deterministic key+nonce for a known-answer. - [ ] Touch only `internal/crypto/crypto_test.go` (or its helpers). No production-code change. ## Acceptance criteria - The test is deterministic (0% spurious failure) and `go test ./internal/crypto/...` passes reliably across many reruns.
Sign in to join this conversation.