From 3d615ef6125d8fc709e21b23d903363a3bad4fa3 Mon Sep 17 00:00:00 2001 From: sneak Date: Sun, 9 Aug 2026 05:03:54 +0000 Subject: [PATCH] Make script/test fail on flaky failures and enable -race (closes #32) script/test ended with `go test ./... || go test -v ./...`, so the verbose rerun's exit status became the script's. A test that failed once and passed on the retry produced exit 0, and since the Dockerfile runs `make test` and CI runs script/cibuild, flaky failures were invisible repo-wide. Replace that with the pattern from REPO_POLICIES.md: run `go test -timeout 30s -race -cover ./...`, and on failure rerun verbosely and then exit 1, so the rerun is diagnostic only and can never turn a failed run green. `go vet ./...` still runs first; the script stays POSIX sh with set -eu, the CGO_ENABLED=1 export, and the repo-root cd idiom. Enabling -race surfaced a genuine timing conflict rather than a data race: no package that ran to completion reported one, but internal/cli now exceeds the 30s timeout because the 99MB/100MB boundary cases in secrets_size_test.go are about 10x slower under race instrumentation. Measured in the CI-equivalent container (script/cibuild, with the memlock ulimit): 18.7s for the make test layer without -race, versus internal/cli alone needing roughly 55-60s with it. Neither the flags nor the tests were weakened to hide this; the conflict is filed as #52 for a decision. --- TODO.md | 9 +++++++++ script/test | 6 +++++- 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/TODO.md b/TODO.md index 909a91e..d1ff5c7 100644 --- a/TODO.md +++ b/TODO.md @@ -25,6 +25,15 @@ Bring the repo into policy compliance in one commit: # Completed Steps +- 2026-08-09: `script/test` now runs + `go test -timeout 30s -race -cover ./...` and, on failure, reruns + verbosely and then exits non-zero, so a test that fails once and + passes on the retry can no longer produce a green build (closes #32). + Enabling `-race` showed the suite cannot meet the 20-second policy + budget: `internal/cli` exceeds the 30-second timeout because the + 99MB/100MB boundary cases in `secrets_size_test.go` run roughly 10x + slower under race instrumentation. No flags were weakened and no + tests skipped; the conflict is tracked in #52 for a decision. - 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints, Makefile shims, README Entrypoints section - 2026-03-11: Secure Enclave unlocker for hardware-backed secret diff --git a/script/test b/script/test index 3735b43..e1e400d 100755 --- a/script/test +++ b/script/test @@ -9,7 +9,11 @@ main() { # CGO is required (Makefile exports this too) export CGO_ENABLED=1 go vet ./... - go test ./... || go test -v ./... + # The rerun is diagnostic only: `exit 1` keeps the script failing + # even if a flaky test passes on the second attempt. + go test -timeout 30s -race -cover ./... || \ + { echo "--- Rerunning with -v for details ---"; \ + go test -timeout 30s -race -v ./...; exit 1; } } main "$@" -- 2.49.1