Make script/test fail on flaky failures and enable -race (closes #32) #53

Merged
clawbot merged 1 commits from fix-script-test-race into next 2026-10-07 03:22:35 +02:00
Collaborator

Closes #32.

script/test ended with go test -count=1 ./... || go test -count=1 -v ./.... The verbose rerun was the last command, so its exit status became the script's: a test that failed once and passed on the retry gave a green make test, and so a green script/cibuild.

It now follows the pattern in REPO_POLICIES.md:

go test -count=1 -timeout 30s -race -cover ./... || \
    { echo "--- Rerunning with -v for details ---"; \
      go test -count=1 -timeout 30s -race -v ./...; exit 1; }

The exit 1 means the rerun only prints details and can never turn a failed run green. go vet ./... still runs first; nothing else in the script changes. -count=1 stays on both lines because the Dockerfile keeps Go's build cache between builds (#125).

With a 30-second timeout per package, the tests that gave the secret binary a minute could never reach their own limit: a hang ended in Go's timeout panic, without the test's message and without killing the binary. They now give it 10 seconds (commandWait in internal/cli/integration_test.go), and the PGP unlocker test's 30-second timer is 10 seconds. That test builds only on macOS, so it was not run here.

The README.md Entrypoints line for script/test describes the new run.

Timing: with -race, the make test step of script/cibuild took 17.2 s with Go's build cache warm and 32.1 s on the first build, which compiles the race-instrumented packages from scratch: inside the 60 s cap, over the 20 s target (sneak/prompts#41 (comment)).

Model: opus-5-5

Closes https://git.eeqj.de/sneak/secret/issues/32. `script/test` ended with `go test -count=1 ./... || go test -count=1 -v ./...`. The verbose rerun was the last command, so its exit status became the script's: a test that failed once and passed on the retry gave a green `make test`, and so a green `script/cibuild`. It now follows the pattern in `REPO_POLICIES.md`: ```sh go test -count=1 -timeout 30s -race -cover ./... || \ { echo "--- Rerunning with -v for details ---"; \ go test -count=1 -timeout 30s -race -v ./...; exit 1; } ``` The `exit 1` means the rerun only prints details and can never turn a failed run green. `go vet ./...` still runs first; nothing else in the script changes. `-count=1` stays on both lines because the `Dockerfile` keeps Go's build cache between builds (https://git.eeqj.de/sneak/secret/pulls/125). With a 30-second timeout per package, the tests that gave the `secret` binary a minute could never reach their own limit: a hang ended in Go's timeout panic, without the test's message and without killing the binary. They now give it 10 seconds (`commandWait` in `internal/cli/integration_test.go`), and the PGP unlocker test's 30-second timer is 10 seconds. That test builds only on macOS, so it was not run here. The `README.md` Entrypoints line for `script/test` describes the new run. Timing: with `-race`, the `make test` step of `script/cibuild` took 17.2 s with Go's build cache warm and 32.1 s on the first build, which compiles the race-instrumented packages from scratch: inside the 60 s cap, over the 20 s target (https://git.eeqj.de/sneak/prompts/issues/41#issuecomment-53166). Model: opus-5-5
clawbot added the needs-checks label 2026-08-09 07:06:26 +02:00
clawbot self-assigned this 2026-08-09 07:06:26 +02:00
clawbot added this to the 1.0.0 milestone 2026-08-09 07:06:26 +02:00
clawbot changed target branch from main to next 2026-08-10 16:13:08 +02:00
clawbot removed their assignment 2026-09-03 15:41:48 +02:00
sneak was assigned by clawbot 2026-09-03 15:41:48 +02:00
sneak was unassigned by clawbot 2026-09-25 11:09:59 +02:00
clawbot self-assigned this 2026-09-25 11:09:59 +02:00
clawbot force-pushed fix-script-test-race from 3d615ef612 to 3aa5f1de6c 2026-10-07 01:16:55 +02:00 Compare
clawbot added needs-review and removed needs-checks labels 2026-10-07 01:30:56 +02:00
Author
Collaborator

Reworked:

  • Rebased onto current next as one new commit replacing the August one. -count=1 stays on both go test lines.
  • The TODO.md entry is rewritten to describe the script as it is now.
  • The PR body is rewritten. It no longer depends on #52, and it gives the -race timing.

Model: opus-5-5

Reworked: - Rebased onto current `next` as one new commit replacing the August one. `-count=1` stays on both `go test` lines. - The `TODO.md` entry is rewritten to describe the script as it is now. - The PR body is rewritten. It no longer depends on https://git.eeqj.de/sneak/secret/issues/52, and it gives the `-race` timing. Model: opus-5-5
Author
Collaborator

FAIL: needs rework.

  1. internal/cli/integration_test.go lines 2586, 2617 and 2657, internal/cli/entry_test.go:55, internal/secret/pgpunlock_test.go:320: these tests give a hung command one minute (the last one 30 seconds). With -timeout 30s, Go stops each package's tests 30 seconds after they start, so none of these limits can end first. A hang now ends in Go's timeout panic instead of the test's own message ("did not exit within a minute", "still open a minute after secret rm started"), the child secret process is no longer killed, and the comment above the two terminal tests (from line 2600), that a terminal left open fails the test at the deadline instead of hanging it, is no longer true. That deadline is what #126 just added. Acceptable: limits well under 30 seconds (such as the 10 seconds the lock tests wait), with the messages and the comment matching, mentioned in the TODO.md entry.
  2. README.md:607: the Entrypoints line for script/test still describes the old run. Acceptable: it says the tests run with the race detector, a 30-second timeout per package and coverage, and that the script fails even when the verbose rerun passes.

Model: opus-5-5

**FAIL: needs rework.** 1. `internal/cli/integration_test.go` lines 2586, 2617 and 2657, `internal/cli/entry_test.go:55`, `internal/secret/pgpunlock_test.go:320`: these tests give a hung command one minute (the last one 30 seconds). With `-timeout 30s`, Go stops each package's tests 30 seconds after they start, so none of these limits can end first. A hang now ends in Go's timeout panic instead of the test's own message ("did not exit within a minute", "still open a minute after secret rm started"), the child `secret` process is no longer killed, and the comment above the two terminal tests (from line 2600), that a terminal left open fails the test at the deadline instead of hanging it, is no longer true. That deadline is what https://git.eeqj.de/sneak/secret/issues/126 just added. Acceptable: limits well under 30 seconds (such as the 10 seconds the lock tests wait), with the messages and the comment matching, mentioned in the `TODO.md` entry. 2. `README.md:607`: the Entrypoints line for `script/test` still describes the old run. Acceptable: it says the tests run with the race detector, a 30-second timeout per package and coverage, and that the script fails even when the verbose rerun passes. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-07 02:07:00 +02:00
clawbot added 1 commit 2026-10-07 02:28:34 +02:00
script/test ended with a verbose rerun whose exit status became the
script's, so a test that failed once and passed on the retry gave a
green build. It now follows the REPO_POLICIES.md pattern: go vet, then
go test -count=1 -timeout 30s -race -cover; on failure a verbose rerun
for the details, then exit 1. -count=1 stays on both go test lines
because the Dockerfile keeps Go's build cache between builds.

The tests that gave the secret binary a minute, and the PGP unlocker
test's 30-second timer, now use 10 seconds, so a hang fails with the
test's own message before the package's 30-second timeout. The README
describes the new run.

Model: opus-5-5
clawbot force-pushed fix-script-test-race from 3aa5f1de6c to 585a7c1993 2026-10-07 02:28:34 +02:00 Compare
clawbot added needs-review and removed needs-rework labels 2026-10-07 02:41:50 +02:00
Author
Collaborator

Reworked:

  1. The limits are now 10 seconds (commandWait in internal/cli/integration_test.go, and the PGP unlocker test's timer), with the failure messages, the comment above the terminal tests and the TODO.md entry to match. A hang planted in TestRemoveIgnoresTerminalOnStdout failed after 10 seconds with the test's own message.
  2. Rewritten as asked.

The PGP unlocker test builds only on macOS and was not run here. The PR body is updated to match.

Model: opus-5-5

Reworked: 1. The limits are now 10 seconds (`commandWait` in `internal/cli/integration_test.go`, and the PGP unlocker test's timer), with the failure messages, the comment above the terminal tests and the `TODO.md` entry to match. A hang planted in `TestRemoveIgnoresTerminalOnStdout` failed after 10 seconds with the test's own message. 2. Rewritten as asked. The PGP unlocker test builds only on macOS and was not run here. The PR body is updated to match. Model: opus-5-5
Author
Collaborator

PASS: script/test now fails whenever the first run fails, runs with the race detector and the 30-second package timeout, and every test's own limit for a hung command ends well before that timeout, as #32 and the previous review asked.

Model: opus-5-5

**PASS:** `script/test` now fails whenever the first run fails, runs with the race detector and the 30-second package timeout, and every test's own limit for a hung command ends well before that timeout, as https://git.eeqj.de/sneak/secret/issues/32 and the previous review asked. Model: opus-5-5
clawbot merged commit ed6af50ea5 into next 2026-10-07 03:22:35 +02:00
clawbot deleted branch fix-script-test-race 2026-10-07 03:22:35 +02:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/secret#53