Make script/test fail on flaky failures and enable -race (closes #32)

script/test ended with a verbose rerun whose exit status became the
script's, so a test that failed once and passed on the retry gave a
green build. It now follows the REPO_POLICIES.md pattern: go vet, then
go test -count=1 -timeout 30s -race -cover; on failure a verbose rerun
for the details, then exit 1. -count=1 stays on both go test lines
because the Dockerfile keeps Go's build cache between builds.

The tests that gave the secret binary a minute, and the PGP unlocker
test's 30-second timer, now use 10 seconds, so a hang fails with the
test's own message before the package's 30-second timeout. The README
describes the new run.

Model: opus-5-5

Dockerfile

@@ -50,7 +50,12 @@ ARG CHECK_EPOCH
 
 COPY . .
 
-RUN make test
+# This cache mount keeps Go's build cache between builds for make test and
+# make build; -count=1 in script/test keeps test results out of it. Go's cache
+# does not notice C header changes, so the mount has its own id: change the id
+# when the C packages installed above change.
+RUN --mount=type=cache,id=sneak/secret/go-build,target=/root/.cache/go-build \
+    make test
 
 # The version stamped into the binary: the VERSION build argument when one
 # is given, otherwise `git describe --tags --always` of the .git the build
@@ -58,7 +63,8 @@ RUN make test
 # one, the short commit when no tag is reachable. A context that carries .git
 # and still yields no version fails the build.
 ARG VERSION
-RUN version="${VERSION:-$(git describe --tags --always)}"; \
+RUN --mount=type=cache,id=sneak/secret/go-build,target=/root/.cache/go-build \
+    version="${VERSION:-$(git describe --tags --always)}"; \
     if [ -e .git ] && { [ -z "$version" ] || [ "$version" = dev ] || \
         [ "$version" = unknown ]; }; then \
         echo "no version could be derived although the build context carries .git" >&2; \

README.md

@@ -604,7 +604,10 @@ provide:
 - `script/build` — build the `secret` binary into the repo root, stamping the
   version (`VERSION` from the environment, else `git describe`) and the git
   commit
-- `script/test` — run `go vet` and the test suite (verbose rerun on failure)
+- `script/test` — run `go vet`, then the test suite with the race detector, a
+  30-second timeout per package and coverage, every test on every run, never a
+  result from Go's test cache; on failure it reruns the tests verbosely for the
+  details and fails even when the rerun passes
 - `script/lint` — run `golangci-lint` in docker only: builds `Dockerfile.lint`,
   where the linter is a build step that runs on every call, also on an unchanged
   tree
@@ -624,7 +627,8 @@ provide:
 - `script/cibuild` — CI entrypoint: `docker build --ulimit memlock=-1:-1 .`
   (memguard needs mlock; the Dockerfile runs the checks), with a new
   `CHECK_EPOCH` build argument on every run so the checks run again on an
-  unchanged tree
+  unchanged tree; the `Dockerfile` keeps Go's build cache between builds, so
+  `make test` and `make build` compile only what changed
 - `script/precommit` — pre-commit checks: `go mod tidy` verification, then
   `script/check`
 - `script/install-precommit` — install the git pre-commit hook that runs

TODO.md

@@ -18,6 +18,53 @@ https://git.eeqj.de/sneak/secret/milestone/12
 
 # Completed Steps
 
+- 2026-10-06: `script/test` runs the tests with the race detector, a 30-second
+  timeout per package and coverage, as `REPO_POLICIES.md` requires
+  (https://git.eeqj.de/sneak/secret/issues/32). When they fail, it reruns them
+  verbosely for the details and then fails anyway, so a test that fails once and
+  passes on the retry no longer gives a green build. `go vet` still runs first,
+  and every `go test` keeps `-count=1`. The tests that gave the `secret` binary
+  a minute now give it 10 seconds, and the PGP unlocker test's 30-second timer
+  is 10 seconds, so a test that hangs fails with its own message before the
+  package's 30-second timeout ends every test in it.
+- 2026-10-06: `make test` in `script/cibuild` no longer compiles the standard
+  library and every dependency from nothing on every build
+  (https://git.eeqj.de/sneak/secret/issues/124). The `Dockerfile` runs it and
+  `make build` with Go's build cache in a BuildKit cache mount, which docker
+  keeps between builds, so each compiles only what changed since the last build.
+  The mount has an id of its own, so other repositories' builds do not share it.
+  `script/test` passes `-count=1`, so every test runs on every build and no
+  result comes from Go's test cache. A build with an empty cache, such as the
+  first after docker's build cache is cleared, compiles everything in
+  `make test` as before.
+- 2026-10-06: `TestRemoveIgnoresTerminalOnStdout` and
+  `TestRemoveAsksAtTerminalOnStdin` no longer wait until Go's test timeout
+  (https://git.eeqj.de/sneak/secret/issues/126). On Linux, `pty.Open` of
+  `github.com/creack/pty` v1.1.24 passed the address of a local variable to the
+  `ioctl` system call as a plain number, through a function call; when Go moved
+  the goroutine's stack in between, the kernel wrote the terminal's number to
+  the old place, and `pty.Open` opened `/dev/pts/0` instead of the terminal it
+  had created. `secret rm` then wrote to that other terminal, and the test read
+  a terminal no program had open, which never ends. `go.mod` now requires the
+  commit on that library's main branch that passes a pointer instead; no release
+  has it yet. Both tests stop reading the terminal when their one-minute context
+  ends and fail saying so.
+- 2026-10-06: The tests run quickly with the race detector on
+  (https://git.eeqj.de/sneak/secret/issues/120). Most of their time went to
+  deriving keys from passphrases with scrypt, which is slow on purpose. The new
+  `secret.ScryptWorkFactor`, when not zero, replaces age's scrypt work factor
+  when a passphrase encrypts; the tests of `internal/secret`, `internal/vault`
+  and `internal/cli` set it to 1 before any test runs, and the program never
+  sets it; `TestGetCommandOutputsToStdout` checks that the passphrase unlocker
+  the built binary's `secret init` writes names age's work factor, 18.
+  `TestRemovalAsksWithoutHoldingLock` and `TestFailedCommandReleasesLock` no
+  longer run in parallel with other tests: each waits at most 10 seconds for the
+  in-memory lock that every test in the package shares, and other tests'
+  commands held it longer. `TestConcurrentAddsKeepEveryVersion`, which times
+  nothing, and `TestGetCommandOutputsToStdout`, which no longer sets an
+  environment variable its commands do not read, now run in parallel. The
+  `script/cibuild` comment no longer says that tests are skipped without its
+  memlock ulimit.
 - 2026-10-05: No test stores a secret larger than 1 MiB
   (https://git.eeqj.de/sneak/secret/issues/52). The size tests for `secret add`,
   `secret import` and the stdin buffer no longer try 2 MB, 10 MB, 99 MB, 100 MB

go.mod

@@ -9,7 +9,7 @@ require (
 	github.com/btcsuite/btcd/btcec/v2 v2.1.3
 	github.com/btcsuite/btcd/btcutil v1.1.6
 	github.com/btcsuite/btcutil v0.0.0-20190425235716-9e5f4b9a998d
-	github.com/creack/pty v1.1.24
+	github.com/creack/pty v1.1.25-0.20260601142114-9246436fffe8 // v1.1.24's Open can return another pty's terminal
 	github.com/dustin/go-humanize v1.0.1
 	github.com/fatih/color v1.18.0
 	github.com/keybase/go-keychain v0.0.0-20230307172405-3e4884637dd1

go.sum

@@ -35,8 +35,8 @@ github.com/btcsuite/snappy-go v1.0.0/go.mod h1:8woku9dyThutzjeg+3xrA5iCpBRH8XEEg
 github.com/btcsuite/websocket v0.0.0-20150119174127-31079b680792/go.mod h1:ghJtEyQwv5/p4Mg4C0fgbePVuGr935/5ddU9Z3TmDRY=
 github.com/btcsuite/winsvc v1.0.0/go.mod h1:jsenWakMcC0zFBFurPLEAyrnc/teJEM1O46fmI40EZs=
 github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
-github.com/creack/pty v1.1.24 h1:bJrF4RRfyJnbTJqzRLHzcGaZK1NeM5kTC9jGgovnR1s=
+github.com/creack/pty v1.1.25-0.20260601142114-9246436fffe8 h1:CY3gjC7naqYGLMiywvj3suPfa1i0p/QEr7o8ujxL/2M=
-github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE=
+github.com/creack/pty v1.1.25-0.20260601142114-9246436fffe8/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE=
 github.com/davecgh/go-spew v0.0.0-20171005155431-ecdeabc65495/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=

internal/cli/confirm_test.go

@@ -353,9 +353,9 @@ func TestRemovalWithoutTerminalFailsAtOnce(t *testing.T) {
 // for its answer, another command can take the state directory lock and
 // change the secret, and that the removal then removes nothing, since the
 // secret is no longer what the question named.
+//
+//nolint:paralleltest // times commands against the in-memory lock all tests share
 func TestRemovalAsksWithoutHoldingLock(t *testing.T) {
-	t.Parallel()
-
 	r := newRemoval(t, "rm")
 
 	answers, answerWriter := io.Pipe()

internal/cli/entry_test.go

@@ -8,7 +8,6 @@ import (
 	"path/filepath"
 	"strings"
 	"testing"
-	"time"
 
 	"github.com/awnumar/memguard"
 	"github.com/stretchr/testify/assert"
@@ -52,7 +51,7 @@ func TestInterruptExitsThroughMemguard(t *testing.T) {
 
 	const waitingForValue = "Reading secret value from stdin"
 
-	ctx, cancel := context.WithTimeout(t.Context(), time.Minute)
+	ctx, cancel := context.WithTimeout(t.Context(), commandWait)
 	defer cancel()
 
 	wd, err := filepath.Abs("../..")

internal/cli/integration_test.go

@@ -32,6 +32,12 @@ const (
 	// testMnemonic is a standard BIP39 mnemonic used for testing
 	//nolint:dupword // BIP39 test mnemonic intentionally repeats a word
 	testMnemonic = "abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about"
+
+	// commandWait is how long a test lets the secret binary run before it
+	// kills it and fails. It stays well under the 30 seconds script/test
+	// gives the whole package, so a command that hangs fails the test with
+	// the test's own message instead of Go's timeout panic.
+	commandWait = 10 * time.Second
 )
 
 // errEmptyValue indicates a concurrent reader received an empty secret value.
@@ -52,8 +58,12 @@ func runSecretWithStdin(stdin string, env map[string]string, args ...string) (st
 	return cli.ExecuteCommandInProcess(args, stdin, env)
 }
 
-// TestMain runs before all tests and ensures the binary is built
+// TestMain runs before all tests and ensures the binary is built. It also
+// makes passphrase encryption in the tests cheap (see
+// secret.ScryptWorkFactor); the binary keeps age's work factor.
 func TestMain(m *testing.M) {
+	secret.ScryptWorkFactor = 1
+
 	// Get the current working directory
 	wd, err := os.Getwd()
 	if err != nil {
@@ -2579,7 +2589,7 @@ func TestRemoveWithoutTerminalFailsAtOnce(t *testing.T) {
 		_ = stdin.Close()
 	}()
 
-	ctx, cancel := context.WithTimeout(t.Context(), time.Minute)
+	ctx, cancel := context.WithTimeout(t.Context(), commandWait)
 	defer cancel()
 
 	cmd, secretDir := secretRmCommand(ctx, t)
@@ -2597,7 +2607,13 @@ func TestRemoveWithoutTerminalFailsAtOnce(t *testing.T) {
 // and stderr, not both: whether it asks must depend on stdin alone, where
 // the answer is read from. pty.Open returns the two ends of a new terminal:
 // tty is the end a program uses as its terminal, and ptmx the end the test
-// reads what the terminal shows from and types into.
+// reads what the terminal shows from and types into. Reading ptmx stops at
+// the context's deadline, commandWait (10 seconds) after the test starts,
+// when secret rm is killed too, so a terminal that stays open fails the test
+// then with its own message instead of hanging it. The deadline works only
+// while ptmx stays non-blocking, as pty.Open of the github.com/creack/pty
+// commit in go.mod leaves it: calling ptmx.Fd() or going back to v1.1.24
+// makes the read ignore the deadline, without any error.
 
 // TestRemoveIgnoresTerminalOnStdout runs `echo y | secret rm x` at a
 // terminal. stdin is a pipe, so nobody can answer there, and the command
@@ -2605,7 +2621,7 @@ func TestRemoveWithoutTerminalFailsAtOnce(t *testing.T) {
 func TestRemoveIgnoresTerminalOnStdout(t *testing.T) {
 	t.Parallel()
 
-	ctx, cancel := context.WithTimeout(t.Context(), time.Minute)
+	ctx, cancel := context.WithTimeout(t.Context(), commandWait)
 	defer cancel()
 
 	cmd, secretDir := secretRmCommand(ctx, t)
@@ -2615,6 +2631,9 @@ func TestRemoveIgnoresTerminalOnStdout(t *testing.T) {
 
 	defer func() { _ = ptmx.Close() }()
 
+	deadline, _ := ctx.Deadline()
+	require.NoError(t, ptmx.SetReadDeadline(deadline))
+
 	cmd.Stdin = strings.NewReader("y\n")
 	cmd.Stdout = tty
 	cmd.Stderr = tty
@@ -2624,9 +2643,16 @@ func TestRemoveIgnoresTerminalOnStdout(t *testing.T) {
 	_ = tty.Close()
 
 	// The read ends once secret rm has exited and so closed the terminal.
-	shown, _ := io.ReadAll(ptmx)
+	shown, err := io.ReadAll(ptmx)
+	require.NotErrorIs(t, err, os.ErrDeadlineExceeded,
+		"the terminal was still open %s after secret rm started: %s",
+		commandWait, shown)
 
-	require.Error(t, cmd.Wait())
+	err = cmd.Wait()
+
+	require.NoError(t, ctx.Err(), "secret rm did not exit within %s",
+		commandWait)
+	require.Error(t, err)
 	assert.Contains(t, string(shown), "pass --force")
 	assert.DirExists(t, secretDir)
 }
@@ -2636,7 +2662,7 @@ func TestRemoveIgnoresTerminalOnStdout(t *testing.T) {
 func TestRemoveAsksAtTerminalOnStdin(t *testing.T) {
 	t.Parallel()
 
-	ctx, cancel := context.WithTimeout(t.Context(), time.Minute)
+	ctx, cancel := context.WithTimeout(t.Context(), commandWait)
 	defer cancel()
 
 	cmd, secretDir := secretRmCommand(ctx, t)
@@ -2646,6 +2672,9 @@ func TestRemoveAsksAtTerminalOnStdin(t *testing.T) {
 
 	defer func() { _ = ptmx.Close() }()
 
+	deadline, _ := ctx.Deadline()
+	require.NoError(t, ptmx.SetReadDeadline(deadline))
+
 	cmd.Stdin = tty
 	// Not a file, so exec.Cmd connects stdout through a pipe.
 	cmd.Stdout = io.Discard
@@ -2663,7 +2692,8 @@ func TestRemoveAsksAtTerminalOnStdin(t *testing.T) {
 	terminal := bufio.NewReader(ptmx)
 	for !bytes.HasSuffix(shown, []byte("[y/N] ")) {
 		char, err = terminal.ReadByte()
-		require.NoError(t, err, "secret rm ended without asking: %s", shown)
+		require.NoError(t, err, "secret rm did not ask on the terminal: %s",
+			shown)
 
 		shown = append(shown, char)
 	}

internal/cli/lock_test.go

@@ -94,9 +94,9 @@ func numbered(prefix string, count int) []string {
 // lock, adds of a new secret all find it absent and replace each other, and
 // forced adds read the same highest version number and overwrite each
 // other's version. With it they behave as if run one after another.
-//
-//nolint:paralleltest // times commands against the in-memory lock all tests share
 func TestConcurrentAddsKeepEveryVersion(t *testing.T) {
+	t.Parallel()
+
 	mnemonic := testMnemonicBuffer(t)
 
 	const adds = 8
@@ -110,6 +110,8 @@ func TestConcurrentAddsKeepEveryVersion(t *testing.T) {
 		{"real", afero.NewOsFs(), t.TempDir()},
 	} {
 		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+
 			_, err := vault.CreateVault(tc.fs, tc.stateDir, "default", mnemonic, nil)
 			require.NoError(t, err)
 
@@ -235,9 +237,9 @@ func TestEncryptPipedIntoAdd(t *testing.T) {
 
 // TestFailedCommandReleasesLock checks that a command failing after it
 // took the state directory lock leaves the lock free for the next command.
+//
+//nolint:paralleltest // times commands against the in-memory lock all tests share
 func TestFailedCommandReleasesLock(t *testing.T) {
-	t.Parallel()
-
 	fs := afero.NewMemMapFs()
 	cli := NewCLIInstanceWithStateDir(fs, testStateDir)
 

internal/cli/stdout_stderr_test.go

@@ -15,11 +15,11 @@ import (
 // TestGetCommandOutputsToStdout tests that 'secret get' outputs the secret
 // value to stdout, not stderr
 func TestGetCommandOutputsToStdout(t *testing.T) {
-	// Create a temporary directory for our vault
+	t.Parallel()
-	tempDir := t.TempDir()
 
-	// Set environment variables for the test
+	// Create a temporary directory for our vault; each command is given it
-	t.Setenv(secret.EnvStateDir, tempDir)
+	// in its environment
+	tempDir := t.TempDir()
 
 	// Find the secret binary path
 	wd, err := filepath.Abs("../..")
@@ -41,6 +41,18 @@ func TestGetCommandOutputsToStdout(t *testing.T) {
 	output, err := cmd.CombinedOutput()
 	require.NoError(t, err, "init should succeed: %s", string(output))
 
+	// The binary, unlike these tests, encrypts the passphrase unlocker's key
+	// at age's scrypt work factor, 18. age writes the work factor last on the
+	// second line of priv.age: "-> scrypt <salt> <work factor>".
+	vaultDir := filepath.Join(tempDir, "vaults.d", "default")
+	unlockerName := readFile(t, filepath.Join(vaultDir, "current-unlocker"))
+	unlockerDir := filepath.Join(vaultDir, "unlockers.d", string(unlockerName))
+	privAge := readFile(t, filepath.Join(unlockerDir, "priv.age"))
+	header := strings.SplitN(string(privAge), "\n", 3)
+	require.Len(t, header, 3, "priv.age should start with an age header")
+	assert.Regexp(t, `^-> scrypt \S+ 18$`, header[1],
+		"the passphrase unlocker should be encrypted at scrypt work factor 18")
+
 	// Add a secret
 	//nolint:gosec // G204: test executes the freshly built secret binary
 	cmd = exec.CommandContext(t.Context(), secretPath, "add", "test/secret")

internal/secret/crypto.go

@@ -28,6 +28,17 @@ var (
 // terminal to read the mnemonic from, reading it failed, or it was empty.
 var ErrMnemonicNotRead = errors.New("failed to read mnemonic")
 
+// ScryptWorkFactor is, when not zero, the scrypt work factor that
+// EncryptWithPassphrase uses instead of age's, 18: log2 of scrypt's cost
+// parameter N. Deriving a key with age's takes about a second and 256 MiB, on
+// purpose, since so does every guess at the passphrase. Only tests set it,
+// lower, before any test runs, so that the passphrase unlockers they create
+// cost nothing; the program leaves it zero. Decryption takes the work factor
+// from the encrypted data, so it needs no setting.
+//
+//nolint:gochecknoglobals // set by the tests of the packages that use this one
+var ScryptWorkFactor int
+
 // EncryptToRecipient encrypts data to a recipient using age
 // The data parameter should be a LockedBuffer for secure memory handling
 func EncryptToRecipient(
@@ -142,6 +153,10 @@ func EncryptWithPassphrase(
 		return nil, fmt.Errorf("failed to create scrypt recipient: %w", err)
 	}
 
+	if ScryptWorkFactor != 0 {
+		recipient.SetWorkFactor(ScryptWorkFactor)
+	}
+
 	return EncryptToRecipient(data, recipient)
 }
 

internal/secret/pgpunlock_test.go

@@ -317,8 +317,8 @@ func testCreatePGPUnlocker(
 	t.Helper()
 
 	// Set a limited test timeout to avoid hanging
-	timer := time.AfterFunc(30*time.Second, func() {
+	timer := time.AfterFunc(10*time.Second, func() {
-		t.Fatalf("Test timed out after 30 seconds")
+		t.Fatalf("Test timed out after 10 seconds")
 	})
 	defer timer.Stop()
 

internal/secret/secret_test.go

@@ -25,6 +25,14 @@ var (
 	errNotImplementedInMock = errors.New("not implemented in mock")
 )
 
+// TestMain makes passphrase encryption in the tests cheap; see
+// ScryptWorkFactor.
+func TestMain(m *testing.M) {
+	ScryptWorkFactor = 1
+
+	os.Exit(m.Run())
+}
+
 // MockVault is a test implementation of the VaultInterface
 type MockVault struct {
 	name            string

internal/vault/vault_test.go

@@ -3,6 +3,7 @@ package vault_test
 import (
 	"bytes"
 	"errors"
+	"os"
 	"path/filepath"
 	"slices"
 	"testing"
@@ -28,6 +29,14 @@ const (
 	testPassphrase = "test-passphrase"
 )
 
+// TestMain makes passphrase encryption in the tests cheap; see
+// secret.ScryptWorkFactor.
+func TestMain(m *testing.M) {
+	secret.ScryptWorkFactor = 1
+
+	os.Exit(m.Run())
+}
+
 // testMnemonicBuffer returns testMnemonic in a locked buffer that is
 // destroyed when the test ends.
 func testMnemonicBuffer(t *testing.T) *memguard.LockedBuffer {

script/cibuild

@@ -1,12 +1,13 @@
 #!/bin/sh
 # script/cibuild: run the CI build. The Dockerfile runs script/check
 # (via make check), so a successful build implies all checks pass.
-# The Gitea workflow runs this on push. The memlock ulimit lets the tests
+# The Gitea workflow runs this on push. The memlock ulimit lifts the limit
-# that lock large secrets in memory (memguard mlocks them) run; under the
+# on memory the tests lock (memguard mlocks secrets); they also pass under
-# lower limit of a plain `docker build .` they are skipped.
+# the lower limit of a plain `docker build .`.
 # A cached build checks nothing: a new CHECK_EPOCH on every run makes the
 # Dockerfile's check steps run again on an unchanged tree, while its base
-# images and module downloads stay cached.
+# images, module downloads and the Go build cache that make test and make
+# build use stay cached.
 set -eu
 
 ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"

script/test

@@ -1,5 +1,6 @@
 #!/bin/sh
-# script/test: run the test suite (vet first, verbose rerun on failure).
+# script/test: run the test suite (vet first, then the tests with the race
+# detector; a verbose rerun on failure).
 set -eu
 
 ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
@@ -9,7 +10,13 @@ main() {
     # CGO is required (Makefile exports this too)
     export CGO_ENABLED=1
     go vet ./...
-    go test ./... || go test -v ./...
+    # -count=1: run every test, never take a result from Go's test cache,
+    # which the Dockerfile keeps between builds. The rerun only prints
+    # details: `exit 1` keeps the script failing even if a flaky test
+    # passes on the second attempt.
+    go test -count=1 -timeout 30s -race -cover ./... || \
+        { echo "--- Rerunning with -v for details ---"; \
+          go test -count=1 -timeout 30s -race -v ./...; exit 1; }
 }
 
 main "$@"