Skip the verbose test rerun after a timeout (closes #61)
check / check (push) Canceled after 0s

A hung test used the 90-second backstop twice, because the verbose
rerun waited for the same hang. The Go recipe now looks in the first
run's output for what Go prints for a hang, and exits 1 with no rerun.
Any other failure still reruns with -v.

The Python make test example takes the same timeout wrapper as its
Dockerfile example. The 60 and 90 second rule and the 5-minute rule say
a hung package uses the backstop once.

Model: opus-5-5
This commit is contained in:
2026-11-03 20:37:22 +00:00
parent 430964ceac
commit 9272826fc7
2 changed files with 63 additions and 17 deletions
+9
View File
@@ -21,6 +21,15 @@ fmt-check, and commit.
# Completed Steps # Completed Steps
- 2026-11-03: A first test run that timed out now gets no verbose rerun
(https://git.eeqj.de/sneak/prompts/issues/61), on sneak's approval at
https://git.eeqj.de/sneak/prompts/issues/61#issuecomment-165195. The Go recipe
in `REPO_POLICIES.md` looks in the first run's output for
`panic: test timed out` or `*** Test killed`, and exits 1 when either is
there. The Python `make test` example takes the same `timeout 90 sh -c`
wrapper as its `Dockerfile` example. The 60 and 90 second rule and the
5-minute rule now say a hung package uses the backstop once. Consuming
repositories pick this up on their next re-vendor.
- 2026-10-27: Added `prompts/API_POLICIES.md` - 2026-10-27: Added `prompts/API_POLICIES.md`
(https://git.eeqj.de/sneak/prompts/issues/137). It holds sneak's rule from (https://git.eeqj.de/sneak/prompts/issues/137). It holds sneak's rule from
https://git.eeqj.de/sneak/routewatch/issues/199 that a value in an API https://git.eeqj.de/sneak/routewatch/issues/199 that a value in an API
+54 -17
View File
@@ -1,6 +1,6 @@
--- ---
title: Repository Policies title: Repository Policies
last_modified: 2026-10-27 last_modified: 2026-11-03
--- ---
This document covers repository structure, tooling, and workflow standards. Code This document covers repository structure, tooling, and workflow standards. Code
@@ -233,6 +233,7 @@ style conventions are in separate documents:
# Test phase. -race needs cgo and so a C compiler, which the Debian Go # Test phase. -race needs cgo and so a C compiler, which the Debian Go
# image ships and the alpine one does not. -count=1 because the mounted # image ships and the alpine one does not. -count=1 because the mounted
# cache also holds test results, which go test would otherwise replay. # cache also holds test results, which go test would otherwise replay.
# A first run that timed out gets no rerun: its output has the stacks.
# golang:1.x, YYYY-MM-DD # golang:1.x, YYYY-MM-DD
FROM golang@sha256:... AS test FROM golang@sha256:... AS test
WORKDIR /src WORKDIR /src
@@ -241,9 +242,16 @@ style conventions are in separate documents:
ARG CACHEBUST ARG CACHEBUST
COPY . . COPY . .
RUN --mount=type=cache,target=/root/.cache/go-build \ RUN --mount=type=cache,target=/root/.cache/go-build \
go test -count=1 -timeout 90s -race -cover ./... || \ output=$(go test -count=1 -timeout 90s -race -cover ./... 2>&1); \
{ echo "--- Rerunning with -v for details ---"; \ status=$?; \
go test -count=1 -timeout 90s -race -v ./...; exit 1; } printf '%s\n' "$output"; \
test "$status" -eq 0 && exit 0; \
case "$output" in \
*"panic: test timed out"*|*"*** Test killed"*) exit 1 ;; \
esac; \
echo "--- Rerunning with -v for details ---"; \
go test -count=1 -timeout 90s -race -v ./...; \
exit 1
# Build stage. Nothing is wanted from either phase above; the copies # Build stage. Nothing is wanted from either phase above; the copies
# are what make BuildKit build them first, so nothing after them can # are what make BuildKit build them first, so nothing after them can
@@ -413,7 +421,8 @@ style conventions are in separate documents:
`pipenv install --dev --deploy`: `pipenv install --dev --deploy`:
```dockerfile ```dockerfile
# Test phase. The 90-second timeout covers both pytest runs. # Test phase. Both pytest runs share one 90-second timeout, so a hang
# uses it once and gets no rerun.
# python:3.x-slim, YYYY-MM-DD # python:3.x-slim, YYYY-MM-DD
FROM python@sha256:... AS test FROM python@sha256:... AS test
# make and git are base tools that script/bootstrap installs when they # make and git are base tools that script/bootstrap installs when they
@@ -484,20 +493,25 @@ style conventions are in separate documents:
improvement bug against that repo. Add a 90-second timeout to the test improvement bug against that repo. Add a 90-second timeout to the test
invocation (`go test -timeout 90s`). The backstop deliberately sits above the invocation (`go test -timeout 90s`). The backstop deliberately sits above the
hard cap so that it catches a genuinely hung test rather than a merely slow hard cap so that it catches a genuinely hung test rather than a merely slow
one. one. A hung package uses the backstop once, because a first run that timed out
gets no rerun. `-timeout` applies to each package, so several hung packages
can each use it.
- **The test command should use the conditional verbose rerun pattern.** Run - **The test command should use the conditional verbose rerun pattern.** Run
tests without `-v` (verbose) first. If tests fail, automatically rerun with tests without `-v` (verbose) first. If tests fail, automatically rerun with
`-v` to show full output. This keeps CI logs and `docker build` output clean `-v` to show full output. This keeps CI logs and `docker build` output clean
on success (just package/suite summaries) while providing full diagnostic on success (just package/suite summaries) while providing full diagnostic
detail on failure (every test case, every assertion). The command lives in the detail on failure (every test case, every assertion). A first run that timed
`test` phase of the `Dockerfile`, since `script/test` builds that phase; the out gets no rerun, because the rerun would only wait for the same hang. The
Makefile form below is the same pattern for any repo-local invocation: command lives in the `test` phase of the `Dockerfile`, since `script/test`
builds that phase; the Makefile form below is the same pattern for any
repo-local invocation:
```makefile ```makefile
test: test:
@<test-command> || \ @<test-command> || \
{ echo "--- Rerunning with -v for details ---"; \ { <exit 1 if that run timed out>; \
echo "--- Rerunning with -v for details ---"; \
<test-command-with-v>; exit 1; } <test-command-with-v>; exit 1; }
``` ```
@@ -505,11 +519,29 @@ style conventions are in separate documents:
```makefile ```makefile
test: test:
@go test -count=1 -timeout 90s -race -cover ./... || \ @output=$$(go test -count=1 -timeout 90s -race -cover ./... 2>&1); \
{ echo "--- Rerunning with -v for details ---"; \ status=$$?; \
go test -count=1 -timeout 90s -race -v ./...; exit 1; } printf '%s\n' "$$output"; \
test "$$status" -eq 0 && exit 0; \
case "$$output" in \
*"panic: test timed out"*|*"*** Test killed"*) exit 1 ;; \
esac; \
echo "--- Rerunning with -v for details ---"; \
go test -count=1 -timeout 90s -race -v ./...; \
exit 1
``` ```
The first run's output is kept so that the recipe can look in it for what Go
prints for a hang:
- `panic: test timed out` means the test's own timer fired.
- `*** Test killed` means `go test` stopped a test that ignored that timer,
about a minute later.
Go prints the goroutine stacks with either line, as long as the test can
still respond. A rerun would only wait for the same hang, so the recipe
exits 1. Any other failure reruns with `-v`. The kept output prints when the
first run ends, not while it runs.
`-count=1` is required on both invocations: it defeats Go's test _result_ `-count=1` is required on both invocations: it defeats Go's test _result_
cache, so neither run can report a stored pass in place of running the cache, so neither run can report a stored pass in place of running the
tests. It leaves the build cache alone, so it costs the runtime of the suite tests. It leaves the build cache alone, so it costs the runtime of the suite
@@ -528,17 +560,22 @@ style conventions are in separate documents:
```makefile ```makefile
test: test:
@python -m pytest || \ @timeout 90 sh -c 'python -m pytest || { \
{ echo "--- Rerunning with -v for details ---"; \ echo "--- Rerunning with -v for details ---"; \
python -m pytest -v; exit 1; } python -m pytest -v; exit 1; }'
``` ```
Both pytest runs share that one 90-second timeout, as in the `Dockerfile`
example above, so a hang uses it once and gets no rerun.
The `exit 1` ensures the target always fails after a rerun — the first run The `exit 1` ensures the target always fails after a rerun — the first run
already proved the tests are broken, so the build must not pass even if a already proved the tests are broken, so the build must not pass even if a
flaky test happens to succeed on the second attempt. The rerun exists solely flaky test happens to succeed on the second attempt. The rerun exists solely
for diagnostic output. for diagnostic output.
- Docker builds must complete in under 5 minutes. - Docker builds must complete in under 5 minutes. A hung package uses the
90-second test backstop once, because a first run that timed out gets no
rerun.
- `make check` must not modify any files in the repo. Tests may use temporary - `make check` must not modify any files in the repo. Tests may use temporary
directories. directories.