Skip the verbose test rerun after a timeout (closes #61)
check / check (push) Canceled after 0s

A hung test used the 90-second backstop twice, because the verbose
rerun waited for the same hang. The Go recipe now looks in the first
run's output for what Go prints for a hang, and exits 1 with no rerun.
Any other failure still reruns with -v.

The Python make test example takes the same timeout wrapper as its
Dockerfile example. The 60 and 90 second rule and the 5-minute rule say
a hung package uses the backstop once.

Model: opus-5-5
This commit is contained in:
2026-11-03 20:37:22 +00:00
parent 430964ceac
commit 9272826fc7
2 changed files with 63 additions and 17 deletions
+9
View File
@@ -21,6 +21,15 @@ fmt-check, and commit.
# Completed Steps
- 2026-11-03: A first test run that timed out now gets no verbose rerun
(https://git.eeqj.de/sneak/prompts/issues/61), on sneak's approval at
https://git.eeqj.de/sneak/prompts/issues/61#issuecomment-165195. The Go recipe
in `REPO_POLICIES.md` looks in the first run's output for
`panic: test timed out` or `*** Test killed`, and exits 1 when either is
there. The Python `make test` example takes the same `timeout 90 sh -c`
wrapper as its `Dockerfile` example. The 60 and 90 second rule and the
5-minute rule now say a hung package uses the backstop once. Consuming
repositories pick this up on their next re-vendor.
- 2026-10-27: Added `prompts/API_POLICIES.md`
(https://git.eeqj.de/sneak/prompts/issues/137). It holds sneak's rule from
https://git.eeqj.de/sneak/routewatch/issues/199 that a value in an API
+54 -17
View File
@@ -1,6 +1,6 @@
---
title: Repository Policies
last_modified: 2026-10-27
last_modified: 2026-11-03
---
This document covers repository structure, tooling, and workflow standards. Code
@@ -233,6 +233,7 @@ style conventions are in separate documents:
# Test phase. -race needs cgo and so a C compiler, which the Debian Go
# image ships and the alpine one does not. -count=1 because the mounted
# cache also holds test results, which go test would otherwise replay.
# A first run that timed out gets no rerun: its output has the stacks.
# golang:1.x, YYYY-MM-DD
FROM golang@sha256:... AS test
WORKDIR /src
@@ -241,9 +242,16 @@ style conventions are in separate documents:
ARG CACHEBUST
COPY . .
RUN --mount=type=cache,target=/root/.cache/go-build \
go test -count=1 -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \
go test -count=1 -timeout 90s -race -v ./...; exit 1; }
output=$(go test -count=1 -timeout 90s -race -cover ./... 2>&1); \
status=$?; \
printf '%s\n' "$output"; \
test "$status" -eq 0 && exit 0; \
case "$output" in \
*"panic: test timed out"*|*"*** Test killed"*) exit 1 ;; \
esac; \
echo "--- Rerunning with -v for details ---"; \
go test -count=1 -timeout 90s -race -v ./...; \
exit 1
# Build stage. Nothing is wanted from either phase above; the copies
# are what make BuildKit build them first, so nothing after them can
@@ -413,7 +421,8 @@ style conventions are in separate documents:
`pipenv install --dev --deploy`:
```dockerfile
# Test phase. The 90-second timeout covers both pytest runs.
# Test phase. Both pytest runs share one 90-second timeout, so a hang
# uses it once and gets no rerun.
# python:3.x-slim, YYYY-MM-DD
FROM python@sha256:... AS test
# make and git are base tools that script/bootstrap installs when they
@@ -484,20 +493,25 @@ style conventions are in separate documents:
improvement bug against that repo. Add a 90-second timeout to the test
invocation (`go test -timeout 90s`). The backstop deliberately sits above the
hard cap so that it catches a genuinely hung test rather than a merely slow
one.
one. A hung package uses the backstop once, because a first run that timed out
gets no rerun. `-timeout` applies to each package, so several hung packages
can each use it.
- **The test command should use the conditional verbose rerun pattern.** Run
tests without `-v` (verbose) first. If tests fail, automatically rerun with
`-v` to show full output. This keeps CI logs and `docker build` output clean
on success (just package/suite summaries) while providing full diagnostic
detail on failure (every test case, every assertion). The command lives in the
`test` phase of the `Dockerfile`, since `script/test` builds that phase; the
Makefile form below is the same pattern for any repo-local invocation:
detail on failure (every test case, every assertion). A first run that timed
out gets no rerun, because the rerun would only wait for the same hang. The
command lives in the `test` phase of the `Dockerfile`, since `script/test`
builds that phase; the Makefile form below is the same pattern for any
repo-local invocation:
```makefile
test:
@<test-command> || \
{ echo "--- Rerunning with -v for details ---"; \
{ <exit 1 if that run timed out>; \
echo "--- Rerunning with -v for details ---"; \
<test-command-with-v>; exit 1; }
```
@@ -505,11 +519,29 @@ style conventions are in separate documents:
```makefile
test:
@go test -count=1 -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \
go test -count=1 -timeout 90s -race -v ./...; exit 1; }
@output=$$(go test -count=1 -timeout 90s -race -cover ./... 2>&1); \
status=$$?; \
printf '%s\n' "$$output"; \
test "$$status" -eq 0 && exit 0; \
case "$$output" in \
*"panic: test timed out"*|*"*** Test killed"*) exit 1 ;; \
esac; \
echo "--- Rerunning with -v for details ---"; \
go test -count=1 -timeout 90s -race -v ./...; \
exit 1
```
The first run's output is kept so that the recipe can look in it for what Go
prints for a hang:
- `panic: test timed out` means the test's own timer fired.
- `*** Test killed` means `go test` stopped a test that ignored that timer,
about a minute later.
Go prints the goroutine stacks with either line, as long as the test can
still respond. A rerun would only wait for the same hang, so the recipe
exits 1. Any other failure reruns with `-v`. The kept output prints when the
first run ends, not while it runs.
`-count=1` is required on both invocations: it defeats Go's test _result_
cache, so neither run can report a stored pass in place of running the
tests. It leaves the build cache alone, so it costs the runtime of the suite
@@ -528,17 +560,22 @@ style conventions are in separate documents:
```makefile
test:
@python -m pytest || \
{ echo "--- Rerunning with -v for details ---"; \
python -m pytest -v; exit 1; }
@timeout 90 sh -c 'python -m pytest || { \
echo "--- Rerunning with -v for details ---"; \
python -m pytest -v; exit 1; }'
```
Both pytest runs share that one 90-second timeout, as in the `Dockerfile`
example above, so a hang uses it once and gets no rerun.
The `exit 1` ensures the target always fails after a rerun — the first run
already proved the tests are broken, so the build must not pass even if a
flaky test happens to succeed on the second attempt. The rerun exists solely
for diagnostic output.
- Docker builds must complete in under 5 minutes.
- Docker builds must complete in under 5 minutes. A hung package uses the
90-second test backstop once, because a first run that timed out gets no
rerun.
- `make check` must not modify any files in the repo. Tests may use temporary
directories.