diff --git a/TODO.md b/TODO.md index 950d6fc..49e890c 100644 --- a/TODO.md +++ b/TODO.md @@ -21,6 +21,16 @@ fmt-check, and commit. # Completed Steps +- 2026-11-03: A first test run that timed out now gets no verbose rerun + (https://git.eeqj.de/sneak/prompts/issues/61), on sneak's approval at + https://git.eeqj.de/sneak/prompts/issues/61#issuecomment-165195. The Go recipe + in `REPO_POLICIES.md` looks in the first run's output for + `panic: test timed out` or `*** Test killed`, and exits 1 when either is + there. The Python `make test` example takes the same `timeout 90 sh -c` + wrapper as its `Dockerfile` example. The 60 and 90 second rule and the + 5-minute rule now say a hung package uses the backstop once. The Go item in + both repo checklists now asks for this, so a repository with the old recipe + fails it. - 2026-10-27: Added `prompts/API_POLICIES.md` (https://git.eeqj.de/sneak/prompts/issues/137). It holds sneak's rule from https://git.eeqj.de/sneak/routewatch/issues/199 that a value in an API diff --git a/prompts/EXISTING_REPO_CHECKLIST.md b/prompts/EXISTING_REPO_CHECKLIST.md index 79625c6..47a92ed 100644 --- a/prompts/EXISTING_REPO_CHECKLIST.md +++ b/prompts/EXISTING_REPO_CHECKLIST.md @@ -1,6 +1,6 @@ --- title: Existing Repo Checklist -last_modified: 2026-10-26 +last_modified: 2026-11-03 --- Use this checklist when beginning work in a repo that may not yet conform to our @@ -70,8 +70,9 @@ with your task. (`--mount=type=cache,target=/root/.cache/go-build`), `golangci-lint run` also mounts golangci-lint's (`--mount=type=cache,target=/root/.cache/golangci-lint`), and no step - mounts anything else; both `go test` runs carry `-count=1`, as in the - canonical Go `Dockerfile` in `REPO_POLICIES.md` + mounts anything else; both `go test` runs carry `-count=1`; a first run + that timed out gets no rerun, as in the canonical Go `Dockerfile` in + `REPO_POLICIES.md` - [ ] Python: a `test` phase that installs its packages from `Pipfile.lock` does it above `ARG CACHEBUST`, with `COPY script/ script/`, `COPY Pipfile Pipfile.lock ./` and `RUN script/bootstrap`. When diff --git a/prompts/NEW_REPO_CHECKLIST.md b/prompts/NEW_REPO_CHECKLIST.md index 51dd89c..fd309cb 100644 --- a/prompts/NEW_REPO_CHECKLIST.md +++ b/prompts/NEW_REPO_CHECKLIST.md @@ -1,6 +1,6 @@ --- title: New Repo Checklist -last_modified: 2026-10-26 +last_modified: 2026-11-03 --- Use this checklist when creating a new repository from scratch. Follow the steps @@ -129,8 +129,9 @@ Template files can be fetched from: (`--mount=type=cache,target=/root/.cache/go-build`), `golangci-lint run` also mounts golangci-lint's (`--mount=type=cache,target=/root/.cache/golangci-lint`), and no step - mounts anything else; both `go test` runs carry `-count=1`, as in the - canonical Go `Dockerfile` in `REPO_POLICIES.md` + mounts anything else; both `go test` runs carry `-count=1`; a first run + that timed out gets no rerun, as in the canonical Go `Dockerfile` in + `REPO_POLICIES.md` - Python: a `test` phase that installs its packages from `Pipfile.lock` does it above `ARG CACHEBUST`, with `COPY script/ script/`, `COPY Pipfile Pipfile.lock ./` and `RUN script/bootstrap`. When diff --git a/prompts/REPO_POLICIES.md b/prompts/REPO_POLICIES.md index 3e8c97f..c075882 100644 --- a/prompts/REPO_POLICIES.md +++ b/prompts/REPO_POLICIES.md @@ -1,6 +1,6 @@ --- title: Repository Policies -last_modified: 2026-10-27 +last_modified: 2026-11-03 --- This document covers repository structure, tooling, and workflow standards. Code @@ -233,6 +233,8 @@ style conventions are in separate documents: # Test phase. -race needs cgo and so a C compiler, which the Debian Go # image ships and the alpine one does not. -count=1 because the mounted # cache also holds test results, which go test would otherwise replay. + # A first run that timed out gets no rerun, because a rerun would only + # wait for the same hang. # golang:1.x, YYYY-MM-DD FROM golang@sha256:... AS test WORKDIR /src @@ -241,9 +243,16 @@ style conventions are in separate documents: ARG CACHEBUST COPY . . RUN --mount=type=cache,target=/root/.cache/go-build \ - go test -count=1 -timeout 90s -race -cover ./... || \ - { echo "--- Rerunning with -v for details ---"; \ - go test -count=1 -timeout 90s -race -v ./...; exit 1; } + status=0; \ + output=$(go test -count=1 -timeout 90s -race -cover ./... 2>&1) || status=$?; \ + printf '%s\n' "$output"; \ + test "$status" -eq 0 && exit 0; \ + case "$output" in \ + *"panic: test timed out"*|*"*** Test killed"*) exit 1 ;; \ + esac; \ + echo "--- Rerunning with -v for details ---"; \ + go test -count=1 -timeout 90s -race -v ./...; \ + exit 1 # Build stage. Nothing is wanted from either phase above; the copies # are what make BuildKit build them first, so nothing after them can @@ -413,7 +422,8 @@ style conventions are in separate documents: `pipenv install --dev --deploy`: ```dockerfile - # Test phase. The 90-second timeout covers both pytest runs. + # Test phase. Both pytest runs share one 90-second timeout, so a hang + # uses it once and gets no rerun. # python:3.x-slim, YYYY-MM-DD FROM python@sha256:... AS test # make and git are base tools that script/bootstrap installs when they @@ -484,15 +494,19 @@ style conventions are in separate documents: improvement bug against that repo. Add a 90-second timeout to the test invocation (`go test -timeout 90s`). The backstop deliberately sits above the hard cap so that it catches a genuinely hung test rather than a merely slow - one. + one. A hung package uses the backstop once, because a first run that timed out + gets no rerun. `-timeout` applies to each package, so several hung packages + can each use it. - **The test command should use the conditional verbose rerun pattern.** Run tests without `-v` (verbose) first. If tests fail, automatically rerun with `-v` to show full output. This keeps CI logs and `docker build` output clean on success (just package/suite summaries) while providing full diagnostic - detail on failure (every test case, every assertion). The command lives in the - `test` phase of the `Dockerfile`, since `script/test` builds that phase; the - Makefile form below is the same pattern for any repo-local invocation: + detail on failure (every test case, every assertion). A first run that timed + out gets no rerun, because the rerun would only wait for the same hang. The + command lives in the `test` phase of the `Dockerfile`, since `script/test` + builds that phase; the Makefile form below is the same pattern for any + repo-local invocation: ```makefile test: @@ -505,11 +519,29 @@ style conventions are in separate documents: ```makefile test: - @go test -count=1 -timeout 90s -race -cover ./... || \ - { echo "--- Rerunning with -v for details ---"; \ - go test -count=1 -timeout 90s -race -v ./...; exit 1; } + @status=0; \ + output=$$(go test -count=1 -timeout 90s -race -cover ./... 2>&1) || status=$$?; \ + printf '%s\n' "$$output"; \ + test "$$status" -eq 0 && exit 0; \ + case "$$output" in \ + *"panic: test timed out"*|*"*** Test killed"*) exit 1 ;; \ + esac; \ + echo "--- Rerunning with -v for details ---"; \ + go test -count=1 -timeout 90s -race -v ./...; \ + exit 1 ``` + The first run's output is kept so that the recipe can look in it for what Go + prints for a hang: + - `panic: test timed out` means the test's own timer fired. + - `*** Test killed` means `go test` stopped a test that ignored that timer, + about a minute later. + + Go prints the goroutine stacks with either line, as long as the test can + still respond. A rerun would only wait for the same hang, so the recipe + exits 1. A failure with neither line in its output reruns with `-v`. The + kept output prints when the first run ends, not while it runs. + `-count=1` is required on both invocations: it defeats Go's test _result_ cache, so neither run can report a stored pass in place of running the tests. It leaves the build cache alone, so it costs the runtime of the suite @@ -528,17 +560,25 @@ style conventions are in separate documents: ```makefile test: - @python -m pytest || \ - { echo "--- Rerunning with -v for details ---"; \ - python -m pytest -v; exit 1; } + @timeout 90 sh -c 'python -m pytest || { \ + echo "--- Rerunning with -v for details ---"; \ + python -m pytest -v; exit 1; }' ``` + Both pytest runs share that one 90-second timeout, as in the `Dockerfile` + example above, so a hang uses it once and gets no rerun. Ctrl-C does not + stop a running `make test`, which goes on until the tests end or the 90 + seconds run out. On a Mac without `timeout`, the target fails until it is + installed. + The `exit 1` ensures the target always fails after a rerun — the first run already proved the tests are broken, so the build must not pass even if a flaky test happens to succeed on the second attempt. The rerun exists solely for diagnostic output. -- Docker builds must complete in under 5 minutes. +- Docker builds must complete in under 5 minutes. A hung package uses the + 90-second test backstop once, because a first run that timed out gets no + rerun. - `make check` must not modify any files in the repo. Tests may use temporary directories.