diff --git a/prompts/REPO_POLICIES.md b/prompts/REPO_POLICIES.md index 5002521..c5f17b9 100644 --- a/prompts/REPO_POLICIES.md +++ b/prompts/REPO_POLICIES.md @@ -167,25 +167,25 @@ style conventions are in separate documents: - in `.golangci.yml`: the entries the repo added to the `deny` list of the `test-support` depguard rule +# Tests and checks + - All repos with software must have tests that run via the platform-standard - test framework (`go test`, `pytest`, `jest`/`vitest`, etc.). If no meaningful - tests exist yet, add the most minimal test possible: importing the module - under test to verify it compiles/parses, for example. There is no excuse for - `make test` to be a no-op. + test framework (`go test`, `pytest`, `jest`/`vitest`, etc.). There is no + excuse for `make test` to be a no-op. If no meaningful tests exist yet, add + the most minimal one: importing the module under test to verify it + compiles/parses, for example. - `make test` must complete in under 60 seconds. That is the hard cap, and a suite that exceeds it fails. Under 20 seconds is the target. A suite between 20 and 60 seconds is still green, but the overage must be filed as an - improvement bug against that repo. Add a 90-second timeout to the test - invocation (`go test -timeout 90s`). The backstop deliberately sits above the - hard cap so that it catches a genuinely hung test rather than a merely slow - one. A hung package uses the backstop once, because a first run that timed out - gets no rerun. `-timeout` applies to each package, so several hung packages + improvement bug against that repo. + +- Add a 90-second timeout to the test invocation (`go test -timeout 90s`). This + backstop sits above the hard cap on purpose: it catches a hung test, not a + merely slow one. `-timeout` applies to each package, so several hung packages can each use it. -- Docker builds must complete in under 5 minutes. A hung package uses the - 90-second test backstop once, because a first run that timed out gets no - rerun. +- Docker builds must complete in under 5 minutes. - `make check` must not modify any files in the repo. Tests may use temporary directories. @@ -193,14 +193,20 @@ style conventions are in separate documents: - `main` must always pass `make check`, no exceptions. - **The test command should use the conditional verbose rerun pattern.** Run - tests without `-v` (verbose) first. If tests fail, automatically rerun with - `-v` to show full output. This keeps CI logs and `docker build` output clean - on success, with just package/suite summaries. On failure it provides full - diagnostic detail: every test case, every assertion. A first run that timed - out gets no rerun, because the rerun would only wait for the same hang. The - command lives in the `test` phase of the `Dockerfile`, since `script/test` - builds that phase. The Makefile form below is the same pattern for any - repo-local invocation: + tests without `-v` (verbose) first. If they fail, rerun automatically with + `-v`. Success then leaves CI logs and `docker build` output clean, with just + package/suite summaries. Failure shows every test case and every assertion. + - The rerun always ends in `exit 1`. The first run already proved the tests + are broken, so the build must not pass even if a flaky test succeeds on + the second attempt. The rerun exists solely for diagnostic output. + - A first run that timed out gets no rerun, which would only wait for the + same hang. A hung package therefore uses the 90-second backstop once. + + The command lives in the `test` phase of the `Dockerfile`, since + `script/test` builds that phase. `examples/go.Dockerfile` and + `examples/python-test-phase.Dockerfile` in the `prompts` repo carry it. The + Makefile form below is the same pattern for any repo-local invocation. In a + Makefile, each `$` of those two commands is written `$$`. ```makefile test: @@ -209,11 +215,6 @@ style conventions are in separate documents: ; exit 1; } ``` - The `exit 1` ensures the target always fails after a rerun. The first run - already proved the tests are broken, so the build must not pass even if a - flaky test happens to succeed on the second attempt. The rerun exists solely - for diagnostic output. - `script/bootstrap` installs all dependencies idempotently and assumes nothing is present. Base tools, node and yarn come from nix, apt, brew, or apk, detected in that order; apt runs noninteractive. Each one is installed