REPO_POLICIES: Tests and checks
check / check (push) Canceled after 0s

States once, under the rerun pattern, that a timed-out first run gets
no rerun and a hung package uses the backstop once, which three rules
each said; splits the timeout rule from the 60-second rule and trims
wording; names the examples that carry the command, and that a
Makefile writes each $ as $$, since the Go and Python recipes go in
their sections' commits; every limit stays.

Model: opus-5-5
This commit is contained in:
clawbot committed 2026-11-16 23:37:18 +00:00
1 parent 93de4f1758
commit 2f9f3dd622
1 file changed
+26 -25
+26 -25
View File
@@ -167,25 +167,25 @@ style conventions are in separate documents:
- in `.golangci.yml`: the entries the repo added to the `deny` list of the - in `.golangci.yml`: the entries the repo added to the `deny` list of the
`test-support` depguard rule `test-support` depguard rule
# Tests and checks
- All repos with software must have tests that run via the platform-standard - All repos with software must have tests that run via the platform-standard
test framework (`go test`, `pytest`, `jest`/`vitest`, etc.). If no meaningful test framework (`go test`, `pytest`, `jest`/`vitest`, etc.). There is no
tests exist yet, add the most minimal test possible: importing the module excuse for `make test` to be a no-op. If no meaningful tests exist yet, add
under test to verify it compiles/parses, for example. There is no excuse for the most minimal one: importing the module under test to verify it
`make test` to be a no-op. compiles/parses, for example.
- `make test` must complete in under 60 seconds. That is the hard cap, and a - `make test` must complete in under 60 seconds. That is the hard cap, and a
suite that exceeds it fails. Under 20 seconds is the target. A suite between suite that exceeds it fails. Under 20 seconds is the target. A suite between
20 and 60 seconds is still green, but the overage must be filed as an 20 and 60 seconds is still green, but the overage must be filed as an
improvement bug against that repo. Add a 90-second timeout to the test improvement bug against that repo.
invocation (`go test -timeout 90s`). The backstop deliberately sits above the
hard cap so that it catches a genuinely hung test rather than a merely slow - Add a 90-second timeout to the test invocation (`go test -timeout 90s`). This
one. A hung package uses the backstop once, because a first run that timed out backstop sits above the hard cap on purpose: it catches a hung test, not a
gets no rerun. `-timeout` applies to each package, so several hung packages merely slow one. `-timeout` applies to each package, so several hung packages
can each use it. can each use it.
- Docker builds must complete in under 5 minutes. A hung package uses the - Docker builds must complete in under 5 minutes.
90-second test backstop once, because a first run that timed out gets no
rerun.
- `make check` must not modify any files in the repo. Tests may use temporary - `make check` must not modify any files in the repo. Tests may use temporary
directories. directories.
@@ -193,14 +193,20 @@ style conventions are in separate documents:
- `main` must always pass `make check`, no exceptions. - `main` must always pass `make check`, no exceptions.
- **The test command should use the conditional verbose rerun pattern.** Run - **The test command should use the conditional verbose rerun pattern.** Run
tests without `-v` (verbose) first. If tests fail, automatically rerun with tests without `-v` (verbose) first. If they fail, rerun automatically with
`-v` to show full output. This keeps CI logs and `docker build` output clean `-v`. Success then leaves CI logs and `docker build` output clean, with just
on success, with just package/suite summaries. On failure it provides full package/suite summaries. Failure shows every test case and every assertion.
diagnostic detail: every test case, every assertion. A first run that timed - The rerun always ends in `exit 1`. The first run already proved the tests
out gets no rerun, because the rerun would only wait for the same hang. The are broken, so the build must not pass even if a flaky test succeeds on
command lives in the `test` phase of the `Dockerfile`, since `script/test` the second attempt. The rerun exists solely for diagnostic output.
builds that phase. The Makefile form below is the same pattern for any - A first run that timed out gets no rerun, which would only wait for the
repo-local invocation: same hang. A hung package therefore uses the 90-second backstop once.
The command lives in the `test` phase of the `Dockerfile`, since
`script/test` builds that phase. `examples/go.Dockerfile` and
`examples/python-test-phase.Dockerfile` in the `prompts` repo carry it. The
Makefile form below is the same pattern for any repo-local invocation. In a
Makefile, each `$` of those two commands is written `$$`.
```makefile ```makefile
test: test:
@@ -209,11 +215,6 @@ style conventions are in separate documents:
<test-command-with-v>; exit 1; } <test-command-with-v>; exit 1; }
``` ```
The `exit 1` ensures the target always fails after a rerun. The first run
already proved the tests are broken, so the build must not pass even if a
flaky test happens to succeed on the second attempt. The rerun exists solely
for diagnostic output.
`script/bootstrap` installs all dependencies idempotently and assumes `script/bootstrap` installs all dependencies idempotently and assumes
nothing is present. Base tools, node and yarn come from nix, apt, brew, or nothing is present. Base tools, node and yarn come from nix, apt, brew, or
apk, detected in that order; apt runs noninteractive. Each one is installed apk, detected in that order; apt runs noninteractive. Each one is installed