REPO_POLICIES: Tests and checks
check / check (push) Canceled after 0s

States once, under the rerun pattern, that a timed-out first run gets
no rerun and a hung package uses the backstop once, which three rules
each said; splits the timeout rule from the 60-second rule and trims
wording; names the examples that carry the command, and that a
Makefile writes each $ as $$, since the Go and Python recipes go in
their sections' commits; every limit stays.

Model: opus-5-5
This commit is contained in:
clawbot committed 2026-11-16 23:37:18 +00:00
1 parent 93de4f1758
commit 2f9f3dd622
1 file changed
+26 -25
+26 -25
View File
@@ -167,25 +167,25 @@ style conventions are in separate documents:
- in `.golangci.yml`: the entries the repo added to the `deny` list of the
`test-support` depguard rule
# Tests and checks
- All repos with software must have tests that run via the platform-standard
test framework (`go test`, `pytest`, `jest`/`vitest`, etc.). If no meaningful
tests exist yet, add the most minimal test possible: importing the module
under test to verify it compiles/parses, for example. There is no excuse for
`make test` to be a no-op.
test framework (`go test`, `pytest`, `jest`/`vitest`, etc.). There is no
excuse for `make test` to be a no-op. If no meaningful tests exist yet, add
the most minimal one: importing the module under test to verify it
compiles/parses, for example.
- `make test` must complete in under 60 seconds. That is the hard cap, and a
suite that exceeds it fails. Under 20 seconds is the target. A suite between
20 and 60 seconds is still green, but the overage must be filed as an
improvement bug against that repo. Add a 90-second timeout to the test
invocation (`go test -timeout 90s`). The backstop deliberately sits above the
hard cap so that it catches a genuinely hung test rather than a merely slow
one. A hung package uses the backstop once, because a first run that timed out
gets no rerun. `-timeout` applies to each package, so several hung packages
improvement bug against that repo.
- Add a 90-second timeout to the test invocation (`go test -timeout 90s`). This
backstop sits above the hard cap on purpose: it catches a hung test, not a
merely slow one. `-timeout` applies to each package, so several hung packages
can each use it.
- Docker builds must complete in under 5 minutes. A hung package uses the
90-second test backstop once, because a first run that timed out gets no
rerun.
- Docker builds must complete in under 5 minutes.
- `make check` must not modify any files in the repo. Tests may use temporary
directories.
@@ -193,14 +193,20 @@ style conventions are in separate documents:
- `main` must always pass `make check`, no exceptions.
- **The test command should use the conditional verbose rerun pattern.** Run
tests without `-v` (verbose) first. If tests fail, automatically rerun with
`-v` to show full output. This keeps CI logs and `docker build` output clean
on success, with just package/suite summaries. On failure it provides full
diagnostic detail: every test case, every assertion. A first run that timed
out gets no rerun, because the rerun would only wait for the same hang. The
command lives in the `test` phase of the `Dockerfile`, since `script/test`
builds that phase. The Makefile form below is the same pattern for any
repo-local invocation:
tests without `-v` (verbose) first. If they fail, rerun automatically with
`-v`. Success then leaves CI logs and `docker build` output clean, with just
package/suite summaries. Failure shows every test case and every assertion.
- The rerun always ends in `exit 1`. The first run already proved the tests
are broken, so the build must not pass even if a flaky test succeeds on
the second attempt. The rerun exists solely for diagnostic output.
- A first run that timed out gets no rerun, which would only wait for the
same hang. A hung package therefore uses the 90-second backstop once.
The command lives in the `test` phase of the `Dockerfile`, since
`script/test` builds that phase. `examples/go.Dockerfile` and
`examples/python-test-phase.Dockerfile` in the `prompts` repo carry it. The
Makefile form below is the same pattern for any repo-local invocation. In a
Makefile, each `$` of those two commands is written `$$`.
```makefile
test:
@@ -209,11 +215,6 @@ style conventions are in separate documents:
<test-command-with-v>; exit 1; }
```
The `exit 1` ensures the target always fails after a rerun. The first run
already proved the tests are broken, so the build must not pass even if a
flaky test happens to succeed on the second attempt. The rerun exists solely
for diagnostic output.
`script/bootstrap` installs all dependencies idempotently and assumes
nothing is present. Base tools, node and yarn come from nix, apt, brew, or
apk, detected in that order; apt runs noninteractive. Each one is installed