Raise org-wide make test cap to 60s, backstop timeout to 90s (#42)
All checks were successful
check / check (push) Successful in 4s
All checks were successful
check / check (push) Successful in 4s
## The question this answers Is the 60-second test-time cap the new **org-wide** ceiling, or an approved **dnswatcher-only** divergence? - Question: #41 - Origin: sneak/dnswatcher#93 This PR implements **org-wide**. ## The ruling On sneak/dnswatcher#93 (comment) (2026-08-09), verbatim: > make the cap 60s in both and never use mocking, always use live resolvers and > assume the build and run environments have full unmodified unrestricted > internet access. it is ok if they fail due to a bad build environment that > alters dns packets. And on #41 (comment), disambiguating the scope: > org wide. the hard cap is 60 for ci/green, but over 20s should be filed as an > improvement bug. That second comment landed after this work was started, and it confirms the option implemented here. The two-tier shape it describes (60s hard, 20s target, overage filed as a bug) is encoded in the policy text. ## What changed, and the number chosen The backstop moves from `30s` to **`90s`**. The old pairing was incoherent under the new cap: a 60-second ceiling with a 30-second `-timeout` means the timeout kills the suite long before the ceiling is reached, so the ceiling would never be the thing that fails. The backstop has to sit above the cap, where it does its actual job of catching a hung test rather than a merely slow one. `90s` preserves the 1.5x backstop-to-cap ratio the old `20s`/`30s` pair already had, so the relationship between the two numbers is unchanged and only the scale moves. Every place a number changed: | File | What | | --- | --- | | `prompts/REPO_POLICIES.md` | Prose ceiling: `20 seconds` to `60 seconds`, plus the new 20s target / improvement-bug tier | | `prompts/REPO_POLICIES.md` | Backstop prose: `30-second timeout` to `90-second timeout`, with the rationale for why it exceeds the cap | | `prompts/REPO_POLICIES.md` | Go example Makefile snippet, first run: `go test -timeout 30s` to `-timeout 90s` | | `prompts/REPO_POLICIES.md` | Go example Makefile snippet, verbose rerun: `go test -timeout 30s` to `-timeout 90s` | | `prompts/EXISTING_REPO_CHECKLIST.md` | `make test` has a `30-second` timeout, to `90-second` timeout plus the 60s hard cap and the 20s filing rule | | `prompts/NEW_REPO_CHECKLIST.md` | `script/test` / `make test` entrypoint line: `30-second timeout` to `90-second timeout, 60-second hard cap on wall time` | I swept the whole repo for `20 second`, `30-second`, `20s`, `30s`, `timeout 20`, `timeout 30`, and `under 20`/`under 30`. After this change the only remaining occurrences of `20` in a test-timing context are the two intentional references to the new 20-second target. The other `timeout` hits in the repo are unrelated (`.golangci.yml` lint timeout, HTTP server `ReadTimeout`/`WriteTimeout` examples, `middleware.Timeout`) and were left alone. One deliberate non-change: the Python example Makefile snippet in `prompts/REPO_POLICIES.md` carries no timeout flag today and still carries none. `pytest` has no built-in timeout, so adding one would mean mandating the `pytest-timeout` plugin org-wide, which is a new dependency requirement rather than a renumbering, and outside what was ruled on. It is a pre-existing gap between the prose and that snippet, not one this PR introduces. Happy to file it separately or add `--timeout=90` here if you want it in scope. ## The alternative that was not implemented **Keep canonical at 20s and let `sneak/dnswatcher` carry a documented per-repo divergence.** That was the recommendation originally written up in #41, on the reasoning that the 20s ceiling is doing real work in repos with fast deterministic suites and only dnswatcher needs the headroom. Org-wide was chosen instead for two reasons. First, the pressure is not specific to DNS: any repo whose tests exercise real infrastructure over the network inherits the same variance, and there is no principled line that admits dnswatcher and excludes the next such repo. Second, and more decisively, `REPO_POLICIES.md` is a vendored file. A sanctioned per-repo divergence in a vendored file is indistinguishable, on inspection, from a stale vendored copy: the next re-vendoring silently reverts the divergence, and nobody reading a consuming repo can tell whether the number they are looking at is an intentional exception or drift. That is the bidirectional-drift problem already tracked in #31. The two-tier cap gets the same outcome without the drift, since a fast repo that regresses from 4s to 45s still generates an improvement bug. ## Status This PR was opened speculatively, ahead of a decision, on the standing "open it rather than wait" instruction. The scope question has since been answered org-wide in the issue, but the specific backstop value of `90s` and the two-tier wording are still my proposals rather than anything ruled on, so closing this or sending it back for a different number is a perfectly fine outcome. `make check` passes; `make fmt` was run and the result is included (it was a no-op, the edits were already prettier-conformant). `sneak/dnswatcher` is landing the matching 60s edit to its vendored copy in parallel, and will match whichever way this is decided. Co-authored-by: clawbot <clawbot@eeqj.de> Reviewed-on: #42 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org>
This commit was merged in pull request #42.
This commit is contained in:
@@ -59,7 +59,9 @@ with your task.
|
||||
- [ ] README has an **Entrypoints** section documenting the `script/`
|
||||
entrypoints and linking the standard
|
||||
- [ ] `make check` does not modify any files in the repo
|
||||
- [ ] `make test` has a 30-second timeout
|
||||
- [ ] `make test` has a 90-second timeout and completes within the 60-second
|
||||
hard cap (over 20 seconds is green but must be filed as an improvement
|
||||
bug)
|
||||
- [ ] `make test` runs real tests, not a no-op (at minimum, import/compile
|
||||
check)
|
||||
- [ ] `make check` passes on current branch
|
||||
|
||||
@@ -80,8 +80,8 @@ are thin shims calling them. Model scripts:
|
||||
installs
|
||||
- [ ] `script/setup` / `make setup` — readies a fresh clone: runs `bootstrap`,
|
||||
then `install-precommit`, plus repo-specific init
|
||||
- [ ] `script/test` / `make test` — runs real tests, not a no-op (30-second
|
||||
timeout)
|
||||
- [ ] `script/test` / `make test` — runs real tests, not a no-op (90-second
|
||||
timeout, 60-second hard cap on wall time)
|
||||
- [ ] `script/lint` / `make lint` — runs linter
|
||||
- [ ] `script/fmt` / `make fmt` — formats code (writes)
|
||||
- [ ] `script/fmt-check` / `make fmt-check` — checks formatting (read-only)
|
||||
|
||||
@@ -189,8 +189,13 @@ style conventions are in separate documents:
|
||||
module under test to verify it compiles/parses. There is no excuse for
|
||||
`make test` to be a no-op.
|
||||
|
||||
- `make test` must complete in under 20 seconds. Add a 30-second timeout in the
|
||||
Makefile.
|
||||
- `make test` must complete in under 60 seconds. That is the hard cap, and a
|
||||
suite that exceeds it fails. Under 20 seconds is the target. A suite between
|
||||
20 and 60 seconds is still green, but the overage must be filed as an
|
||||
improvement bug against that repo. Add a 90-second timeout to the test
|
||||
invocation in the Makefile (`go test -timeout 90s`). The backstop deliberately
|
||||
sits above the hard cap so that it catches a genuinely hung test rather than a
|
||||
merely slow one.
|
||||
|
||||
- **`make test` should use the conditional verbose rerun pattern.** Run tests
|
||||
without `-v` (verbose) first. If tests fail, automatically rerun with `-v` to
|
||||
@@ -209,9 +214,9 @@ style conventions are in separate documents:
|
||||
|
||||
```makefile
|
||||
test:
|
||||
@go test -timeout 30s -race -cover ./... || \
|
||||
@go test -timeout 90s -race -cover ./... || \
|
||||
{ echo "--- Rerunning with -v for details ---"; \
|
||||
go test -timeout 30s -race -v ./...; exit 1; }
|
||||
go test -timeout 90s -race -v ./...; exit 1; }
|
||||
```
|
||||
|
||||
Python example:
|
||||
|
||||
Reference in New Issue
Block a user