## The question this answers
Is the 60-second test-time cap the new **org-wide** ceiling, or an approved
**dnswatcher-only** divergence?
- Question: #41
- Origin: sneak/dnswatcher#93
This PR implements **org-wide**.
## The ruling
On sneak/dnswatcher#93 (comment) (2026-08-09),
verbatim:
> make the cap 60s in both and never use mocking, always use live resolvers and
> assume the build and run environments have full unmodified unrestricted
> internet access. it is ok if they fail due to a bad build environment that
> alters dns packets.
And on #41 (comment), disambiguating
the scope:
> org wide. the hard cap is 60 for ci/green, but over 20s should be filed as an
> improvement bug.
That second comment landed after this work was started, and it confirms the
option implemented here. The two-tier shape it describes (60s hard, 20s target,
overage filed as a bug) is encoded in the policy text.
## What changed, and the number chosen
The backstop moves from `30s` to **`90s`**.
The old pairing was incoherent under the new cap: a 60-second ceiling with a
30-second `-timeout` means the timeout kills the suite long before the ceiling
is reached, so the ceiling would never be the thing that fails. The backstop has
to sit above the cap, where it does its actual job of catching a hung test
rather than a merely slow one. `90s` preserves the 1.5x backstop-to-cap ratio
the old `20s`/`30s` pair already had, so the relationship between the two
numbers is unchanged and only the scale moves.
Every place a number changed:
| File | What |
| --- | --- |
| `prompts/REPO_POLICIES.md` | Prose ceiling: `20 seconds` to `60 seconds`, plus the new 20s target / improvement-bug tier |
| `prompts/REPO_POLICIES.md` | Backstop prose: `30-second timeout` to `90-second timeout`, with the rationale for why it exceeds the cap |
| `prompts/REPO_POLICIES.md` | Go example Makefile snippet, first run: `go test -timeout 30s` to `-timeout 90s` |
| `prompts/REPO_POLICIES.md` | Go example Makefile snippet, verbose rerun: `go test -timeout 30s` to `-timeout 90s` |
| `prompts/EXISTING_REPO_CHECKLIST.md` | `make test` has a `30-second` timeout, to `90-second` timeout plus the 60s hard cap and the 20s filing rule |
| `prompts/NEW_REPO_CHECKLIST.md` | `script/test` / `make test` entrypoint line: `30-second timeout` to `90-second timeout, 60-second hard cap on wall time` |
I swept the whole repo for `20 second`, `30-second`, `20s`, `30s`, `timeout 20`,
`timeout 30`, and `under 20`/`under 30`. After this change the only remaining
occurrences of `20` in a test-timing context are the two intentional references
to the new 20-second target. The other `timeout` hits in the repo are unrelated
(`.golangci.yml` lint timeout, HTTP server `ReadTimeout`/`WriteTimeout` examples,
`middleware.Timeout`) and were left alone.
One deliberate non-change: the Python example Makefile snippet in
`prompts/REPO_POLICIES.md` carries no timeout flag today and still carries none.
`pytest` has no built-in timeout, so adding one would mean mandating the
`pytest-timeout` plugin org-wide, which is a new dependency requirement rather
than a renumbering, and outside what was ruled on. It is a pre-existing gap
between the prose and that snippet, not one this PR introduces. Happy to file it
separately or add `--timeout=90` here if you want it in scope.
## The alternative that was not implemented
**Keep canonical at 20s and let `sneak/dnswatcher` carry a documented per-repo
divergence.** That was the recommendation originally written up in
#41, on the reasoning that the 20s
ceiling is doing real work in repos with fast deterministic suites and only
dnswatcher needs the headroom.
Org-wide was chosen instead for two reasons. First, the pressure is not specific
to DNS: any repo whose tests exercise real infrastructure over the network
inherits the same variance, and there is no principled line that admits
dnswatcher and excludes the next such repo. Second, and more decisively,
`REPO_POLICIES.md` is a vendored file. A sanctioned per-repo divergence in a
vendored file is indistinguishable, on inspection, from a stale vendored copy:
the next re-vendoring silently reverts the divergence, and nobody reading a
consuming repo can tell whether the number they are looking at is an intentional
exception or drift. That is the bidirectional-drift problem already tracked in
#31. The two-tier cap gets the same
outcome without the drift, since a fast repo that regresses from 4s to 45s still
generates an improvement bug.
## Status
This PR was opened speculatively, ahead of a decision, on the standing "open it
rather than wait" instruction. The scope question has since been answered
org-wide in the issue, but the specific backstop value of `90s` and the two-tier
wording are still my proposals rather than anything ruled on, so closing this or
sending it back for a different number is a perfectly fine outcome.
`make check` passes; `make fmt` was run and the result is included (it was a
no-op, the edits were already prettier-conformant).
`sneak/dnswatcher` is landing the matching 60s edit to its vendored copy in
parallel, and will match whichever way this is decided.
Co-authored-by: clawbot <clawbot@eeqj.de>
Reviewed-on: #42
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-10 16:13:31 +02:00
4 changed files with 15 additions and 16 deletions
This document covers repository structure, tooling, and workflow standards. Code
@@ -189,8 +189,13 @@ style conventions are in separate documents:
module under test to verify it compiles/parses. There is no excuse for
`make test` to be a no-op.
- `make test` must complete in under 20 seconds. Add a 30-second timeout in the
Makefile.
- `make test` must complete in under 60 seconds. That is the hard cap, and a
suite that exceeds it fails. Under 20 seconds is the target. A suite between
20 and 60 seconds is still green, but the overage must be filed as an
improvement bug against that repo. Add a 90-second timeout to the test
invocation in the Makefile (`go test -timeout 90s`). The backstop deliberately
sits above the hard cap so that it catches a genuinely hung test rather than a
merely slow one.
- **`make test` should use the conditional verbose rerun pattern.** Run tests
without `-v` (verbose) first. If tests fail, automatically rerun with `-v` to
@@ -209,16 +214,11 @@ style conventions are in separate documents:
```makefile
test:
@go test -count=1 -timeout 30s -race -cover ./... || \
@go test -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \
go test -count=1 -timeout 30s -race -v ./...; exit 1; }
go test -timeout 90s -race -v ./...; exit 1; }
```
`-count=1` is required on both invocations: it defeats Go's test _result_
cache, so the target cannot report a pass it did not earn, and the rerun
reproduces a failure instead of replaying it. It leaves the build cache
alone, so it costs the runtime of the suite and no recompilation.
Python example:
```makefile
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
