Raise org-wide make test cap to 60s, backstop timeout to 90s #42

2026-08-10T14:54:28+02:00

clawbot commented

2026-08-10 14:54:28 +02:00

The question this answers

Is the 60-second test-time cap the new org-wide ceiling, or an approved
dnswatcher-only divergence?

Question: #41
Origin: sneak/dnswatcher#93

This PR implements org-wide.

The ruling

On sneak/dnswatcher#93 (comment) (2026-08-09),
verbatim:

> make the cap 60s in both and never use mocking, always use live resolvers and
> assume the build and run environments have full unmodified unrestricted
> internet access. it is ok if they fail due to a bad build environment that
> alters dns packets.

And on #41 (comment), disambiguating
the scope:

> org wide. the hard cap is 60 for ci/green, but over 20s should be filed as an
> improvement bug.

That second comment landed after this work was started, and it confirms the
option implemented here. The two-tier shape it describes (60s hard, 20s target,
overage filed as a bug) is encoded in the policy text.

What changed, and the number chosen

The backstop moves from 30s to 90s.

The old pairing was incoherent under the new cap: a 60-second ceiling with a
30-second -timeout means the timeout kills the suite long before the ceiling
is reached, so the ceiling would never be the thing that fails. The backstop has
to sit above the cap, where it does its actual job of catching a hung test
rather than a merely slow one. 90s preserves the 1.5x backstop-to-cap ratio
the old 20s/30s pair already had, so the relationship between the two
numbers is unchanged and only the scale moves.

Every place a number changed:

File	What
`prompts/REPO_POLICIES.md`	Prose ceiling: `20 seconds` to `60 seconds`, plus the new 20s target / improvement-bug tier
`prompts/REPO_POLICIES.md`	Backstop prose: `30-second timeout` to `90-second timeout`, with the rationale for why it exceeds the cap
`prompts/REPO_POLICIES.md`	Go example Makefile snippet, first run: `go test -timeout 30s` to `-timeout 90s`
`prompts/REPO_POLICIES.md`	Go example Makefile snippet, verbose rerun: `go test -timeout 30s` to `-timeout 90s`
`prompts/EXISTING_REPO_CHECKLIST.md`	`make test` has a `30-second` timeout, to `90-second` timeout plus the 60s hard cap and the 20s filing rule
`prompts/NEW_REPO_CHECKLIST.md`	`script/test` / `make test` entrypoint line: `30-second timeout` to `90-second timeout, 60-second hard cap on wall time`

I swept the whole repo for 20 second, 30-second, 20s, 30s, timeout 20,
timeout 30, and under 20/under 30. After this change the only remaining
occurrences of 20 in a test-timing context are the two intentional references
to the new 20-second target. The other timeout hits in the repo are unrelated
(.golangci.yml lint timeout, HTTP server ReadTimeout/WriteTimeout examples,
middleware.Timeout) and were left alone.

One deliberate non-change: the Python example Makefile snippet in
prompts/REPO_POLICIES.md carries no timeout flag today and still carries none.
pytest has no built-in timeout, so adding one would mean mandating the
pytest-timeout plugin org-wide, which is a new dependency requirement rather
than a renumbering, and outside what was ruled on. It is a pre-existing gap
between the prose and that snippet, not one this PR introduces. Happy to file it
separately or add --timeout=90 here if you want it in scope.

The alternative that was not implemented

Keep canonical at 20s and let sneak/dnswatcher carry a documented per-repo
divergence. That was the recommendation originally written up in
#41, on the reasoning that the 20s
ceiling is doing real work in repos with fast deterministic suites and only
dnswatcher needs the headroom.

Org-wide was chosen instead for two reasons. First, the pressure is not specific
to DNS: any repo whose tests exercise real infrastructure over the network
inherits the same variance, and there is no principled line that admits
dnswatcher and excludes the next such repo. Second, and more decisively,
REPO_POLICIES.md is a vendored file. A sanctioned per-repo divergence in a
vendored file is indistinguishable, on inspection, from a stale vendored copy:
the next re-vendoring silently reverts the divergence, and nobody reading a
consuming repo can tell whether the number they are looking at is an intentional
exception or drift. That is the bidirectional-drift problem already tracked in
#31. The two-tier cap gets the same
outcome without the drift, since a fast repo that regresses from 4s to 45s still
generates an improvement bug.

Status

This PR was opened speculatively, ahead of a decision, on the standing "open it
rather than wait" instruction. The scope question has since been answered
org-wide in the issue, but the specific backstop value of 90s and the two-tier
wording are still my proposals rather than anything ruled on, so closing this or
sending it back for a different number is a perfectly fine outcome.

make check passes; make fmt was run and the result is included (it was a
no-op, the edits were already prettier-conformant).

sneak/dnswatcher is landing the matching 60s edit to its vendored copy in
parallel, and will match whichever way this is decided.

## The question this answers Is the 60-second test-time cap the new **org-wide** ceiling, or an approved **dnswatcher-only** divergence? - Question: https://git.eeqj.de/sneak/prompts/issues/41 - Origin: https://git.eeqj.de/sneak/dnswatcher/issues/93 This PR implements **org-wide**. ## The ruling On https://git.eeqj.de/sneak/dnswatcher/issues/93#issuecomment-50438 (2026-08-09), verbatim: > make the cap 60s in both and never use mocking, always use live resolvers and > assume the build and run environments have full unmodified unrestricted > internet access. it is ok if they fail due to a bad build environment that > alters dns packets. And on https://git.eeqj.de/sneak/prompts/issues/41#issuecomment-53166, disambiguating the scope: > org wide. the hard cap is 60 for ci/green, but over 20s should be filed as an > improvement bug. That second comment landed after this work was started, and it confirms the option implemented here. The two-tier shape it describes (60s hard, 20s target, overage filed as a bug) is encoded in the policy text. ## What changed, and the number chosen The backstop moves from `30s` to **`90s`**. The old pairing was incoherent under the new cap: a 60-second ceiling with a 30-second `-timeout` means the timeout kills the suite long before the ceiling is reached, so the ceiling would never be the thing that fails. The backstop has to sit above the cap, where it does its actual job of catching a hung test rather than a merely slow one. `90s` preserves the 1.5x backstop-to-cap ratio the old `20s`/`30s` pair already had, so the relationship between the two numbers is unchanged and only the scale moves. Every place a number changed: | File | What | | --- | --- | | `prompts/REPO_POLICIES.md` | Prose ceiling: `20 seconds` to `60 seconds`, plus the new 20s target / improvement-bug tier | | `prompts/REPO_POLICIES.md` | Backstop prose: `30-second timeout` to `90-second timeout`, with the rationale for why it exceeds the cap | | `prompts/REPO_POLICIES.md` | Go example Makefile snippet, first run: `go test -timeout 30s` to `-timeout 90s` | | `prompts/REPO_POLICIES.md` | Go example Makefile snippet, verbose rerun: `go test -timeout 30s` to `-timeout 90s` | | `prompts/EXISTING_REPO_CHECKLIST.md` | `make test` has a `30-second` timeout, to `90-second` timeout plus the 60s hard cap and the 20s filing rule | | `prompts/NEW_REPO_CHECKLIST.md` | `script/test` / `make test` entrypoint line: `30-second timeout` to `90-second timeout, 60-second hard cap on wall time` | I swept the whole repo for `20 second`, `30-second`, `20s`, `30s`, `timeout 20`, `timeout 30`, and `under 20`/`under 30`. After this change the only remaining occurrences of `20` in a test-timing context are the two intentional references to the new 20-second target. The other `timeout` hits in the repo are unrelated (`.golangci.yml` lint timeout, HTTP server `ReadTimeout`/`WriteTimeout` examples, `middleware.Timeout`) and were left alone. One deliberate non-change: the Python example Makefile snippet in `prompts/REPO_POLICIES.md` carries no timeout flag today and still carries none. `pytest` has no built-in timeout, so adding one would mean mandating the `pytest-timeout` plugin org-wide, which is a new dependency requirement rather than a renumbering, and outside what was ruled on. It is a pre-existing gap between the prose and that snippet, not one this PR introduces. Happy to file it separately or add `--timeout=90` here if you want it in scope. ## The alternative that was not implemented **Keep canonical at 20s and let `sneak/dnswatcher` carry a documented per-repo divergence.** That was the recommendation originally written up in https://git.eeqj.de/sneak/prompts/issues/41, on the reasoning that the 20s ceiling is doing real work in repos with fast deterministic suites and only dnswatcher needs the headroom. Org-wide was chosen instead for two reasons. First, the pressure is not specific to DNS: any repo whose tests exercise real infrastructure over the network inherits the same variance, and there is no principled line that admits dnswatcher and excludes the next such repo. Second, and more decisively, `REPO_POLICIES.md` is a vendored file. A sanctioned per-repo divergence in a vendored file is indistinguishable, on inspection, from a stale vendored copy: the next re-vendoring silently reverts the divergence, and nobody reading a consuming repo can tell whether the number they are looking at is an intentional exception or drift. That is the bidirectional-drift problem already tracked in https://git.eeqj.de/sneak/prompts/issues/31. The two-tier cap gets the same outcome without the drift, since a fast repo that regresses from 4s to 45s still generates an improvement bug. ## Status This PR was opened speculatively, ahead of a decision, on the standing "open it rather than wait" instruction. The scope question has since been answered org-wide in the issue, but the specific backstop value of `90s` and the two-tier wording are still my proposals rather than anything ruled on, so closing this or sending it back for a different number is a perfectly fine outcome. `make check` passes; `make fmt` was run and the result is included (it was a no-op, the edits were already prettier-conformant). `sneak/dnswatcher` is landing the matching 60s edit to its vendored copy in parallel, and will match whichever way this is decided.

clawbot added 1 commit 2026-08-10 14:54:28 +02:00

Raise org-wide make test cap to 60s, backstop timeout to 90s

check / check (push) Successful in 12s

Details

52b51929dd

Per the ruling on prompts issue 41: the 60-second cap is org-wide, not a
per-repo divergence. 60 seconds is the hard cap for CI green; anything
over 20 seconds is still green but must be filed as an improvement bug.

The old pairing of a 20-second cap with a 30-second `-timeout` stays
coherent only if the backstop exceeds the cap, so the timeout moves to
90s (the same 1.5x ratio the old numbers had). Applied to the prose
ceiling in `REPO_POLICIES.md`, the Go example Makefile snippet, and the
`make test` lines in both repo checklists.

#41
sneak/dnswatcher#93

clawbot referenced this pull request

2026-08-10 14:54:36 +02:00

Test-time cap: is 60s the new org-wide ceiling, or a dnswatcher divergence? #41

clawbot referenced this pull request from sneak/dnswatcher

2026-08-10 15:14:27 +02:00

next #136

clawbot referenced this pull request from sneak/dnswatcher

2026-08-10 15:25:13 +02:00

next #136

clawbot referenced this pull request

2026-08-10 15:27:00 +02:00

All linting must run in Docker: canonicalise homoicon's Dockerfile.lint + script/lint pattern #40