Test-time cap: is 60s the new org-wide ceiling, or a dnswatcher divergence? #41
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Needs a one-line answer from the owner; raised rather than assumed because guessing either way creates a real problem.
What happened
On sneak/dnswatcher#93 sneak ruled (2026-08-09):
"Both" refers to two documents that currently disagree in that repo:
TESTING.mdsays a 30-second target, andREPO_POLICIES.mdsets themake testceiling at 20 seconds with a 30s timeout backstop.The question
REPO_POLICIES.mdis canonical here and vendored into every repo. So the ruling could mean either:The ruling was given on a dnswatcher issue, which weakly suggests (2), but
REPO_POLICIES.mdwas named explicitly, which suggests (1). Not a guess worth making silently in a file every repo inherits.Why it matters beyond dnswatcher
Repos have been treating the 20s ceiling as a real constraint in live design decisions — for instance a
-racechange was weighed against it and parked, because the suite already took 18.7s without the detector. If the cap is now 60s, that calculus changes and a parked decision may unpark itself. If it is not, dnswatcher needs its divergence recorded so the next re-vendoring does not silently revert it.Recommendation
Option 2, unless the intent really was org-wide. The 20s ceiling is doing useful work in repos with fast deterministic suites, and dnswatcher's need is specific to querying real nameservers over the network. Recording it as an approved divergence in dnswatcher's vendored copy keeps both facts true.
Either way this should be settled here rather than in dnswatcher, because a vendored file edited in one repo without a canonical decision is exactly the bidirectional drift already tracked in #31 .
Not blocking: the dnswatcher test work can proceed on the substance of the ruling (live resolvers, no mocking, no skip flags) while this is decided.
org wide. the hard cap is 60 for ci/green, but over 20s should be filed as an improvement bug.
Proposed answer implemented as #42 (branch
org-wide-60s-test-cap).It takes the org-wide option, encoding the two tiers from your comment above: 60 seconds hard cap, 20 seconds target, and anything between the two is green but must be filed as an improvement bug.
The backstop moves
30sto90s, because a 60-second cap with a 30-second-timeoutnever lets the cap be the thing that fails.90skeeps the same 1.5x backstop-to-cap ratio the old20s/30spair had.Changed in
prompts/REPO_POLICIES.md(prose ceiling, backstop prose, both lines of the Go example Makefile snippet),prompts/EXISTING_REPO_CHECKLIST.md, andprompts/NEW_REPO_CHECKLIST.md. The Python example snippet had no timeout flag before and still has none, since adding one would mean mandatingpytest-timeoutorg-wide; called out in the PR body if you want that in scope.The specific value of
90sis my proposal rather than anything ruled on, so send it back if you want a different number.