Test recipe reruns a hung suite, so a hang costs the 90s backstop twice and can exceed the 5 minute build limit #61

Open
opened 2026-09-23 01:43:41 +02:00 by clawbot · 0 comments
Collaborator

Proposed policy change, moved here from sneak/vaultik#101 per your ruling that policy problems belong upstream.

The 30s timeout that issue objected to is already gone: REPO_POLICIES.md on main sets a 60 second cap and a 90 second -timeout backstop (ruling at #41 (comment)). One conflict from that issue is still open.

The conflict. The canonical test recipe (REPO_POLICIES.md:183-185, :283-285) reruns the suite with -v when it fails. When a test hangs, the first run waits the full 90s and the rerun waits another 90s. That is 180s of test time before the build even counts compiling and linting, and :315 says Docker builds must finish in under 5 minutes. vaultik measured a hang build at about 6.5 minutes with a 120s timeout. At 90s it is still close to or over the limit.

Options.

  1. Skip the verbose rerun when the first run timed out, so a hang uses the backstop only once.
  2. Say that the 5 minute build limit does not apply to a build that fails on a hang.
  3. Leave both rules as they are and accept that a hung build breaks the time limit.

Recommendation: option 1. A timeout already prints every goroutine's stack, so a verbose rerun adds nothing, and the build time limit stays meaningful.

Definition of done. The recipe and the build-time text on main agree, and consuming repos pick it up at their next re-vendor.

Model: opus-5-5

Proposed policy change, moved here from https://git.eeqj.de/sneak/vaultik/issues/101 per your ruling that policy problems belong upstream. The 30s timeout that issue objected to is already gone: `REPO_POLICIES.md` on `main` sets a 60 second cap and a 90 second `-timeout` backstop (ruling at https://git.eeqj.de/sneak/prompts/issues/41#issuecomment-53166). One conflict from that issue is still open. **The conflict.** The canonical test recipe (`REPO_POLICIES.md:183-185`, `:283-285`) reruns the suite with `-v` when it fails. When a test hangs, the first run waits the full 90s and the rerun waits another 90s. That is 180s of test time before the build even counts compiling and linting, and `:315` says Docker builds must finish in under 5 minutes. vaultik measured a hang build at about 6.5 minutes with a 120s timeout. At 90s it is still close to or over the limit. **Options.** 1. Skip the verbose rerun when the first run timed out, so a hang uses the backstop only once. 2. Say that the 5 minute build limit does not apply to a build that fails on a hang. 3. Leave both rules as they are and accept that a hung build breaks the time limit. **Recommendation: option 1.** A timeout already prints every goroutine's stack, so a verbose rerun adds nothing, and the build time limit stays meaningful. **Definition of done.** The recipe and the build-time text on `main` agree, and consuming repos pick it up at their next re-vendor. Model: opus-5-5
sneak was assigned by clawbot 2026-09-23 01:43:41 +02:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/prompts#61