A hung test used the 90-second backstop twice, because the verbose rerun waited for the same hang. That put a hung build at or over the 5-minute limit.
The Go recipe, in both the Dockerfile and make test forms, now keeps the first run's output and looks in it for panic: test timed out or *** Test killed. Either one exits 1 with no rerun. A failure with neither line in its output reruns with -v.
The Python make test example takes the same timeout 90 sh -c wrapper as its Dockerfile example. The test-time rule and the 5-minute rule now say a hung package uses the backstop once. The Go item in both repo checklists now asks for this.
A test that ignores its timer runs about a minute past the backstop before go test stops it. It still runs once.
Disclosures:
Deviation: the brief says stacks are already printed on both paths. A stopped test printed none on the *** Test killed path, so the policy says "as long as the test can still respond".
Judgement call: a plain failure with either line in its output, or alongside a hang, also gets no rerun.
Judgement call: the output is held in a shell variable, so it prints when the first run ends.
Cost: Ctrl-C does not stop a running Python make test.
Unverified: whether current macOS ships timeout; not tried on a Mac.
Model: opus-5-5
Implements option 1 of https://git.eeqj.de/sneak/prompts/issues/61, as approved at https://git.eeqj.de/sneak/prompts/issues/61#issuecomment-165195.
A hung test used the 90-second backstop twice, because the verbose rerun waited for the same hang. That put a hung build at or over the 5-minute limit.
The Go recipe, in both the `Dockerfile` and `make test` forms, now keeps the first run's output and looks in it for `panic: test timed out` or `*** Test killed`. Either one exits 1 with no rerun. A failure with neither line in its output reruns with `-v`.
The Python `make test` example takes the same `timeout 90 sh -c` wrapper as its `Dockerfile` example. The test-time rule and the 5-minute rule now say a hung package uses the backstop once. The Go item in both repo checklists now asks for this.
A test that ignores its timer runs about a minute past the backstop before `go test` stops it. It still runs once.
Disclosures:
- Deviation: the brief says stacks are already printed on both paths. A stopped test printed none on the `*** Test killed` path, so the policy says "as long as the test can still respond".
- Judgement call: a plain failure with either line in its output, or alongside a hang, also gets no rerun.
- Judgement call: the output is held in a shell variable, so it prints when the first run ends.
- Cost: Ctrl-C does not stop a running Python `make test`.
- Unverified: whether current macOS ships `timeout`; not tried on a Mac.
Model: opus-5-5
clawbot
self-assigned this 2026-11-03 21:38:04 +01:00
FAIL. Findings are in prompts/REPO_POLICIES.md unless another file is named.
The recipe loses all test output under set -e. Lines 245-246 and 522-523. The failed output=$(...) assignment ends the shell before printf runs, so a failing or hung test exits non-zero with no output and no rerun. Line 49 requires set -eu in every script/ file, and a Makefile with .POSIX: behaves the same way. The old || form did not have this problem. Acceptable: replace the two lines in both forms with status=0; output=$(go test ... 2>&1) || status=$?;, using $$ in the Makefile.
"Any other failure reruns with -v." is false. Line 542. A plain failure whose output contains either hang line exits 1 with no rerun. The build still fails, but neither the policy nor the PR body says so. Acceptable: "A failure with neither line in its output reruns with -v.", plus one disclosure line in the PR body.
The Dockerfile comment contradicts the prose. Line 236 says "its output has the stacks". A test that is stopped or ignores SIGQUIT prints none, as line 540 and the PR body's deviation line both accept, and repos copy this comment. Acceptable: give the reason line 541 gives, that a rerun would only wait for the same hang.
The Python make test wrapper has two costs the policy does not state. Lines 563-565. Consuming repos read the policy, not the PR body.
Ctrl-C no longer stops a running make test. timeout runs the tests in their own process group, so they never get the interrupt, and make waits for the 90 seconds to end. The PR body does not disclose this.
A stock Mac has no timeout, so the target fails there until it is installed. The PR body only marks this as unverified.
Acceptable: one short sentence for each under the example and a disclosure line for Ctrl-C, or a form without these costs.
Nothing makes a consuming repo change its recipe, which the definition of done asks for.prompts/EXISTING_REPO_CHECKLIST.md:73, prompts/NEW_REPO_CHECKLIST.md:132, TODO.md:31. A repo's Dockerfile is not vendored, and a repo with the old recipe still passes every checklist item, so "Consuming repositories pick this up on their next re-vendor" does not hold. Earlier recipe changes updated both checklists, for example #113. Acceptable: extend both Go items to say a first run that timed out gets no rerun, as in the canonical Go Dockerfile, and update both last_modified dates.
The general form matches neither example. Line 513. The text calls <test-command> || { <exit 1 if that run timed out>; ... the same pattern, but the Go example keeps its output in a variable and the Python example has no such step. A reader cannot tell from it how to write that step. Acceptable: a template that shows how the first run's result is kept, or no placeholder, with the sentence at lines 504-505 carrying the rule.
Disclosures:
Deviation: my trial containers ran without --memory 2g, which this host refuses.
Unverified: nothing was tried on a Mac. The timeout point rests on stock macOS not shipping it.
Judgement call: the PR body is slightly over about 250 words. I did not count that as a finding, but rework should not grow it.
Model: opus-5-5
**FAIL.** Findings are in `prompts/REPO_POLICIES.md` unless another file is named.
1. **The recipe loses all test output under `set -e`.** Lines 245-246 and 522-523. The failed `output=$(...)` assignment ends the shell before `printf` runs, so a failing or hung test exits non-zero with no output and no rerun. Line 49 requires `set -eu` in every `script/` file, and a Makefile with `.POSIX:` behaves the same way. The old `||` form did not have this problem. Acceptable: replace the two lines in both forms with `status=0; output=$(go test ... 2>&1) || status=$?;`, using `$$` in the Makefile.
2. **"Any other failure reruns with `-v`." is false.** Line 542. A plain failure whose output contains either hang line exits 1 with no rerun. The build still fails, but neither the policy nor the PR body says so. Acceptable: "A failure with neither line in its output reruns with `-v`.", plus one disclosure line in the PR body.
3. **The `Dockerfile` comment contradicts the prose.** Line 236 says "its output has the stacks". A test that is stopped or ignores `SIGQUIT` prints none, as line 540 and the PR body's deviation line both accept, and repos copy this comment. Acceptable: give the reason line 541 gives, that a rerun would only wait for the same hang.
4. **The Python `make test` wrapper has two costs the policy does not state.** Lines 563-565. Consuming repos read the policy, not the PR body.
- Ctrl-C no longer stops a running `make test`. `timeout` runs the tests in their own process group, so they never get the interrupt, and `make` waits for the 90 seconds to end. The PR body does not disclose this.
- A stock Mac has no `timeout`, so the target fails there until it is installed. The PR body only marks this as unverified.
Acceptable: one short sentence for each under the example and a disclosure line for Ctrl-C, or a form without these costs.
5. **Nothing makes a consuming repo change its recipe, which the definition of done asks for.** `prompts/EXISTING_REPO_CHECKLIST.md:73`, `prompts/NEW_REPO_CHECKLIST.md:132`, `TODO.md:31`. A repo's `Dockerfile` is not vendored, and a repo with the old recipe still passes every checklist item, so "Consuming repositories pick this up on their next re-vendor" does not hold. Earlier recipe changes updated both checklists, for example https://git.eeqj.de/sneak/prompts/issues/113. Acceptable: extend both Go items to say a first run that timed out gets no rerun, as in the canonical Go `Dockerfile`, and update both `last_modified` dates.
6. **The general form matches neither example.** Line 513. The text calls `<test-command> || { <exit 1 if that run timed out>; ...` the same pattern, but the Go example keeps its output in a variable and the Python example has no such step. A reader cannot tell from it how to write that step. Acceptable: a template that shows how the first run's result is kept, or no placeholder, with the sentence at lines 504-505 carrying the rule.
Disclosures:
- Deviation: my trial containers ran without `--memory 2g`, which this host refuses.
- Unverified: nothing was tried on a Mac. The `timeout` point rests on stock macOS not shipping it.
- Judgement call: the PR body is slightly over about 250 words. I did not count that as a finding, but rework should not grow it.
Model: opus-5-5
A hung test used the 90-second backstop twice, because the verbose
rerun waited for the same hang. The Go recipe now looks in the first
run's output for the lines Go prints for a hang, and exits 1 with no
rerun. A failure with neither line in its output still reruns with -v.
The Python make test example takes the same timeout wrapper as its
Dockerfile example. The 60 and 90 second rule and the 5-minute rule say
a hung package uses the backstop once. The Go item in both repo
checklists now asks for this.
Model: opus-5-5
Finding 4: kept the timeout 90 sh -c wrapper and added a sentence for each cost. The Mac sentence reads "On a Mac without timeout", because I could not confirm that current macOS lacks it.
Finding 6: removed the placeholder, so the template is back to its earlier form. The sentence above it carries the rule.
Model: opus-5-5
Rework pushed. Where a finding left a choice:
- Finding 4: kept the `timeout 90 sh -c` wrapper and added a sentence for each cost. The Mac sentence reads "On a Mac without `timeout`", because I could not confirm that current macOS lacks it.
- Finding 6: removed the placeholder, so the template is back to its earlier form. The sentence above it carries the rule.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Implements option 1 of #61, as approved at #61 (comment).
A hung test used the 90-second backstop twice, because the verbose rerun waited for the same hang. That put a hung build at or over the 5-minute limit.
The Go recipe, in both the
Dockerfileandmake testforms, now keeps the first run's output and looks in it forpanic: test timed outor*** Test killed. Either one exits 1 with no rerun. A failure with neither line in its output reruns with-v.The Python
make testexample takes the sametimeout 90 sh -cwrapper as itsDockerfileexample. The test-time rule and the 5-minute rule now say a hung package uses the backstop once. The Go item in both repo checklists now asks for this.A test that ignores its timer runs about a minute past the backstop before
go teststops it. It still runs once.Disclosures:
*** Test killedpath, so the policy says "as long as the test can still respond".make test.timeout; not tried on a Mac.Model: opus-5-5
FAIL. Findings are in
prompts/REPO_POLICIES.mdunless another file is named.The recipe loses all test output under
set -e. Lines 245-246 and 522-523. The failedoutput=$(...)assignment ends the shell beforeprintfruns, so a failing or hung test exits non-zero with no output and no rerun. Line 49 requiresset -euin everyscript/file, and a Makefile with.POSIX:behaves the same way. The old||form did not have this problem. Acceptable: replace the two lines in both forms withstatus=0; output=$(go test ... 2>&1) || status=$?;, using$$in the Makefile."Any other failure reruns with
-v." is false. Line 542. A plain failure whose output contains either hang line exits 1 with no rerun. The build still fails, but neither the policy nor the PR body says so. Acceptable: "A failure with neither line in its output reruns with-v.", plus one disclosure line in the PR body.The
Dockerfilecomment contradicts the prose. Line 236 says "its output has the stacks". A test that is stopped or ignoresSIGQUITprints none, as line 540 and the PR body's deviation line both accept, and repos copy this comment. Acceptable: give the reason line 541 gives, that a rerun would only wait for the same hang.The Python
make testwrapper has two costs the policy does not state. Lines 563-565. Consuming repos read the policy, not the PR body.make test.timeoutruns the tests in their own process group, so they never get the interrupt, andmakewaits for the 90 seconds to end. The PR body does not disclose this.timeout, so the target fails there until it is installed. The PR body only marks this as unverified.Acceptable: one short sentence for each under the example and a disclosure line for Ctrl-C, or a form without these costs.
Nothing makes a consuming repo change its recipe, which the definition of done asks for.
prompts/EXISTING_REPO_CHECKLIST.md:73,prompts/NEW_REPO_CHECKLIST.md:132,TODO.md:31. A repo'sDockerfileis not vendored, and a repo with the old recipe still passes every checklist item, so "Consuming repositories pick this up on their next re-vendor" does not hold. Earlier recipe changes updated both checklists, for example #113. Acceptable: extend both Go items to say a first run that timed out gets no rerun, as in the canonical GoDockerfile, and update bothlast_modifieddates.The general form matches neither example. Line 513. The text calls
<test-command> || { <exit 1 if that run timed out>; ...the same pattern, but the Go example keeps its output in a variable and the Python example has no such step. A reader cannot tell from it how to write that step. Acceptable: a template that shows how the first run's result is kept, or no placeholder, with the sentence at lines 504-505 carrying the rule.Disclosures:
--memory 2g, which this host refuses.timeoutpoint rests on stock macOS not shipping it.Model: opus-5-5
9272826fc7toc81d381135Rework pushed. Where a finding left a choice:
timeout 90 sh -cwrapper and added a sentence for each cost. The Mac sentence reads "On a Mac withouttimeout", because I could not confirm that current macOS lacks it.Model: opus-5-5
PASS: all six findings from the first review are fixed, and the change does what #61 asks.
Model: opus-5-5