Makefile:35 runs go test ... || go test -v .... When the first pass fails, the retry frequently passes, and make test (and therefore make check and CI) reports green. Real failures are being swallowed.
Observed on next with no PR applied: first go test pass failed, retry passed. Observed again while reviewing #96 at f24e33a: docker build --no-cache-filter=lint,builder . run 1 failed with a data race in TestIntegrationTwoClients and then panic: test timed out after 30s in internal/handlers; run 2 passed clean. Build is nondeterministic, so no commit can currently be certified green.
Contributing factor: internal/handlers takes 23-30s against a hard -timeout 30s, leaving no margin.
Definition of done:
make test fails when any test fails. No || retry that can convert a red run into a green one.
The data race(s) in the ircserver integration tests are fixed (not skipped, not -race disabled).
The internal/handlers timeout margin is resolved, either by making the package faster or by a justified timeout that is not near the runtime.
Ten consecutive docker build --no-cache . runs pass on next.
`Makefile:35` runs `go test ... || go test -v ...`. When the first pass fails, the retry frequently passes, and `make test` (and therefore `make check` and CI) reports green. Real failures are being swallowed.
Observed on `next` with no PR applied: first `go test` pass failed, retry passed. Observed again while reviewing https://git.eeqj.de/sneak/neoirc/pulls/96 at `f24e33a`: `docker build --no-cache-filter=lint,builder .` run 1 failed with a data race in `TestIntegrationTwoClients` and then `panic: test timed out after 30s` in `internal/handlers`; run 2 passed clean. Build is nondeterministic, so no commit can currently be certified green.
Contributing factor: `internal/handlers` takes 23-30s against a hard `-timeout 30s`, leaving no margin.
Definition of done:
- `make test` fails when any test fails. No `||` retry that can convert a red run into a green one.
- The data race(s) in the `ircserver` integration tests are fixed (not skipped, not `-race` disabled).
- The `internal/handlers` timeout margin is resolved, either by making the package faster or by a justified timeout that is not near the runtime.
- Ten consecutive `docker build --no-cache .` runs pass on `next`.
Reassigned to clawbot: this issue asks no decision of sneak, only the fix in its definition of done. neoirc's main/next greening is owned by the green watch manager (sneak/project-management#20); PR 105's race fix is on next as 915f56ee, and the || retry, the internal/handlers timeout margin and the ten consecutive clean builds remain.
model: opus-5-5
Reassigned to clawbot: this issue asks no decision of sneak, only the fix in its definition of done. neoirc's `main`/`next` greening is owned by the green watch manager (https://git.eeqj.de/sneak/project-management/issues/20); PR 105's race fix is on `next` as `915f56ee`, and the `||` retry, the `internal/handlers` timeout margin and the ten consecutive clean builds remain.
model: opus-5-5
sneak
was unassigned by clawbot2026-10-02 03:03:42 +02:00
clawbot
self-assigned this 2026-10-02 03:03:42 +02:00
next (729c671) already has the race fixes and the longer test timeout from #105; Makefile line 35 still runs go test ... || go test -v .... What is left:
make test runs go test once, keeping -timeout 120s -race -cover, so a failing first pass fails make test, make check and the Docker build.
Prove the tests hold without the retry: ten consecutive docker build --no-cache-filter=lint,builder . runs on the branch. Any failure or race that shows up is fixed at its cause in this PR: no t.Skip, no dropping -race, no raising the timeout to hide it. If any package runs anywhere near 120s, make it faster or justify the timeout in one line next to it.
Commit and PR title: Run make test once so a failing test fails the build (closes #101).
Plan, for one worker, one PR against `next`:
`next` (`729c671`) already has the race fixes and the longer test timeout from https://git.eeqj.de/sneak/neoirc/pulls/105; `Makefile` line 35 still runs `go test ... || go test -v ...`. What is left:
1. `make test` runs `go test` once, keeping `-timeout 120s -race -cover`, so a failing first pass fails `make test`, `make check` and the Docker build.
2. Prove the tests hold without the retry: ten consecutive `docker build --no-cache-filter=lint,builder .` runs on the branch. Any failure or race that shows up is fixed at its cause in this PR: no `t.Skip`, no dropping `-race`, no raising the timeout to hide it. If any package runs anywhere near 120s, make it faster or justify the timeout in one line next to it.
3. Commit and PR title: `Run make test once so a failing test fails the build (closes #101)`.
Does not touch https://git.eeqj.de/sneak/neoirc/pulls/96.
Model: opus-5-5
State, paused for host memory: branch fix/101-no-test-retry (37bde0d) has step 1 of the plan above, the single go test run in make test. Steps 2 and 3 remain: the ten consecutive builder-stage runs, then the PR against next. Without the retry, next (729c671) and main (e332440) each passed their first go test run.
Model: opus-5-5
State, paused for host memory: branch `fix/101-no-test-retry` (`37bde0d`) has step 1 of the plan above, the single `go test` run in `make test`. Steps 2 and 3 remain: the ten consecutive builder-stage runs, then the PR against `next`. Without the retry, `next` (`729c671`) and `main` (`e332440`) each passed their first `go test` run.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Makefile:35runsgo test ... || go test -v .... When the first pass fails, the retry frequently passes, andmake test(and thereforemake checkand CI) reports green. Real failures are being swallowed.Observed on
nextwith no PR applied: firstgo testpass failed, retry passed. Observed again while reviewing #96 atf24e33a:docker build --no-cache-filter=lint,builder .run 1 failed with a data race inTestIntegrationTwoClientsand thenpanic: test timed out after 30sininternal/handlers; run 2 passed clean. Build is nondeterministic, so no commit can currently be certified green.Contributing factor:
internal/handlerstakes 23-30s against a hard-timeout 30s, leaving no margin.Definition of done:
make testfails when any test fails. No||retry that can convert a red run into a green one.ircserverintegration tests are fixed (not skipped, not-racedisabled).internal/handlerstimeout margin is resolved, either by making the package faster or by a justified timeout that is not near the runtime.docker build --no-cache .runs pass onnext.clawbot referenced this issue2026-09-03 15:47:57 +02:00
clawbot referenced this issue2026-09-03 17:28:55 +02:00
clawbot referenced this issue2026-09-03 17:29:49 +02:00
clawbot referenced this issue2026-09-03 22:04:18 +02:00
clawbot referenced this issue2026-09-04 08:01:08 +02:00
clawbot referenced this issue2026-10-02 01:00:43 +02:00
Reassigned to clawbot: this issue asks no decision of sneak, only the fix in its definition of done. neoirc's
main/nextgreening is owned by the green watch manager (sneak/project-management#20); PR 105's race fix is onnextas915f56ee, and the||retry, theinternal/handlerstimeout margin and the ten consecutive clean builds remain.model: opus-5-5
Plan, for one worker, one PR against
next:next(729c671) already has the race fixes and the longer test timeout from #105;Makefileline 35 still runsgo test ... || go test -v .... What is left:make testrunsgo testonce, keeping-timeout 120s -race -cover, so a failing first pass failsmake test,make checkand the Docker build.docker build --no-cache-filter=lint,builder .runs on the branch. Any failure or race that shows up is fixed at its cause in this PR: not.Skip, no dropping-race, no raising the timeout to hide it. If any package runs anywhere near 120s, make it faster or justify the timeout in one line next to it.Run make test once so a failing test fails the build (closes #101).Does not touch #96.
Model: opus-5-5
State, paused for host memory: branch
fix/101-no-test-retry(37bde0d) has step 1 of the plan above, the singlego testrun inmake test. Steps 2 and 3 remain: the ten consecutive builder-stage runs, then the PR againstnext. Without the retry,next(729c671) andmain(e332440) each passed their firstgo testrun.Model: opus-5-5