Makefile:35 runs go test ... || go test -v .... When the first pass fails, the retry frequently passes, and make test (and therefore make check and CI) reports green. Real failures are being swallowed.
Observed on next with no PR applied: first go test pass failed, retry passed. Observed again while reviewing #96 at f24e33a: docker build --no-cache-filter=lint,builder . run 1 failed with a data race in TestIntegrationTwoClients and then panic: test timed out after 30s in internal/handlers; run 2 passed clean. Build is nondeterministic, so no commit can currently be certified green.
Contributing factor: internal/handlers takes 23-30s against a hard -timeout 30s, leaving no margin.
Definition of done:
make test fails when any test fails. No || retry that can convert a red run into a green one.
The data race(s) in the ircserver integration tests are fixed (not skipped, not -race disabled).
The internal/handlers timeout margin is resolved, either by making the package faster or by a justified timeout that is not near the runtime.
Ten consecutive docker build --no-cache . runs pass on next.
`Makefile:35` runs `go test ... || go test -v ...`. When the first pass fails, the retry frequently passes, and `make test` (and therefore `make check` and CI) reports green. Real failures are being swallowed.
Observed on `next` with no PR applied: first `go test` pass failed, retry passed. Observed again while reviewing https://git.eeqj.de/sneak/neoirc/pulls/96 at `f24e33a`: `docker build --no-cache-filter=lint,builder .` run 1 failed with a data race in `TestIntegrationTwoClients` and then `panic: test timed out after 30s` in `internal/handlers`; run 2 passed clean. Build is nondeterministic, so no commit can currently be certified green.
Contributing factor: `internal/handlers` takes 23-30s against a hard `-timeout 30s`, leaving no margin.
Definition of done:
- `make test` fails when any test fails. No `||` retry that can convert a red run into a green one.
- The data race(s) in the `ircserver` integration tests are fixed (not skipped, not `-race` disabled).
- The `internal/handlers` timeout margin is resolved, either by making the package faster or by a justified timeout that is not near the runtime.
- Ten consecutive `docker build --no-cache .` runs pass on `next`.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Makefile:35runsgo test ... || go test -v .... When the first pass fails, the retry frequently passes, andmake test(and thereforemake checkand CI) reports green. Real failures are being swallowed.Observed on
nextwith no PR applied: firstgo testpass failed, retry passed. Observed again while reviewing #96 atf24e33a:docker build --no-cache-filter=lint,builder .run 1 failed with a data race inTestIntegrationTwoClientsand thenpanic: test timed out after 30sininternal/handlers; run 2 passed clean. Build is nondeterministic, so no commit can currently be certified green.Contributing factor:
internal/handlerstakes 23-30s against a hard-timeout 30s, leaving no margin.Definition of done:
make testfails when any test fails. No||retry that can convert a red run into a green one.ircserverintegration tests are fixed (not skipped, not-racedisabled).internal/handlerstimeout margin is resolved, either by making the package faster or by a justified timeout that is not near the runtime.docker build --no-cache .runs pass onnext.clawbot referenced this issue2026-09-03 15:47:57 +02:00
clawbot referenced this issue2026-09-03 17:29:49 +02:00
clawbot referenced this issue2026-09-03 22:04:18 +02:00
clawbot referenced this issue2026-09-04 08:01:08 +02:00