Make the CI gate execute the checks it reports on (closes #119) #138

clawbot · 2026-08-12T12:28:57+02:00

clawbot commented

2026-08-12 12:28:57 +02:00

Closes #119.

What changed

Repo-local only: .gitea/workflows/check.yml, Dockerfile, .dockerignore, .gitignore, README.md. script/cibuild is a model script shared across repos and is untouched — see "Layer decision" below.

Cache replay. The workflow writes .ci-fingerprint into the build context before calling script/cibuild. Its value is the hash of the last commit that touched the build context. Any commit that changes code — including a squash merge whose tree matches an already-built branch — gets a new value, which invalidates the COPY . . layer of both the lint and builder stages, so make fmt-check, make lint, make test and make build really execute. A docs-only commit leaves the value unchanged and the image replays from cache in seconds, which is what .dockerignore excluding *.md already intends. The go mod download layer sits above COPY . . and stays cached either way, so this is cheaper than --no-cache-filter=lint,builder, which would also redo the module download.

Note the review established a stronger property than this mechanism alone provides: because .dockerignore excludes no Go file, Docker's own COPY . . content hash already invalidates on any code change independently of the fingerprint. Verified by pinning the fingerprint to a fully-cached value and adding a failing test — the build still went red. The fingerprint is belt-and-braces, not the sole guard.

Cancelled runs recorded as failures. Gitea cancels the in-flight run when a newer commit lands on the same branch and maps StatusCancelled to CommitStatusFailure, so a commit that was never tested reads red. Both behaviours are server-side and unconditional for push events in 1.25.4, and no workflow key disables them — concurrency is not consulted on this path. The superseding run therefore rewrites the exact failure / Has been cancelled status on recent ancestor commits to skipped / "Superseded by a newer commit; never tested". Only that exact pair is matched, so a genuine failure is never overwritten. skipped is Gitea's neutral state: non-blocking in the combined-status calculation, unlike pending, which would leave those commits blocked forever.

Layer decision

Everything here is repo-local. script/cibuild is untouched by this PR and still runs a plain docker build .. Nothing in this PR is pending an org-wide decision.

Correction to an earlier version of this description, which claimed script/cibuild "stays byte-identical to the model script": it is NOT byte-identical today. Its header comment has pre-existing drift from the model, tracked at #147. The executable body IS identical, and this PR neither caused the drift nor changes it.

Baking --no-cache-filter=lint,builder into the shared script/cibuild would have been the org-wide alternative. It is not needed here, and it is strictly worse for this repo: it discards the module-download layer as well, and it removes the cheap docs-only path the issue asks to keep, since the script cannot tell a docs commit from a code commit.

Acceptance check

Run on scratch branch ci-gate-acceptance-119 (since deleted), against this exact change set.

A deliberately broken test makes CI go red. Commit 94ee42f added a test whose body is t.Fatal: https://git.eeqj.de/sneak/webhooker/actions/runs/155 — failure, "Failing after 2m8s". The lint stage ran for real (59.5s) and the test stage caught the break:
```
#16 [lint 8/8] RUN make lint
#16 58.92 0 issues.
#16 DONE 59.5s
#23 [builder  8/10] RUN make test
#23 60.23     ci_gate_probe_test.go:9: deliberate failure: the CI gate must report this commit red
#23 60.23 --- FAIL: TestCIGateRunsTests (0.00s)
ERROR: process "/bin/sh -c make test" did not complete successfully: exit code: 2
```
In the same log the COPY . . layers are DONE, not CACHED, while RUN go mod download is CACHED — the fingerprint invalidated exactly the intended layers.
A real commit runs the checks instead of replaying them. https://git.eeqj.de/sneak/webhooker/actions/runs/142 — success, "Successful in 3m57s". For comparison, next before this PR reads "Successful in 4s" and "Successful in 5s".
A docs-only commit stays cheap. https://git.eeqj.de/sneak/webhooker/actions/runs/147 — success, "Successful in 9s", every stage CACHED including RUN make lint and RUN make test.
A superseded run no longer leaves a red commit. Commit c35abbb was superseded mid-run; Gitea recorded https://git.eeqj.de/sneak/webhooker/actions/runs/149 as failure / "Has been cancelled". The next run rewrote it, combined state now success rather than failure. The same step also cleared the three false reds this issue reported on next (e50a79c, 15a6117, d51cd0f) on its first run.

Gate evidence

make check on the rebased branch: exit 0 (0 issues., all tests pass).
Containerized checks with the cache defeated, via script/cibuild after changing only .ci-fingerprint (no source change): 3m49s, with RUN make lint 79.8s, RUN make test 70.9s, RUN make build 66.2s — none CACHED.
The same script/cibuild with the fingerprint unchanged: 1.256s, 22 layers CACHED, exit 0 — the false green this issue is about, reproduced and now reachable only when the context genuinely did not change.

No cache was pruned; invalidation was scoped to the context fingerprint throughout.

Notes

No Go code changes, so there is no unit test to add; the verification is the live CI evidence above.
TODO.md is untouched, per #112.
Out of scope and untouched: #106 and #109. The lint stage still runs in Docker via the Dockerfile, as before.
A second review finding, that the status-rewrite step hardcodes its context string and so breaks silently on a workflow rename, is tracked at #147.

Closes https://git.eeqj.de/sneak/webhooker/issues/119. ## What changed Repo-local only: `.gitea/workflows/check.yml`, `Dockerfile`, `.dockerignore`, `.gitignore`, `README.md`. `script/cibuild` is a model script shared across repos and is untouched — see "Layer decision" below. **Cache replay.** The workflow writes `.ci-fingerprint` into the build context before calling `script/cibuild`. Its value is the hash of the last commit that touched the build context. Any commit that changes code — including a squash merge whose tree matches an already-built branch — gets a new value, which invalidates the `COPY . .` layer of both the `lint` and `builder` stages, so `make fmt-check`, `make lint`, `make test` and `make build` really execute. A docs-only commit leaves the value unchanged and the image replays from cache in seconds, which is what `.dockerignore` excluding `*.md` already intends. The `go mod download` layer sits above `COPY . .` and stays cached either way, so this is cheaper than `--no-cache-filter=lint,builder`, which would also redo the module download. Note the review established a stronger property than this mechanism alone provides: because `.dockerignore` excludes no Go file, Docker's own `COPY . .` content hash already invalidates on any code change independently of the fingerprint. Verified by pinning the fingerprint to a fully-cached value and adding a failing test — the build still went red. The fingerprint is belt-and-braces, not the sole guard. **Cancelled runs recorded as failures.** Gitea cancels the in-flight run when a newer commit lands on the same branch and maps `StatusCancelled` to `CommitStatusFailure`, so a commit that was never tested reads red. Both behaviours are server-side and unconditional for push events in 1.25.4, and no workflow key disables them — `concurrency` is not consulted on this path. The superseding run therefore rewrites the exact `failure` / `Has been cancelled` status on recent ancestor commits to `skipped` / "Superseded by a newer commit; never tested". Only that exact pair is matched, so a genuine failure is never overwritten. `skipped` is Gitea's neutral state: non-blocking in the combined-status calculation, unlike `pending`, which would leave those commits blocked forever. ## Layer decision Everything here is repo-local. `script/cibuild` is untouched by this PR and still runs a plain `docker build .`. Nothing in this PR is pending an org-wide decision. Correction to an earlier version of this description, which claimed `script/cibuild` "stays byte-identical to the model script": it is NOT byte-identical today. Its header comment has pre-existing drift from the model, tracked at https://git.eeqj.de/sneak/webhooker/issues/147. The executable body IS identical, and this PR neither caused the drift nor changes it. Baking `--no-cache-filter=lint,builder` into the shared `script/cibuild` would have been the org-wide alternative. It is not needed here, and it is strictly worse for this repo: it discards the module-download layer as well, and it removes the cheap docs-only path the issue asks to keep, since the script cannot tell a docs commit from a code commit. ## Acceptance check Run on scratch branch `ci-gate-acceptance-119` (since deleted), against this exact change set. 1. **A deliberately broken test makes CI go red.** Commit `94ee42f` added a test whose body is `t.Fatal`: https://git.eeqj.de/sneak/webhooker/actions/runs/155 — `failure`, "Failing after 2m8s". The lint stage ran for real (59.5s) and the test stage caught the break: ``` #16 [lint 8/8] RUN make lint #16 58.92 0 issues. #16 DONE 59.5s #23 [builder 8/10] RUN make test #23 60.23 ci_gate_probe_test.go:9: deliberate failure: the CI gate must report this commit red #23 60.23 --- FAIL: TestCIGateRunsTests (0.00s) ERROR: process "/bin/sh -c make test" did not complete successfully: exit code: 2 ``` In the same log the `COPY . .` layers are `DONE`, not `CACHED`, while `RUN go mod download` is `CACHED` — the fingerprint invalidated exactly the intended layers. 2. **A real commit runs the checks instead of replaying them.** https://git.eeqj.de/sneak/webhooker/actions/runs/142 — `success`, "Successful in 3m57s". For comparison, `next` before this PR reads "Successful in 4s" and "Successful in 5s". 3. **A docs-only commit stays cheap.** https://git.eeqj.de/sneak/webhooker/actions/runs/147 — `success`, "Successful in 9s", every stage `CACHED` including `RUN make lint` and `RUN make test`. 4. **A superseded run no longer leaves a red commit.** Commit `c35abbb` was superseded mid-run; Gitea recorded https://git.eeqj.de/sneak/webhooker/actions/runs/149 as `failure` / "Has been cancelled". The next run rewrote it, combined state now `success` rather than `failure`. The same step also cleared the three false reds this issue reported on `next` (`e50a79c`, `15a6117`, `d51cd0f`) on its first run. ## Gate evidence - `make check` on the rebased branch: exit 0 (`0 issues.`, all tests pass). - Containerized checks with the cache defeated, via `script/cibuild` after changing only `.ci-fingerprint` (no source change): 3m49s, with `RUN make lint` 79.8s, `RUN make test` 70.9s, `RUN make build` 66.2s — none `CACHED`. - The same `script/cibuild` with the fingerprint unchanged: 1.256s, 22 layers `CACHED`, exit 0 — the false green this issue is about, reproduced and now reachable only when the context genuinely did not change. No cache was pruned; invalidation was scoped to the context fingerprint throughout. ## Notes - No Go code changes, so there is no unit test to add; the verification is the live CI evidence above. - `TODO.md` is untouched, per https://git.eeqj.de/sneak/webhooker/issues/112. - Out of scope and untouched: https://git.eeqj.de/sneak/webhooker/issues/106 and https://git.eeqj.de/sneak/webhooker/issues/109. The lint stage still runs in Docker via the Dockerfile, as before. - A second review finding, that the status-rewrite step hardcodes its context string and so breaks silently on a workflow rename, is tracked at https://git.eeqj.de/sneak/webhooker/issues/147.

clawbot added 1 commit 2026-08-12 12:28:58 +02:00

Make the CI gate execute the checks it reports on (closes #119 )

check / check (push) Successful in 3m0s

Details

5ac43e6b96

The check workflow ran `script/cibuild`, a plain `docker build .`. With a
warm layer cache the lint and builder stages replayed instead of running,
so a commit could report "Successful in 4s" without being formatted,
linted, tested, or built. Squash-merging an already-built branch onto
`next` hits exactly that path, so no merge was actually validated.

The workflow now writes `.ci-fingerprint` into the build context: the
hash of the last commit that touched the context. Any commit that
changes code gets a new value, invalidates the `COPY . .` layer of both
check stages, and really runs `make fmt-check`, `make lint`, `make test`
and `make build`. A docs-only commit leaves it unchanged and still
replays from cache in seconds, as `.dockerignore` already intends. The
module download layer sits above `COPY . .` and stays cached either way,
so this is cheaper than a scoped `--no-cache-filter`. `script/cibuild`
is a model script shared across repos and is left untouched.

The second half of the problem was that Gitea cancels the in-flight run
when a newer commit lands on the same branch and records the
cancellation as `failure`, marking commits red that were never tested.
That cancellation is unconditional server-side for push events, so the
superseding run now rewrites the exact `Has been cancelled` status to
`skipped`. Genuine failures are never touched.

clawbot added the needs-review label 2026-08-12 12:29:06 +02:00

clawbot self-assigned this 2026-08-12 12:29:07 +02:00

clawbot commented

2026-08-12 12:58:50 +02:00

PASS.

Gate verified independently on the PR head, cache warmed on a baseline fingerprint first:

Fingerprint-only change: COPY . . DONE (not CACHED) in both stages; make fmt-check 4.4s, make lint 75.1s, make test 71.7s, make build 55.9s all DONE; go mod download CACHED. 219s total.
Fingerprint unchanged: 2s, exit 0, 22 layers CACHED — the false green this issue reports, reproduced, and now reachable only when the context is genuinely unchanged.
Adversarial: deliberately STALE fingerprint plus a lint-clean failing test still goes RED (make lint ran 63.2s, make test failed). Docker's own COPY . . content hash invalidates independently of the fingerprint, so a stale fingerprint cannot manufacture a green.
Commit shapes against the git log pathspec: squash merge (this repo's default style), 3-way merge with code on both sides, and a fresh branch each yield a new fingerprint; empty commits, docs-only commits, and a merge tree-identical to its branch reuse it — in every one of those the build context is byte-identical, so the replay is honest. Docs-only history returns empty and falls back to $GITHUB_SHA, i.e. a full build.

Two items to correct or track, neither blocking:

The PR body states script/cibuild "stays byte-identical to the model script". It is not. Against https://git.eeqj.de/sneak/prompts/raw/branch/main/script/cibuild only the header comment differs (script/cibuild lines 2-5 against the model's lines 2-3); the executable body is identical. The divergence is pre-existing and byte-for-byte the same on next, so it is not introduced here and does not block this PR — but the claim as written is inaccurate, and the drift deserves its own issue.
.gitea/workflows/check.yml:32 hardcodes ctx='check / check (push)'. Rename the workflow name: or the job id and the match silently stops firing, restoring the false-red behaviour with no signal at all. Deriving it as "${GITHUB_WORKFLOW} / ${GITHUB_JOB} (${GITHUB_EVENT_NAME})" would keep it self-maintaining.

Disclosures: I did not push a scratch branch — the broken-test acceptance check was reproduced locally through the same Dockerfile gate, not server-side, so the Gitea cancellation path itself is verified only from its recorded statuses. clawbot gets 403 on the Actions API, so I could not read run 155's log and do not confirm it beyond the author's report. Via the commit-status API I did confirm head 5ac43e6 is success / "Successful in 3m0s" (run 156 — a real run, not a 4s replay) and that e50a79c, 15a6117, d51cd0f now read skipped / "Superseded by a newer commit; never tested". The rewrite step fails closed: set -eu with curl -sf aborts the job if the API URL, token, or POST is bad. The skipped-folds-to-success caveat the author disclosed is confirmed and I concur with it over pending, which would block those commits permanently.

Also clean: merges into current next; base next; single commit; title ends (closes #119); no attribution trailers or vendor references; make fmt-check clean with git status clean (.ci-fingerprint correctly gitignored); TODO.md untouched; all external references hash-pinned; lint stage and COPY --from=lint /src/go.sum /dev/null intact.

PASS. Gate verified independently on the PR head, cache warmed on a baseline fingerprint first: - Fingerprint-only change: `COPY . .` DONE (not CACHED) in both stages; `make fmt-check` 4.4s, `make lint` 75.1s, `make test` 71.7s, `make build` 55.9s all DONE; `go mod download` CACHED. 219s total. - Fingerprint unchanged: 2s, exit 0, 22 layers CACHED — the false green this issue reports, reproduced, and now reachable only when the context is genuinely unchanged. - Adversarial: deliberately STALE fingerprint plus a lint-clean failing test still goes RED (`make lint` ran 63.2s, `make test` failed). Docker's own `COPY . .` content hash invalidates independently of the fingerprint, so a stale fingerprint cannot manufacture a green. - Commit shapes against the `git log` pathspec: squash merge (this repo's default style), 3-way merge with code on both sides, and a fresh branch each yield a new fingerprint; empty commits, docs-only commits, and a merge tree-identical to its branch reuse it — in every one of those the build context is byte-identical, so the replay is honest. Docs-only history returns empty and falls back to `$GITHUB_SHA`, i.e. a full build. Two items to correct or track, neither blocking: 1. The PR body states `script/cibuild` "stays byte-identical to the model script". It is not. Against https://git.eeqj.de/sneak/prompts/raw/branch/main/script/cibuild only the header comment differs (`script/cibuild` lines 2-5 against the model's lines 2-3); the executable body is identical. The divergence is pre-existing and byte-for-byte the same on `next`, so it is not introduced here and does not block this PR — but the claim as written is inaccurate, and the drift deserves its own issue. 2. `.gitea/workflows/check.yml:32` hardcodes `ctx='check / check (push)'`. Rename the workflow `name:` or the job id and the match silently stops firing, restoring the false-red behaviour with no signal at all. Deriving it as `"${GITHUB_WORKFLOW} / ${GITHUB_JOB} (${GITHUB_EVENT_NAME})"` would keep it self-maintaining. Disclosures: I did not push a scratch branch — the broken-test acceptance check was reproduced locally through the same Dockerfile gate, not server-side, so the Gitea cancellation path itself is verified only from its recorded statuses. `clawbot` gets 403 on the Actions API, so I could not read run 155's log and do not confirm it beyond the author's report. Via the commit-status API I did confirm head `5ac43e6` is `success` / "Successful in 3m0s" (run 156 — a real run, not a 4s replay) and that `e50a79c`, `15a6117`, `d51cd0f` now read `skipped` / "Superseded by a newer commit; never tested". The rewrite step fails closed: `set -eu` with `curl -sf` aborts the job if the API URL, token, or POST is bad. The `skipped`-folds-to-`success` caveat the author disclosed is confirmed and I concur with it over `pending`, which would block those commits permanently. Also clean: merges into current `next`; base `next`; single commit; title ends ` (closes #119)`; no attribution trailers or vendor references; `make fmt-check` clean with `git status` clean (`.ci-fingerprint` correctly gitignored); `TODO.md` untouched; all external references hash-pinned; lint stage and `COPY --from=lint /src/go.sum /dev/null` intact.

clawbot referenced this pull request

2026-08-12 13:00:16 +02:00

CI gate follow-ups: script/cibuild has drifted from the model script, and the status-rewrite context string is hardcoded #147

clawbot merged commit be576096aa into next

2026-08-12 13:00:52 +02:00

clawbot deleted branch issue-119-real-ci-gate

2026-08-12 13:00:52 +02:00

clawbot referenced this pull request

2026-08-12 13:06:55 +02:00

Correct release-blocking documentation inaccuracies (closes #141) #144

clawbot referenced this pull request

2026-08-12 13:07:16 +02:00