Bust the Docker layer cache for the gate steps (closes #32) #37

Merged
clawbot merged 1 commits from cibuild-cache-bust into main 2026-08-09 09:47:03 +02:00
4 changed files with 92 additions and 9 deletions

View File

@@ -5,8 +5,25 @@ WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN make fmt-check
RUN make lint
# Cache-buster for the gate layers, and only for them. Docker
# invalidates COPY only when the copied content changes, so on an
# unchanged tree the gates below would be served from cache and the
# build would exit 0 having run nothing. script/cibuild and
# script/docker pass a fresh CHECK_EPOCH on every invocation.
#
# Two properties this depends on. ARG is per-stage, so the build stage
# below declares it again; one declaration here would leave that
# stage's gate cacheable. And each gate RUN must reference the value,
# because BuildKit hashes the expanded command: a declared but
# unreferenced ARG invalidates nothing.
#
# It sits below the dependency layers deliberately. Everything above it
# (the pinned base image, go mod download) keeps its cache; only the
# gates go cold.
ARG CHECK_EPOCH
RUN echo "gate fmt-check, epoch ${CHECK_EPOCH}" && make fmt-check
RUN echo "gate lint, epoch ${CHECK_EPOCH}" && make lint
# Build stage
# golang:1.25-alpine, 2026-07-23
@@ -40,7 +57,12 @@ USER builder
# Fail the build unless the branch is green. Runs as non-root so the
# permission-denied test paths are exercised legitimately (root would
# bypass the chmod(0) the tests rely on).
RUN make check
#
# Second per-stage declaration of the gate cache-buster; see the lint
# stage above for why one is not enough. It is placed after USER so the
# drop to the unprivileged user still happens before the checks run.
ARG CHECK_EPOCH
RUN echo "gate check, epoch ${CHECK_EPOCH}" && make check
RUN make build

41
TODO.md
View File

@@ -29,6 +29,47 @@
# Completed Steps
- bust the Docker layer cache for the gate steps, so `script/cibuild`
and `script/docker` cannot report a green they did not earn
(2026-08-09, branch `cibuild-cache-bust`, closes #32): both scripts
were bare `docker build` invocations with no cache control, and the
`Dockerfile` copies the tree before running its gates, so on an
unchanged tree Docker served those layers from cache and the build
exited 0 having executed nothing. That is not hypothetical here —
every merge this repo has done is a non-fast-forward merge of an
undiverged branch, so each merge commit's tree is byte-identical to
the branch head's and each merge CI run was almost certainly a full
cache hit; and PR #31's reviewer found `make docker` returning
success as a 17-layer cache hit, catching it only by being
suspicious. The fix is `ARG CHECK_EPOCH` with the scripts passing
`--build-arg CHECK_EPOCH="$(date +%s)"`. Two details make or break
it. `ARG` is scoped per stage and this `Dockerfile` has three gates
across two — `make fmt-check` and `make lint` in the lint stage,
`make check` in the build stage — so a single declaration would have
left one stage silently cacheable; it is declared in both. And
BuildKit hashes the expanded command, not the declaration, so a
declared-but-unreferenced `ARG` invalidates nothing: each gate `RUN`
echoes the epoch, which also puts the value in the build log as
evidence the layer really ran. Placement is below the dependency
layers on purpose — a build that goes cold every time would be a
different bug, not a fix. Verified by running each script twice back
to back on an unchanged tree under `BUILDKIT_PROGRESS=plain`: all
three gates executed on all four runs, each with a fresh epoch in
the log (`script/cibuild` 78.8s then 61.1s; `script/docker` 61.1s
then 53.4s), and twelve steps were still served `CACHED` in the
steady state — both `go mod download`s, `apk add`, `adduser`, the
`chown`, every `go.mod`/`go.sum` and source copy, the linter copy
out of the lint stage, and the binary copy into the runtime stage.
The lint stage still gates the build stage: with a deliberate
`unused` finding planted in the tree, the build failed at
`make lint` in 36.1s and the build-stage `make check` never started.
The build stage also still drops to the unprivileged `builder` user
before `make check`, which the suite depends on rather than merely
prefers: forcing the same image to run the tests as root fails
`TestScanHardlinkRunFailsTogether`, because root reads straight
through the `chmod(0)` the test uses to prove hard links are read
once. This is the local fix only; propagating it to the canonical
templates is `prompts` #26
- check the installed golangci-lint version in `script/bootstrap`
instead of only its presence (2026-08-09, branch
`bootstrap-version-check`, closes #24): `missing golangci-lint` meant

View File

@@ -1,14 +1,23 @@
#!/bin/sh
# script/cibuild: run the CI build. The Dockerfile runs make check (via
# script/check), so a successful build implies all checks pass. The
# Gitea workflow runs this on push.
# script/cibuild: run the CI build. The Dockerfile runs make fmt-check,
# make lint and make check (via the script/ entrypoints) as build steps,
# so a successful build implies all checks pass. The Gitea workflow runs
# this on push.
#
# That implication holds only because of CHECK_EPOCH. A COPY layer is
# invalidated by changed content, and a merge commit's tree is
# byte-identical to the branch head it merges, so without a fresh value
# here Docker serves the gate layers from cache and the build reports a
# green it never earned. Passing the current epoch invalidates the gate
# layers on every run while leaving the pinned base images and
# go mod download cached; see the Dockerfile for the placement.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
docker build .
docker build --build-arg CHECK_EPOCH="$(date +%s)" .
}
main "$@"

View File

@@ -1,6 +1,14 @@
#!/bin/sh
# script/docker: build the Docker image tagged with the project name.
# Identical in all repos; the tag comes from script/projectname.
# The tag comes from script/projectname.
#
# CHECK_EPOCH is passed for the same reason script/cibuild passes it:
# without it Docker serves the Dockerfile's gate layers from cache on an
# unchanged tree and this exits 0 having run neither the lint stage nor
# the builder stage's make check. This is the gate a developer or
# reviewer runs by hand, so a cached pass here is the most misleading
# result the repo can produce. Dependency layers sit above the ARG and
# stay cached.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -8,7 +16,10 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
docker build -t "$("$SCRIPT_DIR/projectname")" .
docker build \
--build-arg CHECK_EPOCH="$(date +%s)" \
-t "$("$SCRIPT_DIR/projectname")" \
.
}
main "$@"