Bust the Docker layer cache for the gate steps (closes #32) #37
28
Dockerfile
28
Dockerfile
@@ -5,8 +5,25 @@ WORKDIR /src
|
|||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN go mod download
|
RUN go mod download
|
||||||
COPY . .
|
COPY . .
|
||||||
RUN make fmt-check
|
|
||||||
RUN make lint
|
# Cache-buster for the gate layers, and only for them. Docker
|
||||||
|
# invalidates COPY only when the copied content changes, so on an
|
||||||
|
# unchanged tree the gates below would be served from cache and the
|
||||||
|
# build would exit 0 having run nothing. script/cibuild and
|
||||||
|
# script/docker pass a fresh CHECK_EPOCH on every invocation.
|
||||||
|
#
|
||||||
|
# Two properties this depends on. ARG is per-stage, so the build stage
|
||||||
|
# below declares it again; one declaration here would leave that
|
||||||
|
# stage's gate cacheable. And each gate RUN must reference the value,
|
||||||
|
# because BuildKit hashes the expanded command: a declared but
|
||||||
|
# unreferenced ARG invalidates nothing.
|
||||||
|
#
|
||||||
|
# It sits below the dependency layers deliberately. Everything above it
|
||||||
|
# (the pinned base image, go mod download) keeps its cache; only the
|
||||||
|
# gates go cold.
|
||||||
|
ARG CHECK_EPOCH
|
||||||
|
RUN echo "gate fmt-check, epoch ${CHECK_EPOCH}" && make fmt-check
|
||||||
|
RUN echo "gate lint, epoch ${CHECK_EPOCH}" && make lint
|
||||||
|
|
||||||
# Build stage
|
# Build stage
|
||||||
# golang:1.25-alpine, 2026-07-23
|
# golang:1.25-alpine, 2026-07-23
|
||||||
@@ -40,7 +57,12 @@ USER builder
|
|||||||
# Fail the build unless the branch is green. Runs as non-root so the
|
# Fail the build unless the branch is green. Runs as non-root so the
|
||||||
# permission-denied test paths are exercised legitimately (root would
|
# permission-denied test paths are exercised legitimately (root would
|
||||||
# bypass the chmod(0) the tests rely on).
|
# bypass the chmod(0) the tests rely on).
|
||||||
RUN make check
|
#
|
||||||
|
# Second per-stage declaration of the gate cache-buster; see the lint
|
||||||
|
# stage above for why one is not enough. It is placed after USER so the
|
||||||
|
# drop to the unprivileged user still happens before the checks run.
|
||||||
|
ARG CHECK_EPOCH
|
||||||
|
RUN echo "gate check, epoch ${CHECK_EPOCH}" && make check
|
||||||
|
|
||||||
RUN make build
|
RUN make build
|
||||||
|
|
||||||
|
|||||||
41
TODO.md
41
TODO.md
@@ -29,6 +29,47 @@
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- bust the Docker layer cache for the gate steps, so `script/cibuild`
|
||||||
|
and `script/docker` cannot report a green they did not earn
|
||||||
|
(2026-08-09, branch `cibuild-cache-bust`, closes #32): both scripts
|
||||||
|
were bare `docker build` invocations with no cache control, and the
|
||||||
|
`Dockerfile` copies the tree before running its gates, so on an
|
||||||
|
unchanged tree Docker served those layers from cache and the build
|
||||||
|
exited 0 having executed nothing. That is not hypothetical here —
|
||||||
|
every merge this repo has done is a non-fast-forward merge of an
|
||||||
|
undiverged branch, so each merge commit's tree is byte-identical to
|
||||||
|
the branch head's and each merge CI run was almost certainly a full
|
||||||
|
cache hit; and PR #31's reviewer found `make docker` returning
|
||||||
|
success as a 17-layer cache hit, catching it only by being
|
||||||
|
suspicious. The fix is `ARG CHECK_EPOCH` with the scripts passing
|
||||||
|
`--build-arg CHECK_EPOCH="$(date +%s)"`. Two details make or break
|
||||||
|
it. `ARG` is scoped per stage and this `Dockerfile` has three gates
|
||||||
|
across two — `make fmt-check` and `make lint` in the lint stage,
|
||||||
|
`make check` in the build stage — so a single declaration would have
|
||||||
|
left one stage silently cacheable; it is declared in both. And
|
||||||
|
BuildKit hashes the expanded command, not the declaration, so a
|
||||||
|
declared-but-unreferenced `ARG` invalidates nothing: each gate `RUN`
|
||||||
|
echoes the epoch, which also puts the value in the build log as
|
||||||
|
evidence the layer really ran. Placement is below the dependency
|
||||||
|
layers on purpose — a build that goes cold every time would be a
|
||||||
|
different bug, not a fix. Verified by running each script twice back
|
||||||
|
to back on an unchanged tree under `BUILDKIT_PROGRESS=plain`: all
|
||||||
|
three gates executed on all four runs, each with a fresh epoch in
|
||||||
|
the log (`script/cibuild` 78.8s then 61.1s; `script/docker` 61.1s
|
||||||
|
then 53.4s), and twelve steps were still served `CACHED` in the
|
||||||
|
steady state — both `go mod download`s, `apk add`, `adduser`, the
|
||||||
|
`chown`, every `go.mod`/`go.sum` and source copy, the linter copy
|
||||||
|
out of the lint stage, and the binary copy into the runtime stage.
|
||||||
|
The lint stage still gates the build stage: with a deliberate
|
||||||
|
`unused` finding planted in the tree, the build failed at
|
||||||
|
`make lint` in 36.1s and the build-stage `make check` never started.
|
||||||
|
The build stage also still drops to the unprivileged `builder` user
|
||||||
|
before `make check`, which the suite depends on rather than merely
|
||||||
|
prefers: forcing the same image to run the tests as root fails
|
||||||
|
`TestScanHardlinkRunFailsTogether`, because root reads straight
|
||||||
|
through the `chmod(0)` the test uses to prove hard links are read
|
||||||
|
once. This is the local fix only; propagating it to the canonical
|
||||||
|
templates is `prompts` #26
|
||||||
- check the installed golangci-lint version in `script/bootstrap`
|
- check the installed golangci-lint version in `script/bootstrap`
|
||||||
instead of only its presence (2026-08-09, branch
|
instead of only its presence (2026-08-09, branch
|
||||||
`bootstrap-version-check`, closes #24): `missing golangci-lint` meant
|
`bootstrap-version-check`, closes #24): `missing golangci-lint` meant
|
||||||
|
|||||||
@@ -1,14 +1,23 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/cibuild: run the CI build. The Dockerfile runs make check (via
|
# script/cibuild: run the CI build. The Dockerfile runs make fmt-check,
|
||||||
# script/check), so a successful build implies all checks pass. The
|
# make lint and make check (via the script/ entrypoints) as build steps,
|
||||||
# Gitea workflow runs this on push.
|
# so a successful build implies all checks pass. The Gitea workflow runs
|
||||||
|
# this on push.
|
||||||
|
#
|
||||||
|
# That implication holds only because of CHECK_EPOCH. A COPY layer is
|
||||||
|
# invalidated by changed content, and a merge commit's tree is
|
||||||
|
# byte-identical to the branch head it merges, so without a fresh value
|
||||||
|
# here Docker serves the gate layers from cache and the build reports a
|
||||||
|
# green it never earned. Passing the current epoch invalidates the gate
|
||||||
|
# layers on every run while leaving the pinned base images and
|
||||||
|
# go mod download cached; see the Dockerfile for the placement.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build .
|
docker build --build-arg CHECK_EPOCH="$(date +%s)" .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
@@ -1,6 +1,14 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/docker: build the Docker image tagged with the project name.
|
# script/docker: build the Docker image tagged with the project name.
|
||||||
# Identical in all repos; the tag comes from script/projectname.
|
# The tag comes from script/projectname.
|
||||||
|
#
|
||||||
|
# CHECK_EPOCH is passed for the same reason script/cibuild passes it:
|
||||||
|
# without it Docker serves the Dockerfile's gate layers from cache on an
|
||||||
|
# unchanged tree and this exits 0 having run neither the lint stage nor
|
||||||
|
# the builder stage's make check. This is the gate a developer or
|
||||||
|
# reviewer runs by hand, so a cached pass here is the most misleading
|
||||||
|
# result the repo can produce. Dependency layers sit above the ARG and
|
||||||
|
# stay cached.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
@@ -8,7 +16,10 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
|||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build -t "$("$SCRIPT_DIR/projectname")" .
|
docker build \
|
||||||
|
--build-arg CHECK_EPOCH="$(date +%s)" \
|
||||||
|
-t "$("$SCRIPT_DIR/projectname")" \
|
||||||
|
.
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
Reference in New Issue
Block a user