Bust the Docker check-layer cache with a per-invocation CHECK_EPOCH (closes #26)
All checks were successful
check / check (push) Successful in 14s
All checks were successful
check / check (push) Successful in 14s
script/cibuild was a plain `docker build .`, and the Dockerfile does `COPY . .` followed by `RUN make check`. Docker invalidates a COPY layer only when the copied content changes, so on an unchanged tree the check layer was served from cache, the suite never ran, and the build still exited 0. Measured here: run 1 took 18.5s and ran the suite; run 2 on a byte-identical tree took 0.286s with `RUN make check` CACHED. script/cibuild and script/docker now assign a per-invocation nonce on its own line and pass it as --build-arg CHECK_EPOCH. The Dockerfile declares ARG CHECK_EPOCH, guards it with `[ -n "$CHECK_EPOCH" ] || exit 1`, and expands it into the check command. Post-fix, two consecutive runs both execute make check (17.4s / 8.1s) with `RUN script/bootstrap` still CACHED, so dependency layers are untouched and the build ceiling is not at risk. The guard is what makes a bare `docker build .` — the command REPO_POLICIES named verbatim — fail closed rather than reuse the empty and therefore stable cache key; verified failing in 0.455s. Holding the epoch constant restores the false green (run 2 fully CACHED), which pins the varying value as the operative mechanism rather than a coincidence. REPO_POLICIES.md carried the false guarantee as org-canonical text in two places, and its Go multistage template had check steps in two stages; ARG is stage-scoped, so both stages get the treatment or the fleet inherits the half-fixed shape.
This commit is contained in:
13
Dockerfile
13
Dockerfile
@@ -12,4 +12,15 @@ COPY package.json yarn.lock ./
|
|||||||
RUN script/bootstrap
|
RUN script/bootstrap
|
||||||
|
|
||||||
COPY . .
|
COPY . .
|
||||||
RUN make check
|
|
||||||
|
# CHECK_EPOCH is a per-invocation nonce supplied by script/cibuild and
|
||||||
|
# script/docker. Without it an unchanged tree serves this layer from
|
||||||
|
# cache and the build reports a green it never ran. ARG is stage-scoped,
|
||||||
|
# so it must be redeclared in every stage that runs checks. The guard
|
||||||
|
# makes a bare `docker build .` fail loudly instead of silently reusing
|
||||||
|
# the empty (and therefore stable) cache key. Expand the value into the
|
||||||
|
# command so the cache miss does not depend on BuildKit's handling of an
|
||||||
|
# unreferenced ARG.
|
||||||
|
ARG CHECK_EPOCH
|
||||||
|
RUN [ -n "$CHECK_EPOCH" ] || exit 1
|
||||||
|
RUN echo "check epoch: ${CHECK_EPOCH}" && make check
|
||||||
|
|||||||
@@ -124,8 +124,11 @@ alpine. We provide:
|
|||||||
extension)
|
extension)
|
||||||
- `script/docker` — build the Docker image, tagged via `script/projectname`
|
- `script/docker` — build the Docker image, tagged via `script/projectname`
|
||||||
(byte-identical across repos)
|
(byte-identical across repos)
|
||||||
- `script/cibuild` — cd to the repo root and `docker build .` (what CI runs; the
|
- `script/cibuild` — cd to the repo root and
|
||||||
image build runs `script/check`)
|
`docker build --build-arg CHECK_EPOCH="$epoch" .` (what CI runs; the image
|
||||||
|
build runs `script/check`, and the per-invocation `CHECK_EPOCH` nonce is what
|
||||||
|
stops Docker serving that check from cache on an unchanged tree — a bare
|
||||||
|
`docker build .` fails closed on purpose)
|
||||||
- `script/precommit` — run by the git pre-commit hook (our own extension); calls
|
- `script/precommit` — run by the git pre-commit hook (our own extension); calls
|
||||||
`script/check`
|
`script/check`
|
||||||
- `script/install-precommit` — installs the git pre-commit hook (our own
|
- `script/install-precommit` — installs the git pre-commit hook (our own
|
||||||
|
|||||||
6
TODO.md
6
TODO.md
@@ -21,6 +21,12 @@ fmt-check, and commit.
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-08-09: Fixed the false green in the canonical CI gate: `script/cibuild`
|
||||||
|
and `script/docker` now pass a per-invocation `CHECK_EPOCH` nonce, and the
|
||||||
|
`Dockerfile` (plus the Go multistage template in REPO_POLICIES.md, in both its
|
||||||
|
lint and builder stages) declares `ARG CHECK_EPOCH` with a guard that makes a
|
||||||
|
bare `docker build .` fail closed. Corrected the org-canonical text that
|
||||||
|
asserted a successful build implies all checks pass.
|
||||||
- 2026-08-07: Set the canonical `.golangci.yml` to the org-standard v2-schema
|
- 2026-08-07: Set the canonical `.golangci.yml` to the org-standard v2-schema
|
||||||
config already deployed byte-identical across the org's Go repos (settings
|
config already deployed byte-identical across the org's Go repos (settings
|
||||||
under `linters.settings` so thresholds like lll/funlen/cyclop/dupl actually
|
under `linters.settings` so thresholds like lll/funlen/cyclop/dupl actually
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
title: Repository Policies
|
title: Repository Policies
|
||||||
last_modified: 2026-08-07
|
last_modified: 2026-08-09
|
||||||
---
|
---
|
||||||
|
|
||||||
This document covers repository structure, tooling, and workflow standards. Code
|
This document covers repository structure, tooling, and workflow standards. Code
|
||||||
@@ -60,17 +60,19 @@ style conventions are in separate documents:
|
|||||||
prerequisite since nvm requires bash. yarn is then pinned via
|
prerequisite since nvm requires bash. yarn is then pinned via
|
||||||
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts";
|
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts";
|
||||||
always exact versions. `script/cibuild` runs the CI build: it changes to the
|
always exact versions. `script/cibuild` runs the CI build: it changes to the
|
||||||
repo root and runs `docker build .`; the Gitea workflow calls it. Four further
|
repo root and runs `docker build --build-arg CHECK_EPOCH="$epoch" .`, where
|
||||||
scripts are our own extensions to the standard: `script/check` runs
|
`epoch` is a per-invocation nonce (see the `CHECK_EPOCH` rule below); the
|
||||||
`script/test`, `script/lint`, and `script/fmt-check`; `script/precommit` is
|
Gitea workflow calls it. Four further scripts are our own extensions to the
|
||||||
what the git pre-commit hook runs, and it calls `script/check`;
|
standard: `script/check` runs `script/test`, `script/lint`, and
|
||||||
`script/install-precommit` installs the git pre-commit hook (the `make hooks`
|
`script/fmt-check`; `script/precommit` is what the git pre-commit hook runs,
|
||||||
target shims to it); and `script/projectname` (literally that filename) simply
|
and it calls `script/check`; `script/install-precommit` installs the git
|
||||||
outputs the project's name. Scripts that need the name call
|
pre-commit hook (the `make hooks` target shims to it); and
|
||||||
`script/projectname` — e.g. `script/docker` assembles its image tag from it —
|
`script/projectname` (literally that filename) simply outputs the project's
|
||||||
so those scripts stay byte-identical across all repos. Repo-type-specific
|
name. Scripts that need the name call `script/projectname` — e.g.
|
||||||
pre-commit extras (e.g. `go mod tidy` verification in Go repos) belong in
|
`script/docker` assembles its image tag from it — so those scripts stay
|
||||||
`script/precommit`, not in the hook itself. Model scripts are at
|
byte-identical across all repos. Repo-type-specific pre-commit extras (e.g.
|
||||||
|
`go mod tidy` verification in Go repos) belong in `script/precommit`, not in
|
||||||
|
the hook itself. Model scripts are at
|
||||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
|
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
|
||||||
must document the provided scripts in an **Entrypoints** section (see the
|
must document the provided scripts in an **Entrypoints** section (see the
|
||||||
README requirements below).
|
README requirements below).
|
||||||
@@ -99,6 +101,51 @@ style conventions are in separate documents:
|
|||||||
`yarn.lock`, `go.mod` + `go.sum`, etc.) before running it so the bootstrap
|
`yarn.lock`, `go.mod` + `go.sum`, etc.) before running it so the bootstrap
|
||||||
layer stays cached until dependencies change.
|
layer stays cached until dependencies change.
|
||||||
|
|
||||||
|
- **Every check-running `RUN` must be cache-busted with `CHECK_EPOCH`.** Docker
|
||||||
|
invalidates a `COPY` layer only when the copied content changes, so on an
|
||||||
|
unchanged tree the `RUN make check` layer is served from cache, the suite
|
||||||
|
never runs, and the build still exits 0. A sub-second `docker build` reporting
|
||||||
|
success is a cache hit, not a result. The canonical form, in **every** stage
|
||||||
|
containing a check-running `RUN`:
|
||||||
|
|
||||||
|
```dockerfile
|
||||||
|
ARG CHECK_EPOCH
|
||||||
|
RUN [ -n "$CHECK_EPOCH" ] || exit 1
|
||||||
|
RUN echo "check epoch: ${CHECK_EPOCH}" && make check
|
||||||
|
```
|
||||||
|
|
||||||
|
and in both `script/cibuild` and `script/docker`:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
epoch="$(date +%s%N)$$"
|
||||||
|
docker build --build-arg CHECK_EPOCH="$epoch" .
|
||||||
|
```
|
||||||
|
|
||||||
|
All four elements are load-bearing; none is optional, and each guards a
|
||||||
|
failure mode that otherwise fails green:
|
||||||
|
- `ARG` is stage-scoped, so a single declaration leaves the other check
|
||||||
|
stages frozen while the fix reviews as complete. Declare it in every stage
|
||||||
|
that runs checks, immediately above the first such `RUN`.
|
||||||
|
- Expand the value into the command. This makes the cache miss contractual
|
||||||
|
rather than dependent on BuildKit's handling of an unreferenced `ARG`, and
|
||||||
|
it puts the epoch in the build log.
|
||||||
|
- The `[ -n ... ]` guard is required: an unset `ARG` is empty, and empty is
|
||||||
|
a stable cache key, so without it a bare `docker build .` still produces
|
||||||
|
the false green. Failed steps are never cached, so the guard fails on
|
||||||
|
every such invocation, loudly. A bare `docker build .` failing is by
|
||||||
|
design.
|
||||||
|
- Assign `epoch=` on its own line, never inline in the `--build-arg`
|
||||||
|
argument: a failing command substitution inside an argument does not trip
|
||||||
|
`set -e`, so the inline form silently degrades to an empty constant. The
|
||||||
|
`$$` suffix is required because busybox `date` drops `%N` and exits 0, so
|
||||||
|
on an alpine host the epoch would degrade to second granularity and
|
||||||
|
concurrent invocations would collide.
|
||||||
|
|
||||||
|
This invalidates the check layers and everything after them while leaving
|
||||||
|
`go mod download`, `script/bootstrap`, and the pinned toolchain install
|
||||||
|
cached, so it does not push against the five-minute Docker build ceiling.
|
||||||
|
Blanket `--no-cache` also works but is wasteful and can blow that ceiling.
|
||||||
|
|
||||||
- **Dockerfiles must use a separate lint stage for fail-fast feedback.** Go
|
- **Dockerfiles must use a separate lint stage for fail-fast feedback.** Go
|
||||||
repos use a multistage build where linting runs in an independent stage based
|
repos use a multistage build where linting runs in an independent stage based
|
||||||
on the `golangci/golangci-lint` image (pinned by hash). This stage runs
|
on the `golangci/golangci-lint` image (pinned by hash). This stage runs
|
||||||
@@ -119,7 +166,9 @@ style conventions are in separate documents:
|
|||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN go mod download
|
RUN go mod download
|
||||||
COPY . .
|
COPY . .
|
||||||
RUN make fmt-check
|
ARG CHECK_EPOCH
|
||||||
|
RUN [ -n "$CHECK_EPOCH" ] || exit 1
|
||||||
|
RUN echo "check epoch: ${CHECK_EPOCH}" && make fmt-check
|
||||||
RUN make lint
|
RUN make lint
|
||||||
|
|
||||||
# Build stage
|
# Build stage
|
||||||
@@ -133,7 +182,9 @@ style conventions are in separate documents:
|
|||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN go mod download
|
RUN go mod download
|
||||||
COPY . .
|
COPY . .
|
||||||
RUN make test
|
ARG CHECK_EPOCH
|
||||||
|
RUN [ -n "$CHECK_EPOCH" ] || exit 1
|
||||||
|
RUN echo "check epoch: ${CHECK_EPOCH}" && make test
|
||||||
|
|
||||||
ARG VERSION=dev
|
ARG VERSION=dev
|
||||||
RUN CGO_ENABLED=0 go build -trimpath \
|
RUN CGO_ENABLED=0 go build -trimpath \
|
||||||
@@ -165,11 +216,26 @@ style conventions are in separate documents:
|
|||||||
- The build stage runs `make test` after compilation setup. Tests run in the
|
- The build stage runs `make test` after compilation setup. Tests run in the
|
||||||
build stage, not the lint stage, because they may require compiled
|
build stage, not the lint stage, because they may require compiled
|
||||||
artifacts or heavier dependencies.
|
artifacts or heavier dependencies.
|
||||||
|
- `ARG CHECK_EPOCH` appears in **both** stages, because `ARG` is
|
||||||
|
stage-scoped: declaring it only in the lint stage leaves `make test`
|
||||||
|
frozen at the last cached result. In each stage the guard sits immediately
|
||||||
|
below the `ARG` so a bare `docker build .` fails instead of reusing the
|
||||||
|
empty cache key, and the value is expanded into the first check `RUN` so
|
||||||
|
the cache miss does not rely on BuildKit's unreferenced-`ARG` handling.
|
||||||
|
The later `RUN`s in the same stage need no expansion of their own: they
|
||||||
|
are already invalidated by their busted parent layer.
|
||||||
|
|
||||||
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
|
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
|
||||||
runs `script/cibuild` (which runs `docker build .`) on push. Since the
|
runs `script/cibuild` (which runs
|
||||||
Dockerfile already runs `make check`, a successful build implies all checks
|
`docker build --build-arg CHECK_EPOCH="$epoch" .`) on push. The Dockerfile
|
||||||
pass.
|
runs `make check`, so a successful build implies all checks pass — but that
|
||||||
|
implication holds **only** because of the `CHECK_EPOCH` cache-bust described
|
||||||
|
above. Without it, an unchanged tree serves the check layer from cache and the
|
||||||
|
build reports a green it never earned. A bare `docker build .` fails closed by
|
||||||
|
design, on the `[ -n "$CHECK_EPOCH" ]` guard; always go through
|
||||||
|
`script/cibuild` or `script/docker`. Never accept a `script/cibuild` pass as
|
||||||
|
evidence without confirming it ran: a sub-second wall time, or `CACHED` on the
|
||||||
|
check layer, means nothing was executed.
|
||||||
|
|
||||||
- Use platform-standard formatters: `black` for Python, `prettier` for
|
- Use platform-standard formatters: `black` for Python, `prettier` for
|
||||||
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
|
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
|
||||||
|
|||||||
@@ -1,13 +1,20 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/cibuild: run the CI build. The Dockerfile runs script/check, so
|
# script/cibuild: run the CI build. The Dockerfile runs script/check, but
|
||||||
# a successful build implies all checks pass.
|
# that only proves anything because CHECK_EPOCH is a fresh nonce on every
|
||||||
|
# invocation: without it Docker serves the check layer from cache on an
|
||||||
|
# unchanged tree and the build exits 0 without running the suite.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build .
|
# Assign on its own line: a failing command substitution inside an
|
||||||
|
# argument does not trip `set -e`, which would silently degrade the
|
||||||
|
# nonce to an empty constant. `$$` is required because busybox `date`
|
||||||
|
# drops %N without erroring.
|
||||||
|
epoch="$(date +%s%N)$$"
|
||||||
|
docker build --build-arg CHECK_EPOCH="$epoch" .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
@@ -1,6 +1,9 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/docker: build the Docker image tagged with the project name.
|
# script/docker: build the Docker image tagged with the project name.
|
||||||
# Identical in all repos; the tag comes from script/projectname.
|
# Identical in all repos; the tag comes from script/projectname. The
|
||||||
|
# Dockerfile's checks only actually run because CHECK_EPOCH is a fresh
|
||||||
|
# nonce on every invocation; without it a warm cache turns this into a
|
||||||
|
# green that proves nothing.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
@@ -8,7 +11,13 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
|||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build -t "$("$SCRIPT_DIR/projectname")" .
|
# Assign on its own line: a failing command substitution inside an
|
||||||
|
# argument does not trip `set -e`, which would silently degrade the
|
||||||
|
# nonce to an empty constant. `$$` is required because busybox `date`
|
||||||
|
# drops %N without erroring.
|
||||||
|
epoch="$(date +%s%N)$$"
|
||||||
|
docker build --build-arg CHECK_EPOCH="$epoch" \
|
||||||
|
-t "$("$SCRIPT_DIR/projectname")" .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
Reference in New Issue
Block a user