Keep modules, lint cache and system packages in check builds (closes #109)

Re-vendors REPO_POLICIES.md, script/cibuild, script/docker, script/lint
and script/test from sneak/prompts next at 0653579, on sneak's ruling in
sneak/prompts#130. In the lint, test and
builder stages of the Dockerfile, the module download, and in builder
the apk install, move above ARG CACHEBUST, so Docker keeps those layers
until the base image, the install line, go.mod or go.sum changes.
golangci-lint run also mounts golangci-lint's cache. Every step that
lints, tests or compiles still comes after ARG CACHEBUST, and go test
keeps -count=1 and -race. README and TODO.md say the same.

Model: opus-5-5

Dockerfile

@@ -1,16 +1,23 @@
-# Every stage declares ARG CACHEBUST first: script/ passes a new value
+# In every stage ARG CACHEBUST comes right after the steps that install
-# on every build, which reruns every RUN step after it. The steps that
+# system packages and download Go's modules, or right after FROM in a
-# compile mount Go's build cache, which survives from build to build.
+# stage with neither, and every step that lints, tests or compiles
+# comes after it. script/ passes a new value on every build, which
+# reruns every RUN step after it. Docker keeps the layers above it
+# until the base image, their own lines or go.mod and go.sum change.
+# The steps that compile mount Go's build cache, and golangci-lint run
+# also mounts golangci-lint's; both survive from build to build. The
+# test phase runs as nobody and so mounts Go's build cache at
+# /tmp/.cache/go-build; its own comment says why.
 
 # Lint phase, built alone by script/lint. The tools are invoked directly
 # rather than through `make lint`, which runs docker itself and so cannot
 # run inside a build step.
 # golangci/golangci-lint:v2.14.0, 2026-10-07
 FROM golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint
-ARG CACHEBUST
 WORKDIR /src
 COPY go.mod go.sum ./
 RUN go mod download
+ARG CACHEBUST
 COPY . .
 
 # The gofmt half of `make fmt-check`. gofmt's output is assigned to a
@@ -24,6 +31,7 @@ RUN files="$(gofmt -s -l .)" && \
 # Validates .golangci.yml against the schema the pinned binary embeds.
 RUN golangci-lint config verify --config .golangci.yml
 RUN --mount=type=cache,target=/root/.cache/go-build \
+    --mount=type=cache,target=/root/.cache/golangci-lint \
     golangci-lint run --config .golangci.yml ./...
 
 # Test phase, built alone by script/test. -race needs cgo and so a C
@@ -38,12 +46,12 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
 # nobody (65534) so the tests can write to it.
 # golang:1.25-trixie, 2026-10-04
 FROM golang@sha256:2c4c60ef415fbfa5e90300722293bef36c5e63fae17570ce18f580af933dbd73 AS test
-ARG CACHEBUST
 USER nobody
 ENV HOME=/tmp
 WORKDIR /src
 COPY go.mod go.sum ./
 RUN go mod download
+ARG CACHEBUST
 COPY . .
 RUN --mount=type=cache,target=/tmp/.cache/go-build,uid=65534,gid=65534 \
     go test -count=1 -timeout 90s -race -cover ./... || \
@@ -51,19 +59,19 @@ RUN --mount=type=cache,target=/tmp/.cache/go-build,uid=65534,gid=65534 \
       go test -count=1 -timeout 90s -race -v ./...; exit 1; }
 
 # Build stage. Nothing is wanted from either phase above; the copies are
-# what make BuildKit build them first, so this stage cannot run unless
+# what make BuildKit build them first, so nothing after them can run
-# lint and test passed.
+# unless lint and test passed.
 # golang:1.25-alpine, 2026-07-23
 FROM golang@sha256:56961d79ea8129efddcc0b8643fd8a5416b4e6228cfd477e3fd61deb2672c587 AS builder
-ARG CACHEBUST
-COPY --from=lint /src/go.sum /dev/null
-COPY --from=test /src/go.sum /dev/null
 RUN apk add --no-cache git make
-# A tar-stream context keeps the sender's file owners, which git refuses.
-RUN git config --system --add safe.directory /src
 WORKDIR /src
 COPY go.mod go.sum ./
 RUN go mod download
+ARG CACHEBUST
+COPY --from=lint /src/go.sum /dev/null
+COPY --from=test /src/go.sum /dev/null
+# A tar-stream context keeps the sender's file owners, which git refuses.
+RUN git config --system --add safe.directory /src
 COPY . .
 
 # The version stamped into the binary: the VERSION build argument when

README.md

@@ -1125,13 +1125,21 @@ entrypoints are:
 - `script/install-precommit` — install the git pre-commit hook, as
   `.git/hooks/pre-commit`, that runs `script/precommit`.
 
-Every `docker build` in `script/` passes a new `CACHEBUST` build argument, which
+Every `docker build` in `script/` passes a new `CACHEBUST` build argument. In
-every stage of the `Dockerfile` declares first, so every `RUN` step runs again
+every stage of the `Dockerfile`, `ARG CACHEBUST` comes right after the steps
-each time. On an unchanged tree Docker would otherwise serve the gate steps from
+that install system packages and download the Go modules, or right after `FROM`
-its cache, and the build would pass having run no test and no linter. Go's build
+in a stage with neither, and every step that lints, tests or compiles comes
-cache is kept between builds on the steps that compile, so only what changed is
+after it, so every `RUN` step after it runs again each time. On an unchanged
-compiled again; the tests run with `-count=1`, so none reports a stored result.
+tree Docker would otherwise serve the gate steps from its cache, and the build
-Every run downloads the Go modules again and needs the network.
+would pass having run no test and no linter. The layers above that line stay in
+Docker's build cache until the base image, their own lines, `go.mod` or `go.sum`
+change, or until that cache is cleared; only then are the packages installed and
+the modules downloaded again. Go's build cache is kept between builds on the
+steps that compile, so only what changed is compiled again; the tests run with
+`-count=1`, so none reports a stored result. `golangci-lint run` also keeps
+golangci-lint's cache, and takes a package's findings from it only when the
+package's files and those of the packages it imports, `go.mod`, `.golangci.yml`
+and the golangci-lint version are all unchanged.
 
 ## Build
 

REPO_POLICIES.md

@@ -1,6 +1,6 @@
 ---
 title: Repository Policies
-last_modified: 2026-10-15
+last_modified: 2026-10-18
 ---
 
 This document covers repository structure, tooling, and workflow standards. Code
@@ -159,8 +159,11 @@ style conventions are in separate documents:
     one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs
     exit non-zero with `parallel golangci-lint is running` — a status a caller
     cannot tell from real findings. Both have produced wrong verdicts in this
-    org, in both directions. A container has its own golangci-lint cache, its
+    org, in both directions. A container has its own `TMPDIR`, so the lock is
-    own `TMPDIR` and a digest-pinned binary, so neither is reachable.
+    out of reach. The lint phase does keep golangci-lint's cache between check
+    builds (below), but it lints every checkout at `/src`, and a finding is
+    taken from that cache only for the same files, settings and golangci-lint
+    version.
 
 - **Any build that runs checks is given a new `CACHEBUST` value.** Docker
   invalidates a `COPY` layer only when the copied content changes, so on an
@@ -170,15 +173,21 @@ style conventions are in separate documents:
   id, so no two builds share a value, not even the three in one
   `script/cibuild`. `script/lint`, `script/test`, `script/cibuild` and
   `script/docker` are the four, and there is no fifth — `script/check` runs the
-  two gate phases and `script/fmt-check`, and builds no image of its own. Every
+  two gate phases and `script/fmt-check`, and builds no image of its own. In
-  stage of the `Dockerfile` declares `ARG CACHEBUST` as its first line after
+  every stage of the `Dockerfile`, `ARG CACHEBUST` comes right after the steps
-  `FROM`, and a new value reruns every `RUN` step after that line; `COPY` steps
+  that install system packages and download Go's modules, or right after `FROM`
-  are compared by content and need nothing. A stage without the `ARG` line is
+  in a stage with neither, and every step that lints, tests or compiles comes
-  served from cache on an unchanged tree, so a repository re-vendors the scripts
+  after it. A step that installs anything else as well, such as a
-  and adds the `ARG` lines in one commit. The scripts do not pass `--no-cache`:
+  `script/bootstrap` that also installs node packages, comes after it too. A new
-  it reruns every step too, but it also empties every cache mount, and Go's
+  value reruns every `RUN` step after that line; `COPY` steps are compared by
-  build cache, which the canonical Go `Dockerfile` below mounts so that compiled
+  content and need nothing. Docker keeps the layers above the line until the
-  packages survive between check builds, would start empty every time. A bare
+  base image, their own lines or the files they copy change. A stage without the
+  `ARG` line is served from cache on an unchanged tree, so a repository
+  re-vendors the scripts and adds the `ARG` lines in one commit. The scripts do
+  not pass `--no-cache`: it reruns every step too, but it also builds the layers
+  above the line again and empties every cache mount, so Go's build cache and
+  golangci-lint's, which the canonical Go `Dockerfile` below mounts so that they
+  survive between check builds, would start empty every time. A bare
   `docker build .` is not evidence that anything ran: a sub-second build
   reporting success is a cache hit, not a result. Never invalidate by pruning —
   `docker builder prune` and friends destroy a build cache shared with every
@@ -194,19 +203,25 @@ style conventions are in separate documents:
   `Dockerfile`:
 
     ```dockerfile
-    # Every stage declares ARG CACHEBUST first: script/ passes a new value
+    # In every stage ARG CACHEBUST comes right after the steps that install
-    # on every build, which reruns every RUN step after it. The steps that
+    # system packages and download Go's modules, or right after FROM in a
-    # compile mount Go's build cache, which survives from build to build.
+    # stage with neither, and every step that lints, tests or compiles
+    # comes after it. script/ passes a new value on every build, which
+    # reruns every RUN step after it. Docker keeps the layers above it
+    # until the base image, their own lines or go.mod and go.sum change.
+    # The steps that compile mount Go's build cache, and golangci-lint run
+    # also mounts golangci-lint's; both survive from build to build.
 
     # Lint phase
     # golangci/golangci-lint:v2.x.x, YYYY-MM-DD
     FROM golangci/golangci-lint@sha256:... AS lint
-    ARG CACHEBUST
     WORKDIR /src
     COPY go.mod go.sum ./
     RUN go mod download
+    ARG CACHEBUST
     COPY . .
     RUN --mount=type=cache,target=/root/.cache/go-build \
+        --mount=type=cache,target=/root/.cache/golangci-lint \
         golangci-lint run --config .golangci.yml ./...
 
     # Test phase. -race needs cgo and so a C compiler, which the Debian Go
@@ -214,10 +229,10 @@ style conventions are in separate documents:
     # cache also holds test results, which go test would otherwise replay.
     # golang:1.x, YYYY-MM-DD
     FROM golang@sha256:... AS test
-    ARG CACHEBUST
     WORKDIR /src
     COPY go.mod go.sum ./
     RUN go mod download
+    ARG CACHEBUST
     COPY . .
     RUN --mount=type=cache,target=/root/.cache/go-build \
         go test -count=1 -timeout 90s -race -cover ./... || \
@@ -225,19 +240,19 @@ style conventions are in separate documents:
           go test -count=1 -timeout 90s -race -v ./...; exit 1; }
 
     # Build stage. Nothing is wanted from either phase above; the copies
-    # are what make BuildKit build them first, so this stage cannot run
+    # are what make BuildKit build them first, so nothing after them can
-    # unless lint and test passed.
+    # run unless lint and test passed.
     # golang:1.x-alpine, YYYY-MM-DD
     FROM golang@sha256:... AS builder
-    ARG CACHEBUST
-    COPY --from=lint /src/go.sum /dev/null
-    COPY --from=test /src/go.sum /dev/null
     RUN apk add --no-cache git
-    # A tar-stream context keeps the sender's file owners, which git refuses.
-    RUN git config --system --add safe.directory /src
     WORKDIR /src
     COPY go.mod go.sum ./
     RUN go mod download
+    ARG CACHEBUST
+    COPY --from=lint /src/go.sum /dev/null
+    COPY --from=test /src/go.sum /dev/null
+    # A tar-stream context keeps the sender's file owners, which git refuses.
+    RUN git config --system --add safe.directory /src
     COPY . .
 
     # The VERSION build arg when one is given, otherwise
@@ -274,16 +289,26 @@ style conventions are in separate documents:
     - Keep the runtime stage last, and if you add a stage after it, give it the
       same two copies. A plain `docker build .` builds the last stage's chain
       and nothing else.
-    - Only Go's build cache is mounted (`/root/.cache/go-build`, where Go keeps
+    - Check builds keep these, and none of them can hand back a stale result:
-      it for root), and only on the steps that compile: `golangci-lint run`,
+        - The layers above `ARG CACHEBUST`: the system packages and Go's
-      both `go test` runs and `go build`. It keeps compiled packages between
+          modules. Docker installs the packages again when the base image or the
-      check builds, and Go reuses one only when all its inputs are unchanged, so
+          install line changes, and downloads the modules again when `go.mod` or
-      it saves the compile without skipping a check. Nothing else is kept:
+          `go.sum` changes. Go checks every module against `go.sum` as it
-      golangci-lint's own cache would replay stored findings in place of a lint
+          downloads it, and no later build can change a kept layer. The modules
-      run, and Go builds from whatever files a kept module cache holds without
+          are kept in a layer rather than a cache mount because a cache mount is
-      checking them again (only `go mod verify` does), so a file changed there
+          shared by every build on the host, and Go builds from whatever files
-      would go unnoticed. Go's build cache also holds test results, which is why
+          it holds without checking them again (only `go mod verify` does), so a
-      both `go test` runs carry `-count=1`.
+          file changed there would go unnoticed.
+        - Go's build cache (`/root/.cache/go-build`, where Go keeps it for
+          root), mounted on the steps that compile: `golangci-lint run`, both
+          `go test` runs and `go build`. Go reuses a compiled package only when
+          all its inputs are unchanged. The cache also holds test results, which
+          is why both `go test` runs carry `-count=1`.
+        - golangci-lint's cache (`/root/.cache/golangci-lint`, where the pinned
+          image keeps it), mounted on `golangci-lint run`. golangci-lint reuses
+          a package's findings only when the package's files and those of every
+          package it imports, `go.mod`, the linters and their settings in
+          `.golangci.yml`, and the golangci-lint version are all unchanged.
     - If the project uses `//go:embed` directives that reference build artifacts
       (e.g. a web frontend compiled in a separate stage), the lint phase must
       create placeholder files so the embed directives resolve. Example:
@@ -292,12 +317,20 @@ style conventions are in separate documents:
       in the lint phase. The `golangci/golangci-lint` image is Debian-based and
       has no `apk`, so install with `apt-get` under the Debian package name
       (`libvips-dev`, where alpine says `vips-dev`), and delete the package
-      lists in the same `RUN`, so the layer does not keep them:
+      lists in the same `RUN`, so the layer does not keep them. The test phase,
+      on the Debian Go image, installs the libraries its tests need the same
+      way. Either install goes right after `FROM`, above `ARG CACHEBUST`, so
+      Docker keeps it until the base image or the install line changes:
 
         ```dockerfile
+        FROM golangci/golangci-lint@sha256:... AS lint
         RUN apt-get update \
             && apt-get install -y --no-install-recommends libvips-dev \
             && rm -rf /var/lib/apt/lists/*
+        WORKDIR /src
+        COPY go.mod go.sum ./
+        RUN go mod download
+        ARG CACHEBUST
         ```
 
     - `.dockerignore` lets `.git` into the build context. It keeps out every git

TODO.md

@@ -29,6 +29,15 @@
 
 # Completed Steps
 
+- re-vendor `REPO_POLICIES.md` and the four scripts that pass `CACHEBUST` from
+  `sneak/prompts` `next` at `0653579`: in the `lint`, `test` and `builder`
+  stages of the `Dockerfile` the module download, and in `builder` the
+  system-package install, now come before `ARG CACHEBUST`, so Docker keeps them
+  until the base image, the install line, `go.mod` or `go.sum` changes, and
+  every step that lints, tests or compiles still comes after it;
+  `golangci-lint run` also keeps golangci-lint's cache. README says the same
+  (2026-10-18, https://git.eeqj.de/sneak/sfdupes/issues/109)
+
 - releases until 1.0 are `v0.0.N` tags on `main`'s tip, so
   `go install sneak.berlin/go/sfdupes@latest` installs `main`'s tip once it has
   passed the build; README "Install" says so, says what `@main` and `v0.0.1`

script/cibuild

@@ -2,11 +2,12 @@
 # script/cibuild: run the CI build. It bootstraps first: a CI runner
 # checks out and runs this and nothing else, and script/fmt-check runs
 # the formatter on the host, which a pristine checkout cannot do.
-# A new CACHEBUST value reruns every RUN step, since a cached gate phase
+# A new CACHEBUST value reruns every RUN step after the ARG CACHEBUST
-# is a check that did not run; unlike --no-cache, it keeps Go's build cache.
+# line, since a cached gate phase is a check that did not run; unlike
+# --no-cache, it keeps the layers above that line and the cache mounts.
 # This relies on every stage of the Dockerfile declaring ARG CACHEBUST
-# first; a stage without that line is served from cache on an unchanged
+# before every step that lints, tests or compiles; a stage without that
-# tree, with no warning.
+# line is served from cache on an unchanged tree, with no warning.
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"

script/docker

@@ -1,11 +1,12 @@
 #!/bin/sh
 # script/docker: build the Docker image tagged with the project name.
 # Identical in all repos; the tag comes from script/projectname.
-# A new CACHEBUST value reruns every RUN step, since a cached gate phase
+# A new CACHEBUST value reruns every RUN step after the ARG CACHEBUST
-# is a check that did not run; unlike --no-cache, it keeps Go's build cache.
+# line, since a cached gate phase is a check that did not run; unlike
+# --no-cache, it keeps the layers above that line and the cache mounts.
 # This relies on every stage of the Dockerfile declaring ARG CACHEBUST
-# first; a stage without that line is served from cache on an unchanged
+# before every step that lints, tests or compiles; a stage without that
-# tree, with no warning.
+# line is served from cache on an unchanged tree, with no warning.
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"

script/lint

@@ -5,11 +5,13 @@
 # make its answer untrustworthy.
 #
 # The phase is not the last stage in the file, so it is built only when
-# --target names it. A new CACHEBUST value reruns every RUN step, since
+# --target names it. A new CACHEBUST value reruns every RUN step after
-# a cached lint layer is a lint that did not run; unlike --no-cache, it
+# the ARG CACHEBUST line, since a cached lint layer is a lint that did
-# keeps Go's build cache. This relies on every stage of the Dockerfile
+# not run; unlike --no-cache, it keeps the layers above that line and
-# declaring ARG CACHEBUST first; a stage without that line is served
+# the cache mounts. This relies on every stage of the Dockerfile
-# from cache on an unchanged tree, with no warning.
+# declaring ARG CACHEBUST before every step that lints, tests or
+# compiles; a stage without that line is served from cache on an
+# unchanged tree, with no warning.
 # --output type=cacheonly writes no image, since nothing uses one.
 set -eu
 

script/test

@@ -3,11 +3,12 @@
 # and this builds that phase alone, on the same terms as script/lint:
 # --target because a phase that is not the last stage is built only when
 # named, and a new CACHEBUST value because a cached test layer is a test
-# that did not run; unlike --no-cache, it keeps Go's build cache. This
+# that did not run; unlike --no-cache, it keeps the layers above the ARG
-# relies on every stage of the Dockerfile declaring ARG CACHEBUST first;
+# CACHEBUST line and the cache mounts. This relies on every stage of the
-# a stage without that line is served from cache on an unchanged tree,
+# Dockerfile declaring ARG CACHEBUST before every step that lints, tests
-# with no warning. --output type=cacheonly writes no image, since
+# or compiles; a stage without that line is served from cache on an
-# nothing uses one.
+# unchanged tree, with no warning. --output type=cacheonly writes no
+# image, since nothing uses one.
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"