sneak/prompts
1
0
Fork 0
Code Issues 5 Pull Requests 1 Actions 2 Packages Projects Releases Wiki Activity

Keep modules, lint cache and system packages in check builds (closes #130)
check / check (push) Canceled after 0s
Details

Browse Source 
On sneak's ruling in #130, check
builds may keep three more things besides Go's build cache. In each stage
of the canonical Go Dockerfile, the system-package install and the module
download now come before ARG CACHEBUST, so Docker keeps those layers until
the base image, the install line, go.mod or go.sum changes. The modules
are kept as a layer, not a cache mount, because Go does not check a cache
mount's files again. golangci-lint run also mounts golangci-lint's cache.
Every step that lints, tests or compiles still comes after ARG CACHEBUST
and reruns on every check build, and both go test runs keep -count=1. The
policy, both checklists, the Go style guide, the README and the script
comments state the rule the same way.

Model: opus-5-5
This commit is contained in:
clawbot
2026-10-18 13:30:16 +00:00
parent e3f79c49da
commit 9d68027287
10 changed files with 156 additions and 79 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+10 -3
View File
@@ -144,9 +144,16 @@ alpine. We provide:
`make hooks` installs the pre-commit hook that runs `script/precommit`.
Every stage of the `Dockerfile` declares `ARG CACHEBUST` as its first line after
`FROM`. The `script/` builds pass a new `CACHEBUST` value on every build, which
reruns every `RUN` step and keeps Go's build cache; `--no-cache` would empty it.
In every stage of a `Dockerfile`, `ARG CACHEBUST` comes right after the steps
that install system packages and download Go's modules, or right after `FROM` in
a stage with neither, and every step that lints, tests or compiles comes after
it. The `script/` builds pass a new `CACHEBUST` value on every build, which
reruns every `RUN` step after that line and leaves the layers above it and the
cache mounts in place; `--no-cache` would build those layers again and empty the
mounts. This repo's own `Dockerfile` declares `ARG CACHEBUST` right after `FROM`
in every stage: each stage installs system packages and node packages together
in one `script/bootstrap` step, and check builds may keep system packages but
not node packages.
## Rationale
TODO.md
+15
View File
@@ -21,6 +21,21 @@ fmt-check, and commit.
# Completed Steps
- 2026-10-18: Check builds may now also keep Go's downloaded modules,
golangci-lint's cache and the layers that install system packages (issue 130),
on sneak's ruling there. In each stage of the canonical Go `Dockerfile`, the
system-package install and the module download (`COPY go.mod go.sum ./`,
`RUN go mod download`) come before `ARG CACHEBUST`, so Docker keeps them until
the base image, the install line, `go.mod` or `go.sum` changes. The modules
are kept as a layer, not a cache mount, because Go does not check the files in
a cache mount again. `golangci-lint run` also mounts golangci-lint's cache,
`/root/.cache/golangci-lint`. Every step that lints, tests or compiles still
comes after `ARG CACHEBUST` and runs on every check build, and both `go test`
runs keep `-count=1`. `REPO_POLICIES.md`, both checklists, the Go style guide,
the README and the script comments state the rule the same way. This repo's
own `Dockerfile` is unchanged: its `script/bootstrap` step also installs node
packages, which the ruling does not cover. Repositories pick this up on their
next re-vendor.
- 2026-10-15: Check builds now keep Go's build cache between runs (issue 113),
on sneak's ruling there that Go's compile cache may survive. `script/lint`,
`script/test`, `script/cibuild` and `script/docker` pass a new `CACHEBUST`
prompts/CODE_STYLEGUIDE_GO.md
+6 -4
View File
@@ -1,6 +1,6 @@
---
title: Code Styleguide — Go
last_modified: 2026-10-15
last_modified: 2026-10-18
---
1. Try to hard wrap long lines at 77 characters or less.
@@ -112,9 +112,11 @@ last_modified: 2026-10-15
both, so a build makes sure the code is in an able-to-be-compiled state,
linted, and its tests run. Go through `script/cibuild` or `script/docker`
rather than a bare `docker build .`: they pass a new `CACHEBUST` value on
every build, which reruns every `RUN` step and keeps Go's build cache;
`--no-cache` would empty it. Without a new value an unchanged tree serves
the gate layers from cache and the build reports a green it never ran.
every build, which reruns every step that lints, tests or compiles and keeps
the system packages, Go's modules and build cache, and golangci-lint's
cache; `--no-cache` would throw all of them away. Without a new value an
unchanged tree serves the gate layers from cache and the build reports a
green it never ran.
1. Every repo must have a `Makefile`. See
[Repository Policies](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/REPO_POLICIES.md)
prompts/EXISTING_REPO_CHECKLIST.md
+17 -9
View File
@@ -1,6 +1,6 @@
---
title: Existing Repo Checklist
last_modified: 2026-10-15
last_modified: 2026-10-18
---
Use this checklist when beginning work in a repo that may not yet conform to our
@@ -49,14 +49,22 @@ with your task.
`--target` names it, so a lost `COPY --from=` edge leaves `docker build .`
passing while the gate never runs. Confirm by planting a violation, not by
reading the file.
- [ ] Every stage declares `ARG CACHEBUST` as its first line after `FROM`. The
`script/` builds pass a new `CACHEBUST` value on every build, which reruns
every `RUN` step and keeps Go's build cache; `--no-cache` would empty it.
A stage without the `ARG` line is served from cache on an unchanged tree,
so the scripts and the `ARG` lines are re-vendored in one commit.
- [ ] Go: the steps that compile (`golangci-lint run`, `go test`, `go build`)
mount Go's build cache (`--mount=type=cache,target=/root/.cache/go-build`)
and nothing else, and both `go test` runs carry `-count=1`, as in the
- [ ] In every stage `ARG CACHEBUST` comes right after the steps that install
system packages and download Go's modules, or right after `FROM` in a
stage with neither, and every step that lints, tests or compiles comes
after it. The `script/` builds pass a new `CACHEBUST` value on every
build, which reruns every `RUN` step after that line and leaves the layers
above it and the cache mounts in place; `--no-cache` would build those
layers again and empty the mounts. A stage without the `ARG` line is
served from cache on an unchanged tree, so the scripts and the `ARG` lines
are re-vendored in one commit.
- [ ] Go: each Go stage downloads the modules (`COPY go.mod go.sum ./`,
`RUN go mod download`) above `ARG CACHEBUST`; the steps that compile
(`golangci-lint run`, `go test`, `go build`) mount Go's build cache
(`--mount=type=cache,target=/root/.cache/go-build`), `golangci-lint run`
also mounts golangci-lint's
(`--mount=type=cache,target=/root/.cache/golangci-lint`), and no step
mounts anything else; both `go test` runs carry `-count=1`, as in the
canonical Go `Dockerfile` in `REPO_POLICIES.md`
- [ ] The gate phases invoke their tools directly, never through `make lint` or
`script/test` — those are themselves a `docker build` and would recurse
prompts/NEW_REPO_CHECKLIST.md
+17 -10
View File
@@ -1,6 +1,6 @@
---
title: New Repo Checklist
last_modified: 2026-10-15
last_modified: 2026-10-18
---
Use this checklist when creating a new repository from scratch. Follow the steps
@@ -108,15 +108,22 @@ Template files can be fetched from:
stage carries a `COPY --from=` of a harmless file from each so the image
cannot be built unless both passed. Keep the final stage last: a stage
nothing depends on is built only when `--target` names it.
- Every stage declares `ARG CACHEBUST` as its first line after `FROM`. The
`script/` builds pass a new `CACHEBUST` value on every build, which reruns
every `RUN` step and keeps Go's build cache; `--no-cache` would empty it.
A stage without the `ARG` line is served from cache on an unchanged tree,
so the scripts and every stage's `ARG CACHEBUST` line are added in the
same commit.
- Go: the steps that compile (`golangci-lint run`, `go test`, `go build`)
mount Go's build cache (`--mount=type=cache,target=/root/.cache/go-build`)
and nothing else, and both `go test` runs carry `-count=1`, as in the
- In every stage `ARG CACHEBUST` comes right after the steps that install
system packages and download Go's modules, or right after `FROM` in a
stage with neither, and every step that lints, tests or compiles comes
after it. The `script/` builds pass a new `CACHEBUST` value on every
build, which reruns every `RUN` step after that line and leaves the layers
above it and the cache mounts in place; `--no-cache` would build those
layers again and empty the mounts. A stage without the `ARG` line is
served from cache on an unchanged tree, so the scripts and every stage's
`ARG CACHEBUST` line are added in the same commit.
- Go: each Go stage downloads the modules (`COPY go.mod go.sum ./`,
`RUN go mod download`) above `ARG CACHEBUST`; the steps that compile
(`golangci-lint run`, `go test`, `go build`) mount Go's build cache
(`--mount=type=cache,target=/root/.cache/go-build`), `golangci-lint run`
also mounts golangci-lint's
(`--mount=type=cache,target=/root/.cache/golangci-lint`), and no step
mounts anything else; both `go test` runs carry `-count=1`, as in the
canonical Go `Dockerfile` in `REPO_POLICIES.md`
- Server: the final stage builds and runs the application
- Non-server: the final stage brings up the dev environment
prompts/REPO_POLICIES.md
+68 -35
View File
@@ -1,6 +1,6 @@
---
title: Repository Policies
last_modified: 2026-10-15
last_modified: 2026-10-18
---
This document covers repository structure, tooling, and workflow standards. Code
@@ -159,8 +159,11 @@ style conventions are in separate documents:
one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs
exit non-zero with `parallel golangci-lint is running` — a status a caller
cannot tell from real findings. Both have produced wrong verdicts in this
org, in both directions. A container has its own golangci-lint cache, its
own `TMPDIR` and a digest-pinned binary, so neither is reachable.
org, in both directions. A container has its own `TMPDIR`, so the lock is
out of reach. The lint phase does keep golangci-lint's cache between check
builds (below), but it lints every checkout at `/src`, and a finding is
taken from that cache only for the same files, settings and golangci-lint
version.
- **Any build that runs checks is given a new `CACHEBUST` value.** Docker
invalidates a `COPY` layer only when the copied content changes, so on an
@@ -170,15 +173,21 @@ style conventions are in separate documents:
id, so no two builds share a value, not even the three in one
`script/cibuild`. `script/lint`, `script/test`, `script/cibuild` and
`script/docker` are the four, and there is no fifth — `script/check` runs the
two gate phases and `script/fmt-check`, and builds no image of its own. Every
stage of the `Dockerfile` declares `ARG CACHEBUST` as its first line after
`FROM`, and a new value reruns every `RUN` step after that line; `COPY` steps
are compared by content and need nothing. A stage without the `ARG` line is
served from cache on an unchanged tree, so a repository re-vendors the scripts
and adds the `ARG` lines in one commit. The scripts do not pass `--no-cache`:
it reruns every step too, but it also empties every cache mount, and Go's
build cache, which the canonical Go `Dockerfile` below mounts so that compiled
packages survive between check builds, would start empty every time. A bare
two gate phases and `script/fmt-check`, and builds no image of its own. In
every stage of the `Dockerfile`, `ARG CACHEBUST` comes right after the steps
that install system packages and download Go's modules, or right after `FROM`
in a stage with neither, and every step that lints, tests or compiles comes
after it. A step that installs anything else as well, such as a
`script/bootstrap` that also installs node packages, comes after it too. A new
value reruns every `RUN` step after that line; `COPY` steps are compared by
content and need nothing. Docker keeps the layers above the line until the
base image, their own lines or the files they copy change. A stage without the
`ARG` line is served from cache on an unchanged tree, so a repository
re-vendors the scripts and adds the `ARG` lines in one commit. The scripts do
not pass `--no-cache`: it reruns every step too, but it also builds the layers
above the line again and empties every cache mount, so Go's build cache and
golangci-lint's, which the canonical Go `Dockerfile` below mounts so that they
survive between check builds, would start empty every time. A bare
`docker build .` is not evidence that anything ran: a sub-second build
reporting success is a cache hit, not a result. Never invalidate by pruning —
`docker builder prune` and friends destroy a build cache shared with every
@@ -194,19 +203,25 @@ style conventions are in separate documents:
`Dockerfile`:
```dockerfile
# Every stage declares ARG CACHEBUST first: script/ passes a new value
# on every build, which reruns every RUN step after it. The steps that
# compile mount Go's build cache, which survives from build to build.
# In every stage ARG CACHEBUST comes right after the steps that install
# system packages and download Go's modules, or right after FROM in a
# stage with neither, and every step that lints, tests or compiles
# comes after it. script/ passes a new value on every build, which
# reruns every RUN step after it. Docker keeps the layers above it
# until the base image, their own lines or go.mod and go.sum change.
# The steps that compile mount Go's build cache, and golangci-lint run
# also mounts golangci-lint's; both survive from build to build.
# Lint phase
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
FROM golangci/golangci-lint@sha256:... AS lint
ARG CACHEBUST
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
ARG CACHEBUST
COPY . .
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/root/.cache/golangci-lint \
golangci-lint run --config .golangci.yml ./...
# Test phase. -race needs cgo and so a C compiler, which the Debian Go
@@ -214,10 +229,10 @@ style conventions are in separate documents:
# cache also holds test results, which go test would otherwise replay.
# golang:1.x, YYYY-MM-DD
FROM golang@sha256:... AS test
ARG CACHEBUST
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
ARG CACHEBUST
COPY . .
RUN --mount=type=cache,target=/root/.cache/go-build \
go test -count=1 -timeout 90s -race -cover ./... || \
@@ -225,19 +240,19 @@ style conventions are in separate documents:
go test -count=1 -timeout 90s -race -v ./...; exit 1; }
# Build stage. Nothing is wanted from either phase above; the copies
# are what make BuildKit build them first, so this stage cannot run
# unless lint and test passed.
# are what make BuildKit build them first, so nothing after them can
# run unless lint and test passed.
# golang:1.x-alpine, YYYY-MM-DD
FROM golang@sha256:... AS builder
ARG CACHEBUST
COPY --from=lint /src/go.sum /dev/null
COPY --from=test /src/go.sum /dev/null
RUN apk add --no-cache git
# A tar-stream context keeps the sender's file owners, which git refuses.
RUN git config --system --add safe.directory /src
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
ARG CACHEBUST
COPY --from=lint /src/go.sum /dev/null
COPY --from=test /src/go.sum /dev/null
# A tar-stream context keeps the sender's file owners, which git refuses.
RUN git config --system --add safe.directory /src
COPY . .
# The VERSION build arg when one is given, otherwise
@@ -274,16 +289,26 @@ style conventions are in separate documents:
- Keep the runtime stage last, and if you add a stage after it, give it the
same two copies. A plain `docker build .` builds the last stage's chain
and nothing else.
- Only Go's build cache is mounted (`/root/.cache/go-build`, where Go keeps
it for root), and only on the steps that compile: `golangci-lint run`,
both `go test` runs and `go build`. It keeps compiled packages between
check builds, and Go reuses one only when all its inputs are unchanged, so
it saves the compile without skipping a check. Nothing else is kept:
golangci-lint's own cache would replay stored findings in place of a lint
run, and Go builds from whatever files a kept module cache holds without
checking them again (only `go mod verify` does), so a file changed there
would go unnoticed. Go's build cache also holds test results, which is why
both `go test` runs carry `-count=1`.
- Check builds keep these, and none of them can hand back a stale result:
- The layers above `ARG CACHEBUST`: the system packages and Go's
modules. Docker installs the packages again when the base image or the
install line changes, and downloads the modules again when `go.mod` or
`go.sum` changes. Go checks every module against `go.sum` as it
downloads it, and no later build can change a kept layer. The modules
are kept in a layer rather than a cache mount because a cache mount is
shared by every build on the host, and Go builds from whatever files
it holds without checking them again (only `go mod verify` does), so a
file changed there would go unnoticed.
- Go's build cache (`/root/.cache/go-build`, where Go keeps it for
root), mounted on the steps that compile: `golangci-lint run`, both
`go test` runs and `go build`. Go reuses a compiled package only when
all its inputs are unchanged. The cache also holds test results, which
is why both `go test` runs carry `-count=1`.
- golangci-lint's cache (`/root/.cache/golangci-lint`, where the pinned
image keeps it), mounted on `golangci-lint run`. golangci-lint reuses
a package's findings only when the package's files and those of every
package it imports, `go.mod`, the linters and their settings in
`.golangci.yml`, and the golangci-lint version are all unchanged.
- If the project uses `//go:embed` directives that reference build artifacts
(e.g. a web frontend compiled in a separate stage), the lint phase must
create placeholder files so the embed directives resolve. Example:
@@ -292,12 +317,20 @@ style conventions are in separate documents:
in the lint phase. The `golangci/golangci-lint` image is Debian-based and
has no `apk`, so install with `apt-get` under the Debian package name
(`libvips-dev`, where alpine says `vips-dev`), and delete the package
lists in the same `RUN`, so the layer does not keep them:
lists in the same `RUN`, so the layer does not keep them. The test phase,
on the Debian Go image, installs the libraries its tests need the same
way. Either install goes right after `FROM`, above `ARG CACHEBUST`, so
Docker keeps it until the base image or the install line changes:
```dockerfile
FROM golangci/golangci-lint@sha256:... AS lint
RUN apt-get update \
&& apt-get install -y --no-install-recommends libvips-dev \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
ARG CACHEBUST
```
- `.dockerignore` lets `.git` into the build context. It keeps out every git
script/cibuild
+5 -4
View File
@@ -2,11 +2,12 @@
# script/cibuild: run the CI build. It bootstraps first: a CI runner
# checks out and runs this and nothing else, and script/fmt-check runs
# the formatter on the host, which a pristine checkout cannot do.
# A new CACHEBUST value reruns every RUN step, since a cached gate phase
# is a check that did not run; unlike --no-cache, it keeps Go's build cache.
# A new CACHEBUST value reruns every RUN step after the ARG CACHEBUST
# line, since a cached gate phase is a check that did not run; unlike
# --no-cache, it keeps the layers above that line and the cache mounts.
# This relies on every stage of the Dockerfile declaring ARG CACHEBUST
# first; a stage without that line is served from cache on an unchanged
# tree, with no warning.
# before every step that lints, tests or compiles; a stage without that
# line is served from cache on an unchanged tree, with no warning.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
script/docker
+5 -4
View File
@@ -1,11 +1,12 @@
#!/bin/sh
# script/docker: build the Docker image tagged with the project name.
# Identical in all repos; the tag comes from script/projectname.
# A new CACHEBUST value reruns every RUN step, since a cached gate phase
# is a check that did not run; unlike --no-cache, it keeps Go's build cache.
# A new CACHEBUST value reruns every RUN step after the ARG CACHEBUST
# line, since a cached gate phase is a check that did not run; unlike
# --no-cache, it keeps the layers above that line and the cache mounts.
# This relies on every stage of the Dockerfile declaring ARG CACHEBUST
# first; a stage without that line is served from cache on an unchanged
# tree, with no warning.
# before every step that lints, tests or compiles; a stage without that
# line is served from cache on an unchanged tree, with no warning.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
script/lint
+7 -5
View File
@@ -5,11 +5,13 @@
# make its answer untrustworthy.
#
# The phase is not the last stage in the file, so it is built only when
# --target names it. A new CACHEBUST value reruns every RUN step, since
# a cached lint layer is a lint that did not run; unlike --no-cache, it
# keeps Go's build cache. This relies on every stage of the Dockerfile
# declaring ARG CACHEBUST first; a stage without that line is served
# from cache on an unchanged tree, with no warning.
# --target names it. A new CACHEBUST value reruns every RUN step after
# the ARG CACHEBUST line, since a cached lint layer is a lint that did
# not run; unlike --no-cache, it keeps the layers above that line and
# the cache mounts. This relies on every stage of the Dockerfile
# declaring ARG CACHEBUST before every step that lints, tests or
# compiles; a stage without that line is served from cache on an
# unchanged tree, with no warning.
# --output type=cacheonly writes no image, since nothing uses one.
set -eu
script/test
+6 -5
View File
@@ -3,11 +3,12 @@
# and this builds that phase alone, on the same terms as script/lint:
# --target because a phase that is not the last stage is built only when
# named, and a new CACHEBUST value because a cached test layer is a test
# that did not run; unlike --no-cache, it keeps Go's build cache. This
# relies on every stage of the Dockerfile declaring ARG CACHEBUST first;
# a stage without that line is served from cache on an unchanged tree,
# with no warning. --output type=cacheonly writes no image, since
# nothing uses one.
# that did not run; unlike --no-cache, it keeps the layers above the ARG
# CACHEBUST line and the cache mounts. This relies on every stage of the
# Dockerfile declaring ARG CACHEBUST before every step that lints, tests
# or compiles; a stage without that line is served from cache on an
# unchanged tree, with no warning. --output type=cacheonly writes no
# image, since nothing uses one.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"