Keep modules, lint cache and system packages in check builds (closes #109)
check / check (push) Waiting to run
check / check (push) Waiting to run
Re-vendors REPO_POLICIES.md and script/cibuild, docker, lint and test from sneak/prompts at 0653579, on sneak's ruling in sneak/prompts#130. In the Dockerfile's lint, test and builder stages, the system-package install and the module download now come before ARG CACHEBUST, so Docker keeps those layers until the base image, the install line, go.mod or go.sum changes. golangci-lint run also mounts golangci-lint's cache. Every step that lints, tests or compiles still runs on every check build, and go test keeps -count=1 and -race. README and TODO.md say the same. Model: opus-5-5
This commit was merged in pull request #131.
This commit is contained in:
+68
-35
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: Repository Policies
|
||||
last_modified: 2026-10-15
|
||||
last_modified: 2026-10-18
|
||||
---
|
||||
|
||||
This document covers repository structure, tooling, and workflow standards. Code
|
||||
@@ -159,8 +159,11 @@ style conventions are in separate documents:
|
||||
one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs
|
||||
exit non-zero with `parallel golangci-lint is running` — a status a caller
|
||||
cannot tell from real findings. Both have produced wrong verdicts in this
|
||||
org, in both directions. A container has its own golangci-lint cache, its
|
||||
own `TMPDIR` and a digest-pinned binary, so neither is reachable.
|
||||
org, in both directions. A container has its own `TMPDIR`, so the lock is
|
||||
out of reach. The lint phase does keep golangci-lint's cache between check
|
||||
builds (below), but it lints every checkout at `/src`, and a finding is
|
||||
taken from that cache only for the same files, settings and golangci-lint
|
||||
version.
|
||||
|
||||
- **Any build that runs checks is given a new `CACHEBUST` value.** Docker
|
||||
invalidates a `COPY` layer only when the copied content changes, so on an
|
||||
@@ -170,15 +173,21 @@ style conventions are in separate documents:
|
||||
id, so no two builds share a value, not even the three in one
|
||||
`script/cibuild`. `script/lint`, `script/test`, `script/cibuild` and
|
||||
`script/docker` are the four, and there is no fifth — `script/check` runs the
|
||||
two gate phases and `script/fmt-check`, and builds no image of its own. Every
|
||||
stage of the `Dockerfile` declares `ARG CACHEBUST` as its first line after
|
||||
`FROM`, and a new value reruns every `RUN` step after that line; `COPY` steps
|
||||
are compared by content and need nothing. A stage without the `ARG` line is
|
||||
served from cache on an unchanged tree, so a repository re-vendors the scripts
|
||||
and adds the `ARG` lines in one commit. The scripts do not pass `--no-cache`:
|
||||
it reruns every step too, but it also empties every cache mount, and Go's
|
||||
build cache, which the canonical Go `Dockerfile` below mounts so that compiled
|
||||
packages survive between check builds, would start empty every time. A bare
|
||||
two gate phases and `script/fmt-check`, and builds no image of its own. In
|
||||
every stage of the `Dockerfile`, `ARG CACHEBUST` comes right after the steps
|
||||
that install system packages and download Go's modules, or right after `FROM`
|
||||
in a stage with neither, and every step that lints, tests or compiles comes
|
||||
after it. A step that installs anything else as well, such as a
|
||||
`script/bootstrap` that also installs node packages, comes after it too. A new
|
||||
value reruns every `RUN` step after that line; `COPY` steps are compared by
|
||||
content and need nothing. Docker keeps the layers above the line until the
|
||||
base image, their own lines or the files they copy change. A stage without the
|
||||
`ARG` line is served from cache on an unchanged tree, so a repository
|
||||
re-vendors the scripts and adds the `ARG` lines in one commit. The scripts do
|
||||
not pass `--no-cache`: it reruns every step too, but it also builds the layers
|
||||
above the line again and empties every cache mount, so Go's build cache and
|
||||
golangci-lint's, which the canonical Go `Dockerfile` below mounts so that they
|
||||
survive between check builds, would start empty every time. A bare
|
||||
`docker build .` is not evidence that anything ran: a sub-second build
|
||||
reporting success is a cache hit, not a result. Never invalidate by pruning —
|
||||
`docker builder prune` and friends destroy a build cache shared with every
|
||||
@@ -194,19 +203,25 @@ style conventions are in separate documents:
|
||||
`Dockerfile`:
|
||||
|
||||
```dockerfile
|
||||
# Every stage declares ARG CACHEBUST first: script/ passes a new value
|
||||
# on every build, which reruns every RUN step after it. The steps that
|
||||
# compile mount Go's build cache, which survives from build to build.
|
||||
# In every stage ARG CACHEBUST comes right after the steps that install
|
||||
# system packages and download Go's modules, or right after FROM in a
|
||||
# stage with neither, and every step that lints, tests or compiles
|
||||
# comes after it. script/ passes a new value on every build, which
|
||||
# reruns every RUN step after it. Docker keeps the layers above it
|
||||
# until the base image, their own lines or go.mod and go.sum change.
|
||||
# The steps that compile mount Go's build cache, and golangci-lint run
|
||||
# also mounts golangci-lint's; both survive from build to build.
|
||||
|
||||
# Lint phase
|
||||
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
|
||||
FROM golangci/golangci-lint@sha256:... AS lint
|
||||
ARG CACHEBUST
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
ARG CACHEBUST
|
||||
COPY . .
|
||||
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
--mount=type=cache,target=/root/.cache/golangci-lint \
|
||||
golangci-lint run --config .golangci.yml ./...
|
||||
|
||||
# Test phase. -race needs cgo and so a C compiler, which the Debian Go
|
||||
@@ -214,10 +229,10 @@ style conventions are in separate documents:
|
||||
# cache also holds test results, which go test would otherwise replay.
|
||||
# golang:1.x, YYYY-MM-DD
|
||||
FROM golang@sha256:... AS test
|
||||
ARG CACHEBUST
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
ARG CACHEBUST
|
||||
COPY . .
|
||||
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
go test -count=1 -timeout 90s -race -cover ./... || \
|
||||
@@ -225,19 +240,19 @@ style conventions are in separate documents:
|
||||
go test -count=1 -timeout 90s -race -v ./...; exit 1; }
|
||||
|
||||
# Build stage. Nothing is wanted from either phase above; the copies
|
||||
# are what make BuildKit build them first, so this stage cannot run
|
||||
# unless lint and test passed.
|
||||
# are what make BuildKit build them first, so nothing after them can
|
||||
# run unless lint and test passed.
|
||||
# golang:1.x-alpine, YYYY-MM-DD
|
||||
FROM golang@sha256:... AS builder
|
||||
ARG CACHEBUST
|
||||
COPY --from=lint /src/go.sum /dev/null
|
||||
COPY --from=test /src/go.sum /dev/null
|
||||
RUN apk add --no-cache git
|
||||
# A tar-stream context keeps the sender's file owners, which git refuses.
|
||||
RUN git config --system --add safe.directory /src
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
ARG CACHEBUST
|
||||
COPY --from=lint /src/go.sum /dev/null
|
||||
COPY --from=test /src/go.sum /dev/null
|
||||
# A tar-stream context keeps the sender's file owners, which git refuses.
|
||||
RUN git config --system --add safe.directory /src
|
||||
COPY . .
|
||||
|
||||
# The VERSION build arg when one is given, otherwise
|
||||
@@ -274,16 +289,26 @@ style conventions are in separate documents:
|
||||
- Keep the runtime stage last, and if you add a stage after it, give it the
|
||||
same two copies. A plain `docker build .` builds the last stage's chain
|
||||
and nothing else.
|
||||
- Only Go's build cache is mounted (`/root/.cache/go-build`, where Go keeps
|
||||
it for root), and only on the steps that compile: `golangci-lint run`,
|
||||
both `go test` runs and `go build`. It keeps compiled packages between
|
||||
check builds, and Go reuses one only when all its inputs are unchanged, so
|
||||
it saves the compile without skipping a check. Nothing else is kept:
|
||||
golangci-lint's own cache would replay stored findings in place of a lint
|
||||
run, and Go builds from whatever files a kept module cache holds without
|
||||
checking them again (only `go mod verify` does), so a file changed there
|
||||
would go unnoticed. Go's build cache also holds test results, which is why
|
||||
both `go test` runs carry `-count=1`.
|
||||
- Check builds keep these, and none of them can hand back a stale result:
|
||||
- The layers above `ARG CACHEBUST`: the system packages and Go's
|
||||
modules. Docker installs the packages again when the base image or the
|
||||
install line changes, and downloads the modules again when `go.mod` or
|
||||
`go.sum` changes. Go checks every module against `go.sum` as it
|
||||
downloads it, and no later build can change a kept layer. The modules
|
||||
are kept in a layer rather than a cache mount because a cache mount is
|
||||
shared by every build on the host, and Go builds from whatever files
|
||||
it holds without checking them again (only `go mod verify` does), so a
|
||||
file changed there would go unnoticed.
|
||||
- Go's build cache (`/root/.cache/go-build`, where Go keeps it for
|
||||
root), mounted on the steps that compile: `golangci-lint run`, both
|
||||
`go test` runs and `go build`. Go reuses a compiled package only when
|
||||
all its inputs are unchanged. The cache also holds test results, which
|
||||
is why both `go test` runs carry `-count=1`.
|
||||
- golangci-lint's cache (`/root/.cache/golangci-lint`, where the pinned
|
||||
image keeps it), mounted on `golangci-lint run`. golangci-lint reuses
|
||||
a package's findings only when the package's files and those of every
|
||||
package it imports, `go.mod`, the linters and their settings in
|
||||
`.golangci.yml`, and the golangci-lint version are all unchanged.
|
||||
- If the project uses `//go:embed` directives that reference build artifacts
|
||||
(e.g. a web frontend compiled in a separate stage), the lint phase must
|
||||
create placeholder files so the embed directives resolve. Example:
|
||||
@@ -292,12 +317,20 @@ style conventions are in separate documents:
|
||||
in the lint phase. The `golangci/golangci-lint` image is Debian-based and
|
||||
has no `apk`, so install with `apt-get` under the Debian package name
|
||||
(`libvips-dev`, where alpine says `vips-dev`), and delete the package
|
||||
lists in the same `RUN`, so the layer does not keep them:
|
||||
lists in the same `RUN`, so the layer does not keep them. The test phase,
|
||||
on the Debian Go image, installs the libraries its tests need the same
|
||||
way. Either install goes right after `FROM`, above `ARG CACHEBUST`, so
|
||||
Docker keeps it until the base image or the install line changes:
|
||||
|
||||
```dockerfile
|
||||
FROM golangci/golangci-lint@sha256:... AS lint
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends libvips-dev \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
ARG CACHEBUST
|
||||
```
|
||||
|
||||
- `.dockerignore` lets `.git` into the build context. It keeps out every git
|
||||
|
||||
Reference in New Issue
Block a user