Compare commits

2 Commits
Author SHA1 Message Date
sneak e4e297aa60 Run the tests under the race detector with make test-race (closes #18)
check / check (push) Failing after 2s
script/test-race runs go test -race in a digest-pinned Debian golang
image that has gcc, since the detector needs cgo and the build keeps it
off. The checkout is mounted read-only and the container is removed
afterwards. The tests run as the calling user, or as nobody when that is
root, so the tests that make a file unreadable still see the read fail.
It is not part of make check. The detector found no races.

Model: opus-5-5
2026-10-04 17:31:36 +00:00
clawbot bebfac1dcb Fail a bare docker build instead of serving cached gates (closes #39)
check / check (push) Failing after 3s
Each Dockerfile stage that runs gates now checks, right after its
ARG CHECK_EPOCH, that the value is not empty, and stops with a message
naming script/cibuild and script/docker. A plain `docker build .` can
no longer report a green from cached gate layers.

script/cibuild and script/docker now append the process id to the
epoch, the form script/lint already uses, so two runs started in the
same second still get different values.

README says both. TODO.md corrects the steady-state CACHED count
recorded for issue 32 from twelve to thirteen.

Model: opus-5-5
2026-10-04 19:30:20 +02:00
8 changed files with 123 additions and 37 deletions
+21 -6
View File
@@ -10,7 +10,9 @@ COPY . .
# invalidates COPY only when the copied content changes, so on an
# unchanged tree the gates below would be served from cache and the
# build would exit 0 having run nothing. script/cibuild and
# script/docker pass a fresh CHECK_EPOCH on every invocation.
# script/docker pass a fresh CHECK_EPOCH on every invocation. A build
# that passes none, such as a bare `docker build .`, fails at the check
# right after the ARG instead of quietly serving the gates from cache.
#
# Two properties this depends on. ARG is per-stage, so the markdown and
# build stages below declare it again; one declaration here would leave
@@ -22,6 +24,10 @@ COPY . .
# (the pinned base image, go mod download) keeps its cache; only the
# gates go cold.
ARG CHECK_EPOCH
RUN if [ -z "${CHECK_EPOCH}" ]; then \
echo "CHECK_EPOCH is unset; build via script/cibuild or script/docker" >&2; \
exit 1; \
fi
# The linter is invoked directly here, not through `make lint`. That
# target now runs `docker build -f Dockerfile.lint`, and a docker build
@@ -71,9 +77,13 @@ WORKDIR /src
# Markdown stage: the Markdown half of `make fmt-check`, as a gate.
FROM prettier AS markdown
COPY . .
# Second per-stage declaration of the gate cache-buster; see the lint
# stage above.
# Second per-stage declaration of the gate cache-buster and its check;
# see the lint stage above.
ARG CHECK_EPOCH
RUN if [ -z "${CHECK_EPOCH}" ]; then \
echo "CHECK_EPOCH is unset; build via script/cibuild or script/docker" >&2; \
exit 1; \
fi
RUN echo "gate prettier, epoch ${CHECK_EPOCH}" && \
prettier --check '**/*.md' --tab-width 4 --prose-wrap always
@@ -153,10 +163,15 @@ USER builder
# prerequisites. `make`, not the script directly, because the Makefile's
# `export CGO_ENABLED = 0` applies only to what it invokes.
#
# Third per-stage declaration of the gate cache-buster; see the lint
# stage above for why one is not enough. It is placed after USER so the
# drop to the unprivileged user still happens before the checks run.
# Third per-stage declaration of the gate cache-buster and its check;
# see the lint stage above for why one is not enough. It is placed after
# USER so the drop to the unprivileged user still happens before the
# checks run.
ARG CHECK_EPOCH
RUN if [ -z "${CHECK_EPOCH}" ]; then \
echo "CHECK_EPOCH is unset; build via script/cibuild or script/docker" >&2; \
exit 1; \
fi
RUN echo "gate test, epoch ${CHECK_EPOCH}" && make test
# The version stamped into the binary: the VERSION build argument when
+4 -1
View File
@@ -6,7 +6,7 @@ BINARY := sfdupes
VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo dev)
LDFLAGS := -X main.Version=$(VERSION)
.PHONY: sfdupes build bootstrap setup test lint fmt fmt-check check docker hooks clean
.PHONY: sfdupes build bootstrap setup test test-race lint fmt fmt-check check docker hooks clean
# Standard targets are thin shims; the implementations live in script/
# per the scripts-to-rule-them-all pattern.
@@ -27,6 +27,9 @@ setup:
test:
@script/test
test-race:
@script/test-race
lint:
@script/lint
+19 -5
View File
@@ -742,14 +742,23 @@ entrypoints are:
installed: they run in Docker (see `script/lint` and `script/fmt`) and never
from a host install, so there is no host copy to drift from the pin. A missing
`docker` is warned about rather than installed or treated as fatal —
everything except linting and formatting works without it. Ends with
`go mod download`.
everything except linting, formatting and `make test-race` works without it.
Ends with `go mod download`.
- `script/setup` — make a fresh clone ready for development: runs
`script/bootstrap`, then `script/install-precommit`.
- `script/projectname` — print this project's name (`sfdupes`). Scripts that
need the name call it, so they stay identical across repositories.
- `script/test` — run the test suite with a 30-second timeout and coverage
enabled, rerunning verbosely on failure so the logs show which test failed.
- `script/test-race` — run the test suite under the race detector with a
60-second timeout. The detector needs cgo and a C compiler, which the build
never uses, so the tests run in a digest-pinned Debian `golang` image that has
`gcc`, with the checkout mounted read-only; the container is removed when it
exits. They run as the calling user, or as `nobody` when that is root, because
several tests make a file unreadable and root reads it anyway; only then must
the checkout be readable by other users. Not part of `script/check`. Every run
starts with empty caches, so it needs the network and takes minutes, and the
mount needs a local docker daemon.
- `script/lint` — run the linter. It builds `Dockerfile.lint`, which copies the
repository into the digest-pinned `golangci/golangci-lint` image and runs
`golangci-lint config verify` and `golangci-lint run` as build steps, so a
@@ -821,9 +830,12 @@ from binary-versus-pin to pin-versus-pin, which is what
gate layers from cache and the build exits 0 having executed no tests and no
lint — a green it never earned, and one this repository has produced twice.
`CHECK_EPOCH` invalidates the gate layers on every run while leaving the pinned
base images and the dependency layers cached. `script/lint`'s value carries the
process id as well as the epoch, because two lint runs land inside the same
second easily and a bare epoch would cache the second one.
base images and the dependency layers cached. Each script's value carries the
process id as well as the epoch, because two runs land inside the same second
easily and a bare epoch would cache the second one. Each `Dockerfile` stage with
gates fails when the value is empty, so a bare `docker build .` stops with
`CHECK_EPOCH is unset; build via script/cibuild or script/docker` instead of
serving the gates from cache.
## Build
@@ -837,6 +849,8 @@ compile recipe:
- `make setup` — prepare a fresh clone: `bootstrap` plus the pre-commit hook.
- `make test` — run the test suite (30-second timeout; reruns with `-v` on
failure).
- `make test-race` — run the test suite under the race detector, in Docker (see
`script/test-race`); requires `docker`. Not part of `make check`.
- `make lint` — run `golangci-lint` with the repo config, in Docker (see
`script/lint`); requires `docker`.
- `make fmt` / `make fmt-check` — format the Go sources and the Markdown /
+24 -14
View File
@@ -28,6 +28,14 @@
# Completed Steps
- `make test-race` runs the test suite under the race detector in a cgo-enabled
container, outside `make check` (2026-10-04,
https://git.eeqj.de/sneak/sfdupes/issues/18)
- a bare `docker build .` fails with a message naming `script/cibuild` and
`script/docker` instead of serving the gates from cache (2026-10-04,
https://git.eeqj.de/sneak/sfdupes/issues/39)
- `make fmt` and `make fmt-check` run prettier over all Markdown, in Docker, and
CI checks it; all Markdown reformatted (2026-10-04,
https://git.eeqj.de/sneak/sfdupes/issues/19)
@@ -298,18 +306,19 @@
bug, not a fix. Verified by running each script twice back to back on an
unchanged tree under `BUILDKIT_PROGRESS=plain`: all three gates executed on
all four runs, each with a fresh epoch in the log (`script/cibuild` 78.8s then
61.1s; `script/docker` 61.1s then 53.4s), and twelve steps were still served
`CACHED` in the steady state — both `go mod download`s, `apk add`, `adduser`,
the `chown`, every `go.mod`/`go.sum` and source copy, the linter copy out of
the lint stage, and the binary copy into the runtime stage. The lint stage
still gates the build stage: with a deliberate `unused` finding planted in the
tree, the build failed at `make lint` in 36.1s and the build-stage
`make check` never started. The build stage also still drops to the
unprivileged `builder` user before `make check`, which the suite depends on
rather than merely prefers: forcing the same image to run the tests as root
fails `TestScanHardlinkRunFailsTogether`, because root reads straight through
the `chmod(0)` the test uses to prove hard links are read once. This is the
local fix only; propagating it to the canonical templates is `prompts` #26
61.1s; `script/docker` 61.1s then 53.4s), and thirteen steps were still served
`CACHED` in the steady state — the lint stage's `WORKDIR /src`, both
`go mod download`s, `apk add`, `adduser`, the `chown`, every `go.mod`/`go.sum`
and source copy, the linter copy out of the lint stage, and the binary copy
into the runtime stage. The lint stage still gates the build stage: with a
deliberate `unused` finding planted in the tree, the build failed at
`make lint` in 36.1s and the build-stage `make check` never started. The build
stage also still drops to the unprivileged `builder` user before `make check`,
which the suite depends on rather than merely prefers: forcing the same image
to run the tests as root fails `TestScanHardlinkRunFailsTogether`, because
root reads straight through the `chmod(0)` the test uses to prove hard links
are read once. This is the local fix only; propagating it to the canonical
templates is `prompts` #26
- check the installed golangci-lint version in `script/bootstrap` instead of
only its presence (2026-08-09, branch `bootstrap-version-check`, closes #24):
`missing golangci-lint` meant any linter already on `PATH` satisfied the
@@ -492,5 +501,6 @@ Accepted divergences (no action):
- flat single-package layout with `.go` files in the repo root — fine for a
small single-binary tool per the Go styleguide; the tracker audit agrees
- `go test` runs without `-race` — the repo mandates `CGO_ENABLED=0` (pure-Go
builds) and the race detector requires cgo
- `make test` runs without `-race` — the repo mandates `CGO_ENABLED=0` (pure-Go
builds) and the race detector requires cgo, so the detector runs in a separate
cgo-enabled container, `make test-race`, which is not part of `make check`
+8 -7
View File
@@ -7,8 +7,8 @@
# installed: golangci-lint (script/lint) and prettier (script/fmt,
# script/fmt-check) run via docker only, pinned by hash, so their only
# prerequisite is a working docker — which is warned about, not
# installed, because everything except linting and formatting works
# without it.
# installed, because everything except linting, formatting and
# make test-race works without it.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
@@ -75,13 +75,14 @@ main() {
# Linting and Markdown formatting run via docker only, so docker is
# their prerequisite rather than something bootstrap installs. Warn,
# do not fail: everything except `make lint`, `make fmt` and
# `make fmt-check` — and, through them, `make check`, `make docker`
# and the pre-commit hook — works without it.
# do not fail: everything except `make lint`, `make fmt`,
# `make fmt-check` and `make test-race` — and, through them,
# `make check`, `make docker` and the pre-commit hook — works
# without it.
if missing docker; then
echo "bootstrap: WARNING: docker not found; make lint, make fmt," >&2
echo "bootstrap: make fmt-check, make check and make docker" >&2
echo "bootstrap: require it." >&2
echo "bootstrap: make fmt-check, make check, make docker and" >&2
echo "bootstrap: make test-race require it." >&2
fi
go mod download
+5 -2
View File
@@ -21,14 +21,17 @@
# serves the gate layers from cache and the build reports a green it
# never earned. Passing the current epoch invalidates the gate
# layers on every run while leaving the pinned base images and
# go mod download cached; see the Dockerfile for the placement.
# go mod download cached; see the Dockerfile for the placement. The
# process id goes in with the epoch so that two runs started in the
# same second still get different values, the same form script/lint
# uses.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
docker build --build-arg CHECK_EPOCH="$(date +%s)" .
docker build --build-arg CHECK_EPOCH="$(date +%s)-$$" .
}
main "$@"
+2 -2
View File
@@ -3,7 +3,7 @@
# The tag comes from script/projectname.
#
# CHECK_EPOCH is passed for the same reason script/cibuild passes it:
# without it Docker serves the Dockerfile's gate layers from cache on an
# without a fresh value Docker serves the gate layers from cache on an
# unchanged tree and this exits 0 having run none of the lint stage's
# gates, the markdown stage's prettier gate or the builder stage's test
# gate. This is the set of gates a developer or reviewer runs by hand,
@@ -17,7 +17,7 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
docker build \
--build-arg CHECK_EPOCH="$(date +%s)" \
--build-arg CHECK_EPOCH="$(date +%s)-$$" \
-t "$("$SCRIPT_DIR/projectname")" \
.
}
+40
View File
@@ -0,0 +1,40 @@
#!/bin/sh
# script/test-race: run the test suite under the race detector. Not part
# of script/check.
#
# The race detector needs cgo and a C compiler, which the host build
# never uses, so the tests run in a golang image that has gcc. The
# checkout is mounted read-only, so the docker daemon must be local. The
# container starts with empty caches every time: each run downloads the
# dependencies and compiles them with the detector, which needs the
# network and takes minutes.
#
# The tests run as the calling user, never as root: several of them make
# a file unreadable and expect reading it to fail, and root reads it
# anyway. When the caller is root they run as nobody, and then the
# checkout must be readable by other users. Neither user has a home
# directory in the image, so HOME is /tmp, where Go puts its build cache.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# golang:1.25-trixie, 2026-10-04. Debian rather than the Alpine image the
# Dockerfile builds with, because this one includes gcc.
IMAGE="golang@sha256:2c4c60ef415fbfa5e90300722293bef36c5e63fae17570ce18f580af933dbd73"
main() {
user="$(id -u):$(id -g)"
if [ "$(id -u)" -eq 0 ]; then
user=65534:65534
fi
docker run --rm \
--user "$user" \
--env HOME=/tmp \
--env CGO_ENABLED=1 \
--volume "$ROOT:/src:ro" \
--workdir /src \
"$IMAGE" \
go test -race -timeout 60s ./...
}
main "$@"