1 Commits
Author SHA1 Message Date
sneak 60d7b97a8d Stop script/lint writing an image it never uses (closes #48)
check / check (push) Failing after 3s
script/lint builds Dockerfile.lint only for the exit status, but every
run exported the result as an image: seconds spent exporting, and one
untagged image left behind each time. It now builds with
--output=type=cacheonly, so nothing is exported. CHECK_EPOCH still
changes on every run, so the gate steps still run each time; the build
cache is kept as before.

Model: opus-5-5
2026-10-04 15:48:00 +00:00
17 changed files with 1379 additions and 1229 deletions
-2
View File
@@ -1,2 +0,0 @@
node_modules/
yarn.lock
-4
View File
@@ -1,4 +0,0 @@
{
"tabWidth": 4,
"proseWrap": "always"
}
+59 -80
View File
@@ -6,36 +6,34 @@ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
COPY . . COPY . .
# Cache-buster for the gate layers, and only for them: on an unchanged # Cache-buster for the gate layers, and only for them. Docker
# tree Docker would serve the gates below from cache and the build would # invalidates COPY only when the copied content changes, so on an
# exit 0 having run nothing. script/cibuild and script/docker pass a # unchanged tree the gates below would be served from cache and the
# fresh CHECK_EPOCH; a build without one, such as a bare # build would exit 0 having run nothing. script/cibuild and
# `docker build .`, fails at the check right after the ARG. # script/docker pass a fresh CHECK_EPOCH on every invocation.
# #
# ARG is per-stage, so the markdown and build stages declare it again. # Two properties this depends on. ARG is per-stage, so the build stage
# Each gate RUN must reference the value: BuildKit hashes the expanded # below declares it again; one declaration here would leave that
# command, so a declared but unreferenced ARG invalidates nothing. Keep # stage's gate cacheable. And each gate RUN must reference the value,
# it below the dependency layers so they stay cached. # because BuildKit hashes the expanded command: a declared but
# unreferenced ARG invalidates nothing.
#
# It sits below the dependency layers deliberately. Everything above it
# (the pinned base image, go mod download) keeps its cache; only the
# gates go cold.
ARG CHECK_EPOCH ARG CHECK_EPOCH
RUN if [ -z "${CHECK_EPOCH}" ]; then \
echo "CHECK_EPOCH is unset; build via script/cibuild or script/docker" >&2; \
exit 1; \
fi
# These gates call the tools directly, not through `make lint` or # The linter is invoked directly here, not through `make lint`. That
# `make fmt-check`: both run docker, which cannot run inside a docker # target now runs `docker build -f Dockerfile.lint`, and a docker build
# build. This step is the gofmt half of `make fmt-check`; the markdown # cannot run a docker build: routing the gate through make would mean
# stage is its prettier half. gofmt's output is assigned to a variable # nesting docker inside this image. Same reason `make check` is gone
# first so that its own exit status, as when it cannot parse a file, # from the build stage below. `make fmt-check` stays as it is — it is a
# still fails the step. # gate, not the aggregate, and it shells out to nothing.
RUN echo "gate gofmt, epoch ${CHECK_EPOCH}" && \ RUN echo "gate fmt-check, epoch ${CHECK_EPOCH}" && make fmt-check
files="$(gofmt -s -l .)" && \
if [ -n "$files" ]; then \
echo "gofmt: files not formatted:" >&2; echo "$files" >&2; exit 1; \
fi
# Fails the build when the FROM above and the one in Dockerfile.lint pin # The FROM above and the one in Dockerfile.lint pin the same linter
# different linter images. # twice, and nothing else keeps them in sync; this fails the build when
# they disagree. See the script for why it restates neither pin.
RUN echo "gate lint-image-pin, epoch ${CHECK_EPOCH}" && \ RUN echo "gate lint-image-pin, epoch ${CHECK_EPOCH}" && \
script/verify-lint-image-pin script/verify-lint-image-pin
@@ -48,47 +46,22 @@ RUN echo "gate config verify, epoch ${CHECK_EPOCH}" && \
RUN echo "gate lint, epoch ${CHECK_EPOCH}" && \ RUN echo "gate lint, epoch ${CHECK_EPOCH}" && \
golangci-lint run --config .golangci.yml ./... golangci-lint run --config .golangci.yml ./...
# Prettier stage: the prettier that formats this repository's Markdown,
# never installed on a host. script/fmt and script/fmt-check build this
# stage alone and run it with the repository mounted on /src. prettier
# is installed in /tools so that the repository, mounted or copied onto
# /src, cannot hide it.
# node:22-alpine, 2026-02-22
FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS prettier
WORKDIR /tools
# yarn.lock pins prettier by hash, and --frozen-lockfile fails rather
# than install anything yarn.lock does not name.
COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile
ENV PATH=/tools/node_modules/.bin:$PATH
WORKDIR /src
# Markdown stage: the Markdown half of `make fmt-check`, as a gate.
FROM prettier AS markdown
COPY . .
# Second per-stage declaration of the gate cache-buster and its check;
# see the lint stage above.
ARG CHECK_EPOCH
RUN if [ -z "${CHECK_EPOCH}" ]; then \
echo "CHECK_EPOCH is unset; build via script/cibuild or script/docker" >&2; \
exit 1; \
fi
RUN echo "gate prettier, epoch ${CHECK_EPOCH}" && \
prettier --check '**/*.md' --tab-width 4 --prose-wrap always
# Build stage # Build stage
# golang:1.25-alpine, 2026-07-23 # golang:1.25-alpine, 2026-07-23
FROM golang@sha256:56961d79ea8129efddcc0b8643fd8a5416b4e6228cfd477e3fd61deb2672c587 AS builder FROM golang@sha256:56961d79ea8129efddcc0b8643fd8a5416b4e6228cfd477e3fd61deb2672c587 AS builder
# We never build or run as root. Create an unprivileged user and point # We never build or run as root. Create an unprivileged user and point
# HOME and the build cache at its home so go build and go test can write # HOME and the build cache at its home so go build and go test can write
# it when we drop to it below. # it when we drop to it below. $GOPATH/bin is deliberately not on PATH:
# script/bootstrap no longer `go install`s anything (the linter runs
# from a pinned image, never from a host install), so nothing lands
# there and adding it would only widen what this image resolves.
# #
# The module cache stays at the base image's default /go/pkg/mod and # The module cache is kept outside that home, at the base image's
# belongs to root: script/bootstrap fills it as root. Do not move it # default /go/pkg/mod, and belongs to root: script/bootstrap fills it as
# into the home and hand it over with `chown -R`: that walks every file # root. Do not move it into the home and hand it over with `chown -R`:
# in it, which took from about 80 s to over ten minutes on a shared # that walks every file in it, which took from about 80 s to over ten
# host, depending on load. # minutes on a shared host, depending on load.
RUN adduser -D -u 1000 builder RUN adduser -D -u 1000 builder
ENV HOME=/home/builder ENV HOME=/home/builder
ENV GOPATH=/home/builder/go ENV GOPATH=/home/builder/go
@@ -97,16 +70,22 @@ ENV GOCACHE=/home/builder/.cache/go-build
WORKDIR /src WORKDIR /src
# No-op file copies whose only purpose is the build-graph edge: they # No-op file copy whose only purpose is the build-graph edge: it is what
# make this stage depend on the lint and markdown stages, so BuildKit # makes this stage depend on the lint stage, and so what forces BuildKit
# finishes those gates before compilation and tests start. Remove one # to finish fmt-check, the pin guard and lint before compilation and
# and the build silently stops gating on that stage and still exits 0. # tests start. Remove it and the fail-fast design dies silently — the
# build stops gating on lint and still exits 0. It replaces a copy of
# the linter binary itself, which is no longer wanted here: nothing in
# this stage runs the linter, because `make lint` is now a docker build
# and a docker build cannot run inside one.
COPY --from=lint /src/go.sum /dev/null COPY --from=lint /src/go.sum /dev/null
COPY --from=markdown /src/go.sum /dev/null
# Install development prerequisites the same way a developer does. Only # Install development prerequisites the same way a developer does,
# script/ and the dependency manifests are copied first, so this layer # rather than duplicating the installs inline. Only script/ and the
# stays cached until they change. Bootstrap ends in `go mod download`. # dependency manifests are copied first, nothing else, so this layer
# stays cached until the scripts or the dependencies change — bootstrap
# ends in `go mod download`, which is why there is no separate
# invocation of it here.
COPY script/ script/ COPY script/ script/
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN script/bootstrap RUN script/bootstrap
@@ -129,23 +108,23 @@ RUN chown builder:builder /src /go/pkg/mod/cache/download && \
COPY --chown=builder:builder . . COPY --chown=builder:builder . .
USER builder USER builder
# Fail the build unless the branch is green. Runs as non-root: root # Fail the build unless the branch is green. Runs as non-root so the
# would bypass the chmod(0) the permission-denied tests rely on. # permission-denied test paths are exercised legitimately (root would
# bypass the chmod(0) the tests rely on).
# #
# The gate is `make test`, not `make check`, which runs docker; lint and # The gates are the individual targets, not `make check`: that aggregate
# the format checks ran in the lint and markdown stages above. `make`, # runs `script/lint`, which is now a docker build, and nothing inside an
# not the script directly, because the Makefile's # image build may shell out to docker. Lint is not skipped by this — it
# ran in the lint stage above, which this stage's COPY --from makes a
# prerequisite. `make`, not the scripts directly, because the Makefile's
# `export CGO_ENABLED = 0` applies only to what it invokes. # `export CGO_ENABLED = 0` applies only to what it invokes.
# #
# Third per-stage declaration of the gate cache-buster and its check; # Second per-stage declaration of the gate cache-buster; see the lint
# see the lint stage above. It is placed after USER so the drop to the # stage above for why one is not enough. It is placed after USER so the
# unprivileged user still happens before the checks run. # drop to the unprivileged user still happens before the checks run.
ARG CHECK_EPOCH ARG CHECK_EPOCH
RUN if [ -z "${CHECK_EPOCH}" ]; then \
echo "CHECK_EPOCH is unset; build via script/cibuild or script/docker" >&2; \
exit 1; \
fi
RUN echo "gate test, epoch ${CHECK_EPOCH}" && make test RUN echo "gate test, epoch ${CHECK_EPOCH}" && make test
RUN echo "gate fmt-check, epoch ${CHECK_EPOCH}" && make fmt-check
# The version stamped into the binary: the VERSION build argument when # The version stamped into the binary: the VERSION build argument when
# one is given, otherwise `git describe --tags --always` of the .git in # one is given, otherwise `git describe --tags --always` of the .git in
+35 -21
View File
@@ -1,12 +1,14 @@
# Lint-only image, built by script/lint: the repo is copied into the # Lint-only image: this is how the linter runs, everywhere. The repo is
# pinned golangci-lint image and the linter runs as a build step, so a # COPYed into the pinned golangci-lint image and the linter runs as a
# successful build is a clean lint. No bind mount, so it works when the # build step, so a successful build IS a clean lint. golangci-lint is
# docker daemon is remote. # never installed on a host — one toolchain, pinned by digest, identical
# on a laptop and in CI — and this works even when the docker daemon is
# remote and bind mounts are impossible.
# #
# It is separate from the main Dockerfile's lint stage because # script/lint builds this file. It is a separate image from the lint
# script/lint must not depend on the rest of that build; the two FROM # stage of the main Dockerfile because script/lint must not depend on
# lines are kept identical by script/verify-lint-image-pin, run as a # the rest of that build; the two FROM lines are kept identical by
# gate below. # script/verify-lint-image-pin, run as a gate below.
# golangci/golangci-lint:v2.12.2, 2026-08-07 # golangci/golangci-lint:v2.12.2, 2026-08-07
FROM golangci/golangci-lint@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240 FROM golangci/golangci-lint@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240
@@ -18,26 +20,38 @@ RUN go mod download
COPY . . COPY . .
# Cache-buster for the gate layers, and only for them; caching of the # Cache-buster for the gate layers, and only for them. Caching of the
# lint run is waived by ruling. On an unchanged tree the gates below # lint run is waived by ruling: COPY is invalidated only by changed
# would be served from cache and this build would exit 0 in under a # content, so on an unchanged tree the gates below would be served from
# second having run no linter. script/lint passes a fresh value on every # cache and this build would exit 0 in under a second having run no
# linter at all. That exact false green has bitten this repo twice
# already (#32, #39). script/lint passes a fresh value on every
# invocation. # invocation.
# #
# Each gate RUN must reference the value: BuildKit hashes the expanded # Each gate RUN must reference the value, because BuildKit hashes the
# command, so a declared but unreferenced ARG invalidates nothing. Keep # expanded command and not the ARG declaration: a declared but
# it below the dependency layers so they stay cached. # unreferenced ARG invalidates nothing. The ARG sits below the
# dependency layers deliberately — everything above it keeps its cache,
# only the gates go cold.
ARG CHECK_EPOCH ARG CHECK_EPOCH
# Fails the build when the FROM above and the main Dockerfile's lint # The linter version is pinned in two places, here and in the main
# stage pin different linter images. # Dockerfile's lint stage. Nothing else keeps them in sync, so a
# half-applied bump is a build failure; see the script.
RUN echo "gate lint-image-pin, epoch ${CHECK_EPOCH}" && \ RUN echo "gate lint-image-pin, epoch ${CHECK_EPOCH}" && \
script/verify-lint-image-pin script/verify-lint-image-pin
# Validates .golangci.yml against golangci-lint's JSON schema, which the # Validates .golangci.yml against golangci-lint's JSON schema. The
# pinned binary embeds: measured under `--network none`, it passes a # concern about this step was that it fetches that schema over a live,
# valid config and rejects an invalid one. No gate step makes a network # unpinned HTTPS call; measured on the pinned image, it does not. The
# call, but `go mod download` above needs the network on a cold cache. # binary carries the schema for its own version, so under
# `--network none` this both passes on a valid config and still rejects
# an invalid one with the jsonschema error. That holds for the gate
# steps generally — none of them makes a network call — but not for
# this build as a whole: `go mod download` above needs the network on a
# cold cache, and under `--network none` a first build fails there
# before reaching any gate. That layer stays cached, so only a warm
# cache lints offline, until go.mod or go.sum changes.
RUN echo "gate config verify, epoch ${CHECK_EPOCH}" && \ RUN echo "gate config verify, epoch ${CHECK_EPOCH}" && \
golangci-lint config verify --config .golangci.yml golangci-lint config verify --config .golangci.yml
+1 -4
View File
@@ -6,7 +6,7 @@ BINARY := sfdupes
VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo dev) VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo dev)
LDFLAGS := -X main.Version=$(VERSION) LDFLAGS := -X main.Version=$(VERSION)
.PHONY: sfdupes build bootstrap setup test test-race lint fmt fmt-check check docker hooks clean .PHONY: sfdupes build bootstrap setup test lint fmt fmt-check check docker hooks clean
# Standard targets are thin shims; the implementations live in script/ # Standard targets are thin shims; the implementations live in script/
# per the scripts-to-rule-them-all pattern. # per the scripts-to-rule-them-all pattern.
@@ -27,9 +27,6 @@ setup:
test: test:
@script/test @script/test
test-race:
@script/test-race
lint: lint:
@script/lint @script/lint
+739 -674
View File
File diff suppressed because it is too large Load Diff
+455 -312
View File
@@ -1,75 +1,60 @@
# Workflow # Workflow
- take an issue from the `1.0.0` milestone on the tracker; work not yet on the - take an issue from the `1.0.0` milestone on the tracker; work not
tracker gets filed as an issue first yet on the tracker gets filed as an issue first
- branch from `next` - branch (from `main`)
- do the work, with tests, in small focused commits - do the work, with tests, in small focused commits
- record it at the top of Completed Steps (`TODO.md` changes in the same commit - record it at the top of Completed Steps (`TODO.md` changes in the
as the work) same commit as the work)
- push the branch and open a PR against `next` whose title ends with - push the branch and open a PR whose title ends with
` (closes #N)` ` (closes #N)`
- an independent review gates each merge to `next`; every finding is addressed - an independent review gates the merge; every finding is addressed
or explicitly rebutted on the PR or explicitly rebutted on the PR
- only the owner merges `next` to `main` - merge to `main` once the review passes
# Status # Status
- pre-1.0 - pre-1.0
- the Gitea tracker is authoritative for the pre-1.0 backlog: the open issues - the Gitea tracker is authoritative for the pre-1.0 backlog: the
under the `1.0.0` milestone are what remains before the tag, and this file open issues under the `1.0.0` milestone are what remains before
records history and process, not the queue the tag, and this file records history and process, not the queue
# Next Step # Next Step
- take the next issue from the `1.0.0` milestone on the tracker: - take the next issue from the `1.0.0` milestone on the tracker:
https://git.eeqj.de/sneak/sfdupes/milestone/17 — the milestone is the source https://git.eeqj.de/sneak/sfdupes/milestone/17 — the milestone is
of truth for what is left before 1.0.0. Individual issues are deliberately not the source of truth for what is left before 1.0.0. Individual
restated here; a copy in this file drifts out of date the moment the tracker issues are deliberately not restated here; a copy in this file
moves drifts out of date the moment the tracker moves
# Completed Steps # Completed Steps
- cut the narration from `TODO.md` Completed Steps and from the comments in - `script/lint` writes no image, so a run no longer leaves an untagged
`script/` and both Dockerfiles; §Workflow now branches from and merges to one behind (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/48)
`next` (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/49)
- `make test-race` runs the test suite under the race detector in a cgo-enabled - tests cover a missing database, `scan` keeping stdout empty, its skip
container, outside `make check` (2026-10-04, warning, the `report` and `trees` summary lines, and every subcommand
https://git.eeqj.de/sneak/sfdupes/issues/18) going through `runE` (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/16)
- a bare `docker build .` fails with a message naming `script/cibuild` and
`script/docker` instead of serving the gates from cache (2026-10-04,
https://git.eeqj.de/sneak/sfdupes/issues/39)
- `make fmt` and `make fmt-check` run prettier over all Markdown, in Docker, and
CI checks it; all Markdown reformatted (2026-10-04,
https://git.eeqj.de/sneak/sfdupes/issues/19)
- `script/lint` writes no image, so a run no longer leaves an untagged one
behind (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/48)
- tests cover a missing database, `scan` keeping stdout empty, its skip warning,
the `report` and `trees` summary lines, and every subcommand going through
`runE` (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/16)
- `.golangci.yml` replaced with the current canonical copy, which uses - `.golangci.yml` replaced with the current canonical copy, which uses
`gomodguard_v2`, so lint no longer prints a deprecation warning (2026-10-04, `gomodguard_v2`, so lint no longer prints a deprecation warning
https://git.eeqj.de/sneak/sfdupes/issues/26) (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/26)
- a test fails when either `hashWorker` cancellation check in `scan.go` is - a test fails when either `hashWorker` cancellation check in `scan.go`
removed (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/83) is removed (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/83)
- a database path holding `?`, `#` or `%` opens exactly the file it names - a database path holding `?`, `#` or `%` opens exactly the file it names
(2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/55) (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/55)
- `scan` rejects `--workers` below 1 as a usage error instead of running - `scan` rejects `--workers` below 1 as a usage error instead of
single-threaded (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/10) running single-threaded (2026-10-04,
https://git.eeqj.de/sneak/sfdupes/issues/10)
- a test fails when either walk cancellation check in `scan.go` is removed - a test fails when either walk cancellation check in `scan.go` is
(2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/81) removed (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/81)
- test that `scan` refuses a database with another schema version (2026-10-04, - test that `scan` refuses a database with another schema version
https://git.eeqj.de/sneak/sfdupes/issues/64) (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/64)
- correct four inaccurate comments in `cancel_test.go` and rename - correct four inaccurate comments in `cancel_test.go` and rename
`walkCancelInFlightDirs` to `walkCancelInFlightFiles` (2026-10-04, `walkCancelInFlightDirs` to `walkCancelInFlightFiles` (2026-10-04,
@@ -82,321 +67,479 @@
`files` table is refused with a clear schema-version error (2026-10-04, `files` table is refused with a clear schema-version error (2026-10-04,
https://git.eeqj.de/sneak/sfdupes/issues/11) https://git.eeqj.de/sneak/sfdupes/issues/11)
- README documents install, Docker, a daily cron scan and how to read and check - README documents install, Docker, a daily cron scan and how to read
the reports (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/54) and check the reports (2026-10-04,
https://git.eeqj.de/sneak/sfdupes/issues/54)
- the `Dockerfile` build stage keeps the Go module cache out of `builder`'s home - the `Dockerfile` build stage keeps the Go module cache out of `builder`'s
and copies the sources with `--chown`, so no `chown -R` walks them home and copies the sources with `--chown`, so no `chown -R` walks them
(2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/43) (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/43)
- `--version` prints `sfdupes VERSION` to stdout; README documents it and - `--version` prints `sfdupes VERSION` to stdout; README documents it and
`--help` (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/15) `--help` (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/15)
- `scan` stops cleanly on `SIGINT` or `SIGTERM`: commits what it has hashed, - `scan` stops cleanly on `SIGINT` or `SIGTERM`: commits what it has
deletes nothing more, exits 1 (2026-10-04, hashed, deletes nothing more, exits 1 (2026-10-04,
https://git.eeqj.de/sneak/sfdupes/issues/5) https://git.eeqj.de/sneak/sfdupes/issues/5)
- `report` and `trees` stream the records instead of holding them all in memory; - `report` and `trees` stream the records instead of holding them all in
the schema gains the `files_signature` index (2026-10-04, memory; the schema gains the `files_signature` index (2026-10-04,
https://git.eeqj.de/sneak/sfdupes/issues/14) https://git.eeqj.de/sneak/sfdupes/issues/14)
- progress prints at once on a non-terminal, uses a real terminal test, and - progress prints at once on a non-terminal, uses a real terminal test,
prints warnings through a spinner instead of racing its redraw (2026-10-03, and prints warnings through a spinner instead of racing its redraw
https://git.eeqj.de/sneak/sfdupes/issues/13) (2026-10-03, https://git.eeqj.de/sneak/sfdupes/issues/13)
- warn about and skip symlink, socket, FIFO, device and `.zfs` operands, keeping - warn about and skip symlink, socket, FIFO, device and `.zfs`
the records beneath them (2026-10-03, operands, keeping the records beneath them (2026-10-03,
https://git.eeqj.de/sneak/sfdupes/issues/9) https://git.eeqj.de/sneak/sfdupes/issues/9)
- `scan` holds a lock on a lock file beside the database for its whole run, so a - `scan` holds a lock on a lock file beside the database for its whole run,
second `scan` fails at once with exit 1 (2026-10-03, so a second `scan` fails at once with exit 1 (2026-10-03,
https://git.eeqj.de/sneak/sfdupes/issues/53) https://git.eeqj.de/sneak/sfdupes/issues/53)
- test stdout write failures in `report` and `trees`; README states that - test stdout write failures in `report` and `trees`; README states that
`| head` ends sfdupes by `SIGPIPE` and `>&-` writes to `/dev/null` `| head` ends sfdupes by `SIGPIPE` and `>&-` writes to `/dev/null`
(2026-10-03, https://git.eeqj.de/sneak/sfdupes/issues/30) (2026-10-03, https://git.eeqj.de/sneak/sfdupes/issues/30)
- `report` and `trees` open the database read-only, and `scan` leaves it out of - `report` and `trees` open the database read-only, and `scan` leaves it
WAL mode, so reading needs only read access (2026-10-03, closes out of WAL mode, so reading needs only read access (2026-10-03, closes
https://git.eeqj.de/sneak/sfdupes/issues/8) https://git.eeqj.de/sneak/sfdupes/issues/8)
- escape tabs, newlines, carriage returns and backslashes in report, trees and - escape tabs, newlines, carriage returns and backslashes in report,
warning paths; the root directory's path is `/` (2026-10-03, trees and warning paths; the root directory's path is `/`
https://git.eeqj.de/sneak/sfdupes/issues/7) (2026-10-03, https://git.eeqj.de/sneak/sfdupes/issues/7)
- stamp the git tag or short commit in a plain `docker build .` instead of `dev` - stamp the git tag or short commit in a plain `docker build .`
(2026-10-02, branch `next`, closes instead of `dev` (2026-10-02, branch `next`, closes
https://git.eeqj.de/sneak/sfdupes/issues/67): `.dockerignore` sends `.git` https://git.eeqj.de/sneak/sfdupes/issues/67): `.dockerignore` now
without `.git/config`; the build stage stamps the `VERSION` build argument, sends `.git`, without `.git/config`, and the `Dockerfile` build
else `git describe --tags --always`, and fails if the context carries `.git` stage takes the `VERSION` build argument when one is given,
and the version is still empty, `dev` or `unknown`. CI checks out the full otherwise `git describe --tags --always` of that `.git`. The build
history (`fetch-depth: 0`) so it stamps the same value as `make build`. fails if the context carries `.git` and the version still comes out
empty, `dev` or `unknown`. The CI checkout step fetches the full
history (`fetch-depth: 0`) so CI sees the tag and stamps the same
value as `make build`.
- replace the 1 KiB end-window sampling with the head/tail plus content-hash - replace the 1 KiB end-window sampling with the head/tail plus
ladder (2026-09-22, branch `next`, closes content-hash ladder (2026-09-22, branch `next`, closes
https://git.eeqj.de/sneak/sfdupes/issues/61); README "Duplicate detection" https://git.eeqj.de/sneak/sfdupes/issues/61): a file under 10 MiB is
documents every rung. A file under 10 MiB is hashed in full, and its `head`, hashed in full and compared directly, with no end-window step — its
`tail` and `content` all hold that hash. A larger file gets only its 64 KiB `head`, `tail`, and `content` all hold the whole-file hash. A file at
`head` and `tail` in the hash phase; the content phase, after the update 10 MiB or above gets only the 64 KiB `head` and `tail` in the hash
phase, reads it for `content` (the whole file below 50 MiB, gigabyte-spaced 1 phase; a new content phase, after the update phase, reads it for its
MiB samples at or above) only when its size, `head` and `tail` match another `content` hash — the whole file below 50 MiB, gigabyte-spaced 1 MiB
record's from this scan or an earlier one, and never reads a file gone or samples at or above — only when its size, `head`, and `tail` match
changed since its record was written. `report` and `trees` leave out any another record's, from the same scan or stored by an earlier one, so
record without a `content` hash. The `content` column is part of the version 1 a stored file gains its content hash when it gains a match. A file
schema. that is gone or has changed since its record was written is not
read. The `content` column is part of the version 1 schema. `report`
and `trees` group by the extended signature and leave out any record
without a `content` hash, so the ladder is applied across the whole
database. README "Duplicate detection" documents every rung including
the probabilistic large-file path.
- remove the dead `files.dat` references from `Makefile`, `.gitignore` and - remove the dead `files.dat` references from `Makefile`, `.gitignore`
`.dockerignore` (2026-09-21, branch `next`, closes and `.dockerignore` (2026-09-21, branch `next`, closes
https://git.eeqj.de/sneak/sfdupes/issues/22) https://git.eeqj.de/sneak/sfdupes/issues/22)
- fix the lint-image pin comments and `FROM` form in `Dockerfile` and - fix the lint-image pin comments and `FROM` form in `Dockerfile` and
`Dockerfile.lint` (2026-08-10, branch `next`, closes `Dockerfile.lint` (2026-08-10, branch `next`, closes
https://git.eeqj.de/sneak/sfdupes/issues/25): both pins are now the policy https://git.eeqj.de/sneak/sfdupes/issues/25): dropped the false
`# image:vX.Y.Z, YYYY-MM-DD` comment over a bare `FROM image@sha256:...`, `(Debian-based)` parenthetical (v2.12.1 was Debian too) and the
without the false `(Debian-based)` note or the tag; digest unchanged. redundant tag, so both pins are the policy `# image:vX.Y.Z,
`script/verify-lint-image-pin` still matches the tagless form, and a tag on YYYY-MM-DD` comment over a bare `FROM image@sha256:...`. Digest
one side only is caught as a plain mismatch. unchanged. `script/verify-lint-image-pin` parses those `FROM` lines
and still matches the tagless form; its advice line lost the now
meaningless "tag and digest". With no tag in either reference, a
tag-only disagreement no longer exists — a one-sided tag is caught as
a plain mismatch.
- run all linting in Docker via `Dockerfile.lint` and `script/lint` (2026-08-10, - run all linting in Docker via `Dockerfile.lint` and `script/lint`
branch `next`, closes https://git.eeqj.de/sneak/sfdupes/issues/46): per the (2026-08-10, branch `next`, closes
owner ruling the linter is never installed on a host. `Dockerfile.lint` copies https://git.eeqj.de/sneak/sfdupes/issues/46): per the owner ruling, the
the repo into the digest-pinned `golangci/golangci-lint:v2.12.2` image and linter runs inside a container invoked through the `script/`
runs `golangci-lint config verify` and `golangci-lint run` as build steps; entrypoint and is never installed on a host. New root
`script/lint` builds it. `script/bootstrap` no longer installs or pins the `Dockerfile.lint` COPYs the repo into the digest-pinned
linter, and warns rather than fails when `docker` is absent; `golangci/golangci-lint:v2.12.2` image and runs
`ENV PATH=/home/builder/go/bin:$PATH` went with its `go install`. `golangci-lint config verify` and `golangci-lint run` as build
`script/verify-linter-pin` is retired; `script/verify-lint-image-pin`, a gate steps, so a successful build IS a clean lint; `script/lint` is
in both files, compares their two `FROM` lines and restates neither pin. reduced to building it. `script/bootstrap` loses the `go install`,
Traps: an unchanged tree lets a lint build pass in under a second having run the pin constants, the version parser and `verify_golangci_lint`
no linter, so every gate `RUN` references `ARG CHECK_EPOCH` (BuildKit hashes outright rather than hardening them — with nothing linting on the
the expanded command) and `script/lint` passes `"$(date +%s)-$$"`, the PID host, the `$GOPATH/bin` versus `PATH` problem that motivated them has
because two runs land in the same second easily. Nothing inside an image build no subject — and now warns rather than fails when `docker` is absent.
may shell out to docker, so the `Dockerfile` lint stage calls `golangci-lint` Two traps handled. A lint build on an unchanged tree returns success
directly and the build stage runs `make test` and `make fmt-check` instead of in well under a second having run no linter, which is
`make check`, through `make` because the Makefile's `export CGO_ENABLED = 0` https://git.eeqj.de/sneak/sfdupes/issues/32 and
only reaches what it invokes. `COPY --from=lint /src/go.sum /dev/null` https://git.eeqj.de/sneak/sfdupes/issues/39 again, so
replaces the copied linter binary as the only edge making the build stage wait `Dockerfile.lint` carries `ARG CHECK_EPOCH` referenced
for lint; dropping it would end fail-fast linting under a still-green build. inside every gate `RUN` (BuildKit hashes the expanded command, not
`golangci-lint config verify`, included per the ruling, validates from an the declaration) and `script/lint` passes `"$(date +%s)-$$"` — the
embedded schema with no network call, but `go mod download` above the gates PID matters because two lint runs land inside the same second easily.
still needs the network on a cold cache. Verified: `make lint` green with no And nothing inside an image build may shell out to docker, so the
`golangci-lint` on `PATH`; two back-to-back `script/lint` runs on an untouched main `Dockerfile`'s lint stage now invokes `golangci-lint` directly
tree both ran the linter (27.7s and 28.7s in the lint step, `COPY . .` instead of `make lint`, and its build stage runs `make test` and
`CACHED` above); a planted unused variable failed `script/lint`, and failed `make fmt-check` instead of the `make check` aggregate (`make`, not
`make docker` at `[lint 9/9]` with the build stage stopped at the scripts bare, because the Makefile's `export CGO_ENABLED = 0`
`[builder 3/12]`; the drift guard fails on a tag-only, a digest-only and an only reaches what it invokes). `COPY --from=lint`
unreadable reference, naming both sides; under `--network none` config verify `/usr/bin/golangci-lint` is replaced by
passes a valid config and rejects an invalid one; `make docker` green in 5m35s `COPY --from=lint /src/go.sum /dev/null`: the copied binary was the
with all six gates run under one epoch (lint 37.6s, test 25.2s reporting only edge forcing BuildKit to finish linting before the build stage
`ok sneak.berlin/go/sfdupes 1.938s coverage: 88.5%`, not `(cached)`); in the starts, and dropping it without replacing the edge would have ended
builder image with the Go test cache off, `--user 0:0` still fails fail-fast linting silently under a still-green build. That is
`TestScanHardlinkRunFailsTogether` where the unprivileged user passes. Noted canonical `REPO_POLICIES.md:107`'s ordering edge, restored.
for follow-up, not fixed here: `golangci-lint` warns that `gomodguard` is `ENV PATH=/home/builder/go/bin:$PATH` is gone with the `go install`
deprecated since v2.12.0 in favour of `gomodguard_v2`. that justified it. `script/verify-linter-pin` is retired, deleted
along with its README entry, because both of its subjects ceased to
exist in the same change: it compared a linter binary against
`GOLANGCI_LINT_VERSION` in `script/bootstrap`, and there is now
neither a binary crossing between stages nor a version pin in
bootstrap. The drift it guarded has not gone away, it has moved — the
linter is still pinned twice, now as the `FROM` line of
`Dockerfile.lint` and the `FROM` line of the `Dockerfile` lint stage,
with nothing syncing them, which is exactly what
https://git.eeqj.de/sneak/sfdupes/issues/42 made a build failure. Its
replacement is one new `script/verify-lint-image-pin`,
run as a gate in both files, which compares the two references to
each other and deliberately restates neither: a hardcoded expected
digest would be a third copy and the same drift one file further out.
`golangci-lint config verify` is included per the ruling, and the
concern about its unpinned live HTTPS schema fetch was measured
rather than assumed — under `--network none` the pinned binary both
passes a valid config and rejects an invalid one with the jsonschema
error, so it validates from an embedded schema and makes no network
call of its own. The README scopes that to the gate steps rather
than to linting as a whole: `Dockerfile.lint` runs `go mod download`
above them, so a cold cache still needs the network and only a warm
one lints offline. Verified: `make lint` green with every `PATH`
directory containing a `golangci-lint` removed
(`/home/user/go/bin`, `/home/user/.local/bin`, `/usr/local/bin`;
`command -v golangci-lint` empty); two consecutive `script/lint` runs
on an untouched tree both executed the linter, 27.7s and 28.7s in the
lint step under distinct epochs with the `COPY . .` layer `CACHED`
above them, at 42.2s and 41.8s wall clock — the no-cache rule was not
weakened to shorten that. Negative control: a planted
`var unusedIssue46Sentinel = 1` failed `script/lint` with
`report.go:173:5: var unusedIssue46Sentinel is unused (unused)`, and
failed `make docker` at `[lint 9/9]` with the build stage stopped at
`[builder 3/12]` — `COPY --from=lint`, `script/bootstrap`, the test
gate and `make build` all zero occurrences — then reverted clean. The
drift guard fails on a tag-only disagreement, on a digest-only
disagreement, and on an unreadable reference, naming both sides.
`make docker` green in 5m35s with all six gates executing under one
epoch (lint 37.6s, test 25.2s reporting
`ok sneak.berlin/go/sfdupes 1.938s coverage: 88.5%`, not `(cached)`).
The non-root quirk still holds: in the builder image with the Go test
cache off, `--user 0:0` fails `TestScanHardlinkRunFailsTogether`
(exit 1) where the unprivileged user passes (exit 0). Noted for
follow-up, not fixed here: `golangci-lint` warns that the
`gomodguard` linter is deprecated since v2.12.0 in favour of
`gomodguard_v2`.
- install the Docker build stage's prerequisites by running `script/bootstrap` - install the Docker build stage's prerequisites by running
instead of `apk add --no-cache make` inline (2026-08-09, branch `script/bootstrap` instead of `apk add --no-cache make` inline
`dockerfile-bootstrap`, closes https://git.eeqj.de/sneak/sfdupes/issues/42): (2026-08-09, branch `dockerfile-bootstrap`, closes #42): canonical
the stage copies `script/` plus `go.mod`/`go.sum` and runs `script/bootstrap`, `REPO_POLICIES.md:97` requires it, and the inline install left the
which ends in `go mod download`, so the separate call to it is gone. build stage maintaining its own notion of the toolchain — exactly
`COPY --from=lint /usr/bin/golangci-lint` stays and moves above the bootstrap the divergence #24 exists to close, one layer down. The stage now
layer: it is the only edge making this stage depend on the lint stage, so copies `script/` plus `go.mod`/`go.sum` and runs `script/bootstrap`,
deleting it would end fail-fast linting silently. A new which ends in `go mod download`, so the separate invocation of that
`script/verify-linter-pin`, run in the build stage before bootstrap, fails the is gone. `COPY --from=lint /usr/bin/golangci-lint` stays, and moves
build naming both versions unless that copied binary is the version above the bootstrap layer. It is the only edge making this stage
`script/bootstrap` pins; a pin it cannot read is a hard failure, not a skip. depend on the lint stage, so deleting it as redundant would end
`$GOPATH/bin` joins `PATH`, where bootstrap's `go install` lands. Everything fail-fast linting silently. Letting bootstrap install its own linter
added sits above `ARG CHECK_EPOCH`, and the `chown` and `USER builder` still here would have reintroduced the second toolchain and paid for a
precede `make check`. Verified: the guard fails the build with both versions from-source build of it. What makes the two stages provably one
named when the lint stage's linter is faked to another version, and passes an toolchain rather than two that happen to agree is a new
unmodified build; bootstrap runs clean under Alpine's `sh` and `apk`, finding `script/verify-linter-pin`, run in the build stage on the binary
the copied linter already at the pin; a second build served the bootstrap and that arrives from the lint stage, before bootstrap: it fails the
dependency layers `CACHED` while both gates ran with a fresh epoch; a planted build naming both versions unless that binary is the version
`unused` finding failed the build at the lint gate in 48.9s with the build `script/bootstrap` pins. Bootstrap's own check could not serve that
stage's `make check` never starting; and the suite run in the image as purpose — it reinstalls its pin from source and then verifies
`--user 0:0` fails `TestScanHardlinkRunFailsTogether`, so the drop to the whatever `PATH` resolves, so drift self-heals silently and a lint
unprivileged user is still needed. That last check needs the Go test cache stage image bumped on its own would lint at the new version while
off: as root it first reported `ok ... (cached)`, reusing the build-time `make check` ran at the old one, green. The linter version is pinned
result. Build times on a noisy shared host: 2m13s on an unchanged tree, 2m17s in two independent places (the lint stage image digest and
and 4m29s after a source change, 5m14s cold, which breaches the policy `GOLANGCI_LINT_VERSION`) and nothing else keeps them in sync, so a
ceiling; `chown -R builder:builder /src /home/builder` walks the module cache half-applied bump is now a build failure. The pin is read out of
and alone varied from 77s to 210s across those builds, and `main` measured `script/bootstrap`, which stays the single source of truth; a pin
5m03s cold with a 209s `chown`. Filed as that cannot be read is a hard failure, not a skip. The check needs
https://git.eeqj.de/sneak/sfdupes/issues/43 no `CHECK_EPOCH`: its only inputs are the copied binary and
- bust the Docker layer cache for the gate steps, so `script/cibuild` and `script/`, so Docker invalidates the layer exactly when a cached
`script/docker` cannot report a green they did not earn (2026-08-09, branch result would stop being true, and it is documented with the other
`cibuild-cache-bust`, closes https://git.eeqj.de/sneak/sfdupes/issues/32): the entrypoints in the README. `$GOPATH/bin` joins `PATH` because
`Dockerfile` copies the tree before its gates, so on an unchanged tree Docker that is where bootstrap's `go install` lands and bootstrap verifies
served them from cache and the build exited 0 having run nothing. Both scripts its installs against what `PATH` resolves — nothing in the image is
now pass `--build-arg CHECK_EPOCH="$(date +%s)"`. `ARG` is per stage and the shadowed by it, the directory does not exist until bootstrap runs.
gates span two stages, so it is declared in both; BuildKit hashes the expanded Everything added sits above `ARG CHECK_EPOCH`, and the `chown` and
command, so each gate `RUN` echoes the epoch, which also logs it as evidence `USER builder` still precede `make check`. Verified: the guard fails
the layer ran. It sits below the dependency layers so they stay cached. the build with both versions named when the lint stage's linter is
Verified under `BUILDKIT_PROGRESS=plain`, each script run twice back to back faked to a different version, and an unmodified build still passes
on an unchanged tree: all three gates ran on all four runs with a fresh epoch it; bootstrap runs clean under Alpine's `sh` and its `apk` branch,
(`script/cibuild` 78.8s then 61.1s; `script/docker` 61.1s then 53.4s), and installing `git` and `make` and finding the copied
thirteen steps were still served `CACHED`. With a planted `unused` finding the linter already at the pin; a second build served the bootstrap and
build failed at `make lint` in 36.1s and the build-stage `make check` never dependency layers `CACHED` while both gates ran with a fresh epoch;
started. Run as root, the same image fails `TestScanHardlinkRunFailsTogether`, a planted `unused` finding failed the build at the lint gate in
because root reads through the `chmod(0)` the test relies on, so the build 48.9s with the build stage's `make check` never starting; and the
stage must drop to the unprivileged `builder` user. Local fix only; suite run in the image as `--user 0:0` fails
propagating it to the canonical templates is `TestScanHardlinkRunFailsTogether`, so the drop to the unprivileged
https://git.eeqj.de/sneak/prompts/issues/26 user is still load-bearing. That last check needs the Go test cache
- check the installed golangci-lint version in `script/bootstrap` instead of disabled — the first attempt reported `ok ... (cached)` as root,
only its presence (2026-08-09, branch `bootstrap-version-check`, closes reusing the result the build-time run had left in the shared cache,
https://git.eeqj.de/sneak/sfdupes/issues/24): the version lives only in which would have read as a pass. Build wall time, on a shared host
`GOLANGCI_LINT_VERSION`, with the `go install` module ref derived from it, and running many concurrent builds and so noisy: 2m13s on an unchanged
any installed version that is not the pin — older, newer, absent or tree, 2m17s and 4m29s for two builds after a source change, 5m14s
unparseable — is reinstalled. `go install` writes into `GOBIN` (or cold. Only the cold one breaches the policy ceiling, and not because
`GOPATH/bin`) while `make lint` runs the first `golangci-lint` on `PATH`, so of this change — `chown -R builder:builder /src /home/builder` walks
bootstrap re-reads the effective version after installing and, on a mismatch, the module cache and re-runs on every source change, and it alone
prints both paths and both versions and exits non-zero; it does not reorder varied between 77s and 210s across those four builds, which is also
`PATH` or delete anyone's binary. The `--version` call keeps its stderr and is the whole spread in the totals. The same cold measurement against
bounded by `timeout(1)` where that exists. `git`, `make` and `go` keep `main` is 5m03s with a 209s `chown`. Filed as #43
presence-only checks. Verified by bootstrapping this host from v2.10.1 to - bust the Docker layer cache for the gate steps, so `script/cibuild`
v2.12.2 and again to a no-op, and by stub runs of the script under `dash` and `script/docker` cannot report a green they did not earn
covering a thirteen-input version-parse matrix, a shadowed install that must (2026-08-09, branch `cibuild-cache-bust`, closes #32): both scripts
exit non-zero, an install destination not on `PATH`, `GOBIN` set, and a wedged were bare `docker build` invocations with no cache control, and the
binary that must hit the timeout; `make check` and `make lint` are clean at `Dockerfile` copies the tree before running its gates, so on an
v2.12.2, so v2.10.1 was not hiding any findings on `main` unchanged tree Docker served those layers from cache and the build
exited 0 having executed nothing. That is not hypothetical here —
every merge this repo has done is a non-fast-forward merge of an
undiverged branch, so each merge commit's tree is byte-identical to
the branch head's and each merge CI run was almost certainly a full
cache hit; and PR #31's reviewer found `make docker` returning
success as a 17-layer cache hit, catching it only by being
suspicious. The fix is `ARG CHECK_EPOCH` with the scripts passing
`--build-arg CHECK_EPOCH="$(date +%s)"`. Two details make or break
it. `ARG` is scoped per stage and this `Dockerfile` has three gates
across two — `make fmt-check` and `make lint` in the lint stage,
`make check` in the build stage — so a single declaration would have
left one stage silently cacheable; it is declared in both. And
BuildKit hashes the expanded command, not the declaration, so a
declared-but-unreferenced `ARG` invalidates nothing: each gate `RUN`
echoes the epoch, which also puts the value in the build log as
evidence the layer really ran. Placement is below the dependency
layers on purpose — a build that goes cold every time would be a
different bug, not a fix. Verified by running each script twice back
to back on an unchanged tree under `BUILDKIT_PROGRESS=plain`: all
three gates executed on all four runs, each with a fresh epoch in
the log (`script/cibuild` 78.8s then 61.1s; `script/docker` 61.1s
then 53.4s), and twelve steps were still served `CACHED` in the
steady state — both `go mod download`s, `apk add`, `adduser`, the
`chown`, every `go.mod`/`go.sum` and source copy, the linter copy
out of the lint stage, and the binary copy into the runtime stage.
The lint stage still gates the build stage: with a deliberate
`unused` finding planted in the tree, the build failed at
`make lint` in 36.1s and the build-stage `make check` never started.
The build stage also still drops to the unprivileged `builder` user
before `make check`, which the suite depends on rather than merely
prefers: forcing the same image to run the tests as root fails
`TestScanHardlinkRunFailsTogether`, because root reads straight
through the `chmod(0)` the test uses to prove hard links are read
once. This is the local fix only; propagating it to the canonical
templates is `prompts` #26
- check the installed golangci-lint version in `script/bootstrap`
instead of only its presence (2026-08-09, branch
`bootstrap-version-check`, closes #24): `missing golangci-lint` meant
any linter already on `PATH` satisfied the check, so the pin was never
consulted and the v2.12.2 bump from #3 was inert on every host that
already had one — this host ran v2.10.1 against a v2.12.2 pin,
`make check` went green, and `make docker` then rejected the same
commit with findings the local gate never saw. The version now lives
in one place, `GOLANGCI_LINT_VERSION`, with the `go install` module
ref derived from it so a bump cannot half-apply; a
`golangci_lint_version` helper parses `golangci-lint --version`
(taking the field after the word `version` and tolerating an optional
leading `v`, which the module ref carries and the binary's output does
not), and any version that is not the pin — older, newer, absent or
unparseable — is reinstalled. The install is then verified against the
binary `PATH` actually resolves: `go install` writes into `GOBIN` (or
`GOPATH/bin`) while `make lint` runs whichever `golangci-lint` comes
first on `PATH`, so a wrong-version one sitting ahead of it — nix,
apt, brew, apk, or the `/usr/local/bin` copy the `Dockerfile` builder
stage makes — would swallow the install and leave the local gate
disagreeing with CI under an affirmative `bootstrap complete`.
Bootstrap now re-reads the effective version after installing and, on
a mismatch, prints both paths and both versions to stderr and exits
non-zero instead of claiming success; it does not reorder anyone's
`PATH` or delete their binary. The `--version` call keeps its stderr
connected, so a present-but-broken binary says why rather than
reinstalling forever in silence, and is bounded by `timeout(1)` where
that exists, so a wedged binary cannot hang bootstrap. `git`, `make`
and `go` keep their presence-only checks and now say why in a
comment: they are host package-manager tools the repo deliberately
does not pin, with `go.mod` governing the language version and the
digest-pinned images covering reproducible builds. Verified on this
host by bootstrapping from v2.10.1 to v2.12.2 and running it again to
a no-op, plus stub runs of the real script under `dash` covering a
thirteen-input parse matrix (absent, older, newer, host-style,
image-style, leading-`v`, stderr-only, empty, non-zero exit, impostor
binary, `(devel)`, trailing `version`), a shadowed install that must
exit non-zero, an install destination not on `PATH` at all, `GOBIN`
set, and a wedged binary that must hit the timeout; `make check` and
`make lint` are clean at v2.12.2, so v2.10.1 was not hiding any
findings on `main`
- unwind the hash worker pool on the error path (2026-08-09, branch - unwind the hash worker pool on the error path (2026-08-09, branch
`hash-pool-cleanup`, closes https://git.eeqj.de/sneak/sfdupes/issues/6): the `hash-pool-cleanup`, closes #6): `hashPhase` used to return the
pool is now an owned, context-aware `hashPool`: every blocking send in the moment `recordRun` failed and abandon the pool — the feeder parked
feeder and the workers selects on `ctx.Done()`, `jobs` is closed on every path forever on a full `jobs` channel and every worker on a full
out, and `hashPhase` defers `pool.stop()`, which cancels and then drains `results` channel. That only stopped being invisible when #4 landed
`results` until the last goroutine has exited — draining is what frees a and `runScan` began unwinding instead of calling `os.Exit`. The
worker already parked on a send. `ctx` is threaded from `cmd.Context()` pool is now an owned, context-aware `hashPool`: every blocking send
through `runScan`, `syncScan`, both worker pools and the whole database layer, in the feeder and the workers selects on `ctx.Done()`, `jobs` is
as the first parameter everywhere. The walk pool gets the same treatment plus closed on every path out, and `hashPhase` defers `pool.stop()`,
a `ctx.Err()` guard after the walk: a cancelled walk yields a partial size which cancels and then drains `results` until the last goroutine
census, and every file it never reached looks vanished to the update phase. has exited — draining is what frees a worker already parked on a
That phase's own `BeginTx` also fails on the cancelled context before deleting send. `ctx` is threaded from `cmd.Context()` through `runScan`,
anything, but the guard is the barrier that still holds once an interrupted `syncScan`, both worker pools and the whole database layer (it is
scan may commit what it has. Tests drive `run(scan)` against a database whose the first parameter everywhere), so #5 can hand this path a signal
insert trigger aborts and assert that the scan fails instead of hanging and and needs to add nothing else. The walk pool never leaked, because
that `runtime.NumGoroutine()` polls back to its pre-scan baseline; others `walkPhase` always drains its events to close, but it has the same
cancel a scan part-way through the walk, deterministically, by counting its unbounded-send shape and #5 will give it an early return, so it
own consultations of `ctx.Done()`, and assert that it stops at the guard gets the same treatment plus a `ctx.Err()` guard after the walk: a
holding a partial census and a still-populated record index, with every record cancelled walk yields a partial size census, and every file it never
intact. Direct tests of `sendEvent`, the walk workers, `dispatchDirs`, reached looks vanished to the update phase. That phase's own
`feedHashJobs`, `hashWorker` and `hashPhase` cover the remaining cancellation `BeginTx` fails on the same cancelled context before deleting
branches of both pools anything, so the guard is defence in depth rather than the only
- guarantee the database is closed on every fatal exit path (2026-08-09, branch barrier — but it is the one that survives #5 deciding an interrupted
`db-close-on-fatal`, closes https://git.eeqj.de/sneak/sfdupes/issues/4): scan may commit what it has. Tests drive `run(scan)` against a
`fatalf` and its `os.Exit(1)` are gone, so the deferred `db.Close()` — and database whose insert trigger aborts, and assert both that the scan
with it the SQLite WAL checkpoint — now actually runs when a subcommand fails; fails instead of hanging and that `runtime.NumGoroutine()` polls
`runScan`, `runReport`, `runTrees`, `loadRecords` and `resolveRoots` return back to its pre-scan baseline; a second set cancels a scan part-way
errors instead. The single exit point is `run` in `main.go`: it maps a through the walk — deterministically, by counting the scan's own
`fatalError` (anything a subcommand returned) to exit 1 and cobra's own consultations of `ctx.Done()` rather than racing a timer — and
argument and flag errors to exit 2, which keeps a runtime failure from being asserts that it stops at the guard holding a partial census and a
reported as a usage error or printing the usage text. New `main_test.go` still-populated record index, with every record intact. The
drives the CLI in-process and asserts the exit codes from README §Error remaining cancellation branches of both pools are covered by direct
handling plus the stdout/stderr split, including that a fatal error raised tests of `sendEvent`, the walk workers, `dispatchDirs`,
after the database is open leaves no `-wal`/`-shm` sidecar behind for `scan`, `feedHashJobs`, `hashWorker` and `hashPhase`
`report` or `trees` - guarantee the database is closed on every fatal exit path
- update golangci-lint to v2.12.2 with the canonical config (2026-08-09, branch (2026-08-09, branch `db-close-on-fatal`, closes #4): `fatalf` and
`golangci-v2.12.2`, merged as `38a01bd`, closes its `os.Exit(1)` are gone, so the deferred `db.Close()` — and with
https://git.eeqj.de/sneak/sfdupes/issues/3): bumped the pinned linter in the it the SQLite WAL checkpoint — now actually runs when a subcommand
`Dockerfile` lint stage and `script/bootstrap` from v2.12.1 to v2.12.2, and fails; `runScan`, `runReport`, `runTrees`, `loadRecords` and
replaced `.golangci.yml` with the canonical file — the linter settings (`lll`, `resolveRoots` return errors instead. The single exit point is `run`
`funlen`, `cyclop`, `dupl` thresholds) now live under `linters.settings` per in `main.go`: it maps a `fatalError` (anything a subcommand
the v2 schema, so they are actually applied; no new lint findings surfaced returned) to exit 1 and cobra's own argument and flag errors to exit
- convert Makefile targets to scripts-to-rule-them-all `script/` entrypoints 2, which keeps a runtime failure from being reported as a usage
like the other managed repos (2026-07-26, commit `3abeacf`, closes error or printing the usage text. New `main_test.go` drives the CLI
https://git.eeqj.de/sneak/sfdupes/issues/1): all 12 `script/` entrypoints in-process and asserts the exit codes from README §Error handling
exist (`bootstrap`, `setup`, `projectname`, `test`, `lint`, `fmt`, plus the stdout/stderr split, including that a fatal error raised
`fmt-check`, `check`, `docker`, `cibuild`, `precommit`, `install-precommit`) after the database is open leaves no `-wal`/`-shm` sidecar behind
and every Makefile target is now a thin shim over them for `scan`, `report` or `trees`
- update golangci-lint to v2.12.2 with the canonical config
(2026-08-09, branch `golangci-v2.12.2`, merged as `38a01bd`,
closes #3): bumped the pinned linter in the `Dockerfile` lint
stage and `script/bootstrap` from v2.12.1 to v2.12.2, and replaced
`.golangci.yml` with the canonical file — the linter settings
(`lll`, `funlen`, `cyclop`, `dupl` thresholds) now live under
`linters.settings` per the v2 schema, so they are actually
applied; no new lint findings surfaced
- convert Makefile targets to scripts-to-rule-them-all `script/`
entrypoints like the other managed repos (2026-07-26, commit
`3abeacf`, closes #1): all 12 `script/` entrypoints exist
(`bootstrap`, `setup`, `projectname`, `test`, `lint`, `fmt`,
`fmt-check`, `check`, `docker`, `cibuild`, `precommit`,
`install-precommit`) and every Makefile target is now a thin shim
over them, matching the other managed repos
- make the binary the default Make target (2026-07-24, branch - make the binary the default Make target (2026-07-24, branch
`make-default-target`): plain `make` now builds `sfdupes` (previously it ran `make-default-target`): plain `make` now builds `sfdupes`
`check` plus `build`); `make build` remains as an alias (previously it ran `check` plus `build`); `make build` remains as
- scan-wide phases, concurrent operands, batched updates (2026-07-24, branch an alias
`scan-wide-phases`): all operands seed the shared walk pool and every pass - scan-wide phases, concurrent operands, batched updates (2026-07-24,
runs once over the whole scan, so totals and ETAs are scan-global; the branch `scan-wide-phases`): all operands seed the shared walk pool
per-operand walk/hash/update cycles and their stderr announcements are gone; and every pass runs once over the whole scan, so totals and ETAs
the update pass commits in batched transactions — the filesystem is are scan-global; the per-operand walk/hash/update cycles and their
authoritative and the database an eventually-consistent reflection, so stderr announcements are gone; the update pass commits in batched
scan-level atomicity is not required transactions — the filesystem is authoritative and the database an
eventually-consistent reflection, so scan-level atomicity is not
required
- split the stat pass back out of the walk (2026-07-24, branch - split the stat pass back out of the walk (2026-07-24, branch
`parallel-phases`): phases are strictly sequential again — walk, stat, hash, `parallel-phases`): phases are strictly sequential again — walk,
update per operand — with parallelism only inside each phase; the walk stat, hash, update per operand — with parallelism only inside each
enumerates paths with per-directory workers and the stat pass lstats them with phase; the walk enumerates paths with per-directory workers and the
per-file workers, restoring the exact total/ETA stat bar stat pass lstats them with per-file workers, restoring the exact
- announce each operand on stderr before its passes (2026-07-24, branch total/ETA stat bar
`scan-operand-progress`): with per-operand walk/hash/update cycles, a - announce each operand on stderr before its passes (2026-07-24,
multi-operand run (e.g. `scan /srv/*`) showed pass totals that looked like the branch `scan-operand-progress`): with per-operand walk/hash/update
whole run's cycles, a multi-operand run (e.g. `scan /srv/*`) showed pass totals
- parallel walk (2026-07-24, branch `parallel-walk`): the walk pass was a single that looked like the whole run's — an operator watching operand 3 of
goroutine and took hours at ~20M files on a busy pool (observed: 22M files in 14 hash 300k files concluded 20M files were being skipped
4h on a ZFS server); it is now a per-directory worker-pool traversal that - parallel walk (2026-07-24, branch `parallel-walk`): the walk pass
records size/mtime during the walk (folding away the separate stat pass, was a single goroutine and took hours at ~20M files on a busy pool
halving metadata I/O), and each `PATH` operand commits in its own transaction (observed: 22M files in 4h on a ZFS server); it is now a
so an interrupted scan keeps completed operands per-directory worker-pool traversal that records size/mtime during
the walk (folding away the separate stat pass, halving metadata
I/O), and each `PATH` operand commits in its own transaction so an
interrupted scan keeps completed operands
- persistent scan database (2026-07-24, branch `persistent-database`): `scan` - persistent scan database (2026-07-24, branch `persistent-database`):
now maintains a SQLite database (`modernc.org/sqlite`, pure Go, cgo stays `scan` now maintains a SQLite database (`modernc.org/sqlite`, pure
disabled) keyed by absolute path that survives between runs — a rescan hashes Go, cgo stays disabled) keyed by absolute path that survives between
only new or changed files (by mtime/size), deletes records for files vanished runs — a rescan hashes only new or changed files (by mtime/size),
from under the scanned operands, and leaves records outside them untouched, so deletes records for files vanished from under the scanned operands,
`scan` can be cronned daily; `report` and `trees` read the database (no and leaves records outside them untouched, so `scan` can be cronned
positional arguments) instead of a scan stream. Database at daily; `report` and `trees` read the database (no positional
`/var/lib/sfdupes/db.sqlite`, overridable via `SFDUPES_DATABASE`; WAL arguments) instead of a scan stream. Database at
journaling plus a single-transaction update keep a report run during a scan `/var/lib/sfdupes/db.sqlite`, overridable via `SFDUPES_DATABASE`;
safe WAL journaling plus a single-transaction update keep a report run
- add the `origin` remote (`git@git.eeqj.de:sneak/sfdupes.git`), tag `v0.0.1`, during a scan safe
and push `main` plus tags (2026-07-23) - add the `origin` remote (`git@git.eeqj.de:sneak/sfdupes.git`), tag
`v0.0.1`, and push `main` plus tags (2026-07-23)
- `scan` CLI rework (2026-07-23, branch `scan-required-paths`): required - `scan` CLI rework (2026-07-23, branch `scan-required-paths`): required
`PATH...` operands via cobra flags replacing the `/srv` `-root` default; new `PATH...` operands via cobra flags replacing the `/srv` `-root`
`-x`/`--one-file-system` flag (GNU convention) to stop at filesystem default; new `-x`/`--one-file-system` flag (GNU convention) to stop
boundaries, which are crossed by default at filesystem boundaries, which are crossed by default
- bring the repo into full policy compliance (2026-07-23, branch - bring the repo into full policy compliance (2026-07-23, branch
`repo-policy-compliance`; checklist below) `repo-policy-compliance`; checklist below)
- `git init` with README-only first commit; code baseline committed on `main` - `git init` with README-only first commit; code baseline committed on
(2026-07-22) `main` (2026-07-22)
- implement `scan`, `report`, and `trees` subcommands (pre-git history) - implement `scan`, `report`, and `trees` subcommands (pre-git history)
# Future Steps # Future Steps
- possible later features (explicitly out of scope per README): full-content - possible later features (explicitly out of scope per README):
verification of candidates, removal-script helpers full-content verification of candidates, removal-script helpers
# Repo Policy Compliance # Repo Policy Compliance
Audited 2026-07-22 against `REPO_POLICIES.md` (2026-07-06), the existing repo Audited 2026-07-22 against `REPO_POLICIES.md` (2026-07-06), the existing
checklist, and the Go styleguide. Code is already gofmt-clean, so no standalone repo checklist, and the Go styleguide. Code is already gofmt-clean, so no
formatting commit is needed. standalone formatting commit is needed.
- [x] `.gitignore` missing — the compiled `sfdupes` binary and `files.dat` sit - [x] `.gitignore` missing — the compiled `sfdupes` binary and
untracked in the tree; needs OS/editor/Go artifacts plus secrets patterns `files.dat` sit untracked in the tree; needs OS/editor/Go
artifacts plus secrets patterns
- [x] `.editorconfig` missing - [x] `.editorconfig` missing
- [x] `LICENSE` missing and README has no License section (MIT assumed from - [x] `LICENSE` missing and README has no License section (MIT assumed
house convention — user to confirm) from house convention — user to confirm)
- [x] `REPO_POLICIES.md` missing from repo root - [x] `REPO_POLICIES.md` missing from repo root
- [x] `.golangci.yml` missing (install canonical copy); code must then pass - [x] `.golangci.yml` missing (install canonical copy); code must then
`make lint` (150 findings fixed; `make lint` is clean) pass `make lint` (150 findings fixed; `make lint` is clean)
- [x] `Makefile` lacks required targets `test`, `lint`, `fmt`, `fmt-check`, - [x] `Makefile` lacks required targets `test`, `lint`, `fmt`,
`docker`, `hooks`; `check` currently depends on `build`, which writes the `fmt-check`, `docker`, `hooks`; `check` currently depends on
binary (`make check` must not modify files) `build`, which writes the binary (`make check` must not modify
- [x] no tests — `go test ./...` has nothing to run; policy requires real tests files)
with a 30-second timeout and the conditional `-v` rerun pattern (suite - [x] no tests — `go test ./...` has nothing to run; policy requires
covers parsing, grouping, digests, suppression, hashing, and the scan real tests with a 30-second timeout and the conditional `-v`
pipeline; 64% coverage) rerun pattern (suite covers parsing, grouping, digests,
- [x] `Dockerfile` missing — Go multistage with hash-pinned images: fail-fast suppression, hashing, and the scan pipeline; 64% coverage)
lint stage, build stage running `make check` - [x] `Dockerfile` missing — Go multistage with hash-pinned images:
fail-fast lint stage, build stage running `make check`
- [x] `.dockerignore` missing - [x] `.dockerignore` missing
- [x] `.gitea/workflows/check.yml` missing (`docker build .` on push, checkout - [x] `.gitea/workflows/check.yml` missing (`docker build .` on push,
action pinned by commit SHA) checkout action pinned by commit SHA)
- [x] README lacks required sections: Description first line - [x] README lacks required sections: Description first line
(name/purpose/category/license/author), Getting Started, Rationale, TODO, (name/purpose/category/license/author), Getting Started,
License, Author Rationale, TODO, License, Author
- [x] README non-goal "no git repository setup and no CI" is stale now that the - [x] README non-goal "no git repository setup and no CI" is stale now
repo is under git with CI that the repo is under git with CI
- [x] pre-commit hook not installed (`make hooks` once the target exists) - [x] pre-commit hook not installed (`make hooks` once the target
exists)
Accepted divergences (no action): Accepted divergences (no action):
- flat single-package layout with `.go` files in the repo root — fine for a - flat single-package layout with `.go` files in the repo root — fine
small single-binary tool per the Go styleguide; the tracker audit agrees for a small single-binary tool per the Go styleguide; the tracker
- `make test` runs without `-race` — the repo mandates `CGO_ENABLED=0` (pure-Go audit agrees
builds) and the race detector requires cgo, so the detector runs in a separate - `go test` runs without `-race` — the repo mandates `CGO_ENABLED=0`
cgo-enabled container, `make test-race`, which is not part of `make check` (pure-Go builds) and the race detector requires cgo
-5
View File
@@ -1,5 +0,0 @@
{
"devDependencies": {
"prettier": "3.8.1"
}
}
+21 -12
View File
@@ -1,9 +1,12 @@
#!/bin/sh #!/bin/sh
# script/bootstrap: install all dependencies needed to build and develop # script/bootstrap: install all dependencies needed to build and develop
# this repo. Idempotent; assumes nothing is present (not git, make, or # this repo. Idempotent: every install is guarded by a check so already
# go). Base tooling comes from nix, apt, brew, or apk (detected in that # installed tools are skipped. Base tooling comes from nix, apt, brew,
# order). golangci-lint and prettier are never installed: they run via # or apk (detected in that order); assumes nothing is present (not git,
# docker only (script/lint, script/fmt, script/fmt-check). # make, or go). The linter is NOT installed: golangci-lint runs via
# docker only (script/lint), pinned by image digest, so the only lint
# prerequisite is a working docker — which is warned about, not
# installed, because everything except linting works without it.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
@@ -58,19 +61,25 @@ missing() {
main() { main() {
cd "$ROOT" cd "$ROOT"
# Deliberately unpinned, so presence is the whole check: go.mod # System tooling, deliberately unpinned: these come from the host
# governs the Go version, and reproducible builds run in the # package manager and whatever version it ships is what the host
# digest-pinned Docker images. # gets, so a presence check is the right check. The repo pins no
# system toolchain versions — the Go language version is governed by
# go.mod, and builds that must be reproducible run in the Docker
# image, whose base images are pinned by digest.
if missing git; then pkg_install git git git git; fi if missing git; then pkg_install git git git git; fi
if missing make; then pkg_install gnumake make make make; fi if missing make; then pkg_install gnumake make make make; fi
if missing go; then pkg_install go golang go go; fi if missing go; then pkg_install go golang go go; fi
# Warn, do not fail: only the targets named below, and the # Linting runs via docker only (script/lint), so docker is a lint
# pre-commit hook, need docker. # prerequisite rather than something bootstrap installs. Warn, do
# not fail: everything except `make lint` — and, through it,
# `make check`, `make docker` and the pre-commit hook — works
# without it.
if missing docker; then if missing docker; then
echo "bootstrap: WARNING: docker not found; make lint, make fmt," >&2 echo "bootstrap: WARNING: docker not found; make lint, make check" >&2
echo "bootstrap: make fmt-check, make check, make docker and" >&2 echo "bootstrap: and make docker require it. Install docker to" >&2
echo "bootstrap: make test-race require it." >&2 echo "bootstrap: run the linter." >&2
fi fi
go mod download go mod download
+22 -7
View File
@@ -1,19 +1,34 @@
#!/bin/sh #!/bin/sh
# script/cibuild: run the CI build. The Gitea workflow runs this on # script/cibuild: run the CI build. The Gitea workflow runs this on
# push. The Dockerfile runs every gate make check runs, as build steps, # push.
# so a successful build means the repo is green.
# #
# Without a fresh CHECK_EPOCH, a rebuild of an unchanged checkout serves # The Dockerfile runs the gates individually as build steps, not the
# the gate layers from cache and passes having run none of them. The # make check aggregate: the lint stage runs make fmt-check,
# process id goes in with the epoch so two runs started in the same # script/verify-lint-image-pin, golangci-lint config verify and
# second still differ. # golangci-lint run; the build stage, dropped to an unprivileged user,
# runs make test and make fmt-check. Neither make lint nor make check
# appears, because both reach script/lint, which is itself a docker
# build, and a docker build cannot run inside one. Lint is not skipped
# by that — the linter is invoked directly in the lint stage, and the
# build stage's COPY --from=lint makes that stage a prerequisite, so
# BuildKit must finish it first. Between the two stages everything
# make check would run has run, which is why a successful build here
# implies the repo is green.
#
# That implication holds only because of CHECK_EPOCH. A COPY layer is
# invalidated only by changed content, and a rebuild of an unchanged
# checkout sends the same content, so without a fresh value here Docker
# serves the gate layers from cache and the build reports a green it
# never earned. Passing the current epoch invalidates the gate
# layers on every run while leaving the pinned base images and
# go mod download cached; see the Dockerfile for the placement.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
docker build --build-arg CHECK_EPOCH="$(date +%s)-$$" . docker build --build-arg CHECK_EPOCH="$(date +%s)" .
} }
main "$@" main "$@"
+10 -5
View File
@@ -1,9 +1,14 @@
#!/bin/sh #!/bin/sh
# script/docker: build the Docker image tagged with the project name. # script/docker: build the Docker image tagged with the project name.
# The tag comes from script/projectname. CHECK_EPOCH is passed for the # The tag comes from script/projectname.
# same reason script/cibuild passes it: without a fresh value an #
# unchanged tree is served from cache and this exits 0 having run no # CHECK_EPOCH is passed for the same reason script/cibuild passes it:
# gate. # without it Docker serves the Dockerfile's gate layers from cache on an
# unchanged tree and this exits 0 having run neither the lint stage's
# gates nor the builder stage's test and fmt-check gates. This is the
# set of gates a developer or reviewer runs by hand, so a cached pass
# here is the most misleading result the repo can produce. Dependency
# layers sit above the ARG and stay cached.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -12,7 +17,7 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
docker build \ docker build \
--build-arg CHECK_EPOCH="$(date +%s)-$$" \ --build-arg CHECK_EPOCH="$(date +%s)" \
-t "$("$SCRIPT_DIR/projectname")" \ -t "$("$SCRIPT_DIR/projectname")" \
. .
} }
+2 -12
View File
@@ -1,22 +1,12 @@
#!/bin/sh #!/bin/sh
# script/fmt: format all files (writes): the Go sources with gofmt, the # script/fmt: format all files (writes).
# Markdown with prettier. prettier is never installed on the host: it
# runs from the Dockerfile's prettier stage with the repository mounted,
# as the calling user so the files it rewrites keep their owner. The tag
# makes each build replace the previous image instead of leaving another
# one behind.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
gofmt -s -w . gofmt -s -w .
image="$("$SCRIPT_DIR/projectname")-prettier"
docker build -q --target prettier -t "$image" . >/dev/null
docker run --rm --user "$(id -u):$(id -g)" -v "$ROOT:/src" "$image" \
prettier --write '**/*.md' --tab-width 4 --prose-wrap always
} }
main "$@" main "$@"
+4 -25
View File
@@ -1,39 +1,18 @@
#!/bin/sh #!/bin/sh
# script/fmt-check: check formatting (read-only). Same scope as # script/fmt-check: check formatting (read-only). Same scope as
# script/fmt, but fails instead of writing. gofmt and prettier both run # script/fmt, but fails instead of writing.
# every time and each reports its own failure, so the output says which
# one failed.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
status=0 files="$(gofmt -s -l .)"
# Under set -e a bare assignment would end the script when gofmt
# fails (a Go file it cannot parse), and prettier would never run.
if ! files="$(gofmt -s -l .)"; then
echo "gofmt: failed; see its errors above" >&2
status=1
fi
if [ -n "$files" ]; then if [ -n "$files" ]; then
echo "gofmt: files not formatted:" >&2 echo "gofmt: files not formatted:" >&2
echo "$files" >&2 echo "$files" >&2
status=1 exit 1
fi fi
# Same image as script/fmt; see there.
image="$("$SCRIPT_DIR/projectname")-prettier"
docker build -q --target prettier -t "$image" . >/dev/null
if ! docker run --rm -v "$ROOT:/src:ro" "$image" \
prettier --check '**/*.md' --tab-width 4 --prose-wrap always; then
echo "prettier: Markdown not formatted; run make fmt" >&2
status=1
fi
exit "$status"
} }
main "$@" main "$@"
+16 -10
View File
@@ -1,17 +1,23 @@
#!/bin/sh #!/bin/sh
# script/lint: run the linter. golangci-lint is never installed on a # script/lint: run the linter. golangci-lint is never installed on a
# host: this builds Dockerfile.lint, which copies the repo into the # host: it runs via docker only, one way, everywhere — this builds
# digest-pinned golangci-lint image and lints as a build step, so a # Dockerfile.lint, which COPYs the repo into the digest-pinned
# successful build is a clean lint. A cold cache needs the network to # golangci-lint image and lints as a build step, so a successful build
# pull the image and for `go mod download`; once warm this runs offline # is a clean lint. The only prerequisite is a working docker. The gate
# until go.mod or go.sum changes. # steps make no network calls of their own, but Dockerfile.lint runs
# `go mod download` above them, so a cold cache does reach the network
# (as does pulling the pinned image); that layer stays cached, and once
# it is warm this runs offline until go.mod or go.sum changes.
# #
# Without a fresh CHECK_EPOCH docker serves the gate layers from cache # CHECK_EPOCH is what makes the result mean anything. Without it docker
# on an unchanged tree and this exits 0 having run no linter. The PID is # serves the gate layers from cache on an unchanged tree and this exits
# in the value because two lint runs land inside the same second easily. # 0 in well under a second having run no linter. The PID is in the value
# as well as the epoch because two lint runs land inside the same second
# easily, and `date +%s` alone would cache the second one.
# #
# The image is never used, so --output=type=cacheonly writes none; # The result is the build's exit status and the image is never used, so
# without it every run leaves an untagged image behind. # --output=type=cacheonly writes none. Without it every run spends
# seconds exporting an image and leaves it behind untagged.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
-40
View File
@@ -1,40 +0,0 @@
#!/bin/sh
# script/test-race: run the test suite under the race detector. Not part
# of script/check.
#
# The race detector needs cgo and a C compiler, which the host build
# never uses, so the tests run in a golang image that has gcc. The
# checkout is mounted read-only, so the docker daemon must be local. The
# container starts with empty caches every time: each run downloads the
# dependencies and compiles them with the detector, which needs the
# network and takes minutes.
#
# The tests run as the calling user, never as root: several of them make
# a file unreadable and expect reading it to fail, and root reads it
# anyway. When the caller is root they run as nobody, and then the
# checkout must be readable by other users. Neither user has a home
# directory in the image, so HOME is /tmp, where Go puts its build cache.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# golang:1.25-trixie, 2026-10-04. Debian rather than the Alpine image the
# Dockerfile builds with, because this one includes gcc.
IMAGE="golang@sha256:2c4c60ef415fbfa5e90300722293bef36c5e63fae17570ce18f580af933dbd73"
main() {
user="$(id -u):$(id -g)"
if [ "$(id -u)" -eq 0 ]; then
user=65534:65534
fi
docker run --rm \
--user "$user" \
--env HOME=/tmp \
--env CGO_ENABLED=1 \
--volume "$ROOT:/src:ro" \
--workdir /src \
"$IMAGE" \
go test -race -timeout 60s ./...
}
main "$@"
+15 -8
View File
@@ -2,16 +2,23 @@
# script/verify-lint-image-pin: fail unless the golangci-lint image # script/verify-lint-image-pin: fail unless the golangci-lint image
# referenced by Dockerfile.lint and the one referenced by the main # referenced by Dockerfile.lint and the one referenced by the main
# Dockerfile's lint stage are the same image at the same digest. Our own # Dockerfile's lint stage are the same image at the same digest. Our own
# extension to scripts-to-rule-them-all, not one of its entrypoints; run # extension to scripts-to-rule-them-all, not one of its entrypoints.
# as a gate in both files. Nothing else keeps the two pins in sync, and
# a bump applied to one alone would lint the same tree against different
# rulesets, both green.
# #
# Do not hardcode the expected digest here: that is a third copy to keep # The linter version is pinned in two independent files. That is the
# in sync. # shape #42 turned into a build failure rather than tolerate: nothing
# else keeps the two in sync, and a bump applied to one file alone would
# leave `make lint` and the fail-fast lint stage of `make docker`
# linting the same tree against different rulesets, both green. This is
# the single guard that stops it, run as a gate in both files.
# #
# A reference that cannot be read is a hard failure, not a skip: two # It deliberately restates neither pin. A hardcoded expected digest here
# empty strings compare equal. # would be a third copy — one more thing to bump, and the same drift one
# file further out. It compares the two files to each other and knows
# nothing about which version is correct.
#
# A reference that cannot be read is a hard failure, not a skip: a
# comparison of two empty strings succeeds, which would turn this guard
# into exactly the unearned green it exists to prevent.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
-8
View File
@@ -1,8 +0,0 @@
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
prettier@3.8.1:
version "3.8.1"
resolved "https://registry.yarnpkg.com/prettier/-/prettier-3.8.1.tgz#edf48977cf991558f4fcbd8a3ba6015ba2a3a173"
integrity sha512-UOnG6LftzbdaHZcKoPFtOcCKztrQ57WkHDeRD9t/PTQtmT0NHSeWWepj6pS0z/N7+08BHFDQVUrfmfMRcZwbMg==