Format Markdown with prettier in make fmt and make fmt-check (closes #19)
check / check (push) Failing after 3s
check / check (push) Failing after 3s
script/fmt and script/fmt-check run prettier over every Markdown file again, next to gofmt. prettier is pinned by hash through package.json and yarn.lock, copied from the prompts repo with .prettierrc and .prettierignore, and is never installed on a host: a new prettier stage of the Dockerfile installs it into a digest-pinned node image, and both scripts build that stage and run it with the repository mounted. CI checks the Markdown in a markdown stage that the build stage waits on. Because make fmt-check now runs docker, the Dockerfile runs gofmt directly in its lint stage instead. All Markdown is reformatted. Model: opus-5-5
This commit was merged in pull request #84.
This commit is contained in:
@@ -0,0 +1,2 @@
|
||||
node_modules/
|
||||
yarn.lock
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"tabWidth": 4,
|
||||
"proseWrap": "always"
|
||||
}
|
||||
+54
-21
@@ -12,9 +12,9 @@ COPY . .
|
||||
# build would exit 0 having run nothing. script/cibuild and
|
||||
# script/docker pass a fresh CHECK_EPOCH on every invocation.
|
||||
#
|
||||
# Two properties this depends on. ARG is per-stage, so the build stage
|
||||
# below declares it again; one declaration here would leave that
|
||||
# stage's gate cacheable. And each gate RUN must reference the value,
|
||||
# Two properties this depends on. ARG is per-stage, so the markdown and
|
||||
# build stages below declare it again; one declaration here would leave
|
||||
# their gates cacheable. And each gate RUN must reference the value,
|
||||
# because BuildKit hashes the expanded command: a declared but
|
||||
# unreferenced ARG invalidates nothing.
|
||||
#
|
||||
@@ -27,9 +27,16 @@ ARG CHECK_EPOCH
|
||||
# target now runs `docker build -f Dockerfile.lint`, and a docker build
|
||||
# cannot run a docker build: routing the gate through make would mean
|
||||
# nesting docker inside this image. Same reason `make check` is gone
|
||||
# from the build stage below. `make fmt-check` stays as it is — it is a
|
||||
# gate, not the aggregate, and it shells out to nothing.
|
||||
RUN echo "gate fmt-check, epoch ${CHECK_EPOCH}" && make fmt-check
|
||||
# from the build stage below, and `make fmt-check` from both stages: it
|
||||
# runs prettier through docker too. Its gofmt half is the step below,
|
||||
# its Markdown half the markdown stage further down. gofmt's output is
|
||||
# assigned to a variable first so that its own exit status, as when it
|
||||
# cannot parse a file, still fails the step.
|
||||
RUN echo "gate gofmt, epoch ${CHECK_EPOCH}" && \
|
||||
files="$(gofmt -s -l .)" && \
|
||||
if [ -n "$files" ]; then \
|
||||
echo "gofmt: files not formatted:" >&2; echo "$files" >&2; exit 1; \
|
||||
fi
|
||||
|
||||
# The FROM above and the one in Dockerfile.lint pin the same linter
|
||||
# twice, and nothing else keeps them in sync; this fails the build when
|
||||
@@ -46,6 +53,30 @@ RUN echo "gate config verify, epoch ${CHECK_EPOCH}" && \
|
||||
RUN echo "gate lint, epoch ${CHECK_EPOCH}" && \
|
||||
golangci-lint run --config .golangci.yml ./...
|
||||
|
||||
# Prettier stage: the prettier that formats this repository's Markdown,
|
||||
# never installed on a host. script/fmt and script/fmt-check build this
|
||||
# stage alone and run it with the repository mounted on /src. prettier
|
||||
# is installed in /tools so that the repository, mounted or copied onto
|
||||
# /src, cannot hide it.
|
||||
# node:22-alpine, 2026-02-22
|
||||
FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS prettier
|
||||
WORKDIR /tools
|
||||
# yarn.lock pins prettier by hash, and --frozen-lockfile fails rather
|
||||
# than install anything yarn.lock does not name.
|
||||
COPY package.json yarn.lock ./
|
||||
RUN yarn install --frozen-lockfile
|
||||
ENV PATH=/tools/node_modules/.bin:$PATH
|
||||
WORKDIR /src
|
||||
|
||||
# Markdown stage: the Markdown half of `make fmt-check`, as a gate.
|
||||
FROM prettier AS markdown
|
||||
COPY . .
|
||||
# Second per-stage declaration of the gate cache-buster; see the lint
|
||||
# stage above.
|
||||
ARG CHECK_EPOCH
|
||||
RUN echo "gate prettier, epoch ${CHECK_EPOCH}" && \
|
||||
prettier --check '**/*.md' --tab-width 4 --prose-wrap always
|
||||
|
||||
# Build stage
|
||||
# golang:1.25-alpine, 2026-07-23
|
||||
FROM golang@sha256:56961d79ea8129efddcc0b8643fd8a5416b4e6228cfd477e3fd61deb2672c587 AS builder
|
||||
@@ -70,15 +101,17 @@ ENV GOCACHE=/home/builder/.cache/go-build
|
||||
|
||||
WORKDIR /src
|
||||
|
||||
# No-op file copy whose only purpose is the build-graph edge: it is what
|
||||
# makes this stage depend on the lint stage, and so what forces BuildKit
|
||||
# to finish fmt-check, the pin guard and lint before compilation and
|
||||
# tests start. Remove it and the fail-fast design dies silently — the
|
||||
# build stops gating on lint and still exits 0. It replaces a copy of
|
||||
# the linter binary itself, which is no longer wanted here: nothing in
|
||||
# this stage runs the linter, because `make lint` is now a docker build
|
||||
# and a docker build cannot run inside one.
|
||||
# No-op file copies whose only purpose is the build-graph edge: they are
|
||||
# what make this stage depend on the lint and markdown stages, and so
|
||||
# what forces BuildKit to finish gofmt, the pin guard, lint and prettier
|
||||
# before compilation and tests start. Remove one and the fail-fast
|
||||
# design dies silently — the build stops gating on that stage and still
|
||||
# exits 0. The first replaces a copy of the linter binary itself, which
|
||||
# is no longer wanted here: nothing in this stage runs the linter,
|
||||
# because `make lint` is now a docker build and a docker build cannot
|
||||
# run inside one.
|
||||
COPY --from=lint /src/go.sum /dev/null
|
||||
COPY --from=markdown /src/go.sum /dev/null
|
||||
|
||||
# Install development prerequisites the same way a developer does,
|
||||
# rather than duplicating the installs inline. Only script/ and the
|
||||
@@ -112,19 +145,19 @@ USER builder
|
||||
# permission-denied test paths are exercised legitimately (root would
|
||||
# bypass the chmod(0) the tests rely on).
|
||||
#
|
||||
# The gates are the individual targets, not `make check`: that aggregate
|
||||
# runs `script/lint`, which is now a docker build, and nothing inside an
|
||||
# image build may shell out to docker. Lint is not skipped by this — it
|
||||
# ran in the lint stage above, which this stage's COPY --from makes a
|
||||
# prerequisite. `make`, not the scripts directly, because the Makefile's
|
||||
# The gate is `make test`, not `make check`: that aggregate runs
|
||||
# `script/lint` and `script/fmt-check`, which both run docker, and
|
||||
# nothing inside an image build may shell out to docker. Lint and the
|
||||
# format checks are not skipped by this — they ran in the lint and
|
||||
# markdown stages above, which this stage's COPY --from lines make
|
||||
# prerequisites. `make`, not the script directly, because the Makefile's
|
||||
# `export CGO_ENABLED = 0` applies only to what it invokes.
|
||||
#
|
||||
# Second per-stage declaration of the gate cache-buster; see the lint
|
||||
# Third per-stage declaration of the gate cache-buster; see the lint
|
||||
# stage above for why one is not enough. It is placed after USER so the
|
||||
# drop to the unprivileged user still happens before the checks run.
|
||||
ARG CHECK_EPOCH
|
||||
RUN echo "gate test, epoch ${CHECK_EPOCH}" && make test
|
||||
RUN echo "gate fmt-check, epoch ${CHECK_EPOCH}" && make fmt-check
|
||||
|
||||
# The version stamped into the binary: the VERSION build argument when
|
||||
# one is given, otherwise `git describe --tags --always` of the .git in
|
||||
|
||||
@@ -1,60 +1,62 @@
|
||||
# Workflow
|
||||
|
||||
- take an issue from the `1.0.0` milestone on the tracker; work not
|
||||
yet on the tracker gets filed as an issue first
|
||||
- take an issue from the `1.0.0` milestone on the tracker; work not yet on the
|
||||
tracker gets filed as an issue first
|
||||
- branch (from `main`)
|
||||
- do the work, with tests, in small focused commits
|
||||
- record it at the top of Completed Steps (`TODO.md` changes in the
|
||||
same commit as the work)
|
||||
- push the branch and open a PR whose title ends with
|
||||
` (closes #N)`
|
||||
- an independent review gates the merge; every finding is addressed
|
||||
or explicitly rebutted on the PR
|
||||
- record it at the top of Completed Steps (`TODO.md` changes in the same commit
|
||||
as the work)
|
||||
- push the branch and open a PR whose title ends with ` (closes #N)`
|
||||
- an independent review gates the merge; every finding is addressed or
|
||||
explicitly rebutted on the PR
|
||||
- merge to `main` once the review passes
|
||||
|
||||
# Status
|
||||
|
||||
- pre-1.0
|
||||
- the Gitea tracker is authoritative for the pre-1.0 backlog: the
|
||||
open issues under the `1.0.0` milestone are what remains before
|
||||
the tag, and this file records history and process, not the queue
|
||||
- the Gitea tracker is authoritative for the pre-1.0 backlog: the open issues
|
||||
under the `1.0.0` milestone are what remains before the tag, and this file
|
||||
records history and process, not the queue
|
||||
|
||||
# Next Step
|
||||
|
||||
- take the next issue from the `1.0.0` milestone on the tracker:
|
||||
https://git.eeqj.de/sneak/sfdupes/milestone/17 — the milestone is
|
||||
the source of truth for what is left before 1.0.0. Individual
|
||||
issues are deliberately not restated here; a copy in this file
|
||||
drifts out of date the moment the tracker moves
|
||||
https://git.eeqj.de/sneak/sfdupes/milestone/17 — the milestone is the source
|
||||
of truth for what is left before 1.0.0. Individual issues are deliberately not
|
||||
restated here; a copy in this file drifts out of date the moment the tracker
|
||||
moves
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- `script/lint` writes no image, so a run no longer leaves an untagged
|
||||
one behind (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/48)
|
||||
- `make fmt` and `make fmt-check` run prettier over all Markdown, in Docker, and
|
||||
CI checks it; all Markdown reformatted (2026-10-04,
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/19)
|
||||
|
||||
- tests cover a missing database, `scan` keeping stdout empty, its skip
|
||||
warning, the `report` and `trees` summary lines, and every subcommand
|
||||
going through `runE` (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/16)
|
||||
- `script/lint` writes no image, so a run no longer leaves an untagged one
|
||||
behind (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/48)
|
||||
|
||||
- tests cover a missing database, `scan` keeping stdout empty, its skip warning,
|
||||
the `report` and `trees` summary lines, and every subcommand going through
|
||||
`runE` (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/16)
|
||||
|
||||
- `.golangci.yml` replaced with the current canonical copy, which uses
|
||||
`gomodguard_v2`, so lint no longer prints a deprecation warning
|
||||
(2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/26)
|
||||
`gomodguard_v2`, so lint no longer prints a deprecation warning (2026-10-04,
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/26)
|
||||
|
||||
- a test fails when either `hashWorker` cancellation check in `scan.go`
|
||||
is removed (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/83)
|
||||
- a test fails when either `hashWorker` cancellation check in `scan.go` is
|
||||
removed (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/83)
|
||||
|
||||
- a database path holding `?`, `#` or `%` opens exactly the file it names
|
||||
(2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/55)
|
||||
|
||||
- `scan` rejects `--workers` below 1 as a usage error instead of
|
||||
running single-threaded (2026-10-04,
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/10)
|
||||
- `scan` rejects `--workers` below 1 as a usage error instead of running
|
||||
single-threaded (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/10)
|
||||
|
||||
- a test fails when either walk cancellation check in `scan.go` is
|
||||
removed (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/81)
|
||||
- a test fails when either walk cancellation check in `scan.go` is removed
|
||||
(2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/81)
|
||||
|
||||
- test that `scan` refuses a database with another schema version
|
||||
(2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/64)
|
||||
- test that `scan` refuses a database with another schema version (2026-10-04,
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/64)
|
||||
|
||||
- correct four inaccurate comments in `cancel_test.go` and rename
|
||||
`walkCancelInFlightDirs` to `walkCancelInFlightFiles` (2026-10-04,
|
||||
@@ -67,479 +69,428 @@
|
||||
`files` table is refused with a clear schema-version error (2026-10-04,
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/11)
|
||||
|
||||
- README documents install, Docker, a daily cron scan and how to read
|
||||
and check the reports (2026-10-04,
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/54)
|
||||
- README documents install, Docker, a daily cron scan and how to read and check
|
||||
the reports (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/54)
|
||||
|
||||
- the `Dockerfile` build stage keeps the Go module cache out of `builder`'s
|
||||
home and copies the sources with `--chown`, so no `chown -R` walks them
|
||||
- the `Dockerfile` build stage keeps the Go module cache out of `builder`'s home
|
||||
and copies the sources with `--chown`, so no `chown -R` walks them
|
||||
(2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/43)
|
||||
|
||||
- `--version` prints `sfdupes VERSION` to stdout; README documents it and
|
||||
`--help` (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/15)
|
||||
|
||||
- `scan` stops cleanly on `SIGINT` or `SIGTERM`: commits what it has
|
||||
hashed, deletes nothing more, exits 1 (2026-10-04,
|
||||
- `scan` stops cleanly on `SIGINT` or `SIGTERM`: commits what it has hashed,
|
||||
deletes nothing more, exits 1 (2026-10-04,
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/5)
|
||||
|
||||
- `report` and `trees` stream the records instead of holding them all in
|
||||
memory; the schema gains the `files_signature` index (2026-10-04,
|
||||
- `report` and `trees` stream the records instead of holding them all in memory;
|
||||
the schema gains the `files_signature` index (2026-10-04,
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/14)
|
||||
|
||||
- progress prints at once on a non-terminal, uses a real terminal test,
|
||||
and prints warnings through a spinner instead of racing its redraw
|
||||
(2026-10-03, https://git.eeqj.de/sneak/sfdupes/issues/13)
|
||||
- progress prints at once on a non-terminal, uses a real terminal test, and
|
||||
prints warnings through a spinner instead of racing its redraw (2026-10-03,
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/13)
|
||||
|
||||
- warn about and skip symlink, socket, FIFO, device and `.zfs`
|
||||
operands, keeping the records beneath them (2026-10-03,
|
||||
- warn about and skip symlink, socket, FIFO, device and `.zfs` operands, keeping
|
||||
the records beneath them (2026-10-03,
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/9)
|
||||
|
||||
- `scan` holds a lock on a lock file beside the database for its whole run,
|
||||
so a second `scan` fails at once with exit 1 (2026-10-03,
|
||||
- `scan` holds a lock on a lock file beside the database for its whole run, so a
|
||||
second `scan` fails at once with exit 1 (2026-10-03,
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/53)
|
||||
|
||||
- test stdout write failures in `report` and `trees`; README states that
|
||||
`| head` ends sfdupes by `SIGPIPE` and `>&-` writes to `/dev/null`
|
||||
(2026-10-03, https://git.eeqj.de/sneak/sfdupes/issues/30)
|
||||
|
||||
- `report` and `trees` open the database read-only, and `scan` leaves it
|
||||
out of WAL mode, so reading needs only read access (2026-10-03, closes
|
||||
- `report` and `trees` open the database read-only, and `scan` leaves it out of
|
||||
WAL mode, so reading needs only read access (2026-10-03, closes
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/8)
|
||||
|
||||
- escape tabs, newlines, carriage returns and backslashes in report,
|
||||
trees and warning paths; the root directory's path is `/`
|
||||
(2026-10-03, https://git.eeqj.de/sneak/sfdupes/issues/7)
|
||||
- escape tabs, newlines, carriage returns and backslashes in report, trees and
|
||||
warning paths; the root directory's path is `/` (2026-10-03,
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/7)
|
||||
|
||||
- stamp the git tag or short commit in a plain `docker build .`
|
||||
instead of `dev` (2026-10-02, branch `next`, closes
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/67): `.dockerignore` now
|
||||
sends `.git`, without `.git/config`, and the `Dockerfile` build
|
||||
stage takes the `VERSION` build argument when one is given,
|
||||
otherwise `git describe --tags --always` of that `.git`. The build
|
||||
fails if the context carries `.git` and the version still comes out
|
||||
empty, `dev` or `unknown`. The CI checkout step fetches the full
|
||||
history (`fetch-depth: 0`) so CI sees the tag and stamps the same
|
||||
value as `make build`.
|
||||
- stamp the git tag or short commit in a plain `docker build .` instead of `dev`
|
||||
(2026-10-02, branch `next`, closes
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/67): `.dockerignore` now sends
|
||||
`.git`, without `.git/config`, and the `Dockerfile` build stage takes the
|
||||
`VERSION` build argument when one is given, otherwise
|
||||
`git describe --tags --always` of that `.git`. The build fails if the context
|
||||
carries `.git` and the version still comes out empty, `dev` or `unknown`. The
|
||||
CI checkout step fetches the full history (`fetch-depth: 0`) so CI sees the
|
||||
tag and stamps the same value as `make build`.
|
||||
|
||||
- replace the 1 KiB end-window sampling with the head/tail plus
|
||||
content-hash ladder (2026-09-22, branch `next`, closes
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/61): a file under 10 MiB is
|
||||
hashed in full and compared directly, with no end-window step — its
|
||||
`head`, `tail`, and `content` all hold the whole-file hash. A file at
|
||||
10 MiB or above gets only the 64 KiB `head` and `tail` in the hash
|
||||
phase; a new content phase, after the update phase, reads it for its
|
||||
`content` hash — the whole file below 50 MiB, gigabyte-spaced 1 MiB
|
||||
samples at or above — only when its size, `head`, and `tail` match
|
||||
another record's, from the same scan or stored by an earlier one, so
|
||||
a stored file gains its content hash when it gains a match. A file
|
||||
that is gone or has changed since its record was written is not
|
||||
read. The `content` column is part of the version 1 schema. `report`
|
||||
and `trees` group by the extended signature and leave out any record
|
||||
without a `content` hash, so the ladder is applied across the whole
|
||||
database. README "Duplicate detection" documents every rung including
|
||||
the probabilistic large-file path.
|
||||
- replace the 1 KiB end-window sampling with the head/tail plus content-hash
|
||||
ladder (2026-09-22, branch `next`, closes
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/61): a file under 10 MiB is hashed in
|
||||
full and compared directly, with no end-window step — its `head`, `tail`, and
|
||||
`content` all hold the whole-file hash. A file at 10 MiB or above gets only
|
||||
the 64 KiB `head` and `tail` in the hash phase; a new content phase, after the
|
||||
update phase, reads it for its `content` hash — the whole file below 50 MiB,
|
||||
gigabyte-spaced 1 MiB samples at or above — only when its size, `head`, and
|
||||
`tail` match another record's, from the same scan or stored by an earlier one,
|
||||
so a stored file gains its content hash when it gains a match. A file that is
|
||||
gone or has changed since its record was written is not read. The `content`
|
||||
column is part of the version 1 schema. `report` and `trees` group by the
|
||||
extended signature and leave out any record without a `content` hash, so the
|
||||
ladder is applied across the whole database. README "Duplicate detection"
|
||||
documents every rung including the probabilistic large-file path.
|
||||
|
||||
- remove the dead `files.dat` references from `Makefile`, `.gitignore`
|
||||
and `.dockerignore` (2026-09-21, branch `next`, closes
|
||||
- remove the dead `files.dat` references from `Makefile`, `.gitignore` and
|
||||
`.dockerignore` (2026-09-21, branch `next`, closes
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/22)
|
||||
|
||||
- fix the lint-image pin comments and `FROM` form in `Dockerfile` and
|
||||
`Dockerfile.lint` (2026-08-10, branch `next`, closes
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/25): dropped the false
|
||||
`(Debian-based)` parenthetical (v2.12.1 was Debian too) and the
|
||||
redundant tag, so both pins are the policy `# image:vX.Y.Z,
|
||||
YYYY-MM-DD` comment over a bare `FROM image@sha256:...`. Digest
|
||||
unchanged. `script/verify-lint-image-pin` parses those `FROM` lines
|
||||
and still matches the tagless form; its advice line lost the now
|
||||
meaningless "tag and digest". With no tag in either reference, a
|
||||
tag-only disagreement no longer exists — a one-sided tag is caught as
|
||||
a plain mismatch.
|
||||
`(Debian-based)` parenthetical (v2.12.1 was Debian too) and the redundant tag,
|
||||
so both pins are the policy `# image:vX.Y.Z, YYYY-MM-DD` comment over a bare
|
||||
`FROM image@sha256:...`. Digest unchanged. `script/verify-lint-image-pin`
|
||||
parses those `FROM` lines and still matches the tagless form; its advice line
|
||||
lost the now meaningless "tag and digest". With no tag in either reference, a
|
||||
tag-only disagreement no longer exists — a one-sided tag is caught as a plain
|
||||
mismatch.
|
||||
|
||||
- run all linting in Docker via `Dockerfile.lint` and `script/lint`
|
||||
(2026-08-10, branch `next`, closes
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/46): per the owner ruling, the
|
||||
linter runs inside a container invoked through the `script/`
|
||||
entrypoint and is never installed on a host. New root
|
||||
`Dockerfile.lint` COPYs the repo into the digest-pinned
|
||||
`golangci/golangci-lint:v2.12.2` image and runs
|
||||
`golangci-lint config verify` and `golangci-lint run` as build
|
||||
steps, so a successful build IS a clean lint; `script/lint` is
|
||||
reduced to building it. `script/bootstrap` loses the `go install`,
|
||||
the pin constants, the version parser and `verify_golangci_lint`
|
||||
outright rather than hardening them — with nothing linting on the
|
||||
host, the `$GOPATH/bin` versus `PATH` problem that motivated them has
|
||||
no subject — and now warns rather than fails when `docker` is absent.
|
||||
Two traps handled. A lint build on an unchanged tree returns success
|
||||
in well under a second having run no linter, which is
|
||||
- run all linting in Docker via `Dockerfile.lint` and `script/lint` (2026-08-10,
|
||||
branch `next`, closes https://git.eeqj.de/sneak/sfdupes/issues/46): per the
|
||||
owner ruling, the linter runs inside a container invoked through the `script/`
|
||||
entrypoint and is never installed on a host. New root `Dockerfile.lint` COPYs
|
||||
the repo into the digest-pinned `golangci/golangci-lint:v2.12.2` image and
|
||||
runs `golangci-lint config verify` and `golangci-lint run` as build steps, so
|
||||
a successful build IS a clean lint; `script/lint` is reduced to building it.
|
||||
`script/bootstrap` loses the `go install`, the pin constants, the version
|
||||
parser and `verify_golangci_lint` outright rather than hardening them — with
|
||||
nothing linting on the host, the `$GOPATH/bin` versus `PATH` problem that
|
||||
motivated them has no subject — and now warns rather than fails when `docker`
|
||||
is absent. Two traps handled. A lint build on an unchanged tree returns
|
||||
success in well under a second having run no linter, which is
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/32 and
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/39 again, so
|
||||
`Dockerfile.lint` carries `ARG CHECK_EPOCH` referenced
|
||||
inside every gate `RUN` (BuildKit hashes the expanded command, not
|
||||
the declaration) and `script/lint` passes `"$(date +%s)-$$"` — the
|
||||
PID matters because two lint runs land inside the same second easily.
|
||||
And nothing inside an image build may shell out to docker, so the
|
||||
main `Dockerfile`'s lint stage now invokes `golangci-lint` directly
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/39 again, so `Dockerfile.lint`
|
||||
carries `ARG CHECK_EPOCH` referenced inside every gate `RUN` (BuildKit hashes
|
||||
the expanded command, not the declaration) and `script/lint` passes
|
||||
`"$(date +%s)-$$"` — the PID matters because two lint runs land inside the
|
||||
same second easily. And nothing inside an image build may shell out to docker,
|
||||
so the main `Dockerfile`'s lint stage now invokes `golangci-lint` directly
|
||||
instead of `make lint`, and its build stage runs `make test` and
|
||||
`make fmt-check` instead of the `make check` aggregate (`make`, not
|
||||
the scripts bare, because the Makefile's `export CGO_ENABLED = 0`
|
||||
only reaches what it invokes). `COPY --from=lint`
|
||||
`/usr/bin/golangci-lint` is replaced by
|
||||
`COPY --from=lint /src/go.sum /dev/null`: the copied binary was the
|
||||
only edge forcing BuildKit to finish linting before the build stage
|
||||
starts, and dropping it without replacing the edge would have ended
|
||||
fail-fast linting silently under a still-green build. That is
|
||||
canonical `REPO_POLICIES.md:107`'s ordering edge, restored.
|
||||
`ENV PATH=/home/builder/go/bin:$PATH` is gone with the `go install`
|
||||
that justified it. `script/verify-linter-pin` is retired, deleted
|
||||
along with its README entry, because both of its subjects ceased to
|
||||
exist in the same change: it compared a linter binary against
|
||||
`GOLANGCI_LINT_VERSION` in `script/bootstrap`, and there is now
|
||||
neither a binary crossing between stages nor a version pin in
|
||||
bootstrap. The drift it guarded has not gone away, it has moved — the
|
||||
linter is still pinned twice, now as the `FROM` line of
|
||||
`Dockerfile.lint` and the `FROM` line of the `Dockerfile` lint stage,
|
||||
with nothing syncing them, which is exactly what
|
||||
`make fmt-check` instead of the `make check` aggregate (`make`, not the
|
||||
scripts bare, because the Makefile's `export CGO_ENABLED = 0` only reaches
|
||||
what it invokes). `COPY --from=lint` `/usr/bin/golangci-lint` is replaced by
|
||||
`COPY --from=lint /src/go.sum /dev/null`: the copied binary was the only edge
|
||||
forcing BuildKit to finish linting before the build stage starts, and dropping
|
||||
it without replacing the edge would have ended fail-fast linting silently
|
||||
under a still-green build. That is canonical `REPO_POLICIES.md:107`'s ordering
|
||||
edge, restored. `ENV PATH=/home/builder/go/bin:$PATH` is gone with the
|
||||
`go install` that justified it. `script/verify-linter-pin` is retired, deleted
|
||||
along with its README entry, because both of its subjects ceased to exist in
|
||||
the same change: it compared a linter binary against `GOLANGCI_LINT_VERSION`
|
||||
in `script/bootstrap`, and there is now neither a binary crossing between
|
||||
stages nor a version pin in bootstrap. The drift it guarded has not gone away,
|
||||
it has moved — the linter is still pinned twice, now as the `FROM` line of
|
||||
`Dockerfile.lint` and the `FROM` line of the `Dockerfile` lint stage, with
|
||||
nothing syncing them, which is exactly what
|
||||
https://git.eeqj.de/sneak/sfdupes/issues/42 made a build failure. Its
|
||||
replacement is one new `script/verify-lint-image-pin`,
|
||||
run as a gate in both files, which compares the two references to
|
||||
each other and deliberately restates neither: a hardcoded expected
|
||||
digest would be a third copy and the same drift one file further out.
|
||||
`golangci-lint config verify` is included per the ruling, and the
|
||||
concern about its unpinned live HTTPS schema fetch was measured
|
||||
rather than assumed — under `--network none` the pinned binary both
|
||||
passes a valid config and rejects an invalid one with the jsonschema
|
||||
error, so it validates from an embedded schema and makes no network
|
||||
call of its own. The README scopes that to the gate steps rather
|
||||
than to linting as a whole: `Dockerfile.lint` runs `go mod download`
|
||||
above them, so a cold cache still needs the network and only a warm
|
||||
one lints offline. Verified: `make lint` green with every `PATH`
|
||||
directory containing a `golangci-lint` removed
|
||||
replacement is one new `script/verify-lint-image-pin`, run as a gate in both
|
||||
files, which compares the two references to each other and deliberately
|
||||
restates neither: a hardcoded expected digest would be a third copy and the
|
||||
same drift one file further out. `golangci-lint config verify` is included per
|
||||
the ruling, and the concern about its unpinned live HTTPS schema fetch was
|
||||
measured rather than assumed — under `--network none` the pinned binary both
|
||||
passes a valid config and rejects an invalid one with the jsonschema error, so
|
||||
it validates from an embedded schema and makes no network call of its own. The
|
||||
README scopes that to the gate steps rather than to linting as a whole:
|
||||
`Dockerfile.lint` runs `go mod download` above them, so a cold cache still
|
||||
needs the network and only a warm one lints offline. Verified: `make lint`
|
||||
green with every `PATH` directory containing a `golangci-lint` removed
|
||||
(`/home/user/go/bin`, `/home/user/.local/bin`, `/usr/local/bin`;
|
||||
`command -v golangci-lint` empty); two consecutive `script/lint` runs
|
||||
on an untouched tree both executed the linter, 27.7s and 28.7s in the
|
||||
lint step under distinct epochs with the `COPY . .` layer `CACHED`
|
||||
above them, at 42.2s and 41.8s wall clock — the no-cache rule was not
|
||||
weakened to shorten that. Negative control: a planted
|
||||
`var unusedIssue46Sentinel = 1` failed `script/lint` with
|
||||
`report.go:173:5: var unusedIssue46Sentinel is unused (unused)`, and
|
||||
failed `make docker` at `[lint 9/9]` with the build stage stopped at
|
||||
`[builder 3/12]` — `COPY --from=lint`, `script/bootstrap`, the test
|
||||
gate and `make build` all zero occurrences — then reverted clean. The
|
||||
drift guard fails on a tag-only disagreement, on a digest-only
|
||||
disagreement, and on an unreadable reference, naming both sides.
|
||||
`make docker` green in 5m35s with all six gates executing under one
|
||||
epoch (lint 37.6s, test 25.2s reporting
|
||||
`ok sneak.berlin/go/sfdupes 1.938s coverage: 88.5%`, not `(cached)`).
|
||||
The non-root quirk still holds: in the builder image with the Go test
|
||||
cache off, `--user 0:0` fails `TestScanHardlinkRunFailsTogether`
|
||||
(exit 1) where the unprivileged user passes (exit 0). Noted for
|
||||
follow-up, not fixed here: `golangci-lint` warns that the
|
||||
`gomodguard` linter is deprecated since v2.12.0 in favour of
|
||||
`gomodguard_v2`.
|
||||
`command -v golangci-lint` empty); two consecutive `script/lint` runs on an
|
||||
untouched tree both executed the linter, 27.7s and 28.7s in the lint step
|
||||
under distinct epochs with the `COPY . .` layer `CACHED` above them, at 42.2s
|
||||
and 41.8s wall clock — the no-cache rule was not weakened to shorten that.
|
||||
Negative control: a planted `var unusedIssue46Sentinel = 1` failed
|
||||
`script/lint` with
|
||||
`report.go:173:5: var unusedIssue46Sentinel is unused (unused)`, and failed
|
||||
`make docker` at `[lint 9/9]` with the build stage stopped at `[builder 3/12]`
|
||||
— `COPY --from=lint`, `script/bootstrap`, the test gate and `make build` all
|
||||
zero occurrences — then reverted clean. The drift guard fails on a tag-only
|
||||
disagreement, on a digest-only disagreement, and on an unreadable reference,
|
||||
naming both sides. `make docker` green in 5m35s with all six gates executing
|
||||
under one epoch (lint 37.6s, test 25.2s reporting
|
||||
`ok sneak.berlin/go/sfdupes 1.938s coverage: 88.5%`, not `(cached)`). The
|
||||
non-root quirk still holds: in the builder image with the Go test cache off,
|
||||
`--user 0:0` fails `TestScanHardlinkRunFailsTogether` (exit 1) where the
|
||||
unprivileged user passes (exit 0). Noted for follow-up, not fixed here:
|
||||
`golangci-lint` warns that the `gomodguard` linter is deprecated since v2.12.0
|
||||
in favour of `gomodguard_v2`.
|
||||
|
||||
- install the Docker build stage's prerequisites by running
|
||||
`script/bootstrap` instead of `apk add --no-cache make` inline
|
||||
(2026-08-09, branch `dockerfile-bootstrap`, closes #42): canonical
|
||||
`REPO_POLICIES.md:97` requires it, and the inline install left the
|
||||
build stage maintaining its own notion of the toolchain — exactly
|
||||
the divergence #24 exists to close, one layer down. The stage now
|
||||
copies `script/` plus `go.mod`/`go.sum` and runs `script/bootstrap`,
|
||||
which ends in `go mod download`, so the separate invocation of that
|
||||
is gone. `COPY --from=lint /usr/bin/golangci-lint` stays, and moves
|
||||
above the bootstrap layer. It is the only edge making this stage
|
||||
depend on the lint stage, so deleting it as redundant would end
|
||||
fail-fast linting silently. Letting bootstrap install its own linter
|
||||
here would have reintroduced the second toolchain and paid for a
|
||||
from-source build of it. What makes the two stages provably one
|
||||
toolchain rather than two that happen to agree is a new
|
||||
`script/verify-linter-pin`, run in the build stage on the binary
|
||||
that arrives from the lint stage, before bootstrap: it fails the
|
||||
build naming both versions unless that binary is the version
|
||||
`script/bootstrap` pins. Bootstrap's own check could not serve that
|
||||
purpose — it reinstalls its pin from source and then verifies
|
||||
whatever `PATH` resolves, so drift self-heals silently and a lint
|
||||
stage image bumped on its own would lint at the new version while
|
||||
`make check` ran at the old one, green. The linter version is pinned
|
||||
in two independent places (the lint stage image digest and
|
||||
- install the Docker build stage's prerequisites by running `script/bootstrap`
|
||||
instead of `apk add --no-cache make` inline (2026-08-09, branch
|
||||
`dockerfile-bootstrap`, closes #42): canonical `REPO_POLICIES.md:97` requires
|
||||
it, and the inline install left the build stage maintaining its own notion of
|
||||
the toolchain — exactly the divergence #24 exists to close, one layer down.
|
||||
The stage now copies `script/` plus `go.mod`/`go.sum` and runs
|
||||
`script/bootstrap`, which ends in `go mod download`, so the separate
|
||||
invocation of that is gone. `COPY --from=lint /usr/bin/golangci-lint` stays,
|
||||
and moves above the bootstrap layer. It is the only edge making this stage
|
||||
depend on the lint stage, so deleting it as redundant would end fail-fast
|
||||
linting silently. Letting bootstrap install its own linter here would have
|
||||
reintroduced the second toolchain and paid for a from-source build of it. What
|
||||
makes the two stages provably one toolchain rather than two that happen to
|
||||
agree is a new `script/verify-linter-pin`, run in the build stage on the
|
||||
binary that arrives from the lint stage, before bootstrap: it fails the build
|
||||
naming both versions unless that binary is the version `script/bootstrap`
|
||||
pins. Bootstrap's own check could not serve that purpose — it reinstalls its
|
||||
pin from source and then verifies whatever `PATH` resolves, so drift
|
||||
self-heals silently and a lint stage image bumped on its own would lint at the
|
||||
new version while `make check` ran at the old one, green. The linter version
|
||||
is pinned in two independent places (the lint stage image digest and
|
||||
`GOLANGCI_LINT_VERSION`) and nothing else keeps them in sync, so a
|
||||
half-applied bump is now a build failure. The pin is read out of
|
||||
`script/bootstrap`, which stays the single source of truth; a pin
|
||||
that cannot be read is a hard failure, not a skip. The check needs
|
||||
no `CHECK_EPOCH`: its only inputs are the copied binary and
|
||||
`script/`, so Docker invalidates the layer exactly when a cached
|
||||
result would stop being true, and it is documented with the other
|
||||
entrypoints in the README. `$GOPATH/bin` joins `PATH` because
|
||||
that is where bootstrap's `go install` lands and bootstrap verifies
|
||||
its installs against what `PATH` resolves — nothing in the image is
|
||||
shadowed by it, the directory does not exist until bootstrap runs.
|
||||
Everything added sits above `ARG CHECK_EPOCH`, and the `chown` and
|
||||
`USER builder` still precede `make check`. Verified: the guard fails
|
||||
the build with both versions named when the lint stage's linter is
|
||||
faked to a different version, and an unmodified build still passes
|
||||
it; bootstrap runs clean under Alpine's `sh` and its `apk` branch,
|
||||
installing `git` and `make` and finding the copied
|
||||
linter already at the pin; a second build served the bootstrap and
|
||||
dependency layers `CACHED` while both gates ran with a fresh epoch;
|
||||
a planted `unused` finding failed the build at the lint gate in
|
||||
48.9s with the build stage's `make check` never starting; and the
|
||||
suite run in the image as `--user 0:0` fails
|
||||
`TestScanHardlinkRunFailsTogether`, so the drop to the unprivileged
|
||||
user is still load-bearing. That last check needs the Go test cache
|
||||
disabled — the first attempt reported `ok ... (cached)` as root,
|
||||
reusing the result the build-time run had left in the shared cache,
|
||||
which would have read as a pass. Build wall time, on a shared host
|
||||
running many concurrent builds and so noisy: 2m13s on an unchanged
|
||||
tree, 2m17s and 4m29s for two builds after a source change, 5m14s
|
||||
cold. Only the cold one breaches the policy ceiling, and not because
|
||||
of this change — `chown -R builder:builder /src /home/builder` walks
|
||||
the module cache and re-runs on every source change, and it alone
|
||||
varied between 77s and 210s across those four builds, which is also
|
||||
the whole spread in the totals. The same cold measurement against
|
||||
`main` is 5m03s with a 209s `chown`. Filed as #43
|
||||
- bust the Docker layer cache for the gate steps, so `script/cibuild`
|
||||
and `script/docker` cannot report a green they did not earn
|
||||
(2026-08-09, branch `cibuild-cache-bust`, closes #32): both scripts
|
||||
were bare `docker build` invocations with no cache control, and the
|
||||
`Dockerfile` copies the tree before running its gates, so on an
|
||||
unchanged tree Docker served those layers from cache and the build
|
||||
exited 0 having executed nothing. That is not hypothetical here —
|
||||
every merge this repo has done is a non-fast-forward merge of an
|
||||
undiverged branch, so each merge commit's tree is byte-identical to
|
||||
the branch head's and each merge CI run was almost certainly a full
|
||||
cache hit; and PR #31's reviewer found `make docker` returning
|
||||
success as a 17-layer cache hit, catching it only by being
|
||||
suspicious. The fix is `ARG CHECK_EPOCH` with the scripts passing
|
||||
`--build-arg CHECK_EPOCH="$(date +%s)"`. Two details make or break
|
||||
it. `ARG` is scoped per stage and this `Dockerfile` has three gates
|
||||
across two — `make fmt-check` and `make lint` in the lint stage,
|
||||
`make check` in the build stage — so a single declaration would have
|
||||
left one stage silently cacheable; it is declared in both. And
|
||||
BuildKit hashes the expanded command, not the declaration, so a
|
||||
declared-but-unreferenced `ARG` invalidates nothing: each gate `RUN`
|
||||
echoes the epoch, which also puts the value in the build log as
|
||||
evidence the layer really ran. Placement is below the dependency
|
||||
layers on purpose — a build that goes cold every time would be a
|
||||
different bug, not a fix. Verified by running each script twice back
|
||||
to back on an unchanged tree under `BUILDKIT_PROGRESS=plain`: all
|
||||
three gates executed on all four runs, each with a fresh epoch in
|
||||
the log (`script/cibuild` 78.8s then 61.1s; `script/docker` 61.1s
|
||||
then 53.4s), and twelve steps were still served `CACHED` in the
|
||||
steady state — both `go mod download`s, `apk add`, `adduser`, the
|
||||
`chown`, every `go.mod`/`go.sum` and source copy, the linter copy
|
||||
out of the lint stage, and the binary copy into the runtime stage.
|
||||
The lint stage still gates the build stage: with a deliberate
|
||||
`unused` finding planted in the tree, the build failed at
|
||||
`make lint` in 36.1s and the build-stage `make check` never started.
|
||||
The build stage also still drops to the unprivileged `builder` user
|
||||
before `make check`, which the suite depends on rather than merely
|
||||
prefers: forcing the same image to run the tests as root fails
|
||||
`TestScanHardlinkRunFailsTogether`, because root reads straight
|
||||
through the `chmod(0)` the test uses to prove hard links are read
|
||||
once. This is the local fix only; propagating it to the canonical
|
||||
templates is `prompts` #26
|
||||
- check the installed golangci-lint version in `script/bootstrap`
|
||||
instead of only its presence (2026-08-09, branch
|
||||
`bootstrap-version-check`, closes #24): `missing golangci-lint` meant
|
||||
any linter already on `PATH` satisfied the check, so the pin was never
|
||||
consulted and the v2.12.2 bump from #3 was inert on every host that
|
||||
already had one — this host ran v2.10.1 against a v2.12.2 pin,
|
||||
`make check` went green, and `make docker` then rejected the same
|
||||
commit with findings the local gate never saw. The version now lives
|
||||
in one place, `GOLANGCI_LINT_VERSION`, with the `go install` module
|
||||
ref derived from it so a bump cannot half-apply; a
|
||||
`golangci_lint_version` helper parses `golangci-lint --version`
|
||||
(taking the field after the word `version` and tolerating an optional
|
||||
leading `v`, which the module ref carries and the binary's output does
|
||||
not), and any version that is not the pin — older, newer, absent or
|
||||
unparseable — is reinstalled. The install is then verified against the
|
||||
binary `PATH` actually resolves: `go install` writes into `GOBIN` (or
|
||||
`GOPATH/bin`) while `make lint` runs whichever `golangci-lint` comes
|
||||
first on `PATH`, so a wrong-version one sitting ahead of it — nix,
|
||||
apt, brew, apk, or the `/usr/local/bin` copy the `Dockerfile` builder
|
||||
stage makes — would swallow the install and leave the local gate
|
||||
disagreeing with CI under an affirmative `bootstrap complete`.
|
||||
Bootstrap now re-reads the effective version after installing and, on
|
||||
a mismatch, prints both paths and both versions to stderr and exits
|
||||
non-zero instead of claiming success; it does not reorder anyone's
|
||||
`script/bootstrap`, which stays the single source of truth; a pin that cannot
|
||||
be read is a hard failure, not a skip. The check needs no `CHECK_EPOCH`: its
|
||||
only inputs are the copied binary and `script/`, so Docker invalidates the
|
||||
layer exactly when a cached result would stop being true, and it is documented
|
||||
with the other entrypoints in the README. `$GOPATH/bin` joins `PATH` because
|
||||
that is where bootstrap's `go install` lands and bootstrap verifies its
|
||||
installs against what `PATH` resolves — nothing in the image is shadowed by
|
||||
it, the directory does not exist until bootstrap runs. Everything added sits
|
||||
above `ARG CHECK_EPOCH`, and the `chown` and `USER builder` still precede
|
||||
`make check`. Verified: the guard fails the build with both versions named
|
||||
when the lint stage's linter is faked to a different version, and an
|
||||
unmodified build still passes it; bootstrap runs clean under Alpine's `sh` and
|
||||
its `apk` branch, installing `git` and `make` and finding the copied linter
|
||||
already at the pin; a second build served the bootstrap and dependency layers
|
||||
`CACHED` while both gates ran with a fresh epoch; a planted `unused` finding
|
||||
failed the build at the lint gate in 48.9s with the build stage's `make check`
|
||||
never starting; and the suite run in the image as `--user 0:0` fails
|
||||
`TestScanHardlinkRunFailsTogether`, so the drop to the unprivileged user is
|
||||
still load-bearing. That last check needs the Go test cache disabled — the
|
||||
first attempt reported `ok ... (cached)` as root, reusing the result the
|
||||
build-time run had left in the shared cache, which would have read as a pass.
|
||||
Build wall time, on a shared host running many concurrent builds and so noisy:
|
||||
2m13s on an unchanged tree, 2m17s and 4m29s for two builds after a source
|
||||
change, 5m14s cold. Only the cold one breaches the policy ceiling, and not
|
||||
because of this change — `chown -R builder:builder /src /home/builder` walks
|
||||
the module cache and re-runs on every source change, and it alone varied
|
||||
between 77s and 210s across those four builds, which is also the whole spread
|
||||
in the totals. The same cold measurement against `main` is 5m03s with a 209s
|
||||
`chown`. Filed as #43
|
||||
- bust the Docker layer cache for the gate steps, so `script/cibuild` and
|
||||
`script/docker` cannot report a green they did not earn (2026-08-09, branch
|
||||
`cibuild-cache-bust`, closes #32): both scripts were bare `docker build`
|
||||
invocations with no cache control, and the `Dockerfile` copies the tree before
|
||||
running its gates, so on an unchanged tree Docker served those layers from
|
||||
cache and the build exited 0 having executed nothing. That is not hypothetical
|
||||
here — every merge this repo has done is a non-fast-forward merge of an
|
||||
undiverged branch, so each merge commit's tree is byte-identical to the branch
|
||||
head's and each merge CI run was almost certainly a full cache hit; and PR
|
||||
#31's reviewer found `make docker` returning success as a 17-layer cache hit,
|
||||
catching it only by being suspicious. The fix is `ARG CHECK_EPOCH` with the
|
||||
scripts passing `--build-arg CHECK_EPOCH="$(date +%s)"`. Two details make or
|
||||
break it. `ARG` is scoped per stage and this `Dockerfile` has three gates
|
||||
across two — `make fmt-check` and `make lint` in the lint stage, `make check`
|
||||
in the build stage — so a single declaration would have left one stage
|
||||
silently cacheable; it is declared in both. And BuildKit hashes the expanded
|
||||
command, not the declaration, so a declared-but-unreferenced `ARG` invalidates
|
||||
nothing: each gate `RUN` echoes the epoch, which also puts the value in the
|
||||
build log as evidence the layer really ran. Placement is below the dependency
|
||||
layers on purpose — a build that goes cold every time would be a different
|
||||
bug, not a fix. Verified by running each script twice back to back on an
|
||||
unchanged tree under `BUILDKIT_PROGRESS=plain`: all three gates executed on
|
||||
all four runs, each with a fresh epoch in the log (`script/cibuild` 78.8s then
|
||||
61.1s; `script/docker` 61.1s then 53.4s), and twelve steps were still served
|
||||
`CACHED` in the steady state — both `go mod download`s, `apk add`, `adduser`,
|
||||
the `chown`, every `go.mod`/`go.sum` and source copy, the linter copy out of
|
||||
the lint stage, and the binary copy into the runtime stage. The lint stage
|
||||
still gates the build stage: with a deliberate `unused` finding planted in the
|
||||
tree, the build failed at `make lint` in 36.1s and the build-stage
|
||||
`make check` never started. The build stage also still drops to the
|
||||
unprivileged `builder` user before `make check`, which the suite depends on
|
||||
rather than merely prefers: forcing the same image to run the tests as root
|
||||
fails `TestScanHardlinkRunFailsTogether`, because root reads straight through
|
||||
the `chmod(0)` the test uses to prove hard links are read once. This is the
|
||||
local fix only; propagating it to the canonical templates is `prompts` #26
|
||||
- check the installed golangci-lint version in `script/bootstrap` instead of
|
||||
only its presence (2026-08-09, branch `bootstrap-version-check`, closes #24):
|
||||
`missing golangci-lint` meant any linter already on `PATH` satisfied the
|
||||
check, so the pin was never consulted and the v2.12.2 bump from #3 was inert
|
||||
on every host that already had one — this host ran v2.10.1 against a v2.12.2
|
||||
pin, `make check` went green, and `make docker` then rejected the same commit
|
||||
with findings the local gate never saw. The version now lives in one place,
|
||||
`GOLANGCI_LINT_VERSION`, with the `go install` module ref derived from it so a
|
||||
bump cannot half-apply; a `golangci_lint_version` helper parses
|
||||
`golangci-lint --version` (taking the field after the word `version` and
|
||||
tolerating an optional leading `v`, which the module ref carries and the
|
||||
binary's output does not), and any version that is not the pin — older, newer,
|
||||
absent or unparseable — is reinstalled. The install is then verified against
|
||||
the binary `PATH` actually resolves: `go install` writes into `GOBIN` (or
|
||||
`GOPATH/bin`) while `make lint` runs whichever `golangci-lint` comes first on
|
||||
`PATH`, so a wrong-version one sitting ahead of it — nix, apt, brew, apk, or
|
||||
the `/usr/local/bin` copy the `Dockerfile` builder stage makes — would swallow
|
||||
the install and leave the local gate disagreeing with CI under an affirmative
|
||||
`bootstrap complete`. Bootstrap now re-reads the effective version after
|
||||
installing and, on a mismatch, prints both paths and both versions to stderr
|
||||
and exits non-zero instead of claiming success; it does not reorder anyone's
|
||||
`PATH` or delete their binary. The `--version` call keeps its stderr
|
||||
connected, so a present-but-broken binary says why rather than
|
||||
reinstalling forever in silence, and is bounded by `timeout(1)` where
|
||||
that exists, so a wedged binary cannot hang bootstrap. `git`, `make`
|
||||
and `go` keep their presence-only checks and now say why in a
|
||||
comment: they are host package-manager tools the repo deliberately
|
||||
does not pin, with `go.mod` governing the language version and the
|
||||
digest-pinned images covering reproducible builds. Verified on this
|
||||
host by bootstrapping from v2.10.1 to v2.12.2 and running it again to
|
||||
a no-op, plus stub runs of the real script under `dash` covering a
|
||||
thirteen-input parse matrix (absent, older, newer, host-style,
|
||||
image-style, leading-`v`, stderr-only, empty, non-zero exit, impostor
|
||||
binary, `(devel)`, trailing `version`), a shadowed install that must
|
||||
exit non-zero, an install destination not on `PATH` at all, `GOBIN`
|
||||
set, and a wedged binary that must hit the timeout; `make check` and
|
||||
`make lint` are clean at v2.12.2, so v2.10.1 was not hiding any
|
||||
findings on `main`
|
||||
connected, so a present-but-broken binary says why rather than reinstalling
|
||||
forever in silence, and is bounded by `timeout(1)` where that exists, so a
|
||||
wedged binary cannot hang bootstrap. `git`, `make` and `go` keep their
|
||||
presence-only checks and now say why in a comment: they are host
|
||||
package-manager tools the repo deliberately does not pin, with `go.mod`
|
||||
governing the language version and the digest-pinned images covering
|
||||
reproducible builds. Verified on this host by bootstrapping from v2.10.1 to
|
||||
v2.12.2 and running it again to a no-op, plus stub runs of the real script
|
||||
under `dash` covering a thirteen-input parse matrix (absent, older, newer,
|
||||
host-style, image-style, leading-`v`, stderr-only, empty, non-zero exit,
|
||||
impostor binary, `(devel)`, trailing `version`), a shadowed install that must
|
||||
exit non-zero, an install destination not on `PATH` at all, `GOBIN` set, and a
|
||||
wedged binary that must hit the timeout; `make check` and `make lint` are
|
||||
clean at v2.12.2, so v2.10.1 was not hiding any findings on `main`
|
||||
- unwind the hash worker pool on the error path (2026-08-09, branch
|
||||
`hash-pool-cleanup`, closes #6): `hashPhase` used to return the
|
||||
moment `recordRun` failed and abandon the pool — the feeder parked
|
||||
forever on a full `jobs` channel and every worker on a full
|
||||
`results` channel. That only stopped being invisible when #4 landed
|
||||
and `runScan` began unwinding instead of calling `os.Exit`. The
|
||||
pool is now an owned, context-aware `hashPool`: every blocking send
|
||||
in the feeder and the workers selects on `ctx.Done()`, `jobs` is
|
||||
closed on every path out, and `hashPhase` defers `pool.stop()`,
|
||||
which cancels and then drains `results` until the last goroutine
|
||||
has exited — draining is what frees a worker already parked on a
|
||||
send. `ctx` is threaded from `cmd.Context()` through `runScan`,
|
||||
`syncScan`, both worker pools and the whole database layer (it is
|
||||
the first parameter everywhere), so #5 can hand this path a signal
|
||||
and needs to add nothing else. The walk pool never leaked, because
|
||||
`walkPhase` always drains its events to close, but it has the same
|
||||
unbounded-send shape and #5 will give it an early return, so it
|
||||
gets the same treatment plus a `ctx.Err()` guard after the walk: a
|
||||
cancelled walk yields a partial size census, and every file it never
|
||||
reached looks vanished to the update phase. That phase's own
|
||||
`BeginTx` fails on the same cancelled context before deleting
|
||||
anything, so the guard is defence in depth rather than the only
|
||||
barrier — but it is the one that survives #5 deciding an interrupted
|
||||
scan may commit what it has. Tests drive `run(scan)` against a
|
||||
database whose insert trigger aborts, and assert both that the scan
|
||||
fails instead of hanging and that `runtime.NumGoroutine()` polls
|
||||
back to its pre-scan baseline; a second set cancels a scan part-way
|
||||
through the walk — deterministically, by counting the scan's own
|
||||
consultations of `ctx.Done()` rather than racing a timer — and
|
||||
asserts that it stops at the guard holding a partial census and a
|
||||
still-populated record index, with every record intact. The
|
||||
remaining cancellation branches of both pools are covered by direct
|
||||
tests of `sendEvent`, the walk workers, `dispatchDirs`,
|
||||
`feedHashJobs`, `hashWorker` and `hashPhase`
|
||||
- guarantee the database is closed on every fatal exit path
|
||||
(2026-08-09, branch `db-close-on-fatal`, closes #4): `fatalf` and
|
||||
its `os.Exit(1)` are gone, so the deferred `db.Close()` — and with
|
||||
it the SQLite WAL checkpoint — now actually runs when a subcommand
|
||||
fails; `runScan`, `runReport`, `runTrees`, `loadRecords` and
|
||||
`resolveRoots` return errors instead. The single exit point is `run`
|
||||
in `main.go`: it maps a `fatalError` (anything a subcommand
|
||||
returned) to exit 1 and cobra's own argument and flag errors to exit
|
||||
2, which keeps a runtime failure from being reported as a usage
|
||||
error or printing the usage text. New `main_test.go` drives the CLI
|
||||
in-process and asserts the exit codes from README §Error handling
|
||||
plus the stdout/stderr split, including that a fatal error raised
|
||||
after the database is open leaves no `-wal`/`-shm` sidecar behind
|
||||
for `scan`, `report` or `trees`
|
||||
- update golangci-lint to v2.12.2 with the canonical config
|
||||
(2026-08-09, branch `golangci-v2.12.2`, merged as `38a01bd`,
|
||||
closes #3): bumped the pinned linter in the `Dockerfile` lint
|
||||
stage and `script/bootstrap` from v2.12.1 to v2.12.2, and replaced
|
||||
`.golangci.yml` with the canonical file — the linter settings
|
||||
`hash-pool-cleanup`, closes #6): `hashPhase` used to return the moment
|
||||
`recordRun` failed and abandon the pool — the feeder parked forever on a full
|
||||
`jobs` channel and every worker on a full `results` channel. That only stopped
|
||||
being invisible when #4 landed and `runScan` began unwinding instead of
|
||||
calling `os.Exit`. The pool is now an owned, context-aware `hashPool`: every
|
||||
blocking send in the feeder and the workers selects on `ctx.Done()`, `jobs` is
|
||||
closed on every path out, and `hashPhase` defers `pool.stop()`, which cancels
|
||||
and then drains `results` until the last goroutine has exited — draining is
|
||||
what frees a worker already parked on a send. `ctx` is threaded from
|
||||
`cmd.Context()` through `runScan`, `syncScan`, both worker pools and the whole
|
||||
database layer (it is the first parameter everywhere), so #5 can hand this
|
||||
path a signal and needs to add nothing else. The walk pool never leaked,
|
||||
because `walkPhase` always drains its events to close, but it has the same
|
||||
unbounded-send shape and #5 will give it an early return, so it gets the same
|
||||
treatment plus a `ctx.Err()` guard after the walk: a cancelled walk yields a
|
||||
partial size census, and every file it never reached looks vanished to the
|
||||
update phase. That phase's own `BeginTx` fails on the same cancelled context
|
||||
before deleting anything, so the guard is defence in depth rather than the
|
||||
only barrier — but it is the one that survives #5 deciding an interrupted scan
|
||||
may commit what it has. Tests drive `run(scan)` against a database whose
|
||||
insert trigger aborts, and assert both that the scan fails instead of hanging
|
||||
and that `runtime.NumGoroutine()` polls back to its pre-scan baseline; a
|
||||
second set cancels a scan part-way through the walk — deterministically, by
|
||||
counting the scan's own consultations of `ctx.Done()` rather than racing a
|
||||
timer — and asserts that it stops at the guard holding a partial census and a
|
||||
still-populated record index, with every record intact. The remaining
|
||||
cancellation branches of both pools are covered by direct tests of
|
||||
`sendEvent`, the walk workers, `dispatchDirs`, `feedHashJobs`, `hashWorker`
|
||||
and `hashPhase`
|
||||
- guarantee the database is closed on every fatal exit path (2026-08-09, branch
|
||||
`db-close-on-fatal`, closes #4): `fatalf` and its `os.Exit(1)` are gone, so
|
||||
the deferred `db.Close()` — and with it the SQLite WAL checkpoint — now
|
||||
actually runs when a subcommand fails; `runScan`, `runReport`, `runTrees`,
|
||||
`loadRecords` and `resolveRoots` return errors instead. The single exit point
|
||||
is `run` in `main.go`: it maps a `fatalError` (anything a subcommand returned)
|
||||
to exit 1 and cobra's own argument and flag errors to exit 2, which keeps a
|
||||
runtime failure from being reported as a usage error or printing the usage
|
||||
text. New `main_test.go` drives the CLI in-process and asserts the exit codes
|
||||
from README §Error handling plus the stdout/stderr split, including that a
|
||||
fatal error raised after the database is open leaves no `-wal`/`-shm` sidecar
|
||||
behind for `scan`, `report` or `trees`
|
||||
- update golangci-lint to v2.12.2 with the canonical config (2026-08-09, branch
|
||||
`golangci-v2.12.2`, merged as `38a01bd`, closes #3): bumped the pinned linter
|
||||
in the `Dockerfile` lint stage and `script/bootstrap` from v2.12.1 to v2.12.2,
|
||||
and replaced `.golangci.yml` with the canonical file — the linter settings
|
||||
(`lll`, `funlen`, `cyclop`, `dupl` thresholds) now live under
|
||||
`linters.settings` per the v2 schema, so they are actually
|
||||
applied; no new lint findings surfaced
|
||||
- convert Makefile targets to scripts-to-rule-them-all `script/`
|
||||
entrypoints like the other managed repos (2026-07-26, commit
|
||||
`3abeacf`, closes #1): all 12 `script/` entrypoints exist
|
||||
(`bootstrap`, `setup`, `projectname`, `test`, `lint`, `fmt`,
|
||||
`fmt-check`, `check`, `docker`, `cibuild`, `precommit`,
|
||||
`install-precommit`) and every Makefile target is now a thin shim
|
||||
over them, matching the other managed repos
|
||||
`linters.settings` per the v2 schema, so they are actually applied; no new
|
||||
lint findings surfaced
|
||||
- convert Makefile targets to scripts-to-rule-them-all `script/` entrypoints
|
||||
like the other managed repos (2026-07-26, commit `3abeacf`, closes #1): all 12
|
||||
`script/` entrypoints exist (`bootstrap`, `setup`, `projectname`, `test`,
|
||||
`lint`, `fmt`, `fmt-check`, `check`, `docker`, `cibuild`, `precommit`,
|
||||
`install-precommit`) and every Makefile target is now a thin shim over them,
|
||||
matching the other managed repos
|
||||
- make the binary the default Make target (2026-07-24, branch
|
||||
`make-default-target`): plain `make` now builds `sfdupes`
|
||||
(previously it ran `check` plus `build`); `make build` remains as
|
||||
an alias
|
||||
- scan-wide phases, concurrent operands, batched updates (2026-07-24,
|
||||
branch `scan-wide-phases`): all operands seed the shared walk pool
|
||||
and every pass runs once over the whole scan, so totals and ETAs
|
||||
are scan-global; the per-operand walk/hash/update cycles and their
|
||||
stderr announcements are gone; the update pass commits in batched
|
||||
transactions — the filesystem is authoritative and the database an
|
||||
eventually-consistent reflection, so scan-level atomicity is not
|
||||
required
|
||||
`make-default-target`): plain `make` now builds `sfdupes` (previously it ran
|
||||
`check` plus `build`); `make build` remains as an alias
|
||||
- scan-wide phases, concurrent operands, batched updates (2026-07-24, branch
|
||||
`scan-wide-phases`): all operands seed the shared walk pool and every pass
|
||||
runs once over the whole scan, so totals and ETAs are scan-global; the
|
||||
per-operand walk/hash/update cycles and their stderr announcements are gone;
|
||||
the update pass commits in batched transactions — the filesystem is
|
||||
authoritative and the database an eventually-consistent reflection, so
|
||||
scan-level atomicity is not required
|
||||
- split the stat pass back out of the walk (2026-07-24, branch
|
||||
`parallel-phases`): phases are strictly sequential again — walk,
|
||||
stat, hash, update per operand — with parallelism only inside each
|
||||
phase; the walk enumerates paths with per-directory workers and the
|
||||
stat pass lstats them with per-file workers, restoring the exact
|
||||
total/ETA stat bar
|
||||
- announce each operand on stderr before its passes (2026-07-24,
|
||||
branch `scan-operand-progress`): with per-operand walk/hash/update
|
||||
cycles, a multi-operand run (e.g. `scan /srv/*`) showed pass totals
|
||||
that looked like the whole run's — an operator watching operand 3 of
|
||||
14 hash 300k files concluded 20M files were being skipped
|
||||
- parallel walk (2026-07-24, branch `parallel-walk`): the walk pass
|
||||
was a single goroutine and took hours at ~20M files on a busy pool
|
||||
(observed: 22M files in 4h on a ZFS server); it is now a
|
||||
per-directory worker-pool traversal that records size/mtime during
|
||||
the walk (folding away the separate stat pass, halving metadata
|
||||
I/O), and each `PATH` operand commits in its own transaction so an
|
||||
interrupted scan keeps completed operands
|
||||
`parallel-phases`): phases are strictly sequential again — walk, stat, hash,
|
||||
update per operand — with parallelism only inside each phase; the walk
|
||||
enumerates paths with per-directory workers and the stat pass lstats them with
|
||||
per-file workers, restoring the exact total/ETA stat bar
|
||||
- announce each operand on stderr before its passes (2026-07-24, branch
|
||||
`scan-operand-progress`): with per-operand walk/hash/update cycles, a
|
||||
multi-operand run (e.g. `scan /srv/*`) showed pass totals that looked like the
|
||||
whole run's — an operator watching operand 3 of 14 hash 300k files concluded
|
||||
20M files were being skipped
|
||||
- parallel walk (2026-07-24, branch `parallel-walk`): the walk pass was a single
|
||||
goroutine and took hours at ~20M files on a busy pool (observed: 22M files in
|
||||
4h on a ZFS server); it is now a per-directory worker-pool traversal that
|
||||
records size/mtime during the walk (folding away the separate stat pass,
|
||||
halving metadata I/O), and each `PATH` operand commits in its own transaction
|
||||
so an interrupted scan keeps completed operands
|
||||
|
||||
- persistent scan database (2026-07-24, branch `persistent-database`):
|
||||
`scan` now maintains a SQLite database (`modernc.org/sqlite`, pure
|
||||
Go, cgo stays disabled) keyed by absolute path that survives between
|
||||
runs — a rescan hashes only new or changed files (by mtime/size),
|
||||
deletes records for files vanished from under the scanned operands,
|
||||
and leaves records outside them untouched, so `scan` can be cronned
|
||||
daily; `report` and `trees` read the database (no positional
|
||||
arguments) instead of a scan stream. Database at
|
||||
`/var/lib/sfdupes/db.sqlite`, overridable via `SFDUPES_DATABASE`;
|
||||
WAL journaling plus a single-transaction update keep a report run
|
||||
during a scan safe
|
||||
- add the `origin` remote (`git@git.eeqj.de:sneak/sfdupes.git`), tag
|
||||
`v0.0.1`, and push `main` plus tags (2026-07-23)
|
||||
- persistent scan database (2026-07-24, branch `persistent-database`): `scan`
|
||||
now maintains a SQLite database (`modernc.org/sqlite`, pure Go, cgo stays
|
||||
disabled) keyed by absolute path that survives between runs — a rescan hashes
|
||||
only new or changed files (by mtime/size), deletes records for files vanished
|
||||
from under the scanned operands, and leaves records outside them untouched, so
|
||||
`scan` can be cronned daily; `report` and `trees` read the database (no
|
||||
positional arguments) instead of a scan stream. Database at
|
||||
`/var/lib/sfdupes/db.sqlite`, overridable via `SFDUPES_DATABASE`; WAL
|
||||
journaling plus a single-transaction update keep a report run during a scan
|
||||
safe
|
||||
- add the `origin` remote (`git@git.eeqj.de:sneak/sfdupes.git`), tag `v0.0.1`,
|
||||
and push `main` plus tags (2026-07-23)
|
||||
- `scan` CLI rework (2026-07-23, branch `scan-required-paths`): required
|
||||
`PATH...` operands via cobra flags replacing the `/srv` `-root`
|
||||
default; new `-x`/`--one-file-system` flag (GNU convention) to stop
|
||||
at filesystem boundaries, which are crossed by default
|
||||
`PATH...` operands via cobra flags replacing the `/srv` `-root` default; new
|
||||
`-x`/`--one-file-system` flag (GNU convention) to stop at filesystem
|
||||
boundaries, which are crossed by default
|
||||
- bring the repo into full policy compliance (2026-07-23, branch
|
||||
`repo-policy-compliance`; checklist below)
|
||||
- `git init` with README-only first commit; code baseline committed on
|
||||
`main` (2026-07-22)
|
||||
- `git init` with README-only first commit; code baseline committed on `main`
|
||||
(2026-07-22)
|
||||
- implement `scan`, `report`, and `trees` subcommands (pre-git history)
|
||||
|
||||
# Future Steps
|
||||
|
||||
- possible later features (explicitly out of scope per README):
|
||||
full-content verification of candidates, removal-script helpers
|
||||
- possible later features (explicitly out of scope per README): full-content
|
||||
verification of candidates, removal-script helpers
|
||||
|
||||
# Repo Policy Compliance
|
||||
|
||||
Audited 2026-07-22 against `REPO_POLICIES.md` (2026-07-06), the existing
|
||||
repo checklist, and the Go styleguide. Code is already gofmt-clean, so no
|
||||
standalone formatting commit is needed.
|
||||
Audited 2026-07-22 against `REPO_POLICIES.md` (2026-07-06), the existing repo
|
||||
checklist, and the Go styleguide. Code is already gofmt-clean, so no standalone
|
||||
formatting commit is needed.
|
||||
|
||||
- [x] `.gitignore` missing — the compiled `sfdupes` binary and
|
||||
`files.dat` sit untracked in the tree; needs OS/editor/Go
|
||||
artifacts plus secrets patterns
|
||||
- [x] `.gitignore` missing — the compiled `sfdupes` binary and `files.dat` sit
|
||||
untracked in the tree; needs OS/editor/Go artifacts plus secrets patterns
|
||||
- [x] `.editorconfig` missing
|
||||
- [x] `LICENSE` missing and README has no License section (MIT assumed
|
||||
from house convention — user to confirm)
|
||||
- [x] `LICENSE` missing and README has no License section (MIT assumed from
|
||||
house convention — user to confirm)
|
||||
- [x] `REPO_POLICIES.md` missing from repo root
|
||||
- [x] `.golangci.yml` missing (install canonical copy); code must then
|
||||
pass `make lint` (150 findings fixed; `make lint` is clean)
|
||||
- [x] `Makefile` lacks required targets `test`, `lint`, `fmt`,
|
||||
`fmt-check`, `docker`, `hooks`; `check` currently depends on
|
||||
`build`, which writes the binary (`make check` must not modify
|
||||
files)
|
||||
- [x] no tests — `go test ./...` has nothing to run; policy requires
|
||||
real tests with a 30-second timeout and the conditional `-v`
|
||||
rerun pattern (suite covers parsing, grouping, digests,
|
||||
suppression, hashing, and the scan pipeline; 64% coverage)
|
||||
- [x] `Dockerfile` missing — Go multistage with hash-pinned images:
|
||||
fail-fast lint stage, build stage running `make check`
|
||||
- [x] `.golangci.yml` missing (install canonical copy); code must then pass
|
||||
`make lint` (150 findings fixed; `make lint` is clean)
|
||||
- [x] `Makefile` lacks required targets `test`, `lint`, `fmt`, `fmt-check`,
|
||||
`docker`, `hooks`; `check` currently depends on `build`, which writes the
|
||||
binary (`make check` must not modify files)
|
||||
- [x] no tests — `go test ./...` has nothing to run; policy requires real tests
|
||||
with a 30-second timeout and the conditional `-v` rerun pattern (suite
|
||||
covers parsing, grouping, digests, suppression, hashing, and the scan
|
||||
pipeline; 64% coverage)
|
||||
- [x] `Dockerfile` missing — Go multistage with hash-pinned images: fail-fast
|
||||
lint stage, build stage running `make check`
|
||||
- [x] `.dockerignore` missing
|
||||
- [x] `.gitea/workflows/check.yml` missing (`docker build .` on push,
|
||||
checkout action pinned by commit SHA)
|
||||
- [x] `.gitea/workflows/check.yml` missing (`docker build .` on push, checkout
|
||||
action pinned by commit SHA)
|
||||
- [x] README lacks required sections: Description first line
|
||||
(name/purpose/category/license/author), Getting Started,
|
||||
Rationale, TODO, License, Author
|
||||
- [x] README non-goal "no git repository setup and no CI" is stale now
|
||||
that the repo is under git with CI
|
||||
- [x] pre-commit hook not installed (`make hooks` once the target
|
||||
exists)
|
||||
(name/purpose/category/license/author), Getting Started, Rationale, TODO,
|
||||
License, Author
|
||||
- [x] README non-goal "no git repository setup and no CI" is stale now that the
|
||||
repo is under git with CI
|
||||
- [x] pre-commit hook not installed (`make hooks` once the target exists)
|
||||
|
||||
Accepted divergences (no action):
|
||||
|
||||
- flat single-package layout with `.go` files in the repo root — fine
|
||||
for a small single-binary tool per the Go styleguide; the tracker
|
||||
audit agrees
|
||||
- `go test` runs without `-race` — the repo mandates `CGO_ENABLED=0`
|
||||
(pure-Go builds) and the race detector requires cgo
|
||||
- flat single-package layout with `.go` files in the repo root — fine for a
|
||||
small single-binary tool per the Go styleguide; the tracker audit agrees
|
||||
- `go test` runs without `-race` — the repo mandates `CGO_ENABLED=0` (pure-Go
|
||||
builds) and the race detector requires cgo
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"devDependencies": {
|
||||
"prettier": "3.8.1"
|
||||
}
|
||||
}
|
||||
+13
-11
@@ -3,10 +3,12 @@
|
||||
# this repo. Idempotent: every install is guarded by a check so already
|
||||
# installed tools are skipped. Base tooling comes from nix, apt, brew,
|
||||
# or apk (detected in that order); assumes nothing is present (not git,
|
||||
# make, or go). The linter is NOT installed: golangci-lint runs via
|
||||
# docker only (script/lint), pinned by image digest, so the only lint
|
||||
# make, or go). Neither the linter nor the Markdown formatter is
|
||||
# installed: golangci-lint (script/lint) and prettier (script/fmt,
|
||||
# script/fmt-check) run via docker only, pinned by hash, so their only
|
||||
# prerequisite is a working docker — which is warned about, not
|
||||
# installed, because everything except linting works without it.
|
||||
# installed, because everything except linting and formatting works
|
||||
# without it.
|
||||
set -eu
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
@@ -71,15 +73,15 @@ main() {
|
||||
if missing make; then pkg_install gnumake make make make; fi
|
||||
if missing go; then pkg_install go golang go go; fi
|
||||
|
||||
# Linting runs via docker only (script/lint), so docker is a lint
|
||||
# prerequisite rather than something bootstrap installs. Warn, do
|
||||
# not fail: everything except `make lint` — and, through it,
|
||||
# `make check`, `make docker` and the pre-commit hook — works
|
||||
# without it.
|
||||
# Linting and Markdown formatting run via docker only, so docker is
|
||||
# their prerequisite rather than something bootstrap installs. Warn,
|
||||
# do not fail: everything except `make lint`, `make fmt` and
|
||||
# `make fmt-check` — and, through them, `make check`, `make docker`
|
||||
# and the pre-commit hook — works without it.
|
||||
if missing docker; then
|
||||
echo "bootstrap: WARNING: docker not found; make lint, make check" >&2
|
||||
echo "bootstrap: and make docker require it. Install docker to" >&2
|
||||
echo "bootstrap: run the linter." >&2
|
||||
echo "bootstrap: WARNING: docker not found; make lint, make fmt," >&2
|
||||
echo "bootstrap: make fmt-check, make check and make docker" >&2
|
||||
echo "bootstrap: require it." >&2
|
||||
fi
|
||||
|
||||
go mod download
|
||||
|
||||
+10
-10
@@ -3,17 +3,17 @@
|
||||
# push.
|
||||
#
|
||||
# The Dockerfile runs the gates individually as build steps, not the
|
||||
# make check aggregate: the lint stage runs make fmt-check,
|
||||
# make check aggregate: the lint stage runs the gofmt check,
|
||||
# script/verify-lint-image-pin, golangci-lint config verify and
|
||||
# golangci-lint run; the build stage, dropped to an unprivileged user,
|
||||
# runs make test and make fmt-check. Neither make lint nor make check
|
||||
# appears, because both reach script/lint, which is itself a docker
|
||||
# build, and a docker build cannot run inside one. Lint is not skipped
|
||||
# by that — the linter is invoked directly in the lint stage, and the
|
||||
# build stage's COPY --from=lint makes that stage a prerequisite, so
|
||||
# BuildKit must finish it first. Between the two stages everything
|
||||
# make check would run has run, which is why a successful build here
|
||||
# implies the repo is green.
|
||||
# golangci-lint run; the markdown stage runs the prettier check; the
|
||||
# build stage, dropped to an unprivileged user, runs make test. None of
|
||||
# make lint, make fmt-check or make check appears, because each runs
|
||||
# docker, and docker cannot run inside a docker build. Nothing is
|
||||
# skipped by that — the linter, gofmt and prettier are invoked directly
|
||||
# in their stages, and the build stage's COPY --from lines make those
|
||||
# stages prerequisites, so BuildKit must finish them first. Between the
|
||||
# three stages everything make check would run has run, which is why a
|
||||
# successful build here implies the repo is green.
|
||||
#
|
||||
# That implication holds only because of CHECK_EPOCH. A COPY layer is
|
||||
# invalidated only by changed content, and a rebuild of an unchanged
|
||||
|
||||
+5
-5
@@ -4,11 +4,11 @@
|
||||
#
|
||||
# CHECK_EPOCH is passed for the same reason script/cibuild passes it:
|
||||
# without it Docker serves the Dockerfile's gate layers from cache on an
|
||||
# unchanged tree and this exits 0 having run neither the lint stage's
|
||||
# gates nor the builder stage's test and fmt-check gates. This is the
|
||||
# set of gates a developer or reviewer runs by hand, so a cached pass
|
||||
# here is the most misleading result the repo can produce. Dependency
|
||||
# layers sit above the ARG and stay cached.
|
||||
# unchanged tree and this exits 0 having run none of the lint stage's
|
||||
# gates, the markdown stage's prettier gate or the builder stage's test
|
||||
# gate. This is the set of gates a developer or reviewer runs by hand,
|
||||
# so a cached pass here is the most misleading result the repo can
|
||||
# produce. Dependency layers sit above the ARG and stay cached.
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
|
||||
+12
-2
@@ -1,12 +1,22 @@
|
||||
#!/bin/sh
|
||||
# script/fmt: format all files (writes).
|
||||
# script/fmt: format all files (writes): the Go sources with gofmt, the
|
||||
# Markdown with prettier. prettier is never installed on the host: it
|
||||
# runs from the Dockerfile's prettier stage with the repository mounted,
|
||||
# as the calling user so the files it rewrites keep their owner. The tag
|
||||
# makes each build replace the previous image instead of leaving another
|
||||
# one behind.
|
||||
set -eu
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
gofmt -s -w .
|
||||
image="$("$SCRIPT_DIR/projectname")-prettier"
|
||||
docker build -q --target prettier -t "$image" . >/dev/null
|
||||
docker run --rm --user "$(id -u):$(id -g)" -v "$ROOT:/src" "$image" \
|
||||
prettier --write '**/*.md' --tab-width 4 --prose-wrap always
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
+25
-4
@@ -1,18 +1,39 @@
|
||||
#!/bin/sh
|
||||
# script/fmt-check: check formatting (read-only). Same scope as
|
||||
# script/fmt, but fails instead of writing.
|
||||
# script/fmt, but fails instead of writing. gofmt and prettier both run
|
||||
# every time and each reports its own failure, so the output says which
|
||||
# one failed.
|
||||
set -eu
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
files="$(gofmt -s -l .)"
|
||||
status=0
|
||||
|
||||
# Under set -e a bare assignment would end the script when gofmt
|
||||
# fails (a Go file it cannot parse), and prettier would never run.
|
||||
if ! files="$(gofmt -s -l .)"; then
|
||||
echo "gofmt: failed; see its errors above" >&2
|
||||
status=1
|
||||
fi
|
||||
if [ -n "$files" ]; then
|
||||
echo "gofmt: files not formatted:" >&2
|
||||
echo "$files" >&2
|
||||
exit 1
|
||||
status=1
|
||||
fi
|
||||
|
||||
# Same image as script/fmt; see there.
|
||||
image="$("$SCRIPT_DIR/projectname")-prettier"
|
||||
docker build -q --target prettier -t "$image" . >/dev/null
|
||||
if ! docker run --rm -v "$ROOT:/src:ro" "$image" \
|
||||
prettier --check '**/*.md' --tab-width 4 --prose-wrap always; then
|
||||
echo "prettier: Markdown not formatted; run make fmt" >&2
|
||||
status=1
|
||||
fi
|
||||
|
||||
exit "$status"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
|
||||
# yarn lockfile v1
|
||||
|
||||
|
||||
prettier@3.8.1:
|
||||
version "3.8.1"
|
||||
resolved "https://registry.yarnpkg.com/prettier/-/prettier-3.8.1.tgz#edf48977cf991558f4fcbd8a3ba6015ba2a3a173"
|
||||
integrity sha512-UOnG6LftzbdaHZcKoPFtOcCKztrQ57WkHDeRD9t/PTQtmT0NHSeWWepj6pS0z/N7+08BHFDQVUrfmfMRcZwbMg==
|
||||
Reference in New Issue
Block a user