sneak/attrsum
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

next → main: report the version from git #9

Open
clawbot wants to merge 10 commits from next into main
pull from: next
merge into: :main
Conversation 0 Commits 10 Files Changed 24
+1627 -220
24 changed files with 1627 additions and 220 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.dockerignore
+75
View File
@@ -0,0 +1,75 @@
# .dockerignore does NOT use .gitignore semantics. Docker matches with
# moby/patternmatcher: filepath.Match plus `**`, so `*` does not cross
# `/` and an unprefixed pattern is anchored at the context root. Every
# depth-independent pattern therefore needs `**/`, or `config/.env` and
# `certs/server.key` still ship while this file reads as solved. Only
# genuinely root-anchored entries go unprefixed. Never transplant these
# into .gitignore, where `**/` is wrong.
#
# Matching is case-sensitive, so secrets use character ranges rather
# than an ALL-CAPS twin, which would still miss `Server.Key`.
#
# Extend with this repo's own host-built artifacts, written anchored:
# `/myapp`, never `**/myapp`, which also matches `cmd/myapp/` and
# deletes the package directory from the context.
# .git is sent without its config. Without a VERSION build argument the
# stage that compiles runs `git describe --tags --always` on .git, which
# does not need .git/config; that file can hold a credential, such as a
# password in a remote URL or the token the CI checkout step stores there.
# Each submodule keeps a config with the same exposure in its git directory
# under .git/modules/, nested again for a submodule's own submodules, or in
# its own .git directory when it keeps one.
# KNOWN GAP: a submodule whose name has a `config` segment (`config`,
# `deploy/config`, `config/lib`) loses its whole git directory, because
# `**/.git/modules/**/config` also matches that segment's directory
# under .git/modules/. Go's version stamping then fails the build;
# nothing leaks. Name such a submodule without that segment:
# `git submodule add --name`.
**/.git/config
**/.git/modules/**/config
# Agent scratch: one full checkout of the repo per in-flight agent.
# Anchored because it occurs once where agents run at the repo root.
# KNOWN GAP: a repo running agents in subdirectories still ships
# `services/api/.claude/` and must add its own anchored entry.
.claude
# Environment files. `*.env` covers bare `.env` and the `prod.env`
# convention. Re-include a committed template with a negation if the
# build needs one: `!docs/example.env`.
**/*.[eE][nN][vV]
**/.[eE][nN][vV].*
**/.[eE][nN][vV][rR][cC]
# Private keys and the bundles carrying them. Public certificates
# (*.crt, *.cer) are deliberately absent: they are legitimate inputs.
**/*.[pP][eE][mM]
**/*.[kK][eE][yY]
**/*.[pP]12
**/*.[pP][fF][xX]
**/[iI][dD]_[rR][sS][aA]
**/[iI][dD]_[dD][sS][aA]
**/[iI][dD]_[eE][cC][dD][sS][aA]
**/[iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK]
**/[iI][dD]_[eE][dD]25519
**/[iI][dD]_[eE][dD]25519_[sS][kK]
# Dependencies: restored inside the image, never copied in.
**/node_modules
# OS metadata.
**/.DS_Store
**/Thumbs.db
# Editor state: never a build input, and it churns COPY.
**/*.swp
**/*.swo
**/*~
**/*.bak
**/.idea
**/.vscode
**/*.sublime-*
# This repo's host-built binary (`make build`); the image builds its own.
/attrsum
.editorconfig
+15
View File
@@ -0,0 +1,15 @@
root = true
[*]
indent_style = space
indent_size = 4
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true
[Makefile]
indent_style = tab
[*.go]
indent_style = tab
.gitea/workflows/check.yml
+5 -1
View File
@@ -4,6 +4,10 @@ jobs:
check:
runs-on: ubuntu-latest
steps:
# actions/checkout v4.2.2, 2026-02-28
# actions/checkout v4.2.2, 2026-02-22
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
# Full history and tags, so the build stamps the same
# `git describe` version as a full clone.
with:
fetch-depth: 0
- run: script/cibuild
.gitignore
+55 -1
View File
@@ -1 +1,55 @@
attrsum
# OS
.DS_Store
Thumbs.db
# Editors
*.swp
*.swo
*~
*.bak
.idea/
.vscode/
*.sublime-*
# Agent scratch (worktrees of this repo, created and destroyed by
# in-flight tooling). Unanchored: .gitignore patterns already match at
# every depth, so no prefix is wanted here. This is not a .dockerignore
# entry and must not be given a `**/` prefix on the way into one.
.claude/
# Node
node_modules/
# Secrets. Unanchored like every entry above, so each matches at every
# depth. Matching is case-sensitive on Linux, so names use character
# ranges rather than a lowercase form that misses `Server.Key`.
# Environment files. `*.env` covers bare `.env` and the `prod.env`
# convention. Only the templates `example.env` and `sample.env` are
# re-included below. A repository that commits any other template adds
# its own negation after these lines, for example `!.env.example`.
*.[eE][nN][vV]
.[eE][nN][vV].*
.[eE][nN][vV][rR][cC]
!example.env
!sample.env
# Private keys and the bundles carrying them.
*.[pP][eE][mM]
*.[kK][eE][yY]
*.[pP]12
*.[pP][fF][xX]
[iI][dD]_[rR][sS][aA]
[iI][dD]_[dD][sS][aA]
[iI][dD]_[eE][cC][dD][sS][aA]
[iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK]
[iI][dD]_[eE][dD]25519
[iI][dD]_[eE][dD]25519_[sS][kK]
# This repo's own entries, kept after the canonical content above: Go
# logs, coverage output and test binaries, and the binary `make build`
# writes.
*.log
*.out
*.test
/attrsum
.golangci.yml
+72 -5
View File
@@ -1,21 +1,31 @@
version: "2"
# Config schema uses the golangci-lint v2 layout (settings live under
# linters.settings, not top-level linters-settings) so that the
# thresholds below are actually applied by golangci-lint >= v2.
run:
timeout: 5m
modules-download-mode: readonly
linters:
default: all
enable:
# Successor to the deprecated gomodguard. Named explicitly, rather than
# left to `default: all`, because it carries the module policy below.
- gomodguard_v2
disable:
# Genuinely incompatible with project patterns
- exhaustruct # Requires all struct fields
- depguard # Dependency allow/block lists
- exhaustruct_v5 # Requires all struct fields (successor to exhaustruct)
- godot # Requires comments to end with periods
- wsl # Deprecated, replaced by wsl_v5
- wrapcheck # Too verbose for internal packages
- varnamelen # Short names like db, id are idiomatic Go
linters-settings:
# Deprecated: the warning is attached to the old name, so it is
# silenced by disabling that name, not by enabling the successor.
- wsl # Deprecated, replaced by wsl_v5
- gomodguard # Deprecated, replaced by gomodguard_v2
settings:
lll:
line-length: 88
funlen:
@@ -25,8 +35,65 @@ linters-settings:
max-complexity: 15
dupl:
threshold: 100
depguard:
# Test-support code must not be compiled into the shipped binary. A
# test-support package exists to hand a test privileges the program
# itself must never have, so a file that is not a test must not import
# one. Test files, and the files inside a package whose directory name
# ends in `test`, are where that code belongs, and are exempt.
#
# The deny list below is the one part of this file a repository is
# expected to extend, and the only part it may. depguard matches an
# import path against a list of prefixes, so it cannot be told "any path
# whose last segment ends in test"; a repository's own test-support
# packages have to be named here one at a time, by full import path,
# under a module path that differs from repository to repository. Add
# them; change nothing else.
rules:
test-support:
list-mode: lax
files:
- "$all"
- "!$test"
- "!**/*test/**"
deny:
- pkg: net/http/httptest
desc: >-
Test-support code belongs in test files and in packages whose
directory name ends in test, not in the shipped binary.
# Only decisions already recorded in the Go package defaults are
# listed here. Every entry matches the module path exactly.
gomodguard_v2:
blocked:
- module: github.com/rs/zerolog
recommendations:
- log/slog
reason: "Structured logging is stdlib log/slog."
# One entry per pre-fork module path, because the later releases
# are separate paths. A prefix match would be shorter but would
# also reach github.com/go-redis/redismock, the test double for
# the successor these entries recommend.
- module: github.com/go-redis/redis
recommendations:
- github.com/redis/go-redis/v9
reason: "Pre-fork module; use the maintained go-redis v9."
- module: github.com/go-redis/redis/v7
recommendations:
- github.com/redis/go-redis/v9
reason: "Pre-fork module; use the maintained go-redis v9."
- module: github.com/go-redis/redis/v8
recommendations:
- github.com/redis/go-redis/v9
reason: "Pre-fork module; use the maintained go-redis v9."
- module: github.com/sergi/go-diff
recommendations:
- github.com/aymanbagabas/go-udiff
reason: "No unified diff output; use go-udiff."
- module: github.com/hexops/gotextdiff
recommendations:
- github.com/aymanbagabas/go-udiff
reason: "Unmaintained fork; use go-udiff."
issues:
exclude-use-default: false
max-issues-per-linter: 0
max-same-issues: 0
.prettierignore
+2
View File
@@ -0,0 +1,2 @@
node_modules/
yarn.lock
.prettierrc
+4
View File
@@ -0,0 +1,4 @@
{
"tabWidth": 4,
"proseWrap": "always"
}
Dockerfile
+48 -26
View File
@@ -1,35 +1,57 @@
# Build stage
# golang 1.25-alpine, 2026-02-28
FROM golang@sha256:f6751d823c26342f9506c03797d2527668d095b0a15f1862cddb4d927a7a4ced AS builder
RUN apk add --no-cache git make gcc musl-dev binutils-gold
# golangci-lint v2.10.1
RUN go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@5d1e709b7be35cb2025444e19de266b056b7b7ee
# goimports v0.42.0
RUN go install golang.org/x/tools/cmd/goimports@009367f5c17a8d4c45a961a3a509277190a9a6f0
# Lint phase
# golangci/golangci-lint:v2.14.0, 2026-10-06
FROM golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN golangci-lint run --config .golangci.yml ./...
# Test phase. -race needs cgo and so a C compiler, which the Debian Go
# image ships and the alpine one does not. The tests run as an
# unprivileged user: root can read a file with mode 0000, so the
# permission tests would fail.
# golang:1.25.7-trixie, 2026-10-06
FROM golang@sha256:2b174ffcf56c7ad0c47d30d2630693265639ddf2a5141149c2da34db921791b4 AS test
RUN useradd --create-home testuser
USER testuser
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN go test -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \
go test -timeout 90s -race -v ./...; exit 1; }
# Build stage. Nothing is wanted from either phase above; the copies
# are what make BuildKit build them first, so this stage cannot run
# unless lint and test passed.
# golang 1.25-alpine, 2026-02-28
FROM golang@sha256:f6751d823c26342f9506c03797d2527668d095b0a15f1862cddb4d927a7a4ced AS builder
COPY --from=lint /src/go.sum /dev/null
COPY --from=test /src/go.sum /dev/null
RUN apk add --no-cache git make
# A tar-stream context keeps the sender's file owners, which git refuses.
RUN git config --system --add safe.directory /src
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
# Run the checks as an unprivileged user. Root bypasses file mode bits, which
# would make the permission tests (expecting EACCES on a 0000 file) spuriously
# pass with no error. Caches live under /tmp (world-writable) so the user needs
# no home directory of its own.
ENV GOCACHE=/tmp/gocache
ENV XDG_CACHE_HOME=/tmp/xdgcache
RUN adduser -D -u 1000 builder && chown -R builder:builder /src /go
USER builder
# Run all checks - build fails if any check fails
RUN make check
# Build the binary (still as the unprivileged user: it owns /src, so git VCS
# stamping sees consistent ownership).
RUN make build
# The version stamped into the binary: the VERSION build argument when one
# is given, otherwise `git describe --tags --always` of the .git the build
# context carries: the tag on a tagged commit, tag-N-gHASH on a commit after
# one, the short commit when no tag is reachable. A context that carries .git
# and still yields no version fails the build. With neither, as from a source
# tarball, the binary reports dev.
ARG VERSION
RUN version="${VERSION:-$(git describe --tags --always)}"; \
if [ -e .git ] && { [ -z "$version" ] || [ "$version" = dev ] || \
[ "$version" = unknown ]; }; then \
echo "version is '$version' although .git is present" >&2; \
exit 1; \
fi; \
make build VERSION="${version:-dev}"
# Runtime stage
# alpine 3.21, 2026-02-28
Makefile
+23 -11
View File
@@ -1,6 +1,9 @@
.PHONY: default bootstrap setup test lint fmt fmt-check check docker hooks build clean try
TESTDIR := $(HOME)/Documents/_SYSADMIN/cyberdyne
# The version `make build` stamps into the binary: the git tag or short
# commit, -dirty with uncommitted changes. `make build VERSION=x` stamps x,
# which is how the Dockerfile passes its version in.
VERSION ?= $(shell git describe --tags --always --dirty 2>/dev/null || echo dev)
# Standard targets are thin shims; the implementations live in script/
# per the scripts-to-rule-them-all pattern.
@@ -35,18 +38,27 @@ hooks:
@script/install-precommit
build: clean
@go build .
@go build -ldflags "-X main.Version=$(VERSION)" .
clean:
@rm -f attrsum
# Runs the binary on three small files in a new temporary directory, which
# the trap removes when the recipe ends, also when a step fails. The check
# after clear is expected to fail, so its exit status is ignored.
try: build
./attrsum sum add -v $(TESTDIR)
./attrsum check -v $(TESTDIR)
./attrsum clear -v $(TESTDIR)
-./attrsum check -v $(TESTDIR)
./attrsum sum add -v $(TESTDIR)
./attrsum check -v $(TESTDIR)
touch $(TESTDIR)/*
./attrsum sum update -v $(TESTDIR)
./attrsum check -v $(TESTDIR)
@set -ex; \
dir=$$(mktemp -d); \
trap 'rm -rf "$$dir"' EXIT; \
echo one >"$$dir/a"; \
echo two >"$$dir/b"; \
echo three >"$$dir/c"; \
./attrsum sum add -v "$$dir"; \
./attrsum check -v "$$dir"; \
./attrsum clear -v "$$dir"; \
./attrsum check -v "$$dir" || true; \
./attrsum sum add -v "$$dir"; \
./attrsum check -v "$$dir"; \
touch "$$dir"/*; \
./attrsum sum update -v "$$dir"; \
./attrsum check -v "$$dir"
README.md
+31 -23
View File
@@ -1,7 +1,7 @@
[**attrsum**](https://git.eeqj.de/sneak/attrsum/) is a **Go
1.22** command-line utility that **adds, updates, verifies, and clears per-file
file content checksums stored in extended attributes (xattrs) on macOS (APFS) and
Linux**, released under the [WTFPL v2](http://www.wtfpl.net/).
[**attrsum**](https://git.eeqj.de/sneak/attrsum/) is a **Go 1.22** command-line
utility that **adds, updates, verifies, and clears per-file file content
checksums stored in extended attributes (xattrs) on macOS (APFS) and Linux**,
released under the [WTFPL v2](http://www.wtfpl.net/).
Original release 2025-05-08.
@@ -54,29 +54,32 @@ attrsum -q sum add DIR
```
| xattr key | meaning |
|---------------------------------------------|--------------------------------|
| ---------------------------------------- | ------------------------------ |
| `user.berlin.sneak.app.attrsum.checksum` | base-58 multihash (sha2-256) |
| `user.berlin.sneak.app.attrsum.sumtime` | RFC 3339 timestamp of checksum |
Flags:
* `-v, --verbose` — per-file log output
* `-q, --quiet` — suppress all output except errors (no progress bar or summary)
* `--exclude PATTERN` — skip paths matching rsync/Doublestar glob
* `--exclude-dotfiles` — skip any path component that starts with `.`
- `-v, --verbose` — per-file log output
- `-q, --quiet` — suppress all output except errors (no progress bar or summary)
- `--exclude PATTERN` — skip paths matching rsync/Doublestar glob
- `--exclude-dotfiles` — skip any path component that starts with `.`
All commands display a progress bar with ETA and print a summary report to stderr on completion (unless `--quiet` is specified).
All commands display a progress bar with ETA and print a summary report to
stderr on completion (unless `--quiet` is specified).
`attrsum` **never follows symlinks** and skips non-regular files (sockets, devices, …).
`attrsum` **never follows symlinks** and skips non-regular files (sockets,
devices, …).
---
## Why?
Apple APFS and Linux ext3/ext4 **store no per-file content checksums**, so
silent data corruption can pass unnoticed. `attrsum` keeps a portable checksum **inside each file’s xattrs**, providing integrity
verification that travels with the file itself—no external database
required. Now you can trust a USB stick didn't eat your data.
silent data corruption can pass unnoticed. `attrsum` keeps a portable checksum
**inside each file’s xattrs**, providing integrity verification that travels
with the file itself—no external database required. Now you can trust a USB
stick didn't eat your data.
---
@@ -84,19 +87,25 @@ required. Now you can trust a USB stick didn't eat your data.
Future improvements under consideration:
- **Dry-run mode (`--dry-run`, `-n`)** — show what would be done without making changes
- **JSON output (`--json`)** — machine-readable output for scripting and integration
- **Parallel processing (`-j N`)** — use multiple goroutines for faster checksumming on large trees
- **Dry-run mode (`--dry-run`, `-n`)** — show what would be done without making
changes
- **JSON output (`--json`)** — machine-readable output for scripting and
integration
- **Parallel processing (`-j N`)** — use multiple goroutines for faster
checksumming on large trees
- **Exit code documentation** — formalize and document exit codes for scripting
---
## Contributing
* Author & maintainer: **sneak** – <sneak@sneak.berlin>
* Issues / PRs: <https://git.eeqj.de/sneak/attrsum/>
* Code must pass `go vet`, `go test ./...`, and `go fmt`.
* No CLA; contributions are under WTFPL v2.
- Author & maintainer: **sneak** – <sneak@sneak.berlin>
- Issues / PRs: <https://git.eeqj.de/sneak/attrsum/>
- Code must pass `make check`, which runs the tests and golangci-lint as phases
of the `Dockerfile` (Docker is required) and checks formatting with `gofmt -s`
and goimports for Go and prettier for Markdown. `make bootstrap` installs
goimports and prettier, and `make fmt` fixes what the check reports.
- No CLA; contributions are under WTFPL v2.
---
@@ -110,5 +119,4 @@ No formal Code of Conduct; be excellent to each other.
## License
*Everything is permitted.*
See [WTFPL v2](http://www.wtfpl.net/txt/copying/).
_Everything is permitted._ See [WTFPL v2](http://www.wtfpl.net/txt/copying/).
REPO_POLICIES.md
+679
View File
@@ -0,0 +1,679 @@
---
title: Repository Policies
last_modified: 2026-10-04
---
This document covers repository structure, tooling, and workflow standards. Code
style conventions are in separate documents:
- [Code Styleguide](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/CODE_STYLEGUIDE.md)
(general, bash, Docker)
- [Go](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/CODE_STYLEGUIDE_GO.md)
- [JavaScript](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/CODE_STYLEGUIDE_JS.md)
- [Python](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/CODE_STYLEGUIDE_PYTHON.md)
- [Go HTTP Server Conventions](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/GO_HTTP_SERVER_CONVENTIONS.md)
---
- Cross-project documentation (such as this file) must include
`last_modified: YYYY-MM-DD` in the YAML front matter so it can be kept in sync
with the authoritative source as policies evolve.
- **ALL external references must be pinned by cryptographic hash.** This
includes Docker base images, Go modules, npm packages, GitHub Actions, and
anything else fetched from a remote source. Version tags (`@v4`, `@latest`,
`:3.21`, etc.) are server-mutable and therefore remote code execution
vulnerabilities. The ONLY acceptable way to reference an external dependency
is by its content hash (Docker `@sha256:...`, Go module hash in `go.sum`, npm
integrity hash in lockfile, GitHub Actions `@<commit-sha>`). No exceptions.
This also means never `curl | bash` to install tools like pyenv, nvm, rustup,
etc. Instead, download a specific release archive from GitHub, verify its hash
(hardcoded in the Dockerfile or script), and only then install. Unverified
install scripts are arbitrary remote code execution. This is the single most
important rule in this document. Double-check every external reference in
every file before committing. There are zero exceptions to this rule.
- Every repo with software must have a root `Makefile` with these targets:
`make bootstrap`, `make setup`, `make test`, `make lint`, `make fmt` (writes),
`make fmt-check` (read-only), `make check` (runs `test`, `lint`, `fmt-check`),
`make docker`, and `make hooks` (installs pre-commit hook). A model Makefile
is at `https://git.eeqj.de/sneak/prompts/raw/branch/main/Makefile`.
- Repos follow the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
pattern: the implementation of each Makefile target lives in an executable
script in `script/` (`script/bootstrap`, `script/setup`, `script/test`,
`script/lint`, `script/fmt`, `script/fmt-check`, `script/check`,
`script/docker`), and the Makefile targets are thin shims that call them. The
scripts must be POSIX sh (`#!/bin/sh`, `set -eu`, no bashisms) so they run in
minimal containers (e.g. alpine images have no bash); locate the repo root
with `$(cd "$(dirname "$0")/.." && pwd -P)` and `cd` there before acting. From
the standard's canonical set we use `bootstrap`, `setup` (make the repo ready
for development after a fresh clone: runs `bootstrap`, then
`install-precommit`, plus any repo-specific initialization), `test`, and
`cibuild`. `script/bootstrap` installs all dependencies idempotently and
assumes nothing is present: base tools come from nix, apt, brew, or apk
(detected in that order; apt runs noninteractive). For node it uses the
installed node if present; otherwise it installs a PINNED node version via
nvm, first installing nvm itself if missing — from a hash-verified GitHub
release archive (never `curl | sh`), with bash installed as an explicit
prerequisite since nvm requires bash. yarn is then pinned via
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts";
always exact versions. `script/cibuild` runs the CI build: it changes to the
repo root, runs `script/bootstrap`, runs `script/check`, and builds the image
with the version; the Gitea workflow calls it. **`script/cibuild` runs
`script/bootstrap` first**, because the workflow checks out the repo and runs
nothing else, while `script/fmt-check` runs the formatter on the host: on a
pristine checkout with nothing installed the run dies there, after the
containerised gates have passed. **The bootstrap alone is not enough**:
`script/bootstrap` installs node and yarn under nvm and leaves neither on the
`PATH` of the shell that called it, so a bare `yarn` still exits 127. The host
entrypoints that need yarn — `script/fmt` and `script/fmt-check` — therefore
source nvm for the pinned node version before invoking it, exactly as
`script/bootstrap`'s own install step does. A runner carrying nothing but
docker and git then gets through `script/check`. Four further scripts are our
own extensions to the standard: `script/check` runs `script/test`,
`script/lint` and `script/fmt-check`; `script/precommit` is what the git
pre-commit hook runs, and it calls `script/check`; `script/install-precommit`
installs the git pre-commit hook (the `make hooks` target shims to it); and
`script/projectname` (literally that filename) simply outputs the project's
name. Scripts that need the name call `script/projectname` — e.g.
`script/docker` assembles its image tag from it — so those scripts stay
byte-identical across all repos. Repo-type-specific pre-commit extras (e.g.
`go mod tidy` verification in Go repos) belong in `script/precommit`, not in
the hook itself. Model scripts are at
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
must document the provided scripts in an **Entrypoints** section (see the
README requirements below).
- Always use Makefile targets (`make fmt`, `make test`, `make lint`, etc.)
instead of invoking the underlying tools directly. The Makefile is the single
source of truth for how these operations are run.
- The Makefile is authoritative documentation for how the repo is used. Beyond
the required targets above, it should have targets for every common operation:
running a local development server (`make run`, `make dev`), re-initializing
or migrating the database (`make db-reset`, `make migrate`), building
artifacts (`make build`), generating code, seeding data, or anything else a
developer would do regularly. If someone checks out the repo and types
`make<tab>`, they should see every meaningful operation available. A new
contributor should be able to understand the entire development workflow by
reading the Makefile.
- Every repo should have a `Dockerfile`, and it carries the repo's gates: a
`lint` phase and a `test` phase, with the final stage depending on both so the
image cannot be built unless they pass. For non-server repos the final stage
brings up a development environment; for server repos it is the runtime image.
The gate phases and the build stage start from their pinned base images and
install what those images lack either inline, as the canonical Go `Dockerfile`
below does for `git`, or by running `script/bootstrap`, as the `prompts`
repo's own `Dockerfile` does for its yarn packages. The development
environment stage installs development prerequisites by running
`script/bootstrap` rather than duplicating its installs inline. A stage that
runs `script/bootstrap` COPYs `script/` and the dependency manifests
(`package.json` + `yarn.lock`, `go.mod` + `go.sum`, etc.) before running it.
- **Linting and testing run in Docker, as phases of the `Dockerfile`.** There is
no separate lint file. `script/lint` and `script/test` each build one phase
and nothing else:
```sh
docker build --no-cache --target lint -t "$(script/projectname)-lint" .
docker build --no-cache --target test -t "$(script/projectname)-test" .
```
**A stage that is not the last one in the file is built only when the final
stage's chain depends on it, or when `--target` names it.** That is why the
two gates are always invoked by name here, and why the final stage carries a
`COPY --from=` of a harmless file from each of them: without that edge a
plain `docker build .` builds the last stage alone and exits 0 having linted
and tested nothing.
**Every `docker build` in `script/` is tagged**, here and in
`script/cibuild` and `script/docker`. An untagged build leaves a dangling
image behind on every invocation, on every developer host and every CI
runner; a tagged one replaces the previous image.
Inside a phase the tool is invoked directly — `golangci-lint`, `go test`,
`eslint`, `prettier` — never through `make lint` or `script/test`, which are
themselves a `docker build` and would recurse into a daemon that does not
exist in a build step. Formatting is the exception and stays on the host:
`script/fmt` writes the working tree, and `script/fmt-check` is its
read-only twin.
**No lint verdict may come from a host invocation of the linter.** On a
shared host golangci-lint reads a result cache keyed on file content rather
than location, so a second checkout of the same content is served the first
one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs
exit non-zero with `parallel golangci-lint is running` — a status a caller
cannot tell from real findings. Both have produced wrong verdicts in this
org, in both directions. A container has its own cache, its own `TMPDIR` and
a digest-pinned binary, so neither is reachable.
- **Any build that runs checks is built with `--no-cache`.** Docker invalidates
a `COPY` layer only when the copied content changes, so on an unchanged tree
the check `RUN` is served from cache, nothing executes, and the build still
exits 0. Every `docker build` in `script/` therefore passes `--no-cache`:
`script/lint`, `script/test`, `script/cibuild` and `script/docker` are the
four, and there is no fifth — `script/check` runs the two gate phases and
`script/fmt-check`, and builds no image of its own. A bare `docker build .` is
not evidence that anything ran: a sub-second build reporting success is a
cache hit, not a result. Never invalidate by pruning — `docker builder prune`
and friends destroy a build cache shared with every other build on the host.
When a check is added or changed, prove it works by planting a defect it must
catch and watching the run fail on it, then revert the defect. A green run
alone shows neither that the check ran nor that it covers what it should.
- **The gate phases are separate stages, and the build stage depends on both.**
The lint phase is based on the `golangci/golangci-lint` image (pinned by
hash), so lint failures surface in seconds rather than after a full compile,
and the test phase is based on the Debian Go image. The canonical Go repo
`Dockerfile`:
```dockerfile
# Lint phase
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
FROM golangci/golangci-lint@sha256:... AS lint
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN golangci-lint run --config .golangci.yml ./...
# Test phase. -race needs cgo and so a C compiler, which the Debian Go
# image ships and the alpine one does not.
# golang:1.x, YYYY-MM-DD
FROM golang@sha256:... AS test
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN go test -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \
go test -timeout 90s -race -v ./...; exit 1; }
# Build stage. Nothing is wanted from either phase above; the copies
# are what make BuildKit build them first, so this stage cannot run
# unless lint and test passed.
# golang:1.x-alpine, YYYY-MM-DD
FROM golang@sha256:... AS builder
COPY --from=lint /src/go.sum /dev/null
COPY --from=test /src/go.sum /dev/null
RUN apk add --no-cache git
# A tar-stream context keeps the sender's file owners, which git refuses.
RUN git config --system --add safe.directory /src
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
# The VERSION build arg when one is given, otherwise
# `git describe --tags --always` on the .git in the build context. With
# .git present, a version that is still empty, dev or unknown fails the
# build: git is missing or could not read the checkout.
ARG VERSION
RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \
if [ -e .git ]; then \
case "$VERSION" in ""|dev|unknown) \
echo "version is '$VERSION' although .git is present" >&2; \
exit 1 ;; \
esac; \
fi; \
CGO_ENABLED=0 go build -trimpath \
-ldflags="-s -w -X main.Version=${VERSION}" \
-o /app ./cmd/app/
# Runtime stage, and the last one
FROM alpine@sha256:...
COPY --from=builder /app /usr/local/bin/app
ENTRYPOINT ["app"]
```
Key points:
- The lint phase uses the `golangci/golangci-lint` image directly (it has
both Go and the linter), so nothing needs installing.
- `COPY --from=<phase> /src/go.sum /dev/null` is a no-op copy whose only
purpose is the ordering edge. BuildKit runs stages in parallel by default,
and a stage nothing depends on is not built at all, so without these two
lines a red gate would not fail the build.
- Keep the runtime stage last, and if you add a stage after it, give it the
same two copies. A plain `docker build .` builds the last stage's chain
and nothing else.
- If the project uses `//go:embed` directives that reference build artifacts
(e.g. a web frontend compiled in a separate stage), the lint phase must
create placeholder files so the embed directives resolve. Example:
`RUN mkdir -p web/dist && touch web/dist/index.html web/dist/style.css`.
- If the project requires CGO or system libraries for linting, install them
in the lint phase. The `golangci/golangci-lint` image is Debian-based and
has no `apk`, so install with `apt-get` under the Debian package name
(`libvips-dev`, where alpine says `vips-dev`), and delete the package
lists in the same `RUN`, so the layer does not keep them:
```dockerfile
RUN apt-get update \
&& apt-get install -y --no-install-recommends libvips-dev \
&& rm -rf /var/lib/apt/lists/*
```
- `.dockerignore` lets `.git` into the build context. It keeps out every git
`config` at any depth (`**/.git/config`, `**/.git/modules/**/config`): the
repository's own, each submodule's under `.git/modules/`, and that of a
submodule keeping its own `.git` directory. `git describe` does not need
them, and each can hold a credential: a password in a remote URL, or the
token the CI checkout step stores there. A submodule whose name has a
`config` segment (`config`, `deploy/config`, `config/lib`) loses its whole
git directory to `**/.git/modules/**/config`, and Go's version stamping
then fails the build: give it a name without that segment
(`git submodule add --name`). The stage that compiles has `git` (the
Debian Go image has it; an alpine one needs `apk add --no-cache git`) and
takes the version from the `VERSION` build argument when one is given,
otherwise from `git describe --tags --always`. That gives the tag on a
tagged commit; on a later commit, the tag, the number of commits since it
and the short commit (`v1.2.3-4-gabc1234`); and the short commit when no
tag is reachable. The stage that compiles also marks its working directory
safe for git (`git config --system --add safe.directory /src`): a context
sent as a tar stream keeps the sender's file owners, and git refuses a
checkout owned by another user, so the version would come out empty.
`ARG VERSION` has no default, and the build fails if the context carries
`.git` and the version still comes out empty, `dev` or `unknown`. A plain
`docker build .` with no build arguments must succeed; a Dockerfile that
refuses an empty build argument drops that refusal and keeps the argument.
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
runs `script/cibuild` on push, and checks out the repo as its only other step.
That script bootstraps, runs the gate phases, and then builds the image, so a
successful run means every check passed; a bare `docker build .` does not
carry the same guarantee, because its gate phases may come from the cache. The
image build is uncached and so runs the gate phases a second time. That is the
price of the rule above, and it is worth paying: the image that ships is built
from a run of its own gates rather than from a cache entry. A separate
workflow limited to `main` by a `branches` list under `on: push` cannot be
checked by review: to try a change to it, add the feature branch to that list
and push, then remove the branch from the list again before merging. Keep any
job in it that publishes behind `if: github.ref_name == 'main'`, so the run
from the feature branch publishes nothing.
- Use platform-standard formatters: `black` for Python, `prettier` for
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
two exceptions: four-space indents (except Go), and `proseWrap: always` for
Markdown (hard-wrap at 80 columns). Documentation and writing repos (Markdown,
HTML, CSS) should also have `.prettierrc` and `.prettierignore`.
- Pre-commit hook: runs `script/precommit`, which calls `script/check`. If local
testing is not possible in the repo, `script/precommit` may skip `script/test`
and run only `script/lint` and `script/fmt-check`. The hook is installed by
`script/install-precommit`; the Makefile must provide a `make hooks` target
that shims to it.
- All repos with software must have tests that run via the platform-standard
test framework (`go test`, `pytest`, `jest`/`vitest`, etc.). If no meaningful
tests exist yet, add the most minimal test possible — e.g. importing the
module under test to verify it compiles/parses. There is no excuse for
`make test` to be a no-op.
- `make test` must complete in under 60 seconds. That is the hard cap, and a
suite that exceeds it fails. Under 20 seconds is the target. A suite between
20 and 60 seconds is still green, but the overage must be filed as an
improvement bug against that repo. Add a 90-second timeout to the test
invocation (`go test -timeout 90s`). The backstop deliberately sits above the
hard cap so that it catches a genuinely hung test rather than a merely slow
one.
- **The test command should use the conditional verbose rerun pattern.** Run
tests without `-v` (verbose) first. If tests fail, automatically rerun with
`-v` to show full output. This keeps CI logs and `docker build` output clean
on success (just package/suite summaries) while providing full diagnostic
detail on failure (every test case, every assertion). The command lives in the
`test` phase of the `Dockerfile`, since `script/test` builds that phase; the
Makefile form below is the same pattern for any repo-local invocation:
```makefile
test:
@<test-command> || \
{ echo "--- Rerunning with -v for details ---"; \
<test-command-with-v>; exit 1; }
```
Go example:
```makefile
test:
@go test -count=1 -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \
go test -count=1 -timeout 90s -race -v ./...; exit 1; }
```
`-count=1` is required on both invocations: it defeats Go's test _result_
cache, so neither run can report a stored pass in place of running the
tests. It leaves the build cache alone, so it costs the runtime of the suite
and no recompilation.
That cache is Go's own, separate from Docker's layer cache. Go stores a
passing result in its cache directory (`GOCACHE`), and when the same tests
run again on unchanged code it prints that result, marked `(cached)`,
without running them. That matters on a developer's machine, where this
target runs and the directory lasts from one run to the next. The `test`
phase of the `Dockerfile` needs no `-count=1`: its base image holds no
result for this repo's tests and nothing before its `go test` step runs a
test, so there is nothing to replay. `--no-cache` (above) is what makes that
step run on an unchanged tree.
Python example:
```makefile
test:
@python -m pytest || \
{ echo "--- Rerunning with -v for details ---"; \
python -m pytest -v; exit 1; }
```
The `exit 1` ensures the target always fails after a rerun — the first run
already proved the tests are broken, so the build must not pass even if a
flaky test happens to succeed on the second attempt. The rerun exists solely
for diagnostic output.
- Docker builds must complete in under 5 minutes.
- `make check` must not modify any files in the repo. Tests may use temporary
directories.
- `main` must always pass `make check`, no exceptions.
- Never commit secrets. `.env` files, credentials, API keys, and private keys
must be in `.gitignore`. No exceptions.
- `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`),
editor files (`.swp`, `*~`), in-repo agent scratch directories (`.claude/`),
language build artifacts, and `node_modules/`. Fetch the standard `.gitignore`
from `https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when
setting up a new repo. These patterns are written to `.gitignore`'s own
semantics, in which an unanchored pattern already matches at every depth; they
are not a `.dockerignore` and must not be transplanted into one unmodified.
- **`.dockerignore` does not use `.gitignore` semantics, and copying patterns
across unmodified leaves secrets in the build context.** Docker matches with
`moby/patternmatcher`: `filepath.Match` semantics plus a `**` extension, so
`*` does not cross `/` and a pattern without a leading `**/` is anchored at
the build-context root. A `.dockerignore` listing `.env`, `*.pem` and `*.key`
therefore excludes only the copies at the repository root, while `config/.env`
and `certs/server.key` still reach the context and can land in an image layer
— which is more dangerous than a short file with no secret patterns at all,
because it reads as solved and stops anyone looking. Give every
depth-independent pattern the `**/` prefix and leave only genuinely
root-anchored entries unprefixed: `.claude`, and the repo's own host-built
binary, written `/myapp` and never `**/myapp`, which would also match
`cmd/myapp/` and delete the package directory from the context. Matching is
case-sensitive, and an ALL-CAPS twin per pattern still misses `Server.Key`, so
secret names use character ranges — `**/*.[kK][eE][yY]`, `**/*.[pP][eE][mM]`,
and likewise for `.envrc` and the extensionless SSH keys. Where such a pattern
also catches something the build needs, re-include it with a negation
(`!docs/example.env`); deleting the pattern reopens the exposure for every
other file it covers. Fetch the standard `.dockerignore` from
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.dockerignore` and extend
it with the repo's own artifacts.
- **In-repo agent scratch belongs in both files, written to each file's own
semantics.** `.claude/` holds one worktree per in-flight agent — an entire
additional checkout of the repo — so under `COPY . .` the build context
inflates by a multiple of the repo and another session's unreviewed work can
be copied into an image layer. In `.gitignore` the entry is `.claude/`,
unanchored. In `.dockerignore` it is `.claude`, anchored and with **no** `**/`
prefix, because the prefixed form would also delete any nested directory of
that name from the build. Anchoring carries a known gap that the canonical
`.dockerignore` states in its own comment, since consuming repos receive the
file and not the tracker: the directory is created in the agent's working
directory, so a repo running agents in subdirectories still ships
`services/api/.claude/` and must add its own anchored entry there.
- **A plain `docker build .` of a clone stamps the version that
`git describe --tags --always` gives**, derived from the `.git` in the build
context as the canonical `Dockerfile` above shows. Without its failure check,
a missing `git` or an unreadable checkout would leave `-X main.Version=` empty
and the build would still exit 0. `script/docker` and `script/cibuild` pass
the version they compute on the host; it takes precedence. They do this
byte-identically across repos:
```sh
# Own line: a failing command substitution inside an argument does not
# trip `set -e`, so the inline form degrades to an empty constant.
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown"
docker build --no-cache \
--build-arg VERSION="$version" \
-t "$(script/projectname)" .
```
`--always` makes an untagged repo yield an abbreviated commit hash rather
than failing, and the `[ -n "$version" ]` line is the single place the
fallback is applied — a live check that fires on a build from an export with
no `.git` and on a repository with no commits yet. Do not fold it into the
substitution as `|| echo unknown`, which makes the guard unreachable. The
Dockerfile's side is `ARG VERSION` in the stage that compiles, declared
there because `ARG` is stage-scoped; passing `VERSION` to a repo whose
Dockerfile declares no such `ARG` is ignored and costs nothing, which is why
the scripts stay byte-identical. One consequence for CI: the standard
checkout action clones shallow and fetches no tags, so a repo that embeds a
tag-derived version must set `fetch-depth: 0` on its checkout step.
- **Verify `.dockerignore` by enumerating the image, not by reading the
patterns.** Plant files at the root _and_ at least two directories deep, build
a probe image that does `COPY . .`, and list what actually landed
(`docker run --rm --entrypoint find IMAGE /app`). The `transferring context`
size is not a substitute: a nested secret is a few bytes, and BuildKit
transfers only the delta from the previous build.
- **No build artifacts in version control.** Code-derived data (compiled
bundles, minified output, generated assets) must never be committed to the
repository if it can be avoided. The build process (e.g. Dockerfile, Makefile)
should generate these at build time. Notable exception: Go protobuf generated
files (`.pb.go`) ARE committed because repos need to work with `go get`, which
downloads code but does not execute code generation.
- Never use `git add -A` or `git add .`. Always stage files explicitly by name.
- Never force-push to `main`.
- Make all changes on a feature branch. You can do whatever you want on a
feature branch.
- `.golangci.yml` is standardized. The vendored copy in a consuming repo must
_NEVER_ be modified by an agent: fetch it from
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.golangci.yml` and keep it
byte-identical, so that no repo can quietly loosen its own linting. Linter
configuration changes are made to the canonical copy in the `prompts` repo and
reach consuming repos by re-vendoring; an agent may open a PR against
canonical, which only the user merges. One list is exempt from byte-identity,
because it cannot be written once for every repo: the `deny` list of the
`test-support` depguard rule, where a repo names its own test-support packages
by full import path. A repo adds entries there and changes nothing else, and a
re-vendor carries its entries forward. The canonical golangci-lint version is
v2.14.0 (released 2026-09-24), pinned as the digest of the lint phase's base
image
(`golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f`,
which reports `2.14.0 built with go1.27.0 from 114493f9`). A module's `go`
directive must not name a newer Go minor version than the one golangci-lint
was built with, or golangci-lint refuses to lint it: this release lints
`go 1.27.1` but not `go 1.28`. That digest is the only pin, since no repo
installs golangci-lint on the host. A repo sets the lint phase digest to the
one named here and re-vendors `.golangci.yml` in the same commit, whichever of
the two prompted the change: the canonical copy can name linters that an older
golangci-lint rejects, and a newer golangci-lint can add linters that
`default: all` switches on until the canonical copy disables them.
- **`script/bootstrap` installs a pinned tool by comparing versions, never by
testing presence.** An `if ! command -v <tool>; then install; fi` guard tests
`PATH` only, so on an already-provisioned machine the pin is inert and a
version bump is a silent no-op — while the Dockerfile, installing into a clean
image, gets the pinned version, so a local `make check` and `make docker` can
disagree about what the tool even is. The canonical form:
- compares the installed version against the pin over the **whole** version
token; a parser that stops at the first `-` reports `2.12.2` for a host
running `2.12.2-rc1` and skips the install;
- treats absent, non-zero, empty or unrecognised `--version` output as a
mismatch, so the failure direction is a redundant install and never a
skipped one;
- after installing, re-resolves the binary the way callers do — `hash -r`,
then through `PATH`, not through the directory the installer wrote to —
and fails naming the resolved path, since an install that a shadowing
binary hides succeeds while changing nothing any caller sees;
- is actually called, and prints the version on both success paths: a
function defined and never invoked has the same exit status and the same
empty output as one that worked.
Keep it POSIX sh: no arrays, no `[[`, no `grep -P`.
A Go tool a repo needs on the host is installed with `go install` pinned to
a commit hash (`go install <package>@<commit hash>`). It is never tracked as
a `go.mod` tool dependency or through a `tools.go` file, either of which
pulls the tool's own dependencies into the repo's `go.mod` and `go.sum`.
- When pinning images or packages by hash, add a comment above the reference
with the version and date (YYYY-MM-DD).
- Use `yarn`, not `npm`.
- Write all dates as YYYY-MM-DD (ISO 8601).
- Simple projects should be configured with environment variables.
- Dockerized web services listen on port 8080 by default, overridable with
`PORT`.
- **HTTP/web services must be hardened for production internet exposure before
tagging 1.0.** This means full compliance with security best practices
including, without limitation, all of the following:
- **Security headers** on every response:
- `Strict-Transport-Security` (HSTS) with `max-age` of at least one year
and `includeSubDomains`.
- `Content-Security-Policy` (CSP) with a restrictive default policy
(`default-src 'self'` as a baseline, tightened per-resource as
needed). Never use `unsafe-inline` or `unsafe-eval` unless
unavoidable, and document the reason.
- `X-Frame-Options: DENY` (or `SAMEORIGIN` if framing is required).
Prefer the `frame-ancestors` CSP directive as the primary control.
- `X-Content-Type-Options: nosniff`.
- `Referrer-Policy: strict-origin-when-cross-origin` (or stricter).
- `Permissions-Policy` restricting access to browser features the
application does not use (camera, microphone, geolocation, etc.).
- **Request and response limits:**
- Maximum request body size enforced on all endpoints (e.g. Go
`http.MaxBytesReader`). Choose a sane default per-route; never accept
unbounded input.
- Maximum response body size where applicable (e.g. paginated APIs).
- `ReadTimeout` and `ReadHeaderTimeout` on the `http.Server` to defend
against slowloris attacks.
- `WriteTimeout` on the `http.Server`.
- `IdleTimeout` on the `http.Server`.
- Per-handler execution time limits via `context.WithTimeout` or
chi/stdlib `middleware.Timeout`.
- **Authentication and session security:**
- Rate limiting on password-based authentication endpoints. API keys are
high-entropy and not susceptible to brute force, so they are exempt.
- CSRF tokens on all state-mutating HTML forms. API endpoints
authenticated via `Authorization` header (Bearer token, API key) are
exempt because the browser does not attach these automatically.
- Passwords stored using bcrypt, scrypt, or argon2 — never plain-text,
MD5, or SHA.
- Session cookies set with `HttpOnly`, `Secure`, and `SameSite=Lax` (or
`Strict`) attributes.
- **Reverse proxy awareness:**
- True client IP detection when behind a reverse proxy
(`X-Forwarded-For`, `X-Real-IP`). The application must accept
forwarded headers only from a configured set of trusted proxy
addresses — never trust `X-Forwarded-For` unconditionally.
- **CORS:**
- Authenticated endpoints must restrict `Access-Control-Allow-Origin` to
an explicit allowlist of known origins. Wildcard (`*`) is acceptable
only for public, unauthenticated read-only APIs.
- **Error handling:**
- Internal errors must never leak stack traces, SQL queries, file paths,
or other implementation details to the client. Return generic error
messages in production; detailed errors only when `DEBUG` is enabled.
- **TLS:**
- Services never terminate TLS directly. They are always deployed behind
a TLS-terminating reverse proxy. The service itself listens on plain
HTTP. However, HSTS headers and `Secure` cookie flags must still be
set by the application so that the browser enforces HTTPS end-to-end.
This list is non-exhaustive. Apply defense-in-depth: if a standard security
hardening measure exists for HTTP services and is not listed here, it is
still expected. When in doubt, harden.
- `README.md` is the primary documentation. Required sections:
- **Description**: First line must include the project name, purpose,
category (web server, SPA, CLI tool, etc.), license, and author. Example:
"µPaaS is an MIT-licensed Go web application by @sneak that receives
git-frontend webhooks and deploys applications via Docker in realtime."
- **Getting Started**: Copy-pasteable install/usage code block.
- **Entrypoints**: Opens by stating that the repo adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard (with that link), then documents each provided `script/`
entrypoint and its purpose.
- **Rationale**: Why does this exist?
- **Design**: How is the program structured?
- **TODO**: Update meticulously, even between commits. When planning, put
the todo list in the README so a new agent can pick up where the last one
left off.
- **License**: MIT, GPL, or WTFPL. Ask the user for new projects. Include a
`LICENSE` file in the repo root and a License section in the README.
- **Author**: [@sneak](https://sneak.berlin).
- First commit of a new repo should contain only `README.md`.
- Go module root: `sneak.berlin/go/<name>`. Always run `go mod tidy` before
committing.
- Use SemVer.
- Database migrations live in `internal/db/migrations/` and must be embedded in
the binary.
- `000_migration.sql` — contains ONLY the creation of the migrations
tracking table itself. Nothing else.
- `001_schema.sql` — the full application schema.
- **Pre-1.0.0:** never add additional migration files (002, 003, etc.).
There is no installed base to migrate. Edit `001_schema.sql` directly.
- **Post-1.0.0:** add new numbered migration files for each schema change.
Never edit existing migrations after release.
- All repos should have an `.editorconfig` enforcing the project's indentation
settings.
- Avoid putting files in the repo root unless necessary. Root should contain
only project-level config files (`README.md`, `AGENTS.md`, `Makefile`,
`Dockerfile`, `LICENSE`, `.gitignore`, `.editorconfig`, `REPO_POLICIES.md`,
and language-specific config). Everything else goes in a subdirectory.
Canonical subdirectory names:
- `bin/` — executable scripts and tools
- `cmd/` — Go command entrypoints; thin only: one `main.go` per binary whose
body is a single call into `internal/` or `pkg/`, no project logic in
`cmd/`
- `configs/` — configuration templates and examples
- `deploy/` — deployment manifests (k8s, compose, terraform)
- `docs/` — documentation and markdown (README.md stays in root)
- `internal/` — Go internal packages
- `internal/db/migrations/` — database migrations
- `pkg/` — Go library packages
- `share/` — systemd units, data files
- `static/` — static assets (images, fonts, etc.)
- `web/` — web frontend source
- When setting up a new repo, files from the `prompts` repo may be used as
templates. Fetch them from
`https://git.eeqj.de/sneak/prompts/raw/branch/main/<path>`.
- New repos must contain at minimum:
- `README.md`, `.git`, `.gitignore`, `.editorconfig`
- `LICENSE`, `REPO_POLICIES.md` (copy from the `prompts` repo)
- `Makefile`
- `script/` entrypoints (`bootstrap`, `setup`, `projectname`, `test`,
`lint`, `fmt`, `fmt-check`, `check`, `docker`, `cibuild`, `precommit`,
`install-precommit`)
- `Dockerfile`, `.dockerignore`
- `.gitea/workflows/check.yml`
- Go: `go.mod`, `go.sum`, `.golangci.yml`
- JS: `package.json`, `yarn.lock`, `.prettierrc`, `.prettierignore`
- Python: `pyproject.toml`
- Guidance for coding agents lives in one `AGENTS.md` at the repository root. It
is never committed under a file or directory named after one agent tool, such
as `CLAUDE.md` or `.claude/`, and never split into separate memory files.
TODO.md
+67 -39
View File
@@ -1,55 +1,83 @@
# Workflow
* branch (from `main`)
* do the work in Next Step
* move Next Step to the top of Completed Steps
* move the top item of Future Steps into Next Step
* commit (`TODO.md` changes in the same commit as the work)
* merge to `main` if the branch is not protected, otherwise open a PR
* push
- branch (from `main`)
- do the work in Next Step
- move Next Step to the top of Completed Steps
- move the top item of Future Steps into Next Step
- commit (`TODO.md` changes in the same commit as the work)
- merge to `main` if the branch is not protected, otherwise open a PR
- push
# Status
1.0+
Tagged 1.0.0 (2025-05-08). Substantial correctness fixes and features
have landed since the tag.
Tagged 1.0.0 (2025-05-08). Substantial correctness fixes and features have
landed since the tag.
# Next Step
Policy scaffold commit: add LICENSE, REPO_POLICIES.md, .editorconfig,
.golangci.yml, and a comprehensive .gitignore (currently only the
attrsum binary), and extend the Makefile (only test/build/clean/try
today) with lint, fmt, fmt-check, check, and hooks targets. Fix the try
target to use a temp fixture instead of the hardcoded
$(HOME)/Documents/_SYSADMIN/cyberdyne path.
Restructure README.md into the standard sections: Description, Getting Started,
Entrypoints, Rationale, Design, TODO, License, Author (Getting Started, Why?,
TODO, License exist; Description, Entrypoints, Design, Author are missing)
# Completed Steps
* 2026-02-02: correctness pass: track actual bytes read instead of
stale file size, atomic failure tracking in ProcessCheck, detect
file modification during checksum (TOCTOU), propagate countFiles
errors, single progress bar across paths, error on empty stdin,
dead code removal
* 2026-02-01: added quiet mode, progress bar, summary report, and stdin
path input; multiple file/directory arguments for all commands
* 2025-07-12: README update
* 2025-05-08: initial working tool with passing tests, skips
non-regular files, Makefile, README; tagged 1.0.0
- 2026-10-06: `make fmt` formats the Markdown files with prettier (four-space
indents, `proseWrap: always`) and `make fmt-check` fails on one it would
change; prettier is pinned in `package.json` and `yarn.lock`, and
`script/bootstrap` installs it, along with the pinned node and yarn when the
yarn on hand is not the pinned version; the Markdown files reformatted once
- 2026-10-06: `script/fmt-check` checks what `script/fmt` writes: it fails and
lists the files when `gofmt -s` or goimports would change one, so a file that
passes `make check` no longer changes on the next `make fmt`
- 2026-10-06: canonical files re-vendored from `sneak/prompts` at `dd4027b`:
`REPO_POLICIES.md` and `.editorconfig` added; `.dockerignore`, `.gitignore`,
`.golangci.yml` and the workflow refreshed, keeping this repo's own entries
and `fetch-depth: 0`; the lint phase runs golangci-lint v2.14.0;
`script/bootstrap` installs goimports unless the installed one has the pinned
version, and it and `script/fmt` put Go's bin directory on `PATH`
- 2026-10-06: the summary line after `sum`, `check` and `clear` prints the byte
unit once (`1.5 KiB`, not `1.5 KiB bytes`)
- 2026-10-06: a path that `--exclude` or `--exclude-dotfiles` excludes is
skipped even when it cannot be read, so an excluded directory that cannot be
listed no longer fails the run
- 2026-10-06: lint and test run as phases of the `Dockerfile`, and the build
stage depends on both; `script/lint` and `script/test` each build their phase
with `--no-cache`; `script/cibuild` bootstraps, runs `script/check`, then
builds the image; golangci-lint is no longer installed on the host
- 2026-10-06: `check --continue` keeps going past a file or directory it cannot
read: it counts it as failed, prints the error and the path on stderr, and
checks the rest of the tree
- 2026-10-05: golangci-lint settings take effect: canonical `.golangci.yml` (v2
layout, settings under `linters.settings`), golangci-lint pinned at v2.12.2 in
`Dockerfile` and `script/bootstrap`, and the code fixed for what the settings
now report (long lines in `attrsum.go` rewrapped)
- 2026-10-05: `make try` runs on three small files in a temporary directory that
it removes afterwards, also when a step fails, instead of on a fixed directory
on one person's machine
- 2026-10-02: `attrsum --version` reports the git tag or short commit, stamped
by `make build` and by a plain `docker build .` of a clone; `.dockerignore`
sends `.git` without `.git/config` and keeps a host-built `attrsum` out; CI
checks out full history so it stamps the same version
- 2026-02-02: correctness pass: track actual bytes read instead of stale file
size, atomic failure tracking in ProcessCheck, detect file modification during
checksum (TOCTOU), propagate countFiles errors, single progress bar across
paths, error on empty stdin, dead code removal
- 2026-02-01: added quiet mode, progress bar, summary report, and stdin path
input; multiple file/directory arguments for all commands
- 2025-07-12: README update
- 2025-05-08: initial working tool with passing tests, skips non-regular files,
Makefile, README; tagged 1.0.0
# Future Steps
* Add Dockerfile and .dockerignore that run make check, images pinned
by sha256, plus a Makefile docker target
* Add .gitea/workflows/check.yml
* Restructure README.md into the standard sections: Description,
Getting Started, Rationale, Design, TODO, License, Author (Getting
Started, Why?, TODO, License exist; Description, Design, Author are
missing)
* Tag a patch release to ship the 2026-02-02 correctness fixes
* Dry-run mode (--dry-run, -n): show what would be done without making
changes (from README TODO)
* JSON output (--json) for scripting and integration (from README TODO)
* Parallel processing (-j N) with multiple goroutines for faster
checksumming on large trees (from README TODO)
* Formalize and document exit codes for scripting (from README TODO)
- Add a `LICENSE` file matching the README's WTFPL v2; sneak's to add, not an
agent's
- Tag a patch release to ship the 2026-02-02 correctness fixes
- Dry-run mode (--dry-run, -n): show what would be done without making changes
(from README TODO)
- JSON output (--json) for scripting and integration (from README TODO)
- Parallel processing (-j N) with multiple goroutines for faster checksumming on
large trees (from README TODO)
- Formalize and document exit codes for scripting (from README TODO)
attrsum.go
+90 -31
View File
@@ -36,6 +36,10 @@ const (
progressThrottle = 250 * time.Millisecond
)
// Version is the git tag or short commit, set at link time with -X by
// `make build`. A build that does not set it reports dev.
var Version = "dev" //nolint:gochecknoglobals // set at link time with -X
// Sentinel errors returned by the command implementations.
var (
errNoPaths = errors.New("no paths provided")
@@ -67,12 +71,13 @@ func (s *Stats) Duration() time.Duration {
return time.Since(s.StartTime)
}
func (s *Stats) Print(opts *options, operation string) {
func (s *Stats) Print(w io.Writer, opts *options, operation string) {
if opts.quiet {
return
}
fmt.Fprintf(os.Stderr, "\n%s complete: %d files processed, %d skipped, %d failed, %s bytes in %s\n",
_, _ = fmt.Fprintf(w,
"\n%s complete: %d files processed, %d skipped, %d failed, %s in %s\n",
operation,
s.FilesProcessed,
s.FilesSkipped,
@@ -103,6 +108,7 @@ func main() {
rootCmd := &cobra.Command{
Use: "attrsum",
Short: "Compute and verify file checksums via xattrs",
Version: Version,
}
rootCmd.SilenceUsage = true
rootCmd.SilenceErrors = true
@@ -166,12 +172,16 @@ func expandPaths(args []string) ([]string, error) {
}
// processFunc processes a single path within a command's run loop.
type processFunc func(opts *options, path string, stats *Stats, bar *progressbar.ProgressBar) error
type processFunc func(
opts *options, path string, stats *Stats, bar *progressbar.ProgressBar,
) error
// countAndBar counts the files under paths and returns a progress bar sized
// to that total. It always returns either a non-nil bar or a non-nil error.
func countAndBar(opts *options, paths []string, desc string) (*progressbar.ProgressBar, error) {
total, err := countFilesMultiple(opts, paths)
func countAndBar(
opts *options, paths []string, desc string, cont bool,
) (*progressbar.ProgressBar, error) {
total, err := countFilesMultiple(opts, paths, cont)
if err != nil {
return nil, err
}
@@ -188,7 +198,9 @@ func finishBar(bar *progressbar.ProgressBar) {
// runOverPaths runs process over each path, sharing the progress/stats
// bookkeeping common to the sum-add, sum-update and clear commands.
func runOverPaths(opts *options, args []string, desc, op string, process processFunc) error {
func runOverPaths(
opts *options, args []string, desc, op string, process processFunc,
) error {
paths, err := expandPaths(args)
if err != nil {
return err
@@ -199,7 +211,7 @@ func runOverPaths(opts *options, args []string, desc, op string, process process
var bar *progressbar.ProgressBar
if !opts.quiet {
bar, err = countAndBar(opts, paths, desc)
bar, err = countAndBar(opts, paths, desc, false)
if err != nil {
return err
}
@@ -215,7 +227,7 @@ func runOverPaths(opts *options, args []string, desc, op string, process process
}
finishBar(bar)
stats.Print(opts, op)
stats.Print(os.Stderr, opts, op)
return nil
}
@@ -253,8 +265,11 @@ func newSumCmd(opts *options) *cobra.Command {
return cmd
}
func processSumAdd(opts *options, dir string, stats *Stats, bar *progressbar.ProgressBar) error {
return walkAndProcess(opts, dir, stats, bar, func(p string, info os.FileInfo, s *Stats) error {
func processSumAdd(
opts *options, dir string, stats *Stats, bar *progressbar.ProgressBar,
) error {
return walkAndProcess(opts, dir, false, stats, bar,
func(p string, info os.FileInfo, s *Stats) error {
if hasXattr(p, checksumKey) {
atomic.AddInt64(&s.FilesSkipped, 1)
@@ -272,8 +287,11 @@ func processSumAdd(opts *options, dir string, stats *Stats, bar *progressbar.Pro
})
}
func processSumUpdate(opts *options, dir string, stats *Stats, bar *progressbar.ProgressBar) error {
return walkAndProcess(opts, dir, stats, bar, func(p string, info os.FileInfo, s *Stats) error {
func processSumUpdate(
opts *options, dir string, stats *Stats, bar *progressbar.ProgressBar,
) error {
return walkAndProcess(opts, dir, false, stats, bar,
func(p string, info os.FileInfo, s *Stats) error {
t, err := readSumTime(p)
if err != nil || info.ModTime().After(t) {
werr := writeChecksumAndTime(opts, p, info, s)
@@ -292,7 +310,9 @@ func processSumUpdate(opts *options, dir string, stats *Stats, bar *progressbar.
})
}
func writeChecksumAndTime(opts *options, path string, info os.FileInfo, stats *Stats) error {
func writeChecksumAndTime(
opts *options, path string, info os.FileInfo, stats *Stats,
) error {
// Record mtime before hashing to detect modifications during hash.
mtimeBefore := info.ModTime()
@@ -363,8 +383,11 @@ func newClearCmd(opts *options) *cobra.Command {
}
}
func processClear(opts *options, dir string, stats *Stats, bar *progressbar.ProgressBar) error {
return walkAndProcess(opts, dir, stats, bar, func(p string, info os.FileInfo, s *Stats) error {
func processClear(
opts *options, dir string, stats *Stats, bar *progressbar.ProgressBar,
) error {
return walkAndProcess(opts, dir, false, stats, bar,
func(p string, info os.FileInfo, s *Stats) error {
cleared, err := clearOne(opts, p)
if err != nil {
atomic.AddInt64(&s.FilesFailed, 1)
@@ -428,7 +451,8 @@ func newCheckCmd(opts *options) *cobra.Command {
return runCheck(opts, a, cont)
},
}
cmd.Flags().BoolVar(&cont, "continue", false, "continue after errors and report each file")
cmd.Flags().BoolVar(&cont, "continue", false,
"continue after errors and report each file")
return cmd
}
@@ -444,7 +468,7 @@ func runCheck(opts *options, args []string, cont bool) error {
var bar *progressbar.ProgressBar
if !opts.quiet {
bar, err = countAndBar(opts, paths, "Verifying checksums")
bar, err = countAndBar(opts, paths, "Verifying checksums", cont)
if err != nil {
return err
}
@@ -457,7 +481,7 @@ func runCheck(opts *options, args []string, cont bool) error {
if perr != nil {
if !cont {
finishBar(bar)
stats.Print(opts, "check")
stats.Print(os.Stderr, opts, "check")
return perr
}
@@ -467,16 +491,19 @@ func runCheck(opts *options, args []string, cont bool) error {
}
finishBar(bar)
stats.Print(opts, "check")
stats.Print(os.Stderr, opts, "check")
return finalErr
}
func processCheck(opts *options, dir string, cont bool, stats *Stats, bar *progressbar.ProgressBar) error {
func processCheck(
opts *options, dir string, cont bool, stats *Stats, bar *progressbar.ProgressBar,
) error {
// Track initial failed count to detect failures during this walk.
initialFailed := atomic.LoadInt64(&stats.FilesFailed)
err := walkAndProcess(opts, dir, stats, bar, func(p string, _ os.FileInfo, s *Stats) error {
err := walkAndProcess(opts, dir, cont, stats, bar,
func(p string, _ os.FileInfo, s *Stats) error {
return checkOne(opts, p, cont, s)
})
if err != nil {
@@ -500,7 +527,7 @@ func checkOne(opts *options, p string, cont bool, s *Stats) error {
exp, err := xattr.Get(p, checksumKey)
if err != nil {
if !errors.Is(err, xattr.ENOATTR) {
return err
return unreadable(cont, s, err)
}
return missingChecksum(opts, p, cont, s)
@@ -508,9 +535,7 @@ func checkOne(opts *options, p string, cont bool, s *Stats) error {
act, bytesRead, err := fileMultihash(p)
if err != nil {
atomic.AddInt64(&s.FilesFailed, 1)
return err
return unreadable(cont, s, err)
}
ok := bytes.Equal(exp, act)
@@ -546,6 +571,21 @@ func missingChecksum(opts *options, p string, cont bool, s *Stats) error {
return errVerification
}
// unreadable counts a file or directory that could not be read as failed.
// With --continue it prints err, which names the path, to stderr and
// returns nil so the walk goes on; otherwise it returns err.
func unreadable(cont bool, s *Stats, err error) error {
atomic.AddInt64(&s.FilesFailed, 1)
if !cont {
return err
}
log.Print(err)
return nil
}
// reportCheck prints a per-file verification result when verbose output is on.
func reportCheck(opts *options, p, actual string, ok bool) {
if !opts.verbose || opts.quiet {
@@ -565,12 +605,24 @@ func reportCheck(opts *options, p, actual string, ok bool) {
///////////////////////////////////////////////////////////////////////////////
// countFiles counts the total number of regular files that will be processed.
func countFiles(opts *options, root string) (int64, error) {
// With cont, a path it cannot read is left out of the count instead of ending
// it; the walk that follows reports that path as failed.
func countFiles(opts *options, root string, cont bool) (int64, error) {
var count int64
root = filepath.Clean(root)
err := filepath.Walk(root, func(p string, info os.FileInfo, err error) error {
// An excluded path is skipped whatever went wrong reading it.
rel, _ := filepath.Rel(root, p)
if err != nil && shouldExclude(opts, rel) {
return nil
}
if err != nil && cont {
return nil
}
if err != nil {
return err
}
@@ -581,7 +633,6 @@ func countFiles(opts *options, root string) (int64, error) {
return nil
}
rel, _ := filepath.Rel(root, p)
if shouldExclude(opts, rel) {
if info.IsDir() {
return filepath.SkipDir
@@ -605,11 +656,11 @@ func countFiles(opts *options, root string) (int64, error) {
}
// countFilesMultiple counts files across multiple roots.
func countFilesMultiple(opts *options, roots []string) (int64, error) {
func countFilesMultiple(opts *options, roots []string, cont bool) (int64, error) {
var total int64
for _, root := range roots {
count, err := countFiles(opts, root)
count, err := countFiles(opts, root, cont)
if err != nil {
return total, err
}
@@ -645,6 +696,7 @@ func newProgressBar(total int64, description string) *progressbar.ProgressBar {
func walkAndProcess(
opts *options,
root string,
cont bool,
stats *Stats,
bar *progressbar.ProgressBar,
fn func(string, os.FileInfo, *Stats) error,
@@ -652,8 +704,14 @@ func walkAndProcess(
root = filepath.Clean(root)
return filepath.Walk(root, func(p string, info os.FileInfo, err error) error {
// An excluded path is skipped whatever went wrong reading it.
rel, _ := filepath.Rel(root, p)
if err != nil && shouldExclude(opts, rel) {
return nil
}
if err != nil {
return err
return unreadable(cont, stats, err)
}
skip, skipErr := walkSkip(opts, root, p, info)
@@ -742,7 +800,8 @@ func hasXattr(path, key string) bool {
func fileMultihash(path string) ([]byte, int64, error) {
// The path is supplied by the operator as the tree to checksum; reading
// it is the entire purpose of the tool.
f, err := os.Open(path) //nolint:gosec // G304: operator-specified path is the intended input
//nolint:gosec // G304: operator-specified path is the intended input
f, err := os.Open(path)
if err != nil {
return nil, 0, err
}
attrsum_test.go
+130
View File
@@ -1,6 +1,8 @@
package main
import (
"bytes"
"errors"
"os"
"path/filepath"
"strings"
@@ -213,6 +215,54 @@ func TestExcludeDotfilesAndPatterns(t *testing.T) {
}
}
func TestExcludeUnreadableDir(t *testing.T) {
t.Parallel()
opts := &options{excludePatterns: []string{"locked"}}
dir := t.TempDir()
skipIfNoXattr(t, dir)
keep := writeFile(t, dir, "keep.txt", "keep")
hidden := writeFile(t, dir, "locked/a.txt", "hidden")
// The excluded directory cannot be listed.
sub := filepath.Join(dir, "locked")
err := os.Chmod(sub, noPerm)
if err != nil {
t.Fatalf("chmod dir: %v", err)
}
defer func() { _ = os.Chmod(sub, dirPerm) }()
err = processSumAdd(opts, dir, newTestStats(), nil)
if err != nil {
t.Fatalf("add: %v", err)
}
_, err = xattr.Get(keep, checksumKey)
if err != nil {
t.Fatalf("expected xattr on keep.txt: %v", err)
}
// Without --quiet, runCheck counts the files for the progress bar
// before it checks any, so this also covers the count.
err = runCheck(opts, []string{dir}, false)
if err != nil {
t.Fatalf("check: %v", err)
}
err = os.Chmod(sub, dirPerm)
if err != nil {
t.Fatalf("chmod dir back: %v", err)
}
_, err = xattr.Get(hidden, checksumKey)
if err == nil {
t.Fatalf("locked/a.txt should have been excluded")
}
}
func TestSkipBrokenSymlink(t *testing.T) {
t.Parallel()
@@ -271,3 +321,83 @@ func TestPermissionErrors(t *testing.T) {
t.Fatalf("expected permission error on check, got nil")
}
}
func TestCheckContinuePastUnreadable(t *testing.T) {
t.Parallel()
opts := &options{}
dir := t.TempDir()
skipIfNoXattr(t, dir)
writeFile(t, dir, "a.txt", "one")
secret := writeFile(t, dir, "b.txt", "two")
writeFile(t, dir, "c/d.txt", "three")
writeFile(t, dir, "e.txt", "four")
err := processSumAdd(opts, dir, newTestStats(), nil)
if err != nil {
t.Fatalf("add: %v", err)
}
// An unreadable file and an unlistable directory sit between the
// readable files a.txt and e.txt.
sub := filepath.Join(dir, "c")
err = os.Chmod(secret, noPerm)
if err != nil {
t.Fatalf("chmod file: %v", err)
}
defer func() { _ = os.Chmod(secret, filePerm) }()
err = os.Chmod(sub, noPerm)
if err != nil {
t.Fatalf("chmod dir: %v", err)
}
defer func() { _ = os.Chmod(sub, dirPerm) }()
stats := newTestStats()
err = processCheck(opts, dir, true, stats, nil)
if !errors.Is(err, errVerification) {
t.Fatalf("expected verification error, got %v", err)
}
if stats.FilesProcessed != 2 || stats.FilesFailed != 2 {
t.Fatalf("expected 2 verified and 2 failed, got %d and %d",
stats.FilesProcessed, stats.FilesFailed)
}
// Without --quiet, runCheck counts the files for the progress bar
// before it checks any, so the count reaches the unlistable directory
// first.
err = runCheck(opts, []string{dir}, true)
if !errors.Is(err, errVerification) {
t.Fatalf("expected verification error from runCheck, got %v", err)
}
}
func TestSummaryPrintsByteUnitOnce(t *testing.T) {
t.Parallel()
tests := []struct {
bytesProcessed int64
want string
}{
{9, ", 9 B in "},
{1536, ", 1.5 KiB in "},
}
for _, tt := range tests {
var out bytes.Buffer
stats := newTestStats()
stats.BytesProcessed = tt.bytesProcessed
stats.Print(&out, &options{}, "check")
if !strings.Contains(out.String(), tt.want) {
t.Errorf("summary %q does not contain %q", out.String(), tt.want)
}
}
}
package.json
+6
View File
@@ -0,0 +1,6 @@
{
"private": true,
"devDependencies": {
"prettier": "3.8.1"
}
}
script/bootstrap
+147 -10
View File
@@ -3,18 +3,30 @@
# this repo. Idempotent: every install is guarded by a check so already
# installed tools are skipped. Base tooling comes from nix, apt, brew,
# or apk (detected in that order); assumes nothing is present.
# golangci-lint and goimports are installed via `go install` at the same
# pinned commits the Dockerfile uses (never "latest").
# goimports is installed via `go install` at a pinned commit (never
# "latest"), unless the installed one already has the pinned version.
# The linter is not installed: it runs only as the lint phase of the
# Dockerfile. yarn is installed via corepack unless the yarn that
# script/fmt runs already has the pinned version. It is installed under
# the node on PATH if that has the pinned version; otherwise the pinned
# node is installed via nvm (installing nvm itself first, from a
# hash-verified release archive, never curl | sh).
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Pinned versions, 2026-07-07 (same pins as the Dockerfile)
# golangci-lint v2.10.1
GOLANGCI_LINT_REF="github.com/golangci/golangci-lint/v2/cmd/golangci-lint@5d1e709b7be35cb2025444e19de266b056b7b7ee"
# goimports v0.42.0
# Pinned versions, 2026-10-05
# GOIMPORTS_REF is the commit tagged GOIMPORTS_VERSION.
GOIMPORTS_VERSION="v0.42.0"
GOIMPORTS_REF="golang.org/x/tools/cmd/goimports@009367f5c17a8d4c45a961a3a509277190a9a6f0"
# Pinned versions, 2026-07-06
NODE_VERSION="22.17.0"
NVM_VERSION="0.40.3"
# sha256 of https://github.com/nvm-sh/nvm/archive/refs/tags/v0.40.3.tar.gz
NVM_SHA256="5f4d6aaa04a177dc93c985e31dbc411ab6b8c6e1e21d8015dbc1372625fcd1d0"
YARN_VERSION="1.22.22"
PKGMGR=""
SUDO=""
APT_UPDATED=""
@@ -62,6 +74,127 @@ missing() {
! command -v "$1" >/dev/null 2>&1
}
# Print the version the goimports on PATH was built from, or nothing.
# goimports has no version flag; `go version -m` reads the binary's
# build info, whose "mod" line names the module and its version.
goimports_version() {
if missing goimports; then return 0; fi
go version -m "$(command -v goimports)" 2>/dev/null |
awk '$1 == "mod" { print $3 }'
}
ensure_goimports() {
if [ "$(goimports_version)" = "$GOIMPORTS_VERSION" ]; then
echo "goimports $GOIMPORTS_VERSION already installed"
return 0
fi
go install "$GOIMPORTS_REF"
hash -r
if [ "$(goimports_version)" != "$GOIMPORTS_VERSION" ]; then
echo "bootstrap: goimports on PATH is not $GOIMPORTS_VERSION:" \
"$(command -v goimports)" >&2
exit 1
fi
echo "goimports $GOIMPORTS_VERSION installed"
}
# verify_sha256 <file> <expected-hash>
verify_sha256() {
if command -v sha256sum >/dev/null 2>&1; then
actual="$(sha256sum "$1" | cut -d' ' -f1)"
else
actual="$(shasum -a 256 "$1" | cut -d' ' -f1)"
fi
if [ "$actual" != "$2" ]; then
echo "bootstrap: sha256 mismatch for $1" >&2
echo " expected: $2" >&2
echo " actual: $actual" >&2
exit 1
fi
}
# nvm is a bash script; run a command in a bash with nvm loaded
nvm_sh() {
bash -c ". \"\$HOME/.nvm/nvm.sh\" && $*"
}
ensure_nvm() {
[ -s "$HOME/.nvm/nvm.sh" ] && return 0
# nvm prerequisites; nvm itself requires bash
if missing bash; then pkg_install bash bash bash bash; fi
if missing curl; then pkg_install curl curl curl curl; fi
if missing git; then pkg_install git git git git; fi
tmp="$(mktemp -d)"
curl -fsSL -o "$tmp/nvm.tar.gz" \
"https://github.com/nvm-sh/nvm/archive/refs/tags/v${NVM_VERSION}.tar.gz"
verify_sha256 "$tmp/nvm.tar.gz" "$NVM_SHA256"
mkdir -p "$HOME/.nvm"
tar -xzf "$tmp/nvm.tar.gz" -C "$HOME/.nvm" --strip-components=1
rm -rf "$tmp"
}
# Print the version of the node on PATH, such as v22.17.0, or nothing.
node_version() {
if missing node; then return 0; fi
node --version 2>/dev/null
}
ensure_node() {
if [ "$(node_version)" = "v$NODE_VERSION" ]; then
echo "node $NODE_VERSION already installed"
return 0
fi
ensure_nvm
nvm_sh "nvm install $NODE_VERSION"
echo "node $NODE_VERSION installed via nvm"
}
# Print the version of the yarn that script/fmt and script/fmt-check
# run, or nothing: the yarn on PATH, else the one under the pinned node
# in nvm.
yarn_version() {
if ! missing yarn; then
yarn --version 2>/dev/null
elif [ -s "$HOME/.nvm/nvm.sh" ]; then
nvm_sh "nvm use $NODE_VERSION >/dev/null && yarn --version" \
2>/dev/null
fi
}
# A yarn that already has the pinned version is used with the node that
# runs it. Otherwise yarn is installed via corepack under the pinned
# node.
ensure_yarn() {
if [ "$(yarn_version)" = "$YARN_VERSION" ]; then
echo "yarn $YARN_VERSION already installed"
return 0
fi
ensure_node
if [ "$(node_version)" = "v$NODE_VERSION" ]; then
corepack enable
corepack prepare "yarn@$YARN_VERSION" --activate
else
nvm_sh "nvm use $NODE_VERSION >/dev/null && corepack enable && \
corepack prepare yarn@$YARN_VERSION --activate"
fi
hash -r
if [ "$(yarn_version)" != "$YARN_VERSION" ]; then
echo "bootstrap: the yarn script/fmt runs is not $YARN_VERSION:" \
"$(command -v yarn || echo "none on PATH")" >&2
exit 1
fi
echo "yarn $YARN_VERSION installed"
}
install_js_deps() {
if missing yarn && [ -s "$HOME/.nvm/nvm.sh" ]; then
nvm_sh "nvm use $NODE_VERSION >/dev/null && cd \"$ROOT\" && \
yarn install --frozen-lockfile"
else
yarn install --frozen-lockfile
fi
}
main() {
cd "$ROOT"
@@ -69,13 +202,17 @@ main() {
if missing make; then pkg_install gnumake make make make; fi
if missing go; then pkg_install go golang go go; fi
# Lint/format tools, pinned via go install (installs into
# "$(go env GOPATH)/bin"; ensure that is on your PATH).
if missing golangci-lint; then go install "$GOLANGCI_LINT_REF"; fi
if missing goimports; then go install "$GOIMPORTS_REF"; fi
# go install writes to Go's bin directory, which need not be on PATH.
gobin="$(go env GOBIN)"
[ -n "$gobin" ] || gobin="$(go env GOPATH)/bin"
PATH="$gobin:$PATH"
ensure_goimports
go mod download
ensure_yarn
install_js_deps
echo "bootstrap complete"
}
script/check
+3 -1
View File
@@ -1,6 +1,8 @@
#!/bin/sh
# script/check: run all checks (test, lint, fmt-check). Our own
# extension to scripts-to-rule-them-all. Must not modify any files.
# extension to scripts-to-rule-them-all. test and lint are Docker
# phases; fmt-check is native, because a formatter writes the working
# tree. Must not modify any files.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
script/cibuild
+19 -4
View File
@@ -1,13 +1,28 @@
#!/bin/sh
# script/cibuild: run the CI build. The Dockerfile runs make check, so
# a successful build implies all checks pass.
# script/cibuild: run the CI build. It bootstraps first: a CI runner
# checks out and runs this and nothing else, and script/fmt-check runs
# the formatter on the host, which a pristine checkout cannot do.
# --no-cache for the same reason as script/docker: the gate phases the
# final stage depends on are RUN steps, and a cached one is a check that
# did not run.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
docker build .
"$SCRIPT_DIR/bootstrap"
"$SCRIPT_DIR/check"
# Own line: a failing command substitution inside an argument does
# not trip `set -e`, so the inline form degrades silently to an
# empty constant. The VERSION build argument takes precedence over
# the version a build stage derives from the .git in the context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown"
docker build --no-cache \
--build-arg VERSION="$version" \
-t "$("$SCRIPT_DIR/projectname")" .
}
main "$@"
script/docker
+11 -1
View File
@@ -1,6 +1,8 @@
#!/bin/sh
# script/docker: build the Docker image tagged with the project name.
# Identical in all repos; the tag comes from script/projectname.
# --no-cache because the gate phases the final stage depends on are RUN
# steps, and a cached one is a check that did not run.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -8,7 +10,15 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
docker build -t "$("$SCRIPT_DIR/projectname")" .
# Own line: a failing command substitution inside an argument does
# not trip `set -e`, so the inline form degrades silently to an
# empty constant. The VERSION build argument takes precedence over
# the version a build stage derives from the .git in the context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown"
docker build --no-cache \
--build-arg VERSION="$version" \
-t "$("$SCRIPT_DIR/projectname")" .
}
main "$@"
script/fmt
+26
View File
@@ -4,10 +4,36 @@ set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Must match the pin in script/bootstrap.
NODE_VERSION="22.17.0"
# script/bootstrap installs node and yarn under nvm and leaves neither
# on the PATH of the shell that called it, so resolve the pinned
# toolchain here the way bootstrap's own install step does. nvm is a
# bash script, hence the subshell.
run_yarn() {
if command -v yarn >/dev/null 2>&1; then
yarn "$@"
return
fi
if [ ! -s "$HOME/.nvm/nvm.sh" ]; then
echo "fmt: no yarn; run script/bootstrap first" >&2
exit 1
fi
bash -c '. "$HOME/.nvm/nvm.sh" && nvm use "$1" >/dev/null &&
shift && exec yarn "$@"' bash "$NODE_VERSION" "$@"
}
main() {
cd "$ROOT"
# script/bootstrap installs goimports into Go's bin directory, which
# need not be on PATH.
gobin="$(go env GOBIN)"
[ -n "$gobin" ] || gobin="$(go env GOPATH)/bin"
PATH="$gobin:$PATH"
gofmt -s -w .
goimports -w .
run_yarn run prettier --write '**/*.md' --tab-width 4 --prose-wrap always
}
main "$@"
script/fmt-check
+32 -6
View File
@@ -5,14 +5,40 @@ set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
files="$(gofmt -l .)"
if [ -n "$files" ]; then
echo "gofmt: files not formatted:" >&2
echo "$files" >&2
# Must match the pin in script/bootstrap.
NODE_VERSION="22.17.0"
# script/bootstrap installs node and yarn under nvm and leaves neither
# on the PATH of the shell that called it, so resolve the pinned
# toolchain here the way bootstrap's own install step does. nvm is a
# bash script, hence the subshell.
run_yarn() {
if command -v yarn >/dev/null 2>&1; then
yarn "$@"
return
fi
if [ ! -s "$HOME/.nvm/nvm.sh" ]; then
echo "fmt-check: no yarn; run script/bootstrap first" >&2
exit 1
fi
bash -c '. "$HOME/.nvm/nvm.sh" && nvm use "$1" >/dev/null &&
shift && exec yarn "$@"' bash "$NODE_VERSION" "$@"
}
main() {
cd "$ROOT"
# script/bootstrap installs goimports into Go's bin directory, which
# need not be on PATH.
gobin="$(go env GOBIN)"
[ -n "$gobin" ] || gobin="$(go env GOPATH)/bin"
PATH="$gobin:$PATH"
files="$(gofmt -s -l .; goimports -l .)"
if [ -n "$files" ]; then
echo "files that make fmt would change:" >&2
echo "$files" | sort -u >&2
exit 1
fi
run_yarn run prettier --check '**/*.md' --tab-width 4 --prose-wrap always
}
main "$@"
script/lint
+14 -3
View File
@@ -1,12 +1,23 @@
#!/bin/sh
# script/lint: run the linter.
# script/lint: run the linter. Linting is a phase of the Dockerfile and
# this builds that phase alone; the linter is never installed or run on
# a developer host, where a shared result cache and a host-global lock
# make its answer untrustworthy.
#
# The phase is not the last stage in the file, so it is built only when
# --target names it. --no-cache because a cached lint layer is a lint
# that did not run. The tag makes each build replace the previous image
# instead of leaving a dangling one behind.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
golangci-lint run --config .golangci.yml ./...
docker build --no-cache \
--target lint \
-t "$("$SCRIPT_DIR/projectname")-lint" .
}
main "$@"
script/test
+10 -3
View File
@@ -1,12 +1,19 @@
#!/bin/sh
# script/test: run the test suite.
# script/test: run the test suite. Testing is a phase of the Dockerfile
# and this builds that phase alone, on the same terms as script/lint:
# --target because a phase that is not the last stage is built only when
# named, --no-cache because a cached test layer is a test that did not
# run, and a tag so each build replaces the previous image.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
go test -v -race -timeout 30s -cover ./...
docker build --no-cache \
--target test \
-t "$("$SCRIPT_DIR/projectname")-test" .
}
main "$@"
yarn.lock
+8
View File
@@ -0,0 +1,8 @@
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
prettier@3.8.1:
version "3.8.1"
resolved "https://registry.yarnpkg.com/prettier/-/prettier-3.8.1.tgz#edf48977cf991558f4fcbd8a3ba6015ba2a3a173"
integrity sha512-UOnG6LftzbdaHZcKoPFtOcCKztrQ57WkHDeRD9t/PTQtmT0NHSeWWepj6pS0z/N7+08BHFDQVUrfmfMRcZwbMg==