Author SHA1 Message Date
clawbot 3e4ca5b999 Keep only open work in TODO.md (closes #198)
check / check (push) Successful in 3m51s
Removes the Completed Steps work log and the Workflow step that fed
it, and drops history notes from the open items. Items that named
removed macOS code now name what is still open.

Model: opus-5-5
2026-11-11 00:10:31 +00:00
clawbot 176aee05aa Point the commandWait comment at the Dockerfile's test limit (closes #195)
check / check (push) Canceled after 0s
The comment said script/test gives the package 30 seconds. script/test
sets no time limit; the test phase of the Dockerfile sets one with
go test -timeout. The comment now names that place instead of a number.

Model: opus-5-5
2026-10-15 20:25:07 +02:00
clawbot 26a4739615 Keep Go's build cache in check builds: re-vendor sneak/prompts e3f79c4 (closes #129)
check / check (push) Canceled after 0s
The vendored files are copies from sneak/prompts e3f79c4, with this
repository's own entries kept. The script/ builds pass a new CACHEBUST
value instead of --no-cache, which emptied Go's build cache mount, so
every make test compiled the standard library and every dependency with
-race. Every Dockerfile stage declares ARG CACHEBUST first, so every RUN
step still reruns on an unchanged tree. The steps that compile mount
Go's build cache, and both go test runs keep -count=1. The lint and test
builds write no image. script/bootstrap, script/fmt and script/fmt-check
are the canonical files plus this repository's Go steps.

Model: opus-5-5
2026-10-15 19:58:38 +02:00
clawbot 8851171877 Set the time zone in main and run Entry in its own test process (closes #192)
check / check (push) Canceled after 0s
cli.Entry() set time.Local to UTC and ended with memguard.Purge(), and
TestEntryWipesBuffersAndReturnsExitCode ran it in the same process as
the other tests. A timer left by an earlier test could read time.Local
as it changed, which the race detector reported on some runs, and the
purge could destroy a buffer still in use, the likeliest cause of a
crash seen on another run.

main() in cmd/secret now sets time.Local before it calls cli.Entry();
the test that runs the built binary with TZ=America/New_York still
covers it. The Entry test runs cli.Entry() in a copy of the test binary
started with SECRET_TEST_RUN_ENTRY set, which init picks up so that
TestMain does not run there.

Model: opus-5-5
2026-10-15 19:25:10 +02:00
clawbot 25f4652e70 Log the audit fixes in TODO.md (closes #173)
check / check (push) Canceled after 0s
Completed Steps gets one entry for each fix merged into next since
v1.0.0-rc2 that had none, newest first in landing order. Each entry says
what its landing commit says and carries that commit's date in UTC. The
five fixes that logged themselves before this issue's decision keep
their entries and get no second one. Status no longer says there are no
git tags.

Model: opus-5-5
2026-10-15 17:58:57 +02:00
22 changed files with 395 additions and 760 deletions

No files matched your search

+3
View File
@@ -13,3 +13,6 @@ indent_style = tab
[*.go]
indent_style = tab
# This repository's own sections, such as one for another language it
# uses, go below this comment, and a re-vendor keeps them.
+11
View File
@@ -1,9 +1,20 @@
name: check
on: [push]
# Free the shared runner: a new push cancels only the same branch's older run.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
check:
runs-on: ubuntu-latest
# Free the shared runner from a hung build.
timeout-minutes: 20
steps:
# actions/checkout v4.2.2, 2026-02-22
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
# script/cibuild needs no token, so none is left in .git/config.
with:
persist-credentials: false
# All history and tags, so git describe finds the version tag.
fetch-depth: 0
- run: script/cibuild
+5 -1
View File
@@ -27,7 +27,7 @@ node_modules/
# Environment files. `*.env` covers bare `.env` and the `prod.env`
# convention. Only the templates `example.env` and `sample.env` are
# re-included below. A repository that commits any other template adds
# its own negation after these lines, for example `!.env.example`.
# its own negation at the end of this file, for example `!.env.example`.
*.[eE][nN][vV]
.[eE][nN][vV].*
.[eE][nN][vV][rR][cC]
@@ -46,6 +46,10 @@ node_modules/
[iI][dD]_[eE][dD]25519
[iI][dD]_[eE][dD]25519_[sS][kK]
# This repository's own entries, such as its build outputs, go below
# this comment, and a re-vendor keeps them. Anchor a binary built at the
# root: `/myapp`, never `myapp`, which also ignores `cmd/myapp/`.
# Go. /secret is the built binary, anchored so that it does not also match
# the internal/secret/ package directory.
*.log
+2
View File
@@ -25,6 +25,8 @@ linters:
# silenced by disabling that name, not by enabling the successor.
- wsl # Deprecated, replaced by wsl_v5
- gomodguard # Deprecated, replaced by gomodguard_v2
# Misses findings at random in v2.14.0; back once a pinned release fixes it
- canonicalheader
settings:
lll:
line-length: 88
+21 -11
View File
@@ -1,33 +1,40 @@
# Every stage declares ARG CACHEBUST first: script/ passes a new value
# on every build, which reruns every RUN step after it. The steps that
# compile mount Go's build cache, which survives from build to build.
# Lint phase. The linter is invoked directly rather than through `make
# lint` or `script/lint`, which are themselves a docker build.
# golangci/golangci-lint:v2.14.0 (Debian-based), 2026-09-24
FROM golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint
ARG CACHEBUST
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN go vet ./...
RUN golangci-lint run --config .golangci.yml ./...
RUN --mount=type=cache,target=/root/.cache/go-build \
go vet ./...
RUN --mount=type=cache,target=/root/.cache/go-build \
golangci-lint run --config .golangci.yml ./...
# The same checks on the code as a macOS build compiles it, which a Linux
# build never compiles. Cgo is off, because compiling cgo code for macOS
# needs Apple's SDK headers.
RUN GOOS=darwin CGO_ENABLED=0 go vet ./...
RUN GOOS=darwin CGO_ENABLED=0 golangci-lint run --config .golangci.yml ./...
RUN --mount=type=cache,target=/root/.cache/go-build \
GOOS=darwin CGO_ENABLED=0 go vet ./...
RUN --mount=type=cache,target=/root/.cache/go-build \
GOOS=darwin CGO_ENABLED=0 golangci-lint run --config .golangci.yml ./...
# Test phase. -race needs cgo and so a C compiler, which the Debian Go
# image ships and the alpine one does not.
# image ships and the alpine one does not. -count=1 because the mounted
# cache also holds test results, which go test would otherwise replay.
# golang:1.24.13-trixie, 2026-02-04
FROM golang@sha256:5835f052b784aa39f2fe9070def3568605c8bc3fcd810f10402066348b61e716 AS test
ARG CACHEBUST
ENV CGO_ENABLED=1
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
# Go's build cache goes in a cache mount, not the image layer, which would
# take seconds longer to export. --no-cache, on every build in script/,
# starts the mount empty; -count=1 keeps a build without it from taking
# test results from there.
RUN --mount=type=cache,id=sneak/secret/go-build-test,target=/root/.cache/go-build \
RUN --mount=type=cache,target=/root/.cache/go-build \
go test -count=1 -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \
go test -count=1 -timeout 90s -race -v ./...; exit 1; }
@@ -37,6 +44,7 @@ RUN --mount=type=cache,id=sneak/secret/go-build-test,target=/root/.cache/go-buil
# lint and test passed.
# golang 1.24.13-alpine, 2026-03-10
FROM golang@sha256:8bee1901f1e530bfb4a7850aa7a479d17ae3a18beb6e09064ed54cfd245b7191 AS builder
ARG CACHEBUST
COPY --from=lint /src/go.sum /dev/null
COPY --from=test /src/go.sum /dev/null
# script/build compiles with cgo, so it needs a C compiler too.
@@ -53,7 +61,8 @@ COPY . .
# .git present, a version that is still empty, dev or unknown fails the
# build: git is missing or could not read the checkout.
ARG VERSION
RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \
RUN --mount=type=cache,target=/root/.cache/go-build \
VERSION="${VERSION:-$(git describe --tags --always)}"; \
if [ -e .git ]; then \
case "$VERSION" in ""|dev|unknown) \
echo "version is '$VERSION' although .git is present" >&2; \
@@ -65,6 +74,7 @@ RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \
# Runtime stage, and the last one
# alpine 3.23, 2026-03-10
FROM alpine@sha256:25109184c71bdad752c8312a8623239686a9a2071e8825f20acb8f2198c3f659
ARG CACHEBUST
RUN apk add --no-cache ca-certificates gnupg
+4
View File
@@ -675,6 +675,10 @@ provide:
- `script/install-precommit` — install the git pre-commit hook that runs
`script/precommit`
Every stage of the `Dockerfile` declares `ARG CACHEBUST` as its first line after
`FROM`. The `script/` builds pass a new `CACHEBUST` value on every build, which
reruns every `RUN` step and keeps Go's build cache; `--no-cache` would empty it.
## Features
- **Multiple Authentication Methods**: Supports passphrase and PGP unlockers
+156 -79
View File
@@ -1,6 +1,6 @@
---
title: Repository Policies
last_modified: 2026-10-04
last_modified: 2026-10-15
---
This document covers repository structure, tooling, and workflow standards. Code
@@ -54,34 +54,39 @@ style conventions are in separate documents:
`cibuild`. `script/bootstrap` installs all dependencies idempotently and
assumes nothing is present: base tools come from nix, apt, brew, or apk
(detected in that order; apt runs noninteractive). For node it uses the
installed node if present; otherwise it installs a PINNED node version via
nvm, first installing nvm itself if missing — from a hash-verified GitHub
release archive (never `curl | sh`), with bash installed as an explicit
prerequisite since nvm requires bash. yarn is then pinned via
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts";
always exact versions. `script/cibuild` runs the CI build: it changes to the
repo root, runs `script/bootstrap`, runs `script/check`, and builds the image
with the version; the Gitea workflow calls it. **`script/cibuild` runs
installed node only when its major version is the pinned one; otherwise (node
missing, or another major, such as the node 24 the CI runner image ships) it
installs the PINNED node version via nvm and installs yarn and the packages
under it. nvm itself is installed first if missing, from a hash-verified
GitHub release archive (never `curl | sh`), with bash installed as an explicit
prerequisite since nvm requires bash. Only the major version is compared
because the `Dockerfile` stages start from a node image whose exact version is
not the pin, and nvm cannot install a prebuilt node on alpine. yarn is pinned
via `corepack prepare yarn@<version> --activate`. Never install "latest" or
"lts"; always exact versions. `script/cibuild` runs the CI build: it changes
to the repo root, runs `script/bootstrap`, runs `script/check`, and builds the
image with the version; the Gitea workflow calls it. **`script/cibuild` runs
`script/bootstrap` first**, because the workflow checks out the repo and runs
nothing else, while `script/fmt-check` runs the formatter on the host: on a
pristine checkout with nothing installed the run dies there, after the
containerised gates have passed. **The bootstrap alone is not enough**:
`script/bootstrap` installs node and yarn under nvm and leaves neither on the
`PATH` of the shell that called it, so a bare `yarn` still exits 127. The host
entrypoints that need yarn — `script/fmt` and `script/fmt-check` — therefore
source nvm for the pinned node version before invoking it, exactly as
`script/bootstrap`'s own install step does. A runner carrying nothing but
docker and git then gets through `script/check`. Four further scripts are our
own extensions to the standard: `script/check` runs `script/test`,
`script/lint` and `script/fmt-check`; `script/precommit` is what the git
pre-commit hook runs, and it calls `script/check`; `script/install-precommit`
installs the git pre-commit hook (the `make hooks` target shims to it); and
`script/projectname` (literally that filename) simply outputs the project's
name. Scripts that need the name call `script/projectname` — e.g.
`script/docker` assembles its image tag from it — so those scripts stay
byte-identical across all repos. Repo-type-specific pre-commit extras (e.g.
`go mod tidy` verification in Go repos) belong in `script/precommit`, not in
the hook itself. Model scripts are at
containerised gates have passed. **The bootstrap alone is not enough**: when
`script/bootstrap` installs node and yarn under nvm it leaves neither on the
`PATH` of the shell that called it, so a bare `yarn` either exits 127 or runs
under another node. The host entrypoints that need yarn — `script/fmt` and
`script/fmt-check` — therefore use the same test as `script/bootstrap`: when
the node on `PATH` has the pinned major version they run the `yarn` on `PATH`,
and otherwise they load nvm through `$HOME/.nvm/nvm.sh` and run yarn under the
pinned node. A runner carrying nothing but docker and git then gets through
`script/check`. Four further scripts are our own extensions to the standard:
`script/check` runs `script/test`, `script/lint` and `script/fmt-check`;
`script/precommit` is what the git pre-commit hook runs, and it calls
`script/check`; `script/install-precommit` installs the git pre-commit hook
(the `make hooks` target shims to it); and `script/projectname` (literally
that filename) simply outputs the project's name. Scripts that need the name
call `script/projectname` — e.g. `script/docker` assembles its image tag from
it — so those scripts stay byte-identical across all repos. Repo-type-specific
pre-commit extras (e.g. `go mod tidy` verification in Go repos) belong in
`script/precommit`, not in the hook itself. Model scripts are at
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
must document the provided scripts in an **Entrypoints** section (see the
README requirements below).
@@ -118,8 +123,10 @@ style conventions are in separate documents:
and nothing else:
```sh
docker build --no-cache --target lint -t "$(script/projectname)-lint" .
docker build --no-cache --target test -t "$(script/projectname)-test" .
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--target lint --output type=cacheonly .
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--target test --output type=cacheonly .
```
**A stage that is not the last one in the file is built only when the final
@@ -129,10 +136,15 @@ style conventions are in separate documents:
plain `docker build .` builds the last stage alone and exits 0 having linted
and tested nothing.
**Every `docker build` in `script/` is tagged**, here and in
`script/cibuild` and `script/docker`. An untagged build leaves a dangling
image behind on every invocation, on every developer host and every CI
runner; a tagged one replaces the previous image.
**The gate builds write no image.** With `--output type=cacheonly` the phase
runs and a failing step fails the build, but the result is not exported.
Nothing uses those images, and writing one out is slow: a Go test phase's
image holds the toolchain. A build given neither `--output` nor `-t` writes
an untagged image and leaves it dangling, on every developer host and every
CI runner. `script/cibuild` and `script/docker` build the image that ships
and tag it, so each build replaces the previous image; each assigns the tag
on its own line before the build, so `set -e` stops it where
`script/projectname` fails.
Inside a phase the tool is invoked directly — `golangci-lint`, `go test`,
`eslint`, `prettier` — never through `make lint` or `script/test`, which are
@@ -147,22 +159,33 @@ style conventions are in separate documents:
one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs
exit non-zero with `parallel golangci-lint is running` — a status a caller
cannot tell from real findings. Both have produced wrong verdicts in this
org, in both directions. A container has its own cache, its own `TMPDIR` and
a digest-pinned binary, so neither is reachable.
org, in both directions. A container has its own golangci-lint cache, its
own `TMPDIR` and a digest-pinned binary, so neither is reachable.
- **Any build that runs checks is built with `--no-cache`.** Docker invalidates
a `COPY` layer only when the copied content changes, so on an unchanged tree
the check `RUN` is served from cache, nothing executes, and the build still
exits 0. Every `docker build` in `script/` therefore passes `--no-cache`:
`script/lint`, `script/test`, `script/cibuild` and `script/docker` are the
four, and there is no fifth — `script/check` runs the two gate phases and
`script/fmt-check`, and builds no image of its own. A bare `docker build .` is
not evidence that anything ran: a sub-second build reporting success is a
cache hit, not a result. Never invalidate by pruning — `docker builder prune`
and friends destroy a build cache shared with every other build on the host.
When a check is added or changed, prove it works by planting a defect it must
catch and watching the run fail on it, then revert the defect. A green run
alone shows neither that the check ran nor that it covers what it should.
- **Any build that runs checks is given a new `CACHEBUST` value.** Docker
invalidates a `COPY` layer only when the copied content changes, so on an
unchanged tree the check `RUN` is served from cache, nothing executes, and the
build still exits 0. Every `docker build` in `script/` therefore passes
`--build-arg CACHEBUST="$(date +%s)-$$"`, the time and the script's process
id, so no two builds share a value, not even the three in one
`script/cibuild`. `script/lint`, `script/test`, `script/cibuild` and
`script/docker` are the four, and there is no fifth — `script/check` runs the
two gate phases and `script/fmt-check`, and builds no image of its own. Every
stage of the `Dockerfile` declares `ARG CACHEBUST` as its first line after
`FROM`, and a new value reruns every `RUN` step after that line; `COPY` steps
are compared by content and need nothing. A stage without the `ARG` line is
served from cache on an unchanged tree, so a repository re-vendors the scripts
and adds the `ARG` lines in one commit. The scripts do not pass `--no-cache`:
it reruns every step too, but it also empties every cache mount, and Go's
build cache, which the canonical Go `Dockerfile` below mounts so that compiled
packages survive between check builds, would start empty every time. A bare
`docker build .` is not evidence that anything ran: a sub-second build
reporting success is a cache hit, not a result. Never invalidate by pruning —
`docker builder prune` and friends destroy a build cache shared with every
other build on the host. When a check is added or changed, prove it works by
planting a defect it must catch and watching the run fail on it, then revert
the defect. A green run alone shows neither that the check ran nor that it
covers what it should.
- **The gate phases are separate stages, and the build stage depends on both.**
The lint phase is based on the `golangci/golangci-lint` image (pinned by
@@ -171,32 +194,42 @@ style conventions are in separate documents:
`Dockerfile`:
```dockerfile
# Every stage declares ARG CACHEBUST first: script/ passes a new value
# on every build, which reruns every RUN step after it. The steps that
# compile mount Go's build cache, which survives from build to build.
# Lint phase
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
FROM golangci/golangci-lint@sha256:... AS lint
ARG CACHEBUST
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN golangci-lint run --config .golangci.yml ./...
RUN --mount=type=cache,target=/root/.cache/go-build \
golangci-lint run --config .golangci.yml ./...
# Test phase. -race needs cgo and so a C compiler, which the Debian Go
# image ships and the alpine one does not.
# image ships and the alpine one does not. -count=1 because the mounted
# cache also holds test results, which go test would otherwise replay.
# golang:1.x, YYYY-MM-DD
FROM golang@sha256:... AS test
ARG CACHEBUST
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN go test -timeout 90s -race -cover ./... || \
RUN --mount=type=cache,target=/root/.cache/go-build \
go test -count=1 -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \
go test -timeout 90s -race -v ./...; exit 1; }
go test -count=1 -timeout 90s -race -v ./...; exit 1; }
# Build stage. Nothing is wanted from either phase above; the copies
# are what make BuildKit build them first, so this stage cannot run
# unless lint and test passed.
# golang:1.x-alpine, YYYY-MM-DD
FROM golang@sha256:... AS builder
ARG CACHEBUST
COPY --from=lint /src/go.sum /dev/null
COPY --from=test /src/go.sum /dev/null
RUN apk add --no-cache git
@@ -212,7 +245,8 @@ style conventions are in separate documents:
# .git present, a version that is still empty, dev or unknown fails the
# build: git is missing or could not read the checkout.
ARG VERSION
RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \
RUN --mount=type=cache,target=/root/.cache/go-build \
VERSION="${VERSION:-$(git describe --tags --always)}"; \
if [ -e .git ]; then \
case "$VERSION" in ""|dev|unknown) \
echo "version is '$VERSION' although .git is present" >&2; \
@@ -225,6 +259,7 @@ style conventions are in separate documents:
# Runtime stage, and the last one
FROM alpine@sha256:...
ARG CACHEBUST
COPY --from=builder /app /usr/local/bin/app
ENTRYPOINT ["app"]
```
@@ -239,6 +274,16 @@ style conventions are in separate documents:
- Keep the runtime stage last, and if you add a stage after it, give it the
same two copies. A plain `docker build .` builds the last stage's chain
and nothing else.
- Only Go's build cache is mounted (`/root/.cache/go-build`, where Go keeps
it for root), and only on the steps that compile: `golangci-lint run`,
both `go test` runs and `go build`. It keeps compiled packages between
check builds, and Go reuses one only when all its inputs are unchanged, so
it saves the compile without skipping a check. Nothing else is kept:
golangci-lint's own cache would replay stored findings in place of a lint
run, and Go builds from whatever files a kept module cache holds without
checking them again (only `go mod verify` does), so a file changed there
would go unnoticed. Go's build cache also holds test results, which is why
both `go test` runs carry `-count=1`.
- If the project uses `//go:embed` directives that reference build artifacts
(e.g. a web frontend compiled in a separate stage), the lint phase must
create placeholder files so the embed directives resolve. Example:
@@ -278,20 +323,39 @@ style conventions are in separate documents:
`.git` and the version still comes out empty, `dev` or `unknown`. A plain
`docker build .` with no build arguments must succeed; a Dockerfile that
refuses an empty build argument drops that refusal and keeps the argument.
A checkout whose `.git` is a file (a linked worktree, or a repository
checked out as a submodule) is the exception: that file points to a git
directory outside the build context, so the build cannot read the version
and a plain `docker build .` fails; pass the version with
`--build-arg VERSION=...`, as `script/docker` and `script/cibuild` already
do.
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
runs `script/cibuild` on push, and checks out the repo as its only other step.
That script bootstraps, runs the gate phases, and then builds the image, so a
successful run means every check passed; a bare `docker build .` does not
runs `script/cibuild` on push, and checks out the repo as its only other step,
with `persist-credentials: false`: `script/cibuild` needs no token, and
without it the checkout leaves the job's token in `.git/config` for every
later step. The checkout step also sets `fetch-depth: 0`, which fetches the
tags `git describe` needs: by default it clones shallow with no tags, and a
tagged repository's CI build would stamp a bare short commit id. The
workflow's `concurrency` block groups runs by workflow and branch
(`${{ github.workflow }}-${{ github.ref }}`) with `cancel-in-progress: true`,
so a new push cancels the older run on the same branch, queued or running, and
no other: runs for replaced commits do not hold up the shared runner.
`script/cibuild` bootstraps, runs the gate phases, and then builds the image,
so a successful run means every check passed; a bare `docker build .` does not
carry the same guarantee, because its gate phases may come from the cache. The
image build is uncached and so runs the gate phases a second time. That is the
price of the rule above, and it is worth paying: the image that ships is built
from a run of its own gates rather than from a cache entry. A separate
workflow limited to `main` by a `branches` list under `on: push` cannot be
checked by review: to try a change to it, add the feature branch to that list
and push, then remove the branch from the list again before merging. Keep any
job in it that publishes behind `if: github.ref_name == 'main'`, so the run
from the feature branch publishes nothing.
image build gets a `CACHEBUST` value of its own and so runs the gate phases a
second time. That is the price of the rule above, and it is worth paying: the
image that ships is built from a run of its own gates rather than from a cache
entry. The `check` job sets `timeout-minutes: 20`, so a hung build frees the
shared runner after 20 minutes. That allows for the three Docker builds
described above (the test phase, the lint phase, then the image), each held to
the 5-minute Docker build limit below, plus the bootstrap. A separate workflow
limited to `main` by a `branches` list under `on: push` cannot be checked by
review: to try a change to it, add the feature branch to that list and push,
then remove the branch from the list again before merging. Keep any job in it
that publishes behind `if: github.ref_name == 'main'`, so the run from the
feature branch publishes nothing.
- Use platform-standard formatters: `black` for Python, `prettier` for
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
@@ -351,12 +415,11 @@ style conventions are in separate documents:
That cache is Go's own, separate from Docker's layer cache. Go stores a
passing result in its cache directory (`GOCACHE`), and when the same tests
run again on unchanged code it prints that result, marked `(cached)`,
without running them. That matters on a developer's machine, where this
target runs and the directory lasts from one run to the next. The `test`
phase of the `Dockerfile` needs no `-count=1`: its base image holds no
result for this repo's tests and nothing before its `go test` step runs a
test, so there is nothing to replay. `--no-cache` (above) is what makes that
step run on an unchanged tree.
without running them. The directory lasts from one run to the next on a
developer's machine, where this target runs, and in the `test` phase of the
`Dockerfile`, which mounts it to keep compiled packages between check
builds, so the phase's `go test` runs carry `-count=1` too. A new
`CACHEBUST` value (above) is what makes that step run on an unchanged tree.
Python example:
@@ -384,9 +447,12 @@ style conventions are in separate documents:
- `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`),
editor files (`.swp`, `*~`), in-repo agent scratch directories (`.claude/`),
language build artifacts, and `node_modules/`. Fetch the standard `.gitignore`
from `https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when
setting up a new repo. These patterns are written to `.gitignore`'s own
`node_modules/`, and the repo's own build outputs. Fetch the standard
`.gitignore` from
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when setting up
a new repo. A repo's `.gitignore` is the standard file followed by the repo's
own entries, such as its binaries; a re-vendor replaces the standard part and
keeps those entries. These patterns are written to `.gitignore`'s own
semantics, in which an unanchored pattern already matches at every depth; they
are not a `.dockerignore` and must not be transplanted into one unmodified.
@@ -434,13 +500,15 @@ style conventions are in separate documents:
byte-identically across repos:
```sh
# Own line: a failing command substitution inside an argument does not
# trip `set -e`, so the inline form degrades to an empty constant.
# The version and the tag each get their own line: a failing command
# substitution inside an argument does not trip `set -e`, so the inline
# form degrades to an empty constant.
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown"
docker build --no-cache \
tag="$(script/projectname)"
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--build-arg VERSION="$version" \
-t "$(script/projectname)" .
-t "$tag" .
```
`--always` makes an untagged repo yield an abbreviated commit hash rather
@@ -452,8 +520,8 @@ style conventions are in separate documents:
there because `ARG` is stage-scoped; passing `VERSION` to a repo whose
Dockerfile declares no such `ARG` is ignored and costs nothing, which is why
the scripts stay byte-identical. One consequence for CI: the standard
checkout action clones shallow and fetches no tags, so a repo that embeds a
tag-derived version must set `fetch-depth: 0` on its checkout step.
checkout action clones shallow and fetches no tags, so the canonical
`.gitea/workflows/check.yml` sets `fetch-depth: 0` on its checkout step.
- **Verify `.dockerignore` by enumerating the image, not by reading the
patterns.** Plant files at the root _and_ at least two directories deep, build
@@ -520,6 +588,11 @@ style conventions are in separate documents:
function defined and never invoked has the same exit status and the same
empty output as one that worked.
Node is the exception to the whole-token comparison: the canonical
`script/bootstrap` compares only its major version, because the node 22
alpine image the `Dockerfile` stages start from is not the exact pin, and
nvm cannot install a prebuilt node on alpine.
Keep it POSIX sh: no arrays, no `[[`, no `grep -P`.
A Go tool a repo needs on the host is installed with `go install` pinned to
@@ -636,7 +709,11 @@ style conventions are in separate documents:
Never edit existing migrations after release.
- All repos should have an `.editorconfig` enforcing the project's indentation
settings.
settings: the standard file from
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.editorconfig`, which sets
tabs for `Makefile` and Go files, followed by the repo's own sections, such as
one for another language it uses. A re-vendor replaces the standard part and
keeps those sections.
- Avoid putting files in the repo root unless necessary. Root should contain
only project-level config files (`README.md`, `AGENTS.md`, `Makefile`,
+9 -581
View File
@@ -2,599 +2,27 @@
- branch from `next`
- do the Next Step: the next open issue in the `1.0.0` milestone
- log it at the top of Completed Steps
- commit (`TODO.md` changes in the same commit as the work)
- push, and open a PR against `next`
# Status
pre-1.0. No git tags. Open work is tracked on the issue tracker, which is
authoritative.
pre-1.0. Open work is tracked on the issue tracker, which is authoritative.
# Next Step
Take the next open issue in the `1.0.0` milestone:
https://git.eeqj.de/sneak/secret/milestone/12
# Completed Steps
- 2026-10-14: A wrong passphrase, from `SB_UNLOCK_PASSPHRASE` or typed at the
prompt, gives the error `wrong passphrase for unlocker <id>`
(https://git.eeqj.de/sneak/secret/issues/157). It used to be age's "no
identity matched any of the recipients", followed by advice to give the vault
a new unlocker from its mnemonic, which would have replaced an unlocker that
works. The error now wraps `secret.ErrWrongPassphrase` and carries no advice
to use the mnemonic. A `priv.age` that is not an age file still gets that
advice.
- 2026-10-14: The passphrase and mnemonic prompts ask on the controlling
terminal (`/dev/tty`) instead of stdin, so `echo v | secret add s1 --force`,
`cat file | secret encrypt key` and `cat file.age | secret decrypt key` ask
for the passphrase at a terminal instead of failing
(https://git.eeqj.de/sneak/secret/issues/144). Only a process with no
controlling terminal gets the error that names the variable to set. On SIGINT
or SIGTERM, `Entry()` restores the settings of that terminal, not of stdin,
since that is where the prompt turns echo off. `secret init` and
`secret vault create` no longer write a newline to stderr after the mnemonic
is typed, since the prompt ends that line on the terminal.
- 2026-10-14: `secret generate mnemonic`, `secret vault list` and
`secret unlocker list` print their output to stdout, as `secret list` does, so
it can be piped or captured (https://git.eeqj.de/sneak/secret/issues/138).
They printed with cobra's `cmd.Println` and `cmd.Printf`, which write to
stderr when no output writer is set, so `secret vault list --json | jq`
received nothing. "No versions found" from `secret version list` moved to
stdout too. Messages such as "Created vault", and the warning after the
mnemonic, stay on stderr.
- 2026-10-14: Removing the current unlocker no longer leaves the vault unable to
open through the unlockers it still has
(https://git.eeqj.de/sneak/secret/issues/137). `current-unlocker` went on
naming the removed unlocker, so every command that opened the vault failed,
with advice to use the mnemonic. `secret unlocker remove` of the current
unlocker now makes the first of the vault's other unlockers that
`secret unlocker select` accepts, in the order `secret unlocker list` shows,
current before removing it, and says so; when there is none, it also removes
`current-unlocker`. `vault.RemoveUnlocker` returns the ID of the unlocker it
made current. When `current-unlocker` names an unlocker that does not exist
and the vault has another that `secret unlocker select` accepts, the error
wraps `vault.ErrUnlockerNotFound` and says to run `secret unlocker select`,
without the advice to use the mnemonic.
- 2026-10-14: A `secret add --force` that cannot unlock the vault, given a wrong
passphrase or mnemonic, leaves the secret as it was
(https://git.eeqj.de/sneak/secret/issues/136). The new version used to be
written before the vault was unlocked to set `notAfter` on the version it
replaces, so each failed add left one more version that `secret version list`
showed. The vault is now unlocked, and that version's metadata read, before
anything is written. `generate secret --force` and `import --force` take the
same path.
- 2026-10-12: A mnemonic typed with extra spaces, tabs or line breaks gives the
same long-term key as its words separated by single spaces
(https://git.eeqj.de/sneak/secret/issues/135). The key was derived from the
text exactly as typed, so a vault made with a double or trailing space did not
open with the same words typed normally. `agehd.DeriveEntropy`, which every
use of the mnemonic goes through, now joins the words with single spaces and
puts them in Unicode NFKD, as BIP39 defines, before making the seed.
- 2026-10-12: The keychain and Secure Enclave unlockers are gone from the code
and the documents (https://git.eeqj.de/sneak/secret/issues/132). They were
research: neither works without macOS code signing and entitlements, which
`secret` does not have. The last commit that has them is `ef0ae90`, tagged
`v1.0.0-rc1`. `secret unlocker add` takes only `passphrase` and `pgp` on every
platform, `internal/macse` and those unlockers' files in `internal/secret` are
deleted, and `github.com/keybase/go-keychain` left `go.mod` and `go.sum`.
`secret` still builds and runs on macOS.
- 2026-10-07: The part of `github.com/tyler-smith/go-bip39` v1.1.0 that `secret`
uses is copied into `internal/bip39`, with upstream's `LICENSE` beside it,
because its repository no longer exists
(https://git.eeqj.de/sneak/secret/issues/122). Every import moved there, and
the module left `go.mod` and `go.sum`. No derived key or mnemonic changes.
- 2026-10-07: The canonical files are re-vendored from `sneak/prompts` commit
`dd4027b` (https://git.eeqj.de/sneak/secret/issues/121), with golangci-lint
v2.14.0 in the lint phase. Lint and test are phases of the `Dockerfile`, and
`script/lint` and `script/test` each build one with `--no-cache`;
`Dockerfile.lint` and `script/lint-darwin` are gone, and the lint phase runs
`go vet` and checks the macOS build too. The tests run on the Debian Go image
with cgo and the race detector, with the policy's 90-second timeout.
`script/cibuild` bootstraps, runs `script/check` and builds the image;
`CHECK_EPOCH` and the memlock ulimit are gone. `--no-cache` starts Go's build
cache mount empty, so `make test` compiles everything on every run. The rules
in `CLAUDE.md` that `AGENTS.md` lacked are in `AGENTS.md`, and `CLAUDE.md` is
deleted.
- 2026-10-06: `script/test` runs the tests with the race detector, a 30-second
timeout per package and coverage, as `REPO_POLICIES.md` requires
(https://git.eeqj.de/sneak/secret/issues/32). When they fail, it reruns them
verbosely for the details and then fails anyway, so a test that fails once and
passes on the retry no longer gives a green build. `go vet` still runs first,
and every `go test` keeps `-count=1`. The tests that gave the `secret` binary
a minute now give it 10 seconds, and the PGP unlocker test's 30-second timer
is 10 seconds, so a test that hangs fails with its own message before the
package's 30-second timeout ends every test in it.
- 2026-10-06: `make test` in `script/cibuild` no longer compiles the standard
library and every dependency from nothing on every build
(https://git.eeqj.de/sneak/secret/issues/124). The `Dockerfile` runs it and
`make build` with Go's build cache in a BuildKit cache mount, which docker
keeps between builds, so each compiles only what changed since the last build.
The mount has an id of its own, so other repositories' builds do not share it.
`script/test` passes `-count=1`, so every test runs on every build and no
result comes from Go's test cache. A build with an empty cache, such as the
first after docker's build cache is cleared, compiles everything in
`make test` as before.
- 2026-10-06: `TestRemoveIgnoresTerminalOnStdout` and
`TestRemoveAsksAtTerminalOnStdin` no longer wait until Go's test timeout
(https://git.eeqj.de/sneak/secret/issues/126). On Linux, `pty.Open` of
`github.com/creack/pty` v1.1.24 passed the address of a local variable to the
`ioctl` system call as a plain number, through a function call; when Go moved
the goroutine's stack in between, the kernel wrote the terminal's number to
the old place, and `pty.Open` opened `/dev/pts/0` instead of the terminal it
had created. `secret rm` then wrote to that other terminal, and the test read
a terminal no program had open, which never ends. `go.mod` now requires the
commit on that library's main branch that passes a pointer instead; no release
has it yet. Both tests stop reading the terminal when their one-minute context
ends and fail saying so.
- 2026-10-06: The tests run quickly with the race detector on
(https://git.eeqj.de/sneak/secret/issues/120). Most of their time went to
deriving keys from passphrases with scrypt, which is slow on purpose. The new
`secret.ScryptWorkFactor`, when not zero, replaces age's scrypt work factor
when a passphrase encrypts; the tests of `internal/secret`, `internal/vault`
and `internal/cli` set it to 1 before any test runs, and the program never
sets it; `TestGetCommandOutputsToStdout` checks that the passphrase unlocker
the built binary's `secret init` writes names age's work factor, 18.
`TestRemovalAsksWithoutHoldingLock` and `TestFailedCommandReleasesLock` no
longer run in parallel with other tests: each waits at most 10 seconds for the
in-memory lock that every test in the package shares, and other tests'
commands held it longer. `TestConcurrentAddsKeepEveryVersion`, which times
nothing, and `TestGetCommandOutputsToStdout`, which no longer sets an
environment variable its commands do not read, now run in parallel. The
`script/cibuild` comment no longer says that tests are skipped without its
memlock ulimit.
- 2026-10-05: No test stores a secret larger than 1 MiB
(https://git.eeqj.de/sneak/secret/issues/52). The size tests for `secret add`,
`secret import` and the stdin buffer no longer try 2 MB, 10 MB, 99 MB, 100 MB
or 101 MB secrets, and nothing tests that a secret over the 100 MB limit is
rejected; the limit itself is unchanged. With nothing large left, the size
tests no longer skip a case for want of locked memory.
- 2026-10-05: The Go module path is `sneak.berlin/go/secret`, as
`REPO_POLICIES.md` requires, not `git.eeqj.de/sneak/secret`
(https://git.eeqj.de/sneak/secret/issues/43). Every import uses it, as do the
`-X` flags in `script/build` that stamp the version and commit shown by
`secret info`, and the examples in `pkg/agehd/README.md` and
`pkg/bip85/README.md`. `go mod tidy` now lists `github.com/dustin/go-humanize`
and `github.com/fatih/color`, which `internal/cli` imports, as direct
requirements. Code that imported the old path must switch to the new one.
- 2026-10-04: `make fmt` formats every markdown file with prettier (4-space
tabs, `proseWrap: always`) as well as the Go code, and `make fmt-check` checks
both, as the model scripts in the `prompts` repo do
(https://git.eeqj.de/sneak/secret/issues/110). Prettier is pinned by hash in
`package.json` and `yarn.lock`, and `script/bootstrap` installs node, yarn and
prettier. The `Dockerfile` lint stage copies node and yarn from a node image
pinned by hash and runs `script/bootstrap`, so its `make fmt-check` fails the
build on an unformatted markdown file. Every markdown file was formatted once,
wording unchanged.
- 2026-10-04: A mnemonic that cannot be read, in `secret init` and
`secret vault create`, gives an error that names the mnemonic only
(https://git.eeqj.de/sneak/secret/issues/115). It is read with
`secret.ReadMnemonic`, whose every error wraps the new
`secret.ErrMnemonicNotRead`; before, it was read with `ReadPassphrase`, so the
message said "failed to read mnemonic: failed to read passphrase:" and advised
setting `SB_UNLOCK_PASSPHRASE`. Without a terminal it now says "failed to read
mnemonic: stdin is not a terminal (piped input or script). Please set the
SB_SECRET_MNEMONIC environment variable or run interactively". The passphrase
messages no longer repeat "cannot read passphrase" after "failed to read
passphrase:", and empty input gives "nothing was entered".
- 2026-10-04: A failure returns the same error value whichever command hits it
(https://git.eeqj.de/sneak/secret/issues/113). `internal/cli` no longer keeps
its own copies of `vault.ErrSecretNotFound`, `ErrVaultNotFound`,
`ErrVersionNotFound` and `ErrSecretExists`: `secret mv`, `rm`, `decrypt`,
`vault import`, `vault remove` and `version list`, `promote` and `rm` wrap the
`vault` errors. `errUnsupportedUnlockerType` is removed: `secret unlocker add`
gives `errInvalidUnlockerType` for an unknown type, whichever check rejects
it. Off macOS, adding a keychain or Secure Enclave unlocker returns the
`secret` package's error for it, not an `internal/cli` copy; on macOS, the
check that the system is macOS is gone, as it could never fail.
`secret vault import` gives `errInvalidMnemonicPhrase` for an invalid
mnemonic, as `init` and `vault create` do. `secret generate secret` gives
`errLengthTooSmall` for a length below 1 wherever it is checked, and
`errUnsupportedSecretType` for `--type mnemonic` too. `secret import` of a
file over 100MB wraps `errSecretTooLarge`, as `secret add` returns it.
`vault.ErrNilValueBuffer` is replaced by `secret.ErrNilValueBuffer`, which
`secret` already returned under another name. Messages are unchanged, except
that `secret decrypt` of a missing secret says "not found", as `secret get`
does, not "does not exist"; `vault import` of an invalid mnemonic says
"invalid BIP39 mnemonic phrase"; `--type mnemonic` says "unsupported type:
mnemonic (use 'secret generate mnemonic' instead)"; and a file too large to
import says
`failed to read secret from file <path>: secret too large: exceeds 100MB limit`.
Every error of `secret.ReadPassphrase` wraps `secret.ErrPassphraseNotRead`,
which supplies the words "failed to read passphrase" that its callers used to
add themselves; so two passphrases that differ now give only "passphrases do
not match", the words now follow "failed to read mnemonic:" and "failed to
read passphrase confirmation:", and a terminal read error no longer repeats
them. A GPG key the keyring does not hold gives `secret.ErrGPGKeyNotFound`,
found by gpg's status line for "No public key"; before, the message repeated
"failed to resolve GPG key fingerprint" and ended in gpg's exit status. The
keychain unlocker returns `errNilDataBuffer` for nil data; this and its test
build only on macOS with cgo and were only read. `bip85.ErrPasswordTooShort`
and `ErrEncodedTooShort` are removed with their checks: 64 bytes of entropy
always give 86 Base64 or 80 Base85 characters, the most a password length may
ask for. Tests that matched these errors' text use `errors.Is`.
- 2026-10-04: Tests check which error a failure returns with `errors.Is`, not by
matching words of its message (https://git.eeqj.de/sneak/secret/issues/49).
Every exported error that can be returned has a test that the function returns
it, and errors wrapping a cause are checked through the wrapping. Checks that
still match text, because the error has no exported value the test can name,
are listed on the issue.
- 2026-10-04: When a vault cannot be opened through its current unlocker,
because a file the unlocker needs is missing or damaged, its keychain item or
Secure Enclave key is gone, or the passphrase is wrong, the error now ends by
naming the vault, saying that it still opens with its mnemonic, and that
`secret unlocker add passphrase`, run with `SB_SECRET_MNEMONIC` set to it,
gives the vault a new unlocker; for a vault that is not the current one, as in
`secret move` between vaults, it says to run `secret vault select` first
(https://git.eeqj.de/sneak/secret/issues/47). Before, it ended with the bare
cause. The advice is given only when the vault metadata records the key the
mnemonic derives, so not for a vault created without a mnemonic, and not when
the passphrase could not be read at all. `secret vault import` is not named:
it refuses a vault that has a long-term key. `secret encrypt` and
`secret decrypt` now read the key secret through `vault.GetSecret`, as
`secret get` does, so they give the same advice; `Secret.GetValue`, the other
way to get the long-term key, is removed. When a secret's `current` file
cannot be read, the error says that `secret version list` lists its versions
and `secret version promote` makes one current. The causes stay wrapped.
- 2026-10-04: An unlocker's ID is the name of its directory in `unlockers.d`, so
no two unlockers of a vault share one
(https://git.eeqj.de/sneak/secret/issues/98). Before, a keychain or Secure
Enclave unlocker's ID was its creation time to the minute and the host name,
and a passphrase unlocker's the time to the minute, so two created within a
minute shared an ID, and `unlocker select`, `unlocker remove` and the
selection `unlocker add` makes acted on the older one. A PGP unlocker's ID was
`pgp-` and its key's fingerprint; a second PGP unlocker for a key is still
refused, now by comparing the fingerprint in the other unlockers' metadata.
`unlocker list` and the shell completion of `unlocker select` and
`unlocker remove` take each ID from the directory the unlocker was read from,
no longer by matching metadata, so two unlockers with the same metadata are
listed apart; an unlocker of an unknown type is listed under its directory
name, and completion now offers Secure Enclave unlockers too. The keychain and
Secure Enclave code was type-checked by `script/lint-darwin`, never run; a
test on Linux lists, completes, selects and removes each of two passphrase
unlockers with the same metadata by its own ID.
- 2026-10-04: README's Storage Architecture, `secret version promote`, Technical
Details and Testing text matches the code
(https://git.eeqj.de/sneak/secret/issues/102). `current` and `currentvault`
are plain files holding a name, not symbolic links; a version's metadata is
the encrypted `metadata.age`; the state directory is `berlin.sneak.pkg.secret`
in the user's configuration directory, not `~/.local/share/secret`, and holds
the `lock` file. Also corrected: the code sets up no Touch ID for the keychain
or Secure Enclave unlocker, and the Secure Enclave only decrypts; per-version
keys give no forward secrecy; `pub.age` is not age-encrypted; vault metadata
holds no vault name. Testing lists only `make test`.
- 2026-10-04: `secret init` and `secret vault create` create a vault whole or
not at all (https://git.eeqj.de/sneak/secret/issues/105). `vault.CreateVault`
now takes the unlocker passphrase too, writes the vault directory with its
metadata, long-term public key and passphrase unlocker, `longterm.age`
included, into a temporary directory, renames that into `vaults.d` once it is
complete, and only then makes the vault current. Before, either command killed
after the passphrase prompt but before the unlocker was written left a vault
with no unlocker, which `vault create` had already made current and which
neither command would create again. Killed part-way now, it leaves no vault,
and the next command that takes the lock deletes the temporary directory; or,
killed between the rename and making the vault current, a complete vault that
is not current, which `secret vault select` makes current.
- 2026-10-04: A failed `secret unlocker add keychain` or
`secret unlocker add secure-enclave` no longer leaves its keychain item or
Secure Enclave key behind (https://git.eeqj.de/sneak/secret/issues/89).
`CreateSecureEnclaveUnlocker` gets the long-term key before it creates the
Secure Enclave key, so that a wrong passphrase creates none, and deletes the
key again if encrypting with it or writing the unlocker then fails.
`macse.CreateKey` finds the new key's hash right after `sc_auth` creates it,
and fails with an error naming the key's label if it cannot; it deletes the
key again if getting its public key then fails. The Objective-C was only read,
never compiled or run, and so was `macse_darwin.go`, which is cgo only.
`CreateKeychainUnlocker` writes all of the unlocker's files, the metadata
among them, before it stores the item in the keychain, and deletes the item
again if moving the unlocker into place then fails. A failure to delete is
reported along with the first error. The tests of this run only on macOS: the
Secure Enclave one in a build with cgo on a Mac with a Secure Enclave, the
keychain one in a build with cgo.
- 2026-10-04: What a command killed part-way left under a `.tmp-` name
(https://git.eeqj.de/sneak/secret/issues/75), the temporary directories of
`secret.TempDirFor` and the temporary files of `secret.WriteFileAtomic`,
encrypted keys included, is deleted by the next command that takes the state
directory lock. Before, it stayed until deleted by hand. A command writes
`finished` into the lock file just before it releases the lock; the next one
to take the lock searches only when it does not find that, so after a command
that finished nothing is searched, however many secrets and versions there
are. The search looks in the state directory, each vault, each secret and each
version, the only directories those helpers make them in. A command that only
reads takes no lock and deletes nothing. A failure to delete is warned about
and the command goes on. An unlocker directory with no metadata file was
already removed by `secret unlocker remove` given its directory name; a test
now shows it.
- 2026-10-04: An age identity's private key goes into a locked buffer through
`secret.IdentityToLockedBuffer` everywhere
(https://git.eeqj.de/sneak/secret/issues/38): the vault's long-term key when a
passphrase, PGP, keychain or Secure Enclave unlocker is created, the new
unlocker's own key, a new secret version's key, and the key `secret encrypt`
generates. Before, each place converted the string age returns to bytes and
left the string in ordinary memory. The function moves the string's own bytes
into the buffer, which overwrites them; the copies age makes while writing the
string remain, as its comment says. The 1.0 memory-security entry below no
longer lists these places, `internal/cli/crypto.go` among them, nor
`version.go:155`, which was `internal/secret/version.go`, not
`internal/cli/version.go`.
- 2026-10-04: `script/lint-darwin` (`make lint-darwin`) runs `go vet` and
`golangci-lint` in docker on the code as a macOS build compiles it
(`GOOS=darwin`), with cgo off (https://git.eeqj.de/sneak/secret/issues/50).
`script/check` runs it, and the `Dockerfile` lint stage runs its commands, so
`script/cibuild` does too. Before, CI on Linux never compiled the files built
only for macOS. Compiling cgo code for macOS needs Apple's SDK headers, and
both `internal/macse` and `github.com/keybase/go-keychain` are cgo on macOS.
So the three functions that call `go-keychain` moved from
`keychainunlocker.go` to `keychainunlocker_cgo.go`, built only with cgo on
macOS like `macse_darwin.go`. A macOS build without cgo, which before did not
compile, gets `keychainunlocker_nocgo.go` and the `macse` stub instead, whose
errors say the keychain or Secure Enclave needs a macOS build with cgo. The
check covers the rest of the keychain unlocker, the Secure Enclave unlocker
and the macOS-only tests other than `keychainunlocker_test.go`, whose lint
findings are fixed. For the length and complexity limits, parts of
`GetIdentity`, `getLongTermPrivateKey` and `CreateKeychainUnlocker` moved into
functions of their own, and the Secure Enclave unlocker derives the long-term
key from the mnemonic through the same function as the keychain unlocker
instead of a copy of it. Lines over 88 columns in the files the check cannot
see are wrapped.
- 2026-10-04: `secret rm`, `secret version rm`, `secret vault remove` and
`secret unlocker remove` ask `[y/N]` before removing anything
(https://git.eeqj.de/sneak/secret/issues/39), naming what they remove: the
secret, its vault and its version count; the version, secret and vault; the
vault and its secret count; the unlocker, its vault and whether it is the
last, and for the last the vault's secret count and that the vault then opens
only with its mnemonic. Only `y` or `yes` goes ahead. Without `--force`, a
command whose stdin is not a terminal fails at once. `--force` (now also on
`rm` and `version rm`) removes without asking; it replaces the old refusals to
remove a vault with secrets or the last unlocker of one without `--force`,
which the question now covers. The checks run, and the question is asked,
before the state directory lock is taken; under the lock the checks run again,
and if they would ask a different question, nothing is removed. `secret rm`
fails when it cannot count the versions.
- 2026-10-04: A crash while an unlocker is being replaced no longer leaves a
current unlocker that cannot open the vault
(https://git.eeqj.de/sneak/secret/issues/71). Every new unlocker gets a
directory of its own, named with the time to the nanosecond:
`passphrase-<time>`, `<host>-pgp-<time>`, and for a keychain or Secure Enclave
unlocker the keychain item or Secure Enclave key, which names the directory,
carries the time instead of the day. `secret.WriteDir` fails on a directory
that exists instead of writing into it. `unlocker add passphrase` writes the
new unlocker, makes it current, and only then removes the vault's other
passphrase unlockers; a crash between the last two steps leaves the old one
beside the new, and the old passphrase still opens the vault through it until
the next `unlocker add passphrase` or an `unlocker remove` removes it. A PGP,
keychain or Secure Enclave unlocker added on the same host and day as another
of its type is added beside it instead of replacing it.
- 2026-10-04: `SB_SECRET_MNEMONIC` and `SB_UNLOCK_PASSPHRASE` are read once per
command, in its `RunE`, into locked buffers on the CLI `Instance`, and unset
at once, so that no program the command runs, `gpg` included, inherits them
(https://git.eeqj.de/sneak/secret/issues/60). Nothing below the command reads
the environment; the buffers are passed down: `vault.CreateVault` takes the
mnemonic (nil for none), a `Vault` derives its long-term key from its
`Mnemonic` and gives its `UnlockPassphrase` to a passphrase unlocker, and the
PGP, keychain and Secure Enclave unlocker constructors take both.
`CreatePGPUnlocker` sets both on the vault it loads, through `SetMnemonic` and
`SetUnlockPassphrase`, now part of `VaultInterface`, before calling its
`GetOrDeriveLongTermKey`. `init` and `vault create` no longer put the mnemonic
into the environment. Unsetting erases nothing: the starting environment
(`/proc/<pid>/environ`) and memory still hold the value. The README warns
against both variables.
- 2026-10-04: `.golangci.yml` is again the canonical file from `sneak/prompts`,
byte for byte (https://git.eeqj.de/sneak/secret/issues/66). It runs
`gomodguard_v2` in place of the deprecated `gomodguard`, so the lint no longer
warns, and enables `depguard` with a rule that keeps `net/http/httptest` out
of non-test files. Neither raised a finding in this repo.
- 2026-10-04: `secret unlocker add pgp` works on Linux
(https://git.eeqj.de/sneak/secret/issues/88). `CreatePGPUnlocker` gets the
vault's long-term key as adding a passphrase unlocker does, with the vault's
`GetOrDeriveLongTermKey`, now part of `VaultInterface`: from the mnemonic,
checked against the vault, or else from the current unlocker. Before, it used
the keychain unlocker's helper, which on every platform but macOS always
failed. A test adds a PGP unlocker for a throwaway GPG key, getting the
long-term key once from the mnemonic and once from a passphrase unlocker, and
reads a secret through the new unlocker.
- 2026-10-04: A vault name may use only lowercase ASCII letters, digits, `.`,
`-` and `_`, and must not be empty, `.` or `..`
(https://git.eeqj.de/sneak/secret/issues/68); the error and `README.md` state
the rule. `vault create`, `vault import`, `vault select`, `vault remove`, both
vault names of `mv` and shell completion of a `vault:secret` argument check
the name as typed with `vault.ValidateVaultName` before building any path from
it. Before, `vault import ..` wrote a long-term key and an unlocker into the
state directory itself, and `vault select ..` made that the current vault.
- 2026-10-04: `script/cibuild` runs the checks again on an unchanged tree
(https://git.eeqj.de/sneak/secret/issues/54). It passes the current time as
the `CHECK_EPOCH` build argument, which both the lint and the build stage of
the `Dockerfile` declare after their module download, so the `RUN` steps below
the argument run again on each build while the base images and module
downloads stay cached. Before, a second run on the same tree took every check
from the build cache and reported success having run nothing.
- 2026-10-04: A failed unlocker add no longer leaves a partial unlocker
directory (https://git.eeqj.de/sneak/secret/issues/48).
`secret unlocker add pgp` resolves the GPG key's fingerprint once, for its
duplicate check, and passes it to `CreatePGPUnlocker` to record.
`CreatePGPUnlocker` and `CreateKeychainUnlocker` get the long-term key and
encrypt everything before writing anything. All four unlocker types write
their files through `secret.WriteDir`: a new unlocker is built in a temporary
directory, renamed into place when complete and removed on a failure.
- 2026-10-04: `secret unlocker select` and `secret unlocker remove` skip, with
the warning `unlocker list` gives, an unlocker directory whose metadata file
cannot be checked for, read or parsed, instead of failing when it sorts before
the unlocker asked for. Such a directory, or one without a metadata file, is
removed by its directory name, the name the warning gives; only the directory
is removed, since its type is unknown. Removing one whose metadata file is
missing or corrupt never counts as removing the last unlocker. Removing one
whose metadata file cannot be checked for or read always does, since it may be
the only working unlocker, so in a vault with secrets it needs `--force`.
- 2026-10-04: A failed command prints its error once, without the usage text
after it (https://git.eeqj.de/sneak/secret/issues/41). Usage is still printed
for a command called wrongly: wrong number of arguments, unknown flag, bad
flag value, missing required flag, or flags that break a flag group (mutually
exclusive, required together, one required). The root command's
`PersistentPreRunE` turns usage off. Cobra checks arguments and flag values
before that hook but required flags and flag groups only after it, so the hook
checks those two first. Root `SilenceUsage` would have hidden usage for all of
these.
- 2026-10-04: `secret get` keeps the secret in locked memory until it writes it
out (https://git.eeqj.de/sneak/secret/issues/37): `Vault.GetSecret` and
`Vault.GetSecretVersion` return a `*memguard.LockedBuffer`, which every caller
destroys, and `secret get` writes its bytes straight to stdout, still with no
trailing newline. Before, the value was copied into ordinary memory that
nothing wiped, and `get --version` also wrote it to the debug log.
- 2026-10-04: The `Makefile` no longer sets `DOCKER_HOST`, so its docker targets
use the local docker daemon, or whatever `DOCKER_HOST` the environment sets.
`make build` calls the new `script/build`, which stamps the version (`VERSION`
from the environment, else `git describe`) and the git commit as before.
`build`, `clean`, `install` and `docker-run` are in `.PHONY`; `make install`
depends on `build`. The `vet` target is gone: `script/test` runs `go vet`
first.
- 2026-10-04: `.gitignore` is the org's standard file, which ignores `.env`,
`.env.*`, `*.pem` and `*.key` and editor and OS files, plus this repo's
`/secret`, `*.log`, `*.test` and `settings.local.json`
(https://git.eeqj.de/sneak/secret/issues/40). `.dockerignore` also leaves out
`node_modules`; `.git` stays in the build context for the version stamp.
- 2026-10-04: `secret init` refuses when the default vault exists, and
`secret vault create NAME` when `NAME` does, with "vault NAME already exists",
before writing anything. The check is in `vault.CreateVault`, which both
commands call while holding the state directory lock, so two creates of one
vault at once cannot both pass the check. Before, either command replaced the
vault's metadata, passphrase unlocker and `longterm.age`, so none of its
secrets could be decrypted any more. Both commands now ask for the unlocker
passphrase before creating the vault, so one stopped at that prompt leaves no
vault behind.
- 2026-10-04: The `internal/cli` tests are back to about their time before the
state directory lock (https://git.eeqj.de/sneak/secret/issues/80). The test
that each changing command waits for the lock releases it as soon as it sees
the command waiting there, instead of after a fixed 100 ms. The two vaults
with passphrase unlockers that the path and move tests start from are made
once and copied for each test.
- 2026-10-04: `secret mv` rejects a move whose destination is the source under
another name, such as `foo` for `Foo` on a case-insensitive filesystem (the
macOS default) or a name reached through a symbolic link, before changing
anything, with or without `--force`, within a vault and between vaults;
before, `--force` removed the destination and so deleted the secret. A rename
that changes only letter case works on a case-sensitive filesystem as before.
- 2026-10-04: Lint runs only in docker: `script/lint` builds `Dockerfile.lint`,
where golangci-lint is a build step rebuilt on every run
(`--no-cache-filter`), so an unchanged tree is linted too; the module download
stays cached. `script/bootstrap` no longer installs golangci-lint, and the
`Dockerfile` lint stage calls it directly instead of `make lint`.
`golangci-lint config verify` is not run: it fetches its schema live over
unpinned HTTPS.
- 2026-10-04: A PGP unlocker whose metadata has no usable GPG key ID no longer
panics: `GetID()` warns with the unlocker's directory and returns
`pgp-unknown`. `ListUnlockers` skips, with a warning, an unlocker whose
metadata file cannot be checked for, read or parsed instead of failing, so
`secret unlocker list` still lists the others; the listing's ID lookup no
longer warns about that directory again.
- 2026-10-03: `secret mv` rejects a move whose destination is the source
(`mv --force x x`, `mv --force work:x work:`, or an empty destination, which
defaults to the source name) before changing anything; before, `--force`
removed the destination first and so deleted the secret. Every vault name
given with `vault:` must be one of the existing vaults by exact name, so
`work:x work/:x` is rejected instead of being taken for a move between two
vaults. A move within a named vault no longer makes that vault the current
one, whether it succeeds or fails.
- 2026-10-03: Commands that change the state directory hold one lock (`flock` on
`lock` in the state directory; a mutex on the in-memory test filesystem), so
concurrent commands no longer lose versions or race on the current pointers.
Every file is written through `secret.WriteFileAtomic` (temporary file, sync,
rename), so no file is ever half-written and `current`, `currentvault` and
`current-unlocker` never go missing. New versions, new secrets and cross-vault
copies are built in a temporary directory and renamed into place, and removals
rename out of the way first, so a version or secret is never half-added and
never half-removed.
- 2026-10-03: The checks run before changing a vault now stop with an error
naming the path and cause when they cannot read what they inspect, instead of
reading the failure as "nothing there": the duplicate check before
`unlocker add pgp` (an unreadable `unlockers.d` or unlocker metadata file),
the secret count that guards removing the last unlocker and removing a vault,
and the existing long-term key check before `vault import`.
- 2026-10-03: `version rm`, `version promote` and `get --version` accept a
version only if it is one of the versions `version list` lists for that
secret, compared as typed before any path is built (`secret.VersionExists`),
and touch nothing otherwise. An empty `--version` is rejected instead of
meaning the current version. Before, `secret version rm x ../../..` deleted
the whole vault, `secret version rm x ..` the secret, and `.` or `""` every
version.
- 2026-10-03: Key material is wiped on every exit: `Entry()` returns the exit
code after its deferred `memguard.Purge()` has run, and only `main` calls
`os.Exit`. SIGINT and SIGTERM go through memguard's handler, which wipes every
buffer before exiting; when the process is in the terminal's foreground
process group it first restores the terminal settings from startup, so an
interrupted passphrase prompt no longer leaves echo off.
- 2026-10-03: Every command that builds a path from a secret name checks the
name first with `vault.ValidateSecretName` and touches nothing when it is
invalid: `rm`, `mv` (both names, within a vault and between vaults, before
switching the current vault), `import`, `version list`/`promote`/`rm`,
`encrypt` and `decrypt`. The error and `README.md` state the naming rule.
Before, `secret rm ..` deleted the whole vault and `secret rm .` every secret
in it.
- 2026-10-03: The keychain unlocker's age key passphrase stays in locked memory:
it is generated into a locked buffer, and the keychain JSON is written and
read by `KeychainData` code in `internal/secret/keychaindata.go` (tested on
Linux) without `encoding/json` holding it; the JSON field names are unchanged.
- 2026-10-02: A plain `docker build .` builds again: the size tests skip a case
that needs more locked memory than the process can lock, and run every case
under `script/cibuild`. The image stamps the `VERSION` build argument, else
`git describe --tags --always`, into `Version`, and fails if `.git` is present
but yields no version; `make build` stamps `git describe` too, not a fixed
`0.1.0`. `.dockerignore` keeps `.git/config` out; `script/docker` is the
canonical copy.
- 2026-08-07: Updated golangci-lint to v2.12.2 with the canonical
`.golangci.yml` (all linters enabled minus the standard disable list, `lll`
88, tests linted); bumped the `Dockerfile` lint-stage image to the tagged
v2.12.2 Debian digest; fixed all ~1550 new findings across `internal/` and
`pkg/` (line wrapping, `wsl_v5` blank lines, sentinel errors for `err113`,
`t.Parallel()` where safe, `_test` package conversions, complexity/`dupl`
helper extraction) on branch `golangci-v2.12.2`. Reworked after review: the
`err113` sentinels in `internal/vault`, `internal/secret`, `internal/cli` and
`pkg/bip85` were reshaped so every composed error message is byte-identical to
`main`, and `findUnlockerIDByMetadata` now returns an error so `unlocker list`
skips an unreadable `unlockers.d` entry with a warning instead of emitting a
fabricated fallback ID.
- 2026-08-07: Added `.editorconfig`
(https://git.eeqj.de/sneak/secret/issues/27).
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints, Makefile
shims, README Entrypoints section
- 2026-07-07: Added `REPO_POLICIES.md` and the `make hooks` target;
`.gitea/workflows/check.yml` now runs `script/cibuild`.
- 2026-03-30: Added the `make fmt-check` target and
`.gitea/workflows/check.yml`, which runs `docker build` on every push; the
`Dockerfile` base images are pinned by sha256.
- 2026-03-11: Secure Enclave unlocker for hardware-backed secret protection,
plus review fixes (stub panics, derivation index, tests, README) on branch
secure-enclave-unlocker.
- 2026-02-28: Repo cleanup, removed stale .cursorrules and coverage.out.
- Audit fix wave (issues #1, #2, #3, #13, #14): skip unlockers with missing
metadata, allow uppercase secret names, fix hardcoded derivation index,
validate names in GetSecretVersion against path traversal, return errors
instead of panicking, add Warn() on silent anomalies.
- Memory security hardening: LockedBuffer used through encrypt/decrypt paths
(Save/EncryptWithPassphrase/GetValue/gpg helpers), deprecated bare-[]byte APIs
removed.
- Per-secret keypair architecture, vault package refactor, versioning with
--version, comprehensive test suite with in-memory filesystem.
- Debug logging system (slog, GODEBUG flag, TTY-aware output).
- Renamed SEP unlocker to Keychain, reorganized import commands.
- 2025-05-28: Initial implementation (vault, age encryption, mnemonic, CLI).
# Future Steps
- Implement version-number shell completion for the second arg of
`secret version promote` and `secret version rm` (`internal/cli/version.go`;
was an in-code TODO removed for godox).
`secret version promote` and `secret version rm` (`internal/cli/version.go`).
- Cover mnemonic-vs-xprv identity consistency in `pkg/agehd/agehd_test.go`
`TestMnemonicVsXPRVConsistency` (was an in-code FIXME removed for godox).
- CI does not compile, lint or test the files built only with cgo on macOS,
since compiling them needs Apple's SDK:
`internal/secret/keychainunlocker_cgo.go` (the three functions that call
`go-keychain`) with `keychainunlocker_test.go`, and `internal/macse`
(`macse_darwin.go`, `macse_test.go`, the Objective-C sources). Lint has never
run on them, so it would likely find more there than the line lengths. No
macOS test runs in CI. A macOS runner would cover all of it (asked on
https://git.eeqj.de/sneak/secret/issues/50).
- 1.0 critical security blockers (from repo TODO.md):
`TestMnemonicVsXPRVConsistency`.
- No macOS test runs in CI: `internal/secret/pgpunlock_test.go` builds only on
macOS. A macOS runner would cover it.
- 1.0 critical security blockers:
- Memory security: age writes an identity's private key out as a string in
ordinary memory, and the copies it makes on the way stay there
(`secret.IdentityToLockedBuffer` overwrites only the string itself).
@@ -604,6 +32,6 @@ https://git.eeqj.de/sneak/secret/milestone/12
- Cleanups: read statedir from environment or default instead of passing it
around.
- Enhancements: help examples, colored output, --quiet flag, name suggestions on
miss, audit logging, hardware integration tests (Keychain, GPG), naming
consistency, vault export/import, batch operations, search, secret metadata
(descriptions, tags).
miss, audit logging, hardware integration tests (GPG), naming consistency,
vault export/import, batch operations, search, secret metadata (descriptions,
tags).
+6
View File
@@ -3,10 +3,16 @@ package main
import (
"os"
"time"
"sneak.berlin/go/secret/internal/cli"
)
func main() {
// Every time secret prints or writes is in UTC, whatever TZ or the
// system's time zone is set to. It is set here, before cli.Entry starts
// any goroutine that could read time.Local while it changes.
time.Local = time.UTC //nolint:gosmopolitan // this makes local time UTC
os.Exit(cli.Entry())
}
+52 -13
View File
@@ -3,6 +3,7 @@ package cli_test
import (
"bufio"
"context"
"fmt"
"os"
"os/exec"
"path/filepath"
@@ -16,29 +17,67 @@ import (
"sneak.berlin/go/secret/internal/secret"
)
// Entry must return its exit code rather than exit, so that its deferred
// memguard purge runs on the success and the error path alike.
//
//nolint:paralleltest // sets os.Args, and Entry wipes every buffer in the process
func TestEntryWipesBuffersAndReturnsExitCode(t *testing.T) {
savedArgs := os.Args
const (
// envRunEntry, when set, makes this test binary run cli.Entry with its
// own arguments and exit, instead of running the tests.
envRunEntry = "SECRET_TEST_RUN_ENTRY"
t.Cleanup(func() { os.Args = savedArgs })
// bufferWiped is what the test binary prints, run with envRunEntry set,
// when cli.Entry wiped the buffer made before it ran.
bufferWiped = "Entry wiped the buffer made before it ran"
)
// With envRunEntry set, the test binary makes a buffer, runs cli.Entry,
// says whether the buffer is wiped, and exits with Entry's exit code. This
// is done in init, not in a test, so that TestMain does not run in that
// process: it would build the secret binary again, and remove it at the end,
// while the other tests run it.
//
//nolint:gochecknoinits // must run before TestMain, as above
func init() {
if os.Getenv(envRunEntry) == "" {
return
}
buf := memguard.NewBufferFromBytes([]byte("key material"))
code := cli.Entry()
if !buf.IsAlive() {
fmt.Fprintln(os.Stderr, bufferWiped)
}
os.Exit(code)
}
// Entry must return its exit code rather than exit, so that its deferred
// memguard purge runs on the success and the error path alike. Entry wipes
// every memguard buffer in the process, including those other tests are
// using, so it runs in a process of its own: this test binary, run again
// with envRunEntry set.
func TestEntryWipesBuffersAndReturnsExitCode(t *testing.T) {
t.Parallel()
tests := []struct {
args []string
exitCode int
}{
{args: []string{"secret", "--help"}, exitCode: 0},
{args: []string{"secret", "no-such-command"}, exitCode: 1},
{args: []string{"--help"}, exitCode: 0},
{args: []string{"no-such-command"}, exitCode: 1},
}
for _, tt := range tests {
buf := memguard.NewBufferFromBytes([]byte("key material"))
os.Args = tt.args
//nolint:gosec // G204: test runs its own test binary
cmd := exec.CommandContext(t.Context(), os.Args[0], tt.args...)
assert.Equal(t, tt.exitCode, cli.Entry(), "exit code for %v", tt.args)
assert.False(t, buf.IsAlive(), "Entry left a buffer unwiped for %v", tt.args)
cmd.Env = append(os.Environ(), envRunEntry+"=1")
output, err := cmd.CombinedOutput()
require.NotNil(t, cmd.ProcessState, "%v did not run: %v", tt.args, err)
assert.Equal(t, tt.exitCode, cmd.ProcessState.ExitCode(),
"exit code for %v: %s", tt.args, output)
assert.Contains(t, string(output), bufferWiped,
"Entry left a buffer unwiped for %v", tt.args)
}
}
+4 -3
View File
@@ -34,9 +34,10 @@ const (
testMnemonic = "abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about"
// commandWait is how long a test lets the secret binary run before it
// kills it and fails. It stays well under the 30 seconds script/test
// gives the whole package, so a command that hangs fails the test with
// the test's own message instead of Go's timeout panic.
// kills it and fails. It stays well under the time limit the test phase
// of the Dockerfile gives the whole package (go test -timeout), so a
// command that hangs fails the test with the test's own message instead
// of Go's timeout panic.
commandWait = 10 * time.Second
)
-5
View File
@@ -2,7 +2,6 @@ package cli
import (
"os"
"time"
"github.com/awnumar/memguard"
"github.com/spf13/cobra"
@@ -15,10 +14,6 @@ import (
// every memguard buffer before it returns, so the caller must do nothing
// but exit with the code.
func Entry() int {
// Every time secret prints or writes is in UTC, whatever TZ or the
// system's time zone is set to.
time.Local = time.UTC //nolint:gosmopolitan // this makes local time UTC
raiseLockedMemoryLimit()
// On SIGINT or SIGTERM memguard runs this function, wipes every buffer
+1
View File
@@ -1,4 +1,5 @@
{
"private": true,
"devDependencies": {
"prettier": "3.8.1"
}
+46 -26
View File
@@ -2,18 +2,18 @@
# script/bootstrap: install all dependencies needed to build and develop
# this repo. Idempotent: every install is guarded by a check so already
# installed tools are skipped. Base tooling comes from nix, apt, brew,
# or apk (detected in that order); assumes NOTHING is present (not git,
# make, node, yarn, go, or python). Node is used directly if installed;
# otherwise a pinned version is installed via nvm (installing nvm
# itself first, from a hash-verified release archive, never curl | sh).
# or apk (detected in that order); assumes nothing is present. The
# installed node is used only when its major version is the pinned one;
# otherwise (node missing, or another major) the pinned version is
# installed via nvm (installing nvm itself first, from a hash-verified
# release archive, never curl | sh), and yarn and the packages are
# installed under it.
# golangci-lint is never installed: script/lint runs it in docker.
#
# Uncomment the language sections in main() that apply to this repo.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Pinned versions, 2026-07-06. Never "latest" or "lts"; exact versions.
# Pinned versions, 2026-07-06
NODE_VERSION="22.17.0"
NVM_VERSION="0.40.3"
# sha256 of https://github.com/nvm-sh/nvm/archive/refs/tags/v0.40.3.tar.gz
@@ -22,6 +22,7 @@ YARN_VERSION="1.22.22"
PKGMGR=""
SUDO=""
APT_UPDATED=""
detect_pkgmgr() {
[ -n "$PKGMGR" ] && return 0
@@ -50,7 +51,14 @@ pkg_install() {
detect_pkgmgr
case "$PKGMGR" in
nix) nix-env -iA "nixpkgs.$1" ;;
apt) $SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2" ;;
apt)
# Package lists may be empty (fresh images); refresh once per run.
if [ -z "$APT_UPDATED" ]; then
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get update
APT_UPDATED=1
fi
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2"
;;
brew) brew install "$3" ;;
apk) apk add --no-cache "$4" ;;
esac
@@ -82,7 +90,7 @@ nvm_sh() {
ensure_nvm() {
[ -s "$HOME/.nvm/nvm.sh" ] && return 0
# nvm prerequisites; nvm itself requires bash, so install it too
# nvm prerequisites; nvm itself requires bash
if missing bash; then pkg_install bash bash bash bash; fi
if missing curl; then pkg_install curl curl curl curl; fi
if missing git; then pkg_install git git git git; fi
@@ -95,56 +103,68 @@ ensure_nvm() {
rm -rf "$tmp"
}
# True when the node on PATH has the pinned major version. Only the
# major is compared: the Dockerfile stages start from a node 22 alpine
# image whose exact version is not the pin, and nvm cannot install a
# prebuilt node on alpine. Another major is not used: the pinned yarn 1
# prints a deprecation warning under node 24. script/fmt and
# script/fmt-check carry this function unchanged, to run the yarn
# installed here.
node_is_pinned_major() {
if ! command -v node >/dev/null 2>&1; then return 1; fi
installed="$(node --version)"
installed="${installed#v}"
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
}
ensure_node() {
if ! missing node; then return 0; fi
if node_is_pinned_major; then
echo "bootstrap: using node $(node --version)"
return 0
fi
ensure_nvm
nvm_sh "nvm install $NODE_VERSION"
nvm_sh "nvm use $NODE_VERSION >/dev/null && \
echo \"bootstrap: using node \$(node --version)\""
}
ensure_yarn() {
if ! node_is_pinned_major; then
nvm_sh "nvm use $NODE_VERSION >/dev/null && corepack enable && \
corepack prepare yarn@$YARN_VERSION --activate"
return 0
fi
if ! missing yarn; then return 0; fi
if ! missing corepack; then
corepack enable
corepack prepare "yarn@$YARN_VERSION" --activate
elif [ -s "$HOME/.nvm/nvm.sh" ]; then
nvm_sh "nvm use $NODE_VERSION >/dev/null && corepack enable && \
corepack prepare yarn@$YARN_VERSION --activate"
else
npm install -g "yarn@$YARN_VERSION"
fi
}
install_js_deps() {
if missing yarn && [ -s "$HOME/.nvm/nvm.sh" ]; then
if node_is_pinned_major; then
yarn install --frozen-lockfile
else
nvm_sh "nvm use $NODE_VERSION >/dev/null && cd \"$ROOT\" && \
yarn install --frozen-lockfile"
else
yarn install --frozen-lockfile
fi
}
main() {
cd "$ROOT"
# Base tooling (every repo)
if missing git; then pkg_install git git git git; fi
if missing make; then pkg_install gnumake make make make; fi
if missing git; then pkg_install git git git git; fi
# ---- JS / docs repos ----
# prettier, pinned in package.json and yarn.lock, formats the markdown
ensure_node
ensure_yarn
install_js_deps
# ---- Go repos ----
if missing go; then pkg_install go golang go go; fi
go mod download
# ---- Python repos ----
# if missing python3; then pkg_install python3 python3 python3 python3; fi
# python3 -m venv .venv
# ./.venv/bin/pip install -e '.[dev]'
echo "bootstrap complete"
}
+13 -9
View File
@@ -2,9 +2,11 @@
# script/cibuild: run the CI build. It bootstraps first: a CI runner
# checks out and runs this and nothing else, and script/fmt-check runs
# the formatter on the host, which a pristine checkout cannot do.
# --no-cache for the same reason as script/docker: the gate phases the
# final stage depends on are RUN steps, and a cached one is a check that
# did not run.
# A new CACHEBUST value reruns every RUN step, since a cached gate phase
# is a check that did not run; unlike --no-cache, it keeps Go's build cache.
# This relies on every stage of the Dockerfile declaring ARG CACHEBUST
# first; a stage without that line is served from cache on an unchanged
# tree, with no warning.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -14,15 +16,17 @@ main() {
cd "$ROOT"
"$SCRIPT_DIR/bootstrap"
"$SCRIPT_DIR/check"
# Own line: a failing command substitution inside an argument does
# not trip `set -e`, so the inline form degrades silently to an
# empty constant. The VERSION build argument takes precedence over
# the version a build stage derives from the .git in the context.
# The version and the tag each get their own line: a failing
# command substitution inside an argument does not trip `set -e`,
# so the inline form degrades silently to an empty constant. The
# VERSION build argument takes precedence over the version a build
# stage derives from the .git in the context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown"
docker build --no-cache \
tag="$("$SCRIPT_DIR/projectname")"
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--build-arg VERSION="$version" \
-t "$("$SCRIPT_DIR/projectname")" .
-t "$tag" .
}
main "$@"
+13 -8
View File
@@ -1,8 +1,11 @@
#!/bin/sh
# script/docker: build the Docker image tagged with the project name.
# Identical in all repos; the tag comes from script/projectname.
# --no-cache because the gate phases the final stage depends on are RUN
# steps, and a cached one is a check that did not run.
# A new CACHEBUST value reruns every RUN step, since a cached gate phase
# is a check that did not run; unlike --no-cache, it keeps Go's build cache.
# This relies on every stage of the Dockerfile declaring ARG CACHEBUST
# first; a stage without that line is served from cache on an unchanged
# tree, with no warning.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -10,15 +13,17 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
# Own line: a failing command substitution inside an argument does
# not trip `set -e`, so the inline form degrades silently to an
# empty constant. The VERSION build argument takes precedence over
# the version a build stage derives from the .git in the context.
# The version and the tag each get their own line: a failing
# command substitution inside an argument does not trip `set -e`,
# so the inline form degrades silently to an empty constant. The
# VERSION build argument takes precedence over the version a build
# stage derives from the .git in the context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown"
docker build --no-cache \
tag="$("$SCRIPT_DIR/projectname")"
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--build-arg VERSION="$version" \
-t "$("$SCRIPT_DIR/projectname")" .
-t "$tag" .
}
main "$@"
+16 -6
View File
@@ -8,15 +8,25 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Must match the pin in script/bootstrap.
NODE_VERSION="22.17.0"
# script/bootstrap installs node and yarn under nvm and leaves neither
# on the PATH of the shell that called it, so resolve the pinned
# toolchain here the way bootstrap's own install step does. nvm is a
# bash script, hence the subshell.
# True when the node on PATH has the pinned major version: the same
# function as in script/bootstrap, which uses it to decide where it
# installs yarn.
node_is_pinned_major() {
if ! command -v node >/dev/null 2>&1; then return 1; fi
installed="$(node --version)"
installed="${installed#v}"
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
}
# Run the yarn script/bootstrap installed: the yarn on PATH when the
# node on PATH has the pinned major version, and otherwise yarn under
# the pinned node in nvm, which bootstrap leaves off the PATH of the
# shell that called it. nvm is a bash script, hence the subshell.
run_yarn() {
if command -v yarn >/dev/null 2>&1; then
if node_is_pinned_major && command -v yarn >/dev/null 2>&1; then
exec yarn "$@"
fi
if [ ! -s "$HOME/.nvm/nvm.sh" ]; then
if node_is_pinned_major || [ ! -s "$HOME/.nvm/nvm.sh" ]; then
echo "fmt: no yarn; run script/bootstrap first" >&2
exit 1
fi
+16 -6
View File
@@ -8,15 +8,25 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Must match the pin in script/bootstrap.
NODE_VERSION="22.17.0"
# script/bootstrap installs node and yarn under nvm and leaves neither
# on the PATH of the shell that called it, so resolve the pinned
# toolchain here the way bootstrap's own install step does. nvm is a
# bash script, hence the subshell.
# True when the node on PATH has the pinned major version: the same
# function as in script/bootstrap, which uses it to decide where it
# installs yarn.
node_is_pinned_major() {
if ! command -v node >/dev/null 2>&1; then return 1; fi
installed="$(node --version)"
installed="${installed#v}"
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
}
# Run the yarn script/bootstrap installed: the yarn on PATH when the
# node on PATH has the pinned major version, and otherwise yarn under
# the pinned node in nvm, which bootstrap leaves off the PATH of the
# shell that called it. nvm is a bash script, hence the subshell.
run_yarn() {
if command -v yarn >/dev/null 2>&1; then
if node_is_pinned_major && command -v yarn >/dev/null 2>&1; then
exec yarn "$@"
fi
if [ ! -s "$HOME/.nvm/nvm.sh" ]; then
if node_is_pinned_major || [ ! -s "$HOME/.nvm/nvm.sh" ]; then
echo "fmt-check: no yarn; run script/bootstrap first" >&2
exit 1
fi
-1
View File
@@ -1,7 +1,6 @@
#!/bin/sh
# script/install-precommit: install the git pre-commit hook that runs
# script/precommit. Our own extension to scripts-to-rule-them-all.
# Generic: needs no adaptation.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
+8 -5
View File
@@ -5,9 +5,12 @@
# make its answer untrustworthy.
#
# The phase is not the last stage in the file, so it is built only when
# --target names it. --no-cache because a cached lint layer is a lint
# that did not run. The tag makes each build replace the previous image
# instead of leaving a dangling one behind.
# --target names it. A new CACHEBUST value reruns every RUN step, since
# a cached lint layer is a lint that did not run; unlike --no-cache, it
# keeps Go's build cache. This relies on every stage of the Dockerfile
# declaring ARG CACHEBUST first; a stage without that line is served
# from cache on an unchanged tree, with no warning.
# --output type=cacheonly writes no image, since nothing uses one.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -15,9 +18,9 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
docker build --no-cache \
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--target lint \
-t "$("$SCRIPT_DIR/projectname")-lint" .
--output type=cacheonly .
}
main "$@"
+1 -2
View File
@@ -1,7 +1,6 @@
#!/bin/sh
# script/setup: set up the repo for development after a fresh clone:
# installs dependencies (script/bootstrap) and the git pre-commit hook.
# Add any repo-specific initialization (db init, .env template) here.
# installs dependencies and the git pre-commit hook.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
+8 -4
View File
@@ -2,8 +2,12 @@
# script/test: run the test suite. Testing is a phase of the Dockerfile
# and this builds that phase alone, on the same terms as script/lint:
# --target because a phase that is not the last stage is built only when
# named, --no-cache because a cached test layer is a test that did not
# run, and a tag so each build replaces the previous image.
# named, and a new CACHEBUST value because a cached test layer is a test
# that did not run; unlike --no-cache, it keeps Go's build cache. This
# relies on every stage of the Dockerfile declaring ARG CACHEBUST first;
# a stage without that line is served from cache on an unchanged tree,
# with no warning. --output type=cacheonly writes no image, since
# nothing uses one.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -11,9 +15,9 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
docker build --no-cache \
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--target test \
-t "$("$SCRIPT_DIR/projectname")-test" .
--output type=cacheonly .
}
main "$@"