5 Commits
Author SHA1 Message Date
clawbot 26a4739615 Keep Go's build cache in check builds: re-vendor sneak/prompts e3f79c4 (closes #129)
check / check (push) Waiting to run
The vendored files are copies from sneak/prompts e3f79c4, with this
repository's own entries kept. The script/ builds pass a new CACHEBUST
value instead of --no-cache, which emptied Go's build cache mount, so
every make test compiled the standard library and every dependency with
-race. Every Dockerfile stage declares ARG CACHEBUST first, so every RUN
step still reruns on an unchanged tree. The steps that compile mount
Go's build cache, and both go test runs keep -count=1. The lint and test
builds write no image. script/bootstrap, script/fmt and script/fmt-check
are the canonical files plus this repository's Go steps.

Model: opus-5-5
2026-10-15 19:58:38 +02:00
clawbot 8851171877 Set the time zone in main and run Entry in its own test process (closes #192)
check / check (push) Waiting to run
cli.Entry() set time.Local to UTC and ended with memguard.Purge(), and
TestEntryWipesBuffersAndReturnsExitCode ran it in the same process as
the other tests. A timer left by an earlier test could read time.Local
as it changed, which the race detector reported on some runs, and the
purge could destroy a buffer still in use, the likeliest cause of a
crash seen on another run.

main() in cmd/secret now sets time.Local before it calls cli.Entry();
the test that runs the built binary with TZ=America/New_York still
covers it. The Entry test runs cli.Entry() in a copy of the test binary
started with SECRET_TEST_RUN_ENTRY set, which init picks up so that
TestMain does not run there.

Model: opus-5-5
2026-10-15 19:25:10 +02:00
clawbot 25f4652e70 Log the audit fixes in TODO.md (closes #173)
check / check (push) Waiting to run
Completed Steps gets one entry for each fix merged into next since
v1.0.0-rc2 that had none, newest first in landing order. Each entry says
what its landing commit says and carries that commit's date in UTC. The
five fixes that logged themselves before this issue's decision keep
their entries and get no second one. Status no longer says there are no
git tags.

Model: opus-5-5
2026-10-15 17:58:57 +02:00
clawbot 4c47b233bb A secret mv between vaults unlocks both vaults before taking the lock (closes #189)
check / check (push) Waiting to run
A move between two vaults unlocked them while holding the state directory
lock, and unlocking can ask for a passphrase, so every other changing
command waited at that prompt. moveSecretCrossVault now checks the move,
refusing an existing destination secret without --force, unlocks both
vaults, and only then takes the lock. Under the lock, it checks that the
destination vault still has the long-term key it unlocked, and
copySecretAndRemoveSource checks the move again before it copies. A move
within a vault takes the lock in moveSecretWithinVault, so MoveSecret
takes none itself. The error for a vault created again with another key,
shared with secret unlocker add, now ends "nothing was changed".

Model: opus-5-5
2026-10-15 10:49:06 +02:00
clawbot fac2753819 Ask for the mnemonic and passphrase before taking the state directory lock (closes #152)
check / check (push) Waiting to run
init, vault create, unlocker add, and add, import and generate secret
with --force asked for the mnemonic or passphrase, or unlocked the
vault, while holding the state directory lock, so every other command
that changes anything waited at the prompt. They now ask and unlock
first and lock after. Under the lock, unlocker add fails if the vault
was removed, or created again with another key, and refuses a GPG key
added meanwhile; a forced add of a secret created meanwhile lets go of
the lock, gets the key and locks again. secret.CreatePGPUnlocker takes
the unlocked vault; SetMnemonic and SetUnlockPassphrase are gone.

Not done: secret move between vaults still unlocks both under the lock
(#189).

Model: opus-5-5
2026-10-15 09:02:13 +02:00
40 changed files with 1280 additions and 419 deletions
+3
View File
@@ -13,3 +13,6 @@ indent_style = tab
[*.go] [*.go]
indent_style = tab indent_style = tab
# This repository's own sections, such as one for another language it
# uses, go below this comment, and a re-vendor keeps them.
+11
View File
@@ -1,9 +1,20 @@
name: check name: check
on: [push] on: [push]
# Free the shared runner: a new push cancels only the same branch's older run.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs: jobs:
check: check:
runs-on: ubuntu-latest runs-on: ubuntu-latest
# Free the shared runner from a hung build.
timeout-minutes: 20
steps: steps:
# actions/checkout v4.2.2, 2026-02-22 # actions/checkout v4.2.2, 2026-02-22
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
# script/cibuild needs no token, so none is left in .git/config.
with:
persist-credentials: false
# All history and tags, so git describe finds the version tag.
fetch-depth: 0
- run: script/cibuild - run: script/cibuild
+5 -1
View File
@@ -27,7 +27,7 @@ node_modules/
# Environment files. `*.env` covers bare `.env` and the `prod.env` # Environment files. `*.env` covers bare `.env` and the `prod.env`
# convention. Only the templates `example.env` and `sample.env` are # convention. Only the templates `example.env` and `sample.env` are
# re-included below. A repository that commits any other template adds # re-included below. A repository that commits any other template adds
# its own negation after these lines, for example `!.env.example`. # its own negation at the end of this file, for example `!.env.example`.
*.[eE][nN][vV] *.[eE][nN][vV]
.[eE][nN][vV].* .[eE][nN][vV].*
.[eE][nN][vV][rR][cC] .[eE][nN][vV][rR][cC]
@@ -46,6 +46,10 @@ node_modules/
[iI][dD]_[eE][dD]25519 [iI][dD]_[eE][dD]25519
[iI][dD]_[eE][dD]25519_[sS][kK] [iI][dD]_[eE][dD]25519_[sS][kK]
# This repository's own entries, such as its build outputs, go below
# this comment, and a re-vendor keeps them. Anchor a binary built at the
# root: `/myapp`, never `myapp`, which also ignores `cmd/myapp/`.
# Go. /secret is the built binary, anchored so that it does not also match # Go. /secret is the built binary, anchored so that it does not also match
# the internal/secret/ package directory. # the internal/secret/ package directory.
*.log *.log
+2
View File
@@ -25,6 +25,8 @@ linters:
# silenced by disabling that name, not by enabling the successor. # silenced by disabling that name, not by enabling the successor.
- wsl # Deprecated, replaced by wsl_v5 - wsl # Deprecated, replaced by wsl_v5
- gomodguard # Deprecated, replaced by gomodguard_v2 - gomodguard # Deprecated, replaced by gomodguard_v2
# Misses findings at random in v2.14.0; back once a pinned release fixes it
- canonicalheader
settings: settings:
lll: lll:
line-length: 88 line-length: 88
+21 -11
View File
@@ -1,33 +1,40 @@
# Every stage declares ARG CACHEBUST first: script/ passes a new value
# on every build, which reruns every RUN step after it. The steps that
# compile mount Go's build cache, which survives from build to build.
# Lint phase. The linter is invoked directly rather than through `make # Lint phase. The linter is invoked directly rather than through `make
# lint` or `script/lint`, which are themselves a docker build. # lint` or `script/lint`, which are themselves a docker build.
# golangci/golangci-lint:v2.14.0 (Debian-based), 2026-09-24 # golangci/golangci-lint:v2.14.0 (Debian-based), 2026-09-24
FROM golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint FROM golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint
ARG CACHEBUST
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
COPY . . COPY . .
RUN go vet ./... RUN --mount=type=cache,target=/root/.cache/go-build \
RUN golangci-lint run --config .golangci.yml ./... go vet ./...
RUN --mount=type=cache,target=/root/.cache/go-build \
golangci-lint run --config .golangci.yml ./...
# The same checks on the code as a macOS build compiles it, which a Linux # The same checks on the code as a macOS build compiles it, which a Linux
# build never compiles. Cgo is off, because compiling cgo code for macOS # build never compiles. Cgo is off, because compiling cgo code for macOS
# needs Apple's SDK headers. # needs Apple's SDK headers.
RUN GOOS=darwin CGO_ENABLED=0 go vet ./... RUN --mount=type=cache,target=/root/.cache/go-build \
RUN GOOS=darwin CGO_ENABLED=0 golangci-lint run --config .golangci.yml ./... GOOS=darwin CGO_ENABLED=0 go vet ./...
RUN --mount=type=cache,target=/root/.cache/go-build \
GOOS=darwin CGO_ENABLED=0 golangci-lint run --config .golangci.yml ./...
# Test phase. -race needs cgo and so a C compiler, which the Debian Go # Test phase. -race needs cgo and so a C compiler, which the Debian Go
# image ships and the alpine one does not. # image ships and the alpine one does not. -count=1 because the mounted
# cache also holds test results, which go test would otherwise replay.
# golang:1.24.13-trixie, 2026-02-04 # golang:1.24.13-trixie, 2026-02-04
FROM golang@sha256:5835f052b784aa39f2fe9070def3568605c8bc3fcd810f10402066348b61e716 AS test FROM golang@sha256:5835f052b784aa39f2fe9070def3568605c8bc3fcd810f10402066348b61e716 AS test
ARG CACHEBUST
ENV CGO_ENABLED=1 ENV CGO_ENABLED=1
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
COPY . . COPY . .
# Go's build cache goes in a cache mount, not the image layer, which would RUN --mount=type=cache,target=/root/.cache/go-build \
# take seconds longer to export. --no-cache, on every build in script/,
# starts the mount empty; -count=1 keeps a build without it from taking
# test results from there.
RUN --mount=type=cache,id=sneak/secret/go-build-test,target=/root/.cache/go-build \
go test -count=1 -timeout 90s -race -cover ./... || \ go test -count=1 -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \ { echo "--- Rerunning with -v for details ---"; \
go test -count=1 -timeout 90s -race -v ./...; exit 1; } go test -count=1 -timeout 90s -race -v ./...; exit 1; }
@@ -37,6 +44,7 @@ RUN --mount=type=cache,id=sneak/secret/go-build-test,target=/root/.cache/go-buil
# lint and test passed. # lint and test passed.
# golang 1.24.13-alpine, 2026-03-10 # golang 1.24.13-alpine, 2026-03-10
FROM golang@sha256:8bee1901f1e530bfb4a7850aa7a479d17ae3a18beb6e09064ed54cfd245b7191 AS builder FROM golang@sha256:8bee1901f1e530bfb4a7850aa7a479d17ae3a18beb6e09064ed54cfd245b7191 AS builder
ARG CACHEBUST
COPY --from=lint /src/go.sum /dev/null COPY --from=lint /src/go.sum /dev/null
COPY --from=test /src/go.sum /dev/null COPY --from=test /src/go.sum /dev/null
# script/build compiles with cgo, so it needs a C compiler too. # script/build compiles with cgo, so it needs a C compiler too.
@@ -53,7 +61,8 @@ COPY . .
# .git present, a version that is still empty, dev or unknown fails the # .git present, a version that is still empty, dev or unknown fails the
# build: git is missing or could not read the checkout. # build: git is missing or could not read the checkout.
ARG VERSION ARG VERSION
RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \ RUN --mount=type=cache,target=/root/.cache/go-build \
VERSION="${VERSION:-$(git describe --tags --always)}"; \
if [ -e .git ]; then \ if [ -e .git ]; then \
case "$VERSION" in ""|dev|unknown) \ case "$VERSION" in ""|dev|unknown) \
echo "version is '$VERSION' although .git is present" >&2; \ echo "version is '$VERSION' although .git is present" >&2; \
@@ -65,6 +74,7 @@ RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \
# Runtime stage, and the last one # Runtime stage, and the last one
# alpine 3.23, 2026-03-10 # alpine 3.23, 2026-03-10
FROM alpine@sha256:25109184c71bdad752c8312a8623239686a9a2071e8825f20acb8f2198c3f659 FROM alpine@sha256:25109184c71bdad752c8312a8623239686a9a2071e8825f20acb8f2198c3f659
ARG CACHEBUST
RUN apk add --no-cache ca-certificates gnupg RUN apk add --no-cache ca-certificates gnupg
+12
View File
@@ -438,6 +438,14 @@ asks when stdin is a pipe, as in `echo value | secret add name --force` or
`cat file | secret encrypt key`. With no controlling terminal, as under cron or `cat file | secret encrypt key`. With no controlling terminal, as under cron or
in a CI job, the command fails and names the variable to set. in a CI job, the command fails and names the variable to set.
A command that changes anything locks the state directory, through its file
`lock`, and another such command waits until the lock is released.
`secret init`, `secret vault create`, `secret unlocker add`, `secret mv` from
one vault to another, and `secret add`, `secret import` and
`secret generate secret` with `--force`, ask for the mnemonic or passphrase
before they take the lock, so a prompt of theirs left open in one terminal holds
up no command in another.
## Security Features ## Security Features
### Encryption ### Encryption
@@ -667,6 +675,10 @@ provide:
- `script/install-precommit` — install the git pre-commit hook that runs - `script/install-precommit` — install the git pre-commit hook that runs
`script/precommit` `script/precommit`
Every stage of the `Dockerfile` declares `ARG CACHEBUST` as its first line after
`FROM`. The `script/` builds pass a new `CACHEBUST` value on every build, which
reruns every `RUN` step and keeps Go's build cache; `--no-cache` would empty it.
## Features ## Features
- **Multiple Authentication Methods**: Supports passphrase and PGP unlockers - **Multiple Authentication Methods**: Supports passphrase and PGP unlockers
+156 -79
View File
@@ -1,6 +1,6 @@
--- ---
title: Repository Policies title: Repository Policies
last_modified: 2026-10-04 last_modified: 2026-10-15
--- ---
This document covers repository structure, tooling, and workflow standards. Code This document covers repository structure, tooling, and workflow standards. Code
@@ -54,34 +54,39 @@ style conventions are in separate documents:
`cibuild`. `script/bootstrap` installs all dependencies idempotently and `cibuild`. `script/bootstrap` installs all dependencies idempotently and
assumes nothing is present: base tools come from nix, apt, brew, or apk assumes nothing is present: base tools come from nix, apt, brew, or apk
(detected in that order; apt runs noninteractive). For node it uses the (detected in that order; apt runs noninteractive). For node it uses the
installed node if present; otherwise it installs a PINNED node version via installed node only when its major version is the pinned one; otherwise (node
nvm, first installing nvm itself if missing — from a hash-verified GitHub missing, or another major, such as the node 24 the CI runner image ships) it
release archive (never `curl | sh`), with bash installed as an explicit installs the PINNED node version via nvm and installs yarn and the packages
prerequisite since nvm requires bash. yarn is then pinned via under it. nvm itself is installed first if missing, from a hash-verified
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts"; GitHub release archive (never `curl | sh`), with bash installed as an explicit
always exact versions. `script/cibuild` runs the CI build: it changes to the prerequisite since nvm requires bash. Only the major version is compared
repo root, runs `script/bootstrap`, runs `script/check`, and builds the image because the `Dockerfile` stages start from a node image whose exact version is
with the version; the Gitea workflow calls it. **`script/cibuild` runs not the pin, and nvm cannot install a prebuilt node on alpine. yarn is pinned
via `corepack prepare yarn@<version> --activate`. Never install "latest" or
"lts"; always exact versions. `script/cibuild` runs the CI build: it changes
to the repo root, runs `script/bootstrap`, runs `script/check`, and builds the
image with the version; the Gitea workflow calls it. **`script/cibuild` runs
`script/bootstrap` first**, because the workflow checks out the repo and runs `script/bootstrap` first**, because the workflow checks out the repo and runs
nothing else, while `script/fmt-check` runs the formatter on the host: on a nothing else, while `script/fmt-check` runs the formatter on the host: on a
pristine checkout with nothing installed the run dies there, after the pristine checkout with nothing installed the run dies there, after the
containerised gates have passed. **The bootstrap alone is not enough**: containerised gates have passed. **The bootstrap alone is not enough**: when
`script/bootstrap` installs node and yarn under nvm and leaves neither on the `script/bootstrap` installs node and yarn under nvm it leaves neither on the
`PATH` of the shell that called it, so a bare `yarn` still exits 127. The host `PATH` of the shell that called it, so a bare `yarn` either exits 127 or runs
entrypoints that need yarn — `script/fmt` and `script/fmt-check` — therefore under another node. The host entrypoints that need yarn — `script/fmt` and
source nvm for the pinned node version before invoking it, exactly as `script/fmt-check` — therefore use the same test as `script/bootstrap`: when
`script/bootstrap`'s own install step does. A runner carrying nothing but the node on `PATH` has the pinned major version they run the `yarn` on `PATH`,
docker and git then gets through `script/check`. Four further scripts are our and otherwise they load nvm through `$HOME/.nvm/nvm.sh` and run yarn under the
own extensions to the standard: `script/check` runs `script/test`, pinned node. A runner carrying nothing but docker and git then gets through
`script/lint` and `script/fmt-check`; `script/precommit` is what the git `script/check`. Four further scripts are our own extensions to the standard:
pre-commit hook runs, and it calls `script/check`; `script/install-precommit` `script/check` runs `script/test`, `script/lint` and `script/fmt-check`;
installs the git pre-commit hook (the `make hooks` target shims to it); and `script/precommit` is what the git pre-commit hook runs, and it calls
`script/projectname` (literally that filename) simply outputs the project's `script/check`; `script/install-precommit` installs the git pre-commit hook
name. Scripts that need the name call `script/projectname` — e.g. (the `make hooks` target shims to it); and `script/projectname` (literally
`script/docker` assembles its image tag from it — so those scripts stay that filename) simply outputs the project's name. Scripts that need the name
byte-identical across all repos. Repo-type-specific pre-commit extras (e.g. call `script/projectname` — e.g. `script/docker` assembles its image tag from
`go mod tidy` verification in Go repos) belong in `script/precommit`, not in it — so those scripts stay byte-identical across all repos. Repo-type-specific
the hook itself. Model scripts are at pre-commit extras (e.g. `go mod tidy` verification in Go repos) belong in
`script/precommit`, not in the hook itself. Model scripts are at
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README `https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
must document the provided scripts in an **Entrypoints** section (see the must document the provided scripts in an **Entrypoints** section (see the
README requirements below). README requirements below).
@@ -118,8 +123,10 @@ style conventions are in separate documents:
and nothing else: and nothing else:
```sh ```sh
docker build --no-cache --target lint -t "$(script/projectname)-lint" . docker build --build-arg CACHEBUST="$(date +%s)-$$" \
docker build --no-cache --target test -t "$(script/projectname)-test" . --target lint --output type=cacheonly .
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--target test --output type=cacheonly .
``` ```
**A stage that is not the last one in the file is built only when the final **A stage that is not the last one in the file is built only when the final
@@ -129,10 +136,15 @@ style conventions are in separate documents:
plain `docker build .` builds the last stage alone and exits 0 having linted plain `docker build .` builds the last stage alone and exits 0 having linted
and tested nothing. and tested nothing.
**Every `docker build` in `script/` is tagged**, here and in **The gate builds write no image.** With `--output type=cacheonly` the phase
`script/cibuild` and `script/docker`. An untagged build leaves a dangling runs and a failing step fails the build, but the result is not exported.
image behind on every invocation, on every developer host and every CI Nothing uses those images, and writing one out is slow: a Go test phase's
runner; a tagged one replaces the previous image. image holds the toolchain. A build given neither `--output` nor `-t` writes
an untagged image and leaves it dangling, on every developer host and every
CI runner. `script/cibuild` and `script/docker` build the image that ships
and tag it, so each build replaces the previous image; each assigns the tag
on its own line before the build, so `set -e` stops it where
`script/projectname` fails.
Inside a phase the tool is invoked directly — `golangci-lint`, `go test`, Inside a phase the tool is invoked directly — `golangci-lint`, `go test`,
`eslint`, `prettier` — never through `make lint` or `script/test`, which are `eslint`, `prettier` — never through `make lint` or `script/test`, which are
@@ -147,22 +159,33 @@ style conventions are in separate documents:
one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs
exit non-zero with `parallel golangci-lint is running` — a status a caller exit non-zero with `parallel golangci-lint is running` — a status a caller
cannot tell from real findings. Both have produced wrong verdicts in this cannot tell from real findings. Both have produced wrong verdicts in this
org, in both directions. A container has its own cache, its own `TMPDIR` and org, in both directions. A container has its own golangci-lint cache, its
a digest-pinned binary, so neither is reachable. own `TMPDIR` and a digest-pinned binary, so neither is reachable.
- **Any build that runs checks is built with `--no-cache`.** Docker invalidates - **Any build that runs checks is given a new `CACHEBUST` value.** Docker
a `COPY` layer only when the copied content changes, so on an unchanged tree invalidates a `COPY` layer only when the copied content changes, so on an
the check `RUN` is served from cache, nothing executes, and the build still unchanged tree the check `RUN` is served from cache, nothing executes, and the
exits 0. Every `docker build` in `script/` therefore passes `--no-cache`: build still exits 0. Every `docker build` in `script/` therefore passes
`script/lint`, `script/test`, `script/cibuild` and `script/docker` are the `--build-arg CACHEBUST="$(date +%s)-$$"`, the time and the script's process
four, and there is no fifth — `script/check` runs the two gate phases and id, so no two builds share a value, not even the three in one
`script/fmt-check`, and builds no image of its own. A bare `docker build .` is `script/cibuild`. `script/lint`, `script/test`, `script/cibuild` and
not evidence that anything ran: a sub-second build reporting success is a `script/docker` are the four, and there is no fifth — `script/check` runs the
cache hit, not a result. Never invalidate by pruning — `docker builder prune` two gate phases and `script/fmt-check`, and builds no image of its own. Every
and friends destroy a build cache shared with every other build on the host. stage of the `Dockerfile` declares `ARG CACHEBUST` as its first line after
When a check is added or changed, prove it works by planting a defect it must `FROM`, and a new value reruns every `RUN` step after that line; `COPY` steps
catch and watching the run fail on it, then revert the defect. A green run are compared by content and need nothing. A stage without the `ARG` line is
alone shows neither that the check ran nor that it covers what it should. served from cache on an unchanged tree, so a repository re-vendors the scripts
and adds the `ARG` lines in one commit. The scripts do not pass `--no-cache`:
it reruns every step too, but it also empties every cache mount, and Go's
build cache, which the canonical Go `Dockerfile` below mounts so that compiled
packages survive between check builds, would start empty every time. A bare
`docker build .` is not evidence that anything ran: a sub-second build
reporting success is a cache hit, not a result. Never invalidate by pruning —
`docker builder prune` and friends destroy a build cache shared with every
other build on the host. When a check is added or changed, prove it works by
planting a defect it must catch and watching the run fail on it, then revert
the defect. A green run alone shows neither that the check ran nor that it
covers what it should.
- **The gate phases are separate stages, and the build stage depends on both.** - **The gate phases are separate stages, and the build stage depends on both.**
The lint phase is based on the `golangci/golangci-lint` image (pinned by The lint phase is based on the `golangci/golangci-lint` image (pinned by
@@ -171,32 +194,42 @@ style conventions are in separate documents:
`Dockerfile`: `Dockerfile`:
```dockerfile ```dockerfile
# Every stage declares ARG CACHEBUST first: script/ passes a new value
# on every build, which reruns every RUN step after it. The steps that
# compile mount Go's build cache, which survives from build to build.
# Lint phase # Lint phase
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD # golangci/golangci-lint:v2.x.x, YYYY-MM-DD
FROM golangci/golangci-lint@sha256:... AS lint FROM golangci/golangci-lint@sha256:... AS lint
ARG CACHEBUST
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
COPY . . COPY . .
RUN golangci-lint run --config .golangci.yml ./... RUN --mount=type=cache,target=/root/.cache/go-build \
golangci-lint run --config .golangci.yml ./...
# Test phase. -race needs cgo and so a C compiler, which the Debian Go # Test phase. -race needs cgo and so a C compiler, which the Debian Go
# image ships and the alpine one does not. # image ships and the alpine one does not. -count=1 because the mounted
# cache also holds test results, which go test would otherwise replay.
# golang:1.x, YYYY-MM-DD # golang:1.x, YYYY-MM-DD
FROM golang@sha256:... AS test FROM golang@sha256:... AS test
ARG CACHEBUST
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
COPY . . COPY . .
RUN go test -timeout 90s -race -cover ./... || \ RUN --mount=type=cache,target=/root/.cache/go-build \
go test -count=1 -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \ { echo "--- Rerunning with -v for details ---"; \
go test -timeout 90s -race -v ./...; exit 1; } go test -count=1 -timeout 90s -race -v ./...; exit 1; }
# Build stage. Nothing is wanted from either phase above; the copies # Build stage. Nothing is wanted from either phase above; the copies
# are what make BuildKit build them first, so this stage cannot run # are what make BuildKit build them first, so this stage cannot run
# unless lint and test passed. # unless lint and test passed.
# golang:1.x-alpine, YYYY-MM-DD # golang:1.x-alpine, YYYY-MM-DD
FROM golang@sha256:... AS builder FROM golang@sha256:... AS builder
ARG CACHEBUST
COPY --from=lint /src/go.sum /dev/null COPY --from=lint /src/go.sum /dev/null
COPY --from=test /src/go.sum /dev/null COPY --from=test /src/go.sum /dev/null
RUN apk add --no-cache git RUN apk add --no-cache git
@@ -212,7 +245,8 @@ style conventions are in separate documents:
# .git present, a version that is still empty, dev or unknown fails the # .git present, a version that is still empty, dev or unknown fails the
# build: git is missing or could not read the checkout. # build: git is missing or could not read the checkout.
ARG VERSION ARG VERSION
RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \ RUN --mount=type=cache,target=/root/.cache/go-build \
VERSION="${VERSION:-$(git describe --tags --always)}"; \
if [ -e .git ]; then \ if [ -e .git ]; then \
case "$VERSION" in ""|dev|unknown) \ case "$VERSION" in ""|dev|unknown) \
echo "version is '$VERSION' although .git is present" >&2; \ echo "version is '$VERSION' although .git is present" >&2; \
@@ -225,6 +259,7 @@ style conventions are in separate documents:
# Runtime stage, and the last one # Runtime stage, and the last one
FROM alpine@sha256:... FROM alpine@sha256:...
ARG CACHEBUST
COPY --from=builder /app /usr/local/bin/app COPY --from=builder /app /usr/local/bin/app
ENTRYPOINT ["app"] ENTRYPOINT ["app"]
``` ```
@@ -239,6 +274,16 @@ style conventions are in separate documents:
- Keep the runtime stage last, and if you add a stage after it, give it the - Keep the runtime stage last, and if you add a stage after it, give it the
same two copies. A plain `docker build .` builds the last stage's chain same two copies. A plain `docker build .` builds the last stage's chain
and nothing else. and nothing else.
- Only Go's build cache is mounted (`/root/.cache/go-build`, where Go keeps
it for root), and only on the steps that compile: `golangci-lint run`,
both `go test` runs and `go build`. It keeps compiled packages between
check builds, and Go reuses one only when all its inputs are unchanged, so
it saves the compile without skipping a check. Nothing else is kept:
golangci-lint's own cache would replay stored findings in place of a lint
run, and Go builds from whatever files a kept module cache holds without
checking them again (only `go mod verify` does), so a file changed there
would go unnoticed. Go's build cache also holds test results, which is why
both `go test` runs carry `-count=1`.
- If the project uses `//go:embed` directives that reference build artifacts - If the project uses `//go:embed` directives that reference build artifacts
(e.g. a web frontend compiled in a separate stage), the lint phase must (e.g. a web frontend compiled in a separate stage), the lint phase must
create placeholder files so the embed directives resolve. Example: create placeholder files so the embed directives resolve. Example:
@@ -278,20 +323,39 @@ style conventions are in separate documents:
`.git` and the version still comes out empty, `dev` or `unknown`. A plain `.git` and the version still comes out empty, `dev` or `unknown`. A plain
`docker build .` with no build arguments must succeed; a Dockerfile that `docker build .` with no build arguments must succeed; a Dockerfile that
refuses an empty build argument drops that refusal and keeps the argument. refuses an empty build argument drops that refusal and keeps the argument.
A checkout whose `.git` is a file (a linked worktree, or a repository
checked out as a submodule) is the exception: that file points to a git
directory outside the build context, so the build cannot read the version
and a plain `docker build .` fails; pass the version with
`--build-arg VERSION=...`, as `script/docker` and `script/cibuild` already
do.
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that - Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
runs `script/cibuild` on push, and checks out the repo as its only other step. runs `script/cibuild` on push, and checks out the repo as its only other step,
That script bootstraps, runs the gate phases, and then builds the image, so a with `persist-credentials: false`: `script/cibuild` needs no token, and
successful run means every check passed; a bare `docker build .` does not without it the checkout leaves the job's token in `.git/config` for every
later step. The checkout step also sets `fetch-depth: 0`, which fetches the
tags `git describe` needs: by default it clones shallow with no tags, and a
tagged repository's CI build would stamp a bare short commit id. The
workflow's `concurrency` block groups runs by workflow and branch
(`${{ github.workflow }}-${{ github.ref }}`) with `cancel-in-progress: true`,
so a new push cancels the older run on the same branch, queued or running, and
no other: runs for replaced commits do not hold up the shared runner.
`script/cibuild` bootstraps, runs the gate phases, and then builds the image,
so a successful run means every check passed; a bare `docker build .` does not
carry the same guarantee, because its gate phases may come from the cache. The carry the same guarantee, because its gate phases may come from the cache. The
image build is uncached and so runs the gate phases a second time. That is the image build gets a `CACHEBUST` value of its own and so runs the gate phases a
price of the rule above, and it is worth paying: the image that ships is built second time. That is the price of the rule above, and it is worth paying: the
from a run of its own gates rather than from a cache entry. A separate image that ships is built from a run of its own gates rather than from a cache
workflow limited to `main` by a `branches` list under `on: push` cannot be entry. The `check` job sets `timeout-minutes: 20`, so a hung build frees the
checked by review: to try a change to it, add the feature branch to that list shared runner after 20 minutes. That allows for the three Docker builds
and push, then remove the branch from the list again before merging. Keep any described above (the test phase, the lint phase, then the image), each held to
job in it that publishes behind `if: github.ref_name == 'main'`, so the run the 5-minute Docker build limit below, plus the bootstrap. A separate workflow
from the feature branch publishes nothing. limited to `main` by a `branches` list under `on: push` cannot be checked by
review: to try a change to it, add the feature branch to that list and push,
then remove the branch from the list again before merging. Keep any job in it
that publishes behind `if: github.ref_name == 'main'`, so the run from the
feature branch publishes nothing.
- Use platform-standard formatters: `black` for Python, `prettier` for - Use platform-standard formatters: `black` for Python, `prettier` for
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
@@ -351,12 +415,11 @@ style conventions are in separate documents:
That cache is Go's own, separate from Docker's layer cache. Go stores a That cache is Go's own, separate from Docker's layer cache. Go stores a
passing result in its cache directory (`GOCACHE`), and when the same tests passing result in its cache directory (`GOCACHE`), and when the same tests
run again on unchanged code it prints that result, marked `(cached)`, run again on unchanged code it prints that result, marked `(cached)`,
without running them. That matters on a developer's machine, where this without running them. The directory lasts from one run to the next on a
target runs and the directory lasts from one run to the next. The `test` developer's machine, where this target runs, and in the `test` phase of the
phase of the `Dockerfile` needs no `-count=1`: its base image holds no `Dockerfile`, which mounts it to keep compiled packages between check
result for this repo's tests and nothing before its `go test` step runs a builds, so the phase's `go test` runs carry `-count=1` too. A new
test, so there is nothing to replay. `--no-cache` (above) is what makes that `CACHEBUST` value (above) is what makes that step run on an unchanged tree.
step run on an unchanged tree.
Python example: Python example:
@@ -384,9 +447,12 @@ style conventions are in separate documents:
- `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`), - `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`),
editor files (`.swp`, `*~`), in-repo agent scratch directories (`.claude/`), editor files (`.swp`, `*~`), in-repo agent scratch directories (`.claude/`),
language build artifacts, and `node_modules/`. Fetch the standard `.gitignore` `node_modules/`, and the repo's own build outputs. Fetch the standard
from `https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when `.gitignore` from
setting up a new repo. These patterns are written to `.gitignore`'s own `https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when setting up
a new repo. A repo's `.gitignore` is the standard file followed by the repo's
own entries, such as its binaries; a re-vendor replaces the standard part and
keeps those entries. These patterns are written to `.gitignore`'s own
semantics, in which an unanchored pattern already matches at every depth; they semantics, in which an unanchored pattern already matches at every depth; they
are not a `.dockerignore` and must not be transplanted into one unmodified. are not a `.dockerignore` and must not be transplanted into one unmodified.
@@ -434,13 +500,15 @@ style conventions are in separate documents:
byte-identically across repos: byte-identically across repos:
```sh ```sh
# Own line: a failing command substitution inside an argument does not # The version and the tag each get their own line: a failing command
# trip `set -e`, so the inline form degrades to an empty constant. # substitution inside an argument does not trip `set -e`, so the inline
# form degrades to an empty constant.
version="$(git describe --tags --always --dirty 2>/dev/null || true)" version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
docker build --no-cache \ tag="$(script/projectname)"
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--build-arg VERSION="$version" \ --build-arg VERSION="$version" \
-t "$(script/projectname)" . -t "$tag" .
``` ```
`--always` makes an untagged repo yield an abbreviated commit hash rather `--always` makes an untagged repo yield an abbreviated commit hash rather
@@ -452,8 +520,8 @@ style conventions are in separate documents:
there because `ARG` is stage-scoped; passing `VERSION` to a repo whose there because `ARG` is stage-scoped; passing `VERSION` to a repo whose
Dockerfile declares no such `ARG` is ignored and costs nothing, which is why Dockerfile declares no such `ARG` is ignored and costs nothing, which is why
the scripts stay byte-identical. One consequence for CI: the standard the scripts stay byte-identical. One consequence for CI: the standard
checkout action clones shallow and fetches no tags, so a repo that embeds a checkout action clones shallow and fetches no tags, so the canonical
tag-derived version must set `fetch-depth: 0` on its checkout step. `.gitea/workflows/check.yml` sets `fetch-depth: 0` on its checkout step.
- **Verify `.dockerignore` by enumerating the image, not by reading the - **Verify `.dockerignore` by enumerating the image, not by reading the
patterns.** Plant files at the root _and_ at least two directories deep, build patterns.** Plant files at the root _and_ at least two directories deep, build
@@ -520,6 +588,11 @@ style conventions are in separate documents:
function defined and never invoked has the same exit status and the same function defined and never invoked has the same exit status and the same
empty output as one that worked. empty output as one that worked.
Node is the exception to the whole-token comparison: the canonical
`script/bootstrap` compares only its major version, because the node 22
alpine image the `Dockerfile` stages start from is not the exact pin, and
nvm cannot install a prebuilt node on alpine.
Keep it POSIX sh: no arrays, no `[[`, no `grep -P`. Keep it POSIX sh: no arrays, no `[[`, no `grep -P`.
A Go tool a repo needs on the host is installed with `go install` pinned to A Go tool a repo needs on the host is installed with `go install` pinned to
@@ -636,7 +709,11 @@ style conventions are in separate documents:
Never edit existing migrations after release. Never edit existing migrations after release.
- All repos should have an `.editorconfig` enforcing the project's indentation - All repos should have an `.editorconfig` enforcing the project's indentation
settings. settings: the standard file from
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.editorconfig`, which sets
tabs for `Makefile` and Go files, followed by the repo's own sections, such as
one for another language it uses. A re-vendor replaces the standard part and
keeps those sections.
- Avoid putting files in the repo root unless necessary. Root should contain - Avoid putting files in the repo root unless necessary. Root should contain
only project-level config files (`README.md`, `AGENTS.md`, `Makefile`, only project-level config files (`README.md`, `AGENTS.md`, `Makefile`,
+190 -2
View File
@@ -8,8 +8,7 @@
# Status # Status
pre-1.0. No git tags. Open work is tracked on the issue tracker, which is pre-1.0. Open work is tracked on the issue tracker, which is authoritative.
authoritative.
# Next Step # Next Step
@@ -18,6 +17,195 @@ https://git.eeqj.de/sneak/secret/milestone/12
# Completed Steps # Completed Steps
- 2026-10-15: The canonical files are re-vendored from `sneak/prompts` commit
`e3f79c4` (https://git.eeqj.de/sneak/secret/issues/129), so Go's build cache
survives between check builds and `make test` no longer compiles the standard
library and every dependency with `-race` on every run. `script/lint`,
`script/test`, `script/cibuild` and `script/docker` pass a new `CACHEBUST`
value instead of `--no-cache`, and every stage of the `Dockerfile` declares
`ARG CACHEBUST` first, so every `RUN` step still reruns on an unchanged tree.
The steps that compile mount Go's build cache, and both `go test` runs keep
`-count=1`, so no test result is reused. `script/lint` and `script/test` write
no image. The CI workflow cancels a branch's older run, stops after 20
minutes, fetches the tags and leaves no token behind. `script/bootstrap` uses
the installed node only when its major version is the pinned one.
- 2026-10-15: `make test` no longer fails on some runs in `internal/cli`
(https://git.eeqj.de/sneak/secret/issues/192). `cli.Entry()` set `time.Local`
to UTC and ended by wiping every memguard buffer, and a test ran it in the
same process as the other tests. A timer left by an earlier test could read
`time.Local` as it changed, which the race detector reported, and the wipe
could destroy a buffer still in use, the likeliest cause of a crash of the
test process on another run. `main()` in `cmd/secret` now sets `time.Local`
before it calls `cli.Entry()`, and that test runs `cli.Entry()` in a process
of its own.
- 2026-10-15: `secret mv` from one vault to another unlocks both vaults before
it takes the state directory lock
(https://git.eeqj.de/sneak/secret/issues/189). It unlocked them while holding
the lock, and unlocking can ask for a passphrase, so every other command that
changes anything waited at that prompt. `moveSecretCrossVault` now checks the
move, refusing an existing destination secret without `--force`, unlocks both
vaults, and only then takes the lock. Under the lock, it checks that the
destination vault still has the long-term key it unlocked, and
`copySecretAndRemoveSource` checks the move again before it copies. A move
within a vault takes the lock in `moveSecretWithinVault`, so `MoveSecret`
takes none itself. The error for a vault created again with another key, which
`secret unlocker add` shares, now ends "nothing was changed".
- 2026-10-15: `secret init`, `secret vault create`, `secret unlocker add`, and
`secret add`, `secret import` and `secret generate secret` with `--force` ask
for the mnemonic or passphrase, or unlock the vault, before they take the
state directory lock (https://git.eeqj.de/sneak/secret/issues/152). They did
so while holding the lock, so every other command that changes anything waited
at the prompt. Under the lock, `secret unlocker add` fails if the vault was
removed, or created again with another key, and refuses a GPG key added
meanwhile; a forced add of a secret created meanwhile lets go of the lock,
gets the key and takes the lock again. `secret.CreatePGPUnlocker` takes the
unlocked vault; `SetMnemonic` and `SetUnlockPassphrase` are gone.
- 2026-10-15: A `secret mv` to another vault that copies the secret but cannot
remove the source fails (https://git.eeqj.de/sneak/secret/issues/159). It
printed a warning and the usual "Moved" line and exited 0. If the source
secret's directory is still in place, the error says the copy holds all
versions and the source remains. If the source left its vault and only the
temporary directory it was moved into could not be deleted, the error says the
secret is moved and names that directory, which the error of `RemoveDirAtomic`
now does.
- 2026-10-15: `secret encrypt` and `secret decrypt` with `--output` hold the
result in memory and open the output file only once encryption or decryption
has succeeded (https://git.eeqj.de/sneak/secret/issues/143). They emptied that
file before reading any input, so a failed command left it empty, and
encrypting a file onto itself gave the encryption of nothing. A failure to
encrypt or decrypt now leaves `--output` untouched, and `--input` can name the
same file. A failure while writing `--output`, such as a full disk, can still
leave it empty or partly written; an error from closing it is reported.
Without `--output` the result still goes to stdout as it is produced.
- 2026-10-15: A command that uses the current vault fails when `currentvault` is
empty or names no vault (https://git.eeqj.de/sneak/secret/issues/156).
`secret list` reported an empty vault with exit status 0, and `secret add`
failed on a missing directory. `ResolveVaultSymlink` now refuses an empty or
invalid name, or one with no directory under `vaults.d`, with an error that
wraps `ErrVaultNotFound`, names the file and says to run
`secret vault select <name>`. `secret vault list` and `secret info` still run.
- 2026-10-15: `secret` prints every time in UTC, whatever the machine's time
zone (https://git.eeqj.de/sneak/secret/issues/169). `secret list`,
`secret version list`, `secret unlocker list` and `secret info` printed times
in the local time zone without naming it, so one vault showed different times
on two machines. `Entry()` now sets `time.Local` to UTC before anything else,
as policy 6 in `AGENTS.md` asks, so every time the program prints or writes is
in UTC. `--json` times end in `Z`, and a new version's name takes the UTC
date. The README says so.
- 2026-10-15: `secret vault`, `secret unlocker`, `secret version` and
`secret generate` fail when given a subcommand they do not have
(https://git.eeqj.de/sneak/secret/issues/183). They printed their help and
exited 0, so a mistyped command looked like one that worked. Each now exits 1
with `unknown command "<name>" for "secret <parent>"`, and with no arguments
still prints its help. Cobra checks a command's `Args` only when it has a
`RunE`, so each gets `Args: cobra.NoArgs` and a `RunE` that prints its help;
their help now shows a `secret <parent> [flags]` usage line.
`secret vault help` fails as an unknown subcommand, while `secret help vault`
and `secret vault --help` still work.
- 2026-10-15: With `SB_UNLOCK_PASSPHRASE` set, `secret unlocker add passphrase`
asks on the terminal for the new unlocker's passphrase and uses the variable
to unlock the vault (https://git.eeqj.de/sneak/secret/issues/154). It gave the
new unlocker the passphrase in the variable without asking. With no
controlling terminal, the new unlocker takes the variable's passphrase and the
output says so. `secret.ErrNoTerminal` is exported so that the command can
tell a missing terminal from other failures to read, such as nothing typed or
a confirmation that does not match, which still fail the command.
- 2026-10-15: `secret init` and `secret vault create` stop with an error naming
the file when a vault's `vault-metadata.json` cannot be read or parsed, and
leave the state directory as it was
(https://git.eeqj.de/sneak/secret/issues/140). `GetNextDerivationIndex` passed
over such a vault, so the next vault made from the same mnemonic could get its
derivation index and so the same long-term key. A vault directory with no
metadata file is still skipped.
- 2026-10-15: The README says that `secret add` drops one trailing newline from
stdin, that `secret import` stores a file's bytes unchanged, and that
`secret get` writes the value with no newline added
(https://git.eeqj.de/sneak/secret/issues/158). It said none of this, so a PEM
key piped into `secret add` came back one byte short with no hint why. A new
test stores `a\n\n` through `add` and through `import` and checks that `add`
kept two bytes and `import` three. No program code changes.
- 2026-10-15: When gpg fails to decrypt, to encrypt or to look up a key's
fingerprint, the error ends with what gpg wrote to stderr, trimmed
(https://git.eeqj.de/sneak/secret/issues/146). It gave only gpg's exit status,
so the user could not tell why. `cmd.Output` keeps gpg's stderr in the
`*exec.ExitError` it returns, and the three errors now use it. When gpg wrote
nothing to stderr the error is as before, and when gpg wrote several lines the
error has them all.
- 2026-10-15: `secret version list` no longer shows a version without
`NOT_AFTER` as expired (https://git.eeqj.de/sneak/secret/issues/150). STATUS
was `expired` for every version other than the current one, so after a promote
the newest version, which has no `NOT_AFTER`, read `expired`. STATUS is now
`current` for the current version, `expired` for another version whose
`NOT_AFTER` has passed, and `inactive` for any other version. The README
describes the three.
- 2026-10-15: `secret unlocker list --json` for a vault with no unlockers, and
`secret vault list --json` when `vaults.d` holds no vaults, print the empty
list as `[]` instead of `null` (https://git.eeqj.de/sneak/secret/issues/148).
Both lists were nil slices, which `encoding/json` writes as `null`; each is
now made as an empty slice before it is filled. `vault.ListVaults` therefore
returns an empty slice instead of nil for an empty `vaults.d`.
- 2026-10-15: `secret add`, `secret import` and `secret generate secret` fail
with an error, not memguard's panic, when a secret is too large for the
locked-memory limit (https://git.eeqj.de/sneak/secret/issues/145). The value
was held twice, so half the usual 8 MiB limit was enough to crash them.
`secret` now raises the limit to the hard limit at startup, and each of the
three commands refuses, before its first locked allocation, a limit that
leaves fewer than 8 pages for its keys and buffers. The value is read a page
at a time into a `memguard.Stream`, which keeps it encrypted, and copied once
into locked memory after its size is checked against 100 MiB and the limit. It
leaves the stream through `Next`, not `Read`, because `Read` re-encrypts a
leftover piece and memguard hangs if that runs out of locked memory.
- 2026-10-15: Total Size in `secret info` counts every file of each version: the
value, the metadata and the version's two keys
(https://git.eeqj.de/sneak/secret/issues/147). It added up each version's
`data.age` and `metadata.age`, but a version keeps its value in `value.age`,
so no secret's value was counted. A version directory that cannot be read adds
nothing and gets a warning naming it.
- 2026-10-15: `secret vault import` is gone, with its tests and its README entry
(https://git.eeqj.de/sneak/secret/issues/139). It took only a vault without a
long-term key, and `secret init` and `secret vault create` give every vault
one, so it always failed; a mnemonic enters through those two commands. The
help and the note of `secret generate mnemonic` named `secret import`, which
imports a file as a secret, and now name `secret init` and
`secret vault create`.
- 2026-10-15: `secret version promote --help` says promote rewrites the
`current` file, not a "current symlink"
(https://git.eeqj.de/sneak/secret/issues/153). `current` is a plain file
holding the version name, as the README already said, and the comment in
`PromoteVersion` now says so too.
- 2026-10-14: `secret init` and `secret vault create` refuse a vault that
already exists before they ask for the mnemonic or the unlocker passphrase
(https://git.eeqj.de/sneak/secret/issues/151). They asked first, and
`secret init` printed "Initialized secrets manager at" before refusing. The
check `vault.CreateVault` made before writing is now
`vault.CheckNewVaultName`, which both commands call before the first prompt
and `CreateVault` still calls. It validates the name too, so
`secret vault create` with an invalid name also fails before the prompts.
`secret init` prints the "Initialized" line only once the vault is created.
- 2026-10-14: `secret get` fails when it cannot write the value
(https://git.eeqj.de/sneak/secret/issues/155). `GetSecret` and
`GetSecretWithVersion` discarded the result of writing it to stdout, so a full
disk or a failing stdout left an empty or cut-short file and exit status 0.
The write error is now returned, so the command prints it and exits 1. A
stdout the shell closed before `secret` started still exits 0, because the Go
runtime opens `/dev/null` in its place and the write there succeeds.
- 2026-10-14: `secret list` and `secret list --json` give each secret's own
creation and last-update times (https://git.eeqj.de/sneak/secret/issues/142).
They gave the time of the listing for every secret, because
`Secret.LoadMetadata` set both to the current time. It now reads file
modification times, so listing still needs no passphrase. The last update is
when the secret's `current` file was written. The creation is when its oldest
version's `value.age` was written, not that version's directory, whose time
changes when the next add rewrites its `metadata.age`. A move to another vault
gives the copied files the times of the originals. `GetSecretObject` no longer
loads the times, so `secret get` keeps its error for a missing `current` file.
- 2026-10-14: `secret unlocker add pgp` without `--keyid` uses the `default-key`
in `gpg.conf` (https://git.eeqj.de/sneak/secret/issues/141).
`getDefaultGPGKey` took that value from `gpgconf --list-options gpg` as
printed, with a leading `"` and percent-escaped, so gpg rejected the key ID.
The new `parseDefaultGPGKey` drops the `"` and percent-decodes the rest with
`url.PathUnescape`, which leaves `+` alone. A value that does not decode is an
error, not a fallback to the first secret key.
- 2026-10-14: A wrong passphrase, from `SB_UNLOCK_PASSPHRASE` or typed at the - 2026-10-14: A wrong passphrase, from `SB_UNLOCK_PASSPHRASE` or typed at the
prompt, gives the error `wrong passphrase for unlocker <id>` prompt, gives the error `wrong passphrase for unlocker <id>`
(https://git.eeqj.de/sneak/secret/issues/157). It used to be age's "no (https://git.eeqj.de/sneak/secret/issues/157). It used to be age's "no
+6
View File
@@ -3,10 +3,16 @@ package main
import ( import (
"os" "os"
"time"
"sneak.berlin/go/secret/internal/cli" "sneak.berlin/go/secret/internal/cli"
) )
func main() { func main() {
// Every time secret prints or writes is in UTC, whatever TZ or the
// system's time zone is set to. It is set here, before cli.Entry starts
// any goroutine that could read time.Local while it changes.
time.Local = time.UTC //nolint:gosmopolitan // this makes local time UTC
os.Exit(cli.Entry()) os.Exit(cli.Entry())
} }
+52 -13
View File
@@ -3,6 +3,7 @@ package cli_test
import ( import (
"bufio" "bufio"
"context" "context"
"fmt"
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
@@ -16,29 +17,67 @@ import (
"sneak.berlin/go/secret/internal/secret" "sneak.berlin/go/secret/internal/secret"
) )
// Entry must return its exit code rather than exit, so that its deferred const (
// memguard purge runs on the success and the error path alike. // envRunEntry, when set, makes this test binary run cli.Entry with its
// // own arguments and exit, instead of running the tests.
//nolint:paralleltest // sets os.Args, and Entry wipes every buffer in the process envRunEntry = "SECRET_TEST_RUN_ENTRY"
func TestEntryWipesBuffersAndReturnsExitCode(t *testing.T) {
savedArgs := os.Args
t.Cleanup(func() { os.Args = savedArgs }) // bufferWiped is what the test binary prints, run with envRunEntry set,
// when cli.Entry wiped the buffer made before it ran.
bufferWiped = "Entry wiped the buffer made before it ran"
)
// With envRunEntry set, the test binary makes a buffer, runs cli.Entry,
// says whether the buffer is wiped, and exits with Entry's exit code. This
// is done in init, not in a test, so that TestMain does not run in that
// process: it would build the secret binary again, and remove it at the end,
// while the other tests run it.
//
//nolint:gochecknoinits // must run before TestMain, as above
func init() {
if os.Getenv(envRunEntry) == "" {
return
}
buf := memguard.NewBufferFromBytes([]byte("key material"))
code := cli.Entry()
if !buf.IsAlive() {
fmt.Fprintln(os.Stderr, bufferWiped)
}
os.Exit(code)
}
// Entry must return its exit code rather than exit, so that its deferred
// memguard purge runs on the success and the error path alike. Entry wipes
// every memguard buffer in the process, including those other tests are
// using, so it runs in a process of its own: this test binary, run again
// with envRunEntry set.
func TestEntryWipesBuffersAndReturnsExitCode(t *testing.T) {
t.Parallel()
tests := []struct { tests := []struct {
args []string args []string
exitCode int exitCode int
}{ }{
{args: []string{"secret", "--help"}, exitCode: 0}, {args: []string{"--help"}, exitCode: 0},
{args: []string{"secret", "no-such-command"}, exitCode: 1}, {args: []string{"no-such-command"}, exitCode: 1},
} }
for _, tt := range tests { for _, tt := range tests {
buf := memguard.NewBufferFromBytes([]byte("key material")) //nolint:gosec // G204: test runs its own test binary
os.Args = tt.args cmd := exec.CommandContext(t.Context(), os.Args[0], tt.args...)
assert.Equal(t, tt.exitCode, cli.Entry(), "exit code for %v", tt.args) cmd.Env = append(os.Environ(), envRunEntry+"=1")
assert.False(t, buf.IsAlive(), "Entry left a buffer unwiped for %v", tt.args)
output, err := cmd.CombinedOutput()
require.NotNil(t, cmd.ProcessState, "%v did not run: %v", tt.args, err)
assert.Equal(t, tt.exitCode, cmd.ProcessState.ExitCode(),
"exit code for %v: %s", tt.args, output)
assert.Contains(t, string(output), bufferWiped,
"Entry left a buffer unwiped for %v", tt.args)
} }
} }
+1 -7
View File
@@ -175,12 +175,6 @@ func (cli *Instance) GenerateSecret(
return fmt.Errorf("failed to generate random secret: %w", err) return fmt.Errorf("failed to generate random secret: %w", err)
} }
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
// Store the secret in the vault // Store the secret in the vault
vlt, err := vault.GetCurrentVault(cli.fs, cli.stateDir) vlt, err := vault.GetCurrentVault(cli.fs, cli.stateDir)
if err != nil { if err != nil {
@@ -193,7 +187,7 @@ func (cli *Instance) GenerateSecret(
secretBuffer := memguard.NewBufferFromBytes([]byte(secretValue)) secretBuffer := memguard.NewBufferFromBytes([]byte(secretValue))
defer secretBuffer.Destroy() defer secretBuffer.Destroy()
err = vlt.AddSecret(secretName, secretBuffer, force) err = cli.storeSecret(vlt, secretName, secretBuffer, force)
if err != nil { if err != nil {
return err return err
} }
+33 -27
View File
@@ -67,36 +67,15 @@ func (cli *Instance) promptMnemonic() (*memguard.LockedBuffer, func(), error) {
return mnemonicBuffer, mnemonicBuffer.Destroy, nil return mnemonicBuffer, mnemonicBuffer.Destroy, nil
} }
// Init initializes the secret manager, holding the state directory lock // Init initializes the secret manager. It asks for the mnemonic and the
// while initialize runs // passphrase before it takes the state directory lock, so that no other
// command waits while the user answers, and holds the lock while
// initialize runs.
func (cli *Instance) Init(cmd *cobra.Command) error { func (cli *Instance) Init(cmd *cobra.Command) error {
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
return cli.initialize(cmd)
}
// initialize creates the state directory, the default vault and its first
// unlocker
func (cli *Instance) initialize(cmd *cobra.Command) error {
secret.Debug("Starting secret manager initialization") secret.Debug("Starting secret manager initialization")
// Create state directory
stateDir := cli.GetStateDir()
secret.DebugWith("Creating state directory", slog.String("path", stateDir))
err := cli.fs.MkdirAll(stateDir, secret.DirPerms)
if err != nil {
secret.Debug("Failed to create state directory", "error", err)
return fmt.Errorf("failed to create state directory: %w", err)
}
// Refuse an existing default vault before asking for anything // Refuse an existing default vault before asking for anything
err = vault.CheckNewVaultName(cli.fs, cli.stateDir, defaultVaultName) err := vault.CheckNewVaultName(cli.fs, cli.stateDir, defaultVaultName)
if err != nil { if err != nil {
return fmt.Errorf("failed to create default vault: %w", err) return fmt.Errorf("failed to create default vault: %w", err)
} }
@@ -135,11 +114,38 @@ func (cli *Instance) initialize(cmd *cobra.Command) error {
} }
defer cleanupPassphrase() defer cleanupPassphrase()
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
return cli.initialize(cmd, mnemonic, passphraseBuffer)
}
// initialize creates the state directory, and the default vault from
// mnemonic with a passphrase unlocker protected by passphrase. Another
// command may have created the vault since Init checked: vault.CreateVault
// checks again.
func (cli *Instance) initialize(
cmd *cobra.Command, mnemonic, passphrase *memguard.LockedBuffer,
) error {
// Create state directory
stateDir := cli.GetStateDir()
secret.DebugWith("Creating state directory", slog.String("path", stateDir))
err := cli.fs.MkdirAll(stateDir, secret.DirPerms)
if err != nil {
secret.Debug("Failed to create state directory", "error", err)
return fmt.Errorf("failed to create state directory: %w", err)
}
// Create the default vault with its passphrase unlocker // Create the default vault with its passphrase unlocker
secret.Debug("Creating default vault") secret.Debug("Creating default vault")
vlt, err := vault.CreateVault(cli.fs, cli.stateDir, defaultVaultName, vlt, err := vault.CreateVault(cli.fs, cli.stateDir, defaultVaultName,
mnemonic, passphraseBuffer) mnemonic, passphrase)
if err != nil { if err != nil {
secret.Debug("Failed to create default vault", "error", err) secret.Debug("Failed to create default vault", "error", err)
+328 -14
View File
@@ -280,9 +280,9 @@ func stateDirModTimes(t *testing.T, fs afero.Fs) map[string]int64 {
return modTimes return modTimes
} }
// setupEveryCommand makes what each command in // setupEveryCommand makes what the commands that the lock tests below run
// TestChangingCommandsWaitForLock needs: the current vault "work" with two // need: the current vault "work" with two versions of "test/secret", the
// versions of "test/secret", the vault "other", and the file testInput. // vault "other", and the file testInput.
// There is no vault "default", which init creates. If withUnlocker is set, // There is no vault "default", which init creates. If withUnlocker is set,
// it also gives "work" a passphrase unlocker, which is slow. It returns the // it also gives "work" a passphrase unlocker, which is slow. It returns the
// older version and the unlocker's ID. // older version and the unlocker's ID.
@@ -346,6 +346,25 @@ func waitingForLock() bool {
return false return false
} }
// requireWaitingForLock returns once a command waits for the state
// directory lock. It fails the test if the command finishes first, sending
// its error on done, or never waits.
func requireWaitingForLock(t *testing.T, done <-chan error) {
t.Helper()
timeout := time.After(lockWait)
for !waitingForLock() {
select {
case err := <-done:
t.Fatalf("finished while the lock was held, with error %v", err)
case <-timeout:
t.Fatal("never waited for the lock")
case <-time.After(time.Millisecond):
}
}
}
// requireWaitsForLock runs a command, given what setupEveryCommand made, // requireWaitsForLock runs a command, given what setupEveryCommand made,
// while holding the state directory lock. The command must neither finish // while holding the state directory lock. The command must neither finish
// nor change anything before it waits for the lock, and must succeed once // nor change anything before it waits for the lock, and must succeed once
@@ -385,17 +404,7 @@ func requireWaitsForLock(
go func() { done <- run(cli, olderVersion, unlockerID) }() go func() { done <- run(cli, olderVersion, unlockerID) }()
timeout := time.After(lockWait) requireWaitingForLock(t, done)
for !waitingForLock() {
select {
case err := <-done:
t.Fatalf("finished while the lock was held, with error %v", err)
case <-timeout:
t.Fatal("never waited for the lock")
case <-time.After(time.Millisecond):
}
}
assert.Equal(t, before, stateDirModTimes(t, fs), assert.Equal(t, before, stateDirModTimes(t, fs),
"changed the state directory before waiting for the lock") "changed the state directory before waiting for the lock")
@@ -472,6 +481,311 @@ func TestChangingCommandsWaitForLock(t *testing.T) {
} }
} }
// TestCommandsAskBeforeLocking is a regression test for
// https://git.eeqj.de/sneak/secret/issues/152: these commands took the
// state directory lock before asking for the mnemonic or a passphrase, so
// while one waited at its prompt, every other command that changes anything
// waited for it. Each runs here while the test holds the lock, given what
// takes it to the last step at which it can ask: the passphrase prompt of
// init and vault create, and the unlock of the vault, which asks for the
// passphrase of its passphrase unlocker, for the others. The tests have no
// terminal, so a prompt fails at once. Each command must get there, and
// fail, without waiting for the lock.
//
//nolint:paralleltest // waitingForLock sees any test's command waiting for the lock
func TestCommandsAskBeforeLocking(t *testing.T) {
newTestGPGKey(t)
mnemonic := testMnemonicBuffer(t)
wrongPassphrase := memguard.NewBufferFromBytes([]byte("wrong-passphrase"))
t.Cleanup(wrongPassphrase.Destroy)
for _, tc := range []struct {
name string
run func(cli *Instance) error
// want is wrapped in the error the command fails with
want error
}{
{"init", func(cli *Instance) error {
cli.Mnemonic = mnemonic
return cli.Init(cli.cmd)
}, secret.ErrNoTerminal},
{"vault create", func(cli *Instance) error {
cli.Mnemonic = mnemonic
return cli.CreateVault(cli.cmd, "created")
}, secret.ErrNoTerminal},
// With no terminal to ask for the new passphrase on, the new
// unlocker takes the one in SB_UNLOCK_PASSPHRASE, which then fails
// to unlock the vault
{"unlocker add passphrase", func(cli *Instance) error {
cli.UnlockPassphrase = wrongPassphrase
return cli.UnlockersAdd("passphrase", cli.cmd)
}, secret.ErrWrongPassphrase},
{"unlocker add pgp", func(cli *Instance) error {
cli.cmd.Flags().String("keyid", unreadableTestGPGUserID, "")
return cli.UnlockersAdd("pgp", cli.cmd)
}, secret.ErrNoTerminal},
{"add --force", func(cli *Instance) error {
return cli.AddSecret("test/secret", true)
}, secret.ErrNoTerminal},
{"import --force", func(cli *Instance) error {
return cli.ImportSecret(cli.cmd, "test/secret", testInput, true)
}, secret.ErrNoTerminal},
{"generate secret --force", func(cli *Instance) error {
return cli.GenerateSecret(cli.cmd, "test/secret", 16, "base58", true)
}, secret.ErrNoTerminal},
} {
t.Run(tc.name, func(t *testing.T) {
requireAsksBeforeLocking(t, tc.run, tc.want)
})
}
}
// startWhileLocked makes what setupEveryCommand makes, with the passphrase
// unlocker, on fs, an empty in-memory filesystem, takes the state directory
// lock there, and starts run, given a CLI instance on fs with "value" on
// stdin. It returns the function that releases the lock, which does so only
// the first time it is called, and the channel that run's error comes on.
// The lock is also released when the test ends, so that later tests can
// take it.
func startWhileLocked(
t *testing.T, fs afero.Fs, run func(cli *Instance) error,
) (func(), <-chan error) {
t.Helper()
setupEveryCommand(t, fs, true)
release, err := vault.LockStateDir(fs, testStateDir)
require.NoError(t, err)
release = sync.OnceFunc(release)
t.Cleanup(release)
cli := NewCLIInstanceWithStateDir(fs, testStateDir)
cli.cmd = &cobra.Command{}
cli.cmd.SetIn(strings.NewReader("value"))
cli.cmd.SetOut(io.Discard)
done := make(chan error, 1)
go func() { done <- run(cli) }()
return release, done
}
// requireAsksBeforeLocking starts run while holding the state directory
// lock, as startWhileLocked does. run must fail, with an error wrapping
// want, without waiting for the lock.
func requireAsksBeforeLocking(
t *testing.T, run func(cli *Instance) error, want error,
) {
t.Helper()
release, done := startWhileLocked(t, afero.NewMemMapFs(), run)
timeout := time.After(lockWait)
for !waitingForLock() {
select {
case err := <-done:
require.ErrorIs(t, err, want)
return
case <-timeout:
t.Fatal("neither failed nor waited for the lock")
case <-time.After(time.Millisecond):
}
}
// Let the command go on, so that it releases the lock
release()
select {
case <-done:
case <-time.After(lockWait):
}
t.Fatal("waited for the lock before asking")
}
// TestCommandsCheckAgainOnceLocked checks that commands which check what
// they change before they ask for anything, and so before they take the
// state directory lock, check again once they hold it: another command may
// have changed it in the meantime. Each runs while the test holds the lock,
// given what it needs to ask for nothing. Once it waits for the lock, the
// test makes the change another command could have made, then releases the
// lock. The command must fail, and change nothing.
//
//nolint:paralleltest // waitingForLock sees any test's command waiting for the lock
func TestCommandsCheckAgainOnceLocked(t *testing.T) {
fingerprint := newTestGPGKey(t)
mnemonic := testMnemonicBuffer(t)
otherMnemonic := memguard.NewBufferFromBytes([]byte("legal winner thank " +
"year wave sausage worth useful legal winner thank yellow"))
t.Cleanup(otherMnemonic.Destroy)
passphrase := memguard.NewBufferFromBytes([]byte(testPassphrase))
t.Cleanup(passphrase.Destroy)
// work returns the vault "work" on fs, opened with the mnemonic
work := func(fs afero.Fs) *vault.Vault {
vlt := vault.NewVault(fs, testStateDir, "work")
vlt.Mnemonic = mnemonic
return vlt
}
for _, tc := range []struct {
name string
run func(cli *Instance) error
// change is what another command changes while run waits
change func(fs afero.Fs) error
// want is wrapped in the error the command fails with
want error
}{
{
"unlocker add, vault removed",
func(cli *Instance) error {
cli.Mnemonic, cli.UnlockPassphrase = mnemonic, passphrase
return cli.UnlockersAdd("passphrase", cli.cmd)
},
func(fs afero.Fs) error {
return fs.RemoveAll(testStateDir + "/vaults.d/work")
},
vault.ErrVaultNotFound,
},
{
"unlocker add, vault created again from another mnemonic",
func(cli *Instance) error {
cli.Mnemonic, cli.UnlockPassphrase = mnemonic, passphrase
return cli.UnlockersAdd("passphrase", cli.cmd)
},
func(fs afero.Fs) error {
err := fs.RemoveAll(testStateDir + "/vaults.d/work")
if err != nil {
return err
}
_, err = vault.CreateVault(fs, testStateDir, "work",
otherMnemonic, passphrase)
return err
},
errVaultCreatedAgain,
},
{
"unlocker add pgp, key added",
func(cli *Instance) error {
cli.Mnemonic = mnemonic
cli.cmd.Flags().String("keyid", unreadableTestGPGUserID, "")
return cli.UnlockersAdd("pgp", cli.cmd)
},
func(fs afero.Fs) error {
_, err := secret.CreatePGPUnlocker(work(fs),
unreadableTestGPGUserID, fingerprint)
return err
},
errGPGKeyAlreadyUnlocker,
},
} {
t.Run(tc.name, func(t *testing.T) {
requireChecksAgainOnceLocked(t, tc.run, tc.change, tc.want)
})
}
}
// requireChecksAgainOnceLocked starts run while holding the state directory
// lock, as startWhileLocked does. Once run waits for the lock, change makes
// the change that another command could have made in the meantime, and the
// lock is released. run must then fail, with an error wrapping want, and
// change nothing.
func requireChecksAgainOnceLocked(
t *testing.T, run func(cli *Instance) error,
change func(fs afero.Fs) error, want error,
) {
t.Helper()
fs := afero.NewMemMapFs()
release, done := startWhileLocked(t, fs, run)
requireWaitingForLock(t, done)
require.NoError(t, change(fs))
// Left out: taking and releasing the lock writes the lock file
lockFile := testStateDir + "/lock"
before := stateDirModTimes(t, fs)
delete(before, lockFile)
release()
select {
case err := <-done:
require.ErrorIs(t, err, want)
case <-time.After(lockWait):
t.Fatal("did not finish once the lock was released")
}
after := stateDirModTimes(t, fs)
delete(after, lockFile)
assert.Equal(t, before, after, "changed the state directory")
}
// TestForcedAddReplacesSecretAddedMeanwhile runs a forced add of a secret
// that does not exist yet while the test holds the state directory lock.
// Once it waits for the lock, the test adds the secret, as another command
// could have, and releases the lock. The forced add must replace that
// secret, as if it had run after the other command.
//
//nolint:paralleltest // waitingForLock sees any test's command waiting for the lock
func TestForcedAddReplacesSecretAddedMeanwhile(t *testing.T) {
mnemonic := testMnemonicBuffer(t)
fs := afero.NewMemMapFs()
release, done := startWhileLocked(t, fs, func(cli *Instance) error {
cli.Mnemonic = mnemonic
return cli.AddSecret("added", true)
})
requireWaitingForLock(t, done)
work := vault.NewVault(fs, testStateDir, "work")
work.Mnemonic = mnemonic
other := memguard.NewBufferFromBytes([]byte("other"))
t.Cleanup(other.Destroy)
require.NoError(t, work.AddSecret("added", other, false))
release()
select {
case err := <-done:
require.NoError(t, err)
case <-time.After(lockWait):
t.Fatal("did not finish once the lock was released")
}
value, err := work.GetSecret("added")
require.NoError(t, err)
t.Cleanup(value.Destroy)
assert.Equal(t, "value", string(value.Bytes()))
}
// TestEncryptWithExistingKeyTakesNoLock checks that secret encrypt with a // TestEncryptWithExistingKeyTakesNoLock checks that secret encrypt with a
// key that already exists, which only reads the state directory, finishes // key that already exists, which only reads the state directory, finishes
// while another command holds the state directory lock. // while another command holds the state directory lock.
+76 -58
View File
@@ -4,7 +4,6 @@ package cli
import ( import (
"errors" "errors"
"path/filepath" "path/filepath"
"sync"
"testing" "testing"
"time" "time"
@@ -230,68 +229,87 @@ func TestCrossVaultMoveUnlocksBeforeLocking(t *testing.T) {
} }
} }
// TestCrossVaultMoveChecksAgainOnceLocked starts a move between two vaults, // TestCrossVaultMoveChecksAgainOnceLocked runs a move between vaults, given
// given the mnemonic, while the test holds the state directory lock. Once // the mnemonic, as TestCommandsCheckAgainOnceLocked runs the commands in its
// the move waits for the lock, the test removes what another command could // table: once the move waits for the state directory lock, the test removes
// have removed in the meantime, then releases the lock. The move must fail // the source secret or the destination vault, or creates the destination
// with the error that says what is missing. // vault again from another mnemonic, and the move must then fail and change
// nothing.
// //
//nolint:paralleltest // waitingForLock sees any test's command waiting for the lock //nolint:paralleltest // waitingForLock sees any test's command waiting for the lock
func TestCrossVaultMoveChecksAgainOnceLocked(t *testing.T) { func TestCrossVaultMoveChecksAgainOnceLocked(t *testing.T) {
for _, tc := range []struct { mnemonic := testMnemonicBuffer(t)
name string
// removed is the directory, under the state directory, that the
// test removes
removed string
want error
}{
{"source", "/vaults.d/default/secrets.d/x", vault.ErrSecretNotFound},
{"destination vault", "/vaults.d/work", vault.ErrVaultNotFound},
} {
t.Run(tc.name, func(t *testing.T) {
fs := afero.NewMemMapFs()
createCrossVaults(t, fs)
release, err := vault.LockStateDir(fs, testStateDir) otherMnemonic := memguard.NewBufferFromBytes([]byte("legal winner thank " +
require.NoError(t, err) "year wave sausage worth useful legal winner thank yellow"))
t.Cleanup(otherMnemonic.Destroy)
// Released at most once, and also if the test fails while move := func(cli *Instance) error {
// holding it, so that later tests can take it cli.Mnemonic = mnemonic
release = sync.OnceFunc(release)
defer release()
instance := NewCLIInstanceWithStateDir(fs, testStateDir) return cli.MoveSecret(cli.cmd, "work:test/secret", "other:moved", false)
instance.Mnemonic = testMnemonicBuffer(t)
done := make(chan error, 1)
go func() {
done <- instance.MoveSecret(&cobra.Command{},
"default:x", "work:y", false)
}()
timeout := time.After(lockWait)
for !waitingForLock() {
select {
case err := <-done:
t.Fatalf("finished while the lock was held, with error %v",
err)
case <-timeout:
t.Fatal("never waited for the lock")
case <-time.After(time.Millisecond):
}
}
require.NoError(t, fs.RemoveAll(testStateDir+tc.removed))
release()
select {
case err := <-done:
require.ErrorIs(t, err, tc.want)
case <-time.After(lockWait):
t.Fatal("did not finish once the lock was released")
}
})
} }
t.Run("source removed", func(t *testing.T) {
requireChecksAgainOnceLocked(t, move, func(fs afero.Fs) error {
return fs.RemoveAll(testStateDir + "/vaults.d/work/secrets.d/test%secret")
}, vault.ErrSecretNotFound)
})
t.Run("destination vault removed", func(t *testing.T) {
requireChecksAgainOnceLocked(t, move, func(fs afero.Fs) error {
return fs.RemoveAll(testStateDir + "/vaults.d/other")
}, vault.ErrVaultNotFound)
})
t.Run("destination vault created again from another mnemonic", func(t *testing.T) {
// Set by the move, which has finished once
// requireChecksAgainOnceLocked returns
var moveErr error
requireChecksAgainOnceLocked(t, func(cli *Instance) error {
moveErr = move(cli)
return moveErr
}, func(fs afero.Fs) error {
err := fs.RemoveAll(testStateDir + "/vaults.d/other")
if err != nil {
return err
}
_, err = vault.CreateVault(fs, testStateDir, "other", otherMnemonic, nil)
return err
}, errVaultCreatedAgain)
// secret unlocker add fails with the same error, so its message
// must fit a move too
require.EqualError(t, moveErr, "vault 'other' was removed and "+
"created again with another key while it was being unlocked; "+
"nothing was changed")
})
}
// TestCrossVaultMoveRefusesExistingSecretBeforeUnlocking checks that a move
// between vaults onto a secret that exists, without --force, is refused
// before either vault is unlocked, since unlocking can ask for a
// passphrase. The vaults have no unlockers and the move is given no
// mnemonic, so unlocking either vault fails.
func TestCrossVaultMoveRefusesExistingSecretBeforeUnlocking(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
createCrossVaults(t, fs)
work := vault.NewVault(fs, testStateDir, "work")
work.Mnemonic = testMnemonicBuffer(t)
err := work.AddSecret("y",
memguard.NewBufferFromBytes([]byte("other")), false)
require.NoError(t, err)
instance := NewCLIInstanceWithStateDir(fs, testStateDir)
err = instance.MoveSecret(&cobra.Command{}, "default:x", "work:y", false)
require.ErrorIs(t, err, vault.ErrSecretExists)
} }
-5
View File
@@ -2,7 +2,6 @@ package cli
import ( import (
"os" "os"
"time"
"github.com/awnumar/memguard" "github.com/awnumar/memguard"
"github.com/spf13/cobra" "github.com/spf13/cobra"
@@ -15,10 +14,6 @@ import (
// every memguard buffer before it returns, so the caller must do nothing // every memguard buffer before it returns, so the caller must do nothing
// but exit with the code. // but exit with the code.
func Entry() int { func Entry() int {
// Every time secret prints or writes is in UTC, whatever TZ or the
// system's time zone is set to.
time.Local = time.UTC //nolint:gosmopolitan // this makes local time UTC
raiseLockedMemoryLimit() raiseLockedMemoryLimit()
// On SIGINT or SIGTERM memguard runs this function, wipes every buffer // On SIGINT or SIGTERM memguard runs this function, wipes every buffer
+111 -33
View File
@@ -422,26 +422,92 @@ func (cli *Instance) AddSecret(secretName string, force bool) error {
// Locked only now that stdin has been read: in `secret encrypt key | // Locked only now that stdin has been read: in `secret encrypt key |
// secret add name`, holding the lock while reading would leave each // secret add name`, holding the lock while reading would leave each
// command waiting for the other. // command waiting for the other.
release, err := vault.LockStateDir(cli.fs, cli.stateDir) return cli.storeSecret(vlt, secretName, valueBuffer, force)
}
// storeSecret adds the secret name with value to vlt, as vault.AddSecret
// does, holding the state directory lock while it does. With force, adding
// an existing secret replaces its current version, which needs the vault's
// long-term key. Getting the key can ask for the passphrase, so storeSecret
// gets it before it takes the lock, and no other command waits while the
// user answers. If another command adds the secret while this one waits for
// the lock, storeSecret lets go of the lock, gets the key, and takes the
// lock again.
func (cli *Instance) storeSecret(
vlt *vault.Vault, name string, value *memguard.LockedBuffer, force bool,
) error {
err := vault.ValidateSecretName(name)
if err != nil { if err != nil {
return err return err
} }
replacing, err := replacesVersion(vlt, name, force)
if err != nil {
return err
}
if replacing {
_, err = vlt.GetOrDeriveLongTermKey()
if err != nil {
return fmt.Errorf("failed to get long-term key: %w", err)
}
}
needsKey, err := cli.addWithoutAsking(vlt, name, value, force)
if !needsKey {
return err
}
_, err = vlt.GetOrDeriveLongTermKey()
if err != nil {
return fmt.Errorf("failed to get long-term key: %w", err)
}
// With the key in memory, this adds the secret
_, err = cli.addWithoutAsking(vlt, name, value, force)
return err
}
// addWithoutAsking takes the state directory lock and adds the secret name
// with value to vlt, as vault.AddSecret does. If that would replace the
// secret's current version while the vault's long-term key is not in
// memory, getting the key could ask for the passphrase with the lock held:
// addWithoutAsking then adds nothing and returns true.
func (cli *Instance) addWithoutAsking(
vlt *vault.Vault, name string, value *memguard.LockedBuffer, force bool,
) (bool, error) {
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return false, err
}
defer release() defer release()
// Add the secret to the vault replacing, err := replacesVersion(vlt, name, force)
secret.Debug("Calling vault.AddSecret", "secret_name", secretName,
"value_length", valueBuffer.Size(), "force", force)
err = vlt.AddSecret(secretName, valueBuffer, force)
if err != nil { if err != nil {
secret.Debug("vault.AddSecret failed", "error", err) return false, err
return err
} }
secret.Debug("vault.AddSecret completed successfully") if replacing && vlt.Locked() {
return true, nil
}
return nil return false, vlt.AddSecret(name, value, force)
}
// replacesVersion reports whether adding the secret name to vlt replaces
// the secret's current version: whether force is set and the secret has one.
func replacesVersion(vlt *vault.Vault, name string, force bool) (bool, error) {
if !force {
return false, nil
}
exists, err := secret.NewSecret(vlt, name).Exists()
if err != nil {
return false, fmt.Errorf("failed to check if secret exists: %w", err)
}
return exists, nil
} }
// GetSecret retrieves and prints the current version of a secret // GetSecret retrieves and prints the current version of a secret
@@ -707,14 +773,7 @@ func (cli *Instance) ImportSecret(
// Locked only now that the file has been read, as in AddSecret: the // Locked only now that the file has been read, as in AddSecret: the
// file may be a pipe written by another secret command. // file may be a pipe written by another secret command.
release, err := vault.LockStateDir(cli.fs, cli.stateDir) err = cli.storeSecret(vlt, secretName, valueBuffer, force)
if err != nil {
return err
}
defer release()
// Store the secret in the vault
err = vlt.AddSecret(secretName, valueBuffer, force)
if err != nil { if err != nil {
return err return err
} }
@@ -1025,18 +1084,24 @@ func (cli *Instance) sameDirectory(dir, other string) (bool, error) {
// moveSecretCrossVault handles moving between two different vaults. Its // moveSecretCrossVault handles moving between two different vaults. Its
// caller, MoveSecret, has already checked both secret names and that both // caller, MoveSecret, has already checked both secret names and that both
// vaults exist. Copying needs the long-term keys of both vaults, and getting // vaults exist. The copy needs the source vault's long-term key and is
// one can ask for a passphrase, so it gets them before it takes the state // encrypted to the destination vault's long-term public key. The
// directory lock, and no other command waits while the user answers. // destination vault is unlocked to check that the copy will open with what
// the user gave, and, once the state directory lock is held, that it still
// has the key unlocked. Either unlock can ask for a passphrase, so both
// happen before the lock is taken, and no other command waits while the
// user answers.
func (cli *Instance) moveSecretCrossVault( func (cli *Instance) moveSecretCrossVault(
cmd *cobra.Command, cmd *cobra.Command,
srcVault *vault.Vault, srcSecretName string, srcVault *vault.Vault, srcSecretName string,
destVault *vault.Vault, destSecretName string, destVault *vault.Vault, destSecretName string,
force bool, force bool,
) error { ) error {
// Refuse a move that cannot be made before asking for anything // Before unlocking, check that the source secret exists, that the
// destination is not the source under another name, and, without
// --force, that the destination secret does not exist
_, err := cli.checkCrossVaultMove( _, err := cli.checkCrossVaultMove(
srcVault, srcSecretName, destVault, destSecretName) srcVault, srcSecretName, destVault, destSecretName, force)
if err != nil { if err != nil {
return err return err
} }
@@ -1045,7 +1110,7 @@ func (cli *Instance) moveSecretCrossVault(
destVault.Mnemonic, destVault.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase destVault.Mnemonic, destVault.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase
// Unlock destination vault (will fail if neither mnemonic nor unlocker available) // Unlock destination vault (will fail if neither mnemonic nor unlocker available)
_, err = destVault.GetOrDeriveLongTermKey() destIdentity, err := destVault.GetOrDeriveLongTermKey()
if err != nil { if err != nil {
return fmt.Errorf("failed to unlock destination vault '%s': %w", destVault.Name, err) return fmt.Errorf("failed to unlock destination vault '%s': %w", destVault.Name, err)
} }
@@ -1062,6 +1127,11 @@ func (cli *Instance) moveSecretCrossVault(
} }
defer release() defer release()
err = cli.checkLongTermKey(destVault, destIdentity)
if err != nil {
return err
}
return cli.copySecretAndRemoveSource( return cli.copySecretAndRemoveSource(
cmd, srcVault, srcSecretName, destVault, destSecretName, force) cmd, srcVault, srcSecretName, destVault, destSecretName, force)
} }
@@ -1078,7 +1148,7 @@ func (cli *Instance) copySecretAndRemoveSource(
force bool, force bool,
) error { ) error {
srcSecretDir, err := cli.checkCrossVaultMove( srcSecretDir, err := cli.checkCrossVaultMove(
srcVault, srcSecretName, destVault, destSecretName) srcVault, srcSecretName, destVault, destSecretName, force)
if err != nil { if err != nil {
return err return err
} }
@@ -1109,11 +1179,13 @@ func (cli *Instance) copySecretAndRemoveSource(
} }
// checkCrossVaultMove checks that the secret srcSecretName exists in // checkCrossVaultMove checks that the secret srcSecretName exists in
// srcVault, that destVault exists, and that destSecretName in destVault is // srcVault, that destSecretName in destVault is not the source under
// not the source under another name. It returns the source's directory. // another name, and, unless force is set, that destSecretName does not
// exist in destVault. It returns the source's directory.
func (cli *Instance) checkCrossVaultMove( func (cli *Instance) checkCrossVaultMove(
srcVault *vault.Vault, srcSecretName string, srcVault *vault.Vault, srcSecretName string,
destVault *vault.Vault, destSecretName string, destVault *vault.Vault, destSecretName string,
force bool,
) (string, error) { ) (string, error) {
srcVaultDir, err := srcVault.GetDirectory() srcVaultDir, err := srcVault.GetDirectory()
if err != nil { if err != nil {
@@ -1130,11 +1202,6 @@ func (cli *Instance) checkCrossVaultMove(
srcSecretName, vault.ErrSecretNotFound, srcVault.Name) srcSecretName, vault.ErrSecretNotFound, srcVault.Name)
} }
_, err = cli.existingVault(destVault.Name)
if err != nil {
return "", err
}
// The source is removed after the copy, so a destination that is the // The source is removed after the copy, so a destination that is the
// source under another name would be lost with it. // source under another name would be lost with it.
destVaultDir, err := destVault.GetDirectory() destVaultDir, err := destVault.GetDirectory()
@@ -1156,6 +1223,17 @@ func (cli *Instance) checkCrossVaultMove(
destVault.Name, destSecretName) destVault.Name, destSecretName)
} }
exists, err = afero.DirExists(cli.fs, destSecretDir)
if err != nil {
return "", fmt.Errorf("failed to check if destination secret exists: %w", err)
}
if exists && !force {
return "", fmt.Errorf(
"secret '%s' %w in vault '%s' (use --force to overwrite)",
destSecretName, vault.ErrSecretExists, destVault.Name)
}
return srcSecretDir, nil return srcSecretDir, nil
} }
+107 -20
View File
@@ -15,6 +15,7 @@ import (
"strings" "strings"
"time" "time"
"filippo.io/age"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"sneak.berlin/go/secret/internal/secret" "sneak.berlin/go/secret/internal/secret"
@@ -39,6 +40,10 @@ var (
// composes "GPG key <id> is already added as an unlocker". // composes "GPG key <id> is already added as an unlocker".
errGPGKeyAlreadyUnlocker = errors.New( errGPGKeyAlreadyUnlocker = errors.New(
"is already added as an unlocker") "is already added as an unlocker")
// errVaultCreatedAgain carries only the message tail; the caller
// composes "vault '<name>' was removed and created again ...".
errVaultCreatedAgain = errors.New("was removed and created again " +
"with another key while it was being unlocked; nothing was changed")
) )
// UnlockerInfo represents unlocker information for display // UnlockerInfo represents unlocker information for display
@@ -404,12 +409,6 @@ func (cli *Instance) printUnlockersTable(unlockers []UnlockerInfo) error {
// UnlockersAdd adds a new unlocker // UnlockersAdd adds a new unlocker
func (cli *Instance) UnlockersAdd(unlockerType string, cmd *cobra.Command) error { func (cli *Instance) UnlockersAdd(unlockerType string, cmd *cobra.Command) error {
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
switch unlockerType { switch unlockerType {
case unlockerTypePassphrase: case unlockerTypePassphrase:
return cli.addPassphraseUnlocker(cmd) return cli.addPassphraseUnlocker(cmd)
@@ -423,6 +422,64 @@ func (cli *Instance) UnlockersAdd(unlockerType string, cmd *cobra.Command) error
} }
} }
// unlockThenLock gets the long-term key of vlt, the vault an unlocker is
// added to, then takes the state directory lock and returns the function
// that releases it. Getting the key can ask for a passphrase, so it comes
// before the lock, as the question of askThenLock does, and no other
// command waits while the user answers. Once the lock is held, it fails if
// another command has removed the vault in the meantime, since writing the
// unlocker would create the vault's directory again, holding nothing else.
// It also fails if the vault was removed and created again with another
// long-term key, which the unlocker would then hold instead of the new
// vault's own.
func (cli *Instance) unlockThenLock(vlt *vault.Vault) (func(), error) {
ltIdentity, err := vlt.GetOrDeriveLongTermKey()
if err != nil {
return nil, fmt.Errorf("failed to get long-term key: %w", err)
}
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return nil, err
}
err = cli.checkLongTermKey(vlt, ltIdentity)
if err != nil {
release()
return nil, err
}
return release, nil
}
// checkLongTermKey checks, for the vault vlt unlocked as ltIdentity before
// the state directory lock was taken, that vlt still exists and that its
// long-term public key is still that of ltIdentity. Another command may have
// removed the vault in the meantime, or removed it and created it again with
// another long-term key. The caller holds the lock.
func (cli *Instance) checkLongTermKey(
vlt *vault.Vault, ltIdentity *age.X25519Identity,
) error {
_, err := cli.existingVault(vlt.GetName())
if err != nil {
return err
}
vaultDir, _ := vlt.GetDirectory()
ltPubKey, err := afero.ReadFile(cli.fs, filepath.Join(vaultDir, "pub.age"))
if err != nil {
return fmt.Errorf("failed to read long-term public key: %w", err)
}
if string(ltPubKey) != ltIdentity.Recipient().String() {
return fmt.Errorf("vault '%s' %w", vlt.GetName(), errVaultCreatedAgain)
}
return nil
}
// autoSelectUnlocker selects the newly created unlocker as current, // autoSelectUnlocker selects the newly created unlocker as current,
// printing a warning if selection fails // printing a warning if selection fails
func autoSelectUnlocker(cmd *cobra.Command, vlt *vault.Vault, unlockerID string) { func autoSelectUnlocker(cmd *cobra.Command, vlt *vault.Vault, unlockerID string) {
@@ -442,9 +499,6 @@ func (cli *Instance) addPassphraseUnlocker(cmd *cobra.Command) error {
return fmt.Errorf("failed to get current vault: %w", err) return fmt.Errorf("failed to get current vault: %w", err)
} }
// For passphrase unlockers, we don't need the vault to be unlocked
// The CreatePassphraseUnlocker method will handle getting the
// long-term key
vlt.Mnemonic, vlt.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase vlt.Mnemonic, vlt.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase
// The new passphrase is asked for even when SB_UNLOCK_PASSPHRASE is // The new passphrase is asked for even when SB_UNLOCK_PASSPHRASE is
@@ -464,6 +518,12 @@ func (cli *Instance) addPassphraseUnlocker(cmd *cobra.Command) error {
defer passphraseBuffer.Destroy() defer passphraseBuffer.Destroy()
} }
release, err := cli.unlockThenLock(vlt)
if err != nil {
return err
}
defer release()
passphraseUnlocker, err := vlt.CreatePassphraseUnlocker(passphraseBuffer) passphraseUnlocker, err := vlt.CreatePassphraseUnlocker(passphraseBuffer)
if err != nil { if err != nil {
return err return err
@@ -502,12 +562,13 @@ func (cli *Instance) addPGPUnlocker(cmd *cobra.Command) error {
cmd.Printf("Using default GPG key: %s\n", gpgKeyID) cmd.Printf("Using default GPG key: %s\n", gpgKeyID)
} }
// Check if this key is already added as an unlocker
vlt, err := vault.GetCurrentVault(cli.fs, cli.stateDir) vlt, err := vault.GetCurrentVault(cli.fs, cli.stateDir)
if err != nil { if err != nil {
return fmt.Errorf("failed to get current vault: %w", err) return fmt.Errorf("failed to get current vault: %w", err)
} }
vlt.Mnemonic, vlt.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase
// Resolve the GPG key ID to its fingerprint, once: the duplicate check // Resolve the GPG key ID to its fingerprint, once: the duplicate check
// and the new unlocker's metadata both use this result // and the new unlocker's metadata both use this result
fingerprint, err := secret.ResolveGPGKeyFingerprint(gpgKeyID) fingerprint, err := secret.ResolveGPGKeyFingerprint(gpgKeyID)
@@ -515,21 +576,26 @@ func (cli *Instance) addPGPUnlocker(cmd *cobra.Command) error {
return fmt.Errorf("failed to resolve GPG key fingerprint: %w", err) return fmt.Errorf("failed to resolve GPG key fingerprint: %w", err)
} }
// Check if this GPG key is already added // Refuse a key that is already an unlocker before asking for anything,
exists, err := cli.pgpUnlockerExists(vlt, fingerprint) // and again once the lock is held, since another command may have added
// it in the meantime
err = cli.checkPGPKeyNotAdded(vlt, gpgKeyID, fingerprint)
if err != nil { if err != nil {
return fmt.Errorf( return err
"could not check whether GPG key %s is already an unlocker: %w",
gpgKeyID, err,
)
} }
if exists { release, err := cli.unlockThenLock(vlt)
return fmt.Errorf("GPG key %s %w", gpgKeyID, errGPGKeyAlreadyUnlocker) if err != nil {
return err
}
defer release()
err = cli.checkPGPKeyNotAdded(vlt, gpgKeyID, fingerprint)
if err != nil {
return err
} }
pgpUnlocker, err := secret.CreatePGPUnlocker(cli.fs, cli.stateDir, pgpUnlocker, err := secret.CreatePGPUnlocker(vlt, gpgKeyID, fingerprint)
gpgKeyID, fingerprint, cli.Mnemonic, cli.UnlockPassphrase)
if err != nil { if err != nil {
return err return err
} }
@@ -704,6 +770,27 @@ func (cli *Instance) UnlockerSelect(unlockerID string) error {
return vlt.SelectUnlocker(unlockerID) return vlt.SelectUnlocker(unlockerID)
} }
// checkPGPKeyNotAdded fails when vlt already has a PGP unlocker for the GPG
// key gpgKeyID, whose fingerprint is fingerprint, or when that cannot be
// checked.
func (cli *Instance) checkPGPKeyNotAdded(
vlt *vault.Vault, gpgKeyID, fingerprint string,
) error {
exists, err := cli.pgpUnlockerExists(vlt, fingerprint)
if err != nil {
return fmt.Errorf(
"could not check whether GPG key %s is already an unlocker: %w",
gpgKeyID, err,
)
}
if exists {
return fmt.Errorf("GPG key %s %w", gpgKeyID, errGPGKeyAlreadyUnlocker)
}
return nil
}
// pgpUnlockerExists reports whether the vault already has a PGP unlocker // pgpUnlockerExists reports whether the vault already has a PGP unlocker
// for the GPG key with the given fingerprint. It returns an error, and no // for the GPG key with the given fingerprint. It returns an error, and no
// answer, when unlockers.d or an unlocker's metadata file cannot be read; // answer, when unlockers.d or an unlocker's metadata file cannot be read;
+6 -5
View File
@@ -9,11 +9,12 @@
// "nothing there" and go ahead. // "nothing there" and go ahead.
// //
// The tests make the look fail with a wrapper around the in-memory // The tests make the look fail with a wrapper around the in-memory
// filesystem, which the state directory lock refuses. So they call the // filesystem, which the state directory lock refuses. So they call a
// function each command runs once it holds the lock, such as addPGPUnlocker // function that makes the look before any lock is taken: addPGPUnlocker for
// for UnlockersAdd, or, for a removal, the function that makes its checks, // UnlockersAdd, which looks again once it holds the lock, or, for a removal,
// such as findVaultToRemove for RemoveVault, which runs again under the // the function that makes its checks, such as findVaultToRemove for
// lock before anything is removed, with --force or without. // RemoveVault, which runs again under the lock before anything is removed,
// with --force or without.
//nolint:testpackage // white-box test of unexported internals //nolint:testpackage // white-box test of unexported internals
package cli package cli
+12 -9
View File
@@ -222,18 +222,14 @@ func (cli *Instance) resolvePassphrase() (*memguard.LockedBuffer, func(), error)
return passphraseBuffer, passphraseBuffer.Destroy, nil return passphraseBuffer, passphraseBuffer.Destroy, nil
} }
// CreateVault creates a new vault // CreateVault creates a new vault. It asks for the mnemonic and the
// passphrase before it takes the state directory lock, so that no other
// command waits while the user answers.
func (cli *Instance) CreateVault(cmd *cobra.Command, name string) error { func (cli *Instance) CreateVault(cmd *cobra.Command, name string) error {
secret.Debug("Creating new vault", "name", name, "state_dir", cli.stateDir) secret.Debug("Creating new vault", "name", name, "state_dir", cli.stateDir)
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
// Refuse an invalid name or an existing vault before asking for anything // Refuse an invalid name or an existing vault before asking for anything
err = vault.CheckNewVaultName(cli.fs, cli.stateDir, name) err := vault.CheckNewVaultName(cli.fs, cli.stateDir, name)
if err != nil { if err != nil {
return err return err
} }
@@ -265,7 +261,14 @@ func (cli *Instance) CreateVault(cmd *cobra.Command, name string) error {
} }
defer cleanupPassphrase() defer cleanupPassphrase()
// Create the vault with its passphrase unlocker release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
// Create the vault with its passphrase unlocker. Another command may
// have created it since the check above: vault.CreateVault checks again.
vlt, err := vault.CreateVault(cli.fs, cli.stateDir, name, vlt, err := vault.CreateVault(cli.fs, cli.stateDir, name,
mnemonic, passphraseBuffer) mnemonic, passphraseBuffer)
if err != nil { if err != nil {
+1 -2
View File
@@ -375,8 +375,7 @@ func testCreatePGPUnlocker(
} }
// Now create a PGP unlock key (this will use our custom GPGEncryptFunc) // Now create a PGP unlock key (this will use our custom GPGEncryptFunc)
pgpUnlocker, err := secret.CreatePGPUnlocker( pgpUnlocker, err := secret.CreatePGPUnlocker(vlt, keyID, fingerprint)
fs, stateDir, keyID, fingerprint, mnemonic, nil)
if err != nil { if err != nil {
t.Fatalf("Failed to create PGP unlock key: %v", err) t.Fatalf("Failed to create PGP unlock key: %v", err)
} }
+14 -27
View File
@@ -192,58 +192,45 @@ func generatePGPUnlockerName() (string, error) {
return fmt.Sprintf("%s-pgp-%s", hostname, enrollmentTime), nil return fmt.Sprintf("%s-pgp-%s", hostname, enrollmentTime), nil
} }
// pgpUnlockerDir returns the current vault and the directory in it for a // pgpUnlockerDir returns the directory in vault for a new PGP unlocker,
// new PGP unlocker, named after the host and the time. // named after the host and the time.
// func pgpUnlockerDir(vault VaultInterface) (string, error) {
//nolint:ireturn // the vault is only available behind VaultInterface
func pgpUnlockerDir(
fs afero.Fs, stateDir string,
) (VaultInterface, string, error) {
// Get current vault
vault, err := GetCurrentVault(fs, stateDir)
if err != nil {
return nil, "", fmt.Errorf("failed to get current vault: %w", err)
}
// Generate the unlocker name based on hostname and time // Generate the unlocker name based on hostname and time
unlockerName, err := generatePGPUnlockerName() unlockerName, err := generatePGPUnlockerName()
if err != nil { if err != nil {
return nil, "", fmt.Errorf("failed to generate unlocker name: %w", err) return "", fmt.Errorf("failed to generate unlocker name: %w", err)
} }
vaultDir, err := vault.GetDirectory() vaultDir, err := vault.GetDirectory()
if err != nil { if err != nil {
return nil, "", fmt.Errorf("failed to get vault directory: %w", err) return "", fmt.Errorf("failed to get vault directory: %w", err)
} }
return vault, filepath.Join(vaultDir, "unlockers.d", unlockerName), nil return filepath.Join(vaultDir, "unlockers.d", unlockerName), nil
} }
// CreatePGPUnlocker creates a new PGP unlocker and stores it in the vault. // CreatePGPUnlocker creates a new PGP unlocker and stores it in vault.
// It encrypts to the GPG key gpgKeyID and records fingerprint, that key's // It encrypts to the GPG key gpgKeyID and records fingerprint, that key's
// fingerprint as ResolveGPGKeyFingerprint returns it, in the metadata. // fingerprint as ResolveGPGKeyFingerprint returns it, in the metadata.
// Everything that can fail short of writing a file is done before anything // Everything that can fail short of writing a file is done before anything
// is written, and the files are written through WriteDir, so a failure // is written, and the files are written through WriteDir, so a failure
// leaves no partial unlocker. The long-term key comes from mnemonic when it // leaves no partial unlocker. The long-term key comes from vault's
// is not nil, else from the current unlocker, which is given passphrase when // GetOrDeriveLongTermKey: the key it holds when it is unlocked, else the
// it is a passphrase unlocker. // key from its mnemonic or its current unlocker.
func CreatePGPUnlocker( func CreatePGPUnlocker(
fs afero.Fs, stateDir, gpgKeyID, fingerprint string, vault VaultInterface, gpgKeyID, fingerprint string,
mnemonic, passphrase *memguard.LockedBuffer,
) (*PGPUnlocker, error) { ) (*PGPUnlocker, error) {
err := checkGPGAvailable() err := checkGPGAvailable()
if err != nil { if err != nil {
return nil, err return nil, err
} }
vault, unlockerDir, err := pgpUnlockerDir(fs, stateDir) unlockerDir, err := pgpUnlockerDir(vault)
if err != nil { if err != nil {
return nil, err return nil, err
} }
// The vault's GetOrDeriveLongTermKey, in step 2, uses both fs := vault.GetFilesystem()
vault.SetMnemonic(mnemonic)
vault.SetUnlockPassphrase(passphrase)
// Step 1: Generate a new age keypair for the PGP unlocker // Step 1: Generate a new age keypair for the PGP unlocker
ageIdentity, err := age.GenerateX25519Identity() ageIdentity, err := age.GenerateX25519Identity()
@@ -295,7 +282,7 @@ func CreatePGPUnlocker(
func encryptPGPUnlockerKeys( func encryptPGPUnlockerKeys(
vault VaultInterface, ageIdentity *age.X25519Identity, gpgKeyID string, vault VaultInterface, ageIdentity *age.X25519Identity, gpgKeyID string,
) ([]byte, []byte, error) { ) ([]byte, []byte, error) {
// From the mnemonic or the current unlocker, as for a passphrase unlocker // The key vault holds, or else from its mnemonic or current unlocker
ltIdentity, err := vault.GetOrDeriveLongTermKey() ltIdentity, err := vault.GetOrDeriveLongTermKey()
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("failed to get long-term key: %w", err) return nil, nil, fmt.Errorf("failed to get long-term key: %w", err)
+5 -6
View File
@@ -58,7 +58,8 @@ func TestCreatePGPUnlockerFailureWritesNothing(t *testing.T) {
}} }}
_, err = secret.CreatePGPUnlocker( _, err = secret.CreatePGPUnlocker(
fs, testVaultStateDir, testGPGKeyID, testGPGFingerprint, nil, nil) vault.NewVault(fs, testVaultStateDir, testVaultName),
testGPGKeyID, testGPGFingerprint)
require.Error(t, err) require.Error(t, err)
vaultDir, err := vlt.GetDirectory() vaultDir, err := vlt.GetDirectory()
@@ -87,17 +88,15 @@ func TestPGPUnlockerAddedTwiceKeepsFirst(t *testing.T) {
fs := afero.NewMemMapFs() fs := afero.NewMemMapFs()
mnemonic := testMnemonicBuffer(t) mnemonic := testMnemonicBuffer(t)
_, err := vault.CreateVault(fs, testVaultStateDir, testVaultName, mnemonic, nil) vlt, err := vault.CreateVault(fs, testVaultStateDir, testVaultName, mnemonic, nil)
require.NoError(t, err) require.NoError(t, err)
first, err := secret.CreatePGPUnlocker( first, err := secret.CreatePGPUnlocker(vlt, testGPGKeyID, testGPGFingerprint)
fs, testVaultStateDir, testGPGKeyID, testGPGFingerprint, mnemonic, nil)
require.NoError(t, err) require.NoError(t, err)
firstFiles := dirFiles(t, fs, first.GetDirectory()) firstFiles := dirFiles(t, fs, first.GetDirectory())
second, err := secret.CreatePGPUnlocker( second, err := secret.CreatePGPUnlocker(vlt, testGPGKeyID, testGPGFingerprint)
fs, testVaultStateDir, testGPGKeyID, testGPGFingerprint, mnemonic, nil)
require.NoError(t, err) require.NoError(t, err)
assert.NotEqual(t, first.GetDirectory(), second.GetDirectory()) assert.NotEqual(t, first.GetDirectory(), second.GetDirectory())
assert.Equal(t, firstFiles, dirFiles(t, fs, first.GetDirectory())) assert.Equal(t, firstFiles, dirFiles(t, fs, first.GetDirectory()))
-5
View File
@@ -28,11 +28,6 @@ type VaultInterface interface {
GetFilesystem() afero.Fs GetFilesystem() afero.Fs
GetCurrentUnlocker() (Unlocker, error) GetCurrentUnlocker() (Unlocker, error)
GetOrDeriveLongTermKey() (*age.X25519Identity, error) GetOrDeriveLongTermKey() (*age.X25519Identity, error)
// SetMnemonic and SetUnlockPassphrase give GetOrDeriveLongTermKey the
// mnemonic to derive the long-term key from, and the passphrase for a
// current passphrase unlocker; nil for none.
SetMnemonic(mnemonic *memguard.LockedBuffer)
SetUnlockPassphrase(passphrase *memguard.LockedBuffer)
CreatePassphraseUnlocker( CreatePassphraseUnlocker(
passphrase *memguard.LockedBuffer) (*PassphraseUnlocker, error) passphrase *memguard.LockedBuffer) (*PassphraseUnlocker, error)
} }
-6
View File
@@ -118,12 +118,6 @@ func (m *MockVault) GetOrDeriveLongTermKey() (*age.X25519Identity, error) {
return nil, errNotImplementedInMock return nil, errNotImplementedInMock
} }
func (m *MockVault) SetMnemonic(mnemonic *memguard.LockedBuffer) {
m.mnemonic = mnemonic
}
func (m *MockVault) SetUnlockPassphrase(_ *memguard.LockedBuffer) {}
func (m *MockVault) CreatePassphraseUnlocker( func (m *MockVault) CreatePassphraseUnlocker(
_ *memguard.LockedBuffer, _ *memguard.LockedBuffer,
) (*PassphraseUnlocker, error) { ) (*PassphraseUnlocker, error) {
-4
View File
@@ -90,10 +90,6 @@ func (m *MockVersionVault) GetOrDeriveLongTermKey() (*age.X25519Identity, error)
return nil, errNotImplementedInMock return nil, errNotImplementedInMock
} }
func (m *MockVersionVault) SetMnemonic(_ *memguard.LockedBuffer) {}
func (m *MockVersionVault) SetUnlockPassphrase(_ *memguard.LockedBuffer) {}
func (m *MockVersionVault) CreatePassphraseUnlocker( func (m *MockVersionVault) CreatePassphraseUnlocker(
_ *memguard.LockedBuffer, _ *memguard.LockedBuffer,
) (*secret.PassphraseUnlocker, error) { ) (*secret.PassphraseUnlocker, error) {
+1 -1
View File
@@ -132,7 +132,7 @@ func TestGetSecretWithWrongMnemonic(t *testing.T) {
t.Cleanup(mnemonic.Destroy) t.Cleanup(mnemonic.Destroy)
vlt := vault.NewVault(created.GetFilesystem(), testStateDir, testVaultName) vlt := vault.NewVault(created.GetFilesystem(), testStateDir, testVaultName)
vlt.SetMnemonic(mnemonic) vlt.Mnemonic = mnemonic
_, err := vlt.GetSecret(testSecretName) _, err := vlt.GetSecret(testSecretName)
require.ErrorIs(t, err, vault.ErrMnemonicMismatch) require.ErrorIs(t, err, vault.ErrMnemonicMismatch)
+3 -3
View File
@@ -235,9 +235,9 @@ func processMnemonicForVault(
// vault: one wrapping ErrInvalidVaultName when it is not a valid vault name, // vault: one wrapping ErrInvalidVaultName when it is not a valid vault name,
// and one wrapping ErrVaultExists when that vault already exists. Creating an // and one wrapping ErrVaultExists when that vault already exists. Creating an
// existing vault again would replace its keys, and its secrets could no // existing vault again would replace its keys, and its secrets could no
// longer be decrypted. CreateVault checks this before writing anything; the // longer be decrypted. The commands that create a vault check it before they
// commands that create a vault also check it before they ask for the // ask for the mnemonic or passphrase, and CreateVault checks it again, under
// mnemonic or passphrase, holding the state directory lock. // the state directory lock, before writing anything.
func CheckNewVaultName(fs afero.Fs, stateDir string, name string) error { func CheckNewVaultName(fs afero.Fs, stateDir string, name string) error {
err := ValidateVaultName(name) err := ValidateVaultName(name)
if err != nil { if err != nil {
+1 -1
View File
@@ -336,7 +336,7 @@ func TestVaultVersionTimestampsAfterUnlock(t *testing.T) {
// A new instance is locked; AddSecret unlocks it with the passphrase // A new instance is locked; AddSecret unlocks it with the passphrase
vault := NewVault(fs, testStateDir, "test") vault := NewVault(fs, testStateDir, "test")
vault.SetUnlockPassphrase(passphrase) vault.UnlockPassphrase = passphrase
addTestSecretToVault(t, vault, testSecretPath, []byte("version-2"), true) addTestSecretToVault(t, vault, testSecretPath, []byte("version-2"), true)
require.False(t, fs.readAt.IsZero(), "AddSecret did not unlock the vault") require.False(t, fs.readAt.IsZero(), "AddSecret did not unlock the vault")
-12
View File
@@ -64,18 +64,6 @@ func (v *Vault) ClearLongTermKey() {
v.longTermKey = nil v.longTermKey = nil
} }
// SetMnemonic sets v.Mnemonic, for code that has v only as a
// secret.VaultInterface.
func (v *Vault) SetMnemonic(mnemonic *memguard.LockedBuffer) {
v.Mnemonic = mnemonic
}
// SetUnlockPassphrase sets v.UnlockPassphrase, for code that has v only as
// a secret.VaultInterface.
func (v *Vault) SetUnlockPassphrase(passphrase *memguard.LockedBuffer) {
v.UnlockPassphrase = passphrase
}
// GetOrDeriveLongTermKey gets the long-term key from memory or derives it // GetOrDeriveLongTermKey gets the long-term key from memory or derives it
// from available sources // from available sources
func (v *Vault) GetOrDeriveLongTermKey() (*age.X25519Identity, error) { func (v *Vault) GetOrDeriveLongTermKey() (*age.X25519Identity, error) {
+1 -1
View File
@@ -128,7 +128,7 @@ func TestAddSecretForceWithWrongPassphraseChangesNothing(t *testing.T) {
t.Cleanup(wrong.Destroy) t.Cleanup(wrong.Destroy)
vlt := vault.NewVault(fs, testStateDir, testVaultName) vlt := vault.NewVault(fs, testStateDir, testVaultName)
vlt.SetUnlockPassphrase(wrong) vlt.UnlockPassphrase = wrong
secretDir := filepath.Join(testStateDir, "vaults.d", testVaultName, secretDir := filepath.Join(testStateDir, "vaults.d", testVaultName,
"secrets.d", testSecretName) "secrets.d", testSecretName)
+1
View File
@@ -1,4 +1,5 @@
{ {
"private": true,
"devDependencies": { "devDependencies": {
"prettier": "3.8.1" "prettier": "3.8.1"
} }
+46 -26
View File
@@ -2,18 +2,18 @@
# script/bootstrap: install all dependencies needed to build and develop # script/bootstrap: install all dependencies needed to build and develop
# this repo. Idempotent: every install is guarded by a check so already # this repo. Idempotent: every install is guarded by a check so already
# installed tools are skipped. Base tooling comes from nix, apt, brew, # installed tools are skipped. Base tooling comes from nix, apt, brew,
# or apk (detected in that order); assumes NOTHING is present (not git, # or apk (detected in that order); assumes nothing is present. The
# make, node, yarn, go, or python). Node is used directly if installed; # installed node is used only when its major version is the pinned one;
# otherwise a pinned version is installed via nvm (installing nvm # otherwise (node missing, or another major) the pinned version is
# itself first, from a hash-verified release archive, never curl | sh). # installed via nvm (installing nvm itself first, from a hash-verified
# release archive, never curl | sh), and yarn and the packages are
# installed under it.
# golangci-lint is never installed: script/lint runs it in docker. # golangci-lint is never installed: script/lint runs it in docker.
#
# Uncomment the language sections in main() that apply to this repo.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Pinned versions, 2026-07-06. Never "latest" or "lts"; exact versions. # Pinned versions, 2026-07-06
NODE_VERSION="22.17.0" NODE_VERSION="22.17.0"
NVM_VERSION="0.40.3" NVM_VERSION="0.40.3"
# sha256 of https://github.com/nvm-sh/nvm/archive/refs/tags/v0.40.3.tar.gz # sha256 of https://github.com/nvm-sh/nvm/archive/refs/tags/v0.40.3.tar.gz
@@ -22,6 +22,7 @@ YARN_VERSION="1.22.22"
PKGMGR="" PKGMGR=""
SUDO="" SUDO=""
APT_UPDATED=""
detect_pkgmgr() { detect_pkgmgr() {
[ -n "$PKGMGR" ] && return 0 [ -n "$PKGMGR" ] && return 0
@@ -50,7 +51,14 @@ pkg_install() {
detect_pkgmgr detect_pkgmgr
case "$PKGMGR" in case "$PKGMGR" in
nix) nix-env -iA "nixpkgs.$1" ;; nix) nix-env -iA "nixpkgs.$1" ;;
apt) $SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2" ;; apt)
# Package lists may be empty (fresh images); refresh once per run.
if [ -z "$APT_UPDATED" ]; then
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get update
APT_UPDATED=1
fi
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2"
;;
brew) brew install "$3" ;; brew) brew install "$3" ;;
apk) apk add --no-cache "$4" ;; apk) apk add --no-cache "$4" ;;
esac esac
@@ -82,7 +90,7 @@ nvm_sh() {
ensure_nvm() { ensure_nvm() {
[ -s "$HOME/.nvm/nvm.sh" ] && return 0 [ -s "$HOME/.nvm/nvm.sh" ] && return 0
# nvm prerequisites; nvm itself requires bash, so install it too # nvm prerequisites; nvm itself requires bash
if missing bash; then pkg_install bash bash bash bash; fi if missing bash; then pkg_install bash bash bash bash; fi
if missing curl; then pkg_install curl curl curl curl; fi if missing curl; then pkg_install curl curl curl curl; fi
if missing git; then pkg_install git git git git; fi if missing git; then pkg_install git git git git; fi
@@ -95,56 +103,68 @@ ensure_nvm() {
rm -rf "$tmp" rm -rf "$tmp"
} }
# True when the node on PATH has the pinned major version. Only the
# major is compared: the Dockerfile stages start from a node 22 alpine
# image whose exact version is not the pin, and nvm cannot install a
# prebuilt node on alpine. Another major is not used: the pinned yarn 1
# prints a deprecation warning under node 24. script/fmt and
# script/fmt-check carry this function unchanged, to run the yarn
# installed here.
node_is_pinned_major() {
if ! command -v node >/dev/null 2>&1; then return 1; fi
installed="$(node --version)"
installed="${installed#v}"
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
}
ensure_node() { ensure_node() {
if ! missing node; then return 0; fi if node_is_pinned_major; then
echo "bootstrap: using node $(node --version)"
return 0
fi
ensure_nvm ensure_nvm
nvm_sh "nvm install $NODE_VERSION" nvm_sh "nvm install $NODE_VERSION"
nvm_sh "nvm use $NODE_VERSION >/dev/null && \
echo \"bootstrap: using node \$(node --version)\""
} }
ensure_yarn() { ensure_yarn() {
if ! node_is_pinned_major; then
nvm_sh "nvm use $NODE_VERSION >/dev/null && corepack enable && \
corepack prepare yarn@$YARN_VERSION --activate"
return 0
fi
if ! missing yarn; then return 0; fi if ! missing yarn; then return 0; fi
if ! missing corepack; then if ! missing corepack; then
corepack enable corepack enable
corepack prepare "yarn@$YARN_VERSION" --activate corepack prepare "yarn@$YARN_VERSION" --activate
elif [ -s "$HOME/.nvm/nvm.sh" ]; then
nvm_sh "nvm use $NODE_VERSION >/dev/null && corepack enable && \
corepack prepare yarn@$YARN_VERSION --activate"
else else
npm install -g "yarn@$YARN_VERSION" npm install -g "yarn@$YARN_VERSION"
fi fi
} }
install_js_deps() { install_js_deps() {
if missing yarn && [ -s "$HOME/.nvm/nvm.sh" ]; then if node_is_pinned_major; then
yarn install --frozen-lockfile
else
nvm_sh "nvm use $NODE_VERSION >/dev/null && cd \"$ROOT\" && \ nvm_sh "nvm use $NODE_VERSION >/dev/null && cd \"$ROOT\" && \
yarn install --frozen-lockfile" yarn install --frozen-lockfile"
else
yarn install --frozen-lockfile
fi fi
} }
main() { main() {
cd "$ROOT" cd "$ROOT"
# Base tooling (every repo)
if missing git; then pkg_install git git git git; fi
if missing make; then pkg_install gnumake make make make; fi if missing make; then pkg_install gnumake make make make; fi
if missing git; then pkg_install git git git git; fi
# ---- JS / docs repos ----
# prettier, pinned in package.json and yarn.lock, formats the markdown
ensure_node ensure_node
ensure_yarn ensure_yarn
install_js_deps install_js_deps
# ---- Go repos ----
if missing go; then pkg_install go golang go go; fi if missing go; then pkg_install go golang go go; fi
go mod download go mod download
# ---- Python repos ----
# if missing python3; then pkg_install python3 python3 python3 python3; fi
# python3 -m venv .venv
# ./.venv/bin/pip install -e '.[dev]'
echo "bootstrap complete" echo "bootstrap complete"
} }
+13 -9
View File
@@ -2,9 +2,11 @@
# script/cibuild: run the CI build. It bootstraps first: a CI runner # script/cibuild: run the CI build. It bootstraps first: a CI runner
# checks out and runs this and nothing else, and script/fmt-check runs # checks out and runs this and nothing else, and script/fmt-check runs
# the formatter on the host, which a pristine checkout cannot do. # the formatter on the host, which a pristine checkout cannot do.
# --no-cache for the same reason as script/docker: the gate phases the # A new CACHEBUST value reruns every RUN step, since a cached gate phase
# final stage depends on are RUN steps, and a cached one is a check that # is a check that did not run; unlike --no-cache, it keeps Go's build cache.
# did not run. # This relies on every stage of the Dockerfile declaring ARG CACHEBUST
# first; a stage without that line is served from cache on an unchanged
# tree, with no warning.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -14,15 +16,17 @@ main() {
cd "$ROOT" cd "$ROOT"
"$SCRIPT_DIR/bootstrap" "$SCRIPT_DIR/bootstrap"
"$SCRIPT_DIR/check" "$SCRIPT_DIR/check"
# Own line: a failing command substitution inside an argument does # The version and the tag each get their own line: a failing
# not trip `set -e`, so the inline form degrades silently to an # command substitution inside an argument does not trip `set -e`,
# empty constant. The VERSION build argument takes precedence over # so the inline form degrades silently to an empty constant. The
# the version a build stage derives from the .git in the context. # VERSION build argument takes precedence over the version a build
# stage derives from the .git in the context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)" version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
docker build --no-cache \ tag="$("$SCRIPT_DIR/projectname")"
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--build-arg VERSION="$version" \ --build-arg VERSION="$version" \
-t "$("$SCRIPT_DIR/projectname")" . -t "$tag" .
} }
main "$@" main "$@"
+13 -8
View File
@@ -1,8 +1,11 @@
#!/bin/sh #!/bin/sh
# script/docker: build the Docker image tagged with the project name. # script/docker: build the Docker image tagged with the project name.
# Identical in all repos; the tag comes from script/projectname. # Identical in all repos; the tag comes from script/projectname.
# --no-cache because the gate phases the final stage depends on are RUN # A new CACHEBUST value reruns every RUN step, since a cached gate phase
# steps, and a cached one is a check that did not run. # is a check that did not run; unlike --no-cache, it keeps Go's build cache.
# This relies on every stage of the Dockerfile declaring ARG CACHEBUST
# first; a stage without that line is served from cache on an unchanged
# tree, with no warning.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -10,15 +13,17 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
# Own line: a failing command substitution inside an argument does # The version and the tag each get their own line: a failing
# not trip `set -e`, so the inline form degrades silently to an # command substitution inside an argument does not trip `set -e`,
# empty constant. The VERSION build argument takes precedence over # so the inline form degrades silently to an empty constant. The
# the version a build stage derives from the .git in the context. # VERSION build argument takes precedence over the version a build
# stage derives from the .git in the context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)" version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
docker build --no-cache \ tag="$("$SCRIPT_DIR/projectname")"
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--build-arg VERSION="$version" \ --build-arg VERSION="$version" \
-t "$("$SCRIPT_DIR/projectname")" . -t "$tag" .
} }
main "$@" main "$@"
+16 -6
View File
@@ -8,15 +8,25 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Must match the pin in script/bootstrap. # Must match the pin in script/bootstrap.
NODE_VERSION="22.17.0" NODE_VERSION="22.17.0"
# script/bootstrap installs node and yarn under nvm and leaves neither # True when the node on PATH has the pinned major version: the same
# on the PATH of the shell that called it, so resolve the pinned # function as in script/bootstrap, which uses it to decide where it
# toolchain here the way bootstrap's own install step does. nvm is a # installs yarn.
# bash script, hence the subshell. node_is_pinned_major() {
if ! command -v node >/dev/null 2>&1; then return 1; fi
installed="$(node --version)"
installed="${installed#v}"
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
}
# Run the yarn script/bootstrap installed: the yarn on PATH when the
# node on PATH has the pinned major version, and otherwise yarn under
# the pinned node in nvm, which bootstrap leaves off the PATH of the
# shell that called it. nvm is a bash script, hence the subshell.
run_yarn() { run_yarn() {
if command -v yarn >/dev/null 2>&1; then if node_is_pinned_major && command -v yarn >/dev/null 2>&1; then
exec yarn "$@" exec yarn "$@"
fi fi
if [ ! -s "$HOME/.nvm/nvm.sh" ]; then if node_is_pinned_major || [ ! -s "$HOME/.nvm/nvm.sh" ]; then
echo "fmt: no yarn; run script/bootstrap first" >&2 echo "fmt: no yarn; run script/bootstrap first" >&2
exit 1 exit 1
fi fi
+16 -6
View File
@@ -8,15 +8,25 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Must match the pin in script/bootstrap. # Must match the pin in script/bootstrap.
NODE_VERSION="22.17.0" NODE_VERSION="22.17.0"
# script/bootstrap installs node and yarn under nvm and leaves neither # True when the node on PATH has the pinned major version: the same
# on the PATH of the shell that called it, so resolve the pinned # function as in script/bootstrap, which uses it to decide where it
# toolchain here the way bootstrap's own install step does. nvm is a # installs yarn.
# bash script, hence the subshell. node_is_pinned_major() {
if ! command -v node >/dev/null 2>&1; then return 1; fi
installed="$(node --version)"
installed="${installed#v}"
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
}
# Run the yarn script/bootstrap installed: the yarn on PATH when the
# node on PATH has the pinned major version, and otherwise yarn under
# the pinned node in nvm, which bootstrap leaves off the PATH of the
# shell that called it. nvm is a bash script, hence the subshell.
run_yarn() { run_yarn() {
if command -v yarn >/dev/null 2>&1; then if node_is_pinned_major && command -v yarn >/dev/null 2>&1; then
exec yarn "$@" exec yarn "$@"
fi fi
if [ ! -s "$HOME/.nvm/nvm.sh" ]; then if node_is_pinned_major || [ ! -s "$HOME/.nvm/nvm.sh" ]; then
echo "fmt-check: no yarn; run script/bootstrap first" >&2 echo "fmt-check: no yarn; run script/bootstrap first" >&2
exit 1 exit 1
fi fi
-1
View File
@@ -1,7 +1,6 @@
#!/bin/sh #!/bin/sh
# script/install-precommit: install the git pre-commit hook that runs # script/install-precommit: install the git pre-commit hook that runs
# script/precommit. Our own extension to scripts-to-rule-them-all. # script/precommit. Our own extension to scripts-to-rule-them-all.
# Generic: needs no adaptation.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
+8 -5
View File
@@ -5,9 +5,12 @@
# make its answer untrustworthy. # make its answer untrustworthy.
# #
# The phase is not the last stage in the file, so it is built only when # The phase is not the last stage in the file, so it is built only when
# --target names it. --no-cache because a cached lint layer is a lint # --target names it. A new CACHEBUST value reruns every RUN step, since
# that did not run. The tag makes each build replace the previous image # a cached lint layer is a lint that did not run; unlike --no-cache, it
# instead of leaving a dangling one behind. # keeps Go's build cache. This relies on every stage of the Dockerfile
# declaring ARG CACHEBUST first; a stage without that line is served
# from cache on an unchanged tree, with no warning.
# --output type=cacheonly writes no image, since nothing uses one.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -15,9 +18,9 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
docker build --no-cache \ docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--target lint \ --target lint \
-t "$("$SCRIPT_DIR/projectname")-lint" . --output type=cacheonly .
} }
main "$@" main "$@"
+1 -2
View File
@@ -1,7 +1,6 @@
#!/bin/sh #!/bin/sh
# script/setup: set up the repo for development after a fresh clone: # script/setup: set up the repo for development after a fresh clone:
# installs dependencies (script/bootstrap) and the git pre-commit hook. # installs dependencies and the git pre-commit hook.
# Add any repo-specific initialization (db init, .env template) here.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
+8 -4
View File
@@ -2,8 +2,12 @@
# script/test: run the test suite. Testing is a phase of the Dockerfile # script/test: run the test suite. Testing is a phase of the Dockerfile
# and this builds that phase alone, on the same terms as script/lint: # and this builds that phase alone, on the same terms as script/lint:
# --target because a phase that is not the last stage is built only when # --target because a phase that is not the last stage is built only when
# named, --no-cache because a cached test layer is a test that did not # named, and a new CACHEBUST value because a cached test layer is a test
# run, and a tag so each build replaces the previous image. # that did not run; unlike --no-cache, it keeps Go's build cache. This
# relies on every stage of the Dockerfile declaring ARG CACHEBUST first;
# a stage without that line is served from cache on an unchanged tree,
# with no warning. --output type=cacheonly writes no image, since
# nothing uses one.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -11,9 +15,9 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
docker build --no-cache \ docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--target test \ --target test \
-t "$("$SCRIPT_DIR/projectname")-test" . --output type=cacheonly .
} }
main "$@" main "$@"