Author SHA1 Message Date
sneak 0982d45868 Ask for the new passphrase in unlocker add when the variable is set (closes #154)
check / check (push) Canceled after 0s
With SB_UNLOCK_PASSPHRASE set, `secret unlocker add passphrase` gave the
new unlocker the passphrase in the variable without asking. It now asks
on the terminal for the new passphrase and uses the variable to unlock
the vault. With no controlling terminal the new unlocker takes the
variable's passphrase, and the output says so.

secret.ErrNoTerminal is exported so that the command can tell a missing
terminal from other failures to read, such as nothing typed or a
mismatched confirmation, which still fail the command.

The in-process tests that add a passphrase unlocker with the variable
set rely on the Docker test phase having no controlling terminal; at a
terminal they would ask.

Model: opus-5-5
2026-10-15 03:28:16 +00:00
52 changed files with 1046 additions and 2153 deletions

No files matched your search

-3
View File
@@ -13,6 +13,3 @@ indent_style = tab
[*.go] [*.go]
indent_style = tab indent_style = tab
# This repository's own sections, such as one for another language it
# uses, go below this comment, and a re-vendor keeps them.
-11
View File
@@ -1,20 +1,9 @@
name: check name: check
on: [push] on: [push]
# Free the shared runner: a new push cancels only the same branch's older run.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs: jobs:
check: check:
runs-on: ubuntu-latest runs-on: ubuntu-latest
# Free the shared runner from a hung build.
timeout-minutes: 20
steps: steps:
# actions/checkout v4.2.2, 2026-02-22 # actions/checkout v4.2.2, 2026-02-22
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
# script/cibuild needs no token, so none is left in .git/config.
with:
persist-credentials: false
# All history and tags, so git describe finds the version tag.
fetch-depth: 0
- run: script/cibuild - run: script/cibuild
+1 -5
View File
@@ -27,7 +27,7 @@ node_modules/
# Environment files. `*.env` covers bare `.env` and the `prod.env` # Environment files. `*.env` covers bare `.env` and the `prod.env`
# convention. Only the templates `example.env` and `sample.env` are # convention. Only the templates `example.env` and `sample.env` are
# re-included below. A repository that commits any other template adds # re-included below. A repository that commits any other template adds
# its own negation at the end of this file, for example `!.env.example`. # its own negation after these lines, for example `!.env.example`.
*.[eE][nN][vV] *.[eE][nN][vV]
.[eE][nN][vV].* .[eE][nN][vV].*
.[eE][nN][vV][rR][cC] .[eE][nN][vV][rR][cC]
@@ -46,10 +46,6 @@ node_modules/
[iI][dD]_[eE][dD]25519 [iI][dD]_[eE][dD]25519
[iI][dD]_[eE][dD]25519_[sS][kK] [iI][dD]_[eE][dD]25519_[sS][kK]
# This repository's own entries, such as its build outputs, go below
# this comment, and a re-vendor keeps them. Anchor a binary built at the
# root: `/myapp`, never `myapp`, which also ignores `cmd/myapp/`.
# Go. /secret is the built binary, anchored so that it does not also match # Go. /secret is the built binary, anchored so that it does not also match
# the internal/secret/ package directory. # the internal/secret/ package directory.
*.log *.log
-2
View File
@@ -25,8 +25,6 @@ linters:
# silenced by disabling that name, not by enabling the successor. # silenced by disabling that name, not by enabling the successor.
- wsl # Deprecated, replaced by wsl_v5 - wsl # Deprecated, replaced by wsl_v5
- gomodguard # Deprecated, replaced by gomodguard_v2 - gomodguard # Deprecated, replaced by gomodguard_v2
# Misses findings at random in v2.14.0; back once a pinned release fixes it
- canonicalheader
settings: settings:
lll: lll:
line-length: 88 line-length: 88
+11 -21
View File
@@ -1,40 +1,33 @@
# Every stage declares ARG CACHEBUST first: script/ passes a new value
# on every build, which reruns every RUN step after it. The steps that
# compile mount Go's build cache, which survives from build to build.
# Lint phase. The linter is invoked directly rather than through `make # Lint phase. The linter is invoked directly rather than through `make
# lint` or `script/lint`, which are themselves a docker build. # lint` or `script/lint`, which are themselves a docker build.
# golangci/golangci-lint:v2.14.0 (Debian-based), 2026-09-24 # golangci/golangci-lint:v2.14.0 (Debian-based), 2026-09-24
FROM golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint FROM golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint
ARG CACHEBUST
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
COPY . . COPY . .
RUN --mount=type=cache,target=/root/.cache/go-build \ RUN go vet ./...
go vet ./... RUN golangci-lint run --config .golangci.yml ./...
RUN --mount=type=cache,target=/root/.cache/go-build \
golangci-lint run --config .golangci.yml ./...
# The same checks on the code as a macOS build compiles it, which a Linux # The same checks on the code as a macOS build compiles it, which a Linux
# build never compiles. Cgo is off, because compiling cgo code for macOS # build never compiles. Cgo is off, because compiling cgo code for macOS
# needs Apple's SDK headers. # needs Apple's SDK headers.
RUN --mount=type=cache,target=/root/.cache/go-build \ RUN GOOS=darwin CGO_ENABLED=0 go vet ./...
GOOS=darwin CGO_ENABLED=0 go vet ./... RUN GOOS=darwin CGO_ENABLED=0 golangci-lint run --config .golangci.yml ./...
RUN --mount=type=cache,target=/root/.cache/go-build \
GOOS=darwin CGO_ENABLED=0 golangci-lint run --config .golangci.yml ./...
# Test phase. -race needs cgo and so a C compiler, which the Debian Go # Test phase. -race needs cgo and so a C compiler, which the Debian Go
# image ships and the alpine one does not. -count=1 because the mounted # image ships and the alpine one does not.
# cache also holds test results, which go test would otherwise replay.
# golang:1.24.13-trixie, 2026-02-04 # golang:1.24.13-trixie, 2026-02-04
FROM golang@sha256:5835f052b784aa39f2fe9070def3568605c8bc3fcd810f10402066348b61e716 AS test FROM golang@sha256:5835f052b784aa39f2fe9070def3568605c8bc3fcd810f10402066348b61e716 AS test
ARG CACHEBUST
ENV CGO_ENABLED=1 ENV CGO_ENABLED=1
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
COPY . . COPY . .
RUN --mount=type=cache,target=/root/.cache/go-build \ # Go's build cache goes in a cache mount, not the image layer, which would
# take seconds longer to export. --no-cache, on every build in script/,
# starts the mount empty; -count=1 keeps a build without it from taking
# test results from there.
RUN --mount=type=cache,id=sneak/secret/go-build-test,target=/root/.cache/go-build \
go test -count=1 -timeout 90s -race -cover ./... || \ go test -count=1 -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \ { echo "--- Rerunning with -v for details ---"; \
go test -count=1 -timeout 90s -race -v ./...; exit 1; } go test -count=1 -timeout 90s -race -v ./...; exit 1; }
@@ -44,7 +37,6 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
# lint and test passed. # lint and test passed.
# golang 1.24.13-alpine, 2026-03-10 # golang 1.24.13-alpine, 2026-03-10
FROM golang@sha256:8bee1901f1e530bfb4a7850aa7a479d17ae3a18beb6e09064ed54cfd245b7191 AS builder FROM golang@sha256:8bee1901f1e530bfb4a7850aa7a479d17ae3a18beb6e09064ed54cfd245b7191 AS builder
ARG CACHEBUST
COPY --from=lint /src/go.sum /dev/null COPY --from=lint /src/go.sum /dev/null
COPY --from=test /src/go.sum /dev/null COPY --from=test /src/go.sum /dev/null
# script/build compiles with cgo, so it needs a C compiler too. # script/build compiles with cgo, so it needs a C compiler too.
@@ -61,8 +53,7 @@ COPY . .
# .git present, a version that is still empty, dev or unknown fails the # .git present, a version that is still empty, dev or unknown fails the
# build: git is missing or could not read the checkout. # build: git is missing or could not read the checkout.
ARG VERSION ARG VERSION
RUN --mount=type=cache,target=/root/.cache/go-build \ RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \
VERSION="${VERSION:-$(git describe --tags --always)}"; \
if [ -e .git ]; then \ if [ -e .git ]; then \
case "$VERSION" in ""|dev|unknown) \ case "$VERSION" in ""|dev|unknown) \
echo "version is '$VERSION' although .git is present" >&2; \ echo "version is '$VERSION' although .git is present" >&2; \
@@ -74,7 +65,6 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
# Runtime stage, and the last one # Runtime stage, and the last one
# alpine 3.23, 2026-03-10 # alpine 3.23, 2026-03-10
FROM alpine@sha256:25109184c71bdad752c8312a8623239686a9a2071e8825f20acb8f2198c3f659 FROM alpine@sha256:25109184c71bdad752c8312a8623239686a9a2071e8825f20acb8f2198c3f659
ARG CACHEBUST
RUN apk add --no-cache ca-certificates gnupg RUN apk add --no-cache ca-certificates gnupg
+11 -45
View File
@@ -67,10 +67,6 @@ and unlocker set. This allows for complete isolation between different contexts
## Commands Reference ## Commands Reference
Every time `secret` prints, in a table or in `--json` output, is in UTC,
whatever `TZ` or the system's time zone is set to. The date in a version's name
is the UTC date too.
### Confirmation Before Removal ### Confirmation Before Removal
`secret rm`, `secret version rm`, `secret vault remove` and `secret rm`, `secret version rm`, `secret vault remove` and
@@ -118,11 +114,7 @@ are allowed, and a name must not be empty, `.` or `..`.
#### `secret vault select <name>` #### `secret vault select <name>`
Switches to the specified vault for subsequent operations. When `currentvault` Switches to the specified vault for subsequent operations.
is empty or names a vault that does not exist, every command that uses the
current vault fails with an error that names the file and says to run
`secret vault select <name>`. `secret vault list` still lists the vaults, and
`secret info` still runs; both show no vault as current.
#### `secret vault remove <name> [--force]` / `secret vault rm` ⚠️ 🛑 #### `secret vault remove <name> [--force]` / `secret vault rm` ⚠️ 🛑
@@ -150,11 +142,7 @@ value may be over 100 MiB, whatever the limit.
#### `secret add <secret-name> [--force]` #### `secret add <secret-name> [--force]`
Adds a secret to the current vault. Reads the secret value from stdin. When the Adds a secret to the current vault. Reads the secret value from stdin.
value ends in a newline, that one newline is dropped, so
`echo value | secret add name` stores `value`; from `printf 'a\n\n'` it stores
`a` and one newline. To keep a file's final newline, as in a PEM key, use
`secret import`, which stores the file's bytes unchanged.
- `--force, -f`: Overwrite existing secret - `--force, -f`: Overwrite existing secret
@@ -167,13 +155,13 @@ contain `//`, or have `..` as a path segment.
#### `secret get <secret-name> [--version <version>]` #### `secret get <secret-name> [--version <version>]`
Writes the secret's value to stdout exactly as stored, with no newline added Retrieves and outputs a secret value to stdout. A failed write of the value,
after it. A failed write of the value, such as to a file on a full disk, prints such as to a file on a full disk, prints the error and exits 1, with two
the error and exits 1, with two exceptions. If the program reading from a pipe exceptions. If the program reading from a pipe has already exited, as in
has already exited, as in `secret get s | true`, the broken-pipe signal `secret get s | true`, the broken-pipe signal (SIGPIPE) ends `secret` with no
(SIGPIPE) ends `secret` with no message and the shell reports exit status 141. message and the shell reports exit status 141. If the shell closed stdout before
If the shell closed stdout before `secret` started (`secret get s >&-`), the `secret` started (`secret get s >&-`), the value is written to `/dev/null` and
value is written to `/dev/null` and `secret get` exits 0. `secret get` exits 0.
- `--version, -v`: Get a specific version (default: current) - `--version, -v`: Get a specific version (default: current)
@@ -318,16 +306,10 @@ Selects an unlocker as the current default for operations.
#### `secret import <secret-name> --source <filename>` #### `secret import <secret-name> --source <filename>`
Imports a secret from a file and stores it in the current vault under the given Imports a secret from a file and stores it in the current vault under the given
name. The file's bytes are stored unchanged, a final newline included. name.
### Encryption Operations ### Encryption Operations
With `--output`, `secret encrypt` and `secret decrypt` hold the result in memory
and write it to `--output` only once encryption or decryption has succeeded, so
a failure to encrypt or decrypt leaves `--output` untouched. A failure while
writing `--output`, such as a full disk, can leave it empty or partly written.
The command can need up to about four times as much memory as the result.
#### `secret encrypt <secret-name> [--input=file] [--output=file]` #### `secret encrypt <secret-name> [--input=file] [--output=file]`
Encrypts data using an Age key stored as a secret. If the secret doesn't exist, Encrypts data using an Age key stored as a secret. If the secret doesn't exist,
@@ -438,14 +420,6 @@ asks when stdin is a pipe, as in `echo value | secret add name --force` or
`cat file | secret encrypt key`. With no controlling terminal, as under cron or `cat file | secret encrypt key`. With no controlling terminal, as under cron or
in a CI job, the command fails and names the variable to set. in a CI job, the command fails and names the variable to set.
A command that changes anything locks the state directory, through its file
`lock`, and another such command waits until the lock is released.
`secret init`, `secret vault create`, `secret unlocker add`, `secret mv` from
one vault to another, and `secret add`, `secret import` and
`secret generate secret` with `--force`, ask for the mnemonic or passphrase
before they take the lock, so a prompt of theirs left open in one terminal holds
up no command in another.
## Security Features ## Security Features
### Encryption ### Encryption
@@ -582,11 +556,7 @@ secret decrypt encryption/mykey --input document.txt.age --output document.txt
### Vault Management ### Vault Management
- **Derivation Index**: Each vault uses a unique derivation index from the - **Derivation Index**: Each vault uses a unique derivation index from the
mnemonic, and thus a unique key pair; creating a vault (`secret init` and mnemonic, and thus a unique key pair
`secret vault create`) fails, with an error naming the file, when any vault's
`vault-metadata.json` exists but cannot be read or parsed; a vault directory
with no `vault-metadata.json` is skipped, so the index it uses can be given to
a new vault
- **Public Key Hash**: Double SHA-256 hash of the vault's public key; the same - **Public Key Hash**: Double SHA-256 hash of the vault's public key; the same
hash of the index-0 public key identifies vaults from the same mnemonic hash of the index-0 public key identifies vaults from the same mnemonic
- **Automatic Key Derivation**: When creating vaults with a mnemonic, keys are - **Automatic Key Derivation**: When creating vaults with a mnemonic, keys are
@@ -675,10 +645,6 @@ provide:
- `script/install-precommit` — install the git pre-commit hook that runs - `script/install-precommit` — install the git pre-commit hook that runs
`script/precommit` `script/precommit`
Every stage of the `Dockerfile` declares `ARG CACHEBUST` as its first line after
`FROM`. The `script/` builds pass a new `CACHEBUST` value on every build, which
reruns every `RUN` step and keeps Go's build cache; `--no-cache` would empty it.
## Features ## Features
- **Multiple Authentication Methods**: Supports passphrase and PGP unlockers - **Multiple Authentication Methods**: Supports passphrase and PGP unlockers
+79 -156
View File
@@ -1,6 +1,6 @@
--- ---
title: Repository Policies title: Repository Policies
last_modified: 2026-10-15 last_modified: 2026-10-04
--- ---
This document covers repository structure, tooling, and workflow standards. Code This document covers repository structure, tooling, and workflow standards. Code
@@ -54,39 +54,34 @@ style conventions are in separate documents:
`cibuild`. `script/bootstrap` installs all dependencies idempotently and `cibuild`. `script/bootstrap` installs all dependencies idempotently and
assumes nothing is present: base tools come from nix, apt, brew, or apk assumes nothing is present: base tools come from nix, apt, brew, or apk
(detected in that order; apt runs noninteractive). For node it uses the (detected in that order; apt runs noninteractive). For node it uses the
installed node only when its major version is the pinned one; otherwise (node installed node if present; otherwise it installs a PINNED node version via
missing, or another major, such as the node 24 the CI runner image ships) it nvm, first installing nvm itself if missing — from a hash-verified GitHub
installs the PINNED node version via nvm and installs yarn and the packages release archive (never `curl | sh`), with bash installed as an explicit
under it. nvm itself is installed first if missing, from a hash-verified prerequisite since nvm requires bash. yarn is then pinned via
GitHub release archive (never `curl | sh`), with bash installed as an explicit `corepack prepare yarn@<version> --activate`. Never install "latest" or "lts";
prerequisite since nvm requires bash. Only the major version is compared always exact versions. `script/cibuild` runs the CI build: it changes to the
because the `Dockerfile` stages start from a node image whose exact version is repo root, runs `script/bootstrap`, runs `script/check`, and builds the image
not the pin, and nvm cannot install a prebuilt node on alpine. yarn is pinned with the version; the Gitea workflow calls it. **`script/cibuild` runs
via `corepack prepare yarn@<version> --activate`. Never install "latest" or
"lts"; always exact versions. `script/cibuild` runs the CI build: it changes
to the repo root, runs `script/bootstrap`, runs `script/check`, and builds the
image with the version; the Gitea workflow calls it. **`script/cibuild` runs
`script/bootstrap` first**, because the workflow checks out the repo and runs `script/bootstrap` first**, because the workflow checks out the repo and runs
nothing else, while `script/fmt-check` runs the formatter on the host: on a nothing else, while `script/fmt-check` runs the formatter on the host: on a
pristine checkout with nothing installed the run dies there, after the pristine checkout with nothing installed the run dies there, after the
containerised gates have passed. **The bootstrap alone is not enough**: when containerised gates have passed. **The bootstrap alone is not enough**:
`script/bootstrap` installs node and yarn under nvm it leaves neither on the `script/bootstrap` installs node and yarn under nvm and leaves neither on the
`PATH` of the shell that called it, so a bare `yarn` either exits 127 or runs `PATH` of the shell that called it, so a bare `yarn` still exits 127. The host
under another node. The host entrypoints that need yarn — `script/fmt` and entrypoints that need yarn — `script/fmt` and `script/fmt-check` — therefore
`script/fmt-check` — therefore use the same test as `script/bootstrap`: when source nvm for the pinned node version before invoking it, exactly as
the node on `PATH` has the pinned major version they run the `yarn` on `PATH`, `script/bootstrap`'s own install step does. A runner carrying nothing but
and otherwise they load nvm through `$HOME/.nvm/nvm.sh` and run yarn under the docker and git then gets through `script/check`. Four further scripts are our
pinned node. A runner carrying nothing but docker and git then gets through own extensions to the standard: `script/check` runs `script/test`,
`script/check`. Four further scripts are our own extensions to the standard: `script/lint` and `script/fmt-check`; `script/precommit` is what the git
`script/check` runs `script/test`, `script/lint` and `script/fmt-check`; pre-commit hook runs, and it calls `script/check`; `script/install-precommit`
`script/precommit` is what the git pre-commit hook runs, and it calls installs the git pre-commit hook (the `make hooks` target shims to it); and
`script/check`; `script/install-precommit` installs the git pre-commit hook `script/projectname` (literally that filename) simply outputs the project's
(the `make hooks` target shims to it); and `script/projectname` (literally name. Scripts that need the name call `script/projectname` — e.g.
that filename) simply outputs the project's name. Scripts that need the name `script/docker` assembles its image tag from it — so those scripts stay
call `script/projectname` — e.g. `script/docker` assembles its image tag from byte-identical across all repos. Repo-type-specific pre-commit extras (e.g.
it — so those scripts stay byte-identical across all repos. Repo-type-specific `go mod tidy` verification in Go repos) belong in `script/precommit`, not in
pre-commit extras (e.g. `go mod tidy` verification in Go repos) belong in the hook itself. Model scripts are at
`script/precommit`, not in the hook itself. Model scripts are at
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README `https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
must document the provided scripts in an **Entrypoints** section (see the must document the provided scripts in an **Entrypoints** section (see the
README requirements below). README requirements below).
@@ -123,10 +118,8 @@ style conventions are in separate documents:
and nothing else: and nothing else:
```sh ```sh
docker build --build-arg CACHEBUST="$(date +%s)-$$" \ docker build --no-cache --target lint -t "$(script/projectname)-lint" .
--target lint --output type=cacheonly . docker build --no-cache --target test -t "$(script/projectname)-test" .
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--target test --output type=cacheonly .
``` ```
**A stage that is not the last one in the file is built only when the final **A stage that is not the last one in the file is built only when the final
@@ -136,15 +129,10 @@ style conventions are in separate documents:
plain `docker build .` builds the last stage alone and exits 0 having linted plain `docker build .` builds the last stage alone and exits 0 having linted
and tested nothing. and tested nothing.
**The gate builds write no image.** With `--output type=cacheonly` the phase **Every `docker build` in `script/` is tagged**, here and in
runs and a failing step fails the build, but the result is not exported. `script/cibuild` and `script/docker`. An untagged build leaves a dangling
Nothing uses those images, and writing one out is slow: a Go test phase's image behind on every invocation, on every developer host and every CI
image holds the toolchain. A build given neither `--output` nor `-t` writes runner; a tagged one replaces the previous image.
an untagged image and leaves it dangling, on every developer host and every
CI runner. `script/cibuild` and `script/docker` build the image that ships
and tag it, so each build replaces the previous image; each assigns the tag
on its own line before the build, so `set -e` stops it where
`script/projectname` fails.
Inside a phase the tool is invoked directly — `golangci-lint`, `go test`, Inside a phase the tool is invoked directly — `golangci-lint`, `go test`,
`eslint`, `prettier` — never through `make lint` or `script/test`, which are `eslint`, `prettier` — never through `make lint` or `script/test`, which are
@@ -159,33 +147,22 @@ style conventions are in separate documents:
one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs
exit non-zero with `parallel golangci-lint is running` — a status a caller exit non-zero with `parallel golangci-lint is running` — a status a caller
cannot tell from real findings. Both have produced wrong verdicts in this cannot tell from real findings. Both have produced wrong verdicts in this
org, in both directions. A container has its own golangci-lint cache, its org, in both directions. A container has its own cache, its own `TMPDIR` and
own `TMPDIR` and a digest-pinned binary, so neither is reachable. a digest-pinned binary, so neither is reachable.
- **Any build that runs checks is given a new `CACHEBUST` value.** Docker - **Any build that runs checks is built with `--no-cache`.** Docker invalidates
invalidates a `COPY` layer only when the copied content changes, so on an a `COPY` layer only when the copied content changes, so on an unchanged tree
unchanged tree the check `RUN` is served from cache, nothing executes, and the the check `RUN` is served from cache, nothing executes, and the build still
build still exits 0. Every `docker build` in `script/` therefore passes exits 0. Every `docker build` in `script/` therefore passes `--no-cache`:
`--build-arg CACHEBUST="$(date +%s)-$$"`, the time and the script's process `script/lint`, `script/test`, `script/cibuild` and `script/docker` are the
id, so no two builds share a value, not even the three in one four, and there is no fifth — `script/check` runs the two gate phases and
`script/cibuild`. `script/lint`, `script/test`, `script/cibuild` and `script/fmt-check`, and builds no image of its own. A bare `docker build .` is
`script/docker` are the four, and there is no fifth — `script/check` runs the not evidence that anything ran: a sub-second build reporting success is a
two gate phases and `script/fmt-check`, and builds no image of its own. Every cache hit, not a result. Never invalidate by pruning — `docker builder prune`
stage of the `Dockerfile` declares `ARG CACHEBUST` as its first line after and friends destroy a build cache shared with every other build on the host.
`FROM`, and a new value reruns every `RUN` step after that line; `COPY` steps When a check is added or changed, prove it works by planting a defect it must
are compared by content and need nothing. A stage without the `ARG` line is catch and watching the run fail on it, then revert the defect. A green run
served from cache on an unchanged tree, so a repository re-vendors the scripts alone shows neither that the check ran nor that it covers what it should.
and adds the `ARG` lines in one commit. The scripts do not pass `--no-cache`:
it reruns every step too, but it also empties every cache mount, and Go's
build cache, which the canonical Go `Dockerfile` below mounts so that compiled
packages survive between check builds, would start empty every time. A bare
`docker build .` is not evidence that anything ran: a sub-second build
reporting success is a cache hit, not a result. Never invalidate by pruning —
`docker builder prune` and friends destroy a build cache shared with every
other build on the host. When a check is added or changed, prove it works by
planting a defect it must catch and watching the run fail on it, then revert
the defect. A green run alone shows neither that the check ran nor that it
covers what it should.
- **The gate phases are separate stages, and the build stage depends on both.** - **The gate phases are separate stages, and the build stage depends on both.**
The lint phase is based on the `golangci/golangci-lint` image (pinned by The lint phase is based on the `golangci/golangci-lint` image (pinned by
@@ -194,42 +171,32 @@ style conventions are in separate documents:
`Dockerfile`: `Dockerfile`:
```dockerfile ```dockerfile
# Every stage declares ARG CACHEBUST first: script/ passes a new value
# on every build, which reruns every RUN step after it. The steps that
# compile mount Go's build cache, which survives from build to build.
# Lint phase # Lint phase
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD # golangci/golangci-lint:v2.x.x, YYYY-MM-DD
FROM golangci/golangci-lint@sha256:... AS lint FROM golangci/golangci-lint@sha256:... AS lint
ARG CACHEBUST
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
COPY . . COPY . .
RUN --mount=type=cache,target=/root/.cache/go-build \ RUN golangci-lint run --config .golangci.yml ./...
golangci-lint run --config .golangci.yml ./...
# Test phase. -race needs cgo and so a C compiler, which the Debian Go # Test phase. -race needs cgo and so a C compiler, which the Debian Go
# image ships and the alpine one does not. -count=1 because the mounted # image ships and the alpine one does not.
# cache also holds test results, which go test would otherwise replay.
# golang:1.x, YYYY-MM-DD # golang:1.x, YYYY-MM-DD
FROM golang@sha256:... AS test FROM golang@sha256:... AS test
ARG CACHEBUST
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
COPY . . COPY . .
RUN --mount=type=cache,target=/root/.cache/go-build \ RUN go test -timeout 90s -race -cover ./... || \
go test -count=1 -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \ { echo "--- Rerunning with -v for details ---"; \
go test -count=1 -timeout 90s -race -v ./...; exit 1; } go test -timeout 90s -race -v ./...; exit 1; }
# Build stage. Nothing is wanted from either phase above; the copies # Build stage. Nothing is wanted from either phase above; the copies
# are what make BuildKit build them first, so this stage cannot run # are what make BuildKit build them first, so this stage cannot run
# unless lint and test passed. # unless lint and test passed.
# golang:1.x-alpine, YYYY-MM-DD # golang:1.x-alpine, YYYY-MM-DD
FROM golang@sha256:... AS builder FROM golang@sha256:... AS builder
ARG CACHEBUST
COPY --from=lint /src/go.sum /dev/null COPY --from=lint /src/go.sum /dev/null
COPY --from=test /src/go.sum /dev/null COPY --from=test /src/go.sum /dev/null
RUN apk add --no-cache git RUN apk add --no-cache git
@@ -245,8 +212,7 @@ style conventions are in separate documents:
# .git present, a version that is still empty, dev or unknown fails the # .git present, a version that is still empty, dev or unknown fails the
# build: git is missing or could not read the checkout. # build: git is missing or could not read the checkout.
ARG VERSION ARG VERSION
RUN --mount=type=cache,target=/root/.cache/go-build \ RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \
VERSION="${VERSION:-$(git describe --tags --always)}"; \
if [ -e .git ]; then \ if [ -e .git ]; then \
case "$VERSION" in ""|dev|unknown) \ case "$VERSION" in ""|dev|unknown) \
echo "version is '$VERSION' although .git is present" >&2; \ echo "version is '$VERSION' although .git is present" >&2; \
@@ -259,7 +225,6 @@ style conventions are in separate documents:
# Runtime stage, and the last one # Runtime stage, and the last one
FROM alpine@sha256:... FROM alpine@sha256:...
ARG CACHEBUST
COPY --from=builder /app /usr/local/bin/app COPY --from=builder /app /usr/local/bin/app
ENTRYPOINT ["app"] ENTRYPOINT ["app"]
``` ```
@@ -274,16 +239,6 @@ style conventions are in separate documents:
- Keep the runtime stage last, and if you add a stage after it, give it the - Keep the runtime stage last, and if you add a stage after it, give it the
same two copies. A plain `docker build .` builds the last stage's chain same two copies. A plain `docker build .` builds the last stage's chain
and nothing else. and nothing else.
- Only Go's build cache is mounted (`/root/.cache/go-build`, where Go keeps
it for root), and only on the steps that compile: `golangci-lint run`,
both `go test` runs and `go build`. It keeps compiled packages between
check builds, and Go reuses one only when all its inputs are unchanged, so
it saves the compile without skipping a check. Nothing else is kept:
golangci-lint's own cache would replay stored findings in place of a lint
run, and Go builds from whatever files a kept module cache holds without
checking them again (only `go mod verify` does), so a file changed there
would go unnoticed. Go's build cache also holds test results, which is why
both `go test` runs carry `-count=1`.
- If the project uses `//go:embed` directives that reference build artifacts - If the project uses `//go:embed` directives that reference build artifacts
(e.g. a web frontend compiled in a separate stage), the lint phase must (e.g. a web frontend compiled in a separate stage), the lint phase must
create placeholder files so the embed directives resolve. Example: create placeholder files so the embed directives resolve. Example:
@@ -323,39 +278,20 @@ style conventions are in separate documents:
`.git` and the version still comes out empty, `dev` or `unknown`. A plain `.git` and the version still comes out empty, `dev` or `unknown`. A plain
`docker build .` with no build arguments must succeed; a Dockerfile that `docker build .` with no build arguments must succeed; a Dockerfile that
refuses an empty build argument drops that refusal and keeps the argument. refuses an empty build argument drops that refusal and keeps the argument.
A checkout whose `.git` is a file (a linked worktree, or a repository
checked out as a submodule) is the exception: that file points to a git
directory outside the build context, so the build cannot read the version
and a plain `docker build .` fails; pass the version with
`--build-arg VERSION=...`, as `script/docker` and `script/cibuild` already
do.
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that - Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
runs `script/cibuild` on push, and checks out the repo as its only other step, runs `script/cibuild` on push, and checks out the repo as its only other step.
with `persist-credentials: false`: `script/cibuild` needs no token, and That script bootstraps, runs the gate phases, and then builds the image, so a
without it the checkout leaves the job's token in `.git/config` for every successful run means every check passed; a bare `docker build .` does not
later step. The checkout step also sets `fetch-depth: 0`, which fetches the
tags `git describe` needs: by default it clones shallow with no tags, and a
tagged repository's CI build would stamp a bare short commit id. The
workflow's `concurrency` block groups runs by workflow and branch
(`${{ github.workflow }}-${{ github.ref }}`) with `cancel-in-progress: true`,
so a new push cancels the older run on the same branch, queued or running, and
no other: runs for replaced commits do not hold up the shared runner.
`script/cibuild` bootstraps, runs the gate phases, and then builds the image,
so a successful run means every check passed; a bare `docker build .` does not
carry the same guarantee, because its gate phases may come from the cache. The carry the same guarantee, because its gate phases may come from the cache. The
image build gets a `CACHEBUST` value of its own and so runs the gate phases a image build is uncached and so runs the gate phases a second time. That is the
second time. That is the price of the rule above, and it is worth paying: the price of the rule above, and it is worth paying: the image that ships is built
image that ships is built from a run of its own gates rather than from a cache from a run of its own gates rather than from a cache entry. A separate
entry. The `check` job sets `timeout-minutes: 20`, so a hung build frees the workflow limited to `main` by a `branches` list under `on: push` cannot be
shared runner after 20 minutes. That allows for the three Docker builds checked by review: to try a change to it, add the feature branch to that list
described above (the test phase, the lint phase, then the image), each held to and push, then remove the branch from the list again before merging. Keep any
the 5-minute Docker build limit below, plus the bootstrap. A separate workflow job in it that publishes behind `if: github.ref_name == 'main'`, so the run
limited to `main` by a `branches` list under `on: push` cannot be checked by from the feature branch publishes nothing.
review: to try a change to it, add the feature branch to that list and push,
then remove the branch from the list again before merging. Keep any job in it
that publishes behind `if: github.ref_name == 'main'`, so the run from the
feature branch publishes nothing.
- Use platform-standard formatters: `black` for Python, `prettier` for - Use platform-standard formatters: `black` for Python, `prettier` for
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
@@ -415,11 +351,12 @@ style conventions are in separate documents:
That cache is Go's own, separate from Docker's layer cache. Go stores a That cache is Go's own, separate from Docker's layer cache. Go stores a
passing result in its cache directory (`GOCACHE`), and when the same tests passing result in its cache directory (`GOCACHE`), and when the same tests
run again on unchanged code it prints that result, marked `(cached)`, run again on unchanged code it prints that result, marked `(cached)`,
without running them. The directory lasts from one run to the next on a without running them. That matters on a developer's machine, where this
developer's machine, where this target runs, and in the `test` phase of the target runs and the directory lasts from one run to the next. The `test`
`Dockerfile`, which mounts it to keep compiled packages between check phase of the `Dockerfile` needs no `-count=1`: its base image holds no
builds, so the phase's `go test` runs carry `-count=1` too. A new result for this repo's tests and nothing before its `go test` step runs a
`CACHEBUST` value (above) is what makes that step run on an unchanged tree. test, so there is nothing to replay. `--no-cache` (above) is what makes that
step run on an unchanged tree.
Python example: Python example:
@@ -447,12 +384,9 @@ style conventions are in separate documents:
- `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`), - `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`),
editor files (`.swp`, `*~`), in-repo agent scratch directories (`.claude/`), editor files (`.swp`, `*~`), in-repo agent scratch directories (`.claude/`),
`node_modules/`, and the repo's own build outputs. Fetch the standard language build artifacts, and `node_modules/`. Fetch the standard `.gitignore`
`.gitignore` from from `https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when setting up setting up a new repo. These patterns are written to `.gitignore`'s own
a new repo. A repo's `.gitignore` is the standard file followed by the repo's
own entries, such as its binaries; a re-vendor replaces the standard part and
keeps those entries. These patterns are written to `.gitignore`'s own
semantics, in which an unanchored pattern already matches at every depth; they semantics, in which an unanchored pattern already matches at every depth; they
are not a `.dockerignore` and must not be transplanted into one unmodified. are not a `.dockerignore` and must not be transplanted into one unmodified.
@@ -500,15 +434,13 @@ style conventions are in separate documents:
byte-identically across repos: byte-identically across repos:
```sh ```sh
# The version and the tag each get their own line: a failing command # Own line: a failing command substitution inside an argument does not
# substitution inside an argument does not trip `set -e`, so the inline # trip `set -e`, so the inline form degrades to an empty constant.
# form degrades to an empty constant.
version="$(git describe --tags --always --dirty 2>/dev/null || true)" version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
tag="$(script/projectname)" docker build --no-cache \
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--build-arg VERSION="$version" \ --build-arg VERSION="$version" \
-t "$tag" . -t "$(script/projectname)" .
``` ```
`--always` makes an untagged repo yield an abbreviated commit hash rather `--always` makes an untagged repo yield an abbreviated commit hash rather
@@ -520,8 +452,8 @@ style conventions are in separate documents:
there because `ARG` is stage-scoped; passing `VERSION` to a repo whose there because `ARG` is stage-scoped; passing `VERSION` to a repo whose
Dockerfile declares no such `ARG` is ignored and costs nothing, which is why Dockerfile declares no such `ARG` is ignored and costs nothing, which is why
the scripts stay byte-identical. One consequence for CI: the standard the scripts stay byte-identical. One consequence for CI: the standard
checkout action clones shallow and fetches no tags, so the canonical checkout action clones shallow and fetches no tags, so a repo that embeds a
`.gitea/workflows/check.yml` sets `fetch-depth: 0` on its checkout step. tag-derived version must set `fetch-depth: 0` on its checkout step.
- **Verify `.dockerignore` by enumerating the image, not by reading the - **Verify `.dockerignore` by enumerating the image, not by reading the
patterns.** Plant files at the root _and_ at least two directories deep, build patterns.** Plant files at the root _and_ at least two directories deep, build
@@ -588,11 +520,6 @@ style conventions are in separate documents:
function defined and never invoked has the same exit status and the same function defined and never invoked has the same exit status and the same
empty output as one that worked. empty output as one that worked.
Node is the exception to the whole-token comparison: the canonical
`script/bootstrap` compares only its major version, because the node 22
alpine image the `Dockerfile` stages start from is not the exact pin, and
nvm cannot install a prebuilt node on alpine.
Keep it POSIX sh: no arrays, no `[[`, no `grep -P`. Keep it POSIX sh: no arrays, no `[[`, no `grep -P`.
A Go tool a repo needs on the host is installed with `go install` pinned to A Go tool a repo needs on the host is installed with `go install` pinned to
@@ -709,11 +636,7 @@ style conventions are in separate documents:
Never edit existing migrations after release. Never edit existing migrations after release.
- All repos should have an `.editorconfig` enforcing the project's indentation - All repos should have an `.editorconfig` enforcing the project's indentation
settings: the standard file from settings.
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.editorconfig`, which sets
tabs for `Makefile` and Go files, followed by the repo's own sections, such as
one for another language it uses. A re-vendor replaces the standard part and
keeps those sections.
- Avoid putting files in the repo root unless necessary. Root should contain - Avoid putting files in the repo root unless necessary. Root should contain
only project-level config files (`README.md`, `AGENTS.md`, `Makefile`, only project-level config files (`README.md`, `AGENTS.md`, `Makefile`,
+581 -9
View File
@@ -2,27 +2,599 @@
- branch from `next` - branch from `next`
- do the Next Step: the next open issue in the `1.0.0` milestone - do the Next Step: the next open issue in the `1.0.0` milestone
- log it at the top of Completed Steps
- commit (`TODO.md` changes in the same commit as the work) - commit (`TODO.md` changes in the same commit as the work)
- push, and open a PR against `next` - push, and open a PR against `next`
# Status # Status
pre-1.0. Open work is tracked on the issue tracker, which is authoritative. pre-1.0. No git tags. Open work is tracked on the issue tracker, which is
authoritative.
# Next Step # Next Step
Take the next open issue in the `1.0.0` milestone: Take the next open issue in the `1.0.0` milestone:
https://git.eeqj.de/sneak/secret/milestone/12 https://git.eeqj.de/sneak/secret/milestone/12
# Completed Steps
- 2026-10-14: A wrong passphrase, from `SB_UNLOCK_PASSPHRASE` or typed at the
prompt, gives the error `wrong passphrase for unlocker <id>`
(https://git.eeqj.de/sneak/secret/issues/157). It used to be age's "no
identity matched any of the recipients", followed by advice to give the vault
a new unlocker from its mnemonic, which would have replaced an unlocker that
works. The error now wraps `secret.ErrWrongPassphrase` and carries no advice
to use the mnemonic. A `priv.age` that is not an age file still gets that
advice.
- 2026-10-14: The passphrase and mnemonic prompts ask on the controlling
terminal (`/dev/tty`) instead of stdin, so `echo v | secret add s1 --force`,
`cat file | secret encrypt key` and `cat file.age | secret decrypt key` ask
for the passphrase at a terminal instead of failing
(https://git.eeqj.de/sneak/secret/issues/144). Only a process with no
controlling terminal gets the error that names the variable to set. On SIGINT
or SIGTERM, `Entry()` restores the settings of that terminal, not of stdin,
since that is where the prompt turns echo off. `secret init` and
`secret vault create` no longer write a newline to stderr after the mnemonic
is typed, since the prompt ends that line on the terminal.
- 2026-10-14: `secret generate mnemonic`, `secret vault list` and
`secret unlocker list` print their output to stdout, as `secret list` does, so
it can be piped or captured (https://git.eeqj.de/sneak/secret/issues/138).
They printed with cobra's `cmd.Println` and `cmd.Printf`, which write to
stderr when no output writer is set, so `secret vault list --json | jq`
received nothing. "No versions found" from `secret version list` moved to
stdout too. Messages such as "Created vault", and the warning after the
mnemonic, stay on stderr.
- 2026-10-14: Removing the current unlocker no longer leaves the vault unable to
open through the unlockers it still has
(https://git.eeqj.de/sneak/secret/issues/137). `current-unlocker` went on
naming the removed unlocker, so every command that opened the vault failed,
with advice to use the mnemonic. `secret unlocker remove` of the current
unlocker now makes the first of the vault's other unlockers that
`secret unlocker select` accepts, in the order `secret unlocker list` shows,
current before removing it, and says so; when there is none, it also removes
`current-unlocker`. `vault.RemoveUnlocker` returns the ID of the unlocker it
made current. When `current-unlocker` names an unlocker that does not exist
and the vault has another that `secret unlocker select` accepts, the error
wraps `vault.ErrUnlockerNotFound` and says to run `secret unlocker select`,
without the advice to use the mnemonic.
- 2026-10-14: A `secret add --force` that cannot unlock the vault, given a wrong
passphrase or mnemonic, leaves the secret as it was
(https://git.eeqj.de/sneak/secret/issues/136). The new version used to be
written before the vault was unlocked to set `notAfter` on the version it
replaces, so each failed add left one more version that `secret version list`
showed. The vault is now unlocked, and that version's metadata read, before
anything is written. `generate secret --force` and `import --force` take the
same path.
- 2026-10-12: A mnemonic typed with extra spaces, tabs or line breaks gives the
same long-term key as its words separated by single spaces
(https://git.eeqj.de/sneak/secret/issues/135). The key was derived from the
text exactly as typed, so a vault made with a double or trailing space did not
open with the same words typed normally. `agehd.DeriveEntropy`, which every
use of the mnemonic goes through, now joins the words with single spaces and
puts them in Unicode NFKD, as BIP39 defines, before making the seed.
- 2026-10-12: The keychain and Secure Enclave unlockers are gone from the code
and the documents (https://git.eeqj.de/sneak/secret/issues/132). They were
research: neither works without macOS code signing and entitlements, which
`secret` does not have. The last commit that has them is `ef0ae90`, tagged
`v1.0.0-rc1`. `secret unlocker add` takes only `passphrase` and `pgp` on every
platform, `internal/macse` and those unlockers' files in `internal/secret` are
deleted, and `github.com/keybase/go-keychain` left `go.mod` and `go.sum`.
`secret` still builds and runs on macOS.
- 2026-10-07: The part of `github.com/tyler-smith/go-bip39` v1.1.0 that `secret`
uses is copied into `internal/bip39`, with upstream's `LICENSE` beside it,
because its repository no longer exists
(https://git.eeqj.de/sneak/secret/issues/122). Every import moved there, and
the module left `go.mod` and `go.sum`. No derived key or mnemonic changes.
- 2026-10-07: The canonical files are re-vendored from `sneak/prompts` commit
`dd4027b` (https://git.eeqj.de/sneak/secret/issues/121), with golangci-lint
v2.14.0 in the lint phase. Lint and test are phases of the `Dockerfile`, and
`script/lint` and `script/test` each build one with `--no-cache`;
`Dockerfile.lint` and `script/lint-darwin` are gone, and the lint phase runs
`go vet` and checks the macOS build too. The tests run on the Debian Go image
with cgo and the race detector, with the policy's 90-second timeout.
`script/cibuild` bootstraps, runs `script/check` and builds the image;
`CHECK_EPOCH` and the memlock ulimit are gone. `--no-cache` starts Go's build
cache mount empty, so `make test` compiles everything on every run. The rules
in `CLAUDE.md` that `AGENTS.md` lacked are in `AGENTS.md`, and `CLAUDE.md` is
deleted.
- 2026-10-06: `script/test` runs the tests with the race detector, a 30-second
timeout per package and coverage, as `REPO_POLICIES.md` requires
(https://git.eeqj.de/sneak/secret/issues/32). When they fail, it reruns them
verbosely for the details and then fails anyway, so a test that fails once and
passes on the retry no longer gives a green build. `go vet` still runs first,
and every `go test` keeps `-count=1`. The tests that gave the `secret` binary
a minute now give it 10 seconds, and the PGP unlocker test's 30-second timer
is 10 seconds, so a test that hangs fails with its own message before the
package's 30-second timeout ends every test in it.
- 2026-10-06: `make test` in `script/cibuild` no longer compiles the standard
library and every dependency from nothing on every build
(https://git.eeqj.de/sneak/secret/issues/124). The `Dockerfile` runs it and
`make build` with Go's build cache in a BuildKit cache mount, which docker
keeps between builds, so each compiles only what changed since the last build.
The mount has an id of its own, so other repositories' builds do not share it.
`script/test` passes `-count=1`, so every test runs on every build and no
result comes from Go's test cache. A build with an empty cache, such as the
first after docker's build cache is cleared, compiles everything in
`make test` as before.
- 2026-10-06: `TestRemoveIgnoresTerminalOnStdout` and
`TestRemoveAsksAtTerminalOnStdin` no longer wait until Go's test timeout
(https://git.eeqj.de/sneak/secret/issues/126). On Linux, `pty.Open` of
`github.com/creack/pty` v1.1.24 passed the address of a local variable to the
`ioctl` system call as a plain number, through a function call; when Go moved
the goroutine's stack in between, the kernel wrote the terminal's number to
the old place, and `pty.Open` opened `/dev/pts/0` instead of the terminal it
had created. `secret rm` then wrote to that other terminal, and the test read
a terminal no program had open, which never ends. `go.mod` now requires the
commit on that library's main branch that passes a pointer instead; no release
has it yet. Both tests stop reading the terminal when their one-minute context
ends and fail saying so.
- 2026-10-06: The tests run quickly with the race detector on
(https://git.eeqj.de/sneak/secret/issues/120). Most of their time went to
deriving keys from passphrases with scrypt, which is slow on purpose. The new
`secret.ScryptWorkFactor`, when not zero, replaces age's scrypt work factor
when a passphrase encrypts; the tests of `internal/secret`, `internal/vault`
and `internal/cli` set it to 1 before any test runs, and the program never
sets it; `TestGetCommandOutputsToStdout` checks that the passphrase unlocker
the built binary's `secret init` writes names age's work factor, 18.
`TestRemovalAsksWithoutHoldingLock` and `TestFailedCommandReleasesLock` no
longer run in parallel with other tests: each waits at most 10 seconds for the
in-memory lock that every test in the package shares, and other tests'
commands held it longer. `TestConcurrentAddsKeepEveryVersion`, which times
nothing, and `TestGetCommandOutputsToStdout`, which no longer sets an
environment variable its commands do not read, now run in parallel. The
`script/cibuild` comment no longer says that tests are skipped without its
memlock ulimit.
- 2026-10-05: No test stores a secret larger than 1 MiB
(https://git.eeqj.de/sneak/secret/issues/52). The size tests for `secret add`,
`secret import` and the stdin buffer no longer try 2 MB, 10 MB, 99 MB, 100 MB
or 101 MB secrets, and nothing tests that a secret over the 100 MB limit is
rejected; the limit itself is unchanged. With nothing large left, the size
tests no longer skip a case for want of locked memory.
- 2026-10-05: The Go module path is `sneak.berlin/go/secret`, as
`REPO_POLICIES.md` requires, not `git.eeqj.de/sneak/secret`
(https://git.eeqj.de/sneak/secret/issues/43). Every import uses it, as do the
`-X` flags in `script/build` that stamp the version and commit shown by
`secret info`, and the examples in `pkg/agehd/README.md` and
`pkg/bip85/README.md`. `go mod tidy` now lists `github.com/dustin/go-humanize`
and `github.com/fatih/color`, which `internal/cli` imports, as direct
requirements. Code that imported the old path must switch to the new one.
- 2026-10-04: `make fmt` formats every markdown file with prettier (4-space
tabs, `proseWrap: always`) as well as the Go code, and `make fmt-check` checks
both, as the model scripts in the `prompts` repo do
(https://git.eeqj.de/sneak/secret/issues/110). Prettier is pinned by hash in
`package.json` and `yarn.lock`, and `script/bootstrap` installs node, yarn and
prettier. The `Dockerfile` lint stage copies node and yarn from a node image
pinned by hash and runs `script/bootstrap`, so its `make fmt-check` fails the
build on an unformatted markdown file. Every markdown file was formatted once,
wording unchanged.
- 2026-10-04: A mnemonic that cannot be read, in `secret init` and
`secret vault create`, gives an error that names the mnemonic only
(https://git.eeqj.de/sneak/secret/issues/115). It is read with
`secret.ReadMnemonic`, whose every error wraps the new
`secret.ErrMnemonicNotRead`; before, it was read with `ReadPassphrase`, so the
message said "failed to read mnemonic: failed to read passphrase:" and advised
setting `SB_UNLOCK_PASSPHRASE`. Without a terminal it now says "failed to read
mnemonic: stdin is not a terminal (piped input or script). Please set the
SB_SECRET_MNEMONIC environment variable or run interactively". The passphrase
messages no longer repeat "cannot read passphrase" after "failed to read
passphrase:", and empty input gives "nothing was entered".
- 2026-10-04: A failure returns the same error value whichever command hits it
(https://git.eeqj.de/sneak/secret/issues/113). `internal/cli` no longer keeps
its own copies of `vault.ErrSecretNotFound`, `ErrVaultNotFound`,
`ErrVersionNotFound` and `ErrSecretExists`: `secret mv`, `rm`, `decrypt`,
`vault import`, `vault remove` and `version list`, `promote` and `rm` wrap the
`vault` errors. `errUnsupportedUnlockerType` is removed: `secret unlocker add`
gives `errInvalidUnlockerType` for an unknown type, whichever check rejects
it. Off macOS, adding a keychain or Secure Enclave unlocker returns the
`secret` package's error for it, not an `internal/cli` copy; on macOS, the
check that the system is macOS is gone, as it could never fail.
`secret vault import` gives `errInvalidMnemonicPhrase` for an invalid
mnemonic, as `init` and `vault create` do. `secret generate secret` gives
`errLengthTooSmall` for a length below 1 wherever it is checked, and
`errUnsupportedSecretType` for `--type mnemonic` too. `secret import` of a
file over 100MB wraps `errSecretTooLarge`, as `secret add` returns it.
`vault.ErrNilValueBuffer` is replaced by `secret.ErrNilValueBuffer`, which
`secret` already returned under another name. Messages are unchanged, except
that `secret decrypt` of a missing secret says "not found", as `secret get`
does, not "does not exist"; `vault import` of an invalid mnemonic says
"invalid BIP39 mnemonic phrase"; `--type mnemonic` says "unsupported type:
mnemonic (use 'secret generate mnemonic' instead)"; and a file too large to
import says
`failed to read secret from file <path>: secret too large: exceeds 100MB limit`.
Every error of `secret.ReadPassphrase` wraps `secret.ErrPassphraseNotRead`,
which supplies the words "failed to read passphrase" that its callers used to
add themselves; so two passphrases that differ now give only "passphrases do
not match", the words now follow "failed to read mnemonic:" and "failed to
read passphrase confirmation:", and a terminal read error no longer repeats
them. A GPG key the keyring does not hold gives `secret.ErrGPGKeyNotFound`,
found by gpg's status line for "No public key"; before, the message repeated
"failed to resolve GPG key fingerprint" and ended in gpg's exit status. The
keychain unlocker returns `errNilDataBuffer` for nil data; this and its test
build only on macOS with cgo and were only read. `bip85.ErrPasswordTooShort`
and `ErrEncodedTooShort` are removed with their checks: 64 bytes of entropy
always give 86 Base64 or 80 Base85 characters, the most a password length may
ask for. Tests that matched these errors' text use `errors.Is`.
- 2026-10-04: Tests check which error a failure returns with `errors.Is`, not by
matching words of its message (https://git.eeqj.de/sneak/secret/issues/49).
Every exported error that can be returned has a test that the function returns
it, and errors wrapping a cause are checked through the wrapping. Checks that
still match text, because the error has no exported value the test can name,
are listed on the issue.
- 2026-10-04: When a vault cannot be opened through its current unlocker,
because a file the unlocker needs is missing or damaged, its keychain item or
Secure Enclave key is gone, or the passphrase is wrong, the error now ends by
naming the vault, saying that it still opens with its mnemonic, and that
`secret unlocker add passphrase`, run with `SB_SECRET_MNEMONIC` set to it,
gives the vault a new unlocker; for a vault that is not the current one, as in
`secret move` between vaults, it says to run `secret vault select` first
(https://git.eeqj.de/sneak/secret/issues/47). Before, it ended with the bare
cause. The advice is given only when the vault metadata records the key the
mnemonic derives, so not for a vault created without a mnemonic, and not when
the passphrase could not be read at all. `secret vault import` is not named:
it refuses a vault that has a long-term key. `secret encrypt` and
`secret decrypt` now read the key secret through `vault.GetSecret`, as
`secret get` does, so they give the same advice; `Secret.GetValue`, the other
way to get the long-term key, is removed. When a secret's `current` file
cannot be read, the error says that `secret version list` lists its versions
and `secret version promote` makes one current. The causes stay wrapped.
- 2026-10-04: An unlocker's ID is the name of its directory in `unlockers.d`, so
no two unlockers of a vault share one
(https://git.eeqj.de/sneak/secret/issues/98). Before, a keychain or Secure
Enclave unlocker's ID was its creation time to the minute and the host name,
and a passphrase unlocker's the time to the minute, so two created within a
minute shared an ID, and `unlocker select`, `unlocker remove` and the
selection `unlocker add` makes acted on the older one. A PGP unlocker's ID was
`pgp-` and its key's fingerprint; a second PGP unlocker for a key is still
refused, now by comparing the fingerprint in the other unlockers' metadata.
`unlocker list` and the shell completion of `unlocker select` and
`unlocker remove` take each ID from the directory the unlocker was read from,
no longer by matching metadata, so two unlockers with the same metadata are
listed apart; an unlocker of an unknown type is listed under its directory
name, and completion now offers Secure Enclave unlockers too. The keychain and
Secure Enclave code was type-checked by `script/lint-darwin`, never run; a
test on Linux lists, completes, selects and removes each of two passphrase
unlockers with the same metadata by its own ID.
- 2026-10-04: README's Storage Architecture, `secret version promote`, Technical
Details and Testing text matches the code
(https://git.eeqj.de/sneak/secret/issues/102). `current` and `currentvault`
are plain files holding a name, not symbolic links; a version's metadata is
the encrypted `metadata.age`; the state directory is `berlin.sneak.pkg.secret`
in the user's configuration directory, not `~/.local/share/secret`, and holds
the `lock` file. Also corrected: the code sets up no Touch ID for the keychain
or Secure Enclave unlocker, and the Secure Enclave only decrypts; per-version
keys give no forward secrecy; `pub.age` is not age-encrypted; vault metadata
holds no vault name. Testing lists only `make test`.
- 2026-10-04: `secret init` and `secret vault create` create a vault whole or
not at all (https://git.eeqj.de/sneak/secret/issues/105). `vault.CreateVault`
now takes the unlocker passphrase too, writes the vault directory with its
metadata, long-term public key and passphrase unlocker, `longterm.age`
included, into a temporary directory, renames that into `vaults.d` once it is
complete, and only then makes the vault current. Before, either command killed
after the passphrase prompt but before the unlocker was written left a vault
with no unlocker, which `vault create` had already made current and which
neither command would create again. Killed part-way now, it leaves no vault,
and the next command that takes the lock deletes the temporary directory; or,
killed between the rename and making the vault current, a complete vault that
is not current, which `secret vault select` makes current.
- 2026-10-04: A failed `secret unlocker add keychain` or
`secret unlocker add secure-enclave` no longer leaves its keychain item or
Secure Enclave key behind (https://git.eeqj.de/sneak/secret/issues/89).
`CreateSecureEnclaveUnlocker` gets the long-term key before it creates the
Secure Enclave key, so that a wrong passphrase creates none, and deletes the
key again if encrypting with it or writing the unlocker then fails.
`macse.CreateKey` finds the new key's hash right after `sc_auth` creates it,
and fails with an error naming the key's label if it cannot; it deletes the
key again if getting its public key then fails. The Objective-C was only read,
never compiled or run, and so was `macse_darwin.go`, which is cgo only.
`CreateKeychainUnlocker` writes all of the unlocker's files, the metadata
among them, before it stores the item in the keychain, and deletes the item
again if moving the unlocker into place then fails. A failure to delete is
reported along with the first error. The tests of this run only on macOS: the
Secure Enclave one in a build with cgo on a Mac with a Secure Enclave, the
keychain one in a build with cgo.
- 2026-10-04: What a command killed part-way left under a `.tmp-` name
(https://git.eeqj.de/sneak/secret/issues/75), the temporary directories of
`secret.TempDirFor` and the temporary files of `secret.WriteFileAtomic`,
encrypted keys included, is deleted by the next command that takes the state
directory lock. Before, it stayed until deleted by hand. A command writes
`finished` into the lock file just before it releases the lock; the next one
to take the lock searches only when it does not find that, so after a command
that finished nothing is searched, however many secrets and versions there
are. The search looks in the state directory, each vault, each secret and each
version, the only directories those helpers make them in. A command that only
reads takes no lock and deletes nothing. A failure to delete is warned about
and the command goes on. An unlocker directory with no metadata file was
already removed by `secret unlocker remove` given its directory name; a test
now shows it.
- 2026-10-04: An age identity's private key goes into a locked buffer through
`secret.IdentityToLockedBuffer` everywhere
(https://git.eeqj.de/sneak/secret/issues/38): the vault's long-term key when a
passphrase, PGP, keychain or Secure Enclave unlocker is created, the new
unlocker's own key, a new secret version's key, and the key `secret encrypt`
generates. Before, each place converted the string age returns to bytes and
left the string in ordinary memory. The function moves the string's own bytes
into the buffer, which overwrites them; the copies age makes while writing the
string remain, as its comment says. The 1.0 memory-security entry below no
longer lists these places, `internal/cli/crypto.go` among them, nor
`version.go:155`, which was `internal/secret/version.go`, not
`internal/cli/version.go`.
- 2026-10-04: `script/lint-darwin` (`make lint-darwin`) runs `go vet` and
`golangci-lint` in docker on the code as a macOS build compiles it
(`GOOS=darwin`), with cgo off (https://git.eeqj.de/sneak/secret/issues/50).
`script/check` runs it, and the `Dockerfile` lint stage runs its commands, so
`script/cibuild` does too. Before, CI on Linux never compiled the files built
only for macOS. Compiling cgo code for macOS needs Apple's SDK headers, and
both `internal/macse` and `github.com/keybase/go-keychain` are cgo on macOS.
So the three functions that call `go-keychain` moved from
`keychainunlocker.go` to `keychainunlocker_cgo.go`, built only with cgo on
macOS like `macse_darwin.go`. A macOS build without cgo, which before did not
compile, gets `keychainunlocker_nocgo.go` and the `macse` stub instead, whose
errors say the keychain or Secure Enclave needs a macOS build with cgo. The
check covers the rest of the keychain unlocker, the Secure Enclave unlocker
and the macOS-only tests other than `keychainunlocker_test.go`, whose lint
findings are fixed. For the length and complexity limits, parts of
`GetIdentity`, `getLongTermPrivateKey` and `CreateKeychainUnlocker` moved into
functions of their own, and the Secure Enclave unlocker derives the long-term
key from the mnemonic through the same function as the keychain unlocker
instead of a copy of it. Lines over 88 columns in the files the check cannot
see are wrapped.
- 2026-10-04: `secret rm`, `secret version rm`, `secret vault remove` and
`secret unlocker remove` ask `[y/N]` before removing anything
(https://git.eeqj.de/sneak/secret/issues/39), naming what they remove: the
secret, its vault and its version count; the version, secret and vault; the
vault and its secret count; the unlocker, its vault and whether it is the
last, and for the last the vault's secret count and that the vault then opens
only with its mnemonic. Only `y` or `yes` goes ahead. Without `--force`, a
command whose stdin is not a terminal fails at once. `--force` (now also on
`rm` and `version rm`) removes without asking; it replaces the old refusals to
remove a vault with secrets or the last unlocker of one without `--force`,
which the question now covers. The checks run, and the question is asked,
before the state directory lock is taken; under the lock the checks run again,
and if they would ask a different question, nothing is removed. `secret rm`
fails when it cannot count the versions.
- 2026-10-04: A crash while an unlocker is being replaced no longer leaves a
current unlocker that cannot open the vault
(https://git.eeqj.de/sneak/secret/issues/71). Every new unlocker gets a
directory of its own, named with the time to the nanosecond:
`passphrase-<time>`, `<host>-pgp-<time>`, and for a keychain or Secure Enclave
unlocker the keychain item or Secure Enclave key, which names the directory,
carries the time instead of the day. `secret.WriteDir` fails on a directory
that exists instead of writing into it. `unlocker add passphrase` writes the
new unlocker, makes it current, and only then removes the vault's other
passphrase unlockers; a crash between the last two steps leaves the old one
beside the new, and the old passphrase still opens the vault through it until
the next `unlocker add passphrase` or an `unlocker remove` removes it. A PGP,
keychain or Secure Enclave unlocker added on the same host and day as another
of its type is added beside it instead of replacing it.
- 2026-10-04: `SB_SECRET_MNEMONIC` and `SB_UNLOCK_PASSPHRASE` are read once per
command, in its `RunE`, into locked buffers on the CLI `Instance`, and unset
at once, so that no program the command runs, `gpg` included, inherits them
(https://git.eeqj.de/sneak/secret/issues/60). Nothing below the command reads
the environment; the buffers are passed down: `vault.CreateVault` takes the
mnemonic (nil for none), a `Vault` derives its long-term key from its
`Mnemonic` and gives its `UnlockPassphrase` to a passphrase unlocker, and the
PGP, keychain and Secure Enclave unlocker constructors take both.
`CreatePGPUnlocker` sets both on the vault it loads, through `SetMnemonic` and
`SetUnlockPassphrase`, now part of `VaultInterface`, before calling its
`GetOrDeriveLongTermKey`. `init` and `vault create` no longer put the mnemonic
into the environment. Unsetting erases nothing: the starting environment
(`/proc/<pid>/environ`) and memory still hold the value. The README warns
against both variables.
- 2026-10-04: `.golangci.yml` is again the canonical file from `sneak/prompts`,
byte for byte (https://git.eeqj.de/sneak/secret/issues/66). It runs
`gomodguard_v2` in place of the deprecated `gomodguard`, so the lint no longer
warns, and enables `depguard` with a rule that keeps `net/http/httptest` out
of non-test files. Neither raised a finding in this repo.
- 2026-10-04: `secret unlocker add pgp` works on Linux
(https://git.eeqj.de/sneak/secret/issues/88). `CreatePGPUnlocker` gets the
vault's long-term key as adding a passphrase unlocker does, with the vault's
`GetOrDeriveLongTermKey`, now part of `VaultInterface`: from the mnemonic,
checked against the vault, or else from the current unlocker. Before, it used
the keychain unlocker's helper, which on every platform but macOS always
failed. A test adds a PGP unlocker for a throwaway GPG key, getting the
long-term key once from the mnemonic and once from a passphrase unlocker, and
reads a secret through the new unlocker.
- 2026-10-04: A vault name may use only lowercase ASCII letters, digits, `.`,
`-` and `_`, and must not be empty, `.` or `..`
(https://git.eeqj.de/sneak/secret/issues/68); the error and `README.md` state
the rule. `vault create`, `vault import`, `vault select`, `vault remove`, both
vault names of `mv` and shell completion of a `vault:secret` argument check
the name as typed with `vault.ValidateVaultName` before building any path from
it. Before, `vault import ..` wrote a long-term key and an unlocker into the
state directory itself, and `vault select ..` made that the current vault.
- 2026-10-04: `script/cibuild` runs the checks again on an unchanged tree
(https://git.eeqj.de/sneak/secret/issues/54). It passes the current time as
the `CHECK_EPOCH` build argument, which both the lint and the build stage of
the `Dockerfile` declare after their module download, so the `RUN` steps below
the argument run again on each build while the base images and module
downloads stay cached. Before, a second run on the same tree took every check
from the build cache and reported success having run nothing.
- 2026-10-04: A failed unlocker add no longer leaves a partial unlocker
directory (https://git.eeqj.de/sneak/secret/issues/48).
`secret unlocker add pgp` resolves the GPG key's fingerprint once, for its
duplicate check, and passes it to `CreatePGPUnlocker` to record.
`CreatePGPUnlocker` and `CreateKeychainUnlocker` get the long-term key and
encrypt everything before writing anything. All four unlocker types write
their files through `secret.WriteDir`: a new unlocker is built in a temporary
directory, renamed into place when complete and removed on a failure.
- 2026-10-04: `secret unlocker select` and `secret unlocker remove` skip, with
the warning `unlocker list` gives, an unlocker directory whose metadata file
cannot be checked for, read or parsed, instead of failing when it sorts before
the unlocker asked for. Such a directory, or one without a metadata file, is
removed by its directory name, the name the warning gives; only the directory
is removed, since its type is unknown. Removing one whose metadata file is
missing or corrupt never counts as removing the last unlocker. Removing one
whose metadata file cannot be checked for or read always does, since it may be
the only working unlocker, so in a vault with secrets it needs `--force`.
- 2026-10-04: A failed command prints its error once, without the usage text
after it (https://git.eeqj.de/sneak/secret/issues/41). Usage is still printed
for a command called wrongly: wrong number of arguments, unknown flag, bad
flag value, missing required flag, or flags that break a flag group (mutually
exclusive, required together, one required). The root command's
`PersistentPreRunE` turns usage off. Cobra checks arguments and flag values
before that hook but required flags and flag groups only after it, so the hook
checks those two first. Root `SilenceUsage` would have hidden usage for all of
these.
- 2026-10-04: `secret get` keeps the secret in locked memory until it writes it
out (https://git.eeqj.de/sneak/secret/issues/37): `Vault.GetSecret` and
`Vault.GetSecretVersion` return a `*memguard.LockedBuffer`, which every caller
destroys, and `secret get` writes its bytes straight to stdout, still with no
trailing newline. Before, the value was copied into ordinary memory that
nothing wiped, and `get --version` also wrote it to the debug log.
- 2026-10-04: The `Makefile` no longer sets `DOCKER_HOST`, so its docker targets
use the local docker daemon, or whatever `DOCKER_HOST` the environment sets.
`make build` calls the new `script/build`, which stamps the version (`VERSION`
from the environment, else `git describe`) and the git commit as before.
`build`, `clean`, `install` and `docker-run` are in `.PHONY`; `make install`
depends on `build`. The `vet` target is gone: `script/test` runs `go vet`
first.
- 2026-10-04: `.gitignore` is the org's standard file, which ignores `.env`,
`.env.*`, `*.pem` and `*.key` and editor and OS files, plus this repo's
`/secret`, `*.log`, `*.test` and `settings.local.json`
(https://git.eeqj.de/sneak/secret/issues/40). `.dockerignore` also leaves out
`node_modules`; `.git` stays in the build context for the version stamp.
- 2026-10-04: `secret init` refuses when the default vault exists, and
`secret vault create NAME` when `NAME` does, with "vault NAME already exists",
before writing anything. The check is in `vault.CreateVault`, which both
commands call while holding the state directory lock, so two creates of one
vault at once cannot both pass the check. Before, either command replaced the
vault's metadata, passphrase unlocker and `longterm.age`, so none of its
secrets could be decrypted any more. Both commands now ask for the unlocker
passphrase before creating the vault, so one stopped at that prompt leaves no
vault behind.
- 2026-10-04: The `internal/cli` tests are back to about their time before the
state directory lock (https://git.eeqj.de/sneak/secret/issues/80). The test
that each changing command waits for the lock releases it as soon as it sees
the command waiting there, instead of after a fixed 100 ms. The two vaults
with passphrase unlockers that the path and move tests start from are made
once and copied for each test.
- 2026-10-04: `secret mv` rejects a move whose destination is the source under
another name, such as `foo` for `Foo` on a case-insensitive filesystem (the
macOS default) or a name reached through a symbolic link, before changing
anything, with or without `--force`, within a vault and between vaults;
before, `--force` removed the destination and so deleted the secret. A rename
that changes only letter case works on a case-sensitive filesystem as before.
- 2026-10-04: Lint runs only in docker: `script/lint` builds `Dockerfile.lint`,
where golangci-lint is a build step rebuilt on every run
(`--no-cache-filter`), so an unchanged tree is linted too; the module download
stays cached. `script/bootstrap` no longer installs golangci-lint, and the
`Dockerfile` lint stage calls it directly instead of `make lint`.
`golangci-lint config verify` is not run: it fetches its schema live over
unpinned HTTPS.
- 2026-10-04: A PGP unlocker whose metadata has no usable GPG key ID no longer
panics: `GetID()` warns with the unlocker's directory and returns
`pgp-unknown`. `ListUnlockers` skips, with a warning, an unlocker whose
metadata file cannot be checked for, read or parsed instead of failing, so
`secret unlocker list` still lists the others; the listing's ID lookup no
longer warns about that directory again.
- 2026-10-03: `secret mv` rejects a move whose destination is the source
(`mv --force x x`, `mv --force work:x work:`, or an empty destination, which
defaults to the source name) before changing anything; before, `--force`
removed the destination first and so deleted the secret. Every vault name
given with `vault:` must be one of the existing vaults by exact name, so
`work:x work/:x` is rejected instead of being taken for a move between two
vaults. A move within a named vault no longer makes that vault the current
one, whether it succeeds or fails.
- 2026-10-03: Commands that change the state directory hold one lock (`flock` on
`lock` in the state directory; a mutex on the in-memory test filesystem), so
concurrent commands no longer lose versions or race on the current pointers.
Every file is written through `secret.WriteFileAtomic` (temporary file, sync,
rename), so no file is ever half-written and `current`, `currentvault` and
`current-unlocker` never go missing. New versions, new secrets and cross-vault
copies are built in a temporary directory and renamed into place, and removals
rename out of the way first, so a version or secret is never half-added and
never half-removed.
- 2026-10-03: The checks run before changing a vault now stop with an error
naming the path and cause when they cannot read what they inspect, instead of
reading the failure as "nothing there": the duplicate check before
`unlocker add pgp` (an unreadable `unlockers.d` or unlocker metadata file),
the secret count that guards removing the last unlocker and removing a vault,
and the existing long-term key check before `vault import`.
- 2026-10-03: `version rm`, `version promote` and `get --version` accept a
version only if it is one of the versions `version list` lists for that
secret, compared as typed before any path is built (`secret.VersionExists`),
and touch nothing otherwise. An empty `--version` is rejected instead of
meaning the current version. Before, `secret version rm x ../../..` deleted
the whole vault, `secret version rm x ..` the secret, and `.` or `""` every
version.
- 2026-10-03: Key material is wiped on every exit: `Entry()` returns the exit
code after its deferred `memguard.Purge()` has run, and only `main` calls
`os.Exit`. SIGINT and SIGTERM go through memguard's handler, which wipes every
buffer before exiting; when the process is in the terminal's foreground
process group it first restores the terminal settings from startup, so an
interrupted passphrase prompt no longer leaves echo off.
- 2026-10-03: Every command that builds a path from a secret name checks the
name first with `vault.ValidateSecretName` and touches nothing when it is
invalid: `rm`, `mv` (both names, within a vault and between vaults, before
switching the current vault), `import`, `version list`/`promote`/`rm`,
`encrypt` and `decrypt`. The error and `README.md` state the naming rule.
Before, `secret rm ..` deleted the whole vault and `secret rm .` every secret
in it.
- 2026-10-03: The keychain unlocker's age key passphrase stays in locked memory:
it is generated into a locked buffer, and the keychain JSON is written and
read by `KeychainData` code in `internal/secret/keychaindata.go` (tested on
Linux) without `encoding/json` holding it; the JSON field names are unchanged.
- 2026-10-02: A plain `docker build .` builds again: the size tests skip a case
that needs more locked memory than the process can lock, and run every case
under `script/cibuild`. The image stamps the `VERSION` build argument, else
`git describe --tags --always`, into `Version`, and fails if `.git` is present
but yields no version; `make build` stamps `git describe` too, not a fixed
`0.1.0`. `.dockerignore` keeps `.git/config` out; `script/docker` is the
canonical copy.
- 2026-08-07: Updated golangci-lint to v2.12.2 with the canonical
`.golangci.yml` (all linters enabled minus the standard disable list, `lll`
88, tests linted); bumped the `Dockerfile` lint-stage image to the tagged
v2.12.2 Debian digest; fixed all ~1550 new findings across `internal/` and
`pkg/` (line wrapping, `wsl_v5` blank lines, sentinel errors for `err113`,
`t.Parallel()` where safe, `_test` package conversions, complexity/`dupl`
helper extraction) on branch `golangci-v2.12.2`. Reworked after review: the
`err113` sentinels in `internal/vault`, `internal/secret`, `internal/cli` and
`pkg/bip85` were reshaped so every composed error message is byte-identical to
`main`, and `findUnlockerIDByMetadata` now returns an error so `unlocker list`
skips an unreadable `unlockers.d` entry with a warning instead of emitting a
fabricated fallback ID.
- 2026-08-07: Added `.editorconfig`
(https://git.eeqj.de/sneak/secret/issues/27).
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints, Makefile
shims, README Entrypoints section
- 2026-07-07: Added `REPO_POLICIES.md` and the `make hooks` target;
`.gitea/workflows/check.yml` now runs `script/cibuild`.
- 2026-03-30: Added the `make fmt-check` target and
`.gitea/workflows/check.yml`, which runs `docker build` on every push; the
`Dockerfile` base images are pinned by sha256.
- 2026-03-11: Secure Enclave unlocker for hardware-backed secret protection,
plus review fixes (stub panics, derivation index, tests, README) on branch
secure-enclave-unlocker.
- 2026-02-28: Repo cleanup, removed stale .cursorrules and coverage.out.
- Audit fix wave (issues #1, #2, #3, #13, #14): skip unlockers with missing
metadata, allow uppercase secret names, fix hardcoded derivation index,
validate names in GetSecretVersion against path traversal, return errors
instead of panicking, add Warn() on silent anomalies.
- Memory security hardening: LockedBuffer used through encrypt/decrypt paths
(Save/EncryptWithPassphrase/GetValue/gpg helpers), deprecated bare-[]byte APIs
removed.
- Per-secret keypair architecture, vault package refactor, versioning with
--version, comprehensive test suite with in-memory filesystem.
- Debug logging system (slog, GODEBUG flag, TTY-aware output).
- Renamed SEP unlocker to Keychain, reorganized import commands.
- 2025-05-28: Initial implementation (vault, age encryption, mnemonic, CLI).
# Future Steps # Future Steps
- Implement version-number shell completion for the second arg of - Implement version-number shell completion for the second arg of
`secret version promote` and `secret version rm` (`internal/cli/version.go`). `secret version promote` and `secret version rm` (`internal/cli/version.go`;
was an in-code TODO removed for godox).
- Cover mnemonic-vs-xprv identity consistency in `pkg/agehd/agehd_test.go` - Cover mnemonic-vs-xprv identity consistency in `pkg/agehd/agehd_test.go`
`TestMnemonicVsXPRVConsistency`. `TestMnemonicVsXPRVConsistency` (was an in-code FIXME removed for godox).
- No macOS test runs in CI: `internal/secret/pgpunlock_test.go` builds only on - CI does not compile, lint or test the files built only with cgo on macOS,
macOS. A macOS runner would cover it. since compiling them needs Apple's SDK:
- 1.0 critical security blockers: `internal/secret/keychainunlocker_cgo.go` (the three functions that call
`go-keychain`) with `keychainunlocker_test.go`, and `internal/macse`
(`macse_darwin.go`, `macse_test.go`, the Objective-C sources). Lint has never
run on them, so it would likely find more there than the line lengths. No
macOS test runs in CI. A macOS runner would cover all of it (asked on
https://git.eeqj.de/sneak/secret/issues/50).
- 1.0 critical security blockers (from repo TODO.md):
- Memory security: age writes an identity's private key out as a string in - Memory security: age writes an identity's private key out as a string in
ordinary memory, and the copies it makes on the way stay there ordinary memory, and the copies it makes on the way stay there
(`secret.IdentityToLockedBuffer` overwrites only the string itself). (`secret.IdentityToLockedBuffer` overwrites only the string itself).
@@ -32,6 +604,6 @@ https://git.eeqj.de/sneak/secret/milestone/12
- Cleanups: read statedir from environment or default instead of passing it - Cleanups: read statedir from environment or default instead of passing it
around. around.
- Enhancements: help examples, colored output, --quiet flag, name suggestions on - Enhancements: help examples, colored output, --quiet flag, name suggestions on
miss, audit logging, hardware integration tests (GPG), naming consistency, miss, audit logging, hardware integration tests (Keychain, GPG), naming
vault export/import, batch operations, search, secret metadata (descriptions, consistency, vault export/import, batch operations, search, secret metadata
tags). (descriptions, tags).
-6
View File
@@ -3,16 +3,10 @@ package main
import ( import (
"os" "os"
"time"
"sneak.berlin/go/secret/internal/cli" "sneak.berlin/go/secret/internal/cli"
) )
func main() { func main() {
// Every time secret prints or writes is in UTC, whatever TZ or the
// system's time zone is set to. It is set here, before cli.Entry starts
// any goroutine that could read time.Local while it changes.
time.Local = time.UTC //nolint:gosmopolitan // this makes local time UTC
os.Exit(cli.Entry()) os.Exit(cli.Entry())
} }
+15 -10
View File
@@ -144,20 +144,25 @@ func TestVaultCreationLeavesNoSecretInEnvironment(t *testing.T) {
// TestNoVaultImportCommand is a regression test for // TestNoVaultImportCommand is a regression test for
// https://git.eeqj.de/sneak/secret/issues/139: `secret vault import` // https://git.eeqj.de/sneak/secret/issues/139: `secret vault import`
// refused every vault that `secret vault create` makes, because each has a // refused every vault that `secret vault create` makes, because each has a
// long-term key. The command is gone, so it fails as any subcommand of // long-term key. The command is gone, so cobra answers it with the help of
// `secret vault` that does not exist does. // `secret vault`, as it does any subcommand of `secret vault` it does not
// know.
func TestNoVaultImportCommand(t *testing.T) { func TestNoVaultImportCommand(t *testing.T) {
t.Setenv(secret.EnvStateDir, t.TempDir()) t.Setenv(secret.EnvStateDir, t.TempDir())
t.Setenv(secret.EnvMnemonic, testMnemonic)
t.Setenv(secret.EnvUnlockPassphrase, testPassphrase)
_, err := cli.ExecuteCommandInProcess( run := func(args ...string) string {
strings.Fields("vault create work"), "", nil) t.Setenv(secret.EnvMnemonic, testMnemonic)
require.NoError(t, err) t.Setenv(secret.EnvUnlockPassphrase, testPassphrase)
_, err = cli.ExecuteCommandInProcess( output, err := cli.ExecuteCommandInProcess(args, "", nil)
strings.Fields("vault import work"), "", nil) require.NoError(t, err, "%v", args)
require.EqualError(t, err, `unknown command "import" for "secret vault"`)
return output
}
run("vault", "create", "work")
require.Equal(t, run("vault"), run("vault", "import", "work"))
} }
// TestStopAtPassphrasePromptLeavesNothing is a regression test for the // TestStopAtPassphrasePromptLeavesNothing is a regression test for the
+19 -50
View File
@@ -1,7 +1,6 @@
package cli package cli
import ( import (
"bytes"
"errors" "errors"
"fmt" "fmt"
"io" "io"
@@ -108,8 +107,8 @@ func (cli *Instance) storeNewEncryptionKey(
// does not exist. The caller must destroy the returned buffer. Only storing // does not exist. The caller must destroy the returned buffer. Only storing
// a new key takes the state directory lock, so that reading an existing key // a new key takes the state directory lock, so that reading an existing key
// works on a read-only state directory and keeps no other command waiting // works on a read-only state directory and keeps no other command waiting
// at the passphrase prompt, and Encrypt reads its input and writes its // at the passphrase prompt, and Encrypt streams its input and output
// output without the lock. // unlocked.
func (cli *Instance) resolveEncryptionKey( func (cli *Instance) resolveEncryptionKey(
vlt *vault.Vault, secretName string, vlt *vault.Vault, secretName string,
) (*memguard.LockedBuffer, error) { ) (*memguard.LockedBuffer, error) {
@@ -189,17 +188,17 @@ func (cli *Instance) Encrypt(secretName, inputFile, outputFile string) error {
input = file input = file
} }
// Set up output writer. With --output the result is held in memory // Set up output writer
// and written only once encryption has succeeded, so a failure to
// encrypt leaves --output untouched and --input can be the same file.
// A failure while writing --output, such as a full disk, can leave it
// empty or partly written.
output := cli.cmd.OutOrStdout() output := cli.cmd.OutOrStdout()
var result bytes.Buffer
if outputFile != "" { if outputFile != "" {
output = &result file, err := cli.fs.Create(outputFile)
if err != nil {
return fmt.Errorf("failed to create output file: %w", err)
}
defer func() { _ = file.Close() }()
output = file
} }
// Encrypt the data // Encrypt the data
@@ -218,11 +217,7 @@ func (cli *Instance) Encrypt(secretName, inputFile, outputFile string) error {
return fmt.Errorf("failed to finalize encryption: %w", err) return fmt.Errorf("failed to finalize encryption: %w", err)
} }
if outputFile == "" { return nil
return nil
}
return cli.writeOutputFile(outputFile, result.Bytes())
} }
// Decrypt decrypts data using an age secret key stored in a secret // Decrypt decrypts data using an age secret key stored in a secret
@@ -283,17 +278,17 @@ func (cli *Instance) Decrypt(secretName, inputFile, outputFile string) error {
input = file input = file
} }
// Set up output writer. With --output the result is held in memory // Set up output writer
// and written only once decryption has succeeded, so a failure to
// decrypt leaves --output untouched and --input can be the same file.
// A failure while writing --output, such as a full disk, can leave it
// empty or partly written.
output := cli.cmd.OutOrStdout() output := cli.cmd.OutOrStdout()
var result bytes.Buffer
if outputFile != "" { if outputFile != "" {
output = &result file, err := cli.fs.Create(outputFile)
if err != nil {
return fmt.Errorf("failed to create output file: %w", err)
}
defer func() { _ = file.Close() }()
output = file
} }
// Decrypt the data // Decrypt the data
@@ -307,32 +302,6 @@ func (cli *Instance) Decrypt(secretName, inputFile, outputFile string) error {
return fmt.Errorf("failed to decrypt data: %w", err) return fmt.Errorf("failed to decrypt data: %w", err)
} }
if outputFile == "" {
return nil
}
return cli.writeOutputFile(outputFile, result.Bytes())
}
// writeOutputFile creates outputFile, or empties it if it exists, and
// writes result into it.
func (cli *Instance) writeOutputFile(outputFile string, result []byte) error {
file, err := cli.fs.Create(outputFile)
if err != nil {
return fmt.Errorf("failed to create output file: %w", err)
}
_, err = file.Write(result)
closeErr := file.Close()
if err == nil {
err = closeErr
}
if err != nil {
return fmt.Errorf("failed to write output file: %w", err)
}
return nil return nil
} }
-185
View File
@@ -1,185 +0,0 @@
//nolint:testpackage // sets the unexported fields of Instance
package cli
import (
"errors"
"fmt"
"io"
"os"
"path/filepath"
"testing"
"github.com/spf13/afero"
"github.com/spf13/cobra"
"github.com/stretchr/testify/require"
"sneak.berlin/go/secret/internal/vault"
)
// plainText is the content of plain.txt in the directory newCryptoTest makes.
const plainText = "document"
// newCryptoTest returns an Instance whose current vault, in a new state
// directory on the real filesystem, holds the age key secret "key", and a
// new directory holding only plain.txt and plain.txt.age, its encryption
// with that key.
func newCryptoTest(t *testing.T) (*Instance, string) {
t.Helper()
fs := afero.NewOsFs()
stateDir := t.TempDir()
_, err := vault.CreateVault(fs, stateDir, "default", testMnemonicBuffer(t), nil)
require.NoError(t, err)
cli := NewCLIInstanceWithStateDir(fs, stateDir)
cli.Mnemonic = testMnemonicBuffer(t)
cli.cmd = &cobra.Command{}
dir := t.TempDir()
plain := filepath.Join(dir, "plain.txt")
require.NoError(t, afero.WriteFile(fs, plain, []byte(plainText), 0o600))
require.NoError(t, cli.Encrypt("key", plain, plain+".age"))
return cli, dir
}
// TestFailedCryptoLeavesOutputUnchanged is a regression test for
// https://git.eeqj.de/sneak/secret/issues/143, where secret encrypt and
// secret decrypt emptied the file named by --output before they failed.
// A failure to encrypt or decrypt must leave an existing file as it was and
// create none, and give the same error as without --output.
func TestFailedCryptoLeavesOutputUnchanged(t *testing.T) {
t.Parallel()
tests := []struct {
name string
run func(cli *Instance, dir, output string) error
}{
// Reading a directory fails once encryption has begun
{"encrypt", func(cli *Instance, dir, output string) error {
return cli.Encrypt("key", dir, output)
}},
// plain.txt is not an age file
{"decrypt", func(cli *Instance, dir, output string) error {
return cli.Decrypt("key", filepath.Join(dir, "plain.txt"), output)
}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
cli, dir := newCryptoTest(t)
existing := filepath.Join(dir, "existing.txt")
require.NoError(t, afero.WriteFile(cli.fs, existing, []byte("precious"), 0o600))
cli.cmd.SetOut(io.Discard)
toStdout := tt.run(cli, dir, "")
require.Error(t, toStdout)
require.EqualError(t, tt.run(cli, dir, existing), toStdout.Error())
require.EqualError(t, tt.run(cli, dir, filepath.Join(dir, "new.txt")),
toStdout.Error())
content, err := afero.ReadFile(cli.fs, existing)
require.NoError(t, err)
require.Equal(t, "precious", string(content))
exists, err := afero.Exists(cli.fs, filepath.Join(dir, "new.txt"))
require.NoError(t, err)
require.False(t, exists, "a failed %s created --output", tt.name)
})
}
}
// TestCryptoInPlace checks that secret encrypt and secret decrypt given
// one file as both --input and --output replace it with the right result.
// Before https://git.eeqj.de/sneak/secret/issues/143 they emptied the file
// before reading it.
func TestCryptoInPlace(t *testing.T) {
t.Parallel()
cli, dir := newCryptoTest(t)
file := filepath.Join(dir, "plain.txt")
require.NoError(t, cli.Encrypt("key", file, file))
require.NoError(t, cli.Decrypt("key", file, file))
content, err := afero.ReadFile(cli.fs, file)
require.NoError(t, err)
require.Equal(t, plainText, string(content))
}
// errCloseFailed is returned by the files closeFailFs creates in place of
// a successful Close, as NFS reports a full disk only when the file is
// closed.
var errCloseFailed = errors.New("no space left on device")
// closeFailFs creates files whose Close fails with errCloseFailed.
type closeFailFs struct {
afero.Fs
}
// closeFailFile is a file created through closeFailFs.
type closeFailFile struct {
afero.File
}
func (f closeFailFile) Close() error {
_ = f.File.Close()
return errCloseFailed
}
//nolint:ireturn // afero.File is the interface required by afero.Fs
func (f closeFailFs) Create(name string) (afero.File, error) {
file, err := f.Fs.Create(name)
if err != nil {
return nil, err
}
return closeFailFile{File: file}, nil
}
// TestDecryptFailsWhenClosingOutputFails checks that secret decrypt fails
// when closing --output fails, since some filesystems report a failed
// write only then.
func TestDecryptFailsWhenClosingOutputFails(t *testing.T) {
t.Parallel()
cli, dir := newCryptoTest(t)
cli.fs = closeFailFs{Fs: cli.fs}
err := cli.Decrypt("key", filepath.Join(dir, "plain.txt.age"),
filepath.Join(dir, "out.txt"))
require.ErrorIs(t, err, errCloseFailed)
}
// TestDecryptToDescriptorOfDeletedFile checks that secret decrypt given
// /dev/fd/N as --output writes through that descriptor. A program may give
// the command a temporary file it has already deleted as stdout, pass
// /dev/stdout as --output and read the result back from that file.
func TestDecryptToDescriptorOfDeletedFile(t *testing.T) {
t.Parallel()
cli, dir := newCryptoTest(t)
file, err := os.CreateTemp(t.TempDir(), "out")
require.NoError(t, err)
defer func() { _ = file.Close() }()
require.NoError(t, os.Remove(file.Name()))
output := fmt.Sprintf("/dev/fd/%d", file.Fd())
require.NoError(t, cli.Decrypt("key", filepath.Join(dir, "plain.txt.age"), output))
// On macOS the command's open of /dev/fd/N shares this descriptor's
// position, which its write leaves at the end.
_, err = file.Seek(0, io.SeekStart)
require.NoError(t, err)
content, err := io.ReadAll(file)
require.NoError(t, err)
require.Equal(t, plainText, string(content))
}
-60
View File
@@ -1,60 +0,0 @@
package cli_test
import (
"bytes"
"path/filepath"
"strconv"
"testing"
"github.com/spf13/afero"
"github.com/spf13/cobra"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/secret/internal/cli"
"sneak.berlin/go/secret/internal/secret"
"sneak.berlin/go/secret/internal/vault"
)
// TestListFailsWhenCurrentVaultDoesNotExist is a regression test for
// https://git.eeqj.de/sneak/secret/issues/156, where a currentvault file
// that was empty, held an invalid vault name, or named a vault that does
// not exist, made `secret list` report an empty vault. Listing must fail
// with an error that names the file and says to run `secret vault select`,
// `secret vault list` must still list the vaults, and selecting a vault
// must make listing work again. The name "." would make vaults.d itself the
// vault, and "../vaults.d/work" would reach the vault work by a path.
func TestListFailsWhenCurrentVaultDoesNotExist(t *testing.T) {
t.Parallel()
for _, name := range []string{"", ".", "../vaults.d/work", "gone"} {
t.Run(strconv.Quote(name), func(t *testing.T) {
t.Parallel()
fs := newTwoVaultFs(t)
currentVaultPath := filepath.Join(testStateDir, "currentvault")
require.NoError(t, afero.WriteFile(fs, currentVaultPath,
[]byte(name), secret.FilePerms))
c := cli.NewCLIInstanceWithStateDir(fs, testStateDir)
var out bytes.Buffer
cmd := &cobra.Command{}
cmd.SetOut(&out)
err := c.ListSecrets(cmd, false, true, "")
require.ErrorIs(t, err, vault.ErrVaultNotFound)
require.ErrorContains(t, err, currentVaultPath)
require.ErrorContains(t, err, "secret vault select")
assert.Empty(t, out.String())
require.NoError(t, c.ListVaults(cmd, false))
assert.Equal(t, "Available vaults:\n default\n work\n", out.String())
require.NoError(t, c.SelectVault(cmd, "default"))
out.Reset()
require.NoError(t, c.ListSecrets(cmd, false, true, ""))
assert.Equal(t, "x\n", out.String())
})
}
}
+12 -51
View File
@@ -3,7 +3,6 @@ package cli_test
import ( import (
"bufio" "bufio"
"context" "context"
"fmt"
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
@@ -17,67 +16,29 @@ import (
"sneak.berlin/go/secret/internal/secret" "sneak.berlin/go/secret/internal/secret"
) )
const (
// envRunEntry, when set, makes this test binary run cli.Entry with its
// own arguments and exit, instead of running the tests.
envRunEntry = "SECRET_TEST_RUN_ENTRY"
// bufferWiped is what the test binary prints, run with envRunEntry set,
// when cli.Entry wiped the buffer made before it ran.
bufferWiped = "Entry wiped the buffer made before it ran"
)
// With envRunEntry set, the test binary makes a buffer, runs cli.Entry,
// says whether the buffer is wiped, and exits with Entry's exit code. This
// is done in init, not in a test, so that TestMain does not run in that
// process: it would build the secret binary again, and remove it at the end,
// while the other tests run it.
//
//nolint:gochecknoinits // must run before TestMain, as above
func init() {
if os.Getenv(envRunEntry) == "" {
return
}
buf := memguard.NewBufferFromBytes([]byte("key material"))
code := cli.Entry()
if !buf.IsAlive() {
fmt.Fprintln(os.Stderr, bufferWiped)
}
os.Exit(code)
}
// Entry must return its exit code rather than exit, so that its deferred // Entry must return its exit code rather than exit, so that its deferred
// memguard purge runs on the success and the error path alike. Entry wipes // memguard purge runs on the success and the error path alike.
// every memguard buffer in the process, including those other tests are //
// using, so it runs in a process of its own: this test binary, run again //nolint:paralleltest // sets os.Args, and Entry wipes every buffer in the process
// with envRunEntry set.
func TestEntryWipesBuffersAndReturnsExitCode(t *testing.T) { func TestEntryWipesBuffersAndReturnsExitCode(t *testing.T) {
t.Parallel() savedArgs := os.Args
t.Cleanup(func() { os.Args = savedArgs })
tests := []struct { tests := []struct {
args []string args []string
exitCode int exitCode int
}{ }{
{args: []string{"--help"}, exitCode: 0}, {args: []string{"secret", "--help"}, exitCode: 0},
{args: []string{"no-such-command"}, exitCode: 1}, {args: []string{"secret", "no-such-command"}, exitCode: 1},
} }
for _, tt := range tests { for _, tt := range tests {
//nolint:gosec // G204: test runs its own test binary buf := memguard.NewBufferFromBytes([]byte("key material"))
cmd := exec.CommandContext(t.Context(), os.Args[0], tt.args...) os.Args = tt.args
cmd.Env = append(os.Environ(), envRunEntry+"=1") assert.Equal(t, tt.exitCode, cli.Entry(), "exit code for %v", tt.args)
assert.False(t, buf.IsAlive(), "Entry left a buffer unwiped for %v", tt.args)
output, err := cmd.CombinedOutput()
require.NotNil(t, cmd.ProcessState, "%v did not run: %v", tt.args, err)
assert.Equal(t, tt.exitCode, cmd.ProcessState.ExitCode(),
"exit code for %v: %s", tt.args, output)
assert.Contains(t, string(output), bufferWiped,
"Entry left a buffer unwiped for %v", tt.args)
} }
} }
+7 -7
View File
@@ -29,12 +29,6 @@ func newGenerateCmd() *cobra.Command {
Use: "generate", Use: "generate",
Short: "Generate random data", Short: "Generate random data",
Long: `Generate various types of random data including mnemonics and secrets.`, Long: `Generate various types of random data including mnemonics and secrets.`,
// Cobra checks Args only on a command that has a RunE; without
// one, an unknown subcommand prints this help and exits 0.
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
return cmd.Help()
},
} }
cmd.AddCommand(newGenerateMnemonicCmd()) cmd.AddCommand(newGenerateMnemonicCmd())
@@ -175,6 +169,12 @@ func (cli *Instance) GenerateSecret(
return fmt.Errorf("failed to generate random secret: %w", err) return fmt.Errorf("failed to generate random secret: %w", err)
} }
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
// Store the secret in the vault // Store the secret in the vault
vlt, err := vault.GetCurrentVault(cli.fs, cli.stateDir) vlt, err := vault.GetCurrentVault(cli.fs, cli.stateDir)
if err != nil { if err != nil {
@@ -187,7 +187,7 @@ func (cli *Instance) GenerateSecret(
secretBuffer := memguard.NewBufferFromBytes([]byte(secretValue)) secretBuffer := memguard.NewBufferFromBytes([]byte(secretValue))
defer secretBuffer.Destroy() defer secretBuffer.Destroy()
err = cli.storeSecret(vlt, secretName, secretBuffer, force) err = vlt.AddSecret(secretName, secretBuffer, force)
if err != nil { if err != nil {
return err return err
} }
+27 -33
View File
@@ -67,15 +67,36 @@ func (cli *Instance) promptMnemonic() (*memguard.LockedBuffer, func(), error) {
return mnemonicBuffer, mnemonicBuffer.Destroy, nil return mnemonicBuffer, mnemonicBuffer.Destroy, nil
} }
// Init initializes the secret manager. It asks for the mnemonic and the // Init initializes the secret manager, holding the state directory lock
// passphrase before it takes the state directory lock, so that no other // while initialize runs
// command waits while the user answers, and holds the lock while
// initialize runs.
func (cli *Instance) Init(cmd *cobra.Command) error { func (cli *Instance) Init(cmd *cobra.Command) error {
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
return cli.initialize(cmd)
}
// initialize creates the state directory, the default vault and its first
// unlocker
func (cli *Instance) initialize(cmd *cobra.Command) error {
secret.Debug("Starting secret manager initialization") secret.Debug("Starting secret manager initialization")
// Create state directory
stateDir := cli.GetStateDir()
secret.DebugWith("Creating state directory", slog.String("path", stateDir))
err := cli.fs.MkdirAll(stateDir, secret.DirPerms)
if err != nil {
secret.Debug("Failed to create state directory", "error", err)
return fmt.Errorf("failed to create state directory: %w", err)
}
// Refuse an existing default vault before asking for anything // Refuse an existing default vault before asking for anything
err := vault.CheckNewVaultName(cli.fs, cli.stateDir, defaultVaultName) err = vault.CheckNewVaultName(cli.fs, cli.stateDir, defaultVaultName)
if err != nil { if err != nil {
return fmt.Errorf("failed to create default vault: %w", err) return fmt.Errorf("failed to create default vault: %w", err)
} }
@@ -114,38 +135,11 @@ func (cli *Instance) Init(cmd *cobra.Command) error {
} }
defer cleanupPassphrase() defer cleanupPassphrase()
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
return cli.initialize(cmd, mnemonic, passphraseBuffer)
}
// initialize creates the state directory, and the default vault from
// mnemonic with a passphrase unlocker protected by passphrase. Another
// command may have created the vault since Init checked: vault.CreateVault
// checks again.
func (cli *Instance) initialize(
cmd *cobra.Command, mnemonic, passphrase *memguard.LockedBuffer,
) error {
// Create state directory
stateDir := cli.GetStateDir()
secret.DebugWith("Creating state directory", slog.String("path", stateDir))
err := cli.fs.MkdirAll(stateDir, secret.DirPerms)
if err != nil {
secret.Debug("Failed to create state directory", "error", err)
return fmt.Errorf("failed to create state directory: %w", err)
}
// Create the default vault with its passphrase unlocker // Create the default vault with its passphrase unlocker
secret.Debug("Creating default vault") secret.Debug("Creating default vault")
vlt, err := vault.CreateVault(cli.fs, cli.stateDir, defaultVaultName, vlt, err := vault.CreateVault(cli.fs, cli.stateDir, defaultVaultName,
mnemonic, passphrase) mnemonic, passphraseBuffer)
if err != nil { if err != nil {
secret.Debug("Failed to create default vault", "error", err) secret.Debug("Failed to create default vault", "error", err)
+3 -4
View File
@@ -34,10 +34,9 @@ const (
testMnemonic = "abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about" testMnemonic = "abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about"
// commandWait is how long a test lets the secret binary run before it // commandWait is how long a test lets the secret binary run before it
// kills it and fails. It stays well under the time limit the test phase // kills it and fails. It stays well under the 30 seconds script/test
// of the Dockerfile gives the whole package (go test -timeout), so a // gives the whole package, so a command that hangs fails the test with
// command that hangs fails the test with the test's own message instead // the test's own message instead of Go's timeout panic.
// of Go's timeout panic.
commandWait = 10 * time.Second commandWait = 10 * time.Second
) )
-54
View File
@@ -3,9 +3,6 @@ package cli_test
import ( import (
"bytes" "bytes"
"encoding/json" "encoding/json"
"os"
"os/exec"
"path/filepath"
"testing" "testing"
"time" "time"
@@ -128,57 +125,6 @@ func TestMovedSecretKeepsItsTimes(t *testing.T) {
"updated %s, then %s", beforeMove.UpdatedAt, afterMove.UpdatedAt) "updated %s, then %s", beforeMove.UpdatedAt, afterMove.UpdatedAt)
} }
// TestListedTimesAreUTC is a regression test for
// https://git.eeqj.de/sneak/secret/issues/169, where times were printed
// in the local time zone. It runs `secret list` on the built binary with
// TZ set to New York, where the secret's last update, 03:04:05 UTC on 2
// January, is 22:04:05 the day before.
func TestListedTimesAreUTC(t *testing.T) {
t.Parallel()
stateDir := t.TempDir()
vlt, err := vault.CreateVault(
afero.NewOsFs(), stateDir, "default", testMnemonicBuffer(t), nil)
require.NoError(t, err)
vlt.Mnemonic = testMnemonicBuffer(t)
err = vlt.AddSecret("x", memguard.NewBufferFromBytes([]byte("one")), false)
require.NoError(t, err)
updated := time.Date(2026, time.January, 2, 3, 4, 5, 0, time.UTC)
current := filepath.Join(stateDir, "vaults.d", "default", "secrets.d", "x",
"current")
require.NoError(t, os.Chtimes(current, updated, updated))
tests := []struct {
flags []string
want string
}{
{flags: nil, want: "2026-01-02 03:04"},
{
flags: []string{"--json"},
want: `"updated_at": "2026-01-02T03:04:05Z"`,
},
}
for _, tt := range tests {
args := append([]string{"list"}, tt.flags...)
//nolint:gosec // G204: test executes the freshly built secret binary
cmd := exec.CommandContext(t.Context(), secretBinaryPath(t), args...)
cmd.Env = []string{
secret.EnvStateDir + "=" + stateDir,
"TZ=America/New_York",
"PATH=" + os.Getenv("PATH"),
}
output, err := cmd.CombinedOutput()
require.NoError(t, err, "%v: %s", args, output)
assert.Contains(t, string(output), tt.want, "%v", args)
}
}
// listSecrets returns the output of `secret list`, or of // listSecrets returns the output of `secret list`, or of
// `secret list --json` when jsonOutput is set. // `secret list --json` when jsonOutput is set.
func listSecrets(t *testing.T, c *cli.Instance, jsonOutput bool) string { func listSecrets(t *testing.T, c *cli.Instance, jsonOutput bool) string {
+14 -328
View File
@@ -280,9 +280,9 @@ func stateDirModTimes(t *testing.T, fs afero.Fs) map[string]int64 {
return modTimes return modTimes
} }
// setupEveryCommand makes what the commands that the lock tests below run // setupEveryCommand makes what each command in
// need: the current vault "work" with two versions of "test/secret", the // TestChangingCommandsWaitForLock needs: the current vault "work" with two
// vault "other", and the file testInput. // versions of "test/secret", the vault "other", and the file testInput.
// There is no vault "default", which init creates. If withUnlocker is set, // There is no vault "default", which init creates. If withUnlocker is set,
// it also gives "work" a passphrase unlocker, which is slow. It returns the // it also gives "work" a passphrase unlocker, which is slow. It returns the
// older version and the unlocker's ID. // older version and the unlocker's ID.
@@ -346,25 +346,6 @@ func waitingForLock() bool {
return false return false
} }
// requireWaitingForLock returns once a command waits for the state
// directory lock. It fails the test if the command finishes first, sending
// its error on done, or never waits.
func requireWaitingForLock(t *testing.T, done <-chan error) {
t.Helper()
timeout := time.After(lockWait)
for !waitingForLock() {
select {
case err := <-done:
t.Fatalf("finished while the lock was held, with error %v", err)
case <-timeout:
t.Fatal("never waited for the lock")
case <-time.After(time.Millisecond):
}
}
}
// requireWaitsForLock runs a command, given what setupEveryCommand made, // requireWaitsForLock runs a command, given what setupEveryCommand made,
// while holding the state directory lock. The command must neither finish // while holding the state directory lock. The command must neither finish
// nor change anything before it waits for the lock, and must succeed once // nor change anything before it waits for the lock, and must succeed once
@@ -404,7 +385,17 @@ func requireWaitsForLock(
go func() { done <- run(cli, olderVersion, unlockerID) }() go func() { done <- run(cli, olderVersion, unlockerID) }()
requireWaitingForLock(t, done) timeout := time.After(lockWait)
for !waitingForLock() {
select {
case err := <-done:
t.Fatalf("finished while the lock was held, with error %v", err)
case <-timeout:
t.Fatal("never waited for the lock")
case <-time.After(time.Millisecond):
}
}
assert.Equal(t, before, stateDirModTimes(t, fs), assert.Equal(t, before, stateDirModTimes(t, fs),
"changed the state directory before waiting for the lock") "changed the state directory before waiting for the lock")
@@ -481,311 +472,6 @@ func TestChangingCommandsWaitForLock(t *testing.T) {
} }
} }
// TestCommandsAskBeforeLocking is a regression test for
// https://git.eeqj.de/sneak/secret/issues/152: these commands took the
// state directory lock before asking for the mnemonic or a passphrase, so
// while one waited at its prompt, every other command that changes anything
// waited for it. Each runs here while the test holds the lock, given what
// takes it to the last step at which it can ask: the passphrase prompt of
// init and vault create, and the unlock of the vault, which asks for the
// passphrase of its passphrase unlocker, for the others. The tests have no
// terminal, so a prompt fails at once. Each command must get there, and
// fail, without waiting for the lock.
//
//nolint:paralleltest // waitingForLock sees any test's command waiting for the lock
func TestCommandsAskBeforeLocking(t *testing.T) {
newTestGPGKey(t)
mnemonic := testMnemonicBuffer(t)
wrongPassphrase := memguard.NewBufferFromBytes([]byte("wrong-passphrase"))
t.Cleanup(wrongPassphrase.Destroy)
for _, tc := range []struct {
name string
run func(cli *Instance) error
// want is wrapped in the error the command fails with
want error
}{
{"init", func(cli *Instance) error {
cli.Mnemonic = mnemonic
return cli.Init(cli.cmd)
}, secret.ErrNoTerminal},
{"vault create", func(cli *Instance) error {
cli.Mnemonic = mnemonic
return cli.CreateVault(cli.cmd, "created")
}, secret.ErrNoTerminal},
// With no terminal to ask for the new passphrase on, the new
// unlocker takes the one in SB_UNLOCK_PASSPHRASE, which then fails
// to unlock the vault
{"unlocker add passphrase", func(cli *Instance) error {
cli.UnlockPassphrase = wrongPassphrase
return cli.UnlockersAdd("passphrase", cli.cmd)
}, secret.ErrWrongPassphrase},
{"unlocker add pgp", func(cli *Instance) error {
cli.cmd.Flags().String("keyid", unreadableTestGPGUserID, "")
return cli.UnlockersAdd("pgp", cli.cmd)
}, secret.ErrNoTerminal},
{"add --force", func(cli *Instance) error {
return cli.AddSecret("test/secret", true)
}, secret.ErrNoTerminal},
{"import --force", func(cli *Instance) error {
return cli.ImportSecret(cli.cmd, "test/secret", testInput, true)
}, secret.ErrNoTerminal},
{"generate secret --force", func(cli *Instance) error {
return cli.GenerateSecret(cli.cmd, "test/secret", 16, "base58", true)
}, secret.ErrNoTerminal},
} {
t.Run(tc.name, func(t *testing.T) {
requireAsksBeforeLocking(t, tc.run, tc.want)
})
}
}
// startWhileLocked makes what setupEveryCommand makes, with the passphrase
// unlocker, on fs, an empty in-memory filesystem, takes the state directory
// lock there, and starts run, given a CLI instance on fs with "value" on
// stdin. It returns the function that releases the lock, which does so only
// the first time it is called, and the channel that run's error comes on.
// The lock is also released when the test ends, so that later tests can
// take it.
func startWhileLocked(
t *testing.T, fs afero.Fs, run func(cli *Instance) error,
) (func(), <-chan error) {
t.Helper()
setupEveryCommand(t, fs, true)
release, err := vault.LockStateDir(fs, testStateDir)
require.NoError(t, err)
release = sync.OnceFunc(release)
t.Cleanup(release)
cli := NewCLIInstanceWithStateDir(fs, testStateDir)
cli.cmd = &cobra.Command{}
cli.cmd.SetIn(strings.NewReader("value"))
cli.cmd.SetOut(io.Discard)
done := make(chan error, 1)
go func() { done <- run(cli) }()
return release, done
}
// requireAsksBeforeLocking starts run while holding the state directory
// lock, as startWhileLocked does. run must fail, with an error wrapping
// want, without waiting for the lock.
func requireAsksBeforeLocking(
t *testing.T, run func(cli *Instance) error, want error,
) {
t.Helper()
release, done := startWhileLocked(t, afero.NewMemMapFs(), run)
timeout := time.After(lockWait)
for !waitingForLock() {
select {
case err := <-done:
require.ErrorIs(t, err, want)
return
case <-timeout:
t.Fatal("neither failed nor waited for the lock")
case <-time.After(time.Millisecond):
}
}
// Let the command go on, so that it releases the lock
release()
select {
case <-done:
case <-time.After(lockWait):
}
t.Fatal("waited for the lock before asking")
}
// TestCommandsCheckAgainOnceLocked checks that commands which check what
// they change before they ask for anything, and so before they take the
// state directory lock, check again once they hold it: another command may
// have changed it in the meantime. Each runs while the test holds the lock,
// given what it needs to ask for nothing. Once it waits for the lock, the
// test makes the change another command could have made, then releases the
// lock. The command must fail, and change nothing.
//
//nolint:paralleltest // waitingForLock sees any test's command waiting for the lock
func TestCommandsCheckAgainOnceLocked(t *testing.T) {
fingerprint := newTestGPGKey(t)
mnemonic := testMnemonicBuffer(t)
otherMnemonic := memguard.NewBufferFromBytes([]byte("legal winner thank " +
"year wave sausage worth useful legal winner thank yellow"))
t.Cleanup(otherMnemonic.Destroy)
passphrase := memguard.NewBufferFromBytes([]byte(testPassphrase))
t.Cleanup(passphrase.Destroy)
// work returns the vault "work" on fs, opened with the mnemonic
work := func(fs afero.Fs) *vault.Vault {
vlt := vault.NewVault(fs, testStateDir, "work")
vlt.Mnemonic = mnemonic
return vlt
}
for _, tc := range []struct {
name string
run func(cli *Instance) error
// change is what another command changes while run waits
change func(fs afero.Fs) error
// want is wrapped in the error the command fails with
want error
}{
{
"unlocker add, vault removed",
func(cli *Instance) error {
cli.Mnemonic, cli.UnlockPassphrase = mnemonic, passphrase
return cli.UnlockersAdd("passphrase", cli.cmd)
},
func(fs afero.Fs) error {
return fs.RemoveAll(testStateDir + "/vaults.d/work")
},
vault.ErrVaultNotFound,
},
{
"unlocker add, vault created again from another mnemonic",
func(cli *Instance) error {
cli.Mnemonic, cli.UnlockPassphrase = mnemonic, passphrase
return cli.UnlockersAdd("passphrase", cli.cmd)
},
func(fs afero.Fs) error {
err := fs.RemoveAll(testStateDir + "/vaults.d/work")
if err != nil {
return err
}
_, err = vault.CreateVault(fs, testStateDir, "work",
otherMnemonic, passphrase)
return err
},
errVaultCreatedAgain,
},
{
"unlocker add pgp, key added",
func(cli *Instance) error {
cli.Mnemonic = mnemonic
cli.cmd.Flags().String("keyid", unreadableTestGPGUserID, "")
return cli.UnlockersAdd("pgp", cli.cmd)
},
func(fs afero.Fs) error {
_, err := secret.CreatePGPUnlocker(work(fs),
unreadableTestGPGUserID, fingerprint)
return err
},
errGPGKeyAlreadyUnlocker,
},
} {
t.Run(tc.name, func(t *testing.T) {
requireChecksAgainOnceLocked(t, tc.run, tc.change, tc.want)
})
}
}
// requireChecksAgainOnceLocked starts run while holding the state directory
// lock, as startWhileLocked does. Once run waits for the lock, change makes
// the change that another command could have made in the meantime, and the
// lock is released. run must then fail, with an error wrapping want, and
// change nothing.
func requireChecksAgainOnceLocked(
t *testing.T, run func(cli *Instance) error,
change func(fs afero.Fs) error, want error,
) {
t.Helper()
fs := afero.NewMemMapFs()
release, done := startWhileLocked(t, fs, run)
requireWaitingForLock(t, done)
require.NoError(t, change(fs))
// Left out: taking and releasing the lock writes the lock file
lockFile := testStateDir + "/lock"
before := stateDirModTimes(t, fs)
delete(before, lockFile)
release()
select {
case err := <-done:
require.ErrorIs(t, err, want)
case <-time.After(lockWait):
t.Fatal("did not finish once the lock was released")
}
after := stateDirModTimes(t, fs)
delete(after, lockFile)
assert.Equal(t, before, after, "changed the state directory")
}
// TestForcedAddReplacesSecretAddedMeanwhile runs a forced add of a secret
// that does not exist yet while the test holds the state directory lock.
// Once it waits for the lock, the test adds the secret, as another command
// could have, and releases the lock. The forced add must replace that
// secret, as if it had run after the other command.
//
//nolint:paralleltest // waitingForLock sees any test's command waiting for the lock
func TestForcedAddReplacesSecretAddedMeanwhile(t *testing.T) {
mnemonic := testMnemonicBuffer(t)
fs := afero.NewMemMapFs()
release, done := startWhileLocked(t, fs, func(cli *Instance) error {
cli.Mnemonic = mnemonic
return cli.AddSecret("added", true)
})
requireWaitingForLock(t, done)
work := vault.NewVault(fs, testStateDir, "work")
work.Mnemonic = mnemonic
other := memguard.NewBufferFromBytes([]byte("other"))
t.Cleanup(other.Destroy)
require.NoError(t, work.AddSecret("added", other, false))
release()
select {
case err := <-done:
require.NoError(t, err)
case <-time.After(lockWait):
t.Fatal("did not finish once the lock was released")
}
value, err := work.GetSecret("added")
require.NoError(t, err)
t.Cleanup(value.Destroy)
assert.Equal(t, "value", string(value.Bytes()))
}
// TestEncryptWithExistingKeyTakesNoLock checks that secret encrypt with a // TestEncryptWithExistingKeyTakesNoLock checks that secret encrypt with a
// key that already exists, which only reads the state directory, finishes // key that already exists, which only reads the state directory, finishes
// while another command holds the state directory lock. // while another command holds the state directory lock.
-315
View File
@@ -1,315 +0,0 @@
//nolint:testpackage // white-box test of unexported internals
package cli
import (
"errors"
"path/filepath"
"testing"
"time"
"github.com/awnumar/memguard"
"github.com/spf13/afero"
"github.com/spf13/cobra"
"github.com/stretchr/testify/require"
"sneak.berlin/go/secret/internal/secret"
"sneak.berlin/go/secret/internal/vault"
)
// errPermissionDenied is returned by renameFailFs and removeAllFailFs in
// place of a rename or removal that succeeds.
var errPermissionDenied = errors.New("permission denied")
// renameFailFs fails every Rename of one path, as a read-only parent
// directory does.
type renameFailFs struct {
afero.Fs
path string
}
func (f *renameFailFs) Rename(oldname, newname string) error {
if oldname == f.path {
return errPermissionDenied
}
return f.Fs.Rename(oldname, newname)
}
// removeAllFailFs fails every RemoveAll of an entry of the directory dir,
// as a file in it that cannot be deleted does, and records that entry's
// path in failed.
type removeAllFailFs struct {
afero.Fs
dir string
failed string
}
func (f *removeAllFailFs) RemoveAll(path string) error {
if filepath.Dir(path) == f.dir {
f.failed = path
return errPermissionDenied
}
return f.Fs.RemoveAll(path)
}
// createCrossVaults creates in fs the vaults "work" and "default", where
// "default" holds the secret "x" with the value "value".
func createCrossVaults(t *testing.T, fs afero.Fs) {
t.Helper()
_, err := vault.CreateVault(fs, testStateDir, "work",
testMnemonicBuffer(t), nil)
require.NoError(t, err)
source, err := vault.CreateVault(fs, testStateDir, "default",
testMnemonicBuffer(t), nil)
require.NoError(t, err)
err = source.AddSecret("x",
memguard.NewBufferFromBytes([]byte("value")), false)
require.NoError(t, err)
}
// requireValue requires that the vault vaultName in fs holds secretName
// with the value createCrossVaults stored.
func requireValue(t *testing.T, fs afero.Fs, vaultName, secretName string) {
t.Helper()
vlt := vault.NewVault(fs, testStateDir, vaultName)
vlt.Mnemonic = testMnemonicBuffer(t)
value, err := vlt.GetSecret(secretName)
require.NoError(t, err, vaultName+":"+secretName)
defer value.Destroy()
require.Equal(t, []byte("value"), value.Bytes())
}
// TestCrossVaultMoveFailsWhenSourceRemains is a regression test for
// https://git.eeqj.de/sneak/secret/issues/159, where a move between two
// vaults whose source could not be removed reported the secret moved and
// succeeded. It must fail, and the copy and the source must both be there.
// The state directory lock refuses the wrapped filesystem, so the test
// calls copySecretAndRemoveSource, which a move between vaults runs once it
// holds the lock.
func TestCrossVaultMoveFailsWhenSourceRemains(t *testing.T) {
t.Parallel()
base := afero.NewMemMapFs()
createCrossVaults(t, base)
instance := NewCLIInstanceWithStateDir(&renameFailFs{
Fs: base,
path: testStateDir + "/vaults.d/default/secrets.d/x",
}, testStateDir)
source := vault.NewVault(instance.fs, testStateDir, "default")
source.Mnemonic = testMnemonicBuffer(t)
err := instance.copySecretAndRemoveSource(&cobra.Command{}, source, "x",
vault.NewVault(instance.fs, testStateDir, "work"), "y", false)
require.ErrorIs(t, err, errPermissionDenied)
require.ErrorContains(t, err, "secret 'default:x' is copied with all "+
"1 version(s) to 'work:y', but the source remains")
requireValue(t, base, "default", "x")
requireValue(t, base, "work", "y")
}
// TestCrossVaultMoveFailsWhenOldFilesRemain makes the removal of the source
// fail after the source has been moved out of its vault, when only the
// temporary directory it was moved into cannot be deleted. The move must
// fail, say that the secret is moved, and name that directory.
func TestCrossVaultMoveFailsWhenOldFilesRemain(t *testing.T) {
t.Parallel()
base := afero.NewMemMapFs()
createCrossVaults(t, base)
sourceVaultDir := testStateDir + "/vaults.d/default"
fs := &removeAllFailFs{Fs: base, dir: sourceVaultDir}
instance := NewCLIInstanceWithStateDir(fs, testStateDir)
source := vault.NewVault(fs, testStateDir, "default")
source.Mnemonic = testMnemonicBuffer(t)
err := instance.copySecretAndRemoveSource(&cobra.Command{}, source, "x",
vault.NewVault(fs, testStateDir, "work"), "y", false)
require.ErrorIs(t, err, errPermissionDenied)
require.ErrorContains(t, err, "secret 'default:x' is moved with all "+
"1 version(s) to 'work:y', but its old files could not all be deleted")
// The error names the directory that holds the old files
require.NotEmpty(t, fs.failed)
require.ErrorContains(t, err, fs.failed)
oldFiles, err := afero.DirExists(base, filepath.Join(fs.failed, "x"))
require.NoError(t, err)
require.True(t, oldFiles)
// The source vault no longer holds the secret
remains, err := afero.DirExists(base, sourceVaultDir+"/secrets.d/x")
require.NoError(t, err)
require.False(t, remains)
requireValue(t, base, "work", "y")
}
// TestCrossVaultMoveUnlocksBeforeLocking is a regression test for
// https://git.eeqj.de/sneak/secret/issues/189: a move between two vaults
// unlocked them while holding the state directory lock, and unlocking a
// vault can ask for its passphrase, so every other command that changes
// anything waited while the user answered. Here each vault has a passphrase
// unlocker whose passphrase is the vault's name, and the move is given the
// passphrase of only one of them. Run while the test holds the lock, it
// must fail to unlock the other vault without waiting for the lock.
//
//nolint:paralleltest // times commands against the in-memory lock all tests share
func TestCrossVaultMoveUnlocksBeforeLocking(t *testing.T) {
fs := afero.NewMemMapFs()
createCrossVaults(t, fs)
vaults, err := vault.ListVaults(fs, testStateDir)
require.NoError(t, err)
for _, name := range vaults {
vlt := vault.NewVault(fs, testStateDir, name)
vlt.Mnemonic = testMnemonicBuffer(t)
passphrase := memguard.NewBufferFromBytes([]byte(name))
_, err = vlt.CreatePassphraseUnlocker(passphrase)
passphrase.Destroy()
require.NoError(t, err)
}
// The move is from "default" to "work"
for _, tc := range []struct {
// failing is the vault that the move cannot unlock, given the
// passphrase of the vault named given
failing, given string
}{
{"source", "work"},
{"destination", "default"},
} {
t.Run(tc.failing, func(t *testing.T) {
passphrase := memguard.NewBufferFromBytes([]byte(tc.given))
defer passphrase.Destroy()
instance := NewCLIInstanceWithStateDir(fs, testStateDir)
instance.UnlockPassphrase = passphrase
release, err := vault.LockStateDir(fs, testStateDir)
require.NoError(t, err)
// Also frees a waiting move if the test fails, so that it
// releases the lock the other tests use
defer release()
done := make(chan error, 1)
go func() {
done <- instance.MoveSecret(&cobra.Command{},
"default:x", "work:y", false)
}()
select {
case err := <-done:
require.ErrorIs(t, err, secret.ErrWrongPassphrase)
require.ErrorContains(t, err,
"failed to unlock "+tc.failing+" vault")
case <-time.After(lockWait):
t.Fatal("waited for the lock before unlocking the " +
tc.failing + " vault")
}
})
}
}
// TestCrossVaultMoveChecksAgainOnceLocked runs a move between vaults, given
// the mnemonic, as TestCommandsCheckAgainOnceLocked runs the commands in its
// table: once the move waits for the state directory lock, the test removes
// the source secret or the destination vault, or creates the destination
// vault again from another mnemonic, and the move must then fail and change
// nothing.
//
//nolint:paralleltest // waitingForLock sees any test's command waiting for the lock
func TestCrossVaultMoveChecksAgainOnceLocked(t *testing.T) {
mnemonic := testMnemonicBuffer(t)
otherMnemonic := memguard.NewBufferFromBytes([]byte("legal winner thank " +
"year wave sausage worth useful legal winner thank yellow"))
t.Cleanup(otherMnemonic.Destroy)
move := func(cli *Instance) error {
cli.Mnemonic = mnemonic
return cli.MoveSecret(cli.cmd, "work:test/secret", "other:moved", false)
}
t.Run("source removed", func(t *testing.T) {
requireChecksAgainOnceLocked(t, move, func(fs afero.Fs) error {
return fs.RemoveAll(testStateDir + "/vaults.d/work/secrets.d/test%secret")
}, vault.ErrSecretNotFound)
})
t.Run("destination vault removed", func(t *testing.T) {
requireChecksAgainOnceLocked(t, move, func(fs afero.Fs) error {
return fs.RemoveAll(testStateDir + "/vaults.d/other")
}, vault.ErrVaultNotFound)
})
t.Run("destination vault created again from another mnemonic", func(t *testing.T) {
// Set by the move, which has finished once
// requireChecksAgainOnceLocked returns
var moveErr error
requireChecksAgainOnceLocked(t, func(cli *Instance) error {
moveErr = move(cli)
return moveErr
}, func(fs afero.Fs) error {
err := fs.RemoveAll(testStateDir + "/vaults.d/other")
if err != nil {
return err
}
_, err = vault.CreateVault(fs, testStateDir, "other", otherMnemonic, nil)
return err
}, errVaultCreatedAgain)
// secret unlocker add fails with the same error, so its message
// must fit a move too
require.EqualError(t, moveErr, "vault 'other' was removed and "+
"created again with another key while it was being unlocked; "+
"nothing was changed")
})
}
// TestCrossVaultMoveRefusesExistingSecretBeforeUnlocking checks that a move
// between vaults onto a secret that exists, without --force, is refused
// before either vault is unlocked, since unlocking can ask for a
// passphrase. The vaults have no unlockers and the move is given no
// mnemonic, so unlocking either vault fails.
func TestCrossVaultMoveRefusesExistingSecretBeforeUnlocking(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
createCrossVaults(t, fs)
work := vault.NewVault(fs, testStateDir, "work")
work.Mnemonic = testMnemonicBuffer(t)
err := work.AddSecret("y",
memguard.NewBufferFromBytes([]byte("other")), false)
require.NoError(t, err)
instance := NewCLIInstanceWithStateDir(fs, testStateDir)
err = instance.MoveSecret(&cobra.Command{}, "default:x", "work:y", false)
require.ErrorIs(t, err, vault.ErrSecretExists)
}
+78 -232
View File
@@ -422,92 +422,26 @@ func (cli *Instance) AddSecret(secretName string, force bool) error {
// Locked only now that stdin has been read: in `secret encrypt key | // Locked only now that stdin has been read: in `secret encrypt key |
// secret add name`, holding the lock while reading would leave each // secret add name`, holding the lock while reading would leave each
// command waiting for the other. // command waiting for the other.
return cli.storeSecret(vlt, secretName, valueBuffer, force)
}
// storeSecret adds the secret name with value to vlt, as vault.AddSecret
// does, holding the state directory lock while it does. With force, adding
// an existing secret replaces its current version, which needs the vault's
// long-term key. Getting the key can ask for the passphrase, so storeSecret
// gets it before it takes the lock, and no other command waits while the
// user answers. If another command adds the secret while this one waits for
// the lock, storeSecret lets go of the lock, gets the key, and takes the
// lock again.
func (cli *Instance) storeSecret(
vlt *vault.Vault, name string, value *memguard.LockedBuffer, force bool,
) error {
err := vault.ValidateSecretName(name)
if err != nil {
return err
}
replacing, err := replacesVersion(vlt, name, force)
if err != nil {
return err
}
if replacing {
_, err = vlt.GetOrDeriveLongTermKey()
if err != nil {
return fmt.Errorf("failed to get long-term key: %w", err)
}
}
needsKey, err := cli.addWithoutAsking(vlt, name, value, force)
if !needsKey {
return err
}
_, err = vlt.GetOrDeriveLongTermKey()
if err != nil {
return fmt.Errorf("failed to get long-term key: %w", err)
}
// With the key in memory, this adds the secret
_, err = cli.addWithoutAsking(vlt, name, value, force)
return err
}
// addWithoutAsking takes the state directory lock and adds the secret name
// with value to vlt, as vault.AddSecret does. If that would replace the
// secret's current version while the vault's long-term key is not in
// memory, getting the key could ask for the passphrase with the lock held:
// addWithoutAsking then adds nothing and returns true.
func (cli *Instance) addWithoutAsking(
vlt *vault.Vault, name string, value *memguard.LockedBuffer, force bool,
) (bool, error) {
release, err := vault.LockStateDir(cli.fs, cli.stateDir) release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil { if err != nil {
return false, err return err
} }
defer release() defer release()
replacing, err := replacesVersion(vlt, name, force) // Add the secret to the vault
secret.Debug("Calling vault.AddSecret", "secret_name", secretName,
"value_length", valueBuffer.Size(), "force", force)
err = vlt.AddSecret(secretName, valueBuffer, force)
if err != nil { if err != nil {
return false, err secret.Debug("vault.AddSecret failed", "error", err)
return err
} }
if replacing && vlt.Locked() { secret.Debug("vault.AddSecret completed successfully")
return true, nil
}
return false, vlt.AddSecret(name, value, force) return nil
}
// replacesVersion reports whether adding the secret name to vlt replaces
// the secret's current version: whether force is set and the secret has one.
func replacesVersion(vlt *vault.Vault, name string, force bool) (bool, error) {
if !force {
return false, nil
}
exists, err := secret.NewSecret(vlt, name).Exists()
if err != nil {
return false, fmt.Errorf("failed to check if secret exists: %w", err)
}
return exists, nil
} }
// GetSecret retrieves and prints the current version of a secret // GetSecret retrieves and prints the current version of a secret
@@ -773,7 +707,14 @@ func (cli *Instance) ImportSecret(
// Locked only now that the file has been read, as in AddSecret: the // Locked only now that the file has been read, as in AddSecret: the
// file may be a pipe written by another secret command. // file may be a pipe written by another secret command.
err = cli.storeSecret(vlt, secretName, valueBuffer, force) release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
// Store the secret in the vault
err = vlt.AddSecret(secretName, valueBuffer, force)
if err != nil { if err != nil {
return err return err
} }
@@ -875,12 +816,24 @@ func (cli *Instance) findSecretToRemove(
}, nil }, nil
} }
// MoveSecret moves or renames a secret (within or across vaults). It // MoveSecret moves or renames a secret (within or across vaults), holding
// leaves taking the state directory lock to moveSecretWithinVault and // the state directory lock.
// moveSecretCrossVault, since a move between vaults unlocks both vaults
// before it takes the lock.
func (cli *Instance) MoveSecret( func (cli *Instance) MoveSecret(
cmd *cobra.Command, source, dest string, force bool, cmd *cobra.Command, source, dest string, force bool,
) error {
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
return cli.moveSecret(cmd, source, dest, force)
}
// moveSecret does the work of MoveSecret. The caller holds the state
// directory lock.
func (cli *Instance) moveSecret(
cmd *cobra.Command, source, dest string, force bool,
) error { ) error {
// Parse source and destination // Parse source and destination
srcVaultName, srcSecretName, srcQualified := ParseVaultSecretRef(source) srcVaultName, srcSecretName, srcQualified := ParseVaultSecretRef(source)
@@ -980,9 +933,8 @@ func (cli *Instance) existingVault(name string) (*vault.Vault, error) {
return vault.NewVault(cli.fs, cli.stateDir, name), nil return vault.NewVault(cli.fs, cli.stateDir, name), nil
} }
// moveSecretWithinVault renames a secret within the vault vlt, holding the // moveSecretWithinVault renames a secret within the vault vlt. Its caller,
// state directory lock. Its caller, MoveSecret, has already checked both // MoveSecret, has already checked both secret names.
// secret names.
func (cli *Instance) moveSecretWithinVault( func (cli *Instance) moveSecretWithinVault(
cmd *cobra.Command, vlt *vault.Vault, source, dest string, force bool, cmd *cobra.Command, vlt *vault.Vault, source, dest string, force bool,
) error { ) error {
@@ -992,12 +944,6 @@ func (cli *Instance) moveSecretWithinVault(
return fmt.Errorf("secret '%s' %w", source, errMoveOntoItself) return fmt.Errorf("secret '%s' %w", source, errMoveOntoItself)
} }
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
vaultDir, err := vlt.GetDirectory() vaultDir, err := vlt.GetDirectory()
if err != nil { if err != nil {
return err return err
@@ -1084,75 +1030,59 @@ func (cli *Instance) sameDirectory(dir, other string) (bool, error) {
// moveSecretCrossVault handles moving between two different vaults. Its // moveSecretCrossVault handles moving between two different vaults. Its
// caller, MoveSecret, has already checked both secret names and that both // caller, MoveSecret, has already checked both secret names and that both
// vaults exist. The copy needs the source vault's long-term key and is // vaults exist.
// encrypted to the destination vault's long-term public key. The
// destination vault is unlocked to check that the copy will open with what
// the user gave, and, once the state directory lock is held, that it still
// has the key unlocked. Either unlock can ask for a passphrase, so both
// happen before the lock is taken, and no other command waits while the
// user answers.
func (cli *Instance) moveSecretCrossVault( func (cli *Instance) moveSecretCrossVault(
cmd *cobra.Command, cmd *cobra.Command,
srcVault *vault.Vault, srcSecretName string, srcVault *vault.Vault, srcSecretName string,
destVault *vault.Vault, destSecretName string, destVault *vault.Vault, destSecretName string,
force bool, force bool,
) error { ) error {
// Before unlocking, check that the source secret exists, that the srcVaultDir, err := srcVault.GetDirectory()
// destination is not the source under another name, and, without if err != nil {
// --force, that the destination secret does not exist return fmt.Errorf("failed to get source vault directory: %w", err)
_, err := cli.checkCrossVaultMove( }
srcVault, srcSecretName, destVault, destSecretName, force)
// Verify source secret exists
srcStorageName := strings.ReplaceAll(srcSecretName, "/", "%")
srcSecretDir := filepath.Join(srcVaultDir, "secrets.d", srcStorageName)
exists, err := afero.DirExists(cli.fs, srcSecretDir)
if err != nil || !exists {
return fmt.Errorf("secret '%s' %w in vault '%s'",
srcSecretName, vault.ErrSecretNotFound, srcVault.Name)
}
// The source is removed after the copy, so a destination that is the
// source under another name would be lost with it.
destVaultDir, err := destVault.GetDirectory()
if err != nil {
return fmt.Errorf("failed to get destination vault directory: %w", err)
}
destStorageName := strings.ReplaceAll(destSecretName, "/", "%")
destSecretDir := filepath.Join(destVaultDir, "secrets.d", destStorageName)
same, err := cli.sameDirectory(srcSecretDir, destSecretDir)
if err != nil { if err != nil {
return err return err
} }
if same {
return fmt.Errorf("secret '%s:%s' %w: '%s:%s' is the same secret on "+
"this filesystem", srcVault.Name, srcSecretName, errMoveOntoItself,
destVault.Name, destSecretName)
}
// Copying needs the long-term keys of both vaults
srcVault.Mnemonic, srcVault.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase srcVault.Mnemonic, srcVault.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase
destVault.Mnemonic, destVault.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase destVault.Mnemonic, destVault.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase
// Unlock destination vault (will fail if neither mnemonic nor unlocker available) // Unlock destination vault (will fail if neither mnemonic nor unlocker available)
destIdentity, err := destVault.GetOrDeriveLongTermKey() _, err = destVault.GetOrDeriveLongTermKey()
if err != nil { if err != nil {
return fmt.Errorf("failed to unlock destination vault '%s': %w", destVault.Name, err) return fmt.Errorf("failed to unlock destination vault '%s': %w", destVault.Name, err)
} }
_, err = srcVault.GetOrDeriveLongTermKey()
if err != nil {
return fmt.Errorf("failed to unlock source vault '%s': %w",
srcVault.Name, err)
}
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
err = cli.checkLongTermKey(destVault, destIdentity)
if err != nil {
return err
}
return cli.copySecretAndRemoveSource(
cmd, srcVault, srcSecretName, destVault, destSecretName, force)
}
// copySecretAndRemoveSource copies the secret srcSecretName of srcVault,
// with all its versions, to destSecretName in destVault, and removes the
// source. moveSecretCrossVault calls it holding the state directory lock.
// Another command may have changed either vault since moveSecretCrossVault
// checked them, so it checks them again first.
func (cli *Instance) copySecretAndRemoveSource(
cmd *cobra.Command,
srcVault *vault.Vault, srcSecretName string,
destVault *vault.Vault, destSecretName string,
force bool,
) error {
srcSecretDir, err := cli.checkCrossVaultMove(
srcVault, srcSecretName, destVault, destSecretName, force)
if err != nil {
return err
}
// Count versions for user feedback // Count versions for user feedback
versions, _ := secret.ListVersions(cli.fs, srcSecretDir) versions, _ := secret.ListVersions(cli.fs, srcSecretDir)
versionCount := len(versions) versionCount := len(versions)
@@ -1167,9 +1097,12 @@ func (cli *Instance) copySecretAndRemoveSource(
// Delete source secret // Delete source secret
err = secret.RemoveDirAtomic(cli.fs, srcSecretDir) err = secret.RemoveDirAtomic(cli.fs, srcSecretDir)
if err != nil { if err != nil {
return cli.sourceRemovalError(err, srcSecretDir, // Copy succeeded but delete failed - warn but don't fail
srcVault.Name+":"+srcSecretName, destVault.Name+":"+destSecretName, cmd.Printf("Warning: copied secret but failed to remove source: %v\n", err)
versionCount) cmd.Printf("Moved secret '%s:%s' to '%s:%s' (%d version(s))\n",
srcVault.Name, srcSecretName, destVault.Name, destSecretName, versionCount)
return nil
} }
cmd.Printf("Moved secret '%s:%s' to '%s:%s' (%d version(s))\n", cmd.Printf("Moved secret '%s:%s' to '%s:%s' (%d version(s))\n",
@@ -1177,90 +1110,3 @@ func (cli *Instance) copySecretAndRemoveSource(
return nil return nil
} }
// checkCrossVaultMove checks that the secret srcSecretName exists in
// srcVault, that destSecretName in destVault is not the source under
// another name, and, unless force is set, that destSecretName does not
// exist in destVault. It returns the source's directory.
func (cli *Instance) checkCrossVaultMove(
srcVault *vault.Vault, srcSecretName string,
destVault *vault.Vault, destSecretName string,
force bool,
) (string, error) {
srcVaultDir, err := srcVault.GetDirectory()
if err != nil {
return "", fmt.Errorf("failed to get source vault directory: %w", err)
}
// Verify source secret exists
srcStorageName := strings.ReplaceAll(srcSecretName, "/", "%")
srcSecretDir := filepath.Join(srcVaultDir, "secrets.d", srcStorageName)
exists, err := afero.DirExists(cli.fs, srcSecretDir)
if err != nil || !exists {
return "", fmt.Errorf("secret '%s' %w in vault '%s'",
srcSecretName, vault.ErrSecretNotFound, srcVault.Name)
}
// The source is removed after the copy, so a destination that is the
// source under another name would be lost with it.
destVaultDir, err := destVault.GetDirectory()
if err != nil {
return "", fmt.Errorf("failed to get destination vault directory: %w", err)
}
destStorageName := strings.ReplaceAll(destSecretName, "/", "%")
destSecretDir := filepath.Join(destVaultDir, "secrets.d", destStorageName)
same, err := cli.sameDirectory(srcSecretDir, destSecretDir)
if err != nil {
return "", err
}
if same {
return "", fmt.Errorf("secret '%s:%s' %w: '%s:%s' is the same secret "+
"on this filesystem", srcVault.Name, srcSecretName, errMoveOntoItself,
destVault.Name, destSecretName)
}
exists, err = afero.DirExists(cli.fs, destSecretDir)
if err != nil {
return "", fmt.Errorf("failed to check if destination secret exists: %w", err)
}
if exists && !force {
return "", fmt.Errorf(
"secret '%s' %w in vault '%s' (use --force to overwrite)",
destSecretName, vault.ErrSecretExists, destVault.Name)
}
return srcSecretDir, nil
}
// sourceRemovalError returns the error of a move between vaults that
// copied all versionCount versions of the secret srcName to destName, and
// then failed with removeErr to remove srcSecretDir, the directory of
// srcName. The removal can fail after the source has left its vault, when
// only the temporary directory it was moved into cannot be deleted, so the
// error says the source remains only if srcSecretDir is still there.
func (cli *Instance) sourceRemovalError(
removeErr error, srcSecretDir, srcName, destName string, versionCount int,
) error {
remains, err := afero.DirExists(cli.fs, srcSecretDir)
switch {
case err != nil:
return fmt.Errorf("secret '%s' is copied with all %d version(s) "+
"to '%s', but removing the source failed and whether it "+
"remains is unknown: %w",
srcName, versionCount, destName, errors.Join(removeErr, err))
case remains:
return fmt.Errorf("secret '%s' is copied with all %d version(s) "+
"to '%s', but the source remains: %w",
srcName, versionCount, destName, removeErr)
default:
return fmt.Errorf("secret '%s' is moved with all %d version(s) "+
"to '%s', but its old files could not all be deleted: %w",
srcName, versionCount, destName, removeErr)
}
}
+7 -1
View File
@@ -36,11 +36,17 @@ func newSizeTestVault(t *testing.T) (afero.Fs, *vault.Vault) {
fs := afero.NewMemMapFs() fs := afero.NewMemMapFs()
// Create vault, which selects it as the current vault // Create vault
_, err := vault.CreateVault(fs, testStateDir, testVaultName, _, err := vault.CreateVault(fs, testStateDir, testVaultName,
testMnemonicBuffer(t), nil) testMnemonicBuffer(t), nil)
require.NoError(t, err) require.NoError(t, err)
// Set current vault
currentVaultPath := filepath.Join(testStateDir, "currentvault")
vaultPath := filepath.Join(testStateDir, "vaults.d", testVaultName)
err = afero.WriteFile(fs, currentVaultPath, []byte(vaultPath), 0o600)
require.NoError(t, err)
// Get vault and set up long-term key // Get vault and set up long-term key
vlt, err := vault.GetCurrentVault(fs, testStateDir) vlt, err := vault.GetCurrentVault(fs, testStateDir)
require.NoError(t, err) require.NoError(t, err)
-47
View File
@@ -1,47 +0,0 @@
//nolint:testpackage // white-box test of unexported internals
package cli
import (
"bytes"
"testing"
"github.com/spf13/afero"
"github.com/spf13/cobra"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestAddDropsOneTrailingNewlineImportKeepsIt pins the rule the README
// gives: `secret add` drops one trailing newline from stdin, and
// `secret import` stores the file's bytes unchanged.
func TestAddDropsOneTrailingNewlineImportKeepsIt(t *testing.T) {
t.Parallel()
fs, vlt := newSizeTestVault(t)
value := []byte("a\n\n")
cmd := &cobra.Command{}
cmd.SetIn(bytes.NewReader(value))
cli := NewCLIInstanceWithStateDir(fs, testStateDir)
cli.cmd = cmd
require.NoError(t, cli.AddSecret("added", false))
sourceFile := "/test/value"
require.NoError(t, afero.WriteFile(fs, sourceFile, value, 0o600))
require.NoError(t, cli.ImportSecret(cmd, "imported", sourceFile, false))
added, err := vlt.GetSecret("added")
require.NoError(t, err)
defer added.Destroy()
imported, err := vlt.GetSecret("imported")
require.NoError(t, err)
defer imported.Destroy()
assert.Equal(t, []byte("a\n"), added.Bytes())
assert.Equal(t, []byte("a\n\n"), imported.Bytes())
}
-58
View File
@@ -1,58 +0,0 @@
package cli_test
import (
"fmt"
"os/exec"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/secret/internal/secret"
)
// TestUnknownSubcommandFails is a regression test for
// https://git.eeqj.de/sneak/secret/issues/183: given a subcommand it does not
// have, `secret vault`, `secret unlocker`, `secret version` and `secret
// generate` printed their help and exited 0, so a script could not tell a
// mistyped command from a successful one. Each runs here as the built binary
// and must exit 1 naming the subcommand, as an unknown command does at the
// top level, and must still print its help and exit 0 when given nothing.
func TestUnknownSubcommandFails(t *testing.T) {
t.Parallel()
stateDir := t.TempDir()
// run runs the built binary with args and returns what it printed and
// its exit code
run := func(args ...string) (string, int) {
//nolint:gosec // G204: test executes the freshly built secret binary
cmd := exec.CommandContext(t.Context(), secretBinaryPath(t), args...)
cmd.Env = []string{secret.EnvStateDir + "=" + stateDir}
output, err := cmd.CombinedOutput()
if err != nil {
var exitErr *exec.ExitError
require.ErrorAs(t, err, &exitErr, "secret %v did not run", args)
}
return string(output), cmd.ProcessState.ExitCode()
}
for _, call := range []string{
"vault improt", "unlocker foo", "version foo", "generate foo",
} {
parent, unknown, _ := strings.Cut(call, " ")
output, exitCode := run(parent, unknown)
assert.Equal(t, 1, exitCode, "secret %s should fail:\n%s", call, output)
assert.Contains(t, output, fmt.Sprintf(
"unknown command %q for \"secret %s\"", unknown, parent))
output, exitCode = run(parent)
assert.Equal(t, 0, exitCode,
"secret %s should succeed:\n%s", parent, output)
assert.Contains(t, output, usageHeading,
"secret %s should print its help", parent)
}
}
+20 -113
View File
@@ -15,7 +15,6 @@ import (
"strings" "strings"
"time" "time"
"filippo.io/age"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"sneak.berlin/go/secret/internal/secret" "sneak.berlin/go/secret/internal/secret"
@@ -40,10 +39,6 @@ var (
// composes "GPG key <id> is already added as an unlocker". // composes "GPG key <id> is already added as an unlocker".
errGPGKeyAlreadyUnlocker = errors.New( errGPGKeyAlreadyUnlocker = errors.New(
"is already added as an unlocker") "is already added as an unlocker")
// errVaultCreatedAgain carries only the message tail; the caller
// composes "vault '<name>' was removed and created again ...".
errVaultCreatedAgain = errors.New("was removed and created again " +
"with another key while it was being unlocked; nothing was changed")
) )
// UnlockerInfo represents unlocker information for display // UnlockerInfo represents unlocker information for display
@@ -131,12 +126,6 @@ func newUnlockerCmd() *cobra.Command {
Use: "unlocker", Use: "unlocker",
Short: "Manage unlockers", Short: "Manage unlockers",
Long: `Create, list, and remove unlockers for the current vault.`, Long: `Create, list, and remove unlockers for the current vault.`,
// Cobra checks Args only on a command that has a RunE; without
// one, an unknown subcommand prints this help and exits 0.
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
return cmd.Help()
},
} }
cmd.AddCommand(newUnlockerListCmd()) cmd.AddCommand(newUnlockerListCmd())
@@ -409,6 +398,12 @@ func (cli *Instance) printUnlockersTable(unlockers []UnlockerInfo) error {
// UnlockersAdd adds a new unlocker // UnlockersAdd adds a new unlocker
func (cli *Instance) UnlockersAdd(unlockerType string, cmd *cobra.Command) error { func (cli *Instance) UnlockersAdd(unlockerType string, cmd *cobra.Command) error {
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
switch unlockerType { switch unlockerType {
case unlockerTypePassphrase: case unlockerTypePassphrase:
return cli.addPassphraseUnlocker(cmd) return cli.addPassphraseUnlocker(cmd)
@@ -422,64 +417,6 @@ func (cli *Instance) UnlockersAdd(unlockerType string, cmd *cobra.Command) error
} }
} }
// unlockThenLock gets the long-term key of vlt, the vault an unlocker is
// added to, then takes the state directory lock and returns the function
// that releases it. Getting the key can ask for a passphrase, so it comes
// before the lock, as the question of askThenLock does, and no other
// command waits while the user answers. Once the lock is held, it fails if
// another command has removed the vault in the meantime, since writing the
// unlocker would create the vault's directory again, holding nothing else.
// It also fails if the vault was removed and created again with another
// long-term key, which the unlocker would then hold instead of the new
// vault's own.
func (cli *Instance) unlockThenLock(vlt *vault.Vault) (func(), error) {
ltIdentity, err := vlt.GetOrDeriveLongTermKey()
if err != nil {
return nil, fmt.Errorf("failed to get long-term key: %w", err)
}
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return nil, err
}
err = cli.checkLongTermKey(vlt, ltIdentity)
if err != nil {
release()
return nil, err
}
return release, nil
}
// checkLongTermKey checks, for the vault vlt unlocked as ltIdentity before
// the state directory lock was taken, that vlt still exists and that its
// long-term public key is still that of ltIdentity. Another command may have
// removed the vault in the meantime, or removed it and created it again with
// another long-term key. The caller holds the lock.
func (cli *Instance) checkLongTermKey(
vlt *vault.Vault, ltIdentity *age.X25519Identity,
) error {
_, err := cli.existingVault(vlt.GetName())
if err != nil {
return err
}
vaultDir, _ := vlt.GetDirectory()
ltPubKey, err := afero.ReadFile(cli.fs, filepath.Join(vaultDir, "pub.age"))
if err != nil {
return fmt.Errorf("failed to read long-term public key: %w", err)
}
if string(ltPubKey) != ltIdentity.Recipient().String() {
return fmt.Errorf("vault '%s' %w", vlt.GetName(), errVaultCreatedAgain)
}
return nil
}
// autoSelectUnlocker selects the newly created unlocker as current, // autoSelectUnlocker selects the newly created unlocker as current,
// printing a warning if selection fails // printing a warning if selection fails
func autoSelectUnlocker(cmd *cobra.Command, vlt *vault.Vault, unlockerID string) { func autoSelectUnlocker(cmd *cobra.Command, vlt *vault.Vault, unlockerID string) {
@@ -499,6 +436,9 @@ func (cli *Instance) addPassphraseUnlocker(cmd *cobra.Command) error {
return fmt.Errorf("failed to get current vault: %w", err) return fmt.Errorf("failed to get current vault: %w", err)
} }
// For passphrase unlockers, we don't need the vault to be unlocked
// The CreatePassphraseUnlocker method will handle getting the
// long-term key
vlt.Mnemonic, vlt.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase vlt.Mnemonic, vlt.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase
// The new passphrase is asked for even when SB_UNLOCK_PASSPHRASE is // The new passphrase is asked for even when SB_UNLOCK_PASSPHRASE is
@@ -518,12 +458,6 @@ func (cli *Instance) addPassphraseUnlocker(cmd *cobra.Command) error {
defer passphraseBuffer.Destroy() defer passphraseBuffer.Destroy()
} }
release, err := cli.unlockThenLock(vlt)
if err != nil {
return err
}
defer release()
passphraseUnlocker, err := vlt.CreatePassphraseUnlocker(passphraseBuffer) passphraseUnlocker, err := vlt.CreatePassphraseUnlocker(passphraseBuffer)
if err != nil { if err != nil {
return err return err
@@ -562,13 +496,12 @@ func (cli *Instance) addPGPUnlocker(cmd *cobra.Command) error {
cmd.Printf("Using default GPG key: %s\n", gpgKeyID) cmd.Printf("Using default GPG key: %s\n", gpgKeyID)
} }
// Check if this key is already added as an unlocker
vlt, err := vault.GetCurrentVault(cli.fs, cli.stateDir) vlt, err := vault.GetCurrentVault(cli.fs, cli.stateDir)
if err != nil { if err != nil {
return fmt.Errorf("failed to get current vault: %w", err) return fmt.Errorf("failed to get current vault: %w", err)
} }
vlt.Mnemonic, vlt.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase
// Resolve the GPG key ID to its fingerprint, once: the duplicate check // Resolve the GPG key ID to its fingerprint, once: the duplicate check
// and the new unlocker's metadata both use this result // and the new unlocker's metadata both use this result
fingerprint, err := secret.ResolveGPGKeyFingerprint(gpgKeyID) fingerprint, err := secret.ResolveGPGKeyFingerprint(gpgKeyID)
@@ -576,26 +509,21 @@ func (cli *Instance) addPGPUnlocker(cmd *cobra.Command) error {
return fmt.Errorf("failed to resolve GPG key fingerprint: %w", err) return fmt.Errorf("failed to resolve GPG key fingerprint: %w", err)
} }
// Refuse a key that is already an unlocker before asking for anything, // Check if this GPG key is already added
// and again once the lock is held, since another command may have added exists, err := cli.pgpUnlockerExists(vlt, fingerprint)
// it in the meantime
err = cli.checkPGPKeyNotAdded(vlt, gpgKeyID, fingerprint)
if err != nil { if err != nil {
return err return fmt.Errorf(
"could not check whether GPG key %s is already an unlocker: %w",
gpgKeyID, err,
)
} }
release, err := cli.unlockThenLock(vlt) if exists {
if err != nil { return fmt.Errorf("GPG key %s %w", gpgKeyID, errGPGKeyAlreadyUnlocker)
return err
}
defer release()
err = cli.checkPGPKeyNotAdded(vlt, gpgKeyID, fingerprint)
if err != nil {
return err
} }
pgpUnlocker, err := secret.CreatePGPUnlocker(vlt, gpgKeyID, fingerprint) pgpUnlocker, err := secret.CreatePGPUnlocker(cli.fs, cli.stateDir,
gpgKeyID, fingerprint, cli.Mnemonic, cli.UnlockPassphrase)
if err != nil { if err != nil {
return err return err
} }
@@ -770,27 +698,6 @@ func (cli *Instance) UnlockerSelect(unlockerID string) error {
return vlt.SelectUnlocker(unlockerID) return vlt.SelectUnlocker(unlockerID)
} }
// checkPGPKeyNotAdded fails when vlt already has a PGP unlocker for the GPG
// key gpgKeyID, whose fingerprint is fingerprint, or when that cannot be
// checked.
func (cli *Instance) checkPGPKeyNotAdded(
vlt *vault.Vault, gpgKeyID, fingerprint string,
) error {
exists, err := cli.pgpUnlockerExists(vlt, fingerprint)
if err != nil {
return fmt.Errorf(
"could not check whether GPG key %s is already an unlocker: %w",
gpgKeyID, err,
)
}
if exists {
return fmt.Errorf("GPG key %s %w", gpgKeyID, errGPGKeyAlreadyUnlocker)
}
return nil
}
// pgpUnlockerExists reports whether the vault already has a PGP unlocker // pgpUnlockerExists reports whether the vault already has a PGP unlocker
// for the GPG key with the given fingerprint. It returns an error, and no // for the GPG key with the given fingerprint. It returns an error, and no
// answer, when unlockers.d or an unlocker's metadata file cannot be read; // answer, when unlockers.d or an unlocker's metadata file cannot be read;
+5 -6
View File
@@ -9,12 +9,11 @@
// "nothing there" and go ahead. // "nothing there" and go ahead.
// //
// The tests make the look fail with a wrapper around the in-memory // The tests make the look fail with a wrapper around the in-memory
// filesystem, which the state directory lock refuses. So they call a // filesystem, which the state directory lock refuses. So they call the
// function that makes the look before any lock is taken: addPGPUnlocker for // function each command runs once it holds the lock, such as addPGPUnlocker
// UnlockersAdd, which looks again once it holds the lock, or, for a removal, // for UnlockersAdd, or, for a removal, the function that makes its checks,
// the function that makes its checks, such as findVaultToRemove for // such as findVaultToRemove for RemoveVault, which runs again under the
// RemoveVault, which runs again under the lock before anything is removed, // lock before anything is removed, with --force or without.
// with --force or without.
//nolint:testpackage // white-box test of unexported internals //nolint:testpackage // white-box test of unexported internals
package cli package cli
+9 -18
View File
@@ -29,12 +29,6 @@ func newVaultCmd() *cobra.Command {
Use: "vault", Use: "vault",
Short: "Manage vaults", Short: "Manage vaults",
Long: `Create, list, and select vaults for organizing secrets.`, Long: `Create, list, and select vaults for organizing secrets.`,
// Cobra checks Args only on a command that has a RunE; without
// one, an unknown subcommand prints this help and exits 0.
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
return cmd.Help()
},
} }
cmd.AddCommand(newVaultListCmd()) cmd.AddCommand(newVaultListCmd())
@@ -222,14 +216,18 @@ func (cli *Instance) resolvePassphrase() (*memguard.LockedBuffer, func(), error)
return passphraseBuffer, passphraseBuffer.Destroy, nil return passphraseBuffer, passphraseBuffer.Destroy, nil
} }
// CreateVault creates a new vault. It asks for the mnemonic and the // CreateVault creates a new vault
// passphrase before it takes the state directory lock, so that no other
// command waits while the user answers.
func (cli *Instance) CreateVault(cmd *cobra.Command, name string) error { func (cli *Instance) CreateVault(cmd *cobra.Command, name string) error {
secret.Debug("Creating new vault", "name", name, "state_dir", cli.stateDir) secret.Debug("Creating new vault", "name", name, "state_dir", cli.stateDir)
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
// Refuse an invalid name or an existing vault before asking for anything // Refuse an invalid name or an existing vault before asking for anything
err := vault.CheckNewVaultName(cli.fs, cli.stateDir, name) err = vault.CheckNewVaultName(cli.fs, cli.stateDir, name)
if err != nil { if err != nil {
return err return err
} }
@@ -261,14 +259,7 @@ func (cli *Instance) CreateVault(cmd *cobra.Command, name string) error {
} }
defer cleanupPassphrase() defer cleanupPassphrase()
release, err := vault.LockStateDir(cli.fs, cli.stateDir) // Create the vault with its passphrase unlocker
if err != nil {
return err
}
defer release()
// Create the vault with its passphrase unlocker. Another command may
// have created it since the check above: vault.CreateVault checks again.
vlt, err := vault.CreateVault(cli.fs, cli.stateDir, name, vlt, err := vault.CreateVault(cli.fs, cli.stateDir, name,
mnemonic, passphraseBuffer) mnemonic, passphraseBuffer)
if err != nil { if err != nil {
-6
View File
@@ -43,12 +43,6 @@ func VersionCommands(cli *Instance) *cobra.Command {
Short: "Manage secret versions", Short: "Manage secret versions",
Long: "Commands for managing secret versions including listing, " + Long: "Commands for managing secret versions including listing, " +
"promoting, and retrieving specific versions", "promoting, and retrieving specific versions",
// Cobra checks Args only on a command that has a RunE; without
// one, an unknown subcommand prints this help and exits 0.
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
return cmd.Help()
},
} }
// List versions command // List versions command
+2 -5
View File
@@ -149,9 +149,7 @@ func WriteDir(fs afero.Fs, dir string, write func(dir string) error) error {
// RemoveDirAtomic deletes the directory dir so that it disappears in one // RemoveDirAtomic deletes the directory dir so that it disappears in one
// rename: dir is moved into a new directory from TempDirFor, which is then // rename: dir is moved into a new directory from TempDirFor, which is then
// deleted. A crash part-way leaves only that temporary directory behind. If // deleted. A crash part-way leaves only that temporary directory behind.
// only deleting the temporary directory fails, dir is already gone, and the
// error names the temporary directory, which holds what is left of it.
func RemoveDirAtomic(fs afero.Fs, dir string) error { func RemoveDirAtomic(fs afero.Fs, dir string) error {
tmp, err := TempDirFor(fs, dir) tmp, err := TempDirFor(fs, dir)
if err != nil { if err != nil {
@@ -167,8 +165,7 @@ func RemoveDirAtomic(fs afero.Fs, dir string) error {
err = fs.RemoveAll(tmp) err = fs.RemoveAll(tmp)
if err != nil { if err != nil {
return fmt.Errorf("failed to remove %s after moving %s into it: %w", return fmt.Errorf("failed to remove %s: %w", dir, err)
tmp, dir, err)
} }
return nil return nil
+2 -1
View File
@@ -375,7 +375,8 @@ func testCreatePGPUnlocker(
} }
// Now create a PGP unlock key (this will use our custom GPGEncryptFunc) // Now create a PGP unlock key (this will use our custom GPGEncryptFunc)
pgpUnlocker, err := secret.CreatePGPUnlocker(vlt, keyID, fingerprint) pgpUnlocker, err := secret.CreatePGPUnlocker(
fs, stateDir, keyID, fingerprint, mnemonic, nil)
if err != nil { if err != nil {
t.Fatalf("Failed to create PGP unlock key: %v", err) t.Fatalf("Failed to create PGP unlock key: %v", err)
} }
+30 -36
View File
@@ -192,45 +192,58 @@ func generatePGPUnlockerName() (string, error) {
return fmt.Sprintf("%s-pgp-%s", hostname, enrollmentTime), nil return fmt.Sprintf("%s-pgp-%s", hostname, enrollmentTime), nil
} }
// pgpUnlockerDir returns the directory in vault for a new PGP unlocker, // pgpUnlockerDir returns the current vault and the directory in it for a
// named after the host and the time. // new PGP unlocker, named after the host and the time.
func pgpUnlockerDir(vault VaultInterface) (string, error) { //
//nolint:ireturn // the vault is only available behind VaultInterface
func pgpUnlockerDir(
fs afero.Fs, stateDir string,
) (VaultInterface, string, error) {
// Get current vault
vault, err := GetCurrentVault(fs, stateDir)
if err != nil {
return nil, "", fmt.Errorf("failed to get current vault: %w", err)
}
// Generate the unlocker name based on hostname and time // Generate the unlocker name based on hostname and time
unlockerName, err := generatePGPUnlockerName() unlockerName, err := generatePGPUnlockerName()
if err != nil { if err != nil {
return "", fmt.Errorf("failed to generate unlocker name: %w", err) return nil, "", fmt.Errorf("failed to generate unlocker name: %w", err)
} }
vaultDir, err := vault.GetDirectory() vaultDir, err := vault.GetDirectory()
if err != nil { if err != nil {
return "", fmt.Errorf("failed to get vault directory: %w", err) return nil, "", fmt.Errorf("failed to get vault directory: %w", err)
} }
return filepath.Join(vaultDir, "unlockers.d", unlockerName), nil return vault, filepath.Join(vaultDir, "unlockers.d", unlockerName), nil
} }
// CreatePGPUnlocker creates a new PGP unlocker and stores it in vault. // CreatePGPUnlocker creates a new PGP unlocker and stores it in the vault.
// It encrypts to the GPG key gpgKeyID and records fingerprint, that key's // It encrypts to the GPG key gpgKeyID and records fingerprint, that key's
// fingerprint as ResolveGPGKeyFingerprint returns it, in the metadata. // fingerprint as ResolveGPGKeyFingerprint returns it, in the metadata.
// Everything that can fail short of writing a file is done before anything // Everything that can fail short of writing a file is done before anything
// is written, and the files are written through WriteDir, so a failure // is written, and the files are written through WriteDir, so a failure
// leaves no partial unlocker. The long-term key comes from vault's // leaves no partial unlocker. The long-term key comes from mnemonic when it
// GetOrDeriveLongTermKey: the key it holds when it is unlocked, else the // is not nil, else from the current unlocker, which is given passphrase when
// key from its mnemonic or its current unlocker. // it is a passphrase unlocker.
func CreatePGPUnlocker( func CreatePGPUnlocker(
vault VaultInterface, gpgKeyID, fingerprint string, fs afero.Fs, stateDir, gpgKeyID, fingerprint string,
mnemonic, passphrase *memguard.LockedBuffer,
) (*PGPUnlocker, error) { ) (*PGPUnlocker, error) {
err := checkGPGAvailable() err := checkGPGAvailable()
if err != nil { if err != nil {
return nil, err return nil, err
} }
unlockerDir, err := pgpUnlockerDir(vault) vault, unlockerDir, err := pgpUnlockerDir(fs, stateDir)
if err != nil { if err != nil {
return nil, err return nil, err
} }
fs := vault.GetFilesystem() // The vault's GetOrDeriveLongTermKey, in step 2, uses both
vault.SetMnemonic(mnemonic)
vault.SetUnlockPassphrase(passphrase)
// Step 1: Generate a new age keypair for the PGP unlocker // Step 1: Generate a new age keypair for the PGP unlocker
ageIdentity, err := age.GenerateX25519Identity() ageIdentity, err := age.GenerateX25519Identity()
@@ -282,7 +295,7 @@ func CreatePGPUnlocker(
func encryptPGPUnlockerKeys( func encryptPGPUnlockerKeys(
vault VaultInterface, ageIdentity *age.X25519Identity, gpgKeyID string, vault VaultInterface, ageIdentity *age.X25519Identity, gpgKeyID string,
) ([]byte, []byte, error) { ) ([]byte, []byte, error) {
// The key vault holds, or else from its mnemonic or current unlocker // From the mnemonic or the current unlocker, as for a passphrase unlocker
ltIdentity, err := vault.GetOrDeriveLongTermKey() ltIdentity, err := vault.GetOrDeriveLongTermKey()
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("failed to get long-term key: %w", err) return nil, nil, fmt.Errorf("failed to get long-term key: %w", err)
@@ -375,8 +388,7 @@ func ResolveGPGKeyFingerprint(keyID string) (string, error) {
return "", fmt.Errorf("%w: %s", ErrGPGKeyNotFound, keyID) return "", fmt.Errorf("%w: %s", ErrGPGKeyNotFound, keyID)
} }
return "", fmt.Errorf( return "", fmt.Errorf("failed to resolve GPG key fingerprint: %w", err)
"failed to resolve GPG key fingerprint: %w", withGPGStderr(err))
} }
// Parse the output to extract the fingerprint // Parse the output to extract the fingerprint
@@ -406,24 +418,6 @@ func checkGPGAvailable() error {
return nil return nil
} }
// withGPGStderr appends gpg's stderr, which says why gpg failed, to err
// from cmd.Output. cmd.Output keeps that stderr in the *exec.ExitError.
// err is returned unchanged when gpg wrote nothing to stderr.
func withGPGStderr(err error) error {
var exitErr *exec.ExitError
if !errors.As(err, &exitErr) {
return err
}
stderr := strings.TrimSpace(string(exitErr.Stderr))
if stderr == "" {
return err
}
return fmt.Errorf("%w: %s", err, stderr)
}
// gpgEncryptDefault is the default implementation of GPG encryption // gpgEncryptDefault is the default implementation of GPG encryption
func gpgEncryptDefault(data *memguard.LockedBuffer, keyID string) ([]byte, error) { func gpgEncryptDefault(data *memguard.LockedBuffer, keyID string) ([]byte, error) {
if data == nil { if data == nil {
@@ -443,7 +437,7 @@ func gpgEncryptDefault(data *memguard.LockedBuffer, keyID string) ([]byte, error
output, err := cmd.Output() output, err := cmd.Output()
if err != nil { if err != nil {
return nil, fmt.Errorf("GPG encryption failed: %w", withGPGStderr(err)) return nil, fmt.Errorf("GPG encryption failed: %w", err)
} }
return output, nil return output, nil
@@ -456,7 +450,7 @@ func gpgDecryptDefault(encryptedData []byte) (*memguard.LockedBuffer, error) {
output, err := cmd.Output() output, err := cmd.Output()
if err != nil { if err != nil {
return nil, fmt.Errorf("GPG decryption failed: %w", withGPGStderr(err)) return nil, fmt.Errorf("GPG decryption failed: %w", err)
} }
// Create a secure buffer for the decrypted data // Create a secure buffer for the decrypted data
+6 -29
View File
@@ -58,8 +58,7 @@ func TestCreatePGPUnlockerFailureWritesNothing(t *testing.T) {
}} }}
_, err = secret.CreatePGPUnlocker( _, err = secret.CreatePGPUnlocker(
vault.NewVault(fs, testVaultStateDir, testVaultName), fs, testVaultStateDir, testGPGKeyID, testGPGFingerprint, nil, nil)
testGPGKeyID, testGPGFingerprint)
require.Error(t, err) require.Error(t, err)
vaultDir, err := vlt.GetDirectory() vaultDir, err := vlt.GetDirectory()
@@ -88,40 +87,18 @@ func TestPGPUnlockerAddedTwiceKeepsFirst(t *testing.T) {
fs := afero.NewMemMapFs() fs := afero.NewMemMapFs()
mnemonic := testMnemonicBuffer(t) mnemonic := testMnemonicBuffer(t)
vlt, err := vault.CreateVault(fs, testVaultStateDir, testVaultName, mnemonic, nil) _, err := vault.CreateVault(fs, testVaultStateDir, testVaultName, mnemonic, nil)
require.NoError(t, err) require.NoError(t, err)
first, err := secret.CreatePGPUnlocker(vlt, testGPGKeyID, testGPGFingerprint) first, err := secret.CreatePGPUnlocker(
fs, testVaultStateDir, testGPGKeyID, testGPGFingerprint, mnemonic, nil)
require.NoError(t, err) require.NoError(t, err)
firstFiles := dirFiles(t, fs, first.GetDirectory()) firstFiles := dirFiles(t, fs, first.GetDirectory())
second, err := secret.CreatePGPUnlocker(vlt, testGPGKeyID, testGPGFingerprint) second, err := secret.CreatePGPUnlocker(
fs, testVaultStateDir, testGPGKeyID, testGPGFingerprint, mnemonic, nil)
require.NoError(t, err) require.NoError(t, err)
assert.NotEqual(t, first.GetDirectory(), second.GetDirectory()) assert.NotEqual(t, first.GetDirectory(), second.GetDirectory())
assert.Equal(t, firstFiles, dirFiles(t, fs, first.GetDirectory())) assert.Equal(t, firstFiles, dirFiles(t, fs, first.GetDirectory()))
} }
// TestGPGDecryptErrorHasGPGReason runs the real gpg on data that is not an
// OpenPGP message and checks that the error carries the reason gpg writes
// to stderr.
func TestGPGDecryptErrorHasGPGReason(t *testing.T) {
// The empty GNUPGHOME keeps gpg away from the user's own keyring;
// LC_ALL=C keeps gpg's message in English.
t.Setenv("GNUPGHOME", t.TempDir())
t.Setenv("LC_ALL", "C")
_, err := secret.GPGDecryptFunc([]byte("not an OpenPGP message"))
require.ErrorContains(t, err, "gpg: no valid OpenPGP data found")
}
// TestGPGDecryptErrorWithoutGPGStderr checks that when gpg fails without
// writing to stderr, the error ends with the exit status and nothing after it.
//
//nolint:paralleltest // installFakeGPG uses t.Setenv
func TestGPGDecryptErrorWithoutGPGStderr(t *testing.T) {
installFakeGPG(t)
_, err := secret.GPGDecryptFunc([]byte("not an OpenPGP message"))
require.EqualError(t, err, "GPG decryption failed: exit status 1")
}
+5
View File
@@ -28,6 +28,11 @@ type VaultInterface interface {
GetFilesystem() afero.Fs GetFilesystem() afero.Fs
GetCurrentUnlocker() (Unlocker, error) GetCurrentUnlocker() (Unlocker, error)
GetOrDeriveLongTermKey() (*age.X25519Identity, error) GetOrDeriveLongTermKey() (*age.X25519Identity, error)
// SetMnemonic and SetUnlockPassphrase give GetOrDeriveLongTermKey the
// mnemonic to derive the long-term key from, and the passphrase for a
// current passphrase unlocker; nil for none.
SetMnemonic(mnemonic *memguard.LockedBuffer)
SetUnlockPassphrase(passphrase *memguard.LockedBuffer)
CreatePassphraseUnlocker( CreatePassphraseUnlocker(
passphrase *memguard.LockedBuffer) (*PassphraseUnlocker, error) passphrase *memguard.LockedBuffer) (*PassphraseUnlocker, error)
} }
+6
View File
@@ -118,6 +118,12 @@ func (m *MockVault) GetOrDeriveLongTermKey() (*age.X25519Identity, error) {
return nil, errNotImplementedInMock return nil, errNotImplementedInMock
} }
func (m *MockVault) SetMnemonic(mnemonic *memguard.LockedBuffer) {
m.mnemonic = mnemonic
}
func (m *MockVault) SetUnlockPassphrase(_ *memguard.LockedBuffer) {}
func (m *MockVault) CreatePassphraseUnlocker( func (m *MockVault) CreatePassphraseUnlocker(
_ *memguard.LockedBuffer, _ *memguard.LockedBuffer,
) (*PassphraseUnlocker, error) { ) (*PassphraseUnlocker, error) {
+4
View File
@@ -90,6 +90,10 @@ func (m *MockVersionVault) GetOrDeriveLongTermKey() (*age.X25519Identity, error)
return nil, errNotImplementedInMock return nil, errNotImplementedInMock
} }
func (m *MockVersionVault) SetMnemonic(_ *memguard.LockedBuffer) {}
func (m *MockVersionVault) SetUnlockPassphrase(_ *memguard.LockedBuffer) {}
func (m *MockVersionVault) CreatePassphraseUnlocker( func (m *MockVersionVault) CreatePassphraseUnlocker(
_ *memguard.LockedBuffer, _ *memguard.LockedBuffer,
) (*secret.PassphraseUnlocker, error) { ) (*secret.PassphraseUnlocker, error) {
+1 -1
View File
@@ -132,7 +132,7 @@ func TestGetSecretWithWrongMnemonic(t *testing.T) {
t.Cleanup(mnemonic.Destroy) t.Cleanup(mnemonic.Destroy)
vlt := vault.NewVault(created.GetFilesystem(), testStateDir, testVaultName) vlt := vault.NewVault(created.GetFilesystem(), testStateDir, testVaultName)
vlt.Mnemonic = mnemonic vlt.SetMnemonic(mnemonic)
_, err := vlt.GetSecret(testSecretName) _, err := vlt.GetSecret(testSecretName)
require.ErrorIs(t, err, vault.ErrMnemonicMismatch) require.ErrorIs(t, err, vault.ErrMnemonicMismatch)
+4 -22
View File
@@ -55,10 +55,7 @@ func ValidateVaultName(name string) error {
} }
// ResolveVaultSymlink reads the currentvault file to get the path to the current vault // ResolveVaultSymlink reads the currentvault file to get the path to the current vault
// The file contains just the vault name (e.g., "default"). An empty or // The file contains just the vault name (e.g., "default")
// invalid name, or the name of a vault that does not exist, is an error
// wrapping ErrVaultNotFound that names the file and says to run
// 'secret vault select <name>'.
func ResolveVaultSymlink(fs afero.Fs, currentVaultPath string) (string, error) { func ResolveVaultSymlink(fs afero.Fs, currentVaultPath string) (string, error) {
secret.Debug("resolveVaultSymlink starting", "path", currentVaultPath) secret.Debug("resolveVaultSymlink starting", "path", currentVaultPath)
@@ -77,21 +74,6 @@ func ResolveVaultSymlink(fs afero.Fs, currentVaultPath string) (string, error) {
stateDir := filepath.Dir(currentVaultPath) stateDir := filepath.Dir(currentVaultPath)
absolutePath := filepath.Join(stateDir, "vaults.d", vaultName) absolutePath := filepath.Join(stateDir, "vaults.d", vaultName)
// An empty name would make vaults.d itself the vault's directory
exists := false
if isValidVaultName(vaultName) {
exists, err = afero.DirExists(fs, absolutePath)
if err != nil {
return "", fmt.Errorf("failed to check if vault exists: %w", err)
}
}
if !exists {
return "", fmt.Errorf("vault '%s' named in %s %w; run "+
"'secret vault select <name>' to select a vault",
vaultName, currentVaultPath, ErrVaultNotFound)
}
secret.Debug("Resolved to absolute path", "absolute_path", absolutePath) secret.Debug("Resolved to absolute path", "absolute_path", absolutePath)
return absolutePath, nil return absolutePath, nil
@@ -235,9 +217,9 @@ func processMnemonicForVault(
// vault: one wrapping ErrInvalidVaultName when it is not a valid vault name, // vault: one wrapping ErrInvalidVaultName when it is not a valid vault name,
// and one wrapping ErrVaultExists when that vault already exists. Creating an // and one wrapping ErrVaultExists when that vault already exists. Creating an
// existing vault again would replace its keys, and its secrets could no // existing vault again would replace its keys, and its secrets could no
// longer be decrypted. The commands that create a vault check it before they // longer be decrypted. CreateVault checks this before writing anything; the
// ask for the mnemonic or passphrase, and CreateVault checks it again, under // commands that create a vault also check it before they ask for the
// the state directory lock, before writing anything. // mnemonic or passphrase, holding the state directory lock.
func CheckNewVaultName(fs afero.Fs, stateDir string, name string) error { func CheckNewVaultName(fs afero.Fs, stateDir string, name string) error {
err := ValidateVaultName(name) err := ValidateVaultName(name)
if err != nil { if err != nil {
+6 -13
View File
@@ -4,9 +4,7 @@ import (
"crypto/sha256" "crypto/sha256"
"encoding/hex" "encoding/hex"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"os"
"path/filepath" "path/filepath"
"github.com/spf13/afero" "github.com/spf13/afero"
@@ -35,10 +33,7 @@ func ComputeDoubleSHA256(data []byte) string {
} }
// GetNextDerivationIndex finds the next available derivation index for a given mnemonic // GetNextDerivationIndex finds the next available derivation index for a given mnemonic
// by deriving the public key for index 0 and using its hash to identify related vaults. // by deriving the public key for index 0 and using its hash to identify related vaults
// It fails when a vault's metadata file exists but cannot be read or parsed: the
// index that vault uses is then unknown, and handing it out again would give a
// second vault the same long-term key.
func GetNextDerivationIndex( func GetNextDerivationIndex(
fs afero.Fs, stateDir string, mnemonic string, fs afero.Fs, stateDir string, mnemonic string,
) (uint32, error) { ) (uint32, error) {
@@ -77,23 +72,21 @@ func GetNextDerivationIndex(
continue continue
} }
// Try to read vault metadata
metadataPath := filepath.Join(vaultsDir, entry.Name(), "vault-metadata.json") metadataPath := filepath.Join(vaultsDir, entry.Name(), "vault-metadata.json")
metadataBytes, err := afero.ReadFile(fs, metadataPath) metadataBytes, err := afero.ReadFile(fs, metadataPath)
if errors.Is(err, os.ErrNotExist) {
// Skip directories without metadata
continue
}
if err != nil { if err != nil {
return 0, fmt.Errorf("failed to read vault metadata %s: %w", metadataPath, err) // Skip vaults without metadata
continue
} }
var metadata Metadata var metadata Metadata
err = json.Unmarshal(metadataBytes, &metadata) err = json.Unmarshal(metadataBytes, &metadata)
if err != nil { if err != nil {
return 0, fmt.Errorf("failed to parse vault metadata %s: %w", metadataPath, err) // Skip vaults with invalid metadata
continue
} }
// Check if this vault uses the same mnemonic by comparing family hashes // Check if this vault uses the same mnemonic by comparing family hashes
-86
View File
@@ -1,15 +1,11 @@
package vault_test package vault_test
import ( import (
"errors"
"path/filepath" "path/filepath"
"strings" "strings"
"testing" "testing"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/secret/internal/secret"
"sneak.berlin/go/secret/internal/vault" "sneak.berlin/go/secret/internal/vault"
"sneak.berlin/go/secret/pkg/agehd" "sneak.berlin/go/secret/pkg/agehd"
) )
@@ -167,88 +163,6 @@ func testGetNextDerivationIndex(t *testing.T, fs afero.Fs) {
} }
} }
// errMetadataUnreadable is returned by the test filesystem in place of a
// successful open of one vault's metadata file.
var errMetadataUnreadable = errors.New("input/output error")
// metadataReadFailFs fails every open of the file at unreadablePath. The
// file still exists, so checking for it succeeds and only reading it fails.
type metadataReadFailFs struct {
afero.Fs
unreadablePath string
}
//nolint:ireturn // afero.File is the interface required by afero.Fs
func (f *metadataReadFailFs) Open(name string) (afero.File, error) {
if name == f.unreadablePath {
return nil, errMetadataUnreadable
}
return f.Fs.Open(name)
}
// TestCreateVaultWithCorruptMetadataFails checks that creating a vault from
// a mnemonic fails, naming the file and leaving no new vault, when another
// vault's metadata file exists but cannot be parsed or read. That vault's
// derivation index is then unknown, so the new vault could get the same one
// and the same key.
func TestCreateVaultWithCorruptMetadataFails(t *testing.T) {
t.Parallel()
vaultsDir := filepath.Join(testStateDir, "vaults.d")
metadataPath := filepath.Join(vaultsDir, "work", "vault-metadata.json")
tests := []struct {
name string
// breakMetadata makes the work vault's metadata file unusable on
// base and returns the filesystem to create the second vault on.
breakMetadata func(t *testing.T, base afero.Fs) afero.Fs
}{
{
name: "parsing the file fails",
breakMetadata: func(t *testing.T, base afero.Fs) afero.Fs {
t.Helper()
err := afero.WriteFile(base, metadataPath, []byte("{"),
secret.FilePerms)
require.NoError(t, err)
return base
},
},
{
name: "reading the file fails",
breakMetadata: func(_ *testing.T, base afero.Fs) afero.Fs {
return &metadataReadFailFs{Fs: base, unreadablePath: metadataPath}
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
base := afero.NewMemMapFs()
_, err := vault.CreateVault(base, testStateDir, "work",
testMnemonicBuffer(t), nil)
require.NoError(t, err)
fs := tt.breakMetadata(t, base)
_, err = vault.CreateVault(fs, testStateDir, "other",
testMnemonicBuffer(t), nil)
require.ErrorContains(t, err, metadataPath)
entries, err := afero.ReadDir(base, vaultsDir)
require.NoError(t, err)
require.Len(t, entries, 1)
assert.Equal(t, "work", entries[0].Name())
})
}
}
func testMetadataPersistence(t *testing.T, fs afero.Fs) { func testMetadataPersistence(t *testing.T, fs afero.Fs) {
t.Helper() t.Helper()
+1 -1
View File
@@ -336,7 +336,7 @@ func TestVaultVersionTimestampsAfterUnlock(t *testing.T) {
// A new instance is locked; AddSecret unlocks it with the passphrase // A new instance is locked; AddSecret unlocks it with the passphrase
vault := NewVault(fs, testStateDir, "test") vault := NewVault(fs, testStateDir, "test")
vault.UnlockPassphrase = passphrase vault.SetUnlockPassphrase(passphrase)
addTestSecretToVault(t, vault, testSecretPath, []byte("version-2"), true) addTestSecretToVault(t, vault, testSecretPath, []byte("version-2"), true)
require.False(t, fs.readAt.IsZero(), "AddSecret did not unlock the vault") require.False(t, fs.readAt.IsZero(), "AddSecret did not unlock the vault")
+12
View File
@@ -64,6 +64,18 @@ func (v *Vault) ClearLongTermKey() {
v.longTermKey = nil v.longTermKey = nil
} }
// SetMnemonic sets v.Mnemonic, for code that has v only as a
// secret.VaultInterface.
func (v *Vault) SetMnemonic(mnemonic *memguard.LockedBuffer) {
v.Mnemonic = mnemonic
}
// SetUnlockPassphrase sets v.UnlockPassphrase, for code that has v only as
// a secret.VaultInterface.
func (v *Vault) SetUnlockPassphrase(passphrase *memguard.LockedBuffer) {
v.UnlockPassphrase = passphrase
}
// GetOrDeriveLongTermKey gets the long-term key from memory or derives it // GetOrDeriveLongTermKey gets the long-term key from memory or derives it
// from available sources // from available sources
func (v *Vault) GetOrDeriveLongTermKey() (*age.X25519Identity, error) { func (v *Vault) GetOrDeriveLongTermKey() (*age.X25519Identity, error) {
+1 -1
View File
@@ -128,7 +128,7 @@ func TestAddSecretForceWithWrongPassphraseChangesNothing(t *testing.T) {
t.Cleanup(wrong.Destroy) t.Cleanup(wrong.Destroy)
vlt := vault.NewVault(fs, testStateDir, testVaultName) vlt := vault.NewVault(fs, testStateDir, testVaultName)
vlt.UnlockPassphrase = wrong vlt.SetUnlockPassphrase(wrong)
secretDir := filepath.Join(testStateDir, "vaults.d", testVaultName, secretDir := filepath.Join(testStateDir, "vaults.d", testVaultName,
"secrets.d", testSecretName) "secrets.d", testSecretName)
-1
View File
@@ -1,5 +1,4 @@
{ {
"private": true,
"devDependencies": { "devDependencies": {
"prettier": "3.8.1" "prettier": "3.8.1"
} }
+26 -46
View File
@@ -2,18 +2,18 @@
# script/bootstrap: install all dependencies needed to build and develop # script/bootstrap: install all dependencies needed to build and develop
# this repo. Idempotent: every install is guarded by a check so already # this repo. Idempotent: every install is guarded by a check so already
# installed tools are skipped. Base tooling comes from nix, apt, brew, # installed tools are skipped. Base tooling comes from nix, apt, brew,
# or apk (detected in that order); assumes nothing is present. The # or apk (detected in that order); assumes NOTHING is present (not git,
# installed node is used only when its major version is the pinned one; # make, node, yarn, go, or python). Node is used directly if installed;
# otherwise (node missing, or another major) the pinned version is # otherwise a pinned version is installed via nvm (installing nvm
# installed via nvm (installing nvm itself first, from a hash-verified # itself first, from a hash-verified release archive, never curl | sh).
# release archive, never curl | sh), and yarn and the packages are
# installed under it.
# golangci-lint is never installed: script/lint runs it in docker. # golangci-lint is never installed: script/lint runs it in docker.
#
# Uncomment the language sections in main() that apply to this repo.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Pinned versions, 2026-07-06 # Pinned versions, 2026-07-06. Never "latest" or "lts"; exact versions.
NODE_VERSION="22.17.0" NODE_VERSION="22.17.0"
NVM_VERSION="0.40.3" NVM_VERSION="0.40.3"
# sha256 of https://github.com/nvm-sh/nvm/archive/refs/tags/v0.40.3.tar.gz # sha256 of https://github.com/nvm-sh/nvm/archive/refs/tags/v0.40.3.tar.gz
@@ -22,7 +22,6 @@ YARN_VERSION="1.22.22"
PKGMGR="" PKGMGR=""
SUDO="" SUDO=""
APT_UPDATED=""
detect_pkgmgr() { detect_pkgmgr() {
[ -n "$PKGMGR" ] && return 0 [ -n "$PKGMGR" ] && return 0
@@ -51,14 +50,7 @@ pkg_install() {
detect_pkgmgr detect_pkgmgr
case "$PKGMGR" in case "$PKGMGR" in
nix) nix-env -iA "nixpkgs.$1" ;; nix) nix-env -iA "nixpkgs.$1" ;;
apt) apt) $SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2" ;;
# Package lists may be empty (fresh images); refresh once per run.
if [ -z "$APT_UPDATED" ]; then
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get update
APT_UPDATED=1
fi
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2"
;;
brew) brew install "$3" ;; brew) brew install "$3" ;;
apk) apk add --no-cache "$4" ;; apk) apk add --no-cache "$4" ;;
esac esac
@@ -90,7 +82,7 @@ nvm_sh() {
ensure_nvm() { ensure_nvm() {
[ -s "$HOME/.nvm/nvm.sh" ] && return 0 [ -s "$HOME/.nvm/nvm.sh" ] && return 0
# nvm prerequisites; nvm itself requires bash # nvm prerequisites; nvm itself requires bash, so install it too
if missing bash; then pkg_install bash bash bash bash; fi if missing bash; then pkg_install bash bash bash bash; fi
if missing curl; then pkg_install curl curl curl curl; fi if missing curl; then pkg_install curl curl curl curl; fi
if missing git; then pkg_install git git git git; fi if missing git; then pkg_install git git git git; fi
@@ -103,68 +95,56 @@ ensure_nvm() {
rm -rf "$tmp" rm -rf "$tmp"
} }
# True when the node on PATH has the pinned major version. Only the
# major is compared: the Dockerfile stages start from a node 22 alpine
# image whose exact version is not the pin, and nvm cannot install a
# prebuilt node on alpine. Another major is not used: the pinned yarn 1
# prints a deprecation warning under node 24. script/fmt and
# script/fmt-check carry this function unchanged, to run the yarn
# installed here.
node_is_pinned_major() {
if ! command -v node >/dev/null 2>&1; then return 1; fi
installed="$(node --version)"
installed="${installed#v}"
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
}
ensure_node() { ensure_node() {
if node_is_pinned_major; then if ! missing node; then return 0; fi
echo "bootstrap: using node $(node --version)"
return 0
fi
ensure_nvm ensure_nvm
nvm_sh "nvm install $NODE_VERSION" nvm_sh "nvm install $NODE_VERSION"
nvm_sh "nvm use $NODE_VERSION >/dev/null && \
echo \"bootstrap: using node \$(node --version)\""
} }
ensure_yarn() { ensure_yarn() {
if ! node_is_pinned_major; then
nvm_sh "nvm use $NODE_VERSION >/dev/null && corepack enable && \
corepack prepare yarn@$YARN_VERSION --activate"
return 0
fi
if ! missing yarn; then return 0; fi if ! missing yarn; then return 0; fi
if ! missing corepack; then if ! missing corepack; then
corepack enable corepack enable
corepack prepare "yarn@$YARN_VERSION" --activate corepack prepare "yarn@$YARN_VERSION" --activate
elif [ -s "$HOME/.nvm/nvm.sh" ]; then
nvm_sh "nvm use $NODE_VERSION >/dev/null && corepack enable && \
corepack prepare yarn@$YARN_VERSION --activate"
else else
npm install -g "yarn@$YARN_VERSION" npm install -g "yarn@$YARN_VERSION"
fi fi
} }
install_js_deps() { install_js_deps() {
if node_is_pinned_major; then if missing yarn && [ -s "$HOME/.nvm/nvm.sh" ]; then
yarn install --frozen-lockfile
else
nvm_sh "nvm use $NODE_VERSION >/dev/null && cd \"$ROOT\" && \ nvm_sh "nvm use $NODE_VERSION >/dev/null && cd \"$ROOT\" && \
yarn install --frozen-lockfile" yarn install --frozen-lockfile"
else
yarn install --frozen-lockfile
fi fi
} }
main() { main() {
cd "$ROOT" cd "$ROOT"
if missing make; then pkg_install gnumake make make make; fi # Base tooling (every repo)
if missing git; then pkg_install git git git git; fi if missing git; then pkg_install git git git git; fi
if missing make; then pkg_install gnumake make make make; fi
# ---- JS / docs repos ----
# prettier, pinned in package.json and yarn.lock, formats the markdown
ensure_node ensure_node
ensure_yarn ensure_yarn
install_js_deps install_js_deps
# ---- Go repos ----
if missing go; then pkg_install go golang go go; fi if missing go; then pkg_install go golang go go; fi
go mod download go mod download
# ---- Python repos ----
# if missing python3; then pkg_install python3 python3 python3 python3; fi
# python3 -m venv .venv
# ./.venv/bin/pip install -e '.[dev]'
echo "bootstrap complete" echo "bootstrap complete"
} }
+9 -13
View File
@@ -2,11 +2,9 @@
# script/cibuild: run the CI build. It bootstraps first: a CI runner # script/cibuild: run the CI build. It bootstraps first: a CI runner
# checks out and runs this and nothing else, and script/fmt-check runs # checks out and runs this and nothing else, and script/fmt-check runs
# the formatter on the host, which a pristine checkout cannot do. # the formatter on the host, which a pristine checkout cannot do.
# A new CACHEBUST value reruns every RUN step, since a cached gate phase # --no-cache for the same reason as script/docker: the gate phases the
# is a check that did not run; unlike --no-cache, it keeps Go's build cache. # final stage depends on are RUN steps, and a cached one is a check that
# This relies on every stage of the Dockerfile declaring ARG CACHEBUST # did not run.
# first; a stage without that line is served from cache on an unchanged
# tree, with no warning.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -16,17 +14,15 @@ main() {
cd "$ROOT" cd "$ROOT"
"$SCRIPT_DIR/bootstrap" "$SCRIPT_DIR/bootstrap"
"$SCRIPT_DIR/check" "$SCRIPT_DIR/check"
# The version and the tag each get their own line: a failing # Own line: a failing command substitution inside an argument does
# command substitution inside an argument does not trip `set -e`, # not trip `set -e`, so the inline form degrades silently to an
# so the inline form degrades silently to an empty constant. The # empty constant. The VERSION build argument takes precedence over
# VERSION build argument takes precedence over the version a build # the version a build stage derives from the .git in the context.
# stage derives from the .git in the context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)" version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
tag="$("$SCRIPT_DIR/projectname")" docker build --no-cache \
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--build-arg VERSION="$version" \ --build-arg VERSION="$version" \
-t "$tag" . -t "$("$SCRIPT_DIR/projectname")" .
} }
main "$@" main "$@"
+8 -13
View File
@@ -1,11 +1,8 @@
#!/bin/sh #!/bin/sh
# script/docker: build the Docker image tagged with the project name. # script/docker: build the Docker image tagged with the project name.
# Identical in all repos; the tag comes from script/projectname. # Identical in all repos; the tag comes from script/projectname.
# A new CACHEBUST value reruns every RUN step, since a cached gate phase # --no-cache because the gate phases the final stage depends on are RUN
# is a check that did not run; unlike --no-cache, it keeps Go's build cache. # steps, and a cached one is a check that did not run.
# This relies on every stage of the Dockerfile declaring ARG CACHEBUST
# first; a stage without that line is served from cache on an unchanged
# tree, with no warning.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -13,17 +10,15 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
# The version and the tag each get their own line: a failing # Own line: a failing command substitution inside an argument does
# command substitution inside an argument does not trip `set -e`, # not trip `set -e`, so the inline form degrades silently to an
# so the inline form degrades silently to an empty constant. The # empty constant. The VERSION build argument takes precedence over
# VERSION build argument takes precedence over the version a build # the version a build stage derives from the .git in the context.
# stage derives from the .git in the context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)" version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
tag="$("$SCRIPT_DIR/projectname")" docker build --no-cache \
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--build-arg VERSION="$version" \ --build-arg VERSION="$version" \
-t "$tag" . -t "$("$SCRIPT_DIR/projectname")" .
} }
main "$@" main "$@"
+6 -16
View File
@@ -8,25 +8,15 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Must match the pin in script/bootstrap. # Must match the pin in script/bootstrap.
NODE_VERSION="22.17.0" NODE_VERSION="22.17.0"
# True when the node on PATH has the pinned major version: the same # script/bootstrap installs node and yarn under nvm and leaves neither
# function as in script/bootstrap, which uses it to decide where it # on the PATH of the shell that called it, so resolve the pinned
# installs yarn. # toolchain here the way bootstrap's own install step does. nvm is a
node_is_pinned_major() { # bash script, hence the subshell.
if ! command -v node >/dev/null 2>&1; then return 1; fi
installed="$(node --version)"
installed="${installed#v}"
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
}
# Run the yarn script/bootstrap installed: the yarn on PATH when the
# node on PATH has the pinned major version, and otherwise yarn under
# the pinned node in nvm, which bootstrap leaves off the PATH of the
# shell that called it. nvm is a bash script, hence the subshell.
run_yarn() { run_yarn() {
if node_is_pinned_major && command -v yarn >/dev/null 2>&1; then if command -v yarn >/dev/null 2>&1; then
exec yarn "$@" exec yarn "$@"
fi fi
if node_is_pinned_major || [ ! -s "$HOME/.nvm/nvm.sh" ]; then if [ ! -s "$HOME/.nvm/nvm.sh" ]; then
echo "fmt: no yarn; run script/bootstrap first" >&2 echo "fmt: no yarn; run script/bootstrap first" >&2
exit 1 exit 1
fi fi
+6 -16
View File
@@ -8,25 +8,15 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Must match the pin in script/bootstrap. # Must match the pin in script/bootstrap.
NODE_VERSION="22.17.0" NODE_VERSION="22.17.0"
# True when the node on PATH has the pinned major version: the same # script/bootstrap installs node and yarn under nvm and leaves neither
# function as in script/bootstrap, which uses it to decide where it # on the PATH of the shell that called it, so resolve the pinned
# installs yarn. # toolchain here the way bootstrap's own install step does. nvm is a
node_is_pinned_major() { # bash script, hence the subshell.
if ! command -v node >/dev/null 2>&1; then return 1; fi
installed="$(node --version)"
installed="${installed#v}"
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
}
# Run the yarn script/bootstrap installed: the yarn on PATH when the
# node on PATH has the pinned major version, and otherwise yarn under
# the pinned node in nvm, which bootstrap leaves off the PATH of the
# shell that called it. nvm is a bash script, hence the subshell.
run_yarn() { run_yarn() {
if node_is_pinned_major && command -v yarn >/dev/null 2>&1; then if command -v yarn >/dev/null 2>&1; then
exec yarn "$@" exec yarn "$@"
fi fi
if node_is_pinned_major || [ ! -s "$HOME/.nvm/nvm.sh" ]; then if [ ! -s "$HOME/.nvm/nvm.sh" ]; then
echo "fmt-check: no yarn; run script/bootstrap first" >&2 echo "fmt-check: no yarn; run script/bootstrap first" >&2
exit 1 exit 1
fi fi
+1
View File
@@ -1,6 +1,7 @@
#!/bin/sh #!/bin/sh
# script/install-precommit: install the git pre-commit hook that runs # script/install-precommit: install the git pre-commit hook that runs
# script/precommit. Our own extension to scripts-to-rule-them-all. # script/precommit. Our own extension to scripts-to-rule-them-all.
# Generic: needs no adaptation.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
+5 -8
View File
@@ -5,12 +5,9 @@
# make its answer untrustworthy. # make its answer untrustworthy.
# #
# The phase is not the last stage in the file, so it is built only when # The phase is not the last stage in the file, so it is built only when
# --target names it. A new CACHEBUST value reruns every RUN step, since # --target names it. --no-cache because a cached lint layer is a lint
# a cached lint layer is a lint that did not run; unlike --no-cache, it # that did not run. The tag makes each build replace the previous image
# keeps Go's build cache. This relies on every stage of the Dockerfile # instead of leaving a dangling one behind.
# declaring ARG CACHEBUST first; a stage without that line is served
# from cache on an unchanged tree, with no warning.
# --output type=cacheonly writes no image, since nothing uses one.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -18,9 +15,9 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
docker build --build-arg CACHEBUST="$(date +%s)-$$" \ docker build --no-cache \
--target lint \ --target lint \
--output type=cacheonly . -t "$("$SCRIPT_DIR/projectname")-lint" .
} }
main "$@" main "$@"
+2 -1
View File
@@ -1,6 +1,7 @@
#!/bin/sh #!/bin/sh
# script/setup: set up the repo for development after a fresh clone: # script/setup: set up the repo for development after a fresh clone:
# installs dependencies and the git pre-commit hook. # installs dependencies (script/bootstrap) and the git pre-commit hook.
# Add any repo-specific initialization (db init, .env template) here.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
+4 -8
View File
@@ -2,12 +2,8 @@
# script/test: run the test suite. Testing is a phase of the Dockerfile # script/test: run the test suite. Testing is a phase of the Dockerfile
# and this builds that phase alone, on the same terms as script/lint: # and this builds that phase alone, on the same terms as script/lint:
# --target because a phase that is not the last stage is built only when # --target because a phase that is not the last stage is built only when
# named, and a new CACHEBUST value because a cached test layer is a test # named, --no-cache because a cached test layer is a test that did not
# that did not run; unlike --no-cache, it keeps Go's build cache. This # run, and a tag so each build replaces the previous image.
# relies on every stage of the Dockerfile declaring ARG CACHEBUST first;
# a stage without that line is served from cache on an unchanged tree,
# with no warning. --output type=cacheonly writes no image, since
# nothing uses one.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -15,9 +11,9 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
docker build --build-arg CACHEBUST="$(date +%s)-$$" \ docker build --no-cache \
--target test \ --target test \
--output type=cacheonly . -t "$("$SCRIPT_DIR/projectname")-test" .
} }
main "$@" main "$@"