26 Commits
Author SHA1 Message Date
clawbot 176aee05aa Point the commandWait comment at the Dockerfile's test limit (closes #195)
check / check (push) Waiting to run
The comment said script/test gives the package 30 seconds. script/test
sets no time limit; the test phase of the Dockerfile sets one with
go test -timeout. The comment now names that place instead of a number.

Model: opus-5-5
2026-10-15 20:25:07 +02:00
clawbot 26a4739615 Keep Go's build cache in check builds: re-vendor sneak/prompts e3f79c4 (closes #129)
check / check (push) Canceled after 0s
The vendored files are copies from sneak/prompts e3f79c4, with this
repository's own entries kept. The script/ builds pass a new CACHEBUST
value instead of --no-cache, which emptied Go's build cache mount, so
every make test compiled the standard library and every dependency with
-race. Every Dockerfile stage declares ARG CACHEBUST first, so every RUN
step still reruns on an unchanged tree. The steps that compile mount
Go's build cache, and both go test runs keep -count=1. The lint and test
builds write no image. script/bootstrap, script/fmt and script/fmt-check
are the canonical files plus this repository's Go steps.

Model: opus-5-5
2026-10-15 19:58:38 +02:00
clawbot 8851171877 Set the time zone in main and run Entry in its own test process (closes #192)
check / check (push) Waiting to run
cli.Entry() set time.Local to UTC and ended with memguard.Purge(), and
TestEntryWipesBuffersAndReturnsExitCode ran it in the same process as
the other tests. A timer left by an earlier test could read time.Local
as it changed, which the race detector reported on some runs, and the
purge could destroy a buffer still in use, the likeliest cause of a
crash seen on another run.

main() in cmd/secret now sets time.Local before it calls cli.Entry();
the test that runs the built binary with TZ=America/New_York still
covers it. The Entry test runs cli.Entry() in a copy of the test binary
started with SECRET_TEST_RUN_ENTRY set, which init picks up so that
TestMain does not run there.

Model: opus-5-5
2026-10-15 19:25:10 +02:00
clawbot 25f4652e70 Log the audit fixes in TODO.md (closes #173)
check / check (push) Waiting to run
Completed Steps gets one entry for each fix merged into next since
v1.0.0-rc2 that had none, newest first in landing order. Each entry says
what its landing commit says and carries that commit's date in UTC. The
five fixes that logged themselves before this issue's decision keep
their entries and get no second one. Status no longer says there are no
git tags.

Model: opus-5-5
2026-10-15 17:58:57 +02:00
clawbot 4c47b233bb A secret mv between vaults unlocks both vaults before taking the lock (closes #189)
check / check (push) Waiting to run
A move between two vaults unlocked them while holding the state directory
lock, and unlocking can ask for a passphrase, so every other changing
command waited at that prompt. moveSecretCrossVault now checks the move,
refusing an existing destination secret without --force, unlocks both
vaults, and only then takes the lock. Under the lock, it checks that the
destination vault still has the long-term key it unlocked, and
copySecretAndRemoveSource checks the move again before it copies. A move
within a vault takes the lock in moveSecretWithinVault, so MoveSecret
takes none itself. The error for a vault created again with another key,
shared with secret unlocker add, now ends "nothing was changed".

Model: opus-5-5
2026-10-15 10:49:06 +02:00
clawbot fac2753819 Ask for the mnemonic and passphrase before taking the state directory lock (closes #152)
check / check (push) Waiting to run
init, vault create, unlocker add, and add, import and generate secret
with --force asked for the mnemonic or passphrase, or unlocked the
vault, while holding the state directory lock, so every other command
that changes anything waited at the prompt. They now ask and unlock
first and lock after. Under the lock, unlocker add fails if the vault
was removed, or created again with another key, and refuses a GPG key
added meanwhile; a forced add of a secret created meanwhile lets go of
the lock, gets the key and locks again. secret.CreatePGPUnlocker takes
the unlocked vault; SetMnemonic and SetUnlockPassphrase are gone.

Not done: secret move between vaults still unlocks both under the lock
(#189).

Model: opus-5-5
2026-10-15 09:02:13 +02:00
clawbot 9c4968c768 A cross-vault secret mv that cannot remove the source fails (closes #159)
check / check (push) Waiting to run
When `secret mv` copied a secret into another vault but could not remove
the source, it printed a warning and the usual "Moved" line and exited 0.
It now fails. If the source secret's directory is still in place, the
error says the copy holds all versions and the source remains. If the
source left its vault and only the temporary directory it was moved into
could not be deleted, the error says the secret is moved and names that
directory, which the error of RemoveDirAtomic now does.

The tests are in package cli and call moveSecret, because the state
directory lock refuses the wrapped filesystems that make the removal fail.

Model: opus-5-5
2026-10-15 07:31:54 +02:00
clawbot 93351cbd36 Hold the encrypt and decrypt result in memory until it is written (closes #143)
check / check (push) Waiting to run
With --output, secret encrypt and secret decrypt emptied that file before
reading any input, so a failed command left it empty, and encrypting a file
onto itself gave the encryption of nothing.

With --output they now hold the result in memory and open --output, with the
same call as before, only once encryption or decryption has succeeded. A
failure to encrypt or decrypt leaves --output untouched, and --input can be the
same file. A failure while writing --output, such as a full disk, can leave it
empty or partly written; an error from closing it is reported. Without
--output the result still goes to stdout as it is produced.

Model: opus-5-5
2026-10-15 07:31:47 +02:00
clawbot 03416d417c Commands fail when currentvault names no vault (closes #156)
check / check (push) Waiting to run
An empty currentvault, or one naming a vault that does not exist, made
`secret list` report an empty vault with exit 0 and `secret add` fail on a
missing directory. ResolveVaultSymlink now refuses an empty or invalid
name, or one with no directory under vaults.d, with an error that wraps
ErrVaultNotFound, names the file and says to run
`secret vault select <name>`. Every command that uses the current vault
gets it; `secret vault list` and `secret info` still run.

newSizeTestVault wrote a full path into currentvault, which resolved only
because the last path element was taken as the name; it now relies on
CreateVault selecting the vault.

Model: opus-5-5
2026-10-15 07:14:44 +02:00
clawbot 42f4961937 Print every time in UTC, whatever the time zone (closes #169)
check / check (push) Waiting to run
secret list, secret version list, secret unlocker list and secret info
printed times in the machine's time zone without naming it, so one vault
showed different times on two machines. Entry() now sets time.Local to
UTC before anything else, as AGENTS.md policy 6 asks, so every time the
program prints or writes is in UTC: --json times end in Z, and a new
version's name takes the UTC date. The README says so.

A test runs the built secret list with TZ=America/New_York and checks the
table and the JSON against the UTC time set on the secret's current file.

Rule suppressed: gosmopolitan, which flags any use of time.Local.

Model: opus-5-5
2026-10-15 06:48:51 +02:00
clawbot f0bf3b24a6 Unknown subcommands of vault, unlocker, version and generate fail (closes #183)
check / check (push) Waiting to run
Given a subcommand they do not have, `secret vault`, `secret unlocker`,
`secret version` and `secret generate` printed their help and exited 0,
so a mistyped command looked like a successful one. Each now exits 1
with `unknown command "<name>" for "secret <parent>"`, and with no
arguments still prints its help.

Cobra checks a command's Args only when it has a RunE, so each parent
gets `Args: cobra.NoArgs` and a RunE that prints its help. Args alone
changes nothing. TestNoVaultImportCommand required the old answer to
`secret vault import` and now requires the error.

Judgement call: their help now shows a `secret <parent> [flags]` usage
line, which cobra adds for any command that runs.
Judgement call: `secret vault help` now fails as an unknown subcommand;
`secret help vault` and `secret vault --help` still work.

Model: opus-5-5
2026-10-15 06:02:45 +02:00
clawbot 67c2829f58 Ask for the new passphrase in unlocker add when the variable is set (closes #154)
check / check (push) Waiting to run
With SB_UNLOCK_PASSPHRASE set, `secret unlocker add passphrase` gave the
new unlocker the passphrase in the variable without asking. It now asks
on the terminal for the new passphrase and uses the variable to unlock
the vault. With no controlling terminal the new unlocker takes the
variable's passphrase, and the output says so.

secret.ErrNoTerminal is exported so that the command can tell a missing
terminal from other failures to read, such as nothing typed or a
mismatched confirmation, which still fail the command.

The in-process tests that add a passphrase unlocker with the variable
set rely on the Docker test phase having no controlling terminal; at a
terminal they would ask.

Model: opus-5-5
2026-10-15 06:02:31 +02:00
clawbot 5922de32df Stop vault creation when a vault's metadata cannot be read (closes #140)
check / check (push) Waiting to run
GetNextDerivationIndex passed over a vault whose vault-metadata.json could
not be read or parsed, so the next vault made from the same mnemonic could
get that vault's derivation index and so the same long-term key. It now
returns an error naming the file, so vault create and init stop with that
error and leave the state directory as it was. A vault directory with no
metadata file is still skipped.

The regression test breaks one vault's metadata, once with content that
does not parse and once with a test filesystem that fails to open it, and
requires CreateVault with the same mnemonic to fail, naming the file, and
leave no new vault.

Model: opus-5-5
2026-10-15 06:02:22 +02:00
clawbot 6bc95f7c69 README says secret add drops one trailing newline and import keeps it (closes #158)
check / check (push) Waiting to run
`secret add` drops one trailing newline from stdin, so
`echo value | secret add name` stores `value`. `secret import` stores a
file's bytes unchanged, and `secret get` writes the value with no
newline added. The README said none of this, so a PEM key piped into
`add` came back one byte short with no hint why. Its entries for `add`,
`get` and `import` now state each rule. A new test stores `a\n\n`
through `add` and through `import` and checks that `add` kept two bytes
and `import` three. No program code changes.

The new test passes on the code before this change; it fails when `add` keeps the newline.

Model: opus-5-5
2026-10-15 05:31:57 +02:00
clawbot fdfe2c92f6 Include gpg's stderr in the error when gpg fails (closes #146)
check / check (push) Waiting to run
When gpg failed while decrypting, encrypting or looking up a key
fingerprint, the error gave only gpg's exit status, so the user could
not tell why. cmd.Output keeps gpg's stderr in the *exec.ExitError it
returns; the three errors now end with that stderr, trimmed. When gpg
wrote nothing to stderr, the error is left as it was.

gpg's stderr can run to more than one line, and the error then does too.

The regression test gives gpg data that is not an OpenPGP message
instead of one encrypted to a missing key: gpg reports both the same
way, and this case starts no gpg-agent the test would have to stop.

Model: opus-5-5
2026-10-15 05:31:50 +02:00
clawbot 00e34b93ec secret version list no longer shows a version without NOT_AFTER as expired (closes #150)
check / check (push) Waiting to run
STATUS was `expired` for every version other than the current one, so after
a promote the newest version, which has no NOT_AFTER, read `expired`. STATUS
is now `current` for the current version, `expired` for another version
whose NOT_AFTER has passed, and `inactive` for any other version. README
describes the three.

Judgement call: the issue left the third word to the owner; `inactive` is
the word the plan comment on the issue chose.

Model: opus-5-5
2026-10-15 04:48:53 +02:00
clawbot 0ffffd5e3f Print an empty unlocker or vault list as [] in JSON (closes #148)
check / check (push) Waiting to run
`secret unlocker list --json` printed "unlockers": null for a vault with
no unlockers, and `secret vault list --json` printed "vaults": null when
vaults.d held no vaults. Both lists were nil slices, which encoding/json
writes as null. Each is now made as an empty slice before it is filled,
so an empty list prints as [].

vault.ListVaults therefore returns an empty slice instead of nil for an
empty vaults.d; its other callers only range over the result or check
its length.

Model: opus-5-5
2026-10-15 04:48:46 +02:00
clawbot 438c36e7a9 A secret too large for the locked-memory limit fails with an error, not a panic (closes #145)
check / check (push) Waiting to run
secret add, import and generate secret crashed with memguard's panic once a
value outgrew the locked-memory limit, and because the value was held twice,
half the usual 8 MiB limit was enough. secret now raises the limit to the hard
limit at startup. Each of the three commands refuses, before its first locked
allocation, a limit that leaves fewer than 8 pages for its keys and buffers.
The value is read a page at a time into a memguard.Stream, which keeps it
encrypted, and copied once into locked memory after its size is checked
against 100 MiB and the limit. It leaves the stream through Next, not Read:
Read re-encrypts a leftover piece, and memguard hangs if that runs out of
locked memory.

Model: opus-5-5
2026-10-15 04:02:14 +02:00
clawbot 220213abfe Count every file of each version in secret info's Total Size (closes #147)
check / check (push) Waiting to run
Total Size added up each version's data.age and metadata.age, but a
version keeps its value in value.age, so no secret's value was counted.
It now adds up every file in each version directory: the value, the
metadata and the version's two keys, as the plan on the issue decided.
A version directory that cannot be read adds nothing and gets a warning
naming it.

The new tests add a 10000-byte secret and check that the total grows by
at least that much, and check the warning for a version directory that
cannot be read.

Model: opus-5-5
2026-10-15 03:15:04 +02:00
clawbot bfb13af302 Remove secret vault import (closes #139)
check / check (push) Waiting to run
`secret vault import` took only a vault without a long-term key, and
`secret init` and `secret vault create` give every vault one, so it
always failed. A mnemonic enters through those two commands, so the
command goes, with its tests and its README entry. The help and the
note of `secret generate mnemonic` named `secret import`, which imports
a file as a secret; they now name `secret init` and
`secret vault create`.

`secret vault import` now prints the help of `secret vault` and exits
0, as cobra does for any subcommand of `secret vault` it does not know.

Judgement call: TestWorkflowMismatch in internal/vault, which replayed
the import step by step, is deleted.

Model: opus-5-5
2026-10-15 03:14:55 +02:00
clawbot acb4b9cb6a version promote --help says the current file is rewritten (closes #153)
check / check (push) Waiting to run
The help text for `secret version promote` and a comment in
PromoteVersion said promote updates a "current symlink". `current` is a
plain file holding the version name, and promote rewrites it; the README
already says so. A new test runs `secret version promote --help` and
checks the output does not mention a symlink and does name the current
file.

Model: opus-5-5
2026-10-15 02:31:47 +02:00
clawbot 13c7721bad Refuse an existing vault before asking for the mnemonic (closes #151)
check / check (push) Waiting to run
`secret init` and `secret vault create` asked for the mnemonic and the
unlocker passphrase, and `secret init` printed "Initialized secrets
manager at", before refusing a vault that already exists. The check
`vault.CreateVault` made before writing is now `vault.CheckNewVaultName`,
which both commands call while holding the state directory lock, before
the first prompt; `CreateVault` still calls it. `secret init` prints the
"Initialized" line only once the vault is created, so
`TestMnemonicNotReadNamesOnlyMnemonic` no longer expects it.

Judgement call: the same check validates the name, so `secret vault
create` with an invalid name also fails before the prompts.

Model: opus-5-5
2026-10-15 01:32:23 +02:00
clawbot b8842b6184 secret get fails when it cannot write the value (closes #155)
check / check (push) Waiting to run
GetSecret and GetSecretWithVersion discarded the result of writing the
value to stdout, so a full disk or a failing stdout left the caller with
an empty or cut-short file and exit status 0. The write error is now
returned, so the command prints it and exits 1.

Deviation: a stdout the shell closed before `secret` started still exits
0, because the Go runtime opens /dev/null in its place and the write
there succeeds.

Model: opus-5-5
2026-10-15 01:32:14 +02:00
clawbot faf46d12ff secret list gives each secret's own creation and last-update times (closes #142)
check / check (push) Waiting to run
secret list and secret list --json gave the time of the listing as every
secret's creation and last-update time, because Secret.LoadMetadata set
both to the current time. It now reads file modification times, so
listing still needs no passphrase: the last update is when the secret's
current file was written, and the creation is when its oldest version's
value.age was written. A move to another vault gives the copied files
the times of the originals.

GetSecretObject no longer loads the times, so secret get keeps its error
for a missing current file; secret list calls LoadMetadata itself.

Judgement call: the creation time comes from value.age, not from the
oldest version's directory, whose time changes when the next add
rewrites that version's metadata.age.

Model: opus-5-5
2026-10-15 00:32:05 +02:00
clawbot 6d95d96477 Use gpg.conf's default-key when unlocker add pgp has no --keyid (closes #141)
check / check (push) Waiting to run
getDefaultGPGKey took the default-key value from `gpgconf --list-options
gpg` as printed. gpgconf prints a string value with a leading `"` and
percent-escaped, so gpg rejected the key ID. The parsing moved into
parseDefaultGPGKey, which drops the `"` and percent-decodes the rest with
url.PathUnescape, which leaves `+` alone. A value that does not decode is
an error, not a fallback to the first secret key.

Rule suppressed: gosec G703 on the test's write of gpg.conf into the
GNUPGHOME directory newTestGPGKey made.

Model: opus-5-5
2026-10-15 00:31:53 +02:00
clawbot 72cd6c30a1 A wrong passphrase gives an error that says so (closes #157)
check / check (push) Waiting to run
A passphrase that did not decrypt the passphrase unlocker failed with age's
"no identity matched any of the recipients" and advice to give the vault a
new unlocker from its mnemonic, which would replace an unlocker that works.
GetIdentity now turns that age error into "wrong passphrase for unlocker
<id>", wrapping the new secret.ErrWrongPassphrase, and withMnemonicAdvice
leaves it alone, as it does when no passphrase was read. A passphrase from
SB_UNLOCK_PASSPHRASE and a typed one both reach this code. A priv.age that
is not an age file keeps the advice.

Judgement call: age cannot tell a wrong passphrase from a damaged key inside
a priv.age whose header reads, so that case now reads as a wrong passphrase.

Model: opus-5-5
2026-10-14 23:48:50 +02:00
68 changed files with 3712 additions and 1120 deletions
+3
View File
@@ -13,3 +13,6 @@ indent_style = tab
[*.go] [*.go]
indent_style = tab indent_style = tab
# This repository's own sections, such as one for another language it
# uses, go below this comment, and a re-vendor keeps them.
+11
View File
@@ -1,9 +1,20 @@
name: check name: check
on: [push] on: [push]
# Free the shared runner: a new push cancels only the same branch's older run.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs: jobs:
check: check:
runs-on: ubuntu-latest runs-on: ubuntu-latest
# Free the shared runner from a hung build.
timeout-minutes: 20
steps: steps:
# actions/checkout v4.2.2, 2026-02-22 # actions/checkout v4.2.2, 2026-02-22
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
# script/cibuild needs no token, so none is left in .git/config.
with:
persist-credentials: false
# All history and tags, so git describe finds the version tag.
fetch-depth: 0
- run: script/cibuild - run: script/cibuild
+5 -1
View File
@@ -27,7 +27,7 @@ node_modules/
# Environment files. `*.env` covers bare `.env` and the `prod.env` # Environment files. `*.env` covers bare `.env` and the `prod.env`
# convention. Only the templates `example.env` and `sample.env` are # convention. Only the templates `example.env` and `sample.env` are
# re-included below. A repository that commits any other template adds # re-included below. A repository that commits any other template adds
# its own negation after these lines, for example `!.env.example`. # its own negation at the end of this file, for example `!.env.example`.
*.[eE][nN][vV] *.[eE][nN][vV]
.[eE][nN][vV].* .[eE][nN][vV].*
.[eE][nN][vV][rR][cC] .[eE][nN][vV][rR][cC]
@@ -46,6 +46,10 @@ node_modules/
[iI][dD]_[eE][dD]25519 [iI][dD]_[eE][dD]25519
[iI][dD]_[eE][dD]25519_[sS][kK] [iI][dD]_[eE][dD]25519_[sS][kK]
# This repository's own entries, such as its build outputs, go below
# this comment, and a re-vendor keeps them. Anchor a binary built at the
# root: `/myapp`, never `myapp`, which also ignores `cmd/myapp/`.
# Go. /secret is the built binary, anchored so that it does not also match # Go. /secret is the built binary, anchored so that it does not also match
# the internal/secret/ package directory. # the internal/secret/ package directory.
*.log *.log
+2
View File
@@ -25,6 +25,8 @@ linters:
# silenced by disabling that name, not by enabling the successor. # silenced by disabling that name, not by enabling the successor.
- wsl # Deprecated, replaced by wsl_v5 - wsl # Deprecated, replaced by wsl_v5
- gomodguard # Deprecated, replaced by gomodguard_v2 - gomodguard # Deprecated, replaced by gomodguard_v2
# Misses findings at random in v2.14.0; back once a pinned release fixes it
- canonicalheader
settings: settings:
lll: lll:
line-length: 88 line-length: 88
+21 -11
View File
@@ -1,33 +1,40 @@
# Every stage declares ARG CACHEBUST first: script/ passes a new value
# on every build, which reruns every RUN step after it. The steps that
# compile mount Go's build cache, which survives from build to build.
# Lint phase. The linter is invoked directly rather than through `make # Lint phase. The linter is invoked directly rather than through `make
# lint` or `script/lint`, which are themselves a docker build. # lint` or `script/lint`, which are themselves a docker build.
# golangci/golangci-lint:v2.14.0 (Debian-based), 2026-09-24 # golangci/golangci-lint:v2.14.0 (Debian-based), 2026-09-24
FROM golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint FROM golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint
ARG CACHEBUST
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
COPY . . COPY . .
RUN go vet ./... RUN --mount=type=cache,target=/root/.cache/go-build \
RUN golangci-lint run --config .golangci.yml ./... go vet ./...
RUN --mount=type=cache,target=/root/.cache/go-build \
golangci-lint run --config .golangci.yml ./...
# The same checks on the code as a macOS build compiles it, which a Linux # The same checks on the code as a macOS build compiles it, which a Linux
# build never compiles. Cgo is off, because compiling cgo code for macOS # build never compiles. Cgo is off, because compiling cgo code for macOS
# needs Apple's SDK headers. # needs Apple's SDK headers.
RUN GOOS=darwin CGO_ENABLED=0 go vet ./... RUN --mount=type=cache,target=/root/.cache/go-build \
RUN GOOS=darwin CGO_ENABLED=0 golangci-lint run --config .golangci.yml ./... GOOS=darwin CGO_ENABLED=0 go vet ./...
RUN --mount=type=cache,target=/root/.cache/go-build \
GOOS=darwin CGO_ENABLED=0 golangci-lint run --config .golangci.yml ./...
# Test phase. -race needs cgo and so a C compiler, which the Debian Go # Test phase. -race needs cgo and so a C compiler, which the Debian Go
# image ships and the alpine one does not. # image ships and the alpine one does not. -count=1 because the mounted
# cache also holds test results, which go test would otherwise replay.
# golang:1.24.13-trixie, 2026-02-04 # golang:1.24.13-trixie, 2026-02-04
FROM golang@sha256:5835f052b784aa39f2fe9070def3568605c8bc3fcd810f10402066348b61e716 AS test FROM golang@sha256:5835f052b784aa39f2fe9070def3568605c8bc3fcd810f10402066348b61e716 AS test
ARG CACHEBUST
ENV CGO_ENABLED=1 ENV CGO_ENABLED=1
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
COPY . . COPY . .
# Go's build cache goes in a cache mount, not the image layer, which would RUN --mount=type=cache,target=/root/.cache/go-build \
# take seconds longer to export. --no-cache, on every build in script/,
# starts the mount empty; -count=1 keeps a build without it from taking
# test results from there.
RUN --mount=type=cache,id=sneak/secret/go-build-test,target=/root/.cache/go-build \
go test -count=1 -timeout 90s -race -cover ./... || \ go test -count=1 -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \ { echo "--- Rerunning with -v for details ---"; \
go test -count=1 -timeout 90s -race -v ./...; exit 1; } go test -count=1 -timeout 90s -race -v ./...; exit 1; }
@@ -37,6 +44,7 @@ RUN --mount=type=cache,id=sneak/secret/go-build-test,target=/root/.cache/go-buil
# lint and test passed. # lint and test passed.
# golang 1.24.13-alpine, 2026-03-10 # golang 1.24.13-alpine, 2026-03-10
FROM golang@sha256:8bee1901f1e530bfb4a7850aa7a479d17ae3a18beb6e09064ed54cfd245b7191 AS builder FROM golang@sha256:8bee1901f1e530bfb4a7850aa7a479d17ae3a18beb6e09064ed54cfd245b7191 AS builder
ARG CACHEBUST
COPY --from=lint /src/go.sum /dev/null COPY --from=lint /src/go.sum /dev/null
COPY --from=test /src/go.sum /dev/null COPY --from=test /src/go.sum /dev/null
# script/build compiles with cgo, so it needs a C compiler too. # script/build compiles with cgo, so it needs a C compiler too.
@@ -53,7 +61,8 @@ COPY . .
# .git present, a version that is still empty, dev or unknown fails the # .git present, a version that is still empty, dev or unknown fails the
# build: git is missing or could not read the checkout. # build: git is missing or could not read the checkout.
ARG VERSION ARG VERSION
RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \ RUN --mount=type=cache,target=/root/.cache/go-build \
VERSION="${VERSION:-$(git describe --tags --always)}"; \
if [ -e .git ]; then \ if [ -e .git ]; then \
case "$VERSION" in ""|dev|unknown) \ case "$VERSION" in ""|dev|unknown) \
echo "version is '$VERSION' although .git is present" >&2; \ echo "version is '$VERSION' although .git is present" >&2; \
@@ -65,6 +74,7 @@ RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \
# Runtime stage, and the last one # Runtime stage, and the last one
# alpine 3.23, 2026-03-10 # alpine 3.23, 2026-03-10
FROM alpine@sha256:25109184c71bdad752c8312a8623239686a9a2071e8825f20acb8f2198c3f659 FROM alpine@sha256:25109184c71bdad752c8312a8623239686a9a2071e8825f20acb8f2198c3f659
ARG CACHEBUST
RUN apk add --no-cache ca-certificates gnupg RUN apk add --no-cache ca-certificates gnupg
+83 -15
View File
@@ -67,6 +67,10 @@ and unlocker set. This allows for complete isolation between different contexts
## Commands Reference ## Commands Reference
Every time `secret` prints, in a table or in `--json` output, is in UTC,
whatever `TZ` or the system's time zone is set to. The date in a version's name
is the UTC date too.
### Confirmation Before Removal ### Confirmation Before Removal
`secret rm`, `secret version rm`, `secret vault remove` and `secret rm`, `secret version rm`, `secret vault remove` and
@@ -90,7 +94,8 @@ remove things pass `--force`.
#### `secret init` #### `secret init`
Initializes the secret manager with a default vault. Prompts for a BIP39 Initializes the secret manager with a default vault. Prompts for a BIP39
mnemonic phrase and creates the initial directory structure. mnemonic phrase and creates the initial directory structure. When the vault
`default` already exists, it fails before asking for the mnemonic or passphrase.
**Environment Variables:** **Environment Variables:**
@@ -105,14 +110,19 @@ Lists all available vaults. The current vault is marked.
#### `secret vault create <name>` #### `secret vault create <name>`
Creates a new vault with the specified name. Creates a new vault with the specified name. When a vault of that name already
exists, it fails before asking for the mnemonic or passphrase.
**Vault Name Format:** only lowercase ASCII letters, digits, `.`, `-` and `_` **Vault Name Format:** only lowercase ASCII letters, digits, `.`, `-` and `_`
are allowed, and a name must not be empty, `.` or `..`. are allowed, and a name must not be empty, `.` or `..`.
#### `secret vault select <name>` #### `secret vault select <name>`
Switches to the specified vault for subsequent operations. Switches to the specified vault for subsequent operations. When `currentvault`
is empty or names a vault that does not exist, every command that uses the
current vault fails with an error that names the file and says to run
`secret vault select <name>`. `secret vault list` still lists the vaults, and
`secret info` still runs; both show no vault as current.
#### `secret vault remove <name> [--force]` / `secret vault rm` ⚠️ 🛑 #### `secret vault remove <name> [--force]` / `secret vault rm` ⚠️ 🛑
@@ -127,9 +137,24 @@ one.
### Secret Management ### Secret Management
#### Secret Size
While `secret` works on a secret's value it keeps the value in locked memory, so
the value counts against the locked-memory limit (`ulimit -l`). `secret` raises
that limit to the hard limit when it starts. `secret add`, `secret import` and
`secret generate secret` refuse a value that does not fit in the limit with 8
memory pages to spare (32 KiB with 4 KiB pages), with an error that names the
limit. Linux's usual limit of 8 MiB allows a value of nearly 8 MiB; a larger one
needs a higher hard limit, such as `memlock` in `/etc/security/limits.conf`. No
value may be over 100 MiB, whatever the limit.
#### `secret add <secret-name> [--force]` #### `secret add <secret-name> [--force]`
Adds a secret to the current vault. Reads the secret value from stdin. Adds a secret to the current vault. Reads the secret value from stdin. When the
value ends in a newline, that one newline is dropped, so
`echo value | secret add name` stores `value`; from `printf 'a\n\n'` it stores
`a` and one newline. To keep a file's final newline, as in a PEM key, use
`secret import`, which stores the file's bytes unchanged.
- `--force, -f`: Overwrite existing secret - `--force, -f`: Overwrite existing secret
@@ -142,7 +167,13 @@ contain `//`, or have `..` as a path segment.
#### `secret get <secret-name> [--version <version>]` #### `secret get <secret-name> [--version <version>]`
Retrieves and outputs a secret value to stdout. Writes the secret's value to stdout exactly as stored, with no newline added
after it. A failed write of the value, such as to a file on a full disk, prints
the error and exits 1, with two exceptions. If the program reading from a pipe
has already exited, as in `secret get s | true`, the broken-pipe signal
(SIGPIPE) ends `secret` with no message and the shell reports exit status 141.
If the shell closed stdout before `secret` started (`secret get s >&-`), the
value is written to `/dev/null` and `secret get` exits 0.
- `--version, -v`: Get a specific version (default: current) - `--version, -v`: Get a specific version (default: current)
@@ -150,6 +181,13 @@ Retrieves and outputs a secret value to stdout.
Lists all secrets in the current vault. Optional filter for substring matching. Lists all secrets in the current vault. Optional filter for substring matching.
The `LAST UPDATED` column, and `updated_at` in the `--json` output, give the
time the secret's `current` file was last written: every command that adds a
version replaces it, and so does `secret version promote`. `created_at` gives
the time the `value.age` of the secret's oldest version was written. Both are
the files' modification times, so listing asks for no passphrase. Moving a
secret to another vault keeps both times.
#### `secret remove <secret-name> [--force]` / `secret rm` ⚠️ 🛑 #### `secret remove <secret-name> [--force]` / `secret rm` ⚠️ 🛑
**DANGER**: Permanently removes a secret and ALL its versions. It first asks for **DANGER**: Permanently removes a secret and ALL its versions. It first asks for
@@ -178,10 +216,18 @@ Moves or renames a secret within the current vault.
Lists all versions of a secret showing creation time, status, and validity Lists all versions of a secret showing creation time, status, and validity
period. period.
`STATUS` is `current` for the secret's current version, `expired` for another
version whose `NOT_AFTER` has passed, and `inactive` for any other version.
Adding a version sets `NOT_AFTER` on the version that was current;
`secret version promote` sets no times. A version that is not current therefore
shows `inactive` when no version was added while it was current.
#### `secret version promote <secret-name> <version>` #### `secret version promote <secret-name> <version>`
Promotes a specific version to current by rewriting the secret's `current` file Promotes a specific version to current by rewriting the secret's `current` file
to name it. Does not modify any timestamps, allowing for rollback scenarios. to name it. The versions' own times are left unchanged, allowing for rollback
scenarios; `secret list` then gives the time of the promote as the secret's last
update.
#### `secret version remove <secret-name> <version> [--force]` / `secret version rm` ⚠️ 🛑 #### `secret version remove <secret-name> <version> [--force]` / `secret version rm` ⚠️ 🛑
@@ -233,6 +279,12 @@ Creates a new unlocker of the specified type:
A vault has one passphrase unlocker: adding one replaces the one the vault has, A vault has one passphrase unlocker: adding one replaces the one the vault has,
which is removed only once the new one is the current unlocker. which is removed only once the new one is the current unlocker.
`secret unlocker add passphrase` asks on the terminal for the new passphrase
even when `SB_UNLOCK_PASSPHRASE` is set, and uses the variable to unlock the
vault. With no controlling terminal, as under cron or in a CI job, the new
unlocker takes its passphrase from `SB_UNLOCK_PASSPHRASE`, and the command says
so.
#### `secret unlocker remove <unlocker-id> [--force]` / `secret unlocker rm` ⚠️ 🛑 #### `secret unlocker remove <unlocker-id> [--force]` / `secret unlocker rm` ⚠️ 🛑
**DANGER**: Permanently removes an unlocker. It first asks for confirmation, **DANGER**: Permanently removes an unlocker. It first asks for confirmation,
@@ -266,14 +318,16 @@ Selects an unlocker as the current default for operations.
#### `secret import <secret-name> --source <filename>` #### `secret import <secret-name> --source <filename>`
Imports a secret from a file and stores it in the current vault under the given Imports a secret from a file and stores it in the current vault under the given
name. name. The file's bytes are stored unchanged, a final newline included.
#### `secret vault import [vault-name]`
Imports a mnemonic phrase into the specified vault (defaults to "default").
### Encryption Operations ### Encryption Operations
With `--output`, `secret encrypt` and `secret decrypt` hold the result in memory
and write it to `--output` only once encryption or decryption has succeeded, so
a failure to encrypt or decrypt leaves `--output` untouched. A failure while
writing `--output`, such as a full disk, can leave it empty or partly written.
The command can need up to about four times as much memory as the result.
#### `secret encrypt <secret-name> [--input=file] [--output=file]` #### `secret encrypt <secret-name> [--input=file] [--output=file]`
Encrypts data using an Age key stored as a secret. If the secret doesn't exist, Encrypts data using an Age key stored as a secret. If the secret doesn't exist,
@@ -377,15 +431,21 @@ line or in a CI job, they end up in shell history and CI logs. `secret` unsets
each one as soon as it has read it, so that the programs it runs itself, such as each one as soon as it has read it, so that the programs it runs itself, such as
`gpg`, do not inherit it, but that erases nothing: the environment the process `gpg`, do not inherit it, but that erases nothing: the environment the process
started with, and its memory, still hold the value. The interactive prompt, started with, and its memory, still hold the value. The interactive prompt,
which every command except `secret vault import` offers when the variable is not which every command offers when the variable is not set, is the safer default.
set, is the safer default; `secret vault import` has no prompt and needs both
variables.
The prompt asks on the terminal the command runs in, not on stdin, so it also The prompt asks on the terminal the command runs in, not on stdin, so it also
asks when stdin is a pipe, as in `echo value | secret add name --force` or asks when stdin is a pipe, as in `echo value | secret add name --force` or
`cat file | secret encrypt key`. With no controlling terminal, as under cron or `cat file | secret encrypt key`. With no controlling terminal, as under cron or
in a CI job, the command fails and names the variable to set. in a CI job, the command fails and names the variable to set.
A command that changes anything locks the state directory, through its file
`lock`, and another such command waits until the lock is released.
`secret init`, `secret vault create`, `secret unlocker add`, `secret mv` from
one vault to another, and `secret add`, `secret import` and
`secret generate secret` with `--force`, ask for the mnemonic or passphrase
before they take the lock, so a prompt of theirs left open in one terminal holds
up no command in another.
## Security Features ## Security Features
### Encryption ### Encryption
@@ -522,7 +582,11 @@ secret decrypt encryption/mykey --input document.txt.age --output document.txt
### Vault Management ### Vault Management
- **Derivation Index**: Each vault uses a unique derivation index from the - **Derivation Index**: Each vault uses a unique derivation index from the
mnemonic, and thus a unique key pair mnemonic, and thus a unique key pair; creating a vault (`secret init` and
`secret vault create`) fails, with an error naming the file, when any vault's
`vault-metadata.json` exists but cannot be read or parsed; a vault directory
with no `vault-metadata.json` is skipped, so the index it uses can be given to
a new vault
- **Public Key Hash**: Double SHA-256 hash of the vault's public key; the same - **Public Key Hash**: Double SHA-256 hash of the vault's public key; the same
hash of the index-0 public key identifies vaults from the same mnemonic hash of the index-0 public key identifies vaults from the same mnemonic
- **Automatic Key Derivation**: When creating vaults with a mnemonic, keys are - **Automatic Key Derivation**: When creating vaults with a mnemonic, keys are
@@ -611,6 +675,10 @@ provide:
- `script/install-precommit` — install the git pre-commit hook that runs - `script/install-precommit` — install the git pre-commit hook that runs
`script/precommit` `script/precommit`
Every stage of the `Dockerfile` declares `ARG CACHEBUST` as its first line after
`FROM`. The `script/` builds pass a new `CACHEBUST` value on every build, which
reruns every `RUN` step and keeps Go's build cache; `--no-cache` would empty it.
## Features ## Features
- **Multiple Authentication Methods**: Supports passphrase and PGP unlockers - **Multiple Authentication Methods**: Supports passphrase and PGP unlockers
+156 -79
View File
@@ -1,6 +1,6 @@
--- ---
title: Repository Policies title: Repository Policies
last_modified: 2026-10-04 last_modified: 2026-10-15
--- ---
This document covers repository structure, tooling, and workflow standards. Code This document covers repository structure, tooling, and workflow standards. Code
@@ -54,34 +54,39 @@ style conventions are in separate documents:
`cibuild`. `script/bootstrap` installs all dependencies idempotently and `cibuild`. `script/bootstrap` installs all dependencies idempotently and
assumes nothing is present: base tools come from nix, apt, brew, or apk assumes nothing is present: base tools come from nix, apt, brew, or apk
(detected in that order; apt runs noninteractive). For node it uses the (detected in that order; apt runs noninteractive). For node it uses the
installed node if present; otherwise it installs a PINNED node version via installed node only when its major version is the pinned one; otherwise (node
nvm, first installing nvm itself if missing — from a hash-verified GitHub missing, or another major, such as the node 24 the CI runner image ships) it
release archive (never `curl | sh`), with bash installed as an explicit installs the PINNED node version via nvm and installs yarn and the packages
prerequisite since nvm requires bash. yarn is then pinned via under it. nvm itself is installed first if missing, from a hash-verified
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts"; GitHub release archive (never `curl | sh`), with bash installed as an explicit
always exact versions. `script/cibuild` runs the CI build: it changes to the prerequisite since nvm requires bash. Only the major version is compared
repo root, runs `script/bootstrap`, runs `script/check`, and builds the image because the `Dockerfile` stages start from a node image whose exact version is
with the version; the Gitea workflow calls it. **`script/cibuild` runs not the pin, and nvm cannot install a prebuilt node on alpine. yarn is pinned
via `corepack prepare yarn@<version> --activate`. Never install "latest" or
"lts"; always exact versions. `script/cibuild` runs the CI build: it changes
to the repo root, runs `script/bootstrap`, runs `script/check`, and builds the
image with the version; the Gitea workflow calls it. **`script/cibuild` runs
`script/bootstrap` first**, because the workflow checks out the repo and runs `script/bootstrap` first**, because the workflow checks out the repo and runs
nothing else, while `script/fmt-check` runs the formatter on the host: on a nothing else, while `script/fmt-check` runs the formatter on the host: on a
pristine checkout with nothing installed the run dies there, after the pristine checkout with nothing installed the run dies there, after the
containerised gates have passed. **The bootstrap alone is not enough**: containerised gates have passed. **The bootstrap alone is not enough**: when
`script/bootstrap` installs node and yarn under nvm and leaves neither on the `script/bootstrap` installs node and yarn under nvm it leaves neither on the
`PATH` of the shell that called it, so a bare `yarn` still exits 127. The host `PATH` of the shell that called it, so a bare `yarn` either exits 127 or runs
entrypoints that need yarn — `script/fmt` and `script/fmt-check` — therefore under another node. The host entrypoints that need yarn — `script/fmt` and
source nvm for the pinned node version before invoking it, exactly as `script/fmt-check` — therefore use the same test as `script/bootstrap`: when
`script/bootstrap`'s own install step does. A runner carrying nothing but the node on `PATH` has the pinned major version they run the `yarn` on `PATH`,
docker and git then gets through `script/check`. Four further scripts are our and otherwise they load nvm through `$HOME/.nvm/nvm.sh` and run yarn under the
own extensions to the standard: `script/check` runs `script/test`, pinned node. A runner carrying nothing but docker and git then gets through
`script/lint` and `script/fmt-check`; `script/precommit` is what the git `script/check`. Four further scripts are our own extensions to the standard:
pre-commit hook runs, and it calls `script/check`; `script/install-precommit` `script/check` runs `script/test`, `script/lint` and `script/fmt-check`;
installs the git pre-commit hook (the `make hooks` target shims to it); and `script/precommit` is what the git pre-commit hook runs, and it calls
`script/projectname` (literally that filename) simply outputs the project's `script/check`; `script/install-precommit` installs the git pre-commit hook
name. Scripts that need the name call `script/projectname` — e.g. (the `make hooks` target shims to it); and `script/projectname` (literally
`script/docker` assembles its image tag from it — so those scripts stay that filename) simply outputs the project's name. Scripts that need the name
byte-identical across all repos. Repo-type-specific pre-commit extras (e.g. call `script/projectname` — e.g. `script/docker` assembles its image tag from
`go mod tidy` verification in Go repos) belong in `script/precommit`, not in it — so those scripts stay byte-identical across all repos. Repo-type-specific
the hook itself. Model scripts are at pre-commit extras (e.g. `go mod tidy` verification in Go repos) belong in
`script/precommit`, not in the hook itself. Model scripts are at
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README `https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
must document the provided scripts in an **Entrypoints** section (see the must document the provided scripts in an **Entrypoints** section (see the
README requirements below). README requirements below).
@@ -118,8 +123,10 @@ style conventions are in separate documents:
and nothing else: and nothing else:
```sh ```sh
docker build --no-cache --target lint -t "$(script/projectname)-lint" . docker build --build-arg CACHEBUST="$(date +%s)-$$" \
docker build --no-cache --target test -t "$(script/projectname)-test" . --target lint --output type=cacheonly .
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--target test --output type=cacheonly .
``` ```
**A stage that is not the last one in the file is built only when the final **A stage that is not the last one in the file is built only when the final
@@ -129,10 +136,15 @@ style conventions are in separate documents:
plain `docker build .` builds the last stage alone and exits 0 having linted plain `docker build .` builds the last stage alone and exits 0 having linted
and tested nothing. and tested nothing.
**Every `docker build` in `script/` is tagged**, here and in **The gate builds write no image.** With `--output type=cacheonly` the phase
`script/cibuild` and `script/docker`. An untagged build leaves a dangling runs and a failing step fails the build, but the result is not exported.
image behind on every invocation, on every developer host and every CI Nothing uses those images, and writing one out is slow: a Go test phase's
runner; a tagged one replaces the previous image. image holds the toolchain. A build given neither `--output` nor `-t` writes
an untagged image and leaves it dangling, on every developer host and every
CI runner. `script/cibuild` and `script/docker` build the image that ships
and tag it, so each build replaces the previous image; each assigns the tag
on its own line before the build, so `set -e` stops it where
`script/projectname` fails.
Inside a phase the tool is invoked directly — `golangci-lint`, `go test`, Inside a phase the tool is invoked directly — `golangci-lint`, `go test`,
`eslint`, `prettier` — never through `make lint` or `script/test`, which are `eslint`, `prettier` — never through `make lint` or `script/test`, which are
@@ -147,22 +159,33 @@ style conventions are in separate documents:
one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs
exit non-zero with `parallel golangci-lint is running` — a status a caller exit non-zero with `parallel golangci-lint is running` — a status a caller
cannot tell from real findings. Both have produced wrong verdicts in this cannot tell from real findings. Both have produced wrong verdicts in this
org, in both directions. A container has its own cache, its own `TMPDIR` and org, in both directions. A container has its own golangci-lint cache, its
a digest-pinned binary, so neither is reachable. own `TMPDIR` and a digest-pinned binary, so neither is reachable.
- **Any build that runs checks is built with `--no-cache`.** Docker invalidates - **Any build that runs checks is given a new `CACHEBUST` value.** Docker
a `COPY` layer only when the copied content changes, so on an unchanged tree invalidates a `COPY` layer only when the copied content changes, so on an
the check `RUN` is served from cache, nothing executes, and the build still unchanged tree the check `RUN` is served from cache, nothing executes, and the
exits 0. Every `docker build` in `script/` therefore passes `--no-cache`: build still exits 0. Every `docker build` in `script/` therefore passes
`script/lint`, `script/test`, `script/cibuild` and `script/docker` are the `--build-arg CACHEBUST="$(date +%s)-$$"`, the time and the script's process
four, and there is no fifth — `script/check` runs the two gate phases and id, so no two builds share a value, not even the three in one
`script/fmt-check`, and builds no image of its own. A bare `docker build .` is `script/cibuild`. `script/lint`, `script/test`, `script/cibuild` and
not evidence that anything ran: a sub-second build reporting success is a `script/docker` are the four, and there is no fifth — `script/check` runs the
cache hit, not a result. Never invalidate by pruning — `docker builder prune` two gate phases and `script/fmt-check`, and builds no image of its own. Every
and friends destroy a build cache shared with every other build on the host. stage of the `Dockerfile` declares `ARG CACHEBUST` as its first line after
When a check is added or changed, prove it works by planting a defect it must `FROM`, and a new value reruns every `RUN` step after that line; `COPY` steps
catch and watching the run fail on it, then revert the defect. A green run are compared by content and need nothing. A stage without the `ARG` line is
alone shows neither that the check ran nor that it covers what it should. served from cache on an unchanged tree, so a repository re-vendors the scripts
and adds the `ARG` lines in one commit. The scripts do not pass `--no-cache`:
it reruns every step too, but it also empties every cache mount, and Go's
build cache, which the canonical Go `Dockerfile` below mounts so that compiled
packages survive between check builds, would start empty every time. A bare
`docker build .` is not evidence that anything ran: a sub-second build
reporting success is a cache hit, not a result. Never invalidate by pruning —
`docker builder prune` and friends destroy a build cache shared with every
other build on the host. When a check is added or changed, prove it works by
planting a defect it must catch and watching the run fail on it, then revert
the defect. A green run alone shows neither that the check ran nor that it
covers what it should.
- **The gate phases are separate stages, and the build stage depends on both.** - **The gate phases are separate stages, and the build stage depends on both.**
The lint phase is based on the `golangci/golangci-lint` image (pinned by The lint phase is based on the `golangci/golangci-lint` image (pinned by
@@ -171,32 +194,42 @@ style conventions are in separate documents:
`Dockerfile`: `Dockerfile`:
```dockerfile ```dockerfile
# Every stage declares ARG CACHEBUST first: script/ passes a new value
# on every build, which reruns every RUN step after it. The steps that
# compile mount Go's build cache, which survives from build to build.
# Lint phase # Lint phase
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD # golangci/golangci-lint:v2.x.x, YYYY-MM-DD
FROM golangci/golangci-lint@sha256:... AS lint FROM golangci/golangci-lint@sha256:... AS lint
ARG CACHEBUST
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
COPY . . COPY . .
RUN golangci-lint run --config .golangci.yml ./... RUN --mount=type=cache,target=/root/.cache/go-build \
golangci-lint run --config .golangci.yml ./...
# Test phase. -race needs cgo and so a C compiler, which the Debian Go # Test phase. -race needs cgo and so a C compiler, which the Debian Go
# image ships and the alpine one does not. # image ships and the alpine one does not. -count=1 because the mounted
# cache also holds test results, which go test would otherwise replay.
# golang:1.x, YYYY-MM-DD # golang:1.x, YYYY-MM-DD
FROM golang@sha256:... AS test FROM golang@sha256:... AS test
ARG CACHEBUST
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
COPY . . COPY . .
RUN go test -timeout 90s -race -cover ./... || \ RUN --mount=type=cache,target=/root/.cache/go-build \
go test -count=1 -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \ { echo "--- Rerunning with -v for details ---"; \
go test -timeout 90s -race -v ./...; exit 1; } go test -count=1 -timeout 90s -race -v ./...; exit 1; }
# Build stage. Nothing is wanted from either phase above; the copies # Build stage. Nothing is wanted from either phase above; the copies
# are what make BuildKit build them first, so this stage cannot run # are what make BuildKit build them first, so this stage cannot run
# unless lint and test passed. # unless lint and test passed.
# golang:1.x-alpine, YYYY-MM-DD # golang:1.x-alpine, YYYY-MM-DD
FROM golang@sha256:... AS builder FROM golang@sha256:... AS builder
ARG CACHEBUST
COPY --from=lint /src/go.sum /dev/null COPY --from=lint /src/go.sum /dev/null
COPY --from=test /src/go.sum /dev/null COPY --from=test /src/go.sum /dev/null
RUN apk add --no-cache git RUN apk add --no-cache git
@@ -212,7 +245,8 @@ style conventions are in separate documents:
# .git present, a version that is still empty, dev or unknown fails the # .git present, a version that is still empty, dev or unknown fails the
# build: git is missing or could not read the checkout. # build: git is missing or could not read the checkout.
ARG VERSION ARG VERSION
RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \ RUN --mount=type=cache,target=/root/.cache/go-build \
VERSION="${VERSION:-$(git describe --tags --always)}"; \
if [ -e .git ]; then \ if [ -e .git ]; then \
case "$VERSION" in ""|dev|unknown) \ case "$VERSION" in ""|dev|unknown) \
echo "version is '$VERSION' although .git is present" >&2; \ echo "version is '$VERSION' although .git is present" >&2; \
@@ -225,6 +259,7 @@ style conventions are in separate documents:
# Runtime stage, and the last one # Runtime stage, and the last one
FROM alpine@sha256:... FROM alpine@sha256:...
ARG CACHEBUST
COPY --from=builder /app /usr/local/bin/app COPY --from=builder /app /usr/local/bin/app
ENTRYPOINT ["app"] ENTRYPOINT ["app"]
``` ```
@@ -239,6 +274,16 @@ style conventions are in separate documents:
- Keep the runtime stage last, and if you add a stage after it, give it the - Keep the runtime stage last, and if you add a stage after it, give it the
same two copies. A plain `docker build .` builds the last stage's chain same two copies. A plain `docker build .` builds the last stage's chain
and nothing else. and nothing else.
- Only Go's build cache is mounted (`/root/.cache/go-build`, where Go keeps
it for root), and only on the steps that compile: `golangci-lint run`,
both `go test` runs and `go build`. It keeps compiled packages between
check builds, and Go reuses one only when all its inputs are unchanged, so
it saves the compile without skipping a check. Nothing else is kept:
golangci-lint's own cache would replay stored findings in place of a lint
run, and Go builds from whatever files a kept module cache holds without
checking them again (only `go mod verify` does), so a file changed there
would go unnoticed. Go's build cache also holds test results, which is why
both `go test` runs carry `-count=1`.
- If the project uses `//go:embed` directives that reference build artifacts - If the project uses `//go:embed` directives that reference build artifacts
(e.g. a web frontend compiled in a separate stage), the lint phase must (e.g. a web frontend compiled in a separate stage), the lint phase must
create placeholder files so the embed directives resolve. Example: create placeholder files so the embed directives resolve. Example:
@@ -278,20 +323,39 @@ style conventions are in separate documents:
`.git` and the version still comes out empty, `dev` or `unknown`. A plain `.git` and the version still comes out empty, `dev` or `unknown`. A plain
`docker build .` with no build arguments must succeed; a Dockerfile that `docker build .` with no build arguments must succeed; a Dockerfile that
refuses an empty build argument drops that refusal and keeps the argument. refuses an empty build argument drops that refusal and keeps the argument.
A checkout whose `.git` is a file (a linked worktree, or a repository
checked out as a submodule) is the exception: that file points to a git
directory outside the build context, so the build cannot read the version
and a plain `docker build .` fails; pass the version with
`--build-arg VERSION=...`, as `script/docker` and `script/cibuild` already
do.
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that - Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
runs `script/cibuild` on push, and checks out the repo as its only other step. runs `script/cibuild` on push, and checks out the repo as its only other step,
That script bootstraps, runs the gate phases, and then builds the image, so a with `persist-credentials: false`: `script/cibuild` needs no token, and
successful run means every check passed; a bare `docker build .` does not without it the checkout leaves the job's token in `.git/config` for every
later step. The checkout step also sets `fetch-depth: 0`, which fetches the
tags `git describe` needs: by default it clones shallow with no tags, and a
tagged repository's CI build would stamp a bare short commit id. The
workflow's `concurrency` block groups runs by workflow and branch
(`${{ github.workflow }}-${{ github.ref }}`) with `cancel-in-progress: true`,
so a new push cancels the older run on the same branch, queued or running, and
no other: runs for replaced commits do not hold up the shared runner.
`script/cibuild` bootstraps, runs the gate phases, and then builds the image,
so a successful run means every check passed; a bare `docker build .` does not
carry the same guarantee, because its gate phases may come from the cache. The carry the same guarantee, because its gate phases may come from the cache. The
image build is uncached and so runs the gate phases a second time. That is the image build gets a `CACHEBUST` value of its own and so runs the gate phases a
price of the rule above, and it is worth paying: the image that ships is built second time. That is the price of the rule above, and it is worth paying: the
from a run of its own gates rather than from a cache entry. A separate image that ships is built from a run of its own gates rather than from a cache
workflow limited to `main` by a `branches` list under `on: push` cannot be entry. The `check` job sets `timeout-minutes: 20`, so a hung build frees the
checked by review: to try a change to it, add the feature branch to that list shared runner after 20 minutes. That allows for the three Docker builds
and push, then remove the branch from the list again before merging. Keep any described above (the test phase, the lint phase, then the image), each held to
job in it that publishes behind `if: github.ref_name == 'main'`, so the run the 5-minute Docker build limit below, plus the bootstrap. A separate workflow
from the feature branch publishes nothing. limited to `main` by a `branches` list under `on: push` cannot be checked by
review: to try a change to it, add the feature branch to that list and push,
then remove the branch from the list again before merging. Keep any job in it
that publishes behind `if: github.ref_name == 'main'`, so the run from the
feature branch publishes nothing.
- Use platform-standard formatters: `black` for Python, `prettier` for - Use platform-standard formatters: `black` for Python, `prettier` for
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
@@ -351,12 +415,11 @@ style conventions are in separate documents:
That cache is Go's own, separate from Docker's layer cache. Go stores a That cache is Go's own, separate from Docker's layer cache. Go stores a
passing result in its cache directory (`GOCACHE`), and when the same tests passing result in its cache directory (`GOCACHE`), and when the same tests
run again on unchanged code it prints that result, marked `(cached)`, run again on unchanged code it prints that result, marked `(cached)`,
without running them. That matters on a developer's machine, where this without running them. The directory lasts from one run to the next on a
target runs and the directory lasts from one run to the next. The `test` developer's machine, where this target runs, and in the `test` phase of the
phase of the `Dockerfile` needs no `-count=1`: its base image holds no `Dockerfile`, which mounts it to keep compiled packages between check
result for this repo's tests and nothing before its `go test` step runs a builds, so the phase's `go test` runs carry `-count=1` too. A new
test, so there is nothing to replay. `--no-cache` (above) is what makes that `CACHEBUST` value (above) is what makes that step run on an unchanged tree.
step run on an unchanged tree.
Python example: Python example:
@@ -384,9 +447,12 @@ style conventions are in separate documents:
- `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`), - `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`),
editor files (`.swp`, `*~`), in-repo agent scratch directories (`.claude/`), editor files (`.swp`, `*~`), in-repo agent scratch directories (`.claude/`),
language build artifacts, and `node_modules/`. Fetch the standard `.gitignore` `node_modules/`, and the repo's own build outputs. Fetch the standard
from `https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when `.gitignore` from
setting up a new repo. These patterns are written to `.gitignore`'s own `https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when setting up
a new repo. A repo's `.gitignore` is the standard file followed by the repo's
own entries, such as its binaries; a re-vendor replaces the standard part and
keeps those entries. These patterns are written to `.gitignore`'s own
semantics, in which an unanchored pattern already matches at every depth; they semantics, in which an unanchored pattern already matches at every depth; they
are not a `.dockerignore` and must not be transplanted into one unmodified. are not a `.dockerignore` and must not be transplanted into one unmodified.
@@ -434,13 +500,15 @@ style conventions are in separate documents:
byte-identically across repos: byte-identically across repos:
```sh ```sh
# Own line: a failing command substitution inside an argument does not # The version and the tag each get their own line: a failing command
# trip `set -e`, so the inline form degrades to an empty constant. # substitution inside an argument does not trip `set -e`, so the inline
# form degrades to an empty constant.
version="$(git describe --tags --always --dirty 2>/dev/null || true)" version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
docker build --no-cache \ tag="$(script/projectname)"
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--build-arg VERSION="$version" \ --build-arg VERSION="$version" \
-t "$(script/projectname)" . -t "$tag" .
``` ```
`--always` makes an untagged repo yield an abbreviated commit hash rather `--always` makes an untagged repo yield an abbreviated commit hash rather
@@ -452,8 +520,8 @@ style conventions are in separate documents:
there because `ARG` is stage-scoped; passing `VERSION` to a repo whose there because `ARG` is stage-scoped; passing `VERSION` to a repo whose
Dockerfile declares no such `ARG` is ignored and costs nothing, which is why Dockerfile declares no such `ARG` is ignored and costs nothing, which is why
the scripts stay byte-identical. One consequence for CI: the standard the scripts stay byte-identical. One consequence for CI: the standard
checkout action clones shallow and fetches no tags, so a repo that embeds a checkout action clones shallow and fetches no tags, so the canonical
tag-derived version must set `fetch-depth: 0` on its checkout step. `.gitea/workflows/check.yml` sets `fetch-depth: 0` on its checkout step.
- **Verify `.dockerignore` by enumerating the image, not by reading the - **Verify `.dockerignore` by enumerating the image, not by reading the
patterns.** Plant files at the root _and_ at least two directories deep, build patterns.** Plant files at the root _and_ at least two directories deep, build
@@ -520,6 +588,11 @@ style conventions are in separate documents:
function defined and never invoked has the same exit status and the same function defined and never invoked has the same exit status and the same
empty output as one that worked. empty output as one that worked.
Node is the exception to the whole-token comparison: the canonical
`script/bootstrap` compares only its major version, because the node 22
alpine image the `Dockerfile` stages start from is not the exact pin, and
nvm cannot install a prebuilt node on alpine.
Keep it POSIX sh: no arrays, no `[[`, no `grep -P`. Keep it POSIX sh: no arrays, no `[[`, no `grep -P`.
A Go tool a repo needs on the host is installed with `go install` pinned to A Go tool a repo needs on the host is installed with `go install` pinned to
@@ -636,7 +709,11 @@ style conventions are in separate documents:
Never edit existing migrations after release. Never edit existing migrations after release.
- All repos should have an `.editorconfig` enforcing the project's indentation - All repos should have an `.editorconfig` enforcing the project's indentation
settings. settings: the standard file from
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.editorconfig`, which sets
tabs for `Makefile` and Go files, followed by the repo's own sections, such as
one for another language it uses. A re-vendor replaces the standard part and
keeps those sections.
- Avoid putting files in the repo root unless necessary. Root should contain - Avoid putting files in the repo root unless necessary. Root should contain
only project-level config files (`README.md`, `AGENTS.md`, `Makefile`, only project-level config files (`README.md`, `AGENTS.md`, `Makefile`,
+203 -2
View File
@@ -8,8 +8,7 @@
# Status # Status
pre-1.0. No git tags. Open work is tracked on the issue tracker, which is pre-1.0. Open work is tracked on the issue tracker, which is authoritative.
authoritative.
# Next Step # Next Step
@@ -18,6 +17,208 @@ https://git.eeqj.de/sneak/secret/milestone/12
# Completed Steps # Completed Steps
- 2026-10-15: The comment on `commandWait` in `internal/cli/integration_test.go`
no longer says `script/test` gives the package 30 seconds
(https://git.eeqj.de/sneak/secret/issues/195). `script/test` sets no time
limit; the test phase of the `Dockerfile` sets one with `go test -timeout`.
The comment now names that place instead of a number.
- 2026-10-15: The canonical files are re-vendored from `sneak/prompts` commit
`e3f79c4` (https://git.eeqj.de/sneak/secret/issues/129), so Go's build cache
survives between check builds and `make test` no longer compiles the standard
library and every dependency with `-race` on every run. `script/lint`,
`script/test`, `script/cibuild` and `script/docker` pass a new `CACHEBUST`
value instead of `--no-cache`, and every stage of the `Dockerfile` declares
`ARG CACHEBUST` first, so every `RUN` step still reruns on an unchanged tree.
The steps that compile mount Go's build cache, and both `go test` runs keep
`-count=1`, so no test result is reused. `script/lint` and `script/test` write
no image. The CI workflow cancels a branch's older run, stops after 20
minutes, fetches the tags and leaves no token behind. `script/bootstrap` uses
the installed node only when its major version is the pinned one.
- 2026-10-15: `make test` no longer fails on some runs in `internal/cli`
(https://git.eeqj.de/sneak/secret/issues/192). `cli.Entry()` set `time.Local`
to UTC and ended by wiping every memguard buffer, and a test ran it in the
same process as the other tests. A timer left by an earlier test could read
`time.Local` as it changed, which the race detector reported, and the wipe
could destroy a buffer still in use, the likeliest cause of a crash of the
test process on another run. `main()` in `cmd/secret` now sets `time.Local`
before it calls `cli.Entry()`, and that test runs `cli.Entry()` in a process
of its own.
- 2026-10-15: `secret mv` from one vault to another unlocks both vaults before
it takes the state directory lock
(https://git.eeqj.de/sneak/secret/issues/189). It unlocked them while holding
the lock, and unlocking can ask for a passphrase, so every other command that
changes anything waited at that prompt. `moveSecretCrossVault` now checks the
move, refusing an existing destination secret without `--force`, unlocks both
vaults, and only then takes the lock. Under the lock, it checks that the
destination vault still has the long-term key it unlocked, and
`copySecretAndRemoveSource` checks the move again before it copies. A move
within a vault takes the lock in `moveSecretWithinVault`, so `MoveSecret`
takes none itself. The error for a vault created again with another key, which
`secret unlocker add` shares, now ends "nothing was changed".
- 2026-10-15: `secret init`, `secret vault create`, `secret unlocker add`, and
`secret add`, `secret import` and `secret generate secret` with `--force` ask
for the mnemonic or passphrase, or unlock the vault, before they take the
state directory lock (https://git.eeqj.de/sneak/secret/issues/152). They did
so while holding the lock, so every other command that changes anything waited
at the prompt. Under the lock, `secret unlocker add` fails if the vault was
removed, or created again with another key, and refuses a GPG key added
meanwhile; a forced add of a secret created meanwhile lets go of the lock,
gets the key and takes the lock again. `secret.CreatePGPUnlocker` takes the
unlocked vault; `SetMnemonic` and `SetUnlockPassphrase` are gone.
- 2026-10-15: A `secret mv` to another vault that copies the secret but cannot
remove the source fails (https://git.eeqj.de/sneak/secret/issues/159). It
printed a warning and the usual "Moved" line and exited 0. If the source
secret's directory is still in place, the error says the copy holds all
versions and the source remains. If the source left its vault and only the
temporary directory it was moved into could not be deleted, the error says the
secret is moved and names that directory, which the error of `RemoveDirAtomic`
now does.
- 2026-10-15: `secret encrypt` and `secret decrypt` with `--output` hold the
result in memory and open the output file only once encryption or decryption
has succeeded (https://git.eeqj.de/sneak/secret/issues/143). They emptied that
file before reading any input, so a failed command left it empty, and
encrypting a file onto itself gave the encryption of nothing. A failure to
encrypt or decrypt now leaves `--output` untouched, and `--input` can name the
same file. A failure while writing `--output`, such as a full disk, can still
leave it empty or partly written; an error from closing it is reported.
Without `--output` the result still goes to stdout as it is produced.
- 2026-10-15: A command that uses the current vault fails when `currentvault` is
empty or names no vault (https://git.eeqj.de/sneak/secret/issues/156).
`secret list` reported an empty vault with exit status 0, and `secret add`
failed on a missing directory. `ResolveVaultSymlink` now refuses an empty or
invalid name, or one with no directory under `vaults.d`, with an error that
wraps `ErrVaultNotFound`, names the file and says to run
`secret vault select <name>`. `secret vault list` and `secret info` still run.
- 2026-10-15: `secret` prints every time in UTC, whatever the machine's time
zone (https://git.eeqj.de/sneak/secret/issues/169). `secret list`,
`secret version list`, `secret unlocker list` and `secret info` printed times
in the local time zone without naming it, so one vault showed different times
on two machines. `Entry()` now sets `time.Local` to UTC before anything else,
as policy 6 in `AGENTS.md` asks, so every time the program prints or writes is
in UTC. `--json` times end in `Z`, and a new version's name takes the UTC
date. The README says so.
- 2026-10-15: `secret vault`, `secret unlocker`, `secret version` and
`secret generate` fail when given a subcommand they do not have
(https://git.eeqj.de/sneak/secret/issues/183). They printed their help and
exited 0, so a mistyped command looked like one that worked. Each now exits 1
with `unknown command "<name>" for "secret <parent>"`, and with no arguments
still prints its help. Cobra checks a command's `Args` only when it has a
`RunE`, so each gets `Args: cobra.NoArgs` and a `RunE` that prints its help;
their help now shows a `secret <parent> [flags]` usage line.
`secret vault help` fails as an unknown subcommand, while `secret help vault`
and `secret vault --help` still work.
- 2026-10-15: With `SB_UNLOCK_PASSPHRASE` set, `secret unlocker add passphrase`
asks on the terminal for the new unlocker's passphrase and uses the variable
to unlock the vault (https://git.eeqj.de/sneak/secret/issues/154). It gave the
new unlocker the passphrase in the variable without asking. With no
controlling terminal, the new unlocker takes the variable's passphrase and the
output says so. `secret.ErrNoTerminal` is exported so that the command can
tell a missing terminal from other failures to read, such as nothing typed or
a confirmation that does not match, which still fail the command.
- 2026-10-15: `secret init` and `secret vault create` stop with an error naming
the file when a vault's `vault-metadata.json` cannot be read or parsed, and
leave the state directory as it was
(https://git.eeqj.de/sneak/secret/issues/140). `GetNextDerivationIndex` passed
over such a vault, so the next vault made from the same mnemonic could get its
derivation index and so the same long-term key. A vault directory with no
metadata file is still skipped.
- 2026-10-15: The README says that `secret add` drops one trailing newline from
stdin, that `secret import` stores a file's bytes unchanged, and that
`secret get` writes the value with no newline added
(https://git.eeqj.de/sneak/secret/issues/158). It said none of this, so a PEM
key piped into `secret add` came back one byte short with no hint why. A new
test stores `a\n\n` through `add` and through `import` and checks that `add`
kept two bytes and `import` three. No program code changes.
- 2026-10-15: When gpg fails to decrypt, to encrypt or to look up a key's
fingerprint, the error ends with what gpg wrote to stderr, trimmed
(https://git.eeqj.de/sneak/secret/issues/146). It gave only gpg's exit status,
so the user could not tell why. `cmd.Output` keeps gpg's stderr in the
`*exec.ExitError` it returns, and the three errors now use it. When gpg wrote
nothing to stderr the error is as before, and when gpg wrote several lines the
error has them all.
- 2026-10-15: `secret version list` no longer shows a version without
`NOT_AFTER` as expired (https://git.eeqj.de/sneak/secret/issues/150). STATUS
was `expired` for every version other than the current one, so after a promote
the newest version, which has no `NOT_AFTER`, read `expired`. STATUS is now
`current` for the current version, `expired` for another version whose
`NOT_AFTER` has passed, and `inactive` for any other version. The README
describes the three.
- 2026-10-15: `secret unlocker list --json` for a vault with no unlockers, and
`secret vault list --json` when `vaults.d` holds no vaults, print the empty
list as `[]` instead of `null` (https://git.eeqj.de/sneak/secret/issues/148).
Both lists were nil slices, which `encoding/json` writes as `null`; each is
now made as an empty slice before it is filled. `vault.ListVaults` therefore
returns an empty slice instead of nil for an empty `vaults.d`.
- 2026-10-15: `secret add`, `secret import` and `secret generate secret` fail
with an error, not memguard's panic, when a secret is too large for the
locked-memory limit (https://git.eeqj.de/sneak/secret/issues/145). The value
was held twice, so half the usual 8 MiB limit was enough to crash them.
`secret` now raises the limit to the hard limit at startup, and each of the
three commands refuses, before its first locked allocation, a limit that
leaves fewer than 8 pages for its keys and buffers. The value is read a page
at a time into a `memguard.Stream`, which keeps it encrypted, and copied once
into locked memory after its size is checked against 100 MiB and the limit. It
leaves the stream through `Next`, not `Read`, because `Read` re-encrypts a
leftover piece and memguard hangs if that runs out of locked memory.
- 2026-10-15: Total Size in `secret info` counts every file of each version: the
value, the metadata and the version's two keys
(https://git.eeqj.de/sneak/secret/issues/147). It added up each version's
`data.age` and `metadata.age`, but a version keeps its value in `value.age`,
so no secret's value was counted. A version directory that cannot be read adds
nothing and gets a warning naming it.
- 2026-10-15: `secret vault import` is gone, with its tests and its README entry
(https://git.eeqj.de/sneak/secret/issues/139). It took only a vault without a
long-term key, and `secret init` and `secret vault create` give every vault
one, so it always failed; a mnemonic enters through those two commands. The
help and the note of `secret generate mnemonic` named `secret import`, which
imports a file as a secret, and now name `secret init` and
`secret vault create`.
- 2026-10-15: `secret version promote --help` says promote rewrites the
`current` file, not a "current symlink"
(https://git.eeqj.de/sneak/secret/issues/153). `current` is a plain file
holding the version name, as the README already said, and the comment in
`PromoteVersion` now says so too.
- 2026-10-14: `secret init` and `secret vault create` refuse a vault that
already exists before they ask for the mnemonic or the unlocker passphrase
(https://git.eeqj.de/sneak/secret/issues/151). They asked first, and
`secret init` printed "Initialized secrets manager at" before refusing. The
check `vault.CreateVault` made before writing is now
`vault.CheckNewVaultName`, which both commands call before the first prompt
and `CreateVault` still calls. It validates the name too, so
`secret vault create` with an invalid name also fails before the prompts.
`secret init` prints the "Initialized" line only once the vault is created.
- 2026-10-14: `secret get` fails when it cannot write the value
(https://git.eeqj.de/sneak/secret/issues/155). `GetSecret` and
`GetSecretWithVersion` discarded the result of writing it to stdout, so a full
disk or a failing stdout left an empty or cut-short file and exit status 0.
The write error is now returned, so the command prints it and exits 1. A
stdout the shell closed before `secret` started still exits 0, because the Go
runtime opens `/dev/null` in its place and the write there succeeds.
- 2026-10-14: `secret list` and `secret list --json` give each secret's own
creation and last-update times (https://git.eeqj.de/sneak/secret/issues/142).
They gave the time of the listing for every secret, because
`Secret.LoadMetadata` set both to the current time. It now reads file
modification times, so listing still needs no passphrase. The last update is
when the secret's `current` file was written. The creation is when its oldest
version's `value.age` was written, not that version's directory, whose time
changes when the next add rewrites its `metadata.age`. A move to another vault
gives the copied files the times of the originals. `GetSecretObject` no longer
loads the times, so `secret get` keeps its error for a missing `current` file.
- 2026-10-14: `secret unlocker add pgp` without `--keyid` uses the `default-key`
in `gpg.conf` (https://git.eeqj.de/sneak/secret/issues/141).
`getDefaultGPGKey` took that value from `gpgconf --list-options gpg` as
printed, with a leading `"` and percent-escaped, so gpg rejected the key ID.
The new `parseDefaultGPGKey` drops the `"` and percent-decodes the rest with
`url.PathUnescape`, which leaves `+` alone. A value that does not decode is an
error, not a fallback to the first secret key.
- 2026-10-14: A wrong passphrase, from `SB_UNLOCK_PASSPHRASE` or typed at the
prompt, gives the error `wrong passphrase for unlocker <id>`
(https://git.eeqj.de/sneak/secret/issues/157). It used to be age's "no
identity matched any of the recipients", followed by advice to give the vault
a new unlocker from its mnemonic, which would have replaced an unlocker that
works. The error now wraps `secret.ErrWrongPassphrase` and carries no advice
to use the mnemonic. A `priv.age` that is not an age file still gets that
advice.
- 2026-10-14: The passphrase and mnemonic prompts ask on the controlling - 2026-10-14: The passphrase and mnemonic prompts ask on the controlling
terminal (`/dev/tty`) instead of stdin, so `echo v | secret add s1 --force`, terminal (`/dev/tty`) instead of stdin, so `echo v | secret add s1 --force`,
`cat file | secret encrypt key` and `cat file.age | secret decrypt key` ask `cat file | secret encrypt key` and `cat file.age | secret decrypt key` ask
+6
View File
@@ -3,10 +3,16 @@ package main
import ( import (
"os" "os"
"time"
"sneak.berlin/go/secret/internal/cli" "sneak.berlin/go/secret/internal/cli"
) )
func main() { func main() {
// Every time secret prints or writes is in UTC, whatever TZ or the
// system's time zone is set to. It is set here, before cli.Entry starts
// any goroutine that could read time.Local while it changes.
time.Local = time.UTC //nolint:gosmopolitan // this makes local time UTC
os.Exit(cli.Entry()) os.Exit(cli.Entry())
} }
+61 -3
View File
@@ -141,6 +141,25 @@ func TestVaultCreationLeavesNoSecretInEnvironment(t *testing.T) {
run("vault", "create", "work") run("vault", "create", "work")
} }
// TestNoVaultImportCommand is a regression test for
// https://git.eeqj.de/sneak/secret/issues/139: `secret vault import`
// refused every vault that `secret vault create` makes, because each has a
// long-term key. The command is gone, so it fails as any subcommand of
// `secret vault` that does not exist does.
func TestNoVaultImportCommand(t *testing.T) {
t.Setenv(secret.EnvStateDir, t.TempDir())
t.Setenv(secret.EnvMnemonic, testMnemonic)
t.Setenv(secret.EnvUnlockPassphrase, testPassphrase)
_, err := cli.ExecuteCommandInProcess(
strings.Fields("vault create work"), "", nil)
require.NoError(t, err)
_, err = cli.ExecuteCommandInProcess(
strings.Fields("vault import work"), "", nil)
require.EqualError(t, err, `unknown command "import" for "secret vault"`)
}
// TestStopAtPassphrasePromptLeavesNothing is a regression test for the // TestStopAtPassphrasePromptLeavesNothing is a regression test for the
// review of https://git.eeqj.de/sneak/secret/pulls/82: `secret init` or // review of https://git.eeqj.de/sneak/secret/pulls/82: `secret init` or
// `secret vault create` stopped at the passphrase prompt left a vault with // `secret vault create` stopped at the passphrase prompt left a vault with
@@ -222,12 +241,51 @@ func TestMnemonicNotReadNamesOnlyMnemonic(t *testing.T) {
output, err := cmd.CombinedOutput() output, err := cmd.CombinedOutput()
require.Error(t, err) require.Error(t, err)
require.Equal(t, "Initialized secrets manager at: "+stateDir+"\n"+ require.Equal(t, "Error: failed to read mnemonic: there is no terminal to "+
"Error: failed to read mnemonic: there is no terminal to ask on. "+ "ask on. Please set the SB_SECRET_MNEMONIC environment variable or run "+
"Please set the SB_SECRET_MNEMONIC environment variable or run "+
"interactively\n", string(output)) "interactively\n", string(output))
} }
// TestCreateExistingVaultAsksForNothing is a regression test for
// https://git.eeqj.de/sneak/secret/issues/151: `secret init` and `secret
// vault create` asked for the mnemonic and the unlocker passphrase before
// refusing a vault that already exists, and `secret init` printed
// "Initialized secrets manager at" first. Each runs here as the built binary,
// in a new session, which has no terminal, and with neither
// SB_SECRET_MNEMONIC nor SB_UNLOCK_PASSPHRASE set, so asking for either
// fails. Each must fail saying only that the vault exists.
func TestCreateExistingVaultAsksForNothing(t *testing.T) {
t.Parallel()
stateDir := t.TempDir()
_, err := vault.CreateVault(afero.NewOsFs(), stateDir, "default", nil, nil)
require.NoError(t, err)
// run runs the built binary with args, which must fail, and returns what
// it printed
run := func(args ...string) string {
//nolint:gosec // G204: test executes the freshly built secret binary
cmd := exec.CommandContext(t.Context(), secretBinaryPath(t), args...)
cmd.Env = []string{
secret.EnvStateDir + "=" + stateDir,
"PATH=" + os.Getenv("PATH"),
"HOME=" + os.Getenv("HOME"),
}
cmd.SysProcAttr = &syscall.SysProcAttr{Setsid: true}
output, err := cmd.CombinedOutput()
require.Error(t, err, "secret %v", args)
return string(output)
}
require.Equal(t,
"Error: failed to create default vault: vault default already exists\n",
run("init"))
require.Equal(t, "Error: vault default already exists\n",
run("vault", "create", "default"))
}
// TestStopDuringCreateLeavesWholeVaultOrNone is a regression test for // TestStopDuringCreateLeavesWholeVaultOrNone is a regression test for
// https://git.eeqj.de/sneak/secret/issues/105: `secret init` or `secret vault // https://git.eeqj.de/sneak/secret/issues/105: `secret init` or `secret vault
// create` killed after the passphrase prompt but before the unlocker was // create` killed after the passphrase prompt but before the unlocker was
+50 -19
View File
@@ -1,6 +1,7 @@
package cli package cli
import ( import (
"bytes"
"errors" "errors"
"fmt" "fmt"
"io" "io"
@@ -107,8 +108,8 @@ func (cli *Instance) storeNewEncryptionKey(
// does not exist. The caller must destroy the returned buffer. Only storing // does not exist. The caller must destroy the returned buffer. Only storing
// a new key takes the state directory lock, so that reading an existing key // a new key takes the state directory lock, so that reading an existing key
// works on a read-only state directory and keeps no other command waiting // works on a read-only state directory and keeps no other command waiting
// at the passphrase prompt, and Encrypt streams its input and output // at the passphrase prompt, and Encrypt reads its input and writes its
// unlocked. // output without the lock.
func (cli *Instance) resolveEncryptionKey( func (cli *Instance) resolveEncryptionKey(
vlt *vault.Vault, secretName string, vlt *vault.Vault, secretName string,
) (*memguard.LockedBuffer, error) { ) (*memguard.LockedBuffer, error) {
@@ -188,17 +189,17 @@ func (cli *Instance) Encrypt(secretName, inputFile, outputFile string) error {
input = file input = file
} }
// Set up output writer // Set up output writer. With --output the result is held in memory
// and written only once encryption has succeeded, so a failure to
// encrypt leaves --output untouched and --input can be the same file.
// A failure while writing --output, such as a full disk, can leave it
// empty or partly written.
output := cli.cmd.OutOrStdout() output := cli.cmd.OutOrStdout()
if outputFile != "" { var result bytes.Buffer
file, err := cli.fs.Create(outputFile)
if err != nil {
return fmt.Errorf("failed to create output file: %w", err)
}
defer func() { _ = file.Close() }()
output = file if outputFile != "" {
output = &result
} }
// Encrypt the data // Encrypt the data
@@ -217,7 +218,11 @@ func (cli *Instance) Encrypt(secretName, inputFile, outputFile string) error {
return fmt.Errorf("failed to finalize encryption: %w", err) return fmt.Errorf("failed to finalize encryption: %w", err)
} }
return nil if outputFile == "" {
return nil
}
return cli.writeOutputFile(outputFile, result.Bytes())
} }
// Decrypt decrypts data using an age secret key stored in a secret // Decrypt decrypts data using an age secret key stored in a secret
@@ -278,17 +283,17 @@ func (cli *Instance) Decrypt(secretName, inputFile, outputFile string) error {
input = file input = file
} }
// Set up output writer // Set up output writer. With --output the result is held in memory
// and written only once decryption has succeeded, so a failure to
// decrypt leaves --output untouched and --input can be the same file.
// A failure while writing --output, such as a full disk, can leave it
// empty or partly written.
output := cli.cmd.OutOrStdout() output := cli.cmd.OutOrStdout()
if outputFile != "" { var result bytes.Buffer
file, err := cli.fs.Create(outputFile)
if err != nil {
return fmt.Errorf("failed to create output file: %w", err)
}
defer func() { _ = file.Close() }()
output = file if outputFile != "" {
output = &result
} }
// Decrypt the data // Decrypt the data
@@ -302,6 +307,32 @@ func (cli *Instance) Decrypt(secretName, inputFile, outputFile string) error {
return fmt.Errorf("failed to decrypt data: %w", err) return fmt.Errorf("failed to decrypt data: %w", err)
} }
if outputFile == "" {
return nil
}
return cli.writeOutputFile(outputFile, result.Bytes())
}
// writeOutputFile creates outputFile, or empties it if it exists, and
// writes result into it.
func (cli *Instance) writeOutputFile(outputFile string, result []byte) error {
file, err := cli.fs.Create(outputFile)
if err != nil {
return fmt.Errorf("failed to create output file: %w", err)
}
_, err = file.Write(result)
closeErr := file.Close()
if err == nil {
err = closeErr
}
if err != nil {
return fmt.Errorf("failed to write output file: %w", err)
}
return nil return nil
} }
+185
View File
@@ -0,0 +1,185 @@
//nolint:testpackage // sets the unexported fields of Instance
package cli
import (
"errors"
"fmt"
"io"
"os"
"path/filepath"
"testing"
"github.com/spf13/afero"
"github.com/spf13/cobra"
"github.com/stretchr/testify/require"
"sneak.berlin/go/secret/internal/vault"
)
// plainText is the content of plain.txt in the directory newCryptoTest makes.
const plainText = "document"
// newCryptoTest returns an Instance whose current vault, in a new state
// directory on the real filesystem, holds the age key secret "key", and a
// new directory holding only plain.txt and plain.txt.age, its encryption
// with that key.
func newCryptoTest(t *testing.T) (*Instance, string) {
t.Helper()
fs := afero.NewOsFs()
stateDir := t.TempDir()
_, err := vault.CreateVault(fs, stateDir, "default", testMnemonicBuffer(t), nil)
require.NoError(t, err)
cli := NewCLIInstanceWithStateDir(fs, stateDir)
cli.Mnemonic = testMnemonicBuffer(t)
cli.cmd = &cobra.Command{}
dir := t.TempDir()
plain := filepath.Join(dir, "plain.txt")
require.NoError(t, afero.WriteFile(fs, plain, []byte(plainText), 0o600))
require.NoError(t, cli.Encrypt("key", plain, plain+".age"))
return cli, dir
}
// TestFailedCryptoLeavesOutputUnchanged is a regression test for
// https://git.eeqj.de/sneak/secret/issues/143, where secret encrypt and
// secret decrypt emptied the file named by --output before they failed.
// A failure to encrypt or decrypt must leave an existing file as it was and
// create none, and give the same error as without --output.
func TestFailedCryptoLeavesOutputUnchanged(t *testing.T) {
t.Parallel()
tests := []struct {
name string
run func(cli *Instance, dir, output string) error
}{
// Reading a directory fails once encryption has begun
{"encrypt", func(cli *Instance, dir, output string) error {
return cli.Encrypt("key", dir, output)
}},
// plain.txt is not an age file
{"decrypt", func(cli *Instance, dir, output string) error {
return cli.Decrypt("key", filepath.Join(dir, "plain.txt"), output)
}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
cli, dir := newCryptoTest(t)
existing := filepath.Join(dir, "existing.txt")
require.NoError(t, afero.WriteFile(cli.fs, existing, []byte("precious"), 0o600))
cli.cmd.SetOut(io.Discard)
toStdout := tt.run(cli, dir, "")
require.Error(t, toStdout)
require.EqualError(t, tt.run(cli, dir, existing), toStdout.Error())
require.EqualError(t, tt.run(cli, dir, filepath.Join(dir, "new.txt")),
toStdout.Error())
content, err := afero.ReadFile(cli.fs, existing)
require.NoError(t, err)
require.Equal(t, "precious", string(content))
exists, err := afero.Exists(cli.fs, filepath.Join(dir, "new.txt"))
require.NoError(t, err)
require.False(t, exists, "a failed %s created --output", tt.name)
})
}
}
// TestCryptoInPlace checks that secret encrypt and secret decrypt given
// one file as both --input and --output replace it with the right result.
// Before https://git.eeqj.de/sneak/secret/issues/143 they emptied the file
// before reading it.
func TestCryptoInPlace(t *testing.T) {
t.Parallel()
cli, dir := newCryptoTest(t)
file := filepath.Join(dir, "plain.txt")
require.NoError(t, cli.Encrypt("key", file, file))
require.NoError(t, cli.Decrypt("key", file, file))
content, err := afero.ReadFile(cli.fs, file)
require.NoError(t, err)
require.Equal(t, plainText, string(content))
}
// errCloseFailed is returned by the files closeFailFs creates in place of
// a successful Close, as NFS reports a full disk only when the file is
// closed.
var errCloseFailed = errors.New("no space left on device")
// closeFailFs creates files whose Close fails with errCloseFailed.
type closeFailFs struct {
afero.Fs
}
// closeFailFile is a file created through closeFailFs.
type closeFailFile struct {
afero.File
}
func (f closeFailFile) Close() error {
_ = f.File.Close()
return errCloseFailed
}
//nolint:ireturn // afero.File is the interface required by afero.Fs
func (f closeFailFs) Create(name string) (afero.File, error) {
file, err := f.Fs.Create(name)
if err != nil {
return nil, err
}
return closeFailFile{File: file}, nil
}
// TestDecryptFailsWhenClosingOutputFails checks that secret decrypt fails
// when closing --output fails, since some filesystems report a failed
// write only then.
func TestDecryptFailsWhenClosingOutputFails(t *testing.T) {
t.Parallel()
cli, dir := newCryptoTest(t)
cli.fs = closeFailFs{Fs: cli.fs}
err := cli.Decrypt("key", filepath.Join(dir, "plain.txt.age"),
filepath.Join(dir, "out.txt"))
require.ErrorIs(t, err, errCloseFailed)
}
// TestDecryptToDescriptorOfDeletedFile checks that secret decrypt given
// /dev/fd/N as --output writes through that descriptor. A program may give
// the command a temporary file it has already deleted as stdout, pass
// /dev/stdout as --output and read the result back from that file.
func TestDecryptToDescriptorOfDeletedFile(t *testing.T) {
t.Parallel()
cli, dir := newCryptoTest(t)
file, err := os.CreateTemp(t.TempDir(), "out")
require.NoError(t, err)
defer func() { _ = file.Close() }()
require.NoError(t, os.Remove(file.Name()))
output := fmt.Sprintf("/dev/fd/%d", file.Fd())
require.NoError(t, cli.Decrypt("key", filepath.Join(dir, "plain.txt.age"), output))
// On macOS the command's open of /dev/fd/N shares this descriptor's
// position, which its write leaves at the end.
_, err = file.Seek(0, io.SeekStart)
require.NoError(t, err)
content, err := io.ReadAll(file)
require.NoError(t, err)
require.Equal(t, plainText, string(content))
}
+60
View File
@@ -0,0 +1,60 @@
package cli_test
import (
"bytes"
"path/filepath"
"strconv"
"testing"
"github.com/spf13/afero"
"github.com/spf13/cobra"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/secret/internal/cli"
"sneak.berlin/go/secret/internal/secret"
"sneak.berlin/go/secret/internal/vault"
)
// TestListFailsWhenCurrentVaultDoesNotExist is a regression test for
// https://git.eeqj.de/sneak/secret/issues/156, where a currentvault file
// that was empty, held an invalid vault name, or named a vault that does
// not exist, made `secret list` report an empty vault. Listing must fail
// with an error that names the file and says to run `secret vault select`,
// `secret vault list` must still list the vaults, and selecting a vault
// must make listing work again. The name "." would make vaults.d itself the
// vault, and "../vaults.d/work" would reach the vault work by a path.
func TestListFailsWhenCurrentVaultDoesNotExist(t *testing.T) {
t.Parallel()
for _, name := range []string{"", ".", "../vaults.d/work", "gone"} {
t.Run(strconv.Quote(name), func(t *testing.T) {
t.Parallel()
fs := newTwoVaultFs(t)
currentVaultPath := filepath.Join(testStateDir, "currentvault")
require.NoError(t, afero.WriteFile(fs, currentVaultPath,
[]byte(name), secret.FilePerms))
c := cli.NewCLIInstanceWithStateDir(fs, testStateDir)
var out bytes.Buffer
cmd := &cobra.Command{}
cmd.SetOut(&out)
err := c.ListSecrets(cmd, false, true, "")
require.ErrorIs(t, err, vault.ErrVaultNotFound)
require.ErrorContains(t, err, currentVaultPath)
require.ErrorContains(t, err, "secret vault select")
assert.Empty(t, out.String())
require.NoError(t, c.ListVaults(cmd, false))
assert.Equal(t, "Available vaults:\n default\n work\n", out.String())
require.NoError(t, c.SelectVault(cmd, "default"))
out.Reset()
require.NoError(t, c.ListSecrets(cmd, false, true, ""))
assert.Equal(t, "x\n", out.String())
})
}
}
+52 -13
View File
@@ -3,6 +3,7 @@ package cli_test
import ( import (
"bufio" "bufio"
"context" "context"
"fmt"
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
@@ -16,29 +17,67 @@ import (
"sneak.berlin/go/secret/internal/secret" "sneak.berlin/go/secret/internal/secret"
) )
// Entry must return its exit code rather than exit, so that its deferred const (
// memguard purge runs on the success and the error path alike. // envRunEntry, when set, makes this test binary run cli.Entry with its
// // own arguments and exit, instead of running the tests.
//nolint:paralleltest // sets os.Args, and Entry wipes every buffer in the process envRunEntry = "SECRET_TEST_RUN_ENTRY"
func TestEntryWipesBuffersAndReturnsExitCode(t *testing.T) {
savedArgs := os.Args
t.Cleanup(func() { os.Args = savedArgs }) // bufferWiped is what the test binary prints, run with envRunEntry set,
// when cli.Entry wiped the buffer made before it ran.
bufferWiped = "Entry wiped the buffer made before it ran"
)
// With envRunEntry set, the test binary makes a buffer, runs cli.Entry,
// says whether the buffer is wiped, and exits with Entry's exit code. This
// is done in init, not in a test, so that TestMain does not run in that
// process: it would build the secret binary again, and remove it at the end,
// while the other tests run it.
//
//nolint:gochecknoinits // must run before TestMain, as above
func init() {
if os.Getenv(envRunEntry) == "" {
return
}
buf := memguard.NewBufferFromBytes([]byte("key material"))
code := cli.Entry()
if !buf.IsAlive() {
fmt.Fprintln(os.Stderr, bufferWiped)
}
os.Exit(code)
}
// Entry must return its exit code rather than exit, so that its deferred
// memguard purge runs on the success and the error path alike. Entry wipes
// every memguard buffer in the process, including those other tests are
// using, so it runs in a process of its own: this test binary, run again
// with envRunEntry set.
func TestEntryWipesBuffersAndReturnsExitCode(t *testing.T) {
t.Parallel()
tests := []struct { tests := []struct {
args []string args []string
exitCode int exitCode int
}{ }{
{args: []string{"secret", "--help"}, exitCode: 0}, {args: []string{"--help"}, exitCode: 0},
{args: []string{"secret", "no-such-command"}, exitCode: 1}, {args: []string{"no-such-command"}, exitCode: 1},
} }
for _, tt := range tests { for _, tt := range tests {
buf := memguard.NewBufferFromBytes([]byte("key material")) //nolint:gosec // G204: test runs its own test binary
os.Args = tt.args cmd := exec.CommandContext(t.Context(), os.Args[0], tt.args...)
assert.Equal(t, tt.exitCode, cli.Entry(), "exit code for %v", tt.args) cmd.Env = append(os.Environ(), envRunEntry+"=1")
assert.False(t, buf.IsAlive(), "Entry left a buffer unwiped for %v", tt.args)
output, err := cmd.CombinedOutput()
require.NotNil(t, cmd.ProcessState, "%v did not run: %v", tt.args, err)
assert.Equal(t, tt.exitCode, cmd.ProcessState.ExitCode(),
"exit code for %v: %s", tt.args, output)
assert.Contains(t, string(output), bufferWiped,
"Entry left a buffer unwiped for %v", tt.args)
} }
} }
+28 -14
View File
@@ -29,6 +29,12 @@ func newGenerateCmd() *cobra.Command {
Use: "generate", Use: "generate",
Short: "Generate random data", Short: "Generate random data",
Long: `Generate various types of random data including mnemonics and secrets.`, Long: `Generate various types of random data including mnemonics and secrets.`,
// Cobra checks Args only on a command that has a RunE; without
// one, an unknown subcommand prints this help and exits 0.
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
return cmd.Help()
},
} }
cmd.AddCommand(newGenerateMnemonicCmd()) cmd.AddCommand(newGenerateMnemonicCmd())
@@ -43,7 +49,7 @@ func newGenerateMnemonicCmd() *cobra.Command {
Short: "Generate a random BIP39 mnemonic phrase", Short: "Generate a random BIP39 mnemonic phrase",
Long: `Generate a cryptographically secure random BIP39 ` + Long: `Generate a cryptographically secure random BIP39 ` +
`mnemonic phrase that can be used with 'secret init' ` + `mnemonic phrase that can be used with 'secret init' ` +
`or 'secret import'.`, `or 'secret vault create'.`,
RunE: func(cmd *cobra.Command, _ []string) error { RunE: func(cmd *cobra.Command, _ []string) error {
cli, err := NewCLIInstance() cli, err := NewCLIInstance()
if err != nil { if err != nil {
@@ -72,6 +78,12 @@ func newGenerateSecretCmd() *cobra.Command {
return fmt.Errorf("failed to initialize CLI: %w", err) return fmt.Errorf("failed to initialize CLI: %w", err)
} }
// Before readSecretEnv, which makes the first locked allocation
err = checkLockedMemoryLimit(0)
if err != nil {
return err
}
destroySecrets := cli.readSecretEnv() destroySecrets := cli.readSecretEnv()
defer destroySecrets() defer destroySecrets()
@@ -115,8 +127,10 @@ func (cli *Instance) GenerateMnemonic(cmd *cobra.Command) error {
" • If you lose this phrase, your secrets cannot be recovered") " • If you lose this phrase, your secrets cannot be recovered")
fmt.Fprintln(os.Stderr, "") fmt.Fprintln(os.Stderr, "")
fmt.Fprintln(os.Stderr, "Use this mnemonic with:") fmt.Fprintln(os.Stderr, "Use this mnemonic with:")
fmt.Fprintln(os.Stderr, " secret init (to initialize a new secret manager)") fmt.Fprintln(os.Stderr,
fmt.Fprintln(os.Stderr, " secret import (to import into an existing vault)") " secret init (to initialize a new secret manager)")
fmt.Fprintln(os.Stderr,
" secret vault create <name> (to create another vault)")
return nil return nil
} }
@@ -133,10 +147,16 @@ func (cli *Instance) GenerateSecret(
return errLengthTooSmall return errLengthTooSmall
} }
var ( if length > maxSecretSize {
secretValue string return errSecretTooLarge
err error }
)
err := checkLockedMemoryLimit(length)
if err != nil {
return err
}
var secretValue string
switch secretType { switch secretType {
case "base58": case "base58":
@@ -155,12 +175,6 @@ func (cli *Instance) GenerateSecret(
return fmt.Errorf("failed to generate random secret: %w", err) return fmt.Errorf("failed to generate random secret: %w", err)
} }
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
// Store the secret in the vault // Store the secret in the vault
vlt, err := vault.GetCurrentVault(cli.fs, cli.stateDir) vlt, err := vault.GetCurrentVault(cli.fs, cli.stateDir)
if err != nil { if err != nil {
@@ -173,7 +187,7 @@ func (cli *Instance) GenerateSecret(
secretBuffer := memguard.NewBufferFromBytes([]byte(secretValue)) secretBuffer := memguard.NewBufferFromBytes([]byte(secretValue))
defer secretBuffer.Destroy() defer secretBuffer.Destroy()
err = vlt.AddSecret(secretName, secretBuffer, force) err = cli.storeSecret(vlt, secretName, secretBuffer, force)
if err != nil { if err != nil {
return err return err
} }
+12 -13
View File
@@ -18,24 +18,23 @@ type vaultStats struct {
// addVersion accumulates size and timestamp info for one version directory // addVersion accumulates size and timestamp info for one version directory
func (s *vaultStats) addVersion(fs afero.Fs, versionPath string) { func (s *vaultStats) addVersion(fs afero.Fs, versionPath string) {
// Add size of encrypted data // Add the size of every file in the version directory: its value,
dataPath := filepath.Join(versionPath, "data.age") // metadata and keys. A directory that cannot be read adds nothing and
// gets a warning; its timestamp is still tracked below.
stat, err := fs.Stat(dataPath) files, err := afero.ReadDir(fs, versionPath)
if err == nil { if err != nil {
s.totalSize += stat.Size() secret.Warn("Could not read version directory",
"directory", versionPath, "error", err)
} }
// Add size of metadata for _, file := range files {
metaPath := filepath.Join(versionPath, "metadata.age") if !file.IsDir() {
s.totalSize += file.Size()
stat, err = fs.Stat(metaPath) }
if err == nil {
s.totalSize += stat.Size()
} }
// Track timestamps // Track timestamps
stat, err = fs.Stat(versionPath) stat, err := fs.Stat(versionPath)
if err == nil { if err == nil {
modTime := stat.ModTime() modTime := stat.ModTime()
if s.oldestTime.IsZero() || modTime.Before(s.oldestTime) { if s.oldestTime.IsZero() || modTime.Before(s.oldestTime) {
+51
View File
@@ -0,0 +1,51 @@
//nolint:testpackage // white-box test of unexported internals
package cli
import (
"io"
"os"
"path/filepath"
"testing"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestAddVersionWarnsWhenVersionDirUnreadable asserts that a version
// directory that cannot be listed, so that its files are left out of the
// total size, gets a warning naming it and the error, and that its
// timestamp is still tracked.
//
//nolint:paralleltest // replaces os.Stderr, where secret.Warn writes
func TestAddVersionWarnsWhenVersionDirUnreadable(t *testing.T) {
versionDir := filepath.Join(testVaultDir(listTestVaultName),
unreadableTestSecretsDirName, unreadableTestSecretName,
"versions", "20260809.001")
base := afero.NewMemMapFs()
require.NoError(t, base.MkdirAll(versionDir, listTestDirPerm))
stderrReader, stderrWriter, err := os.Pipe()
require.NoError(t, err)
defer func() { _ = stderrReader.Close() }()
realStderr := os.Stderr
os.Stderr = stderrWriter
var stats vaultStats
stats.addVersion(&openFailFs{Fs: base, path: versionDir}, versionDir)
os.Stderr = realStderr
require.NoError(t, stderrWriter.Close())
warning, err := io.ReadAll(stderrReader)
require.NoError(t, err)
assert.Contains(t, string(warning), versionDir)
assert.Contains(t, string(warning), errOpenFailed.Error())
assert.False(t, stats.latestTime.IsZero(),
"the timestamp must still be tracked")
}
+64
View File
@@ -0,0 +1,64 @@
package cli_test
import (
"bytes"
"crypto/rand"
"encoding/json"
"testing"
"github.com/awnumar/memguard"
"github.com/spf13/afero"
"github.com/spf13/cobra"
"github.com/stretchr/testify/require"
"sneak.berlin/go/secret/internal/cli"
"sneak.berlin/go/secret/internal/vault"
)
// TestInfoTotalSizeCountsSecretValues is a regression test for
// https://git.eeqj.de/sneak/secret/issues/147, where the total size shown by
// `secret info` left out every secret's value. Adding a secret must make it
// grow by at least the size of the value.
func TestInfoTotalSizeCountsSecretValues(t *testing.T) {
t.Parallel()
const valueSize = 10000
fs := afero.NewMemMapFs()
vlt, err := vault.CreateVault(fs, testStateDir, "default",
testMnemonicBuffer(t), nil)
require.NoError(t, err)
before := infoTotalSize(t, fs)
valueBytes := make([]byte, valueSize)
_, err = rand.Read(valueBytes)
require.NoError(t, err)
value := memguard.NewBufferFromBytes(valueBytes)
defer value.Destroy()
require.NoError(t, vlt.AddSecret("s1", value, false))
after := infoTotalSize(t, fs)
require.GreaterOrEqual(t, after-before, int64(valueSize))
}
// infoTotalSize returns the total size that `secret info --json` reports for
// the state directory on fs.
func infoTotalSize(t *testing.T, fs afero.Fs) int64 {
t.Helper()
var out bytes.Buffer
cmd := &cobra.Command{}
cmd.SetOut(&out)
err := cli.NewCLIInstanceWithStateDir(fs, testStateDir).Info(cmd, true)
require.NoError(t, err)
var info cli.InfoOutput
require.NoError(t, json.Unmarshal(out.Bytes(), &info))
return info.TotalSize
}
+40 -28
View File
@@ -17,6 +17,9 @@ import (
// errPassphraseMismatch is returned when passphrase confirmation fails // errPassphraseMismatch is returned when passphrase confirmation fails
var errPassphraseMismatch = errors.New("passphrases do not match") var errPassphraseMismatch = errors.New("passphrases do not match")
// defaultVaultName is the name of the vault `secret init` creates
const defaultVaultName = "default"
// NewInitCmd creates the init command // NewInitCmd creates the init command
func NewInitCmd() *cobra.Command { func NewInitCmd() *cobra.Command {
return &cobra.Command{ return &cobra.Command{
@@ -64,36 +67,17 @@ func (cli *Instance) promptMnemonic() (*memguard.LockedBuffer, func(), error) {
return mnemonicBuffer, mnemonicBuffer.Destroy, nil return mnemonicBuffer, mnemonicBuffer.Destroy, nil
} }
// Init initializes the secret manager, holding the state directory lock // Init initializes the secret manager. It asks for the mnemonic and the
// while initialize runs // passphrase before it takes the state directory lock, so that no other
// command waits while the user answers, and holds the lock while
// initialize runs.
func (cli *Instance) Init(cmd *cobra.Command) error { func (cli *Instance) Init(cmd *cobra.Command) error {
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
return cli.initialize(cmd)
}
// initialize creates the state directory, the default vault and its first
// unlocker
func (cli *Instance) initialize(cmd *cobra.Command) error {
secret.Debug("Starting secret manager initialization") secret.Debug("Starting secret manager initialization")
// Create state directory // Refuse an existing default vault before asking for anything
stateDir := cli.GetStateDir() err := vault.CheckNewVaultName(cli.fs, cli.stateDir, defaultVaultName)
secret.DebugWith("Creating state directory", slog.String("path", stateDir))
err := cli.fs.MkdirAll(stateDir, secret.DirPerms)
if err != nil { if err != nil {
secret.Debug("Failed to create state directory", "error", err) return fmt.Errorf("failed to create default vault: %w", err)
return fmt.Errorf("failed to create state directory: %w", err)
}
if cmd != nil {
cmd.Printf("Initialized secrets manager at: %s\n", stateDir)
} }
// Prompt for mnemonic // Prompt for mnemonic
@@ -130,11 +114,38 @@ func (cli *Instance) initialize(cmd *cobra.Command) error {
} }
defer cleanupPassphrase() defer cleanupPassphrase()
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
return cli.initialize(cmd, mnemonic, passphraseBuffer)
}
// initialize creates the state directory, and the default vault from
// mnemonic with a passphrase unlocker protected by passphrase. Another
// command may have created the vault since Init checked: vault.CreateVault
// checks again.
func (cli *Instance) initialize(
cmd *cobra.Command, mnemonic, passphrase *memguard.LockedBuffer,
) error {
// Create state directory
stateDir := cli.GetStateDir()
secret.DebugWith("Creating state directory", slog.String("path", stateDir))
err := cli.fs.MkdirAll(stateDir, secret.DirPerms)
if err != nil {
secret.Debug("Failed to create state directory", "error", err)
return fmt.Errorf("failed to create state directory: %w", err)
}
// Create the default vault with its passphrase unlocker // Create the default vault with its passphrase unlocker
secret.Debug("Creating default vault") secret.Debug("Creating default vault")
vlt, err := vault.CreateVault(cli.fs, cli.stateDir, "default", vlt, err := vault.CreateVault(cli.fs, cli.stateDir, defaultVaultName,
mnemonic, passphraseBuffer) mnemonic, passphrase)
if err != nil { if err != nil {
secret.Debug("Failed to create default vault", "error", err) secret.Debug("Failed to create default vault", "error", err)
@@ -152,6 +163,7 @@ func (cli *Instance) initialize(cmd *cobra.Command) error {
} }
if cmd != nil { if cmd != nil {
cmd.Printf("Initialized secrets manager at: %s\n", stateDir)
cmd.Printf("\nDefault vault created and configured\n") cmd.Printf("\nDefault vault created and configured\n")
cmd.Printf("Long-term public key: %s\n", ltIdentity.Recipient().String()) cmd.Printf("Long-term public key: %s\n", ltIdentity.Recipient().String())
cmd.Printf("Unlocker ID: %s\n", unlocker.GetID()) cmd.Printf("Unlocker ID: %s\n", unlocker.GetID())
+2 -11
View File
@@ -7,12 +7,10 @@ import (
"github.com/awnumar/memguard" "github.com/awnumar/memguard"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/secret/internal/vault"
) )
// TestInvalidMnemonicError checks that every command that takes a mnemonic // TestInvalidMnemonicError checks that every command that takes a mnemonic
// returns errInvalidMnemonicPhrase for one that is not valid BIP39. The vault // returns errInvalidMnemonicPhrase for one that is not valid BIP39.
// "other" has no long-term key, as vault import needs.
func TestInvalidMnemonicError(t *testing.T) { func TestInvalidMnemonicError(t *testing.T) {
t.Parallel() t.Parallel()
@@ -24,20 +22,13 @@ func TestInvalidMnemonicError(t *testing.T) {
{"secret vault create work", func(c *Instance) error { {"secret vault create work", func(c *Instance) error {
return c.CreateVault(c.cmd, "work") return c.CreateVault(c.cmd, "work")
}}, }},
{"secret vault import other", func(c *Instance) error {
return c.VaultImport(c.cmd, "other")
}},
} }
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.command, func(t *testing.T) { t.Run(tt.command, func(t *testing.T) {
t.Parallel() t.Parallel()
fs := afero.NewMemMapFs() instance, _ := newTestInstance(afero.NewMemMapFs())
_, err := vault.CreateVault(fs, listTestStateDir, "other", nil, nil)
require.NoError(t, err)
instance, _ := newTestInstance(fs)
instance.Mnemonic = memguard.NewBufferFromBytes([]byte("not a mnemonic")) instance.Mnemonic = memguard.NewBufferFromBytes([]byte("not a mnemonic"))
t.Cleanup(instance.Mnemonic.Destroy) t.Cleanup(instance.Mnemonic.Destroy)
+4 -80
View File
@@ -34,9 +34,10 @@ const (
testMnemonic = "abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about" testMnemonic = "abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about"
// commandWait is how long a test lets the secret binary run before it // commandWait is how long a test lets the secret binary run before it
// kills it and fails. It stays well under the 30 seconds script/test // kills it and fails. It stays well under the time limit the test phase
// gives the whole package, so a command that hangs fails the test with // of the Dockerfile gives the whole package (go test -timeout), so a
// the test's own message instead of Go's timeout panic. // command that hangs fails the test with the test's own message instead
// of Go's timeout panic.
commandWait = 10 * time.Second commandWait = 10 * time.Second
) )
@@ -140,16 +141,6 @@ func TestSecretManagerIntegration(t *testing.T) {
// - currentvault symlink updated to point to work // - currentvault symlink updated to point to work
test03CreateVault(t, tempDir, runSecret) test03CreateVault(t, tempDir, runSecret)
// Test 4: Import mnemonic into work vault
// Command: secret vault import work
// Purpose: Import mnemonic with passphrase unlocker
// Expected filesystem:
// - work vault has pub.age file
// - work vault has unlockers.d/passphrase directory
// - Unlocker metadata and encrypted keys present
// NOTE: Skipped because vault creation now includes mnemonic import
// test04ImportMnemonic(t, tempDir, testMnemonic, testPassphrase, runSecretWithEnv)
// Test 5: Add secrets with versioning // Test 5: Add secrets with versioning
// Command: echo "password123" | secret add database/password // Command: echo "password123" | secret add database/password
// Purpose: Add a secret and verify versioned storage // Purpose: Add a secret and verify versioned storage
@@ -535,65 +526,6 @@ func test03CreateVault(t *testing.T, tempDir string, runSecret func(...string) (
assert.Contains(t, output, "work", "should list work vault") assert.Contains(t, output, "work", "should list work vault")
} }
//nolint:unused // TODO: re-enable when vault import is implemented
func test04ImportMnemonic(t *testing.T, tempDir, testMnemonic, testPassphrase string, runSecretWithEnv func(map[string]string, ...string) (string, error)) {
t.Helper()
// Import mnemonic into work vault
output, err := runSecretWithEnv(map[string]string{
secret.EnvMnemonic: testMnemonic,
secret.EnvUnlockPassphrase: testPassphrase,
}, "vault", "import", "work")
require.NoError(t, err, "vault import should succeed")
assert.Contains(t, output, "Successfully imported mnemonic into vault 'work'", "should confirm import")
// Verify work vault now has long-term key
workVaultDir := filepath.Join(tempDir, "vaults.d", "work")
pubKeyFile := filepath.Join(workVaultDir, "pub.age")
verifyFileExists(t, pubKeyFile)
// Verify passphrase unlocker was created
currentUnlocker := readFile(t, filepath.Join(workVaultDir, "current-unlocker"))
passphraseUnlockerDir := filepath.Join(workVaultDir, "unlockers.d", string(currentUnlocker))
verifyFileExists(t, passphraseUnlockerDir)
// Check unlocker files
unlockerMetadata := filepath.Join(passphraseUnlockerDir, "unlocker-metadata.json")
verifyFileExists(t, unlockerMetadata)
encryptedLTPrivKey := filepath.Join(passphraseUnlockerDir, "priv.age")
verifyFileExists(t, encryptedLTPrivKey)
encryptedLTPubKey := filepath.Join(passphraseUnlockerDir, "pub.age")
verifyFileExists(t, encryptedLTPubKey)
longtermKeyFile := filepath.Join(passphraseUnlockerDir, "longterm.age")
verifyFileExists(t, longtermKeyFile)
// Verify vault metadata was created with derivation index
vaultMetadata := filepath.Join(workVaultDir, "vault-metadata.json")
verifyFileExists(t, vaultMetadata)
metadataBytes := readFile(t, vaultMetadata)
var metadata map[string]any
err = json.Unmarshal(metadataBytes, &metadata)
require.NoError(t, err, "vault metadata should be valid JSON")
// Work vault should have a different derivation index than default (0)
derivIndex, ok := metadata["derivationIndex"].(float64)
require.True(t, ok, "derivationIndex should be a number")
assert.NotEqual(t, float64(0), derivIndex, "work vault should have non-zero derivation index")
// Verify public key hash is stored
assert.Contains(t, metadata, "publicKeyHash", "should contain public key hash")
pubKeyHash, ok := metadata["publicKeyHash"].(string)
require.True(t, ok, "publicKeyHash should be a string")
assert.NotEmpty(t, pubKeyHash, "public key hash should not be empty")
}
func test05AddSecret(t *testing.T, tempDir, testMnemonic string, runSecret func(...string) (string, error), runSecretWithEnv func(map[string]string, ...string) (string, error), runSecretWithStdin func(string, map[string]string, ...string) (string, error)) { func test05AddSecret(t *testing.T, tempDir, testMnemonic string, runSecret func(...string) (string, error), runSecretWithEnv func(map[string]string, ...string) (string, error), runSecretWithStdin func(string, map[string]string, ...string) (string, error)) {
t.Helper() t.Helper()
@@ -1939,14 +1871,6 @@ func test23ErrorHandling(t *testing.T, tempDir, secretPath, testMnemonic string,
_, err = runSecret("vault", "select", "nonexistent") _, err = runSecret("vault", "select", "nonexistent")
require.ErrorIs(t, err, vault.ErrVaultNotFound, "select non-existent vault should fail") require.ErrorIs(t, err, vault.ErrVaultNotFound, "select non-existent vault should fail")
// Import to non-existent vault with test passphrase
testPassphrase := "test-passphrase-123" // Define testPassphrase locally
_, err = runSecretWithEnv(map[string]string{
secret.EnvMnemonic: testMnemonic,
secret.EnvUnlockPassphrase: testPassphrase,
}, "vault", "import", "nonexistent")
require.ErrorIs(t, err, vault.ErrVaultNotFound, "import to non-existent vault should fail")
// Get specific version that doesn't exist // Get specific version that doesn't exist
_, err = runSecretWithEnv(map[string]string{ _, err = runSecretWithEnv(map[string]string{
secret.EnvMnemonic: testMnemonic, secret.EnvMnemonic: testMnemonic,
+215
View File
@@ -0,0 +1,215 @@
package cli_test
import (
"bytes"
"encoding/json"
"os"
"os/exec"
"path/filepath"
"testing"
"time"
"github.com/awnumar/memguard"
"github.com/spf13/afero"
"github.com/spf13/cobra"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/secret/internal/cli"
"sneak.berlin/go/secret/internal/secret"
"sneak.berlin/go/secret/internal/vault"
)
const (
// listClockStep is how long TestListShowsWhenSecretWasWritten waits
// between its steps, so that a time read in one step cannot be the
// time read in the next.
listClockStep = 200 * time.Millisecond
// fileTimeSlack allows for the clock the kernel stamps files with,
// which advances only every few milliseconds: a file's modification
// time can be slightly earlier than a time.Now() read before the file
// was written.
fileTimeSlack = 100 * time.Millisecond
)
// TestListShowsWhenSecretWasWritten is a regression test for
// https://git.eeqj.de/sneak/secret/issues/142, where `secret list` gave
// the time of the listing as every secret's creation and last-update time.
// Two listings with a clock step between must be the same, and give the
// time of the add; after a forced add, the creation time is unchanged and
// the last update is the time of that add. It runs on a real filesystem,
// where writing a file into a directory also changes the directory's time.
func TestListShowsWhenSecretWasWritten(t *testing.T) {
t.Parallel()
fs := afero.NewOsFs()
stateDir := t.TempDir()
vlt, err := vault.CreateVault(fs, stateDir, "default", testMnemonicBuffer(t), nil)
require.NoError(t, err)
vlt.Mnemonic = testMnemonicBuffer(t)
c := cli.NewCLIInstanceWithStateDir(fs, stateDir)
beforeAdd := time.Now()
err = vlt.AddSecret("x", memguard.NewBufferFromBytes([]byte("one")), false)
require.NoError(t, err)
afterAdd := time.Now()
firstListing := listSecrets(t, c, true)
time.Sleep(listClockStep)
require.Equal(t, firstListing, listSecrets(t, c, true))
added := listedTimes(t, firstListing)
assert.WithinRange(t, added.CreatedAt, beforeAdd.Add(-fileTimeSlack), afterAdd)
assert.WithinRange(t, added.UpdatedAt, beforeAdd.Add(-fileTimeSlack), afterAdd)
time.Sleep(listClockStep)
beforeUpdate := time.Now()
err = vlt.AddSecret("x", memguard.NewBufferFromBytes([]byte("two")), true)
require.NoError(t, err)
afterUpdate := time.Now()
updated := listedTimes(t, listSecrets(t, c, true))
assert.True(t, added.CreatedAt.Equal(updated.CreatedAt),
"created %s, then %s", added.CreatedAt, updated.CreatedAt)
assert.WithinRange(t, updated.UpdatedAt,
beforeUpdate.Add(-fileTimeSlack), afterUpdate)
// The table's LAST UPDATED column gives the same time, to the minute
assert.Contains(t, listSecrets(t, c, false),
updated.UpdatedAt.Format("2006-01-02 15:04"))
}
// TestMovedSecretKeepsItsTimes checks that a secret moved to another vault
// is listed there with the times it was listed with before the move, and
// not with the time of the move.
func TestMovedSecretKeepsItsTimes(t *testing.T) {
t.Parallel()
fs := afero.NewOsFs()
stateDir := t.TempDir()
_, err := vault.CreateVault(fs, stateDir, "other", testMnemonicBuffer(t), nil)
require.NoError(t, err)
// "default" is created last, so it is the current vault
vlt, err := vault.CreateVault(fs, stateDir, "default", testMnemonicBuffer(t), nil)
require.NoError(t, err)
vlt.Mnemonic = testMnemonicBuffer(t)
// Two versions written apart, so that the creation and the last update
// are different times
err = vlt.AddSecret("x", memguard.NewBufferFromBytes([]byte("one")), false)
require.NoError(t, err)
time.Sleep(listClockStep)
err = vlt.AddSecret("x", memguard.NewBufferFromBytes([]byte("two")), true)
require.NoError(t, err)
c := cli.NewCLIInstanceWithStateDir(fs, stateDir)
c.Mnemonic = testMnemonicBuffer(t)
beforeMove := listedTimes(t, listSecrets(t, c, true))
time.Sleep(listClockStep)
require.NoError(t, c.MoveSecret(&cobra.Command{}, "default:x", "other", false))
require.NoError(t, vault.SelectVault(fs, stateDir, "other"))
afterMove := listedTimes(t, listSecrets(t, c, true))
assert.True(t, beforeMove.CreatedAt.Equal(afterMove.CreatedAt),
"created %s, then %s", beforeMove.CreatedAt, afterMove.CreatedAt)
assert.True(t, beforeMove.UpdatedAt.Equal(afterMove.UpdatedAt),
"updated %s, then %s", beforeMove.UpdatedAt, afterMove.UpdatedAt)
}
// TestListedTimesAreUTC is a regression test for
// https://git.eeqj.de/sneak/secret/issues/169, where times were printed
// in the local time zone. It runs `secret list` on the built binary with
// TZ set to New York, where the secret's last update, 03:04:05 UTC on 2
// January, is 22:04:05 the day before.
func TestListedTimesAreUTC(t *testing.T) {
t.Parallel()
stateDir := t.TempDir()
vlt, err := vault.CreateVault(
afero.NewOsFs(), stateDir, "default", testMnemonicBuffer(t), nil)
require.NoError(t, err)
vlt.Mnemonic = testMnemonicBuffer(t)
err = vlt.AddSecret("x", memguard.NewBufferFromBytes([]byte("one")), false)
require.NoError(t, err)
updated := time.Date(2026, time.January, 2, 3, 4, 5, 0, time.UTC)
current := filepath.Join(stateDir, "vaults.d", "default", "secrets.d", "x",
"current")
require.NoError(t, os.Chtimes(current, updated, updated))
tests := []struct {
flags []string
want string
}{
{flags: nil, want: "2026-01-02 03:04"},
{
flags: []string{"--json"},
want: `"updated_at": "2026-01-02T03:04:05Z"`,
},
}
for _, tt := range tests {
args := append([]string{"list"}, tt.flags...)
//nolint:gosec // G204: test executes the freshly built secret binary
cmd := exec.CommandContext(t.Context(), secretBinaryPath(t), args...)
cmd.Env = []string{
secret.EnvStateDir + "=" + stateDir,
"TZ=America/New_York",
"PATH=" + os.Getenv("PATH"),
}
output, err := cmd.CombinedOutput()
require.NoError(t, err, "%v: %s", args, output)
assert.Contains(t, string(output), tt.want, "%v", args)
}
}
// listSecrets returns the output of `secret list`, or of
// `secret list --json` when jsonOutput is set.
func listSecrets(t *testing.T, c *cli.Instance, jsonOutput bool) string {
t.Helper()
var out bytes.Buffer
cmd := &cobra.Command{}
cmd.SetOut(&out)
require.NoError(t, c.ListSecrets(cmd, jsonOutput, false, ""))
return out.String()
}
// listedTimes returns the times that the output of `secret list --json`
// gives for the one secret in the vault.
func listedTimes(t *testing.T, listing string) secret.Metadata {
t.Helper()
var parsed struct {
Secrets []map[string]string `json:"secrets"`
}
require.NoError(t, json.Unmarshal([]byte(listing), &parsed))
require.Len(t, parsed.Secrets, 1)
createdAt, err := time.Parse(time.RFC3339Nano, parsed.Secrets[0]["created_at"])
require.NoError(t, err)
updatedAt, err := time.Parse(time.RFC3339Nano, parsed.Secrets[0]["updated_at"])
require.NoError(t, err)
return secret.Metadata{CreatedAt: createdAt, UpdatedAt: updatedAt}
}
+332 -26
View File
@@ -280,13 +280,12 @@ func stateDirModTimes(t *testing.T, fs afero.Fs) map[string]int64 {
return modTimes return modTimes
} }
// setupEveryCommand makes what each command in // setupEveryCommand makes what the commands that the lock tests below run
// TestChangingCommandsWaitForLock needs: the current vault "work" with two // need: the current vault "work" with two versions of "test/secret", the
// versions of "test/secret", the vault "other" without a long-term key, for // vault "other", and the file testInput.
// vault import, and the file testInput. There is no vault "default", which // There is no vault "default", which init creates. If withUnlocker is set,
// init creates. If withUnlocker is set, it also gives "work" a passphrase // it also gives "work" a passphrase unlocker, which is slow. It returns the
// unlocker, which is slow. It returns the older version and the unlocker's // older version and the unlocker's ID.
// ID.
func setupEveryCommand( func setupEveryCommand(
t *testing.T, fs afero.Fs, withUnlocker bool, t *testing.T, fs afero.Fs, withUnlocker bool,
) (string, string) { ) (string, string) {
@@ -294,13 +293,9 @@ func setupEveryCommand(
mnemonic := testMnemonicBuffer(t) mnemonic := testMnemonicBuffer(t)
other, err := vault.CreateVault(fs, testStateDir, "other", mnemonic, nil) _, err := vault.CreateVault(fs, testStateDir, "other", mnemonic, nil)
require.NoError(t, err) require.NoError(t, err)
otherDir, err := other.GetDirectory()
require.NoError(t, err)
require.NoError(t, fs.Remove(filepath.Join(otherDir, "pub.age")))
vlt, err := vault.CreateVault(fs, testStateDir, "work", mnemonic, nil) vlt, err := vault.CreateVault(fs, testStateDir, "work", mnemonic, nil)
require.NoError(t, err) require.NoError(t, err)
@@ -351,6 +346,25 @@ func waitingForLock() bool {
return false return false
} }
// requireWaitingForLock returns once a command waits for the state
// directory lock. It fails the test if the command finishes first, sending
// its error on done, or never waits.
func requireWaitingForLock(t *testing.T, done <-chan error) {
t.Helper()
timeout := time.After(lockWait)
for !waitingForLock() {
select {
case err := <-done:
t.Fatalf("finished while the lock was held, with error %v", err)
case <-timeout:
t.Fatal("never waited for the lock")
case <-time.After(time.Millisecond):
}
}
}
// requireWaitsForLock runs a command, given what setupEveryCommand made, // requireWaitsForLock runs a command, given what setupEveryCommand made,
// while holding the state directory lock. The command must neither finish // while holding the state directory lock. The command must neither finish
// nor change anything before it waits for the lock, and must succeed once // nor change anything before it waits for the lock, and must succeed once
@@ -390,17 +404,7 @@ func requireWaitsForLock(
go func() { done <- run(cli, olderVersion, unlockerID) }() go func() { done <- run(cli, olderVersion, unlockerID) }()
timeout := time.After(lockWait) requireWaitingForLock(t, done)
for !waitingForLock() {
select {
case err := <-done:
t.Fatalf("finished while the lock was held, with error %v", err)
case <-timeout:
t.Fatal("never waited for the lock")
case <-time.After(time.Millisecond):
}
}
assert.Equal(t, before, stateDirModTimes(t, fs), assert.Equal(t, before, stateDirModTimes(t, fs),
"changed the state directory before waiting for the lock") "changed the state directory before waiting for the lock")
@@ -455,9 +459,6 @@ func TestChangingCommandsWaitForLock(t *testing.T) {
{"vault select", false, func(cli *Instance, _, _ string) error { {"vault select", false, func(cli *Instance, _, _ string) error {
return cli.SelectVault(cli.cmd, "other") return cli.SelectVault(cli.cmd, "other")
}}, }},
{"vault import", false, func(cli *Instance, _, _ string) error {
return cli.VaultImport(cli.cmd, "other")
}},
{"vault rm --force", false, func(cli *Instance, _, _ string) error { {"vault rm --force", false, func(cli *Instance, _, _ string) error {
return cli.RemoveVault(cli.cmd, "other", true) return cli.RemoveVault(cli.cmd, "other", true)
}}, }},
@@ -480,6 +481,311 @@ func TestChangingCommandsWaitForLock(t *testing.T) {
} }
} }
// TestCommandsAskBeforeLocking is a regression test for
// https://git.eeqj.de/sneak/secret/issues/152: these commands took the
// state directory lock before asking for the mnemonic or a passphrase, so
// while one waited at its prompt, every other command that changes anything
// waited for it. Each runs here while the test holds the lock, given what
// takes it to the last step at which it can ask: the passphrase prompt of
// init and vault create, and the unlock of the vault, which asks for the
// passphrase of its passphrase unlocker, for the others. The tests have no
// terminal, so a prompt fails at once. Each command must get there, and
// fail, without waiting for the lock.
//
//nolint:paralleltest // waitingForLock sees any test's command waiting for the lock
func TestCommandsAskBeforeLocking(t *testing.T) {
newTestGPGKey(t)
mnemonic := testMnemonicBuffer(t)
wrongPassphrase := memguard.NewBufferFromBytes([]byte("wrong-passphrase"))
t.Cleanup(wrongPassphrase.Destroy)
for _, tc := range []struct {
name string
run func(cli *Instance) error
// want is wrapped in the error the command fails with
want error
}{
{"init", func(cli *Instance) error {
cli.Mnemonic = mnemonic
return cli.Init(cli.cmd)
}, secret.ErrNoTerminal},
{"vault create", func(cli *Instance) error {
cli.Mnemonic = mnemonic
return cli.CreateVault(cli.cmd, "created")
}, secret.ErrNoTerminal},
// With no terminal to ask for the new passphrase on, the new
// unlocker takes the one in SB_UNLOCK_PASSPHRASE, which then fails
// to unlock the vault
{"unlocker add passphrase", func(cli *Instance) error {
cli.UnlockPassphrase = wrongPassphrase
return cli.UnlockersAdd("passphrase", cli.cmd)
}, secret.ErrWrongPassphrase},
{"unlocker add pgp", func(cli *Instance) error {
cli.cmd.Flags().String("keyid", unreadableTestGPGUserID, "")
return cli.UnlockersAdd("pgp", cli.cmd)
}, secret.ErrNoTerminal},
{"add --force", func(cli *Instance) error {
return cli.AddSecret("test/secret", true)
}, secret.ErrNoTerminal},
{"import --force", func(cli *Instance) error {
return cli.ImportSecret(cli.cmd, "test/secret", testInput, true)
}, secret.ErrNoTerminal},
{"generate secret --force", func(cli *Instance) error {
return cli.GenerateSecret(cli.cmd, "test/secret", 16, "base58", true)
}, secret.ErrNoTerminal},
} {
t.Run(tc.name, func(t *testing.T) {
requireAsksBeforeLocking(t, tc.run, tc.want)
})
}
}
// startWhileLocked makes what setupEveryCommand makes, with the passphrase
// unlocker, on fs, an empty in-memory filesystem, takes the state directory
// lock there, and starts run, given a CLI instance on fs with "value" on
// stdin. It returns the function that releases the lock, which does so only
// the first time it is called, and the channel that run's error comes on.
// The lock is also released when the test ends, so that later tests can
// take it.
func startWhileLocked(
t *testing.T, fs afero.Fs, run func(cli *Instance) error,
) (func(), <-chan error) {
t.Helper()
setupEveryCommand(t, fs, true)
release, err := vault.LockStateDir(fs, testStateDir)
require.NoError(t, err)
release = sync.OnceFunc(release)
t.Cleanup(release)
cli := NewCLIInstanceWithStateDir(fs, testStateDir)
cli.cmd = &cobra.Command{}
cli.cmd.SetIn(strings.NewReader("value"))
cli.cmd.SetOut(io.Discard)
done := make(chan error, 1)
go func() { done <- run(cli) }()
return release, done
}
// requireAsksBeforeLocking starts run while holding the state directory
// lock, as startWhileLocked does. run must fail, with an error wrapping
// want, without waiting for the lock.
func requireAsksBeforeLocking(
t *testing.T, run func(cli *Instance) error, want error,
) {
t.Helper()
release, done := startWhileLocked(t, afero.NewMemMapFs(), run)
timeout := time.After(lockWait)
for !waitingForLock() {
select {
case err := <-done:
require.ErrorIs(t, err, want)
return
case <-timeout:
t.Fatal("neither failed nor waited for the lock")
case <-time.After(time.Millisecond):
}
}
// Let the command go on, so that it releases the lock
release()
select {
case <-done:
case <-time.After(lockWait):
}
t.Fatal("waited for the lock before asking")
}
// TestCommandsCheckAgainOnceLocked checks that commands which check what
// they change before they ask for anything, and so before they take the
// state directory lock, check again once they hold it: another command may
// have changed it in the meantime. Each runs while the test holds the lock,
// given what it needs to ask for nothing. Once it waits for the lock, the
// test makes the change another command could have made, then releases the
// lock. The command must fail, and change nothing.
//
//nolint:paralleltest // waitingForLock sees any test's command waiting for the lock
func TestCommandsCheckAgainOnceLocked(t *testing.T) {
fingerprint := newTestGPGKey(t)
mnemonic := testMnemonicBuffer(t)
otherMnemonic := memguard.NewBufferFromBytes([]byte("legal winner thank " +
"year wave sausage worth useful legal winner thank yellow"))
t.Cleanup(otherMnemonic.Destroy)
passphrase := memguard.NewBufferFromBytes([]byte(testPassphrase))
t.Cleanup(passphrase.Destroy)
// work returns the vault "work" on fs, opened with the mnemonic
work := func(fs afero.Fs) *vault.Vault {
vlt := vault.NewVault(fs, testStateDir, "work")
vlt.Mnemonic = mnemonic
return vlt
}
for _, tc := range []struct {
name string
run func(cli *Instance) error
// change is what another command changes while run waits
change func(fs afero.Fs) error
// want is wrapped in the error the command fails with
want error
}{
{
"unlocker add, vault removed",
func(cli *Instance) error {
cli.Mnemonic, cli.UnlockPassphrase = mnemonic, passphrase
return cli.UnlockersAdd("passphrase", cli.cmd)
},
func(fs afero.Fs) error {
return fs.RemoveAll(testStateDir + "/vaults.d/work")
},
vault.ErrVaultNotFound,
},
{
"unlocker add, vault created again from another mnemonic",
func(cli *Instance) error {
cli.Mnemonic, cli.UnlockPassphrase = mnemonic, passphrase
return cli.UnlockersAdd("passphrase", cli.cmd)
},
func(fs afero.Fs) error {
err := fs.RemoveAll(testStateDir + "/vaults.d/work")
if err != nil {
return err
}
_, err = vault.CreateVault(fs, testStateDir, "work",
otherMnemonic, passphrase)
return err
},
errVaultCreatedAgain,
},
{
"unlocker add pgp, key added",
func(cli *Instance) error {
cli.Mnemonic = mnemonic
cli.cmd.Flags().String("keyid", unreadableTestGPGUserID, "")
return cli.UnlockersAdd("pgp", cli.cmd)
},
func(fs afero.Fs) error {
_, err := secret.CreatePGPUnlocker(work(fs),
unreadableTestGPGUserID, fingerprint)
return err
},
errGPGKeyAlreadyUnlocker,
},
} {
t.Run(tc.name, func(t *testing.T) {
requireChecksAgainOnceLocked(t, tc.run, tc.change, tc.want)
})
}
}
// requireChecksAgainOnceLocked starts run while holding the state directory
// lock, as startWhileLocked does. Once run waits for the lock, change makes
// the change that another command could have made in the meantime, and the
// lock is released. run must then fail, with an error wrapping want, and
// change nothing.
func requireChecksAgainOnceLocked(
t *testing.T, run func(cli *Instance) error,
change func(fs afero.Fs) error, want error,
) {
t.Helper()
fs := afero.NewMemMapFs()
release, done := startWhileLocked(t, fs, run)
requireWaitingForLock(t, done)
require.NoError(t, change(fs))
// Left out: taking and releasing the lock writes the lock file
lockFile := testStateDir + "/lock"
before := stateDirModTimes(t, fs)
delete(before, lockFile)
release()
select {
case err := <-done:
require.ErrorIs(t, err, want)
case <-time.After(lockWait):
t.Fatal("did not finish once the lock was released")
}
after := stateDirModTimes(t, fs)
delete(after, lockFile)
assert.Equal(t, before, after, "changed the state directory")
}
// TestForcedAddReplacesSecretAddedMeanwhile runs a forced add of a secret
// that does not exist yet while the test holds the state directory lock.
// Once it waits for the lock, the test adds the secret, as another command
// could have, and releases the lock. The forced add must replace that
// secret, as if it had run after the other command.
//
//nolint:paralleltest // waitingForLock sees any test's command waiting for the lock
func TestForcedAddReplacesSecretAddedMeanwhile(t *testing.T) {
mnemonic := testMnemonicBuffer(t)
fs := afero.NewMemMapFs()
release, done := startWhileLocked(t, fs, func(cli *Instance) error {
cli.Mnemonic = mnemonic
return cli.AddSecret("added", true)
})
requireWaitingForLock(t, done)
work := vault.NewVault(fs, testStateDir, "work")
work.Mnemonic = mnemonic
other := memguard.NewBufferFromBytes([]byte("other"))
t.Cleanup(other.Destroy)
require.NoError(t, work.AddSecret("added", other, false))
release()
select {
case err := <-done:
require.NoError(t, err)
case <-time.After(lockWait):
t.Fatal("did not finish once the lock was released")
}
value, err := work.GetSecret("added")
require.NoError(t, err)
t.Cleanup(value.Destroy)
assert.Equal(t, "value", string(value.Bytes()))
}
// TestEncryptWithExistingKeyTakesNoLock checks that secret encrypt with a // TestEncryptWithExistingKeyTakesNoLock checks that secret encrypt with a
// key that already exists, which only reads the state directory, finishes // key that already exists, which only reads the state directory, finishes
// while another command holds the state directory lock. // while another command holds the state directory lock.
+315
View File
@@ -0,0 +1,315 @@
//nolint:testpackage // white-box test of unexported internals
package cli
import (
"errors"
"path/filepath"
"testing"
"time"
"github.com/awnumar/memguard"
"github.com/spf13/afero"
"github.com/spf13/cobra"
"github.com/stretchr/testify/require"
"sneak.berlin/go/secret/internal/secret"
"sneak.berlin/go/secret/internal/vault"
)
// errPermissionDenied is returned by renameFailFs and removeAllFailFs in
// place of a rename or removal that succeeds.
var errPermissionDenied = errors.New("permission denied")
// renameFailFs fails every Rename of one path, as a read-only parent
// directory does.
type renameFailFs struct {
afero.Fs
path string
}
func (f *renameFailFs) Rename(oldname, newname string) error {
if oldname == f.path {
return errPermissionDenied
}
return f.Fs.Rename(oldname, newname)
}
// removeAllFailFs fails every RemoveAll of an entry of the directory dir,
// as a file in it that cannot be deleted does, and records that entry's
// path in failed.
type removeAllFailFs struct {
afero.Fs
dir string
failed string
}
func (f *removeAllFailFs) RemoveAll(path string) error {
if filepath.Dir(path) == f.dir {
f.failed = path
return errPermissionDenied
}
return f.Fs.RemoveAll(path)
}
// createCrossVaults creates in fs the vaults "work" and "default", where
// "default" holds the secret "x" with the value "value".
func createCrossVaults(t *testing.T, fs afero.Fs) {
t.Helper()
_, err := vault.CreateVault(fs, testStateDir, "work",
testMnemonicBuffer(t), nil)
require.NoError(t, err)
source, err := vault.CreateVault(fs, testStateDir, "default",
testMnemonicBuffer(t), nil)
require.NoError(t, err)
err = source.AddSecret("x",
memguard.NewBufferFromBytes([]byte("value")), false)
require.NoError(t, err)
}
// requireValue requires that the vault vaultName in fs holds secretName
// with the value createCrossVaults stored.
func requireValue(t *testing.T, fs afero.Fs, vaultName, secretName string) {
t.Helper()
vlt := vault.NewVault(fs, testStateDir, vaultName)
vlt.Mnemonic = testMnemonicBuffer(t)
value, err := vlt.GetSecret(secretName)
require.NoError(t, err, vaultName+":"+secretName)
defer value.Destroy()
require.Equal(t, []byte("value"), value.Bytes())
}
// TestCrossVaultMoveFailsWhenSourceRemains is a regression test for
// https://git.eeqj.de/sneak/secret/issues/159, where a move between two
// vaults whose source could not be removed reported the secret moved and
// succeeded. It must fail, and the copy and the source must both be there.
// The state directory lock refuses the wrapped filesystem, so the test
// calls copySecretAndRemoveSource, which a move between vaults runs once it
// holds the lock.
func TestCrossVaultMoveFailsWhenSourceRemains(t *testing.T) {
t.Parallel()
base := afero.NewMemMapFs()
createCrossVaults(t, base)
instance := NewCLIInstanceWithStateDir(&renameFailFs{
Fs: base,
path: testStateDir + "/vaults.d/default/secrets.d/x",
}, testStateDir)
source := vault.NewVault(instance.fs, testStateDir, "default")
source.Mnemonic = testMnemonicBuffer(t)
err := instance.copySecretAndRemoveSource(&cobra.Command{}, source, "x",
vault.NewVault(instance.fs, testStateDir, "work"), "y", false)
require.ErrorIs(t, err, errPermissionDenied)
require.ErrorContains(t, err, "secret 'default:x' is copied with all "+
"1 version(s) to 'work:y', but the source remains")
requireValue(t, base, "default", "x")
requireValue(t, base, "work", "y")
}
// TestCrossVaultMoveFailsWhenOldFilesRemain makes the removal of the source
// fail after the source has been moved out of its vault, when only the
// temporary directory it was moved into cannot be deleted. The move must
// fail, say that the secret is moved, and name that directory.
func TestCrossVaultMoveFailsWhenOldFilesRemain(t *testing.T) {
t.Parallel()
base := afero.NewMemMapFs()
createCrossVaults(t, base)
sourceVaultDir := testStateDir + "/vaults.d/default"
fs := &removeAllFailFs{Fs: base, dir: sourceVaultDir}
instance := NewCLIInstanceWithStateDir(fs, testStateDir)
source := vault.NewVault(fs, testStateDir, "default")
source.Mnemonic = testMnemonicBuffer(t)
err := instance.copySecretAndRemoveSource(&cobra.Command{}, source, "x",
vault.NewVault(fs, testStateDir, "work"), "y", false)
require.ErrorIs(t, err, errPermissionDenied)
require.ErrorContains(t, err, "secret 'default:x' is moved with all "+
"1 version(s) to 'work:y', but its old files could not all be deleted")
// The error names the directory that holds the old files
require.NotEmpty(t, fs.failed)
require.ErrorContains(t, err, fs.failed)
oldFiles, err := afero.DirExists(base, filepath.Join(fs.failed, "x"))
require.NoError(t, err)
require.True(t, oldFiles)
// The source vault no longer holds the secret
remains, err := afero.DirExists(base, sourceVaultDir+"/secrets.d/x")
require.NoError(t, err)
require.False(t, remains)
requireValue(t, base, "work", "y")
}
// TestCrossVaultMoveUnlocksBeforeLocking is a regression test for
// https://git.eeqj.de/sneak/secret/issues/189: a move between two vaults
// unlocked them while holding the state directory lock, and unlocking a
// vault can ask for its passphrase, so every other command that changes
// anything waited while the user answered. Here each vault has a passphrase
// unlocker whose passphrase is the vault's name, and the move is given the
// passphrase of only one of them. Run while the test holds the lock, it
// must fail to unlock the other vault without waiting for the lock.
//
//nolint:paralleltest // times commands against the in-memory lock all tests share
func TestCrossVaultMoveUnlocksBeforeLocking(t *testing.T) {
fs := afero.NewMemMapFs()
createCrossVaults(t, fs)
vaults, err := vault.ListVaults(fs, testStateDir)
require.NoError(t, err)
for _, name := range vaults {
vlt := vault.NewVault(fs, testStateDir, name)
vlt.Mnemonic = testMnemonicBuffer(t)
passphrase := memguard.NewBufferFromBytes([]byte(name))
_, err = vlt.CreatePassphraseUnlocker(passphrase)
passphrase.Destroy()
require.NoError(t, err)
}
// The move is from "default" to "work"
for _, tc := range []struct {
// failing is the vault that the move cannot unlock, given the
// passphrase of the vault named given
failing, given string
}{
{"source", "work"},
{"destination", "default"},
} {
t.Run(tc.failing, func(t *testing.T) {
passphrase := memguard.NewBufferFromBytes([]byte(tc.given))
defer passphrase.Destroy()
instance := NewCLIInstanceWithStateDir(fs, testStateDir)
instance.UnlockPassphrase = passphrase
release, err := vault.LockStateDir(fs, testStateDir)
require.NoError(t, err)
// Also frees a waiting move if the test fails, so that it
// releases the lock the other tests use
defer release()
done := make(chan error, 1)
go func() {
done <- instance.MoveSecret(&cobra.Command{},
"default:x", "work:y", false)
}()
select {
case err := <-done:
require.ErrorIs(t, err, secret.ErrWrongPassphrase)
require.ErrorContains(t, err,
"failed to unlock "+tc.failing+" vault")
case <-time.After(lockWait):
t.Fatal("waited for the lock before unlocking the " +
tc.failing + " vault")
}
})
}
}
// TestCrossVaultMoveChecksAgainOnceLocked runs a move between vaults, given
// the mnemonic, as TestCommandsCheckAgainOnceLocked runs the commands in its
// table: once the move waits for the state directory lock, the test removes
// the source secret or the destination vault, or creates the destination
// vault again from another mnemonic, and the move must then fail and change
// nothing.
//
//nolint:paralleltest // waitingForLock sees any test's command waiting for the lock
func TestCrossVaultMoveChecksAgainOnceLocked(t *testing.T) {
mnemonic := testMnemonicBuffer(t)
otherMnemonic := memguard.NewBufferFromBytes([]byte("legal winner thank " +
"year wave sausage worth useful legal winner thank yellow"))
t.Cleanup(otherMnemonic.Destroy)
move := func(cli *Instance) error {
cli.Mnemonic = mnemonic
return cli.MoveSecret(cli.cmd, "work:test/secret", "other:moved", false)
}
t.Run("source removed", func(t *testing.T) {
requireChecksAgainOnceLocked(t, move, func(fs afero.Fs) error {
return fs.RemoveAll(testStateDir + "/vaults.d/work/secrets.d/test%secret")
}, vault.ErrSecretNotFound)
})
t.Run("destination vault removed", func(t *testing.T) {
requireChecksAgainOnceLocked(t, move, func(fs afero.Fs) error {
return fs.RemoveAll(testStateDir + "/vaults.d/other")
}, vault.ErrVaultNotFound)
})
t.Run("destination vault created again from another mnemonic", func(t *testing.T) {
// Set by the move, which has finished once
// requireChecksAgainOnceLocked returns
var moveErr error
requireChecksAgainOnceLocked(t, func(cli *Instance) error {
moveErr = move(cli)
return moveErr
}, func(fs afero.Fs) error {
err := fs.RemoveAll(testStateDir + "/vaults.d/other")
if err != nil {
return err
}
_, err = vault.CreateVault(fs, testStateDir, "other", otherMnemonic, nil)
return err
}, errVaultCreatedAgain)
// secret unlocker add fails with the same error, so its message
// must fit a move too
require.EqualError(t, moveErr, "vault 'other' was removed and "+
"created again with another key while it was being unlocked; "+
"nothing was changed")
})
}
// TestCrossVaultMoveRefusesExistingSecretBeforeUnlocking checks that a move
// between vaults onto a secret that exists, without --force, is refused
// before either vault is unlocked, since unlocking can ask for a
// passphrase. The vaults have no unlockers and the move is given no
// mnemonic, so unlocking either vault fails.
func TestCrossVaultMoveRefusesExistingSecretBeforeUnlocking(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
createCrossVaults(t, fs)
work := vault.NewVault(fs, testStateDir, "work")
work.Mnemonic = testMnemonicBuffer(t)
err := work.AddSecret("y",
memguard.NewBufferFromBytes([]byte("other")), false)
require.NoError(t, err)
instance := NewCLIInstanceWithStateDir(fs, testStateDir)
err = instance.MoveSecret(&cobra.Command{}, "default:x", "work:y", false)
require.ErrorIs(t, err, vault.ErrSecretExists)
}
+142
View File
@@ -138,6 +138,148 @@ func TestInitWritesNoBlankLineAfterMnemonic(t *testing.T) {
"stderr: %q", stderr.String()) "stderr: %q", stderr.String())
} }
// TestUnlockerAddAsksForNewPassphrase is a regression test for
// https://git.eeqj.de/sneak/secret/issues/154: with SB_UNLOCK_PASSPHRASE set,
// `secret unlocker add passphrase` gave the new unlocker the passphrase in
// the variable without asking. At a terminal it must ask for the new
// passphrase there, twice, and unlock the vault with the variable, so that
// `secret get` then opens the vault with the passphrase typed. pty.Open is
// described above TestRemoveIgnoresTerminalOnStdout.
func TestUnlockerAddAsksForNewPassphrase(t *testing.T) {
t.Parallel()
const newPassphrase = "new-passphrase"
stateDir := t.TempDir()
passphrase := memguard.NewBufferFromBytes([]byte(testPassphrase))
defer passphrase.Destroy()
vlt, err := vault.CreateVault(afero.NewOsFs(), stateDir, "default",
testMnemonicBuffer(t), passphrase)
require.NoError(t, err)
value := memguard.NewBufferFromBytes([]byte("v1"))
defer value.Destroy()
require.NoError(t, vlt.AddSecret("s1", value, false))
ctx, cancel := context.WithTimeout(t.Context(), commandWait)
defer cancel()
//nolint:gosec // G204: test executes the freshly built secret binary
cmd := exec.CommandContext(ctx, secretBinaryPath(t),
"unlocker", "add", "passphrase")
cmd.Env = []string{
secret.EnvStateDir + "=" + stateDir,
secret.EnvUnlockPassphrase + "=" + testPassphrase,
"PATH=" + os.Getenv("PATH"),
"HOME=" + os.Getenv("HOME"),
}
ptmx, tty, err := pty.Open()
require.NoError(t, err)
defer func() { _ = ptmx.Close() }()
deadline, _ := ctx.Deadline()
require.NoError(t, ptmx.SetReadDeadline(deadline))
cmd.Stderr = tty
// Ctty is a descriptor in the new process: 2, its stderr.
cmd.SysProcAttr = &syscall.SysProcAttr{Setsid: true, Setctty: true, Ctty: 2}
require.NoError(t, cmd.Start())
_ = tty.Close()
var (
shown []byte
char byte
)
terminal := bufio.NewReader(ptmx)
for _, prompt := range []string{
"Enter passphrase for unlocker: ", "Confirm passphrase: ",
} {
for !bytes.HasSuffix(shown, []byte(prompt)) {
char, err = terminal.ReadByte()
require.NoError(t, err,
"secret unlocker add passphrase did not ask on the terminal: %s",
shown)
shown = append(shown, char)
}
_, err = ptmx.WriteString(newPassphrase + "\n")
require.NoError(t, err)
}
// The read ends once the command has exited and so closed the terminal.
rest, _ := io.ReadAll(terminal)
require.NoError(t, cmd.Wait(),
"secret unlocker add passphrase: %s%s", shown, rest)
// The binary, not this test process, opens the vault: the new unlocker
// has age's full scrypt work factor, which is slow under the race
// detector.
assert.Equal(t, "v1",
runWithoutTerminal(t, stateDir, newPassphrase, "get", "s1"))
}
// TestUnlockerAddWithoutTerminalSaysPassphraseIsVariable is a regression
// test for https://git.eeqj.de/sneak/secret/issues/154: with no terminal to
// ask on, `secret unlocker add passphrase` gives the new unlocker the
// passphrase in SB_UNLOCK_PASSPHRASE, and its output must say so.
func TestUnlockerAddWithoutTerminalSaysPassphraseIsVariable(t *testing.T) {
t.Parallel()
stateDir := t.TempDir()
passphrase := memguard.NewBufferFromBytes([]byte(testPassphrase))
defer passphrase.Destroy()
_, err := vault.CreateVault(afero.NewOsFs(), stateDir, "default",
testMnemonicBuffer(t), passphrase)
require.NoError(t, err)
output := runWithoutTerminal(t, stateDir, testPassphrase,
"unlocker", "add", "passphrase")
assert.Contains(t, output, "The new unlocker takes its passphrase "+
"from SB_UNLOCK_PASSPHRASE, since there is no terminal to ask for a "+
"new one on\n")
}
// runWithoutTerminal runs the secret binary with args and SB_UNLOCK_PASSPHRASE
// set to passphrase, in a new session, which has no terminal, and returns
// what it printed once it has succeeded.
func runWithoutTerminal(
t *testing.T, stateDir, passphrase string, args ...string,
) string {
t.Helper()
ctx, cancel := context.WithTimeout(t.Context(), commandWait)
defer cancel()
//nolint:gosec // G204: test executes the freshly built secret binary
cmd := exec.CommandContext(ctx, secretBinaryPath(t), args...)
cmd.Env = []string{
secret.EnvStateDir + "=" + stateDir,
secret.EnvUnlockPassphrase + "=" + passphrase,
"PATH=" + os.Getenv("PATH"),
"HOME=" + os.Getenv("HOME"),
}
cmd.SysProcAttr = &syscall.SysProcAttr{Setsid: true}
output, err := cmd.CombinedOutput()
require.NoError(t, err, "secret %v: %s", args, output)
return string(output)
}
// runAtTerminal runs the secret binary with args and input on stdin, in a // runAtTerminal runs the secret binary with args and input on stdin, in a
// new session whose controlling terminal is a new terminal, which is also // new session whose controlling terminal is a new terminal, which is also
// its stderr: a command typed at a terminal with its input piped. pty.Open // its stderr: a command typed at a terminal with its input piped. pty.Open
+1 -5
View File
@@ -306,8 +306,7 @@ func TestInvalidVersionLeavesVaultsUnchanged(t *testing.T) {
// TestInvalidVaultNameLeavesStateUnchanged is a regression test for // TestInvalidVaultNameLeavesStateUnchanged is a regression test for
// https://git.eeqj.de/sneak/secret/issues/68, where // https://git.eeqj.de/sneak/secret/issues/68, where
// `secret vault import ..` wrote a long-term key and an unlocker into the // `secret vault select ..` made the state directory itself the current
// state directory itself, and `secret vault select ..` made it the current
// vault. Each command that takes a vault name must reject an invalid one // vault. Each command that takes a vault name must reject an invalid one
// before building a path from it. The instance is given the mnemonic and // before building a path from it. The instance is given the mnemonic and
// the passphrase, and moves and removals use --force, so that only the name // the passphrase, and moves and removals use --force, so that only the name
@@ -331,9 +330,6 @@ func TestInvalidVaultNameLeavesStateUnchanged(t *testing.T) {
{"vault create %q", func(c *cli.Instance, name string) error { {"vault create %q", func(c *cli.Instance, name string) error {
return c.CreateVault(cmd, name) return c.CreateVault(cmd, name)
}}, }},
{"vault import %q", func(c *cli.Instance, name string) error {
return c.VaultImport(cmd, name)
}},
{"vault select %q", func(c *cli.Instance, name string) error { {"vault select %q", func(c *cli.Instance, name string) error {
return c.SelectVault(cmd, name) return c.SelectVault(cmd, name)
}}, }},
+15
View File
@@ -14,6 +14,8 @@ import (
// every memguard buffer before it returns, so the caller must do nothing // every memguard buffer before it returns, so the caller must do nothing
// but exit with the code. // but exit with the code.
func Entry() int { func Entry() int {
raiseLockedMemoryLimit()
// On SIGINT or SIGTERM memguard runs this function, wipes every buffer // On SIGINT or SIGTERM memguard runs this function, wipes every buffer
// and exits with status 1. The passphrase prompt turns echo off on the // and exits with status 1. The passphrase prompt turns echo off on the
// controlling terminal until the read finishes, so a signal there would // controlling terminal until the read finishes, so a signal there would
@@ -51,6 +53,19 @@ func Entry() int {
return 0 return 0
} }
// raiseLockedMemoryLimit raises this process's limit on locked memory, which
// holds every secret value, to the hard limit, the most a process may set
// without privileges. When that fails the limit stays as it was, and
// checkLockedMemoryLimit names it in its error.
func raiseLockedMemoryLimit() {
var limit unix.Rlimit
if unix.Getrlimit(unix.RLIMIT_MEMLOCK, &limit) == nil {
limit.Cur = limit.Max
_ = unix.Setrlimit(unix.RLIMIT_MEMLOCK, &limit)
}
}
func newRootCmd() *cobra.Command { func newRootCmd() *cobra.Command {
secret.Debug("newRootCmd starting") secret.Debug("newRootCmd starting")
+360 -176
View File
@@ -14,6 +14,7 @@ import (
"github.com/awnumar/memguard" "github.com/awnumar/memguard"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"golang.org/x/sys/unix"
"sneak.berlin/go/secret/internal/secret" "sneak.berlin/go/secret/internal/secret"
"sneak.berlin/go/secret/internal/vault" "sneak.berlin/go/secret/internal/vault"
) )
@@ -24,26 +25,30 @@ const (
// vaultSecretParts is the number of parts when splitting vault:secret // vaultSecretParts is the number of parts when splitting vault:secret
vaultSecretParts = 2 vaultSecretParts = 2
// initialBufferSize is the starting size for secret read buffers (4KB)
initialBufferSize = 4 * 1024
// maxSecretSize is the maximum allowed size of a secret (100MB) // maxSecretSize is the maximum allowed size of a secret (100MB)
maxSecretSize = 100 * 1024 * 1024 maxSecretSize = 100 * 1024 * 1024
// lockedMemoryReservePages is the locked memory, in pages, that `secret
// add`, `secret import` and `secret generate secret` may need besides
// the value they store, for keys and the read buffer: memguard locks
// whole pages, at least one for each buffer. `secret add` was measured
// to need 7 with both SB_SECRET_MNEMONIC and SB_UNLOCK_PASSPHRASE set,
// and 6 with one; the eighth is to spare.
lockedMemoryReservePages = 8
// kibibyte is the unit of the locked-memory limit in `ulimit -l`.
kibibyte = 1024
) )
// Sentinel errors for secret operations // Sentinel errors for secret operations
var ( var (
errSecretTooLarge = errors.New("secret too large: exceeds 100MB limit") errSecretTooLarge = errors.New("secret too large: exceeds 100MB limit")
errLockedMemoryLimit = errors.New(
"secret too large for the locked-memory limit")
errCrossVaultSourceUnqualified = errors.New( errCrossVaultSourceUnqualified = errors.New(
"source must specify vault (e.g., vault:secret) for cross-vault move") "source must specify vault (e.g., vault:secret) for cross-vault move")
errMoveOntoItself = errors.New("cannot be moved onto itself") errMoveOntoItself = errors.New("cannot be moved onto itself")
) )
// bufferInfo tracks a protected buffer and the number of bytes used in it
type bufferInfo struct {
buffer *memguard.LockedBuffer
used int
}
// ParseVaultSecretRef parses a "vault:secret" or just "secret" reference // ParseVaultSecretRef parses a "vault:secret" or just "secret" reference
// Returns (vaultName, secretName, isQualified) // Returns (vaultName, secretName, isQualified)
// If no vault is specified, returns empty vaultName and isQualified=false // If no vault is specified, returns empty vaultName and isQualified=false
@@ -75,6 +80,12 @@ func newAddCmd() *cobra.Command {
cli.cmd = cmd // Set the command for stdin access cli.cmd = cmd // Set the command for stdin access
// Before readSecretEnv, which makes the first locked allocation
err = checkLockedMemoryLimit(0)
if err != nil {
return err
}
destroySecrets := cli.readSecretEnv() destroySecrets := cli.readSecretEnv()
defer destroySecrets() defer destroySecrets()
@@ -174,6 +185,12 @@ func newImportCmd() *cobra.Command {
return fmt.Errorf("failed to initialize CLI: %w", err) return fmt.Errorf("failed to initialize CLI: %w", err)
} }
// Before readSecretEnv, which makes the first locked allocation
err = checkLockedMemoryLimit(0)
if err != nil {
return err
}
destroySecrets := cli.readSecretEnv() destroySecrets := cli.readSecretEnv()
defer destroySecrets() defer destroySecrets()
@@ -270,88 +287,105 @@ The source secret is deleted after successful copy.`,
return cmd return cmd
} }
// updateBufferSize updates the buffer size based on usage pattern // checkLockedMemoryLimit returns an error naming the locked-memory limit
func updateBufferSize(currentSize int, sameSize *int) int { // (`ulimit -l`) when a value of size bytes does not fit in it with
const ( // lockedMemoryReservePages. memguard, which holds secret values in locked
doubleAfterBuffers = 2 // memory, crashes or hangs the process when it cannot lock more, so a
growthFactor = 2 // command calls this with size 0 before its first locked allocation, and
) // again with the value's size before a buffer for the value is allocated.
func checkLockedMemoryLimit(size int) error {
var limit unix.Rlimit
*sameSize++ err := unix.Getrlimit(unix.RLIMIT_MEMLOCK, &limit)
if *sameSize >= doubleAfterBuffers { if err != nil {
*sameSize = 0 return fmt.Errorf("failed to read the locked-memory limit: %w", err)
return currentSize * growthFactor
} }
return currentSize // memguard locks whole pages, so the value is counted in whole pages.
} pageSize := os.Getpagesize()
valuePages := (size + pageSize - 1) / pageSize
// destroyBuffers destroys every buffer in the list //nolint:gosec // G115: a page count and a page size are never negative
func destroyBuffers(buffers []bufferInfo) { needed := uint64((valuePages + lockedMemoryReservePages) * pageSize)
for _, b := range buffers { if needed > limit.Cur {
b.buffer.Destroy() return fmt.Errorf("%w of %d KiB (ulimit -l)",
errLockedMemoryLimit, limit.Cur/kibibyte)
} }
return nil
} }
// readSecretFromReader reads all data from reader into protected buffers, // readSecretFromReader reads all of reader into a new locked buffer, which
// enforcing the maximum secret size. On failure the accumulated buffers // the caller must destroy, leaving out one trailing newline when trimNewline
// are destroyed; on success the caller must destroy them. // is set. It reads a page at a time into a memguard.Stream, which keeps what
func readSecretFromReader(reader io.Reader) ([]bufferInfo, int, error) { // it is given encrypted, so that the value is in locked memory only once, in
var buffers []bufferInfo // the buffer returned, and both size checks come before that buffer is
// allocated. It takes the value back out with Stream.Next, not Stream.Read:
// Read puts back the part of a piece that does not fit, and memguard hangs
// when it runs out of locked memory doing that.
func readSecretFromReader(
reader io.Reader, trimNewline bool,
) (*memguard.LockedBuffer, error) {
stream := memguard.NewStream()
totalSize := 0 chunk := memguard.NewBuffer(os.Getpagesize())
currentBufferSize := initialBufferSize defer chunk.Destroy()
sameSize := 0
size := 0
endsWithNewline := false
for { for {
// Create a new buffer n, err := io.ReadFull(reader, chunk.Bytes())
buffer := memguard.NewBuffer(currentBufferSize) if n > 0 {
n, err := io.ReadFull(reader, buffer.Bytes()) size += n
if size > maxSecretSize {
if n == 0 { return nil, errSecretTooLarge
// No data read, destroy the unused buffer
buffer.Destroy()
} else {
buffers = append(buffers, bufferInfo{buffer: buffer, used: n})
totalSize += n
if totalSize > maxSecretSize {
destroyBuffers(buffers)
return nil, 0, errSecretTooLarge
} }
// If we filled the buffer, consider growing for next iteration endsWithNewline = chunk.Bytes()[n-1] == '\n'
if n == currentBufferSize {
currentBufferSize = updateBufferSize(currentBufferSize, &sameSize) // Write encrypts the bytes and wipes them from chunk.
} _, _ = stream.Write(chunk.Bytes()[:n])
} }
if err == io.EOF || errors.Is(err, io.ErrUnexpectedEOF) { if errors.Is(err, io.EOF) || errors.Is(err, io.ErrUnexpectedEOF) {
break break
} else if err != nil { }
destroyBuffers(buffers)
return nil, 0, err if err != nil {
return nil, err
} }
} }
return buffers, totalSize, nil // Freed before the value's buffer is allocated, so that the two are
} // never locked at once.
chunk.Destroy()
// combineBuffers copies the used portions of buffers into a single if trimNewline && endsWithNewline {
// protected buffer of totalSize bytes size--
func combineBuffers(buffers []bufferInfo, totalSize int) *memguard.LockedBuffer {
valueBuffer := memguard.NewBuffer(totalSize)
offset := 0
for _, b := range buffers {
copy(valueBuffer.Bytes()[offset:], b.buffer.Bytes()[:b.used])
offset += b.used
} }
return valueBuffer err := checkLockedMemoryLimit(size)
if err != nil {
return nil, err
}
value := memguard.NewBuffer(size)
// copy stops at the value's size, so a trimmed newline is not copied.
for offset := 0; offset < size; {
decrypted, err := stream.Next()
if err != nil {
value.Destroy()
return nil, fmt.Errorf("failed to decrypt the value read: %w", err)
}
offset += copy(value.Bytes()[offset:], decrypted.Bytes())
decrypted.Destroy()
}
return value, nil
} }
// AddSecret adds a secret to the current vault // AddSecret adds a secret to the current vault
@@ -370,58 +404,110 @@ func (cli *Instance) AddSecret(secretName string, force bool) error {
secret.Debug("Got current vault", "vault_name", vlt.GetName()) secret.Debug("Got current vault", "vault_name", vlt.GetName())
// Read secret value directly into protected buffers // Read the secret value, less a trailing newline, into locked memory
secret.Debug("Reading secret value from stdin into protected buffers") secret.Debug("Reading secret value from stdin into a protected buffer")
buffers, totalSize, err := readSecretFromReader(cli.cmd.InOrStdin()) valueBuffer, err := readSecretFromReader(cli.cmd.InOrStdin(), true)
if err != nil { if err != nil {
if errors.Is(err, errSecretTooLarge) { if errors.Is(err, errSecretTooLarge) || errors.Is(err, errLockedMemoryLimit) {
return err return err
} }
return fmt.Errorf("failed to read secret value: %w", err) return fmt.Errorf("failed to read secret value: %w", err)
} }
defer destroyBuffers(buffers)
// Check for trailing newline in the last buffer
if len(buffers) > 0 && totalSize > 0 {
lastBuffer := &buffers[len(buffers)-1]
if lastBuffer.buffer.Bytes()[lastBuffer.used-1] == '\n' {
lastBuffer.used--
totalSize--
}
}
secret.Debug("Read secret value from stdin",
"value_length", totalSize, "buffers", len(buffers))
// Combine all buffers into a single protected buffer
valueBuffer := combineBuffers(buffers, totalSize)
defer valueBuffer.Destroy() defer valueBuffer.Destroy()
secret.Debug("Read secret value from stdin", "value_length", valueBuffer.Size())
// Locked only now that stdin has been read: in `secret encrypt key | // Locked only now that stdin has been read: in `secret encrypt key |
// secret add name`, holding the lock while reading would leave each // secret add name`, holding the lock while reading would leave each
// command waiting for the other. // command waiting for the other.
release, err := vault.LockStateDir(cli.fs, cli.stateDir) return cli.storeSecret(vlt, secretName, valueBuffer, force)
}
// storeSecret adds the secret name with value to vlt, as vault.AddSecret
// does, holding the state directory lock while it does. With force, adding
// an existing secret replaces its current version, which needs the vault's
// long-term key. Getting the key can ask for the passphrase, so storeSecret
// gets it before it takes the lock, and no other command waits while the
// user answers. If another command adds the secret while this one waits for
// the lock, storeSecret lets go of the lock, gets the key, and takes the
// lock again.
func (cli *Instance) storeSecret(
vlt *vault.Vault, name string, value *memguard.LockedBuffer, force bool,
) error {
err := vault.ValidateSecretName(name)
if err != nil { if err != nil {
return err return err
} }
replacing, err := replacesVersion(vlt, name, force)
if err != nil {
return err
}
if replacing {
_, err = vlt.GetOrDeriveLongTermKey()
if err != nil {
return fmt.Errorf("failed to get long-term key: %w", err)
}
}
needsKey, err := cli.addWithoutAsking(vlt, name, value, force)
if !needsKey {
return err
}
_, err = vlt.GetOrDeriveLongTermKey()
if err != nil {
return fmt.Errorf("failed to get long-term key: %w", err)
}
// With the key in memory, this adds the secret
_, err = cli.addWithoutAsking(vlt, name, value, force)
return err
}
// addWithoutAsking takes the state directory lock and adds the secret name
// with value to vlt, as vault.AddSecret does. If that would replace the
// secret's current version while the vault's long-term key is not in
// memory, getting the key could ask for the passphrase with the lock held:
// addWithoutAsking then adds nothing and returns true.
func (cli *Instance) addWithoutAsking(
vlt *vault.Vault, name string, value *memguard.LockedBuffer, force bool,
) (bool, error) {
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return false, err
}
defer release() defer release()
// Add the secret to the vault replacing, err := replacesVersion(vlt, name, force)
secret.Debug("Calling vault.AddSecret", "secret_name", secretName,
"value_length", valueBuffer.Size(), "force", force)
err = vlt.AddSecret(secretName, valueBuffer, force)
if err != nil { if err != nil {
secret.Debug("vault.AddSecret failed", "error", err) return false, err
return err
} }
secret.Debug("vault.AddSecret completed successfully") if replacing && vlt.Locked() {
return true, nil
}
return nil return false, vlt.AddSecret(name, value, force)
}
// replacesVersion reports whether adding the secret name to vlt replaces
// the secret's current version: whether force is set and the secret has one.
func replacesVersion(vlt *vault.Vault, name string, force bool) (bool, error) {
if !force {
return false, nil
}
exists, err := secret.NewSecret(vlt, name).Exists()
if err != nil {
return false, fmt.Errorf("failed to check if secret exists: %w", err)
}
return exists, nil
} }
// GetSecret retrieves and prints the current version of a secret // GetSecret retrieves and prints the current version of a secret
@@ -443,7 +529,10 @@ func (cli *Instance) GetSecret(cmd *cobra.Command, secretName string) error {
defer value.Destroy() defer value.Destroy()
// Write the value straight from locked memory, with no trailing newline // Write the value straight from locked memory, with no trailing newline
_, _ = cmd.OutOrStdout().Write(value.Bytes()) _, err = cmd.OutOrStdout().Write(value.Bytes())
if err != nil {
return fmt.Errorf("failed to write secret value: %w", err)
}
return nil return nil
} }
@@ -478,7 +567,10 @@ func (cli *Instance) GetSecretWithVersion(
secret.Debug("Got secret value", "valueLength", value.Size()) secret.Debug("Got secret value", "valueLength", value.Size())
// Write the value straight from locked memory, with no trailing newline // Write the value straight from locked memory, with no trailing newline
_, _ = cmd.OutOrStdout().Write(value.Bytes()) _, err = cmd.OutOrStdout().Write(value.Bytes())
if err != nil {
return fmt.Errorf("failed to write secret value: %w", err)
}
secret.Debug("Printed value to stdout") secret.Debug("Printed value to stdout")
@@ -541,8 +633,12 @@ func printSecretsJSON(
"name": secretName, "name": secretName,
} }
// Try to get metadata using GetSecretObject // A secret whose times cannot be read is listed without them
secretObj, err := vlt.GetSecretObject(secretName) secretObj, err := vlt.GetSecretObject(secretName)
if err == nil {
err = secretObj.LoadMetadata()
}
if err == nil { if err == nil {
metadata := secretObj.GetMetadata() metadata := secretObj.GetMetadata()
secretInfo["created_at"] = metadata.CreatedAt secretInfo["created_at"] = metadata.CreatedAt
@@ -618,6 +714,10 @@ func printSecretsTable(
lastUpdated := "unknown" lastUpdated := "unknown"
secretObj, err := vlt.GetSecretObject(secretName) secretObj, err := vlt.GetSecretObject(secretName)
if err == nil {
err = secretObj.LoadMetadata()
}
if err == nil { if err == nil {
metadata := secretObj.GetMetadata() metadata := secretObj.GetMetadata()
lastUpdated = metadata.UpdatedAt.Format("2006-01-02 15:04") lastUpdated = metadata.UpdatedAt.Format("2006-01-02 15:04")
@@ -653,7 +753,7 @@ func (cli *Instance) ImportSecret(
vlt.Mnemonic, vlt.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase vlt.Mnemonic, vlt.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase
// Read secret value from the source file into protected buffers // Read secret value from the source file into a protected buffer
file, err := cli.fs.Open(sourceFile) file, err := cli.fs.Open(sourceFile)
if err != nil { if err != nil {
return fmt.Errorf("failed to open file %s: %w", sourceFile, err) return fmt.Errorf("failed to open file %s: %w", sourceFile, err)
@@ -665,26 +765,15 @@ func (cli *Instance) ImportSecret(
} }
}() }()
buffers, totalSize, err := readSecretFromReader(file) valueBuffer, err := readSecretFromReader(file, false)
if err != nil { if err != nil {
return fmt.Errorf("failed to read secret from file %s: %w", sourceFile, err) return fmt.Errorf("failed to read secret from file %s: %w", sourceFile, err)
} }
defer destroyBuffers(buffers)
// Combine all buffers into a single protected buffer
valueBuffer := combineBuffers(buffers, totalSize)
defer valueBuffer.Destroy() defer valueBuffer.Destroy()
// Locked only now that the file has been read, as in AddSecret: the // Locked only now that the file has been read, as in AddSecret: the
// file may be a pipe written by another secret command. // file may be a pipe written by another secret command.
release, err := vault.LockStateDir(cli.fs, cli.stateDir) err = cli.storeSecret(vlt, secretName, valueBuffer, force)
if err != nil {
return err
}
defer release()
// Store the secret in the vault
err = vlt.AddSecret(secretName, valueBuffer, force)
if err != nil { if err != nil {
return err return err
} }
@@ -786,24 +875,12 @@ func (cli *Instance) findSecretToRemove(
}, nil }, nil
} }
// MoveSecret moves or renames a secret (within or across vaults), holding // MoveSecret moves or renames a secret (within or across vaults). It
// the state directory lock. // leaves taking the state directory lock to moveSecretWithinVault and
// moveSecretCrossVault, since a move between vaults unlocks both vaults
// before it takes the lock.
func (cli *Instance) MoveSecret( func (cli *Instance) MoveSecret(
cmd *cobra.Command, source, dest string, force bool, cmd *cobra.Command, source, dest string, force bool,
) error {
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
return cli.moveSecret(cmd, source, dest, force)
}
// moveSecret does the work of MoveSecret. The caller holds the state
// directory lock.
func (cli *Instance) moveSecret(
cmd *cobra.Command, source, dest string, force bool,
) error { ) error {
// Parse source and destination // Parse source and destination
srcVaultName, srcSecretName, srcQualified := ParseVaultSecretRef(source) srcVaultName, srcSecretName, srcQualified := ParseVaultSecretRef(source)
@@ -903,8 +980,9 @@ func (cli *Instance) existingVault(name string) (*vault.Vault, error) {
return vault.NewVault(cli.fs, cli.stateDir, name), nil return vault.NewVault(cli.fs, cli.stateDir, name), nil
} }
// moveSecretWithinVault renames a secret within the vault vlt. Its caller, // moveSecretWithinVault renames a secret within the vault vlt, holding the
// MoveSecret, has already checked both secret names. // state directory lock. Its caller, MoveSecret, has already checked both
// secret names.
func (cli *Instance) moveSecretWithinVault( func (cli *Instance) moveSecretWithinVault(
cmd *cobra.Command, vlt *vault.Vault, source, dest string, force bool, cmd *cobra.Command, vlt *vault.Vault, source, dest string, force bool,
) error { ) error {
@@ -914,6 +992,12 @@ func (cli *Instance) moveSecretWithinVault(
return fmt.Errorf("secret '%s' %w", source, errMoveOntoItself) return fmt.Errorf("secret '%s' %w", source, errMoveOntoItself)
} }
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
vaultDir, err := vlt.GetDirectory() vaultDir, err := vlt.GetDirectory()
if err != nil { if err != nil {
return err return err
@@ -1000,59 +1084,75 @@ func (cli *Instance) sameDirectory(dir, other string) (bool, error) {
// moveSecretCrossVault handles moving between two different vaults. Its // moveSecretCrossVault handles moving between two different vaults. Its
// caller, MoveSecret, has already checked both secret names and that both // caller, MoveSecret, has already checked both secret names and that both
// vaults exist. // vaults exist. The copy needs the source vault's long-term key and is
// encrypted to the destination vault's long-term public key. The
// destination vault is unlocked to check that the copy will open with what
// the user gave, and, once the state directory lock is held, that it still
// has the key unlocked. Either unlock can ask for a passphrase, so both
// happen before the lock is taken, and no other command waits while the
// user answers.
func (cli *Instance) moveSecretCrossVault( func (cli *Instance) moveSecretCrossVault(
cmd *cobra.Command, cmd *cobra.Command,
srcVault *vault.Vault, srcSecretName string, srcVault *vault.Vault, srcSecretName string,
destVault *vault.Vault, destSecretName string, destVault *vault.Vault, destSecretName string,
force bool, force bool,
) error { ) error {
srcVaultDir, err := srcVault.GetDirectory() // Before unlocking, check that the source secret exists, that the
if err != nil { // destination is not the source under another name, and, without
return fmt.Errorf("failed to get source vault directory: %w", err) // --force, that the destination secret does not exist
} _, err := cli.checkCrossVaultMove(
srcVault, srcSecretName, destVault, destSecretName, force)
// Verify source secret exists
srcStorageName := strings.ReplaceAll(srcSecretName, "/", "%")
srcSecretDir := filepath.Join(srcVaultDir, "secrets.d", srcStorageName)
exists, err := afero.DirExists(cli.fs, srcSecretDir)
if err != nil || !exists {
return fmt.Errorf("secret '%s' %w in vault '%s'",
srcSecretName, vault.ErrSecretNotFound, srcVault.Name)
}
// The source is removed after the copy, so a destination that is the
// source under another name would be lost with it.
destVaultDir, err := destVault.GetDirectory()
if err != nil {
return fmt.Errorf("failed to get destination vault directory: %w", err)
}
destStorageName := strings.ReplaceAll(destSecretName, "/", "%")
destSecretDir := filepath.Join(destVaultDir, "secrets.d", destStorageName)
same, err := cli.sameDirectory(srcSecretDir, destSecretDir)
if err != nil { if err != nil {
return err return err
} }
if same {
return fmt.Errorf("secret '%s:%s' %w: '%s:%s' is the same secret on "+
"this filesystem", srcVault.Name, srcSecretName, errMoveOntoItself,
destVault.Name, destSecretName)
}
// Copying needs the long-term keys of both vaults
srcVault.Mnemonic, srcVault.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase srcVault.Mnemonic, srcVault.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase
destVault.Mnemonic, destVault.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase destVault.Mnemonic, destVault.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase
// Unlock destination vault (will fail if neither mnemonic nor unlocker available) // Unlock destination vault (will fail if neither mnemonic nor unlocker available)
_, err = destVault.GetOrDeriveLongTermKey() destIdentity, err := destVault.GetOrDeriveLongTermKey()
if err != nil { if err != nil {
return fmt.Errorf("failed to unlock destination vault '%s': %w", destVault.Name, err) return fmt.Errorf("failed to unlock destination vault '%s': %w", destVault.Name, err)
} }
_, err = srcVault.GetOrDeriveLongTermKey()
if err != nil {
return fmt.Errorf("failed to unlock source vault '%s': %w",
srcVault.Name, err)
}
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
err = cli.checkLongTermKey(destVault, destIdentity)
if err != nil {
return err
}
return cli.copySecretAndRemoveSource(
cmd, srcVault, srcSecretName, destVault, destSecretName, force)
}
// copySecretAndRemoveSource copies the secret srcSecretName of srcVault,
// with all its versions, to destSecretName in destVault, and removes the
// source. moveSecretCrossVault calls it holding the state directory lock.
// Another command may have changed either vault since moveSecretCrossVault
// checked them, so it checks them again first.
func (cli *Instance) copySecretAndRemoveSource(
cmd *cobra.Command,
srcVault *vault.Vault, srcSecretName string,
destVault *vault.Vault, destSecretName string,
force bool,
) error {
srcSecretDir, err := cli.checkCrossVaultMove(
srcVault, srcSecretName, destVault, destSecretName, force)
if err != nil {
return err
}
// Count versions for user feedback // Count versions for user feedback
versions, _ := secret.ListVersions(cli.fs, srcSecretDir) versions, _ := secret.ListVersions(cli.fs, srcSecretDir)
versionCount := len(versions) versionCount := len(versions)
@@ -1067,12 +1167,9 @@ func (cli *Instance) moveSecretCrossVault(
// Delete source secret // Delete source secret
err = secret.RemoveDirAtomic(cli.fs, srcSecretDir) err = secret.RemoveDirAtomic(cli.fs, srcSecretDir)
if err != nil { if err != nil {
// Copy succeeded but delete failed - warn but don't fail return cli.sourceRemovalError(err, srcSecretDir,
cmd.Printf("Warning: copied secret but failed to remove source: %v\n", err) srcVault.Name+":"+srcSecretName, destVault.Name+":"+destSecretName,
cmd.Printf("Moved secret '%s:%s' to '%s:%s' (%d version(s))\n", versionCount)
srcVault.Name, srcSecretName, destVault.Name, destSecretName, versionCount)
return nil
} }
cmd.Printf("Moved secret '%s:%s' to '%s:%s' (%d version(s))\n", cmd.Printf("Moved secret '%s:%s' to '%s:%s' (%d version(s))\n",
@@ -1080,3 +1177,90 @@ func (cli *Instance) moveSecretCrossVault(
return nil return nil
} }
// checkCrossVaultMove checks that the secret srcSecretName exists in
// srcVault, that destSecretName in destVault is not the source under
// another name, and, unless force is set, that destSecretName does not
// exist in destVault. It returns the source's directory.
func (cli *Instance) checkCrossVaultMove(
srcVault *vault.Vault, srcSecretName string,
destVault *vault.Vault, destSecretName string,
force bool,
) (string, error) {
srcVaultDir, err := srcVault.GetDirectory()
if err != nil {
return "", fmt.Errorf("failed to get source vault directory: %w", err)
}
// Verify source secret exists
srcStorageName := strings.ReplaceAll(srcSecretName, "/", "%")
srcSecretDir := filepath.Join(srcVaultDir, "secrets.d", srcStorageName)
exists, err := afero.DirExists(cli.fs, srcSecretDir)
if err != nil || !exists {
return "", fmt.Errorf("secret '%s' %w in vault '%s'",
srcSecretName, vault.ErrSecretNotFound, srcVault.Name)
}
// The source is removed after the copy, so a destination that is the
// source under another name would be lost with it.
destVaultDir, err := destVault.GetDirectory()
if err != nil {
return "", fmt.Errorf("failed to get destination vault directory: %w", err)
}
destStorageName := strings.ReplaceAll(destSecretName, "/", "%")
destSecretDir := filepath.Join(destVaultDir, "secrets.d", destStorageName)
same, err := cli.sameDirectory(srcSecretDir, destSecretDir)
if err != nil {
return "", err
}
if same {
return "", fmt.Errorf("secret '%s:%s' %w: '%s:%s' is the same secret "+
"on this filesystem", srcVault.Name, srcSecretName, errMoveOntoItself,
destVault.Name, destSecretName)
}
exists, err = afero.DirExists(cli.fs, destSecretDir)
if err != nil {
return "", fmt.Errorf("failed to check if destination secret exists: %w", err)
}
if exists && !force {
return "", fmt.Errorf(
"secret '%s' %w in vault '%s' (use --force to overwrite)",
destSecretName, vault.ErrSecretExists, destVault.Name)
}
return srcSecretDir, nil
}
// sourceRemovalError returns the error of a move between vaults that
// copied all versionCount versions of the secret srcName to destName, and
// then failed with removeErr to remove srcSecretDir, the directory of
// srcName. The removal can fail after the source has left its vault, when
// only the temporary directory it was moved into cannot be deleted, so the
// error says the source remains only if srcSecretDir is still there.
func (cli *Instance) sourceRemovalError(
removeErr error, srcSecretDir, srcName, destName string, versionCount int,
) error {
remains, err := afero.DirExists(cli.fs, srcSecretDir)
switch {
case err != nil:
return fmt.Errorf("secret '%s' is copied with all %d version(s) "+
"to '%s', but removing the source failed and whether it "+
"remains is unknown: %w",
srcName, versionCount, destName, errors.Join(removeErr, err))
case remains:
return fmt.Errorf("secret '%s' is copied with all %d version(s) "+
"to '%s', but the source remains: %w",
srcName, versionCount, destName, removeErr)
default:
return fmt.Errorf("secret '%s' is moved with all %d version(s) "+
"to '%s', but its old files could not all be deleted: %w",
srcName, versionCount, destName, removeErr)
}
}
+154 -14
View File
@@ -3,17 +3,23 @@ package cli
import ( import (
"bytes" "bytes"
"context"
"crypto/rand" "crypto/rand"
"fmt" "fmt"
"io" "io"
"os"
"os/exec"
"path/filepath" "path/filepath"
"strconv"
"strings" "strings"
"testing" "testing"
"time"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/secret/internal/secret"
"sneak.berlin/go/secret/internal/vault" "sneak.berlin/go/secret/internal/vault"
"sneak.berlin/go/secret/pkg/agehd" "sneak.berlin/go/secret/pkg/agehd"
) )
@@ -30,17 +36,11 @@ func newSizeTestVault(t *testing.T) (afero.Fs, *vault.Vault) {
fs := afero.NewMemMapFs() fs := afero.NewMemMapFs()
// Create vault // Create vault, which selects it as the current vault
_, err := vault.CreateVault(fs, testStateDir, testVaultName, _, err := vault.CreateVault(fs, testStateDir, testVaultName,
testMnemonicBuffer(t), nil) testMnemonicBuffer(t), nil)
require.NoError(t, err) require.NoError(t, err)
// Set current vault
currentVaultPath := filepath.Join(testStateDir, "currentvault")
vaultPath := filepath.Join(testStateDir, "vaults.d", testVaultName)
err = afero.WriteFile(fs, currentVaultPath, []byte(vaultPath), 0o600)
require.NoError(t, err)
// Get vault and set up long-term key // Get vault and set up long-term key
vlt, err := vault.GetCurrentVault(fs, testStateDir) vlt, err := vault.GetCurrentVault(fs, testStateDir)
require.NoError(t, err) require.NoError(t, err)
@@ -208,23 +208,25 @@ func TestImportSecretVariousSizes(t *testing.T) {
} }
} }
// TestAddSecretBufferGrowth tests that our buffer growth strategy works correctly // TestAddSecretAroundPageSize tests that secret add, which reads its input
// one memory page (usually 4 KiB) at a time, stores values just under, at and
// just over one and two pages unchanged.
// //
//nolint:paralleltest // in parallel, size tests could exceed the memlock limit //nolint:paralleltest // in parallel, size tests could exceed the memlock limit
func TestAddSecretBufferGrowth(t *testing.T) { func TestAddSecretAroundPageSize(t *testing.T) {
// Test various sizes that should trigger buffer growth // Sizes around one and two pages of 4 KiB, then larger ones up to 1 MiB
sizes := []int{ sizes := []int{
1, // Single byte 1, // Single byte
100, // Small 100, // Small
4095, // Just under initial 4KB 4095, // Just under 4KB
4096, // Exactly 4KB 4096, // Exactly 4KB
4097, // Just over 4KB 4097, // Just over 4KB
8191, // Just under 8KB (first double) 8191, // Just under 8KB
8192, // Exactly 8KB 8192, // Exactly 8KB
8193, // Just over 8KB 8193, // Just over 8KB
12288, // 12KB (should trigger second double) 12288, // 12KB
16384, // 16KB 16384, // 16KB
32768, // 32KB (after more doublings) 32768, // 32KB
65536, // 64KB 65536, // 64KB
131072, // 128KB 131072, // 128KB
524288, // 512KB 524288, // 512KB
@@ -315,6 +317,144 @@ func TestAddSecretStreamingBehavior(t *testing.T) {
"Retrieved secret should match original") "Retrieved secret should match original")
} }
// newBinaryTestStateDir returns a new state directory holding a vault made
// from the test mnemonic, for the secret binary to use.
func newBinaryTestStateDir(t *testing.T) string {
t.Helper()
stateDir := t.TempDir()
_, err := vault.CreateVault(afero.NewOsFs(), stateDir, "default",
testMnemonicBuffer(t), nil)
require.NoError(t, err)
return stateDir
}
// runAtLockedMemoryLimit runs the secret binary with args, stdin, the state
// directory stateDir and the test mnemonic, its locked-memory limit (ulimit
// -l) lowered to limitKiB, and returns what it printed.
func runAtLockedMemoryLimit(
t *testing.T, stateDir string, limitKiB int, stdin io.Reader,
args ...string,
) (string, error) {
t.Helper()
const commandWait = 10 * time.Second
secretPath, err := filepath.Abs("../../secret")
require.NoError(t, err)
ctx, cancel := context.WithTimeout(t.Context(), commandWait)
defer cancel()
// The shell lowers the limit for itself and the binary only.
script := fmt.Sprintf(`ulimit -l %d && exec "$0" "$@"`, limitKiB)
//nolint:gosec // G204: runs the freshly built secret binary
cmd := exec.CommandContext(ctx, "sh",
append([]string{"-c", script, secretPath}, args...)...)
cmd.Env = []string{
secret.EnvStateDir + "=" + stateDir,
secret.EnvMnemonic + "=" + testMnemonic,
}
cmd.Stdin = stdin
output, err := cmd.CombinedOutput()
return string(output), err
}
// requireRefusedAtLockedMemoryLimit runs the secret binary as
// runAtLockedMemoryLimit does, and requires it to exit with status 1 and
// print wantError, with no panic.
func requireRefusedAtLockedMemoryLimit(
t *testing.T, stateDir string, limitKiB int, wantError string,
stdin io.Reader, args ...string,
) {
t.Helper()
output, err := runAtLockedMemoryLimit(t, stateDir, limitKiB, stdin, args...)
var exitErr *exec.ExitError
require.ErrorAs(t, err, &exitErr, "%v: %s", args, output)
assert.Equal(t, 1, exitErr.ExitCode(), "%v: %s", args, output)
assert.Contains(t, output, wantError, args)
assert.NotContains(t, output, "panic", args)
}
// TestSecretsAtLockedMemoryLimit runs the secret binary with the
// locked-memory limit (ulimit -l) lowered to 1 MiB, where memguard would crash
// the process with a panic instead of returning an error. add must store the
// largest value that fits, which takes holding it in locked memory only once.
// add, import and generate secret must refuse a value one byte larger with
// exit status 1 and an error saying why.
func TestSecretsAtLockedMemoryLimit(t *testing.T) {
t.Parallel()
const limitKiB = 1024
fits := limitKiB*kibibyte - lockedMemoryReservePages*os.Getpagesize()
limitError := fmt.Sprintf("locked-memory limit of %d KiB", limitKiB)
stateDir := newBinaryTestStateDir(t)
tooBigFile := filepath.Join(t.TempDir(), "too-big")
require.NoError(t, os.WriteFile(tooBigFile, make([]byte, fits+1), 0o600))
output, err := runAtLockedMemoryLimit(t, stateDir, limitKiB,
bytes.NewReader(make([]byte, fits)), "add", "fits")
require.NoError(t, err, "the largest value that fits: %s", output)
requireRefusedAtLockedMemoryLimit(t, stateDir, limitKiB, limitError,
bytes.NewReader(make([]byte, fits+1)), "add", "too-big")
requireRefusedAtLockedMemoryLimit(t, stateDir, limitKiB, limitError,
nil, "import", "too-big", "--source", tooBigFile)
requireRefusedAtLockedMemoryLimit(t, stateDir, limitKiB, limitError,
nil, "generate", "secret", "too-big", "--length", strconv.Itoa(fits+1))
}
// TestSecretsUnderSmallLockedMemoryLimits runs add, import and generate
// secret with a short value under two small locked-memory limits (ulimit
// -l). Under 20 KiB, too little for the buffers they need besides the value,
// each must refuse it with exit status 1 and an error naming the limit, where
// memguard would panic or hang. Under 64 KiB, the default of older Linux
// kernels, each must store it.
func TestSecretsUnderSmallLockedMemoryLimits(t *testing.T) {
t.Parallel()
const (
tooSmallLimitKiB = 20
oldDefaultLimitKiB = 64
shortValue = "hunter2"
)
tooSmallLimitError := fmt.Sprintf("locked-memory limit of %d KiB",
tooSmallLimitKiB)
stateDir := newBinaryTestStateDir(t)
shortFile := filepath.Join(t.TempDir(), "short")
require.NoError(t, os.WriteFile(shortFile, []byte(shortValue), 0o600))
requireRefusedAtLockedMemoryLimit(t, stateDir, tooSmallLimitKiB,
tooSmallLimitError, strings.NewReader(shortValue), "add", "added")
requireRefusedAtLockedMemoryLimit(t, stateDir, tooSmallLimitKiB,
tooSmallLimitError, nil, "import", "imported", "--source", shortFile)
requireRefusedAtLockedMemoryLimit(t, stateDir, tooSmallLimitKiB,
tooSmallLimitError, nil, "generate", "secret", "generated")
output, err := runAtLockedMemoryLimit(t, stateDir, oldDefaultLimitKiB,
strings.NewReader(shortValue), "add", "added")
require.NoError(t, err, "add: %s", output)
output, err = runAtLockedMemoryLimit(t, stateDir, oldDefaultLimitKiB,
nil, "import", "imported", "--source", shortFile)
require.NoError(t, err, "import: %s", output)
output, err = runAtLockedMemoryLimit(t, stateDir, oldDefaultLimitKiB,
nil, "generate", "secret", "generated")
require.NoError(t, err, "generate secret: %s", output)
}
// slowReader simulates a reader that returns data in small chunks // slowReader simulates a reader that returns data in small chunks
type slowReader struct { type slowReader struct {
data []byte data []byte
+47
View File
@@ -0,0 +1,47 @@
//nolint:testpackage // white-box test of unexported internals
package cli
import (
"bytes"
"testing"
"github.com/spf13/afero"
"github.com/spf13/cobra"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestAddDropsOneTrailingNewlineImportKeepsIt pins the rule the README
// gives: `secret add` drops one trailing newline from stdin, and
// `secret import` stores the file's bytes unchanged.
func TestAddDropsOneTrailingNewlineImportKeepsIt(t *testing.T) {
t.Parallel()
fs, vlt := newSizeTestVault(t)
value := []byte("a\n\n")
cmd := &cobra.Command{}
cmd.SetIn(bytes.NewReader(value))
cli := NewCLIInstanceWithStateDir(fs, testStateDir)
cli.cmd = cmd
require.NoError(t, cli.AddSecret("added", false))
sourceFile := "/test/value"
require.NoError(t, afero.WriteFile(fs, sourceFile, value, 0o600))
require.NoError(t, cli.ImportSecret(cmd, "imported", sourceFile, false))
added, err := vlt.GetSecret("added")
require.NoError(t, err)
defer added.Destroy()
imported, err := vlt.GetSecret("imported")
require.NoError(t, err)
defer imported.Destroy()
assert.Equal(t, []byte("a\n"), added.Bytes())
assert.Equal(t, []byte("a\n\n"), imported.Bytes())
}
+58
View File
@@ -0,0 +1,58 @@
package cli_test
import (
"fmt"
"os/exec"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/secret/internal/secret"
)
// TestUnknownSubcommandFails is a regression test for
// https://git.eeqj.de/sneak/secret/issues/183: given a subcommand it does not
// have, `secret vault`, `secret unlocker`, `secret version` and `secret
// generate` printed their help and exited 0, so a script could not tell a
// mistyped command from a successful one. Each runs here as the built binary
// and must exit 1 naming the subcommand, as an unknown command does at the
// top level, and must still print its help and exit 0 when given nothing.
func TestUnknownSubcommandFails(t *testing.T) {
t.Parallel()
stateDir := t.TempDir()
// run runs the built binary with args and returns what it printed and
// its exit code
run := func(args ...string) (string, int) {
//nolint:gosec // G204: test executes the freshly built secret binary
cmd := exec.CommandContext(t.Context(), secretBinaryPath(t), args...)
cmd.Env = []string{secret.EnvStateDir + "=" + stateDir}
output, err := cmd.CombinedOutput()
if err != nil {
var exitErr *exec.ExitError
require.ErrorAs(t, err, &exitErr, "secret %v did not run", args)
}
return string(output), cmd.ProcessState.ExitCode()
}
for _, call := range []string{
"vault improt", "unlocker foo", "version foo", "generate foo",
} {
parent, unknown, _ := strings.Cut(call, " ")
output, exitCode := run(parent, unknown)
assert.Equal(t, 1, exitCode, "secret %s should fail:\n%s", call, output)
assert.Contains(t, output, fmt.Sprintf(
"unknown command %q for \"secret %s\"", unknown, parent))
output, exitCode = run(parent)
assert.Equal(t, 0, exitCode,
"secret %s should succeed:\n%s", parent, output)
assert.Contains(t, output, usageHeading,
"secret %s should print its help", parent)
}
}
+39 -18
View File
@@ -1,12 +1,12 @@
// Unlock Failure Tests // Unlock Failure Tests
// //
// When a vault cannot be opened through its current unlocker, because a // When a vault cannot be opened through its current unlocker, because a
// file the unlocker needs is missing or the passphrase is wrong, the error // file the unlocker needs is missing, the error keeps its cause and ends by
// keeps its cause and ends by saying that the mnemonic still opens that // saying that the mnemonic still opens that vault, but only for a vault that
// vault, but only for a vault that the mnemonic does open, and not when the // the mnemonic does open, and not when the passphrase could not be read at
// passphrase could not be read at all. When the current unlocker does not // all. A wrong passphrase gives an error that says so, without that advice.
// exist and the vault has another that `secret unlocker select` accepts, the // When the current unlocker does not exist and the vault has another that
// error says to select one instead. // `secret unlocker select` accepts, the error says to select one instead.
// When a secret's current file is missing, the error says how to make a // When a secret's current file is missing, the error says how to make a
// version current again. Each test that pins such advice also follows it. // version current again. Each test that pins such advice also follows it.
@@ -21,7 +21,6 @@ import (
"syscall" "syscall"
"testing" "testing"
"filippo.io/age"
"github.com/awnumar/memguard" "github.com/awnumar/memguard"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/spf13/cobra" "github.com/spf13/cobra"
@@ -165,26 +164,48 @@ func TestUnlockFailureNamesMnemonic(t *testing.T) {
} }
} }
// TestWrongPassphraseNamesMnemonic checks the error of `secret get` given a // TestWrongPassphraseNamesPassphrase checks the error of `secret get` given
// passphrase that does not decrypt the passphrase unlocker: it keeps age's // a passphrase that does not decrypt the passphrase unlocker: it says that
// error and ends with the advice that the mnemonic still opens the vault. // the passphrase is wrong, without the advice to use the mnemonic, since the
func TestWrongPassphraseNamesMnemonic(t *testing.T) { // unlocker still opens the vault with the right passphrase. The test then
// gives the right passphrase.
func TestWrongPassphraseNamesPassphrase(t *testing.T) {
t.Parallel() t.Parallel()
_, c := newUnlockTestCLI(t, newTwoVaultFs(t)) unlockerDir, c := newUnlockTestCLI(t, newTwoVaultFs(t))
c.UnlockPassphrase = memguard.NewBufferFromBytes([]byte("wrong passphrase")) c.UnlockPassphrase = memguard.NewBufferFromBytes([]byte("wrong passphrase"))
t.Cleanup(c.UnlockPassphrase.Destroy) t.Cleanup(c.UnlockPassphrase.Destroy)
err := c.GetSecret(discardCmd(), "x") err := c.GetSecret(discardCmd(), "x")
var noMatch *age.NoIdentityMatchError require.ErrorIs(t, err, secret.ErrWrongPassphrase)
assert.NotContains(t, err.Error(), mnemonicAdvice)
require.ErrorAs(t, err, &noMatch)
require.EqualError(t, err, "failed to unlock vault: "+ require.EqualError(t, err, "failed to unlock vault: "+
"failed to get long-term key: failed to get unlocker identity: "+ "failed to get long-term key: failed to get unlocker identity: "+
"failed to decrypt unlocker private key: failed to create decryptor: "+ "wrong passphrase for unlocker "+filepath.Base(unlockerDir))
noMatch.Error()+mnemonicAdvice)
c.UnlockPassphrase = memguard.NewBufferFromBytes([]byte(testPassphrase))
t.Cleanup(c.UnlockPassphrase.Destroy)
assert.Equal(t, "value", getSecret(t, c, "x"))
}
// TestDamagedUnlockerNamesMnemonic checks the error of `secret get` when the
// passphrase unlocker's priv.age is not an age file: the passphrase is not
// blamed, and the error ends with the advice that the mnemonic still opens
// the vault.
func TestDamagedUnlockerNamesMnemonic(t *testing.T) {
t.Parallel()
fs := newTwoVaultFs(t)
unlockerDir, c := newUnlockTestCLI(t, fs)
require.NoError(t, afero.WriteFile(fs, filepath.Join(unlockerDir, "priv.age"),
[]byte("damaged"), secret.FilePerms))
err := c.GetSecret(discardCmd(), "x")
require.NotErrorIs(t, err, secret.ErrWrongPassphrase)
require.ErrorContains(t, err, mnemonicAdvice)
} }
// TestMoveUnlockFailureNamesVault checks the error of `secret move` into // TestMoveUnlockFailureNamesVault checks the error of `secret move` into
+164 -36
View File
@@ -7,6 +7,7 @@ import (
"fmt" "fmt"
"log" "log"
"maps" "maps"
"net/url"
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
@@ -14,6 +15,7 @@ import (
"strings" "strings"
"time" "time"
"filippo.io/age"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"sneak.berlin/go/secret/internal/secret" "sneak.berlin/go/secret/internal/secret"
@@ -38,6 +40,10 @@ var (
// composes "GPG key <id> is already added as an unlocker". // composes "GPG key <id> is already added as an unlocker".
errGPGKeyAlreadyUnlocker = errors.New( errGPGKeyAlreadyUnlocker = errors.New(
"is already added as an unlocker") "is already added as an unlocker")
// errVaultCreatedAgain carries only the message tail; the caller
// composes "vault '<name>' was removed and created again ...".
errVaultCreatedAgain = errors.New("was removed and created again " +
"with another key while it was being unlocked; nothing was changed")
) )
// UnlockerInfo represents unlocker information for display // UnlockerInfo represents unlocker information for display
@@ -66,12 +72,13 @@ func getDefaultGPGKey() (string, error) {
output, err := cmd.Output() output, err := cmd.Output()
if err == nil { if err == nil {
for line := range strings.SplitSeq(string(output), "\n") { defaultKey, parseErr := parseDefaultGPGKey(string(output))
fields := strings.Split(line, ":") if parseErr != nil {
if len(fields) > 9 && fields[0] == "default-key" && fields[9] != "" { return "", parseErr
// The default key is in field 10 (index 9) }
return fields[9], nil
} if defaultKey != "" {
return defaultKey, nil
} }
} }
@@ -98,11 +105,38 @@ func getDefaultGPGKey() (string, error) {
return "", errNoGPGSecretKeys return "", errNoGPGSecretKeys
} }
// parseDefaultGPGKey returns the default-key value in the output of
// `gpgconf --list-options gpg`, or "" when gpg.conf sets none. gpgconf(1)
// prints a string value with a leading `"` and percent-escaped.
func parseDefaultGPGKey(gpgconfOutput string) (string, error) {
for line := range strings.SplitSeq(gpgconfOutput, "\n") {
fields := strings.Split(line, ":")
if len(fields) > 9 && fields[0] == "default-key" && fields[9] != "" {
// The default key is in field 10 (index 9)
keyID, err := url.PathUnescape(strings.TrimPrefix(fields[9], `"`))
if err != nil {
return "", fmt.Errorf("failed to decode gpgconf default-key %s: %w",
fields[9], err)
}
return keyID, nil
}
}
return "", nil
}
func newUnlockerCmd() *cobra.Command { func newUnlockerCmd() *cobra.Command {
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "unlocker", Use: "unlocker",
Short: "Manage unlockers", Short: "Manage unlockers",
Long: `Create, list, and remove unlockers for the current vault.`, Long: `Create, list, and remove unlockers for the current vault.`,
// Cobra checks Args only on a command that has a RunE; without
// one, an unknown subcommand prints this help and exits 0.
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
return cmd.Help()
},
} }
cmd.AddCommand(newUnlockerListCmd()) cmd.AddCommand(newUnlockerListCmd())
@@ -291,7 +325,8 @@ func (cli *Instance) UnlockersList(jsonOutput bool) error {
return err return err
} }
var unlockers []UnlockerInfo // Not nil, so that a vault with no unlockers prints [] in JSON, not null
unlockers := make([]UnlockerInfo, 0, len(unlockerMetadata))
for _, unlockerID := range slices.Sorted(maps.Keys(unlockerMetadata)) { for _, unlockerID := range slices.Sorted(maps.Keys(unlockerMetadata)) {
metadata := unlockerMetadata[unlockerID] metadata := unlockerMetadata[unlockerID]
@@ -374,12 +409,6 @@ func (cli *Instance) printUnlockersTable(unlockers []UnlockerInfo) error {
// UnlockersAdd adds a new unlocker // UnlockersAdd adds a new unlocker
func (cli *Instance) UnlockersAdd(unlockerType string, cmd *cobra.Command) error { func (cli *Instance) UnlockersAdd(unlockerType string, cmd *cobra.Command) error {
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
switch unlockerType { switch unlockerType {
case unlockerTypePassphrase: case unlockerTypePassphrase:
return cli.addPassphraseUnlocker(cmd) return cli.addPassphraseUnlocker(cmd)
@@ -393,6 +422,64 @@ func (cli *Instance) UnlockersAdd(unlockerType string, cmd *cobra.Command) error
} }
} }
// unlockThenLock gets the long-term key of vlt, the vault an unlocker is
// added to, then takes the state directory lock and returns the function
// that releases it. Getting the key can ask for a passphrase, so it comes
// before the lock, as the question of askThenLock does, and no other
// command waits while the user answers. Once the lock is held, it fails if
// another command has removed the vault in the meantime, since writing the
// unlocker would create the vault's directory again, holding nothing else.
// It also fails if the vault was removed and created again with another
// long-term key, which the unlocker would then hold instead of the new
// vault's own.
func (cli *Instance) unlockThenLock(vlt *vault.Vault) (func(), error) {
ltIdentity, err := vlt.GetOrDeriveLongTermKey()
if err != nil {
return nil, fmt.Errorf("failed to get long-term key: %w", err)
}
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return nil, err
}
err = cli.checkLongTermKey(vlt, ltIdentity)
if err != nil {
release()
return nil, err
}
return release, nil
}
// checkLongTermKey checks, for the vault vlt unlocked as ltIdentity before
// the state directory lock was taken, that vlt still exists and that its
// long-term public key is still that of ltIdentity. Another command may have
// removed the vault in the meantime, or removed it and created it again with
// another long-term key. The caller holds the lock.
func (cli *Instance) checkLongTermKey(
vlt *vault.Vault, ltIdentity *age.X25519Identity,
) error {
_, err := cli.existingVault(vlt.GetName())
if err != nil {
return err
}
vaultDir, _ := vlt.GetDirectory()
ltPubKey, err := afero.ReadFile(cli.fs, filepath.Join(vaultDir, "pub.age"))
if err != nil {
return fmt.Errorf("failed to read long-term public key: %w", err)
}
if string(ltPubKey) != ltIdentity.Recipient().String() {
return fmt.Errorf("vault '%s' %w", vlt.GetName(), errVaultCreatedAgain)
}
return nil
}
// autoSelectUnlocker selects the newly created unlocker as current, // autoSelectUnlocker selects the newly created unlocker as current,
// printing a warning if selection fails // printing a warning if selection fails
func autoSelectUnlocker(cmd *cobra.Command, vlt *vault.Vault, unlockerID string) { func autoSelectUnlocker(cmd *cobra.Command, vlt *vault.Vault, unlockerID string) {
@@ -412,23 +499,31 @@ func (cli *Instance) addPassphraseUnlocker(cmd *cobra.Command) error {
return fmt.Errorf("failed to get current vault: %w", err) return fmt.Errorf("failed to get current vault: %w", err)
} }
// For passphrase unlockers, we don't need the vault to be unlocked
// The CreatePassphraseUnlocker method will handle getting the
// long-term key
vlt.Mnemonic, vlt.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase vlt.Mnemonic, vlt.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase
// The new unlocker gets the passphrase from the environment, which also // The new passphrase is asked for even when SB_UNLOCK_PASSPHRASE is
// unlocks the current passphrase unlocker, else the one entered here // set, and the variable unlocks the vault. With no terminal to ask on,
passphraseBuffer := cli.UnlockPassphrase // the new unlocker takes the passphrase in the variable.
if passphraseBuffer == nil { fromVariable := false
// Use secure passphrase input with confirmation
passphraseBuffer, err = readSecurePassphrase("Enter passphrase for unlocker: ") passphraseBuffer, err := readSecurePassphrase("Enter passphrase for unlocker: ")
if err != nil {
return err switch {
} case errors.Is(err, secret.ErrNoTerminal) && cli.UnlockPassphrase != nil:
passphraseBuffer = cli.UnlockPassphrase
fromVariable = true
case err != nil:
return err
default:
defer passphraseBuffer.Destroy() defer passphraseBuffer.Destroy()
} }
release, err := cli.unlockThenLock(vlt)
if err != nil {
return err
}
defer release()
passphraseUnlocker, err := vlt.CreatePassphraseUnlocker(passphraseBuffer) passphraseUnlocker, err := vlt.CreatePassphraseUnlocker(passphraseBuffer)
if err != nil { if err != nil {
return err return err
@@ -436,6 +531,12 @@ func (cli *Instance) addPassphraseUnlocker(cmd *cobra.Command) error {
cmd.Printf("Created passphrase unlocker: %s\n", passphraseUnlocker.GetID()) cmd.Printf("Created passphrase unlocker: %s\n", passphraseUnlocker.GetID())
if fromVariable {
cmd.Printf("The new unlocker takes its passphrase from %s, since "+
"there is no terminal to ask for a new one on\n",
secret.EnvUnlockPassphrase)
}
// CreatePassphraseUnlocker has already made it the current unlocker // CreatePassphraseUnlocker has already made it the current unlocker
cmd.Printf("Automatically selected as current unlocker\n") cmd.Printf("Automatically selected as current unlocker\n")
@@ -461,12 +562,13 @@ func (cli *Instance) addPGPUnlocker(cmd *cobra.Command) error {
cmd.Printf("Using default GPG key: %s\n", gpgKeyID) cmd.Printf("Using default GPG key: %s\n", gpgKeyID)
} }
// Check if this key is already added as an unlocker
vlt, err := vault.GetCurrentVault(cli.fs, cli.stateDir) vlt, err := vault.GetCurrentVault(cli.fs, cli.stateDir)
if err != nil { if err != nil {
return fmt.Errorf("failed to get current vault: %w", err) return fmt.Errorf("failed to get current vault: %w", err)
} }
vlt.Mnemonic, vlt.UnlockPassphrase = cli.Mnemonic, cli.UnlockPassphrase
// Resolve the GPG key ID to its fingerprint, once: the duplicate check // Resolve the GPG key ID to its fingerprint, once: the duplicate check
// and the new unlocker's metadata both use this result // and the new unlocker's metadata both use this result
fingerprint, err := secret.ResolveGPGKeyFingerprint(gpgKeyID) fingerprint, err := secret.ResolveGPGKeyFingerprint(gpgKeyID)
@@ -474,21 +576,26 @@ func (cli *Instance) addPGPUnlocker(cmd *cobra.Command) error {
return fmt.Errorf("failed to resolve GPG key fingerprint: %w", err) return fmt.Errorf("failed to resolve GPG key fingerprint: %w", err)
} }
// Check if this GPG key is already added // Refuse a key that is already an unlocker before asking for anything,
exists, err := cli.pgpUnlockerExists(vlt, fingerprint) // and again once the lock is held, since another command may have added
// it in the meantime
err = cli.checkPGPKeyNotAdded(vlt, gpgKeyID, fingerprint)
if err != nil { if err != nil {
return fmt.Errorf( return err
"could not check whether GPG key %s is already an unlocker: %w",
gpgKeyID, err,
)
} }
if exists { release, err := cli.unlockThenLock(vlt)
return fmt.Errorf("GPG key %s %w", gpgKeyID, errGPGKeyAlreadyUnlocker) if err != nil {
return err
}
defer release()
err = cli.checkPGPKeyNotAdded(vlt, gpgKeyID, fingerprint)
if err != nil {
return err
} }
pgpUnlocker, err := secret.CreatePGPUnlocker(cli.fs, cli.stateDir, pgpUnlocker, err := secret.CreatePGPUnlocker(vlt, gpgKeyID, fingerprint)
gpgKeyID, fingerprint, cli.Mnemonic, cli.UnlockPassphrase)
if err != nil { if err != nil {
return err return err
} }
@@ -663,6 +770,27 @@ func (cli *Instance) UnlockerSelect(unlockerID string) error {
return vlt.SelectUnlocker(unlockerID) return vlt.SelectUnlocker(unlockerID)
} }
// checkPGPKeyNotAdded fails when vlt already has a PGP unlocker for the GPG
// key gpgKeyID, whose fingerprint is fingerprint, or when that cannot be
// checked.
func (cli *Instance) checkPGPKeyNotAdded(
vlt *vault.Vault, gpgKeyID, fingerprint string,
) error {
exists, err := cli.pgpUnlockerExists(vlt, fingerprint)
if err != nil {
return fmt.Errorf(
"could not check whether GPG key %s is already an unlocker: %w",
gpgKeyID, err,
)
}
if exists {
return fmt.Errorf("GPG key %s %w", gpgKeyID, errGPGKeyAlreadyUnlocker)
}
return nil
}
// pgpUnlockerExists reports whether the vault already has a PGP unlocker // pgpUnlockerExists reports whether the vault already has a PGP unlocker
// for the GPG key with the given fingerprint. It returns an error, and no // for the GPG key with the given fingerprint. It returns an error, and no
// answer, when unlockers.d or an unlocker's metadata file cannot be read; // answer, when unlockers.d or an unlocker's metadata file cannot be read;
+73
View File
@@ -2,6 +2,8 @@
package cli package cli
import ( import (
"bytes"
"os"
"path/filepath" "path/filepath"
"strings" "strings"
"testing" "testing"
@@ -83,6 +85,77 @@ func TestAddPGPUnlocker(t *testing.T) {
} }
} }
// TestAddPGPUnlockerDefaultKey is a regression test for
// https://git.eeqj.de/sneak/secret/issues/141: with default-key set in
// gpg.conf and no --keyid, `secret unlocker add pgp` uses that key.
// gpg.conf names the key by its user ID, and the printed key ID must be that
// user ID. The first secret key, which is used when gpg.conf sets no
// default-key, prints as a hex key ID.
func TestAddPGPUnlockerDefaultKey(t *testing.T) {
newTestGPGKey(t)
// The key comes from neither --keyid nor the environment.
t.Setenv(secret.EnvGPGKeyID, "")
gpgConf := filepath.Join(os.Getenv("GNUPGHOME"), "gpg.conf")
//nolint:gosec // G703: GNUPGHOME is the directory newTestGPGKey made
require.NoError(t, os.WriteFile(gpgConf,
[]byte("default-key "+unreadableTestGPGUserID+"\n"), 0o600))
fs := afero.NewMemMapFs()
_, err := vault.CreateVault(fs, listTestStateDir, listTestVaultName,
testMnemonicBuffer(t), nil)
require.NoError(t, err)
instance, cmd := newTestInstance(fs)
instance.Mnemonic = testMnemonicBuffer(t)
var output bytes.Buffer
cmd.SetOut(&output)
require.NoError(t, instance.UnlockersAdd(unlockerTypePGP, cmd))
assert.Contains(t, output.String(),
"Using default GPG key: "+unreadableTestGPGUserID+"\n")
}
// TestParseDefaultGPGKey is a regression test for
// https://git.eeqj.de/sneak/secret/issues/141: gpgconf prints the
// default-key value with a leading `"` and percent-escaped, and the key ID
// is that value decoded.
func TestParseDefaultGPGKey(t *testing.T) {
t.Parallel()
// gpgconfLine is the default-key line of `gpgconf --list-options gpg`
// up to the value, its last field.
const gpgconfLine = "default-key:0:0:use NAME as default secret key:" +
"1:1:NAME:::"
tests := []struct {
name string
value string
want string
}{
{
"fingerprint", `"56163D7F3FAFF709608855BEC1A360216D007F41`,
"56163D7F3FAFF709608855BEC1A360216D007F41",
},
{
"user ID with a comma", `"Doe%2c Alice <alice@example.com>`,
"Doe, Alice <alice@example.com>",
},
{"no default-key in gpg.conf", "", ""},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
t.Parallel()
keyID, err := parseDefaultGPGKey(gpgconfLine + test.value + "\n")
require.NoError(t, err)
assert.Equal(t, test.want, keyID)
})
}
}
// TestAddPGPUnlockerUnknownKey asserts that adding a PGP unlocker for a key // TestAddPGPUnlockerUnknownKey asserts that adding a PGP unlocker for a key
// the keyring does not hold fails at looking up the key's fingerprint and // the keyring does not hold fails at looking up the key's fingerprint and
// leaves no new unlocker directory. The error must come from the lookup: a // leaves no new unlocker directory. The error must come from the lookup: a
+36 -2
View File
@@ -1,13 +1,14 @@
// Unlocker List Tests // Unlocker List Tests
// //
// Tests for `secret unlocker list` behavior when an unlocker's metadata // Tests for `secret unlocker list`:
// cannot be read or used:
// //
// - TestUnlockersListToleratesCorruptMetadata: one unlocker's corrupt // - TestUnlockersListToleratesCorruptMetadata: one unlocker's corrupt
// metadata does not stop the others from being listed. // metadata does not stop the others from being listed.
// - TestUnlockersListSkipsUnreadableMetadata: an unlocker whose metadata // - TestUnlockersListSkipsUnreadableMetadata: an unlocker whose metadata
// file cannot be checked for or read is left out, and the other is // file cannot be checked for or read is left out, and the other is
// still listed. // still listed.
// - TestUnlockersListEmptyJSON: a vault with no unlockers lists them as
// an empty JSON list.
//nolint:testpackage // white-box test of unexported internals //nolint:testpackage // white-box test of unexported internals
package cli package cli
@@ -321,3 +322,36 @@ func TestUnlockersListSkipsUnreadableMetadata(t *testing.T) {
}) })
} }
} }
// TestUnlockersListEmptyJSON is a regression test for
// https://git.eeqj.de/sneak/secret/issues/148: listing the unlockers of a
// vault that has none, as after its last unlocker is removed, printed
// "unlockers": null in JSON form instead of an empty list.
func TestUnlockersListEmptyJSON(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
vaultDir := filepath.Join(listTestStateDir, "vaults.d", listTestVaultName)
require.NoError(t, afero.WriteFile(
fs, filepath.Join(listTestStateDir, "currentvault"),
[]byte(listTestVaultName), listTestFilePerm,
))
require.NoError(t, fs.MkdirAll(
filepath.Join(vaultDir, listTestUnlockersDirName), listTestDirPerm,
))
var buf bytes.Buffer
cmd := &cobra.Command{}
cmd.SetOut(&buf)
instance := &Instance{fs: fs, stateDir: listTestStateDir, cmd: cmd}
require.NoError(t, instance.UnlockersList(true))
var decoded map[string]any
require.NoError(t, json.Unmarshal(buf.Bytes(), &decoded))
assert.Equal(t, []any{}, decoded["unlockers"],
"no unlockers must be printed as [], not %s", buf.String())
}
+6 -21
View File
@@ -9,11 +9,12 @@
// "nothing there" and go ahead. // "nothing there" and go ahead.
// //
// The tests make the look fail with a wrapper around the in-memory // The tests make the look fail with a wrapper around the in-memory
// filesystem, which the state directory lock refuses. So they call the // filesystem, which the state directory lock refuses. So they call a
// function each command runs once it holds the lock, such as addPGPUnlocker // function that makes the look before any lock is taken: addPGPUnlocker for
// for UnlockersAdd, or, for a removal, the function that makes its checks, // UnlockersAdd, which looks again once it holds the lock, or, for a removal,
// such as findVaultToRemove for RemoveVault, which runs again under the // the function that makes its checks, such as findVaultToRemove for
// lock before anything is removed, with --force or without. // RemoveVault, which runs again under the lock before anything is removed,
// with --force or without.
//nolint:testpackage // white-box test of unexported internals //nolint:testpackage // white-box test of unexported internals
package cli package cli
@@ -371,19 +372,3 @@ func TestRemoveSecretAbortsWhenVersionsUnreadable(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
assert.True(t, exists, "the secret must not be removed") assert.True(t, exists, "the secret must not be removed")
} }
// TestVaultImportAbortsWhenPubKeyUnreadable asserts that a mnemonic import
// stops when whether the vault already has a long-term key cannot be
// determined.
func TestVaultImportAbortsWhenPubKeyUnreadable(t *testing.T) {
t.Parallel()
base := newListTestVault(t, 1)
instance, cmd := newTestInstance(&statFailFs{
Fs: base, path: filepath.Join(testVaultDir(listTestVaultName), "pub.age"),
})
err := instance.importMnemonic(cmd, listTestVaultName)
require.ErrorIs(t, err, errStatFailed)
}
+19 -235
View File
@@ -8,7 +8,6 @@ import (
"path/filepath" "path/filepath"
"slices" "slices"
"strings" "strings"
"time"
"github.com/awnumar/memguard" "github.com/awnumar/memguard"
"github.com/spf13/afero" "github.com/spf13/afero"
@@ -16,19 +15,12 @@ import (
"sneak.berlin/go/secret/internal/bip39" "sneak.berlin/go/secret/internal/bip39"
"sneak.berlin/go/secret/internal/secret" "sneak.berlin/go/secret/internal/secret"
"sneak.berlin/go/secret/internal/vault" "sneak.berlin/go/secret/internal/vault"
"sneak.berlin/go/secret/pkg/agehd"
) )
// Sentinel errors for vault operations // Sentinel errors for vault operations
var ( var (
errMnemonicEmpty = errors.New("mnemonic cannot be empty") errMnemonicEmpty = errors.New("mnemonic cannot be empty")
errInvalidMnemonicPhrase = errors.New("invalid BIP39 mnemonic phrase") errInvalidMnemonicPhrase = errors.New("invalid BIP39 mnemonic phrase")
errVaultHasLongTermKey = errors.New(
"already has a long-term key configured")
errMnemonicEnvNotSet = errors.New(
"SB_SECRET_MNEMONIC environment variable not set")
errPassphraseEnvNotSet = errors.New(
"SB_UNLOCK_PASSPHRASE environment variable not set")
errCannotRemoveLastVault = errors.New("cannot remove the last vault") errCannotRemoveLastVault = errors.New("cannot remove the last vault")
) )
@@ -37,12 +29,17 @@ func newVaultCmd() *cobra.Command {
Use: "vault", Use: "vault",
Short: "Manage vaults", Short: "Manage vaults",
Long: `Create, list, and select vaults for organizing secrets.`, Long: `Create, list, and select vaults for organizing secrets.`,
// Cobra checks Args only on a command that has a RunE; without
// one, an unknown subcommand prints this help and exits 0.
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
return cmd.Help()
},
} }
cmd.AddCommand(newVaultListCmd()) cmd.AddCommand(newVaultListCmd())
cmd.AddCommand(newVaultCreateCmd()) cmd.AddCommand(newVaultCreateCmd())
cmd.AddCommand(newVaultSelectCmd()) cmd.AddCommand(newVaultSelectCmd())
cmd.AddCommand(newVaultImportCmd())
cmd.AddCommand(newVaultRemoveCmd()) cmd.AddCommand(newVaultRemoveCmd())
return cmd return cmd
@@ -111,38 +108,6 @@ func newVaultSelectCmd() *cobra.Command {
} }
} }
func newVaultImportCmd() *cobra.Command {
cli, err := NewCLIInstance()
if err != nil {
log.Fatalf("failed to initialize CLI: %v", err)
}
return &cobra.Command{
Use: "import <vault-name>",
Short: "Import a mnemonic into a vault",
Long: `Import a BIP39 mnemonic phrase into the specified vault ` +
`(default if not specified).`,
Args: cobra.MaximumNArgs(1),
ValidArgsFunction: getVaultNamesCompletionFunc(cli.fs, cli.stateDir),
RunE: func(cmd *cobra.Command, args []string) error {
vaultName := "default"
if len(args) > 0 {
vaultName = args[0]
}
cli, err := NewCLIInstance()
if err != nil {
return fmt.Errorf("failed to initialize CLI: %w", err)
}
destroySecrets := cli.readSecretEnv()
defer destroySecrets()
return cli.VaultImport(cmd, vaultName)
},
}
}
func newVaultRemoveCmd() *cobra.Command { func newVaultRemoveCmd() *cobra.Command {
cli, err := NewCLIInstance() cli, err := NewCLIInstance()
if err != nil { if err != nil {
@@ -257,15 +222,17 @@ func (cli *Instance) resolvePassphrase() (*memguard.LockedBuffer, func(), error)
return passphraseBuffer, passphraseBuffer.Destroy, nil return passphraseBuffer, passphraseBuffer.Destroy, nil
} }
// CreateVault creates a new vault // CreateVault creates a new vault. It asks for the mnemonic and the
// passphrase before it takes the state directory lock, so that no other
// command waits while the user answers.
func (cli *Instance) CreateVault(cmd *cobra.Command, name string) error { func (cli *Instance) CreateVault(cmd *cobra.Command, name string) error {
secret.Debug("Creating new vault", "name", name, "state_dir", cli.stateDir) secret.Debug("Creating new vault", "name", name, "state_dir", cli.stateDir)
release, err := vault.LockStateDir(cli.fs, cli.stateDir) // Refuse an invalid name or an existing vault before asking for anything
err := vault.CheckNewVaultName(cli.fs, cli.stateDir, name)
if err != nil { if err != nil {
return err return err
} }
defer release()
mnemonic, cleanupMnemonic, err := cli.promptMnemonic() mnemonic, cleanupMnemonic, err := cli.promptMnemonic()
if err != nil { if err != nil {
@@ -294,7 +261,14 @@ func (cli *Instance) CreateVault(cmd *cobra.Command, name string) error {
} }
defer cleanupPassphrase() defer cleanupPassphrase()
// Create the vault with its passphrase unlocker release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
// Create the vault with its passphrase unlocker. Another command may
// have created it since the check above: vault.CreateVault checks again.
vlt, err := vault.CreateVault(cli.fs, cli.stateDir, name, vlt, err := vault.CreateVault(cli.fs, cli.stateDir, name,
mnemonic, passphraseBuffer) mnemonic, passphraseBuffer)
if err != nil { if err != nil {
@@ -336,196 +310,6 @@ func (cli *Instance) SelectVault(cmd *cobra.Command, name string) error {
return nil return nil
} }
// vaultImportPreflight verifies the vault exists without a long-term key
// and returns the vault directory, public key path, and validated mnemonic
func (cli *Instance) vaultImportPreflight(
vlt *vault.Vault, vaultName string,
) (string, string, string, error) {
// Check if vault exists
vaultDir, err := vlt.GetDirectory()
if err != nil {
return "", "", "", err
}
exists, err := afero.DirExists(cli.fs, vaultDir)
if err != nil {
return "", "", "", fmt.Errorf("failed to check if vault exists: %w", err)
}
if !exists {
return "", "", "", fmt.Errorf("vault '%s' %w",
vaultName, vault.ErrVaultNotFound)
}
// Check if vault already has a public key
pubKeyPath := vaultDir + "/pub.age"
exists, err = afero.Exists(cli.fs, pubKeyPath)
if err != nil {
return "", "", "", fmt.Errorf("failed to check %s: %w", pubKeyPath, err)
}
if exists {
return "", "", "", fmt.Errorf("vault '%s' %w",
vaultName, errVaultHasLongTermKey)
}
// Get mnemonic from environment
if cli.Mnemonic == nil {
return "", "", "", errMnemonicEnvNotSet
}
mnemonic := cli.Mnemonic.String()
// Validate the mnemonic
mnemonicWords := strings.Fields(mnemonic)
secret.Debug("Validating BIP39 mnemonic", "word_count", len(mnemonicWords))
if !bip39.IsMnemonicValid(mnemonic) {
return "", "", "", errInvalidMnemonicPhrase
}
return vaultDir, pubKeyPath, mnemonic, nil
}
// updateVaultImportMetadata stores the derivation info in vault metadata
func updateVaultImportMetadata(
fs afero.Fs, vaultDir string, derivationIndex uint32,
publicKeyHash, familyHash string,
) error {
// Load existing metadata
existingMetadata, err := vault.LoadVaultMetadata(fs, vaultDir)
if err != nil {
// If metadata doesn't exist, create new
existingMetadata = &vault.Metadata{
CreatedAt: time.Now(),
}
}
// Update metadata with new derivation info
existingMetadata.DerivationIndex = derivationIndex
existingMetadata.PublicKeyHash = publicKeyHash
existingMetadata.MnemonicFamilyHash = familyHash
err = vault.SaveVaultMetadata(fs, vaultDir, existingMetadata)
if err != nil {
secret.Debug("Failed to save vault metadata", "error", err)
return fmt.Errorf("failed to save vault metadata: %w", err)
}
secret.Debug("Saved vault metadata with derivation index and public key hash")
return nil
}
// VaultImport imports a mnemonic into a specific vault, holding the state
// directory lock while importMnemonic runs
func (cli *Instance) VaultImport(cmd *cobra.Command, vaultName string) error {
err := vault.ValidateVaultName(vaultName)
if err != nil {
return err
}
release, err := vault.LockStateDir(cli.fs, cli.stateDir)
if err != nil {
return err
}
defer release()
return cli.importMnemonic(cmd, vaultName)
}
// importMnemonic gives the vault a long-term key derived from the mnemonic
// and a passphrase unlocker
func (cli *Instance) importMnemonic(cmd *cobra.Command, vaultName string) error {
secret.Debug("Importing mnemonic into vault",
"vault_name", vaultName, "state_dir", cli.stateDir)
// Get the specific vault by name
vlt := vault.NewVault(cli.fs, cli.stateDir, vaultName)
vaultDir, pubKeyPath, mnemonic, err := cli.vaultImportPreflight(vlt, vaultName)
if err != nil {
return err
}
// Get the next available derivation index for this mnemonic
derivationIndex, err := vault.GetNextDerivationIndex(cli.fs, cli.stateDir, mnemonic)
if err != nil {
secret.Debug("Failed to get next derivation index", "error", err)
return fmt.Errorf("failed to get next derivation index: %w", err)
}
secret.Debug("Using derivation index", "index", derivationIndex)
// Derive long-term key from mnemonic with the appropriate index
secret.Debug("Deriving long-term key from mnemonic", "index", derivationIndex)
ltIdentity, err := agehd.DeriveIdentity(mnemonic, derivationIndex)
if err != nil {
return fmt.Errorf("failed to derive long-term key: %w", err)
}
// Store long-term public key in vault
ltPublicKey := ltIdentity.Recipient().String()
secret.Debug("Storing long-term public key",
"pubkey", ltPublicKey, "vault_dir", vaultDir)
err = secret.WriteFileAtomic(cli.fs, pubKeyPath, []byte(ltPublicKey))
if err != nil {
return fmt.Errorf("failed to store long-term public key: %w", err)
}
// Calculate public key hash from the actual derivation index being used
// This is used to verify that the derived key matches what was stored
publicKeyHash := vault.ComputeDoubleSHA256([]byte(ltIdentity.Recipient().String()))
// Calculate family hash from index 0 (same for all vaults with this
// mnemonic). This is used to identify which vaults belong to the same
// mnemonic family.
identity0, err := agehd.DeriveIdentity(mnemonic, 0)
if err != nil {
return fmt.Errorf("failed to derive identity for index 0: %w", err)
}
familyHash := vault.ComputeDoubleSHA256([]byte(identity0.Recipient().String()))
err = updateVaultImportMetadata(
cli.fs, vaultDir, derivationIndex, publicKeyHash, familyHash)
if err != nil {
return err
}
// Get passphrase from environment variable
passphraseBuffer := cli.UnlockPassphrase
if passphraseBuffer == nil {
return errPassphraseEnvNotSet
}
secret.Debug("Using unlock passphrase from environment variable")
// Unlock the vault with the derived long-term key
vlt.Unlock(ltIdentity)
// Create passphrase-protected unlocker
secret.Debug("Creating passphrase-protected unlocker")
passphraseUnlocker, err := vlt.CreatePassphraseUnlocker(passphraseBuffer)
if err != nil {
secret.Debug("Failed to create unlocker", "error", err)
return fmt.Errorf("failed to create unlocker: %w", err)
}
cmd.Printf("Successfully imported mnemonic into vault '%s'\n", vaultName)
cmd.Printf("Long-term public key: %s\n", ltPublicKey)
cmd.Printf("Unlocker ID: %s\n", passphraseUnlocker.GetID())
return nil
}
// countVaultSecrets returns the number of secrets in the vault directory // countVaultSecrets returns the number of secrets in the vault directory
func (cli *Instance) countVaultSecrets(vaultDir string) (int, error) { func (cli *Instance) countVaultSecrets(vaultDir string) (int, error) {
secretsDir := filepath.Join(vaultDir, "secrets.d") secretsDir := filepath.Join(vaultDir, "secrets.d")
+42
View File
@@ -0,0 +1,42 @@
package cli_test
import (
"bytes"
"encoding/json"
"path/filepath"
"testing"
"github.com/spf13/afero"
"github.com/spf13/cobra"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/secret/internal/cli"
"sneak.berlin/go/secret/internal/secret"
)
// TestVaultListEmptyJSON is a regression test for
// https://git.eeqj.de/sneak/secret/issues/148: listing the vaults when
// vaults.d holds none printed "vaults": null in JSON form instead of an
// empty list.
func TestVaultListEmptyJSON(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll(
filepath.Join(testStateDir, "vaults.d"), secret.DirPerms,
))
var buf bytes.Buffer
cmd := &cobra.Command{}
cmd.SetOut(&buf)
c := cli.NewCLIInstanceWithStateDir(fs, testStateDir)
require.NoError(t, c.ListVaults(cmd, true))
var decoded map[string]any
require.NoError(t, json.Unmarshal(buf.Bytes(), &decoded))
assert.Equal(t, []any{}, decoded["vaults"],
"no vaults must be printed as [], not %s", buf.String())
}
+17 -5
View File
@@ -43,6 +43,12 @@ func VersionCommands(cli *Instance) *cobra.Command {
Short: "Manage secret versions", Short: "Manage secret versions",
Long: "Commands for managing secret versions including listing, " + Long: "Commands for managing secret versions including listing, " +
"promoting, and retrieving specific versions", "promoting, and retrieving specific versions",
// Cobra checks Args only on a command that has a RunE; without
// one, an unknown subcommand prints this help and exits 0.
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
return cmd.Help()
},
} }
// List versions command // List versions command
@@ -64,8 +70,10 @@ func VersionCommands(cli *Instance) *cobra.Command {
promoteCmd := &cobra.Command{ promoteCmd := &cobra.Command{
Use: "promote <secret-name> <version>", Use: "promote <secret-name> <version>",
Short: "Promote a specific version to current", Short: "Promote a specific version to current",
Long: "Updates the current symlink to point to the specified " + Long: "Rewrites the secret's current file to name the specified " +
"version without modifying timestamps", "version. The versions' own times are left unchanged; secret " +
"list then gives the time of the promote as the secret's last " +
"update.",
Args: cobra.ExactArgs(2), //nolint:mnd // secret-name and version args Args: cobra.ExactArgs(2), //nolint:mnd // secret-name and version args
ValidArgsFunction: func( ValidArgsFunction: func(
cmd *cobra.Command, args []string, toComplete string, cmd *cobra.Command, args []string, toComplete string,
@@ -227,9 +235,13 @@ func printVersionRow(
} }
// Determine status // Determine status
status := "expired" status := "inactive"
if version == currentVersion {
switch {
case version == currentVersion:
status = "current" status = "current"
case sv.Metadata.NotAfter != nil && sv.Metadata.NotAfter.Before(time.Now()):
status = "expired"
} }
// Format timestamps // Format timestamps
@@ -291,7 +303,7 @@ func (cli *Instance) PromoteVersion(
version, vault.ErrVersionNotFound, secretName) version, vault.ErrVersionNotFound, secretName)
} }
// Update the current symlink using the proper function // Rewrite the secret's current file to name the version
err = secret.SetCurrentVersion(cli.fs, secretDir, version) err = secret.SetCurrentVersion(cli.fs, secretDir, version)
if err != nil { if err != nil {
return fmt.Errorf("failed to update current version: %w", err) return fmt.Errorf("failed to update current version: %w", err)
+122
View File
@@ -5,9 +5,14 @@
// - TestListVersionsCommand: Tests `secret version list` command output // - TestListVersionsCommand: Tests `secret version list` command output
// - TestListVersionsNonExistentSecret: Tests error handling for missing secrets // - TestListVersionsNonExistentSecret: Tests error handling for missing secrets
// - TestPromoteVersionCommand: Tests `secret version promote` command // - TestPromoteVersionCommand: Tests `secret version promote` command
// - TestListVersionsStatusAfterPromote: Tests each version's status after
// promoting an older version
// - TestPromoteNonExistentVersion: Tests error handling for invalid promotion // - TestPromoteNonExistentVersion: Tests error handling for invalid promotion
// - TestPromoteHelpSaysCurrentFileIsRewritten: Tests `secret version promote
// --help` text
// - TestGetSecretWithVersion: Tests `secret get --version` flag functionality // - TestGetSecretWithVersion: Tests `secret get --version` flag functionality
// - TestGetSecretWritesBinaryValue: Tests `secret get` output of binary values // - TestGetSecretWritesBinaryValue: Tests `secret get` output of binary values
// - TestGetSecretReturnsWriteError: Tests `secret get` when stdout fails
// - TestVersionCommandStructure: Tests command structure and help text // - TestVersionCommandStructure: Tests command structure and help text
// - TestListVersionsEmptyOutput: Tests edge case with no versions // - TestListVersionsEmptyOutput: Tests edge case with no versions
// //
@@ -20,6 +25,7 @@ package cli
import ( import (
"bytes" "bytes"
"os"
"path/filepath" "path/filepath"
"strings" "strings"
"testing" "testing"
@@ -240,6 +246,66 @@ func TestPromoteVersionCommand(t *testing.T) {
assert.Equal(t, []byte("version-1"), promoted.Bytes()) assert.Equal(t, []byte("version-1"), promoted.Bytes())
} }
func TestListVersionsStatusAfterPromote(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
cli := NewCLIInstanceWithStateDir(fs, testStateDir)
cli.Mnemonic = testMnemonicBuffer(t)
setupTestVault(t, fs)
vlt, err := vault.GetCurrentVault(fs, testStateDir)
require.NoError(t, err)
vlt.Mnemonic = cli.Mnemonic
addTestSecret(t, vlt, []byte("version-1"), false)
addTestSecret(t, vlt, []byte("version-2"), true)
addTestSecret(t, vlt, []byte("version-3"), true)
vaultDir, err := vlt.GetDirectory()
require.NoError(t, err)
// Newest first
versions, err := secret.ListVersions(fs,
filepath.Join(vaultDir, "secrets.d", "test%secret"))
require.NoError(t, err)
require.Len(t, versions, 3)
newest, middle, oldest := versions[0], versions[1], versions[2]
cmd := newRootCmd()
var buf bytes.Buffer
cmd.SetOut(&buf)
cmd.SetErr(&buf)
err = cli.PromoteVersion(cmd, "test/secret", oldest)
require.NoError(t, err)
buf.Reset()
err = cli.ListVersions(cmd, "test/secret")
require.NoError(t, err)
// STATUS is the fourth field of a row, since CREATED is a date and a time
statuses := map[string]string{}
for line := range strings.SplitSeq(buf.String(), "\n") {
fields := strings.Fields(line)
if len(fields) > 3 {
statuses[fields[0]] = fields[3]
}
}
assert.Equal(t, "current", statuses[oldest])
// Its NOT_AFTER was set when the newest version was added
assert.Equal(t, "expired", statuses[middle])
// It has no NOT_AFTER
assert.Equal(t, "inactive", statuses[newest])
}
func TestPromoteNonExistentVersion(t *testing.T) { func TestPromoteNonExistentVersion(t *testing.T) {
t.Parallel() t.Parallel()
@@ -268,6 +334,23 @@ func TestPromoteNonExistentVersion(t *testing.T) {
require.ErrorIs(t, err, vault.ErrVersionNotFound) require.ErrorIs(t, err, vault.ErrVersionNotFound)
} }
// TestPromoteHelpSaysCurrentFileIsRewritten is a regression test for
// https://git.eeqj.de/sneak/secret/issues/153: `secret version promote
// --help` said it updates a symlink, but promote rewrites the secret's
// `current` file, which is a plain file.
//
//nolint:paralleltest // executes the CLI in-process and sets the environment
func TestPromoteHelpSaysCurrentFileIsRewritten(t *testing.T) {
env := map[string]string{secret.EnvStateDir: t.TempDir()}
output, err := ExecuteCommandInProcess(
[]string{"version", "promote", "--help"}, "", env)
require.NoError(t, err)
assert.NotContains(t, output, "symlink")
assert.Contains(t, output, "current file")
}
func TestGetSecretWithVersion(t *testing.T) { func TestGetSecretWithVersion(t *testing.T) {
t.Parallel() t.Parallel()
@@ -360,6 +443,45 @@ func TestGetSecretWritesBinaryValue(t *testing.T) {
assert.Equal(t, value, buf.Bytes()) assert.Equal(t, value, buf.Bytes())
} }
// TestGetSecretReturnsWriteError checks that `secret get`, with and without
// a version, returns the error of a failed write of the value.
func TestGetSecretReturnsWriteError(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
cli := NewCLIInstanceWithStateDir(fs, testStateDir)
cli.Mnemonic = testMnemonicBuffer(t)
setupTestVault(t, fs)
vlt, err := vault.GetCurrentVault(fs, testStateDir)
require.NoError(t, err)
addTestSecret(t, vlt, []byte("value"), false)
vaultDir, err := vlt.GetDirectory()
require.NoError(t, err)
versions, err := secret.ListVersions(fs,
filepath.Join(vaultDir, "secrets.d", "test%secret"))
require.NoError(t, err)
require.Len(t, versions, 1)
// Stdout is a file closed before the value is written to it
out, err := os.Create(filepath.Join(t.TempDir(), "stdout"))
require.NoError(t, err)
require.NoError(t, out.Close())
cmd := newRootCmd()
cmd.SetOut(out)
err = cli.GetSecret(cmd, "test/secret")
require.ErrorIs(t, err, os.ErrClosed)
err = cli.GetSecretWithVersion(cmd, "test/secret", versions[0])
require.ErrorIs(t, err, os.ErrClosed)
}
//nolint:paralleltest // reads process environment to determine the state dir //nolint:paralleltest // reads process environment to determine the state dir
func TestVersionCommandStructure(t *testing.T) { func TestVersionCommandStructure(t *testing.T) {
// Test that version commands are properly structured // Test that version commands are properly structured
+5 -2
View File
@@ -149,7 +149,9 @@ func WriteDir(fs afero.Fs, dir string, write func(dir string) error) error {
// RemoveDirAtomic deletes the directory dir so that it disappears in one // RemoveDirAtomic deletes the directory dir so that it disappears in one
// rename: dir is moved into a new directory from TempDirFor, which is then // rename: dir is moved into a new directory from TempDirFor, which is then
// deleted. A crash part-way leaves only that temporary directory behind. // deleted. A crash part-way leaves only that temporary directory behind. If
// only deleting the temporary directory fails, dir is already gone, and the
// error names the temporary directory, which holds what is left of it.
func RemoveDirAtomic(fs afero.Fs, dir string) error { func RemoveDirAtomic(fs afero.Fs, dir string) error {
tmp, err := TempDirFor(fs, dir) tmp, err := TempDirFor(fs, dir)
if err != nil { if err != nil {
@@ -165,7 +167,8 @@ func RemoveDirAtomic(fs afero.Fs, dir string) error {
err = fs.RemoveAll(tmp) err = fs.RemoveAll(tmp)
if err != nil { if err != nil {
return fmt.Errorf("failed to remove %s: %w", dir, err) return fmt.Errorf("failed to remove %s after moving %s into it: %w",
tmp, dir, err)
} }
return nil return nil
+5 -2
View File
@@ -15,10 +15,13 @@ import (
var ( var (
errNilPassphraseBuffer = errors.New("passphrase buffer is nil") errNilPassphraseBuffer = errors.New("passphrase buffer is nil")
errNoTerminal = errors.New("there is no terminal to ask on")
errNothingEntered = errors.New("nothing was entered") errNothingEntered = errors.New("nothing was entered")
) )
// ErrNoTerminal is wrapped in the error of ReadPassphrase and ReadMnemonic
// when the process has no controlling terminal to ask on.
var ErrNoTerminal = errors.New("there is no terminal to ask on")
// ErrMnemonicNotRead is wrapped in every error of ReadMnemonic: there is no // ErrMnemonicNotRead is wrapped in every error of ReadMnemonic: there is no
// terminal to read the mnemonic from, reading it failed, or it was empty. // terminal to read the mnemonic from, reading it failed, or it was empty.
var ErrMnemonicNotRead = errors.New("failed to read mnemonic") var ErrMnemonicNotRead = errors.New("failed to read mnemonic")
@@ -203,7 +206,7 @@ func readFromTerminal(
return nil, fmt.Errorf( return nil, fmt.Errorf(
"%w: %w. Please set the %s environment variable or run interactively", "%w: %w. Please set the %s environment variable or run interactively",
notRead, errNoTerminal, envVar) notRead, ErrNoTerminal, envVar)
} }
defer func() { _ = terminal.Close() }() defer func() { _ = terminal.Close() }()
+10
View File
@@ -16,6 +16,10 @@ import (
// empty. A passphrase unlocker that fails with it was not tried. // empty. A passphrase unlocker that fails with it was not tried.
var ErrPassphraseNotRead = errors.New("failed to read passphrase") var ErrPassphraseNotRead = errors.New("failed to read passphrase")
// ErrWrongPassphrase is wrapped in the error of GetIdentity when the
// passphrase does not decrypt the unlocker's private key.
var ErrWrongPassphrase = errors.New("wrong passphrase")
// PassphraseUnlocker represents a passphrase-protected unlocker // PassphraseUnlocker represents a passphrase-protected unlocker
type PassphraseUnlocker struct { type PassphraseUnlocker struct {
Directory string Directory string
@@ -73,6 +77,12 @@ func (p *PassphraseUnlocker) GetIdentity() (*age.X25519Identity, error) {
Debug("Failed to decrypt unlocker private key", Debug("Failed to decrypt unlocker private key",
"error", err, "unlocker_id", p.GetID()) "error", err, "unlocker_id", p.GetID())
// age says that no identity matched when the passphrase is wrong
var noMatch *age.NoIdentityMatchError
if errors.As(err, &noMatch) {
return nil, fmt.Errorf("%w for unlocker %s", ErrWrongPassphrase, p.GetID())
}
return nil, fmt.Errorf("failed to decrypt unlocker private key: %w", err) return nil, fmt.Errorf("failed to decrypt unlocker private key: %w", err)
} }
defer privKeyBuffer.Destroy() defer privKeyBuffer.Destroy()
+1 -2
View File
@@ -375,8 +375,7 @@ func testCreatePGPUnlocker(
} }
// Now create a PGP unlock key (this will use our custom GPGEncryptFunc) // Now create a PGP unlock key (this will use our custom GPGEncryptFunc)
pgpUnlocker, err := secret.CreatePGPUnlocker( pgpUnlocker, err := secret.CreatePGPUnlocker(vlt, keyID, fingerprint)
fs, stateDir, keyID, fingerprint, mnemonic, nil)
if err != nil { if err != nil {
t.Fatalf("Failed to create PGP unlock key: %v", err) t.Fatalf("Failed to create PGP unlock key: %v", err)
} }
+36 -30
View File
@@ -192,58 +192,45 @@ func generatePGPUnlockerName() (string, error) {
return fmt.Sprintf("%s-pgp-%s", hostname, enrollmentTime), nil return fmt.Sprintf("%s-pgp-%s", hostname, enrollmentTime), nil
} }
// pgpUnlockerDir returns the current vault and the directory in it for a // pgpUnlockerDir returns the directory in vault for a new PGP unlocker,
// new PGP unlocker, named after the host and the time. // named after the host and the time.
// func pgpUnlockerDir(vault VaultInterface) (string, error) {
//nolint:ireturn // the vault is only available behind VaultInterface
func pgpUnlockerDir(
fs afero.Fs, stateDir string,
) (VaultInterface, string, error) {
// Get current vault
vault, err := GetCurrentVault(fs, stateDir)
if err != nil {
return nil, "", fmt.Errorf("failed to get current vault: %w", err)
}
// Generate the unlocker name based on hostname and time // Generate the unlocker name based on hostname and time
unlockerName, err := generatePGPUnlockerName() unlockerName, err := generatePGPUnlockerName()
if err != nil { if err != nil {
return nil, "", fmt.Errorf("failed to generate unlocker name: %w", err) return "", fmt.Errorf("failed to generate unlocker name: %w", err)
} }
vaultDir, err := vault.GetDirectory() vaultDir, err := vault.GetDirectory()
if err != nil { if err != nil {
return nil, "", fmt.Errorf("failed to get vault directory: %w", err) return "", fmt.Errorf("failed to get vault directory: %w", err)
} }
return vault, filepath.Join(vaultDir, "unlockers.d", unlockerName), nil return filepath.Join(vaultDir, "unlockers.d", unlockerName), nil
} }
// CreatePGPUnlocker creates a new PGP unlocker and stores it in the vault. // CreatePGPUnlocker creates a new PGP unlocker and stores it in vault.
// It encrypts to the GPG key gpgKeyID and records fingerprint, that key's // It encrypts to the GPG key gpgKeyID and records fingerprint, that key's
// fingerprint as ResolveGPGKeyFingerprint returns it, in the metadata. // fingerprint as ResolveGPGKeyFingerprint returns it, in the metadata.
// Everything that can fail short of writing a file is done before anything // Everything that can fail short of writing a file is done before anything
// is written, and the files are written through WriteDir, so a failure // is written, and the files are written through WriteDir, so a failure
// leaves no partial unlocker. The long-term key comes from mnemonic when it // leaves no partial unlocker. The long-term key comes from vault's
// is not nil, else from the current unlocker, which is given passphrase when // GetOrDeriveLongTermKey: the key it holds when it is unlocked, else the
// it is a passphrase unlocker. // key from its mnemonic or its current unlocker.
func CreatePGPUnlocker( func CreatePGPUnlocker(
fs afero.Fs, stateDir, gpgKeyID, fingerprint string, vault VaultInterface, gpgKeyID, fingerprint string,
mnemonic, passphrase *memguard.LockedBuffer,
) (*PGPUnlocker, error) { ) (*PGPUnlocker, error) {
err := checkGPGAvailable() err := checkGPGAvailable()
if err != nil { if err != nil {
return nil, err return nil, err
} }
vault, unlockerDir, err := pgpUnlockerDir(fs, stateDir) unlockerDir, err := pgpUnlockerDir(vault)
if err != nil { if err != nil {
return nil, err return nil, err
} }
// The vault's GetOrDeriveLongTermKey, in step 2, uses both fs := vault.GetFilesystem()
vault.SetMnemonic(mnemonic)
vault.SetUnlockPassphrase(passphrase)
// Step 1: Generate a new age keypair for the PGP unlocker // Step 1: Generate a new age keypair for the PGP unlocker
ageIdentity, err := age.GenerateX25519Identity() ageIdentity, err := age.GenerateX25519Identity()
@@ -295,7 +282,7 @@ func CreatePGPUnlocker(
func encryptPGPUnlockerKeys( func encryptPGPUnlockerKeys(
vault VaultInterface, ageIdentity *age.X25519Identity, gpgKeyID string, vault VaultInterface, ageIdentity *age.X25519Identity, gpgKeyID string,
) ([]byte, []byte, error) { ) ([]byte, []byte, error) {
// From the mnemonic or the current unlocker, as for a passphrase unlocker // The key vault holds, or else from its mnemonic or current unlocker
ltIdentity, err := vault.GetOrDeriveLongTermKey() ltIdentity, err := vault.GetOrDeriveLongTermKey()
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("failed to get long-term key: %w", err) return nil, nil, fmt.Errorf("failed to get long-term key: %w", err)
@@ -388,7 +375,8 @@ func ResolveGPGKeyFingerprint(keyID string) (string, error) {
return "", fmt.Errorf("%w: %s", ErrGPGKeyNotFound, keyID) return "", fmt.Errorf("%w: %s", ErrGPGKeyNotFound, keyID)
} }
return "", fmt.Errorf("failed to resolve GPG key fingerprint: %w", err) return "", fmt.Errorf(
"failed to resolve GPG key fingerprint: %w", withGPGStderr(err))
} }
// Parse the output to extract the fingerprint // Parse the output to extract the fingerprint
@@ -418,6 +406,24 @@ func checkGPGAvailable() error {
return nil return nil
} }
// withGPGStderr appends gpg's stderr, which says why gpg failed, to err
// from cmd.Output. cmd.Output keeps that stderr in the *exec.ExitError.
// err is returned unchanged when gpg wrote nothing to stderr.
func withGPGStderr(err error) error {
var exitErr *exec.ExitError
if !errors.As(err, &exitErr) {
return err
}
stderr := strings.TrimSpace(string(exitErr.Stderr))
if stderr == "" {
return err
}
return fmt.Errorf("%w: %s", err, stderr)
}
// gpgEncryptDefault is the default implementation of GPG encryption // gpgEncryptDefault is the default implementation of GPG encryption
func gpgEncryptDefault(data *memguard.LockedBuffer, keyID string) ([]byte, error) { func gpgEncryptDefault(data *memguard.LockedBuffer, keyID string) ([]byte, error) {
if data == nil { if data == nil {
@@ -437,7 +443,7 @@ func gpgEncryptDefault(data *memguard.LockedBuffer, keyID string) ([]byte, error
output, err := cmd.Output() output, err := cmd.Output()
if err != nil { if err != nil {
return nil, fmt.Errorf("GPG encryption failed: %w", err) return nil, fmt.Errorf("GPG encryption failed: %w", withGPGStderr(err))
} }
return output, nil return output, nil
@@ -450,7 +456,7 @@ func gpgDecryptDefault(encryptedData []byte) (*memguard.LockedBuffer, error) {
output, err := cmd.Output() output, err := cmd.Output()
if err != nil { if err != nil {
return nil, fmt.Errorf("GPG decryption failed: %w", err) return nil, fmt.Errorf("GPG decryption failed: %w", withGPGStderr(err))
} }
// Create a secure buffer for the decrypted data // Create a secure buffer for the decrypted data
+29 -6
View File
@@ -58,7 +58,8 @@ func TestCreatePGPUnlockerFailureWritesNothing(t *testing.T) {
}} }}
_, err = secret.CreatePGPUnlocker( _, err = secret.CreatePGPUnlocker(
fs, testVaultStateDir, testGPGKeyID, testGPGFingerprint, nil, nil) vault.NewVault(fs, testVaultStateDir, testVaultName),
testGPGKeyID, testGPGFingerprint)
require.Error(t, err) require.Error(t, err)
vaultDir, err := vlt.GetDirectory() vaultDir, err := vlt.GetDirectory()
@@ -87,18 +88,40 @@ func TestPGPUnlockerAddedTwiceKeepsFirst(t *testing.T) {
fs := afero.NewMemMapFs() fs := afero.NewMemMapFs()
mnemonic := testMnemonicBuffer(t) mnemonic := testMnemonicBuffer(t)
_, err := vault.CreateVault(fs, testVaultStateDir, testVaultName, mnemonic, nil) vlt, err := vault.CreateVault(fs, testVaultStateDir, testVaultName, mnemonic, nil)
require.NoError(t, err) require.NoError(t, err)
first, err := secret.CreatePGPUnlocker( first, err := secret.CreatePGPUnlocker(vlt, testGPGKeyID, testGPGFingerprint)
fs, testVaultStateDir, testGPGKeyID, testGPGFingerprint, mnemonic, nil)
require.NoError(t, err) require.NoError(t, err)
firstFiles := dirFiles(t, fs, first.GetDirectory()) firstFiles := dirFiles(t, fs, first.GetDirectory())
second, err := secret.CreatePGPUnlocker( second, err := secret.CreatePGPUnlocker(vlt, testGPGKeyID, testGPGFingerprint)
fs, testVaultStateDir, testGPGKeyID, testGPGFingerprint, mnemonic, nil)
require.NoError(t, err) require.NoError(t, err)
assert.NotEqual(t, first.GetDirectory(), second.GetDirectory()) assert.NotEqual(t, first.GetDirectory(), second.GetDirectory())
assert.Equal(t, firstFiles, dirFiles(t, fs, first.GetDirectory())) assert.Equal(t, firstFiles, dirFiles(t, fs, first.GetDirectory()))
} }
// TestGPGDecryptErrorHasGPGReason runs the real gpg on data that is not an
// OpenPGP message and checks that the error carries the reason gpg writes
// to stderr.
func TestGPGDecryptErrorHasGPGReason(t *testing.T) {
// The empty GNUPGHOME keeps gpg away from the user's own keyring;
// LC_ALL=C keeps gpg's message in English.
t.Setenv("GNUPGHOME", t.TempDir())
t.Setenv("LC_ALL", "C")
_, err := secret.GPGDecryptFunc([]byte("not an OpenPGP message"))
require.ErrorContains(t, err, "gpg: no valid OpenPGP data found")
}
// TestGPGDecryptErrorWithoutGPGStderr checks that when gpg fails without
// writing to stderr, the error ends with the exit status and nothing after it.
//
//nolint:paralleltest // installFakeGPG uses t.Setenv
func TestGPGDecryptErrorWithoutGPGStderr(t *testing.T) {
installFakeGPG(t)
_, err := secret.GPGDecryptFunc([]byte("not an OpenPGP message"))
require.EqualError(t, err, "GPG decryption failed: exit status 1")
}
+38 -20
View File
@@ -2,10 +2,11 @@ package secret
import ( import (
"errors" "errors"
"fmt"
"log/slog" "log/slog"
"os"
"path/filepath" "path/filepath"
"strings" "strings"
"time"
"filippo.io/age" "filippo.io/age"
"github.com/awnumar/memguard" "github.com/awnumar/memguard"
@@ -27,11 +28,6 @@ type VaultInterface interface {
GetFilesystem() afero.Fs GetFilesystem() afero.Fs
GetCurrentUnlocker() (Unlocker, error) GetCurrentUnlocker() (Unlocker, error)
GetOrDeriveLongTermKey() (*age.X25519Identity, error) GetOrDeriveLongTermKey() (*age.X25519Identity, error)
// SetMnemonic and SetUnlockPassphrase give GetOrDeriveLongTermKey the
// mnemonic to derive the long-term key from, and the passphrase for a
// current passphrase unlocker; nil for none.
SetMnemonic(mnemonic *memguard.LockedBuffer)
SetUnlockPassphrase(passphrase *memguard.LockedBuffer)
CreatePassphraseUnlocker( CreatePassphraseUnlocker(
passphrase *memguard.LockedBuffer) (*PassphraseUnlocker, error) passphrase *memguard.LockedBuffer) (*PassphraseUnlocker, error)
} }
@@ -66,31 +62,53 @@ func NewSecret(vault VaultInterface, name string) *Secret {
Name: name, Name: name,
Directory: secretDir, Directory: secretDir,
vault: vault, vault: vault,
Metadata: Metadata{
CreatedAt: time.Now(),
UpdatedAt: time.Now(),
},
} }
} }
// LoadMetadata is deprecated - metadata is now per-version and encrypted // LoadMetadata sets the secret's times from the modification times of its
// files, which can be read without unlocking the vault. UpdatedAt is when
// the secret's current file was last written, by an add or a promote.
// CreatedAt is when its oldest version's value.age was written. That file
// is never rewritten; the version's directory is no use, as its time
// changes when the next add rewrites the version's metadata.age.
func (s *Secret) LoadMetadata() error { func (s *Secret) LoadMetadata() error {
Debug("LoadMetadata called but is deprecated in versioned model", fs := s.vault.GetFilesystem()
"secret_name", s.Name)
// For backward compatibility, we'll populate with basic info current, err := fs.Stat(filepath.Join(s.Directory, "current"))
now := time.Now() if err != nil {
return fmt.Errorf("failed to read when secret %s was updated: %w",
s.Name, err)
}
versions, err := ListVersions(fs, s.Directory)
if err != nil {
return err
}
if len(versions) == 0 {
return fmt.Errorf("secret %s has no versions: %w", s.Name, os.ErrNotExist)
}
// ListVersions lists the newest version first
oldest := versions[len(versions)-1]
value, err := fs.Stat(
filepath.Join(s.Directory, "versions", oldest, "value.age"))
if err != nil {
return fmt.Errorf("failed to read when secret %s was created: %w",
s.Name, err)
}
s.Metadata = Metadata{ s.Metadata = Metadata{
CreatedAt: now, CreatedAt: value.ModTime(),
UpdatedAt: now, UpdatedAt: current.ModTime(),
} }
return nil return nil
} }
// GetMetadata returns the secret metadata (deprecated) // GetMetadata returns the times LoadMetadata read
func (s *Secret) GetMetadata() Metadata { func (s *Secret) GetMetadata() Metadata {
Debug("GetMetadata called but is deprecated in versioned model", "secret_name", s.Name)
return s.Metadata return s.Metadata
} }
-6
View File
@@ -118,12 +118,6 @@ func (m *MockVault) GetOrDeriveLongTermKey() (*age.X25519Identity, error) {
return nil, errNotImplementedInMock return nil, errNotImplementedInMock
} }
func (m *MockVault) SetMnemonic(mnemonic *memguard.LockedBuffer) {
m.mnemonic = mnemonic
}
func (m *MockVault) SetUnlockPassphrase(_ *memguard.LockedBuffer) {}
func (m *MockVault) CreatePassphraseUnlocker( func (m *MockVault) CreatePassphraseUnlocker(
_ *memguard.LockedBuffer, _ *memguard.LockedBuffer,
) (*PassphraseUnlocker, error) { ) (*PassphraseUnlocker, error) {
+2 -1
View File
@@ -32,7 +32,8 @@ type VersionMetadata struct {
ID string `json:"id"` // ULID ID string `json:"id"` // ULID
CreatedAt *time.Time `json:"createdAt,omitempty"` // When version was created CreatedAt *time.Time `json:"createdAt,omitempty"` // When version was created
NotBefore *time.Time `json:"notBefore,omitempty"` // When this version becomes active NotBefore *time.Time `json:"notBefore,omitempty"` // When this version becomes active
NotAfter *time.Time `json:"notAfter,omitempty"` // Expiry (nil = current) // NotAfter is set when another version is added while this one is current
NotAfter *time.Time `json:"notAfter,omitempty"`
} }
// Version represents a version of a secret // Version represents a version of a secret
-4
View File
@@ -90,10 +90,6 @@ func (m *MockVersionVault) GetOrDeriveLongTermKey() (*age.X25519Identity, error)
return nil, errNotImplementedInMock return nil, errNotImplementedInMock
} }
func (m *MockVersionVault) SetMnemonic(_ *memguard.LockedBuffer) {}
func (m *MockVersionVault) SetUnlockPassphrase(_ *memguard.LockedBuffer) {}
func (m *MockVersionVault) CreatePassphraseUnlocker( func (m *MockVersionVault) CreatePassphraseUnlocker(
_ *memguard.LockedBuffer, _ *memguard.LockedBuffer,
) (*secret.PassphraseUnlocker, error) { ) (*secret.PassphraseUnlocker, error) {
+1 -1
View File
@@ -132,7 +132,7 @@ func TestGetSecretWithWrongMnemonic(t *testing.T) {
t.Cleanup(mnemonic.Destroy) t.Cleanup(mnemonic.Destroy)
vlt := vault.NewVault(created.GetFilesystem(), testStateDir, testVaultName) vlt := vault.NewVault(created.GetFilesystem(), testStateDir, testVaultName)
vlt.SetMnemonic(mnemonic) vlt.Mnemonic = mnemonic
_, err := vlt.GetSecret(testSecretName) _, err := vlt.GetSecret(testSecretName)
require.ErrorIs(t, err, vault.ErrMnemonicMismatch) require.ErrorIs(t, err, vault.ErrMnemonicMismatch)
+56 -26
View File
@@ -55,7 +55,10 @@ func ValidateVaultName(name string) error {
} }
// ResolveVaultSymlink reads the currentvault file to get the path to the current vault // ResolveVaultSymlink reads the currentvault file to get the path to the current vault
// The file contains just the vault name (e.g., "default") // The file contains just the vault name (e.g., "default"). An empty or
// invalid name, or the name of a vault that does not exist, is an error
// wrapping ErrVaultNotFound that names the file and says to run
// 'secret vault select <name>'.
func ResolveVaultSymlink(fs afero.Fs, currentVaultPath string) (string, error) { func ResolveVaultSymlink(fs afero.Fs, currentVaultPath string) (string, error) {
secret.Debug("resolveVaultSymlink starting", "path", currentVaultPath) secret.Debug("resolveVaultSymlink starting", "path", currentVaultPath)
@@ -74,6 +77,21 @@ func ResolveVaultSymlink(fs afero.Fs, currentVaultPath string) (string, error) {
stateDir := filepath.Dir(currentVaultPath) stateDir := filepath.Dir(currentVaultPath)
absolutePath := filepath.Join(stateDir, "vaults.d", vaultName) absolutePath := filepath.Join(stateDir, "vaults.d", vaultName)
// An empty name would make vaults.d itself the vault's directory
exists := false
if isValidVaultName(vaultName) {
exists, err = afero.DirExists(fs, absolutePath)
if err != nil {
return "", fmt.Errorf("failed to check if vault exists: %w", err)
}
}
if !exists {
return "", fmt.Errorf("vault '%s' named in %s %w; run "+
"'secret vault select <name>' to select a vault",
vaultName, currentVaultPath, ErrVaultNotFound)
}
secret.Debug("Resolved to absolute path", "absolute_path", absolutePath) secret.Debug("Resolved to absolute path", "absolute_path", absolutePath)
return absolutePath, nil return absolutePath, nil
@@ -140,8 +158,9 @@ func ListVaults(fs afero.Fs, stateDir string) ([]string, error) {
return nil, fmt.Errorf("failed to read vaults directory: %w", err) return nil, fmt.Errorf("failed to read vaults directory: %w", err)
} }
// Extract vault names // Extract vault names. Not nil, so that no vaults prints [] in JSON,
var vaults []string // not null.
vaults := make([]string, 0, len(entries))
for _, entry := range entries { for _, entry := range entries {
if entry.IsDir() { if entry.IsDir() {
@@ -212,16 +231,40 @@ func processMnemonicForVault(
return ltIdentity, derivationIndex, publicKeyHash, familyHash, nil return ltIdentity, derivationIndex, publicKeyHash, familyHash, nil
} }
// CheckNewVaultName returns an error when name cannot be the name of a new
// vault: one wrapping ErrInvalidVaultName when it is not a valid vault name,
// and one wrapping ErrVaultExists when that vault already exists. Creating an
// existing vault again would replace its keys, and its secrets could no
// longer be decrypted. The commands that create a vault check it before they
// ask for the mnemonic or passphrase, and CreateVault checks it again, under
// the state directory lock, before writing anything.
func CheckNewVaultName(fs afero.Fs, stateDir string, name string) error {
err := ValidateVaultName(name)
if err != nil {
return err
}
exists, err := afero.DirExists(fs, filepath.Join(stateDir, "vaults.d", name))
if err != nil {
return fmt.Errorf("failed to check if vault exists: %w", err)
}
if exists {
return fmt.Errorf("vault %s %w", name, ErrVaultExists)
}
return nil
}
// CreateVault creates a new vault and selects it as the current vault. When // CreateVault creates a new vault and selects it as the current vault. When
// mnemonic is not nil, the vault's long-term key is derived from it, and the // mnemonic is not nil, the vault's long-term key is derived from it, and the
// returned vault has it as its Mnemonic; when it is nil, the vault has no // returned vault has it as its Mnemonic; when it is nil, the vault has no
// long-term key until one is imported. When passphrase is not nil, the vault // long-term key. When passphrase is not nil, the vault gets a passphrase
// gets a passphrase unlocker protected by it, as its current unlocker; that // unlocker protected by it, as its current unlocker; that needs a mnemonic.
// needs a mnemonic. It refuses a vault that already exists before writing // It refuses an invalid name or a vault that already exists, as
// anything: creating it again would replace its keys, and its secrets could // CheckNewVaultName does, before writing anything. The commands that call it
// no longer be decrypted. The commands that call it hold the state directory // hold the state directory lock, so no other command can create the vault
// lock, so no other command can create the vault between the check and the // between the check and the writes.
// writes.
// //
// The vault is written whole into a temporary directory, which is renamed // The vault is written whole into a temporary directory, which is renamed
// into vaults.d only once complete, and only then selected: a crash at any // into vaults.d only once complete, and only then selected: a crash at any
@@ -233,30 +276,17 @@ func CreateVault(
) (*Vault, error) { ) (*Vault, error) {
secret.Debug("Creating new vault", "name", name, "state_dir", stateDir) secret.Debug("Creating new vault", "name", name, "state_dir", stateDir)
err := ValidateVaultName(name) err := CheckNewVaultName(fs, stateDir, name)
if err != nil { if err != nil {
secret.Debug("Invalid vault name provided", "vault_name", name)
return nil, err return nil, err
} }
secret.Debug("Vault name validation passed", "vault_name", name)
vaultDir := filepath.Join(stateDir, "vaults.d", name)
exists, err := afero.DirExists(fs, vaultDir)
if err != nil {
return nil, fmt.Errorf("failed to check if vault exists: %w", err)
}
if exists {
return nil, fmt.Errorf("vault %s %w", name, ErrVaultExists)
}
if passphrase != nil && mnemonic == nil { if passphrase != nil && mnemonic == nil {
return nil, fmt.Errorf("vault %s %w", name, ErrUnlockerWithoutMnemonic) return nil, fmt.Errorf("vault %s %w", name, ErrUnlockerWithoutMnemonic)
} }
vaultDir := filepath.Join(stateDir, "vaults.d", name)
secret.Debug("Creating vault directory structure", "vault_dir", vaultDir) secret.Debug("Creating vault directory structure", "vault_dir", vaultDir)
err = secret.WriteDir(fs, vaultDir, func(dir string) error { err = secret.WriteDir(fs, vaultDir, func(dir string) error {
+13 -6
View File
@@ -4,7 +4,9 @@ import (
"crypto/sha256" "crypto/sha256"
"encoding/hex" "encoding/hex"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"os"
"path/filepath" "path/filepath"
"github.com/spf13/afero" "github.com/spf13/afero"
@@ -33,7 +35,10 @@ func ComputeDoubleSHA256(data []byte) string {
} }
// GetNextDerivationIndex finds the next available derivation index for a given mnemonic // GetNextDerivationIndex finds the next available derivation index for a given mnemonic
// by deriving the public key for index 0 and using its hash to identify related vaults // by deriving the public key for index 0 and using its hash to identify related vaults.
// It fails when a vault's metadata file exists but cannot be read or parsed: the
// index that vault uses is then unknown, and handing it out again would give a
// second vault the same long-term key.
func GetNextDerivationIndex( func GetNextDerivationIndex(
fs afero.Fs, stateDir string, mnemonic string, fs afero.Fs, stateDir string, mnemonic string,
) (uint32, error) { ) (uint32, error) {
@@ -72,21 +77,23 @@ func GetNextDerivationIndex(
continue continue
} }
// Try to read vault metadata
metadataPath := filepath.Join(vaultsDir, entry.Name(), "vault-metadata.json") metadataPath := filepath.Join(vaultsDir, entry.Name(), "vault-metadata.json")
metadataBytes, err := afero.ReadFile(fs, metadataPath) metadataBytes, err := afero.ReadFile(fs, metadataPath)
if err != nil { if errors.Is(err, os.ErrNotExist) {
// Skip vaults without metadata // Skip directories without metadata
continue continue
} }
if err != nil {
return 0, fmt.Errorf("failed to read vault metadata %s: %w", metadataPath, err)
}
var metadata Metadata var metadata Metadata
err = json.Unmarshal(metadataBytes, &metadata) err = json.Unmarshal(metadataBytes, &metadata)
if err != nil { if err != nil {
// Skip vaults with invalid metadata return 0, fmt.Errorf("failed to parse vault metadata %s: %w", metadataPath, err)
continue
} }
// Check if this vault uses the same mnemonic by comparing family hashes // Check if this vault uses the same mnemonic by comparing family hashes
+86 -85
View File
@@ -1,11 +1,15 @@
package vault_test package vault_test
import ( import (
"errors"
"path/filepath" "path/filepath"
"strings" "strings"
"testing" "testing"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/secret/internal/secret"
"sneak.berlin/go/secret/internal/vault" "sneak.berlin/go/secret/internal/vault"
"sneak.berlin/go/secret/pkg/agehd" "sneak.berlin/go/secret/pkg/agehd"
) )
@@ -163,6 +167,88 @@ func testGetNextDerivationIndex(t *testing.T, fs afero.Fs) {
} }
} }
// errMetadataUnreadable is returned by the test filesystem in place of a
// successful open of one vault's metadata file.
var errMetadataUnreadable = errors.New("input/output error")
// metadataReadFailFs fails every open of the file at unreadablePath. The
// file still exists, so checking for it succeeds and only reading it fails.
type metadataReadFailFs struct {
afero.Fs
unreadablePath string
}
//nolint:ireturn // afero.File is the interface required by afero.Fs
func (f *metadataReadFailFs) Open(name string) (afero.File, error) {
if name == f.unreadablePath {
return nil, errMetadataUnreadable
}
return f.Fs.Open(name)
}
// TestCreateVaultWithCorruptMetadataFails checks that creating a vault from
// a mnemonic fails, naming the file and leaving no new vault, when another
// vault's metadata file exists but cannot be parsed or read. That vault's
// derivation index is then unknown, so the new vault could get the same one
// and the same key.
func TestCreateVaultWithCorruptMetadataFails(t *testing.T) {
t.Parallel()
vaultsDir := filepath.Join(testStateDir, "vaults.d")
metadataPath := filepath.Join(vaultsDir, "work", "vault-metadata.json")
tests := []struct {
name string
// breakMetadata makes the work vault's metadata file unusable on
// base and returns the filesystem to create the second vault on.
breakMetadata func(t *testing.T, base afero.Fs) afero.Fs
}{
{
name: "parsing the file fails",
breakMetadata: func(t *testing.T, base afero.Fs) afero.Fs {
t.Helper()
err := afero.WriteFile(base, metadataPath, []byte("{"),
secret.FilePerms)
require.NoError(t, err)
return base
},
},
{
name: "reading the file fails",
breakMetadata: func(_ *testing.T, base afero.Fs) afero.Fs {
return &metadataReadFailFs{Fs: base, unreadablePath: metadataPath}
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
base := afero.NewMemMapFs()
_, err := vault.CreateVault(base, testStateDir, "work",
testMnemonicBuffer(t), nil)
require.NoError(t, err)
fs := tt.breakMetadata(t, base)
_, err = vault.CreateVault(fs, testStateDir, "other",
testMnemonicBuffer(t), nil)
require.ErrorContains(t, err, metadataPath)
entries, err := afero.ReadDir(base, vaultsDir)
require.NoError(t, err)
require.Len(t, entries, 1)
assert.Equal(t, "work", entries[0].Name())
})
}
}
func testMetadataPersistence(t *testing.T, fs afero.Fs) { func testMetadataPersistence(t *testing.T, fs afero.Fs) {
t.Helper() t.Helper()
@@ -296,91 +382,6 @@ func TestSampleHashCalculation(t *testing.T) {
} }
} }
func TestWorkflowMismatch(t *testing.T) {
t.Parallel()
// Create a temporary directory for testing
tempDir := t.TempDir()
fs := afero.NewOsFs()
// Test Case 1: Create vault WITH mnemonic (like init command)
_, err := vault.CreateVault(fs, tempDir, "default", testMnemonicBuffer(t), nil)
if err != nil {
t.Fatalf("Failed to create vault with mnemonic: %v", err)
}
// Load metadata for vault1
vault1Dir := filepath.Join(tempDir, "vaults.d", "default")
metadata1, err := vault.LoadVaultMetadata(fs, vault1Dir)
if err != nil {
t.Fatalf("Failed to load vault1 metadata: %v", err)
}
t.Logf("Vault1 (with mnemonic) - DerivationIndex: %d, PublicKeyHash: %s",
metadata1.DerivationIndex, metadata1.PublicKeyHash)
// Test Case 2: Create vault WITHOUT mnemonic, then import (work vault)
_, err = vault.CreateVault(fs, tempDir, "work", nil, nil)
if err != nil {
t.Fatalf("Failed to create vault without mnemonic: %v", err)
}
vault2Dir := filepath.Join(tempDir, "vaults.d", "work")
// Simulate the vault import process: get the next available derivation
// index for this mnemonic
derivationIndex, err := vault.GetNextDerivationIndex(fs, tempDir, testMnemonic)
if err != nil {
t.Fatalf("Failed to get next derivation index: %v", err)
}
t.Logf("Next derivation index for import: %d", derivationIndex)
// Calculate public key hash from index 0 (same as in VaultImport)
identity0, err := agehd.DeriveIdentity(testMnemonic, 0)
if err != nil {
t.Fatalf("Failed to derive identity for index 0: %v", err)
}
publicKeyHash := vault.ComputeDoubleSHA256(
[]byte(identity0.Recipient().String()))
// Load existing metadata and update it (same as in VaultImport)
existingMetadata, err := vault.LoadVaultMetadata(fs, vault2Dir)
if err != nil {
t.Fatalf("Failed to load existing metadata: %v", err)
}
// Update metadata with new derivation info
existingMetadata.DerivationIndex = derivationIndex
existingMetadata.PublicKeyHash = publicKeyHash
err = vault.SaveVaultMetadata(fs, vault2Dir, existingMetadata)
if err != nil {
t.Fatalf("Failed to save vault metadata: %v", err)
}
// Load updated metadata for vault2
metadata2, err := vault.LoadVaultMetadata(fs, vault2Dir)
if err != nil {
t.Fatalf("Failed to load vault2 metadata: %v", err)
}
t.Logf("Vault2 (imported mnemonic) - DerivationIndex: %d, PublicKeyHash: %s",
metadata2.DerivationIndex, metadata2.PublicKeyHash)
// Verify that both vaults have the same public key hash
if metadata1.PublicKeyHash != metadata2.PublicKeyHash {
t.Errorf("Public key hashes don't match!")
t.Logf("Vault1 hash: %s", metadata1.PublicKeyHash)
t.Logf("Vault2 hash: %s", metadata2.PublicKeyHash)
} else {
t.Logf("SUCCESS: Both vaults have the same public key hash: %s",
metadata1.PublicKeyHash)
}
}
func TestReverseEngineerHash(t *testing.T) { func TestReverseEngineerHash(t *testing.T) {
t.Parallel() t.Parallel()
+44 -15
View File
@@ -427,7 +427,8 @@ func (v *Vault) UnlockVault() (*age.X25519Identity, error) {
return longTermIdentity, nil return longTermIdentity, nil
} }
// GetSecretObject retrieves a Secret object with metadata loaded from this vault // GetSecretObject retrieves a Secret object from this vault. Its times are
// zero until its LoadMetadata reads them.
func (v *Vault) GetSecretObject(name string) (*secret.Secret, error) { func (v *Vault) GetSecretObject(name string) (*secret.Secret, error) {
err := ValidateSecretName(name) err := ValidateSecretName(name)
if err != nil { if err != nil {
@@ -454,16 +455,7 @@ func (v *Vault) GetSecretObject(name string) (*secret.Secret, error) {
return nil, fmt.Errorf("secret %s %w", name, ErrSecretNotFound) return nil, fmt.Errorf("secret %s %w", name, ErrSecretNotFound)
} }
// Create a Secret object return secret.NewSecret(v, name), nil
secretObj := secret.NewSecret(v, name)
// Load the metadata from disk
err = secretObj.LoadMetadata()
if err != nil {
return nil, err
}
return secretObj, nil
} }
// CopySecretVersion copies a single version from source into destSecretDir // CopySecretVersion copies a single version from source into destSecretDir
@@ -581,8 +573,8 @@ func (v *Vault) CopySecretAllVersions(
} }
// Copy each version and the current pointer, then move the copy into place // Copy each version and the current pointer, then move the copy into place
err = v.copyVersions(srcVault, srcIdentity, err = v.copyVersions(srcVault, srcIdentity, srcSecretName, srcSecretDir,
srcSecretName, destSecretName, destSecretDir, versions, currentVersion) destSecretName, destSecretDir, versions, currentVersion)
if err != nil { if err != nil {
return err return err
} }
@@ -801,10 +793,13 @@ func (v *Vault) createAndSaveVersion(
// pointer into a temporary directory, then moves that directory to // pointer into a temporary directory, then moves that directory to
// destSecretDir, replacing a secret already there. Nothing in this vault // destSecretDir, replacing a secret already there. Nothing in this vault
// changes until the copy is complete, so an interrupted copy leaves only a // changes until the copy is complete, so an interrupted copy leaves only a
// temporary directory behind. // temporary directory behind. Each copied value.age and current file gets
// the modification time of the file it was copied from, so `secret list`,
// which reads a secret's times from them, lists the copy with the times of
// the source.
func (v *Vault) copyVersions( func (v *Vault) copyVersions(
srcVault *Vault, srcIdentity *age.X25519Identity, srcVault *Vault, srcIdentity *age.X25519Identity,
srcSecretName, destSecretName, destSecretDir string, srcSecretName, srcSecretDir, destSecretName, destSecretDir string,
versions []string, currentVersion string, versions []string, currentVersion string,
) error { ) error {
buildDir, err := secret.TempDirFor(v.fs, destSecretDir) buildDir, err := secret.TempDirFor(v.fs, destSecretDir)
@@ -823,6 +818,14 @@ func (v *Vault) copyVersions(
if err != nil { if err != nil {
return fmt.Errorf("failed to copy version %s: %w", versionName, err) return fmt.Errorf("failed to copy version %s: %w", versionName, err)
} }
valuePath := filepath.Join("versions", versionName, "value.age")
err = copyModTime(srcVault.fs, filepath.Join(srcSecretDir, valuePath),
v.fs, filepath.Join(buildDir, valuePath))
if err != nil {
return err
}
} }
err = secret.SetCurrentVersion(v.fs, buildDir, currentVersion) err = secret.SetCurrentVersion(v.fs, buildDir, currentVersion)
@@ -830,6 +833,12 @@ func (v *Vault) copyVersions(
return fmt.Errorf("failed to set current version: %w", err) return fmt.Errorf("failed to set current version: %w", err)
} }
err = copyModTime(srcVault.fs, filepath.Join(srcSecretDir, "current"),
v.fs, filepath.Join(buildDir, "current"))
if err != nil {
return err
}
// With --force, the secret being replaced goes only now that its // With --force, the secret being replaced goes only now that its
// replacement is complete // replacement is complete
exists, err := afero.DirExists(v.fs, destSecretDir) exists, err := afero.DirExists(v.fs, destSecretDir)
@@ -854,6 +863,26 @@ func (v *Vault) copyVersions(
return nil return nil
} }
// copyModTime gives destPath in destFs the modification time of srcPath in
// srcFs.
func copyModTime(
srcFs afero.Fs, srcPath string, destFs afero.Fs, destPath string,
) error {
info, err := srcFs.Stat(srcPath)
if err != nil {
return fmt.Errorf("failed to read modification time of %s: %w",
srcPath, err)
}
err = destFs.Chtimes(destPath, info.ModTime(), info.ModTime())
if err != nil {
return fmt.Errorf("failed to set modification time of %s: %w",
destPath, err)
}
return nil
}
// checkCopyDestination refuses to copy over an existing secret unless force // checkCopyDestination refuses to copy over an existing secret unless force
// is set. A secret being replaced is removed by copyVersions, once its // is set. A secret being replaced is removed by copyVersions, once its
// replacement is complete. // replacement is complete.
+1 -1
View File
@@ -336,7 +336,7 @@ func TestVaultVersionTimestampsAfterUnlock(t *testing.T) {
// A new instance is locked; AddSecret unlocks it with the passphrase // A new instance is locked; AddSecret unlocks it with the passphrase
vault := NewVault(fs, testStateDir, "test") vault := NewVault(fs, testStateDir, "test")
vault.SetUnlockPassphrase(passphrase) vault.UnlockPassphrase = passphrase
addTestSecretToVault(t, vault, testSecretPath, []byte("version-2"), true) addTestSecretToVault(t, vault, testSecretPath, []byte("version-2"), true)
require.False(t, fs.readAt.IsZero(), "AddSecret did not unlock the vault") require.False(t, fs.readAt.IsZero(), "AddSecret did not unlock the vault")
+5 -14
View File
@@ -64,18 +64,6 @@ func (v *Vault) ClearLongTermKey() {
v.longTermKey = nil v.longTermKey = nil
} }
// SetMnemonic sets v.Mnemonic, for code that has v only as a
// secret.VaultInterface.
func (v *Vault) SetMnemonic(mnemonic *memguard.LockedBuffer) {
v.Mnemonic = mnemonic
}
// SetUnlockPassphrase sets v.UnlockPassphrase, for code that has v only as
// a secret.VaultInterface.
func (v *Vault) SetUnlockPassphrase(passphrase *memguard.LockedBuffer) {
v.UnlockPassphrase = passphrase
}
// GetOrDeriveLongTermKey gets the long-term key from memory or derives it // GetOrDeriveLongTermKey gets the long-term key from memory or derives it
// from available sources // from available sources
func (v *Vault) GetOrDeriveLongTermKey() (*age.X25519Identity, error) { func (v *Vault) GetOrDeriveLongTermKey() (*age.X25519Identity, error) {
@@ -291,10 +279,13 @@ func (v *Vault) unlockLongTermKey(
// without a mnemonic records none, and without its metadata the key cannot // without a mnemonic records none, and without its metadata the key cannot
// be derived. It is not added when the passphrase could not be read: the // be derived. It is not added when the passphrase could not be read: the
// unlocker was not tried, and adding one would need a passphrase read the // unlocker was not tried, and adding one would need a passphrase read the
// same way. Nor is it added when the current unlocker does not exist and the // same way. Nor is it added when the passphrase is wrong: the unlocker still
// vault has others: the error already says to select one of them. // opens the vault with the right one. Nor is it added when the current
// unlocker does not exist and the vault has others: the error already says
// to select one of them.
func (v *Vault) withMnemonicAdvice(err error) error { func (v *Vault) withMnemonicAdvice(err error) error {
if errors.Is(err, secret.ErrPassphraseNotRead) || if errors.Is(err, secret.ErrPassphraseNotRead) ||
errors.Is(err, secret.ErrWrongPassphrase) ||
errors.Is(err, ErrUnlockerNotFound) { errors.Is(err, ErrUnlockerNotFound) {
return err return err
} }
+2 -5
View File
@@ -5,7 +5,6 @@ import (
"path/filepath" "path/filepath"
"testing" "testing"
"filippo.io/age"
"github.com/awnumar/memguard" "github.com/awnumar/memguard"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
@@ -129,7 +128,7 @@ func TestAddSecretForceWithWrongPassphraseChangesNothing(t *testing.T) {
t.Cleanup(wrong.Destroy) t.Cleanup(wrong.Destroy)
vlt := vault.NewVault(fs, testStateDir, testVaultName) vlt := vault.NewVault(fs, testStateDir, testVaultName)
vlt.SetUnlockPassphrase(wrong) vlt.UnlockPassphrase = wrong
secretDir := filepath.Join(testStateDir, "vaults.d", testVaultName, secretDir := filepath.Join(testStateDir, "vaults.d", testVaultName,
"secrets.d", testSecretName) "secrets.d", testSecretName)
@@ -140,9 +139,7 @@ func TestAddSecretForceWithWrongPassphraseChangesNothing(t *testing.T) {
err = vlt.AddSecret(testSecretName, second, true) err = vlt.AddSecret(testSecretName, second, true)
var noMatch *age.NoIdentityMatchError require.ErrorIs(t, err, secret.ErrWrongPassphrase)
require.ErrorAs(t, err, &noMatch)
assert.Equal(t, before, dirContents(t, fs, secretDir)) assert.Equal(t, before, dirContents(t, fs, secretDir))
} }
+1
View File
@@ -1,4 +1,5 @@
{ {
"private": true,
"devDependencies": { "devDependencies": {
"prettier": "3.8.1" "prettier": "3.8.1"
} }
+46 -26
View File
@@ -2,18 +2,18 @@
# script/bootstrap: install all dependencies needed to build and develop # script/bootstrap: install all dependencies needed to build and develop
# this repo. Idempotent: every install is guarded by a check so already # this repo. Idempotent: every install is guarded by a check so already
# installed tools are skipped. Base tooling comes from nix, apt, brew, # installed tools are skipped. Base tooling comes from nix, apt, brew,
# or apk (detected in that order); assumes NOTHING is present (not git, # or apk (detected in that order); assumes nothing is present. The
# make, node, yarn, go, or python). Node is used directly if installed; # installed node is used only when its major version is the pinned one;
# otherwise a pinned version is installed via nvm (installing nvm # otherwise (node missing, or another major) the pinned version is
# itself first, from a hash-verified release archive, never curl | sh). # installed via nvm (installing nvm itself first, from a hash-verified
# release archive, never curl | sh), and yarn and the packages are
# installed under it.
# golangci-lint is never installed: script/lint runs it in docker. # golangci-lint is never installed: script/lint runs it in docker.
#
# Uncomment the language sections in main() that apply to this repo.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Pinned versions, 2026-07-06. Never "latest" or "lts"; exact versions. # Pinned versions, 2026-07-06
NODE_VERSION="22.17.0" NODE_VERSION="22.17.0"
NVM_VERSION="0.40.3" NVM_VERSION="0.40.3"
# sha256 of https://github.com/nvm-sh/nvm/archive/refs/tags/v0.40.3.tar.gz # sha256 of https://github.com/nvm-sh/nvm/archive/refs/tags/v0.40.3.tar.gz
@@ -22,6 +22,7 @@ YARN_VERSION="1.22.22"
PKGMGR="" PKGMGR=""
SUDO="" SUDO=""
APT_UPDATED=""
detect_pkgmgr() { detect_pkgmgr() {
[ -n "$PKGMGR" ] && return 0 [ -n "$PKGMGR" ] && return 0
@@ -50,7 +51,14 @@ pkg_install() {
detect_pkgmgr detect_pkgmgr
case "$PKGMGR" in case "$PKGMGR" in
nix) nix-env -iA "nixpkgs.$1" ;; nix) nix-env -iA "nixpkgs.$1" ;;
apt) $SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2" ;; apt)
# Package lists may be empty (fresh images); refresh once per run.
if [ -z "$APT_UPDATED" ]; then
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get update
APT_UPDATED=1
fi
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2"
;;
brew) brew install "$3" ;; brew) brew install "$3" ;;
apk) apk add --no-cache "$4" ;; apk) apk add --no-cache "$4" ;;
esac esac
@@ -82,7 +90,7 @@ nvm_sh() {
ensure_nvm() { ensure_nvm() {
[ -s "$HOME/.nvm/nvm.sh" ] && return 0 [ -s "$HOME/.nvm/nvm.sh" ] && return 0
# nvm prerequisites; nvm itself requires bash, so install it too # nvm prerequisites; nvm itself requires bash
if missing bash; then pkg_install bash bash bash bash; fi if missing bash; then pkg_install bash bash bash bash; fi
if missing curl; then pkg_install curl curl curl curl; fi if missing curl; then pkg_install curl curl curl curl; fi
if missing git; then pkg_install git git git git; fi if missing git; then pkg_install git git git git; fi
@@ -95,56 +103,68 @@ ensure_nvm() {
rm -rf "$tmp" rm -rf "$tmp"
} }
# True when the node on PATH has the pinned major version. Only the
# major is compared: the Dockerfile stages start from a node 22 alpine
# image whose exact version is not the pin, and nvm cannot install a
# prebuilt node on alpine. Another major is not used: the pinned yarn 1
# prints a deprecation warning under node 24. script/fmt and
# script/fmt-check carry this function unchanged, to run the yarn
# installed here.
node_is_pinned_major() {
if ! command -v node >/dev/null 2>&1; then return 1; fi
installed="$(node --version)"
installed="${installed#v}"
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
}
ensure_node() { ensure_node() {
if ! missing node; then return 0; fi if node_is_pinned_major; then
echo "bootstrap: using node $(node --version)"
return 0
fi
ensure_nvm ensure_nvm
nvm_sh "nvm install $NODE_VERSION" nvm_sh "nvm install $NODE_VERSION"
nvm_sh "nvm use $NODE_VERSION >/dev/null && \
echo \"bootstrap: using node \$(node --version)\""
} }
ensure_yarn() { ensure_yarn() {
if ! node_is_pinned_major; then
nvm_sh "nvm use $NODE_VERSION >/dev/null && corepack enable && \
corepack prepare yarn@$YARN_VERSION --activate"
return 0
fi
if ! missing yarn; then return 0; fi if ! missing yarn; then return 0; fi
if ! missing corepack; then if ! missing corepack; then
corepack enable corepack enable
corepack prepare "yarn@$YARN_VERSION" --activate corepack prepare "yarn@$YARN_VERSION" --activate
elif [ -s "$HOME/.nvm/nvm.sh" ]; then
nvm_sh "nvm use $NODE_VERSION >/dev/null && corepack enable && \
corepack prepare yarn@$YARN_VERSION --activate"
else else
npm install -g "yarn@$YARN_VERSION" npm install -g "yarn@$YARN_VERSION"
fi fi
} }
install_js_deps() { install_js_deps() {
if missing yarn && [ -s "$HOME/.nvm/nvm.sh" ]; then if node_is_pinned_major; then
yarn install --frozen-lockfile
else
nvm_sh "nvm use $NODE_VERSION >/dev/null && cd \"$ROOT\" && \ nvm_sh "nvm use $NODE_VERSION >/dev/null && cd \"$ROOT\" && \
yarn install --frozen-lockfile" yarn install --frozen-lockfile"
else
yarn install --frozen-lockfile
fi fi
} }
main() { main() {
cd "$ROOT" cd "$ROOT"
# Base tooling (every repo)
if missing git; then pkg_install git git git git; fi
if missing make; then pkg_install gnumake make make make; fi if missing make; then pkg_install gnumake make make make; fi
if missing git; then pkg_install git git git git; fi
# ---- JS / docs repos ----
# prettier, pinned in package.json and yarn.lock, formats the markdown
ensure_node ensure_node
ensure_yarn ensure_yarn
install_js_deps install_js_deps
# ---- Go repos ----
if missing go; then pkg_install go golang go go; fi if missing go; then pkg_install go golang go go; fi
go mod download go mod download
# ---- Python repos ----
# if missing python3; then pkg_install python3 python3 python3 python3; fi
# python3 -m venv .venv
# ./.venv/bin/pip install -e '.[dev]'
echo "bootstrap complete" echo "bootstrap complete"
} }
+13 -9
View File
@@ -2,9 +2,11 @@
# script/cibuild: run the CI build. It bootstraps first: a CI runner # script/cibuild: run the CI build. It bootstraps first: a CI runner
# checks out and runs this and nothing else, and script/fmt-check runs # checks out and runs this and nothing else, and script/fmt-check runs
# the formatter on the host, which a pristine checkout cannot do. # the formatter on the host, which a pristine checkout cannot do.
# --no-cache for the same reason as script/docker: the gate phases the # A new CACHEBUST value reruns every RUN step, since a cached gate phase
# final stage depends on are RUN steps, and a cached one is a check that # is a check that did not run; unlike --no-cache, it keeps Go's build cache.
# did not run. # This relies on every stage of the Dockerfile declaring ARG CACHEBUST
# first; a stage without that line is served from cache on an unchanged
# tree, with no warning.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -14,15 +16,17 @@ main() {
cd "$ROOT" cd "$ROOT"
"$SCRIPT_DIR/bootstrap" "$SCRIPT_DIR/bootstrap"
"$SCRIPT_DIR/check" "$SCRIPT_DIR/check"
# Own line: a failing command substitution inside an argument does # The version and the tag each get their own line: a failing
# not trip `set -e`, so the inline form degrades silently to an # command substitution inside an argument does not trip `set -e`,
# empty constant. The VERSION build argument takes precedence over # so the inline form degrades silently to an empty constant. The
# the version a build stage derives from the .git in the context. # VERSION build argument takes precedence over the version a build
# stage derives from the .git in the context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)" version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
docker build --no-cache \ tag="$("$SCRIPT_DIR/projectname")"
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--build-arg VERSION="$version" \ --build-arg VERSION="$version" \
-t "$("$SCRIPT_DIR/projectname")" . -t "$tag" .
} }
main "$@" main "$@"
+13 -8
View File
@@ -1,8 +1,11 @@
#!/bin/sh #!/bin/sh
# script/docker: build the Docker image tagged with the project name. # script/docker: build the Docker image tagged with the project name.
# Identical in all repos; the tag comes from script/projectname. # Identical in all repos; the tag comes from script/projectname.
# --no-cache because the gate phases the final stage depends on are RUN # A new CACHEBUST value reruns every RUN step, since a cached gate phase
# steps, and a cached one is a check that did not run. # is a check that did not run; unlike --no-cache, it keeps Go's build cache.
# This relies on every stage of the Dockerfile declaring ARG CACHEBUST
# first; a stage without that line is served from cache on an unchanged
# tree, with no warning.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -10,15 +13,17 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
# Own line: a failing command substitution inside an argument does # The version and the tag each get their own line: a failing
# not trip `set -e`, so the inline form degrades silently to an # command substitution inside an argument does not trip `set -e`,
# empty constant. The VERSION build argument takes precedence over # so the inline form degrades silently to an empty constant. The
# the version a build stage derives from the .git in the context. # VERSION build argument takes precedence over the version a build
# stage derives from the .git in the context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)" version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
docker build --no-cache \ tag="$("$SCRIPT_DIR/projectname")"
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--build-arg VERSION="$version" \ --build-arg VERSION="$version" \
-t "$("$SCRIPT_DIR/projectname")" . -t "$tag" .
} }
main "$@" main "$@"
+16 -6
View File
@@ -8,15 +8,25 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Must match the pin in script/bootstrap. # Must match the pin in script/bootstrap.
NODE_VERSION="22.17.0" NODE_VERSION="22.17.0"
# script/bootstrap installs node and yarn under nvm and leaves neither # True when the node on PATH has the pinned major version: the same
# on the PATH of the shell that called it, so resolve the pinned # function as in script/bootstrap, which uses it to decide where it
# toolchain here the way bootstrap's own install step does. nvm is a # installs yarn.
# bash script, hence the subshell. node_is_pinned_major() {
if ! command -v node >/dev/null 2>&1; then return 1; fi
installed="$(node --version)"
installed="${installed#v}"
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
}
# Run the yarn script/bootstrap installed: the yarn on PATH when the
# node on PATH has the pinned major version, and otherwise yarn under
# the pinned node in nvm, which bootstrap leaves off the PATH of the
# shell that called it. nvm is a bash script, hence the subshell.
run_yarn() { run_yarn() {
if command -v yarn >/dev/null 2>&1; then if node_is_pinned_major && command -v yarn >/dev/null 2>&1; then
exec yarn "$@" exec yarn "$@"
fi fi
if [ ! -s "$HOME/.nvm/nvm.sh" ]; then if node_is_pinned_major || [ ! -s "$HOME/.nvm/nvm.sh" ]; then
echo "fmt: no yarn; run script/bootstrap first" >&2 echo "fmt: no yarn; run script/bootstrap first" >&2
exit 1 exit 1
fi fi
+16 -6
View File
@@ -8,15 +8,25 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Must match the pin in script/bootstrap. # Must match the pin in script/bootstrap.
NODE_VERSION="22.17.0" NODE_VERSION="22.17.0"
# script/bootstrap installs node and yarn under nvm and leaves neither # True when the node on PATH has the pinned major version: the same
# on the PATH of the shell that called it, so resolve the pinned # function as in script/bootstrap, which uses it to decide where it
# toolchain here the way bootstrap's own install step does. nvm is a # installs yarn.
# bash script, hence the subshell. node_is_pinned_major() {
if ! command -v node >/dev/null 2>&1; then return 1; fi
installed="$(node --version)"
installed="${installed#v}"
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
}
# Run the yarn script/bootstrap installed: the yarn on PATH when the
# node on PATH has the pinned major version, and otherwise yarn under
# the pinned node in nvm, which bootstrap leaves off the PATH of the
# shell that called it. nvm is a bash script, hence the subshell.
run_yarn() { run_yarn() {
if command -v yarn >/dev/null 2>&1; then if node_is_pinned_major && command -v yarn >/dev/null 2>&1; then
exec yarn "$@" exec yarn "$@"
fi fi
if [ ! -s "$HOME/.nvm/nvm.sh" ]; then if node_is_pinned_major || [ ! -s "$HOME/.nvm/nvm.sh" ]; then
echo "fmt-check: no yarn; run script/bootstrap first" >&2 echo "fmt-check: no yarn; run script/bootstrap first" >&2
exit 1 exit 1
fi fi
-1
View File
@@ -1,7 +1,6 @@
#!/bin/sh #!/bin/sh
# script/install-precommit: install the git pre-commit hook that runs # script/install-precommit: install the git pre-commit hook that runs
# script/precommit. Our own extension to scripts-to-rule-them-all. # script/precommit. Our own extension to scripts-to-rule-them-all.
# Generic: needs no adaptation.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
+8 -5
View File
@@ -5,9 +5,12 @@
# make its answer untrustworthy. # make its answer untrustworthy.
# #
# The phase is not the last stage in the file, so it is built only when # The phase is not the last stage in the file, so it is built only when
# --target names it. --no-cache because a cached lint layer is a lint # --target names it. A new CACHEBUST value reruns every RUN step, since
# that did not run. The tag makes each build replace the previous image # a cached lint layer is a lint that did not run; unlike --no-cache, it
# instead of leaving a dangling one behind. # keeps Go's build cache. This relies on every stage of the Dockerfile
# declaring ARG CACHEBUST first; a stage without that line is served
# from cache on an unchanged tree, with no warning.
# --output type=cacheonly writes no image, since nothing uses one.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -15,9 +18,9 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
docker build --no-cache \ docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--target lint \ --target lint \
-t "$("$SCRIPT_DIR/projectname")-lint" . --output type=cacheonly .
} }
main "$@" main "$@"
+1 -2
View File
@@ -1,7 +1,6 @@
#!/bin/sh #!/bin/sh
# script/setup: set up the repo for development after a fresh clone: # script/setup: set up the repo for development after a fresh clone:
# installs dependencies (script/bootstrap) and the git pre-commit hook. # installs dependencies and the git pre-commit hook.
# Add any repo-specific initialization (db init, .env template) here.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
+8 -4
View File
@@ -2,8 +2,12 @@
# script/test: run the test suite. Testing is a phase of the Dockerfile # script/test: run the test suite. Testing is a phase of the Dockerfile
# and this builds that phase alone, on the same terms as script/lint: # and this builds that phase alone, on the same terms as script/lint:
# --target because a phase that is not the last stage is built only when # --target because a phase that is not the last stage is built only when
# named, --no-cache because a cached test layer is a test that did not # named, and a new CACHEBUST value because a cached test layer is a test
# run, and a tag so each build replaces the previous image. # that did not run; unlike --no-cache, it keeps Go's build cache. This
# relies on every stage of the Dockerfile declaring ARG CACHEBUST first;
# a stage without that line is served from cache on an unchanged tree,
# with no warning. --output type=cacheonly writes no image, since
# nothing uses one.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -11,9 +15,9 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
docker build --no-cache \ docker build --build-arg CACHEBUST="$(date +%s)-$$" \
--target test \ --target test \
-t "$("$SCRIPT_DIR/projectname")-test" . --output type=cacheonly .
} }
main "$@" main "$@"