secret/README.md

# secret - Local Secret Manager

## Description

`secret` is a WTFPL-licensed Go command-line local secret manager by
[@sneak](https://sneak.berlin) that implements a hierarchical key architecture
for storing and managing sensitive data. It supports multiple vaults, various
unlock mechanisms, and provides secure storage using the `age` encryption
library.

## Getting Started

Build from source, then install the binary as `~/bin/secret`:

```bash
git clone https://git.eeqj.de/sneak/secret.git
cd secret
make build     # writes the binary to ./secret
make install   # builds it and copies it to ~/bin/secret
```

Generate a mnemonic, create the default vault, then store and read a secret:

```bash
secret generate mnemonic   # prints a new BIP39 mnemonic; write it down
secret init                # asks for that mnemonic and an unlocker passphrase
echo "my-password" | secret add myservice/password
secret get myservice/password
```

## Rationale

I created `secret` to scratch an itch: I wanted a secure key/value store to
replace a bunch of PGP-encrypted files in a directory structure. It could be
used as a password manager, but was not designed as one.

## Design

### Three-Layer Key Hierarchy

Secret implements a three-layer key architecture:

1. **Long-term Keys**: Derived from BIP39 mnemonic phrases, these provide the
   foundation for all encryption
2. **Unlockers**: Short-term keys that encrypt the long-term keys, supporting
   multiple authentication methods
3. **Version-specific Keys**: Per-version keys that encrypt individual secret
   values

### Version Management

Each secret maintains a history of versions, with each version having:

- Its own encryption key pair
- Metadata including creation time and validity period, encrypted to the
  version's key pair
- Immutable value storage

The secret's `current` file names its current version. Switching versions
replaces that file in one rename, so it is never half-written.

### Vault System

Vaults provide logical separation of secrets, each with its own long-term key
and unlocker set. This allows for complete isolation between different contexts
(work, personal, projects).

## Commands Reference

### Confirmation Before Removal

`secret rm`, `secret version rm`, `secret vault remove` and
`secret unlocker remove` destroy data that exists nowhere else. On a terminal
each one first asks `[y/N]`, naming exactly what it is about to remove, and goes
ahead only on `y` or `yes`; any other answer, a bare Enter included, cancels and
removes nothing. The question is asked only after the command's checks have
passed, and before it changes anything.

Whether to ask is decided by stdin, where the answer is read from, so
`secret rm foo | tee log` still asks. When stdin is not a terminal, as in a
script or a CI job, nobody is there to answer: the command fails at once,
removes nothing, and says to pass `--force`.

`--force` (`-f`) removes without asking, whatever the command removes: a vault
that holds secrets and the last unlocker of a vault included. Scripts that
remove things pass `--force`.

### Initialization

#### `secret init`

Initializes the secret manager with a default vault. Prompts for a BIP39
mnemonic phrase and creates the initial directory structure.

**Environment Variables:**

- `SB_SECRET_MNEMONIC`: Pre-set mnemonic phrase
- `SB_UNLOCK_PASSPHRASE`: Pre-set unlock passphrase

### Vault Management

#### `secret vault list [--json]` / `secret vault ls`

Lists all available vaults. The current vault is marked.

#### `secret vault create <name>`

Creates a new vault with the specified name.

**Vault Name Format:** only lowercase ASCII letters, digits, `.`, `-` and `_`
are allowed, and a name must not be empty, `.` or `..`.

#### `secret vault select <name>`

Switches to the specified vault for subsequent operations.

#### `secret vault remove <name> [--force]` / `secret vault rm` ⚠️ 🛑

**DANGER**: Permanently removes a vault and all its secrets. It first asks for
confirmation, naming the vault and how many secrets it holds (see
[Confirmation Before Removal](#confirmation-before-removal)). The last vault
cannot be removed. Removing the current vault makes another vault the current
one.

- `--force, -f`: Remove without asking, also a vault that contains secrets
- **NO RECOVERY**: All secrets in the vault will be permanently deleted

### Secret Management

#### `secret add <secret-name> [--force]`

Adds a secret to the current vault. Reads the secret value from stdin.

- `--force, -f`: Overwrite existing secret

**Secret Name Format:** only ASCII letters, digits, `.`, `-`, `_` and `/` are
allowed, and a name must not be empty, start with `.` or `/`, end with `/`,
contain `//`, or have `..` as a path segment.

- Forward slashes (`/`) are converted to percent signs (`%`) for storage
- Examples: `database/password`, `api.key`, `ssh_private_key`

#### `secret get <secret-name> [--version <version>]`

Retrieves and outputs a secret value to stdout.

- `--version, -v`: Get a specific version (default: current)

#### `secret list [filter] [--json]` / `secret ls`

Lists all secrets in the current vault. Optional filter for substring matching.

#### `secret remove <secret-name> [--force]` / `secret rm` ⚠️ 🛑

**DANGER**: Permanently removes a secret and ALL its versions. It first asks for
confirmation, naming the secret, its vault and how many versions it has (see
[Confirmation Before Removal](#confirmation-before-removal)).

- `--force, -f`: Remove without asking
- **NO RECOVERY**: Once removed, the secret cannot be recovered
- **ALL VERSIONS DELETED**: Every version of the secret will be permanently
  deleted

#### `secret move <source> <destination>` / `secret mv` / `secret rename`

Moves or renames a secret within the current vault.

- Fails if the destination already exists
- Fails if the destination is the source under another name, such as `foo` for
  `Foo` on a case-insensitive filesystem (the macOS default); there, to change
  only the case of a name, move the secret to a third name first
- Preserves all versions and metadata

### Version Management

#### `secret version list <secret-name>` / `secret version ls`

Lists all versions of a secret showing creation time, status, and validity
period.

#### `secret version promote <secret-name> <version>`

Promotes a specific version to current by rewriting the secret's `current` file
to name it. Does not modify any timestamps, allowing for rollback scenarios.

#### `secret version remove <secret-name> <version> [--force]` / `secret version rm` ⚠️ 🛑

**DANGER**: Permanently removes a specific version of a secret. It first asks
for confirmation, naming the version, the secret and its vault (see
[Confirmation Before Removal](#confirmation-before-removal)).

- `--force, -f`: Remove without asking
- **NO RECOVERY**: Once removed, this version cannot be recovered
- Cannot remove the current version (must promote another version first)

### Key Generation

#### `secret generate mnemonic`

Generates a cryptographically secure BIP39 mnemonic phrase.

#### `secret generate secret <name> [--length=16] [--type=base58] [--force]`

Generates and stores a random secret.

- `--length, -l`: Length of generated secret (default: 16)
- `--type, -t`: Type of secret (`base58`, `alnum`)
- `--force, -f`: Overwrite existing secret

### Unlocker Management

#### `secret unlocker list [--json]` / `secret unlocker ls`

Lists all unlockers in the current vault with their metadata. An unlocker's ID,
which `secret unlocker select` and `secret unlocker remove` take, is the name of
its directory in `unlockers.d`.

#### `secret unlocker add <type> [options]`

Creates a new unlocker of the specified type:

**Types:**

- `passphrase`: Traditional passphrase-protected unlocker
- `pgp`: Uses an existing GPG key for encryption/decryption
- `keychain`: macOS Keychain integration (macOS only)
- `secure-enclave`: Hardware-backed Secure Enclave protection (macOS only)

**Options:**

- `--keyid <id>`: GPG key ID (optional for PGP type, uses default key if not
  specified)

A vault has one passphrase unlocker: adding one replaces the one the vault has,
which is removed only once the new one is the current unlocker.

#### `secret unlocker remove <unlocker-id> [--force]` / `secret unlocker rm` ⚠️ 🛑

**DANGER**: Permanently removes an unlocker. It first asks for confirmation,
naming the unlocker and its vault and saying whether it is the vault's last
unlocker; for the last one it says how many secrets the vault holds and warns
that the vault then opens only with its mnemonic (see
[Confirmation Before Removal](#confirmation-before-removal)). An unlocker
directory that `secret unlocker list` skips with a warning, because its metadata
cannot be read or parsed, is removed by the directory name the warning gives.

- `--force, -f`: Remove without asking, even the last unlocker
- **CRITICAL WARNING**: Without unlockers and without your mnemonic phrase,
  vault data will be PERMANENTLY INACCESSIBLE
- **NO RECOVERY**: Removing all unlockers without having your mnemonic means
  losing access to all secrets forever

#### `secret unlocker select <unlocker-id>`

Selects an unlocker as the current default for operations.

### Import Operations

#### `secret import <secret-name> --source <filename>`

Imports a secret from a file and stores it in the current vault under the given
name.

#### `secret vault import [vault-name]`

Imports a mnemonic phrase into the specified vault (defaults to "default").

### Encryption Operations

#### `secret encrypt <secret-name> [--input=file] [--output=file]`

Encrypts data using an Age key stored as a secret. If the secret doesn't exist,
generates a new Age key.

#### `secret decrypt <secret-name> [--input=file] [--output=file]`

Decrypts data using an Age key stored as a secret.

## Storage Architecture

### Directory Structure

The state directory is `berlin.sneak.pkg.secret` in the user's configuration
directory: on Linux `$XDG_CONFIG_HOME`, or `~/.config` when that is unset; on
macOS `~/Library/Application Support`. When `SB_SECRET_STATE_DIR` is set, it is
the state directory instead. On Linux:

```
~/.config/berlin.sneak.pkg.secret/
├── vaults.d/
│   ├── default/
│   │   ├── unlockers.d/
│   │   │   ├── passphrase-<time>/       # Passphrase unlocker
│   │   │   └── <host>-pgp-<time>/       # PGP unlocker
│   │   ├── secrets.d/
│   │   │   ├── api%key/                 # Secret: api/key
│   │   │   │   ├── versions/
│   │   │   │   │   ├── 20231215.001/   # Version directory
│   │   │   │   │   │   ├── pub.age     # Version public key
│   │   │   │   │   │   ├── priv.age    # Version private key (encrypted)
│   │   │   │   │   │   ├── value.age   # Encrypted value
│   │   │   │   │   │   └── metadata.age # Encrypted metadata
│   │   │   │   │   └── 20231216.001/   # Another version
│   │   │   │   └── current              # Current version's name: 20231216.001
│   │   │   └── database%password/       # Secret: database/password
│   │   │       ├── versions/
│   │   │       └── current              # Current version's name: 20231215.001
│   │   ├── vault-metadata.json          # Vault metadata
│   │   ├── pub.age                      # Long-term public key
│   │   └── current-unlocker             # Current unlocker's directory name
│   └── work/
│       ├── unlockers.d/
│       ├── secrets.d/
│       ├── vault-metadata.json
│       ├── pub.age
│       └── current-unlocker
├── currentvault                         # Current vault's name: default
└── lock                                 # Locked by each command that changes anything
```

`current`, `currentvault` and `current-unlocker` are plain files that each hold
one name. Changing one replaces it in one rename, so it is never half-written.

### Key Management and Encryption Flow

#### 1: Long-term Keys

- **Source**: Derived from BIP39 mnemonic phrases using hierarchical
  deterministic (HD) key derivation
- **Purpose**: Master keys for each vault, used to encrypt secret-specific keys
- **Storage**: Public key stored as `pub.age`, private key encrypted by
  unlockers

#### 2: Unlockers

Unlockers provide different authentication methods to access the long-term keys:

1. **Passphrase Unlockers**:
    - Encrypted with user-provided passphrase
    - Stored as encrypted Age keys
    - Cross-platform compatible

2. **PGP Unlockers**:
    - Uses existing GPG key infrastructure
    - Leverages existing key management workflows
    - Strong authentication through GPG

3. **Keychain Unlockers** (macOS only):
    - Stores unlock keys in macOS Keychain
    - Kept on this Mac only: the keychain item is never synced to other devices
    - Automatic unlocking when Keychain is unlocked
    - Cross-application integration

4. **Secure Enclave Unlockers** (macOS):
    - Hardware-backed key storage using Apple Secure Enclave
    - Uses `sc_auth` / CryptoTokenKit for SE key management (no Apple Developer
      Program required)
    - ECIES encryption: the vault long-term key is encrypted directly to the SE
      key, and only the SE can decrypt it
    - The SE key cannot leave this Mac; using it asks for no Touch ID or
      password

Each vault maintains its own set of unlockers and one long-term key. The
long-term key is encrypted to each unlocker, allowing any authorized unlocker to
access vault secrets.

#### 3: Secret-specific Keys

- Each secret version has its own encryption key pair
- Private key encrypted to the vault's long-term key
- A version's private key decrypts only that version's value and metadata

### Environment Variables

- `SB_SECRET_STATE_DIR`: Custom state directory location
- `SB_SECRET_MNEMONIC`: Pre-set mnemonic phrase (avoids interactive prompt)
- `SB_UNLOCK_PASSPHRASE`: Pre-set unlock passphrase (avoids interactive prompt)
- `SB_GPG_KEY_ID`: GPG key ID for PGP unlockers

**Warning:** `SB_SECRET_MNEMONIC` and `SB_UNLOCK_PASSPHRASE` expose the secret
they hold. Other processes running as the same user can read a process's
environment (on Linux, from `/proc/<pid>/environ`). Every child process of the
shell or script that sets them inherits them, `gpg` included. Set on a command
line or in a CI job, they end up in shell history and CI logs. `secret` unsets
each one as soon as it has read it, so that the programs it runs itself, such as
`gpg`, do not inherit it, but that erases nothing: the environment the process
started with, and its memory, still hold the value. The interactive prompt,
which every command except `secret vault import` offers when the variable is not
set, is the safer default; `secret vault import` has no prompt and needs both
variables.

## Security Features

### Encryption

- Uses the [age encryption library](https://age-encryption.org/) with X25519
  keys
- All private keys are encrypted at rest
- No plaintext secrets stored on disk

### Access Control

- Multiple authentication methods supported
- Vault isolation prevents cross-contamination

### Forward Secrecy

- Per-version encryption keys limit exposure if compromised
- Each version is independently encrypted
- Historical versions remain encrypted with their original keys

### Hardware Integration

- Hardware token support via PGP/GPG integration
- macOS Keychain integration for system-level security
- Secure Enclave integration for hardware-backed key protection (macOS, via
  `sc_auth` / CryptoTokenKit)

## Examples

### Basic Workflow

```bash
# Initialize with a new mnemonic
secret generate mnemonic  # Copy the output
secret init              # Paste the mnemonic when prompted

# Add some secrets
echo "supersecret123" | secret add database/prod/password
echo "api-key-xyz" | secret add services/api/key
echo "ssh-private-key-content" | secret add ssh/servers/web01

# List and retrieve secrets
secret list
secret get database/prod/password
secret get services/api/key

# Remove a secret ⚠️ 🛑 (asks first - PERMANENT!)
secret remove ssh/servers/web01
```

### Multi-vault Setup

```bash
# Create separate vaults for different contexts
secret vault create work
secret vault create personal

# Work with work vault
secret vault select work
echo "work-db-pass" | secret add database/password
secret unlocker add passphrase  # Add passphrase authentication

# Switch to personal vault
secret vault select personal
echo "personal-email-pass" | secret add email/password

# List all vaults
secret vault list

# Remove a vault ⚠️ 🛑 (--force: NO CONFIRMATION - PERMANENT!)
secret vault remove personal --force
```

### Advanced Authentication

```bash
# Add multiple unlock methods
secret unlocker add passphrase              # Password-based
secret unlocker add pgp --keyid ABCD1234    # GPG key
secret unlocker add keychain                # macOS Keychain (macOS only)
secret unlocker add secure-enclave          # macOS Secure Enclave (macOS only)

# List unlockers
secret unlocker list

# Select a specific unlocker
secret unlocker select <unlocker-id>

# Remove an unlocker ⚠️ 🛑 (asks first!)
secret unlocker remove <unlocker-id>
```

### Version Management

```bash
# List all versions of a secret
secret version list database/prod/password

# Promote an older version to current
secret version promote database/prod/password 20231215.001

# Remove an old version ⚠️ 🛑 (asks first - PERMANENT!)
secret version remove database/prod/password 20231214.001
```

### Encryption/Decryption with Age Keys

```bash
# Generate an Age key and store it as a secret
secret generate secret encryption/mykey

# Encrypt a file using the stored key
secret encrypt encryption/mykey --input document.txt --output document.txt.age

# Decrypt the file
secret decrypt encryption/mykey --input document.txt.age --output document.txt
```

## Technical Details

### Cryptographic Primitives

- **Key Derivation**: BIP32/BIP39 hierarchical deterministic key derivation
- **Encryption**: Age (X25519 + ChaCha20-Poly1305)
- **Authentication**: Poly1305 MAC
- **Hashing**: Double SHA-256 for public key identification

### File Formats

- **age Files**: Standard age encryption format (.age extension), except
  `pub.age`, which holds an age public key as text
- **Metadata**: `vault-metadata.json` and `unlocker-metadata.json` are
  unencrypted JSON with a creation time, and `unlocker-metadata.json` also
  records the unlocker's type; a version's `metadata.age` is JSON encrypted to
  the version's public key
- **Vault Metadata**: JSON containing creation time, derivation index, and the
  public key hashes described below

### Vault Management

- **Derivation Index**: Each vault uses a unique derivation index from the
  mnemonic, and thus a unique key pair
- **Public Key Hash**: Double SHA-256 hash of the vault's public key; the same
  hash of the index-0 public key identifies vaults from the same mnemonic
- **Automatic Key Derivation**: When creating vaults with a mnemonic, keys are
  automatically derived

### Cross-Platform Support

- **macOS**: Full support including Keychain and Secure Enclave integration
- **Linux**: Full support (excluding macOS-specific features)

The keychain and Secure Enclave unlockers need a macOS build with cgo. A macOS
build without cgo, such as one cross-compiled from Linux, offers them but fails
to add or use them.

## Security Considerations

### Threat Model

- Protects against unauthorized access to secret values
- Provides defense against compromise of individual components
- Supports hardware-backed authentication where available

### Best Practices

1. Use strong, unique passphrases for unlockers
2. Enable hardware authentication (Keychain, hardware tokens) when available
3. Regularly audit unlockers and remove unused ones
4. Keep mnemonic phrases securely backed up offline
5. Use separate vaults for different security contexts

### Limitations

- Requires access to unlockers for secret retrieval
- Mnemonic phrases must be securely stored and backed up
- Hardware features limited to supported platforms

## Development

### Building

```bash
make build    # Build binary
make test     # Run tests
make lint     # Run linter
```

### Testing

The project includes comprehensive tests:

```bash
make test     # Run all tests
```

## Entrypoints

This repository adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard: normalized scripts in `script/` are the entrypoints for the
development workflow, and the Makefile targets are thin shims that call them. We
provide:

- `script/bootstrap` — install all dependencies (Go, Go module download),
  idempotently; golangci-lint is not installed, it runs in docker
- `script/setup` — make a fresh clone ready for development: runs
  `script/bootstrap`, then `script/install-precommit`
- `script/projectname` — output the project name (`secret`); used by other
  scripts such as `script/docker`
- `script/build` — build the `secret` binary into the repo root, stamping the
  version (`VERSION` from the environment, else `git describe`) and the git
  commit
- `script/test` — run `go vet` and the test suite (verbose rerun on failure)
- `script/lint` — run `golangci-lint` in docker only: builds `Dockerfile.lint`,
  where the linter is a build step that runs on every call, also on an unchanged
  tree
- `script/lint-darwin` — run `go vet` and `golangci-lint` in docker on the code
  as a macOS build compiles it (`GOOS=darwin`), which a Linux build never
  compiles; cgo is off, so the keychain unlocker's calls into the keychain
  (`internal/secret/keychainunlocker_cgo.go`, and `keychainunlocker_test.go`)
  and the Secure Enclave bindings (`internal/macse`) are not checked
- `script/fmt` — format all Go code (writes)
- `script/fmt-check` — check formatting without writing
- `script/check` — run `script/test`, `script/lint`, `script/lint-darwin`, and
  `script/fmt-check`
- `script/docker` — build the Docker image tagged with the project name
- `script/cibuild` — CI entrypoint: `docker build --ulimit memlock=-1:-1 .`
  (memguard needs mlock; the Dockerfile runs the checks), with a new
  `CHECK_EPOCH` build argument on every run so the checks run again on an
  unchanged tree
- `script/precommit` — pre-commit checks: `go mod tidy` verification, then
  `script/check`
- `script/install-precommit` — install the git pre-commit hook that runs
  `script/precommit`

## Features

- **Multiple Authentication Methods**: Supports passphrase, PGP, macOS Keychain,
  and Secure Enclave unlockers
- **Vault Isolation**: Complete separation between different vaults
- **Per-Secret Encryption**: Each secret has its own encryption key
- **BIP39 Mnemonic Support**: Keyless operation using mnemonic phrases
- **Cross-Platform**: Works on macOS, Linux, and other Unix-like systems

## TODO

Open work is tracked on the
[issue tracker](https://git.eeqj.de/sneak/secret/issues), which is
authoritative. The work to be done before 1.0 is the
[`1.0.0` milestone](https://git.eeqj.de/sneak/secret/milestone/12). `TODO.md`
records the steps completed so far.

## License

Released as a free software gift to the world, no strings attached, under the
[WTFPL](https://www.wtfpl.net/) license; see [`LICENSE`](LICENSE).

## Author

Made with love and lots of expensive SOTA AI by [@sneak](https://sneak.berlin)
in Berlin in the summer of 2025.

Contact: [sneak@sneak.berlin](mailto:sneak@sneak.berlin)

[https://keys.openpgp.org/vks/v1/by-fingerprint/5539AD00DE4C42F3AFE11575052443F4DF2A55C2](https://keys.openpgp.org/vks/v1/by-fingerprint/5539AD00DE4C42F3AFE11575052443F4DF2A55C2)