Rewrite README with required sections and package docs
This commit is contained in:
@@ -1,16 +1,276 @@
|
||||
# x
|
||||
|
||||
Experimental code.
|
||||
## Description
|
||||
|
||||
# Build Status
|
||||
`sneak.berlin/go/x` is a WTFPL-licensed Go module and CLI tool by
|
||||
[@sneak](https://sneak.berlin) that serves as an umbrella publishing space for
|
||||
small reusable packages, plus `sutil`, a command-line utility that exposes them
|
||||
from the shell.
|
||||
|
||||
[](https://drone.datavi.be/sneak/x)
|
||||
Importable packages live in `pkg/`. The CLI is `cmd/sutil/`, and the code that
|
||||
exists only to support it lives in `internal/`.
|
||||
|
||||
# License
|
||||
## Getting Started
|
||||
|
||||
WTFPL
|
||||
### Install the CLI
|
||||
|
||||
# Author
|
||||
```sh
|
||||
go install sneak.berlin/go/x/cmd/sutil@latest
|
||||
```
|
||||
|
||||
sneak@sneak.berlin
|
||||
Or, from a checkout:
|
||||
|
||||
```sh
|
||||
make build # builds ./sutil with the git version embedded
|
||||
make install # installs sutil into $(go env GOBIN)
|
||||
```
|
||||
|
||||
### Use the CLI
|
||||
|
||||
```sh
|
||||
# An endless stream of fast pseudorandom bytes; stop it with Ctrl-C.
|
||||
sutil generate random | pv > /dev/null
|
||||
|
||||
# The same, but deterministic: the same seed always produces the same bytes.
|
||||
sutil generate random --count 1073741824 "my seed" > testdata.bin
|
||||
|
||||
# A memorable password: three dictionary words, each followed by a digit,
|
||||
# joined by a randomly chosen delimiter. Requires a system word list.
|
||||
sutil generate password
|
||||
# marble7-sprout2-velvet9
|
||||
|
||||
# A new 24-word BIP39 mnemonic phrase (or --words 12).
|
||||
sutil generate mnemonic
|
||||
|
||||
# A crypto_box keypair derived from a mnemonic. The mnemonic is read from
|
||||
# stdin, never from the command line, so it stays out of shell history.
|
||||
sutil generate mnemonic > mnemonic.txt
|
||||
sutil cbk derive < mnemonic.txt
|
||||
sutil cbk derive --public < mnemonic.txt
|
||||
|
||||
# Build metadata.
|
||||
sutil version
|
||||
sutil info
|
||||
```
|
||||
|
||||
### Use the packages
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"sneak.berlin/go/x/pkg/bip39wordlist"
|
||||
"sneak.berlin/go/x/pkg/cbk"
|
||||
)
|
||||
|
||||
func main() {
|
||||
mnemonic, err := bip39wordlist.GenerateMnemonic(bip39wordlist.MaxWordCount)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
// The same mnemonic (and optional passphrase) always yields the same
|
||||
// keypair, on any machine.
|
||||
alice, err := cbk.New(cbk.Params{Mnemonic: mnemonic})
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
bob, err := cbk.FromMnemonic("abandon abandon abandon abandon abandon " +
|
||||
"abandon abandon abandon abandon abandon abandon about")
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
sealed, err := alice.Seal([]byte("hello bob"), bob.PublicKey)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
plain, err := bob.Open(sealed, alice.PublicKey)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
fmt.Println(string(plain), alice.PublicKeyHex())
|
||||
}
|
||||
```
|
||||
|
||||
## Rationale
|
||||
|
||||
- **Reusable odds and ends need a home.** Several small packages are worth
|
||||
sharing between projects but are too small to each deserve a repository, CI
|
||||
pipeline, and release cadence of their own. This module collects them under
|
||||
one import path.
|
||||
- **`/dev/urandom` is slow.** Operating system random devices are tuned for
|
||||
small requests of key material, not for filling a disk or producing gigabytes
|
||||
of test data. `sutil generate random` streams pseudorandom bytes at close to
|
||||
memory bandwidth, and can do so deterministically from a seed so test data is
|
||||
reproducible.
|
||||
- **Keys you can carry in your head.** A crypto_box keypair derived from a BIP39
|
||||
mnemonic can be regenerated anywhere from a phrase a human can write down or
|
||||
memorize, with no key file to lose. `pkg/cbk` makes that derivation explicit
|
||||
and reproducible.
|
||||
- **Passwords you can type.** Three dictionary words with digits and a delimiter
|
||||
are far easier to transcribe than line noise, and with the macOS word list
|
||||
they still carry about 62 bits of entropy.
|
||||
|
||||
## Design
|
||||
|
||||
### Layout
|
||||
|
||||
| Path | Purpose |
|
||||
| ---------------------- | --------------------------------------------------- |
|
||||
| `cmd/sutil/` | `package main`; the whole entrypoint is a few lines |
|
||||
| `internal/cli/` | The cobra command tree, built by `cli.New` |
|
||||
| `internal/randstream/` | Fast seeded pseudorandom byte stream |
|
||||
| `internal/pwgen/` | Password generator over the system dictionary |
|
||||
| `pkg/bip39wordlist/` | BIP39 English wordlist and mnemonic helpers |
|
||||
| `pkg/cbk/` | crypto_box keypairs derived from BIP39 mnemonics |
|
||||
|
||||
The split between `pkg/` and `internal/` is deliberate and should be kept:
|
||||
`pkg/` holds only packages designed to be imported by other programs, with a
|
||||
stable, documented API. Anything that exists to support `sutil` goes under
|
||||
`internal/`, even if it would be harmless to export, so that the public surface
|
||||
of the module stays small and intentional.
|
||||
|
||||
The command tree holds no global state: `cli.New` builds every command from a
|
||||
`Params` struct, so tests construct it repeatedly with captured input and
|
||||
output, and nothing happens in `init` functions.
|
||||
|
||||
### `pkg/cbk`
|
||||
|
||||
Derives NaCl crypto_box keypairs (Curve25519, XSalsa20, Poly1305)
|
||||
deterministically from BIP39 mnemonics, and seals and opens messages with them.
|
||||
The derivation is fixed and documented so keys are reproducible:
|
||||
|
||||
1. The mnemonic is validated and canonicalized with
|
||||
`bip39wordlist.NormalizeWordlist`.
|
||||
2. The standard 64-byte BIP39 seed is computed with PBKDF2-HMAC-SHA512 (2048
|
||||
rounds, salt `"mnemonic" + passphrase`), exactly as a BIP39 wallet would, so
|
||||
the optional passphrase works as the usual "25th word".
|
||||
3. The 32-byte private key is
|
||||
`HKDF-SHA256(seed, salt = nil, info = cbk.DerivationInfo)`.
|
||||
4. The public key is the Curve25519 base-point multiple of the private key.
|
||||
|
||||
A sealed message is a 24-byte random nonce followed by the box ciphertext, so it
|
||||
can be handled as a single opaque byte string. The API:
|
||||
|
||||
- `cbk.New(cbk.Params{Mnemonic, Passphrase})` and `cbk.FromMnemonic(m)` return a
|
||||
`CryptoBoxKeypair`.
|
||||
- `CryptoBoxKeypair.Seal(message, recipientPublicKey)` and
|
||||
`CryptoBoxKeypair.Open(sealed, senderPublicKey)`.
|
||||
- `CryptoBoxKey.Hex()`, `CryptoBoxKey.Bytes()`, `cbk.KeyFromHex(s)`, and the
|
||||
`PublicKeyHex`, `PrivateKeyHex`, `PublicKeyBytes`, `PrivateKeyBytes` accessors
|
||||
on the keypair.
|
||||
|
||||
The test suite pins the keypair derived from the standard all-zero-entropy BIP39
|
||||
test vector. If that test ever fails, the derivation has changed and existing
|
||||
keys are no longer reproducible.
|
||||
|
||||
### `pkg/bip39wordlist`
|
||||
|
||||
The BIP39 English wordlist, embedded from `english.txt` and checked against the
|
||||
reference implementation in the tests, plus mnemonic helpers:
|
||||
|
||||
- `ValidateWordlist(mnemonic)` checks word count, every word, and the checksum,
|
||||
ignoring letter case and whitespace. It reports what is wrong with sentinel
|
||||
errors (`ErrInvalidWordCount`, `ErrUnknownWord` naming the word and its
|
||||
position, `ErrInvalidChecksum`).
|
||||
- `NormalizeWordlist(mnemonic)` returns the canonical lowercase,
|
||||
single-space-separated form, and refuses anything invalid.
|
||||
- `GenerateMnemonic(wordCount)` makes a new mnemonic of 12, 15, 18, 21, or 24
|
||||
words from `crypto/rand`.
|
||||
- `Contains(word)`, `ValidWordCount(n)`, and the `WordlistEnglish` and
|
||||
`Wordlist` tables.
|
||||
|
||||
### `internal/randstream`
|
||||
|
||||
An `io.Reader` that expands a 32-byte key with the ChaCha8 generator from
|
||||
`math/rand/v2`. With a seed string the key is its SHA-256 and the stream is
|
||||
deterministic; without one the key comes from `crypto/rand`, which is the only
|
||||
time the operating system's random source is touched. The output is for bulk
|
||||
data, not for cryptographic keys.
|
||||
|
||||
### `internal/pwgen`
|
||||
|
||||
Produces passwords of the form
|
||||
|
||||
```
|
||||
<word><digit><delim><word><digit><delim><word><digit>
|
||||
```
|
||||
|
||||
where the delimiter is drawn once per password from `.-_|~+=:,!@#%^&*` and used
|
||||
in both positions. Words come from the system word list
|
||||
(`/usr/share/dict/words`) through `github.com/kavfixnel/words`, filtered to
|
||||
lowercase ASCII words of four to eight letters, which drops proper nouns and
|
||||
abbreviations. All choices use `crypto/rand`. On a system without a word list,
|
||||
`sutil generate password` panics rather than falling back to something weaker;
|
||||
the upstream library is only known to work on macOS.
|
||||
|
||||
### CLI
|
||||
|
||||
```
|
||||
sutil
|
||||
├── version print the build version
|
||||
├── info print version, Go version, module, VCS metadata
|
||||
├── generate (gen)
|
||||
│ ├── random [seedstr] stream pseudorandom bytes (--count N to stop)
|
||||
│ ├── password (pw) print a memorable password
|
||||
│ └── mnemonic print a new BIP39 mnemonic (--words N)
|
||||
└── cbk
|
||||
└── derive keypair from a mnemonic on stdin (--public)
|
||||
```
|
||||
|
||||
### Build, test, and CI
|
||||
|
||||
The `Makefile` is the single entry point: `make test` (30-second timeout, race
|
||||
detector, quiet unless something fails), `make lint`, `make fmt` and
|
||||
`make fmt-check` (gofmt for Go, prettier for markdown), `make check` for all
|
||||
three, `make build`, `make install`, `make docker`, `make hooks`, `make tidy`,
|
||||
and `make setup` to install the pinned prettier.
|
||||
|
||||
`make lint` runs golangci-lint inside the same hash-pinned Docker image the CI
|
||||
build uses, so local results match CI exactly. The `Dockerfile` splits
|
||||
`make check` across stages: a lint stage and a markdown-check stage fail fast in
|
||||
seconds, and the build stage, which runs the tests and compiles `sutil`, depends
|
||||
on both. The runtime image carries the binary and the source it was built from.
|
||||
Every external reference, from base images to the GitHub Action in
|
||||
`.gitea/workflows/check.yml`, is pinned by content hash with a version and date
|
||||
comment. `.golangci.yml` is the standard configuration and is never edited by
|
||||
hand here.
|
||||
|
||||
## Development
|
||||
|
||||
```sh
|
||||
make setup # once: install the pinned prettier with yarn
|
||||
make check # test + lint + fmt-check, exactly what CI gates on
|
||||
make fmt # gofmt and prettier, writes files
|
||||
make hooks # install the pre-commit hook that runs make check
|
||||
make docker # the full CI build
|
||||
```
|
||||
|
||||
## TODO
|
||||
|
||||
- [ ] Tag `v0.1.0` once `main` is green in CI.
|
||||
- [ ] Bump the pinned `golangci/golangci-lint` image (in `Makefile` and
|
||||
`Dockerfile`) past v2.12.2 once the canonical `.golangci.yml` disables
|
||||
`exhaustruct_v5`, which v2.13.0 added alongside the already-disabled
|
||||
`exhaustruct`, and replaces the deprecated `gomodguard` with
|
||||
`gomodguard_v2`. With `default: all`, newer images enable `exhaustruct_v5`
|
||||
and fail every partial struct literal.
|
||||
- [ ] Add `sutil cbk seal` and `sutil cbk open` so messages can be sealed and
|
||||
opened from the shell, not just from Go.
|
||||
- [ ] Let `sutil cbk derive` take the BIP39 passphrase from an environment
|
||||
variable; the library supports it, the CLI does not expose it yet.
|
||||
|
||||
## License
|
||||
|
||||
WTFPL. See `LICENSE`.
|
||||
|
||||
## Author
|
||||
|
||||
[@sneak](https://sneak.berlin)
|
||||
<[sneak@sneak.berlin](mailto:sneak@sneak.berlin)>
|
||||
|
||||
Reference in New Issue
Block a user