Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 sneak
|
437c158fec |
@@ -257,9 +257,8 @@ are now tracked only in the [issues](https://git.eeqj.de/sneak/mfer/issues).
|
||||
|
||||
- Should the manifest signature format be GnuPG signatures, or those from
|
||||
OpenBSD's signify (of which there is a good
|
||||
[golang implementation](https://github.com/frankbraun/gosignify))? Settled
|
||||
under question 10 on [issue 82](https://git.eeqj.de/sneak/mfer/issues/82):
|
||||
OpenPGP signatures, which mfer makes and checks itself without running `gpg`.
|
||||
[golang implementation](https://github.com/frankbraun/gosignify))? Still open,
|
||||
as question 10 on [issue 82](https://git.eeqj.de/sneak/mfer/issues/82).
|
||||
|
||||
- Should the on-disk serialization format be proto3 or json? Settled: it is
|
||||
proto3, see `docs/FORMAT.md` and `mfer/mf.proto`.
|
||||
@@ -268,49 +267,15 @@ are now tracked only in the [issues](https://git.eeqj.de/sneak/mfer/issues).
|
||||
|
||||
- `mfer gen` / `mfer gen .`
|
||||
- recurses under current directory and writes out an `index.mf`
|
||||
- lists regular files only, and skips a named pipe, socket or device, which
|
||||
it names with `-v`
|
||||
- records every file's mode as `0000` unless given `--include-permissions`,
|
||||
which records each file's permission bits (`0777` at most)
|
||||
- `mfer gen /media/drive`
|
||||
- writes `/media/drive/index.mf`, listing each file by its path under
|
||||
`/media/drive`, so `mfer check /media/drive` verifies it. Given a file,
|
||||
gen writes `index.mf` beside it and lists the file by its name; given
|
||||
several paths, it writes `index.mf` in the current directory
|
||||
- `--output` names another file to write instead. What gen lists depends
|
||||
only on the paths it is given and the files under them, so with the same
|
||||
`--seed` and an unchanged tree it writes the same bytes wherever the
|
||||
manifest goes. The file it writes to is never listed
|
||||
- `mfer check` / `mfer check .`
|
||||
- verifies checksums of all files in manifest, displaying error and exiting
|
||||
nonzero if any files are missing or corrupted, or have permission bits
|
||||
other than the mode the manifest records, unless that is `0000`
|
||||
- looks for those files under the base directory: the one `--base` names, or
|
||||
else the directory holding the manifest, or the current directory for a
|
||||
manifest given by URL. So `mfer check /media/drive` checks a drive against
|
||||
the `index.mf` at its root, from any directory
|
||||
- warns about each file under the base directory that the manifest does not
|
||||
list, hidden files included; with `--no-extra-files` each one is a failure
|
||||
instead
|
||||
- `mfer freshen` / `mfer freshen .`
|
||||
- rewrites `index.mf` to list the files now under the directory holding it,
|
||||
or under the one `--base` names, hashing only the files that are new or
|
||||
changed
|
||||
- lists regular files only, as gen does
|
||||
- leaves out hidden files unless given `--include-dotfiles`, and symlinks
|
||||
unless given `--follow-symlinks`, which lists each symlink to a regular
|
||||
file under its own name with the contents of the file it points to
|
||||
- `mfer gen --sign-key key.asc` / `mfer freshen --sign-key key.asc`
|
||||
- signs the manifest with the OpenPGP secret key in `key.asc`, armored or
|
||||
binary, as `gpg --export-secret-keys` writes it; `MFER_SIGN_KEY` names the
|
||||
file too. mfer signs it itself and does not need `gpg`. A file holding
|
||||
more than one key is refused, and a key held only on a smartcard cannot
|
||||
sign. The key must be a version 4 key, whose 40-character fingerprint is
|
||||
what `--require-signature` takes, and not a DSA key. A key that cannot
|
||||
sign, because it has expired or been revoked or its passphrase is wrong,
|
||||
stops `gen` and `freshen` before they read any file
|
||||
- takes a protected key's passphrase from `MFER_SIGN_KEY_PASSPHRASE`, or
|
||||
else asks for it at the terminal
|
||||
- `mfer fetch https://example.com/stuff/`
|
||||
- fetches `/stuff/index.mf` and downloads all files listed in manifest into
|
||||
the current directory, or the one given with `--dest`, and assures
|
||||
|
||||
+43
-236
@@ -6,7 +6,7 @@ Version 1.0
|
||||
|
||||
An `.mf` file is a binary manifest that describes a directory tree of files,
|
||||
including their paths, sizes, and cryptographic checksums. It supports optional
|
||||
OpenPGP signatures for integrity verification and optional timestamps and file
|
||||
GPG signatures for integrity verification and optional timestamps and file
|
||||
permissions for metadata preservation.
|
||||
|
||||
Nothing goes in the 1.0 manifest that 1.0 does not read or write: no field is
|
||||
@@ -20,14 +20,9 @@ An `.mf` file consists of two parts, concatenated:
|
||||
2. **Outer message**: a Protocol Buffers serialized `MFFileOuter` message
|
||||
|
||||
There is no length prefix or version byte between the magic and the protobuf
|
||||
message. The protobuf message extends to the end of the file, so a `.mf` is
|
||||
always a whole file; a different framing, such as one with a length prefix,
|
||||
would need a different magic. The format's version is the outer message's
|
||||
`version` field.
|
||||
message. The protobuf message extends to the end of the file.
|
||||
|
||||
See [`mfer/mf.proto`](../mfer/mf.proto) for exact field numbers and types.
|
||||
Readers ignore any field they do not know, in any message, so a later version
|
||||
can add a field without breaking 1.0 readers.
|
||||
|
||||
## Outer Message (`MFFileOuter`)
|
||||
|
||||
@@ -39,11 +34,11 @@ The outer message contains:
|
||||
| `compressionType` | 102 | enum | Compression of `innerMessage`; must be `COMPRESSION_ZSTD` (1) |
|
||||
| `size` | 103 | int64 | Uncompressed size of `innerMessage` (corruption detection) |
|
||||
| `sha256` | 104 | bytes | SHA-256 hash of the **compressed** `innerMessage` (corruption detection) |
|
||||
| `uuid` | 105 | bytes | 16-byte UUID; must match the inner message UUID |
|
||||
| `uuid` | 105 | bytes | Random v4 UUID; must match the inner message UUID |
|
||||
| `innerMessage` | 199 | bytes | Zstd-compressed serialized `MFFile` message |
|
||||
| `signature` | 201 | bytes (optional) | One OpenPGP signature over the canonical string, armored or binary |
|
||||
| `signer` | 202 | bytes (optional) | Fingerprint of the signing primary key, in upper-case hex |
|
||||
| `signingPubKey` | 203 | bytes (optional) | Full OpenPGP public key of the signing key (ASCII-armored or binary) |
|
||||
| `signature` | 201 | bytes (optional) | GPG signature (ASCII-armored or binary) |
|
||||
| `signer` | 202 | bytes (optional) | Fingerprint of the signing key |
|
||||
| `signingPubKey` | 203 | bytes (optional) | Full GPG signing public key |
|
||||
|
||||
### SHA-256 Hash
|
||||
|
||||
@@ -69,26 +64,26 @@ for the signature, the signing key and the other outer fields.
|
||||
After decompressing `innerMessage`, the result is a serialized `MFFile`
|
||||
(referred to as the manifest):
|
||||
|
||||
| Field | Number | Type | Description |
|
||||
| ----------- | ------ | --------------------- | ----------------------------------- |
|
||||
| `version` | 100 | enum | Must be `VERSION_ONE` (1) |
|
||||
| `files` | 101 | repeated `MFFilePath` | List of files in the manifest |
|
||||
| `uuid` | 102 | bytes | 16-byte UUID; must match outer UUID |
|
||||
| `createdAt` | 201 | Timestamp (optional) | When the manifest was created |
|
||||
| Field | Number | Type | Description |
|
||||
| ----------- | ------ | --------------------- | ------------------------------------- |
|
||||
| `version` | 100 | enum | Must be `VERSION_ONE` (1) |
|
||||
| `files` | 101 | repeated `MFFilePath` | List of files in the manifest |
|
||||
| `uuid` | 102 | bytes | Random v4 UUID; must match outer UUID |
|
||||
| `createdAt` | 201 | Timestamp (optional) | When the manifest was created |
|
||||
|
||||
## File Entries (`MFFilePath`)
|
||||
|
||||
Each file entry contains:
|
||||
|
||||
| Field | Number | Type | Description |
|
||||
| ---------- | ------ | ------------------------- | ------------------------------------------------------------- |
|
||||
| `path` | 1 | string | Relative file path (see Path Rules) |
|
||||
| `size` | 2 | int64 | File size in bytes |
|
||||
| `hashes` | 3 | repeated `MFFileChecksum` | At least one hash required; readers reject an entry with none |
|
||||
| `mimeType` | 301 | string (optional) | MIME type |
|
||||
| `mtime` | 302 | Timestamp (optional) | Modification time |
|
||||
| `ctime` | 303 | Timestamp (optional) | Change time (inode metadata change) |
|
||||
| `mode` | 304 | uint32 | Permission bits (see File Mode) |
|
||||
| Field | Number | Type | Description |
|
||||
| ---------- | ------ | ------------------------- | ----------------------------------- |
|
||||
| `path` | 1 | string | Relative file path (see Path Rules) |
|
||||
| `size` | 2 | int64 | File size in bytes |
|
||||
| `hashes` | 3 | repeated `MFFileChecksum` | At least one hash required |
|
||||
| `mimeType` | 301 | string (optional) | MIME type |
|
||||
| `mtime` | 302 | Timestamp (optional) | Modification time |
|
||||
| `ctime` | 303 | Timestamp (optional) | Change time (inode metadata change) |
|
||||
| `mode` | 304 | uint32 | Permission bits (see File Mode) |
|
||||
|
||||
## File Mode
|
||||
|
||||
@@ -121,10 +116,6 @@ Implementations must validate these invariants when reading and writing
|
||||
manifests. Paths that violate these rules must be rejected, and a reader must
|
||||
reject a manifest that lists a path more than once.
|
||||
|
||||
A path holds each name exactly as the file system gives it, with no Unicode
|
||||
normalization, and readers compare paths byte for byte. So `é` stored as one
|
||||
code point and `é` stored as `e` followed by a combining accent are two paths.
|
||||
|
||||
## Hash Format (`MFFileChecksum`)
|
||||
|
||||
Each checksum is a single `bytes multiHash` field containing a
|
||||
@@ -137,227 +128,43 @@ Implementations must be able to verify SHA-256 multihashes at minimum.
|
||||
|
||||
## Signature Scheme
|
||||
|
||||
Signing is optional. A signature is stored only in the outer message's
|
||||
`signature` field, never in a separate file. It is one OpenPGP signature over
|
||||
the canonical string below, armored or binary, and does not contain the string,
|
||||
which a verifier builds again from the outer message.
|
||||
|
||||
The canonical string is three parts separated by hyphens (`-`):
|
||||
Signing is optional. When present, the signature covers a canonical string
|
||||
constructed as:
|
||||
|
||||
```
|
||||
ZNAVSRFG-<UUID>-<SHA256>
|
||||
```
|
||||
|
||||
- `ZNAVSRFG` is the magic, in ASCII.
|
||||
- `<UUID>` is the 16 bytes of the outer message's `uuid` as 32 lower-case hex
|
||||
digits, with no hyphens.
|
||||
- `<SHA256>` is the lower-case hex of the outer message's `sha256` encoded as a
|
||||
SHA-256 multihash (see Hash Format). That multihash is the algorithm code
|
||||
`0x12`, the digest length `0x20` and the 32 bytes of the hash, so this part is
|
||||
68 hex digits starting `1220`.
|
||||
Where:
|
||||
|
||||
The signature covers exactly the 110 ASCII characters of the string, with no
|
||||
line ending. For example, the UUID `b0fb6918-6298-443b-ada2-1b162f4dc0b3` and a
|
||||
`sha256` of `e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855`
|
||||
in hex give this canonical string:
|
||||
- `ZNAVSRFG` is the magic bytes string (literal ASCII)
|
||||
- `<UUID>` is the hex-encoded UUID from the outer message
|
||||
- `<SHA256>` is the hex-encoded SHA-256 hash from the outer message (covering
|
||||
compressed data)
|
||||
|
||||
```
|
||||
ZNAVSRFG-b0fb69186298443bada21b162f4dc0b3-1220e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
|
||||
```
|
||||
|
||||
The signature covers `innerMessage` only through `sha256`, so a verifier checks
|
||||
`sha256` against `innerMessage` (see SHA-256 Hash) as well as the signature.
|
||||
|
||||
The signing key's public key goes in `signingPubKey` and the fingerprint of its
|
||||
primary key, in upper-case hex, in `signer`. The reference implementation signs
|
||||
only with version 4 keys, whose fingerprints are 40 hex digits, and writes both
|
||||
the signature and the public key armored.
|
||||
Components are separated by hyphens. The signature is produced by GPG over this
|
||||
canonical string and stored in the `signature` field of the outer message. The
|
||||
signing key's public key goes in `signingPubKey` and its fingerprint, in hex, in
|
||||
`signer`.
|
||||
|
||||
A verifier accepts a signed manifest only if `signingPubKey` holds exactly one
|
||||
primary key, `signature` is one good signature over the canonical string made by
|
||||
that key (or one of its subkeys), and `signer` is that primary key's
|
||||
fingerprint, compared without regard to case. The reference implementation
|
||||
refuses to load a manifest that fails these checks; `check` and `fetch` given
|
||||
`--require-signature` then compare the required fingerprint with `signer`. It
|
||||
also refuses a manifest whose `signingPubKey` holds a DSA key or subkey, or any
|
||||
secret key or subkey, since checking the self-signatures of a DSA key or the
|
||||
numbers of a secret key can take hours when those numbers are very large.
|
||||
|
||||
A signing key that has expired does not make the signature bad, because a
|
||||
manifest outlives its key. A signature past its own expiry time is not good,
|
||||
whether or not the key has expired too.
|
||||
that key (or one of its subkeys), and `signer` is that key's fingerprint. The
|
||||
reference implementation refuses to load a manifest that fails these checks;
|
||||
`check` and `fetch` given `--require-signature` then compare the required
|
||||
fingerprint with `signer`.
|
||||
|
||||
## Deterministic Serialization
|
||||
|
||||
The reference implementation writes the same inner message for the same files,
|
||||
UUID and options, unless the options include `--include-timestamps`. It
|
||||
compresses the same inner message to the same bytes:
|
||||
By default, manifests are generated deterministically:
|
||||
|
||||
- File entries are sorted by `path` in byte order, so `B.txt` comes before
|
||||
`a.txt`.
|
||||
- Each message is encoded by the Go protobuf library with deterministic
|
||||
marshaling, which writes fields in field-number order and leaves out the
|
||||
fields that are not set. A field not declared `optional` counts as not set
|
||||
when it holds its default value, zero or empty.
|
||||
- Each entry holds only `path`, `size`, `hashes` with exactly one hash (the
|
||||
SHA-256 multihash of the file's content), `mtime` and `mode`. `mtime` is the
|
||||
file's modification time as the file system gives it, to the nanosecond, and
|
||||
`mode` is `0000` unless `gen` or `freshen` is given `--include-permissions`.
|
||||
- `createdAt` is left out unless `gen` or `freshen` is given
|
||||
`--include-timestamps`. Then it holds the time the run started building the
|
||||
manifest, to the nanosecond, so it differs on every run.
|
||||
- The inner message is compressed into one zstd frame, with a checksum of its
|
||||
content, by the encoder of the Go library `github.com/klauspost/compress` at
|
||||
its `SpeedBestCompression` level and otherwise its default settings. That
|
||||
encoder gives the same bytes for the same input only within one version of the
|
||||
library. Another zstd encoder gives other bytes for the same inner message,
|
||||
and so another `sha256`.
|
||||
- `gen` and `freshen` choose a new random version 4 UUID on every run, except
|
||||
that `gen` given `--seed` takes the first 16 bytes of the SHA-256 of the seed
|
||||
string instead. Those bytes are not a version 4 UUID; readers accept any 16
|
||||
bytes.
|
||||
- File entries are sorted by `path` in **lexicographic byte order**
|
||||
- `createdAt` is omitted unless explicitly requested
|
||||
- `mode` is `0000` unless explicitly requested
|
||||
|
||||
So two runs of `gen` over an unchanged tree write byte-identical files only when
|
||||
both are given the same `--seed`, neither is given `--include-timestamps`, and
|
||||
neither signs the manifest: a signature records the time it was made, and the
|
||||
OpenPGP library adds a random salt to each one.
|
||||
|
||||
## Test Vector
|
||||
|
||||
[`mfer/testdata/test-vector.mf`](../mfer/testdata/test-vector.mf) is a signed
|
||||
manifest the reference implementation made once, with the UUID
|
||||
`92197fd5-32bc-4090-8aa9-f86e5518b56c` and an Ed25519 key that the OpenPGP
|
||||
library made for this file and nothing else; its secret key was not kept.
|
||||
`TestFormatTestVector` in `mfer/deserialize_test.go` checks that the file's
|
||||
SHA-256, its canonical string and `signer` are the ones shown here, and verifies
|
||||
its signature.
|
||||
|
||||
A reader should load the file and accept its signature. A writer given the same
|
||||
files and UUID should give the same inner message; its compressed bytes, and so
|
||||
its `sha256` and signature, depend on its zstd encoder (see Deterministic
|
||||
Serialization).
|
||||
|
||||
The manifest has no `createdAt` and lists two files, each with mode `0000` and
|
||||
the modification time 2026-10-13 12:00:00.5 UTC (`seconds` 1791892800, `nanos`
|
||||
500000000):
|
||||
|
||||
| `path` | `size` | Content |
|
||||
| --------------- | ------ | ----------------------- |
|
||||
| `dir/world.txt` | 6 | `world` and a line feed |
|
||||
| `hello.txt` | 6 | `hello` and a line feed |
|
||||
|
||||
The inner message, 164 bytes before compression:
|
||||
|
||||
```
|
||||
00000000 a0 06 01 aa 06 46 0a 0d 64 69 72 2f 77 6f 72 6c |.....F..dir/worl|
|
||||
00000010 64 2e 74 78 74 10 06 1a 24 0a 22 12 20 e2 58 d2 |d.txt...$.". .X.|
|
||||
00000020 48 fd a9 4c 63 75 36 07 f7 c4 49 4e e0 fc be 92 |H..Lcu6...IN....|
|
||||
00000030 f1 a7 6b fd ac 79 5c 9d 84 10 1e b3 17 f2 12 0c |..k..y\.........|
|
||||
00000040 08 c0 ba b8 d6 06 10 80 ca b5 ee 01 aa 06 42 0a |..............B.|
|
||||
00000050 09 68 65 6c 6c 6f 2e 74 78 74 10 06 1a 24 0a 22 |.hello.txt...$."|
|
||||
00000060 12 20 58 91 b5 b5 22 d5 df 08 6d 0f f0 b1 10 fb |. X..."...m.....|
|
||||
00000070 d9 d2 1b b4 fc 71 63 af 34 d0 82 86 a2 e8 46 f6 |.....qc.4.....F.|
|
||||
00000080 be 03 f2 12 0c 08 c0 ba b8 d6 06 10 80 ca b5 ee |................|
|
||||
00000090 01 b2 06 10 92 19 7f d5 32 bc 40 90 8a a9 f8 6e |........2.@....n|
|
||||
000000a0 55 18 b5 6c |U..l|
|
||||
```
|
||||
|
||||
Its compressed form is hashed into `sha256`:
|
||||
|
||||
```
|
||||
2353553205f036a88cefee3d0d9049ccd8e68bb24011a0b462b5867f1fcc6352
|
||||
```
|
||||
|
||||
So the canonical string is:
|
||||
|
||||
```
|
||||
ZNAVSRFG-92197fd532bc40908aa9f86e5518b56c-12202353553205f036a88cefee3d0d9049ccd8e68bb24011a0b462b5867f1fcc6352
|
||||
```
|
||||
|
||||
`signer` is `5E962A37590D855B467D20294947A60319611E6F`, and `signature` holds:
|
||||
|
||||
```
|
||||
-----BEGIN PGP SIGNATURE-----
|
||||
|
||||
wqsEABYIAF0FgmrOejEJEElHpgMZYR5vNRQAAAAAABwAEHNhbHRAbm90YXRpb25z
|
||||
Lm9wZW5wZ3Bqcy5vcmddxfykLLTqmTD7G/jHGzY+FiEEXpYqN1kNhVtGfSApSUem
|
||||
AxlhHm8AAH0hAQDI6oGxvk1sGqIcu3Abvs7DM/7EalxkND614N75FCg+ywEApP6L
|
||||
oIvrZUgEw8DUb9JeK7TChOi6PUAHUFY75Fp6hAE=
|
||||
=nfX5
|
||||
-----END PGP SIGNATURE-----
|
||||
```
|
||||
|
||||
The whole file, 1056 bytes, whose SHA-256 is
|
||||
`5864187d671b2a55f8e015f858248f92c51956b549cf548b6bb54477c2407fd4`:
|
||||
|
||||
```
|
||||
00000000 5a 4e 41 56 53 52 46 47 a8 06 01 b0 06 01 b8 06 |ZNAVSRFG........|
|
||||
00000010 a4 01 c2 06 20 23 53 55 32 05 f0 36 a8 8c ef ee |.... #SU2..6....|
|
||||
00000020 3d 0d 90 49 cc d8 e6 8b b2 40 11 a0 b4 62 b5 86 |=..I.....@...b..|
|
||||
00000030 7f 1f cc 63 52 ca 06 10 92 19 7f d5 32 bc 40 90 |...cR.......2.@.|
|
||||
00000040 8a a9 f8 6e 55 18 b5 6c ba 0c a1 01 28 b5 2f fd |...nU..l....(./.|
|
||||
00000050 04 00 a5 04 00 94 08 a0 06 01 aa 06 46 0a 0d 64 |............F..d|
|
||||
00000060 69 72 2f 77 6f 72 6c 64 2e 74 78 74 10 06 1a 24 |ir/world.txt...$|
|
||||
00000070 0a 22 12 20 e2 58 d2 48 fd a9 4c 63 75 36 07 f7 |.". .X.H..Lcu6..|
|
||||
00000080 c4 49 4e e0 fc be 92 f1 a7 6b fd ac 79 5c 9d 84 |.IN......k..y\..|
|
||||
00000090 10 1e b3 17 f2 12 0c 08 c0 ba b8 d6 06 10 80 ca |................|
|
||||
000000a0 b5 ee 01 aa 06 42 0a 09 68 65 6c 6c 6f 58 91 b5 |.....B..helloX..|
|
||||
000000b0 b5 22 d5 df 08 6d 0f f0 b1 10 fb d9 d2 1b b4 fc |."...m..........|
|
||||
000000c0 71 63 af 34 d0 82 86 a2 e8 46 f6 be 03 b2 06 10 |qc.4.....F......|
|
||||
000000d0 92 19 7f d5 32 bc 40 90 8a a9 f8 6e 55 18 b5 6c |....2.@....nU..l|
|
||||
000000e0 02 10 06 40 ba b7 10 ce 0c f8 c2 24 13 ca 0c ac |...@.......$....|
|
||||
000000f0 02 2d 2d 2d 2d 2d 42 45 47 49 4e 20 50 47 50 20 |.-----BEGIN PGP |
|
||||
00000100 53 49 47 4e 41 54 55 52 45 2d 2d 2d 2d 2d 0a 0a |SIGNATURE-----..|
|
||||
00000110 77 71 73 45 41 42 59 49 41 46 30 46 67 6d 72 4f |wqsEABYIAF0FgmrO|
|
||||
00000120 65 6a 45 4a 45 45 6c 48 70 67 4d 5a 59 52 35 76 |ejEJEElHpgMZYR5v|
|
||||
00000130 4e 52 51 41 41 41 41 41 41 42 77 41 45 48 4e 68 |NRQAAAAAABwAEHNh|
|
||||
00000140 62 48 52 41 62 6d 39 30 59 58 52 70 62 32 35 7a |bHRAbm90YXRpb25z|
|
||||
00000150 0a 4c 6d 39 77 5a 57 35 77 5a 33 42 71 63 79 35 |.Lm9wZW5wZ3Bqcy5|
|
||||
00000160 76 63 6d 64 64 78 66 79 6b 4c 4c 54 71 6d 54 44 |vcmddxfykLLTqmTD|
|
||||
00000170 37 47 2f 6a 48 47 7a 59 2b 46 69 45 45 58 70 59 |7G/jHGzY+FiEEXpY|
|
||||
00000180 71 4e 31 6b 4e 68 56 74 47 66 53 41 70 53 55 65 |qN1kNhVtGfSApSUe|
|
||||
00000190 6d 0a 41 78 6c 68 48 6d 38 41 41 48 30 68 41 51 |m.AxlhHm8AAH0hAQ|
|
||||
000001a0 44 49 36 6f 47 78 76 6b 31 73 47 71 49 63 75 33 |DI6oGxvk1sGqIcu3|
|
||||
000001b0 41 62 76 73 37 44 4d 2f 37 45 61 6c 78 6b 4e 44 |Abvs7DM/7EalxkND|
|
||||
000001c0 36 31 34 4e 37 35 46 43 67 2b 79 77 45 41 70 50 |614N75FCg+ywEApP|
|
||||
000001d0 36 4c 0a 6f 49 76 72 5a 55 67 45 77 38 44 55 62 |6L.oIvrZUgEw8DUb|
|
||||
000001e0 39 4a 65 4b 37 54 43 68 4f 69 36 50 55 41 48 55 |9JeK7TChOi6PUAHU|
|
||||
000001f0 46 59 37 35 46 70 36 68 41 45 3d 0a 3d 6e 66 58 |FY75Fp6hAE=.=nfX|
|
||||
00000200 35 0a 2d 2d 2d 2d 2d 45 4e 44 20 50 47 50 20 53 |5.-----END PGP S|
|
||||
00000210 49 47 4e 41 54 55 52 45 2d 2d 2d 2d 2d d2 0c 28 |IGNATURE-----..(|
|
||||
00000220 35 45 39 36 32 41 33 37 35 39 30 44 38 35 35 42 |5E962A37590D855B|
|
||||
00000230 34 36 37 44 32 30 32 39 34 39 34 37 41 36 30 33 |467D20294947A603|
|
||||
00000240 31 39 36 31 31 45 36 46 da 0c d4 03 2d 2d 2d 2d |19611E6F....----|
|
||||
00000250 2d 42 45 47 49 4e 20 50 47 50 20 50 55 42 4c 49 |-BEGIN PGP PUBLI|
|
||||
00000260 43 20 4b 45 59 20 42 4c 4f 43 4b 2d 2d 2d 2d 2d |C KEY BLOCK-----|
|
||||
00000270 0a 0a 78 6a 4d 45 61 73 31 30 67 42 59 4a 4b 77 |..xjMEas10gBYJKw|
|
||||
00000280 59 42 42 41 48 61 52 77 38 42 41 51 64 41 70 71 |YBBAHaRw8BAQdApq|
|
||||
00000290 33 32 53 79 4f 72 4e 75 76 31 4d 71 53 55 66 45 |32SyOrNuv1MqSUfE|
|
||||
000002a0 44 67 74 42 7a 6e 4b 59 32 39 38 2f 30 47 46 55 |DgtBznKY298/0GFU|
|
||||
000002b0 76 6b 0a 6c 6f 42 71 35 69 6e 4e 4b 47 31 6d 5a |vk.loBq5inNKG1mZ|
|
||||
000002c0 58 49 67 64 47 56 7a 64 43 42 32 5a 57 4e 30 62 |XIgdGVzdCB2ZWN0b|
|
||||
000002d0 33 49 67 50 48 52 6c 63 33 51 74 64 6d 56 6a 64 |3IgPHRlc3QtdmVjd|
|
||||
000002e0 47 39 79 51 47 31 6d 5a 58 49 75 64 47 56 7a 64 |G9yQG1mZXIudGVzd|
|
||||
000002f0 44 37 43 0a 76 51 51 54 46 67 67 41 62 77 57 43 |D7C.vQQTFggAbwWC|
|
||||
00000300 61 73 31 30 67 41 49 4c 42 77 6b 51 53 55 65 6d |as10gAILBwkQSUem|
|
||||
00000310 41 78 6c 68 48 6d 38 31 46 41 41 41 41 41 41 41 |AxlhHm81FAAAAAAA|
|
||||
00000320 48 41 41 51 63 32 46 73 64 45 42 75 62 33 52 68 |HAAQc2FsdEBub3Rh|
|
||||
00000330 64 47 6c 76 0a 62 6e 4d 75 62 33 42 6c 62 6e 42 |dGlv.bnMub3BlbnB|
|
||||
00000340 6e 63 47 70 7a 4c 6d 39 79 5a 30 79 62 6c 73 34 |ncGpzLm9yZ0ybls4|
|
||||
00000350 71 53 43 6d 77 4a 64 45 49 45 6d 4c 72 54 55 45 |qSCmwJdEIEmLrTUE|
|
||||
00000360 43 46 51 67 43 46 67 41 43 47 51 45 43 6d 77 4d |CFQgCFgACGQECmwM|
|
||||
00000370 43 48 67 45 57 0a 49 51 52 65 6c 69 6f 33 57 51 |CHgEW.IQRelio3WQ|
|
||||
00000380 32 46 57 30 5a 39 49 43 6c 4a 52 36 59 44 47 57 |2FW0Z9IClJR6YDGW|
|
||||
00000390 45 65 62 77 41 41 31 67 73 41 2f 32 61 2f 50 58 |EebwAA1gsA/2a/PX|
|
||||
000003a0 68 6c 44 49 4b 78 67 32 79 4e 30 4c 58 39 48 68 |hlDIKxg2yN0LX9Hh|
|
||||
000003b0 49 44 43 54 7a 69 0a 49 32 43 53 71 44 74 2b 34 |IDCTzi.I2CSqDt+4|
|
||||
000003c0 62 2f 73 7a 6d 59 64 41 51 43 66 2b 63 6a 6b 4f |b/szmYdAQCf+cjkO|
|
||||
000003d0 71 6c 34 4c 47 5a 41 43 6c 2f 73 2f 2f 6d 31 71 |ql4LGZACl/s//m1q|
|
||||
000003e0 62 74 6e 2f 4a 31 46 45 53 52 56 73 4e 68 65 2f |btn/J1FESRVsNhe/|
|
||||
000003f0 76 6d 69 41 67 3d 3d 0a 3d 56 6b 4f 33 0a 2d 2d |vmiAg==.=VkO3.--|
|
||||
00000400 2d 2d 2d 45 4e 44 20 50 47 50 20 50 55 42 4c 49 |---END PGP PUBLI|
|
||||
00000410 43 20 4b 45 59 20 42 4c 4f 43 4b 2d 2d 2d 2d 2d |C KEY BLOCK-----|
|
||||
```
|
||||
This ensures that two independent runs over the same directory tree produce
|
||||
byte-identical `.mf` files (assuming file contents and metadata have not
|
||||
changed).
|
||||
|
||||
## MIME Type
|
||||
|
||||
|
||||
@@ -3,8 +3,6 @@ module sneak.berlin/go/mfer
|
||||
go 1.27.1
|
||||
|
||||
require (
|
||||
github.com/ProtonMail/go-crypto v1.5.2
|
||||
github.com/creack/pty v1.1.25-0.20260601142114-9246436fffe8
|
||||
github.com/davecgh/go-spew v1.1.1
|
||||
github.com/dustin/go-humanize v1.1.0
|
||||
github.com/klauspost/compress v1.20.1
|
||||
@@ -17,7 +15,6 @@ require (
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/cloudflare/circl v1.6.3 // indirect
|
||||
github.com/klauspost/cpuid/v2 v2.4.0 // indirect
|
||||
github.com/minio/sha256-simd v1.0.1 // indirect
|
||||
github.com/mr-tron/base58 v1.3.0 // indirect
|
||||
|
||||
@@ -1,9 +1,3 @@
|
||||
github.com/ProtonMail/go-crypto v1.5.2 h1:cucYnvqcY7UOXVD//mSyjeaPY0SSN3v5cDkYPxumINk=
|
||||
github.com/ProtonMail/go-crypto v1.5.2/go.mod h1:/RaSu30DaKO4RY+XdV/ACcCcZkGr7AhUIduq5sjzzCo=
|
||||
github.com/cloudflare/circl v1.6.3 h1:9GPOhQGF9MCYUeXyMYlqTR6a5gTrgR/fBLXvUgtVcg8=
|
||||
github.com/cloudflare/circl v1.6.3/go.mod h1:2eXP6Qfat4O/Yhh8BznvKnJ+uzEoTQ6jVKJRn81BiS4=
|
||||
github.com/creack/pty v1.1.25-0.20260601142114-9246436fffe8 h1:CY3gjC7naqYGLMiywvj3suPfa1i0p/QEr7o8ujxL/2M=
|
||||
github.com/creack/pty v1.1.25-0.20260601142114-9246436fffe8/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/dustin/go-humanize v1.1.0 h1:dbKTrvD0klcbBV/h4AWJdMuZogJACoMlvWIWZ5b2xWg=
|
||||
|
||||
+10
-12
@@ -21,8 +21,8 @@ import (
|
||||
"sneak.berlin/go/mfer/mfer"
|
||||
)
|
||||
|
||||
// fingerprintHexLen is the length in hex characters of the fingerprint of
|
||||
// an OpenPGP version 4 key, the only version mfer signs with.
|
||||
// fingerprintHexLen is the length of a full GPG key fingerprint in hex
|
||||
// characters.
|
||||
const fingerprintHexLen = 40
|
||||
|
||||
var (
|
||||
@@ -189,8 +189,8 @@ func reportCheckProgress(progress <-chan mfer.CheckStatus, wg *sync.WaitGroup) {
|
||||
log.ProgressDone()
|
||||
}
|
||||
|
||||
// countCheckFailures consumes check results, counting failures and logging
|
||||
// each at error level so that --quiet still shows them, then closes done.
|
||||
// countCheckFailures consumes check results, counting and logging
|
||||
// failures, then closes done.
|
||||
func countCheckFailures(
|
||||
results <-chan mfer.Result, failures *int64, done chan<- struct{},
|
||||
) {
|
||||
@@ -198,7 +198,7 @@ func countCheckFailures(
|
||||
if result.Status != mfer.StatusOK {
|
||||
*failures++
|
||||
|
||||
log.Errorf("%s: %s (%s)", result.Status, result.Path, result.Message)
|
||||
log.Infof("%s: %s (%s)", result.Status, result.Path, result.Message)
|
||||
} else {
|
||||
log.Verbosef("%s: %s", result.Status, result.Path)
|
||||
}
|
||||
@@ -208,8 +208,8 @@ func countCheckFailures(
|
||||
}
|
||||
|
||||
// findExtraFiles reports files present on disk but absent from the
|
||||
// manifest, and anything the search cannot read: each is a failure, logged
|
||||
// at error level, under --no-extra-files, otherwise a warning.
|
||||
// manifest, and anything the search cannot read: each is a failure under
|
||||
// --no-extra-files, otherwise a warning.
|
||||
func findExtraFiles(
|
||||
ctx context.Context, cmd *cli.Command, chk *mfer.Checker, failures *int64,
|
||||
) error {
|
||||
@@ -221,7 +221,7 @@ func findExtraFiles(
|
||||
if cmd.Bool("no-extra-files") {
|
||||
*failures++
|
||||
|
||||
log.Errorf("%s: %s (%s)", result.Status, result.Path, result.Message)
|
||||
log.Infof("%s: %s (%s)", result.Status, result.Path, result.Message)
|
||||
} else {
|
||||
log.Warnf("%s: %s (%s)", result.Status, result.Path, result.Message)
|
||||
}
|
||||
@@ -299,10 +299,6 @@ func (mfa *CLIApp) checkManifestOperation(
|
||||
return err
|
||||
}
|
||||
|
||||
// Done before a URL is swapped for the temp file it is downloaded to,
|
||||
// whose directory is not the base.
|
||||
basePath := resolveBasePath(cmd, manifestPath)
|
||||
|
||||
// URL manifests need to be downloaded to a temp file for the checker
|
||||
if isHTTPURL(manifestPath) {
|
||||
tmpPath, tmpErr := mfa.fetchManifestToTemp(ctx, manifestPath)
|
||||
@@ -315,11 +311,13 @@ func (mfa *CLIApp) checkManifestOperation(
|
||||
manifestPath = tmpPath
|
||||
}
|
||||
|
||||
basePath := cmd.String("base")
|
||||
showProgress := cmd.Bool("progress")
|
||||
|
||||
log.Infof("checking manifest %s with base %s", manifestPath, basePath)
|
||||
|
||||
// Create checker
|
||||
//nolint:contextcheck // mfer loads a manifest without a context
|
||||
chk, err := mfer.NewChecker(&mfer.CheckerOptions{
|
||||
ManifestPath: manifestPath,
|
||||
BasePath: basePath,
|
||||
|
||||
+14
-444
@@ -8,7 +8,6 @@ import (
|
||||
"fmt"
|
||||
"io"
|
||||
"math/rand"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
@@ -654,10 +653,11 @@ func runCheckAfterRewrite(t *testing.T, rewritten, msg string) {
|
||||
|
||||
// TestCheckRequireSignatureRefusesOtherSigningKey runs check
|
||||
// --require-signature on a manifest signed by another key whose embedded
|
||||
// public key block also holds the required key. check must refuse it.
|
||||
// public key block also holds the required key. check must refuse it. It
|
||||
// needs gpg and is skipped without it, as the other signing tests are.
|
||||
//
|
||||
//nolint:paralleltest // signedManifest calls t.Setenv, which bars t.Parallel
|
||||
func TestCheckRequireSignatureRefusesOtherSigningKey(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
content := []byte("signed file")
|
||||
manifest, required := manifestSignedByAnotherKey(t,
|
||||
map[string][]byte{testFileTxt: content})
|
||||
@@ -694,33 +694,6 @@ func TestCheckCommandWithSizeMismatch(t *testing.T) {
|
||||
"check should have failed for size mismatch")
|
||||
}
|
||||
|
||||
// TestCheckQuietShowsFailedFiles asserts that check -q prints nothing for a
|
||||
// tree that passes and still names each file that fails: --quiet hides
|
||||
// everything except errors, and a failed file is an error.
|
||||
func TestCheckQuietShowsFailedFiles(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
fs := afero.NewMemMapFs()
|
||||
require.NoError(t, fs.MkdirAll(testDir, 0o755))
|
||||
writeTestFile(t, fs, testFile1, "hello world")
|
||||
|
||||
opts := testOpts([]string{testApp, cmdGenerate, "-q", "-o", testManifest, testDir}, fs)
|
||||
require.Equal(t, 0, runCLI(opts), "generate failed: %s", testStderr(t, opts))
|
||||
|
||||
args := []string{testApp, cmdCheck, "-q", testFlagBase, testDir, testManifest}
|
||||
|
||||
opts = testOpts(args, fs)
|
||||
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
|
||||
assert.Empty(t, testStdout(t, opts))
|
||||
assert.Empty(t, testStderr(t, opts))
|
||||
|
||||
writeTestFile(t, fs, testFile1, "HELLO WORLD")
|
||||
|
||||
opts = testOpts(args, fs)
|
||||
assert.Equal(t, 1, runCLI(opts))
|
||||
assert.Contains(t, testStderr(t, opts), "HASH_MISMATCH: file1.txt")
|
||||
}
|
||||
|
||||
func TestBannerOutput(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -953,9 +926,8 @@ func TestCheckWarnsAboutUnlistedFiles(t *testing.T) {
|
||||
assert.Equal(t, 1, exitCode, "stderr: %s", stderr)
|
||||
assert.Contains(t, stderr, unlisted)
|
||||
|
||||
exitCode, stderr = check("-q", testFlagNoExtra)
|
||||
exitCode, _ = check("-q", testFlagNoExtra)
|
||||
assert.Equal(t, 1, exitCode)
|
||||
assert.Contains(t, stderr, unlisted)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -1007,90 +979,6 @@ func TestCheckNeverReportsManifest(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// The directory setupManifestInSubdir makes and the manifest it writes there,
|
||||
// relative to the working directory it sets.
|
||||
const (
|
||||
testSubdir = "sub"
|
||||
testSubdirManifest = testSubdir + "/" + defaultManifestName
|
||||
)
|
||||
|
||||
// setupManifestInSubdir makes a temp dir holding file.txt and sub/b.txt,
|
||||
// where sub/index.mf is the manifest gen writes for sub, and makes it the
|
||||
// working directory, so a test calling it cannot run in parallel. It returns
|
||||
// the temp dir.
|
||||
func setupManifestInSubdir(t *testing.T) string {
|
||||
t.Helper()
|
||||
|
||||
root := t.TempDir()
|
||||
sub := filepath.Join(root, testSubdir)
|
||||
|
||||
fs := afero.NewOsFs()
|
||||
require.NoError(t, fs.MkdirAll(sub, 0o750))
|
||||
writeTestFile(t, fs, filepath.Join(root, testFileTxt), "not in the manifest")
|
||||
writeTestFile(t, fs, filepath.Join(sub, "b.txt"), "in the manifest")
|
||||
|
||||
opts := testOpts([]string{
|
||||
testApp, cmdGenerate, "-q", "-o", filepath.Join(sub, defaultManifestName), sub,
|
||||
}, fs)
|
||||
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
|
||||
|
||||
t.Chdir(root)
|
||||
|
||||
return root
|
||||
}
|
||||
|
||||
// TestCheckResolvesEntriesAgainstManifestDirectory runs check from the
|
||||
// directory above sub, on the manifest in sub. Without --base, the
|
||||
// manifest's entries are looked for in sub, whether check is given the
|
||||
// manifest or sub, and the files above sub are not reported. --base names
|
||||
// the directory to look in instead, the current one included.
|
||||
//
|
||||
//nolint:paralleltest // changes the process-global working directory
|
||||
func TestCheckResolvesEntriesAgainstManifestDirectory(t *testing.T) {
|
||||
root := setupManifestInSubdir(t)
|
||||
|
||||
for _, tc := range []struct {
|
||||
args []string
|
||||
exitCode int
|
||||
failure string // a line check must print, if any
|
||||
}{
|
||||
{[]string{testSubdir}, 0, ""},
|
||||
{[]string{testSubdirManifest}, 0, ""},
|
||||
{[]string{filepath.Join(root, testSubdir)}, 0, ""},
|
||||
{[]string{testFlagBase, testSubdir, testSubdirManifest}, 0, ""},
|
||||
{[]string{testFlagBase, ".", testSubdirManifest}, 1, "MISSING: b.txt"},
|
||||
} {
|
||||
t.Run(strings.Join(tc.args, " "), func(t *testing.T) {
|
||||
opts := testOpts(slices.Concat(
|
||||
[]string{testApp, cmdCheck, testFlagNoExtra}, tc.args,
|
||||
), afero.NewOsFs())
|
||||
assert.Equal(t, tc.exitCode, runCLI(opts), "stderr: %s", testStderr(t, opts))
|
||||
assert.Contains(t, testStderr(t, opts), tc.failure)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestCheckURLManifestResolvesEntriesAgainstCurrentDirectory runs check on
|
||||
// a manifest given by URL, from a directory holding the file it lists: the
|
||||
// file is looked for there.
|
||||
//
|
||||
//nolint:paralleltest // changes the process-global working directory
|
||||
func TestCheckURLManifestResolvesEntriesAgainstCurrentDirectory(t *testing.T) {
|
||||
files := map[string][]byte{testFileTxt: []byte("hello")}
|
||||
|
||||
server := httptest.NewServer(fetchTestHandler(manifestOf(t, files), files))
|
||||
defer server.Close()
|
||||
|
||||
cwd := chdirTemp(t)
|
||||
require.NoError(t,
|
||||
os.WriteFile(filepath.Join(cwd, testFileTxt), files[testFileTxt], 0o600))
|
||||
|
||||
opts := testOpts([]string{
|
||||
testApp, cmdCheck, testFlagNoExtra, server.URL + "/" + defaultManifestName,
|
||||
}, afero.NewOsFs())
|
||||
assert.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
|
||||
}
|
||||
|
||||
// unlistableDirFs is a filesystem on which one directory cannot be listed.
|
||||
type unlistableDirFs struct {
|
||||
afero.Fs
|
||||
@@ -1316,225 +1204,6 @@ func TestGenerateLeavesLeftoverTempFileOutOfListing(t *testing.T) {
|
||||
assert.Equal(t, []string{testFileTxt}, manifestPaths(t, fs, output))
|
||||
}
|
||||
|
||||
// writeTestTree writes file.txt and sub/nested.txt under dir.
|
||||
func writeTestTree(t *testing.T, fs afero.Fs, dir string) {
|
||||
t.Helper()
|
||||
|
||||
require.NoError(t, fs.MkdirAll(filepath.Join(dir, testSubdir), 0o750))
|
||||
writeTestFile(t, fs, filepath.Join(dir, testFileTxt), "hello")
|
||||
writeTestFile(t, fs, filepath.Join(dir, testSubdir, "nested.txt"), "in sub")
|
||||
}
|
||||
|
||||
// TestGenerateDefaultOutput runs gen without --output on one directory or
|
||||
// one file: it writes index.mf in that directory, or beside that file,
|
||||
// listing each file by its path under the directory index.mf is in, and
|
||||
// check given that directory passes.
|
||||
func TestGenerateDefaultOutput(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// Paths are relative to a temp dir holding file.txt and sub/nested.txt.
|
||||
for name, tc := range map[string]struct {
|
||||
input, output string
|
||||
listed []string
|
||||
}{
|
||||
"directory": {
|
||||
".", defaultManifestName, []string{testFileTxt, "sub/nested.txt"},
|
||||
},
|
||||
"subdirectory": {testSubdir, testSubdirManifest, []string{"nested.txt"}},
|
||||
"file": {testFileTxt, defaultManifestName, []string{testFileTxt}},
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
root := t.TempDir()
|
||||
fs := afero.NewOsFs()
|
||||
writeTestTree(t, fs, root)
|
||||
|
||||
opts := testOpts([]string{
|
||||
testApp, cmdGenerate, "-q", filepath.Join(root, tc.input),
|
||||
}, fs)
|
||||
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
|
||||
|
||||
output := filepath.Join(root, tc.output)
|
||||
assert.ElementsMatch(t, tc.listed, manifestPaths(t, fs, output))
|
||||
|
||||
opts = testOpts([]string{
|
||||
testApp, cmdCheck, "-q", filepath.Dir(output),
|
||||
}, fs)
|
||||
assert.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestGenerateSeveralPathsDefaultOutput runs gen without --output on two
|
||||
// directories: it writes index.mf in the current directory, listing both
|
||||
// directories' files, and writes no index.mf in either directory.
|
||||
//
|
||||
//nolint:paralleltest // changes the process-global working directory
|
||||
func TestGenerateSeveralPathsDefaultOutput(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
fs := afero.NewOsFs()
|
||||
dirs := []string{"first", "second"}
|
||||
|
||||
for _, dir := range dirs {
|
||||
require.NoError(t, fs.MkdirAll(filepath.Join(root, dir), 0o750))
|
||||
writeTestFile(t, fs, filepath.Join(root, dir, dir+".txt"), dir)
|
||||
}
|
||||
|
||||
t.Chdir(root)
|
||||
|
||||
opts := testOpts(append([]string{testApp, cmdGenerate, "-q"}, dirs...), fs)
|
||||
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
|
||||
|
||||
assert.ElementsMatch(t, []string{"first.txt", "second.txt"},
|
||||
manifestPaths(t, fs, filepath.Join(root, defaultManifestName)))
|
||||
|
||||
for _, dir := range dirs {
|
||||
exists, err := afero.Exists(fs, filepath.Join(root, dir, defaultManifestName))
|
||||
require.NoError(t, err)
|
||||
assert.False(t, exists, "index.mf written in %s", dir)
|
||||
}
|
||||
}
|
||||
|
||||
// testLink is the name of the symlink to a tree that the
|
||||
// DirectoryNamedThroughSymlink tests make in a temp dir.
|
||||
const testLink = "link"
|
||||
|
||||
// TestGenerateDirectoryNamedThroughSymlink runs gen on a directory named
|
||||
// through a symlink, given as the argument or as the working directory: the
|
||||
// manifest lists the files in the directory the symlink points to, and
|
||||
// leaves out a symlink inside it, as gen does without --follow-symlinks.
|
||||
//
|
||||
//nolint:paralleltest // changes the process-global working directory
|
||||
func TestGenerateDirectoryNamedThroughSymlink(t *testing.T) {
|
||||
// Paths are relative to a temp dir holding data and link, a symlink to
|
||||
// data.
|
||||
for name, tc := range map[string]struct {
|
||||
workDir string
|
||||
args []string
|
||||
}{
|
||||
"argument": {".", []string{testLink}},
|
||||
"working directory": {testLink, nil},
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
data := filepath.Join(root, "data")
|
||||
|
||||
fs := afero.NewOsFs()
|
||||
writeTestTree(t, fs, data)
|
||||
require.NoError(t,
|
||||
os.Symlink(testFileTxt, filepath.Join(data, "alias.txt")))
|
||||
require.NoError(t, os.Symlink(data, filepath.Join(root, testLink)))
|
||||
|
||||
// t.Chdir sets PWD to the path it is given, as a shell does, and
|
||||
// os.Getwd returns PWD when it names the working directory.
|
||||
t.Chdir(filepath.Join(root, tc.workDir))
|
||||
|
||||
opts := testOpts(slices.Concat(
|
||||
[]string{testApp, cmdGenerate, "-q"}, tc.args,
|
||||
), fs)
|
||||
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
|
||||
|
||||
assert.ElementsMatch(t, []string{testFileTxt, "sub/nested.txt"},
|
||||
manifestPaths(t, fs, filepath.Join(data, defaultManifestName)))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestGenerateBytesDoNotDependOnOutput runs gen --seed on one tree, each
|
||||
// time writing to another file, over a file already there and beside an
|
||||
// earlier run's temp file: neither is listed, and what is listed depends
|
||||
// only on the tree, so every manifest has the same bytes.
|
||||
func TestGenerateBytesDoNotDependOnOutput(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
root := t.TempDir()
|
||||
tree := filepath.Join(root, "tree")
|
||||
defaultOutput := filepath.Join(tree, defaultManifestName)
|
||||
|
||||
fs := afero.NewOsFs()
|
||||
writeTestTree(t, fs, tree)
|
||||
|
||||
var first []byte
|
||||
|
||||
for _, output := range []string{
|
||||
defaultOutput,
|
||||
filepath.Join(tree, "listing.mf"),
|
||||
filepath.Join(tree, testSubdir, "listing.mf"),
|
||||
filepath.Join(root, "outside.mf"),
|
||||
} {
|
||||
writeTestFile(t, fs, output, "previous manifest")
|
||||
writeTestFile(t, fs, manifestTempPath(output), "part of a manifest")
|
||||
|
||||
args := []string{testApp, cmdGenerate, "-q", "-f", "--seed", "mfer"}
|
||||
if output != defaultOutput {
|
||||
args = append(args, "-o", output)
|
||||
}
|
||||
|
||||
args = append(args, tree)
|
||||
|
||||
opts := testOpts(args, fs)
|
||||
require.Equal(t, 0, runCLI(opts),
|
||||
"output %s, stderr: %s", output, testStderr(t, opts))
|
||||
|
||||
got, err := afero.ReadFile(fs, output)
|
||||
require.NoError(t, err)
|
||||
require.NoError(t, fs.Remove(output))
|
||||
|
||||
if first == nil {
|
||||
first = got
|
||||
}
|
||||
|
||||
assert.Equal(t, first, got, "output %s", output)
|
||||
}
|
||||
}
|
||||
|
||||
// TestGenerateRefusesExistingDefaultOutput runs gen without --output or
|
||||
// --force on a directory already holding index.mf: gen fails, naming that
|
||||
// file, and leaves it as it was.
|
||||
func TestGenerateRefusesExistingDefaultOutput(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
output := filepath.Join(testDir, defaultManifestName)
|
||||
|
||||
fs := afero.NewMemMapFs()
|
||||
require.NoError(t, fs.MkdirAll(testDir, 0o755))
|
||||
writeTestFile(t, fs, testFile1, "hello")
|
||||
writeTestFile(t, fs, output, "previous manifest")
|
||||
|
||||
opts := testOpts([]string{testApp, cmdGenerate, "-q", testDir}, fs)
|
||||
assert.Equal(t, 1, runCLI(opts))
|
||||
assert.Contains(t, testStderr(t, opts),
|
||||
"output file "+output+" already exists (use --force to overwrite)")
|
||||
|
||||
content, err := afero.ReadFile(fs, output)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "previous manifest", string(content))
|
||||
}
|
||||
|
||||
// TestGenerateRefusesEmptyOutput runs gen with --force and an --output
|
||||
// given an empty value, as an unset shell variable gives it, on a
|
||||
// directory already holding index.mf: gen fails and leaves that file as it
|
||||
// was.
|
||||
func TestGenerateRefusesEmptyOutput(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
output := filepath.Join(testDir, defaultManifestName)
|
||||
|
||||
fs := afero.NewMemMapFs()
|
||||
require.NoError(t, fs.MkdirAll(testDir, 0o755))
|
||||
writeTestFile(t, fs, testFile1, "hello")
|
||||
writeTestFile(t, fs, output, "previous manifest")
|
||||
|
||||
opts := testOpts([]string{testApp, cmdGenerate, "-q", "-f", "-o", "", testDir}, fs)
|
||||
assert.Equal(t, 1, runCLI(opts))
|
||||
assert.Contains(t, testStderr(t, opts), errEmptyOutput.Error())
|
||||
|
||||
content, err := afero.ReadFile(fs, output)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "previous manifest", string(content))
|
||||
}
|
||||
|
||||
func TestGenerateAtomicWriteUsesTemp(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -1692,119 +1361,20 @@ func TestGenerateValidatesInputPaths(t *testing.T) {
|
||||
})
|
||||
}
|
||||
|
||||
// generateTestDir writes testFile1 and runs gen on testDir, which writes
|
||||
// the manifest to testDir/index.mf.
|
||||
func generateTestDir(t *testing.T, fs afero.Fs) {
|
||||
t.Helper()
|
||||
|
||||
require.NoError(t, fs.MkdirAll(testDir, 0o755))
|
||||
writeTestFile(t, fs, testFile1, "content")
|
||||
|
||||
opts := testOpts([]string{testApp, cmdGenerate, "-q", testDir}, fs)
|
||||
require.Equal(t, 0, runCLI(opts), testStderr(t, opts))
|
||||
}
|
||||
|
||||
// TestCheckReadsOptionAfterPath runs check on an unsigned manifest with
|
||||
// --require-signature after the directory. check must refuse the manifest.
|
||||
func TestCheckReadsOptionAfterPath(t *testing.T) {
|
||||
// TestFlagAfterArgumentIsArgument asserts that flags are read only before a
|
||||
// command's first argument: after it, -v is a path, not the verbose flag.
|
||||
func TestFlagAfterArgumentIsArgument(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
fs := afero.NewMemMapFs()
|
||||
generateTestDir(t, fs)
|
||||
require.NoError(t, fs.MkdirAll(testDir, 0o755))
|
||||
writeTestFile(t, fs, testFile1, "content")
|
||||
|
||||
opts := testOpts([]string{
|
||||
testApp, cmdCheck, "-q", testDir, "--" + flagRequireSignature, msgFpA,
|
||||
}, fs)
|
||||
assert.Equal(t, 1, runCLI(opts))
|
||||
assert.Contains(t, testStderr(t, opts), "manifest is not signed")
|
||||
}
|
||||
opts := testOpts([]string{testApp, cmdGenerate, testDir, "-v"}, fs)
|
||||
exitCode := runCLI(opts)
|
||||
|
||||
// TestFreshenReadsOptionAfterPath runs freshen with --include-permissions
|
||||
// after the directory. freshen must record the file's mode.
|
||||
func TestFreshenReadsOptionAfterPath(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
fs := afero.NewOsFs()
|
||||
root, manifestPath := setupFreshenDir(t, fs,
|
||||
map[string]string{testFileTxt: "a file"})
|
||||
require.NoError(t, fs.Chmod(filepath.Join(root, testFileTxt), 0o640))
|
||||
|
||||
opts := testOpts([]string{
|
||||
testApp, cmdFreshen, "-q", root, "--" + flagIncludePermissions,
|
||||
}, fs)
|
||||
require.Equal(t, 0, runCLI(opts), testStderr(t, opts))
|
||||
|
||||
files := manifestFiles(t, fs, manifestPath)
|
||||
require.Len(t, files, 1)
|
||||
assert.Equal(t, uint32(0o640), files[0].GetMode())
|
||||
}
|
||||
|
||||
// TestFetchReadsOptionAfterURL runs fetch with --dest after the URL. The
|
||||
// file and the manifest must land in that directory and nothing in the
|
||||
// current one.
|
||||
//
|
||||
//nolint:paralleltest // changes the process-global working directory
|
||||
func TestFetchReadsOptionAfterURL(t *testing.T) {
|
||||
content := []byte("fetched")
|
||||
manifest := manifestOf(t, map[string][]byte{testFileTxt: content})
|
||||
|
||||
server := httptest.NewServer(fetchTestHandler(manifest,
|
||||
map[string][]byte{testFileTxt: content}))
|
||||
defer server.Close()
|
||||
|
||||
cwd := chdirTemp(t)
|
||||
dest := t.TempDir()
|
||||
|
||||
opts := testOpts([]string{
|
||||
testApp, cmdFetch, "-q", server.URL, "--" + flagDest, dest,
|
||||
}, afero.NewOsFs())
|
||||
require.Equal(t, 0, runCLI(opts), testStderr(t, opts))
|
||||
|
||||
assert.Equal(t,
|
||||
map[string][]byte{testFileTxt: content, defaultManifestName: manifest},
|
||||
filesUnder(t, dest))
|
||||
assert.Empty(t, filesUnder(t, cwd), "fetch wrote outside --dest")
|
||||
}
|
||||
|
||||
// TestOneArgumentCommandsRefuseSecond runs each command that takes one
|
||||
// argument with two. It must fail naming the second.
|
||||
func TestOneArgumentCommandsRefuseSecond(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, command := range []string{
|
||||
cmdCheck, cmdFreshen, cmdList, cmdExport, cmdFetch,
|
||||
} {
|
||||
t.Run(command, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
fs := afero.NewMemMapFs()
|
||||
generateTestDir(t, fs)
|
||||
|
||||
opts := testOpts([]string{testApp, command, testDir, "/second"}, fs)
|
||||
assert.Equal(t, 1, runCLI(opts))
|
||||
assert.Contains(t, testStderr(t, opts),
|
||||
`unexpected argument "/second"`)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestGenerateReadsPathAfterDoubleDash runs gen -- -x where -x is a
|
||||
// directory. -x must be read as the path to list, not as an option.
|
||||
//
|
||||
//nolint:paralleltest // changes the process-global working directory
|
||||
func TestGenerateReadsPathAfterDoubleDash(t *testing.T) {
|
||||
chdirTemp(t)
|
||||
|
||||
fs := afero.NewOsFs()
|
||||
require.NoError(t, fs.Mkdir("-x", 0o755))
|
||||
writeTestFile(t, fs, filepath.Join("-x", testFileTxt), "content")
|
||||
|
||||
opts := testOpts([]string{testApp, cmdGenerate, "-q", "--", "-x"}, fs)
|
||||
require.Equal(t, 0, runCLI(opts), testStderr(t, opts))
|
||||
|
||||
files := manifestFiles(t, fs, filepath.Join("-x", defaultManifestName))
|
||||
require.Len(t, files, 1)
|
||||
assert.Equal(t, testFileTxt, files[0].GetPath())
|
||||
assert.Equal(t, 1, exitCode)
|
||||
assert.Contains(t, testStderr(t, opts), "path does not exist: -v")
|
||||
}
|
||||
|
||||
func TestCheckDetectsManifestCorruption(t *testing.T) {
|
||||
|
||||
+40
-66
@@ -4,18 +4,14 @@ package cli
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/hex"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"slices"
|
||||
"testing"
|
||||
|
||||
"github.com/ProtonMail/go-crypto/openpgp"
|
||||
"github.com/ProtonMail/go-crypto/openpgp/armor"
|
||||
"github.com/ProtonMail/go-crypto/openpgp/packet"
|
||||
"github.com/spf13/afero"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
@@ -112,10 +108,11 @@ func TestVerifyRequiredSignerMessages(t *testing.T) {
|
||||
// manifest. The signing key's fingerprint is whatever the generated key
|
||||
// produced, so it is read back from the checker and substituted into the
|
||||
// expected string; the required signer is a fixed value that cannot match
|
||||
// it.
|
||||
// it. Requires gpg and is skipped where it is absent, as the other signing
|
||||
// tests are.
|
||||
//
|
||||
//nolint:paralleltest // signedManifest calls t.Setenv, which bars t.Parallel
|
||||
func TestSignerMismatchMessage(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
chk := signedChecker(t,
|
||||
signedManifest(t, map[string][]byte{"f.txt": []byte("signed file")}))
|
||||
|
||||
@@ -126,48 +123,43 @@ func TestSignerMismatchMessage(t *testing.T) {
|
||||
" does not match required "+msgFpB)
|
||||
}
|
||||
|
||||
// testSecretKey returns a new OpenPGP key with its secret key, armored, as
|
||||
// gpg --export-secret-keys --armor writes it, and the key's fingerprint.
|
||||
// The key is protected by passphrase unless that is nil. config sets how
|
||||
// the key is made; without one it is an Ed25519 key, which is quick to
|
||||
// make.
|
||||
func testSecretKey(
|
||||
t *testing.T, passphrase []byte, config *packet.Config,
|
||||
) ([]byte, string) {
|
||||
t.Helper()
|
||||
|
||||
if config == nil {
|
||||
config = &packet.Config{Algorithm: packet.PubKeyAlgoEdDSA}
|
||||
}
|
||||
|
||||
key, err := openpgp.NewEntity("MFER Test Key", "", "test@mfer.test", config)
|
||||
require.NoError(t, err)
|
||||
|
||||
if passphrase != nil {
|
||||
require.NoError(t, key.EncryptPrivateKeys(passphrase, nil))
|
||||
}
|
||||
|
||||
var buf bytes.Buffer
|
||||
|
||||
w, err := armor.Encode(&buf, openpgp.PrivateKeyType, nil)
|
||||
require.NoError(t, err)
|
||||
require.NoError(t, key.SerializePrivateWithoutSigning(w, nil))
|
||||
require.NoError(t, w.Close())
|
||||
|
||||
return buf.Bytes(), strings.ToUpper(hex.EncodeToString(key.PrimaryKey.Fingerprint))
|
||||
}
|
||||
|
||||
// signedManifest returns a manifest of files signed by a new OpenPGP key.
|
||||
// signedManifest returns a manifest of files signed by a throwaway GPG key
|
||||
// generated in a temporary GNUPGHOME, which it leaves set for the rest of
|
||||
// the test.
|
||||
func signedManifest(t *testing.T, files map[string][]byte) []byte {
|
||||
t.Helper()
|
||||
|
||||
secretKey, _ := testSecretKey(t, nil, nil)
|
||||
_, err := exec.LookPath("gpg")
|
||||
if err != nil {
|
||||
t.Skip("gpg not installed, skipping signing test")
|
||||
}
|
||||
|
||||
gpgHome := t.TempDir()
|
||||
params := "%no-protection\n" +
|
||||
"Key-Type: RSA\nKey-Length: 2048\n" +
|
||||
"Name-Real: MFER Test Key\nName-Email: test@mfer.test\n" +
|
||||
"Expire-Date: 0\n%commit\n"
|
||||
paramsFile := filepath.Join(gpgHome, "key-params")
|
||||
require.NoError(t, os.WriteFile(paramsFile, []byte(params), 0o600))
|
||||
|
||||
//nolint:gosec // paramsFile is a test-controlled path inside t.TempDir()
|
||||
cmd := exec.CommandContext(context.Background(), "gpg",
|
||||
"--batch", "--gen-key", paramsFile)
|
||||
|
||||
cmd.Env = append(os.Environ(), "GNUPGHOME="+gpgHome)
|
||||
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Skipf("failed to generate test GPG key: %v: %s", err, out)
|
||||
}
|
||||
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
b := mfer.NewBuilder()
|
||||
b.SetSigningOptions(&mfer.SigningOptions{SecretKey: secretKey})
|
||||
b.SetSigningOptions(&mfer.SigningOptions{KeyID: mfer.GPGKeyID("test@mfer.test")})
|
||||
|
||||
for path, content := range files {
|
||||
_, err := b.AddFile(mfer.RelFilePath(path), mfer.FileSize(len(content)),
|
||||
_, err = b.AddFile(mfer.RelFilePath(path), mfer.FileSize(len(content)),
|
||||
mfer.ModTime{}, 0, bytes.NewReader(content), nil)
|
||||
require.NoError(t, err)
|
||||
}
|
||||
@@ -198,8 +190,8 @@ func signedChecker(t *testing.T, manifest []byte) *mfer.Checker {
|
||||
}
|
||||
|
||||
// manifestSignedByAnotherKey returns a manifest of files and the
|
||||
// fingerprint of a new key, the required key, that did not sign it.
|
||||
// The manifest is signed by a second new key; its embedded public key
|
||||
// fingerprint of a throwaway key, the required key, that did not sign it.
|
||||
// The manifest is signed by a second throwaway key; its embedded public key
|
||||
// block holds the required key followed by the second key, and its signer
|
||||
// field names the required key.
|
||||
func manifestSignedByAnotherKey(
|
||||
@@ -215,26 +207,8 @@ func manifestSignedByAnotherKey(
|
||||
require.NoError(t, proto.Unmarshal(
|
||||
signedManifest(t, files)[len(mfer.MAGIC):], outer))
|
||||
|
||||
// One armored block holding both keys, as gpg --export --armor writes
|
||||
// two keys.
|
||||
var block bytes.Buffer
|
||||
|
||||
w, err := armor.Encode(&block, openpgp.PublicKeyType, nil)
|
||||
require.NoError(t, err)
|
||||
|
||||
for _, key := range [][]byte{
|
||||
required.GetSigningPubKey(), outer.GetSigningPubKey(),
|
||||
} {
|
||||
decoded, err := armor.Decode(bytes.NewReader(key))
|
||||
require.NoError(t, err)
|
||||
|
||||
_, err = io.Copy(w, decoded.Body)
|
||||
require.NoError(t, err)
|
||||
}
|
||||
|
||||
require.NoError(t, w.Close())
|
||||
|
||||
outer.SigningPubKey = block.Bytes()
|
||||
outer.SigningPubKey = slices.Concat(
|
||||
required.GetSigningPubKey(), outer.GetSigningPubKey())
|
||||
outer.Signer = required.GetSigner()
|
||||
|
||||
data, err := proto.Marshal(outer)
|
||||
|
||||
@@ -36,6 +36,7 @@ func (mfa *CLIApp) exportManifestOperation(
|
||||
|
||||
defer func() { _ = rc.Close() }()
|
||||
|
||||
//nolint:contextcheck // mfer loads a manifest without a context
|
||||
manifest, err := mfer.NewManifestFromReader(rc)
|
||||
if err != nil {
|
||||
return fmt.Errorf("parse manifest: %w", err)
|
||||
|
||||
@@ -455,6 +455,7 @@ func (mfa *CLIApp) fetchManifest(
|
||||
}
|
||||
|
||||
// Parse manifest
|
||||
//nolint:contextcheck // mfer loads a manifest without a context
|
||||
manifest, err := mfer.NewManifestFromReader(bytes.NewReader(manifestData))
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("parse manifest: %w", err)
|
||||
@@ -462,6 +463,7 @@ func (mfa *CLIApp) fetchManifest(
|
||||
|
||||
requiredSigner := cmd.String(flagRequireSignature)
|
||||
if requiredSigner != "" {
|
||||
//nolint:contextcheck // mfer loads a manifest without a context
|
||||
err = verifyFetchedSigner(manifestData, requiredSigner)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
|
||||
@@ -1191,23 +1191,20 @@ func TestFetchIntoDest(t *testing.T) {
|
||||
// with check's message before it downloads or writes anything; the
|
||||
// required key lets it through. A manifest signed by another key whose
|
||||
// embedded public key block also holds the required key must stop fetch
|
||||
// too.
|
||||
// too. The signed cases need gpg and are skipped without it, as the other
|
||||
// signing tests are.
|
||||
//
|
||||
//nolint:paralleltest // signedManifest calls t.Setenv, which bars t.Parallel
|
||||
func TestFetchRequireSignature(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
files := map[string][]byte{testFileTxt: []byte("signed file")}
|
||||
|
||||
t.Run("unsigned", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
assertFetchRefused(t, manifestOf(t, files), files,
|
||||
"manifest is not signed, but signature from "+msgFpA+" is required",
|
||||
"--"+flagRequireSignature, msgFpA)
|
||||
})
|
||||
|
||||
t.Run("signed", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
manifest := signedManifest(t, files)
|
||||
|
||||
signer := string(signedChecker(t, manifest).Signer())
|
||||
@@ -1230,8 +1227,6 @@ func TestFetchRequireSignature(t *testing.T) {
|
||||
})
|
||||
|
||||
t.Run("signed by another key embedded after the required one", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
manifest, required := manifestSignedByAnotherKey(t, files)
|
||||
|
||||
assertFetchRefused(t, manifest, files,
|
||||
|
||||
+20
-48
@@ -28,15 +28,9 @@ const (
|
||||
scanProgressInterval = 100
|
||||
)
|
||||
|
||||
var (
|
||||
// errEntryMissingMtime indicates a manifest entry that carries no
|
||||
// modification time where one is required to carry it forward unchanged.
|
||||
errEntryMissingMtime = errors.New("manifest entry has no mtime")
|
||||
|
||||
// errEntryMissingHash indicates a manifest entry that carries no hash
|
||||
// to carry forward.
|
||||
errEntryMissingHash = errors.New("manifest entry has no hash")
|
||||
)
|
||||
// errEntryMissingMtime indicates a manifest entry that carries no
|
||||
// modification time where one is required to carry it forward unchanged.
|
||||
var errEntryMissingMtime = errors.New("manifest entry has no mtime")
|
||||
|
||||
// FreshenStatus contains progress information for the freshen operation.
|
||||
type FreshenStatus struct {
|
||||
@@ -198,13 +192,6 @@ func (s *freshenScanner) walk(path string, info fs.FileInfo, walkErr error) erro
|
||||
info = realInfo
|
||||
}
|
||||
|
||||
// Opening a named pipe can block, and reading a device may never end.
|
||||
if !info.Mode().IsRegular() {
|
||||
log.Verbosef("skipping %s: not a regular file", relPath)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
for _, excluded := range s.excluded {
|
||||
if os.SameFile(info, excluded) {
|
||||
return nil
|
||||
@@ -352,7 +339,7 @@ func writeFreshenedManifest(
|
||||
|
||||
// newFreshenBuilder constructs the manifest builder configured from CLI
|
||||
// flags.
|
||||
func (mfa *CLIApp) newFreshenBuilder(cmd *cli.Command) (*mfer.Builder, error) {
|
||||
func newFreshenBuilder(cmd *cli.Command) *mfer.Builder {
|
||||
builder := mfer.NewBuilder()
|
||||
if cmd.Bool("include-timestamps") {
|
||||
builder.SetIncludeTimestamps(true)
|
||||
@@ -360,15 +347,13 @@ func (mfa *CLIApp) newFreshenBuilder(cmd *cli.Command) (*mfer.Builder, error) {
|
||||
|
||||
// Set up signing options if sign-key is provided
|
||||
if signKey := cmd.String("sign-key"); signKey != "" {
|
||||
signing, err := mfa.signingOptions(signKey)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
builder.SetSigningOptions(signing)
|
||||
builder.SetSigningOptions(&mfer.SigningOptions{
|
||||
KeyID: mfer.GPGKeyID(signKey),
|
||||
})
|
||||
log.Infof("signing manifest with GPG key: %s", signKey)
|
||||
}
|
||||
|
||||
return builder, nil
|
||||
return builder
|
||||
}
|
||||
|
||||
// freshenScan runs the scan phase against the loaded manifest entries
|
||||
@@ -394,14 +379,6 @@ func (mfa *CLIApp) freshenScan(
|
||||
}
|
||||
}
|
||||
|
||||
// The walk does not follow a symlink at its top, so a base directory
|
||||
// named through one is resolved first. If that fails, the base is
|
||||
// walked as named and the walk reports the problem.
|
||||
resolved, err := filepath.EvalSymlinks(absBase)
|
||||
if err == nil {
|
||||
absBase = resolved
|
||||
}
|
||||
|
||||
scanner := &freshenScanner{
|
||||
fs: mfa.Fs,
|
||||
absBase: absBase,
|
||||
@@ -413,7 +390,7 @@ func (mfa *CLIApp) freshenScan(
|
||||
existingByPath: existingByPath,
|
||||
}
|
||||
|
||||
err = afero.Walk(mfa.Fs, absBase, scanner.walk)
|
||||
err := afero.Walk(mfa.Fs, absBase, scanner.walk)
|
||||
|
||||
if showProgress {
|
||||
log.ProgressDone()
|
||||
@@ -456,7 +433,7 @@ func hashTotals(entries []*freshenEntry) (int64, int64) {
|
||||
}
|
||||
|
||||
// runFreshenHash processes every entry through the hasher, aborting if
|
||||
// the context is canceled, and ends the hasher's progress line.
|
||||
// the context is canceled.
|
||||
func runFreshenHash(
|
||||
ctx context.Context, hasher *freshenHasher, entries []*freshenEntry,
|
||||
) error {
|
||||
@@ -473,10 +450,6 @@ func runFreshenHash(
|
||||
}
|
||||
}
|
||||
|
||||
if hasher.showProgress && hasher.filesToHash > 0 {
|
||||
log.ProgressDone()
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -513,6 +486,7 @@ func (mfa *CLIApp) freshenManifestOperation(
|
||||
) error {
|
||||
log.Debug("freshenManifestOperation()")
|
||||
|
||||
basePath := cmd.String("base")
|
||||
showProgress := cmd.Bool("progress")
|
||||
|
||||
// Find manifest file
|
||||
@@ -521,17 +495,13 @@ func (mfa *CLIApp) freshenManifestOperation(
|
||||
return err
|
||||
}
|
||||
|
||||
builder, err := mfa.newFreshenBuilder(cmd)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
//nolint:contextcheck // mfer loads a manifest without a context
|
||||
existingByPath, err := mfa.loadExistingEntries(manifestPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
absBase, err := filepath.Abs(resolveBasePath(cmd, manifestPath))
|
||||
absBase, err := filepath.Abs(basePath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("invalid base path: %w", err)
|
||||
}
|
||||
@@ -559,7 +529,7 @@ func (mfa *CLIApp) freshenManifestOperation(
|
||||
totalHashBytes: totalHashBytes,
|
||||
filesToHash: filesToHash,
|
||||
startHash: time.Now(),
|
||||
builder: builder,
|
||||
builder: newFreshenBuilder(cmd),
|
||||
}
|
||||
|
||||
err = runFreshenHash(ctx, hasher, scanner.entries)
|
||||
@@ -567,6 +537,10 @@ func (mfa *CLIApp) freshenManifestOperation(
|
||||
return err
|
||||
}
|
||||
|
||||
if showProgress && filesToHash > 0 {
|
||||
log.ProgressDone()
|
||||
}
|
||||
|
||||
// Print summary
|
||||
log.Infof("freshen complete: %d unchanged, %d changed, %d added, %d removed",
|
||||
scanner.unchanged, scanner.changed, scanner.added, removed)
|
||||
@@ -650,8 +624,6 @@ func addFileToBuilder(
|
||||
// Entries reach this path only when recordEntry classified them as
|
||||
// unchanged, which requires a recorded mtime, so an absent mtime here is
|
||||
// an error rather than something to paper over with the Unix epoch.
|
||||
// Loading refuses an entry with no hash; one that got here anyway is an
|
||||
// error too, not a file to leave out of the manifest.
|
||||
func addExistingToBuilder(b *mfer.Builder, entry *mfer.MFFilePath) error {
|
||||
mtime, ok := entryMtime(entry)
|
||||
if !ok {
|
||||
@@ -659,7 +631,7 @@ func addExistingToBuilder(b *mfer.Builder, entry *mfer.MFFilePath) error {
|
||||
}
|
||||
|
||||
if len(entry.GetHashes()) == 0 {
|
||||
return fmt.Errorf("%w: %s", errEntryMissingHash, entry.GetPath())
|
||||
return nil
|
||||
}
|
||||
|
||||
err := b.AddFileWithHash(mfer.RelFilePath(entry.GetPath()),
|
||||
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
@@ -301,84 +300,6 @@ func TestFreshenLeavesLeftoverTempFileOutOfListing(t *testing.T) {
|
||||
manifestPaths(t, fs, manifestPath))
|
||||
}
|
||||
|
||||
// TestFreshenResolvesEntriesAgainstManifestDirectory adds sub/c.txt, then
|
||||
// runs freshen from the directory above sub, on the manifest in sub.
|
||||
// Without --base, the manifest then lists the files in sub, whether freshen
|
||||
// is given the manifest or sub. --base names the directory to list instead,
|
||||
// the current one included.
|
||||
//
|
||||
//nolint:paralleltest // changes the process-global working directory
|
||||
func TestFreshenResolvesEntriesAgainstManifestDirectory(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
args []string
|
||||
want []string // the paths the manifest lists afterwards
|
||||
}{
|
||||
{[]string{testSubdir}, []string{"b.txt", "c.txt"}},
|
||||
{[]string{testSubdirManifest}, []string{"b.txt", "c.txt"}},
|
||||
{
|
||||
[]string{testFlagBase, ".", testSubdirManifest},
|
||||
[]string{testFileTxt, "sub/b.txt", "sub/c.txt"},
|
||||
},
|
||||
} {
|
||||
t.Run(strings.Join(tc.args, " "), func(t *testing.T) {
|
||||
fs := afero.NewOsFs()
|
||||
root := setupManifestInSubdir(t)
|
||||
writeTestFile(t, fs, filepath.Join(root, testSubdir, "c.txt"), "added")
|
||||
|
||||
opts := testOpts(slices.Concat(
|
||||
[]string{testApp, cmdFreshen, "-q"}, tc.args,
|
||||
), fs)
|
||||
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
|
||||
|
||||
assert.ElementsMatch(t, tc.want, manifestPaths(t, fs,
|
||||
filepath.Join(root, testSubdirManifest)))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestFreshenDirectoryNamedThroughSymlink adds a file to a tree after gen
|
||||
// made its manifest, then freshens it with the tree named through a symlink,
|
||||
// given as the argument or as the working directory: the manifest lists the
|
||||
// files in the tree, and leaves out a symlink inside it, as freshen does
|
||||
// without --follow-symlinks.
|
||||
//
|
||||
//nolint:paralleltest // changes the process-global working directory
|
||||
func TestFreshenDirectoryNamedThroughSymlink(t *testing.T) {
|
||||
// Paths are relative to a temp dir holding link, a symlink to the tree.
|
||||
for name, tc := range map[string]struct {
|
||||
workDir string
|
||||
args []string
|
||||
}{
|
||||
"argument": {".", []string{testLink}},
|
||||
"working directory": {testLink, nil},
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
fs := afero.NewOsFs()
|
||||
tree, manifestPath := setupFreshenDir(t, fs,
|
||||
map[string]string{testFileTxt: "in the tree"})
|
||||
writeTestFile(t, fs, filepath.Join(tree, "later.txt"), "added later")
|
||||
require.NoError(t,
|
||||
os.Symlink(testFileTxt, filepath.Join(tree, "alias.txt")))
|
||||
|
||||
root := t.TempDir()
|
||||
require.NoError(t, os.Symlink(tree, filepath.Join(root, testLink)))
|
||||
|
||||
// t.Chdir sets PWD to the path it is given, as a shell does, and
|
||||
// os.Getwd returns PWD when it names the working directory.
|
||||
t.Chdir(filepath.Join(root, tc.workDir))
|
||||
|
||||
opts := testOpts(slices.Concat(
|
||||
[]string{testApp, cmdFreshen, "-q"}, tc.args,
|
||||
), fs)
|
||||
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
|
||||
|
||||
assertManifestLists(t, fs, tree, manifestPath, map[string]string{
|
||||
testFileTxt: "in the tree", "later.txt": "added later",
|
||||
})
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestFreshenRecordEntryMtimePresence pins the behavior of recordEntry
|
||||
// with respect to MFFilePath.Mtime, which is a message pointer with
|
||||
// proto3 field presence and may legitimately be absent.
|
||||
@@ -471,24 +392,6 @@ func TestFreshenAddExistingRejectsMissingMtime(t *testing.T) {
|
||||
assert.Contains(t, err.Error(), "file1.txt")
|
||||
}
|
||||
|
||||
// TestFreshenAddExistingRejectsMissingHash pins that an entry with no hash
|
||||
// is an error rather than a file left out of the rebuilt manifest.
|
||||
func TestFreshenAddExistingRejectsMissingHash(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
b := mfer.NewBuilder()
|
||||
entry := &mfer.MFFilePath{
|
||||
Path: "data.bin",
|
||||
Size: 8,
|
||||
Mtime: &mfer.Timestamp{Seconds: 1_700_000_000},
|
||||
}
|
||||
|
||||
err := addExistingToBuilder(b, entry)
|
||||
require.ErrorIs(t, err, errEntryMissingHash)
|
||||
assert.Contains(t, err.Error(), "data.bin")
|
||||
assert.Zero(t, b.FileCount())
|
||||
}
|
||||
|
||||
// TestFreshenAddExistingRejectsShortHash pins that an existing manifest
|
||||
// entry whose hash the builder refuses is reported as a problem with the
|
||||
// manifest, with the command that fixes it.
|
||||
|
||||
+30
-86
@@ -4,7 +4,6 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"os/signal"
|
||||
"path/filepath"
|
||||
@@ -27,8 +26,6 @@ var (
|
||||
// rendered message stays exactly as mfer has always printed it.
|
||||
errOutputExists = errors.New(
|
||||
"already exists (use --force to overwrite)")
|
||||
// errEmptyOutput indicates --output given with an empty value.
|
||||
errEmptyOutput = errors.New("--output must not be empty")
|
||||
)
|
||||
|
||||
// reportEnumProgress renders enumeration progress until the channel
|
||||
@@ -91,40 +88,9 @@ func (mfa *CLIApp) collectInputPaths(args cli.Args) ([]string, error) {
|
||||
return paths, nil
|
||||
}
|
||||
|
||||
// outputPath returns the file gen writes the manifest to: the one --output
|
||||
// names, or else index.mf in the directory the only argument names, or
|
||||
// beside the file it names, or else in the current directory. An --output
|
||||
// given with an empty value is refused.
|
||||
func (mfa *CLIApp) outputPath(cmd *cli.Command) (string, error) {
|
||||
if cmd.IsSet("output") {
|
||||
output := cmd.String("output")
|
||||
if output == "" {
|
||||
return "", errEmptyOutput
|
||||
}
|
||||
|
||||
return output, nil
|
||||
}
|
||||
|
||||
if cmd.Args().Len() != 1 {
|
||||
return defaultManifestName, nil
|
||||
}
|
||||
|
||||
arg := cmd.Args().First()
|
||||
|
||||
// A path that does not exist is refused when it is enumerated.
|
||||
info, err := mfa.Fs.Stat(arg)
|
||||
if err == nil && !info.IsDir() {
|
||||
return filepath.Join(filepath.Dir(arg), defaultManifestName), nil
|
||||
}
|
||||
|
||||
return filepath.Join(arg, defaultManifestName), nil
|
||||
}
|
||||
|
||||
// buildScannerOptions constructs scanner options from the CLI flags and
|
||||
// the path the manifest is written to.
|
||||
func (mfa *CLIApp) buildScannerOptions(
|
||||
cmd *cli.Command, output string,
|
||||
) (*mfer.ScannerOptions, error) {
|
||||
// buildScannerOptions constructs scanner options from the CLI flags.
|
||||
func (mfa *CLIApp) buildScannerOptions(cmd *cli.Command) *mfer.ScannerOptions {
|
||||
output := cmd.String("output")
|
||||
opts := &mfer.ScannerOptions{
|
||||
IncludeDotfiles: cmd.Bool("include-dotfiles"),
|
||||
FollowSymLinks: cmd.Bool("follow-symlinks"),
|
||||
@@ -145,15 +111,13 @@ func (mfa *CLIApp) buildScannerOptions(
|
||||
|
||||
// Set up signing options if sign-key is provided
|
||||
if signKey := cmd.String("sign-key"); signKey != "" {
|
||||
signing, err := mfa.signingOptions(signKey)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
opts.SigningOptions = &mfer.SigningOptions{
|
||||
KeyID: mfer.GPGKeyID(signKey),
|
||||
}
|
||||
|
||||
opts.SigningOptions = signing
|
||||
log.Infof("signing manifest with GPG key: %s", signKey)
|
||||
}
|
||||
|
||||
return opts, nil
|
||||
return opts
|
||||
}
|
||||
|
||||
// enumerateInputs runs the enumeration phase over the argument paths,
|
||||
@@ -238,59 +202,23 @@ func (mfa *CLIApp) runEnumeratePhase(cmd *cli.Command, s *mfer.Scanner) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// runScanPhase reads the enumerated files and writes the manifest to out,
|
||||
// with optional progress reporting.
|
||||
func (mfa *CLIApp) runScanPhase(
|
||||
ctx context.Context, cmd *cli.Command, s *mfer.Scanner, out io.Writer,
|
||||
) error {
|
||||
var (
|
||||
scanProgress chan mfer.ScanStatus
|
||||
scanWg sync.WaitGroup
|
||||
)
|
||||
|
||||
if cmd.Bool("progress") {
|
||||
scanProgress = make(chan mfer.ScanStatus, 1)
|
||||
|
||||
scanWg.Add(1)
|
||||
|
||||
go reportScanProgress(scanProgress, &scanWg)
|
||||
}
|
||||
|
||||
err := s.ToManifest(ctx, out, scanProgress)
|
||||
|
||||
scanWg.Wait()
|
||||
|
||||
if err != nil {
|
||||
return fmt.Errorf("generate manifest: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (mfa *CLIApp) generateManifestOperation(
|
||||
ctx context.Context, cmd *cli.Command,
|
||||
) error {
|
||||
log.Debug("generateManifestOperation()")
|
||||
|
||||
outputPath, err := mfa.outputPath(cmd)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
opts, err := mfa.buildScannerOptions(cmd, outputPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
s := mfer.NewScannerWithOptions(opts)
|
||||
s := mfer.NewScannerWithOptions(mfa.buildScannerOptions(cmd))
|
||||
|
||||
// Phase 1: Enumeration - collect paths and stat files
|
||||
err = mfa.runEnumeratePhase(cmd, s)
|
||||
err := mfa.runEnumeratePhase(cmd, s)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
showProgress := cmd.Bool("progress")
|
||||
|
||||
// Check if output file exists
|
||||
outputPath := cmd.String("output")
|
||||
if exists, _ := afero.Exists(mfa.Fs, outputPath); exists && !cmd.Bool("force") {
|
||||
return fmt.Errorf("output file %s %w", outputPath, errOutputExists)
|
||||
}
|
||||
@@ -321,9 +249,25 @@ func (mfa *CLIApp) generateManifestOperation(
|
||||
}()
|
||||
|
||||
// Phase 2: Scan - read file contents and generate manifest
|
||||
err = mfa.runScanPhase(ctx, cmd, s, outFile)
|
||||
var (
|
||||
scanProgress chan mfer.ScanStatus
|
||||
scanWg sync.WaitGroup
|
||||
)
|
||||
|
||||
if showProgress {
|
||||
scanProgress = make(chan mfer.ScanStatus, 1)
|
||||
|
||||
scanWg.Add(1)
|
||||
|
||||
go reportScanProgress(scanProgress, &scanWg)
|
||||
}
|
||||
|
||||
err = s.ToManifest(ctx, outFile, scanProgress)
|
||||
|
||||
scanWg.Wait()
|
||||
|
||||
if err != nil {
|
||||
return err
|
||||
return fmt.Errorf("generate manifest: %w", err)
|
||||
}
|
||||
|
||||
// Close file before rename to ensure all data is flushed
|
||||
|
||||
@@ -29,6 +29,7 @@ func (mfa *CLIApp) listManifestOperation(ctx context.Context, cmd *cli.Command)
|
||||
|
||||
defer func() { _ = rc.Close() }()
|
||||
|
||||
//nolint:contextcheck // mfer loads a manifest without a context
|
||||
manifest, err := mfer.NewManifestFromReader(rc)
|
||||
if err != nil {
|
||||
return fmt.Errorf("parse manifest: %w", err)
|
||||
|
||||
@@ -6,7 +6,6 @@ import (
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -87,17 +86,3 @@ func (mfa *CLIApp) resolveManifestArg(cmd *cli.Command) (string, error) {
|
||||
|
||||
return findManifest(mfa.Fs, ".")
|
||||
}
|
||||
|
||||
// resolveBasePath returns the directory a manifest's paths are resolved
|
||||
// against: the one --base names, or else the directory holding the manifest,
|
||||
// or the current directory for a manifest URL.
|
||||
func resolveBasePath(cmd *cli.Command, manifestPath string) string {
|
||||
switch {
|
||||
case cmd.IsSet(flagBase):
|
||||
return cmd.String(flagBase)
|
||||
case isHTTPURL(manifestPath):
|
||||
return "."
|
||||
default:
|
||||
return filepath.Dir(manifestPath)
|
||||
}
|
||||
}
|
||||
|
||||
+33
-38
@@ -24,7 +24,6 @@ const (
|
||||
cmdList = "list"
|
||||
cmdVersion = "version"
|
||||
|
||||
flagBase = "base"
|
||||
flagProgress = "progress"
|
||||
flagTimeout = "timeout"
|
||||
flagDest = "dest"
|
||||
@@ -48,10 +47,6 @@ func manifestTempPath(out string) string {
|
||||
// errUnknownCommand indicates an unrecognized command argument.
|
||||
var errUnknownCommand = errors.New("unknown command")
|
||||
|
||||
// errUnexpectedArgument indicates a second argument to a command that takes
|
||||
// one.
|
||||
var errUnexpectedArgument = errors.New("unexpected argument")
|
||||
|
||||
// CLIApp is the main CLI application container. It holds configuration,
|
||||
// I/O streams, and filesystem abstraction to enable testing and flexibility.
|
||||
//
|
||||
@@ -158,14 +153,13 @@ func commonFlags() []cli.Flag {
|
||||
}
|
||||
}
|
||||
|
||||
// refuseExtraArgs is the ArgValidator of the commands that take one
|
||||
// argument: it refuses a second one, naming it.
|
||||
func refuseExtraArgs(_ context.Context, cmd *cli.Command) error {
|
||||
if cmd.Args().Len() > 1 {
|
||||
return fmt.Errorf("%w %q", errUnexpectedArgument, cmd.Args().Get(1))
|
||||
}
|
||||
// stopOnFirstArg returns the StopOnNthArg setting every command uses: flags
|
||||
// are read only before the command's first argument, and everything after it
|
||||
// is an argument, as with urfave/cli v2. So `mfer gen d -v` names a path "-v".
|
||||
func stopOnFirstArg() *int {
|
||||
n := 1
|
||||
|
||||
return nil
|
||||
return &n
|
||||
}
|
||||
|
||||
// requireSignatureFlag returns the --require-signature flag taken by the
|
||||
@@ -174,7 +168,7 @@ func requireSignatureFlag() *cli.StringFlag {
|
||||
return &cli.StringFlag{
|
||||
Name: flagRequireSignature,
|
||||
Aliases: []string{"S"},
|
||||
Usage: "Require manifest to be signed by the OpenPGP key with this fingerprint",
|
||||
Usage: "Require manifest to be signed by the specified GPG key ID",
|
||||
Sources: cli.EnvVars("MFER_REQUIRE_SIGNATURE"),
|
||||
}
|
||||
}
|
||||
@@ -191,10 +185,11 @@ func includePermissionsFlag() *cli.BoolFlag {
|
||||
|
||||
func (mfa *CLIApp) generateCommand() *cli.Command {
|
||||
return &cli.Command{
|
||||
Name: cmdGenerate,
|
||||
Aliases: []string{"gen"},
|
||||
Usage: "Generate manifest file",
|
||||
ArgsUsage: "[path ...]",
|
||||
Name: cmdGenerate,
|
||||
Aliases: []string{"gen"},
|
||||
Usage: "Generate manifest file",
|
||||
ArgsUsage: "[path ...]",
|
||||
StopOnNthArg: stopOnFirstArg(),
|
||||
Action: func(ctx context.Context, cmd *cli.Command) error {
|
||||
mfa.setVerbosity(cmd)
|
||||
mfa.printBanner()
|
||||
@@ -215,10 +210,9 @@ func (mfa *CLIApp) generateCommand() *cli.Command {
|
||||
},
|
||||
&cli.StringFlag{
|
||||
Name: "output",
|
||||
Value: defaultManifestName,
|
||||
Aliases: []string{"o"},
|
||||
Usage: "File to write the manifest to (default: index.mf in " +
|
||||
"the directory given, or beside the file given; with no " +
|
||||
"path or several, index.mf in the current directory)",
|
||||
Usage: "Specify output filename",
|
||||
},
|
||||
&cli.BoolFlag{
|
||||
Name: "force",
|
||||
@@ -233,7 +227,7 @@ func (mfa *CLIApp) generateCommand() *cli.Command {
|
||||
&cli.StringFlag{
|
||||
Name: "sign-key",
|
||||
Aliases: []string{"s"},
|
||||
Usage: "OpenPGP secret key file to sign the manifest with",
|
||||
Usage: "GPG key ID to sign the manifest with",
|
||||
Sources: cli.EnvVars("MFER_SIGN_KEY"),
|
||||
},
|
||||
&cli.StringFlag{
|
||||
@@ -256,7 +250,7 @@ func (mfa *CLIApp) checkCommand() *cli.Command {
|
||||
Name: cmdCheck,
|
||||
Usage: "Validate files using manifest file",
|
||||
ArgsUsage: manifestArgsUsage,
|
||||
ArgValidator: refuseExtraArgs,
|
||||
StopOnNthArg: stopOnFirstArg(),
|
||||
Action: func(ctx context.Context, cmd *cli.Command) error {
|
||||
mfa.setVerbosity(cmd)
|
||||
mfa.printBanner()
|
||||
@@ -265,11 +259,10 @@ func (mfa *CLIApp) checkCommand() *cli.Command {
|
||||
},
|
||||
Flags: append(commonFlags(),
|
||||
&cli.StringFlag{
|
||||
Name: flagBase,
|
||||
Name: "base",
|
||||
Aliases: []string{"b"},
|
||||
Usage: "Base directory for resolving relative paths from manifest " +
|
||||
"(by default the directory holding the manifest, or the " +
|
||||
"current directory for a manifest URL)",
|
||||
Value: ".",
|
||||
Usage: "Base directory for resolving relative paths from manifest",
|
||||
},
|
||||
&cli.BoolFlag{
|
||||
Name: flagProgress,
|
||||
@@ -290,7 +283,7 @@ func (mfa *CLIApp) freshenCommand() *cli.Command {
|
||||
Name: cmdFreshen,
|
||||
Usage: "Update manifest with changed, new, and removed files",
|
||||
ArgsUsage: manifestArgsUsage,
|
||||
ArgValidator: refuseExtraArgs,
|
||||
StopOnNthArg: stopOnFirstArg(),
|
||||
Action: func(ctx context.Context, cmd *cli.Command) error {
|
||||
mfa.setVerbosity(cmd)
|
||||
mfa.printBanner()
|
||||
@@ -299,10 +292,10 @@ func (mfa *CLIApp) freshenCommand() *cli.Command {
|
||||
},
|
||||
Flags: append(commonFlags(),
|
||||
&cli.StringFlag{
|
||||
Name: flagBase,
|
||||
Name: "base",
|
||||
Aliases: []string{"b"},
|
||||
Usage: "Base directory for resolving relative paths " +
|
||||
"(by default the directory holding the manifest)",
|
||||
Value: ".",
|
||||
Usage: "Base directory for resolving relative paths",
|
||||
},
|
||||
&cli.BoolFlag{
|
||||
Name: "follow-symlinks",
|
||||
@@ -323,7 +316,7 @@ func (mfa *CLIApp) freshenCommand() *cli.Command {
|
||||
&cli.StringFlag{
|
||||
Name: "sign-key",
|
||||
Aliases: []string{"s"},
|
||||
Usage: "OpenPGP secret key file to sign the manifest with",
|
||||
Usage: "GPG key ID to sign the manifest with",
|
||||
Sources: cli.EnvVars("MFER_SIGN_KEY"),
|
||||
},
|
||||
&cli.BoolFlag{
|
||||
@@ -341,7 +334,7 @@ func (mfa *CLIApp) exportCommand() *cli.Command {
|
||||
Name: cmdExport,
|
||||
Usage: "Export manifest contents as JSON",
|
||||
ArgsUsage: "[manifest file or URL]",
|
||||
ArgValidator: refuseExtraArgs,
|
||||
StopOnNthArg: stopOnFirstArg(),
|
||||
Action: func(ctx context.Context, cmd *cli.Command) error {
|
||||
mfa.setVerbosity(cmd)
|
||||
|
||||
@@ -352,8 +345,9 @@ func (mfa *CLIApp) exportCommand() *cli.Command {
|
||||
|
||||
func (mfa *CLIApp) versionCommand() *cli.Command {
|
||||
return &cli.Command{
|
||||
Name: cmdVersion,
|
||||
Usage: "Show version",
|
||||
Name: cmdVersion,
|
||||
Usage: "Show version",
|
||||
StopOnNthArg: stopOnFirstArg(),
|
||||
Action: func(context.Context, *cli.Command) error {
|
||||
mfa.printVersion()
|
||||
|
||||
@@ -368,7 +362,7 @@ func (mfa *CLIApp) listCommand() *cli.Command {
|
||||
Aliases: []string{"ls"},
|
||||
Usage: "List files in manifest",
|
||||
ArgsUsage: manifestArgsUsage,
|
||||
ArgValidator: refuseExtraArgs,
|
||||
StopOnNthArg: stopOnFirstArg(),
|
||||
Action: mfa.listManifestOperation,
|
||||
Flags: []cli.Flag{
|
||||
&cli.BoolFlag{
|
||||
@@ -389,7 +383,7 @@ func (mfa *CLIApp) fetchCommand() *cli.Command {
|
||||
Name: cmdFetch,
|
||||
Usage: "fetch manifest and referenced files",
|
||||
ArgsUsage: "URL",
|
||||
ArgValidator: refuseExtraArgs,
|
||||
StopOnNthArg: stopOnFirstArg(),
|
||||
Action: func(ctx context.Context, cmd *cli.Command) error {
|
||||
mfa.setVerbosity(cmd)
|
||||
mfa.printBanner()
|
||||
@@ -448,8 +442,9 @@ func (mfa *CLIApp) run(args []string) {
|
||||
Writer: mfa.Stdout,
|
||||
// v3 writes its "Incorrect Usage" line to ErrWriter; v2 wrote it to
|
||||
// stdout, before the help, so it stays on stdout.
|
||||
ErrWriter: mfa.Stdout,
|
||||
Flags: commonFlags(),
|
||||
ErrWriter: mfa.Stdout,
|
||||
Flags: commonFlags(),
|
||||
StopOnNthArg: stopOnFirstArg(),
|
||||
Action: func(_ context.Context, cmd *cli.Command) error {
|
||||
if cmd.Args().Len() > 0 {
|
||||
return fmt.Errorf("%w %q", errUnknownCommand, cmd.Args().First())
|
||||
|
||||
@@ -1,94 +0,0 @@
|
||||
//go:build unix
|
||||
|
||||
//nolint:testpackage // white-box tests exercise unexported internals
|
||||
package cli
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"syscall"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/spf13/afero"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
const (
|
||||
// testPipe is the name of the named pipe the pipe tests make in a tree.
|
||||
testPipe = "pipe"
|
||||
|
||||
// pipeDeadline is how long a test lets a run go before it opens the
|
||||
// test's named pipes for writing.
|
||||
pipeDeadline = 5 * time.Second
|
||||
)
|
||||
|
||||
// makePipe makes a named pipe at path. Opening a named pipe for reading
|
||||
// blocks until it is opened for writing, so once pipeDeadline has passed,
|
||||
// unless the test has ended, makePipe opens it for writing and closes it
|
||||
// again. A run that wrongly reads the pipe then ends, and the test fails on
|
||||
// what it reports instead of hanging. That lets one read through, so each
|
||||
// path a wrong run would read gets a pipe of its own.
|
||||
func makePipe(t *testing.T, path string) {
|
||||
t.Helper()
|
||||
|
||||
require.NoError(t, syscall.Mkfifo(path, 0o600))
|
||||
|
||||
timer := time.AfterFunc(pipeDeadline, func() {
|
||||
f, err := os.OpenFile(path, os.O_WRONLY, 0) //nolint:gosec // test-controlled path
|
||||
if err == nil {
|
||||
_ = f.Close()
|
||||
}
|
||||
})
|
||||
|
||||
t.Cleanup(func() { timer.Stop() })
|
||||
}
|
||||
|
||||
// TestGenerateSkipsPipe runs gen -v on a tree holding a named pipe: the
|
||||
// manifest lists the regular file only, and gen names the pipe it skipped.
|
||||
func TestGenerateSkipsPipe(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dir := t.TempDir()
|
||||
fs := afero.NewOsFs()
|
||||
writeTestFile(t, fs, filepath.Join(dir, testFileTxt), "regular")
|
||||
makePipe(t, filepath.Join(dir, testPipe))
|
||||
|
||||
opts := testOpts([]string{testApp, cmdGenerate, "-v", dir}, fs)
|
||||
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
|
||||
|
||||
assert.Equal(t, []string{testFileTxt},
|
||||
manifestPaths(t, fs, filepath.Join(dir, defaultManifestName)))
|
||||
assert.Contains(t, testStderr(t, opts), "skipping pipe: not a regular file")
|
||||
}
|
||||
|
||||
// TestFreshenSkipsPipes adds a named pipe to a tree after gen made its
|
||||
// manifest, and a symlink to another pipe. freshen -v --follow-symlinks
|
||||
// leaves the manifest listing the regular file only, and names both entries
|
||||
// it skipped.
|
||||
func TestFreshenSkipsPipes(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
fs := afero.NewOsFs()
|
||||
root, manifestPath := setupFreshenDir(t, fs,
|
||||
map[string]string{testFileTxt: "regular"})
|
||||
makePipe(t, filepath.Join(root, testPipe))
|
||||
|
||||
// The symlink points outside the tree, so no pipe is read twice.
|
||||
target := filepath.Join(t.TempDir(), testPipe)
|
||||
makePipe(t, target)
|
||||
require.NoError(t, os.Symlink(target, filepath.Join(root, testLink)))
|
||||
|
||||
opts := testOpts([]string{
|
||||
testApp, cmdFreshen, "-v", "--follow-symlinks",
|
||||
testFlagBase, root, manifestPath,
|
||||
}, fs)
|
||||
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
|
||||
|
||||
assert.Equal(t, []string{testFileTxt}, manifestPaths(t, fs, manifestPath))
|
||||
|
||||
stderr := testStderr(t, opts)
|
||||
assert.Contains(t, stderr, "skipping pipe: not a regular file")
|
||||
assert.Contains(t, stderr, "skipping link: not a regular file")
|
||||
}
|
||||
@@ -1,86 +0,0 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
|
||||
"github.com/spf13/afero"
|
||||
"golang.org/x/term"
|
||||
"sneak.berlin/go/mfer/internal/log"
|
||||
"sneak.berlin/go/mfer/mfer"
|
||||
)
|
||||
|
||||
// envSignKeyPassphrase names the environment variable holding the
|
||||
// passphrase of a protected signing key.
|
||||
//
|
||||
//nolint:gosec // G101: the name of a variable, not a credential
|
||||
const envSignKeyPassphrase = "MFER_SIGN_KEY_PASSPHRASE"
|
||||
|
||||
// errNoPassphrase indicates a protected signing key whose passphrase is
|
||||
// neither in the environment nor can be asked for on a terminal.
|
||||
var errNoPassphrase = errors.New(
|
||||
"signing key is protected: set " + envSignKeyPassphrase + " to its passphrase")
|
||||
|
||||
// signingOptions returns the signing options for the OpenPGP secret key in
|
||||
// the file path, which must be able to sign. The passphrase of a protected
|
||||
// key comes from MFER_SIGN_KEY_PASSPHRASE, or else from the terminal on
|
||||
// stdin, and must unlock the key.
|
||||
func (mfa *CLIApp) signingOptions(path string) (*mfer.SigningOptions, error) {
|
||||
secretKey, err := afero.ReadFile(mfa.Fs, path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read signing key: %w", err)
|
||||
}
|
||||
|
||||
protected, err := mfer.SecretKeyIsProtected(secretKey)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", path, err)
|
||||
}
|
||||
|
||||
log.Infof("signing manifest with the OpenPGP key in %s", path)
|
||||
|
||||
opts := &mfer.SigningOptions{SecretKey: secretKey}
|
||||
if protected {
|
||||
opts.Passphrase, err = mfa.readPassphrase(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
// gen and freshen read the signing options before any file, so a key
|
||||
// that cannot sign, or a wrong passphrase, stops them before they hash
|
||||
// anything.
|
||||
err = mfer.CheckSigningKey(opts)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", path, err)
|
||||
}
|
||||
|
||||
return opts, nil
|
||||
}
|
||||
|
||||
// readPassphrase returns MFER_SIGN_KEY_PASSPHRASE when it is set, or else
|
||||
// asks for the passphrase of the key in the file path on the terminal on
|
||||
// stdin.
|
||||
func (mfa *CLIApp) readPassphrase(path string) ([]byte, error) {
|
||||
passphrase := os.Getenv(envSignKeyPassphrase)
|
||||
if passphrase != "" {
|
||||
return []byte(passphrase), nil
|
||||
}
|
||||
|
||||
stdin, ok := mfa.Stdin.(*os.File)
|
||||
if !ok || !term.IsTerminal(int(stdin.Fd())) {
|
||||
return nil, errNoPassphrase
|
||||
}
|
||||
|
||||
_, _ = fmt.Fprintf(mfa.Stderr, "Passphrase for %s: ", path)
|
||||
|
||||
typed, err := term.ReadPassword(int(stdin.Fd()))
|
||||
|
||||
_, _ = fmt.Fprintln(mfa.Stderr)
|
||||
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read passphrase: %w", err)
|
||||
}
|
||||
|
||||
return typed, nil
|
||||
}
|
||||
@@ -1,209 +0,0 @@
|
||||
//nolint:testpackage // white-box tests exercise unexported internals
|
||||
package cli
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"io"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/ProtonMail/go-crypto/openpgp/packet"
|
||||
"github.com/creack/pty"
|
||||
"github.com/spf13/afero"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
const (
|
||||
testFlagSignKey = "--sign-key"
|
||||
testKeyFile = "/key.asc"
|
||||
)
|
||||
|
||||
// TestGenAndFreshenSignWithKeyFile runs gen, then freshen after a file is
|
||||
// added, with --sign-key naming a key file: one key with no passphrase and
|
||||
// one protected by the passphrase in MFER_SIGN_KEY_PASSPHRASE. check
|
||||
// --require-signature must accept each manifest as signed by that key.
|
||||
// freshen leaves its manifest out of the listing only on the real
|
||||
// filesystem, so the test uses that.
|
||||
func TestGenAndFreshenSignWithKeyFile(t *testing.T) {
|
||||
for name, passphrase := range map[string][]byte{
|
||||
"unprotected": nil,
|
||||
"protected": []byte("passphrase"),
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
t.Setenv(envSignKeyPassphrase, string(passphrase))
|
||||
|
||||
secretKey, fingerprint := testSecretKey(t, passphrase, nil)
|
||||
|
||||
fs := afero.NewOsFs()
|
||||
keyFile := filepath.Join(t.TempDir(), "key.asc")
|
||||
root := t.TempDir()
|
||||
manifestPath := filepath.Join(root, defaultManifestName)
|
||||
|
||||
require.NoError(t, afero.WriteFile(fs, keyFile, secretKey, 0o600))
|
||||
writeTestFile(t, fs, filepath.Join(root, testFileTxt), "hello")
|
||||
|
||||
opts := testOpts([]string{
|
||||
testApp, cmdGenerate, "-q", testFlagSignKey, keyFile,
|
||||
"-o", manifestPath, root,
|
||||
}, fs)
|
||||
require.Equal(t, 0, runCLI(opts), testStderr(t, opts))
|
||||
|
||||
check := []string{
|
||||
testApp, cmdCheck, "-q",
|
||||
"--" + flagRequireSignature, fingerprint, manifestPath,
|
||||
}
|
||||
|
||||
opts = testOpts(check, fs)
|
||||
require.Equal(t, 0, runCLI(opts), testStderr(t, opts))
|
||||
|
||||
writeTestFile(t, fs, filepath.Join(root, "added.txt"), "added")
|
||||
|
||||
opts = testOpts([]string{
|
||||
testApp, cmdFreshen, "-q", testFlagSignKey, keyFile, manifestPath,
|
||||
}, fs)
|
||||
require.Equal(t, 0, runCLI(opts), testStderr(t, opts))
|
||||
|
||||
opts = testOpts(check, fs)
|
||||
require.Equal(t, 0, runCLI(opts), testStderr(t, opts))
|
||||
assert.Len(t, manifestFiles(t, fs, manifestPath), 2)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestSignWithProtectedKeyNeedsPassphrase runs gen with a protected key,
|
||||
// with MFER_SIGN_KEY_PASSPHRASE empty and no terminal to ask on. gen must
|
||||
// fail, naming the variable, and write no manifest.
|
||||
func TestSignWithProtectedKeyNeedsPassphrase(t *testing.T) {
|
||||
t.Setenv(envSignKeyPassphrase, "")
|
||||
|
||||
secretKey, _ := testSecretKey(t, []byte("secret"), nil)
|
||||
|
||||
fs := afero.NewMemMapFs()
|
||||
require.NoError(t, afero.WriteFile(fs, testKeyFile, secretKey, 0o600))
|
||||
require.NoError(t, fs.MkdirAll(testDir, 0o755))
|
||||
writeTestFile(t, fs, testFile1, "hello")
|
||||
|
||||
opts := testOpts([]string{
|
||||
testApp, cmdGenerate, "-q", testFlagSignKey, testKeyFile,
|
||||
"-o", testMF, testDir,
|
||||
}, fs)
|
||||
assert.Equal(t, 1, runCLI(opts))
|
||||
assert.Contains(t, testStderr(t, opts),
|
||||
"signing key is protected: set MFER_SIGN_KEY_PASSPHRASE to its passphrase")
|
||||
|
||||
exists, err := afero.Exists(fs, testMF)
|
||||
require.NoError(t, err)
|
||||
assert.False(t, exists)
|
||||
}
|
||||
|
||||
// TestSignWithKeyThatCannotSignFailsFirst runs gen on a directory and
|
||||
// freshen on a manifest, neither of which exists, with keys that cannot
|
||||
// sign: a protected key with a wrong MFER_SIGN_KEY_PASSPHRASE, a key that
|
||||
// expired in 2020, and a version 6 key. Each run must fail on the key: it
|
||||
// checks the key before it reads any file, so a missing file goes
|
||||
// unnoticed.
|
||||
func TestSignWithKeyThatCannotSignFailsFirst(t *testing.T) {
|
||||
t.Setenv(envSignKeyPassphrase, "wrong")
|
||||
|
||||
wrongPassphrase, _ := testSecretKey(t, []byte("right"), nil)
|
||||
|
||||
made := time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)
|
||||
expired, _ := testSecretKey(t, nil, &packet.Config{
|
||||
Algorithm: packet.PubKeyAlgoEdDSA,
|
||||
Time: func() time.Time { return made },
|
||||
KeyLifetimeSecs: uint32((24 * time.Hour).Seconds()),
|
||||
})
|
||||
|
||||
version6, _ := testSecretKey(t, nil, &packet.Config{
|
||||
Algorithm: packet.PubKeyAlgoEd25519,
|
||||
V6Keys: true,
|
||||
})
|
||||
|
||||
for want, secretKey := range map[string][]byte{
|
||||
"unlock signing key": wrongPassphrase,
|
||||
"signing key cannot sign": expired,
|
||||
"signing key must be an OpenPGP version 4 key": version6,
|
||||
} {
|
||||
fs := afero.NewMemMapFs()
|
||||
require.NoError(t, afero.WriteFile(fs, testKeyFile, secretKey, 0o600))
|
||||
|
||||
for _, args := range [][]string{
|
||||
{
|
||||
testApp, cmdGenerate, "-q", testFlagSignKey, testKeyFile,
|
||||
"-o", testMF, "/missing",
|
||||
},
|
||||
{testApp, cmdFreshen, "-q", testFlagSignKey, testKeyFile, "/missing.mf"},
|
||||
} {
|
||||
opts := testOpts(args, fs)
|
||||
assert.Equal(t, 1, runCLI(opts), args[1], want)
|
||||
assert.Contains(t, testStderr(t, opts), testKeyFile+": "+want, args[1])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestGenAsksForPassphraseOnTerminal runs gen with a protected key, no
|
||||
// MFER_SIGN_KEY_PASSPHRASE, and a terminal as stdin and stderr. gen must
|
||||
// ask for the passphrase on stderr, and sign with what is typed after the
|
||||
// prompt.
|
||||
func TestGenAsksForPassphraseOnTerminal(t *testing.T) {
|
||||
t.Setenv(envSignKeyPassphrase, "")
|
||||
|
||||
secretKey, fingerprint := testSecretKey(t, []byte("passphrase"), nil)
|
||||
|
||||
fs := afero.NewOsFs()
|
||||
keyFile := filepath.Join(t.TempDir(), "key.asc")
|
||||
root := t.TempDir()
|
||||
manifestPath := filepath.Join(root, defaultManifestName)
|
||||
|
||||
require.NoError(t, afero.WriteFile(fs, keyFile, secretKey, 0o600))
|
||||
writeTestFile(t, fs, filepath.Join(root, testFileTxt), "hello")
|
||||
|
||||
terminal, tty, err := pty.Open()
|
||||
require.NoError(t, err)
|
||||
|
||||
t.Cleanup(func() { _ = terminal.Close() })
|
||||
|
||||
opts := testOpts([]string{
|
||||
testApp, cmdGenerate, "-q", testFlagSignKey, keyFile,
|
||||
"-o", manifestPath, root,
|
||||
}, fs)
|
||||
opts.Stdin = tty
|
||||
opts.Stderr = tty
|
||||
|
||||
exitCode := make(chan int, 1)
|
||||
|
||||
go func() {
|
||||
exitCode <- runCLI(opts)
|
||||
|
||||
// Once gen has ended, reading the terminal fails instead of
|
||||
// waiting for a prompt that will not come.
|
||||
_ = tty.Close()
|
||||
}()
|
||||
|
||||
prompt := "Passphrase for " + keyFile + ": "
|
||||
output := bufio.NewReader(terminal)
|
||||
written := ""
|
||||
|
||||
for !strings.HasSuffix(written, prompt) {
|
||||
b, err := output.ReadByte()
|
||||
require.NoError(t, err, "gen wrote %q and no prompt", written)
|
||||
|
||||
written += string(b)
|
||||
}
|
||||
|
||||
_, err = terminal.WriteString("passphrase\n")
|
||||
require.NoError(t, err)
|
||||
|
||||
code := <-exitCode
|
||||
rest, _ := io.ReadAll(output)
|
||||
require.Equal(t, 0, code, "gen wrote %q", rest)
|
||||
|
||||
check := testOpts([]string{
|
||||
testApp, cmdCheck, "-q",
|
||||
"--" + flagRequireSignature, fingerprint, manifestPath,
|
||||
}, fs)
|
||||
require.Equal(t, 0, runCLI(check), testStderr(t, check))
|
||||
}
|
||||
+3
-3
@@ -290,7 +290,7 @@ func (b *Builder) SetIncludeTimestamps(include bool) {
|
||||
b.includeTimestamps = include
|
||||
}
|
||||
|
||||
// SetSigningOptions sets the key the manifest is signed with.
|
||||
// SetSigningOptions sets the GPG signing options for the manifest.
|
||||
// If opts is non-nil, the manifest will be signed when Build() is called.
|
||||
func (b *Builder) SetSigningOptions(opts *SigningOptions) {
|
||||
b.mu.Lock()
|
||||
@@ -299,8 +299,8 @@ func (b *Builder) SetSigningOptions(opts *SigningOptions) {
|
||||
b.signingOptions = opts
|
||||
}
|
||||
|
||||
// Build finalizes the manifest and writes it to the writer. When signing
|
||||
// options are set, it does not sign once ctx has ended.
|
||||
// Build finalizes the manifest and writes it to the writer. ctx bounds the
|
||||
// gpg runs that sign the manifest when signing options are set.
|
||||
func (b *Builder) Build(ctx context.Context, w io.Writer) error {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
|
||||
@@ -318,11 +318,6 @@ func (c *Checker) FindExtraFiles(ctx context.Context, results chan<- Result) err
|
||||
return nil
|
||||
}
|
||||
|
||||
// gen never lists a named pipe, a socket or a device.
|
||||
if info.Mode()&(os.ModeNamedPipe|os.ModeSocket|os.ModeDevice) != 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
// A symlink is compared by what it points to, so a manifest reached
|
||||
// through one is not reported either.
|
||||
if info.Mode()&os.ModeSymlink != 0 {
|
||||
@@ -376,15 +371,6 @@ func (c *Checker) checkFile(entry *MFFilePath, checkedBytes *FileSize) Result {
|
||||
return Result{Path: relPath, Status: StatusError, Message: err.Error()}
|
||||
}
|
||||
|
||||
// Opening a named pipe can block, and reading a device may never end.
|
||||
if !info.Mode().IsRegular() {
|
||||
return Result{
|
||||
Path: relPath,
|
||||
Status: StatusError,
|
||||
Message: "not a regular file",
|
||||
}
|
||||
}
|
||||
|
||||
// Check size
|
||||
if info.Size() != entry.GetSize() {
|
||||
*checkedBytes += FileSize(info.Size())
|
||||
|
||||
+5
-8
@@ -2,6 +2,7 @@ package mfer
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"errors"
|
||||
"fmt"
|
||||
@@ -27,7 +28,6 @@ var (
|
||||
errUUIDMismatch = errors.New("outer and inner UUID mismatch")
|
||||
errInvalidFileFormat = errors.New("invalid file format")
|
||||
errInvalidManifestPath = errors.New("invalid file entry")
|
||||
errEntryMissingHash = errors.New("file entry has no hash")
|
||||
errDecodedTooLarge = errors.New("too much memory needed")
|
||||
errSignerNotSigningKey = errors.New(
|
||||
"signer is not the fingerprint of the key that made the signature")
|
||||
@@ -93,7 +93,9 @@ func (m *manifest) verifyOuterIntegrity() error {
|
||||
return fmt.Errorf("build signature string: %w", err)
|
||||
}
|
||||
|
||||
signingKey, err := verifySignature(
|
||||
// Loading a manifest takes no context; gpgTimeout still bounds gpg.
|
||||
signingKey, err := gpgVerify(
|
||||
context.Background(),
|
||||
[]byte(sigString),
|
||||
m.pbOuter.GetSignature(),
|
||||
m.pbOuter.GetSigningPubKey(),
|
||||
@@ -294,8 +296,7 @@ func (m *manifest) deserializeInner() error {
|
||||
// untrusted .mf. Reject loudly on the first offender rather than
|
||||
// dropping entries, which would let a hostile manifest hide files from a
|
||||
// check. A path listed twice is refused too: check would check the one
|
||||
// file against both entries. So is an entry with no hash: there is
|
||||
// nothing to check the file against.
|
||||
// file against both entries.
|
||||
seen := make(map[string]bool, len(m.pbInner.GetFiles()))
|
||||
|
||||
for _, f := range m.pbInner.GetFiles() {
|
||||
@@ -304,10 +305,6 @@ func (m *manifest) deserializeInner() error {
|
||||
return fmt.Errorf("%w: %w", errInvalidManifestPath, err)
|
||||
}
|
||||
|
||||
if len(f.GetHashes()) == 0 {
|
||||
return fmt.Errorf("%w: %q", errEntryMissingHash, f.GetPath())
|
||||
}
|
||||
|
||||
if seen[f.GetPath()] {
|
||||
return fmt.Errorf("%w %q", errDuplicatePath, f.GetPath())
|
||||
}
|
||||
|
||||
@@ -19,6 +19,14 @@ import (
|
||||
// input and of the decompressed data it may read, plus room for the
|
||||
// decoder's window buffers. A panic or a hang fails the test on its own.
|
||||
func FuzzNewManifestFromReader(f *testing.F) {
|
||||
// A signed manifest makes the parser write the key and signature to a
|
||||
// temporary directory and run gpg on them. With gpg off the PATH and
|
||||
// temporary files kept in the test's own directory, no process is
|
||||
// started and nothing is written elsewhere; such input ends in an
|
||||
// error instead.
|
||||
f.Setenv("PATH", "")
|
||||
f.Setenv("TMPDIR", f.TempDir())
|
||||
|
||||
f.Fuzz(func(t *testing.T, data []byte) {
|
||||
var before, after runtime.MemStats
|
||||
|
||||
|
||||
@@ -165,24 +165,6 @@ func TestDeserializeRefusesPathListedTwice(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// A manifest holding a file entry with no hash is refused as it is loaded,
|
||||
// naming the path: there is nothing to check the file against.
|
||||
func TestDeserializeRefusesEntryWithNoHash(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
id := uuid.NewV4()
|
||||
inner, err := proto.Marshal(&MFFile{
|
||||
Version: MFFile_VERSION_ONE,
|
||||
Files: []*MFFilePath{{Path: "x", Size: 1}},
|
||||
Uuid: id[:],
|
||||
})
|
||||
require.NoError(t, err)
|
||||
|
||||
_, err = NewManifestFromReader(bytes.NewReader(wrapInner(t, id, inner)))
|
||||
require.ErrorIs(t, err, errEntryMissingHash)
|
||||
require.EqualError(t, err, `file entry has no hash: "x"`)
|
||||
}
|
||||
|
||||
// Entries of a path, an empty hash, an empty MIME type and empty modification
|
||||
// and change times are counted at 432 bytes each (176 + 112 + 16 + 64 + 64)
|
||||
// and take 16 bytes plus the path to encode. A 37-character path makes that
|
||||
@@ -247,8 +229,6 @@ func TestDeserializeDropsUnknownFields(t *testing.T) {
|
||||
|
||||
entry := protowire.AppendTag(nil, 1, protowire.BytesType) // MFFilePath.path
|
||||
entry = protowire.AppendString(entry, "a")
|
||||
entry = protowire.AppendTag(entry, 3, protowire.BytesType) // MFFilePath.hashes
|
||||
entry = protowire.AppendBytes(entry, nil)
|
||||
entry = append(entry, unknown...)
|
||||
|
||||
id := uuid.NewV4()
|
||||
|
||||
@@ -3,9 +3,6 @@ package mfer
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"os"
|
||||
"testing"
|
||||
"uuid"
|
||||
|
||||
@@ -55,38 +52,3 @@ func TestReadAtMost(t *testing.T) {
|
||||
"file exceeds maximum allowed size of 65536 bytes")
|
||||
assert.Equal(t, maxSize-1, input.Len(), "bytes left unread")
|
||||
}
|
||||
|
||||
// TestFormatTestVector checks the test vector that docs/FORMAT.md shows: the
|
||||
// SHA-256 of the whole file, so that any change to the file fails, then loads
|
||||
// it, verifies its signature, and checks the canonical string and the signer
|
||||
// the document gives for it.
|
||||
func TestFormatTestVector(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const (
|
||||
path = "testdata/test-vector.mf"
|
||||
fileSHA256 = "5864187d671b2a55f8e015f858248f92c51956b549cf548b6bb54477c2407fd4"
|
||||
signer = "5E962A37590D855B467D20294947A60319611E6F"
|
||||
)
|
||||
|
||||
data, err := os.ReadFile(path)
|
||||
require.NoError(t, err)
|
||||
|
||||
sum := sha256.Sum256(data)
|
||||
require.Equal(t, fileSHA256, hex.EncodeToString(sum[:]))
|
||||
|
||||
m, err := NewManifestFromFile(&ManifestFromFileOptions{Path: path})
|
||||
require.NoError(t, err)
|
||||
|
||||
canonicalString, err := m.signatureString()
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "ZNAVSRFG-92197fd532bc40908aa9f86e5518b56c-"+
|
||||
"12202353553205f036a88cefee3d0d9049ccd8e68bb24011a0b462b5867f1fcc6352",
|
||||
canonicalString)
|
||||
|
||||
keyFingerprint, err := verifySignature([]byte(canonicalString),
|
||||
m.pbOuter.GetSignature(), m.pbOuter.GetSigningPubKey())
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, signer, keyFingerprint)
|
||||
assert.Equal(t, signer, string(m.pbOuter.GetSigner()))
|
||||
}
|
||||
|
||||
+358
@@ -0,0 +1,358 @@
|
||||
package mfer
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
const (
|
||||
// gpgTimeout bounds every gpg run, which can otherwise wait forever on
|
||||
// a passphrase prompt or a stalled gpg-agent. A minute leaves a person
|
||||
// time to type a passphrase or touch a smartcard.
|
||||
gpgTimeout = time.Minute
|
||||
|
||||
// gpgWaitDelay is how long a gpg run keeps waiting for gpg's stdout
|
||||
// and stderr to close once gpg has been killed or has exited. Reading
|
||||
// what gpg itself wrote takes far less; only a process gpg left behind
|
||||
// holds them open longer.
|
||||
gpgWaitDelay = time.Second
|
||||
|
||||
// privateDirPerms is the permission mode for temporary GPG home
|
||||
// directories.
|
||||
privateDirPerms os.FileMode = 0o700
|
||||
|
||||
// privateFilePerms is the permission mode for temporary key,
|
||||
// signature, and data files.
|
||||
privateFilePerms os.FileMode = 0o600
|
||||
|
||||
// gpgFingerprintField is the record type tag for fingerprint lines
|
||||
// in gpg --with-colons output.
|
||||
gpgFingerprintField = "fpr"
|
||||
|
||||
// gpgFingerprintMinFields is the minimum number of colon-separated
|
||||
// fields in a gpg fingerprint record (the fingerprint is field 10).
|
||||
gpgFingerprintMinFields = 10
|
||||
|
||||
// gpgStatusPrefix starts each status line gpg writes to the file
|
||||
// descriptor named by --status-fd.
|
||||
gpgStatusPrefix = "[GNUPG:]"
|
||||
|
||||
// gpg option names used from more than one call site.
|
||||
gpgOptArmor = "--armor"
|
||||
gpgOptHomedir = "--homedir"
|
||||
gpgOptStatusFD = "--status-fd"
|
||||
gpgOptVerify = "--verify"
|
||||
)
|
||||
|
||||
var (
|
||||
errGPGKeyNotFound = errors.New("GPG key not found")
|
||||
errFingerprintNotFound = errors.New("fingerprint not found for key")
|
||||
errSigningKeyCount = errors.New(
|
||||
"embedded public key block must hold exactly one key")
|
||||
errNotOneGoodSignature = errors.New(
|
||||
"gpg did not report exactly one good signature")
|
||||
errSigningKeyNotReported = errors.New(
|
||||
"gpg did not report the key that made the signature")
|
||||
)
|
||||
|
||||
// GPGKeyID represents a GPG key identifier (fingerprint or key ID).
|
||||
type GPGKeyID string
|
||||
|
||||
// SigningOptions contains options for GPG signing.
|
||||
type SigningOptions struct {
|
||||
KeyID GPGKeyID
|
||||
}
|
||||
|
||||
// gpgArgs builds a gpg argument list from opts followed by positional
|
||||
// arguments, separated by an explicit "--" end-of-options marker.
|
||||
//
|
||||
// This matters because key IDs reach gpg as bare positional arguments
|
||||
// (from --sign-key / MFER_SIGN_KEY) and gpg would otherwise parse a value
|
||||
// beginning with "-" as one of its own options. Callers must route every
|
||||
// non-option argument through here.
|
||||
func gpgArgs(opts []string, positional ...string) []string {
|
||||
args := make([]string, 0, len(opts)+1+len(positional))
|
||||
args = append(args, opts...)
|
||||
args = append(args, "--")
|
||||
args = append(args, positional...)
|
||||
|
||||
return args
|
||||
}
|
||||
|
||||
// runGPG runs the gpg binary in batch mode with the given arguments and
|
||||
// optional stdin, returning captured stdout and stderr. If gpg fails, the
|
||||
// error ends with what gpg wrote to stderr. gpg is killed when ctx ends or
|
||||
// gpgTimeout passes, whichever comes first.
|
||||
func runGPG(
|
||||
ctx context.Context, stdin io.Reader, args ...string,
|
||||
) (*bytes.Buffer, *bytes.Buffer, error) {
|
||||
// exec.CommandContext kills only gpg itself. A gpg-agent that gpg
|
||||
// starts runs detached and holds none of gpg's output, but another
|
||||
// process gpg leaves behind (a wrapper script that runs the real gpg
|
||||
// without exec, for example) can keep gpg's stdout or stderr open, and
|
||||
// Run would wait for it to exit. WaitDelay stops that wait
|
||||
// gpgWaitDelay after the kill; that process is left running.
|
||||
ctx, cancel := context.WithTimeout(ctx, gpgTimeout)
|
||||
defer cancel()
|
||||
|
||||
fullArgs := append([]string{"--batch", "--no-tty"}, args...)
|
||||
|
||||
// G204: the executable name is a compile-time constant. The arguments
|
||||
// are not, so the guarantee that matters is placement: every
|
||||
// caller-supplied value is passed either as the value of a named
|
||||
// option or after the "--" end-of-options marker inserted by gpgArgs,
|
||||
// and therefore cannot be reinterpreted by gpg as an option.
|
||||
cmd := exec.CommandContext( //nolint:gosec // G204: see comment above
|
||||
ctx, "gpg", fullArgs...)
|
||||
cmd.WaitDelay = gpgWaitDelay
|
||||
cmd.Stdin = stdin
|
||||
|
||||
var stdout, stderr bytes.Buffer
|
||||
|
||||
cmd.Stdout = &stdout
|
||||
cmd.Stderr = &stderr
|
||||
|
||||
err := cmd.Run()
|
||||
if err != nil && ctx.Err() != nil {
|
||||
// gpg was killed because ctx ended, which Run reports only as
|
||||
// "signal: killed"; return the reason instead.
|
||||
err = ctx.Err()
|
||||
if errors.Is(err, context.DeadlineExceeded) {
|
||||
err = fmt.Errorf("timed out: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
err = withStderr(err, &stderr)
|
||||
}
|
||||
|
||||
return &stdout, &stderr, err
|
||||
}
|
||||
|
||||
// withStderr returns err followed by what gpg wrote to stderr, or err alone
|
||||
// when gpg wrote nothing.
|
||||
func withStderr(err error, stderr *bytes.Buffer) error {
|
||||
messages := strings.TrimSpace(stderr.String())
|
||||
if messages == "" {
|
||||
return err
|
||||
}
|
||||
|
||||
return fmt.Errorf("%w: %s", err, messages)
|
||||
}
|
||||
|
||||
// parseFingerprint extracts the first fingerprint from gpg --with-colons
|
||||
// output, or returns ok=false if none is present.
|
||||
func parseFingerprint(colonOutput string) (string, bool) {
|
||||
for line := range strings.SplitSeq(colonOutput, "\n") {
|
||||
fields := strings.Split(line, ":")
|
||||
if len(fields) >= gpgFingerprintMinFields &&
|
||||
fields[0] == gpgFingerprintField {
|
||||
return fields[9], true
|
||||
}
|
||||
}
|
||||
|
||||
return "", false
|
||||
}
|
||||
|
||||
// parseStatusLine returns the arguments of the status line for keyword in
|
||||
// gpg --status-fd output, or ok=false unless there is exactly one such line
|
||||
// and it has arguments.
|
||||
func parseStatusLine(statusOutput, keyword string) ([]string, bool) {
|
||||
var found [][]string
|
||||
|
||||
for line := range strings.SplitSeq(statusOutput, "\n") {
|
||||
fields := strings.Fields(line)
|
||||
if len(fields) > 2 && fields[0] == gpgStatusPrefix && fields[1] == keyword {
|
||||
found = append(found, fields[2:])
|
||||
}
|
||||
}
|
||||
|
||||
if len(found) != 1 {
|
||||
return nil, false
|
||||
}
|
||||
|
||||
return found[0], true
|
||||
}
|
||||
|
||||
// gpgSign creates an armored detached signature of data with the key gpg
|
||||
// picks for keyID, and returns it with the fingerprint of the key that made
|
||||
// it, which is a subkey's when gpg signed with a subkey.
|
||||
func gpgSign(
|
||||
ctx context.Context, data []byte, keyID GPGKeyID,
|
||||
) ([]byte, string, error) {
|
||||
tmpDir, err := os.MkdirTemp("", "mfer-gpg-sign-*")
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
|
||||
defer func() { _ = os.RemoveAll(tmpDir) }()
|
||||
|
||||
sigFile := filepath.Join(tmpDir, "signature.asc")
|
||||
|
||||
// The signature goes to sigFile, so --status-fd 1 can send gpg's status
|
||||
// lines to stdout; its messages go to stderr.
|
||||
stdout, stderr, err := runGPG(ctx, bytes.NewReader(data),
|
||||
"--detach-sign",
|
||||
gpgOptArmor,
|
||||
"--output", sigFile,
|
||||
gpgOptStatusFD, "1",
|
||||
"--local-user", string(keyID),
|
||||
)
|
||||
if err != nil {
|
||||
return nil, "", fmt.Errorf("gpg sign: %w", err)
|
||||
}
|
||||
|
||||
// The last argument of SIG_CREATED is the fingerprint of the key that
|
||||
// made the signature.
|
||||
created, ok := parseStatusLine(stdout.String(), "SIG_CREATED")
|
||||
if !ok {
|
||||
return nil, "", withStderr(errSigningKeyNotReported, stderr)
|
||||
}
|
||||
|
||||
sig, err := os.ReadFile(sigFile) //nolint:gosec // G304: inside tmpDir, made above
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
|
||||
return sig, created[len(created)-1], nil
|
||||
}
|
||||
|
||||
// gpgExportPublicKey exports the public key for the specified key ID.
|
||||
// Returns the armored public key.
|
||||
func gpgExportPublicKey(ctx context.Context, keyID GPGKeyID) ([]byte, error) {
|
||||
stdout, _, err := runGPG(ctx, nil,
|
||||
gpgArgs([]string{"--export", gpgOptArmor}, string(keyID))...,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("gpg export: %w", err)
|
||||
}
|
||||
|
||||
if stdout.Len() == 0 {
|
||||
return nil, fmt.Errorf("%w: %s", errGPGKeyNotFound, keyID)
|
||||
}
|
||||
|
||||
return stdout.Bytes(), nil
|
||||
}
|
||||
|
||||
// gpgGetKeyFingerprint gets the full fingerprint for a key ID.
|
||||
func gpgGetKeyFingerprint(ctx context.Context, keyID GPGKeyID) ([]byte, error) {
|
||||
stdout, _, err := runGPG(ctx, nil,
|
||||
gpgArgs([]string{"--with-colons", "--fingerprint"}, string(keyID))...,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("gpg fingerprint lookup: %w", err)
|
||||
}
|
||||
|
||||
fpr, ok := parseFingerprint(stdout.String())
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%w: %s", errFingerprintNotFound, keyID)
|
||||
}
|
||||
|
||||
return []byte(fpr), nil
|
||||
}
|
||||
|
||||
// gpgImportOneKey imports the public key block in pubKeyFile into the
|
||||
// keyring in gpgHome. The block must hold exactly one primary key.
|
||||
func gpgImportOneKey(ctx context.Context, gpgHome, pubKeyFile string) error {
|
||||
// --status-fd 1 sends gpg's status lines to stdout, which importing
|
||||
// otherwise leaves empty; its messages go to stderr.
|
||||
importStdout, _, err := runGPG(ctx, nil,
|
||||
gpgArgs([]string{gpgOptHomedir, gpgHome, gpgOptStatusFD, "1", "--import"},
|
||||
pubKeyFile)...,
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("gpg import: %w", err)
|
||||
}
|
||||
|
||||
// The first argument of IMPORT_RES counts the primary keys gpg read
|
||||
// from the block, those it then skipped (one with no user ID, for
|
||||
// example) included.
|
||||
result, ok := parseStatusLine(importStdout.String(), "IMPORT_RES")
|
||||
if !ok {
|
||||
return fmt.Errorf("%w, gpg reported no count", errSigningKeyCount)
|
||||
}
|
||||
|
||||
if result[0] != "1" {
|
||||
return fmt.Errorf("%w, found %s", errSigningKeyCount, result[0])
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// gpgVerify verifies a detached signature against data using the provided
|
||||
// public key, imported into a temporary keyring, and returns the
|
||||
// fingerprint of the primary key that made the signature. The public key
|
||||
// must hold exactly one primary key, so that a good signature can come
|
||||
// from no other key.
|
||||
func gpgVerify(ctx context.Context, data, signature, pubKey []byte) (string, error) {
|
||||
// Create temporary directory for GPG operations
|
||||
tmpDir, err := os.MkdirTemp("", "mfer-gpg-verify-*")
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
defer func() { _ = os.RemoveAll(tmpDir) }()
|
||||
|
||||
// Set restrictive permissions
|
||||
err = os.Chmod(tmpDir, privateDirPerms)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
// Write public key to temp file
|
||||
pubKeyFile := filepath.Join(tmpDir, "pubkey.asc")
|
||||
|
||||
err = os.WriteFile(pubKeyFile, pubKey, privateFilePerms)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
// Write signature to temp file
|
||||
sigFile := filepath.Join(tmpDir, "signature.asc")
|
||||
|
||||
err = os.WriteFile(sigFile, signature, privateFilePerms)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
// Write data to temp file
|
||||
dataFile := filepath.Join(tmpDir, "data")
|
||||
|
||||
err = os.WriteFile(dataFile, data, privateFilePerms)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
err = gpgImportOneKey(ctx, tmpDir, pubKeyFile)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
// --status-fd 1 sends gpg's status lines to stdout, which verifying a
|
||||
// detached signature otherwise leaves empty; its messages go to stderr.
|
||||
verifyStdout, _, err := runGPG(ctx, nil,
|
||||
gpgArgs([]string{gpgOptHomedir, tmpDir, gpgOptStatusFD, "1", gpgOptVerify},
|
||||
sigFile, dataFile)...,
|
||||
)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("gpg verify: %w", err)
|
||||
}
|
||||
|
||||
// gpg writes a VALIDSIG line for each good signature. Its first
|
||||
// argument is the fingerprint of the key that made the signature,
|
||||
// which may be a subkey; its last is that of the primary key.
|
||||
valid, ok := parseStatusLine(verifyStdout.String(), "VALIDSIG")
|
||||
if !ok {
|
||||
return "", errNotOneGoodSignature
|
||||
}
|
||||
|
||||
return valid[len(valid)-1], nil
|
||||
}
|
||||
@@ -0,0 +1,703 @@
|
||||
//nolint:testpackage // white-box tests exercise unexported internals
|
||||
package mfer
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"io"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"syscall"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/spf13/afero"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"google.golang.org/protobuf/proto"
|
||||
)
|
||||
|
||||
// testGPGEnv sets up a temporary GPG home directory with a test key.
|
||||
// Returns the key ID and the GPG home directory; callers must point
|
||||
// GNUPGHOME at the returned directory (via t.Setenv) before using the
|
||||
// gpg helpers under test.
|
||||
func testGPGEnv(t *testing.T) (GPGKeyID, string) {
|
||||
t.Helper()
|
||||
|
||||
// Check if gpg is installed
|
||||
_, err := exec.LookPath("gpg")
|
||||
if err != nil {
|
||||
t.Skip("gpg not installed, skipping signing test")
|
||||
}
|
||||
|
||||
// Create temporary GPG home directory (0700 by default)
|
||||
gpgHome := t.TempDir()
|
||||
|
||||
genTestKey(t, gpgHome, testKeyParams)
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), gpgTimeout)
|
||||
defer cancel()
|
||||
|
||||
// Get the key fingerprint
|
||||
cmd := exec.CommandContext(ctx, "gpg",
|
||||
"--list-keys", "--with-colons", "test@mfer.test")
|
||||
|
||||
cmd.Env = append(os.Environ(), "GNUPGHOME="+gpgHome)
|
||||
|
||||
output, err := cmd.Output()
|
||||
if err != nil {
|
||||
t.Fatalf("failed to list test key: %v", err)
|
||||
}
|
||||
|
||||
// Parse fingerprint from output
|
||||
var keyID string
|
||||
|
||||
for line := range strings.SplitSeq(string(output), "\n") {
|
||||
fields := strings.Split(line, ":")
|
||||
if len(fields) >= gpgFingerprintMinFields &&
|
||||
fields[0] == gpgFingerprintField {
|
||||
keyID = fields[9]
|
||||
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if keyID == "" {
|
||||
t.Fatal("failed to find test key fingerprint")
|
||||
}
|
||||
|
||||
return GPGKeyID(keyID), gpgHome
|
||||
}
|
||||
|
||||
// testKeyParams are the gpg key generation parameters of an RSA key that
|
||||
// signs and does not expire.
|
||||
const testKeyParams = "Key-Type: RSA\nKey-Length: 2048\nExpire-Date: 0\n"
|
||||
|
||||
// genTestKey generates a key with no passphrase for
|
||||
// "MFER Test Key <test@mfer.test>" from the gpg key generation parameters
|
||||
// keyParams in gpgHome, which may already hold one.
|
||||
func genTestKey(t *testing.T, gpgHome, keyParams string) {
|
||||
t.Helper()
|
||||
|
||||
params := "%no-protection\n" + keyParams +
|
||||
"Name-Real: MFER Test Key\nName-Email: test@mfer.test\n%commit\n"
|
||||
paramsFile := filepath.Join(gpgHome, "key-params")
|
||||
require.NoError(t, os.WriteFile(paramsFile, []byte(params), 0o600))
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), gpgTimeout)
|
||||
defer cancel()
|
||||
|
||||
//nolint:gosec // paramsFile is a test-controlled path inside t.TempDir()
|
||||
cmd := exec.CommandContext(ctx, "gpg",
|
||||
"--batch", "--gen-key", paramsFile)
|
||||
|
||||
cmd.Env = append(os.Environ(), "GNUPGHOME="+gpgHome)
|
||||
|
||||
output, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Skipf("failed to generate test GPG key: %v: %s", err, output)
|
||||
}
|
||||
}
|
||||
|
||||
// signedTestManifest returns a manifest of one file signed with keyID.
|
||||
func signedTestManifest(t *testing.T, keyID GPGKeyID) []byte {
|
||||
t.Helper()
|
||||
|
||||
b := NewBuilder()
|
||||
b.SetSigningOptions(&SigningOptions{KeyID: keyID})
|
||||
|
||||
content := []byte("signed file content")
|
||||
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, 0,
|
||||
bytes.NewReader(content), nil)
|
||||
require.NoError(t, err)
|
||||
|
||||
var buf bytes.Buffer
|
||||
|
||||
require.NoError(t, b.Build(context.Background(), &buf))
|
||||
|
||||
return buf.Bytes()
|
||||
}
|
||||
|
||||
// rewriteOuter returns manifest with its outer message changed by edit.
|
||||
// A signature stays good as long as edit leaves the UUID and hash alone.
|
||||
func rewriteOuter(t *testing.T, manifest []byte, edit func(*MFFileOuter)) []byte {
|
||||
t.Helper()
|
||||
|
||||
outer := new(MFFileOuter)
|
||||
require.NoError(t, proto.Unmarshal(manifest[len(MAGIC):], outer))
|
||||
|
||||
edit(outer)
|
||||
|
||||
data, err := proto.Marshal(outer)
|
||||
require.NoError(t, err)
|
||||
|
||||
return append([]byte(MAGIC), data...)
|
||||
}
|
||||
|
||||
func TestGPGSign(t *testing.T) {
|
||||
keyID, gpgHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
data := []byte("test data to sign")
|
||||
sig, signingKey, err := gpgSign(context.Background(), data, keyID)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, string(keyID), signingKey)
|
||||
assert.NotEmpty(t, sig)
|
||||
assert.Contains(t, string(sig), "-----BEGIN PGP SIGNATURE-----")
|
||||
assert.Contains(t, string(sig), "-----END PGP SIGNATURE-----")
|
||||
}
|
||||
|
||||
func TestGPGExportPublicKey(t *testing.T) {
|
||||
keyID, gpgHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
pubKey, err := gpgExportPublicKey(context.Background(), keyID)
|
||||
require.NoError(t, err)
|
||||
assert.NotEmpty(t, pubKey)
|
||||
assert.Contains(t, string(pubKey), "-----BEGIN PGP PUBLIC KEY BLOCK-----")
|
||||
assert.Contains(t, string(pubKey), "-----END PGP PUBLIC KEY BLOCK-----")
|
||||
}
|
||||
|
||||
func TestGPGGetKeyFingerprint(t *testing.T) {
|
||||
keyID, gpgHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
fingerprint, err := gpgGetKeyFingerprint(context.Background(), keyID)
|
||||
require.NoError(t, err)
|
||||
assert.NotEmpty(t, fingerprint)
|
||||
// The fingerprint should be 40 hex chars
|
||||
assert.Len(t, fingerprint, 40, "fingerprint should be 40 hex chars")
|
||||
}
|
||||
|
||||
// TestGPGArgsSeparatesPositionals pins that caller-supplied values are
|
||||
// placed after an end-of-options marker. Key IDs arrive from --sign-key
|
||||
// and MFER_SIGN_KEY as bare positional arguments, so without the marker
|
||||
// a value beginning with "-" would be parsed by gpg as one of its own
|
||||
// options.
|
||||
func TestGPGArgsSeparatesPositionals(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
assert.Equal(t,
|
||||
[]string{"--opt-a", "--opt-b", "--", "--version"},
|
||||
gpgArgs([]string{"--opt-a", "--opt-b"}, "--version"))
|
||||
|
||||
assert.Equal(t,
|
||||
[]string{"--opt-c", "--", "sig", "data"},
|
||||
gpgArgs([]string{"--opt-c"}, "sig", "data"))
|
||||
|
||||
assert.Equal(t, []string{"--opt-d", "--"},
|
||||
gpgArgs([]string{"--opt-d"}))
|
||||
}
|
||||
|
||||
// TestGPGOptionLikeKeyIDIsNotAnOption drives real gpg with a key ID that
|
||||
// looks like an option and asserts it is treated as a (nonexistent) key
|
||||
// rather than executed as gpg's own --version.
|
||||
func TestGPGOptionLikeKeyIDIsNotAnOption(t *testing.T) {
|
||||
_, gpgHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
pubKey, err := gpgExportPublicKey(context.Background(), GPGKeyID("--version"))
|
||||
require.Error(t, err)
|
||||
require.ErrorIs(t, err, errGPGKeyNotFound)
|
||||
assert.NotContains(t, string(pubKey), "gpg (GnuPG)")
|
||||
|
||||
fpr, err := gpgGetKeyFingerprint(context.Background(), GPGKeyID("--version"))
|
||||
require.Error(t, err)
|
||||
assert.NotContains(t, string(fpr), "gpg (GnuPG)")
|
||||
}
|
||||
|
||||
// TestGPGSignInvalidKey signs with a key that has no secret key in the
|
||||
// keyring. The error must hold gpg's messages and none of its status lines.
|
||||
func TestGPGSignInvalidKey(t *testing.T) {
|
||||
// Set up test environment (we need GNUPGHOME set)
|
||||
_, gpgHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
data := []byte("test data")
|
||||
_, _, err := gpgSign(context.Background(), data,
|
||||
GPGKeyID("NONEXISTENT_KEY_ID_12345"))
|
||||
require.Error(t, err)
|
||||
assert.NotContains(t, err.Error(), gpgStatusPrefix)
|
||||
}
|
||||
|
||||
func TestBuilderWithSigning(t *testing.T) {
|
||||
keyID, gpgHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
// Create a builder with signing options
|
||||
b := NewBuilder()
|
||||
b.SetSigningOptions(&SigningOptions{
|
||||
KeyID: keyID,
|
||||
})
|
||||
|
||||
// Add a test file
|
||||
content := []byte("test file content")
|
||||
reader := bytes.NewReader(content)
|
||||
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, 0, reader, nil)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Build the manifest
|
||||
var buf bytes.Buffer
|
||||
|
||||
err = b.Build(context.Background(), &buf)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Parse the manifest and verify signature fields are populated
|
||||
manifest, err := NewManifestFromReader(&buf)
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, manifest.pbOuter)
|
||||
|
||||
assert.NotEmpty(t, manifest.pbOuter.GetSignature(),
|
||||
"signature should be populated")
|
||||
assert.NotEmpty(t, manifest.pbOuter.GetSigner(), "signer should be populated")
|
||||
assert.NotEmpty(t, manifest.pbOuter.GetSigningPubKey(),
|
||||
"signing public key should be populated")
|
||||
|
||||
// Verify signature is a valid PGP signature
|
||||
assert.Contains(t, string(manifest.pbOuter.GetSignature()),
|
||||
"-----BEGIN PGP SIGNATURE-----")
|
||||
|
||||
// Verify public key is a valid PGP public key block
|
||||
assert.Contains(t, string(manifest.pbOuter.GetSigningPubKey()),
|
||||
"-----BEGIN PGP PUBLIC KEY BLOCK-----")
|
||||
}
|
||||
|
||||
func TestScannerWithSigning(t *testing.T) {
|
||||
keyID, gpgHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
// Create in-memory filesystem with test files
|
||||
fs := afero.NewMemMapFs()
|
||||
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
|
||||
require.NoError(t,
|
||||
afero.WriteFile(fs, "/testdir/file1.txt", []byte("content1"), 0o644))
|
||||
require.NoError(t,
|
||||
afero.WriteFile(fs, "/testdir/file2.txt", []byte("content2"), 0o644))
|
||||
|
||||
// Create scanner with signing options
|
||||
opts := &ScannerOptions{
|
||||
Fs: fs,
|
||||
SigningOptions: &SigningOptions{
|
||||
KeyID: keyID,
|
||||
},
|
||||
}
|
||||
s := NewScannerWithOptions(opts)
|
||||
|
||||
// Enumerate files
|
||||
require.NoError(t, s.EnumeratePath("/testdir", nil))
|
||||
assert.Equal(t, FileCount(2), s.FileCount())
|
||||
|
||||
// Generate signed manifest
|
||||
var buf bytes.Buffer
|
||||
require.NoError(t, s.ToManifest(context.Background(), &buf, nil))
|
||||
|
||||
// Parse and verify
|
||||
manifest, err := NewManifestFromReader(&buf)
|
||||
require.NoError(t, err)
|
||||
|
||||
assert.NotEmpty(t, manifest.pbOuter.GetSignature())
|
||||
assert.NotEmpty(t, manifest.pbOuter.GetSigner())
|
||||
assert.NotEmpty(t, manifest.pbOuter.GetSigningPubKey())
|
||||
}
|
||||
|
||||
func TestGPGVerify(t *testing.T) {
|
||||
keyID, gpgHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
data := []byte("test data to sign and verify")
|
||||
sig, _, err := gpgSign(context.Background(), data, keyID)
|
||||
require.NoError(t, err)
|
||||
|
||||
pubKey, err := gpgExportPublicKey(context.Background(), keyID)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Verify the signature; it names the key that made it
|
||||
signingKey, err := gpgVerify(context.Background(), data, sig, pubKey)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, string(keyID), signingKey)
|
||||
}
|
||||
|
||||
func TestGPGVerifyInvalidSignature(t *testing.T) {
|
||||
keyID, gpgHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
data := []byte("test data to sign")
|
||||
sig, _, err := gpgSign(context.Background(), data, keyID)
|
||||
require.NoError(t, err)
|
||||
|
||||
pubKey, err := gpgExportPublicKey(context.Background(), keyID)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Try to verify with different data - should fail
|
||||
wrongData := []byte("different data")
|
||||
_, err = gpgVerify(context.Background(), wrongData, sig, pubKey)
|
||||
assert.Error(t, err)
|
||||
}
|
||||
|
||||
func TestGPGVerifyBadPublicKey(t *testing.T) {
|
||||
keyID, gpgHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
data := []byte("test data")
|
||||
sig, _, err := gpgSign(context.Background(), data, keyID)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Try to verify with invalid public key - should fail
|
||||
badPubKey := []byte("not a valid public key")
|
||||
_, err = gpgVerify(context.Background(), data, sig, badPubKey)
|
||||
assert.Error(t, err)
|
||||
}
|
||||
|
||||
func TestManifestSignatureVerification(t *testing.T) {
|
||||
keyID, gpgHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
// Create a builder with signing options
|
||||
b := NewBuilder()
|
||||
b.SetSigningOptions(&SigningOptions{
|
||||
KeyID: keyID,
|
||||
})
|
||||
|
||||
// Add a test file
|
||||
content := []byte("test file content for verification")
|
||||
reader := bytes.NewReader(content)
|
||||
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, 0, reader, nil)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Build the manifest
|
||||
var buf bytes.Buffer
|
||||
|
||||
err = b.Build(context.Background(), &buf)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Parse the manifest - signature should be verified during load
|
||||
manifest, err := NewManifestFromReader(&buf)
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, manifest)
|
||||
|
||||
// Signature should be present and valid
|
||||
assert.NotEmpty(t, manifest.pbOuter.GetSignature())
|
||||
}
|
||||
|
||||
func TestManifestTamperedSignatureFails(t *testing.T) {
|
||||
keyID, gpgHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
// Create a signed manifest
|
||||
b := NewBuilder()
|
||||
b.SetSigningOptions(&SigningOptions{
|
||||
KeyID: keyID,
|
||||
})
|
||||
|
||||
content := []byte("test file content")
|
||||
reader := bytes.NewReader(content)
|
||||
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, 0, reader, nil)
|
||||
require.NoError(t, err)
|
||||
|
||||
var buf bytes.Buffer
|
||||
|
||||
err = b.Build(context.Background(), &buf)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Tamper with the signature by replacing some bytes
|
||||
data := buf.Bytes()
|
||||
// Find and modify a byte in the signature portion
|
||||
for i := range data {
|
||||
if i > 100 && data[i] == 'A' {
|
||||
data[i] = 'B'
|
||||
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
// Try to load the tampered manifest - should fail
|
||||
_, err = NewManifestFromReader(bytes.NewReader(data))
|
||||
assert.Error(t, err)
|
||||
}
|
||||
|
||||
// TestManifestRefusesSecondEmbeddedKey loads a manifest whose embedded
|
||||
// public key block holds another key before the key that signed it.
|
||||
// Loading must refuse it, although the signature is good and the signer
|
||||
// field names the key that made it.
|
||||
func TestManifestRefusesSecondEmbeddedKey(t *testing.T) {
|
||||
otherKey, otherHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", otherHome)
|
||||
|
||||
otherPubKey, err := gpgExportPublicKey(context.Background(), otherKey)
|
||||
require.NoError(t, err)
|
||||
|
||||
keyID, gpgHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
manifest := rewriteOuter(t, signedTestManifest(t, keyID),
|
||||
func(outer *MFFileOuter) {
|
||||
outer.SigningPubKey = slices.Concat(otherPubKey, outer.GetSigningPubKey())
|
||||
})
|
||||
|
||||
_, err = NewManifestFromReader(bytes.NewReader(manifest))
|
||||
require.ErrorIs(t, err, errSigningKeyCount)
|
||||
}
|
||||
|
||||
// TestManifestRefusesSecondEmbeddedKeyWithoutUserID loads a manifest whose
|
||||
// embedded public key block holds, before the key that signed it, another
|
||||
// key with its user ID removed, which gpg skips on import. Loading must
|
||||
// refuse it: the block holds two keys.
|
||||
func TestManifestRefusesSecondEmbeddedKeyWithoutUserID(t *testing.T) {
|
||||
otherKey, otherHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", otherHome)
|
||||
|
||||
// Keeping only the user IDs that match "nobody" exports none.
|
||||
otherPubKey, _, err := runGPG(context.Background(), nil,
|
||||
gpgArgs([]string{
|
||||
"--export", gpgOptArmor, "--export-filter", "keep-uid=uid = nobody",
|
||||
}, string(otherKey))...)
|
||||
require.NoError(t, err)
|
||||
|
||||
keyID, gpgHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
manifest := rewriteOuter(t, signedTestManifest(t, keyID),
|
||||
func(outer *MFFileOuter) {
|
||||
outer.SigningPubKey = slices.Concat(
|
||||
otherPubKey.Bytes(), outer.GetSigningPubKey())
|
||||
})
|
||||
|
||||
_, err = NewManifestFromReader(bytes.NewReader(manifest))
|
||||
require.ErrorIs(t, err, errSigningKeyCount)
|
||||
}
|
||||
|
||||
// TestManifestRefusesTwoSignatures loads a manifest whose signature field
|
||||
// holds its good signature twice. Loading must refuse it.
|
||||
func TestManifestRefusesTwoSignatures(t *testing.T) {
|
||||
keyID, gpgHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
manifest := rewriteOuter(t, signedTestManifest(t, keyID),
|
||||
func(outer *MFFileOuter) {
|
||||
outer.Signature = slices.Concat(
|
||||
outer.GetSignature(), outer.GetSignature())
|
||||
})
|
||||
|
||||
_, err := NewManifestFromReader(bytes.NewReader(manifest))
|
||||
require.ErrorIs(t, err, errNotOneGoodSignature)
|
||||
}
|
||||
|
||||
// TestManifestSignedWithSubkey signs with a key whose primary key can only
|
||||
// certify, so gpg signs with its signing subkey. The manifest must load,
|
||||
// with the primary key's fingerprint as signer.
|
||||
func TestManifestSignedWithSubkey(t *testing.T) {
|
||||
gpgHome := t.TempDir()
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
genTestKey(t, gpgHome, "Key-Type: RSA\nKey-Length: 2048\nKey-Usage: cert\n"+
|
||||
"Subkey-Type: RSA\nSubkey-Length: 2048\nSubkey-Usage: sign\n"+
|
||||
"Expire-Date: 0\n")
|
||||
|
||||
primary, err := gpgGetKeyFingerprint(context.Background(), "test@mfer.test")
|
||||
require.NoError(t, err)
|
||||
|
||||
m, err := NewManifestFromReader(bytes.NewReader(
|
||||
signedTestManifest(t, GPGKeyID("test@mfer.test"))))
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, primary, m.pbOuter.GetSigner())
|
||||
}
|
||||
|
||||
// TestManifestRefusesSignerOtherThanSigningKey loads a manifest whose
|
||||
// signer field names a key other than the one that made the signature.
|
||||
func TestManifestRefusesSignerOtherThanSigningKey(t *testing.T) {
|
||||
keyID, gpgHome := testGPGEnv(t)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
manifest := rewriteOuter(t, signedTestManifest(t, keyID),
|
||||
func(outer *MFFileOuter) {
|
||||
outer.Signer = []byte(strings.Repeat("A", len(keyID)))
|
||||
})
|
||||
|
||||
_, err := NewManifestFromReader(bytes.NewReader(manifest))
|
||||
require.ErrorIs(t, err, errSignerNotSigningKey)
|
||||
}
|
||||
|
||||
// TestBuilderSigningKeyIDMatchingTwoKeys signs with a key ID that two keys
|
||||
// in the keyring match. The manifest must embed and name only the key that
|
||||
// signed it, or loading refuses it.
|
||||
func TestBuilderSigningKeyIDMatchingTwoKeys(t *testing.T) {
|
||||
_, gpgHome := testGPGEnv(t)
|
||||
genTestKey(t, gpgHome, testKeyParams)
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
manifest := signedTestManifest(t, GPGKeyID("test@mfer.test"))
|
||||
|
||||
_, err := NewManifestFromReader(bytes.NewReader(manifest))
|
||||
require.NoError(t, err)
|
||||
}
|
||||
|
||||
// TestBuilderSigningUserIDWithExpiredFirstKey signs with a user ID whose
|
||||
// first key in the keyring has expired. gpg signs with the other key for
|
||||
// that user ID, and the manifest must name and embed that key.
|
||||
func TestBuilderSigningUserIDWithExpiredFirstKey(t *testing.T) {
|
||||
gpgHome := t.TempDir()
|
||||
t.Setenv("GNUPGHOME", gpgHome)
|
||||
|
||||
// Made in 2020 and valid for one day.
|
||||
genTestKey(t, gpgHome, "Key-Type: RSA\nKey-Length: 2048\n"+
|
||||
"Creation-Date: 20200101T000000\nExpire-Date: 1d\n")
|
||||
|
||||
expired, err := gpgGetKeyFingerprint(context.Background(), "test@mfer.test")
|
||||
require.NoError(t, err)
|
||||
|
||||
genTestKey(t, gpgHome, testKeyParams)
|
||||
|
||||
m, err := NewManifestFromReader(bytes.NewReader(
|
||||
signedTestManifest(t, GPGKeyID("test@mfer.test"))))
|
||||
require.NoError(t, err)
|
||||
assert.NotEqual(t, expired, m.pbOuter.GetSigner())
|
||||
}
|
||||
|
||||
func TestBuilderWithoutSigning(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// Create a builder without signing options
|
||||
b := NewBuilder()
|
||||
|
||||
// Add a test file
|
||||
content := []byte("test file content")
|
||||
reader := bytes.NewReader(content)
|
||||
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, 0, reader, nil)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Build the manifest
|
||||
var buf bytes.Buffer
|
||||
|
||||
err = b.Build(context.Background(), &buf)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Parse the manifest and verify signature fields are empty
|
||||
manifest, err := NewManifestFromReader(&buf)
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, manifest.pbOuter)
|
||||
|
||||
assert.Empty(t, manifest.pbOuter.GetSignature(),
|
||||
"signature should be empty when not signing")
|
||||
assert.Empty(t, manifest.pbOuter.GetSigner(),
|
||||
"signer should be empty when not signing")
|
||||
assert.Empty(t, manifest.pbOuter.GetSigningPubKey(),
|
||||
"signing public key should be empty when not signing")
|
||||
}
|
||||
|
||||
// fakeGPGPath writes script as an executable named gpg into a temporary
|
||||
// directory and returns a PATH value with that directory first.
|
||||
func fakeGPGPath(t *testing.T, script string) string {
|
||||
t.Helper()
|
||||
|
||||
binDir := t.TempDir()
|
||||
//nolint:gosec // G306: the fake gpg has to be executable
|
||||
require.NoError(t, os.WriteFile(filepath.Join(binDir, "gpg"),
|
||||
[]byte(script), 0o700))
|
||||
|
||||
return binDir + string(os.PathListSeparator) + os.Getenv("PATH")
|
||||
}
|
||||
|
||||
// TestGPGTimeoutKillsGPG puts a fake gpg that never finishes first on
|
||||
// PATH and checks that a run past its deadline is killed and reported as
|
||||
// a timeout of the named operation, instead of hanging. The fake gpg writes
|
||||
// nothing to stderr, so the message ends with the timeout.
|
||||
func TestGPGTimeoutKillsGPG(t *testing.T) {
|
||||
t.Setenv("PATH", fakeGPGPath(t, "#!/bin/sh\nexec sleep 10\n"))
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
|
||||
defer cancel()
|
||||
|
||||
_, _, err := gpgSign(ctx, []byte("data"), GPGKeyID("any"))
|
||||
require.ErrorIs(t, err, context.DeadlineExceeded)
|
||||
assert.EqualError(t, err, "gpg sign: timed out: context deadline exceeded")
|
||||
}
|
||||
|
||||
// TestGPGSignKeyNotReportedKeepsStderr puts a fake gpg first on PATH that
|
||||
// exits cleanly without reporting the key that signed, and checks that what
|
||||
// it wrote to stderr is in the message.
|
||||
func TestGPGSignKeyNotReportedKeepsStderr(t *testing.T) {
|
||||
t.Setenv("PATH", fakeGPGPath(t,
|
||||
"#!/bin/sh\necho 'gpg: note from the fake gpg' >&2\n"))
|
||||
|
||||
_, _, err := gpgSign(context.Background(), []byte("data"), GPGKeyID("any"))
|
||||
require.ErrorIs(t, err, errSigningKeyNotReported)
|
||||
assert.EqualError(t, err,
|
||||
"gpg did not report the key that made the signature: "+
|
||||
"gpg: note from the fake gpg")
|
||||
}
|
||||
|
||||
// TestGPGFailureKeepsStderr puts a fake gpg first on PATH that writes to
|
||||
// stderr and exits non-zero, and checks that what it wrote ends the message.
|
||||
func TestGPGFailureKeepsStderr(t *testing.T) {
|
||||
t.Setenv("PATH", fakeGPGPath(t,
|
||||
"#!/bin/sh\necho 'gpg: signing failed: No secret key' >&2\nexit 2\n"))
|
||||
|
||||
_, _, err := gpgSign(context.Background(), []byte("data"), GPGKeyID("any"))
|
||||
assert.EqualError(t, err,
|
||||
"gpg sign: exit status 2: gpg: signing failed: No secret key")
|
||||
}
|
||||
|
||||
// TestGPGCancelWhenChildHoldsOutput uses a fake gpg that runs sleep as a
|
||||
// child instead of exec-ing it, the way a wrapper script around the real
|
||||
// gpg might. Killing the fake gpg leaves sleep holding its stdout and
|
||||
// stderr open; the call must still return once ctx ends instead of waiting
|
||||
// for sleep to exit. The fake gpg writes the process ID of sleep to a named
|
||||
// pipe; the test ends ctx only after reading it, so sleep is running by
|
||||
// then, and kills sleep before returning.
|
||||
func TestGPGCancelWhenChildHoldsOutput(t *testing.T) {
|
||||
pidPipe := filepath.Join(t.TempDir(), "sleep.pid")
|
||||
require.NoError(t, syscall.Mkfifo(pidPipe, 0o600))
|
||||
// sleep outlasts the 10 s wait below, so a call that waits for it fails.
|
||||
t.Setenv("PATH", fakeGPGPath(t,
|
||||
"#!/bin/sh\nsleep 60 &\necho $! >'"+pidPipe+"'\nwait\n"))
|
||||
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
|
||||
signErr := make(chan error, 1)
|
||||
|
||||
go func() {
|
||||
_, _, err := gpgSign(ctx, []byte("data"), GPGKeyID("any"))
|
||||
signErr <- err
|
||||
}()
|
||||
|
||||
pid, err := os.ReadFile(pidPipe) //nolint:gosec // G304: path inside t.TempDir()
|
||||
require.NoError(t, err)
|
||||
|
||||
n, err := strconv.Atoi(strings.TrimSpace(string(pid)))
|
||||
require.NoError(t, err)
|
||||
|
||||
sleep, err := os.FindProcess(n)
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { require.NoError(t, sleep.Kill()) })
|
||||
|
||||
cancel()
|
||||
|
||||
// The call should return about gpgWaitDelay (one second) after the
|
||||
// cancel. 10 s is far above that and well under the 30 s test timeout,
|
||||
// which would abort the whole package before the cleanup kills sleep.
|
||||
select {
|
||||
case err := <-signErr:
|
||||
require.ErrorIs(t, err, context.Canceled)
|
||||
case <-time.After(10 * time.Second):
|
||||
t.Fatal("the call waited for the child holding gpg's output to exit")
|
||||
}
|
||||
}
|
||||
|
||||
// TestBuildPassesContextToSigning checks that a caller can cancel the gpg
|
||||
// runs that sign a manifest through the context given to Build.
|
||||
func TestBuildPassesContextToSigning(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
b := NewBuilder()
|
||||
b.SetSigningOptions(&SigningOptions{KeyID: "any"})
|
||||
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
|
||||
require.ErrorIs(t, b.Build(ctx, io.Discard), context.Canceled)
|
||||
}
|
||||
@@ -1,30 +0,0 @@
|
||||
//nolint:testpackage // white-box tests exercise unexported internals
|
||||
package mfer
|
||||
|
||||
import (
|
||||
"encoding/hex"
|
||||
"testing"
|
||||
"uuid"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// TestSignatureString checks the canonical string for the UUID and sha256
|
||||
// of the example in the Signature Scheme section of docs/FORMAT.md, which
|
||||
// shows the same string.
|
||||
func TestSignatureString(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
manifestUUID := uuid.MustParse("b0fb6918-6298-443b-ada2-1b162f4dc0b3")
|
||||
sha256Hash, err := hex.DecodeString(
|
||||
"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855")
|
||||
require.NoError(t, err)
|
||||
|
||||
m := &manifest{pbOuter: &MFFileOuter{Uuid: manifestUUID[:], Sha256: sha256Hash}}
|
||||
|
||||
got, err := m.signatureString()
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "ZNAVSRFG-b0fb69186298443bada21b162f4dc0b3-"+
|
||||
"1220e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", got)
|
||||
}
|
||||
+3
-6
@@ -223,14 +223,11 @@ type MFFileOuter struct {
|
||||
// uuid must match the uuid in the inner message
|
||||
Uuid []byte `protobuf:"bytes,105,opt,name=uuid,proto3" json:"uuid,omitempty"`
|
||||
InnerMessage []byte `protobuf:"bytes,199,opt,name=innerMessage,proto3" json:"innerMessage,omitempty"`
|
||||
// one OpenPGP signature, armored or binary, over the canonical string
|
||||
// ZNAVSRFG-<uuid>-<sha256 as a multihash>, with the uuid as 32
|
||||
// lower-case hex digits and no hyphens, and the multihash in lower-case
|
||||
// hex (see docs/FORMAT.md). A signature is never stored anywhere else.
|
||||
//detached signature, ascii or binary
|
||||
Signature []byte `protobuf:"bytes,201,opt,name=signature,proto3,oneof" json:"signature,omitempty"`
|
||||
// fingerprint of the signing primary key, in upper-case hex
|
||||
//full GPG key id
|
||||
Signer []byte `protobuf:"bytes,202,opt,name=signer,proto3,oneof" json:"signer,omitempty"`
|
||||
// OpenPGP public key of the signer, one primary key, armored or binary
|
||||
//full GPG signing public key, ascii or binary
|
||||
SigningPubKey []byte `protobuf:"bytes,203,opt,name=signingPubKey,proto3,oneof" json:"signingPubKey,omitempty"`
|
||||
unknownFields protoimpl.UnknownFields
|
||||
sizeCache protoimpl.SizeCache
|
||||
|
||||
+5
-7
@@ -32,17 +32,15 @@ message MFFileOuter {
|
||||
bytes uuid = 105;
|
||||
|
||||
bytes innerMessage = 199;
|
||||
|
||||
// 2xx for optional manifest root attributes
|
||||
// think we might use gosignify instead of gpg:
|
||||
// github.com/frankbraun/gosignify
|
||||
|
||||
// one OpenPGP signature, armored or binary, over the canonical string
|
||||
// ZNAVSRFG-<uuid>-<sha256 as a multihash>, with the uuid as 32
|
||||
// lower-case hex digits and no hyphens, and the multihash in lower-case
|
||||
// hex (see docs/FORMAT.md). A signature is never stored anywhere else.
|
||||
//detached signature, ascii or binary
|
||||
optional bytes signature = 201;
|
||||
// fingerprint of the signing primary key, in upper-case hex
|
||||
//full GPG key id
|
||||
optional bytes signer = 202;
|
||||
// OpenPGP public key of the signer, one primary key, armored or binary
|
||||
//full GPG signing public key, ascii or binary
|
||||
optional bytes signingPubKey = 203;
|
||||
}
|
||||
|
||||
|
||||
@@ -1 +1 @@
|
||||
4f8ea5276a8e66590e71ed154a9176b0f68ffa9ab69d8de5aff0e0d9b731164d mf.proto
|
||||
3d4dcb0b2f4640dd3ae6bb48b833e9f26ae9771e2d3e88bd646e7f1724dda654 mf.proto
|
||||
|
||||
-383
@@ -1,383 +0,0 @@
|
||||
package mfer
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/ProtonMail/go-crypto/openpgp"
|
||||
"github.com/ProtonMail/go-crypto/openpgp/armor"
|
||||
pgperrors "github.com/ProtonMail/go-crypto/openpgp/errors"
|
||||
"github.com/ProtonMail/go-crypto/openpgp/packet"
|
||||
)
|
||||
|
||||
const (
|
||||
// The tags of OpenPGP signature, key and subkey packets (RFC 9580,
|
||||
// section 5). Subkeys have tags of their own, so each secret or public
|
||||
// key packet is one primary key.
|
||||
signaturePacketTag = 2
|
||||
secretKeyPacketTag = 5
|
||||
publicKeyPacketTag = 6
|
||||
secretSubkeyPacketTag = 7
|
||||
publicSubkeyPacketTag = 14
|
||||
|
||||
// In the body of a key or subkey packet the algorithm octet follows
|
||||
// the version octet and the four-octet creation time, and from version
|
||||
// 5 on a four-octet length as well (RFC 9580, section 5.5.2).
|
||||
keyAlgorithmOffset = 5
|
||||
firstKeyVersionWithLength = 5
|
||||
keyAlgorithmOffsetAfterLength = 9
|
||||
|
||||
// signingKeyVersion is the only OpenPGP key version mfer signs with.
|
||||
// Its fingerprints are 40 hex characters, the length
|
||||
// --require-signature takes.
|
||||
signingKeyVersion = 4
|
||||
|
||||
// armorBegin and armorEnd start the lines that begin and end an
|
||||
// armored block.
|
||||
armorBegin = "-----BEGIN "
|
||||
armorEnd = "-----END "
|
||||
)
|
||||
|
||||
var (
|
||||
errKeyCount = errors.New("must hold exactly one key")
|
||||
errDSAKey = errors.New("must not hold a DSA key")
|
||||
errSecretKey = errors.New("must not hold a secret key")
|
||||
errNoSecretKey = errors.New("signing key file holds no secret key")
|
||||
errNotV4Key = errors.New("signing key must be an OpenPGP version 4 key, " +
|
||||
"the only kind whose fingerprint --require-signature takes")
|
||||
errNoPassphrase = errors.New(
|
||||
"signing key is protected and no passphrase was given")
|
||||
errNotOneSignature = errors.New(
|
||||
"signature must hold exactly one signature")
|
||||
errNotOneArmoredBlock = errors.New(
|
||||
"must be exactly one armored block and nothing else")
|
||||
errMalformedArmor = errors.New("armor is malformed")
|
||||
)
|
||||
|
||||
// SigningOptions holds the key a manifest is signed with.
|
||||
type SigningOptions struct {
|
||||
// SecretKey is an OpenPGP secret key, armored or binary, as
|
||||
// gpg --export-secret-keys writes it. It must hold one primary key.
|
||||
SecretKey []byte
|
||||
// Passphrase unlocks SecretKey when it is protected.
|
||||
Passphrase []byte
|
||||
}
|
||||
|
||||
// SecretKeyIsProtected reports whether the OpenPGP secret key secretKey,
|
||||
// armored or binary, needs a passphrase to sign. It fails unless
|
||||
// secretKey holds one version 4 primary key with its secret key.
|
||||
func SecretKeyIsProtected(secretKey []byte) (bool, error) {
|
||||
key, err := readSecretKey(secretKey)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
|
||||
return isProtected(key), nil
|
||||
}
|
||||
|
||||
// CheckSigningKey fails unless opts can sign now: opts.SecretKey must hold
|
||||
// one version 4 primary key with a secret key that may sign and has not
|
||||
// expired or been revoked, and opts.Passphrase must unlock it when it is
|
||||
// protected. It lets a caller find a key that cannot sign before it builds
|
||||
// a manifest.
|
||||
func CheckSigningKey(opts *SigningOptions) error {
|
||||
key, err := readSigningKey(opts)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Signing nothing fails wherever signing the manifest would.
|
||||
err = openpgp.DetachSign(io.Discard, key, bytes.NewReader(nil), nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("signing key cannot sign: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// readSigningKey returns the key in opts.SecretKey, unlocked with
|
||||
// opts.Passphrase if it is protected.
|
||||
func readSigningKey(opts *SigningOptions) (*openpgp.Entity, error) {
|
||||
key, err := readSecretKey(opts.SecretKey)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if !isProtected(key) {
|
||||
return key, nil
|
||||
}
|
||||
|
||||
if len(opts.Passphrase) == 0 {
|
||||
return nil, errNoPassphrase
|
||||
}
|
||||
|
||||
err = key.DecryptPrivateKeys(opts.Passphrase)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("unlock signing key: %w", err)
|
||||
}
|
||||
|
||||
return key, nil
|
||||
}
|
||||
|
||||
// readSecretKey returns the one key in secretKey, armored or binary,
|
||||
// which must be a version 4 key and include its secret key.
|
||||
func readSecretKey(secretKey []byte) (*openpgp.Entity, error) {
|
||||
key, err := readOneKey(secretKey, "signing key file", false)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if key.PrimaryKey.Version != signingKeyVersion {
|
||||
return nil, fmt.Errorf("%w; this key is version %d",
|
||||
errNotV4Key, key.PrimaryKey.Version)
|
||||
}
|
||||
|
||||
if key.PrivateKey == nil {
|
||||
return nil, errNoSecretKey
|
||||
}
|
||||
|
||||
return key, nil
|
||||
}
|
||||
|
||||
// readOneKey returns the key in data, armored or binary, which must hold
|
||||
// exactly one primary key and no DSA key or subkey, and when publicOnly no
|
||||
// secret key or subkey either. what names data in errors.
|
||||
func readOneKey(data []byte, what string, publicOnly bool) (*openpgp.Entity, error) {
|
||||
packets, err := dearmor(data)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read %s: %w", what, err)
|
||||
}
|
||||
|
||||
keys, err := countPackets(packets, secretKeyPacketTag, publicKeyPacketTag)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read %s: %w", what, err)
|
||||
}
|
||||
|
||||
if keys != 1 {
|
||||
return nil, fmt.Errorf("%s %w, found %d", what, errKeyCount, keys)
|
||||
}
|
||||
|
||||
// openpgp.ReadKeyRing checks the numbers of every secret key it reads,
|
||||
// and an ElGamal secret subkey with a very large prime makes that take
|
||||
// minutes.
|
||||
secretKeys, err := countPackets(packets, secretKeyPacketTag, secretSubkeyPacketTag)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read %s: %w", what, err)
|
||||
}
|
||||
|
||||
if publicOnly && secretKeys != 0 {
|
||||
return nil, fmt.Errorf("%s %w", what, errSecretKey)
|
||||
}
|
||||
|
||||
// openpgp.ReadKeyRing also checks every self-signature, and a DSA key
|
||||
// with very large numbers makes each check take seconds to minutes.
|
||||
dsa, err := holdsDSAKey(packets)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read %s: %w", what, err)
|
||||
}
|
||||
|
||||
if dsa {
|
||||
return nil, fmt.Errorf("%s %w", what, errDSAKey)
|
||||
}
|
||||
|
||||
keyring, err := openpgp.ReadKeyRing(bytes.NewReader(packets))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read %s: %w", what, err)
|
||||
}
|
||||
|
||||
// openpgp.ReadKeyRing also reads a subkey packet at the start as a
|
||||
// primary key.
|
||||
if len(keyring) != 1 {
|
||||
return nil, fmt.Errorf("%s %w, found %d", what, errKeyCount, len(keyring))
|
||||
}
|
||||
|
||||
return keyring[0], nil
|
||||
}
|
||||
|
||||
// isProtected reports whether any secret key in key needs a passphrase.
|
||||
func isProtected(key *openpgp.Entity) bool {
|
||||
if key.PrivateKey.Encrypted {
|
||||
return true
|
||||
}
|
||||
|
||||
for _, subkey := range key.Subkeys {
|
||||
if subkey.PrivateKey != nil && subkey.PrivateKey.Encrypted {
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
|
||||
// armoredPublicKey returns the public part of key, armored.
|
||||
func armoredPublicKey(key *openpgp.Entity) ([]byte, error) {
|
||||
var buf bytes.Buffer
|
||||
|
||||
w, err := armor.Encode(&buf, openpgp.PublicKeyType, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
err = key.Serialize(w)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("write public key: %w", err)
|
||||
}
|
||||
|
||||
err = w.Close()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return buf.Bytes(), nil
|
||||
}
|
||||
|
||||
// fingerprint returns the fingerprint of key's primary key in upper-case
|
||||
// hex, as gpg prints it.
|
||||
func fingerprint(key *openpgp.Entity) string {
|
||||
return strings.ToUpper(hex.EncodeToString(key.PrimaryKey.Fingerprint))
|
||||
}
|
||||
|
||||
// verifySignature checks that signature is one good, unexpired OpenPGP
|
||||
// signature over data, made by the one primary key in pubKey or one of its
|
||||
// subkeys, and returns that primary key's fingerprint. signature and pubKey
|
||||
// may each be armored or binary.
|
||||
func verifySignature(data, signature, pubKey []byte) (string, error) {
|
||||
key, err := readOneKey(pubKey, "embedded public key block", true)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
sigData, err := dearmor(signature)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("read signature: %w", err)
|
||||
}
|
||||
|
||||
sigs, err := countPackets(sigData, signaturePacketTag)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("read signature: %w", err)
|
||||
}
|
||||
|
||||
if sigs != 1 {
|
||||
return "", fmt.Errorf("%w, found %d", errNotOneSignature, sigs)
|
||||
}
|
||||
|
||||
sig, _, err := openpgp.VerifyDetachedSignature(openpgp.EntityList{key},
|
||||
bytes.NewReader(data), bytes.NewReader(sigData), nil)
|
||||
// A manifest outlives its signing key, so a signature by a key that
|
||||
// has expired since is still good, unless the signature has expired
|
||||
// itself. The library does not check that once it finds the key
|
||||
// expired.
|
||||
if errors.Is(err, pgperrors.ErrKeyExpired) && sig.SigExpired(time.Now()) {
|
||||
err = pgperrors.ErrSignatureExpired
|
||||
}
|
||||
|
||||
if err != nil && !errors.Is(err, pgperrors.ErrKeyExpired) {
|
||||
return "", fmt.Errorf("verify signature: %w", err)
|
||||
}
|
||||
|
||||
return fingerprint(key), nil
|
||||
}
|
||||
|
||||
// dearmor returns the binary OpenPGP data in data: data itself when it is
|
||||
// not armored, or else the body of its armored block. Armored data must be
|
||||
// one block and nothing else: its first line is the only BEGIN line and
|
||||
// its last line the only END line, white space around them aside.
|
||||
// armor.Decode skips any text before a BEGIN line and reads only the first
|
||||
// block, so without this a second key or signature would go unseen.
|
||||
func dearmor(data []byte) ([]byte, error) {
|
||||
if !bytes.Contains(data, []byte(armorBegin)) {
|
||||
return data, nil
|
||||
}
|
||||
|
||||
text := bytes.TrimSpace(data)
|
||||
lastLine := text[bytes.LastIndexByte(text, '\n')+1:]
|
||||
|
||||
if !bytes.HasPrefix(text, []byte(armorBegin)) ||
|
||||
bytes.Count(text, []byte(armorBegin)) != 1 ||
|
||||
!bytes.HasPrefix(lastLine, []byte(armorEnd)) ||
|
||||
bytes.Count(text, []byte(armorEnd)) != 1 {
|
||||
return nil, errNotOneArmoredBlock
|
||||
}
|
||||
|
||||
// armor.Decode passes over a block it cannot read, such as one with a
|
||||
// header line that has no colon, and returns io.EOF on finding no
|
||||
// other.
|
||||
block, err := armor.Decode(bytes.NewReader(text))
|
||||
if err != nil {
|
||||
return nil, errMalformedArmor
|
||||
}
|
||||
|
||||
body, err := io.ReadAll(block.Body)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: %w", errMalformedArmor, err)
|
||||
}
|
||||
|
||||
return body, nil
|
||||
}
|
||||
|
||||
// countPackets returns how many packets in the binary OpenPGP data have
|
||||
// one of tags. It reads only each packet's header, so it also counts
|
||||
// packets that openpgp.ReadKeyRing skips, such as a key with no user ID
|
||||
// or of an algorithm it does not know.
|
||||
func countPackets(data []byte, tags ...uint8) (int, error) {
|
||||
packets := packet.NewOpaqueReader(bytes.NewReader(data))
|
||||
count := 0
|
||||
|
||||
for {
|
||||
p, err := packets.Next()
|
||||
if errors.Is(err, io.EOF) {
|
||||
return count, nil
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
if slices.Contains(tags, p.Tag) {
|
||||
count++
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// holdsDSAKey reports whether any key or subkey packet in the binary
|
||||
// OpenPGP data holds a DSA key. It reads each packet's algorithm octet
|
||||
// rather than parsing the packet, since parsing a secret key packet checks
|
||||
// its numbers, which for a DSA key with very large numbers is as slow as
|
||||
// checking a self-signature.
|
||||
func holdsDSAKey(data []byte) (bool, error) {
|
||||
packets := packet.NewOpaqueReader(bytes.NewReader(data))
|
||||
|
||||
for {
|
||||
p, err := packets.Next()
|
||||
if errors.Is(err, io.EOF) {
|
||||
return false, nil
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
|
||||
if !slices.Contains([]uint8{
|
||||
secretKeyPacketTag, publicKeyPacketTag,
|
||||
secretSubkeyPacketTag, publicSubkeyPacketTag,
|
||||
}, p.Tag) {
|
||||
continue
|
||||
}
|
||||
|
||||
offset := keyAlgorithmOffset
|
||||
if len(p.Contents) > 0 && p.Contents[0] >= firstKeyVersionWithLength {
|
||||
offset = keyAlgorithmOffsetAfterLength
|
||||
}
|
||||
|
||||
if len(p.Contents) > offset &&
|
||||
packet.PublicKeyAlgorithm(p.Contents[offset]) == packet.PubKeyAlgoDSA {
|
||||
return true, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,814 +0,0 @@
|
||||
//nolint:testpackage // white-box tests exercise unexported internals
|
||||
package mfer
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/dsa" //nolint:staticcheck // SA1019: tests need a DSA key to refuse
|
||||
"io"
|
||||
"math/big"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/ProtonMail/go-crypto/openpgp"
|
||||
"github.com/ProtonMail/go-crypto/openpgp/armor"
|
||||
"github.com/ProtonMail/go-crypto/openpgp/elgamal"
|
||||
pgperrors "github.com/ProtonMail/go-crypto/openpgp/errors"
|
||||
"github.com/ProtonMail/go-crypto/openpgp/packet"
|
||||
"github.com/spf13/afero"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"google.golang.org/protobuf/proto"
|
||||
)
|
||||
|
||||
// newTestKey returns a new Ed25519 key, which is quick to make, for
|
||||
// "MFER Test Key <test@mfer.test>". config may set when it is made and how
|
||||
// long it lasts.
|
||||
func newTestKey(t *testing.T, config *packet.Config) *openpgp.Entity {
|
||||
t.Helper()
|
||||
|
||||
if config == nil {
|
||||
config = &packet.Config{}
|
||||
}
|
||||
|
||||
config.Algorithm = packet.PubKeyAlgoEdDSA
|
||||
|
||||
key, err := openpgp.NewEntity("MFER Test Key", "", "test@mfer.test", config)
|
||||
require.NoError(t, err)
|
||||
|
||||
return key
|
||||
}
|
||||
|
||||
// armoredSecretKeys returns keys with their secret keys in one armored
|
||||
// block, as gpg --export-secret-keys --armor writes them.
|
||||
func armoredSecretKeys(t *testing.T, keys ...*openpgp.Entity) []byte {
|
||||
t.Helper()
|
||||
|
||||
var buf bytes.Buffer
|
||||
|
||||
w, err := armor.Encode(&buf, openpgp.PrivateKeyType, nil)
|
||||
require.NoError(t, err)
|
||||
|
||||
for _, key := range keys {
|
||||
require.NoError(t, key.SerializePrivateWithoutSigning(w, nil))
|
||||
}
|
||||
|
||||
require.NoError(t, w.Close())
|
||||
|
||||
return buf.Bytes()
|
||||
}
|
||||
|
||||
// armoredPublicKeys returns the public parts of keys in one armored block,
|
||||
// as gpg --export --armor writes them.
|
||||
func armoredPublicKeys(t *testing.T, keys ...*openpgp.Entity) []byte {
|
||||
t.Helper()
|
||||
|
||||
var buf bytes.Buffer
|
||||
|
||||
w, err := armor.Encode(&buf, openpgp.PublicKeyType, nil)
|
||||
require.NoError(t, err)
|
||||
|
||||
for _, key := range keys {
|
||||
require.NoError(t, key.Serialize(w))
|
||||
}
|
||||
|
||||
require.NoError(t, w.Close())
|
||||
|
||||
return buf.Bytes()
|
||||
}
|
||||
|
||||
// testSigningOptions returns signing options for a new key with no
|
||||
// passphrase.
|
||||
func testSigningOptions(t *testing.T) *SigningOptions {
|
||||
t.Helper()
|
||||
|
||||
return &SigningOptions{SecretKey: armoredSecretKeys(t, newTestKey(t, nil))}
|
||||
}
|
||||
|
||||
// joinArmored returns the armored block first followed by the armored
|
||||
// block second on the next line. armor.Encode ends a block without a
|
||||
// newline, unlike gpg, and a block only starts at the start of a line.
|
||||
func joinArmored(first, second []byte) []byte {
|
||||
return slices.Concat(first, []byte("\n"), second)
|
||||
}
|
||||
|
||||
// signedTestManifest returns a manifest of one file signed with opts.
|
||||
func signedTestManifest(t *testing.T, opts *SigningOptions) []byte {
|
||||
t.Helper()
|
||||
|
||||
b := NewBuilder()
|
||||
b.SetSigningOptions(opts)
|
||||
|
||||
content := []byte("signed file content")
|
||||
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, 0,
|
||||
bytes.NewReader(content), nil)
|
||||
require.NoError(t, err)
|
||||
|
||||
var buf bytes.Buffer
|
||||
|
||||
require.NoError(t, b.Build(context.Background(), &buf))
|
||||
|
||||
return buf.Bytes()
|
||||
}
|
||||
|
||||
// rewriteOuter returns manifest with its outer message changed by edit.
|
||||
// A signature stays good as long as edit leaves the UUID and hash alone.
|
||||
func rewriteOuter(t *testing.T, manifest []byte, edit func(*MFFileOuter)) []byte {
|
||||
t.Helper()
|
||||
|
||||
outer := new(MFFileOuter)
|
||||
require.NoError(t, proto.Unmarshal(manifest[len(MAGIC):], outer))
|
||||
|
||||
edit(outer)
|
||||
|
||||
data, err := proto.Marshal(outer)
|
||||
require.NoError(t, err)
|
||||
|
||||
return append([]byte(MAGIC), data...)
|
||||
}
|
||||
|
||||
func TestBuilderWithSigning(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
key := newTestKey(t, nil)
|
||||
|
||||
// Create a builder with signing options
|
||||
b := NewBuilder()
|
||||
b.SetSigningOptions(&SigningOptions{SecretKey: armoredSecretKeys(t, key)})
|
||||
|
||||
// Add a test file
|
||||
content := []byte("test file content")
|
||||
reader := bytes.NewReader(content)
|
||||
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, 0, reader, nil)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Build the manifest
|
||||
var buf bytes.Buffer
|
||||
|
||||
err = b.Build(context.Background(), &buf)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Parse the manifest and verify signature fields are populated
|
||||
manifest, err := NewManifestFromReader(&buf)
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, manifest.pbOuter)
|
||||
|
||||
assert.NotEmpty(t, manifest.pbOuter.GetSignature(),
|
||||
"signature should be populated")
|
||||
assert.NotEmpty(t, manifest.pbOuter.GetSigningPubKey(),
|
||||
"signing public key should be populated")
|
||||
|
||||
// The signer is the key's fingerprint in 40 upper-case hex characters.
|
||||
assert.Equal(t, fingerprint(key), string(manifest.pbOuter.GetSigner()))
|
||||
assert.Regexp(t, "^[0-9A-F]{40}$", string(manifest.pbOuter.GetSigner()))
|
||||
|
||||
// Verify signature is a valid PGP signature
|
||||
assert.Contains(t, string(manifest.pbOuter.GetSignature()),
|
||||
"-----BEGIN PGP SIGNATURE-----")
|
||||
|
||||
// Verify public key is a valid PGP public key block
|
||||
assert.Contains(t, string(manifest.pbOuter.GetSigningPubKey()),
|
||||
"-----BEGIN PGP PUBLIC KEY BLOCK-----")
|
||||
}
|
||||
|
||||
func TestScannerWithSigning(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// Create in-memory filesystem with test files
|
||||
fs := afero.NewMemMapFs()
|
||||
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
|
||||
require.NoError(t,
|
||||
afero.WriteFile(fs, "/testdir/file1.txt", []byte("content1"), 0o644))
|
||||
require.NoError(t,
|
||||
afero.WriteFile(fs, "/testdir/file2.txt", []byte("content2"), 0o644))
|
||||
|
||||
// Create scanner with signing options
|
||||
opts := &ScannerOptions{
|
||||
Fs: fs,
|
||||
SigningOptions: testSigningOptions(t),
|
||||
}
|
||||
s := NewScannerWithOptions(opts)
|
||||
|
||||
// Enumerate files
|
||||
require.NoError(t, s.EnumeratePath("/testdir", nil))
|
||||
assert.Equal(t, FileCount(2), s.FileCount())
|
||||
|
||||
// Generate signed manifest
|
||||
var buf bytes.Buffer
|
||||
require.NoError(t, s.ToManifest(context.Background(), &buf, nil))
|
||||
|
||||
// Parse and verify
|
||||
manifest, err := NewManifestFromReader(&buf)
|
||||
require.NoError(t, err)
|
||||
|
||||
assert.NotEmpty(t, manifest.pbOuter.GetSignature())
|
||||
assert.NotEmpty(t, manifest.pbOuter.GetSigner())
|
||||
assert.NotEmpty(t, manifest.pbOuter.GetSigningPubKey())
|
||||
}
|
||||
|
||||
// TestSigningWithBinarySecretKey signs with a secret key that is not
|
||||
// armored, as gpg --export-secret-keys writes it without --armor.
|
||||
func TestSigningWithBinarySecretKey(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var secretKey bytes.Buffer
|
||||
|
||||
require.NoError(t, newTestKey(t, nil).SerializePrivateWithoutSigning(&secretKey, nil))
|
||||
|
||||
_, err := NewManifestFromReader(bytes.NewReader(
|
||||
signedTestManifest(t, &SigningOptions{SecretKey: secretKey.Bytes()})))
|
||||
require.NoError(t, err)
|
||||
}
|
||||
|
||||
// TestSigningWithProtectedKey signs with a key protected by a passphrase:
|
||||
// with the passphrase, without one, and with a wrong one.
|
||||
func TestSigningWithProtectedKey(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
key := newTestKey(t, nil)
|
||||
require.NoError(t, key.EncryptPrivateKeys([]byte("right"), nil))
|
||||
secretKey := armoredSecretKeys(t, key)
|
||||
|
||||
protected, err := SecretKeyIsProtected(secretKey)
|
||||
require.NoError(t, err)
|
||||
assert.True(t, protected)
|
||||
|
||||
_, err = NewManifestFromReader(bytes.NewReader(signedTestManifest(t,
|
||||
&SigningOptions{SecretKey: secretKey, Passphrase: []byte("right")})))
|
||||
require.NoError(t, err)
|
||||
|
||||
b := NewBuilder()
|
||||
b.SetSigningOptions(&SigningOptions{SecretKey: secretKey})
|
||||
require.ErrorIs(t, b.Build(context.Background(), io.Discard), errNoPassphrase)
|
||||
|
||||
b.SetSigningOptions(&SigningOptions{
|
||||
SecretKey: secretKey, Passphrase: []byte("wrong"),
|
||||
})
|
||||
assert.ErrorContains(t, b.Build(context.Background(), io.Discard),
|
||||
"unlock signing key")
|
||||
}
|
||||
|
||||
func TestSecretKeyIsProtectedWithoutPassphrase(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
protected, err := SecretKeyIsProtected(testSigningOptions(t).SecretKey)
|
||||
require.NoError(t, err)
|
||||
assert.False(t, protected)
|
||||
}
|
||||
|
||||
// TestSigningKeyFileWithTwoKeys signs with a key file that holds two keys,
|
||||
// as gpg writes it for a user ID that two keys have. It names no one key
|
||||
// to sign with, so signing must fail.
|
||||
func TestSigningKeyFileWithTwoKeys(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
b := NewBuilder()
|
||||
b.SetSigningOptions(&SigningOptions{SecretKey: armoredSecretKeys(t,
|
||||
newTestKey(t, nil), newTestKey(t, nil))})
|
||||
|
||||
err := b.Build(context.Background(), io.Discard)
|
||||
require.ErrorIs(t, err, errKeyCount)
|
||||
assert.EqualError(t, err, "signing key file must hold exactly one key, found 2")
|
||||
}
|
||||
|
||||
// TestSigningKeyFileWithoutSecretKey signs with a file that holds only a
|
||||
// public key.
|
||||
func TestSigningKeyFileWithoutSecretKey(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
b := NewBuilder()
|
||||
b.SetSigningOptions(&SigningOptions{
|
||||
SecretKey: armoredPublicKeys(t, newTestKey(t, nil)),
|
||||
})
|
||||
require.ErrorIs(t, b.Build(context.Background(), io.Discard), errNoSecretKey)
|
||||
}
|
||||
|
||||
// TestCheckSigningKeyRefusesKeyThatCannotSign checks a key that can sign,
|
||||
// and keys that read and need no passphrase but cannot sign now: one that
|
||||
// expired in 2020 and one that has been revoked. CheckSigningKey must
|
||||
// refuse each of the two, as signing a manifest with it would fail.
|
||||
func TestCheckSigningKeyRefusesKeyThatCannotSign(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
require.NoError(t, CheckSigningKey(testSigningOptions(t)))
|
||||
|
||||
made := time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)
|
||||
expired := newTestKey(t, &packet.Config{
|
||||
Time: func() time.Time { return made },
|
||||
KeyLifetimeSecs: uint32((24 * time.Hour).Seconds()),
|
||||
})
|
||||
|
||||
revoked := newTestKey(t, nil)
|
||||
require.NoError(t, revoked.RevokeKey(packet.KeyRetired, "", nil))
|
||||
|
||||
for name, key := range map[string]*openpgp.Entity{
|
||||
"expired": expired,
|
||||
"revoked": revoked,
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
assert.ErrorContains(t, CheckSigningKey(
|
||||
&SigningOptions{SecretKey: armoredSecretKeys(t, key)}),
|
||||
"signing key cannot sign")
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestSigningRefusesVersion6Key signs with an OpenPGP version 6 key, whose
|
||||
// fingerprint is 64 hex characters. mfer signs only with version 4 keys.
|
||||
func TestSigningRefusesVersion6Key(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
key, err := openpgp.NewEntity("MFER Test Key", "", "test@mfer.test",
|
||||
&packet.Config{V6Keys: true, Algorithm: packet.PubKeyAlgoEd25519})
|
||||
require.NoError(t, err)
|
||||
|
||||
secretKey := armoredSecretKeys(t, key)
|
||||
|
||||
_, err = SecretKeyIsProtected(secretKey)
|
||||
require.ErrorIs(t, err, errNotV4Key)
|
||||
|
||||
b := NewBuilder()
|
||||
b.SetSigningOptions(&SigningOptions{SecretKey: secretKey})
|
||||
require.ErrorIs(t, b.Build(context.Background(), io.Discard), errNotV4Key)
|
||||
}
|
||||
|
||||
// TestMalformedArmorIsNamed reads a signing key file, a signature and an
|
||||
// embedded public key block whose armor is malformed: a header line with
|
||||
// no colon, or no blank line after the BEGIN line. Each must fail saying
|
||||
// the armor is malformed.
|
||||
func TestMalformedArmorIsNamed(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
opts := testSigningOptions(t)
|
||||
manifest := signedTestManifest(t, opts)
|
||||
|
||||
for name, change := range map[string]func([]byte) []byte{
|
||||
"header line with no colon": func(block []byte) []byte {
|
||||
return bytes.Replace(block,
|
||||
[]byte("-----\n"), []byte("-----\nno colon\n"), 1)
|
||||
},
|
||||
"no blank line after the BEGIN line": func(block []byte) []byte {
|
||||
return bytes.Replace(block, []byte("-----\n\n"), []byte("-----\n"), 1)
|
||||
},
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
_, err := SecretKeyIsProtected(change(opts.SecretKey))
|
||||
require.ErrorIs(t, err, errMalformedArmor)
|
||||
|
||||
for _, changed := range [][]byte{
|
||||
rewriteOuter(t, manifest, func(outer *MFFileOuter) {
|
||||
outer.Signature = change(outer.GetSignature())
|
||||
}),
|
||||
rewriteOuter(t, manifest, func(outer *MFFileOuter) {
|
||||
outer.SigningPubKey = change(outer.GetSigningPubKey())
|
||||
}),
|
||||
} {
|
||||
_, err = NewManifestFromReader(bytes.NewReader(changed))
|
||||
require.ErrorIs(t, err, errMalformedArmor)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifySignature(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
key := newTestKey(t, nil)
|
||||
data := []byte("test data to sign and verify")
|
||||
|
||||
var armored, binary bytes.Buffer
|
||||
|
||||
require.NoError(t, openpgp.ArmoredDetachSign(&armored, key,
|
||||
bytes.NewReader(data), nil))
|
||||
require.NoError(t, openpgp.DetachSign(&binary, key, bytes.NewReader(data), nil))
|
||||
|
||||
pubKey, err := armoredPublicKey(key)
|
||||
require.NoError(t, err)
|
||||
|
||||
var binaryPubKey bytes.Buffer
|
||||
|
||||
require.NoError(t, key.Serialize(&binaryPubKey))
|
||||
|
||||
// Verifying names the key that made the signature, whether the
|
||||
// signature and key are armored or not.
|
||||
signer, err := verifySignature(data, armored.Bytes(), pubKey)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, fingerprint(key), signer)
|
||||
|
||||
signer, err = verifySignature(data, binary.Bytes(), binaryPubKey.Bytes())
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, fingerprint(key), signer)
|
||||
|
||||
// A signature over other data is bad.
|
||||
_, err = verifySignature([]byte("different data"), armored.Bytes(), pubKey)
|
||||
require.Error(t, err)
|
||||
|
||||
// A public key that is not one cannot verify anything.
|
||||
_, err = verifySignature(data, armored.Bytes(), []byte("not a public key"))
|
||||
assert.Error(t, err)
|
||||
}
|
||||
|
||||
// TestVerifySignatureKeyExpiredSince verifies two signatures made in 2020
|
||||
// by a key that expired a day after it was made. A signature with no
|
||||
// expiry time is still good. One that expired an hour after it was made is
|
||||
// refused, although the key has expired too.
|
||||
func TestVerifySignatureKeyExpiredSince(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
made := time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)
|
||||
config := &packet.Config{
|
||||
Time: func() time.Time { return made },
|
||||
KeyLifetimeSecs: uint32((24 * time.Hour).Seconds()),
|
||||
}
|
||||
key := newTestKey(t, config)
|
||||
data := []byte("signed in 2020")
|
||||
|
||||
var sig, expiringSig bytes.Buffer
|
||||
|
||||
require.NoError(t, openpgp.ArmoredDetachSign(&sig, key, bytes.NewReader(data), config))
|
||||
require.NoError(t, openpgp.ArmoredDetachSign(&expiringSig, key,
|
||||
bytes.NewReader(data), &packet.Config{
|
||||
Time: config.Time,
|
||||
SigLifetimeSecs: uint32(time.Hour.Seconds()),
|
||||
}))
|
||||
|
||||
pubKey, err := armoredPublicKey(key)
|
||||
require.NoError(t, err)
|
||||
|
||||
signer, err := verifySignature(data, sig.Bytes(), pubKey)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, fingerprint(key), signer)
|
||||
|
||||
_, err = verifySignature(data, expiringSig.Bytes(), pubKey)
|
||||
require.ErrorIs(t, err, pgperrors.ErrSignatureExpired)
|
||||
}
|
||||
|
||||
func TestManifestSignatureVerification(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// Parse the manifest - signature should be verified during load
|
||||
manifest, err := NewManifestFromReader(bytes.NewReader(
|
||||
signedTestManifest(t, testSigningOptions(t))))
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, manifest)
|
||||
|
||||
// Signature should be present and valid
|
||||
assert.NotEmpty(t, manifest.pbOuter.GetSignature())
|
||||
}
|
||||
|
||||
func TestManifestTamperedSignatureFails(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// Change one character of the signature's base64 body, which starts
|
||||
// after the blank line that ends the armor headers.
|
||||
data := rewriteOuter(t, signedTestManifest(t, testSigningOptions(t)),
|
||||
func(outer *MFFileOuter) {
|
||||
sig := outer.GetSignature()
|
||||
i := bytes.Index(sig, []byte("\n\n")) + len("\n\n") + 20
|
||||
|
||||
sig[i]++
|
||||
})
|
||||
|
||||
// Try to load the tampered manifest - should fail
|
||||
_, err := NewManifestFromReader(bytes.NewReader(data))
|
||||
assert.Error(t, err)
|
||||
}
|
||||
|
||||
// TestManifestSignedByGPGLoads loads the signed seed of
|
||||
// FuzzNewManifestFromReader, a manifest signed with gpg before mfer signed
|
||||
// and verified manifests itself.
|
||||
func TestManifestSignedByGPGLoads(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
seed, err := os.ReadFile(filepath.Join(
|
||||
"testdata", "fuzz", "FuzzNewManifestFromReader", "signed"))
|
||||
require.NoError(t, err)
|
||||
|
||||
// After its header line the seed holds the manifest as []byte("...").
|
||||
_, quoted, found := strings.Cut(string(seed), "[]byte(")
|
||||
require.True(t, found)
|
||||
|
||||
manifest, err := strconv.Unquote(
|
||||
strings.TrimSuffix(strings.TrimSpace(quoted), ")"))
|
||||
require.NoError(t, err)
|
||||
|
||||
m, err := NewManifestFromReader(strings.NewReader(manifest))
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "4F562BFB863FDC6B51B4EE88872A51176CEF23AE",
|
||||
string(m.pbOuter.GetSigner()))
|
||||
}
|
||||
|
||||
// TestManifestRefusesSecondEmbeddedKey loads manifests whose embedded
|
||||
// public key block holds another key besides the key that signed it: as a
|
||||
// public key, as a secret key with no user ID, which openpgp.ReadKeyRing
|
||||
// skips, and written as a subkey packet at the start of the block, which
|
||||
// openpgp.ReadKeyRing reads as a primary key. Loading must refuse each,
|
||||
// although the signature is good and the signer field names the key that
|
||||
// made it.
|
||||
func TestManifestRefusesSecondEmbeddedKey(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
other := newTestKey(t, nil)
|
||||
signer := newTestKey(t, nil)
|
||||
manifest := signedTestManifest(t,
|
||||
&SigningOptions{SecretKey: armoredSecretKeys(t, signer)})
|
||||
|
||||
otherWithoutUserID := newTestKey(t, nil)
|
||||
otherWithoutUserID.Identities = map[string]*openpgp.Identity{}
|
||||
|
||||
otherAsSubkey := newTestKey(t, nil)
|
||||
otherAsSubkey.PrimaryKey.IsSubkey = true
|
||||
|
||||
for name, block := range map[string][]byte{
|
||||
"public key": armoredPublicKeys(t, other, signer),
|
||||
"secret key without user ID": armoredSecretKeys(t, signer, otherWithoutUserID),
|
||||
"subkey packet first": armoredPublicKeys(t, otherAsSubkey, signer),
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
embedded := rewriteOuter(t, manifest, func(outer *MFFileOuter) {
|
||||
outer.SigningPubKey = block
|
||||
})
|
||||
|
||||
_, err := NewManifestFromReader(bytes.NewReader(embedded))
|
||||
require.ErrorIs(t, err, errKeyCount)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestManifestRefusesSecondEmbeddedKeyWithoutUserID loads a manifest whose
|
||||
// embedded public key block holds, before the key that signed it, another
|
||||
// key with no user ID, which openpgp.ReadKeyRing skips. Loading must
|
||||
// refuse it: the block holds two keys.
|
||||
func TestManifestRefusesSecondEmbeddedKeyWithoutUserID(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
other := newTestKey(t, nil)
|
||||
other.Identities = map[string]*openpgp.Identity{}
|
||||
|
||||
signer := newTestKey(t, nil)
|
||||
|
||||
manifest := rewriteOuter(t, signedTestManifest(t,
|
||||
&SigningOptions{SecretKey: armoredSecretKeys(t, signer)}),
|
||||
func(outer *MFFileOuter) {
|
||||
outer.SigningPubKey = armoredPublicKeys(t, other, signer)
|
||||
})
|
||||
|
||||
_, err := NewManifestFromReader(bytes.NewReader(manifest))
|
||||
require.ErrorIs(t, err, errKeyCount)
|
||||
}
|
||||
|
||||
// dsaKeyPacket returns a public key packet holding a DSA key, or a public
|
||||
// subkey packet when isSubkey. Its numbers are not a working key: loading
|
||||
// must refuse the packet before it uses them.
|
||||
func dsaKeyPacket(t *testing.T, isSubkey bool) []byte {
|
||||
t.Helper()
|
||||
|
||||
key := packet.NewDSAPublicKey(time.Now(), &dsa.PublicKey{
|
||||
P: big.NewInt(23), Q: big.NewInt(11), G: big.NewInt(4), Y: big.NewInt(8),
|
||||
})
|
||||
key.IsSubkey = isSubkey
|
||||
|
||||
var buf bytes.Buffer
|
||||
|
||||
require.NoError(t, key.Serialize(&buf))
|
||||
|
||||
return buf.Bytes()
|
||||
}
|
||||
|
||||
// TestManifestRefusesDSAKey loads manifests whose embedded public key
|
||||
// block holds a DSA key: alone, or as a subkey after the key that signed
|
||||
// the manifest. openpgp.ReadKeyRing checks a key's self-signatures, which
|
||||
// for a DSA key with very large numbers takes minutes each. Loading must
|
||||
// refuse each block before that.
|
||||
func TestManifestRefusesDSAKey(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
signer := newTestKey(t, nil)
|
||||
manifest := signedTestManifest(t,
|
||||
&SigningOptions{SecretKey: armoredSecretKeys(t, signer)})
|
||||
|
||||
var signerKey bytes.Buffer
|
||||
|
||||
require.NoError(t, signer.Serialize(&signerKey))
|
||||
|
||||
for name, block := range map[string][]byte{
|
||||
"DSA key": dsaKeyPacket(t, false),
|
||||
"DSA subkey": slices.Concat(signerKey.Bytes(), dsaKeyPacket(t, true)),
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
embedded := rewriteOuter(t, manifest, func(outer *MFFileOuter) {
|
||||
outer.SigningPubKey = block
|
||||
})
|
||||
|
||||
_, err := NewManifestFromReader(bytes.NewReader(embedded))
|
||||
require.ErrorIs(t, err, errDSAKey)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// elGamalSecretSubkeyPacket returns a secret subkey packet holding an
|
||||
// ElGamal key. Its numbers are not a working key: loading must refuse the
|
||||
// packet before it uses them.
|
||||
func elGamalSecretSubkeyPacket(t *testing.T) []byte {
|
||||
t.Helper()
|
||||
|
||||
key := packet.NewElGamalPrivateKey(time.Now(), &elgamal.PrivateKey{
|
||||
P: big.NewInt(23), G: big.NewInt(4), Y: big.NewInt(8), X: big.NewInt(3),
|
||||
})
|
||||
key.IsSubkey = true
|
||||
|
||||
var buf bytes.Buffer
|
||||
|
||||
require.NoError(t, key.Serialize(&buf))
|
||||
|
||||
return buf.Bytes()
|
||||
}
|
||||
|
||||
// TestManifestRefusesSecretKey loads manifests whose embedded public key
|
||||
// block holds a secret key: the key that signed the manifest with its
|
||||
// secret key, or its public key followed by an ElGamal secret subkey.
|
||||
// openpgp.ReadKeyRing checks the numbers of every secret key it reads,
|
||||
// which for an ElGamal key with a very large prime takes minutes. Loading
|
||||
// must refuse each block before that.
|
||||
func TestManifestRefusesSecretKey(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
signer := newTestKey(t, nil)
|
||||
manifest := signedTestManifest(t,
|
||||
&SigningOptions{SecretKey: armoredSecretKeys(t, signer)})
|
||||
|
||||
var signerKey bytes.Buffer
|
||||
|
||||
require.NoError(t, signer.Serialize(&signerKey))
|
||||
|
||||
for _, block := range [][]byte{
|
||||
armoredSecretKeys(t, signer),
|
||||
slices.Concat(signerKey.Bytes(), elGamalSecretSubkeyPacket(t)),
|
||||
} {
|
||||
embedded := rewriteOuter(t, manifest, func(outer *MFFileOuter) {
|
||||
outer.SigningPubKey = block
|
||||
})
|
||||
|
||||
_, err := NewManifestFromReader(bytes.NewReader(embedded))
|
||||
require.ErrorIs(t, err, errSecretKey)
|
||||
}
|
||||
}
|
||||
|
||||
// TestManifestRefusesTwoSignatures loads a manifest whose signature field
|
||||
// holds its good signature twice, not armored. Loading must refuse it.
|
||||
func TestManifestRefusesTwoSignatures(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
manifest := rewriteOuter(t, signedTestManifest(t, testSigningOptions(t)),
|
||||
func(outer *MFFileOuter) {
|
||||
sig, err := dearmor(outer.GetSignature())
|
||||
require.NoError(t, err)
|
||||
|
||||
outer.Signature = slices.Concat(sig, sig)
|
||||
})
|
||||
|
||||
_, err := NewManifestFromReader(bytes.NewReader(manifest))
|
||||
require.ErrorIs(t, err, errNotOneSignature)
|
||||
}
|
||||
|
||||
// TestManifestRefusesFieldNotOneArmoredBlock loads manifests whose
|
||||
// signature or embedded public key block holds its good armored block with
|
||||
// something else: a second armored block, text after the END line, or many
|
||||
// END lines before the block. Decoding the block once for each END line
|
||||
// before it would take time and memory that grow with the square of the
|
||||
// field's size. Loading must refuse each.
|
||||
func TestManifestRefusesFieldNotOneArmoredBlock(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
manifest := signedTestManifest(t, testSigningOptions(t))
|
||||
|
||||
for name, change := range map[string]func([]byte) []byte{
|
||||
"second armored block": func(block []byte) []byte {
|
||||
return joinArmored(block, block)
|
||||
},
|
||||
"text after the END line": func(block []byte) []byte {
|
||||
return slices.Concat(block, []byte("\nmore text\n"))
|
||||
},
|
||||
"END lines before the block": func(block []byte) []byte {
|
||||
return slices.Concat(
|
||||
[]byte(strings.Repeat(armorEnd+"\n", 1000)), block)
|
||||
},
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, changed := range [][]byte{
|
||||
rewriteOuter(t, manifest, func(outer *MFFileOuter) {
|
||||
outer.Signature = change(outer.GetSignature())
|
||||
}),
|
||||
rewriteOuter(t, manifest, func(outer *MFFileOuter) {
|
||||
outer.SigningPubKey = change(outer.GetSigningPubKey())
|
||||
}),
|
||||
} {
|
||||
_, err := NewManifestFromReader(bytes.NewReader(changed))
|
||||
require.ErrorIs(t, err, errNotOneArmoredBlock)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestManifestSignedWithSubkey signs with a key that has a signing subkey,
|
||||
// which signs in place of the primary key. The manifest must load, with
|
||||
// the primary key's fingerprint as signer.
|
||||
func TestManifestSignedWithSubkey(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
key := newTestKey(t, nil)
|
||||
require.NoError(t, key.AddSigningSubkey(
|
||||
&packet.Config{Algorithm: packet.PubKeyAlgoEdDSA}))
|
||||
|
||||
m, err := NewManifestFromReader(bytes.NewReader(signedTestManifest(t,
|
||||
&SigningOptions{SecretKey: armoredSecretKeys(t, key)})))
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, fingerprint(key), string(m.pbOuter.GetSigner()))
|
||||
|
||||
block, err := armor.Decode(bytes.NewReader(m.pbOuter.GetSignature()))
|
||||
require.NoError(t, err)
|
||||
|
||||
p, err := packet.Read(block.Body)
|
||||
require.NoError(t, err)
|
||||
|
||||
sig, ok := p.(*packet.Signature)
|
||||
require.True(t, ok)
|
||||
|
||||
subkey := key.Subkeys[len(key.Subkeys)-1].PublicKey
|
||||
assert.Equal(t, subkey.KeyId, *sig.IssuerKeyId,
|
||||
"the signing subkey made the signature")
|
||||
}
|
||||
|
||||
// TestManifestRefusesSignerOtherThanSigningKey loads a manifest whose
|
||||
// signer field names a key other than the one that made the signature.
|
||||
func TestManifestRefusesSignerOtherThanSigningKey(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
manifest := rewriteOuter(t, signedTestManifest(t, testSigningOptions(t)),
|
||||
func(outer *MFFileOuter) {
|
||||
outer.Signer = []byte(strings.Repeat("A", len(outer.GetSigner())))
|
||||
})
|
||||
|
||||
_, err := NewManifestFromReader(bytes.NewReader(manifest))
|
||||
require.ErrorIs(t, err, errSignerNotSigningKey)
|
||||
}
|
||||
|
||||
func TestBuilderWithoutSigning(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// Create a builder without signing options
|
||||
b := NewBuilder()
|
||||
|
||||
// Add a test file
|
||||
content := []byte("test file content")
|
||||
reader := bytes.NewReader(content)
|
||||
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, 0, reader, nil)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Build the manifest
|
||||
var buf bytes.Buffer
|
||||
|
||||
err = b.Build(context.Background(), &buf)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Parse the manifest and verify signature fields are empty
|
||||
manifest, err := NewManifestFromReader(&buf)
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, manifest.pbOuter)
|
||||
|
||||
assert.Empty(t, manifest.pbOuter.GetSignature(),
|
||||
"signature should be empty when not signing")
|
||||
assert.Empty(t, manifest.pbOuter.GetSigner(),
|
||||
"signer should be empty when not signing")
|
||||
assert.Empty(t, manifest.pbOuter.GetSigningPubKey(),
|
||||
"signing public key should be empty when not signing")
|
||||
}
|
||||
|
||||
// TestBuildPassesContextToSigning checks that Build does not sign once the
|
||||
// context given to it has ended.
|
||||
func TestBuildPassesContextToSigning(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
b := NewBuilder()
|
||||
b.SetSigningOptions(&SigningOptions{SecretKey: []byte("any")})
|
||||
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
|
||||
require.ErrorIs(t, b.Build(ctx, io.Discard), context.Canceled)
|
||||
}
|
||||
@@ -1,123 +0,0 @@
|
||||
//go:build unix
|
||||
|
||||
//nolint:testpackage // white-box tests exercise unexported internals
|
||||
package mfer
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"syscall"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/spf13/afero"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// pipeDeadline is how long a test lets a run go before it opens the test's
|
||||
// named pipes for writing.
|
||||
const pipeDeadline = 5 * time.Second
|
||||
|
||||
// makePipe makes a named pipe at path. Opening a named pipe for reading
|
||||
// blocks until it is opened for writing, so once pipeDeadline has passed,
|
||||
// unless the test has ended, makePipe opens it for writing and closes it
|
||||
// again. A run that wrongly reads the pipe then ends, and the test fails on
|
||||
// what it reports instead of hanging. That lets one read through, so each
|
||||
// path a wrong run would read gets a pipe of its own.
|
||||
func makePipe(t *testing.T, path string) {
|
||||
t.Helper()
|
||||
|
||||
require.NoError(t, syscall.Mkfifo(path, 0o600))
|
||||
|
||||
timer := time.AfterFunc(pipeDeadline, func() {
|
||||
f, err := os.OpenFile(path, os.O_WRONLY, 0) //nolint:gosec // test-controlled path
|
||||
if err == nil {
|
||||
_ = f.Close()
|
||||
}
|
||||
})
|
||||
|
||||
t.Cleanup(func() { timer.Stop() })
|
||||
}
|
||||
|
||||
// TestScannerSkipsPipes scans a tree holding a named pipe and a symlink to
|
||||
// another one, following symlinks: the manifest lists the regular file only.
|
||||
func TestScannerSkipsPipes(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dir := t.TempDir()
|
||||
require.NoError(t, os.WriteFile(filepath.Join(dir, testFile1), []byte("a"), 0o600))
|
||||
makePipe(t, filepath.Join(dir, "pipe"))
|
||||
|
||||
// The symlink points outside the tree, so no pipe is read twice.
|
||||
target := filepath.Join(t.TempDir(), "pipe")
|
||||
makePipe(t, target)
|
||||
require.NoError(t, os.Symlink(target, filepath.Join(dir, "link")))
|
||||
|
||||
s := NewScannerWithOptions(&ScannerOptions{FollowSymLinks: true})
|
||||
require.NoError(t, s.EnumeratePath(dir, nil))
|
||||
|
||||
var buf bytes.Buffer
|
||||
require.NoError(t, s.ToManifest(context.Background(), &buf, nil))
|
||||
|
||||
m, err := NewManifestFromReader(&buf)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, m.Files(), 1)
|
||||
assert.Equal(t, testFile1, m.Files()[0].GetPath())
|
||||
}
|
||||
|
||||
// TestCheckPipes checks a tree against a manifest that lists, as an empty
|
||||
// file, a path that is a named pipe on disk. The tree holds another pipe the
|
||||
// manifest does not list. check reports the listed pipe as an error without
|
||||
// reading it, and the search for extra files does not report the other.
|
||||
func TestCheckPipes(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dir := t.TempDir()
|
||||
manifestPath := filepath.Join(dir, "index.mf")
|
||||
content := []byte("in manifest")
|
||||
listedPipe := RelFilePath("listed-pipe")
|
||||
|
||||
fs := afero.NewOsFs()
|
||||
createTestManifest(t, fs, manifestPath, map[string][]byte{
|
||||
testFile1: content,
|
||||
string(listedPipe): {},
|
||||
})
|
||||
require.NoError(t, afero.WriteFile(fs, filepath.Join(dir, testFile1), content, 0o600))
|
||||
makePipe(t, filepath.Join(dir, string(listedPipe)))
|
||||
makePipe(t, filepath.Join(dir, "unlisted-pipe"))
|
||||
|
||||
chk, err := NewChecker(&CheckerOptions{
|
||||
ManifestPath: manifestPath,
|
||||
BasePath: dir,
|
||||
Fs: fs,
|
||||
})
|
||||
require.NoError(t, err)
|
||||
|
||||
results := make(chan Result, 10)
|
||||
require.NoError(t, chk.Check(context.Background(), results, nil))
|
||||
|
||||
byPath := map[RelFilePath]Result{}
|
||||
for r := range results {
|
||||
byPath[r.Path] = r
|
||||
}
|
||||
|
||||
assert.Equal(t, StatusOK, byPath[testFile1].Status)
|
||||
assert.Equal(t, Result{
|
||||
Path: listedPipe,
|
||||
Status: StatusError,
|
||||
Message: "not a regular file",
|
||||
}, byPath[listedPipe])
|
||||
|
||||
extras := make(chan Result, 10)
|
||||
require.NoError(t, chk.FindExtraFiles(context.Background(), extras))
|
||||
|
||||
var extraPaths []RelFilePath
|
||||
for r := range extras {
|
||||
extraPaths = append(extraPaths, r.Path)
|
||||
}
|
||||
|
||||
assert.Empty(t, extraPaths)
|
||||
}
|
||||
+17
-34
@@ -58,7 +58,7 @@ type ScannerOptions struct {
|
||||
IncludePermissions bool
|
||||
// Fs is the filesystem to use, defaults to OsFs if nil.
|
||||
Fs afero.Fs
|
||||
// SigningOptions holds the key to sign with (nil = no signing).
|
||||
// SigningOptions holds GPG signing options (nil = no signing).
|
||||
SigningOptions *SigningOptions
|
||||
// Seed, if set, derives a deterministic UUID from this seed.
|
||||
Seed string
|
||||
@@ -139,20 +139,28 @@ func (s *Scanner) EnumerateFile(filePath string) error {
|
||||
return s.enumerateFileWithInfo(filepath.Base(abs), basePath, info, nil)
|
||||
}
|
||||
|
||||
// EnumeratePath adds inputPath, a directory or a file, to the scanner as
|
||||
// EnumeratePaths adds each of its paths.
|
||||
// EnumeratePath walks a directory path and adds all files to the scanner.
|
||||
// If progress is non-nil, status updates are sent as files are discovered.
|
||||
// The progress channel is closed when the method returns.
|
||||
func (s *Scanner) EnumeratePath(
|
||||
inputPath string,
|
||||
progress chan<- EnumerateStatus,
|
||||
) error {
|
||||
return s.EnumeratePaths(progress, inputPath)
|
||||
if progress != nil {
|
||||
defer close(progress)
|
||||
}
|
||||
|
||||
abs, err := filepath.Abs(inputPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
afs := afero.NewReadOnlyFs(afero.NewBasePathFs(s.fs, abs))
|
||||
|
||||
return s.enumerateFS(afs, abs, progress)
|
||||
}
|
||||
|
||||
// EnumeratePaths adds to the scanner the files under each directory path,
|
||||
// listed by their paths under it, and each file path, listed by its name
|
||||
// as EnumerateFile lists it.
|
||||
// EnumeratePaths walks multiple directory paths and adds all files to the scanner.
|
||||
// If progress is non-nil, status updates are sent as files are discovered.
|
||||
// The progress channel is closed when the method returns.
|
||||
func (s *Scanner) EnumeratePaths(
|
||||
@@ -169,27 +177,9 @@ func (s *Scanner) EnumeratePaths(
|
||||
return err
|
||||
}
|
||||
|
||||
info, err := s.fs.Stat(abs)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if info.IsDir() {
|
||||
// The walk does not follow a symlink at its top, so a directory
|
||||
// named through one is resolved first. If that fails, the
|
||||
// directory is walked as named and the walk reports the problem.
|
||||
resolved, evalErr := filepath.EvalSymlinks(abs)
|
||||
if evalErr == nil {
|
||||
abs = resolved
|
||||
}
|
||||
|
||||
afs := afero.NewReadOnlyFs(afero.NewBasePathFs(s.fs, abs))
|
||||
err = s.enumerateFS(afs, abs, progress)
|
||||
} else {
|
||||
err = s.enumerateFileWithInfo(
|
||||
filepath.Base(abs), filepath.Dir(abs), info, progress)
|
||||
}
|
||||
afs := afero.NewReadOnlyFs(afero.NewBasePathFs(s.fs, abs))
|
||||
|
||||
err = s.enumerateFS(afs, abs, progress)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -474,13 +464,6 @@ func (s *Scanner) enumerateFileWithInfo(
|
||||
info = realInfo
|
||||
}
|
||||
|
||||
// Opening a named pipe can block, and reading a device may never end.
|
||||
if !info.Mode().IsRegular() {
|
||||
log.Verbosef("skipping %s: not a regular file", cleanPath)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
for _, excluded := range s.excluded {
|
||||
if os.SameFile(info, excluded) {
|
||||
return nil
|
||||
|
||||
@@ -118,27 +118,6 @@ func TestScannerEnumeratePathWithProgress(t *testing.T) {
|
||||
assert.Equal(t, FileSize(6), final.BytesFound)
|
||||
}
|
||||
|
||||
// TestScannerEnumeratePathFile gives EnumeratePath a file: it is listed
|
||||
// by its name, as EnumerateFile lists it, and the manifest can be built.
|
||||
func TestScannerEnumeratePathFile(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
fs := afero.NewMemMapFs()
|
||||
require.NoError(t, fs.MkdirAll("/dir", 0o755))
|
||||
require.NoError(t, afero.WriteFile(fs, "/dir/one.txt", []byte("1"), 0o644))
|
||||
|
||||
s := NewScannerWithOptions(&ScannerOptions{Fs: fs})
|
||||
require.NoError(t, s.EnumeratePath("/dir/one.txt", nil))
|
||||
|
||||
var buf bytes.Buffer
|
||||
require.NoError(t, s.ToManifest(context.Background(), &buf, nil))
|
||||
|
||||
m, err := NewManifestFromReader(&buf)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, m.Files(), 1)
|
||||
assert.Equal(t, "one.txt", m.Files()[0].GetPath())
|
||||
}
|
||||
|
||||
func TestScannerEnumeratePaths(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -155,28 +134,6 @@ func TestScannerEnumeratePaths(t *testing.T) {
|
||||
assert.Equal(t, FileCount(2), s.FileCount())
|
||||
}
|
||||
|
||||
// TestScannerEnumeratePathsFile gives EnumeratePaths a directory and a
|
||||
// file: the file is listed by its name, as EnumerateFile lists it.
|
||||
func TestScannerEnumeratePathsFile(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
fs := afero.NewMemMapFs()
|
||||
require.NoError(t, fs.MkdirAll("/dir/sub", 0o755))
|
||||
require.NoError(t, fs.MkdirAll("/other", 0o755))
|
||||
require.NoError(t, afero.WriteFile(fs, "/dir/sub/one.txt", []byte("1"), 0o644))
|
||||
require.NoError(t, afero.WriteFile(fs, "/other/two.txt", []byte("2"), 0o644))
|
||||
|
||||
s := NewScannerWithOptions(&ScannerOptions{Fs: fs})
|
||||
require.NoError(t, s.EnumeratePaths(nil, "/dir", "/other/two.txt"))
|
||||
|
||||
paths := make([]RelFilePath, 0, s.FileCount())
|
||||
for _, f := range s.Files() {
|
||||
paths = append(paths, f.Path)
|
||||
}
|
||||
|
||||
assert.Equal(t, []RelFilePath{"sub/one.txt", "two.txt"}, paths)
|
||||
}
|
||||
|
||||
func TestScannerExcludeDotfiles(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
+14
-20
@@ -7,11 +7,9 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"math"
|
||||
"strings"
|
||||
"time"
|
||||
"uuid"
|
||||
|
||||
"github.com/ProtonMail/go-crypto/openpgp"
|
||||
"github.com/klauspost/compress/zstd"
|
||||
"google.golang.org/protobuf/proto"
|
||||
)
|
||||
@@ -135,48 +133,44 @@ func (m *manifest) generateOuter(ctx context.Context) error {
|
||||
}
|
||||
|
||||
// Sign the manifest if signing options are provided
|
||||
if m.signingOptions != nil {
|
||||
if m.signingOptions != nil && m.signingOptions.KeyID != "" {
|
||||
return m.signOuter(ctx)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// signOuter signs the outer message with the secret key in the signing
|
||||
// options and embeds the signature, the key's fingerprint and its public
|
||||
// key.
|
||||
// signOuter signs the outer message with the configured GPG key and
|
||||
// embeds the signature, signer fingerprint, and public key. The signer
|
||||
// and public key are those of the key gpg reports it signed with, so that
|
||||
// a key ID matching more than one key cannot name or embed another key.
|
||||
func (m *manifest) signOuter(ctx context.Context) error {
|
||||
// Unlocking a protected key can take a while; do not start once ctx
|
||||
// has ended.
|
||||
err := ctx.Err()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
sigString, err := m.signatureString()
|
||||
if err != nil {
|
||||
return fmt.Errorf("build signature string: %w", err)
|
||||
}
|
||||
|
||||
key, err := readSigningKey(m.signingOptions)
|
||||
sig, signingKey, err := gpgSign(ctx, []byte(sigString), m.signingOptions.KeyID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
var sig bytes.Buffer
|
||||
m.pbOuter.Signature = sig
|
||||
|
||||
err = openpgp.ArmoredDetachSign(&sig, key, strings.NewReader(sigString), nil)
|
||||
// Listing the signing key, a subkey's included, puts its primary key's
|
||||
// fingerprint first.
|
||||
fingerprint, err := gpgGetKeyFingerprint(ctx, GPGKeyID(signingKey))
|
||||
if err != nil {
|
||||
return fmt.Errorf("sign manifest: %w", err)
|
||||
return err
|
||||
}
|
||||
|
||||
pubKey, err := armoredPublicKey(key)
|
||||
m.pbOuter.Signer = fingerprint
|
||||
|
||||
pubKey, err := gpgExportPublicKey(ctx, GPGKeyID(fingerprint))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
m.pbOuter.Signature = sig.Bytes()
|
||||
m.pbOuter.Signer = []byte(fingerprint(key))
|
||||
m.pbOuter.SigningPubKey = pubKey
|
||||
|
||||
return nil
|
||||
|
||||
Vendored
BIN
Binary file not shown.
Reference in New Issue
Block a user