9 Commits
Author SHA1 Message Date
clawbot e35cd4a045 Resolve check and freshen paths against the manifest's directory (closes #177)
check / check (push) Waiting to run
Without --base, check and freshen now look for a manifest's files in the
directory that holds it, the file named or the one found in a directory
argument, instead of the current directory. So `mfer check /media/drive`
checks a drive against its own index.mf from anywhere. check of a manifest
given by URL still uses the current directory, and --base still overrides,
even when it names the current directory. The --base help text of both
commands and the README's Tool Examples state the default; the README gains
a freshen entry for this, which also names the hidden files and symlinks
freshen leaves out by default and the flags that include them.

Model: opus-5-5
2026-10-07 17:28:38 +02:00
clawbot c0b099cc48 Make error message wording consistent (closes #165)
check / check (push) Waiting to run
Error messages in mfer/ and internal/cli/ are lowercase except names and
acronyms, carry no "failed to" or command-name prefix, and each wrap names
only the operation and thing the wrapped error does not already name, so a
stacked message names what failed once. Wraps around errors that already
name their operation and path (os and afero path errors, url.Error, the
builder's path errors, the gpg helpers' own errors) are dropped. gpg's
stderr is appended to a gpg failure, and to the error for a signing key gpg
did not report, only when gpg wrote some. errHTTPStatus reads "unexpected
HTTP status"; both inner-not-set sentinels read "inner message not set".
No sentinel, errors.Is result or exit status changes.

Model: opus-5-5
2026-10-07 17:25:41 +02:00
clawbot dce5e050c3 Link docs/FORMAT.md as the format specification in the README (closes #166)
check / check (push) Waiting to run
The README's opening paragraph called mfer/mf.proto the format specification
and linked to it. It now links docs/FORMAT.md, which is the specification, and
names mfer/mf.proto as the protobuf schema that document refers to for field
numbers and types. The link is relative, as the README's link to
REPO_POLICIES.md is, because docs/FORMAT.md is not on main yet.

Model: opus-5-5
2026-10-07 15:59:10 +02:00
clawbot 4fe1ff2fe1 Refuse a path listed twice in a manifest (closes #170)
check / check (push) Waiting to run
gen given arguments whose files share a path, such as gen a b with a.txt
in both, or gen . ., now fails while listing the files, before hashing
any, naming the path and both files. Builder.AddFile and
Builder.AddFileWithHash refuse a path already added. Loading refuses a
manifest that lists a path twice, compared byte for byte; fetch keeps
its own letter-case check. The Path Rules in docs/FORMAT.md say each
path appears at most once. The decode-size test listed one path 1000
times; each entry now has its own path of the same length.

Model: opus-5-5
2026-10-07 15:28:57 +02:00
clawbot 01ff67a38e Refuse a manifest whose inner message version is not one (closes #169)
check / check (push) Waiting to run
Loading a manifest checked only the outer message's version, so an
inner message of version 0 or a later version loaded as if it were
version one, although docs/FORMAT.md requires VERSION_ONE in both.
deserializeInner now refuses any other inner version with the same
error as an unknown outer version. Two existing tests built inner
messages with no version and expected them to load; they now write
version one.

Model: opus-5-5
2026-10-07 15:25:47 +02:00
clawbot f663f4242d Limit how much fetch and check read for a manifest or a file (closes #168)
check / check (push) Waiting to run
NewManifestFromReader reads at most one byte past MaxManifestSize, a new
constant of 258 MiB: the 256 MiB decompressed limit grown by zstd's worst
case of 1/256, plus 1 MiB for the signature, the signing key and the
other outer fields. It refuses a larger manifest. fetch, and check given
a URL, stop downloading a manifest one byte past the same size and report
it as too large; tests lower that size to keep their memory small. fetch
stops reading a file one byte past its listed size, so a longer body ends
in the size mismatch at once instead of filling the disk. docs/FORMAT.md
states the limit and gives the decompressed limit as 256 MiB, the size
the code uses.

Model: opus-5-5
2026-10-07 14:25:45 +02:00
clawbot 0762a728d4 Compare --require-signature with the key that signed (closes #167)
check / check (push) Waiting to run
check and fetch --require-signature compared the required fingerprint
with the first key in the manifest's embedded public key block, while
gpg accepted a good signature by any key in that block.

Loading a signed manifest now refuses one whose embedded block holds
more than one primary key, counted as gpg reads the block, or whose
signer field is not the primary key fingerprint gpg reports for the
signature. --require-signature compares with the signer field, which
loading has checked. Signing names and embeds the key gpg reports it
signed with, so a key ID matching several keys still writes a manifest
that loads. docs/FORMAT.md states what a verifier checks.

Model: opus-5-5
2026-10-07 13:59:17 +02:00
clawbot 2a174e3ba2 Raise Go to the latest release, update dependencies, use the standard library uuid, add a vulnerability check (closes #102)
check / check (push) Failing after 3s
Go 1.27.1 in go.mod and in the Dockerfile's test and build images.
Every module go.mod requires is at its current release; protoc-gen-go
follows protobuf to v1.36.12 and mf.pb.go is regenerated. The standard
library uuid package replaces github.com/google/uuid; FromBytes could
only fail on a length validateUUID already checks, so that call and its
unreachable error are gone. make vulncheck runs govulncheck v1.8.0,
installed with go install at its release commit, in a vulncheck stage
of the Dockerfile; script/check does not run it. The newer go directive
switches on lint checks for strings.SplitSeq and t.Chdir, now used. A
new test pins the bytes of a seeded manifest written by an mfer built
before this change.

Model: opus-5-5
2026-10-06 20:26:15 +02:00
clawbot 2a270b40c5 Record file mode in the manifest, 0000 unless asked (closes #161)
check / check (push) Failing after 4s
MFFilePath gains mode (field 304): a file's permission bits, 0777 at
most, or 0000, meaning none recorded. gen and freshen record real modes
only with --include-permissions (ScannerOptions.IncludePermissions); the
builder keeps only mode.Perm(), so setuid, setgid and sticky are never
written. list -l and export show the mode in octal. check reports
MODE_MISMATCH for a recorded mode other than 0000 the file lacks. fetch
refuses a manifest with a mode above 0777 before requesting any file,
sets only the permission bits of each recorded mode on the files it
writes, and downloads again a present file whose mode differs. The decoding-cost bound counts a file
entry at 176 bytes, up from 160.

Model: opus-5-5
2026-10-06 11:43:18 +02:00
43 changed files with 2083 additions and 1007 deletions
+17 -5
View File
@@ -11,8 +11,8 @@ RUN golangci-lint run --config .golangci.yml ./...
# Test phase. -race needs cgo and so a C compiler, which the Debian Go # Test phase. -race needs cgo and so a C compiler, which the Debian Go
# image ships and the alpine one does not. # image ships and the alpine one does not.
# golang:1.23.12, 2026-03-14 # golang:1.27.1, 2026-10-06
FROM golang@sha256:60deed95d3888cc5e4d9ff8a10c54e5edc008c6ae3fba6187be6fb592e19e8c0 AS test FROM golang@sha256:1e93e00a31255c07e9a34c4207f3006e1501730c5323697cee7dfb827fdae44c AS test
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
@@ -21,12 +21,24 @@ RUN go test -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \ { echo "--- Rerunning with -v for details ---"; \
go test -timeout 90s -race -v ./...; exit 1; } go test -timeout 90s -race -v ./...; exit 1; }
# Build stage. Nothing is wanted from either phase above; the copies # Vulnerability check, built only by script/vulncheck (make vulncheck).
# No stage depends on it, so the image build does not run it.
# golang:1.27.1, 2026-10-06
FROM golang@sha256:1e93e00a31255c07e9a34c4207f3006e1501730c5323697cee7dfb827fdae44c AS vulncheck
# govulncheck v1.8.0, 2026-10-06
RUN go install golang.org/x/vuln/cmd/govulncheck@709015412431dd2b5b28a53c06c70bc02d49074c
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN govulncheck ./...
# Build stage. Nothing is wanted from the lint or test phase; the copies
# are what make BuildKit build them first, so this stage cannot run # are what make BuildKit build them first, so this stage cannot run
# unless lint and test passed. The Debian Go image ships git, which the # unless lint and test passed. The Debian Go image ships git, which the
# version step below needs. # version step below needs.
# golang:1.23.12, 2026-03-14 # golang:1.27.1, 2026-10-06
FROM golang@sha256:60deed95d3888cc5e4d9ff8a10c54e5edc008c6ae3fba6187be6fb592e19e8c0 AS builder FROM golang@sha256:1e93e00a31255c07e9a34c4207f3006e1501730c5323697cee7dfb827fdae44c AS builder
COPY --from=lint /src/go.sum /dev/null COPY --from=lint /src/go.sum /dev/null
COPY --from=test /src/go.sum /dev/null COPY --from=test /src/go.sum /dev/null
# A tar-stream context keeps the sender's file owners, which git refuses. # A tar-stream context keeps the sender's file owners, which git refuses.
+4 -1
View File
@@ -1,4 +1,4 @@
.PHONY: bootstrap setup test lint fmt fmt-check check docker hooks build generate fuzz .PHONY: bootstrap setup test lint fmt fmt-check check docker hooks build generate fuzz vulncheck
# Makefile targets are thin shims; the implementations live in script/ # Makefile targets are thin shims; the implementations live in script/
# per the scripts-to-rule-them-all pattern (see the Entrypoints section # per the scripts-to-rule-them-all pattern (see the Entrypoints section
@@ -39,3 +39,6 @@ generate:
fuzz: fuzz:
@script/fuzz @script/fuzz
vulncheck:
@script/vulncheck
+37 -15
View File
@@ -9,8 +9,9 @@ downloading, streaming, and mirroring. It was first published in 2022. The
manifest files' data is serialized with Google's manifest files' data is serialized with Google's
[protobuf serialization format](https://developers.google.com/protocol-buffers). [protobuf serialization format](https://developers.google.com/protocol-buffers).
The structure of these files can be found The structure of these files can be found
[in the format specification](https://git.eeqj.de/sneak/mfer/src/branch/main/mfer/mf.proto) [in the format specification](docs/FORMAT.md), which refers to the protobuf
which is included in the [project repository](https://git.eeqj.de/sneak/mfer). schema `mfer/mf.proto` for exact field numbers and types. Both are included in
the [project repository](https://git.eeqj.de/sneak/mfer).
The current version is pre-1.0 and while the repo was published in 2022, there The current version is pre-1.0 and while the repo was published in 2022, there
has not yet been any versioned release. [SemVer](https://semver.org) will be has not yet been any versioned release. [SemVer](https://semver.org) will be
@@ -23,7 +24,7 @@ javascript library is planned.
# Getting Started # Getting Started
`mfer` builds from source with a Go 1.23+ toolchain. The generated protobuf code `mfer` builds from source with Go 1.27.1 or later. The generated protobuf code
is committed, so no `protoc` toolchain is required: is committed, so no `protoc` toolchain is required:
```sh ```sh
@@ -70,7 +71,7 @@ provide:
- `script/bootstrap` — install all dependencies, idempotently: Go and the - `script/bootstrap` — install all dependencies, idempotently: Go and the
modules of `go.mod`; node (the version `.nvmrc` names, through nvm when there modules of `go.mod`; node (the version `.nvmrc` names, through nvm when there
is no node on `PATH`) and yarn, plus the prettier version pinned in is no node on `PATH`) and yarn, plus the prettier version pinned in
`package.json`/`yarn.lock`; `gofumpt` v0.12.0 and `protoc-gen-go` v1.36.11, `package.json`/`yarn.lock`; `gofumpt` v0.12.0 and `protoc-gen-go` v1.36.12,
installed into `bin/` with `go install`, each pinned to a commit; and `protoc` installed into `bin/` with `go install`, each pinned to a commit; and `protoc`
33.4, unpacked into `bin/protoc` from its release archive once the archive 33.4, unpacked into `bin/protoc` from its release archive once the archive
matches the sha256 the script holds for this platform. Each of those three is matches the sha256 the script holds for this platform. Each of those three is
@@ -98,6 +99,11 @@ provide:
as ordinary tests as ordinary tests
- `script/lint` — run `golangci-lint` in Docker: builds only the `lint` stage of - `script/lint` — run `golangci-lint` in Docker: builds only the `lint` stage of
the `Dockerfile`, whose build runs the linter, uncached so it runs every time the `Dockerfile`, whose build runs the linter, uncached so it runs every time
- `script/vulncheck` (`make vulncheck`) — run `govulncheck` in Docker: builds
only the `vulncheck` stage of the `Dockerfile`, uncached, which reports known
vulnerabilities in the code `mfer` calls, from the Go vulnerability database.
`script/check` does not run it, so an advisory published later never turns the
gate red
- `script/fmt` — format all code and docs (writes): `script/gofumpt --write` and - `script/fmt` — format all code and docs (writes): `script/gofumpt --write` and
`script/prettier --write` `script/prettier --write`
- `script/gofumpt` — run `gofumpt` over every Go file in the repository in the - `script/gofumpt` — run `gofumpt` over every Go file in the repository in the
@@ -261,26 +267,42 @@ are now tracked only in the [issues](https://git.eeqj.de/sneak/mfer/issues).
- `mfer gen` / `mfer gen .` - `mfer gen` / `mfer gen .`
- recurses under current directory and writes out an `index.mf` - recurses under current directory and writes out an `index.mf`
- records every file's mode as `0000` unless given `--include-permissions`,
which records each file's permission bits (`0777` at most)
- `mfer check` / `mfer check .` - `mfer check` / `mfer check .`
- verifies checksums of all files in manifest, displaying error and exiting - verifies checksums of all files in manifest, displaying error and exiting
nonzero if any files are missing or corrupted nonzero if any files are missing or corrupted, or have permission bits
other than the mode the manifest records, unless that is `0000`
- looks for those files under the base directory: the one `--base` names, or
else the directory holding the manifest, or the current directory for a
manifest given by URL. So `mfer check /media/drive` checks a drive against
the `index.mf` at its root, from any directory
- warns about each file under the base directory that the manifest does not - warns about each file under the base directory that the manifest does not
list, hidden files included; with `--no-extra-files` each one is a failure list, hidden files included; with `--no-extra-files` each one is a failure
instead instead
- `mfer freshen` / `mfer freshen .`
- rewrites `index.mf` to list the files now under the directory holding it,
or under the one `--base` names, hashing only the files that are new or
changed
- leaves out hidden files unless given `--include-dotfiles`, and symlinks
unless given `--follow-symlinks`, which lists each symlink to a file under
its own name with the contents of the file it points to
- `mfer fetch https://example.com/stuff/` - `mfer fetch https://example.com/stuff/`
- fetches `/stuff/index.mf` and downloads all files listed in manifest into - fetches `/stuff/index.mf` and downloads all files listed in manifest into
the current directory, or the one given with `--dest`, and assures the current directory, or the one given with `--dest`, and assures
cryptographic integrity of downloaded files. A file already there with the cryptographic integrity of downloaded files. A file already there with the
size and hash the manifest lists is skipped. Once every file is in place, size, hash and recorded mode the manifest lists is skipped. Once every
the manifest is saved there as `index.mf`, so `mfer check` can verify the file is in place, the manifest is saved there as `index.mf`, so
tree later. Each file is downloaded to a temp file beside it, such as `mfer check` can verify the tree later. Each file is downloaded to a temp
`.a.txt.tmp` for `a.txt`, then moved into place. A manifest is refused file beside it, such as `.a.txt.tmp` for `a.txt`, given the mode the
before any file is downloaded if it lists a file where fetch writes manifest records unless that is `0000`, then moved into place. A manifest
another: at another listed file or a directory one is in, at the temp file is refused before any file is downloaded if it records a mode above
of a listed file, or at `index.mf` or `.index.mf.tmp` at the top of the `0777`, or lists a file where fetch writes another: at another listed file
tree. Names are compared in any letter case, on every filesystem, since on or a directory one is in, at the temp file of a listed file, or at
a case-insensitive one `A.txt` and `a.txt` are one file; a directory two `index.mf` or `.index.mf.tmp` at the top of the tree. Names are compared
listed files are in must be spelled alike in both. in any letter case, on every filesystem, since on a case-insensitive one
`A.txt` and `a.txt` are one file; a directory two listed files are in must
be spelled alike in both.
- `mfer fetch --require-signature <fingerprint> https://example.com/stuff/` - `mfer fetch --require-signature <fingerprint> https://example.com/stuff/`
- as above, but first refuses a manifest not signed by the key with that - as above, but first refuses a manifest not signed by the key with that
fingerprint, as `mfer check --require-signature` does, before downloading fingerprint, as `mfer check --require-signature` does, before downloading
+42 -8
View File
@@ -6,8 +6,11 @@ Version 1.0
An `.mf` file is a binary manifest that describes a directory tree of files, An `.mf` file is a binary manifest that describes a directory tree of files,
including their paths, sizes, and cryptographic checksums. It supports optional including their paths, sizes, and cryptographic checksums. It supports optional
GPG signatures for integrity verification and optional timestamps for metadata GPG signatures for integrity verification and optional timestamps and file
preservation. permissions for metadata preservation.
Nothing goes in the 1.0 manifest that 1.0 does not read or write: no field is
reserved or kept for later use.
## File Structure ## File Structure
@@ -34,7 +37,7 @@ The outer message contains:
| `uuid` | 105 | bytes | Random v4 UUID; must match the inner message UUID | | `uuid` | 105 | bytes | Random v4 UUID; must match the inner message UUID |
| `innerMessage` | 199 | bytes | Zstd-compressed serialized `MFFile` message | | `innerMessage` | 199 | bytes | Zstd-compressed serialized `MFFile` message |
| `signature` | 201 | bytes (optional) | GPG signature (ASCII-armored or binary) | | `signature` | 201 | bytes (optional) | GPG signature (ASCII-armored or binary) |
| `signer` | 202 | bytes (optional) | Full GPG key ID of the signer | | `signer` | 202 | bytes (optional) | Fingerprint of the signing key |
| `signingPubKey` | 203 | bytes (optional) | Full GPG signing public key | | `signingPubKey` | 203 | bytes (optional) | Full GPG signing public key |
### SHA-256 Hash ### SHA-256 Hash
@@ -47,11 +50,14 @@ allows verifying data integrity before decompression.
The `innerMessage` field is compressed with The `innerMessage` field is compressed with
[Zstandard (zstd)](https://facebook.github.io/zstd/). Implementations must [Zstandard (zstd)](https://facebook.github.io/zstd/). Implementations must
enforce a decompression size limit to prevent decompression bombs. The reference enforce a decompression size limit to prevent decompression bombs. The reference
implementation limits decompressed size to 256 MB. It writes zstd frames with a implementation limits decompressed size to 256 MiB. It writes zstd frames with a
window of at most 8 MiB, the largest window the zstd format recommends decoders window of at most 8 MiB, the largest window the zstd format recommends decoders
support, and refuses frames that ask for a larger one. It also refuses an inner support, and refuses frames that ask for a larger one. It also refuses an inner
message whose file entries, hashes, timestamps and MIME types, counted at 160, message whose file entries, hashes, timestamps and MIME types, counted at 176,
112, 64 and 16 bytes each, add up to more than 8 times its size. 112, 64 and 16 bytes each, add up to more than 8 times its size. It refuses a
manifest file larger than 258 MiB without reading the rest of it: zstd's worst
case grows a 256 MiB inner message by 1/256 to 257 MiB, and the last MiB is room
for the signature, the signing key and the other outer fields.
## Inner Message (`MFFile`) ## Inner Message (`MFFile`)
@@ -77,6 +83,21 @@ Each file entry contains:
| `mimeType` | 301 | string (optional) | MIME type | | `mimeType` | 301 | string (optional) | MIME type |
| `mtime` | 302 | Timestamp (optional) | Modification time | | `mtime` | 302 | Timestamp (optional) | Modification time |
| `ctime` | 303 | Timestamp (optional) | Change time (inode metadata change) | | `ctime` | 303 | Timestamp (optional) | Change time (inode metadata change) |
| `mode` | 304 | uint32 | Permission bits (see File Mode) |
## File Mode
`mode` holds a file's Unix permission bits, the nine `rwx` bits, so it is never
above `0777` (octal); the setuid, setgid and sticky bits are never recorded.
Writers record `0000` unless whoever creates the manifest asks for permissions.
`0000`, the proto3 default, means no mode was recorded: readers never check or
apply it.
The reference implementation records modes when `gen` or `freshen` is given
`--include-permissions`. `check` fails a file whose permission bits differ from
a recorded mode other than `0000`. `fetch` sets a recorded mode other than
`0000` on each file it writes, and refuses a manifest that records a mode above
`0777` before it requests any file.
## Path Rules ## Path Rules
@@ -88,9 +109,12 @@ All `path` values must satisfy these invariants:
- **No parent traversal**: no `..` path segments - **No parent traversal**: no `..` path segments
- **No empty segments**: no `//` sequences - **No empty segments**: no `//` sequences
- **No trailing slash**: paths refer to files, not directories - **No trailing slash**: paths refer to files, not directories
- **Listed once**: each path appears at most once in a manifest, compared byte
for byte, so `A.txt` and `a.txt` are two paths
Implementations must validate these invariants when reading and writing Implementations must validate these invariants when reading and writing
manifests. Paths that violate these rules must be rejected. manifests. Paths that violate these rules must be rejected, and a reader must
reject a manifest that lists a path more than once.
## Hash Format (`MFFileChecksum`) ## Hash Format (`MFFileChecksum`)
@@ -119,7 +143,16 @@ Where:
compressed data) compressed data)
Components are separated by hyphens. The signature is produced by GPG over this Components are separated by hyphens. The signature is produced by GPG over this
canonical string and stored in the `signature` field of the outer message. canonical string and stored in the `signature` field of the outer message. The
signing key's public key goes in `signingPubKey` and its fingerprint, in hex, in
`signer`.
A verifier accepts a signed manifest only if `signingPubKey` holds exactly one
primary key, `signature` is one good signature over the canonical string made by
that key (or one of its subkeys), and `signer` is that key's fingerprint. The
reference implementation refuses to load a manifest that fails these checks;
`check` and `fetch` given `--require-signature` then compare the required
fingerprint with `signer`.
## Deterministic Serialization ## Deterministic Serialization
@@ -127,6 +160,7 @@ By default, manifests are generated deterministically:
- File entries are sorted by `path` in **lexicographic byte order** - File entries are sorted by `path` in **lexicographic byte order**
- `createdAt` is omitted unless explicitly requested - `createdAt` is omitted unless explicitly requested
- `mode` is `0000` unless explicitly requested
This ensures that two independent runs over the same directory tree produce This ensures that two independent runs over the same directory tree produce
byte-identical `.mf` files (assuming file contents and metadata have not byte-identical `.mf` files (assuming file contents and metadata have not
+14 -15
View File
@@ -1,29 +1,28 @@
module sneak.berlin/go/mfer module sneak.berlin/go/mfer
go 1.23 go 1.27.1
require ( require (
github.com/davecgh/go-spew v1.1.1 github.com/davecgh/go-spew v1.1.1
github.com/dustin/go-humanize v1.0.1 github.com/dustin/go-humanize v1.1.0
github.com/google/uuid v1.1.2 github.com/klauspost/compress v1.20.1
github.com/klauspost/compress v1.18.2
github.com/multiformats/go-multihash v0.2.3 github.com/multiformats/go-multihash v0.2.3
github.com/spf13/afero v1.8.0 github.com/spf13/afero v1.15.0
github.com/stretchr/testify v1.12.1 github.com/stretchr/testify v1.12.1
github.com/urfave/cli/v3 v3.14.0 github.com/urfave/cli/v3 v3.14.0
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 golang.org/x/term v0.46.0
google.golang.org/protobuf v1.28.1 google.golang.org/protobuf v1.36.12
) )
require ( require (
github.com/klauspost/cpuid/v2 v2.0.9 // indirect github.com/klauspost/cpuid/v2 v2.4.0 // indirect
github.com/minio/sha256-simd v1.0.0 // indirect github.com/minio/sha256-simd v1.0.1 // indirect
github.com/mr-tron/base58 v1.2.0 // indirect github.com/mr-tron/base58 v1.3.0 // indirect
github.com/multiformats/go-varint v0.0.6 // indirect github.com/multiformats/go-varint v0.1.0 // indirect
github.com/spaolacci/murmur3 v1.1.0 // indirect github.com/spaolacci/murmur3 v1.1.0 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect go.yaml.in/yaml/v3 v3.0.5 // indirect
golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e // indirect golang.org/x/crypto v0.57.0 // indirect
golang.org/x/sys v0.1.0 // indirect golang.org/x/sys v0.48.0 // indirect
golang.org/x/text v0.3.6 // indirect golang.org/x/text v0.42.0 // indirect
lukechampine.com/blake3 v1.1.6 // indirect lukechampine.com/blake3 v1.4.1 // indirect
) )
+28 -463
View File
@@ -1,475 +1,40 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=
cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU=
cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=
cloud.google.com/go v0.44.3/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=
cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=
cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=
cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To=
cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4=
cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M=
cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc=
cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk=
cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=
cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=
cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=
cloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI=
cloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk=
cloud.google.com/go v0.75.0/go.mod h1:VGuuCn7PG0dwsd5XPVm2Mm3wlh3EL55/79EKB6hlPTY=
cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=
cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=
cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc=
cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=
cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=
cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=
cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=
cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU=
cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=
cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=
cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=
cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=
cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
cloud.google.com/go/storage v1.14.0/go.mod h1:GrKmX003DSIwi9o29oFT7YDnHYwZoctc3fOKtUw0Xmo=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= github.com/dustin/go-humanize v1.1.0 h1:dbKTrvD0klcbBV/h4AWJdMuZogJACoMlvWIWZ5b2xWg=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= github.com/dustin/go-humanize v1.1.0/go.mod h1:hc1CvRkJMsgxqjmjMQF3QNRAZBwY8AXBAzKYoSX9sFI=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98= github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po= github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk= github.com/klauspost/cpuid/v2 v2.4.0 h1:S6Hrbc7+ywsr0r+RLapfGBHfyefhCTwEh3A0tV913Dw=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= github.com/klauspost/cpuid/v2 v2.4.0/go.mod h1:19jmZ9mjzoF//ddRSUsv0zfBTJWh3QJh9FNxZTMrGxU=
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU= github.com/minio/sha256-simd v1.0.1 h1:6kaan5IFmwTNynnKKpDHe6FWHohJOHhCPchzK49dzMM=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= github.com/minio/sha256-simd v1.0.1/go.mod h1:Pz6AKMiUdngCLpeTL/RJY1M9rUuPMYujV5xJjtbRSN8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= github.com/mr-tron/base58 v1.3.0 h1:K6Y13R2h+dku0wOqKtecgRnBUBPrZzLZy5aIj8lCcJI=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= github.com/mr-tron/base58 v1.3.0/go.mod h1:2BuubE67DCSWwVfx37JWNG8emOC0sHEU4/HpcYgCLX8=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=
github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20201218002935-b9804c9f04c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/uuid v1.1.2 h1:EVhdT+1Kseyi1/pUmXKaFxYsDNy9RQYkMWRH68J/W7Y=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
github.com/googleapis/google-cloud-go-testing v0.0.0-20200911160855-bcd43fbb19e8/go.mod h1:dvDLG8qkwmyD9a/MJJN3XJcT3xFxOKAvTZGvuZmac9g=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk=
github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/klauspost/cpuid/v2 v2.0.4/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.0.9 h1:lgaqFMSdTdQYdZ04uHyN2d/eKdOMyi2YLSvlQIBFYa4=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/minio/sha256-simd v1.0.0 h1:v1ta+49hkWZyvaKwrQB8elexRqm6Y0aMLjCNsrYxo6g=
github.com/minio/sha256-simd v1.0.0/go.mod h1:OuYzVNI5vcoYIAmbIvHPl3N3jUzVedXbKy5RFepssQM=
github.com/mr-tron/base58 v1.2.0 h1:T/HDJBh4ZCPbU39/+c3rRvE0uKBQlU27+QI8LJ4t64o=
github.com/mr-tron/base58 v1.2.0/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=
github.com/multiformats/go-multihash v0.2.3 h1:7Lyc8XfX/IY2jWb/gI7JP+o7JEq9hOa7BFvVU9RSh+U= github.com/multiformats/go-multihash v0.2.3 h1:7Lyc8XfX/IY2jWb/gI7JP+o7JEq9hOa7BFvVU9RSh+U=
github.com/multiformats/go-multihash v0.2.3/go.mod h1:dXgKXCXjBzdscBLk9JkjINiEsCKRVch90MdaGiKsvSM= github.com/multiformats/go-multihash v0.2.3/go.mod h1:dXgKXCXjBzdscBLk9JkjINiEsCKRVch90MdaGiKsvSM=
github.com/multiformats/go-varint v0.0.6 h1:gk85QWKxh3TazbLxED/NlDVv8+q+ReFJk7Y2W/KhfNY= github.com/multiformats/go-varint v0.1.0 h1:i2wqFp4sdl3IcIxfAonHQV9qU5OsZ4Ts9IOoETFs5dI=
github.com/multiformats/go-varint v0.0.6/go.mod h1:3Ls8CIEsrijN6+B7PbrXRPxHRPuXSrVKRY101jdMZYE= github.com/multiformats/go-varint v0.1.0/go.mod h1:5KVAVXegtfmNQQm/lCY+ATvDzvJJhSkUlGQV9wgObdI=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/sftp v1.13.1/go.mod h1:3HaPG6Dq1ILlpPZRO0HVMrsydcdLt6HRDccSgb87qRg=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI= github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI=
github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/spf13/afero v1.8.0 h1:5MmtuhAgYeU6qpa7w7bP0dv6MBYuup0vekhSpSkoq60= github.com/spf13/afero v1.15.0 h1:b/YBCLWAJdFWJTN9cLhiXXcD7mzKn9Dm86dNnfyQw1I=
github.com/spf13/afero v1.8.0/go.mod h1:CtAatgMJh6bJEIs48Ay/FOnkljP3WeGUG0MC1RfAqwo= github.com/spf13/afero v1.15.0/go.mod h1:NC2ByUVxtQs4b3sIUphxK0NioZnmxgyCrfzeuq8lxMg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE= github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg= github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
github.com/urfave/cli/v3 v3.14.0 h1:a8414NQlHJs0c/iBsulKLzlES0n/lEAskbL2LKpU4/s= github.com/urfave/cli/v3 v3.14.0 h1:a8414NQlHJs0c/iBsulKLzlES0n/lEAskbL2LKpU4/s=
github.com/urfave/cli/v3 v3.14.0/go.mod h1:vXn6HxPNccJSzQr2QvwVncOKrgYGIHU0HY5h8B2nQj4= github.com/urfave/cli/v3 v3.14.0/go.mod h1:vXn6HxPNccJSzQr2QvwVncOKrgYGIHU0HY5h8B2nQj4=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw= go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg= go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/term v0.46.0 h1:3+OXuTbaKDgwk8jTi3aSLHRlmWqHEUDUtxnbFigO4YE=
golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4= golang.org/x/term v0.46.0/go.mod h1:+K02xbkittuwc0Am4abfA3Fc+XRGXkvBXNO88NCXPoc=
golang.org/x/crypto v0.0.0-20211108221036-ceb1ce70b4fa/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI=
golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e h1:T8NU3HyQ8ClP4SEE+KbFlg6n0NhuTsN4MyznaarGsZM= golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E=
golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= google.golang.org/protobuf v1.36.12 h1:pJOKDDOyeXErUroCihFAd5LQuwXBSpVnKGrj5o/fwxc=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= google.golang.org/protobuf v1.36.12/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= lukechampine.com/blake3 v1.4.1 h1:I3Smz7gso8w4/TunLKec6K2fn+kyKtDxr/xcQEN84Wg=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= lukechampine.com/blake3 v1.4.1/go.mod h1:QFosUxmjB8mnrWFSNwKmvxHpfY72bmD2tQ0kBMM3kwo=
golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=
golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=
golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=
golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210225134936-a50acf3fe073/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423185535-09eb48e85fd7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.1.0 h1:kunALQeHf1/185U1i0GOB/fy1IPRDDpuoOOqRReG57U=
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 h1:JGgROgKl9N8DuW20oFS5gxc+lE67/N3FcwmBPMe7ArY=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=
golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=
golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210108195828-e2f9c7f1fc8e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM=
google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc=
google.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=
google.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE=
google.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=
google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=
google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA=
google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=
google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20210108203827-ffc7fda8c3d7/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20210226172003-ab064af71705/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=
google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=
google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.31.1/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
google.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8=
google.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.28.1 h1:d0NfwRgPtno5B1Wa6L2DAG+KivqkdutMf1UhdNx175w=
google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
lukechampine.com/blake3 v1.1.6 h1:H3cROdztr7RCfoaTpGZFQsrqvweFLrqS73j7L7cmR5c=
lukechampine.com/blake3 v1.1.6/go.mod h1:tkKEOtDkNtklkXtLNEOGNq5tcV90tJiA1vAA12R78LA=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
+29 -26
View File
@@ -108,28 +108,33 @@ func (mfa *CLIApp) fetchManifestToTemp(
if tmpErr != nil { if tmpErr != nil {
_ = rc.Close() _ = rc.Close()
return "", fmt.Errorf("failed to create temp file: %w", tmpErr) return "", tmpErr
} }
tmpPath := tmpFile.Name() tmpPath := tmpFile.Name()
_, cpErr := io.Copy(tmpFile, rc)
// Copying stops one byte past mfa.maxManifestSize, which is enough to
// tell that the manifest is too large.
written, cpErr := io.Copy(tmpFile, io.LimitReader(rc, mfa.maxManifestSize+1))
_ = rc.Close() _ = rc.Close()
_ = tmpFile.Close() _ = tmpFile.Close()
if cpErr == nil && written > mfa.maxManifestSize {
cpErr = fmt.Errorf("%w of %d bytes", errManifestTooLarge, mfa.maxManifestSize)
}
if cpErr != nil { if cpErr != nil {
_ = mfa.Fs.Remove(tmpPath) _ = mfa.Fs.Remove(tmpPath)
return "", fmt.Errorf("failed to download manifest: %w", cpErr) return "", fmt.Errorf("download manifest: %w", cpErr)
} }
return tmpPath, nil return tmpPath, nil
} }
// verifyRequiredSigner enforces the --require-signature fingerprint // verifyRequiredSigner enforces the --require-signature fingerprint
// against the manifest's embedded signing key. // against the key that made the manifest's signature.
func verifyRequiredSigner( func verifyRequiredSigner(chk *mfer.Checker, requiredSigner string) error {
ctx context.Context, chk *mfer.Checker, requiredSigner string,
) error {
// Validate fingerprint format: must be exactly 40 hex characters // Validate fingerprint format: must be exactly 40 hex characters
if len(requiredSigner) != fingerprintHexLen { if len(requiredSigner) != fingerprintHexLen {
return fmt.Errorf("%w, got %d", errInvalidFingerprint, len(requiredSigner)) return fmt.Errorf("%w, got %d", errInvalidFingerprint, len(requiredSigner))
@@ -137,7 +142,7 @@ func verifyRequiredSigner(
_, err := hex.DecodeString(requiredSigner) _, err := hex.DecodeString(requiredSigner)
if err != nil { if err != nil {
return fmt.Errorf("invalid fingerprint: must be valid hex: %w", err) return fmt.Errorf("invalid fingerprint: %w", err)
} }
if !chk.IsSigned() { if !chk.IsSigned() {
@@ -145,22 +150,17 @@ func verifyRequiredSigner(
errManifestNotSigned, requiredSigner) errManifestNotSigned, requiredSigner)
} }
// Extract fingerprint from the embedded public key (not from the // Loading the manifest checked that the signer is the fingerprint of
// signer field). This validates the key is importable and gets its // the key that made the signature.
// actual fingerprint. signer := string(chk.Signer())
embeddedFP, err := chk.ExtractEmbeddedSigningKeyFP(ctx)
if err != nil {
return fmt.Errorf(
"failed to extract fingerprint from embedded signing key: %w", err)
}
// Compare fingerprints - must be exact match (case-insensitive) // Compare fingerprints - must be exact match (case-insensitive)
if !strings.EqualFold(embeddedFP, requiredSigner) { if !strings.EqualFold(signer, requiredSigner) {
return fmt.Errorf("embedded signing key fingerprint %s %w %s", return fmt.Errorf("embedded signing key fingerprint %s %w %s",
embeddedFP, errSignerMismatch, requiredSigner) signer, errSignerMismatch, requiredSigner)
} }
log.Infof("manifest signature verified (signer: %s)", embeddedFP) log.Infof("manifest signature verified (signer: %s)", signer)
return nil return nil
} }
@@ -232,7 +232,7 @@ func findExtraFiles(
err := chk.FindExtraFiles(ctx, extraResults) err := chk.FindExtraFiles(ctx, extraResults)
if err != nil { if err != nil {
return fmt.Errorf("failed to check for extra files: %w", err) return fmt.Errorf("find extra files: %w", err)
} }
<-extraDone <-extraDone
@@ -275,7 +275,7 @@ func runCheck(
progressWg.Wait() progressWg.Wait()
if err != nil { if err != nil {
return 0, fmt.Errorf("check failed: %w", err) return 0, fmt.Errorf("check files: %w", err)
} }
// Wait for results processing to complete // Wait for results processing to complete
@@ -296,14 +296,18 @@ func (mfa *CLIApp) checkManifestOperation(
manifestPath, err := mfa.resolveManifestArg(cmd) manifestPath, err := mfa.resolveManifestArg(cmd)
if err != nil { if err != nil {
return fmt.Errorf("check: %w", err) return err
} }
// Done before a URL is swapped for the temp file it is downloaded to,
// whose directory is not the base.
basePath := resolveBasePath(cmd, manifestPath)
// URL manifests need to be downloaded to a temp file for the checker // URL manifests need to be downloaded to a temp file for the checker
if isHTTPURL(manifestPath) { if isHTTPURL(manifestPath) {
tmpPath, tmpErr := mfa.fetchManifestToTemp(ctx, manifestPath) tmpPath, tmpErr := mfa.fetchManifestToTemp(ctx, manifestPath)
if tmpErr != nil { if tmpErr != nil {
return fmt.Errorf("check: %w", tmpErr) return tmpErr
} }
defer func() { _ = mfa.Fs.Remove(tmpPath) }() defer func() { _ = mfa.Fs.Remove(tmpPath) }()
@@ -311,7 +315,6 @@ func (mfa *CLIApp) checkManifestOperation(
manifestPath = tmpPath manifestPath = tmpPath
} }
basePath := cmd.String("base")
showProgress := cmd.Bool("progress") showProgress := cmd.Bool("progress")
log.Infof("checking manifest %s with base %s", manifestPath, basePath) log.Infof("checking manifest %s with base %s", manifestPath, basePath)
@@ -324,13 +327,13 @@ func (mfa *CLIApp) checkManifestOperation(
Fs: mfa.Fs, Fs: mfa.Fs,
}) })
if err != nil { if err != nil {
return fmt.Errorf("failed to load manifest: %w", err) return fmt.Errorf("load manifest: %w", err)
} }
// Check signature requirement // Check signature requirement
requiredSigner := cmd.String(flagRequireSignature) requiredSigner := cmd.String(flagRequireSignature)
if requiredSigner != "" { if requiredSigner != "" {
err = verifyRequiredSigner(ctx, chk, requiredSigner) err = verifyRequiredSigner(chk, requiredSigner)
if err != nil { if err != nil {
return err return err
} }
+10 -8
View File
@@ -5,6 +5,7 @@ import (
"os" "os"
"github.com/spf13/afero" "github.com/spf13/afero"
"sneak.berlin/go/mfer/mfer"
) )
// NoColor disables colored output when set. Automatically true if the // NoColor disables colored output when set. Automatically true if the
@@ -56,14 +57,15 @@ func Run(appname, version, gitrev string) int {
// RunWithOptions creates and runs the CLI application with the given options. // RunWithOptions creates and runs the CLI application with the given options.
func RunWithOptions(opts *RunOptions) int { func RunWithOptions(opts *RunOptions) int {
m := &CLIApp{ m := &CLIApp{
appname: opts.Appname, appname: opts.Appname,
version: opts.Version, version: opts.Version,
gitrev: opts.Gitrev, gitrev: opts.Gitrev,
exitCode: 0, exitCode: 0,
Stdin: opts.Stdin, maxManifestSize: mfer.MaxManifestSize,
Stdout: opts.Stdout, Stdin: opts.Stdin,
Stderr: opts.Stderr, Stdout: opts.Stdout,
Fs: opts.Fs, Stderr: opts.Stderr,
Fs: opts.Fs,
} }
m.run(opts.Args) m.run(opts.Args)
+276
View File
@@ -3,10 +3,12 @@ package cli
import ( import (
"bytes" "bytes"
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"io" "io"
"math/rand" "math/rand"
"net/http/httptest"
"os" "os"
"path/filepath" "path/filepath"
"slices" "slices"
@@ -353,6 +355,77 @@ func TestGenerateCommand(t *testing.T) {
assert.True(t, exists) assert.True(t, exists)
} }
// TestGenerateRefusesTwoFilesAtOnePath runs gen on arguments whose files
// would share a path in the manifest: two directories that each hold a.txt,
// and one directory given twice. gen must fail while it lists the files,
// before it hashes any, naming the path, and write no manifest.
func TestGenerateRefusesTwoFilesAtOnePath(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
name string
first, second string
}{
{"two directories", testDir, "/other"},
{"one directory twice", testDir, testDir},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll(testDir, 0o755))
require.NoError(t, fs.MkdirAll("/other", 0o755))
writeTestFile(t, fs, "/testdir/a.txt", "first")
writeTestFile(t, fs, "/other/a.txt", "second")
opts := testOpts([]string{
testApp, cmdGenerate, "-q", "-o", testOutput, tc.first, tc.second,
}, fs)
assert.Equal(t, 1, runCLI(opts))
assert.Contains(t, testStderr(t, opts),
`enumerate files: duplicate path "a.txt": `+
tc.first+"/a.txt and "+tc.second+"/a.txt")
exists, err := afero.Exists(fs, testOutput)
require.NoError(t, err)
assert.False(t, exists)
})
}
}
// TestGenerateSeededManifestBytes pins the exact bytes `gen --seed` writes
// for a fixed tree, so that a Go or dependency update that changes what
// mfer writes fails here. testdata/seeded.mf was written by an mfer built
// before such an update. The tree has enough files that zstd compresses
// the manifest instead of storing it as it is.
func TestGenerateSeededManifestBytes(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
start := time.Date(2025, 6, 1, 0, 0, 0, 0, time.UTC)
for i := range 200 {
path := fmt.Sprintf("/testdir/d%d/f%03d.txt", i%10, i)
mtime := start.Add(time.Duration(i) * time.Second)
require.NoError(t, fs.MkdirAll(filepath.Dir(path), 0o755))
writeTestFile(t, fs, path, fmt.Sprintf("file %d\n", i))
require.NoError(t, fs.Chtimes(path, mtime, mtime))
}
opts := testOpts([]string{
testApp, cmdGenerate, "-q", "--seed", "mfer", "-o", testOutput, testDir,
}, fs)
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
got, err := afero.ReadFile(fs, testOutput)
require.NoError(t, err)
want, err := os.ReadFile("testdata/seeded.mf")
require.NoError(t, err)
assert.Equal(t, want, got)
}
func TestGenerateAndCheckCommand(t *testing.T) { func TestGenerateAndCheckCommand(t *testing.T) {
t.Parallel() t.Parallel()
@@ -374,6 +447,98 @@ func TestGenerateAndCheckCommand(t *testing.T) {
assert.Equal(t, 0, exitCode, "check failed: %s", testStderr(t, opts)) assert.Equal(t, 0, exitCode, "check failed: %s", testStderr(t, opts))
} }
// TestGenerateRecordsModeOnlyWhenAsked runs gen with and without
// --include-permissions: without it every mode is recorded as 0000, with
// it each file's permission bits. list -l and export show the recorded
// modes.
func TestGenerateRecordsModeOnlyWhenAsked(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll(testDir, 0o755))
writeTestFile(t, fs, "/testdir/notes.txt", "hello world")
writeTestFile(t, fs, "/testdir/run.sh", "#!/bin/sh\n")
require.NoError(t, fs.Chmod("/testdir/run.sh", 0o755))
for _, tc := range []struct {
flags []string
want map[string]string // recorded mode by path
}{
{nil, map[string]string{"notes.txt": "0000", "run.sh": "0000"}},
{
[]string{"--" + flagIncludePermissions},
map[string]string{"notes.txt": "0644", "run.sh": "0755"},
},
} {
opts := testOpts(slices.Concat(
[]string{testApp, cmdGenerate, "-q", "-f", "-o", testOutput}, tc.flags,
[]string{testDir},
), fs)
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
opts = testOpts([]string{testApp, cmdList, "-l", testOutput}, fs)
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
listed := map[string]string{}
out := strings.TrimSuffix(testStdout(t, opts), "\n")
for line := range strings.SplitSeq(out, "\n") {
fields := strings.Split(line, "\t") // mode, size, mtime, path
require.Len(t, fields, 4, line)
listed[fields[3]] = fields[0]
}
assert.Equal(t, tc.want, listed, "list -l %v", tc.flags)
opts = testOpts([]string{testApp, cmdExport, testOutput}, fs)
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
var entries []ExportEntry
require.NoError(t, json.Unmarshal([]byte(testStdout(t, opts)), &entries))
exported := map[string]string{}
for _, e := range entries {
exported[e.Path] = e.Mode
}
assert.Equal(t, tc.want, exported, "export %v", tc.flags)
}
}
// TestCheckComparesRecordedMode changes a file's mode after gen: check
// must fail on it when gen recorded the file's mode, and pass when gen
// recorded 0000.
func TestCheckComparesRecordedMode(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
flags []string
exitCode int
}{
{nil, 0},
{[]string{"--" + flagIncludePermissions}, 1},
} {
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll(testDir, 0o755))
writeTestFile(t, fs, testFile1, "hello world")
opts := testOpts(slices.Concat(
[]string{testApp, cmdGenerate, "-q", "-o", testMF}, tc.flags,
[]string{testDir},
), fs)
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
require.NoError(t, fs.Chmod(testFile1, 0o600))
opts = testOpts([]string{testApp, cmdCheck, testFlagBase, testDir, testMF}, fs)
assert.Equal(t, tc.exitCode, runCLI(opts), "%v: %s", tc.flags, testStderr(t, opts))
if tc.exitCode != 0 {
assert.Contains(t, testStderr(t, opts), "MODE_MISMATCH: file1.txt")
}
}
}
// sharedWriter appends to a buffer shared with other sharedWriters, so // sharedWriter appends to a buffer shared with other sharedWriters, so
// output written to stdout and stderr is kept in the order it was written. // output written to stdout and stderr is kept in the order it was written.
// Each write first waits for delay. // Each write first waits for delay.
@@ -487,6 +652,33 @@ func runCheckAfterRewrite(t *testing.T, rewritten, msg string) {
assert.Equal(t, 1, exitCode, msg) assert.Equal(t, 1, exitCode, msg)
} }
// TestCheckRequireSignatureRefusesOtherSigningKey runs check
// --require-signature on a manifest signed by another key whose embedded
// public key block also holds the required key. check must refuse it. It
// needs gpg and is skipped without it, as the other signing tests are.
//
//nolint:paralleltest // signedManifest calls t.Setenv, which bars t.Parallel
func TestCheckRequireSignatureRefusesOtherSigningKey(t *testing.T) {
content := []byte("signed file")
manifest, required := manifestSignedByAnotherKey(t,
map[string][]byte{testFileTxt: content})
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll(testDir, 0o755))
require.NoError(t, afero.WriteFile(fs,
filepath.Join(testDir, testFileTxt), content, 0o644))
require.NoError(t, afero.WriteFile(fs, testManifest, manifest, 0o644))
opts := testOpts([]string{
testApp, cmdCheck, "-q", testFlagBase, testDir,
"--" + flagRequireSignature, required, testManifest,
}, fs)
assert.Equal(t, 1, runCLI(opts))
assert.Contains(t, testStderr(t, opts),
"load manifest: "+
"embedded public key block must hold exactly one key, found 2")
}
func TestCheckCommandWithCorruptedFile(t *testing.T) { func TestCheckCommandWithCorruptedFile(t *testing.T) {
t.Parallel() t.Parallel()
@@ -788,6 +980,90 @@ func TestCheckNeverReportsManifest(t *testing.T) {
} }
} }
// The directory setupManifestInSubdir makes and the manifest it writes there,
// relative to the working directory it sets.
const (
testSubdir = "sub"
testSubdirManifest = testSubdir + "/" + defaultManifestName
)
// setupManifestInSubdir makes a temp dir holding file.txt and sub/b.txt,
// where sub/index.mf is the manifest gen writes for sub, and makes it the
// working directory, so a test calling it cannot run in parallel. It returns
// the temp dir.
func setupManifestInSubdir(t *testing.T) string {
t.Helper()
root := t.TempDir()
sub := filepath.Join(root, testSubdir)
fs := afero.NewOsFs()
require.NoError(t, fs.MkdirAll(sub, 0o750))
writeTestFile(t, fs, filepath.Join(root, testFileTxt), "not in the manifest")
writeTestFile(t, fs, filepath.Join(sub, "b.txt"), "in the manifest")
opts := testOpts([]string{
testApp, cmdGenerate, "-q", "-o", filepath.Join(sub, defaultManifestName), sub,
}, fs)
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
t.Chdir(root)
return root
}
// TestCheckResolvesEntriesAgainstManifestDirectory runs check from the
// directory above sub, on the manifest in sub. Without --base, the
// manifest's entries are looked for in sub, whether check is given the
// manifest or sub, and the files above sub are not reported. --base names
// the directory to look in instead, the current one included.
//
//nolint:paralleltest // changes the process-global working directory
func TestCheckResolvesEntriesAgainstManifestDirectory(t *testing.T) {
root := setupManifestInSubdir(t)
for _, tc := range []struct {
args []string
exitCode int
failure string // a line check must print, if any
}{
{[]string{testSubdir}, 0, ""},
{[]string{testSubdirManifest}, 0, ""},
{[]string{filepath.Join(root, testSubdir)}, 0, ""},
{[]string{testFlagBase, testSubdir, testSubdirManifest}, 0, ""},
{[]string{testFlagBase, ".", testSubdirManifest}, 1, "MISSING: b.txt"},
} {
t.Run(strings.Join(tc.args, " "), func(t *testing.T) {
opts := testOpts(slices.Concat(
[]string{testApp, cmdCheck, testFlagNoExtra}, tc.args,
), afero.NewOsFs())
assert.Equal(t, tc.exitCode, runCLI(opts), "stderr: %s", testStderr(t, opts))
assert.Contains(t, testStderr(t, opts), tc.failure)
})
}
}
// TestCheckURLManifestResolvesEntriesAgainstCurrentDirectory runs check on
// a manifest given by URL, from a directory holding the file it lists: the
// file is looked for there.
//
//nolint:paralleltest // changes the process-global working directory
func TestCheckURLManifestResolvesEntriesAgainstCurrentDirectory(t *testing.T) {
files := map[string][]byte{testFileTxt: []byte("hello")}
server := httptest.NewServer(fetchTestHandler(manifestOf(t, files), files))
defer server.Close()
cwd := chdirTemp(t)
require.NoError(t,
os.WriteFile(filepath.Join(cwd, testFileTxt), files[testFileTxt], 0o600))
opts := testOpts([]string{
testApp, cmdCheck, testFlagNoExtra, server.URL + "/" + defaultManifestName,
}, afero.NewOsFs())
assert.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
}
// unlistableDirFs is a filesystem on which one directory cannot be listed. // unlistableDirFs is a filesystem on which one directory cannot be listed.
type unlistableDirFs struct { type unlistableDirFs struct {
afero.Fs afero.Fs
+144 -17
View File
@@ -9,12 +9,14 @@ import (
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
"slices"
"testing" "testing"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
urfcli "github.com/urfave/cli/v3" urfcli "github.com/urfave/cli/v3"
"google.golang.org/protobuf/proto"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -86,8 +88,7 @@ func TestVerifyRequiredSignerMessages(t *testing.T) {
t.Run("invalid fingerprint length", func(t *testing.T) { t.Run("invalid fingerprint length", func(t *testing.T) {
t.Parallel() t.Parallel()
err := verifyRequiredSigner(context.Background(), err := verifyRequiredSigner(unsignedChecker(t), "12345678")
unsignedChecker(t), "12345678")
require.ErrorIs(t, err, errInvalidFingerprint) require.ErrorIs(t, err, errInvalidFingerprint)
assert.EqualError(t, err, assert.EqualError(t, err,
"invalid fingerprint: must be exactly 40 hex characters, got 8") "invalid fingerprint: must be exactly 40 hex characters, got 8")
@@ -96,8 +97,7 @@ func TestVerifyRequiredSignerMessages(t *testing.T) {
t.Run("manifest not signed", func(t *testing.T) { t.Run("manifest not signed", func(t *testing.T) {
t.Parallel() t.Parallel()
err := verifyRequiredSigner(context.Background(), err := verifyRequiredSigner(unsignedChecker(t), msgFpA)
unsignedChecker(t), msgFpA)
require.ErrorIs(t, err, errManifestNotSigned) require.ErrorIs(t, err, errManifestNotSigned)
assert.EqualError(t, err, assert.EqualError(t, err,
"manifest is not signed, but signature from "+msgFpA+" is required") "manifest is not signed, but signature from "+msgFpA+" is required")
@@ -105,23 +105,21 @@ func TestVerifyRequiredSignerMessages(t *testing.T) {
} }
// TestSignerMismatchMessage drives verifyRequiredSigner against a real signed // TestSignerMismatchMessage drives verifyRequiredSigner against a real signed
// manifest. The embedded fingerprint is whatever the generated key produced, // manifest. The signing key's fingerprint is whatever the generated key
// so it is read back from the checker and substituted into the expected // produced, so it is read back from the checker and substituted into the
// string; the required signer is a fixed value that cannot match it. Requires // expected string; the required signer is a fixed value that cannot match
// gpg and is skipped where it is absent, as the other signing tests are. // it. Requires gpg and is skipped where it is absent, as the other signing
// tests are.
// //
//nolint:paralleltest // signedManifest calls t.Setenv, which bars t.Parallel //nolint:paralleltest // signedManifest calls t.Setenv, which bars t.Parallel
func TestSignerMismatchMessage(t *testing.T) { func TestSignerMismatchMessage(t *testing.T) {
chk := signedChecker(t, chk := signedChecker(t,
signedManifest(t, map[string][]byte{"f.txt": []byte("signed file")})) signedManifest(t, map[string][]byte{"f.txt": []byte("signed file")}))
embeddedFP, err := chk.ExtractEmbeddedSigningKeyFP(context.Background()) err := verifyRequiredSigner(chk, msgFpB)
require.NoError(t, err)
err = verifyRequiredSigner(context.Background(), chk, msgFpB)
require.ErrorIs(t, err, errSignerMismatch) require.ErrorIs(t, err, errSignerMismatch)
assert.EqualError(t, err, assert.EqualError(t, err,
"embedded signing key fingerprint "+embeddedFP+ "embedded signing key fingerprint "+string(chk.Signer())+
" does not match required "+msgFpB) " does not match required "+msgFpB)
} }
@@ -162,7 +160,7 @@ func signedManifest(t *testing.T, files map[string][]byte) []byte {
for path, content := range files { for path, content := range files {
_, err = b.AddFile(mfer.RelFilePath(path), mfer.FileSize(len(content)), _, err = b.AddFile(mfer.RelFilePath(path), mfer.FileSize(len(content)),
mfer.ModTime{}, bytes.NewReader(content), nil) mfer.ModTime{}, 0, bytes.NewReader(content), nil)
require.NoError(t, err) require.NoError(t, err)
} }
@@ -191,6 +189,34 @@ func signedChecker(t *testing.T, manifest []byte) *mfer.Checker {
return chk return chk
} }
// manifestSignedByAnotherKey returns a manifest of files and the
// fingerprint of a throwaway key, the required key, that did not sign it.
// The manifest is signed by a second throwaway key; its embedded public key
// block holds the required key followed by the second key, and its signer
// field names the required key.
func manifestSignedByAnotherKey(
t *testing.T, files map[string][]byte,
) ([]byte, string) {
t.Helper()
required := new(mfer.MFFileOuter)
require.NoError(t, proto.Unmarshal(
signedManifest(t, files)[len(mfer.MAGIC):], required))
outer := new(mfer.MFFileOuter)
require.NoError(t, proto.Unmarshal(
signedManifest(t, files)[len(mfer.MAGIC):], outer))
outer.SigningPubKey = slices.Concat(
required.GetSigningPubKey(), outer.GetSigningPubKey())
outer.Signer = required.GetSigner()
data, err := proto.Marshal(outer)
require.NoError(t, err)
return append([]byte(mfer.MAGIC), data...), string(required.GetSigner())
}
func TestPathDoesNotExistMessage(t *testing.T) { func TestPathDoesNotExistMessage(t *testing.T) {
t.Parallel() t.Parallel()
@@ -277,7 +303,7 @@ func TestManifestLoaderHTTPStatusMessage(t *testing.T) {
_, err := mfa.openManifestReader(context.Background(), server.URL+"/foo.mf") _, err := mfa.openManifestReader(context.Background(), server.URL+"/foo.mf")
require.ErrorIs(t, err, errHTTPStatus) require.ErrorIs(t, err, errHTTPStatus)
assert.EqualError(t, err, assert.EqualError(t, err,
"failed to fetch "+server.URL+"/foo.mf: HTTP 404") "download manifest "+server.URL+"/foo.mf: unexpected HTTP status 404")
} }
func TestFetchManifestHTTPStatusMessage(t *testing.T) { func TestFetchManifestHTTPStatusMessage(t *testing.T) {
@@ -299,7 +325,7 @@ func TestFetchManifestHTTPStatusMessage(t *testing.T) {
return cmd.Run(context.Background(), []string{cmdFetch, server.URL}) return cmd.Run(context.Background(), []string{cmdFetch, server.URL})
}) })
require.ErrorIs(t, err, errHTTPStatus) require.ErrorIs(t, err, errHTTPStatus)
assert.EqualError(t, err, "failed to fetch manifest: HTTP 404") assert.EqualError(t, err, "download manifest: unexpected HTTP status 404")
} }
func TestFetchFileHTTPStatusMessage(t *testing.T) { func TestFetchFileHTTPStatusMessage(t *testing.T) {
@@ -317,7 +343,108 @@ func TestFetchFileHTTPStatusMessage(t *testing.T) {
&mfer.MFFilePath{}, nil) &mfer.MFFilePath{}, nil)
}) })
require.ErrorIs(t, err, errHTTPStatus) require.ErrorIs(t, err, errHTTPStatus)
assert.EqualError(t, err, "HTTP 500") assert.EqualError(t, err, "unexpected HTTP status 500")
}
// TestCheckCorruptManifestMessage runs check on a file that is not a
// manifest.
func TestCheckCorruptManifestMessage(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
require.NoError(t, afero.WriteFile(fs, "/bad.mf", []byte("not a manifest"), 0o644))
mfa := &CLIApp{Fs: fs}
cmd := mfa.checkCommand()
cmd.Action = mfa.checkManifestOperation
// checkManifestOperation logs to the process-global logger.
err := runLocked(func() error {
return cmd.Run(context.Background(), []string{cmdCheck, "/bad.mf"})
})
assert.EqualError(t, err, "load manifest: invalid file format")
}
// TestListMissingManifestMessage runs list on a manifest file that does not
// exist.
func TestListMissingManifestMessage(t *testing.T) {
t.Parallel()
mfa := &CLIApp{Fs: afero.NewMemMapFs()}
cmd := mfa.listCommand()
// listManifestOperation sets the process-global log level.
err := runLocked(func() error {
return cmd.Run(context.Background(), []string{cmdList, "/nope.mf"})
})
require.ErrorIs(t, err, os.ErrNotExist)
assert.EqualError(t, err, "open /nope.mf: file does not exist")
}
// TestFetchHashMismatchMessage runs fetch against a server that sends a
// listed file with other content of the same size.
func TestFetchHashMismatchMessage(t *testing.T) {
t.Parallel()
manifest := builtManifest(t, map[string][]byte{testFileTxt: []byte("listed")})
server := httptest.NewServer(fetchTestHandler(manifest,
map[string][]byte{testFileTxt: []byte("served")}))
defer server.Close()
mfa := &CLIApp{Fs: afero.NewMemMapFs(), maxManifestSize: mfer.MaxManifestSize}
cmd := mfa.fetchCommand()
cmd.Action = mfa.fetchManifestOperation
// fetchManifestOperation logs to the process-global logger.
err := runLocked(func() error {
return cmd.Run(context.Background(),
[]string{cmdFetch, "--" + flagDest, t.TempDir(), server.URL})
})
require.ErrorIs(t, err, errHashMismatch)
assert.EqualError(t, err, "download "+testFileTxt+": hash mismatch")
}
// TestFreshenBackslashPathMessage runs freshen on a tree that has gained a
// file whose name holds a backslash, which a manifest path may not contain.
func TestFreshenBackslashPathMessage(t *testing.T) {
t.Parallel()
fs := afero.NewOsFs()
root, manifestPath := setupFreshenDir(t, fs,
map[string]string{testFileTxt: "content"})
writeTestFile(t, fs, filepath.Join(root, `a\b.txt`), "new")
mfa := &CLIApp{Fs: fs}
cmd := mfa.freshenCommand()
cmd.Action = mfa.freshenManifestOperation
// freshenManifestOperation logs to the process-global logger.
err := runLocked(func() error {
return cmd.Run(context.Background(),
[]string{cmdFreshen, testFlagBase, root, manifestPath})
})
assert.EqualError(t, err,
`path "a\\b.txt" contains backslash; use forward slashes only`)
}
// TestFreshenReadErrorMessage has freshen hash a directory as though it
// were a file, so reading it fails.
func TestFreshenReadErrorMessage(t *testing.T) {
t.Parallel()
root := t.TempDir()
require.NoError(t, os.Mkdir(filepath.Join(root, "sub"), 0o750))
hasher := &freshenHasher{
fs: afero.NewOsFs(),
absBase: root,
builder: mfer.NewBuilder(),
}
err := hasher.processEntry(&freshenEntry{path: "sub", needsHash: true})
assert.EqualError(t, err,
"read "+filepath.Join(root, "sub")+": is a directory")
} }
func TestURLRequiredMessage(t *testing.T) { func TestURLRequiredMessage(t *testing.T) {
+6 -4
View File
@@ -18,6 +18,7 @@ type ExportEntry struct {
Hashes []string `json:"hashes"` Hashes []string `json:"hashes"`
Mtime *string `json:"mtime,omitempty"` Mtime *string `json:"mtime,omitempty"`
Ctime *string `json:"ctime,omitempty"` Ctime *string `json:"ctime,omitempty"`
Mode string `json:"mode"` // octal, "0000" when none was recorded
} }
func (mfa *CLIApp) exportManifestOperation( func (mfa *CLIApp) exportManifestOperation(
@@ -25,12 +26,12 @@ func (mfa *CLIApp) exportManifestOperation(
) error { ) error {
pathOrURL, err := mfa.resolveManifestArg(cmd) pathOrURL, err := mfa.resolveManifestArg(cmd)
if err != nil { if err != nil {
return fmt.Errorf("export: %w", err) return err
} }
rc, err := mfa.openManifestReader(ctx, pathOrURL) rc, err := mfa.openManifestReader(ctx, pathOrURL)
if err != nil { if err != nil {
return fmt.Errorf("export: %w", err) return err
} }
defer func() { _ = rc.Close() }() defer func() { _ = rc.Close() }()
@@ -38,7 +39,7 @@ func (mfa *CLIApp) exportManifestOperation(
//nolint:contextcheck // mfer loads a manifest without a context //nolint:contextcheck // mfer loads a manifest without a context
manifest, err := mfer.NewManifestFromReader(rc) manifest, err := mfer.NewManifestFromReader(rc)
if err != nil { if err != nil {
return fmt.Errorf("export: failed to parse manifest: %w", err) return fmt.Errorf("parse manifest: %w", err)
} }
files := manifest.Files() files := manifest.Files()
@@ -49,6 +50,7 @@ func (mfa *CLIApp) exportManifestOperation(
Path: f.GetPath(), Path: f.GetPath(),
Size: f.GetSize(), Size: f.GetSize(),
Hashes: make([]string, 0, len(f.GetHashes())), Hashes: make([]string, 0, len(f.GetHashes())),
Mode: fmt.Sprintf("%04o", f.GetMode()),
} }
for _, h := range f.GetHashes() { for _, h := range f.GetHashes() {
@@ -74,7 +76,7 @@ func (mfa *CLIApp) exportManifestOperation(
err = enc.Encode(entries) err = enc.Encode(entries)
if err != nil { if err != nil {
return fmt.Errorf("export: failed to encode JSON: %w", err) return fmt.Errorf("encode JSON: %w", err)
} }
return nil return nil
+1 -1
View File
@@ -132,7 +132,7 @@ func TestListFromHTTPURL(t *testing.T) {
exitCode := runCLI(&RunOptions{ exitCode := runCLI(&RunOptions{
Appname: testApp, Appname: testApp,
Args: []string{testApp, "list", server.URL + "/index.mf"}, Args: []string{testApp, cmdList, server.URL + "/index.mf"},
Stdin: &bytes.Buffer{}, Stdin: &bytes.Buffer{},
Stdout: &stdout, Stdout: &stdout,
Stderr: &stderr, Stderr: &stderr,
+74 -40
View File
@@ -95,6 +95,9 @@ var (
// writes another file. // writes another file.
errNameClash = errors.New( errNameClash = errors.New(
"manifest lists a file where fetch writes another file") "manifest lists a file where fetch writes another file")
// errModeOutOfRange indicates a manifest that lists a mode with more
// than the permission bits, such as setuid.
errModeOutOfRange = errors.New("manifest lists a mode outside 0777")
) )
// DownloadProgress reports the progress of a single file download. // DownloadProgress reports the progress of a single file download.
@@ -236,7 +239,7 @@ func reportDownloadProgress(progress <-chan DownloadProgress, done chan<- struct
func manifestBaseURL(manifestURL string) (*url.URL, error) { func manifestBaseURL(manifestURL string) (*url.URL, error) {
parsed, err := url.Parse(manifestURL) parsed, err := url.Parse(manifestURL)
if err != nil { if err != nil {
return nil, fmt.Errorf("fetch: invalid manifest URL: %w", err) return nil, fmt.Errorf("invalid manifest URL: %w", err)
} }
// JoinPath cleans the path it builds, so ".." drops the manifest's // JoinPath cleans the path it builds, so ".." drops the manifest's
@@ -265,7 +268,7 @@ func downloadManifestFiles(
// Sanitize the path to prevent path traversal attacks // Sanitize the path to prevent path traversal attacks
localPath, err := sanitizePath(f.GetPath()) localPath, err := sanitizePath(f.GetPath())
if err != nil { if err != nil {
return 0, 0, fmt.Errorf("invalid path in manifest: %w", err) return 0, 0, fmt.Errorf("invalid file entry: %w", err)
} }
if alreadyPresent(dest, localPath, f) { if alreadyPresent(dest, localPath, f) {
@@ -281,7 +284,7 @@ func downloadManifestFiles(
err = downloadFile(ctx, client, fileURL, dest, localPath, f, progress) err = downloadFile(ctx, client, fileURL, dest, localPath, f, progress)
if err != nil { if err != nil {
return 0, 0, fmt.Errorf("failed to download %s: %w", f.GetPath(), err) return 0, 0, fmt.Errorf("download %s: %w", f.GetPath(), err)
} }
downloaded++ downloaded++
@@ -292,11 +295,11 @@ func downloadManifestFiles(
} }
// alreadyPresent reports whether localPath under dest is a regular file // alreadyPresent reports whether localPath under dest is a regular file
// with the size and one of the hashes the manifest lists for entry. It // with the size, the recorded mode if any, and one of the hashes the
// hashes the whole file, since a matching size alone would accept a // manifest lists for entry. It hashes the whole file, since a matching
// corrupted or partly written one. A file it cannot read, or reaches only // size alone would accept a corrupted or partly written one. A file it
// through a symlink, is not present: fetch downloads it, and the download // cannot read, or reaches only through a symlink, is not present: fetch
// reports the problem. // downloads it, and the download reports the problem.
func alreadyPresent(dest, localPath string, entry *mfer.MFFilePath) bool { func alreadyPresent(dest, localPath string, entry *mfer.MFFilePath) bool {
if checkNoSymlinks(dest, localPath) != nil { if checkNoSymlinks(dest, localPath) != nil {
return false return false
@@ -309,6 +312,12 @@ func alreadyPresent(dest, localPath string, entry *mfer.MFFilePath) bool {
return false return false
} }
// A recorded mode of 0 means none was recorded.
if entry.GetMode() != 0 &&
info.Mode().Perm() != os.FileMode(entry.GetMode()).Perm() {
return false
}
// G304: localPath is a relative path that sanitizePath keeps inside // G304: localPath is a relative path that sanitizePath keeps inside
// dest as text, and checkNoSymlinks just found no symlink in it. // dest as text, and checkNoSymlinks just found no symlink in it.
f, err := os.Open(path) //nolint:gosec // G304: see comment above f, err := os.Open(path) //nolint:gosec // G304: see comment above
@@ -352,7 +361,7 @@ func (mfa *CLIApp) fetchManifestOperation(
firstDelay: firstRetryDelay, firstDelay: firstRetryDelay,
} }
manifestData, files, err := fetchManifest(ctx, cmd, client, manifestURL) manifestData, files, err := mfa.fetchManifest(ctx, cmd, client, manifestURL)
if err != nil { if err != nil {
return err return err
} }
@@ -367,7 +376,7 @@ func (mfa *CLIApp) fetchManifestOperation(
err = os.MkdirAll(dest, dirPerms) err = os.MkdirAll(dest, dirPerms)
if err != nil { if err != nil {
return fmt.Errorf("failed to create destination directory %s: %w", dest, err) return err
} }
// Create progress channel and start progress reporter goroutine // Create progress channel and start progress reporter goroutine
@@ -394,7 +403,7 @@ func (mfa *CLIApp) fetchManifestOperation(
// "mfer check" can verify the tree later. // "mfer check" can verify the tree later.
err = saveManifest(dest, manifestData) err = saveManifest(dest, manifestData)
if err != nil { if err != nil {
return fmt.Errorf("failed to save manifest: %w", err) return fmt.Errorf("save manifest: %w", err)
} }
// Print summary // Print summary
@@ -414,40 +423,48 @@ func (mfa *CLIApp) fetchManifestOperation(
// fetchManifest downloads the manifest at manifestURL and parses it, // fetchManifest downloads the manifest at manifestURL and parses it,
// enforcing --require-signature if it is given and refusing a manifest // enforcing --require-signature if it is given and refusing a manifest
// that lists a file where fetch writes another. It returns the manifest as // that lists a file where fetch writes another or a mode outside 0777. It
// downloaded, to be saved once the files are in place, and the files it // returns the manifest as downloaded, to be saved once the files are in
// lists. // place, and the files it lists.
func fetchManifest( func (mfa *CLIApp) fetchManifest(
ctx context.Context, cmd *cli.Command, client retryingClient, manifestURL string, ctx context.Context, cmd *cli.Command, client retryingClient, manifestURL string,
) ([]byte, []*mfer.MFFilePath, error) { ) ([]byte, []*mfer.MFFilePath, error) {
log.Infof("fetching manifest from %s", manifestURL) log.Infof("fetching manifest from %s", manifestURL)
// Read the whole manifest before parsing it, so that a connection // Read the whole manifest before parsing it, so that a connection
// lost partway through is retried rather than reported as a bad // lost partway through is retried rather than reported as a bad
// manifest. // manifest. Reading stops one byte past mfa.maxManifestSize, which is
// enough to tell that the manifest is too large.
var manifestData []byte var manifestData []byte
err := client.get(ctx, manifestURL, func(resp *http.Response) error { err := client.get(ctx, manifestURL, func(resp *http.Response) error {
var readErr error var readErr error
manifestData, readErr = io.ReadAll(resp.Body) manifestData, readErr = io.ReadAll(
io.LimitReader(resp.Body, mfa.maxManifestSize+1))
return readErr return readErr
}) })
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("failed to fetch manifest: %w", err) return nil, nil, fmt.Errorf("download manifest: %w", err)
}
if int64(len(manifestData)) > mfa.maxManifestSize {
return nil, nil, fmt.Errorf("download manifest: %w of %d bytes",
errManifestTooLarge, mfa.maxManifestSize)
} }
// Parse manifest // Parse manifest
//nolint:contextcheck // mfer loads a manifest without a context //nolint:contextcheck // mfer loads a manifest without a context
manifest, err := mfer.NewManifestFromReader(bytes.NewReader(manifestData)) manifest, err := mfer.NewManifestFromReader(bytes.NewReader(manifestData))
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("failed to parse manifest: %w", err) return nil, nil, fmt.Errorf("parse manifest: %w", err)
} }
requiredSigner := cmd.String(flagRequireSignature) requiredSigner := cmd.String(flagRequireSignature)
if requiredSigner != "" { if requiredSigner != "" {
err = verifyFetchedSigner(ctx, manifestData, requiredSigner) //nolint:contextcheck // mfer loads a manifest without a context
err = verifyFetchedSigner(manifestData, requiredSigner)
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
} }
@@ -460,6 +477,16 @@ func fetchManifest(
return nil, nil, err return nil, nil, err
} }
// fetch sets each recorded mode on the file it writes, so a mode above
// 0777, which could carry setuid, setgid or sticky bits, is refused
// before any file is requested.
for _, f := range files {
if f.GetMode() > uint32(os.ModePerm) {
return nil, nil, fmt.Errorf("%w: %s (%#o)",
errModeOutOfRange, f.GetPath(), f.GetMode())
}
}
log.Infof("manifest contains %d files", len(files)) log.Infof("manifest contains %d files", len(files))
return manifestData, files, nil return manifestData, files, nil
@@ -519,9 +546,7 @@ func checkNoNameClash(files []*mfer.MFFilePath) error {
// exactly as check does. verifyRequiredSigner takes a Checker, which loads // exactly as check does. verifyRequiredSigner takes a Checker, which loads
// its manifest from a file, so the manifest is handed to it as a file in // its manifest from a file, so the manifest is handed to it as a file in
// memory. // memory.
func verifyFetchedSigner( func verifyFetchedSigner(manifestData []byte, requiredSigner string) error {
ctx context.Context, manifestData []byte, requiredSigner string,
) error {
memFs := afero.NewMemMapFs() memFs := afero.NewMemMapFs()
manifestPath := "/" + defaultManifestName manifestPath := "/" + defaultManifestName
@@ -530,17 +555,16 @@ func verifyFetchedSigner(
return err return err
} }
//nolint:contextcheck // mfer loads a manifest without a context
chk, err := mfer.NewChecker(&mfer.CheckerOptions{ chk, err := mfer.NewChecker(&mfer.CheckerOptions{
ManifestPath: manifestPath, ManifestPath: manifestPath,
BasePath: "/", BasePath: "/",
Fs: memFs, Fs: memFs,
}) })
if err != nil { if err != nil {
return fmt.Errorf("failed to load manifest: %w", err) return fmt.Errorf("load manifest: %w", err)
} }
return verifyRequiredSigner(ctx, chk, requiredSigner) return verifyRequiredSigner(chk, requiredSigner)
} }
// saveManifest writes the fetched manifest into dest under the default // saveManifest writes the fetched manifest into dest under the default
@@ -620,7 +644,7 @@ func sanitizePath(p string) (string, error) {
func checkNoSymlinks(dest, p string) error { func checkNoSymlinks(dest, p string) error {
current := dest current := dest
for _, part := range strings.Split(p, string(filepath.Separator)) { for part := range strings.SplitSeq(p, string(filepath.Separator)) {
current = filepath.Join(current, part) current = filepath.Join(current, part)
info, err := os.Lstat(current) info, err := os.Lstat(current)
@@ -629,7 +653,7 @@ func checkNoSymlinks(dest, p string) error {
} }
if err != nil { if err != nil {
return fmt.Errorf("failed to check %s for a symlink: %w", current, err) return err
} }
if info.Mode()&os.ModeSymlink != 0 { if info.Mode()&os.ModeSymlink != 0 {
@@ -746,7 +770,7 @@ func tempPathFor(localPath string) string {
func verifyDownloadedHash(digest []byte, entry *mfer.MFFilePath) error { func verifyDownloadedHash(digest []byte, entry *mfer.MFFilePath) error {
computed, err := multihash.Encode(digest, multihash.SHA2_256) computed, err := multihash.Encode(digest, multihash.SHA2_256)
if err != nil { if err != nil {
return fmt.Errorf("failed to encode hash: %w", err) return fmt.Errorf("encode hash: %w", err)
} }
for _, hash := range entry.GetHashes() { for _, hash := range entry.GetHashes() {
@@ -773,7 +797,7 @@ func downloadFile(
// so every entry point to downloadFile gets the same treatment. // so every entry point to downloadFile gets the same treatment.
localPath, err := sanitizePath(localPath) localPath, err := sanitizePath(localPath)
if err != nil { if err != nil {
return fmt.Errorf("invalid path: %w", err) return fmt.Errorf("invalid file entry: %w", err)
} }
// Create parent directories if needed // Create parent directories if needed
@@ -788,7 +812,7 @@ func downloadFile(
err = os.MkdirAll(dir, dirPerms) err = os.MkdirAll(dir, dirPerms)
if err != nil { if err != nil {
return fmt.Errorf("failed to create directory %s: %w", dir, err) return err
} }
} }
@@ -822,7 +846,7 @@ func createTempFile(dest, tmpPath string) (*os.File, error) {
out, err := os.OpenFile( //nolint:gosec // G304: see comment above out, err := os.OpenFile( //nolint:gosec // G304: see comment above
path, os.O_RDWR|os.O_CREATE|os.O_EXCL, filePerms) path, os.O_RDWR|os.O_CREATE|os.O_EXCL, filePerms)
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to create temp file: %w", err) return nil, err
} }
return out, nil return out, nil
@@ -835,12 +859,7 @@ func moveIntoPlace(dest, tmpPath, localPath string) error {
return err return err
} }
err = os.Rename(filepath.Join(dest, tmpPath), filepath.Join(dest, localPath)) return os.Rename(filepath.Join(dest, tmpPath), filepath.Join(dest, localPath))
if err != nil {
return fmt.Errorf("failed to rename temp file: %w", err)
}
return nil
} }
// saveResponse writes resp's body to tmpPath, verifies it against entry, // saveResponse writes resp's body to tmpPath, verifies it against entry,
@@ -866,6 +885,19 @@ func saveResponse(
return err return err
} }
// A recorded mode of 0 means none was recorded. Of any other, only the
// permission bits are set, whatever the umask, so setuid, setgid and
// sticky never are, whichever caller passed the entry.
if entry.GetMode() != 0 {
err = out.Chmod(os.FileMode(entry.GetMode()).Perm())
if err != nil {
_ = out.Close()
_ = os.Remove(filepath.Join(dest, tmpPath))
return err
}
}
// Set up hash computation // Set up hash computation
h := sha256.New() h := sha256.New()
@@ -878,8 +910,10 @@ func saveResponse(
progress: progress, progress: progress,
} }
// Copy content while hashing and reporting progress // Copy content while hashing and reporting progress. One byte past
written, copyErr := io.Copy(pw, resp.Body) // the listed size is enough for finishDownload to report a size
// mismatch.
written, copyErr := io.Copy(pw, io.LimitReader(resp.Body, expectedSize+1))
// Close file before checking errors (to flush writes) // Close file before checking errors (to flush writes)
closeErr := out.Close() closeErr := out.Close()
+288 -20
View File
@@ -4,6 +4,7 @@ package cli
import ( import (
"bytes" "bytes"
"context" "context"
"crypto/sha256"
"fmt" "fmt"
"io" "io"
"maps" "maps"
@@ -18,10 +19,15 @@ import (
"sync/atomic" "sync/atomic"
"testing" "testing"
"time" "time"
"uuid"
"github.com/klauspost/compress/zstd"
"github.com/multiformats/go-multihash"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
urfcli "github.com/urfave/cli/v3"
"google.golang.org/protobuf/proto"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -181,12 +187,7 @@ func chdirTemp(t *testing.T) string {
t.Helper() t.Helper()
destDir := t.TempDir() destDir := t.TempDir()
t.Chdir(destDir)
origDir, err := os.Getwd()
require.NoError(t, err)
require.NoError(t, os.Chdir(destDir))
t.Cleanup(func() { _ = os.Chdir(origDir) })
return destDir return destDir
} }
@@ -441,6 +442,76 @@ func TestFetchSizeMismatch(t *testing.T) {
"temp file should be cleaned up on size mismatch") "temp file should be cleaned up on size mismatch")
} }
// zeros is an io.Reader of zero bytes without end.
type zeros struct{}
func (zeros) Read(p []byte) (int, error) {
clear(p)
return len(p), nil
}
// TestFetchStopsReadingFilePastListedSize serves a body that never ends
// for a file listed at 16 bytes. fetch must stop reading one byte past
// the listed size, report the size mismatch and remove its temp file.
//
//nolint:paralleltest // changes the process-global working directory
func TestFetchStopsReadingFilePastListedSize(t *testing.T) {
server := httptest.NewServer(
http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = io.Copy(w, zeros{})
}))
defer server.Close()
chdirTemp(t)
err := downloadFile(context.Background(), testClient(),
server.URL+"/"+testFileTxt, ".", testFileTxt,
&mfer.MFFilePath{Path: testFileTxt, Size: 16}, nil)
require.ErrorIs(t, err, errSizeMismatch)
require.EqualError(t, err, "size mismatch: expected 16 bytes, got 17")
assert.NoFileExists(t, tempPathFor(testFileTxt))
}
// TestManifestDownloadStopsAtLimit serves a manifest that never ends to
// fetch and to check, with the most they download of a manifest lowered
// to 64 KiB. Each must stop reading at that limit, fail with an error
// naming it and leave no temp file.
func TestManifestDownloadStopsAtLimit(t *testing.T) {
server := httptest.NewServer(
http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = io.Copy(w, zeros{})
}))
defer server.Close()
tmpDir := t.TempDir()
t.Setenv("TMPDIR", tmpDir)
mfa := &CLIApp{Fs: afero.NewOsFs(), maxManifestSize: 64 << 10}
fetch := mfa.fetchCommand()
fetch.Action = mfa.fetchManifestOperation
check := mfa.checkCommand()
check.Action = mfa.checkManifestOperation
for _, cmd := range []*urfcli.Command{fetch, check} {
// Both operations log to the process-global logger.
err := runLocked(func() error {
return cmd.Run(context.Background(),
[]string{cmd.Name, server.URL + "/index.mf"})
})
require.ErrorIs(t, err, errManifestTooLarge, cmd.Name)
require.EqualError(t, err,
"download manifest: file exceeds maximum allowed size of 65536 bytes",
cmd.Name)
}
leftover, err := os.ReadDir(tmpDir)
require.NoError(t, err)
assert.Empty(t, leftover)
}
//nolint:paralleltest // changes the process-global working directory //nolint:paralleltest // changes the process-global working directory
func TestFetchProgress(t *testing.T) { func TestFetchProgress(t *testing.T) {
// Create source filesystem with a larger test file // Create source filesystem with a larger test file
@@ -535,27 +606,27 @@ func TestFetchRefusesSymlinks(t *testing.T) {
}{ }{
{ {
"parent directory", "sub/deeper/file.txt", "sub", ".", "parent directory", "sub/deeper/file.txt", "sub", ".",
"failed to download sub/deeper/file.txt", "download sub/deeper/file.txt",
}, },
{ {
"directory inside a plain directory", "docs/data/passwd", "docs/data", ".", "directory inside a plain directory", "docs/data/passwd", "docs/data", ".",
"failed to download docs/data/passwd", "download docs/data/passwd",
}, },
{ {
"temp file", testFileTxt, ".file.txt.tmp", newFile, "temp file", testFileTxt, ".file.txt.tmp", newFile,
"failed to download " + testFileTxt, "download " + testFileTxt,
}, },
{ {
"file", testFileTxt, testFileTxt, newFile, "file", testFileTxt, testFileTxt, newFile,
"failed to download " + testFileTxt, "download " + testFileTxt,
}, },
{ {
"manifest temp file", testFileTxt, tempPathFor(defaultManifestName), newFile, "manifest temp file", testFileTxt, tempPathFor(defaultManifestName), newFile,
"failed to save manifest", "save manifest",
}, },
{ {
"manifest", testFileTxt, defaultManifestName, newFile, "manifest", testFileTxt, defaultManifestName, newFile,
"failed to save manifest", "save manifest",
}, },
} }
@@ -622,7 +693,7 @@ func TestFetchDoesNotSkipThroughSymlink(t *testing.T) {
}, afero.NewOsFs()) }, afero.NewOsFs())
assert.Equal(t, 1, runCLI(opts)) assert.Equal(t, 1, runCLI(opts))
assert.Contains(t, testStderr(t, opts), assert.Contains(t, testStderr(t, opts),
"failed to download sub/"+testFileTxt+": symlink in path not allowed: "+link) "download sub/"+testFileTxt+": symlink in path not allowed: "+link)
assert.Equal(t, map[string][]byte{testFileTxt: content}, filesUnder(t, outside)) assert.Equal(t, map[string][]byte{testFileTxt: content}, filesUnder(t, outside))
} }
@@ -692,7 +763,8 @@ func TestGetRetriesTransientStatusesOnly(t *testing.T) {
err := getNothing(testClient(), server.URL, 10*time.Second) err := getNothing(testClient(), server.URL, 10*time.Second)
require.ErrorIs(t, err, errHTTPStatus) require.ErrorIs(t, err, errHTTPStatus)
require.EqualError(t, err, fmt.Sprintf("HTTP %d", tt.status)) require.EqualError(t, err,
fmt.Sprintf("unexpected HTTP status %d", tt.status))
assert.Equal(t, tt.requests, requests.Load()) assert.Equal(t, tt.requests, requests.Load())
}) })
} }
@@ -1117,8 +1189,10 @@ func TestFetchIntoDest(t *testing.T) {
// TestFetchRequireSignature runs fetch with --require-signature. A // TestFetchRequireSignature runs fetch with --require-signature. A
// manifest that is unsigned, or signed by another key, must stop fetch // manifest that is unsigned, or signed by another key, must stop fetch
// with check's message before it downloads or writes anything; the // with check's message before it downloads or writes anything; the
// required key lets it through. The signed cases need gpg and are skipped // required key lets it through. A manifest signed by another key whose
// without it, as the other signing tests are. // embedded public key block also holds the required key must stop fetch
// too. The signed cases need gpg and are skipped without it, as the other
// signing tests are.
// //
//nolint:paralleltest // signedManifest calls t.Setenv, which bars t.Parallel //nolint:paralleltest // signedManifest calls t.Setenv, which bars t.Parallel
func TestFetchRequireSignature(t *testing.T) { func TestFetchRequireSignature(t *testing.T) {
@@ -1133,9 +1207,7 @@ func TestFetchRequireSignature(t *testing.T) {
t.Run("signed", func(t *testing.T) { t.Run("signed", func(t *testing.T) {
manifest := signedManifest(t, files) manifest := signedManifest(t, files)
signer, err := signedChecker(t, manifest). signer := string(signedChecker(t, manifest).Signer())
ExtractEmbeddedSigningKeyFP(context.Background())
require.NoError(t, err)
assertFetchRefused(t, manifest, files, assertFetchRefused(t, manifest, files,
"embedded signing key fingerprint "+signer+" does not match required "+msgFpB, "embedded signing key fingerprint "+signer+" does not match required "+msgFpB,
@@ -1153,6 +1225,15 @@ func TestFetchRequireSignature(t *testing.T) {
require.Equal(t, 0, runCLI(opts), testStderr(t, opts)) require.Equal(t, 0, runCLI(opts), testStderr(t, opts))
assert.Equal(t, files[testFileTxt], filesUnder(t, dest)[testFileTxt]) assert.Equal(t, files[testFileTxt], filesUnder(t, dest)[testFileTxt])
}) })
t.Run("signed by another key embedded after the required one", func(t *testing.T) {
manifest, required := manifestSignedByAnotherKey(t, files)
assertFetchRefused(t, manifest, files,
"parse manifest: "+
"embedded public key block must hold exactly one key, found 2",
"--"+flagRequireSignature, required)
})
} }
// TestFetchRefusesListedManifestName fetches manifests that list, at the // TestFetchRefusesListedManifestName fetches manifests that list, at the
@@ -1299,6 +1380,193 @@ func TestFetchRefusesNamesEqualIgnoringCase(t *testing.T) {
}) })
} }
// TestFetchSetsRecordedMode fetches a tree whose manifest records the
// modes 0640 and 0755. Each file must get its mode whatever the umask, and
// check must pass on the result. After one file's mode is changed, a
// second fetch must download that file again, and only it, to restore its
// mode.
func TestFetchSetsRecordedMode(t *testing.T) {
t.Parallel()
files := map[string][]byte{
testFileTxt: []byte("a file"),
"tool.sh": []byte("#!/bin/sh\n"),
}
modes := map[string]os.FileMode{testFileTxt: 0o640, "tool.sh": 0o755}
sourceFs := afero.NewMemMapFs()
for p, content := range files {
require.NoError(t, afero.WriteFile(sourceFs, "/"+p, content, modes[p]))
}
scanner := mfer.NewScannerWithOptions(&mfer.ScannerOptions{
Fs: sourceFs,
IncludePermissions: true,
})
require.NoError(t, scanner.EnumerateFS(sourceFs, "/", nil))
var manifest bytes.Buffer
require.NoError(t, scanner.ToManifest(context.Background(), &manifest, nil))
tree := fetchTestHandler(manifest.Bytes(), files)
var (
mu sync.Mutex
requested []string
)
server := httptest.NewServer(
http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
mu.Lock()
requested = append(requested, r.URL.Path)
mu.Unlock()
tree.ServeHTTP(w, r)
}))
defer server.Close()
dest := t.TempDir()
fetch := []string{testApp, cmdFetch, "-q", "--" + flagDest, dest, server.URL}
opts := testOpts(fetch, afero.NewOsFs())
require.Equal(t, 0, runCLI(opts), testStderr(t, opts))
for p, mode := range modes {
info, err := os.Stat(filepath.Join(dest, p))
require.NoError(t, err)
assert.Equal(t, mode, info.Mode().Perm(), p)
}
opts = testOpts([]string{
testApp, cmdCheck, "-q", testFlagBase, dest,
filepath.Join(dest, defaultManifestName),
}, afero.NewOsFs())
require.Equal(t, 0, runCLI(opts), testStderr(t, opts))
require.NoError(t, os.Chmod(filepath.Join(dest, testFileTxt), 0o600))
mu.Lock()
requested = nil
mu.Unlock()
opts = testOpts(fetch, afero.NewOsFs())
require.Equal(t, 0, runCLI(opts), testStderr(t, opts))
info, err := os.Stat(filepath.Join(dest, testFileTxt))
require.NoError(t, err)
assert.Equal(t, os.FileMode(0o640), info.Mode().Perm())
mu.Lock()
defer mu.Unlock()
assert.ElementsMatch(t,
[]string{"/" + defaultManifestName, "/" + testFileTxt}, requested)
}
// TestFetchRefusesModeOutsidePermissionBits fetches manifests that record
// a mode with the setuid bit, once as Unix writes it (04755) and once as
// Go keeps it in os.FileMode. fetch must refuse both before it creates the
// destination or requests any file.
func TestFetchRefusesModeOutsidePermissionBits(t *testing.T) {
t.Parallel()
files := map[string][]byte{testFileTxt: []byte("a file")}
assertFetchRefused(t, manifestWithMode(t, testFileTxt, files[testFileTxt], 0o4755),
files, "manifest lists a mode outside 0777: file.txt (04755)")
assertFetchRefused(t,
manifestWithMode(t, testFileTxt, files[testFileTxt],
uint32(os.ModeSetuid|0o755)),
files, "manifest lists a mode outside 0777: file.txt (040000755)")
}
// TestDownloadFileSetsOnlyPermissionBits downloads a file whose entry
// records mode 0755 together with setuid, setgid or sticky, as Go keeps
// them in os.FileMode. fetchManifest would refuse such an entry; called
// directly, downloadFile must still set only the permission bits.
func TestDownloadFileSetsOnlyPermissionBits(t *testing.T) {
t.Parallel()
content := []byte("#!/bin/sh\n")
digest := sha256.Sum256(content)
hash, err := multihash.Encode(digest[:], multihash.SHA2_256)
require.NoError(t, err)
server := httptest.NewServer(
http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write(content)
}))
defer server.Close()
for _, special := range []os.FileMode{os.ModeSetuid, os.ModeSetgid, os.ModeSticky} {
entry := &mfer.MFFilePath{
Path: testFileTxt,
Size: int64(len(content)),
Hashes: []*mfer.MFFileChecksum{{MultiHash: hash}},
Mode: uint32(special | 0o755),
}
dest := t.TempDir()
err := downloadFile(context.Background(), testClient(),
server.URL+"/"+testFileTxt, dest, testFileTxt, entry, nil)
require.NoError(t, err, special)
info, err := os.Stat(filepath.Join(dest, testFileTxt))
require.NoError(t, err)
assert.Equal(t, os.FileMode(0o755), info.Mode(), special)
}
}
// manifestWithMode returns a manifest listing one file, path, with content
// and the given recorded mode. It is assembled by hand, since the builder
// never records a mode outside 0777.
func manifestWithMode(t *testing.T, path string, content []byte, mode uint32) []byte {
t.Helper()
digest := sha256.Sum256(content)
hash, err := multihash.Encode(digest[:], multihash.SHA2_256)
require.NoError(t, err)
id := uuid.NewV4()
inner, err := proto.Marshal(&mfer.MFFile{
Version: mfer.MFFile_VERSION_ONE,
Files: []*mfer.MFFilePath{{
Path: path,
Size: int64(len(content)),
Hashes: []*mfer.MFFileChecksum{{MultiHash: hash}},
Mode: mode,
}},
Uuid: id[:],
})
require.NoError(t, err)
encoder, err := zstd.NewWriter(nil)
require.NoError(t, err)
compressed := encoder.EncodeAll(inner, nil)
require.NoError(t, encoder.Close())
sum := sha256.Sum256(compressed)
outer, err := proto.Marshal(&mfer.MFFileOuter{
Version: mfer.MFFileOuter_VERSION_ONE,
CompressionType: mfer.MFFileOuter_COMPRESSION_ZSTD,
Size: int64(len(inner)),
Sha256: sum[:],
Uuid: id[:],
InnerMessage: compressed,
})
require.NoError(t, err)
return append([]byte(mfer.MAGIC), outer...)
}
// builtManifest returns a manifest of files, built directly rather than // builtManifest returns a manifest of files, built directly rather than
// scanned, since a scan lists no hidden files and never a path starting // scanned, since a scan lists no hidden files and never a path starting
// with "./". // with "./".
@@ -1309,7 +1577,7 @@ func builtManifest(t *testing.T, files map[string][]byte) []byte {
for p, content := range files { for p, content := range files {
_, err := builder.AddFile(mfer.RelFilePath(p), mfer.FileSize(len(content)), _, err := builder.AddFile(mfer.RelFilePath(p), mfer.FileSize(len(content)),
mfer.ModTime(time.Now()), bytes.NewReader(content), nil) mfer.ModTime(time.Now()), 0, bytes.NewReader(content), nil)
require.NoError(t, err) require.NoError(t, err)
} }
+53 -52
View File
@@ -48,6 +48,7 @@ type freshenEntry struct {
path string path string
size int64 size int64
mtime time.Time mtime time.Time
mode fs.FileMode // mode to record, 0 for none
needsHash bool // true if new or changed needsHash bool // true if new or changed
existing *mfer.MFFilePath // existing manifest entry if unchanged existing *mfer.MFFilePath // existing manifest entry if unchanged
} }
@@ -55,13 +56,14 @@ type freshenEntry struct {
// freshenScanner walks the filesystem and compares it against the // freshenScanner walks the filesystem and compares it against the
// entries of an existing manifest. // entries of an existing manifest.
type freshenScanner struct { type freshenScanner struct {
fs afero.Fs fs afero.Fs
absBase string absBase string
excluded []fs.FileInfo // files left out of the listing excluded []fs.FileInfo // files left out of the listing
includeDotfiles bool includeDotfiles bool
followSymlinks bool followSymlinks bool
showProgress bool includePermissions bool
existingByPath map[string]*mfer.MFFilePath showProgress bool
existingByPath map[string]*mfer.MFFilePath
entries []*freshenEntry entries []*freshenEntry
scanCount int64 scanCount int64
@@ -93,6 +95,12 @@ func (s *freshenScanner) resolveSymlink(path string) (fs.FileInfo, bool) {
// recordEntry classifies a scanned file as changed, unchanged, or added // recordEntry classifies a scanned file as changed, unchanged, or added
// relative to the existing manifest. // relative to the existing manifest.
func (s *freshenScanner) recordEntry(relPath string, info fs.FileInfo) { func (s *freshenScanner) recordEntry(relPath string, info fs.FileInfo) {
// A mode of 0 records 0000, which means none was recorded.
var mode fs.FileMode
if s.includePermissions {
mode = info.Mode().Perm()
}
existing, inManifest := s.existingByPath[relPath] existing, inManifest := s.existingByPath[relPath]
if !inManifest { if !inManifest {
s.added++ s.added++
@@ -102,16 +110,17 @@ func (s *freshenScanner) recordEntry(relPath string, info fs.FileInfo) {
path: relPath, path: relPath,
size: info.Size(), size: info.Size(),
mtime: info.ModTime(), mtime: info.ModTime(),
mode: mode,
needsHash: true, needsHash: true,
}) })
return return
} }
// Check if changed (size or mtime). An entry with no recorded mtime // Check if changed (size, mtime, or the mode to record). An entry
// cannot be compared, so it counts as changed and gets re-hashed; // with no recorded mtime cannot be compared, so it counts as changed
// silently treating the absent mtime as the Unix epoch would classify // and gets re-hashed; silently treating the absent mtime as the Unix
// every such entry as changed without saying why. // epoch would classify every such entry as changed without saying why.
existingMtime, haveMtime := entryMtime(existing) existingMtime, haveMtime := entryMtime(existing)
if !haveMtime { if !haveMtime {
log.Debugf("%s: manifest entry has no mtime, treating as changed", log.Debugf("%s: manifest entry has no mtime, treating as changed",
@@ -119,7 +128,8 @@ func (s *freshenScanner) recordEntry(relPath string, info fs.FileInfo) {
} }
if !haveMtime || existing.GetSize() != info.Size() || if !haveMtime || existing.GetSize() != info.Size() ||
!existingMtime.Equal(info.ModTime()) { !existingMtime.Equal(info.ModTime()) ||
fs.FileMode(existing.GetMode()) != mode {
s.changed++ s.changed++
log.Verbosef("M %s", relPath) log.Verbosef("M %s", relPath)
@@ -127,6 +137,7 @@ func (s *freshenScanner) recordEntry(relPath string, info fs.FileInfo) {
path: relPath, path: relPath,
size: info.Size(), size: info.Size(),
mtime: info.ModTime(), mtime: info.ModTime(),
mode: mode,
needsHash: true, needsHash: true,
}) })
} else { } else {
@@ -136,6 +147,7 @@ func (s *freshenScanner) recordEntry(relPath string, info fs.FileInfo) {
path: relPath, path: relPath,
size: info.Size(), size: info.Size(),
mtime: info.ModTime(), mtime: info.ModTime(),
mode: mode,
needsHash: false, needsHash: false,
existing: existing, existing: existing,
}) })
@@ -153,8 +165,7 @@ func (s *freshenScanner) walk(path string, info fs.FileInfo, walkErr error) erro
// Get relative path // Get relative path
relPath, err := filepath.Rel(s.absBase, path) relPath, err := filepath.Rel(s.absBase, path)
if err != nil { if err != nil {
return fmt.Errorf( return err
"freshen: failed to compute relative path for %s: %w", path, err)
} }
// Handle dotfiles // Handle dotfiles
@@ -268,13 +279,8 @@ func (h *freshenHasher) reportProgress(n int64) {
// processEntry hashes the entry if needed and adds it to the builder. // processEntry hashes the entry if needed and adds it to the builder.
func (h *freshenHasher) processEntry(e *freshenEntry) error { func (h *freshenHasher) processEntry(e *freshenEntry) error {
if !e.needsHash { if !e.needsHash {
// Use existing entry // Use existing entry; the error names the entry
err := addExistingToBuilder(h.builder, e.existing) return addExistingToBuilder(h.builder, e.existing)
if err != nil {
return fmt.Errorf("failed to add %s: %w", e.path, err)
}
return nil
} }
// Need to read and hash the file // Need to read and hash the file
@@ -282,26 +288,21 @@ func (h *freshenHasher) processEntry(e *freshenEntry) error {
f, err := h.fs.Open(absPath) f, err := h.fs.Open(absPath)
if err != nil { if err != nil {
return fmt.Errorf("failed to open %s: %w", e.path, err) return err
} }
hash, bytesRead, err := hashFile(f, h.reportProgress) hash, bytesRead, err := hashFile(f, h.reportProgress)
_ = f.Close() _ = f.Close()
if err != nil { if err != nil {
return fmt.Errorf("failed to hash %s: %w", e.path, err) return err
} }
h.hashedBytes += bytesRead h.hashedBytes += bytesRead
h.hashedFiles++ h.hashedFiles++
// Add to builder with computed hash // Add to builder with computed hash; a refused path is named in the error
err = addFileToBuilder(h.builder, e.path, e.size, e.mtime, hash) return addFileToBuilder(h.builder, e.path, e.size, e.mtime, e.mode, hash)
if err != nil {
return fmt.Errorf("failed to add %s: %w", e.path, err)
}
return nil
} }
// writeFreshenedManifest writes the manifest atomically (write to a // writeFreshenedManifest writes the manifest atomically (write to a
@@ -313,7 +314,7 @@ func writeFreshenedManifest(
outFile, err := afs.Create(tmpPath) outFile, err := afs.Create(tmpPath)
if err != nil { if err != nil {
return fmt.Errorf("failed to create temp file: %w", err) return err
} }
err = builder.Build(ctx, outFile) err = builder.Build(ctx, outFile)
@@ -322,7 +323,7 @@ func writeFreshenedManifest(
if err != nil { if err != nil {
_ = afs.Remove(tmpPath) _ = afs.Remove(tmpPath)
return fmt.Errorf("failed to write manifest: %w", err) return fmt.Errorf("build manifest: %w", err)
} }
// Rename temp to final // Rename temp to final
@@ -330,7 +331,7 @@ func writeFreshenedManifest(
if err != nil { if err != nil {
_ = afs.Remove(tmpPath) _ = afs.Remove(tmpPath)
return fmt.Errorf("failed to rename manifest: %w", err) return err
} }
return nil return nil
@@ -379,13 +380,14 @@ func (mfa *CLIApp) freshenScan(
} }
scanner := &freshenScanner{ scanner := &freshenScanner{
fs: mfa.Fs, fs: mfa.Fs,
absBase: absBase, absBase: absBase,
excluded: excluded, excluded: excluded,
includeDotfiles: cmd.Bool("include-dotfiles"), includeDotfiles: cmd.Bool("include-dotfiles"),
followSymlinks: cmd.Bool("follow-symlinks"), followSymlinks: cmd.Bool("follow-symlinks"),
showProgress: showProgress, includePermissions: cmd.Bool(flagIncludePermissions),
existingByPath: existingByPath, showProgress: showProgress,
existingByPath: existingByPath,
} }
err := afero.Walk(mfa.Fs, absBase, scanner.walk) err := afero.Walk(mfa.Fs, absBase, scanner.walk)
@@ -395,7 +397,7 @@ func (mfa *CLIApp) freshenScan(
} }
if err != nil { if err != nil {
return nil, 0, fmt.Errorf("failed to scan filesystem: %w", err) return nil, 0, fmt.Errorf("scan filesystem: %w", err)
} }
// Remaining entries in existingByPath are removed files // Remaining entries in existingByPath are removed files
@@ -464,7 +466,7 @@ func (mfa *CLIApp) loadExistingEntries(
Fs: mfa.Fs, Fs: mfa.Fs,
}) })
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to load manifest: %w", err) return nil, fmt.Errorf("load manifest: %w", err)
} }
existingFiles := manifest.Files() existingFiles := manifest.Files()
@@ -484,13 +486,12 @@ func (mfa *CLIApp) freshenManifestOperation(
) error { ) error {
log.Debug("freshenManifestOperation()") log.Debug("freshenManifestOperation()")
basePath := cmd.String("base")
showProgress := cmd.Bool("progress") showProgress := cmd.Bool("progress")
// Find manifest file // Find manifest file
manifestPath, err := mfa.resolveFreshenManifestPath(cmd) manifestPath, err := mfa.resolveFreshenManifestPath(cmd)
if err != nil { if err != nil {
return fmt.Errorf("freshen: %w", err) return err
} }
//nolint:contextcheck // mfer loads a manifest without a context //nolint:contextcheck // mfer loads a manifest without a context
@@ -499,9 +500,9 @@ func (mfa *CLIApp) freshenManifestOperation(
return err return err
} }
absBase, err := filepath.Abs(basePath) absBase, err := filepath.Abs(resolveBasePath(cmd, manifestPath))
if err != nil { if err != nil {
return fmt.Errorf("freshen: invalid base path: %w", err) return fmt.Errorf("invalid base path: %w", err)
} }
// Phase 1: Scan filesystem // Phase 1: Scan filesystem
@@ -593,9 +594,7 @@ func hashFile(r io.Reader, progress func(int64)) ([]byte, int64, error) {
break break
} }
// Returned unwrapped: the caller renders this as // Returned unwrapped: a read error already names the file.
// "failed to hash <path>: <err>" and adding a second layer here
// would change that message.
if err != nil { if err != nil {
return nil, total, err return nil, total, err
} }
@@ -611,10 +610,12 @@ func hashFile(r io.Reader, progress func(int64)) ([]byte, int64, error) {
// addFileToBuilder adds a new file entry to the builder // addFileToBuilder adds a new file entry to the builder
func addFileToBuilder( func addFileToBuilder(
b *mfer.Builder, path string, size int64, mtime time.Time, hash []byte, b *mfer.Builder, path string, size int64, mtime time.Time, mode fs.FileMode,
hash []byte,
) error { ) error {
return b.AddFileWithHash( return b.AddFileWithHash(
mfer.RelFilePath(path), mfer.FileSize(size), mfer.ModTime(mtime), hash) mfer.RelFilePath(path), mfer.FileSize(size), mfer.ModTime(mtime), mode,
hash)
} }
// addExistingToBuilder adds an existing manifest entry to the builder. // addExistingToBuilder adds an existing manifest entry to the builder.
@@ -634,7 +635,7 @@ func addExistingToBuilder(b *mfer.Builder, entry *mfer.MFFilePath) error {
err := b.AddFileWithHash(mfer.RelFilePath(entry.GetPath()), err := b.AddFileWithHash(mfer.RelFilePath(entry.GetPath()),
mfer.FileSize(entry.GetSize()), mfer.ModTime(mtime), mfer.FileSize(entry.GetSize()), mfer.ModTime(mtime),
entry.GetHashes()[0].GetMultiHash()) fs.FileMode(entry.GetMode()), entry.GetHashes()[0].GetMultiHash())
if err != nil { if err != nil {
return fmt.Errorf( return fmt.Errorf(
"manifest entry %s: %w (regenerate the manifest with mfer generate)", "manifest entry %s: %w (regenerate the manifest with mfer generate)",
+72
View File
@@ -7,6 +7,7 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"slices" "slices"
"strings"
"testing" "testing"
"time" "time"
@@ -99,6 +100,42 @@ func TestFreshenUnchanged(t *testing.T) {
} }
} }
// TestFreshenRecordsModeOnlyWhenAsked changes only the modes of a tree
// after gen made its manifest, which recorded every mode as 0000. freshen
// --include-permissions must record each file's permission bits, and a
// later freshen without it must record 0000 again, although no file's
// content or mtime changed.
func TestFreshenRecordsModeOnlyWhenAsked(t *testing.T) {
t.Parallel()
fs := afero.NewOsFs()
root, manifestPath := setupFreshenDir(t, fs,
map[string]string{testFileTxt: "a file", testDirFile: "a file in dir"})
require.NoError(t, fs.Chmod(filepath.Join(root, testFileTxt), 0o640))
require.NoError(t, fs.Chmod(filepath.Join(root, testDirFile), 0o755))
recordedModes := func() map[string]uint32 {
modes := map[string]uint32{}
for _, f := range manifestFiles(t, fs, manifestPath) {
modes[f.GetPath()] = f.GetMode()
}
return modes
}
opts := testOpts([]string{
testApp, cmdFreshen, "-q", "--" + flagIncludePermissions,
testFlagBase, root, manifestPath,
}, fs)
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
assert.Equal(t, map[string]uint32{testFileTxt: 0o640, testDirFile: 0o755},
recordedModes())
runFreshen(t, fs, root, manifestPath)
assert.Equal(t, map[string]uint32{testFileTxt: 0, testDirFile: 0},
recordedModes())
}
// assertManifestLists asserts that the manifest at manifestPath lists // assertManifestLists asserts that the manifest at manifestPath lists
// exactly the files in want, each with the size and SHA-256 hash of its // exactly the files in want, each with the size and SHA-256 hash of its
// content in want and the mtime of the file of that name under root. // content in want and the mtime of the file of that name under root.
@@ -264,6 +301,41 @@ func TestFreshenLeavesLeftoverTempFileOutOfListing(t *testing.T) {
manifestPaths(t, fs, manifestPath)) manifestPaths(t, fs, manifestPath))
} }
// TestFreshenResolvesEntriesAgainstManifestDirectory adds sub/c.txt, then
// runs freshen from the directory above sub, on the manifest in sub.
// Without --base, the manifest then lists the files in sub, whether freshen
// is given the manifest or sub. --base names the directory to list instead,
// the current one included.
//
//nolint:paralleltest // changes the process-global working directory
func TestFreshenResolvesEntriesAgainstManifestDirectory(t *testing.T) {
for _, tc := range []struct {
args []string
want []string // the paths the manifest lists afterwards
}{
{[]string{testSubdir}, []string{"b.txt", "c.txt"}},
{[]string{testSubdirManifest}, []string{"b.txt", "c.txt"}},
{
[]string{testFlagBase, ".", testSubdirManifest},
[]string{testFileTxt, "sub/b.txt", "sub/c.txt"},
},
} {
t.Run(strings.Join(tc.args, " "), func(t *testing.T) {
fs := afero.NewOsFs()
root := setupManifestInSubdir(t)
writeTestFile(t, fs, filepath.Join(root, testSubdir, "c.txt"), "added")
opts := testOpts(slices.Concat(
[]string{testApp, cmdFreshen, "-q"}, tc.args,
), fs)
require.Equal(t, 0, runCLI(opts), "stderr: %s", testStderr(t, opts))
assert.ElementsMatch(t, tc.want, manifestPaths(t, fs,
filepath.Join(root, testSubdirManifest)))
})
}
}
// TestFreshenRecordEntryMtimePresence pins the behavior of recordEntry // TestFreshenRecordEntryMtimePresence pins the behavior of recordEntry
// with respect to MFFilePath.Mtime, which is a message pointer with // with respect to MFFilePath.Mtime, which is a message pointer with
// proto3 field presence and may legitimately be absent. // proto3 field presence and may legitimately be absent.
+12 -12
View File
@@ -74,7 +74,7 @@ func (mfa *CLIApp) collectInputPaths(args cli.Args) ([]string, error) {
ap, err := filepath.Abs(inputPath) ap, err := filepath.Abs(inputPath)
if err != nil { if err != nil {
return nil, fmt.Errorf("generate: invalid path %q: %w", inputPath, err) return nil, fmt.Errorf("invalid path %q: %w", inputPath, err)
} }
// Validate path exists before adding to list // Validate path exists before adding to list
if exists, _ := afero.Exists(mfa.Fs, ap); !exists { if exists, _ := afero.Exists(mfa.Fs, ap); !exists {
@@ -92,10 +92,11 @@ func (mfa *CLIApp) collectInputPaths(args cli.Args) ([]string, error) {
func (mfa *CLIApp) buildScannerOptions(cmd *cli.Command) *mfer.ScannerOptions { func (mfa *CLIApp) buildScannerOptions(cmd *cli.Command) *mfer.ScannerOptions {
output := cmd.String("output") output := cmd.String("output")
opts := &mfer.ScannerOptions{ opts := &mfer.ScannerOptions{
IncludeDotfiles: cmd.Bool("include-dotfiles"), IncludeDotfiles: cmd.Bool("include-dotfiles"),
FollowSymLinks: cmd.Bool("follow-symlinks"), FollowSymLinks: cmd.Bool("follow-symlinks"),
IncludeTimestamps: cmd.Bool("include-timestamps"), IncludeTimestamps: cmd.Bool("include-timestamps"),
Fs: mfa.Fs, IncludePermissions: cmd.Bool(flagIncludePermissions),
Fs: mfa.Fs,
// Neither a manifest being replaced nor a temp file left by an // Neither a manifest being replaced nor a temp file left by an
// interrupted run belongs in the new manifest. // interrupted run belongs in the new manifest.
ExcludePaths: []string{output, manifestTempPath(output)}, ExcludePaths: []string{output, manifestTempPath(output)},
@@ -128,8 +129,7 @@ func (mfa *CLIApp) enumerateInputs(
// Default to current directory // Default to current directory
err := s.EnumeratePath(".", enumProgress) err := s.EnumeratePath(".", enumProgress)
if err != nil { if err != nil {
return fmt.Errorf( return fmt.Errorf("enumerate current directory: %w", err)
"generate: failed to enumerate current directory: %w", err)
} }
return nil return nil
@@ -143,7 +143,7 @@ func (mfa *CLIApp) enumerateInputs(
err = s.EnumeratePaths(enumProgress, paths...) err = s.EnumeratePaths(enumProgress, paths...)
if err != nil { if err != nil {
return fmt.Errorf("generate: failed to enumerate paths: %w", err) return fmt.Errorf("enumerate files: %w", err)
} }
return nil return nil
@@ -228,7 +228,7 @@ func (mfa *CLIApp) generateManifestOperation(
outFile, err := mfa.Fs.Create(tmpPath) outFile, err := mfa.Fs.Create(tmpPath)
if err != nil { if err != nil {
return fmt.Errorf("failed to create temp file: %w", err) return err
} }
// Set up signal handler to clean up temp file on Ctrl-C // Set up signal handler to clean up temp file on Ctrl-C
@@ -267,19 +267,19 @@ func (mfa *CLIApp) generateManifestOperation(
scanWg.Wait() scanWg.Wait()
if err != nil { if err != nil {
return fmt.Errorf("failed to generate manifest: %w", err) return fmt.Errorf("generate manifest: %w", err)
} }
// Close file before rename to ensure all data is flushed // Close file before rename to ensure all data is flushed
err = outFile.Close() err = outFile.Close()
if err != nil { if err != nil {
return fmt.Errorf("failed to close temp file: %w", err) return err
} }
// Atomic rename // Atomic rename
err = mfa.Fs.Rename(tmpPath, outputPath) err = mfa.Fs.Rename(tmpPath, outputPath)
if err != nil { if err != nil {
return fmt.Errorf("failed to rename temp file: %w", err) return err
} }
success = true success = true
+5 -5
View File
@@ -19,12 +19,12 @@ func (mfa *CLIApp) listManifestOperation(ctx context.Context, cmd *cli.Command)
pathOrURL, err := mfa.resolveManifestArg(cmd) pathOrURL, err := mfa.resolveManifestArg(cmd)
if err != nil { if err != nil {
return fmt.Errorf("list: %w", err) return err
} }
rc, err := mfa.openManifestReader(ctx, pathOrURL) rc, err := mfa.openManifestReader(ctx, pathOrURL)
if err != nil { if err != nil {
return fmt.Errorf("list: %w", err) return err
} }
defer func() { _ = rc.Close() }() defer func() { _ = rc.Close() }()
@@ -32,7 +32,7 @@ func (mfa *CLIApp) listManifestOperation(ctx context.Context, cmd *cli.Command)
//nolint:contextcheck // mfer loads a manifest without a context //nolint:contextcheck // mfer loads a manifest without a context
manifest, err := mfer.NewManifestFromReader(rc) manifest, err := mfer.NewManifestFromReader(rc)
if err != nil { if err != nil {
return fmt.Errorf("list: failed to parse manifest: %w", err) return fmt.Errorf("parse manifest: %w", err)
} }
files := manifest.Files() files := manifest.Files()
@@ -52,8 +52,8 @@ func (mfa *CLIApp) listManifestOperation(ctx context.Context, cmd *cli.Command)
mtimeStr = mtime.Format(time.RFC3339) mtimeStr = mtime.Format(time.RFC3339)
} }
_, _ = fmt.Fprintf(mfa.Stdout, "%d\t%s\t%s%s", _, _ = fmt.Fprintf(mfa.Stdout, "%04o\t%d\t%s\t%s%s",
f.GetSize(), mtimeStr, f.GetPath(), lineEnd) f.GetMode(), f.GetSize(), mtimeStr, f.GetPath(), lineEnd)
} else { } else {
_, _ = fmt.Fprintf(mfa.Stdout, "%s%s", f.GetPath(), lineEnd) _, _ = fmt.Fprintf(mfa.Stdout, "%s%s", f.GetPath(), lineEnd)
} }
+27 -10
View File
@@ -6,6 +6,7 @@ import (
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
"path/filepath"
"strings" "strings"
"time" "time"
@@ -15,13 +16,13 @@ import (
// manifestFetchTimeout bounds HTTP requests made to fetch a manifest. // manifestFetchTimeout bounds HTTP requests made to fetch a manifest.
const manifestFetchTimeout = 30 * time.Second const manifestFetchTimeout = 30 * time.Second
// errHTTPStatus indicates an HTTP response with a non-OK status code. // errHTTPStatus indicates an HTTP response with a non-OK status code. It is
// // followed by the code, as in "unexpected HTTP status 404".
// Its text is the literal "HTTP" prefix of the rendered "HTTP <code>" var errHTTPStatus = errors.New("unexpected HTTP status")
// message that mfer has always printed, so that wrapping it does not
// change any user-visible output. Match it with errors.Is; do not read // errManifestTooLarge indicates a manifest download that passed
// its message. // CLIApp.maxManifestSize.
var errHTTPStatus = errors.New("HTTP") var errManifestTooLarge = errors.New("file exceeds maximum allowed size")
// isHTTPURL returns true if the string starts with http:// or https://. // isHTTPURL returns true if the string starts with http:// or https://.
func isHTTPURL(s string) bool { func isHTTPURL(s string) bool {
@@ -36,20 +37,22 @@ func (mfa *CLIApp) openManifestReader(
if isHTTPURL(pathOrURL) { if isHTTPURL(pathOrURL) {
client := &http.Client{Timeout: manifestFetchTimeout} client := &http.Client{Timeout: manifestFetchTimeout}
// The *url.Error that NewRequestWithContext and Do return names
// the URL.
req, err := http.NewRequestWithContext(ctx, http.MethodGet, pathOrURL, nil) req, err := http.NewRequestWithContext(ctx, http.MethodGet, pathOrURL, nil)
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to fetch %s: %w", pathOrURL, err) return nil, fmt.Errorf("download manifest: %w", err)
} }
resp, err := client.Do(req) resp, err := client.Do(req)
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to fetch %s: %w", pathOrURL, err) return nil, fmt.Errorf("download manifest: %w", err)
} }
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
_ = resp.Body.Close() _ = resp.Body.Close()
return nil, fmt.Errorf("failed to fetch %s: %w %d", return nil, fmt.Errorf("download manifest %s: %w %d",
pathOrURL, errHTTPStatus, resp.StatusCode) pathOrURL, errHTTPStatus, resp.StatusCode)
} }
@@ -84,3 +87,17 @@ func (mfa *CLIApp) resolveManifestArg(cmd *cli.Command) (string, error) {
return findManifest(mfa.Fs, ".") return findManifest(mfa.Fs, ".")
} }
// resolveBasePath returns the directory a manifest's paths are resolved
// against: the one --base names, or else the directory holding the manifest,
// or the current directory for a manifest URL.
func resolveBasePath(cmd *cli.Command, manifestPath string) string {
switch {
case cmd.IsSet(flagBase):
return cmd.String(flagBase)
case isHTTPURL(manifestPath):
return "."
default:
return filepath.Dir(manifestPath)
}
}
+32 -12
View File
@@ -21,12 +21,15 @@ const (
cmdFreshen = "freshen" cmdFreshen = "freshen"
cmdExport = "export" cmdExport = "export"
cmdFetch = "fetch" cmdFetch = "fetch"
cmdList = "list"
cmdVersion = "version" cmdVersion = "version"
flagProgress = "progress" flagBase = "base"
flagTimeout = "timeout" flagProgress = "progress"
flagDest = "dest" flagTimeout = "timeout"
flagRequireSignature = "require-signature" flagDest = "dest"
flagRequireSignature = "require-signature"
flagIncludePermissions = "include-permissions"
manifestArgsUsage = "[manifest file]" manifestArgsUsage = "[manifest file]"
@@ -57,6 +60,10 @@ type CLIApp struct {
exitCode int exitCode int
app *cli.Command app *cli.Command
// maxManifestSize is the most of a manifest that fetch, and check
// given a URL, download: mfer.MaxManifestSize, which tests lower.
maxManifestSize int64
Stdin io.Reader // Standard input stream Stdin io.Reader // Standard input stream
Stdout io.Writer // Standard output stream for normal output Stdout io.Writer // Standard output stream for normal output
Stderr io.Writer // Standard error stream for diagnostics Stderr io.Writer // Standard error stream for diagnostics
@@ -167,6 +174,16 @@ func requireSignatureFlag() *cli.StringFlag {
} }
} }
// includePermissionsFlag returns the --include-permissions flag taken by the
// generate and freshen subcommands.
func includePermissionsFlag() *cli.BoolFlag {
return &cli.BoolFlag{
Name: flagIncludePermissions,
Usage: "Record each file's permission bits in manifest " +
"(recorded as 0000 by default)",
}
}
func (mfa *CLIApp) generateCommand() *cli.Command { func (mfa *CLIApp) generateCommand() *cli.Command {
return &cli.Command{ return &cli.Command{
Name: cmdGenerate, Name: cmdGenerate,
@@ -224,6 +241,7 @@ func (mfa *CLIApp) generateCommand() *cli.Command {
Usage: "Include createdAt timestamp in manifest " + Usage: "Include createdAt timestamp in manifest " +
"(omitted by default for determinism)", "(omitted by default for determinism)",
}, },
includePermissionsFlag(),
), ),
} }
} }
@@ -242,10 +260,11 @@ func (mfa *CLIApp) checkCommand() *cli.Command {
}, },
Flags: append(commonFlags(), Flags: append(commonFlags(),
&cli.StringFlag{ &cli.StringFlag{
Name: "base", Name: flagBase,
Aliases: []string{"b"}, Aliases: []string{"b"},
Value: ".", Usage: "Base directory for resolving relative paths from manifest " +
Usage: "Base directory for resolving relative paths from manifest", "(by default the directory holding the manifest, or the " +
"current directory for a manifest URL)",
}, },
&cli.BoolFlag{ &cli.BoolFlag{
Name: flagProgress, Name: flagProgress,
@@ -275,10 +294,10 @@ func (mfa *CLIApp) freshenCommand() *cli.Command {
}, },
Flags: append(commonFlags(), Flags: append(commonFlags(),
&cli.StringFlag{ &cli.StringFlag{
Name: "base", Name: flagBase,
Aliases: []string{"b"}, Aliases: []string{"b"},
Value: ".", Usage: "Base directory for resolving relative paths " +
Usage: "Base directory for resolving relative paths", "(by default the directory holding the manifest)",
}, },
&cli.BoolFlag{ &cli.BoolFlag{
Name: "follow-symlinks", Name: "follow-symlinks",
@@ -307,6 +326,7 @@ func (mfa *CLIApp) freshenCommand() *cli.Command {
Usage: "Include createdAt timestamp in manifest " + Usage: "Include createdAt timestamp in manifest " +
"(omitted by default for determinism)", "(omitted by default for determinism)",
}, },
includePermissionsFlag(),
), ),
} }
} }
@@ -340,7 +360,7 @@ func (mfa *CLIApp) versionCommand() *cli.Command {
func (mfa *CLIApp) listCommand() *cli.Command { func (mfa *CLIApp) listCommand() *cli.Command {
return &cli.Command{ return &cli.Command{
Name: "list", Name: cmdList,
Aliases: []string{"ls"}, Aliases: []string{"ls"},
Usage: "List files in manifest", Usage: "List files in manifest",
ArgsUsage: manifestArgsUsage, ArgsUsage: manifestArgsUsage,
@@ -350,7 +370,7 @@ func (mfa *CLIApp) listCommand() *cli.Command {
&cli.BoolFlag{ &cli.BoolFlag{
Name: "long", Name: "long",
Aliases: []string{"l"}, Aliases: []string{"l"},
Usage: "Show size and mtime", Usage: "Show mode, size and mtime",
}, },
&cli.BoolFlag{ &cli.BoolFlag{
Name: "print0", Name: "print0",
BIN
View File
Binary file not shown.
+35 -18
View File
@@ -8,6 +8,7 @@ import (
"errors" "errors"
"fmt" "fmt"
"io" "io"
"io/fs"
"sort" "sort"
"strings" "strings"
"sync" "sync"
@@ -37,6 +38,7 @@ var (
errNegativeSize = errors.New("size cannot be negative") errNegativeSize = errors.New("size cannot be negative")
errHashNotMultihash = errors.New("hash is not a valid multihash") errHashNotMultihash = errors.New("hash is not a valid multihash")
errHashTooShort = errors.New("hash digest is too short") errHashTooShort = errors.New("hash digest is too short")
errDuplicatePath = errors.New("duplicate path")
) )
// ValidatePath checks that a file path conforms to manifest path invariants: // ValidatePath checks that a file path conforms to manifest path invariants:
@@ -63,7 +65,7 @@ func ValidatePath(p string) error {
return fmt.Errorf("path %q %w", p, errPathAbsolute) return fmt.Errorf("path %q %w", p, errPathAbsolute)
} }
for _, seg := range strings.Split(p, "/") { for seg := range strings.SplitSeq(p, "/") {
if seg == "" { if seg == "" {
return fmt.Errorf("path %q %w", p, errPathEmptySegment) return fmt.Errorf("path %q %w", p, errPathEmptySegment)
} }
@@ -114,6 +116,7 @@ type FileHashProgress struct {
type Builder struct { type Builder struct {
mu sync.Mutex mu sync.Mutex
files []*MFFilePath files []*MFFilePath
paths map[string]bool // the path of each entry in files
createdAt time.Time createdAt time.Time
includeTimestamps bool includeTimestamps bool
signingOptions *SigningOptions signingOptions *SigningOptions
@@ -124,6 +127,7 @@ type Builder struct {
func NewBuilder() *Builder { func NewBuilder() *Builder {
return &Builder{ return &Builder{
files: make([]*MFFilePath, 0), files: make([]*MFFilePath, 0),
paths: make(map[string]bool),
createdAt: time.Now(), createdAt: time.Now(),
} }
} }
@@ -137,12 +141,15 @@ func (b *Builder) SetSeed(seed string) {
} }
// AddFile reads file content from reader, computes hashes, and adds to manifest. // AddFile reads file content from reader, computes hashes, and adds to manifest.
// A path already added is refused once the file is read.
// Only mode's permission bits (mode.Perm()) are recorded; 0 records none.
// Progress updates are sent to the progress channel (if non-nil) without blocking. // Progress updates are sent to the progress channel (if non-nil) without blocking.
// Returns the number of bytes read. // Returns the number of bytes read.
func (b *Builder) AddFile( func (b *Builder) AddFile(
path RelFilePath, path RelFilePath,
size FileSize, size FileSize,
mtime ModTime, mtime ModTime,
mode fs.FileMode,
reader io.Reader, reader io.Reader,
progress chan<- FileHashProgress, progress chan<- FileHashProgress,
) (FileSize, error) { ) (FileSize, error) {
@@ -198,13 +205,10 @@ func (b *Builder) AddFile(
{MultiHash: mh}, {MultiHash: mh},
}, },
Mtime: mtime.Timestamp(), Mtime: mtime.Timestamp(),
Mode: uint32(mode.Perm()),
} }
b.mu.Lock() return totalRead, b.addEntry(entry)
b.files = append(b.files, entry)
b.mu.Unlock()
return totalRead, nil
} }
// sendFileHashProgress sends a progress update without blocking. // sendFileHashProgress sends a progress update without blocking.
@@ -229,17 +233,20 @@ func (b *Builder) FileCount() int {
// AddFileWithHash adds a file entry with a pre-computed hash. // AddFileWithHash adds a file entry with a pre-computed hash.
// This is useful when the hash is already known (e.g., from an existing manifest). // This is useful when the hash is already known (e.g., from an existing manifest).
// Returns an error if path is invalid, size is negative, or hash is not a // Only mode's permission bits (mode.Perm()) are recorded; 0 records none.
// multihash with a digest of at least 32 bytes, as long as SHA-256's. // Returns an error if path is invalid or already added, size is negative,
// or hash is not a multihash with a digest of at least 32 bytes, as long
// as SHA-256's.
func (b *Builder) AddFileWithHash( func (b *Builder) AddFileWithHash(
path RelFilePath, path RelFilePath,
size FileSize, size FileSize,
mtime ModTime, mtime ModTime,
mode fs.FileMode,
hash Multihash, hash Multihash,
) error { ) error {
err := ValidatePath(string(path)) err := ValidatePath(string(path))
if err != nil { if err != nil {
return fmt.Errorf("add file: %w", err) return err
} }
if size < 0 { if size < 0 {
@@ -268,13 +275,10 @@ func (b *Builder) AddFileWithHash(
{MultiHash: hash}, {MultiHash: hash},
}, },
Mtime: mtime.Timestamp(), Mtime: mtime.Timestamp(),
Mode: uint32(mode.Perm()),
} }
b.mu.Lock() return b.addEntry(entry)
b.files = append(b.files, entry)
b.mu.Unlock()
return nil
} }
// SetIncludeTimestamps controls whether the manifest includes a createdAt timestamp. // SetIncludeTimestamps controls whether the manifest includes a createdAt timestamp.
@@ -325,20 +329,33 @@ func (b *Builder) Build(ctx context.Context, w io.Writer) error {
// Generate outer wrapper // Generate outer wrapper
err := m.generateOuter(ctx) err := m.generateOuter(ctx)
if err != nil { if err != nil {
return fmt.Errorf("build: generate outer: %w", err) return err
} }
// Generate final output // Generate final output
err = m.generate(ctx) err = m.generate(ctx)
if err != nil { if err != nil {
return fmt.Errorf("build: generate: %w", err) return err
} }
// Write to output // Write to output
_, err = w.Write(m.output.Bytes()) _, err = w.Write(m.output.Bytes())
if err != nil {
return fmt.Errorf("build: write output: %w", err) return err
}
// addEntry adds entry to the manifest unless an entry with its path is
// already there.
func (b *Builder) addEntry(entry *MFFilePath) error {
b.mu.Lock()
defer b.mu.Unlock()
if b.paths[entry.GetPath()] {
return fmt.Errorf("%w %q", errDuplicatePath, entry.GetPath())
} }
b.paths[entry.GetPath()] = true
b.files = append(b.files, entry)
return nil return nil
} }
+44 -18
View File
@@ -35,7 +35,7 @@ func TestBuilderAddFile(t *testing.T) {
reader := bytes.NewReader(content) reader := bytes.NewReader(content)
bytesRead, err := b.AddFile( bytesRead, err := b.AddFile(
"test.txt", FileSize(len(content)), ModTime(time.Now()), reader, nil, "test.txt", FileSize(len(content)), ModTime(time.Now()), 0, reader, nil,
) )
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, FileSize(len(content)), bytesRead) assert.Equal(t, FileSize(len(content)), bytesRead)
@@ -49,7 +49,7 @@ func TestBuilderAddFileWithHash(t *testing.T) {
hash, err := multihash.Encode(make([]byte, sha256.Size), multihash.SHA2_256) hash, err := multihash.Encode(make([]byte, sha256.Size), multihash.SHA2_256)
require.NoError(t, err) require.NoError(t, err)
err = b.AddFileWithHash("test.txt", 100, ModTime(time.Now()), hash) err = b.AddFileWithHash("test.txt", 100, ModTime(time.Now()), 0, hash)
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, 1, b.FileCount()) assert.Equal(t, 1, b.FileCount())
} }
@@ -64,7 +64,7 @@ func TestBuilderAddFileWithHashValidation(t *testing.T) {
t.Parallel() t.Parallel()
b := NewBuilder() b := NewBuilder()
err := b.AddFileWithHash("", 100, ModTime(time.Now()), sha256Hash) err := b.AddFileWithHash("", 100, ModTime(time.Now()), 0, sha256Hash)
require.Error(t, err) require.Error(t, err)
assert.Contains(t, err.Error(), "path") assert.Contains(t, err.Error(), "path")
}) })
@@ -73,7 +73,7 @@ func TestBuilderAddFileWithHashValidation(t *testing.T) {
t.Parallel() t.Parallel()
b := NewBuilder() b := NewBuilder()
err := b.AddFileWithHash("test.txt", -1, ModTime(time.Now()), sha256Hash) err := b.AddFileWithHash("test.txt", -1, ModTime(time.Now()), 0, sha256Hash)
require.Error(t, err) require.Error(t, err)
assert.Contains(t, err.Error(), "size") assert.Contains(t, err.Error(), "size")
}) })
@@ -82,7 +82,7 @@ func TestBuilderAddFileWithHashValidation(t *testing.T) {
t.Parallel() t.Parallel()
b := NewBuilder() b := NewBuilder()
err := b.AddFileWithHash("test.txt", 100, ModTime(time.Now()), sha256Hash) err := b.AddFileWithHash("test.txt", 100, ModTime(time.Now()), 0, sha256Hash)
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, 1, b.FileCount()) assert.Equal(t, 1, b.FileCount())
}) })
@@ -118,13 +118,39 @@ func TestBuilderAddFileWithHashRejectsBadHashes(t *testing.T) {
t.Parallel() t.Parallel()
b := NewBuilder() b := NewBuilder()
err := b.AddFileWithHash("test.txt", 100, ModTime(time.Now()), tt.hash) err := b.AddFileWithHash("test.txt", 100, ModTime(time.Now()), 0, tt.hash)
require.ErrorIs(t, err, tt.want) require.ErrorIs(t, err, tt.want)
assert.Equal(t, 0, b.FileCount()) assert.Equal(t, 0, b.FileCount())
}) })
} }
} }
// TestBuilderRefusesPathAlreadyAdded adds a path, then adds it again with
// AddFile and with AddFileWithHash. Each must refuse it, naming it, and
// keep the one entry already added.
func TestBuilderRefusesPathAlreadyAdded(t *testing.T) {
t.Parallel()
hash, err := multihash.Encode(make([]byte, sha256.Size), multihash.SHA2_256)
require.NoError(t, err)
b := NewBuilder()
require.NoError(t, b.AddFileWithHash("dir/a.txt", 4, ModTime{}, 0, hash))
content := []byte("data")
_, err = b.AddFile(
"dir/a.txt", FileSize(len(content)), ModTime{}, 0, bytes.NewReader(content), nil,
)
require.ErrorIs(t, err, errDuplicatePath)
require.EqualError(t, err, `duplicate path "dir/a.txt"`)
err = b.AddFileWithHash("dir/a.txt", 4, ModTime{}, 0, hash)
require.ErrorIs(t, err, errDuplicatePath)
require.EqualError(t, err, `duplicate path "dir/a.txt"`)
assert.Equal(t, 1, b.FileCount())
}
func TestBuilderBuild(t *testing.T) { func TestBuilderBuild(t *testing.T) {
t.Parallel() t.Parallel()
@@ -133,7 +159,7 @@ func TestBuilderBuild(t *testing.T) {
reader := bytes.NewReader(content) reader := bytes.NewReader(content)
_, err := b.AddFile( _, err := b.AddFile(
"test.txt", FileSize(len(content)), ModTime(time.Now()), reader, nil, "test.txt", FileSize(len(content)), ModTime(time.Now()), 0, reader, nil,
) )
require.NoError(t, err) require.NoError(t, err)
@@ -196,7 +222,7 @@ func TestBuilderDeterministicOutput(t *testing.T) {
for _, f := range files { for _, f := range files {
r := bytes.NewReader([]byte(f.content)) r := bytes.NewReader([]byte(f.content))
_, err := b.AddFile( _, err := b.AddFile(
RelFilePath(f.path), FileSize(len(f.content)), mtime, r, nil, RelFilePath(f.path), FileSize(len(f.content)), mtime, 0, r, nil,
) )
require.NoError(t, err) require.NoError(t, err)
} }
@@ -279,7 +305,7 @@ func TestBuilderAddFileSizeMismatch(t *testing.T) {
reader := bytes.NewReader(content) reader := bytes.NewReader(content)
// Declare wrong size // Declare wrong size
_, err := b.AddFile("test.txt", FileSize(100), ModTime(time.Now()), reader, nil) _, err := b.AddFile("test.txt", FileSize(100), ModTime(time.Now()), 0, reader, nil)
require.Error(t, err) require.Error(t, err)
assert.Contains(t, err.Error(), "size mismatch") assert.Contains(t, err.Error(), "size mismatch")
} }
@@ -291,12 +317,12 @@ func TestBuilderAddFileInvalidPath(t *testing.T) {
content := []byte("data") content := []byte("data")
reader := bytes.NewReader(content) reader := bytes.NewReader(content)
_, err := b.AddFile("", FileSize(len(content)), ModTime(time.Now()), reader, nil) _, err := b.AddFile("", FileSize(len(content)), ModTime(time.Now()), 0, reader, nil)
require.Error(t, err) require.Error(t, err)
reader.Reset(content) reader.Reset(content)
_, err = b.AddFile( _, err = b.AddFile(
"/absolute", FileSize(len(content)), ModTime(time.Now()), reader, nil, "/absolute", FileSize(len(content)), ModTime(time.Now()), 0, reader, nil,
) )
assert.Error(t, err) assert.Error(t, err)
} }
@@ -310,7 +336,7 @@ func TestBuilderAddFileWithProgress(t *testing.T) {
progress := make(chan FileHashProgress, 100) progress := make(chan FileHashProgress, 100)
bytesRead, err := b.AddFile( bytesRead, err := b.AddFile(
"test.txt", FileSize(len(content)), ModTime(time.Now()), reader, progress, "test.txt", FileSize(len(content)), ModTime(time.Now()), 0, reader, progress,
) )
close(progress) close(progress)
require.NoError(t, err) require.NoError(t, err)
@@ -345,7 +371,7 @@ func TestBuilderBuildRoundTrip(t *testing.T) {
for _, f := range files { for _, f := range files {
reader := bytes.NewReader(f.content) reader := bytes.NewReader(f.content)
_, err := b.AddFile( _, err := b.AddFile(
RelFilePath(f.path), FileSize(len(f.content)), ModTime(now), reader, nil, RelFilePath(f.path), FileSize(len(f.content)), ModTime(now), 0, reader, nil,
) )
require.NoError(t, err) require.NoError(t, err)
} }
@@ -387,7 +413,7 @@ func TestBuilderBuildRoundTripLargeManifest(t *testing.T) {
for i := range 4000 { for i := range 4000 {
path := RelFilePath(fmt.Sprintf("dir/file-%05d.txt", i)) path := RelFilePath(fmt.Sprintf("dir/file-%05d.txt", i))
require.NoError(t, b.AddFileWithHash(path, FileSize(i), ModTime{}, hash)) require.NoError(t, b.AddFileWithHash(path, FileSize(i), ModTime{}, 0, hash))
} }
var buf bytes.Buffer var buf bytes.Buffer
@@ -452,7 +478,7 @@ func TestManifestString(t *testing.T) {
content := []byte("test") content := []byte("test")
reader := bytes.NewReader(content) reader := bytes.NewReader(content)
_, err := b.AddFile( _, err := b.AddFile(
"test.txt", FileSize(len(content)), ModTime(time.Now()), reader, nil, "test.txt", FileSize(len(content)), ModTime(time.Now()), 0, reader, nil,
) )
require.NoError(t, err) require.NoError(t, err)
@@ -484,7 +510,7 @@ func TestBuilderOmitsCreatedAtByDefault(t *testing.T) {
b := NewBuilder() b := NewBuilder()
content := []byte("hello") content := []byte("hello")
_, err := b.AddFile( _, err := b.AddFile(
"test.txt", FileSize(len(content)), ModTime(time.Now()), "test.txt", FileSize(len(content)), ModTime(time.Now()), 0,
bytes.NewReader(content), nil, bytes.NewReader(content), nil,
) )
require.NoError(t, err) require.NoError(t, err)
@@ -506,7 +532,7 @@ func TestBuilderIncludesCreatedAtWhenRequested(t *testing.T) {
content := []byte("hello") content := []byte("hello")
_, err := b.AddFile( _, err := b.AddFile(
"test.txt", FileSize(len(content)), ModTime(time.Now()), "test.txt", FileSize(len(content)), ModTime(time.Now()), 0,
bytes.NewReader(content), nil, bytes.NewReader(content), nil,
) )
require.NoError(t, err) require.NoError(t, err)
@@ -532,7 +558,7 @@ func TestBuilderDeterministicFileOrder(t *testing.T) {
content := []byte("content of " + name) content := []byte("content of " + name)
_, err := b.AddFile( _, err := b.AddFile(
RelFilePath(name), FileSize(len(content)), RelFilePath(name), FileSize(len(content)),
ModTime(time.Unix(1000, 0)), bytes.NewReader(content), nil, ModTime(time.Unix(1000, 0)), 0, bytes.NewReader(content), nil,
) )
require.NoError(t, err) require.NoError(t, err)
} }
+24 -16
View File
@@ -15,7 +15,6 @@ import (
) )
var ( var (
errNoSigningPubKey = errors.New("manifest has no signing public key")
errManifestPathEmpty = errors.New("manifest path cannot be empty") errManifestPathEmpty = errors.New("manifest path cannot be empty")
errBasePathEmpty = errors.New("base path cannot be empty") errBasePathEmpty = errors.New("base path cannot be empty")
) )
@@ -36,6 +35,7 @@ const (
StatusMissing // File not found on disk StatusMissing // File not found on disk
StatusSizeMismatch // File size differs from manifest StatusSizeMismatch // File size differs from manifest
StatusHashMismatch // File hash differs from manifest StatusHashMismatch // File hash differs from manifest
StatusModeMismatch // File permission bits differ from a recorded mode
StatusExtra // File exists on disk but not in manifest StatusExtra // File exists on disk but not in manifest
StatusError // Error occurred during verification StatusError // Error occurred during verification
) )
@@ -50,6 +50,8 @@ func (s Status) String() string {
return "SIZE_MISMATCH" return "SIZE_MISMATCH"
case StatusHashMismatch: case StatusHashMismatch:
return "HASH_MISMATCH" return "HASH_MISMATCH"
case StatusModeMismatch:
return "MODE_MISMATCH"
case StatusExtra: case StatusExtra:
return "EXTRA" return "EXTRA"
case StatusError: case StatusError:
@@ -170,8 +172,14 @@ func (c *Checker) IsSigned() bool {
return len(c.signature) > 0 return len(c.signature) > 0
} }
// Signer returns the signer fingerprint if the manifest is signed, nil otherwise. // Signer returns the fingerprint of the key that made the manifest's
// signature, which loading the manifest checked, or nil if the manifest is
// not signed.
func (c *Checker) Signer() []byte { func (c *Checker) Signer() []byte {
if !c.IsSigned() {
return nil
}
return c.signer return c.signer
} }
@@ -181,17 +189,6 @@ func (c *Checker) SigningPubKey() []byte {
return c.signingPubKey return c.signingPubKey
} }
// ExtractEmbeddedSigningKeyFP imports the manifest's embedded public key into a
// temporary keyring and extracts its fingerprint. This validates the key and
// returns its actual fingerprint from the key material itself.
func (c *Checker) ExtractEmbeddedSigningKeyFP(ctx context.Context) (string, error) {
if len(c.signingPubKey) == 0 {
return "", errNoSigningPubKey
}
return gpgExtractPubKeyFingerprint(ctx, c.signingPubKey)
}
// Check verifies all files against the manifest. // Check verifies all files against the manifest.
// Results are sent to the results channel as files are checked. // Results are sent to the results channel as files are checked.
// Progress updates are sent to the progress channel approximately once per second. // Progress updates are sent to the progress channel approximately once per second.
@@ -408,11 +405,22 @@ func (c *Checker) checkFile(entry *MFFilePath, checkedBytes *FileSize) Result {
return Result{Path: relPath, Status: StatusError, Message: err.Error()} return Result{Path: relPath, Status: StatusError, Message: err.Error()}
} }
// Check against all hashes in manifest (at least one must match) // Check against all hashes in manifest (at least one must match),
// then against the recorded mode, where one is: 0 means none was.
for _, hash := range entry.GetHashes() { for _, hash := range entry.GetHashes() {
if bytes.Equal(computed, hash.GetMultiHash()) { if !bytes.Equal(computed, hash.GetMultiHash()) {
return Result{Path: relPath, Status: StatusOK} continue
} }
if entry.GetMode() != 0 && info.Mode().Perm() != os.FileMode(entry.GetMode()) {
return Result{
Path: relPath,
Status: StatusModeMismatch,
Message: "mode mismatch",
}
}
return Result{Path: relPath, Status: StatusOK}
} }
return Result{ return Result{
+52 -2
View File
@@ -34,6 +34,7 @@ func TestStatusString(t *testing.T) {
{StatusMissing, "MISSING"}, {StatusMissing, "MISSING"},
{StatusSizeMismatch, "SIZE_MISMATCH"}, {StatusSizeMismatch, "SIZE_MISMATCH"},
{StatusHashMismatch, "HASH_MISMATCH"}, {StatusHashMismatch, "HASH_MISMATCH"},
{StatusModeMismatch, "MODE_MISMATCH"},
{StatusExtra, "EXTRA"}, {StatusExtra, "EXTRA"},
{StatusError, "ERROR"}, {StatusError, "ERROR"},
{Status(99), "UNKNOWN"}, {Status(99), "UNKNOWN"},
@@ -59,7 +60,7 @@ func createTestManifest(
for path, content := range files { for path, content := range files {
reader := bytes.NewReader(content) reader := bytes.NewReader(content)
_, err := builder.AddFile( _, err := builder.AddFile(
RelFilePath(path), FileSize(len(content)), ModTime(time.Now()), reader, nil, RelFilePath(path), FileSize(len(content)), ModTime(time.Now()), 0, reader, nil,
) )
require.NoError(t, err) require.NoError(t, err)
} }
@@ -279,7 +280,8 @@ func TestCheckMissingFile(t *testing.T) {
missingCount++ missingCount++
assert.Equal(t, RelFilePath("missing.txt"), r.Path) assert.Equal(t, RelFilePath("missing.txt"), r.Path)
case StatusSizeMismatch, StatusHashMismatch, StatusExtra, StatusError: case StatusSizeMismatch, StatusHashMismatch, StatusModeMismatch,
StatusExtra, StatusError:
// Not expected in this test; counted assertions below will fail. // Not expected in this test; counted assertions below will fail.
} }
} }
@@ -349,6 +351,54 @@ func TestCheckHashMismatch(t *testing.T) {
assert.Equal(t, RelFilePath(testFileName), r.Path) assert.Equal(t, RelFilePath(testFileName), r.Path)
} }
// A recorded mode other than 0000 that differs from the file's permission
// bits fails the check; a recorded 0000 is never checked.
func TestCheckMode(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
name string
recorded os.FileMode
onDisk os.FileMode
want Status
}{
{"recorded mode matches", 0o640, 0o640, StatusOK},
{"recorded mode differs", 0o640, 0o600, StatusModeMismatch},
{"0000 is not checked", 0, 0o600, StatusOK},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
content := []byte("content")
b := NewBuilder()
_, err := b.AddFile(testFileName, FileSize(len(content)), ModTime{},
tc.recorded, bytes.NewReader(content), nil)
require.NoError(t, err)
var buf bytes.Buffer
require.NoError(t, b.Build(context.Background(), &buf))
require.NoError(t, afero.WriteFile(fs, testManifestPath, buf.Bytes(), 0o644))
require.NoError(t, fs.MkdirAll(testDataDir, 0o755))
require.NoError(t, afero.WriteFile(fs,
testDataDir+"/"+testFileName, content, tc.onDisk))
chk, err := NewChecker(&CheckerOptions{
ManifestPath: testManifestPath,
BasePath: testDataDir,
Fs: fs,
})
require.NoError(t, err)
results := make(chan Result, 1)
require.NoError(t, chk.Check(context.Background(), results, nil))
assert.Equal(t, tc.want, (<-results).Status)
})
}
}
func TestCheckWithProgress(t *testing.T) { func TestCheckWithProgress(t *testing.T) {
t.Parallel() t.Parallel()
+12 -5
View File
@@ -8,10 +8,17 @@ const (
ReleaseDate = "2025-12-17" ReleaseDate = "2025-12-17"
// MaxDecompressedSize is the maximum allowed size of decompressed manifest // MaxDecompressedSize is the maximum allowed size of decompressed manifest
// data (256 MB). This prevents decompression bombs from consuming excessive // data (256 MiB). This prevents decompression bombs from consuming excessive
// memory. // memory.
MaxDecompressedSize int64 = 256 * 1024 * 1024 MaxDecompressedSize int64 = 256 * 1024 * 1024
// MaxManifestSize is the largest manifest file mfer reads (258 MiB).
// zstd's worst case grows data it cannot compress by 1/256, so an inner
// message of MaxDecompressedSize compresses to at most 257 MiB; the
// last MiB is room for the signature, the signing key and the other
// outer fields.
MaxManifestSize = MaxDecompressedSize + MaxDecompressedSize/256 + 1<<20
// zstdWindowSize is the zstd window zstd.SpeedBestCompression gives mfer's writer. // zstdWindowSize is the zstd window zstd.SpeedBestCompression gives mfer's writer.
zstdWindowSize = 8 << 20 zstdWindowSize = 8 << 20
@@ -29,8 +36,8 @@ const (
// Bytes decoding sets aside for each file entry, hash, timestamp and // Bytes decoding sets aside for each file entry, hash, timestamp and
// MIME type, however short its encoding. checkDecodedSize refuses an // MIME type, however short its encoding. checkDecodedSize refuses an
// inner message for which these add up to more than maxDecodedGrowth // inner message for which these add up to more than maxDecodedGrowth
// times its size. // times its size. The mode is held in the file entry itself.
decodedFileEntrySize = 160 decodedFileEntrySize = 176
decodedHashSize = 112 decodedHashSize = 112
decodedTimestampSize = 64 decodedTimestampSize = 64
decodedMIMETypeSize = 16 decodedMIMETypeSize = 16
@@ -38,7 +45,7 @@ const (
// Each file entry mfer writes holds a path of at least one byte, a // Each file entry mfer writes holds a path of at least one byte, a
// multihash at least as long as SHA-256's 34 bytes (AddFileWithHash // multihash at least as long as SHA-256's 34 bytes (AddFileWithHash
// refuses shorter ones) and a modification time: at least 47 bytes, // refuses shorter ones) and a modification time: at least 47 bytes,
// counted at 336. So its manifests add up to at most about 7.15 times // counted at 352. So its manifests add up to at most about 7.49 times
// their size, and this limit is about 12% above that. // their size, and this limit is about 7% above that.
maxDecodedGrowth = 8 maxDecodedGrowth = 8
) )
+59 -28
View File
@@ -7,8 +7,8 @@ import (
"errors" "errors"
"fmt" "fmt"
"io" "io"
"strings"
"github.com/google/uuid"
"github.com/klauspost/compress/zstd" "github.com/klauspost/compress/zstd"
"github.com/spf13/afero" "github.com/spf13/afero"
"google.golang.org/protobuf/encoding/protowire" "google.golang.org/protobuf/encoding/protowire"
@@ -19,29 +19,26 @@ import (
var ( var (
errInvalidUUIDLength = errors.New("invalid UUID length") errInvalidUUIDLength = errors.New("invalid UUID length")
errInvalidUUIDFormat = errors.New("invalid UUID format")
errUnknownVersion = errors.New("unknown version") errUnknownVersion = errors.New("unknown version")
errUnknownCompression = errors.New("unknown compression type") errUnknownCompression = errors.New("unknown compression type")
errCompressedHashWrong = errors.New("compressed data hash mismatch") errCompressedHashWrong = errors.New("compressed data hash mismatch")
errSignatureNoPubKey = errors.New("signature present but no public key") errSignatureNoPubKey = errors.New("signature present but no public key")
errDecompressedTooLarge = errors.New("decompressed data exceeds maximum allowed size") errDecompressedTooLarge = errors.New("decompressed data exceeds maximum allowed size")
errManifestTooLarge = errors.New("file exceeds maximum allowed size")
errUUIDMismatch = errors.New("outer and inner UUID mismatch") errUUIDMismatch = errors.New("outer and inner UUID mismatch")
errInvalidFileFormat = errors.New("invalid file format") errInvalidFileFormat = errors.New("invalid file format")
errInvalidManifestPath = errors.New("manifest contains invalid path") errInvalidManifestPath = errors.New("invalid file entry")
errDecodedTooLarge = errors.New( errDecodedTooLarge = errors.New("too much memory needed")
"manifest would take too much memory to decode") errSignerNotSigningKey = errors.New(
"signer is not the fingerprint of the key that made the signature")
) )
// validateUUID checks that the byte slice is a valid UUID (16 bytes, parseable). // validateUUID checks that the byte slice is the 16 bytes of a binary UUID.
// Any 16 bytes are one, so the length is all there is to check.
func validateUUID(data []byte) error { func validateUUID(data []byte) error {
if len(data) != uuidLength { if len(data) != uuidLength {
return errInvalidUUIDLength return errInvalidUUIDLength
} }
// Try to parse as UUID to validate format
_, err := uuid.FromBytes(data)
if err != nil {
return errInvalidUUIDFormat
}
return nil return nil
} }
@@ -60,20 +57,22 @@ func (m *manifest) validateOuterHeader() error {
// Validate outer UUID before any decompression // Validate outer UUID before any decompression
err := validateUUID(m.pbOuter.GetUuid()) err := validateUUID(m.pbOuter.GetUuid())
if err != nil { if err != nil {
return fmt.Errorf("outer UUID invalid: %w", err) return fmt.Errorf("outer message: %w", err)
} }
return nil return nil
} }
// verifyOuterIntegrity checks the hash of the compressed payload and, // verifyOuterIntegrity checks the hash of the compressed payload and, if a
// if a signature is present, verifies it against the embedded public key. // signature is present, verifies it against the embedded public key, which
// must be one key, and checks that the signer field is that key's
// fingerprint.
func (m *manifest) verifyOuterIntegrity() error { func (m *manifest) verifyOuterIntegrity() error {
h := sha256.New() h := sha256.New()
_, err := h.Write(m.pbOuter.GetInnerMessage()) _, err := h.Write(m.pbOuter.GetInnerMessage())
if err != nil { if err != nil {
return fmt.Errorf("deserialize: hash write: %w", err) return fmt.Errorf("hash inner message: %w", err)
} }
sha256Hash := h.Sum(nil) sha256Hash := h.Sum(nil)
@@ -91,20 +90,23 @@ func (m *manifest) verifyOuterIntegrity() error {
sigString, err := m.signatureString() sigString, err := m.signatureString()
if err != nil { if err != nil {
return fmt.Errorf( return fmt.Errorf("build signature string: %w", err)
"failed to generate signature string for verification: %w", err,
)
} }
// Loading a manifest takes no context; gpgTimeout still bounds gpg. // Loading a manifest takes no context; gpgTimeout still bounds gpg.
err = gpgVerify( signingKey, err := gpgVerify(
context.Background(), context.Background(),
[]byte(sigString), []byte(sigString),
m.pbOuter.GetSignature(), m.pbOuter.GetSignature(),
m.pbOuter.GetSigningPubKey(), m.pbOuter.GetSigningPubKey(),
) )
if err != nil { if err != nil {
return fmt.Errorf("signature verification failed: %w", err) return err
}
if !strings.EqualFold(string(m.pbOuter.GetSigner()), signingKey) {
return fmt.Errorf("%w: signer %q, signing key %s",
errSignerNotSigningKey, m.pbOuter.GetSigner(), signingKey)
} }
log.Infof("signature verified successfully") log.Infof("signature verified successfully")
@@ -130,7 +132,7 @@ func (m *manifest) decompressInner() ([]byte, error) {
zstd.WithDecodeBuffersBelow(0), zstd.WithDecodeBuffersBelow(0),
zstd.WithDecoderMaxWindow(zstdWindowSize)) zstd.WithDecoderMaxWindow(zstdWindowSize))
if err != nil { if err != nil {
return nil, fmt.Errorf("deserialize: zstd reader: %w", err) return nil, fmt.Errorf("create decompressor: %w", err)
} }
defer zr.Close() defer zr.Close()
@@ -145,7 +147,7 @@ func (m *manifest) decompressInner() ([]byte, error) {
dat, err := io.ReadAll(limitedReader) dat, err := io.ReadAll(limitedReader)
if err != nil { if err != nil {
return nil, fmt.Errorf("deserialize: decompress: %w", err) return nil, fmt.Errorf("decompress inner message: %w", err)
} }
if int64(len(dat)) >= MaxDecompressedSize { if int64(len(dat)) >= MaxDecompressedSize {
@@ -261,7 +263,7 @@ func (m *manifest) deserializeInner() error {
err = checkDecodedSize(dat) err = checkDecodedSize(dat)
if err != nil { if err != nil {
return fmt.Errorf("deserialize: unmarshal inner: %w", err) return fmt.Errorf("unmarshal inner message: %w", err)
} }
// Deserialize inner message // Deserialize inner message
@@ -270,13 +272,17 @@ func (m *manifest) deserializeInner() error {
// Unknown fields would cost memory; mfer never writes a loaded manifest out. // Unknown fields would cost memory; mfer never writes a loaded manifest out.
err = proto.UnmarshalOptions{DiscardUnknown: true}.Unmarshal(dat, m.pbInner) err = proto.UnmarshalOptions{DiscardUnknown: true}.Unmarshal(dat, m.pbInner)
if err != nil { if err != nil {
return fmt.Errorf("deserialize: unmarshal inner: %w", err) return fmt.Errorf("unmarshal inner message: %w", err)
}
if m.pbInner.GetVersion() != MFFile_VERSION_ONE {
return errUnknownVersion
} }
// Validate inner UUID // Validate inner UUID
err = validateUUID(m.pbInner.GetUuid()) err = validateUUID(m.pbInner.GetUuid())
if err != nil { if err != nil {
return fmt.Errorf("inner UUID invalid: %w", err) return fmt.Errorf("inner message: %w", err)
} }
// Verify UUIDs match // Verify UUIDs match
@@ -289,12 +295,21 @@ func (m *manifest) deserializeInner() error {
// extract path tomorrow — acts on a traversal or absolute path from an // extract path tomorrow — acts on a traversal or absolute path from an
// untrusted .mf. Reject loudly on the first offender rather than // untrusted .mf. Reject loudly on the first offender rather than
// dropping entries, which would let a hostile manifest hide files from a // dropping entries, which would let a hostile manifest hide files from a
// check. // check. A path listed twice is refused too: check would check the one
// file against both entries.
seen := make(map[string]bool, len(m.pbInner.GetFiles()))
for _, f := range m.pbInner.GetFiles() { for _, f := range m.pbInner.GetFiles() {
err = ValidatePath(f.GetPath()) err = ValidatePath(f.GetPath())
if err != nil { if err != nil {
return fmt.Errorf("%w: %w", errInvalidManifestPath, err) return fmt.Errorf("%w: %w", errInvalidManifestPath, err)
} }
if seen[f.GetPath()] {
return fmt.Errorf("%w %q", errDuplicatePath, f.GetPath())
}
seen[f.GetPath()] = true
} }
log.Infof("loaded manifest with %d files", len(m.pbInner.GetFiles())) log.Infof("loaded manifest with %d files", len(m.pbInner.GetFiles()))
@@ -314,13 +329,14 @@ func validateMagic(dat []byte) bool {
return bytes.Equal(got, expected) return bytes.Equal(got, expected)
} }
// NewManifestFromReader reads a manifest from an io.Reader. // NewManifestFromReader reads a manifest from an io.Reader. It refuses a
// manifest larger than MaxManifestSize, reading at most one byte past it.
// //
//nolint:revive // unexported-return: exporting manifest is owner question 13 //nolint:revive // unexported-return: exporting manifest is owner question 13
func NewManifestFromReader(input io.Reader) (*manifest, error) { func NewManifestFromReader(input io.Reader) (*manifest, error) {
m := &manifest{} m := &manifest{}
dat, err := io.ReadAll(input) dat, err := readAtMost(input, MaxManifestSize)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -352,6 +368,21 @@ func NewManifestFromReader(input io.Reader) (*manifest, error) {
return m, nil return m, nil
} }
// readAtMost reads all of input, or refuses it with errManifestTooLarge
// once it passes maxSize bytes, after reading one byte past maxSize.
func readAtMost(input io.Reader, maxSize int64) ([]byte, error) {
dat, err := io.ReadAll(io.LimitReader(input, maxSize+1))
if err != nil {
return nil, err
}
if int64(len(dat)) > maxSize {
return nil, fmt.Errorf("%w of %d bytes", errManifestTooLarge, maxSize)
}
return dat, nil
}
// ManifestFromFileOptions configures NewManifestFromFile. // ManifestFromFileOptions configures NewManifestFromFile.
type ManifestFromFileOptions struct { type ManifestFromFileOptions struct {
// Path is the manifest file to read (required). // Path is the manifest file to read (required).
+80 -28
View File
@@ -7,11 +7,10 @@ import (
"crypto/sha256" "crypto/sha256"
"fmt" "fmt"
"strconv" "strconv"
"strings"
"testing" "testing"
"time" "time"
"uuid"
"github.com/google/uuid"
"github.com/klauspost/compress/zstd" "github.com/klauspost/compress/zstd"
"github.com/multiformats/go-multihash" "github.com/multiformats/go-multihash"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
@@ -92,7 +91,7 @@ func TestDeserializeRejectsInvalidEntryPaths(t *testing.T) {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
t.Parallel() t.Parallel()
id := uuid.New() id := uuid.NewV4()
data := wrapInner(t, id, craftInnerBytes(id, tt.path)) data := wrapInner(t, id, craftInnerBytes(id, tt.path))
_, err := NewManifestFromReader(bytes.NewReader(data)) _, err := NewManifestFromReader(bytes.NewReader(data))
@@ -118,12 +117,61 @@ func TestDeserializeRejectsInvalidEntryPaths(t *testing.T) {
} }
} }
// A manifest that lists a path twice is refused as it is loaded, naming the
// path. Paths are compared byte for byte: two that differ only in letter case
// load, and fetch refuses those itself. Each entry has a hash, as entries mfer
// writes do; entries of a path alone would take too much memory to decode.
func TestDeserializeRefusesPathListedTwice(t *testing.T) {
t.Parallel()
hash, err := multihash.Encode(make([]byte, sha256.Size), multihash.SHA2_256)
require.NoError(t, err)
tests := []struct {
name string
paths []string
refused bool
}{
{"same path twice", []string{"dir/a.txt", "other.txt", "dir/a.txt"}, true},
{"paths differing in letter case", []string{"dir/b.txt", "dir/B.txt"}, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
id := uuid.NewV4()
inner := &MFFile{Version: MFFile_VERSION_ONE, Uuid: id[:]}
for _, p := range tt.paths {
inner.Files = append(inner.Files, &MFFilePath{
Path: p,
Hashes: []*MFFileChecksum{{MultiHash: hash}},
})
}
innerData, err := proto.Marshal(inner)
require.NoError(t, err)
m, err := NewManifestFromReader(bytes.NewReader(wrapInner(t, id, innerData)))
if tt.refused {
require.ErrorIs(t, err, errDuplicatePath)
require.EqualError(t, err, `duplicate path "dir/a.txt"`)
} else {
require.NoError(t, err)
assert.Len(t, m.Files(), len(tt.paths))
}
})
}
}
// Entries of a path, an empty hash, an empty MIME type and empty modification // Entries of a path, an empty hash, an empty MIME type and empty modification
// and change times are counted at 416 bytes each (160 + 112 + 16 + 64 + 64) // and change times are counted at 432 bytes each (176 + 112 + 16 + 64 + 64)
// and take 16 bytes plus the path to encode. A 35-character path makes that // and take 16 bytes plus the path to encode. A 37-character path makes that
// 51 bytes, about 8.2 times: refused, and leaving any one of the five // 53 bytes, about 8.2 times: refused, and leaving any one of the five
// uncounted, even the MIME type, brings it under 8. A 37-character path makes // uncounted, even the MIME type, brings it under 8. A 39-character path makes
// it 53 bytes, about 7.8 times: loaded. // it 55 bytes, about 7.9 times: loaded. Each entry's path is its number,
// padded with zeros to that length, since a manifest lists a path only once.
func TestDeserializeRefusesEntriesThatDecodeTooLarge(t *testing.T) { func TestDeserializeRefusesEntriesThatDecodeTooLarge(t *testing.T) {
t.Parallel() t.Parallel()
@@ -131,30 +179,32 @@ func TestDeserializeRefusesEntriesThatDecodeTooLarge(t *testing.T) {
pathLen int pathLen int
refused bool refused bool
}{ }{
{35, true}, {37, true},
{37, false}, {39, false},
} }
for _, tt := range tests { for _, tt := range tests {
t.Run(strconv.Itoa(tt.pathLen), func(t *testing.T) { t.Run(strconv.Itoa(tt.pathLen), func(t *testing.T) {
t.Parallel() t.Parallel()
entry := protowire.AppendTag(nil, 1, protowire.BytesType) // MFFilePath.path id := uuid.NewV4()
entry = protowire.AppendString(entry, strings.Repeat("a", tt.pathLen)) inner := protowire.AppendTag(nil, 100, protowire.VarintType) // MFFile.version
entry = protowire.AppendTag(entry, 3, protowire.BytesType) // MFFilePath.hashes inner = protowire.AppendVarint(inner, uint64(MFFile_VERSION_ONE))
entry = protowire.AppendBytes(entry, nil) inner = protowire.AppendTag(inner, 102, protowire.BytesType) // MFFile.uuid
entry = protowire.AppendTag(entry, 301, protowire.BytesType) // MFFilePath.mimeType
entry = protowire.AppendBytes(entry, nil)
entry = protowire.AppendTag(entry, 302, protowire.BytesType) // MFFilePath.mtime
entry = protowire.AppendBytes(entry, nil)
entry = protowire.AppendTag(entry, 303, protowire.BytesType) // MFFilePath.ctime
entry = protowire.AppendBytes(entry, nil)
id := uuid.New()
inner := protowire.AppendTag(nil, 102, protowire.BytesType) // MFFile.uuid
inner = protowire.AppendBytes(inner, id[:]) inner = protowire.AppendBytes(inner, id[:])
for range 1000 { for i := range 1000 {
entry := protowire.AppendTag(nil, 1, protowire.BytesType) // MFFilePath.path
entry = protowire.AppendString(entry, fmt.Sprintf("%0*d", tt.pathLen, i))
entry = protowire.AppendTag(entry, 3, protowire.BytesType) // MFFilePath.hashes
entry = protowire.AppendBytes(entry, nil)
entry = protowire.AppendTag(entry, 301, protowire.BytesType) // MFFilePath.mimeType
entry = protowire.AppendBytes(entry, nil)
entry = protowire.AppendTag(entry, 302, protowire.BytesType) // MFFilePath.mtime
entry = protowire.AppendBytes(entry, nil)
entry = protowire.AppendTag(entry, 303, protowire.BytesType) // MFFilePath.ctime
entry = protowire.AppendBytes(entry, nil)
inner = protowire.AppendTag(inner, 101, protowire.BytesType) // MFFile.files inner = protowire.AppendTag(inner, 101, protowire.BytesType) // MFFile.files
inner = protowire.AppendBytes(inner, entry) inner = protowire.AppendBytes(inner, entry)
} }
@@ -181,8 +231,10 @@ func TestDeserializeDropsUnknownFields(t *testing.T) {
entry = protowire.AppendString(entry, "a") entry = protowire.AppendString(entry, "a")
entry = append(entry, unknown...) entry = append(entry, unknown...)
id := uuid.New() id := uuid.NewV4()
inner := protowire.AppendTag(nil, 101, protowire.BytesType) // MFFile.files inner := protowire.AppendTag(nil, 100, protowire.VarintType) // MFFile.version
inner = protowire.AppendVarint(inner, uint64(MFFile_VERSION_ONE))
inner = protowire.AppendTag(inner, 101, protowire.BytesType) // MFFile.files
inner = protowire.AppendBytes(inner, entry) inner = protowire.AppendBytes(inner, entry)
inner = protowire.AppendTag(inner, 102, protowire.BytesType) // MFFile.uuid inner = protowire.AppendTag(inner, 102, protowire.BytesType) // MFFile.uuid
inner = protowire.AppendBytes(inner, id[:]) inner = protowire.AppendBytes(inner, id[:])
@@ -215,7 +267,7 @@ func TestDeserializeLoadsDensestManifest(t *testing.T) {
const files = 10000 const files = 10000
for i := range files { for i := range files {
name := RelFilePath(strconv.FormatInt(int64(i), 36)) name := RelFilePath(strconv.FormatInt(int64(i), 36))
require.NoError(t, b.AddFileWithHash(name, 0, ModTime(time.Unix(0, 0)), hash)) require.NoError(t, b.AddFileWithHash(name, 0, ModTime(time.Unix(0, 0)), 0, hash))
} }
var buf bytes.Buffer var buf bytes.Buffer
@@ -233,7 +285,7 @@ func TestDeserializeValidManifestRoundTrips(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
b := NewBuilder() b := NewBuilder()
require.NoError(t, b.AddFileWithHash("dir/file.txt", 123, ModTime{}, hash)) require.NoError(t, b.AddFileWithHash("dir/file.txt", 123, ModTime{}, 0, hash))
var buf bytes.Buffer var buf bytes.Buffer
require.NoError(t, b.Build(context.Background(), &buf)) require.NoError(t, b.Build(context.Background(), &buf))
+54
View File
@@ -0,0 +1,54 @@
//nolint:testpackage // white-box tests exercise unexported internals
package mfer
import (
"bytes"
"testing"
"uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"google.golang.org/protobuf/proto"
)
// An inner message whose version is not VERSION_ONE, whether version 0 or a
// later one, is refused with the same error as an outer message's.
func TestDeserializeRefusesUnknownInnerVersion(t *testing.T) {
t.Parallel()
for _, version := range []MFFile_Version{MFFile_VERSION_NONE, MFFile_VERSION_ONE + 1} {
t.Run(version.String(), func(t *testing.T) {
t.Parallel()
id := uuid.NewV4()
inner, err := proto.Marshal(&MFFile{Version: version, Uuid: id[:]})
require.NoError(t, err)
_, err = NewManifestFromReader(bytes.NewReader(wrapInner(t, id, inner)))
require.ErrorIs(t, err, errUnknownVersion)
})
}
}
// TestReadAtMost gives readAtMost exactly its maximum, which it must
// return whole, and twice its maximum, which it must refuse after reading
// one byte past the maximum, and no more. NewManifestFromReader reads
// through it with MaxManifestSize; the test uses 64 KiB, since reading
// MaxManifestSize under the race detector takes gigabytes of memory.
func TestReadAtMost(t *testing.T) {
t.Parallel()
const maxSize = 64 << 10
dat, err := readAtMost(bytes.NewReader(make([]byte, maxSize)), maxSize)
require.NoError(t, err)
assert.Len(t, dat, maxSize)
input := bytes.NewReader(make([]byte, 2*maxSize))
_, err = readAtMost(input, maxSize)
require.ErrorIs(t, err, errManifestTooLarge)
require.EqualError(t, err,
"file exceeds maximum allowed size of 65536 bytes")
assert.Equal(t, maxSize-1, input.Len(), "bytes left unread")
}
+11 -7
View File
@@ -74,14 +74,18 @@ func TestValidatePathMessagesVerbatim(t *testing.T) {
} }
} }
// TestSerializeInternalErrorMessagesVerbatim pins the two distinct // TestSerializeInnerNotSetMessagesVerbatim pins the messages generate and
// "internal error" messages, which differ between generate and // generateOuter return when the inner message is missing.
// generateOuter and have always done so. func TestSerializeInnerNotSetMessagesVerbatim(t *testing.T) {
func TestSerializeInternalErrorMessagesVerbatim(t *testing.T) {
t.Parallel() t.Parallel()
m := &manifest{} m := &manifest{}
require.EqualError(t, m.generate(context.Background()),
"internal error: pbInner not set") err := m.generate(context.Background())
require.EqualError(t, m.generateOuter(context.Background()), "internal error") require.ErrorIs(t, err, errInnerNotSet)
require.EqualError(t, err, "inner message not set")
err = m.generateOuter(context.Background())
require.ErrorIs(t, err, errInternal)
require.EqualError(t, err, "inner message not set")
} }
+138 -89
View File
@@ -41,16 +41,26 @@ const (
// fields in a gpg fingerprint record (the fingerprint is field 10). // fields in a gpg fingerprint record (the fingerprint is field 10).
gpgFingerprintMinFields = 10 gpgFingerprintMinFields = 10
// gpgStatusPrefix starts each status line gpg writes to the file
// descriptor named by --status-fd.
gpgStatusPrefix = "[GNUPG:]"
// gpg option names used from more than one call site. // gpg option names used from more than one call site.
gpgOptArmor = "--armor" gpgOptArmor = "--armor"
gpgOptHomedir = "--homedir" gpgOptHomedir = "--homedir"
gpgOptVerify = "--verify" gpgOptStatusFD = "--status-fd"
gpgOptVerify = "--verify"
) )
var ( var (
errGPGKeyNotFound = errors.New("gpg key not found") errGPGKeyNotFound = errors.New("GPG key not found")
errFingerprintNotFound = errors.New("fingerprint not found for key") errFingerprintNotFound = errors.New("fingerprint not found for key")
errImportedFPRNotFound = errors.New("fingerprint not found in imported key") errSigningKeyCount = errors.New(
"embedded public key block must hold exactly one key")
errNotOneGoodSignature = errors.New(
"gpg did not report exactly one good signature")
errSigningKeyNotReported = errors.New(
"gpg did not report the key that made the signature")
) )
// GPGKeyID represents a GPG key identifier (fingerprint or key ID). // GPGKeyID represents a GPG key identifier (fingerprint or key ID).
@@ -78,8 +88,9 @@ func gpgArgs(opts []string, positional ...string) []string {
} }
// runGPG runs the gpg binary in batch mode with the given arguments and // runGPG runs the gpg binary in batch mode with the given arguments and
// optional stdin, returning captured stdout and stderr. gpg is killed when // optional stdin, returning captured stdout and stderr. If gpg fails, the
// ctx ends or gpgTimeout passes, whichever comes first. // error ends with what gpg wrote to stderr. gpg is killed when ctx ends or
// gpgTimeout passes, whichever comes first.
func runGPG( func runGPG(
ctx context.Context, stdin io.Reader, args ...string, ctx context.Context, stdin io.Reader, args ...string,
) (*bytes.Buffer, *bytes.Buffer, error) { ) (*bytes.Buffer, *bytes.Buffer, error) {
@@ -115,17 +126,32 @@ func runGPG(
// "signal: killed"; return the reason instead. // "signal: killed"; return the reason instead.
err = ctx.Err() err = ctx.Err()
if errors.Is(err, context.DeadlineExceeded) { if errors.Is(err, context.DeadlineExceeded) {
err = fmt.Errorf("gpg timed out: %w", err) err = fmt.Errorf("timed out: %w", err)
} }
} }
if err != nil {
err = withStderr(err, &stderr)
}
return &stdout, &stderr, err return &stdout, &stderr, err
} }
// withStderr returns err followed by what gpg wrote to stderr, or err alone
// when gpg wrote nothing.
func withStderr(err error, stderr *bytes.Buffer) error {
messages := strings.TrimSpace(stderr.String())
if messages == "" {
return err
}
return fmt.Errorf("%w: %s", err, messages)
}
// parseFingerprint extracts the first fingerprint from gpg --with-colons // parseFingerprint extracts the first fingerprint from gpg --with-colons
// output, or returns ok=false if none is present. // output, or returns ok=false if none is present.
func parseFingerprint(colonOutput string) (string, bool) { func parseFingerprint(colonOutput string) (string, bool) {
for _, line := range strings.Split(colonOutput, "\n") { for line := range strings.SplitSeq(colonOutput, "\n") {
fields := strings.Split(line, ":") fields := strings.Split(line, ":")
if len(fields) >= gpgFingerprintMinFields && if len(fields) >= gpgFingerprintMinFields &&
fields[0] == gpgFingerprintField { fields[0] == gpgFingerprintField {
@@ -136,29 +162,77 @@ func parseFingerprint(colonOutput string) (string, bool) {
return "", false return "", false
} }
// gpgSign creates a detached signature of the data using the specified key. // parseStatusLine returns the arguments of the status line for keyword in
// Returns the armored detached signature. // gpg --status-fd output, or ok=false unless there is exactly one such line
func gpgSign(ctx context.Context, data []byte, keyID GPGKeyID) ([]byte, error) { // and it has arguments.
func parseStatusLine(statusOutput, keyword string) ([]string, bool) {
var found [][]string
for line := range strings.SplitSeq(statusOutput, "\n") {
fields := strings.Fields(line)
if len(fields) > 2 && fields[0] == gpgStatusPrefix && fields[1] == keyword {
found = append(found, fields[2:])
}
}
if len(found) != 1 {
return nil, false
}
return found[0], true
}
// gpgSign creates an armored detached signature of data with the key gpg
// picks for keyID, and returns it with the fingerprint of the key that made
// it, which is a subkey's when gpg signed with a subkey.
func gpgSign(
ctx context.Context, data []byte, keyID GPGKeyID,
) ([]byte, string, error) {
tmpDir, err := os.MkdirTemp("", "mfer-gpg-sign-*")
if err != nil {
return nil, "", err
}
defer func() { _ = os.RemoveAll(tmpDir) }()
sigFile := filepath.Join(tmpDir, "signature.asc")
// The signature goes to sigFile, so --status-fd 1 can send gpg's status
// lines to stdout; its messages go to stderr.
stdout, stderr, err := runGPG(ctx, bytes.NewReader(data), stdout, stderr, err := runGPG(ctx, bytes.NewReader(data),
"--detach-sign", "--detach-sign",
gpgOptArmor, gpgOptArmor,
"--output", sigFile,
gpgOptStatusFD, "1",
"--local-user", string(keyID), "--local-user", string(keyID),
) )
if err != nil { if err != nil {
return nil, fmt.Errorf("gpg sign failed: %w: %s", err, stderr.String()) return nil, "", fmt.Errorf("gpg sign: %w", err)
} }
return stdout.Bytes(), nil // The last argument of SIG_CREATED is the fingerprint of the key that
// made the signature.
created, ok := parseStatusLine(stdout.String(), "SIG_CREATED")
if !ok {
return nil, "", withStderr(errSigningKeyNotReported, stderr)
}
sig, err := os.ReadFile(sigFile) //nolint:gosec // G304: inside tmpDir, made above
if err != nil {
return nil, "", err
}
return sig, created[len(created)-1], nil
} }
// gpgExportPublicKey exports the public key for the specified key ID. // gpgExportPublicKey exports the public key for the specified key ID.
// Returns the armored public key. // Returns the armored public key.
func gpgExportPublicKey(ctx context.Context, keyID GPGKeyID) ([]byte, error) { func gpgExportPublicKey(ctx context.Context, keyID GPGKeyID) ([]byte, error) {
stdout, stderr, err := runGPG(ctx, nil, stdout, _, err := runGPG(ctx, nil,
gpgArgs([]string{"--export", gpgOptArmor}, string(keyID))..., gpgArgs([]string{"--export", gpgOptArmor}, string(keyID))...,
) )
if err != nil { if err != nil {
return nil, fmt.Errorf("gpg export failed: %w: %s", err, stderr.String()) return nil, fmt.Errorf("gpg export: %w", err)
} }
if stdout.Len() == 0 { if stdout.Len() == 0 {
@@ -170,13 +244,11 @@ func gpgExportPublicKey(ctx context.Context, keyID GPGKeyID) ([]byte, error) {
// gpgGetKeyFingerprint gets the full fingerprint for a key ID. // gpgGetKeyFingerprint gets the full fingerprint for a key ID.
func gpgGetKeyFingerprint(ctx context.Context, keyID GPGKeyID) ([]byte, error) { func gpgGetKeyFingerprint(ctx context.Context, keyID GPGKeyID) ([]byte, error) {
stdout, stderr, err := runGPG(ctx, nil, stdout, _, err := runGPG(ctx, nil,
gpgArgs([]string{"--with-colons", "--fingerprint"}, string(keyID))..., gpgArgs([]string{"--with-colons", "--fingerprint"}, string(keyID))...,
) )
if err != nil { if err != nil {
return nil, fmt.Errorf( return nil, fmt.Errorf("gpg fingerprint lookup: %w", err)
"gpg fingerprint lookup failed: %w: %s", err, stderr.String(),
)
} }
fpr, ok := parseFingerprint(stdout.String()) fpr, ok := parseFingerprint(stdout.String())
@@ -187,69 +259,44 @@ func gpgGetKeyFingerprint(ctx context.Context, keyID GPGKeyID) ([]byte, error) {
return []byte(fpr), nil return []byte(fpr), nil
} }
// gpgExtractPubKeyFingerprint imports a public key into a temporary keyring // gpgImportOneKey imports the public key block in pubKeyFile into the
// and extracts its fingerprint. This verifies the key is valid and returns // keyring in gpgHome. The block must hold exactly one primary key.
// the actual fingerprint from the key material. func gpgImportOneKey(ctx context.Context, gpgHome, pubKeyFile string) error {
func gpgExtractPubKeyFingerprint(ctx context.Context, pubKey []byte) (string, error) { // --status-fd 1 sends gpg's status lines to stdout, which importing
// Create temporary directory for GPG operations // otherwise leaves empty; its messages go to stderr.
tmpDir, err := os.MkdirTemp("", "mfer-gpg-fingerprint-*") importStdout, _, err := runGPG(ctx, nil,
if err != nil { gpgArgs([]string{gpgOptHomedir, gpgHome, gpgOptStatusFD, "1", "--import"},
return "", fmt.Errorf("failed to create temp dir: %w", err) pubKeyFile)...,
}
defer func() { _ = os.RemoveAll(tmpDir) }()
// Set restrictive permissions
err = os.Chmod(tmpDir, privateDirPerms)
if err != nil {
return "", fmt.Errorf("failed to set temp dir permissions: %w", err)
}
// Write public key to temp file
pubKeyFile := filepath.Join(tmpDir, "pubkey.asc")
err = os.WriteFile(pubKeyFile, pubKey, privateFilePerms)
if err != nil {
return "", fmt.Errorf("failed to write public key: %w", err)
}
// Import the public key into the temporary keyring
_, importStderr, err := runGPG(ctx, nil,
gpgArgs([]string{gpgOptHomedir, tmpDir, "--import"}, pubKeyFile)...,
) )
if err != nil { if err != nil {
return "", fmt.Errorf( return fmt.Errorf("gpg import: %w", err)
"failed to import public key: %w: %s", err, importStderr.String(),
)
} }
// List keys to get fingerprint // The first argument of IMPORT_RES counts the primary keys gpg read
listStdout, listStderr, err := runGPG(ctx, nil, // from the block, those it then skipped (one with no user ID, for
"--homedir", tmpDir, // example) included.
"--with-colons", result, ok := parseStatusLine(importStdout.String(), "IMPORT_RES")
"--fingerprint",
)
if err != nil {
return "", fmt.Errorf(
"failed to list keys: %w: %s", err, listStderr.String(),
)
}
fpr, ok := parseFingerprint(listStdout.String())
if !ok { if !ok {
return "", errImportedFPRNotFound return fmt.Errorf("%w, gpg reported no count", errSigningKeyCount)
} }
return fpr, nil if result[0] != "1" {
return fmt.Errorf("%w, found %s", errSigningKeyCount, result[0])
}
return nil
} }
// gpgVerify verifies a detached signature against data using the provided public key. // gpgVerify verifies a detached signature against data using the provided
// It creates a temporary keyring to import the public key for verification. // public key, imported into a temporary keyring, and returns the
func gpgVerify(ctx context.Context, data, signature, pubKey []byte) error { // fingerprint of the primary key that made the signature. The public key
// must hold exactly one primary key, so that a good signature can come
// from no other key.
func gpgVerify(ctx context.Context, data, signature, pubKey []byte) (string, error) {
// Create temporary directory for GPG operations // Create temporary directory for GPG operations
tmpDir, err := os.MkdirTemp("", "mfer-gpg-verify-*") tmpDir, err := os.MkdirTemp("", "mfer-gpg-verify-*")
if err != nil { if err != nil {
return fmt.Errorf("failed to create temp dir: %w", err) return "", err
} }
defer func() { _ = os.RemoveAll(tmpDir) }() defer func() { _ = os.RemoveAll(tmpDir) }()
@@ -257,7 +304,7 @@ func gpgVerify(ctx context.Context, data, signature, pubKey []byte) error {
// Set restrictive permissions // Set restrictive permissions
err = os.Chmod(tmpDir, privateDirPerms) err = os.Chmod(tmpDir, privateDirPerms)
if err != nil { if err != nil {
return fmt.Errorf("failed to set temp dir permissions: %w", err) return "", err
} }
// Write public key to temp file // Write public key to temp file
@@ -265,7 +312,7 @@ func gpgVerify(ctx context.Context, data, signature, pubKey []byte) error {
err = os.WriteFile(pubKeyFile, pubKey, privateFilePerms) err = os.WriteFile(pubKeyFile, pubKey, privateFilePerms)
if err != nil { if err != nil {
return fmt.Errorf("failed to write public key: %w", err) return "", err
} }
// Write signature to temp file // Write signature to temp file
@@ -273,7 +320,7 @@ func gpgVerify(ctx context.Context, data, signature, pubKey []byte) error {
err = os.WriteFile(sigFile, signature, privateFilePerms) err = os.WriteFile(sigFile, signature, privateFilePerms)
if err != nil { if err != nil {
return fmt.Errorf("failed to write signature: %w", err) return "", err
} }
// Write data to temp file // Write data to temp file
@@ -281,29 +328,31 @@ func gpgVerify(ctx context.Context, data, signature, pubKey []byte) error {
err = os.WriteFile(dataFile, data, privateFilePerms) err = os.WriteFile(dataFile, data, privateFilePerms)
if err != nil { if err != nil {
return fmt.Errorf("failed to write data: %w", err) return "", err
} }
// Import the public key into the temporary keyring err = gpgImportOneKey(ctx, tmpDir, pubKeyFile)
_, importStderr, err := runGPG(ctx, nil,
gpgArgs([]string{gpgOptHomedir, tmpDir, "--import"}, pubKeyFile)...,
)
if err != nil { if err != nil {
return fmt.Errorf( return "", err
"failed to import public key: %w: %s", err, importStderr.String(),
)
} }
// Verify the signature // --status-fd 1 sends gpg's status lines to stdout, which verifying a
_, verifyStderr, err := runGPG(ctx, nil, // detached signature otherwise leaves empty; its messages go to stderr.
gpgArgs([]string{gpgOptHomedir, tmpDir, gpgOptVerify}, verifyStdout, _, err := runGPG(ctx, nil,
gpgArgs([]string{gpgOptHomedir, tmpDir, gpgOptStatusFD, "1", gpgOptVerify},
sigFile, dataFile)..., sigFile, dataFile)...,
) )
if err != nil { if err != nil {
return fmt.Errorf( return "", fmt.Errorf("gpg verify: %w", err)
"signature verification failed: %w: %s", err, verifyStderr.String(),
)
} }
return nil // gpg writes a VALIDSIG line for each good signature. Its first
// argument is the fingerprint of the key that made the signature,
// which may be a subkey; its last is that of the primary key.
valid, ok := parseStatusLine(verifyStdout.String(), "VALIDSIG")
if !ok {
return "", errNotOneGoodSignature
}
return valid[len(valid)-1], nil
} }
+258 -43
View File
@@ -8,6 +8,7 @@ import (
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
"slices"
"strconv" "strconv"
"strings" "strings"
"syscall" "syscall"
@@ -17,6 +18,7 @@ import (
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"google.golang.org/protobuf/proto"
) )
// testGPGEnv sets up a temporary GPG home directory with a test key. // testGPGEnv sets up a temporary GPG home directory with a test key.
@@ -35,39 +37,18 @@ func testGPGEnv(t *testing.T) (GPGKeyID, string) {
// Create temporary GPG home directory (0700 by default) // Create temporary GPG home directory (0700 by default)
gpgHome := t.TempDir() gpgHome := t.TempDir()
// Generate a test key with no passphrase genTestKey(t, gpgHome, testKeyParams)
keyParams := `%no-protection
Key-Type: RSA
Key-Length: 2048
Name-Real: MFER Test Key
Name-Email: test@mfer.test
Expire-Date: 0
%commit
`
paramsFile := filepath.Join(gpgHome, "key-params")
require.NoError(t, os.WriteFile(paramsFile, []byte(keyParams), 0o600))
ctx, cancel := context.WithTimeout(context.Background(), gpgTimeout) ctx, cancel := context.WithTimeout(context.Background(), gpgTimeout)
defer cancel() defer cancel()
//nolint:gosec // paramsFile is a test-controlled path inside t.TempDir()
cmd := exec.CommandContext(ctx, "gpg",
"--batch", "--gen-key", paramsFile)
cmd.Env = append(os.Environ(), "GNUPGHOME="+gpgHome)
output, err := cmd.CombinedOutput()
if err != nil {
t.Skipf("failed to generate test GPG key: %v: %s", err, output)
}
// Get the key fingerprint // Get the key fingerprint
cmd = exec.CommandContext(ctx, "gpg", cmd := exec.CommandContext(ctx, "gpg",
"--list-keys", "--with-colons", "test@mfer.test") "--list-keys", "--with-colons", "test@mfer.test")
cmd.Env = append(os.Environ(), "GNUPGHOME="+gpgHome) cmd.Env = append(os.Environ(), "GNUPGHOME="+gpgHome)
output, err = cmd.Output() output, err := cmd.Output()
if err != nil { if err != nil {
t.Fatalf("failed to list test key: %v", err) t.Fatalf("failed to list test key: %v", err)
} }
@@ -75,7 +56,7 @@ Expire-Date: 0
// Parse fingerprint from output // Parse fingerprint from output
var keyID string var keyID string
for _, line := range strings.Split(string(output), "\n") { for line := range strings.SplitSeq(string(output), "\n") {
fields := strings.Split(line, ":") fields := strings.Split(line, ":")
if len(fields) >= gpgFingerprintMinFields && if len(fields) >= gpgFingerprintMinFields &&
fields[0] == gpgFingerprintField { fields[0] == gpgFingerprintField {
@@ -92,13 +73,79 @@ Expire-Date: 0
return GPGKeyID(keyID), gpgHome return GPGKeyID(keyID), gpgHome
} }
// testKeyParams are the gpg key generation parameters of an RSA key that
// signs and does not expire.
const testKeyParams = "Key-Type: RSA\nKey-Length: 2048\nExpire-Date: 0\n"
// genTestKey generates a key with no passphrase for
// "MFER Test Key <test@mfer.test>" from the gpg key generation parameters
// keyParams in gpgHome, which may already hold one.
func genTestKey(t *testing.T, gpgHome, keyParams string) {
t.Helper()
params := "%no-protection\n" + keyParams +
"Name-Real: MFER Test Key\nName-Email: test@mfer.test\n%commit\n"
paramsFile := filepath.Join(gpgHome, "key-params")
require.NoError(t, os.WriteFile(paramsFile, []byte(params), 0o600))
ctx, cancel := context.WithTimeout(context.Background(), gpgTimeout)
defer cancel()
//nolint:gosec // paramsFile is a test-controlled path inside t.TempDir()
cmd := exec.CommandContext(ctx, "gpg",
"--batch", "--gen-key", paramsFile)
cmd.Env = append(os.Environ(), "GNUPGHOME="+gpgHome)
output, err := cmd.CombinedOutput()
if err != nil {
t.Skipf("failed to generate test GPG key: %v: %s", err, output)
}
}
// signedTestManifest returns a manifest of one file signed with keyID.
func signedTestManifest(t *testing.T, keyID GPGKeyID) []byte {
t.Helper()
b := NewBuilder()
b.SetSigningOptions(&SigningOptions{KeyID: keyID})
content := []byte("signed file content")
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, 0,
bytes.NewReader(content), nil)
require.NoError(t, err)
var buf bytes.Buffer
require.NoError(t, b.Build(context.Background(), &buf))
return buf.Bytes()
}
// rewriteOuter returns manifest with its outer message changed by edit.
// A signature stays good as long as edit leaves the UUID and hash alone.
func rewriteOuter(t *testing.T, manifest []byte, edit func(*MFFileOuter)) []byte {
t.Helper()
outer := new(MFFileOuter)
require.NoError(t, proto.Unmarshal(manifest[len(MAGIC):], outer))
edit(outer)
data, err := proto.Marshal(outer)
require.NoError(t, err)
return append([]byte(MAGIC), data...)
}
func TestGPGSign(t *testing.T) { func TestGPGSign(t *testing.T) {
keyID, gpgHome := testGPGEnv(t) keyID, gpgHome := testGPGEnv(t)
t.Setenv("GNUPGHOME", gpgHome) t.Setenv("GNUPGHOME", gpgHome)
data := []byte("test data to sign") data := []byte("test data to sign")
sig, err := gpgSign(context.Background(), data, keyID) sig, signingKey, err := gpgSign(context.Background(), data, keyID)
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, string(keyID), signingKey)
assert.NotEmpty(t, sig) assert.NotEmpty(t, sig)
assert.Contains(t, string(sig), "-----BEGIN PGP SIGNATURE-----") assert.Contains(t, string(sig), "-----BEGIN PGP SIGNATURE-----")
assert.Contains(t, string(sig), "-----END PGP SIGNATURE-----") assert.Contains(t, string(sig), "-----END PGP SIGNATURE-----")
@@ -163,15 +210,18 @@ func TestGPGOptionLikeKeyIDIsNotAnOption(t *testing.T) {
assert.NotContains(t, string(fpr), "gpg (GnuPG)") assert.NotContains(t, string(fpr), "gpg (GnuPG)")
} }
// TestGPGSignInvalidKey signs with a key that has no secret key in the
// keyring. The error must hold gpg's messages and none of its status lines.
func TestGPGSignInvalidKey(t *testing.T) { func TestGPGSignInvalidKey(t *testing.T) {
// Set up test environment (we need GNUPGHOME set) // Set up test environment (we need GNUPGHOME set)
_, gpgHome := testGPGEnv(t) _, gpgHome := testGPGEnv(t)
t.Setenv("GNUPGHOME", gpgHome) t.Setenv("GNUPGHOME", gpgHome)
data := []byte("test data") data := []byte("test data")
_, err := gpgSign(context.Background(), data, _, _, err := gpgSign(context.Background(), data,
GPGKeyID("NONEXISTENT_KEY_ID_12345")) GPGKeyID("NONEXISTENT_KEY_ID_12345"))
assert.Error(t, err) require.Error(t, err)
assert.NotContains(t, err.Error(), gpgStatusPrefix)
} }
func TestBuilderWithSigning(t *testing.T) { func TestBuilderWithSigning(t *testing.T) {
@@ -187,7 +237,7 @@ func TestBuilderWithSigning(t *testing.T) {
// Add a test file // Add a test file
content := []byte("test file content") content := []byte("test file content")
reader := bytes.NewReader(content) reader := bytes.NewReader(content)
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, reader, nil) _, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, 0, reader, nil)
require.NoError(t, err) require.NoError(t, err)
// Build the manifest // Build the manifest
@@ -259,15 +309,16 @@ func TestGPGVerify(t *testing.T) {
t.Setenv("GNUPGHOME", gpgHome) t.Setenv("GNUPGHOME", gpgHome)
data := []byte("test data to sign and verify") data := []byte("test data to sign and verify")
sig, err := gpgSign(context.Background(), data, keyID) sig, _, err := gpgSign(context.Background(), data, keyID)
require.NoError(t, err) require.NoError(t, err)
pubKey, err := gpgExportPublicKey(context.Background(), keyID) pubKey, err := gpgExportPublicKey(context.Background(), keyID)
require.NoError(t, err) require.NoError(t, err)
// Verify the signature // Verify the signature; it names the key that made it
err = gpgVerify(context.Background(), data, sig, pubKey) signingKey, err := gpgVerify(context.Background(), data, sig, pubKey)
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, string(keyID), signingKey)
} }
func TestGPGVerifyInvalidSignature(t *testing.T) { func TestGPGVerifyInvalidSignature(t *testing.T) {
@@ -275,7 +326,7 @@ func TestGPGVerifyInvalidSignature(t *testing.T) {
t.Setenv("GNUPGHOME", gpgHome) t.Setenv("GNUPGHOME", gpgHome)
data := []byte("test data to sign") data := []byte("test data to sign")
sig, err := gpgSign(context.Background(), data, keyID) sig, _, err := gpgSign(context.Background(), data, keyID)
require.NoError(t, err) require.NoError(t, err)
pubKey, err := gpgExportPublicKey(context.Background(), keyID) pubKey, err := gpgExportPublicKey(context.Background(), keyID)
@@ -283,7 +334,7 @@ func TestGPGVerifyInvalidSignature(t *testing.T) {
// Try to verify with different data - should fail // Try to verify with different data - should fail
wrongData := []byte("different data") wrongData := []byte("different data")
err = gpgVerify(context.Background(), wrongData, sig, pubKey) _, err = gpgVerify(context.Background(), wrongData, sig, pubKey)
assert.Error(t, err) assert.Error(t, err)
} }
@@ -292,12 +343,12 @@ func TestGPGVerifyBadPublicKey(t *testing.T) {
t.Setenv("GNUPGHOME", gpgHome) t.Setenv("GNUPGHOME", gpgHome)
data := []byte("test data") data := []byte("test data")
sig, err := gpgSign(context.Background(), data, keyID) sig, _, err := gpgSign(context.Background(), data, keyID)
require.NoError(t, err) require.NoError(t, err)
// Try to verify with invalid public key - should fail // Try to verify with invalid public key - should fail
badPubKey := []byte("not a valid public key") badPubKey := []byte("not a valid public key")
err = gpgVerify(context.Background(), data, sig, badPubKey) _, err = gpgVerify(context.Background(), data, sig, badPubKey)
assert.Error(t, err) assert.Error(t, err)
} }
@@ -314,7 +365,7 @@ func TestManifestSignatureVerification(t *testing.T) {
// Add a test file // Add a test file
content := []byte("test file content for verification") content := []byte("test file content for verification")
reader := bytes.NewReader(content) reader := bytes.NewReader(content)
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, reader, nil) _, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, 0, reader, nil)
require.NoError(t, err) require.NoError(t, err)
// Build the manifest // Build the manifest
@@ -344,7 +395,7 @@ func TestManifestTamperedSignatureFails(t *testing.T) {
content := []byte("test file content") content := []byte("test file content")
reader := bytes.NewReader(content) reader := bytes.NewReader(content)
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, reader, nil) _, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, 0, reader, nil)
require.NoError(t, err) require.NoError(t, err)
var buf bytes.Buffer var buf bytes.Buffer
@@ -368,6 +419,144 @@ func TestManifestTamperedSignatureFails(t *testing.T) {
assert.Error(t, err) assert.Error(t, err)
} }
// TestManifestRefusesSecondEmbeddedKey loads a manifest whose embedded
// public key block holds another key before the key that signed it.
// Loading must refuse it, although the signature is good and the signer
// field names the key that made it.
func TestManifestRefusesSecondEmbeddedKey(t *testing.T) {
otherKey, otherHome := testGPGEnv(t)
t.Setenv("GNUPGHOME", otherHome)
otherPubKey, err := gpgExportPublicKey(context.Background(), otherKey)
require.NoError(t, err)
keyID, gpgHome := testGPGEnv(t)
t.Setenv("GNUPGHOME", gpgHome)
manifest := rewriteOuter(t, signedTestManifest(t, keyID),
func(outer *MFFileOuter) {
outer.SigningPubKey = slices.Concat(otherPubKey, outer.GetSigningPubKey())
})
_, err = NewManifestFromReader(bytes.NewReader(manifest))
require.ErrorIs(t, err, errSigningKeyCount)
}
// TestManifestRefusesSecondEmbeddedKeyWithoutUserID loads a manifest whose
// embedded public key block holds, before the key that signed it, another
// key with its user ID removed, which gpg skips on import. Loading must
// refuse it: the block holds two keys.
func TestManifestRefusesSecondEmbeddedKeyWithoutUserID(t *testing.T) {
otherKey, otherHome := testGPGEnv(t)
t.Setenv("GNUPGHOME", otherHome)
// Keeping only the user IDs that match "nobody" exports none.
otherPubKey, _, err := runGPG(context.Background(), nil,
gpgArgs([]string{
"--export", gpgOptArmor, "--export-filter", "keep-uid=uid = nobody",
}, string(otherKey))...)
require.NoError(t, err)
keyID, gpgHome := testGPGEnv(t)
t.Setenv("GNUPGHOME", gpgHome)
manifest := rewriteOuter(t, signedTestManifest(t, keyID),
func(outer *MFFileOuter) {
outer.SigningPubKey = slices.Concat(
otherPubKey.Bytes(), outer.GetSigningPubKey())
})
_, err = NewManifestFromReader(bytes.NewReader(manifest))
require.ErrorIs(t, err, errSigningKeyCount)
}
// TestManifestRefusesTwoSignatures loads a manifest whose signature field
// holds its good signature twice. Loading must refuse it.
func TestManifestRefusesTwoSignatures(t *testing.T) {
keyID, gpgHome := testGPGEnv(t)
t.Setenv("GNUPGHOME", gpgHome)
manifest := rewriteOuter(t, signedTestManifest(t, keyID),
func(outer *MFFileOuter) {
outer.Signature = slices.Concat(
outer.GetSignature(), outer.GetSignature())
})
_, err := NewManifestFromReader(bytes.NewReader(manifest))
require.ErrorIs(t, err, errNotOneGoodSignature)
}
// TestManifestSignedWithSubkey signs with a key whose primary key can only
// certify, so gpg signs with its signing subkey. The manifest must load,
// with the primary key's fingerprint as signer.
func TestManifestSignedWithSubkey(t *testing.T) {
gpgHome := t.TempDir()
t.Setenv("GNUPGHOME", gpgHome)
genTestKey(t, gpgHome, "Key-Type: RSA\nKey-Length: 2048\nKey-Usage: cert\n"+
"Subkey-Type: RSA\nSubkey-Length: 2048\nSubkey-Usage: sign\n"+
"Expire-Date: 0\n")
primary, err := gpgGetKeyFingerprint(context.Background(), "test@mfer.test")
require.NoError(t, err)
m, err := NewManifestFromReader(bytes.NewReader(
signedTestManifest(t, GPGKeyID("test@mfer.test"))))
require.NoError(t, err)
assert.Equal(t, primary, m.pbOuter.GetSigner())
}
// TestManifestRefusesSignerOtherThanSigningKey loads a manifest whose
// signer field names a key other than the one that made the signature.
func TestManifestRefusesSignerOtherThanSigningKey(t *testing.T) {
keyID, gpgHome := testGPGEnv(t)
t.Setenv("GNUPGHOME", gpgHome)
manifest := rewriteOuter(t, signedTestManifest(t, keyID),
func(outer *MFFileOuter) {
outer.Signer = []byte(strings.Repeat("A", len(keyID)))
})
_, err := NewManifestFromReader(bytes.NewReader(manifest))
require.ErrorIs(t, err, errSignerNotSigningKey)
}
// TestBuilderSigningKeyIDMatchingTwoKeys signs with a key ID that two keys
// in the keyring match. The manifest must embed and name only the key that
// signed it, or loading refuses it.
func TestBuilderSigningKeyIDMatchingTwoKeys(t *testing.T) {
_, gpgHome := testGPGEnv(t)
genTestKey(t, gpgHome, testKeyParams)
t.Setenv("GNUPGHOME", gpgHome)
manifest := signedTestManifest(t, GPGKeyID("test@mfer.test"))
_, err := NewManifestFromReader(bytes.NewReader(manifest))
require.NoError(t, err)
}
// TestBuilderSigningUserIDWithExpiredFirstKey signs with a user ID whose
// first key in the keyring has expired. gpg signs with the other key for
// that user ID, and the manifest must name and embed that key.
func TestBuilderSigningUserIDWithExpiredFirstKey(t *testing.T) {
gpgHome := t.TempDir()
t.Setenv("GNUPGHOME", gpgHome)
// Made in 2020 and valid for one day.
genTestKey(t, gpgHome, "Key-Type: RSA\nKey-Length: 2048\n"+
"Creation-Date: 20200101T000000\nExpire-Date: 1d\n")
expired, err := gpgGetKeyFingerprint(context.Background(), "test@mfer.test")
require.NoError(t, err)
genTestKey(t, gpgHome, testKeyParams)
m, err := NewManifestFromReader(bytes.NewReader(
signedTestManifest(t, GPGKeyID("test@mfer.test"))))
require.NoError(t, err)
assert.NotEqual(t, expired, m.pbOuter.GetSigner())
}
func TestBuilderWithoutSigning(t *testing.T) { func TestBuilderWithoutSigning(t *testing.T) {
t.Parallel() t.Parallel()
@@ -377,7 +566,7 @@ func TestBuilderWithoutSigning(t *testing.T) {
// Add a test file // Add a test file
content := []byte("test file content") content := []byte("test file content")
reader := bytes.NewReader(content) reader := bytes.NewReader(content)
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, reader, nil) _, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, 0, reader, nil)
require.NoError(t, err) require.NoError(t, err)
// Build the manifest // Build the manifest
@@ -414,16 +603,42 @@ func fakeGPGPath(t *testing.T, script string) string {
// TestGPGTimeoutKillsGPG puts a fake gpg that never finishes first on // TestGPGTimeoutKillsGPG puts a fake gpg that never finishes first on
// PATH and checks that a run past its deadline is killed and reported as // PATH and checks that a run past its deadline is killed and reported as
// a timeout of the named operation, instead of hanging. // a timeout of the named operation, instead of hanging. The fake gpg writes
// nothing to stderr, so the message ends with the timeout.
func TestGPGTimeoutKillsGPG(t *testing.T) { func TestGPGTimeoutKillsGPG(t *testing.T) {
t.Setenv("PATH", fakeGPGPath(t, "#!/bin/sh\nexec sleep 10\n")) t.Setenv("PATH", fakeGPGPath(t, "#!/bin/sh\nexec sleep 10\n"))
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond) ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel() defer cancel()
_, err := gpgSign(ctx, []byte("data"), GPGKeyID("any")) _, _, err := gpgSign(ctx, []byte("data"), GPGKeyID("any"))
require.ErrorIs(t, err, context.DeadlineExceeded) require.ErrorIs(t, err, context.DeadlineExceeded)
assert.Contains(t, err.Error(), "gpg sign failed: gpg timed out") assert.EqualError(t, err, "gpg sign: timed out: context deadline exceeded")
}
// TestGPGSignKeyNotReportedKeepsStderr puts a fake gpg first on PATH that
// exits cleanly without reporting the key that signed, and checks that what
// it wrote to stderr is in the message.
func TestGPGSignKeyNotReportedKeepsStderr(t *testing.T) {
t.Setenv("PATH", fakeGPGPath(t,
"#!/bin/sh\necho 'gpg: note from the fake gpg' >&2\n"))
_, _, err := gpgSign(context.Background(), []byte("data"), GPGKeyID("any"))
require.ErrorIs(t, err, errSigningKeyNotReported)
assert.EqualError(t, err,
"gpg did not report the key that made the signature: "+
"gpg: note from the fake gpg")
}
// TestGPGFailureKeepsStderr puts a fake gpg first on PATH that writes to
// stderr and exits non-zero, and checks that what it wrote ends the message.
func TestGPGFailureKeepsStderr(t *testing.T) {
t.Setenv("PATH", fakeGPGPath(t,
"#!/bin/sh\necho 'gpg: signing failed: No secret key' >&2\nexit 2\n"))
_, _, err := gpgSign(context.Background(), []byte("data"), GPGKeyID("any"))
assert.EqualError(t, err,
"gpg sign: exit status 2: gpg: signing failed: No secret key")
} }
// TestGPGCancelWhenChildHoldsOutput uses a fake gpg that runs sleep as a // TestGPGCancelWhenChildHoldsOutput uses a fake gpg that runs sleep as a
@@ -446,7 +661,7 @@ func TestGPGCancelWhenChildHoldsOutput(t *testing.T) {
signErr := make(chan error, 1) signErr := make(chan error, 1)
go func() { go func() {
_, err := gpgSign(ctx, []byte("data"), GPGKeyID("any")) _, _, err := gpgSign(ctx, []byte("data"), GPGKeyID("any"))
signErr <- err signErr <- err
}() }()
+2 -2
View File
@@ -10,7 +10,7 @@ import (
) )
var ( var (
errOuterNotSet = errors.New("pbOuter not set") errOuterNotSet = errors.New("outer message not set")
errUUIDNotSet = errors.New("UUID not set") errUUIDNotSet = errors.New("UUID not set")
errSHA256NotSet = errors.New("SHA256 hash not set") errSHA256NotSet = errors.New("SHA256 hash not set")
) )
@@ -65,7 +65,7 @@ func (m *manifest) signatureString() (string, error) {
mh, err := multihash.Encode(m.pbOuter.GetSha256(), multihash.SHA2_256) mh, err := multihash.Encode(m.pbOuter.GetSha256(), multihash.SHA2_256)
if err != nil { if err != nil {
return "", fmt.Errorf("failed to encode multihash: %w", err) return "", fmt.Errorf("encode multihash: %w", err)
} }
uuidStr := hex.EncodeToString(m.pbOuter.GetUuid()) uuidStr := hex.EncodeToString(m.pbOuter.GetUuid())
+19 -9
View File
@@ -1,6 +1,6 @@
// Code generated by protoc-gen-go. DO NOT EDIT. // Code generated by protoc-gen-go. DO NOT EDIT.
// versions: // versions:
// protoc-gen-go v1.36.11 // protoc-gen-go v1.36.12
// protoc v6.33.4 // protoc v6.33.4
// source: mf.proto // source: mf.proto
@@ -223,11 +223,11 @@ type MFFileOuter struct {
// uuid must match the uuid in the inner message // uuid must match the uuid in the inner message
Uuid []byte `protobuf:"bytes,105,opt,name=uuid,proto3" json:"uuid,omitempty"` Uuid []byte `protobuf:"bytes,105,opt,name=uuid,proto3" json:"uuid,omitempty"`
InnerMessage []byte `protobuf:"bytes,199,opt,name=innerMessage,proto3" json:"innerMessage,omitempty"` InnerMessage []byte `protobuf:"bytes,199,opt,name=innerMessage,proto3" json:"innerMessage,omitempty"`
// detached signature, ascii or binary //detached signature, ascii or binary
Signature []byte `protobuf:"bytes,201,opt,name=signature,proto3,oneof" json:"signature,omitempty"` Signature []byte `protobuf:"bytes,201,opt,name=signature,proto3,oneof" json:"signature,omitempty"`
// full GPG key id //full GPG key id
Signer []byte `protobuf:"bytes,202,opt,name=signer,proto3,oneof" json:"signer,omitempty"` Signer []byte `protobuf:"bytes,202,opt,name=signer,proto3,oneof" json:"signer,omitempty"`
// full GPG signing public key, ascii or binary //full GPG signing public key, ascii or binary
SigningPubKey []byte `protobuf:"bytes,203,opt,name=signingPubKey,proto3,oneof" json:"signingPubKey,omitempty"` SigningPubKey []byte `protobuf:"bytes,203,opt,name=signingPubKey,proto3,oneof" json:"signingPubKey,omitempty"`
unknownFields protoimpl.UnknownFields unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache sizeCache protoimpl.SizeCache
@@ -337,9 +337,11 @@ type MFFilePath struct {
// gotta have at least one: // gotta have at least one:
Hashes []*MFFileChecksum `protobuf:"bytes,3,rep,name=hashes,proto3" json:"hashes,omitempty"` Hashes []*MFFileChecksum `protobuf:"bytes,3,rep,name=hashes,proto3" json:"hashes,omitempty"`
// optional per-file metadata // optional per-file metadata
MimeType *string `protobuf:"bytes,301,opt,name=mimeType,proto3,oneof" json:"mimeType,omitempty"` MimeType *string `protobuf:"bytes,301,opt,name=mimeType,proto3,oneof" json:"mimeType,omitempty"`
Mtime *Timestamp `protobuf:"bytes,302,opt,name=mtime,proto3,oneof" json:"mtime,omitempty"` Mtime *Timestamp `protobuf:"bytes,302,opt,name=mtime,proto3,oneof" json:"mtime,omitempty"`
Ctime *Timestamp `protobuf:"bytes,303,opt,name=ctime,proto3,oneof" json:"ctime,omitempty"` Ctime *Timestamp `protobuf:"bytes,303,opt,name=ctime,proto3,oneof" json:"ctime,omitempty"`
// permission bits, at most 0777; 0 when not recorded
Mode uint32 `protobuf:"varint,304,opt,name=mode,proto3" json:"mode,omitempty"`
unknownFields protoimpl.UnknownFields unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache sizeCache protoimpl.SizeCache
} }
@@ -416,6 +418,13 @@ func (x *MFFilePath) GetCtime() *Timestamp {
return nil return nil
} }
func (x *MFFilePath) GetMode() uint32 {
if x != nil {
return x.Mode
}
return 0
}
type MFFileChecksum struct { type MFFileChecksum struct {
state protoimpl.MessageState `protogen:"open.v1"` state protoimpl.MessageState `protogen:"open.v1"`
// 1.0 golang implementation must write a multihash here // 1.0 golang implementation must write a multihash here
@@ -561,7 +570,7 @@ const file_mf_proto_rawDesc = "" +
"\n" + "\n" +
"_signatureB\t\n" + "_signatureB\t\n" +
"\a_signerB\x10\n" + "\a_signerB\x10\n" +
"\x0e_signingPubKey\"\xf0\x01\n" + "\x0e_signingPubKey\"\x85\x02\n" +
"\n" + "\n" +
"MFFilePath\x12\x12\n" + "MFFilePath\x12\x12\n" +
"\x04path\x18\x01 \x01(\tR\x04path\x12\x12\n" + "\x04path\x18\x01 \x01(\tR\x04path\x12\x12\n" +
@@ -571,7 +580,8 @@ const file_mf_proto_rawDesc = "" +
"\x05mtime\x18\xae\x02 \x01(\v2\n" + "\x05mtime\x18\xae\x02 \x01(\v2\n" +
".TimestampH\x01R\x05mtime\x88\x01\x01\x12&\n" + ".TimestampH\x01R\x05mtime\x88\x01\x01\x12&\n" +
"\x05ctime\x18\xaf\x02 \x01(\v2\n" + "\x05ctime\x18\xaf\x02 \x01(\v2\n" +
".TimestampH\x02R\x05ctime\x88\x01\x01B\v\n" + ".TimestampH\x02R\x05ctime\x88\x01\x01\x12\x13\n" +
"\x04mode\x18\xb0\x02 \x01(\rR\x04modeB\v\n" +
"\t_mimeTypeB\b\n" + "\t_mimeTypeB\b\n" +
"\x06_mtimeB\b\n" + "\x06_mtimeB\b\n" +
"\x06_ctime\".\n" + "\x06_ctime\".\n" +
+2
View File
@@ -59,6 +59,8 @@ message MFFilePath {
optional string mimeType = 301; optional string mimeType = 301;
optional Timestamp mtime = 302; optional Timestamp mtime = 302;
optional Timestamp ctime = 303; optional Timestamp ctime = 303;
// permission bits, at most 0777; 0 when not recorded
uint32 mode = 304;
} }
message MFFileChecksum { message MFFileChecksum {
+1 -1
View File
@@ -1 +1 @@
fa6fceaba5553c8667c631994535fe5c307c8adb19f3ad289babf79a0f438650 mf.proto 3d4dcb0b2f4640dd3ae6bb48b833e9f26ae9771e2d3e88bd646e7f1724dda654 mf.proto
+1
View File
@@ -41,6 +41,7 @@ func TestFileEntryFieldsMatchSpec(t *testing.T) {
"mimeType": 301, "mimeType": 301,
"mtime": 302, "mtime": 302,
"ctime": 303, "ctime": 303,
"mode": 304,
} }
got := map[string]protoreflect.FieldNumber{} got := map[string]protoreflect.FieldNumber{}
+28 -1
View File
@@ -2,6 +2,7 @@ package mfer
import ( import (
"context" "context"
"fmt"
"io" "io"
"io/fs" "io/fs"
"os" "os"
@@ -52,6 +53,9 @@ type ScannerOptions struct {
// IncludeTimestamps includes a createdAt timestamp in the manifest // IncludeTimestamps includes a createdAt timestamp in the manifest
// (default: omit for determinism). // (default: omit for determinism).
IncludeTimestamps bool IncludeTimestamps bool
// IncludePermissions records each file's permission bits, 0777 at
// most, in the manifest (default: record 0000).
IncludePermissions bool
// Fs is the filesystem to use, defaults to OsFs if nil. // Fs is the filesystem to use, defaults to OsFs if nil.
Fs afero.Fs Fs afero.Fs
// SigningOptions holds GPG signing options (nil = no signing). // SigningOptions holds GPG signing options (nil = no signing).
@@ -69,13 +73,15 @@ type FileEntry struct {
Size FileSize // File size in bytes Size FileSize // File size in bytes
Mtime ModTime // Last modification time Mtime ModTime // Last modification time
Ctime time.Time // Creation time (platform-dependent) Ctime time.Time // Creation time (platform-dependent)
Mode fs.FileMode // Permission bits (Perm() of the file's mode)
} }
// Scanner accumulates files and generates manifests from them. // Scanner accumulates files and generates manifests from them.
type Scanner struct { type Scanner struct {
mu sync.RWMutex mu sync.RWMutex
files []*FileEntry files []*FileEntry
totalBytes FileSize // cached sum of all file sizes paths map[RelFilePath]AbsFilePath // the file at each path in files
totalBytes FileSize // cached sum of all file sizes
options *ScannerOptions options *ScannerOptions
fs afero.Fs fs afero.Fs
excluded []fs.FileInfo // the files named in ExcludePaths that exist excluded []fs.FileInfo // the files named in ExcludePaths that exist
@@ -99,6 +105,7 @@ func NewScannerWithOptions(opts *ScannerOptions) *Scanner {
s := &Scanner{ s := &Scanner{
files: make([]*FileEntry, 0), files: make([]*FileEntry, 0),
paths: make(map[RelFilePath]AbsFilePath),
options: opts, options: opts,
fs: fs, fs: fs,
} }
@@ -353,11 +360,18 @@ func (s *Scanner) scanFile(
}(scannedBytes, scannedFiles) }(scannedBytes, scannedFiles)
} }
// A mode of 0 records 0000, which means none was recorded.
var mode fs.FileMode
if s.options.IncludePermissions {
mode = entry.Mode
}
// Add to manifest with progress channel // Add to manifest with progress channel
bytesRead, err := builder.AddFile( bytesRead, err := builder.AddFile(
entry.Path, entry.Path,
entry.Size, entry.Size,
entry.Mtime, entry.Mtime,
mode,
f, f,
fileProgress, fileProgress,
) )
@@ -461,11 +475,24 @@ func (s *Scanner) enumerateFileWithInfo(
AbsPath: AbsFilePath(absPath), AbsPath: AbsFilePath(absPath),
Size: FileSize(info.Size()), Size: FileSize(info.Size()),
Mtime: ModTime(info.ModTime()), Mtime: ModTime(info.ModTime()),
Mode: info.Mode().Perm(),
// Note: Ctime not available from fs.FileInfo on all platforms // Note: Ctime not available from fs.FileInfo on all platforms
// Will need platform-specific code to extract it // Will need platform-specific code to extract it
} }
s.mu.Lock() s.mu.Lock()
// Each path is relative to the input path it was found under, so files
// under two input paths can share one.
first, ok := s.paths[entry.Path]
if ok {
s.mu.Unlock()
return fmt.Errorf("%w %q: %s and %s",
errDuplicatePath, entry.Path, first, entry.AbsPath)
}
s.paths[entry.Path] = entry.AbsPath
s.files = append(s.files, entry) s.files = append(s.files, entry)
s.totalBytes += entry.Size s.totalBytes += entry.Size
filesFound := FileCount(len(s.files)) filesFound := FileCount(len(s.files))
+41
View File
@@ -4,6 +4,7 @@ package mfer
import ( import (
"bytes" "bytes"
"context" "context"
"os"
"testing" "testing"
"time" "time"
@@ -350,6 +351,46 @@ func TestScannerFileEntryFields(t *testing.T) {
assert.WithinDuration(t, now, time.Time(entry.Mtime), 2*time.Second) assert.WithinDuration(t, now, time.Time(entry.Mtime), 2*time.Second)
} }
// A manifest records every mode as 0000 unless the creator asks for
// permissions; then it records each file's permission bits and never the
// setuid, setgid or sticky bits.
func TestScannerRecordsModeOnlyWhenAsked(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
require.NoError(t, afero.WriteFile(fs, "/"+testFile1, []byte("a"), 0o640))
require.NoError(t, afero.WriteFile(fs, "/run.sh", []byte("b"), 0o755))
require.NoError(t, afero.WriteFile(fs, "/su", []byte("c"), 0o755))
require.NoError(t, fs.Chmod("/su", 0o755|os.ModeSetuid|os.ModeSetgid|os.ModeSticky))
for _, tc := range []struct {
includePermissions bool
want map[string]uint32
}{
{false, map[string]uint32{testFile1: 0, "run.sh": 0, "su": 0}},
{true, map[string]uint32{testFile1: 0o640, "run.sh": 0o755, "su": 0o755}},
} {
s := NewScannerWithOptions(&ScannerOptions{
Fs: fs,
IncludePermissions: tc.includePermissions,
})
require.NoError(t, s.EnumerateFS(fs, "/", nil))
var buf bytes.Buffer
require.NoError(t, s.ToManifest(context.Background(), &buf, nil))
m, err := NewManifestFromReader(&buf)
require.NoError(t, err)
got := map[string]uint32{}
for _, f := range m.Files() {
got[f.GetPath()] = f.GetMode()
}
assert.Equal(t, tc.want, got, "IncludePermissions: %v", tc.includePermissions)
}
}
func TestScannerLargeFileEnumeration(t *testing.T) { func TestScannerLargeFileEnumeration(t *testing.T) {
t.Parallel() t.Parallel()
+22 -20
View File
@@ -8,8 +8,8 @@ import (
"fmt" "fmt"
"math" "math"
"time" "time"
"uuid"
"github.com/google/uuid"
"github.com/klauspost/compress/zstd" "github.com/klauspost/compress/zstd"
"google.golang.org/protobuf/proto" "google.golang.org/protobuf/proto"
) )
@@ -20,11 +20,9 @@ const MAGIC string = "ZNAVSRFG"
var ( var (
// errInnerNotSet is returned by generate when the inner manifest is // errInnerNotSet is returned by generate when the inner manifest is
// missing. // missing.
errInnerNotSet = errors.New("internal error: pbInner not set") errInnerNotSet = errors.New("inner message not set")
// errInternal is returned by generateOuter for the same condition. // errInternal is returned by generateOuter for the same condition.
// The two messages differ, and both are load-bearing for callers that errInternal = errors.New("inner message not set")
// match on text, so they are kept distinct.
errInternal = errors.New("internal error")
) )
// nanosecondsInt32 converts t's nanosecond component to int32. // nanosecondsInt32 converts t's nanosecond component to int32.
@@ -65,14 +63,14 @@ func (m *manifest) generate(ctx context.Context) error {
dat, err := proto.MarshalOptions{Deterministic: true}.Marshal(m.pbOuter) dat, err := proto.MarshalOptions{Deterministic: true}.Marshal(m.pbOuter)
if err != nil { if err != nil {
return fmt.Errorf("serialize: marshal outer: %w", err) return fmt.Errorf("marshal outer message: %w", err)
} }
m.output = bytes.NewBufferString(MAGIC) m.output = bytes.NewBufferString(MAGIC)
_, err = m.output.Write(dat) _, err = m.output.Write(dat)
if err != nil { if err != nil {
return fmt.Errorf("serialize: write output: %w", err) return fmt.Errorf("write outer message: %w", err)
} }
return nil return nil
@@ -88,14 +86,14 @@ func (m *manifest) generateOuter(ctx context.Context) error {
if len(m.fixedUUID) == uuidLength { if len(m.fixedUUID) == uuidLength {
copy(manifestUUID[:], m.fixedUUID) copy(manifestUUID[:], m.fixedUUID)
} else { } else {
manifestUUID = uuid.New() manifestUUID = uuid.NewV4()
} }
m.pbInner.Uuid = manifestUUID[:] m.pbInner.Uuid = manifestUUID[:]
innerData, err := proto.MarshalOptions{Deterministic: true}.Marshal(m.pbInner) innerData, err := proto.MarshalOptions{Deterministic: true}.Marshal(m.pbInner)
if err != nil { if err != nil {
return fmt.Errorf("serialize: marshal inner: %w", err) return fmt.Errorf("marshal inner message: %w", err)
} }
// Compress the inner data // Compress the inner data
@@ -103,12 +101,12 @@ func (m *manifest) generateOuter(ctx context.Context) error {
zw, err := zstd.NewWriter(idc, zstd.WithEncoderLevel(zstd.SpeedBestCompression)) zw, err := zstd.NewWriter(idc, zstd.WithEncoderLevel(zstd.SpeedBestCompression))
if err != nil { if err != nil {
return fmt.Errorf("serialize: create compressor: %w", err) return fmt.Errorf("create compressor: %w", err)
} }
_, err = zw.Write(innerData) _, err = zw.Write(innerData)
if err != nil { if err != nil {
return fmt.Errorf("serialize: compress: %w", err) return fmt.Errorf("compress inner message: %w", err)
} }
_ = zw.Close() _ = zw.Close()
@@ -120,7 +118,7 @@ func (m *manifest) generateOuter(ctx context.Context) error {
_, err = h.Write(compressedData) _, err = h.Write(compressedData)
if err != nil { if err != nil {
return fmt.Errorf("serialize: hash write: %w", err) return fmt.Errorf("hash inner message: %w", err)
} }
sha256Hash := h.Sum(nil) sha256Hash := h.Sum(nil)
@@ -143,30 +141,34 @@ func (m *manifest) generateOuter(ctx context.Context) error {
} }
// signOuter signs the outer message with the configured GPG key and // signOuter signs the outer message with the configured GPG key and
// embeds the signature, signer fingerprint, and public key. // embeds the signature, signer fingerprint, and public key. The signer
// and public key are those of the key gpg reports it signed with, so that
// a key ID matching more than one key cannot name or embed another key.
func (m *manifest) signOuter(ctx context.Context) error { func (m *manifest) signOuter(ctx context.Context) error {
sigString, err := m.signatureString() sigString, err := m.signatureString()
if err != nil { if err != nil {
return fmt.Errorf("failed to generate signature string: %w", err) return fmt.Errorf("build signature string: %w", err)
} }
sig, err := gpgSign(ctx, []byte(sigString), m.signingOptions.KeyID) sig, signingKey, err := gpgSign(ctx, []byte(sigString), m.signingOptions.KeyID)
if err != nil { if err != nil {
return fmt.Errorf("failed to sign manifest: %w", err) return err
} }
m.pbOuter.Signature = sig m.pbOuter.Signature = sig
fingerprint, err := gpgGetKeyFingerprint(ctx, m.signingOptions.KeyID) // Listing the signing key, a subkey's included, puts its primary key's
// fingerprint first.
fingerprint, err := gpgGetKeyFingerprint(ctx, GPGKeyID(signingKey))
if err != nil { if err != nil {
return fmt.Errorf("failed to get key fingerprint: %w", err) return err
} }
m.pbOuter.Signer = fingerprint m.pbOuter.Signer = fingerprint
pubKey, err := gpgExportPublicKey(ctx, m.signingOptions.KeyID) pubKey, err := gpgExportPublicKey(ctx, GPGKeyID(fingerprint))
if err != nil { if err != nil {
return fmt.Errorf("failed to export public key: %w", err) return err
} }
m.pbOuter.SigningPubKey = pubKey m.pbOuter.SigningPubKey = pubKey
+6 -5
View File
@@ -22,14 +22,15 @@ YARN_VERSION="1.22.22"
# protoc v33.4, 2026-10-04, for script/generate. The sha256 of each # protoc v33.4, 2026-10-04, for script/generate. The sha256 of each
# platform's release archive is in ensure_protoc. # platform's release archive is in ensure_protoc.
PROTOC_VERSION="33.4" PROTOC_VERSION="33.4"
# gofumpt v0.12.0 for script/gofumpt and protoc-gen-go v1.36.11 for # gofumpt v0.12.0 for script/gofumpt, 2026-10-04, and protoc-gen-go
# script/generate, 2026-10-04: each is installed into bin/ with # v1.36.12 for script/generate, 2026-10-06: each is installed into bin/
# `go install`, pinned to the commit its release tag names. Those two # with `go install`, pinned to the commit its release tag names. Those two
# scripts refuse any other version, so a new pin is changed there too. # scripts refuse any other version, so a new pin is changed there too.
# protoc-gen-go stays at the google.golang.org/protobuf version in go.mod.
GOFUMPT_VERSION="v0.12.0" GOFUMPT_VERSION="v0.12.0"
GOFUMPT_COMMIT="3e07e7e70ac93761d8e79ca0083a19e3d59f753d" GOFUMPT_COMMIT="3e07e7e70ac93761d8e79ca0083a19e3d59f753d"
PROTOC_GEN_GO_VERSION="v1.36.11" PROTOC_GEN_GO_VERSION="v1.36.12"
PROTOC_GEN_GO_COMMIT="96a179180f0ad6bba9b1e7b6e38d0affb0168e9a" PROTOC_GEN_GO_COMMIT="cdd4c5f7406e82462949c7a65defa9f3029c162d"
PKGMGR="" PKGMGR=""
SUDO="" SUDO=""
+1 -1
View File
@@ -15,7 +15,7 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# in the mf.pb.go header. # in the mf.pb.go header.
PROTOC_VERSION="33.4" PROTOC_VERSION="33.4"
PROTOC="$ROOT/bin/protoc/bin/protoc" PROTOC="$ROOT/bin/protoc/bin/protoc"
PROTOC_GEN_GO_VERSION="v1.36.11" PROTOC_GEN_GO_VERSION="v1.36.12"
PROTOC_GEN_GO="$ROOT/bin/protoc-gen-go" PROTOC_GEN_GO="$ROOT/bin/protoc-gen-go"
# sha256 <file>: print "<hash> <file>", with sha256sum, or with shasum # sha256 <file>: print "<hash> <file>", with sha256sum, or with shasum
+22
View File
@@ -0,0 +1,22 @@
#!/bin/sh
# script/vulncheck: report known vulnerabilities in the code mfer calls,
# with govulncheck, which reads the Go vulnerability database online.
# It runs as the vulncheck stage of the Dockerfile, on the same Go as the
# test phase, and this builds that stage alone, on the same terms as
# script/lint and script/test.
#
# script/check does not run it: the gate's result depends on this tree
# alone, and this one changes whenever a new advisory is published.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
docker build --no-cache \
--target vulncheck \
-t "$("$SCRIPT_DIR/projectname")-vulncheck" .
}
main "$@"