Author SHA1 Message Date
sneak 114ba0a29f Disable the deprecated gomodguard linter (closes #116)
check / check (push) Waiting to run
golangci-lint v2.12 deprecates gomodguard in favour of gomodguard_v2.
With `default: all` both ran, and every lint run printed the
deprecation warning. Disable the old name, the same way wsl is
disabled for wsl_v5; gomodguard_v2 stays enabled. The old linter had
no settings here, so there is nothing to carry over.

Model: opus-5-5
2026-10-04 08:06:44 +00:00
clawbot 64eb5cbd40 Scanner status tests no longer depend on a 100 ms timer (closes #124)
check / check (push) Failing after 3s
The two status-send tests now call the send directly on a channel nobody
receives from, so a blocking send hangs the test into its timeout instead
of racing a 100 ms timer that a loaded host can miss.

The gpg test for a child holding gpg's output had the same problem: its
100 ms deadline could fire before the fake gpg wrote the PID of sleep,
and the cleanup then failed. The fake gpg now writes that PID to a named
pipe, and the test cancels only after reading it. It then waits up to
10 s for the call, so a call that waits for sleep fails the test and the
cleanup still kills sleep.

Model: opus-5-5
2026-10-04 09:48:51 +02:00
clawbot 45eac1f6f8 Run all linting in Docker via the Dockerfile lint stage (closes #90)
check / check (push) Failing after 3s
script/lint now builds only the lint stage of the main Dockerfile
(docker build --no-cache --target lint), whose build runs the linter, so
a successful build is a clean lint. It is uncached because a cached
build runs no linter, and a trap removes the image it tagged; the tag
carries the process ID so concurrent runs do not collide. The lint stage
calls golangci-lint directly, since make lint now needs Docker. Nothing
installs or runs golangci-lint on the host any more: bootstrap and the
Makefile drop the install, and script/fmt drops golangci-lint run --fix.

Model: opus-5-5
2026-10-04 09:31:54 +02:00
clawbot b91e92b070 Build a static binary so the scratch image runs (closes #126)
check / check (push) Failing after 1s
The final stage is scratch, which has no C library, but the builder
compiled mfer with cgo on (the golang image's default). The binary
imports net (mfer's own HTTP code does, and so does google/uuid), so
with cgo on it came out dynamically linked and the image could not
start. The image's go build now sets CGO_ENABLED=0; nothing in mfer
needs cgo. A new builder step runs ldd on the binary and fails the
build unless it reports a static executable.

Model: opus-5-5
2026-10-04 08:48:52 +02:00
clawbot a2732cf8da Add the required README sections (closes #75)
check / check (push) Successful in 1m28s
The Description first line now names the project, purpose, category,
WTFPL license, and author. A new Getting Started section gives a
copy-pasteable build-from-source block and gen/check/fetch usage. Problem
Statement and Proposed Solution move under a new Rationale heading, the
prose kept. A new Design section documents the package layout: the mfer/
library with its committed protobuf code, the internal/cli commands,
internal/log and internal/bork, and the cmd/mfer entrypoint. Authors is
renamed Author with the canonical link.

Getting Started uses go build because the make target that builds the
binary runs protoc, which script/bootstrap does not install.

Model: opus-4-8 (implementation); opus-5-5 (rebase, rework)
2026-10-04 06:32:07 +02:00
clawbot a789eb3083 Give -v to verbose, move --version to -V (closes #64)
check / check (push) Successful in 1m45s
urfave/cli's built-in version flag claimed -v, so "mfer -v --version"
failed to parse, and -v meant version at the root but verbose on
generate, check, freshen and fetch. Verbose is the more common meaning,
so the root now takes -v and -q too, and the version flag takes -V. A
-v or -q before generate, check, freshen, fetch or export applies to
that subcommand; list keeps its fixed quiet logging.

User-visible changes: -v no longer prints the version; use -V or
--version. "mfer version" prints "mfer version 0.1.0 (...)", the same
line as --version, instead of the bare "0.1.0 (...)".

Tests: runCLI now points the logger at io.Discard after each run, so
other tests' log lines cannot land in a finished run's output.

Model: opus-4-8 (implementation); opus-5-5 (rework)
2026-10-04 06:02:24 +02:00
clawbot d00982b329 Fuzz NewManifestFromReader and cap the zstd decoder (closes #65)
check / check (push) Successful in 1m5s
FuzzNewManifestFromReader fails when the parser returns both or neither
of a manifest and an error, or allocates more than a fixed multiple of
its input and the decompressed data it may read, plus room for the
decoder's window buffers. make test runs the seed corpus; make fuzz
fuzzes for one minute.

Parser bug: MaxDecompressedSize did not bound decompression; the zstd
decoder decoded a payload under 128 KiB in full before the LimitReader
read any of it. It now decodes only what the LimitReader reads and
refuses windows over the 8 MiB mfer writes with. Seeds: a frame claiming
8 GiB, two frames together over the limit, empty frames with growing
windows.

Model: opus-5-5
2026-10-04 05:31:51 +02:00
clawbot 51f69c960d Enforce real timeouts on gpg subprocess calls (closes #62)
check / check (push) Successful in 2m2s
Every gpg run now has a one-minute deadline (gpgTimeout) on top of its
caller's context and is killed when either ends. A timeout is reported
as "gpg timed out" under the failing operation instead of "signal:
killed". Only gpg itself is killed; WaitDelay (one second) stops the run
from waiting on a process gpg left behind that still holds its output,
such as a wrapper script that does not exec the real gpg.
Builder.Build and Checker.ExtractEmbeddedSigningKeyFP take a context, so
ToManifest's context now reaches signing and the contextcheck
suppression calling signing non-cancellable is gone. Manifest loading
takes no context, so its signature check is bounded by the timeout
alone.

Model: opus-5-5
2026-10-04 04:31:53 +02:00
44 changed files with 671 additions and 158 deletions
+1
View File
@@ -16,6 +16,7 @@ linters:
- depguard # Dependency allow/block lists
- godot # Requires comments to end with periods
- wsl # Deprecated, replaced by wsl_v5
- gomodguard # Deprecated, replaced by gomodguard_v2
- wrapcheck # Too verbose for internal packages
- varnamelen # Short names like db, id are idiomatic Go
settings:
+7 -2
View File
@@ -14,7 +14,9 @@ RUN touch mfer/mf.pb.go
# Go half of fmt-check only: this image has no node, so no prettier. The
# markdown half runs in the mdfmt stage below.
RUN make fmt-check-go
RUN make lint
# The linter directly, not `make lint`: script/lint builds this stage, and
# there is no docker inside this build.
RUN golangci-lint run --config .golangci.yml ./...
# Markdown/JSON format stage — prettier needs node, which the Go images
# do not have. node:22.17.0-bookworm-slim (2026-08-09); ships node
@@ -67,7 +69,10 @@ RUN version="${VERSION:-$(git describe --tags --always)}"; \
exit 1; \
fi; \
cd cmd/mfer && \
go build -tags urfave_cli_no_docs -ldflags "-X main.Gitrev=$version" -o /mfer .
CGO_ENABLED=0 go build -tags urfave_cli_no_docs -ldflags "-X main.Gitrev=$version" -o /mfer .
# Fail unless /mfer is statically linked: scratch has no C library to run it.
RUN ldd /mfer 2>&1 | grep -q 'not a dynamic executable'
FROM scratch
COPY --from=builder /mfer /mfer
+3 -1
View File
@@ -47,7 +47,9 @@ allows verifying data integrity before decompression.
The `innerMessage` field is compressed with
[Zstandard (zstd)](https://facebook.github.io/zstd/). Implementations must
enforce a decompression size limit to prevent decompression bombs. The reference
implementation limits decompressed size to 256 MB.
implementation limits decompressed size to 256 MB. It writes zstd frames with a
window of at most 8 MiB, the largest window the zstd format recommends decoders
support, and refuses frames that ask for a larger one.
## Inner Message (`MFFile`)
+4 -4
View File
@@ -13,7 +13,7 @@ GOLDFLAGS += -X main.Version=$(VERSION)
GOLDFLAGS += -X main.Gitrev=$(GITREV_BUILD)
GOFLAGS := -ldflags "$(GOLDFLAGS)"
.PHONY: bootstrap setup docker default run ci test check lint fmt fmt-check fmt-check-go fmt-check-md hooks fixme
.PHONY: bootstrap setup docker default run ci test fuzz check lint fmt fmt-check fmt-check-go fmt-check-md hooks fixme
default: fmt test
@@ -32,6 +32,9 @@ ci: test
test:
@script/test
fuzz:
@script/fuzz
$(PROTOC_GEN_GO):
test -e $(PROTOC_GEN_GO) || go install -v google.golang.org/protobuf/cmd/protoc-gen-go@v1.28.1
@@ -55,9 +58,6 @@ fmt-check-md:
hooks:
@script/install-precommit
devprereqs:
which golangci-lint || go install -v github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.12.2
mfer/mf.pb.go: mfer/mf.proto
cd mfer && go generate .
+80 -22
View File
@@ -1,12 +1,12 @@
# mfer
[mfer](https://git.eeqj.de/sneak/mfer) is a reference implementation library and
thin wrapper command-line utility written in [Go](https://golang.org) and first
published in 2022 under the [WTFPL](https://wtfpl.net) (public domain) license.
It specifies and generates `.mf` manifest files over a directory tree of files
to encapsulate metadata about them (such as cryptographic checksums or
signatures over same) to aid in archiving, downloading, and streaming, or
mirroring. The manifest files' data is serialized with Google's
[mfer](https://git.eeqj.de/sneak/mfer) is a [WTFPL](https://wtfpl.net)-licensed
(public domain) [Go](https://golang.org) library and command-line tool by
[@sneak](https://sneak.berlin) that specifies and generates `.mf` manifest files
over a directory tree to encapsulate metadata about the files — such as
cryptographic checksums and signatures over same — to aid in archiving,
downloading, streaming, and mirroring. It was first published in 2022. The
manifest files' data is serialized with Google's
[protobuf serialization format](https://developers.google.com/protocol-buffers).
The structure of these files can be found
[in the format specification](https://git.eeqj.de/sneak/mfer/src/branch/main/mfer/mf.proto)
@@ -21,6 +21,36 @@ This project was started by [@sneak](https://sneak.berlin) to scratch an itch in
as a de-facto standard and be incorporated into other software. A compatible
javascript library is planned.
# Getting Started
`mfer` builds from source with a Go 1.23+ toolchain. The generated protobuf code
is committed, so no `protoc` toolchain is required:
```sh
git clone https://git.eeqj.de/sneak/mfer.git
cd mfer
go build -o bin/mfer ./cmd/mfer
```
Generate a manifest for a directory tree, verify it later, and fetch a published
tree by URL:
```sh
# Write .index.mf, a manifest of the files under the current directory.
bin/mfer gen .
# Verify the files on disk against the manifest. Exits nonzero if any file
# is missing or corrupted.
bin/mfer check .index.mf
# Download and cryptographically verify a tree published over HTTP: mfer
# fetches <url>/index.mf, then downloads every file it lists.
bin/mfer fetch https://example.com/tree/
```
Run `bin/mfer help` for the full command list, or `bin/mfer <command> --help`
for a single command's options.
# Build Status
CI runs `script/cibuild`, which builds the Docker image with `--no-cache`, so
@@ -35,18 +65,23 @@ standard: normalized scripts in `script/` are the entrypoints for the
development workflow, and the Makefile targets are thin shims that call them. We
provide:
- `script/bootstrap` — install all dependencies (Go, golangci-lint, Go module
download, and node/yarn plus the prettier version pinned in
`package.json`/`yarn.lock`), idempotently
- `script/bootstrap` — install all dependencies (Go, Go module download, and
node/yarn plus the prettier version pinned in `package.json`/`yarn.lock`),
idempotently; golangci-lint is not installed, it runs only in Docker
- `script/setup` — make a fresh clone ready for development: runs
`script/bootstrap`, then `script/install-precommit`
- `script/projectname` — output the project name (`mfer`); used by other scripts
such as `script/docker`
- `script/test` — run the test suite (`go test`), regenerating the protobuf code
first if it is stale
- `script/lint` — run `golangci-lint` and verify `gofmt` cleanliness
- `script/fmt` — format all code and docs (writes): `gofumpt`,
`golangci-lint run --fix`, and `script/prettier --write`
- `script/fuzz` — fuzz the manifest parser for one minute; run by hand
(`make fuzz`), never by CI, while `script/test` runs its committed seed corpus
as ordinary tests
- `script/lint` — run `golangci-lint` in Docker: builds only the `lint` stage of
the `Dockerfile` (the Go format check, then the linter), uncached so it runs
every time, then removes the image
- `script/fmt` — format all code and docs (writes): `gofumpt` and
`script/prettier --write`
- `script/prettier` — run prettier over the repository's canonical file set
(Markdown and JSON, minus `.prettierignore`) in the given mode, `--write` or
`--check`; the single definition of that file set, so `script/fmt` and
@@ -72,17 +107,18 @@ yet. Primary development happens on a privately-run Gitea instance at
[tracked there](https://git.eeqj.de/sneak/mfer/issues).
Changes must always be formatted with a standard `go fmt`, syntactically valid,
and must pass the linting defined in the repository (presently only the
`golangci-lint` defaults), which can be run with a `make lint`. The `main`
branch is protected and all changes must be made via
[pull requests](https://git.eeqj.de/sneak/mfer/pulls) and pass CI to be merged.
Any changes submitted to this project must also be
and must pass the linting defined in the repository's `.golangci.yml`, which
`make lint` runs in Docker. The `main` branch is protected and all changes must
be made via [pull requests](https://git.eeqj.de/sneak/mfer/pulls) and pass CI to
be merged. Any changes submitted to this project must also be
[WTFPL-licensed](https://wtfpl.net) to be considered.
See [`REPO_POLICIES.md`](REPO_POLICIES.md) for detailed coding standards,
tooling requirements, and workflow conventions.
# Problem Statement
# Rationale
## The problem
Given a plain URL, there is no standard way to safely and programmatically
download everything "under" that URL path. `wget -r` can traverse directory
@@ -106,7 +142,7 @@ Real issues I face:
- when I download a large file via HTTP, I have no way of knowing if the file
content is what it's supposed to be
# Proposed Solution
## The solution
A standard, a manifest file format, and a tool for generating same.
@@ -138,6 +174,28 @@ The manifest file would do several important things:
- maybe a bittorrent chunklist for torrent client compatibility? perhaps a
top-level infohash for the whole manifest?
# Design
The repository is split into a reusable library and a thin command-line wrapper
around it.
- `mfer/` is the reusable library and the heart of the project: it defines the
manifest format and implements building, scanning, checking, serialization,
and signing. The protobuf schema is `mfer/mf.proto`, and the generated code it
produces (`mfer/mf.pb.go`) is committed alongside it so the library builds
with `go get` and needs no `protoc` toolchain.
- `internal/cli/` holds the command implementations — `generate` (alias `gen`),
`check`, `freshen`, `export`, `list` (alias `ls`), `fetch`, and `version` —
that wire the library to the command-line interface.
- `internal/log/` provides the logging used by the commands and the library.
- `internal/bork/` provides the error the library returns when a manifest's
decompressed contents are not the size the manifest records.
- `cmd/mfer/` is the entrypoint: its `main` package runs `internal/cli` and
exits with the status it returns.
Everything under `internal/` is private to this repository; only the `mfer/`
package is intended for import by other software.
# Design Goals
- Replace SHASUMS/SHASUMS.asc files
@@ -271,9 +329,9 @@ Open work, open design questions included, is tracked in this repo's issues:
- Issues:
[https://git.eeqj.de/sneak/mfer/issues](https://git.eeqj.de/sneak/mfer/issues)
# Authors
# Author
- [@sneak &lt;sneak@sneak.berlin&gt;](mailto:sneak@sneak.berlin)
- [@sneak](https://sneak.berlin)
# License
+6 -3
View File
@@ -2,6 +2,7 @@
package cli
import (
"context"
"encoding/hex"
"errors"
"fmt"
@@ -126,7 +127,9 @@ func (mfa *CLIApp) fetchManifestToTemp(url string) (string, error) {
// verifyRequiredSigner enforces the --require-signature fingerprint
// against the manifest's embedded signing key.
func verifyRequiredSigner(chk *mfer.Checker, requiredSigner string) error {
func verifyRequiredSigner(
ctx context.Context, chk *mfer.Checker, requiredSigner string,
) error {
// Validate fingerprint format: must be exactly 40 hex characters
if len(requiredSigner) != fingerprintHexLen {
return fmt.Errorf("%w, got %d", errInvalidFingerprint, len(requiredSigner))
@@ -145,7 +148,7 @@ func verifyRequiredSigner(chk *mfer.Checker, requiredSigner string) error {
// Extract fingerprint from the embedded public key (not from the
// signer field). This validates the key is importable and gets its
// actual fingerprint.
embeddedFP, err := chk.ExtractEmbeddedSigningKeyFP()
embeddedFP, err := chk.ExtractEmbeddedSigningKeyFP(ctx)
if err != nil {
return fmt.Errorf(
"failed to extract fingerprint from embedded signing key: %w", err)
@@ -303,7 +306,7 @@ func (mfa *CLIApp) checkManifestOperation(ctx *cli.Context) error {
// Check signature requirement
requiredSigner := ctx.String("require-signature")
if requiredSigner != "" {
err = verifyRequiredSigner(chk, requiredSigner)
err = verifyRequiredSigner(ctx.Context, chk, requiredSigner)
if err != nil {
return err
}
+125 -3
View File
@@ -5,6 +5,7 @@ import (
"bytes"
"errors"
"fmt"
"io"
"math/rand"
"os"
"sync"
@@ -14,6 +15,7 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
urfcli "github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer"
)
@@ -27,6 +29,7 @@ const (
testManifest = "/manifest.mf"
testFlagBase = "--base"
testFlagNoExtra = "--no-extra-files"
testFlagVersion = "--version"
)
var errSimulatedWrite = errors.New("simulated write failure")
@@ -39,12 +42,20 @@ var errSimulatedWrite = errors.New("simulated write failure")
var runMu sync.Mutex
// runCLI invokes RunWithOptions while holding runMu so parallel tests
// capture their own output.
// capture their own output. Before releasing the lock it points the
// process-global logger at io.Discard: other tests log outside the lock
// (manifest loads, scans), and those lines must not land in this run's
// buffers once it has returned and its test is reading them.
func runCLI(opts *RunOptions) int {
runMu.Lock()
defer runMu.Unlock()
return RunWithOptions(opts)
exitCode := RunWithOptions(opts)
log.SetOutput(io.Discard, io.Discard)
log.Init()
return exitCode
}
func TestMain(m *testing.M) {
@@ -102,7 +113,7 @@ func TestVersionCommand(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
opts := testOpts([]string{testApp, "version"}, fs)
opts := testOpts([]string{testApp, cmdVersion}, fs)
exitCode := runCLI(opts)
@@ -113,6 +124,117 @@ func TestVersionCommand(t *testing.T) {
assert.Contains(t, stdout, "abc123")
}
// TestVFlagCollision covers the -v/--verbose vs --version flag interaction
// (issue #64). Verbose owns -v; version answers to --version and -V. None of
// these invocations may produce a parser error, and the two ways of asking
// for the version must print the same thing.
func TestVFlagCollision(t *testing.T) {
t.Parallel()
// Invocations that must print the version and exit 0.
versionCases := map[string][]string{
"long version flag": {testApp, testFlagVersion},
"short version flag": {testApp, "-V"},
"verbose then version": {testApp, "-v", testFlagVersion},
"long verbose and version": {testApp, "--verbose", testFlagVersion},
}
for name, args := range versionCases {
t.Run(name, func(t *testing.T) {
t.Parallel()
opts := testOpts(args, afero.NewMemMapFs())
exitCode := runCLI(opts)
assert.Equal(t, 0, exitCode, "stderr: %s", testStderr(t, opts))
assert.Contains(t, testStdout(t, opts), mfer.Version)
assert.NotContains(t, testStderr(t, opts), "two forms of the same flag")
})
}
// Invocations that must enable verbose and exit 0 without a parser error.
verboseCases := map[string][]string{
"short verbose flag": {testApp, "-v"},
"long verbose flag": {testApp, "--verbose"},
}
for name, args := range verboseCases {
t.Run(name, func(t *testing.T) {
t.Parallel()
opts := testOpts(args, afero.NewMemMapFs())
exitCode := runCLI(opts)
assert.Equal(t, 0, exitCode, "stderr: %s", testStderr(t, opts))
assert.Contains(t, testStdout(t, opts), cmdGenerate,
"root should show help listing subcommands")
})
}
}
// TestVersionFlagAndCommandMatch asserts that "mfer --version" and
// "mfer version" produce identical output (issue #64).
func TestVersionFlagAndCommandMatch(t *testing.T) {
t.Parallel()
flagOpts := testOpts([]string{testApp, testFlagVersion}, afero.NewMemMapFs())
require.Equal(t, 0, runCLI(flagOpts))
cmdOpts := testOpts([]string{testApp, cmdVersion}, afero.NewMemMapFs())
require.Equal(t, 0, runCLI(cmdOpts))
assert.Equal(t, testStdout(t, flagOpts), testStdout(t, cmdOpts))
}
// TestRootVerbosityFlags asserts that -q and -v given before any subcommand
// name take effect: in the root itself, and in the fetch and export
// subcommands (issue #64). It does not run in parallel: other tests log
// outside runMu (manifest loads, scans), and a line logged while one of these
// runs is in progress lands in its output.
//
//nolint:paralleltest // see above
func TestRootVerbosityFlags(t *testing.T) {
t.Run("quiet with no subcommand hides the banner", func(t *testing.T) {
loud := testOpts([]string{testApp}, afero.NewMemMapFs())
require.Equal(t, 0, runCLI(loud))
assert.Contains(t, testStdout(t, loud), banner)
quiet := testOpts([]string{testApp, "-q"}, afero.NewMemMapFs())
require.Equal(t, 0, runCLI(quiet))
assert.Contains(t, testStdout(t, quiet), cmdGenerate)
assert.NotContains(t, testStdout(t, quiet), banner)
})
t.Run("quiet before fetch hides the banner", func(t *testing.T) {
opts := testOpts([]string{testApp, "-q", cmdFetch}, afero.NewMemMapFs())
assert.Equal(t, 1, runCLI(opts))
assert.Contains(t, testStderr(t, opts), errURLRequired.Error())
assert.NotContains(t, testStdout(t, opts), banner)
})
t.Run("verbose twice before fetch enables debug logging", func(t *testing.T) {
opts := testOpts([]string{testApp, "-v", "-v", cmdFetch}, afero.NewMemMapFs())
assert.Equal(t, 1, runCLI(opts))
assert.Contains(t, testStderr(t, opts), "fetchManifestOperation()")
})
t.Run("quiet before export hides the manifest summary", func(t *testing.T) {
fs := afero.NewMemMapFs()
manifest := buildTestManifest(t, map[string][]byte{"a.txt": []byte("a")})
require.NoError(t, afero.WriteFile(fs, testManifest, manifest, 0o644))
loud := testOpts([]string{testApp, cmdExport, testManifest}, fs)
require.Equal(t, 0, runCLI(loud))
assert.Contains(t, testStderr(t, loud), "loaded manifest")
quiet := testOpts([]string{testApp, "-q", cmdExport, testManifest}, fs)
require.Equal(t, 0, runCLI(quiet))
assert.NotContains(t, testStderr(t, quiet), "loaded manifest")
})
}
func TestHelpCommand(t *testing.T) {
t.Parallel()
+7 -5
View File
@@ -83,7 +83,8 @@ func TestVerifyRequiredSignerMessages(t *testing.T) {
t.Run("invalid fingerprint length", func(t *testing.T) {
t.Parallel()
err := verifyRequiredSigner(unsignedChecker(t), "12345678")
err := verifyRequiredSigner(context.Background(),
unsignedChecker(t), "12345678")
require.ErrorIs(t, err, errInvalidFingerprint)
assert.EqualError(t, err,
"invalid fingerprint: must be exactly 40 hex characters, got 8")
@@ -92,7 +93,8 @@ func TestVerifyRequiredSignerMessages(t *testing.T) {
t.Run("manifest not signed", func(t *testing.T) {
t.Parallel()
err := verifyRequiredSigner(unsignedChecker(t), msgFpA)
err := verifyRequiredSigner(context.Background(),
unsignedChecker(t), msgFpA)
require.ErrorIs(t, err, errManifestNotSigned)
assert.EqualError(t, err,
"manifest is not signed, but signature from "+msgFpA+" is required")
@@ -109,10 +111,10 @@ func TestVerifyRequiredSignerMessages(t *testing.T) {
func TestSignerMismatchMessage(t *testing.T) {
chk := signedChecker(t)
embeddedFP, err := chk.ExtractEmbeddedSigningKeyFP()
embeddedFP, err := chk.ExtractEmbeddedSigningKeyFP(context.Background())
require.NoError(t, err)
err = verifyRequiredSigner(chk, msgFpB)
err = verifyRequiredSigner(context.Background(), chk, msgFpB)
require.ErrorIs(t, err, errSignerMismatch)
assert.EqualError(t, err,
"embedded signing key fingerprint "+embeddedFP+
@@ -160,7 +162,7 @@ func signedChecker(t *testing.T) *mfer.Checker {
var buf bytes.Buffer
require.NoError(t, b.Build(&buf))
require.NoError(t, b.Build(context.Background(), &buf))
fs := afero.NewMemMapFs()
require.NoError(t, afero.WriteFile(fs, "/index.mf", buf.Bytes(), 0o644))
+4 -3
View File
@@ -1,6 +1,7 @@
package cli
import (
"context"
"crypto/sha256"
"errors"
"fmt"
@@ -304,7 +305,7 @@ func (h *freshenHasher) processEntry(e *freshenEntry) error {
// writeFreshenedManifest writes the manifest atomically (write to a
// temp file, then rename over the target).
func writeFreshenedManifest(
afs afero.Fs, builder *mfer.Builder, manifestPath string,
ctx context.Context, afs afero.Fs, builder *mfer.Builder, manifestPath string,
) error {
tmpPath := manifestPath + ".tmp"
@@ -313,7 +314,7 @@ func writeFreshenedManifest(
return fmt.Errorf("failed to create temp file: %w", err)
}
err = builder.Build(outFile)
err = builder.Build(ctx, outFile)
_ = outFile.Close()
if err != nil {
@@ -530,7 +531,7 @@ func (mfa *CLIApp) freshenManifestOperation(ctx *cli.Context) error {
}
// Write updated manifest atomically (write to temp, then rename)
err = writeFreshenedManifest(mfa.Fs, hasher.builder, manifestPath)
err = writeFreshenedManifest(ctx.Context, mfa.Fs, hasher.builder, manifestPath)
if err != nil {
return err
}
+45 -5
View File
@@ -19,6 +19,7 @@ const (
cmdCheck = "check"
cmdExport = "export"
cmdFetch = "fetch"
cmdVersion = "version"
flagProgress = "progress"
@@ -68,6 +69,12 @@ func (mfa *CLIApp) VersionString() string {
return mfer.Version
}
// printVersion writes the version line shared by the --version flag and the
// version subcommand, so both produce identical output.
func (mfa *CLIApp) printVersion() {
_, _ = fmt.Fprintf(mfa.Stdout, "%s version %s\n", mfa.appname, mfa.VersionString())
}
func (mfa *CLIApp) printBanner() {
if log.GetLevel() <= log.InfoLevel {
_, _ = fmt.Fprintln(mfa.Stdout, banner)
@@ -78,20 +85,34 @@ func (mfa *CLIApp) printBanner() {
}
}
// setVerbosity sets the log level from -v and -q, given before the subcommand
// name (the root's copies), after it (the subcommand's own copies), or both.
// urfave/cli reads a flag from the nearest command that defines it, so each
// command in the lineage is asked. The highest -v count wins rather than the
// sum, because a subcommand without its own copies reads the root's.
func (mfa *CLIApp) setVerbosity(c *cli.Context) {
_, present := os.LookupEnv("MFER_DEBUG")
verbosity := 0
quiet := false
for _, ctx := range c.Lineage() {
verbosity = max(verbosity, ctx.Count("verbose"))
quiet = quiet || ctx.Bool("quiet")
}
switch {
case present:
log.EnableDebugLogging()
case c.Bool("quiet"):
case quiet:
log.SetLevel(log.ErrorLevel)
default:
log.SetLevelFromVerbosity(c.Count("verbose"))
log.SetLevelFromVerbosity(verbosity)
}
}
// commonFlags returns the flags shared by most commands (-v, -q)
// commonFlags returns the -v and -q flags taken by the root and by the
// generate, check, freshen and fetch subcommands.
func commonFlags() []cli.Flag {
return []cli.Flag{
&cli.BoolFlag{
@@ -259,6 +280,8 @@ func (mfa *CLIApp) exportCommand() *cli.Command {
Usage: "Export manifest contents as JSON",
ArgsUsage: "[manifest file or URL]",
Action: func(c *cli.Context) error {
mfa.setVerbosity(c)
return mfa.exportManifestOperation(c)
},
}
@@ -266,10 +289,10 @@ func (mfa *CLIApp) exportCommand() *cli.Command {
func (mfa *CLIApp) versionCommand() *cli.Command {
return &cli.Command{
Name: "version",
Name: cmdVersion,
Usage: "Show version",
Action: func(_ *cli.Context) error {
_, _ = fmt.Fprintln(mfa.Stdout, mfa.VersionString())
mfa.printVersion()
return nil
},
@@ -325,6 +348,21 @@ func (mfa *CLIApp) run(args []string) {
log.SetOutput(mfa.Stdout, mfa.Stderr)
log.Init()
// -v means verbose, not version. urfave/cli's built-in version flag
// claims -v by default, which made "mfer -v --version" fail to parse and
// gave -v a different meaning at the root than on the generate, check,
// freshen and fetch subcommands, where it means verbose. Verbose is the
// more common meaning of -v in tools that offer both, so -v means verbose
// at the root too and the version flag takes the capital -V.
// VersionFlag and VersionPrinter are urfave/cli package globals; run() is
// serialized in tests, so assigning them here is safe.
cli.VersionFlag = &cli.BoolFlag{
Name: cmdVersion,
Aliases: []string{"V"},
Usage: "print the version",
}
cli.VersionPrinter = func(_ *cli.Context) { mfa.printVersion() }
mfa.app = &cli.App{
Name: mfa.appname,
Usage: "Manifest generator",
@@ -332,11 +370,13 @@ func (mfa *CLIApp) run(args []string) {
EnableBashCompletion: true,
Writer: mfa.Stdout,
ErrWriter: mfa.Stderr,
Flags: commonFlags(),
Action: func(c *cli.Context) error {
if c.Args().Len() > 0 {
return fmt.Errorf("%w %q", errUnknownCommand, c.Args().First())
}
mfa.setVerbosity(c)
mfa.printBanner()
return cli.ShowAppHelp(c)
+6 -4
View File
@@ -3,6 +3,7 @@
package mfer
import (
"context"
"crypto/sha256"
"errors"
"fmt"
@@ -281,8 +282,9 @@ func (b *Builder) SetSigningOptions(opts *SigningOptions) {
b.signingOptions = opts
}
// Build finalizes the manifest and writes it to the writer.
func (b *Builder) Build(w io.Writer) error {
// Build finalizes the manifest and writes it to the writer. ctx bounds the
// gpg runs that sign the manifest when signing options are set.
func (b *Builder) Build(ctx context.Context, w io.Writer) error {
b.mu.Lock()
defer b.mu.Unlock()
@@ -308,13 +310,13 @@ func (b *Builder) Build(w io.Writer) error {
}
// Generate outer wrapper
err := m.generateOuter()
err := m.generateOuter(ctx)
if err != nil {
return fmt.Errorf("build: generate outer: %w", err)
}
// Generate final output
err = m.generate()
err = m.generate(ctx)
if err != nil {
return fmt.Errorf("build: generate: %w", err)
}
+34 -8
View File
@@ -3,6 +3,8 @@ package mfer
import (
"bytes"
"context"
"fmt"
"strings"
"testing"
"time"
@@ -113,7 +115,7 @@ func TestBuilderBuild(t *testing.T) {
var buf bytes.Buffer
err = b.Build(&buf)
err = b.Build(context.Background(), &buf)
require.NoError(t, err)
// Should have magic bytes
@@ -177,7 +179,7 @@ func TestBuilderDeterministicOutput(t *testing.T) {
var buf bytes.Buffer
err := b.Build(&buf)
err := b.Build(context.Background(), &buf)
require.NoError(t, err)
return buf.Bytes()
@@ -325,7 +327,7 @@ func TestBuilderBuildRoundTrip(t *testing.T) {
}
var buf bytes.Buffer
require.NoError(t, b.Build(&buf))
require.NoError(t, b.Build(context.Background(), &buf))
m, err := NewManifestFromReader(&buf)
require.NoError(t, err)
@@ -348,6 +350,30 @@ func TestBuilderBuildRoundTrip(t *testing.T) {
}
}
// A manifest whose payload is larger than one zstd block (128 KiB) is
// written as a frame asking for the writer's whole window, zstdWindowSize,
// which is the most the parser accepts.
func TestBuilderBuildRoundTripLargeManifest(t *testing.T) {
t.Parallel()
hash := make([]byte, 34) // multihash: 2-byte prefix + 32-byte SHA-256
b := NewBuilder()
for i := range 4000 {
path := RelFilePath(fmt.Sprintf("dir/file-%05d.txt", i))
require.NoError(t, b.AddFileWithHash(path, FileSize(i), ModTime{}, hash))
}
var buf bytes.Buffer
require.NoError(t, b.Build(context.Background(), &buf))
m, err := NewManifestFromReader(&buf)
require.NoError(t, err)
require.Greater(t, m.pbOuter.GetSize(), int64(128<<10))
assert.Len(t, m.Files(), 4000)
}
func TestNewManifestFromReaderInvalidMagic(t *testing.T) {
t.Parallel()
@@ -383,7 +409,7 @@ func TestManifestString(t *testing.T) {
require.NoError(t, err)
var buf bytes.Buffer
require.NoError(t, b.Build(&buf))
require.NoError(t, b.Build(context.Background(), &buf))
m, err := NewManifestFromReader(&buf)
require.NoError(t, err)
@@ -397,7 +423,7 @@ func TestBuilderBuildEmpty(t *testing.T) {
var buf bytes.Buffer
err := b.Build(&buf)
err := b.Build(context.Background(), &buf)
require.NoError(t, err)
// Should still produce valid manifest with 0 files
@@ -416,7 +442,7 @@ func TestBuilderOmitsCreatedAtByDefault(t *testing.T) {
require.NoError(t, err)
var buf bytes.Buffer
require.NoError(t, b.Build(&buf))
require.NoError(t, b.Build(context.Background(), &buf))
m, err := NewManifestFromReader(&buf)
require.NoError(t, err)
@@ -438,7 +464,7 @@ func TestBuilderIncludesCreatedAtWhenRequested(t *testing.T) {
require.NoError(t, err)
var buf bytes.Buffer
require.NoError(t, b.Build(&buf))
require.NoError(t, b.Build(context.Background(), &buf))
m, err := NewManifestFromReader(&buf)
require.NoError(t, err)
@@ -464,7 +490,7 @@ func TestBuilderDeterministicFileOrder(t *testing.T) {
}
var buf bytes.Buffer
require.NoError(t, b.Build(&buf))
require.NoError(t, b.Build(context.Background(), &buf))
m, err := NewManifestFromReader(&buf)
require.NoError(t, err)
+2 -2
View File
@@ -164,12 +164,12 @@ func (c *Checker) SigningPubKey() []byte {
// ExtractEmbeddedSigningKeyFP imports the manifest's embedded public key into a
// temporary keyring and extracts its fingerprint. This validates the key and
// returns its actual fingerprint from the key material itself.
func (c *Checker) ExtractEmbeddedSigningKeyFP() (string, error) {
func (c *Checker) ExtractEmbeddedSigningKeyFP(ctx context.Context) (string, error) {
if len(c.signingPubKey) == 0 {
return "", errNoSigningPubKey
}
return gpgExtractPubKeyFingerprint(c.signingPubKey)
return gpgExtractPubKeyFingerprint(ctx, c.signingPubKey)
}
// Check verifies all files against the manifest.
+1 -1
View File
@@ -61,7 +61,7 @@ func createTestManifest(
}
var buf bytes.Buffer
require.NoError(t, builder.Build(&buf))
require.NoError(t, builder.Build(context.Background(), &buf))
require.NoError(t, afero.WriteFile(fs, manifestPath, buf.Bytes(), 0o644))
}
+3
View File
@@ -12,6 +12,9 @@ const (
// memory.
MaxDecompressedSize int64 = 256 * 1024 * 1024
// zstdWindowSize is the zstd window zstd.SpeedBestCompression gives mfer's writer.
zstdWindowSize = 8 << 20
// uuidLength is the length in bytes of a binary UUID.
uuidLength = 16
)
+15 -1
View File
@@ -2,6 +2,7 @@ package mfer
import (
"bytes"
"context"
"crypto/sha256"
"errors"
"fmt"
@@ -92,7 +93,9 @@ func (m *manifest) verifyOuterIntegrity() error {
)
}
// Loading a manifest takes no context; gpgTimeout still bounds gpg.
err = gpgVerify(
context.Background(),
[]byte(sigString),
m.pbOuter.GetSignature(),
m.pbOuter.GetSigningPubKey(),
@@ -111,7 +114,18 @@ func (m *manifest) verifyOuterIntegrity() error {
func (m *manifest) decompressInner() ([]byte, error) {
bb := bytes.NewBuffer(m.pbOuter.GetInnerMessage())
zr, err := zstd.NewReader(bb)
// By default the decoder decodes a payload under 128 KiB in full,
// each frame up to the decoder's limit, before the LimitReader below
// reads any of it. Decoding synchronously and never in full makes it
// decode only what the LimitReader asks for. It also sets aside a new
// buffer for each frame that asks for a larger window than the frames
// before it, even a frame holding no data. Refusing windows above
// zstdWindowSize keeps each buffer to a little over zstdWindowSize, and
// the buffers of frames holding no data to about 16 times it in total.
zr, err := zstd.NewReader(bb,
zstd.WithDecoderConcurrency(1),
zstd.WithDecodeBuffersBelow(0),
zstd.WithDecoderMaxWindow(zstdWindowSize))
if err != nil {
return nil, fmt.Errorf("deserialize: zstd reader: %w", err)
}
+83
View File
@@ -0,0 +1,83 @@
//nolint:testpackage // white-box tests exercise unexported internals
package mfer
import (
"bytes"
"runtime"
"testing"
"google.golang.org/protobuf/proto"
)
// FuzzNewManifestFromReader feeds arbitrary bytes to the manifest parser.
// `make test` runs it on the seed corpus in
// testdata/fuzz/FuzzNewManifestFromReader; `make fuzz` searches for new
// inputs.
//
// For every input the parser must return a manifest or an error, not both
// and not neither, and must not allocate more than a fixed multiple of its
// input and of the decompressed data it may read, plus room for the
// decoder's window buffers. A panic or a hang fails the test on its own.
func FuzzNewManifestFromReader(f *testing.F) {
// A signed manifest makes the parser write the key and signature to a
// temporary directory and run gpg on them. With gpg off the PATH and
// temporary files kept in the test's own directory, no process is
// started and nothing is written elsewhere; such input ends in an
// error instead.
f.Setenv("PATH", "")
f.Setenv("TMPDIR", f.TempDir())
f.Fuzz(func(t *testing.T, data []byte) {
var before, after runtime.MemStats
runtime.ReadMemStats(&before)
m, err := NewManifestFromReader(bytes.NewReader(data))
runtime.ReadMemStats(&after)
if (m == nil) == (err == nil) {
t.Fatalf("got manifest %p and error %v, want exactly one", m, err)
}
// The parser reads at most the declared size plus one byte of
// decompressed data, and never more than MaxDecompressedSize.
decompressed := uint64(MaxDecompressedSize)
outer := new(MFFileOuter)
if validateMagic(data) &&
proto.Unmarshal(data[len(MAGIC):], outer) == nil {
size := outer.GetSize()
if size > 0 && size < MaxDecompressedSize {
decompressed = uint64(size) + 1
}
}
// It also keeps a few copies of its input. Buffers grow by
// copying, so reaching those sizes allocates a few times them in
// total: sixteen times the input and the decompressed data leaves
// room for that.
//
// The decoder also sets aside a new buffer of one to two times the
// window for each frame that asks for a larger window than the
// frames before it, and refuses windows above zstdWindowSize. A
// frame that gives its content size instead of a window has that
// size as its window, refused above zstdWindowSize like any other.
// Frames asking for every window size up to that make it set aside
// about 16 times zstdWindowSize in total; 24 times leaves room.
//
// The seed whose frame claims 8 GiB fails if the decoder sets that
// size aside; the seed whose frames ask for ever larger windows
// fails if the decoder accepts windows of twice zstdWindowSize; the
// seed whose two frames together exceed MaxDecompressedSize fails
// if the decoder decodes them in full instead of stopping at the
// declared size.
limit := 16*(uint64(len(data))+decompressed) + 24*zstdWindowSize
allocated := after.TotalAlloc - before.TotalAlloc
if allocated > limit {
t.Fatalf("allocated %d bytes for %d bytes of input, limit %d",
allocated, len(data), limit)
}
})
}
+2 -1
View File
@@ -3,6 +3,7 @@ package mfer
import (
"bytes"
"context"
"crypto/sha256"
"fmt"
"testing"
@@ -122,7 +123,7 @@ func TestDeserializeValidManifestRoundTrips(t *testing.T) {
require.NoError(t, b.AddFileWithHash("dir/file.txt", 123, ModTime{}, hash))
var buf bytes.Buffer
require.NoError(t, b.Build(&buf))
require.NoError(t, b.Build(context.Background(), &buf))
m, err := NewManifestFromReader(bytes.NewReader(buf.Bytes()))
require.NoError(t, err)
+4 -2
View File
@@ -2,6 +2,7 @@
package mfer
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
@@ -80,6 +81,7 @@ func TestSerializeInternalErrorMessagesVerbatim(t *testing.T) {
t.Parallel()
m := &manifest{}
require.EqualError(t, m.generate(), "internal error: pbInner not set")
require.EqualError(t, m.generateOuter(), "internal error")
require.EqualError(t, m.generate(context.Background()),
"internal error: pbInner not set")
require.EqualError(t, m.generateOuter(context.Background()), "internal error")
}
+48 -15
View File
@@ -10,9 +10,21 @@ import (
"os/exec"
"path/filepath"
"strings"
"time"
)
const (
// gpgTimeout bounds every gpg run, which can otherwise wait forever on
// a passphrase prompt or a stalled gpg-agent. A minute leaves a person
// time to type a passphrase or touch a smartcard.
gpgTimeout = time.Minute
// gpgWaitDelay is how long a gpg run keeps waiting for gpg's stdout
// and stderr to close once gpg has been killed or has exited. Reading
// what gpg itself wrote takes far less; only a process gpg left behind
// holds them open longer.
gpgWaitDelay = time.Second
// privateDirPerms is the permission mode for temporary GPG home
// directories.
privateDirPerms os.FileMode = 0o700
@@ -66,8 +78,20 @@ func gpgArgs(opts []string, positional ...string) []string {
}
// runGPG runs the gpg binary in batch mode with the given arguments and
// optional stdin, returning captured stdout and stderr.
func runGPG(stdin io.Reader, args ...string) (*bytes.Buffer, *bytes.Buffer, error) {
// optional stdin, returning captured stdout and stderr. gpg is killed when
// ctx ends or gpgTimeout passes, whichever comes first.
func runGPG(
ctx context.Context, stdin io.Reader, args ...string,
) (*bytes.Buffer, *bytes.Buffer, error) {
// exec.CommandContext kills only gpg itself. A gpg-agent that gpg
// starts runs detached and holds none of gpg's output, but another
// process gpg leaves behind (a wrapper script that runs the real gpg
// without exec, for example) can keep gpg's stdout or stderr open, and
// Run would wait for it to exit. WaitDelay stops that wait
// gpgWaitDelay after the kill; that process is left running.
ctx, cancel := context.WithTimeout(ctx, gpgTimeout)
defer cancel()
fullArgs := append([]string{"--batch", "--no-tty"}, args...)
// G204: the executable name is a compile-time constant. The arguments
@@ -76,7 +100,8 @@ func runGPG(stdin io.Reader, args ...string) (*bytes.Buffer, *bytes.Buffer, erro
// option or after the "--" end-of-options marker inserted by gpgArgs,
// and therefore cannot be reinterpreted by gpg as an option.
cmd := exec.CommandContext( //nolint:gosec // G204: see comment above
context.Background(), "gpg", fullArgs...)
ctx, "gpg", fullArgs...)
cmd.WaitDelay = gpgWaitDelay
cmd.Stdin = stdin
var stdout, stderr bytes.Buffer
@@ -85,6 +110,14 @@ func runGPG(stdin io.Reader, args ...string) (*bytes.Buffer, *bytes.Buffer, erro
cmd.Stderr = &stderr
err := cmd.Run()
if err != nil && ctx.Err() != nil {
// gpg was killed because ctx ended, which Run reports only as
// "signal: killed"; return the reason instead.
err = ctx.Err()
if errors.Is(err, context.DeadlineExceeded) {
err = fmt.Errorf("gpg timed out: %w", err)
}
}
return &stdout, &stderr, err
}
@@ -105,8 +138,8 @@ func parseFingerprint(colonOutput string) (string, bool) {
// gpgSign creates a detached signature of the data using the specified key.
// Returns the armored detached signature.
func gpgSign(data []byte, keyID GPGKeyID) ([]byte, error) {
stdout, stderr, err := runGPG(bytes.NewReader(data),
func gpgSign(ctx context.Context, data []byte, keyID GPGKeyID) ([]byte, error) {
stdout, stderr, err := runGPG(ctx, bytes.NewReader(data),
"--detach-sign",
gpgOptArmor,
"--local-user", string(keyID),
@@ -120,8 +153,8 @@ func gpgSign(data []byte, keyID GPGKeyID) ([]byte, error) {
// gpgExportPublicKey exports the public key for the specified key ID.
// Returns the armored public key.
func gpgExportPublicKey(keyID GPGKeyID) ([]byte, error) {
stdout, stderr, err := runGPG(nil,
func gpgExportPublicKey(ctx context.Context, keyID GPGKeyID) ([]byte, error) {
stdout, stderr, err := runGPG(ctx, nil,
gpgArgs([]string{"--export", gpgOptArmor}, string(keyID))...,
)
if err != nil {
@@ -136,8 +169,8 @@ func gpgExportPublicKey(keyID GPGKeyID) ([]byte, error) {
}
// gpgGetKeyFingerprint gets the full fingerprint for a key ID.
func gpgGetKeyFingerprint(keyID GPGKeyID) ([]byte, error) {
stdout, stderr, err := runGPG(nil,
func gpgGetKeyFingerprint(ctx context.Context, keyID GPGKeyID) ([]byte, error) {
stdout, stderr, err := runGPG(ctx, nil,
gpgArgs([]string{"--with-colons", "--fingerprint"}, string(keyID))...,
)
if err != nil {
@@ -157,7 +190,7 @@ func gpgGetKeyFingerprint(keyID GPGKeyID) ([]byte, error) {
// gpgExtractPubKeyFingerprint imports a public key into a temporary keyring
// and extracts its fingerprint. This verifies the key is valid and returns
// the actual fingerprint from the key material.
func gpgExtractPubKeyFingerprint(pubKey []byte) (string, error) {
func gpgExtractPubKeyFingerprint(ctx context.Context, pubKey []byte) (string, error) {
// Create temporary directory for GPG operations
tmpDir, err := os.MkdirTemp("", "mfer-gpg-fingerprint-*")
if err != nil {
@@ -181,7 +214,7 @@ func gpgExtractPubKeyFingerprint(pubKey []byte) (string, error) {
}
// Import the public key into the temporary keyring
_, importStderr, err := runGPG(nil,
_, importStderr, err := runGPG(ctx, nil,
gpgArgs([]string{gpgOptHomedir, tmpDir, "--import"}, pubKeyFile)...,
)
if err != nil {
@@ -191,7 +224,7 @@ func gpgExtractPubKeyFingerprint(pubKey []byte) (string, error) {
}
// List keys to get fingerprint
listStdout, listStderr, err := runGPG(nil,
listStdout, listStderr, err := runGPG(ctx, nil,
"--homedir", tmpDir,
"--with-colons",
"--fingerprint",
@@ -212,7 +245,7 @@ func gpgExtractPubKeyFingerprint(pubKey []byte) (string, error) {
// gpgVerify verifies a detached signature against data using the provided public key.
// It creates a temporary keyring to import the public key for verification.
func gpgVerify(data, signature, pubKey []byte) error {
func gpgVerify(ctx context.Context, data, signature, pubKey []byte) error {
// Create temporary directory for GPG operations
tmpDir, err := os.MkdirTemp("", "mfer-gpg-verify-*")
if err != nil {
@@ -252,7 +285,7 @@ func gpgVerify(data, signature, pubKey []byte) error {
}
// Import the public key into the temporary keyring
_, importStderr, err := runGPG(nil,
_, importStderr, err := runGPG(ctx, nil,
gpgArgs([]string{gpgOptHomedir, tmpDir, "--import"}, pubKeyFile)...,
)
if err != nil {
@@ -262,7 +295,7 @@ func gpgVerify(data, signature, pubKey []byte) error {
}
// Verify the signature
_, verifyStderr, err := runGPG(nil,
_, verifyStderr, err := runGPG(ctx, nil,
gpgArgs([]string{gpgOptHomedir, tmpDir, gpgOptVerify},
sigFile, dataFile)...,
)
+116 -20
View File
@@ -4,11 +4,15 @@ package mfer
import (
"bytes"
"context"
"io"
"os"
"os/exec"
"path/filepath"
"strconv"
"strings"
"syscall"
"testing"
"time"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
@@ -43,8 +47,11 @@ Expire-Date: 0
paramsFile := filepath.Join(gpgHome, "key-params")
require.NoError(t, os.WriteFile(paramsFile, []byte(keyParams), 0o600))
ctx, cancel := context.WithTimeout(context.Background(), gpgTimeout)
defer cancel()
//nolint:gosec // paramsFile is a test-controlled path inside t.TempDir()
cmd := exec.CommandContext(context.Background(), "gpg",
cmd := exec.CommandContext(ctx, "gpg",
"--batch", "--gen-key", paramsFile)
cmd.Env = append(os.Environ(), "GNUPGHOME="+gpgHome)
@@ -55,7 +62,7 @@ Expire-Date: 0
}
// Get the key fingerprint
cmd = exec.CommandContext(context.Background(), "gpg",
cmd = exec.CommandContext(ctx, "gpg",
"--list-keys", "--with-colons", "test@mfer.test")
cmd.Env = append(os.Environ(), "GNUPGHOME="+gpgHome)
@@ -90,7 +97,7 @@ func TestGPGSign(t *testing.T) {
t.Setenv("GNUPGHOME", gpgHome)
data := []byte("test data to sign")
sig, err := gpgSign(data, keyID)
sig, err := gpgSign(context.Background(), data, keyID)
require.NoError(t, err)
assert.NotEmpty(t, sig)
assert.Contains(t, string(sig), "-----BEGIN PGP SIGNATURE-----")
@@ -101,7 +108,7 @@ func TestGPGExportPublicKey(t *testing.T) {
keyID, gpgHome := testGPGEnv(t)
t.Setenv("GNUPGHOME", gpgHome)
pubKey, err := gpgExportPublicKey(keyID)
pubKey, err := gpgExportPublicKey(context.Background(), keyID)
require.NoError(t, err)
assert.NotEmpty(t, pubKey)
assert.Contains(t, string(pubKey), "-----BEGIN PGP PUBLIC KEY BLOCK-----")
@@ -112,7 +119,7 @@ func TestGPGGetKeyFingerprint(t *testing.T) {
keyID, gpgHome := testGPGEnv(t)
t.Setenv("GNUPGHOME", gpgHome)
fingerprint, err := gpgGetKeyFingerprint(keyID)
fingerprint, err := gpgGetKeyFingerprint(context.Background(), keyID)
require.NoError(t, err)
assert.NotEmpty(t, fingerprint)
// The fingerprint should be 40 hex chars
@@ -146,12 +153,12 @@ func TestGPGOptionLikeKeyIDIsNotAnOption(t *testing.T) {
_, gpgHome := testGPGEnv(t)
t.Setenv("GNUPGHOME", gpgHome)
pubKey, err := gpgExportPublicKey(GPGKeyID("--version"))
pubKey, err := gpgExportPublicKey(context.Background(), GPGKeyID("--version"))
require.Error(t, err)
require.ErrorIs(t, err, errGPGKeyNotFound)
assert.NotContains(t, string(pubKey), "gpg (GnuPG)")
fpr, err := gpgGetKeyFingerprint(GPGKeyID("--version"))
fpr, err := gpgGetKeyFingerprint(context.Background(), GPGKeyID("--version"))
require.Error(t, err)
assert.NotContains(t, string(fpr), "gpg (GnuPG)")
}
@@ -162,7 +169,8 @@ func TestGPGSignInvalidKey(t *testing.T) {
t.Setenv("GNUPGHOME", gpgHome)
data := []byte("test data")
_, err := gpgSign(data, GPGKeyID("NONEXISTENT_KEY_ID_12345"))
_, err := gpgSign(context.Background(), data,
GPGKeyID("NONEXISTENT_KEY_ID_12345"))
assert.Error(t, err)
}
@@ -185,7 +193,7 @@ func TestBuilderWithSigning(t *testing.T) {
// Build the manifest
var buf bytes.Buffer
err = b.Build(&buf)
err = b.Build(context.Background(), &buf)
require.NoError(t, err)
// Parse the manifest and verify signature fields are populated
@@ -251,14 +259,14 @@ func TestGPGVerify(t *testing.T) {
t.Setenv("GNUPGHOME", gpgHome)
data := []byte("test data to sign and verify")
sig, err := gpgSign(data, keyID)
sig, err := gpgSign(context.Background(), data, keyID)
require.NoError(t, err)
pubKey, err := gpgExportPublicKey(keyID)
pubKey, err := gpgExportPublicKey(context.Background(), keyID)
require.NoError(t, err)
// Verify the signature
err = gpgVerify(data, sig, pubKey)
err = gpgVerify(context.Background(), data, sig, pubKey)
require.NoError(t, err)
}
@@ -267,15 +275,15 @@ func TestGPGVerifyInvalidSignature(t *testing.T) {
t.Setenv("GNUPGHOME", gpgHome)
data := []byte("test data to sign")
sig, err := gpgSign(data, keyID)
sig, err := gpgSign(context.Background(), data, keyID)
require.NoError(t, err)
pubKey, err := gpgExportPublicKey(keyID)
pubKey, err := gpgExportPublicKey(context.Background(), keyID)
require.NoError(t, err)
// Try to verify with different data - should fail
wrongData := []byte("different data")
err = gpgVerify(wrongData, sig, pubKey)
err = gpgVerify(context.Background(), wrongData, sig, pubKey)
assert.Error(t, err)
}
@@ -284,12 +292,12 @@ func TestGPGVerifyBadPublicKey(t *testing.T) {
t.Setenv("GNUPGHOME", gpgHome)
data := []byte("test data")
sig, err := gpgSign(data, keyID)
sig, err := gpgSign(context.Background(), data, keyID)
require.NoError(t, err)
// Try to verify with invalid public key - should fail
badPubKey := []byte("not a valid public key")
err = gpgVerify(data, sig, badPubKey)
err = gpgVerify(context.Background(), data, sig, badPubKey)
assert.Error(t, err)
}
@@ -312,7 +320,7 @@ func TestManifestSignatureVerification(t *testing.T) {
// Build the manifest
var buf bytes.Buffer
err = b.Build(&buf)
err = b.Build(context.Background(), &buf)
require.NoError(t, err)
// Parse the manifest - signature should be verified during load
@@ -341,7 +349,7 @@ func TestManifestTamperedSignatureFails(t *testing.T) {
var buf bytes.Buffer
err = b.Build(&buf)
err = b.Build(context.Background(), &buf)
require.NoError(t, err)
// Tamper with the signature by replacing some bytes
@@ -375,7 +383,7 @@ func TestBuilderWithoutSigning(t *testing.T) {
// Build the manifest
var buf bytes.Buffer
err = b.Build(&buf)
err = b.Build(context.Background(), &buf)
require.NoError(t, err)
// Parse the manifest and verify signature fields are empty
@@ -390,3 +398,91 @@ func TestBuilderWithoutSigning(t *testing.T) {
assert.Empty(t, manifest.pbOuter.GetSigningPubKey(),
"signing public key should be empty when not signing")
}
// fakeGPGPath writes script as an executable named gpg into a temporary
// directory and returns a PATH value with that directory first.
func fakeGPGPath(t *testing.T, script string) string {
t.Helper()
binDir := t.TempDir()
//nolint:gosec // G306: the fake gpg has to be executable
require.NoError(t, os.WriteFile(filepath.Join(binDir, "gpg"),
[]byte(script), 0o700))
return binDir + string(os.PathListSeparator) + os.Getenv("PATH")
}
// TestGPGTimeoutKillsGPG puts a fake gpg that never finishes first on
// PATH and checks that a run past its deadline is killed and reported as
// a timeout of the named operation, instead of hanging.
func TestGPGTimeoutKillsGPG(t *testing.T) {
t.Setenv("PATH", fakeGPGPath(t, "#!/bin/sh\nexec sleep 10\n"))
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
_, err := gpgSign(ctx, []byte("data"), GPGKeyID("any"))
require.ErrorIs(t, err, context.DeadlineExceeded)
assert.Contains(t, err.Error(), "gpg sign failed: gpg timed out")
}
// TestGPGCancelWhenChildHoldsOutput uses a fake gpg that runs sleep as a
// child instead of exec-ing it, the way a wrapper script around the real
// gpg might. Killing the fake gpg leaves sleep holding its stdout and
// stderr open; the call must still return once ctx ends instead of waiting
// for sleep to exit. The fake gpg writes the process ID of sleep to a named
// pipe; the test ends ctx only after reading it, so sleep is running by
// then, and kills sleep before returning.
func TestGPGCancelWhenChildHoldsOutput(t *testing.T) {
pidPipe := filepath.Join(t.TempDir(), "sleep.pid")
require.NoError(t, syscall.Mkfifo(pidPipe, 0o600))
// sleep outlasts the 10 s wait below, so a call that waits for it fails.
t.Setenv("PATH", fakeGPGPath(t,
"#!/bin/sh\nsleep 60 &\necho $! >'"+pidPipe+"'\nwait\n"))
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
signErr := make(chan error, 1)
go func() {
_, err := gpgSign(ctx, []byte("data"), GPGKeyID("any"))
signErr <- err
}()
pid, err := os.ReadFile(pidPipe) //nolint:gosec // G304: path inside t.TempDir()
require.NoError(t, err)
n, err := strconv.Atoi(strings.TrimSpace(string(pid)))
require.NoError(t, err)
sleep, err := os.FindProcess(n)
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, sleep.Kill()) })
cancel()
// The call should return about gpgWaitDelay (one second) after the
// cancel. 10 s is far above that and well under the 30 s test timeout,
// which would abort the whole package before the cleanup kills sleep.
select {
case err := <-signErr:
require.ErrorIs(t, err, context.Canceled)
case <-time.After(10 * time.Second):
t.Fatal("the call waited for the child holding gpg's output to exit")
}
}
// TestBuildPassesContextToSigning checks that a caller can cancel the gpg
// runs that sign a manifest through the context given to Build.
func TestBuildPassesContextToSigning(t *testing.T) {
t.Parallel()
b := NewBuilder()
b.SetSigningOptions(&SigningOptions{KeyID: "any"})
ctx, cancel := context.WithCancel(context.Background())
cancel()
require.ErrorIs(t, b.Build(ctx, io.Discard), context.Canceled)
}
+1 -2
View File
@@ -283,8 +283,7 @@ func (s *Scanner) ToManifest(
}
// Build and write manifest
//nolint:contextcheck // Build's GPG signing exec is not cancellable by design
return builder.Build(w)
return builder.Build(ctx, w)
}
// configureBuilder constructs a manifest builder configured from the
+4 -31
View File
@@ -304,46 +304,19 @@ func TestScannerEnumerateFS(t *testing.T) {
func TestSendEnumerateStatusNonBlocking(t *testing.T) {
t.Parallel()
// Channel with no buffer - send should not block
// Nobody receives, so a blocking send would hang the test into its timeout.
ch := make(chan EnumerateStatus)
// This should not block
done := make(chan bool)
go func() {
sendEnumerateStatus(ch, EnumerateStatus{FilesFound: 1})
done <- true
}()
select {
case <-done:
// Success - did not block
case <-time.After(100 * time.Millisecond):
t.Fatal("sendEnumerateStatus blocked on full channel")
}
sendEnumerateStatus(ch, EnumerateStatus{FilesFound: 1})
}
func TestSendScanStatusNonBlocking(t *testing.T) {
t.Parallel()
// Channel with no buffer - send should not block
// Nobody receives, so a blocking send would hang the test into its timeout.
ch := make(chan ScanStatus)
done := make(chan bool)
go func() {
sendScanStatus(ch, ScanStatus{ScannedFiles: 1})
done <- true
}()
select {
case <-done:
// Success - did not block
case <-time.After(100 * time.Millisecond):
t.Fatal("sendScanStatus blocked on full channel")
}
sendScanStatus(ch, ScanStatus{ScannedFiles: 1})
}
func TestSendStatusNilChannel(t *testing.T) {
+9 -8
View File
@@ -2,6 +2,7 @@ package mfer
import (
"bytes"
"context"
"crypto/sha256"
"errors"
"fmt"
@@ -50,13 +51,13 @@ func newTimestampFromTime(t time.Time) *Timestamp {
}
}
func (m *manifest) generate() error {
func (m *manifest) generate(ctx context.Context) error {
if m.pbInner == nil {
return errInnerNotSet
}
if m.pbOuter == nil {
e := m.generateOuter()
e := m.generateOuter(ctx)
if e != nil {
return e
}
@@ -77,7 +78,7 @@ func (m *manifest) generate() error {
return nil
}
func (m *manifest) generateOuter() error {
func (m *manifest) generateOuter(ctx context.Context) error {
if m.pbInner == nil {
return errInternal
}
@@ -135,7 +136,7 @@ func (m *manifest) generateOuter() error {
// Sign the manifest if signing options are provided
if m.signingOptions != nil && m.signingOptions.KeyID != "" {
return m.signOuter()
return m.signOuter(ctx)
}
return nil
@@ -143,27 +144,27 @@ func (m *manifest) generateOuter() error {
// signOuter signs the outer message with the configured GPG key and
// embeds the signature, signer fingerprint, and public key.
func (m *manifest) signOuter() error {
func (m *manifest) signOuter(ctx context.Context) error {
sigString, err := m.signatureString()
if err != nil {
return fmt.Errorf("failed to generate signature string: %w", err)
}
sig, err := gpgSign([]byte(sigString), m.signingOptions.KeyID)
sig, err := gpgSign(ctx, []byte(sigString), m.signingOptions.KeyID)
if err != nil {
return fmt.Errorf("failed to sign manifest: %w", err)
}
m.pbOuter.Signature = sig
fingerprint, err := gpgGetKeyFingerprint(m.signingOptions.KeyID)
fingerprint, err := gpgGetKeyFingerprint(ctx, m.signingOptions.KeyID)
if err != nil {
return fmt.Errorf("failed to get key fingerprint: %w", err)
}
m.pbOuter.Signer = fingerprint
pubKey, err := gpgExportPublicKey(m.signingOptions.KeyID)
pubKey, err := gpgExportPublicKey(ctx, m.signingOptions.KeyID)
if err != nil {
return fmt.Errorf("failed to export public key: %w", err)
}
+2
View File
@@ -0,0 +1,2 @@
go test fuzz v1
[]byte("")
@@ -0,0 +1,2 @@
go test fuzz v1
[]byte("ZNAVSRFGy[i\x04\xe5O\x82A\x1d\xf4\xb0\xe2z7:U\xee\xa3\xf9\xd6m\xacZ\x9b\xce\x1d\xd9/{@\x1d\xa5y[i\x04\xe5O\x82A\x1d\xf4\xb0\xe2z7:U\xee\xa3\xf9\xd6m\xacZ\x9b\xce\x1d\xd9/{@\x1d\xa5")
+2
View File
@@ -0,0 +1,2 @@
go test fuzz v1
[]byte("ZNAVSRFG\xa8\x06\x01\xb0\x06\x01\xb8\x06V\xc2\x06 \xa3\xf7\x97\xa7\xf3\x87:\x90)\\ӊj\xb9\xf7\xfaTJ\x1b\xe7:\xee\xbe\"V\xe0:\x8d(Z\xd6%\xca\x06\x10\x93\x85\vpu\x85\xe4\x04\xe4\x95\x1a=\xdc\x1f\x05\xa3\xba\fc(\xb5/\xfd\x04\x00\xb1\x02\x00\xa0\x06\x01\xaa\x06=\n\x05a.txt\x10\x01\x1a$\n\"\x12 ʗ\x81\x12\xca\x1b\xbd\xca\xfa\xc21\xb3\x9a#\xdcM\xa7\x86\xef\xf8\x14|Nr\xb9\x80w\x85\xaf\xeeH\xbb\xf2\x12\v\b\x80\x92\xb8Ø\xfe\xff\xff\xff\x01\xb2\x06\x10\x93\x85\vpu\x85\xe4\x04\xe4\x95\x1a=\xdc\x1f\x05\xa3s\xeeG\x80")
+2
View File
@@ -0,0 +1,2 @@
go test fuzz v1
[]byte("ZNAVSRFG\xa8\x06\x01\xb0\x06\x01\xb8\x06V\xc2\x06 \xa3\xf7\x97\xa7\xf3\x87:\x90)\\ӊj\xb9\xf7\xfaTJ\x1b\xe7:\xee\xbe\"V\xe0:\x8d(Z\xd6%\xca\x06\x10\x93\x85\vpu\x85\xe4\x04\xe4\x95\x1a=\xdc\x1f\x05\xa3\xba\fc(\xb5/\xfd\x04\x00\xb1\x02\x00\xa0\x06\x01\xaa\x06=\n\x05a.txt\x10\x01\x1a$\n\"\x12 ʗ\x81\x12\xca\x1b\xbd\xca\xfa\xc21\xb3\x9a#\xdcM\xa7\x86\xef\xf8\x14|Nr\xb9\x80w\x85\xaf\xeeH\xbb\xf2\x12\v\b\x80\x92\xb8Ø\xfe\xff\xff\xff\x01\xb2\x06\x10\x93\x85\vpu\x85\xe4\x04\xe4\x95\x1a=\xdc\x1f\x05\xa3s\xeeG\x80\xca\f\xe8\x03-----BEGIN PGP SIGNATURE-----\n\niQEzBAABCgAdFiEET1Yr+4Y/3GtRtO6IhypRF2zvI64FAmrBIXAACgkQhypRF2zv\nI67BQAf/QrpX2MjY15YGMGkjR5oIhnx/YV96aGYZyZThzb+l/R/N75iVFVkhX21d\nZhQqdCsORrodTPAXic2g2UGVXP9PhNMh7n6Wm3LsvQYjrRQGrQnqtCkut+3tUt8K\n7pt4OAnnwRSieaVImA1COmzxIrQQKNOs6UkgmAstGuPV0XZoeDiSG8TUYJ/vieCn\np5hC0FFXtzfw4NtkxSmkewE0xBxIwFCA/RfSHCGH3m5K+tRz41vMEgGbL1iEp6+V\nuBaoEc4hqCgEt+Af2pA8VHfqeu2vKiwggOpYpaILXZKVqH9+tWHL1EBv9t0vTsYE\n9D57euuR9+kOdngYNPieP1yn5dOSHg==\n=kvgL\n-----END PGP SIGNATURE-----\n\xd2\f(4F562BFB863FDC6B51B4EE88872A51176CEF23AE\xda\f\xb5\a-----BEGIN PGP PUBLIC KEY BLOCK-----\n\nmQENBGrBIW8BCADESetN5EdxIe7Fafgxl99Yoo5cOexf7wJyYT0wfUYlRaxt3neR\nhir7LOfH4PZWWoDx7qghxCS4+vs7yGypl6JOm7jnJlhn4HneDa2zeIlgGW2TamyE\nua9KPWBQqkFOYmKPmzp+KnL6ncnBLR5mDkNKFyON812KVvteu6Dp/DNk4Meufe44\nWWr49LSFZa9gEbmRCoQGKby9F0H0yIi4FAc74VdQudy0+fMKcfkKjEvByMzlbBEK\n92Hq3sRFzWd3kvPliNjZTmlh5n5m9aBhMpoy3GkKy8gpDdFc6NLA9iAJe7oNMriR\nkVoa5EjQL1xCXAiAWTYA9NScFfU/574sCTxZABEBAAG0Hk1GRVIgVGVzdCBLZXkg\nPHRlc3RAbWZlci50ZXN0PokBTwQTAQoAORYhBE9WK/uGP9xrUbTuiIcqURds7yOu\nBQJqwSFvAxsvBAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRCHKlEXbO8jrjml\nCAC8wUK9wmvxq0+NZUpFyP+P29klLZYzBDaBrLPJFs0GjnG4kvfUAktWx0Ro80F7\ncjTJ4f44XjDj4glvSjbe2VaDnZl9FTfzUfG+xjD4462NgntQ4fHk/uG4F6d1ikWx\nkEoMpIn1PlSMas1jTQSGlxUr+zFwWuUbGq4n6hRxEnwLlwJlwQt/Aw1vPDYuPDE3\nOYDhJIAJyP+6e9W8ToaAG9byg/22KA1u1qxnNQqsx5Tped2VltAzdYub+yeCuNc8\nIUo5ILo/fQq3GM5sUEaHjPolv88WlDm3vcdbSbDoh5m2inDtg5zuUKJwu32UGu8l\nKoTjp4nxgQGy5WmeBzs4Hdm/\n=TCB8\n-----END PGP PUBLIC KEY BLOCK-----\n")
@@ -0,0 +1,2 @@
go test fuzz v1
[]byte("ZNAVSRFG\xa8\x06\x01\xb0\x06\x01\xb8\x06!\xc2\x06 \x91\x90*\xa5>\fݐ \x87\xbeaL\xc1\x05?\x0eR\xc18\xa4eՕ\xa95\xb9KʺoZ\xca\x06\x10\x93\x85\vpu\x85\xe4\x04\xe4\x95\x1a=\xdc\x1f\x05\xa3\xba\f-(\xb5/\xfd\x04\x00\x01\x01\x00\xa0\x06\x01\xaa\x06\a\n\x05a.txt\xb2\x06\x10\x93\x85\vpu\x85\xe4\x04\xe4\x95\x1a=\xdc\x1f\x05\xa3a[k'")
@@ -0,0 +1,2 @@
go test fuzz v1
[]byte("ZNAVSRFG\xa8\x06\x01\xb0\x06\x01\xb8\x06\x1f\xc2\x06 \x91\x90*\xa5>\fݐ \x87\xbeaL\xc1\x05?\x0eR\xc18\xa4eՕ\xa95\xb9KʺoZ\xca\x06\x10\x93\x85\vpu\x85\xe4\x04\xe4\x95\x1a=\xdc\x1f\x05\xa3\xba\f-(\xb5/\xfd\x04\x00\x01\x01\x00\xa0\x06\x01\xaa\x06\a\n\x05a.txt\xb2\x06\x10\x93\x85\vpu\x85\xe4\x04\xe4\x95\x1a=\xdc\x1f\x05\xa3a[k'")
@@ -0,0 +1,2 @@
go test fuzz v1
[]byte("ZNAVSRFG\xa8\x06\x01\xb0\x06\x01\xb8\x06V\xc2\x06 \xa3\xf7\x97\xa7\xf3\x87:\x90)\\ӊj\xb9\xf7\xfaTJ\x1b\xe7:\xee\xbe\"V\xe0:\x8d(Z\xd6%\xca\x06\x10\x93\x85\vpu\x85\xe4\x04\xe4\x95\x1a=\xdc\x1f\x05\xa3\xba\fc(\xb5/\xfd\x04\x00\xb1\x02\x00\xa0\x06\x01\xaa\x06=\n\x05a.txt\x10\x01\x1a$\n\"\x12 ʗ\x81\x12\xca\x1b\xbd\xca\xfa\xc21\xb3\x9a#\xdcM\xa7\x86\xef\xf8\x14|Nr\xb9\x80w\x85\xaf\xeeH\xbb\xf2\x12\v\b\x80\x92\xb8Ø\xfe\xff\xff\xff\x01\xb2\x06\x10\x93\x85\vpu\x85\xe4\x04\xe4\x95\x1a=\xdc\x1f\x05\xa3s\xeeG")
@@ -0,0 +1,2 @@
go test fuzz v1
[]byte("ZNAVSRFG\xa8\x06\x01\xb0\x06\x01\xb8\x06V\xc2\x06 ")
@@ -0,0 +1,2 @@
go test fuzz v1
[]byte("ZNAV")
@@ -0,0 +1,2 @@
go test fuzz v1
[]byte("ZNAVSRFG")
@@ -0,0 +1,2 @@
go test fuzz v1
[]byte("ZNAVSRFG\xa8\x06\x01\xb0\x06\x01\xb8\x06V\xc2\x06 \xa3\xf7\x97\xa7\xf3\x87:\x90)\\ӊj\xb9\xf7\xfaTJ\x1b\xe7:\xee\xbe\"V\xe0:\x8d(Z\xd6%\xca\x06\x10\x93\x85\vpu\x85\xe4\x04\xe4\x95\x1a=\xdc\x1f\x05\xa3\xba\fc(\xb5/\xfd\x04\x00\xb1\x02\x00\xa0\x06\x01")
@@ -0,0 +1,2 @@
go test fuzz v1
[]byte("ZNAVSRFX\xa8\x06\x01\xb0\x06\x01\xb8\x06V\xc2\x06 \xa3\xf7\x97\xa7\xf3\x87:\x90)\\ӊj\xb9\xf7\xfaTJ\x1b\xe7:\xee\xbe\"V\xe0:\x8d(Z\xd6%\xca\x06\x10\x93\x85\vpu\x85\xe4\x04\xe4\x95\x1a=\xdc\x1f\x05\xa3\xba\fc(\xb5/\xfd\x04\x00\xb1\x02\x00\xa0\x06\x01\xaa\x06=\n\x05a.txt\x10\x01\x1a$\n\"\x12 ʗ\x81\x12\xca\x1b\xbd\xca\xfa\xc21\xb3\x9a#\xdcM\xa7\x86\xef\xf8\x14|Nr\xb9\x80w\x85\xaf\xeeH\xbb\xf2\x12\v\b\x80\x92\xb8Ø\xfe\xff\xff\xff\x01\xb2\x06\x10\x93\x85\vpu\x85\xe4\x04\xe4\x95\x1a=\xdc\x1f\x05\xa3s\xeeG\x80")
File diff suppressed because one or more lines are too long
@@ -0,0 +1,2 @@
go test fuzz v1
[]byte("ZNAVSRFG\xa8\x06\x01\xb0\x06\x01\xb8\x06\x01\xc2\x06 \xd6aQ\xa4\x85\xc0+\xbb\xca\x11\x11\a<\x019\x97\xb3\xbb3\xd0 \xd5U\xfa!\xaeAf<N@\x9d\xca\x06\x10\x93\x85\vpu\x85\xe4\x04\xe4\x95\x1a=\xdc\x1f\x05\xa3\xba\f\x12(\xb5/\xfd\xc0\x00\x00\x00\x00\x00\x02\x00\x00\x00\v\x00\x00\x00")
@@ -0,0 +1,2 @@
go test fuzz v1
[]byte("ZNAVSRFG\xa8\x06\x01\xb0\x06\x01\xb8\x06\x01\xc2\x06 \xc8g?\xa0\xc9\xc5]\xb57M\xe5O#\x04\xb2\x12\xc6)@=\xd2\xf3f/͉~\x10\x15\xfbkD\xca\x06\x10o\x1c*N;]L~\x9a\x8b\x1d.?@Qb\xba\f\x89\t(\xb5/\xfd\x00\x00\x01\x00\x00(\xb5/\xfd\x00\x01\x01\x00\x00(\xb5/\xfd\x00\x02\x01\x00\x00(\xb5/\xfd\x00\x03\x01\x00\x00(\xb5/\xfd\x00\x04\x01\x00\x00(\xb5/\xfd\x00\x05\x01\x00\x00(\xb5/\xfd\x00\x06\x01\x00\x00(\xb5/\xfd\x00\a\x01\x00\x00(\xb5/\xfd\x00\b\x01\x00\x00(\xb5/\xfd\x00\t\x01\x00\x00(\xb5/\xfd\x00\n\x01\x00\x00(\xb5/\xfd\x00\v\x01\x00\x00(\xb5/\xfd\x00\f\x01\x00\x00(\xb5/\xfd\x00\r\x01\x00\x00(\xb5/\xfd\x00\x0e\x01\x00\x00(\xb5/\xfd\x00\x0f\x01\x00\x00(\xb5/\xfd\x00\x10\x01\x00\x00(\xb5/\xfd\x00\x11\x01\x00\x00(\xb5/\xfd\x00\x12\x01\x00\x00(\xb5/\xfd\x00\x13\x01\x00\x00(\xb5/\xfd\x00\x14\x01\x00\x00(\xb5/\xfd\x00\x15\x01\x00\x00(\xb5/\xfd\x00\x16\x01\x00\x00(\xb5/\xfd\x00\x17\x01\x00\x00(\xb5/\xfd\x00\x18\x01\x00\x00(\xb5/\xfd\x00\x19\x01\x00\x00(\xb5/\xfd\x00\x1a\x01\x00\x00(\xb5/\xfd\x00\x1b\x01\x00\x00(\xb5/\xfd\x00\x1c\x01\x00\x00(\xb5/\xfd\x00\x1d\x01\x00\x00(\xb5/\xfd\x00\x1e\x01\x00\x00(\xb5/\xfd\x00\x1f\x01\x00\x00(\xb5/\xfd\x00 \x01\x00\x00(\xb5/\xfd\x00!\x01\x00\x00(\xb5/\xfd\x00\"\x01\x00\x00(\xb5/\xfd\x00#\x01\x00\x00(\xb5/\xfd\x00$\x01\x00\x00(\xb5/\xfd\x00%\x01\x00\x00(\xb5/\xfd\x00&\x01\x00\x00(\xb5/\xfd\x00'\x01\x00\x00(\xb5/\xfd\x00(\x01\x00\x00(\xb5/\xfd\x00)\x01\x00\x00(\xb5/\xfd\x00*\x01\x00\x00(\xb5/\xfd\x00+\x01\x00\x00(\xb5/\xfd\x00,\x01\x00\x00(\xb5/\xfd\x00-\x01\x00\x00(\xb5/\xfd\x00.\x01\x00\x00(\xb5/\xfd\x00/\x01\x00\x00(\xb5/\xfd\x000\x01\x00\x00(\xb5/\xfd\x001\x01\x00\x00(\xb5/\xfd\x002\x01\x00\x00(\xb5/\xfd\x003\x01\x00\x00(\xb5/\xfd\x004\x01\x00\x00(\xb5/\xfd\x005\x01\x00\x00(\xb5/\xfd\x006\x01\x00\x00(\xb5/\xfd\x007\x01\x00\x00(\xb5/\xfd\x008\x01\x00\x00(\xb5/\xfd\x009\x01\x00\x00(\xb5/\xfd\x00:\x01\x00\x00(\xb5/\xfd\x00;\x01\x00\x00(\xb5/\xfd\x00<\x01\x00\x00(\xb5/\xfd\x00=\x01\x00\x00(\xb5/\xfd\x00>\x01\x00\x00(\xb5/\xfd\x00?\x01\x00\x00(\xb5/\xfd\x00@\x01\x00\x00(\xb5/\xfd\x00A\x01\x00\x00(\xb5/\xfd\x00B\x01\x00\x00(\xb5/\xfd\x00C\x01\x00\x00(\xb5/\xfd\x00D\x01\x00\x00(\xb5/\xfd\x00E\x01\x00\x00(\xb5/\xfd\x00F\x01\x00\x00(\xb5/\xfd\x00G\x01\x00\x00(\xb5/\xfd\x00H\x01\x00\x00(\xb5/\xfd\x00I\x01\x00\x00(\xb5/\xfd\x00J\x01\x00\x00(\xb5/\xfd\x00K\x01\x00\x00(\xb5/\xfd\x00L\x01\x00\x00(\xb5/\xfd\x00M\x01\x00\x00(\xb5/\xfd\x00N\x01\x00\x00(\xb5/\xfd\x00O\x01\x00\x00(\xb5/\xfd\x00P\x01\x00\x00(\xb5/\xfd\x00Q\x01\x00\x00(\xb5/\xfd\x00R\x01\x00\x00(\xb5/\xfd\x00S\x01\x00\x00(\xb5/\xfd\x00T\x01\x00\x00(\xb5/\xfd\x00U\x01\x00\x00(\xb5/\xfd\x00V\x01\x00\x00(\xb5/\xfd\x00W\x01\x00\x00(\xb5/\xfd\x00X\x01\x00\x00(\xb5/\xfd\x00Y\x01\x00\x00(\xb5/\xfd\x00Z\x01\x00\x00(\xb5/\xfd\x00[\x01\x00\x00(\xb5/\xfd\x00\\\x01\x00\x00(\xb5/\xfd\x00]\x01\x00\x00(\xb5/\xfd\x00^\x01\x00\x00(\xb5/\xfd\x00_\x01\x00\x00(\xb5/\xfd\x00`\x01\x00\x00(\xb5/\xfd\x00a\x01\x00\x00(\xb5/\xfd\x00b\x01\x00\x00(\xb5/\xfd\x00c\x01\x00\x00(\xb5/\xfd\x00d\x01\x00\x00(\xb5/\xfd\x00e\x01\x00\x00(\xb5/\xfd\x00f\x01\x00\x00(\xb5/\xfd\x00g\x01\x00\x00(\xb5/\xfd\x00h\x01\x00\x00(\xb5/\xfd\x00i\x01\x00\x00(\xb5/\xfd\x00j\x01\x00\x00(\xb5/\xfd\x00k\x01\x00\x00(\xb5/\xfd\x00l\x01\x00\x00(\xb5/\xfd\x00m\x01\x00\x00(\xb5/\xfd\x00n\x01\x00\x00(\xb5/\xfd\x00o\x01\x00\x00(\xb5/\xfd\x00p\x01\x00\x00(\xb5/\xfd\x00q\x01\x00\x00(\xb5/\xfd\x00r\x01\x00\x00(\xb5/\xfd\x00s\x01\x00\x00(\xb5/\xfd\x00t\x01\x00\x00(\xb5/\xfd\x00u\x01\x00\x00(\xb5/\xfd\x00v\x01\x00\x00(\xb5/\xfd\x00w\x01\x00\x00(\xb5/\xfd\x00x\x01\x00\x00(\xb5/\xfd\x00y\x01\x00\x00(\xb5/\xfd\x00z\x01\x00\x00(\xb5/\xfd\x00{\x01\x00\x00(\xb5/\xfd\x00|\x01\x00\x00(\xb5/\xfd\x00}\x01\x00\x00(\xb5/\xfd\x00~\x01\x00\x00(\xb5/\xfd\x00\x7f\x01\x00\x00(\xb5/\xfd\x00\x80\x01\x00\x00")
File diff suppressed because one or more lines are too long
+1 -6
View File
@@ -140,12 +140,7 @@ main() {
# ---- Go repos ----
if missing go; then pkg_install go golang go go; fi
# golangci-lint: packaged in nix, brew, and apk. On apt there is no
# package: download a specific release archive from GitHub and
# verify its hash (verify_sha256), never curl | sh.
if missing golangci-lint; then
pkg_install golangci-lint golangci-lint golangci-lint golangci-lint
fi
# No golangci-lint: script/lint runs it in Docker only.
go mod download
# ---- Python repos ----
-1
View File
@@ -19,7 +19,6 @@ main() {
cd "$ROOT"
ensure_pb
gofumpt -l -w mfer internal cmd
golangci-lint run --fix
# Markdown and JSON, over the same file set script/fmt-check verifies.
"$SCRIPT_DIR/prettier" --write
}
Executable
+17
View File
@@ -0,0 +1,17 @@
#!/bin/sh
# script/fuzz: fuzz the manifest parser for one minute. Run by hand only:
# script/test already runs the committed seed corpus as ordinary tests,
# and CI never fuzzes. An input that fails is written to
# mfer/testdata/fuzz/FuzzNewManifestFromReader/; once the parser is fixed,
# commit it there as a regression seed.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
go test -run '^$' -fuzz '^FuzzNewManifestFromReader$' \
-fuzztime 1m -parallel 2 ./mfer
}
main "$@"
+11 -8
View File
@@ -1,17 +1,20 @@
#!/bin/sh
# script/lint: run the linter.
# script/lint: run golangci-lint, in Docker only. Builds the lint stage of
# the Dockerfile, whose build runs the linter, so a successful build is a
# clean lint. --no-cache because a cached build runs no linter. The image
# is removed afterwards, whatever the outcome.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
golangci-lint run
if [ -n "$(gofmt -l .)" ]; then
echo "gofmt: files need formatting:" >&2
gofmt -l . >&2
exit 1
fi
# Tagged per run, so concurrent runs never remove each other's image.
image="$("$SCRIPT_DIR/projectname")-lint:$$"
# A failed build leaves no image, so there is nothing to remove then.
trap 'docker image rm "$image" >/dev/null 2>&1 || true' EXIT INT TERM
docker build --no-cache --target lint -t "$image" .
}
main "$@"