Author SHA1 Message Date
sneak ee49371459 Enforce real timeouts on gpg subprocess calls (closes #62)
check / check (push) Successful in 55s
Every gpg run now has a one-minute deadline (gpgTimeout) on top of its
caller's context and is killed when either ends. A timeout is reported
as "gpg timed out" under the failing operation instead of "signal:
killed". Builder.Build and Checker.ExtractEmbeddedSigningKeyFP take a
context, so ToManifest's context now reaches signing and the
contextcheck suppression calling signing non-cancellable is gone.
Manifest loading takes no context, so its signature check is bounded by
the timeout alone. Killing gpg itself is enough: the gpg-agent it starts
runs detached and holds none of its output.

Model: opus-5-5
2026-10-03 15:37:23 +00:00
clawbot fa97c4519c Never write fetch's temp file into an existing file (closes #115)
check / check (push) Successful in 53s
downloadFile opened the temp file with os.Create, which opens and
empties a file already at that name. If that file was a hard link to a
file outside the destination directory, fetch overwrote the outside
file. It now removes whatever is at the temp name, which removes only
that name, and creates the temp file with O_EXCL, so the create fails if
anything is still or again there. A leftover temp file from an
interrupted run is still replaced. The new test puts a hard link at the
temp name and checks that fetch succeeds and the outside file is
unchanged. The command name is now the constant cmdFetch, like the other
command names, because lint requires it once a third test uses it.

Model: opus-5-5
2026-10-03 16:58:35 +02:00
clawbot 7e601929c8 Refuse fetch writes through a symlink in the destination (closes #86)
check / check (push) Successful in 1m10s
sanitizePath checks manifest paths only as text, so a symlink already
inside the destination directory could send fetch's writes outside it.
checkNoSymlinks now looks at each existing part of a path with os.Lstat
and refuses the path if any part is a symlink, wherever it points.
fetch runs it immediately before each write: creating the parent
directories, creating the temp file, and renaming it into place. The new
test puts such a symlink at each of those three places, and once inside
a plain directory, and checks that the fetch fails and nothing outside
changes. The G304 comment now states what holds. A symlink swapped in
between a check and its write is not caught; os.Root closes that once
the Go version is raised.

Model: opus-5-5
2026-10-03 16:08:09 +02:00
clawbot c3b5fe651a Stamp the tag or short commit in a plain docker build (closes #112)
check / check (push) Successful in 48s
.dockerignore now sends .git but not .git/config, which can hold a
credential and which git describe does not need. The build stage stamps
main.Gitrev from the VERSION build argument when one is given, otherwise
from git describe --tags --always, and fails if .git is present and no
version comes out. git there trusts /src whoever owns it, since a
context sent as a tar archive keeps its files' owners. script/docker is
replaced by the canonical copy, which passes VERSION; bin/gitrev.sh uses
--tags too, so every entrypoint stamps the same value for a clean
commit.

Model: opus-5-5
2026-10-02 08:53:34 +02:00
clawbot 0deacfc7ed Validate manifest entry paths on deserialize (closes #61)
check / check (push) Failing after 1s
Untrusted .mf files were parsed with no path validation, so an entry like ../../etc/passwd reached filepath.Join against the checker's base path and mfer check could stat and read outside it. ValidatePath ran only when building a manifest. It now runs on every entry as the manifest loads, so every consumer is covered. The whole manifest is rejected on the first bad entry instead of dropping it, which could hide files from a check; the error wraps errInvalidManifestPath and names the path.

Disclosure: a path that is not valid UTF-8 is refused earlier, by the protobuf string decoder, so that error does not name the path.

Model: opus-4-8 (implementation); fable-5-1 (summary)
2026-09-22 00:47:26 +02:00
clawbot 7de4d6ec1c Rewrite script/test to the canonical race-enabled pattern (closes #67)
check / check (push) Successful in 51s
script/test now runs go test -timeout 30s -race -cover ./..., quiet on success and rerunning verbose on failure. -race exposed a data race on the process-global apex/log logger: Init reconfigured it on every CLI run while other goroutines logged through it. internal/log now mutates the global under the write lock and reads it under the read lock; WithError is dropped because its Entry logged outside that lock, and run() reports a failed command's error via log.Errorf instead (still shown under -q). The corruption fuzz test is scaled to 1500 files / 100 iterations to fit the budget under -race; it pins the same claims at reduced breadth. Independent review ran the Docker lint gate uncached.

Model: opus-4-8 (implementation, review)
model: claude-fable-5
2026-09-21 15:17:50 +02:00
clawbot 6de3f1d714 Add .editorconfig, broaden .gitignore, drop dead Drone refs (closes #72)
check / check (push) Failing after 1s
Add the canonical .editorconfig from sneak/prompts verbatim. Broaden .gitignore to cover secrets (.env, .env.*, *.key, *.pem), OS files, editor files, and Go artifacts while keeping every repo-specific entry; no tracked file becomes ignored, and bin/gitrev.sh stays tracked despite the /bin/ rule. Remove the stale .drone.yml ignore entry and the DRONE_COMMIT_SHA branch from bin/gitrev.sh, left from the Gitea Actions migration; the GITREV and git-describe paths still work.

A reader may trip over .editorconfig saying four spaces for every file while Go files are tab-indented: gofmt governs Go, editorconfig does not, and the file is taken verbatim by policy.

Disclosure: the worker's cold docker build hit the 10s test timeout in the gpg signature test; this change has no Go code, and that timeout is tracked by issues 67 and 62.

Model: opus-4-8 (implementation); fable-5-1 (merge)
2026-09-21 09:56:15 +02:00
26 changed files with 664 additions and 238 deletions
+62 -2
View File
@@ -1,4 +1,64 @@
# .dockerignore does NOT use .gitignore semantics. Docker matches with
# moby/patternmatcher: filepath.Match plus `**`, so `*` does not cross
# `/` and an unprefixed pattern is anchored at the context root. Every
# depth-independent pattern therefore needs `**/`, or `config/.env` and
# `certs/server.key` still ship while this file reads as solved. Only
# genuinely root-anchored entries go unprefixed. Never transplant these
# into .gitignore, where `**/` is wrong.
#
# Matching is case-sensitive, so secrets use character ranges rather
# than an ALL-CAPS twin, which would still miss `Server.Key`.
#
# Extend with this repo's own host-built artifacts, written anchored:
# `/myapp`, never `**/myapp`, which also matches `cmd/myapp/` and
# deletes the package directory from the context.
# .git is sent without its config. Without a VERSION build argument the
# stage that compiles runs `git describe --tags --always` on .git, which
# does not need .git/config; that file can hold a credential, such as a
# password in a remote URL or the token the CI checkout step stores there.
.git/config
# Agent scratch: one full checkout of the repo per in-flight agent.
# Anchored because it occurs once where agents run at the repo root.
# KNOWN GAP: a repo running agents in subdirectories still ships
# `services/api/.claude/` and must add its own anchored entry.
.claude
# Environment files. `*.env` covers bare `.env` and the `prod.env`
# convention. Re-include a committed template with a negation if the
# build needs one: `!docs/example.env`.
**/*.[eE][nN][vV]
**/.[eE][nN][vV].*
**/.[eE][nN][vV][rR][cC]
# Private keys and the bundles carrying them. Public certificates
# (*.crt, *.cer) are deliberately absent: they are legitimate inputs.
**/*.[pP][eE][mM]
**/*.[kK][eE][yY]
**/*.[pP]12
**/*.[pP][fF][xX]
**/[iI][dD]_[rR][sS][aA]
**/[iI][dD]_[dD][sS][aA]
**/[iI][dD]_[eE][cC][dD][sS][aA]
**/[iI][dD]_[eE][dD]25519
# Dependencies: restored inside the image, never copied in.
**/node_modules
# OS metadata.
**/.DS_Store
**/Thumbs.db
# Editor state: never a build input, and it churns COPY.
**/*.swp
**/*.swo
**/*~
**/*.bak
**/.idea
**/.vscode
**/*.sublime-*
# This repo's own host-built archives (Makefile).
*.tmp *.tmp
*.dockerimage *.dockerimage
.git
node_modules
+12
View File
@@ -0,0 +1,12 @@
root = true
[*]
indent_style = space
indent_size = 4
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true
[Makefile]
indent_style = tab
+21 -2
View File
@@ -10,5 +10,24 @@ modcache.tzst
# Generated manifest files # Generated manifest files
.index.mf .index.mf
# Stale files # Secrets
.drone.yml .env
.env.*
*.key
*.pem
# OS files
.DS_Store
Thumbs.db
# Editor files
*.swp
*.swo
*~
.idea/
.vscode/
# Go build artifacts
*.log
*.out
*.test
+20 -1
View File
@@ -48,7 +48,26 @@ COPY . .
RUN touch mfer/mf.pb.go RUN touch mfer/mf.pb.go
RUN make test RUN make test
RUN cd cmd/mfer && go build -tags urfave_cli_no_docs -o /mfer .
# A build context sent as a tar archive, as upaas sends it, keeps its files'
# owners, and git refuses to read a checkout owned by another user.
RUN git config --system --add safe.directory /src
# The revision `mfer version` prints, stamped into main.Gitrev: the VERSION
# build argument when one is given (script/docker passes one), otherwise
# `git describe --tags --always` of the .git the build context carries: the
# tag on a tagged commit, tag-N-gHASH on a commit after one, the short commit
# when no tag is reachable. git ships in this base image. A context that
# carries .git and still yields no version fails the build.
ARG VERSION
RUN version="${VERSION:-$(git describe --tags --always)}"; \
if [ -e .git ] && { [ -z "$version" ] || [ "$version" = dev ] || \
[ "$version" = unknown ]; }; then \
echo "no version could be derived although the build context carries .git" >&2; \
exit 1; \
fi; \
cd cmd/mfer && \
go build -tags urfave_cli_no_docs -ldflags "-X main.Gitrev=$version" -o /mfer .
FROM scratch FROM scratch
COPY --from=builder /mfer /mfer COPY --from=builder /mfer /mfer
+26 -5
View File
@@ -24,8 +24,31 @@ only thing left of the `chore/align-repo-policies` branch is the list below.
# Completed Steps # Completed Steps
- 2026-09-21: fixed the `-v` collision between `--verbose` and `--version`; - 2026-10-03: every gpg run is killed after one minute or when its caller's
verbose owns `-v`, and version answers to `--version` and `-V` (#64) context ends, and a timeout reads as "gpg timed out" under the failing
operation; `Builder.Build` and `Checker.ExtractEmbeddedSigningKeyFP` take a
context, which reaches gpg (#62)
- 2026-10-03: `fetch` removes whatever sits at a file's temp name and then
creates the temp file only if that name is free, so a hard link left there
cannot make it write into a file outside the destination directory (#115)
- 2026-10-03: `fetch` refuses any manifest path that runs through a symlink
already in the destination directory, checked before each of its writes
(directories, temp file, rename), so such a symlink cannot send a write
outside it (#86)
- 2026-10-02: a plain `docker build .` of a clone now stamps the tag or short
commit into `mfer version` instead of nothing: `.dockerignore` sends `.git`
(not `.git/config`), and the build stage takes the `VERSION` build argument,
otherwise `git describe --tags --always`, failing if `.git` is present and no
version comes out. `script/docker` is the canonical copy, which passes
`VERSION`; `bin/gitrev.sh` uses `--tags` too (#112)
- 2026-09-21: validate manifest entry paths on deserialize so untrusted `.mf`
files cannot make `Checker` stat or read outside `basePath` (#61)
- 2026-09-21: rewrote `script/test` to the canonical pattern (30s timeout,
`-race -cover`, quiet-first with verbose-on-failure rerun) and fixed the
process-global logger data race it surfaced (#67)
- 2026-09-21: added the canonical `.editorconfig`, made `.gitignore` cover
secrets, OS, editor, and Go artifacts, and removed the dead Drone CI
references from `.gitignore` and `bin/gitrev.sh` (#72)
- 2026-08-09: added `.prettierrc`/`.prettierignore`, gave `script/fmt` and - 2026-08-09: added `.prettierrc`/`.prettierignore`, gave `script/fmt` and
`script/fmt-check` one shared prettier file set via `script/prettier`, dropped `script/fmt-check` one shared prettier file set via `script/prettier`, dropped
the `|| true` that hid prettier failures, and added a node-based Dockerfile the `|| true` that hid prettier failures, and added a node-based Dockerfile
@@ -64,7 +87,6 @@ only thing left of the `chore/align-repo-policies` branch is the list below.
- Move FORMAT.md from repo root to docs/ and update the AGENTS.md reference - Move FORMAT.md from repo root to docs/ and update the AGENTS.md reference
- Pin Makefile-installed Go tools (`protoc-gen-go@v1.28.1`, - Pin Makefile-installed Go tools (`protoc-gen-go@v1.28.1`,
`golangci-lint@v2.12.2`) by module hash, not mutable tag `golangci-lint@v2.12.2`) by module hash, not mutable tag
- Set `make test` timeout to 30s (currently 10s)
- Add explicit README "Rationale" heading (content exists under other - Add explicit README "Rationale" heading (content exists under other
names); name the author in the README Description first line names); name the author in the README Description first line
- Reconcile root-level AGENTS.md with directory-hygiene policy (keep or - Reconcile root-level AGENTS.md with directory-hygiene policy (keep or
@@ -94,8 +116,7 @@ only thing left of the `chore/align-repo-policies` branch is the list below.
- Add decompression size limit via io.LimitReader in deserializeInner() - Add decompression size limit via io.LimitReader in deserializeInner()
- Fix errors.Is dead code in checker; make AddFile verify totalRead == size - Fix errors.Is dead code in checker; make AddFile verify totalRead == size
- Export manifest type or define a public interface (pending) - Export manifest type or define a public interface (pending)
- Replace GPG subprocess with pure-Go crypto (pending); add timeouts to - Replace GPG subprocess with pure-Go crypto (pending)
remaining subprocess calls
- CLI: - CLI:
- Kebab-case primary flag names; fix fetch URL construction with - Kebab-case primary flag names; fix fetch URL construction with
url.JoinPath; add http.Client timeout and retry with backoff to fetch; url.JoinPath; add http.Client timeout and retry with backoff to fetch;
+1 -6
View File
@@ -1,12 +1,7 @@
#!/bin/bash #!/bin/bash
# #
if [[ ! -z "$DRONE_COMMIT_SHA" ]]; then
echo "${DRONE_COMMIT_SHA:0:7}"
exit 0
fi
if [[ ! -z "$GITREV" ]]; then if [[ ! -z "$GITREV" ]]; then
echo $GITREV echo $GITREV
else else
git describe --always --dirty=-dirty git describe --tags --always --dirty=-dirty
fi fi
+6 -3
View File
@@ -2,6 +2,7 @@
package cli package cli
import ( import (
"context"
"encoding/hex" "encoding/hex"
"errors" "errors"
"fmt" "fmt"
@@ -126,7 +127,9 @@ func (mfa *CLIApp) fetchManifestToTemp(url string) (string, error) {
// verifyRequiredSigner enforces the --require-signature fingerprint // verifyRequiredSigner enforces the --require-signature fingerprint
// against the manifest's embedded signing key. // against the manifest's embedded signing key.
func verifyRequiredSigner(chk *mfer.Checker, requiredSigner string) error { func verifyRequiredSigner(
ctx context.Context, chk *mfer.Checker, requiredSigner string,
) error {
// Validate fingerprint format: must be exactly 40 hex characters // Validate fingerprint format: must be exactly 40 hex characters
if len(requiredSigner) != fingerprintHexLen { if len(requiredSigner) != fingerprintHexLen {
return fmt.Errorf("%w, got %d", errInvalidFingerprint, len(requiredSigner)) return fmt.Errorf("%w, got %d", errInvalidFingerprint, len(requiredSigner))
@@ -145,7 +148,7 @@ func verifyRequiredSigner(chk *mfer.Checker, requiredSigner string) error {
// Extract fingerprint from the embedded public key (not from the // Extract fingerprint from the embedded public key (not from the
// signer field). This validates the key is importable and gets its // signer field). This validates the key is importable and gets its
// actual fingerprint. // actual fingerprint.
embeddedFP, err := chk.ExtractEmbeddedSigningKeyFP() embeddedFP, err := chk.ExtractEmbeddedSigningKeyFP(ctx)
if err != nil { if err != nil {
return fmt.Errorf( return fmt.Errorf(
"failed to extract fingerprint from embedded signing key: %w", err) "failed to extract fingerprint from embedded signing key: %w", err)
@@ -303,7 +306,7 @@ func (mfa *CLIApp) checkManifestOperation(ctx *cli.Context) error {
// Check signature requirement // Check signature requirement
requiredSigner := ctx.String("require-signature") requiredSigner := ctx.String("require-signature")
if requiredSigner != "" { if requiredSigner != "" {
err = verifyRequiredSigner(chk, requiredSigner) err = verifyRequiredSigner(ctx.Context, chk, requiredSigner)
if err != nil { if err != nil {
return err return err
} }
+12 -74
View File
@@ -27,7 +27,6 @@ const (
testManifest = "/manifest.mf" testManifest = "/manifest.mf"
testFlagBase = "--base" testFlagBase = "--base"
testFlagNoExtra = "--no-extra-files" testFlagNoExtra = "--no-extra-files"
testFlagVersion = "--version"
) )
var errSimulatedWrite = errors.New("simulated write failure") var errSimulatedWrite = errors.New("simulated write failure")
@@ -103,7 +102,7 @@ func TestVersionCommand(t *testing.T) {
t.Parallel() t.Parallel()
fs := afero.NewMemMapFs() fs := afero.NewMemMapFs()
opts := testOpts([]string{testApp, cmdVersion}, fs) opts := testOpts([]string{testApp, "version"}, fs)
exitCode := runCLI(opts) exitCode := runCLI(opts)
@@ -114,69 +113,6 @@ func TestVersionCommand(t *testing.T) {
assert.Contains(t, stdout, "abc123") assert.Contains(t, stdout, "abc123")
} }
// TestVFlagCollision covers the -v/--verbose vs --version flag interaction
// (issue #64). Verbose owns -v; version answers to --version and -V. None of
// these invocations may produce a parser error, and the two ways of asking
// for the version must print the same thing.
func TestVFlagCollision(t *testing.T) {
t.Parallel()
// Invocations that must print the version and exit 0.
versionCases := map[string][]string{
"long version flag": {testApp, testFlagVersion},
"short version flag": {testApp, "-V"},
"verbose then version": {testApp, "-v", testFlagVersion},
"long verbose and version": {testApp, "--verbose", testFlagVersion},
}
for name, args := range versionCases {
t.Run(name, func(t *testing.T) {
t.Parallel()
opts := testOpts(args, afero.NewMemMapFs())
exitCode := runCLI(opts)
assert.Equal(t, 0, exitCode, "stderr: %s", testStderr(t, opts))
assert.Contains(t, testStdout(t, opts), mfer.Version)
assert.NotContains(t, testStderr(t, opts), "two forms of the same flag")
})
}
// Invocations that must enable verbose and exit 0 without a parser error.
verboseCases := map[string][]string{
"short verbose flag": {testApp, "-v"},
"long verbose flag": {testApp, "--verbose"},
}
for name, args := range verboseCases {
t.Run(name, func(t *testing.T) {
t.Parallel()
opts := testOpts(args, afero.NewMemMapFs())
exitCode := runCLI(opts)
assert.Equal(t, 0, exitCode, "stderr: %s", testStderr(t, opts))
assert.Contains(t, testStdout(t, opts), cmdGenerate,
"root should show help listing subcommands")
assert.Empty(t, testStderr(t, opts))
})
}
}
// TestVersionFlagAndCommandMatch asserts that "mfer --version" and
// "mfer version" produce identical output (issue #64).
func TestVersionFlagAndCommandMatch(t *testing.T) {
t.Parallel()
flagOpts := testOpts([]string{testApp, testFlagVersion}, afero.NewMemMapFs())
require.Equal(t, 0, runCLI(flagOpts))
cmdOpts := testOpts([]string{testApp, cmdVersion}, afero.NewMemMapFs())
require.Equal(t, 0, runCLI(cmdOpts))
assert.Equal(t, testStdout(t, flagOpts), testStdout(t, cmdOpts))
}
func TestHelpCommand(t *testing.T) { func TestHelpCommand(t *testing.T) {
t.Parallel() t.Parallel()
@@ -190,7 +126,7 @@ func TestHelpCommand(t *testing.T) {
stdout := testStdout(t, opts) stdout := testStdout(t, opts)
assert.Contains(t, stdout, cmdGenerate) assert.Contains(t, stdout, cmdGenerate)
assert.Contains(t, stdout, cmdCheck) assert.Contains(t, stdout, cmdCheck)
assert.Contains(t, stdout, "fetch") assert.Contains(t, stdout, cmdFetch)
} }
func TestGenerateCommand(t *testing.T) { func TestGenerateCommand(t *testing.T) {
@@ -743,11 +679,13 @@ func TestCheckDetectsManifestCorruption(t *testing.T) {
fs := afero.NewMemMapFs() fs := afero.NewMemMapFs()
rng := rand.New(rand.NewSource(42)) //nolint:gosec // deterministic test data rng := rand.New(rand.NewSource(42)) //nolint:gosec // deterministic test data
// Create many small files with random names to generate a ~1MB manifest // Create many small files with random names so the manifest has many
// Each manifest entry is roughly 50-60 bytes, so we need ~20000 files // entries and random single-byte flips land at varied offsets. Each
// manifest entry is roughly 50-60 bytes. Kept modest so the suite stays
// within its wall-clock budget under -race.
require.NoError(t, fs.MkdirAll(testDir, 0o755)) require.NoError(t, fs.MkdirAll(testDir, 0o755))
numFiles := 20000 numFiles := 1500
for range numFiles { for range numFiles {
// Generate random filename // Generate random filename
filename := fmt.Sprintf("/testdir/%08x%08x%08x.dat", filename := fmt.Sprintf("/testdir/%08x%08x%08x.dat",
@@ -763,11 +701,11 @@ func TestCheckDetectsManifestCorruption(t *testing.T) {
exitCode := runCLI(opts) exitCode := runCLI(opts)
require.Equal(t, 0, exitCode, "generate should succeed") require.Equal(t, 0, exitCode, "generate should succeed")
// Read the valid manifest and verify it's approximately 1MB // Read the valid manifest and verify it has real size.
validManifest, err := afero.ReadFile(fs, testManifest) validManifest, err := afero.ReadFile(fs, testManifest)
require.NoError(t, err) require.NoError(t, err)
require.GreaterOrEqual(t, len(validManifest), 1024*1024, require.GreaterOrEqual(t, len(validManifest), 64*1024,
"manifest should be at least 1MB, got %d bytes", len(validManifest)) "manifest should be at least 64KB, got %d bytes", len(validManifest))
t.Logf("manifest size: %d bytes (%d files)", len(validManifest), numFiles) t.Logf("manifest size: %d bytes (%d files)", len(validManifest), numFiles)
// First corruption: truncate the manifest // First corruption: truncate the manifest
@@ -790,8 +728,8 @@ func TestCheckDetectsManifestCorruption(t *testing.T) {
exitCode = runCLI(opts) exitCode = runCLI(opts)
require.Equal(t, 0, exitCode, "check should pass with valid manifest") require.Equal(t, 0, exitCode, "check should pass with valid manifest")
// Now do 500 random corruption iterations // Now do 100 random corruption iterations
for i := range 500 { for i := range 100 {
// Corrupt: write a random byte at a random offset // Corrupt: write a random byte at a random offset
corrupted := make([]byte, len(validManifest)) corrupted := make([]byte, len(validManifest))
copy(corrupted, validManifest) copy(corrupted, validManifest)
+66 -9
View File
@@ -36,6 +36,11 @@ const (
// traversal bit for group and other must stay set. // traversal bit for group and other must stay set.
dirPerms os.FileMode = 0o755 dirPerms os.FileMode = 0o755
// filePerms is the permission mode, before the umask, for downloaded
// files. It is the mode os.Create uses; like dirPerms, it keeps group
// and other read access.
filePerms os.FileMode = 0o666
// Bitrate unit thresholds in bits per second. // Bitrate unit thresholds in bits per second.
bpsPerGbps = 1e9 bpsPerGbps = 1e9
bpsPerMbps = 1e6 bpsPerMbps = 1e6
@@ -53,6 +58,9 @@ var (
// errPathTraversal indicates a manifest path escaping the target // errPathTraversal indicates a manifest path escaping the target
// directory. // directory.
errPathTraversal = errors.New("path traversal not allowed") errPathTraversal = errors.New("path traversal not allowed")
// errSymlinkInPath indicates a manifest path running through a
// symlink that already exists in the target directory.
errSymlinkInPath = errors.New("symlink in path not allowed")
// errSizeMismatch indicates a downloaded file with an unexpected // errSizeMismatch indicates a downloaded file with an unexpected
// size. // size.
errSizeMismatch = errors.New("size mismatch") errSizeMismatch = errors.New("size mismatch")
@@ -274,6 +282,35 @@ func sanitizePath(p string) (string, error) {
return cleaned, nil return cleaned, nil
} }
// checkNoSymlinks returns an error if any part of the relative path p
// already exists as a symlink. sanitizePath checks p only as text, so
// without this a symlink inside the target directory could send a write
// to p outside of it. Parts that do not exist yet are fine: fetch creates
// them as plain directories and files. Call it immediately before each
// write: a symlink created after it returns is not caught.
func checkNoSymlinks(p string) error {
current := ""
for _, part := range strings.Split(p, string(filepath.Separator)) {
current = filepath.Join(current, part)
info, err := os.Lstat(current)
if errors.Is(err, os.ErrNotExist) {
return nil
}
if err != nil {
return fmt.Errorf("failed to check %s for a symlink: %w", current, err)
}
if info.Mode()&os.ModeSymlink != 0 {
return fmt.Errorf("%w: %s", errSymlinkInPath, current)
}
}
return nil
}
// resolveManifestURL takes a URL and returns the manifest URL. // resolveManifestURL takes a URL and returns the manifest URL.
// If the URL already ends with .mf, it's returned as-is. // If the URL already ends with .mf, it's returned as-is.
// Otherwise, index.mf is appended. // Otherwise, index.mf is appended.
@@ -419,7 +456,12 @@ func downloadFile(
// Create parent directories if needed // Create parent directories if needed
dir := filepath.Dir(localPath) dir := filepath.Dir(localPath)
if dir != "" && dir != "." { if dir != "" && dir != "." {
err := os.MkdirAll(dir, dirPerms) err = checkNoSymlinks(dir)
if err != nil {
return err
}
err = os.MkdirAll(dir, dirPerms)
if err != nil { if err != nil {
return fmt.Errorf("failed to create directory %s: %w", dir, err) return fmt.Errorf("failed to create directory %s: %w", dir, err)
} }
@@ -447,15 +489,25 @@ func downloadFile(
totalBytes = expectedSize totalBytes = expectedSize
} }
// Create temp file. err = checkNoSymlinks(tmpPath)
if err != nil {
return err
}
// Remove whatever is at tmpPath, such as a leftover from an
// interrupted run, rather than write into it: it may be a hard link
// to a file outside the target directory, and removing a hard link
// removes only this name. If the removal fails, O_EXCL below makes
// the create fail.
_ = os.Remove(tmpPath)
// Create the temp file only if nothing is at tmpPath (O_EXCL).
// //
// G304: tmpPath is derived from localPath, which sanitizePath above // G304: tmpPath is a relative path that sanitizePath keeps inside the
// constrains lexically to a relative path that does not escape the // target directory as text, and checkNoSymlinks just found no symlink
// destination directory. That is a purely lexical guarantee: it does // in it.
// not resolve symlinks, so a pre-existing symlink inside the out, err := os.OpenFile( //nolint:gosec // G304: see comment above
// destination tree can still redirect this write outside of it tmpPath, os.O_RDWR|os.O_CREATE|os.O_EXCL, filePerms)
// (tracked in issue #86).
out, err := os.Create(tmpPath) //nolint:gosec // G304: see comment above
if err != nil { if err != nil {
return fmt.Errorf("failed to create temp file: %w", err) return fmt.Errorf("failed to create temp file: %w", err)
} }
@@ -519,6 +571,11 @@ func finishDownload(
return err return err
} }
err = checkNoSymlinks(localPath)
if err != nil {
return err
}
// Rename temp file to final path // Rename temp file to final path
err = os.Rename(tmpPath, localPath) err = os.Rename(tmpPath, localPath)
if err != nil { if err != nil {
+83
View File
@@ -440,3 +440,86 @@ func TestFetchProgress(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, content, downloaded) assert.Equal(t, content, downloaded)
} }
// TestFetchRefusesSymlinks runs fetch into a destination directory that
// holds a symlink pointing outside it, in each of the three places fetch
// writes: a parent directory, the temp file, and the file itself, which
// the temp file is renamed onto; and once as a directory inside a plain
// directory. The fetch must fail and nothing outside may change.
//
//nolint:paralleltest // changes the process-global working directory
func TestFetchRefusesSymlinks(t *testing.T) {
tests := []struct {
name string
entry string // the manifest's only file
link string // symlink placed in the destination directory
target string // what link points to, relative to the outside directory
}{
{"parent directory", "sub/deeper/file.txt", "sub", "."},
{"directory inside a plain directory", "docs/data/passwd", "docs/data", "."},
{"temp file", testFileTxt, ".file.txt.tmp", "new.txt"},
{"file", testFileTxt, testFileTxt, "new.txt"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
content := []byte("fetched")
sourceFs := afero.NewMemMapFs()
require.NoError(t, sourceFs.MkdirAll(filepath.Dir("/"+tt.entry), 0o755))
require.NoError(t, afero.WriteFile(sourceFs, "/"+tt.entry, content, 0o644))
server := httptest.NewServer(fetchTestHandler(
scanToManifest(t, sourceFs), map[string][]byte{tt.entry: content}))
defer server.Close()
outside := t.TempDir()
chdirTemp(t)
require.NoError(t, os.MkdirAll(filepath.Dir(tt.link), 0o750))
require.NoError(t, os.Symlink(filepath.Join(outside, tt.target), tt.link))
opts := testOpts([]string{testApp, cmdFetch, "-q", server.URL}, afero.NewOsFs())
assert.Equal(t, 1, runCLI(opts))
assert.Contains(t, testStderr(t, opts), "failed to download "+tt.entry+
": symlink in path not allowed: "+tt.link)
written, err := os.ReadDir(outside)
require.NoError(t, err)
assert.Empty(t, written, "fetch wrote outside the destination")
})
}
}
// TestFetchReplacesHardLinkAtTempName runs fetch into a destination
// directory that holds, at the temp file's name, a hard link to a file
// outside it. To fetch that is an ordinary leftover from an interrupted
// earlier run: it must replace it and succeed, and the outside file must
// not change.
//
//nolint:paralleltest // changes the process-global working directory
func TestFetchReplacesHardLinkAtTempName(t *testing.T) {
content := []byte("fetched")
sourceFs := afero.NewMemMapFs()
require.NoError(t, afero.WriteFile(sourceFs, "/"+testFileTxt, content, 0o644))
server := httptest.NewServer(fetchTestHandler(
scanToManifest(t, sourceFs), map[string][]byte{testFileTxt: content}))
defer server.Close()
outsideFile := filepath.Join(t.TempDir(), "secret.txt")
require.NoError(t, os.WriteFile(outsideFile, []byte("outside"), 0o600))
chdirTemp(t)
require.NoError(t, os.Link(outsideFile, ".file.txt.tmp"))
opts := testOpts([]string{testApp, cmdFetch, "-q", server.URL}, afero.NewOsFs())
require.Equal(t, 0, runCLI(opts), testStderr(t, opts))
fetched, err := os.ReadFile(testFileTxt)
require.NoError(t, err)
assert.Equal(t, content, fetched)
outside, err := os.ReadFile(outsideFile) //nolint:gosec // test-controlled path
require.NoError(t, err)
assert.Equal(t, "outside", string(outside), "fetch wrote outside the destination")
}
+4 -3
View File
@@ -1,6 +1,7 @@
package cli package cli
import ( import (
"context"
"crypto/sha256" "crypto/sha256"
"errors" "errors"
"fmt" "fmt"
@@ -304,7 +305,7 @@ func (h *freshenHasher) processEntry(e *freshenEntry) error {
// writeFreshenedManifest writes the manifest atomically (write to a // writeFreshenedManifest writes the manifest atomically (write to a
// temp file, then rename over the target). // temp file, then rename over the target).
func writeFreshenedManifest( func writeFreshenedManifest(
afs afero.Fs, builder *mfer.Builder, manifestPath string, ctx context.Context, afs afero.Fs, builder *mfer.Builder, manifestPath string,
) error { ) error {
tmpPath := manifestPath + ".tmp" tmpPath := manifestPath + ".tmp"
@@ -313,7 +314,7 @@ func writeFreshenedManifest(
return fmt.Errorf("failed to create temp file: %w", err) return fmt.Errorf("failed to create temp file: %w", err)
} }
err = builder.Build(outFile) err = builder.Build(ctx, outFile)
_ = outFile.Close() _ = outFile.Close()
if err != nil { if err != nil {
@@ -530,7 +531,7 @@ func (mfa *CLIApp) freshenManifestOperation(ctx *cli.Context) error {
} }
// Write updated manifest atomically (write to temp, then rename) // Write updated manifest atomically (write to temp, then rename)
err = writeFreshenedManifest(mfa.Fs, hasher.builder, manifestPath) err = writeFreshenedManifest(ctx.Context, mfa.Fs, hasher.builder, manifestPath)
if err != nil { if err != nil {
return err return err
} }
+5 -27
View File
@@ -18,7 +18,7 @@ const (
cmdGenerate = "generate" cmdGenerate = "generate"
cmdCheck = "check" cmdCheck = "check"
cmdExport = "export" cmdExport = "export"
cmdVersion = "version" cmdFetch = "fetch"
flagProgress = "progress" flagProgress = "progress"
@@ -68,12 +68,6 @@ func (mfa *CLIApp) VersionString() string {
return mfer.Version return mfer.Version
} }
// printVersion writes the version line shared by the --version flag and the
// version subcommand, so both produce identical output.
func (mfa *CLIApp) printVersion() {
_, _ = fmt.Fprintf(mfa.Stdout, "%s version %s\n", mfa.appname, mfa.VersionString())
}
func (mfa *CLIApp) printBanner() { func (mfa *CLIApp) printBanner() {
if log.GetLevel() <= log.InfoLevel { if log.GetLevel() <= log.InfoLevel {
_, _ = fmt.Fprintln(mfa.Stdout, banner) _, _ = fmt.Fprintln(mfa.Stdout, banner)
@@ -272,10 +266,10 @@ func (mfa *CLIApp) exportCommand() *cli.Command {
func (mfa *CLIApp) versionCommand() *cli.Command { func (mfa *CLIApp) versionCommand() *cli.Command {
return &cli.Command{ return &cli.Command{
Name: cmdVersion, Name: "version",
Usage: "Show version", Usage: "Show version",
Action: func(_ *cli.Context) error { Action: func(_ *cli.Context) error {
mfa.printVersion() _, _ = fmt.Fprintln(mfa.Stdout, mfa.VersionString())
return nil return nil
}, },
@@ -307,7 +301,7 @@ func (mfa *CLIApp) listCommand() *cli.Command {
func (mfa *CLIApp) fetchCommand() *cli.Command { func (mfa *CLIApp) fetchCommand() *cli.Command {
return &cli.Command{ return &cli.Command{
Name: "fetch", Name: cmdFetch,
Usage: "fetch manifest and referenced files", Usage: "fetch manifest and referenced files",
Action: func(c *cli.Context) error { Action: func(c *cli.Context) error {
mfa.setVerbosity(c) mfa.setVerbosity(c)
@@ -331,20 +325,6 @@ func (mfa *CLIApp) run(args []string) {
log.SetOutput(mfa.Stdout, mfa.Stderr) log.SetOutput(mfa.Stdout, mfa.Stderr)
log.Init() log.Init()
// -v means verbose, not version. urfave/cli's built-in version flag
// claims -v by default, colliding with the -v verbose alias used here and
// on every subcommand; that collision makes "mfer -v --version" fail to
// parse. Verbose is the more common meaning of -v in tools that offer
// both, so verbose keeps -v and the version flag takes the capital -V.
// VersionFlag and VersionPrinter are urfave/cli package globals; run() is
// serialized in tests, so assigning them here is safe.
cli.VersionFlag = &cli.BoolFlag{
Name: cmdVersion,
Aliases: []string{"V"},
Usage: "print the version",
}
cli.VersionPrinter = func(_ *cli.Context) { mfa.printVersion() }
mfa.app = &cli.App{ mfa.app = &cli.App{
Name: mfa.appname, Name: mfa.appname,
Usage: "Manifest generator", Usage: "Manifest generator",
@@ -352,13 +332,11 @@ func (mfa *CLIApp) run(args []string) {
EnableBashCompletion: true, EnableBashCompletion: true,
Writer: mfa.Stdout, Writer: mfa.Stdout,
ErrWriter: mfa.Stderr, ErrWriter: mfa.Stderr,
Flags: commonFlags(),
Action: func(c *cli.Context) error { Action: func(c *cli.Context) error {
if c.Args().Len() > 0 { if c.Args().Len() > 0 {
return fmt.Errorf("%w %q", errUnknownCommand, c.Args().First()) return fmt.Errorf("%w %q", errUnknownCommand, c.Args().First())
} }
mfa.setVerbosity(c)
mfa.printBanner() mfa.printBanner()
return cli.ShowAppHelp(c) return cli.ShowAppHelp(c)
@@ -380,6 +358,6 @@ func (mfa *CLIApp) run(args []string) {
if err != nil { if err != nil {
mfa.exitCode = 1 mfa.exitCode = 1
log.WithError(err).Debugf("exiting") log.Errorf("%s", err)
} }
} }
+34 -41
View File
@@ -112,13 +112,16 @@ func DisableStyling() {
} }
// Init initializes the logger with the CLI handler and default log level. // Init initializes the logger with the CLI handler and default log level.
//
// It reconfigures the process-global apex/log logger under the write lock so
// the global is never mutated while another goroutine holds the read lock to
// read it in emit. Without this, parallel callers (e.g. the test suite) race
// Init's SetLevel/SetHandler against concurrent log calls.
func Init() { func Init() {
mu.RLock() mu.Lock()
defer mu.Unlock()
w := stderr log.SetHandler(acli.New(stderr))
mu.RUnlock()
log.SetHandler(acli.New(w))
log.SetLevel(log.DebugLevel) // Let apex/log pass everything; we filter ourselves log.SetLevel(log.DebugLevel) // Let apex/log pass everything; we filter ourselves
} }
@@ -130,74 +133,66 @@ func isEnabled(l Level) bool {
return l >= currentLevel return l >= currentLevel
} }
// emit calls fn while holding the read lock if messages at level l are
// enabled. Holding the read lock across the apex/log call keeps the global
// logger from being read while Init reconfigures it under the write lock.
func emit(l Level, fn func()) {
mu.RLock()
defer mu.RUnlock()
if l >= currentLevel {
fn()
}
}
// Fatalf logs a formatted message at fatal level. // Fatalf logs a formatted message at fatal level.
func Fatalf(format string, args ...any) { func Fatalf(format string, args ...any) {
if isEnabled(FatalLevel) { emit(FatalLevel, func() { log.Fatalf(format, args...) })
log.Fatalf(format, args...)
}
} }
// Fatal logs a message at fatal level. // Fatal logs a message at fatal level.
func Fatal(arg string) { func Fatal(arg string) {
if isEnabled(FatalLevel) { emit(FatalLevel, func() { log.Fatal(arg) })
log.Fatal(arg)
}
} }
// Errorf logs a formatted message at error level. // Errorf logs a formatted message at error level.
func Errorf(format string, args ...any) { func Errorf(format string, args ...any) {
if isEnabled(ErrorLevel) { emit(ErrorLevel, func() { log.Errorf(format, args...) })
log.Errorf(format, args...)
}
} }
// Error logs a message at error level. // Error logs a message at error level.
func Error(arg string) { func Error(arg string) {
if isEnabled(ErrorLevel) { emit(ErrorLevel, func() { log.Error(arg) })
log.Error(arg)
}
} }
// Warnf logs a formatted message at warn level. // Warnf logs a formatted message at warn level.
func Warnf(format string, args ...any) { func Warnf(format string, args ...any) {
if isEnabled(WarnLevel) { emit(WarnLevel, func() { log.Warnf(format, args...) })
log.Warnf(format, args...)
}
} }
// Warn logs a message at warn level. // Warn logs a message at warn level.
func Warn(arg string) { func Warn(arg string) {
if isEnabled(WarnLevel) { emit(WarnLevel, func() { log.Warn(arg) })
log.Warn(arg)
}
} }
// Infof logs a formatted message at info level. // Infof logs a formatted message at info level.
func Infof(format string, args ...any) { func Infof(format string, args ...any) {
if isEnabled(InfoLevel) { emit(InfoLevel, func() { log.Infof(format, args...) })
log.Infof(format, args...)
}
} }
// Info logs a message at info level. // Info logs a message at info level.
func Info(arg string) { func Info(arg string) {
if isEnabled(InfoLevel) { emit(InfoLevel, func() { log.Info(arg) })
log.Info(arg)
}
} }
// Verbosef logs a formatted message at verbose level. // Verbosef logs a formatted message at verbose level.
func Verbosef(format string, args ...any) { func Verbosef(format string, args ...any) {
if isEnabled(VerboseLevel) { emit(VerboseLevel, func() { log.Infof(format, args...) })
log.Infof(format, args...)
}
} }
// Verbose logs a message at verbose level. // Verbose logs a message at verbose level.
func Verbose(arg string) { func Verbose(arg string) {
if isEnabled(VerboseLevel) { emit(VerboseLevel, func() { log.Info(arg) })
log.Info(arg)
}
} }
// Debugf logs a formatted message at debug level with caller location. // Debugf logs a formatted message at debug level with caller location.
@@ -216,7 +211,10 @@ func Debug(arg string) {
// DebugReal logs at debug level with caller info from the specified stack depth. // DebugReal logs at debug level with caller info from the specified stack depth.
func DebugReal(arg string, cs int) { func DebugReal(arg string, cs int) {
if !isEnabled(DebugLevel) { mu.RLock()
defer mu.RUnlock()
if DebugLevel < currentLevel {
return return
} }
@@ -275,11 +273,6 @@ func GetLevel() Level {
return currentLevel return currentLevel
} }
// WithError returns a log entry with the error attached.
func WithError(e error) *log.Entry {
return log.Log.WithError(e)
}
// Progressf prints a progress message that overwrites the current line. // Progressf prints a progress message that overwrites the current line.
// Use ProgressDone() when progress is complete to move to the next line. // Use ProgressDone() when progress is complete to move to the next line.
func Progressf(format string, args ...any) { func Progressf(format string, args ...any) {
+6 -4
View File
@@ -3,6 +3,7 @@
package mfer package mfer
import ( import (
"context"
"crypto/sha256" "crypto/sha256"
"errors" "errors"
"fmt" "fmt"
@@ -281,8 +282,9 @@ func (b *Builder) SetSigningOptions(opts *SigningOptions) {
b.signingOptions = opts b.signingOptions = opts
} }
// Build finalizes the manifest and writes it to the writer. // Build finalizes the manifest and writes it to the writer. ctx bounds the
func (b *Builder) Build(w io.Writer) error { // gpg runs that sign the manifest when signing options are set.
func (b *Builder) Build(ctx context.Context, w io.Writer) error {
b.mu.Lock() b.mu.Lock()
defer b.mu.Unlock() defer b.mu.Unlock()
@@ -308,13 +310,13 @@ func (b *Builder) Build(w io.Writer) error {
} }
// Generate outer wrapper // Generate outer wrapper
err := m.generateOuter() err := m.generateOuter(ctx)
if err != nil { if err != nil {
return fmt.Errorf("build: generate outer: %w", err) return fmt.Errorf("build: generate outer: %w", err)
} }
// Generate final output // Generate final output
err = m.generate() err = m.generate(ctx)
if err != nil { if err != nil {
return fmt.Errorf("build: generate: %w", err) return fmt.Errorf("build: generate: %w", err)
} }
+9 -8
View File
@@ -3,6 +3,7 @@ package mfer
import ( import (
"bytes" "bytes"
"context"
"strings" "strings"
"testing" "testing"
"time" "time"
@@ -113,7 +114,7 @@ func TestBuilderBuild(t *testing.T) {
var buf bytes.Buffer var buf bytes.Buffer
err = b.Build(&buf) err = b.Build(context.Background(), &buf)
require.NoError(t, err) require.NoError(t, err)
// Should have magic bytes // Should have magic bytes
@@ -177,7 +178,7 @@ func TestBuilderDeterministicOutput(t *testing.T) {
var buf bytes.Buffer var buf bytes.Buffer
err := b.Build(&buf) err := b.Build(context.Background(), &buf)
require.NoError(t, err) require.NoError(t, err)
return buf.Bytes() return buf.Bytes()
@@ -325,7 +326,7 @@ func TestBuilderBuildRoundTrip(t *testing.T) {
} }
var buf bytes.Buffer var buf bytes.Buffer
require.NoError(t, b.Build(&buf)) require.NoError(t, b.Build(context.Background(), &buf))
m, err := NewManifestFromReader(&buf) m, err := NewManifestFromReader(&buf)
require.NoError(t, err) require.NoError(t, err)
@@ -383,7 +384,7 @@ func TestManifestString(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
var buf bytes.Buffer var buf bytes.Buffer
require.NoError(t, b.Build(&buf)) require.NoError(t, b.Build(context.Background(), &buf))
m, err := NewManifestFromReader(&buf) m, err := NewManifestFromReader(&buf)
require.NoError(t, err) require.NoError(t, err)
@@ -397,7 +398,7 @@ func TestBuilderBuildEmpty(t *testing.T) {
var buf bytes.Buffer var buf bytes.Buffer
err := b.Build(&buf) err := b.Build(context.Background(), &buf)
require.NoError(t, err) require.NoError(t, err)
// Should still produce valid manifest with 0 files // Should still produce valid manifest with 0 files
@@ -416,7 +417,7 @@ func TestBuilderOmitsCreatedAtByDefault(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
var buf bytes.Buffer var buf bytes.Buffer
require.NoError(t, b.Build(&buf)) require.NoError(t, b.Build(context.Background(), &buf))
m, err := NewManifestFromReader(&buf) m, err := NewManifestFromReader(&buf)
require.NoError(t, err) require.NoError(t, err)
@@ -438,7 +439,7 @@ func TestBuilderIncludesCreatedAtWhenRequested(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
var buf bytes.Buffer var buf bytes.Buffer
require.NoError(t, b.Build(&buf)) require.NoError(t, b.Build(context.Background(), &buf))
m, err := NewManifestFromReader(&buf) m, err := NewManifestFromReader(&buf)
require.NoError(t, err) require.NoError(t, err)
@@ -464,7 +465,7 @@ func TestBuilderDeterministicFileOrder(t *testing.T) {
} }
var buf bytes.Buffer var buf bytes.Buffer
require.NoError(t, b.Build(&buf)) require.NoError(t, b.Build(context.Background(), &buf))
m, err := NewManifestFromReader(&buf) m, err := NewManifestFromReader(&buf)
require.NoError(t, err) require.NoError(t, err)
+6 -2
View File
@@ -164,12 +164,12 @@ func (c *Checker) SigningPubKey() []byte {
// ExtractEmbeddedSigningKeyFP imports the manifest's embedded public key into a // ExtractEmbeddedSigningKeyFP imports the manifest's embedded public key into a
// temporary keyring and extracts its fingerprint. This validates the key and // temporary keyring and extracts its fingerprint. This validates the key and
// returns its actual fingerprint from the key material itself. // returns its actual fingerprint from the key material itself.
func (c *Checker) ExtractEmbeddedSigningKeyFP() (string, error) { func (c *Checker) ExtractEmbeddedSigningKeyFP(ctx context.Context) (string, error) {
if len(c.signingPubKey) == 0 { if len(c.signingPubKey) == 0 {
return "", errNoSigningPubKey return "", errNoSigningPubKey
} }
return gpgExtractPubKeyFingerprint(c.signingPubKey) return gpgExtractPubKeyFingerprint(ctx, c.signingPubKey)
} }
// Check verifies all files against the manifest. // Check verifies all files against the manifest.
@@ -312,6 +312,10 @@ func (c *Checker) FindExtraFiles(ctx context.Context, results chan<- Result) err
} }
func (c *Checker) checkFile(entry *MFFilePath, checkedBytes *FileSize) Result { func (c *Checker) checkFile(entry *MFFilePath, checkedBytes *FileSize) Result {
// entry.GetPath() is safe to join here: a manifest's entry paths are
// validated against the path invariants when it is loaded (see
// deserializeInner) or built (see Builder.AddFile), so a traversal or
// absolute path can never reach this point.
absPath := filepath.Join(string(c.basePath), entry.GetPath()) absPath := filepath.Join(string(c.basePath), entry.GetPath())
relPath := RelFilePath(entry.GetPath()) relPath := RelFilePath(entry.GetPath())
+1 -1
View File
@@ -61,7 +61,7 @@ func createTestManifest(
} }
var buf bytes.Buffer var buf bytes.Buffer
require.NoError(t, builder.Build(&buf)) require.NoError(t, builder.Build(context.Background(), &buf))
require.NoError(t, afero.WriteFile(fs, manifestPath, buf.Bytes(), 0o644)) require.NoError(t, afero.WriteFile(fs, manifestPath, buf.Bytes(), 0o644))
} }
+17
View File
@@ -2,6 +2,7 @@ package mfer
import ( import (
"bytes" "bytes"
"context"
"crypto/sha256" "crypto/sha256"
"errors" "errors"
"fmt" "fmt"
@@ -25,6 +26,7 @@ var (
errDecompressedTooLarge = errors.New("decompressed data exceeds maximum allowed size") errDecompressedTooLarge = errors.New("decompressed data exceeds maximum allowed size")
errUUIDMismatch = errors.New("outer and inner UUID mismatch") errUUIDMismatch = errors.New("outer and inner UUID mismatch")
errInvalidFileFormat = errors.New("invalid file format") errInvalidFileFormat = errors.New("invalid file format")
errInvalidManifestPath = errors.New("manifest contains invalid path")
) )
// validateUUID checks that the byte slice is a valid UUID (16 bytes, parseable). // validateUUID checks that the byte slice is a valid UUID (16 bytes, parseable).
@@ -91,7 +93,9 @@ func (m *manifest) verifyOuterIntegrity() error {
) )
} }
// Loading a manifest takes no context; gpgTimeout still bounds gpg.
err = gpgVerify( err = gpgVerify(
context.Background(),
[]byte(sigString), []byte(sigString),
m.pbOuter.GetSignature(), m.pbOuter.GetSignature(),
m.pbOuter.GetSigningPubKey(), m.pbOuter.GetSigningPubKey(),
@@ -181,6 +185,19 @@ func (m *manifest) deserializeInner() error {
return errUUIDMismatch return errUUIDMismatch
} }
// Enforce the manifest path invariants on every entry as it is loaded,
// so that no consumer of a manifest — Checker today, any restore or
// extract path tomorrow — acts on a traversal or absolute path from an
// untrusted .mf. Reject loudly on the first offender rather than
// dropping entries, which would let a hostile manifest hide files from a
// check.
for _, f := range m.pbInner.GetFiles() {
err = ValidatePath(f.GetPath())
if err != nil {
return fmt.Errorf("%w: %w", errInvalidManifestPath, err)
}
}
log.Infof("loaded manifest with %d files", len(m.pbInner.GetFiles())) log.Infof("loaded manifest with %d files", len(m.pbInner.GetFiles()))
return nil return nil
+146
View File
@@ -0,0 +1,146 @@
//nolint:testpackage // white-box tests exercise unexported internals
package mfer
import (
"bytes"
"context"
"crypto/sha256"
"fmt"
"testing"
"github.com/google/uuid"
"github.com/klauspost/compress/zstd"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"google.golang.org/protobuf/encoding/protowire"
"google.golang.org/protobuf/proto"
)
// craftInnerBytes builds the wire bytes of an inner MFFile holding a single
// file entry whose path is exactly pathBytes. It writes the wire form by hand
// so a hostile path — including one that is not valid UTF-8 — can be embedded
// without proto.Marshal's own UTF-8 enforcement rejecting it first.
func craftInnerBytes(id uuid.UUID, pathBytes string) []byte {
entry := protowire.AppendTag(nil, 1, protowire.BytesType) // MFFilePath.path
entry = protowire.AppendString(entry, pathBytes)
inner := protowire.AppendTag(nil, 100, protowire.VarintType) // MFFile.version
inner = protowire.AppendVarint(inner, uint64(MFFile_VERSION_ONE))
inner = protowire.AppendTag(inner, 101, protowire.BytesType) // MFFile.files
inner = protowire.AppendBytes(inner, entry)
inner = protowire.AppendTag(inner, 102, protowire.BytesType) // MFFile.uuid
inner = protowire.AppendBytes(inner, id[:])
return inner
}
// wrapInner wraps inner MFFile wire bytes in a complete, well-formed .mf
// envelope (magic prefix, zstd-compressed payload, matching hash and UUID) so
// that deserialization reaches path validation rather than failing earlier on
// an integrity check.
func wrapInner(t *testing.T, id uuid.UUID, innerData []byte) []byte {
t.Helper()
var cbuf bytes.Buffer
zw, err := zstd.NewWriter(&cbuf, zstd.WithEncoderLevel(zstd.SpeedBestCompression))
require.NoError(t, err)
_, err = zw.Write(innerData)
require.NoError(t, err)
require.NoError(t, zw.Close())
compressed := cbuf.Bytes()
sum := sha256.Sum256(compressed)
outer := &MFFileOuter{
InnerMessage: compressed,
Size: int64(len(innerData)),
Sha256: sum[:],
Uuid: id[:],
Version: MFFileOuter_VERSION_ONE,
CompressionType: MFFileOuter_COMPRESSION_ZSTD,
}
ob, err := proto.Marshal(outer)
require.NoError(t, err)
return append([]byte(MAGIC), ob...)
}
func TestDeserializeRejectsInvalidEntryPaths(t *testing.T) {
t.Parallel()
tests := []struct {
name string
path string
}{
{"parent traversal", "../escape"},
{"interior traversal", "a/../../escape"},
{"absolute path", "/etc/passwd"},
{"backslash path", `a\b`},
{"double slash", "a//b"},
{"empty path", ""},
{"invalid utf-8", "abc\xff"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
id := uuid.New()
data := wrapInner(t, id, craftInnerBytes(id, tt.path))
_, err := NewManifestFromReader(bytes.NewReader(data))
require.Error(t, err)
if tt.path == "abc\xff" {
// A path that is not valid UTF-8 cannot survive the proto3
// string decoder, which rejects it before path validation
// runs; the manifest is still refused at load time.
return
}
require.ErrorIs(t, err, errInvalidManifestPath)
if tt.path != "" {
// ValidatePath quotes the path with %q; assert against the
// same rendering so escaped characters (e.g. a backslash)
// still match.
assert.Contains(t, err.Error(), fmt.Sprintf("%q", tt.path),
"error must name the offending path")
}
})
}
}
func TestDeserializeValidManifestRoundTrips(t *testing.T) {
t.Parallel()
hash := make([]byte, 34) // multihash: 2-byte prefix + 32-byte SHA-256
b := NewBuilder()
require.NoError(t, b.AddFileWithHash("dir/file.txt", 123, ModTime{}, hash))
var buf bytes.Buffer
require.NoError(t, b.Build(context.Background(), &buf))
m, err := NewManifestFromReader(bytes.NewReader(buf.Bytes()))
require.NoError(t, err)
files := m.Files()
require.Len(t, files, 1)
assert.Equal(t, "dir/file.txt", files[0].GetPath())
assert.Equal(t, int64(123), files[0].GetSize())
}
// TestValidatePathRejectsInvalidUTF8 pins the ValidatePath rule that a manifest
// path must be valid UTF-8, independent of the proto decoder that also enforces
// it on the wire.
func TestValidatePathRejectsInvalidUTF8(t *testing.T) {
t.Parallel()
err := ValidatePath("abc\xff")
require.ErrorIs(t, err, errPathNotUTF8)
assert.Contains(t, err.Error(), "UTF-8")
}
+4 -2
View File
@@ -2,6 +2,7 @@
package mfer package mfer
import ( import (
"context"
"testing" "testing"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
@@ -80,6 +81,7 @@ func TestSerializeInternalErrorMessagesVerbatim(t *testing.T) {
t.Parallel() t.Parallel()
m := &manifest{} m := &manifest{}
require.EqualError(t, m.generate(), "internal error: pbInner not set") require.EqualError(t, m.generate(context.Background()),
require.EqualError(t, m.generateOuter(), "internal error") "internal error: pbInner not set")
require.EqualError(t, m.generateOuter(context.Background()), "internal error")
} }
+38 -15
View File
@@ -10,9 +10,15 @@ import (
"os/exec" "os/exec"
"path/filepath" "path/filepath"
"strings" "strings"
"time"
) )
const ( const (
// gpgTimeout bounds every gpg run, which can otherwise wait forever on
// a passphrase prompt or a stalled gpg-agent. A minute leaves a person
// time to type a passphrase or touch a smartcard.
gpgTimeout = time.Minute
// privateDirPerms is the permission mode for temporary GPG home // privateDirPerms is the permission mode for temporary GPG home
// directories. // directories.
privateDirPerms os.FileMode = 0o700 privateDirPerms os.FileMode = 0o700
@@ -66,8 +72,17 @@ func gpgArgs(opts []string, positional ...string) []string {
} }
// runGPG runs the gpg binary in batch mode with the given arguments and // runGPG runs the gpg binary in batch mode with the given arguments and
// optional stdin, returning captured stdout and stderr. // optional stdin, returning captured stdout and stderr. gpg is killed when
func runGPG(stdin io.Reader, args ...string) (*bytes.Buffer, *bytes.Buffer, error) { // ctx ends or gpgTimeout passes, whichever comes first.
func runGPG(
ctx context.Context, stdin io.Reader, args ...string,
) (*bytes.Buffer, *bytes.Buffer, error) {
// exec.CommandContext kills only gpg itself, and that is enough: a
// gpg-agent that gpg starts runs detached in its own session and does
// not hold gpg's output open, so Run returns as soon as gpg dies.
ctx, cancel := context.WithTimeout(ctx, gpgTimeout)
defer cancel()
fullArgs := append([]string{"--batch", "--no-tty"}, args...) fullArgs := append([]string{"--batch", "--no-tty"}, args...)
// G204: the executable name is a compile-time constant. The arguments // G204: the executable name is a compile-time constant. The arguments
@@ -76,7 +91,7 @@ func runGPG(stdin io.Reader, args ...string) (*bytes.Buffer, *bytes.Buffer, erro
// option or after the "--" end-of-options marker inserted by gpgArgs, // option or after the "--" end-of-options marker inserted by gpgArgs,
// and therefore cannot be reinterpreted by gpg as an option. // and therefore cannot be reinterpreted by gpg as an option.
cmd := exec.CommandContext( //nolint:gosec // G204: see comment above cmd := exec.CommandContext( //nolint:gosec // G204: see comment above
context.Background(), "gpg", fullArgs...) ctx, "gpg", fullArgs...)
cmd.Stdin = stdin cmd.Stdin = stdin
var stdout, stderr bytes.Buffer var stdout, stderr bytes.Buffer
@@ -85,6 +100,14 @@ func runGPG(stdin io.Reader, args ...string) (*bytes.Buffer, *bytes.Buffer, erro
cmd.Stderr = &stderr cmd.Stderr = &stderr
err := cmd.Run() err := cmd.Run()
if err != nil && ctx.Err() != nil {
// gpg was killed because ctx ended, which Run reports only as
// "signal: killed"; return the reason instead.
err = ctx.Err()
if errors.Is(err, context.DeadlineExceeded) {
err = fmt.Errorf("gpg timed out: %w", err)
}
}
return &stdout, &stderr, err return &stdout, &stderr, err
} }
@@ -105,8 +128,8 @@ func parseFingerprint(colonOutput string) (string, bool) {
// gpgSign creates a detached signature of the data using the specified key. // gpgSign creates a detached signature of the data using the specified key.
// Returns the armored detached signature. // Returns the armored detached signature.
func gpgSign(data []byte, keyID GPGKeyID) ([]byte, error) { func gpgSign(ctx context.Context, data []byte, keyID GPGKeyID) ([]byte, error) {
stdout, stderr, err := runGPG(bytes.NewReader(data), stdout, stderr, err := runGPG(ctx, bytes.NewReader(data),
"--detach-sign", "--detach-sign",
gpgOptArmor, gpgOptArmor,
"--local-user", string(keyID), "--local-user", string(keyID),
@@ -120,8 +143,8 @@ func gpgSign(data []byte, keyID GPGKeyID) ([]byte, error) {
// gpgExportPublicKey exports the public key for the specified key ID. // gpgExportPublicKey exports the public key for the specified key ID.
// Returns the armored public key. // Returns the armored public key.
func gpgExportPublicKey(keyID GPGKeyID) ([]byte, error) { func gpgExportPublicKey(ctx context.Context, keyID GPGKeyID) ([]byte, error) {
stdout, stderr, err := runGPG(nil, stdout, stderr, err := runGPG(ctx, nil,
gpgArgs([]string{"--export", gpgOptArmor}, string(keyID))..., gpgArgs([]string{"--export", gpgOptArmor}, string(keyID))...,
) )
if err != nil { if err != nil {
@@ -136,8 +159,8 @@ func gpgExportPublicKey(keyID GPGKeyID) ([]byte, error) {
} }
// gpgGetKeyFingerprint gets the full fingerprint for a key ID. // gpgGetKeyFingerprint gets the full fingerprint for a key ID.
func gpgGetKeyFingerprint(keyID GPGKeyID) ([]byte, error) { func gpgGetKeyFingerprint(ctx context.Context, keyID GPGKeyID) ([]byte, error) {
stdout, stderr, err := runGPG(nil, stdout, stderr, err := runGPG(ctx, nil,
gpgArgs([]string{"--with-colons", "--fingerprint"}, string(keyID))..., gpgArgs([]string{"--with-colons", "--fingerprint"}, string(keyID))...,
) )
if err != nil { if err != nil {
@@ -157,7 +180,7 @@ func gpgGetKeyFingerprint(keyID GPGKeyID) ([]byte, error) {
// gpgExtractPubKeyFingerprint imports a public key into a temporary keyring // gpgExtractPubKeyFingerprint imports a public key into a temporary keyring
// and extracts its fingerprint. This verifies the key is valid and returns // and extracts its fingerprint. This verifies the key is valid and returns
// the actual fingerprint from the key material. // the actual fingerprint from the key material.
func gpgExtractPubKeyFingerprint(pubKey []byte) (string, error) { func gpgExtractPubKeyFingerprint(ctx context.Context, pubKey []byte) (string, error) {
// Create temporary directory for GPG operations // Create temporary directory for GPG operations
tmpDir, err := os.MkdirTemp("", "mfer-gpg-fingerprint-*") tmpDir, err := os.MkdirTemp("", "mfer-gpg-fingerprint-*")
if err != nil { if err != nil {
@@ -181,7 +204,7 @@ func gpgExtractPubKeyFingerprint(pubKey []byte) (string, error) {
} }
// Import the public key into the temporary keyring // Import the public key into the temporary keyring
_, importStderr, err := runGPG(nil, _, importStderr, err := runGPG(ctx, nil,
gpgArgs([]string{gpgOptHomedir, tmpDir, "--import"}, pubKeyFile)..., gpgArgs([]string{gpgOptHomedir, tmpDir, "--import"}, pubKeyFile)...,
) )
if err != nil { if err != nil {
@@ -191,7 +214,7 @@ func gpgExtractPubKeyFingerprint(pubKey []byte) (string, error) {
} }
// List keys to get fingerprint // List keys to get fingerprint
listStdout, listStderr, err := runGPG(nil, listStdout, listStderr, err := runGPG(ctx, nil,
"--homedir", tmpDir, "--homedir", tmpDir,
"--with-colons", "--with-colons",
"--fingerprint", "--fingerprint",
@@ -212,7 +235,7 @@ func gpgExtractPubKeyFingerprint(pubKey []byte) (string, error) {
// gpgVerify verifies a detached signature against data using the provided public key. // gpgVerify verifies a detached signature against data using the provided public key.
// It creates a temporary keyring to import the public key for verification. // It creates a temporary keyring to import the public key for verification.
func gpgVerify(data, signature, pubKey []byte) error { func gpgVerify(ctx context.Context, data, signature, pubKey []byte) error {
// Create temporary directory for GPG operations // Create temporary directory for GPG operations
tmpDir, err := os.MkdirTemp("", "mfer-gpg-verify-*") tmpDir, err := os.MkdirTemp("", "mfer-gpg-verify-*")
if err != nil { if err != nil {
@@ -252,7 +275,7 @@ func gpgVerify(data, signature, pubKey []byte) error {
} }
// Import the public key into the temporary keyring // Import the public key into the temporary keyring
_, importStderr, err := runGPG(nil, _, importStderr, err := runGPG(ctx, nil,
gpgArgs([]string{gpgOptHomedir, tmpDir, "--import"}, pubKeyFile)..., gpgArgs([]string{gpgOptHomedir, tmpDir, "--import"}, pubKeyFile)...,
) )
if err != nil { if err != nil {
@@ -262,7 +285,7 @@ func gpgVerify(data, signature, pubKey []byte) error {
} }
// Verify the signature // Verify the signature
_, verifyStderr, err := runGPG(nil, _, verifyStderr, err := runGPG(ctx, nil,
gpgArgs([]string{gpgOptHomedir, tmpDir, gpgOptVerify}, gpgArgs([]string{gpgOptHomedir, tmpDir, gpgOptVerify},
sigFile, dataFile)..., sigFile, dataFile)...,
) )
+58 -20
View File
@@ -4,11 +4,13 @@ package mfer
import ( import (
"bytes" "bytes"
"context" "context"
"io"
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
"strings" "strings"
"testing" "testing"
"time"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
@@ -43,8 +45,11 @@ Expire-Date: 0
paramsFile := filepath.Join(gpgHome, "key-params") paramsFile := filepath.Join(gpgHome, "key-params")
require.NoError(t, os.WriteFile(paramsFile, []byte(keyParams), 0o600)) require.NoError(t, os.WriteFile(paramsFile, []byte(keyParams), 0o600))
ctx, cancel := context.WithTimeout(context.Background(), gpgTimeout)
defer cancel()
//nolint:gosec // paramsFile is a test-controlled path inside t.TempDir() //nolint:gosec // paramsFile is a test-controlled path inside t.TempDir()
cmd := exec.CommandContext(context.Background(), "gpg", cmd := exec.CommandContext(ctx, "gpg",
"--batch", "--gen-key", paramsFile) "--batch", "--gen-key", paramsFile)
cmd.Env = append(os.Environ(), "GNUPGHOME="+gpgHome) cmd.Env = append(os.Environ(), "GNUPGHOME="+gpgHome)
@@ -55,7 +60,7 @@ Expire-Date: 0
} }
// Get the key fingerprint // Get the key fingerprint
cmd = exec.CommandContext(context.Background(), "gpg", cmd = exec.CommandContext(ctx, "gpg",
"--list-keys", "--with-colons", "test@mfer.test") "--list-keys", "--with-colons", "test@mfer.test")
cmd.Env = append(os.Environ(), "GNUPGHOME="+gpgHome) cmd.Env = append(os.Environ(), "GNUPGHOME="+gpgHome)
@@ -90,7 +95,7 @@ func TestGPGSign(t *testing.T) {
t.Setenv("GNUPGHOME", gpgHome) t.Setenv("GNUPGHOME", gpgHome)
data := []byte("test data to sign") data := []byte("test data to sign")
sig, err := gpgSign(data, keyID) sig, err := gpgSign(context.Background(), data, keyID)
require.NoError(t, err) require.NoError(t, err)
assert.NotEmpty(t, sig) assert.NotEmpty(t, sig)
assert.Contains(t, string(sig), "-----BEGIN PGP SIGNATURE-----") assert.Contains(t, string(sig), "-----BEGIN PGP SIGNATURE-----")
@@ -101,7 +106,7 @@ func TestGPGExportPublicKey(t *testing.T) {
keyID, gpgHome := testGPGEnv(t) keyID, gpgHome := testGPGEnv(t)
t.Setenv("GNUPGHOME", gpgHome) t.Setenv("GNUPGHOME", gpgHome)
pubKey, err := gpgExportPublicKey(keyID) pubKey, err := gpgExportPublicKey(context.Background(), keyID)
require.NoError(t, err) require.NoError(t, err)
assert.NotEmpty(t, pubKey) assert.NotEmpty(t, pubKey)
assert.Contains(t, string(pubKey), "-----BEGIN PGP PUBLIC KEY BLOCK-----") assert.Contains(t, string(pubKey), "-----BEGIN PGP PUBLIC KEY BLOCK-----")
@@ -112,7 +117,7 @@ func TestGPGGetKeyFingerprint(t *testing.T) {
keyID, gpgHome := testGPGEnv(t) keyID, gpgHome := testGPGEnv(t)
t.Setenv("GNUPGHOME", gpgHome) t.Setenv("GNUPGHOME", gpgHome)
fingerprint, err := gpgGetKeyFingerprint(keyID) fingerprint, err := gpgGetKeyFingerprint(context.Background(), keyID)
require.NoError(t, err) require.NoError(t, err)
assert.NotEmpty(t, fingerprint) assert.NotEmpty(t, fingerprint)
// The fingerprint should be 40 hex chars // The fingerprint should be 40 hex chars
@@ -146,12 +151,12 @@ func TestGPGOptionLikeKeyIDIsNotAnOption(t *testing.T) {
_, gpgHome := testGPGEnv(t) _, gpgHome := testGPGEnv(t)
t.Setenv("GNUPGHOME", gpgHome) t.Setenv("GNUPGHOME", gpgHome)
pubKey, err := gpgExportPublicKey(GPGKeyID("--version")) pubKey, err := gpgExportPublicKey(context.Background(), GPGKeyID("--version"))
require.Error(t, err) require.Error(t, err)
require.ErrorIs(t, err, errGPGKeyNotFound) require.ErrorIs(t, err, errGPGKeyNotFound)
assert.NotContains(t, string(pubKey), "gpg (GnuPG)") assert.NotContains(t, string(pubKey), "gpg (GnuPG)")
fpr, err := gpgGetKeyFingerprint(GPGKeyID("--version")) fpr, err := gpgGetKeyFingerprint(context.Background(), GPGKeyID("--version"))
require.Error(t, err) require.Error(t, err)
assert.NotContains(t, string(fpr), "gpg (GnuPG)") assert.NotContains(t, string(fpr), "gpg (GnuPG)")
} }
@@ -162,7 +167,8 @@ func TestGPGSignInvalidKey(t *testing.T) {
t.Setenv("GNUPGHOME", gpgHome) t.Setenv("GNUPGHOME", gpgHome)
data := []byte("test data") data := []byte("test data")
_, err := gpgSign(data, GPGKeyID("NONEXISTENT_KEY_ID_12345")) _, err := gpgSign(context.Background(), data,
GPGKeyID("NONEXISTENT_KEY_ID_12345"))
assert.Error(t, err) assert.Error(t, err)
} }
@@ -185,7 +191,7 @@ func TestBuilderWithSigning(t *testing.T) {
// Build the manifest // Build the manifest
var buf bytes.Buffer var buf bytes.Buffer
err = b.Build(&buf) err = b.Build(context.Background(), &buf)
require.NoError(t, err) require.NoError(t, err)
// Parse the manifest and verify signature fields are populated // Parse the manifest and verify signature fields are populated
@@ -251,14 +257,14 @@ func TestGPGVerify(t *testing.T) {
t.Setenv("GNUPGHOME", gpgHome) t.Setenv("GNUPGHOME", gpgHome)
data := []byte("test data to sign and verify") data := []byte("test data to sign and verify")
sig, err := gpgSign(data, keyID) sig, err := gpgSign(context.Background(), data, keyID)
require.NoError(t, err) require.NoError(t, err)
pubKey, err := gpgExportPublicKey(keyID) pubKey, err := gpgExportPublicKey(context.Background(), keyID)
require.NoError(t, err) require.NoError(t, err)
// Verify the signature // Verify the signature
err = gpgVerify(data, sig, pubKey) err = gpgVerify(context.Background(), data, sig, pubKey)
require.NoError(t, err) require.NoError(t, err)
} }
@@ -267,15 +273,15 @@ func TestGPGVerifyInvalidSignature(t *testing.T) {
t.Setenv("GNUPGHOME", gpgHome) t.Setenv("GNUPGHOME", gpgHome)
data := []byte("test data to sign") data := []byte("test data to sign")
sig, err := gpgSign(data, keyID) sig, err := gpgSign(context.Background(), data, keyID)
require.NoError(t, err) require.NoError(t, err)
pubKey, err := gpgExportPublicKey(keyID) pubKey, err := gpgExportPublicKey(context.Background(), keyID)
require.NoError(t, err) require.NoError(t, err)
// Try to verify with different data - should fail // Try to verify with different data - should fail
wrongData := []byte("different data") wrongData := []byte("different data")
err = gpgVerify(wrongData, sig, pubKey) err = gpgVerify(context.Background(), wrongData, sig, pubKey)
assert.Error(t, err) assert.Error(t, err)
} }
@@ -284,12 +290,12 @@ func TestGPGVerifyBadPublicKey(t *testing.T) {
t.Setenv("GNUPGHOME", gpgHome) t.Setenv("GNUPGHOME", gpgHome)
data := []byte("test data") data := []byte("test data")
sig, err := gpgSign(data, keyID) sig, err := gpgSign(context.Background(), data, keyID)
require.NoError(t, err) require.NoError(t, err)
// Try to verify with invalid public key - should fail // Try to verify with invalid public key - should fail
badPubKey := []byte("not a valid public key") badPubKey := []byte("not a valid public key")
err = gpgVerify(data, sig, badPubKey) err = gpgVerify(context.Background(), data, sig, badPubKey)
assert.Error(t, err) assert.Error(t, err)
} }
@@ -312,7 +318,7 @@ func TestManifestSignatureVerification(t *testing.T) {
// Build the manifest // Build the manifest
var buf bytes.Buffer var buf bytes.Buffer
err = b.Build(&buf) err = b.Build(context.Background(), &buf)
require.NoError(t, err) require.NoError(t, err)
// Parse the manifest - signature should be verified during load // Parse the manifest - signature should be verified during load
@@ -341,7 +347,7 @@ func TestManifestTamperedSignatureFails(t *testing.T) {
var buf bytes.Buffer var buf bytes.Buffer
err = b.Build(&buf) err = b.Build(context.Background(), &buf)
require.NoError(t, err) require.NoError(t, err)
// Tamper with the signature by replacing some bytes // Tamper with the signature by replacing some bytes
@@ -375,7 +381,7 @@ func TestBuilderWithoutSigning(t *testing.T) {
// Build the manifest // Build the manifest
var buf bytes.Buffer var buf bytes.Buffer
err = b.Build(&buf) err = b.Build(context.Background(), &buf)
require.NoError(t, err) require.NoError(t, err)
// Parse the manifest and verify signature fields are empty // Parse the manifest and verify signature fields are empty
@@ -390,3 +396,35 @@ func TestBuilderWithoutSigning(t *testing.T) {
assert.Empty(t, manifest.pbOuter.GetSigningPubKey(), assert.Empty(t, manifest.pbOuter.GetSigningPubKey(),
"signing public key should be empty when not signing") "signing public key should be empty when not signing")
} }
// TestGPGTimeoutKillsGPG puts a fake gpg that never finishes first on
// PATH and checks that a run past its deadline is killed and reported as
// a timeout of the named operation, instead of hanging.
func TestGPGTimeoutKillsGPG(t *testing.T) {
binDir := t.TempDir()
fakeGPG := []byte("#!/bin/sh\nexec sleep 10\n")
//nolint:gosec // G306: the fake gpg has to be executable
require.NoError(t, os.WriteFile(filepath.Join(binDir, "gpg"), fakeGPG, 0o700))
t.Setenv("PATH", binDir+string(os.PathListSeparator)+os.Getenv("PATH"))
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
_, err := gpgSign(ctx, []byte("data"), GPGKeyID("any"))
require.ErrorIs(t, err, context.DeadlineExceeded)
assert.Contains(t, err.Error(), "gpg sign failed: gpg timed out")
}
// TestBuildPassesContextToSigning checks that a caller can cancel the gpg
// runs that sign a manifest through the context given to Build.
func TestBuildPassesContextToSigning(t *testing.T) {
t.Parallel()
b := NewBuilder()
b.SetSigningOptions(&SigningOptions{KeyID: "any"})
ctx, cancel := context.WithCancel(context.Background())
cancel()
require.ErrorIs(t, b.Build(ctx, io.Discard), context.Canceled)
}
+1 -2
View File
@@ -283,8 +283,7 @@ func (s *Scanner) ToManifest(
} }
// Build and write manifest // Build and write manifest
//nolint:contextcheck // Build's GPG signing exec is not cancellable by design return builder.Build(ctx, w)
return builder.Build(w)
} }
// configureBuilder constructs a manifest builder configured from the // configureBuilder constructs a manifest builder configured from the
+9 -8
View File
@@ -2,6 +2,7 @@ package mfer
import ( import (
"bytes" "bytes"
"context"
"crypto/sha256" "crypto/sha256"
"errors" "errors"
"fmt" "fmt"
@@ -50,13 +51,13 @@ func newTimestampFromTime(t time.Time) *Timestamp {
} }
} }
func (m *manifest) generate() error { func (m *manifest) generate(ctx context.Context) error {
if m.pbInner == nil { if m.pbInner == nil {
return errInnerNotSet return errInnerNotSet
} }
if m.pbOuter == nil { if m.pbOuter == nil {
e := m.generateOuter() e := m.generateOuter(ctx)
if e != nil { if e != nil {
return e return e
} }
@@ -77,7 +78,7 @@ func (m *manifest) generate() error {
return nil return nil
} }
func (m *manifest) generateOuter() error { func (m *manifest) generateOuter(ctx context.Context) error {
if m.pbInner == nil { if m.pbInner == nil {
return errInternal return errInternal
} }
@@ -135,7 +136,7 @@ func (m *manifest) generateOuter() error {
// Sign the manifest if signing options are provided // Sign the manifest if signing options are provided
if m.signingOptions != nil && m.signingOptions.KeyID != "" { if m.signingOptions != nil && m.signingOptions.KeyID != "" {
return m.signOuter() return m.signOuter(ctx)
} }
return nil return nil
@@ -143,27 +144,27 @@ func (m *manifest) generateOuter() error {
// signOuter signs the outer message with the configured GPG key and // signOuter signs the outer message with the configured GPG key and
// embeds the signature, signer fingerprint, and public key. // embeds the signature, signer fingerprint, and public key.
func (m *manifest) signOuter() error { func (m *manifest) signOuter(ctx context.Context) error {
sigString, err := m.signatureString() sigString, err := m.signatureString()
if err != nil { if err != nil {
return fmt.Errorf("failed to generate signature string: %w", err) return fmt.Errorf("failed to generate signature string: %w", err)
} }
sig, err := gpgSign([]byte(sigString), m.signingOptions.KeyID) sig, err := gpgSign(ctx, []byte(sigString), m.signingOptions.KeyID)
if err != nil { if err != nil {
return fmt.Errorf("failed to sign manifest: %w", err) return fmt.Errorf("failed to sign manifest: %w", err)
} }
m.pbOuter.Signature = sig m.pbOuter.Signature = sig
fingerprint, err := gpgGetKeyFingerprint(m.signingOptions.KeyID) fingerprint, err := gpgGetKeyFingerprint(ctx, m.signingOptions.KeyID)
if err != nil { if err != nil {
return fmt.Errorf("failed to get key fingerprint: %w", err) return fmt.Errorf("failed to get key fingerprint: %w", err)
} }
m.pbOuter.Signer = fingerprint m.pbOuter.Signer = fingerprint
pubKey, err := gpgExportPublicKey(m.signingOptions.KeyID) pubKey, err := gpgExportPublicKey(ctx, m.signingOptions.KeyID)
if err != nil { if err != nil {
return fmt.Errorf("failed to export public key: %w", err) return fmt.Errorf("failed to export public key: %w", err)
} }
+11 -2
View File
@@ -1,7 +1,8 @@
#!/bin/sh #!/bin/sh
# script/docker: build the Docker image tagged with the project name. # script/docker: build the Docker image tagged with the project name.
# Identical in all repos; the tag comes from script/projectname. # Identical in all repos; the tag comes from script/projectname.
# Generic: needs no adaptation. # --no-cache because the gate phases the final stage depends on are RUN
# steps, and a cached one is a check that did not run.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -9,7 +10,15 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
docker build -t "$("$SCRIPT_DIR/projectname")" . # Own line: a failing command substitution inside an argument does
# not trip `set -e`, so the inline form degrades silently to an
# empty constant. The VERSION build argument takes precedence over
# the version a build stage derives from the .git in the context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown"
docker build --no-cache \
--build-arg VERSION="$version" \
-t "$("$SCRIPT_DIR/projectname")" .
} }
main "$@" main "$@"
+6 -1
View File
@@ -17,7 +17,12 @@ ensure_pb() {
main() { main() {
cd "$ROOT" cd "$ROOT"
ensure_pb ensure_pb
go test -v --timeout 10s ./... go test -timeout 30s -race -cover ./... ||
{
echo "--- Rerunning with -v for details ---"
go test -timeout 30s -race -v ./...
exit 1
}
} }
main "$@" main "$@"