sneak/mfer
1
0
Fork 0
Code Issues 10 Pull Requests 1 Actions Releases Wiki Activity

Compare commits

base: sneak/mfer:next
Branches Tags
sneak/mfer:main sneak/mfer:next sneak/mfer:issue-116-gomodguard
..
compare: sneak/mfer:ef56eeeae09650f6ca13baf1f4079d801bfa00a2
Branches Tags
sneak/mfer:next sneak/mfer:issue-116-gomodguard sneak/mfer:main
1 Commits
next .. ef56eeeae0
Author SHA1 Message Date
clawbot ef56eeeae0 Pin the remaining developer tool installs (closes #68)
check / check (push) Failing after 2s
Details 
gofumpt was pinned only by its version tag. It is now a tool of a
separate module in tools/, so `go tool` builds it from source checked
against the hashes in tools/go.sum, and mfer's own module gains no
dependencies. script/prettier no longer falls back to a prettier on
PATH, and fails when node_modules holds a different version than
package.json pins. The bootstrap comment now says what --frozen-lockfile
really does, package.json drops its made-up version, and the golang
image comment names its exact version.

Model: opus-5-5
 2026-10-04 17:09:48 +00:00
27 changed files with 364 additions and 534 deletions
Expand all files Collapse all files
.dockerignore
-3
View File
@@ -61,6 +61,3 @@
# This repo's own host-built binary (make build). # This repo's own host-built binary (make build).
/bin/mfer /bin/mfer
# The protoc script/bootstrap unpacks for script/generate.
/bin/protoc
.gitignore
-1
View File
@@ -1,5 +1,4 @@
/bin/mfer /bin/mfer
/bin/protoc/
/tmp /tmp
/node_modules/ /node_modules/
.nvmrc
-1
View File
@@ -1 +0,0 @@
22.17.0
Dockerfile
+3 -3
View File
@@ -10,7 +10,7 @@ COPY . .
# Go half of fmt-check only: this image has no node, so no prettier. The # Go half of fmt-check only: this image has no node, so no prettier. The
# markdown half runs in the mdfmt stage below. The image has no gofumpt # markdown half runs in the mdfmt stage below. The image has no gofumpt
# either; script/gofumpt builds the version bin/tools/go.mod pins. # either; script/gofumpt builds the version tools/go.mod requires.
RUN script/gofumpt --check RUN script/gofumpt --check
# The linter directly, not `make lint`: script/lint builds this stage, and # The linter directly, not `make lint`: script/lint builds this stage, and
# there is no docker inside this build. # there is no docker inside this build.
@@ -18,7 +18,7 @@ RUN golangci-lint run --config .golangci.yml ./...
# Markdown/JSON format stage — prettier needs node, which the Go images # Markdown/JSON format stage — prettier needs node, which the Go images
# do not have. node:22.17.0-bookworm-slim (2026-08-09); ships node # do not have. node:22.17.0-bookworm-slim (2026-08-09); ships node
# 22.17.0 and yarn 1.22.22, the versions .nvmrc and script/bootstrap pin. # 22.17.0 and yarn 1.22.22, the versions script/bootstrap pins.
FROM node@sha256:b04ce4ae4e95b522112c2e5c52f781471a5cbc3b594527bcddedee9bc48c03a0 AS mdfmt FROM node@sha256:b04ce4ae4e95b522112c2e5c52f781471a5cbc3b594527bcddedee9bc48c03a0 AS mdfmt
WORKDIR /src WORKDIR /src
@@ -64,7 +64,7 @@ RUN version="${VERSION:-$(git describe --tags --always)}"; \
exit 1; \ exit 1; \
fi; \ fi; \
cd cmd/mfer && \ cd cmd/mfer && \
CGO_ENABLED=0 go build -ldflags "-X main.Gitrev=$version" -o /mfer . CGO_ENABLED=0 go build -tags urfave_cli_no_docs -ldflags "-X main.Gitrev=$version" -o /mfer .
# Fail unless /mfer is statically linked: scratch has no C library to run it. # Fail unless /mfer is statically linked: scratch has no C library to run it.
RUN ldd /mfer 2>&1 | grep -q 'not a dynamic executable' RUN ldd /mfer 2>&1 | grep -q 'not a dynamic executable'
README.md
+19 -24
View File
@@ -67,13 +67,9 @@ standard: normalized scripts in `script/` are the entrypoints for the
development workflow, and the Makefile targets are thin shims that call them. We development workflow, and the Makefile targets are thin shims that call them. We
provide: provide:
- `script/bootstrap` — install all dependencies, idempotently: Go and the - `script/bootstrap` — install all dependencies (Go, Go module download, and
modules of both `go.mod` and `bin/tools/go.mod`; node (the version `.nvmrc` node/yarn plus the prettier version pinned in `package.json`/`yarn.lock`),
names, through nvm when there is no node on `PATH`) and yarn, plus the idempotently; golangci-lint is not installed, it runs only in Docker
prettier version pinned in `package.json`/`yarn.lock`; and `protoc` 33.4,
unpacked into `bin/protoc` from its release archive once the archive matches
the sha256 the script holds for this platform. golangci-lint is not installed,
it runs only in Docker
- `script/setup` — make a fresh clone ready for development: runs - `script/setup` — make a fresh clone ready for development: runs
`script/bootstrap`, then `script/install-precommit` `script/bootstrap`, then `script/install-precommit`
- `script/projectname` — output the project name (`mfer`); used by other scripts - `script/projectname` — output the project name (`mfer`); used by other scripts
@@ -86,10 +82,14 @@ provide:
- `script/generate` (`make generate`) — regenerate `mfer/mf.pb.go` from - `script/generate` (`make generate`) — regenerate `mfer/mf.pb.go` from
`mfer/mf.proto` and record the hash of that `mfer/mf.proto` in `mfer/mf.proto` and record the hash of that `mfer/mf.proto` in
`mfer/mf.proto.sha256`; the only thing that regenerates the committed `mfer/mf.proto.sha256`; the only thing that regenerates the committed
`mfer/mf.pb.go`. It runs the `protoc` that `script/bootstrap` unpacks into `mfer/mf.pb.go`. It needs the exact versions that wrote the committed file,
`bin/protoc`, refusing any version but 33.4, and the `protoc-gen-go` that and refuses to run with any other: `protoc` 33.4 (unpack
`bin/tools/go.mod` pins, which `go tool` builds from source checked against `protoc-33.4-<platform>.zip` from
the hashes in `bin/tools/go.sum` [its release](https://github.com/protocolbuffers/protobuf/releases/tag/v33.4)
and put its `bin/protoc` on `PATH`) and `protoc-gen-go` v1.36.11
(`go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11`, which
installs it in `$(go env GOPATH)/bin`; `script/generate` adds that directory
to `PATH`)
- `script/fuzz` — fuzz the manifest parser for one minute; run by hand - `script/fuzz` — fuzz the manifest parser for one minute; run by hand
(`make fuzz`), never by CI, while `script/test` runs its committed seed corpus (`make fuzz`), never by CI, while `script/test` runs its committed seed corpus
as ordinary tests as ordinary tests
@@ -99,17 +99,15 @@ provide:
- `script/fmt` — format all code and docs (writes): `script/gofumpt --write` and - `script/fmt` — format all code and docs (writes): `script/gofumpt --write` and
`script/prettier --write` `script/prettier --write`
- `script/gofumpt` — run `gofumpt` over every Go file in the repository in the - `script/gofumpt` — run `gofumpt` over every Go file in the repository in the
given mode, `--write` or `--check`, at the version `bin/tools/go.mod` pins given mode, `--write` or `--check`, at the version `tools/go.mod` requires
(built on demand by `go tool` from source checked against the hashes in (built on demand by `go tool` from source checked against the hashes in
`bin/tools/go.sum`, so nothing installs it); `script/fmt`, `script/fmt-check` `tools/go.sum`, so nothing installs it); `script/fmt`, `script/fmt-check` and
and the Docker lint stage all go through it, so they cannot disagree about Go the Docker lint stage all go through it, so they cannot disagree about Go
formatting formatting
- `script/prettier` — run prettier over the repository's canonical file set - `script/prettier` — run prettier over the repository's canonical file set
(Markdown and JSON, minus `.prettierignore`) in the given mode, `--write` or (Markdown and JSON, minus `.prettierignore`) in the given mode, `--write` or
`--check`, with the prettier version `yarn.lock` pins, run by the node on `--check`; the single definition of that file set, so `script/fmt` and
`PATH` or else the one `script/bootstrap` installed through nvm; the single `script/fmt-check` cannot disagree about it
definition of that file set, so `script/fmt` and `script/fmt-check` cannot
disagree about it
- `script/fmt-check` — check formatting without writing: - `script/fmt-check` — check formatting without writing:
`script/gofumpt --check` plus `script/prettier --check` `script/gofumpt --check` plus `script/prettier --check`
- `script/check` — run `script/test`, `script/lint`, and `script/fmt-check` - `script/check` — run `script/test`, `script/lint`, and `script/fmt-check`
@@ -271,12 +269,9 @@ are now tracked only in the [issues](https://git.eeqj.de/sneak/mfer/issues).
cryptographic integrity of downloaded files. A file already there with the cryptographic integrity of downloaded files. A file already there with the
size and hash the manifest lists is skipped. Once every file is in place, size and hash the manifest lists is skipped. Once every file is in place,
the manifest is saved there as `index.mf`, so `mfer check` can verify the the manifest is saved there as `index.mf`, so `mfer check` can verify the
tree later. Each file is downloaded to a temp file beside it, such as tree later. A manifest that lists `index.mf` (in any letter case) or
`.a.txt.tmp` for `a.txt`, then moved into place. A manifest is refused `.index.mf.tmp` at the top of the tree is refused before any file is
before any file is downloaded if it lists a file where fetch writes downloaded, since saving the manifest would replace it.
another: at the temp file of a listed file, or at `index.mf` or
`.index.mf.tmp` at the top of the tree. Names are compared in any letter
case.
- `mfer fetch --require-signature <fingerprint> https://example.com/stuff/` - `mfer fetch --require-signature <fingerprint> https://example.com/stuff/`
- as above, but first refuses a manifest not signed by the key with that - as above, but first refuses a manifest not signed by the key with that
fingerprint, as `mfer check --require-signature` does, before downloading fingerprint, as `mfer check --require-signature` does, before downloading
bin/tools/go.mod
-22
View File
@@ -1,22 +0,0 @@
// The developer tools this repo runs with `go tool`: gofumpt for
// script/gofumpt and protoc-gen-go for script/generate. Kept out of the mfer
// module so they add nothing to what mfer's users download. `go tool` builds
// exactly the source whose hashes go.sum here records.
module sneak.berlin/go/mfer/bin/tools
go 1.26.0
tool (
google.golang.org/protobuf/cmd/protoc-gen-go
mvdan.cc/gofumpt
)
require (
golang.org/x/mod v0.40.0 // indirect
golang.org/x/sync v0.22.0 // indirect
golang.org/x/tools v0.49.0 // indirect
// protoc-gen-go v1.36.11, 2026-10-04
google.golang.org/protobuf v1.36.11 // indirect
// gofumpt v0.12.0, 2026-10-04
mvdan.cc/gofumpt v0.12.0 // indirect
)
go.mod
+9 -3
View File
@@ -9,21 +9,27 @@ require (
github.com/klauspost/compress v1.18.2 github.com/klauspost/compress v1.18.2
github.com/multiformats/go-multihash v0.2.3 github.com/multiformats/go-multihash v0.2.3
github.com/spf13/afero v1.8.0 github.com/spf13/afero v1.8.0
github.com/stretchr/testify v1.12.1 github.com/stretchr/testify v1.8.1
github.com/urfave/cli/v3 v3.14.0 github.com/urfave/cli/v2 v2.27.7
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 golang.org/x/term v0.0.0-20210927222741-03fcf44c2211
google.golang.org/protobuf v1.28.1 google.golang.org/protobuf v1.28.1
) )
require ( require (
github.com/cpuguy83/go-md2man/v2 v2.0.7 // indirect
github.com/klauspost/cpuid/v2 v2.0.9 // indirect github.com/klauspost/cpuid/v2 v2.0.9 // indirect
github.com/kr/pretty v0.2.0 // indirect
github.com/minio/sha256-simd v1.0.0 // indirect github.com/minio/sha256-simd v1.0.0 // indirect
github.com/mr-tron/base58 v1.2.0 // indirect github.com/mr-tron/base58 v1.2.0 // indirect
github.com/multiformats/go-varint v0.0.6 // indirect github.com/multiformats/go-varint v0.0.6 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/spaolacci/murmur3 v1.1.0 // indirect github.com/spaolacci/murmur3 v1.1.0 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1 // indirect
golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e // indirect golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e // indirect
golang.org/x/sys v0.1.0 // indirect golang.org/x/sys v0.1.0 // indirect
golang.org/x/text v0.3.6 // indirect golang.org/x/text v0.3.6 // indirect
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
lukechampine.com/blake3 v1.1.6 // indirect lukechampine.com/blake3 v1.1.6 // indirect
) )
go.sum
+22 -6
View File
@@ -46,6 +46,8 @@ github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDk
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc= github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/cpuguy83/go-md2man/v2 v2.0.7 h1:zbFlGlXEAKlwXpmvle3d8Oe3YnkKIK4xSRTd3sHPnBo=
github.com/cpuguy83/go-md2man/v2 v2.0.7/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -132,7 +134,10 @@ github.com/klauspost/cpuid/v2 v2.0.9 h1:lgaqFMSdTdQYdZ04uHyN2d/eKdOMyi2YLSvlQIBF
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg= github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg= github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.2.0 h1:s5hAObm+yFO5uHYt5dYjxi2rXrsnmRpJx4OYvIWUaQs=
github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/minio/sha256-simd v1.0.0 h1:v1ta+49hkWZyvaKwrQB8elexRqm6Y0aMLjCNsrYxo6g= github.com/minio/sha256-simd v1.0.0 h1:v1ta+49hkWZyvaKwrQB8elexRqm6Y0aMLjCNsrYxo6g=
github.com/minio/sha256-simd v1.0.0/go.mod h1:OuYzVNI5vcoYIAmbIvHPl3N3jUzVedXbKy5RFepssQM= github.com/minio/sha256-simd v1.0.0/go.mod h1:OuYzVNI5vcoYIAmbIvHPl3N3jUzVedXbKy5RFepssQM=
@@ -144,21 +149,30 @@ github.com/multiformats/go-varint v0.0.6 h1:gk85QWKxh3TazbLxED/NlDVv8+q+ReFJk7Y2
github.com/multiformats/go-varint v0.0.6/go.mod h1:3Ls8CIEsrijN6+B7PbrXRPxHRPuXSrVKRY101jdMZYE= github.com/multiformats/go-varint v0.0.6/go.mod h1:3Ls8CIEsrijN6+B7PbrXRPxHRPuXSrVKRY101jdMZYE=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/sftp v1.13.1/go.mod h1:3HaPG6Dq1ILlpPZRO0HVMrsydcdLt6HRDccSgb87qRg= github.com/pkg/sftp v1.13.1/go.mod h1:3HaPG6Dq1ILlpPZRO0HVMrsydcdLt6HRDccSgb87qRg=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI= github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI=
github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/spf13/afero v1.8.0 h1:5MmtuhAgYeU6qpa7w7bP0dv6MBYuup0vekhSpSkoq60= github.com/spf13/afero v1.8.0 h1:5MmtuhAgYeU6qpa7w7bP0dv6MBYuup0vekhSpSkoq60=
github.com/spf13/afero v1.8.0/go.mod h1:CtAatgMJh6bJEIs48Ay/FOnkljP3WeGUG0MC1RfAqwo= github.com/spf13/afero v1.8.0/go.mod h1:CtAatgMJh6bJEIs48Ay/FOnkljP3WeGUG0MC1RfAqwo=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE= github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg= github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/urfave/cli/v3 v3.14.0 h1:a8414NQlHJs0c/iBsulKLzlES0n/lEAskbL2LKpU4/s= github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
github.com/urfave/cli/v3 v3.14.0/go.mod h1:vXn6HxPNccJSzQr2QvwVncOKrgYGIHU0HY5h8B2nQj4= github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/urfave/cli/v2 v2.27.7 h1:bH59vdhbjLv3LAvIu6gd0usJHgoTTPhCFib8qqOwXYU=
github.com/urfave/cli/v2 v2.27.7/go.mod h1:CyNAG/xg+iAOg0N4MPGZqVmv2rCoP267496AOXUZjA4=
github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1 h1:gEOO8jv9F4OT7lGCjxCBTO/36wtF6j2nSip77qHd4x4=
github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1/go.mod h1:Ohn+xnUBiLI6FVj/9LpzZWtj1/D6lUovWYBkxHVV3aM=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
@@ -169,8 +183,6 @@ go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk= go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
@@ -458,9 +470,13 @@ google.golang.org/protobuf v1.28.1 h1:d0NfwRgPtno5B1Wa6L2DAG+KivqkdutMf1UhdNx175
google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
internal/cli/check.go
+17 -26
View File
@@ -16,7 +16,7 @@ import (
"github.com/dustin/go-humanize" "github.com/dustin/go-humanize"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/urfave/cli/v3" "github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/internal/log" "sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -96,10 +96,8 @@ func findManifest(fs afero.Fs, dir string) (string, error) {
// fetchManifestToTemp downloads a manifest URL to a temporary file and // fetchManifestToTemp downloads a manifest URL to a temporary file and
// returns the temp file path. The caller is responsible for removing it. // returns the temp file path. The caller is responsible for removing it.
func (mfa *CLIApp) fetchManifestToTemp( func (mfa *CLIApp) fetchManifestToTemp(url string) (string, error) {
ctx context.Context, url string, rc, fetchErr := mfa.openManifestReader(url)
) (string, error) {
rc, fetchErr := mfa.openManifestReader(ctx, url)
if fetchErr != nil { if fetchErr != nil {
return "", fetchErr return "", fetchErr
} }
@@ -210,15 +208,13 @@ func countCheckFailures(
// findExtraFiles reports files present on disk but absent from the // findExtraFiles reports files present on disk but absent from the
// manifest, and anything the search cannot read: each is a failure under // manifest, and anything the search cannot read: each is a failure under
// --no-extra-files, otherwise a warning. // --no-extra-files, otherwise a warning.
func findExtraFiles( func findExtraFiles(ctx *cli.Context, chk *mfer.Checker, failures *int64) error {
ctx context.Context, cmd *cli.Command, chk *mfer.Checker, failures *int64,
) error {
extraResults := make(chan mfer.Result, 1) extraResults := make(chan mfer.Result, 1)
extraDone := make(chan struct{}) extraDone := make(chan struct{})
go func() { go func() {
for result := range extraResults { for result := range extraResults {
if cmd.Bool("no-extra-files") { if ctx.Bool("no-extra-files") {
*failures++ *failures++
log.Infof("%s: %s (%s)", result.Status, result.Path, result.Message) log.Infof("%s: %s (%s)", result.Status, result.Path, result.Message)
@@ -230,7 +226,7 @@ func findExtraFiles(
close(extraDone) close(extraDone)
}() }()
err := chk.FindExtraFiles(ctx, extraResults) err := chk.FindExtraFiles(ctx.Context, extraResults)
if err != nil { if err != nil {
return fmt.Errorf("failed to check for extra files: %w", err) return fmt.Errorf("failed to check for extra files: %w", err)
} }
@@ -242,9 +238,7 @@ func findExtraFiles(
// runCheck runs the manifest check with progress and result reporting // runCheck runs the manifest check with progress and result reporting
// and returns the number of failures. // and returns the number of failures.
func runCheck( func runCheck(ctx *cli.Context, chk *mfer.Checker, showProgress bool) (int64, error) {
ctx context.Context, cmd *cli.Command, chk *mfer.Checker, showProgress bool,
) (int64, error) {
// Set up results channel // Set up results channel
results := make(chan mfer.Result, 1) results := make(chan mfer.Result, 1)
@@ -270,7 +264,7 @@ func runCheck(
go countCheckFailures(results, &failures, done) go countCheckFailures(results, &failures, done)
// Run check // Run check
err := chk.Check(ctx, results, progress) err := chk.Check(ctx.Context, results, progress)
progressWg.Wait() progressWg.Wait()
@@ -281,7 +275,7 @@ func runCheck(
// Wait for results processing to complete // Wait for results processing to complete
<-done <-done
err = findExtraFiles(ctx, cmd, chk, &failures) err = findExtraFiles(ctx, chk, &failures)
if err != nil { if err != nil {
return 0, err return 0, err
} }
@@ -289,19 +283,17 @@ func runCheck(
return failures, nil return failures, nil
} }
func (mfa *CLIApp) checkManifestOperation( func (mfa *CLIApp) checkManifestOperation(ctx *cli.Context) error {
ctx context.Context, cmd *cli.Command,
) error {
log.Debug("checkManifestOperation()") log.Debug("checkManifestOperation()")
manifestPath, err := mfa.resolveManifestArg(cmd) manifestPath, err := mfa.resolveManifestArg(ctx)
if err != nil { if err != nil {
return fmt.Errorf("check: %w", err) return fmt.Errorf("check: %w", err)
} }
// URL manifests need to be downloaded to a temp file for the checker // URL manifests need to be downloaded to a temp file for the checker
if isHTTPURL(manifestPath) { if isHTTPURL(manifestPath) {
tmpPath, tmpErr := mfa.fetchManifestToTemp(ctx, manifestPath) tmpPath, tmpErr := mfa.fetchManifestToTemp(manifestPath)
if tmpErr != nil { if tmpErr != nil {
return fmt.Errorf("check: %w", tmpErr) return fmt.Errorf("check: %w", tmpErr)
} }
@@ -311,13 +303,12 @@ func (mfa *CLIApp) checkManifestOperation(
manifestPath = tmpPath manifestPath = tmpPath
} }
basePath := cmd.String("base") basePath := ctx.String("base")
showProgress := cmd.Bool("progress") showProgress := ctx.Bool("progress")
log.Infof("checking manifest %s with base %s", manifestPath, basePath) log.Infof("checking manifest %s with base %s", manifestPath, basePath)
// Create checker // Create checker
//nolint:contextcheck // mfer loads a manifest without a context
chk, err := mfer.NewChecker(&mfer.CheckerOptions{ chk, err := mfer.NewChecker(&mfer.CheckerOptions{
ManifestPath: manifestPath, ManifestPath: manifestPath,
BasePath: basePath, BasePath: basePath,
@@ -328,9 +319,9 @@ func (mfa *CLIApp) checkManifestOperation(
} }
// Check signature requirement // Check signature requirement
requiredSigner := cmd.String(flagRequireSignature) requiredSigner := ctx.String(flagRequireSignature)
if requiredSigner != "" { if requiredSigner != "" {
err = verifyRequiredSigner(ctx, chk, requiredSigner) err = verifyRequiredSigner(ctx.Context, chk, requiredSigner)
if err != nil { if err != nil {
return err return err
} }
@@ -339,7 +330,7 @@ func (mfa *CLIApp) checkManifestOperation(
log.Infof("manifest contains %d files, %s", chk.FileCount(), log.Infof("manifest contains %d files, %s", chk.FileCount(),
humanize.IBytes(safeUint64(int64(chk.TotalBytes())))) humanize.IBytes(safeUint64(int64(chk.TotalBytes()))))
failures, err := runCheck(ctx, cmd, chk, showProgress) failures, err := runCheck(ctx, chk, showProgress)
if err != nil { if err != nil {
return err return err
} }
internal/cli/entry_test.go
+25 -21
View File
@@ -18,7 +18,7 @@ import (
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
urfcli "github.com/urfave/cli/v3" urfcli "github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/internal/log" "sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -264,9 +264,8 @@ func commandsTakingVerbose() [][]string {
} }
// TestVerboseCount asserts that one -v or --verbose gives verbose output and // TestVerboseCount asserts that one -v or --verbose gives verbose output and
// two -v, or -v and --verbose, give debug output (issue #125). urfave/cli // two -v give debug output (issue #125). urfave/cli before v2.25.5 counted a
// before v2.25.5 counted a flag given by its alias twice, so one -v gave debug // flag given by its alias twice, so one -v gave debug output.
// output.
func TestVerboseCount(t *testing.T) { func TestVerboseCount(t *testing.T) {
t.Parallel() t.Parallel()
@@ -277,7 +276,6 @@ func TestVerboseCount(t *testing.T) {
{[]string{"-v"}, log.VerboseLevel}, {[]string{"-v"}, log.VerboseLevel},
{[]string{testFlagVerbose}, log.VerboseLevel}, {[]string{testFlagVerbose}, log.VerboseLevel},
{[]string{"-v", "-v"}, log.DebugLevel}, {[]string{"-v", "-v"}, log.DebugLevel},
{[]string{"-v", testFlagVerbose}, log.DebugLevel},
} }
for _, command := range commandsTakingVerbose() { for _, command := range commandsTakingVerbose() {
@@ -315,6 +313,28 @@ func TestCombinedShortVerboseRefused(t *testing.T) {
} }
} }
// TestShortAndLongVerboseRefused asserts that -v and --verbose given together
// are refused (issue #125): urfave/cli v2 refuses a flag given under two of its
// names, and one flag with an alias keeps help and parsing simple.
func TestShortAndLongVerboseRefused(t *testing.T) {
t.Parallel()
for _, command := range commandsTakingVerbose() {
args := slices.Concat(command, []string{"-v", testFlagVerbose})
t.Run(strings.Join(args, " "), func(t *testing.T) {
t.Parallel()
opts := testOpts(args, afero.NewMemMapFs())
exitCode, level := runCLIWithLevel(opts)
assert.Equal(t, 1, exitCode)
assert.Contains(t, testStderr(t, opts), "Cannot use two forms of the same flag")
assert.Equal(t, log.InfoLevel, level)
})
}
}
func TestHelpCommand(t *testing.T) { func TestHelpCommand(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1170,22 +1190,6 @@ func TestGenerateValidatesInputPaths(t *testing.T) {
}) })
} }
// TestFlagAfterArgumentIsArgument asserts that flags are read only before a
// command's first argument: after it, -v is a path, not the verbose flag.
func TestFlagAfterArgumentIsArgument(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll(testDir, 0o755))
writeTestFile(t, fs, testFile1, "content")
opts := testOpts([]string{testApp, cmdGenerate, testDir, "-v"}, fs)
exitCode := runCLI(opts)
assert.Equal(t, 1, exitCode)
assert.Contains(t, testStderr(t, opts), "path does not exist: -v")
}
func TestCheckDetectsManifestCorruption(t *testing.T) { func TestCheckDetectsManifestCorruption(t *testing.T) {
t.Parallel() t.Parallel()
internal/cli/errmsg_test.go
+31 -33
View File
@@ -4,6 +4,7 @@ package cli
import ( import (
"bytes" "bytes"
"context" "context"
"flag"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"os" "os"
@@ -14,7 +15,7 @@ import (
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
urfcli "github.com/urfave/cli/v3" urfcli "github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -194,17 +195,13 @@ func signedChecker(t *testing.T, manifest []byte) *mfer.Checker {
func TestPathDoesNotExistMessage(t *testing.T) { func TestPathDoesNotExistMessage(t *testing.T) {
t.Parallel() t.Parallel()
set := flag.NewFlagSet("gen", flag.ContinueOnError)
require.NoError(t, set.Parse([]string{"nope"}))
mfa := &CLIApp{Fs: afero.NewMemMapFs()} mfa := &CLIApp{Fs: afero.NewMemMapFs()}
cmd := &urfcli.Command{ ctx := urfcli.NewContext(nil, set, nil)
Name: cmdGenerate,
Action: func(_ context.Context, c *urfcli.Command) error {
_, err := mfa.collectInputPaths(c.Args())
return err _, err := mfa.collectInputPaths(ctx.Args())
},
}
err := cmd.Run(context.Background(), []string{cmdGenerate, "nope"})
require.ErrorIs(t, err, errPathNotExist) require.ErrorIs(t, err, errPathNotExist)
assert.EqualError(t, err, "path does not exist: nope") assert.EqualError(t, err, "path does not exist: nope")
} }
@@ -217,22 +214,18 @@ func TestOutputFileExistsMessage(t *testing.T) {
require.NoError(t, afero.WriteFile(fs, "/d/f.txt", []byte("hi"), 0o644)) require.NoError(t, afero.WriteFile(fs, "/d/f.txt", []byte("hi"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/out.mf", []byte("old"), 0o644)) require.NoError(t, afero.WriteFile(fs, "/out.mf", []byte("old"), 0o644))
set := flag.NewFlagSet("gen", flag.ContinueOnError)
set.String("output", "", "")
set.Bool("force", false, "")
require.NoError(t, set.Parse([]string{"/d"}))
require.NoError(t, set.Set("output", "/out.mf"))
mfa := &CLIApp{Fs: fs} mfa := &CLIApp{Fs: fs}
cmd := &urfcli.Command{ ctx := urfcli.NewContext(nil, set, nil)
Name: cmdGenerate,
Flags: []urfcli.Flag{
&urfcli.StringFlag{Name: "output"},
&urfcli.BoolFlag{Name: "force"},
},
Action: mfa.generateManifestOperation,
}
// generateManifestOperation writes to the process-global logger during // generateManifestOperation writes to the process-global logger during
// enumeration, so serialize with the other CLI runs. // enumeration, so serialize with the other CLI runs.
err := runLocked(func() error { err := runLocked(func() error { return mfa.generateManifestOperation(ctx) })
return cmd.Run(context.Background(),
[]string{cmdGenerate, "--output", "/out.mf", "/d"})
})
require.ErrorIs(t, err, errOutputExists) require.ErrorIs(t, err, errOutputExists)
assert.EqualError(t, err, assert.EqualError(t, err,
"output file /out.mf already exists (use --force to overwrite)") "output file /out.mf already exists (use --force to overwrite)")
@@ -257,7 +250,7 @@ func TestUnknownCommandMessage(t *testing.T) {
err := runLocked(func() error { err := runLocked(func() error {
mfa.run([]string{testApp}) mfa.run([]string{testApp})
return mfa.app.Run(context.Background(), []string{testApp, "bogus"}) return mfa.app.Run([]string{testApp, "bogus"})
}) })
require.ErrorIs(t, err, errUnknownCommand) require.ErrorIs(t, err, errUnknownCommand)
assert.EqualError(t, err, `unknown command "bogus"`) assert.EqualError(t, err, `unknown command "bogus"`)
@@ -274,7 +267,7 @@ func TestManifestLoaderHTTPStatusMessage(t *testing.T) {
mfa := &CLIApp{Fs: afero.NewMemMapFs()} mfa := &CLIApp{Fs: afero.NewMemMapFs()}
_, err := mfa.openManifestReader(context.Background(), server.URL+"/foo.mf") _, err := mfa.openManifestReader(server.URL + "/foo.mf")
require.ErrorIs(t, err, errHTTPStatus) require.ErrorIs(t, err, errHTTPStatus)
assert.EqualError(t, err, assert.EqualError(t, err,
"failed to fetch "+server.URL+"/foo.mf: HTTP 404") "failed to fetch "+server.URL+"/foo.mf: HTTP 404")
@@ -291,13 +284,17 @@ func TestFetchManifestHTTPStatusMessage(t *testing.T) {
mfa := &CLIApp{Fs: afero.NewMemMapFs()} mfa := &CLIApp{Fs: afero.NewMemMapFs()}
cmd := mfa.fetchCommand() set := flag.NewFlagSet(cmdFetch, flag.ContinueOnError)
cmd.Action = mfa.fetchManifestOperation for _, f := range mfa.fetchCommand().Flags {
require.NoError(t, f.Apply(set))
}
require.NoError(t, set.Parse([]string{server.URL}))
ctx := urfcli.NewContext(nil, set, nil)
// fetchManifestOperation logs to the process-global logger. // fetchManifestOperation logs to the process-global logger.
err := runLocked(func() error { err := runLocked(func() error { return mfa.fetchManifestOperation(ctx) })
return cmd.Run(context.Background(), []string{cmdFetch, server.URL})
})
require.ErrorIs(t, err, errHTTPStatus) require.ErrorIs(t, err, errHTTPStatus)
assert.EqualError(t, err, "failed to fetch manifest: HTTP 404") assert.EqualError(t, err, "failed to fetch manifest: HTTP 404")
} }
@@ -323,13 +320,14 @@ func TestFetchFileHTTPStatusMessage(t *testing.T) {
func TestURLRequiredMessage(t *testing.T) { func TestURLRequiredMessage(t *testing.T) {
t.Parallel() t.Parallel()
set := flag.NewFlagSet("fetch", flag.ContinueOnError)
require.NoError(t, set.Parse([]string{}))
mfa := &CLIApp{Fs: afero.NewMemMapFs()} mfa := &CLIApp{Fs: afero.NewMemMapFs()}
cmd := &urfcli.Command{Name: cmdFetch, Action: mfa.fetchManifestOperation} ctx := urfcli.NewContext(nil, set, nil)
// fetchManifestOperation logs to the process-global logger. // fetchManifestOperation logs to the process-global logger.
err := runLocked(func() error { err := runLocked(func() error { return mfa.fetchManifestOperation(ctx) })
return cmd.Run(context.Background(), []string{cmdFetch})
})
require.ErrorIs(t, err, errURLRequired) require.ErrorIs(t, err, errURLRequired)
assert.EqualError(t, err, "URL argument required") assert.EqualError(t, err, "URL argument required")
} }
internal/cli/export.go
+4 -8
View File
@@ -1,13 +1,12 @@
package cli package cli
import ( import (
"context"
"encoding/hex" "encoding/hex"
"encoding/json" "encoding/json"
"fmt" "fmt"
"time" "time"
"github.com/urfave/cli/v3" "github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -20,22 +19,19 @@ type ExportEntry struct {
Ctime *string `json:"ctime,omitempty"` Ctime *string `json:"ctime,omitempty"`
} }
func (mfa *CLIApp) exportManifestOperation( func (mfa *CLIApp) exportManifestOperation(ctx *cli.Context) error {
ctx context.Context, cmd *cli.Command, pathOrURL, err := mfa.resolveManifestArg(ctx)
) error {
pathOrURL, err := mfa.resolveManifestArg(cmd)
if err != nil { if err != nil {
return fmt.Errorf("export: %w", err) return fmt.Errorf("export: %w", err)
} }
rc, err := mfa.openManifestReader(ctx, pathOrURL) rc, err := mfa.openManifestReader(pathOrURL)
if err != nil { if err != nil {
return fmt.Errorf("export: %w", err) return fmt.Errorf("export: %w", err)
} }
defer func() { _ = rc.Close() }() defer func() { _ = rc.Close() }()
//nolint:contextcheck // mfer loads a manifest without a context
manifest, err := mfer.NewManifestFromReader(rc) manifest, err := mfer.NewManifestFromReader(rc)
if err != nil { if err != nil {
return fmt.Errorf("export: failed to parse manifest: %w", err) return fmt.Errorf("export: failed to parse manifest: %w", err)
internal/cli/fetch.go
+30 -52
View File
@@ -20,7 +20,7 @@ import (
"github.com/dustin/go-humanize" "github.com/dustin/go-humanize"
"github.com/multiformats/go-multihash" "github.com/multiformats/go-multihash"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/urfave/cli/v3" "github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/internal/log" "sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -91,10 +91,10 @@ var (
// errHashMismatch indicates a downloaded file whose hash matches no // errHashMismatch indicates a downloaded file whose hash matches no
// manifest hash. // manifest hash.
errHashMismatch = errors.New("hash mismatch") errHashMismatch = errors.New("hash mismatch")
// errNameClash indicates a manifest that lists a file where fetch // errManifestNameListed indicates a manifest that lists a file where
// writes another file. // fetch saves the manifest.
errNameClash = errors.New( errManifestNameListed = errors.New(
"manifest lists a file where fetch writes another file") "manifest lists a file where fetch saves the manifest")
) )
// DownloadProgress reports the progress of a single file download. // DownloadProgress reports the progress of a single file download.
@@ -328,21 +328,19 @@ func alreadyPresent(dest, localPath string, entry *mfer.MFFilePath) bool {
return verifyDownloadedHash(h.Sum(nil), entry) == nil return verifyDownloadedHash(h.Sum(nil), entry) == nil
} }
func (mfa *CLIApp) fetchManifestOperation( func (mfa *CLIApp) fetchManifestOperation(ctx *cli.Context) error {
ctx context.Context, cmd *cli.Command,
) error {
log.Debug("fetchManifestOperation()") log.Debug("fetchManifestOperation()")
if cmd.Args().Len() == 0 { if ctx.Args().Len() == 0 {
return errURLRequired return errURLRequired
} }
timeout := cmd.Duration(flagTimeout) timeout := ctx.Duration(flagTimeout)
if timeout <= 0 { if timeout <= 0 {
return errInvalidTimeout return errInvalidTimeout
} }
manifestURL, err := resolveManifestURL(cmd.Args().Get(0)) manifestURL, err := resolveManifestURL(ctx.Args().Get(0))
if err != nil { if err != nil {
return fmt.Errorf("invalid URL: %w", err) return fmt.Errorf("invalid URL: %w", err)
} }
@@ -352,7 +350,7 @@ func (mfa *CLIApp) fetchManifestOperation(
firstDelay: firstRetryDelay, firstDelay: firstRetryDelay,
} }
manifestData, files, err := fetchManifest(ctx, cmd, client, manifestURL) manifestData, files, err := fetchManifest(ctx, client, manifestURL)
if err != nil { if err != nil {
return err return err
} }
@@ -363,7 +361,7 @@ func (mfa *CLIApp) fetchManifestOperation(
return err return err
} }
dest := cmd.String(flagDest) dest := ctx.String(flagDest)
err = os.MkdirAll(dest, dirPerms) err = os.MkdirAll(dest, dirPerms)
if err != nil { if err != nil {
@@ -381,7 +379,7 @@ func (mfa *CLIApp) fetchManifestOperation(
// Download each file // Download each file
downloaded, downloadedBytes, dlErr := downloadManifestFiles( downloaded, downloadedBytes, dlErr := downloadManifestFiles(
ctx, client, baseURL, dest, files, progress) ctx.Context, client, baseURL, dest, files, progress)
close(progress) close(progress)
<-done <-done
@@ -414,11 +412,11 @@ func (mfa *CLIApp) fetchManifestOperation(
// fetchManifest downloads the manifest at manifestURL and parses it, // fetchManifest downloads the manifest at manifestURL and parses it,
// enforcing --require-signature if it is given and refusing a manifest // enforcing --require-signature if it is given and refusing a manifest
// that lists a file where fetch writes another. It returns the manifest as // that lists a file where it will be saved. It returns the manifest as
// downloaded, to be saved once the files are in place, and the files it // downloaded, to be saved once the files are in place, and the files it
// lists. // lists.
func fetchManifest( func fetchManifest(
ctx context.Context, cmd *cli.Command, client retryingClient, manifestURL string, ctx *cli.Context, client retryingClient, manifestURL string,
) ([]byte, []*mfer.MFFilePath, error) { ) ([]byte, []*mfer.MFFilePath, error) {
log.Infof("fetching manifest from %s", manifestURL) log.Infof("fetching manifest from %s", manifestURL)
@@ -427,7 +425,7 @@ func fetchManifest(
// manifest. // manifest.
var manifestData []byte var manifestData []byte
err := client.get(ctx, manifestURL, func(resp *http.Response) error { err := client.get(ctx.Context, manifestURL, func(resp *http.Response) error {
var readErr error var readErr error
manifestData, readErr = io.ReadAll(resp.Body) manifestData, readErr = io.ReadAll(resp.Body)
@@ -439,13 +437,12 @@ func fetchManifest(
} }
// Parse manifest // Parse manifest
//nolint:contextcheck // mfer loads a manifest without a context
manifest, err := mfer.NewManifestFromReader(bytes.NewReader(manifestData)) manifest, err := mfer.NewManifestFromReader(bytes.NewReader(manifestData))
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("failed to parse manifest: %w", err) return nil, nil, fmt.Errorf("failed to parse manifest: %w", err)
} }
requiredSigner := cmd.String(flagRequireSignature) requiredSigner := ctx.String(flagRequireSignature)
if requiredSigner != "" { if requiredSigner != "" {
err = verifyFetchedSigner(ctx, manifestData, requiredSigner) err = verifyFetchedSigner(ctx, manifestData, requiredSigner)
if err != nil { if err != nil {
@@ -455,7 +452,7 @@ func fetchManifest(
files := manifest.Files() files := manifest.Files()
err = checkNoNameClash(files) err = checkManifestNameUnlisted(files)
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
} }
@@ -465,37 +462,19 @@ func fetchManifest(
return manifestData, files, nil return manifestData, files, nil
} }
// checkNoNameClash returns an error if files lists a file, or a directory // checkManifestNameUnlisted returns an error if files lists a file or
// a file is in, under a name where fetch writes another file: the temp // directory at the top of the tree under the name fetch saves the
// file it downloads a listed file to, or, at the top of the tree, the // manifest as, or under that name's temp file. Saving the manifest would
// saved manifest or its temp file. fetch would remove or replace what is // replace or remove it, or fail once every file was downloaded, leaving a
// listed there, or fail partway, leaving a tree check rejects. Names are // tree check rejects. Names are compared ignoring case, since on a
// compared ignoring case, since on a case-insensitive filesystem INDEX.MF // case-insensitive filesystem INDEX.MF and index.mf are one file.
// and index.mf are one file. func checkManifestNameUnlisted(files []*mfer.MFFilePath) error {
func checkNoNameClash(files []*mfer.MFFilePath) error {
sep := string(filepath.Separator)
// written maps each name fetch writes, other than the listed files
// themselves, in lower case, to the file it writes there.
written := map[string]string{
defaultManifestName: "the saved manifest",
tempPathFor(defaultManifestName): "the saved manifest's temp file",
}
for _, f := range files { for _, f := range files {
tmpPath := tempPathFor(filepath.Clean(f.GetPath())) top, _, _ := strings.Cut(filepath.Clean(f.GetPath()), string(filepath.Separator))
written[strings.ToLower(tmpPath)] = "the temp file for " + f.GetPath()
}
for _, f := range files { if strings.EqualFold(top, defaultManifestName) ||
// Look up each directory on the file's path, then the file itself. strings.EqualFold(top, tempPathFor(defaultManifestName)) {
parts := strings.Split(strings.ToLower(filepath.Clean(f.GetPath())), sep) return fmt.Errorf("%w: %s", errManifestNameListed, f.GetPath())
for i := range parts {
what, ok := written[strings.Join(parts[:i+1], sep)]
if ok {
return fmt.Errorf("%w: %s (%s)", errNameClash, f.GetPath(), what)
}
} }
} }
@@ -507,7 +486,7 @@ func checkNoNameClash(files []*mfer.MFFilePath) error {
// its manifest from a file, so the manifest is handed to it as a file in // its manifest from a file, so the manifest is handed to it as a file in
// memory. // memory.
func verifyFetchedSigner( func verifyFetchedSigner(
ctx context.Context, manifestData []byte, requiredSigner string, ctx *cli.Context, manifestData []byte, requiredSigner string,
) error { ) error {
memFs := afero.NewMemMapFs() memFs := afero.NewMemMapFs()
manifestPath := "/" + defaultManifestName manifestPath := "/" + defaultManifestName
@@ -517,7 +496,6 @@ func verifyFetchedSigner(
return err return err
} }
//nolint:contextcheck // mfer loads a manifest without a context
chk, err := mfer.NewChecker(&mfer.CheckerOptions{ chk, err := mfer.NewChecker(&mfer.CheckerOptions{
ManifestPath: manifestPath, ManifestPath: manifestPath,
BasePath: "/", BasePath: "/",
@@ -527,7 +505,7 @@ func verifyFetchedSigner(
return fmt.Errorf("failed to load manifest: %w", err) return fmt.Errorf("failed to load manifest: %w", err)
} }
return verifyRequiredSigner(ctx, chk, requiredSigner) return verifyRequiredSigner(ctx.Context, chk, requiredSigner)
} }
// saveManifest writes the fetched manifest into dest under the default // saveManifest writes the fetched manifest into dest under the default
internal/cli/fetch_test.go
+24 -83
View File
@@ -4,6 +4,7 @@ package cli
import ( import (
"bytes" "bytes"
"context" "context"
"flag"
"fmt" "fmt"
"io" "io"
"maps" "maps"
@@ -22,6 +23,7 @@ import (
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
urfcli "github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -1174,89 +1176,23 @@ func TestFetchRefusesListedManifestName(t *testing.T) {
t.Run(listed, func(t *testing.T) { t.Run(listed, func(t *testing.T) {
t.Parallel() t.Parallel()
files := map[string][]byte{listed: []byte("listed")} // Built directly rather than scanned, since a scan lists no
// hidden files and never a path starting with "./".
content := []byte("listed")
builder := mfer.NewBuilder()
_, err := builder.AddFile(mfer.RelFilePath(listed), mfer.FileSize(len(content)),
mfer.ModTime(time.Now()), bytes.NewReader(content), nil)
require.NoError(t, err)
assertFetchRefused(t, builtManifest(t, files), files, var manifest bytes.Buffer
"manifest lists a file where fetch writes another file: "+listed) require.NoError(t, builder.Build(context.Background(), &manifest))
assertFetchRefused(t, manifest.Bytes(), map[string][]byte{listed: content},
"manifest lists a file where fetch saves the manifest: "+listed)
}) })
} }
} }
// TestFetchRefusesListedTempName fetches manifests that list a.txt and
// .a.txt.tmp, the temp file fetch downloads a.txt to, at the top of the
// tree and in a directory. Downloading a.txt would remove .a.txt.tmp, so
// fetch must refuse the manifest before it creates the destination or
// requests any file. README with .README.tmp checks that temp names are
// compared ignoring case. A manifest that lists only one of the two is
// fetched in full.
func TestFetchRefusesListedTempName(t *testing.T) {
t.Parallel()
for _, dir := range []string{"", "sub/"} {
for file, tmp := range map[string]string{
dir + "a.txt": dir + ".a.txt.tmp",
dir + "README": dir + ".README.tmp",
} {
both := map[string][]byte{
file: []byte("a file"),
tmp: []byte("a file at its temp name"),
}
t.Run(file+" and "+tmp, func(t *testing.T) {
t.Parallel()
assertFetchRefused(t, builtManifest(t, both), both,
"manifest lists a file where fetch writes another file: "+
tmp+" (the temp file for "+file+")")
})
for listed, content := range both {
t.Run("only "+listed, func(t *testing.T) {
t.Parallel()
files := map[string][]byte{listed: content}
manifest := builtManifest(t, files)
server := httptest.NewServer(fetchTestHandler(manifest, files))
defer server.Close()
dest := t.TempDir()
opts := testOpts([]string{
testApp, cmdFetch, "-q", "--" + flagDest, dest, server.URL,
}, afero.NewOsFs())
require.Equal(t, 0, runCLI(opts), testStderr(t, opts))
want := maps.Clone(files)
want[defaultManifestName] = manifest
assert.Equal(t, want, filesUnder(t, dest))
})
}
}
}
}
// builtManifest returns a manifest of files, built directly rather than
// scanned, since a scan lists no hidden files and never a path starting
// with "./".
func builtManifest(t *testing.T, files map[string][]byte) []byte {
t.Helper()
builder := mfer.NewBuilder()
for p, content := range files {
_, err := builder.AddFile(mfer.RelFilePath(p), mfer.FileSize(len(content)),
mfer.ModTime(time.Now()), bytes.NewReader(content), nil)
require.NoError(t, err)
}
var manifest bytes.Buffer
require.NoError(t, builder.Build(context.Background(), &manifest))
return manifest.Bytes()
}
// assertFetchRefused serves manifest, a manifest of files, and fetches it // assertFetchRefused serves manifest, a manifest of files, and fetches it
// with flags into a directory that does not exist yet. fetch must fail // with flags into a directory that does not exist yet. fetch must fail
// with message after requesting only the manifest, and must not create // with message after requesting only the manifest, and must not create
@@ -1320,13 +1256,18 @@ func TestFetchTimeoutFlag(t *testing.T) {
mfa := &CLIApp{Fs: afero.NewMemMapFs()} mfa := &CLIApp{Fs: afero.NewMemMapFs()}
cmd := mfa.fetchCommand() set := flag.NewFlagSet(cmdFetch, flag.ContinueOnError)
cmd.Action = mfa.fetchManifestOperation for _, f := range mfa.fetchCommand().Flags {
require.NoError(t, f.Apply(set))
}
require.NoError(t, set.Parse([]string{"--" + flagTimeout, "100ms", server.URL}))
cliCtx := urfcli.NewContext(nil, set, nil)
cliCtx.Context = ctx
// fetchManifestOperation logs to the process-global logger. // fetchManifestOperation logs to the process-global logger.
err := runLocked(func() error { err := runLocked(func() error { return mfa.fetchManifestOperation(cliCtx) })
return cmd.Run(ctx, []string{cmdFetch, "--" + flagTimeout, "100ms", server.URL})
})
require.Error(t, err) require.Error(t, err)
} }
internal/cli/freshen.go
+19 -22
View File
@@ -14,7 +14,7 @@ import (
"github.com/dustin/go-humanize" "github.com/dustin/go-humanize"
"github.com/multiformats/go-multihash" "github.com/multiformats/go-multihash"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/urfave/cli/v3" "github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/internal/log" "sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -202,12 +202,12 @@ func (s *freshenScanner) walk(path string, info fs.FileInfo, walkErr error) erro
// resolveFreshenManifestPath determines the manifest path from the CLI // resolveFreshenManifestPath determines the manifest path from the CLI
// arguments, searching directories for a manifest where needed. // arguments, searching directories for a manifest where needed.
func (mfa *CLIApp) resolveFreshenManifestPath(cmd *cli.Command) (string, error) { func (mfa *CLIApp) resolveFreshenManifestPath(ctx *cli.Context) (string, error) {
if cmd.Args().Len() == 0 { if ctx.Args().Len() == 0 {
return findManifest(mfa.Fs, ".") return findManifest(mfa.Fs, ".")
} }
arg := cmd.Args().Get(0) arg := ctx.Args().Get(0)
info, statErr := mfa.Fs.Stat(arg) info, statErr := mfa.Fs.Stat(arg)
if statErr == nil && info.IsDir() { if statErr == nil && info.IsDir() {
@@ -338,14 +338,14 @@ func writeFreshenedManifest(
// newFreshenBuilder constructs the manifest builder configured from CLI // newFreshenBuilder constructs the manifest builder configured from CLI
// flags. // flags.
func newFreshenBuilder(cmd *cli.Command) *mfer.Builder { func newFreshenBuilder(ctx *cli.Context) *mfer.Builder {
builder := mfer.NewBuilder() builder := mfer.NewBuilder()
if cmd.Bool("include-timestamps") { if ctx.Bool("include-timestamps") {
builder.SetIncludeTimestamps(true) builder.SetIncludeTimestamps(true)
} }
// Set up signing options if sign-key is provided // Set up signing options if sign-key is provided
if signKey := cmd.String("sign-key"); signKey != "" { if signKey := ctx.String("sign-key"); signKey != "" {
builder.SetSigningOptions(&mfer.SigningOptions{ builder.SetSigningOptions(&mfer.SigningOptions{
KeyID: mfer.GPGKeyID(signKey), KeyID: mfer.GPGKeyID(signKey),
}) })
@@ -358,13 +358,13 @@ func newFreshenBuilder(cmd *cli.Command) *mfer.Builder {
// freshenScan runs the scan phase against the loaded manifest entries // freshenScan runs the scan phase against the loaded manifest entries
// and returns the populated scanner and the count of removed files. // and returns the populated scanner and the count of removed files.
func (mfa *CLIApp) freshenScan( func (mfa *CLIApp) freshenScan(
cmd *cli.Command, manifestPath, absBase string, ctx *cli.Context, manifestPath, absBase string,
existingByPath map[string]*mfer.MFFilePath, existingByPath map[string]*mfer.MFFilePath,
) (*freshenScanner, int64, error) { ) (*freshenScanner, int64, error) {
log.Infof("scanning filesystem...") log.Infof("scanning filesystem...")
startScan := time.Now() startScan := time.Now()
showProgress := cmd.Bool("progress") showProgress := ctx.Bool("progress")
// Leave out the manifest and a temp file left by an interrupted run, // Leave out the manifest and a temp file left by an interrupted run,
// as gen does. A path that cannot be stat'd, normally because no file // as gen does. A path that cannot be stat'd, normally because no file
@@ -382,8 +382,8 @@ func (mfa *CLIApp) freshenScan(
fs: mfa.Fs, fs: mfa.Fs,
absBase: absBase, absBase: absBase,
excluded: excluded, excluded: excluded,
includeDotfiles: cmd.Bool("include-dotfiles"), includeDotfiles: ctx.Bool("include-dotfiles"),
followSymlinks: cmd.Bool("follow-symlinks"), followSymlinks: ctx.Bool("follow-symlinks"),
showProgress: showProgress, showProgress: showProgress,
existingByPath: existingByPath, existingByPath: existingByPath,
} }
@@ -433,7 +433,7 @@ func hashTotals(entries []*freshenEntry) (int64, int64) {
// runFreshenHash processes every entry through the hasher, aborting if // runFreshenHash processes every entry through the hasher, aborting if
// the context is canceled. // the context is canceled.
func runFreshenHash( func runFreshenHash(
ctx context.Context, hasher *freshenHasher, entries []*freshenEntry, ctx *cli.Context, hasher *freshenHasher, entries []*freshenEntry,
) error { ) error {
for _, e := range entries { for _, e := range entries {
select { select {
@@ -479,21 +479,18 @@ func (mfa *CLIApp) loadExistingEntries(
return existingByPath, nil return existingByPath, nil
} }
func (mfa *CLIApp) freshenManifestOperation( func (mfa *CLIApp) freshenManifestOperation(ctx *cli.Context) error {
ctx context.Context, cmd *cli.Command,
) error {
log.Debug("freshenManifestOperation()") log.Debug("freshenManifestOperation()")
basePath := cmd.String("base") basePath := ctx.String("base")
showProgress := cmd.Bool("progress") showProgress := ctx.Bool("progress")
// Find manifest file // Find manifest file
manifestPath, err := mfa.resolveFreshenManifestPath(cmd) manifestPath, err := mfa.resolveFreshenManifestPath(ctx)
if err != nil { if err != nil {
return fmt.Errorf("freshen: %w", err) return fmt.Errorf("freshen: %w", err)
} }
//nolint:contextcheck // mfer loads a manifest without a context
existingByPath, err := mfa.loadExistingEntries(manifestPath) existingByPath, err := mfa.loadExistingEntries(manifestPath)
if err != nil { if err != nil {
return err return err
@@ -505,7 +502,7 @@ func (mfa *CLIApp) freshenManifestOperation(
} }
// Phase 1: Scan filesystem // Phase 1: Scan filesystem
scanner, removed, err := mfa.freshenScan(cmd, manifestPath, absBase, scanner, removed, err := mfa.freshenScan(ctx, manifestPath, absBase,
existingByPath) existingByPath)
if err != nil { if err != nil {
return err return err
@@ -527,7 +524,7 @@ func (mfa *CLIApp) freshenManifestOperation(
totalHashBytes: totalHashBytes, totalHashBytes: totalHashBytes,
filesToHash: filesToHash, filesToHash: filesToHash,
startHash: time.Now(), startHash: time.Now(),
builder: newFreshenBuilder(cmd), builder: newFreshenBuilder(ctx),
} }
err = runFreshenHash(ctx, hasher, scanner.entries) err = runFreshenHash(ctx, hasher, scanner.entries)
@@ -551,7 +548,7 @@ func (mfa *CLIApp) freshenManifestOperation(
} }
// Write updated manifest atomically (write to temp, then rename) // Write updated manifest atomically (write to temp, then rename)
err = writeFreshenedManifest(ctx, mfa.Fs, hasher.builder, manifestPath) err = writeFreshenedManifest(ctx.Context, mfa.Fs, hasher.builder, manifestPath)
if err != nil { if err != nil {
return err return err
} }
internal/cli/gen.go
+18 -21
View File
@@ -1,7 +1,6 @@
package cli package cli
import ( import (
"context"
"errors" "errors"
"fmt" "fmt"
"os" "os"
@@ -13,7 +12,7 @@ import (
"github.com/dustin/go-humanize" "github.com/dustin/go-humanize"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/urfave/cli/v3" "github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/internal/log" "sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -89,12 +88,12 @@ func (mfa *CLIApp) collectInputPaths(args cli.Args) ([]string, error) {
} }
// buildScannerOptions constructs scanner options from the CLI flags. // buildScannerOptions constructs scanner options from the CLI flags.
func (mfa *CLIApp) buildScannerOptions(cmd *cli.Command) *mfer.ScannerOptions { func (mfa *CLIApp) buildScannerOptions(ctx *cli.Context) *mfer.ScannerOptions {
output := cmd.String("output") output := ctx.String("output")
opts := &mfer.ScannerOptions{ opts := &mfer.ScannerOptions{
IncludeDotfiles: cmd.Bool("include-dotfiles"), IncludeDotfiles: ctx.Bool("include-dotfiles"),
FollowSymLinks: cmd.Bool("follow-symlinks"), FollowSymLinks: ctx.Bool("follow-symlinks"),
IncludeTimestamps: cmd.Bool("include-timestamps"), IncludeTimestamps: ctx.Bool("include-timestamps"),
Fs: mfa.Fs, Fs: mfa.Fs,
// Neither a manifest being replaced nor a temp file left by an // Neither a manifest being replaced nor a temp file left by an
// interrupted run belongs in the new manifest. // interrupted run belongs in the new manifest.
@@ -102,14 +101,14 @@ func (mfa *CLIApp) buildScannerOptions(cmd *cli.Command) *mfer.ScannerOptions {
} }
// Set seed for deterministic UUID if provided // Set seed for deterministic UUID if provided
if seed := cmd.String("seed"); seed != "" { if seed := ctx.String("seed"); seed != "" {
opts.Seed = seed opts.Seed = seed
log.Infof("using deterministic seed for manifest UUID") log.Infof("using deterministic seed for manifest UUID")
} }
// Set up signing options if sign-key is provided // Set up signing options if sign-key is provided
if signKey := cmd.String("sign-key"); signKey != "" { if signKey := ctx.String("sign-key"); signKey != "" {
opts.SigningOptions = &mfer.SigningOptions{ opts.SigningOptions = &mfer.SigningOptions{
KeyID: mfer.GPGKeyID(signKey), KeyID: mfer.GPGKeyID(signKey),
} }
@@ -174,14 +173,14 @@ func (mfa *CLIApp) cleanupOnSignal(outFile afero.File, tmpPath string) chan os.S
// runEnumeratePhase enumerates all input paths with optional progress // runEnumeratePhase enumerates all input paths with optional progress
// reporting and logs the totals. // reporting and logs the totals.
func (mfa *CLIApp) runEnumeratePhase(cmd *cli.Command, s *mfer.Scanner) error { func (mfa *CLIApp) runEnumeratePhase(ctx *cli.Context, s *mfer.Scanner) error {
// Set up enumeration progress reporting // Set up enumeration progress reporting
var ( var (
enumProgress chan mfer.EnumerateStatus enumProgress chan mfer.EnumerateStatus
enumWg sync.WaitGroup enumWg sync.WaitGroup
) )
if cmd.Bool("progress") { if ctx.Bool("progress") {
enumProgress = make(chan mfer.EnumerateStatus, 1) enumProgress = make(chan mfer.EnumerateStatus, 1)
enumWg.Add(1) enumWg.Add(1)
@@ -189,7 +188,7 @@ func (mfa *CLIApp) runEnumeratePhase(cmd *cli.Command, s *mfer.Scanner) error {
go reportEnumProgress(enumProgress, &enumWg) go reportEnumProgress(enumProgress, &enumWg)
} }
err := mfa.enumerateInputs(s, cmd.Args(), enumProgress) err := mfa.enumerateInputs(s, ctx.Args(), enumProgress)
if err != nil { if err != nil {
return err return err
} }
@@ -202,24 +201,22 @@ func (mfa *CLIApp) runEnumeratePhase(cmd *cli.Command, s *mfer.Scanner) error {
return nil return nil
} }
func (mfa *CLIApp) generateManifestOperation( func (mfa *CLIApp) generateManifestOperation(ctx *cli.Context) error {
ctx context.Context, cmd *cli.Command,
) error {
log.Debug("generateManifestOperation()") log.Debug("generateManifestOperation()")
s := mfer.NewScannerWithOptions(mfa.buildScannerOptions(cmd)) s := mfer.NewScannerWithOptions(mfa.buildScannerOptions(ctx))
// Phase 1: Enumeration - collect paths and stat files // Phase 1: Enumeration - collect paths and stat files
err := mfa.runEnumeratePhase(cmd, s) err := mfa.runEnumeratePhase(ctx, s)
if err != nil { if err != nil {
return err return err
} }
showProgress := cmd.Bool("progress") showProgress := ctx.Bool("progress")
// Check if output file exists // Check if output file exists
outputPath := cmd.String("output") outputPath := ctx.String("output")
if exists, _ := afero.Exists(mfa.Fs, outputPath); exists && !cmd.Bool("force") { if exists, _ := afero.Exists(mfa.Fs, outputPath); exists && !ctx.Bool("force") {
return fmt.Errorf("output file %s %w", outputPath, errOutputExists) return fmt.Errorf("output file %s %w", outputPath, errOutputExists)
} }
@@ -262,7 +259,7 @@ func (mfa *CLIApp) generateManifestOperation(
go reportScanProgress(scanProgress, &scanWg) go reportScanProgress(scanProgress, &scanWg)
} }
err = s.ToManifest(ctx, outFile, scanProgress) err = s.ToManifest(ctx.Context, outFile, scanProgress)
scanWg.Wait() scanWg.Wait()
internal/cli/list.go
+6 -8
View File
@@ -1,35 +1,33 @@
package cli package cli
import ( import (
"context"
"fmt" "fmt"
"time" "time"
"github.com/urfave/cli/v3" "github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/internal/log" "sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
func (mfa *CLIApp) listManifestOperation(ctx context.Context, cmd *cli.Command) error { func (mfa *CLIApp) listManifestOperation(ctx *cli.Context) error {
// Default to ErrorLevel for clean output // Default to ErrorLevel for clean output
log.SetLevel(log.ErrorLevel) log.SetLevel(log.ErrorLevel)
longFormat := cmd.Bool("long") longFormat := ctx.Bool("long")
print0 := cmd.Bool("print0") print0 := ctx.Bool("print0")
pathOrURL, err := mfa.resolveManifestArg(cmd) pathOrURL, err := mfa.resolveManifestArg(ctx)
if err != nil { if err != nil {
return fmt.Errorf("list: %w", err) return fmt.Errorf("list: %w", err)
} }
rc, err := mfa.openManifestReader(ctx, pathOrURL) rc, err := mfa.openManifestReader(pathOrURL)
if err != nil { if err != nil {
return fmt.Errorf("list: %w", err) return fmt.Errorf("list: %w", err)
} }
defer func() { _ = rc.Close() }() defer func() { _ = rc.Close() }()
//nolint:contextcheck // mfer loads a manifest without a context
manifest, err := mfer.NewManifestFromReader(rc) manifest, err := mfer.NewManifestFromReader(rc)
if err != nil { if err != nil {
return fmt.Errorf("list: failed to parse manifest: %w", err) return fmt.Errorf("list: failed to parse manifest: %w", err)
internal/cli/manifest_loader.go
+8 -8
View File
@@ -9,7 +9,7 @@ import (
"strings" "strings"
"time" "time"
"github.com/urfave/cli/v3" "github.com/urfave/cli/v2"
) )
// manifestFetchTimeout bounds HTTP requests made to fetch a manifest. // manifestFetchTimeout bounds HTTP requests made to fetch a manifest.
@@ -30,13 +30,13 @@ func isHTTPURL(s string) bool {
// openManifestReader opens a manifest from a path or URL and returns a ReadCloser. // openManifestReader opens a manifest from a path or URL and returns a ReadCloser.
// The caller must close the returned reader. // The caller must close the returned reader.
func (mfa *CLIApp) openManifestReader( func (mfa *CLIApp) openManifestReader(pathOrURL string) (io.ReadCloser, error) {
ctx context.Context, pathOrURL string,
) (io.ReadCloser, error) {
if isHTTPURL(pathOrURL) { if isHTTPURL(pathOrURL) {
client := &http.Client{Timeout: manifestFetchTimeout} client := &http.Client{Timeout: manifestFetchTimeout}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, pathOrURL, nil) req, err := http.NewRequestWithContext(
context.Background(), http.MethodGet, pathOrURL, nil,
)
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to fetch %s: %w", pathOrURL, err) return nil, fmt.Errorf("failed to fetch %s: %w", pathOrURL, err)
} }
@@ -67,9 +67,9 @@ func (mfa *CLIApp) openManifestReader(
// resolveManifestArg resolves the manifest path from CLI arguments. // resolveManifestArg resolves the manifest path from CLI arguments.
// HTTP(S) URLs are returned as-is. Directories are searched for index.mf. // HTTP(S) URLs are returned as-is. Directories are searched for index.mf.
// If no argument is given, the current directory is searched. // If no argument is given, the current directory is searched.
func (mfa *CLIApp) resolveManifestArg(cmd *cli.Command) (string, error) { func (mfa *CLIApp) resolveManifestArg(ctx *cli.Context) (string, error) {
if cmd.Args().Len() > 0 { if ctx.Args().Len() > 0 {
arg := cmd.Args().Get(0) arg := ctx.Args().Get(0)
if isHTTPURL(arg) { if isHTTPURL(arg) {
return arg, nil return arg, nil
} }
internal/cli/mfer.go
+74 -93
View File
@@ -1,7 +1,6 @@
package cli package cli
import ( import (
"context"
"errors" "errors"
"fmt" "fmt"
"io" "io"
@@ -9,7 +8,7 @@ import (
"time" "time"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/urfave/cli/v3" "github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/internal/log" "sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -55,7 +54,7 @@ type CLIApp struct {
gitrev string gitrev string
startupTime time.Time startupTime time.Time
exitCode int exitCode int
app *cli.Command app *cli.App
Stdin io.Reader // Standard input stream Stdin io.Reader // Standard input stream
Stdout io.Writer // Standard output stream for normal output Stdout io.Writer // Standard output stream for normal output
@@ -106,15 +105,15 @@ func (mfa *CLIApp) printBanner() {
// urfave/cli reads a flag from the nearest command that defines it, so each // urfave/cli reads a flag from the nearest command that defines it, so each
// command in the lineage is asked. The highest -v count wins rather than the // command in the lineage is asked. The highest -v count wins rather than the
// sum, because a subcommand without its own copies reads the root's. // sum, because a subcommand without its own copies reads the root's.
func (mfa *CLIApp) setVerbosity(cmd *cli.Command) { func (mfa *CLIApp) setVerbosity(c *cli.Context) {
_, present := os.LookupEnv("MFER_DEBUG") _, present := os.LookupEnv("MFER_DEBUG")
verbosity := 0 verbosity := 0
quiet := false quiet := false
for _, c := range cmd.Lineage() { for _, ctx := range c.Lineage() {
verbosity = max(verbosity, c.Count("verbose")) verbosity = max(verbosity, ctx.Count("verbose"))
quiet = quiet || c.Bool("quiet") quiet = quiet || ctx.Bool("quiet")
} }
switch { switch {
@@ -128,34 +127,23 @@ func (mfa *CLIApp) setVerbosity(cmd *cli.Command) {
} }
// commonFlags returns the -v and -q flags taken by the root and by the // commonFlags returns the -v and -q flags taken by the root and by the
// generate, check, freshen and fetch subcommands. They are local, so the // generate, check, freshen and fetch subcommands.
// root's copies are not inherited by the subcommands that do not take them.
func commonFlags() []cli.Flag { func commonFlags() []cli.Flag {
return []cli.Flag{ return []cli.Flag{
&cli.BoolFlag{ &cli.BoolFlag{
Name: "verbose", Name: "verbose",
Aliases: []string{"v"}, Aliases: []string{"v"},
Usage: "Increase verbosity (-v for verbose, -v -v for debug)", Usage: "Increase verbosity (-v for verbose, -v -v for debug)",
Local: true, Count: new(int),
}, },
&cli.BoolFlag{ &cli.BoolFlag{
Name: "quiet", Name: "quiet",
Aliases: []string{"q"}, Aliases: []string{"q"},
Usage: "Suppress output except errors", Usage: "Suppress output except errors",
Local: true,
}, },
} }
} }
// stopOnFirstArg returns the StopOnNthArg setting every command uses: flags
// are read only before the command's first argument, and everything after it
// is an argument, as with urfave/cli v2. So `mfer gen d -v` names a path "-v".
func stopOnFirstArg() *int {
n := 1
return &n
}
// requireSignatureFlag returns the --require-signature flag taken by the // requireSignatureFlag returns the --require-signature flag taken by the
// check and fetch subcommands. // check and fetch subcommands.
func requireSignatureFlag() *cli.StringFlag { func requireSignatureFlag() *cli.StringFlag {
@@ -163,22 +151,21 @@ func requireSignatureFlag() *cli.StringFlag {
Name: flagRequireSignature, Name: flagRequireSignature,
Aliases: []string{"S"}, Aliases: []string{"S"},
Usage: "Require manifest to be signed by the specified GPG key ID", Usage: "Require manifest to be signed by the specified GPG key ID",
Sources: cli.EnvVars("MFER_REQUIRE_SIGNATURE"), EnvVars: []string{"MFER_REQUIRE_SIGNATURE"},
} }
} }
func (mfa *CLIApp) generateCommand() *cli.Command { func (mfa *CLIApp) generateCommand() *cli.Command {
return &cli.Command{ return &cli.Command{
Name: cmdGenerate, Name: cmdGenerate,
Aliases: []string{"gen"}, Aliases: []string{"gen"},
Usage: "Generate manifest file", Usage: "Generate manifest file",
ArgsUsage: "[path ...]", ArgsUsage: "[path ...]",
StopOnNthArg: stopOnFirstArg(), Action: func(c *cli.Context) error {
Action: func(ctx context.Context, cmd *cli.Command) error { mfa.setVerbosity(c)
mfa.setVerbosity(cmd)
mfa.printBanner() mfa.printBanner()
return mfa.generateManifestOperation(ctx, cmd) return mfa.generateManifestOperation(c)
}, },
Flags: append(commonFlags(), Flags: append(commonFlags(),
&cli.BoolFlag{ &cli.BoolFlag{
@@ -212,12 +199,12 @@ func (mfa *CLIApp) generateCommand() *cli.Command {
Name: "sign-key", Name: "sign-key",
Aliases: []string{"s"}, Aliases: []string{"s"},
Usage: "GPG key ID to sign the manifest with", Usage: "GPG key ID to sign the manifest with",
Sources: cli.EnvVars("MFER_SIGN_KEY"), EnvVars: []string{"MFER_SIGN_KEY"},
}, },
&cli.StringFlag{ &cli.StringFlag{
Name: "seed", Name: "seed",
Usage: "Seed value for deterministic manifest UUID", Usage: "Seed value for deterministic manifest UUID",
Sources: cli.EnvVars("MFER_SEED"), EnvVars: []string{"MFER_SEED"},
}, },
&cli.BoolFlag{ &cli.BoolFlag{
Name: "include-timestamps", Name: "include-timestamps",
@@ -230,15 +217,14 @@ func (mfa *CLIApp) generateCommand() *cli.Command {
func (mfa *CLIApp) checkCommand() *cli.Command { func (mfa *CLIApp) checkCommand() *cli.Command {
return &cli.Command{ return &cli.Command{
Name: cmdCheck, Name: cmdCheck,
Usage: "Validate files using manifest file", Usage: "Validate files using manifest file",
ArgsUsage: manifestArgsUsage, ArgsUsage: manifestArgsUsage,
StopOnNthArg: stopOnFirstArg(), Action: func(c *cli.Context) error {
Action: func(ctx context.Context, cmd *cli.Command) error { mfa.setVerbosity(c)
mfa.setVerbosity(cmd)
mfa.printBanner() mfa.printBanner()
return mfa.checkManifestOperation(ctx, cmd) return mfa.checkManifestOperation(c)
}, },
Flags: append(commonFlags(), Flags: append(commonFlags(),
&cli.StringFlag{ &cli.StringFlag{
@@ -263,15 +249,14 @@ func (mfa *CLIApp) checkCommand() *cli.Command {
func (mfa *CLIApp) freshenCommand() *cli.Command { func (mfa *CLIApp) freshenCommand() *cli.Command {
return &cli.Command{ return &cli.Command{
Name: cmdFreshen, Name: cmdFreshen,
Usage: "Update manifest with changed, new, and removed files", Usage: "Update manifest with changed, new, and removed files",
ArgsUsage: manifestArgsUsage, ArgsUsage: manifestArgsUsage,
StopOnNthArg: stopOnFirstArg(), Action: func(c *cli.Context) error {
Action: func(ctx context.Context, cmd *cli.Command) error { mfa.setVerbosity(c)
mfa.setVerbosity(cmd)
mfa.printBanner() mfa.printBanner()
return mfa.freshenManifestOperation(ctx, cmd) return mfa.freshenManifestOperation(c)
}, },
Flags: append(commonFlags(), Flags: append(commonFlags(),
&cli.StringFlag{ &cli.StringFlag{
@@ -300,7 +285,7 @@ func (mfa *CLIApp) freshenCommand() *cli.Command {
Name: "sign-key", Name: "sign-key",
Aliases: []string{"s"}, Aliases: []string{"s"},
Usage: "GPG key ID to sign the manifest with", Usage: "GPG key ID to sign the manifest with",
Sources: cli.EnvVars("MFER_SIGN_KEY"), EnvVars: []string{"MFER_SIGN_KEY"},
}, },
&cli.BoolFlag{ &cli.BoolFlag{
Name: "include-timestamps", Name: "include-timestamps",
@@ -313,24 +298,22 @@ func (mfa *CLIApp) freshenCommand() *cli.Command {
func (mfa *CLIApp) exportCommand() *cli.Command { func (mfa *CLIApp) exportCommand() *cli.Command {
return &cli.Command{ return &cli.Command{
Name: cmdExport, Name: cmdExport,
Usage: "Export manifest contents as JSON", Usage: "Export manifest contents as JSON",
ArgsUsage: "[manifest file or URL]", ArgsUsage: "[manifest file or URL]",
StopOnNthArg: stopOnFirstArg(), Action: func(c *cli.Context) error {
Action: func(ctx context.Context, cmd *cli.Command) error { mfa.setVerbosity(c)
mfa.setVerbosity(cmd)
return mfa.exportManifestOperation(ctx, cmd) return mfa.exportManifestOperation(c)
}, },
} }
} }
func (mfa *CLIApp) versionCommand() *cli.Command { func (mfa *CLIApp) versionCommand() *cli.Command {
return &cli.Command{ return &cli.Command{
Name: cmdVersion, Name: cmdVersion,
Usage: "Show version", Usage: "Show version",
StopOnNthArg: stopOnFirstArg(), Action: func(_ *cli.Context) error {
Action: func(context.Context, *cli.Command) error {
mfa.printVersion() mfa.printVersion()
return nil return nil
@@ -340,12 +323,13 @@ func (mfa *CLIApp) versionCommand() *cli.Command {
func (mfa *CLIApp) listCommand() *cli.Command { func (mfa *CLIApp) listCommand() *cli.Command {
return &cli.Command{ return &cli.Command{
Name: "list", Name: "list",
Aliases: []string{"ls"}, Aliases: []string{"ls"},
Usage: "List files in manifest", Usage: "List files in manifest",
ArgsUsage: manifestArgsUsage, ArgsUsage: manifestArgsUsage,
StopOnNthArg: stopOnFirstArg(), Action: func(c *cli.Context) error {
Action: mfa.listManifestOperation, return mfa.listManifestOperation(c)
},
Flags: []cli.Flag{ Flags: []cli.Flag{
&cli.BoolFlag{ &cli.BoolFlag{
Name: "long", Name: "long",
@@ -362,15 +346,14 @@ func (mfa *CLIApp) listCommand() *cli.Command {
func (mfa *CLIApp) fetchCommand() *cli.Command { func (mfa *CLIApp) fetchCommand() *cli.Command {
return &cli.Command{ return &cli.Command{
Name: cmdFetch, Name: cmdFetch,
Usage: "fetch manifest and referenced files", Usage: "fetch manifest and referenced files",
ArgsUsage: "URL", ArgsUsage: "URL",
StopOnNthArg: stopOnFirstArg(), Action: func(c *cli.Context) error {
Action: func(ctx context.Context, cmd *cli.Command) error { mfa.setVerbosity(c)
mfa.setVerbosity(cmd)
mfa.printBanner() mfa.printBanner()
return mfa.fetchManifestOperation(ctx, cmd) return mfa.fetchManifestOperation(c)
}, },
Flags: append(commonFlags(), Flags: append(commonFlags(),
&cli.DurationFlag{ &cli.DurationFlag{
@@ -402,40 +385,38 @@ func (mfa *CLIApp) run(args []string) {
log.SetOutput(mfa.Stdout, mfa.Stderr) log.SetOutput(mfa.Stdout, mfa.Stderr)
log.Init() log.Init()
// -v means verbose, not version, at the root as on the generate, check, // -v means verbose, not version. urfave/cli's built-in version flag
// freshen and fetch subcommands: verbose is the more common meaning of -v // claims -v by default, which made "mfer -v --version" fail to parse and
// in tools that offer both. urfave/cli's built-in version flag claims -v // gave -v a different meaning at the root than on the generate, check,
// by default, so the version flag takes the capital -V instead. // freshen and fetch subcommands, where it means verbose. Verbose is the
// more common meaning of -v in tools that offer both, so -v means verbose
// at the root too and the version flag takes the capital -V.
// VersionFlag and VersionPrinter are urfave/cli package globals; run() is // VersionFlag and VersionPrinter are urfave/cli package globals; run() is
// serialized in tests, so assigning them here is safe. // serialized in tests, so assigning them here is safe.
cli.VersionFlag = &cli.BoolFlag{ cli.VersionFlag = &cli.BoolFlag{
Name: cmdVersion, Name: cmdVersion,
Aliases: []string{"V"}, Aliases: []string{"V"},
Usage: "print the version", Usage: "print the version",
Local: true,
} }
cli.VersionPrinter = func(*cli.Command) { mfa.printVersion() } cli.VersionPrinter = func(_ *cli.Context) { mfa.printVersion() }
mfa.app = &cli.Command{ mfa.app = &cli.App{
Name: mfa.appname, Name: mfa.appname,
Usage: "Manifest generator", Usage: "Manifest generator",
Version: mfa.VersionString(), Version: mfa.VersionString(),
EnableShellCompletion: true, EnableBashCompletion: true,
Writer: mfa.Stdout, Writer: mfa.Stdout,
// v3 writes its "Incorrect Usage" line to ErrWriter; v2 wrote it to ErrWriter: mfa.Stderr,
// stdout, before the help, so it stays on stdout. Flags: commonFlags(),
ErrWriter: mfa.Stdout, Action: func(c *cli.Context) error {
Flags: commonFlags(), if c.Args().Len() > 0 {
StopOnNthArg: stopOnFirstArg(), return fmt.Errorf("%w %q", errUnknownCommand, c.Args().First())
Action: func(_ context.Context, cmd *cli.Command) error {
if cmd.Args().Len() > 0 {
return fmt.Errorf("%w %q", errUnknownCommand, cmd.Args().First())
} }
mfa.setVerbosity(cmd) mfa.setVerbosity(c)
mfa.printBanner() mfa.printBanner()
return cli.ShowRootCommandHelp(cmd) return cli.ShowAppHelp(c)
}, },
Commands: []*cli.Command{ Commands: []*cli.Command{
mfa.generateCommand(), mfa.generateCommand(),
@@ -450,7 +431,7 @@ func (mfa *CLIApp) run(args []string) {
mfa.app.HideVersion = false mfa.app.HideVersion = false
err := mfa.app.Run(context.Background(), args) err := mfa.app.Run(args)
if err != nil { if err != nil {
mfa.exitCode = 1 mfa.exitCode = 1
mfer/mf.go
+3
View File
@@ -0,0 +1,3 @@
package mfer
//go:generate protoc ./mf.proto --go_out=paths=source_relative:.
script/bootstrap
+5 -56
View File
@@ -13,15 +13,11 @@ set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Pinned versions, 2026-07-06. Never "latest" or "lts"; exact versions. # Pinned versions, 2026-07-06. Never "latest" or "lts"; exact versions.
# The node version is in .nvmrc, where script/prettier reads it too. NODE_VERSION="22.17.0"
NODE_VERSION="$(cat "$ROOT/.nvmrc")"
NVM_VERSION="0.40.3" NVM_VERSION="0.40.3"
# sha256 of https://github.com/nvm-sh/nvm/archive/refs/tags/v0.40.3.tar.gz # sha256 of https://github.com/nvm-sh/nvm/archive/refs/tags/v0.40.3.tar.gz
NVM_SHA256="5f4d6aaa04a177dc93c985e31dbc411ab6b8c6e1e21d8015dbc1372625fcd1d0" NVM_SHA256="5f4d6aaa04a177dc93c985e31dbc411ab6b8c6e1e21d8015dbc1372625fcd1d0"
YARN_VERSION="1.22.22" YARN_VERSION="1.22.22"
# protoc v33.4, 2026-10-04, for script/generate. The sha256 of each
# platform's release archive is in ensure_protoc.
PROTOC_VERSION="33.4"
PKGMGR="" PKGMGR=""
SUDO="" SUDO=""
@@ -78,11 +74,9 @@ verify_sha256() {
fi fi
} }
# nvm is a bash script; run a command in a bash with nvm loaded. # nvm is a bash script; run a command in a bash with nvm loaded
# --no-use: otherwise loading nvm here switches to the version .nvmrc
# names, and fails silently while that version is not installed yet.
nvm_sh() { nvm_sh() {
bash -c ". \"\$HOME/.nvm/nvm.sh\" --no-use && $*" bash -c ". \"\$HOME/.nvm/nvm.sh\" && $*"
} }
ensure_nvm() { ensure_nvm() {
@@ -128,48 +122,6 @@ install_js_deps() {
fi fi
} }
# Unpack protoc's release archive for this platform into bin/protoc, after
# checking the archive's sha256, unless bin/protoc already holds the pinned
# version.
ensure_protoc() {
dir="$ROOT/bin/protoc"
if [ "$("$dir/bin/protoc" --version 2>/dev/null)" = \
"libprotoc $PROTOC_VERSION" ]; then
return 0
fi
case "$(uname -s) $(uname -m)" in
"Linux x86_64")
platform="linux-x86_64"
sha256="c0040ea9aef08fdeb2c74ca609b18d5fdbfc44ea0042fcfbfb38860d35f7dd66"
;;
"Linux aarch64" | "Linux arm64")
platform="linux-aarch_64"
sha256="15aa988f4a6090636525ec236a8e4b3aab41eef402751bd5bb2df6afd9b7b5a5"
;;
"Darwin x86_64")
platform="osx-x86_64"
sha256="a49bec10d039e902d3b43e49938c42526f90011467609864fa6386ac4014da58"
;;
"Darwin arm64")
platform="osx-aarch_64"
sha256="726297dcfed58592fd35620a5a6246ae020c39e88f3fd4cb1827df7bcf3dfcf1"
;;
*)
echo "bootstrap: no protoc archive pinned for $(uname -s) $(uname -m)" >&2
exit 1
;;
esac
if missing curl; then pkg_install curl curl curl curl; fi
if missing unzip; then pkg_install unzip unzip unzip unzip; fi
tmp="$(mktemp -d)"
curl -fsSL -o "$tmp/protoc.zip" \
"https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOC_VERSION}/protoc-${PROTOC_VERSION}-${platform}.zip"
verify_sha256 "$tmp/protoc.zip" "$sha256"
rm -rf "$dir"
unzip -q "$tmp/protoc.zip" -d "$dir"
rm -rf "$tmp"
}
main() { main() {
cd "$ROOT" cd "$ROOT"
@@ -190,12 +142,9 @@ main() {
# ---- Go repos ---- # ---- Go repos ----
if missing go; then pkg_install go golang go go; fi if missing go; then pkg_install go golang go go; fi
# No golangci-lint: script/lint runs it in Docker only. # No golangci-lint: script/lint runs it in Docker only. No gofumpt:
# script/gofumpt builds the version tools/go.mod requires.
go mod download go mod download
# gofumpt and protoc-gen-go: bin/tools/go.mod pins them, and
# script/gofumpt and script/generate build them from there.
(cd "$ROOT/bin/tools" && go mod download)
ensure_protoc
# ---- Python repos ---- # ---- Python repos ----
# if missing python3; then pkg_install python3 python3 python3 python3; fi # if missing python3; then pkg_install python3 python3 python3 python3; fi
script/build
+3 -1
View File
@@ -12,7 +12,9 @@ main() {
# empty constant. # empty constant.
version="$(git describe --tags --always --dirty 2>/dev/null || true)" version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
go build -ldflags "-X main.Gitrev=$version" -o bin/mfer ./cmd/mfer go build -tags urfave_cli_no_docs \
-ldflags "-X main.Gitrev=$version" \
-o bin/mfer ./cmd/mfer
} }
main "$@" main "$@"
script/generate
+21 -20
View File
@@ -4,17 +4,26 @@
# regenerates mf.pb.go: it is committed, so building and checking need no # regenerates mf.pb.go: it is committed, so building and checking need no
# protoc. A test fails while mf.proto no longer matches the recorded hash. # protoc. A test fails while mf.proto no longer matches the recorded hash.
# #
# Runs the protoc that script/bootstrap unpacks into bin/protoc, and the # Needs exactly the protoc and protoc-gen-go versions named in the header of
# protoc-gen-go that bin/tools/go.mod pins. Another version of either # the committed mf.pb.go (README.md says how to install them). Another
# writes a different mf.pb.go. # version writes a different mf.pb.go, so the script refuses to run.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# The protoc version script/bootstrap installs. protoc 33.4 names itself # protoc 33.4 names itself v6.33.4 in the mf.pb.go header.
# v6.33.4 in the mf.pb.go header.
PROTOC_VERSION="33.4" PROTOC_VERSION="33.4"
PROTOC="$ROOT/bin/protoc/bin/protoc" PROTOC_GEN_GO_VERSION="v1.36.11"
# require_version <command> <its exact --version output>
require_version() {
actual="$("$1" --version 2>/dev/null || true)"
if [ "$actual" != "$2" ]; then
echo "generate: needs $2 on PATH, found: ${actual:-none}" >&2
echo " README.md says how to install it." >&2
exit 1
fi
}
# sha256 <file>: print "<hash> <file>", with sha256sum, or with shasum # sha256 <file>: print "<hash> <file>", with sha256sum, or with shasum
# where there is no sha256sum. # where there is no sha256sum.
@@ -30,24 +39,16 @@ sha256() {
} }
main() { main() {
# A bin/protoc left from before the pin moved fails here, until
# script/bootstrap replaces it.
actual="$("$PROTOC" --version 2>/dev/null || true)"
if [ "$actual" != "libprotoc $PROTOC_VERSION" ]; then
echo "generate: needs protoc $PROTOC_VERSION in bin/protoc," \
"found: ${actual:-none}; run script/bootstrap" >&2
exit 1
fi
# `go tool -n` builds protoc-gen-go from bin/tools and prints where the
# binary is, without running it.
plugin="$(cd "$ROOT/bin/tools" && go tool -n protoc-gen-go)"
cd "$ROOT/mfer" cd "$ROOT/mfer"
# `go install` puts protoc-gen-go in $(go env GOPATH)/bin, which is
# often not on PATH.
PATH="$PATH:$(go env GOPATH)/bin"
require_version protoc "libprotoc $PROTOC_VERSION"
require_version protoc-gen-go "protoc-gen-go $PROTOC_GEN_GO_VERSION"
# Hashed before regenerating, so a missing hash tool stops the script # Hashed before regenerating, so a missing hash tool stops the script
# before it changes anything. Regenerating leaves mf.proto as it is. # before it changes anything. Regenerating leaves mf.proto as it is.
proto_hash="$(sha256 mf.proto)" proto_hash="$(sha256 mf.proto)"
"$PROTOC" --plugin=protoc-gen-go="$plugin" \ go generate .
--go_out=paths=source_relative:. ./mf.proto
echo "$proto_hash" >mf.proto.sha256 echo "$proto_hash" >mf.proto.sha256
} }
script/gofumpt
+4 -5
View File
@@ -10,10 +10,9 @@ set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# The gofumpt version is the one bin/tools/go.mod pins. `go tool` run in # gofumpt v0.12.0, 2026-10-04, required by tools/go.mod. `go tool` run in
# bin/tools builds it from source checked against the hashes in # tools/ builds it from source checked against the hashes in tools/go.sum,
# bin/tools/go.sum, so neither a developer machine nor the lint image needs # so neither a developer machine nor the lint image needs it installed.
# it installed.
usage() { usage() {
echo "usage: script/gofumpt --write|--check" >&2 echo "usage: script/gofumpt --write|--check" >&2
@@ -22,7 +21,7 @@ usage() {
main() { main() {
[ "$#" -eq 1 ] || usage [ "$#" -eq 1 ] || usage
cd "$ROOT/bin/tools" cd "$ROOT/tools"
# Every Go file in the repo, from $ROOT down. gofumpt holds generated # Every Go file in the repo, from $ROOT down. gofumpt holds generated
# files, such as mfer/mf.pb.go, to gofmt's rules only. # files, such as mfer/mf.pb.go, to gofmt's rules only.
case "$1" in case "$1" in
script/prettier
+4 -12
View File
@@ -31,18 +31,9 @@ main() {
cd "$ROOT" cd "$ROOT"
# Where there is no node on PATH, script/bootstrap installs the version
# .nvmrc names through nvm, which keeps it in this directory.
if ! command -v node >/dev/null 2>&1; then
PATH="$HOME/.nvm/versions/node/v$(cat .nvmrc)/bin:$PATH"
fi
if ! command -v node >/dev/null 2>&1; then
echo "prettier: node is missing; run script/bootstrap" >&2
exit 1
fi
# node_modules keeps the old prettier after package.json moves to a new # node_modules keeps the old prettier after package.json moves to a new
# one, until script/bootstrap runs again, so compare the two. # one, until script/bootstrap runs again, so compare the two. A prettier
# that runs also means node is on PATH, which reading package.json needs.
if ! installed="$("$PRETTIER" --version 2>/dev/null)"; then if ! installed="$("$PRETTIER" --version 2>/dev/null)"; then
echo "prettier: not installed; run script/bootstrap" >&2 echo "prettier: not installed; run script/bootstrap" >&2
exit 1 exit 1
@@ -63,7 +54,8 @@ main() {
# patterns always match at least one tracked file (README.md, # patterns always match at least one tracked file (README.md,
# package.json), so an empty match means the glob broke, and prettier # package.json), so an empty match means the glob broke, and prettier
# erroring out is exactly what we want rather than a vacuous pass. # erroring out is exactly what we want rather than a vacuous pass.
"$PRETTIER" "$mode" "**/*.md" "**/*.json" "$PRETTIER" "$mode" "**/*.md"
"$PRETTIER" "$mode" "**/*.json"
} }
main "$@" main "$@"
tools/go.mod
+15
View File
@@ -0,0 +1,15 @@
// The developer tools this repo runs with `go tool`, kept out of the mfer
// module so they add nothing to what mfer's users download. `go tool` builds
// exactly the source whose hashes go.sum here records.
module sneak.berlin/go/mfer/tools
go 1.26.0
tool mvdan.cc/gofumpt
require (
golang.org/x/mod v0.40.0 // indirect
golang.org/x/sync v0.22.0 // indirect
golang.org/x/tools v0.49.0 // indirect
mvdan.cc/gofumpt v0.12.0 // indirect
)
bin/tools/go.sum → tools/go.sum
-2
View File
@@ -16,7 +16,5 @@ golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/tools v0.49.0 h1:3NI7VXzL9+1WZD52Dx2ttoPwD5DWrFGpl9mFZDlmisI= golang.org/x/tools v0.49.0 h1:3NI7VXzL9+1WZD52Dx2ttoPwD5DWrFGpl9mFZDlmisI=
golang.org/x/tools v0.49.0/go.mod h1:SJNXV9DBKT0UbdttsQjbfJlAE/q+y36++zo3uL3N0Oo= golang.org/x/tools v0.49.0/go.mod h1:SJNXV9DBKT0UbdttsQjbfJlAE/q+y36++zo3uL3N0Oo=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
mvdan.cc/gofumpt v0.12.0 h1:1Lbudkz2kpM9Cjz2pL4M19u7q+GaEhCTNf7N9mfpcho= mvdan.cc/gofumpt v0.12.0 h1:1Lbudkz2kpM9Cjz2pL4M19u7q+GaEhCTNf7N9mfpcho=
mvdan.cc/gofumpt v0.12.0/go.mod h1:SmBHHrljiZu/uoypeKup3rFzP6eoC9UwCp2iH5E3jZA= mvdan.cc/gofumpt v0.12.0/go.mod h1:SmBHHrljiZu/uoypeKup3rFzP6eoC9UwCp2iH5E3jZA=