2 Commits
Author SHA1 Message Date
clawbot ce024baaed Move the CLI to urfave/cli v3 (closes #110)
check / check (push) Failing after 3s
Ported per the library's v2-to-v3 migration guide: the app is a root
cli.Command, actions take a context and the command, and flag
environment variables become value sources. -v, -q and the version flag
are local so, as before, only the commands defining them accept them.
Every command stops reading flags at its first argument, as v2 did.
ErrWriter is stdout so usage errors print with their help. The action's
context reaches the manifest download in check, export and list.
testify rises to v1.12.1, which v3 requires; the urfave_cli_no_docs
build tag, which v3 lacks, is dropped. v3 accepts a flag given under two
names, so -v --verbose gives debug output, and the test pinning the
refusal becomes a TestVerboseCount case.

Model: opus-5-5
2026-10-04 22:02:12 +02:00
clawbot ab72692439 Pin the remaining developer tool installs (closes #68)
check / check (push) Failing after 3s
gofumpt and protoc-gen-go are tools of a separate Go module in bin/tools,
so `go tool` builds them from source checked against bin/tools/go.sum and
mfer's own module gains no dependencies. script/bootstrap downloads that
module, and unpacks protoc 33.4 into bin/protoc from its release archive
after checking the sha256 it holds for the platform. script/generate runs
that protoc with the pinned plugin, so mfer/mf.go and its go:generate line
go. script/prettier runs only the node_modules prettier, fails when it
differs from the package.json pin, finds the node bootstrap installed
through nvm by the version in .nvmrc, and runs prettier once. Comment and
package.json fixes.

Model: opus-5-5
2026-10-04 20:48:51 +02:00
27 changed files with 455 additions and 343 deletions
+3
View File
@@ -61,3 +61,6 @@
# This repo's own host-built binary (make build). # This repo's own host-built binary (make build).
/bin/mfer /bin/mfer
# The protoc script/bootstrap unpacks for script/generate.
/bin/protoc
+1
View File
@@ -1,4 +1,5 @@
/bin/mfer /bin/mfer
/bin/protoc/
/tmp /tmp
/node_modules/ /node_modules/
+1
View File
@@ -0,0 +1 @@
22.17.0
+4 -4
View File
@@ -10,7 +10,7 @@ COPY . .
# Go half of fmt-check only: this image has no node, so no prettier. The # Go half of fmt-check only: this image has no node, so no prettier. The
# markdown half runs in the mdfmt stage below. The image has no gofumpt # markdown half runs in the mdfmt stage below. The image has no gofumpt
# either; script/gofumpt builds the version it pins with `go run`. # either; script/gofumpt builds the version bin/tools/go.mod pins.
RUN script/gofumpt --check RUN script/gofumpt --check
# The linter directly, not `make lint`: script/lint builds this stage, and # The linter directly, not `make lint`: script/lint builds this stage, and
# there is no docker inside this build. # there is no docker inside this build.
@@ -18,7 +18,7 @@ RUN golangci-lint run --config .golangci.yml ./...
# Markdown/JSON format stage — prettier needs node, which the Go images # Markdown/JSON format stage — prettier needs node, which the Go images
# do not have. node:22.17.0-bookworm-slim (2026-08-09); ships node # do not have. node:22.17.0-bookworm-slim (2026-08-09); ships node
# 22.17.0 and yarn 1.22.22, the versions script/bootstrap pins. # 22.17.0 and yarn 1.22.22, the versions .nvmrc and script/bootstrap pin.
FROM node@sha256:b04ce4ae4e95b522112c2e5c52f781471a5cbc3b594527bcddedee9bc48c03a0 AS mdfmt FROM node@sha256:b04ce4ae4e95b522112c2e5c52f781471a5cbc3b594527bcddedee9bc48c03a0 AS mdfmt
WORKDIR /src WORKDIR /src
@@ -31,7 +31,7 @@ COPY . .
RUN script/prettier --check RUN script/prettier --check
# Build stage — tests and compilation # Build stage — tests and compilation
# golang:1.23 (2026-03-14) # golang:1.23.12, 2026-03-14
FROM golang@sha256:60deed95d3888cc5e4d9ff8a10c54e5edc008c6ae3fba6187be6fb592e19e8c0 AS builder FROM golang@sha256:60deed95d3888cc5e4d9ff8a10c54e5edc008c6ae3fba6187be6fb592e19e8c0 AS builder
# Force BuildKit to run the lint and mdfmt stages by creating stage dependencies # Force BuildKit to run the lint and mdfmt stages by creating stage dependencies
@@ -64,7 +64,7 @@ RUN version="${VERSION:-$(git describe --tags --always)}"; \
exit 1; \ exit 1; \
fi; \ fi; \
cd cmd/mfer && \ cd cmd/mfer && \
CGO_ENABLED=0 go build -tags urfave_cli_no_docs -ldflags "-X main.Gitrev=$version" -o /mfer . CGO_ENABLED=0 go build -ldflags "-X main.Gitrev=$version" -o /mfer .
# Fail unless /mfer is statically linked: scratch has no C library to run it. # Fail unless /mfer is statically linked: scratch has no C library to run it.
RUN ldd /mfer 2>&1 | grep -q 'not a dynamic executable' RUN ldd /mfer 2>&1 | grep -q 'not a dynamic executable'
+18 -15
View File
@@ -67,9 +67,13 @@ standard: normalized scripts in `script/` are the entrypoints for the
development workflow, and the Makefile targets are thin shims that call them. We development workflow, and the Makefile targets are thin shims that call them. We
provide: provide:
- `script/bootstrap` — install all dependencies (Go, Go module download, and - `script/bootstrap` — install all dependencies, idempotently: Go and the
node/yarn plus the prettier version pinned in `package.json`/`yarn.lock`), modules of both `go.mod` and `bin/tools/go.mod`; node (the version `.nvmrc`
idempotently; golangci-lint is not installed, it runs only in Docker names, through nvm when there is no node on `PATH`) and yarn, plus the
prettier version pinned in `package.json`/`yarn.lock`; and `protoc` 33.4,
unpacked into `bin/protoc` from its release archive once the archive matches
the sha256 the script holds for this platform. golangci-lint is not installed,
it runs only in Docker
- `script/setup` — make a fresh clone ready for development: runs - `script/setup` — make a fresh clone ready for development: runs
`script/bootstrap`, then `script/install-precommit` `script/bootstrap`, then `script/install-precommit`
- `script/projectname` — output the project name (`mfer`); used by other scripts - `script/projectname` — output the project name (`mfer`); used by other scripts
@@ -82,14 +86,10 @@ provide:
- `script/generate` (`make generate`) — regenerate `mfer/mf.pb.go` from - `script/generate` (`make generate`) — regenerate `mfer/mf.pb.go` from
`mfer/mf.proto` and record the hash of that `mfer/mf.proto` in `mfer/mf.proto` and record the hash of that `mfer/mf.proto` in
`mfer/mf.proto.sha256`; the only thing that regenerates the committed `mfer/mf.proto.sha256`; the only thing that regenerates the committed
`mfer/mf.pb.go`. It needs the exact versions that wrote the committed file, `mfer/mf.pb.go`. It runs the `protoc` that `script/bootstrap` unpacks into
and refuses to run with any other: `protoc` 33.4 (unpack `bin/protoc`, refusing any version but 33.4, and the `protoc-gen-go` that
`protoc-33.4-<platform>.zip` from `bin/tools/go.mod` pins, which `go tool` builds from source checked against
[its release](https://github.com/protocolbuffers/protobuf/releases/tag/v33.4) the hashes in `bin/tools/go.sum`
and put its `bin/protoc` on `PATH`) and `protoc-gen-go` v1.36.11
(`go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11`, which
installs it in `$(go env GOPATH)/bin`; `script/generate` adds that directory
to `PATH`)
- `script/fuzz` — fuzz the manifest parser for one minute; run by hand - `script/fuzz` — fuzz the manifest parser for one minute; run by hand
(`make fuzz`), never by CI, while `script/test` runs its committed seed corpus (`make fuzz`), never by CI, while `script/test` runs its committed seed corpus
as ordinary tests as ordinary tests
@@ -99,14 +99,17 @@ provide:
- `script/fmt` — format all code and docs (writes): `script/gofumpt --write` and - `script/fmt` — format all code and docs (writes): `script/gofumpt --write` and
`script/prettier --write` `script/prettier --write`
- `script/gofumpt` — run `gofumpt` over every Go file in the repository in the - `script/gofumpt` — run `gofumpt` over every Go file in the repository in the
given mode, `--write` or `--check`, at the one version it pins (built on given mode, `--write` or `--check`, at the version `bin/tools/go.mod` pins
demand by `go run`, so nothing installs it); `script/fmt`, `script/fmt-check` (built on demand by `go tool` from source checked against the hashes in
`bin/tools/go.sum`, so nothing installs it); `script/fmt`, `script/fmt-check`
and the Docker lint stage all go through it, so they cannot disagree about Go and the Docker lint stage all go through it, so they cannot disagree about Go
formatting formatting
- `script/prettier` — run prettier over the repository's canonical file set - `script/prettier` — run prettier over the repository's canonical file set
(Markdown and JSON, minus `.prettierignore`) in the given mode, `--write` or (Markdown and JSON, minus `.prettierignore`) in the given mode, `--write` or
`--check`; the single definition of that file set, so `script/fmt` and `--check`, with the prettier version `yarn.lock` pins, run by the node on
`script/fmt-check` cannot disagree about it `PATH` or else the one `script/bootstrap` installed through nvm; the single
definition of that file set, so `script/fmt` and `script/fmt-check` cannot
disagree about it
- `script/fmt-check` — check formatting without writing: - `script/fmt-check` — check formatting without writing:
`script/gofumpt --check` plus `script/prettier --check` `script/gofumpt --check` plus `script/prettier --check`
- `script/check` — run `script/test`, `script/lint`, and `script/fmt-check` - `script/check` — run `script/test`, `script/lint`, and `script/fmt-check`
+22
View File
@@ -0,0 +1,22 @@
// The developer tools this repo runs with `go tool`: gofumpt for
// script/gofumpt and protoc-gen-go for script/generate. Kept out of the mfer
// module so they add nothing to what mfer's users download. `go tool` builds
// exactly the source whose hashes go.sum here records.
module sneak.berlin/go/mfer/bin/tools
go 1.26.0
tool (
google.golang.org/protobuf/cmd/protoc-gen-go
mvdan.cc/gofumpt
)
require (
golang.org/x/mod v0.40.0 // indirect
golang.org/x/sync v0.22.0 // indirect
golang.org/x/tools v0.49.0 // indirect
// protoc-gen-go v1.36.11, 2026-10-04
google.golang.org/protobuf v1.36.11 // indirect
// gofumpt v0.12.0, 2026-10-04
mvdan.cc/gofumpt v0.12.0 // indirect
)
+22
View File
@@ -0,0 +1,22 @@
github.com/go-quicktest/qt v1.102.0 h1:HSQxCeh5YZH3EL3W39ixjtyaEhcWSXQHtHnMBzSs474=
github.com/go-quicktest/qt v1.102.0/go.mod h1:p4lGIVX+8Wa6ZPNDvqcxq36XpUDLh42FLetFU7odllI=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/rogpeppe/go-internal v1.16.0 h1:O9DK+vNMDVGLr2BeZqmpLeMjiMNkuXfcqntWbZV6S5g=
github.com/rogpeppe/go-internal v1.16.0/go.mod h1:DrUVZyrJU+txYW5/1kwtXQSMFio52ZOxX7yM1VHvnxs=
golang.org/x/mod v0.40.0 h1:hUv+3cXcdRHz08UmSiOob7sadHig73uo5bkXxQ/tvUs=
golang.org/x/mod v0.40.0/go.mod h1:0/weTWkPWGBikyTWAX3dkjVztMmBA5hM0DH6BElSupE=
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/tools v0.49.0 h1:3NI7VXzL9+1WZD52Dx2ttoPwD5DWrFGpl9mFZDlmisI=
golang.org/x/tools v0.49.0/go.mod h1:SJNXV9DBKT0UbdttsQjbfJlAE/q+y36++zo3uL3N0Oo=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
mvdan.cc/gofumpt v0.12.0 h1:1Lbudkz2kpM9Cjz2pL4M19u7q+GaEhCTNf7N9mfpcho=
mvdan.cc/gofumpt v0.12.0/go.mod h1:SmBHHrljiZu/uoypeKup3rFzP6eoC9UwCp2iH5E3jZA=
+3 -9
View File
@@ -9,27 +9,21 @@ require (
github.com/klauspost/compress v1.18.2 github.com/klauspost/compress v1.18.2
github.com/multiformats/go-multihash v0.2.3 github.com/multiformats/go-multihash v0.2.3
github.com/spf13/afero v1.8.0 github.com/spf13/afero v1.8.0
github.com/stretchr/testify v1.8.1 github.com/stretchr/testify v1.12.1
github.com/urfave/cli/v2 v2.27.7 github.com/urfave/cli/v3 v3.14.0
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 golang.org/x/term v0.0.0-20210927222741-03fcf44c2211
google.golang.org/protobuf v1.28.1 google.golang.org/protobuf v1.28.1
) )
require ( require (
github.com/cpuguy83/go-md2man/v2 v2.0.7 // indirect
github.com/klauspost/cpuid/v2 v2.0.9 // indirect github.com/klauspost/cpuid/v2 v2.0.9 // indirect
github.com/kr/pretty v0.2.0 // indirect
github.com/minio/sha256-simd v1.0.0 // indirect github.com/minio/sha256-simd v1.0.0 // indirect
github.com/mr-tron/base58 v1.2.0 // indirect github.com/mr-tron/base58 v1.2.0 // indirect
github.com/multiformats/go-varint v0.0.6 // indirect github.com/multiformats/go-varint v0.0.6 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/spaolacci/murmur3 v1.1.0 // indirect github.com/spaolacci/murmur3 v1.1.0 // indirect
github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1 // indirect go.yaml.in/yaml/v3 v3.0.5 // indirect
golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e // indirect golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e // indirect
golang.org/x/sys v0.1.0 // indirect golang.org/x/sys v0.1.0 // indirect
golang.org/x/text v0.3.6 // indirect golang.org/x/text v0.3.6 // indirect
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
lukechampine.com/blake3 v1.1.6 // indirect lukechampine.com/blake3 v1.1.6 // indirect
) )
+6 -22
View File
@@ -46,8 +46,6 @@ github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDk
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc= github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/cpuguy83/go-md2man/v2 v2.0.7 h1:zbFlGlXEAKlwXpmvle3d8Oe3YnkKIK4xSRTd3sHPnBo=
github.com/cpuguy83/go-md2man/v2 v2.0.7/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -134,10 +132,7 @@ github.com/klauspost/cpuid/v2 v2.0.9 h1:lgaqFMSdTdQYdZ04uHyN2d/eKdOMyi2YLSvlQIBF
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg= github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg= github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.2.0 h1:s5hAObm+yFO5uHYt5dYjxi2rXrsnmRpJx4OYvIWUaQs=
github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/minio/sha256-simd v1.0.0 h1:v1ta+49hkWZyvaKwrQB8elexRqm6Y0aMLjCNsrYxo6g= github.com/minio/sha256-simd v1.0.0 h1:v1ta+49hkWZyvaKwrQB8elexRqm6Y0aMLjCNsrYxo6g=
github.com/minio/sha256-simd v1.0.0/go.mod h1:OuYzVNI5vcoYIAmbIvHPl3N3jUzVedXbKy5RFepssQM= github.com/minio/sha256-simd v1.0.0/go.mod h1:OuYzVNI5vcoYIAmbIvHPl3N3jUzVedXbKy5RFepssQM=
@@ -149,30 +144,21 @@ github.com/multiformats/go-varint v0.0.6 h1:gk85QWKxh3TazbLxED/NlDVv8+q+ReFJk7Y2
github.com/multiformats/go-varint v0.0.6/go.mod h1:3Ls8CIEsrijN6+B7PbrXRPxHRPuXSrVKRY101jdMZYE= github.com/multiformats/go-varint v0.0.6/go.mod h1:3Ls8CIEsrijN6+B7PbrXRPxHRPuXSrVKRY101jdMZYE=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/sftp v1.13.1/go.mod h1:3HaPG6Dq1ILlpPZRO0HVMrsydcdLt6HRDccSgb87qRg= github.com/pkg/sftp v1.13.1/go.mod h1:3HaPG6Dq1ILlpPZRO0HVMrsydcdLt6HRDccSgb87qRg=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI= github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI=
github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/spf13/afero v1.8.0 h1:5MmtuhAgYeU6qpa7w7bP0dv6MBYuup0vekhSpSkoq60= github.com/spf13/afero v1.8.0 h1:5MmtuhAgYeU6qpa7w7bP0dv6MBYuup0vekhSpSkoq60=
github.com/spf13/afero v1.8.0/go.mod h1:CtAatgMJh6bJEIs48Ay/FOnkljP3WeGUG0MC1RfAqwo= github.com/spf13/afero v1.8.0/go.mod h1:CtAatgMJh6bJEIs48Ay/FOnkljP3WeGUG0MC1RfAqwo=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk= github.com/urfave/cli/v3 v3.14.0 h1:a8414NQlHJs0c/iBsulKLzlES0n/lEAskbL2LKpU4/s=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= github.com/urfave/cli/v3 v3.14.0/go.mod h1:vXn6HxPNccJSzQr2QvwVncOKrgYGIHU0HY5h8B2nQj4=
github.com/urfave/cli/v2 v2.27.7 h1:bH59vdhbjLv3LAvIu6gd0usJHgoTTPhCFib8qqOwXYU=
github.com/urfave/cli/v2 v2.27.7/go.mod h1:CyNAG/xg+iAOg0N4MPGZqVmv2rCoP267496AOXUZjA4=
github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1 h1:gEOO8jv9F4OT7lGCjxCBTO/36wtF6j2nSip77qHd4x4=
github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1/go.mod h1:Ohn+xnUBiLI6FVj/9LpzZWtj1/D6lUovWYBkxHVV3aM=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
@@ -183,6 +169,8 @@ go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk= go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
@@ -470,13 +458,9 @@ google.golang.org/protobuf v1.28.1 h1:d0NfwRgPtno5B1Wa6L2DAG+KivqkdutMf1UhdNx175
google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+26 -17
View File
@@ -16,7 +16,7 @@ import (
"github.com/dustin/go-humanize" "github.com/dustin/go-humanize"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/urfave/cli/v2" "github.com/urfave/cli/v3"
"sneak.berlin/go/mfer/internal/log" "sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -96,8 +96,10 @@ func findManifest(fs afero.Fs, dir string) (string, error) {
// fetchManifestToTemp downloads a manifest URL to a temporary file and // fetchManifestToTemp downloads a manifest URL to a temporary file and
// returns the temp file path. The caller is responsible for removing it. // returns the temp file path. The caller is responsible for removing it.
func (mfa *CLIApp) fetchManifestToTemp(url string) (string, error) { func (mfa *CLIApp) fetchManifestToTemp(
rc, fetchErr := mfa.openManifestReader(url) ctx context.Context, url string,
) (string, error) {
rc, fetchErr := mfa.openManifestReader(ctx, url)
if fetchErr != nil { if fetchErr != nil {
return "", fetchErr return "", fetchErr
} }
@@ -208,13 +210,15 @@ func countCheckFailures(
// findExtraFiles reports files present on disk but absent from the // findExtraFiles reports files present on disk but absent from the
// manifest, and anything the search cannot read: each is a failure under // manifest, and anything the search cannot read: each is a failure under
// --no-extra-files, otherwise a warning. // --no-extra-files, otherwise a warning.
func findExtraFiles(ctx *cli.Context, chk *mfer.Checker, failures *int64) error { func findExtraFiles(
ctx context.Context, cmd *cli.Command, chk *mfer.Checker, failures *int64,
) error {
extraResults := make(chan mfer.Result, 1) extraResults := make(chan mfer.Result, 1)
extraDone := make(chan struct{}) extraDone := make(chan struct{})
go func() { go func() {
for result := range extraResults { for result := range extraResults {
if ctx.Bool("no-extra-files") { if cmd.Bool("no-extra-files") {
*failures++ *failures++
log.Infof("%s: %s (%s)", result.Status, result.Path, result.Message) log.Infof("%s: %s (%s)", result.Status, result.Path, result.Message)
@@ -226,7 +230,7 @@ func findExtraFiles(ctx *cli.Context, chk *mfer.Checker, failures *int64) error
close(extraDone) close(extraDone)
}() }()
err := chk.FindExtraFiles(ctx.Context, extraResults) err := chk.FindExtraFiles(ctx, extraResults)
if err != nil { if err != nil {
return fmt.Errorf("failed to check for extra files: %w", err) return fmt.Errorf("failed to check for extra files: %w", err)
} }
@@ -238,7 +242,9 @@ func findExtraFiles(ctx *cli.Context, chk *mfer.Checker, failures *int64) error
// runCheck runs the manifest check with progress and result reporting // runCheck runs the manifest check with progress and result reporting
// and returns the number of failures. // and returns the number of failures.
func runCheck(ctx *cli.Context, chk *mfer.Checker, showProgress bool) (int64, error) { func runCheck(
ctx context.Context, cmd *cli.Command, chk *mfer.Checker, showProgress bool,
) (int64, error) {
// Set up results channel // Set up results channel
results := make(chan mfer.Result, 1) results := make(chan mfer.Result, 1)
@@ -264,7 +270,7 @@ func runCheck(ctx *cli.Context, chk *mfer.Checker, showProgress bool) (int64, er
go countCheckFailures(results, &failures, done) go countCheckFailures(results, &failures, done)
// Run check // Run check
err := chk.Check(ctx.Context, results, progress) err := chk.Check(ctx, results, progress)
progressWg.Wait() progressWg.Wait()
@@ -275,7 +281,7 @@ func runCheck(ctx *cli.Context, chk *mfer.Checker, showProgress bool) (int64, er
// Wait for results processing to complete // Wait for results processing to complete
<-done <-done
err = findExtraFiles(ctx, chk, &failures) err = findExtraFiles(ctx, cmd, chk, &failures)
if err != nil { if err != nil {
return 0, err return 0, err
} }
@@ -283,17 +289,19 @@ func runCheck(ctx *cli.Context, chk *mfer.Checker, showProgress bool) (int64, er
return failures, nil return failures, nil
} }
func (mfa *CLIApp) checkManifestOperation(ctx *cli.Context) error { func (mfa *CLIApp) checkManifestOperation(
ctx context.Context, cmd *cli.Command,
) error {
log.Debug("checkManifestOperation()") log.Debug("checkManifestOperation()")
manifestPath, err := mfa.resolveManifestArg(ctx) manifestPath, err := mfa.resolveManifestArg(cmd)
if err != nil { if err != nil {
return fmt.Errorf("check: %w", err) return fmt.Errorf("check: %w", err)
} }
// URL manifests need to be downloaded to a temp file for the checker // URL manifests need to be downloaded to a temp file for the checker
if isHTTPURL(manifestPath) { if isHTTPURL(manifestPath) {
tmpPath, tmpErr := mfa.fetchManifestToTemp(manifestPath) tmpPath, tmpErr := mfa.fetchManifestToTemp(ctx, manifestPath)
if tmpErr != nil { if tmpErr != nil {
return fmt.Errorf("check: %w", tmpErr) return fmt.Errorf("check: %w", tmpErr)
} }
@@ -303,12 +311,13 @@ func (mfa *CLIApp) checkManifestOperation(ctx *cli.Context) error {
manifestPath = tmpPath manifestPath = tmpPath
} }
basePath := ctx.String("base") basePath := cmd.String("base")
showProgress := ctx.Bool("progress") showProgress := cmd.Bool("progress")
log.Infof("checking manifest %s with base %s", manifestPath, basePath) log.Infof("checking manifest %s with base %s", manifestPath, basePath)
// Create checker // Create checker
//nolint:contextcheck // mfer loads a manifest without a context
chk, err := mfer.NewChecker(&mfer.CheckerOptions{ chk, err := mfer.NewChecker(&mfer.CheckerOptions{
ManifestPath: manifestPath, ManifestPath: manifestPath,
BasePath: basePath, BasePath: basePath,
@@ -319,9 +328,9 @@ func (mfa *CLIApp) checkManifestOperation(ctx *cli.Context) error {
} }
// Check signature requirement // Check signature requirement
requiredSigner := ctx.String(flagRequireSignature) requiredSigner := cmd.String(flagRequireSignature)
if requiredSigner != "" { if requiredSigner != "" {
err = verifyRequiredSigner(ctx.Context, chk, requiredSigner) err = verifyRequiredSigner(ctx, chk, requiredSigner)
if err != nil { if err != nil {
return err return err
} }
@@ -330,7 +339,7 @@ func (mfa *CLIApp) checkManifestOperation(ctx *cli.Context) error {
log.Infof("manifest contains %d files, %s", chk.FileCount(), log.Infof("manifest contains %d files, %s", chk.FileCount(),
humanize.IBytes(safeUint64(int64(chk.TotalBytes())))) humanize.IBytes(safeUint64(int64(chk.TotalBytes()))))
failures, err := runCheck(ctx, chk, showProgress) failures, err := runCheck(ctx, cmd, chk, showProgress)
if err != nil { if err != nil {
return err return err
} }
+21 -25
View File
@@ -18,7 +18,7 @@ import (
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
urfcli "github.com/urfave/cli/v2" urfcli "github.com/urfave/cli/v3"
"sneak.berlin/go/mfer/internal/log" "sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -264,8 +264,9 @@ func commandsTakingVerbose() [][]string {
} }
// TestVerboseCount asserts that one -v or --verbose gives verbose output and // TestVerboseCount asserts that one -v or --verbose gives verbose output and
// two -v give debug output (issue #125). urfave/cli before v2.25.5 counted a // two -v, or -v and --verbose, give debug output (issue #125). urfave/cli
// flag given by its alias twice, so one -v gave debug output. // before v2.25.5 counted a flag given by its alias twice, so one -v gave debug
// output.
func TestVerboseCount(t *testing.T) { func TestVerboseCount(t *testing.T) {
t.Parallel() t.Parallel()
@@ -276,6 +277,7 @@ func TestVerboseCount(t *testing.T) {
{[]string{"-v"}, log.VerboseLevel}, {[]string{"-v"}, log.VerboseLevel},
{[]string{testFlagVerbose}, log.VerboseLevel}, {[]string{testFlagVerbose}, log.VerboseLevel},
{[]string{"-v", "-v"}, log.DebugLevel}, {[]string{"-v", "-v"}, log.DebugLevel},
{[]string{"-v", testFlagVerbose}, log.DebugLevel},
} }
for _, command := range commandsTakingVerbose() { for _, command := range commandsTakingVerbose() {
@@ -313,28 +315,6 @@ func TestCombinedShortVerboseRefused(t *testing.T) {
} }
} }
// TestShortAndLongVerboseRefused asserts that -v and --verbose given together
// are refused (issue #125): urfave/cli v2 refuses a flag given under two of its
// names, and one flag with an alias keeps help and parsing simple.
func TestShortAndLongVerboseRefused(t *testing.T) {
t.Parallel()
for _, command := range commandsTakingVerbose() {
args := slices.Concat(command, []string{"-v", testFlagVerbose})
t.Run(strings.Join(args, " "), func(t *testing.T) {
t.Parallel()
opts := testOpts(args, afero.NewMemMapFs())
exitCode, level := runCLIWithLevel(opts)
assert.Equal(t, 1, exitCode)
assert.Contains(t, testStderr(t, opts), "Cannot use two forms of the same flag")
assert.Equal(t, log.InfoLevel, level)
})
}
}
func TestHelpCommand(t *testing.T) { func TestHelpCommand(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1190,6 +1170,22 @@ func TestGenerateValidatesInputPaths(t *testing.T) {
}) })
} }
// TestFlagAfterArgumentIsArgument asserts that flags are read only before a
// command's first argument: after it, -v is a path, not the verbose flag.
func TestFlagAfterArgumentIsArgument(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll(testDir, 0o755))
writeTestFile(t, fs, testFile1, "content")
opts := testOpts([]string{testApp, cmdGenerate, testDir, "-v"}, fs)
exitCode := runCLI(opts)
assert.Equal(t, 1, exitCode)
assert.Contains(t, testStderr(t, opts), "path does not exist: -v")
}
func TestCheckDetectsManifestCorruption(t *testing.T) { func TestCheckDetectsManifestCorruption(t *testing.T) {
t.Parallel() t.Parallel()
+33 -31
View File
@@ -4,7 +4,6 @@ package cli
import ( import (
"bytes" "bytes"
"context" "context"
"flag"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"os" "os"
@@ -15,7 +14,7 @@ import (
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
urfcli "github.com/urfave/cli/v2" urfcli "github.com/urfave/cli/v3"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -195,13 +194,17 @@ func signedChecker(t *testing.T, manifest []byte) *mfer.Checker {
func TestPathDoesNotExistMessage(t *testing.T) { func TestPathDoesNotExistMessage(t *testing.T) {
t.Parallel() t.Parallel()
set := flag.NewFlagSet("gen", flag.ContinueOnError)
require.NoError(t, set.Parse([]string{"nope"}))
mfa := &CLIApp{Fs: afero.NewMemMapFs()} mfa := &CLIApp{Fs: afero.NewMemMapFs()}
ctx := urfcli.NewContext(nil, set, nil) cmd := &urfcli.Command{
Name: cmdGenerate,
Action: func(_ context.Context, c *urfcli.Command) error {
_, err := mfa.collectInputPaths(c.Args())
_, err := mfa.collectInputPaths(ctx.Args()) return err
},
}
err := cmd.Run(context.Background(), []string{cmdGenerate, "nope"})
require.ErrorIs(t, err, errPathNotExist) require.ErrorIs(t, err, errPathNotExist)
assert.EqualError(t, err, "path does not exist: nope") assert.EqualError(t, err, "path does not exist: nope")
} }
@@ -214,18 +217,22 @@ func TestOutputFileExistsMessage(t *testing.T) {
require.NoError(t, afero.WriteFile(fs, "/d/f.txt", []byte("hi"), 0o644)) require.NoError(t, afero.WriteFile(fs, "/d/f.txt", []byte("hi"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/out.mf", []byte("old"), 0o644)) require.NoError(t, afero.WriteFile(fs, "/out.mf", []byte("old"), 0o644))
set := flag.NewFlagSet("gen", flag.ContinueOnError)
set.String("output", "", "")
set.Bool("force", false, "")
require.NoError(t, set.Parse([]string{"/d"}))
require.NoError(t, set.Set("output", "/out.mf"))
mfa := &CLIApp{Fs: fs} mfa := &CLIApp{Fs: fs}
ctx := urfcli.NewContext(nil, set, nil) cmd := &urfcli.Command{
Name: cmdGenerate,
Flags: []urfcli.Flag{
&urfcli.StringFlag{Name: "output"},
&urfcli.BoolFlag{Name: "force"},
},
Action: mfa.generateManifestOperation,
}
// generateManifestOperation writes to the process-global logger during // generateManifestOperation writes to the process-global logger during
// enumeration, so serialize with the other CLI runs. // enumeration, so serialize with the other CLI runs.
err := runLocked(func() error { return mfa.generateManifestOperation(ctx) }) err := runLocked(func() error {
return cmd.Run(context.Background(),
[]string{cmdGenerate, "--output", "/out.mf", "/d"})
})
require.ErrorIs(t, err, errOutputExists) require.ErrorIs(t, err, errOutputExists)
assert.EqualError(t, err, assert.EqualError(t, err,
"output file /out.mf already exists (use --force to overwrite)") "output file /out.mf already exists (use --force to overwrite)")
@@ -250,7 +257,7 @@ func TestUnknownCommandMessage(t *testing.T) {
err := runLocked(func() error { err := runLocked(func() error {
mfa.run([]string{testApp}) mfa.run([]string{testApp})
return mfa.app.Run([]string{testApp, "bogus"}) return mfa.app.Run(context.Background(), []string{testApp, "bogus"})
}) })
require.ErrorIs(t, err, errUnknownCommand) require.ErrorIs(t, err, errUnknownCommand)
assert.EqualError(t, err, `unknown command "bogus"`) assert.EqualError(t, err, `unknown command "bogus"`)
@@ -267,7 +274,7 @@ func TestManifestLoaderHTTPStatusMessage(t *testing.T) {
mfa := &CLIApp{Fs: afero.NewMemMapFs()} mfa := &CLIApp{Fs: afero.NewMemMapFs()}
_, err := mfa.openManifestReader(server.URL + "/foo.mf") _, err := mfa.openManifestReader(context.Background(), server.URL+"/foo.mf")
require.ErrorIs(t, err, errHTTPStatus) require.ErrorIs(t, err, errHTTPStatus)
assert.EqualError(t, err, assert.EqualError(t, err,
"failed to fetch "+server.URL+"/foo.mf: HTTP 404") "failed to fetch "+server.URL+"/foo.mf: HTTP 404")
@@ -284,17 +291,13 @@ func TestFetchManifestHTTPStatusMessage(t *testing.T) {
mfa := &CLIApp{Fs: afero.NewMemMapFs()} mfa := &CLIApp{Fs: afero.NewMemMapFs()}
set := flag.NewFlagSet(cmdFetch, flag.ContinueOnError) cmd := mfa.fetchCommand()
for _, f := range mfa.fetchCommand().Flags { cmd.Action = mfa.fetchManifestOperation
require.NoError(t, f.Apply(set))
}
require.NoError(t, set.Parse([]string{server.URL}))
ctx := urfcli.NewContext(nil, set, nil)
// fetchManifestOperation logs to the process-global logger. // fetchManifestOperation logs to the process-global logger.
err := runLocked(func() error { return mfa.fetchManifestOperation(ctx) }) err := runLocked(func() error {
return cmd.Run(context.Background(), []string{cmdFetch, server.URL})
})
require.ErrorIs(t, err, errHTTPStatus) require.ErrorIs(t, err, errHTTPStatus)
assert.EqualError(t, err, "failed to fetch manifest: HTTP 404") assert.EqualError(t, err, "failed to fetch manifest: HTTP 404")
} }
@@ -320,14 +323,13 @@ func TestFetchFileHTTPStatusMessage(t *testing.T) {
func TestURLRequiredMessage(t *testing.T) { func TestURLRequiredMessage(t *testing.T) {
t.Parallel() t.Parallel()
set := flag.NewFlagSet("fetch", flag.ContinueOnError)
require.NoError(t, set.Parse([]string{}))
mfa := &CLIApp{Fs: afero.NewMemMapFs()} mfa := &CLIApp{Fs: afero.NewMemMapFs()}
ctx := urfcli.NewContext(nil, set, nil) cmd := &urfcli.Command{Name: cmdFetch, Action: mfa.fetchManifestOperation}
// fetchManifestOperation logs to the process-global logger. // fetchManifestOperation logs to the process-global logger.
err := runLocked(func() error { return mfa.fetchManifestOperation(ctx) }) err := runLocked(func() error {
return cmd.Run(context.Background(), []string{cmdFetch})
})
require.ErrorIs(t, err, errURLRequired) require.ErrorIs(t, err, errURLRequired)
assert.EqualError(t, err, "URL argument required") assert.EqualError(t, err, "URL argument required")
} }
+8 -4
View File
@@ -1,12 +1,13 @@
package cli package cli
import ( import (
"context"
"encoding/hex" "encoding/hex"
"encoding/json" "encoding/json"
"fmt" "fmt"
"time" "time"
"github.com/urfave/cli/v2" "github.com/urfave/cli/v3"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -19,19 +20,22 @@ type ExportEntry struct {
Ctime *string `json:"ctime,omitempty"` Ctime *string `json:"ctime,omitempty"`
} }
func (mfa *CLIApp) exportManifestOperation(ctx *cli.Context) error { func (mfa *CLIApp) exportManifestOperation(
pathOrURL, err := mfa.resolveManifestArg(ctx) ctx context.Context, cmd *cli.Command,
) error {
pathOrURL, err := mfa.resolveManifestArg(cmd)
if err != nil { if err != nil {
return fmt.Errorf("export: %w", err) return fmt.Errorf("export: %w", err)
} }
rc, err := mfa.openManifestReader(pathOrURL) rc, err := mfa.openManifestReader(ctx, pathOrURL)
if err != nil { if err != nil {
return fmt.Errorf("export: %w", err) return fmt.Errorf("export: %w", err)
} }
defer func() { _ = rc.Close() }() defer func() { _ = rc.Close() }()
//nolint:contextcheck // mfer loads a manifest without a context
manifest, err := mfer.NewManifestFromReader(rc) manifest, err := mfer.NewManifestFromReader(rc)
if err != nil { if err != nil {
return fmt.Errorf("export: failed to parse manifest: %w", err) return fmt.Errorf("export: failed to parse manifest: %w", err)
+17 -13
View File
@@ -20,7 +20,7 @@ import (
"github.com/dustin/go-humanize" "github.com/dustin/go-humanize"
"github.com/multiformats/go-multihash" "github.com/multiformats/go-multihash"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/urfave/cli/v2" "github.com/urfave/cli/v3"
"sneak.berlin/go/mfer/internal/log" "sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -328,19 +328,21 @@ func alreadyPresent(dest, localPath string, entry *mfer.MFFilePath) bool {
return verifyDownloadedHash(h.Sum(nil), entry) == nil return verifyDownloadedHash(h.Sum(nil), entry) == nil
} }
func (mfa *CLIApp) fetchManifestOperation(ctx *cli.Context) error { func (mfa *CLIApp) fetchManifestOperation(
ctx context.Context, cmd *cli.Command,
) error {
log.Debug("fetchManifestOperation()") log.Debug("fetchManifestOperation()")
if ctx.Args().Len() == 0 { if cmd.Args().Len() == 0 {
return errURLRequired return errURLRequired
} }
timeout := ctx.Duration(flagTimeout) timeout := cmd.Duration(flagTimeout)
if timeout <= 0 { if timeout <= 0 {
return errInvalidTimeout return errInvalidTimeout
} }
manifestURL, err := resolveManifestURL(ctx.Args().Get(0)) manifestURL, err := resolveManifestURL(cmd.Args().Get(0))
if err != nil { if err != nil {
return fmt.Errorf("invalid URL: %w", err) return fmt.Errorf("invalid URL: %w", err)
} }
@@ -350,7 +352,7 @@ func (mfa *CLIApp) fetchManifestOperation(ctx *cli.Context) error {
firstDelay: firstRetryDelay, firstDelay: firstRetryDelay,
} }
manifestData, files, err := fetchManifest(ctx, client, manifestURL) manifestData, files, err := fetchManifest(ctx, cmd, client, manifestURL)
if err != nil { if err != nil {
return err return err
} }
@@ -361,7 +363,7 @@ func (mfa *CLIApp) fetchManifestOperation(ctx *cli.Context) error {
return err return err
} }
dest := ctx.String(flagDest) dest := cmd.String(flagDest)
err = os.MkdirAll(dest, dirPerms) err = os.MkdirAll(dest, dirPerms)
if err != nil { if err != nil {
@@ -379,7 +381,7 @@ func (mfa *CLIApp) fetchManifestOperation(ctx *cli.Context) error {
// Download each file // Download each file
downloaded, downloadedBytes, dlErr := downloadManifestFiles( downloaded, downloadedBytes, dlErr := downloadManifestFiles(
ctx.Context, client, baseURL, dest, files, progress) ctx, client, baseURL, dest, files, progress)
close(progress) close(progress)
<-done <-done
@@ -416,7 +418,7 @@ func (mfa *CLIApp) fetchManifestOperation(ctx *cli.Context) error {
// downloaded, to be saved once the files are in place, and the files it // downloaded, to be saved once the files are in place, and the files it
// lists. // lists.
func fetchManifest( func fetchManifest(
ctx *cli.Context, client retryingClient, manifestURL string, ctx context.Context, cmd *cli.Command, client retryingClient, manifestURL string,
) ([]byte, []*mfer.MFFilePath, error) { ) ([]byte, []*mfer.MFFilePath, error) {
log.Infof("fetching manifest from %s", manifestURL) log.Infof("fetching manifest from %s", manifestURL)
@@ -425,7 +427,7 @@ func fetchManifest(
// manifest. // manifest.
var manifestData []byte var manifestData []byte
err := client.get(ctx.Context, manifestURL, func(resp *http.Response) error { err := client.get(ctx, manifestURL, func(resp *http.Response) error {
var readErr error var readErr error
manifestData, readErr = io.ReadAll(resp.Body) manifestData, readErr = io.ReadAll(resp.Body)
@@ -437,12 +439,13 @@ func fetchManifest(
} }
// Parse manifest // Parse manifest
//nolint:contextcheck // mfer loads a manifest without a context
manifest, err := mfer.NewManifestFromReader(bytes.NewReader(manifestData)) manifest, err := mfer.NewManifestFromReader(bytes.NewReader(manifestData))
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("failed to parse manifest: %w", err) return nil, nil, fmt.Errorf("failed to parse manifest: %w", err)
} }
requiredSigner := ctx.String(flagRequireSignature) requiredSigner := cmd.String(flagRequireSignature)
if requiredSigner != "" { if requiredSigner != "" {
err = verifyFetchedSigner(ctx, manifestData, requiredSigner) err = verifyFetchedSigner(ctx, manifestData, requiredSigner)
if err != nil { if err != nil {
@@ -504,7 +507,7 @@ func checkNoNameClash(files []*mfer.MFFilePath) error {
// its manifest from a file, so the manifest is handed to it as a file in // its manifest from a file, so the manifest is handed to it as a file in
// memory. // memory.
func verifyFetchedSigner( func verifyFetchedSigner(
ctx *cli.Context, manifestData []byte, requiredSigner string, ctx context.Context, manifestData []byte, requiredSigner string,
) error { ) error {
memFs := afero.NewMemMapFs() memFs := afero.NewMemMapFs()
manifestPath := "/" + defaultManifestName manifestPath := "/" + defaultManifestName
@@ -514,6 +517,7 @@ func verifyFetchedSigner(
return err return err
} }
//nolint:contextcheck // mfer loads a manifest without a context
chk, err := mfer.NewChecker(&mfer.CheckerOptions{ chk, err := mfer.NewChecker(&mfer.CheckerOptions{
ManifestPath: manifestPath, ManifestPath: manifestPath,
BasePath: "/", BasePath: "/",
@@ -523,7 +527,7 @@ func verifyFetchedSigner(
return fmt.Errorf("failed to load manifest: %w", err) return fmt.Errorf("failed to load manifest: %w", err)
} }
return verifyRequiredSigner(ctx.Context, chk, requiredSigner) return verifyRequiredSigner(ctx, chk, requiredSigner)
} }
// saveManifest writes the fetched manifest into dest under the default // saveManifest writes the fetched manifest into dest under the default
+5 -12
View File
@@ -4,7 +4,6 @@ package cli
import ( import (
"bytes" "bytes"
"context" "context"
"flag"
"fmt" "fmt"
"io" "io"
"maps" "maps"
@@ -23,7 +22,6 @@ import (
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
urfcli "github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -1322,18 +1320,13 @@ func TestFetchTimeoutFlag(t *testing.T) {
mfa := &CLIApp{Fs: afero.NewMemMapFs()} mfa := &CLIApp{Fs: afero.NewMemMapFs()}
set := flag.NewFlagSet(cmdFetch, flag.ContinueOnError) cmd := mfa.fetchCommand()
for _, f := range mfa.fetchCommand().Flags { cmd.Action = mfa.fetchManifestOperation
require.NoError(t, f.Apply(set))
}
require.NoError(t, set.Parse([]string{"--" + flagTimeout, "100ms", server.URL}))
cliCtx := urfcli.NewContext(nil, set, nil)
cliCtx.Context = ctx
// fetchManifestOperation logs to the process-global logger. // fetchManifestOperation logs to the process-global logger.
err := runLocked(func() error { return mfa.fetchManifestOperation(cliCtx) }) err := runLocked(func() error {
return cmd.Run(ctx, []string{cmdFetch, "--" + flagTimeout, "100ms", server.URL})
})
require.Error(t, err) require.Error(t, err)
} }
+22 -19
View File
@@ -14,7 +14,7 @@ import (
"github.com/dustin/go-humanize" "github.com/dustin/go-humanize"
"github.com/multiformats/go-multihash" "github.com/multiformats/go-multihash"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/urfave/cli/v2" "github.com/urfave/cli/v3"
"sneak.berlin/go/mfer/internal/log" "sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -202,12 +202,12 @@ func (s *freshenScanner) walk(path string, info fs.FileInfo, walkErr error) erro
// resolveFreshenManifestPath determines the manifest path from the CLI // resolveFreshenManifestPath determines the manifest path from the CLI
// arguments, searching directories for a manifest where needed. // arguments, searching directories for a manifest where needed.
func (mfa *CLIApp) resolveFreshenManifestPath(ctx *cli.Context) (string, error) { func (mfa *CLIApp) resolveFreshenManifestPath(cmd *cli.Command) (string, error) {
if ctx.Args().Len() == 0 { if cmd.Args().Len() == 0 {
return findManifest(mfa.Fs, ".") return findManifest(mfa.Fs, ".")
} }
arg := ctx.Args().Get(0) arg := cmd.Args().Get(0)
info, statErr := mfa.Fs.Stat(arg) info, statErr := mfa.Fs.Stat(arg)
if statErr == nil && info.IsDir() { if statErr == nil && info.IsDir() {
@@ -338,14 +338,14 @@ func writeFreshenedManifest(
// newFreshenBuilder constructs the manifest builder configured from CLI // newFreshenBuilder constructs the manifest builder configured from CLI
// flags. // flags.
func newFreshenBuilder(ctx *cli.Context) *mfer.Builder { func newFreshenBuilder(cmd *cli.Command) *mfer.Builder {
builder := mfer.NewBuilder() builder := mfer.NewBuilder()
if ctx.Bool("include-timestamps") { if cmd.Bool("include-timestamps") {
builder.SetIncludeTimestamps(true) builder.SetIncludeTimestamps(true)
} }
// Set up signing options if sign-key is provided // Set up signing options if sign-key is provided
if signKey := ctx.String("sign-key"); signKey != "" { if signKey := cmd.String("sign-key"); signKey != "" {
builder.SetSigningOptions(&mfer.SigningOptions{ builder.SetSigningOptions(&mfer.SigningOptions{
KeyID: mfer.GPGKeyID(signKey), KeyID: mfer.GPGKeyID(signKey),
}) })
@@ -358,13 +358,13 @@ func newFreshenBuilder(ctx *cli.Context) *mfer.Builder {
// freshenScan runs the scan phase against the loaded manifest entries // freshenScan runs the scan phase against the loaded manifest entries
// and returns the populated scanner and the count of removed files. // and returns the populated scanner and the count of removed files.
func (mfa *CLIApp) freshenScan( func (mfa *CLIApp) freshenScan(
ctx *cli.Context, manifestPath, absBase string, cmd *cli.Command, manifestPath, absBase string,
existingByPath map[string]*mfer.MFFilePath, existingByPath map[string]*mfer.MFFilePath,
) (*freshenScanner, int64, error) { ) (*freshenScanner, int64, error) {
log.Infof("scanning filesystem...") log.Infof("scanning filesystem...")
startScan := time.Now() startScan := time.Now()
showProgress := ctx.Bool("progress") showProgress := cmd.Bool("progress")
// Leave out the manifest and a temp file left by an interrupted run, // Leave out the manifest and a temp file left by an interrupted run,
// as gen does. A path that cannot be stat'd, normally because no file // as gen does. A path that cannot be stat'd, normally because no file
@@ -382,8 +382,8 @@ func (mfa *CLIApp) freshenScan(
fs: mfa.Fs, fs: mfa.Fs,
absBase: absBase, absBase: absBase,
excluded: excluded, excluded: excluded,
includeDotfiles: ctx.Bool("include-dotfiles"), includeDotfiles: cmd.Bool("include-dotfiles"),
followSymlinks: ctx.Bool("follow-symlinks"), followSymlinks: cmd.Bool("follow-symlinks"),
showProgress: showProgress, showProgress: showProgress,
existingByPath: existingByPath, existingByPath: existingByPath,
} }
@@ -433,7 +433,7 @@ func hashTotals(entries []*freshenEntry) (int64, int64) {
// runFreshenHash processes every entry through the hasher, aborting if // runFreshenHash processes every entry through the hasher, aborting if
// the context is canceled. // the context is canceled.
func runFreshenHash( func runFreshenHash(
ctx *cli.Context, hasher *freshenHasher, entries []*freshenEntry, ctx context.Context, hasher *freshenHasher, entries []*freshenEntry,
) error { ) error {
for _, e := range entries { for _, e := range entries {
select { select {
@@ -479,18 +479,21 @@ func (mfa *CLIApp) loadExistingEntries(
return existingByPath, nil return existingByPath, nil
} }
func (mfa *CLIApp) freshenManifestOperation(ctx *cli.Context) error { func (mfa *CLIApp) freshenManifestOperation(
ctx context.Context, cmd *cli.Command,
) error {
log.Debug("freshenManifestOperation()") log.Debug("freshenManifestOperation()")
basePath := ctx.String("base") basePath := cmd.String("base")
showProgress := ctx.Bool("progress") showProgress := cmd.Bool("progress")
// Find manifest file // Find manifest file
manifestPath, err := mfa.resolveFreshenManifestPath(ctx) manifestPath, err := mfa.resolveFreshenManifestPath(cmd)
if err != nil { if err != nil {
return fmt.Errorf("freshen: %w", err) return fmt.Errorf("freshen: %w", err)
} }
//nolint:contextcheck // mfer loads a manifest without a context
existingByPath, err := mfa.loadExistingEntries(manifestPath) existingByPath, err := mfa.loadExistingEntries(manifestPath)
if err != nil { if err != nil {
return err return err
@@ -502,7 +505,7 @@ func (mfa *CLIApp) freshenManifestOperation(ctx *cli.Context) error {
} }
// Phase 1: Scan filesystem // Phase 1: Scan filesystem
scanner, removed, err := mfa.freshenScan(ctx, manifestPath, absBase, scanner, removed, err := mfa.freshenScan(cmd, manifestPath, absBase,
existingByPath) existingByPath)
if err != nil { if err != nil {
return err return err
@@ -524,7 +527,7 @@ func (mfa *CLIApp) freshenManifestOperation(ctx *cli.Context) error {
totalHashBytes: totalHashBytes, totalHashBytes: totalHashBytes,
filesToHash: filesToHash, filesToHash: filesToHash,
startHash: time.Now(), startHash: time.Now(),
builder: newFreshenBuilder(ctx), builder: newFreshenBuilder(cmd),
} }
err = runFreshenHash(ctx, hasher, scanner.entries) err = runFreshenHash(ctx, hasher, scanner.entries)
@@ -548,7 +551,7 @@ func (mfa *CLIApp) freshenManifestOperation(ctx *cli.Context) error {
} }
// Write updated manifest atomically (write to temp, then rename) // Write updated manifest atomically (write to temp, then rename)
err = writeFreshenedManifest(ctx.Context, mfa.Fs, hasher.builder, manifestPath) err = writeFreshenedManifest(ctx, mfa.Fs, hasher.builder, manifestPath)
if err != nil { if err != nil {
return err return err
} }
+21 -18
View File
@@ -1,6 +1,7 @@
package cli package cli
import ( import (
"context"
"errors" "errors"
"fmt" "fmt"
"os" "os"
@@ -12,7 +13,7 @@ import (
"github.com/dustin/go-humanize" "github.com/dustin/go-humanize"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/urfave/cli/v2" "github.com/urfave/cli/v3"
"sneak.berlin/go/mfer/internal/log" "sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -88,12 +89,12 @@ func (mfa *CLIApp) collectInputPaths(args cli.Args) ([]string, error) {
} }
// buildScannerOptions constructs scanner options from the CLI flags. // buildScannerOptions constructs scanner options from the CLI flags.
func (mfa *CLIApp) buildScannerOptions(ctx *cli.Context) *mfer.ScannerOptions { func (mfa *CLIApp) buildScannerOptions(cmd *cli.Command) *mfer.ScannerOptions {
output := ctx.String("output") output := cmd.String("output")
opts := &mfer.ScannerOptions{ opts := &mfer.ScannerOptions{
IncludeDotfiles: ctx.Bool("include-dotfiles"), IncludeDotfiles: cmd.Bool("include-dotfiles"),
FollowSymLinks: ctx.Bool("follow-symlinks"), FollowSymLinks: cmd.Bool("follow-symlinks"),
IncludeTimestamps: ctx.Bool("include-timestamps"), IncludeTimestamps: cmd.Bool("include-timestamps"),
Fs: mfa.Fs, Fs: mfa.Fs,
// Neither a manifest being replaced nor a temp file left by an // Neither a manifest being replaced nor a temp file left by an
// interrupted run belongs in the new manifest. // interrupted run belongs in the new manifest.
@@ -101,14 +102,14 @@ func (mfa *CLIApp) buildScannerOptions(ctx *cli.Context) *mfer.ScannerOptions {
} }
// Set seed for deterministic UUID if provided // Set seed for deterministic UUID if provided
if seed := ctx.String("seed"); seed != "" { if seed := cmd.String("seed"); seed != "" {
opts.Seed = seed opts.Seed = seed
log.Infof("using deterministic seed for manifest UUID") log.Infof("using deterministic seed for manifest UUID")
} }
// Set up signing options if sign-key is provided // Set up signing options if sign-key is provided
if signKey := ctx.String("sign-key"); signKey != "" { if signKey := cmd.String("sign-key"); signKey != "" {
opts.SigningOptions = &mfer.SigningOptions{ opts.SigningOptions = &mfer.SigningOptions{
KeyID: mfer.GPGKeyID(signKey), KeyID: mfer.GPGKeyID(signKey),
} }
@@ -173,14 +174,14 @@ func (mfa *CLIApp) cleanupOnSignal(outFile afero.File, tmpPath string) chan os.S
// runEnumeratePhase enumerates all input paths with optional progress // runEnumeratePhase enumerates all input paths with optional progress
// reporting and logs the totals. // reporting and logs the totals.
func (mfa *CLIApp) runEnumeratePhase(ctx *cli.Context, s *mfer.Scanner) error { func (mfa *CLIApp) runEnumeratePhase(cmd *cli.Command, s *mfer.Scanner) error {
// Set up enumeration progress reporting // Set up enumeration progress reporting
var ( var (
enumProgress chan mfer.EnumerateStatus enumProgress chan mfer.EnumerateStatus
enumWg sync.WaitGroup enumWg sync.WaitGroup
) )
if ctx.Bool("progress") { if cmd.Bool("progress") {
enumProgress = make(chan mfer.EnumerateStatus, 1) enumProgress = make(chan mfer.EnumerateStatus, 1)
enumWg.Add(1) enumWg.Add(1)
@@ -188,7 +189,7 @@ func (mfa *CLIApp) runEnumeratePhase(ctx *cli.Context, s *mfer.Scanner) error {
go reportEnumProgress(enumProgress, &enumWg) go reportEnumProgress(enumProgress, &enumWg)
} }
err := mfa.enumerateInputs(s, ctx.Args(), enumProgress) err := mfa.enumerateInputs(s, cmd.Args(), enumProgress)
if err != nil { if err != nil {
return err return err
} }
@@ -201,22 +202,24 @@ func (mfa *CLIApp) runEnumeratePhase(ctx *cli.Context, s *mfer.Scanner) error {
return nil return nil
} }
func (mfa *CLIApp) generateManifestOperation(ctx *cli.Context) error { func (mfa *CLIApp) generateManifestOperation(
ctx context.Context, cmd *cli.Command,
) error {
log.Debug("generateManifestOperation()") log.Debug("generateManifestOperation()")
s := mfer.NewScannerWithOptions(mfa.buildScannerOptions(ctx)) s := mfer.NewScannerWithOptions(mfa.buildScannerOptions(cmd))
// Phase 1: Enumeration - collect paths and stat files // Phase 1: Enumeration - collect paths and stat files
err := mfa.runEnumeratePhase(ctx, s) err := mfa.runEnumeratePhase(cmd, s)
if err != nil { if err != nil {
return err return err
} }
showProgress := ctx.Bool("progress") showProgress := cmd.Bool("progress")
// Check if output file exists // Check if output file exists
outputPath := ctx.String("output") outputPath := cmd.String("output")
if exists, _ := afero.Exists(mfa.Fs, outputPath); exists && !ctx.Bool("force") { if exists, _ := afero.Exists(mfa.Fs, outputPath); exists && !cmd.Bool("force") {
return fmt.Errorf("output file %s %w", outputPath, errOutputExists) return fmt.Errorf("output file %s %w", outputPath, errOutputExists)
} }
@@ -259,7 +262,7 @@ func (mfa *CLIApp) generateManifestOperation(ctx *cli.Context) error {
go reportScanProgress(scanProgress, &scanWg) go reportScanProgress(scanProgress, &scanWg)
} }
err = s.ToManifest(ctx.Context, outFile, scanProgress) err = s.ToManifest(ctx, outFile, scanProgress)
scanWg.Wait() scanWg.Wait()
+8 -6
View File
@@ -1,33 +1,35 @@
package cli package cli
import ( import (
"context"
"fmt" "fmt"
"time" "time"
"github.com/urfave/cli/v2" "github.com/urfave/cli/v3"
"sneak.berlin/go/mfer/internal/log" "sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
func (mfa *CLIApp) listManifestOperation(ctx *cli.Context) error { func (mfa *CLIApp) listManifestOperation(ctx context.Context, cmd *cli.Command) error {
// Default to ErrorLevel for clean output // Default to ErrorLevel for clean output
log.SetLevel(log.ErrorLevel) log.SetLevel(log.ErrorLevel)
longFormat := ctx.Bool("long") longFormat := cmd.Bool("long")
print0 := ctx.Bool("print0") print0 := cmd.Bool("print0")
pathOrURL, err := mfa.resolveManifestArg(ctx) pathOrURL, err := mfa.resolveManifestArg(cmd)
if err != nil { if err != nil {
return fmt.Errorf("list: %w", err) return fmt.Errorf("list: %w", err)
} }
rc, err := mfa.openManifestReader(pathOrURL) rc, err := mfa.openManifestReader(ctx, pathOrURL)
if err != nil { if err != nil {
return fmt.Errorf("list: %w", err) return fmt.Errorf("list: %w", err)
} }
defer func() { _ = rc.Close() }() defer func() { _ = rc.Close() }()
//nolint:contextcheck // mfer loads a manifest without a context
manifest, err := mfer.NewManifestFromReader(rc) manifest, err := mfer.NewManifestFromReader(rc)
if err != nil { if err != nil {
return fmt.Errorf("list: failed to parse manifest: %w", err) return fmt.Errorf("list: failed to parse manifest: %w", err)
+8 -8
View File
@@ -9,7 +9,7 @@ import (
"strings" "strings"
"time" "time"
"github.com/urfave/cli/v2" "github.com/urfave/cli/v3"
) )
// manifestFetchTimeout bounds HTTP requests made to fetch a manifest. // manifestFetchTimeout bounds HTTP requests made to fetch a manifest.
@@ -30,13 +30,13 @@ func isHTTPURL(s string) bool {
// openManifestReader opens a manifest from a path or URL and returns a ReadCloser. // openManifestReader opens a manifest from a path or URL and returns a ReadCloser.
// The caller must close the returned reader. // The caller must close the returned reader.
func (mfa *CLIApp) openManifestReader(pathOrURL string) (io.ReadCloser, error) { func (mfa *CLIApp) openManifestReader(
ctx context.Context, pathOrURL string,
) (io.ReadCloser, error) {
if isHTTPURL(pathOrURL) { if isHTTPURL(pathOrURL) {
client := &http.Client{Timeout: manifestFetchTimeout} client := &http.Client{Timeout: manifestFetchTimeout}
req, err := http.NewRequestWithContext( req, err := http.NewRequestWithContext(ctx, http.MethodGet, pathOrURL, nil)
context.Background(), http.MethodGet, pathOrURL, nil,
)
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to fetch %s: %w", pathOrURL, err) return nil, fmt.Errorf("failed to fetch %s: %w", pathOrURL, err)
} }
@@ -67,9 +67,9 @@ func (mfa *CLIApp) openManifestReader(pathOrURL string) (io.ReadCloser, error) {
// resolveManifestArg resolves the manifest path from CLI arguments. // resolveManifestArg resolves the manifest path from CLI arguments.
// HTTP(S) URLs are returned as-is. Directories are searched for index.mf. // HTTP(S) URLs are returned as-is. Directories are searched for index.mf.
// If no argument is given, the current directory is searched. // If no argument is given, the current directory is searched.
func (mfa *CLIApp) resolveManifestArg(ctx *cli.Context) (string, error) { func (mfa *CLIApp) resolveManifestArg(cmd *cli.Command) (string, error) {
if ctx.Args().Len() > 0 { if cmd.Args().Len() > 0 {
arg := ctx.Args().Get(0) arg := cmd.Args().Get(0)
if isHTTPURL(arg) { if isHTTPURL(arg) {
return arg, nil return arg, nil
} }
+66 -47
View File
@@ -1,6 +1,7 @@
package cli package cli
import ( import (
"context"
"errors" "errors"
"fmt" "fmt"
"io" "io"
@@ -8,7 +9,7 @@ import (
"time" "time"
"github.com/spf13/afero" "github.com/spf13/afero"
"github.com/urfave/cli/v2" "github.com/urfave/cli/v3"
"sneak.berlin/go/mfer/internal/log" "sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer" "sneak.berlin/go/mfer/mfer"
) )
@@ -54,7 +55,7 @@ type CLIApp struct {
gitrev string gitrev string
startupTime time.Time startupTime time.Time
exitCode int exitCode int
app *cli.App app *cli.Command
Stdin io.Reader // Standard input stream Stdin io.Reader // Standard input stream
Stdout io.Writer // Standard output stream for normal output Stdout io.Writer // Standard output stream for normal output
@@ -105,15 +106,15 @@ func (mfa *CLIApp) printBanner() {
// urfave/cli reads a flag from the nearest command that defines it, so each // urfave/cli reads a flag from the nearest command that defines it, so each
// command in the lineage is asked. The highest -v count wins rather than the // command in the lineage is asked. The highest -v count wins rather than the
// sum, because a subcommand without its own copies reads the root's. // sum, because a subcommand without its own copies reads the root's.
func (mfa *CLIApp) setVerbosity(c *cli.Context) { func (mfa *CLIApp) setVerbosity(cmd *cli.Command) {
_, present := os.LookupEnv("MFER_DEBUG") _, present := os.LookupEnv("MFER_DEBUG")
verbosity := 0 verbosity := 0
quiet := false quiet := false
for _, ctx := range c.Lineage() { for _, c := range cmd.Lineage() {
verbosity = max(verbosity, ctx.Count("verbose")) verbosity = max(verbosity, c.Count("verbose"))
quiet = quiet || ctx.Bool("quiet") quiet = quiet || c.Bool("quiet")
} }
switch { switch {
@@ -127,23 +128,34 @@ func (mfa *CLIApp) setVerbosity(c *cli.Context) {
} }
// commonFlags returns the -v and -q flags taken by the root and by the // commonFlags returns the -v and -q flags taken by the root and by the
// generate, check, freshen and fetch subcommands. // generate, check, freshen and fetch subcommands. They are local, so the
// root's copies are not inherited by the subcommands that do not take them.
func commonFlags() []cli.Flag { func commonFlags() []cli.Flag {
return []cli.Flag{ return []cli.Flag{
&cli.BoolFlag{ &cli.BoolFlag{
Name: "verbose", Name: "verbose",
Aliases: []string{"v"}, Aliases: []string{"v"},
Usage: "Increase verbosity (-v for verbose, -v -v for debug)", Usage: "Increase verbosity (-v for verbose, -v -v for debug)",
Count: new(int), Local: true,
}, },
&cli.BoolFlag{ &cli.BoolFlag{
Name: "quiet", Name: "quiet",
Aliases: []string{"q"}, Aliases: []string{"q"},
Usage: "Suppress output except errors", Usage: "Suppress output except errors",
Local: true,
}, },
} }
} }
// stopOnFirstArg returns the StopOnNthArg setting every command uses: flags
// are read only before the command's first argument, and everything after it
// is an argument, as with urfave/cli v2. So `mfer gen d -v` names a path "-v".
func stopOnFirstArg() *int {
n := 1
return &n
}
// requireSignatureFlag returns the --require-signature flag taken by the // requireSignatureFlag returns the --require-signature flag taken by the
// check and fetch subcommands. // check and fetch subcommands.
func requireSignatureFlag() *cli.StringFlag { func requireSignatureFlag() *cli.StringFlag {
@@ -151,7 +163,7 @@ func requireSignatureFlag() *cli.StringFlag {
Name: flagRequireSignature, Name: flagRequireSignature,
Aliases: []string{"S"}, Aliases: []string{"S"},
Usage: "Require manifest to be signed by the specified GPG key ID", Usage: "Require manifest to be signed by the specified GPG key ID",
EnvVars: []string{"MFER_REQUIRE_SIGNATURE"}, Sources: cli.EnvVars("MFER_REQUIRE_SIGNATURE"),
} }
} }
@@ -161,11 +173,12 @@ func (mfa *CLIApp) generateCommand() *cli.Command {
Aliases: []string{"gen"}, Aliases: []string{"gen"},
Usage: "Generate manifest file", Usage: "Generate manifest file",
ArgsUsage: "[path ...]", ArgsUsage: "[path ...]",
Action: func(c *cli.Context) error { StopOnNthArg: stopOnFirstArg(),
mfa.setVerbosity(c) Action: func(ctx context.Context, cmd *cli.Command) error {
mfa.setVerbosity(cmd)
mfa.printBanner() mfa.printBanner()
return mfa.generateManifestOperation(c) return mfa.generateManifestOperation(ctx, cmd)
}, },
Flags: append(commonFlags(), Flags: append(commonFlags(),
&cli.BoolFlag{ &cli.BoolFlag{
@@ -199,12 +212,12 @@ func (mfa *CLIApp) generateCommand() *cli.Command {
Name: "sign-key", Name: "sign-key",
Aliases: []string{"s"}, Aliases: []string{"s"},
Usage: "GPG key ID to sign the manifest with", Usage: "GPG key ID to sign the manifest with",
EnvVars: []string{"MFER_SIGN_KEY"}, Sources: cli.EnvVars("MFER_SIGN_KEY"),
}, },
&cli.StringFlag{ &cli.StringFlag{
Name: "seed", Name: "seed",
Usage: "Seed value for deterministic manifest UUID", Usage: "Seed value for deterministic manifest UUID",
EnvVars: []string{"MFER_SEED"}, Sources: cli.EnvVars("MFER_SEED"),
}, },
&cli.BoolFlag{ &cli.BoolFlag{
Name: "include-timestamps", Name: "include-timestamps",
@@ -220,11 +233,12 @@ func (mfa *CLIApp) checkCommand() *cli.Command {
Name: cmdCheck, Name: cmdCheck,
Usage: "Validate files using manifest file", Usage: "Validate files using manifest file",
ArgsUsage: manifestArgsUsage, ArgsUsage: manifestArgsUsage,
Action: func(c *cli.Context) error { StopOnNthArg: stopOnFirstArg(),
mfa.setVerbosity(c) Action: func(ctx context.Context, cmd *cli.Command) error {
mfa.setVerbosity(cmd)
mfa.printBanner() mfa.printBanner()
return mfa.checkManifestOperation(c) return mfa.checkManifestOperation(ctx, cmd)
}, },
Flags: append(commonFlags(), Flags: append(commonFlags(),
&cli.StringFlag{ &cli.StringFlag{
@@ -252,11 +266,12 @@ func (mfa *CLIApp) freshenCommand() *cli.Command {
Name: cmdFreshen, Name: cmdFreshen,
Usage: "Update manifest with changed, new, and removed files", Usage: "Update manifest with changed, new, and removed files",
ArgsUsage: manifestArgsUsage, ArgsUsage: manifestArgsUsage,
Action: func(c *cli.Context) error { StopOnNthArg: stopOnFirstArg(),
mfa.setVerbosity(c) Action: func(ctx context.Context, cmd *cli.Command) error {
mfa.setVerbosity(cmd)
mfa.printBanner() mfa.printBanner()
return mfa.freshenManifestOperation(c) return mfa.freshenManifestOperation(ctx, cmd)
}, },
Flags: append(commonFlags(), Flags: append(commonFlags(),
&cli.StringFlag{ &cli.StringFlag{
@@ -285,7 +300,7 @@ func (mfa *CLIApp) freshenCommand() *cli.Command {
Name: "sign-key", Name: "sign-key",
Aliases: []string{"s"}, Aliases: []string{"s"},
Usage: "GPG key ID to sign the manifest with", Usage: "GPG key ID to sign the manifest with",
EnvVars: []string{"MFER_SIGN_KEY"}, Sources: cli.EnvVars("MFER_SIGN_KEY"),
}, },
&cli.BoolFlag{ &cli.BoolFlag{
Name: "include-timestamps", Name: "include-timestamps",
@@ -301,10 +316,11 @@ func (mfa *CLIApp) exportCommand() *cli.Command {
Name: cmdExport, Name: cmdExport,
Usage: "Export manifest contents as JSON", Usage: "Export manifest contents as JSON",
ArgsUsage: "[manifest file or URL]", ArgsUsage: "[manifest file or URL]",
Action: func(c *cli.Context) error { StopOnNthArg: stopOnFirstArg(),
mfa.setVerbosity(c) Action: func(ctx context.Context, cmd *cli.Command) error {
mfa.setVerbosity(cmd)
return mfa.exportManifestOperation(c) return mfa.exportManifestOperation(ctx, cmd)
}, },
} }
} }
@@ -313,7 +329,8 @@ func (mfa *CLIApp) versionCommand() *cli.Command {
return &cli.Command{ return &cli.Command{
Name: cmdVersion, Name: cmdVersion,
Usage: "Show version", Usage: "Show version",
Action: func(_ *cli.Context) error { StopOnNthArg: stopOnFirstArg(),
Action: func(context.Context, *cli.Command) error {
mfa.printVersion() mfa.printVersion()
return nil return nil
@@ -327,9 +344,8 @@ func (mfa *CLIApp) listCommand() *cli.Command {
Aliases: []string{"ls"}, Aliases: []string{"ls"},
Usage: "List files in manifest", Usage: "List files in manifest",
ArgsUsage: manifestArgsUsage, ArgsUsage: manifestArgsUsage,
Action: func(c *cli.Context) error { StopOnNthArg: stopOnFirstArg(),
return mfa.listManifestOperation(c) Action: mfa.listManifestOperation,
},
Flags: []cli.Flag{ Flags: []cli.Flag{
&cli.BoolFlag{ &cli.BoolFlag{
Name: "long", Name: "long",
@@ -349,11 +365,12 @@ func (mfa *CLIApp) fetchCommand() *cli.Command {
Name: cmdFetch, Name: cmdFetch,
Usage: "fetch manifest and referenced files", Usage: "fetch manifest and referenced files",
ArgsUsage: "URL", ArgsUsage: "URL",
Action: func(c *cli.Context) error { StopOnNthArg: stopOnFirstArg(),
mfa.setVerbosity(c) Action: func(ctx context.Context, cmd *cli.Command) error {
mfa.setVerbosity(cmd)
mfa.printBanner() mfa.printBanner()
return mfa.fetchManifestOperation(c) return mfa.fetchManifestOperation(ctx, cmd)
}, },
Flags: append(commonFlags(), Flags: append(commonFlags(),
&cli.DurationFlag{ &cli.DurationFlag{
@@ -385,38 +402,40 @@ func (mfa *CLIApp) run(args []string) {
log.SetOutput(mfa.Stdout, mfa.Stderr) log.SetOutput(mfa.Stdout, mfa.Stderr)
log.Init() log.Init()
// -v means verbose, not version. urfave/cli's built-in version flag // -v means verbose, not version, at the root as on the generate, check,
// claims -v by default, which made "mfer -v --version" fail to parse and // freshen and fetch subcommands: verbose is the more common meaning of -v
// gave -v a different meaning at the root than on the generate, check, // in tools that offer both. urfave/cli's built-in version flag claims -v
// freshen and fetch subcommands, where it means verbose. Verbose is the // by default, so the version flag takes the capital -V instead.
// more common meaning of -v in tools that offer both, so -v means verbose
// at the root too and the version flag takes the capital -V.
// VersionFlag and VersionPrinter are urfave/cli package globals; run() is // VersionFlag and VersionPrinter are urfave/cli package globals; run() is
// serialized in tests, so assigning them here is safe. // serialized in tests, so assigning them here is safe.
cli.VersionFlag = &cli.BoolFlag{ cli.VersionFlag = &cli.BoolFlag{
Name: cmdVersion, Name: cmdVersion,
Aliases: []string{"V"}, Aliases: []string{"V"},
Usage: "print the version", Usage: "print the version",
Local: true,
} }
cli.VersionPrinter = func(_ *cli.Context) { mfa.printVersion() } cli.VersionPrinter = func(*cli.Command) { mfa.printVersion() }
mfa.app = &cli.App{ mfa.app = &cli.Command{
Name: mfa.appname, Name: mfa.appname,
Usage: "Manifest generator", Usage: "Manifest generator",
Version: mfa.VersionString(), Version: mfa.VersionString(),
EnableBashCompletion: true, EnableShellCompletion: true,
Writer: mfa.Stdout, Writer: mfa.Stdout,
ErrWriter: mfa.Stderr, // v3 writes its "Incorrect Usage" line to ErrWriter; v2 wrote it to
// stdout, before the help, so it stays on stdout.
ErrWriter: mfa.Stdout,
Flags: commonFlags(), Flags: commonFlags(),
Action: func(c *cli.Context) error { StopOnNthArg: stopOnFirstArg(),
if c.Args().Len() > 0 { Action: func(_ context.Context, cmd *cli.Command) error {
return fmt.Errorf("%w %q", errUnknownCommand, c.Args().First()) if cmd.Args().Len() > 0 {
return fmt.Errorf("%w %q", errUnknownCommand, cmd.Args().First())
} }
mfa.setVerbosity(c) mfa.setVerbosity(cmd)
mfa.printBanner() mfa.printBanner()
return cli.ShowAppHelp(c) return cli.ShowRootCommandHelp(cmd)
}, },
Commands: []*cli.Command{ Commands: []*cli.Command{
mfa.generateCommand(), mfa.generateCommand(),
@@ -431,7 +450,7 @@ func (mfa *CLIApp) run(args []string) {
mfa.app.HideVersion = false mfa.app.HideVersion = false
err := mfa.app.Run(args) err := mfa.app.Run(context.Background(), args)
if err != nil { if err != nil {
mfa.exitCode = 1 mfa.exitCode = 1
-3
View File
@@ -1,3 +0,0 @@
package mfer
//go:generate protoc ./mf.proto --go_out=paths=source_relative:.
-1
View File
@@ -1,6 +1,5 @@
{ {
"name": "mfer", "name": "mfer",
"version": "0.1.0",
"private": true, "private": true,
"description": "Development tooling for the mfer repository: prettier, used by script/fmt and script/fmt-check to format and verify Markdown and JSON.", "description": "Development tooling for the mfer repository: prettier, used by script/fmt and script/fmt-check to format and verify Markdown and JSON.",
"license": "WTFPL", "license": "WTFPL",
+59 -5
View File
@@ -13,11 +13,15 @@ set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Pinned versions, 2026-07-06. Never "latest" or "lts"; exact versions. # Pinned versions, 2026-07-06. Never "latest" or "lts"; exact versions.
NODE_VERSION="22.17.0" # The node version is in .nvmrc, where script/prettier reads it too.
NODE_VERSION="$(cat "$ROOT/.nvmrc")"
NVM_VERSION="0.40.3" NVM_VERSION="0.40.3"
# sha256 of https://github.com/nvm-sh/nvm/archive/refs/tags/v0.40.3.tar.gz # sha256 of https://github.com/nvm-sh/nvm/archive/refs/tags/v0.40.3.tar.gz
NVM_SHA256="5f4d6aaa04a177dc93c985e31dbc411ab6b8c6e1e21d8015dbc1372625fcd1d0" NVM_SHA256="5f4d6aaa04a177dc93c985e31dbc411ab6b8c6e1e21d8015dbc1372625fcd1d0"
YARN_VERSION="1.22.22" YARN_VERSION="1.22.22"
# protoc v33.4, 2026-10-04, for script/generate. The sha256 of each
# platform's release archive is in ensure_protoc.
PROTOC_VERSION="33.4"
PKGMGR="" PKGMGR=""
SUDO="" SUDO=""
@@ -74,9 +78,11 @@ verify_sha256() {
fi fi
} }
# nvm is a bash script; run a command in a bash with nvm loaded # nvm is a bash script; run a command in a bash with nvm loaded.
# --no-use: otherwise loading nvm here switches to the version .nvmrc
# names, and fails silently while that version is not installed yet.
nvm_sh() { nvm_sh() {
bash -c ". \"\$HOME/.nvm/nvm.sh\" && $*" bash -c ". \"\$HOME/.nvm/nvm.sh\" --no-use && $*"
} }
ensure_nvm() { ensure_nvm() {
@@ -122,6 +128,48 @@ install_js_deps() {
fi fi
} }
# Unpack protoc's release archive for this platform into bin/protoc, after
# checking the archive's sha256, unless bin/protoc already holds the pinned
# version.
ensure_protoc() {
dir="$ROOT/bin/protoc"
if [ "$("$dir/bin/protoc" --version 2>/dev/null)" = \
"libprotoc $PROTOC_VERSION" ]; then
return 0
fi
case "$(uname -s) $(uname -m)" in
"Linux x86_64")
platform="linux-x86_64"
sha256="c0040ea9aef08fdeb2c74ca609b18d5fdbfc44ea0042fcfbfb38860d35f7dd66"
;;
"Linux aarch64" | "Linux arm64")
platform="linux-aarch_64"
sha256="15aa988f4a6090636525ec236a8e4b3aab41eef402751bd5bb2df6afd9b7b5a5"
;;
"Darwin x86_64")
platform="osx-x86_64"
sha256="a49bec10d039e902d3b43e49938c42526f90011467609864fa6386ac4014da58"
;;
"Darwin arm64")
platform="osx-aarch_64"
sha256="726297dcfed58592fd35620a5a6246ae020c39e88f3fd4cb1827df7bcf3dfcf1"
;;
*)
echo "bootstrap: no protoc archive pinned for $(uname -s) $(uname -m)" >&2
exit 1
;;
esac
if missing curl; then pkg_install curl curl curl curl; fi
if missing unzip; then pkg_install unzip unzip unzip unzip; fi
tmp="$(mktemp -d)"
curl -fsSL -o "$tmp/protoc.zip" \
"https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOC_VERSION}/protoc-${PROTOC_VERSION}-${platform}.zip"
verify_sha256 "$tmp/protoc.zip" "$sha256"
rm -rf "$dir"
unzip -q "$tmp/protoc.zip" -d "$dir"
rm -rf "$tmp"
}
main() { main() {
cd "$ROOT" cd "$ROOT"
@@ -132,8 +180,10 @@ main() {
# ---- JS / docs repos ---- # ---- JS / docs repos ----
# This is a Go repo, but node and yarn are required anyway: prettier # This is a Go repo, but node and yarn are required anyway: prettier
# formats the Markdown and JSON, and script/fmt-check verifies it. # formats the Markdown and JSON, and script/fmt-check verifies it.
# The version is pinned by package.json/yarn.lock, whose integrity # The version is pinned by package.json/yarn.lock: yarn checks every
# hashes --frozen-lockfile enforces. # package it fetches against its yarn.lock integrity hash, and
# --frozen-lockfile fails instead of rewriting a yarn.lock that no
# longer matches package.json.
ensure_node ensure_node
ensure_yarn ensure_yarn
install_js_deps install_js_deps
@@ -142,6 +192,10 @@ main() {
if missing go; then pkg_install go golang go go; fi if missing go; then pkg_install go golang go go; fi
# No golangci-lint: script/lint runs it in Docker only. # No golangci-lint: script/lint runs it in Docker only.
go mod download go mod download
# gofumpt and protoc-gen-go: bin/tools/go.mod pins them, and
# script/gofumpt and script/generate build them from there.
(cd "$ROOT/bin/tools" && go mod download)
ensure_protoc
# ---- Python repos ---- # ---- Python repos ----
# if missing python3; then pkg_install python3 python3 python3 python3; fi # if missing python3; then pkg_install python3 python3 python3 python3; fi
+1 -3
View File
@@ -12,9 +12,7 @@ main() {
# empty constant. # empty constant.
version="$(git describe --tags --always --dirty 2>/dev/null || true)" version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
go build -tags urfave_cli_no_docs \ go build -ldflags "-X main.Gitrev=$version" -o bin/mfer ./cmd/mfer
-ldflags "-X main.Gitrev=$version" \
-o bin/mfer ./cmd/mfer
} }
main "$@" main "$@"
+20 -21
View File
@@ -4,26 +4,17 @@
# regenerates mf.pb.go: it is committed, so building and checking need no # regenerates mf.pb.go: it is committed, so building and checking need no
# protoc. A test fails while mf.proto no longer matches the recorded hash. # protoc. A test fails while mf.proto no longer matches the recorded hash.
# #
# Needs exactly the protoc and protoc-gen-go versions named in the header of # Runs the protoc that script/bootstrap unpacks into bin/protoc, and the
# the committed mf.pb.go (README.md says how to install them). Another # protoc-gen-go that bin/tools/go.mod pins. Another version of either
# version writes a different mf.pb.go, so the script refuses to run. # writes a different mf.pb.go.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# protoc 33.4 names itself v6.33.4 in the mf.pb.go header. # The protoc version script/bootstrap installs. protoc 33.4 names itself
# v6.33.4 in the mf.pb.go header.
PROTOC_VERSION="33.4" PROTOC_VERSION="33.4"
PROTOC_GEN_GO_VERSION="v1.36.11" PROTOC="$ROOT/bin/protoc/bin/protoc"
# require_version <command> <its exact --version output>
require_version() {
actual="$("$1" --version 2>/dev/null || true)"
if [ "$actual" != "$2" ]; then
echo "generate: needs $2 on PATH, found: ${actual:-none}" >&2
echo " README.md says how to install it." >&2
exit 1
fi
}
# sha256 <file>: print "<hash> <file>", with sha256sum, or with shasum # sha256 <file>: print "<hash> <file>", with sha256sum, or with shasum
# where there is no sha256sum. # where there is no sha256sum.
@@ -39,16 +30,24 @@ sha256() {
} }
main() { main() {
# A bin/protoc left from before the pin moved fails here, until
# script/bootstrap replaces it.
actual="$("$PROTOC" --version 2>/dev/null || true)"
if [ "$actual" != "libprotoc $PROTOC_VERSION" ]; then
echo "generate: needs protoc $PROTOC_VERSION in bin/protoc," \
"found: ${actual:-none}; run script/bootstrap" >&2
exit 1
fi
# `go tool -n` builds protoc-gen-go from bin/tools and prints where the
# binary is, without running it.
plugin="$(cd "$ROOT/bin/tools" && go tool -n protoc-gen-go)"
cd "$ROOT/mfer" cd "$ROOT/mfer"
# `go install` puts protoc-gen-go in $(go env GOPATH)/bin, which is
# often not on PATH.
PATH="$PATH:$(go env GOPATH)/bin"
require_version protoc "libprotoc $PROTOC_VERSION"
require_version protoc-gen-go "protoc-gen-go $PROTOC_GEN_GO_VERSION"
# Hashed before regenerating, so a missing hash tool stops the script # Hashed before regenerating, so a missing hash tool stops the script
# before it changes anything. Regenerating leaves mf.proto as it is. # before it changes anything. Regenerating leaves mf.proto as it is.
proto_hash="$(sha256 mf.proto)" proto_hash="$(sha256 mf.proto)"
go generate . "$PROTOC" --plugin=protoc-gen-go="$plugin" \
--go_out=paths=source_relative:. ./mf.proto
echo "$proto_hash" >mf.proto.sha256 echo "$proto_hash" >mf.proto.sha256
} }
+9 -9
View File
@@ -10,10 +10,10 @@ set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# gofumpt v0.12.0, 2026-10-04. `go run` fetches and builds exactly this # The gofumpt version is the one bin/tools/go.mod pins. `go tool` run in
# version, so neither a developer machine nor the lint image needs # bin/tools builds it from source checked against the hashes in
# gofumpt installed. # bin/tools/go.sum, so neither a developer machine nor the lint image needs
GOFUMPT="mvdan.cc/gofumpt@v0.12.0" # it installed.
usage() { usage() {
echo "usage: script/gofumpt --write|--check" >&2 echo "usage: script/gofumpt --write|--check" >&2
@@ -22,15 +22,15 @@ usage() {
main() { main() {
[ "$#" -eq 1 ] || usage [ "$#" -eq 1 ] || usage
cd "$ROOT" cd "$ROOT/bin/tools"
# Every Go file in the repo. gofumpt holds generated files, such as # Every Go file in the repo, from $ROOT down. gofumpt holds generated
# mfer/mf.pb.go, to gofmt's rules only. # files, such as mfer/mf.pb.go, to gofmt's rules only.
case "$1" in case "$1" in
--write) go run "$GOFUMPT" -l -w . ;; --write) go tool gofumpt -l -w "$ROOT" ;;
--check) --check)
# Own line: a failing command inside `[ -n "$(...)" ]` does # Own line: a failing command inside `[ -n "$(...)" ]` does
# not trip `set -e`, so a gofumpt that never ran would pass. # not trip `set -e`, so a gofumpt that never ran would pass.
unformatted="$(go run "$GOFUMPT" -l .)" unformatted="$(go tool gofumpt -l "$ROOT")"
if [ -n "$unformatted" ]; then if [ -n "$unformatted" ]; then
echo "gofumpt: files need formatting (run make fmt):" >&2 echo "gofumpt: files need formatting (run make fmt):" >&2
echo "$unformatted" >&2 echo "$unformatted" >&2
+24 -24
View File
@@ -18,24 +18,9 @@ usage() {
exit 2 exit 2
} }
# Prefer the version pinned by package.json/yarn.lock so that CI and # Only the prettier yarn installed from yarn.lock, never one on PATH: a
# developer machines format identically. Fall back to a prettier on PATH, # different version formats differently.
# but say so, because a different version formats differently. PRETTIER="$ROOT/node_modules/.bin/prettier"
find_prettier() {
if [ -x "$ROOT/node_modules/.bin/prettier" ]; then
printf '%s\n' "$ROOT/node_modules/.bin/prettier"
return 0
fi
if command -v prettier >/dev/null 2>&1; then
echo "prettier: node_modules/.bin/prettier is absent; using the" \
"prettier on PATH, which may be a different version than the" \
"one pinned in package.json. Run script/bootstrap to install" \
"the pinned version." >&2
command -v prettier
return 0
fi
return 1
}
main() { main() {
[ "$#" -eq 1 ] || usage [ "$#" -eq 1 ] || usage
@@ -46,10 +31,26 @@ main() {
cd "$ROOT" cd "$ROOT"
if ! prettier_bin="$(find_prettier)"; then # Where there is no node on PATH, script/bootstrap installs the version
echo "prettier: not found." >&2 # .nvmrc names through nvm, which keeps it in this directory.
echo " Install it with: script/bootstrap" >&2 if ! command -v node >/dev/null 2>&1; then
echo " (installs the version pinned in package.json/yarn.lock)" >&2 PATH="$HOME/.nvm/versions/node/v$(cat .nvmrc)/bin:$PATH"
fi
if ! command -v node >/dev/null 2>&1; then
echo "prettier: node is missing; run script/bootstrap" >&2
exit 1
fi
# node_modules keeps the old prettier after package.json moves to a new
# one, until script/bootstrap runs again, so compare the two.
if ! installed="$("$PRETTIER" --version 2>/dev/null)"; then
echo "prettier: not installed; run script/bootstrap" >&2
exit 1
fi
pinned="$(node -p 'require("./package.json").devDependencies.prettier')"
if [ "$installed" != "$pinned" ]; then
echo "prettier: package.json pins $pinned but $installed is" \
"installed; run script/bootstrap" >&2
exit 1 exit 1
fi fi
@@ -62,8 +63,7 @@ main() {
# patterns always match at least one tracked file (README.md, # patterns always match at least one tracked file (README.md,
# package.json), so an empty match means the glob broke, and prettier # package.json), so an empty match means the glob broke, and prettier
# erroring out is exactly what we want rather than a vacuous pass. # erroring out is exactly what we want rather than a vacuous pass.
"$prettier_bin" "$mode" "**/*.md" "$PRETTIER" "$mode" "**/*.md" "**/*.json"
"$prettier_bin" "$mode" "**/*.json"
} }
main "$@" main "$@"