2 Commits
Author SHA1 Message Date
sneak 08139beab2 fetch refuses a manifest that lists another file's temp name (closes #151)
check / check (push) Failing after 2s
fetch downloads each file to a temp name beside it and first removes
whatever is there. A manifest listing both a.txt and .a.txt.tmp had
fetch delete the second while fetching the first, then exit 0 with a
tree check rejects.

The refusal of a manifest that lists the saved manifest's own name or
temp name now covers this too: a listed file, or a directory a listed
file is in, may not sit at any name fetch writes besides the listed
files themselves. Temp names come from tempPathFor, names are compared
ignoring case as before, and the refusal still happens before the
destination is created or any file requested.

Model: opus-5-5
2026-10-04 17:05:02 +00:00
clawbot d3394bd2a2 Makefile: thin shims only, add make build (closes #73)
check / check (push) Failing after 2s
The Makefile is now one shim per script/ entrypoint, in the order of
the canonical model Makefile, plus build, generate and fuzz. The new
script/build writes bin/mfer with the urfave_cli_no_docs tag and the
git describe revision that script/docker stamps. script/generate now
adds the Go bin directory to PATH itself, and the Docker lint stage
calls script/gofumpt --check directly. Removed: the tarball-caching
rules and their ignore entries, godoc, ci, fixme, clean, run, default,
fmt-check-go, fmt-check-md, and bin/gitrev.sh, which only the
Makefile called.

Model: opus-5-5
2026-10-04 18:38:13 +02:00
10 changed files with 76 additions and 123 deletions

No files matched your search

+2 -3
View File
@@ -59,6 +59,5 @@
**/.vscode **/.vscode
**/*.sublime-* **/*.sublime-*
# This repo's own host-built archives (Makefile). # This repo's own host-built binary (make build).
*.tmp /bin/mfer
*.dockerimage
-5
View File
@@ -1,11 +1,6 @@
/bin/mfer /bin/mfer
/tmp /tmp
/node_modules/ /node_modules/
*.tmp
*.dockerimage
/vendor
vendor.tzst
modcache.tzst
# Generated manifest files # Generated manifest files
/index.mf /index.mf
+1 -1
View File
@@ -11,7 +11,7 @@ COPY . .
# Go half of fmt-check only: this image has no node, so no prettier. The # Go half of fmt-check only: this image has no node, so no prettier. The
# markdown half runs in the mdfmt stage below. The image has no gofumpt # markdown half runs in the mdfmt stage below. The image has no gofumpt
# either; script/gofumpt builds the version it pins with `go run`. # either; script/gofumpt builds the version it pins with `go run`.
RUN make fmt-check-go RUN script/gofumpt --check
# The linter directly, not `make lint`: script/lint builds this stage, and # The linter directly, not `make lint`: script/lint builds this stage, and
# there is no docker inside this build. # there is no docker inside this build.
RUN golangci-lint run --config .golangci.yml ./... RUN golangci-lint run --config .golangci.yml ./...
+21 -76
View File
@@ -1,20 +1,8 @@
export DOCKER_BUILDKIT := 1 .PHONY: bootstrap setup test lint fmt fmt-check check docker hooks build generate fuzz
export PROGRESS_NO_TRUNC := 1
GOPATH := $(shell go env GOPATH)
export PATH := $(PATH):$(GOPATH)/bin
SOURCEFILES := mfer/*.go mfer/*.proto internal/*/*.go cmd/*/*.go go.mod go.sum
ARCH := $(shell uname -m)
GITREV_BUILD := $(shell bash $(PWD)/bin/gitrev.sh 2>/dev/null || echo unknown)
APPNAME := mfer
VERSION := 0.1.0
export DOCKER_IMAGE_CACHE_DIR := $(HOME)/Library/Caches/Docker/$(APPNAME)-$(ARCH)
GOLDFLAGS += -X main.Version=$(VERSION)
GOLDFLAGS += -X main.Gitrev=$(GITREV_BUILD)
GOFLAGS := -ldflags "$(GOLDFLAGS)"
.PHONY: bootstrap setup docker default run ci test fuzz check lint fmt fmt-check fmt-check-go fmt-check-md hooks fixme generate # Makefile targets are thin shims; the implementations live in script/
# per the scripts-to-rule-them-all pattern (see the Entrypoints section
default: fmt test # of README.md).
bootstrap: bootstrap:
@script/bootstrap @script/bootstrap
@@ -22,75 +10,32 @@ bootstrap:
setup: setup:
@script/setup @script/setup
run: ./bin/mfer
./$<
./$< gen
ci: test
test: test:
@script/test @script/test
fuzz:
@script/fuzz
fixme:
@grep -nir fixme . | grep -v Makefile
check:
@script/check
fmt-check:
@script/fmt-check
# Halves of fmt-check, for environments that have only one toolchain:
# the Docker lint stage has Go but no node, the markdown stage the reverse.
fmt-check-go:
@script/gofumpt --check
fmt-check-md:
@script/prettier --check
hooks:
@script/install-precommit
generate:
@script/generate
bin/mfer: $(SOURCEFILES)
cd cmd/mfer && go build -tags urfave_cli_no_docs -o ../../bin/mfer $(GOFLAGS) .
clean:
rm -rfv bin/mfer cmd/mfer/mfer *.dockerimage
fmt:
@script/fmt
lint: lint:
@script/lint @script/lint
fmt:
@script/fmt
fmt-check:
@script/fmt-check
check:
@script/check
docker: docker:
@script/docker @script/docker
sneak-mfer.$(ARCH).tzst.dockerimage: $(SOURCEFILES) vendor.tzst modcache.tzst hooks:
docker build --progress plain --build-arg GITREV=$(GITREV_BUILD) -t sneak/mfer . @script/install-precommit
docker save sneak/mfer | pv | zstdmt -19 > $@
du -sh $@
godoc: build:
open http://127.0.0.1:6060 @script/build
godoc -http=:6060
vendor.tzst: go.mod go.sum generate:
go mod tidy @script/generate
go mod vendor
cd vendor && tar -c . | pv | zstdmt -19 > $(PWD)/$@.tmp
rm -rf vendor
mv $@.tmp $@
modcache.tzst: go.mod go.sum fuzz:
go mod tidy @script/fuzz
cd $(HOME)/go/pkg && chmod -R u+rw . && rm -rf mod sumdb
go mod download -x
cd $(shell go env GOMODCACHE) && tar -c . | pv | zstdmt -19 > $(PWD)/$@.tmp
mv $@.tmp $@
+8 -5
View File
@@ -29,7 +29,7 @@ is committed, so no `protoc` toolchain is required:
```sh ```sh
git clone https://git.eeqj.de/sneak/mfer.git git clone https://git.eeqj.de/sneak/mfer.git
cd mfer cd mfer
go build -o bin/mfer ./cmd/mfer make build
``` ```
Generate a manifest for a directory tree, verify it later, and fetch a published Generate a manifest for a directory tree, verify it later, and fetch a published
@@ -76,6 +76,9 @@ provide:
such as `script/docker` such as `script/docker`
- `script/test` — run the test suite (`go test`); one test fails when - `script/test` — run the test suite (`go test`); one test fails when
`mfer/mf.proto` no longer matches the hash `script/generate` recorded `mfer/mf.proto` no longer matches the hash `script/generate` recorded
- `script/build` (`make build`) — build the `mfer` binary into `bin/mfer`,
stamped with the revision `mfer version` prints: the output of
`git describe --tags --always --dirty`, as `script/docker` passes it
- `script/generate` (`make generate`) — regenerate `mfer/mf.pb.go` from - `script/generate` (`make generate`) — regenerate `mfer/mf.pb.go` from
`mfer/mf.proto` and record the hash of that `mfer/mf.proto` in `mfer/mf.proto` and record the hash of that `mfer/mf.proto` in
`mfer/mf.proto.sha256`; the only thing that regenerates the committed `mfer/mf.proto.sha256`; the only thing that regenerates the committed
@@ -85,8 +88,8 @@ provide:
[its release](https://github.com/protocolbuffers/protobuf/releases/tag/v33.4) [its release](https://github.com/protocolbuffers/protobuf/releases/tag/v33.4)
and put its `bin/protoc` on `PATH`) and `protoc-gen-go` v1.36.11 and put its `bin/protoc` on `PATH`) and `protoc-gen-go` v1.36.11
(`go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11`, which (`go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11`, which
installs it in `$(go env GOPATH)/bin`; `make generate` adds that directory to installs it in `$(go env GOPATH)/bin`; `script/generate` adds that directory
`PATH`) to `PATH`)
- `script/fuzz` — fuzz the manifest parser for one minute; run by hand - `script/fuzz` — fuzz the manifest parser for one minute; run by hand
(`make fuzz`), never by CI, while `script/test` runs its committed seed corpus (`make fuzz`), never by CI, while `script/test` runs its committed seed corpus
as ordinary tests as ordinary tests
@@ -98,8 +101,8 @@ provide:
- `script/gofumpt` — run `gofumpt` over every Go file in the repository in the - `script/gofumpt` — run `gofumpt` over every Go file in the repository in the
given mode, `--write` or `--check`, at the one version it pins (built on given mode, `--write` or `--check`, at the one version it pins (built on
demand by `go run`, so nothing installs it); `script/fmt`, `script/fmt-check` demand by `go run`, so nothing installs it); `script/fmt`, `script/fmt-check`
and the Docker lint stage (`make fmt-check-go`) all go through it, so they and the Docker lint stage all go through it, so they cannot disagree about Go
cannot disagree about Go formatting formatting
- `script/prettier` — run prettier over the repository's canonical file set - `script/prettier` — run prettier over the repository's canonical file set
(Markdown and JSON, minus `.prettierignore`) in the given mode, `--write` or (Markdown and JSON, minus `.prettierignore`) in the given mode, `--write` or
`--check`; the single definition of that file set, so `script/fmt` and `--check`; the single definition of that file set, so `script/fmt` and
-12
View File
@@ -1,12 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
main() {
if [[ -n "${GITREV:-}" ]]; then
echo "$GITREV"
else
git describe --tags --always --dirty=-dirty
fi
}
main "$@"
+19 -19
View File
@@ -1166,12 +1166,12 @@ func TestFetchRequireSignature(t *testing.T) {
func TestFetchRefusesListedManifestName(t *testing.T) { func TestFetchRefusesListedManifestName(t *testing.T) {
t.Parallel() t.Parallel()
for listed, writtenThere := range map[string]string{ for _, listed := range []string{
defaultManifestName: "the saved manifest", defaultManifestName,
tempPathFor(defaultManifestName): "the saved manifest's temp file", tempPathFor(defaultManifestName),
defaultManifestName + "/" + testFileTxt: "the saved manifest", defaultManifestName + "/" + testFileTxt,
"INDEX.MF": "the saved manifest", "INDEX.MF",
"./" + defaultManifestName: "the saved manifest", "./" + defaultManifestName,
} { } {
t.Run(listed, func(t *testing.T) { t.Run(listed, func(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1179,8 +1179,7 @@ func TestFetchRefusesListedManifestName(t *testing.T) {
files := map[string][]byte{listed: []byte("listed")} files := map[string][]byte{listed: []byte("listed")}
assertFetchRefused(t, builtManifest(t, files), files, assertFetchRefused(t, builtManifest(t, files), files,
"manifest lists a file where fetch writes another file: "+ "manifest lists a file where fetch writes another file: "+listed)
listed+" ("+writtenThere+")")
}) })
} }
} }
@@ -1195,25 +1194,25 @@ func TestFetchRefusesListedTempName(t *testing.T) {
t.Parallel() t.Parallel()
for _, dir := range []string{"", "sub/"} { for _, dir := range []string{"", "sub/"} {
t.Run(dir+"a.txt and "+dir+".a.txt.tmp", func(t *testing.T) { file, tmp := dir+"a.txt", dir+".a.txt.tmp"
both := map[string][]byte{
file: []byte("a file"),
tmp: []byte("a file at its temp name"),
}
t.Run(file+" and "+tmp, func(t *testing.T) {
t.Parallel() t.Parallel()
files := map[string][]byte{ assertFetchRefused(t, builtManifest(t, both), both,
dir + "a.txt": []byte("a file"),
dir + ".a.txt.tmp": []byte("a file at its temp name"),
}
assertFetchRefused(t, builtManifest(t, files), files,
"manifest lists a file where fetch writes another file: "+ "manifest lists a file where fetch writes another file: "+
dir+".a.txt.tmp (the temp file for "+dir+"a.txt)") tmp+" (the temp file for "+file+")")
}) })
}
for _, listed := range []string{"a.txt", ".a.txt.tmp"} { for listed, content := range both {
t.Run("only "+listed, func(t *testing.T) { t.Run("only "+listed, func(t *testing.T) {
t.Parallel() t.Parallel()
files := map[string][]byte{listed: []byte("listed")} files := map[string][]byte{listed: content}
manifest := builtManifest(t, files) manifest := builtManifest(t, files)
server := httptest.NewServer(fetchTestHandler(manifest, files)) server := httptest.NewServer(fetchTestHandler(manifest, files))
@@ -1231,6 +1230,7 @@ func TestFetchRefusesListedTempName(t *testing.T) {
assert.Equal(t, want, filesUnder(t, dest)) assert.Equal(t, want, filesUnder(t, dest))
}) })
} }
}
} }
// builtManifest returns a manifest of files, built directly rather than // builtManifest returns a manifest of files, built directly rather than
Executable
+20
View File
@@ -0,0 +1,20 @@
#!/bin/sh
# script/build: build the mfer binary into bin/mfer, stamped with the
# revision `mfer version` prints, derived as script/docker derives it.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
# Own line: a failing command substitution inside an argument does
# not trip `set -e`, so the inline form degrades silently to an
# empty constant.
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown"
go build -tags urfave_cli_no_docs \
-ldflags "-X main.Gitrev=$version" \
-o bin/mfer ./cmd/mfer
}
main "$@"
+3
View File
@@ -40,6 +40,9 @@ sha256() {
main() { main() {
cd "$ROOT/mfer" cd "$ROOT/mfer"
# `go install` puts protoc-gen-go in $(go env GOPATH)/bin, which is
# often not on PATH.
PATH="$PATH:$(go env GOPATH)/bin"
require_version protoc "libprotoc $PROTOC_VERSION" require_version protoc "libprotoc $PROTOC_VERSION"
require_version protoc-gen-go "protoc-gen-go $PROTOC_GEN_GO_VERSION" require_version protoc-gen-go "protoc-gen-go $PROTOC_GEN_GO_VERSION"
# Hashed before regenerating, so a missing hash tool stops the script # Hashed before regenerating, so a missing hash tool stops the script
+2 -2
View File
@@ -4,8 +4,8 @@
# Takes exactly one mode argument, --write or --check, and runs the same # Takes exactly one mode argument, --write or --check, and runs the same
# gofumpt version over the same files in both modes. script/fmt and # gofumpt version over the same files in both modes. script/fmt and
# script/fmt-check both go through here, and so does the Docker lint # script/fmt-check both go through here, and so does the Docker lint
# stage (make fmt-check-go), so what gets formatted and what gets # stage, so what gets formatted and what gets verified cannot drift
# verified cannot drift apart. # apart.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"