The final stage is scratch, which has no CA certificates, so fetch from
an HTTPS URL failed to verify any server. Copy the CA bundle from the
pinned builder image into the final stage.
Model: opus-5-5
The two status-send tests now call the send directly on a channel nobody
receives from, so a blocking send hangs the test into its timeout instead
of racing a 100 ms timer that a loaded host can miss.
The gpg test for a child holding gpg's output had the same problem: its
100 ms deadline could fire before the fake gpg wrote the PID of sleep,
and the cleanup then failed. The fake gpg now writes that PID to a named
pipe, and the test cancels only after reading it. It then waits up to
10 s for the call, so a call that waits for sleep fails the test and the
cleanup still kills sleep.
Model: opus-5-5
script/lint now builds only the lint stage of the main Dockerfile
(docker build --no-cache --target lint), whose build runs the linter, so
a successful build is a clean lint. It is uncached because a cached
build runs no linter, and a trap removes the image it tagged; the tag
carries the process ID so concurrent runs do not collide. The lint stage
calls golangci-lint directly, since make lint now needs Docker. Nothing
installs or runs golangci-lint on the host any more: bootstrap and the
Makefile drop the install, and script/fmt drops golangci-lint run --fix.
Model: opus-5-5
The final stage is scratch, which has no C library, but the builder
compiled mfer with cgo on (the golang image's default). The binary
imports net (mfer's own HTTP code does, and so does google/uuid), so
with cgo on it came out dynamically linked and the image could not
start. The image's go build now sets CGO_ENABLED=0; nothing in mfer
needs cgo. A new builder step runs ldd on the binary and fails the
build unless it reports a static executable.
Model: opus-5-5
The Description first line now names the project, purpose, category,
WTFPL license, and author. A new Getting Started section gives a
copy-pasteable build-from-source block and gen/check/fetch usage. Problem
Statement and Proposed Solution move under a new Rationale heading, the
prose kept. A new Design section documents the package layout: the mfer/
library with its committed protobuf code, the internal/cli commands,
internal/log and internal/bork, and the cmd/mfer entrypoint. Authors is
renamed Author with the canonical link.
Getting Started uses go build because the make target that builds the
binary runs protoc, which script/bootstrap does not install.
Model: opus-4-8 (implementation); opus-5-5 (rebase, rework)
urfave/cli's built-in version flag claimed -v, so "mfer -v --version"
failed to parse, and -v meant version at the root but verbose on
generate, check, freshen and fetch. Verbose is the more common meaning,
so the root now takes -v and -q too, and the version flag takes -V. A
-v or -q before generate, check, freshen, fetch or export applies to
that subcommand; list keeps its fixed quiet logging.
User-visible changes: -v no longer prints the version; use -V or
--version. "mfer version" prints "mfer version 0.1.0 (...)", the same
line as --version, instead of the bare "0.1.0 (...)".
Tests: runCLI now points the logger at io.Discard after each run, so
other tests' log lines cannot land in a finished run's output.
Model: opus-4-8 (implementation); opus-5-5 (rework)
FuzzNewManifestFromReader fails when the parser returns both or neither
of a manifest and an error, or allocates more than a fixed multiple of
its input and the decompressed data it may read, plus room for the
decoder's window buffers. make test runs the seed corpus; make fuzz
fuzzes for one minute.
Parser bug: MaxDecompressedSize did not bound decompression; the zstd
decoder decoded a payload under 128 KiB in full before the LimitReader
read any of it. It now decodes only what the LimitReader reads and
refuses windows over the 8 MiB mfer writes with. Seeds: a frame claiming
8 GiB, two frames together over the limit, empty frames with growing
windows.
Model: opus-5-5
Every gpg run now has a one-minute deadline (gpgTimeout) on top of its
caller's context and is killed when either ends. A timeout is reported
as "gpg timed out" under the failing operation instead of "signal:
killed". Only gpg itself is killed; WaitDelay (one second) stops the run
from waiting on a process gpg left behind that still holds its output,
such as a wrapper script that does not exec the real gpg.
Builder.Build and Checker.ExtractEmbeddedSigningKeyFP take a context, so
ToManifest's context now reaches signing and the contextcheck
suppression calling signing non-cancellable is gone. Manifest loading
takes no context, so its signature check is bounded by the timeout
alone.
Model: opus-5-5
TODO.md is deleted and the README TODO section now only points to the
issue tracker, where open work and open design questions live from now
on. AGENTS.md says so too, and says this overrides the shared policy's
README todo list for this repo. The README Open Questions section
becomes Original Design Questions, each noting where it now stands.
Every open item in both lists was already done or already owned by an
issue, apart from one, now #121,
and the chunk checksum question, now on
#81.
Model: opus-5-5
errmsg_test.go now calls the function that emits each user-visible CLI
error message and asserts on the full text it returns, instead of
rendering format strings copied from production, so rewording any pinned
message fails the suite. The unknown-command message is driven through
the root command's action.
The signer-mismatch case signs a manifest with a throwaway gpg key and
is skipped where gpg is absent, like the repo's other signing tests.
The freshen mtime-presence test gives the scanned file an epoch mtime,
so it fails if recordEntry reads an absent manifest mtime as the epoch.
Model: opus-4-8 (first implementation); opus-5-5 (completion, this message)
A bare `docker build .` served the Dockerfile's check steps from the
layer cache on an unchanged tree and exited 0 without running them.
script/cibuild now computes the version on its own line and runs the
same build command as script/docker and the shared policy, --no-cache
included, so every CI build runs the checks. README.md describes
script/cibuild accordingly.
Model: opus-5-5
downloadFile opened the temp file with os.Create, which opens and
empties a file already at that name. If that file was a hard link to a
file outside the destination directory, fetch overwrote the outside
file. It now removes whatever is at the temp name, which removes only
that name, and creates the temp file with O_EXCL, so the create fails if
anything is still or again there. A leftover temp file from an
interrupted run is still replaced. The new test puts a hard link at the
temp name and checks that fetch succeeds and the outside file is
unchanged. The command name is now the constant cmdFetch, like the other
command names, because lint requires it once a third test uses it.
Model: opus-5-5
sanitizePath checks manifest paths only as text, so a symlink already
inside the destination directory could send fetch's writes outside it.
checkNoSymlinks now looks at each existing part of a path with os.Lstat
and refuses the path if any part is a symlink, wherever it points.
fetch runs it immediately before each write: creating the parent
directories, creating the temp file, and renaming it into place. The new
test puts such a symlink at each of those three places, and once inside
a plain directory, and checks that the fetch fails and nothing outside
changes. The G304 comment now states what holds. A symlink swapped in
between a check and its write is not caught; os.Root closes that once
the Go version is raised.
Model: opus-5-5
.dockerignore now sends .git but not .git/config, which can hold a
credential and which git describe does not need. The build stage stamps
main.Gitrev from the VERSION build argument when one is given, otherwise
from git describe --tags --always, and fails if .git is present and no
version comes out. git there trusts /src whoever owns it, since a
context sent as a tar archive keeps its files' owners. script/docker is
replaced by the canonical copy, which passes VERSION; bin/gitrev.sh uses
--tags too, so every entrypoint stamps the same value for a clean
commit.
Model: opus-5-5
Untrusted .mf files were parsed with no path validation, so an entry like ../../etc/passwd reached filepath.Join against the checker's base path and mfer check could stat and read outside it. ValidatePath ran only when building a manifest. It now runs on every entry as the manifest loads, so every consumer is covered. The whole manifest is rejected on the first bad entry instead of dropping it, which could hide files from a check; the error wraps errInvalidManifestPath and names the path.
Disclosure: a path that is not valid UTF-8 is refused earlier, by the protobuf string decoder, so that error does not name the path.
Model: opus-4-8 (implementation); fable-5-1 (summary)
script/test now runs go test -timeout 30s -race -cover ./..., quiet on success and rerunning verbose on failure. -race exposed a data race on the process-global apex/log logger: Init reconfigured it on every CLI run while other goroutines logged through it. internal/log now mutates the global under the write lock and reads it under the read lock; WithError is dropped because its Entry logged outside that lock, and run() reports a failed command's error via log.Errorf instead (still shown under -q). The corruption fuzz test is scaled to 1500 files / 100 iterations to fit the budget under -race; it pins the same claims at reduced breadth. Independent review ran the Docker lint gate uncached.
Model: opus-4-8 (implementation, review)
model: claude-fable-5
Add the canonical .editorconfig from sneak/prompts verbatim. Broaden .gitignore to cover secrets (.env, .env.*, *.key, *.pem), OS files, editor files, and Go artifacts while keeping every repo-specific entry; no tracked file becomes ignored, and bin/gitrev.sh stays tracked despite the /bin/ rule. Remove the stale .drone.yml ignore entry and the DRONE_COMMIT_SHA branch from bin/gitrev.sh, left from the Gitea Actions migration; the GITREV and git-describe paths still work.
A reader may trip over .editorconfig saying four spaces for every file while Go files are tab-indented: gofmt governs Go, editorconfig does not, and the file is taken verbatim by policy.
Disclosure: the worker's cold docker build hit the 10s test timeout in the gpg signature test; this change has no Go code, and that timeout is tracked by issues 67 and 62.
Model: opus-4-8 (implementation); fable-5-1 (merge)
Adds .prettierrc and .prettierignore, a single script/prettier entrypoint shared by fmt and fmt-check so the two cannot drift, and prettier 3.9.6 pinned by yarn.lock integrity hash.
Markdown formatting is now gated in the authoritative Docker build via a new mdfmt stage, since the golangci-lint image has no node. REPO_POLICIES.md is ignored so local tooling cannot drift it from upstream.
Removes the || true that made the previous prettier invocation unable to fail.
Adopts golangci-lint v2.12.2 and the canonical .golangci.yml (default: all), and fixes all resulting findings across the tree.
Two intended behavior changes: absent MFFilePath.Mtime is handled explicitly in freshen, list and export rather than dereferenced (main panicked); gpg positional key IDs now follow an explicit -- end-of-options marker.
All twelve reworded user-visible error messages restored to byte-identical parity with main and pinned by tests.
## Summary
Performs a design and status review of the codebase and adds a comprehensive TODO section to `README.md` listing remaining work for a 1.0 release.
### What changed
- **README.md**: Added a `TODO: Remaining Work for 1.0` section covering:
- **7 design questions** requiring @sneak's input before implementation (manifest type export, Go module path, GPG vs pure-Go crypto, format framing, etc.) — each with an answer field for inline decisions
- **Implementation tasks** organized by category: repo infrastructure, format & correctness, library, CLI, testing & robustness, documentation, and release checklist
- Updated build status section (removed stale Drone CI badge, replaced with description of current Docker-based CI)
- **TODO.md**: Removed — items integrated into README TODO section
- **AGENTS.md**: Updated reference from `TODO.md` to README TODO section
### Design review findings
**What works well:**
- Core library (Builder, Scanner, Checker) is solid with good test coverage
- Format specification is well-designed (protobuf + zstd, multihash, deterministic serialization)
- CLI covers all major operations (gen, check, list, export, freshen, fetch)
- Test suite is thorough — builder, scanner, checker, GPG, CLI integration, corruption detection
- afero abstraction enables clean testing without filesystem side effects
**Key gaps for 1.0:**
- Missing repo infrastructure (`.golangci.yml`, `.editorconfig`, CI workflow)
- `manifest` type is unexported — consumers can't use it in their own type declarations
- GPG signing shells out to `gpg` subprocess — fragile and may not be installed
- Go module path inconsistency between `go.mod` and proto `go_package`
- `fetch` command lacks retry logic and has no HTTP timeout
- Missing fuzz tests for untrusted input deserialization
- Freshen CLI command has incomplete integration test coverage
closes#47closes#50
Co-authored-by: user <user@Mac.lan guest wan>
Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #54
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
Closes #49
`.index.mf` is a generated manifest file produced at runtime by `mfer gen`. It was committed to the repo but should not be tracked in version control.
Changes:
- `git rm .index.mf` to remove it from tracking
- Added `.index.mf` to `.gitignore` to prevent accidental re-commits
Co-authored-by: user <user@Mac.lan guest wan>
Reviewed-on: #52
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
Closes#39
Splits the Dockerfile into a dedicated lint stage using the `golangci/golangci-lint` image directly, rather than copying the binary into the builder stage.
## Changes
### Dockerfile
- **Lint stage** (`AS lint`): Uses the pre-built `golangci/golangci-lint` image (pinned by sha256) to run `make fmt-check` and `make lint`. This is a self-contained stage with its own `go mod download` and source copy.
- **Builder stage** (`AS builder`): Runs only `make test` and the final binary build. No longer needs golangci-lint installed.
- **Stage dependency**: `COPY --from=lint /src/go.sum /dev/null` forces BuildKit to always execute the lint stage (without this, unused stages are silently skipped).
- Both stages touch `mfer/mf.pb.go` to prevent make from trying to regenerate via protoc.
With BuildKit, the lint and builder stages run in parallel after their shared `go mod download` layers complete, so lint/formatting failures surface much faster without blocking on test execution.
### Makefile
- Added `lint` to the `check` target prereqs: `check: test lint fmt-check` (was `check: test fmt-check`), matching the [REPO_POLICIES](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/REPO_POLICIES.md) requirement.
Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #45
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
Remove obsolete Drone CI config. Added to .gitignore.
.golangci.yaml was already removed from next branch.
Co-authored-by: user <user@Mac.lan guest wan>
Reviewed-on: #37
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
Remove unused `internal/scanner/` and `internal/checker/` packages. The CLI already uses `mfer.Scanner` and `mfer.Checker` from the `mfer/` package directly, so these were dead code.
Co-authored-by: clawbot <clawbot@openclaw>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #27
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
clawbot
