Every gpg invocation went through runGPG, which built its command with
exec.CommandContext(context.Background(), ...). That is the right call
with the wrong context: context.Background() never expires, so no
deadline was ever enforced on any of the five gpg call sites.
runGPG now takes a context and derives a gpgTimeout deadline from it,
honouring an earlier caller deadline when there is one. The gpg-touching
library entry points take a ctx as their first argument so cancellation
propagates from above: Builder.Build, NewManifestFromReader,
NewManifestFromFile, NewChecker, Checker.ExtractEmbeddedSigningKeyFP.
Scanner.ToManifest already had a ctx and now passes it down, which
withdraws the //nolint:contextcheck claiming signing was "not
cancellable by design" -- it is, and now it is.
A deadline alone is not enough, and the added test proves it. gpg
delegates to helpers (gpg-agent, pinentry) that inherit the captured
stdout and stderr pipes. Go's default cancellation kills only the direct
child, so the helper keeps the pipes open and Cmd.Wait blocks on the
output-copying goroutines forever -- a dead process and a call that
still never returns. Two additions fix that: the child runs in its own
process group and cancellation kills the group, and Cmd.WaitDelay caps
how long Wait will hold on for the pipes if something escapes the group
anyway. Measured with the stand-in gpg from the new test: neither
mechanism, hangs until `go test` gives up; WaitDelay only, returns in
2.2s; both, returns in 0.20s.
Timeout errors now name the operation and how long gpg ran instead of
surfacing a bare "signal: killed" or "context deadline exceeded", and a
cancellation from above is reported as a cancellation rather than a
timeout, so an abort is distinguishable from a stall.
The test helper's own keygen invocations had the same unbounded
context.Background() and the same pipe-inheriting agent problem, which
makes them the actual mechanism behind the intermittent suite timeout
noted in the issue: keygen starts gpg-agent, and a stalled agent hung
the suite rather than failing it. They now run under a deadline with the
same hardening, so a broken gpg environment skips instead of hanging.
Verified with a cold `docker buildx build --no-cache`: prettier, gofmt,
`make lint` (0 issues) and `make test` all executed and passed. The
golang:1.23 image ships gpg, so the real signing, export, fingerprint,
import and verify tests run against real gpg there, not skipped.
The process-group kill is unix-only and lives in a build-tagged file; on
other platforms the deadline is still enforced via cancellation plus
WaitDelay, only the group kill of helpers is unavailable.
Adds .prettierrc and .prettierignore, a single script/prettier entrypoint shared by fmt and fmt-check so the two cannot drift, and prettier 3.9.6 pinned by yarn.lock integrity hash.
Markdown formatting is now gated in the authoritative Docker build via a new mdfmt stage, since the golangci-lint image has no node. REPO_POLICIES.md is ignored so local tooling cannot drift it from upstream.
Removes the || true that made the previous prettier invocation unable to fail.
Adopts golangci-lint v2.12.2 and the canonical .golangci.yml (default: all), and fixes all resulting findings across the tree.
Two intended behavior changes: absent MFFilePath.Mtime is handled explicitly in freshen, list and export rather than dereferenced (main panicked); gpg positional key IDs now follow an explicit -- end-of-options marker.
All twelve reworded user-visible error messages restored to byte-identical parity with main and pinned by tests.
## Summary
Performs a design and status review of the codebase and adds a comprehensive TODO section to `README.md` listing remaining work for a 1.0 release.
### What changed
- **README.md**: Added a `TODO: Remaining Work for 1.0` section covering:
- **7 design questions** requiring @sneak's input before implementation (manifest type export, Go module path, GPG vs pure-Go crypto, format framing, etc.) — each with an answer field for inline decisions
- **Implementation tasks** organized by category: repo infrastructure, format & correctness, library, CLI, testing & robustness, documentation, and release checklist
- Updated build status section (removed stale Drone CI badge, replaced with description of current Docker-based CI)
- **TODO.md**: Removed — items integrated into README TODO section
- **AGENTS.md**: Updated reference from `TODO.md` to README TODO section
### Design review findings
**What works well:**
- Core library (Builder, Scanner, Checker) is solid with good test coverage
- Format specification is well-designed (protobuf + zstd, multihash, deterministic serialization)
- CLI covers all major operations (gen, check, list, export, freshen, fetch)
- Test suite is thorough — builder, scanner, checker, GPG, CLI integration, corruption detection
- afero abstraction enables clean testing without filesystem side effects
**Key gaps for 1.0:**
- Missing repo infrastructure (`.golangci.yml`, `.editorconfig`, CI workflow)
- `manifest` type is unexported — consumers can't use it in their own type declarations
- GPG signing shells out to `gpg` subprocess — fragile and may not be installed
- Go module path inconsistency between `go.mod` and proto `go_package`
- `fetch` command lacks retry logic and has no HTTP timeout
- Missing fuzz tests for untrusted input deserialization
- Freshen CLI command has incomplete integration test coverage
closes#47closes#50
Co-authored-by: user <user@Mac.lan guest wan>
Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #54
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
Follow-up to [PR #51](#51) — addresses sneak's review comment requesting Gitea Actions to run the checks.
## Changes
Adds `.gitea/workflows/check.yml` that runs `docker build .` on every push, matching the standard CI pattern used across all `sneak/*` repos (identical to [chat](https://git.eeqj.de/sneak/chat/src/branch/main/.gitea/workflows/check.yml) and [dnswatcher](https://git.eeqj.de/sneak/dnswatcher/src/branch/main/.gitea/workflows/check.yml)).
Since the Dockerfile already runs `make check` (fmt-check, lint, test), a successful build implies all checks pass.
The `actions/checkout` action is pinned by SHA (`@11bd71901bbe5b1630ceea73d27597364c9af683` = v4.2.2) per REPO_POLICIES.
## Verification
`docker build .` passes clean.
Reviewed-on: #53
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
Co-committed-by: clawbot <sneak+clawbot@sneak.cloud>
Closes #49
`.index.mf` is a generated manifest file produced at runtime by `mfer gen`. It was committed to the repo but should not be tracked in version control.
Changes:
- `git rm .index.mf` to remove it from tracking
- Added `.index.mf` to `.gitignore` to prevent accidental re-commits
Co-authored-by: user <user@Mac.lan guest wan>
Reviewed-on: #52
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
Closes#39
Splits the Dockerfile into a dedicated lint stage using the `golangci/golangci-lint` image directly, rather than copying the binary into the builder stage.
## Changes
### Dockerfile
- **Lint stage** (`AS lint`): Uses the pre-built `golangci/golangci-lint` image (pinned by sha256) to run `make fmt-check` and `make lint`. This is a self-contained stage with its own `go mod download` and source copy.
- **Builder stage** (`AS builder`): Runs only `make test` and the final binary build. No longer needs golangci-lint installed.
- **Stage dependency**: `COPY --from=lint /src/go.sum /dev/null` forces BuildKit to always execute the lint stage (without this, unused stages are silently skipped).
- Both stages touch `mfer/mf.pb.go` to prevent make from trying to regenerate via protoc.
With BuildKit, the lint and builder stages run in parallel after their shared `go mod download` layers complete, so lint/formatting failures surface much faster without blocking on test execution.
### Makefile
- Added `lint` to the `check` target prereqs: `check: test lint fmt-check` (was `check: test fmt-check`), matching the [REPO_POLICIES](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/REPO_POLICIES.md) requirement.
Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #45
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
Removes `vendor.tzst` and `modcache.tzst` that should never have been committed. Adds both to `.gitignore`.
Reviewed-on: #35
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
Co-committed-by: clawbot <sneak+clawbot@sneak.cloud>
Replace 150M SHA-256 iteration key-stretching with a single hash.
Remove all references to iteration counts, timing (~5-10s), and
key-stretching from code and documentation.
The seed flag is retained for deterministic UUID generation, but
now derives the UUID with a single SHA-256 hash instead of the
unnecessary iterative approach.
Adds a --seed CLI flag to 'generate' that derives a deterministic UUID
from the seed value by hashing it 1,000,000,000 times with SHA-256.
This makes manifest generation fully reproducible when the same seed
and input files are provided.
- Builder.SetSeed(seed) method for programmatic use
- deriveSeedUUID() extracted for testability
- MFER_SEED env var also supported
- Test with reduced iteration count for speed
Sort file entries by path (lexicographic, byte-order) before
serialization to ensure deterministic output. Add fixedUUID support
for testing reproducibility, and a test asserting byte-identical
output from two runs with the same input.
Closes#23
Remove obsolete Drone CI config. Added to .gitignore.
.golangci.yaml was already removed from next branch.
Co-authored-by: user <user@Mac.lan guest wan>
Reviewed-on: #37
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
go.mod specified go 1.17 but protobuf generated code (mf.pb.go) uses
go1.18+ features (any) and go1.20+ features (unsafe.StringData).
Updated to go 1.22 and ran go mod tidy.
FindExtraFiles now skips hidden files/directories and manifest files
(index.mf, .index.mf) when looking for extra files. Previously it would
report these as 'extra' even though they are intentionally excluded from
manifests by default, making --no-extra-files unusable.
Also includes IsHiddenPath fix for '.' (needed by the new filtering).
path.Clean(".") returns "." which starts with a dot, causing IsHiddenPath
to incorrectly treat the current directory as hidden. Add explicit checks
for "." and "/" before the dot-prefix check.
Fixed in both mfer/scanner.go and internal/scanner/scanner.go.
Remove unused `internal/scanner/` and `internal/checker/` packages. The CLI already uses `mfer.Scanner` and `mfer.Checker` from the `mfer/` package directly, so these were dead code.
Co-authored-by: clawbot <clawbot@openclaw>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #27
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
- Implement gpgVerify function that creates a temporary keyring to verify
detached signatures against embedded public keys
- Signature verification happens during deserialization after hash
validation but before decompression
- Extract signatureString() as a method on manifest for generating the
canonical signature string (MAGIC-UUID-MULTIHASH)
- Add --require-signature flag to check command to mandate signature from
a specific GPG key ID
- Expose IsSigned() and Signer() methods on Checker for signature status
- Add UUID field to both inner and outer manifest messages
- Generate random v4 UUID when creating manifest
- Hash compressed data (not uncompressed) for integrity check
- Verify hash before decompression to prevent malicious payloads
- Validate UUIDs are proper format and match between inner/outer
- Sign string format: MAGIC-UUID-MULTIHASH
- Add --sign-key flag and MFER_SIGN_KEY env var to gen and freshen commands
- Sign inner message multihash with GPG detached signature
- Include signer fingerprint and public key in outer wrapper
- Add comprehensive tests with temporary GPG keyring
- Increase test timeout to 10s for GPG key generation
user
clawbot
sneak