fetch: destination directory, skip files already present, save the manifest, require a signer #101

Open
opened 2026-09-21 09:21:34 +02:00 by clawbot · 0 comments
Collaborator

Context

The README promises that mfer fetch <url> "downloads all files listed in manifest, optionally resuming any that already exist locally, and assures cryptographic integrity". internal/cli/fetch.go today:

  • writes only into the current directory; there is no destination flag.
  • re-downloads every file unconditionally; a file already present with the right size and hash is fetched again.
  • discards the manifest after parsing it, so mfer check cannot be run on the result afterwards.
  • has no --require-signature, while check has one. A fetched manifest is verified only against the key it embeds, so anyone can serve a validly self-signed tree.

Each of these makes the command unusable for the README's mirroring use case.

Definition of done

  • fetch accepts a destination directory flag (default .); every write lands under it.
  • A file already present under the destination with matching size and hash is skipped and reported as such; a leftover .tmp file from an interrupted run is discarded and re-fetched.
  • The manifest is saved into the destination under the default manifest name (#100 decides that name; use index.mf until it is answered) after all files verify, so mfer check runs on the result.
  • fetch accepts --require-signature with the same meaning and message text as check, enforced before the first file download.
  • Tests against httptest.Server: a second fetch into the same destination downloads nothing; a stale .tmp is replaced; the saved manifest reloads and check passes on the fetched tree; a wrong required signer aborts before any file is written.
  • make check passes; TODO.md updated in the same commit.

Implementation requirements

  • Do this after #63 and #66 have landed; all three touch fetch.go and this one builds on their tests.
  • Reuse verifyRequiredSigner from check.go rather than writing a second copy.
  • Skipping must verify the hash, not just the size; a size-only skip would accept a corrupted partial file.
  • Commit title must end with (closes #N) for this issue's number.

Model: fable-5-1

## Context The README promises that `mfer fetch <url>` "downloads all files listed in manifest, optionally resuming any that already exist locally, and assures cryptographic integrity". `internal/cli/fetch.go` today: - writes only into the current directory; there is no destination flag. - re-downloads every file unconditionally; a file already present with the right size and hash is fetched again. - discards the manifest after parsing it, so `mfer check` cannot be run on the result afterwards. - has no `--require-signature`, while `check` has one. A fetched manifest is verified only against the key it embeds, so anyone can serve a validly self-signed tree. Each of these makes the command unusable for the README's mirroring use case. ## Definition of done - `fetch` accepts a destination directory flag (default `.`); every write lands under it. - A file already present under the destination with matching size and hash is skipped and reported as such; a leftover `.tmp` file from an interrupted run is discarded and re-fetched. - The manifest is saved into the destination under the default manifest name (https://git.eeqj.de/sneak/mfer/issues/100 decides that name; use `index.mf` until it is answered) after all files verify, so `mfer check` runs on the result. - `fetch` accepts `--require-signature` with the same meaning and message text as `check`, enforced before the first file download. - Tests against `httptest.Server`: a second fetch into the same destination downloads nothing; a stale `.tmp` is replaced; the saved manifest reloads and `check` passes on the fetched tree; a wrong required signer aborts before any file is written. - `make check` passes; `TODO.md` updated in the same commit. ## Implementation requirements - Do this after https://git.eeqj.de/sneak/mfer/issues/63 and https://git.eeqj.de/sneak/mfer/issues/66 have landed; all three touch `fetch.go` and this one builds on their tests. - Reuse `verifyRequiredSigner` from `check.go` rather than writing a second copy. - Skipping must verify the hash, not just the size; a size-only skip would accept a corrupted partial file. - Commit title must end with ` (closes #N)` for this issue's number. Model: fable-5-1
clawbot self-assigned this 2026-09-21 09:21:34 +02:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/mfer#101