Send fetch progress at most once per second (closes #206) #215

Merged
clawbot merged 1 commits from issue-206-fetch-progress-rate into next 2026-10-14 00:01:23 +02:00
Collaborator

mfer fetch logged a progress line for every 32 KiB it downloaded, because progressWriter.Write sent an update on each chunk io.Copy handed it. It now sends at most one update per second per file, plus one when the last byte is written, as gen and check do.

Closes #206

What the diff does not show:

  • The first update waits a second from the start of the download, as in Checker.Check and the scanner. A file that downloads in under a second prints only its completion line.
  • The completion update fires when the bytes written equal the expected total (Content-Length, or the manifest's size when the server sends none). A body that ends short of that total gets no completion line, and the download fails its size check anyway.
  • The bound in TestFetchProgress allows one more update for each whole second the download took, so a slow run on a loaded host does not fail it. A download that finishes within a second must send no more than two.

Model: opus-5-5

`mfer fetch` logged a progress line for every 32 KiB it downloaded, because `progressWriter.Write` sent an update on each chunk `io.Copy` handed it. It now sends at most one update per second per file, plus one when the last byte is written, as `gen` and `check` do. Closes https://git.eeqj.de/sneak/mfer/issues/206 What the diff does not show: - The first update waits a second from the start of the download, as in `Checker.Check` and the scanner. A file that downloads in under a second prints only its completion line. - The completion update fires when the bytes written equal the expected total (`Content-Length`, or the manifest's size when the server sends none). A body that ends short of that total gets no completion line, and the download fails its size check anyway. - The bound in `TestFetchProgress` allows one more update for each whole second the download took, so a slow run on a loaded host does not fail it. A download that finishes within a second must send no more than two. Model: opus-5-5
clawbot added the needs-review label 2026-10-13 21:03:29 +02:00
clawbot self-assigned this 2026-10-13 21:03:29 +02:00
clawbot added 1 commit 2026-10-13 21:03:29 +02:00
progressWriter sent an update for every chunk io.Copy wrote, so fetch
logged a line per 32 KiB. It now sends at most one per second, waiting
a second before the first as gen and check do, and one more when the
last byte is written.

TestFetchProgress now checks that a download of several chunks sends no
more than one update per second plus the last.

Model: opus-5-5
Author
Collaborator

Review passed.

  • Judgement call: freshen --progress has no rate limit on next, although #206 says it does. I checked fetch against the once-per-second limit that gen and check apply, and it matches.

Model: opus-5-5

Review passed. - Judgement call: `freshen --progress` has no rate limit on `next`, although https://git.eeqj.de/sneak/mfer/issues/206 says it does. I checked `fetch` against the once-per-second limit that `gen` and `check` apply, and it matches. Model: opus-5-5
clawbot merged commit 2ac238d767 into next 2026-10-14 00:01:23 +02:00
clawbot deleted branch issue-206-fetch-progress-rate 2026-10-14 00:01:24 +02:00
Sign in to join this conversation.