fetch: client timeout, retry with backoff, url.JoinPath (closes #63)
check / check (push) Failing after 1s

fetch now makes every request through an http.Client with a time
limit, ten minutes by default and set with --timeout; the manifest
loader shares that default. A connection error, a timeout, or a 5xx
or 429 response is retried, up to five tries in all, after a random
wait whose limit doubles from one second, or after the wait the
server's Retry-After asks for, up to one minute. Each try of a file
starts a new temp file, so a retry never keeps a partial file; the
size and hash checks are unchanged. Manifest and file URLs are built
with URL.JoinPath, so a base URL with or without a trailing slash or
with a query string, and names that need escaping, all work.

Model: opus-5-5
This commit is contained in:
2026-10-04 12:18:02 +00:00
parent e412d20c20
commit 35d771b704
5 changed files with 541 additions and 66 deletions
+5 -2
View File
@@ -293,8 +293,11 @@ func TestFetchFileHTTPStatusMessage(t *testing.T) {
}))
defer server.Close()
err := downloadFile(context.Background(), server.URL+"/x", "x",
&mfer.MFFilePath{}, nil)
// downloadFile logs each retry of the 500 to the process-global logger.
err := runLocked(func() error {
return downloadFile(context.Background(), testClient(), server.URL+"/x", "x",
&mfer.MFFilePath{}, nil)
})
require.ErrorIs(t, err, errHTTPStatus)
assert.EqualError(t, err, "HTTP 500")
}