fetch: client timeout, retry with backoff, url.JoinPath (closes #63)
check / check (push) Failing after 1s
check / check (push) Failing after 1s
fetch now makes every request through an http.Client with a time limit, ten minutes by default and set with --timeout; the manifest loader shares that default. A connection error, a timeout, or a 5xx or 429 response is retried, up to five tries in all, after a random wait whose limit doubles from one second, or after the wait the server's Retry-After asks for, up to one minute. Each try of a file starts a new temp file, so a retry never keeps a partial file; the size and hash checks are unchanged. Manifest and file URLs are built with URL.JoinPath, so a base URL with or without a trailing slash or with a query string, and names that need escaping, all work. Model: opus-5-5
This commit is contained in:
@@ -7,14 +7,10 @@ import (
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/urfave/cli/v2"
|
||||
)
|
||||
|
||||
// manifestFetchTimeout bounds HTTP requests made to fetch a manifest.
|
||||
const manifestFetchTimeout = 30 * time.Second
|
||||
|
||||
// errHTTPStatus indicates an HTTP response with a non-OK status code.
|
||||
//
|
||||
// Its text is the literal "HTTP" prefix of the rendered "HTTP <code>"
|
||||
@@ -32,7 +28,7 @@ func isHTTPURL(s string) bool {
|
||||
// The caller must close the returned reader.
|
||||
func (mfa *CLIApp) openManifestReader(pathOrURL string) (io.ReadCloser, error) {
|
||||
if isHTTPURL(pathOrURL) {
|
||||
client := &http.Client{Timeout: manifestFetchTimeout}
|
||||
client := &http.Client{Timeout: httpTimeout}
|
||||
|
||||
req, err := http.NewRequestWithContext(
|
||||
context.Background(), http.MethodGet, pathOrURL, nil,
|
||||
|
||||
Reference in New Issue
Block a user