Limit how much fetch and check read for a manifest or a file (closes #168)
check / check (push) Waiting to run
check / check (push) Waiting to run
NewManifestFromReader reads at most one byte past MaxManifestSize, a new constant of 258 MiB: the 256 MiB decompressed limit grown by zstd's worst case of 1/256, plus 1 MiB for the signature, the signing key and the other outer fields. It refuses a larger manifest. fetch, and check given a URL, stop downloading a manifest at the same point, so the refusal comes from the library. fetch stops reading a file one byte past its listed size, so a longer body ends in the size mismatch at once instead of filling the disk. docs/FORMAT.md states the limit and gives the decompressed limit as 256 MiB, the size the code uses. Model: opus-5-5
This commit is contained in:
@@ -433,13 +433,15 @@ func fetchManifest(
|
||||
|
||||
// Read the whole manifest before parsing it, so that a connection
|
||||
// lost partway through is retried rather than reported as a bad
|
||||
// manifest.
|
||||
// manifest. One byte past mfer.MaxManifestSize is enough for
|
||||
// mfer.NewManifestFromReader to refuse a manifest that is too large.
|
||||
var manifestData []byte
|
||||
|
||||
err := client.get(ctx, manifestURL, func(resp *http.Response) error {
|
||||
var readErr error
|
||||
|
||||
manifestData, readErr = io.ReadAll(resp.Body)
|
||||
manifestData, readErr = io.ReadAll(
|
||||
io.LimitReader(resp.Body, mfer.MaxManifestSize+1))
|
||||
|
||||
return readErr
|
||||
})
|
||||
@@ -910,8 +912,10 @@ func saveResponse(
|
||||
progress: progress,
|
||||
}
|
||||
|
||||
// Copy content while hashing and reporting progress
|
||||
written, copyErr := io.Copy(pw, resp.Body)
|
||||
// Copy content while hashing and reporting progress. One byte past
|
||||
// the listed size is enough for finishDownload to report a size
|
||||
// mismatch.
|
||||
written, copyErr := io.Copy(pw, io.LimitReader(resp.Body, expectedSize+1))
|
||||
|
||||
// Close file before checking errors (to flush writes)
|
||||
closeErr := out.Close()
|
||||
|
||||
Reference in New Issue
Block a user