Limit how much fetch and check read for a manifest or a file (closes #168)
check / check (push) Canceled after 0s

NewManifestFromReader reads at most one byte past MaxManifestSize, a new
constant of 258 MiB: the 256 MiB decompressed limit grown by zstd's worst
case of 1/256, plus 1 MiB for the signature, the signing key and the
other outer fields. It refuses a larger manifest. fetch, and check given
a URL, stop downloading a manifest one byte past the same size and report
it as too large; tests lower that size to keep their memory small. fetch
stops reading a file one byte past its listed size, so a longer body ends
in the size mismatch at once instead of filling the disk. docs/FORMAT.md
states the limit and gives the decompressed limit as 256 MiB, the size
the code uses.

Model: opus-5-5
This commit was merged in pull request #172.
This commit is contained in:
2026-10-07 14:25:45 +02:00
parent 0762a728d4
commit f663f4242d
10 changed files with 176 additions and 20 deletions
+33
View File
@@ -0,0 +1,33 @@
//nolint:testpackage // white-box tests exercise unexported internals
package mfer
import (
"bytes"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestReadAtMost gives readAtMost exactly its maximum, which it must
// return whole, and twice its maximum, which it must refuse after reading
// one byte past the maximum, and no more. NewManifestFromReader reads
// through it with MaxManifestSize; the test uses 64 KiB, since reading
// MaxManifestSize under the race detector takes gigabytes of memory.
func TestReadAtMost(t *testing.T) {
t.Parallel()
const maxSize = 64 << 10
dat, err := readAtMost(bytes.NewReader(make([]byte, maxSize)), maxSize)
require.NoError(t, err)
assert.Len(t, dat, maxSize)
input := bytes.NewReader(make([]byte, 2*maxSize))
_, err = readAtMost(input, maxSize)
require.ErrorIs(t, err, errManifestTooLarge)
require.EqualError(t, err,
"manifest exceeds maximum allowed size of 65536 bytes")
assert.Equal(t, maxSize-1, input.Len(), "bytes left unread")
}