Reject a blob_size_limit below the largest possible chunk (closes #167)
Validate only rejected blob_size_limit below chunk_size, but the chunker can emit chunks up to chunk_size times the FastCDC size spread (four times), and the packer puts a single chunk of any size into an otherwise empty blob. A limit between one and four times chunk_size therefore let a blob reach four times the configured maximum, with most blobs holding a single chunk and so exposing individual chunk lengths to anyone who can list the destination. Validate now rejects blob_size_limit below chunk_size times the spread, reusing the chunker's one constant (now exported as ChunkSizeSpread) instead of a second literal. The rule is stated in the error text, the Validate comment, the README config table, config.example.yml, and the generated config template. Model: opus-4-8
This commit was merged in pull request #181.
This commit is contained in:
@@ -33,9 +33,10 @@ type Chunker struct {
|
||||
maxChunkSize int
|
||||
}
|
||||
|
||||
// chunkSizeSpread is the FastCDC-recommended factor between the average
|
||||
// ChunkSizeSpread is the FastCDC-recommended factor between the average
|
||||
// chunk size and the minimum (avg/spread) and maximum (avg*spread) sizes.
|
||||
const chunkSizeSpread = 4
|
||||
// The largest chunk the chunker can emit is therefore avg*ChunkSizeSpread.
|
||||
const ChunkSizeSpread = 4
|
||||
|
||||
// NewChunker creates a new chunker with the specified average chunk size.
|
||||
// The actual chunk sizes will vary between avgChunkSize/4 and avgChunkSize*4
|
||||
@@ -45,8 +46,8 @@ func NewChunker(avgChunkSize int64) *Chunker {
|
||||
// FastCDC recommends min = avg/4 and max = avg*4
|
||||
return &Chunker{
|
||||
avgChunkSize: int(avgChunkSize),
|
||||
minChunkSize: int(avgChunkSize / chunkSizeSpread),
|
||||
maxChunkSize: int(avgChunkSize * chunkSizeSpread),
|
||||
minChunkSize: int(avgChunkSize / ChunkSizeSpread),
|
||||
maxChunkSize: int(avgChunkSize * ChunkSizeSpread),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user