fix: close TOCTOU window between blob eviction commit and unlink
StoreSource now hashes content itself and holds the per-hash contentLock across the whole store (file write plus accounting row inserts); evictSourceBlob holds the same lock across its whole operation (row deletion transaction through file unlink). A concurrent store and eviction of identical content bytes can no longer interleave: either runs to completion before the other starts, so a fresh row can never be left pointing at a file the other side is mid-unlink on. ContentStorage gains StoreHashed for callers that need the hash before writing; Store is refactored to share the write-if-absent logic with it, with no change to its existing behavior or signature. internal/imgcache/eviction_test.go: TestEvictSourceBlobExcludesConcurrentStoreOfIdenticalContent proves it: pauses eviction (via evictSourceBlobTestHook) in the exact window between commit and unlink, asserts a concurrent StoreSource for identical content blocks rather than completing, then verifies no dangling reference and that the store's data survives once eviction releases the hash.
This commit is contained in:
@@ -2,7 +2,9 @@ package imgcache
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"database/sql"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
@@ -215,8 +217,33 @@ func (c *Cache) StoreSource(
|
||||
return "", nil
|
||||
}
|
||||
|
||||
// Store content
|
||||
contentHash, size, err := c.srcContent.Store(content)
|
||||
// Hash the content ourselves (rather than via srcContent.Store,
|
||||
// which would hash internally) so the content hash is known before
|
||||
// any file or database work happens: that lets the entire store be
|
||||
// serialized, per hash, against a concurrent eviction of the same
|
||||
// content below.
|
||||
data, err := io.ReadAll(content)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to read source content: %w", err)
|
||||
}
|
||||
|
||||
sum := sha256.Sum256(data)
|
||||
contentHash := ContentHash(hex.EncodeToString(sum[:]))
|
||||
|
||||
// Hold the content hash's lock for the whole store operation. A
|
||||
// concurrent eviction of this exact hash (the real SHA-256 dedup
|
||||
// case: a different source path whose bytes hash identically)
|
||||
// deletes the accounting rows and unlinks the file inside the same
|
||||
// lock, so the two can never interleave: either this store
|
||||
// completes first (and a subsequent eviction removes it together
|
||||
// with its rows and file, correctly), or eviction completes first
|
||||
// (and this store finds the file already gone and recreates it
|
||||
// fresh) — never a fresh row left pointing at a file eviction is
|
||||
// mid-unlink on.
|
||||
unlock := c.contentLocks.Lock(string(contentHash))
|
||||
defer unlock()
|
||||
|
||||
size, err := c.srcContent.StoreHashed(contentHash, data)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to store source content: %w", err)
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user