Files
vaultik/internal
sneak 683fb0b103 Replace LRU eviction in restore with reference-counted sweeper
Restore previously capped the blob disk cache at 4× the configured
blob_size_limit (so 40 GB by default). With large or heavily-deduped
snapshots a chunk-by-chunk file walk could blow past that cap and
trigger LRU eviction of blobs that were still needed by later files,
forcing repeated re-downloads — observed during a real restore as
single-stream throughput collapsing to under 1 MB/s.

Restore now allocates the cache with no practical size cap and drives
eviction explicitly:

  * An in-memory set of restored file IDs accumulates as files finish.
  * Every blob_size_limit/100 bytes of restored data (≈100 sweeps per
    blob's worth of writes) the sweeper iterates the cache. For each
    cached blob it queries the snapshot's local SQLite DB for every
    file that references any chunk in the blob and deletes the cache
    entry only when every such file is already in the restored set.
  * blobStillNeeded returns true on any error so an unreadable DB
    never causes premature eviction.

The cache itself gains Delete(key) and Keys() so the sweeper can drive
removal without touching internal LRU state.
2026-06-17 07:15:22 +02:00
..