sneak/vaultik
1
0
Fork 0
Code Issues 7 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

Restore: blobCache grows unboundedly, will OOM with large backups #11

New Issue
Closed
opened 2026-02-08 17:16:25 +01:00 by clawbot · 1 comment
clawbot commented 2026-02-08 17:16:25 +01:00
Collaborator
Copy Link

In restore.go Restore(), downloaded+decrypted blobs are cached in memory:

blobCache := make(map[string][]byte)
...
blobCache[blobHashStr] = blobData

This cache is never evicted. For large restores with many blobs (each up to 10GB per config), this will exhaust memory. Should implement an LRU cache with a configurable size limit, or write cached blobs to temp files.

Ref: parent issue #1

In `restore.go` `Restore()`, downloaded+decrypted blobs are cached in memory: ```go blobCache := make(map[string][]byte) ... blobCache[blobHashStr] = blobData ``` This cache is never evicted. For large restores with many blobs (each up to 10GB per config), this will exhaust memory. Should implement an LRU cache with a configurable size limit, or write cached blobs to temp files. Ref: parent issue #1
clawbot self-assigned this 2026-02-20 09:29:09 +01:00
clawbot added the needs-review label 2026-02-20 09:29:09 +01:00
clawbot commented 2026-03-17 13:46:54 +01:00
Author
Collaborator
Copy Link

This issue is already resolved on main. The unbounded in-memory map[string][]byte blob cache was replaced with a disk-backed LRU cache (blobDiskCache in internal/vaultik/blobcache.go) via PR #34, which closed duplicate issue #29.

The implementation:

  • Writes cached blobs to a temp directory instead of holding them in memory
  • Tracks total cached bytes with LRU eviction when the limit is exceeded
  • Cache max is set to 4 × BlobSizeLimit (configurable via config)
  • Entries larger than maxBytes are silently skipped
  • Has comprehensive unit tests in blobcache_test.go

Closing as duplicate of #29.

This issue is already resolved on `main`. The unbounded in-memory `map[string][]byte` blob cache was replaced with a disk-backed LRU cache (`blobDiskCache` in `internal/vaultik/blobcache.go`) via [PR #34](https://git.eeqj.de/sneak/vaultik/pulls/34), which closed duplicate [issue #29](https://git.eeqj.de/sneak/vaultik/issues/29). The implementation: - Writes cached blobs to a temp directory instead of holding them in memory - Tracks total cached bytes with LRU eviction when the limit is exceeded - Cache max is set to `4 × BlobSizeLimit` (configurable via config) - Entries larger than `maxBytes` are silently skipped - Has comprehensive unit tests in `blobcache_test.go` Closing as duplicate of [#29](https://git.eeqj.de/sneak/vaultik/issues/29).
clawbot removed the needs-review label 2026-03-17 13:47:31 +01:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
merge-ready
All checks pass, reviewed, rebased, ready for merge
 needs-checks
make check does not pass cleanly
 needs-rebase
PR has merge conflicts or is behind main
 needs-review
Code review not yet done
 needs-review
needs-review
needs-review
needs-review
needs-rework
Code review found issues to fix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
clawbot sneak
No Assignees clawbot
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/vaultik#11
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?