Originals cache size limit with least-recently-used eviction #47

New Issue

2026-09-22T09:11:07+02:00

clawbot commented

2026-09-22 09:11:07 +02:00

Cache/API implementation (#36), phase 3.
Depends on the content/thumbnail-store unit.

Goal

Bound the originals cache and evict least-recently-used (owner ruling: 100 GiB,
adapt down when the disk has under 50 GiB free).

Scope

cacheOriginalsMaxBytes default 100 GiB; freeBelowBytes default 50 GiB; both
open() options.
Effective limit before each original write = min(configured limit,
bytesUsedByOriginals + bytesFree - freeBelowBytes) using fs.statfs on the
volume holding cacheDirectory (Node 22 here). So the limit falls on a full
disk and rises as space returns; status().originalsLimitBytes shows it.
When the next write would cross the limit, remove originals least-recently-used
first until it fits. Pinned files (favorites + latest week — the precache unit
defines the pinned set; expose a predicate/hook this unit consumes) are
skipped; if only pinned remain, the write proceeds over-limit until the pinned
set shrinks.
Last-use = the file's mtime; touch (utimes) when a read returns the path,
so ordering survives restarts with no ledger. Do not rely on atime.
Only cacheDirectory/originals is ever evicted. downloadDirectory (the
backup) is never counted or touched. Thumbnails are never evicted.

Definition of done

Eviction removes LRU originals to fit; pinned skipped; over-limit-when-only-
pinned handled. Adaptive limit from statfs tested (mock statfs).
mtime touch on read tested. make check green.

Grounding

src/library/. Depends on content/thumbnail store; coordinates with precache
for the pinned set.

Dispatch notes: TDD; no scripted edits; no interactive questions; plain language.
Squash subject ends (closes #<this issue>). End every message with
Model: opus-4-8.

Model: opus-4-8

Cache/API implementation (https://git.eeqj.de/sneak/quak/issues/36), phase 3. Depends on the content/thumbnail-store unit. ## Goal Bound the originals cache and evict least-recently-used (owner ruling: 100 GiB, adapt down when the disk has under 50 GiB free). ## Scope - `cacheOriginalsMaxBytes` default 100 GiB; `freeBelowBytes` default 50 GiB; both `open()` options. - Effective limit before each original write = min(configured limit, `bytesUsedByOriginals + bytesFree - freeBelowBytes`) using `fs.statfs` on the volume holding `cacheDirectory` (Node 22 here). So the limit falls on a full disk and rises as space returns; `status().originalsLimitBytes` shows it. - When the next write would cross the limit, remove originals least-recently-used first until it fits. Pinned files (favorites + latest week — the precache unit defines the pinned set; expose a predicate/hook this unit consumes) are skipped; if only pinned remain, the write proceeds over-limit until the pinned set shrinks. - Last-use = the file's `mtime`; touch (`utimes`) when a read returns the path, so ordering survives restarts with no ledger. Do not rely on `atime`. - Only `cacheDirectory/originals` is ever evicted. `downloadDirectory` (the backup) is never counted or touched. Thumbnails are never evicted. ## Definition of done - Eviction removes LRU originals to fit; pinned skipped; over-limit-when-only- pinned handled. Adaptive limit from `statfs` tested (mock statfs). - `mtime` touch on read tested. `make check` green. ## Grounding `src/library/`. Depends on content/thumbnail store; coordinates with precache for the pinned set. Dispatch notes: TDD; no scripted edits; no interactive questions; plain language. Squash subject ends ` (closes #<this issue>)`. End every message with `Model: opus-4-8`. Model: opus-4-8

clawbot referenced this issue

2026-09-22 09:11:56 +02:00

Design for review: a cache layer (SQLite metadata + on-disk content/thumbnail store) and a crash-safe backup on it #36

clawbot referenced this issue from a commit

2026-09-22 19:03:25 +02:00

Bound the originals cache with adaptive LRU eviction (closes #47)

clawbot referenced a pull request that will close this issue

2026-09-22 19:04:05 +02:00

Bound the originals cache with adaptive LRU eviction (closes #47) #71

clawbot commented

2026-09-22 19:04:25 +02:00

Implemented in #71 (branch issue-47-lru-eviction, base next).

Bounds cacheDirectory/originals on the content cache (#46): open() options cacheOriginalsMaxBytes (default 100 GiB) and freeBelowBytes (default 50 GiB). Before each original write the effective limit is min(cacheOriginalsMaxBytes, bytesUsedByOriginals + bytesFree - freeBelowBytes) from fs.statfs, so it falls on a full disk and rises as space returns; status().originalsLimitBytes exposes it. A crossing write evicts least-recently-used originals until it fits, skipping pinned files, and proceeds over-limit when only pinned remain. Last-use is the file mtime, touched on every read that returns a path, so order survives restarts with no ledger. The backup download directory and thumbnails are never counted or evicted.

The pinned predicate is a hook (isOriginalPinned) left for the precache unit (#48) to supply.

One judgement call: eviction runs just after the write, once the plaintext size is known on disk (the streaming download layer cannot report it beforehand); the freeBelowBytes reserve absorbs the single-file overshoot.

make check and make build green.

Model: opus-4-8

Implemented in https://git.eeqj.de/sneak/quak/pulls/71 (branch `issue-47-lru-eviction`, base `next`). Bounds `cacheDirectory/originals` on the content cache (#46): `open()` options `cacheOriginalsMaxBytes` (default 100 GiB) and `freeBelowBytes` (default 50 GiB). Before each original write the effective limit is `min(cacheOriginalsMaxBytes, bytesUsedByOriginals + bytesFree - freeBelowBytes)` from `fs.statfs`, so it falls on a full disk and rises as space returns; `status().originalsLimitBytes` exposes it. A crossing write evicts least-recently-used originals until it fits, skipping pinned files, and proceeds over-limit when only pinned remain. Last-use is the file `mtime`, touched on every read that returns a path, so order survives restarts with no ledger. The backup download directory and thumbnails are never counted or evicted. The pinned predicate is a hook (`isOriginalPinned`) left for the precache unit (https://git.eeqj.de/sneak/quak/issues/48) to supply. One judgement call: eviction runs just after the write, once the plaintext size is known on disk (the streaming download layer cannot report it beforehand); the `freeBelowBytes` reserve absorbs the single-file overshoot. `make check` and `make build` green. Model: opus-4-8

clawbot referenced this issue

2026-09-22 19:09:15 +02:00

Bound the originals cache with adaptive LRU eviction (closes #47) #71

clawbot referenced this issue from a commit

2026-09-22 19:19:20 +02:00

Bound the originals cache with adaptive LRU eviction (closes #47)

clawbot referenced this issue from a commit

2026-09-22 19:57:39 +02:00

Bound the originals cache with adaptive LRU eviction (closes #47)

clawbot closed this issue

2026-09-22 20:13:22 +02:00

clawbot referenced this issue from a commit