sneak
bdd86a4c1e
Migration 002 adds a variant_content table (processed variants were untracked on disk) and an LRU timestamp on source_content. Total usage is two SUMs, never a directory scan on the hot path; hits touch LRU timestamps best-effort. A background goroutine evicts globally least-recently-used entries (variants and source blobs merged) until usage is under MaxBytes, woken by a periodic ticker and by non-blocking write-pressure notifications from stores. Evicting a source blob deletes all source_metadata rows referencing it plus its source_content row in one transaction before the file is unlinked, so multi-referenced blobs are removed only with all their references and rows never point at deleted files; JSON sidecars are cleaned up too. A one-time startup reconciliation walk adopts untracked variant files, drops rows whose files are missing, removes unreachable source blobs, and sweeps stale temp files. CacheConfig.DisableDiskCache turns the disk cache off entirely (config maps cache_max_bytes: 0 to it): no directories, lookups miss, stores no-op, no evictor. Handlers wire the limit, start eviction on startup, and stop it on shutdown.
26 lines
1.1 KiB
SQL
26 lines
1.1 KiB
SQL
-- Migration 002: cache size accounting and eviction
|
|
--
|
|
-- Tracks processed variants in the database (source content blobs are
|
|
-- already tracked in source_content) so total cache usage can be
|
|
-- computed without directory scans, and adds last-access timestamps
|
|
-- for LRU eviction ordering.
|
|
|
|
-- Processed variant blobs
|
|
-- Files stored at: cache/variants/<ab>/<cd>/<cache_key> (plus a
|
|
-- .meta sidecar with the content type)
|
|
CREATE TABLE IF NOT EXISTS variant_content (
|
|
cache_key TEXT PRIMARY KEY,
|
|
size_bytes INTEGER NOT NULL,
|
|
content_type TEXT NOT NULL,
|
|
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
last_accessed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
|
|
);
|
|
CREATE INDEX IF NOT EXISTS idx_variant_content_last_accessed
|
|
ON variant_content(last_accessed_at);
|
|
|
|
-- LRU timestamp for source content blobs. Rows written before this
|
|
-- migration have NULL here; eviction falls back to fetched_at.
|
|
ALTER TABLE source_content ADD COLUMN last_accessed_at DATETIME;
|
|
CREATE INDEX IF NOT EXISTS idx_source_content_last_accessed
|
|
ON source_content(last_accessed_at);
|