Commit Graph
10 Commits
Author SHA1 Message Date
sneak 4a83b25bf6 Bound the originals cache with adaptive LRU eviction (closes #47)
check / check (push) Successful in 26s
Extends the content cache (#46) so cacheDirectory/originals stays within a
size limit. Before each original write the effective limit is
min(cacheOriginalsMaxBytes, bytesUsedByOriginals + bytesFree - freeBelowBytes),
with bytesFree from fs.statfs on the cache volume: it falls as the disk fills
and rises as space returns. status().originalsLimitBytes exposes it. A write
that crosses the limit evicts least-recently-used originals until it fits;
pinned originals are skipped, and when only pinned remain the write proceeds
over-limit. Last-use is the file mtime, bumped on every read that returns a
path, so order survives restarts with no ledger.

Eviction runs just after the write, once the plaintext size is known on disk
(the download layer cannot report it beforehand); the freeBelowBytes reserve
absorbs the transient. Defaults: 100 GiB limit, 50 GiB reserve. The pinned
predicate is a hook the precache unit (#48) will supply; only
cacheDirectory/originals is touched — the backup download directory and
thumbnails are never counted or evicted.

Model: opus-4-8
2026-09-22 17:03:16 +00:00
clawbot 5db59a6e2b On-disk content and thumbnail cache with per-photo fetch and prefetch (closes #46)
check / check (push) Successful in 33s
Adds the on-disk content and thumbnail cache keyed by fileID: originals/ and thumbnails/ under cacheDirectory, present-means-complete (streaming atomic rename), orphan temp reaping on open. Photo.original/thumbnail return a cached path with no network when present, else fetch through the shared request pool; thumbnails.ensure drives the thumbnail pool with priority, dedup and AbortSignal. One shared RequestPools set serves both the ML fetch and the content cache. Content-hash integrity is deferred (#68); authenticated streaming decrypt guarantees integrity now.

Model: opus-4-8
2026-09-22 18:38:22 +02:00
clawbot 61dfec8d38 Revert ML search (#50) to restore a green next (build fix pending)
check / check (push) Successful in 18s
Reverts the #50 merge (224bd101): mlsearch.ts failed tsc in the CI build (make check does not run make build, so it slipped past review). next restored to green; #50 to be redone with make build in its gate.

Model: opus-4-8
2026-09-22 18:21:54 +02:00
clawbot 224bd101ab Content-similarity search surface over the CLIP index (closes #50)
check / check (push) Failing after 23s
Adds lib.mldata search over the CLIP index (#49): forFile returns a file's stored payload; similar ranks nearest files by cosine on the CLIP embedding; searchByEmbedding ranks the index against a caller-supplied query vector. All RAM-only, reusing the packed Float32Array index and id list. No text encoder is bundled — the caller provides the query embedding.

Model: opus-4-8
2026-09-22 18:14:13 +02:00
clawbot d7f415fe29 Fetch, store, and index per-file ML data (closes #49)
check / check (push) Successful in 14s
Adds the machine-learning (magic) data layer: fetches per-file ML payloads (face detections + CLIP embeddings) via the existing metadata-backup fetch through the metadata pool after each refresh, decrypts and gunzips them, and stores one mldata/<fileID>.json per file by rename (present-means-complete). A derived index (mldata/clip.f32 + clip.json) loads in one read and is rebuilt whenever it disagrees with the payloads on disk in either direction, so an interrupted backfill self-heals. Never in metadata.json; incremental on later refreshes; progress via onProgress/status.

Model: opus-4-8
2026-09-22 17:54:40 +02:00
clawbot c5c1f387df In-process read surface: albums, photos, and timeline grouping (closes #44)
check / check (push) Successful in 25s
Adds the in-process read surface, served from RAM with args-object signatures: albums (list/byName/byID), photos (byID/records -> plain PhotoRecord[]), thin Album/Photo wrappers, and timeline.groups (day/week/month) with a PhotoFilter (albumID/text/fileTypes/hasLocation/includeArchived; hidden excluded). Week keys use ISO YYYY-Www; each file appears once per group, newest first. Built on the #43 snapshot projection; no network.

Model: opus-4-8
2026-09-22 17:21:48 +02:00
clawbot 000d395c87 Three bounded request pools for metadata, content, and thumbnails (closes #45)
check / check (push) Successful in 55s
Adds three independent bounded request pools — metadata (10 in flight), content (5), thumbnails (25), each overridable — with on-demand-before-background priority and in-flight dedup (a shared key runs once); an idle pool never lends slots, and retries run inside a slot. Self-contained module; the content cache wires it into the library later.

Model: opus-4-8
2026-09-22 15:31:21 +02:00
clawbot fbb8ae44a7 Plain-record library snapshot and change subscription (closes #43)
check / check (push) Successful in 30s
Adds a synchronous, key-free snapshot() returning LibrarySnapshot (one PhotoRecord per fileID, deduped, newest first) with edited-name/time precedence (pubMagicMetadata over basic metadata) in milliseconds, plus AlbumRecord (favorites identified by type). subscribe({onChange}) delivers LibraryChange (changed/removed albums and files, refreshedAt) only when a refresh changes something; unsubscribe stops delivery. Built on the existing refresh loop; served from RAM, safe to send over IPC.

Model: opus-4-8
2026-09-22 15:22:52 +02:00
clawbot 7570055a5b Library.open with a transparent background refresh (closes #42)
check / check (push) Successful in 24s
Library.open loads metadata.json and serves reads from RAM: an empty cache awaits the first refresh, an existing cache returns at once and refreshes in the background so an unreachable server never stalls open(). A background timer refreshes every refreshIntervalSeconds (default 3), diffing only changed albums via the resumable cursor+tombstone enumerators and rewriting metadata.json only when something changed. A refresh failure is invisible to reads and surfaced via status()/onProgress; a failed save keeps status().lastError set and retries until one lands, so a stale disk is never masked. No sync()/refresh()/serverReachable surface; status() and close() included.

Model: opus-4-8
2026-09-22 14:52:02 +02:00
clawbot 72ea8dcb01 On-disk JSON metadata store for the local cache (closes #41)
check / check (push) Successful in 23s
Adds the metadata.json store: loads whole into RAM with id-lookup Maps, rewrites whole through the exported fsync atomic writer (temp, fsync, rename, dir fsync); a missing, unparseable, or wrong-schema file loads as empty (it is a cache); directory 0700, file 0600. No lock file, no public sync().

Model: opus-4-8
2026-09-22 12:52:50 +02:00