Commit Graph
11 Commits
Author SHA1 Message Date
sneak 4d400f5a0d Precache all thumbnails and pinned originals from Library.open (closes #48)
check / check (push) Successful in 31s
Two background fills start inside open() with no caller input, through the
shared pools (#45) at background priority, so on-demand work always preempts
them. Thumbnails: every file newest first until all are on disk, sharing the
pool with thumbnails.ensure. Originals: the pinned set — favorites, then the
latest-week window ending at the newest file — through the content pool. The
pinned set is the eviction predicate (#47): a pinned original is never evicted,
and a file that leaves the set (favorite removed, or window moved on a later
refresh) becomes an ordinary evictable original. Each refresh re-kicks the
fills to pick up new files and retry failures. Progress via open() onProgress
and status().

Model: opus-4-8
2026-09-22 19:02:08 +00:00
clawbot e8575780e4 Content-similarity search surface over the CLIP index (closes #50)
check / check (push) Successful in 15s
Adds lib.mldata search over the CLIP index (#49): forFile returns a file's stored payload; similar ranks nearest files by cosine on the CLIP embedding; searchByEmbedding ranks the index against a caller-supplied query vector. All RAM-only, reusing the packed Float32Array index and id list. No text encoder is bundled — the caller provides the query embedding. (Redo of the reverted first attempt, now tsc-clean.)

Model: opus-4-8
2026-09-22 20:45:59 +02:00
clawbot 17d1d74615 Rewrite the backup command on the library API with a durable failure ledger (closes #51)
check / check (push) Successful in 25s
Rewrites backup on the library API. backup() refreshes, fetches pending originals (and optionally thumbnails) through the pools reusing the content cache, then materialises the unchanged collections/ symlink views + per-collection JSON + sidecars from the model. A symlink failure no longer aborts the run (closes #8). failures.json reconciles against each run's attempted set — since-deleted/out-of-scope/resolved entries clear, still-failing retained, one attempt per file per run — and the exit-code contract is preserved.

Model: opus-4-8
2026-09-22 20:21:49 +02:00
clawbot c05d63a2f0 Originals cache size limit with least-recently-used eviction (closes #47)
check / check (push) Successful in 25s
Bounds the originals cache and evicts least-recently-used. cacheOriginalsMaxBytes default 100 GiB; the effective limit adapts down via fs.statfs to keep freeBelowBytes (default 50 GiB) free. Over-limit writes evict unpinned originals oldest-mtime-first (mtime touched on read); pinned files (favorites + latest week) are skipped and an over-budget on-demand fetch proceeds over-limit. Every in-flight write is excluded from eviction, so concurrent fetches never delete each other's just-stored file. Only cacheDirectory/originals is evicted; downloadDirectory and thumbnails never.

Model: opus-4-8
2026-09-22 20:13:22 +02:00
clawbot 5db59a6e2b On-disk content and thumbnail cache with per-photo fetch and prefetch (closes #46)
check / check (push) Successful in 33s
Adds the on-disk content and thumbnail cache keyed by fileID: originals/ and thumbnails/ under cacheDirectory, present-means-complete (streaming atomic rename), orphan temp reaping on open. Photo.original/thumbnail return a cached path with no network when present, else fetch through the shared request pool; thumbnails.ensure drives the thumbnail pool with priority, dedup and AbortSignal. One shared RequestPools set serves both the ML fetch and the content cache. Content-hash integrity is deferred (#68); authenticated streaming decrypt guarantees integrity now.

Model: opus-4-8
2026-09-22 18:38:22 +02:00
clawbot 61dfec8d38 Revert ML search (#50) to restore a green next (build fix pending)
check / check (push) Successful in 18s
Reverts the #50 merge (224bd101): mlsearch.ts failed tsc in the CI build (make check does not run make build, so it slipped past review). next restored to green; #50 to be redone with make build in its gate.

Model: opus-4-8
2026-09-22 18:21:54 +02:00
clawbot 224bd101ab Content-similarity search surface over the CLIP index (closes #50)
check / check (push) Failing after 23s
Adds lib.mldata search over the CLIP index (#49): forFile returns a file's stored payload; similar ranks nearest files by cosine on the CLIP embedding; searchByEmbedding ranks the index against a caller-supplied query vector. All RAM-only, reusing the packed Float32Array index and id list. No text encoder is bundled — the caller provides the query embedding.

Model: opus-4-8
2026-09-22 18:14:13 +02:00
clawbot d7f415fe29 Fetch, store, and index per-file ML data (closes #49)
check / check (push) Successful in 14s
Adds the machine-learning (magic) data layer: fetches per-file ML payloads (face detections + CLIP embeddings) via the existing metadata-backup fetch through the metadata pool after each refresh, decrypts and gunzips them, and stores one mldata/<fileID>.json per file by rename (present-means-complete). A derived index (mldata/clip.f32 + clip.json) loads in one read and is rebuilt whenever it disagrees with the payloads on disk in either direction, so an interrupted backfill self-heals. Never in metadata.json; incremental on later refreshes; progress via onProgress/status.

Model: opus-4-8
2026-09-22 17:54:40 +02:00
clawbot c5c1f387df In-process read surface: albums, photos, and timeline grouping (closes #44)
check / check (push) Successful in 25s
Adds the in-process read surface, served from RAM with args-object signatures: albums (list/byName/byID), photos (byID/records -> plain PhotoRecord[]), thin Album/Photo wrappers, and timeline.groups (day/week/month) with a PhotoFilter (albumID/text/fileTypes/hasLocation/includeArchived; hidden excluded). Week keys use ISO YYYY-Www; each file appears once per group, newest first. Built on the #43 snapshot projection; no network.

Model: opus-4-8
2026-09-22 17:21:48 +02:00
clawbot fbb8ae44a7 Plain-record library snapshot and change subscription (closes #43)
check / check (push) Successful in 30s
Adds a synchronous, key-free snapshot() returning LibrarySnapshot (one PhotoRecord per fileID, deduped, newest first) with edited-name/time precedence (pubMagicMetadata over basic metadata) in milliseconds, plus AlbumRecord (favorites identified by type). subscribe({onChange}) delivers LibraryChange (changed/removed albums and files, refreshedAt) only when a refresh changes something; unsubscribe stops delivery. Built on the existing refresh loop; served from RAM, safe to send over IPC.

Model: opus-4-8
2026-09-22 15:22:52 +02:00
clawbot 7570055a5b Library.open with a transparent background refresh (closes #42)
check / check (push) Successful in 24s
Library.open loads metadata.json and serves reads from RAM: an empty cache awaits the first refresh, an existing cache returns at once and refreshes in the background so an unreachable server never stalls open(). A background timer refreshes every refreshIntervalSeconds (default 3), diffing only changed albums via the resumable cursor+tombstone enumerators and rewriting metadata.json only when something changed. A refresh failure is invisible to reads and surfaced via status()/onProgress; a failed save keeps status().lastError set and retries until one lands, so a stale disk is never masked. No sync()/refresh()/serverReachable surface; status() and close() included.

Model: opus-4-8
2026-09-22 14:52:02 +02:00