Commit Graph
12 Commits
Author SHA1 Message Date
sneak 4d400f5a0d Precache all thumbnails and pinned originals from Library.open (closes #48)
check / check (push) Successful in 31s
Two background fills start inside open() with no caller input, through the
shared pools (#45) at background priority, so on-demand work always preempts
them. Thumbnails: every file newest first until all are on disk, sharing the
pool with thumbnails.ensure. Originals: the pinned set — favorites, then the
latest-week window ending at the newest file — through the content pool. The
pinned set is the eviction predicate (#47): a pinned original is never evicted,
and a file that leaves the set (favorite removed, or window moved on a later
refresh) becomes an ordinary evictable original. Each refresh re-kicks the
fills to pick up new files and retry failures. Progress via open() onProgress
and status().

Model: opus-4-8
2026-09-22 19:02:08 +00:00
clawbot e8575780e4 Content-similarity search surface over the CLIP index (closes #50)
check / check (push) Successful in 15s
Adds lib.mldata search over the CLIP index (#49): forFile returns a file's stored payload; similar ranks nearest files by cosine on the CLIP embedding; searchByEmbedding ranks the index against a caller-supplied query vector. All RAM-only, reusing the packed Float32Array index and id list. No text encoder is bundled — the caller provides the query embedding. (Redo of the reverted first attempt, now tsc-clean.)

Model: opus-4-8
2026-09-22 20:45:59 +02:00
clawbot c05d63a2f0 Originals cache size limit with least-recently-used eviction (closes #47)
check / check (push) Successful in 25s
Bounds the originals cache and evicts least-recently-used. cacheOriginalsMaxBytes default 100 GiB; the effective limit adapts down via fs.statfs to keep freeBelowBytes (default 50 GiB) free. Over-limit writes evict unpinned originals oldest-mtime-first (mtime touched on read); pinned files (favorites + latest week) are skipped and an over-budget on-demand fetch proceeds over-limit. Every in-flight write is excluded from eviction, so concurrent fetches never delete each other's just-stored file. Only cacheDirectory/originals is evicted; downloadDirectory and thumbnails never.

Model: opus-4-8
2026-09-22 20:13:22 +02:00
clawbot 5db59a6e2b On-disk content and thumbnail cache with per-photo fetch and prefetch (closes #46)
check / check (push) Successful in 33s
Adds the on-disk content and thumbnail cache keyed by fileID: originals/ and thumbnails/ under cacheDirectory, present-means-complete (streaming atomic rename), orphan temp reaping on open. Photo.original/thumbnail return a cached path with no network when present, else fetch through the shared request pool; thumbnails.ensure drives the thumbnail pool with priority, dedup and AbortSignal. One shared RequestPools set serves both the ML fetch and the content cache. Content-hash integrity is deferred (#68); authenticated streaming decrypt guarantees integrity now.

Model: opus-4-8
2026-09-22 18:38:22 +02:00
clawbot 61dfec8d38 Revert ML search (#50) to restore a green next (build fix pending)
check / check (push) Successful in 18s
Reverts the #50 merge (224bd101): mlsearch.ts failed tsc in the CI build (make check does not run make build, so it slipped past review). next restored to green; #50 to be redone with make build in its gate.

Model: opus-4-8
2026-09-22 18:21:54 +02:00
clawbot 224bd101ab Content-similarity search surface over the CLIP index (closes #50)
check / check (push) Failing after 23s
Adds lib.mldata search over the CLIP index (#49): forFile returns a file's stored payload; similar ranks nearest files by cosine on the CLIP embedding; searchByEmbedding ranks the index against a caller-supplied query vector. All RAM-only, reusing the packed Float32Array index and id list. No text encoder is bundled — the caller provides the query embedding.

Model: opus-4-8
2026-09-22 18:14:13 +02:00
clawbot d7f415fe29 Fetch, store, and index per-file ML data (closes #49)
check / check (push) Successful in 14s
Adds the machine-learning (magic) data layer: fetches per-file ML payloads (face detections + CLIP embeddings) via the existing metadata-backup fetch through the metadata pool after each refresh, decrypts and gunzips them, and stores one mldata/<fileID>.json per file by rename (present-means-complete). A derived index (mldata/clip.f32 + clip.json) loads in one read and is rebuilt whenever it disagrees with the payloads on disk in either direction, so an interrupted backfill self-heals. Never in metadata.json; incremental on later refreshes; progress via onProgress/status.

Model: opus-4-8
2026-09-22 17:54:40 +02:00
clawbot c5c1f387df In-process read surface: albums, photos, and timeline grouping (closes #44)
check / check (push) Successful in 25s
Adds the in-process read surface, served from RAM with args-object signatures: albums (list/byName/byID), photos (byID/records -> plain PhotoRecord[]), thin Album/Photo wrappers, and timeline.groups (day/week/month) with a PhotoFilter (albumID/text/fileTypes/hasLocation/includeArchived; hidden excluded). Week keys use ISO YYYY-Www; each file appears once per group, newest first. Built on the #43 snapshot projection; no network.

Model: opus-4-8
2026-09-22 17:21:48 +02:00
clawbot 000d395c87 Three bounded request pools for metadata, content, and thumbnails (closes #45)
check / check (push) Successful in 55s
Adds three independent bounded request pools — metadata (10 in flight), content (5), thumbnails (25), each overridable — with on-demand-before-background priority and in-flight dedup (a shared key runs once); an idle pool never lends slots, and retries run inside a slot. Self-contained module; the content cache wires it into the library later.

Model: opus-4-8
2026-09-22 15:31:21 +02:00
clawbot fbb8ae44a7 Plain-record library snapshot and change subscription (closes #43)
check / check (push) Successful in 30s
Adds a synchronous, key-free snapshot() returning LibrarySnapshot (one PhotoRecord per fileID, deduped, newest first) with edited-name/time precedence (pubMagicMetadata over basic metadata) in milliseconds, plus AlbumRecord (favorites identified by type). subscribe({onChange}) delivers LibraryChange (changed/removed albums and files, refreshedAt) only when a refresh changes something; unsubscribe stops delivery. Built on the existing refresh loop; served from RAM, safe to send over IPC.

Model: opus-4-8
2026-09-22 15:22:52 +02:00
clawbot 7570055a5b Library.open with a transparent background refresh (closes #42)
check / check (push) Successful in 24s
Library.open loads metadata.json and serves reads from RAM: an empty cache awaits the first refresh, an existing cache returns at once and refreshes in the background so an unreachable server never stalls open(). A background timer refreshes every refreshIntervalSeconds (default 3), diffing only changed albums via the resumable cursor+tombstone enumerators and rewriting metadata.json only when something changed. A refresh failure is invisible to reads and surfaced via status()/onProgress; a failed save keeps status().lastError set and retries until one lands, so a stale disk is never masked. No sync()/refresh()/serverReachable surface; status() and close() included.

Model: opus-4-8
2026-09-22 14:52:02 +02:00
clawbot 72ea8dcb01 On-disk JSON metadata store for the local cache (closes #41)
check / check (push) Successful in 23s
Adds the metadata.json store: loads whole into RAM with id-lookup Maps, rewrites whole through the exported fsync atomic writer (temp, fsync, rename, dir fsync); a missing, unparseable, or wrong-schema file loads as empty (it is a cache); directory 0700, file 0600. No lock file, no public sync().

Model: opus-4-8
2026-09-22 12:52:50 +02:00