Ente's "magic" search data (face detections + CLIP embeddings) is now
fetched, decrypted, and cached under cacheDirectory/mldata/, never in
metadata.json.
A new mldata-fetch module holds the /files/data/fetch (type mldata)
decrypt+gunzip, reused by both the metadata backup and the library.
MLDataStore writes one payload file per fileID by rename (present means
complete) and maintains a derived index: clip.json (fileIDs in order +
embedding length) and clip.f32 (embeddings packed as one Float32Array,
loaded in a single read). The index is rebuilt from the payloads when
missing or structurally inconsistent with the files present, and appended
to as payloads arrive; fetched.json records each file's fetch-time
updationTime for refetch decisions.
After each refresh the library fetches, through the #45 metadata pool, the
ML data for every known file absent from mldata/ or whose updationTime
advanced, reporting via onProgress (operation fetchMLData) and status().
RAM holds only the id list and Float32Array; payloads are read on demand.
Model: opus-4-8
Adds the in-process read surface, served from RAM with args-object signatures: albums (list/byName/byID), photos (byID/records -> plain PhotoRecord[]), thin Album/Photo wrappers, and timeline.groups (day/week/month) with a PhotoFilter (albumID/text/fileTypes/hasLocation/includeArchived; hidden excluded). Week keys use ISO YYYY-Www; each file appears once per group, newest first. Built on the #43 snapshot projection; no network.
Model: opus-4-8
Adds three independent bounded request pools — metadata (10 in flight), content (5), thumbnails (25), each overridable — with on-demand-before-background priority and in-flight dedup (a shared key runs once); an idle pool never lends slots, and retries run inside a slot. Self-contained module; the content cache wires it into the library later.
Model: opus-4-8
Adds a synchronous, key-free snapshot() returning LibrarySnapshot (one PhotoRecord per fileID, deduped, newest first) with edited-name/time precedence (pubMagicMetadata over basic metadata) in milliseconds, plus AlbumRecord (favorites identified by type). subscribe({onChange}) delivers LibraryChange (changed/removed albums and files, refreshedAt) only when a refresh changes something; unsubscribe stops delivery. Built on the existing refresh loop; served from RAM, safe to send over IPC.
Model: opus-4-8
Library.open loads metadata.json and serves reads from RAM: an empty cache awaits the first refresh, an existing cache returns at once and refreshes in the background so an unreachable server never stalls open(). A background timer refreshes every refreshIntervalSeconds (default 3), diffing only changed albums via the resumable cursor+tombstone enumerators and rewriting metadata.json only when something changed. A refresh failure is invisible to reads and surfaced via status()/onProgress; a failed save keeps status().lastError set and retries until one lands, so a stale disk is never masked. No sync()/refresh()/serverReachable surface; status() and close() included.
Model: opus-4-8
Adds the metadata.json store: loads whole into RAM with id-lookup Maps, rewrites whole through the exported fsync atomic writer (temp, fsync, rename, dir fsync); a missing, unparseable, or wrong-schema file loads as empty (it is a cache); directory 0700, file 0600. No lock file, no public sync().
Model: opus-4-8
sneak
clawbot