Commit Graph
1 Commits
Author SHA1 Message Date
sneak 6fe099e524 Fetch, store, and index per-file ML data and CLIP embeddings (closes #49)
check / check (push) Failing after 31s
Ente's "magic" search data (face detections + CLIP embeddings) is now
fetched, decrypted, and cached under cacheDirectory/mldata/, never in
metadata.json.

A new mldata-fetch module holds the /files/data/fetch (type mldata)
decrypt+gunzip, reused by both the metadata backup and the library.
MLDataStore writes one payload file per fileID by rename (present means
complete) and maintains a derived index: clip.json (fileIDs in order +
embedding length) and clip.f32 (embeddings packed as one Float32Array,
loaded in a single read). The index is rebuilt from the payloads when
missing or structurally inconsistent with the files present, and appended
to as payloads arrive; fetched.json records each file's fetch-time
updationTime for refetch decisions.

After each refresh the library fetches, through the #45 metadata pool, the
ML data for every known file absent from mldata/ or whose updationTime
advanced, reporting via onProgress (operation fetchMLData) and status().
RAM holds only the id list and Float32Array; payloads are read on demand.

Model: opus-4-8
2026-09-22 15:10:35 +00:00