3 Commits

Include renames

Author	SHA1	Message	Date
clawbot	7ec71bf7ec	ML data that does not decrypt is recorded with its reason (closes #198) ... `fetchMLDataBatch` dropped an entry that did not decrypt, gunzip or parse, so the file looked like one with no ML data and the run exited 0. It now returns the reason beside the payloads, also for ML data that is not a JSON object. The library gives the reasons from its last ML data fetch that succeeded in `status().mlDataErrors` and the `fetchMLData` "done" event; a fetch that fails leaves them as they were. `quak backup` writes the reason as `mlDataError` and records the file in `failures.json`; `backup-metadata` writes `mlDataError` and exits 1. Judgement call: a file whose ML data is cached keeps it in its JSON when a newer copy does not decode, as when the fetch fails. Model: opus-5-5	2026-10-16 07:23:16 +02:00
clawbot	bf3b20df2f	backup-metadata: keep going when an ML data request fails (closes #101) ... Each ML data request of up to 200 files is now tried on its own. A request that still fails after its retries is logged, its files are written with the reason in `mlDataError`, and the command exits 1 once the dump is complete. `fetchMLData`, used only here, is removed in favour of a per-batch loop over `fetchMLDataBatch`. Model: opus-5-5	2026-09-23 05:49:38 +02:00
clawbot	d7f415fe29	Fetch, store, and index per-file ML data (closes #49) ... Adds the machine-learning (magic) data layer: fetches per-file ML payloads (face detections + CLIP embeddings) via the existing metadata-backup fetch through the metadata pool after each refresh, decrypts and gunzips them, and stores one mldata/<fileID>.json per file by rename (present-means-complete). A derived index (mldata/clip.f32 + clip.json) loads in one read and is rebuilt whenever it disagrees with the payloads on disk in either direction, so an interrupted backfill self-heals. Never in metadata.json; incremental on later refreshes; progress via onProgress/status. Model: opus-4-8	2026-09-22 17:54:40 +02:00