ML data that does not decrypt is recorded with its reason (closes #198)

`fetchMLDataBatch` dropped an entry that did not decrypt, gunzip or parse,
so the file looked like one with no ML data and the run exited 0. It now
returns the reason beside the payloads, also for ML data that is not a
JSON object. The library gives the reasons from its last ML data fetch
that succeeded in `status().mlDataErrors` and the `fetchMLData` "done"
event; a fetch that fails leaves them as they were. `quak backup` writes
the reason as `mlDataError` and records the file in `failures.json`;
`backup-metadata` writes `mlDataError` and exits 1.

Judgement call: a file whose ML data is cached keeps it in its JSON when a
newer copy does not decode, as when the fetch fails.

Model: opus-5-5

src/metadata-backup.ts

@@ -2,6 +2,7 @@ import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
 import { join } from "node:path";
 import * as jpeg from "jpeg-js";
 import type { Client } from "./client.js";
+import { describeError } from "./errors.js";
 import { readExifTags } from "./exif.js";
 import type { Library, Photo } from "./library/index.js";
 import { sanitizeFileName } from "./filename.js";
@@ -91,14 +92,15 @@ const extractExif = async (
 // of plain JSON: account, per-collection, and per-file records including the
 // private and public magic metadata and (by default) the ML data. Collections
 // and files are enumerated from the library's cache, which the caller refreshes
-// first. Returns how many ML data requests failed; their files are still
+// first. Returns how many files have no ML data because their ML data request
+// failed or their ML data did not decrypt, gunzip or parse; they are still
 // written, with `mlDataError` in place of `mlData`.
 export const runMetadataBackup = async (
     lib: Library,
     client: Client,
     outDir: string,
     opts?: MetadataBackupOptions,
-): Promise<{ failedMLBatches: number }> => {
+): Promise<{ filesWithMLDataError: number }> => {
     const log = opts?.onProgress ?? (() => {});
     const wantExif = opts?.exif ?? false;
 
@@ -164,24 +166,28 @@ export const runMetadataBackup = async (
     }
 
     // One failed request (retries exhausted) must not end the dump: its files
-    // get the reason in `mlDataError` and the other batches go on.
+    // get the reason in `mlDataError` and the other batches go on. So does a
+    // file whose ML data does not decrypt, gunzip or parse.
     log("Fetching ML data (face detections, CLIP embeddings)...");
     const mlDataMap = new Map<number, MLData>();
     const mlDataErrors = new Map<number, string>();
-    let failedMLBatches = 0;
     const fileIDs = [...fileKeys.keys()];
     for (let i = 0; i < fileIDs.length; i += MLDATA_BATCH_SIZE) {
         const batch = fileIDs.slice(i, i + MLDATA_BATCH_SIZE);
         try {
-            const result = await fetchMLDataBatch(
+            const { payloads, errors } = await fetchMLDataBatch(
                 client.getApiClient(),
                 batch,
                 fileKeys,
             );
-            for (const [id, payload] of result) mlDataMap.set(id, payload);
+            for (const [id, payload] of payloads) mlDataMap.set(id, payload);
+            for (const [id, err] of errors) {
+                const reason = describeError(err);
+                log(`ML data of file ${id} could not be read: ${reason}`);
+                mlDataErrors.set(id, reason);
+            }
         } catch (err) {
             const reason = err instanceof Error ? err.message : String(err);
-            failedMLBatches++;
             log(
                 `ML data request for ${batch.length} file(s) failed: ${reason}`,
             );
@@ -229,5 +235,5 @@ export const runMetadataBackup = async (
     }
 
     log("Metadata backup complete.");
-    return { failedMLBatches };
+    return { filesWithMLDataError: mlDataErrors.size };
 };