Give backup album folders distinct names and remove stale entries (closes #103)

Within a collection's folder, files whose sanitized titles match (ignoring
case) each get their file ID added before the extension, and collections
whose sanitized names match get their ID added, so no symlink or JSON
replaces another. Names are chosen across all collections, so a scoped run
names folders the same as a full one.

Each run first removes symlinks into originals/ that no longer belong to a
collection, and the folders quak wrote (a sibling JSON with an album ID) for
collections that are gone or renamed. Anything else is left alone; a folder
still holding user files keeps its JSON.

Model: opus-5-5

README.md

@@ -471,6 +471,11 @@ accepted for backward compatibility but ignored. `backup-metadata --exif` (alias
 metadata. The listing and backup commands support `--json` for machine-readable
 output.
 
+`backup-metadata` fetches ML data in requests of up to 200 files. When a request
+still fails after its retries, the error is logged, each of its files is written
+with the reason in an `mlDataError` field instead of `mlData`, and the dump goes
+on. The exit code is non-zero if any ML data request failed.
+
 `helper fix-missing-thumbnails` regenerates thumbnails for baseline JPEG images
 only, because the bundled decoder (`jpeg-js`) decodes only JPEG. A non-JPEG
 image (PNG, HEIC) or a video is reported as `skipped` (unsupported format), kept
@@ -492,6 +497,20 @@ failure still exits non-zero.
         <name>.json             collection metadata + file list
 ```
 
+A collection's directory and JSON are named after the collection, and a symlink
+after the file's title, both with unsafe characters replaced. When two
+collections would get the same name, or two files in one collection the same
+title (ignoring case in both), each of them gets its ID added: two albums named
+`Trip` become `Trip (10)/` and `Trip (11)/`, and two files titled `IMG_0001.JPG`
+become `IMG_0001 (12345).JPG` and `IMG_0001 (12346).JPG`. IDs never change, so a
+name stays the same from run to run until such a clash appears or goes away.
+
+Each run removes the symlinks into `originals/` that no longer belong in their
+collection's directory, and the directories (and JSON) of collections that were
+deleted or renamed. Nothing else in `collections/` is touched: a file or a
+symlink you put there stays, and a directory that still holds one after its
+symlinks are removed stays too, with its JSON.
+
 Each file is downloaded exactly once regardless of how many collections it
 appears in. On subsequent runs, existing originals are skipped. If a download
 fails, the error is logged and the backup continues with the next file. The exit
@@ -676,6 +695,11 @@ Under `cacheDirectory`:
         fetched.json              per-file fetch bookkeeping
 ```
 
+When `metadata.json` belongs to a different account than the client's,
+`Library.open` deletes it and `mldata/` and starts from an empty cache. Cached
+originals and thumbnails are kept; they are reached only through the files the
+current account's records name.
+
 A stored file appears only via an atomic temp-then-rename, so its presence means
 it is complete. The design also calls for a content-hash comparison against
 `FileMetadata.hash` on each fetched original; that check is deferred (issue

TODO.md

@@ -18,6 +18,24 @@ Tag v1.0.0.
 
 # Completed Steps
 
+- 2026-09-23: Fixed the backup's per-collection folders (issue 103). Two files
+  in one collection with the same title, and two collections with the same name,
+  each get their ID added to the name (`IMG_0001 (12345).JPG`, `Trip (10)/`), so
+  none replaces another's symlink or JSON. Each run removes symlinks into
+  `originals/` for files no longer in the collection, and the folders of deleted
+  or renamed collections, leaving anything else in `collections/` alone. The
+  README backup layout states the naming rule.
+- 2026-09-23: Kept one account's cache from mixing with another's (issue 104).
+  When `metadata.json` in the cache directory was written for a different,
+  non-zero user ID than the client's, `Library.open` deletes it and `mldata/`
+  and starts empty, so the first refresh enumerates from 0. This only happens
+  with `--cache-dir` or an explicit `cacheDirectory`; the default path already
+  includes the user ID. A test opens one account's cache as another account.
+- 2026-09-23: `backup-metadata` no longer stops on one failed ML data request
+  (issue 101). Each request of up to 200 files is tried on its own; a failed one
+  is logged, its files are written with the reason in `mlDataError`, and the
+  command exits 1 once the dump is complete. `fetchMLData`, which only this
+  command used, is gone; the command calls `fetchMLDataBatch` per batch.
 - 2026-09-23: Single-sourced the version string (issue 5). `package.json` is the
   only place it is written: `src/index.ts` imports it for `VERSION` and
   `bin/quak.ts` passes `VERSION` to commander. tsc copies `package.json` to

src/backup.ts

@@ -17,7 +17,9 @@
 // temp-then-rename, so a file that exists is whole and is never re-fetched — an
 // interrupted run resumes by listing the directory. The derived views hold no
 // unique state, so they are rebuilt every run; that repairs stale sidecars and
-// missing or broken symlinks left by an earlier crash.
+// missing or broken symlinks left by an earlier crash. A rebuild also removes
+// the symlinks into originals/ that no longer belong to an album, and the
+// directories of albums that no longer exist.
 //
 // Resilience (issue #8): no per-file condition aborts the run. A failed
 // download or a failed symlink is caught, recorded in `failures.json` with a
@@ -34,13 +36,14 @@ import {
     readdirSync,
     readFileSync,
     readlinkSync,
+    rmdirSync,
     rmSync,
     statSync,
     symlinkSync,
     writeFileSync,
 } from "node:fs";
 import { copyFile, rename, rm } from "node:fs/promises";
-import { basename, dirname, join, relative } from "node:path";
+import { basename, dirname, extname, join, relative } from "node:path";
 
 import { fsyncPath } from "./download/index.js";
 import { safeExtension, sanitizeFileName } from "./filename.js";
@@ -227,6 +230,83 @@ const rebuildSymlink = (linkPath: string, target: string): void => {
     symlinkSync(target, linkPath);
 };
 
+// The on-disk names for the entries of one directory, keyed by ID. Each name
+// is used as is unless another entry has the same name, ignoring case (two
+// names that differ only in case are one entry on a case-insensitive file
+// system); then every entry sharing it gets ` (<id>)`, before the extension
+// when `beforeExtension` is set. IDs are stable, so the names are too.
+const namesByID = (
+    entries: { id: number; name: string }[],
+    beforeExtension: boolean,
+): Map<number, string> => {
+    const counts = new Map<string, number>();
+    for (const { name } of entries) {
+        const key = name.toLowerCase();
+        counts.set(key, (counts.get(key) ?? 0) + 1);
+    }
+    const names = new Map<number, string>();
+    for (const { id, name } of entries) {
+        if (counts.get(name.toLowerCase()) === 1) {
+            names.set(id, name);
+            continue;
+        }
+        const ext = beforeExtension ? extname(name) : "";
+        const stem = name.slice(0, name.length - ext.length);
+        names.set(id, `${stem} (${id})${ext}`);
+    }
+    return names;
+};
+
+// Remove the symlinks in the album directory `dir` that point into
+// `originalsDir` and are not named in `keep`. Nothing else in the directory
+// is touched: anything else there was put there by the user.
+const removeStaleLinks = (
+    dir: string,
+    keep: Set<string>,
+    originalsDir: string,
+): void => {
+    const target = relative(dir, originalsDir);
+    for (const name of readdirSync(dir)) {
+        if (keep.has(name)) continue;
+        const path = join(dir, name);
+        if (
+            lstatSync(path).isSymbolicLink() &&
+            dirname(readlinkSync(path)) === target
+        ) {
+            rmSync(path);
+        }
+    }
+};
+
+// Remove the directories under `collectionsDir` that an earlier run wrote for
+// an album that is gone or renamed: a directory not named in `current` with a
+// `<name>.json` beside it holding an album ID, which is what a run writes. Its
+// symlinks into originals/ are removed; if that leaves it empty, it and its
+// JSON are deleted, otherwise both stay for what the user put there.
+const removeStaleAlbumDirs = (
+    collectionsDir: string,
+    current: Set<string>,
+    originalsDir: string,
+): void => {
+    for (const entry of readdirSync(collectionsDir, { withFileTypes: true })) {
+        if (!entry.isDirectory() || current.has(entry.name)) continue;
+        const jsonPath = join(collectionsDir, `${entry.name}.json`);
+        try {
+            const album = JSON.parse(readFileSync(jsonPath, "utf-8")) as {
+                id?: unknown;
+            };
+            if (typeof album.id !== "number") continue;
+        } catch {
+            continue;
+        }
+        const dir = join(collectionsDir, entry.name);
+        removeStaleLinks(dir, new Set(), originalsDir);
+        if (readdirSync(dir).length > 0) continue;
+        rmdirSync(dir);
+        rmSync(jsonPath);
+    }
+};
+
 const loadLedger = (path: string): Map<number, FailureEntry> => {
     const ledger = new Map<number, FailureEntry>();
     try {
@@ -303,9 +383,10 @@ export const runBackup = async (
 
     // Collections in scope, and the distinct files across them (a file shared
     // by two albums is one original).
-    const collections = lib
-        .listCollections()
-        .filter((c) => (only ? only.has(c.name) : true));
+    const allCollections = lib.listCollections();
+    const collections = allCollections.filter((c) =>
+        only ? only.has(c.name) : true,
+    );
     const collectionName = new Map<number, string>();
     for (const c of collections) collectionName.set(c.id, c.name);
 
@@ -406,23 +487,55 @@ export const runBackup = async (
         }
     }
 
-    // Then the per-collection symlink trees and JSON.
+    // Then the per-collection symlink trees and JSON. Directory names are
+    // chosen across every album, not just those in scope, so a scoped run
+    // names an album the same as a full one and never takes the directory of
+    // an album it skipped. Stale entries are removed before anything is
+    // rebuilt, so on a case-insensitive file system removing an old name can
+    // never remove the new one.
+    const albumDirNames = namesByID(
+        allCollections.map((c) => ({
+            id: c.id,
+            name: sanitizeFileName(c.name, `collection-${c.id}`),
+        })),
+        false,
+    );
+    try {
+        removeStaleAlbumDirs(
+            collectionsDir,
+            new Set(albumDirNames.values()),
+            originalsDir,
+        );
+    } catch (err) {
+        log(`FAILED removing old album directories: ${errorMessage(err)}`);
+    }
+
     for (const c of collections) {
-        const colDirName = sanitizeFileName(c.name, `collection-${c.id}`);
+        const colDirName = albumDirNames.get(c.id)!;
         const colDir = join(collectionsDir, colDirName);
         mkdirSync(colDir, { recursive: true });
 
         const files = filesByCollection.get(c.id) ?? [];
+        const linkNames = namesByID(
+            files.map((f) => ({
+                id: f.id,
+                name: sanitizeFileName(f.metadata.title, `file-${f.id}`),
+            })),
+            true,
+        );
+        try {
+            removeStaleLinks(colDir, new Set(linkNames.values()), originalsDir);
+        } catch (err) {
+            log(`FAILED removing old links in ${c.name}: ${errorMessage(err)}`);
+        }
+
         const metaFiles: { id: number; metadata: EnteFile["metadata"] }[] = [];
         for (const file of files) {
             metaFiles.push({ id: file.id, metadata: file.metadata });
             if (!includeOriginals) continue;
             const orig = join(originalsDir, originalName(file));
             if (!isPresent(orig)) continue;
-            const linkName = sanitizeFileName(
-                file.metadata.title,
-                `file-${file.id}`,
-            );
+            const linkName = linkNames.get(file.id)!;
             const linkPath = join(colDir, linkName);
             try {
                 rebuildSymlink(linkPath, relative(colDir, orig));

src/cli-commands.ts

@@ -323,11 +323,11 @@ export const backupMetadataCommand = async (
     if (!client) return 1;
     const lib = await openReadLibrary(ctx, client);
     try {
-        await runMetadataBackup(lib, client, dir, {
+        const { failedMLBatches } = await runMetadataBackup(lib, client, dir, {
             exif: opts.exif || opts.all,
             onProgress: (msg) => ctx.stderr.write(msg + "\n"),
         });
-        return 0;
+        return failedMLBatches > 0 ? 1 : 0;
     } finally {
         await lib.close();
     }

src/library/index.ts

@@ -26,6 +26,7 @@
 // store marked unsaved until a later save actually lands, so a stuck disk is
 // never masked by a subsequent empty refresh.
 
+import { rm } from "node:fs/promises";
 import { join } from "node:path";
 import envPaths from "env-paths";
 
@@ -345,9 +346,20 @@ export class Library {
         const cacheDirectory =
             opts.cacheDirectory ??
             join(envPaths("quak", { suffix: "" }).cache, String(userID));
-        const store = await MetadataStore.load(
-            join(cacheDirectory, "metadata.json"),
-        );
+        const metadataPath = join(cacheDirectory, "metadata.json");
+        let store = await MetadataStore.load(metadataPath);
+        // A cache directory given explicitly can hold another account's cache.
+        // Its records and cursor are not this account's, so delete it and the
+        // ML data beside it and start empty. A user ID of 0 means the cache
+        // was never refreshed and so holds nothing to discard.
+        if (store.userID !== 0 && store.userID !== userID) {
+            await rm(metadataPath, { force: true });
+            await rm(join(cacheDirectory, "mldata"), {
+                recursive: true,
+                force: true,
+            });
+            store = await MetadataStore.load(metadataPath);
+        }
         const intervalMs =
             (opts.refreshIntervalSeconds ?? DEFAULT_REFRESH_INTERVAL_SECONDS) *
             1000;

src/metadata-backup.ts

@@ -5,7 +5,11 @@ import exifReader from "exif-reader";
 import type { Client } from "./client.js";
 import type { Library, Photo } from "./library/index.js";
 import { sanitizeFileName } from "./filename.js";
-import { fetchMLData } from "./mldata-fetch.js";
+import {
+    fetchMLDataBatch,
+    MLDATA_BATCH_SIZE,
+    type MLData,
+} from "./mldata-fetch.js";
 import type { EnteFile } from "./model/types.js";
 
 export type ProgressCallback = (message: string) => void;
@@ -137,13 +141,14 @@ const extractExif = async (
 // of plain JSON: account, per-collection, and per-file records including the
 // private and public magic metadata and (by default) the ML data. Collections
 // and files are enumerated from the library's cache rather than a fresh server
-// scan; the ML fetch and EXIF extraction are unchanged.
+// scan. Returns how many ML data requests failed; their files are still
+// written, with `mlDataError` in place of `mlData`.
 export const runMetadataBackup = async (
     lib: Library,
     client: Client,
     outDir: string,
     opts?: MetadataBackupOptions,
-): Promise<void> => {
+): Promise<{ failedMLBatches: number }> => {
     const log = opts?.onProgress ?? (() => {});
     const wantExif = opts?.exif ?? false;
 
@@ -208,12 +213,31 @@ export const runMetadataBackup = async (
         }
     }
 
+    // One failed request (retries exhausted) must not end the dump: its files
+    // get the reason in `mlDataError` and the other batches go on.
     log("Fetching ML data (face detections, CLIP embeddings)...");
-    const mlDataMap = await fetchMLData(
-        client.getApiClient(),
-        [...fileKeys.keys()],
-        fileKeys,
-    );
+    const mlDataMap = new Map<number, MLData>();
+    const mlDataErrors = new Map<number, string>();
+    let failedMLBatches = 0;
+    const fileIDs = [...fileKeys.keys()];
+    for (let i = 0; i < fileIDs.length; i += MLDATA_BATCH_SIZE) {
+        const batch = fileIDs.slice(i, i + MLDATA_BATCH_SIZE);
+        try {
+            const result = await fetchMLDataBatch(
+                client.getApiClient(),
+                batch,
+                fileKeys,
+            );
+            for (const [id, payload] of result) mlDataMap.set(id, payload);
+        } catch (err) {
+            const reason = err instanceof Error ? err.message : String(err);
+            failedMLBatches++;
+            log(
+                `ML data request for ${batch.length} file(s) failed: ${reason}`,
+            );
+            for (const id of batch) mlDataErrors.set(id, reason);
+        }
+    }
     log(`Got ML data for ${mlDataMap.size} file(s)`);
 
     const writtenFileIDs = new Set<number>();
@@ -233,6 +257,8 @@ export const runMetadataBackup = async (
 
         const ml = mlDataMap.get(file.id);
         if (ml) fileMeta.mlData = ml;
+        const mlError = mlDataErrors.get(file.id);
+        if (mlError) fileMeta.mlDataError = mlError;
 
         if (wantExif && !writtenFileIDs.has(file.id)) {
             log(`[${file.metadata.title}] Extracting EXIF...`);
@@ -253,4 +279,5 @@ export const runMetadataBackup = async (
     }
 
     log("Metadata backup complete.");
+    return { failedMLBatches };
 };

src/mldata-fetch.ts

@@ -5,9 +5,8 @@
 // comes back encrypted under the file's own key and gzipped; decrypting and
 // gunzipping yields the JSON payload
 // `{ face: { faces: [...] }, clip: { embedding } }`. Ente caps a request at 200
-// ids, so `fetchMLData` batches for callers that want many at once while
-// `fetchMLDataBatch` is the single-request unit the library submits to its
-// request pool.
+// ids, so callers that want many at once split them into batches of
+// `MLDATA_BATCH_SIZE` and call `fetchMLDataBatch` once per batch.
 
 import { gunzipSync } from "node:zlib";
 
@@ -69,25 +68,3 @@ export const fetchMLDataBatch = async (
     }
     return result;
 };
-
-// Fetch ML data for arbitrarily many ids, batching at `MLDATA_BATCH_SIZE`. Used
-// by the one-shot metadata backup; the library fetches through its request pool
-// with `fetchMLDataBatch` instead.
-export const fetchMLData = async (
-    api: ApiClient,
-    fileIDs: number[],
-    fileKeys: Map<number, Uint8Array>,
-): Promise<Map<number, MLData>> => {
-    const result = new Map<number, MLData>();
-    for (let i = 0; i < fileIDs.length; i += MLDATA_BATCH_SIZE) {
-        const batch = fileIDs.slice(i, i + MLDATA_BATCH_SIZE);
-        for (const [id, payload] of await fetchMLDataBatch(
-            api,
-            batch,
-            fileKeys,
-        )) {
-            result.set(id, payload);
-        }
-    }
-    return result;
-};

test/cli/backup.test.ts

@@ -40,6 +40,7 @@ import {
     readFileSync,
     readlinkSync,
     rmSync,
+    symlinkSync,
     writeFileSync,
 } from "node:fs";
 import { spawnSync } from "node:child_process";
@@ -47,6 +48,7 @@ import { join } from "node:path";
 import { tmpdir } from "node:os";
 import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
 
+import { runBackup, type BackupLibrary } from "../../src/backup.js";
 import { Library } from "../../src/library/index.js";
 import type { ContentSource } from "../../src/library/content.js";
 import type { CollectionsPage, FilesPage } from "../../src/client.js";
@@ -622,3 +624,219 @@ describe("lib.backup", () => {
         lib.close();
     });
 });
+
+// The album folders under collections/, driven through `runBackup` with a
+// stand-in library whose albums a test changes between runs.
+describe("backup album folders", () => {
+    interface Album {
+        collection: Collection;
+        files: EnteFile[];
+    }
+
+    const libraryOf = (albums: Album[]): BackupLibrary => ({
+        refresh: async () => {},
+        listCollections: () => albums.map((a) => a.collection),
+        listFiles: (id) =>
+            albums.find((a) => a.collection.id === id)?.files ?? [],
+        original: async (fileID) => {
+            const path = join(root, `source-${fileID}`);
+            writeFileSync(path, `original ${fileID}`);
+            return { path };
+        },
+        thumbnail: async () => {
+            throw new Error("no thumbnails in this stand-in");
+        },
+    });
+
+    // Every entry under collections/, one level of directories deep, with each
+    // symlink's target.
+    const tree = (outDir: string): string[] => {
+        const lines: string[] = [];
+        const list = (dir: string, prefix: string): void => {
+            for (const name of readdirSync(dir).sort()) {
+                const path = join(dir, name);
+                const st = lstatSync(path);
+                if (st.isSymbolicLink()) {
+                    lines.push(`${prefix}${name} -> ${readlinkSync(path)}`);
+                } else if (st.isDirectory() && prefix === "") {
+                    lines.push(`${name}/`);
+                    list(path, `${name}/`);
+                } else {
+                    lines.push(`${prefix}${name}`);
+                }
+            }
+        };
+        list(join(outDir, "collections"), "");
+        return lines;
+    };
+
+    const albumID = (outDir: string, jsonName: string): number =>
+        JSON.parse(readFileSync(join(outDir, "collections", jsonName), "utf-8"))
+            .id;
+
+    it("gives every file and every album its own name when names repeat", async () => {
+        const outDir = join(root, "backup");
+        const lib = libraryOf([
+            {
+                collection: collection(10, "Trip"),
+                files: [
+                    file(1, 10, "IMG_0001.JPG"),
+                    file(2, 10, "IMG_0001.JPG"),
+                    file(4, 10, "img_0001.jpg"),
+                    file(3, 10, "other.jpg"),
+                ],
+            },
+            {
+                collection: collection(11, "Trip"),
+                files: [file(3, 11, "other.jpg")],
+            },
+        ]);
+
+        const result = await runBackup(lib, { downloadDirectory: outDir });
+
+        expect(result.failed).toBe(0);
+        expect(tree(outDir)).toEqual([
+            "Trip (10)/",
+            "Trip (10)/IMG_0001 (1).JPG -> ../../originals/1.JPG",
+            "Trip (10)/IMG_0001 (2).JPG -> ../../originals/2.JPG",
+            "Trip (10)/img_0001 (4).jpg -> ../../originals/4.jpg",
+            "Trip (10)/other.jpg -> ../../originals/3.jpg",
+            "Trip (10).json",
+            "Trip (11)/",
+            "Trip (11)/other.jpg -> ../../originals/3.jpg",
+            "Trip (11).json",
+        ]);
+        expect(albumID(outDir, "Trip (10).json")).toBe(10);
+        expect(albumID(outDir, "Trip (11).json")).toBe(11);
+    });
+
+    it("changes nothing on a second run over an unchanged account", async () => {
+        const outDir = join(root, "backup");
+        const lib = libraryOf([
+            {
+                collection: collection(10, "Trip"),
+                files: [
+                    file(1, 10, "IMG_0001.JPG"),
+                    file(2, 10, "IMG_0001.JPG"),
+                ],
+            },
+            {
+                collection: collection(11, "Trip"),
+                files: [file(3, 11, "other.jpg")],
+            },
+        ]);
+
+        await runBackup(lib, { downloadDirectory: outDir });
+        const before = tree(outDir);
+        const second = await runBackup(lib, { downloadDirectory: outDir });
+
+        expect(second.downloaded).toBe(0);
+        expect(second.failed).toBe(0);
+        expect(tree(outDir)).toEqual(before);
+    });
+
+    it("leaves the albums an onlyAlbumNames run skips as they were", async () => {
+        const outDir = join(root, "backup");
+        // "trip" is skipped by the scoped run but its name clashes with the
+        // in-scope "Trip", so "Trip" must keep its ID suffix.
+        const lib = libraryOf([
+            {
+                collection: collection(10, "Trip"),
+                files: [file(1, 10, "a.jpg")],
+            },
+            {
+                collection: collection(11, "trip"),
+                files: [file(2, 11, "b.jpg")],
+            },
+            {
+                collection: collection(12, "Work"),
+                files: [file(3, 12, "c.jpg")],
+            },
+        ]);
+        const json = (name: string): string =>
+            readFileSync(join(outDir, "collections", name), "utf-8");
+
+        await runBackup(lib, { downloadDirectory: outDir });
+        const before = tree(outDir);
+        const skippedJSON = [json("trip (11).json"), json("Work.json")];
+        const scoped = await runBackup(lib, {
+            downloadDirectory: outDir,
+            onlyAlbumNames: ["Trip"],
+        });
+
+        expect(scoped.failed).toBe(0);
+        expect(before).toEqual([
+            "Trip (10)/",
+            "Trip (10)/a.jpg -> ../../originals/1.jpg",
+            "Trip (10).json",
+            "Work/",
+            "Work/c.jpg -> ../../originals/3.jpg",
+            "Work.json",
+            "trip (11)/",
+            "trip (11)/b.jpg -> ../../originals/2.jpg",
+            "trip (11).json",
+        ]);
+        expect(tree(outDir)).toEqual(before);
+        expect([json("trip (11).json"), json("Work.json")]).toEqual(
+            skippedJSON,
+        );
+    });
+
+    it("removes links and album folders that are gone, and nothing the user added", async () => {
+        const outDir = join(root, "backup");
+        const albums: Album[] = [
+            {
+                collection: collection(10, "Trip"),
+                files: [
+                    file(1, 10, "IMG_0001.JPG"),
+                    file(2, 10, "IMG_0001.JPG"),
+                    file(3, 10, "other.jpg"),
+                ],
+            },
+            {
+                collection: collection(12, "Work"),
+                files: [file(5, 12, "a.jpg")],
+            },
+            {
+                collection: collection(13, "Old"),
+                files: [file(5, 13, "a.jpg")],
+            },
+        ];
+        const lib = libraryOf(albums);
+        await runBackup(lib, { downloadDirectory: outDir });
+
+        // What the user put in the tree: a note and a symlink of their own in
+        // an album, a note in an album about to be renamed, and a folder quak
+        // did not create.
+        const collectionsDir = join(outDir, "collections");
+        writeFileSync(join(collectionsDir, "Trip", "notes.txt"), "mine");
+        symlinkSync("../elsewhere", join(collectionsDir, "Trip", "mine"));
+        writeFileSync(join(collectionsDir, "Work", "keep.txt"), "mine");
+        mkdirSync(join(collectionsDir, "Mine"));
+        writeFileSync(join(collectionsDir, "Mine", "keep.txt"), "mine");
+
+        // File 2 leaves Trip, Work is renamed Office, Old is deleted.
+        albums[0]!.files.splice(1, 1);
+        albums[1]!.collection = collection(12, "Office");
+        albums.splice(2, 1);
+        const result = await runBackup(lib, { downloadDirectory: outDir });
+
+        expect(result.failed).toBe(0);
+        expect(tree(outDir)).toEqual([
+            "Mine/",
+            "Mine/keep.txt",
+            "Office/",
+            "Office/a.jpg -> ../../originals/5.jpg",
+            "Office.json",
+            "Trip/",
+            "Trip/IMG_0001.JPG -> ../../originals/1.JPG",
+            "Trip/mine -> ../elsewhere",
+            "Trip/notes.txt",
+            "Trip/other.jpg -> ../../originals/3.jpg",
+            "Trip.json",
+            "Work/",
+            "Work/keep.txt",
+            "Work.json",
+        ]);
+    });
+});

test/cli/metadata-backup.test.ts

@@ -38,7 +38,7 @@ import { join } from "node:path";
 import { tmpdir } from "node:os";
 import sodium from "libsodium-wrappers-sumo";
 import { SRP, SrpServer } from "fast-srp-hap";
-import { beforeAll, afterAll, describe, expect, it } from "vitest";
+import { beforeAll, afterAll, describe, expect, it, vi } from "vitest";
 import {
     init,
     toBase64,
@@ -53,8 +53,16 @@ import {
     runMetadataBackup,
     type MetadataBackupOptions,
 } from "../../src/metadata-backup.js";
+import { backupMetadataCommand } from "../../src/cli-commands.js";
 import type { KeyAttributes } from "../../src/auth/types.js";
 
+// One file per ML data request, so the two files of the mock account are
+// fetched in two requests and one of them can fail on its own.
+vi.mock("../../src/mldata-fetch.js", async (importOriginal) => ({
+    ...(await importOriginal<typeof import("../../src/mldata-fetch.js")>()),
+    MLDATA_BATCH_SIZE: 1,
+}));
+
 const TEST_EMAIL = "metabackup@example.com";
 const TEST_PASSWORD = "metapass";
 const TEST_OPS = 2;
@@ -347,7 +355,8 @@ const buildMetaMock = async (): Promise<MetaMockState> => {
     };
 };
 
-const buildMetaFetch = (m: MetaMockState) => {
+// `failMLDataFor`: answer 500 to every ML data request that asks for this file.
+const buildMetaFetch = (m: MetaMockState, failMLDataFor?: number) => {
     let srpServer: SrpServer;
     return (async (
         input: RequestInfo | URL,
@@ -403,6 +412,8 @@ const buildMetaFetch = (m: MetaMockState) => {
         }
         if (path === "/files/data/fetch") {
             const body = JSON.parse(init?.body as string);
+            if ((body.fileIDs as number[]).includes(failMLDataFor!))
+                return new Response("server error", { status: 500 });
             const data = (body.fileIDs as number[])
                 .filter((id: number) => m.encryptedMLData[id])
                 .map((id: number) => ({
@@ -636,3 +647,63 @@ describe("quak backup-metadata", () => {
         expect(failedMeta.imageMetadataError).toEqual(expect.any(String));
     });
 });
+
+describe("quak backup-metadata when an ML data request fails", () => {
+    // Run the CLI command against the mock and return its exit code, stderr
+    // and output directory.
+    const runCommand = async (failMLDataFor?: number) => {
+        const client = await Client.login({
+            email: TEST_EMAIL,
+            password: TEST_PASSWORD,
+            apiOptions: {
+                fetch: buildMetaFetch(mock, failMLDataFor),
+                retry: { sleep: async () => {} },
+            },
+        });
+        const outDir = mkdtempSync(join(testDir, "ml-fail-"));
+        let stderr = "";
+        const code = await backupMetadataCommand(
+            {
+                stdout: { write: () => true },
+                stderr: { write: (text: string) => (stderr += text) },
+                sessionDir: testDir,
+                cacheDir: mkdtempSync(join(testDir, "cache-")),
+                loadSession: () => client,
+            },
+            outDir,
+            {},
+        );
+        return { code, stderr, outDir };
+    };
+
+    it("writes every file, marks the failed batch's files, and exits 1", async () => {
+        const { code, stderr, outDir } = await runCommand(200);
+
+        expect(code).toBe(1);
+        expect(stderr).toContain("ML data request for 1 file(s) failed");
+
+        const ok = JSON.parse(
+            readFileSync(
+                join(outDir, "collections", "10-Vacation", "100.json"),
+                "utf-8",
+            ),
+        );
+        expect(ok.mlData.clip.embedding).toEqual([0.5, 0.6, 0.7]);
+        expect(ok.mlDataError).toBeUndefined();
+
+        const failed = JSON.parse(
+            readFileSync(
+                join(outDir, "collections", "20-__Work", "200.json"),
+                "utf-8",
+            ),
+        );
+        expect(failed.metadata.title).toBe("diagram.png");
+        expect(failed.mlData).toBeUndefined();
+        expect(failed.mlDataError).toContain("500");
+    });
+
+    it("exits 0 when every ML data request succeeds", async () => {
+        const { code } = await runCommand();
+        expect(code).toBe(0);
+    });
+});

test/library/mldata.test.ts

@@ -13,6 +13,8 @@
  *   2. `Library` wiring: after each refresh the library fetches ML data through
  *      the metadata pool for every known file not yet cached, is incremental on
  *      later refreshes, and refetches a file whose `updationTime` advanced.
+ *      Opening a cache directory written by another account starts empty,
+ *      its ML data included (issue #104).
  *
  * Embedding values are chosen to be exactly representable as float32 so the
  * round-trip through `clip.f32` compares equal.
@@ -498,4 +500,63 @@ describe("Library ML-data fetch on refresh", () => {
             await lib.close();
         }
     });
+
+    it("starts empty when the cache directory holds another account's cache", async () => {
+        // Account A fills the cache directory: metadata and ML data.
+        const clientA = new MLMockClient();
+        clientA.collectionsQueue.push({
+            collections: [collection(1, 100)],
+            deleted: [],
+            cursor: 100,
+        });
+        clientA.filesFor(1, {
+            files: [file(1001, 1, 90)],
+            deleted: [],
+            cursor: 90,
+        });
+        clientA.mlByFile.set(1001, payload([0.5, 0.25, 0.75]));
+        const libA = await Library.open({
+            client: clientA,
+            cacheDirectory,
+            refreshIntervalSeconds: 3600,
+        });
+        try {
+            await vi.waitFor(
+                () => expect(libA.status().lastMLFetchAt).toBeGreaterThan(0),
+                { timeout: 2000, interval: 5 },
+            );
+        } finally {
+            await libA.close();
+        }
+
+        // Account B opens the same directory.
+        const clientB = new MLMockClient();
+        clientB.userID = USER_ID + 1;
+        const sinceTimes: number[] = [];
+        const realCollectionsSince = clientB.collectionsSince.bind(clientB);
+        clientB.collectionsSince = async (args) => {
+            sinceTimes.push(args.sinceTime);
+            return realCollectionsSince(args);
+        };
+        const libB = await Library.open({
+            client: clientB,
+            cacheDirectory,
+            refreshIntervalSeconds: 3600,
+        });
+        try {
+            expect(sinceTimes[0]).toBe(0);
+            expect(libB.status().userID).toBe(USER_ID + 1);
+            expect(libB.listCollections()).toEqual([]);
+            expect(libB.getFile(1, 1001)).toBeUndefined();
+            expect(await libB.mldata.forFile({ fileID: 1001 })).toBeUndefined();
+            expect(
+                libB.mldata.searchByEmbedding({ embedding: [0.5, 0.25, 0.75] }),
+            ).toEqual([]);
+            expect(
+                existsSync(join(cacheDirectory, "mldata", "1001.json")),
+            ).toBe(false);
+        } finally {
+            await libB.close();
+        }
+    });
 });