Make lint and test phases of the Dockerfile (closes #96)

Follows the template: Dockerfile.lint is gone; the Dockerfile has a lint
phase (eslint, prettier --check .) and a test phase (vitest, run as the
node user, which the not-writable-directory tests need), and its
last stage compiles and depends on both. script/lint and script/test
build one phase each with --no-cache; script/docker and script/cibuild
pass --no-cache, so CHECK_EPOCH and LINT_EPOCH are removed.
script/cibuild is the single image build, so CI runs lint and the tests
once each. The tests that checked the old layout are deleted,
REPO_POLICIES.md is re-copied and the README describes the new layout.

Model: opus-5-5

Dockerfile

@@ -33,7 +33,7 @@ COPY . .
 
 # Unlike the template, the suite runs as the image's non-root `node` user:
 # root ignores directory permissions, so the tests of a destination that is
-# not writable would otherwise be skipped. vitest writes into /app.
+# not writable would otherwise fail. vitest writes into /app.
 RUN chown -R node:node /app
 USER node
 

README.md

@@ -475,6 +475,20 @@ failure still exits non-zero.
         <name>.json             collection metadata + file list
 ```
 
+A collection's directory and JSON are named after the collection, and a symlink
+after the file's title, both with unsafe characters replaced. When two
+collections would get the same name, or two files in one collection the same
+title (ignoring case in both), each of them gets its ID added: two albums named
+`Trip` become `Trip (10)/` and `Trip (11)/`, and two files titled `IMG_0001.JPG`
+become `IMG_0001 (12345).JPG` and `IMG_0001 (12346).JPG`. IDs never change, so a
+name stays the same from run to run until such a clash appears or goes away.
+
+Each run removes the symlinks into `originals/` that no longer belong in their
+collection's directory, and the directories (and JSON) of collections that were
+deleted or renamed. Nothing else in `collections/` is touched: a file or a
+symlink you put there stays, and a directory that still holds one after its
+symlinks are removed stays too, with its JSON.
+
 Each file is downloaded exactly once regardless of how many collections it
 appears in. On subsequent runs, existing originals are skipped. If a download
 fails, the error is logged and the backup continues with the next file. The exit
@@ -665,10 +679,13 @@ originals and thumbnails are kept; they are reached only through the files the
 current account's records name.
 
 A stored file appears only via an atomic temp-then-rename, so its presence means
-it is complete. The design also calls for a content-hash comparison against
-`FileMetadata.hash` on each fetched original; that check is deferred (issue
-https://git.eeqj.de/sneak/quak/issues/68) because the exact hash construction
-cannot yet be confirmed against the repo's fixtures.
+it is complete. Every downloaded original (by `quak get`, the cache, or
+`backup`) whose metadata records a content hash (`FileMetadata.hash`) is hashed
+as it is written: unkeyed BLAKE2b with a 64-byte output, standard base64. For a
+live photo, which is stored as a ZIP, the image and the video are hashed
+separately and joined as `<imageHash>:<videoHash>`. A mismatch stores nothing
+and fails the download with an error naming the file ID. An original with no
+recorded hash, from a very old client, is stored unchecked.
 
 ### Key types by source file
 

TODO.md

@@ -24,6 +24,21 @@ Tag v1.0.0.
   stage compiles and depends on both, so `script/cibuild` is one build.
   `Dockerfile.lint`, `CHECK_EPOCH`, `LINT_EPOCH` and the tests that checked them
   are gone; `REPO_POLICIES.md` is re-copied.
+- 2026-09-23: Fixed the backup's per-collection folders (issue 103). Two files
+  in one collection with the same title, and two collections with the same name,
+  each get their ID added to the name (`IMG_0001 (12345).JPG`, `Trip (10)/`), so
+  none replaces another's symlink or JSON. Each run removes symlinks into
+  `originals/` for files no longer in the collection, and the folders of deleted
+  or renamed collections, leaving anything else in `collections/` alone. The
+  README backup layout states the naming rule.
+- 2026-09-23: Checked downloaded originals against their recorded content hash
+  (issue 68). `downloadFile`, which `quak get`, the content cache and backup all
+  use, hashes the decrypted bytes (unkeyed BLAKE2b-512, standard base64) and
+  stores nothing on a mismatch, failing with an error naming the file ID. A live
+  photo ZIP is unpacked as it streams with `fflate` and its image and video
+  hashed separately as `<imageHash>:<videoHash>`. `decryptFile` reads older
+  clients' `imageHash` and `videoHash` fields for live photos. A file with no
+  recorded hash is stored unchecked.
 - 2026-09-23: Kept one account's cache from mixing with another's (issue 104).
   When `metadata.json` in the cache directory was written for a different,
   non-zero user ID than the client's, `Library.open` deletes it and `mldata/`

package.json

@@ -42,6 +42,7 @@
         "env-paths": "4.0.0",
         "exif-reader": "2.0.3",
         "fast-srp-hap": "2.0.4",
+        "fflate": "0.8.3",
         "jpeg-js": "0.4.4",
         "libsodium-wrappers-sumo": "0.8.4"
     }

src/backup.ts

@@ -17,7 +17,9 @@
 // temp-then-rename, so a file that exists is whole and is never re-fetched — an
 // interrupted run resumes by listing the directory. The derived views hold no
 // unique state, so they are rebuilt every run; that repairs stale sidecars and
-// missing or broken symlinks left by an earlier crash.
+// missing or broken symlinks left by an earlier crash. A rebuild also removes
+// the symlinks into originals/ that no longer belong to an album, and the
+// directories of albums that no longer exist.
 //
 // Resilience (issue #8): no per-file condition aborts the run. A failed
 // download or a failed symlink is caught, recorded in `failures.json` with a
@@ -34,13 +36,14 @@ import {
     readdirSync,
     readFileSync,
     readlinkSync,
+    rmdirSync,
     rmSync,
     statSync,
     symlinkSync,
     writeFileSync,
 } from "node:fs";
 import { copyFile, rename, rm } from "node:fs/promises";
-import { basename, dirname, join, relative } from "node:path";
+import { basename, dirname, extname, join, relative } from "node:path";
 
 import { fsyncPath } from "./download/index.js";
 import { safeExtension, sanitizeFileName } from "./filename.js";
@@ -227,6 +230,93 @@ const rebuildSymlink = (linkPath: string, target: string): void => {
     symlinkSync(target, linkPath);
 };
 
+// The on-disk names for the entries of one directory, keyed by ID. Each name
+// is used as is unless another entry would get the same name, ignoring case
+// (two names that differ only in case are one entry on a case-insensitive
+// file system); then every entry sharing it gets ` (<id>)`, before the
+// extension when `beforeExtension` is set. A name with an ID added can match
+// another entry's own name (`IMG (6).JPG`), so this repeats until no name is
+// shared. IDs are stable, so the names are too.
+const namesByID = (
+    entries: { id: number; name: string }[],
+    beforeExtension: boolean,
+): Map<number, string> => {
+    const withID = (id: number, name: string): string => {
+        const ext = beforeExtension ? extname(name) : "";
+        const stem = name.slice(0, name.length - ext.length);
+        return `${stem} (${id})${ext}`;
+    };
+    const names = new Map<number, string>();
+    for (const { id, name } of entries) names.set(id, name);
+    const suffixed = new Set<number>();
+    for (;;) {
+        const counts = new Map<string, number>();
+        for (const name of names.values()) {
+            const key = name.toLowerCase();
+            counts.set(key, (counts.get(key) ?? 0) + 1);
+        }
+        let changed = false;
+        for (const { id, name } of entries) {
+            if (suffixed.has(id)) continue;
+            if (counts.get(name.toLowerCase()) === 1) continue;
+            names.set(id, withID(id, name));
+            suffixed.add(id);
+            changed = true;
+        }
+        if (!changed) return names;
+    }
+};
+
+// Remove the symlinks in the album directory `dir` that point into
+// `originalsDir` and are not named in `keep`. Nothing else in the directory
+// is touched: anything else there was put there by the user.
+const removeStaleLinks = (
+    dir: string,
+    keep: Set<string>,
+    originalsDir: string,
+): void => {
+    const target = relative(dir, originalsDir);
+    for (const name of readdirSync(dir)) {
+        if (keep.has(name)) continue;
+        const path = join(dir, name);
+        if (
+            lstatSync(path).isSymbolicLink() &&
+            dirname(readlinkSync(path)) === target
+        ) {
+            rmSync(path);
+        }
+    }
+};
+
+// Remove the directories under `collectionsDir` that an earlier run wrote for
+// an album that is gone or renamed: a directory not named in `current` with a
+// `<name>.json` beside it holding an album ID, which is what a run writes. Its
+// symlinks into originals/ are removed; if that leaves it empty, it and its
+// JSON are deleted, otherwise both stay for what the user put there.
+const removeStaleAlbumDirs = (
+    collectionsDir: string,
+    current: Set<string>,
+    originalsDir: string,
+): void => {
+    for (const entry of readdirSync(collectionsDir, { withFileTypes: true })) {
+        if (!entry.isDirectory() || current.has(entry.name)) continue;
+        const jsonPath = join(collectionsDir, `${entry.name}.json`);
+        try {
+            const album = JSON.parse(readFileSync(jsonPath, "utf-8")) as {
+                id?: unknown;
+            };
+            if (typeof album.id !== "number") continue;
+        } catch {
+            continue;
+        }
+        const dir = join(collectionsDir, entry.name);
+        removeStaleLinks(dir, new Set(), originalsDir);
+        if (readdirSync(dir).length > 0) continue;
+        rmdirSync(dir);
+        rmSync(jsonPath);
+    }
+};
+
 const loadLedger = (path: string): Map<number, FailureEntry> => {
     const ledger = new Map<number, FailureEntry>();
     try {
@@ -303,9 +393,10 @@ export const runBackup = async (
 
     // Collections in scope, and the distinct files across them (a file shared
     // by two albums is one original).
-    const collections = lib
-        .listCollections()
-        .filter((c) => (only ? only.has(c.name) : true));
+    const allCollections = lib.listCollections();
+    const collections = allCollections.filter((c) =>
+        only ? only.has(c.name) : true,
+    );
     const collectionName = new Map<number, string>();
     for (const c of collections) collectionName.set(c.id, c.name);
 
@@ -406,23 +497,55 @@ export const runBackup = async (
         }
     }
 
-    // Then the per-collection symlink trees and JSON.
+    // Then the per-collection symlink trees and JSON. Directory names are
+    // chosen across every album, not just those in scope, so a scoped run
+    // names an album the same as a full one and never takes the directory of
+    // an album it skipped. Stale entries are removed before anything is
+    // rebuilt, so on a case-insensitive file system removing an old name can
+    // never remove the new one.
+    const albumDirNames = namesByID(
+        allCollections.map((c) => ({
+            id: c.id,
+            name: sanitizeFileName(c.name, `collection-${c.id}`),
+        })),
+        false,
+    );
+    try {
+        removeStaleAlbumDirs(
+            collectionsDir,
+            new Set(albumDirNames.values()),
+            originalsDir,
+        );
+    } catch (err) {
+        log(`FAILED removing old album directories: ${errorMessage(err)}`);
+    }
+
     for (const c of collections) {
-        const colDirName = sanitizeFileName(c.name, `collection-${c.id}`);
+        const colDirName = albumDirNames.get(c.id)!;
         const colDir = join(collectionsDir, colDirName);
         mkdirSync(colDir, { recursive: true });
 
         const files = filesByCollection.get(c.id) ?? [];
+        const linkNames = namesByID(
+            files.map((f) => ({
+                id: f.id,
+                name: sanitizeFileName(f.metadata.title, `file-${f.id}`),
+            })),
+            true,
+        );
+        try {
+            removeStaleLinks(colDir, new Set(linkNames.values()), originalsDir);
+        } catch (err) {
+            log(`FAILED removing old links in ${c.name}: ${errorMessage(err)}`);
+        }
+
         const metaFiles: { id: number; metadata: EnteFile["metadata"] }[] = [];
         for (const file of files) {
             metaFiles.push({ id: file.id, metadata: file.metadata });
             if (!includeOriginals) continue;
             const orig = join(originalsDir, originalName(file));
             if (!isPresent(orig)) continue;
-            const linkName = sanitizeFileName(
-                file.metadata.title,
-                `file-${file.id}`,
-            );
+            const linkName = linkNames.get(file.id)!;
             const linkPath = join(colDir, linkName);
             try {
                 rebuildSymlink(linkPath, relative(colDir, orig));

src/crypto/hash.ts

@@ -0,0 +1,22 @@
+import sodium, { type StateAddress } from "libsodium-wrappers-sumo";
+import { toBase64 } from "./encoding.js";
+
+// The content hash an uploading client records in a file's metadata: unkeyed
+// BLAKE2b with a 64-byte output over the original's bytes, fed in chunks, as
+// standard base64 with padding. Named after the upstream client's functions.
+// The output length is read at call time for the same reason as
+// `streamTagFinal` in stream.ts: libsodium sets its constants only once ready.
+
+export const chunkHashInit = (): StateAddress =>
+    sodium.crypto_generichash_init(null, sodium.crypto_generichash_BYTES_MAX);
+
+export const chunkHashUpdate = (state: StateAddress, chunk: Uint8Array): void =>
+    sodium.crypto_generichash_update(state, chunk);
+
+export const chunkHashFinal = (state: StateAddress): string =>
+    toBase64(
+        sodium.crypto_generichash_final(
+            state,
+            sodium.crypto_generichash_BYTES_MAX,
+        ),
+    );

src/crypto/index.ts

@@ -7,6 +7,7 @@ export {
 } from "./encoding.js";
 export { deriveKEK, deriveLoginSubkey } from "./kdf.js";
 export { decryptBox, decryptSealed } from "./box.js";
+export { chunkHashFinal, chunkHashInit, chunkHashUpdate } from "./hash.js";
 export {
     decryptBlob,
     encryptBlob,

src/download/index.ts

@@ -2,7 +2,11 @@ import { randomUUID } from "node:crypto";
 import { open, rename, rm } from "node:fs/promises";
 import type { FileHandle } from "node:fs/promises";
 import { dirname, join } from "node:path";
+import { Unzip, UnzipInflate } from "fflate";
 import {
+    chunkHashFinal,
+    chunkHashInit,
+    chunkHashUpdate,
     fromBase64,
     initStreamPull,
     pullStreamChunk,
@@ -237,19 +241,109 @@ export const writeAtomic = async (
 ): Promise<void> =>
     stageAtomic(destination, (handle) => handle.writeFile(plaintext));
 
+// Hashes an original's bytes as they are decrypted, for comparison with the
+// hash its uploader recorded.
+interface ContentHasher {
+    update: (plaintext: Uint8Array) => void;
+    digest: () => string;
+}
+
+const fileHasher = (): ContentHasher => {
+    const state = chunkHashInit();
+    return {
+        update: (plaintext) => chunkHashUpdate(state, plaintext),
+        digest: () => chunkHashFinal(state),
+    };
+};
+
+// A live photo is stored as a ZIP of its image and its video, and its recorded
+// hash is `<imageHash>:<videoHash>`, each over that part's own bytes. Like the
+// upstream client's decoder, this takes the first entries whose names start
+// with `image` and `video`.
+//
+// The ZIP is chosen by its uploader and may expand enormously, so entries are
+// hashed as they decompress and never held. fflate's `Unzip` inflates each
+// push in one piece, and deflate expands at most about 1000-fold, so the ZIP
+// is pushed in 4 KiB slices to keep each decompressed piece near 4 MiB, one
+// plaintext chunk. Every entry is started, even one that is not hashed,
+// because fflate keeps an unstarted entry's data in memory.
+const livePhotoHasher = (fileID: number): ContentHasher => {
+    const sliceSize = 4096;
+    const fail = (message: string, cause?: unknown): Error =>
+        new Error(`download: file ${fileID}: ${message}`, { cause });
+    const claimed = new Set<string>();
+    const hashes = new Map<string, string>();
+    const unzip = new Unzip((entry) => {
+        const part = ["image", "video"].find((p) => entry.name.startsWith(p));
+        const target =
+            part === undefined || claimed.has(part)
+                ? undefined
+                : { part, state: chunkHashInit() };
+        if (target !== undefined) claimed.add(target.part);
+        entry.ondata = (err, data, final) => {
+            if (err) throw err;
+            if (target === undefined) return;
+            chunkHashUpdate(target.state, data);
+            if (final) hashes.set(target.part, chunkHashFinal(target.state));
+        };
+        entry.start();
+    });
+    unzip.register(UnzipInflate);
+    // fflate reports a bad ZIP by throwing, sometimes a TypeError, which the
+    // retry would take for a network failure; a bad ZIP is never retried.
+    const push = (data: Uint8Array, final: boolean): void => {
+        try {
+            unzip.push(data, final);
+        } catch (err) {
+            throw fail("live photo is not a readable ZIP", err);
+        }
+    };
+    return {
+        update: (plaintext) => {
+            for (let i = 0; i < plaintext.length; i += sliceSize) {
+                push(plaintext.subarray(i, i + sliceSize), false);
+            }
+        },
+        digest: () => {
+            push(new Uint8Array(0), true);
+            const image = hashes.get("image");
+            const video = hashes.get("video");
+            if (image === undefined || video === undefined) {
+                throw fail(
+                    "live photo ZIP does not hold both an image and a video",
+                );
+            }
+            return `${image}:${video}`;
+        },
+    };
+};
+
 // Decrypt `stream` straight to `destination`, one plaintext chunk at a time,
 // under the atomic writer's temp-then-rename discipline. Memory stays bounded
 // by the chunk size: each decrypted chunk is written to the temp file and
 // dropped. The rename happens only after the stream authenticates as terminated
 // on TAG_FINAL; a truncated stream throws and leaves the destination untouched.
 // Returns the plaintext length written.
+//
+// `original` is the file whose original this is (none for a thumbnail, which
+// has no recorded hash). When its metadata has a hash, the decrypted bytes
+// must match it or nothing is stored. Both a plain file and a live photo's
+// parts are hashed as they stream. The mismatch error is not retried.
 const decryptToTemp = async (
     destination: string,
     stream: ReadableStream<Uint8Array>,
     header: Uint8Array,
     key: Uint8Array,
     onProgress?: ProgressCallback,
+    original?: EnteFile,
 ): Promise<number> => {
+    const expected = original?.metadata.hash;
+    const hasher =
+        original === undefined || expected === undefined
+            ? undefined
+            : original.metadata.fileType === "livePhoto"
+              ? livePhotoHasher(original.id)
+              : fileHasher();
     let bytesWritten = 0;
     try {
         await stageAtomic(destination, async (handle) => {
@@ -258,10 +352,18 @@ const decryptToTemp = async (
                 header,
                 key,
                 async (plaintext) => {
+                    hasher?.update(plaintext);
                     await handle.write(plaintext);
                 },
                 onProgress,
             );
+            if (original === undefined || hasher === undefined) return;
+            const actual = hasher.digest();
+            if (actual !== expected) {
+                throw new Error(
+                    `download: file ${original.id}: content hash ${actual} does not match the hash its uploader recorded, ${expected}`,
+                );
+            }
         });
     } catch (err) {
         // Cancel the body so its connection is closed now rather than held
@@ -302,10 +404,18 @@ const fetchAndDecrypt = async (
     key: Uint8Array,
     destination: string,
     onProgress?: ProgressCallback,
+    original?: EnteFile,
 ): Promise<number> =>
     withRetry(async () => {
         const stream = await openStream();
-        return decryptToTemp(destination, stream, header, key, onProgress);
+        return decryptToTemp(
+            destination,
+            stream,
+            header,
+            key,
+            onProgress,
+            original,
+        );
     }, api.getRetryOptions());
 
 export const downloadFile = async (
@@ -326,6 +436,7 @@ export const downloadFile = async (
         file.key,
         resolvedPath,
         onProgress,
+        file,
     );
     return { path: resolvedPath, bytesWritten };
 };

src/library/content.ts

@@ -15,12 +15,12 @@
 // Integrity. The reused streaming decrypt is the enforced guarantee: every
 // chunk is authenticated and the writer renames the file into place only once
 // the stream ends on TAG_FINAL, so a truncated or corrupt fetch throws and
-// nothing is stored. On top of that this module refuses to record a stored file
-// that came out empty. The design also asks for a content-hash comparison
-// against `FileMetadata.hash` (with a `fileSize` fallback); that is deferred —
-// see the PR — because the exact hash construction cannot be confirmed against
-// the repo's fixtures and `FileBlob.size` is the encrypted object size, not the
-// decrypted length this layer has.
+// nothing is stored. For an original whose metadata records a content hash
+// (`FileMetadata.hash`), the writer also hashes the decrypted bytes and stores
+// nothing if they differ, failing the fetch with an error naming the file. An
+// original with no recorded hash is stored unchecked, as the upstream client
+// does; thumbnails have none. On top of that this module refuses to record a
+// stored file that came out empty.
 
 import { existsSync, statSync } from "node:fs";
 import {

src/model/decrypt.ts

@@ -34,6 +34,28 @@ const FILE_TYPE_MAP: Record<number, FileType> = {
 
 const parseFileType = (n: number): FileType => FILE_TYPE_MAP[n] ?? "unknown";
 
+// The hash the uploading client recorded for the original's bytes, read the
+// way the upstream client's `metadataHash` reads it: `hash` if present,
+// otherwise, for a live photo from an older client that wrote the two parts
+// separately, `<imageHash>:<videoHash>`. A field that is not a non-empty
+// string counts as absent, and a file with no hash at all is normal.
+const expectedHash = (json: Record<string, unknown>): string | undefined => {
+    const text = (v: unknown): string | undefined =>
+        typeof v === "string" && v !== "" ? v : undefined;
+    const hash = text(json.hash);
+    if (hash !== undefined) return hash;
+    const imageHash = text(json.imageHash);
+    const videoHash = text(json.videoHash);
+    if (
+        json.fileType === 2 &&
+        imageHash !== undefined &&
+        videoHash !== undefined
+    ) {
+        return `${imageHash}:${videoHash}`;
+    }
+    return undefined;
+};
+
 export const decryptCollection = (
     raw: RawCollection,
     keys: KeyMaterial,
@@ -115,7 +137,7 @@ export const decryptFile = (
         modificationTime: metadataJSON.modificationTime ?? 0,
         latitude: metadataJSON.latitude,
         longitude: metadataJSON.longitude,
-        hash: metadataJSON.hash,
+        hash: expectedHash(metadataJSON),
     };
 
     const magicMetadata = decryptMagicMetadata(raw.magicMetadata, key);

src/model/types.ts

@@ -29,6 +29,9 @@ export interface FileMetadata {
     modificationTime: Microseconds;
     latitude?: number;
     longitude?: number;
+    // The content hash the uploader recorded (see `expectedHash` in
+    // decrypt.ts); `downloadFile` refuses an original that does not match it.
+    // Absent for files from very old clients.
     hash?: string;
 }
 

test/cli/backup.test.ts

@@ -40,6 +40,7 @@ import {
     readFileSync,
     readlinkSync,
     rmSync,
+    symlinkSync,
     writeFileSync,
 } from "node:fs";
 import { spawnSync } from "node:child_process";
@@ -47,6 +48,7 @@ import { join } from "node:path";
 import { tmpdir } from "node:os";
 import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
 
+import { runBackup, type BackupLibrary } from "../../src/backup.js";
 import { Library } from "../../src/library/index.js";
 import type { ContentSource } from "../../src/library/content.js";
 import type { CollectionsPage, FilesPage } from "../../src/client.js";
@@ -622,3 +624,318 @@ describe("lib.backup", () => {
         lib.close();
     });
 });
+
+// The album folders under collections/, driven through `runBackup` with a
+// stand-in library whose albums a test changes between runs.
+describe("backup album folders", () => {
+    interface Album {
+        collection: Collection;
+        files: EnteFile[];
+    }
+
+    const libraryOf = (albums: Album[]): BackupLibrary => ({
+        refresh: async () => {},
+        listCollections: () => albums.map((a) => a.collection),
+        listFiles: (id) =>
+            albums.find((a) => a.collection.id === id)?.files ?? [],
+        original: async (fileID) => {
+            const path = join(root, `source-${fileID}`);
+            writeFileSync(path, `original ${fileID}`);
+            return { path };
+        },
+        thumbnail: async () => {
+            throw new Error("no thumbnails in this stand-in");
+        },
+    });
+
+    // Every entry under collections/, one level of directories deep, with each
+    // symlink's target.
+    const tree = (outDir: string): string[] => {
+        const lines: string[] = [];
+        const list = (dir: string, prefix: string): void => {
+            for (const name of readdirSync(dir).sort()) {
+                const path = join(dir, name);
+                const st = lstatSync(path);
+                if (st.isSymbolicLink()) {
+                    lines.push(`${prefix}${name} -> ${readlinkSync(path)}`);
+                } else if (st.isDirectory() && prefix === "") {
+                    lines.push(`${name}/`);
+                    list(path, `${name}/`);
+                } else {
+                    lines.push(`${prefix}${name}`);
+                }
+            }
+        };
+        list(join(outDir, "collections"), "");
+        return lines;
+    };
+
+    const albumID = (outDir: string, jsonName: string): number =>
+        JSON.parse(readFileSync(join(outDir, "collections", jsonName), "utf-8"))
+            .id;
+
+    it("gives every file and every album its own name when names repeat", async () => {
+        const outDir = join(root, "backup");
+        const lib = libraryOf([
+            {
+                collection: collection(10, "Trip"),
+                files: [
+                    file(1, 10, "IMG_0001.JPG"),
+                    file(2, 10, "IMG_0001.JPG"),
+                    file(4, 10, "img_0001.jpg"),
+                    file(3, 10, "other.jpg"),
+                ],
+            },
+            {
+                collection: collection(11, "Trip"),
+                files: [file(3, 11, "other.jpg")],
+            },
+        ]);
+
+        const result = await runBackup(lib, { downloadDirectory: outDir });
+
+        expect(result.failed).toBe(0);
+        expect(tree(outDir)).toEqual([
+            "Trip (10)/",
+            "Trip (10)/IMG_0001 (1).JPG -> ../../originals/1.JPG",
+            "Trip (10)/IMG_0001 (2).JPG -> ../../originals/2.JPG",
+            "Trip (10)/img_0001 (4).jpg -> ../../originals/4.jpg",
+            "Trip (10)/other.jpg -> ../../originals/3.jpg",
+            "Trip (10).json",
+            "Trip (11)/",
+            "Trip (11)/other.jpg -> ../../originals/3.jpg",
+            "Trip (11).json",
+        ]);
+        expect(albumID(outDir, "Trip (10).json")).toBe(10);
+        expect(albumID(outDir, "Trip (11).json")).toBe(11);
+    });
+
+    it("keeps names unique when a name with an ID added is another entry's own name", async () => {
+        const outDir = join(root, "backup");
+        const lib = libraryOf([
+            {
+                collection: collection(10, "Trip"),
+                files: [
+                    file(5, 10, "IMG (6).JPG"),
+                    file(6, 10, "IMG.JPG"),
+                    file(7, 10, "IMG.JPG"),
+                ],
+            },
+            {
+                collection: collection(11, "Trip"),
+                files: [file(8, 11, "a.jpg")],
+            },
+            {
+                collection: collection(12, "Trip (11)"),
+                files: [file(9, 12, "b.jpg")],
+            },
+        ]);
+
+        const result = await runBackup(lib, { downloadDirectory: outDir });
+
+        expect(result.failed).toBe(0);
+        expect(tree(outDir)).toEqual([
+            "Trip (10)/",
+            "Trip (10)/IMG (6) (5).JPG -> ../../originals/5.JPG",
+            "Trip (10)/IMG (6).JPG -> ../../originals/6.JPG",
+            "Trip (10)/IMG (7).JPG -> ../../originals/7.JPG",
+            "Trip (10).json",
+            "Trip (11)/",
+            "Trip (11)/a.jpg -> ../../originals/8.jpg",
+            "Trip (11) (12)/",
+            "Trip (11) (12)/b.jpg -> ../../originals/9.jpg",
+            "Trip (11) (12).json",
+            "Trip (11).json",
+        ]);
+        expect(albumID(outDir, "Trip (10).json")).toBe(10);
+        expect(albumID(outDir, "Trip (11).json")).toBe(11);
+        expect(albumID(outDir, "Trip (11) (12).json")).toBe(12);
+    });
+
+    it("changes nothing on a second run over an unchanged account", async () => {
+        const outDir = join(root, "backup");
+        const lib = libraryOf([
+            {
+                collection: collection(10, "Trip"),
+                files: [
+                    file(1, 10, "IMG_0001.JPG"),
+                    file(2, 10, "IMG_0001.JPG"),
+                ],
+            },
+            {
+                collection: collection(11, "Trip"),
+                files: [file(3, 11, "other.jpg")],
+            },
+        ]);
+
+        await runBackup(lib, { downloadDirectory: outDir });
+        const before = tree(outDir);
+        const second = await runBackup(lib, { downloadDirectory: outDir });
+
+        expect(second.downloaded).toBe(0);
+        expect(second.failed).toBe(0);
+        expect(tree(outDir)).toEqual(before);
+    });
+
+    it("leaves the albums an onlyAlbumNames run skips as they were", async () => {
+        const outDir = join(root, "backup");
+        // "trip" is skipped by the scoped run but its name clashes with the
+        // in-scope "Trip", so "Trip" must keep its ID suffix.
+        const lib = libraryOf([
+            {
+                collection: collection(10, "Trip"),
+                files: [file(1, 10, "a.jpg")],
+            },
+            {
+                collection: collection(11, "trip"),
+                files: [file(2, 11, "b.jpg")],
+            },
+            {
+                collection: collection(12, "Work"),
+                files: [file(3, 12, "c.jpg")],
+            },
+        ]);
+        const json = (name: string): string =>
+            readFileSync(join(outDir, "collections", name), "utf-8");
+
+        await runBackup(lib, { downloadDirectory: outDir });
+        const before = tree(outDir);
+        const skippedJSON = [json("trip (11).json"), json("Work.json")];
+        const scoped = await runBackup(lib, {
+            downloadDirectory: outDir,
+            onlyAlbumNames: ["Trip"],
+        });
+
+        expect(scoped.failed).toBe(0);
+        expect(before).toEqual([
+            "Trip (10)/",
+            "Trip (10)/a.jpg -> ../../originals/1.jpg",
+            "Trip (10).json",
+            "Work/",
+            "Work/c.jpg -> ../../originals/3.jpg",
+            "Work.json",
+            "trip (11)/",
+            "trip (11)/b.jpg -> ../../originals/2.jpg",
+            "trip (11).json",
+        ]);
+        expect(tree(outDir)).toEqual(before);
+        expect([json("trip (11).json"), json("Work.json")]).toEqual(
+            skippedJSON,
+        );
+    });
+
+    it("removes links and album folders that are gone, and nothing the user added", async () => {
+        const outDir = join(root, "backup");
+        const albums: Album[] = [
+            {
+                collection: collection(10, "Trip"),
+                files: [
+                    file(1, 10, "IMG_0001.JPG"),
+                    file(2, 10, "IMG_0001.JPG"),
+                    file(3, 10, "other.jpg"),
+                ],
+            },
+            {
+                collection: collection(12, "Work"),
+                files: [file(5, 12, "a.jpg")],
+            },
+            {
+                collection: collection(13, "Old"),
+                files: [file(5, 13, "a.jpg")],
+            },
+        ];
+        const lib = libraryOf(albums);
+        await runBackup(lib, { downloadDirectory: outDir });
+
+        // What the user put in the tree: a note and a symlink of their own in
+        // an album, a note in an album about to be renamed, and a folder quak
+        // did not create.
+        const collectionsDir = join(outDir, "collections");
+        writeFileSync(join(collectionsDir, "Trip", "notes.txt"), "mine");
+        symlinkSync("../elsewhere", join(collectionsDir, "Trip", "mine"));
+        writeFileSync(join(collectionsDir, "Work", "keep.txt"), "mine");
+        mkdirSync(join(collectionsDir, "Mine"));
+        writeFileSync(join(collectionsDir, "Mine", "keep.txt"), "mine");
+
+        // File 2 leaves Trip, Work is renamed Office, Old is deleted.
+        albums[0]!.files.splice(1, 1);
+        albums[1]!.collection = collection(12, "Office");
+        albums.splice(2, 1);
+        const result = await runBackup(lib, { downloadDirectory: outDir });
+
+        expect(result.failed).toBe(0);
+        expect(tree(outDir)).toEqual([
+            "Mine/",
+            "Mine/keep.txt",
+            "Office/",
+            "Office/a.jpg -> ../../originals/5.jpg",
+            "Office.json",
+            "Trip/",
+            "Trip/IMG_0001.JPG -> ../../originals/1.JPG",
+            "Trip/mine -> ../elsewhere",
+            "Trip/notes.txt",
+            "Trip/other.jpg -> ../../originals/3.jpg",
+            "Trip.json",
+            "Work/",
+            "Work/keep.txt",
+            "Work.json",
+        ]);
+    });
+
+    // One album backed up, then a folder the user made beside it holding a
+    // symlink into originals/, with `json` (if given) as its sibling JSON.
+    const backupWithUserFolder = async (
+        json: string | undefined,
+    ): Promise<{ outDir: string; failed: number }> => {
+        const outDir = join(root, "backup");
+        const lib = libraryOf([
+            {
+                collection: collection(10, "Trip"),
+                files: [file(1, 10, "a.jpg")],
+            },
+        ]);
+        await runBackup(lib, { downloadDirectory: outDir });
+        const collectionsDir = join(outDir, "collections");
+        mkdirSync(join(collectionsDir, "Mine"));
+        symlinkSync(
+            "../../originals/1.jpg",
+            join(collectionsDir, "Mine", "a.jpg"),
+        );
+        if (json !== undefined) {
+            writeFileSync(join(collectionsDir, "Mine.json"), json);
+        }
+        const result = await runBackup(lib, { downloadDirectory: outDir });
+        return { outDir, failed: result.failed };
+    };
+
+    it("leaves a user folder with no JSON beside it as it was", async () => {
+        const { outDir, failed } = await backupWithUserFolder(undefined);
+
+        expect(failed).toBe(0);
+        expect(tree(outDir)).toEqual([
+            "Mine/",
+            "Mine/a.jpg -> ../../originals/1.jpg",
+            "Trip/",
+            "Trip/a.jpg -> ../../originals/1.jpg",
+            "Trip.json",
+        ]);
+    });
+
+    it("leaves a user folder whose JSON has no album ID as it was", async () => {
+        const json = '{"name":"Mine"}';
+        const { outDir, failed } = await backupWithUserFolder(json);
+
+        expect(failed).toBe(0);
+        expect(tree(outDir)).toEqual([
+            "Mine/",
+            "Mine/a.jpg -> ../../originals/1.jpg",
+            "Mine.json",
+            "Trip/",
+            "Trip/a.jpg -> ../../originals/1.jpg",
+            "Trip.json",
+        ]);
+        expect(
+            readFileSync(join(outDir, "collections", "Mine.json"), "utf-8"),
+        ).toBe(json);
+    });
+});

test/crypto/hash.test.ts

@@ -0,0 +1,33 @@
+import { beforeAll, describe, expect, it } from "vitest";
+import {
+    chunkHashFinal,
+    chunkHashInit,
+    chunkHashUpdate,
+    init,
+} from "../../src/crypto/index.js";
+
+beforeAll(async () => {
+    await init();
+});
+
+describe("content hash", () => {
+    // RFC 7693 Appendix A: BLAKE2b-512 of "abc".
+    const abc = Buffer.from(
+        "ba80a53f981c4d0d6a2797b69f12f6e94c212f14685ac4b74b12bb6fdbffa2d1" +
+            "7d87c5392aab792dc252d5de4533cc9518d38aa8dbf1925ab92386edd4009923",
+        "hex",
+    ).toString("base64");
+
+    it("is unkeyed BLAKE2b-512 in standard base64", () => {
+        const state = chunkHashInit();
+        chunkHashUpdate(state, new TextEncoder().encode("abc"));
+        expect(chunkHashFinal(state)).toBe(abc);
+    });
+
+    it("gives the same hash when the input arrives in chunks", () => {
+        const state = chunkHashInit();
+        chunkHashUpdate(state, new TextEncoder().encode("a"));
+        chunkHashUpdate(state, new TextEncoder().encode("bc"));
+        expect(chunkHashFinal(state)).toBe(abc);
+    });
+});

test/crypto/kdf.test.ts

@@ -29,10 +29,10 @@ describe("crypto.deriveKEK (Argon2id)", () => {
     });
 
     /**
-     * Cheap parameters used so the test suite stays under the 30-second
-     * budget. The real production parameters Ente uses are larger
-     * (memLimit up to 1 GiB, opsLimit 3-16). The algorithm is the same
-     * regardless of parameters.
+     * Cheap parameters used so the test suite stays under the 90-second
+     * `timeout` in the `test` phase of the `Dockerfile`. The real production
+     * parameters Ente uses are larger (memLimit up to 1 GiB, opsLimit 3-16).
+     * The algorithm is the same regardless of parameters.
      */
     const TEST_OPS = 2;
     const TEST_MEM = 64 * 1024 * 1024; // 64 MiB

test/download/download.test.ts

@@ -61,6 +61,7 @@ import { dirname, join } from "node:path";
 import { tmpdir } from "node:os";
 import { createHash } from "node:crypto";
 import sodium from "libsodium-wrappers-sumo";
+import { zipSync } from "fflate";
 import {
     beforeAll,
     beforeEach,
@@ -188,7 +189,31 @@ vi.mock("node:fs/promises", async (importOriginal) => {
     };
 });
 
+/**
+ * `chunkHashUpdate` is wrapped to record the length of every piece hashed, so
+ * a test can show that a live photo entry reaches the hash in pieces far
+ * smaller than the entry, rather than decompressed whole first.
+ */
+const hashHook = vi.hoisted(() => ({
+    lengths: [] as number[],
+}));
+
+vi.mock("../../src/crypto/index.js", async (importOriginal) => {
+    const actual =
+        await importOriginal<typeof import("../../src/crypto/index.js")>();
+    return {
+        ...actual,
+        chunkHashUpdate: (
+            ...args: Parameters<typeof actual.chunkHashUpdate>
+        ): void => {
+            hashHook.lengths.push(args[1].length);
+            actual.chunkHashUpdate(...args);
+        },
+    };
+});
+
 beforeEach(() => {
+    hashHook.lengths.length = 0;
     renameHook.calls.length = 0;
     renameHook.failWith = null;
     durabilityHook.events.length = 0;
@@ -1010,33 +1035,28 @@ describe.each(entryPoints)(
             expect(readdirSync(dir)).toEqual([]);
         });
 
-        // Root ignores directory permissions, so this cannot fail as root.
-        // The `test` phase of the `Dockerfile` runs as the `node` user so
-        // that `make check` runs it.
-        it.skipIf(process.getuid?.() === 0)(
-            "fails without creating anything when the destination directory is not writable",
-            async () => {
-                const key =
-                    sodium.crypto_secretstream_xchacha20poly1305_keygen();
-                const { header, ciphertext } = encryptFileBody(
-                    patternBytes(64, 34),
-                    key,
-                );
-                const { api, file } = fixtureFor(key, header, ciphertext);
-                const dir = freshDir();
-                const outPath = join(dir, "never.bin");
-                chmodSync(dir, 0o500);
-                try {
-                    await expect(
-                        download(api, file, outPath),
-                    ).rejects.toMatchObject({ code: "EACCES" });
-                } finally {
-                    chmodSync(dir, 0o700);
-                }
+        // Root ignores directory permissions, so this fails when run as root.
+        // The `test` phase of the `Dockerfile` runs as the `node` user.
+        it("fails without creating anything when the destination directory is not writable", async () => {
+            const key = sodium.crypto_secretstream_xchacha20poly1305_keygen();
+            const { header, ciphertext } = encryptFileBody(
+                patternBytes(64, 34),
+                key,
+            );
+            const { api, file } = fixtureFor(key, header, ciphertext);
+            const dir = freshDir();
+            const outPath = join(dir, "never.bin");
+            chmodSync(dir, 0o500);
+            try {
+                await expect(
+                    download(api, file, outPath),
+                ).rejects.toMatchObject({ code: "EACCES" });
+            } finally {
+                chmodSync(dir, 0o700);
+            }
 
-                expect(readdirSync(dir)).toEqual([]);
-            },
-        );
+            expect(readdirSync(dir)).toEqual([]);
+        });
     },
 );
 
@@ -1615,3 +1635,113 @@ describe.each(entryPoints)("$name progress", ({ name, download }) => {
         expectSameBytes(readFileSync(outPath), plaintext);
     });
 });
+
+describe("downloadFile content hash", () => {
+    // Node's own BLAKE2b-512 is the reference, so these tests do not depend
+    // on the code under test to compute what they expect.
+    const blake2b = (bytes: Uint8Array): string =>
+        createHash("blake2b512").update(bytes).digest("base64");
+
+    // Serve `plaintext` encrypted as file 999 with the given metadata. Four
+    // responses are scripted so a retried mismatch would show in `requests`.
+    const setup = (plaintext: Uint8Array, metadata: Partial<FileMetadata>) => {
+        const key = sodium.crypto_secretstream_xchacha20poly1305_keygen();
+        const { header, ciphertext } = encryptFileBody(plaintext, key);
+        const file = buildMockEnteFile(key, header, header);
+        file.metadata = { ...file.metadata, ...metadata };
+        const body = { kind: "body", bytes: ciphertext } as const;
+        const { fetch, requests } = scriptedCdnFetch(body, body, body, body);
+        const api = new ApiClient({ fetch, retry: { ...noWait, attempts: 4 } });
+        const dir = mkdtempSync(join(testDir, "hash-"));
+        const outPath = join(dir, "f.bin");
+        return {
+            run: () => downloadFile(api, file, outPath),
+            dir,
+            outPath,
+            requests,
+        };
+    };
+
+    const livePhotoZip = zipSync({
+        "image.heic": patternBytes(500, 81),
+        "video.mov": patternBytes(900, 82),
+    });
+    const livePhotoHash = `${blake2b(patternBytes(500, 81))}:${blake2b(patternBytes(900, 82))}`;
+
+    it("stores a file whose hash matches", async () => {
+        const plaintext = patternBytes(700, 80);
+        const t = setup(plaintext, { hash: blake2b(plaintext) });
+
+        await t.run();
+
+        expectSameBytes(readFileSync(t.outPath), plaintext);
+    });
+
+    it("rejects a mismatch, stores nothing, names the file and does not retry", async () => {
+        const t = setup(patternBytes(700, 80), {
+            hash: blake2b(patternBytes(700, 79)),
+        });
+
+        await expect(t.run()).rejects.toThrow(
+            /file 999: content hash .* does not match/,
+        );
+
+        expect(readdirSync(t.dir)).toEqual([]);
+        expect(t.requests()).toBe(1);
+    });
+
+    it("stores a file with no recorded hash unchecked", async () => {
+        const plaintext = patternBytes(700, 80);
+        const t = setup(plaintext, { hash: undefined });
+
+        await t.run();
+
+        expectSameBytes(readFileSync(t.outPath), plaintext);
+    });
+
+    it("stores a live photo whose image and video hashes match", async () => {
+        const t = setup(livePhotoZip, {
+            fileType: "livePhoto",
+            hash: livePhotoHash,
+        });
+
+        await t.run();
+
+        expectSameBytes(readFileSync(t.outPath), livePhotoZip);
+    });
+
+    it("hashes a large live photo entry as it decompresses, never whole", async () => {
+        // 64 MiB of zeros deflates to a few kilobytes, the shape of a ZIP
+        // that would exhaust memory if expanded whole.
+        const image = new Uint8Array(64 * 1024 * 1024);
+        const video = patternBytes(900, 83);
+        const zip = zipSync({ "image.heic": image, "video.mov": video });
+        const t = setup(zip, {
+            fileType: "livePhoto",
+            hash: `${blake2b(image)}:${blake2b(video)}`,
+        });
+
+        await t.run();
+
+        expectSameBytes(readFileSync(t.outPath), zip);
+        const hashed = hashHook.lengths.reduce((a, b) => a + b, 0);
+        expect(hashed).toBe(image.length + video.length);
+        expect(Math.max(...hashHook.lengths)).toBeLessThanOrEqual(
+            2 * STREAM_CHUNK_SIZE,
+        );
+    });
+
+    it("rejects a live photo whose hash does not match", async () => {
+        // The whole ZIP's hash is not the recorded one: each part is hashed.
+        const t = setup(livePhotoZip, {
+            fileType: "livePhoto",
+            hash: blake2b(livePhotoZip),
+        });
+
+        await expect(t.run()).rejects.toThrow(
+            /file 999: content hash .* does not match/,
+        );
+
+        expect(readdirSync(t.dir)).toEqual([]);
+    });
+});

test/model/decrypt.test.ts

@@ -351,6 +351,46 @@ describe("model.decryptFile", () => {
         }
     });
 
+    it("reads the recorded content hash, joining an older live photo's two parts", () => {
+        const masterKey = sodium.crypto_secretbox_keygen();
+        const { collectionKey } = buildRawCollection(masterKey);
+        const hashOf = (metadata: Record<string, unknown>) =>
+            decryptFile(
+                buildRawFile(collectionKey, {
+                    metadata: { title: "x", ...metadata },
+                }),
+                collectionKey,
+            ).metadata.hash;
+
+        expect(hashOf({ fileType: 0, hash: "H" })).toBe("H");
+        expect(
+            hashOf({ fileType: 2, hash: "H", imageHash: "I", videoHash: "V" }),
+        ).toBe("H");
+        expect(hashOf({ fileType: 2, imageHash: "I", videoHash: "V" })).toBe(
+            "I:V",
+        );
+        expect(hashOf({ fileType: 2, imageHash: "I" })).toBeUndefined();
+        expect(
+            hashOf({ fileType: 0, imageHash: "I", videoHash: "V" }),
+        ).toBeUndefined();
+        expect(hashOf({ fileType: 0 })).toBeUndefined();
+        expect(hashOf({ fileType: 0, hash: 42 })).toBeUndefined();
+        expect(
+            hashOf({ fileType: 2, imageHash: "I", videoHash: 7 }),
+        ).toBeUndefined();
+        // An empty string counts as absent, not as a hash to match.
+        expect(hashOf({ fileType: 0, hash: "" })).toBeUndefined();
+        expect(
+            hashOf({ fileType: 2, hash: "", imageHash: "I", videoHash: "V" }),
+        ).toBe("I:V");
+        expect(
+            hashOf({ fileType: 2, imageHash: "", videoHash: "V" }),
+        ).toBeUndefined();
+        expect(
+            hashOf({ fileType: 2, imageHash: "I", videoHash: "" }),
+        ).toBeUndefined();
+    });
+
     it("maps fileType numbers to FileType strings", () => {
         // Ente uses: 0=image, 1=video, 2=livePhoto
         const masterKey = sodium.crypto_secretbox_keygen();

yarn.lock

@@ -1069,6 +1069,11 @@ fastq@^1.6.0:
   dependencies:
     reusify "^1.0.4"
 
+fflate@0.8.3:
+  version "0.8.3"
+  resolved "https://registry.yarnpkg.com/fflate/-/fflate-0.8.3.tgz#bc27d8eb30343d4d512abb03480202ce65d825fc"
+  integrity sha512-tbZNuJrLwGUp3zshBtdy4W+ORxZuIh8a5ilyIEQDC5rY1f3U20JMry0Ll3WBzU58EZKsEuJFXhb5gwv8CsPvgA==
+
 file-entry-cache@^8.0.0:
   version "8.0.0"
   resolved "https://registry.yarnpkg.com/file-entry-cache/-/file-entry-cache-8.0.0.tgz#7787bddcf1131bffb92636c69457bbc0edd6d81f"