diff --git a/README.md b/README.md index 5654819..0a7e7a3 100644 --- a/README.md +++ b/README.md @@ -127,19 +127,21 @@ matters. ### Exit codes -| Code | Meaning | -| ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `0` | The backup is complete: every original is at its save path, and no file failed. | -| `1` | The run finished with files in `failures.json`, or it stopped on another error, printed as one line `quak: `. The next run tries again. | -| `2` | Another backup of the same directory is running. This run sent no request and changed nothing. | -| `3` | There is no usable session: none is saved, the saved one is corrupt, or the server no longer accepts it. Run `quak login` as the user the cron job runs as. | +| Code | Meaning | +| ---- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `0` | The backup is complete: every original is at its save path, and no file failed. | +| `1` | The run finished with files in `failures.json` or an album it could not decrypt, or it stopped on another error, printed as one line `quak: `. The next run tries again. | +| `2` | Another backup of the same directory is running. This run sent no request and changed nothing. | +| `3` | There is no usable session: none is saved, the saved one is corrupt, or the server no longer accepts it. Run `quak login` as the user the cron job runs as. | After a `1`, the next run fetches each missing original again, fetches the ML -data that is not cached, and rebuilds the album links. An original that a -`--verify` run could not read, or put back with bytes that still do not match, -stays at its save path. Only a later `--verify` run checks it again: a run -without `--verify` leaves it as it is and takes the file out of `failures.json`, -so that run can exit 0. +data that is not cached, and rebuilds the album links. An album quak could not +decrypt, or a file in `failures.json` because quak could not decrypt it, keeps +the exit code at `1` until the server sends a copy that decrypts, or deletes it +(see "Backup layout"). An original that a `--verify` run could not read, or put +back with bytes that still do not match, stays at its save path. Only a later +`--verify` run checks it again: a run without `--verify` leaves it as it is and +takes the file out of `failures.json`, so that run can exit 0. ## Examples @@ -623,8 +625,10 @@ library. The read commands — `collections`, `files`, `get`, `get-thumb`, answer, so they report current account state rather than whatever the cache last held. If that round-trip fails, the command prints the error on one line and exits 1, or 3 when the server no longer accepts the saved session (see "Session -handling"). `--cache-dir` overrides where the cache lives; without it each -account gets its own directory under the per-user cache path. +handling"). An album or file quak cannot decrypt does not fail it; the command +answers from what did decrypt (see "Default reads vs. fresh reads"). +`--cache-dir` overrides where the cache lives; without it each account gets its +own directory under the per-user cache path. An ID given to `files --collection`, `get`, `get-thumb` or `helper fix-missing-thumbnails --file` is written in decimal digits, no larger @@ -741,6 +745,19 @@ succeeds, or once it is no longer in the library or in the backup's scope. The library's `lib.backup({ includeThumbnails: true })` also writes `thumbnails/.jpg` beside `collections/`; `quak backup` does not. +A file in the backup's scope that the library could not decrypt (see "Default +reads vs. fresh reads") goes into `failures.json` with an error that starts +`could not be decrypted:` and gives the reason. If a copy of it in another album +decrypted, it does not: its original is stored from that copy, and the album +where it does not decrypt neither links nor lists it. An album the library could +not decrypt has no name quak can read and no files it can list. A run without +`--json` prints one line for it, +`FAILED album : could not be decrypted: `, and `--json` and the +`BackupResult` list its ID in `undecryptableAlbums`. Such a file in +`failures.json`, or such an album, keeps the exit code at 1, run after run, +until the server sends a copy that decrypts or deletes it; the next run then +drops it. + `backup.lock` keeps two backups of the same directory from running at once, such as a cron run that starts while the previous one is still going. A backup creates `` if it is missing and takes the lock before its refresh, and @@ -777,9 +794,12 @@ both, each named after the title with that file's extension (`IMG_0001.heic` and Each run removes the symlinks into the date folders that no longer belong in their collection's directory, and the directories (and JSON) of collections that -were deleted or renamed. Nothing else in `collections/` is touched: a file or a -symlink you put there stays, and a directory that still holds one after its -symlinks are removed stays too, with its JSON. +were deleted or renamed. An album quak can no longer decrypt loses its directory +and JSON the same way, and a file whose copy in an album quak can no longer +decrypt loses its symlink there and its entry in that album's JSON; the +originals stay at their save paths. Nothing else in `collections/` is touched: a +file or a symlink you put there stays, and a directory that still holds one +after its symlinks are removed stays too, with its JSON. Each file is downloaded exactly once regardless of how many collections it appears in, and written once: straight to its save path, with no copy left in @@ -915,6 +935,16 @@ rejects the caller (default reads stay silent and keep serving the last good copy). The CLI's read commands use fresh reads (issue https://git.eeqj.de/sneak/quak/issues/75). +An album or file in the server's answer that quak cannot decrypt, or whose +metadata is not JSON, does not fail the refresh. The refresh leaves it out, an +album with all its files, and drops any older copy the cache held, so reads +answer from what did decrypt. The library keeps a list of what it left out, with +the reason, in `metadata.json`: the next refresh asks the server only for what +changed, so it would not see the record again. A record stays on that list until +a later refresh gets a copy that decrypts or learns that the server deleted it. +`lib.backup()` reports each such album, and each such file in its scope that has +no copy that decrypted (see "Backup layout"). + ### Read surface - `lib.albums.list()` → `Album[]`, newest-updated first. diff --git a/TODO.md b/TODO.md index 25a6bfb..e170323 100644 --- a/TODO.md +++ b/TODO.md @@ -25,6 +25,19 @@ declares one. # Completed Steps +- 2026-10-14: An album or file quak cannot decrypt, or whose metadata is not + JSON, no longer fails the refresh, so one bad record no longer stops + `quak backup` from storing anything (issue 181). The refresh leaves it out, an + album with its files, and the library keeps it with the reason in + `metadata.json` until the server sends a copy that decrypts or deletes it. + `quak backup` stores every other file, records each such file in + `failures.json` with an error starting `could not be decrypted:`, names each + such album's ID in a printed line, or with `--json` in `undecryptableAlbums`, + and exits 1 while any remain. A file whose copy in another album decrypts is + not recorded: its original is stored from that copy, and the album where it + does not decrypt neither links nor lists it. The read commands answer from + what decrypted. + - 2026-10-14: `quak backup` no longer asks the server for changes every 3 seconds while it runs (issue 183). It opens its library with `refreshIntervalSeconds: Infinity`, which the library now takes as no diff --git a/src/backup.ts b/src/backup.ts index 184d466..5deae14 100644 --- a/src/backup.ts +++ b/src/backup.ts @@ -47,6 +47,9 @@ // download, a failed symlink, or ML data missing because the ML data fetch // failed is caught, recorded in `failures.json` with a classification, a // running attempt count, and the last-tried time, and the run continues. +// A file the library could not decrypt is recorded there too, unless a copy of +// it in another album decrypted; an album it could not decrypt is logged by ID +// and listed in `result.undecryptableAlbums`, as its name is not known. // `result.failed` — and thus the CLI's exit code — stays non-zero while any // failure remains unresolved and clears once every one succeeds. Each run // reconciles the ledger against the files it attempted, so an entry for a file @@ -86,6 +89,7 @@ import { storedAtSavePath, } from "./library/content.js"; import { representative } from "./library/records.js"; +import type { Undecryptable } from "./library/store.js"; import { extractImageMetadata } from "./metadata-backup.js"; import type { MLData } from "./mldata-fetch.js"; import type { Collection, EnteFile, FileMetadata } from "./model/types.js"; @@ -141,6 +145,9 @@ export interface BackupResult { failed: number; // This run's per-file errors, in encounter order. errors: BackupError[]; + // The IDs of the albums the library could not decrypt, whose files it + // could not list; the CLI exits non-zero while there are any. + undecryptableAlbums: number[]; } // The slice of the library that backup drives. `Library` implements it; a test @@ -151,6 +158,9 @@ export interface BackupLibrary { refresh(): Promise; listCollections(): Collection[]; listFiles(collectionID: number): EnteFile[]; + // The albums, and files in albums, that the library could not decrypt, + // which `listCollections` and `listFiles` leave out. + undecryptable(): Undecryptable[]; // Get an original's bytes onto disk through the content cache/pools, // returning where they landed: `destination` when they were fetched now, // otherwise wherever they already were (the cache, or the library's save @@ -574,33 +584,56 @@ const runLockedBackup = async ( let unchecked = 0; const recordFailure = ( - file: EnteFile, + fileID: number, + title: string, collection: string, err: unknown, ): void => { // Count at most one attempt per file per run: a file whose original // and thumbnail both fail this run must not double its attempt count // or appear twice in errors. - if (failedThisRun.has(file.id)) return; + if (failedThisRun.has(fileID)) return; const error = errorMessage(err); - errors.push({ - fileID: file.id, - title: file.metadata.title, - collection, - error, - }); - const prior = ledger.get(file.id); - ledger.set(file.id, { - fileID: file.id, - title: file.metadata.title, + errors.push({ fileID, title, collection, error }); + const prior = ledger.get(fileID); + ledger.set(fileID, { + fileID, + title, classification: classify(err), attempts: (prior?.attempts ?? 0) + 1, lastTriedAt: now, error, }); - failedThisRun.add(file.id); + failedThisRun.add(fileID); }; + // What the library could not decrypt. An album is logged by its ID, as + // its name is not known, so a run limited to some albums logs it too. A + // file in an album in scope is backed up from a copy of it in another + // album that decrypted, even an album out of scope. With no such copy it + // is recorded as failed, with an empty title as its metadata is not known. + const scope = new Set(collections.map((c) => c.id)); + const undecryptableAlbums: number[] = []; + for (const record of lib.undecryptable()) { + const error = `could not be decrypted: ${record.error}`; + if (record.fileID === undefined) { + log(`FAILED album ${record.collectionID}: ${error}`); + undecryptableAlbums.push(record.collectionID); + } else if (scope.has(record.collectionID)) { + const copies = memberships.get(record.fileID); + if (copies) { + distinct.set(record.fileID, representative(copies)); + } else { + recordFailure( + record.fileID, + "", + collectionName.get(record.collectionID) ?? "", + new Error(error), + ); + } + } + } + // Phase 1: get the bytes. Put each pending original at its save path // through the content cache/pools, as `Photo.download()` does, and fetch // the optional thumbnails; a present file is left as is. With `verify`, a @@ -636,7 +669,8 @@ const runLockedBackup = async ( `FAILED verifying original ${file.metadata.title}: ${errorMessage(err)}`, ); recordFailure( - file, + file.id, + file.metadata.title, collectionName.get(file.collectionID) ?? "", err, ); @@ -676,7 +710,8 @@ const runLockedBackup = async ( `FAILED original ${file.metadata.title}: ${errorMessage(err)}`, ); recordFailure( - file, + file.id, + file.metadata.title, collectionName.get(file.collectionID) ?? "", err, ); @@ -693,7 +728,8 @@ const runLockedBackup = async ( await copyAtomic(path, dest); } catch (err) { recordFailure( - file, + file.id, + file.metadata.title, collectionName.get(file.collectionID) ?? "", err, ); @@ -733,7 +769,8 @@ const runLockedBackup = async ( if (mlData === undefined && mlDataError !== undefined) { writeSidecar(path, file, { mlDataError }, image); recordFailure( - file, + file.id, + file.metadata.title, collectionName.get(file.collectionID) ?? "", new Error(`ML data: ${mlDataError}`), ); @@ -806,7 +843,12 @@ const runLockedBackup = async ( log( `FAILED symlink ${c.name}/${linkName}: ${errorMessage(err)}`, ); - recordFailure(link.file, c.name, err); + recordFailure( + link.file.id, + link.file.metadata.title, + c.name, + err, + ); } } @@ -837,6 +879,7 @@ const runLockedBackup = async ( unchecked, failed: ledger.size, errors, + undecryptableAlbums, }; }; diff --git a/src/cli-commands.ts b/src/cli-commands.ts index 154c180..076d1ef 100644 --- a/src/cli-commands.ts +++ b/src/cli-commands.ts @@ -470,7 +470,9 @@ export const backupCommand = async ( } } - return result.failed > 0 ? 1 : 0; + return result.failed > 0 || result.undecryptableAlbums.length > 0 + ? 1 + : 0; } finally { await lib.close(); } diff --git a/src/client.ts b/src/client.ts index 62784c1..bcb3dac 100644 --- a/src/client.ts +++ b/src/client.ts @@ -44,20 +44,28 @@ export interface ClientSnapshot { // The result of a resumable enumeration. Live decrypted records and deleted // ids are kept apart on purpose: a tombstone carries no key or metadata to -// decrypt, so it is a bare id rather than a hollowed-out record. `cursor` is -// the max `updationTime` seen, to pass back into the next call. +// decrypt, so it is a bare id rather than a hollowed-out record. A live row +// that cannot be decrypted, or whose metadata cannot be parsed, is in +// `undecryptable` with the reason instead, so one bad row does not fail the +// call; absent means there was none. `cursor` is the max `updationTime` seen, +// those rows included, to pass back into the next call. export interface CollectionsPage { collections: Collection[]; deleted: number[]; + undecryptable?: { id: number; error: string }[]; cursor: number; } export interface FilesPage { files: EnteFile[]; deleted: number[]; + undecryptable?: { id: number; error: string }[]; cursor: number; } +const errorMessage = (err: unknown): string => + err instanceof Error ? err.message : String(err); + export class Client { private readonly api: ApiClient; private readonly email: string; @@ -237,11 +245,12 @@ export class Client { } // Enumerate collections changed since `sinceTime`. Live collections are - // decrypted; tombstoned ones (isDeleted) are surfaced as bare ids. The - // returned cursor is the max `updationTime` seen — including tombstones, so - // the next sync resumes past them — falling back to `sinceTime` when the - // response is empty. `/collections/v2` returns the whole changed set in one - // response, so there is no pagination here. + // decrypted; tombstoned ones (isDeleted) are surfaced as bare ids, and + // ones that do not decrypt as ids with the reason. The returned cursor is + // the max `updationTime` seen — including tombstones, so the next sync + // resumes past them — falling back to `sinceTime` when the response is + // empty. `/collections/v2` returns the whole changed set in one response, + // so there is no pagination here. async collectionsSince(args: { sinceTime: number; }): Promise { @@ -254,31 +263,40 @@ export class Client { const collections: Collection[] = []; const deleted: number[] = []; + const undecryptable: { id: number; error: string }[] = []; let cursor = args.sinceTime; for (const raw of raws) { if (raw.isDeleted) { deleted.push(raw.id); } else { - collections.push( - decryptCollection( - raw, - { - masterKey: this.masterKey, - publicKey: this.publicKey, - secretKey: this.secretKey, - }, - this.userID, - ), - ); + try { + collections.push( + decryptCollection( + raw, + { + masterKey: this.masterKey, + publicKey: this.publicKey, + secretKey: this.secretKey, + }, + this.userID, + ), + ); + } catch (err) { + undecryptable.push({ + id: raw.id, + error: errorMessage(err), + }); + } } if (raw.updationTime > cursor) cursor = raw.updationTime; } - return { collections, deleted, cursor }; + return { collections, deleted, undecryptable, cursor }; } // Enumerate a collection's files changed since `sinceTime`, paginating the // diff from that cursor. Live rows are decrypted; tombstoned ones are - // surfaced as bare ids. Returns the final cursor to resume from. + // surfaced as bare ids, and ones that do not decrypt or parse as ids with + // the reason. Returns the final cursor to resume from. async filesSince(args: { collectionID: number; collectionKey: Uint8Array; @@ -288,6 +306,7 @@ export class Client { const { collectionID, collectionKey } = args; const files: EnteFile[] = []; const deleted: number[] = []; + const undecryptable: { id: number; error: string }[] = []; let cursor = args.sinceTime; for (;;) { const { diff, hasMore } = await this.api.getJSON<{ @@ -300,7 +319,14 @@ export class Client { if (raw.isDeleted) { deleted.push(raw.id); } else { - files.push(decryptFile(raw, collectionKey)); + try { + files.push(decryptFile(raw, collectionKey)); + } catch (err) { + undecryptable.push({ + id: raw.id, + error: errorMessage(err), + }); + } } if (raw.updationTime > pageMax) pageMax = raw.updationTime; } @@ -321,18 +347,19 @@ export class Client { } cursor = pageMax; } - return { files, deleted, cursor }; + return { files, deleted, undecryptable, cursor }; } - // Whole-account listing: every live collection, deletions hidden. A thin - // wrapper over `collectionsSince` from the beginning of time. + // Whole-account listing: every live collection that decrypts, deletions + // hidden. A thin wrapper over `collectionsSince` from the beginning of + // time. async listCollections(): Promise { const { collections } = await this.collectionsSince({ sinceTime: 0 }); return collections; } - // Every live file in a collection, deletions hidden. A thin wrapper over - // `filesSince` from the beginning of time. + // Every live file in a collection that decrypts, deletions hidden. A thin + // wrapper over `filesSince` from the beginning of time. async listFiles( collectionID: number, collectionKey: Uint8Array, diff --git a/src/library/index.ts b/src/library/index.ts index 080020b..443f4d4 100644 --- a/src/library/index.ts +++ b/src/library/index.ts @@ -601,6 +601,7 @@ export class Library { refresh: () => this.refreshNow(), listCollections: () => this.store.listCollections(), listFiles: (id) => this.store.listFiles(id), + undecryptable: () => this.store.listUndecryptable(), original: (fileID, destination) => cache!.backupOriginal(fileID, destination), thumbnail: (fileID) => cache!.thumbnail(fileID), @@ -703,7 +704,10 @@ export class Library { // Fetch every change since the stored cursor, then commit. All network // reads happen before any store mutation, so a fetch that throws leaves the - // store untouched and the previous snapshot intact. + // store untouched and the previous snapshot intact. A collection or file + // that does not decrypt is left out instead, as if deleted, with the reason + // kept in the store: the cursor moves past it, so no later refresh sees it + // again until the server's copy changes. private async refreshOnce(): Promise { const page = await this.client.collectionsSince({ sinceTime: this.store.collectionsSinceTime, @@ -740,9 +744,17 @@ export class Library { this.store.deleteCollection(id); changed = true; } + if (this.store.deleteUndecryptable(id)) changed = true; + } + // A collection left out takes its files with it. + for (const { id, error } of page.undecryptable ?? []) { + this.store.deleteCollection(id); + this.store.putUndecryptable({ collectionID: id, error }); + changed = true; } for (const collection of page.collections) { this.store.putCollection(collection); + this.store.deleteUndecryptable(collection.id); changed = true; } @@ -752,9 +764,22 @@ export class Library { this.store.deleteFile(collectionID, id); changed = true; } + if (this.store.deleteUndecryptable(collectionID, id)) { + changed = true; + } + } + for (const { id, error } of filePage.undecryptable ?? []) { + this.store.deleteFile(collectionID, id); + this.store.putUndecryptable({ + collectionID, + fileID: id, + error, + }); + changed = true; } for (const f of filePage.files) { this.store.putFile(f); + this.store.deleteUndecryptable(collectionID, f.id); changed = true; } } diff --git a/src/library/store.ts b/src/library/store.ts index e74f8e1..fc59b78 100644 --- a/src/library/store.ts +++ b/src/library/store.ts @@ -2,10 +2,11 @@ // // The store keeps one `metadata.json` file holding the account's server // state: the user id, a schema version, the cursor for the incremental -// collections listing, and the decrypted collection and file records. The -// whole file is read into RAM on load and rewritten as a whole on save; there -// is no partial update and no lock file. A separate refresh unit populates the -// store from the server — this module only stores what it is given. +// collections listing, the decrypted collection and file records, and the +// collections and files the refresh could not decrypt. The whole file is read +// into RAM on load and rewritten as a whole on save; there is no partial +// update and no lock file. A separate refresh unit populates the store from +// the server — this module only stores what it is given. // // The file is a cache, so it is never trusted to exist or to be intact: a // missing or unreadable file loads as an empty store rather than an error, and @@ -33,12 +34,22 @@ const FILE_MODE = 0o600; type StoredCollection = Omit & { key: string }; type StoredFile = Omit & { key: string }; +// A collection, or with `fileID` a file in it, that the last refresh to fetch +// it could not decrypt or parse, and why. It is kept until a later refresh +// fetches it again, because a refresh fetches only what changed. +export interface Undecryptable { + collectionID: number; + fileID?: number; + error: string; +} + interface StoredMetadata { schemaVersion: number; userID: number; collectionsSinceTime: Microseconds; collections: StoredCollection[]; files: StoredFile[]; + undecryptable: Undecryptable[]; } const encodeKey = (key: Uint8Array): string => @@ -67,6 +78,7 @@ export class MetadataStore { private readonly collections = new Map(); private readonly files = new Map(); + private undecryptable: Undecryptable[] = []; private constructor(path: string) { this.path = path; @@ -107,6 +119,12 @@ export class MetadataStore { }; store.files.set(fileKey(file.collectionID, file.id), file); } + for (const record of parsed.undecryptable) { + if (typeof record.collectionID !== "number") { + throw new Error("damaged record of what did not decrypt"); + } + store.undecryptable.push(record); + } } catch { // Any corruption discards the partial result: a half-read cache is // worse than an empty one, since the refresh unit will rebuild it. @@ -134,6 +152,7 @@ export class MetadataStore { ...f, key: encodeKey(f.key), })), + undecryptable: this.undecryptable, }; const dir = dirname(this.path); @@ -165,7 +184,8 @@ export class MetadataStore { this.collections.set(collection.id, collection); } - // Removing a collection also drops its file memberships: a file record is + // Removing a collection also drops its file memberships, and any record + // that it or one of its files could not be decrypted: a file record is // only meaningful as part of a collection the cache still knows about. deleteCollection(id: number): void { this.collections.delete(id); @@ -174,6 +194,9 @@ export class MetadataStore { this.files.delete(key); } } + this.undecryptable = this.undecryptable.filter( + (record) => record.collectionID !== id, + ); } getFile(collectionID: number, fileID: number): EnteFile | undefined { @@ -205,4 +228,26 @@ export class MetadataStore { deleteFile(collectionID: number, fileID: number): void { this.files.delete(fileKey(collectionID, fileID)); } + + listUndecryptable(): Undecryptable[] { + return [...this.undecryptable]; + } + + // Replaces any earlier record of the same collection or file. + putUndecryptable(record: Undecryptable): void { + this.deleteUndecryptable(record.collectionID, record.fileID); + this.undecryptable.push(record); + } + + // Forgets the record of the collection, or with `fileID` of that file in + // it. Returns whether there was one. + deleteUndecryptable(collectionID: number, fileID?: number): boolean { + const before = this.undecryptable.length; + this.undecryptable = this.undecryptable.filter( + (record) => + record.collectionID !== collectionID || + record.fileID !== fileID, + ); + return this.undecryptable.length < before; + } } diff --git a/test/cli/backup-undecryptable.test.ts b/test/cli/backup-undecryptable.test.ts new file mode 100644 index 0000000..7344ba1 --- /dev/null +++ b/test/cli/backup-undecryptable.test.ts @@ -0,0 +1,554 @@ +/** + * `quak backup` over an account holding an album or a file quak cannot decrypt + * (issue 181). + * + * The server sends every album and file encrypted, and a refresh decrypts each + * one it fetches. A record whose key no key of the account opens, or whose + * metadata is not JSON, must not fail the refresh: a refresh that fails commits + * nothing, so `quak backup` would store nothing, on every run, until the + * server's copy changed. Instead the refresh leaves that record out, and the + * library keeps a list of what it left out in its cache. It has to: the next + * refresh asks the server only for what changed since the last one, so it will + * not see the record again. + * + * The backup stores every other file. It records each file it could not + * decrypt in `failures.json`, prints one line for each album it could not + * decrypt, and exits 1 while any remain. Once the server sends a copy that + * decrypts, or deletes the record, the next run drops it and can exit 0. The + * read commands answer from what did decrypt. + * + * These tests run the commands with a real saved session over a stand-in Ente + * server, which serves albums, files and originals encrypted with libsodium as + * the real server does. A record is broken the way a corrupt one arrives: its + * key replaced by bytes no key opens, or its metadata text that is not JSON. + */ + +import { + existsSync, + mkdtempSync, + readdirSync, + readFileSync, + rmSync, +} from "node:fs"; +import { join } from "node:path"; +import { tmpdir } from "node:os"; +import sodium from "libsodium-wrappers-sumo"; +import { afterEach, beforeAll, beforeEach, describe, expect, it } from "vitest"; + +import { + backupCommand, + collectionsCommand, + filesCommand, + saveSession, + type CliContext, +} from "../../src/cli-commands.js"; +import { loadSession } from "../../src/cli-session.js"; +import { Client, type ClientSnapshot } from "../../src/client.js"; +import { encryptBlob, init, toBase64 } from "../../src/crypto/index.js"; +import { Library } from "../../src/library/index.js"; + +const USER_ID = 42; + +// Every file is taken at noon local time on 2026-03-01, in microseconds as Ente +// stores times, so its original is saved in the folder `DAY` whatever the time +// zone of the machine running the tests. +const TAKEN = new Date(2026, 2, 1, 12).getTime() * 1000; +const DAY = join("2026", "2026-03", "2026-03-01"); + +type Row = { id: number; updationTime: number } & Record; + +// `plaintext` sealed under `key` with a fresh nonce, both base64, as Ente +// sends a key or an album name. +const secretbox = ( + plaintext: Uint8Array, + key: Uint8Array, +): { ciphertext: string; nonce: string } => { + const nonce = sodium.randombytes_buf(sodium.crypto_secretbox_NONCEBYTES); + return { + ciphertext: toBase64( + sodium.crypto_secretbox_easy(plaintext, nonce, key), + ), + nonce: toBase64(nonce), + }; +}; + +// What a corrupt row carries in place of its sealed key: random bytes of the +// same length, which no key opens. +const corruptKey = (): string => + toBase64( + sodium.randombytes_buf( + sodium.crypto_secretbox_KEYBYTES + sodium.crypto_secretbox_MACBYTES, + ), + ); + +const json = (body: unknown): Response => + new Response(JSON.stringify(body), { + status: 200, + headers: { "content-type": "application/json" }, + }); + +// A stand-in Ente server for one account. It holds each album, and each file in +// an album, as the row `/collections/v2` and `/collections/v2/diff` serve, and +// answers a request with the rows whose `updationTime` is past the request's +// `sinceTime`, as the real server does. A change to a file also moves its +// album's `updationTime`, which is what tells a client to list the album's +// files again. Originals are served by file ID, as the files CDN serves them. +class FakeEnte { + private readonly masterKey = sodium.crypto_secretbox_keygen(); + private readonly keyPair = sodium.crypto_box_keypair(); + private readonly albums = new Map(); + private readonly albumKeys = new Map(); + // Each album's file rows, by album ID and then file ID. + private readonly files = new Map>(); + // Each file's key and its original as served, by file ID. A file in two + // albums is one original under one key, sealed under each album's key. + private readonly originals = new Map< + number, + { key: Uint8Array; header: string; body: Uint8Array } + >(); + + // The session `quak login` would have saved for the account. + snapshot(): ClientSnapshot { + return { + email: "user@example.com", + userID: USER_ID, + token: "token", + masterKey: toBase64(this.masterKey), + secretKey: toBase64(this.keyPair.privateKey), + publicKey: toBase64(this.keyPair.publicKey), + }; + } + + // Album `id`, named `name`, as of time `t`. With `corrupt`, its key cannot + // be decrypted. An album keeps its key from one call to the next, so its + // files' rows still decrypt once it is put back without `corrupt`. + putAlbum(id: number, name: string, t: number, corrupt = false): void { + const key = this.albumKeys.get(id) ?? sodium.crypto_secretbox_keygen(); + this.albumKeys.set(id, key); + const sealedKey = secretbox(key, this.masterKey); + const sealedName = secretbox(new TextEncoder().encode(name), key); + this.albums.set(id, { + id, + owner: { id: USER_ID }, + encryptedKey: corrupt ? corruptKey() : sealedKey.ciphertext, + keyDecryptionNonce: sealedKey.nonce, + encryptedName: sealedName.ciphertext, + nameDecryptionNonce: sealedName.nonce, + type: "album", + updationTime: t, + }); + if (!this.files.has(id)) this.files.set(id, new Map()); + } + + // File `id`, titled `photo-.jpg`, in album `albumID` as of time `t`. + // With `corruptKey`, its key cannot be decrypted. With `metadata`, its + // metadata decrypts to that text instead of the JSON a client uploads. + putFile( + albumID: number, + id: number, + t: number, + opts: { corruptKey?: boolean; metadata?: string } = {}, + ): void { + const original = this.originalOf(id); + const sealedKey = secretbox(original.key, this.albumKeys.get(albumID)!); + const metadata = + opts.metadata ?? + JSON.stringify({ + title: `photo-${id}.jpg`, + fileType: 0, + creationTime: TAKEN, + modificationTime: TAKEN, + }); + const sealedMetadata = encryptBlob( + new TextEncoder().encode(metadata), + original.key, + ); + this.files.get(albumID)!.set(id, { + id, + collectionID: albumID, + ownerID: USER_ID, + encryptedKey: opts.corruptKey ? corruptKey() : sealedKey.ciphertext, + keyDecryptionNonce: sealedKey.nonce, + metadata: { + encryptedData: toBase64(sealedMetadata.ciphertext), + decryptionHeader: toBase64(sealedMetadata.header), + }, + file: { decryptionHeader: original.header }, + thumbnail: { decryptionHeader: original.header }, + updationTime: t, + }); + this.moveAlbum(albumID, t); + } + + // A deleted record stays on the server as a tombstone holding only its + // ID, its time and `isDeleted`. + deleteFile(albumID: number, id: number, t: number): void { + this.files + .get(albumID)! + .set(id, { id, updationTime: t, isDeleted: true }); + this.moveAlbum(albumID, t); + } + + deleteAlbum(id: number, t: number): void { + this.albums.set(id, { id, updationTime: t, isDeleted: true }); + } + + // The `fetch` the saved session's client is given. + readonly fetch = (async (input: string | URL | Request) => { + const url = new URL(input instanceof Request ? input.url : input); + const since = Number(url.searchParams.get("sinceTime")); + const changed = (rows: Iterable): Row[] => + [...rows].filter((row) => row.updationTime > since); + switch (url.pathname) { + case "/collections/v2": + return json({ collections: changed(this.albums.values()) }); + case "/collections/v2/diff": { + const albumID = Number(url.searchParams.get("collectionID")); + const rows = this.files.get(albumID)?.values() ?? []; + return json({ diff: changed(rows), hasMore: false }); + } + case "/files/data/fetch": + // No file has ML data. + return json({ data: [] }); + default: { + const fileID = Number(url.searchParams.get("fileID")); + const original = this.originals.get(fileID); + return original + ? new Response(original.body) + : new Response("not found", { status: 404 }); + } + } + }) as typeof globalThis.fetch; + + private originalOf(id: number): { + key: Uint8Array; + header: string; + body: Uint8Array; + } { + const known = this.originals.get(id); + if (known) return known; + const key = sodium.crypto_secretbox_keygen(); + const { header, ciphertext } = encryptBlob( + new TextEncoder().encode(`original of ${id}`), + key, + ); + const original = { key, header: toBase64(header), body: ciphertext }; + this.originals.set(id, original); + return original; + } + + private moveAlbum(albumID: number, t: number): void { + this.albums.set(albumID, { + ...this.albums.get(albumID)!, + updationTime: t, + }); + } +} + +// Album 1, "Trip", holds file 10, which decrypts, file 11, whose key does not, +// and file 12, whose metadata is not JSON. Album 2, "Family", holds file 20, +// but the album's own key does not decrypt. All of it as of time 1. +const brokenAccount = (): FakeEnte => { + const ente = new FakeEnte(); + ente.putAlbum(1, "Trip", 1); + ente.putFile(1, 10, 1); + ente.putFile(1, 11, 1, { corruptKey: true }); + ente.putFile(1, 12, 1, { metadata: "not json" }); + ente.putAlbum(2, "Family", 1, true); + ente.putFile(2, 20, 1); + return ente; +}; + +// Album 1, "Trip", holds file 10, and album 2, "Family", holds file 20. Both +// files decrypt, but album 2's own key does not, so the album is the only +// record left out. All of it as of time 1. +const brokenAlbumAccount = (): FakeEnte => { + const ente = new FakeEnte(); + ente.putAlbum(1, "Trip", 1); + ente.putFile(1, 10, 1); + ente.putAlbum(2, "Family", 1, true); + ente.putFile(2, 20, 1); + return ente; +}; + +// Collects everything written to it. +class Output { + text = ""; + write(text: string): void { + this.text += text; + } +} + +let root: string; +let stdout: Output; +let stderr: Output; + +beforeAll(async () => { + await init(); + await sodium.ready; +}); + +beforeEach(() => { + root = mkdtempSync(join(tmpdir(), "quak-undecryptable-")); +}); + +afterEach(() => { + rmSync(root, { recursive: true, force: true }); +}); + +// A context for one command, with fresh output, whose saved session talks to +// `ente`. Every run shares one cache directory, as runs of `quak` do. +const context = (ente: FakeEnte): CliContext => { + stdout = new Output(); + stderr = new Output(); + saveSession(join(root, "session"), ente.snapshot()); + return { + stdout, + stderr, + sessionDir: join(root, "session"), + cacheDir: join(root, "cache"), + loadSession: (path) => + loadSession(path, { fetch: ente.fetch, retry: { attempts: 1 } }), + login: async () => { + throw new Error("login not expected"); + }, + prompt: async () => { + throw new Error("prompt not expected"); + }, + promptSecret: async () => { + throw new Error("prompt not expected"); + }, + }; +}; + +const backupDir = (): string => join(root, "backup"); + +// Runs `quak backup` into the backup directory and returns its exit code. +const backup = (ente: FakeEnte, opts: { json?: boolean } = {}) => + backupCommand(context(ente), backupDir(), opts); + +// The backup's `failures.json` entries by file ID, or {} when it has none. +const failures = (): Record => { + const path = join(backupDir(), "failures.json"); + return existsSync(path) + ? JSON.parse(readFileSync(path, "utf-8")).files + : {}; +}; + +// Whether file `id`'s original is at its save path in the backup. +const stored = (id: number): boolean => + existsSync(join(backupDir(), DAY, `2026-03-01.${id}.jpg`)); + +// The IDs of the files that album `name`'s JSON lists. +const albumFiles = (name: string): number[] => + JSON.parse( + readFileSync(join(backupDir(), "collections", `${name}.json`), "utf-8"), + ).files.map((f: { id: number }) => f.id); + +describe("quak backup with records it cannot decrypt", () => { + it("backs up every other file, records each file it cannot decrypt in failures.json, names each such album, and exits 1", async () => { + expect(await backup(brokenAccount())).toBe(1); + + expect(stored(10)).toBe(true); + // Without its album's key, file 20 cannot even be listed. + expect(stored(20)).toBe(false); + expect(failures()).toEqual({ + "11": expect.objectContaining({ + fileID: 11, + error: expect.stringMatching( + /^could not be decrypted: .*wrong secret key/, + ), + }), + "12": expect.objectContaining({ + fileID: 12, + error: expect.stringMatching(/^could not be decrypted: .*JSON/), + }), + }); + expect(stderr.text).toMatch( + /^FAILED album 2: could not be decrypted: .*wrong secret key/m, + ); + expect(stderr.text).toContain(" Failed: 2\n"); + }); + + it("names each album it cannot decrypt in the --json result", async () => { + expect(await backup(brokenAccount(), { json: true })).toBe(1); + + expect(JSON.parse(stdout.text)).toMatchObject({ + downloaded: 1, + failed: 2, + undecryptableAlbums: [2], + }); + }); + + it("still reports them on the next run, though the server does not send them again", async () => { + const ente = brokenAccount(); + await backup(ente); + + // Nothing changed on the server, so this run's refresh gets no rows at + // all. What it reports, it remembered from the first run. + expect(await backup(ente)).toBe(1); + + expect(Object.keys(failures())).toEqual(["11", "12"]); + expect(failures()["11"]!.attempts).toBe(2); + expect(stderr.text).toMatch(/^FAILED album 2: could not be decrypted/m); + }); + + it("drops each once the server sends a copy that decrypts, and then exits 0", async () => { + const ente = brokenAccount(); + await backup(ente); + + // At time 2 files 11 and 12 and album 2 change on the server, and now + // each decrypts. Album 2 brings file 20 with it. + ente.putFile(1, 11, 2); + ente.putFile(1, 12, 2); + ente.putAlbum(2, "Family", 2); + expect(await backup(ente)).toBe(0); + + expect([11, 12, 20].map(stored)).toEqual([true, true, true]); + expect(existsSync(join(backupDir(), "failures.json"))).toBe(false); + expect(stderr.text).not.toContain("FAILED"); + }); + + it("drops each once the server deletes it, and then exits 0", async () => { + const ente = brokenAccount(); + await backup(ente); + + ente.deleteFile(1, 11, 2); + ente.deleteFile(1, 12, 2); + ente.deleteAlbum(2, 2); + expect(await backup(ente)).toBe(0); + + expect(existsSync(join(backupDir(), "failures.json"))).toBe(false); + expect(stderr.text).not.toContain("FAILED"); + }); + + it("exits 1 when the only record it cannot decrypt is an album, and 0 once the album decrypts", async () => { + const ente = brokenAlbumAccount(); + + // No file is in `failures.json`: the album alone keeps the exit code + // at 1. + expect(await backup(ente)).toBe(1); + expect(failures()).toEqual({}); + + ente.putAlbum(2, "Family", 2); + expect(await backup(ente)).toBe(0); + expect(stored(20)).toBe(true); + }); + + it("exits 1 when the only record it cannot decrypt is an album, and 0 once the album is deleted", async () => { + const ente = brokenAlbumAccount(); + + expect(await backup(ente)).toBe(1); + expect(failures()).toEqual({}); + + ente.deleteAlbum(2, 2); + expect(await backup(ente)).toBe(0); + }); + + it("removes what an earlier run linked for an album or a file that no longer decrypts, and keeps the originals", async () => { + // At time 1 everything decrypts: the first run links files 10 and 11 + // in Trip and file 20 in Family. + const ente = new FakeEnte(); + ente.putAlbum(1, "Trip", 1); + ente.putFile(1, 10, 1); + ente.putFile(1, 11, 1); + ente.putAlbum(2, "Family", 1); + ente.putFile(2, 20, 1); + expect(await backup(ente)).toBe(0); + const collections = join(backupDir(), "collections"); + expect(readdirSync(join(collections, "Trip")).sort()).toEqual([ + "photo-10.jpg", + "photo-11.jpg", + ]); + expect(readdirSync(join(collections, "Family"))).toEqual([ + "photo-20.jpg", + ]); + + // At time 2 the server's copy of file 11 in Trip, and album 2 itself, + // no longer decrypt. A run builds the album folders, their JSON and + // their links from what the library holds, so it removes them as if + // the album and the file had been deleted: Family's folder and JSON + // go, and Trip neither links nor lists file 11. The originals stay at + // their save paths, and the run exits 1. + ente.putFile(1, 11, 2, { corruptKey: true }); + ente.putAlbum(2, "Family", 2, true); + expect(await backup(ente)).toBe(1); + + expect(readdirSync(collections).sort()).toEqual(["Trip", "Trip.json"]); + expect(readdirSync(join(collections, "Trip"))).toEqual([ + "photo-10.jpg", + ]); + expect(albumFiles("Trip")).toEqual([10]); + expect([10, 11, 20].map(stored)).toEqual([true, true, true]); + expect(Object.keys(failures())).toEqual(["11"]); + }); + + it("backs up a file that does not decrypt in one album but does in another, and does not report it", async () => { + // File 30 is in albums 1 and 3, each holding its key sealed under the + // album's own key. In album 1 that is corrupt; in album 3 it is not, + // and one copy that decrypts is all the backup needs. + const ente = new FakeEnte(); + ente.putAlbum(1, "Trip", 1); + ente.putFile(1, 30, 1, { corruptKey: true }); + ente.putAlbum(3, "Favorites", 1); + ente.putFile(3, 30, 1); + + expect(await backup(ente)).toBe(0); + + expect(stored(30)).toBe(true); + expect(failures()).toEqual({}); + // Album 1 has no copy of file 30 that decrypts, so it neither links + // the original nor lists the file; album 3 lists it. + expect(readdirSync(join(backupDir(), "collections", "Trip"))).toEqual( + [], + ); + expect(albumFiles("Trip")).toEqual([]); + expect(albumFiles("Favorites")).toEqual([30]); + }); + + it("backs up that file in a run limited to the album where it does not decrypt", async () => { + // The same account, backed up by `lib.backup()` limited to album 1, + // "Trip", as `quak backup` cannot be. File 30 is in Trip, so the run + // must store it, from its copy in album 3 that the run otherwise + // skips. + const ente = new FakeEnte(); + ente.putAlbum(1, "Trip", 1); + ente.putFile(1, 30, 1, { corruptKey: true }); + ente.putAlbum(3, "Favorites", 1); + ente.putFile(3, 30, 1); + const lib = await Library.open({ + client: Client.fromJSON(ente.snapshot(), { + fetch: ente.fetch, + retry: { attempts: 1 }, + }), + downloadDirectory: backupDir(), + cacheDirectory: join(root, "cache"), + precacheThumbnails: false, + precacheOriginals: false, + }); + try { + const result = await lib.backup({ + downloadDirectory: backupDir(), + onlyAlbumNames: ["Trip"], + }); + expect(result).toMatchObject({ downloaded: 1, failed: 0 }); + } finally { + await lib.close(); + } + + expect(stored(30)).toBe(true); + expect(failures()).toEqual({}); + }); +}); + +describe("the read commands with records quak cannot decrypt", () => { + it("quak collections and quak files answer from what decrypts", async () => { + const ente = brokenAccount(); + + expect(await collectionsCommand(context(ente), {})).toBe(0); + expect(stdout.text).toBe("1\talbum\tTrip\n"); + + expect(await filesCommand(context(ente), { collection: "1" })).toBe(0); + expect(stdout.text).toBe("10\timage\tphoto-10.jpg\n"); + }); +}); diff --git a/test/cli/backup.test.ts b/test/cli/backup.test.ts index 977a584..065a2c5 100644 --- a/test/cli/backup.test.ts +++ b/test/cli/backup.test.ts @@ -1482,6 +1482,7 @@ describe("backup album folders", () => { listCollections: () => albums.map((a) => a.collection), listFiles: (id) => albums.find((a) => a.collection.id === id)?.files ?? [], + undecryptable: () => [], original: async (fileID) => { const path = join(root, `source-${fileID}`); writeFileSync(path, `original ${fileID}`); diff --git a/test/client/enumeration.test.ts b/test/client/enumeration.test.ts index 363f27d..49fcb07 100644 --- a/test/client/enumeration.test.ts +++ b/test/client/enumeration.test.ts @@ -10,7 +10,9 @@ * The return shape keeps live records and tombstones apart — `collections` / * `files` are decrypted live records, `deleted` is a plain list of the ids the * server tombstoned. A tombstone carries no decryptable key or metadata, so it - * is a bare id rather than a hollowed-out `Collection` / `EnteFile`. + * is a bare id rather than a hollowed-out `Collection` / `EnteFile`. A live row + * that does not decrypt, or whose metadata is not JSON, is listed under + * `undecryptable` with the reason, so one bad row does not fail the call. * * All tests inject a fake `fetch` and drive a real `Client` (built with * `Client.fromJSON`) so the decryption path runs for real. Live rows are built @@ -91,27 +93,38 @@ const ownedCollectionRow = ( }; }; -/** A live file row inside a collection, keyed under that collection's key. */ +/** + * A live file row inside a collection, keyed under that collection's key. Its + * metadata decrypts to `metadata` when given, otherwise to the JSON a client + * uploads for `title`. + */ const fileRow = ( collectionKey: Uint8Array, - opts: { id: number; title: string; updationTime: number }, + opts: { + id: number; + title: string; + updationTime: number; + metadata?: string; + }, ): Record => { const fileKey = sodium.crypto_secretbox_keygen(); const { ciphertext: encFileKey, nonce: fileKeyNonce } = secretboxEncrypt( fileKey, collectionKey, ); - const metadata = { - title: opts.title, - fileType: 0, - creationTime: opts.updationTime, - modificationTime: opts.updationTime, - }; + const metadata = + opts.metadata ?? + JSON.stringify({ + title: opts.title, + fileType: 0, + creationTime: opts.updationTime, + modificationTime: opts.updationTime, + }); const push = sodium.crypto_secretstream_xchacha20poly1305_init_push(fileKey); const encMeta = sodium.crypto_secretstream_xchacha20poly1305_push( push.state, - new TextEncoder().encode(JSON.stringify(metadata)), + new TextEncoder().encode(metadata), null, sodium.crypto_secretstream_xchacha20poly1305_TAG_FINAL, ); @@ -131,6 +144,21 @@ const fileRow = ( }; }; +/** + * `row` with its sealed key replaced by random bytes of the same length, as a + * corrupt row arrives: no key opens it, so the row cannot be decrypted. + */ +const withCorruptKey = ( + row: Record, +): Record => ({ + ...row, + encryptedKey: toBase64( + sodium.randombytes_buf( + sodium.crypto_secretbox_KEYBYTES + sodium.crypto_secretbox_MACBYTES, + ), + ), +}); + /** A tombstone row. Never decrypted, so only these fields are ever read. */ const tombstoneRow = ( id: number, @@ -278,6 +306,58 @@ describe("Client.filesSince", () => { // It gave up after the first page rather than looping. expect(sinceTimes).toEqual(["50"]); }); + + it("leaves out each file it cannot decrypt or parse, with the reason, and decrypts the rest", async () => { + const keys = buildKeys(); + const collectionKey = sodium.crypto_secretbox_keygen(); + + // File 1002's key is sealed under no key quak holds, and file 1003's + // metadata decrypts to text that is not JSON. Neither may cost the + // caller file 1001: each is left out of `files` and reported under + // `undecryptable` with the reason instead of failing the call. + const { fetch } = recordingFetch( + jsonResponse({ + diff: [ + fileRow(collectionKey, { + id: 1001, + title: "fine.jpg", + updationTime: 100, + }), + withCorruptKey( + fileRow(collectionKey, { + id: 1002, + title: "bad-key.jpg", + updationTime: 200, + }), + ), + fileRow(collectionKey, { + id: 1003, + title: "bad-metadata.jpg", + updationTime: 300, + metadata: "not json", + }), + ], + hasMore: false, + }), + ); + const client = Client.fromJSON(snapshotFor(keys), { fetch }); + + const { files, undecryptable, cursor } = await client.filesSince({ + collectionID: 1, + collectionKey, + sinceTime: 0, + }); + + expect(files.map((f) => f.id)).toEqual([1001]); + expect(undecryptable).toEqual([ + { id: 1002, error: expect.stringContaining("wrong secret key") }, + { id: 1003, error: expect.stringContaining("JSON") }, + ]); + // The cursor moves past the two left out, as past any other row, so + // the server does not send them again until its copy changes. The + // caller has to keep what it was told here. + expect(cursor).toBe(300); + }); }); describe("Client.collectionsSince", () => { @@ -331,6 +411,42 @@ describe("Client.collectionsSince", () => { expect(result.cursor).toBe(150); expect(sinceTimes).toEqual(["150"]); }); + + it("leaves out an album it cannot decrypt, with the reason, and decrypts the rest", async () => { + const keys = buildKeys(); + + // Album 2's key is sealed under no key quak holds. Without its key + // neither its name nor its files can be read, so it is left out and + // reported under `undecryptable`; album 1 is unaffected. + const { fetch } = recordingFetch( + jsonResponse({ + collections: [ + ownedCollectionRow(keys.masterKey, { + id: 1, + name: "Vacation", + updationTime: 100, + }), + withCorruptKey( + ownedCollectionRow(keys.masterKey, { + id: 2, + name: "Broken", + updationTime: 200, + }), + ), + ], + }), + ); + const client = Client.fromJSON(snapshotFor(keys), { fetch }); + + const { collections, undecryptable, cursor } = + await client.collectionsSince({ sinceTime: 0 }); + + expect(collections.map((c) => c.name)).toEqual(["Vacation"]); + expect(undecryptable).toEqual([ + { id: 2, error: expect.stringContaining("wrong secret key") }, + ]); + expect(cursor).toBe(200); + }); }); describe("Client list wrappers still hide deletions", () => { diff --git a/test/library/store.test.ts b/test/library/store.test.ts index 21212b3..38e7e25 100644 --- a/test/library/store.test.ts +++ b/test/library/store.test.ts @@ -172,6 +172,29 @@ describe("MetadataStore", () => { expect(store.listCollections()).toEqual([]); }); + it("loads an empty store when the list of what the refresh could not decrypt is damaged", async () => { + // A backup reads that list on every run, so a damaged one that loaded + // would fail every backup until the cache was deleted. A string is + // damaged too, though JavaScript can loop over its characters. + mkdirSync(join(dir, "cache"), { recursive: true }); + for (const undecryptable of [{}, "not a list", [1]]) { + writeFileSync( + path, + JSON.stringify({ + schemaVersion: METADATA_SCHEMA_VERSION, + userID: 42, + collectionsSinceTime: 0, + collections: [], + files: [], + undecryptable, + }), + ); + const store = await MetadataStore.load(path); + expect(store.userID).toBe(0); + expect(store.listUndecryptable()).toEqual([]); + } + }); + it("creates the directory 0700 and the file 0600", async () => { const store = await MetadataStore.load(path); store.putCollection(sampleCollection());