1 Commits
Author SHA1 Message Date
sneak 2e885d54e5 Refresh before backup-metadata and the thumbnail helpers answer (closes #100)
check / check (push) Successful in 1m27s
backup-metadata, helper list-missing-thumbnails and helper
fix-missing-thumbnails now await lib.fresh() before reading the library,
as collections, files, get and get-thumb already do. A file added since
the cache was written is included, and a failed refresh is thrown, so
the CLI prints one line and exits 1 instead of answering from a stale
or empty cache. The README lists them among the commands that refresh
first.

Model: opus-5-5
2026-09-23 06:08:00 +00:00
10 changed files with 59 additions and 349 deletions
+28 -61
View File
@@ -214,23 +214,11 @@ quak/
auth/ login flow (SRP + email OTP + TOTP), key unwrap auth/ login flow (SRP + email OTP + TOTP), key unwrap
model/ decrypted Collection, File, Metadata types + decrypt fns model/ decrypted Collection, File, Metadata types + decrypt fns
download/ streaming file/thumbnail download + decryption download/ streaming file/thumbnail download + decryption
library/ the cache-backed Library: metadata store, read
surface, records, content cache, precache, ML data
and search, request pools
backup.ts resilient full-account backup with dedup backup.ts resilient full-account backup with dedup
metadata-backup.ts
backup-metadata: all decrypted metadata as JSON
mldata-fetch.ts fetch + decrypt per-file ML data
filename.ts safe file names from server metadata
errors.ts error types shared across layers errors.ts error types shared across layers
retry.ts retry classifier + exponential backoff with jitter retry.ts retry classifier + exponential backoff with jitter
thumbnails.ts detect + regenerate missing thumbnails thumbnails.ts detect + regenerate missing thumbnails
client.ts high-level Client class assembled from the above client.ts high-level Client class assembled from the above
cli-commands.ts the CLI's commands as functions returning exit codes
cli-output.ts how the CLI prints a file's title and time
cli-read.ts fresh reads for the CLI's read commands
cli-run.ts run a command, print its error, exit with its code
cli-session.ts read the saved session file back into a Client
index.ts public library exports index.ts public library exports
bin/ bin/
quak.ts CLI entrypoint (commander.js) quak.ts CLI entrypoint (commander.js)
@@ -505,15 +493,8 @@ the smallest does not.
<name>/ <name>/
<title> -> ../../originals/<fileID>.<ext> (symlink) <title> -> ../../originals/<fileID>.<ext> (symlink)
<name>.json collection metadata + file list <name>.json collection metadata + file list
failures.json files that failed and have not yet succeeded
``` ```
`failures.json` records each failed file with the kind of failure, how many
times it has been tried and when it was last tried. A file leaves it once it
succeeds, or once it is no longer in the library or in the backup's scope. The
library's `lib.backup({ includeThumbnails: true })` also writes
`thumbnails/<fileID>.jpg` beside `originals/`; `quak backup` does not.
A collection's directory and JSON are named after the collection, and a symlink A collection's directory and JSON are named after the collection, and a symlink
after the file's title, both with unsafe characters replaced. When two after the file's title, both with unsafe characters replaced. When two
collections would get the same name, or two files in one collection the same collections would get the same name, or two files in one collection the same
@@ -529,25 +510,22 @@ symlink you put there stays, and a directory that still holds one after its
symlinks are removed stays too, with its JSON. symlinks are removed stays too, with its JSON.
Each file is downloaded exactly once regardless of how many collections it Each file is downloaded exactly once regardless of how many collections it
appears in, and written once: straight into `originals/`, with no copy left in appears in. On subsequent runs, existing originals are skipped. If a download
the cache. An original the cache already held is copied from there instead. On fails, the error is logged and the backup continues with the next file. The exit
subsequent runs, existing originals are skipped. If a download fails, the error code is non-zero if any files failed.
is logged and the backup continues with the next file. The exit code is non-zero
if any files failed. `quak backup` opens its library with the thumbnail and
originals precache off, so it fetches only what the backup stores.
Each original is written to a temporary file in the same directory, synced to Each original is copied to a temporary file named
disk, and renamed into place, so an original is either complete or absent, even `.quak-backup-<fileID>.<ext>-<pid>-<random>.tmp` in the same directory, synced
after a power cut. A downloaded original's temporary file is named to disk, and renamed into place, so an original is either complete or absent,
`.quak-<pid>-<random>.tmp`, one copied from the cache even after a power cut. A run that is killed can leave one of these temporary
`.quak-backup-<fileID>.<ext>-<pid>-<random>.tmp`. A run that is killed can leave files behind; the next backup deletes those whose process is no longer running.
one of these temporary files behind; the next backup deletes those whose process Downloads and the content cache use the same scheme with
is no longer running. The content cache uses the same scheme, and opening a `.quak-<pid>-<random>.tmp` names, and opening a library deletes those in the
library deletes the temporary files in the cache whose process is no longer cache whose process is no longer running, so a download another process has in
running, so a download another process has in progress in the same cache is left progress in the same cache is left alone. The rename replaces whatever was at
alone. The rename replaces whatever was at the destination rather than writing the destination rather than writing through it: a symlink there is replaced, not
through it: a symlink there is replaced, not followed, and the new file has the followed, and the new file has the temporary file's permissions, not those of
temporary file's permissions, not those of the file it replaced. the file it replaced.
## TODO ## TODO
@@ -555,12 +533,7 @@ temporary file's permissions, not those of the file it replaced.
errors errors
- [x] Update the API reference section below to match the current implementation - [x] Update the API reference section below to match the current implementation
- [x] `make docker` green - [x] `make docker` green
- [ ] Store live photos in a form a photo viewer can open - [ ] Tag `v1.0.0`
(https://git.eeqj.de/sneak/quak/issues/107), once sneak has chosen between
keeping the ZIP and unpacking it
Tagging and releases are decided by sneak alone, and happen only when he
declares one.
Future (desktop client, separate repo): Future (desktop client, separate repo):
@@ -580,11 +553,10 @@ test suite is the canonical, executable documentation — `test/library/` and
### Opening a library ### Opening a library
`Library.open(options)` loads the on-disk cache, starts the background refresh `Library.open(options)` loads the on-disk cache, starts the background refresh
loop, and resolves to a `Library`. On an empty cache it awaits the first loop, and resolves to a `Library`. On an empty cache it awaits the first refresh
refresh, so it opens onto the account's data whenever the server is reachable; so it never opens onto empty data; on an existing cache it returns immediately
if that refresh fails, it opens with no data and records the error in and refreshes in the background, so an unreachable server does not block
`lib.status()`. On an existing cache it returns immediately and refreshes in the opening.
background, so an unreachable server does not block opening.
`LibraryOptions`: `LibraryOptions`:
@@ -695,15 +667,12 @@ photos newest first). `lib.subscribe({ onChange })` delivers a `LibraryChange`
default limit 20). quak bundles no text encoder, so `searchByEmbedding` takes default limit 20). quak bundles no text encoder, so `searchByEmbedding` takes
a query vector the caller produced elsewhere. a query vector the caller produced elsewhere.
- `await lib.backup(opts?)` → `BackupResult`. It refreshes, fetches every - `await lib.backup(opts?)` → `BackupResult`. It refreshes, fetches every
in-scope original not already in the backup (and, with `includeThumbnails`, in-scope original (and, with `includeThumbnails`, thumbnails) through the
thumbnails) through the content cache, and rebuilds the on-disk backup tree content cache, and rebuilds the on-disk backup tree with a durable failure
with a durable failure ledger. A fetched original is written straight into the ledger. `BackupOptions`: `downloadDirectory` (falls back to the one `open()`
backup's `originals/` and not into the cache, which then counts it as present; was given), `includeOriginals` (default `true`), `includeThumbnails` (default
one the cache already held is copied from there. `BackupOptions`: `false`), `onlyAlbumNames`, and `onProgress`. See Backup layout above for the
`downloadDirectory` (falls back to the one `open()` was given), tree it writes.
`includeOriginals` (default `true`), `includeThumbnails` (default `false`),
`onlyAlbumNames`, and `onProgress`. See Backup layout above for the tree it
writes.
### Request pools ### Request pools
@@ -800,10 +769,8 @@ documents:
markdown. Use `make fmt` to format. Use `yarn` not `npm`. markdown. Use `make fmt` to format. Use `yarn` not `npm`.
- **Testing:** vitest. Tests go in `test/` mirroring the `src/` structure. - **Testing:** vitest. Tests go in `test/` mirroring the `src/` structure.
`make test` must finish in under 60 seconds (the hard cap) and should finish `make test` must complete in under 20 seconds. Use `mkdtempSync` for temporary
in under 20. The 90-second `timeout` in the `test` phase of the `Dockerfile` directories, never manual timestamp paths.
is a backstop that catches a hung test, not the time limit. Use `mkdtempSync`
for temporary directories, never manual timestamp paths.
- **Code style:** `const` for everything, `let` if reassignment is needed, never - **Code style:** `const` for everything, `let` if reassignment is needed, never
`var`. Avoid unnecessary comments. No hand-rolled crypto. The `var`. Avoid unnecessary comments. No hand-rolled crypto. The
+1 -35
View File
@@ -14,44 +14,10 @@ pre-1.0
# Next Step # Next Step
Store live photos in a form a photo viewer can open Tag v1.0.0.
(https://git.eeqj.de/sneak/quak/issues/107). This waits on sneak's choice
between keeping the ZIP and unpacking it into the image and the video.
Tagging and releases are decided by sneak alone, and happen only when he
declares one.
# Completed Steps # Completed Steps
- 2026-09-23: Settled the package metadata (issue 6). quak is not published, so
`package.json` is marked `"private": true` and the `files` field is gone.
`engines.node` is `>=22`, the major version `script/bootstrap` and the
`Dockerfile` use. An `exports` map makes `.` and `./package.json` the only
importable paths; `runMetadataBackup`, the thumbnail helpers and their types
stay internal to the CLI.
- 2026-09-23: Brought the README and this file in line with the tree (issue
111). The layout lists `src/library/` and the other source files, the backup
layout names `failures.json` and the optional `thumbnails/`, "Opening a
library" says what happens when the first refresh fails, the Testing section
gives the 60-second hard cap and 20-second target for `make test` and names
the 90-second `timeout` in the `Dockerfile` as the backstop for a hung test,
and "Tag v1.0.0" is no longer listed as the next step.
- 2026-09-23: Tested `quak login` and `backup-metadata --exif` (issue 110).
`loginCommand` takes its login function and its prompts from `CliContext`, and
`bin/quak.ts` passes `Client.login` and the terminal prompts. Tests cover a
login from `QUAK_EMAIL` and `QUAK_PASSWORD` with no prompt, the TOTP prompt, a
failed login, and the saved session's modes, and show that `--exif` and
`--all` each turn on EXIF extraction and that it is off without them.
- 2026-09-23: `quak backup` writes each original once and no longer fills the
cache (issue 106). An original fetched for a backup is written by the download
writer straight into the backup's `originals/`, and the content cache records
it there instead of keeping its own copy; one the cache already held is still
copied. `quak backup` opens its library with the thumbnail and originals
precache off.
- 2026-09-23: `backup-metadata`, `helper list-missing-thumbnails` and - 2026-09-23: `backup-metadata`, `helper list-missing-thumbnails` and
`helper fix-missing-thumbnails` refresh before they answer (issue 100). Each `helper fix-missing-thumbnails` refresh before they answer (issue 100). Each
awaits `lib.fresh()` before reading, so a file added since the cache was awaits `lib.fresh()` before reading, so a file added since the cache was
-5
View File
@@ -1,7 +1,6 @@
#!/usr/bin/env node #!/usr/bin/env node
import { stdout, stderr } from "node:process"; import { stdout, stderr } from "node:process";
import { input, password } from "@inquirer/prompts";
import { Command } from "commander"; import { Command } from "commander";
import envPaths from "env-paths"; import envPaths from "env-paths";
import { init } from "../src/crypto/index.js"; import { init } from "../src/crypto/index.js";
@@ -21,7 +20,6 @@ import {
} from "../src/cli-commands.js"; } from "../src/cli-commands.js";
import { run as runCommand } from "../src/cli-run.js"; import { run as runCommand } from "../src/cli-run.js";
import { loadSession } from "../src/cli-session.js"; import { loadSession } from "../src/cli-session.js";
import { Client } from "../src/client.js";
import { VERSION } from "../src/index.js"; import { VERSION } from "../src/index.js";
const paths = envPaths("quak", { suffix: "" }); const paths = envPaths("quak", { suffix: "" });
@@ -44,9 +42,6 @@ const context = (): CliContext => ({
sessionDir: paths.data, sessionDir: paths.data,
cacheDir: program.opts<{ cacheDir?: string }>().cacheDir, cacheDir: program.opts<{ cacheDir?: string }>().cacheDir,
loadSession, loadSession,
login: (opts) => Client.login(opts),
prompt: (message) => input({ message }),
promptSecret: (message) => password({ message, mask: true }),
}); });
const run = (command: Promise<number>): Promise<void> => const run = (command: Promise<number>): Promise<void> =>
+5 -11
View File
@@ -9,23 +9,17 @@
"type": "git", "type": "git",
"url": "https://git.eeqj.de/sneak/quak.git" "url": "https://git.eeqj.de/sneak/quak.git"
}, },
"private": true,
"type": "module", "type": "module",
"engines": {
"node": ">=22"
},
"main": "./dist/src/index.js", "main": "./dist/src/index.js",
"types": "./dist/src/index.d.ts", "types": "./dist/src/index.d.ts",
"exports": {
".": {
"types": "./dist/src/index.d.ts",
"import": "./dist/src/index.js"
},
"./package.json": "./package.json"
},
"bin": { "bin": {
"quak": "./dist/bin/quak.js" "quak": "./dist/bin/quak.js"
}, },
"files": [
"dist/",
"README.md",
"LICENSE"
],
"scripts": { "scripts": {
"build": "script/build", "build": "script/build",
"quak": "node ./dist/bin/quak.js", "quak": "node ./dist/bin/quak.js",
+4 -8
View File
@@ -4,8 +4,7 @@
// fails the backup before any file is touched), then, for every file in scope, // fails the backup before any file is touched), then, for every file in scope,
// gets its original bytes onto disk under `downloadDirectory` and rebuilds the // gets its original bytes onto disk under `downloadDirectory` and rebuilds the
// derived views (per-file sidecars, per-collection symlink trees, // derived views (per-file sidecars, per-collection symlink trees,
// per-collection JSON) from the model. The on-disk layout is the historical // per-collection JSON) from the model. The on-disk layout is the historical one, unchanged:
// one, unchanged:
// //
// <downloadDirectory>/ // <downloadDirectory>/
// originals/<fileID>.<ext> the decrypted bytes // originals/<fileID>.<ext> the decrypted bytes
@@ -97,9 +96,8 @@ export interface BackupLibrary {
listCollections(): Collection[]; listCollections(): Collection[];
listFiles(collectionID: number): EnteFile[]; listFiles(collectionID: number): EnteFile[];
// Get an original's bytes onto disk through the content cache/pools, // Get an original's bytes onto disk through the content cache/pools,
// returning where they landed: `destination` when they were fetched now, // returning where they landed (the cache, or a prior backup).
// otherwise wherever they already were (the cache, or a prior backup). original(fileID: number): Promise<{ path: string }>;
original(fileID: number, destination: string): Promise<{ path: string }>;
thumbnail(fileID: number): Promise<{ path: string }>; thumbnail(fileID: number): Promise<{ path: string }>;
} }
@@ -426,9 +424,7 @@ export const runBackup = async (
} }
try { try {
log(`Fetching original ${file.metadata.title} (${fileID})...`); log(`Fetching original ${file.metadata.title} (${fileID})...`);
// A fetched original is written straight to `dest`; only one const { path } = await lib.original(fileID);
// that was already cached elsewhere is copied.
const { path } = await lib.original(fileID, dest);
await copyAtomic(path, dest); await copyAtomic(path, dest);
downloaded++; downloaded++;
} catch (err) { } catch (err) {
+14 -21
View File
@@ -4,8 +4,10 @@
// code; a thrown error is left to the caller. Nothing here calls // code; a thrown error is left to the caller. Nothing here calls
// `process.exit`: `bin/quak.ts` wires these to the command line, and `run` in // `process.exit`: `bin/quak.ts` wires these to the command line, and `run` in
// `cli-run.ts` prints a thrown error as one line and exits once output has // `cli-run.ts` prints a thrown error as one line and exits once output has
// drained. Output must stay byte-identical (see `cli-output.ts`). // drained. Output must stay byte-identical
// (see `cli-output.ts`).
import { input, password as passwordPrompt } from "@inquirer/prompts";
import { import {
copyFileSync, copyFileSync,
existsSync, existsSync,
@@ -14,11 +16,7 @@ import {
writeFileSync, writeFileSync,
} from "node:fs"; } from "node:fs";
import { join } from "node:path"; import { join } from "node:path";
import { import { Client, type ClientSnapshot } from "./client.js";
type Client,
type ClientSnapshot,
type LoginOptions,
} from "./client.js";
import { init } from "./crypto/index.js"; import { init } from "./crypto/index.js";
import { import {
defaultCacheDirectory, defaultCacheDirectory,
@@ -46,12 +44,6 @@ export interface CliContext {
// Reads the session file into a client, or null when there is none. The // Reads the session file into a client, or null when there is none. The
// CLI passes `loadSession` from `cli-session.ts`; tests pass a fake client. // CLI passes `loadSession` from `cli-session.ts`; tests pass a fake client.
loadSession: (path: string) => Client | null; loadSession: (path: string) => Client | null;
// Used by `login` only. The CLI passes `Client.login` and terminal
// prompts; tests pass fakes.
login: (opts: LoginOptions) => Promise<Client>;
prompt: (message: string) => Promise<string>;
// Like `prompt`, but the answer is masked as it is typed.
promptSecret: (message: string) => Promise<string>;
} }
const sessionPath = (ctx: CliContext): string => const sessionPath = (ctx: CliContext): string =>
@@ -117,19 +109,24 @@ const openReadLibrary = (ctx: CliContext, client: Client): Promise<Library> =>
precacheOriginals: false, precacheOriginals: false,
}); });
const prompt = async (message: string): Promise<string> => input({ message });
const promptSecret = async (message: string): Promise<string> =>
passwordPrompt({ message, mask: true });
export const loginCommand = async (ctx: CliContext): Promise<number> => { export const loginCommand = async (ctx: CliContext): Promise<number> => {
await init(); await init();
const email = process.env.QUAK_EMAIL ?? (await ctx.prompt("Email")); const email = process.env.QUAK_EMAIL ?? (await prompt("Email"));
const password = const password =
process.env.QUAK_PASSWORD ?? (await ctx.promptSecret("Password")); process.env.QUAK_PASSWORD ?? (await promptSecret("Password"));
ctx.stderr.write("Authenticating...\n"); ctx.stderr.write("Authenticating...\n");
try { try {
const client = await ctx.login({ const client = await Client.login({
email, email,
password, password,
totp: async () => ctx.prompt("TOTP code: "), totp: async () => prompt("TOTP code: "),
emailOTP: async () => ctx.prompt("Email verification code: "), emailOTP: async () => prompt("Email verification code: "),
}); });
saveSession(ctx.sessionDir, client.toJSON()); saveSession(ctx.sessionDir, client.toJSON());
@@ -383,14 +380,10 @@ export const backupCommand = async (
if (!client) return 1; if (!client) return 1;
ctx.stderr.write("Starting backup...\n"); ctx.stderr.write("Starting backup...\n");
// The precache is off: the backup fetches what it needs, and must not
// also fill the cache with every thumbnail and the recent originals.
const lib = await Library.open({ const lib = await Library.open({
client, client,
downloadDirectory: dir, downloadDirectory: dir,
cacheDirectory: ctx.cacheDir, cacheDirectory: ctx.cacheDir,
precacheThumbnails: false,
precacheOriginals: false,
}); });
try { try {
const result = await lib.backup({ const result = await lib.backup({
+3 -21
View File
@@ -321,23 +321,6 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
return this.get(fileID, "thumbnail", "on-demand", opts?.onProgress); return this.get(fileID, "thumbnail", "on-demand", opts?.onProgress);
} }
// Get an original for a backup. One not present anywhere is written
// straight to `destination` and recorded there, so no second copy lands
// in the cache; one already present is returned where it is.
async backupOriginal(
fileID: number,
destination: string,
): Promise<ContentResult> {
const result = await this.acquire(
fileID,
"original",
"on-demand",
undefined,
{ destination },
);
return { path: result.path, bytes: result.bytes };
}
async ensure(args: EnsureOptions): Promise<EnsureResult[]> { async ensure(args: EnsureOptions): Promise<EnsureResult[]> {
return this.ensureThumbnails(args); return this.ensureThumbnails(args);
} }
@@ -439,14 +422,13 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
// The core: return the cached path if present, else fetch through the pool, // The core: return the cached path if present, else fetch through the pool,
// store, and return it. `cached` distinguishes a present hit (no network, // store, and return it. `cached` distinguishes a present hit (no network,
// no download event) from a fresh fetch. A fetched original is stored at // no download event) from a fresh fetch.
// `opts.destination` when given, instead of in `originalsDir`.
private async acquire( private async acquire(
fileID: number, fileID: number,
kind: Kind, kind: Kind,
priority: Priority, priority: Priority,
signal: AbortSignal | undefined, signal: AbortSignal | undefined,
opts?: { onByte?: ProgressCallback; destination?: string }, opts?: { onByte?: ProgressCallback },
): Promise<{ path: string; bytes: number; cached: boolean }> { ): Promise<{ path: string; bytes: number; cached: boolean }> {
const file = this.getFile(fileID); const file = this.getFile(fileID);
if (!file) throw new Error(`content cache: unknown file ${fileID}`); if (!file) throw new Error(`content cache: unknown file ${fileID}`);
@@ -487,7 +469,7 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
kind === "original" ? this.originalsDir : this.thumbnailsDir; kind === "original" ? this.originalsDir : this.thumbnailsDir;
const dest = const dest =
kind === "original" kind === "original"
? (opts?.destination ?? join(dir, originalName(file))) ? join(dir, originalName(file))
: join(dir, `${fileID}${THUMBNAIL_EXT}`); : join(dir, `${fileID}${THUMBNAIL_EXT}`);
const pool = const pool =
kind === "original" ? this.pools.content : this.pools.thumbnails; kind === "original" ? this.pools.content : this.pools.thumbnails;
+1 -2
View File
@@ -571,8 +571,7 @@ export class Library {
refresh: () => this.refreshNow(), refresh: () => this.refreshNow(),
listCollections: () => this.store.listCollections(), listCollections: () => this.store.listCollections(),
listFiles: (id) => this.store.listFiles(id), listFiles: (id) => this.store.listFiles(id),
original: (fileID, destination) => original: (fileID) => cache!.original(fileID),
cache!.backupOriginal(fileID, destination),
thumbnail: (fileID) => cache!.thumbnail(fileID), thumbnail: (fileID) => cache!.thumbnail(fileID),
}, },
{ ...opts, downloadDirectory }, { ...opts, downloadDirectory },
+1 -29
View File
@@ -562,39 +562,11 @@ describe("lib.backup", () => {
lib.close(); lib.close();
}); });
it("fetches each original once and writes it only into the backup", async () => { it("fsyncs a copied original before the rename and its directory after", async () => {
// A backup of a 500 GB account must write 500 GB, not a copy in the
// cache as well: an original fetched for the backup goes straight
// into its originals/, and the cache records it there.
const source = stubSource();
const lib = await openLibrary(source);
const outDir = join(root, "backup");
const result = await lib.backup({ downloadDirectory: outDir });
expect(result.downloaded).toBe(3);
expect(source.originalCalls).toBe(3);
expect(readdirSync(join(root, "cache", "originals"))).toEqual([]);
const stored = readdirSync(join(outDir, "originals")).filter(
(name) => !name.endsWith(".json"),
);
expect(stored.sort()).toEqual(["100.jpg", "101.jpg", "200.png"]);
// The cache counts the backup's copy as present: reading the
// original afterwards fetches nothing and answers with that copy.
const read = await lib.photos.byID({ fileID: 100 })!.original();
expect(read.path).toBe(join(outDir, "originals", "100.jpg"));
expect(source.originalCalls).toBe(3);
await lib.close();
});
it("fsyncs an original copied from the cache before the rename and its directory after", async () => {
const lib = await openLibrary(stubSource()); const lib = await openLibrary(stubSource());
const outDir = join(root, "backup"); const outDir = join(root, "backup");
const originals = join(outDir, "originals"); const originals = join(outDir, "originals");
const dest = join(originals, "100.jpg"); const dest = join(originals, "100.jpg");
// Only an original already in the cache is copied into the backup;
// one fetched for the backup is written there by the download writer.
await lib.photos.byID({ fileID: 100 })!.original();
fsEvents.length = 0; fsEvents.length = 0;
await lib.backup({ downloadDirectory: outDir }); await lib.backup({ downloadDirectory: outDir });
+2 -156
View File
@@ -13,7 +13,6 @@
import { import {
existsSync, existsSync,
mkdtempSync, mkdtempSync,
readdirSync,
readFileSync, readFileSync,
rmSync, rmSync,
statSync, statSync,
@@ -22,20 +21,11 @@ import {
import { join } from "node:path"; import { join } from "node:path";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import { PassThrough } from "node:stream"; import { PassThrough } from "node:stream";
import { import { describe, it, expect, beforeAll, beforeEach, afterEach } from "vitest";
describe,
it,
expect,
vi,
beforeAll,
beforeEach,
afterEach,
} from "vitest";
import { import {
type CliContext, type CliContext,
saveSession, saveSession,
loginCommand,
whoamiCommand, whoamiCommand,
logoutCommand, logoutCommand,
collectionsCommand, collectionsCommand,
@@ -49,7 +39,7 @@ import {
} from "../../src/cli-commands.js"; } from "../../src/cli-commands.js";
import { run } from "../../src/cli-run.js"; import { run } from "../../src/cli-run.js";
import { loadSession } from "../../src/cli-session.js"; import { loadSession } from "../../src/cli-session.js";
import type { Client, ClientSnapshot, LoginOptions } from "../../src/client.js"; import type { Client, ClientSnapshot } from "../../src/client.js";
import type { ContentSource } from "../../src/library/content.js"; import type { ContentSource } from "../../src/library/content.js";
import type { Collection, EnteFile } from "../../src/model/types.js"; import type { Collection, EnteFile } from "../../src/model/types.js";
import { init, toBase64 } from "../../src/crypto/index.js"; import { init, toBase64 } from "../../src/crypto/index.js";
@@ -175,15 +165,6 @@ const context = (client: Client | null = fakeClient()): CliContext => ({
sessionDir: join(root, "session"), sessionDir: join(root, "session"),
cacheDir: join(root, "cache"), cacheDir: join(root, "cache"),
loadSession: () => client, loadSession: () => client,
login: async () => {
throw new Error("login not expected");
},
prompt: async () => {
throw new Error("prompt not expected");
},
promptSecret: async () => {
throw new Error("prompt not expected");
},
}); });
beforeAll(async () => { beforeAll(async () => {
@@ -241,105 +222,6 @@ describe("session file", () => {
}); });
}); });
// The login function is a fake that hands back a client whose snapshot is
// `snapshot`; each prompt is recorded and answered with "123456".
describe("login", () => {
const snapshot: ClientSnapshot = {
email: "cli@example.com",
userID: USER_ID,
token: "token",
masterKey: "a",
secretKey: "b",
publicKey: "c",
};
const loggedIn = {
whoami: () => ({ email: "cli@example.com", userID: USER_ID }),
toJSON: () => snapshot,
} as unknown as Client;
let prompts: string[];
const loginContext = (
login: (opts: LoginOptions) => Promise<Client>,
): CliContext => ({
...context(),
login,
prompt: async (message) => {
prompts.push(message);
return "123456";
},
promptSecret: async (message) => {
prompts.push(message);
return "123456";
},
});
beforeEach(() => {
prompts = [];
vi.stubEnv("QUAK_EMAIL", "cli@example.com");
vi.stubEnv("QUAK_PASSWORD", "hunter2");
});
afterEach(() => {
vi.unstubAllEnvs();
});
it("takes the email and password from the environment without a prompt", async () => {
const calls: LoginOptions[] = [];
const ctx = loginContext(async (opts) => {
calls.push(opts);
return loggedIn;
});
expect(await loginCommand(ctx)).toBe(0);
expect(prompts).toEqual([]);
expect(calls).toHaveLength(1);
expect(calls[0]!.email).toBe("cli@example.com");
expect(calls[0]!.password).toBe("hunter2");
const path = join(ctx.sessionDir, "session.json");
expect(stderr.text).toBe(
"Authenticating...\n" +
`Logged in as cli@example.com (user ${USER_ID})\n` +
`Session saved to ${path}\n`,
);
});
it("saves the session with mode 0600 in a directory with mode 0700", async () => {
const ctx = loginContext(async () => loggedIn);
expect(await loginCommand(ctx)).toBe(0);
expect(statSync(ctx.sessionDir).mode & 0o777).toBe(0o700);
const path = join(ctx.sessionDir, "session.json");
expect(statSync(path).mode & 0o777).toBe(0o600);
expect(JSON.parse(readFileSync(path, "utf-8"))).toEqual(snapshot);
});
it("asks for the TOTP code when the account needs one", async () => {
let code: string | undefined;
const ctx = loginContext(async (opts) => {
code = await opts.totp!();
return loggedIn;
});
expect(await loginCommand(ctx)).toBe(0);
expect(prompts).toEqual(["TOTP code: "]);
expect(code).toBe("123456");
});
it("a failed login exits 1, says why and writes no session", async () => {
const ctx = loginContext(async () => {
throw new Error("HTTP 401 from server");
});
expect(await loginCommand(ctx)).toBe(1);
expect(stderr.text).toBe(
"Authenticating...\nLogin failed: HTTP 401 from server\n",
);
expect(existsSync(join(ctx.sessionDir, "session.json"))).toBe(false);
});
});
// These use a real client read from the session file, over a fake API that // These use a real client read from the session file, over a fake API that
// records each request and answers with `status`. // records each request and answers with `status`.
describe("logout", () => { describe("logout", () => {
@@ -545,16 +427,6 @@ describe("backup", () => {
expect(stdout.text).toBe(""); expect(stdout.text).toBe("");
}); });
// The backup opens its library with the precache off: it fetches the
// originals it needs into the backup, and must not also fetch every
// thumbnail in the account, or keep originals, in the per-user cache.
it("leaves nothing in the cache's originals and thumbnails", async () => {
const dir = join(root, "backup");
expect(await backupCommand(context(), dir, {})).toBe(0);
expect(readdirSync(join(root, "cache", "thumbnails"))).toEqual([]);
expect(readdirSync(join(root, "cache", "originals"))).toEqual([]);
});
it("exits 1 and lists the file when one download fails", async () => { it("exits 1 and lists the file when one download fails", async () => {
const ctx = context(fakeClient({ failID: 101 })); const ctx = context(fakeClient({ failID: 101 }));
expect(await backupCommand(ctx, join(root, "backup"), {})).toBe(1); expect(await backupCommand(ctx, join(root, "backup"), {})).toBe(1);
@@ -642,32 +514,6 @@ describe("helper list-missing-thumbnails", () => {
}); });
}); });
describe("backup-metadata --exif", () => {
// Runs the command and returns what it printed to stderr.
const backupMetadata = async (opts: { exif?: boolean; all?: boolean }) => {
expect(
await backupMetadataCommand(context(), join(root, "dump"), opts),
).toBe(0);
return stderr.text;
};
it("--exif extracts EXIF", async () => {
expect(await backupMetadata({ exif: true })).toContain(
"[beach.jpg] Extracting EXIF...\n",
);
});
it("--all extracts EXIF", async () => {
expect(await backupMetadata({ all: true })).toContain(
"[beach.jpg] Extracting EXIF...\n",
);
});
it("without either flag extracts no EXIF", async () => {
expect(await backupMetadata({})).not.toContain("Extracting EXIF");
});
});
// Each test first runs `collections` so the cache holds the account as it was, // Each test first runs `collections` so the cache holds the account as it was,
// then changes the server under it. // then changes the server under it.
describe("backup-metadata and the thumbnail helpers refresh first", () => { describe("backup-metadata and the thumbnail helpers refresh first", () => {