2 Commits
Author SHA1 Message Date
clawbot 41304fb7db An expired session exits 3 with one line saying to run quak login (closes #164)
check / check (push) Successful in 1m39s
A 401 from the server reaches run in src/cli-run.ts as the ApiError the
refresh threw, so run recognises it there, prints one line saying to run
"quak login" and exits 3. A missing or corrupt session file keeps its message
and also exits 3. quak backup meets the 401 on its first refresh, before it
touches any file. Every other error still exits 1.

Judgement call: quak logout is unchanged; it handles its own errors and
deletes the session file whatever the server answers.
Judgement call: quak backup still prints its two progress lines before the
error line.

Model: opus-5-5
2026-10-06 05:44:04 +00:00
clawbot 2483c85321 quak backup writes the account and album records backup-metadata writes (closes #166)
check / check (push) Successful in 1m23s
quak backup now writes account.json with the account's email and user ID,
adds each album's ownerID, isShared, updationTime and, when present, its
magicMetadata, pubMagicMetadata and sharedMagicMetadata to the album's
JSON, and adds updationTime to each file's JSON. The fields and their order
match backup-metadata's account.json, _collection.json and per-file JSON.
The interface runBackup drives gains whoami, which the library passes
through from its client.

Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
2026-10-06 07:13:24 +02:00
9 changed files with 300 additions and 47 deletions
+26 -6
View File
@@ -516,8 +516,17 @@ The CLI stores the snapshot at the platform-appropriate data directory via
`0600`. The key material is stored in cleartext in the JSON; treat this file as
you would treat the password itself. A missing file is reported as "not logged
in"; a file that exists but is corrupt is reported as such, naming the bad
field. Both exit with status 1, except that `quak logout` with no file says
there is no session and exits 0.
field. When the refresh a command starts with gets HTTP 401 from the server,
because it no longer accepts the saved session's token, the command prints one
line, `quak: the saved session is no longer valid; run "quak login"`, with no
stack trace. All three exit with status 3, which means the user must run
`quak login` again. `quak logout` is the exception: with no file it says there
is no session and exits 0, and it handles a corrupt file or a failed server call
as described below. `quak backup` meets an expired session on the refresh that
starts every run, before it touches any file. A session that stops working
partway through a backup instead fails each remaining file into `failures.json`,
so that run exits 1 and the next one stops at its refresh with status 3. No
command but `quak login` ever prompts.
`quak logout` ends the session on the server, so the token in `session.json`
stops working even in a copy of the file, and then deletes the file. If the
@@ -551,8 +560,9 @@ library. The read commands — `collections`, `files`, `get`, `get-thumb`,
`helper fix-missing-thumbnails` — force a fresh server round-trip before they
answer, so they report current account state rather than whatever the cache last
held. If that round-trip fails, the command prints the error on one line and
exits 1. `--cache-dir` overrides where the cache lives; without it each account
gets its own directory under the per-user cache path.
exits 1, or 3 when the server no longer accepts the saved session (see "Session
handling"). `--cache-dir` overrides where the cache lives; without it each
account gets its own directory under the per-user cache path.
`get` and `get-thumb` resolve the file by ID directly, so `--collection` is
accepted for backward compatibility but ignored. For a live photo, `get` writes
@@ -599,8 +609,8 @@ the smallest does not.
per unique file, two for a live photo: see
below)
YYYY-MM-DD.<fileID>.json the file's basic metadata fields quak
keeps, its private and public magic
metadata, and its ML data
keeps, its update time, its private and
public magic metadata, and its ML data
YYYY-MM-DD.<fileID>.livephoto.json
which of a live photo's two files is which
collections/
@@ -608,6 +618,7 @@ the smallest does not.
<title> -> ../../YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>
(symlink)
<name>.json collection metadata + file list
account.json the account's email and user ID
failures.json files that failed and have not yet succeeded
```
@@ -619,6 +630,15 @@ name as uploaded, case kept, or `.bin` when it has none or it holds anything but
letters and digits. When the date or the time zone changes, the next run saves
the original at its new path and leaves the old copy where it is.
`account.json` holds the account's `email` and `userID`, as `backup-metadata`
writes it. A collection's JSON holds its `id`, `name`, `type`, `ownerID`,
`isShared` and `updationTime`, its `magicMetadata`, `pubMagicMetadata` and
`sharedMagicMetadata` when it has them, as `backup-metadata`'s
`_collection.json` does, and `files`, each file's `id` and `metadata`. A file's
JSON holds its `id`, `collectionID`, `ownerID`, `metadata` and `updationTime`,
and its `magicMetadata` and `pubMagicMetadata` when it has them. Update times
are in microseconds, as Ente records them.
A file's JSON holds Ente's ML data for it (its faces and its CLIP embedding) as
`mlData`, the same payload `backup-metadata` writes; a file Ente has no ML data
for has no `mlData`. The backup waits for the library's ML data fetch to finish
+15
View File
@@ -25,6 +25,21 @@ declares one.
# Completed Steps
- 2026-10-06: When the refresh a command starts with gets HTTP 401, because the
server no longer accepts the saved session's token, the command prints one
line, `quak: the saved session is no longer valid; run "quak login"`, and
exits 3 (issue 164). A missing or corrupt session file keeps its message and
now also exits 3, so a cron job can tell that the user must log in again.
`run` in `src/cli-run.ts` recognises the 401, which reaches it unchanged from
the refresh, so `quak backup` stops there before it touches any file. Every
other error still exits 1.
- 2026-10-06: `quak backup` writes the account and album records
`backup-metadata` writes (issue 166): `account.json` with the account's
`email` and `userID`, and in each album's JSON its `ownerID`, `isShared`,
`updationTime` and, when present, its three layers of magic metadata. Each
file's JSON gains its `updationTime`.
- 2026-10-05: `quak backup` writes each file's ML data (its faces and CLIP
embedding) into the file's JSON as `mlData`, the payload
`lib.mldata.forFile()` returns (issue 163). `lib.backup()` waits for an ML
+36 -7
View File
@@ -14,6 +14,7 @@
// the file's ML data
// collections/<name>/<title> symlink to the original
// collections/<name>.json per-collection metadata
// account.json the account's email and user ID
// failures.json durable ledger of unresolved failures
//
// A live photo's original is its image and its video, each with its own
@@ -64,7 +65,7 @@ import {
} from "./library/content.js";
import { representative } from "./library/records.js";
import type { MLData } from "./mldata-fetch.js";
import type { Collection, EnteFile } from "./model/types.js";
import type { Collection, EnteFile, FileMetadata } from "./model/types.js";
export type ProgressCallback = (message: string) => void;
@@ -108,6 +109,8 @@ export interface BackupResult {
// The slice of the library that backup drives. `Library` implements it; a test
// can drive backup with a stand-in.
export interface BackupLibrary {
// The account the library belongs to.
whoami(): { email: string; userID: number };
refresh(): Promise<void>;
listCollections(): Collection[];
listFiles(collectionID: number): EnteFile[];
@@ -363,6 +366,7 @@ const writeSidecar = (
collectionID: file.collectionID,
ownerID: file.ownerID,
metadata: file.metadata,
updationTime: file.updationTime,
};
if (file.magicMetadata) meta.magicMetadata = file.magicMetadata;
if (file.pubMagicMetadata) meta.pubMagicMetadata = file.pubMagicMetadata;
@@ -371,6 +375,29 @@ const writeSidecar = (
writeFileSync(path, JSON.stringify(meta, null, 2));
};
// The album's JSON: its basic fields, its magic metadata, and its files.
const writeAlbumJSON = (
path: string,
c: Collection,
files: { id: number; metadata: FileMetadata }[],
): void => {
const album: Record<string, unknown> = {
id: c.id,
name: c.name,
type: c.type,
ownerID: c.ownerID,
isShared: c.isShared,
updationTime: c.updationTime,
};
if (c.magicMetadata) album.magicMetadata = c.magicMetadata;
if (c.pubMagicMetadata) album.pubMagicMetadata = c.pubMagicMetadata;
if (c.sharedMagicMetadata) {
album.sharedMagicMetadata = c.sharedMagicMetadata;
}
album.files = files;
writeFileSync(path, JSON.stringify(album, null, 2));
};
export const runBackup = async (
lib: BackupLibrary,
opts: BackupOptions,
@@ -393,6 +420,11 @@ export const runBackup = async (
const collectionsDir = join(downloadDirectory, "collections");
const thumbnailsDir = join(downloadDirectory, "thumbnails");
mkdirSync(collectionsDir, { recursive: true });
const { email, userID } = lib.whoami();
writeFileSync(
join(downloadDirectory, "account.json"),
JSON.stringify({ email, userID }, null, 2),
);
if (includeThumbnails) mkdirSync(thumbnailsDir, { recursive: true });
removeLeftoverTempFiles(thumbnailsDir);
for (const dir of dateFolders(downloadDirectory)) {
@@ -618,13 +650,10 @@ export const runBackup = async (
}
}
writeFileSync(
writeAlbumJSON(
join(collectionsDir, `${colDirName}.json`),
JSON.stringify(
{ id: c.id, name: c.name, type: c.type, files: metaFiles },
null,
2,
),
c,
metaFiles,
);
}
+12 -10
View File
@@ -73,7 +73,9 @@ export const saveSession = (
);
};
// The saved client, or undefined after telling the user why there is none.
// The saved client, or undefined after telling the user why there is none. The
// command then exits 3, the code for "log in again", as `run` in `cli-run.ts`
// does when the server no longer accepts the saved session.
const requireSession = (ctx: CliContext): Client | undefined => {
let client: Client | null;
try {
@@ -150,7 +152,7 @@ export const loginCommand = async (ctx: CliContext): Promise<number> => {
export const whoamiCommand = async (ctx: CliContext): Promise<number> => {
await init();
const client = requireSession(ctx);
if (!client) return 1;
if (!client) return 3;
const info = client.whoami();
ctx.stdout.write(JSON.stringify(info) + "\n");
return 0;
@@ -201,7 +203,7 @@ export const collectionsCommand = async (
): Promise<number> => {
await init();
const client = requireSession(ctx);
if (!client) return 1;
if (!client) return 3;
const lib = await openReadLibrary(ctx, client);
try {
// Force a server round-trip and list in enumeration order (issue #36
@@ -243,7 +245,7 @@ export const filesCommand = async (
): Promise<number> => {
await init();
const client = requireSession(ctx);
if (!client) return 1;
if (!client) return 3;
const collectionID = Number(opts.collection);
if (!Number.isFinite(collectionID)) {
ctx.stderr.write("Invalid collection ID\n");
@@ -285,7 +287,7 @@ export const getCommand = async (
): Promise<number> => {
await init();
const client = requireSession(ctx);
if (!client) return 1;
if (!client) return 3;
const fileID = Number(fileIDStr);
if (!Number.isFinite(fileID)) {
ctx.stderr.write("Invalid file ID\n");
@@ -343,7 +345,7 @@ export const getThumbCommand = async (
): Promise<number> => {
await init();
const client = requireSession(ctx);
if (!client) return 1;
if (!client) return 3;
const fileID = Number(fileIDStr);
if (!Number.isFinite(fileID)) {
ctx.stderr.write("Invalid file ID\n");
@@ -380,7 +382,7 @@ export const backupMetadataCommand = async (
): Promise<number> => {
await init();
const client = requireSession(ctx);
if (!client) return 1;
if (!client) return 3;
const lib = await openReadLibrary(ctx, client);
try {
// Refresh first so the dump holds current account state, not what the
@@ -403,7 +405,7 @@ export const backupCommand = async (
): Promise<number> => {
await init();
const client = requireSession(ctx);
if (!client) return 1;
if (!client) return 3;
ctx.stderr.write("Starting backup...\n");
// The precache is off: the backup fetches what it needs, and must not
@@ -453,7 +455,7 @@ export const listMissingThumbnailsCommand = async (
): Promise<number> => {
await init();
const client = requireSession(ctx);
if (!client) return 1;
if (!client) return 3;
const lib = await openReadLibrary(ctx, client);
try {
// Refresh first so files added since the cache was written are
@@ -491,7 +493,7 @@ export const fixMissingThumbnailsCommand = async (
): Promise<number> => {
await init();
const client = requireSession(ctx);
if (!client) return 1;
if (!client) return 3;
const lib = await openReadLibrary(ctx, client);
try {
// Refresh first so files added since the cache was written are found;
+15 -5
View File
@@ -1,12 +1,15 @@
// Runs one CLI command for `bin/quak.ts` and exits with its code.
import type { Writable } from "node:stream";
import { ApiError } from "./api/client.js";
// Run a command and exit with its code once stdout/stderr have drained.
// Exiting before the drain can truncate piped output, and the library can keep
// the event loop alive after a command returns, so a plain return could hang.
// An error the command throws is printed as one `quak: MESSAGE` line, without
// the stack trace, and exits 1.
// the stack trace, and exits 1. A 401 from the server means it no longer
// accepts the saved session: that prints one line saying to log in again and
// exits 3, as a missing or corrupt session file does.
export const run = async (
command: Promise<number>,
stdout: Writable,
@@ -17,10 +20,17 @@ export const run = async (
try {
code = await command;
} catch (err) {
stderr.write(
`quak: ${err instanceof Error ? err.message : String(err)}\n`,
);
code = 1;
if (err instanceof ApiError && err.status === 401) {
stderr.write(
`quak: the saved session is no longer valid; run "quak login"\n`,
);
code = 3;
} else {
stderr.write(
`quak: ${err instanceof Error ? err.message : String(err)}\n`,
);
code = 1;
}
}
const pending = [stdout, stderr].filter((s) => s.writableLength > 0);
if (pending.length === 0) {
+3 -1
View File
@@ -571,7 +571,8 @@ export class Library {
// when it fails. Then puts pending originals at their save paths as
// `Photo.download()` does (and optional thumbnails) through the content
// cache and pools, waits for an ML data fetch, and rebuilds the derived
// symlink/JSON views from the model, each file's JSON with its ML data.
// symlink/JSON views from the model, each file's JSON with its ML data,
// beside an `account.json` with the account's email and user ID.
// Throws before any network work when no content cache backs the
// originals it must fetch.
backup(opts?: BackupOptions): Promise<BackupResult> {
@@ -589,6 +590,7 @@ export class Library {
const cache = this.cache;
return runBackup(
{
whoami: () => this.client.whoami(),
refresh: () => this.refreshNow(),
listCollections: () => this.store.listCollections(),
listFiles: (id) => this.store.listFiles(id),
+102
View File
@@ -12,6 +12,7 @@
* collections/
* <name>/<title> symlink to the original (rebuilt each run)
* <name>.json per-collection metadata (rebuilt each run)
* account.json the account's email and user ID
* failures.json durable ledger of unresolved failures
*
* The properties that distinguish backup from a naive download loop, and that
@@ -917,6 +918,106 @@ describe("ML data in each file's JSON", () => {
});
});
// The fields `RecordsClient` gives the Vacation album: shared by another
// account, with all three layers of magic metadata.
const SHARED_ALBUM = {
ownerID: 7,
isShared: true,
updationTime: 1700,
magicMetadata: { visibility: 0 },
pubMagicMetadata: { coverID: 100 },
sharedMagicMetadata: { visibility: 2 },
};
// Serves the Vacation album with `SHARED_ALBUM`'s fields, and each file with
// the update time 5000 + its ID.
class RecordsClient extends MockClient {
override async collectionsSince(): Promise<CollectionsPage> {
const page = await super.collectionsSince();
return {
...page,
collections: page.collections.map((c) =>
c.id === 1 ? { ...c, ...SHARED_ALBUM } : c,
),
};
}
override async filesSince(args: {
collectionID: number;
}): Promise<FilesPage> {
const page = await super.filesSince(args);
return {
...page,
files: page.files.map((f) => ({
...f,
updationTime: 5000 + f.id,
})),
};
}
}
// The JSON the backup in `outDir` wrote for the album named `name`.
const albumJSON = (outDir: string, name: string): Record<string, unknown> =>
JSON.parse(
readFileSync(join(outDir, "collections", `${name}.json`), "utf-8"),
);
describe("account and album records", () => {
it("writes account.json with the account's email and user ID", async () => {
const lib = await openLibrary(stubSource());
const outDir = join(root, "backup");
await lib.backup({ downloadDirectory: outDir });
expect(
JSON.parse(readFileSync(join(outDir, "account.json"), "utf-8")),
).toEqual({ email: "backup@example.com", userID: USER_ID });
await lib.close();
});
it("writes each album's owner, sharing, update time and magic metadata into its JSON", async () => {
const lib = await openLibrary(stubSource(), new RecordsClient());
const outDir = join(root, "backup");
await lib.backup({ downloadDirectory: outDir });
expect(albumJSON(outDir, "Vacation")).toEqual({
id: 1,
name: "Vacation",
type: "album",
...SHARED_ALBUM,
files: [
{ id: 100, metadata: file(100, 1, "beach.jpg").metadata },
{ id: 101, metadata: file(101, 1, "sunset.jpg").metadata },
],
});
// An album with no magic metadata gets no magic metadata fields.
expect(albumJSON(outDir, "Work")).toEqual({
id: 2,
name: "Work",
type: "album",
ownerID: USER_ID,
isShared: false,
updationTime: 1,
files: [
{ id: 200, metadata: file(200, 2, "diagram.png").metadata },
],
});
await lib.close();
});
it("writes each file's update time into its JSON", async () => {
const lib = await openLibrary(stubSource(), new RecordsClient());
const outDir = join(root, "backup");
await lib.backup({ downloadDirectory: outDir });
for (const fileID of [100, 101, 200]) {
expect(fileJSON(outDir, fileID).updationTime).toBe(5000 + fileID);
}
await lib.close();
});
});
// Every entry under collections/, one level of directories deep, with each
// symlink's target.
const tree = (outDir: string): string[] => {
@@ -962,6 +1063,7 @@ describe("backup album folders", () => {
},
fetchMLData: async () => {},
mlData: async () => undefined,
whoami: () => ({ email: "backup@example.com", userID: USER_ID }),
});
const albumID = (outDir: string, jsonName: string): number =>
+68 -18
View File
@@ -230,20 +230,30 @@ describe("session file", () => {
expect(JSON.parse(readFileSync(path, "utf-8"))).toEqual(snapshot);
});
it("a missing session exits 1 with 'Not logged in'", async () => {
it("a missing session exits 3 with 'Not logged in' from every command that needs one", async () => {
const ctx = { ...context(), loadSession };
expect(await whoamiCommand(ctx)).toBe(1);
expect(stderr.text).toBe(
const dir = join(root, "backup");
expect(await whoamiCommand(ctx)).toBe(3);
expect(await collectionsCommand(ctx, {})).toBe(3);
expect(await filesCommand(ctx, { collection: "1" })).toBe(3);
expect(await getCommand(ctx, "100", {})).toBe(3);
expect(await getThumbCommand(ctx, "100", {})).toBe(3);
expect(await backupMetadataCommand(ctx, dir, {})).toBe(3);
expect(await backupCommand(ctx, dir, {})).toBe(3);
expect(await listMissingThumbnailsCommand(ctx, {})).toBe(3);
expect(await fixMissingThumbnailsCommand(ctx, {})).toBe(3);
const notLoggedIn =
`Not logged in. Run "quak login" first.\n` +
`Session file: ${join(ctx.sessionDir, "session.json")}\n`,
);
`Session file: ${join(ctx.sessionDir, "session.json")}\n`;
expect(stderr.text).toBe(notLoggedIn.repeat(9));
expect(stdout.text).toBe("");
expect(existsSync(dir)).toBe(false);
});
it("a corrupt session exits 1 and says it is corrupt", async () => {
it("a corrupt session exits 3 and says it is corrupt", async () => {
const ctx = { ...context(), loadSession };
saveSession(ctx.sessionDir, snapshot);
expect(await collectionsCommand(ctx, {})).toBe(1);
expect(await collectionsCommand(ctx, {})).toBe(3);
expect(stderr.text).toContain("is corrupt");
expect(stderr.text).toContain(
`Run "quak logout" and then "quak login" to replace it.\n`,
@@ -748,15 +758,12 @@ describe("backup", () => {
);
});
it("exits 1 with the error on one line when the refresh fails", async () => {
const client = {
...fakeClient(),
collectionsSince: async () => {
throw new Error("HTTP 401 from server");
},
} as unknown as Client;
const dir = join(root, "backup");
// Through `run`, as `bin/quak.ts` does, which prints a thrown error.
// Runs `backup` through `run`, as `bin/quak.ts` does, which prints a thrown
// error; returns the exit code and what `run` printed.
const backupThroughRun = async (
ctx: CliContext,
dir: string,
): Promise<{ code: number; runText: string }> => {
const runStderr = new PassThrough();
let runText = "";
runStderr.on("data", (chunk: Buffer) => {
@@ -764,14 +771,57 @@ describe("backup", () => {
});
const code = await new Promise<number>((resolve) => {
void run(
backupCommand(context(client), dir, {}),
backupCommand(ctx, dir, {}),
new PassThrough(),
runStderr,
resolve,
);
});
return { code, runText };
};
it("exits 1 with the error on one line when the refresh fails", async () => {
const client = {
...fakeClient(),
collectionsSince: async () => {
throw new Error("HTTP 503 from server");
},
} as unknown as Client;
const dir = join(root, "backup");
const { code, runText } = await backupThroughRun(context(client), dir);
expect(code).toBe(1);
expect(runText).toBe("quak: HTTP 401 from server\n");
expect(runText).toBe("quak: HTTP 503 from server\n");
expect(stderr.text).toBe("Starting backup...\nRefreshing library...\n");
expect(existsSync(dir)).toBe(false);
});
// A real saved session, read back by `loadSession`, whose server answers
// every request with 401, as it does once it no longer accepts the token.
// The context's prompts throw, so a prompt would end the run with another
// line and exit 1.
it("exits 3 with one line saying to log in again when the server answers 401", async () => {
const key = toBase64(new Uint8Array(32));
saveSession(join(root, "session"), {
email: "cli@example.com",
userID: USER_ID,
token: "expired",
masterKey: key,
secretKey: key,
publicKey: key,
});
const unauthorized = async (): Promise<Response> =>
new Response(null, { status: 401 });
const ctx = {
...context(),
loadSession: (path: string) =>
loadSession(path, { fetch: unauthorized }),
};
const dir = join(root, "backup");
const { code, runText } = await backupThroughRun(ctx, dir);
expect(code).toBe(3);
expect(runText).toBe(
`quak: the saved session is no longer valid; run "quak login"\n`,
);
expect(stderr.text).toBe("Starting backup...\nRefreshing library...\n");
expect(existsSync(dir)).toBe(false);
});
+23
View File
@@ -5,6 +5,7 @@
import { PassThrough } from "node:stream";
import { describe, it, expect } from "vitest";
import { ApiError } from "../../src/api/client.js";
import { run } from "../../src/cli-run.js";
// A stream whose written text is kept in `text`; writes finish at once, so
@@ -55,4 +56,26 @@ describe("run", () => {
stderr: "quak: offline\n",
});
});
it("on a 401 from the server says to run quak login, on one line, and exits 3", async () => {
const result = await runToExit(
Promise.reject(new ApiError("unauthorized", 401)),
);
expect(result).toEqual({
code: 3,
stdout: "",
stderr: `quak: the saved session is no longer valid; run "quak login"\n`,
});
});
it("prints another HTTP error as it is and exits 1", async () => {
const result = await runToExit(
Promise.reject(new ApiError("forbidden", 403)),
);
expect(result).toEqual({
code: 1,
stdout: "",
stderr: "quak: forbidden\n",
});
});
});