Compare commits
2
Commits
6967117cee
...
41304fb7db
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
41304fb7db | ||
|
|
2483c85321 |
@@ -516,8 +516,17 @@ The CLI stores the snapshot at the platform-appropriate data directory via
|
|||||||
`0600`. The key material is stored in cleartext in the JSON; treat this file as
|
`0600`. The key material is stored in cleartext in the JSON; treat this file as
|
||||||
you would treat the password itself. A missing file is reported as "not logged
|
you would treat the password itself. A missing file is reported as "not logged
|
||||||
in"; a file that exists but is corrupt is reported as such, naming the bad
|
in"; a file that exists but is corrupt is reported as such, naming the bad
|
||||||
field. Both exit with status 1, except that `quak logout` with no file says
|
field. When the refresh a command starts with gets HTTP 401 from the server,
|
||||||
there is no session and exits 0.
|
because it no longer accepts the saved session's token, the command prints one
|
||||||
|
line, `quak: the saved session is no longer valid; run "quak login"`, with no
|
||||||
|
stack trace. All three exit with status 3, which means the user must run
|
||||||
|
`quak login` again. `quak logout` is the exception: with no file it says there
|
||||||
|
is no session and exits 0, and it handles a corrupt file or a failed server call
|
||||||
|
as described below. `quak backup` meets an expired session on the refresh that
|
||||||
|
starts every run, before it touches any file. A session that stops working
|
||||||
|
partway through a backup instead fails each remaining file into `failures.json`,
|
||||||
|
so that run exits 1 and the next one stops at its refresh with status 3. No
|
||||||
|
command but `quak login` ever prompts.
|
||||||
|
|
||||||
`quak logout` ends the session on the server, so the token in `session.json`
|
`quak logout` ends the session on the server, so the token in `session.json`
|
||||||
stops working even in a copy of the file, and then deletes the file. If the
|
stops working even in a copy of the file, and then deletes the file. If the
|
||||||
@@ -551,8 +560,9 @@ library. The read commands — `collections`, `files`, `get`, `get-thumb`,
|
|||||||
`helper fix-missing-thumbnails` — force a fresh server round-trip before they
|
`helper fix-missing-thumbnails` — force a fresh server round-trip before they
|
||||||
answer, so they report current account state rather than whatever the cache last
|
answer, so they report current account state rather than whatever the cache last
|
||||||
held. If that round-trip fails, the command prints the error on one line and
|
held. If that round-trip fails, the command prints the error on one line and
|
||||||
exits 1. `--cache-dir` overrides where the cache lives; without it each account
|
exits 1, or 3 when the server no longer accepts the saved session (see "Session
|
||||||
gets its own directory under the per-user cache path.
|
handling"). `--cache-dir` overrides where the cache lives; without it each
|
||||||
|
account gets its own directory under the per-user cache path.
|
||||||
|
|
||||||
`get` and `get-thumb` resolve the file by ID directly, so `--collection` is
|
`get` and `get-thumb` resolve the file by ID directly, so `--collection` is
|
||||||
accepted for backward compatibility but ignored. For a live photo, `get` writes
|
accepted for backward compatibility but ignored. For a live photo, `get` writes
|
||||||
@@ -599,8 +609,8 @@ the smallest does not.
|
|||||||
per unique file, two for a live photo: see
|
per unique file, two for a live photo: see
|
||||||
below)
|
below)
|
||||||
YYYY-MM-DD.<fileID>.json the file's basic metadata fields quak
|
YYYY-MM-DD.<fileID>.json the file's basic metadata fields quak
|
||||||
keeps, its private and public magic
|
keeps, its update time, its private and
|
||||||
metadata, and its ML data
|
public magic metadata, and its ML data
|
||||||
YYYY-MM-DD.<fileID>.livephoto.json
|
YYYY-MM-DD.<fileID>.livephoto.json
|
||||||
which of a live photo's two files is which
|
which of a live photo's two files is which
|
||||||
collections/
|
collections/
|
||||||
@@ -608,6 +618,7 @@ the smallest does not.
|
|||||||
<title> -> ../../YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>
|
<title> -> ../../YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>
|
||||||
(symlink)
|
(symlink)
|
||||||
<name>.json collection metadata + file list
|
<name>.json collection metadata + file list
|
||||||
|
account.json the account's email and user ID
|
||||||
failures.json files that failed and have not yet succeeded
|
failures.json files that failed and have not yet succeeded
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -619,6 +630,15 @@ name as uploaded, case kept, or `.bin` when it has none or it holds anything but
|
|||||||
letters and digits. When the date or the time zone changes, the next run saves
|
letters and digits. When the date or the time zone changes, the next run saves
|
||||||
the original at its new path and leaves the old copy where it is.
|
the original at its new path and leaves the old copy where it is.
|
||||||
|
|
||||||
|
`account.json` holds the account's `email` and `userID`, as `backup-metadata`
|
||||||
|
writes it. A collection's JSON holds its `id`, `name`, `type`, `ownerID`,
|
||||||
|
`isShared` and `updationTime`, its `magicMetadata`, `pubMagicMetadata` and
|
||||||
|
`sharedMagicMetadata` when it has them, as `backup-metadata`'s
|
||||||
|
`_collection.json` does, and `files`, each file's `id` and `metadata`. A file's
|
||||||
|
JSON holds its `id`, `collectionID`, `ownerID`, `metadata` and `updationTime`,
|
||||||
|
and its `magicMetadata` and `pubMagicMetadata` when it has them. Update times
|
||||||
|
are in microseconds, as Ente records them.
|
||||||
|
|
||||||
A file's JSON holds Ente's ML data for it (its faces and its CLIP embedding) as
|
A file's JSON holds Ente's ML data for it (its faces and its CLIP embedding) as
|
||||||
`mlData`, the same payload `backup-metadata` writes; a file Ente has no ML data
|
`mlData`, the same payload `backup-metadata` writes; a file Ente has no ML data
|
||||||
for has no `mlData`. The backup waits for the library's ML data fetch to finish
|
for has no `mlData`. The backup waits for the library's ML data fetch to finish
|
||||||
|
|||||||
@@ -25,6 +25,21 @@ declares one.
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-10-06: When the refresh a command starts with gets HTTP 401, because the
|
||||||
|
server no longer accepts the saved session's token, the command prints one
|
||||||
|
line, `quak: the saved session is no longer valid; run "quak login"`, and
|
||||||
|
exits 3 (issue 164). A missing or corrupt session file keeps its message and
|
||||||
|
now also exits 3, so a cron job can tell that the user must log in again.
|
||||||
|
`run` in `src/cli-run.ts` recognises the 401, which reaches it unchanged from
|
||||||
|
the refresh, so `quak backup` stops there before it touches any file. Every
|
||||||
|
other error still exits 1.
|
||||||
|
|
||||||
|
- 2026-10-06: `quak backup` writes the account and album records
|
||||||
|
`backup-metadata` writes (issue 166): `account.json` with the account's
|
||||||
|
`email` and `userID`, and in each album's JSON its `ownerID`, `isShared`,
|
||||||
|
`updationTime` and, when present, its three layers of magic metadata. Each
|
||||||
|
file's JSON gains its `updationTime`.
|
||||||
|
|
||||||
- 2026-10-05: `quak backup` writes each file's ML data (its faces and CLIP
|
- 2026-10-05: `quak backup` writes each file's ML data (its faces and CLIP
|
||||||
embedding) into the file's JSON as `mlData`, the payload
|
embedding) into the file's JSON as `mlData`, the payload
|
||||||
`lib.mldata.forFile()` returns (issue 163). `lib.backup()` waits for an ML
|
`lib.mldata.forFile()` returns (issue 163). `lib.backup()` waits for an ML
|
||||||
|
|||||||
+36
-7
@@ -14,6 +14,7 @@
|
|||||||
// the file's ML data
|
// the file's ML data
|
||||||
// collections/<name>/<title> symlink to the original
|
// collections/<name>/<title> symlink to the original
|
||||||
// collections/<name>.json per-collection metadata
|
// collections/<name>.json per-collection metadata
|
||||||
|
// account.json the account's email and user ID
|
||||||
// failures.json durable ledger of unresolved failures
|
// failures.json durable ledger of unresolved failures
|
||||||
//
|
//
|
||||||
// A live photo's original is its image and its video, each with its own
|
// A live photo's original is its image and its video, each with its own
|
||||||
@@ -64,7 +65,7 @@ import {
|
|||||||
} from "./library/content.js";
|
} from "./library/content.js";
|
||||||
import { representative } from "./library/records.js";
|
import { representative } from "./library/records.js";
|
||||||
import type { MLData } from "./mldata-fetch.js";
|
import type { MLData } from "./mldata-fetch.js";
|
||||||
import type { Collection, EnteFile } from "./model/types.js";
|
import type { Collection, EnteFile, FileMetadata } from "./model/types.js";
|
||||||
|
|
||||||
export type ProgressCallback = (message: string) => void;
|
export type ProgressCallback = (message: string) => void;
|
||||||
|
|
||||||
@@ -108,6 +109,8 @@ export interface BackupResult {
|
|||||||
// The slice of the library that backup drives. `Library` implements it; a test
|
// The slice of the library that backup drives. `Library` implements it; a test
|
||||||
// can drive backup with a stand-in.
|
// can drive backup with a stand-in.
|
||||||
export interface BackupLibrary {
|
export interface BackupLibrary {
|
||||||
|
// The account the library belongs to.
|
||||||
|
whoami(): { email: string; userID: number };
|
||||||
refresh(): Promise<void>;
|
refresh(): Promise<void>;
|
||||||
listCollections(): Collection[];
|
listCollections(): Collection[];
|
||||||
listFiles(collectionID: number): EnteFile[];
|
listFiles(collectionID: number): EnteFile[];
|
||||||
@@ -363,6 +366,7 @@ const writeSidecar = (
|
|||||||
collectionID: file.collectionID,
|
collectionID: file.collectionID,
|
||||||
ownerID: file.ownerID,
|
ownerID: file.ownerID,
|
||||||
metadata: file.metadata,
|
metadata: file.metadata,
|
||||||
|
updationTime: file.updationTime,
|
||||||
};
|
};
|
||||||
if (file.magicMetadata) meta.magicMetadata = file.magicMetadata;
|
if (file.magicMetadata) meta.magicMetadata = file.magicMetadata;
|
||||||
if (file.pubMagicMetadata) meta.pubMagicMetadata = file.pubMagicMetadata;
|
if (file.pubMagicMetadata) meta.pubMagicMetadata = file.pubMagicMetadata;
|
||||||
@@ -371,6 +375,29 @@ const writeSidecar = (
|
|||||||
writeFileSync(path, JSON.stringify(meta, null, 2));
|
writeFileSync(path, JSON.stringify(meta, null, 2));
|
||||||
};
|
};
|
||||||
|
|
||||||
|
// The album's JSON: its basic fields, its magic metadata, and its files.
|
||||||
|
const writeAlbumJSON = (
|
||||||
|
path: string,
|
||||||
|
c: Collection,
|
||||||
|
files: { id: number; metadata: FileMetadata }[],
|
||||||
|
): void => {
|
||||||
|
const album: Record<string, unknown> = {
|
||||||
|
id: c.id,
|
||||||
|
name: c.name,
|
||||||
|
type: c.type,
|
||||||
|
ownerID: c.ownerID,
|
||||||
|
isShared: c.isShared,
|
||||||
|
updationTime: c.updationTime,
|
||||||
|
};
|
||||||
|
if (c.magicMetadata) album.magicMetadata = c.magicMetadata;
|
||||||
|
if (c.pubMagicMetadata) album.pubMagicMetadata = c.pubMagicMetadata;
|
||||||
|
if (c.sharedMagicMetadata) {
|
||||||
|
album.sharedMagicMetadata = c.sharedMagicMetadata;
|
||||||
|
}
|
||||||
|
album.files = files;
|
||||||
|
writeFileSync(path, JSON.stringify(album, null, 2));
|
||||||
|
};
|
||||||
|
|
||||||
export const runBackup = async (
|
export const runBackup = async (
|
||||||
lib: BackupLibrary,
|
lib: BackupLibrary,
|
||||||
opts: BackupOptions,
|
opts: BackupOptions,
|
||||||
@@ -393,6 +420,11 @@ export const runBackup = async (
|
|||||||
const collectionsDir = join(downloadDirectory, "collections");
|
const collectionsDir = join(downloadDirectory, "collections");
|
||||||
const thumbnailsDir = join(downloadDirectory, "thumbnails");
|
const thumbnailsDir = join(downloadDirectory, "thumbnails");
|
||||||
mkdirSync(collectionsDir, { recursive: true });
|
mkdirSync(collectionsDir, { recursive: true });
|
||||||
|
const { email, userID } = lib.whoami();
|
||||||
|
writeFileSync(
|
||||||
|
join(downloadDirectory, "account.json"),
|
||||||
|
JSON.stringify({ email, userID }, null, 2),
|
||||||
|
);
|
||||||
if (includeThumbnails) mkdirSync(thumbnailsDir, { recursive: true });
|
if (includeThumbnails) mkdirSync(thumbnailsDir, { recursive: true });
|
||||||
removeLeftoverTempFiles(thumbnailsDir);
|
removeLeftoverTempFiles(thumbnailsDir);
|
||||||
for (const dir of dateFolders(downloadDirectory)) {
|
for (const dir of dateFolders(downloadDirectory)) {
|
||||||
@@ -618,13 +650,10 @@ export const runBackup = async (
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
writeFileSync(
|
writeAlbumJSON(
|
||||||
join(collectionsDir, `${colDirName}.json`),
|
join(collectionsDir, `${colDirName}.json`),
|
||||||
JSON.stringify(
|
c,
|
||||||
{ id: c.id, name: c.name, type: c.type, files: metaFiles },
|
metaFiles,
|
||||||
null,
|
|
||||||
2,
|
|
||||||
),
|
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+12
-10
@@ -73,7 +73,9 @@ export const saveSession = (
|
|||||||
);
|
);
|
||||||
};
|
};
|
||||||
|
|
||||||
// The saved client, or undefined after telling the user why there is none.
|
// The saved client, or undefined after telling the user why there is none. The
|
||||||
|
// command then exits 3, the code for "log in again", as `run` in `cli-run.ts`
|
||||||
|
// does when the server no longer accepts the saved session.
|
||||||
const requireSession = (ctx: CliContext): Client | undefined => {
|
const requireSession = (ctx: CliContext): Client | undefined => {
|
||||||
let client: Client | null;
|
let client: Client | null;
|
||||||
try {
|
try {
|
||||||
@@ -150,7 +152,7 @@ export const loginCommand = async (ctx: CliContext): Promise<number> => {
|
|||||||
export const whoamiCommand = async (ctx: CliContext): Promise<number> => {
|
export const whoamiCommand = async (ctx: CliContext): Promise<number> => {
|
||||||
await init();
|
await init();
|
||||||
const client = requireSession(ctx);
|
const client = requireSession(ctx);
|
||||||
if (!client) return 1;
|
if (!client) return 3;
|
||||||
const info = client.whoami();
|
const info = client.whoami();
|
||||||
ctx.stdout.write(JSON.stringify(info) + "\n");
|
ctx.stdout.write(JSON.stringify(info) + "\n");
|
||||||
return 0;
|
return 0;
|
||||||
@@ -201,7 +203,7 @@ export const collectionsCommand = async (
|
|||||||
): Promise<number> => {
|
): Promise<number> => {
|
||||||
await init();
|
await init();
|
||||||
const client = requireSession(ctx);
|
const client = requireSession(ctx);
|
||||||
if (!client) return 1;
|
if (!client) return 3;
|
||||||
const lib = await openReadLibrary(ctx, client);
|
const lib = await openReadLibrary(ctx, client);
|
||||||
try {
|
try {
|
||||||
// Force a server round-trip and list in enumeration order (issue #36
|
// Force a server round-trip and list in enumeration order (issue #36
|
||||||
@@ -243,7 +245,7 @@ export const filesCommand = async (
|
|||||||
): Promise<number> => {
|
): Promise<number> => {
|
||||||
await init();
|
await init();
|
||||||
const client = requireSession(ctx);
|
const client = requireSession(ctx);
|
||||||
if (!client) return 1;
|
if (!client) return 3;
|
||||||
const collectionID = Number(opts.collection);
|
const collectionID = Number(opts.collection);
|
||||||
if (!Number.isFinite(collectionID)) {
|
if (!Number.isFinite(collectionID)) {
|
||||||
ctx.stderr.write("Invalid collection ID\n");
|
ctx.stderr.write("Invalid collection ID\n");
|
||||||
@@ -285,7 +287,7 @@ export const getCommand = async (
|
|||||||
): Promise<number> => {
|
): Promise<number> => {
|
||||||
await init();
|
await init();
|
||||||
const client = requireSession(ctx);
|
const client = requireSession(ctx);
|
||||||
if (!client) return 1;
|
if (!client) return 3;
|
||||||
const fileID = Number(fileIDStr);
|
const fileID = Number(fileIDStr);
|
||||||
if (!Number.isFinite(fileID)) {
|
if (!Number.isFinite(fileID)) {
|
||||||
ctx.stderr.write("Invalid file ID\n");
|
ctx.stderr.write("Invalid file ID\n");
|
||||||
@@ -343,7 +345,7 @@ export const getThumbCommand = async (
|
|||||||
): Promise<number> => {
|
): Promise<number> => {
|
||||||
await init();
|
await init();
|
||||||
const client = requireSession(ctx);
|
const client = requireSession(ctx);
|
||||||
if (!client) return 1;
|
if (!client) return 3;
|
||||||
const fileID = Number(fileIDStr);
|
const fileID = Number(fileIDStr);
|
||||||
if (!Number.isFinite(fileID)) {
|
if (!Number.isFinite(fileID)) {
|
||||||
ctx.stderr.write("Invalid file ID\n");
|
ctx.stderr.write("Invalid file ID\n");
|
||||||
@@ -380,7 +382,7 @@ export const backupMetadataCommand = async (
|
|||||||
): Promise<number> => {
|
): Promise<number> => {
|
||||||
await init();
|
await init();
|
||||||
const client = requireSession(ctx);
|
const client = requireSession(ctx);
|
||||||
if (!client) return 1;
|
if (!client) return 3;
|
||||||
const lib = await openReadLibrary(ctx, client);
|
const lib = await openReadLibrary(ctx, client);
|
||||||
try {
|
try {
|
||||||
// Refresh first so the dump holds current account state, not what the
|
// Refresh first so the dump holds current account state, not what the
|
||||||
@@ -403,7 +405,7 @@ export const backupCommand = async (
|
|||||||
): Promise<number> => {
|
): Promise<number> => {
|
||||||
await init();
|
await init();
|
||||||
const client = requireSession(ctx);
|
const client = requireSession(ctx);
|
||||||
if (!client) return 1;
|
if (!client) return 3;
|
||||||
|
|
||||||
ctx.stderr.write("Starting backup...\n");
|
ctx.stderr.write("Starting backup...\n");
|
||||||
// The precache is off: the backup fetches what it needs, and must not
|
// The precache is off: the backup fetches what it needs, and must not
|
||||||
@@ -453,7 +455,7 @@ export const listMissingThumbnailsCommand = async (
|
|||||||
): Promise<number> => {
|
): Promise<number> => {
|
||||||
await init();
|
await init();
|
||||||
const client = requireSession(ctx);
|
const client = requireSession(ctx);
|
||||||
if (!client) return 1;
|
if (!client) return 3;
|
||||||
const lib = await openReadLibrary(ctx, client);
|
const lib = await openReadLibrary(ctx, client);
|
||||||
try {
|
try {
|
||||||
// Refresh first so files added since the cache was written are
|
// Refresh first so files added since the cache was written are
|
||||||
@@ -491,7 +493,7 @@ export const fixMissingThumbnailsCommand = async (
|
|||||||
): Promise<number> => {
|
): Promise<number> => {
|
||||||
await init();
|
await init();
|
||||||
const client = requireSession(ctx);
|
const client = requireSession(ctx);
|
||||||
if (!client) return 1;
|
if (!client) return 3;
|
||||||
const lib = await openReadLibrary(ctx, client);
|
const lib = await openReadLibrary(ctx, client);
|
||||||
try {
|
try {
|
||||||
// Refresh first so files added since the cache was written are found;
|
// Refresh first so files added since the cache was written are found;
|
||||||
|
|||||||
+15
-5
@@ -1,12 +1,15 @@
|
|||||||
// Runs one CLI command for `bin/quak.ts` and exits with its code.
|
// Runs one CLI command for `bin/quak.ts` and exits with its code.
|
||||||
|
|
||||||
import type { Writable } from "node:stream";
|
import type { Writable } from "node:stream";
|
||||||
|
import { ApiError } from "./api/client.js";
|
||||||
|
|
||||||
// Run a command and exit with its code once stdout/stderr have drained.
|
// Run a command and exit with its code once stdout/stderr have drained.
|
||||||
// Exiting before the drain can truncate piped output, and the library can keep
|
// Exiting before the drain can truncate piped output, and the library can keep
|
||||||
// the event loop alive after a command returns, so a plain return could hang.
|
// the event loop alive after a command returns, so a plain return could hang.
|
||||||
// An error the command throws is printed as one `quak: MESSAGE` line, without
|
// An error the command throws is printed as one `quak: MESSAGE` line, without
|
||||||
// the stack trace, and exits 1.
|
// the stack trace, and exits 1. A 401 from the server means it no longer
|
||||||
|
// accepts the saved session: that prints one line saying to log in again and
|
||||||
|
// exits 3, as a missing or corrupt session file does.
|
||||||
export const run = async (
|
export const run = async (
|
||||||
command: Promise<number>,
|
command: Promise<number>,
|
||||||
stdout: Writable,
|
stdout: Writable,
|
||||||
@@ -17,10 +20,17 @@ export const run = async (
|
|||||||
try {
|
try {
|
||||||
code = await command;
|
code = await command;
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
stderr.write(
|
if (err instanceof ApiError && err.status === 401) {
|
||||||
`quak: ${err instanceof Error ? err.message : String(err)}\n`,
|
stderr.write(
|
||||||
);
|
`quak: the saved session is no longer valid; run "quak login"\n`,
|
||||||
code = 1;
|
);
|
||||||
|
code = 3;
|
||||||
|
} else {
|
||||||
|
stderr.write(
|
||||||
|
`quak: ${err instanceof Error ? err.message : String(err)}\n`,
|
||||||
|
);
|
||||||
|
code = 1;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
const pending = [stdout, stderr].filter((s) => s.writableLength > 0);
|
const pending = [stdout, stderr].filter((s) => s.writableLength > 0);
|
||||||
if (pending.length === 0) {
|
if (pending.length === 0) {
|
||||||
|
|||||||
@@ -571,7 +571,8 @@ export class Library {
|
|||||||
// when it fails. Then puts pending originals at their save paths as
|
// when it fails. Then puts pending originals at their save paths as
|
||||||
// `Photo.download()` does (and optional thumbnails) through the content
|
// `Photo.download()` does (and optional thumbnails) through the content
|
||||||
// cache and pools, waits for an ML data fetch, and rebuilds the derived
|
// cache and pools, waits for an ML data fetch, and rebuilds the derived
|
||||||
// symlink/JSON views from the model, each file's JSON with its ML data.
|
// symlink/JSON views from the model, each file's JSON with its ML data,
|
||||||
|
// beside an `account.json` with the account's email and user ID.
|
||||||
// Throws before any network work when no content cache backs the
|
// Throws before any network work when no content cache backs the
|
||||||
// originals it must fetch.
|
// originals it must fetch.
|
||||||
backup(opts?: BackupOptions): Promise<BackupResult> {
|
backup(opts?: BackupOptions): Promise<BackupResult> {
|
||||||
@@ -589,6 +590,7 @@ export class Library {
|
|||||||
const cache = this.cache;
|
const cache = this.cache;
|
||||||
return runBackup(
|
return runBackup(
|
||||||
{
|
{
|
||||||
|
whoami: () => this.client.whoami(),
|
||||||
refresh: () => this.refreshNow(),
|
refresh: () => this.refreshNow(),
|
||||||
listCollections: () => this.store.listCollections(),
|
listCollections: () => this.store.listCollections(),
|
||||||
listFiles: (id) => this.store.listFiles(id),
|
listFiles: (id) => this.store.listFiles(id),
|
||||||
|
|||||||
@@ -12,6 +12,7 @@
|
|||||||
* collections/
|
* collections/
|
||||||
* <name>/<title> symlink to the original (rebuilt each run)
|
* <name>/<title> symlink to the original (rebuilt each run)
|
||||||
* <name>.json per-collection metadata (rebuilt each run)
|
* <name>.json per-collection metadata (rebuilt each run)
|
||||||
|
* account.json the account's email and user ID
|
||||||
* failures.json durable ledger of unresolved failures
|
* failures.json durable ledger of unresolved failures
|
||||||
*
|
*
|
||||||
* The properties that distinguish backup from a naive download loop, and that
|
* The properties that distinguish backup from a naive download loop, and that
|
||||||
@@ -917,6 +918,106 @@ describe("ML data in each file's JSON", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// The fields `RecordsClient` gives the Vacation album: shared by another
|
||||||
|
// account, with all three layers of magic metadata.
|
||||||
|
const SHARED_ALBUM = {
|
||||||
|
ownerID: 7,
|
||||||
|
isShared: true,
|
||||||
|
updationTime: 1700,
|
||||||
|
magicMetadata: { visibility: 0 },
|
||||||
|
pubMagicMetadata: { coverID: 100 },
|
||||||
|
sharedMagicMetadata: { visibility: 2 },
|
||||||
|
};
|
||||||
|
|
||||||
|
// Serves the Vacation album with `SHARED_ALBUM`'s fields, and each file with
|
||||||
|
// the update time 5000 + its ID.
|
||||||
|
class RecordsClient extends MockClient {
|
||||||
|
override async collectionsSince(): Promise<CollectionsPage> {
|
||||||
|
const page = await super.collectionsSince();
|
||||||
|
return {
|
||||||
|
...page,
|
||||||
|
collections: page.collections.map((c) =>
|
||||||
|
c.id === 1 ? { ...c, ...SHARED_ALBUM } : c,
|
||||||
|
),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
override async filesSince(args: {
|
||||||
|
collectionID: number;
|
||||||
|
}): Promise<FilesPage> {
|
||||||
|
const page = await super.filesSince(args);
|
||||||
|
return {
|
||||||
|
...page,
|
||||||
|
files: page.files.map((f) => ({
|
||||||
|
...f,
|
||||||
|
updationTime: 5000 + f.id,
|
||||||
|
})),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The JSON the backup in `outDir` wrote for the album named `name`.
|
||||||
|
const albumJSON = (outDir: string, name: string): Record<string, unknown> =>
|
||||||
|
JSON.parse(
|
||||||
|
readFileSync(join(outDir, "collections", `${name}.json`), "utf-8"),
|
||||||
|
);
|
||||||
|
|
||||||
|
describe("account and album records", () => {
|
||||||
|
it("writes account.json with the account's email and user ID", async () => {
|
||||||
|
const lib = await openLibrary(stubSource());
|
||||||
|
const outDir = join(root, "backup");
|
||||||
|
|
||||||
|
await lib.backup({ downloadDirectory: outDir });
|
||||||
|
|
||||||
|
expect(
|
||||||
|
JSON.parse(readFileSync(join(outDir, "account.json"), "utf-8")),
|
||||||
|
).toEqual({ email: "backup@example.com", userID: USER_ID });
|
||||||
|
await lib.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("writes each album's owner, sharing, update time and magic metadata into its JSON", async () => {
|
||||||
|
const lib = await openLibrary(stubSource(), new RecordsClient());
|
||||||
|
const outDir = join(root, "backup");
|
||||||
|
|
||||||
|
await lib.backup({ downloadDirectory: outDir });
|
||||||
|
|
||||||
|
expect(albumJSON(outDir, "Vacation")).toEqual({
|
||||||
|
id: 1,
|
||||||
|
name: "Vacation",
|
||||||
|
type: "album",
|
||||||
|
...SHARED_ALBUM,
|
||||||
|
files: [
|
||||||
|
{ id: 100, metadata: file(100, 1, "beach.jpg").metadata },
|
||||||
|
{ id: 101, metadata: file(101, 1, "sunset.jpg").metadata },
|
||||||
|
],
|
||||||
|
});
|
||||||
|
// An album with no magic metadata gets no magic metadata fields.
|
||||||
|
expect(albumJSON(outDir, "Work")).toEqual({
|
||||||
|
id: 2,
|
||||||
|
name: "Work",
|
||||||
|
type: "album",
|
||||||
|
ownerID: USER_ID,
|
||||||
|
isShared: false,
|
||||||
|
updationTime: 1,
|
||||||
|
files: [
|
||||||
|
{ id: 200, metadata: file(200, 2, "diagram.png").metadata },
|
||||||
|
],
|
||||||
|
});
|
||||||
|
await lib.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("writes each file's update time into its JSON", async () => {
|
||||||
|
const lib = await openLibrary(stubSource(), new RecordsClient());
|
||||||
|
const outDir = join(root, "backup");
|
||||||
|
|
||||||
|
await lib.backup({ downloadDirectory: outDir });
|
||||||
|
|
||||||
|
for (const fileID of [100, 101, 200]) {
|
||||||
|
expect(fileJSON(outDir, fileID).updationTime).toBe(5000 + fileID);
|
||||||
|
}
|
||||||
|
await lib.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
// Every entry under collections/, one level of directories deep, with each
|
// Every entry under collections/, one level of directories deep, with each
|
||||||
// symlink's target.
|
// symlink's target.
|
||||||
const tree = (outDir: string): string[] => {
|
const tree = (outDir: string): string[] => {
|
||||||
@@ -962,6 +1063,7 @@ describe("backup album folders", () => {
|
|||||||
},
|
},
|
||||||
fetchMLData: async () => {},
|
fetchMLData: async () => {},
|
||||||
mlData: async () => undefined,
|
mlData: async () => undefined,
|
||||||
|
whoami: () => ({ email: "backup@example.com", userID: USER_ID }),
|
||||||
});
|
});
|
||||||
|
|
||||||
const albumID = (outDir: string, jsonName: string): number =>
|
const albumID = (outDir: string, jsonName: string): number =>
|
||||||
|
|||||||
+68
-18
@@ -230,20 +230,30 @@ describe("session file", () => {
|
|||||||
expect(JSON.parse(readFileSync(path, "utf-8"))).toEqual(snapshot);
|
expect(JSON.parse(readFileSync(path, "utf-8"))).toEqual(snapshot);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("a missing session exits 1 with 'Not logged in'", async () => {
|
it("a missing session exits 3 with 'Not logged in' from every command that needs one", async () => {
|
||||||
const ctx = { ...context(), loadSession };
|
const ctx = { ...context(), loadSession };
|
||||||
expect(await whoamiCommand(ctx)).toBe(1);
|
const dir = join(root, "backup");
|
||||||
expect(stderr.text).toBe(
|
expect(await whoamiCommand(ctx)).toBe(3);
|
||||||
|
expect(await collectionsCommand(ctx, {})).toBe(3);
|
||||||
|
expect(await filesCommand(ctx, { collection: "1" })).toBe(3);
|
||||||
|
expect(await getCommand(ctx, "100", {})).toBe(3);
|
||||||
|
expect(await getThumbCommand(ctx, "100", {})).toBe(3);
|
||||||
|
expect(await backupMetadataCommand(ctx, dir, {})).toBe(3);
|
||||||
|
expect(await backupCommand(ctx, dir, {})).toBe(3);
|
||||||
|
expect(await listMissingThumbnailsCommand(ctx, {})).toBe(3);
|
||||||
|
expect(await fixMissingThumbnailsCommand(ctx, {})).toBe(3);
|
||||||
|
const notLoggedIn =
|
||||||
`Not logged in. Run "quak login" first.\n` +
|
`Not logged in. Run "quak login" first.\n` +
|
||||||
`Session file: ${join(ctx.sessionDir, "session.json")}\n`,
|
`Session file: ${join(ctx.sessionDir, "session.json")}\n`;
|
||||||
);
|
expect(stderr.text).toBe(notLoggedIn.repeat(9));
|
||||||
expect(stdout.text).toBe("");
|
expect(stdout.text).toBe("");
|
||||||
|
expect(existsSync(dir)).toBe(false);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("a corrupt session exits 1 and says it is corrupt", async () => {
|
it("a corrupt session exits 3 and says it is corrupt", async () => {
|
||||||
const ctx = { ...context(), loadSession };
|
const ctx = { ...context(), loadSession };
|
||||||
saveSession(ctx.sessionDir, snapshot);
|
saveSession(ctx.sessionDir, snapshot);
|
||||||
expect(await collectionsCommand(ctx, {})).toBe(1);
|
expect(await collectionsCommand(ctx, {})).toBe(3);
|
||||||
expect(stderr.text).toContain("is corrupt");
|
expect(stderr.text).toContain("is corrupt");
|
||||||
expect(stderr.text).toContain(
|
expect(stderr.text).toContain(
|
||||||
`Run "quak logout" and then "quak login" to replace it.\n`,
|
`Run "quak logout" and then "quak login" to replace it.\n`,
|
||||||
@@ -748,15 +758,12 @@ describe("backup", () => {
|
|||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("exits 1 with the error on one line when the refresh fails", async () => {
|
// Runs `backup` through `run`, as `bin/quak.ts` does, which prints a thrown
|
||||||
const client = {
|
// error; returns the exit code and what `run` printed.
|
||||||
...fakeClient(),
|
const backupThroughRun = async (
|
||||||
collectionsSince: async () => {
|
ctx: CliContext,
|
||||||
throw new Error("HTTP 401 from server");
|
dir: string,
|
||||||
},
|
): Promise<{ code: number; runText: string }> => {
|
||||||
} as unknown as Client;
|
|
||||||
const dir = join(root, "backup");
|
|
||||||
// Through `run`, as `bin/quak.ts` does, which prints a thrown error.
|
|
||||||
const runStderr = new PassThrough();
|
const runStderr = new PassThrough();
|
||||||
let runText = "";
|
let runText = "";
|
||||||
runStderr.on("data", (chunk: Buffer) => {
|
runStderr.on("data", (chunk: Buffer) => {
|
||||||
@@ -764,14 +771,57 @@ describe("backup", () => {
|
|||||||
});
|
});
|
||||||
const code = await new Promise<number>((resolve) => {
|
const code = await new Promise<number>((resolve) => {
|
||||||
void run(
|
void run(
|
||||||
backupCommand(context(client), dir, {}),
|
backupCommand(ctx, dir, {}),
|
||||||
new PassThrough(),
|
new PassThrough(),
|
||||||
runStderr,
|
runStderr,
|
||||||
resolve,
|
resolve,
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
return { code, runText };
|
||||||
|
};
|
||||||
|
|
||||||
|
it("exits 1 with the error on one line when the refresh fails", async () => {
|
||||||
|
const client = {
|
||||||
|
...fakeClient(),
|
||||||
|
collectionsSince: async () => {
|
||||||
|
throw new Error("HTTP 503 from server");
|
||||||
|
},
|
||||||
|
} as unknown as Client;
|
||||||
|
const dir = join(root, "backup");
|
||||||
|
const { code, runText } = await backupThroughRun(context(client), dir);
|
||||||
expect(code).toBe(1);
|
expect(code).toBe(1);
|
||||||
expect(runText).toBe("quak: HTTP 401 from server\n");
|
expect(runText).toBe("quak: HTTP 503 from server\n");
|
||||||
|
expect(stderr.text).toBe("Starting backup...\nRefreshing library...\n");
|
||||||
|
expect(existsSync(dir)).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
// A real saved session, read back by `loadSession`, whose server answers
|
||||||
|
// every request with 401, as it does once it no longer accepts the token.
|
||||||
|
// The context's prompts throw, so a prompt would end the run with another
|
||||||
|
// line and exit 1.
|
||||||
|
it("exits 3 with one line saying to log in again when the server answers 401", async () => {
|
||||||
|
const key = toBase64(new Uint8Array(32));
|
||||||
|
saveSession(join(root, "session"), {
|
||||||
|
email: "cli@example.com",
|
||||||
|
userID: USER_ID,
|
||||||
|
token: "expired",
|
||||||
|
masterKey: key,
|
||||||
|
secretKey: key,
|
||||||
|
publicKey: key,
|
||||||
|
});
|
||||||
|
const unauthorized = async (): Promise<Response> =>
|
||||||
|
new Response(null, { status: 401 });
|
||||||
|
const ctx = {
|
||||||
|
...context(),
|
||||||
|
loadSession: (path: string) =>
|
||||||
|
loadSession(path, { fetch: unauthorized }),
|
||||||
|
};
|
||||||
|
const dir = join(root, "backup");
|
||||||
|
const { code, runText } = await backupThroughRun(ctx, dir);
|
||||||
|
expect(code).toBe(3);
|
||||||
|
expect(runText).toBe(
|
||||||
|
`quak: the saved session is no longer valid; run "quak login"\n`,
|
||||||
|
);
|
||||||
expect(stderr.text).toBe("Starting backup...\nRefreshing library...\n");
|
expect(stderr.text).toBe("Starting backup...\nRefreshing library...\n");
|
||||||
expect(existsSync(dir)).toBe(false);
|
expect(existsSync(dir)).toBe(false);
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -5,6 +5,7 @@
|
|||||||
import { PassThrough } from "node:stream";
|
import { PassThrough } from "node:stream";
|
||||||
import { describe, it, expect } from "vitest";
|
import { describe, it, expect } from "vitest";
|
||||||
|
|
||||||
|
import { ApiError } from "../../src/api/client.js";
|
||||||
import { run } from "../../src/cli-run.js";
|
import { run } from "../../src/cli-run.js";
|
||||||
|
|
||||||
// A stream whose written text is kept in `text`; writes finish at once, so
|
// A stream whose written text is kept in `text`; writes finish at once, so
|
||||||
@@ -55,4 +56,26 @@ describe("run", () => {
|
|||||||
stderr: "quak: offline\n",
|
stderr: "quak: offline\n",
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("on a 401 from the server says to run quak login, on one line, and exits 3", async () => {
|
||||||
|
const result = await runToExit(
|
||||||
|
Promise.reject(new ApiError("unauthorized", 401)),
|
||||||
|
);
|
||||||
|
expect(result).toEqual({
|
||||||
|
code: 3,
|
||||||
|
stdout: "",
|
||||||
|
stderr: `quak: the saved session is no longer valid; run "quak login"\n`,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("prints another HTTP error as it is and exits 1", async () => {
|
||||||
|
const result = await runToExit(
|
||||||
|
Promise.reject(new ApiError("forbidden", 403)),
|
||||||
|
);
|
||||||
|
expect(result).toEqual({
|
||||||
|
code: 1,
|
||||||
|
stdout: "",
|
||||||
|
stderr: "quak: forbidden\n",
|
||||||
|
});
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
Reference in New Issue
Block a user