Failed runs end with one fatal line; failure lines are errors
check / check (push) Canceled after 0s

A command that exits 1 because files or albums failed now ends with one
fatal line saying how many failed and where to look, so -q never leaves
a non-zero exit unexplained: backup points to failures.json, backup-metadata
to mlDataError in each file's JSON, and fix-missing-thumbnails to its
Failed files list. The progress callbacks of the backup, the metadata
backup and the thumbnail helpers take a level, so every line that reports
a failure is logged at error. The library's operation names are shown in
plain words. The README says which lines are error and warning, and which
summary lines --json replaces.

Model: opus-5-5
This commit is contained in:
2026-10-18 19:15:16 +00:00
parent f9969b2525
commit d87a5e7ef5
12 changed files with 253 additions and 63 deletions
+31 -11
View File
@@ -138,9 +138,9 @@ matters.
### Exit codes ### Exit codes
| Code | Meaning | | Code | Meaning |
| ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `0` | The backup is complete: every original is at its save path, and no file failed. | | `0` | The backup is complete: every original is at its save path, and no file failed. |
| `1` | The run finished with files in `failures.json` or albums it could not decrypt or write (see "Backup layout"), or it stopped on another error, printed as one line `quak: <message>`. The next run tries again. | | `1` | The run finished with files in `failures.json` or albums it could not decrypt or write (see "Backup layout"), and its last line says how many, or it stopped on another error, printed as one line `quak: <message>`. The next run tries again. |
| `2` | Another backup of the same directory is running. This run sent no request and changed nothing. | | `2` | Another backup of the same directory is running. This run sent no request and changed nothing. |
| `3` | There is no usable session: none is saved, the saved one is corrupt, or the server no longer accepts it. Run `quak login` as the user the cron job runs as. | | `3` | There is no usable session: none is saved, the saved one is corrupt, or the server no longer accepts it. Run `quak login` as the user the cron job runs as. |
@@ -695,8 +695,31 @@ own directory under the per-user cache path.
prints on stderr is a log line at one of six levels, from the quietest: fatal, prints on stderr is a log line at one of six levels, from the quietest: fatal,
error, warning, info, debug and trace. A run prints the lines at its own level error, warning, info, debug and trace. A run prints the lines at its own level
and at every quieter one. stdout holds only the command's output, at every and at every quieter one. stdout holds only the command's output, at every
level, and `--json` changes stdout only, so a run with `--json` logs the same level.
lines on stderr as one without it.
- Every command that exits non-zero logs a fatal line saying why, such as
`quak: <message>`, `Not logged in`, an invalid ID or a collection that does
not exist. A command that exits 1 because files or albums failed logs it last,
saying how many failed and where to look: `backup` points to `failures.json`
and its `FAILED` lines, `backup-metadata` to the `mlDataError` in the JSON of
each such file, and `helper fix-missing-thumbnails` to its `Failed files:`
list, or to its JSON with `--json`.
- An error line reports something that failed: the `FAILED` lines that `backup`
and `helper fix-missing-thumbnails` log as they work,
`ML data of file <id> could not be read` and
`ML data request for <n> file(s) failed` from `backup-metadata`,
`Could not check <title>` from both thumbnail helpers, and the `Failed files:`
list at the end of `backup` and of `helper fix-missing-thumbnails`.
- A warning reports something that did not fail but needs your attention: after
`logout`, that the cache directory still holds decrypted data.
- Every other line quak logs by default is info.
With `--json`, a command's result on stdout takes the place of its summary on
stderr: for `backup`, the lines from `--- Backup complete ---` to the end of its
`Failed files:` list; for `helper list-missing-thumbnails`,
`No missing thumbnails found.` or `<n> file(s) with missing thumbnails:`; and
for `helper fix-missing-thumbnails`, the lines from `--- Done ---` to the end of
its `Failed files:` list. Every other line is logged as it is without `--json`.
- With neither option the level is info: the lines quak prints as it works, such - With neither option the level is info: the lines quak prints as it works, such
as a refresh's progress, the files a backup fetches, its summary and the files as a refresh's progress, the files a backup fetches, its summary and the files
@@ -709,12 +732,9 @@ lines on stderr as one without it.
URL of an upload of a thumbnail is logged without its query, which holds the URL of an upload of a thumbnail is logged without its query, which holds the
upload's signature. More repeats stay at trace, and `--verbose --verbose` is upload's signature. More repeats stay at trace, and `--verbose --verbose` is
`-vv`. `-vv`.
- `-q` lowers it to fatal: quak prints only the line a command prints as it - `-q` lowers it to fatal: a run that exits 0 prints nothing on stderr, and one
fails and exits non-zero, such as `quak: <message>`, `Not logged in`, an that exits non-zero prints only its fatal lines. The prompts of `login` are
invalid ID or a collection that does not exist. A file that fails inside not log lines, and `-q` leaves them as they are.
`backup` is not fatal: it is in `failures.json` and in the exit code, and
nothing is printed for it. The prompts of `login` are not log lines, and `-q`
leaves them as they are.
`-q` with `-v` is refused with one line, `-q` with `-v` is refused with one line,
`error: option '-q, --quiet' cannot be used with option '-v, --verbose'`, and `error: option '-q, --quiet' cannot be used with option '-v, --verbose'`, and
@@ -772,7 +792,7 @@ fails, the error is logged, each of its files is written with the reason in an
`mlDataError` field instead of `mlData`, and the dump goes on. A file whose ML `mlDataError` field instead of `mlData`, and the dump goes on. A file whose ML
data does not decrypt, gunzip or parse is logged and written the same way, with data does not decrypt, gunzip or parse is logged and written the same way, with
why in `mlDataError`. The exit code is non-zero if any file was written with why in `mlDataError`. The exit code is non-zero if any file was written with
`mlDataError`. `mlDataError`, and the last line then says how many were.
`helper fix-missing-thumbnails` regenerates thumbnails for JPEG images only, `helper fix-missing-thumbnails` regenerates thumbnails for JPEG images only,
because the bundled decoder (`jpeg-js`) decodes only JPEG. A non-JPEG image because the bundled decoder (`jpeg-js`) decodes only JPEG. A non-JPEG image
+4 -1
View File
@@ -36,7 +36,10 @@ declares one.
end of each refresh and ML data fetch; trace adds each request, which end of each refresh and ML data fetch; trace adds each request, which
`ApiClient` reports through the new `onRequest`. `--json` no longer keeps the `ApiClient` reports through the new `onRequest`. `--json` no longer keeps the
progress lines off stderr, so `backup` and `--json` runs log a refresh's progress lines off stderr, so `backup` and `--json` runs log a refresh's
progress too. progress too. Every line that reports a failed file or album is at error,
which the progress callbacks of `backup`, `backup-metadata` and the thumbnail
helpers now say, and a command that exits 1 because files or albums failed
ends with one fatal line saying how many and where to look.
- 2026-10-18: `quak files` takes the collection ID as an optional argument - 2026-10-18: `quak files` takes the collection ID as an optional argument
instead of the `--collection` option, which is gone instead of the `--collection` option, which is gone
+15 -5
View File
@@ -101,7 +101,10 @@ import type { MLData } from "./mldata-fetch.js";
import type { Collection, EnteFile, FileMetadata } from "./model/types.js"; import type { Collection, EnteFile, FileMetadata } from "./model/types.js";
import { hasTransportCode } from "./retry.js"; import { hasTransportCode } from "./retry.js";
export type ProgressCallback = (message: string) => void; // Reports one line of the backup's progress. `level` is "error" when the line
// reports something that failed, such as a file or an album, and is left out
// for every other line.
export type ProgressCallback = (message: string, level?: "error") => void;
// The lock in a backup directory, as `lockBackupDirectory` returns it. // The lock in a backup directory, as `lockBackupDirectory` returns it.
export interface BackupLock { export interface BackupLock {
@@ -654,7 +657,7 @@ const runLockedBackup = async (
for (const record of lib.undecryptable()) { for (const record of lib.undecryptable()) {
const error = `could not be decrypted: ${record.error}`; const error = `could not be decrypted: ${record.error}`;
if (record.fileID === undefined) { if (record.fileID === undefined) {
log(`FAILED album ${record.collectionID}: ${error}`); log(`FAILED album ${record.collectionID}: ${error}`, "error");
undecryptableAlbums.push(record.collectionID); undecryptableAlbums.push(record.collectionID);
} else if (scope.has(record.collectionID)) { } else if (scope.has(record.collectionID)) {
const copies = memberships.get(record.fileID); const copies = memberships.get(record.fileID);
@@ -699,6 +702,7 @@ const runLockedBackup = async (
} catch (err) { } catch (err) {
log( log(
`FAILED verifying original ${file.metadata.title}: ${describeError(err)}`, `FAILED verifying original ${file.metadata.title}: ${describeError(err)}`,
"error",
); );
recordFailure( recordFailure(
file.id, file.id,
@@ -732,6 +736,7 @@ const runLockedBackup = async (
} catch (err) { } catch (err) {
log( log(
`FAILED original ${file.metadata.title}: ${describeError(err)}`, `FAILED original ${file.metadata.title}: ${describeError(err)}`,
"error",
); );
recordFailure( recordFailure(
file.id, file.id,
@@ -807,7 +812,7 @@ const runLockedBackup = async (
await lib.fetchMLData(); await lib.fetchMLData();
} catch (err) { } catch (err) {
mlDataFailure = err; mlDataFailure = err;
log(`FAILED ML data: ${describeError(err)}`); log(`FAILED ML data: ${describeError(err)}`, "error");
} }
for (const file of distinct.values()) { for (const file of distinct.values()) {
if (lock.lost) throw lock.lost; if (lock.lost) throw lock.lost;
@@ -868,7 +873,10 @@ const runLockedBackup = async (
downloadDirectory, downloadDirectory,
); );
} catch (err) { } catch (err) {
log(`FAILED removing old album directories: ${describeError(err)}`); log(
`FAILED removing old album directories: ${describeError(err)}`,
"error",
);
} }
// An album whose folder or JSON cannot be written, such as one whose // An album whose folder or JSON cannot be written, such as one whose
@@ -897,6 +905,7 @@ const runLockedBackup = async (
} catch (err) { } catch (err) {
log( log(
`FAILED removing old links in ${c.name}: ${describeError(err)}`, `FAILED removing old links in ${c.name}: ${describeError(err)}`,
"error",
); );
} }
@@ -915,6 +924,7 @@ const runLockedBackup = async (
} catch (err) { } catch (err) {
log( log(
`FAILED symlink ${c.name}/${linkName}: ${describeError(err)}`, `FAILED symlink ${c.name}/${linkName}: ${describeError(err)}`,
"error",
); );
recordFailure( recordFailure(
link.file.id, link.file.id,
@@ -931,7 +941,7 @@ const runLockedBackup = async (
metaFiles, metaFiles,
); );
} catch (err) { } catch (err) {
log(`FAILED album ${c.name}: ${describeError(err)}`); log(`FAILED album ${c.name}: ${describeError(err)}`, "error");
failedAlbums.push(c.id); failedAlbums.push(c.id);
} }
} }
+33 -17
View File
@@ -5,8 +5,9 @@
// `process.exit`: `bin/quak.ts` wires these to the command line, and `run` in // `process.exit`: `bin/quak.ts` wires these to the command line, and `run` in
// `cli-run.ts` prints a thrown error as one line and exits once output has // `cli-run.ts` prints a thrown error as one line and exits once output has
// drained. Output must stay byte-identical (see `cli-output.ts`). Each line on // drained. Output must stay byte-identical (see `cli-output.ts`). Each line on
// stderr goes through `ctx.log` at its level (see `cli-log.ts`); a line a // stderr goes through `ctx.log` at its level (see `cli-log.ts`). Every command
// command prints as it fails with a non-zero exit code is fatal. // that exits non-zero logs a fatal line saying why; one that exits 1 because
// files or albums failed logs it last, saying how many and where to look.
import { import {
copyFileSync, copyFileSync,
@@ -414,9 +415,14 @@ export const backupMetadataCommand = async (
await lib.fresh(); await lib.fresh();
const result = await runMetadataBackup(lib, client, dir, { const result = await runMetadataBackup(lib, client, dir, {
exif: opts.exif || opts.all, exif: opts.exif || opts.all,
onProgress: (msg) => ctx.log("info", msg), onProgress: (msg, level) => ctx.log(level ?? "info", msg),
}); });
return result.filesWithMLDataError > 0 ? 1 : 0; if (result.filesWithMLDataError === 0) return 0;
ctx.log(
"fatal",
`quak: the ML data of ${result.filesWithMLDataError} file(s) could not be read; see mlDataError in their JSON in ${dir}`,
);
return 1;
} finally { } finally {
await lib.close(); await lib.close();
} }
@@ -461,7 +467,7 @@ export const backupCommand = async (
downloadDirectory: dir, downloadDirectory: dir,
lock, lock,
verify: opts.verify, verify: opts.verify,
onProgress: (msg) => ctx.log("info", msg), onProgress: (msg, level) => ctx.log(level ?? "info", msg),
}); });
if (opts.json) { if (opts.json) {
@@ -488,11 +494,14 @@ export const backupCommand = async (
} }
} }
return result.failed > 0 || const albums =
result.undecryptableAlbums.length > 0 || result.undecryptableAlbums.length + result.failedAlbums.length;
result.failedAlbums.length > 0 if (result.failed === 0 && albums === 0) return 0;
? 1 ctx.log(
: 0; "fatal",
`quak: ${result.failed} file(s) and ${albums} album(s) failed; see ${join(dir, "failures.json")} and the FAILED lines`,
);
return 1;
} finally { } finally {
await lib.close(); await lib.close();
} }
@@ -513,8 +522,8 @@ export const listMissingThumbnailsCommand = async (
// Refresh first so files added since the cache was written are // Refresh first so files added since the cache was written are
// checked; a failed refresh throws. // checked; a failed refresh throws.
await lib.fresh(); await lib.fresh();
const missing = await listMissingThumbnails(lib, client, (msg) => const missing = await listMissingThumbnails(lib, client, (msg, level) =>
ctx.log("info", msg), ctx.log(level ?? "info", msg),
); );
if (opts.json) { if (opts.json) {
@@ -561,8 +570,10 @@ export const fixMissingThumbnailsCommand = async (
await lib.fresh(); await lib.fresh();
if (fileIDs.length === 0) { if (fileIDs.length === 0) {
ctx.log("info", "Scanning for missing thumbnails..."); ctx.log("info", "Scanning for missing thumbnails...");
const missing = await listMissingThumbnails(lib, client, (msg) => const missing = await listMissingThumbnails(
ctx.log("info", msg), lib,
client,
(msg, level) => ctx.log(level ?? "info", msg),
); );
fileIDs = missing.map((m) => m.fileID); fileIDs = missing.map((m) => m.fileID);
if (fileIDs.length === 0) { if (fileIDs.length === 0) {
@@ -576,9 +587,10 @@ export const fixMissingThumbnailsCommand = async (
lib, lib,
client, client,
fileIDs, fileIDs,
(msg) => ctx.log("info", msg), (msg, level) => ctx.log(level ?? "info", msg),
); );
const failed = results.filter((r) => r.status === "failed").length;
if (opts.json) { if (opts.json) {
ctx.stdout.write(JSON.stringify(results, null, 2) + "\n"); ctx.stdout.write(JSON.stringify(results, null, 2) + "\n");
} else { } else {
@@ -586,7 +598,6 @@ export const fixMissingThumbnailsCommand = async (
const skipped = results.filter( const skipped = results.filter(
(r) => r.status === "skipped", (r) => r.status === "skipped",
).length; ).length;
const failed = results.filter((r) => r.status === "failed").length;
ctx.log("info", `\n--- Done ---`); ctx.log("info", `\n--- Done ---`);
ctx.log("info", ` Fixed: ${fixed}`); ctx.log("info", ` Fixed: ${fixed}`);
ctx.log("info", ` Skipped: ${skipped}`); ctx.log("info", ` Skipped: ${skipped}`);
@@ -605,7 +616,12 @@ export const fixMissingThumbnailsCommand = async (
} }
} }
return results.some((r) => r.status === "failed") ? 1 : 0; if (failed === 0) return 0;
ctx.log(
"fatal",
`quak: ${failed} file(s) failed; see the "Failed files:" list, or the JSON with --json`,
);
return 1;
} finally { } finally {
await lib.close(); await lib.close();
} }
+19 -4
View File
@@ -4,9 +4,12 @@
// Every line quak prints on stderr is a log line at one of six levels, from // Every line quak prints on stderr is a log line at one of six levels, from
// the quietest: fatal, error, warning, info, debug, trace. A run prints the // the quietest: fatal, error, warning, info, debug, trace. A run prints the
// lines at its own level and at every quieter one. Its level is info unless // lines at its own level and at every quieter one. Its level is info unless
// `-v` or `-q` asks for another (see "CLI surface" in the README). A fatal line // `-v` or `-q` asks for another (see "CLI surface" in the README). Every
// is the one a command prints when it fails and exits non-zero, so even `-q` // command that exits non-zero logs a fatal line saying why, so even `-q`, which
// prints it. stdout carries only a command's output and is not a log. // prints only fatal lines, never leaves a non-zero exit unexplained. An error
// line reports something that failed, such as a file or an album, and a
// warning something that did not fail but needs the user's attention. stdout
// carries only a command's output and is not a log.
// //
// The library itself writes no lines: its layers report through callbacks, and // The library itself writes no lines: its layers report through callbacks, and
// the CLI logs what they report. `logRequests` gives the API client options // the CLI logs what they report. `logRequests` gives the API client options
@@ -66,6 +69,15 @@ export const logRequests = (
), ),
}); });
// How a log line names each operation the library reports the start and end
// of.
const OPERATION_NAMES: Record<RefreshEvent["operation"], string> = {
refresh: "Refresh",
fetchMLData: "ML data fetch",
precacheThumbnails: "Thumbnail precache",
precacheOriginals: "Original precache",
};
// A library `onProgress` that logs, at info, each collection whose files a // A library `onProgress` that logs, at info, each collection whose files a
// refresh has fetched, and at debug the start and end of each refresh and ML // refresh has fetched, and at debug the start and end of each refresh and ML
// data fetch, with the error of one that failed. // data fetch, with the error of one that failed.
@@ -80,5 +92,8 @@ export const logRefresh =
return; return;
} }
const error = event.error === undefined ? "" : `: ${event.error}`; const error = event.error === undefined ? "" : `: ${event.error}`;
log("debug", `${event.operation} ${event.status}${error}`); log(
"debug",
`${OPERATION_NAMES[event.operation]} ${event.status}${error}`,
);
}; };
+9 -2
View File
@@ -17,7 +17,10 @@ import {
} from "./mldata-fetch.js"; } from "./mldata-fetch.js";
import type { EnteFile } from "./model/types.js"; import type { EnteFile } from "./model/types.js";
export type ProgressCallback = (message: string) => void; // Reports one line of the dump's progress. `level` is "error" when the line
// reports ML data that could not be read, and is left out for every other
// line.
export type ProgressCallback = (message: string, level?: "error") => void;
export interface MetadataBackupOptions { export interface MetadataBackupOptions {
exif?: boolean; exif?: boolean;
@@ -195,13 +198,17 @@ export const runMetadataBackup = async (
for (const [id, payload] of payloads) mlDataMap.set(id, payload); for (const [id, payload] of payloads) mlDataMap.set(id, payload);
for (const [id, err] of errors) { for (const [id, err] of errors) {
const reason = describeError(err); const reason = describeError(err);
log(`ML data of file ${id} could not be read: ${reason}`); log(
`ML data of file ${id} could not be read: ${reason}`,
"error",
);
mlDataErrors.set(id, reason); mlDataErrors.set(id, reason);
} }
} catch (err) { } catch (err) {
const reason = err instanceof Error ? err.message : String(err); const reason = err instanceof Error ? err.message : String(err);
log( log(
`ML data request for ${batch.length} file(s) failed: ${reason}`, `ML data request for ${batch.length} file(s) failed: ${reason}`,
"error",
); );
for (const id of batch) mlDataErrors.set(id, reason); for (const id of batch) mlDataErrors.set(id, reason);
} }
+6 -1
View File
@@ -49,7 +49,10 @@ export interface ThumbnailFixResult {
reason?: string; reason?: string;
} }
export type ProgressCallback = (message: string) => void; // Reports one line of progress. `level` is "error" when the line reports a
// file that could not be checked or fixed, and is left out for every other
// line.
export type ProgressCallback = (message: string, level?: "error") => void;
// Enumerate every file the library knows about, newest album first, each file // Enumerate every file the library knows about, newest album first, each file
// once, and report those whose server-side thumbnail is missing. "Missing" is // once, and report those whose server-side thumbnail is missing. "Missing" is
@@ -111,6 +114,7 @@ export const listMissingThumbnails = async (
} else { } else {
log( log(
`[${album.name}] Could not check ${photo.title}: ${err instanceof Error ? err.message : String(err)} (not reported as missing)`, `[${album.name}] Could not check ${photo.title}: ${err instanceof Error ? err.message : String(err)} (not reported as missing)`,
"error",
); );
} }
} }
@@ -359,6 +363,7 @@ export const fixMissingThumbnails = async (
} catch (err) { } catch (err) {
log( log(
`[${collectionName}] FAILED ${title}: ${err instanceof Error ? err.message : err}`, `[${collectionName}] FAILED ${title}: ${err instanceof Error ? err.message : err}`,
"error",
); );
results.push({ results.push({
fileID, fileID,
+18
View File
@@ -371,6 +371,24 @@ describe("quak backup with records it cannot decrypt", () => {
expect(stderr.text).toContain(" Failed: 2\n"); expect(stderr.text).toContain(" Failed: 2\n");
}); });
// The album is an error line. The last line is fatal and counts the files
// and the album, so `-q`, which prints fatal lines only, still says why
// the backup exits 1.
it("logs the album at error, and ends with one fatal line counting the files and the album", async () => {
const ctx = {
...context(brokenAccount()),
log: stderrLog(stderr, "error"),
};
expect(await backupCommand(ctx, backupDir(), {})).toBe(1);
expect(stderr.text).toMatch(
/^FAILED album 2: could not be decrypted: /,
);
expect(stderr.text.split("\n").at(-2)).toBe(
`quak: 2 file(s) and 1 album(s) failed; see ${join(backupDir(), "failures.json")} and the FAILED lines`,
);
});
it("names each album it cannot decrypt in the --json result", async () => { it("names each album it cannot decrypt in the --json result", async () => {
expect(await backup(brokenAccount(), { json: true })).toBe(1); expect(await backup(brokenAccount(), { json: true })).toBe(1);
+60
View File
@@ -913,6 +913,38 @@ describe("backup", () => {
); );
}); });
// A failed file is an error line both where the backup fails it and in
// the list at the end. The last line is fatal, so `-q`, which prints
// fatal lines only, still says why the backup exits 1.
it("logs a failed file at error, and ends with one fatal line counting the failures", async () => {
const dir = join(root, "backup");
const last = `quak: 1 file(s) and 0 album(s) failed; see ${join(dir, "failures.json")} and the FAILED lines\n`;
const ctx = context(fakeClient({ failID: 101 }));
expect(
await backupCommand(
{ ...ctx, log: stderrLog(stderr, "error") },
dir,
{},
),
).toBe(1);
expect(stderr.text).toBe(
"FAILED original sunset.jpg: HTTP 500 from server\n" +
"\nFailed files:\n" +
" [Vacation] sunset.jpg (id 101): HTTP 500 from server\n" +
last,
);
stderr.text = "";
expect(
await backupCommand(
{ ...ctx, log: stderrLog(stderr, "fatal") },
dir,
{ json: true },
),
).toBe(1);
expect(stderr.text).toBe(last);
});
// `--json` shapes stdout only: the log lines stay on stderr, and the JSON // `--json` shapes stdout only: the log lines stay on stderr, and the JSON
// takes the place of the summary. // takes the place of the summary.
it("prints the result as JSON with --json, still exiting 1 on a failure", async () => { it("prints the result as JSON with --json, still exiting 1 on a failure", async () => {
@@ -1303,6 +1335,34 @@ describe("helper list-missing-thumbnails", () => {
}); });
}); });
describe("helper fix-missing-thumbnails", () => {
// File 999 is in no album, so fixing it fails. The list of failed files
// is at error, and the last line is fatal, so `-q`, which prints fatal
// lines only, still says why the command exits 1.
it("lists a failed file at error, and ends with one fatal line counting the failures", async () => {
const last = `quak: 1 file(s) failed; see the "Failed files:" list, or the JSON with --json\n`;
const ctx = { ...context(), log: stderrLog(stderr, "error") };
expect(await fixMissingThumbnailsCommand(ctx, { file: ["999"] })).toBe(
1,
);
expect(stderr.text).toBe(
"\nFailed files:\n" +
" 999\tunknown\tfile not found in any collection\n" +
last,
);
stderr.text = "";
const quiet = { ...context(), log: stderrLog(stderr, "fatal") };
expect(
await fixMissingThumbnailsCommand(quiet, {
file: ["999"],
json: true,
}),
).toBe(1);
expect(stderr.text).toBe(last);
});
});
describe("backup-metadata --exif", () => { describe("backup-metadata --exif", () => {
// Runs the command and returns what it printed to stderr. // Runs the command and returns what it printed to stderr.
const backupMetadata = async (opts: { exif?: boolean; all?: boolean }) => { const backupMetadata = async (opts: { exif?: boolean; all?: boolean }) => {
+9 -6
View File
@@ -173,13 +173,13 @@ const FILES = "100\timage\tbeach.jpg\n";
// line for the one album whose files it fetched. Debug adds the start and end // line for the one album whose files it fetched. Debug adds the start and end
// of the refresh, and trace each request with the status of its answer. // of the refresh, and trace each request with the status of its answer.
const INFO = "Fetched the files of 1 of 1 collections\n"; const INFO = "Fetched the files of 1 of 1 collections\n";
const DEBUG = "refresh started\n" + INFO + "refresh done\n"; const DEBUG = "Refresh started\n" + INFO + "Refresh done\n";
const TRACE = const TRACE =
"refresh started\n" + "Refresh started\n" +
"GET https://api.ente.io/collections/v2?sinceTime=0: HTTP 200\n" + "GET https://api.ente.io/collections/v2?sinceTime=0: HTTP 200\n" +
"GET https://api.ente.io/collections/v2/diff?collectionID=1&sinceTime=0: HTTP 200\n" + "GET https://api.ente.io/collections/v2/diff?collectionID=1&sinceTime=0: HTTP 200\n" +
INFO + INFO +
"refresh done\n"; "Refresh done\n";
describe("the log level", () => { describe("the log level", () => {
it("is info by default", async () => { it("is info by default", async () => {
@@ -326,6 +326,7 @@ describe("what the library reports", () => {
]); ]);
}); });
// Each operation is named in plain words, as the README names it.
it("logs a refresh's progress at info, and its start and end at debug", () => { it("logs a refresh's progress at info, and its start and end at debug", () => {
const { log, lines } = collect(); const { log, lines } = collect();
const onProgress = logRefresh(log); const onProgress = logRefresh(log);
@@ -342,13 +343,15 @@ describe("what the library reports", () => {
status: "failed", status: "failed",
error: "HTTP 503", error: "HTTP 503",
}); });
onProgress({ operation: "fetchMLData", status: "started" });
onProgress({ operation: "fetchMLData", status: "done", fetched: 3 }); onProgress({ operation: "fetchMLData", status: "done", fetched: 3 });
expect(lines).toEqual([ expect(lines).toEqual([
"debug: refresh started", "debug: Refresh started",
"info: Fetched the files of 1 of 2 collections", "info: Fetched the files of 1 of 2 collections",
"debug: refresh failed: HTTP 503", "debug: Refresh failed: HTTP 503",
"debug: fetchMLData done", "debug: ML data fetch started",
"debug: ML data fetch done",
]); ]);
}); });
}); });
+19 -9
View File
@@ -47,7 +47,7 @@ import {
type MetadataBackupOptions, type MetadataBackupOptions,
} from "../../src/metadata-backup.js"; } from "../../src/metadata-backup.js";
import { backupMetadataCommand } from "../../src/cli-commands.js"; import { backupMetadataCommand } from "../../src/cli-commands.js";
import { stderrLog } from "../../src/cli-log.js"; import { stderrLog, type LogLevel } from "../../src/cli-log.js";
import { import {
buildAccount, buildAccount,
buildMetaFetch, buildMetaFetch,
@@ -491,9 +491,12 @@ describe("quak backup-metadata", () => {
}); });
describe("quak backup-metadata when a file's ML data cannot be read", () => { describe("quak backup-metadata when a file's ML data cannot be read", () => {
// Run the CLI command against `fetch`, the mock unless given, and return // Run the CLI command against `fetch`, the mock unless given, logging at
// its exit code, stderr and output directory. // `level`, and return its exit code, stderr and output directory.
const runCommand = async (fetch = buildMetaFetch(mock)) => { const runCommand = async (
fetch = buildMetaFetch(mock),
level: LogLevel = "info",
) => {
const client = await Client.login({ const client = await Client.login({
email: TEST_EMAIL, email: TEST_EMAIL,
password: TEST_PASSWORD, password: TEST_PASSWORD,
@@ -509,7 +512,7 @@ describe("quak backup-metadata when a file's ML data cannot be read", () => {
stdout: { write: () => true }, stdout: { write: () => true },
log: stderrLog( log: stderrLog(
{ write: (text: string) => (stderr += text) }, { write: (text: string) => (stderr += text) },
"info", level,
), ),
sessionDir: testDir, sessionDir: testDir,
cacheDir: mkdtempSync(join(testDir, "cache-")), cacheDir: mkdtempSync(join(testDir, "cache-")),
@@ -521,13 +524,16 @@ describe("quak backup-metadata when a file's ML data cannot be read", () => {
return { code, stderr, outDir }; return { code, stderr, outDir };
}; };
// The failed request is logged at error, so it is the first line printed
// at that level.
it("writes every file, marks the failed batch's files, and exits 1", async () => { it("writes every file, marks the failed batch's files, and exits 1", async () => {
const { code, stderr, outDir } = await runCommand( const { code, stderr, outDir } = await runCommand(
buildMetaFetch(mock, 200), buildMetaFetch(mock, 200),
"error",
); );
expect(code).toBe(1); expect(code).toBe(1);
expect(stderr).toContain("ML data request for 1 file(s) failed"); expect(stderr).toMatch(/^ML data request for 1 file\(s\) failed: /);
const ok = JSON.parse( const ok = JSON.parse(
readFileSync( readFileSync(
@@ -550,7 +556,9 @@ describe("quak backup-metadata when a file's ML data cannot be read", () => {
}); });
// The server sends ML data for diagram.png encrypted under a key that is // The server sends ML data for diagram.png encrypted under a key that is
// not the file's, so quak cannot decrypt it. // not the file's, so quak cannot decrypt it. That is logged at error, and
// the last line is fatal, so `-q`, which prints fatal lines only, still
// says why the command exits 1.
it("writes the reason for ML data that does not decrypt, and exits 1", async () => { it("writes the reason for ML data that does not decrypt, and exits 1", async () => {
const otherKey = sodium.crypto_secretstream_xchacha20poly1305_keygen(); const otherKey = sodium.crypto_secretstream_xchacha20poly1305_keygen();
const { header, ciphertext } = encryptBlob( const { header, ciphertext } = encryptBlob(
@@ -568,12 +576,14 @@ describe("quak backup-metadata when a file's ML data cannot be read", () => {
}, },
}, },
}), }),
"error",
); );
expect(code).toBe(1); expect(code).toBe(1);
const reason = "secretstream chunk authentication failed"; const reason = "secretstream chunk authentication failed";
expect(stderr).toContain( expect(stderr).toBe(
`ML data of file 200 could not be read: ${reason}\n`, `ML data of file 200 could not be read: ${reason}\n` +
`quak: the ML data of 1 file(s) could not be read; see mlDataError in their JSON in ${outDir}\n`,
); );
const failed = JSON.parse( const failed = JSON.parse(
readFileSync( readFileSync(
+26 -3
View File
@@ -544,11 +544,22 @@ describe("listMissingThumbnails", () => {
const client = await login(counted.fetch, { ...noWait }); const client = await login(counted.fetch, { ...noWait });
const lib = await openLib(client); const lib = await openLib(client);
const missing = await listMissingThumbnails(lib, client); const errors: string[] = [];
const missing = await listMissingThumbnails(
lib,
client,
(msg, level) => {
if (level === "error") errors.push(msg);
},
);
lib.close(); lib.close();
// Only the genuinely empty thumbnail is reported. // Only the genuinely empty thumbnail is reported. The file that
// could not be checked is reported as an error.
expect(missing.map((m) => m.fileID)).toEqual([101]); expect(missing.map((m) => m.fileID)).toEqual([101]);
expect(errors).toEqual([
expect.stringMatching(/^\[.*\] Could not check file-102\.jpg: /),
]);
// And the 500 was retried rather than accepted as an answer: four // And the 500 was retried rather than accepted as an answer: four
// attempts is the library default. // attempts is the library default.
expect(counted.matched()).toBe(4); expect(counted.matched()).toBe(4);
@@ -699,7 +710,15 @@ describe("fixMissingThumbnails", () => {
const client = await login(buildThumbFetch(fixMock)); const client = await login(buildThumbFetch(fixMock));
const lib = await openLib(client); const lib = await openLib(client);
const results = await fixMissingThumbnails(lib, client, [101, 102]); const errors: string[] = [];
const results = await fixMissingThumbnails(
lib,
client,
[101, 102],
(msg, level) => {
if (level === "error") errors.push(msg);
},
);
lib.close(); lib.close();
expect(results.length).toBe(2); expect(results.length).toBe(2);
@@ -707,6 +726,10 @@ describe("fixMissingThumbnails", () => {
const failure = results.find((r) => r.fileID === 102)!; const failure = results.find((r) => r.fileID === 102)!;
expect(success.status).toBe("fixed"); expect(success.status).toBe("fixed");
expect(failure.status).toBe("failed"); expect(failure.status).toBe("failed");
// The failed file, and only it, is reported as an error.
expect(errors).toEqual([
expect.stringMatching(/^\[.*\] FAILED file-102\.jpg: /),
]);
}); });
it("skips a non-JPEG image and a video as unsupported, not failed (issue #17)", async () => { it("skips a non-JPEG image and a video as unsupported, not failed (issue #17)", async () => {