A command that exits 1 because files or albums failed now ends with one fatal line saying how many failed and where to look, so -q never leaves a non-zero exit unexplained: backup points to failures.json, backup-metadata to mlDataError in each file's JSON, and fix-missing-thumbnails to its Failed files list. The progress callbacks of the backup, the metadata backup and the thumbnail helpers take a level, so every line that reports a failure is logged at error. The library's operation names are shown in plain words. The README says which lines are error and warning, and which summary lines --json replaces. Model: opus-5-5
This commit is contained in:
@@ -137,12 +137,12 @@ matters.
|
||||
|
||||
### Exit codes
|
||||
|
||||
| Code | Meaning |
|
||||
| ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `0` | The backup is complete: every original is at its save path, and no file failed. |
|
||||
| `1` | The run finished with files in `failures.json` or albums it could not decrypt or write (see "Backup layout"), or it stopped on another error, printed as one line `quak: <message>`. The next run tries again. |
|
||||
| `2` | Another backup of the same directory is running. This run sent no request and changed nothing. |
|
||||
| `3` | There is no usable session: none is saved, the saved one is corrupt, or the server no longer accepts it. Run `quak login` as the user the cron job runs as. |
|
||||
| Code | Meaning |
|
||||
| ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `0` | The backup is complete: every original is at its save path, and no file failed. |
|
||||
| `1` | The run finished with files in `failures.json` or albums it could not decrypt or write (see "Backup layout"), and its last line says how many, or it stopped on another error, printed as one line `quak: <message>`. The next run tries again. |
|
||||
| `2` | Another backup of the same directory is running. This run sent no request and changed nothing. |
|
||||
| `3` | There is no usable session: none is saved, the saved one is corrupt, or the server no longer accepts it. Run `quak login` as the user the cron job runs as. |
|
||||
|
||||
`<message>` is the error's message, followed by the message of each error that
|
||||
caused it. Under Node, as `yarn quak` runs quak, `fetch` reports every failure
|
||||
@@ -695,8 +695,31 @@ own directory under the per-user cache path.
|
||||
prints on stderr is a log line at one of six levels, from the quietest: fatal,
|
||||
error, warning, info, debug and trace. A run prints the lines at its own level
|
||||
and at every quieter one. stdout holds only the command's output, at every
|
||||
level, and `--json` changes stdout only, so a run with `--json` logs the same
|
||||
lines on stderr as one without it.
|
||||
level.
|
||||
|
||||
- Every command that exits non-zero logs a fatal line saying why, such as
|
||||
`quak: <message>`, `Not logged in`, an invalid ID or a collection that does
|
||||
not exist. A command that exits 1 because files or albums failed logs it last,
|
||||
saying how many failed and where to look: `backup` points to `failures.json`
|
||||
and its `FAILED` lines, `backup-metadata` to the `mlDataError` in the JSON of
|
||||
each such file, and `helper fix-missing-thumbnails` to its `Failed files:`
|
||||
list, or to its JSON with `--json`.
|
||||
- An error line reports something that failed: the `FAILED` lines that `backup`
|
||||
and `helper fix-missing-thumbnails` log as they work,
|
||||
`ML data of file <id> could not be read` and
|
||||
`ML data request for <n> file(s) failed` from `backup-metadata`,
|
||||
`Could not check <title>` from both thumbnail helpers, and the `Failed files:`
|
||||
list at the end of `backup` and of `helper fix-missing-thumbnails`.
|
||||
- A warning reports something that did not fail but needs your attention: after
|
||||
`logout`, that the cache directory still holds decrypted data.
|
||||
- Every other line quak logs by default is info.
|
||||
|
||||
With `--json`, a command's result on stdout takes the place of its summary on
|
||||
stderr: for `backup`, the lines from `--- Backup complete ---` to the end of its
|
||||
`Failed files:` list; for `helper list-missing-thumbnails`,
|
||||
`No missing thumbnails found.` or `<n> file(s) with missing thumbnails:`; and
|
||||
for `helper fix-missing-thumbnails`, the lines from `--- Done ---` to the end of
|
||||
its `Failed files:` list. Every other line is logged as it is without `--json`.
|
||||
|
||||
- With neither option the level is info: the lines quak prints as it works, such
|
||||
as a refresh's progress, the files a backup fetches, its summary and the files
|
||||
@@ -709,12 +732,9 @@ lines on stderr as one without it.
|
||||
URL of an upload of a thumbnail is logged without its query, which holds the
|
||||
upload's signature. More repeats stay at trace, and `--verbose --verbose` is
|
||||
`-vv`.
|
||||
- `-q` lowers it to fatal: quak prints only the line a command prints as it
|
||||
fails and exits non-zero, such as `quak: <message>`, `Not logged in`, an
|
||||
invalid ID or a collection that does not exist. A file that fails inside
|
||||
`backup` is not fatal: it is in `failures.json` and in the exit code, and
|
||||
nothing is printed for it. The prompts of `login` are not log lines, and `-q`
|
||||
leaves them as they are.
|
||||
- `-q` lowers it to fatal: a run that exits 0 prints nothing on stderr, and one
|
||||
that exits non-zero prints only its fatal lines. The prompts of `login` are
|
||||
not log lines, and `-q` leaves them as they are.
|
||||
|
||||
`-q` with `-v` is refused with one line,
|
||||
`error: option '-q, --quiet' cannot be used with option '-v, --verbose'`, and
|
||||
@@ -772,7 +792,7 @@ fails, the error is logged, each of its files is written with the reason in an
|
||||
`mlDataError` field instead of `mlData`, and the dump goes on. A file whose ML
|
||||
data does not decrypt, gunzip or parse is logged and written the same way, with
|
||||
why in `mlDataError`. The exit code is non-zero if any file was written with
|
||||
`mlDataError`.
|
||||
`mlDataError`, and the last line then says how many were.
|
||||
|
||||
`helper fix-missing-thumbnails` regenerates thumbnails for JPEG images only,
|
||||
because the bundled decoder (`jpeg-js`) decodes only JPEG. A non-JPEG image
|
||||
|
||||
@@ -36,7 +36,10 @@ declares one.
|
||||
end of each refresh and ML data fetch; trace adds each request, which
|
||||
`ApiClient` reports through the new `onRequest`. `--json` no longer keeps the
|
||||
progress lines off stderr, so `backup` and `--json` runs log a refresh's
|
||||
progress too.
|
||||
progress too. Every line that reports a failed file or album is at error,
|
||||
which the progress callbacks of `backup`, `backup-metadata` and the thumbnail
|
||||
helpers now say, and a command that exits 1 because files or albums failed
|
||||
ends with one fatal line saying how many and where to look.
|
||||
|
||||
- 2026-10-18: `quak files` takes the collection ID as an optional argument
|
||||
instead of the `--collection` option, which is gone
|
||||
|
||||
+15
-5
@@ -101,7 +101,10 @@ import type { MLData } from "./mldata-fetch.js";
|
||||
import type { Collection, EnteFile, FileMetadata } from "./model/types.js";
|
||||
import { hasTransportCode } from "./retry.js";
|
||||
|
||||
export type ProgressCallback = (message: string) => void;
|
||||
// Reports one line of the backup's progress. `level` is "error" when the line
|
||||
// reports something that failed, such as a file or an album, and is left out
|
||||
// for every other line.
|
||||
export type ProgressCallback = (message: string, level?: "error") => void;
|
||||
|
||||
// The lock in a backup directory, as `lockBackupDirectory` returns it.
|
||||
export interface BackupLock {
|
||||
@@ -654,7 +657,7 @@ const runLockedBackup = async (
|
||||
for (const record of lib.undecryptable()) {
|
||||
const error = `could not be decrypted: ${record.error}`;
|
||||
if (record.fileID === undefined) {
|
||||
log(`FAILED album ${record.collectionID}: ${error}`);
|
||||
log(`FAILED album ${record.collectionID}: ${error}`, "error");
|
||||
undecryptableAlbums.push(record.collectionID);
|
||||
} else if (scope.has(record.collectionID)) {
|
||||
const copies = memberships.get(record.fileID);
|
||||
@@ -699,6 +702,7 @@ const runLockedBackup = async (
|
||||
} catch (err) {
|
||||
log(
|
||||
`FAILED verifying original ${file.metadata.title}: ${describeError(err)}`,
|
||||
"error",
|
||||
);
|
||||
recordFailure(
|
||||
file.id,
|
||||
@@ -732,6 +736,7 @@ const runLockedBackup = async (
|
||||
} catch (err) {
|
||||
log(
|
||||
`FAILED original ${file.metadata.title}: ${describeError(err)}`,
|
||||
"error",
|
||||
);
|
||||
recordFailure(
|
||||
file.id,
|
||||
@@ -807,7 +812,7 @@ const runLockedBackup = async (
|
||||
await lib.fetchMLData();
|
||||
} catch (err) {
|
||||
mlDataFailure = err;
|
||||
log(`FAILED ML data: ${describeError(err)}`);
|
||||
log(`FAILED ML data: ${describeError(err)}`, "error");
|
||||
}
|
||||
for (const file of distinct.values()) {
|
||||
if (lock.lost) throw lock.lost;
|
||||
@@ -868,7 +873,10 @@ const runLockedBackup = async (
|
||||
downloadDirectory,
|
||||
);
|
||||
} catch (err) {
|
||||
log(`FAILED removing old album directories: ${describeError(err)}`);
|
||||
log(
|
||||
`FAILED removing old album directories: ${describeError(err)}`,
|
||||
"error",
|
||||
);
|
||||
}
|
||||
|
||||
// An album whose folder or JSON cannot be written, such as one whose
|
||||
@@ -897,6 +905,7 @@ const runLockedBackup = async (
|
||||
} catch (err) {
|
||||
log(
|
||||
`FAILED removing old links in ${c.name}: ${describeError(err)}`,
|
||||
"error",
|
||||
);
|
||||
}
|
||||
|
||||
@@ -915,6 +924,7 @@ const runLockedBackup = async (
|
||||
} catch (err) {
|
||||
log(
|
||||
`FAILED symlink ${c.name}/${linkName}: ${describeError(err)}`,
|
||||
"error",
|
||||
);
|
||||
recordFailure(
|
||||
link.file.id,
|
||||
@@ -931,7 +941,7 @@ const runLockedBackup = async (
|
||||
metaFiles,
|
||||
);
|
||||
} catch (err) {
|
||||
log(`FAILED album ${c.name}: ${describeError(err)}`);
|
||||
log(`FAILED album ${c.name}: ${describeError(err)}`, "error");
|
||||
failedAlbums.push(c.id);
|
||||
}
|
||||
}
|
||||
|
||||
+33
-17
@@ -5,8 +5,9 @@
|
||||
// `process.exit`: `bin/quak.ts` wires these to the command line, and `run` in
|
||||
// `cli-run.ts` prints a thrown error as one line and exits once output has
|
||||
// drained. Output must stay byte-identical (see `cli-output.ts`). Each line on
|
||||
// stderr goes through `ctx.log` at its level (see `cli-log.ts`); a line a
|
||||
// command prints as it fails with a non-zero exit code is fatal.
|
||||
// stderr goes through `ctx.log` at its level (see `cli-log.ts`). Every command
|
||||
// that exits non-zero logs a fatal line saying why; one that exits 1 because
|
||||
// files or albums failed logs it last, saying how many and where to look.
|
||||
|
||||
import {
|
||||
copyFileSync,
|
||||
@@ -414,9 +415,14 @@ export const backupMetadataCommand = async (
|
||||
await lib.fresh();
|
||||
const result = await runMetadataBackup(lib, client, dir, {
|
||||
exif: opts.exif || opts.all,
|
||||
onProgress: (msg) => ctx.log("info", msg),
|
||||
onProgress: (msg, level) => ctx.log(level ?? "info", msg),
|
||||
});
|
||||
return result.filesWithMLDataError > 0 ? 1 : 0;
|
||||
if (result.filesWithMLDataError === 0) return 0;
|
||||
ctx.log(
|
||||
"fatal",
|
||||
`quak: the ML data of ${result.filesWithMLDataError} file(s) could not be read; see mlDataError in their JSON in ${dir}`,
|
||||
);
|
||||
return 1;
|
||||
} finally {
|
||||
await lib.close();
|
||||
}
|
||||
@@ -461,7 +467,7 @@ export const backupCommand = async (
|
||||
downloadDirectory: dir,
|
||||
lock,
|
||||
verify: opts.verify,
|
||||
onProgress: (msg) => ctx.log("info", msg),
|
||||
onProgress: (msg, level) => ctx.log(level ?? "info", msg),
|
||||
});
|
||||
|
||||
if (opts.json) {
|
||||
@@ -488,11 +494,14 @@ export const backupCommand = async (
|
||||
}
|
||||
}
|
||||
|
||||
return result.failed > 0 ||
|
||||
result.undecryptableAlbums.length > 0 ||
|
||||
result.failedAlbums.length > 0
|
||||
? 1
|
||||
: 0;
|
||||
const albums =
|
||||
result.undecryptableAlbums.length + result.failedAlbums.length;
|
||||
if (result.failed === 0 && albums === 0) return 0;
|
||||
ctx.log(
|
||||
"fatal",
|
||||
`quak: ${result.failed} file(s) and ${albums} album(s) failed; see ${join(dir, "failures.json")} and the FAILED lines`,
|
||||
);
|
||||
return 1;
|
||||
} finally {
|
||||
await lib.close();
|
||||
}
|
||||
@@ -513,8 +522,8 @@ export const listMissingThumbnailsCommand = async (
|
||||
// Refresh first so files added since the cache was written are
|
||||
// checked; a failed refresh throws.
|
||||
await lib.fresh();
|
||||
const missing = await listMissingThumbnails(lib, client, (msg) =>
|
||||
ctx.log("info", msg),
|
||||
const missing = await listMissingThumbnails(lib, client, (msg, level) =>
|
||||
ctx.log(level ?? "info", msg),
|
||||
);
|
||||
|
||||
if (opts.json) {
|
||||
@@ -561,8 +570,10 @@ export const fixMissingThumbnailsCommand = async (
|
||||
await lib.fresh();
|
||||
if (fileIDs.length === 0) {
|
||||
ctx.log("info", "Scanning for missing thumbnails...");
|
||||
const missing = await listMissingThumbnails(lib, client, (msg) =>
|
||||
ctx.log("info", msg),
|
||||
const missing = await listMissingThumbnails(
|
||||
lib,
|
||||
client,
|
||||
(msg, level) => ctx.log(level ?? "info", msg),
|
||||
);
|
||||
fileIDs = missing.map((m) => m.fileID);
|
||||
if (fileIDs.length === 0) {
|
||||
@@ -576,9 +587,10 @@ export const fixMissingThumbnailsCommand = async (
|
||||
lib,
|
||||
client,
|
||||
fileIDs,
|
||||
(msg) => ctx.log("info", msg),
|
||||
(msg, level) => ctx.log(level ?? "info", msg),
|
||||
);
|
||||
|
||||
const failed = results.filter((r) => r.status === "failed").length;
|
||||
if (opts.json) {
|
||||
ctx.stdout.write(JSON.stringify(results, null, 2) + "\n");
|
||||
} else {
|
||||
@@ -586,7 +598,6 @@ export const fixMissingThumbnailsCommand = async (
|
||||
const skipped = results.filter(
|
||||
(r) => r.status === "skipped",
|
||||
).length;
|
||||
const failed = results.filter((r) => r.status === "failed").length;
|
||||
ctx.log("info", `\n--- Done ---`);
|
||||
ctx.log("info", ` Fixed: ${fixed}`);
|
||||
ctx.log("info", ` Skipped: ${skipped}`);
|
||||
@@ -605,7 +616,12 @@ export const fixMissingThumbnailsCommand = async (
|
||||
}
|
||||
}
|
||||
|
||||
return results.some((r) => r.status === "failed") ? 1 : 0;
|
||||
if (failed === 0) return 0;
|
||||
ctx.log(
|
||||
"fatal",
|
||||
`quak: ${failed} file(s) failed; see the "Failed files:" list, or the JSON with --json`,
|
||||
);
|
||||
return 1;
|
||||
} finally {
|
||||
await lib.close();
|
||||
}
|
||||
|
||||
+19
-4
@@ -4,9 +4,12 @@
|
||||
// Every line quak prints on stderr is a log line at one of six levels, from
|
||||
// the quietest: fatal, error, warning, info, debug, trace. A run prints the
|
||||
// lines at its own level and at every quieter one. Its level is info unless
|
||||
// `-v` or `-q` asks for another (see "CLI surface" in the README). A fatal line
|
||||
// is the one a command prints when it fails and exits non-zero, so even `-q`
|
||||
// prints it. stdout carries only a command's output and is not a log.
|
||||
// `-v` or `-q` asks for another (see "CLI surface" in the README). Every
|
||||
// command that exits non-zero logs a fatal line saying why, so even `-q`, which
|
||||
// prints only fatal lines, never leaves a non-zero exit unexplained. An error
|
||||
// line reports something that failed, such as a file or an album, and a
|
||||
// warning something that did not fail but needs the user's attention. stdout
|
||||
// carries only a command's output and is not a log.
|
||||
//
|
||||
// The library itself writes no lines: its layers report through callbacks, and
|
||||
// the CLI logs what they report. `logRequests` gives the API client options
|
||||
@@ -66,6 +69,15 @@ export const logRequests = (
|
||||
),
|
||||
});
|
||||
|
||||
// How a log line names each operation the library reports the start and end
|
||||
// of.
|
||||
const OPERATION_NAMES: Record<RefreshEvent["operation"], string> = {
|
||||
refresh: "Refresh",
|
||||
fetchMLData: "ML data fetch",
|
||||
precacheThumbnails: "Thumbnail precache",
|
||||
precacheOriginals: "Original precache",
|
||||
};
|
||||
|
||||
// A library `onProgress` that logs, at info, each collection whose files a
|
||||
// refresh has fetched, and at debug the start and end of each refresh and ML
|
||||
// data fetch, with the error of one that failed.
|
||||
@@ -80,5 +92,8 @@ export const logRefresh =
|
||||
return;
|
||||
}
|
||||
const error = event.error === undefined ? "" : `: ${event.error}`;
|
||||
log("debug", `${event.operation} ${event.status}${error}`);
|
||||
log(
|
||||
"debug",
|
||||
`${OPERATION_NAMES[event.operation]} ${event.status}${error}`,
|
||||
);
|
||||
};
|
||||
|
||||
@@ -17,7 +17,10 @@ import {
|
||||
} from "./mldata-fetch.js";
|
||||
import type { EnteFile } from "./model/types.js";
|
||||
|
||||
export type ProgressCallback = (message: string) => void;
|
||||
// Reports one line of the dump's progress. `level` is "error" when the line
|
||||
// reports ML data that could not be read, and is left out for every other
|
||||
// line.
|
||||
export type ProgressCallback = (message: string, level?: "error") => void;
|
||||
|
||||
export interface MetadataBackupOptions {
|
||||
exif?: boolean;
|
||||
@@ -195,13 +198,17 @@ export const runMetadataBackup = async (
|
||||
for (const [id, payload] of payloads) mlDataMap.set(id, payload);
|
||||
for (const [id, err] of errors) {
|
||||
const reason = describeError(err);
|
||||
log(`ML data of file ${id} could not be read: ${reason}`);
|
||||
log(
|
||||
`ML data of file ${id} could not be read: ${reason}`,
|
||||
"error",
|
||||
);
|
||||
mlDataErrors.set(id, reason);
|
||||
}
|
||||
} catch (err) {
|
||||
const reason = err instanceof Error ? err.message : String(err);
|
||||
log(
|
||||
`ML data request for ${batch.length} file(s) failed: ${reason}`,
|
||||
"error",
|
||||
);
|
||||
for (const id of batch) mlDataErrors.set(id, reason);
|
||||
}
|
||||
|
||||
+6
-1
@@ -49,7 +49,10 @@ export interface ThumbnailFixResult {
|
||||
reason?: string;
|
||||
}
|
||||
|
||||
export type ProgressCallback = (message: string) => void;
|
||||
// Reports one line of progress. `level` is "error" when the line reports a
|
||||
// file that could not be checked or fixed, and is left out for every other
|
||||
// line.
|
||||
export type ProgressCallback = (message: string, level?: "error") => void;
|
||||
|
||||
// Enumerate every file the library knows about, newest album first, each file
|
||||
// once, and report those whose server-side thumbnail is missing. "Missing" is
|
||||
@@ -111,6 +114,7 @@ export const listMissingThumbnails = async (
|
||||
} else {
|
||||
log(
|
||||
`[${album.name}] Could not check ${photo.title}: ${err instanceof Error ? err.message : String(err)} (not reported as missing)`,
|
||||
"error",
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -359,6 +363,7 @@ export const fixMissingThumbnails = async (
|
||||
} catch (err) {
|
||||
log(
|
||||
`[${collectionName}] FAILED ${title}: ${err instanceof Error ? err.message : err}`,
|
||||
"error",
|
||||
);
|
||||
results.push({
|
||||
fileID,
|
||||
|
||||
@@ -371,6 +371,24 @@ describe("quak backup with records it cannot decrypt", () => {
|
||||
expect(stderr.text).toContain(" Failed: 2\n");
|
||||
});
|
||||
|
||||
// The album is an error line. The last line is fatal and counts the files
|
||||
// and the album, so `-q`, which prints fatal lines only, still says why
|
||||
// the backup exits 1.
|
||||
it("logs the album at error, and ends with one fatal line counting the files and the album", async () => {
|
||||
const ctx = {
|
||||
...context(brokenAccount()),
|
||||
log: stderrLog(stderr, "error"),
|
||||
};
|
||||
expect(await backupCommand(ctx, backupDir(), {})).toBe(1);
|
||||
|
||||
expect(stderr.text).toMatch(
|
||||
/^FAILED album 2: could not be decrypted: /,
|
||||
);
|
||||
expect(stderr.text.split("\n").at(-2)).toBe(
|
||||
`quak: 2 file(s) and 1 album(s) failed; see ${join(backupDir(), "failures.json")} and the FAILED lines`,
|
||||
);
|
||||
});
|
||||
|
||||
it("names each album it cannot decrypt in the --json result", async () => {
|
||||
expect(await backup(brokenAccount(), { json: true })).toBe(1);
|
||||
|
||||
|
||||
@@ -913,6 +913,38 @@ describe("backup", () => {
|
||||
);
|
||||
});
|
||||
|
||||
// A failed file is an error line both where the backup fails it and in
|
||||
// the list at the end. The last line is fatal, so `-q`, which prints
|
||||
// fatal lines only, still says why the backup exits 1.
|
||||
it("logs a failed file at error, and ends with one fatal line counting the failures", async () => {
|
||||
const dir = join(root, "backup");
|
||||
const last = `quak: 1 file(s) and 0 album(s) failed; see ${join(dir, "failures.json")} and the FAILED lines\n`;
|
||||
const ctx = context(fakeClient({ failID: 101 }));
|
||||
expect(
|
||||
await backupCommand(
|
||||
{ ...ctx, log: stderrLog(stderr, "error") },
|
||||
dir,
|
||||
{},
|
||||
),
|
||||
).toBe(1);
|
||||
expect(stderr.text).toBe(
|
||||
"FAILED original sunset.jpg: HTTP 500 from server\n" +
|
||||
"\nFailed files:\n" +
|
||||
" [Vacation] sunset.jpg (id 101): HTTP 500 from server\n" +
|
||||
last,
|
||||
);
|
||||
|
||||
stderr.text = "";
|
||||
expect(
|
||||
await backupCommand(
|
||||
{ ...ctx, log: stderrLog(stderr, "fatal") },
|
||||
dir,
|
||||
{ json: true },
|
||||
),
|
||||
).toBe(1);
|
||||
expect(stderr.text).toBe(last);
|
||||
});
|
||||
|
||||
// `--json` shapes stdout only: the log lines stay on stderr, and the JSON
|
||||
// takes the place of the summary.
|
||||
it("prints the result as JSON with --json, still exiting 1 on a failure", async () => {
|
||||
@@ -1303,6 +1335,34 @@ describe("helper list-missing-thumbnails", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("helper fix-missing-thumbnails", () => {
|
||||
// File 999 is in no album, so fixing it fails. The list of failed files
|
||||
// is at error, and the last line is fatal, so `-q`, which prints fatal
|
||||
// lines only, still says why the command exits 1.
|
||||
it("lists a failed file at error, and ends with one fatal line counting the failures", async () => {
|
||||
const last = `quak: 1 file(s) failed; see the "Failed files:" list, or the JSON with --json\n`;
|
||||
const ctx = { ...context(), log: stderrLog(stderr, "error") };
|
||||
expect(await fixMissingThumbnailsCommand(ctx, { file: ["999"] })).toBe(
|
||||
1,
|
||||
);
|
||||
expect(stderr.text).toBe(
|
||||
"\nFailed files:\n" +
|
||||
" 999\tunknown\tfile not found in any collection\n" +
|
||||
last,
|
||||
);
|
||||
|
||||
stderr.text = "";
|
||||
const quiet = { ...context(), log: stderrLog(stderr, "fatal") };
|
||||
expect(
|
||||
await fixMissingThumbnailsCommand(quiet, {
|
||||
file: ["999"],
|
||||
json: true,
|
||||
}),
|
||||
).toBe(1);
|
||||
expect(stderr.text).toBe(last);
|
||||
});
|
||||
});
|
||||
|
||||
describe("backup-metadata --exif", () => {
|
||||
// Runs the command and returns what it printed to stderr.
|
||||
const backupMetadata = async (opts: { exif?: boolean; all?: boolean }) => {
|
||||
|
||||
@@ -173,13 +173,13 @@ const FILES = "100\timage\tbeach.jpg\n";
|
||||
// line for the one album whose files it fetched. Debug adds the start and end
|
||||
// of the refresh, and trace each request with the status of its answer.
|
||||
const INFO = "Fetched the files of 1 of 1 collections\n";
|
||||
const DEBUG = "refresh started\n" + INFO + "refresh done\n";
|
||||
const DEBUG = "Refresh started\n" + INFO + "Refresh done\n";
|
||||
const TRACE =
|
||||
"refresh started\n" +
|
||||
"Refresh started\n" +
|
||||
"GET https://api.ente.io/collections/v2?sinceTime=0: HTTP 200\n" +
|
||||
"GET https://api.ente.io/collections/v2/diff?collectionID=1&sinceTime=0: HTTP 200\n" +
|
||||
INFO +
|
||||
"refresh done\n";
|
||||
"Refresh done\n";
|
||||
|
||||
describe("the log level", () => {
|
||||
it("is info by default", async () => {
|
||||
@@ -326,6 +326,7 @@ describe("what the library reports", () => {
|
||||
]);
|
||||
});
|
||||
|
||||
// Each operation is named in plain words, as the README names it.
|
||||
it("logs a refresh's progress at info, and its start and end at debug", () => {
|
||||
const { log, lines } = collect();
|
||||
const onProgress = logRefresh(log);
|
||||
@@ -342,13 +343,15 @@ describe("what the library reports", () => {
|
||||
status: "failed",
|
||||
error: "HTTP 503",
|
||||
});
|
||||
onProgress({ operation: "fetchMLData", status: "started" });
|
||||
onProgress({ operation: "fetchMLData", status: "done", fetched: 3 });
|
||||
|
||||
expect(lines).toEqual([
|
||||
"debug: refresh started",
|
||||
"debug: Refresh started",
|
||||
"info: Fetched the files of 1 of 2 collections",
|
||||
"debug: refresh failed: HTTP 503",
|
||||
"debug: fetchMLData done",
|
||||
"debug: Refresh failed: HTTP 503",
|
||||
"debug: ML data fetch started",
|
||||
"debug: ML data fetch done",
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -47,7 +47,7 @@ import {
|
||||
type MetadataBackupOptions,
|
||||
} from "../../src/metadata-backup.js";
|
||||
import { backupMetadataCommand } from "../../src/cli-commands.js";
|
||||
import { stderrLog } from "../../src/cli-log.js";
|
||||
import { stderrLog, type LogLevel } from "../../src/cli-log.js";
|
||||
import {
|
||||
buildAccount,
|
||||
buildMetaFetch,
|
||||
@@ -491,9 +491,12 @@ describe("quak backup-metadata", () => {
|
||||
});
|
||||
|
||||
describe("quak backup-metadata when a file's ML data cannot be read", () => {
|
||||
// Run the CLI command against `fetch`, the mock unless given, and return
|
||||
// its exit code, stderr and output directory.
|
||||
const runCommand = async (fetch = buildMetaFetch(mock)) => {
|
||||
// Run the CLI command against `fetch`, the mock unless given, logging at
|
||||
// `level`, and return its exit code, stderr and output directory.
|
||||
const runCommand = async (
|
||||
fetch = buildMetaFetch(mock),
|
||||
level: LogLevel = "info",
|
||||
) => {
|
||||
const client = await Client.login({
|
||||
email: TEST_EMAIL,
|
||||
password: TEST_PASSWORD,
|
||||
@@ -509,7 +512,7 @@ describe("quak backup-metadata when a file's ML data cannot be read", () => {
|
||||
stdout: { write: () => true },
|
||||
log: stderrLog(
|
||||
{ write: (text: string) => (stderr += text) },
|
||||
"info",
|
||||
level,
|
||||
),
|
||||
sessionDir: testDir,
|
||||
cacheDir: mkdtempSync(join(testDir, "cache-")),
|
||||
@@ -521,13 +524,16 @@ describe("quak backup-metadata when a file's ML data cannot be read", () => {
|
||||
return { code, stderr, outDir };
|
||||
};
|
||||
|
||||
// The failed request is logged at error, so it is the first line printed
|
||||
// at that level.
|
||||
it("writes every file, marks the failed batch's files, and exits 1", async () => {
|
||||
const { code, stderr, outDir } = await runCommand(
|
||||
buildMetaFetch(mock, 200),
|
||||
"error",
|
||||
);
|
||||
|
||||
expect(code).toBe(1);
|
||||
expect(stderr).toContain("ML data request for 1 file(s) failed");
|
||||
expect(stderr).toMatch(/^ML data request for 1 file\(s\) failed: /);
|
||||
|
||||
const ok = JSON.parse(
|
||||
readFileSync(
|
||||
@@ -550,7 +556,9 @@ describe("quak backup-metadata when a file's ML data cannot be read", () => {
|
||||
});
|
||||
|
||||
// The server sends ML data for diagram.png encrypted under a key that is
|
||||
// not the file's, so quak cannot decrypt it.
|
||||
// not the file's, so quak cannot decrypt it. That is logged at error, and
|
||||
// the last line is fatal, so `-q`, which prints fatal lines only, still
|
||||
// says why the command exits 1.
|
||||
it("writes the reason for ML data that does not decrypt, and exits 1", async () => {
|
||||
const otherKey = sodium.crypto_secretstream_xchacha20poly1305_keygen();
|
||||
const { header, ciphertext } = encryptBlob(
|
||||
@@ -568,12 +576,14 @@ describe("quak backup-metadata when a file's ML data cannot be read", () => {
|
||||
},
|
||||
},
|
||||
}),
|
||||
"error",
|
||||
);
|
||||
|
||||
expect(code).toBe(1);
|
||||
const reason = "secretstream chunk authentication failed";
|
||||
expect(stderr).toContain(
|
||||
`ML data of file 200 could not be read: ${reason}\n`,
|
||||
expect(stderr).toBe(
|
||||
`ML data of file 200 could not be read: ${reason}\n` +
|
||||
`quak: the ML data of 1 file(s) could not be read; see mlDataError in their JSON in ${outDir}\n`,
|
||||
);
|
||||
const failed = JSON.parse(
|
||||
readFileSync(
|
||||
|
||||
@@ -544,11 +544,22 @@ describe("listMissingThumbnails", () => {
|
||||
const client = await login(counted.fetch, { ...noWait });
|
||||
const lib = await openLib(client);
|
||||
|
||||
const missing = await listMissingThumbnails(lib, client);
|
||||
const errors: string[] = [];
|
||||
const missing = await listMissingThumbnails(
|
||||
lib,
|
||||
client,
|
||||
(msg, level) => {
|
||||
if (level === "error") errors.push(msg);
|
||||
},
|
||||
);
|
||||
lib.close();
|
||||
|
||||
// Only the genuinely empty thumbnail is reported.
|
||||
// Only the genuinely empty thumbnail is reported. The file that
|
||||
// could not be checked is reported as an error.
|
||||
expect(missing.map((m) => m.fileID)).toEqual([101]);
|
||||
expect(errors).toEqual([
|
||||
expect.stringMatching(/^\[.*\] Could not check file-102\.jpg: /),
|
||||
]);
|
||||
// And the 500 was retried rather than accepted as an answer: four
|
||||
// attempts is the library default.
|
||||
expect(counted.matched()).toBe(4);
|
||||
@@ -699,7 +710,15 @@ describe("fixMissingThumbnails", () => {
|
||||
const client = await login(buildThumbFetch(fixMock));
|
||||
const lib = await openLib(client);
|
||||
|
||||
const results = await fixMissingThumbnails(lib, client, [101, 102]);
|
||||
const errors: string[] = [];
|
||||
const results = await fixMissingThumbnails(
|
||||
lib,
|
||||
client,
|
||||
[101, 102],
|
||||
(msg, level) => {
|
||||
if (level === "error") errors.push(msg);
|
||||
},
|
||||
);
|
||||
lib.close();
|
||||
|
||||
expect(results.length).toBe(2);
|
||||
@@ -707,6 +726,10 @@ describe("fixMissingThumbnails", () => {
|
||||
const failure = results.find((r) => r.fileID === 102)!;
|
||||
expect(success.status).toBe("fixed");
|
||||
expect(failure.status).toBe("failed");
|
||||
// The failed file, and only it, is reported as an error.
|
||||
expect(errors).toEqual([
|
||||
expect.stringMatching(/^\[.*\] FAILED file-102\.jpg: /),
|
||||
]);
|
||||
});
|
||||
|
||||
it("skips a non-JPEG image and a video as unsupported, not failed (issue #17)", async () => {
|
||||
|
||||
Reference in New Issue
Block a user