Compare commits

..
1 Commits
Author SHA1 Message Date
clawbot 80f691413a Make lint and test phases of the Dockerfile (closes #96)
check / check (push) Successful in 1m40s
Follows the template: Dockerfile.lint is gone; the Dockerfile has a lint
phase (eslint, prettier --check .) and a test phase (vitest, run as the
node user, which the not-writable-directory tests need), and its
last stage compiles and depends on both. script/lint and script/test
build one phase each with --no-cache; script/docker and script/cibuild
pass --no-cache, so CHECK_EPOCH and LINT_EPOCH are removed.
script/cibuild is the single image build, so CI runs lint and the tests
once each. The tests that checked the old layout are deleted,
REPO_POLICIES.md is re-copied and the README describes the new layout.

Model: opus-5-5
2026-09-23 04:50:28 +00:00
10 changed files with 57 additions and 448 deletions
+3 -20
View File
@@ -296,7 +296,6 @@ Endpoints used:
encrypted token plus key attributes. encrypted token plus key attributes.
- `POST /users/ott` and `POST /users/verify-email`: email OTP fallback path. - `POST /users/ott` and `POST /users/verify-email`: email OTP fallback path.
- `POST /users/two-factor/verify`: TOTP second factor. - `POST /users/two-factor/verify`: TOTP second factor.
- `POST /users/logout`: end the calling token's session (`quak logout`).
- `GET /collections/v2?sinceTime=<usec>`: list collections changed since - `GET /collections/v2?sinceTime=<usec>`: list collections changed since
microsecond timestamp; pass 0 for a full enumeration. microsecond timestamp; pass 0 for a full enumeration.
- `GET /collections/v2/diff?collectionID=<id>&sinceTime=<usec>`: list files in a - `GET /collections/v2/diff?collectionID=<id>&sinceTime=<usec>`: list files in a
@@ -411,9 +410,7 @@ whatever else fits their use case. `Client.fromJSON(snapshot)` restores a
working client from that snapshot without re-authenticating; it checks every working client from that snapshot without re-authenticating; it checks every
field and each key's length first, and throws an error naming the bad field. field and each key's length first, and throws an error naming the bad field.
`client.logout()` clears the token and zeroes the key buffers in place; every `client.logout()` clears the token and zeroes the key buffers in place; every
later call on that client throws. It does not contact the server, so the token later call on that client throws.
stays valid there and in any saved snapshot; `await client.logoutOnServer()`
first ends the session on the server (`POST /users/logout`).
The CLI stores the snapshot at the platform-appropriate data directory via The CLI stores the snapshot at the platform-appropriate data directory via
`env-paths`: `~/Library/Application Support/quak/session.json` on macOS, `env-paths`: `~/Library/Application Support/quak/session.json` on macOS,
@@ -423,21 +420,13 @@ you would treat the password itself. A missing file is reported as "not logged
in"; a file that exists but is corrupt is reported as such, naming the bad in"; a file that exists but is corrupt is reported as such, naming the bad
field. Both exit with status 1. field. Both exit with status 1.
`quak logout` ends the session on the server, so the token in `session.json`
stops working even in a copy of the file, and then deletes the file. If the
server call fails (or the file is corrupt), the file is still deleted, the
command says the server session could not be ended, and it exits with status 1.
It does not delete the cache: it prints the account's cache directory and says
it still holds decrypted data (file keys in `metadata.json`, cached originals
and thumbnails), for the user to delete if they want it gone.
### CLI surface ### CLI surface
``` ```
quak [--cache-dir <path>] <command> global: local metadata/content cache location quak [--cache-dir <path>] <command> global: local metadata/content cache location
quak login interactive or QUAK_EMAIL/QUAK_PASSWORD quak login interactive or QUAK_EMAIL/QUAK_PASSWORD
quak whoami print logged-in account as JSON quak whoami print logged-in account as JSON
quak logout end the session, delete it quak logout delete saved session
quak collections [--json] list all collections quak collections [--json] list all collections
quak files --collection <id> [--json] list files in a collection quak files --collection <id> [--json] list files in a collection
quak get <fileID> [--out path] [--collection] download and decrypt a file quak get <fileID> [--out path] [--collection] download and decrypt a file
@@ -469,13 +458,7 @@ on. The exit code is non-zero if any ML data request failed.
only, because the bundled decoder (`jpeg-js`) decodes only JPEG. A non-JPEG only, because the bundled decoder (`jpeg-js`) decodes only JPEG. A non-JPEG
image (PNG, HEIC) or a video is reported as `skipped` (unsupported format), kept image (PNG, HEIC) or a video is reported as `skipped` (unsupported format), kept
distinct from a `failed` repair, and does not affect the exit code; a genuine distinct from a `failed` repair, and does not affect the exit code; a genuine
failure still exits non-zero. The server accepts a new thumbnail only from the failure still exits non-zero.
file's owner and only when it is no larger than the thumbnail size it records
for the file. So a file another account owns, in an album shared with you, is
skipped by both thumbnail helpers without being fetched, and the fixer skips a
file whose recorded thumbnail size is 0 or unknown. Otherwise the fixer lowers
the quality and size of the thumbnail until it fits, and skips the file if even
the smallest does not.
### Backup layout ### Backup layout
-23
View File
@@ -24,26 +24,6 @@ Tag v1.0.0.
stage compiles and depends on both, so `script/cibuild` is one build. stage compiles and depends on both, so `script/cibuild` is one build.
`Dockerfile.lint`, `CHECK_EPOCH`, `LINT_EPOCH` and the tests that checked them `Dockerfile.lint`, `CHECK_EPOCH`, `LINT_EPOCH` and the tests that checked them
are gone; `REPO_POLICIES.md` is re-copied. are gone; `REPO_POLICIES.md` is re-copied.
- 2026-09-23: Stopped `helper fix-missing-thumbnails` retrying files the server
always refuses (issue 109). Both thumbnail helpers skip a file another account
owns without fetching it. The fixer skips a file whose recorded thumbnail size
is 0 or unknown before downloading it, and otherwise tries smaller encodings
(720 px quality 50 down to 160 px quality 20) until the encrypted thumbnail is
no larger than that size, skipping the file if none fits.
- 2026-09-23: Tested the live-photo hash check's error paths (issue 117). Tests
download a live photo whose ZIP names an unknown compression method, one whose
ZIP has no image entry and one with no video entry, and check that nothing is
stored and the error names the file ID; the unreadable one is not retried.
- 2026-09-23: `quak logout` ends the session on the server (issue 108). It calls
`POST /users/logout` through the new `Client.logoutOnServer()`, then deletes
`session.json` even when that call fails, says so and exits 1. It prints the
account's cache directory and says it still holds decrypted data. The default
cache path is now `defaultCacheDirectory()` in the library, shared with
`Library.open`.
- 2026-09-23: Fixed the backup's per-collection folders (issue 103). Two files - 2026-09-23: Fixed the backup's per-collection folders (issue 103). Two files
in one collection with the same title, and two collections with the same name, in one collection with the same title, and two collections with the same name,
each get their ID added to the name (`IMG_0001 (12345).JPG`, `Trip (10)/`), so each get their ID added to the name (`IMG_0001 (12345).JPG`, `Trip (10)/`), so
@@ -51,7 +31,6 @@ Tag v1.0.0.
`originals/` for files no longer in the collection, and the folders of deleted `originals/` for files no longer in the collection, and the folders of deleted
or renamed collections, leaving anything else in `collections/` alone. The or renamed collections, leaving anything else in `collections/` alone. The
README backup layout states the naming rule. README backup layout states the naming rule.
- 2026-09-23: Checked downloaded originals against their recorded content hash - 2026-09-23: Checked downloaded originals against their recorded content hash
(issue 68). `downloadFile`, which `quak get`, the content cache and backup all (issue 68). `downloadFile`, which `quak get`, the content cache and backup all
use, hashes the decrypted bytes (unkeyed BLAKE2b-512, standard base64) and use, hashes the decrypted bytes (unkeyed BLAKE2b-512, standard base64) and
@@ -60,14 +39,12 @@ Tag v1.0.0.
hashed separately as `<imageHash>:<videoHash>`. `decryptFile` reads older hashed separately as `<imageHash>:<videoHash>`. `decryptFile` reads older
clients' `imageHash` and `videoHash` fields for live photos. A file with no clients' `imageHash` and `videoHash` fields for live photos. A file with no
recorded hash is stored unchecked. recorded hash is stored unchecked.
- 2026-09-23: Kept one account's cache from mixing with another's (issue 104). - 2026-09-23: Kept one account's cache from mixing with another's (issue 104).
When `metadata.json` in the cache directory was written for a different, When `metadata.json` in the cache directory was written for a different,
non-zero user ID than the client's, `Library.open` deletes it and `mldata/` non-zero user ID than the client's, `Library.open` deletes it and `mldata/`
and starts empty, so the first refresh enumerates from 0. This only happens and starts empty, so the first refresh enumerates from 0. This only happens
with `--cache-dir` or an explicit `cacheDirectory`; the default path already with `--cache-dir` or an explicit `cacheDirectory`; the default path already
includes the user ID. A test opens one account's cache as another account. includes the user ID. A test opens one account's cache as another account.
- 2026-09-23: `backup-metadata` no longer stops on one failed ML data request - 2026-09-23: `backup-metadata` no longer stops on one failed ML data request
(issue 101). Each request of up to 200 files is tried on its own; a failed one (issue 101). Each request of up to 200 files is tried on its own; a failed one
is logged, its files are written with the reason in `mlDataError`, and the is logged, its files are written with the reason in `mlDataError`, and the
+1 -1
View File
@@ -73,7 +73,7 @@ program
program program
.command("logout") .command("logout")
.description("End the session on the server and delete the saved session") .description("Delete the saved session")
.action(() => run(logoutCommand(context()))); .action(() => run(logoutCommand(context())));
program program
+7 -40
View File
@@ -17,11 +17,7 @@ import {
import { join } from "node:path"; import { join } from "node:path";
import { Client, type ClientSnapshot } from "./client.js"; import { Client, type ClientSnapshot } from "./client.js";
import { init } from "./crypto/index.js"; import { init } from "./crypto/index.js";
import { import { Library, type LibraryClient } from "./library/index.js";
defaultCacheDirectory,
Library,
type LibraryClient,
} from "./library/index.js";
import { import {
fileListRow, fileListRow,
fileListLine, fileListLine,
@@ -150,43 +146,14 @@ export const whoamiCommand = async (ctx: CliContext): Promise<number> => {
return 0; return 0;
}; };
// Ends the session on the server, then deletes the session file even when that
// failed, and exits 1 if it did. The cache is left in place; the user is told
// where it is.
export const logoutCommand = async (ctx: CliContext): Promise<number> => { export const logoutCommand = async (ctx: CliContext): Promise<number> => {
const path = sessionPath(ctx); if (existsSync(sessionPath(ctx))) {
if (!existsSync(path)) { unlinkSync(sessionPath(ctx));
ctx.stderr.write("No session found.\n");
return 0;
}
await init();
let cacheDir = ctx.cacheDir;
let failure: string | undefined;
try {
const client = ctx.loadSession(path);
if (client) {
cacheDir ??= defaultCacheDirectory(client.whoami().userID);
await client.logoutOnServer();
client.logout();
}
} catch (err) {
failure = err instanceof Error ? err.message : String(err);
}
unlinkSync(path);
if (failure === undefined) {
ctx.stderr.write("Session ended on the server.\n");
} else {
ctx.stderr.write(
`Could not end the session on the server: ${failure}\n`,
);
}
ctx.stderr.write("Session deleted.\n"); ctx.stderr.write("Session deleted.\n");
if (cacheDir !== undefined) { } else {
ctx.stderr.write( ctx.stderr.write("No session found.\n");
`Cache directory ${cacheDir} still holds decrypted data; delete it to remove that data.\n`,
);
} }
return failure === undefined ? 0 : 1; return 0;
}; };
export const collectionsCommand = async ( export const collectionsCommand = async (
@@ -495,7 +462,7 @@ export const fixMissingThumbnailsCommand = async (
ctx.stderr.write(` Skipped: ${skipped}\n`); ctx.stderr.write(` Skipped: ${skipped}\n`);
ctx.stderr.write(` Failed: ${failed}\n`); ctx.stderr.write(` Failed: ${failed}\n`);
if (skipped > 0) { if (skipped > 0) {
ctx.stderr.write("\nSkipped:\n"); ctx.stderr.write("\nSkipped (unsupported format):\n");
for (const r of results.filter((r) => r.status === "skipped")) { for (const r of results.filter((r) => r.status === "skipped")) {
ctx.stderr.write( ctx.stderr.write(
` ${r.fileID}\t${r.title}\t${r.reason}\n`, ` ${r.fileID}\t${r.title}\t${r.reason}\n`,
-8
View File
@@ -217,14 +217,6 @@ export class Client {
}; };
} }
// Ends this client's session on the server (`POST /users/logout`), so the
// token stops working everywhere, including in any saved copy of it. This
// client is left as it was; call `logout()` to clear it.
async logoutOnServer(): Promise<void> {
this.assertLoggedIn();
await this.api.postJSON("/users/logout", {});
}
// Zeroes the key buffers in place, so any copy of the reference held // Zeroes the key buffers in place, so any copy of the reference held
// elsewhere is wiped too. Every method checks `assertLoggedIn` before // elsewhere is wiped too. Every method checks `assertLoggedIn` before
// touching the keys, so nothing decrypts with the zeroed keys. // touching the keys, so nothing decrypts with the zeroed keys.
+2 -6
View File
@@ -98,11 +98,6 @@ export {
export const DEFAULT_REFRESH_INTERVAL_SECONDS = 3; export const DEFAULT_REFRESH_INTERVAL_SECONDS = 3;
// The account's cache directory when `cacheDirectory` is not given: the
// env-paths cache directory plus the user id, so each account has its own.
export const defaultCacheDirectory = (userID: number): string =>
join(envPaths("quak", { suffix: "" }).cache, String(userID));
// Project a metadata store into by-id records, filling each record's cache // Project a metadata store into by-id records, filling each record's cache
// paths from the content cache when one is given. Shared by the live read // paths from the content cache when one is given. Shared by the live read
// projection and the precache's initial seeding at open(). // projection and the precache's initial seeding at open().
@@ -349,7 +344,8 @@ export class Library {
static async open(opts: LibraryOptions): Promise<Library> { static async open(opts: LibraryOptions): Promise<Library> {
const { userID } = opts.client.whoami(); const { userID } = opts.client.whoami();
const cacheDirectory = const cacheDirectory =
opts.cacheDirectory ?? defaultCacheDirectory(userID); opts.cacheDirectory ??
join(envPaths("quak", { suffix: "" }).cache, String(userID));
const metadataPath = join(cacheDirectory, "metadata.json"); const metadataPath = join(cacheDirectory, "metadata.json");
let store = await MetadataStore.load(metadataPath); let store = await MetadataStore.load(metadataPath);
// A cache directory given explicitly can hold another account's cache. // A cache directory given explicitly can hold another account's cache.
+30 -88
View File
@@ -7,21 +7,8 @@ import { ApiError } from "./api/client.js";
import { encryptBlob, toBase64 } from "./crypto/index.js"; import { encryptBlob, toBase64 } from "./crypto/index.js";
import type { EnteFile } from "./model/types.js"; import type { EnteFile } from "./model/types.js";
// The server refuses a thumbnail larger than the one it already records for the const THUMB_MAX_DIMENSION = 720;
// file (`thumbnail.size`, the encrypted size), so these encodings are tried const THUMB_JPEG_QUALITY = 50;
// from largest to smallest and the first that fits is uploaded.
const THUMB_ENCODINGS = [
{ maxDimension: 720, quality: 50 },
{ maxDimension: 720, quality: 30 },
{ maxDimension: 480, quality: 30 },
{ maxDimension: 320, quality: 20 },
{ maxDimension: 160, quality: 20 },
];
// The server accepts a new thumbnail only from the file's owner, so files other
// people own in albums shared with this account are never checked or repaired.
const NOT_OWNED_REASON =
"owned by another account (only the owner can replace its thumbnail)";
export interface MissingThumbnailInfo { export interface MissingThumbnailInfo {
fileID: number; fileID: number;
@@ -32,12 +19,11 @@ export interface MissingThumbnailInfo {
// Three outcomes, not two. "fixed": a thumbnail was generated and uploaded. // Three outcomes, not two. "fixed": a thumbnail was generated and uploaded.
// "failed": something went wrong (download, encode, upload) and the file still // "failed": something went wrong (download, encode, upload) and the file still
// has no thumbnail. "skipped": the server would refuse any thumbnail for the // has no thumbnail. "skipped": the file is a format this helper cannot
// file or this helper cannot regenerate it — a file another account owns, a // regenerate — a video, or an image that is not a baseline JPEG. Skipped is a
// recorded thumbnail size nothing fits within, a video, or an image that is // deliberate, expected outcome, not an error (issue #17): the repair path is
// not a baseline JPEG. Skipped is a deliberate, expected outcome, not an error // JPEG-only because `jpeg-js` is, and a PNG or HEIC is left for a format-aware
// (issue #17): the repair path is JPEG-only because `jpeg-js` is, and a PNG or // tool rather than reported as a failure.
// HEIC is left for a format-aware tool rather than reported as a failure.
export type ThumbnailFixStatus = "fixed" | "skipped" | "failed"; export type ThumbnailFixStatus = "fixed" | "skipped" | "failed";
export interface ThumbnailFixResult { export interface ThumbnailFixResult {
@@ -59,7 +45,6 @@ export type ProgressCallback = (message: string) => void;
// exists, so it is logged and the file is left unreported. That distinction is // exists, so it is logged and the file is left unreported. That distinction is
// what stops `fix-missing-thumbnails` from regenerating and uploading over // what stops `fix-missing-thumbnails` from regenerating and uploading over
// thumbnails that were fine all along while the CDN was briefly returning 500s. // thumbnails that were fine all along while the CDN was briefly returning 500s.
// Files another account owns are logged as skipped and not checked.
export const listMissingThumbnails = async ( export const listMissingThumbnails = async (
lib: Library, lib: Library,
client: Client, client: Client,
@@ -67,7 +52,6 @@ export const listMissingThumbnails = async (
): Promise<MissingThumbnailInfo[]> => { ): Promise<MissingThumbnailInfo[]> => {
const log = onProgress ?? (() => {}); const log = onProgress ?? (() => {});
const api = client.getApiClient(); const api = client.getApiClient();
const { userID } = client.whoami();
const missing: MissingThumbnailInfo[] = []; const missing: MissingThumbnailInfo[] = [];
const seen = new Set<number>(); const seen = new Set<number>();
@@ -76,13 +60,6 @@ export const listMissingThumbnails = async (
for (const photo of album.photos.list()) { for (const photo of album.photos.list()) {
if (seen.has(photo.fileID)) continue; if (seen.has(photo.fileID)) continue;
seen.add(photo.fileID); seen.add(photo.fileID);
const file = lib.getFile(album.collectionID, photo.fileID);
if (file && file.ownerID !== userID) {
log(
`[${album.name}] Skipping ${photo.title}: ${NOT_OWNED_REASON}`,
);
continue;
}
try { try {
const stream = await api.getThumbnailStream(photo.fileID); const stream = await api.getThumbnailStream(photo.fileID);
const reader = stream.getReader(); const reader = stream.getReader();
@@ -158,13 +135,17 @@ const resizeRGBA = (
return dst; return dst;
}; };
const generateThumbnail = ( const generateThumbnail = (fileBytes: Uint8Array): Uint8Array => {
decoded: { data: Uint8Array; width: number; height: number }, const decoded = jpeg.decode(fileBytes, {
maxDimension: number, useTArray: true,
quality: number, formatAsRGBA: true,
): Uint8Array => { });
const { width: srcW, height: srcH } = decoded; const { width: srcW, height: srcH } = decoded;
const scale = Math.min(maxDimension / srcW, maxDimension / srcH, 1); const scale = Math.min(
THUMB_MAX_DIMENSION / srcW,
THUMB_MAX_DIMENSION / srcH,
1,
);
const dstW = Math.round(srcW * scale); const dstW = Math.round(srcW * scale);
const dstH = Math.round(srcH * scale); const dstH = Math.round(srcH * scale);
@@ -177,7 +158,7 @@ const generateThumbnail = (
const encoded = jpeg.encode( const encoded = jpeg.encode(
{ data: pixels, width: dstW, height: dstH }, { data: pixels, width: dstW, height: dstH },
quality, THUMB_JPEG_QUALITY,
); );
return new Uint8Array(encoded.data); return new Uint8Array(encoded.data);
}; };
@@ -190,35 +171,14 @@ const generateThumbnail = (
const isJpeg = (bytes: Uint8Array): boolean => const isJpeg = (bytes: Uint8Array): boolean =>
bytes.length >= 2 && bytes[0] === 0xff && bytes[1] === 0xd8; bytes.length >= 2 && bytes[0] === 0xff && bytes[1] === 0xd8;
// The reason a file cannot have a JPEG thumbnail regenerated for it, known from // The reason a file cannot have a JPEG thumbnail regenerated for it from its
// its record alone before any bytes are fetched, or undefined when it might. A // metadata alone, before any bytes are fetched, or undefined when it might. A
// still image still has to be checked against its actual bytes once // non-image (video, live photo) is unsupported outright; a still image still
// downloaded. // has to be checked against its actual bytes once downloaded.
const reasonToSkip = (file: EnteFile, userID: number): string | undefined => { const unsupportedByType = (file: EnteFile): string | undefined => {
if (file.ownerID !== userID) {
return NOT_OWNED_REASON;
}
if (file.metadata.fileType !== "image") { if (file.metadata.fileType !== "image") {
return `unsupported file type: ${file.metadata.fileType} (only JPEG images can be regenerated)`; return `unsupported file type: ${file.metadata.fileType} (only JPEG images can be regenerated)`;
} }
if (!file.thumbnail.size) {
return `recorded thumbnail size is ${file.thumbnail.size ?? "unknown"} (the server refuses a thumbnail larger than the one it records)`;
}
return undefined;
};
// Encrypt the largest encoding of the decoded image whose ciphertext is no
// larger than `maxSize`, or return undefined when even the smallest is larger.
const encryptThumbnailWithin = (
decoded: { data: Uint8Array; width: number; height: number },
key: Uint8Array,
maxSize: number,
): { header: Uint8Array; ciphertext: Uint8Array } | undefined => {
for (const { maxDimension, quality } of THUMB_ENCODINGS) {
const thumbJpeg = generateThumbnail(decoded, maxDimension, quality);
const encrypted = encryptBlob(thumbJpeg, key);
if (encrypted.ciphertext.length <= maxSize) return encrypted;
}
return undefined; return undefined;
}; };
@@ -237,7 +197,6 @@ export const fixMissingThumbnails = async (
const log = onProgress ?? (() => {}); const log = onProgress ?? (() => {});
const results: ThumbnailFixResult[] = []; const results: ThumbnailFixResult[] = [];
const api = client.getApiClient(); const api = client.getApiClient();
const { userID } = client.whoami();
// Resolve each requested fileID to its file record and owning album by // Resolve each requested fileID to its file record and owning album by
// enumerating the library, each file taken from the first album that holds // enumerating the library, each file taken from the first album that holds
@@ -278,19 +237,18 @@ export const fixMissingThumbnails = async (
const { file, collectionName } = entry; const { file, collectionName } = entry;
const title = file.metadata.title; const title = file.metadata.title;
const skipReason = reasonToSkip(file, userID); const typeReason = unsupportedByType(file);
if (skipReason) { if (typeReason) {
log(`[${collectionName}] Skipping ${title}: ${skipReason}`); log(`[${collectionName}] Skipping ${title}: ${typeReason}`);
results.push({ results.push({
fileID, fileID,
title, title,
collection: collectionName, collection: collectionName,
status: "skipped", status: "skipped",
reason: skipReason, reason: typeReason,
}); });
continue; continue;
} }
const maxSize = file.thumbnail.size!;
try { try {
const photo = lib.photos.byID({ fileID }); const photo = lib.photos.byID({ fileID });
@@ -319,28 +277,12 @@ export const fixMissingThumbnails = async (
} }
log(`[${collectionName}] Generating thumbnail for ${title}...`); log(`[${collectionName}] Generating thumbnail for ${title}...`);
const decoded = jpeg.decode(fileBytes, { const thumbJpeg = generateThumbnail(fileBytes);
useTArray: true,
formatAsRGBA: true,
});
const fitting = encryptThumbnailWithin(decoded, file.key, maxSize);
if (!fitting) {
const reason = `no thumbnail encoding fits the recorded thumbnail size of ${maxSize} bytes`;
log(`[${collectionName}] Skipping ${title}: ${reason}`);
results.push({
fileID,
title,
collection: collectionName,
status: "skipped",
reason,
});
continue;
}
const { header, ciphertext } = fitting;
log( log(
`[${collectionName}] Uploading thumbnail (${ciphertext.length} bytes)...`, `[${collectionName}] Encrypting and uploading thumbnail (${thumbJpeg.length} bytes)...`,
); );
const { header, ciphertext } = encryptBlob(thumbJpeg, file.key);
const md5 = createHash("md5").update(ciphertext).digest("base64"); const md5 = createHash("md5").update(ciphertext).digest("base64");
const { objectKey, url } = await api.getUploadURL( const { objectKey, url } = await api.getUploadURL(
ciphertext.length, ciphertext.length,
+14 -81
View File
@@ -38,8 +38,7 @@ import { loadSession } from "../../src/cli-session.js";
import type { Client, ClientSnapshot } from "../../src/client.js"; import type { Client, ClientSnapshot } from "../../src/client.js";
import type { ContentSource } from "../../src/library/content.js"; import type { ContentSource } from "../../src/library/content.js";
import type { Collection, EnteFile } from "../../src/model/types.js"; import type { Collection, EnteFile } from "../../src/model/types.js";
import { init, toBase64 } from "../../src/crypto/index.js"; import { init } from "../../src/crypto/index.js";
import { defaultCacheDirectory } from "../../src/library/index.js";
const USER_ID = 42; const USER_ID = 42;
@@ -177,6 +176,19 @@ describe("session file", () => {
expect(JSON.parse(readFileSync(path, "utf-8"))).toEqual(snapshot); expect(JSON.parse(readFileSync(path, "utf-8"))).toEqual(snapshot);
}); });
it("is removed by logout", async () => {
const ctx = context();
saveSession(ctx.sessionDir, snapshot);
expect(await logoutCommand(ctx)).toBe(0);
expect(existsSync(join(ctx.sessionDir, "session.json"))).toBe(false);
expect(stderr.text).toBe("Session deleted.\n");
});
it("logout without a session says so and exits 0", async () => {
expect(await logoutCommand(context())).toBe(0);
expect(stderr.text).toBe("No session found.\n");
});
it("a missing session exits 1 with 'Not logged in'", async () => { it("a missing session exits 1 with 'Not logged in'", async () => {
const ctx = { ...context(), loadSession }; const ctx = { ...context(), loadSession };
expect(await whoamiCommand(ctx)).toBe(1); expect(await whoamiCommand(ctx)).toBe(1);
@@ -199,85 +211,6 @@ describe("session file", () => {
}); });
}); });
// These use a real client read from the session file, over a fake API that
// records each request and answers with `status`.
describe("logout", () => {
const snapshot: ClientSnapshot = {
email: "cli@example.com",
userID: USER_ID,
token: "saved-token",
masterKey: toBase64(new Uint8Array(32)),
secretKey: toBase64(new Uint8Array(32)),
publicKey: toBase64(new Uint8Array(32)),
};
const requests: Request[] = [];
const logoutContext = (status: number): CliContext => ({
...context(),
loadSession: (path) =>
loadSession(path, {
fetch: async (url, init) => {
requests.push(new Request(url, init));
return new Response(JSON.stringify({}), {
status,
headers: { "content-type": "application/json" },
});
},
}),
});
beforeEach(() => {
requests.length = 0;
});
it("ends the session on the server, then deletes the file", async () => {
const ctx = logoutContext(200);
saveSession(ctx.sessionDir, snapshot);
expect(await logoutCommand(ctx)).toBe(0);
expect(requests).toHaveLength(1);
expect(requests[0]!.method).toBe("POST");
expect(new URL(requests[0]!.url).pathname).toBe("/users/logout");
expect(requests[0]!.headers.get("X-Auth-Token")).toBe("saved-token");
expect(existsSync(join(ctx.sessionDir, "session.json"))).toBe(false);
expect(stderr.text).toBe(
"Session ended on the server.\n" +
"Session deleted.\n" +
`Cache directory ${ctx.cacheDir} still holds decrypted data; delete it to remove that data.\n`,
);
});
it("still deletes the file when the server call fails, and says so", async () => {
const ctx = logoutContext(500);
saveSession(ctx.sessionDir, snapshot);
expect(await logoutCommand(ctx)).toBe(1);
expect(requests).toHaveLength(1);
expect(existsSync(join(ctx.sessionDir, "session.json"))).toBe(false);
expect(stderr.text).toBe(
"Could not end the session on the server: HTTP 500\n" +
"Session deleted.\n" +
`Cache directory ${ctx.cacheDir} still holds decrypted data; delete it to remove that data.\n`,
);
});
it("names the account's default cache directory without --cache-dir", async () => {
const ctx = { ...logoutContext(200), cacheDir: undefined };
saveSession(ctx.sessionDir, snapshot);
expect(await logoutCommand(ctx)).toBe(0);
expect(stderr.text).toContain(
`Cache directory ${defaultCacheDirectory(USER_ID)} still holds decrypted data`,
);
});
it("without a session says so, calls nothing and exits 0", async () => {
expect(await logoutCommand(logoutContext(200))).toBe(0);
expect(requests).toHaveLength(0);
expect(stderr.text).toBe("No session found.\n");
});
});
describe("whoami", () => { describe("whoami", () => {
it("prints the account as one line of JSON", async () => { it("prints the account as one line of JSON", async () => {
expect(await whoamiCommand(context())).toBe(0); expect(await whoamiCommand(context())).toBe(0);
-38
View File
@@ -1744,42 +1744,4 @@ describe("downloadFile content hash", () => {
expect(readdirSync(t.dir)).toEqual([]); expect(readdirSync(t.dir)).toEqual([]);
}); });
it("rejects a live photo that is not a readable ZIP and does not retry", async () => {
// Bytes 8-9 of a ZIP entry's local header name its compression
// method; 99 is one no reader knows, so the entry cannot be read.
const zip = livePhotoZip.slice();
zip[8] = 99;
zip[9] = 0;
const t = setup(zip, { fileType: "livePhoto", hash: livePhotoHash });
await expect(t.run()).rejects.toThrow(
/file 999: live photo is not a readable ZIP/,
);
expect(readdirSync(t.dir)).toEqual([]);
expect(t.requests()).toBe(1);
});
it("rejects a live photo ZIP with no image entry", async () => {
const zip = zipSync({ "video.mov": patternBytes(900, 82) });
const t = setup(zip, { fileType: "livePhoto", hash: livePhotoHash });
await expect(t.run()).rejects.toThrow(
/file 999: live photo ZIP does not hold both an image and a video/,
);
expect(readdirSync(t.dir)).toEqual([]);
});
it("rejects a live photo ZIP with no video entry", async () => {
const zip = zipSync({ "image.heic": patternBytes(500, 81) });
const t = setup(zip, { fileType: "livePhoto", hash: livePhotoHash });
await expect(t.run()).rejects.toThrow(
/file 999: live photo ZIP does not hold both an image and a video/,
);
expect(readdirSync(t.dir)).toEqual([]);
});
}); });
-143
View File
@@ -215,9 +215,6 @@ const buildThumbMock = async (opts?: {
thumbnail: { thumbnail: {
decryptionHeader: toBase64(sodium.randombytes_buf(24)), decryptionHeader: toBase64(sodium.randombytes_buf(24)),
}, },
// The encrypted size of the thumbnail the server records; large
// enough here that the default encoding fits.
info: { thumbSize: 1_000_000 },
updationTime: TEST_TIME, updationTime: TEST_TIME,
}; };
}; };
@@ -449,32 +446,6 @@ const openLib = (client: Client): Promise<Library> =>
precacheOriginals: false, precacheOriginals: false,
}); });
/** The mock's raw record for one file, for a test to change before login. */
const rawFile = (m: ThumbMockState, fileID: number): Record<string, unknown> =>
m.filesByCollection[1]!.find((f) => f.id === fileID)!;
/** Replace the original the mock serves for one file. */
const replaceOriginal = (
m: ThumbMockState,
fileID: number,
body: Uint8Array,
): void => {
const push = sodium.crypto_secretstream_xchacha20poly1305_init_push(
m.fileKeys[fileID]!,
);
m.fileCiphertexts[fileID] =
sodium.crypto_secretstream_xchacha20poly1305_push(
push.state,
body,
null,
sodium.crypto_secretstream_xchacha20poly1305_TAG_FINAL,
);
rawFile(m, fileID).file = { decryptionHeader: toBase64(push.header) };
};
const isOriginalDownload = (url: string): boolean =>
url.includes("files.ente.io") || url.includes("/files/download/");
const login = (fetch: typeof globalThis.fetch, retry?: RetryOptions) => const login = (fetch: typeof globalThis.fetch, retry?: RetryOptions) =>
Client.login({ Client.login({
email: TEST_EMAIL, email: TEST_EMAIL,
@@ -610,33 +581,6 @@ describe("listMissingThumbnails", () => {
// Should still be 2, not 4 (each file checked only once) // Should still be 2, not 4 (each file checked only once)
expect(missing.length).toBe(2); expect(missing.length).toBe(2);
}); });
it("skips a file another account owns without fetching its thumbnail", async () => {
const otherMock = await buildThumbMock();
rawFile(otherMock, 102).ownerID = 7;
const logs: string[] = [];
const counted = countingFetch(
buildThumbFetch(otherMock),
(url) => url.includes("thumbnails.ente.io") && url.includes("102"),
);
const client = await login(counted.fetch);
const lib = await openLib(client);
const missing = await listMissingThumbnails(lib, client, (msg) =>
logs.push(msg),
);
lib.close();
expect(missing.map((m) => m.fileID)).toEqual([101]);
expect(counted.matched()).toBe(0);
expect(
logs.some(
(l) =>
l.includes("Skipping file-102.jpg") &&
l.includes("another account"),
),
).toBe(true);
});
}); });
describe("fixMissingThumbnails", () => { describe("fixMissingThumbnails", () => {
@@ -744,93 +688,6 @@ describe("fixMissingThumbnails", () => {
expect(fixMock.uploadedThumbnails.length).toBe(1); expect(fixMock.uploadedThumbnails.length).toBe(1);
expect(fixMock.uploadedThumbnails[0]!.fileID).toBe(101); expect(fixMock.uploadedThumbnails[0]!.fileID).toBe(101);
}); });
it("skips a file another account owns without downloading it", async () => {
// The server accepts a thumbnail only from the file's owner.
const fixMock = await buildThumbMock();
rawFile(fixMock, 101).ownerID = 7;
const counted = countingFetch(
buildThumbFetch(fixMock),
isOriginalDownload,
);
const client = await login(counted.fetch);
const lib = await openLib(client);
const results = await fixMissingThumbnails(lib, client, [101]);
lib.close();
expect(results[0]!.status).toBe("skipped");
expect(results[0]!.reason).toContain("another account");
expect(counted.matched()).toBe(0);
expect(fixMock.uploadedThumbnails.length).toBe(0);
});
it("skips a file whose recorded thumbnail size is 0 without downloading it", async () => {
// The server refuses a thumbnail larger than the one it records, and
// no thumbnail is 0 bytes.
const fixMock = await buildThumbMock();
rawFile(fixMock, 101).info = { thumbSize: 0 };
const counted = countingFetch(
buildThumbFetch(fixMock),
isOriginalDownload,
);
const client = await login(counted.fetch);
const lib = await openLib(client);
const results = await fixMissingThumbnails(lib, client, [101]);
lib.close();
expect(results[0]!.status).toBe("skipped");
expect(results[0]!.reason).toContain("recorded thumbnail size is 0");
expect(counted.matched()).toBe(0);
expect(fixMock.uploadedThumbnails.length).toBe(0);
});
it("re-encodes smaller until the thumbnail fits the recorded size", async () => {
// A noisy 400x300 JPEG, which the default encoding (quality 50, not
// resized because it is under 720 px) cannot compress below the size
// recorded here: one byte less than that encoding's ciphertext.
const fixMock = await buildThumbMock();
const w = 400;
const h = 300;
const noisy = new Uint8Array(
jpegJs.encode(
{
data: sodium.randombytes_buf(w * h * 4),
width: w,
height: h,
},
90,
).data,
);
replaceOriginal(fixMock, 101, noisy);
const decoded = jpegJs.decode(noisy, {
useTArray: true,
formatAsRGBA: true,
});
const defaultSize =
jpegJs.encode(decoded, 50).data.length +
sodium.crypto_secretstream_xchacha20poly1305_ABYTES;
const recordedSize = defaultSize - 1;
rawFile(fixMock, 101).info = { thumbSize: recordedSize };
const client = await login(buildThumbFetch(fixMock));
const lib = await openLib(client);
const results = await fixMissingThumbnails(lib, client, [101]);
lib.close();
expect(results[0]!.status).toBe("fixed");
const upload = fixMock.uploadedThumbnails[0]!;
expect(upload.ciphertext.length).toBeLessThanOrEqual(recordedSize);
const decrypted = decryptBlob(
upload.ciphertext,
fromBase64(upload.decryptionHeader),
fixMock.fileKeys[101]!,
);
expect(decrypted[0]).toBe(0xff);
expect(decrypted[1]).toBe(0xd8);
});
}); });
describe("Client.getApiClient", () => { describe("Client.getApiClient", () => {