Compare commits
2
Commits
573de7e104
...
6e238bc1bd
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6e238bc1bd | ||
|
|
c19943a520 |
@@ -318,6 +318,7 @@ Endpoints used:
|
||||
encrypted token plus key attributes.
|
||||
- `POST /users/ott` and `POST /users/verify-email`: email OTP fallback path.
|
||||
- `POST /users/two-factor/verify`: TOTP second factor.
|
||||
- `POST /users/logout`: end the calling token's session (`quak logout`).
|
||||
- `GET /collections/v2?sinceTime=<usec>`: list collections changed since
|
||||
microsecond timestamp; pass 0 for a full enumeration.
|
||||
- `GET /collections/v2/diff?collectionID=<id>&sinceTime=<usec>`: list files in a
|
||||
@@ -432,7 +433,9 @@ whatever else fits their use case. `Client.fromJSON(snapshot)` restores a
|
||||
working client from that snapshot without re-authenticating; it checks every
|
||||
field and each key's length first, and throws an error naming the bad field.
|
||||
`client.logout()` clears the token and zeroes the key buffers in place; every
|
||||
later call on that client throws.
|
||||
later call on that client throws. It does not contact the server, so the token
|
||||
stays valid there and in any saved snapshot; `await client.logoutOnServer()`
|
||||
first ends the session on the server (`POST /users/logout`).
|
||||
|
||||
The CLI stores the snapshot at the platform-appropriate data directory via
|
||||
`env-paths`: `~/Library/Application Support/quak/session.json` on macOS,
|
||||
@@ -442,13 +445,21 @@ you would treat the password itself. A missing file is reported as "not logged
|
||||
in"; a file that exists but is corrupt is reported as such, naming the bad
|
||||
field. Both exit with status 1.
|
||||
|
||||
`quak logout` ends the session on the server, so the token in `session.json`
|
||||
stops working even in a copy of the file, and then deletes the file. If the
|
||||
server call fails (or the file is corrupt), the file is still deleted, the
|
||||
command says the server session could not be ended, and it exits with status 1.
|
||||
It does not delete the cache: it prints the account's cache directory and says
|
||||
it still holds decrypted data (file keys in `metadata.json`, cached originals
|
||||
and thumbnails), for the user to delete if they want it gone.
|
||||
|
||||
### CLI surface
|
||||
|
||||
```
|
||||
quak [--cache-dir <path>] <command> global: local metadata/content cache location
|
||||
quak login interactive or QUAK_EMAIL/QUAK_PASSWORD
|
||||
quak whoami print logged-in account as JSON
|
||||
quak logout delete saved session
|
||||
quak logout end the session, delete it
|
||||
quak collections [--json] list all collections
|
||||
quak files --collection <id> [--json] list files in a collection
|
||||
quak get <fileID> [--out path] [--collection] download and decrypt a file
|
||||
@@ -687,10 +698,13 @@ originals and thumbnails are kept; they are reached only through the files the
|
||||
current account's records name.
|
||||
|
||||
A stored file appears only via an atomic temp-then-rename, so its presence means
|
||||
it is complete. The design also calls for a content-hash comparison against
|
||||
`FileMetadata.hash` on each fetched original; that check is deferred (issue
|
||||
https://git.eeqj.de/sneak/quak/issues/68) because the exact hash construction
|
||||
cannot yet be confirmed against the repo's fixtures.
|
||||
it is complete. Every downloaded original (by `quak get`, the cache, or
|
||||
`backup`) whose metadata records a content hash (`FileMetadata.hash`) is hashed
|
||||
as it is written: unkeyed BLAKE2b with a 64-byte output, standard base64. For a
|
||||
live photo, which is stored as a ZIP, the image and the video are hashed
|
||||
separately and joined as `<imageHash>:<videoHash>`. A mismatch stores nothing
|
||||
and fails the download with an error naming the file ID. An original with no
|
||||
recorded hash, from a very old client, is stored unchecked.
|
||||
|
||||
### Key types by source file
|
||||
|
||||
|
||||
@@ -18,12 +18,29 @@ Tag v1.0.0.
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-09-23: `quak logout` ends the session on the server (issue 108). It calls
|
||||
`POST /users/logout` through the new `Client.logoutOnServer()`, then deletes
|
||||
`session.json` even when that call fails, says so and exits 1. It prints the
|
||||
account's cache directory and says it still holds decrypted data. The default
|
||||
cache path is now `defaultCacheDirectory()` in the library, shared with
|
||||
`Library.open`.
|
||||
|
||||
- 2026-09-23: Checked downloaded originals against their recorded content hash
|
||||
(issue 68). `downloadFile`, which `quak get`, the content cache and backup all
|
||||
use, hashes the decrypted bytes (unkeyed BLAKE2b-512, standard base64) and
|
||||
stores nothing on a mismatch, failing with an error naming the file ID. A live
|
||||
photo ZIP is unpacked as it streams with `fflate` and its image and video
|
||||
hashed separately as `<imageHash>:<videoHash>`. `decryptFile` reads older
|
||||
clients' `imageHash` and `videoHash` fields for live photos. A file with no
|
||||
recorded hash is stored unchecked.
|
||||
|
||||
- 2026-09-23: Kept one account's cache from mixing with another's (issue 104).
|
||||
When `metadata.json` in the cache directory was written for a different,
|
||||
non-zero user ID than the client's, `Library.open` deletes it and `mldata/`
|
||||
and starts empty, so the first refresh enumerates from 0. This only happens
|
||||
with `--cache-dir` or an explicit `cacheDirectory`; the default path already
|
||||
includes the user ID. A test opens one account's cache as another account.
|
||||
|
||||
- 2026-09-23: `backup-metadata` no longer stops on one failed ML data request
|
||||
(issue 101). Each request of up to 200 files is tried on its own; a failed one
|
||||
is logged, its files are written with the reason in `mlDataError`, and the
|
||||
|
||||
+1
-1
@@ -73,7 +73,7 @@ program
|
||||
|
||||
program
|
||||
.command("logout")
|
||||
.description("Delete the saved session")
|
||||
.description("End the session on the server and delete the saved session")
|
||||
.action(() => run(logoutCommand(context())));
|
||||
|
||||
program
|
||||
|
||||
@@ -42,6 +42,7 @@
|
||||
"env-paths": "4.0.0",
|
||||
"exif-reader": "2.0.3",
|
||||
"fast-srp-hap": "2.0.4",
|
||||
"fflate": "0.8.3",
|
||||
"jpeg-js": "0.4.4",
|
||||
"libsodium-wrappers-sumo": "0.8.4"
|
||||
}
|
||||
|
||||
+39
-6
@@ -17,7 +17,11 @@ import {
|
||||
import { join } from "node:path";
|
||||
import { Client, type ClientSnapshot } from "./client.js";
|
||||
import { init } from "./crypto/index.js";
|
||||
import { Library, type LibraryClient } from "./library/index.js";
|
||||
import {
|
||||
defaultCacheDirectory,
|
||||
Library,
|
||||
type LibraryClient,
|
||||
} from "./library/index.js";
|
||||
import {
|
||||
fileListRow,
|
||||
fileListLine,
|
||||
@@ -146,14 +150,43 @@ export const whoamiCommand = async (ctx: CliContext): Promise<number> => {
|
||||
return 0;
|
||||
};
|
||||
|
||||
// Ends the session on the server, then deletes the session file even when that
|
||||
// failed, and exits 1 if it did. The cache is left in place; the user is told
|
||||
// where it is.
|
||||
export const logoutCommand = async (ctx: CliContext): Promise<number> => {
|
||||
if (existsSync(sessionPath(ctx))) {
|
||||
unlinkSync(sessionPath(ctx));
|
||||
ctx.stderr.write("Session deleted.\n");
|
||||
} else {
|
||||
const path = sessionPath(ctx);
|
||||
if (!existsSync(path)) {
|
||||
ctx.stderr.write("No session found.\n");
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
await init();
|
||||
let cacheDir = ctx.cacheDir;
|
||||
let failure: string | undefined;
|
||||
try {
|
||||
const client = ctx.loadSession(path);
|
||||
if (client) {
|
||||
cacheDir ??= defaultCacheDirectory(client.whoami().userID);
|
||||
await client.logoutOnServer();
|
||||
client.logout();
|
||||
}
|
||||
} catch (err) {
|
||||
failure = err instanceof Error ? err.message : String(err);
|
||||
}
|
||||
unlinkSync(path);
|
||||
if (failure === undefined) {
|
||||
ctx.stderr.write("Session ended on the server.\n");
|
||||
} else {
|
||||
ctx.stderr.write(
|
||||
`Could not end the session on the server: ${failure}\n`,
|
||||
);
|
||||
}
|
||||
ctx.stderr.write("Session deleted.\n");
|
||||
if (cacheDir !== undefined) {
|
||||
ctx.stderr.write(
|
||||
`Cache directory ${cacheDir} still holds decrypted data; delete it to remove that data.\n`,
|
||||
);
|
||||
}
|
||||
return failure === undefined ? 0 : 1;
|
||||
};
|
||||
|
||||
export const collectionsCommand = async (
|
||||
|
||||
@@ -217,6 +217,14 @@ export class Client {
|
||||
};
|
||||
}
|
||||
|
||||
// Ends this client's session on the server (`POST /users/logout`), so the
|
||||
// token stops working everywhere, including in any saved copy of it. This
|
||||
// client is left as it was; call `logout()` to clear it.
|
||||
async logoutOnServer(): Promise<void> {
|
||||
this.assertLoggedIn();
|
||||
await this.api.postJSON("/users/logout", {});
|
||||
}
|
||||
|
||||
// Zeroes the key buffers in place, so any copy of the reference held
|
||||
// elsewhere is wiped too. Every method checks `assertLoggedIn` before
|
||||
// touching the keys, so nothing decrypts with the zeroed keys.
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
import sodium, { type StateAddress } from "libsodium-wrappers-sumo";
|
||||
import { toBase64 } from "./encoding.js";
|
||||
|
||||
// The content hash an uploading client records in a file's metadata: unkeyed
|
||||
// BLAKE2b with a 64-byte output over the original's bytes, fed in chunks, as
|
||||
// standard base64 with padding. Named after the upstream client's functions.
|
||||
// The output length is read at call time for the same reason as
|
||||
// `streamTagFinal` in stream.ts: libsodium sets its constants only once ready.
|
||||
|
||||
export const chunkHashInit = (): StateAddress =>
|
||||
sodium.crypto_generichash_init(null, sodium.crypto_generichash_BYTES_MAX);
|
||||
|
||||
export const chunkHashUpdate = (state: StateAddress, chunk: Uint8Array): void =>
|
||||
sodium.crypto_generichash_update(state, chunk);
|
||||
|
||||
export const chunkHashFinal = (state: StateAddress): string =>
|
||||
toBase64(
|
||||
sodium.crypto_generichash_final(
|
||||
state,
|
||||
sodium.crypto_generichash_BYTES_MAX,
|
||||
),
|
||||
);
|
||||
@@ -7,6 +7,7 @@ export {
|
||||
} from "./encoding.js";
|
||||
export { deriveKEK, deriveLoginSubkey } from "./kdf.js";
|
||||
export { decryptBox, decryptSealed } from "./box.js";
|
||||
export { chunkHashFinal, chunkHashInit, chunkHashUpdate } from "./hash.js";
|
||||
export {
|
||||
decryptBlob,
|
||||
encryptBlob,
|
||||
|
||||
+112
-1
@@ -2,7 +2,11 @@ import { randomUUID } from "node:crypto";
|
||||
import { open, rename, rm } from "node:fs/promises";
|
||||
import type { FileHandle } from "node:fs/promises";
|
||||
import { dirname, join } from "node:path";
|
||||
import { Unzip, UnzipInflate } from "fflate";
|
||||
import {
|
||||
chunkHashFinal,
|
||||
chunkHashInit,
|
||||
chunkHashUpdate,
|
||||
fromBase64,
|
||||
initStreamPull,
|
||||
pullStreamChunk,
|
||||
@@ -237,19 +241,109 @@ export const writeAtomic = async (
|
||||
): Promise<void> =>
|
||||
stageAtomic(destination, (handle) => handle.writeFile(plaintext));
|
||||
|
||||
// Hashes an original's bytes as they are decrypted, for comparison with the
|
||||
// hash its uploader recorded.
|
||||
interface ContentHasher {
|
||||
update: (plaintext: Uint8Array) => void;
|
||||
digest: () => string;
|
||||
}
|
||||
|
||||
const fileHasher = (): ContentHasher => {
|
||||
const state = chunkHashInit();
|
||||
return {
|
||||
update: (plaintext) => chunkHashUpdate(state, plaintext),
|
||||
digest: () => chunkHashFinal(state),
|
||||
};
|
||||
};
|
||||
|
||||
// A live photo is stored as a ZIP of its image and its video, and its recorded
|
||||
// hash is `<imageHash>:<videoHash>`, each over that part's own bytes. Like the
|
||||
// upstream client's decoder, this takes the first entries whose names start
|
||||
// with `image` and `video`.
|
||||
//
|
||||
// The ZIP is chosen by its uploader and may expand enormously, so entries are
|
||||
// hashed as they decompress and never held. fflate's `Unzip` inflates each
|
||||
// push in one piece, and deflate expands at most about 1000-fold, so the ZIP
|
||||
// is pushed in 4 KiB slices to keep each decompressed piece near 4 MiB, one
|
||||
// plaintext chunk. Every entry is started, even one that is not hashed,
|
||||
// because fflate keeps an unstarted entry's data in memory.
|
||||
const livePhotoHasher = (fileID: number): ContentHasher => {
|
||||
const sliceSize = 4096;
|
||||
const fail = (message: string, cause?: unknown): Error =>
|
||||
new Error(`download: file ${fileID}: ${message}`, { cause });
|
||||
const claimed = new Set<string>();
|
||||
const hashes = new Map<string, string>();
|
||||
const unzip = new Unzip((entry) => {
|
||||
const part = ["image", "video"].find((p) => entry.name.startsWith(p));
|
||||
const target =
|
||||
part === undefined || claimed.has(part)
|
||||
? undefined
|
||||
: { part, state: chunkHashInit() };
|
||||
if (target !== undefined) claimed.add(target.part);
|
||||
entry.ondata = (err, data, final) => {
|
||||
if (err) throw err;
|
||||
if (target === undefined) return;
|
||||
chunkHashUpdate(target.state, data);
|
||||
if (final) hashes.set(target.part, chunkHashFinal(target.state));
|
||||
};
|
||||
entry.start();
|
||||
});
|
||||
unzip.register(UnzipInflate);
|
||||
// fflate reports a bad ZIP by throwing, sometimes a TypeError, which the
|
||||
// retry would take for a network failure; a bad ZIP is never retried.
|
||||
const push = (data: Uint8Array, final: boolean): void => {
|
||||
try {
|
||||
unzip.push(data, final);
|
||||
} catch (err) {
|
||||
throw fail("live photo is not a readable ZIP", err);
|
||||
}
|
||||
};
|
||||
return {
|
||||
update: (plaintext) => {
|
||||
for (let i = 0; i < plaintext.length; i += sliceSize) {
|
||||
push(plaintext.subarray(i, i + sliceSize), false);
|
||||
}
|
||||
},
|
||||
digest: () => {
|
||||
push(new Uint8Array(0), true);
|
||||
const image = hashes.get("image");
|
||||
const video = hashes.get("video");
|
||||
if (image === undefined || video === undefined) {
|
||||
throw fail(
|
||||
"live photo ZIP does not hold both an image and a video",
|
||||
);
|
||||
}
|
||||
return `${image}:${video}`;
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
// Decrypt `stream` straight to `destination`, one plaintext chunk at a time,
|
||||
// under the atomic writer's temp-then-rename discipline. Memory stays bounded
|
||||
// by the chunk size: each decrypted chunk is written to the temp file and
|
||||
// dropped. The rename happens only after the stream authenticates as terminated
|
||||
// on TAG_FINAL; a truncated stream throws and leaves the destination untouched.
|
||||
// Returns the plaintext length written.
|
||||
//
|
||||
// `original` is the file whose original this is (none for a thumbnail, which
|
||||
// has no recorded hash). When its metadata has a hash, the decrypted bytes
|
||||
// must match it or nothing is stored. Both a plain file and a live photo's
|
||||
// parts are hashed as they stream. The mismatch error is not retried.
|
||||
const decryptToTemp = async (
|
||||
destination: string,
|
||||
stream: ReadableStream<Uint8Array>,
|
||||
header: Uint8Array,
|
||||
key: Uint8Array,
|
||||
onProgress?: ProgressCallback,
|
||||
original?: EnteFile,
|
||||
): Promise<number> => {
|
||||
const expected = original?.metadata.hash;
|
||||
const hasher =
|
||||
original === undefined || expected === undefined
|
||||
? undefined
|
||||
: original.metadata.fileType === "livePhoto"
|
||||
? livePhotoHasher(original.id)
|
||||
: fileHasher();
|
||||
let bytesWritten = 0;
|
||||
try {
|
||||
await stageAtomic(destination, async (handle) => {
|
||||
@@ -258,10 +352,18 @@ const decryptToTemp = async (
|
||||
header,
|
||||
key,
|
||||
async (plaintext) => {
|
||||
hasher?.update(plaintext);
|
||||
await handle.write(plaintext);
|
||||
},
|
||||
onProgress,
|
||||
);
|
||||
if (original === undefined || hasher === undefined) return;
|
||||
const actual = hasher.digest();
|
||||
if (actual !== expected) {
|
||||
throw new Error(
|
||||
`download: file ${original.id}: content hash ${actual} does not match the hash its uploader recorded, ${expected}`,
|
||||
);
|
||||
}
|
||||
});
|
||||
} catch (err) {
|
||||
// Cancel the body so its connection is closed now rather than held
|
||||
@@ -302,10 +404,18 @@ const fetchAndDecrypt = async (
|
||||
key: Uint8Array,
|
||||
destination: string,
|
||||
onProgress?: ProgressCallback,
|
||||
original?: EnteFile,
|
||||
): Promise<number> =>
|
||||
withRetry(async () => {
|
||||
const stream = await openStream();
|
||||
return decryptToTemp(destination, stream, header, key, onProgress);
|
||||
return decryptToTemp(
|
||||
destination,
|
||||
stream,
|
||||
header,
|
||||
key,
|
||||
onProgress,
|
||||
original,
|
||||
);
|
||||
}, api.getRetryOptions());
|
||||
|
||||
export const downloadFile = async (
|
||||
@@ -326,6 +436,7 @@ export const downloadFile = async (
|
||||
file.key,
|
||||
resolvedPath,
|
||||
onProgress,
|
||||
file,
|
||||
);
|
||||
return { path: resolvedPath, bytesWritten };
|
||||
};
|
||||
|
||||
@@ -15,12 +15,12 @@
|
||||
// Integrity. The reused streaming decrypt is the enforced guarantee: every
|
||||
// chunk is authenticated and the writer renames the file into place only once
|
||||
// the stream ends on TAG_FINAL, so a truncated or corrupt fetch throws and
|
||||
// nothing is stored. On top of that this module refuses to record a stored file
|
||||
// that came out empty. The design also asks for a content-hash comparison
|
||||
// against `FileMetadata.hash` (with a `fileSize` fallback); that is deferred —
|
||||
// see the PR — because the exact hash construction cannot be confirmed against
|
||||
// the repo's fixtures and `FileBlob.size` is the encrypted object size, not the
|
||||
// decrypted length this layer has.
|
||||
// nothing is stored. For an original whose metadata records a content hash
|
||||
// (`FileMetadata.hash`), the writer also hashes the decrypted bytes and stores
|
||||
// nothing if they differ, failing the fetch with an error naming the file. An
|
||||
// original with no recorded hash is stored unchecked, as the upstream client
|
||||
// does; thumbnails have none. On top of that this module refuses to record a
|
||||
// stored file that came out empty.
|
||||
|
||||
import { existsSync, statSync } from "node:fs";
|
||||
import {
|
||||
|
||||
@@ -98,6 +98,11 @@ export {
|
||||
|
||||
export const DEFAULT_REFRESH_INTERVAL_SECONDS = 3;
|
||||
|
||||
// The account's cache directory when `cacheDirectory` is not given: the
|
||||
// env-paths cache directory plus the user id, so each account has its own.
|
||||
export const defaultCacheDirectory = (userID: number): string =>
|
||||
join(envPaths("quak", { suffix: "" }).cache, String(userID));
|
||||
|
||||
// Project a metadata store into by-id records, filling each record's cache
|
||||
// paths from the content cache when one is given. Shared by the live read
|
||||
// projection and the precache's initial seeding at open().
|
||||
@@ -344,8 +349,7 @@ export class Library {
|
||||
static async open(opts: LibraryOptions): Promise<Library> {
|
||||
const { userID } = opts.client.whoami();
|
||||
const cacheDirectory =
|
||||
opts.cacheDirectory ??
|
||||
join(envPaths("quak", { suffix: "" }).cache, String(userID));
|
||||
opts.cacheDirectory ?? defaultCacheDirectory(userID);
|
||||
const metadataPath = join(cacheDirectory, "metadata.json");
|
||||
let store = await MetadataStore.load(metadataPath);
|
||||
// A cache directory given explicitly can hold another account's cache.
|
||||
|
||||
+23
-1
@@ -34,6 +34,28 @@ const FILE_TYPE_MAP: Record<number, FileType> = {
|
||||
|
||||
const parseFileType = (n: number): FileType => FILE_TYPE_MAP[n] ?? "unknown";
|
||||
|
||||
// The hash the uploading client recorded for the original's bytes, read the
|
||||
// way the upstream client's `metadataHash` reads it: `hash` if present,
|
||||
// otherwise, for a live photo from an older client that wrote the two parts
|
||||
// separately, `<imageHash>:<videoHash>`. A field that is not a non-empty
|
||||
// string counts as absent, and a file with no hash at all is normal.
|
||||
const expectedHash = (json: Record<string, unknown>): string | undefined => {
|
||||
const text = (v: unknown): string | undefined =>
|
||||
typeof v === "string" && v !== "" ? v : undefined;
|
||||
const hash = text(json.hash);
|
||||
if (hash !== undefined) return hash;
|
||||
const imageHash = text(json.imageHash);
|
||||
const videoHash = text(json.videoHash);
|
||||
if (
|
||||
json.fileType === 2 &&
|
||||
imageHash !== undefined &&
|
||||
videoHash !== undefined
|
||||
) {
|
||||
return `${imageHash}:${videoHash}`;
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
|
||||
export const decryptCollection = (
|
||||
raw: RawCollection,
|
||||
keys: KeyMaterial,
|
||||
@@ -115,7 +137,7 @@ export const decryptFile = (
|
||||
modificationTime: metadataJSON.modificationTime ?? 0,
|
||||
latitude: metadataJSON.latitude,
|
||||
longitude: metadataJSON.longitude,
|
||||
hash: metadataJSON.hash,
|
||||
hash: expectedHash(metadataJSON),
|
||||
};
|
||||
|
||||
const magicMetadata = decryptMagicMetadata(raw.magicMetadata, key);
|
||||
|
||||
@@ -29,6 +29,9 @@ export interface FileMetadata {
|
||||
modificationTime: Microseconds;
|
||||
latitude?: number;
|
||||
longitude?: number;
|
||||
// The content hash the uploader recorded (see `expectedHash` in
|
||||
// decrypt.ts); `downloadFile` refuses an original that does not match it.
|
||||
// Absent for files from very old clients.
|
||||
hash?: string;
|
||||
}
|
||||
|
||||
|
||||
+81
-14
@@ -38,7 +38,8 @@ import { loadSession } from "../../src/cli-session.js";
|
||||
import type { Client, ClientSnapshot } from "../../src/client.js";
|
||||
import type { ContentSource } from "../../src/library/content.js";
|
||||
import type { Collection, EnteFile } from "../../src/model/types.js";
|
||||
import { init } from "../../src/crypto/index.js";
|
||||
import { init, toBase64 } from "../../src/crypto/index.js";
|
||||
import { defaultCacheDirectory } from "../../src/library/index.js";
|
||||
|
||||
const USER_ID = 42;
|
||||
|
||||
@@ -176,19 +177,6 @@ describe("session file", () => {
|
||||
expect(JSON.parse(readFileSync(path, "utf-8"))).toEqual(snapshot);
|
||||
});
|
||||
|
||||
it("is removed by logout", async () => {
|
||||
const ctx = context();
|
||||
saveSession(ctx.sessionDir, snapshot);
|
||||
expect(await logoutCommand(ctx)).toBe(0);
|
||||
expect(existsSync(join(ctx.sessionDir, "session.json"))).toBe(false);
|
||||
expect(stderr.text).toBe("Session deleted.\n");
|
||||
});
|
||||
|
||||
it("logout without a session says so and exits 0", async () => {
|
||||
expect(await logoutCommand(context())).toBe(0);
|
||||
expect(stderr.text).toBe("No session found.\n");
|
||||
});
|
||||
|
||||
it("a missing session exits 1 with 'Not logged in'", async () => {
|
||||
const ctx = { ...context(), loadSession };
|
||||
expect(await whoamiCommand(ctx)).toBe(1);
|
||||
@@ -211,6 +199,85 @@ describe("session file", () => {
|
||||
});
|
||||
});
|
||||
|
||||
// These use a real client read from the session file, over a fake API that
|
||||
// records each request and answers with `status`.
|
||||
describe("logout", () => {
|
||||
const snapshot: ClientSnapshot = {
|
||||
email: "cli@example.com",
|
||||
userID: USER_ID,
|
||||
token: "saved-token",
|
||||
masterKey: toBase64(new Uint8Array(32)),
|
||||
secretKey: toBase64(new Uint8Array(32)),
|
||||
publicKey: toBase64(new Uint8Array(32)),
|
||||
};
|
||||
|
||||
const requests: Request[] = [];
|
||||
|
||||
const logoutContext = (status: number): CliContext => ({
|
||||
...context(),
|
||||
loadSession: (path) =>
|
||||
loadSession(path, {
|
||||
fetch: async (url, init) => {
|
||||
requests.push(new Request(url, init));
|
||||
return new Response(JSON.stringify({}), {
|
||||
status,
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
},
|
||||
}),
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
requests.length = 0;
|
||||
});
|
||||
|
||||
it("ends the session on the server, then deletes the file", async () => {
|
||||
const ctx = logoutContext(200);
|
||||
saveSession(ctx.sessionDir, snapshot);
|
||||
expect(await logoutCommand(ctx)).toBe(0);
|
||||
|
||||
expect(requests).toHaveLength(1);
|
||||
expect(requests[0]!.method).toBe("POST");
|
||||
expect(new URL(requests[0]!.url).pathname).toBe("/users/logout");
|
||||
expect(requests[0]!.headers.get("X-Auth-Token")).toBe("saved-token");
|
||||
expect(existsSync(join(ctx.sessionDir, "session.json"))).toBe(false);
|
||||
expect(stderr.text).toBe(
|
||||
"Session ended on the server.\n" +
|
||||
"Session deleted.\n" +
|
||||
`Cache directory ${ctx.cacheDir} still holds decrypted data; delete it to remove that data.\n`,
|
||||
);
|
||||
});
|
||||
|
||||
it("still deletes the file when the server call fails, and says so", async () => {
|
||||
const ctx = logoutContext(500);
|
||||
saveSession(ctx.sessionDir, snapshot);
|
||||
expect(await logoutCommand(ctx)).toBe(1);
|
||||
|
||||
expect(requests).toHaveLength(1);
|
||||
expect(existsSync(join(ctx.sessionDir, "session.json"))).toBe(false);
|
||||
expect(stderr.text).toBe(
|
||||
"Could not end the session on the server: HTTP 500\n" +
|
||||
"Session deleted.\n" +
|
||||
`Cache directory ${ctx.cacheDir} still holds decrypted data; delete it to remove that data.\n`,
|
||||
);
|
||||
});
|
||||
|
||||
it("names the account's default cache directory without --cache-dir", async () => {
|
||||
const ctx = { ...logoutContext(200), cacheDir: undefined };
|
||||
saveSession(ctx.sessionDir, snapshot);
|
||||
expect(await logoutCommand(ctx)).toBe(0);
|
||||
expect(stderr.text).toContain(
|
||||
`Cache directory ${defaultCacheDirectory(USER_ID)} still holds decrypted data`,
|
||||
);
|
||||
});
|
||||
|
||||
it("without a session says so, calls nothing and exits 0", async () => {
|
||||
expect(await logoutCommand(logoutContext(200))).toBe(0);
|
||||
expect(requests).toHaveLength(0);
|
||||
expect(stderr.text).toBe("No session found.\n");
|
||||
});
|
||||
});
|
||||
|
||||
describe("whoami", () => {
|
||||
it("prints the account as one line of JSON", async () => {
|
||||
expect(await whoamiCommand(context())).toBe(0);
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
import { beforeAll, describe, expect, it } from "vitest";
|
||||
import {
|
||||
chunkHashFinal,
|
||||
chunkHashInit,
|
||||
chunkHashUpdate,
|
||||
init,
|
||||
} from "../../src/crypto/index.js";
|
||||
|
||||
beforeAll(async () => {
|
||||
await init();
|
||||
});
|
||||
|
||||
describe("content hash", () => {
|
||||
// RFC 7693 Appendix A: BLAKE2b-512 of "abc".
|
||||
const abc = Buffer.from(
|
||||
"ba80a53f981c4d0d6a2797b69f12f6e94c212f14685ac4b74b12bb6fdbffa2d1" +
|
||||
"7d87c5392aab792dc252d5de4533cc9518d38aa8dbf1925ab92386edd4009923",
|
||||
"hex",
|
||||
).toString("base64");
|
||||
|
||||
it("is unkeyed BLAKE2b-512 in standard base64", () => {
|
||||
const state = chunkHashInit();
|
||||
chunkHashUpdate(state, new TextEncoder().encode("abc"));
|
||||
expect(chunkHashFinal(state)).toBe(abc);
|
||||
});
|
||||
|
||||
it("gives the same hash when the input arrives in chunks", () => {
|
||||
const state = chunkHashInit();
|
||||
chunkHashUpdate(state, new TextEncoder().encode("a"));
|
||||
chunkHashUpdate(state, new TextEncoder().encode("bc"));
|
||||
expect(chunkHashFinal(state)).toBe(abc);
|
||||
});
|
||||
});
|
||||
@@ -61,6 +61,7 @@ import { dirname, join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { createHash } from "node:crypto";
|
||||
import sodium from "libsodium-wrappers-sumo";
|
||||
import { zipSync } from "fflate";
|
||||
import {
|
||||
beforeAll,
|
||||
beforeEach,
|
||||
@@ -188,7 +189,31 @@ vi.mock("node:fs/promises", async (importOriginal) => {
|
||||
};
|
||||
});
|
||||
|
||||
/**
|
||||
* `chunkHashUpdate` is wrapped to record the length of every piece hashed, so
|
||||
* a test can show that a live photo entry reaches the hash in pieces far
|
||||
* smaller than the entry, rather than decompressed whole first.
|
||||
*/
|
||||
const hashHook = vi.hoisted(() => ({
|
||||
lengths: [] as number[],
|
||||
}));
|
||||
|
||||
vi.mock("../../src/crypto/index.js", async (importOriginal) => {
|
||||
const actual =
|
||||
await importOriginal<typeof import("../../src/crypto/index.js")>();
|
||||
return {
|
||||
...actual,
|
||||
chunkHashUpdate: (
|
||||
...args: Parameters<typeof actual.chunkHashUpdate>
|
||||
): void => {
|
||||
hashHook.lengths.push(args[1].length);
|
||||
actual.chunkHashUpdate(...args);
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
hashHook.lengths.length = 0;
|
||||
renameHook.calls.length = 0;
|
||||
renameHook.failWith = null;
|
||||
durabilityHook.events.length = 0;
|
||||
@@ -1613,3 +1638,113 @@ describe.each(entryPoints)("$name progress", ({ name, download }) => {
|
||||
expectSameBytes(readFileSync(outPath), plaintext);
|
||||
});
|
||||
});
|
||||
|
||||
describe("downloadFile content hash", () => {
|
||||
// Node's own BLAKE2b-512 is the reference, so these tests do not depend
|
||||
// on the code under test to compute what they expect.
|
||||
const blake2b = (bytes: Uint8Array): string =>
|
||||
createHash("blake2b512").update(bytes).digest("base64");
|
||||
|
||||
// Serve `plaintext` encrypted as file 999 with the given metadata. Four
|
||||
// responses are scripted so a retried mismatch would show in `requests`.
|
||||
const setup = (plaintext: Uint8Array, metadata: Partial<FileMetadata>) => {
|
||||
const key = sodium.crypto_secretstream_xchacha20poly1305_keygen();
|
||||
const { header, ciphertext } = encryptFileBody(plaintext, key);
|
||||
const file = buildMockEnteFile(key, header, header);
|
||||
file.metadata = { ...file.metadata, ...metadata };
|
||||
const body = { kind: "body", bytes: ciphertext } as const;
|
||||
const { fetch, requests } = scriptedCdnFetch(body, body, body, body);
|
||||
const api = new ApiClient({ fetch, retry: { ...noWait, attempts: 4 } });
|
||||
const dir = mkdtempSync(join(testDir, "hash-"));
|
||||
const outPath = join(dir, "f.bin");
|
||||
return {
|
||||
run: () => downloadFile(api, file, outPath),
|
||||
dir,
|
||||
outPath,
|
||||
requests,
|
||||
};
|
||||
};
|
||||
|
||||
const livePhotoZip = zipSync({
|
||||
"image.heic": patternBytes(500, 81),
|
||||
"video.mov": patternBytes(900, 82),
|
||||
});
|
||||
const livePhotoHash = `${blake2b(patternBytes(500, 81))}:${blake2b(patternBytes(900, 82))}`;
|
||||
|
||||
it("stores a file whose hash matches", async () => {
|
||||
const plaintext = patternBytes(700, 80);
|
||||
const t = setup(plaintext, { hash: blake2b(plaintext) });
|
||||
|
||||
await t.run();
|
||||
|
||||
expectSameBytes(readFileSync(t.outPath), plaintext);
|
||||
});
|
||||
|
||||
it("rejects a mismatch, stores nothing, names the file and does not retry", async () => {
|
||||
const t = setup(patternBytes(700, 80), {
|
||||
hash: blake2b(patternBytes(700, 79)),
|
||||
});
|
||||
|
||||
await expect(t.run()).rejects.toThrow(
|
||||
/file 999: content hash .* does not match/,
|
||||
);
|
||||
|
||||
expect(readdirSync(t.dir)).toEqual([]);
|
||||
expect(t.requests()).toBe(1);
|
||||
});
|
||||
|
||||
it("stores a file with no recorded hash unchecked", async () => {
|
||||
const plaintext = patternBytes(700, 80);
|
||||
const t = setup(plaintext, { hash: undefined });
|
||||
|
||||
await t.run();
|
||||
|
||||
expectSameBytes(readFileSync(t.outPath), plaintext);
|
||||
});
|
||||
|
||||
it("stores a live photo whose image and video hashes match", async () => {
|
||||
const t = setup(livePhotoZip, {
|
||||
fileType: "livePhoto",
|
||||
hash: livePhotoHash,
|
||||
});
|
||||
|
||||
await t.run();
|
||||
|
||||
expectSameBytes(readFileSync(t.outPath), livePhotoZip);
|
||||
});
|
||||
|
||||
it("hashes a large live photo entry as it decompresses, never whole", async () => {
|
||||
// 64 MiB of zeros deflates to a few kilobytes, the shape of a ZIP
|
||||
// that would exhaust memory if expanded whole.
|
||||
const image = new Uint8Array(64 * 1024 * 1024);
|
||||
const video = patternBytes(900, 83);
|
||||
const zip = zipSync({ "image.heic": image, "video.mov": video });
|
||||
const t = setup(zip, {
|
||||
fileType: "livePhoto",
|
||||
hash: `${blake2b(image)}:${blake2b(video)}`,
|
||||
});
|
||||
|
||||
await t.run();
|
||||
|
||||
expectSameBytes(readFileSync(t.outPath), zip);
|
||||
const hashed = hashHook.lengths.reduce((a, b) => a + b, 0);
|
||||
expect(hashed).toBe(image.length + video.length);
|
||||
expect(Math.max(...hashHook.lengths)).toBeLessThanOrEqual(
|
||||
2 * STREAM_CHUNK_SIZE,
|
||||
);
|
||||
});
|
||||
|
||||
it("rejects a live photo whose hash does not match", async () => {
|
||||
// The whole ZIP's hash is not the recorded one: each part is hashed.
|
||||
const t = setup(livePhotoZip, {
|
||||
fileType: "livePhoto",
|
||||
hash: blake2b(livePhotoZip),
|
||||
});
|
||||
|
||||
await expect(t.run()).rejects.toThrow(
|
||||
/file 999: content hash .* does not match/,
|
||||
);
|
||||
|
||||
expect(readdirSync(t.dir)).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -351,6 +351,46 @@ describe("model.decryptFile", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("reads the recorded content hash, joining an older live photo's two parts", () => {
|
||||
const masterKey = sodium.crypto_secretbox_keygen();
|
||||
const { collectionKey } = buildRawCollection(masterKey);
|
||||
const hashOf = (metadata: Record<string, unknown>) =>
|
||||
decryptFile(
|
||||
buildRawFile(collectionKey, {
|
||||
metadata: { title: "x", ...metadata },
|
||||
}),
|
||||
collectionKey,
|
||||
).metadata.hash;
|
||||
|
||||
expect(hashOf({ fileType: 0, hash: "H" })).toBe("H");
|
||||
expect(
|
||||
hashOf({ fileType: 2, hash: "H", imageHash: "I", videoHash: "V" }),
|
||||
).toBe("H");
|
||||
expect(hashOf({ fileType: 2, imageHash: "I", videoHash: "V" })).toBe(
|
||||
"I:V",
|
||||
);
|
||||
expect(hashOf({ fileType: 2, imageHash: "I" })).toBeUndefined();
|
||||
expect(
|
||||
hashOf({ fileType: 0, imageHash: "I", videoHash: "V" }),
|
||||
).toBeUndefined();
|
||||
expect(hashOf({ fileType: 0 })).toBeUndefined();
|
||||
expect(hashOf({ fileType: 0, hash: 42 })).toBeUndefined();
|
||||
expect(
|
||||
hashOf({ fileType: 2, imageHash: "I", videoHash: 7 }),
|
||||
).toBeUndefined();
|
||||
// An empty string counts as absent, not as a hash to match.
|
||||
expect(hashOf({ fileType: 0, hash: "" })).toBeUndefined();
|
||||
expect(
|
||||
hashOf({ fileType: 2, hash: "", imageHash: "I", videoHash: "V" }),
|
||||
).toBe("I:V");
|
||||
expect(
|
||||
hashOf({ fileType: 2, imageHash: "", videoHash: "V" }),
|
||||
).toBeUndefined();
|
||||
expect(
|
||||
hashOf({ fileType: 2, imageHash: "I", videoHash: "" }),
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
it("maps fileType numbers to FileType strings", () => {
|
||||
// Ente uses: 0=image, 1=video, 2=livePhoto
|
||||
const masterKey = sodium.crypto_secretbox_keygen();
|
||||
|
||||
@@ -1069,6 +1069,11 @@ fastq@^1.6.0:
|
||||
dependencies:
|
||||
reusify "^1.0.4"
|
||||
|
||||
fflate@0.8.3:
|
||||
version "0.8.3"
|
||||
resolved "https://registry.yarnpkg.com/fflate/-/fflate-0.8.3.tgz#bc27d8eb30343d4d512abb03480202ce65d825fc"
|
||||
integrity sha512-tbZNuJrLwGUp3zshBtdy4W+ORxZuIh8a5ilyIEQDC5rY1f3U20JMry0Ll3WBzU58EZKsEuJFXhb5gwv8CsPvgA==
|
||||
|
||||
file-entry-cache@^8.0.0:
|
||||
version "8.0.0"
|
||||
resolved "https://registry.yarnpkg.com/file-entry-cache/-/file-entry-cache-8.0.0.tgz#7787bddcf1131bffb92636c69457bbc0edd6d81f"
|
||||
|
||||
Reference in New Issue
Block a user