Author SHA1 Message Date
sneak b6020aca4e quak login --server: log in to a self-hosted Ente server (closes #243)
check / check (push) Successful in 4m25s
`quak login` takes the server's API address from `--server` or
`QUAK_SERVER`, refusing anything that is not an http or https URL. The
client's snapshot now keeps a non-default `apiOrigin`, so the session
file names the server and every later command uses it. A snapshot's
`apiOrigin` wins over the one in `fromJSON`'s `apiOptions`, since the
token belongs to that server.

Model: opus-5-5
2026-11-13 18:48:55 +00:00
9 changed files with 63 additions and 537 deletions

No files matched your search

+3 -20
View File
@@ -632,11 +632,6 @@ Without `--cache-dir`, each account has its own cache directory:
the variable is not set; the variable is not set;
- on macOS, `~/Library/Caches/quak/<userID>`. - on macOS, `~/Library/Caches/quak/<userID>`.
Two servers can give two accounts the same user ID. So an account on a
self-hosted server has the server's address, URL-encoded, between `quak` and the
user ID. For `https://ente.example.com`, that is
`quak/https%3A%2F%2Fente.example.com/<userID>`.
The directory is readable by its owner only (mode `0700`), because it holds The directory is readable by its owner only (mode `0700`), because it holds
decrypted data, file keys included. decrypted data, file keys included.
@@ -661,13 +656,6 @@ the library deletes it and `mldata/` and starts from an empty cache. Cached
originals and thumbnails are kept. They are reached only through the records of originals and thumbnails are kept. They are reached only through the records of
the current account, so another account's files are never served. the current account, so another account's files are never served.
When `metadata.json` belongs to an account on another server, opening the
library also deletes the originals and thumbnails. Two servers can give two
files the same ID, so one server's cached original could be served for the
other's file. quak records the account in `metadata.json` before it stores any
original or thumbnail. It does so also when `quak get` finds a file on an empty
cache.
A file appears in the cache only once it is complete: each one is written to a A file appears in the cache only once it is complete: each one is written to a
temporary file and renamed into place. Ente's clients record a hash of each temporary file and renamed into place. Ente's clients record a hash of each
file's bytes at upload, the content hash. Every original whose metadata records file's bytes at upload, the content hash. Every original whose metadata records
@@ -1071,8 +1059,7 @@ onto what the cache holds, empty or not. A refresh then starts only when
- `client`, required: the account's `Client`, or anything that satisfies - `client`, required: the account's `Client`, or anything that satisfies
`LibraryClient`. `LibraryClient`.
- `cacheDirectory`: where `metadata.json` and the content cache live. Default: - `cacheDirectory`: where `metadata.json` and the content cache live. Default:
`quak/<userID>` under the per-user cache directory, with the URL-encoded `quak/<userID>` under the per-user cache directory (see "The cache").
`apiOrigin` before `<userID>` for a client that has one (see "The cache").
- `downloadDirectory`: the root of the save paths. Default: `photos` in the - `downloadDirectory`: the root of the save paths. Default: `photos` in the
working directory, resolved once when the library opens; working directory, resolved once when the library opens;
`lib.downloadDirectory` holds the result. An original stored at its save path `lib.downloadDirectory` holds the result. An original stored at its save path
@@ -1269,8 +1256,7 @@ do.
fetch and cache as `photo.original()` and `photo.thumbnail()` do. `file` is an fetch and cache as `photo.original()` and `photo.thumbnail()` do. `file` is an
`EnteFile` the caller holds, and the library need not hold it. `quak get` and `EnteFile` the caller holds, and the library need not hold it. `quak get` and
`quak get-thumb` use them; on an empty cache they pass a file that `quak get-thumb` use them; on an empty cache they pass a file that
`lib.freshFiles()` returned. In a cache that names no account yet, they first `lib.freshFiles()` returned.
save the client's user ID and `apiOrigin` in `metadata.json`.
Lower-level reads return the decrypted model objects, which hold key material: Lower-level reads return the decrypted model objects, which hold key material:
`lib.listCollections()`, `lib.getCollection(id)`, `lib.listFiles(collectionID)`, `lib.listCollections()`, `lib.getCollection(id)`, `lib.listFiles(collectionID)`,
@@ -1509,10 +1495,7 @@ field and each key's length first, and throws an error naming the bad field.
A client whose `apiOptions` named an `apiOrigin` puts it in its snapshot. The A client whose `apiOptions` named an `apiOrigin` puts it in its snapshot. The
restored client then uses that server, even when the `apiOptions` given to restored client then uses that server, even when the `apiOptions` given to
`Client.fromJSON` name another. A snapshot without `apiOrigin` uses the `Client.fromJSON` name another. A snapshot without `apiOrigin` uses the
`apiOrigin` of those `apiOptions`, or the default server. `client.apiOrigin` `apiOrigin` of those `apiOptions`, or the default server.
holds the one in use, or undefined when none was named. `Client.fromJSON`
refuses an `apiOrigin` that is not an http or https URL, or that has a query or
fragment, as `quak login --server` does.
`Client.fromJSON` works only once libsodium, the cryptography library quak uses, `Client.fromJSON` works only once libsodium, the cryptography library quak uses,
is loaded. `Client.login` loads it, and `Client.fromJSON` does not. A program is loaded. `Client.login` loads it, and `Client.fromJSON` does not. A program
+10 -7
View File
@@ -21,7 +21,6 @@ import { extname, join } from "node:path";
import { import {
type Client, type Client,
type ClientSnapshot, type ClientSnapshot,
isServerAddress,
type LoginOptions, type LoginOptions,
} from "./client.js"; } from "./client.js";
import { lockBackupDirectory, type BackupLock } from "./backup.js"; import { lockBackupDirectory, type BackupLock } from "./backup.js";
@@ -57,7 +56,7 @@ export interface CliContext {
// Directory holding `session.json`. // Directory holding `session.json`.
sessionDir: string; sessionDir: string;
// The `--cache-dir` global, or undefined to let the library pick its // The `--cache-dir` global, or undefined to let the library pick its
// per-user default keyed by the account's server and user ID. // per-user default keyed by the account id.
cacheDir?: string; cacheDir?: string;
// Reads the session file into a client, or null when there is none. The // Reads the session file into a client, or null when there is none. The
// CLI passes `loadSession` from `cli-session.ts`; tests pass a fake client. // CLI passes `loadSession` from `cli-session.ts`; tests pass a fake client.
@@ -130,7 +129,6 @@ const parseID = (
// own terms. The content source is kept so `get`/`get-thumb`/`--exif` can fetch // own terms. The content source is kept so `get`/`get-thumb`/`--exif` can fetch
// originals through the on-disk cache. // originals through the on-disk cache.
const readLibraryClient = (client: Client): LibraryClient => ({ const readLibraryClient = (client: Client): LibraryClient => ({
apiOrigin: client.apiOrigin,
whoami: () => client.whoami(), whoami: () => client.whoami(),
collectionsSince: (args) => client.collectionsSince(args), collectionsSince: (args) => client.collectionsSince(args),
filesSince: (args) => client.filesSince(args), filesSince: (args) => client.filesSince(args),
@@ -160,6 +158,14 @@ const openReadLibrary = (ctx: CliContext, client: Client): Promise<Library> =>
// Logs in to the server `--server` or `QUAK_SERVER` names, the flag first, or // Logs in to the server `--server` or `QUAK_SERVER` names, the flag first, or
// to Ente's hosted service when neither is set. The saved session keeps the // to Ente's hosted service when neither is set. The saved session keeps the
// server, so every later command uses it. // server, so every later command uses it.
// Whether `text` can be an Ente server's address: an http or https URL, which
// may have a path, and no query or fragment, which request URLs could not
// carry.
const isServerAddress = (text: string): boolean =>
URL.canParse(text) &&
["http:", "https:"].includes(new URL(text).protocol) &&
!/[?#]/.test(text);
export const loginCommand = async ( export const loginCommand = async (
ctx: CliContext, ctx: CliContext,
opts: { server?: string }, opts: { server?: string },
@@ -227,10 +233,7 @@ export const logoutCommand = async (ctx: CliContext): Promise<number> => {
try { try {
const client = ctx.loadSession(path); const client = ctx.loadSession(path);
if (client) { if (client) {
cacheDir ??= defaultCacheDirectory( cacheDir ??= defaultCacheDirectory(client.whoami().userID);
client.whoami().userID,
client.apiOrigin,
);
await client.logoutOnServer(); await client.logoutOnServer();
client.logout(); client.logout();
} }
+3 -17
View File
@@ -69,25 +69,14 @@ export interface FilesPage {
const errorMessage = (err: unknown): string => const errorMessage = (err: unknown): string =>
err instanceof Error ? err.message : String(err); err instanceof Error ? err.message : String(err);
// Whether `text` can be an Ente server's address: an http or https URL, which
// may have a path, and no query or fragment, which request URLs could not
// carry. `quak login --server` and `Client.fromJSON` both check with it.
export const isServerAddress = (text: string): boolean =>
URL.canParse(text) &&
["http:", "https:"].includes(new URL(text).protocol) &&
!/[?#]/.test(text);
export class Client { export class Client {
// The `apiOrigin` the client was given, or undefined when it was given
// none, which means Ente's hosted service. The library keeps a cache per
// server as well as per user ID.
readonly apiOrigin: string | undefined;
private readonly api: ApiClient; private readonly api: ApiClient;
private readonly email: string; private readonly email: string;
private readonly userID: number; private readonly userID: number;
private readonly masterKey: Uint8Array; private readonly masterKey: Uint8Array;
private readonly secretKey: Uint8Array; private readonly secretKey: Uint8Array;
private readonly publicKey: Uint8Array; private readonly publicKey: Uint8Array;
private readonly apiOrigin: string | undefined;
private loggedOut = false; private loggedOut = false;
private constructor( private constructor(
@@ -174,12 +163,9 @@ export class Client {
} }
if ( if (
s.apiOrigin !== undefined && s.apiOrigin !== undefined &&
(typeof s.apiOrigin !== "string" || !isServerAddress(s.apiOrigin)) (typeof s.apiOrigin !== "string" || s.apiOrigin === "")
) { ) {
throw invalid( throw invalid("apiOrigin", "must be a non-empty string");
"apiOrigin",
"must be an http or https URL with no query or fragment",
);
} }
const apiOrigin = const apiOrigin =
(s.apiOrigin as string | undefined) ?? apiOptions?.apiOrigin; (s.apiOrigin as string | undefined) ?? apiOptions?.apiOrigin;
+14 -62
View File
@@ -119,18 +119,9 @@ export {
export const DEFAULT_REFRESH_INTERVAL_SECONDS = 3; export const DEFAULT_REFRESH_INTERVAL_SECONDS = 3;
// The account's cache directory when `cacheDirectory` is not given: the // The account's cache directory when `cacheDirectory` is not given: the
// env-paths cache directory plus the user id, so each account has its own. An // env-paths cache directory plus the user id, so each account has its own.
// account on a self-hosted server has the server's address, URL-encoded, in export const defaultCacheDirectory = (userID: number): string =>
// between, because two servers can give two accounts the same user id. join(envPaths("quak", { suffix: "" }).cache, String(userID));
export const defaultCacheDirectory = (
userID: number,
apiOrigin?: string,
): string => {
const cache = envPaths("quak", { suffix: "" }).cache;
return apiOrigin === undefined
? join(cache, String(userID))
: join(cache, encodeURIComponent(apiOrigin), String(userID));
};
// Project a metadata store into by-id records, filling each record's cache // Project a metadata store into by-id records, filling each record's cache
// paths from the content cache when one is given. Shared by the live read // paths from the content cache when one is given. Shared by the live read
@@ -153,9 +144,6 @@ const deriveRecordsFromStore = (
// tests drive a mock with no crypto or network; the real `Client` satisfies it // tests drive a mock with no crypto or network; the real `Client` satisfies it
// structurally. // structurally.
export interface LibraryClient { export interface LibraryClient {
// The server the client talks to, as `Client.apiOrigin` names it. Absent
// means Ente's hosted service.
readonly apiOrigin?: string;
whoami(): { email: string; userID: number }; whoami(): { email: string; userID: number };
collectionsSince(args: { sinceTime: number }): Promise<CollectionsPage>; collectionsSince(args: { sinceTime: number }): Promise<CollectionsPage>;
filesSince(args: { filesSince(args: {
@@ -208,8 +196,8 @@ export type RefreshProgressCallback = (event: RefreshEvent) => void;
export interface LibraryOptions { export interface LibraryOptions {
client: LibraryClient; client: LibraryClient;
// Where `metadata.json` lives. Defaults to `defaultCacheDirectory`, so // Where `metadata.json` lives. Defaults to the env-paths cache directory
// each account has its own cache. // plus the user id, so each account has its own cache.
cacheDirectory?: string; cacheDirectory?: string;
// The root of every photo's save path, where `Photo.download()` and // The root of every photo's save path, where `Photo.download()` and
// `lib.backup()` put originals. Defaults to `photos` in the working // `lib.backup()` put originals. Defaults to `photos` in the working
@@ -312,7 +300,6 @@ export class Library {
// available (a metadata-only client with no explicit source). // available (a metadata-only client with no explicit source).
private readonly cache?: ContentCache; private readonly cache?: ContentCache;
private readonly userID: number; private readonly userID: number;
private readonly apiOrigin: string | undefined;
private readonly intervalMs: number; private readonly intervalMs: number;
private readonly onProgress?: RefreshProgressCallback; private readonly onProgress?: RefreshProgressCallback;
private readonly pools: RequestPools; private readonly pools: RequestPools;
@@ -359,7 +346,6 @@ export class Library {
client: LibraryClient; client: LibraryClient;
store: MetadataStore; store: MetadataStore;
userID: number; userID: number;
apiOrigin: string | undefined;
cacheDirectory: string; cacheDirectory: string;
downloadDirectory: string; downloadDirectory: string;
intervalMs: number; intervalMs: number;
@@ -372,7 +358,6 @@ export class Library {
this.client = args.client; this.client = args.client;
this.store = args.store; this.store = args.store;
this.userID = args.userID; this.userID = args.userID;
this.apiOrigin = args.apiOrigin;
this.cacheDirectory = args.cacheDirectory; this.cacheDirectory = args.cacheDirectory;
this.downloadDirectory = args.downloadDirectory; this.downloadDirectory = args.downloadDirectory;
this.intervalMs = args.intervalMs; this.intervalMs = args.intervalMs;
@@ -421,9 +406,8 @@ export class Library {
// refresh: `open()` returns onto the cache as it is. // refresh: `open()` returns onto the cache as it is.
static async open(opts: LibraryOptions): Promise<Library> { static async open(opts: LibraryOptions): Promise<Library> {
const { userID } = opts.client.whoami(); const { userID } = opts.client.whoami();
const { apiOrigin } = opts.client;
const cacheDirectory = const cacheDirectory =
opts.cacheDirectory ?? defaultCacheDirectory(userID, apiOrigin); opts.cacheDirectory ?? defaultCacheDirectory(userID);
if (opts.downloadDirectory === "") { if (opts.downloadDirectory === "") {
throw new Error( throw new Error(
"library: downloadDirectory is empty (leave it out to save " + "library: downloadDirectory is empty (leave it out to save " +
@@ -435,24 +419,14 @@ export class Library {
let store = await MetadataStore.load(metadataPath); let store = await MetadataStore.load(metadataPath);
// A cache directory given explicitly can hold another account's cache. // A cache directory given explicitly can hold another account's cache.
// Its records and cursor are not this account's, so delete it and the // Its records and cursor are not this account's, so delete it and the
// ML data beside it and start empty. A cache from another server goes // ML data beside it and start empty. A user ID of 0 means the cache
// whole: two servers can give two files the same ID, so its originals // was never refreshed and so holds nothing to discard.
// and thumbnails could be served for this account's files. A user ID if (store.userID !== 0 && store.userID !== userID) {
// of 0 means the cache names no account, so nothing is stored in it. await rm(metadataPath, { force: true });
if ( await rm(join(cacheDirectory, "mldata"), {
store.userID !== 0 &&
(store.userID !== userID || store.apiOrigin !== apiOrigin)
) {
const discard = ["metadata.json", "mldata"];
if (store.apiOrigin !== apiOrigin) {
discard.push("originals", "thumbnails");
}
for (const name of discard) {
await rm(join(cacheDirectory, name), {
recursive: true, recursive: true,
force: true, force: true,
}); });
}
store = await MetadataStore.load(metadataPath); store = await MetadataStore.load(metadataPath);
} }
const intervalMs = const intervalMs =
@@ -516,7 +490,6 @@ export class Library {
client: opts.client, client: opts.client,
store, store,
userID, userID,
apiOrigin,
cacheDirectory, cacheDirectory,
downloadDirectory, downloadDirectory,
intervalMs, intervalMs,
@@ -536,11 +509,9 @@ export class Library {
if (opts.refreshOnOpen === false) { if (opts.refreshOnOpen === false) {
// The caller asks for each refresh it needs. // The caller asks for each refresh it needs.
lib.scheduleNext(); lib.scheduleNext();
} else if (store.listCollections().length > 0) { } else if (store.loadedFromDisk) {
// An existing copy already answers reads; refresh in the background // An existing copy already answers reads; refresh in the background
// and start the interval once that first cycle settles. A cache // and start the interval once that first cycle settles.
// that names the account but holds no collection has nothing to
// answer with, such as after `originalOf` on an empty cache.
void lib.runRefresh().then(() => lib.scheduleNext()); void lib.runRefresh().then(() => lib.scheduleNext());
} else { } else {
// Nothing was cached: wait for the first refresh to fill the store // Nothing was cached: wait for the first refresh to fill the store
@@ -687,7 +658,6 @@ export class Library {
"originalOf requires a library opened with a content cache", "originalOf requires a library opened with a content cache",
); );
} }
await this.saveAccountInEmptyCache();
return this.cache.originalOf(file, opts); return this.cache.originalOf(file, opts);
} }
@@ -703,23 +673,9 @@ export class Library {
"thumbnailOf requires a library opened with a content cache", "thumbnailOf requires a library opened with a content cache",
); );
} }
await this.saveAccountInEmptyCache();
return this.cache.thumbnailOf(file, opts); return this.cache.thumbnailOf(file, opts);
} }
// Saves the account's user ID and server in a cache that names no account
// yet, before `originalOf` or `thumbnailOf` stores a file there. A later
// `open()` for an account on another server then deletes that file (see
// `open()`), although no refresh has stored any records.
private async saveAccountInEmptyCache(): Promise<void> {
if (this.store.userID !== 0) return;
this.store.userID = this.userID;
this.store.apiOrigin = this.apiOrigin;
this.unsaved = true;
await this.store.save();
this.unsaved = false;
}
// Back up every in-scope file to `opts.downloadDirectory`, or else the // Back up every in-scope file to `opts.downloadDirectory`, or else the
// library's, each original at its save path, with a durable failure // library's, each original at its save path, with a durable failure
// ledger (issue #51). Takes the lock in that directory first, unless the // ledger (issue #51). Takes the lock in that directory first, unless the
@@ -901,12 +857,8 @@ export class Library {
// actually changed. // actually changed.
let changed = false; let changed = false;
if ( if (this.store.userID !== this.userID) {
this.store.userID !== this.userID ||
this.store.apiOrigin !== this.apiOrigin
) {
this.store.userID = this.userID; this.store.userID = this.userID;
this.store.apiOrigin = this.apiOrigin;
changed = true; changed = true;
} }
+12 -11
View File
@@ -1,12 +1,11 @@
// On-disk JSON metadata store for the local cache. // On-disk JSON metadata store for the local cache.
// //
// The store keeps one `metadata.json` file holding the account's server // The store keeps one `metadata.json` file holding the account's server
// state: the user id, the server's address for an account on a self-hosted // state: the user id, a schema version, the cursor for the incremental
// server, a schema version, the cursor for the incremental collections // collections listing, the decrypted collection and file records, and the
// listing, the decrypted collection and file records, and the collections and // collections and files the refresh could not decrypt. The whole file is read
// files the refresh could not decrypt. The whole file is read into RAM on // into RAM on load and rewritten as a whole on save; there is no partial
// load and rewritten as a whole on save; there is no partial update and no // update and no lock file. A separate refresh unit populates the store from
// lock file. A separate refresh unit populates the store from
// the server — this module only stores what it is given. // the server — this module only stores what it is given.
// //
// The file is a cache, so it is never trusted to exist or to be intact: a // The file is a cache, so it is never trusted to exist or to be intact: a
@@ -47,7 +46,6 @@ export interface Undecryptable {
interface StoredMetadata { interface StoredMetadata {
schemaVersion: number; schemaVersion: number;
userID: number; userID: number;
apiOrigin?: string;
collectionsSinceTime: Microseconds; collectionsSinceTime: Microseconds;
collections: StoredCollection[]; collections: StoredCollection[];
files: StoredFile[]; files: StoredFile[];
@@ -70,10 +68,14 @@ export class MetadataStore {
readonly path: string; readonly path: string;
readonly schemaVersion = METADATA_SCHEMA_VERSION; readonly schemaVersion = METADATA_SCHEMA_VERSION;
userID = 0; userID = 0;
// The account's `apiOrigin`, or undefined for Ente's hosted service.
apiOrigin: string | undefined;
collectionsSinceTime: Microseconds = 0; collectionsSinceTime: Microseconds = 0;
// True when `load` populated this store from a valid existing file; false
// on a first run or a missing/corrupt/wrong-version file that loaded empty.
// `Library.open` reads it to decide whether the first refresh may run in
// the background (an existing copy already serves reads) or must be awaited.
loadedFromDisk = false;
private readonly collections = new Map<number, Collection>(); private readonly collections = new Map<number, Collection>();
private readonly files = new Map<string, EnteFile>(); private readonly files = new Map<string, EnteFile>();
private undecryptable: Undecryptable[] = []; private undecryptable: Undecryptable[] = [];
@@ -100,8 +102,8 @@ export class MetadataStore {
if (parsed.schemaVersion !== METADATA_SCHEMA_VERSION) { if (parsed.schemaVersion !== METADATA_SCHEMA_VERSION) {
return store; return store;
} }
store.loadedFromDisk = true;
store.userID = parsed.userID ?? 0; store.userID = parsed.userID ?? 0;
store.apiOrigin = parsed.apiOrigin;
store.collectionsSinceTime = parsed.collectionsSinceTime ?? 0; store.collectionsSinceTime = parsed.collectionsSinceTime ?? 0;
for (const stored of parsed.collections ?? []) { for (const stored of parsed.collections ?? []) {
const collection: Collection = { const collection: Collection = {
@@ -141,7 +143,6 @@ export class MetadataStore {
const model: StoredMetadata = { const model: StoredMetadata = {
schemaVersion: METADATA_SCHEMA_VERSION, schemaVersion: METADATA_SCHEMA_VERSION,
userID: this.userID, userID: this.userID,
apiOrigin: this.apiOrigin,
collectionsSinceTime: this.collectionsSinceTime, collectionsSinceTime: this.collectionsSinceTime,
collections: [...this.collections.values()].map((c) => ({ collections: [...this.collections.values()].map((c) => ({
...c, ...c,
+4 -25
View File
@@ -4,12 +4,11 @@
* each with the callbacks that log its requests and retries, for `quak files` * each with the callbacks that log its requests and retries, for `quak files`
* taking its collection ID as an optional argument, for `quak get` and * taking its collection ID as an optional argument, for `quak get` and
* `quak get-thumb` refusing `--collection`, and for `quak login` taking * `quak get-thumb` refusing `--collection`, and for `quak login` taking
* `--server` and sending its first request to the server it names. * `--server`.
* *
* Each test runs `bin/quak.ts`, as the smoke test does, with its session loader * Each test runs `bin/quak.ts`, as the smoke test does, with its session loader
* replaced by one that records the options it is given and reports no session, * replaced by one that records the options it is given and reports no session,
* so the command stops before it makes any request. `quak login` reads no * so the command stops before it makes any request.
* session, so its test stubs `fetch` instead and records each request.
* *
* The first load of the sources `bin/quak.ts` imports compiles them, and the * The first load of the sources `bin/quak.ts` imports compiles them, and the
* first `init()` starts libsodium. On a loaded host that takes longer than a * first `init()` starts libsodium. On a loaded host that takes longer than a
@@ -60,8 +59,6 @@ afterEach(() => {
process.argv = argv; process.argv = argv;
loaded.length = 0; loaded.length = 0;
vi.restoreAllMocks(); vi.restoreAllMocks();
vi.unstubAllGlobals();
vi.unstubAllEnvs();
}); });
afterAll(() => { afterAll(() => {
@@ -122,30 +119,12 @@ describe("bin/quak.ts get and get-thumb", () => {
); );
}); });
// The command refuses an address that is not a URL before it asks for anything
// or sends any request.
describe("bin/quak.ts login", () => { describe("bin/quak.ts login", () => {
// The command refuses an address that is not a URL before it asks for
// anything or sends any request.
it("hands --server to the command", async () => { it("hands --server to the command", async () => {
expect(await quak("login", "--server", "ente.example.com")).toMatch( expect(await quak("login", "--server", "ente.example.com")).toMatch(
/^Invalid server address "ente\.example\.com"/, /^Invalid server address "ente\.example\.com"/,
); );
}); });
// `fetch` answers 404, so the login fails at its first request. A login
// that succeeded would save its session in this user's real data
// directory, since env-paths is not mocked here.
it("sends its first request to the server --server names", async () => {
const requests: string[] = [];
vi.stubGlobal("fetch", async (input: string) => {
const { origin, pathname } = new URL(input);
requests.push(origin + pathname);
return new Response("not found", { status: 404 });
});
vi.stubEnv("QUAK_EMAIL", "login@example.com");
vi.stubEnv("QUAK_PASSWORD", "password");
await quak("login", "--server", "https://ente.example.com");
expect(requests).toEqual([
"https://ente.example.com/users/srp/attributes",
]);
});
}); });
+9 -158
View File
@@ -445,21 +445,6 @@ describe("login", () => {
); );
}, },
); );
it("refuses an empty QUAK_SERVER before asking for anything", async () => {
vi.stubEnv("QUAK_SERVER", "");
const ctx = loginContext(async () => {
throw new Error("login must not be called");
});
expect(await loginCommand(ctx, {})).toBe(1);
expect(prompts).toEqual([]);
expect(stderr.text).toBe(
`Invalid server address "": it must be an http or https URL ` +
`with no query or fragment\n`,
);
expect(existsSync(join(ctx.sessionDir, "session.json"))).toBe(false);
});
}); });
// These use a real client read from the session file, over a fake API that // These use a real client read from the session file, over a fake API that
@@ -534,16 +519,6 @@ describe("logout", () => {
); );
}); });
it("names a self-hosted account's default cache directory, under its server's address", async () => {
const ctx = { ...logoutContext(200), cacheDir: undefined };
const apiOrigin = "https://ente.example.com";
saveSession(ctx.sessionDir, { ...snapshot, apiOrigin });
expect(await logoutCommand(ctx)).toBe(0);
expect(stderr.text).toContain(
`Cache directory ${defaultCacheDirectory(USER_ID, apiOrigin)} still holds decrypted data`,
);
});
it("without a session says so, calls nothing and exits 0", async () => { it("without a session says so, calls nothing and exits 0", async () => {
expect(await logoutCommand(logoutContext(200))).toBe(0); expect(await logoutCommand(logoutContext(200))).toBe(0);
expect(requests).toHaveLength(0); expect(requests).toHaveLength(0);
@@ -1701,19 +1676,18 @@ describe("an account with many albums", () => {
// encrypted as the server stores it. The original's bytes are the title. // encrypted as the server stores it. The original's bytes are the title.
// These tests fetch no thumbnail. The server records a thumbnail size, so // These tests fetch no thumbnail. The server records a thumbnail size, so
// `helper fix-missing-thumbnails` downloads the original before it skips // `helper fix-missing-thumbnails` downloads the original before it skips
// the photo as not a JPEG. `albumKey` is that album's key, by default the // the photo as not a JPEG.
// one in `albumKeys`. const photoRow = (photo: {
const photoRow = (
photo: {
id: number; id: number;
collectionID: number; collectionID: number;
title: string; title: string;
updationTime: number; updationTime: number;
}, }): { row: Record<string, unknown>; original: Uint8Array } => {
albumKey = albumKeys.get(photo.collectionID)!,
): { row: Record<string, unknown>; original: Uint8Array } => {
const photoKey = sodium.crypto_secretstream_xchacha20poly1305_keygen(); const photoKey = sodium.crypto_secretstream_xchacha20poly1305_keygen();
const photoKeyBox = encryptSecretbox(photoKey, albumKey); const photoKeyBox = encryptSecretbox(
photoKey,
albumKeys.get(photo.collectionID)!,
);
const metadata = encryptStreamBlob( const metadata = encryptStreamBlob(
new TextEncoder().encode( new TextEncoder().encode(
JSON.stringify({ JSON.stringify({
@@ -1826,77 +1800,6 @@ describe("an account with many albums", () => {
}; };
}; };
// The albums in the cache's `metadata.json`. A search on an empty cache
// stores none there, only the account whose original or thumbnail it
// stored.
const cachedAlbums = (): unknown[] =>
JSON.parse(readFileSync(join(root, "cache", "metadata.json"), "utf-8"))
.collections;
// A context whose login and saved session send each request to the server
// its host names: `ente.example.com` is the fake server above, and
// `photos.example.org` a second one. Every account the fake server logs in
// has user ID 42, so the second server's account has the first one's user
// ID. Its one album holds photo 200, `other-200.jpg`; the first server
// gives ID 200 to `photo-2.jpg`.
const twoServerContext = async (): Promise<CliContext> => {
const account = await buildAccount(PASSWORD);
const albumKey = sodium.crypto_secretbox_keygen();
const albumKeyBox = encryptSecretbox(albumKey, account.masterKey);
const name = encryptSecretbox(
new TextEncoder().encode("Other album"),
albumKey,
);
const { row, original } = photoRow(
{
id: 200,
collectionID: 1,
title: "other-200.jpg",
updationTime: 1,
},
albumKey,
);
const other = buildMetaFetch({
verifier: account.verifier,
srpAttributes: account.srpAttributes,
keyAttributes: account.keyAttributes,
encryptedToken: account.encryptedToken,
collections: [
{
id: 1,
owner: { id: USER_ID },
encryptedKey: toBase64(albumKeyBox.ciphertext),
keyDecryptionNonce: toBase64(albumKeyBox.nonce),
encryptedName: toBase64(name.ciphertext),
nameDecryptionNonce: toBase64(name.nonce),
type: "album",
updationTime: 1,
},
],
filesByCollection: { 1: [row] },
encryptedMLData: {},
fileCiphertexts: { 200: original },
});
const fetch = (async (input: RequestInfo | URL, init?: RequestInit) => {
const { host } = new URL(
input instanceof Request ? input.url : input,
);
return (host === "photos.example.org" ? other : server)(
input,
init,
);
}) as typeof globalThis.fetch;
return {
...context(),
login: (opts) =>
Client.login({
...opts,
apiOptions: { ...opts.apiOptions, fetch },
}),
loadSession: (path) => loadSession(path, { fetch }),
};
};
it("collections lists every album from one request, fetching no album's files", async () => { it("collections lists every album from one request, fetching no album's files", async () => {
const ctx = accountContext(); const ctx = accountContext();
expect(await collectionsCommand(ctx, {})).toBe(0); expect(await collectionsCommand(ctx, {})).toBe(0);
@@ -2003,7 +1906,7 @@ describe("an account with many albums", () => {
"/collections/v2/diff?collectionID=2&sinceTime=0", "/collections/v2/diff?collectionID=2&sinceTime=0",
"/?fileID=200", "/?fileID=200",
]); ]);
expect(cachedAlbums()).toEqual([]); expect(existsSync(join(root, "cache", "metadata.json"))).toBe(false);
}); });
// A self-hosted server serves downloads under its own address, at // A self-hosted server serves downloads under its own address, at
@@ -2047,58 +1950,6 @@ describe("an account with many albums", () => {
expect([...hosts]).toEqual(["ente.example.com"]); expect([...hosts]).toEqual(["ente.example.com"]);
}); });
// The two servers' accounts share the cache directory `--cache-dir` names.
it("after a login to another server whose account has the same user ID, files and get see none of the first server's albums and originals", async () => {
const ctx = await twoServerContext();
const first = join(root, "first.jpg");
const second = join(root, "second.jpg");
vi.stubEnv("QUAK_EMAIL", "many@example.com");
vi.stubEnv("QUAK_PASSWORD", PASSWORD);
vi.stubEnv("QUAK_SERVER", undefined);
try {
const opts = { server: "https://ente.example.com" };
expect(await loginCommand(ctx, opts)).toBe(0);
expect(await filesCommand(ctx, undefined, {})).toBe(0);
expect(await getCommand(ctx, "200", { out: first })).toBe(0);
const otherOpts = { server: "https://photos.example.org" };
expect(await loginCommand(ctx, otherOpts)).toBe(0);
stdout.text = "";
expect(await filesCommand(ctx, undefined, {})).toBe(0);
expect(await getCommand(ctx, "200", { out: second })).toBe(0);
} finally {
vi.unstubAllEnvs();
}
expect(stdout.text).toBe("200\timage\tother-200.jpg\n");
expect(readFileSync(first, "utf-8")).toBe("photo-2.jpg");
expect(readFileSync(second, "utf-8")).toBe("other-200.jpg");
});
// `get` on an empty cache stores the original but no album.
it("get on an empty cache, after the same on another server whose account has the same user ID, saves this server's photo", async () => {
const ctx = await twoServerContext();
const first = join(root, "first.jpg");
const second = join(root, "second.jpg");
vi.stubEnv("QUAK_EMAIL", "many@example.com");
vi.stubEnv("QUAK_PASSWORD", PASSWORD);
vi.stubEnv("QUAK_SERVER", undefined);
try {
const opts = { server: "https://ente.example.com" };
expect(await loginCommand(ctx, opts)).toBe(0);
expect(await getCommand(ctx, "200", { out: first })).toBe(0);
const otherOpts = { server: "https://photos.example.org" };
expect(await loginCommand(ctx, otherOpts)).toBe(0);
expect(await getCommand(ctx, "200", { out: second })).toBe(0);
} finally {
vi.unstubAllEnvs();
}
expect(readFileSync(first, "utf-8")).toBe("photo-2.jpg");
expect(readFileSync(second, "utf-8")).toBe("other-200.jpg");
});
// `files` first fills the cache, so `get` waits for a refresh. No album has // `files` first fills the cache, so `get` waits for a refresh. No album has
// changed since, so the refresh asks for no album's files. // changed since, so the refresh asks for no album's files.
it("get on a cache that holds the account asks for no unchanged album's files, also for a photo in no album", async () => { it("get on a cache that holds the account asks for no unchanged album's files, also for a photo in no album", async () => {
@@ -2154,7 +2005,7 @@ describe("an account with many albums", () => {
"/collections/v2/diff?collectionID=39&sinceTime=0", "/collections/v2/diff?collectionID=39&sinceTime=0",
"/?fileID=3900", "/?fileID=3900",
]); ]);
expect(cachedAlbums()).toEqual([]); expect(existsSync(join(root, "cache", "metadata.json"))).toBe(false);
}); });
// `files` first fills the cache, so the helper waits for a refresh. No // `files` first fills the cache, so the helper waits for a refresh. No
+2 -19
View File
@@ -55,25 +55,10 @@ describe("Client.toJSON", () => {
expect(Client.fromJSON(snapshot).toJSON()).toEqual(snapshot); expect(Client.fromJSON(snapshot).toJSON()).toEqual(snapshot);
}); });
it("names the server apiOptions gave fromJSON", async () => { it("names the server apiOptions gave fromJSON", () => {
const apiOrigin = "https://ente.example.com"; const apiOrigin = "https://ente.example.com";
const urls: string[] = []; const client = Client.fromJSON(validSnapshot(), { apiOrigin });
const client = Client.fromJSON(validSnapshot(), {
apiOrigin,
fetch: async (input) => {
urls.push(String(input));
return new Response(JSON.stringify({ collections: [] }), {
status: 200,
headers: { "content-type": "application/json" },
});
},
});
expect(client.toJSON().apiOrigin).toBe(apiOrigin); expect(client.toJSON().apiOrigin).toBe(apiOrigin);
await client.listCollections();
expect(urls).toEqual([
"https://ente.example.com/collections/v2?sinceTime=0",
]);
}); });
it("throws instead of emitting a snapshot without a token", () => { it("throws instead of emitting a snapshot without a token", () => {
@@ -108,8 +93,6 @@ describe("Client.fromJSON", () => {
["publicKey", shortKey], ["publicKey", shortKey],
["apiOrigin", 7], ["apiOrigin", 7],
["apiOrigin", ""], ["apiOrigin", ""],
["apiOrigin", "ente.example.com"],
["apiOrigin", "ftp://ente.example.com"],
])("rejects %s = %j, naming the field", (field, value) => { ])("rejects %s = %j, naming the field", (field, value) => {
const snapshot: Record<string, unknown> = { ...validSnapshot() }; const snapshot: Record<string, unknown> = { ...validSnapshot() };
snapshot[field] = value; snapshot[field] = value;
+6 -218
View File
@@ -19,22 +19,16 @@
* refresh fails (offline start from cache). * refresh fails (offline start from cache).
* 5. `close()` stops the timer and is idempotent, and its promise resolves * 5. `close()` stops the timer and is idempotent, and its promise resolves
* only once an in-flight refresh has written the cache file. * only once an in-flight refresh has written the cache file.
* 6. `cacheDirectory` defaults to the env-paths cache dir plus the user id, * 6. `cacheDirectory` defaults to the env-paths cache dir plus the user id.
* with a self-hosted server's address before the user id. * 7. `open()` branches on the cache: an empty cache awaits the first refresh
* 7. `open()` branches on the cache: a cache that holds no collection awaits * (it has nothing to serve yet); an existing cache serves its copy at once
* the first refresh (it has nothing to serve yet); an existing cache serves * and refreshes in the background, so a slow or dead server never stalls
* its copy at once and refreshes in the background, so a slow or dead * opening.
* server never stalls opening.
* 8. A save failure that leaves RAM ahead of disk keeps `status().lastError` * 8. A save failure that leaves RAM ahead of disk keeps `status().lastError`
* set and keeps retrying the write; a later empty refresh does not clear it. * set and keeps retrying the write; a later empty refresh does not clear it.
* 9. With `refreshOnOpen: false`, `open()` starts no refresh at all, and a * 9. With `refreshOnOpen: false`, `open()` starts no refresh at all, and a
* refresh reports through `onProgress` each collection whose files it * refresh reports through `onProgress` each collection whose files it
* fetched. * fetched.
* 10. A cache another account filled starts empty. One filled from another
* server also loses its originals and thumbnails, since two servers can
* give two files the same ID. `thumbnailOf` on an empty cache saves the
* account's user ID and server. A refresh saves the account's server, so a
* self-hosted account keeps its own cache from one `open()` to the next.
* *
* The client is a mock: no crypto, no network. It serves scripted pages and * The client is a mock: no crypto, no network. It serves scripted pages and
* records the `sinceTime` each call carried so cursor threading is provable. * records the `sinceTime` each call carried so cursor threading is provable.
@@ -55,13 +49,7 @@
*/ */
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest"; import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
import { import { mkdtempSync, rmSync } from "node:fs";
existsSync,
mkdirSync,
mkdtempSync,
rmSync,
writeFileSync,
} from "node:fs";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import { join } from "node:path"; import { join } from "node:path";
import envPaths from "env-paths"; import envPaths from "env-paths";
@@ -119,7 +107,6 @@ const file = (
*/ */
class MockClient { class MockClient {
userID = USER_ID; userID = USER_ID;
apiOrigin?: string;
failCollections = false; failCollections = false;
collectionsQueue: CollectionsPage[] = []; collectionsQueue: CollectionsPage[] = [];
filesByCollection = new Map<number, FilesPage[]>(); filesByCollection = new Map<number, FilesPage[]>();
@@ -597,44 +584,6 @@ describe("Library.open and background refresh", () => {
} }
}); });
// `originalOf` on an empty cache leaves such a cache behind: it names the
// account before it stores the original.
it("awaits the first refresh on a cache that names the account but holds no collection", async () => {
const seed = await MetadataStore.load(
join(cacheDirectory, "metadata.json"),
);
seed.userID = USER_ID;
await seed.save();
let releaseFirstFetch: (page: CollectionsPage) => void = () => {};
const gate = new Promise<CollectionsPage>((resolve) => {
releaseFirstFetch = resolve;
});
const client = new MockClient();
client.collectionsSince = async () => gate;
let opened = false;
const openPromise = Library.open({ client, cacheDirectory }).then(
(l) => {
opened = true;
return l;
},
);
await new Promise((r) => setTimeout(r, 20));
expect(opened).toBe(false);
releaseFirstFetch({
collections: [collection(1, 100)],
deleted: [],
cursor: 100,
});
const lib = await openPromise;
try {
expect(lib.listCollections().map((c) => c.id)).toEqual([1]);
} finally {
await lib.close();
}
});
// A caller that asks for each refresh it needs, as the CLI's read commands // A caller that asks for each refresh it needs, as the CLI's read commands
// do, opens without one: on an empty cache the refresh `open()` would // do, opens without one: on an empty cache the refresh `open()` would
// await fetches the files of every collection, which `quak collections` // await fetches the files of every collection, which `quak collections`
@@ -893,165 +842,4 @@ describe("Library.open and background refresh", () => {
else process.env.XDG_CACHE_HOME = prev; else process.env.XDG_CACHE_HOME = prev;
} }
}); });
it("puts a self-hosted server's address, URL-encoded, before the user id in the default cacheDirectory", async () => {
const xdg = join(dir, "xdg-cache");
const prev = process.env.XDG_CACHE_HOME;
process.env.XDG_CACHE_HOME = xdg;
try {
const client = new MockClient();
client.apiOrigin = "https://ente.example.com:8443/api";
const lib = await Library.open({ client });
try {
expect(lib.cacheDirectory).toBe(
join(
envPaths("quak", { suffix: "" }).cache,
"https%3A%2F%2Fente.example.com%3A8443%2Fapi",
String(USER_ID),
),
);
expect(lib.cacheDirectory.startsWith(xdg)).toBe(true);
} finally {
await lib.close();
}
} finally {
if (prev === undefined) delete process.env.XDG_CACHE_HOME;
else process.env.XDG_CACHE_HOME = prev;
}
});
// A cache that account USER_ID on `apiOrigin` filled: its records, and an
// original and a thumbnail of file 1001.
const fillCache = async (apiOrigin?: string): Promise<void> => {
const seed = await MetadataStore.load(
join(cacheDirectory, "metadata.json"),
);
seed.userID = USER_ID;
seed.apiOrigin = apiOrigin;
seed.collectionsSinceTime = 500;
seed.putCollection(collection(1, 400));
seed.putFile(file(1001, 1, 400));
await seed.save();
for (const name of ["originals", "thumbnails"]) {
mkdirSync(join(cacheDirectory, name));
writeFileSync(join(cacheDirectory, name, "1001.jpg"), "1001");
}
};
it("deletes the whole cache an account on another server filled", async () => {
await fillCache("https://ente.example.com");
const client = new MockClient();
client.apiOrigin = "https://photos.example.org";
const lib = await Library.open({
client,
cacheDirectory,
refreshIntervalSeconds: 3600,
});
try {
expect(client.collectionsSinceTimes).toEqual([0]);
expect(lib.listCollections()).toEqual([]);
for (const name of ["originals", "thumbnails"]) {
expect(existsSync(join(cacheDirectory, name, "1001.jpg"))).toBe(
false,
);
}
} finally {
await lib.close();
}
});
// A server gives each file its own ID, so another account's cached file
// is never served for this account's.
it("keeps the originals and thumbnails another account on the same server cached", async () => {
await fillCache();
const client = new MockClient();
client.userID = USER_ID + 1;
const lib = await Library.open({
client,
cacheDirectory,
refreshIntervalSeconds: 3600,
});
try {
expect(client.collectionsSinceTimes).toEqual([0]);
expect(lib.listCollections()).toEqual([]);
for (const name of ["originals", "thumbnails"]) {
expect(existsSync(join(cacheDirectory, name, "1001.jpg"))).toBe(
true,
);
}
} finally {
await lib.close();
}
});
// `quak get-thumb` on an empty cache stores a thumbnail before any refresh
// runs. `open()` for an account on another server deletes that thumbnail
// only if the cache names the account that stored it.
it("thumbnailOf on an empty cache saves the account's user ID and server in metadata.json", async () => {
const client = new MockClient();
client.apiOrigin = "https://ente.example.com";
const lib = await Library.open({
client,
cacheDirectory,
refreshOnOpen: false,
refreshIntervalSeconds: Infinity,
contentSource: {
original: async () => {
throw new Error("original not expected");
},
thumbnail: async ({ destination }) => {
writeFileSync(destination, "thumb");
return { bytesWritten: 5 };
},
},
});
try {
await lib.thumbnailOf(file(1001, 1, 400));
} finally {
await lib.close();
}
const saved = await MetadataStore.load(
join(cacheDirectory, "metadata.json"),
);
expect(saved.userID).toBe(USER_ID);
expect(saved.apiOrigin).toBe("https://ente.example.com");
});
// A refresh saves the account's server in `metadata.json`. Without it,
// `open()` would take a self-hosted account's own cache for another
// server's and delete it.
it("keeps a self-hosted account's own cache from one open to the next", async () => {
const client = new MockClient();
client.apiOrigin = "https://ente.example.com";
client.collectionsQueue.push({
collections: [collection(1, 100)],
deleted: [],
cursor: 100,
});
const first = await Library.open({
client,
cacheDirectory,
refreshIntervalSeconds: Infinity,
});
await first.close();
mkdirSync(join(cacheDirectory, "originals"));
writeFileSync(join(cacheDirectory, "originals", "1001.jpg"), "1001");
const lib = await Library.open({
client,
cacheDirectory,
refreshIntervalSeconds: Infinity,
});
try {
expect(lib.listCollections().map((c) => c.id)).toEqual([1]);
expect(
existsSync(join(cacheDirectory, "originals", "1001.jpg")),
).toBe(true);
} finally {
await lib.close();
}
// The second open's refresh started from the saved cursor.
expect(client.collectionsSinceTimes).toEqual([0, 100]);
});
}); });