quak/README.md

# quack

quack is a WTFPL-licensed TypeScript client library and CLI by
[@sneak](https://sneak.berlin) for the [Ente](https://ente.io) end-to-end
encrypted photo hosting service. It logs in, enumerates collections and files,
and downloads individual images while decrypting them on the way to disk.

## Getting Started

```bash
git clone https://git.eeqj.de/sneak/quack.git
cd quack
yarn install
yarn build

# Log in (prompts for email, password, and OTP/TOTP if required).
# Stores an encrypted session under $XDG_CONFIG_HOME/quack/.
yarn quack login

# List the user's collections (albums).
yarn quack collections

# List files in a collection.
yarn quack files --collection 12345

# Download and decrypt a single file to ./out/.
yarn quack get 67890 --out ./out/
```

For library use:

```ts
import { Client } from "quack";

const client = await Client.fromSavedSession();
for (const c of await client.listCollections()) {
    console.log(c.id, c.name);
}
const file = await client.getFile(67890);
await client.downloadFile(file, "./out/");
```

## Rationale

Ente is one of very few photo services with a credible end-to-end encryption
story. The shipping clients (mobile Flutter, web React, desktop Electron, and Go
CLI) work, but they are slow, buggy, and difficult to script against. The
Flutter app fails to sync reliably. The web app is heavy. The desktop app is the
web app inside a slow Electron wrapper. The Go CLI is the closest thing to a
usable tool, but it is awkward to integrate from anything that is not a shell.

quack is the first step in fixing that. This repo ships a small, correct,
well-tested implementation of Ente's cryptographic protocol and its read-only
API surface, plus a CLI that proves the library is enough to do real work
without a UI.

The longer-term goal of this project is a simple desktop client for Ente, built
on this library in Electron (or a comparable runtime), with two priorities above
everything else: correctness and stability. Performance and simplicity follow
from those. Features will be added only after the protocol layer is correct, the
local cache is reliable, and the UI is responsive on a five-year-old laptop.

This first release is deliberately scoped to read operations: log in, walk the
account, decrypt and save files. Upload, sharing, deletion, and bidirectional
sync are out of scope. Adding them later is straightforward; doing them right
requires the protocol layer to be correct first.

## Design

quack is a TypeScript library with a thin CLI wrapper. The library does the
work; the CLI is for humans.

### Layout

```
quack/
    src/
        crypto/         libsodium primitives (boxes, secretstreams, KDF, SRP)
        api/            HTTP client + typed endpoint wrappers
        auth/           login flow (SRP + email OTP + TOTP), key unwrap
        model/          decrypted Collection, File, Metadata types
        session/        on-disk session persistence (token + master key)
        client.ts       high-level Client class assembled from the above
        index.ts        public library exports
    bin/
        quack.ts        CLI entrypoint (commander.js)
    test/               unit + integration tests (vitest)
    Makefile
    Dockerfile
    package.json
    tsconfig.json
```

### Cryptography

All cryptography is done by `libsodium-wrappers-sumo` (the "sumo" build is
required for `crypto_pwhash` / Argon2id). No hand-rolled crypto.

The key hierarchy, derived during login, is:

1. The user enters their password.
2. Argon2id (`crypto_pwhash`) over the password and a server-issued `kekSalt`,
   with server-issued `memLimit` and `opsLimit`, produces a 32-byte Key
   Encryption Key (KEK).
3. SRP login: a 16-byte SRP login subkey is derived from the KEK using
   `crypto_kdf_derive_from_key` (BLAKE2b) with subkey id 1 and context
   `loginctx`. That 16-byte value is the SRP password.
4. After SRP completes (or after email-OTP fallback), the server returns a blob
   of "key attributes" plus an encrypted auth token.
5. `crypto_secretbox_open_easy` over the encrypted master key with the KEK
   yields the 32-byte master key.
6. `crypto_secretbox_open_easy` over the encrypted secret key with the master
   key yields the user's X25519 private key. The matching public key is
   delivered in cleartext.
7. `crypto_box_seal_open` over the encrypted token with the user's keypair
   yields the URL-safe base64 auth token used in `X-Auth-Token` for all
   subsequent calls.

Per-collection keys are decrypted with `crypto_secretbox_open_easy` using the
master key (for owned collections). Per-file keys are decrypted with
`crypto_secretbox_open_easy` using the collection key. File metadata is a
secretbox under the file key. File content is a chunked
`crypto_secretstream_xchacha20poly1305` stream under the file key, with a 4 MiB
plaintext chunk size and a 17-byte authentication overhead per chunk.

### HTTP API

Production endpoints:

- API: `https://api.ente.io`
- File download CDN: `https://files.ente.io/?fileID=<id>`
- Thumbnail CDN: `https://thumbnails.ente.io/?fileID=<id>`

A custom API endpoint is configurable for self-hosted servers via the
`ENTE_API_ENDPOINT` environment variable. When set, file downloads route through
`<endpoint>/files/download/<id>` instead of the dedicated CDN host.

Required request headers on every authenticated call:

- `X-Auth-Token`: the decrypted auth token from login.
- `X-Client-Package`: identifies the client. quack uses `berlin.sneak.quack`.

Endpoints used:

- `GET /users/srp/attributes?email=<email>`: fetch SRP and KDF parameters.
- `POST /users/srp/create-session`: begin SRP handshake.
- `POST /users/srp/verify-session`: complete SRP, receive 2FA challenge or the
  encrypted token plus key attributes.
- `POST /users/ott` and `POST /users/verify-email`: email OTP fallback path.
- `POST /users/two-factor/verify`: TOTP second factor.
- `GET /collections/v2?sinceTime=<usec>`: list collections changed since
  microsecond timestamp; pass 0 for a full enumeration.
- `GET /collections/v2/diff?collectionID=<id>&sinceTime=<usec>`: list files in a
  collection; paginate while `hasMore` is true.
- `GET https://files.ente.io/?fileID=<id>`: download encrypted file bytes.

### Session persistence

After login, quack writes an encrypted session blob to
`$XDG_CONFIG_HOME/quack/session.json` (default `~/.config/quack/session.json`)
containing the auth token, the user's master key, the user's secret key, and the
user's email. The session file is itself encrypted with a key derived from a
per-machine random value stored in the OS keychain when available, falling back
to a key file at mode `0600` in the same config directory. The master key and
secret key are never written to disk in cleartext.

### CLI surface

- `quack login`: interactive login, writes session.
- `quack logout`: deletes the session.
- `quack whoami`: prints the logged-in email.
- `quack collections`: list collections (id, name, type, file count).
- `quack files --collection <id>`: list files in a collection (id, name, type,
  creation time, size).
- `quack get <fileID> --out <dir>`: download and decrypt a file.
- `quack get-thumb <fileID> --out <dir>`: download and decrypt a thumbnail.

All commands accept `--json` for machine-readable output.

## API reference

The complete public surface of the library, expressed as TypeScript
declarations. Every exported name is listed here. Anything not listed is
internal.

### Type aliases

```ts
// src/model/types.ts
export type Bytes = Uint8Array;
export type Base64 = string; // standard base64 unless noted
export type Base64URL = string; // URL-safe base64
export type Microseconds = number; // unix epoch microseconds (int64-ish)
```

### Crypto module

```ts
// src/crypto/index.ts

// Lazily initializes libsodium. Safe to call repeatedly; the first call
// performs the init, subsequent calls are no-ops.
export function init(): Promise<void>;

// Argon2id over a UTF-8 password and a 16-byte salt, producing a 32-byte
// key. memLimit is in bytes, opsLimit is the iteration count, both as
// returned by the server in SRP / key attributes.
export function deriveKEK(
    password: string,
    salt: Bytes,
    opsLimit: number,
    memLimit: number,
): Promise<Bytes>;

// crypto_kdf_derive_from_key with subkey id 1 and context "loginctx",
// returning the first 16 bytes. Used as the SRP password.
export function deriveLoginSubkey(kek: Bytes): Bytes;

// crypto_secretbox_open_easy. Returns plaintext or throws on auth failure.
export function decryptBox(ciphertext: Bytes, nonce: Bytes, key: Bytes): Bytes;

// crypto_box_seal_open. Used to recover the auth token after login.
export function decryptSealed(
    ciphertext: Bytes,
    publicKey: Bytes,
    secretKey: Bytes,
): Bytes;

// crypto_secretstream_xchacha20poly1305_init_pull. Returned state is opaque
// and threaded through pullStreamChunk.
export function initStreamPull(header: Bytes, key: Bytes): StreamPullState;

// crypto_secretstream_xchacha20poly1305_pull. Tag values follow libsodium's
// constants: 0=MESSAGE, 1=PUSH, 2=REKEY, 3=FINAL.
export function pullStreamChunk(
    state: StreamPullState,
    ciphertext: Bytes,
): { plaintext: Bytes; tag: number };

// Convenience helpers. fromBase64 accepts both standard and URL-safe.
export function fromBase64(s: Base64 | Base64URL): Bytes;
export function toBase64(b: Bytes): Base64;
export function toBase64URL(b: Bytes): Base64URL;

// Plaintext chunk size used by Ente for file content streams.
export const STREAM_CHUNK_SIZE: number; // 4 * 1024 * 1024

// Encrypted-chunk overhead: secretstream auth tag (16) + tag byte (1).
export const STREAM_CHUNK_OVERHEAD: number; // 17

export interface StreamPullState {
    /* opaque */
}
```

### Auth module

```ts
// src/auth/types.ts

export interface KDFParams {
    kekSalt: Base64;
    memLimit: number;
    opsLimit: number;
}

export interface KeyAttributes {
    kekSalt: Base64;
    encryptedKey: Base64;
    keyDecryptionNonce: Base64;
    publicKey: Base64;
    encryptedSecretKey: Base64;
    secretKeyDecryptionNonce: Base64;
    memLimit: number;
    opsLimit: number;
    masterKeyEncryptedWithRecoveryKey?: Base64;
    masterKeyDecryptionNonce?: Base64;
    recoveryKeyEncryptedWithMasterKey?: Base64;
    recoveryKeyDecryptionNonce?: Base64;
}

export interface SRPAttributes {
    srpUserID: string;
    srpSalt: Base64;
    memLimit: number;
    opsLimit: number;
    kekSalt: Base64;
    isEmailMFAEnabled: boolean;
}

export interface AuthorizationResponse {
    id: number; // user ID
    keyAttributes?: KeyAttributes;
    encryptedToken?: Base64URL; // sealed-box-encrypted to user's pubkey
    twoFactorSessionID?: string;
    passkeySessionID?: string;
}

export type LoginChallenge =
    | { kind: "complete"; response: AuthorizationResponse }
    | { kind: "totp"; sessionID: string }
    | { kind: "passkey"; sessionID: string }
    | { kind: "emailOTP" };
```

```ts
// src/auth/index.ts

// Begin login. Returns a challenge that tells the caller what to do next:
// supply a TOTP code, supply an email OTP, follow a passkey URL, or stop
// because login is already complete.
export function beginLogin(
    api: ApiClient,
    email: string,
    password: string,
): Promise<LoginChallenge>;

// Submit a TOTP code from an authenticator app. Returns the final
// AuthorizationResponse on success.
export function submitTOTP(
    api: ApiClient,
    sessionID: string,
    code: string,
): Promise<AuthorizationResponse>;

// Request and submit an email-delivered one-time code. Two calls, because
// the first triggers email delivery and the second verifies it.
export function requestEmailOTP(api: ApiClient, email: string): Promise<void>;
export function submitEmailOTP(
    api: ApiClient,
    email: string,
    code: string,
): Promise<AuthorizationResponse>;

// Given an AuthorizationResponse and the user's password, decrypt the master
// key, secret key, and auth token. Throws on bad password or tampered data.
export function unwrapAuth(
    response: AuthorizationResponse,
    password: string,
): Promise<{
    masterKey: Bytes;
    secretKey: Bytes;
    publicKey: Bytes;
    token: string; // base64 URL-safe; goes into X-Auth-Token
}>;
```

### Model module

```ts
// src/model/index.ts

export type CollectionType =
    | "album"
    | "folder"
    | "favorites"
    | "uncategorized"
    | "unknown";

export interface Collection {
    id: number;
    ownerID: number;
    key: Bytes; // decrypted
    name: string; // decrypted
    type: CollectionType;
    updationTime: Microseconds;
    isShared: boolean; // true if owner != current user
}

export type FileType = "image" | "video" | "livePhoto" | "unknown";

export interface FileMetadata {
    title: string;
    fileType: FileType;
    creationTime: Microseconds;
    modificationTime: Microseconds;
    latitude?: number;
    longitude?: number;
    hash?: string; // base64 of file SHA256
}

export interface FileBlob {
    decryptionHeader: Base64;
    size?: number; // size of the encrypted body, if known from server
}

export interface EnteFile {
    id: number;
    collectionID: number;
    ownerID: number;
    key: Bytes; // decrypted file key
    metadata: FileMetadata;
    file: FileBlob;
    thumbnail: FileBlob;
    updationTime: Microseconds;
}
```

### HTTP client

```ts
// src/api/client.ts

export interface ApiClientOptions {
    apiOrigin?: string; // default https://api.ente.io
    filesOrigin?: string; // default https://files.ente.io
    thumbsOrigin?: string; // default https://thumbnails.ente.io
    authToken?: string;
    fetch?: typeof fetch; // injectable for tests
    userAgent?: string; // default "quack/<version>"
}

export class ApiError extends Error {
    readonly status: number;
    readonly code?: string;
    readonly requestID?: string;
    readonly body?: unknown;
}

export class ApiClient {
    constructor(opts?: ApiClientOptions);
    setAuthToken(token: string): void;
    clearAuthToken(): void;

    getJSON<T>(
        path: string,
        query?: Record<string, string | number | undefined>,
    ): Promise<T>;
    postJSON<T>(path: string, body: unknown): Promise<T>;

    // Streaming download from the file CDN. Caller is responsible for
    // consuming the stream.
    getFileStream(fileID: number): Promise<ReadableStream<Uint8Array>>;
    getThumbnailStream(fileID: number): Promise<ReadableStream<Uint8Array>>;
}
```

### Session

```ts
// src/session/index.ts

export interface Session {
    email: string;
    userID: number;
    token: string; // base64 URL-safe
    masterKey: Bytes; // 32 bytes, never serialized in cleartext
    secretKey: Bytes; // 32 bytes, never serialized in cleartext
    publicKey: Bytes; // 32 bytes
}

export interface SessionStoreOptions {
    path?: string; // default $XDG_CONFIG_HOME/quack/session.json
    keychainService?: string; // default "berlin.sneak.quack"
}

export class SessionStore {
    constructor(opts?: SessionStoreOptions);
    load(): Promise<Session | null>;
    save(s: Session): Promise<void>;
    clear(): Promise<void>;
}
```

### Client

```ts
// src/client.ts

export interface LoginPrompt {
    password: () => Promise<string>;
    emailOTP?: () => Promise<string>; // called when account uses email OTP
    totp?: () => Promise<string>; // called when TOTP is required
}

export interface ClientOptions extends ApiClientOptions {
    sessionStore?: SessionStore;
}

export interface DownloadResult {
    path: string;
    bytesWritten: number;
}

export class Client {
    // Static constructors. Both end with a Client that has a populated
    // session and a ready ApiClient.
    static login(
        email: string,
        prompt: LoginPrompt,
        opts?: ClientOptions,
    ): Promise<Client>;
    static fromSavedSession(opts?: ClientOptions): Promise<Client>;

    readonly api: ApiClient;
    readonly session: Readonly<Session>;

    // Account.
    whoami(): { email: string; userID: number };
    saveSession(): Promise<void>;
    logout(): Promise<void>; // clears session on disk and in memory

    // Collections.
    listCollections(opts?: { sinceTime?: Microseconds }): Promise<Collection[]>;
    getCollection(id: number): Promise<Collection>;

    // Files.
    listFiles(
        collectionID: number,
        opts?: { sinceTime?: Microseconds },
    ): Promise<EnteFile[]>;
    getFile(fileID: number): Promise<EnteFile>;

    // Downloads. If outPath is omitted, a path is constructed from the
    // decrypted metadata title in the current working directory. If outPath
    // is a directory, the filename is taken from the metadata title. If
    // outPath is a file path, that file is written.
    downloadFile(
        file: EnteFile | number,
        outPath?: string,
    ): Promise<DownloadResult>;
    downloadThumbnail(
        file: EnteFile | number,
        outPath?: string,
    ): Promise<DownloadResult>;
}
```

### Public exports (`src/index.ts`)

```ts
export { Client } from "./client";
export { ApiClient, ApiError } from "./api/client";
export { SessionStore } from "./session";
export * from "./model";
export type {
    Session,
    LoginPrompt,
    ClientOptions,
    DownloadResult,
} from "./client";
```

## TODO

Phase 1: scaffolding (this commit and the next)

- [x] `git init`, write README
- [ ] Create `initial-scaffolding` feature branch
- [ ] Add `LICENSE` (WTFPL), `REPO_POLICIES.md`, `.gitignore`, `.editorconfig`,
      `.prettierrc`, `.prettierignore`, `.dockerignore`
- [ ] Add `Makefile` with `test`, `lint`, `fmt`, `fmt-check`, `check`, `docker`,
      `hooks`, plus `build`, `dev`, `clean`
- [ ] Add `Dockerfile` running `make check` against pinned node image
- [ ] Add `.gitea/workflows/check.yml` running `docker build .`
- [ ] Add `package.json`, `tsconfig.json`, install pinned versions of
      `typescript`, `libsodium-wrappers-sumo`, `secure-remote-password`,
      `commander`, `vitest`, `prettier`, `eslint`, `@types/node`
- [ ] Smoke test: `make check` and `make docker` both pass

Phase 2: crypto primitives

- [ ] Wrap libsodium init as an awaitable singleton
- [ ] `deriveKEK(password, kekSalt, memLimit, opsLimit)` (Argon2id)
- [ ] `deriveLoginSubkey(kek)` (KDF with subkey id 1, context `loginctx`, 16
      bytes)
- [ ] `decryptBox(ciphertext, nonce, key)` for secretbox
- [ ] `decryptSealed(ciphertext, publicKey, secretKey)` for sealed box
- [ ] `initStreamPull` and `pullStreamChunk` for chunked secretstream (4 MiB
      plaintext chunks, 17-byte overhead)
- [ ] Round-trip tests against vectors generated by libsodium directly

Phase 3: SRP + auth

- [ ] SRP-6a client using `secure-remote-password` with the same group as the
      server
- [ ] `beginLogin(email, password)` returning a `LoginChallenge`
- [ ] `requestEmailOTP` and `submitEmailOTP` for accounts without SRP
- [ ] `submitTOTP(sessionID, code)`
- [ ] `unwrapAuth(response, password)` returning master key, secret key, public
      key, and decrypted token
- [ ] Tests against recorded HTTP fixtures

Phase 4: HTTP client + endpoints

- [ ] `ApiClient` that attaches `X-Auth-Token` and `X-Client-Package`
- [ ] Typed wrappers for the endpoints listed above
- [ ] Retry policy: no retry on 4xx, exponential backoff on 5xx and network
      errors
- [ ] `ApiError` that surfaces the server's error code and request id

Phase 5: collections and files

- [ ] `listCollections()` paginating on `sinceTime` until empty
- [ ] Decrypt per-collection key with master key
- [ ] Decrypt collection name with collection key
- [ ] `listFiles(collectionID)` paginating on `sinceTime` while `hasMore`
- [ ] Decrypt per-file key with collection key
- [ ] Decrypt file metadata blob with file key, expose typed `FileMetadata`

Phase 6: download

- [ ] `downloadFile(fileID, outPath)` streams the encrypted body, decrypts it
      chunk by chunk, writes plaintext to `outPath`. Resolves the filename from
      the decrypted metadata title when no `outPath` is supplied.
- [ ] `downloadThumbnail(fileID, outPath)` for the thumbnail CDN
- [ ] Live integration test against a throwaway Ente account if one is available

Phase 7: session persistence

- [ ] `SessionStore` writing an encrypted session blob with a key from the OS
      keychain (`keytar`) or a `0600` keyfile fallback
- [ ] `Client.fromSavedSession()` and `Client.saveSession()`
- [ ] `quack logout` deletes the session and the keychain entry

Phase 8: CLI

- [ ] `commander`-based CLI that matches the surface in the Design section
- [ ] `--json` output for every command
- [ ] Reasonable progress output for long downloads (only when stdout is a TTY)

Phase 9: docs and 1.0

- [ ] README usage examples for both library and CLI verified by hand
- [ ] All TODO items above checked
- [ ] Tag `v1.0.0`

Phase 10 and beyond: desktop client (separate repo)

- [ ] Spike Electron app skeleton consuming this library
- [ ] Local cache (SQLite) keyed on `(collectionID, fileID, updationTime)`
- [ ] Background sync worker that streams new files into the cache
- [ ] Read-only gallery UI: thumbnails, full-image view, basic search
- [ ] Add upload, delete, and share back into the library before the desktop UI
      exposes them

## Source attribution

The cryptographic protocol and wire format implemented here are Ente's, taken
from the Ente open source clients at <https://github.com/ente-io/ente>. No code
is imported or vendored from those projects; any reference code that is copied
is rewritten in TypeScript in this repository. Protocol fidelity is verified
against the upstream implementations in `web/packages/base/`,
`mobile/apps/photos/lib/`, and `cli/`.

## License

WTFPL. See [LICENSE](LICENSE).

## Author

[@sneak](https://sneak.berlin)