commit 0006af4f701bff1d5fdd81c898ab3f0c77e6fef1 Author: sneak Date: Sat May 9 18:57:58 2026 +0200 Initial commit: README with project spec and implementation plan diff --git a/README.md b/README.md new file mode 100644 index 0000000..b78c8f9 --- /dev/null +++ b/README.md @@ -0,0 +1,276 @@ +# quack + +quack is a WTFPL-licensed TypeScript client library and CLI by +[@sneak](https://sneak.berlin) for the [Ente](https://ente.io) end-to-end +encrypted photo hosting service. It logs in, enumerates collections and files, +and downloads individual images while decrypting them on the way to disk. + +## Getting Started + +```bash +git clone https://git.eeqj.de/sneak/quack.git +cd quack +yarn install +yarn build + +# Log in (prompts for email, password, and OTP/TOTP if required). +# Stores an encrypted session under $XDG_CONFIG_HOME/quack/. +yarn quack login + +# List the user's collections (albums). +yarn quack collections + +# List files in a collection. +yarn quack files --collection 12345 + +# Download and decrypt a single file to ./out/. +yarn quack get 67890 --out ./out/ +``` + +For library use: + +```ts +import { Client } from "quack"; + +const client = await Client.fromSavedSession(); +for (const c of await client.listCollections()) { + console.log(c.id, c.name); +} +const file = await client.getFile(67890); +await client.downloadFile(file, "./out/"); +``` + +## Rationale + +Ente is one of very few photo services with a credible end-to-end encryption +story. The official clients (mobile Flutter app, web React app, and Go CLI) are +high quality but each is bound up with a UI or with sync semantics that aren't +useful for scripted access. Pulling individual photos out of an Ente account +from a script (for backup, migration, archival, or programmatic processing) is +awkward without a small TypeScript library that does just the cryptography and +nothing else. + +This project exists to provide that library. It is deliberately scoped to read +operations: log in, walk the account, decrypt and save files. Upload, sharing, +deletion, and sync state management are out of scope for the first release. + +## Design + +quack is a TypeScript library with a thin CLI wrapper. The library does the +work; the CLI is for humans. + +### Layout + +``` +quack/ + src/ + crypto/ libsodium primitives (boxes, secretstreams, KDF, SRP) + api/ HTTP client + typed endpoint wrappers + auth/ login flow (SRP + email OTP + TOTP), key unwrap + model/ decrypted Collection, File, Metadata types + session/ on-disk session persistence (token + master key) + client.ts high-level Client class assembled from the above + index.ts public library exports + bin/ + quack.ts CLI entrypoint (commander.js) + test/ unit + integration tests (vitest) + Makefile + Dockerfile + package.json + tsconfig.json +``` + +### Cryptography + +All cryptography is done by `libsodium-wrappers-sumo` (the "sumo" build is +required for `crypto_pwhash` / Argon2id). No hand-rolled crypto. + +The key hierarchy, derived during login, looks like this: + +1. The user enters their password. +2. Argon2id (`crypto_pwhash`) over the password and a server-issued + `kekSalt`, with server-issued `memLimit` and `opsLimit`, produces a + 32-byte Key Encryption Key (KEK). +3. SRP login: a 16-byte SRP login subkey is derived from the KEK using + `crypto_kdf_derive_from_key` (BLAKE2b) with subkey id 1 and context + `loginctx`. That 16-byte value is the SRP password. +4. After SRP completes (or after email-OTP fallback), the server returns a + blob of "key attributes" plus an encrypted auth token. +5. `crypto_secretbox_open_easy` over the encrypted master key with the KEK + yields the 32-byte master key. +6. `crypto_secretbox_open_easy` over the encrypted secret key with the master + key yields the user's X25519 private key. The matching public key is + delivered in cleartext. +7. `crypto_box_seal_open` over the encrypted token with the user's keypair + yields the URL-safe base64 auth token used in `X-Auth-Token` for all + subsequent calls. + +Per-collection keys are decrypted with `crypto_secretbox_open_easy` using +the master key (for owned collections). Per-file keys are decrypted with +`crypto_secretbox_open_easy` using the collection key. File metadata is a +secretbox under the file key. File content is a chunked +`crypto_secretstream_xchacha20poly1305` stream under the file key, with a +4 MiB plaintext chunk size and a 17-byte authentication overhead per chunk. + +### HTTP API + +Production endpoints: + +- API: `https://api.ente.io` +- File download CDN: `https://files.ente.io/?fileID=` +- Thumbnail CDN: `https://thumbnails.ente.io/?fileID=` + +A custom API endpoint is configurable for self-hosted servers via the +`ENTE_API_ENDPOINT` environment variable. When set, file downloads route +through `/files/download/` instead of the dedicated CDN host. + +Required request headers on every authenticated call: + +- `X-Auth-Token`: the decrypted auth token from login. +- `X-Client-Package`: identifies the client. quack uses + `berlin.sneak.quack`. + +Endpoints used: + +- `GET /users/srp/attributes?email=` — fetch SRP + KDF parameters. +- `POST /users/srp/create-session` — begin SRP handshake. +- `POST /users/srp/verify-session` — complete SRP, receive 2FA challenge or + the encrypted token + key attributes. +- `POST /users/ott` and `POST /users/verify-email` — email OTP fallback path. +- `POST /users/two-factor/verify` — TOTP second factor. +- `GET /collections/v2?sinceTime=` — list collections changed since + microsecond timestamp; pass 0 for a full enumeration. +- `GET /collections/v2/diff?collectionID=&sinceTime=` — list + files in a collection; paginate while `hasMore` is true. +- `GET https://files.ente.io/?fileID=` — download encrypted file bytes. + +### Session persistence + +After login, quack writes an encrypted session blob to +`$XDG_CONFIG_HOME/quack/session.json` (default +`~/.config/quack/session.json`) containing the auth token, the user's master +key, the user's secret key, and the user's email. The session file is itself +encrypted with a key derived from a per-machine random value stored in the OS +keychain when available, falling back to a key file at mode `0600` in the +same config directory. The master key and secret key are never written to +disk in cleartext. + +### CLI surface + +- `quack login` — interactive login, writes session. +- `quack logout` — deletes the session. +- `quack whoami` — prints the logged-in email. +- `quack collections` — list collections (id, name, type, file count). +- `quack files --collection ` — list files in a collection (id, name, + type, creation time, size). +- `quack get --out ` — download and decrypt a file. +- `quack get-thumb --out ` — download and decrypt a + thumbnail. + +All commands accept `--json` for machine-readable output. + +## TODO + +Phase 1: scaffolding (this commit and the next) + +- [x] `git init`, write README +- [ ] Create `initial-scaffolding` feature branch +- [ ] Add `LICENSE` (WTFPL), `REPO_POLICIES.md`, `.gitignore`, + `.editorconfig`, `.prettierrc`, `.prettierignore`, `.dockerignore` +- [ ] Add `Makefile` with `test`, `lint`, `fmt`, `fmt-check`, `check`, + `docker`, `hooks`, plus `build`, `dev`, `clean` +- [ ] Add `Dockerfile` running `make check` against pinned node image +- [ ] Add `.gitea/workflows/check.yml` running `docker build .` +- [ ] Add `package.json`, `tsconfig.json`, install pinned versions of + `typescript`, `libsodium-wrappers-sumo`, `secure-remote-password`, + `commander`, `vitest`, `prettier`, `eslint`, `@types/node` +- [ ] Smoke test: `make check` and `make docker` both pass + +Phase 2: crypto primitives + +- [ ] Wrap libsodium init as an awaitable singleton +- [ ] `deriveKEK(password, kekSalt, memLimit, opsLimit)` (Argon2id) +- [ ] `deriveLoginSubkey(kek)` (KDF with subkey id 1, context `loginctx`, + 16 bytes) +- [ ] `decryptBox(ciphertext, nonce, key)` for secretbox +- [ ] `decryptSealed(ciphertext, publicKey, secretKey)` for sealed box +- [ ] `decryptStream(reader, header, key, writer)` for chunked secretstream + (4 MiB plaintext chunks, 17-byte overhead) +- [ ] Round-trip tests against vectors generated by libsodium directly + +Phase 3: SRP + auth + +- [ ] SRP-6a client using `secure-remote-password` with the same group as + the server +- [ ] `loginViaSRP(email, password)` returning either a 2FA challenge or + `KeyAttributes + encryptedToken` +- [ ] `loginViaEmailOTP(email)` for accounts without SRP enabled +- [ ] `submitTOTP(sessionID, code)` +- [ ] `unwrapMasterKey(keyAttributes, password)` returning master key, + secret key, public key, and decrypted token +- [ ] Tests against recorded HTTP fixtures + +Phase 4: HTTP client + endpoints + +- [ ] Tiny fetch wrapper that attaches `X-Auth-Token` and `X-Client-Package` +- [ ] Typed wrappers for the endpoints listed above +- [ ] Retry policy: no retry on 4xx, exponential backoff on 5xx and network + errors +- [ ] Error type that surfaces the server's error code and request id + +Phase 5: collections and files + +- [ ] `listCollections()` paginating on `sinceTime` until empty +- [ ] Decrypt per-collection key with master key +- [ ] Decrypt collection name with collection key +- [ ] `listFiles(collectionID)` paginating on `sinceTime` while `hasMore` +- [ ] Decrypt per-file key with collection key +- [ ] Decrypt file metadata blob with file key, expose typed `FileMetadata` + +Phase 6: download + +- [ ] `downloadFile(fileID, outPath)` streams the encrypted body, decrypts + it chunk by chunk, writes plaintext to `outPath`. Resolves the + filename from the decrypted metadata title when no `outPath` is + supplied. +- [ ] `downloadThumbnail(fileID, outPath)` for the thumbnail CDN +- [ ] Live integration test against a throwaway Ente account if one is + available + +Phase 7: session persistence + +- [ ] Write session blob encrypted with a key from the OS keychain + (`keytar`) or a `0600` keyfile fallback +- [ ] `Client.fromSavedSession()` and `Client.saveSession()` +- [ ] `quack logout` deletes the session and the keychain entry + +Phase 8: CLI + +- [ ] `commander`-based CLI that matches the surface in the Design section +- [ ] `--json` output for every command +- [ ] Reasonable progress output for long downloads (only when stdout is a + TTY) + +Phase 9: docs and 1.0 + +- [ ] README usage examples for both library and CLI verified by hand +- [ ] All TODO items above checked +- [ ] Tag `v1.0.0` + +## Source attribution + +The cryptographic protocol and wire format implemented here are Ente's, taken +from the Ente open source clients at +. No code is imported or vendored from those +projects; any reference code that is copied is rewritten in TypeScript in +this repository. Protocol fidelity is verified against the upstream +implementations in `web/packages/base/`, `mobile/apps/photos/lib/`, and +`cli/`. + +## License + +WTFPL. See [LICENSE](LICENSE). + +## Author + +[@sneak](https://sneak.berlin)