quak

quak is a WTFPL-licensed TypeScript client library and CLI by @sneak for the Ente end-to-end encrypted photo hosting service. It logs in, enumerates collections and files, and downloads individual images while decrypting them on the way to disk.

quak also includes a resilient backup command that downloads every file in the account into a deduplicated local directory tree, skipping files that already exist on disk and continuing past individual download failures instead of crashing. It decrypts and persists all three metadata layers (basic, private magic, public magic) per file, including camera info, GPS coordinates, captions, and any face/keyword labels the Ente clients have added. A helper subcommand can detect and regenerate missing thumbnails, encrypting and uploading them back to the server.

Getting Started

git clone https://git.eeqj.de/sneak/quak.git
cd quak
yarn install
yarn build

# Log in (prompts for email, password, and OTP/TOTP if required).
yarn quak login

# List the user's collections (albums).
yarn quak collections

# List files in a collection.
yarn quak files --collection 12345

# Download and decrypt a single file.
yarn quak get 67890 --out ./photo.jpg

# Back up every file in the account.
yarn quak backup ./my-backup

For library use, the primary surface is the cache-backed Library:

import { Client, Library } from "quak";

// Log in once; the client satisfies the library's client interface.
const client = await Client.login({
    email: "you@example.com",
    password: "your-password",
});

// Open a cache-backed library. On an empty cache this awaits one server
// refresh; on an existing cache it returns immediately and refreshes in the
// background every `refreshIntervalSeconds` (default 3).
const lib = await Library.open({ client });

// Default reads answer synchronously from the local cache — no network.
for (const album of lib.albums.list()) {
    console.log(album.collectionID, album.name);
    for (const photo of album.photos.list()) {
        console.log(`  ${photo.title} [${photo.fileType}]`);
    }
}

// Fresh reads await a server round-trip and answer with current state.
const { albums } = await lib.fresh();
console.log(`${albums.list().length} albums as of now`);

// Fetch (and cache) one photo's full-resolution bytes.
const photo = lib.photos.byID({ fileID: 12345 });
if (photo) {
    const { path } = await photo.original();
    console.log(`original at ${path}`);
}

await lib.close();

The lower-level Client (login, session serialization, and the raw enumeration/download calls) is exported too and documented under Design below.

Entrypoints

This repository adheres to the Scripts to Rule Them All standard: normalized scripts in script/ are the entrypoints for the development workflow, and the Makefile targets are thin shims that call them. The scripts are POSIX sh (not bash) so they run in minimal containers such as alpine. We provide:

• script/bootstrap — install all dependencies (node/yarn if missing, then yarn install --frozen-lockfile)
• script/setup — set up the repo for development after a fresh clone: runs script/bootstrap, then script/install-precommit
• script/projectname — output the project name (our own extension); used by script/docker for the image tag
• script/build — compile the TypeScript sources into dist/, then verify that the entrypoints package.json declares (main, types, bin) are among the files the compiler wrote, and make the CLI executable (our own extension)
• script/test — run the test suite, by building the test phase of the Dockerfile (vitest, 90s timeout, verbose rerun on failure); requires docker
• script/lint — run eslint and a prettier check, by building the lint phase of the Dockerfile; requires docker (see Linting and testing below)
• script/fmt — format all files with prettier (writes)
• script/fmt-check — check formatting on the host (read-only); standalone, and not called by script/check or script/precommit, because script/lint already checks formatting in the container
• script/check — run all checks: test, lint (our own extension)
• script/docker — build the image, tagged via script/projectname
• script/cibuild — build the image (what CI runs); its last stage depends on the lint and test phases, so this one build lints, tests and compiles
• script/precommit — run by the git pre-commit hook (our own extension); runs script/lint, which checks both lint and formatting, but deliberately not the tests, so the TDD red-phase commit can land
• script/install-precommit — installs the git pre-commit hook (our own extension); make hooks shims to it

make hooks installs the pre-commit hook that runs script/precommit.

Linting and testing

Linting and testing are phases of the Dockerfile. The lint phase copies the repo into a digest-pinned node image and runs eslint and prettier --check .; the test phase does the same with the suite. script/lint and script/test each build one phase with docker build --no-cache --target <phase>. There is no host lint or test path: docker is required, and that also works where the docker daemon is remote and bind mounts are impossible.

The last stage of the Dockerfile compiles the package, and it copies a file from each phase, so it cannot be built unless lint and the tests pass. That is why script/cibuild is a single docker build: it runs lint and the tests once each and then compiles.

Every docker build in script/ passes --no-cache. On an unchanged tree Docker would otherwise serve the lint and test steps from cache, nothing would run, and the build would still exit 0.

The formatting check is part of the lint phase, not a step beside it, so script/check and script/precommit do not call script/fmt-check as well; that would run prettier a second time over the same tree for the same verdict. script/fmt-check remains as a standalone entrypoint for asking the formatting question on the host. Its verdict matches the container's: prettier is pinned to an exact version, installed from yarn.lock under --frozen-lockfile in both places, and reads .gitignore as its default ignore file — which is why .dockerignore keeps .gitignore in the build context.

Rationale

Ente is one of very few photo services with a credible end-to-end encryption story. The shipping clients (mobile Flutter, web React, desktop Electron, and Go CLI) work, but they are slow, buggy, and difficult to script against. The Flutter app fails to sync reliably. The web app is heavy. The desktop app is the web app inside a slow Electron wrapper. The Go CLI is the closest thing to a usable tool, but it is awkward to integrate from anything that is not a shell. The Go CLI's backup mode crashes entirely when a single file download fails, which makes it useless as an actual backup tool.

quak fixes these problems. This repo ships a correct, well-tested implementation of Ente's cryptographic protocol and API surface, plus a CLI that proves the library is enough to do real work without a UI. The backup command is resilient by design: per-file errors are logged and the run continues.

The longer-term goal of this project is a simple desktop client for Ente, built on this library in Electron (or a comparable runtime), with two priorities above everything else: correctness and stability. Performance and simplicity follow from those. Features will be added only after the protocol layer is correct, the local cache is reliable, and the UI is responsive on a five-year-old laptop.

Development workflow

All work on quak is test-driven. No exceptions.

1. Every change starts on a feature branch off main.
2. The first commit on the branch is the test suite for what is being added or changed. Those tests must fail at that commit; the branch is red until the implementation lands.
3. Subsequent commits add the implementation and any refactors needed to make the tests pass.
4. A feature branch can only be merged into main when make check is green. main is always green. CI runs script/cibuild, which builds the Dockerfile: its lint and test phases, then the compile, so neither a red branch nor one that does not compile can pass CI.
5. Tests are the canonical API documentation for this library. Every test file is commented thoroughly enough that a reader who has never seen quak can learn how to use it from the tests alone. Comments explain why a behavior matters, not just what the assertion checks.
6. Test fixtures (cryptographic vectors, recorded HTTP responses, sample files) are committed alongside their tests. Where possible they are generated by deterministic helpers in the test/ tree so any reviewer can reproduce them by running the helper.
7. git rebase -i is allowed on a feature branch before merge to clean up the test-then-implementation sequence into reviewable commits, but the final history must still show tests landing before (or with) the matching implementation.
8. The pre-commit hook installed by make hooks runs script/precommit, which runs script/lint — eslint and the prettier check, in the container — but not the tests, and so not the full make check. This is deliberate so the TDD red-phase commit (failing tests, no implementation yet) can land. The test phase is part of the image build, which is what CI executes via script/cibuild, so a red branch still cannot reach main.

Design

quak is a TypeScript library with a thin CLI wrapper. The library does the work; the CLI is for humans.

Layout

quak/
    src/
        crypto/         libsodium primitives (boxes, secretstreams, KDF, SRP)
        api/            HTTP client (ApiClient class)
        auth/           login flow (SRP + email OTP + TOTP), key unwrap
        model/          decrypted Collection, File, Metadata types + decrypt fns
        download/       streaming file/thumbnail download + decryption
        backup.ts       resilient full-account backup with dedup
        errors.ts       error types shared across layers
        retry.ts        retry classifier + exponential backoff with jitter
        thumbnails.ts   detect + regenerate missing thumbnails
        client.ts       high-level Client class assembled from the above
        index.ts        public library exports
    bin/
        quak.ts         CLI entrypoint (commander.js)
    test/               unit + integration tests (vitest)
    Makefile
    Dockerfile          lint phase, test phase, compile
    package.json
    tsconfig.json

make build compiles that tree into dist/, preserving its shape: the library lands in dist/src/ and the CLI in dist/bin/quak.js, which is what package.json points main, types and bin at. The compiler's rootDir is the repository root rather than src/, because bin/ is compiled too and rootDir has to contain everything that is compiled.

Cryptography

All cryptography is done by libsodium-wrappers-sumo (the "sumo" build is required for crypto_pwhash / Argon2id). No hand-rolled crypto.

The key hierarchy, derived during login, is:

1. The user enters their password.
2. Argon2id (crypto_pwhash) over the password and a server-issued kekSalt, with server-issued memLimit and opsLimit, produces a 32-byte Key Encryption Key (KEK).
3. SRP login: a 16-byte SRP login subkey is derived from the KEK using crypto_kdf_derive_from_key (BLAKE2b) with subkey id 1 and context loginctx. That 16-byte value is the SRP password.
4. After SRP completes (or after email-OTP fallback), the server returns a blob of "key attributes" plus an encrypted auth token.
5. crypto_secretbox_open_easy over the encrypted master key with the KEK yields the 32-byte master key.
6. crypto_secretbox_open_easy over the encrypted secret key with the master key yields the user's X25519 private key. The matching public key is delivered in cleartext.
7. crypto_box_seal_open over the encrypted token with the user's keypair yields the URL-safe base64 auth token used in X-Auth-Token for all subsequent calls.

Per-collection keys are decrypted with crypto_secretbox_open_easy using the master key (for owned collections). Per-file keys are decrypted with crypto_secretbox_open_easy using the collection key. File metadata is a secretstream blob (single chunk, TAG_FINAL) under the file key. File content is a chunked crypto_secretstream_xchacha20poly1305 stream under the file key, with a 4 MiB plaintext chunk size and a 17-byte authentication overhead per chunk. Thumbnails use the same secretstream blob format as metadata.

For upload (thumbnail repair), encryptBlob performs the push side: a single secretstream chunk with TAG_FINAL, returning the header and ciphertext.

HTTP API

Production endpoints:

• API: https://api.ente.io
• File download CDN: https://files.ente.io/?fileID=<id>
• Thumbnail CDN: https://thumbnails.ente.io/?fileID=<id>

A custom API endpoint is configurable for self-hosted servers via the constructor option apiOrigin. When set, file downloads route through <apiOrigin>/files/download/<id> instead of the dedicated CDN host.

Required request headers on every authenticated call:

• X-Auth-Token: the decrypted auth token from login.
• X-Client-Package: identifies the client. quak uses berlin.sneak.quak.

Endpoints used:

• GET /users/srp/attributes?email=<email>: fetch SRP and KDF parameters.
• POST /users/srp/create-session: begin SRP handshake.
• POST /users/srp/verify-session: complete SRP, receive 2FA challenge or the encrypted token plus key attributes.
• POST /users/ott and POST /users/verify-email: email OTP fallback path.
• POST /users/two-factor/verify: TOTP second factor.
• POST /users/logout: end the calling token's session (quak logout).
• GET /collections/v2?sinceTime=<usec>: list collections changed since microsecond timestamp; pass 0 for a full enumeration.
• GET /collections/v2/diff?collectionID=<id>&sinceTime=<usec>: list files in a collection; paginate while hasMore is true.
• GET https://files.ente.io/?fileID=<id>: download encrypted file bytes.
• POST /files/data/fetch: fetch encrypted ML data for a batch of files.
• POST /files/upload-url: mint a presigned upload URL (for thumbnail repair).
• PUT /files/thumbnail: register an uploaded thumbnail's object key.

Retries and timeouts

Every request in the library goes through one policy, in src/retry.ts. A request is repeated only when repeating it could produce a different answer:

• ApiError with a 5xx status: retried. So are 408 and 429, the two 4xx codes that are statements about timing rather than about the request.
• Every other 4xx: not retried. A 404 in particular is an answer, and listMissingThumbnails depends on getting it promptly and once.
• Transport failures — a fetch rejection, ECONNRESET, ETIMEDOUT, a DNS or TLS failure — and deadline aborts: retried. The errno is looked for in the error's cause chain, because that is where Node's fetch puts it.
• A truncated download: retried.
• Anything else, including a secretstream authentication failure that is not truncation: not retried. The default answer is no. For a backup tool, retrying a permanent failure spends round trips and delays every remaining file, while declining to retry a transient one costs a single file that the next run picks up.

Backoff is exponential with full jitter: the delay before retry n is random() * min(maxDelayMs, baseDelayMs * 2 ** (n - 1)). The exponential term is the ceiling and the wait is drawn below it, so a client that lost many parallel downloads to one CDN blip does not send them all again at the same instant. Defaults, configurable through ApiClientOptions.retry:

Option	Default	Meaning
attempts	4	total calls, not retries
baseDelayMs	500	ceiling for the first retry's delay
maxDelayMs	10000	upper bound on that ceiling

With those defaults a file that is going to fail gives up after at most three and a half seconds of waiting. sleep and random are injectable through the same option, which is how the test suite exercises the whole policy without waiting.

Two deadlines, renewed for each attempt:

Option	Default	Applies to	Kind
requestTimeoutMs	30000	getJSON, postJSON, putJSON, putFile	the whole request
downloadTimeoutMs	60000	file and thumbnail downloads	idle: no bytes received for this long

They are different kinds because a download's length depends on the file and the link: a whole-transfer deadline short enough to catch a hung connection would cancel a large video on a slow link that is still making progress. The download deadline restarts every time bytes arrive, so a slow download runs as long as it keeps moving, and one that stalls is aborted after 60 seconds of silence. It covers the wait for the headers and the body — getFileStream returns as soon as headers arrive, so a deadline that only guarded the initial request would leave the same hang one layer down. There is no limit on the total length of a download.

Non-idempotent requests are not blindly replayed. postJSON and putJSON send every POST and PUT in the endpoint list above; some of them change server state, and /users/two-factor/verify consumes one of a small number of second-factor attempts. They are retried only when every errno in the error's cause chain is one of the three that establish no TCP connection to the server ever existed, so no request byte can have been transmitted: ENOTFOUND and EAI_AGAIN (name resolution produced no address) and ECONNREFUSED (the peer refused the connection). A 5xx, a mid-flight reset and a deadline are all left to the caller, because each of them can happen after the server has already acted. These two do not follow redirects either: a redirect means the server already received the request, so it is reported as an error and not retried. The routing errnos EHOSTUNREACH, ENETUNREACH and ENETDOWN are excluded for the same reason, despite looking like connect-time failures: on Linux an ICMP unreachable arriving mid-flight, or a local interface going down after the request was written, delivers them on an already-established socket. They stay retryable for the idempotent calls. putFile is exempt: a presigned PUT stores one whole object at one key in one request, so replaying it has no partial state to damage.

A download is retried as a whole — request, stream consumption, and decryption — because a socket reset after the response headers have arrived surfaces in the download layer rather than in ApiClient, and that is the common failure for multi-megabyte photos over a CDN. The secretstream pull state is not resumable and these endpoints have no Range support, so a retry starts the file over. The atomic write stays outside the retry, so a download that needed three attempts still performs exactly one write and one rename. runBackup and runMetadataBackup are unchanged: the retry sits below them, and a file that fails after exhausting it is still logged, counted, and stepped over.

One imprecision is deliberate and worth knowing about. When a body ends part-way through a secretstream chunk, Poly1305 fails and carries no framing signal, so a cut connection and genuinely corrupt bytes are indistinguishable. quak reports that as truncation, which means it is retried. For a body of more than one chunk the distinction is real — a chunk that failed while the stream carried on past it stays an authentication failure and is not retried — but for a single-chunk body, which is most thumbnails and every small file, a wrong key, server-side corruption and a mid-chunk cutoff all present alike and all get retried. The cost is bounded by the attempt count, and it buys never silently keeping a truncated file.

Session handling

The Client class holds the auth token, master key, secret key, and public key in memory. There is no on-disk session store in the library; the consumer decides how to persist sessions.

client.toJSON() returns a ClientSnapshot (a plain serializable object with base64-encoded keys) that the consumer can write to disk, a database, or whatever else fits their use case. Client.fromJSON(snapshot) restores a working client from that snapshot without re-authenticating; it checks every field and each key's length first, and throws an error naming the bad field. client.logout() clears the token and zeroes the key buffers in place; every later call on that client throws. It does not contact the server, so the token stays valid there and in any saved snapshot; await client.logoutOnServer() first ends the session on the server (POST /users/logout).

The CLI stores the snapshot at the platform-appropriate data directory via env-paths: ~/Library/Application Support/quak/session.json on macOS, $XDG_DATA_HOME/quak/session.json on Linux. The file is written with mode 0600. The key material is stored in cleartext in the JSON; treat this file as you would treat the password itself. A missing file is reported as "not logged in"; a file that exists but is corrupt is reported as such, naming the bad field. Both exit with status 1.

quak logout ends the session on the server, so the token in session.json stops working even in a copy of the file, and then deletes the file. If the server call fails (or the file is corrupt), the file is still deleted, the command says the server session could not be ended, and it exits with status 1. It does not delete the cache: it prints the account's cache directory and says it still holds decrypted data (file keys in metadata.json, cached originals and thumbnails), for the user to delete if they want it gone.

CLI surface

quak [--cache-dir <path>] <command>             global: local metadata/content cache location
quak login                                      interactive or QUAK_EMAIL/QUAK_PASSWORD
quak whoami                                     print logged-in account as JSON
quak logout                                     end the session, delete it
quak collections [--json]                       list all collections
quak files --collection <id> [--json]           list files in a collection
quak get <fileID> [--out path] [--collection]   download and decrypt a file
quak get-thumb <fileID> [--out] [--collection]  download and decrypt a thumbnail
quak backup <dir> [--json]                      full incremental backup
quak backup-metadata <dir> [--exif]             dump all decrypted metadata as JSON
quak helper list-missing-thumbnails [--json]    find files with missing thumbnails
quak helper fix-missing-thumbnails [--file ids] generate + upload missing thumbnails

Every command runs on the same cache-backed library. The read commands — collections, files, get, and get-thumb — force a fresh server round-trip before they answer, so they report current account state rather than whatever the cache last held. --cache-dir overrides where the cache lives; without it each account gets its own directory under the per-user cache path.

get and get-thumb resolve the file by ID directly, so --collection is accepted for backward compatibility but ignored. backup-metadata --exif (alias --all) additionally downloads each file to extract full EXIF/IPTC/XMP metadata. The listing and backup commands support --json for machine-readable output.

backup-metadata fetches ML data in requests of up to 200 files. When a request still fails after its retries, the error is logged, each of its files is written with the reason in an mlDataError field instead of mlData, and the dump goes on. The exit code is non-zero if any ML data request failed.

helper fix-missing-thumbnails regenerates thumbnails for baseline JPEG images only, because the bundled decoder (jpeg-js) decodes only JPEG. A non-JPEG image (PNG, HEIC) or a video is reported as skipped (unsupported format), kept distinct from a failed repair, and does not affect the exit code; a genuine failure still exits non-zero. The server accepts a new thumbnail only from the file's owner and only when it is no larger than the thumbnail size it records for the file. So a file another account owns, in an album shared with you, is skipped by both thumbnail helpers without being fetched, and the fixer skips a file whose recorded thumbnail size is 0 or unknown. Otherwise the fixer lowers the quality and size of the thumbnail until it fits, and skips the file if even the smallest does not.

Backup layout

quak backup <dir> produces:

<dir>/
    originals/
        <fileID>.<ext>          actual file content (one per unique file)
        <fileID>.json           all decrypted metadata for that file
    collections/
        <name>/
            <title> -> ../../originals/<fileID>.<ext>   (symlink)
        <name>.json             collection metadata + file list

A collection's directory and JSON are named after the collection, and a symlink after the file's title, both with unsafe characters replaced. When two collections would get the same name, or two files in one collection the same title (ignoring case in both), each of them gets its ID added: two albums named Trip become Trip (10)/ and Trip (11)/, and two files titled IMG_0001.JPG become IMG_0001 (12345).JPG and IMG_0001 (12346).JPG. IDs never change, so a name stays the same from run to run until such a clash appears or goes away.

Each run removes the symlinks into originals/ that no longer belong in their collection's directory, and the directories (and JSON) of collections that were deleted or renamed. Nothing else in collections/ is touched: a file or a symlink you put there stays, and a directory that still holds one after its symlinks are removed stays too, with its JSON.

Each file is downloaded exactly once regardless of how many collections it appears in. On subsequent runs, existing originals are skipped. If a download fails, the error is logged and the backup continues with the next file. The exit code is non-zero if any files failed.

Each original is copied to a temporary file named .quak-backup-<fileID>.<ext>-<pid>-<random>.tmp in the same directory, synced to disk, and renamed into place, so an original is either complete or absent, even after a power cut. A run that is killed can leave one of these temporary files behind; the next backup deletes those whose process is no longer running. Downloads and the content cache use the same scheme with .quak-<pid>-<random>.tmp names, and opening a library deletes those in the cache whose process is no longer running, so a download another process has in progress in the same cache is left alone. The rename replaces whatever was at the destination rather than writing through it: a symlink there is replaced, not followed, and the new file has the temporary file's permissions, not those of the file it replaced.

TODO

• Retry policy: no retry on 4xx, exponential backoff on 5xx and network errors
• Update the API reference section below to match the current implementation
• make docker green
• Tag v1.0.0

Future (desktop client, separate repo):

• Electron app skeleton consuming this library
• Local cache (SQLite) keyed on (collectionID, fileID, updationTime)
• Background sync worker that streams new files into the cache
• Gallery UI: thumbnails, full-image view, basic search
• Upload, delete, and share operations in the library

API reference

The library's primary surface is the cache-backed Library; the lower-level Client sits underneath it and is covered by the Design sections above. The test suite is the canonical, executable documentation — test/library/ and test/client/usage.test.ts walk every operation, and yarn test verifies them.

Opening a library

Library.open(options) loads the on-disk cache, starts the background refresh loop, and resolves to a Library. On an empty cache it awaits the first refresh so it never opens onto empty data; on an existing cache it returns immediately and refreshes in the background, so an unreachable server does not block opening.

LibraryOptions:

Option	Default	Meaning
client	required	the account client (a Client, or any LibraryClient)
cacheDirectory	<XDG cache>/quak/<userID>	where metadata.json and the content cache live
downloadDirectory	none	backup destination; an original already stored there counts as cached
refreshIntervalSeconds	3	background refresh cadence
precacheThumbnails	true	prefetch every thumbnail, newest first
precacheOriginals	true	prefetch the favorites album and the latest-window originals
precacheOriginalsDays	7	length in days of that latest window
cacheOriginalsMaxBytes	100 GiB	hard ceiling on the originals cache
freeBelowBytes	50 GiB	free space to protect on the volume; the effective limit adapts down
isOriginalPinned	none	extra predicate for originals that must never be evicted
pools	fresh RequestPools	the bounded request pools (sets concurrency)
onProgress	none	refresh/ML/precache progress callback (RefreshEvent)
contentSource	the client's own	override the byte source (mainly for tests)

Concurrency is set through pools: construct new RequestPools({ metadataConcurrency, contentConcurrency, thumbnailConcurrency }) and pass it. The three pools default to 10 / 5 / 25 (see Request pools below).

lib.status() returns a LibraryStatus (collection/file counts, last refresh/ML times and errors, originals usage and effective limit, precache progress, and closed). lib.close() stops the background timer; it is idempotent, and an in-flight refresh is left to finish. The promise it returns resolves once that refresh (including its cache write), the ML data fetch and the precache fetches already running have all finished, so the cache directory can then be removed.

Default reads vs. fresh reads

Default reads — lib.albums, lib.photos, lib.timeline — answer synchronously from the last refreshed copy held in RAM and never touch the network. The background timer refreshes that copy every refreshIntervalSeconds, so a default read is immediate but may be up to one interval stale.

await lib.fresh() forces a refresh, waits for it to complete and persist, and returns the same { albums, photos, timeline } namespaces — now guaranteed to reflect a completed server round-trip. Concurrent fresh() calls coalesce onto one refresh, and a refresh that fails rejects the caller (default reads stay silent and keep serving the last good copy). The CLI's read commands use fresh reads (issue #75).

Read surface

• lib.albums.list() → Album[], newest-updated first. lib.albums.byID({ collectionID }) and byName({ albumName }) → Album | undefined.
• lib.photos.byID({ fileID }) → Photo | undefined. lib.photos.records({ fileIDs }) → PhotoRecord[] in the requested order, each id once, unknown ids dropped.
• lib.timeline.groups({ groupBy, filter? }) → TimelineGroup[], grouped by "day" | "week" | "month" (keys YYYY-MM-DD, ISO YYYY-Www, YYYY-MM), newest group first. A PhotoFilter combines albumID, text (title/caption/album-name substring), fileTypes, hasLocation, and includeArchived; hidden photos are always excluded.

An Album exposes its record fields and album.photos.list() → Photo[] (newest first). A Photo exposes its record fields, photo.record() → PhotoRecord, and two content methods:

• await photo.original(opts?) → { path, bytes } — the full-resolution file.
• await photo.thumbnail(opts?) → { path, bytes }.

Both serve from the on-disk content cache when the bytes are present and otherwise fetch through the pools; opts.onProgress reports per-file progress. They throw when the library was opened without a content source.

Lower-level accessors that return decrypted model objects (which hold key material) are also available: listCollections(), getCollection(id), listFiles(collectionID), getFile(collectionID, fileID), and getFileByID(fileID).

Records and change notifications

The GUI-facing records hold no key material and no binary, so they survive structuredClone/JSON across the Electron IPC boundary:

• PhotoRecord: fileID, albumIDs, title, takenAt (milliseconds), fileType, optional caption / width / height / latitude / longitude, isArchived, isHidden, and thumbnailPath / originalPath once the bytes are cached.
• AlbumRecord: collectionID, name, type, isShared, updationTime, and fileIDs (newest first).
• LibrarySnapshot: { albums, photos, takenAt }.

lib.snapshot() returns a LibrarySnapshot (albums newest-updated first, photos newest first). lib.subscribe({ onChange }) delivers a LibraryChange (albumsChanged, photosChanged, fileIDsRemoved, albumIDsRemoved, refreshedAt) whenever a refresh alters the projection, and returns { unsubscribe }; a refresh that changes nothing delivers nothing.

Thumbnails, ML search, and backup

• lib.thumbnails.ensure({ fileIDs, priority, signal?, onProgress? }) prefetches thumbnails through the thumbnail pool, deduped by fileID, returning one EnsureResult ({ fileID, path?, error? }) per file. priority is "visible" | "ahead" | "background"; only "visible" preempts background work.
• lib.mldata searches the CLIP index built from Ente's per-file ML data: forFile({ fileID }) → Promise<MLData | undefined> (the whole stored payload — face boxes, landmarks, embedding — read from disk on demand); similar({ fileID, limit? }) and searchByEmbedding({ embedding, limit? }) → SimilarResult[] ({ fileID, score }, cosine similarity, most similar first, default limit 20). quak bundles no text encoder, so searchByEmbedding takes a query vector the caller produced elsewhere.
• await lib.backup(opts?) → BackupResult. It refreshes, fetches every in-scope original (and, with includeThumbnails, thumbnails) through the content cache, and rebuilds the on-disk backup tree with a durable failure ledger. BackupOptions: downloadDirectory (falls back to the one open() was given), includeOriginals (default true), includeThumbnails (default false), onlyAlbumNames, and onProgress. See Backup layout above for the tree it writes.

Request pools

RequestPools holds three independent bounded pools — metadata (10), content (5), thumbnails (25) — because Ente meters these traffic classes differently. Each pool orders on-demand work ahead of background/precache work and dedups in-flight fetches by key, and an idle pool never lends its slots to a busy one.

On-disk cache layout

Under cacheDirectory:

<cacheDirectory>/
    metadata.json                 decrypted account state + refresh cursor
    originals/<fileID>.<ext>      cached full-resolution files
    thumbnails/<fileID>.jpg       cached thumbnails
    mldata/
        <fileID>.json             one decrypted ML payload per file
        clip.f32, clip.json       the packed CLIP index and its id list
        fetched.json              per-file fetch bookkeeping

When metadata.json belongs to a different account than the client's, Library.open deletes it and mldata/ and starts from an empty cache. Cached originals and thumbnails are kept; they are reached only through the files the current account's records name.

A stored file appears only via an atomic temp-then-rename, so its presence means it is complete. Every downloaded original (by quak get, the cache, or backup) whose metadata records a content hash (FileMetadata.hash) is hashed as it is written: unkeyed BLAKE2b with a 64-byte output, standard base64. For a live photo, which is stored as a ZIP, the image and the video are hashed separately and joined as <imageHash>:<videoHash>. A mismatch stores nothing and fails the download with an error naming the file ID. An original with no recorded hash, from a very old client, is stored unchecked.

Key types by source file

• src/library/index.ts: Library, LibraryOptions, LibraryStatus, LibraryClient, RefreshEvent
• src/library/read.ts: Album, Photo, AlbumsAPI, PhotosAPI, TimelineAPI, PhotoFilter, TimelineGroup, GroupBy
• src/library/content.ts: ContentResult, ContentOptions, ThumbnailsAPI, EnsureOptions, EnsureResult, ContentSource
• src/library/records.ts: PhotoRecord, AlbumRecord, LibrarySnapshot, LibraryChange
• src/library/mlsearch.ts: MLDataAPI, SimilarResult
• src/library/pools.ts: RequestPools, RequestPoolsOptions, BoundedPool
• src/backup.ts: BackupOptions, BackupResult, BackupError
• src/client.ts: Client, LoginOptions, ClientSnapshot
• src/api/client.ts: ApiClient, ApiClientOptions, StreamOptions
• src/errors.ts: ApiError, TruncatedStreamError
• src/retry.ts: withRetry, isRetryable, isSafeToReplay, RetryOptions
• src/model/types.ts: Collection, EnteFile, FileMetadata, FileType, CollectionType, RawCollection, RawEnteFile
• src/thumbnails.ts: MissingThumbnailInfo, ThumbnailFixResult

Source attribution

The cryptographic protocol and wire format implemented here are Ente's, taken from the Ente open source clients at https://github.com/ente-io/ente. No code is imported or vendored from those projects; any reference code that is copied is rewritten in TypeScript in this repository. Protocol fidelity is verified against the upstream implementations in web/packages/base/, mobile/apps/photos/lib/, and cli/.

For LLMs

If you are an LLM agent working on this repository, read and follow these documents:

• REPO_POLICIES.md in the repo root. It is copied from https://git.eeqj.de/sneak/prompts and covers repository structure, tooling, Makefile targets, Dockerfile conventions, dependency pinning, and commit hygiene. All external dependencies must be pinned by cryptographic hash in yarn.lock. Never git add -A. Never force-push to main.

• The "Development workflow" section above. All changes go on feature branches. Tests are written first and committed in a failing state before the implementation. Tests are the canonical API documentation and must be commented thoroughly. main is always green.

• Required checks before every commit: make lint must pass — that is eslint plus the prettier check, and it builds the lint phase of the Dockerfile, so it needs docker. The pre-commit hook enforces exactly that. make check (which also runs the tests) must pass before merging to main. make fmt-check is available for a host-side formatting check on its own, but it is not a separate requirement: make lint already covers it, and running both would check formatting twice. Never invoke eslint or prettier directly; linting runs in the container only.

• Formatting: prettier with 4-space indents and proseWrap: always for markdown. Use make fmt to format. Use yarn not npm.

• Testing: vitest. Tests go in test/ mirroring the src/ structure. make test must complete in under 20 seconds. Use mkdtempSync for temporary directories, never manual timestamp paths.

• Code style: const for everything, let if reassignment is needed, never var. Avoid unnecessary comments. No hand-rolled crypto. The LLM_PROSE_TELLS.md document in the prompts repo applies to any prose written in this repository (README, comments, commit messages).

License

WTFPL. See LICENSE.

Author

@sneak