Commit Graph

57 Commits

Author SHA1 Message Date
bf0eca4b80 Merge: complete CLI command surface
Adds collections, files, get, get-thumb commands. Full CLI:
  quak login / whoami / logout
  quak collections [--json]
  quak files --collection <id> [--json]
  quak get <fileID> [--out <path>] [--collection <id>]
  quak get-thumb <fileID> [--out <path>] [--collection <id>]
  quak backup <dir> [--json]

get/get-thumb search all collections when --collection is omitted.
All listing commands support --json. Live-tested against dev account.
2026-05-13 20:49:23 -07:00
ec2d12b986 Add collections, files, get, get-thumb CLI commands
Complete CLI surface:
  quak login          interactive or QUAK_EMAIL/QUAK_PASSWORD
  quak whoami         print logged-in account
  quak logout         delete session
  quak collections    list all albums (--json)
  quak files          list files in a collection (--json)
  quak get <id>       download+decrypt a file (--out, --collection)
  quak get-thumb <id> download+decrypt a thumbnail
  quak backup <dir>   full incremental backup

get/get-thumb search all collections for the file ID when --collection
is not specified. All listing commands support --json.

Live-tested: collections list, file list, single file download (472 KB
JPEG from the dev account, verified as valid JPEG with EXIF intact).
2026-05-13 20:49:13 -07:00
5499effa91 Merge: decrypt magic metadata + per-file JSON in backups
All three metadata layers (basic, magicMetadata, pubMagicMetadata) are
now decrypted from secretstream blobs and exposed on EnteFile. Backup
writes originals/<fileID>.json with the full decrypted metadata
including camera make/model, dimensions, datetime, and any face/keyword
data the Ente clients have added.
2026-05-13 19:07:26 -07:00
d4098c711a Decrypt and persist all file metadata layers
Extends RawEnteFile and EnteFile with optional magicMetadata and
pubMagicMetadata fields. Both are secretstream blobs under the file
key, decrypted to arbitrary JSON (Record<string, unknown>).

pubMagicMetadata carries ML-derived data from the Ente clients:
camera make/model, image dimensions, datetime with timezone offset,
and (when present) captions, editedName, face labels, keywords.

magicMetadata carries private mutable fields like visibility.

Backup now writes per-file JSON at originals/<fileID>.json containing
all three metadata layers (basic + magic + pubMagic).

Live-tested: all 11 files in the dev account have pubMagicMetadata
with SONY DSC-RX1RM3 camera info and 3000x2000 dimensions.
2026-05-13 19:07:16 -07:00
7baa9b585a Merge: CLI with login + backup
quak login (interactive or QUAK_EMAIL/QUAK_PASSWORD env vars), quak
backup <dir> with originals/ dedup, collections/ symlinks, per-collection
JSON metadata, incremental skip, and per-file error resilience.

Session at ~/Library/Application Support/quak/session.json (macOS) or
XDG_DATA_HOME/quak/ (Linux) via env-paths. 90 tests, all green.
2026-05-13 18:47:17 -07:00
8ee1be1cc2 CLI: quak login + quak backup with dedup symlink layout
bin/quak.ts: commander-based CLI with login (interactive + QUAK_EMAIL/
QUAK_PASSWORD env vars), whoami, logout, backup commands. Session
stored at env-paths('quak').data/session.json (~/Library/Application
Support/quak/ on macOS, XDG on Linux).

src/backup.ts: runBackup downloads all files into originals/<id>.<ext>,
symlinks into collections/<name>/<title>, writes per-collection JSON
metadata at collections/<name>.json. Deduplicates across collections
(each file downloaded once). Skips existing originals on incremental
runs. Never crashes on single-file failure.

4 backup tests + live-tested against real Ente account.
2026-05-13 18:47:06 -07:00
30a13eeeaf CLI red: backup tests, commander + env-paths deps, stub
4 tests for runBackup: full download into collection-named dirs,
incremental skip of existing files, resilient continuation after
single-file HTTP 500, and metadata.json output.

Adds commander 14.0.3 and env-paths 4.0.0 as runtime deps.
2026-05-13 18:36:07 -07:00
c1b1d12bcc Rename quack to quak in .gitignore 2026-05-13 18:04:35 -07:00
f493918777 Merge: rename quack to quak (Ente = duck, quak = German for quack) 2026-05-13 18:03:03 -07:00
d8a4b0291e Rename quack to quak
German for 'quack', matching the Ente (German for 'duck') naming. All
references updated: package name, CLI binary, X-Client-Package header,
test descriptions, temp dir prefixes, README, Makefile docker tag.
2026-05-13 18:02:55 -07:00
f87680cfd4 Merge: Client class (OO API)
Client.login() performs full SRP + key unwrap and returns a ready
object. toJSON/fromJSON for consumer-managed persistence.
listCollections, listFiles (with pagination), downloadFile,
downloadThumbnail, whoami, logout. 8 new tests in a literate
tutorial-as-test format. 86 total tests, all green.
2026-05-13 18:01:21 -07:00
58c8db4ea9 Update public exports and README for Client class
Exports Client, all lower-level modules, and all types from
src/index.ts. Replaces Phase 7 (on-disk session persistence) with
the Client class phase: session lives in the object, consumer
handles persistence via toJSON/fromJSON.
2026-05-13 18:01:18 -07:00
a8641cbfe8 Client class green: OO API wrapping the entire library
Client.login(email, password) performs the full SRP handshake, key
unwrap, and returns a ready Client. Session lives in the object.
Client.fromJSON(snapshot) restores from a serialized snapshot.
client.toJSON() produces a plain object with base64-encoded keys
that the consumer can persist however they like.

Methods: whoami, listCollections, listFiles (with pagination),
downloadFile, downloadThumbnail, logout.

All 86 tests pass including the 8-part literate usage tutorial.
2026-05-13 18:00:10 -07:00
ca6857d3fe Client class red: literate usage tests and stub
test/client/usage.test.ts is a tutorial-as-test-suite that walks
through the entire quack API in order: login, whoami, listCollections,
listFiles, downloadFile, downloadThumbnail, toJSON/fromJSON, logout.

Each it() block is a self-contained example with prose commentary
explaining what the code does and why, with code samples showing the
API as a consumer would use it. The mock server performs real SRP and
crypto so the test data is structurally identical to production.

8 tests, all failing against the stub.
2026-05-13 17:59:18 -07:00
3b04a8134f Merge: Phase 6 file download and decryption
downloadFile/downloadThumbnail: stream from CDN, buffer to 4 MiB chunk
boundary, secretstream pull decrypt, write to disk. 4 unit tests + live
integration test downloading a real JPEG from the dev Ente account.
2026-05-13 17:54:21 -07:00
75af244815 Tick off Phase 6 download TODO 2026-05-13 17:54:19 -07:00
f55d216252 Phase 6 green: download and decrypt files to disk
downloadFile streams the encrypted body from the CDN, buffers it to
the 4 MiB + 17 encrypted chunk boundary, decrypts each chunk via
secretstream pull, and writes the concatenated plaintext to disk.
downloadThumbnail does the same for the thumbnail CDN.

4 unit tests (single-chunk, large single-chunk, filename fallback,
thumbnail) + live integration test that downloads a real 472 KB JPEG
from the dev account and verifies it lands on disk.

Uses mkdtempSync for temp directories (not manual timestamp paths).
2026-05-13 17:54:03 -07:00
8f4afb06c0 Phase 6 red: file download + decryption tests
4 tests: single-chunk download, metadata-title fallback filename,
multi-chunk stream (50-byte chunks to exercise the buffering/split
logic without allocating 4 MiB), and thumbnail download.

Tests encrypt payloads with sodium's secretstream push, serve them
from a mock fetch, and verify the decrypted file on disk.
2026-05-13 17:40:51 -07:00
cfeec0a009 Merge: Phase 5 collection and file decryption
decryptCollection: secretbox key+name from raw server JSON, type mapping,
isShared. decryptFile: secretbox key + secretstream-blob metadata, fileType
mapping, header passthrough. decryptBlob: convenience for single-chunk
secretstream (file metadata uses this form, not secretbox).

10 unit tests + live integration test against real Ente API. All 74 pass.
2026-05-13 17:38:39 -07:00
9ea829aaa6 Tick off Phase 5 collection/file decryption TODO 2026-05-13 17:38:36 -07:00
44718a92a9 Fix file metadata decryption: use secretstream blob, not secretbox
File metadata is encrypted as a single-chunk secretstream blob (the
'decryptionHeader' is the secretstream init header, not a secretbox
nonce). Collection keys and names correctly use secretbox.

Adds decryptBlob(ciphertext, header, key) to the crypto module as a
convenience wrapper for single-chunk secretstream decryption (init +
pull + verify TAG_FINAL).

Live-tested: collection names and file metadata (titles, types, dates)
decrypt correctly from the real Ente API.
2026-05-13 17:38:18 -07:00
f81216333e Phase 5 red: collection and file decryption tests
10 tests covering decryptCollection (key + name decryption from raw
server JSON, type mapping, isShared, missing name, wrong key) and
decryptFile (key + metadata decryption, fileType number-to-string
mapping, file/thumbnail header passthrough, wrong key).

Adds src/model/types.ts with both raw (server) and decrypted (library)
type definitions. src/model/decrypt.ts has throwing stubs.
2026-05-13 17:12:46 -07:00
ab6e59bd4d Merge: fix auth token encoding (URL-safe base64 with padding)
Live-tested against real Ente API. Login, key unwrap, and authenticated
collection listing all succeed.
2026-05-13 17:10:11 -07:00
e3e40229a5 Fix auth token encoding: use URL-safe base64 WITH padding
The Ente server validates the auth token as URL-safe base64 with
padding (matching Go's base64.URLEncoding). Our toBase64URL strips
padding, producing a 43-char token where the server expects 44. This
caused HTTP 401 'invalid token' on every authenticated call.

Adds toBase64URLPadded to the crypto module and uses it in unwrapAuth
for the token specifically. toBase64URL (no-padding) is kept for
general use (JWT-style contexts).

Adds test/integration/live-login.ts which logs into the dev account
(entedev2026jp@acidhou.se), unwraps keys, and fetches collections
from the real Ente API. Verified: 4 collections returned successfully.
2026-05-13 17:10:04 -07:00
e182d95d80 Merge: Phase 3b SRP login flow
Complete login flow: beginLogin (SRP-6a with 4096-bit group via
fast-srp-hap, with email-OTP and TOTP 2FA fallback paths),
submitTOTP, requestEmailOTP, submitEmailOTP. Built test-first with
a mock server performing real SRP math; 7 new tests, 64 total.
2026-05-11 10:11:39 -07:00
22260c142f Tick off Phase 3 SRP + auth TODO 2026-05-11 10:11:37 -07:00
dcec9b92ad Phase 3b green: implement login flow (SRP + email OTP + TOTP)
beginLogin(api, email, password):
  1. Fetches SRP attributes for the email
  2. If isEmailMFAEnabled, returns { kind: 'emailOTP' } immediately
  3. Otherwise: derives KEK + login subkey, creates SRP client with
     4096-bit group, runs the two-round handshake (create-session,
     verify-session), verifies server M2
  4. Returns { kind: 'complete' } with AuthorizationResponse, or
     { kind: 'totp' } / { kind: 'passkey' } if 2FA is required

submitTOTP(api, sessionID, code): POST /users/two-factor/verify
requestEmailOTP(api, email): POST /users/ott
submitEmailOTP(api, email, code): POST /users/verify-email

All 64 tests pass including real SRP-6a handshakes against a mock
server built with fast-srp-hap's SrpServer.
2026-05-11 10:11:19 -07:00
75b57cfb29 Phase 3b red: login flow tests with SRP mock server
Adds fast-srp-hap (the same SRP library Ente's web client uses, pinned
to 2.0.4) as a runtime dependency.

Tests build a full mock Ente server using fast-srp-hap's SrpServer to
exercise real SRP-6a math end-to-end. The mock handles:
  GET /users/srp/attributes
  POST /users/srp/create-session
  POST /users/srp/verify-session
  POST /users/two-factor/verify
  POST /users/ott
  POST /users/verify-email

7 tests covering:
  * SRP login completing successfully
  * SRP login requiring TOTP (returns { kind: 'totp' })
  * Wrong password (SRP M1 fails server-side checkM1)
  * Email MFA fallback (returns { kind: 'emailOTP' })
  * submitTOTP
  * requestEmailOTP + submitEmailOTP
2026-05-11 01:04:10 -07:00
7d19d16b1b Merge: Phase 4 ApiClient
HTTP client with injectable fetch, production/self-hosted origin
routing, X-Auth-Token/X-Client-Package headers, JSON helpers, streaming
file/thumbnail downloads, and ApiError mapping. 19 tests, all green.
2026-05-11 01:02:15 -07:00
87ff5f3108 Tick off Phase 4 ApiClient TODO 2026-05-11 01:02:13 -07:00
0f70409fd8 Phase 4 green: implement ApiClient
Production origins: api.ente.io, files.ente.io, thumbnails.ente.io.
Custom apiOrigin routes file/thumbnail downloads through the same host
at /files/download/<id> and /files/preview/<id>.

Every request carries X-Client-Package (berlin.sneak.quack).
Authenticated requests carry X-Auth-Token, toggled via setAuthToken /
clearAuthToken or the constructor authToken option.

getJSON appends query params (skipping undefined values), parses JSON.
postJSON sends JSON with Content-Type header, parses JSON response.
getFileStream / getThumbnailStream return the response body stream.

Non-2xx responses throw ApiError with status, code (from JSON body),
requestID (from x-request-id header), and raw body. Retry logic is
deferred to a follow-on branch.

All 57 tests pass.
2026-05-11 01:02:03 -07:00
ef3f10fecc Phase 4 red: ApiClient tests and stub
19 tests covering ApiClient's full public surface: default and custom
origins, X-Client-Package and X-Auth-Token headers, getJSON with query
params, postJSON with JSON body, ApiError on 4xx/5xx, streaming file
and thumbnail downloads, and self-hosted origin routing.

Tests inject a recording fetch via the constructor, so nothing hits the
network. The test file is documented to serve as canonical usage
reference per the development workflow.
2026-05-11 01:01:34 -07:00
d8466be0a7 Merge: Phase 3a auth/unwrap
unwrapAuth: password to master/secret/public key and auth token. The
password-only side of Ente login. Built test-first; 6 tests describe
the protocol and verify byte-for-byte recovery of the inputs from a
synthetic AuthorizationResponse.
2026-05-11 00:59:45 -07:00
78fdabe54a Phase 3a green: implement auth.unwrapAuth
The implementation is exactly the decryption chain documented in the
test file: deriveKEK -> decryptBox(masterKey) -> decryptBox(secretKey)
-> decryptSealed(token) -> toBase64URL. Errors from the underlying
crypto primitives propagate; the only added validation is the up-front
check that the response actually contains both keyAttributes and
encryptedToken (caller bug if not).

Also re-exports the auth/unwrap and auth/types public surface from
src/index.ts.

All 38 tests pass; make check and make docker are green.
2026-05-11 00:59:43 -07:00
6386a0ec9f Phase 3a red: auth.unwrapAuth tests and stub
Tests for the password-only decryption chain that follows a successful
login (SRP or email OTP, with or without 2FA). The unwrap covers:
  password -> KEK (Argon2id) -> masterKey (secretbox) ->
  secretKey (secretbox) -> tokenBytes (sealed box) -> base64url token

Each test builds a synthetic AuthorizationResponse using libsodium
directly and asserts unwrapAuth recovers the inputs byte for byte. The
test file also functions as the canonical description of the protocol.

Adds src/auth/types.ts with KeyAttributes, SRPAttributes,
AuthorizationResponse, and LoginChallenge declarations matching the
README's API reference. src/auth/unwrap.ts is the throwing stub; the
real implementation lands next.
2026-05-11 00:58:27 -07:00
2e2238fa5f Merge: Phase 2 crypto primitives
Implements all the libsodium primitives quack needs to unwrap key
material and decrypt files (KDF, secretbox, sealed box, secretstream
pull). Built test-first per the development workflow; 32 tests pass.
2026-05-09 12:45:55 -07:00
52c2fa844c Tick off Phase 2 crypto primitives TODO 2026-05-09 12:45:53 -07:00
8aecf977e9 Phase 2 green: implement crypto primitives
Each stub is replaced with a thin wrapper over libsodium-wrappers-sumo:

  * init() awaits sodium.ready
  * toBase64 / toBase64URL / fromBase64 use sodium's base64 variants;
    fromBase64 tries all four (standard, standard-no-pad, URL-safe,
    URL-safe-no-pad) so callers don't have to know which form Ente
    delivered
  * deriveKEK is sodium.crypto_pwhash with ALG_ARGON2ID13 and 32-byte
    output
  * deriveLoginSubkey is sodium.crypto_kdf_derive_from_key(32, 1,
    'loginctx', kek).slice(0, 16) per the upstream Ente clients
  * decryptBox is sodium.crypto_secretbox_open_easy
  * decryptSealed is sodium.crypto_box_seal_open
  * initStreamPull / pullStreamChunk wrap the secretstream pull API,
    throwing on authentication failure rather than returning false

All 32 tests pass; make check is green.
2026-05-09 12:44:59 -07:00
676d42c5eb Phase 2 red: crypto primitive tests and stub modules
Tests for the entire crypto/ public surface, written against the API
shape declared in the README. The accompanying src/crypto/ modules are
stubs that throw 'not implemented' so the test files compile and tests
fail with clear errors rather than module-not-found.

Tests cover:
  * init() resolves and is idempotent
  * fromBase64 / toBase64 / toBase64URL round-trips, including URL-safe
    input with stripped padding (the form Ente uses for auth tokens)
  * deriveKEK matches sodium.crypto_pwhash with Argon2id parameters
  * deriveLoginSubkey matches sodium.crypto_kdf_derive_from_key with
    subkey id 1 and ctx 'loginctx', truncated to 16 bytes
  * decryptBox round-trips, rejects tampering, wrong key, wrong nonce
  * decryptSealed round-trips, rejects wrong keypair and tampering
  * Secretstream pull decrypts multi-chunk streams in order, exposes
    per-chunk tags, rejects tampering, wrong key, and out-of-order chunks
  * Constants STREAM_CHUNK_SIZE (4 MiB) and STREAM_CHUNK_OVERHEAD (17)

Tests are commented to serve as the canonical API documentation per the
README development workflow policy. Verified: 29 tests fail (red), 3
trivial constant tests pass; lint and fmt-check are green.

eslint.config.mjs is updated to honour the leading-underscore convention
for intentionally unused parameters (the stubs).
2026-05-09 12:43:52 -07:00
64a3ace33a Add libsodium-wrappers-sumo runtime dependency
The 'sumo' build is required for crypto_pwhash (Argon2id), which Ente
uses to derive the KEK from the user's password. Pinned to exact
versions in package.json with integrity hashes in yarn.lock per repo
policy on hash-pinned external references.
2026-05-09 21:40:38 +02:00
5fc01f4558 Merge: relax pre-commit hook to support TDD red-phase commits 2026-05-09 21:39:34 +02:00
0d504294b5 Pre-commit hook runs lint+fmt-check, not full check (TDD)
Tests are deliberately excluded from the pre-commit hook so that the
TDD red-phase commit (failing tests landing before implementation) can
land on a feature branch. CI still runs full make check via docker build
so a red branch cannot reach main.
2026-05-09 21:39:32 +02:00
22c80d07ee Merge: TDD development workflow 2026-05-09 21:37:40 +02:00
3ccb00d6c8 Mandate TDD in README development workflow
All changes go on a feature branch; the first commit on the branch is the
failing test suite for the change; the branch can only merge to main when
make check is green. Tests are the canonical API documentation and must be
commented thoroughly so a reader can learn the library from them.
2026-05-09 21:37:33 +02:00
b35c677e23 Merge initial scaffolding
Initial repo scaffolding per sneak/prompts NEW_REPO_CHECKLIST: WTFPL
LICENSE, REPO_POLICIES.md, editor and prettier dotfiles, JS toolchain
(TypeScript, ESLint, Prettier, Vitest with pinned versions), Makefile,
Dockerfile (node:22-alpine pinned by sha256), Gitea Actions workflow.
make check and make docker both pass.
2026-05-09 21:33:08 +02:00
42cc1f4a77 Tick off Phase 1 scaffolding TODO items in README 2026-05-09 21:32:32 +02:00
d64db876ca Add Gitea Actions workflow that runs docker build on push 2026-05-09 21:31:01 +02:00
7d087ba7f9 Add Dockerfile pinning node:22-alpine by sha256
Non-server image: brings up the dev environment and runs make check, per
repo policy on Dockerfiles for non-server repos. Base image hash matches
the sneak/prompts template (node 22-alpine, 2026-02-22).
2026-05-09 21:31:00 +02:00
d7d3cd1af7 Add Makefile with check, build, docker, hooks targets
Targets: test (vitest, with conditional verbose rerun pattern), lint
(eslint + prettier), fmt / fmt-check (prettier), check (test + lint +
fmt-check), build (tsc), dev (tsc --watch), clean, docker, hooks.

Uses GNU timeout when available to hard-cap make test at 30 seconds, per
repo policy. Falls through without a cap on systems where timeout is
absent.
2026-05-09 21:30:34 +02:00
47cdb5ca8b Add JS toolchain: TypeScript, ESLint, Prettier, Vitest
package.json declares the project as ESM with NodeNext module resolution,
exposing dist/index.js as the library entry and dist/bin/quack.js as the
CLI binary. Dev dependencies are pinned to exact versions (yarn.lock holds
the integrity hashes per repo policy on hash-pinned external references).
Adds a placeholder src/index.ts and a single smoke test so make check is
not a no-op.
2026-05-09 21:29:13 +02:00