ML data (face detections, CLIP embeddings) is now fetched by default
in backup-metadata. Use --no-ml to skip it. EXIF extraction (which
requires downloading every file) remains opt-in via --exif. --all is
an alias for --exif.
--ml fetches face detections and CLIP embeddings from the /files/data/fetch
endpoint (type 'mldata'). Each blob is encrypted with the file's key and
gzipped; we decrypt with decryptBlob, gunzip, and include the parsed JSON
as 'mlData' in the per-file output. Fetched in batches of 200 file IDs.
--exif downloads each file, runs sharp().metadata() to extract image
properties (format, dimensions, color space, orientation), then parses
the raw EXIF buffer with exif-reader for structured tags (lens, ISO,
shutter, aperture, GPS altitude, etc.). Also captures raw IPTC, XMP,
and ICC profile data. Included as 'imageMetadata' in the per-file output.
Without either flag, behavior is unchanged (fast metadata-only dump).
Adds exif-reader 2.0.3 as a runtime dependency.
3 new tests (ML data decrypted, ML data absent when flag not set, EXIF
extraction). 119 total tests, all green.
New command: quak backup-metadata <dir>
Dumps every piece of decrypted account metadata into a directory tree
of plain JSON files without downloading any file content. Layout:
<dir>/
account.json { email, userID }
collections/
<id>-<name>/
_collection.json { id, name, type, pubMagicMetadata?, ... }
<fileID>.json { id, metadata, magicMetadata?, pubMagicMetadata? }
Also adds collection-level magic metadata decryption (magicMetadata,
pubMagicMetadata, sharedMagicMetadata) to decryptCollection, which was
previously only done for files. The server sends these for visibility
settings, sort order, cover photo selection, etc.
6 new tests covering: account.json, per-collection dirs with
_collection.json, collection pubMagicMetadata decryption, per-file
JSON with all three metadata layers, graceful handling of files with
no magic metadata, and incremental re-run safety. 116 total.
list-missing-thumbnails: iterates all files across all collections,
fetches each thumbnail from the CDN, reports any that are missing or
empty. Deduplicates by file ID across collections.
fix-missing-thumbnails: for each missing thumbnail, downloads the
original file, generates a 720px JPEG thumbnail via sharp, encrypts
it with secretstream push (encryptBlob), uploads to a presigned URL,
and registers the new thumbnail via PUT /files/thumbnail.
New crypto: encryptBlob (secretstream push, single chunk TAG_FINAL).
New ApiClient methods: getUploadURL, putFile, putJSON, updateThumbnail.
New Client method: getApiClient() for modules that need raw API access.
Deps: sharp 0.34.5 (image processing), @types/sharp 0.32.0.
Complete CLI surface:
quak login interactive or QUAK_EMAIL/QUAK_PASSWORD
quak whoami print logged-in account
quak logout delete session
quak collections list all albums (--json)
quak files list files in a collection (--json)
quak get <id> download+decrypt a file (--out, --collection)
quak get-thumb <id> download+decrypt a thumbnail
quak backup <dir> full incremental backup
get/get-thumb search all collections for the file ID when --collection
is not specified. All listing commands support --json.
Live-tested: collections list, file list, single file download (472 KB
JPEG from the dev account, verified as valid JPEG with EXIF intact).
bin/quak.ts: commander-based CLI with login (interactive + QUAK_EMAIL/
QUAK_PASSWORD env vars), whoami, logout, backup commands. Session
stored at env-paths('quak').data/session.json (~/Library/Application
Support/quak/ on macOS, XDG on Linux).
src/backup.ts: runBackup downloads all files into originals/<id>.<ext>,
symlinks into collections/<name>/<title>, writes per-collection JSON
metadata at collections/<name>.json. Deduplicates across collections
(each file downloaded once). Skips existing originals on incremental
runs. Never crashes on single-file failure.
4 backup tests + live-tested against real Ente account.
