8 Commits

Author SHA1 Message Date
0cbe338b58 Add failing tests for the download retry policy
Tests only; the retry module they import does not exist yet, so the
branch is red at this commit.

New test/retry/retry.test.ts documents the classifier and the backoff:
which errors are worth another attempt, which are not, and how the delay
before each retry is derived. It asserts on the arguments handed to an
injected sleep function rather than on elapsed time, so the suite never
waits and the numbers are exact.

test/api/client.test.ts gains the request-count contract for each of the
six call sites, the deadline behaviour, the ApiError typing that the
presigned PUT and the null-body paths need in order to be classified at
all, and the replay rule for the two non-idempotent methods.

test/download/download.test.ts gains the case that motivates the whole
design: a socket reset after the response headers arrived, which happens
below ApiClient and can only be caught by retrying the request, the
stream consumption and the decryption together. It also pins that a
retried download stages exactly one temp file, and that the two retry
layers do not compose into a multiplied request budget. The existing
truncation tests now assert on the error type rather than its wording,
since that type is what the classifier reads.

test/thumbnails/thumbnails.test.ts separates a genuine 404 from an
exhausted retry, so a failing server can no longer make
fix-missing-thumbnails re-upload thumbnails that already exist.
2026-08-09 05:11:07 +00:00
25d3c612cf Replace sharp with jpeg-js + exif-reader; add bun compile binary
sharp was the only native dependency preventing a single-file binary.
Replaced with:
  - jpeg-js (pure JS) for JPEG decode/resize/encode in thumbnail gen
  - exif-reader (pure JS) for EXIF tag parsing
  - Raw JPEG APP1 marker extraction for EXIF segment discovery
  - Raw XMP packet extraction from file bytes

make build-bin produces a ~59MB self-contained Mach-O binary via
bun build --compile (bun installed via nix-shell). Zero runtime
dependencies. Tested: login, whoami, collections, files all work
from the compiled binary.

bin/quak.ts: init() called once at program start before commander
parses, so libsodium is ready for all commands including those that
restore sessions from disk.

118 tests pass.
2026-06-10 10:44:26 -07:00
5e6069f574 ML data always included in backup-metadata; remove --no-ml
ML metadata (face detections, CLIP embeddings) is not a separate
category from the rest of the metadata. It is always fetched and
included. The only opt-in is --exif (or --all) which requires
downloading every file for EXIF extraction.
2026-06-09 17:42:30 -04:00
21a1a78f07 ML data included by default, --exif is the opt-in, --all aliases --exif
ML data (face detections, CLIP embeddings) is now fetched by default
in backup-metadata. Use --no-ml to skip it. EXIF extraction (which
requires downloading every file) remains opt-in via --exif. --all is
an alias for --exif.
2026-06-09 17:38:15 -04:00
c8e7971445 Add --ml and --exif flags to backup-metadata
--ml fetches face detections and CLIP embeddings from the /files/data/fetch
endpoint (type 'mldata'). Each blob is encrypted with the file's key and
gzipped; we decrypt with decryptBlob, gunzip, and include the parsed JSON
as 'mlData' in the per-file output. Fetched in batches of 200 file IDs.

--exif downloads each file, runs sharp().metadata() to extract image
properties (format, dimensions, color space, orientation), then parses
the raw EXIF buffer with exif-reader for structured tags (lens, ISO,
shutter, aperture, GPS altitude, etc.). Also captures raw IPTC, XMP,
and ICC profile data. Included as 'imageMetadata' in the per-file output.

Without either flag, behavior is unchanged (fast metadata-only dump).

Adds exif-reader 2.0.3 as a runtime dependency.
3 new tests (ML data decrypted, ML data absent when flag not set, EXIF
extraction). 119 total tests, all green.
2026-06-09 17:35:35 -04:00
f3958e911d Add quak backup-metadata: dump all decrypted metadata to plain JSON
New command: quak backup-metadata <dir>

Dumps every piece of decrypted account metadata into a directory tree
of plain JSON files without downloading any file content. Layout:

    <dir>/
        account.json                    { email, userID }
        collections/
            <id>-<name>/
                _collection.json        { id, name, type, pubMagicMetadata?, ... }
                <fileID>.json           { id, metadata, magicMetadata?, pubMagicMetadata? }

Also adds collection-level magic metadata decryption (magicMetadata,
pubMagicMetadata, sharedMagicMetadata) to decryptCollection, which was
previously only done for files. The server sends these for visibility
settings, sort order, cover photo selection, etc.

6 new tests covering: account.json, per-collection dirs with
_collection.json, collection pubMagicMetadata decryption, per-file
JSON with all three metadata layers, graceful handling of files with
no magic metadata, and incremental re-run safety. 116 total.
2026-06-09 12:41:34 -04:00
8ee1be1cc2 CLI: quak login + quak backup with dedup symlink layout
bin/quak.ts: commander-based CLI with login (interactive + QUAK_EMAIL/
QUAK_PASSWORD env vars), whoami, logout, backup commands. Session
stored at env-paths('quak').data/session.json (~/Library/Application
Support/quak/ on macOS, XDG on Linux).

src/backup.ts: runBackup downloads all files into originals/<id>.<ext>,
symlinks into collections/<name>/<title>, writes per-collection JSON
metadata at collections/<name>.json. Deduplicates across collections
(each file downloaded once). Skips existing originals on incremental
runs. Never crashes on single-file failure.

4 backup tests + live-tested against real Ente account.
2026-05-13 18:47:06 -07:00
30a13eeeaf CLI red: backup tests, commander + env-paths deps, stub
4 tests for runBackup: full download into collection-named dirs,
incremental skip of existing files, resilient continuation after
single-file HTTP 500, and metadata.json output.

Adds commander 14.0.3 and env-paths 4.0.0 as runtime deps.
2026-05-13 18:36:07 -07:00