sneak/quak
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

next → main: docs brought in line with the code #134

Open
clawbot wants to merge 3 commits from next into main
pull from: next
merge into: :main
Conversation 0 Commits 3 Files Changed 2
+191 -118
2 changed files with 191 additions and 118 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+145 -115
View File
@@ -8,9 +8,9 @@ and downloads individual images while decrypting them on the way to disk.
quak also includes a resilient backup command that downloads every file in the
account into a deduplicated local directory tree, skipping files that already
exist on disk and continuing past individual download failures instead of
crashing. It decrypts and persists all three metadata layers (basic, private
magic, public magic) per file, including camera info, GPS coordinates, captions,
and any face/keyword labels the Ente clients have added. A helper subcommand can
crashing. For each file it persists the basic metadata fields quak keeps (title,
file type, creation and modification time, latitude, longitude, content hash),
and the private and public magic metadata in full. A helper subcommand can
detect and regenerate missing thumbnails, encrypting and uploading them back to
the server.
@@ -51,7 +51,8 @@ const client = await Client.login({
// Open a cache-backed library. On an empty cache this awaits one server
// refresh; on an existing cache it returns immediately and refreshes in the
// background every `refreshIntervalSeconds` (default 3).
// background. Later refreshes start `refreshIntervalSeconds` (default 3) after
// the previous one ends.
const lib = await Library.open({ client });
// Default reads answer synchronously from the local cache — no network.
@@ -62,7 +63,7 @@ for (const album of lib.albums.list()) {
}
}
// Fresh reads await a server round-trip and answer with current state.
// Fresh reads await a server round-trip, joining one already running.
const { albums } = await lib.fresh();
console.log(`${albums.list().length} albums as of now`);
@@ -84,7 +85,7 @@ enumeration/download calls) is exported too and documented under Design below.
This repository adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard: normalized scripts in `script/` are the entrypoints for the
development workflow, and the Makefile targets are thin shims that call them.
development workflow, and most Makefile targets are thin shims that call them.
The scripts are POSIX sh (not bash) so they run in minimal containers such as
alpine. We provide:
@@ -122,9 +123,9 @@ alpine. We provide:
Linting and testing are phases of the `Dockerfile`. The `lint` phase copies the
repo into a digest-pinned node image and runs eslint and `prettier --check .`;
the `test` phase does the same with the suite. `script/lint` and `script/test`
each build one phase with `docker build --no-cache --target <phase>`. There is
no host lint or test path: docker is required, and that also works where the
docker daemon is remote and bind mounts are impossible.
each build one phase with `docker build --no-cache --target <phase>`. Neither
script runs the tools on the host: docker is required, and that also works where
the docker daemon is remote and bind mounts are impossible.
The last stage of the `Dockerfile` compiles the package, and it copies a file
from each phase, so it cannot be built unless lint and the tests pass. That is
@@ -170,16 +171,19 @@ local cache is reliable, and the UI is responsive on a five-year-old laptop.
All work on quak is test-driven. No exceptions.
1. Every change starts on a feature branch off `main`.
1. Every change starts on a feature branch off `next`, and its pull request
targets `next`.
2. The first commit on the branch is the test suite for what is being added or
changed. Those tests must fail at that commit; the branch is red until the
implementation lands.
3. Subsequent commits add the implementation and any refactors needed to make
the tests pass.
4. A feature branch can only be merged into `main` when `make check` is green.
`main` is always green. CI runs `script/cibuild`, which builds the
`Dockerfile`: its `lint` and `test` phases, then the compile, so neither a
red branch nor one that does not compile can pass CI.
4. A pull request can only be merged into `next` when `make check` is green.
Once it has passed review, the repository manager squash-merges it into
`next`. Only sneak merges `next` into `main`. `main` and `next` are always
green. CI runs `script/cibuild`, which builds the `Dockerfile`: its `lint`
and `test` phases, then the compile, so neither a red branch nor one that
does not compile can pass CI.
5. Tests are the canonical API documentation for this library. Every test file
is commented thoroughly enough that a reader who has never seen quak can
learn how to use it from the tests alone. Comments explain why a behavior
@@ -197,7 +201,7 @@ All work on quak is test-driven. No exceptions.
not the tests, and so not the full `make check`. This is deliberate so the
TDD red-phase commit (failing tests, no implementation yet) can land. The
`test` phase is part of the image build, which is what CI executes via
`script/cibuild`, so a red branch still cannot reach `main`.
`script/cibuild`, so a red branch still cannot reach `next`.
## Design
@@ -209,7 +213,7 @@ the CLI is for humans.
```
quak/
src/
crypto/ libsodium primitives (boxes, secretstreams, KDF, SRP)
crypto/ libsodium primitives (boxes, secretstreams, KDF, hash)
api/ HTTP client (ApiClient class)
auth/ login flow (SRP + email OTP + TOTP), key unwrap
model/ decrypted Collection, File, Metadata types + decrypt fns
@@ -219,7 +223,7 @@ quak/
and search, request pools
backup.ts resilient full-account backup with dedup
metadata-backup.ts
backup-metadata: all decrypted metadata as JSON
backup-metadata: the metadata quak keeps, as JSON
mldata-fetch.ts fetch + decrypt per-file ML data
filename.ts safe file names from server metadata
errors.ts error types shared across layers
@@ -250,7 +254,9 @@ the repository root rather than `src/`, because `bin/` is compiled too and
### Cryptography
All cryptography is done by `libsodium-wrappers-sumo` (the "sumo" build is
required for `crypto_pwhash` / Argon2id). No hand-rolled crypto.
required for `crypto_pwhash` / Argon2id), except the SRP handshake, which uses
`fast-srp-hap`, and the MD5 checksum sent with a thumbnail upload, which uses
Node's built-in `node:crypto`. No hand-rolled crypto.
The key hierarchy, derived during login, is:
@@ -258,19 +264,23 @@ The key hierarchy, derived during login, is:
2. Argon2id (`crypto_pwhash`) over the password and a server-issued `kekSalt`,
with server-issued `memLimit` and `opsLimit`, produces a 32-byte Key
Encryption Key (KEK).
3. SRP login: a 16-byte SRP login subkey is derived from the KEK using
`crypto_kdf_derive_from_key` (BLAKE2b) with subkey id 1 and context
`loginctx`. That 16-byte value is the SRP password.
4. After SRP completes (or after email-OTP fallback), the server returns a blob
of "key attributes" plus an encrypted auth token.
3. SRP login: `crypto_kdf_derive_from_key` (BLAKE2b) derives a 32-byte subkey
from the KEK with subkey id 1 and context `loginctx`. Its first 16 bytes are
the SRP password.
4. When SRP completes, the server returns a blob of "key attributes" plus an
encrypted auth token, or first asks for a second factor. quak answers a TOTP
request with the code (`POST /users/two-factor/verify`), after which the
server returns them, and cannot answer a passkey request. When the account
has email MFA on (`isEmailMFAEnabled`), an email OTP replaces SRP and the
server returns them after it.
5. `crypto_secretbox_open_easy` over the encrypted master key with the KEK
yields the 32-byte master key.
6. `crypto_secretbox_open_easy` over the encrypted secret key with the master
key yields the user's X25519 private key. The matching public key is
delivered in cleartext.
7. `crypto_box_seal_open` over the encrypted token with the user's keypair
yields the URL-safe base64 auth token used in `X-Auth-Token` for all
subsequent calls.
yields the auth token's bytes. Encoded as URL-safe base64 with padding, they
are the `X-Auth-Token` value for all subsequent calls.
Per-collection keys are decrypted with `crypto_secretbox_open_easy` using the
master key (for owned collections). Per-file keys are decrypted with
@@ -306,7 +316,8 @@ Endpoints used:
- `POST /users/srp/create-session`: begin SRP handshake.
- `POST /users/srp/verify-session`: complete SRP, receive 2FA challenge or the
encrypted token plus key attributes.
- `POST /users/ott` and `POST /users/verify-email`: email OTP fallback path.
- `POST /users/ott` and `POST /users/verify-email`: email OTP, used instead of
SRP when the SRP attributes have `isEmailMFAEnabled` set.
- `POST /users/two-factor/verify`: TOTP second factor.
- `POST /users/logout`: end the calling token's session (`quak logout`).
- `GET /collections/v2?sinceTime=<usec>`: list collections changed since
@@ -328,8 +339,9 @@ request is repeated only when repeating it could produce a different answer:
- Every other 4xx: not retried. A 404 in particular is an answer, and
`listMissingThumbnails` depends on getting it promptly and once.
- Transport failures — a `fetch` rejection, `ECONNRESET`, `ETIMEDOUT`, a DNS or
TLS failure — and deadline aborts: retried. The errno is looked for in the
error's `cause` chain, because that is where Node's `fetch` puts it.
TLS failure — and deadline aborts: retried. Node's `fetch` rejects with a
plain `TypeError`, so every `TypeError` is retried. The errno is looked for in
the error's `cause` chain, because that is where Node's `fetch` puts it.
- A truncated download: retried.
- Anything else, including a secretstream authentication failure that is not
truncation: not retried. The default answer is no. For a backup tool, retrying
@@ -395,10 +407,13 @@ because a socket reset after the response headers have arrived surfaces in the
download layer rather than in `ApiClient`, and that is the common failure for
multi-megabyte photos over a CDN. The secretstream pull state is not resumable
and these endpoints have no Range support, so a retry starts the file over. The
atomic write stays outside the retry, so a download that needed three attempts
still performs exactly one write and one rename. `runBackup` and
`runMetadataBackup` are unchanged: the retry sits below them, and a file that
fails after exhausting it is still logged, counted, and stepped over.
atomic write is part of the retried unit: each attempt writes its own temporary
files, one for most files and two for a live photo (its image and its video),
and removes them if it fails. Only the attempt that completes renames anything
into place. The retry sits below `runBackup` and `runMetadataBackup`.
`runBackup` logs, counts and steps over a file that still fails after its
retries; `runMetadataBackup`, which downloads only with `--exif`, records the
error in that file's JSON as `imageMetadataError` and goes on.
One imprecision is deliberate and worth knowing about. When a body ends part-way
through a secretstream chunk, Poly1305 fails and carries no framing signal, so a
@@ -422,10 +437,11 @@ base64-encoded keys) that the consumer can write to disk, a database, or
whatever else fits their use case. `Client.fromJSON(snapshot)` restores a
working client from that snapshot without re-authenticating; it checks every
field and each key's length first, and throws an error naming the bad field.
`client.logout()` clears the token and zeroes the key buffers in place; every
later call on that client throws. It does not contact the server, so the token
stays valid there and in any saved snapshot; `await client.logoutOnServer()`
first ends the session on the server (`POST /users/logout`).
`client.logout()` clears the token and zeroes the key buffers in place; after
it, every other method on that client throws. It does not contact the server, so
the token stays valid there and in any saved snapshot;
`await client.logoutOnServer()` first ends the session on the server
(`POST /users/logout`).
The CLI stores the snapshot at the platform-appropriate data directory via
`env-paths`: `~/Library/Application Support/quak/session.json` on macOS,
@@ -433,41 +449,43 @@ The CLI stores the snapshot at the platform-appropriate data directory via
`0600`. The key material is stored in cleartext in the JSON; treat this file as
you would treat the password itself. A missing file is reported as "not logged
in"; a file that exists but is corrupt is reported as such, naming the bad
field. Both exit with status 1.
field. Both exit with status 1, except that `quak logout` with no file says
there is no session and exits 0.
`quak logout` ends the session on the server, so the token in `session.json`
stops working even in a copy of the file, and then deletes the file. If the
server call fails (or the file is corrupt), the file is still deleted, the
command says the server session could not be ended, and it exits with status 1.
It does not delete the cache: it prints the account's cache directory and says
it still holds decrypted data (file keys in `metadata.json`, cached originals
and thumbnails), for the user to delete if they want it gone.
It does not delete the cache. When it knows the cache directory, from
`--cache-dir` or from a session file it could read, it prints it and says it
still holds decrypted data (file keys in `metadata.json`, cached originals and
thumbnails), for the user to delete if they want it gone.
### CLI surface
```
quak [--cache-dir <path>] <command> global: local metadata/content cache location
quak login interactive or QUAK_EMAIL/QUAK_PASSWORD
quak whoami print logged-in account as JSON
quak logout end the session, delete it
quak collections [--json] list all collections
quak files --collection <id> [--json] list files in a collection
quak get <fileID> [--out path] [--collection] download and decrypt a file
quak get-thumb <fileID> [--out] [--collection] download and decrypt a thumbnail
quak backup <dir> [--json] full incremental backup
quak backup-metadata <dir> [--exif] dump all decrypted metadata as JSON
quak helper list-missing-thumbnails [--json] find files with missing thumbnails
quak helper fix-missing-thumbnails [--file ids] generate + upload missing thumbnails
quak [--cache-dir <path>] <command> global: local metadata/content cache location
quak login interactive or QUAK_EMAIL/QUAK_PASSWORD
quak whoami print logged-in account as JSON
quak logout end the session, delete it
quak collections [--json] list all collections
quak files --collection <id> [--json] list files in a collection
quak get <fileID> [--out path] [--collection] download and decrypt a file
quak get-thumb <fileID> [--out] [--collection] download and decrypt a thumbnail
quak backup <dir> [--json] full incremental backup
quak backup-metadata <dir> [--exif] dump the metadata quak keeps as JSON
quak helper list-missing-thumbnails [--json] find files with missing thumbnails
quak helper fix-missing-thumbnails [--file ids] [--json] generate + upload missing thumbnails
```
Every command runs on the same cache-backed library. The read commands —
`collections`, `files`, `get`, `get-thumb`, `backup-metadata`,
`helper list-missing-thumbnails` and `helper fix-missing-thumbnails` — force a
fresh server round-trip before they answer, so they report current account state
rather than whatever the cache last held. If that round-trip fails, the command
prints the error on one line and exits 1. `--cache-dir` overrides where the
cache lives; without it each account gets its own directory under the per-user
cache path.
Every command except `login`, `whoami` and `logout` runs on the cache-backed
library. The read commands — `collections`, `files`, `get`, `get-thumb`,
`backup-metadata`, `helper list-missing-thumbnails` and
`helper fix-missing-thumbnails` — force a fresh server round-trip before they
answer, so they report current account state rather than whatever the cache last
held. If that round-trip fails, the command prints the error on one line and
exits 1. `--cache-dir` overrides where the cache lives; without it each account
gets its own directory under the per-user cache path.
`get` and `get-thumb` resolve the file by ID directly, so `--collection` is
accepted for backward compatibility but ignored. For a live photo, `get` writes
@@ -475,18 +493,20 @@ its image and its video, each named after the title with its own extension, as
Ente's clients name them (`IMG_0001.heic` and `IMG_0001.mov`). With
`--out PATH`, the image is written to `PATH` and the video beside it, with
`PATH`'s name and the video's extension; a `PATH` with the video's extension is
refused. `backup-metadata --exif` (alias `--all`) additionally downloads each
file to extract full EXIF/IPTC/XMP metadata. The listing and backup commands
support `--json` for machine-readable output.
refused. `backup-metadata --exif` (alias `--all`) additionally fetches each
file's original through the cache and records its XMP metadata and, for a JPEG,
its EXIF metadata and dimensions. `collections`, `files`, `backup`,
`helper list-missing-thumbnails` and `helper fix-missing-thumbnails` take
`--json` for machine-readable output.
`backup-metadata` fetches ML data in requests of up to 200 files. When a request
still fails after its retries, the error is logged, each of its files is written
with the reason in an `mlDataError` field instead of `mlData`, and the dump goes
on. The exit code is non-zero if any ML data request failed.
fails, the error is logged, each of its files is written with the reason in an
`mlDataError` field instead of `mlData`, and the dump goes on. The exit code is
non-zero if any ML data request failed.
`helper fix-missing-thumbnails` regenerates thumbnails for baseline JPEG images
only, because the bundled decoder (`jpeg-js`) decodes only JPEG. A non-JPEG
image (PNG, HEIC) or a video is reported as `skipped` (unsupported format), kept
`helper fix-missing-thumbnails` regenerates thumbnails for JPEG images only,
because the bundled decoder (`jpeg-js`) decodes only JPEG. A non-JPEG image
(PNG, HEIC) or a video is reported as `skipped` (unsupported format), kept
distinct from a `failed` repair, and does not affect the exit code; a genuine
failure still exits non-zero. The server accepts a new thumbnail only from the
file's owner and only when it is no larger than the thumbnail size it records
@@ -505,7 +525,9 @@ the smallest does not.
originals/
<fileID>.<ext> actual file content (one per unique file,
two for a live photo: see below)
<fileID>.json all decrypted metadata for that file
<fileID>.json the file's basic metadata fields quak
keeps, and its private and public magic
metadata
<fileID>.livephoto.json which of a live photo's two files is which
collections/
<name>/
@@ -550,7 +572,9 @@ the cache. An original the cache already held is copied from there instead. On
subsequent runs, existing originals are skipped. If a download fails, the error
is logged and the backup continues with the next file. The exit code is non-zero
if any files failed. `quak backup` opens its library with the thumbnail and
originals precache off, so it fetches only what the backup stores.
originals precache off, so the only file content it fetches is the originals the
backup stores. The library's ML data fetch still runs and fills the cache's
`mldata/`.
Each original is written to a temporary file in the same directory, synced to
disk, and renamed into place, so an original is either complete or absent, even
@@ -567,8 +591,8 @@ temporary file's permissions, not those of the file it replaced.
## TODO
- [x] Retry policy: no retry on 4xx, exponential backoff on 5xx and network
errors
- [x] Retry policy: no retry on 4xx (except `408` and `429`), exponential
backoff on 5xx and network errors
- [x] Update the API reference section below to match the current implementation
- [x] `make docker` green
- [x] Store live photos in a form a photo viewer can open
@@ -591,7 +615,8 @@ Future (desktop client, separate repo):
The library's primary surface is the cache-backed `Library`; the lower-level
`Client` sits underneath it and is covered by the Design sections above. The
test suite is the canonical, executable documentation — `test/library/` and
`test/client/usage.test.ts` walk every operation, and `yarn test` verifies them.
`test/client/usage.test.ts` walk most operations, `test/cli/backup.test.ts`
walks `lib.backup()`, and `yarn test` verifies them.
### Opening a library
@@ -604,21 +629,21 @@ background, so an unreachable server does not block opening.
`LibraryOptions`:
| Option | Default | Meaning |
| ------------------------ | --------------------------- | --------------------------------------------------------------------- |
| `client` | required | the account client (a `Client`, or any `LibraryClient`) |
| `cacheDirectory` | `<XDG cache>/quak/<userID>` | where `metadata.json` and the content cache live |
| `downloadDirectory` | none | backup destination; an original already stored there counts as cached |
| `refreshIntervalSeconds` | `3` | background refresh cadence |
| `precacheThumbnails` | `true` | prefetch every thumbnail, newest first |
| `precacheOriginals` | `true` | prefetch the favorites album and the latest-window originals |
| `precacheOriginalsDays` | `7` | length in days of that latest window |
| `cacheOriginalsMaxBytes` | 100 GiB | hard ceiling on the originals cache |
| `freeBelowBytes` | 50 GiB | free space to protect on the volume; the effective limit adapts down |
| `isOriginalPinned` | none | extra predicate for originals that must never be evicted |
| `pools` | fresh `RequestPools` | the bounded request pools (sets concurrency) |
| `onProgress` | none | refresh/ML/precache progress callback (`RefreshEvent`) |
| `contentSource` | the client's own | override the byte source (mainly for tests) |
| Option | Default | Meaning |
| ------------------------ | -------------------------------------------------- | --------------------------------------------------------------------- |
| `client` | required | the account client (a `Client`, or any `LibraryClient`) |
| `cacheDirectory` | `quak/<userID>` under the per-user cache directory | where `metadata.json` and the content cache live |
| `downloadDirectory` | none | backup destination; an original already stored there counts as cached |
| `refreshIntervalSeconds` | `3` | background refresh cadence |
| `precacheThumbnails` | `true` | prefetch every thumbnail, newest first |
| `precacheOriginals` | `true` | prefetch the favorites album and the latest-window originals |
| `precacheOriginalsDays` | `7` | length in days of that latest window |
| `cacheOriginalsMaxBytes` | 100 GiB | size limit on the originals cache; pinned originals can exceed it |
| `freeBelowBytes` | 50 GiB | free space to protect on the volume; the effective limit adapts down |
| `isOriginalPinned` | none | extra predicate for originals that must never be evicted |
| `pools` | fresh `RequestPools` | the bounded request pools (sets concurrency) |
| `onProgress` | none | refresh/ML/precache progress callback (`RefreshEvent`) |
| `contentSource` | the client's own | override the byte source (mainly for tests) |
Concurrency is set through `pools`: construct
`new RequestPools({ metadataConcurrency, contentConcurrency, thumbnailConcurrency })`
@@ -635,17 +660,21 @@ can then be removed.
### Default reads vs. fresh reads
Default reads — `lib.albums`, `lib.photos`, `lib.timeline` — answer
synchronously from the last refreshed copy held in RAM and never touch the
network. The background timer refreshes that copy every
`refreshIntervalSeconds`, so a default read is immediate but may be up to one
interval stale.
synchronously from the copy held in RAM and never touch the network. A refresh
changes that copy only once all its server requests have succeeded, and the
background timer starts the next refresh `refreshIntervalSeconds` after the
previous one ends. So a default read is immediate, but only as current as the
last refresh whose requests all succeeded; right after opening an existing
cache, it is the copy on disk.
`await lib.fresh()` forces a refresh, waits for it to complete and persist, and
returns the same `{ albums, photos, timeline }` namespaces — now guaranteed to
reflect a completed server round-trip. Concurrent `fresh()` calls coalesce onto
one refresh, and a refresh that fails rejects the caller (default reads stay
silent and keep serving the last good copy). The CLI's read commands use fresh
reads (issue https://git.eeqj.de/sneak/quak/issues/75).
`await lib.fresh()` waits for a refresh to complete and persist, and returns the
same `{ albums, photos, timeline }` namespaces, which then reflect a completed
server round-trip. When a refresh is already running, background or not,
`fresh()` waits for that one, so its answer can come from requests made before
the call; only when none is running does it start one. A refresh that fails
rejects the caller (default reads stay silent and keep serving the last good
copy). The CLI's read commands use fresh reads (issue
https://git.eeqj.de/sneak/quak/issues/75).
### Read surface
@@ -662,8 +691,8 @@ reads (issue https://git.eeqj.de/sneak/quak/issues/75).
`includeArchived`; hidden photos are always excluded.
An `Album` exposes its record fields and `album.photos.list()` → `Photo[]`
(newest first). A `Photo` exposes its record fields, `photo.record()` →
`PhotoRecord`, and two content methods:
(newest first). A `Photo` exposes its record fields other than `thumbnailPath`
and `originalPath`, `photo.record()` → `PhotoRecord`, and two content methods:
- `await photo.original(opts?)` → `{ path, bytes, videoPath? }` — the
full-resolution file. For a live photo, `path` and `bytes` are its image's and
@@ -712,16 +741,16 @@ photos newest first). `lib.subscribe({ onChange })` delivers a `LibraryChange`
`SimilarResult[]` (`{ fileID, score }`, cosine similarity, most similar first,
default limit 20). quak bundles no text encoder, so `searchByEmbedding` takes
a query vector the caller produced elsewhere.
- `await lib.backup(opts?)` → `BackupResult`. It refreshes, fetches every
in-scope original not already in the backup (and, with `includeThumbnails`,
thumbnails) through the content cache, and rebuilds the on-disk backup tree
with a durable failure ledger. A fetched original is written straight into the
backup's `originals/` and not into the cache, which then counts it as present;
one the cache already held is copied from there. `BackupOptions`:
`downloadDirectory` (falls back to the one `open()` was given),
`includeOriginals` (default `true`), `includeThumbnails` (default `false`),
`onlyAlbumNames`, and `onProgress`. See Backup layout above for the tree it
writes.
- `await lib.backup(opts?)` → `BackupResult`. It waits for a refresh as
`fresh()` does, fetches every in-scope original not already in the backup
(and, with `includeThumbnails`, thumbnails) through the content cache, and
rebuilds the on-disk backup tree with a durable failure ledger. A fetched
original is written straight into the backup's `originals/` and not into the
cache, which then counts it as present; one the cache already held is copied
from there. `BackupOptions`: `downloadDirectory` (falls back to the one
`open()` was given), `includeOriginals` (default `true`), `includeThumbnails`
(default `false`), `onlyAlbumNames`, and `onProgress`. See Backup layout above
for the tree it writes.
### Request pools
@@ -813,14 +842,15 @@ documents:
`yarn.lock`. Never `git add -A`. Never force-push to main.
- **The "Development workflow" section above.** All changes go on feature
branches. Tests are written first and committed in a failing state before the
implementation. Tests are the canonical API documentation and must be
commented thoroughly. `main` is always green.
branches off `next`, and every pull request targets `next`; only sneak merges
`next` into `main`. Tests are written first and committed in a failing state
before the implementation. Tests are the canonical API documentation and must
be commented thoroughly. `main` and `next` are always green.
- **Required checks before every commit:** `make lint` must pass — that is
eslint plus the prettier check, and it builds the `lint` phase of the
`Dockerfile`, so it needs docker. The pre-commit hook enforces exactly that.
`make check` (which also runs the tests) must pass before merging to `main`.
`make check` (which also runs the tests) must pass before merging into `next`.
`make fmt-check` is available for a host-side formatting check on its own, but
it is not a separate requirement: `make lint` already covers it, and running
both would check formatting twice. Never invoke eslint or prettier directly;
TODO.md
+46 -3
View File
@@ -1,12 +1,15 @@
# Workflow
- branch (from `main`)
- branch from `next`
- do the work in Next Step
- move Next Step to the top of Completed Steps
- move the top item of Future Steps into Next Step
- commit (`TODO.md` changes in the same commit as the work)
- merge to `main` if the branch is not protected, otherwise open a PR
- push
- open a pull request that targets `next`
- once the pull request has passed review, the repository manager squash-merges
it into `next`
- only sneak merges `next` into `main`
# Status
@@ -14,13 +17,53 @@ pre-1.0
# Next Step
None: every issue still open is done on `next` and waits for it to reach `main`.
None: no implementation work is open. The cache design,
https://git.eeqj.de/sneak/quak/issues/36, waits on sneak's review.
Tagging and releases are decided by sneak alone, and happen only when he
declares one.
# Completed Steps
- 2026-09-29: The "Workflow" list at the top of this file now says to branch
from `next` and open a pull request that targets `next`, that the repository
manager squash-merges a reviewed pull request into `next`, and that only sneak
merges `next` into `main`, as the README does (issue 137).
- 2026-09-29: The README's "Development workflow" and "For LLMs" sections now
say work branches from `next`, every pull request targets `next`, the
repository manager squash-merges reviewed pull requests into `next`, and only
sneak merges `next` into `main` (issue 135).
- 2026-09-29: Brought the README and this file in line with `next` after the
milestone merge (issue 132). The Next Step says no implementation work is open
and the cache design (issue 36) waits on sneak's review. README corrections:
the Getting Started comments say when the library refreshes; most, not all,
Makefile targets call a script; `script/lint` and `script/test` have no host
path, though `yarn test` does; the SRP handshake uses `fast-srp-hap`, outside
`crypto/`, and a thumbnail upload's MD5 uses `node:crypto`; the SRP password
is the first 16 bytes of a 32-byte subkey; the key attributes and token come
after SRP, after the TOTP code SRP may ask for, or after the email OTP that
replaces SRP when the account has email MFA on, and quak cannot answer a
passkey; the auth token is sent as URL-safe base64 with padding; every
`TypeError` is retried; each download attempt writes its own temporary files,
two for a live photo, and only the attempt that completes renames them into
place; `runMetadataBackup` records a failed download in the file's JSON; a
second `client.logout()` does not throw; `quak logout` with no session exits
0, and names the cache directory only when it knows it; `login`, `whoami` and
`logout` open no library; `--exif` records XMP and, for a JPEG, EXIF, and no
IPTC; which commands take `--json`; a failed ML data request may not have been
retried; the thumbnail fixer is not limited to baseline JPEG; `quak backup`
still fetches ML data; a backup's JSON holds the basic metadata fields quak
keeps and the private and public magic metadata, not every decrypted field,
and the README no longer lists what the magic metadata holds; the default
cache directory is the per-user one, not an XDG path on macOS; pinned
originals can exceed `cacheOriginalsMaxBytes`; which tests cover which
operations; a default read is only as current as the last refresh whose
requests all succeeded; `fresh()` and `lib.backup()` join a refresh already
running; a `Photo` has no `thumbnailPath` or `originalPath`; and `408` and
`429` are retried.
- 2026-09-28: Tested the live-photo writer's fsyncs (issue 130). A test checks
that the image's and the video's temp files are fsynced before either is
renamed into place, and the directory after both renames, as the `writeAtomic`