1 Commits
Author SHA1 Message Date
sneak 6d9852af13 README.md rewritten sentence by sentence (closes #219)
check / check (push) Waiting to run
The old text served as the list of what a reader needs covered. The new
text leads with what quak is, what it is for and how to use it. Every
command, option, exit code and file layout follows in reference sections,
then the library reference, then design, entrypoints and development.
Every statement was checked against the code on next; the places where
the old text disagreed with it are listed in the pull request. No sentence
runs past about 30 words, every table is now a list, and no issue links or
project history remain in text meant for users. Two code comments that
named README sections now name the new ones.

Model: fable-5-1
2026-10-26 12:11:36 +00:00
3 changed files with 158 additions and 202 deletions
+150 -191
View File
@@ -73,7 +73,8 @@ and `dist/` is not committed. Clone and build it as above, then add the clone to
your own program's `package.json` by its path, for example
`yarn add file:../quak`. The package's `main`, `types` and `exports` fields
point into `dist/`, so the clone must be built first. The package is an ES
module only, so a CommonJS program cannot `require` it.
module only: `package.json` sets `"type": "module"`, and its `exports` field
offers only `import`, so a CommonJS program cannot `require` it.
```ts
import { Client, Library } from "quak";
@@ -89,9 +90,8 @@ const client = await Client.login({
const lib = await Library.open({
client,
// Without these two, open() also starts downloading every thumbnail in
// the account, and the originals of the favorites album and the latest
// week, into the cache. It fetches machine-learning data after each
// refresh either way.
// the account and the latest week's originals into the cache. It fetches
// machine-learning data after each refresh either way.
precacheThumbnails: false,
precacheOriginals: false,
});
@@ -122,12 +122,10 @@ await lib.close();
for. It refreshes every 3 seconds until `close()`. It also downloads, in the
background, every thumbnail in the account and the originals of the favorites
album and of the latest week (see "Opening a library"). The example turns those
two downloads off. After each refresh that succeeds, the library also fetches
each file's machine-learning data into the cache. That is the faces and the
other data Ente's apps compute for a photo (see "The JSON beside each
original"). No option in `LibraryOptions` turns this off. A `client` without a
`fetchMLData` method fetches none, which is how quak's own read commands open
their library.
two downloads off. After each refresh the library also fetches each file's
machine-learning data into the cache. That is the faces and the other data
Ente's apps compute for a photo (see "The JSON beside each original"). No option
turns this off.
The `Client` under the library handles login, saved sessions and the raw
requests. The "Library reference" below covers both.
@@ -136,9 +134,9 @@ requests. The "Library reference" below covers both.
`quak backup` never prompts, so cron can run it. `make install` builds quak as
one binary and copies it to `~/bin/quak`. Besides what Getting Started needs,
`make install` needs `make` and [nix](https://nixos.org/download). It uses bun,
a JavaScript runtime, to bundle the built `dist/` into one executable. It runs
bun through `nix-shell -p bun`.
`make install` needs [nix](https://nixos.org/download). It uses bun, a
JavaScript runtime, to bundle the built `dist/` into one executable. It runs bun
through `nix-shell -p bun`.
Log in once with `~/bin/quak`, as the user the cron job will run as:
@@ -158,10 +156,9 @@ output to `~/quak-backup.log`. Sunday's run adds `--verify`:
A run with `--verify` does all a plain run does. It also hashes each original
already in the backup and compares the result with the content hash. The content
hash is the hash of the file's bytes that Ente recorded at upload. The run
fetches a new copy of each original that does not match, and replaces the
original once the copy matches. Sunday's run is the `--verify` run, rather than
a second job that night, because two backups of one directory cannot run at
hash is the hash of the file's bytes that Ente recorded at upload. It replaces
any original that does not match. Sunday's run is the `--verify` run, rather
than a second job that night, because two backups of one directory cannot run at
once. The second one exits 2 without doing anything.
Cron runs the job with a short `PATH`, usually `/usr/bin:/bin`, so the crontab
@@ -230,12 +227,9 @@ set, it asks on the terminal. The password shows as `*` characters while it is
typed.
When the account has a second factor, quak asks for the code. It asks
`TOTP code: ` for a time-based code from an authenticator app. An account whose
only second factor is a passkey cannot log in with quak. When the account has
email verification on, quak asks `Email verification code: ` for the code sent
by email. The password is then used only to unlock the keys. After the email
code quak expects the login to complete. When the server asks for a second
factor at that point, login fails.
`TOTP code: ` for a time-based code from an authenticator app, and
`Email verification code: ` for a code sent by email. An account whose only
second factor is a passkey cannot log in with quak.
On success it logs `Logged in as <email> (user <id>)` and
`Session saved to <path>`. On failure it logs `Login failed: <reason>`, and the
@@ -245,9 +239,7 @@ to stdout.
#### `quak whoami`
Prints the saved session's account on stdout as one line of JSON:
`{"email":"you@example.com","userID":12345}`. It reads the session file only and
sends no request, so it succeeds even when the server no longer accepts the
session.
`{"email":"you@example.com","userID":12345}`.
#### `quak logout`
@@ -314,16 +306,16 @@ refused, because both files would land on it.
#### `quak get-thumb <fileID> [--out <path>]`
Downloads and decrypts one file's thumbnail, the small JPEG Ente shows in a
grid. It writes one file, also for a live photo. Without `--out`, the name is
`thumb_` followed by the title. In all else it works as `get` does.
grid. It works as `get` does. Without `--out`, the name is `thumb_` followed by
the title.
#### `quak backup <dir> [--json] [--verify]`
Backs up every file in the account to `<dir>`. The layout is described under
"The backup directory". The command downloads only the originals that are not
yet in the backup, and it goes on past a file that fails. A file is downloaded
once, however many collections hold it. An original the cache already holds is
copied from there instead.
"The backup directory". It downloads only the originals that are not yet in the
backup, and it goes on past a file that fails. A file is downloaded once,
however many collections hold it. An original the cache already holds is copied
from there instead.
Progress goes to stderr: one line per original fetched, and a `FAILED` line for
each original or album that failed. The run ends with a summary of counts and,
@@ -362,8 +354,7 @@ Writes the metadata quak keeps about the account, its collections and its files
as JSON files under `<dir>`. The layout is described under "The metadata dump
directory". Photo content is not written. With `--exif`, or its alias `--all`,
the command also fetches each file's original and records the image's own
metadata from it (see "The metadata dump directory"). Each original it fetches
stays in the cache.
metadata from it (see "The metadata dump directory").
Ente's apps compute machine-learning data for each photo, such as the faces in
it (see "The JSON beside each original"). The command fetches it in requests of
@@ -378,13 +369,12 @@ word, and does not change the exit status.
#### `quak helper list-missing-thumbnails [--json]`
Finds the files whose thumbnail is missing on the server. It downloads the
thumbnail of every file you own, one collection at a time, newest updated first,
so on a large account it takes a while. A thumbnail counts as missing when the
Finds the files whose thumbnail is missing on the server. It downloads every
thumbnail in the account, one collection at a time, newest collection first, so
on a large account it takes a while. A thumbnail counts as missing when the
server answers with an empty body or with HTTP 404. Any other failure is logged
as `[<collection>] Could not check <title>: <reason> (not reported as missing)`,
and the file is left out of the list. The command exits 0 even when some files
could not be checked.
and the file is left out of the list.
The server lets only a file's owner replace its thumbnail. A file another
account owns, in a collection shared with you, is skipped and not checked.
@@ -427,13 +417,12 @@ Each file ends in one of three states:
`--file` is held by no collection.
Without `--json`, the command ends with counts of fixed, skipped and failed
files. A `Skipped:` list and a `Failed files:` list follow when they have
entries. With `--json`, one JSON array on stdout replaces them; each object
holds `fileID`, `title`, `collection`, `status` and, for a skipped or failed
file, `reason`. When any file failed, the exit status is 1. A skipped file does
not change it. When nothing needs fixing, the command logs
`No missing thumbnails found.` and exits 0. With `--json` it then writes nothing
on stdout.
files, a `Skipped:` list and a `Failed files:` list. With `--json`, one JSON
array on stdout replaces them; each object holds `fileID`, `title`,
`collection`, `status` and, for a skipped or failed file, `reason`. When any
file failed, the exit status is 1. A skipped file does not change it. When
nothing needs fixing, the command logs `No missing thumbnails found.` and
exits 0. With `--json` it then writes nothing on stdout.
quak refuses the whole command, before any request, when one `--file` ID is
invalid (see "IDs").
@@ -464,9 +453,9 @@ collection fetched, such as `Fetched the files of 3 of 120 collections`.
When a refresh or a search fails, the command logs the error on one line and
exits 1, or 3 when the server no longer accepts the saved session. A collection
or file that quak cannot decrypt does not stop a refresh or a search: the
command leaves it out and answers from what did decrypt (see "Default reads and
fresh reads"). `backup` counts it as failed (see "What quak cannot decrypt").
or file that quak cannot decrypt does not stop a refresh: the command answers
from what did decrypt (see "Default reads and fresh reads"). `backup` counts it
as failed (see "What quak cannot decrypt").
### IDs
@@ -478,11 +467,20 @@ before any request is sent. The command logs `Invalid file ID: <arg>` or
### What quak prints
stdout holds a command's output, such as the listing or the JSON, and the
prompts of `login`. Every other line goes to stderr as a log line at one of six
levels. From the quietest, the levels are fatal, error, warning, info, debug and
trace. A run prints the lines at its own level and at every quieter one. The
level is info unless `-v`, `-vv` or `-q` says otherwise.
stdout holds only a command's output: the listing, the JSON, or nothing. The
prompts of `login` go to stdout too. Every other line goes to stderr as a log
line at one of six levels. From the quietest, the levels are fatal, error,
warning, info, debug and trace. A run prints the lines at its own level and at
every quieter one. The level is info unless `-v`, `-vv` or `-q` says otherwise.
The one exception is a command line quak cannot parse, such as a missing
argument, an unknown option or `-q` with `-v`. quak refuses it with one
`error: …` line that is not a log line, and exits 1. `-q` does not hide that
line. `quak` with no command prints the usage on stderr and exits 1.
`yarn quak` prints yarn's own lines on stdout as well. To pipe a command's
output, run `yarn --silent quak files --json` or
`node dist/bin/quak.js files --json` instead.
- Fatal: why a command exits non-zero. A command that runs and exits non-zero
logs at least one fatal line. A command that exits 1 because files or albums
@@ -508,22 +506,12 @@ level is info unless `-v`, `-vv` or `-q` says otherwise.
one that exits non-zero prints only its fatal lines. The prompts of `login` are
not log lines, and `-q` leaves them alone.
Two things on stderr are not log lines. A command line that quak cannot parse,
such as a missing argument, an unknown option or `-q` with `-v`, is refused with
an `error: …` message, and quak exits 1. `-q` does not hide the message. `quak`
with no command prints the usage on stderr and exits 1.
`yarn quak` prints yarn's own lines on stdout as well. To pipe a command's
output, run `yarn --silent quak files --json` or
`node dist/bin/quak.js files --json` instead.
With `--json`, a command's JSON on stdout takes the place of its summary on
stderr. For `backup`, that is the `--- Backup complete ---` line, the counts and
the `Failed files:` list. For `list-missing-thumbnails`, it is
stderr. For `backup`, that is the lines from `--- Backup complete ---` to the
end of the `Failed files:` list. For `list-missing-thumbnails`, it is
`No missing thumbnails found.` or `<n> file(s) with missing thumbnails:`. For
`fix-missing-thumbnails`, it is the `--- Done ---` line, the counts, the
`Skipped:` list and the `Failed files:` list. Every other line is logged as
without `--json`.
`fix-missing-thumbnails`, it is the lines from `--- Done ---` to the end of the
`Failed files:` list. Every other line is logged as without `--json`.
### Exit codes
@@ -689,13 +677,13 @@ The JSON file beside an original holds:
Update times are in microseconds, as Ente records them.
The JSON also holds the machine-learning data Ente computed for the photo, as
The file also holds the machine-learning data Ente computed for the file, as
`mlData`. Ente's apps detect faces in each photo and compute a CLIP embedding
for it, a list of 512 numbers that describes what the photo shows. `mlData`
holds everything but the `clip` block, which holds that embedding. In its place
the JSON holds `keywords`, derived from the embedding as described under
"Keywords". A file with no machine-learning data has neither field. A file whose
data holds no usable embedding has no `keywords`.
holds everything but that embedding. In its place the file holds `keywords`,
derived from the embedding as described under "Keywords". A file with no
machine-learning data has neither field. A file whose data holds no usable
embedding has no `keywords`.
The backup waits for the library's fetch of machine-learning data to finish
before it writes the JSON files. When a file's data cannot be fetched,
@@ -748,11 +736,11 @@ derives a file's keywords the way the app sorts it. The categories are:
Each category has a query, and a text embedding of that query from Ente's CLIP
text model. A file's score for a category is the cosine similarity of the file's
embedding and the category's. That is a number from -1 to 1; the more alike the
two embeddings are, the higher it is. The file falls in the category when its
score reaches the category's minimum score. The keywords are the titles of the
categories the file falls in, highest score first. A video never falls in a
category marked "no videos", as in the app.
embedding and the category's: a number from -1 to 1, higher the more alike the
two embeddings are. The file falls in the category when its score reaches the
category's minimum score. The keywords are the titles of the categories the file
falls in, highest score first. A video never falls in a category marked "no
videos", as in the app.
#### Collection directories
@@ -775,21 +763,17 @@ ignoring case, each gets its ID added: two albums named `Trip` become
`Trip (10)/` and `Trip (11)/`. Two files in one collection whose titles match,
ignoring case, get their IDs added before the extension: `IMG_0001 (12345).JPG`
and `IMG_0001 (12346).JPG`. A collection named like another collection's JSON
file, such as `Trip.json` beside `Trip`, also gets its ID added. A file name
longer than 255 bytes, the limit on most file systems, gets its ID added too. So
does a collection name longer than 250 bytes, which leaves room for `.json`.
Either name is then cut short before the ID, just far enough to fit. IDs never
file, such as `Trip.json` beside `Trip`, also gets its ID added. A name too long
for a file name, which most file systems limit to 255 bytes, gets its ID added
too. It is then cut short before the ID, just far enough to fit. IDs never
change, so a name stays the same from run to run until such a clash appears or
goes away.
Each run removes every symlink in a collection's directory that points into the
date folders and is not one of its current links, yours included. A file or
symlink of yours at the name of one of its links is replaced by the link. A
directory of yours at that name stays, and the photo whose link it blocks goes
into `failures.json`. Anything else you put under `collections/` stays. A run
removes the directory and JSON of a collection that was deleted or renamed, once
the directory holds nothing else. One that still holds something of yours stays,
with its JSON.
Each run removes the symlinks that no longer belong in a collection's directory.
It removes the directory and JSON of a collection that was deleted or renamed,
once the directory holds nothing else. Nothing else under `collections/` is
touched: a file or symlink you put there stays, and a directory that still holds
one stays too, with its JSON.
A collection whose directory or JSON cannot be written does not stop the run.
For example, a file you put there may already have the directory's name. The run
@@ -834,9 +818,8 @@ Each entry holds:
- `fileID` and `title`;
- `classification`: the kind of failure;
- `attempts`: how many runs in a row have failed the file;
- `lastTriedAt`: when the run that last tried it finished its refresh, as a Unix
time in milliseconds;
- `attempts`: how many runs have tried the file;
- `lastTriedAt`: when it was last tried, as a Unix time in milliseconds;
- `error`: the error, as quak printed it.
The kinds are:
@@ -854,13 +837,13 @@ file outside them. `failures.json` is deleted when the list is empty.
`backup.lock` keeps two backups of the same directory from running at once, such
as a cron run that starts while the previous one is still going. A backup
creates `<dir>` if it is missing and takes the lock before its refresh, so a
refused run sends no request. A run that still holds the lock removes it when it
ends, whether it succeeded or failed.
refused run sends no request. It removes the lock when it ends, whether it
succeeded or failed.
The lock is a directory that
[proper-lockfile](https://github.com/moxystudio/node-proper-lockfile) creates.
While the backup runs, it updates the modification time of `backup.lock` every 5
seconds. A second backup of `<dir>` fails at once: it prints
While the backup runs, the backup updates the directory's modification time
every 5 seconds. A second backup of the directory fails at once: it prints
`quak: another backup of <dir> is running` and exits 2. A run that is killed, or
cut off by a crash, can leave the lock behind. Once the lock has gone 10 seconds
without an update, the next run takes it over, so nobody has to remove it by
@@ -903,13 +886,13 @@ The summary and the `--json` output add the counts `verified`, `mismatched` and
`unchecked`. Each counts only originals that were already stored. A mismatch
that was replaced with matching bytes does not make the exit code non-zero.
A run without `--verify` drops two kinds of failure from `failures.json`:
Two kinds of failure stay in `failures.json` only until the next run:
- a mismatched original whose new copy failed;
- a stored original that cannot be read.
That run finds the original in place, counts it as skipped and drops the entry,
so it can exit 0. Only a `--verify` run hashes the original again.
A run without `--verify` finds the original in place, counts it as skipped, and
drops the entry, so that run can exit 0. Only a `--verify` run hashes it again.
Without `--verify`, no stored original is hashed, the summary is unchanged, and
the three counts are 0 in the JSON.
@@ -935,11 +918,7 @@ scheme. Opening a library deletes such leftovers in the cache's `originals/` and
The rename replaces whatever was at the destination rather than writing through
it. A symlink there is replaced, not followed, and the new file has the
temporary file's permissions, not those of the file it replaced. Each original
is written with mode `0600`, readable by its owner only. Everything else the
backup writes, its folders and JSON files included, gets the mode your umask
allows, which usually lets every user on the machine read it. The JSON beside an
original holds the file's location and faces. The `backup-metadata` tree is
written the same way.
is written with mode `0600`, readable by its owner only.
### The metadata dump directory
@@ -964,9 +943,9 @@ of the same name apart.
`files`. Each `<fileID>.json` holds the fields a backup's JSON beside an
original holds, with these differences:
- `imageMetadata` is written only with `--exif`. The original of every file,
videos included, is then downloaded and read for its EXIF, XMP and JPEG
dimensions.
- `imageMetadata` is written only with `--exif`. The original of every file is
then downloaded, videos included, and read as a backup reads it (see "The JSON
beside each original").
- A file whose original holds no image metadata gets no `imageMetadata` field.
- An original that cannot be downloaded or read gives `imageMetadataError`
instead.
@@ -1047,24 +1026,23 @@ onto what the cache holds, empty or not. A refresh then starts only when
- `contentSource`: a replacement for the client's own byte source. Mainly for
tests.
In the content and thumbnail pools, your own requests take the next free slot
ahead of the two background downloads. A file already in the cache is not
fetched again. When the originals cache grows past its limit, the least recently
used originals are evicted, but never a pinned one, and never one whose fetch is
still in flight.
In each pool, your own requests take the next free slot ahead of the two
background downloads. A file already in the cache is not fetched again. When the
originals cache grows past its limit, the least recently used originals are
evicted, but never a pinned one, and never one whose fetch is still in flight.
`onProgress` receives a `RefreshEvent`. Its `operation` names the work:
`"refresh"`, `"fetchMLData"` for a machine-learning fetch, and
`"precacheThumbnails"` or `"precacheOriginals"` for the two background
downloads. Its `status` is `"started"` as the work starts and `"done"` as it
ends. A refresh or a machine-learning fetch that fails ends with `"failed"`
instead, with the message in `error`. In between, a refresh sends `"progress"`
after each collection whose files it fetched: `fetched` is the number of
collections fetched so far, and `total` the number it will fetch. A refresh
fetches the files of each collection that is new or changed since the cache's
copy, so on an empty cache it fetches those of every collection. The `"done"`
event of a machine-learning fetch, and of each background download, also carries
`fetched`: how many items it newly stored.
ends. A refresh or a machine-learning fetch that fails ends with `"failed"` and
the error message instead. In between, a refresh sends `"progress"` after each
collection whose files it fetched: `fetched` is the number of collections
fetched so far, and `total` the number it will fetch. A refresh fetches the
files of each collection that is new or changed since the cache's copy, so on an
empty cache it fetches those of every collection. The `"done"` event of a
machine-learning fetch, and of each background download, also carries `fetched`:
how many items it newly stored.
`lib.status()` returns a `LibraryStatus`. It holds the collection and file
counts. `lastRefreshAt` is the time of the last refresh that succeeded, and
@@ -1079,10 +1057,10 @@ cached, so the next fetch asks for it again. The reasons are kept in memory
only.
`lib.close()` stops the background timer. It can be called more than once. A
refresh in flight is left to finish, with its write to the cache. `close()`
returns a promise. It resolves once that refresh, the machine-learning fetch and
the background downloads already running have all finished. The cache directory
can then be removed. After `close()`, `fresh()`, `freshCollections()`,
refresh in flight is left to finish, with its write to the cache. The promise
`close()` returns resolves once that refresh, the machine-learning fetch and the
background downloads already running have all finished. The cache directory can
then be removed. After `close()`, `fresh()`, `freshCollections()`,
`freshFiles()` and `lib.backup()` reject with `the library is closed`.
### Default reads and fresh reads
@@ -1187,11 +1165,11 @@ progress of that one file.
keyed by tag name, each as exifreader decodes it with its `id`, `value`,
`description` and `computed` value. For example, `Make` is
`{ id: 271, value: ["Example"], description: "Example", computed: "Example" }`.
A tag that exifreader has no name for is keyed `undefined-<tag number>`. The
tags of the thumbnail embedded in the file are under `Thumbnail`, so they
cannot hide the main image's tags of the same name. The thumbnail image itself
is left out. Any file exifreader cannot read gives `{}`, and a video gives
`{}` without being downloaded.
A tag exifreader has no name for is keyed `undefined-<tag number>`. The tags
of the thumbnail embedded in the file are under `Thumbnail`, so they cannot
hide the main image's tags of the same name. The thumbnail image itself is
left out. Any file exifreader cannot read gives `{}`, and a video gives `{}`
without being downloaded.
- `await photo.make(opts?)` returns one common EXIF field, picked from the tags
`exif()` returns and typed as in `PhotoExif`, or `undefined` when the file
lacks it. These methods do the same for their fields:
@@ -1279,9 +1257,7 @@ symlinks. Once the lock is lost, it rejects with
`BackupOptions`:
- `downloadDirectory`: where the tree lives. Default: the library's.
- `includeOriginals`: fetch and store originals. Default `true`. With `false`,
the backup also fetches no machine-learning data, and writes no JSON beside
the originals and no symlinks.
- `includeOriginals`: fetch and store originals. Default `true`.
- `includeThumbnails`: also store thumbnails as `thumbnails/<fileID>.jpg` beside
`collections/`. Default `false`. `quak backup` does not set it.
- `onlyAlbumNames`: back up only the albums with these names.
@@ -1302,18 +1278,15 @@ releases it once the backup ends.
many requests run at once, because Ente meters these three kinds of traffic
differently:
- metadata, with 10 slots: only the machine-learning data requests;
- content, with 5: originals;
- thumbnails, with 25: thumbnails.
- metadata, with 10 slots;
- content, with 5;
- thumbnails, with 25.
Only the requests named above go through the pools. A refresh,
`freshCollections()` and `freshFiles()` send theirs outside them. Construct one
with
Construct one with
`new RequestPools({ metadataConcurrency, contentConcurrency, thumbnailConcurrency })`
and pass it as `pools`. Each pool serves on-demand work ahead of background
work. Two tasks given to `BoundedPool.run()` with the same `key` run once while
the first is pending, and both get its result. An idle pool never lends its
slots to a busy one.
work. A task submitted under a key that is still pending is not run a second
time. An idle pool never lends its slots to a busy one.
`lib.backup()` works on as many originals at once as the content pool allows, 5
by default. `quak backup` uses the default. The limit covers everything it does
@@ -1383,9 +1356,9 @@ They differ because a download's length depends on the file and the link. A
whole-transfer deadline short enough to catch a hung connection would cancel a
large video on a slow link that is still making progress. The download deadline
restarts each time bytes arrive, so a slow download runs as long as it keeps
moving, and one that stalls is aborted after 60 seconds of silence. The download
deadline covers the wait for the headers as well as the body. There is no limit
on the total length of a download.
moving, and one that stalls is aborted after 60 seconds of silence. It covers
the wait for the headers as well as the body. There is no limit on the total
length of a download.
A request that changes state is not blindly replayed. Every `POST` and `PUT` in
the endpoint list under "HTTP API" goes through this rule. Verifying a second
@@ -1407,13 +1380,13 @@ that rule although they look like connect-time failures. On Linux they are also
delivered on an established socket, after the request may have been written.
They stay retryable for requests that change nothing. Bun's `ConnectionRefused`
and `FailedToOpenSocket` are treated the same way: a request that changes
nothing is retried on them, and a `POST` or `PUT` is not replayed on them. Bun
reuses a connection it kept open from an earlier request. When that connection
closes before any answer, bun sends the request again on a new one. If that one
fails, bun gives one of these codes although the server may already have the
request. The upload of a thumbnail to its presigned URL is exempt from the rule.
It stores one whole object at one key in one request, so replaying it has no
partial state to damage.
nothing is retried on them, and a `POST` or `PUT` is not replayed on them. When
a connection bun kept open from an earlier request closes before any answer, bun
sends the request again on a new connection. If that one fails, bun gives one of
these codes although the server may already have the request. The upload of a
thumbnail to its presigned URL is exempt from the rule. It stores one whole
object at one key in one request, so replaying it has no partial state to
damage.
A download is retried as a whole: the request, the reading of the body and the
decryption. A socket reset after the response headers have arrived surfaces in
@@ -1438,9 +1411,8 @@ return, quak never keeps a truncated file.
`Client.login({ email, password, totp?, emailOTP?, apiOptions? })` logs in and
resolves to a `Client`. `totp` and `emailOTP` are callbacks that return the code
when the account asks for one. Without the callback the account needs, login
throws. A passkey challenge always throws. After an email code, login expects
the key attributes and the token. When the server asks for a second factor
instead, login throws. `apiOptions` configures the HTTP client (see below).
throws. A passkey challenge always throws. `apiOptions` configures the HTTP
client (see below).
The `Client` holds the session token, the master key, the secret key and the
public key in memory. There is no on-disk session store in the library; the
@@ -1450,16 +1422,9 @@ write wherever it likes. `Client.fromJSON(snapshot, apiOptions?)` restores a
working client from that snapshot without logging in again. It checks every
field and each key's length first, and throws an error naming the bad field.
`Client.fromJSON` works only once libsodium, the cryptography library quak uses,
is loaded. `Client.login` loads it, and `Client.fromJSON` does not. A program
that restores a session in a new process first runs `await sodium.ready`, with
`sodium` imported from `libsodium-wrappers-sumo`, which quak depends on. The
exported `beginLogin`, `unwrapAuth`, `decryptCollection`, `decryptFile`,
`downloadFile` and `downloadThumbnail` need the same.
`client.logout()` clears the token and zeroes the key buffers in place. After
it, every other method on that client throws. `logout()` does not contact the
server, so the token stays valid there and in any saved snapshot. Call
it, every other method on that client throws. It does not contact the server, so
the token stays valid there and in any saved snapshot. Call
`await client.logoutOnServer()` before `logout()` to end the session on the
server.
@@ -1479,10 +1444,8 @@ return only the decrypted records that are not deleted, from `sinceTime` `0`.
`client.fetchMLData({ fileIDs, fileKeys })` fetches and decrypts the
machine-learning data of up to 200 files. `client.downloadFile(file, outPath?)`
and `client.downloadThumbnail(file, outPath?)` download and decrypt one file to
`outPath`. For a live photo, `downloadFile` writes the image and the video, each
named `outPath` with its own extension. `quak get --out` instead puts the image
at the path itself. Without `outPath`, they write to the working directory under
the names `quak get` and `quak get-thumb` would use.
`outPath`. Without `outPath`, they write to the working directory under the
names `quak get` and `quak get-thumb` would use.
`ApiClientOptions`:
@@ -1677,8 +1640,8 @@ Login derives the keys in this order:
the code (`POST /users/two-factor/verify`); the server then returns the key
attributes and the token. quak cannot answer a passkey request. When the
account has email verification on (`isEmailMFAEnabled`), an email code
replaces SRP. quak then expects the key attributes and the token in the
answer, and fails the login when the server asks for a second factor instead.
replaces SRP, and the server returns the key attributes and the token after
it.
5. `crypto_secretbox_open_easy` over the encrypted master key with the KEK
yields the 32-byte master key.
6. `crypto_secretbox_open_easy` over the encrypted secret key with the master
@@ -1757,35 +1720,36 @@ This repository adheres to the
standard. Normalized scripts in `script/` are the entrypoints for the
development workflow, and most Makefile targets are thin shims that call them.
The scripts are POSIX `sh`, not bash, so they run in minimal containers such as
Alpine. Of these, only `bootstrap`, `setup`, `test` and `cibuild` come from the
standard. The others are additions to it.
Alpine. The scripts marked "ours" are this repository's own additions to the
standard.
- `script/bootstrap`: install all dependencies. It installs `make`, `git`, Node
and `yarn` when they are missing, then runs `yarn install --frozen-lockfile`.
- `script/setup`: set up the repo for development after a fresh clone. It runs
`script/bootstrap`, then `script/install-precommit`.
- `script/projectname`: print the project name. `script/docker` uses it for the
image tag.
- `script/build`: compile the TypeScript sources into `dist/` and stamp the
version into `dist/package.json`. Then check that the entrypoints
- `script/projectname` (ours): print the project name. `script/docker` uses it
for the image tag.
- `script/build` (ours): compile the TypeScript sources into `dist/` and stamp
the version into `dist/package.json`. Then check that the entrypoints
`package.json` declares (`main`, `types`, `bin`) were written, and make the
command line executable.
- `script/version`: print the version `script/build` stamps (see "Version").
- `script/version` (ours): print the version `script/build` stamps (see
"Version").
- `script/test`: run the test suite, by building the `test` phase of the
`Dockerfile`. Requires Docker.
- `script/lint`: run eslint and a prettier check, by building the `lint` phase
of the `Dockerfile`. Requires Docker.
- `script/fmt`: format all files with prettier. It rewrites files in place.
- `script/fmt-check`: check formatting on the host. It is read-only.
- `script/check`: run all checks: `test`, `lint`, `fmt-check`.
- `script/check` (ours): run all checks: `test`, `lint`, `fmt-check`.
- `script/docker`: build the image, tagged with `script/projectname`.
- `script/cibuild`: what CI runs: `script/bootstrap`, then `script/check`, then
the image build.
- `script/precommit`: run by the git pre-commit hook. It runs `script/lint` and
`script/fmt-check`, and not the tests, so that a commit of failing tests can
land (see "Development workflow").
- `script/install-precommit`: install the git pre-commit hook. `make hooks`
calls it.
- `script/precommit` (ours): run by the git pre-commit hook. It runs
`script/lint` and `script/fmt-check`, and not the tests, so that a commit of
failing tests can land (see "Development workflow").
- `script/install-precommit` (ours): install the git pre-commit hook.
`make hooks` calls it.
The Makefile targets are:
@@ -1852,8 +1816,8 @@ empty, `dev` or `unknown`, because such a build could not be traced to its
commit.
`.dockerignore` therefore does not leave out `.git`, so any `docker build .` of
a clone stamps the commit it was built from. A clone of depth 1 stamps a tag
only when its commit carries one, and otherwise the short commit.
a clone stamps the commit it was built from. A shallow clone stamps a tag only
when the cloned commit itself carries one, and otherwise the short commit.
`.dockerignore` does leave out `.git/config`, which holds the clone's remote URL
and any credential in it; `git describe` does not need that file.
`script/docker` and `script/cibuild` work out the version on the host, with
@@ -1909,12 +1873,7 @@ These apply to people and to coding agents alike.
- the nvm archive `script/bootstrap` downloads, and the Python packages
`bin/clip-keyword-vectors` installs, by SHA-256 hash;
- Node and yarn, when `script/bootstrap` installs them, by version.
- Not pinned:
- bun, which `make build-bin` takes from `nix-shell -p bun`;
- `make` and `git`, which `script/bootstrap` installs from the system
package manager when they are missing, as every `Dockerfile` stage does;
- `bash` and `curl`, which `script/bootstrap` installs the same way when it
has to install Node.
- bun is not pinned: `make build-bin` uses whatever `nix-shell -p bun` provides.
- Follow the workflow above: branches off `next`, tests first and committed
failing, and `next` merged into `main` by sneak alone.
- Before every commit, `make lint` and `make fmt-check` must pass. `make lint`
+7 -10
View File
@@ -1,18 +1,15 @@
// The CLI's log level, and the one function the commands write stderr lines
// through.
//
// Every line quak logs goes to stderr, at one of six levels, from the
// quietest: fatal, error, warning, info, debug, trace. A run prints the lines
// at its own level and at every quieter one. Its level is info unless `-v` or
// `-q` asks for another (see "What quak prints" in the README). A command that
// runs and exits non-zero logs a fatal line saying why, so even `-q`, which
// prints only fatal lines, never leaves such an exit unexplained. An error
// Every line quak prints on stderr is a log line at one of six levels, from
// the quietest: fatal, error, warning, info, debug, trace. A run prints the
// lines at its own level and at every quieter one. Its level is info unless
// `-v` or `-q` asks for another (see "What quak prints" in the README). Every
// command that exits non-zero logs a fatal line saying why, so even `-q`, which
// prints only fatal lines, never leaves a non-zero exit unexplained. An error
// line reports something that failed, such as a file or an album, and a
// warning something that did not fail but needs the user's attention. stdout
// carries a command's output and the prompts of `login`, and is not a log.
// Two things on stderr are not log lines: the `error:` message for a
// command line that quak cannot parse, and the usage printed when no command
// is given.
// carries only a command's output and is not a log.
//
// The library itself writes no lines: its layers report through callbacks, and
// the CLI logs what they report. `logRequests` gives the API client options
+1 -1
View File
@@ -328,7 +328,7 @@ describe("what the library reports", () => {
]);
});
// Each operation is named in plain words.
// Each operation is named in plain words, as the README names it.
it("logs a refresh's progress at info, and its start and end at debug", () => {
const { log, lines } = collect();
const onProgress = logRefresh(log);