3 Commits
Author SHA1 Message Date
clawbot aeb275e137 README: the close() sentence split in two (closes #231)
check / check (push) Waiting to run
The sentence in "Opening a library" had a relative clause with its
"that" left out, so "returns resolves" read as two verbs in a row. It
is now two sentences; the behaviour described is unchanged and matches
`close()` in `src/library/index.ts`.

Model: fable-5-1 (change); opus-5-5 (landing message)
2026-10-26 19:32:03 +01:00
clawbot 67794bf5f8 README: four sentences fixed for the documents gate (closes #229)
check / check (push) Waiting to run
The documents gate on `next` failed five README sentences
(#219 (comment)). Four
are rewritten after a check against the code:

- the keywords score sentence is split in two;
- `mlData` drops the whole `clip` block, as `src/backup.ts` and
  `src/metadata-backup.ts` do;
- a file name gets its ID past 255 bytes and a collection name past
  250, and either is then cut short, as `uniqueNames` does;
- `lastTriedAt` is taken once per run, after its refresh.

The IDs sentence keeps `-1`: the pinned option parser passes a negative
number to a command with no subcommands as an argument, so
`quak get -1` reaches the ID check. Nothing else changes; `make fmt`
reflowed the touched paragraphs.

Model: fable-5-1 (change); opus-5-5 (landing message)
2026-10-26 18:59:28 +01:00
clawbot b5fcbf4400 README.md rewritten sentence by sentence (closes #219)
check / check (push) Waiting to run
The old text served as the list of what to cover. The new text leads
with what quak is, what it is for and how to use it. Every command,
option, exit code and file layout follows in reference sections, then the
library reference, then design, entrypoints and development. Every
statement was checked against the code on next; the pull request lists
each disagreement. Sentences stay under about 30 words, tables are lists,
and no issue links or project history remain in text for users. Two code
comments now name the new README sections, and the one in
`src/cli-log.ts` matches "What quak prints" on stderr, stdout and exit
status; a third no longer cites the README.

Model: fable-5-1
2026-10-26 17:48:25 +01:00
3 changed files with 67 additions and 62 deletions
+56 -54
View File
@@ -158,11 +158,11 @@ output to `~/quak-backup.log`. Sunday's run adds `--verify`:
A run with `--verify` does all a plain run does. It also hashes each original
already in the backup and compares the result with the content hash. The content
hash is the hash of the file's bytes that Ente recorded at upload. It fetches a
new copy of each original that does not match, and replaces the original once
the copy matches. Sunday's run is the `--verify` run, rather than a second job
that night, because two backups of one directory cannot run at once. The second
one exits 2 without doing anything.
hash is the hash of the file's bytes that Ente recorded at upload. The run
fetches a new copy of each original that does not match, and replaces the
original once the copy matches. Sunday's run is the `--verify` run, rather than
a second job that night, because two backups of one directory cannot run at
once. The second one exits 2 without doing anything.
Cron runs the job with a short `PATH`, usually `/usr/bin:/bin`, so the crontab
names quak by its full path. quak finds the saved session through `HOME`, which
@@ -320,10 +320,10 @@ grid. It writes one file, also for a live photo. Without `--out`, the name is
#### `quak backup <dir> [--json] [--verify]`
Backs up every file in the account to `<dir>`. The layout is described under
"The backup directory". It downloads only the originals that are not yet in the
backup, and it goes on past a file that fails. A file is downloaded once,
however many collections hold it. An original the cache already holds is copied
from there instead.
"The backup directory". The command downloads only the originals that are not
yet in the backup, and it goes on past a file that fails. A file is downloaded
once, however many collections hold it. An original the cache already holds is
copied from there instead.
Progress goes to stderr: one line per original fetched, and a `FAILED` line for
each original or album that failed. The run ends with a summary of counts and,
@@ -484,15 +484,6 @@ levels. From the quietest, the levels are fatal, error, warning, info, debug and
trace. A run prints the lines at its own level and at every quieter one. The
level is info unless `-v`, `-vv` or `-q` says otherwise.
Two things on stderr are not log lines. A command line quak cannot parse, such
as a missing argument, an unknown option or `-q` with `-v`, is refused with an
`error: …` message, and quak exits 1. `-q` does not hide the message. `quak`
with no command prints the usage on stderr and exits 1.
`yarn quak` prints yarn's own lines on stdout as well. To pipe a command's
output, run `yarn --silent quak files --json` or
`node dist/bin/quak.js files --json` instead.
- Fatal: why a command exits non-zero. A command that runs and exits non-zero
logs at least one fatal line. A command that exits 1 because files or albums
failed logs it last, saying how many failed and where to look.
@@ -517,6 +508,15 @@ output, run `yarn --silent quak files --json` or
one that exits non-zero prints only its fatal lines. The prompts of `login` are
not log lines, and `-q` leaves them alone.
Two things on stderr are not log lines. A command line that quak cannot parse,
such as a missing argument, an unknown option or `-q` with `-v`, is refused with
an `error: …` message, and quak exits 1. `-q` does not hide the message. `quak`
with no command prints the usage on stderr and exits 1.
`yarn quak` prints yarn's own lines on stdout as well. To pipe a command's
output, run `yarn --silent quak files --json` or
`node dist/bin/quak.js files --json` instead.
With `--json`, a command's JSON on stdout takes the place of its summary on
stderr. For `backup`, that is the `--- Backup complete ---` line, the counts and
the `Failed files:` list. For `list-missing-thumbnails`, it is
@@ -689,13 +689,13 @@ The JSON file beside an original holds:
Update times are in microseconds, as Ente records them.
The file also holds the machine-learning data Ente computed for the file, as
The JSON also holds the machine-learning data Ente computed for the photo, as
`mlData`. Ente's apps detect faces in each photo and compute a CLIP embedding
for it, a list of 512 numbers that describes what the photo shows. `mlData`
holds everything but that embedding. In its place the file holds `keywords`,
derived from the embedding as described under "Keywords". A file with no
machine-learning data has neither field. A file whose data holds no usable
embedding has no `keywords`.
holds everything but the `clip` block, which holds that embedding. In its place
the JSON holds `keywords`, derived from the embedding as described under
"Keywords". A file with no machine-learning data has neither field. A file whose
data holds no usable embedding has no `keywords`.
The backup waits for the library's fetch of machine-learning data to finish
before it writes the JSON files. When a file's data cannot be fetched,
@@ -748,11 +748,11 @@ derives a file's keywords the way the app sorts it. The categories are:
Each category has a query, and a text embedding of that query from Ente's CLIP
text model. A file's score for a category is the cosine similarity of the file's
embedding and the category's: a number from -1 to 1, higher the more alike the
two embeddings are. The file falls in the category when its score reaches the
category's minimum score. The keywords are the titles of the categories the file
falls in, highest score first. A video never falls in a category marked "no
videos", as in the app.
embedding and the category's. That is a number from -1 to 1; the more alike the
two embeddings are, the higher it is. The file falls in the category when its
score reaches the category's minimum score. The keywords are the titles of the
categories the file falls in, highest score first. A video never falls in a
category marked "no videos", as in the app.
#### Collection directories
@@ -775,9 +775,10 @@ ignoring case, each gets its ID added: two albums named `Trip` become
`Trip (10)/` and `Trip (11)/`. Two files in one collection whose titles match,
ignoring case, get their IDs added before the extension: `IMG_0001 (12345).JPG`
and `IMG_0001 (12346).JPG`. A collection named like another collection's JSON
file, such as `Trip.json` beside `Trip`, also gets its ID added. A name too long
for a file name, which most file systems limit to 255 bytes, gets its ID added
too. It is then cut short before the ID, just far enough to fit. IDs never
file, such as `Trip.json` beside `Trip`, also gets its ID added. A file name
longer than 255 bytes, the limit on most file systems, gets its ID added too. So
does a collection name longer than 250 bytes, which leaves room for `.json`.
Either name is then cut short before the ID, just far enough to fit. IDs never
change, so a name stays the same from run to run until such a clash appears or
goes away.
@@ -834,7 +835,8 @@ Each entry holds:
- `fileID` and `title`;
- `classification`: the kind of failure;
- `attempts`: how many runs in a row have failed the file;
- `lastTriedAt`: when it was last tried, as a Unix time in milliseconds;
- `lastTriedAt`: when the run that last tried it finished its refresh, as a Unix
time in milliseconds;
- `error`: the error, as quak printed it.
The kinds are:
@@ -857,8 +859,8 @@ ends, whether it succeeded or failed.
The lock is a directory that
[proper-lockfile](https://github.com/moxystudio/node-proper-lockfile) creates.
While the backup runs, the backup updates the directory's modification time
every 5 seconds. A second backup of the directory fails at once: it prints
While the backup runs, it updates the modification time of `backup.lock` every 5
seconds. A second backup of `<dir>` fails at once: it prints
`quak: another backup of <dir> is running` and exits 2. A run that is killed, or
cut off by a crash, can leave the lock behind. Once the lock has gone 10 seconds
without an update, the next run takes it over, so nobody has to remove it by
@@ -962,9 +964,9 @@ of the same name apart.
`files`. Each `<fileID>.json` holds the fields a backup's JSON beside an
original holds, with these differences:
- `imageMetadata` is written only with `--exif`. The original of every file is
then downloaded, videos included, and read as a backup reads it (see "The JSON
beside each original").
- `imageMetadata` is written only with `--exif`. The original of every file,
videos included, is then downloaded and read for its EXIF, XMP and JPEG
dimensions.
- A file whose original holds no image metadata gets no `imageMetadata` field.
- An original that cannot be downloaded or read gives `imageMetadataError`
instead.
@@ -1055,14 +1057,14 @@ still in flight.
`"refresh"`, `"fetchMLData"` for a machine-learning fetch, and
`"precacheThumbnails"` or `"precacheOriginals"` for the two background
downloads. Its `status` is `"started"` as the work starts and `"done"` as it
ends. A refresh or a machine-learning fetch that fails ends with `"failed"` and
the error message instead. In between, a refresh sends `"progress"` after each
collection whose files it fetched: `fetched` is the number of collections
fetched so far, and `total` the number it will fetch. A refresh fetches the
files of each collection that is new or changed since the cache's copy, so on an
empty cache it fetches those of every collection. The `"done"` event of a
machine-learning fetch, and of each background download, also carries `fetched`:
how many items it newly stored.
ends. A refresh or a machine-learning fetch that fails ends with `"failed"`
instead, with the message in `error`. In between, a refresh sends `"progress"`
after each collection whose files it fetched: `fetched` is the number of
collections fetched so far, and `total` the number it will fetch. A refresh
fetches the files of each collection that is new or changed since the cache's
copy, so on an empty cache it fetches those of every collection. The `"done"`
event of a machine-learning fetch, and of each background download, also carries
`fetched`: how many items it newly stored.
`lib.status()` returns a `LibraryStatus`. It holds the collection and file
counts. `lastRefreshAt` is the time of the last refresh that succeeded, and
@@ -1077,10 +1079,10 @@ cached, so the next fetch asks for it again. The reasons are kept in memory
only.
`lib.close()` stops the background timer. It can be called more than once. A
refresh in flight is left to finish, with its write to the cache. The promise
`close()` returns resolves once that refresh, the machine-learning fetch and the
background downloads already running have all finished. The cache directory can
then be removed. After `close()`, `fresh()`, `freshCollections()`,
refresh in flight is left to finish, with its write to the cache. `close()`
returns a promise. It resolves once that refresh, the machine-learning fetch and
the background downloads already running have all finished. The cache directory
can then be removed. After `close()`, `fresh()`, `freshCollections()`,
`freshFiles()` and `lib.backup()` reject with `the library is closed`.
### Default reads and fresh reads
@@ -1381,9 +1383,9 @@ They differ because a download's length depends on the file and the link. A
whole-transfer deadline short enough to catch a hung connection would cancel a
large video on a slow link that is still making progress. The download deadline
restarts each time bytes arrive, so a slow download runs as long as it keeps
moving, and one that stalls is aborted after 60 seconds of silence. It covers
the wait for the headers as well as the body. There is no limit on the total
length of a download.
moving, and one that stalls is aborted after 60 seconds of silence. The download
deadline covers the wait for the headers as well as the body. There is no limit
on the total length of a download.
A request that changes state is not blindly replayed. Every `POST` and `PUT` in
the endpoint list under "HTTP API" goes through this rule. Verifying a second
@@ -1456,8 +1458,8 @@ exported `beginLogin`, `unwrapAuth`, `decryptCollection`, `decryptFile`,
`downloadFile` and `downloadThumbnail` need the same.
`client.logout()` clears the token and zeroes the key buffers in place. After
it, every other method on that client throws. It does not contact the server, so
the token stays valid there and in any saved snapshot. Call
it, every other method on that client throws. `logout()` does not contact the
server, so the token stays valid there and in any saved snapshot. Call
`await client.logoutOnServer()` before `logout()` to end the session on the
server.
+10 -7
View File
@@ -1,15 +1,18 @@
// The CLI's log level, and the one function the commands write stderr lines
// through.
//
// Every line quak prints on stderr is a log line at one of six levels, from
// the quietest: fatal, error, warning, info, debug, trace. A run prints the
// lines at its own level and at every quieter one. Its level is info unless
// `-v` or `-q` asks for another (see "What quak prints" in the README). Every
// command that exits non-zero logs a fatal line saying why, so even `-q`, which
// prints only fatal lines, never leaves a non-zero exit unexplained. An error
// Every line quak logs goes to stderr, at one of six levels, from the
// quietest: fatal, error, warning, info, debug, trace. A run prints the lines
// at its own level and at every quieter one. Its level is info unless `-v` or
// `-q` asks for another (see "What quak prints" in the README). A command that
// runs and exits non-zero logs a fatal line saying why, so even `-q`, which
// prints only fatal lines, never leaves such an exit unexplained. An error
// line reports something that failed, such as a file or an album, and a
// warning something that did not fail but needs the user's attention. stdout
// carries only a command's output and is not a log.
// carries a command's output and the prompts of `login`, and is not a log.
// Two things on stderr are not log lines: the `error:` message for a
// command line that quak cannot parse, and the usage printed when no command
// is given.
//
// The library itself writes no lines: its layers report through callbacks, and
// the CLI logs what they report. `logRequests` gives the API client options
+1 -1
View File
@@ -328,7 +328,7 @@ describe("what the library reports", () => {
]);
});
// Each operation is named in plain words, as the README names it.
// Each operation is named in plain words.
it("logs a refresh's progress at info, and its start and end at debug", () => {
const { log, lines } = collect();
const onProgress = logRefresh(log);