1 Commits
Author SHA1 Message Date
sneak 94f25ea0e5 README.md rewritten sentence by sentence (closes #219)
check / check (push) Waiting to run
The old text served as the list of what a reader needs covered. The new
text leads with what quak is, what it is for and how to use it. Every
command, option, exit code and file layout follows in reference sections,
then the library reference, then design, entrypoints and development.
Every statement was checked against the code on next; the places where
the old text disagreed with it are listed in the pull request. No sentence
runs past about 30 words, every table is now a list, and no issue links or
project history remain in text meant for users. Two code comments that
named README sections now name the new ones.

Model: fable-5-1
2026-10-26 14:06:47 +00:00
3 changed files with 50 additions and 55 deletions
+42 -44
View File
@@ -158,11 +158,11 @@ output to `~/quak-backup.log`. Sunday's run adds `--verify`:
A run with `--verify` does all a plain run does. It also hashes each original A run with `--verify` does all a plain run does. It also hashes each original
already in the backup and compares the result with the content hash. The content already in the backup and compares the result with the content hash. The content
hash is the hash of the file's bytes that Ente recorded at upload. The run hash is the hash of the file's bytes that Ente recorded at upload. It fetches a
fetches a new copy of each original that does not match, and replaces the new copy of each original that does not match, and replaces the original once
original once the copy matches. Sunday's run is the `--verify` run, rather than the copy matches. Sunday's run is the `--verify` run, rather than a second job
a second job that night, because two backups of one directory cannot run at that night, because two backups of one directory cannot run at once. The second
once. The second one exits 2 without doing anything. one exits 2 without doing anything.
Cron runs the job with a short `PATH`, usually `/usr/bin:/bin`, so the crontab Cron runs the job with a short `PATH`, usually `/usr/bin:/bin`, so the crontab
names quak by its full path. quak finds the saved session through `HOME`, which names quak by its full path. quak finds the saved session through `HOME`, which
@@ -320,10 +320,10 @@ grid. It writes one file, also for a live photo. Without `--out`, the name is
#### `quak backup <dir> [--json] [--verify]` #### `quak backup <dir> [--json] [--verify]`
Backs up every file in the account to `<dir>`. The layout is described under Backs up every file in the account to `<dir>`. The layout is described under
"The backup directory". The command downloads only the originals that are not "The backup directory". It downloads only the originals that are not yet in the
yet in the backup, and it goes on past a file that fails. A file is downloaded backup, and it goes on past a file that fails. A file is downloaded once,
once, however many collections hold it. An original the cache already holds is however many collections hold it. An original the cache already holds is copied
copied from there instead. from there instead.
Progress goes to stderr: one line per original fetched, and a `FAILED` line for Progress goes to stderr: one line per original fetched, and a `FAILED` line for
each original or album that failed. The run ends with a summary of counts and, each original or album that failed. The run ends with a summary of counts and,
@@ -689,13 +689,13 @@ The JSON file beside an original holds:
Update times are in microseconds, as Ente records them. Update times are in microseconds, as Ente records them.
The JSON also holds the machine-learning data Ente computed for the photo, as The file also holds the machine-learning data Ente computed for the file, as
`mlData`. Ente's apps detect faces in each photo and compute a CLIP embedding `mlData`. Ente's apps detect faces in each photo and compute a CLIP embedding
for it, a list of 512 numbers that describes what the photo shows. `mlData` for it, a list of 512 numbers that describes what the photo shows. `mlData`
holds everything but the `clip` block, which holds that embedding. In its place holds everything but that embedding. In its place the file holds `keywords`,
the JSON holds `keywords`, derived from the embedding as described under derived from the embedding as described under "Keywords". A file with no
"Keywords". A file with no machine-learning data has neither field. A file whose machine-learning data has neither field. A file whose data holds no usable
data holds no usable embedding has no `keywords`. embedding has no `keywords`.
The backup waits for the library's fetch of machine-learning data to finish The backup waits for the library's fetch of machine-learning data to finish
before it writes the JSON files. When a file's data cannot be fetched, before it writes the JSON files. When a file's data cannot be fetched,
@@ -748,11 +748,11 @@ derives a file's keywords the way the app sorts it. The categories are:
Each category has a query, and a text embedding of that query from Ente's CLIP Each category has a query, and a text embedding of that query from Ente's CLIP
text model. A file's score for a category is the cosine similarity of the file's text model. A file's score for a category is the cosine similarity of the file's
embedding and the category's. That is a number from -1 to 1; the more alike the embedding and the category's: a number from -1 to 1, higher the more alike the
two embeddings are, the higher it is. The file falls in the category when its two embeddings are. The file falls in the category when its score reaches the
score reaches the category's minimum score. The keywords are the titles of the category's minimum score. The keywords are the titles of the categories the file
categories the file falls in, highest score first. A video never falls in a falls in, highest score first. A video never falls in a category marked "no
category marked "no videos", as in the app. videos", as in the app.
#### Collection directories #### Collection directories
@@ -775,10 +775,9 @@ ignoring case, each gets its ID added: two albums named `Trip` become
`Trip (10)/` and `Trip (11)/`. Two files in one collection whose titles match, `Trip (10)/` and `Trip (11)/`. Two files in one collection whose titles match,
ignoring case, get their IDs added before the extension: `IMG_0001 (12345).JPG` ignoring case, get their IDs added before the extension: `IMG_0001 (12345).JPG`
and `IMG_0001 (12346).JPG`. A collection named like another collection's JSON and `IMG_0001 (12346).JPG`. A collection named like another collection's JSON
file, such as `Trip.json` beside `Trip`, also gets its ID added. A file name file, such as `Trip.json` beside `Trip`, also gets its ID added. A name too long
longer than 255 bytes, the limit on most file systems, gets its ID added too. So for a file name, which most file systems limit to 255 bytes, gets its ID added
does a collection name longer than 250 bytes, which leaves room for `.json`. too. It is then cut short before the ID, just far enough to fit. IDs never
Either name is then cut short before the ID, just far enough to fit. IDs never
change, so a name stays the same from run to run until such a clash appears or change, so a name stays the same from run to run until such a clash appears or
goes away. goes away.
@@ -835,8 +834,7 @@ Each entry holds:
- `fileID` and `title`; - `fileID` and `title`;
- `classification`: the kind of failure; - `classification`: the kind of failure;
- `attempts`: how many runs in a row have failed the file; - `attempts`: how many runs in a row have failed the file;
- `lastTriedAt`: when the run that last tried it finished its refresh, as a Unix - `lastTriedAt`: when it was last tried, as a Unix time in milliseconds;
time in milliseconds;
- `error`: the error, as quak printed it. - `error`: the error, as quak printed it.
The kinds are: The kinds are:
@@ -859,8 +857,8 @@ ends, whether it succeeded or failed.
The lock is a directory that The lock is a directory that
[proper-lockfile](https://github.com/moxystudio/node-proper-lockfile) creates. [proper-lockfile](https://github.com/moxystudio/node-proper-lockfile) creates.
While the backup runs, it updates the modification time of `backup.lock` every 5 While the backup runs, the backup updates the directory's modification time
seconds. A second backup of `<dir>` fails at once: it prints every 5 seconds. A second backup of the directory fails at once: it prints
`quak: another backup of <dir> is running` and exits 2. A run that is killed, or `quak: another backup of <dir> is running` and exits 2. A run that is killed, or
cut off by a crash, can leave the lock behind. Once the lock has gone 10 seconds cut off by a crash, can leave the lock behind. Once the lock has gone 10 seconds
without an update, the next run takes it over, so nobody has to remove it by without an update, the next run takes it over, so nobody has to remove it by
@@ -1057,14 +1055,14 @@ still in flight.
`"refresh"`, `"fetchMLData"` for a machine-learning fetch, and `"refresh"`, `"fetchMLData"` for a machine-learning fetch, and
`"precacheThumbnails"` or `"precacheOriginals"` for the two background `"precacheThumbnails"` or `"precacheOriginals"` for the two background
downloads. Its `status` is `"started"` as the work starts and `"done"` as it downloads. Its `status` is `"started"` as the work starts and `"done"` as it
ends. A refresh or a machine-learning fetch that fails ends with `"failed"` ends. A refresh or a machine-learning fetch that fails ends with `"failed"` and
instead, with the message in `error`. In between, a refresh sends `"progress"` the error message instead. In between, a refresh sends `"progress"` after each
after each collection whose files it fetched: `fetched` is the number of collection whose files it fetched: `fetched` is the number of collections
collections fetched so far, and `total` the number it will fetch. A refresh fetched so far, and `total` the number it will fetch. A refresh fetches the
fetches the files of each collection that is new or changed since the cache's files of each collection that is new or changed since the cache's copy, so on an
copy, so on an empty cache it fetches those of every collection. The `"done"` empty cache it fetches those of every collection. The `"done"` event of a
event of a machine-learning fetch, and of each background download, also carries machine-learning fetch, and of each background download, also carries `fetched`:
`fetched`: how many items it newly stored. how many items it newly stored.
`lib.status()` returns a `LibraryStatus`. It holds the collection and file `lib.status()` returns a `LibraryStatus`. It holds the collection and file
counts. `lastRefreshAt` is the time of the last refresh that succeeded, and counts. `lastRefreshAt` is the time of the last refresh that succeeded, and
@@ -1079,10 +1077,10 @@ cached, so the next fetch asks for it again. The reasons are kept in memory
only. only.
`lib.close()` stops the background timer. It can be called more than once. A `lib.close()` stops the background timer. It can be called more than once. A
refresh in flight is left to finish, with its write to the cache. `close()` refresh in flight is left to finish, with its write to the cache. The promise
returns a promise. It resolves once that refresh, the machine-learning fetch and `close()` returns resolves once that refresh, the machine-learning fetch and the
the background downloads already running have all finished. The cache directory background downloads already running have all finished. The cache directory can
can then be removed. After `close()`, `fresh()`, `freshCollections()`, then be removed. After `close()`, `fresh()`, `freshCollections()`,
`freshFiles()` and `lib.backup()` reject with `the library is closed`. `freshFiles()` and `lib.backup()` reject with `the library is closed`.
### Default reads and fresh reads ### Default reads and fresh reads
@@ -1383,9 +1381,9 @@ They differ because a download's length depends on the file and the link. A
whole-transfer deadline short enough to catch a hung connection would cancel a whole-transfer deadline short enough to catch a hung connection would cancel a
large video on a slow link that is still making progress. The download deadline large video on a slow link that is still making progress. The download deadline
restarts each time bytes arrive, so a slow download runs as long as it keeps restarts each time bytes arrive, so a slow download runs as long as it keeps
moving, and one that stalls is aborted after 60 seconds of silence. The download moving, and one that stalls is aborted after 60 seconds of silence. It covers
deadline covers the wait for the headers as well as the body. There is no limit the wait for the headers as well as the body. There is no limit on the total
on the total length of a download. length of a download.
A request that changes state is not blindly replayed. Every `POST` and `PUT` in A request that changes state is not blindly replayed. Every `POST` and `PUT` in
the endpoint list under "HTTP API" goes through this rule. Verifying a second the endpoint list under "HTTP API" goes through this rule. Verifying a second
@@ -1458,8 +1456,8 @@ exported `beginLogin`, `unwrapAuth`, `decryptCollection`, `decryptFile`,
`downloadFile` and `downloadThumbnail` need the same. `downloadFile` and `downloadThumbnail` need the same.
`client.logout()` clears the token and zeroes the key buffers in place. After `client.logout()` clears the token and zeroes the key buffers in place. After
it, every other method on that client throws. `logout()` does not contact the it, every other method on that client throws. It does not contact the server, so
server, so the token stays valid there and in any saved snapshot. Call the token stays valid there and in any saved snapshot. Call
`await client.logoutOnServer()` before `logout()` to end the session on the `await client.logoutOnServer()` before `logout()` to end the session on the
server. server.
+7 -10
View File
@@ -1,18 +1,15 @@
// The CLI's log level, and the one function the commands write stderr lines // The CLI's log level, and the one function the commands write stderr lines
// through. // through.
// //
// Every line quak logs goes to stderr, at one of six levels, from the // Every line quak prints on stderr is a log line at one of six levels, from
// quietest: fatal, error, warning, info, debug, trace. A run prints the lines // the quietest: fatal, error, warning, info, debug, trace. A run prints the
// at its own level and at every quieter one. Its level is info unless `-v` or // lines at its own level and at every quieter one. Its level is info unless
// `-q` asks for another (see "What quak prints" in the README). A command that // `-v` or `-q` asks for another (see "What quak prints" in the README). Every
// runs and exits non-zero logs a fatal line saying why, so even `-q`, which // command that exits non-zero logs a fatal line saying why, so even `-q`, which
// prints only fatal lines, never leaves such an exit unexplained. An error // prints only fatal lines, never leaves a non-zero exit unexplained. An error
// line reports something that failed, such as a file or an album, and a // line reports something that failed, such as a file or an album, and a
// warning something that did not fail but needs the user's attention. stdout // warning something that did not fail but needs the user's attention. stdout
// carries a command's output and the prompts of `login`, and is not a log. // carries only a command's output and is not a log.
// Two things on stderr are not log lines: the `error:` message for a
// command line that quak cannot parse, and the usage printed when no command
// is given.
// //
// The library itself writes no lines: its layers report through callbacks, and // The library itself writes no lines: its layers report through callbacks, and
// the CLI logs what they report. `logRequests` gives the API client options // the CLI logs what they report. `logRequests` gives the API client options
+1 -1
View File
@@ -328,7 +328,7 @@ describe("what the library reports", () => {
]); ]);
}); });
// Each operation is named in plain words. // Each operation is named in plain words, as the README names it.
it("logs a refresh's progress at info, and its start and end at debug", () => { it("logs a refresh's progress at info, and its start and end at debug", () => {
const { log, lines } = collect(); const { log, lines } = collect();
const onProgress = logRefresh(log); const onProgress = logRefresh(log);