Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6d9852af13 |
@@ -73,7 +73,8 @@ and `dist/` is not committed. Clone and build it as above, then add the clone to
|
|||||||
your own program's `package.json` by its path, for example
|
your own program's `package.json` by its path, for example
|
||||||
`yarn add file:../quak`. The package's `main`, `types` and `exports` fields
|
`yarn add file:../quak`. The package's `main`, `types` and `exports` fields
|
||||||
point into `dist/`, so the clone must be built first. The package is an ES
|
point into `dist/`, so the clone must be built first. The package is an ES
|
||||||
module only, so a CommonJS program cannot `require` it.
|
module only: `package.json` sets `"type": "module"`, and its `exports` field
|
||||||
|
offers only `import`, so a CommonJS program cannot `require` it.
|
||||||
|
|
||||||
```ts
|
```ts
|
||||||
import { Client, Library } from "quak";
|
import { Client, Library } from "quak";
|
||||||
@@ -89,9 +90,8 @@ const client = await Client.login({
|
|||||||
const lib = await Library.open({
|
const lib = await Library.open({
|
||||||
client,
|
client,
|
||||||
// Without these two, open() also starts downloading every thumbnail in
|
// Without these two, open() also starts downloading every thumbnail in
|
||||||
// the account, and the originals of the favorites album and the latest
|
// the account and the latest week's originals into the cache. It fetches
|
||||||
// week, into the cache. It fetches machine-learning data after each
|
// machine-learning data after each refresh either way.
|
||||||
// refresh either way.
|
|
||||||
precacheThumbnails: false,
|
precacheThumbnails: false,
|
||||||
precacheOriginals: false,
|
precacheOriginals: false,
|
||||||
});
|
});
|
||||||
@@ -122,12 +122,10 @@ await lib.close();
|
|||||||
for. It refreshes every 3 seconds until `close()`. It also downloads, in the
|
for. It refreshes every 3 seconds until `close()`. It also downloads, in the
|
||||||
background, every thumbnail in the account and the originals of the favorites
|
background, every thumbnail in the account and the originals of the favorites
|
||||||
album and of the latest week (see "Opening a library"). The example turns those
|
album and of the latest week (see "Opening a library"). The example turns those
|
||||||
two downloads off. After each refresh that succeeds, the library also fetches
|
two downloads off. After each refresh the library also fetches each file's
|
||||||
each file's machine-learning data into the cache. That is the faces and the
|
machine-learning data into the cache. That is the faces and the other data
|
||||||
other data Ente's apps compute for a photo (see "The JSON beside each
|
Ente's apps compute for a photo (see "The JSON beside each original"). No option
|
||||||
original"). No option in `LibraryOptions` turns this off. A `client` without a
|
turns this off.
|
||||||
`fetchMLData` method fetches none, which is how quak's own read commands open
|
|
||||||
their library.
|
|
||||||
|
|
||||||
The `Client` under the library handles login, saved sessions and the raw
|
The `Client` under the library handles login, saved sessions and the raw
|
||||||
requests. The "Library reference" below covers both.
|
requests. The "Library reference" below covers both.
|
||||||
@@ -136,9 +134,9 @@ requests. The "Library reference" below covers both.
|
|||||||
|
|
||||||
`quak backup` never prompts, so cron can run it. `make install` builds quak as
|
`quak backup` never prompts, so cron can run it. `make install` builds quak as
|
||||||
one binary and copies it to `~/bin/quak`. Besides what Getting Started needs,
|
one binary and copies it to `~/bin/quak`. Besides what Getting Started needs,
|
||||||
`make install` needs `make` and [nix](https://nixos.org/download). It uses bun,
|
`make install` needs [nix](https://nixos.org/download). It uses bun, a
|
||||||
a JavaScript runtime, to bundle the built `dist/` into one executable. It runs
|
JavaScript runtime, to bundle the built `dist/` into one executable. It runs bun
|
||||||
bun through `nix-shell -p bun`.
|
through `nix-shell -p bun`.
|
||||||
|
|
||||||
Log in once with `~/bin/quak`, as the user the cron job will run as:
|
Log in once with `~/bin/quak`, as the user the cron job will run as:
|
||||||
|
|
||||||
@@ -158,10 +156,9 @@ output to `~/quak-backup.log`. Sunday's run adds `--verify`:
|
|||||||
|
|
||||||
A run with `--verify` does all a plain run does. It also hashes each original
|
A run with `--verify` does all a plain run does. It also hashes each original
|
||||||
already in the backup and compares the result with the content hash. The content
|
already in the backup and compares the result with the content hash. The content
|
||||||
hash is the hash of the file's bytes that Ente recorded at upload. The run
|
hash is the hash of the file's bytes that Ente recorded at upload. It replaces
|
||||||
fetches a new copy of each original that does not match, and replaces the
|
any original that does not match. Sunday's run is the `--verify` run, rather
|
||||||
original once the copy matches. Sunday's run is the `--verify` run, rather than
|
than a second job that night, because two backups of one directory cannot run at
|
||||||
a second job that night, because two backups of one directory cannot run at
|
|
||||||
once. The second one exits 2 without doing anything.
|
once. The second one exits 2 without doing anything.
|
||||||
|
|
||||||
Cron runs the job with a short `PATH`, usually `/usr/bin:/bin`, so the crontab
|
Cron runs the job with a short `PATH`, usually `/usr/bin:/bin`, so the crontab
|
||||||
@@ -230,12 +227,9 @@ set, it asks on the terminal. The password shows as `*` characters while it is
|
|||||||
typed.
|
typed.
|
||||||
|
|
||||||
When the account has a second factor, quak asks for the code. It asks
|
When the account has a second factor, quak asks for the code. It asks
|
||||||
`TOTP code: ` for a time-based code from an authenticator app. An account whose
|
`TOTP code: ` for a time-based code from an authenticator app, and
|
||||||
only second factor is a passkey cannot log in with quak. When the account has
|
`Email verification code: ` for a code sent by email. An account whose only
|
||||||
email verification on, quak asks `Email verification code: ` for the code sent
|
second factor is a passkey cannot log in with quak.
|
||||||
by email. The password is then used only to unlock the keys. After the email
|
|
||||||
code quak expects the login to complete. When the server asks for a second
|
|
||||||
factor at that point, login fails.
|
|
||||||
|
|
||||||
On success it logs `Logged in as <email> (user <id>)` and
|
On success it logs `Logged in as <email> (user <id>)` and
|
||||||
`Session saved to <path>`. On failure it logs `Login failed: <reason>`, and the
|
`Session saved to <path>`. On failure it logs `Login failed: <reason>`, and the
|
||||||
@@ -245,9 +239,7 @@ to stdout.
|
|||||||
#### `quak whoami`
|
#### `quak whoami`
|
||||||
|
|
||||||
Prints the saved session's account on stdout as one line of JSON:
|
Prints the saved session's account on stdout as one line of JSON:
|
||||||
`{"email":"you@example.com","userID":12345}`. It reads the session file only and
|
`{"email":"you@example.com","userID":12345}`.
|
||||||
sends no request, so it succeeds even when the server no longer accepts the
|
|
||||||
session.
|
|
||||||
|
|
||||||
#### `quak logout`
|
#### `quak logout`
|
||||||
|
|
||||||
@@ -314,16 +306,16 @@ refused, because both files would land on it.
|
|||||||
#### `quak get-thumb <fileID> [--out <path>]`
|
#### `quak get-thumb <fileID> [--out <path>]`
|
||||||
|
|
||||||
Downloads and decrypts one file's thumbnail, the small JPEG Ente shows in a
|
Downloads and decrypts one file's thumbnail, the small JPEG Ente shows in a
|
||||||
grid. It writes one file, also for a live photo. Without `--out`, the name is
|
grid. It works as `get` does. Without `--out`, the name is `thumb_` followed by
|
||||||
`thumb_` followed by the title. In all else it works as `get` does.
|
the title.
|
||||||
|
|
||||||
#### `quak backup <dir> [--json] [--verify]`
|
#### `quak backup <dir> [--json] [--verify]`
|
||||||
|
|
||||||
Backs up every file in the account to `<dir>`. The layout is described under
|
Backs up every file in the account to `<dir>`. The layout is described under
|
||||||
"The backup directory". The command downloads only the originals that are not
|
"The backup directory". It downloads only the originals that are not yet in the
|
||||||
yet in the backup, and it goes on past a file that fails. A file is downloaded
|
backup, and it goes on past a file that fails. A file is downloaded once,
|
||||||
once, however many collections hold it. An original the cache already holds is
|
however many collections hold it. An original the cache already holds is copied
|
||||||
copied from there instead.
|
from there instead.
|
||||||
|
|
||||||
Progress goes to stderr: one line per original fetched, and a `FAILED` line for
|
Progress goes to stderr: one line per original fetched, and a `FAILED` line for
|
||||||
each original or album that failed. The run ends with a summary of counts and,
|
each original or album that failed. The run ends with a summary of counts and,
|
||||||
@@ -362,8 +354,7 @@ Writes the metadata quak keeps about the account, its collections and its files
|
|||||||
as JSON files under `<dir>`. The layout is described under "The metadata dump
|
as JSON files under `<dir>`. The layout is described under "The metadata dump
|
||||||
directory". Photo content is not written. With `--exif`, or its alias `--all`,
|
directory". Photo content is not written. With `--exif`, or its alias `--all`,
|
||||||
the command also fetches each file's original and records the image's own
|
the command also fetches each file's original and records the image's own
|
||||||
metadata from it (see "The metadata dump directory"). Each original it fetches
|
metadata from it (see "The metadata dump directory").
|
||||||
stays in the cache.
|
|
||||||
|
|
||||||
Ente's apps compute machine-learning data for each photo, such as the faces in
|
Ente's apps compute machine-learning data for each photo, such as the faces in
|
||||||
it (see "The JSON beside each original"). The command fetches it in requests of
|
it (see "The JSON beside each original"). The command fetches it in requests of
|
||||||
@@ -378,13 +369,12 @@ word, and does not change the exit status.
|
|||||||
|
|
||||||
#### `quak helper list-missing-thumbnails [--json]`
|
#### `quak helper list-missing-thumbnails [--json]`
|
||||||
|
|
||||||
Finds the files whose thumbnail is missing on the server. It downloads the
|
Finds the files whose thumbnail is missing on the server. It downloads every
|
||||||
thumbnail of every file you own, one collection at a time, newest updated first,
|
thumbnail in the account, one collection at a time, newest collection first, so
|
||||||
so on a large account it takes a while. A thumbnail counts as missing when the
|
on a large account it takes a while. A thumbnail counts as missing when the
|
||||||
server answers with an empty body or with HTTP 404. Any other failure is logged
|
server answers with an empty body or with HTTP 404. Any other failure is logged
|
||||||
as `[<collection>] Could not check <title>: <reason> (not reported as missing)`,
|
as `[<collection>] Could not check <title>: <reason> (not reported as missing)`,
|
||||||
and the file is left out of the list. The command exits 0 even when some files
|
and the file is left out of the list.
|
||||||
could not be checked.
|
|
||||||
|
|
||||||
The server lets only a file's owner replace its thumbnail. A file another
|
The server lets only a file's owner replace its thumbnail. A file another
|
||||||
account owns, in a collection shared with you, is skipped and not checked.
|
account owns, in a collection shared with you, is skipped and not checked.
|
||||||
@@ -427,13 +417,12 @@ Each file ends in one of three states:
|
|||||||
`--file` is held by no collection.
|
`--file` is held by no collection.
|
||||||
|
|
||||||
Without `--json`, the command ends with counts of fixed, skipped and failed
|
Without `--json`, the command ends with counts of fixed, skipped and failed
|
||||||
files. A `Skipped:` list and a `Failed files:` list follow when they have
|
files, a `Skipped:` list and a `Failed files:` list. With `--json`, one JSON
|
||||||
entries. With `--json`, one JSON array on stdout replaces them; each object
|
array on stdout replaces them; each object holds `fileID`, `title`,
|
||||||
holds `fileID`, `title`, `collection`, `status` and, for a skipped or failed
|
`collection`, `status` and, for a skipped or failed file, `reason`. When any
|
||||||
file, `reason`. When any file failed, the exit status is 1. A skipped file does
|
file failed, the exit status is 1. A skipped file does not change it. When
|
||||||
not change it. When nothing needs fixing, the command logs
|
nothing needs fixing, the command logs `No missing thumbnails found.` and
|
||||||
`No missing thumbnails found.` and exits 0. With `--json` it then writes nothing
|
exits 0. With `--json` it then writes nothing on stdout.
|
||||||
on stdout.
|
|
||||||
|
|
||||||
quak refuses the whole command, before any request, when one `--file` ID is
|
quak refuses the whole command, before any request, when one `--file` ID is
|
||||||
invalid (see "IDs").
|
invalid (see "IDs").
|
||||||
@@ -464,9 +453,9 @@ collection fetched, such as `Fetched the files of 3 of 120 collections`.
|
|||||||
|
|
||||||
When a refresh or a search fails, the command logs the error on one line and
|
When a refresh or a search fails, the command logs the error on one line and
|
||||||
exits 1, or 3 when the server no longer accepts the saved session. A collection
|
exits 1, or 3 when the server no longer accepts the saved session. A collection
|
||||||
or file that quak cannot decrypt does not stop a refresh or a search: the
|
or file that quak cannot decrypt does not stop a refresh: the command answers
|
||||||
command leaves it out and answers from what did decrypt (see "Default reads and
|
from what did decrypt (see "Default reads and fresh reads"). `backup` counts it
|
||||||
fresh reads"). `backup` counts it as failed (see "What quak cannot decrypt").
|
as failed (see "What quak cannot decrypt").
|
||||||
|
|
||||||
### IDs
|
### IDs
|
||||||
|
|
||||||
@@ -478,11 +467,20 @@ before any request is sent. The command logs `Invalid file ID: <arg>` or
|
|||||||
|
|
||||||
### What quak prints
|
### What quak prints
|
||||||
|
|
||||||
stdout holds a command's output, such as the listing or the JSON, and the
|
stdout holds only a command's output: the listing, the JSON, or nothing. The
|
||||||
prompts of `login`. Every other line goes to stderr as a log line at one of six
|
prompts of `login` go to stdout too. Every other line goes to stderr as a log
|
||||||
levels. From the quietest, the levels are fatal, error, warning, info, debug and
|
line at one of six levels. From the quietest, the levels are fatal, error,
|
||||||
trace. A run prints the lines at its own level and at every quieter one. The
|
warning, info, debug and trace. A run prints the lines at its own level and at
|
||||||
level is info unless `-v`, `-vv` or `-q` says otherwise.
|
every quieter one. The level is info unless `-v`, `-vv` or `-q` says otherwise.
|
||||||
|
|
||||||
|
The one exception is a command line quak cannot parse, such as a missing
|
||||||
|
argument, an unknown option or `-q` with `-v`. quak refuses it with one
|
||||||
|
`error: …` line that is not a log line, and exits 1. `-q` does not hide that
|
||||||
|
line. `quak` with no command prints the usage on stderr and exits 1.
|
||||||
|
|
||||||
|
`yarn quak` prints yarn's own lines on stdout as well. To pipe a command's
|
||||||
|
output, run `yarn --silent quak files --json` or
|
||||||
|
`node dist/bin/quak.js files --json` instead.
|
||||||
|
|
||||||
- Fatal: why a command exits non-zero. A command that runs and exits non-zero
|
- Fatal: why a command exits non-zero. A command that runs and exits non-zero
|
||||||
logs at least one fatal line. A command that exits 1 because files or albums
|
logs at least one fatal line. A command that exits 1 because files or albums
|
||||||
@@ -508,22 +506,12 @@ level is info unless `-v`, `-vv` or `-q` says otherwise.
|
|||||||
one that exits non-zero prints only its fatal lines. The prompts of `login` are
|
one that exits non-zero prints only its fatal lines. The prompts of `login` are
|
||||||
not log lines, and `-q` leaves them alone.
|
not log lines, and `-q` leaves them alone.
|
||||||
|
|
||||||
Two things on stderr are not log lines. A command line that quak cannot parse,
|
|
||||||
such as a missing argument, an unknown option or `-q` with `-v`, is refused with
|
|
||||||
an `error: …` message, and quak exits 1. `-q` does not hide the message. `quak`
|
|
||||||
with no command prints the usage on stderr and exits 1.
|
|
||||||
|
|
||||||
`yarn quak` prints yarn's own lines on stdout as well. To pipe a command's
|
|
||||||
output, run `yarn --silent quak files --json` or
|
|
||||||
`node dist/bin/quak.js files --json` instead.
|
|
||||||
|
|
||||||
With `--json`, a command's JSON on stdout takes the place of its summary on
|
With `--json`, a command's JSON on stdout takes the place of its summary on
|
||||||
stderr. For `backup`, that is the `--- Backup complete ---` line, the counts and
|
stderr. For `backup`, that is the lines from `--- Backup complete ---` to the
|
||||||
the `Failed files:` list. For `list-missing-thumbnails`, it is
|
end of the `Failed files:` list. For `list-missing-thumbnails`, it is
|
||||||
`No missing thumbnails found.` or `<n> file(s) with missing thumbnails:`. For
|
`No missing thumbnails found.` or `<n> file(s) with missing thumbnails:`. For
|
||||||
`fix-missing-thumbnails`, it is the `--- Done ---` line, the counts, the
|
`fix-missing-thumbnails`, it is the lines from `--- Done ---` to the end of the
|
||||||
`Skipped:` list and the `Failed files:` list. Every other line is logged as
|
`Failed files:` list. Every other line is logged as without `--json`.
|
||||||
without `--json`.
|
|
||||||
|
|
||||||
### Exit codes
|
### Exit codes
|
||||||
|
|
||||||
@@ -689,13 +677,13 @@ The JSON file beside an original holds:
|
|||||||
|
|
||||||
Update times are in microseconds, as Ente records them.
|
Update times are in microseconds, as Ente records them.
|
||||||
|
|
||||||
The JSON also holds the machine-learning data Ente computed for the photo, as
|
The file also holds the machine-learning data Ente computed for the file, as
|
||||||
`mlData`. Ente's apps detect faces in each photo and compute a CLIP embedding
|
`mlData`. Ente's apps detect faces in each photo and compute a CLIP embedding
|
||||||
for it, a list of 512 numbers that describes what the photo shows. `mlData`
|
for it, a list of 512 numbers that describes what the photo shows. `mlData`
|
||||||
holds everything but the `clip` block, which holds that embedding. In its place
|
holds everything but that embedding. In its place the file holds `keywords`,
|
||||||
the JSON holds `keywords`, derived from the embedding as described under
|
derived from the embedding as described under "Keywords". A file with no
|
||||||
"Keywords". A file with no machine-learning data has neither field. A file whose
|
machine-learning data has neither field. A file whose data holds no usable
|
||||||
data holds no usable embedding has no `keywords`.
|
embedding has no `keywords`.
|
||||||
|
|
||||||
The backup waits for the library's fetch of machine-learning data to finish
|
The backup waits for the library's fetch of machine-learning data to finish
|
||||||
before it writes the JSON files. When a file's data cannot be fetched,
|
before it writes the JSON files. When a file's data cannot be fetched,
|
||||||
@@ -748,11 +736,11 @@ derives a file's keywords the way the app sorts it. The categories are:
|
|||||||
|
|
||||||
Each category has a query, and a text embedding of that query from Ente's CLIP
|
Each category has a query, and a text embedding of that query from Ente's CLIP
|
||||||
text model. A file's score for a category is the cosine similarity of the file's
|
text model. A file's score for a category is the cosine similarity of the file's
|
||||||
embedding and the category's. That is a number from -1 to 1; the more alike the
|
embedding and the category's: a number from -1 to 1, higher the more alike the
|
||||||
two embeddings are, the higher it is. The file falls in the category when its
|
two embeddings are. The file falls in the category when its score reaches the
|
||||||
score reaches the category's minimum score. The keywords are the titles of the
|
category's minimum score. The keywords are the titles of the categories the file
|
||||||
categories the file falls in, highest score first. A video never falls in a
|
falls in, highest score first. A video never falls in a category marked "no
|
||||||
category marked "no videos", as in the app.
|
videos", as in the app.
|
||||||
|
|
||||||
#### Collection directories
|
#### Collection directories
|
||||||
|
|
||||||
@@ -775,21 +763,17 @@ ignoring case, each gets its ID added: two albums named `Trip` become
|
|||||||
`Trip (10)/` and `Trip (11)/`. Two files in one collection whose titles match,
|
`Trip (10)/` and `Trip (11)/`. Two files in one collection whose titles match,
|
||||||
ignoring case, get their IDs added before the extension: `IMG_0001 (12345).JPG`
|
ignoring case, get their IDs added before the extension: `IMG_0001 (12345).JPG`
|
||||||
and `IMG_0001 (12346).JPG`. A collection named like another collection's JSON
|
and `IMG_0001 (12346).JPG`. A collection named like another collection's JSON
|
||||||
file, such as `Trip.json` beside `Trip`, also gets its ID added. A file name
|
file, such as `Trip.json` beside `Trip`, also gets its ID added. A name too long
|
||||||
longer than 255 bytes, the limit on most file systems, gets its ID added too. So
|
for a file name, which most file systems limit to 255 bytes, gets its ID added
|
||||||
does a collection name longer than 250 bytes, which leaves room for `.json`.
|
too. It is then cut short before the ID, just far enough to fit. IDs never
|
||||||
Either name is then cut short before the ID, just far enough to fit. IDs never
|
|
||||||
change, so a name stays the same from run to run until such a clash appears or
|
change, so a name stays the same from run to run until such a clash appears or
|
||||||
goes away.
|
goes away.
|
||||||
|
|
||||||
Each run removes every symlink in a collection's directory that points into the
|
Each run removes the symlinks that no longer belong in a collection's directory.
|
||||||
date folders and is not one of its current links, yours included. A file or
|
It removes the directory and JSON of a collection that was deleted or renamed,
|
||||||
symlink of yours at the name of one of its links is replaced by the link. A
|
once the directory holds nothing else. Nothing else under `collections/` is
|
||||||
directory of yours at that name stays, and the photo whose link it blocks goes
|
touched: a file or symlink you put there stays, and a directory that still holds
|
||||||
into `failures.json`. Anything else you put under `collections/` stays. A run
|
one stays too, with its JSON.
|
||||||
removes the directory and JSON of a collection that was deleted or renamed, once
|
|
||||||
the directory holds nothing else. One that still holds something of yours stays,
|
|
||||||
with its JSON.
|
|
||||||
|
|
||||||
A collection whose directory or JSON cannot be written does not stop the run.
|
A collection whose directory or JSON cannot be written does not stop the run.
|
||||||
For example, a file you put there may already have the directory's name. The run
|
For example, a file you put there may already have the directory's name. The run
|
||||||
@@ -834,9 +818,8 @@ Each entry holds:
|
|||||||
|
|
||||||
- `fileID` and `title`;
|
- `fileID` and `title`;
|
||||||
- `classification`: the kind of failure;
|
- `classification`: the kind of failure;
|
||||||
- `attempts`: how many runs in a row have failed the file;
|
- `attempts`: how many runs have tried the file;
|
||||||
- `lastTriedAt`: when the run that last tried it finished its refresh, as a Unix
|
- `lastTriedAt`: when it was last tried, as a Unix time in milliseconds;
|
||||||
time in milliseconds;
|
|
||||||
- `error`: the error, as quak printed it.
|
- `error`: the error, as quak printed it.
|
||||||
|
|
||||||
The kinds are:
|
The kinds are:
|
||||||
@@ -854,13 +837,13 @@ file outside them. `failures.json` is deleted when the list is empty.
|
|||||||
`backup.lock` keeps two backups of the same directory from running at once, such
|
`backup.lock` keeps two backups of the same directory from running at once, such
|
||||||
as a cron run that starts while the previous one is still going. A backup
|
as a cron run that starts while the previous one is still going. A backup
|
||||||
creates `<dir>` if it is missing and takes the lock before its refresh, so a
|
creates `<dir>` if it is missing and takes the lock before its refresh, so a
|
||||||
refused run sends no request. A run that still holds the lock removes it when it
|
refused run sends no request. It removes the lock when it ends, whether it
|
||||||
ends, whether it succeeded or failed.
|
succeeded or failed.
|
||||||
|
|
||||||
The lock is a directory that
|
The lock is a directory that
|
||||||
[proper-lockfile](https://github.com/moxystudio/node-proper-lockfile) creates.
|
[proper-lockfile](https://github.com/moxystudio/node-proper-lockfile) creates.
|
||||||
While the backup runs, it updates the modification time of `backup.lock` every 5
|
While the backup runs, the backup updates the directory's modification time
|
||||||
seconds. A second backup of `<dir>` fails at once: it prints
|
every 5 seconds. A second backup of the directory fails at once: it prints
|
||||||
`quak: another backup of <dir> is running` and exits 2. A run that is killed, or
|
`quak: another backup of <dir> is running` and exits 2. A run that is killed, or
|
||||||
cut off by a crash, can leave the lock behind. Once the lock has gone 10 seconds
|
cut off by a crash, can leave the lock behind. Once the lock has gone 10 seconds
|
||||||
without an update, the next run takes it over, so nobody has to remove it by
|
without an update, the next run takes it over, so nobody has to remove it by
|
||||||
@@ -903,13 +886,13 @@ The summary and the `--json` output add the counts `verified`, `mismatched` and
|
|||||||
`unchecked`. Each counts only originals that were already stored. A mismatch
|
`unchecked`. Each counts only originals that were already stored. A mismatch
|
||||||
that was replaced with matching bytes does not make the exit code non-zero.
|
that was replaced with matching bytes does not make the exit code non-zero.
|
||||||
|
|
||||||
A run without `--verify` drops two kinds of failure from `failures.json`:
|
Two kinds of failure stay in `failures.json` only until the next run:
|
||||||
|
|
||||||
- a mismatched original whose new copy failed;
|
- a mismatched original whose new copy failed;
|
||||||
- a stored original that cannot be read.
|
- a stored original that cannot be read.
|
||||||
|
|
||||||
That run finds the original in place, counts it as skipped and drops the entry,
|
A run without `--verify` finds the original in place, counts it as skipped, and
|
||||||
so it can exit 0. Only a `--verify` run hashes the original again.
|
drops the entry, so that run can exit 0. Only a `--verify` run hashes it again.
|
||||||
|
|
||||||
Without `--verify`, no stored original is hashed, the summary is unchanged, and
|
Without `--verify`, no stored original is hashed, the summary is unchanged, and
|
||||||
the three counts are 0 in the JSON.
|
the three counts are 0 in the JSON.
|
||||||
@@ -935,11 +918,7 @@ scheme. Opening a library deletes such leftovers in the cache's `originals/` and
|
|||||||
The rename replaces whatever was at the destination rather than writing through
|
The rename replaces whatever was at the destination rather than writing through
|
||||||
it. A symlink there is replaced, not followed, and the new file has the
|
it. A symlink there is replaced, not followed, and the new file has the
|
||||||
temporary file's permissions, not those of the file it replaced. Each original
|
temporary file's permissions, not those of the file it replaced. Each original
|
||||||
is written with mode `0600`, readable by its owner only. Everything else the
|
is written with mode `0600`, readable by its owner only.
|
||||||
backup writes, its folders and JSON files included, gets the mode your umask
|
|
||||||
allows, which usually lets every user on the machine read it. The JSON beside an
|
|
||||||
original holds the file's location and faces. The `backup-metadata` tree is
|
|
||||||
written the same way.
|
|
||||||
|
|
||||||
### The metadata dump directory
|
### The metadata dump directory
|
||||||
|
|
||||||
@@ -964,9 +943,9 @@ of the same name apart.
|
|||||||
`files`. Each `<fileID>.json` holds the fields a backup's JSON beside an
|
`files`. Each `<fileID>.json` holds the fields a backup's JSON beside an
|
||||||
original holds, with these differences:
|
original holds, with these differences:
|
||||||
|
|
||||||
- `imageMetadata` is written only with `--exif`. The original of every file,
|
- `imageMetadata` is written only with `--exif`. The original of every file is
|
||||||
videos included, is then downloaded and read for its EXIF, XMP and JPEG
|
then downloaded, videos included, and read as a backup reads it (see "The JSON
|
||||||
dimensions.
|
beside each original").
|
||||||
- A file whose original holds no image metadata gets no `imageMetadata` field.
|
- A file whose original holds no image metadata gets no `imageMetadata` field.
|
||||||
- An original that cannot be downloaded or read gives `imageMetadataError`
|
- An original that cannot be downloaded or read gives `imageMetadataError`
|
||||||
instead.
|
instead.
|
||||||
@@ -1047,24 +1026,23 @@ onto what the cache holds, empty or not. A refresh then starts only when
|
|||||||
- `contentSource`: a replacement for the client's own byte source. Mainly for
|
- `contentSource`: a replacement for the client's own byte source. Mainly for
|
||||||
tests.
|
tests.
|
||||||
|
|
||||||
In the content and thumbnail pools, your own requests take the next free slot
|
In each pool, your own requests take the next free slot ahead of the two
|
||||||
ahead of the two background downloads. A file already in the cache is not
|
background downloads. A file already in the cache is not fetched again. When the
|
||||||
fetched again. When the originals cache grows past its limit, the least recently
|
originals cache grows past its limit, the least recently used originals are
|
||||||
used originals are evicted, but never a pinned one, and never one whose fetch is
|
evicted, but never a pinned one, and never one whose fetch is still in flight.
|
||||||
still in flight.
|
|
||||||
|
|
||||||
`onProgress` receives a `RefreshEvent`. Its `operation` names the work:
|
`onProgress` receives a `RefreshEvent`. Its `operation` names the work:
|
||||||
`"refresh"`, `"fetchMLData"` for a machine-learning fetch, and
|
`"refresh"`, `"fetchMLData"` for a machine-learning fetch, and
|
||||||
`"precacheThumbnails"` or `"precacheOriginals"` for the two background
|
`"precacheThumbnails"` or `"precacheOriginals"` for the two background
|
||||||
downloads. Its `status` is `"started"` as the work starts and `"done"` as it
|
downloads. Its `status` is `"started"` as the work starts and `"done"` as it
|
||||||
ends. A refresh or a machine-learning fetch that fails ends with `"failed"`
|
ends. A refresh or a machine-learning fetch that fails ends with `"failed"` and
|
||||||
instead, with the message in `error`. In between, a refresh sends `"progress"`
|
the error message instead. In between, a refresh sends `"progress"` after each
|
||||||
after each collection whose files it fetched: `fetched` is the number of
|
collection whose files it fetched: `fetched` is the number of collections
|
||||||
collections fetched so far, and `total` the number it will fetch. A refresh
|
fetched so far, and `total` the number it will fetch. A refresh fetches the
|
||||||
fetches the files of each collection that is new or changed since the cache's
|
files of each collection that is new or changed since the cache's copy, so on an
|
||||||
copy, so on an empty cache it fetches those of every collection. The `"done"`
|
empty cache it fetches those of every collection. The `"done"` event of a
|
||||||
event of a machine-learning fetch, and of each background download, also carries
|
machine-learning fetch, and of each background download, also carries `fetched`:
|
||||||
`fetched`: how many items it newly stored.
|
how many items it newly stored.
|
||||||
|
|
||||||
`lib.status()` returns a `LibraryStatus`. It holds the collection and file
|
`lib.status()` returns a `LibraryStatus`. It holds the collection and file
|
||||||
counts. `lastRefreshAt` is the time of the last refresh that succeeded, and
|
counts. `lastRefreshAt` is the time of the last refresh that succeeded, and
|
||||||
@@ -1079,10 +1057,10 @@ cached, so the next fetch asks for it again. The reasons are kept in memory
|
|||||||
only.
|
only.
|
||||||
|
|
||||||
`lib.close()` stops the background timer. It can be called more than once. A
|
`lib.close()` stops the background timer. It can be called more than once. A
|
||||||
refresh in flight is left to finish, with its write to the cache. `close()`
|
refresh in flight is left to finish, with its write to the cache. The promise
|
||||||
returns a promise. It resolves once that refresh, the machine-learning fetch and
|
`close()` returns resolves once that refresh, the machine-learning fetch and the
|
||||||
the background downloads already running have all finished. The cache directory
|
background downloads already running have all finished. The cache directory can
|
||||||
can then be removed. After `close()`, `fresh()`, `freshCollections()`,
|
then be removed. After `close()`, `fresh()`, `freshCollections()`,
|
||||||
`freshFiles()` and `lib.backup()` reject with `the library is closed`.
|
`freshFiles()` and `lib.backup()` reject with `the library is closed`.
|
||||||
|
|
||||||
### Default reads and fresh reads
|
### Default reads and fresh reads
|
||||||
@@ -1187,11 +1165,11 @@ progress of that one file.
|
|||||||
keyed by tag name, each as exifreader decodes it with its `id`, `value`,
|
keyed by tag name, each as exifreader decodes it with its `id`, `value`,
|
||||||
`description` and `computed` value. For example, `Make` is
|
`description` and `computed` value. For example, `Make` is
|
||||||
`{ id: 271, value: ["Example"], description: "Example", computed: "Example" }`.
|
`{ id: 271, value: ["Example"], description: "Example", computed: "Example" }`.
|
||||||
A tag that exifreader has no name for is keyed `undefined-<tag number>`. The
|
A tag exifreader has no name for is keyed `undefined-<tag number>`. The tags
|
||||||
tags of the thumbnail embedded in the file are under `Thumbnail`, so they
|
of the thumbnail embedded in the file are under `Thumbnail`, so they cannot
|
||||||
cannot hide the main image's tags of the same name. The thumbnail image itself
|
hide the main image's tags of the same name. The thumbnail image itself is
|
||||||
is left out. Any file exifreader cannot read gives `{}`, and a video gives
|
left out. Any file exifreader cannot read gives `{}`, and a video gives `{}`
|
||||||
`{}` without being downloaded.
|
without being downloaded.
|
||||||
- `await photo.make(opts?)` returns one common EXIF field, picked from the tags
|
- `await photo.make(opts?)` returns one common EXIF field, picked from the tags
|
||||||
`exif()` returns and typed as in `PhotoExif`, or `undefined` when the file
|
`exif()` returns and typed as in `PhotoExif`, or `undefined` when the file
|
||||||
lacks it. These methods do the same for their fields:
|
lacks it. These methods do the same for their fields:
|
||||||
@@ -1279,9 +1257,7 @@ symlinks. Once the lock is lost, it rejects with
|
|||||||
`BackupOptions`:
|
`BackupOptions`:
|
||||||
|
|
||||||
- `downloadDirectory`: where the tree lives. Default: the library's.
|
- `downloadDirectory`: where the tree lives. Default: the library's.
|
||||||
- `includeOriginals`: fetch and store originals. Default `true`. With `false`,
|
- `includeOriginals`: fetch and store originals. Default `true`.
|
||||||
the backup also fetches no machine-learning data, and writes no JSON beside
|
|
||||||
the originals and no symlinks.
|
|
||||||
- `includeThumbnails`: also store thumbnails as `thumbnails/<fileID>.jpg` beside
|
- `includeThumbnails`: also store thumbnails as `thumbnails/<fileID>.jpg` beside
|
||||||
`collections/`. Default `false`. `quak backup` does not set it.
|
`collections/`. Default `false`. `quak backup` does not set it.
|
||||||
- `onlyAlbumNames`: back up only the albums with these names.
|
- `onlyAlbumNames`: back up only the albums with these names.
|
||||||
@@ -1302,18 +1278,15 @@ releases it once the backup ends.
|
|||||||
many requests run at once, because Ente meters these three kinds of traffic
|
many requests run at once, because Ente meters these three kinds of traffic
|
||||||
differently:
|
differently:
|
||||||
|
|
||||||
- metadata, with 10 slots: only the machine-learning data requests;
|
- metadata, with 10 slots;
|
||||||
- content, with 5: originals;
|
- content, with 5;
|
||||||
- thumbnails, with 25: thumbnails.
|
- thumbnails, with 25.
|
||||||
|
|
||||||
Only the requests named above go through the pools. A refresh,
|
Construct one with
|
||||||
`freshCollections()` and `freshFiles()` send theirs outside them. Construct one
|
|
||||||
with
|
|
||||||
`new RequestPools({ metadataConcurrency, contentConcurrency, thumbnailConcurrency })`
|
`new RequestPools({ metadataConcurrency, contentConcurrency, thumbnailConcurrency })`
|
||||||
and pass it as `pools`. Each pool serves on-demand work ahead of background
|
and pass it as `pools`. Each pool serves on-demand work ahead of background
|
||||||
work. Two tasks given to `BoundedPool.run()` with the same `key` run once while
|
work. A task submitted under a key that is still pending is not run a second
|
||||||
the first is pending, and both get its result. An idle pool never lends its
|
time. An idle pool never lends its slots to a busy one.
|
||||||
slots to a busy one.
|
|
||||||
|
|
||||||
`lib.backup()` works on as many originals at once as the content pool allows, 5
|
`lib.backup()` works on as many originals at once as the content pool allows, 5
|
||||||
by default. `quak backup` uses the default. The limit covers everything it does
|
by default. `quak backup` uses the default. The limit covers everything it does
|
||||||
@@ -1383,9 +1356,9 @@ They differ because a download's length depends on the file and the link. A
|
|||||||
whole-transfer deadline short enough to catch a hung connection would cancel a
|
whole-transfer deadline short enough to catch a hung connection would cancel a
|
||||||
large video on a slow link that is still making progress. The download deadline
|
large video on a slow link that is still making progress. The download deadline
|
||||||
restarts each time bytes arrive, so a slow download runs as long as it keeps
|
restarts each time bytes arrive, so a slow download runs as long as it keeps
|
||||||
moving, and one that stalls is aborted after 60 seconds of silence. The download
|
moving, and one that stalls is aborted after 60 seconds of silence. It covers
|
||||||
deadline covers the wait for the headers as well as the body. There is no limit
|
the wait for the headers as well as the body. There is no limit on the total
|
||||||
on the total length of a download.
|
length of a download.
|
||||||
|
|
||||||
A request that changes state is not blindly replayed. Every `POST` and `PUT` in
|
A request that changes state is not blindly replayed. Every `POST` and `PUT` in
|
||||||
the endpoint list under "HTTP API" goes through this rule. Verifying a second
|
the endpoint list under "HTTP API" goes through this rule. Verifying a second
|
||||||
@@ -1407,13 +1380,13 @@ that rule although they look like connect-time failures. On Linux they are also
|
|||||||
delivered on an established socket, after the request may have been written.
|
delivered on an established socket, after the request may have been written.
|
||||||
They stay retryable for requests that change nothing. Bun's `ConnectionRefused`
|
They stay retryable for requests that change nothing. Bun's `ConnectionRefused`
|
||||||
and `FailedToOpenSocket` are treated the same way: a request that changes
|
and `FailedToOpenSocket` are treated the same way: a request that changes
|
||||||
nothing is retried on them, and a `POST` or `PUT` is not replayed on them. Bun
|
nothing is retried on them, and a `POST` or `PUT` is not replayed on them. When
|
||||||
reuses a connection it kept open from an earlier request. When that connection
|
a connection bun kept open from an earlier request closes before any answer, bun
|
||||||
closes before any answer, bun sends the request again on a new one. If that one
|
sends the request again on a new connection. If that one fails, bun gives one of
|
||||||
fails, bun gives one of these codes although the server may already have the
|
these codes although the server may already have the request. The upload of a
|
||||||
request. The upload of a thumbnail to its presigned URL is exempt from the rule.
|
thumbnail to its presigned URL is exempt from the rule. It stores one whole
|
||||||
It stores one whole object at one key in one request, so replaying it has no
|
object at one key in one request, so replaying it has no partial state to
|
||||||
partial state to damage.
|
damage.
|
||||||
|
|
||||||
A download is retried as a whole: the request, the reading of the body and the
|
A download is retried as a whole: the request, the reading of the body and the
|
||||||
decryption. A socket reset after the response headers have arrived surfaces in
|
decryption. A socket reset after the response headers have arrived surfaces in
|
||||||
@@ -1438,9 +1411,8 @@ return, quak never keeps a truncated file.
|
|||||||
`Client.login({ email, password, totp?, emailOTP?, apiOptions? })` logs in and
|
`Client.login({ email, password, totp?, emailOTP?, apiOptions? })` logs in and
|
||||||
resolves to a `Client`. `totp` and `emailOTP` are callbacks that return the code
|
resolves to a `Client`. `totp` and `emailOTP` are callbacks that return the code
|
||||||
when the account asks for one. Without the callback the account needs, login
|
when the account asks for one. Without the callback the account needs, login
|
||||||
throws. A passkey challenge always throws. After an email code, login expects
|
throws. A passkey challenge always throws. `apiOptions` configures the HTTP
|
||||||
the key attributes and the token. When the server asks for a second factor
|
client (see below).
|
||||||
instead, login throws. `apiOptions` configures the HTTP client (see below).
|
|
||||||
|
|
||||||
The `Client` holds the session token, the master key, the secret key and the
|
The `Client` holds the session token, the master key, the secret key and the
|
||||||
public key in memory. There is no on-disk session store in the library; the
|
public key in memory. There is no on-disk session store in the library; the
|
||||||
@@ -1450,16 +1422,9 @@ write wherever it likes. `Client.fromJSON(snapshot, apiOptions?)` restores a
|
|||||||
working client from that snapshot without logging in again. It checks every
|
working client from that snapshot without logging in again. It checks every
|
||||||
field and each key's length first, and throws an error naming the bad field.
|
field and each key's length first, and throws an error naming the bad field.
|
||||||
|
|
||||||
`Client.fromJSON` works only once libsodium, the cryptography library quak uses,
|
|
||||||
is loaded. `Client.login` loads it, and `Client.fromJSON` does not. A program
|
|
||||||
that restores a session in a new process first runs `await sodium.ready`, with
|
|
||||||
`sodium` imported from `libsodium-wrappers-sumo`, which quak depends on. The
|
|
||||||
exported `beginLogin`, `unwrapAuth`, `decryptCollection`, `decryptFile`,
|
|
||||||
`downloadFile` and `downloadThumbnail` need the same.
|
|
||||||
|
|
||||||
`client.logout()` clears the token and zeroes the key buffers in place. After
|
`client.logout()` clears the token and zeroes the key buffers in place. After
|
||||||
it, every other method on that client throws. `logout()` does not contact the
|
it, every other method on that client throws. It does not contact the server, so
|
||||||
server, so the token stays valid there and in any saved snapshot. Call
|
the token stays valid there and in any saved snapshot. Call
|
||||||
`await client.logoutOnServer()` before `logout()` to end the session on the
|
`await client.logoutOnServer()` before `logout()` to end the session on the
|
||||||
server.
|
server.
|
||||||
|
|
||||||
@@ -1479,10 +1444,8 @@ return only the decrypted records that are not deleted, from `sinceTime` `0`.
|
|||||||
`client.fetchMLData({ fileIDs, fileKeys })` fetches and decrypts the
|
`client.fetchMLData({ fileIDs, fileKeys })` fetches and decrypts the
|
||||||
machine-learning data of up to 200 files. `client.downloadFile(file, outPath?)`
|
machine-learning data of up to 200 files. `client.downloadFile(file, outPath?)`
|
||||||
and `client.downloadThumbnail(file, outPath?)` download and decrypt one file to
|
and `client.downloadThumbnail(file, outPath?)` download and decrypt one file to
|
||||||
`outPath`. For a live photo, `downloadFile` writes the image and the video, each
|
`outPath`. Without `outPath`, they write to the working directory under the
|
||||||
named `outPath` with its own extension. `quak get --out` instead puts the image
|
names `quak get` and `quak get-thumb` would use.
|
||||||
at the path itself. Without `outPath`, they write to the working directory under
|
|
||||||
the names `quak get` and `quak get-thumb` would use.
|
|
||||||
|
|
||||||
`ApiClientOptions`:
|
`ApiClientOptions`:
|
||||||
|
|
||||||
@@ -1677,8 +1640,8 @@ Login derives the keys in this order:
|
|||||||
the code (`POST /users/two-factor/verify`); the server then returns the key
|
the code (`POST /users/two-factor/verify`); the server then returns the key
|
||||||
attributes and the token. quak cannot answer a passkey request. When the
|
attributes and the token. quak cannot answer a passkey request. When the
|
||||||
account has email verification on (`isEmailMFAEnabled`), an email code
|
account has email verification on (`isEmailMFAEnabled`), an email code
|
||||||
replaces SRP. quak then expects the key attributes and the token in the
|
replaces SRP, and the server returns the key attributes and the token after
|
||||||
answer, and fails the login when the server asks for a second factor instead.
|
it.
|
||||||
5. `crypto_secretbox_open_easy` over the encrypted master key with the KEK
|
5. `crypto_secretbox_open_easy` over the encrypted master key with the KEK
|
||||||
yields the 32-byte master key.
|
yields the 32-byte master key.
|
||||||
6. `crypto_secretbox_open_easy` over the encrypted secret key with the master
|
6. `crypto_secretbox_open_easy` over the encrypted secret key with the master
|
||||||
@@ -1757,35 +1720,36 @@ This repository adheres to the
|
|||||||
standard. Normalized scripts in `script/` are the entrypoints for the
|
standard. Normalized scripts in `script/` are the entrypoints for the
|
||||||
development workflow, and most Makefile targets are thin shims that call them.
|
development workflow, and most Makefile targets are thin shims that call them.
|
||||||
The scripts are POSIX `sh`, not bash, so they run in minimal containers such as
|
The scripts are POSIX `sh`, not bash, so they run in minimal containers such as
|
||||||
Alpine. Of these, only `bootstrap`, `setup`, `test` and `cibuild` come from the
|
Alpine. The scripts marked "ours" are this repository's own additions to the
|
||||||
standard. The others are additions to it.
|
standard.
|
||||||
|
|
||||||
- `script/bootstrap`: install all dependencies. It installs `make`, `git`, Node
|
- `script/bootstrap`: install all dependencies. It installs `make`, `git`, Node
|
||||||
and `yarn` when they are missing, then runs `yarn install --frozen-lockfile`.
|
and `yarn` when they are missing, then runs `yarn install --frozen-lockfile`.
|
||||||
- `script/setup`: set up the repo for development after a fresh clone. It runs
|
- `script/setup`: set up the repo for development after a fresh clone. It runs
|
||||||
`script/bootstrap`, then `script/install-precommit`.
|
`script/bootstrap`, then `script/install-precommit`.
|
||||||
- `script/projectname`: print the project name. `script/docker` uses it for the
|
- `script/projectname` (ours): print the project name. `script/docker` uses it
|
||||||
image tag.
|
for the image tag.
|
||||||
- `script/build`: compile the TypeScript sources into `dist/` and stamp the
|
- `script/build` (ours): compile the TypeScript sources into `dist/` and stamp
|
||||||
version into `dist/package.json`. Then check that the entrypoints
|
the version into `dist/package.json`. Then check that the entrypoints
|
||||||
`package.json` declares (`main`, `types`, `bin`) were written, and make the
|
`package.json` declares (`main`, `types`, `bin`) were written, and make the
|
||||||
command line executable.
|
command line executable.
|
||||||
- `script/version`: print the version `script/build` stamps (see "Version").
|
- `script/version` (ours): print the version `script/build` stamps (see
|
||||||
|
"Version").
|
||||||
- `script/test`: run the test suite, by building the `test` phase of the
|
- `script/test`: run the test suite, by building the `test` phase of the
|
||||||
`Dockerfile`. Requires Docker.
|
`Dockerfile`. Requires Docker.
|
||||||
- `script/lint`: run eslint and a prettier check, by building the `lint` phase
|
- `script/lint`: run eslint and a prettier check, by building the `lint` phase
|
||||||
of the `Dockerfile`. Requires Docker.
|
of the `Dockerfile`. Requires Docker.
|
||||||
- `script/fmt`: format all files with prettier. It rewrites files in place.
|
- `script/fmt`: format all files with prettier. It rewrites files in place.
|
||||||
- `script/fmt-check`: check formatting on the host. It is read-only.
|
- `script/fmt-check`: check formatting on the host. It is read-only.
|
||||||
- `script/check`: run all checks: `test`, `lint`, `fmt-check`.
|
- `script/check` (ours): run all checks: `test`, `lint`, `fmt-check`.
|
||||||
- `script/docker`: build the image, tagged with `script/projectname`.
|
- `script/docker`: build the image, tagged with `script/projectname`.
|
||||||
- `script/cibuild`: what CI runs: `script/bootstrap`, then `script/check`, then
|
- `script/cibuild`: what CI runs: `script/bootstrap`, then `script/check`, then
|
||||||
the image build.
|
the image build.
|
||||||
- `script/precommit`: run by the git pre-commit hook. It runs `script/lint` and
|
- `script/precommit` (ours): run by the git pre-commit hook. It runs
|
||||||
`script/fmt-check`, and not the tests, so that a commit of failing tests can
|
`script/lint` and `script/fmt-check`, and not the tests, so that a commit of
|
||||||
land (see "Development workflow").
|
failing tests can land (see "Development workflow").
|
||||||
- `script/install-precommit`: install the git pre-commit hook. `make hooks`
|
- `script/install-precommit` (ours): install the git pre-commit hook.
|
||||||
calls it.
|
`make hooks` calls it.
|
||||||
|
|
||||||
The Makefile targets are:
|
The Makefile targets are:
|
||||||
|
|
||||||
@@ -1852,8 +1816,8 @@ empty, `dev` or `unknown`, because such a build could not be traced to its
|
|||||||
commit.
|
commit.
|
||||||
|
|
||||||
`.dockerignore` therefore does not leave out `.git`, so any `docker build .` of
|
`.dockerignore` therefore does not leave out `.git`, so any `docker build .` of
|
||||||
a clone stamps the commit it was built from. A clone of depth 1 stamps a tag
|
a clone stamps the commit it was built from. A shallow clone stamps a tag only
|
||||||
only when its commit carries one, and otherwise the short commit.
|
when the cloned commit itself carries one, and otherwise the short commit.
|
||||||
`.dockerignore` does leave out `.git/config`, which holds the clone's remote URL
|
`.dockerignore` does leave out `.git/config`, which holds the clone's remote URL
|
||||||
and any credential in it; `git describe` does not need that file.
|
and any credential in it; `git describe` does not need that file.
|
||||||
`script/docker` and `script/cibuild` work out the version on the host, with
|
`script/docker` and `script/cibuild` work out the version on the host, with
|
||||||
@@ -1909,12 +1873,7 @@ These apply to people and to coding agents alike.
|
|||||||
- the nvm archive `script/bootstrap` downloads, and the Python packages
|
- the nvm archive `script/bootstrap` downloads, and the Python packages
|
||||||
`bin/clip-keyword-vectors` installs, by SHA-256 hash;
|
`bin/clip-keyword-vectors` installs, by SHA-256 hash;
|
||||||
- Node and yarn, when `script/bootstrap` installs them, by version.
|
- Node and yarn, when `script/bootstrap` installs them, by version.
|
||||||
- Not pinned:
|
- bun is not pinned: `make build-bin` uses whatever `nix-shell -p bun` provides.
|
||||||
- bun, which `make build-bin` takes from `nix-shell -p bun`;
|
|
||||||
- `make` and `git`, which `script/bootstrap` installs from the system
|
|
||||||
package manager when they are missing, as every `Dockerfile` stage does;
|
|
||||||
- `bash` and `curl`, which `script/bootstrap` installs the same way when it
|
|
||||||
has to install Node.
|
|
||||||
- Follow the workflow above: branches off `next`, tests first and committed
|
- Follow the workflow above: branches off `next`, tests first and committed
|
||||||
failing, and `next` merged into `main` by sneak alone.
|
failing, and `next` merged into `main` by sneak alone.
|
||||||
- Before every commit, `make lint` and `make fmt-check` must pass. `make lint`
|
- Before every commit, `make lint` and `make fmt-check` must pass. `make lint`
|
||||||
|
|||||||
+7
-10
@@ -1,18 +1,15 @@
|
|||||||
// The CLI's log level, and the one function the commands write stderr lines
|
// The CLI's log level, and the one function the commands write stderr lines
|
||||||
// through.
|
// through.
|
||||||
//
|
//
|
||||||
// Every line quak logs goes to stderr, at one of six levels, from the
|
// Every line quak prints on stderr is a log line at one of six levels, from
|
||||||
// quietest: fatal, error, warning, info, debug, trace. A run prints the lines
|
// the quietest: fatal, error, warning, info, debug, trace. A run prints the
|
||||||
// at its own level and at every quieter one. Its level is info unless `-v` or
|
// lines at its own level and at every quieter one. Its level is info unless
|
||||||
// `-q` asks for another (see "What quak prints" in the README). A command that
|
// `-v` or `-q` asks for another (see "What quak prints" in the README). Every
|
||||||
// runs and exits non-zero logs a fatal line saying why, so even `-q`, which
|
// command that exits non-zero logs a fatal line saying why, so even `-q`, which
|
||||||
// prints only fatal lines, never leaves such an exit unexplained. An error
|
// prints only fatal lines, never leaves a non-zero exit unexplained. An error
|
||||||
// line reports something that failed, such as a file or an album, and a
|
// line reports something that failed, such as a file or an album, and a
|
||||||
// warning something that did not fail but needs the user's attention. stdout
|
// warning something that did not fail but needs the user's attention. stdout
|
||||||
// carries a command's output and the prompts of `login`, and is not a log.
|
// carries only a command's output and is not a log.
|
||||||
// Two things on stderr are not log lines: the `error:` message for a
|
|
||||||
// command line that quak cannot parse, and the usage printed when no command
|
|
||||||
// is given.
|
|
||||||
//
|
//
|
||||||
// The library itself writes no lines: its layers report through callbacks, and
|
// The library itself writes no lines: its layers report through callbacks, and
|
||||||
// the CLI logs what they report. `logRequests` gives the API client options
|
// the CLI logs what they report. `logRequests` gives the API client options
|
||||||
|
|||||||
@@ -328,7 +328,7 @@ describe("what the library reports", () => {
|
|||||||
]);
|
]);
|
||||||
});
|
});
|
||||||
|
|
||||||
// Each operation is named in plain words.
|
// Each operation is named in plain words, as the README names it.
|
||||||
it("logs a refresh's progress at info, and its start and end at debug", () => {
|
it("logs a refresh's progress at info, and its start and end at debug", () => {
|
||||||
const { log, lines } = collect();
|
const { log, lines } = collect();
|
||||||
const onProgress = logRefresh(log);
|
const onProgress = logRefresh(log);
|
||||||
|
|||||||
Reference in New Issue
Block a user