README: running quak backup from cron, and its exit codes (closes #170)
check / check (push) Failing after 50s
check / check (push) Failing after 50s
A new README section shows how to run quak backup unattended: log in once as the job's user, a crontab with a backup every night and --verify on Sundays, output appended to a log file, and the HOME and XDG_DATA_HOME the job needs to find the saved session. A table gives each exit code as the code returns it, and says which failures the next run retries. The introduction now lists everything the backup keeps for each file. It, the backup layout tree and the lib.backup() entry say EXIF and XMP are kept for an image only, and dimensions for a JPEG only. Judgement call: the --verify run takes Sunday's slot rather than a second job that night, since an overlapping run would exit 2. Model: opus-5-5
This commit is contained in:
@@ -10,9 +10,12 @@ account into a deduplicated local directory tree, skipping files that already
|
||||
exist on disk and continuing past individual download failures instead of
|
||||
crashing. For each file it persists the basic metadata fields quak keeps (title,
|
||||
file type, creation and modification time, latitude, longitude, content hash),
|
||||
and the private and public magic metadata in full. A helper subcommand can
|
||||
detect and regenerate missing thumbnails, encrypting and uploading them back to
|
||||
the server.
|
||||
its update time, the private and public magic metadata in full, Ente's ML data
|
||||
for it when there is any, and, for an image (for a live photo, its image), its
|
||||
original's EXIF and XMP, with its dimensions for a JPEG only; for a video it
|
||||
keeps none of these three. It runs unattended from cron (see "Running the backup
|
||||
from cron"). A helper subcommand can detect and regenerate missing thumbnails,
|
||||
encrypting and uploading them back to the server.
|
||||
|
||||
## Getting Started
|
||||
|
||||
@@ -80,6 +83,64 @@ await lib.close();
|
||||
The lower-level `Client` (login, session serialization, and the raw
|
||||
enumeration/download calls) is exported too and documented under Design below.
|
||||
|
||||
## Running the backup from cron
|
||||
|
||||
`quak backup` never prompts, so cron can run it. `make install` builds quak as a
|
||||
single binary and copies it to `~/bin/quak`. Log in once with it, as the user
|
||||
the cron job will run as; the session is saved in that user's data directory
|
||||
(see "Session handling"):
|
||||
|
||||
```bash
|
||||
~/bin/quak login
|
||||
```
|
||||
|
||||
Then add the backup to that user's crontab with `crontab -e`. These two lines
|
||||
back up the account to `~/photos-backup` at 03:30 every night, with `--verify`
|
||||
on Sundays, and append all output to `~/quak-backup.log`:
|
||||
|
||||
```
|
||||
30 3 * * 1-6 $HOME/bin/quak backup $HOME/photos-backup >> $HOME/quak-backup.log 2>&1
|
||||
30 3 * * 0 $HOME/bin/quak backup --verify $HOME/photos-backup >> $HOME/quak-backup.log 2>&1
|
||||
```
|
||||
|
||||
A run with `--verify` does all a plain run does, and also checks each original
|
||||
already in the backup against the content hash Ente records for it, and replaces
|
||||
any that do not match (see "Backup layout"). Sunday's run is the `--verify` one,
|
||||
not a second job that night, because a backup that starts while another backup
|
||||
of the same directory is running exits 2 without backing anything up.
|
||||
|
||||
Cron runs the job with a short `PATH`, usually `/usr/bin:/bin`, so the crontab
|
||||
names quak by its full path. To find the saved session, quak needs the same
|
||||
`HOME` as when you logged in, which cron sets from the password file, and on
|
||||
Linux the same `XDG_DATA_HOME`: quak looks for the session in
|
||||
`$XDG_DATA_HOME/quak`, or in `~/.local/share/quak` when that is not set. Cron
|
||||
does not set `XDG_DATA_HOME`, so if your login shell does, set it at the top of
|
||||
the crontab too. Cron does not expand variables in such a line, so give the full
|
||||
path:
|
||||
|
||||
```
|
||||
XDG_DATA_HOME=/home/you/.data
|
||||
```
|
||||
|
||||
On macOS the session is in `~/Library/Application Support/quak`, and only `HOME`
|
||||
matters.
|
||||
|
||||
### Exit codes
|
||||
|
||||
| Code | Meaning |
|
||||
| ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `0` | The backup is complete: every original is at its save path, and no file failed. |
|
||||
| `1` | The run finished with files in `failures.json`, or it stopped on another error, printed as one line `quak: <message>`. The next run tries again. |
|
||||
| `2` | Another backup of the same directory is running. This run sent no request and changed nothing. |
|
||||
| `3` | There is no usable session: none is saved, the saved one is corrupt, or the server no longer accepts it. Run `quak login` as the user the cron job runs as. |
|
||||
|
||||
After a `1`, the next run fetches each missing original again, fetches the ML
|
||||
data that is not cached, and rebuilds the album links. An original that a
|
||||
`--verify` run could not read, or put back with bytes that still do not match,
|
||||
stays at its save path. Only a later `--verify` run checks it again: a run
|
||||
without `--verify` leaves it as it is and takes the file out of `failures.json`,
|
||||
so that run can exit 0.
|
||||
|
||||
## Examples
|
||||
|
||||
`examples/download-albums.ts` downloads every album's photos and their metadata
|
||||
@@ -615,8 +676,11 @@ the smallest does not.
|
||||
below)
|
||||
YYYY-MM-DD.<fileID>.json the file's basic metadata fields quak
|
||||
keeps, its update time, its private and
|
||||
public magic metadata, its ML data, and
|
||||
its original's EXIF, XMP and dimensions
|
||||
public magic metadata, its ML data, and,
|
||||
for an image (for a live photo, its
|
||||
image), its original's EXIF and XMP, with
|
||||
dimensions for a JPEG only; none of these
|
||||
three for a video
|
||||
YYYY-MM-DD.<fileID>.livephoto.json
|
||||
which of a live photo's two files is which
|
||||
collections/
|
||||
@@ -962,14 +1026,16 @@ photos newest first). `lib.subscribe({ onChange })` delivers a `LibraryChange`
|
||||
`fresh()` does, puts every in-scope original not already at its save path
|
||||
there as `photo.download()` does (and, with `includeThumbnails`, fetches
|
||||
thumbnails) through the content cache, waits for an ML data fetch, and
|
||||
rebuilds the on-disk backup tree, each file's JSON with its ML data and its
|
||||
original's EXIF, XMP and dimensions, with a durable failure ledger. A fetched
|
||||
original is written straight to its save path and not into the cache, which
|
||||
then counts it as present; one the cache already held is copied from there.
|
||||
`BackupOptions`: `downloadDirectory` (falls back to the library's),
|
||||
`includeOriginals` (default `true`), `includeThumbnails` (default `false`),
|
||||
`onlyAlbumNames`, `verify` (default `false`), `onProgress`, and `lockHeld`
|
||||
(default `false`). See Backup layout above for the tree it writes.
|
||||
rebuilds the on-disk backup tree with a durable failure ledger. Each file's
|
||||
JSON holds its ML data and, for an image (for a live photo, its image), its
|
||||
original's EXIF and XMP, with its dimensions for a JPEG only; a video's JSON
|
||||
holds none of these three. A fetched original is written straight to its save
|
||||
path and not into the cache, which then counts it as present; one the cache
|
||||
already held is copied from there. `BackupOptions`: `downloadDirectory` (falls
|
||||
back to the library's), `includeOriginals` (default `true`),
|
||||
`includeThumbnails` (default `false`), `onlyAlbumNames`, `verify` (default
|
||||
`false`), `onProgress`, and `lockHeld` (default `false`). See Backup layout
|
||||
above for the tree it writes.
|
||||
|
||||
### Request pools
|
||||
|
||||
|
||||
@@ -25,6 +25,15 @@ declares one.
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-10-06: The README says how to run `quak backup` from cron (issue 170):
|
||||
log in once as the job's user, a crontab with a backup every night and
|
||||
`--verify` on Sundays appending to a log file, and the `HOME` and
|
||||
`XDG_DATA_HOME` the job needs to find the saved session. A table gives each
|
||||
exit code: 0, the backup is complete; 1, files are in `failures.json` or
|
||||
another error stopped the run; 2, another backup of the directory is running;
|
||||
3, there is no usable session. The introduction lists everything the backup
|
||||
keeps for each file.
|
||||
|
||||
- 2026-10-06: `quak backup --verify` and `lib.backup({ verify: true })` hash
|
||||
each original already at its save path as the download check does, streamed, a
|
||||
live photo as `<imageHash>:<videoHash>` (issue 168). One that does not match
|
||||
|
||||
Reference in New Issue
Block a user