Compare commits
1
Commits
next
..
1bd6d72876
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1bd6d72876 |
@@ -10,12 +10,9 @@ account into a deduplicated local directory tree, skipping files that already
|
|||||||
exist on disk and continuing past individual download failures instead of
|
exist on disk and continuing past individual download failures instead of
|
||||||
crashing. For each file it persists the basic metadata fields quak keeps (title,
|
crashing. For each file it persists the basic metadata fields quak keeps (title,
|
||||||
file type, creation and modification time, latitude, longitude, content hash),
|
file type, creation and modification time, latitude, longitude, content hash),
|
||||||
its update time, the private and public magic metadata in full, Ente's ML data
|
and the private and public magic metadata in full. A helper subcommand can
|
||||||
for it when there is any, and, for an image (for a live photo, its image), its
|
detect and regenerate missing thumbnails, encrypting and uploading them back to
|
||||||
original's EXIF and XMP, with its dimensions for a JPEG only; for a video it
|
the server.
|
||||||
keeps none of these three. It runs unattended from cron (see "Running the backup
|
|
||||||
from cron"). A helper subcommand can detect and regenerate missing thumbnails,
|
|
||||||
encrypting and uploading them back to the server.
|
|
||||||
|
|
||||||
## Getting Started
|
## Getting Started
|
||||||
|
|
||||||
@@ -83,64 +80,6 @@ await lib.close();
|
|||||||
The lower-level `Client` (login, session serialization, and the raw
|
The lower-level `Client` (login, session serialization, and the raw
|
||||||
enumeration/download calls) is exported too and documented under Design below.
|
enumeration/download calls) is exported too and documented under Design below.
|
||||||
|
|
||||||
## Running the backup from cron
|
|
||||||
|
|
||||||
`quak backup` never prompts, so cron can run it. `make install` builds quak as a
|
|
||||||
single binary and copies it to `~/bin/quak`. Log in once with it, as the user
|
|
||||||
the cron job will run as; the session is saved in that user's data directory
|
|
||||||
(see "Session handling"):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
~/bin/quak login
|
|
||||||
```
|
|
||||||
|
|
||||||
Then add the backup to that user's crontab with `crontab -e`. These two lines
|
|
||||||
back up the account to `~/photos-backup` at 03:30 every night, with `--verify`
|
|
||||||
on Sundays, and append all output to `~/quak-backup.log`:
|
|
||||||
|
|
||||||
```
|
|
||||||
30 3 * * 1-6 $HOME/bin/quak backup $HOME/photos-backup >> $HOME/quak-backup.log 2>&1
|
|
||||||
30 3 * * 0 $HOME/bin/quak backup --verify $HOME/photos-backup >> $HOME/quak-backup.log 2>&1
|
|
||||||
```
|
|
||||||
|
|
||||||
A run with `--verify` does all a plain run does, and also checks each original
|
|
||||||
already in the backup against the content hash Ente records for it, and replaces
|
|
||||||
any that do not match (see "Backup layout"). Sunday's run is the `--verify` one,
|
|
||||||
not a second job that night, because a backup that starts while another backup
|
|
||||||
of the same directory is running exits 2 without backing anything up.
|
|
||||||
|
|
||||||
Cron runs the job with a short `PATH`, usually `/usr/bin:/bin`, so the crontab
|
|
||||||
names quak by its full path. To find the saved session, quak needs the same
|
|
||||||
`HOME` as when you logged in, which cron sets from the password file, and on
|
|
||||||
Linux the same `XDG_DATA_HOME`: quak looks for the session in
|
|
||||||
`$XDG_DATA_HOME/quak`, or in `~/.local/share/quak` when that is not set. Cron
|
|
||||||
does not set `XDG_DATA_HOME`, so if your login shell does, set it at the top of
|
|
||||||
the crontab too. Cron does not expand variables in such a line, so give the full
|
|
||||||
path:
|
|
||||||
|
|
||||||
```
|
|
||||||
XDG_DATA_HOME=/home/you/.data
|
|
||||||
```
|
|
||||||
|
|
||||||
On macOS the session is in `~/Library/Application Support/quak`, and only `HOME`
|
|
||||||
matters.
|
|
||||||
|
|
||||||
### Exit codes
|
|
||||||
|
|
||||||
| Code | Meaning |
|
|
||||||
| ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| `0` | The backup is complete: every original is at its save path, and no file failed. |
|
|
||||||
| `1` | The run finished with files in `failures.json`, or it stopped on another error, printed as one line `quak: <message>`. The next run tries again. |
|
|
||||||
| `2` | Another backup of the same directory is running. This run sent no request and changed nothing. |
|
|
||||||
| `3` | There is no usable session: none is saved, the saved one is corrupt, or the server no longer accepts it. Run `quak login` as the user the cron job runs as. |
|
|
||||||
|
|
||||||
After a `1`, the next run fetches each missing original again, fetches the ML
|
|
||||||
data that is not cached, and rebuilds the album links. An original that a
|
|
||||||
`--verify` run could not read, or put back with bytes that still do not match,
|
|
||||||
stays at its save path. Only a later `--verify` run checks it again: a run
|
|
||||||
without `--verify` leaves it as it is and takes the file out of `failures.json`,
|
|
||||||
so that run can exit 0.
|
|
||||||
|
|
||||||
## Examples
|
## Examples
|
||||||
|
|
||||||
`examples/download-albums.ts` downloads every album's photos and their metadata
|
`examples/download-albums.ts` downloads every album's photos and their metadata
|
||||||
@@ -676,11 +615,8 @@ the smallest does not.
|
|||||||
below)
|
below)
|
||||||
YYYY-MM-DD.<fileID>.json the file's basic metadata fields quak
|
YYYY-MM-DD.<fileID>.json the file's basic metadata fields quak
|
||||||
keeps, its update time, its private and
|
keeps, its update time, its private and
|
||||||
public magic metadata, its ML data, and,
|
public magic metadata, its ML data, and
|
||||||
for an image (for a live photo, its
|
its original's EXIF, XMP and dimensions
|
||||||
image), its original's EXIF and XMP, with
|
|
||||||
dimensions for a JPEG only; none of these
|
|
||||||
three for a video
|
|
||||||
YYYY-MM-DD.<fileID>.livephoto.json
|
YYYY-MM-DD.<fileID>.livephoto.json
|
||||||
which of a live photo's two files is which
|
which of a live photo's two files is which
|
||||||
collections/
|
collections/
|
||||||
@@ -1026,16 +962,14 @@ photos newest first). `lib.subscribe({ onChange })` delivers a `LibraryChange`
|
|||||||
`fresh()` does, puts every in-scope original not already at its save path
|
`fresh()` does, puts every in-scope original not already at its save path
|
||||||
there as `photo.download()` does (and, with `includeThumbnails`, fetches
|
there as `photo.download()` does (and, with `includeThumbnails`, fetches
|
||||||
thumbnails) through the content cache, waits for an ML data fetch, and
|
thumbnails) through the content cache, waits for an ML data fetch, and
|
||||||
rebuilds the on-disk backup tree with a durable failure ledger. Each file's
|
rebuilds the on-disk backup tree, each file's JSON with its ML data and its
|
||||||
JSON holds its ML data and, for an image (for a live photo, its image), its
|
original's EXIF, XMP and dimensions, with a durable failure ledger. A fetched
|
||||||
original's EXIF and XMP, with its dimensions for a JPEG only; a video's JSON
|
original is written straight to its save path and not into the cache, which
|
||||||
holds none of these three. A fetched original is written straight to its save
|
then counts it as present; one the cache already held is copied from there.
|
||||||
path and not into the cache, which then counts it as present; one the cache
|
`BackupOptions`: `downloadDirectory` (falls back to the library's),
|
||||||
already held is copied from there. `BackupOptions`: `downloadDirectory` (falls
|
`includeOriginals` (default `true`), `includeThumbnails` (default `false`),
|
||||||
back to the library's), `includeOriginals` (default `true`),
|
`onlyAlbumNames`, `verify` (default `false`), `onProgress`, and `lockHeld`
|
||||||
`includeThumbnails` (default `false`), `onlyAlbumNames`, `verify` (default
|
(default `false`). See Backup layout above for the tree it writes.
|
||||||
`false`), `onProgress`, and `lockHeld` (default `false`). See Backup layout
|
|
||||||
above for the tree it writes.
|
|
||||||
|
|
||||||
### Request pools
|
### Request pools
|
||||||
|
|
||||||
|
|||||||
@@ -25,15 +25,6 @@ declares one.
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
- 2026-10-06: The README says how to run `quak backup` from cron (issue 170):
|
|
||||||
log in once as the job's user, a crontab with a backup every night and
|
|
||||||
`--verify` on Sundays appending to a log file, and the `HOME` and
|
|
||||||
`XDG_DATA_HOME` the job needs to find the saved session. A table gives each
|
|
||||||
exit code: 0, the backup is complete; 1, files are in `failures.json` or
|
|
||||||
another error stopped the run; 2, another backup of the directory is running;
|
|
||||||
3, there is no usable session. The introduction lists everything the backup
|
|
||||||
keeps for each file.
|
|
||||||
|
|
||||||
- 2026-10-06: `quak backup --verify` and `lib.backup({ verify: true })` hash
|
- 2026-10-06: `quak backup --verify` and `lib.backup({ verify: true })` hash
|
||||||
each original already at its save path as the download check does, streamed, a
|
each original already at its save path as the download check does, streamed, a
|
||||||
live photo as `<imageHash>:<videoHash>` (issue 168). One that does not match
|
live photo as `<imageHash>:<videoHash>` (issue 168). One that does not match
|
||||||
|
|||||||
Reference in New Issue
Block a user