Write each backed-up original once and skip the precache (closes #106)
check / check (push) Successful in 49s
check / check (push) Successful in 49s
An original fetched for a backup is now written by the download writer straight into the backup's originals/, and the content cache records it there instead of keeping its own copy. One the cache already held is still copied. `quak backup` opens its library with the thumbnail and originals precache off, as the one-shot commands do. Model: opus-5-5
This commit was merged in pull request #124.
This commit is contained in:
@@ -510,22 +510,25 @@ symlink you put there stays, and a directory that still holds one after its
|
||||
symlinks are removed stays too, with its JSON.
|
||||
|
||||
Each file is downloaded exactly once regardless of how many collections it
|
||||
appears in. On subsequent runs, existing originals are skipped. If a download
|
||||
fails, the error is logged and the backup continues with the next file. The exit
|
||||
code is non-zero if any files failed.
|
||||
appears in, and written once: straight into `originals/`, with no copy left in
|
||||
the cache. An original the cache already held is copied from there instead. On
|
||||
subsequent runs, existing originals are skipped. If a download fails, the error
|
||||
is logged and the backup continues with the next file. The exit code is non-zero
|
||||
if any files failed. `quak backup` opens its library with the thumbnail and
|
||||
originals precache off, so it fetches only what the backup stores.
|
||||
|
||||
Each original is copied to a temporary file named
|
||||
`.quak-backup-<fileID>.<ext>-<pid>-<random>.tmp` in the same directory, synced
|
||||
to disk, and renamed into place, so an original is either complete or absent,
|
||||
even after a power cut. A run that is killed can leave one of these temporary
|
||||
files behind; the next backup deletes those whose process is no longer running.
|
||||
Downloads and the content cache use the same scheme with
|
||||
`.quak-<pid>-<random>.tmp` names, and opening a library deletes those in the
|
||||
cache whose process is no longer running, so a download another process has in
|
||||
progress in the same cache is left alone. The rename replaces whatever was at
|
||||
the destination rather than writing through it: a symlink there is replaced, not
|
||||
followed, and the new file has the temporary file's permissions, not those of
|
||||
the file it replaced.
|
||||
Each original is written to a temporary file in the same directory, synced to
|
||||
disk, and renamed into place, so an original is either complete or absent, even
|
||||
after a power cut. A downloaded original's temporary file is named
|
||||
`.quak-<pid>-<random>.tmp`, one copied from the cache
|
||||
`.quak-backup-<fileID>.<ext>-<pid>-<random>.tmp`. A run that is killed can leave
|
||||
one of these temporary files behind; the next backup deletes those whose process
|
||||
is no longer running. The content cache uses the same scheme, and opening a
|
||||
library deletes the temporary files in the cache whose process is no longer
|
||||
running, so a download another process has in progress in the same cache is left
|
||||
alone. The rename replaces whatever was at the destination rather than writing
|
||||
through it: a symlink there is replaced, not followed, and the new file has the
|
||||
temporary file's permissions, not those of the file it replaced.
|
||||
|
||||
## TODO
|
||||
|
||||
@@ -667,12 +670,15 @@ photos newest first). `lib.subscribe({ onChange })` delivers a `LibraryChange`
|
||||
default limit 20). quak bundles no text encoder, so `searchByEmbedding` takes
|
||||
a query vector the caller produced elsewhere.
|
||||
- `await lib.backup(opts?)` → `BackupResult`. It refreshes, fetches every
|
||||
in-scope original (and, with `includeThumbnails`, thumbnails) through the
|
||||
content cache, and rebuilds the on-disk backup tree with a durable failure
|
||||
ledger. `BackupOptions`: `downloadDirectory` (falls back to the one `open()`
|
||||
was given), `includeOriginals` (default `true`), `includeThumbnails` (default
|
||||
`false`), `onlyAlbumNames`, and `onProgress`. See Backup layout above for the
|
||||
tree it writes.
|
||||
in-scope original not already in the backup (and, with `includeThumbnails`,
|
||||
thumbnails) through the content cache, and rebuilds the on-disk backup tree
|
||||
with a durable failure ledger. A fetched original is written straight into the
|
||||
backup's `originals/` and not into the cache, which then counts it as present;
|
||||
one the cache already held is copied from there. `BackupOptions`:
|
||||
`downloadDirectory` (falls back to the one `open()` was given),
|
||||
`includeOriginals` (default `true`), `includeThumbnails` (default `false`),
|
||||
`onlyAlbumNames`, and `onProgress`. See Backup layout above for the tree it
|
||||
writes.
|
||||
|
||||
### Request pools
|
||||
|
||||
|
||||
Reference in New Issue
Block a user