Store live photos as their image and their video (closes #107)
check / check (push) Successful in 41s
check / check (push) Successful in 41s
A live photo, which Ente stores as one ZIP, is unpacked as it downloads into its image and its video, each `<fileID>.<ext>` with its extension from the ZIP, beside `<fileID>.livephoto.json`, which names the two. Both are checked against the recorded hash and renamed into place only when both are complete. The backup and the content cache count a live photo as stored only with both files, album folders link both, `quak get` writes both, and the content result gives the video as `videoPath`. A ZIP an earlier version stored is replaced. Model: opus-5-5
This commit is contained in:
@@ -470,10 +470,14 @@ cache lives; without it each account gets its own directory under the per-user
|
||||
cache path.
|
||||
|
||||
`get` and `get-thumb` resolve the file by ID directly, so `--collection` is
|
||||
accepted for backward compatibility but ignored. `backup-metadata --exif` (alias
|
||||
`--all`) additionally downloads each file to extract full EXIF/IPTC/XMP
|
||||
metadata. The listing and backup commands support `--json` for machine-readable
|
||||
output.
|
||||
accepted for backward compatibility but ignored. For a live photo, `get` writes
|
||||
its image and its video, each named after the title with its own extension, as
|
||||
Ente's clients name them (`IMG_0001.heic` and `IMG_0001.mov`). With
|
||||
`--out PATH`, the image is written to `PATH` and the video beside it, with
|
||||
`PATH`'s name and the video's extension; a `PATH` with the video's extension is
|
||||
refused. `backup-metadata --exif` (alias `--all`) additionally downloads each
|
||||
file to extract full EXIF/IPTC/XMP metadata. The listing and backup commands
|
||||
support `--json` for machine-readable output.
|
||||
|
||||
`backup-metadata` fetches ML data in requests of up to 200 files. When a request
|
||||
still fails after its retries, the error is logged, each of its files is written
|
||||
@@ -499,8 +503,10 @@ the smallest does not.
|
||||
```
|
||||
<dir>/
|
||||
originals/
|
||||
<fileID>.<ext> actual file content (one per unique file)
|
||||
<fileID>.<ext> actual file content (one per unique file,
|
||||
two for a live photo: see below)
|
||||
<fileID>.json all decrypted metadata for that file
|
||||
<fileID>.livephoto.json which of a live photo's two files is which
|
||||
collections/
|
||||
<name>/
|
||||
<title> -> ../../originals/<fileID>.<ext> (symlink)
|
||||
@@ -516,12 +522,22 @@ library's `lib.backup({ includeThumbnails: true })` also writes
|
||||
|
||||
A collection's directory and JSON are named after the collection, and a symlink
|
||||
after the file's title, both with unsafe characters replaced. When two
|
||||
collections would get the same name, or two files in one collection the same
|
||||
title (ignoring case in both), each of them gets its ID added: two albums named
|
||||
collections would get the same name, or two symlinks in one collection the same
|
||||
name (ignoring case in both), each of them gets its ID added: two albums named
|
||||
`Trip` become `Trip (10)/` and `Trip (11)/`, and two files titled `IMG_0001.JPG`
|
||||
become `IMG_0001 (12345).JPG` and `IMG_0001 (12346).JPG`. IDs never change, so a
|
||||
name stays the same from run to run until such a clash appears or goes away.
|
||||
|
||||
A live photo, which Ente stores as one ZIP of its image and its video, is stored
|
||||
as those two files, which a photo viewer can open: each is
|
||||
`originals/<fileID>.<ext>` with the extension it has inside the ZIP (for example
|
||||
`12345.heic` and `12345.mov`), and `<fileID>.livephoto.json` names the two. The
|
||||
live photo counts as stored only when both files are present and not empty. Its
|
||||
album folder links both, each named after the title with that file's extension
|
||||
(`IMG_0001.heic` and `IMG_0001.mov`). A live photo that an earlier version of
|
||||
quak stored as the ZIP, under the image's name, is replaced by its two files on
|
||||
the next run, and the ZIP and its link are removed.
|
||||
|
||||
Each run removes the symlinks into `originals/` that no longer belong in their
|
||||
collection's directory, and the directories (and JSON) of collections that were
|
||||
deleted or renamed. Nothing else in `collections/` is touched: a file or a
|
||||
@@ -555,9 +571,9 @@ temporary file's permissions, not those of the file it replaced.
|
||||
errors
|
||||
- [x] Update the API reference section below to match the current implementation
|
||||
- [x] `make docker` green
|
||||
- [ ] Store live photos in a form a photo viewer can open
|
||||
(https://git.eeqj.de/sneak/quak/issues/107), once sneak has chosen between
|
||||
keeping the ZIP and unpacking it
|
||||
- [x] Store live photos in a form a photo viewer can open
|
||||
(https://git.eeqj.de/sneak/quak/issues/107): unpacked into the image and
|
||||
the video
|
||||
|
||||
Tagging and releases are decided by sneak alone, and happen only when he
|
||||
declares one.
|
||||
@@ -649,7 +665,9 @@ An `Album` exposes its record fields and `album.photos.list()` → `Photo[]`
|
||||
(newest first). A `Photo` exposes its record fields, `photo.record()` →
|
||||
`PhotoRecord`, and two content methods:
|
||||
|
||||
- `await photo.original(opts?)` → `{ path, bytes }` — the full-resolution file.
|
||||
- `await photo.original(opts?)` → `{ path, bytes, videoPath? }` — the
|
||||
full-resolution file. For a live photo, `path` and `bytes` are its image's and
|
||||
`videoPath` is its video.
|
||||
- `await photo.thumbnail(opts?)` → `{ path, bytes }`.
|
||||
|
||||
Both serve from the on-disk content cache when the bytes are present and
|
||||
@@ -669,7 +687,7 @@ The GUI-facing records hold no key material and no binary, so they survive
|
||||
- `PhotoRecord`: `fileID`, `albumIDs`, `title`, `takenAt` (milliseconds),
|
||||
`fileType`, optional `caption` / `width` / `height` / `latitude` /
|
||||
`longitude`, `isArchived`, `isHidden`, and `thumbnailPath` / `originalPath`
|
||||
once the bytes are cached.
|
||||
once the bytes are cached (for a live photo, `originalPath` is its image).
|
||||
- `AlbumRecord`: `collectionID`, `name`, `type`, `isShared`, `updationTime`, and
|
||||
`fileIDs` (newest first).
|
||||
- `LibrarySnapshot`: `{ albums, photos, takenAt }`.
|
||||
@@ -720,6 +738,8 @@ Under `cacheDirectory`:
|
||||
<cacheDirectory>/
|
||||
metadata.json decrypted account state + refresh cursor
|
||||
originals/<fileID>.<ext> cached full-resolution files
|
||||
originals/<fileID>.livephoto.json
|
||||
which of a live photo's two files is which
|
||||
thumbnails/<fileID>.jpg cached thumbnails
|
||||
mldata/
|
||||
<fileID>.json one decrypted ML payload per file
|
||||
@@ -732,14 +752,21 @@ When `metadata.json` belongs to a different account than the client's,
|
||||
originals and thumbnails are kept; they are reached only through the files the
|
||||
current account's records name.
|
||||
|
||||
A live photo's original is cached as in the backup: its image and its video,
|
||||
each `originals/<fileID>.<ext>` with its own extension, and
|
||||
`originals/<fileID>.livephoto.json` naming them; the two are evicted together. A
|
||||
live photo that an earlier version cached as its ZIP is not served, and is
|
||||
replaced by its two files the next time it is read.
|
||||
|
||||
A stored file appears only via an atomic temp-then-rename, so its presence means
|
||||
it is complete. Every downloaded original (by `quak get`, the cache, or
|
||||
`backup`) whose metadata records a content hash (`FileMetadata.hash`) is hashed
|
||||
as it is written: unkeyed BLAKE2b with a 64-byte output, standard base64. For a
|
||||
live photo, which is stored as a ZIP, the image and the video are hashed
|
||||
separately and joined as `<imageHash>:<videoHash>`. A mismatch stores nothing
|
||||
and fails the download with an error naming the file ID. An original with no
|
||||
recorded hash, from a very old client, is stored unchecked.
|
||||
as it is written: unkeyed BLAKE2b with a 64-byte output, standard base64. A live
|
||||
photo arrives as a ZIP and is unpacked as it is written; its image and its video
|
||||
are hashed separately and joined as `<imageHash>:<videoHash>`, and neither is
|
||||
stored unless both are complete and match. A mismatch stores nothing and fails
|
||||
the download with an error naming the file ID. An original with no recorded
|
||||
hash, from a very old client, is stored unchecked.
|
||||
|
||||
### Key types by source file
|
||||
|
||||
|
||||
Reference in New Issue
Block a user