Compare commits

..
8 Commits
Author SHA1 Message Date
clawbot 9e94542a57 docker build . stamps the git tag or short commit, not dev (closes #154)
check / check (push) Waiting to run
A plain `docker build .` now stamps the version from git rather than `dev`/`0.0.0`. After `tsc`, `script/build` writes into `dist/package.json` the version `script/version` decides: `VERSION` when given, otherwise `git describe --tags --always` (the tag; or tag, commits since and short commit; or the short commit), otherwise `package.json`'s. A checkout with `.git` that yields an empty, `dev` or `unknown` version fails the build. `.dockerignore` sends `.git` but not `.git/config`, so no remote URL or credential reaches the image. `ARG VERSION` has no default, and the host scripts' version still wins.

Not changed: `REPO_POLICIES.md` still says `ARG VERSION=dev` until the shared policy changes.

Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
2026-10-02 06:12:56 +02:00
clawbot e50d2a78c8 exif() returns every EXIF tag in the file (closes #156)
check / check (push) Failing after 59s
`photo.exif()` now returns every EXIF tag in the file, as the owner ruled, typed `ExifTags`: each tag keyed by name with `exifreader`'s `id`, `value`, `description` and `computed`. A tag with no name is keyed `undefined-` plus its number, and the embedded thumbnail's tags sit under `Thumbnail`. The thirteen typed methods stay, with the same names and types; each now picks its field from `exif()`'s tags. GPS latitude and longitude are worked out from the GPS tags and their reference tags, and a position with no reference tags gives neither. `backup-metadata --exif` output is unchanged.

Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
2026-10-02 05:30:05 +02:00
clawbot 0c995a8c4f Remove the content cache's handling of an earlier version's live-photo ZIP (closes #151)
check / check (push) Failing after 44s
quak is pre-1.0 and keeps no handling of old data. The content cache no longer recognises or removes a live photo that an earlier quak version cached as one ZIP. A live-photo download no longer removes what was at its destination before renaming the image and video into place; the rename already replaces it. The README sentences and the tests about that old ZIP are gone. The two removed tests that also covered current behaviour are replaced by tests with no ZIP: the cache re-fetching a live photo recorded with no video, and the library telling the cache which files are live photos when it opens.

Model: opus-5-5
2026-10-02 04:14:30 +02:00
clawbot d788c5457d Thumbnail test: re-encode a small image so it cannot time out (closes #153)
check / check (push) Failing after 45s
The test "re-encodes smaller until the thumbnail fits the recorded size" built a noisy 400x300 JPEG and encoded it several times. That came close to vitest's 5 s limit and timed out on a busy host, which made `script/cibuild` fail on `next` and `main`. It now uses a noisy 64x48 JPEG, still too big for the first, default-quality encoding to fit, and keeps every assertion. It runs in about 1 s.

Model: opus-5-5
2026-10-02 03:49:23 +02:00
clawbot 10afa7a7f4 Example script: download every album's photos and metadata (closes #144)
check / check (push) Successful in 1m20s
`examples/download-albums.ts` logs in from `QUAK_EMAIL` and `QUAK_PASSWORD`, then walks every album from `await lib.fresh()`. For each photo it runs `photo.download()` to the photo's save path and writes `{savePath}.json`, which holds the photo's record and EXIF. For each album it writes `{dir}/albums/{collectionID}.json` with the album's name and its photos' save paths. A file is written only when its content changes, so a second run downloads and rewrites nothing.

The script opens the library with prefetching off, as `quak backup` does. The build type-checks `examples/`. The README says how to run it.

Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
2026-10-02 00:33:00 +02:00
clawbot d40f339c0b Photo: one async method per EXIF field (closes #148)
check / check (push) Successful in 1m37s
`Photo` gains thirteen async methods, one per `PhotoExif` field and named after it: `make()`, `model()`, `lensModel()`, `dateTimeOriginal()`, `offsetTimeOriginal()`, `exposureTime()`, `fNumber()`, `iso()`, `focalLength()`, `orientation()`, `gpsLatitude()`, `gpsLongitude()` and `gpsAltitude()`. Each calls `exif()` and returns its one field, or `undefined` when the file lacks it. `exif()` is unchanged.

`Photo` implements a type built from `PhotoExif`'s keys, so the type check fails when a field has no method. Each call reads the original again; a caller that wants several fields calls `exif()` once.

Model: opus-5-5
2026-10-02 00:09:24 +02:00
clawbot 10e1a9ef39 Save path ./photos/YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.fileID.ext; download() from the cache first (closes #143)
check / check (push) Successful in 1m25s
Originals are saved at `{downloadDirectory}/YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.{fileID}{ext}`. The date is the photo's `takenAt` in local time, and `downloadDirectory` defaults to `./photos`, resolved when the library opens. The old `originals/` layout is gone.

`photo.download()` writes the original to `savePath`. It copies from the cache when the cache holds the original, and fetches otherwise. `lib.backup()` uses the same path and rule, and every album in `collections/` links to it. `isLocal` is true only when the original is at `savePath`.

For a file in several albums, one rule picks the copy everything uses: the most recently synced, with the lowest album ID breaking a tie.

Model: opus-5-5
2026-10-01 23:20:58 +02:00
clawbot 67d554fb46 exif(): read HEIF/HEIC originals with exifreader (closes #145)
check / check (push) Successful in 2m10s
`photo.exif()` and `quak backup-metadata --exif` now read EXIF through `exifreader`. HEIC/HEIF originals get EXIF, including a live photo's image, as do the other formats `exifreader` reads. It replaces `exif-reader` and the hand-written JPEG scan. `PhotoExif` is unchanged.

The `backup-metadata` dump now holds `exifreader`'s tag output, with unnamed tags keyed `undefined-` plus their number. GPS altitude without a reference counts as above sea level. A latitude or longitude without its hemisphere tag, an unreadable text tag, and a date the parser rejects each give no field.

Licence: `exifreader` is MPL-2.0, used unmodified.

Model: opus-5-5
2026-10-01 22:34:09 +02:00
30 changed files with 1678 additions and 372 deletions
+6 -3
View File
@@ -1,9 +1,12 @@
# Mirrors .gitignore, with one deliberate exception: .gitignore itself stays # Mirrors .gitignore, with one deliberate exception: .gitignore itself stays
# in the build context, because prettier 3 reads it as a default ignore file # in the build context, because prettier 3 reads it as a default ignore file
# and dropping it would change what the lint phase's prettier check sees. # and dropping it would change what the lint phase's prettier check sees.
#
# VCS # .git is deliberately NOT excluded: the build derives the version it stamps
.git # from it (script/version). It is sent without its config, which holds the
# clone's remote URL and any credential in it, and which the build stage, the
# final image, would otherwise carry. git describe does not need it.
.git/config
# OS # OS
.DS_Store .DS_Store
+6 -3
View File
@@ -61,9 +61,12 @@ RUN script/bootstrap
COPY . . COPY . .
# The version is computed on the host and passed in, because # Version stamped into the build: the VERSION build arg when one is given,
# .dockerignore excludes .git. # otherwise what script/version derives from the .git the build context
ARG VERSION=dev # carries (script/bootstrap installed git), so any `docker build .` of a
# clone stamps its commit. The label can only carry the build arg, and is
# empty without one.
ARG VERSION
LABEL org.opencontainers.image.version="${VERSION}" LABEL org.opencontainers.image.version="${VERSION}"
RUN make build RUN make build
+4 -2
View File
@@ -26,8 +26,10 @@ check:
build: build:
@script/build @script/build
build-bin: # Bundles the built dist/, so the binary reports the version script/build
nix-shell -p bun --run "bun build bin/quak.ts --compile --outfile bin/quak" # stamped.
build-bin: build
nix-shell -p bun --run "bun build dist/bin/quak.js --compile --outfile bin/quak"
install: build-bin install: build-bin
mkdir -p ~/bin mkdir -p ~/bin
+105 -26
View File
@@ -80,6 +80,37 @@ await lib.close();
The lower-level `Client` (login, session serialization, and the raw The lower-level `Client` (login, session serialization, and the raw
enumeration/download calls) is exported too and documented under Design below. enumeration/download calls) is exported too and documented under Design below.
## Examples
`examples/download-albums.ts` downloads every album's photos and their metadata
into a directory, `photos` in the working directory unless you name another. The
build compiles it; run it after `yarn install`:
```bash
yarn build
QUAK_EMAIL=… QUAK_PASSWORD=… node dist/examples/download-albums.js [dir]
```
It opens the library with `precacheThumbnails` and `precacheOriginals` off, as
`quak backup` does, so the only file content it fetches is the originals it
saves. It asks on the terminal for a two-factor or email code when the account
requires one, and writes:
- each photo's original at its save path under `dir`, as `photo.download()`
writes it: `YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>`, and for a live
photo its image, its video and the `.livephoto.json` file naming them
- beside each original, a JSON file named after it with `.json` added, for
example `2026-03-01.12345.jpg.json`: the photo's record (`photo.record()`)
without its cache paths, and every EXIF tag of the photo (`photo.exif()`)
under `exif`
- `albums/<collectionID>.json` for each album: its `collectionID`, its `name`,
and under `savePaths` the save paths of its photos relative to `dir`, newest
first
A photo in several albums is downloaded once. A second run downloads nothing and
rewrites only the JSON files whose content changed. A failed download stops the
run; running it again carries on, since every photo already saved is skipped.
## Entrypoints ## Entrypoints
This repository adheres to the This repository adheres to the
@@ -95,9 +126,12 @@ alpine. We provide:
`script/bootstrap`, then `script/install-precommit` `script/bootstrap`, then `script/install-precommit`
- `script/projectname` — output the project name (our own extension); used by - `script/projectname` — output the project name (our own extension); used by
`script/docker` for the image tag `script/docker` for the image tag
- `script/build` — compile the TypeScript sources into `dist/`, then verify that - `script/build` — compile the TypeScript sources into `dist/`, stamp the
the entrypoints `package.json` declares (`main`, `types`, `bin`) are among the version into `dist/package.json`, then verify that the entrypoints
files the compiler wrote, and make the CLI executable (our own extension) `package.json` declares (`main`, `types`, `bin`) are among the files the
compiler wrote, and make the CLI executable (our own extension)
- `script/version` — print the version `script/build` stamps (our own
extension); see Version below
- `script/test` — run the test suite, by building the `test` phase of the - `script/test` — run the test suite, by building the `test` phase of the
`Dockerfile` (vitest, 90s timeout, verbose rerun on failure); requires docker `Dockerfile` (vitest, 90s timeout, verbose rerun on failure); requires docker
- `script/lint` — run eslint and a prettier check, by building the `lint` phase - `script/lint` — run eslint and a prettier check, by building the `lint` phase
@@ -145,6 +179,34 @@ an exact version, installed from `yarn.lock` under `--frozen-lockfile` in both
places, and reads `.gitignore` as its default ignore file — which is why places, and reads `.gitignore` as its default ignore file — which is why
`.dockerignore` keeps `.gitignore` in the build context. `.dockerignore` keeps `.gitignore` in the build context.
### Version
`quak --version` reports the `version` of `dist/package.json`, which
`script/build` writes after compiling; the repo's own `package.json` keeps
`0.0.0`, and that is what the tests, which run from source, report.
`script/version` decides what is written:
- the `VERSION` environment variable, or the `Dockerfile`'s `VERSION` build arg
(`--build-arg VERSION=...`), when one is given and not empty;
- otherwise, in a checkout with `.git`, `git describe --tags --always`: the tag
on a tagged commit; the tag, the commits since it and the short commit on a
later commit (`v1.2.3-4-gabc1234`); the short commit when no tag is reachable;
- otherwise, as in a source tarball, the version `package.json` declares.
The build fails if the checkout has `.git` and the version still comes out
empty, `dev` or `unknown`: such a build could not be traced back to its commit.
`.dockerignore` therefore does not leave out `.git`, so any `docker build .` of
a clone stamps the commit it was built from; a shallow clone stamps a tag only
when the cloned commit itself carries one, and otherwise the short commit. It
leaves out `.git/config`, which holds the clone's remote URL and any credential
in it, so the image carries `.git` without its config; `git describe` does not
need that file. `script/docker` (and so `make docker`) and `script/cibuild` pass
the version they resolve on the host, with `--dirty`, as the build arg, which
takes precedence. The image's `org.opencontainers.image.version` label carries
that build arg only, so a build given none leaves it empty. `make build-bin`
bundles the built `dist/`, so the single binary reports the stamped version too.
## Rationale ## Rationale
Ente is one of very few photo services with a credible end-to-end encryption Ente is one of very few photo services with a credible end-to-end encryption
@@ -224,7 +286,7 @@ quak/
backup.ts resilient full-account backup with dedup backup.ts resilient full-account backup with dedup
metadata-backup.ts metadata-backup.ts
backup-metadata: the metadata quak keeps, as JSON backup-metadata: the metadata quak keeps, as JSON
exif.ts EXIF read from a JPEG's bytes exif.ts EXIF read from an image's bytes with exifreader
mldata-fetch.ts fetch + decrypt per-file ML data mldata-fetch.ts fetch + decrypt per-file ML data
filename.ts safe file names from server metadata filename.ts safe file names from server metadata
errors.ts error types shared across layers errors.ts error types shared across layers
@@ -239,6 +301,9 @@ quak/
index.ts public library exports index.ts public library exports
bin/ bin/
quak.ts CLI entrypoint (commander.js) quak.ts CLI entrypoint (commander.js)
examples/
download-albums.ts
download every album's photos and metadata
test/ unit + integration tests (vitest) test/ unit + integration tests (vitest)
Makefile Makefile
Dockerfile lint phase, test phase, compile Dockerfile lint phase, test phase, compile
@@ -247,10 +312,11 @@ quak/
``` ```
`make build` compiles that tree into `dist/`, preserving its shape: the library `make build` compiles that tree into `dist/`, preserving its shape: the library
lands in `dist/src/` and the CLI in `dist/bin/quak.js`, which is what lands in `dist/src/`, the examples in `dist/examples/`, and the CLI in
`package.json` points `main`, `types` and `bin` at. The compiler's `rootDir` is `dist/bin/quak.js`, which is what `package.json` points `main`, `types` and
the repository root rather than `src/`, because `bin/` is compiled too and `bin` at. The compiler's `rootDir` is the repository root rather than `src/`,
`rootDir` has to contain everything that is compiled. because `bin/` is compiled too and `rootDir` has to contain everything that is
compiled.
### Cryptography ### Cryptography
@@ -495,10 +561,15 @@ Ente's clients name them (`IMG_0001.heic` and `IMG_0001.mov`). With
`--out PATH`, the image is written to `PATH` and the video beside it, with `--out PATH`, the image is written to `PATH` and the video beside it, with
`PATH`'s name and the video's extension; a `PATH` with the video's extension is `PATH`'s name and the video's extension; a `PATH` with the video's extension is
refused. `backup-metadata --exif` (alias `--all`) additionally fetches each refused. `backup-metadata --exif` (alias `--all`) additionally fetches each
file's original through the cache and records its XMP metadata and, for a JPEG, file's original through the cache and records, from it or a live photo's image,
its EXIF metadata and dimensions. `collections`, `files`, `backup`, its XMP metadata, its EXIF metadata and, for a JPEG, its dimensions. EXIF is
`helper list-missing-thumbnails` and `helper fix-missing-thumbnails` take read with [exifreader](https://github.com/mattiasw/ExifReader) from any image
`--json` for machine-readable output. format it reads, JPEG, HEIC/HEIF, AVIF, PNG and WebP among them. The record's
`exif` field is exifreader's EXIF tag output: each tag by name, with its
`value`, `description` and `computed` value. An EXIF block exifreader finds but
reads no tag from is recorded, base64, as `exifRaw`, with the reason in
`exifError`. `collections`, `files`, `backup`, `helper list-missing-thumbnails`
and `helper fix-missing-thumbnails` take `--json` for machine-readable output.
`backup-metadata` fetches ML data in requests of up to 200 files. When a request `backup-metadata` fetches ML data in requests of up to 200 files. When a request
fails, the error is logged, each of its files is written with the reason in an fails, the error is logged, each of its files is written with the reason in an
@@ -721,7 +792,7 @@ synchronous getters look at the disk and never touch the network:
- `photo.isLocal` → `boolean` — whether the whole original is at `savePath`. A - `photo.isLocal` → `boolean` — whether the whole original is at `savePath`. A
copy only in the cache does not count. copy only in the cache does not count.
Five async methods may download: These async methods may download:
- `await photo.original(opts?)` → `{ path, bytes, videoPath? }` — the - `await photo.original(opts?)` → `{ path, bytes, videoPath? }` — the
full-resolution file. For a live photo, `path` and `bytes` are its image's and full-resolution file. For a live photo, `path` and `bytes` are its image's and
@@ -734,14 +805,25 @@ Five async methods may download:
- `await photo.thumbnail(opts?)` → `{ path, bytes }`. - `await photo.thumbnail(opts?)` → `{ path, bytes }`.
- `await photo.content(opts?)` → `Uint8Array` — the original's bytes, read - `await photo.content(opts?)` → `Uint8Array` — the original's bytes, read
through `original()`; for a live photo, its image's. through `original()`; for a live photo, its image's.
- `await photo.exif(opts?)` → `PhotoExif` — `make`, `model`, `lensModel`, - `await photo.exif(opts?)` → `ExifTags` — every EXIF tag in the file, keyed by
`dateTimeOriginal`, `offsetTimeOriginal`, `exposureTime`, `fNumber`, `iso`, tag name, each as exifreader decodes it, with its `id`, `value`, `description`
`focalLength`, `orientation`, `gpsLatitude`, `gpsLongitude` and `gpsAltitude`, and `computed` value: for example `Make` is
each absent when the file lacks it. GPS values are signed decimal degrees and `{ id: 271, value: ["Canon"], description: "Canon", computed: "Canon" }`. A
metres. `dateTimeOriginal` is the camera's clock reading held in the `Date`'s tag exifreader has no name for is keyed `undefined-<tag number>`. The embedded
UTC fields; `offsetTimeOriginal`, when present, is that clock's offset from thumbnail's tags are under `Thumbnail`, so they cannot hide the main image's
UTC. Only a JPEG's EXIF is read: any other original gives `{}`, and a video tags of the same name; the thumbnail image itself is left out. EXIF is read
gives `{}` without being downloaded. from any image format exifreader reads (such as JPEG, HEIC/HEIF, AVIF, PNG,
WebP and TIFF), a live photo's image included. Any other original gives `{}`,
and a video gives `{}` without being downloaded.
- `await photo.make(opts?)`, and likewise `model()`, `lensModel()`,
`dateTimeOriginal()`, `offsetTimeOriginal()`, `exposureTime()`, `fNumber()`,
`iso()`, `focalLength()`, `orientation()`, `gpsLatitude()`, `gpsLongitude()`
and `gpsAltitude()` → one common field each, picked from the tags `exif()`
returns and typed as in `PhotoExif`, or `undefined` when the file lacks it.
GPS values are signed decimal degrees and metres. `dateTimeOriginal()` is the
camera's clock reading held in the `Date`'s UTC fields;
`offsetTimeOriginal()`, when present, is that clock's offset from UTC. Each
calls `exif()` with its `opts`, so each call reads the original again.
They serve from the on-disk content cache when the bytes are present and They serve from the on-disk content cache when the bytes are present and
otherwise fetch through the pools; `original()`, `content()` and `exif()` also otherwise fetch through the pools; `original()`, `content()` and `exif()` also
@@ -833,10 +915,7 @@ current account's records name.
A live photo's original is cached as at its save path: its image and its video, A live photo's original is cached as at its save path: its image and its video,
each `originals/<fileID>.<ext>` with its own extension, and each `originals/<fileID>.<ext>` with its own extension, and
`originals/<fileID>.livephoto.json` naming them; the two are evicted together. A `originals/<fileID>.livephoto.json` naming them; the two are evicted together.
live photo that an earlier version cached as its ZIP is not served: the library
removes the ZIP when it opens the cache, and fetches the two files when the
photo is next read or precached.
A stored file appears only via an atomic temp-then-rename, so its presence means A stored file appears only via an atomic temp-then-rename, so its presence means
it is complete. Every downloaded original (by `quak get`, the cache, or it is complete. Every downloaded original (by `quak get`, the cache, or
@@ -861,7 +940,7 @@ from a very old client, is stored unchecked.
- `src/library/records.ts`: `PhotoRecord`, `AlbumRecord`, `LibrarySnapshot`, - `src/library/records.ts`: `PhotoRecord`, `AlbumRecord`, `LibrarySnapshot`,
`LibraryChange` `LibraryChange`
- `src/library/mlsearch.ts`: `MLDataAPI`, `SimilarResult` - `src/library/mlsearch.ts`: `MLDataAPI`, `SimilarResult`
- `src/exif.ts`: `PhotoExif` - `src/exif.ts`: `ExifTags`, `PhotoExif`
- `src/library/pools.ts`: `RequestPools`, `RequestPoolsOptions`, `BoundedPool` - `src/library/pools.ts`: `RequestPools`, `RequestPoolsOptions`, `BoundedPool`
- `src/backup.ts`: `BackupOptions`, `BackupResult`, `BackupError` - `src/backup.ts`: `BackupOptions`, `BackupResult`, `BackupError`
- `src/client.ts`: `Client`, `LoginOptions`, `ClientSnapshot` - `src/client.ts`: `Client`, `LoginOptions`, `ClientSnapshot`
+44
View File
@@ -25,6 +25,42 @@ declares one.
# Completed Steps # Completed Steps
- 2026-10-02: `docker build .` stamps the commit's tag or short commit, not
`dev` (issue 154). `script/build` writes the version `script/version` prints
into `dist/package.json`: the `VERSION` environment variable or build arg when
one is given, otherwise `git describe --tags --always`, otherwise the version
`package.json` declares. `.dockerignore` sends `.git`, and a checkout with
`.git` whose version comes out empty, `dev` or `unknown` fails the build.
`make build-bin` bundles the built `dist/`, so the single binary reports the
same version.
- 2026-10-02: `photo.exif()` returns every EXIF tag in the file as `ExifTags`,
keyed by tag name, each as exifreader decodes it, not only the thirteen common
fields (issue 156). The embedded thumbnail's tags are under `Thumbnail`,
without the thumbnail image. The thirteen typed methods stay, each picking its
field from the tags `exif()` returns, typed as in `PhotoExif`. The example
script's JSON files now carry every tag.
- 2026-10-01: The content cache no longer looks for a live photo that an earlier
version cached as one ZIP (issue 151). When the cache opens, a live photo's
file that no JSON file names is now always left alone.
- 2026-10-01: `examples/download-albums.ts` logs in, opens the library, and for
every album downloads each photo to its save path, writes the photo's record
and EXIF fields to a JSON file beside it, and writes the album's photos to
`albums/<collectionID>.json` (issue 144). The build compiles it to
`dist/examples/`; the README's "Examples" section says how to run it.
- 2026-10-01: A `Photo` has one async method for each field of `exif()`, named
and typed as in `PhotoExif`: `make()`, `model()`, `lensModel()`,
`dateTimeOriginal()`, `offsetTimeOriginal()`, `exposureTime()`, `fNumber()`,
`iso()`, `focalLength()`, `orientation()`, `gpsLatitude()`, `gpsLongitude()`
and `gpsAltitude()` (issue 148). Each calls `exif()` and returns its one
field, or undefined when the file lacks it. `Photo` implements a type with one
method per `PhotoExif` field, so the build's type check fails when a field has
no method. A test checks, on the JPEG and the HEIC, that each method gives the
same value as `exif()`.
- 2026-10-01: Each original's save path is - 2026-10-01: Each original's save path is
`YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>` under the library's `YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>` under the library's
download directory, which defaults to `photos` in the working directory (issue download directory, which defaults to `photos` in the working directory (issue
@@ -36,6 +72,14 @@ declares one.
file, writes each file's JSON beside its original, and links `collections/` to file, writes each file's JSON beside its original, and links `collections/` to
the save paths. the save paths.
- 2026-10-01: `photo.exif()` and `backup-metadata --exif` read EXIF from
HEIC/HEIF originals, a live photo's HEIC image included, as well as JPEG and
the other image formats `exifreader` reads (issue 145). `exifreader` replaces
`exif-reader` and the JPEG segment scan; `PhotoExif` is unchanged. The `exif`
field of `backup-metadata --exif` is now exifreader's tag output, and
`exifRaw` holds the whole EXIF block it could not read. The tests use a real
HEIC, `test/exif.heic`.
- 2026-10-01: A `Photo` has `savePath`, `isLocal`, `content()`, `exif()`, - 2026-10-01: A `Photo` has `savePath`, `isLocal`, `content()`, `exif()`,
`modifiedAt`, `hash` and `year` (issue 141). `savePath` is where `modifiedAt`, `hash` and `year` (issue 141). `savePath` is where
`lib.backup()` writes the original under the library's download directory; for `lib.backup()` writes the original under the library's download directory; for
+129
View File
@@ -0,0 +1,129 @@
// Download every album's photos to a directory, with each photo's metadata
// beside it, using only quak's public API. The README's "Examples" section
// describes the files it writes.
//
// yarn build
// QUAK_EMAIL=… QUAK_PASSWORD=… node dist/examples/download-albums.js [dir]
import { realpathSync } from "node:fs";
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { join, relative } from "node:path";
import { stdin, stdout } from "node:process";
import { createInterface } from "node:readline/promises";
import { pathToFileURL } from "node:url";
import { Client, Library } from "../src/index.js";
// Write `text` to `path` unless the file already holds exactly that, so a
// second run rewrites nothing.
async function writeIfChanged(path: string, text: string): Promise<void> {
const current = await readFile(path, "utf-8").catch(() => undefined);
if (current !== text) await writeFile(path, text);
}
const pretty = (value: unknown): string =>
JSON.stringify(value, null, 2) + "\n";
// For every album in `lib`, download each photo to its save path, write the
// photo's metadata to `{savePath}.json`, and write the album's photos to
// `{dir}/albums/{collectionID}.json`. Returns how many photos it downloaded
// and how many were already at their save paths.
export async function downloadAlbums(
lib: Library,
dir: string,
): Promise<{ downloaded: number; alreadyLocal: number }> {
let downloaded = 0;
let alreadyLocal = 0;
// A photo in several albums is handled once.
const done = new Set<number>();
// fresh() waits for a refresh from the server and throws if it fails, so
// albums and photos added since the cache was last written are included.
const { albums } = await lib.fresh();
for (const album of albums.list()) {
const savePaths: string[] = [];
for (const photo of album.photos.list()) {
if (!done.has(photo.fileID)) {
done.add(photo.fileID);
if (photo.isLocal) alreadyLocal++;
else downloaded++;
await photo.download();
// The cache paths say where quak's cache keeps copies, not
// anything about the photo.
const record = { ...photo.record() };
delete record.thumbnailPath;
delete record.originalPath;
const exif = await photo.exif();
// savePath is read after download(): a live photo's names its
// image only once the image is stored.
await writeIfChanged(
`${photo.savePath}.json`,
pretty({ ...record, exif }),
);
}
savePaths.push(relative(dir, photo.savePath));
}
await mkdir(join(dir, "albums"), { recursive: true });
await writeIfChanged(
join(dir, "albums", `${album.collectionID}.json`),
pretty({
collectionID: album.collectionID,
name: album.name,
savePaths,
}),
);
}
return { downloaded, alreadyLocal };
}
// Ask for a login code on the terminal. Client.login calls this only when the
// account requires a code.
async function ask(question: string): Promise<string> {
const terminal = createInterface({ input: stdin, output: stdout });
try {
return await terminal.question(question);
} finally {
terminal.close();
}
}
async function main(): Promise<void> {
const email = process.env.QUAK_EMAIL;
const password = process.env.QUAK_PASSWORD;
if (!email || !password) {
console.error(
"Set QUAK_EMAIL and QUAK_PASSWORD to the account's email and password.",
);
process.exit(1);
}
const dir = process.argv[2] ?? "photos";
const client = await Client.login({
email,
password,
totp: () => ask("Two-factor code: "),
emailOTP: () => ask("Code sent to your email: "),
});
const lib = await Library.open({
client,
downloadDirectory: dir,
// As in `quak backup`: fetch only the originals this script saves,
// not every thumbnail and the recent originals into the cache too.
precacheThumbnails: false,
precacheOriginals: false,
});
try {
const { downloaded, alreadyLocal } = await downloadAlbums(lib, dir);
console.log(
`${downloaded} photos downloaded, ${alreadyLocal} already local, in ${dir}`,
);
} finally {
await lib.close();
}
}
// Run main() when node runs this file, not when a test imports it. argv[1] is
// the path as given, and import.meta.url has symlinks resolved.
const script = process.argv[1];
if (script && pathToFileURL(realpathSync(script)).href === import.meta.url) {
await main();
}
+1 -1
View File
@@ -46,7 +46,7 @@
"@inquirer/prompts": "8.5.2", "@inquirer/prompts": "8.5.2",
"commander": "14.0.3", "commander": "14.0.3",
"env-paths": "4.0.0", "env-paths": "4.0.0",
"exif-reader": "2.0.3", "exifreader": "4.46.0",
"fast-srp-hap": "2.0.4", "fast-srp-hap": "2.0.4",
"fflate": "0.8.3", "fflate": "0.8.3",
"jpeg-js": "0.4.4", "jpeg-js": "0.4.4",
+23 -9
View File
@@ -1,7 +1,7 @@
#!/bin/sh #!/bin/sh
# script/build: compile the TypeScript sources into dist/, then verify that # script/build: compile the TypeScript sources into dist/, stamp the version
# the artifacts package.json advertises are among the files the compiler # script/version prints into it, then verify that the artifacts package.json
# actually wrote. tsc reports success by exit status alone and knows nothing # advertises are among the files the compiler actually wrote. tsc reports success by exit status alone and knows nothing
# about the manifest, so without this step a green build can still ship a # about the manifest, so without this step a green build can still ship a
# package whose main, types or bin resolve to nothing. Our own extension to # package whose main, types or bin resolve to nothing. Our own extension to
# scripts-to-rule-them-all. # scripts-to-rule-them-all.
@@ -46,13 +46,24 @@ for (const bin of bins) {
} }
# src/index.ts imports ../package.json for the version, which tsc copies to # src/index.ts imports ../package.json for the version, which tsc copies to
# dist/package.json. Running the built CLI proves that import resolves from # dist/package.json. The version script/version prints is written into that
# dist/ and reports the version package.json declares. # copy only; the repo's own package.json is left as it is.
stamp_version() {
node -e '
const { readFileSync, writeFileSync } = require("node:fs");
const pkg = JSON.parse(readFileSync("dist/package.json", "utf-8"));
pkg.version = process.argv[1];
writeFileSync("dist/package.json", JSON.stringify(pkg, null, 4) + "\n");
' "$1"
}
# Running the built CLI proves the import resolves from dist/ and reports
# the stamped version.
verify_version() { verify_version() {
built="$(node dist/bin/quak.js --version)" built="$(node dist/bin/quak.js --version)"
declared="$(node -p 'require("./package.json").version')" if [ "$built" != "$1" ]; then
if [ "$built" != "$declared" ]; then echo "build: dist/bin/quak.js reports $built, the build stamped $1" >&2
echo "build: dist/bin/quak.js reports $built, package.json declares $declared" >&2
exit 1 exit 1
fi fi
echo "build: dist/bin/quak.js reports version $built" echo "build: dist/bin/quak.js reports version $built"
@@ -60,9 +71,12 @@ verify_version() {
main() { main() {
cd "$ROOT" cd "$ROOT"
# Own line, so that a failing script/version stops the build.
version="$("$ROOT/script/version")"
yarn run tsc yarn run tsc
stamp_version "$version"
verify_entrypoints verify_entrypoints
verify_version verify_version "$version"
} }
main "$@" main "$@"
+3 -3
View File
@@ -15,9 +15,9 @@ main() {
cd "$ROOT" cd "$ROOT"
# Own line: a failing command substitution inside an argument does # Own line: a failing command substitution inside an argument does
# not trip `set -e`, so the inline form degrades silently to an # not trip `set -e`, so the inline form degrades silently to an
# empty constant. VERSION is computed here because .dockerignore # empty constant. The version resolved here goes in as the VERSION
# excludes .git, so `git describe` in a build stage yields an empty # build arg, which takes precedence over what the build would derive
# version without failing. # from the .git in its context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)" version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
docker build --no-cache \ docker build --no-cache \
+3 -3
View File
@@ -12,9 +12,9 @@ main() {
cd "$ROOT" cd "$ROOT"
# Own line: a failing command substitution inside an argument does # Own line: a failing command substitution inside an argument does
# not trip `set -e`, so the inline form degrades silently to an # not trip `set -e`, so the inline form degrades silently to an
# empty constant. VERSION is computed here because .dockerignore # empty constant. The version resolved here goes in as the VERSION
# excludes .git, so `git describe` in a build stage yields an empty # build arg, which takes precedence over what the build would derive
# version without failing. # from the .git in its context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)" version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
docker build --no-cache \ docker build --no-cache \
Executable
+41
View File
@@ -0,0 +1,41 @@
#!/bin/sh
# script/version: print the version script/build stamps into the built
# package. Our own extension to scripts-to-rule-them-all.
#
# Order of precedence:
#
# 1. $VERSION, if set and not empty: an explicit value, such as the
# Dockerfile's VERSION build arg.
# 2. If this checkout has .git, `git describe --tags --always`: the tag
# on a tagged commit; the tag, the commits since it and the short
# commit on a later commit (v1.2.3-4-gabc1234); the short commit when
# no tag is reachable.
# 3. Otherwise, as in a source tarball, the version package.json declares.
#
# A checkout with .git whose version still comes out empty, dev or unknown
# fails: git is missing or could not read the checkout, and the build could
# not be traced back to its commit.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
version="${VERSION:-}"
if [ -e .git ]; then
if [ -z "$version" ]; then
version="$(git describe --tags --always || true)"
fi
case "$version" in
"" | dev | unknown)
echo "version: $ROOT has .git, but the version came out '$version'" >&2
exit 1
;;
esac
elif [ -z "$version" ]; then
version="$(node -p 'require("./package.json").version')"
fi
echo "$version"
}
main "$@"
+3 -5
View File
@@ -361,8 +361,8 @@ const openPart = async (
// written unpacked: each part is named `destination` with the extension // written unpacked: each part is named `destination` with the extension
// replaced by its own entry's, and the two must differ ignoring case. When the // replaced by its own entry's, and the two must differ ignoring case. When the
// file records a hash, `<imageHash>:<videoHash>` must match it, each over that // file records a hash, `<imageHash>:<videoHash>` must match it, each over that
// part's own bytes. Only then is whatever was at `destination` removed and the // part's own bytes. Only then are the image, then the video, renamed into
// image, then the video, renamed into place; on any failure neither is stored. // place; on any failure neither is stored.
// //
// The ZIP is chosen by its uploader and may expand enormously, so each part is // The ZIP is chosen by its uploader and may expand enormously, so each part is
// written as it decompresses and never held, and the ZIP is refused once the // written as it decompresses and never held, and the ZIP is refused once the
@@ -486,7 +486,6 @@ const decryptLivePhoto = async (
await part.handle.sync(); await part.handle.sync();
await part.handle.close(); await part.handle.close();
} }
await rm(destination, { force: true });
await rename(image.tmpPath, path); await rename(image.tmpPath, path);
try { try {
await rename(video.tmpPath, videoPath); await rename(video.tmpPath, videoPath);
@@ -570,8 +569,7 @@ const fetchAndDecrypt = async (
}, api.getRetryOptions()); }, api.getRetryOptions());
// Write `file`'s original to `outPath`. A live photo is written as its image // Write `file`'s original to `outPath`. A live photo is written as its image
// and its video beside `outPath` instead, and whatever was at `outPath` is // and its video beside `outPath` instead (see `decryptLivePhoto`).
// removed (see `decryptLivePhoto`).
export const downloadFile = async ( export const downloadFile = async (
api: ApiClient, api: ApiClient,
file: EnteFile, file: EnteFile,
+117 -98
View File
@@ -1,71 +1,73 @@
// EXIF in a JPEG's bytes. `backup-metadata --exif` records the whole EXIF block // EXIF in an original's bytes, read with exifreader, which reads it from JPEG,
// it finds; `Photo.exif()` returns the common fields picked from it here. // HEIC/HEIF, AVIF, PNG, WebP and the other image formats it supports.
// `backup-metadata --exif` records every EXIF tag it finds except the
// thumbnail's. `Photo.exif()` returns every tag, the thumbnail's included, and
// `Photo`'s typed methods return the common fields picked from them here.
import exifReader from "exif-reader"; import ExifReader, { type ExpandedTags } from "exifreader";
// Find the raw EXIF APP1 segment in JPEG bytes. Returns `exif` (the segment // The EXIF tags in `bytes` (`exif`), the embedded thumbnail's tags
// data, starting at the "Exif\0\0" header) when there is one, nothing when the // (`Thumbnail`), and where the EXIF block lies in `bytes` (`metadataRange`).
// bytes are not a JPEG or carry no EXIF, and `error` when the segment layout is // Undefined when exifreader cannot read the file at all, such as a video. An
// malformed. Each segment length is checked against the bytes that remain and // EXIF block it finds but reads no tag from comes back as an empty `exif`. A
// each step moves forward by at least 4 bytes, so the scan ends on any input. // tag exifreader has no name for is keyed `undefined-<tag number>`. Each tag's
export const extractExifFromJpeg = ( // `computed` holds its value as a string or number, or as an array of them for
buf: Uint8Array, // a tag with several values, such as `GPSLatitude`'s `[40, 26, 46]`. A
): { exif?: Buffer; error?: string } => { // fraction with a zero denominator computes to null.
if (buf[0] !== 0xff || buf[1] !== 0xd8) return {}; export const readExifTags = (bytes: Uint8Array): ExpandedTags | undefined => {
let offset = 2; try {
while (offset < buf.length) { return ExifReader.loadView(
if (offset + 2 > buf.length) new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength),
return { error: `truncated segment marker at byte ${offset}` }; {
if (buf[offset] !== 0xff) expanded: true,
return { error: `no segment marker at byte ${offset}` }; computed: true,
const marker = buf[offset + 1]!; includeOffsets: true,
if (marker === 0xda) return {}; // start of scan, no more markers includeUnknown: true,
if (offset + 4 > buf.length) includeTags: { exif: true, thumbnail: true },
return { error: `truncated segment length at byte ${offset}` }; },
const len = (buf[offset + 2]! << 8) | buf[offset + 3]!; );
// The length counts its own two bytes, so anything under 2 is invalid. } catch {
if (len < 2) return undefined;
return {
error: `segment length ${len} at byte ${offset} is too small`,
};
if (offset + 2 + len > buf.length)
return {
error: `segment length ${len} at byte ${offset} runs past the end of the file`,
};
if (marker === 0xe1) {
// APP1 — check for "Exif\0\0" header. A length under 8 cannot hold
// the six-byte header, so the segment is not EXIF; below 6 the
// bytes compared would also lie past the segment.
if (
len >= 8 &&
buf[offset + 4] === 0x45 &&
buf[offset + 5] === 0x78 &&
buf[offset + 6] === 0x69 &&
buf[offset + 7] === 0x66
) {
return {
exif: Buffer.from(
buf.buffer,
buf.byteOffset + offset + 4,
len - 2,
),
};
}
}
offset += 2 + len;
} }
return { error: "file ends before the image data" };
}; };
// The common EXIF fields of an original. Each is absent when the file lacks it. // Every EXIF tag of an original, keyed by name, each as exifreader decodes it
// (see `readExifTags`). The embedded thumbnail's own tags are under
// `Thumbnail`, so its `Orientation` or `ImageWidth` cannot hide the main
// image's.
export type ExifTags = Omit<NonNullable<ExpandedTags["exif"]>, "Thumbnail"> & {
Thumbnail?: Omit<
NonNullable<ExpandedTags["Thumbnail"]>,
"type" | "image" | "base64"
>;
};
// Every EXIF tag in `bytes`: `{}` when the file has no EXIF, exifreader cannot
// read its EXIF, or it is not an image exifreader reads.
export const readAllExifTags = (bytes: Uint8Array): ExifTags => {
const tags = readExifTags(bytes);
if (!tags?.Thumbnail) return tags?.exif ?? {};
// exifreader puts the thumbnail's JPEG image beside its tags, as `type`,
// `image` and `base64`. The image is not a tag, so it is left out.
const {
type: _type,
image: _image,
base64: _base64,
...thumbnail
} = tags.Thumbnail;
return { ...tags.exif, Thumbnail: thumbnail };
};
// The common EXIF fields of an original, one for each of `Photo`'s typed
// methods. Each is absent when the file lacks it.
export interface PhotoExif { export interface PhotoExif {
make?: string; make?: string;
model?: string; model?: string;
lensModel?: string; lensModel?: string;
// When the photo was taken, by the camera's clock. EXIF writes this as text // When the photo was taken, by the camera's clock. EXIF writes this as text
// with no time zone, and exif-reader reads that text as if it were UTC: the // with no time zone, and it is read as if it were UTC: the Date's UTC
// Date's UTC fields are the clock reading, which is the moment it was taken // fields are the clock reading, which is the moment it was taken only when
// only when the clock was set to UTC. // the clock was set to UTC.
dateTimeOriginal?: Date; dateTimeOriginal?: Date;
// The camera clock's offset from UTC, such as "+02:00". // The camera clock's offset from UTC, such as "+02:00".
offsetTimeOriginal?: string; offsetTimeOriginal?: string;
@@ -84,56 +86,73 @@ export interface PhotoExif {
gpsAltitude?: number; gpsAltitude?: number;
} }
// exifreader gives "<faulty value>" for a tag whose value lies outside the
// file; that tag is left out like one the file lacks.
const asString = (v: unknown): string | undefined => const asString = (v: unknown): string | undefined =>
typeof v === "string" && v.length > 0 ? v : undefined; typeof v === "string" && v.length > 0 && v !== "<faulty value>"
? v
: undefined;
const asNumber = (v: unknown): number | undefined => const asNumber = (v: unknown): number | undefined =>
typeof v === "number" && Number.isFinite(v) ? v : undefined; typeof v === "number" && Number.isFinite(v) ? v : undefined;
const asDate = (v: unknown): Date | undefined => // EXIF writes a date and time as "2021:07:15 14:30:00". This is that reading
v instanceof Date && !Number.isNaN(v.getTime()) ? v : undefined; // in a Date's UTC fields.
const asDate = (v: unknown): Date | undefined => {
// EXIF writes a GPS coordinate as three numbers: degrees, minutes and seconds. const m =
// This is them as decimal degrees, negated when `negative`. typeof v === "string"
const asDegrees = (v: unknown, negative: boolean): number | undefined => { ? /^(\d{4}):(\d{2}):(\d{2}) (\d{2}:\d{2}:\d{2})$/.exec(v)
if (!Array.isArray(v) || v.length !== 3) return undefined; : null;
const [d, m, s] = v.map(asNumber); if (!m) return undefined;
if (d === undefined || m === undefined || s === undefined) return undefined; const date = new Date(`${m[1]}-${m[2]}-${m[3]}T${m[4]}Z`);
const degrees = d + m / 60 + s / 3600; return Number.isNaN(date.getTime()) ? undefined : date;
return negative ? -degrees : degrees;
}; };
// The common fields of a JPEG's EXIF block: `{}` when the bytes are not a JPEG, // GPSLatitude and GPSLongitude hold degrees, minutes and seconds, computed as
// have no EXIF block, or exif-reader cannot parse it. // three numbers. This is them in decimal degrees, negative when `ref`, the
export const readPhotoExif = (bytes: Uint8Array): PhotoExif => { // GPSLatitudeRef or GPSLongitudeRef tag, is `negativeRef` ("S" or "W").
const { exif } = extractExifFromJpeg(bytes); // Without that tag the hemisphere is unknown, so it is undefined.
if (exif === undefined) return {}; const asDegrees = (
let tags: ReturnType<typeof exifReader>; dms: unknown,
try { ref: unknown,
tags = exifReader(exif); negativeRef: string,
} catch { ): number | undefined => {
return {}; if (!Array.isArray(dms) || ref === undefined) return undefined;
} const [d, m, s] = dms.map(asNumber);
const image = tags.Image ?? {}; if (d === undefined || m === undefined || s === undefined) return undefined;
const photo = tags.Photo ?? {}; const degrees = d + m / 60 + s / 3600;
const gps = tags.GPSInfo ?? {}; return ref === negativeRef ? -degrees : degrees;
const altitude = asNumber(gps.GPSAltitude); };
// The common fields picked from an original's EXIF tags, `readAllExifTags`'s
// result: `{}` when there are none.
export const readPhotoExif = (tags: ExifTags): PhotoExif => {
const altitude = asNumber(tags.GPSAltitude?.computed);
const fields: PhotoExif = { const fields: PhotoExif = {
make: asString(image.Make), make: asString(tags.Make?.computed),
model: asString(image.Model), model: asString(tags.Model?.computed),
lensModel: asString(photo.LensModel), lensModel: asString(tags.LensModel?.computed),
dateTimeOriginal: asDate(photo.DateTimeOriginal), dateTimeOriginal: asDate(tags.DateTimeOriginal?.computed),
offsetTimeOriginal: asString(photo.OffsetTimeOriginal), offsetTimeOriginal: asString(tags.OffsetTimeOriginal?.computed),
exposureTime: asNumber(photo.ExposureTime), exposureTime: asNumber(tags.ExposureTime?.computed),
fNumber: asNumber(photo.FNumber), fNumber: asNumber(tags.FNumber?.computed),
iso: asNumber(photo.ISOSpeedRatings), // Only when the tag holds a single number, as most cameras write it.
focalLength: asNumber(photo.FocalLength), iso: asNumber(tags.ISOSpeedRatings?.computed),
orientation: asNumber(image.Orientation), focalLength: asNumber(tags.FocalLength?.computed),
gpsLatitude: asDegrees(gps.GPSLatitude, gps.GPSLatitudeRef === "S"), orientation: asNumber(tags.Orientation?.computed),
gpsLongitude: asDegrees(gps.GPSLongitude, gps.GPSLongitudeRef === "W"), gpsLatitude: asDegrees(
tags.GPSLatitude?.computed,
tags.GPSLatitudeRef?.computed,
"S",
),
gpsLongitude: asDegrees(
tags.GPSLongitude?.computed,
tags.GPSLongitudeRef?.computed,
"W",
),
// A GPSAltitudeRef of 1 means the altitude is below sea level. // A GPSAltitudeRef of 1 means the altitude is below sea level.
gpsAltitude: gpsAltitude:
altitude !== undefined && gps.GPSAltitudeRef === 1 altitude !== undefined && tags.GPSAltitudeRef?.value === 1
? -altitude ? -altitude
: altitude, : altitude,
}; };
+5 -3
View File
@@ -1,5 +1,7 @@
// package.json is the one place the version is written. tsc copies it to // A build reports the version script/build stamps into dist/package.json;
// dist/package.json, so this path resolves from source and from dist/src/. // package.json's own version is reported only when running from source. tsc
// copies package.json to dist/package.json, so this path resolves from source
// and from dist/src/.
import pkg from "../package.json" with { type: "json" }; import pkg from "../package.json" with { type: "json" };
export const VERSION: string = pkg.version; export const VERSION: string = pkg.version;
@@ -85,7 +87,7 @@ export type {
LibrarySnapshot, LibrarySnapshot,
LibraryChange, LibraryChange,
} from "./library/records.js"; } from "./library/records.js";
export type { PhotoExif } from "./exif.js"; export type { ExifTags, PhotoExif } from "./exif.js";
export { decryptCollection, decryptFile } from "./model/index.js"; export { decryptCollection, decryptFile } from "./model/index.js";
export { downloadFile, downloadThumbnail } from "./download/index.js"; export { downloadFile, downloadThumbnail } from "./download/index.js";
export type { export type {
+19 -48
View File
@@ -29,14 +29,7 @@
// the cache does not count as saved there, but is copied there rather than // the cache does not count as saved there, but is copied there rather than
// fetched again. // fetched again.
import { import { existsSync, readFileSync, statSync } from "node:fs";
closeSync,
existsSync,
openSync,
readFileSync,
readSync,
statSync,
} from "node:fs";
import { import {
chmod, chmod,
copyFile, copyFile,
@@ -117,8 +110,9 @@ export interface ContentOptions {
export interface PhotoContent { export interface PhotoContent {
original(fileID: number, opts?: ContentOptions): Promise<ContentResult>; original(fileID: number, opts?: ContentOptions): Promise<ContentResult>;
thumbnail(fileID: number, opts?: ContentOptions): Promise<ContentResult>; thumbnail(fileID: number, opts?: ContentOptions): Promise<ContentResult>;
// Put the original at its save path and return it there. // Put the original at the save path of `file`, the copy the `Photo` holds,
download(fileID: number): Promise<ContentResult>; // and return it there.
download(file: EnteFile): Promise<ContentResult>;
} }
export interface EnsureResult { export interface EnsureResult {
@@ -280,25 +274,6 @@ const fileSize = (path: string): number | undefined => {
const hasContent = (path: string | undefined): boolean => const hasContent = (path: string | undefined): boolean =>
path !== undefined && (fileSize(path) ?? 0) > 0; path !== undefined && (fileSize(path) ?? 0) > 0;
// Whether the file at `path` begins as a ZIP does, with `PK\x03\x04`. False
// when it cannot be read.
const isZip = (path: string): boolean => {
try {
const fd = openSync(path, "r");
try {
const head = Buffer.alloc(4);
return (
readSync(fd, head, 0, 4, 0) === 4 &&
head.toString("latin1") === "PK\x03\x04"
);
} finally {
closeSync(fd);
}
} catch {
return false;
}
};
// A live photo's image and video are named with the extensions from inside its // A live photo's image and video are named with the extensions from inside its
// ZIP, so their names alone do not say which is which. Wherever the cache or a // ZIP, so their names alone do not say which is which. Wherever the cache or a
// save path stores one, a JSON file of this name beside them names both. // save path stores one, a JSON file of this name beside them names both.
@@ -531,18 +506,18 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
return this.get(fileID, "thumbnail", "on-demand", opts?.onProgress); return this.get(fileID, "thumbnail", "on-demand", opts?.onProgress);
} }
// Put the original at its save path under the download directory and // Put the original at the save path of `file` under the download directory
// return it there. One already stored there is returned as it is; one the // and return it there. `file` is the copy the `Photo` holds, so the path is
// cache holds is copied from it; any other is fetched straight to the // the one its `savePath` names, even after a refresh changed the date. One
// save path, with no copy left in the cache. // already stored there is returned as it is; one the cache holds is copied
async download(fileID: number): Promise<ContentResult> { // from it; any other is fetched straight to the save path, with no copy
const file = this.getFile(fileID); // left in the cache.
if (!file) throw new Error(`content cache: unknown file ${fileID}`); async download(file: EnteFile): Promise<ContentResult> {
const root = this.downloadDirectory; const root = this.downloadDirectory;
const saved = const saved =
storedAtSavePath(root, file) ?? storedAtSavePath(root, file) ??
(await placeOriginal(root, file, (dest) => (await placeOriginal(root, file, (dest) =>
this.backupOriginal(fileID, dest), this.backupOriginal(file.id, dest),
)); ));
return { ...saved, bytes: fileSize(saved.path) ?? 0 }; return { ...saved, bytes: fileSize(saved.path) ?? 0 };
} }
@@ -712,8 +687,10 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
await this.touch(cached.path); await this.touch(cached.path);
return { ...cached, bytes: size, cached: true }; return { ...cached, bytes: size, cached: true };
} }
// A recorded file that has since gone, or a live photo an earlier // A recorded file that has since gone re-fetches below. So does a
// version stored as one ZIP, re-fetches below. // live photo recorded with no video: the cache opened before the
// library's records said it is a live photo, while its image and
// video had no JSON file beside them yet.
known.delete(fileID); known.delete(fileID);
} }
@@ -982,20 +959,14 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
if (id === undefined || !existsSync(path)) continue; if (id === undefined || !existsSync(path)) continue;
// A live photo's image and video are one entry, as the JSON file // A live photo's image and video are one entry, as the JSON file
// beside them names them. A live photo's file with no such JSON // beside them names them. A live photo's file with no such JSON
// file is not its original. If it is a ZIP, it is the one an // file is not its original and is left alone: another process may
// earlier version stored under the image's name, and is removed. // have just stored it and not yet written the JSON file.
// Any other is left alone: another process may have just stored
// it and not yet written the JSON file.
const livePhoto = names.has(livePhotoJSONName(String(id))) const livePhoto = names.has(livePhotoJSONName(String(id)))
? readLivePhotoJSON(dir, String(id)) ? readLivePhotoJSON(dir, String(id))
: undefined; : undefined;
if (livePhoto !== undefined) { if (livePhoto !== undefined) {
into.set(id, livePhoto); into.set(id, livePhoto);
} else if (isLivePhoto(id)) { } else if (!isLivePhoto(id)) {
if (isZip(path)) {
await rm(path, { force: true }).catch(() => undefined);
}
} else {
into.set(id, { path }); into.set(id, { path });
} }
} }
+87 -14
View File
@@ -12,15 +12,20 @@
// plain records are the serializable surface, and `record()` returns one. // plain records are the serializable surface, and `record()` returns one.
// //
// A `Photo` also fetches its own bytes: `original()`, `thumbnail()`, // A `Photo` also fetches its own bytes: `original()`, `thumbnail()`,
// `download()`, `content()` and `exif()` go through the on-disk content cache // `download()`, `content()`, `exif()` and the methods that each return one
// (issue #46), and are the one place in this module that may touch the // EXIF field go through the on-disk content cache (issue #46), and are
// network. A library opened without a content source leaves that cache absent, // the one place in this module that may touch the network. A library opened
// and those methods then throw. `savePath` and `isLocal` look only at the disk // without a content source leaves that cache absent, and those methods then
// and need no cache. // throw. `savePath` and `isLocal` look only at the disk and need no cache.
import { readFile } from "node:fs/promises"; import { readFile } from "node:fs/promises";
import { readPhotoExif, type PhotoExif } from "../exif.js"; import {
readAllExifTags,
readPhotoExif,
type ExifTags,
type PhotoExif,
} from "../exif.js";
import type { CollectionType, EnteFile, FileType } from "../model/types.js"; import type { CollectionType, EnteFile, FileType } from "../model/types.js";
import type { ContentOptions, ContentResult, PhotoContent } from "./content.js"; import type { ContentOptions, ContentResult, PhotoContent } from "./content.js";
import type { AlbumRecord, PhotoRecord, DerivedRecords } from "./records.js"; import type { AlbumRecord, PhotoRecord, DerivedRecords } from "./records.js";
@@ -42,11 +47,18 @@ const byNewest = (a: PhotoRecord, b: PhotoRecord): number =>
const byNewestAlbum = (a: AlbumRecord, b: AlbumRecord): number => const byNewestAlbum = (a: AlbumRecord, b: AlbumRecord): number =>
b.updationTime - a.updationTime || b.collectionID - a.collectionID; b.updationTime - a.updationTime || b.collectionID - a.collectionID;
// A method for each `PhotoExif` field, named after it, taking the options
// `exif()` takes and giving that field. `Photo` implements it, so the build's
// type check fails when `PhotoExif` has a field `Photo` has no method for.
type PhotoExifMethods = {
[K in keyof PhotoExif]-?: (opts?: ContentOptions) => Promise<PhotoExif[K]>;
};
// A single photo. Field access mirrors `PhotoRecord`; `record()` returns the // A single photo. Field access mirrors `PhotoRecord`; `record()` returns the
// underlying plain record for callers that need the IPC-safe value. `file` is // underlying plain record for callers that need the IPC-safe value. `file` is
// the membership the record is read from, so the save path carries the date of // the membership the record is read from, so the save path carries the date of
// `takenAt` and stays known after a refresh removes the file from the library. // `takenAt` and stays known after a refresh removes the file from the library.
export class Photo { export class Photo implements PhotoExifMethods {
constructor( constructor(
private readonly rec: PhotoRecord, private readonly rec: PhotoRecord,
private readonly file: EnteFile, private readonly file: EnteFile,
@@ -134,7 +146,7 @@ export class Photo {
// it, it is copied from there; otherwise it is fetched straight to // it, it is copied from there; otherwise it is fetched straight to
// `savePath`. // `savePath`.
async download(): Promise<ContentResult> { async download(): Promise<ContentResult> {
return this.cacheOrThrow().download(this.rec.fileID); return this.cacheOrThrow().download(this.file);
} }
// As `original`, for the thumbnail, through the thumbnail pool. // As `original`, for the thumbnail, through the thumbnail pool.
@@ -149,14 +161,75 @@ export class Photo {
return readFile(path); return readFile(path);
} }
// The common EXIF fields of the original, read from `content()`, so this // Every EXIF tag of the original, keyed by name (see `ExifTags`), read
// may download it. Only a JPEG's EXIF is read; any other file gives `{}`, // from `content()`, so this may download it. EXIF is read from any image
// and a video gives it without fetching anything. Like the other content // format exifreader reads, JPEG and HEIC/HEIF among them; any other file
// methods, it throws when there is no content cache, video or not. // gives `{}`, and a video gives it without fetching anything. Like the
async exif(opts?: ContentOptions): Promise<PhotoExif> { // other content methods, it throws when there is no content cache, video
// or not.
async exif(opts?: ContentOptions): Promise<ExifTags> {
this.cacheOrThrow(); this.cacheOrThrow();
if (this.rec.fileType === "video") return {}; if (this.rec.fileType === "video") return {};
return readPhotoExif(await this.content(opts)); return readAllExifTags(await this.content(opts));
}
// One field each, named and typed as in `PhotoExif`, picked from the tags
// `exif()` returns, and undefined when the file lacks it. Each call runs
// `exif()`, which reads the original again.
async make(opts?: ContentOptions): Promise<PhotoExif["make"]> {
return readPhotoExif(await this.exif(opts)).make;
}
async model(opts?: ContentOptions): Promise<PhotoExif["model"]> {
return readPhotoExif(await this.exif(opts)).model;
}
async lensModel(opts?: ContentOptions): Promise<PhotoExif["lensModel"]> {
return readPhotoExif(await this.exif(opts)).lensModel;
}
async dateTimeOriginal(
opts?: ContentOptions,
): Promise<PhotoExif["dateTimeOriginal"]> {
return readPhotoExif(await this.exif(opts)).dateTimeOriginal;
}
async offsetTimeOriginal(
opts?: ContentOptions,
): Promise<PhotoExif["offsetTimeOriginal"]> {
return readPhotoExif(await this.exif(opts)).offsetTimeOriginal;
}
async exposureTime(
opts?: ContentOptions,
): Promise<PhotoExif["exposureTime"]> {
return readPhotoExif(await this.exif(opts)).exposureTime;
}
async fNumber(opts?: ContentOptions): Promise<PhotoExif["fNumber"]> {
return readPhotoExif(await this.exif(opts)).fNumber;
}
async iso(opts?: ContentOptions): Promise<PhotoExif["iso"]> {
return readPhotoExif(await this.exif(opts)).iso;
}
async focalLength(
opts?: ContentOptions,
): Promise<PhotoExif["focalLength"]> {
return readPhotoExif(await this.exif(opts)).focalLength;
}
async orientation(
opts?: ContentOptions,
): Promise<PhotoExif["orientation"]> {
return readPhotoExif(await this.exif(opts)).orientation;
}
async gpsLatitude(
opts?: ContentOptions,
): Promise<PhotoExif["gpsLatitude"]> {
return readPhotoExif(await this.exif(opts)).gpsLatitude;
}
async gpsLongitude(
opts?: ContentOptions,
): Promise<PhotoExif["gpsLongitude"]> {
return readPhotoExif(await this.exif(opts)).gpsLongitude;
}
async gpsAltitude(
opts?: ContentOptions,
): Promise<PhotoExif["gpsAltitude"]> {
return readPhotoExif(await this.exif(opts)).gpsAltitude;
} }
private cacheOrThrow(): PhotoContent { private cacheOrThrow(): PhotoContent {
+16 -14
View File
@@ -1,9 +1,8 @@
import { mkdirSync, readFileSync, writeFileSync } from "node:fs"; import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
import { join } from "node:path"; import { join } from "node:path";
import * as jpeg from "jpeg-js"; import * as jpeg from "jpeg-js";
import exifReader from "exif-reader";
import type { Client } from "./client.js"; import type { Client } from "./client.js";
import { extractExifFromJpeg } from "./exif.js"; import { readExifTags } from "./exif.js";
import type { Library, Photo } from "./library/index.js"; import type { Library, Photo } from "./library/index.js";
import { sanitizeFileName } from "./filename.js"; import { sanitizeFileName } from "./filename.js";
import { import {
@@ -20,9 +19,10 @@ export interface MetadataBackupOptions {
onProgress?: ProgressCallback; onProgress?: ProgressCallback;
} }
// Extract dimensions, EXIF and XMP from a file's bytes. When the EXIF segment // Extract dimensions, EXIF and XMP from a file's bytes. `exif` is the EXIF tags
// is malformed or cannot be parsed, the record carries the reason in // exifreader returns, from any image format it reads. When it finds an EXIF
// `exifError`. // block but reads no tag from it, the record keeps the block's bytes, base64,
// in `exifRaw`, with the reason in `exifError`.
export const extractImageMetadata = ( export const extractImageMetadata = (
fileBytes: Uint8Array, fileBytes: Uint8Array,
): Record<string, unknown> | undefined => { ): Record<string, unknown> | undefined => {
@@ -39,19 +39,21 @@ export const extractImageMetadata = (
result.height = decoded.height; result.height = decoded.height;
} catch { } catch {
// Not every original is a JPEG (PNG, HEIC, video), so a failed decode // Not every original is a JPEG (PNG, HEIC, video), so a failed decode
// is expected and only means no dimensions; a malformed JPEG is still // is expected and only means no dimensions; unreadable EXIF is still
// reported below through `exifError`. // reported below through `exifError`.
} }
const { exif, error } = extractExifFromJpeg(fileBytes); const tags = readExifTags(fileBytes);
if (error) result.exifError = error; if (tags?.exif && Object.keys(tags.exif).length > 0) {
if (exif) { result.exif = tags.exif;
try { } else if (tags?.exif) {
result.exif = exifReader(exif); const block = tags.metadataRange?.blocks.find((b) => b.type === "exif");
} catch (err) { if (block) {
result.exifRaw = exif.toString("base64"); result.exifRaw = Buffer.from(
result.exifError = err instanceof Error ? err.message : String(err); fileBytes.subarray(block.start, block.end),
).toString("base64");
} }
result.exifError = "no tag could be read from the EXIF block";
} }
// Extract XMP (look for "http://ns.adobe.com/xap" in the bytes) // Extract XMP (look for "http://ns.adobe.com/xap" in the bytes)
+26
View File
@@ -56,6 +56,7 @@ import type { ContentSource } from "../../src/library/content.js";
import type { Collection, EnteFile } from "../../src/model/types.js"; import type { Collection, EnteFile } from "../../src/model/types.js";
import { init, toBase64 } from "../../src/crypto/index.js"; import { init, toBase64 } from "../../src/crypto/index.js";
import { defaultCacheDirectory } from "../../src/library/index.js"; import { defaultCacheDirectory } from "../../src/library/index.js";
import { HEIC_WITH_EXIF } from "../exif-heic.js";
import { import {
asLivePhoto, asLivePhoto,
cdnSource, cdnSource,
@@ -653,6 +654,31 @@ describe("a live photo", () => {
height: 4, height: 4,
}); });
}); });
it("backup-metadata --exif records the EXIF of a HEIC image", async () => {
const dir = join(root, "dump");
expect(
await backupMetadataCommand(
context(await livePhotoClient(HEIC_WITH_EXIF)),
dir,
{ exif: true },
),
).toBe(0);
const record = JSON.parse(
readFileSync(
join(dir, "collections", "1-Vacation", "300.json"),
"utf-8",
),
);
expect(record.imageMetadata.exifError).toBeUndefined();
expect(record.imageMetadata.exif).toMatchObject({
Make: { value: ["Canon"] },
Model: { value: ["EOS R5"] },
DateTimeOriginal: { value: ["2021:07:15 14:30:00"] },
});
});
}); });
describe("backup", () => { describe("backup", () => {
+301 -67
View File
@@ -1,19 +1,28 @@
/** /**
* Tests for the JPEG EXIF scan behind `quak backup-metadata --exif`. * Tests for reading EXIF (`src/exif.ts`) and the image metadata
* `quak backup-metadata --exif` records.
* *
* The originals come from users' libraries, so a truncated or corrupt JPEG * The originals come from users' libraries, so a truncated or corrupt file
* must neither hang the scan nor throw out of it, and a malformed file must be * must neither hang the read nor throw out of it: `readAllExifTags` gives `{}`,
* told apart from one that simply has no EXIF: the record carries the reason in * and `backup-metadata` tells an EXIF block it cannot read apart from a file
* `exifError`. Each input below is a short hand-built byte array. * that simply has no EXIF, carrying the reason in `exifError`. Each JPEG below
* is a short hand-built byte array; the HEIC is a real file.
*/ */
import { describe, expect, it } from "vitest"; import { describe, expect, it } from "vitest";
import { extractExifFromJpeg } from "../../src/exif.js"; import {
readAllExifTags,
readExifTags,
readPhotoExif,
} from "../../src/exif.js";
import { extractImageMetadata } from "../../src/metadata-backup.js"; import { extractImageMetadata } from "../../src/metadata-backup.js";
import { HEIC_WITH_EXIF } from "../exif-heic.js";
const SOI = [0xff, 0xd8]; // start of image const SOI = [0xff, 0xd8]; // start of image
const SOS = [0xff, 0xda, 0x00, 0x02]; // start of scan, where the scan stops const SOS = [0xff, 0xda, 0x00, 0x02]; // start of scan
const EXIF_HEADER = [0x45, 0x78, 0x69, 0x66, 0x00, 0x00]; // "Exif\0\0" const EXIF_HEADER = [0x45, 0x78, 0x69, 0x66, 0x00, 0x00]; // "Exif\0\0"
const APP0 = [0xff, 0xe0, 0x00, 0x04, 0x00, 0x00];
const ZERO_LENGTH_APP0 = [0xff, 0xe0, 0x00, 0x00];
// A big-endian TIFF block with one IFD entry: Orientation (0x0112), SHORT, 6. // A big-endian TIFF block with one IFD entry: Orientation (0x0112), SHORT, 6.
const TIFF_ORIENTATION_6 = [ const TIFF_ORIENTATION_6 = [
@@ -22,6 +31,132 @@ const TIFF_ORIENTATION_6 = [
0x00, 0x00, 0x00, 0x00,
]; ];
// A big-endian TIFF block holding Orientation 6 and DateTimeOriginal
// "0000:00:00 00:00:00", which a camera with an unset clock writes.
const TIFF_UNSET_DATE = [
...[0x4d, 0x4d, 0x00, 0x2a, 0x00, 0x00, 0x00, 0x08], // the first IFD at 8
// The first IFD, at 8: two entries, then no next IFD.
...[0x00, 0x02],
// Orientation (0x0112), SHORT, 6.
...[0x01, 0x12, 0x00, 0x03, 0x00, 0x00, 0x00, 0x01, 0x00, 0x06, 0x00, 0x00],
// The Exif IFD's offset (0x8769), LONG, 38.
...[0x87, 0x69, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x26],
...[0x00, 0x00, 0x00, 0x00],
// The Exif IFD, at 38: one entry, then no next IFD.
...[0x00, 0x01],
// DateTimeOriginal (0x9003), 20 ASCII bytes at 56.
...[0x90, 0x03, 0x00, 0x02, 0x00, 0x00, 0x00, 0x14, 0x00, 0x00, 0x00, 0x38],
...[0x00, 0x00, 0x00, 0x00],
...new TextEncoder().encode("0000:00:00 00:00:00\0"), // at 56
];
// A big-endian TIFF block holding a GPSAltitude of 12.5 m and no
// GPSAltitudeRef.
const TIFF_ALTITUDE_WITHOUT_REF = [
...[0x4d, 0x4d, 0x00, 0x2a, 0x00, 0x00, 0x00, 0x08], // the first IFD at 8
// The first IFD, at 8: one entry, then no next IFD.
...[0x00, 0x01],
// The GPS IFD's offset (0x8825), LONG, 26.
...[0x88, 0x25, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x1a],
...[0x00, 0x00, 0x00, 0x00],
// The GPS IFD, at 26: one entry, then no next IFD.
...[0x00, 0x01],
// GPSAltitude (0x0006), one RATIONAL at 44.
...[0x00, 0x06, 0x00, 0x05, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x2c],
...[0x00, 0x00, 0x00, 0x00],
...[0x00, 0x00, 0x00, 0x19, 0x00, 0x00, 0x00, 0x02], // 25/2, at 44
];
// A big-endian TIFF block holding a GPSLatitude of 33° 30' 0" and a
// GPSLatitudeRef of "S".
const TIFF_SOUTHERN_LATITUDE = [
...[0x4d, 0x4d, 0x00, 0x2a, 0x00, 0x00, 0x00, 0x08], // the first IFD at 8
// The first IFD, at 8: one entry, then no next IFD.
...[0x00, 0x01],
// The GPS IFD's offset (0x8825), LONG, 26.
...[0x88, 0x25, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x1a],
...[0x00, 0x00, 0x00, 0x00],
// The GPS IFD, at 26: two entries, then no next IFD.
...[0x00, 0x02],
// GPSLatitudeRef (0x0001), 2 ASCII bytes: "S".
...[0x00, 0x01, 0x00, 0x02, 0x00, 0x00, 0x00, 0x02, 0x53, 0x00, 0x00, 0x00],
// GPSLatitude (0x0002), three RATIONALs at 56.
...[0x00, 0x02, 0x00, 0x05, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x38],
...[0x00, 0x00, 0x00, 0x00],
...[0x00, 0x00, 0x00, 0x21, 0x00, 0x00, 0x00, 0x01], // 33/1, at 56
...[0x00, 0x00, 0x00, 0x1e, 0x00, 0x00, 0x00, 0x01], // 30/1
...[0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01], // 0/1
];
// A big-endian TIFF block holding a GPSLatitude of 40° 26' 46" and a
// GPSLongitude of 79° 58' 56", and neither GPSLatitudeRef nor GPSLongitudeRef.
const TIFF_POSITION_WITHOUT_REFS = [
...[0x4d, 0x4d, 0x00, 0x2a, 0x00, 0x00, 0x00, 0x08], // the first IFD at 8
// The first IFD, at 8: one entry, then no next IFD.
...[0x00, 0x01],
// The GPS IFD's offset (0x8825), LONG, 26.
...[0x88, 0x25, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x1a],
...[0x00, 0x00, 0x00, 0x00],
// The GPS IFD, at 26: two entries, then no next IFD.
...[0x00, 0x02],
// GPSLatitude (0x0002), three RATIONALs at 56.
...[0x00, 0x02, 0x00, 0x05, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x38],
// GPSLongitude (0x0004), three RATIONALs at 80.
...[0x00, 0x04, 0x00, 0x05, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x50],
...[0x00, 0x00, 0x00, 0x00],
...[0x00, 0x00, 0x00, 0x28, 0x00, 0x00, 0x00, 0x01], // 40/1, at 56
...[0x00, 0x00, 0x00, 0x1a, 0x00, 0x00, 0x00, 0x01], // 26/1
...[0x00, 0x00, 0x00, 0x2e, 0x00, 0x00, 0x00, 0x01], // 46/1
...[0x00, 0x00, 0x00, 0x4f, 0x00, 0x00, 0x00, 0x01], // 79/1, at 80
...[0x00, 0x00, 0x00, 0x3a, 0x00, 0x00, 0x00, 0x01], // 58/1
...[0x00, 0x00, 0x00, 0x38, 0x00, 0x00, 0x00, 0x01], // 56/1
];
// A big-endian TIFF block holding Orientation 6 and a Make whose value lies
// past the end of the file.
const TIFF_MAKE_PAST_END = [
...[0x4d, 0x4d, 0x00, 0x2a, 0x00, 0x00, 0x00, 0x08], // the first IFD at 8
// The first IFD, at 8: two entries, then no next IFD.
...[0x00, 0x02],
// Make (0x010f), 6 ASCII bytes at 4096.
...[0x01, 0x0f, 0x00, 0x02, 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, 0x10, 0x00],
// Orientation (0x0112), SHORT, 6.
...[0x01, 0x12, 0x00, 0x03, 0x00, 0x00, 0x00, 0x01, 0x00, 0x06, 0x00, 0x00],
...[0x00, 0x00, 0x00, 0x00],
];
// A big-endian TIFF block with one IFD entry that exifreader has no name for:
// tag 0xc000 (49152), SHORT, 7.
const TIFF_UNNAMED_TAG = [
...[0x4d, 0x4d, 0x00, 0x2a, 0x00, 0x00, 0x00, 0x08], // the first IFD at 8
// The first IFD, at 8: one entry, then no next IFD.
...[0x00, 0x01],
// Tag 0xc000, SHORT, 7.
...[0xc0, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x01, 0x00, 0x07, 0x00, 0x00],
...[0x00, 0x00, 0x00, 0x00],
];
// A big-endian TIFF block holding Orientation 6, and a thumbnail IFD holding
// its own Orientation 1 and a 4-byte JPEG thumbnail.
const TIFF_WITH_THUMBNAIL = [
...[0x4d, 0x4d, 0x00, 0x2a, 0x00, 0x00, 0x00, 0x08], // the first IFD at 8
// The first IFD, at 8: one entry, then the thumbnail IFD at 26.
...[0x00, 0x01],
// Orientation (0x0112), SHORT, 6.
...[0x01, 0x12, 0x00, 0x03, 0x00, 0x00, 0x00, 0x01, 0x00, 0x06, 0x00, 0x00],
...[0x00, 0x00, 0x00, 0x1a],
// The thumbnail IFD, at 26: three entries, then no next IFD.
...[0x00, 0x03],
// Orientation (0x0112), SHORT, 1.
...[0x01, 0x12, 0x00, 0x03, 0x00, 0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x00],
// JPEGInterchangeFormat (0x0201), LONG: the thumbnail is at 68.
...[0x02, 0x01, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x44],
// JPEGInterchangeFormatLength (0x0202), LONG: 4 bytes.
...[0x02, 0x02, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x04],
...[0x00, 0x00, 0x00, 0x00],
...[0xff, 0xd8, 0xff, 0xd9], // the thumbnail, at 68: an empty JPEG
];
// An APP1 segment whose length field matches its data. // An APP1 segment whose length field matches its data.
const app1 = (data: number[]): number[] => { const app1 = (data: number[]): number[] => {
const len = data.length + 2; const len = data.length + 2;
@@ -31,74 +166,147 @@ const app1 = (data: number[]): number[] => {
const bytes = (...parts: number[][]): Uint8Array => const bytes = (...parts: number[][]): Uint8Array =>
new Uint8Array(parts.flat()); new Uint8Array(parts.flat());
describe("extractExifFromJpeg", () => { describe("readAllExifTags", () => {
it("returns the EXIF segment of a valid JPEG", () => { it("keys a tag exifreader has no name for by its number", () => {
const data = [...EXIF_HEADER, ...TIFF_UNNAMED_TAG];
expect(readAllExifTags(bytes(SOI, app1(data), SOS))).toStrictEqual({
"undefined-49152": {
id: 49152,
value: 7,
description: 7,
computed: 7,
},
});
});
it("puts the thumbnail's tags under Thumbnail, without its image", () => {
const input = bytes(
SOI,
app1([...EXIF_HEADER, ...TIFF_WITH_THUMBNAIL]),
SOS,
);
// exifreader finds the thumbnail's image.
expect(readExifTags(input)?.Thumbnail?.type).toBe("image/jpeg");
const tags = readAllExifTags(input);
expect(tags.Orientation?.value).toBe(6);
expect(Object.keys(tags.Thumbnail ?? {}).sort()).toEqual([
"JPEGInterchangeFormat",
"JPEGInterchangeFormatLength",
"Orientation",
]);
expect(tags.Thumbnail?.Orientation?.value).toBe(1);
expect(readPhotoExif(tags)).toStrictEqual({ orientation: 6 });
});
});
describe("readPhotoExif", () => {
it("reads the common fields of a valid JPEG", () => {
const data = [...EXIF_HEADER, ...TIFF_ORIENTATION_6]; const data = [...EXIF_HEADER, ...TIFF_ORIENTATION_6];
const scan = extractExifFromJpeg(bytes(SOI, app1(data), SOS)); expect(
expect(scan.error).toBeUndefined(); readPhotoExif(readAllExifTags(bytes(SOI, app1(data), SOS))),
expect([...scan.exif!]).toEqual(data); ).toStrictEqual({
orientation: 6,
});
}); });
it("returns nothing for a file that is not a JPEG", () => { it("gives no dateTimeOriginal for a DateTimeOriginal of 0000:00:00 00:00:00", () => {
const png = bytes([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]); const data = [...EXIF_HEADER, ...TIFF_UNSET_DATE];
expect(extractExifFromJpeg(png)).toEqual({}); expect(
readPhotoExif(readAllExifTags(bytes(SOI, app1(data), SOS))),
).toStrictEqual({
orientation: 6,
});
}); });
it("returns nothing for a JPEG without EXIF", () => { it("reads a GPSAltitude without GPSAltitudeRef as above sea level", () => {
const app0 = [0xff, 0xe0, 0x00, 0x04, 0x00, 0x00]; const data = [...EXIF_HEADER, ...TIFF_ALTITUDE_WITHOUT_REF];
expect(extractExifFromJpeg(bytes(SOI, app0, SOS))).toEqual({}); expect(
readPhotoExif(readAllExifTags(bytes(SOI, app1(data), SOS))),
).toStrictEqual({
gpsAltitude: 12.5,
});
}); });
it("ignores an APP1 segment too short to hold the Exif header", () => { it("reads a GPSLatitude with GPSLatitudeRef S as south of the equator", () => {
// A length under 8 cannot hold the six-byte "Exif\0\0" header, so the const data = [...EXIF_HEADER, ...TIFF_SOUTHERN_LATITUDE];
// segment is not EXIF. This one has length 7 and holds only "Exif\0", expect(
// which the old code, lacking the length check, returned as EXIF. readPhotoExif(readAllExifTags(bytes(SOI, app1(data), SOS))),
const short = app1(EXIF_HEADER.slice(0, 5)); ).toStrictEqual({
expect(extractExifFromJpeg(bytes(SOI, short, SOS))).toEqual({}); gpsLatitude: -33.5,
});
}); });
it("accepts an APP1 segment of length 8 holding just the Exif header", () => { it("gives no gpsLatitude or gpsLongitude without their reference tags", () => {
const scan = extractExifFromJpeg(bytes(SOI, app1(EXIF_HEADER), SOS)); const data = [...EXIF_HEADER, ...TIFF_POSITION_WITHOUT_REFS];
expect(scan.error).toBeUndefined(); const tags = readAllExifTags(bytes(SOI, app1(data), SOS));
expect([...scan.exif!]).toEqual(EXIF_HEADER); // The position is read; only its hemisphere is unknown.
expect(tags.GPSLatitude?.computed).toStrictEqual([40, 26, 46]);
expect(tags.GPSLongitude?.computed).toStrictEqual([79, 58, 56]);
expect(readPhotoExif(tags)).toStrictEqual({});
}); });
it("reports a JPEG truncated inside a segment header", () => { it("gives no make for a Make whose value lies past the end of the file", () => {
const scan = extractExifFromJpeg(bytes(SOI, [0xff, 0xe1, 0x00])); const data = [...EXIF_HEADER, ...TIFF_MAKE_PAST_END];
expect(scan.exif).toBeUndefined(); expect(
expect(scan.error).toMatch(/truncated segment length/); readPhotoExif(readAllExifTags(bytes(SOI, app1(data), SOS))),
).toStrictEqual({
orientation: 6,
});
}); });
it("reports a JPEG that ends before the image data", () => { it.each([
const app0 = [0xff, 0xe0, 0x00, 0x04, 0x00, 0x00]; [
const scan = extractExifFromJpeg(bytes(SOI, app0)); "a file that is not an image",
expect(scan.error).toMatch(/ends before the image data/); new TextEncoder().encode("just some text, not an image"),
}); ],
[
it("stops on a zero-length segment instead of looping", () => { "a PNG without EXIF",
// A length of 0 would otherwise step the scan by 2 bytes at a time bytes([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
// through the rest of the file, reading garbage as markers. ],
const zero = [0xff, 0xe0, 0x00, 0x00]; ["a JPEG without EXIF", bytes(SOI, APP0, SOS)],
const scan = extractExifFromJpeg( // A length under 8 cannot hold the six-byte "Exif\0\0" header. This one
bytes(SOI, zero, zero, zero, zero, SOS), // has length 7 and holds only "Exif\0", so a read past its end would
); // take the next segment's bytes as EXIF.
expect(scan.error).toMatch(/segment length 0 at byte 2 is too small/); [
}); "an APP1 segment too short to hold the Exif header",
bytes(SOI, app1(EXIF_HEADER.slice(0, 5)), SOS),
it("stops on a segment length of 1", () => { ],
const scan = extractExifFromJpeg( [
bytes(SOI, [0xff, 0xe0, 0x00, 0x01], SOS), "an APP1 segment holding just the Exif header",
); bytes(SOI, app1(EXIF_HEADER), SOS),
expect(scan.error).toMatch(/segment length 1 at byte 2 is too small/); ],
}); [
"a JPEG truncated inside a segment header",
it("reports a segment length that runs past the end of the file", () => { bytes(SOI, [0xff, 0xe1, 0x00]),
],
["a JPEG that ends before the image data", bytes(SOI, APP0)],
// A length of 0 would step a scan by 2 bytes at a time through the
// rest of the file, reading garbage as markers.
[
"a zero-length segment",
bytes(
SOI,
ZERO_LENGTH_APP0,
ZERO_LENGTH_APP0,
ZERO_LENGTH_APP0,
ZERO_LENGTH_APP0,
SOS,
),
],
["a segment length of 1", bytes(SOI, [0xff, 0xe0, 0x00, 0x01], SOS)],
// APP1 claims 0x4000 bytes but only the "Exif\0\0" header follows. // APP1 claims 0x4000 bytes but only the "Exif\0\0" header follows.
const scan = extractExifFromJpeg( [
"a segment length that runs past the end of the file",
bytes(SOI, [0xff, 0xe1, 0x40, 0x00], EXIF_HEADER), bytes(SOI, [0xff, 0xe1, 0x40, 0x00], EXIF_HEADER),
); ],
expect(scan.exif).toBeUndefined(); // "XX" where the TIFF byte order belongs.
expect(scan.error).toMatch(/runs past the end of the file/); [
"an EXIF block that cannot be parsed",
bytes(SOI, app1([...EXIF_HEADER, 0x58, 0x58]), SOS),
],
])("returns no tags and no fields for %s", (_, input) => {
expect(readAllExifTags(input)).toStrictEqual({});
expect(readPhotoExif(readAllExifTags(input))).toStrictEqual({});
}); });
}); });
@@ -108,10 +316,30 @@ describe("extractImageMetadata", () => {
bytes(SOI, app1([...EXIF_HEADER, ...TIFF_ORIENTATION_6]), SOS), bytes(SOI, app1([...EXIF_HEADER, ...TIFF_ORIENTATION_6]), SOS),
); );
expect(meta?.exifError).toBeUndefined(); expect(meta?.exifError).toBeUndefined();
expect(meta?.exif).toMatchObject({ Image: { Orientation: 6 } }); expect(meta?.exif).toMatchObject({ Orientation: { value: 6 } });
}); });
it("returns nothing for a file that is not a JPEG", () => { it("parses EXIF from a HEIC", () => {
const meta = extractImageMetadata(HEIC_WITH_EXIF);
expect(meta?.exifError).toBeUndefined();
expect(meta?.exif).toMatchObject({
Make: { value: ["Canon"] },
Model: { value: ["EOS R5"] },
DateTimeOriginal: { value: ["2021:07:15 14:30:00"] },
Orientation: { value: 6 },
GPSLatitudeRef: { value: ["N"] },
});
});
it("keys a tag exifreader has no name for by its number", () => {
const meta = extractImageMetadata(
bytes(SOI, app1([...EXIF_HEADER, ...TIFF_UNNAMED_TAG]), SOS),
);
expect(meta?.exifError).toBeUndefined();
expect(meta?.exif).toMatchObject({ "undefined-49152": { value: 7 } });
});
it("returns nothing for a file that is not an image", () => {
const text = new TextEncoder().encode("just some text, not an image"); const text = new TextEncoder().encode("just some text, not an image");
expect(extractImageMetadata(text)).toBeUndefined(); expect(extractImageMetadata(text)).toBeUndefined();
}); });
@@ -121,14 +349,20 @@ describe("extractImageMetadata", () => {
bytes(SOI, [0xff, 0xe1, 0x40, 0x00], EXIF_HEADER), bytes(SOI, [0xff, 0xe1, 0x40, 0x00], EXIF_HEADER),
); );
expect(meta?.exif).toBeUndefined(); expect(meta?.exif).toBeUndefined();
expect(meta?.exifError).toMatch(/runs past the end of the file/); expect(meta?.exifError).toBe(
"no tag could be read from the EXIF block",
);
}); });
it("keeps the raw bytes and the reason when EXIF cannot be parsed", () => { it("keeps the raw bytes and the reason when EXIF cannot be parsed", () => {
const data = [...EXIF_HEADER, 0x58, 0x58]; // The raw bytes are the whole EXIF block as exifreader finds it: for a
const meta = extractImageMetadata(bytes(SOI, app1(data), SOS)); // JPEG, the APP1 segment, marker and length included.
const segment = app1([...EXIF_HEADER, 0x58, 0x58]);
const meta = extractImageMetadata(bytes(SOI, segment, SOS));
expect(meta?.exif).toBeUndefined(); expect(meta?.exif).toBeUndefined();
expect(meta?.exifRaw).toBe(Buffer.from(data).toString("base64")); expect(meta?.exifRaw).toBe(Buffer.from(segment).toString("base64"));
expect(meta?.exifError).toEqual(expect.any(String)); expect(meta?.exifError).toBe(
"no tag could be read from the EXIF block",
);
}); });
}); });
-9
View File
@@ -1883,15 +1883,6 @@ describe("downloadFile live photos", () => {
expect(readdirSync(t.dir).sort()).toEqual(["f.JPG", "f.bin"]); expect(readdirSync(t.dir).sort()).toEqual(["f.JPG", "f.bin"]);
}); });
it("replaces what was at the destination, such as an earlier ZIP of the two", async () => {
const t = setup(livePhotoZip(), livePhoto);
writeFileSync(t.outPath, livePhotoZip());
await t.run();
expect(readdirSync(t.dir).sort()).toEqual(["f.heic", "f.mov"]);
});
it("renames the image and then the video into place, each from its own temp file", async () => { it("renames the image and then the video into place, each from its own temp file", async () => {
const t = setup(livePhotoZip(), livePhoto); const t = setup(livePhotoZip(), livePhoto);
+303
View File
@@ -0,0 +1,303 @@
/**
* The example script `examples/download-albums.ts` (issue #144), run twice
* against a stand-in account, as a user would run it twice.
*
* The account has two albums sharing one photo, and one of its photos is a
* live photo whose image is a HEIC with EXIF. The first run puts every
* original at its save path, writes each photo's metadata beside it and each
* album's photos under `albums/`. Before the second run the account gains an
* album holding a new photo. The second run downloads that photo and writes
* its files and the new album's, and fetches nothing else and changes no
* other file.
*/
import { describe, it, expect, beforeEach, afterEach } from "vitest";
import {
existsSync,
mkdtempSync,
readdirSync,
readFileSync,
rmSync,
statSync,
utimesSync,
writeFileSync,
} from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { downloadAlbums } from "../../examples/download-albums.js";
import { Library, type ContentSource } from "../../src/index.js";
import type { CollectionsPage, FilesPage } from "../../src/client.js";
import type { Collection, EnteFile } from "../../src/model/types.js";
import { readAllExifTags } from "../../src/exif.js";
import { HEIC_WITH_EXIF } from "../exif-heic.js";
import {
asLivePhoto,
cdnSource,
livePhotoHash,
livePhotoZip,
VIDEO,
} from "../live-photo.js";
const USER_ID = 7;
// Every photo is taken at noon local time on 2026-03-01, so the machine's time
// zone cannot move it to another day; it is saved in the folder `DAY`. Ente
// stores times in microseconds.
const TAKEN_MS = new Date(2026, 2, 1, 12).getTime();
const DAY = join("2026", "2026-03", "2026-03-01");
const collection = (id: number, name: string): Collection => ({
id,
ownerID: USER_ID,
key: new Uint8Array([id]),
name,
type: "album",
updationTime: 1,
isShared: false,
});
const file = (id: number, collectionID: number): EnteFile => ({
id,
collectionID,
ownerID: USER_ID,
key: new Uint8Array([id]),
metadata: {
title: `file-${id}.jpg`,
fileType: "image",
creationTime: TAKEN_MS * 1000,
modificationTime: TAKEN_MS * 1000,
},
file: { decryptionHeader: "aGVhZGVy" },
thumbnail: { decryptionHeader: "dGh1bWI=" },
updationTime: 1,
});
let root: string;
beforeEach(() => {
root = mkdtempSync(join(tmpdir(), "quak-download-albums-"));
});
afterEach(() => {
if (root && existsSync(root))
rmSync(root, { recursive: true, force: true });
});
// Every file and directory under `dir`, by path.
const entries = (dir: string): string[] =>
readdirSync(dir, { recursive: true, encoding: "utf-8" });
// Set the modification time of everything under `dir` to the epoch, so that
// anything written there afterwards has a later one, however soon it comes.
const backdate = (dir: string): void => {
for (const name of entries(dir)) utimesSync(join(dir, name), 0, 0);
};
// The modification time of everything under `dir`, by path.
const mtimes = (dir: string): Map<string, number> =>
new Map(
entries(dir).map((name) => [name, statSync(join(dir, name)).mtimeMs]),
);
const readJSON = (path: string): unknown =>
JSON.parse(readFileSync(path, "utf-8"));
describe("examples/download-albums.ts", () => {
it("downloads every album's photos with their metadata, and on a second run only what the account gained", async () => {
// Album 1, "Trip", holds photos 1 and 2. Album 2, "Family", holds
// photo 2 and photo 3, a live photo.
const live = await asLivePhoto(
file(3, 2),
livePhotoZip({ "image.heic": HEIC_WITH_EXIF, "video.mov": VIDEO }),
livePhotoHash(HEIC_WITH_EXIF, VIDEO),
);
const collections = [collection(1, "Trip"), collection(2, "Family")];
const filesByAlbum = new Map<number, EnteFile[]>([
[1, [file(1, 1), file(2, 1)]],
[2, [file(2, 2), live.file]],
]);
const client = {
whoami: () => ({ email: "u@example.com", userID: USER_ID }),
collectionsSince: async (): Promise<CollectionsPage> => ({
collections: [...collections],
deleted: [],
cursor: 1,
}),
filesSince: async (args: {
collectionID: number;
}): Promise<FilesPage> => ({
files: filesByAlbum.get(args.collectionID) ?? [],
deleted: [],
cursor: 1,
}),
};
// Photo 3 comes from a stand-in server, encrypted as Ente serves a
// live photo. Any other original is a few bytes naming its photo.
// `calls` counts every fetch, thumbnails included.
const server = cdnSource(new Map([[3, live.body]]));
let calls = 0;
const source: ContentSource = {
original: async (args) => {
calls++;
if (args.file.id === 3) return server.original(args);
const bytes = `original-${args.file.id}`;
writeFileSync(args.destination, bytes);
return { bytesWritten: bytes.length };
},
thumbnail: async (args) => {
calls++;
return server.thumbnail(args);
},
};
const dir = join(root, "photos");
const open = (): Promise<Library> =>
Library.open({
client,
cacheDirectory: join(root, "cache"),
downloadDirectory: dir,
contentSource: source,
refreshIntervalSeconds: 3600,
precacheThumbnails: false,
precacheOriginals: false,
});
const first = await open();
// The cache already holds photo 1's original, so its record names a
// cache path, which the metadata leaves out. download() copies it
// from the cache rather than fetching it again.
await first.photos.byID({ fileID: 1 })!.original();
expect(await downloadAlbums(first, dir)).toEqual({
downloaded: 3,
alreadyLocal: 0,
});
await first.close();
expect(calls).toBe(3);
// Each original at its save path, a live photo as its image, its video
// and the file naming them, and each photo's metadata beside it.
const day = join(dir, DAY);
expect(readdirSync(day).sort()).toEqual([
"2026-03-01.1.jpg",
"2026-03-01.1.jpg.json",
"2026-03-01.2.jpg",
"2026-03-01.2.jpg.json",
"2026-03-01.3.heic",
"2026-03-01.3.heic.json",
"2026-03-01.3.livephoto.json",
"2026-03-01.3.mov",
]);
expect(readFileSync(join(day, "2026-03-01.1.jpg"), "utf-8")).toBe(
"original-1",
);
expect(readFileSync(join(day, "2026-03-01.2.jpg"), "utf-8")).toBe(
"original-2",
);
expect(readFileSync(join(day, "2026-03-01.3.heic"))).toEqual(
Buffer.from(HEIC_WITH_EXIF),
);
expect(readFileSync(join(day, "2026-03-01.3.mov"))).toEqual(
Buffer.from(VIDEO),
);
// The metadata is the photo's record and its EXIF tags. The originals
// of photos 1 and 2 are not image data, so they have no EXIF tags.
// Photo 3's are every tag of its image, as `photo.exif()` returns
// them. `record` holds the fields the three records share.
const record = {
takenAt: TAKEN_MS,
modifiedAt: TAKEN_MS,
fileType: "image",
isArchived: false,
isHidden: false,
};
expect(readJSON(join(day, "2026-03-01.1.jpg.json"))).toEqual({
...record,
fileID: 1,
albumIDs: [1],
title: "file-1.jpg",
exif: {},
});
expect(readJSON(join(day, "2026-03-01.2.jpg.json"))).toEqual({
...record,
fileID: 2,
albumIDs: [1, 2],
title: "file-2.jpg",
exif: {},
});
expect(readJSON(join(day, "2026-03-01.3.heic.json"))).toEqual({
...record,
fileID: 3,
albumIDs: [2],
title: "file-3.jpg",
fileType: "livePhoto",
hash: livePhotoHash(HEIC_WITH_EXIF, VIDEO),
exif: readAllExifTags(HEIC_WITH_EXIF),
});
// Each album's photos, newest first, by save path relative to `dir`.
// Photo 2 is in both.
expect(readdirSync(join(dir, "albums")).sort()).toEqual([
"1.json",
"2.json",
]);
expect(readJSON(join(dir, "albums", "1.json"))).toEqual({
collectionID: 1,
name: "Trip",
savePaths: [
join(DAY, "2026-03-01.2.jpg"),
join(DAY, "2026-03-01.1.jpg"),
],
});
expect(readJSON(join(dir, "albums", "2.json"))).toEqual({
collectionID: 2,
name: "Family",
savePaths: [
join(DAY, "2026-03-01.3.heic"),
join(DAY, "2026-03-01.2.jpg"),
],
});
// Before the second run the account gains album 3, "Garden", holding
// a new photo 4. The second run, with a newly opened library, opens
// the cache written by the first and still downloads photo 4. It finds
// the other photos already local and fetches nothing else.
collections.push(collection(3, "Garden"));
filesByAlbum.set(3, [file(4, 3)]);
backdate(dir);
const before = mtimes(dir);
const second = await open();
expect(await downloadAlbums(second, dir)).toEqual({
downloaded: 1,
alreadyLocal: 3,
});
await second.close();
expect(calls).toBe(4);
expect(readFileSync(join(day, "2026-03-01.4.jpg"), "utf-8")).toBe(
"original-4",
);
expect(readJSON(join(dir, "albums", "3.json"))).toEqual({
collectionID: 3,
name: "Garden",
savePaths: [join(DAY, "2026-03-01.4.jpg")],
});
// The second run adds only photo 4's files and album 3's, and
// rewrites, renames or removes no file from the first run. The two
// directories that gain a file are the only other changes.
const after = mtimes(dir);
expect(
[...after.keys()].filter((name) => !before.has(name)).sort(),
).toEqual([
join(DAY, "2026-03-01.4.jpg"),
join(DAY, "2026-03-01.4.jpg.json"),
join("albums", "3.json"),
]);
for (const [name, mtime] of before) {
if (name === DAY || name === "albums") continue;
expect(after.get(name), name).toBe(mtime);
}
});
});
+27
View File
@@ -0,0 +1,27 @@
/**
* `exif.heic`, beside this file: a real 64x64 HEIC whose EXIF holds the same
* values as the hand-built JPEG in `library/content-library.test.ts`, for the
* tests of `exif()` and `backup-metadata --exif`.
*
* It was made once, in a throwaway node:22-alpine container (Alpine 3.23.3),
* with libheif 1.23.0 and exiftool 13.55:
*
* apk add libheif-tools exiftool imagemagick
* magick -size 64x64 gradient:red-blue -depth 8 in.png
* heif-enc -q 30 -o exif.heic in.png
* exiftool -overwrite_original \
* -Make=Canon -Model="EOS R5" -LensModel="RF50mm F1.8 STM" \
* -DateTimeOriginal="2021:07:15 14:30:00" -OffsetTimeOriginal="+02:00" \
* -ExposureTime=1/250 -FNumber=2.8 -ISO=400 -FocalLength=50 \
* -Orientation#=6 \
* -GPSLatitude="40 26 46" -GPSLatitudeRef=N \
* -GPSLongitude="79 58 56" -GPSLongitudeRef=W \
* -GPSAltitude=12.5 -GPSAltitudeRef#=1 \
* exif.heic
*/
import { readFileSync } from "node:fs";
export const HEIC_WITH_EXIF = new Uint8Array(
readFileSync(new URL("exif.heic", import.meta.url)),
);
BIN
View File
Binary file not shown.
+211 -17
View File
@@ -7,7 +7,7 @@
* a cached path shows up on the projected record. A library opened without a * a cached path shows up on the projected record. A library opened without a
* content source leaves those methods throwing rather than silently doing * content source leaves those methods throwing rather than silently doing
* nothing. It also covers a `Photo`'s `savePath`, `isLocal`, `download()`, * nothing. It also covers a `Photo`'s `savePath`, `isLocal`, `download()`,
* `content()` and `exif()`. * `content()`, `exif()` and the methods that each return one EXIF field.
*/ */
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest"; import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
@@ -27,7 +27,16 @@ import { Library, type LibraryOptions } from "../../src/library/index.js";
import type { ContentSource } from "../../src/library/content.js"; import type { ContentSource } from "../../src/library/content.js";
import type { CollectionsPage, FilesPage } from "../../src/client.js"; import type { CollectionsPage, FilesPage } from "../../src/client.js";
import type { Collection, EnteFile } from "../../src/model/types.js"; import type { Collection, EnteFile } from "../../src/model/types.js";
import { asLivePhoto, cdnSource, IMAGE, livePhotoZip } from "../live-photo.js"; import { readPhotoExif, type PhotoExif } from "../../src/exif.js";
import { HEIC_WITH_EXIF } from "../exif-heic.js";
import {
asLivePhoto,
cdnSource,
IMAGE,
livePhotoHash,
livePhotoZip,
VIDEO,
} from "../live-photo.js";
const USER_ID = 7; const USER_ID = 7;
@@ -167,7 +176,7 @@ describe("Library content wiring", () => {
await lib.close(); await lib.close();
}); });
it("removes a live photo's ZIP an earlier version cached when it opens, and precaches its image and video", async () => { it("does not take a live photo's image or video with no JSON file as its original when it opens, and precaches both", async () => {
const { file: live, body } = await asLivePhoto(file(1, 1)); const { file: live, body } = await asLivePhoto(file(1, 1));
class LiveClient extends MockClient { class LiveClient extends MockClient {
override async filesSince(): Promise<FilesPage> { override async filesSince(): Promise<FilesPage> {
@@ -188,7 +197,8 @@ describe("Library content wiring", () => {
// A first run records the library, so the next one knows that file 1 // A first run records the library, so the next one knows that file 1
// is a live photo when it opens the cache. // is a live photo when it opens the cache.
await (await open({})).close(); await (await open({})).close();
writeFileSync(join(originals, "1.jpg"), livePhotoZip()); writeFileSync(join(originals, "1.heic"), "an image");
writeFileSync(join(originals, "1.mov"), "a video");
let precached!: () => void; let precached!: () => void;
const done = new Promise<void>((r) => (precached = r)); const done = new Promise<void>((r) => (precached = r));
@@ -199,7 +209,9 @@ describe("Library content wiring", () => {
precached(); precached();
}, },
}); });
expect(existsSync(join(originals, "1.jpg"))).toBe(false); expect(
lib.photos.byID({ fileID: 1 })!.record().originalPath,
).toBeUndefined();
await done; await done;
expect(readdirSync(originals).sort()).toEqual([ expect(readdirSync(originals).sort()).toEqual([
@@ -207,6 +219,17 @@ describe("Library content wiring", () => {
"1.livephoto.json", "1.livephoto.json",
"1.mov", "1.mov",
]); ]);
expect(readFileSync(join(originals, "1.heic"))).toEqual(
Buffer.from(IMAGE),
);
expect(readFileSync(join(originals, "1.mov"))).toEqual(
Buffer.from(VIDEO),
);
expect(
JSON.parse(
readFileSync(join(originals, "1.livephoto.json"), "utf-8"),
),
).toEqual({ image: "1.heic", video: "1.mov" });
expect(lib.photos.byID({ fileID: 1 })!.record().originalPath).toBe( expect(lib.photos.byID({ fileID: 1 })!.record().originalPath).toBe(
join(originals, "1.heic"), join(originals, "1.heic"),
); );
@@ -250,10 +273,10 @@ const entry = (
value: number[], value: number[],
): number[] => [...u16(tag), ...u16(type), ...u32(count), ...value]; ): number[] => [...u16(tag), ...u16(type), ...u32(count), ...value];
// The TIFF block of a JPEG's EXIF segment, holding every field `exif()` picks: // The TIFF block of a JPEG's EXIF segment, holding every field `Photo`'s typed
// the camera in the first IFD, the exposure in the Exif IFD, and a GPS position // methods return: the camera in the first IFD, the exposure in the Exif IFD,
// of 40°26'46" N, 79°58'56" W, 12.5 m below sea level. Offsets count from the // and a GPS position of 40°26'46" N, 79°58'56" W, 12.5 m below sea level.
// start of this block. // Offsets count from the start of this block.
const TIFF = [ const TIFF = [
...[0x4d, 0x4d, 0x00, 0x2a], // big-endian TIFF ...[0x4d, 0x4d, 0x00, 0x2a], // big-endian TIFF
...u32(8), // the first IFD's offset ...u32(8), // the first IFD's offset
@@ -308,7 +331,7 @@ const JPEG_WITH_EXIF = new Uint8Array([
]); ]);
// A JPEG whose EXIF segment is laid out correctly but holds "XX" where the TIFF // A JPEG whose EXIF segment is laid out correctly but holds "XX" where the TIFF
// byte order belongs, so exif-reader cannot parse it. // byte order belongs, so exifreader cannot parse it.
const JPEG_WITH_BAD_EXIF = new Uint8Array([ const JPEG_WITH_BAD_EXIF = new Uint8Array([
...[0xff, 0xd8], // start of image ...[0xff, 0xd8], // start of image
...[0xff, 0xe1, ...u16(2 + 6 + 2)], // APP1 and its length ...[0xff, 0xe1, ...u16(2 + 6 + 2)], // APP1 and its length
@@ -457,6 +480,48 @@ describe("Photo save path, local copy, content and EXIF", () => {
await lib.close(); await lib.close();
}); });
it("downloads a photo held across a refresh that edits its date to the save path it names", async () => {
// The same account, whose second refresh brings a date edited in Ente.
const edited = file(1, 1);
edited.updationTime = 2;
edited.pubMagicMetadata = {
editedTime: new Date(2026, 3, 15, 12).getTime() * 1000,
};
class DateEditedClient extends MockClient {
refreshes = 0;
override async collectionsSince(): Promise<CollectionsPage> {
this.refreshes++;
return {
collections: [
{ ...collection(1), updationTime: this.refreshes },
],
deleted: [],
cursor: this.refreshes,
};
}
override async filesSince(): Promise<FilesPage> {
return {
files: [this.refreshes === 1 ? file(1, 1) : edited],
deleted: [],
cursor: this.refreshes,
};
}
}
const lib = await open({ client: new DateEditedClient() });
const photo = lib.photos.byID({ fileID: 1 })!;
await lib.fresh();
expect(lib.photos.byID({ fileID: 1 })!.takenAt).toBe(
new Date(2026, 3, 15, 12).getTime(),
);
const saved = await photo.download();
expect(saved.path).toBe(photo.savePath);
expect(saved.path).toBe(join(root, "backup", DAY, "2026-03-01.1.jpg"));
expect(photo.isLocal).toBe(true);
await lib.close();
});
it("has a save path, and is not local, without a content source", async () => { it("has a save path, and is not local, without a content source", async () => {
const lib = await open({ contentSource: undefined }); const lib = await open({ contentSource: undefined });
const photo = lib.photos.byID({ fileID: 1 })!; const photo = lib.photos.byID({ fileID: 1 })!;
@@ -572,9 +637,72 @@ describe("Photo save path, local copy, content and EXIF", () => {
await lib.close(); await lib.close();
}); });
it("reads the common EXIF fields of a JPEG original", async () => { // Every tag in JPEG_WITH_EXIF, by name, in the order of its IFDs.
const jpegTags = [
"Make",
"Model",
"Orientation",
"Exif IFD Pointer",
"GPS Info IFD Pointer",
"ExposureTime",
"FNumber",
"ISOSpeedRatings",
"DateTimeOriginal",
"OffsetTimeOriginal",
"FocalLength",
"LensModel",
"GPSLatitudeRef",
"GPSLatitude",
"GPSLongitudeRef",
"GPSLongitude",
"GPSAltitudeRef",
"GPSAltitude",
];
// HEIC_WITH_EXIF holds those and the tags exiftool adds to every file.
const heicTags = [
...jpegTags,
"YCbCrPositioning",
"ExifVersion",
"ComponentsConfiguration",
"ColorSpace",
"GPSVersionID",
];
it.each([
[
"JPEG",
JPEG_WITH_EXIF,
jpegTags,
{
"Exif IFD Pointer": { value: 88 },
GPSLatitudeRef: { value: ["N"], description: "North latitude" },
},
],
[
"HEIC",
HEIC_WITH_EXIF,
heicTags,
{
ColorSpace: { value: 0xffff, description: "Uncalibrated" },
ExifVersion: { description: "0232" },
},
],
])(
"returns every EXIF tag of a %s original, by name",
async (_, bytes, names, others) => {
const lib = await open({ contentSource: stubSource(bytes) });
const exif = await lib.photos.byID({ fileID: 1 })!.exif();
expect(Object.keys(exif).sort()).toEqual([...names].sort());
// Tags outside the thirteen fields, as exifreader decodes them.
expect(exif).toMatchObject(others);
await lib.close();
},
);
it("picks the common EXIF fields from a JPEG original's tags", async () => {
const lib = await open({ contentSource: stubSource(JPEG_WITH_EXIF) }); const lib = await open({ contentSource: stubSource(JPEG_WITH_EXIF) });
expect(await lib.photos.byID({ fileID: 1 })!.exif()).toStrictEqual({ const exif = await lib.photos.byID({ fileID: 1 })!.exif();
expect(readPhotoExif(exif)).toStrictEqual({
make: "Canon", make: "Canon",
model: "EOS R5", model: "EOS R5",
lensModel: "RF50mm F1.8 STM", lensModel: "RF50mm F1.8 STM",
@@ -593,13 +721,79 @@ describe("Photo save path, local copy, content and EXIF", () => {
await lib.close(); await lib.close();
}); });
it("returns no EXIF fields for an original that is not a JPEG", async () => { // The fields picked from HEIC_WITH_EXIF's tags, and from JPEG_WITH_EXIF's,
const lib = await open(); // which hold the same values.
expect(await lib.photos.byID({ fileID: 1 })!.exif()).toStrictEqual({}); const heicFields: PhotoExif = {
make: "Canon",
model: "EOS R5",
lensModel: "RF50mm F1.8 STM",
dateTimeOriginal: new Date(Date.UTC(2021, 6, 15, 14, 30)),
offsetTimeOriginal: "+02:00",
exposureTime: 1 / 250,
fNumber: 2.8,
iso: 400,
focalLength: 50,
orientation: 6,
gpsLatitude: 40 + 26 / 60 + 46 / 3600,
gpsLongitude: -(79 + 58 / 60 + 56 / 3600),
gpsAltitude: -12.5,
};
it("picks the same common EXIF fields from a HEIC original's tags", async () => {
const lib = await open({ contentSource: stubSource(HEIC_WITH_EXIF) });
const exif = await lib.photos.byID({ fileID: 1 })!.exif();
expect(readPhotoExif(exif)).toStrictEqual(heicFields);
await lib.close(); await lib.close();
}); });
it("returns no EXIF fields for a JPEG whose EXIF cannot be parsed", async () => { it("reads the EXIF of a live photo whose image is a HEIC", async () => {
const { file: live, body } = await asLivePhoto(
file(1, 1),
livePhotoZip({ "image.heic": HEIC_WITH_EXIF, "video.mov": VIDEO }),
livePhotoHash(HEIC_WITH_EXIF, VIDEO),
);
const lib = await open({
client: new FilesClient([live]),
contentSource: cdnSource(new Map([[1, body]])),
});
const exif = await lib.photos.byID({ fileID: 1 })!.exif();
expect(readPhotoExif(exif)).toStrictEqual(heicFields);
await lib.close();
});
// The build's type check, not this test, makes sure `Photo` has a method
// for every `PhotoExif` field, whatever the fixtures hold: `Photo`
// implements a type with one method per field. This test checks that each
// method gives the field picked from the tags exif() returns.
it.each([
["JPEG", JPEG_WITH_EXIF],
["HEIC", HEIC_WITH_EXIF],
])(
"has a method for each field, agreeing with the tags exif() returns, for a %s",
async (_, bytes) => {
const lib = await open({ contentSource: stubSource(bytes) });
const photo = lib.photos.byID({ fileID: 1 })!;
const fields = readPhotoExif(await photo.exif());
// The file holds every field, so every method is checked.
expect(fields).toStrictEqual(heicFields);
for (const [field, value] of Object.entries(fields)) {
expect(await photo[field as keyof PhotoExif]()).toStrictEqual(
value,
);
}
await lib.close();
},
);
it("returns no EXIF tags for an original that is not an image", async () => {
const lib = await open();
const photo = lib.photos.byID({ fileID: 1 })!;
expect(await photo.exif()).toStrictEqual({});
expect(await photo.dateTimeOriginal()).toBeUndefined();
await lib.close();
});
it("returns no EXIF tags for a JPEG whose EXIF cannot be parsed", async () => {
const lib = await open({ const lib = await open({
contentSource: stubSource(JPEG_WITH_BAD_EXIF), contentSource: stubSource(JPEG_WITH_BAD_EXIF),
}); });
@@ -620,7 +814,7 @@ describe("Photo save path, local copy, content and EXIF", () => {
await lib.close(); await lib.close();
}); });
it("returns no EXIF fields for a video, without fetching it", async () => { it("returns no EXIF tags for a video, without fetching it", async () => {
const video = file(1, 1); const video = file(1, 1);
video.metadata.fileType = "video"; video.metadata.fileType = "video";
const source = stubSource(JPEG_WITH_EXIF); const source = stubSource(JPEG_WITH_EXIF);
+30 -37
View File
@@ -566,43 +566,6 @@ describe("ContentCache live photos", () => {
expect(events).toEqual(["skipped"]); expect(events).toEqual(["skipped"]);
}); });
it("replaces a live photo an earlier version stored as a ZIP under the image's name", async () => {
const { file: live, body } = await asLivePhoto(file(5, "IMG_5.HEIC"));
mkdirSync(originals(), { recursive: true });
writeFileSync(join(originals(), "5.HEIC"), livePhotoZip());
const cache = cacheOf([live], new Map([[5, body]]));
// Opened without being told that file 5 is a live photo, the cache
// records the ZIP, and does not serve it.
await cache.open();
const result = await cache.original(5);
expect(result.videoPath).toBe(join(originals(), "5.mov"));
expect(readdirSync(originals()).sort()).toEqual([
"5.heic",
"5.livephoto.json",
"5.mov",
]);
});
it("removes a live photo's ZIP an earlier version stored when it opens, so the precache fetches the image and video", async () => {
const { file: live, body } = await asLivePhoto(file(5, "IMG_5.HEIC"));
mkdirSync(originals(), { recursive: true });
writeFileSync(join(originals(), "5.HEIC"), livePhotoZip());
const cache = cacheOf([live], new Map([[5, body]]));
await cache.open((fileID) => fileID === 5);
expect(readdirSync(originals())).toEqual([]);
expect(cache.pathsFor(5)).toEqual({});
const [fetched] = await cache.ensureOriginals({ fileIDs: [5] });
expect(fetched).toEqual({
fileID: 5,
path: join(originals(), "5.heic"),
});
expect(cache.pathsFor(5)).toEqual({ originalPath: fetched!.path });
});
it("leaves the image and video another process has just stored when it opens before their JSON file is written", async () => { it("leaves the image and video another process has just stored when it opens before their JSON file is written", async () => {
const { file: live, body } = await asLivePhoto(file(5, "IMG_5.HEIC")); const { file: live, body } = await asLivePhoto(file(5, "IMG_5.HEIC"));
const server = cdnSource(new Map([[5, body]])); const server = cdnSource(new Map([[5, body]]));
@@ -633,6 +596,36 @@ describe("ContentCache live photos", () => {
expect(second!.pathsFor(5)).toEqual({}); expect(second!.pathsFor(5)).toEqual({});
}); });
it("fetches a live photo's image and video again when the cache opened before knowing it is a live photo and no JSON file names them", async () => {
const { file: live, body } = await asLivePhoto(file(5, "IMG_5.HEIC"));
mkdirSync(originals(), { recursive: true });
writeFileSync(join(originals(), "5.heic"), "an image");
writeFileSync(join(originals(), "5.mov"), "a video");
const cache = cacheOf([live], new Map([[5, body]]));
// Opened without being told that file 5 is a live photo, the cache
// records one of the two files as its original, with no video.
await cache.open();
const events: string[] = [];
const result = await cache.original(5, {
onProgress: (e) => events.push(e.status),
});
expect(events.at(-1)).toBe("done");
expect(result).toEqual({
path: join(originals(), "5.heic"),
videoPath: join(originals(), "5.mov"),
bytes: IMAGE.length,
});
expect(readFileSync(result.path)).toEqual(Buffer.from(IMAGE));
expect(readFileSync(result.videoPath!)).toEqual(Buffer.from(VIDEO));
expect(
JSON.parse(
readFileSync(join(originals(), "5.livephoto.json"), "utf-8"),
),
).toEqual({ image: "5.heic", video: "5.mov" });
});
it.each(["missing", "empty"])( it.each(["missing", "empty"])(
"fetches a live photo again when the video its JSON file names is %s", "fetches a live photo again when the video its JSON file names is %s",
async (state) => { async (state) => {
+14 -1
View File
@@ -9,8 +9,10 @@
// Excluding too much: Prettier 3 reads `.gitignore` as a default ignore file, // Excluding too much: Prettier 3 reads `.gitignore` as a default ignore file,
// so dropping it from the context silently changes which files the lint // so dropping it from the context silently changes which files the lint
// phase's prettier check looks at compared to `make fmt-check` on the host. // phase's prettier check looks at compared to `make fmt-check` on the host.
// And without `.git`, a `docker build .` given no `VERSION` build arg cannot
// derive the version (`script/version`) and stamps `package.json`'s instead.
// //
// Neither shows up as a build failure, so they are asserted here. // None of these shows up as a build failure, so they are asserted here.
import { describe, expect, it } from "vitest"; import { describe, expect, it } from "vitest";
import { existsSync, readFileSync } from "node:fs"; import { existsSync, readFileSync } from "node:fs";
import { fileURLToPath } from "node:url"; import { fileURLToPath } from "node:url";
@@ -47,6 +49,17 @@ describe(".dockerignore", () => {
expect(dockerignore).not.toContain(".gitignore"); expect(dockerignore).not.toContain(".gitignore");
}); });
it("leaves .git in the build context for the version", () => {
expect(dockerignore).not.toContain(".git");
expect(dockerignore).not.toContain(".git/");
});
// The build stage is the final image, so a .git/config sent in would
// ship the clone's remote URL and any credential in it.
it("sends .git without its config", () => {
expect(dockerignore).toContain(".git/config");
});
// BuildKit lets a `Dockerfile.dockerignore` shadow the root one; such a // BuildKit lets a `Dockerfile.dockerignore` shadow the root one; such a
// file would silently give the build a different, unreviewed context — // file would silently give the build a different, unreviewed context —
// and eslint's flat config does not ignore dot-directories, so a stray // and eslint's flat config does not ignore dot-directories, so a stray
+137
View File
@@ -0,0 +1,137 @@
// `script/version` prints the version `script/build` stamps into
// `dist/package.json`, which is what `quak --version` reports from a build.
// A `docker build .` of a clone is given no `VERSION` build arg, so the
// version has to come from the `.git` in its context: the tag on a tagged
// commit; the tag, the commits since it and the short commit on a later commit;
// the short commit when no tag is reachable. A checkout with `.git` that still
// yields no usable version must fail the build, not ship a version nobody can
// trace back to its commit.
//
// Each test copies the script into a fresh directory, which the script then
// treats as the checkout, and executes it there.
import { afterEach, describe, expect, it } from "vitest";
import { execFileSync, spawnSync } from "node:child_process";
import {
chmodSync,
copyFileSync,
mkdirSync,
mkdtempSync,
rmSync,
writeFileSync,
} from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
const repoRoot = fileURLToPath(new URL("../../", import.meta.url));
let checkout = "";
afterEach(() => {
rmSync(checkout, { recursive: true, force: true });
});
// A checkout holding the script and a package.json that declares 0.0.0, with
// no .git yet.
const makeCheckout = (): void => {
checkout = mkdtempSync(join(tmpdir(), "quak-version-"));
mkdirSync(join(checkout, "script"));
copyFileSync(
join(repoRoot, "script/version"),
join(checkout, "script/version"),
);
chmodSync(join(checkout, "script/version"), 0o755);
writeFileSync(join(checkout, "package.json"), '{ "version": "0.0.0" }\n');
};
// git in the checkout, with an identity and no commit signing, whatever the
// host's own git config says.
const git = (...args: string[]): string =>
execFileSync(
"git",
[
"-c",
"user.name=quak",
"-c",
"user.email=quak@example.invalid",
"-c",
"commit.gpgsign=false",
...args,
],
{ cwd: checkout, encoding: "utf-8", stdio: ["ignore", "pipe", "pipe"] },
).trim();
const makeCommittedCheckout = (): void => {
makeCheckout();
git("init", "-q");
git("add", "package.json");
git("commit", "-q", "-m", "first");
};
// Runs the script with nothing in its environment but PATH and, when given,
// VERSION.
const runVersion = (version?: string) =>
spawnSync(join(checkout, "script/version"), {
cwd: checkout,
encoding: "utf-8",
env: { PATH: process.env.PATH, VERSION: version },
});
describe("script/version", () => {
it("prints the short commit of an untagged commit", () => {
makeCommittedCheckout();
expect(runVersion().stdout.trim()).toBe(
git("rev-parse", "--short", "HEAD"),
);
});
it("prints the tag of a tagged commit", () => {
makeCommittedCheckout();
git("tag", "v1.2.3");
expect(runVersion().stdout.trim()).toBe("v1.2.3");
});
// script/docker and script/cibuild pass the version they resolve on the
// host as the VERSION build arg.
it("prints the VERSION it is given over what git would derive", () => {
makeCommittedCheckout();
expect(runVersion("x").stdout.trim()).toBe("x");
});
// `--build-arg VERSION=` must not stamp an empty version.
it("treats an empty VERSION as unset", () => {
makeCommittedCheckout();
expect(runVersion("").stdout.trim()).toBe(
git("rev-parse", "--short", "HEAD"),
);
});
// A source tarball has no .git: it keeps the version package.json
// declares, and must still build.
it("prints package.json's version where there is no .git", () => {
makeCheckout();
const result = runVersion();
expect(result.status).toBe(0);
expect(result.stdout.trim()).toBe("0.0.0");
});
// A repository with no commits stands in for any .git that git cannot
// describe: git missing from the image, or refusing to read the checkout.
it("fails where .git yields no version", () => {
makeCheckout();
git("init", "-q");
const result = runVersion();
expect(result.status).not.toBe(0);
expect(result.stdout).toBe("");
});
it.each(["dev", "unknown"])(
"fails where there is .git and the version is %s",
(version) => {
makeCommittedCheckout();
const result = runVersion(version);
expect(result.status).not.toBe(0);
expect(result.stdout).toBe("");
},
);
});
+5 -4
View File
@@ -787,12 +787,13 @@ describe("fixMissingThumbnails", () => {
}); });
it("re-encodes smaller until the thumbnail fits the recorded size", async () => { it("re-encodes smaller until the thumbnail fits the recorded size", async () => {
// A noisy 400x300 JPEG, which the default encoding (quality 50, not // A noisy 64x48 JPEG, which the default encoding (quality 50, not
// resized because it is under 720 px) cannot compress below the size // resized because it is under 720 px) cannot compress below the size
// recorded here: one byte less than that encoding's ciphertext. // recorded here: one byte less than that encoding's ciphertext. It is
// small so that each encode is quick even on a busy host.
const fixMock = await buildThumbMock(); const fixMock = await buildThumbMock();
const w = 400; const w = 64;
const h = 300; const h = 48;
const noisy = new Uint8Array( const noisy = new Uint8Array(
jpegJs.encode( jpegJs.encode(
{ {
+1 -1
View File
@@ -18,5 +18,5 @@
"sourceMap": true, "sourceMap": true,
"resolveJsonModule": true "resolveJsonModule": true
}, },
"include": ["src/**/*", "bin/**/*"] "include": ["src/**/*", "bin/**/*", "examples/**/*"]
} }
+11 -4
View File
@@ -702,6 +702,11 @@
loupe "^3.1.2" loupe "^3.1.2"
tinyrainbow "^1.2.0" tinyrainbow "^1.2.0"
"@xmldom/xmldom@^0.9.10":
version "0.9.12"
resolved "https://registry.yarnpkg.com/@xmldom/xmldom/-/xmldom-0.9.12.tgz#1f84c07cb95ccf28202299f77b5fd7fc257151e8"
integrity sha512-5AXjrcMClTryPe9LgZrygpB1lj7s0S9E0+W+AHaVKAVyHanafK86iPSvG5xHVSp/jC+VH1UXu0TAEmY279xH7A==
acorn-jsx@^5.3.2: acorn-jsx@^5.3.2:
version "5.3.2" version "5.3.2"
resolved "https://registry.yarnpkg.com/acorn-jsx/-/acorn-jsx-5.3.2.tgz#7ed5bb55908b3b2f1bc55c6af1653bada7f07937" resolved "https://registry.yarnpkg.com/acorn-jsx/-/acorn-jsx-5.3.2.tgz#7ed5bb55908b3b2f1bc55c6af1653bada7f07937"
@@ -1002,10 +1007,12 @@ esutils@^2.0.2:
resolved "https://registry.yarnpkg.com/esutils/-/esutils-2.0.3.tgz#74d2eb4de0b8da1293711910d50775b9b710ef64" resolved "https://registry.yarnpkg.com/esutils/-/esutils-2.0.3.tgz#74d2eb4de0b8da1293711910d50775b9b710ef64"
integrity sha512-kVscqXk4OCp68SZ0dkgEKVi6/8ij300KBWTJq32P/dYeWTSwK41WyTxalN1eRmA5Z9UU/LX9D7FWSmV9SAYx6g== integrity sha512-kVscqXk4OCp68SZ0dkgEKVi6/8ij300KBWTJq32P/dYeWTSwK41WyTxalN1eRmA5Z9UU/LX9D7FWSmV9SAYx6g==
exif-reader@2.0.3: exifreader@4.46.0:
version "2.0.3" version "4.46.0"
resolved "https://registry.yarnpkg.com/exif-reader/-/exif-reader-2.0.3.tgz#259997735080bc6bb959c37b32c60f004ec4391d" resolved "https://registry.yarnpkg.com/exifreader/-/exifreader-4.46.0.tgz#b6216eae512997587c45114f972cc14ca979205f"
integrity sha512-zFbQvguwT9JkqyYhR7pjE1Yn8SagwaGLNRU0Oh14xFa1paSf5Gzxn4gxgk0XhnudI0UIqU+HgnBX93+nva592A== integrity sha512-ksHTpjXKWzbckY+bYlGaomG0EobHJkaMWLg5OPbzlTPda04F2dfrDfYSz8iAPp/kXYUSQdINBVI6nCAjQ8PQ/Q==
optionalDependencies:
"@xmldom/xmldom" "^0.9.10"
expect-type@^1.1.0: expect-type@^1.1.0:
version "1.3.0" version "1.3.0"