Compare commits

..
2 Commits
Author SHA1 Message Date
sneak b42c5b0e0d download-albums example: walk the albums from lib.fresh()
check / check (push) Successful in 1m32s
downloadAlbums now waits for a refresh from the server through lib.fresh()
before walking the albums, so albums and photos added since the cache was
last written are downloaded, and a failed refresh throws instead of
reporting an empty or stale library as done. The test's stand-in account
gains an album holding a new photo before the second run, and that run must
download it. The README's "Examples" section says the script opens the
library with the thumbnail and originals precache off, as `quak backup` does.

Model: opus-5-5
2026-10-01 22:10:19 +00:00
clawbot 56641c6785 Example script: download every album's photos and metadata (closes #144)
`examples/download-albums.ts` logs in with `QUAK_EMAIL` and `QUAK_PASSWORD`,
opens the library, and for every album downloads each photo to its save path,
writes `{savePath}.json` with the photo's record (cache paths left out) and its
EXIF fields, and writes `albums/{collectionID}.json` with the album's save
paths. A JSON file is written only when its content changed, so a second run
downloads and rewrites nothing. `tsconfig.json` includes `examples/`, so the
build type-checks it. A test runs it twice against a stand-in account.

Model: opus-5-5
2026-10-01 22:10:19 +00:00
22 changed files with 253 additions and 720 deletions
+3 -6
View File
@@ -1,12 +1,9 @@
# Mirrors .gitignore, with one deliberate exception: .gitignore itself stays # Mirrors .gitignore, with one deliberate exception: .gitignore itself stays
# in the build context, because prettier 3 reads it as a default ignore file # in the build context, because prettier 3 reads it as a default ignore file
# and dropping it would change what the lint phase's prettier check sees. # and dropping it would change what the lint phase's prettier check sees.
#
# .git is deliberately NOT excluded: the build derives the version it stamps # VCS
# from it (script/version). It is sent without its config, which holds the .git
# clone's remote URL and any credential in it, and which the build stage, the
# final image, would otherwise carry. git describe does not need it.
.git/config
# OS # OS
.DS_Store .DS_Store
+3 -6
View File
@@ -61,12 +61,9 @@ RUN script/bootstrap
COPY . . COPY . .
# Version stamped into the build: the VERSION build arg when one is given, # The version is computed on the host and passed in, because
# otherwise what script/version derives from the .git the build context # .dockerignore excludes .git.
# carries (script/bootstrap installed git), so any `docker build .` of a ARG VERSION=dev
# clone stamps its commit. The label can only carry the build arg, and is
# empty without one.
ARG VERSION
LABEL org.opencontainers.image.version="${VERSION}" LABEL org.opencontainers.image.version="${VERSION}"
RUN make build RUN make build
+2 -4
View File
@@ -26,10 +26,8 @@ check:
build: build:
@script/build @script/build
# Bundles the built dist/, so the binary reports the version script/build build-bin:
# stamped. nix-shell -p bun --run "bun build bin/quak.ts --compile --outfile bin/quak"
build-bin: build
nix-shell -p bun --run "bun build dist/bin/quak.js --compile --outfile bin/quak"
install: build-bin install: build-bin
mkdir -p ~/bin mkdir -p ~/bin
+21 -54
View File
@@ -101,8 +101,7 @@ requires one, and writes:
photo its image, its video and the `.livephoto.json` file naming them photo its image, its video and the `.livephoto.json` file naming them
- beside each original, a JSON file named after it with `.json` added, for - beside each original, a JSON file named after it with `.json` added, for
example `2026-03-01.12345.jpg.json`: the photo's record (`photo.record()`) example `2026-03-01.12345.jpg.json`: the photo's record (`photo.record()`)
without its cache paths, and every EXIF tag of the photo (`photo.exif()`) without its cache paths, and its EXIF fields (`photo.exif()`) under `exif`
under `exif`
- `albums/<collectionID>.json` for each album: its `collectionID`, its `name`, - `albums/<collectionID>.json` for each album: its `collectionID`, its `name`,
and under `savePaths` the save paths of its photos relative to `dir`, newest and under `savePaths` the save paths of its photos relative to `dir`, newest
first first
@@ -126,12 +125,9 @@ alpine. We provide:
`script/bootstrap`, then `script/install-precommit` `script/bootstrap`, then `script/install-precommit`
- `script/projectname` — output the project name (our own extension); used by - `script/projectname` — output the project name (our own extension); used by
`script/docker` for the image tag `script/docker` for the image tag
- `script/build` — compile the TypeScript sources into `dist/`, stamp the - `script/build` — compile the TypeScript sources into `dist/`, then verify that
version into `dist/package.json`, then verify that the entrypoints the entrypoints `package.json` declares (`main`, `types`, `bin`) are among the
`package.json` declares (`main`, `types`, `bin`) are among the files the files the compiler wrote, and make the CLI executable (our own extension)
compiler wrote, and make the CLI executable (our own extension)
- `script/version` — print the version `script/build` stamps (our own
extension); see Version below
- `script/test` — run the test suite, by building the `test` phase of the - `script/test` — run the test suite, by building the `test` phase of the
`Dockerfile` (vitest, 90s timeout, verbose rerun on failure); requires docker `Dockerfile` (vitest, 90s timeout, verbose rerun on failure); requires docker
- `script/lint` — run eslint and a prettier check, by building the `lint` phase - `script/lint` — run eslint and a prettier check, by building the `lint` phase
@@ -179,34 +175,6 @@ an exact version, installed from `yarn.lock` under `--frozen-lockfile` in both
places, and reads `.gitignore` as its default ignore file — which is why places, and reads `.gitignore` as its default ignore file — which is why
`.dockerignore` keeps `.gitignore` in the build context. `.dockerignore` keeps `.gitignore` in the build context.
### Version
`quak --version` reports the `version` of `dist/package.json`, which
`script/build` writes after compiling; the repo's own `package.json` keeps
`0.0.0`, and that is what the tests, which run from source, report.
`script/version` decides what is written:
- the `VERSION` environment variable, or the `Dockerfile`'s `VERSION` build arg
(`--build-arg VERSION=...`), when one is given and not empty;
- otherwise, in a checkout with `.git`, `git describe --tags --always`: the tag
on a tagged commit; the tag, the commits since it and the short commit on a
later commit (`v1.2.3-4-gabc1234`); the short commit when no tag is reachable;
- otherwise, as in a source tarball, the version `package.json` declares.
The build fails if the checkout has `.git` and the version still comes out
empty, `dev` or `unknown`: such a build could not be traced back to its commit.
`.dockerignore` therefore does not leave out `.git`, so any `docker build .` of
a clone stamps the commit it was built from; a shallow clone stamps a tag only
when the cloned commit itself carries one, and otherwise the short commit. It
leaves out `.git/config`, which holds the clone's remote URL and any credential
in it, so the image carries `.git` without its config; `git describe` does not
need that file. `script/docker` (and so `make docker`) and `script/cibuild` pass
the version they resolve on the host, with `--dirty`, as the build arg, which
takes precedence. The image's `org.opencontainers.image.version` label carries
that build arg only, so a build given none leaves it empty. `make build-bin`
bundles the built `dist/`, so the single binary reports the stamped version too.
## Rationale ## Rationale
Ente is one of very few photo services with a credible end-to-end encryption Ente is one of very few photo services with a credible end-to-end encryption
@@ -805,25 +773,21 @@ These async methods may download:
- `await photo.thumbnail(opts?)` → `{ path, bytes }`. - `await photo.thumbnail(opts?)` → `{ path, bytes }`.
- `await photo.content(opts?)` → `Uint8Array` — the original's bytes, read - `await photo.content(opts?)` → `Uint8Array` — the original's bytes, read
through `original()`; for a live photo, its image's. through `original()`; for a live photo, its image's.
- `await photo.exif(opts?)` → `ExifTags` — every EXIF tag in the file, keyed by - `await photo.exif(opts?)` → `PhotoExif` — `make`, `model`, `lensModel`,
tag name, each as exifreader decodes it, with its `id`, `value`, `description` `dateTimeOriginal`, `offsetTimeOriginal`, `exposureTime`, `fNumber`, `iso`,
and `computed` value: for example `Make` is `focalLength`, `orientation`, `gpsLatitude`, `gpsLongitude` and `gpsAltitude`,
`{ id: 271, value: ["Canon"], description: "Canon", computed: "Canon" }`. A each absent when the file lacks it. GPS values are signed decimal degrees and
tag exifreader has no name for is keyed `undefined-<tag number>`. The embedded metres. `dateTimeOriginal` is the camera's clock reading held in the `Date`'s
thumbnail's tags are under `Thumbnail`, so they cannot hide the main image's UTC fields; `offsetTimeOriginal`, when present, is that clock's offset from
tags of the same name; the thumbnail image itself is left out. EXIF is read UTC. EXIF is read from any image format exifreader reads (such as JPEG,
from any image format exifreader reads (such as JPEG, HEIC/HEIF, AVIF, PNG, HEIC/HEIF, AVIF, PNG, WebP and TIFF), a live photo's image included. Any other
WebP and TIFF), a live photo's image included. Any other original gives `{}`, original gives `{}`, and a video gives `{}` without being downloaded.
and a video gives `{}` without being downloaded.
- `await photo.make(opts?)`, and likewise `model()`, `lensModel()`, - `await photo.make(opts?)`, and likewise `model()`, `lensModel()`,
`dateTimeOriginal()`, `offsetTimeOriginal()`, `exposureTime()`, `fNumber()`, `dateTimeOriginal()`, `offsetTimeOriginal()`, `exposureTime()`, `fNumber()`,
`iso()`, `focalLength()`, `orientation()`, `gpsLatitude()`, `gpsLongitude()` `iso()`, `focalLength()`, `orientation()`, `gpsLatitude()`, `gpsLongitude()`
and `gpsAltitude()` → one common field each, picked from the tags `exif()` and `gpsAltitude()` → one field of `exif()` each, typed as in `PhotoExif`, or
returns and typed as in `PhotoExif`, or `undefined` when the file lacks it. `undefined` when the file lacks it. Each calls `exif()` with its `opts`, so
GPS values are signed decimal degrees and metres. `dateTimeOriginal()` is the each call reads the original again.
camera's clock reading held in the `Date`'s UTC fields;
`offsetTimeOriginal()`, when present, is that clock's offset from UTC. Each
calls `exif()` with its `opts`, so each call reads the original again.
They serve from the on-disk content cache when the bytes are present and They serve from the on-disk content cache when the bytes are present and
otherwise fetch through the pools; `original()`, `content()` and `exif()` also otherwise fetch through the pools; `original()`, `content()` and `exif()` also
@@ -915,7 +879,10 @@ current account's records name.
A live photo's original is cached as at its save path: its image and its video, A live photo's original is cached as at its save path: its image and its video,
each `originals/<fileID>.<ext>` with its own extension, and each `originals/<fileID>.<ext>` with its own extension, and
`originals/<fileID>.livephoto.json` naming them; the two are evicted together. `originals/<fileID>.livephoto.json` naming them; the two are evicted together. A
live photo that an earlier version cached as its ZIP is not served: the library
removes the ZIP when it opens the cache, and fetches the two files when the
photo is next read or precached.
A stored file appears only via an atomic temp-then-rename, so its presence means A stored file appears only via an atomic temp-then-rename, so its presence means
it is complete. Every downloaded original (by `quak get`, the cache, or it is complete. Every downloaded original (by `quak get`, the cache, or
@@ -940,7 +907,7 @@ from a very old client, is stored unchecked.
- `src/library/records.ts`: `PhotoRecord`, `AlbumRecord`, `LibrarySnapshot`, - `src/library/records.ts`: `PhotoRecord`, `AlbumRecord`, `LibrarySnapshot`,
`LibraryChange` `LibraryChange`
- `src/library/mlsearch.ts`: `MLDataAPI`, `SimilarResult` - `src/library/mlsearch.ts`: `MLDataAPI`, `SimilarResult`
- `src/exif.ts`: `ExifTags`, `PhotoExif` - `src/exif.ts`: `PhotoExif`
- `src/library/pools.ts`: `RequestPools`, `RequestPoolsOptions`, `BoundedPool` - `src/library/pools.ts`: `RequestPools`, `RequestPoolsOptions`, `BoundedPool`
- `src/backup.ts`: `BackupOptions`, `BackupResult`, `BackupError` - `src/backup.ts`: `BackupOptions`, `BackupResult`, `BackupError`
- `src/client.ts`: `Client`, `LoginOptions`, `ClientSnapshot` - `src/client.ts`: `Client`, `LoginOptions`, `ClientSnapshot`
-20
View File
@@ -25,26 +25,6 @@ declares one.
# Completed Steps # Completed Steps
- 2026-10-02: `docker build .` stamps the commit's tag or short commit, not
`dev` (issue 154). `script/build` writes the version `script/version` prints
into `dist/package.json`: the `VERSION` environment variable or build arg when
one is given, otherwise `git describe --tags --always`, otherwise the version
`package.json` declares. `.dockerignore` sends `.git`, and a checkout with
`.git` whose version comes out empty, `dev` or `unknown` fails the build.
`make build-bin` bundles the built `dist/`, so the single binary reports the
same version.
- 2026-10-02: `photo.exif()` returns every EXIF tag in the file as `ExifTags`,
keyed by tag name, each as exifreader decodes it, not only the thirteen common
fields (issue 156). The embedded thumbnail's tags are under `Thumbnail`,
without the thumbnail image. The thirteen typed methods stay, each picking its
field from the tags `exif()` returns, typed as in `PhotoExif`. The example
script's JSON files now carry every tag.
- 2026-10-01: The content cache no longer looks for a live photo that an earlier
version cached as one ZIP (issue 151). When the cache opens, a live photo's
file that no JSON file names is now always left alone.
- 2026-10-01: `examples/download-albums.ts` logs in, opens the library, and for - 2026-10-01: `examples/download-albums.ts` logs in, opens the library, and for
every album downloads each photo to its save path, writes the photo's record every album downloads each photo to its save path, writes the photo's record
and EXIF fields to a JSON file beside it, and writes the album's photos to and EXIF fields to a JSON file beside it, and writes the album's photos to
+9 -23
View File
@@ -1,7 +1,7 @@
#!/bin/sh #!/bin/sh
# script/build: compile the TypeScript sources into dist/, stamp the version # script/build: compile the TypeScript sources into dist/, then verify that
# script/version prints into it, then verify that the artifacts package.json # the artifacts package.json advertises are among the files the compiler
# advertises are among the files the compiler actually wrote. tsc reports success by exit status alone and knows nothing # actually wrote. tsc reports success by exit status alone and knows nothing
# about the manifest, so without this step a green build can still ship a # about the manifest, so without this step a green build can still ship a
# package whose main, types or bin resolve to nothing. Our own extension to # package whose main, types or bin resolve to nothing. Our own extension to
# scripts-to-rule-them-all. # scripts-to-rule-them-all.
@@ -46,24 +46,13 @@ for (const bin of bins) {
} }
# src/index.ts imports ../package.json for the version, which tsc copies to # src/index.ts imports ../package.json for the version, which tsc copies to
# dist/package.json. The version script/version prints is written into that # dist/package.json. Running the built CLI proves that import resolves from
# copy only; the repo's own package.json is left as it is. # dist/ and reports the version package.json declares.
stamp_version() {
node -e '
const { readFileSync, writeFileSync } = require("node:fs");
const pkg = JSON.parse(readFileSync("dist/package.json", "utf-8"));
pkg.version = process.argv[1];
writeFileSync("dist/package.json", JSON.stringify(pkg, null, 4) + "\n");
' "$1"
}
# Running the built CLI proves the import resolves from dist/ and reports
# the stamped version.
verify_version() { verify_version() {
built="$(node dist/bin/quak.js --version)" built="$(node dist/bin/quak.js --version)"
if [ "$built" != "$1" ]; then declared="$(node -p 'require("./package.json").version')"
echo "build: dist/bin/quak.js reports $built, the build stamped $1" >&2 if [ "$built" != "$declared" ]; then
echo "build: dist/bin/quak.js reports $built, package.json declares $declared" >&2
exit 1 exit 1
fi fi
echo "build: dist/bin/quak.js reports version $built" echo "build: dist/bin/quak.js reports version $built"
@@ -71,12 +60,9 @@ verify_version() {
main() { main() {
cd "$ROOT" cd "$ROOT"
# Own line, so that a failing script/version stops the build.
version="$("$ROOT/script/version")"
yarn run tsc yarn run tsc
stamp_version "$version"
verify_entrypoints verify_entrypoints
verify_version "$version" verify_version
} }
main "$@" main "$@"
+3 -3
View File
@@ -15,9 +15,9 @@ main() {
cd "$ROOT" cd "$ROOT"
# Own line: a failing command substitution inside an argument does # Own line: a failing command substitution inside an argument does
# not trip `set -e`, so the inline form degrades silently to an # not trip `set -e`, so the inline form degrades silently to an
# empty constant. The version resolved here goes in as the VERSION # empty constant. VERSION is computed here because .dockerignore
# build arg, which takes precedence over what the build would derive # excludes .git, so `git describe` in a build stage yields an empty
# from the .git in its context. # version without failing.
version="$(git describe --tags --always --dirty 2>/dev/null || true)" version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
docker build --no-cache \ docker build --no-cache \
+3 -3
View File
@@ -12,9 +12,9 @@ main() {
cd "$ROOT" cd "$ROOT"
# Own line: a failing command substitution inside an argument does # Own line: a failing command substitution inside an argument does
# not trip `set -e`, so the inline form degrades silently to an # not trip `set -e`, so the inline form degrades silently to an
# empty constant. The version resolved here goes in as the VERSION # empty constant. VERSION is computed here because .dockerignore
# build arg, which takes precedence over what the build would derive # excludes .git, so `git describe` in a build stage yields an empty
# from the .git in its context. # version without failing.
version="$(git describe --tags --always --dirty 2>/dev/null || true)" version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
docker build --no-cache \ docker build --no-cache \
-41
View File
@@ -1,41 +0,0 @@
#!/bin/sh
# script/version: print the version script/build stamps into the built
# package. Our own extension to scripts-to-rule-them-all.
#
# Order of precedence:
#
# 1. $VERSION, if set and not empty: an explicit value, such as the
# Dockerfile's VERSION build arg.
# 2. If this checkout has .git, `git describe --tags --always`: the tag
# on a tagged commit; the tag, the commits since it and the short
# commit on a later commit (v1.2.3-4-gabc1234); the short commit when
# no tag is reachable.
# 3. Otherwise, as in a source tarball, the version package.json declares.
#
# A checkout with .git whose version still comes out empty, dev or unknown
# fails: git is missing or could not read the checkout, and the build could
# not be traced back to its commit.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
version="${VERSION:-}"
if [ -e .git ]; then
if [ -z "$version" ]; then
version="$(git describe --tags --always || true)"
fi
case "$version" in
"" | dev | unknown)
echo "version: $ROOT has .git, but the version came out '$version'" >&2
exit 1
;;
esac
elif [ -z "$version" ]; then
version="$(node -p 'require("./package.json").version')"
fi
echo "$version"
}
main "$@"
+5 -3
View File
@@ -361,8 +361,8 @@ const openPart = async (
// written unpacked: each part is named `destination` with the extension // written unpacked: each part is named `destination` with the extension
// replaced by its own entry's, and the two must differ ignoring case. When the // replaced by its own entry's, and the two must differ ignoring case. When the
// file records a hash, `<imageHash>:<videoHash>` must match it, each over that // file records a hash, `<imageHash>:<videoHash>` must match it, each over that
// part's own bytes. Only then are the image, then the video, renamed into // part's own bytes. Only then is whatever was at `destination` removed and the
// place; on any failure neither is stored. // image, then the video, renamed into place; on any failure neither is stored.
// //
// The ZIP is chosen by its uploader and may expand enormously, so each part is // The ZIP is chosen by its uploader and may expand enormously, so each part is
// written as it decompresses and never held, and the ZIP is refused once the // written as it decompresses and never held, and the ZIP is refused once the
@@ -486,6 +486,7 @@ const decryptLivePhoto = async (
await part.handle.sync(); await part.handle.sync();
await part.handle.close(); await part.handle.close();
} }
await rm(destination, { force: true });
await rename(image.tmpPath, path); await rename(image.tmpPath, path);
try { try {
await rename(video.tmpPath, videoPath); await rename(video.tmpPath, videoPath);
@@ -569,7 +570,8 @@ const fetchAndDecrypt = async (
}, api.getRetryOptions()); }, api.getRetryOptions());
// Write `file`'s original to `outPath`. A live photo is written as its image // Write `file`'s original to `outPath`. A live photo is written as its image
// and its video beside `outPath` instead (see `decryptLivePhoto`). // and its video beside `outPath` instead, and whatever was at `outPath` is
// removed (see `decryptLivePhoto`).
export const downloadFile = async ( export const downloadFile = async (
api: ApiClient, api: ApiClient,
file: EnteFile, file: EnteFile,
+31 -80
View File
@@ -1,19 +1,19 @@
// EXIF in an original's bytes, read with exifreader, which reads it from JPEG, // EXIF in an original's bytes, read with exifreader, which reads it from JPEG,
// HEIC/HEIF, AVIF, PNG, WebP and the other image formats it supports. // HEIC/HEIF, AVIF, PNG, WebP and the other image formats it supports.
// `backup-metadata --exif` records every EXIF tag it finds except the // `backup-metadata --exif` records every EXIF tag it finds except the
// thumbnail's. `Photo.exif()` returns every tag, the thumbnail's included, and // thumbnail's; `Photo.exif()` returns the common fields picked from them here.
// `Photo`'s typed methods return the common fields picked from them here.
import ExifReader, { type ExpandedTags } from "exifreader"; import ExifReader, { type ExpandedTags } from "exifreader";
// The EXIF tags in `bytes` (`exif`), the embedded thumbnail's tags // The EXIF tags in `bytes` (`exif`), the GPS position exifreader computes from
// (`Thumbnail`), and where the EXIF block lies in `bytes` (`metadataRange`). // them (`gps`), and where the EXIF block lies in `bytes` (`metadataRange`).
// Undefined when exifreader cannot read the file at all, such as a video. An // Undefined when exifreader cannot read the file at all, such as a video. An
// EXIF block it finds but reads no tag from comes back as an empty `exif`. A // EXIF block it finds but reads no tag from comes back as an empty `exif`.
// tag exifreader has no name for is keyed `undefined-<tag number>`. Each tag's // `exif` holds every tag except the thumbnail's; a tag exifreader has no name
// `computed` holds its value as a string or number, or as an array of them for // for is keyed `undefined-<tag number>`. Each tag's `computed` holds its value
// a tag with several values, such as `GPSLatitude`'s `[40, 26, 46]`. A // as a string or number, or as an array of them for a tag with several values,
// fraction with a zero denominator computes to null. // such as `GPSLatitude`'s `[40, 26, 46]`. A fraction with a zero denominator
// computes to null.
export const readExifTags = (bytes: Uint8Array): ExpandedTags | undefined => { export const readExifTags = (bytes: Uint8Array): ExpandedTags | undefined => {
try { try {
return ExifReader.loadView( return ExifReader.loadView(
@@ -23,7 +23,7 @@ export const readExifTags = (bytes: Uint8Array): ExpandedTags | undefined => {
computed: true, computed: true,
includeOffsets: true, includeOffsets: true,
includeUnknown: true, includeUnknown: true,
includeTags: { exif: true, thumbnail: true }, includeTags: { exif: true, gps: true },
}, },
); );
} catch { } catch {
@@ -31,35 +31,7 @@ export const readExifTags = (bytes: Uint8Array): ExpandedTags | undefined => {
} }
}; };
// Every EXIF tag of an original, keyed by name, each as exifreader decodes it // The common EXIF fields of an original. Each is absent when the file lacks it.
// (see `readExifTags`). The embedded thumbnail's own tags are under
// `Thumbnail`, so its `Orientation` or `ImageWidth` cannot hide the main
// image's.
export type ExifTags = Omit<NonNullable<ExpandedTags["exif"]>, "Thumbnail"> & {
Thumbnail?: Omit<
NonNullable<ExpandedTags["Thumbnail"]>,
"type" | "image" | "base64"
>;
};
// Every EXIF tag in `bytes`: `{}` when the file has no EXIF, exifreader cannot
// read its EXIF, or it is not an image exifreader reads.
export const readAllExifTags = (bytes: Uint8Array): ExifTags => {
const tags = readExifTags(bytes);
if (!tags?.Thumbnail) return tags?.exif ?? {};
// exifreader puts the thumbnail's JPEG image beside its tags, as `type`,
// `image` and `base64`. The image is not a tag, so it is left out.
const {
type: _type,
image: _image,
base64: _base64,
...thumbnail
} = tags.Thumbnail;
return { ...tags.exif, Thumbnail: thumbnail };
};
// The common EXIF fields of an original, one for each of `Photo`'s typed
// methods. Each is absent when the file lacks it.
export interface PhotoExif { export interface PhotoExif {
make?: string; make?: string;
model?: string; model?: string;
@@ -108,51 +80,30 @@ const asDate = (v: unknown): Date | undefined => {
return Number.isNaN(date.getTime()) ? undefined : date; return Number.isNaN(date.getTime()) ? undefined : date;
}; };
// GPSLatitude and GPSLongitude hold degrees, minutes and seconds, computed as // The common fields of an original's EXIF: `{}` when the file has no EXIF,
// three numbers. This is them in decimal degrees, negative when `ref`, the // exifreader cannot read its EXIF, or it is not an image exifreader reads.
// GPSLatitudeRef or GPSLongitudeRef tag, is `negativeRef` ("S" or "W"). export const readPhotoExif = (bytes: Uint8Array): PhotoExif => {
// Without that tag the hemisphere is unknown, so it is undefined. const tags = readExifTags(bytes);
const asDegrees = ( const exif = tags?.exif;
dms: unknown, const gps = tags?.gps;
ref: unknown, const altitude = asNumber(exif?.GPSAltitude?.computed);
negativeRef: string,
): number | undefined => {
if (!Array.isArray(dms) || ref === undefined) return undefined;
const [d, m, s] = dms.map(asNumber);
if (d === undefined || m === undefined || s === undefined) return undefined;
const degrees = d + m / 60 + s / 3600;
return ref === negativeRef ? -degrees : degrees;
};
// The common fields picked from an original's EXIF tags, `readAllExifTags`'s
// result: `{}` when there are none.
export const readPhotoExif = (tags: ExifTags): PhotoExif => {
const altitude = asNumber(tags.GPSAltitude?.computed);
const fields: PhotoExif = { const fields: PhotoExif = {
make: asString(tags.Make?.computed), make: asString(exif?.Make?.computed),
model: asString(tags.Model?.computed), model: asString(exif?.Model?.computed),
lensModel: asString(tags.LensModel?.computed), lensModel: asString(exif?.LensModel?.computed),
dateTimeOriginal: asDate(tags.DateTimeOriginal?.computed), dateTimeOriginal: asDate(exif?.DateTimeOriginal?.computed),
offsetTimeOriginal: asString(tags.OffsetTimeOriginal?.computed), offsetTimeOriginal: asString(exif?.OffsetTimeOriginal?.computed),
exposureTime: asNumber(tags.ExposureTime?.computed), exposureTime: asNumber(exif?.ExposureTime?.computed),
fNumber: asNumber(tags.FNumber?.computed), fNumber: asNumber(exif?.FNumber?.computed),
// Only when the tag holds a single number, as most cameras write it. // Only when the tag holds a single number, as most cameras write it.
iso: asNumber(tags.ISOSpeedRatings?.computed), iso: asNumber(exif?.ISOSpeedRatings?.computed),
focalLength: asNumber(tags.FocalLength?.computed), focalLength: asNumber(exif?.FocalLength?.computed),
orientation: asNumber(tags.Orientation?.computed), orientation: asNumber(exif?.Orientation?.computed),
gpsLatitude: asDegrees( gpsLatitude: asNumber(gps?.Latitude),
tags.GPSLatitude?.computed, gpsLongitude: asNumber(gps?.Longitude),
tags.GPSLatitudeRef?.computed,
"S",
),
gpsLongitude: asDegrees(
tags.GPSLongitude?.computed,
tags.GPSLongitudeRef?.computed,
"W",
),
// A GPSAltitudeRef of 1 means the altitude is below sea level. // A GPSAltitudeRef of 1 means the altitude is below sea level.
gpsAltitude: gpsAltitude:
altitude !== undefined && tags.GPSAltitudeRef?.value === 1 altitude !== undefined && exif?.GPSAltitudeRef?.value === 1
? -altitude ? -altitude
: altitude, : altitude,
}; };
+3 -5
View File
@@ -1,7 +1,5 @@
// A build reports the version script/build stamps into dist/package.json; // package.json is the one place the version is written. tsc copies it to
// package.json's own version is reported only when running from source. tsc // dist/package.json, so this path resolves from source and from dist/src/.
// copies package.json to dist/package.json, so this path resolves from source
// and from dist/src/.
import pkg from "../package.json" with { type: "json" }; import pkg from "../package.json" with { type: "json" };
export const VERSION: string = pkg.version; export const VERSION: string = pkg.version;
@@ -87,7 +85,7 @@ export type {
LibrarySnapshot, LibrarySnapshot,
LibraryChange, LibraryChange,
} from "./library/records.js"; } from "./library/records.js";
export type { ExifTags, PhotoExif } from "./exif.js"; export type { PhotoExif } from "./exif.js";
export { decryptCollection, decryptFile } from "./model/index.js"; export { decryptCollection, decryptFile } from "./model/index.js";
export { downloadFile, downloadThumbnail } from "./download/index.js"; export { downloadFile, downloadThumbnail } from "./download/index.js";
export type { export type {
+38 -8
View File
@@ -29,7 +29,14 @@
// the cache does not count as saved there, but is copied there rather than // the cache does not count as saved there, but is copied there rather than
// fetched again. // fetched again.
import { existsSync, readFileSync, statSync } from "node:fs"; import {
closeSync,
existsSync,
openSync,
readFileSync,
readSync,
statSync,
} from "node:fs";
import { import {
chmod, chmod,
copyFile, copyFile,
@@ -274,6 +281,25 @@ const fileSize = (path: string): number | undefined => {
const hasContent = (path: string | undefined): boolean => const hasContent = (path: string | undefined): boolean =>
path !== undefined && (fileSize(path) ?? 0) > 0; path !== undefined && (fileSize(path) ?? 0) > 0;
// Whether the file at `path` begins as a ZIP does, with `PK\x03\x04`. False
// when it cannot be read.
const isZip = (path: string): boolean => {
try {
const fd = openSync(path, "r");
try {
const head = Buffer.alloc(4);
return (
readSync(fd, head, 0, 4, 0) === 4 &&
head.toString("latin1") === "PK\x03\x04"
);
} finally {
closeSync(fd);
}
} catch {
return false;
}
};
// A live photo's image and video are named with the extensions from inside its // A live photo's image and video are named with the extensions from inside its
// ZIP, so their names alone do not say which is which. Wherever the cache or a // ZIP, so their names alone do not say which is which. Wherever the cache or a
// save path stores one, a JSON file of this name beside them names both. // save path stores one, a JSON file of this name beside them names both.
@@ -687,10 +713,8 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
await this.touch(cached.path); await this.touch(cached.path);
return { ...cached, bytes: size, cached: true }; return { ...cached, bytes: size, cached: true };
} }
// A recorded file that has since gone re-fetches below. So does a // A recorded file that has since gone, or a live photo an earlier
// live photo recorded with no video: the cache opened before the // version stored as one ZIP, re-fetches below.
// library's records said it is a live photo, while its image and
// video had no JSON file beside them yet.
known.delete(fileID); known.delete(fileID);
} }
@@ -959,14 +983,20 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
if (id === undefined || !existsSync(path)) continue; if (id === undefined || !existsSync(path)) continue;
// A live photo's image and video are one entry, as the JSON file // A live photo's image and video are one entry, as the JSON file
// beside them names them. A live photo's file with no such JSON // beside them names them. A live photo's file with no such JSON
// file is not its original and is left alone: another process may // file is not its original. If it is a ZIP, it is the one an
// have just stored it and not yet written the JSON file. // earlier version stored under the image's name, and is removed.
// Any other is left alone: another process may have just stored
// it and not yet written the JSON file.
const livePhoto = names.has(livePhotoJSONName(String(id))) const livePhoto = names.has(livePhotoJSONName(String(id)))
? readLivePhotoJSON(dir, String(id)) ? readLivePhotoJSON(dir, String(id))
: undefined; : undefined;
if (livePhoto !== undefined) { if (livePhoto !== undefined) {
into.set(id, livePhoto); into.set(id, livePhoto);
} else if (!isLivePhoto(id)) { } else if (isLivePhoto(id)) {
if (isZip(path)) {
await rm(path, { force: true }).catch(() => undefined);
}
} else {
into.set(id, { path }); into.set(id, { path });
} }
} }
+25 -31
View File
@@ -13,19 +13,14 @@
// //
// A `Photo` also fetches its own bytes: `original()`, `thumbnail()`, // A `Photo` also fetches its own bytes: `original()`, `thumbnail()`,
// `download()`, `content()`, `exif()` and the methods that each return one // `download()`, `content()`, `exif()` and the methods that each return one
// EXIF field go through the on-disk content cache (issue #46), and are // field of `exif()` go through the on-disk content cache (issue #46), and are
// the one place in this module that may touch the network. A library opened // the one place in this module that may touch the network. A library opened
// without a content source leaves that cache absent, and those methods then // without a content source leaves that cache absent, and those methods then
// throw. `savePath` and `isLocal` look only at the disk and need no cache. // throw. `savePath` and `isLocal` look only at the disk and need no cache.
import { readFile } from "node:fs/promises"; import { readFile } from "node:fs/promises";
import { import { readPhotoExif, type PhotoExif } from "../exif.js";
readAllExifTags,
readPhotoExif,
type ExifTags,
type PhotoExif,
} from "../exif.js";
import type { CollectionType, EnteFile, FileType } from "../model/types.js"; import type { CollectionType, EnteFile, FileType } from "../model/types.js";
import type { ContentOptions, ContentResult, PhotoContent } from "./content.js"; import type { ContentOptions, ContentResult, PhotoContent } from "./content.js";
import type { AlbumRecord, PhotoRecord, DerivedRecords } from "./records.js"; import type { AlbumRecord, PhotoRecord, DerivedRecords } from "./records.js";
@@ -161,75 +156,74 @@ export class Photo implements PhotoExifMethods {
return readFile(path); return readFile(path);
} }
// Every EXIF tag of the original, keyed by name (see `ExifTags`), read // The common EXIF fields of the original, read from `content()`, so this
// from `content()`, so this may download it. EXIF is read from any image // may download it. EXIF is read from any image format exifreader reads,
// format exifreader reads, JPEG and HEIC/HEIF among them; any other file // JPEG and HEIC/HEIF among them; any other file gives `{}`, and a video
// gives `{}`, and a video gives it without fetching anything. Like the // gives it without fetching anything. Like the other content methods, it
// other content methods, it throws when there is no content cache, video // throws when there is no content cache, video or not.
// or not. async exif(opts?: ContentOptions): Promise<PhotoExif> {
async exif(opts?: ContentOptions): Promise<ExifTags> {
this.cacheOrThrow(); this.cacheOrThrow();
if (this.rec.fileType === "video") return {}; if (this.rec.fileType === "video") return {};
return readAllExifTags(await this.content(opts)); return readPhotoExif(await this.content(opts));
} }
// One field each, named and typed as in `PhotoExif`, picked from the tags // One field of `exif()` each, named and typed as in `PhotoExif`, and
// `exif()` returns, and undefined when the file lacks it. Each call runs // undefined when the file lacks it. Each call runs `exif()`, which reads
// `exif()`, which reads the original again. // the original again.
async make(opts?: ContentOptions): Promise<PhotoExif["make"]> { async make(opts?: ContentOptions): Promise<PhotoExif["make"]> {
return readPhotoExif(await this.exif(opts)).make; return (await this.exif(opts)).make;
} }
async model(opts?: ContentOptions): Promise<PhotoExif["model"]> { async model(opts?: ContentOptions): Promise<PhotoExif["model"]> {
return readPhotoExif(await this.exif(opts)).model; return (await this.exif(opts)).model;
} }
async lensModel(opts?: ContentOptions): Promise<PhotoExif["lensModel"]> { async lensModel(opts?: ContentOptions): Promise<PhotoExif["lensModel"]> {
return readPhotoExif(await this.exif(opts)).lensModel; return (await this.exif(opts)).lensModel;
} }
async dateTimeOriginal( async dateTimeOriginal(
opts?: ContentOptions, opts?: ContentOptions,
): Promise<PhotoExif["dateTimeOriginal"]> { ): Promise<PhotoExif["dateTimeOriginal"]> {
return readPhotoExif(await this.exif(opts)).dateTimeOriginal; return (await this.exif(opts)).dateTimeOriginal;
} }
async offsetTimeOriginal( async offsetTimeOriginal(
opts?: ContentOptions, opts?: ContentOptions,
): Promise<PhotoExif["offsetTimeOriginal"]> { ): Promise<PhotoExif["offsetTimeOriginal"]> {
return readPhotoExif(await this.exif(opts)).offsetTimeOriginal; return (await this.exif(opts)).offsetTimeOriginal;
} }
async exposureTime( async exposureTime(
opts?: ContentOptions, opts?: ContentOptions,
): Promise<PhotoExif["exposureTime"]> { ): Promise<PhotoExif["exposureTime"]> {
return readPhotoExif(await this.exif(opts)).exposureTime; return (await this.exif(opts)).exposureTime;
} }
async fNumber(opts?: ContentOptions): Promise<PhotoExif["fNumber"]> { async fNumber(opts?: ContentOptions): Promise<PhotoExif["fNumber"]> {
return readPhotoExif(await this.exif(opts)).fNumber; return (await this.exif(opts)).fNumber;
} }
async iso(opts?: ContentOptions): Promise<PhotoExif["iso"]> { async iso(opts?: ContentOptions): Promise<PhotoExif["iso"]> {
return readPhotoExif(await this.exif(opts)).iso; return (await this.exif(opts)).iso;
} }
async focalLength( async focalLength(
opts?: ContentOptions, opts?: ContentOptions,
): Promise<PhotoExif["focalLength"]> { ): Promise<PhotoExif["focalLength"]> {
return readPhotoExif(await this.exif(opts)).focalLength; return (await this.exif(opts)).focalLength;
} }
async orientation( async orientation(
opts?: ContentOptions, opts?: ContentOptions,
): Promise<PhotoExif["orientation"]> { ): Promise<PhotoExif["orientation"]> {
return readPhotoExif(await this.exif(opts)).orientation; return (await this.exif(opts)).orientation;
} }
async gpsLatitude( async gpsLatitude(
opts?: ContentOptions, opts?: ContentOptions,
): Promise<PhotoExif["gpsLatitude"]> { ): Promise<PhotoExif["gpsLatitude"]> {
return readPhotoExif(await this.exif(opts)).gpsLatitude; return (await this.exif(opts)).gpsLatitude;
} }
async gpsLongitude( async gpsLongitude(
opts?: ContentOptions, opts?: ContentOptions,
): Promise<PhotoExif["gpsLongitude"]> { ): Promise<PhotoExif["gpsLongitude"]> {
return readPhotoExif(await this.exif(opts)).gpsLongitude; return (await this.exif(opts)).gpsLongitude;
} }
async gpsAltitude( async gpsAltitude(
opts?: ContentOptions, opts?: ContentOptions,
): Promise<PhotoExif["gpsAltitude"]> { ): Promise<PhotoExif["gpsAltitude"]> {
return readPhotoExif(await this.exif(opts)).gpsAltitude; return (await this.exif(opts)).gpsAltitude;
} }
private cacheOrThrow(): PhotoContent { private cacheOrThrow(): PhotoContent {
+8 -138
View File
@@ -3,18 +3,14 @@
* `quak backup-metadata --exif` records. * `quak backup-metadata --exif` records.
* *
* The originals come from users' libraries, so a truncated or corrupt file * The originals come from users' libraries, so a truncated or corrupt file
* must neither hang the read nor throw out of it: `readAllExifTags` gives `{}`, * must neither hang the read nor throw out of it: `readPhotoExif` gives `{}`,
* and `backup-metadata` tells an EXIF block it cannot read apart from a file * and `backup-metadata` tells an EXIF block it cannot read apart from a file
* that simply has no EXIF, carrying the reason in `exifError`. Each JPEG below * that simply has no EXIF, carrying the reason in `exifError`. Each JPEG below
* is a short hand-built byte array; the HEIC is a real file. * is a short hand-built byte array; the HEIC is a real file.
*/ */
import { describe, expect, it } from "vitest"; import { describe, expect, it } from "vitest";
import { import { readPhotoExif } from "../../src/exif.js";
readAllExifTags,
readExifTags,
readPhotoExif,
} from "../../src/exif.js";
import { extractImageMetadata } from "../../src/metadata-backup.js"; import { extractImageMetadata } from "../../src/metadata-backup.js";
import { HEIC_WITH_EXIF } from "../exif-heic.js"; import { HEIC_WITH_EXIF } from "../exif-heic.js";
@@ -67,51 +63,6 @@ const TIFF_ALTITUDE_WITHOUT_REF = [
...[0x00, 0x00, 0x00, 0x19, 0x00, 0x00, 0x00, 0x02], // 25/2, at 44 ...[0x00, 0x00, 0x00, 0x19, 0x00, 0x00, 0x00, 0x02], // 25/2, at 44
]; ];
// A big-endian TIFF block holding a GPSLatitude of 33° 30' 0" and a
// GPSLatitudeRef of "S".
const TIFF_SOUTHERN_LATITUDE = [
...[0x4d, 0x4d, 0x00, 0x2a, 0x00, 0x00, 0x00, 0x08], // the first IFD at 8
// The first IFD, at 8: one entry, then no next IFD.
...[0x00, 0x01],
// The GPS IFD's offset (0x8825), LONG, 26.
...[0x88, 0x25, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x1a],
...[0x00, 0x00, 0x00, 0x00],
// The GPS IFD, at 26: two entries, then no next IFD.
...[0x00, 0x02],
// GPSLatitudeRef (0x0001), 2 ASCII bytes: "S".
...[0x00, 0x01, 0x00, 0x02, 0x00, 0x00, 0x00, 0x02, 0x53, 0x00, 0x00, 0x00],
// GPSLatitude (0x0002), three RATIONALs at 56.
...[0x00, 0x02, 0x00, 0x05, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x38],
...[0x00, 0x00, 0x00, 0x00],
...[0x00, 0x00, 0x00, 0x21, 0x00, 0x00, 0x00, 0x01], // 33/1, at 56
...[0x00, 0x00, 0x00, 0x1e, 0x00, 0x00, 0x00, 0x01], // 30/1
...[0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01], // 0/1
];
// A big-endian TIFF block holding a GPSLatitude of 40° 26' 46" and a
// GPSLongitude of 79° 58' 56", and neither GPSLatitudeRef nor GPSLongitudeRef.
const TIFF_POSITION_WITHOUT_REFS = [
...[0x4d, 0x4d, 0x00, 0x2a, 0x00, 0x00, 0x00, 0x08], // the first IFD at 8
// The first IFD, at 8: one entry, then no next IFD.
...[0x00, 0x01],
// The GPS IFD's offset (0x8825), LONG, 26.
...[0x88, 0x25, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x1a],
...[0x00, 0x00, 0x00, 0x00],
// The GPS IFD, at 26: two entries, then no next IFD.
...[0x00, 0x02],
// GPSLatitude (0x0002), three RATIONALs at 56.
...[0x00, 0x02, 0x00, 0x05, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x38],
// GPSLongitude (0x0004), three RATIONALs at 80.
...[0x00, 0x04, 0x00, 0x05, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x50],
...[0x00, 0x00, 0x00, 0x00],
...[0x00, 0x00, 0x00, 0x28, 0x00, 0x00, 0x00, 0x01], // 40/1, at 56
...[0x00, 0x00, 0x00, 0x1a, 0x00, 0x00, 0x00, 0x01], // 26/1
...[0x00, 0x00, 0x00, 0x2e, 0x00, 0x00, 0x00, 0x01], // 46/1
...[0x00, 0x00, 0x00, 0x4f, 0x00, 0x00, 0x00, 0x01], // 79/1, at 80
...[0x00, 0x00, 0x00, 0x3a, 0x00, 0x00, 0x00, 0x01], // 58/1
...[0x00, 0x00, 0x00, 0x38, 0x00, 0x00, 0x00, 0x01], // 56/1
];
// A big-endian TIFF block holding Orientation 6 and a Make whose value lies // A big-endian TIFF block holding Orientation 6 and a Make whose value lies
// past the end of the file. // past the end of the file.
const TIFF_MAKE_PAST_END = [ const TIFF_MAKE_PAST_END = [
@@ -136,27 +87,6 @@ const TIFF_UNNAMED_TAG = [
...[0x00, 0x00, 0x00, 0x00], ...[0x00, 0x00, 0x00, 0x00],
]; ];
// A big-endian TIFF block holding Orientation 6, and a thumbnail IFD holding
// its own Orientation 1 and a 4-byte JPEG thumbnail.
const TIFF_WITH_THUMBNAIL = [
...[0x4d, 0x4d, 0x00, 0x2a, 0x00, 0x00, 0x00, 0x08], // the first IFD at 8
// The first IFD, at 8: one entry, then the thumbnail IFD at 26.
...[0x00, 0x01],
// Orientation (0x0112), SHORT, 6.
...[0x01, 0x12, 0x00, 0x03, 0x00, 0x00, 0x00, 0x01, 0x00, 0x06, 0x00, 0x00],
...[0x00, 0x00, 0x00, 0x1a],
// The thumbnail IFD, at 26: three entries, then no next IFD.
...[0x00, 0x03],
// Orientation (0x0112), SHORT, 1.
...[0x01, 0x12, 0x00, 0x03, 0x00, 0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x00],
// JPEGInterchangeFormat (0x0201), LONG: the thumbnail is at 68.
...[0x02, 0x01, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x44],
// JPEGInterchangeFormatLength (0x0202), LONG: 4 bytes.
...[0x02, 0x02, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x04],
...[0x00, 0x00, 0x00, 0x00],
...[0xff, 0xd8, 0xff, 0xd9], // the thumbnail, at 68: an empty JPEG
];
// An APP1 segment whose length field matches its data. // An APP1 segment whose length field matches its data.
const app1 = (data: number[]): number[] => { const app1 = (data: number[]): number[] => {
const len = data.length + 2; const len = data.length + 2;
@@ -166,90 +96,31 @@ const app1 = (data: number[]): number[] => {
const bytes = (...parts: number[][]): Uint8Array => const bytes = (...parts: number[][]): Uint8Array =>
new Uint8Array(parts.flat()); new Uint8Array(parts.flat());
describe("readAllExifTags", () => {
it("keys a tag exifreader has no name for by its number", () => {
const data = [...EXIF_HEADER, ...TIFF_UNNAMED_TAG];
expect(readAllExifTags(bytes(SOI, app1(data), SOS))).toStrictEqual({
"undefined-49152": {
id: 49152,
value: 7,
description: 7,
computed: 7,
},
});
});
it("puts the thumbnail's tags under Thumbnail, without its image", () => {
const input = bytes(
SOI,
app1([...EXIF_HEADER, ...TIFF_WITH_THUMBNAIL]),
SOS,
);
// exifreader finds the thumbnail's image.
expect(readExifTags(input)?.Thumbnail?.type).toBe("image/jpeg");
const tags = readAllExifTags(input);
expect(tags.Orientation?.value).toBe(6);
expect(Object.keys(tags.Thumbnail ?? {}).sort()).toEqual([
"JPEGInterchangeFormat",
"JPEGInterchangeFormatLength",
"Orientation",
]);
expect(tags.Thumbnail?.Orientation?.value).toBe(1);
expect(readPhotoExif(tags)).toStrictEqual({ orientation: 6 });
});
});
describe("readPhotoExif", () => { describe("readPhotoExif", () => {
it("reads the common fields of a valid JPEG", () => { it("reads the common fields of a valid JPEG", () => {
const data = [...EXIF_HEADER, ...TIFF_ORIENTATION_6]; const data = [...EXIF_HEADER, ...TIFF_ORIENTATION_6];
expect( expect(readPhotoExif(bytes(SOI, app1(data), SOS))).toStrictEqual({
readPhotoExif(readAllExifTags(bytes(SOI, app1(data), SOS))),
).toStrictEqual({
orientation: 6, orientation: 6,
}); });
}); });
it("gives no dateTimeOriginal for a DateTimeOriginal of 0000:00:00 00:00:00", () => { it("gives no dateTimeOriginal for a DateTimeOriginal of 0000:00:00 00:00:00", () => {
const data = [...EXIF_HEADER, ...TIFF_UNSET_DATE]; const data = [...EXIF_HEADER, ...TIFF_UNSET_DATE];
expect( expect(readPhotoExif(bytes(SOI, app1(data), SOS))).toStrictEqual({
readPhotoExif(readAllExifTags(bytes(SOI, app1(data), SOS))),
).toStrictEqual({
orientation: 6, orientation: 6,
}); });
}); });
it("reads a GPSAltitude without GPSAltitudeRef as above sea level", () => { it("reads a GPSAltitude without GPSAltitudeRef as above sea level", () => {
const data = [...EXIF_HEADER, ...TIFF_ALTITUDE_WITHOUT_REF]; const data = [...EXIF_HEADER, ...TIFF_ALTITUDE_WITHOUT_REF];
expect( expect(readPhotoExif(bytes(SOI, app1(data), SOS))).toStrictEqual({
readPhotoExif(readAllExifTags(bytes(SOI, app1(data), SOS))),
).toStrictEqual({
gpsAltitude: 12.5, gpsAltitude: 12.5,
}); });
}); });
it("reads a GPSLatitude with GPSLatitudeRef S as south of the equator", () => {
const data = [...EXIF_HEADER, ...TIFF_SOUTHERN_LATITUDE];
expect(
readPhotoExif(readAllExifTags(bytes(SOI, app1(data), SOS))),
).toStrictEqual({
gpsLatitude: -33.5,
});
});
it("gives no gpsLatitude or gpsLongitude without their reference tags", () => {
const data = [...EXIF_HEADER, ...TIFF_POSITION_WITHOUT_REFS];
const tags = readAllExifTags(bytes(SOI, app1(data), SOS));
// The position is read; only its hemisphere is unknown.
expect(tags.GPSLatitude?.computed).toStrictEqual([40, 26, 46]);
expect(tags.GPSLongitude?.computed).toStrictEqual([79, 58, 56]);
expect(readPhotoExif(tags)).toStrictEqual({});
});
it("gives no make for a Make whose value lies past the end of the file", () => { it("gives no make for a Make whose value lies past the end of the file", () => {
const data = [...EXIF_HEADER, ...TIFF_MAKE_PAST_END]; const data = [...EXIF_HEADER, ...TIFF_MAKE_PAST_END];
expect( expect(readPhotoExif(bytes(SOI, app1(data), SOS))).toStrictEqual({
readPhotoExif(readAllExifTags(bytes(SOI, app1(data), SOS))),
).toStrictEqual({
orientation: 6, orientation: 6,
}); });
}); });
@@ -304,9 +175,8 @@ describe("readPhotoExif", () => {
"an EXIF block that cannot be parsed", "an EXIF block that cannot be parsed",
bytes(SOI, app1([...EXIF_HEADER, 0x58, 0x58]), SOS), bytes(SOI, app1([...EXIF_HEADER, 0x58, 0x58]), SOS),
], ],
])("returns no tags and no fields for %s", (_, input) => { ])("returns no fields for %s", (_, input) => {
expect(readAllExifTags(input)).toStrictEqual({}); expect(readPhotoExif(input)).toStrictEqual({});
expect(readPhotoExif(readAllExifTags(input))).toStrictEqual({});
}); });
}); });
+9
View File
@@ -1883,6 +1883,15 @@ describe("downloadFile live photos", () => {
expect(readdirSync(t.dir).sort()).toEqual(["f.JPG", "f.bin"]); expect(readdirSync(t.dir).sort()).toEqual(["f.JPG", "f.bin"]);
}); });
it("replaces what was at the destination, such as an earlier ZIP of the two", async () => {
const t = setup(livePhotoZip(), livePhoto);
writeFileSync(t.outPath, livePhotoZip());
await t.run();
expect(readdirSync(t.dir).sort()).toEqual(["f.heic", "f.mov"]);
});
it("renames the image and then the video into place, each from its own temp file", async () => { it("renames the image and then the video into place, each from its own temp file", async () => {
const t = setup(livePhotoZip(), livePhoto); const t = setup(livePhotoZip(), livePhoto);
+20 -6
View File
@@ -29,7 +29,6 @@ import { downloadAlbums } from "../../examples/download-albums.js";
import { Library, type ContentSource } from "../../src/index.js"; import { Library, type ContentSource } from "../../src/index.js";
import type { CollectionsPage, FilesPage } from "../../src/client.js"; import type { CollectionsPage, FilesPage } from "../../src/client.js";
import type { Collection, EnteFile } from "../../src/model/types.js"; import type { Collection, EnteFile } from "../../src/model/types.js";
import { readAllExifTags } from "../../src/exif.js";
import { HEIC_WITH_EXIF } from "../exif-heic.js"; import { HEIC_WITH_EXIF } from "../exif-heic.js";
import { import {
asLivePhoto, asLivePhoto,
@@ -202,10 +201,11 @@ describe("examples/download-albums.ts", () => {
Buffer.from(VIDEO), Buffer.from(VIDEO),
); );
// The metadata is the photo's record and its EXIF tags. The originals // The metadata is the photo's record and its EXIF fields. The
// of photos 1 and 2 are not image data, so they have no EXIF tags. // originals of photos 1 and 2 are not image data, so they have no
// Photo 3's are every tag of its image, as `photo.exif()` returns // EXIF fields. Photo 3's image holds a camera, an exposure and a
// them. `record` holds the fields the three records share. // position, and the date it was taken is written as an ISO 8601
// string. `record` holds the fields the three records share.
const record = { const record = {
takenAt: TAKEN_MS, takenAt: TAKEN_MS,
modifiedAt: TAKEN_MS, modifiedAt: TAKEN_MS,
@@ -234,7 +234,21 @@ describe("examples/download-albums.ts", () => {
title: "file-3.jpg", title: "file-3.jpg",
fileType: "livePhoto", fileType: "livePhoto",
hash: livePhotoHash(HEIC_WITH_EXIF, VIDEO), hash: livePhotoHash(HEIC_WITH_EXIF, VIDEO),
exif: readAllExifTags(HEIC_WITH_EXIF), exif: {
make: "Canon",
model: "EOS R5",
lensModel: "RF50mm F1.8 STM",
dateTimeOriginal: "2021-07-15T14:30:00.000Z",
offsetTimeOriginal: "+02:00",
exposureTime: 1 / 250,
fNumber: 2.8,
iso: 400,
focalLength: 50,
orientation: 6,
gpsLatitude: 40 + 26 / 60 + 46 / 3600,
gpsLongitude: -(79 + 58 / 60 + 56 / 3600),
gpsAltitude: -12.5,
},
}); });
// Each album's photos, newest first, by save path relative to `dir`. // Each album's photos, newest first, by save path relative to `dir`.
+28 -103
View File
@@ -7,7 +7,7 @@
* a cached path shows up on the projected record. A library opened without a * a cached path shows up on the projected record. A library opened without a
* content source leaves those methods throwing rather than silently doing * content source leaves those methods throwing rather than silently doing
* nothing. It also covers a `Photo`'s `savePath`, `isLocal`, `download()`, * nothing. It also covers a `Photo`'s `savePath`, `isLocal`, `download()`,
* `content()`, `exif()` and the methods that each return one EXIF field. * `content()`, `exif()` and the methods that each return one field of `exif()`.
*/ */
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest"; import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
@@ -27,7 +27,7 @@ import { Library, type LibraryOptions } from "../../src/library/index.js";
import type { ContentSource } from "../../src/library/content.js"; import type { ContentSource } from "../../src/library/content.js";
import type { CollectionsPage, FilesPage } from "../../src/client.js"; import type { CollectionsPage, FilesPage } from "../../src/client.js";
import type { Collection, EnteFile } from "../../src/model/types.js"; import type { Collection, EnteFile } from "../../src/model/types.js";
import { readPhotoExif, type PhotoExif } from "../../src/exif.js"; import type { PhotoExif } from "../../src/exif.js";
import { HEIC_WITH_EXIF } from "../exif-heic.js"; import { HEIC_WITH_EXIF } from "../exif-heic.js";
import { import {
asLivePhoto, asLivePhoto,
@@ -176,7 +176,7 @@ describe("Library content wiring", () => {
await lib.close(); await lib.close();
}); });
it("does not take a live photo's image or video with no JSON file as its original when it opens, and precaches both", async () => { it("removes a live photo's ZIP an earlier version cached when it opens, and precaches its image and video", async () => {
const { file: live, body } = await asLivePhoto(file(1, 1)); const { file: live, body } = await asLivePhoto(file(1, 1));
class LiveClient extends MockClient { class LiveClient extends MockClient {
override async filesSince(): Promise<FilesPage> { override async filesSince(): Promise<FilesPage> {
@@ -197,8 +197,7 @@ describe("Library content wiring", () => {
// A first run records the library, so the next one knows that file 1 // A first run records the library, so the next one knows that file 1
// is a live photo when it opens the cache. // is a live photo when it opens the cache.
await (await open({})).close(); await (await open({})).close();
writeFileSync(join(originals, "1.heic"), "an image"); writeFileSync(join(originals, "1.jpg"), livePhotoZip());
writeFileSync(join(originals, "1.mov"), "a video");
let precached!: () => void; let precached!: () => void;
const done = new Promise<void>((r) => (precached = r)); const done = new Promise<void>((r) => (precached = r));
@@ -209,9 +208,7 @@ describe("Library content wiring", () => {
precached(); precached();
}, },
}); });
expect( expect(existsSync(join(originals, "1.jpg"))).toBe(false);
lib.photos.byID({ fileID: 1 })!.record().originalPath,
).toBeUndefined();
await done; await done;
expect(readdirSync(originals).sort()).toEqual([ expect(readdirSync(originals).sort()).toEqual([
@@ -219,17 +216,6 @@ describe("Library content wiring", () => {
"1.livephoto.json", "1.livephoto.json",
"1.mov", "1.mov",
]); ]);
expect(readFileSync(join(originals, "1.heic"))).toEqual(
Buffer.from(IMAGE),
);
expect(readFileSync(join(originals, "1.mov"))).toEqual(
Buffer.from(VIDEO),
);
expect(
JSON.parse(
readFileSync(join(originals, "1.livephoto.json"), "utf-8"),
),
).toEqual({ image: "1.heic", video: "1.mov" });
expect(lib.photos.byID({ fileID: 1 })!.record().originalPath).toBe( expect(lib.photos.byID({ fileID: 1 })!.record().originalPath).toBe(
join(originals, "1.heic"), join(originals, "1.heic"),
); );
@@ -273,10 +259,10 @@ const entry = (
value: number[], value: number[],
): number[] => [...u16(tag), ...u16(type), ...u32(count), ...value]; ): number[] => [...u16(tag), ...u16(type), ...u32(count), ...value];
// The TIFF block of a JPEG's EXIF segment, holding every field `Photo`'s typed // The TIFF block of a JPEG's EXIF segment, holding every field `exif()` picks:
// methods return: the camera in the first IFD, the exposure in the Exif IFD, // the camera in the first IFD, the exposure in the Exif IFD, and a GPS position
// and a GPS position of 40°26'46" N, 79°58'56" W, 12.5 m below sea level. // of 40°26'46" N, 79°58'56" W, 12.5 m below sea level. Offsets count from the
// Offsets count from the start of this block. // start of this block.
const TIFF = [ const TIFF = [
...[0x4d, 0x4d, 0x00, 0x2a], // big-endian TIFF ...[0x4d, 0x4d, 0x00, 0x2a], // big-endian TIFF
...u32(8), // the first IFD's offset ...u32(8), // the first IFD's offset
@@ -637,72 +623,9 @@ describe("Photo save path, local copy, content and EXIF", () => {
await lib.close(); await lib.close();
}); });
// Every tag in JPEG_WITH_EXIF, by name, in the order of its IFDs. it("reads the common EXIF fields of a JPEG original", async () => {
const jpegTags = [
"Make",
"Model",
"Orientation",
"Exif IFD Pointer",
"GPS Info IFD Pointer",
"ExposureTime",
"FNumber",
"ISOSpeedRatings",
"DateTimeOriginal",
"OffsetTimeOriginal",
"FocalLength",
"LensModel",
"GPSLatitudeRef",
"GPSLatitude",
"GPSLongitudeRef",
"GPSLongitude",
"GPSAltitudeRef",
"GPSAltitude",
];
// HEIC_WITH_EXIF holds those and the tags exiftool adds to every file.
const heicTags = [
...jpegTags,
"YCbCrPositioning",
"ExifVersion",
"ComponentsConfiguration",
"ColorSpace",
"GPSVersionID",
];
it.each([
[
"JPEG",
JPEG_WITH_EXIF,
jpegTags,
{
"Exif IFD Pointer": { value: 88 },
GPSLatitudeRef: { value: ["N"], description: "North latitude" },
},
],
[
"HEIC",
HEIC_WITH_EXIF,
heicTags,
{
ColorSpace: { value: 0xffff, description: "Uncalibrated" },
ExifVersion: { description: "0232" },
},
],
])(
"returns every EXIF tag of a %s original, by name",
async (_, bytes, names, others) => {
const lib = await open({ contentSource: stubSource(bytes) });
const exif = await lib.photos.byID({ fileID: 1 })!.exif();
expect(Object.keys(exif).sort()).toEqual([...names].sort());
// Tags outside the thirteen fields, as exifreader decodes them.
expect(exif).toMatchObject(others);
await lib.close();
},
);
it("picks the common EXIF fields from a JPEG original's tags", async () => {
const lib = await open({ contentSource: stubSource(JPEG_WITH_EXIF) }); const lib = await open({ contentSource: stubSource(JPEG_WITH_EXIF) });
const exif = await lib.photos.byID({ fileID: 1 })!.exif(); expect(await lib.photos.byID({ fileID: 1 })!.exif()).toStrictEqual({
expect(readPhotoExif(exif)).toStrictEqual({
make: "Canon", make: "Canon",
model: "EOS R5", model: "EOS R5",
lensModel: "RF50mm F1.8 STM", lensModel: "RF50mm F1.8 STM",
@@ -721,8 +644,8 @@ describe("Photo save path, local copy, content and EXIF", () => {
await lib.close(); await lib.close();
}); });
// The fields picked from HEIC_WITH_EXIF's tags, and from JPEG_WITH_EXIF's, // What exif() returns for HEIC_WITH_EXIF, and for JPEG_WITH_EXIF, which
// which hold the same values. // holds the same values.
const heicFields: PhotoExif = { const heicFields: PhotoExif = {
make: "Canon", make: "Canon",
model: "EOS R5", model: "EOS R5",
@@ -739,10 +662,11 @@ describe("Photo save path, local copy, content and EXIF", () => {
gpsAltitude: -12.5, gpsAltitude: -12.5,
}; };
it("picks the same common EXIF fields from a HEIC original's tags", async () => { it("reads the same common EXIF fields from a HEIC original", async () => {
const lib = await open({ contentSource: stubSource(HEIC_WITH_EXIF) }); const lib = await open({ contentSource: stubSource(HEIC_WITH_EXIF) });
const exif = await lib.photos.byID({ fileID: 1 })!.exif(); expect(await lib.photos.byID({ fileID: 1 })!.exif()).toStrictEqual(
expect(readPhotoExif(exif)).toStrictEqual(heicFields); heicFields,
);
await lib.close(); await lib.close();
}); });
@@ -756,27 +680,28 @@ describe("Photo save path, local copy, content and EXIF", () => {
client: new FilesClient([live]), client: new FilesClient([live]),
contentSource: cdnSource(new Map([[1, body]])), contentSource: cdnSource(new Map([[1, body]])),
}); });
const exif = await lib.photos.byID({ fileID: 1 })!.exif(); expect(await lib.photos.byID({ fileID: 1 })!.exif()).toStrictEqual(
expect(readPhotoExif(exif)).toStrictEqual(heicFields); heicFields,
);
await lib.close(); await lib.close();
}); });
// The build's type check, not this test, makes sure `Photo` has a method // The build's type check, not this test, makes sure `Photo` has a method
// for every `PhotoExif` field, whatever the fixtures hold: `Photo` // for every `PhotoExif` field, whatever the fixtures hold: `Photo`
// implements a type with one method per field. This test checks that each // implements a type with one method per field. This test checks that each
// method gives the field picked from the tags exif() returns. // method gives the same value as exif().
it.each([ it.each([
["JPEG", JPEG_WITH_EXIF], ["JPEG", JPEG_WITH_EXIF],
["HEIC", HEIC_WITH_EXIF], ["HEIC", HEIC_WITH_EXIF],
])( ])(
"has a method for each field, agreeing with the tags exif() returns, for a %s", "has a method for each field exif() returns, giving the same value, for a %s",
async (_, bytes) => { async (_, bytes) => {
const lib = await open({ contentSource: stubSource(bytes) }); const lib = await open({ contentSource: stubSource(bytes) });
const photo = lib.photos.byID({ fileID: 1 })!; const photo = lib.photos.byID({ fileID: 1 })!;
const fields = readPhotoExif(await photo.exif()); const exif = await photo.exif();
// The file holds every field, so every method is checked. // The file holds every field, so every method is checked.
expect(fields).toStrictEqual(heicFields); expect(exif).toStrictEqual(heicFields);
for (const [field, value] of Object.entries(fields)) { for (const [field, value] of Object.entries(exif)) {
expect(await photo[field as keyof PhotoExif]()).toStrictEqual( expect(await photo[field as keyof PhotoExif]()).toStrictEqual(
value, value,
); );
@@ -785,7 +710,7 @@ describe("Photo save path, local copy, content and EXIF", () => {
}, },
); );
it("returns no EXIF tags for an original that is not an image", async () => { it("returns no EXIF fields for an original that is not an image", async () => {
const lib = await open(); const lib = await open();
const photo = lib.photos.byID({ fileID: 1 })!; const photo = lib.photos.byID({ fileID: 1 })!;
expect(await photo.exif()).toStrictEqual({}); expect(await photo.exif()).toStrictEqual({});
@@ -793,7 +718,7 @@ describe("Photo save path, local copy, content and EXIF", () => {
await lib.close(); await lib.close();
}); });
it("returns no EXIF tags for a JPEG whose EXIF cannot be parsed", async () => { it("returns no EXIF fields for a JPEG whose EXIF cannot be parsed", async () => {
const lib = await open({ const lib = await open({
contentSource: stubSource(JPEG_WITH_BAD_EXIF), contentSource: stubSource(JPEG_WITH_BAD_EXIF),
}); });
@@ -814,7 +739,7 @@ describe("Photo save path, local copy, content and EXIF", () => {
await lib.close(); await lib.close();
}); });
it("returns no EXIF tags for a video, without fetching it", async () => { it("returns no EXIF fields for a video, without fetching it", async () => {
const video = file(1, 1); const video = file(1, 1);
video.metadata.fileType = "video"; video.metadata.fileType = "video";
const source = stubSource(JPEG_WITH_EXIF); const source = stubSource(JPEG_WITH_EXIF);
+37 -30
View File
@@ -566,6 +566,43 @@ describe("ContentCache live photos", () => {
expect(events).toEqual(["skipped"]); expect(events).toEqual(["skipped"]);
}); });
it("replaces a live photo an earlier version stored as a ZIP under the image's name", async () => {
const { file: live, body } = await asLivePhoto(file(5, "IMG_5.HEIC"));
mkdirSync(originals(), { recursive: true });
writeFileSync(join(originals(), "5.HEIC"), livePhotoZip());
const cache = cacheOf([live], new Map([[5, body]]));
// Opened without being told that file 5 is a live photo, the cache
// records the ZIP, and does not serve it.
await cache.open();
const result = await cache.original(5);
expect(result.videoPath).toBe(join(originals(), "5.mov"));
expect(readdirSync(originals()).sort()).toEqual([
"5.heic",
"5.livephoto.json",
"5.mov",
]);
});
it("removes a live photo's ZIP an earlier version stored when it opens, so the precache fetches the image and video", async () => {
const { file: live, body } = await asLivePhoto(file(5, "IMG_5.HEIC"));
mkdirSync(originals(), { recursive: true });
writeFileSync(join(originals(), "5.HEIC"), livePhotoZip());
const cache = cacheOf([live], new Map([[5, body]]));
await cache.open((fileID) => fileID === 5);
expect(readdirSync(originals())).toEqual([]);
expect(cache.pathsFor(5)).toEqual({});
const [fetched] = await cache.ensureOriginals({ fileIDs: [5] });
expect(fetched).toEqual({
fileID: 5,
path: join(originals(), "5.heic"),
});
expect(cache.pathsFor(5)).toEqual({ originalPath: fetched!.path });
});
it("leaves the image and video another process has just stored when it opens before their JSON file is written", async () => { it("leaves the image and video another process has just stored when it opens before their JSON file is written", async () => {
const { file: live, body } = await asLivePhoto(file(5, "IMG_5.HEIC")); const { file: live, body } = await asLivePhoto(file(5, "IMG_5.HEIC"));
const server = cdnSource(new Map([[5, body]])); const server = cdnSource(new Map([[5, body]]));
@@ -596,36 +633,6 @@ describe("ContentCache live photos", () => {
expect(second!.pathsFor(5)).toEqual({}); expect(second!.pathsFor(5)).toEqual({});
}); });
it("fetches a live photo's image and video again when the cache opened before knowing it is a live photo and no JSON file names them", async () => {
const { file: live, body } = await asLivePhoto(file(5, "IMG_5.HEIC"));
mkdirSync(originals(), { recursive: true });
writeFileSync(join(originals(), "5.heic"), "an image");
writeFileSync(join(originals(), "5.mov"), "a video");
const cache = cacheOf([live], new Map([[5, body]]));
// Opened without being told that file 5 is a live photo, the cache
// records one of the two files as its original, with no video.
await cache.open();
const events: string[] = [];
const result = await cache.original(5, {
onProgress: (e) => events.push(e.status),
});
expect(events.at(-1)).toBe("done");
expect(result).toEqual({
path: join(originals(), "5.heic"),
videoPath: join(originals(), "5.mov"),
bytes: IMAGE.length,
});
expect(readFileSync(result.path)).toEqual(Buffer.from(IMAGE));
expect(readFileSync(result.videoPath!)).toEqual(Buffer.from(VIDEO));
expect(
JSON.parse(
readFileSync(join(originals(), "5.livephoto.json"), "utf-8"),
),
).toEqual({ image: "5.heic", video: "5.mov" });
});
it.each(["missing", "empty"])( it.each(["missing", "empty"])(
"fetches a live photo again when the video its JSON file names is %s", "fetches a live photo again when the video its JSON file names is %s",
async (state) => { async (state) => {
+1 -14
View File
@@ -9,10 +9,8 @@
// Excluding too much: Prettier 3 reads `.gitignore` as a default ignore file, // Excluding too much: Prettier 3 reads `.gitignore` as a default ignore file,
// so dropping it from the context silently changes which files the lint // so dropping it from the context silently changes which files the lint
// phase's prettier check looks at compared to `make fmt-check` on the host. // phase's prettier check looks at compared to `make fmt-check` on the host.
// And without `.git`, a `docker build .` given no `VERSION` build arg cannot
// derive the version (`script/version`) and stamps `package.json`'s instead.
// //
// None of these shows up as a build failure, so they are asserted here. // Neither shows up as a build failure, so they are asserted here.
import { describe, expect, it } from "vitest"; import { describe, expect, it } from "vitest";
import { existsSync, readFileSync } from "node:fs"; import { existsSync, readFileSync } from "node:fs";
import { fileURLToPath } from "node:url"; import { fileURLToPath } from "node:url";
@@ -49,17 +47,6 @@ describe(".dockerignore", () => {
expect(dockerignore).not.toContain(".gitignore"); expect(dockerignore).not.toContain(".gitignore");
}); });
it("leaves .git in the build context for the version", () => {
expect(dockerignore).not.toContain(".git");
expect(dockerignore).not.toContain(".git/");
});
// The build stage is the final image, so a .git/config sent in would
// ship the clone's remote URL and any credential in it.
it("sends .git without its config", () => {
expect(dockerignore).toContain(".git/config");
});
// BuildKit lets a `Dockerfile.dockerignore` shadow the root one; such a // BuildKit lets a `Dockerfile.dockerignore` shadow the root one; such a
// file would silently give the build a different, unreviewed context — // file would silently give the build a different, unreviewed context —
// and eslint's flat config does not ignore dot-directories, so a stray // and eslint's flat config does not ignore dot-directories, so a stray
-137
View File
@@ -1,137 +0,0 @@
// `script/version` prints the version `script/build` stamps into
// `dist/package.json`, which is what `quak --version` reports from a build.
// A `docker build .` of a clone is given no `VERSION` build arg, so the
// version has to come from the `.git` in its context: the tag on a tagged
// commit; the tag, the commits since it and the short commit on a later commit;
// the short commit when no tag is reachable. A checkout with `.git` that still
// yields no usable version must fail the build, not ship a version nobody can
// trace back to its commit.
//
// Each test copies the script into a fresh directory, which the script then
// treats as the checkout, and executes it there.
import { afterEach, describe, expect, it } from "vitest";
import { execFileSync, spawnSync } from "node:child_process";
import {
chmodSync,
copyFileSync,
mkdirSync,
mkdtempSync,
rmSync,
writeFileSync,
} from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
const repoRoot = fileURLToPath(new URL("../../", import.meta.url));
let checkout = "";
afterEach(() => {
rmSync(checkout, { recursive: true, force: true });
});
// A checkout holding the script and a package.json that declares 0.0.0, with
// no .git yet.
const makeCheckout = (): void => {
checkout = mkdtempSync(join(tmpdir(), "quak-version-"));
mkdirSync(join(checkout, "script"));
copyFileSync(
join(repoRoot, "script/version"),
join(checkout, "script/version"),
);
chmodSync(join(checkout, "script/version"), 0o755);
writeFileSync(join(checkout, "package.json"), '{ "version": "0.0.0" }\n');
};
// git in the checkout, with an identity and no commit signing, whatever the
// host's own git config says.
const git = (...args: string[]): string =>
execFileSync(
"git",
[
"-c",
"user.name=quak",
"-c",
"user.email=quak@example.invalid",
"-c",
"commit.gpgsign=false",
...args,
],
{ cwd: checkout, encoding: "utf-8", stdio: ["ignore", "pipe", "pipe"] },
).trim();
const makeCommittedCheckout = (): void => {
makeCheckout();
git("init", "-q");
git("add", "package.json");
git("commit", "-q", "-m", "first");
};
// Runs the script with nothing in its environment but PATH and, when given,
// VERSION.
const runVersion = (version?: string) =>
spawnSync(join(checkout, "script/version"), {
cwd: checkout,
encoding: "utf-8",
env: { PATH: process.env.PATH, VERSION: version },
});
describe("script/version", () => {
it("prints the short commit of an untagged commit", () => {
makeCommittedCheckout();
expect(runVersion().stdout.trim()).toBe(
git("rev-parse", "--short", "HEAD"),
);
});
it("prints the tag of a tagged commit", () => {
makeCommittedCheckout();
git("tag", "v1.2.3");
expect(runVersion().stdout.trim()).toBe("v1.2.3");
});
// script/docker and script/cibuild pass the version they resolve on the
// host as the VERSION build arg.
it("prints the VERSION it is given over what git would derive", () => {
makeCommittedCheckout();
expect(runVersion("x").stdout.trim()).toBe("x");
});
// `--build-arg VERSION=` must not stamp an empty version.
it("treats an empty VERSION as unset", () => {
makeCommittedCheckout();
expect(runVersion("").stdout.trim()).toBe(
git("rev-parse", "--short", "HEAD"),
);
});
// A source tarball has no .git: it keeps the version package.json
// declares, and must still build.
it("prints package.json's version where there is no .git", () => {
makeCheckout();
const result = runVersion();
expect(result.status).toBe(0);
expect(result.stdout.trim()).toBe("0.0.0");
});
// A repository with no commits stands in for any .git that git cannot
// describe: git missing from the image, or refusing to read the checkout.
it("fails where .git yields no version", () => {
makeCheckout();
git("init", "-q");
const result = runVersion();
expect(result.status).not.toBe(0);
expect(result.stdout).toBe("");
});
it.each(["dev", "unknown"])(
"fails where there is .git and the version is %s",
(version) => {
makeCommittedCheckout();
const result = runVersion(version);
expect(result.status).not.toBe(0);
expect(result.stdout).toBe("");
},
);
});
+4 -5
View File
@@ -787,13 +787,12 @@ describe("fixMissingThumbnails", () => {
}); });
it("re-encodes smaller until the thumbnail fits the recorded size", async () => { it("re-encodes smaller until the thumbnail fits the recorded size", async () => {
// A noisy 64x48 JPEG, which the default encoding (quality 50, not // A noisy 400x300 JPEG, which the default encoding (quality 50, not
// resized because it is under 720 px) cannot compress below the size // resized because it is under 720 px) cannot compress below the size
// recorded here: one byte less than that encoding's ciphertext. It is // recorded here: one byte less than that encoding's ciphertext.
// small so that each encode is quick even on a busy host.
const fixMock = await buildThumbMock(); const fixMock = await buildThumbMock();
const w = 64; const w = 400;
const h = 48; const h = 300;
const noisy = new Uint8Array( const noisy = new Uint8Array(
jpegJs.encode( jpegJs.encode(
{ {