Compare commits

..
2 Commits
Author SHA1 Message Date
clawbot 9e94542a57 docker build . stamps the git tag or short commit, not dev (closes #154)
check / check (push) Failing after 46s
A plain `docker build .` now stamps the version from git rather than `dev`/`0.0.0`. After `tsc`, `script/build` writes into `dist/package.json` the version `script/version` decides: `VERSION` when given, otherwise `git describe --tags --always` (the tag; or tag, commits since and short commit; or the short commit), otherwise `package.json`'s. A checkout with `.git` that yields an empty, `dev` or `unknown` version fails the build. `.dockerignore` sends `.git` but not `.git/config`, so no remote URL or credential reaches the image. `ARG VERSION` has no default, and the host scripts' version still wins.

Not changed: `REPO_POLICIES.md` still says `ARG VERSION=dev` until the shared policy changes.

Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
2026-10-02 06:12:56 +02:00
clawbot e50d2a78c8 exif() returns every EXIF tag in the file (closes #156)
check / check (push) Failing after 59s
`photo.exif()` now returns every EXIF tag in the file, as the owner ruled, typed `ExifTags`: each tag keyed by name with `exifreader`'s `id`, `value`, `description` and `computed`. A tag with no name is keyed `undefined-` plus its number, and the embedded thumbnail's tags sit under `Thumbnail`. The thirteen typed methods stay, with the same names and types; each now picks its field from `exif()`'s tags. GPS latitude and longitude are worked out from the GPS tags and their reference tags, and a position with no reference tags gives neither. `backup-metadata --exif` output is unchanged.

Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
2026-10-02 05:30:05 +02:00
10 changed files with 377 additions and 129 deletions
+21 -15
View File
@@ -101,7 +101,8 @@ requires one, and writes:
photo its image, its video and the `.livephoto.json` file naming them
- beside each original, a JSON file named after it with `.json` added, for
example `2026-03-01.12345.jpg.json`: the photo's record (`photo.record()`)
without its cache paths, and its EXIF fields (`photo.exif()`) under `exif`
without its cache paths, and every EXIF tag of the photo (`photo.exif()`)
under `exif`
- `albums/<collectionID>.json` for each album: its `collectionID`, its `name`,
and under `savePaths` the save paths of its photos relative to `dir`, newest
first
@@ -188,7 +189,8 @@ places, and reads `.gitignore` as its default ignore file — which is why
- the `VERSION` environment variable, or the `Dockerfile`'s `VERSION` build arg
(`--build-arg VERSION=...`), when one is given and not empty;
- otherwise, in a checkout with `.git`, `git describe --tags --always`: the tag
on a tagged commit, otherwise the short commit;
on a tagged commit; the tag, the commits since it and the short commit on a
later commit (`v1.2.3-4-gabc1234`); the short commit when no tag is reachable;
- otherwise, as in a source tarball, the version `package.json` declares.
The build fails if the checkout has `.git` and the version still comes out
@@ -803,21 +805,25 @@ These async methods may download:
- `await photo.thumbnail(opts?)` → `{ path, bytes }`.
- `await photo.content(opts?)` → `Uint8Array` — the original's bytes, read
through `original()`; for a live photo, its image's.
- `await photo.exif(opts?)` → `PhotoExif` — `make`, `model`, `lensModel`,
`dateTimeOriginal`, `offsetTimeOriginal`, `exposureTime`, `fNumber`, `iso`,
`focalLength`, `orientation`, `gpsLatitude`, `gpsLongitude` and `gpsAltitude`,
each absent when the file lacks it. GPS values are signed decimal degrees and
metres. `dateTimeOriginal` is the camera's clock reading held in the `Date`'s
UTC fields; `offsetTimeOriginal`, when present, is that clock's offset from
UTC. EXIF is read from any image format exifreader reads (such as JPEG,
HEIC/HEIF, AVIF, PNG, WebP and TIFF), a live photo's image included. Any other
original gives `{}`, and a video gives `{}` without being downloaded.
- `await photo.exif(opts?)` → `ExifTags` — every EXIF tag in the file, keyed by
tag name, each as exifreader decodes it, with its `id`, `value`, `description`
and `computed` value: for example `Make` is
`{ id: 271, value: ["Canon"], description: "Canon", computed: "Canon" }`. A
tag exifreader has no name for is keyed `undefined-<tag number>`. The embedded
thumbnail's tags are under `Thumbnail`, so they cannot hide the main image's
tags of the same name; the thumbnail image itself is left out. EXIF is read
from any image format exifreader reads (such as JPEG, HEIC/HEIF, AVIF, PNG,
WebP and TIFF), a live photo's image included. Any other original gives `{}`,
and a video gives `{}` without being downloaded.
- `await photo.make(opts?)`, and likewise `model()`, `lensModel()`,
`dateTimeOriginal()`, `offsetTimeOriginal()`, `exposureTime()`, `fNumber()`,
`iso()`, `focalLength()`, `orientation()`, `gpsLatitude()`, `gpsLongitude()`
and `gpsAltitude()` → one field of `exif()` each, typed as in `PhotoExif`, or
`undefined` when the file lacks it. Each calls `exif()` with its `opts`, so
each call reads the original again.
and `gpsAltitude()` → one common field each, picked from the tags `exif()`
returns and typed as in `PhotoExif`, or `undefined` when the file lacks it.
GPS values are signed decimal degrees and metres. `dateTimeOriginal()` is the
camera's clock reading held in the `Date`'s UTC fields;
`offsetTimeOriginal()`, when present, is that clock's offset from UTC. Each
calls `exif()` with its `opts`, so each call reads the original again.
They serve from the on-disk content cache when the bytes are present and
otherwise fetch through the pools; `original()`, `content()` and `exif()` also
@@ -934,7 +940,7 @@ from a very old client, is stored unchecked.
- `src/library/records.ts`: `PhotoRecord`, `AlbumRecord`, `LibrarySnapshot`,
`LibraryChange`
- `src/library/mlsearch.ts`: `MLDataAPI`, `SimilarResult`
- `src/exif.ts`: `PhotoExif`
- `src/exif.ts`: `ExifTags`, `PhotoExif`
- `src/library/pools.ts`: `RequestPools`, `RequestPoolsOptions`, `BoundedPool`
- `src/backup.ts`: `BackupOptions`, `BackupResult`, `BackupError`
- `src/client.ts`: `Client`, `LoginOptions`, `ClientSnapshot`
+7
View File
@@ -34,6 +34,13 @@ declares one.
`make build-bin` bundles the built `dist/`, so the single binary reports the
same version.
- 2026-10-02: `photo.exif()` returns every EXIF tag in the file as `ExifTags`,
keyed by tag name, each as exifreader decodes it, not only the thirteen common
fields (issue 156). The embedded thumbnail's tags are under `Thumbnail`,
without the thumbnail image. The thirteen typed methods stay, each picking its
field from the tags `exif()` returns, typed as in `PhotoExif`. The example
script's JSON files now carry every tag.
- 2026-10-01: The content cache no longer looks for a live photo that an earlier
version cached as one ZIP (issue 151). When the cache opens, a live photo's
file that no JSON file names is now always left alone.
+3 -1
View File
@@ -7,7 +7,9 @@
# 1. $VERSION, if set and not empty: an explicit value, such as the
# Dockerfile's VERSION build arg.
# 2. If this checkout has .git, `git describe --tags --always`: the tag
# on a tagged commit, otherwise the short commit.
# on a tagged commit; the tag, the commits since it and the short
# commit on a later commit (v1.2.3-4-gabc1234); the short commit when
# no tag is reachable.
# 3. Otherwise, as in a source tarball, the version package.json declares.
#
# A checkout with .git whose version still comes out empty, dev or unknown
+80 -31
View File
@@ -1,19 +1,19 @@
// EXIF in an original's bytes, read with exifreader, which reads it from JPEG,
// HEIC/HEIF, AVIF, PNG, WebP and the other image formats it supports.
// `backup-metadata --exif` records every EXIF tag it finds except the
// thumbnail's; `Photo.exif()` returns the common fields picked from them here.
// thumbnail's. `Photo.exif()` returns every tag, the thumbnail's included, and
// `Photo`'s typed methods return the common fields picked from them here.
import ExifReader, { type ExpandedTags } from "exifreader";
// The EXIF tags in `bytes` (`exif`), the GPS position exifreader computes from
// them (`gps`), and where the EXIF block lies in `bytes` (`metadataRange`).
// The EXIF tags in `bytes` (`exif`), the embedded thumbnail's tags
// (`Thumbnail`), and where the EXIF block lies in `bytes` (`metadataRange`).
// Undefined when exifreader cannot read the file at all, such as a video. An
// EXIF block it finds but reads no tag from comes back as an empty `exif`.
// `exif` holds every tag except the thumbnail's; a tag exifreader has no name
// for is keyed `undefined-<tag number>`. Each tag's `computed` holds its value
// as a string or number, or as an array of them for a tag with several values,
// such as `GPSLatitude`'s `[40, 26, 46]`. A fraction with a zero denominator
// computes to null.
// EXIF block it finds but reads no tag from comes back as an empty `exif`. A
// tag exifreader has no name for is keyed `undefined-<tag number>`. Each tag's
// `computed` holds its value as a string or number, or as an array of them for
// a tag with several values, such as `GPSLatitude`'s `[40, 26, 46]`. A
// fraction with a zero denominator computes to null.
export const readExifTags = (bytes: Uint8Array): ExpandedTags | undefined => {
try {
return ExifReader.loadView(
@@ -23,7 +23,7 @@ export const readExifTags = (bytes: Uint8Array): ExpandedTags | undefined => {
computed: true,
includeOffsets: true,
includeUnknown: true,
includeTags: { exif: true, gps: true },
includeTags: { exif: true, thumbnail: true },
},
);
} catch {
@@ -31,7 +31,35 @@ export const readExifTags = (bytes: Uint8Array): ExpandedTags | undefined => {
}
};
// The common EXIF fields of an original. Each is absent when the file lacks it.
// Every EXIF tag of an original, keyed by name, each as exifreader decodes it
// (see `readExifTags`). The embedded thumbnail's own tags are under
// `Thumbnail`, so its `Orientation` or `ImageWidth` cannot hide the main
// image's.
export type ExifTags = Omit<NonNullable<ExpandedTags["exif"]>, "Thumbnail"> & {
Thumbnail?: Omit<
NonNullable<ExpandedTags["Thumbnail"]>,
"type" | "image" | "base64"
>;
};
// Every EXIF tag in `bytes`: `{}` when the file has no EXIF, exifreader cannot
// read its EXIF, or it is not an image exifreader reads.
export const readAllExifTags = (bytes: Uint8Array): ExifTags => {
const tags = readExifTags(bytes);
if (!tags?.Thumbnail) return tags?.exif ?? {};
// exifreader puts the thumbnail's JPEG image beside its tags, as `type`,
// `image` and `base64`. The image is not a tag, so it is left out.
const {
type: _type,
image: _image,
base64: _base64,
...thumbnail
} = tags.Thumbnail;
return { ...tags.exif, Thumbnail: thumbnail };
};
// The common EXIF fields of an original, one for each of `Photo`'s typed
// methods. Each is absent when the file lacks it.
export interface PhotoExif {
make?: string;
model?: string;
@@ -80,30 +108,51 @@ const asDate = (v: unknown): Date | undefined => {
return Number.isNaN(date.getTime()) ? undefined : date;
};
// The common fields of an original's EXIF: `{}` when the file has no EXIF,
// exifreader cannot read its EXIF, or it is not an image exifreader reads.
export const readPhotoExif = (bytes: Uint8Array): PhotoExif => {
const tags = readExifTags(bytes);
const exif = tags?.exif;
const gps = tags?.gps;
const altitude = asNumber(exif?.GPSAltitude?.computed);
// GPSLatitude and GPSLongitude hold degrees, minutes and seconds, computed as
// three numbers. This is them in decimal degrees, negative when `ref`, the
// GPSLatitudeRef or GPSLongitudeRef tag, is `negativeRef` ("S" or "W").
// Without that tag the hemisphere is unknown, so it is undefined.
const asDegrees = (
dms: unknown,
ref: unknown,
negativeRef: string,
): number | undefined => {
if (!Array.isArray(dms) || ref === undefined) return undefined;
const [d, m, s] = dms.map(asNumber);
if (d === undefined || m === undefined || s === undefined) return undefined;
const degrees = d + m / 60 + s / 3600;
return ref === negativeRef ? -degrees : degrees;
};
// The common fields picked from an original's EXIF tags, `readAllExifTags`'s
// result: `{}` when there are none.
export const readPhotoExif = (tags: ExifTags): PhotoExif => {
const altitude = asNumber(tags.GPSAltitude?.computed);
const fields: PhotoExif = {
make: asString(exif?.Make?.computed),
model: asString(exif?.Model?.computed),
lensModel: asString(exif?.LensModel?.computed),
dateTimeOriginal: asDate(exif?.DateTimeOriginal?.computed),
offsetTimeOriginal: asString(exif?.OffsetTimeOriginal?.computed),
exposureTime: asNumber(exif?.ExposureTime?.computed),
fNumber: asNumber(exif?.FNumber?.computed),
make: asString(tags.Make?.computed),
model: asString(tags.Model?.computed),
lensModel: asString(tags.LensModel?.computed),
dateTimeOriginal: asDate(tags.DateTimeOriginal?.computed),
offsetTimeOriginal: asString(tags.OffsetTimeOriginal?.computed),
exposureTime: asNumber(tags.ExposureTime?.computed),
fNumber: asNumber(tags.FNumber?.computed),
// Only when the tag holds a single number, as most cameras write it.
iso: asNumber(exif?.ISOSpeedRatings?.computed),
focalLength: asNumber(exif?.FocalLength?.computed),
orientation: asNumber(exif?.Orientation?.computed),
gpsLatitude: asNumber(gps?.Latitude),
gpsLongitude: asNumber(gps?.Longitude),
iso: asNumber(tags.ISOSpeedRatings?.computed),
focalLength: asNumber(tags.FocalLength?.computed),
orientation: asNumber(tags.Orientation?.computed),
gpsLatitude: asDegrees(
tags.GPSLatitude?.computed,
tags.GPSLatitudeRef?.computed,
"S",
),
gpsLongitude: asDegrees(
tags.GPSLongitude?.computed,
tags.GPSLongitudeRef?.computed,
"W",
),
// A GPSAltitudeRef of 1 means the altitude is below sea level.
gpsAltitude:
altitude !== undefined && exif?.GPSAltitudeRef?.value === 1
altitude !== undefined && tags.GPSAltitudeRef?.value === 1
? -altitude
: altitude,
};
+1 -1
View File
@@ -87,7 +87,7 @@ export type {
LibrarySnapshot,
LibraryChange,
} from "./library/records.js";
export type { PhotoExif } from "./exif.js";
export type { ExifTags, PhotoExif } from "./exif.js";
export { decryptCollection, decryptFile } from "./model/index.js";
export { downloadFile, downloadThumbnail } from "./download/index.js";
export type {
+31 -25
View File
@@ -13,14 +13,19 @@
//
// A `Photo` also fetches its own bytes: `original()`, `thumbnail()`,
// `download()`, `content()`, `exif()` and the methods that each return one
// field of `exif()` go through the on-disk content cache (issue #46), and are
// EXIF field go through the on-disk content cache (issue #46), and are
// the one place in this module that may touch the network. A library opened
// without a content source leaves that cache absent, and those methods then
// throw. `savePath` and `isLocal` look only at the disk and need no cache.
import { readFile } from "node:fs/promises";
import { readPhotoExif, type PhotoExif } from "../exif.js";
import {
readAllExifTags,
readPhotoExif,
type ExifTags,
type PhotoExif,
} from "../exif.js";
import type { CollectionType, EnteFile, FileType } from "../model/types.js";
import type { ContentOptions, ContentResult, PhotoContent } from "./content.js";
import type { AlbumRecord, PhotoRecord, DerivedRecords } from "./records.js";
@@ -156,74 +161,75 @@ export class Photo implements PhotoExifMethods {
return readFile(path);
}
// The common EXIF fields of the original, read from `content()`, so this
// may download it. EXIF is read from any image format exifreader reads,
// JPEG and HEIC/HEIF among them; any other file gives `{}`, and a video
// gives it without fetching anything. Like the other content methods, it
// throws when there is no content cache, video or not.
async exif(opts?: ContentOptions): Promise<PhotoExif> {
// Every EXIF tag of the original, keyed by name (see `ExifTags`), read
// from `content()`, so this may download it. EXIF is read from any image
// format exifreader reads, JPEG and HEIC/HEIF among them; any other file
// gives `{}`, and a video gives it without fetching anything. Like the
// other content methods, it throws when there is no content cache, video
// or not.
async exif(opts?: ContentOptions): Promise<ExifTags> {
this.cacheOrThrow();
if (this.rec.fileType === "video") return {};
return readPhotoExif(await this.content(opts));
return readAllExifTags(await this.content(opts));
}
// One field of `exif()` each, named and typed as in `PhotoExif`, and
// undefined when the file lacks it. Each call runs `exif()`, which reads
// the original again.
// One field each, named and typed as in `PhotoExif`, picked from the tags
// `exif()` returns, and undefined when the file lacks it. Each call runs
// `exif()`, which reads the original again.
async make(opts?: ContentOptions): Promise<PhotoExif["make"]> {
return (await this.exif(opts)).make;
return readPhotoExif(await this.exif(opts)).make;
}
async model(opts?: ContentOptions): Promise<PhotoExif["model"]> {
return (await this.exif(opts)).model;
return readPhotoExif(await this.exif(opts)).model;
}
async lensModel(opts?: ContentOptions): Promise<PhotoExif["lensModel"]> {
return (await this.exif(opts)).lensModel;
return readPhotoExif(await this.exif(opts)).lensModel;
}
async dateTimeOriginal(
opts?: ContentOptions,
): Promise<PhotoExif["dateTimeOriginal"]> {
return (await this.exif(opts)).dateTimeOriginal;
return readPhotoExif(await this.exif(opts)).dateTimeOriginal;
}
async offsetTimeOriginal(
opts?: ContentOptions,
): Promise<PhotoExif["offsetTimeOriginal"]> {
return (await this.exif(opts)).offsetTimeOriginal;
return readPhotoExif(await this.exif(opts)).offsetTimeOriginal;
}
async exposureTime(
opts?: ContentOptions,
): Promise<PhotoExif["exposureTime"]> {
return (await this.exif(opts)).exposureTime;
return readPhotoExif(await this.exif(opts)).exposureTime;
}
async fNumber(opts?: ContentOptions): Promise<PhotoExif["fNumber"]> {
return (await this.exif(opts)).fNumber;
return readPhotoExif(await this.exif(opts)).fNumber;
}
async iso(opts?: ContentOptions): Promise<PhotoExif["iso"]> {
return (await this.exif(opts)).iso;
return readPhotoExif(await this.exif(opts)).iso;
}
async focalLength(
opts?: ContentOptions,
): Promise<PhotoExif["focalLength"]> {
return (await this.exif(opts)).focalLength;
return readPhotoExif(await this.exif(opts)).focalLength;
}
async orientation(
opts?: ContentOptions,
): Promise<PhotoExif["orientation"]> {
return (await this.exif(opts)).orientation;
return readPhotoExif(await this.exif(opts)).orientation;
}
async gpsLatitude(
opts?: ContentOptions,
): Promise<PhotoExif["gpsLatitude"]> {
return (await this.exif(opts)).gpsLatitude;
return readPhotoExif(await this.exif(opts)).gpsLatitude;
}
async gpsLongitude(
opts?: ContentOptions,
): Promise<PhotoExif["gpsLongitude"]> {
return (await this.exif(opts)).gpsLongitude;
return readPhotoExif(await this.exif(opts)).gpsLongitude;
}
async gpsAltitude(
opts?: ContentOptions,
): Promise<PhotoExif["gpsAltitude"]> {
return (await this.exif(opts)).gpsAltitude;
return readPhotoExif(await this.exif(opts)).gpsAltitude;
}
private cacheOrThrow(): PhotoContent {
+138 -8
View File
@@ -3,14 +3,18 @@
* `quak backup-metadata --exif` records.
*
* The originals come from users' libraries, so a truncated or corrupt file
* must neither hang the read nor throw out of it: `readPhotoExif` gives `{}`,
* must neither hang the read nor throw out of it: `readAllExifTags` gives `{}`,
* and `backup-metadata` tells an EXIF block it cannot read apart from a file
* that simply has no EXIF, carrying the reason in `exifError`. Each JPEG below
* is a short hand-built byte array; the HEIC is a real file.
*/
import { describe, expect, it } from "vitest";
import { readPhotoExif } from "../../src/exif.js";
import {
readAllExifTags,
readExifTags,
readPhotoExif,
} from "../../src/exif.js";
import { extractImageMetadata } from "../../src/metadata-backup.js";
import { HEIC_WITH_EXIF } from "../exif-heic.js";
@@ -63,6 +67,51 @@ const TIFF_ALTITUDE_WITHOUT_REF = [
...[0x00, 0x00, 0x00, 0x19, 0x00, 0x00, 0x00, 0x02], // 25/2, at 44
];
// A big-endian TIFF block holding a GPSLatitude of 33° 30' 0" and a
// GPSLatitudeRef of "S".
const TIFF_SOUTHERN_LATITUDE = [
...[0x4d, 0x4d, 0x00, 0x2a, 0x00, 0x00, 0x00, 0x08], // the first IFD at 8
// The first IFD, at 8: one entry, then no next IFD.
...[0x00, 0x01],
// The GPS IFD's offset (0x8825), LONG, 26.
...[0x88, 0x25, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x1a],
...[0x00, 0x00, 0x00, 0x00],
// The GPS IFD, at 26: two entries, then no next IFD.
...[0x00, 0x02],
// GPSLatitudeRef (0x0001), 2 ASCII bytes: "S".
...[0x00, 0x01, 0x00, 0x02, 0x00, 0x00, 0x00, 0x02, 0x53, 0x00, 0x00, 0x00],
// GPSLatitude (0x0002), three RATIONALs at 56.
...[0x00, 0x02, 0x00, 0x05, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x38],
...[0x00, 0x00, 0x00, 0x00],
...[0x00, 0x00, 0x00, 0x21, 0x00, 0x00, 0x00, 0x01], // 33/1, at 56
...[0x00, 0x00, 0x00, 0x1e, 0x00, 0x00, 0x00, 0x01], // 30/1
...[0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01], // 0/1
];
// A big-endian TIFF block holding a GPSLatitude of 40° 26' 46" and a
// GPSLongitude of 79° 58' 56", and neither GPSLatitudeRef nor GPSLongitudeRef.
const TIFF_POSITION_WITHOUT_REFS = [
...[0x4d, 0x4d, 0x00, 0x2a, 0x00, 0x00, 0x00, 0x08], // the first IFD at 8
// The first IFD, at 8: one entry, then no next IFD.
...[0x00, 0x01],
// The GPS IFD's offset (0x8825), LONG, 26.
...[0x88, 0x25, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x1a],
...[0x00, 0x00, 0x00, 0x00],
// The GPS IFD, at 26: two entries, then no next IFD.
...[0x00, 0x02],
// GPSLatitude (0x0002), three RATIONALs at 56.
...[0x00, 0x02, 0x00, 0x05, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x38],
// GPSLongitude (0x0004), three RATIONALs at 80.
...[0x00, 0x04, 0x00, 0x05, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x50],
...[0x00, 0x00, 0x00, 0x00],
...[0x00, 0x00, 0x00, 0x28, 0x00, 0x00, 0x00, 0x01], // 40/1, at 56
...[0x00, 0x00, 0x00, 0x1a, 0x00, 0x00, 0x00, 0x01], // 26/1
...[0x00, 0x00, 0x00, 0x2e, 0x00, 0x00, 0x00, 0x01], // 46/1
...[0x00, 0x00, 0x00, 0x4f, 0x00, 0x00, 0x00, 0x01], // 79/1, at 80
...[0x00, 0x00, 0x00, 0x3a, 0x00, 0x00, 0x00, 0x01], // 58/1
...[0x00, 0x00, 0x00, 0x38, 0x00, 0x00, 0x00, 0x01], // 56/1
];
// A big-endian TIFF block holding Orientation 6 and a Make whose value lies
// past the end of the file.
const TIFF_MAKE_PAST_END = [
@@ -87,6 +136,27 @@ const TIFF_UNNAMED_TAG = [
...[0x00, 0x00, 0x00, 0x00],
];
// A big-endian TIFF block holding Orientation 6, and a thumbnail IFD holding
// its own Orientation 1 and a 4-byte JPEG thumbnail.
const TIFF_WITH_THUMBNAIL = [
...[0x4d, 0x4d, 0x00, 0x2a, 0x00, 0x00, 0x00, 0x08], // the first IFD at 8
// The first IFD, at 8: one entry, then the thumbnail IFD at 26.
...[0x00, 0x01],
// Orientation (0x0112), SHORT, 6.
...[0x01, 0x12, 0x00, 0x03, 0x00, 0x00, 0x00, 0x01, 0x00, 0x06, 0x00, 0x00],
...[0x00, 0x00, 0x00, 0x1a],
// The thumbnail IFD, at 26: three entries, then no next IFD.
...[0x00, 0x03],
// Orientation (0x0112), SHORT, 1.
...[0x01, 0x12, 0x00, 0x03, 0x00, 0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x00],
// JPEGInterchangeFormat (0x0201), LONG: the thumbnail is at 68.
...[0x02, 0x01, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x44],
// JPEGInterchangeFormatLength (0x0202), LONG: 4 bytes.
...[0x02, 0x02, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x04],
...[0x00, 0x00, 0x00, 0x00],
...[0xff, 0xd8, 0xff, 0xd9], // the thumbnail, at 68: an empty JPEG
];
// An APP1 segment whose length field matches its data.
const app1 = (data: number[]): number[] => {
const len = data.length + 2;
@@ -96,31 +166,90 @@ const app1 = (data: number[]): number[] => {
const bytes = (...parts: number[][]): Uint8Array =>
new Uint8Array(parts.flat());
describe("readAllExifTags", () => {
it("keys a tag exifreader has no name for by its number", () => {
const data = [...EXIF_HEADER, ...TIFF_UNNAMED_TAG];
expect(readAllExifTags(bytes(SOI, app1(data), SOS))).toStrictEqual({
"undefined-49152": {
id: 49152,
value: 7,
description: 7,
computed: 7,
},
});
});
it("puts the thumbnail's tags under Thumbnail, without its image", () => {
const input = bytes(
SOI,
app1([...EXIF_HEADER, ...TIFF_WITH_THUMBNAIL]),
SOS,
);
// exifreader finds the thumbnail's image.
expect(readExifTags(input)?.Thumbnail?.type).toBe("image/jpeg");
const tags = readAllExifTags(input);
expect(tags.Orientation?.value).toBe(6);
expect(Object.keys(tags.Thumbnail ?? {}).sort()).toEqual([
"JPEGInterchangeFormat",
"JPEGInterchangeFormatLength",
"Orientation",
]);
expect(tags.Thumbnail?.Orientation?.value).toBe(1);
expect(readPhotoExif(tags)).toStrictEqual({ orientation: 6 });
});
});
describe("readPhotoExif", () => {
it("reads the common fields of a valid JPEG", () => {
const data = [...EXIF_HEADER, ...TIFF_ORIENTATION_6];
expect(readPhotoExif(bytes(SOI, app1(data), SOS))).toStrictEqual({
expect(
readPhotoExif(readAllExifTags(bytes(SOI, app1(data), SOS))),
).toStrictEqual({
orientation: 6,
});
});
it("gives no dateTimeOriginal for a DateTimeOriginal of 0000:00:00 00:00:00", () => {
const data = [...EXIF_HEADER, ...TIFF_UNSET_DATE];
expect(readPhotoExif(bytes(SOI, app1(data), SOS))).toStrictEqual({
expect(
readPhotoExif(readAllExifTags(bytes(SOI, app1(data), SOS))),
).toStrictEqual({
orientation: 6,
});
});
it("reads a GPSAltitude without GPSAltitudeRef as above sea level", () => {
const data = [...EXIF_HEADER, ...TIFF_ALTITUDE_WITHOUT_REF];
expect(readPhotoExif(bytes(SOI, app1(data), SOS))).toStrictEqual({
expect(
readPhotoExif(readAllExifTags(bytes(SOI, app1(data), SOS))),
).toStrictEqual({
gpsAltitude: 12.5,
});
});
it("reads a GPSLatitude with GPSLatitudeRef S as south of the equator", () => {
const data = [...EXIF_HEADER, ...TIFF_SOUTHERN_LATITUDE];
expect(
readPhotoExif(readAllExifTags(bytes(SOI, app1(data), SOS))),
).toStrictEqual({
gpsLatitude: -33.5,
});
});
it("gives no gpsLatitude or gpsLongitude without their reference tags", () => {
const data = [...EXIF_HEADER, ...TIFF_POSITION_WITHOUT_REFS];
const tags = readAllExifTags(bytes(SOI, app1(data), SOS));
// The position is read; only its hemisphere is unknown.
expect(tags.GPSLatitude?.computed).toStrictEqual([40, 26, 46]);
expect(tags.GPSLongitude?.computed).toStrictEqual([79, 58, 56]);
expect(readPhotoExif(tags)).toStrictEqual({});
});
it("gives no make for a Make whose value lies past the end of the file", () => {
const data = [...EXIF_HEADER, ...TIFF_MAKE_PAST_END];
expect(readPhotoExif(bytes(SOI, app1(data), SOS))).toStrictEqual({
expect(
readPhotoExif(readAllExifTags(bytes(SOI, app1(data), SOS))),
).toStrictEqual({
orientation: 6,
});
});
@@ -175,8 +304,9 @@ describe("readPhotoExif", () => {
"an EXIF block that cannot be parsed",
bytes(SOI, app1([...EXIF_HEADER, 0x58, 0x58]), SOS),
],
])("returns no fields for %s", (_, input) => {
expect(readPhotoExif(input)).toStrictEqual({});
])("returns no tags and no fields for %s", (_, input) => {
expect(readAllExifTags(input)).toStrictEqual({});
expect(readPhotoExif(readAllExifTags(input))).toStrictEqual({});
});
});
+6 -20
View File
@@ -29,6 +29,7 @@ import { downloadAlbums } from "../../examples/download-albums.js";
import { Library, type ContentSource } from "../../src/index.js";
import type { CollectionsPage, FilesPage } from "../../src/client.js";
import type { Collection, EnteFile } from "../../src/model/types.js";
import { readAllExifTags } from "../../src/exif.js";
import { HEIC_WITH_EXIF } from "../exif-heic.js";
import {
asLivePhoto,
@@ -201,11 +202,10 @@ describe("examples/download-albums.ts", () => {
Buffer.from(VIDEO),
);
// The metadata is the photo's record and its EXIF fields. The
// originals of photos 1 and 2 are not image data, so they have no
// EXIF fields. Photo 3's image holds a camera, an exposure and a
// position, and the date it was taken is written as an ISO 8601
// string. `record` holds the fields the three records share.
// The metadata is the photo's record and its EXIF tags. The originals
// of photos 1 and 2 are not image data, so they have no EXIF tags.
// Photo 3's are every tag of its image, as `photo.exif()` returns
// them. `record` holds the fields the three records share.
const record = {
takenAt: TAKEN_MS,
modifiedAt: TAKEN_MS,
@@ -234,21 +234,7 @@ describe("examples/download-albums.ts", () => {
title: "file-3.jpg",
fileType: "livePhoto",
hash: livePhotoHash(HEIC_WITH_EXIF, VIDEO),
exif: {
make: "Canon",
model: "EOS R5",
lensModel: "RF50mm F1.8 STM",
dateTimeOriginal: "2021-07-15T14:30:00.000Z",
offsetTimeOriginal: "+02:00",
exposureTime: 1 / 250,
fNumber: 2.8,
iso: 400,
focalLength: 50,
orientation: 6,
gpsLatitude: 40 + 26 / 60 + 46 / 3600,
gpsLongitude: -(79 + 58 / 60 + 56 / 3600),
gpsAltitude: -12.5,
},
exif: readAllExifTags(HEIC_WITH_EXIF),
});
// Each album's photos, newest first, by save path relative to `dir`.
+86 -25
View File
@@ -7,7 +7,7 @@
* a cached path shows up on the projected record. A library opened without a
* content source leaves those methods throwing rather than silently doing
* nothing. It also covers a `Photo`'s `savePath`, `isLocal`, `download()`,
* `content()`, `exif()` and the methods that each return one field of `exif()`.
* `content()`, `exif()` and the methods that each return one EXIF field.
*/
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
@@ -27,7 +27,7 @@ import { Library, type LibraryOptions } from "../../src/library/index.js";
import type { ContentSource } from "../../src/library/content.js";
import type { CollectionsPage, FilesPage } from "../../src/client.js";
import type { Collection, EnteFile } from "../../src/model/types.js";
import type { PhotoExif } from "../../src/exif.js";
import { readPhotoExif, type PhotoExif } from "../../src/exif.js";
import { HEIC_WITH_EXIF } from "../exif-heic.js";
import {
asLivePhoto,
@@ -273,10 +273,10 @@ const entry = (
value: number[],
): number[] => [...u16(tag), ...u16(type), ...u32(count), ...value];
// The TIFF block of a JPEG's EXIF segment, holding every field `exif()` picks:
// the camera in the first IFD, the exposure in the Exif IFD, and a GPS position
// of 40°26'46" N, 79°58'56" W, 12.5 m below sea level. Offsets count from the
// start of this block.
// The TIFF block of a JPEG's EXIF segment, holding every field `Photo`'s typed
// methods return: the camera in the first IFD, the exposure in the Exif IFD,
// and a GPS position of 40°26'46" N, 79°58'56" W, 12.5 m below sea level.
// Offsets count from the start of this block.
const TIFF = [
...[0x4d, 0x4d, 0x00, 0x2a], // big-endian TIFF
...u32(8), // the first IFD's offset
@@ -637,9 +637,72 @@ describe("Photo save path, local copy, content and EXIF", () => {
await lib.close();
});
it("reads the common EXIF fields of a JPEG original", async () => {
// Every tag in JPEG_WITH_EXIF, by name, in the order of its IFDs.
const jpegTags = [
"Make",
"Model",
"Orientation",
"Exif IFD Pointer",
"GPS Info IFD Pointer",
"ExposureTime",
"FNumber",
"ISOSpeedRatings",
"DateTimeOriginal",
"OffsetTimeOriginal",
"FocalLength",
"LensModel",
"GPSLatitudeRef",
"GPSLatitude",
"GPSLongitudeRef",
"GPSLongitude",
"GPSAltitudeRef",
"GPSAltitude",
];
// HEIC_WITH_EXIF holds those and the tags exiftool adds to every file.
const heicTags = [
...jpegTags,
"YCbCrPositioning",
"ExifVersion",
"ComponentsConfiguration",
"ColorSpace",
"GPSVersionID",
];
it.each([
[
"JPEG",
JPEG_WITH_EXIF,
jpegTags,
{
"Exif IFD Pointer": { value: 88 },
GPSLatitudeRef: { value: ["N"], description: "North latitude" },
},
],
[
"HEIC",
HEIC_WITH_EXIF,
heicTags,
{
ColorSpace: { value: 0xffff, description: "Uncalibrated" },
ExifVersion: { description: "0232" },
},
],
])(
"returns every EXIF tag of a %s original, by name",
async (_, bytes, names, others) => {
const lib = await open({ contentSource: stubSource(bytes) });
const exif = await lib.photos.byID({ fileID: 1 })!.exif();
expect(Object.keys(exif).sort()).toEqual([...names].sort());
// Tags outside the thirteen fields, as exifreader decodes them.
expect(exif).toMatchObject(others);
await lib.close();
},
);
it("picks the common EXIF fields from a JPEG original's tags", async () => {
const lib = await open({ contentSource: stubSource(JPEG_WITH_EXIF) });
expect(await lib.photos.byID({ fileID: 1 })!.exif()).toStrictEqual({
const exif = await lib.photos.byID({ fileID: 1 })!.exif();
expect(readPhotoExif(exif)).toStrictEqual({
make: "Canon",
model: "EOS R5",
lensModel: "RF50mm F1.8 STM",
@@ -658,8 +721,8 @@ describe("Photo save path, local copy, content and EXIF", () => {
await lib.close();
});
// What exif() returns for HEIC_WITH_EXIF, and for JPEG_WITH_EXIF, which
// holds the same values.
// The fields picked from HEIC_WITH_EXIF's tags, and from JPEG_WITH_EXIF's,
// which hold the same values.
const heicFields: PhotoExif = {
make: "Canon",
model: "EOS R5",
@@ -676,11 +739,10 @@ describe("Photo save path, local copy, content and EXIF", () => {
gpsAltitude: -12.5,
};
it("reads the same common EXIF fields from a HEIC original", async () => {
it("picks the same common EXIF fields from a HEIC original's tags", async () => {
const lib = await open({ contentSource: stubSource(HEIC_WITH_EXIF) });
expect(await lib.photos.byID({ fileID: 1 })!.exif()).toStrictEqual(
heicFields,
);
const exif = await lib.photos.byID({ fileID: 1 })!.exif();
expect(readPhotoExif(exif)).toStrictEqual(heicFields);
await lib.close();
});
@@ -694,28 +756,27 @@ describe("Photo save path, local copy, content and EXIF", () => {
client: new FilesClient([live]),
contentSource: cdnSource(new Map([[1, body]])),
});
expect(await lib.photos.byID({ fileID: 1 })!.exif()).toStrictEqual(
heicFields,
);
const exif = await lib.photos.byID({ fileID: 1 })!.exif();
expect(readPhotoExif(exif)).toStrictEqual(heicFields);
await lib.close();
});
// The build's type check, not this test, makes sure `Photo` has a method
// for every `PhotoExif` field, whatever the fixtures hold: `Photo`
// implements a type with one method per field. This test checks that each
// method gives the same value as exif().
// method gives the field picked from the tags exif() returns.
it.each([
["JPEG", JPEG_WITH_EXIF],
["HEIC", HEIC_WITH_EXIF],
])(
"has a method for each field exif() returns, giving the same value, for a %s",
"has a method for each field, agreeing with the tags exif() returns, for a %s",
async (_, bytes) => {
const lib = await open({ contentSource: stubSource(bytes) });
const photo = lib.photos.byID({ fileID: 1 })!;
const exif = await photo.exif();
const fields = readPhotoExif(await photo.exif());
// The file holds every field, so every method is checked.
expect(exif).toStrictEqual(heicFields);
for (const [field, value] of Object.entries(exif)) {
expect(fields).toStrictEqual(heicFields);
for (const [field, value] of Object.entries(fields)) {
expect(await photo[field as keyof PhotoExif]()).toStrictEqual(
value,
);
@@ -724,7 +785,7 @@ describe("Photo save path, local copy, content and EXIF", () => {
},
);
it("returns no EXIF fields for an original that is not an image", async () => {
it("returns no EXIF tags for an original that is not an image", async () => {
const lib = await open();
const photo = lib.photos.byID({ fileID: 1 })!;
expect(await photo.exif()).toStrictEqual({});
@@ -732,7 +793,7 @@ describe("Photo save path, local copy, content and EXIF", () => {
await lib.close();
});
it("returns no EXIF fields for a JPEG whose EXIF cannot be parsed", async () => {
it("returns no EXIF tags for a JPEG whose EXIF cannot be parsed", async () => {
const lib = await open({
contentSource: stubSource(JPEG_WITH_BAD_EXIF),
});
@@ -753,7 +814,7 @@ describe("Photo save path, local copy, content and EXIF", () => {
await lib.close();
});
it("returns no EXIF fields for a video, without fetching it", async () => {
it("returns no EXIF tags for a video, without fetching it", async () => {
const video = file(1, 1);
video.metadata.fileType = "video";
const source = stubSource(JPEG_WITH_EXIF);
+4 -3
View File
@@ -2,9 +2,10 @@
// `dist/package.json`, which is what `quak --version` reports from a build.
// A `docker build .` of a clone is given no `VERSION` build arg, so the
// version has to come from the `.git` in its context: the tag on a tagged
// commit, otherwise the short commit. A checkout with `.git` that still yields
// no usable version must fail the build, not ship a version nobody can trace
// back to its commit.
// commit; the tag, the commits since it and the short commit on a later commit;
// the short commit when no tag is reachable. A checkout with `.git` that still
// yields no usable version must fail the build, not ship a version nobody can
// trace back to its commit.
//
// Each test copies the script into a fresh directory, which the script then
// treats as the checkout, and executes it there.