Compare commits

..
3 Commits
Author SHA1 Message Date
sneak b42c5b0e0d download-albums example: walk the albums from lib.fresh()
check / check (push) Successful in 1m32s
downloadAlbums now waits for a refresh from the server through lib.fresh()
before walking the albums, so albums and photos added since the cache was
last written are downloaded, and a failed refresh throws instead of
reporting an empty or stale library as done. The test's stand-in account
gains an album holding a new photo before the second run, and that run must
download it. The README's "Examples" section says the script opens the
library with the thumbnail and originals precache off, as `quak backup` does.

Model: opus-5-5
2026-10-01 22:10:19 +00:00
clawbot 56641c6785 Example script: download every album's photos and metadata (closes #144)
`examples/download-albums.ts` logs in with `QUAK_EMAIL` and `QUAK_PASSWORD`,
opens the library, and for every album downloads each photo to its save path,
writes `{savePath}.json` with the photo's record (cache paths left out) and its
EXIF fields, and writes `albums/{collectionID}.json` with the album's save
paths. A JSON file is written only when its content changed, so a second run
downloads and rewrites nothing. `tsconfig.json` includes `examples/`, so the
build type-checks it. A test runs it twice against a stand-in account.

Model: opus-5-5
2026-10-01 22:10:19 +00:00
clawbot d40f339c0b Photo: one async method per EXIF field (closes #148)
check / check (push) Successful in 1m37s
`Photo` gains thirteen async methods, one per `PhotoExif` field and named after it: `make()`, `model()`, `lensModel()`, `dateTimeOriginal()`, `offsetTimeOriginal()`, `exposureTime()`, `fNumber()`, `iso()`, `focalLength()`, `orientation()`, `gpsLatitude()`, `gpsLongitude()` and `gpsAltitude()`. Each calls `exif()` and returns its one field, or `undefined` when the file lacks it. `exif()` is unchanged.

`Photo` implements a type built from `PhotoExif`'s keys, so the type check fails when a field has no method. Each call reads the original again; a caller that wants several fields calls `exif()` once.

Model: opus-5-5
2026-10-02 00:09:24 +02:00
6 changed files with 171 additions and 28 deletions
+11 -3
View File
@@ -91,8 +91,10 @@ yarn build
QUAK_EMAIL=… QUAK_PASSWORD=… node dist/examples/download-albums.js [dir] QUAK_EMAIL=… QUAK_PASSWORD=… node dist/examples/download-albums.js [dir]
``` ```
It asks on the terminal for a two-factor or email code when the account requires It opens the library with `precacheThumbnails` and `precacheOriginals` off, as
one, and writes: `quak backup` does, so the only file content it fetches is the originals it
saves. It asks on the terminal for a two-factor or email code when the account
requires one, and writes:
- each photo's original at its save path under `dir`, as `photo.download()` - each photo's original at its save path under `dir`, as `photo.download()`
writes it: `YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>`, and for a live writes it: `YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>`, and for a live
@@ -758,7 +760,7 @@ synchronous getters look at the disk and never touch the network:
- `photo.isLocal` → `boolean` — whether the whole original is at `savePath`. A - `photo.isLocal` → `boolean` — whether the whole original is at `savePath`. A
copy only in the cache does not count. copy only in the cache does not count.
Five async methods may download: These async methods may download:
- `await photo.original(opts?)` → `{ path, bytes, videoPath? }` — the - `await photo.original(opts?)` → `{ path, bytes, videoPath? }` — the
full-resolution file. For a live photo, `path` and `bytes` are its image's and full-resolution file. For a live photo, `path` and `bytes` are its image's and
@@ -780,6 +782,12 @@ Five async methods may download:
UTC. EXIF is read from any image format exifreader reads (such as JPEG, UTC. EXIF is read from any image format exifreader reads (such as JPEG,
HEIC/HEIF, AVIF, PNG, WebP and TIFF), a live photo's image included. Any other HEIC/HEIF, AVIF, PNG, WebP and TIFF), a live photo's image included. Any other
original gives `{}`, and a video gives `{}` without being downloaded. original gives `{}`, and a video gives `{}` without being downloaded.
- `await photo.make(opts?)`, and likewise `model()`, `lensModel()`,
`dateTimeOriginal()`, `offsetTimeOriginal()`, `exposureTime()`, `fNumber()`,
`iso()`, `focalLength()`, `orientation()`, `gpsLatitude()`, `gpsLongitude()`
and `gpsAltitude()` → one field of `exif()` each, typed as in `PhotoExif`, or
`undefined` when the file lacks it. Each calls `exif()` with its `opts`, so
each call reads the original again.
They serve from the on-disk content cache when the bytes are present and They serve from the on-disk content cache when the bytes are present and
otherwise fetch through the pools; `original()`, `content()` and `exif()` also otherwise fetch through the pools; `original()`, `content()` and `exif()` also
+10
View File
@@ -31,6 +31,16 @@ declares one.
`albums/<collectionID>.json` (issue 144). The build compiles it to `albums/<collectionID>.json` (issue 144). The build compiles it to
`dist/examples/`; the README's "Examples" section says how to run it. `dist/examples/`; the README's "Examples" section says how to run it.
- 2026-10-01: A `Photo` has one async method for each field of `exif()`, named
and typed as in `PhotoExif`: `make()`, `model()`, `lensModel()`,
`dateTimeOriginal()`, `offsetTimeOriginal()`, `exposureTime()`, `fNumber()`,
`iso()`, `focalLength()`, `orientation()`, `gpsLatitude()`, `gpsLongitude()`
and `gpsAltitude()` (issue 148). Each calls `exif()` and returns its one
field, or undefined when the file lacks it. `Photo` implements a type with one
method per `PhotoExif` field, so the build's type check fails when a field has
no method. A test checks, on the JPEG and the HEIC, that each method gives the
same value as `exif()`.
- 2026-10-01: Each original's save path is - 2026-10-01: Each original's save path is
`YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>` under the library's `YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>` under the library's
download directory, which defaults to `photos` in the working directory (issue download directory, which defaults to `photos` in the working directory (issue
+4 -1
View File
@@ -36,7 +36,10 @@ export async function downloadAlbums(
let alreadyLocal = 0; let alreadyLocal = 0;
// A photo in several albums is handled once. // A photo in several albums is handled once.
const done = new Set<number>(); const done = new Set<number>();
for (const album of lib.albums.list()) { // fresh() waits for a refresh from the server and throws if it fails, so
// albums and photos added since the cache was last written are included.
const { albums } = await lib.fresh();
for (const album of albums.list()) {
const savePaths: string[] = []; const savePaths: string[] = [];
for (const photo of album.photos.list()) { for (const photo of album.photos.list()) {
if (!done.has(photo.fileID)) { if (!done.has(photo.fileID)) {
+72 -6
View File
@@ -12,11 +12,11 @@
// plain records are the serializable surface, and `record()` returns one. // plain records are the serializable surface, and `record()` returns one.
// //
// A `Photo` also fetches its own bytes: `original()`, `thumbnail()`, // A `Photo` also fetches its own bytes: `original()`, `thumbnail()`,
// `download()`, `content()` and `exif()` go through the on-disk content cache // `download()`, `content()`, `exif()` and the methods that each return one
// (issue #46), and are the one place in this module that may touch the // field of `exif()` go through the on-disk content cache (issue #46), and are
// network. A library opened without a content source leaves that cache absent, // the one place in this module that may touch the network. A library opened
// and those methods then throw. `savePath` and `isLocal` look only at the disk // without a content source leaves that cache absent, and those methods then
// and need no cache. // throw. `savePath` and `isLocal` look only at the disk and need no cache.
import { readFile } from "node:fs/promises"; import { readFile } from "node:fs/promises";
@@ -42,11 +42,18 @@ const byNewest = (a: PhotoRecord, b: PhotoRecord): number =>
const byNewestAlbum = (a: AlbumRecord, b: AlbumRecord): number => const byNewestAlbum = (a: AlbumRecord, b: AlbumRecord): number =>
b.updationTime - a.updationTime || b.collectionID - a.collectionID; b.updationTime - a.updationTime || b.collectionID - a.collectionID;
// A method for each `PhotoExif` field, named after it, taking the options
// `exif()` takes and giving that field. `Photo` implements it, so the build's
// type check fails when `PhotoExif` has a field `Photo` has no method for.
type PhotoExifMethods = {
[K in keyof PhotoExif]-?: (opts?: ContentOptions) => Promise<PhotoExif[K]>;
};
// A single photo. Field access mirrors `PhotoRecord`; `record()` returns the // A single photo. Field access mirrors `PhotoRecord`; `record()` returns the
// underlying plain record for callers that need the IPC-safe value. `file` is // underlying plain record for callers that need the IPC-safe value. `file` is
// the membership the record is read from, so the save path carries the date of // the membership the record is read from, so the save path carries the date of
// `takenAt` and stays known after a refresh removes the file from the library. // `takenAt` and stays known after a refresh removes the file from the library.
export class Photo { export class Photo implements PhotoExifMethods {
constructor( constructor(
private readonly rec: PhotoRecord, private readonly rec: PhotoRecord,
private readonly file: EnteFile, private readonly file: EnteFile,
@@ -160,6 +167,65 @@ export class Photo {
return readPhotoExif(await this.content(opts)); return readPhotoExif(await this.content(opts));
} }
// One field of `exif()` each, named and typed as in `PhotoExif`, and
// undefined when the file lacks it. Each call runs `exif()`, which reads
// the original again.
async make(opts?: ContentOptions): Promise<PhotoExif["make"]> {
return (await this.exif(opts)).make;
}
async model(opts?: ContentOptions): Promise<PhotoExif["model"]> {
return (await this.exif(opts)).model;
}
async lensModel(opts?: ContentOptions): Promise<PhotoExif["lensModel"]> {
return (await this.exif(opts)).lensModel;
}
async dateTimeOriginal(
opts?: ContentOptions,
): Promise<PhotoExif["dateTimeOriginal"]> {
return (await this.exif(opts)).dateTimeOriginal;
}
async offsetTimeOriginal(
opts?: ContentOptions,
): Promise<PhotoExif["offsetTimeOriginal"]> {
return (await this.exif(opts)).offsetTimeOriginal;
}
async exposureTime(
opts?: ContentOptions,
): Promise<PhotoExif["exposureTime"]> {
return (await this.exif(opts)).exposureTime;
}
async fNumber(opts?: ContentOptions): Promise<PhotoExif["fNumber"]> {
return (await this.exif(opts)).fNumber;
}
async iso(opts?: ContentOptions): Promise<PhotoExif["iso"]> {
return (await this.exif(opts)).iso;
}
async focalLength(
opts?: ContentOptions,
): Promise<PhotoExif["focalLength"]> {
return (await this.exif(opts)).focalLength;
}
async orientation(
opts?: ContentOptions,
): Promise<PhotoExif["orientation"]> {
return (await this.exif(opts)).orientation;
}
async gpsLatitude(
opts?: ContentOptions,
): Promise<PhotoExif["gpsLatitude"]> {
return (await this.exif(opts)).gpsLatitude;
}
async gpsLongitude(
opts?: ContentOptions,
): Promise<PhotoExif["gpsLongitude"]> {
return (await this.exif(opts)).gpsLongitude;
}
async gpsAltitude(
opts?: ContentOptions,
): Promise<PhotoExif["gpsAltitude"]> {
return (await this.exif(opts)).gpsAltitude;
}
private cacheOrThrow(): PhotoContent { private cacheOrThrow(): PhotoContent {
if (!this.cache) { if (!this.cache) {
throw new Error( throw new Error(
+44 -14
View File
@@ -5,8 +5,10 @@
* The account has two albums sharing one photo, and one of its photos is a * The account has two albums sharing one photo, and one of its photos is a
* live photo whose image is a HEIC with EXIF. The first run puts every * live photo whose image is a HEIC with EXIF. The first run puts every
* original at its save path, writes each photo's metadata beside it and each * original at its save path, writes each photo's metadata beside it and each
* album's photos under `albums/`. The second run fetches nothing and changes * album's photos under `albums/`. Before the second run the account gains an
* no file. * album holding a new photo. The second run downloads that photo and writes
* its files and the new album's, and fetches nothing else and changes no
* other file.
*/ */
import { describe, it, expect, beforeEach, afterEach } from "vitest"; import { describe, it, expect, beforeEach, afterEach } from "vitest";
@@ -101,7 +103,7 @@ const readJSON = (path: string): unknown =>
JSON.parse(readFileSync(path, "utf-8")); JSON.parse(readFileSync(path, "utf-8"));
describe("examples/download-albums.ts", () => { describe("examples/download-albums.ts", () => {
it("downloads every album's photos with their metadata, and nothing on a second run", async () => { it("downloads every album's photos with their metadata, and on a second run only what the account gained", async () => {
// Album 1, "Trip", holds photos 1 and 2. Album 2, "Family", holds // Album 1, "Trip", holds photos 1 and 2. Album 2, "Family", holds
// photo 2 and photo 3, a live photo. // photo 2 and photo 3, a live photo.
const live = await asLivePhoto( const live = await asLivePhoto(
@@ -109,20 +111,22 @@ describe("examples/download-albums.ts", () => {
livePhotoZip({ "image.heic": HEIC_WITH_EXIF, "video.mov": VIDEO }), livePhotoZip({ "image.heic": HEIC_WITH_EXIF, "video.mov": VIDEO }),
livePhotoHash(HEIC_WITH_EXIF, VIDEO), livePhotoHash(HEIC_WITH_EXIF, VIDEO),
); );
const collections = [collection(1, "Trip"), collection(2, "Family")];
const filesByAlbum = new Map<number, EnteFile[]>([
[1, [file(1, 1), file(2, 1)]],
[2, [file(2, 2), live.file]],
]);
const client = { const client = {
whoami: () => ({ email: "u@example.com", userID: USER_ID }), whoami: () => ({ email: "u@example.com", userID: USER_ID }),
collectionsSince: async (): Promise<CollectionsPage> => ({ collectionsSince: async (): Promise<CollectionsPage> => ({
collections: [collection(1, "Trip"), collection(2, "Family")], collections: [...collections],
deleted: [], deleted: [],
cursor: 1, cursor: 1,
}), }),
filesSince: async (args: { filesSince: async (args: {
collectionID: number; collectionID: number;
}): Promise<FilesPage> => ({ }): Promise<FilesPage> => ({
files: files: filesByAlbum.get(args.collectionID) ?? [],
args.collectionID === 1
? [file(1, 1), file(2, 1)]
: [file(2, 2), live.file],
deleted: [], deleted: [],
cursor: 1, cursor: 1,
}), }),
@@ -270,18 +274,44 @@ describe("examples/download-albums.ts", () => {
], ],
}); });
// The second run, with a newly opened library, finds every photo // Before the second run the account gains album 3, "Garden", holding
// already local, fetches nothing, and writes, renames or adds no // a new photo 4. The second run, with a newly opened library, opens
// file. // the cache written by the first and still downloads photo 4. It finds
// the other photos already local and fetches nothing else.
collections.push(collection(3, "Garden"));
filesByAlbum.set(3, [file(4, 3)]);
backdate(dir); backdate(dir);
const before = mtimes(dir); const before = mtimes(dir);
const second = await open(); const second = await open();
expect(await downloadAlbums(second, dir)).toEqual({ expect(await downloadAlbums(second, dir)).toEqual({
downloaded: 0, downloaded: 1,
alreadyLocal: 3, alreadyLocal: 3,
}); });
await second.close(); await second.close();
expect(calls).toBe(3); expect(calls).toBe(4);
expect(mtimes(dir)).toEqual(before); expect(readFileSync(join(day, "2026-03-01.4.jpg"), "utf-8")).toBe(
"original-4",
);
expect(readJSON(join(dir, "albums", "3.json"))).toEqual({
collectionID: 3,
name: "Garden",
savePaths: [join(DAY, "2026-03-01.4.jpg")],
});
// The second run adds only photo 4's files and album 3's, and
// rewrites, renames or removes no file from the first run. The two
// directories that gain a file are the only other changes.
const after = mtimes(dir);
expect(
[...after.keys()].filter((name) => !before.has(name)).sort(),
).toEqual([
join(DAY, "2026-03-01.4.jpg"),
join(DAY, "2026-03-01.4.jpg.json"),
join("albums", "3.json"),
]);
for (const [name, mtime] of before) {
if (name === DAY || name === "albums") continue;
expect(after.get(name), name).toBe(mtime);
}
}); });
}); });
+30 -4
View File
@@ -7,7 +7,7 @@
* a cached path shows up on the projected record. A library opened without a * a cached path shows up on the projected record. A library opened without a
* content source leaves those methods throwing rather than silently doing * content source leaves those methods throwing rather than silently doing
* nothing. It also covers a `Photo`'s `savePath`, `isLocal`, `download()`, * nothing. It also covers a `Photo`'s `savePath`, `isLocal`, `download()`,
* `content()` and `exif()`. * `content()`, `exif()` and the methods that each return one field of `exif()`.
*/ */
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest"; import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
@@ -644,8 +644,8 @@ describe("Photo save path, local copy, content and EXIF", () => {
await lib.close(); await lib.close();
}); });
// What exif() returns for HEIC_WITH_EXIF, which holds the same values as // What exif() returns for HEIC_WITH_EXIF, and for JPEG_WITH_EXIF, which
// JPEG_WITH_EXIF. // holds the same values.
const heicFields: PhotoExif = { const heicFields: PhotoExif = {
make: "Canon", make: "Canon",
model: "EOS R5", model: "EOS R5",
@@ -686,9 +686,35 @@ describe("Photo save path, local copy, content and EXIF", () => {
await lib.close(); await lib.close();
}); });
// The build's type check, not this test, makes sure `Photo` has a method
// for every `PhotoExif` field, whatever the fixtures hold: `Photo`
// implements a type with one method per field. This test checks that each
// method gives the same value as exif().
it.each([
["JPEG", JPEG_WITH_EXIF],
["HEIC", HEIC_WITH_EXIF],
])(
"has a method for each field exif() returns, giving the same value, for a %s",
async (_, bytes) => {
const lib = await open({ contentSource: stubSource(bytes) });
const photo = lib.photos.byID({ fileID: 1 })!;
const exif = await photo.exif();
// The file holds every field, so every method is checked.
expect(exif).toStrictEqual(heicFields);
for (const [field, value] of Object.entries(exif)) {
expect(await photo[field as keyof PhotoExif]()).toStrictEqual(
value,
);
}
await lib.close();
},
);
it("returns no EXIF fields for an original that is not an image", async () => { it("returns no EXIF fields for an original that is not an image", async () => {
const lib = await open(); const lib = await open();
expect(await lib.photos.byID({ fileID: 1 })!.exif()).toStrictEqual({}); const photo = lib.photos.byID({ fileID: 1 })!;
expect(await photo.exif()).toStrictEqual({});
expect(await photo.dateTimeOriginal()).toBeUndefined();
await lib.close(); await lib.close();
}); });