Example script: download every album's photos and metadata to the filesystem #144

Closed
opened 2026-10-01 19:52:12 +02:00 by clawbot · 2 comments
Collaborator

Owner's words (chat, 2026-10-01 ~17:5x UTC):

write an example script to iterate over all albums and download all of the album’s photos and metadata to the filesystem.

Depends on #143 (default save path layout, cache-first download). Build it on 143's API once 143 is on next.

Definition of done:

  • The repo has a runnable example script that logs in and opens the library. For every album, it downloads every photo to its save path and writes the photo's metadata to the filesystem.
  • It uses only the public API, in the style of the usage examples on #140.
  • A comment on this issue states, before implementation, where the metadata goes and in what format, and how album membership is recorded on disk, since save paths are by date and not by album.
  • Running it twice downloads nothing the second time.
  • It is type-checked by the repo's check gate and run end to end in a test against a stand-in account.
  • The README says how to run it.
  • One PR to next, independently reviewed and squash-merged.

model: opus-5-5

Owner's words (chat, 2026-10-01 ~17:5x UTC): > write an example script to iterate over all albums and download all of the album’s photos and metadata to the filesystem. Depends on https://git.eeqj.de/sneak/quak/issues/143 (default save path layout, cache-first download). Build it on 143's API once 143 is on `next`. Definition of done: - The repo has a runnable example script that logs in and opens the library. For every album, it downloads every photo to its save path and writes the photo's metadata to the filesystem. - It uses only the public API, in the style of the usage examples on https://git.eeqj.de/sneak/quak/issues/140. - A comment on this issue states, before implementation, where the metadata goes and in what format, and how album membership is recorded on disk, since save paths are by date and not by album. - Running it twice downloads nothing the second time. - It is type-checked by the repo's check gate and run end to end in a test against a stand-in account. - The README says how to run it. - One PR to `next`, independently reviewed and squash-merged. model: opus-5-5
clawbot self-assigned this 2026-10-01 19:52:12 +02:00
Author
Collaborator

Reading, built on #143. Work starts once 143 is on next.

  • Script: examples/download-albums.ts. The check gate type-checks it, and node dist/examples/download-albums.js [dir] runs it after yarn build. dir defaults to ./photos.
  • Login: email and password come from the QUAK_EMAIL and QUAK_PASSWORD environment variables. A 2FA code is asked for on the terminal only when the account needs one. The script then opens the library with downloadDirectory: dir.
  • Loop: for every album in lib.albums.list(), and every photo in album.photos.list(), it runs await photo.download(), which does nothing when the photo is already local. A photo in several albums is downloaded once.
  • Metadata: one JSON file beside each photo, named after it with .json added, for example 2026-03-01.12345.jpg.json. That name does not clash with the backup's own 2026-03-01.12345.json. It holds the photo's record (photo.record(): ID, title, dates, type, caption, size, location, album IDs and hash) and its EXIF fields (await photo.exif()). The script writes it only when it is missing or its contents changed.
  • Albums on disk: the save paths are by date, so membership is recorded in {dir}/albums/{collectionID}.json. Each file holds the album's ID, its name, and the save paths of its photos relative to dir. Each photo's JSON also lists the IDs of its albums.
  • Second run: nothing is downloaded, because every photo is already local.
  • Test: a test runs the script's main function twice against a stand-in account. It checks the files and metadata on disk, and that the second run fetches nothing.

Model: opus-5-5

Reading, built on https://git.eeqj.de/sneak/quak/issues/143. Work starts once 143 is on `next`. - **Script:** `examples/download-albums.ts`. The check gate type-checks it, and `node dist/examples/download-albums.js [dir]` runs it after `yarn build`. `dir` defaults to `./photos`. - **Login:** email and password come from the `QUAK_EMAIL` and `QUAK_PASSWORD` environment variables. A 2FA code is asked for on the terminal only when the account needs one. The script then opens the library with `downloadDirectory: dir`. - **Loop:** for every album in `lib.albums.list()`, and every photo in `album.photos.list()`, it runs `await photo.download()`, which does nothing when the photo is already local. A photo in several albums is downloaded once. - **Metadata:** one JSON file beside each photo, named after it with `.json` added, for example `2026-03-01.12345.jpg.json`. That name does not clash with the backup's own `2026-03-01.12345.json`. It holds the photo's record (`photo.record()`: ID, title, dates, type, caption, size, location, album IDs and hash) and its EXIF fields (`await photo.exif()`). The script writes it only when it is missing or its contents changed. - **Albums on disk:** the save paths are by date, so membership is recorded in `{dir}/albums/{collectionID}.json`. Each file holds the album's ID, its name, and the save paths of its photos relative to `dir`. Each photo's JSON also lists the IDs of its albums. - **Second run:** nothing is downloaded, because every photo is already local. - **Test:** a test runs the script's main function twice against a stand-in account. It checks the files and metadata on disk, and that the second run fetches nothing. Model: opus-5-5
Author
Collaborator

Implementer's brief. Implement the reading in #144 (comment). #143 is on next: photo.download(), savePath, isLocal, and ./photos as the default directory.

Standing rule: quak is pre-1.0, with no installed base and no data anywhere. No migration, no upgrade path, and no backward compatibility.

Script: examples/download-albums.ts. Add examples/**/* to tsconfig.json's include, so the build type-checks it and emits dist/examples/download-albums.js. Import only from ../src/index.js, the public entry.

  • export async function downloadAlbums(lib: Library, dir: string) does the work, so a test can call it:
    • For every album in lib.albums.list(), and every photo in album.photos.list(), it runs await photo.download(). It skips a photo it has already handled in this run, so a photo in several albums is done once.
    • It then writes the photo's metadata to {savePath}.json, a JSON object holding photo.record() without thumbnailPath and originalPath (those are cache paths, not facts about the photo), plus exif: await photo.exif(). Pretty-print it.
    • It writes that file only when the file is missing or its content differs, so a second run rewrites nothing.
    • After each album, it writes {dir}/albums/{collectionID}.json: the album's collectionID, its name, and the save paths of its photos relative to dir. Again, it writes only on a change.
    • Use node:fs/promises.
  • main() runs only when the file is executed directly, not when a test imports it:
    • It reads QUAK_EMAIL and QUAK_PASSWORD, and exits with a clear message if either is missing.
    • It calls Client.login. Its totp and emailOTP callbacks ask for the code on the terminal with node:readline/promises, only when the account requires one.
    • It opens Library.open({ client, downloadDirectory: dir }), where dir is the first argument or photos.
    • It runs downloadAlbums, prints how many photos it downloaded and how many were already local, and closes the library.

Test: test/examples/download-albums.test.ts.

  • Open a Library on a stand-in client and a stand-in content source that counts its calls, the way test/library/content-library.test.ts does. Use two albums sharing one photo, and a live photo.
  • Run downloadAlbums twice. After the first run, check each original at its save path, each .json file's content, and each album file. On the second run the source is never called, and no file's mtime changes.

README. Add an "Examples" section with the exact commands: yarn build, then QUAK_EMAIL=… QUAK_PASSWORD=… node dist/examples/download-albums.js [dir]. Describe the files it writes.

Process

  • Branch from current next, with one PR whose base is next. Title: Example script: download every album's photos and metadata (closes #144). Label it needs-review and assign clawbot.
  • Gate only with make check, and format only with make fmt. Both run in Docker. Run Docker builds under flock -w 1800 /srv/code/tmp/.docker-build.lock. A timeout there exits 1 without running anything; retry, and never read it as red. Remove every container you start.
  • Pull and rebase onto next before pushing. #149 may land first.
  • Edit files by hand. No sed -i, perl -pi, awk or scripted replacements.
  • Keep the change small and plain enough for a newcomer to follow in one reading. Coin no new terms.
  • Ask no interactive questions.
  • Commits and the PR body end with a Model: line naming your model id. Never name the company.
  • Stay under 2 GiB of RAM.

Model: opus-5-5

**Implementer's brief.** Implement the reading in https://git.eeqj.de/sneak/quak/issues/144#issuecomment-107573. https://git.eeqj.de/sneak/quak/issues/143 is on `next`: `photo.download()`, `savePath`, `isLocal`, and `./photos` as the default directory. **Standing rule: quak is pre-1.0, with no installed base and no data anywhere.** No migration, no upgrade path, and no backward compatibility. **Script: `examples/download-albums.ts`.** Add `examples/**/*` to `tsconfig.json`'s `include`, so the build type-checks it and emits `dist/examples/download-albums.js`. Import only from `../src/index.js`, the public entry. - **`export async function downloadAlbums(lib: Library, dir: string)`** does the work, so a test can call it: - For every album in `lib.albums.list()`, and every photo in `album.photos.list()`, it runs `await photo.download()`. It skips a photo it has already handled in this run, so a photo in several albums is done once. - It then writes the photo's metadata to `{savePath}.json`, a JSON object holding `photo.record()` without `thumbnailPath` and `originalPath` (those are cache paths, not facts about the photo), plus `exif: await photo.exif()`. Pretty-print it. - It writes that file only when the file is missing or its content differs, so a second run rewrites nothing. - After each album, it writes `{dir}/albums/{collectionID}.json`: the album's `collectionID`, its `name`, and the save paths of its photos relative to `dir`. Again, it writes only on a change. - Use `node:fs/promises`. - **`main()`** runs only when the file is executed directly, not when a test imports it: - It reads `QUAK_EMAIL` and `QUAK_PASSWORD`, and exits with a clear message if either is missing. - It calls `Client.login`. Its `totp` and `emailOTP` callbacks ask for the code on the terminal with `node:readline/promises`, only when the account requires one. - It opens `Library.open({ client, downloadDirectory: dir })`, where `dir` is the first argument or `photos`. - It runs `downloadAlbums`, prints how many photos it downloaded and how many were already local, and closes the library. **Test: `test/examples/download-albums.test.ts`.** - Open a `Library` on a stand-in client and a stand-in content source that counts its calls, the way `test/library/content-library.test.ts` does. Use two albums sharing one photo, and a live photo. - Run `downloadAlbums` twice. After the first run, check each original at its save path, each `.json` file's content, and each album file. On the second run the source is never called, and no file's mtime changes. **README.** Add an "Examples" section with the exact commands: `yarn build`, then `QUAK_EMAIL=… QUAK_PASSWORD=… node dist/examples/download-albums.js [dir]`. Describe the files it writes. **Process** - Branch from current `next`, with one PR whose base is `next`. Title: `Example script: download every album's photos and metadata (closes #144)`. Label it `needs-review` and assign `clawbot`. - Gate only with `make check`, and format only with `make fmt`. Both run in Docker. Run Docker builds under `flock -w 1800 /srv/code/tmp/.docker-build.lock`. A timeout there exits 1 without running anything; retry, and never read it as red. Remove every container you start. - Pull and rebase onto `next` before pushing. https://git.eeqj.de/sneak/quak/pulls/149 may land first. - Edit files by hand. No `sed -i`, `perl -pi`, `awk` or scripted replacements. - Keep the change small and plain enough for a newcomer to follow in one reading. Coin no new terms. - Ask no interactive questions. - Commits and the PR body end with a `Model:` line naming your model id. Never name the company. - Stay under 2 GiB of RAM. Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/quak#144