From 657266a60e3bf427dc7b6397d635d6b287033c7f Mon Sep 17 00:00:00 2001 From: clawbot <35+clawbot@noreply.example.org> Date: Thu, 22 Oct 2026 23:32:25 +0200 Subject: [PATCH 1/7] A file's JSON holds keywords in place of its CLIP embedding (closes #218) `quak backup` and `quak backup-metadata` write each file's ML data without its CLIP embedding, and `keywords` in its place: the titles of the Discover categories of Ente's photos app that the file falls in, highest score first. A file falls in a category when the cosine similarity of its CLIP embedding and the text embedding of the category's query reaches the category's minimum score; as in Ente's app, five categories leave out videos. The 16 text embeddings ship in `src/clip-keyword-vectors.ts`, written by `bin/clip-keyword-vectors` from Ente's list and CLIP text model. quak runs no model and gains no dependency. Disclosure: the text embeddings are committed model output, against REPO_POLICIES' rule on generated files. Model: opus-5-5 --- README.md | 130 +- TODO.md | 12 + bin/clip-keyword-vectors | 31 + bin/clip-keyword-vectors.py | 158 +++ src/backup.ts | 20 +- src/clip-keyword-vectors.ts | 1836 ++++++++++++++++++++++++++++ src/clip-keywords.ts | 57 + src/library/mldata.ts | 2 +- src/metadata-backup.ts | 14 +- test/cli/backup-end-to-end.test.ts | 17 +- test/cli/backup.test.ts | 108 +- test/cli/metadata-backup.test.ts | 139 ++- test/clip-keywords-sunset.json | 109 ++ test/clip-keywords.test.ts | 129 ++ 14 files changed, 2694 insertions(+), 68 deletions(-) create mode 100755 bin/clip-keyword-vectors create mode 100644 bin/clip-keyword-vectors.py create mode 100644 src/clip-keyword-vectors.ts create mode 100644 src/clip-keywords.ts create mode 100644 test/clip-keywords-sunset.json create mode 100644 test/clip-keywords.test.ts diff --git a/README.md b/README.md index 06e77ea..8e34535 100644 --- a/README.md +++ b/README.md @@ -10,12 +10,13 @@ account into a deduplicated local directory tree, skipping files that already exist on disk and continuing past individual download failures instead of crashing. For each file it persists the basic metadata fields quak keeps (title, file type, creation and modification time, latitude, longitude, content hash), -its update time, the private and public magic metadata in full, Ente's ML data -for it when there is any, and, for an image (for a live photo, its image), its -original's EXIF and XMP, with its dimensions for a JPEG only; for a video it -keeps none of these three. It runs unattended from cron (see "Running the backup -from cron"). A helper subcommand can detect and regenerate missing thumbnails, -encrypting and uploading them back to the server. +its update time, and the private and public magic metadata in full. It keeps any +ML data Ente has for the file, except the CLIP embedding, and writes the file's +keywords in place of the embedding. For an image (for a live photo, its image), +it keeps its original's EXIF and XMP, with its dimensions for a JPEG only. For a +video it keeps none of these three. It runs unattended from cron (see "Running +the backup from cron"). A helper subcommand can detect and regenerate missing +thumbnails, encrypting and uploading them back to the server. ## Getting Started @@ -371,6 +372,11 @@ quak/ backup-metadata: the metadata quak keeps, as JSON exif.ts EXIF read from an image's bytes with exifreader mldata-fetch.ts fetch + decrypt per-file ML data + clip-keywords.ts + a file's keywords from its CLIP embedding + clip-keyword-vectors.ts + Ente's Discover categories with their text + embeddings, written by bin/clip-keyword-vectors filename.ts safe file names from server metadata errors.ts error types shared across layers, and how an error is described @@ -387,6 +393,9 @@ quak/ index.ts public library exports bin/ quak.ts CLI entrypoint (commander.js) + clip-keyword-vectors, clip-keyword-vectors.py + write src/clip-keyword-vectors.ts and a test + fixture (see Source attribution) examples/ download-albums.ts download every album's photos and metadata @@ -783,9 +792,11 @@ as it was and goes into `failures.json` (see "Backup layout"). `backup-metadata ` writes `account.json` and, for each collection, a directory `collections/-/` holding `_collection.json` and a -`.json` for each of its files. The collection's name has unsafe -characters replaced and is cut short between two characters so that the -directory's name fits in 255 bytes; the ID keeps the directories apart. +`.json` for each of its files. Each `.json` holds the file's ML +data and keywords as `quak backup` writes them (see "Backup layout"). The +collection's name has unsafe characters replaced and is cut short between two +characters so that the directory's name fits in 255 bytes; the ID keeps the +directories apart. `backup-metadata` fetches ML data in requests of up to 200 files. When a request fails, the error is logged, each of its files is written with the reason in an @@ -818,11 +829,11 @@ the smallest does not. below) YYYY-MM-DD..json the file's basic metadata fields quak keeps, its update time, its private and - public magic metadata, its ML data, and, - for an image (for a live photo, its - image), its original's EXIF and XMP, with - dimensions for a JPEG only; none of these - three for a video + public magic metadata, its ML data and + keywords, and, for an image (for a live + photo, its image), its original's EXIF + and XMP, with dimensions for a JPEG + only; none of these three for a video YYYY-MM-DD..livephoto.json which of a live photo's two files is which collections/ @@ -853,20 +864,51 @@ JSON holds its `id`, `collectionID`, `ownerID`, `metadata` and `updationTime`, and its `magicMetadata` and `pubMagicMetadata` when it has them. Update times are in microseconds, as Ente records them. -A file's JSON holds Ente's ML data for it (its faces and its CLIP embedding) as -`mlData`, the same payload `backup-metadata` writes; a file Ente has no ML data -for has no `mlData`. The backup waits for the library's ML data fetch to finish -before it writes the JSON files. If that fetch fails, each file whose ML data is -not in the cache gets the reason in `mlDataError` instead and counts as failed, -and the next run fetches it again. The same goes for a file with no ML data in -the cache whose ML data the server sent but quak could not decrypt, gunzip or -parse. Its `mlDataError` says why, and `failures.json` records it with an error -starting `ML data: `. quak keeps that reason in memory only and caches nothing -for the file, so the next run asks for its ML data again. An ML data fetch that -fails, such as one the library's refresh starts while the backup writes the JSON -files, leaves the reasons as the last fetch that succeeded left them. The JSON -files are rewritten on every run, so ML data that arrived since the last run -appears. +A file's JSON holds Ente's ML data for it as `mlData`, such as the faces Ente +found, without its CLIP embedding. It holds `keywords` in place of the +embedding. + +The Discover section of Ente's photos app has 16 categories: + +- Identity (no videos) +- Screenshots (no videos) +- QR Codes (no videos) +- Receipts +- Notes +- Memes +- Visiting Cards (no videos) +- Babies +- Pets +- Selfies +- Wallpapers (no videos) +- Food +- Celebrations +- Sunset +- Hills +- Greenery + +A file's keywords are the titles of the categories it falls in. A file's score +for a category is the cosine similarity of its CLIP embedding and the text +embedding of the category's query. As in Ente's app, the file falls in the +category when that score is at least the category's minimum score. A video never +falls in a category marked "no videos". The keywords are listed highest score +first. + +A file with no ML data has neither `mlData` nor `keywords`. A file whose ML data +holds no usable CLIP embedding has no `keywords`. `backup-metadata` writes both +fields the same way. + +The backup waits for the library's ML data fetch to finish before it writes the +JSON files. If that fetch fails, each file whose ML data is not in the cache +gets the reason in `mlDataError` instead and counts as failed, and the next run +fetches it again. The same goes for a file with no ML data in the cache whose ML +data the server sent but quak could not decrypt, gunzip or parse. Its +`mlDataError` says why, and `failures.json` records it with an error starting +`ML data: `. quak keeps that reason in memory only and caches nothing for the +file, so the next run asks for its ML data again. An ML data fetch that fails, +such as one the library's refresh starts while the backup writes the JSON files, +leaves the reasons as the last fetch that succeeded left them. The JSON files +are rewritten on every run, so ML data that arrived since the last run appears. A file's JSON holds, as `imageMetadata`, what `backup-metadata --exif` records from its original, or from a live photo's image: `format`, `width` and `height` @@ -1270,18 +1312,18 @@ photos newest first). `lib.subscribe({ onChange })` delivers a `LibraryChange` not already at its save path there as `photo.download()` does (and, with `includeThumbnails`, fetches thumbnails) through the content cache, waits for an ML data fetch, and rebuilds the on-disk backup tree with a durable failure - ledger. Each file's JSON holds its ML data and, for an image (for a live - photo, its image), its original's EXIF and XMP, with its dimensions for a JPEG - only; a video's JSON holds none of these three. A fetched original is written - straight to its save path and not into the cache, which then counts it as - present; one the cache already held is copied from there. `BackupOptions`: - `downloadDirectory` (falls back to the library's), `includeOriginals` (default - `true`), `includeThumbnails` (default `false`), `onlyAlbumNames`, `verify` - (default `false`), `onProgress`, and `lock`. See Backup layout above for the - tree it writes, and for the albums `BackupResult` lists in - `undecryptableAlbums` and `failedAlbums`. Whether the backup took the lock or - its caller did, once the lock is lost the backup rejects with - `lost the lock on : ` before it starts another file. + ledger. Each file's JSON holds its ML data and keywords. An image's JSON also + holds the EXIF and XMP of its original, and its dimensions if it is a JPEG. A + live photo's JSON holds these for its image. A video's JSON holds no EXIF, XMP + or dimensions. A fetched original is written straight to its save path and not + into the cache, which then counts it as present; one the cache already held is + copied from there. `BackupOptions`: `downloadDirectory` (falls back to the + library's), `includeOriginals` (default `true`), `includeThumbnails` (default + `false`), `onlyAlbumNames`, `verify` (default `false`), `onProgress`, and + `lock`. See Backup layout above for the tree it writes, and for the albums + `BackupResult` lists in `undecryptableAlbums` and `failedAlbums`. Whether the + backup took the lock or its caller did, once the lock is lost the backup + rejects with `lost the lock on : ` before it starts another file. ### Request pools @@ -1364,6 +1406,14 @@ is rewritten in TypeScript in this repository. Protocol fidelity is verified against the upstream implementations in `web/packages/base/`, `mobile/apps/photos/lib/`, and `cli/`. +The keywords come from Ente too. `src/clip-keyword-vectors.ts` holds the +Discover categories of Ente's photos app, from +`mobile/apps/photos/assets/discover.json`, each with its query's text embedding +from Ente's CLIP text model. `bin/clip-keyword-vectors` writes that file and the +CLIP embedding of a public-domain photo that the tests use. It runs Ente's +models in Docker, with every download pinned by sha256. It is run by hand only, +followed by `make fmt`. + ## For LLMs If you are an LLM agent working on this repository, read and follow these diff --git a/TODO.md b/TODO.md index f1de6ec..99ae803 100644 --- a/TODO.md +++ b/TODO.md @@ -26,6 +26,18 @@ declares one. # Completed Steps +- 2026-10-22: The JSON `quak backup` and `quak backup-metadata` write for a file + holds `keywords` in place of the CLIP embedding in its ML data + (https://git.eeqj.de/sneak/quak/issues/218). The keywords are the titles of + the Discover categories of Ente's photos app that the file falls in, by the + app's rule: the cosine similarity of the file's CLIP embedding and the text + embedding of a category's query is at least the category's minimum score, and + videos stay out of the categories that leave them out. `src/clip-keywords.ts` + applies the rule. `src/clip-keyword-vectors.ts` holds the 16 categories with + their text embeddings. `bin/clip-keyword-vectors` writes it with Ente's CLIP + text model, and writes the CLIP embedding of a public-domain photo of a sunset + that the tests use. + - 2026-10-18: `-v`/`--verbose` raises the CLI's log level once for each repeat, and `-q`/`--quiet` prints only fatal errors (https://git.eeqj.de/sneak/quak/issues/212). The levels are fatal, error, diff --git a/bin/clip-keyword-vectors b/bin/clip-keyword-vectors new file mode 100755 index 0000000..df4a75f --- /dev/null +++ b/bin/clip-keyword-vectors @@ -0,0 +1,31 @@ +#!/bin/sh +# Writes src/clip-keyword-vectors.ts and test/clip-keywords-sunset.json by +# running bin/clip-keyword-vectors.py in a Python image pinned by digest, with +# its packages pinned by hash. Run it by hand from a checkout, then `make fmt`; +# no gate runs it. Besides the Python image, it downloads about 265 MB, most of +# it Ente's two CLIP models. +set -eu + +ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" + +# python:3.13-slim (Python 3.13.17 on Debian 13.7), 2026-10-21. The wheels +# below are the ones for this image on linux/amd64. +IMAGE=python@sha256:57b29a7c050c4e679e4bea572e14b3965cf4b5839de2da49d55e49ffc4024f39 + +main() { + cd "$ROOT" + docker run --rm -i --platform linux/amd64 \ + --user "$(id -u):$(id -g)" -e HOME=/tmp \ + -v "$ROOT:/repo" -w /repo "$IMAGE" sh -c ' + python -m venv /tmp/venv && + /tmp/venv/bin/pip install --quiet --disable-pip-version-check \ + --no-cache-dir --no-deps --require-hashes -r /dev/stdin && + /tmp/venv/bin/python bin/clip-keyword-vectors.py' <<'EOF' +numpy==2.5.3 --hash=sha256:a5fa86b80fd24bcd1aff83ad23be44ea323de3f787be8f8b15d4a65621e25321 +onnxruntime==1.30.0 --hash=sha256:86f940afc801ea9681a4da8af84fbe95e1d9ea7d80903952cc1bfad54faad38f +pillow==12.3.0 --hash=sha256:0847a763afefb695bc912d7c131e7e0632d4edc1d8698f58ddabec8e46b8b6d3 +tokenizers==0.23.2 --hash=sha256:41c2f84d172449b4dadb9cdc508e3e364076613c35b16e76ecfe47a60d1e3305 +EOF +} + +main "$@" diff --git a/bin/clip-keyword-vectors.py b/bin/clip-keyword-vectors.py new file mode 100644 index 0000000..af888fa --- /dev/null +++ b/bin/clip-keyword-vectors.py @@ -0,0 +1,158 @@ +"""Write src/clip-keyword-vectors.ts and test/clip-keywords-sunset.json. + +bin/clip-keyword-vectors runs this in a pinned Python image. It downloads +Ente's Discover list, Ente's CLIP text and image models, the CLIP tokenizer +and a public-domain photo of a sunset, checks each against its sha256, and +writes: + +- src/clip-keyword-vectors.ts: each Discover category of Ente's photos app, + with the text embedding Ente's CLIP text model gives its query; +- test/clip-keywords-sunset.json: the CLIP embedding Ente's image model gives + the photo, prepared as Ente's apps prepare a photo. + +Both embeddings are scaled to length 1, as Ente's apps scale them. +""" + +import hashlib +import io +import json +import urllib.request + +import numpy as np +import onnxruntime +from PIL import Image, ImageOps +from tokenizers import Tokenizer + +ENTE_COMMIT = "f5ad70a860ed25d0c8057b0efb4bd6eaefe4b0c5" +DISCOVER_PATH = "mobile/apps/photos/assets/discover.json" +DISCOVER_URL = ( + f"https://raw.githubusercontent.com/ente-io/ente/{ENTE_COMMIT}/{DISCOVER_PATH}" +) +DISCOVER_SHA256 = "cf70ac7116b592f7921ca4cf67cdbb421a8e6b287fbe44b2be678b8d198e5c24" + +TEXT_MODEL = "mobileclip_s2_text_opset18_quant.onnx" +TEXT_MODEL_SHA256 = "d92f33dfcff83077fc2e0d3414250710efbb51795dfd89767bdbefb5fdc47322" +IMAGE_MODEL = "mobileclip_s2_image_gelu_opset20.onnx" +IMAGE_MODEL_SHA256 = "205a430af825e501c5138e5bb9abea942482a7a4fd4a680e98e47cf0830dce7e" + +# The CLIP tokenizer Hugging Face publishes for OpenAI's CLIP. Its vocabulary +# and merges are those of bpe_simple_vocab_16e6.txt, the vocabulary Ente's +# apps tokenize with. +TOKENIZER_URL = ( + "https://huggingface.co/openai/clip-vit-base-patch32/resolve/" + "3d74acf9a28c67741b2f4f2ea7635f0aaf6f0268/tokenizer.json" +) +TOKENIZER_SHA256 = "b556ac8c99757ffb677208af34bc8c6721572114111a6e0aaf5fa69ff0b8d842" + +# "Sunset off the Massachusetts coast", a NOAA photo in the public domain: +# https://commons.wikimedia.org/wiki/File:Sunset_off_the_Massachusetts_coast_-_NOAA.jpg +PHOTO_URL = ( + "https://upload.wikimedia.org/wikipedia/commons/d/db/" + "Sunset_off_the_Massachusetts_coast_-_NOAA.jpg" +) +PHOTO_SHA256 = "be62f3289c9bf7b75af68cd6978021d54e8a531d5d4cc493f6e9fbf5e1ab74d3" + +# Ente's text model takes 77 token ids, padded with 0; its image model takes +# a 256x256 RGB image. +TOKEN_COUNT = 77 +IMAGE_SIZE = 256 + + +def download(url, sha256): + # Wikimedia refuses requests without a User-Agent. + request = urllib.request.Request(url, headers={"User-Agent": "quak"}) + with urllib.request.urlopen(request) as response: + data = response.read() + digest = hashlib.sha256(data).hexdigest() + if digest != sha256: + raise SystemExit(f"{url} has sha256 {digest}, not {sha256}") + return data + + +def ente_model(name, sha256): + return onnxruntime.InferenceSession( + download(f"https://models.ente.com/{name}", sha256), + providers=["CPUExecutionProvider"], + ) + + +def run(session, tensor): + (output,) = session.run(None, {session.get_inputs()[0].name: tensor}) + embedding = output[0] + return embedding / np.linalg.norm(embedding) + + +def text_embedding(session, tokenizer, query): + ids = tokenizer.encode(query).ids + ids += [0] * (TOKEN_COUNT - len(ids)) + return run(session, np.array([ids], dtype=np.int32)) + + +def image_embedding(session, photo): + image = ImageOps.exif_transpose(Image.open(io.BytesIO(photo))).convert("RGB") + # Scale to cover IMAGE_SIZE x IMAGE_SIZE and crop the centre. + square = ImageOps.fit(image, (IMAGE_SIZE, IMAGE_SIZE), Image.Resampling.BILINEAR) + pixels = np.asarray(square, dtype=np.float32) / 255 + return run(session, pixels.transpose(2, 0, 1)[np.newaxis]) + + +def numbers(embedding): + return ", ".join(str(value) for value in embedding) + + +def write_categories(prompts, embeddings): + lines = [ + "// Written by bin/clip-keyword-vectors; do not edit by hand.", + "//", + "// The Discover categories of Ente's photos app, in their order, from", + f"// {DISCOVER_PATH} in https://github.com/ente-io/ente", + f"// at commit {ENTE_COMMIT}. Each has the text", + f"// embedding Ente's CLIP text model ({TEXT_MODEL})", + "// gives its query, scaled to length 1. `showVideo` is false for a category", + "// that leaves out videos.", + "", + "export const DISCOVER_CATEGORIES = [", + ] + for prompt, embedding in zip(prompts, embeddings): + lines += [ + " {", + f" title: {json.dumps(prompt['title'])},", + f" query: {json.dumps(prompt['query'])},", + f" minScore: {json.dumps(prompt['minScore'])},", + f" showVideo: {json.dumps(prompt.get('showVideo', True))},", + f" textEmbedding: [{numbers(embedding)}],", + " },", + ] + lines.append("];") + with open("src/clip-keyword-vectors.ts", "w", encoding="utf-8") as file: + file.write("\n".join(lines) + "\n") + + +def write_fixture(embedding): + with open("test/clip-keywords-sunset.json", "w", encoding="utf-8") as file: + file.write( + "{\n" + f' "photo": {json.dumps(PHOTO_URL)},\n' + f' "sha256": {json.dumps(PHOTO_SHA256)},\n' + f' "embedding": [{numbers(embedding)}]\n' + "}\n" + ) + + +def main(): + prompts = json.loads(download(DISCOVER_URL, DISCOVER_SHA256))["prompts"] + tokenizer = Tokenizer.from_str( + download(TOKENIZER_URL, TOKENIZER_SHA256).decode("utf-8") + ) + text_model = ente_model(TEXT_MODEL, TEXT_MODEL_SHA256) + embeddings = [ + text_embedding(text_model, tokenizer, prompt["query"]) for prompt in prompts + ] + write_categories(prompts, embeddings) + + image_model = ente_model(IMAGE_MODEL, IMAGE_MODEL_SHA256) + write_fixture(image_embedding(image_model, download(PHOTO_URL, PHOTO_SHA256))) + + +if __name__ == "__main__": + main() diff --git a/src/backup.ts b/src/backup.ts index 37e5116..659275c 100644 --- a/src/backup.ts +++ b/src/backup.ts @@ -14,9 +14,9 @@ // YYYY/YYYY-MM/YYYY-MM-DD/ // YYYY-MM-DD.. the decrypted bytes (the save path) // YYYY-MM-DD..json per-file metadata sidecar, with -// the file's ML data and its -// original's EXIF, XMP and -// dimensions +// the file's ML data, its +// keywords, and its original's +// EXIF, XMP and dimensions // collections// symlink to the original // collections/<name>.json per-collection metadata // account.json the account's email and user ID @@ -77,6 +77,7 @@ import { readFile } from "node:fs/promises"; import { dirname, extname, join, relative, resolve } from "node:path"; import lockfile from "proper-lockfile"; +import { clipKeywords } from "./clip-keywords.js"; import { removeLeftoverTempFiles } from "./download/index.js"; import { describeError } from "./errors.js"; import { @@ -497,8 +498,9 @@ const imageMetadataFor = async ( } }; -// The file's JSON: its basic fields, its magic metadata, its ML data or the -// reason the ML data is missing, and its image metadata. +// The file's JSON: its basic fields, its magic metadata, its ML data without +// the CLIP embedding and the keywords that embedding gives, or the reason the +// ML data is missing, and its image metadata. const writeSidecar = ( path: string, file: EnteFile, @@ -514,7 +516,13 @@ const writeSidecar = ( }; if (file.magicMetadata) meta.magicMetadata = file.magicMetadata; if (file.pubMagicMetadata) meta.pubMagicMetadata = file.pubMagicMetadata; - if (ml.mlData) meta.mlData = ml.mlData; + if (ml.mlData) { + const { clip: _clip, ...mlDataWithoutClip } = ml.mlData; + meta.mlData = mlDataWithoutClip; + const isVideo = file.metadata.fileType === "video"; + const keywords = clipKeywords(ml.mlData, isVideo); + if (keywords) meta.keywords = keywords; + } if (ml.mlDataError) meta.mlDataError = ml.mlDataError; if (image.imageMetadata) meta.imageMetadata = image.imageMetadata; if (image.imageMetadataError) { diff --git a/src/clip-keyword-vectors.ts b/src/clip-keyword-vectors.ts new file mode 100644 index 0000000..9dd3543 --- /dev/null +++ b/src/clip-keyword-vectors.ts @@ -0,0 +1,1836 @@ +// Written by bin/clip-keyword-vectors; do not edit by hand. +// +// The Discover categories of Ente's photos app, in their order, from +// mobile/apps/photos/assets/discover.json in https://github.com/ente-io/ente +// at commit f5ad70a860ed25d0c8057b0efb4bd6eaefe4b0c5. Each has the text +// embedding Ente's CLIP text model (mobileclip_s2_text_opset18_quant.onnx) +// gives its query, scaled to length 1. `showVideo` is false for a category +// that leaves out videos. + +export const DISCOVER_CATEGORIES = [ + { + title: "Identity", + query: "identity documents and cards", + minScore: 0.2, + showVideo: false, + textEmbedding: [ + 0.03607934, -0.015425337, -0.0043316674, 0.009292002, -0.025645005, + 0.014152668, -0.0076436675, 0.014114335, -0.041070342, 0.007275668, + 0.015916003, 0.017763669, 0.033603005, 0.009023668, -0.03352634, + 0.5298818, 0.0017940004, -0.0076360013, -0.042412005, -0.0015103335, + 0.025008671, -0.023552004, -0.024357004, 0.008088335, -0.013907337, + -0.0021313336, -0.019741671, 0.017365003, 0.037566673, -0.018898336, + 0.023245338, -0.008395001, 0.018032003, 0.041430674, -0.007728001, + 0.019948669, -0.010189001, 0.03670034, -0.022923337, -0.04807001, + -0.060191013, 0.01716567, -0.013171336, -0.011308336, 0.011837335, + -0.015893003, -0.018377002, 0.026296671, 0.027684337, 0.043914672, + -0.038801007, -0.024265004, 0.010127668, -0.043102007, + 0.00044466674, -0.02084567, -0.003664667, -0.008786001, + -0.020301336, -0.026036004, 0.019933337, -0.033258006, 0.014275335, + -0.039866675, -0.11761435, 0.02468667, -0.007950335, -0.024817005, + -0.0035036672, 0.0030590005, -0.038517337, 0.035282005, + -0.059524007, -0.029716006, 0.0070456676, -0.024610003, + -0.042941008, -0.015663004, 0.016935669, -0.00097366684, + -0.00071300013, 0.016859002, 0.02664167, -0.0052976673, + -0.005075334, 0.013930335, -0.0065626674, -0.031096006, 0.019120669, + 0.010434335, 0.008287668, -0.006401668, 0.006647001, 0.006340334, + 0.0077510015, 0.02337567, 0.099183686, 0.011914002, -0.011776002, + -0.0072296676, 0.00088166684, 0.029363338, 0.0118986685, + -0.030551674, -0.04252701, -0.012105669, -0.05200301, 0.007666668, + 0.0038180007, -0.011132002, -0.009713668, 0.0006516668, 0.2377357, + -0.031441007, -0.028305337, -0.010825335, 0.02919467, -0.012190002, + 0.050508007, -0.019389004, -0.05011701, 0.0058113346, -0.017020002, + 0.013363002, -0.0026526672, 0.038640004, 0.01987967, -0.013577669, + -0.012542669, -0.008701668, -0.03423934, -0.034431007, 0.013570002, + -0.0051903343, -0.043056007, -0.0021773337, -0.022777671, + 0.021497337, -0.066209346, 0.004262667, 0.010388334, -0.009644669, + -0.015778001, 0.038494337, 0.026105003, 0.010741002, -0.048790675, + 0.002913334, 0.030812338, -0.016299335, 0.0048990007, -0.02036267, + -0.023306672, 0.0006823334, -0.029478338, 0.03545067, -0.01394567, + -0.0044083344, 0.04194434, 0.020025337, 0.012987335, 0.006969001, + 0.019941004, -0.0055966675, 0.009253668, -0.06916101, -0.006210001, + -0.53356177, -0.023207005, -0.020102004, 0.003204667, 0.057454012, + 0.013715669, 0.0029670005, -0.016238004, 0.01852267, 0.0058573345, + -0.0033580007, 0.0023613337, -0.01596967, -0.00085866684, + -0.016138336, 0.02236367, 0.00042166674, -0.02270867, 0.013301668, + -0.009000667, 0.024832336, 0.013447335, -0.033971004, -0.06357968, + -0.030689672, 0.0049220007, 0.012259002, 0.019059336, 0.005826668, + 0.008195668, -0.031111337, 0.0090620015, 0.0037413337, -0.017035335, + 0.00093533343, -0.054249346, 0.0041246675, 0.016912669, 0.020493003, + 0.025361337, -0.014306002, 0.011392668, 0.04227401, -0.014283002, + -0.0013953336, -0.059715677, -0.045777675, -0.012481336, + 0.022141337, 0.062138345, 0.031763006, -0.0031740007, -0.0062483344, + -0.043431673, 0.07113135, 0.016092336, 0.008218668, 0.016284002, + 0.03701467, 0.0007283335, -0.007620668, -0.0062866677, -0.039897338, + -0.006095001, 0.036884338, -0.011630335, 0.003626334, -0.0059416676, + -0.053567007, -0.007275668, 0.005704001, 0.0066623343, -0.026289005, + 0.017664002, 0.014482335, 0.01930467, 0.0073523344, -0.009805668, + 0.005397334, -0.04258067, -0.008149668, 0.025744673, 0.0076820017, + 0.019818338, 0.007291001, -0.040802006, -0.047065675, 0.0024150002, + -0.014950003, -0.0072680013, 0.04578534, 0.00057500013, -0.01806267, + 0.015410002, -0.013830668, 0.0106413355, -0.0013646669, 0.019910336, + -0.0023076672, 0.018262004, 0.0036876672, 0.020953003, + -0.0038793338, -0.011055335, 0.005650334, 0.07518701, -0.022931004, + -0.006210001, -0.016077003, 0.007291001, -0.010449668, -0.03227667, + -0.014957669, -0.021850005, -0.010871335, 0.02323767, -0.046237674, + 0.026925337, -0.010357669, 0.01642967, -0.020002337, 0.032614004, + -0.0039483337, -0.004937334, 0.03311234, -0.06581835, -0.01863767, + 0.007344668, 0.028650336, 0.006041334, -0.07347734, 0.033534005, + 0.029815672, 0.008272334, 0.063426346, 0.0032506671, -0.026818004, + -0.03571134, -0.017472336, -0.01576267, -0.006447668, 0.030712672, + 0.003626334, 0.012696003, 0.026404003, -0.025790673, 0.0023230005, + -0.025016338, -0.11878735, -0.04511834, 0.03575734, -0.041292675, + -0.014681669, -0.038709007, -0.042205006, -0.0078276675, + -0.028888004, -0.070548676, 0.038923673, 0.014037669, 0.0005366667, + -0.007981001, -0.03969034, 0.026818004, -0.010526335, -0.0039483337, + 0.02360567, -0.01921267, -0.02351367, -0.009069668, -0.0029363339, + 0.0018553336, 0.013508668, 0.003626334, -0.0011653336, -0.07035702, + 0.024924338, 0.025392005, -0.0013570002, 0.008441001, 0.021750337, + -0.07450468, -0.0063480013, 0.027906671, 0.0018630002, 0.007038001, + 0.026810339, 0.031625006, -0.01576267, -0.027707336, 0.011684002, + -0.021413004, -0.0032736673, -0.0057193344, -0.018055003, + -0.012857001, -0.04456634, 0.025085337, 0.03508267, -0.02153567, + -0.015172335, 0.016376004, 0.005152001, 0.0011040001, -0.0027983338, + -0.025085337, -0.05013234, 0.012711335, 0.095787354, -0.034423336, + -0.020723004, -0.065412015, 0.012427669, 0.0026833336, 0.0066546677, + -0.008134334, 0.014313668, 0.007919668, -0.009813335, -0.011607335, + -0.05103701, 0.025568336, -0.008379668, -0.0008740002, -0.064837016, + -0.018346336, -0.0062330007, -0.015954336, 0.005443334, + -0.007620668, 0.030766338, -0.004055667, -0.10353068, -0.009568001, + -0.010802335, -0.004692001, 0.0068386677, -0.018599337, 0.02758467, + 0.037727673, -0.07813101, 0.027822338, -0.019642003, -0.02181167, + -0.0015256669, -0.02726267, -0.03633234, -0.03617134, -0.010656669, + -0.008617334, 0.013501002, -0.003166334, -0.033219673, -0.015026669, + -0.0046996675, -0.038003676, 0.020439336, 0.012596336, -0.019864338, + -0.010488002, -0.021098671, 0.01631467, 0.041844673, -0.0133706685, + -0.04556301, 0.009491335, -0.007981001, 0.016935669, -0.03246067, + -0.029693006, 0.015272003, 0.010618336, 0.0025760005, -0.040863343, + 0.006164001, 0.010327002, 0.019642003, 0.023069004, -0.01665967, + -0.01769467, 0.009575669, -0.028198004, 0.011645668, -0.018921336, + 0.016537003, -0.02751567, 0.007544001, -0.017572002, 0.021045003, + -0.011553668, -0.029601006, -0.0057423343, -0.0029286672, + -0.03575734, -0.02243267, 0.020263003, -0.01442867, -0.0030436672, + 0.00042166674, 0.02266267, 0.04010434, -0.009460668, -0.022103004, + -0.0059186677, 0.022256337, -0.043914672, 0.0064706677, + -0.055591006, 0.010871335, -0.008878002, -0.009468335, 0.011024669, + -0.0070456676, -0.005282334, 0.0034730004, 0.010150668, + -0.013508668, 0.0069536683, -0.0011730002, -0.02929434, 0.024717337, + 0.0071683344, 0.04610734, -0.020378003, 0.04194434, -0.03824901, + -0.008195668, -0.02967767, -0.019381337, 0.01836167, 0.034898672, + -0.03818001, -0.010679669, 0.016874336, -0.008870334, 0.004339334, + 0.010986335, 0.008203334, 0.013117669, 0.044420674, -0.013501002, + -0.011960002, -0.013930335, -0.0070533347, -0.015456002, + ], + }, + { + title: "Screenshots", + query: "screenshots", + minScore: 0.18, + showVideo: false, + textEmbedding: [ + 0.043836724, 0.016966958, 0.017219765, -0.005612314, -0.024269465, + -0.010343416, 0.00104012, -0.02295487, 0.017357003, 0.03262654, + 0.013608237, -0.005966244, -0.020267894, -0.021791957, 0.0, + 0.5200383, -0.015457339, -0.009093828, 0.019870626, 0.010646784, + 0.025995778, 0.0034237283, -0.0064574117, -0.02659529, 0.0030481294, + 0.0038571118, -0.025750192, -0.013528784, -0.0042038183, + -0.020137878, -0.0064357426, -0.03201258, 0.049723517, + -0.0018418792, 0.0118096955, 0.012430879, -0.02231924, + -0.0026436383, 0.011788026, 0.017530356, 0.020939637, -0.007649216, + 0.007865908, 0.02413223, 0.043872837, -0.0033081593, -0.008003145, + 0.012336979, 0.030264601, -0.013962167, -0.0075553157, -0.008855466, + -0.031925905, -0.001170135, 0.011564112, -0.0023980546, + -0.031940352, 0.029282266, 0.034937922, -0.009548879, -0.023229348, + 0.028581629, -0.026768643, 0.0015168416, -0.05953965, 0.04403897, + 0.0075769853, 0.015334547, 0.005836229, 0.020585708, -0.006471858, + 0.015544015, 0.01524787, 0.003936565, 0.020672385, -0.01849102, + -0.028112132, 0.014525565, -0.09122719, -0.010733461, 0.02137302, + 0.027808763, -0.0100183785, 0.0661054, -0.034764566, -0.023178786, + -0.020462917, 0.0028314379, -0.026891435, -0.00867489, 0.011224628, + -0.00978724, -0.021019092, 0.0150095085, 0.0060529206, -0.008176499, + 0.14191137, 0.045209102, 0.048683394, -0.014366657, 0.040774148, + -0.013796036, 0.0137021355, -0.040933058, 0.01283537, 0.020773508, + -0.076333255, -0.0111451745, -0.01945891, 0.026855322, + -0.0023041547, -0.057965018, 0.2242542, -0.031131368, -0.007526424, + 0.07724336, 0.010574553, 0.023301577, -0.0034309514, -0.005735106, + -0.019921187, 0.030084027, -0.020390686, -0.0037704348, 0.04054301, + -0.008718228, 0.030315163, -0.0071291556, -0.034779012, 0.017450903, + -0.008624328, -0.034461197, 0.023344915, -0.016779158, -0.03203425, + -0.0041315877, 0.0064501883, 0.028740536, -0.021625828, -0.03971958, + 0.009158835, -0.030437957, -0.006941356, 0.030582415, -0.008494314, + -0.004572194, 0.0015674031, 0.010162839, 0.019516697, 0.002224701, + -0.02555517, -0.0058145598, -0.018230991, -0.0025497386, + 0.027750978, 0.011210182, 0.0017335333, 0.0007295286, -0.02125023, + 0.016598582, -0.0069991406, 0.015710145, 0.023186008, 0.020556815, + 0.019227773, -0.01526954, -0.014706141, -0.52540505, 0.018700492, + -0.023055993, -0.0026941996, 0.076246575, 0.009548879, 0.014496672, + -0.0005778444, 0.0048538935, -0.01636022, -0.018498246, + -0.048134442, -0.09872472, 0.0043916176, -0.006731888, 0.019588927, + -0.008891582, 0.025894655, -0.010668453, 0.05787834, -0.023012655, + 0.035783015, -0.0048972317, 0.0073747393, -0.011101836, + -0.015999068, 0.039437883, -0.03089301, -0.030452402, 0.035544656, + -0.052388825, -0.013897159, 0.03860001, -0.03905506, -0.006731888, + -0.025757415, -0.0015962953, 0.03382557, -0.03923564, 0.017096972, + 0.049774073, -0.011578558, 0.0129364915, -0.021329682, + -0.0021958088, -0.0033876128, -0.0137021355, 0.024197236, + 0.02167639, -0.030452402, 0.0023330469, -0.0100978315, -0.028747762, + -0.015832938, -0.038419433, 0.007865908, 0.0039510117, -0.005453407, + 0.04182149, 0.016598582, -0.0048611164, 7.2230555e-6, -0.03695315, + 0.032308728, 0.02316434, -0.016511904, -0.052749973, 0.0002961453, + -0.048567824, -0.032308728, 0.021069653, 0.005077808, -0.009462202, + -0.0321426, 0.013015946, 0.011997496, -0.011072944, -0.014843379, + 0.07510533, 0.028242148, -0.04544024, -0.05197711, -0.0107840225, + -0.010942929, -0.023539938, 0.020051202, 0.035494093, -0.04436401, + -0.0033081593, 0.013550451, -0.0076564387, -0.0065657576, + 0.0059445747, 0.014482226, -0.021900306, -0.013796036, -0.030669093, + 0.005467853, 0.0077142236, -0.007909246, -0.06104204, 0.02491232, + -0.0065802033, 0.0052439384, 0.010105055, 0.015565684, -0.0129076, + -0.034222838, 7.945361e-5, 0.01978395, -0.010906814, 0.010256739, + 2.8892222e-5, 0.034143385, 0.013391544, 0.005467853, -0.01212751, + 0.031405848, -0.010935706, -0.008552098, 0.0047094324, -0.012445324, + -0.02347493, -0.017060857, 0.027606517, -0.048524488, 0.010523992, + -0.0066163186, -0.006370735, -0.00048394472, 0.03725652, + -0.005937352, -0.0125753395, 0.010704569, 0.007887577, 0.0014518342, + 0.032055918, -0.012979831, 0.017956516, 0.008335406, -0.06319451, + 0.01430165, -0.019978972, 0.012806477, -0.0154140005, -0.006529642, + -0.012185294, 0.011535219, 0.07436857, -0.06877793, 0.009823356, + -0.021221336, 0.0028819991, 0.0064790808, -0.017927622, -0.01604963, + 0.0046516475, 0.043266103, 0.01014117, 0.0035176282, -0.05184709, + 0.02659529, -0.045895297, 0.022853747, -0.031875346, 0.0060673663, + -0.046632048, -0.0391923, -0.015406777, 0.015486231, 0.017523132, + -0.036209177, 0.0011629119, -0.0064285193, 0.00910105, -0.04517299, + -0.0026653074, -0.03222205, 0.0006717442, -0.025143456, + -0.005489522, 0.0025641848, 0.0030770216, 0.0035320742, + -0.012098618, 0.018541582, 0.007244725, 0.010386754, 0.010495099, + -0.1167968, 0.019581703, -0.007056925, 0.023597723, 0.043193873, + -0.028581629, -0.009216619, -0.013998281, -0.014908387, + -0.020217333, 0.03558077, -0.039488442, 0.022283128, 0.050575834, + 0.011751911, -0.031521413, -0.0065585347, 0.034872912, -0.00685468, + 0.0025280693, 0.014893941, -0.015298432, -0.102076225, 0.008328184, + -0.0020152326, 0.012676462, 0.016757488, 0.02319323, 0.004247157, + 0.0012640347, 0.0010545661, 0.01158578, -0.0022391472, 0.061634332, + -0.00026003, -0.089724794, 0.0062046046, -0.015442892, + -0.0103000775, -0.015038402, 0.046913747, 0.019372234, + -0.0041965954, -0.12853427, 0.0047599934, 0.007909246, -0.03179589, + -0.020571262, -0.0013868266, -0.006349066, -0.042016514, + -0.019632265, 0.022023097, -0.012228633, 0.05901236, -0.025287917, + -0.0015601801, 0.010942929, -0.062046047, -0.021365799, + -0.001170135, 0.016519127, -0.045656934, 0.007295286, -0.0009028819, + 0.03857834, -0.0066596568, -0.004536079, -0.004955016, -0.020224554, + -0.016634697, -0.017609809, 0.0028025454, 0.006002359, -0.024377814, + -0.011029606, 0.019379457, -0.007403632, 0.03589136, -0.028307155, + 0.002231924, -0.017602587, 0.018230991, -0.0048900084, -0.025562393, + 0.0012857039, -0.0021669166, -0.04036966, -0.0126259, -0.031564754, + -0.012770362, -0.019733388, -0.010372308, -0.023344915, + -0.0019718942, -0.0020224554, -0.027953224, 0.005113923, + 0.058268387, 0.046054203, -0.0065946495, -0.019791171, -0.0103073, + -0.006341843, -0.0074903085, 0.020470139, -0.010206178, + -0.0061395974, -0.009036043, 0.039026167, 0.0077575613, + 0.0034526207, 0.0100978315, -0.021864189, 0.0487484, -0.007085818, + 0.020686831, 0.009346633, -0.022817632, -0.0030481294, 0.0046516475, + -0.0049911314, 0.006211828, 0.006768003, 0.0051211463, -0.055826996, + -0.031694766, 0.018324891, 0.050951436, -0.017689263, -0.13221803, + 0.0153128775, 0.021777513, 0.0024991771, -0.010755129, -0.01959615, + 0.0067896727, -0.006529642, 0.0033081593, 0.023446038, -0.021329682, + 0.012929269, -0.0098955855, -0.02793878, -0.007461416, + -0.0048900084, -0.02316434, -0.0043771714, 0.036324747, + -0.0049622394, 0.0009462203, -0.0017624255, -0.06352678, + -0.029015014, 0.022694841, 0.0683301, + ], + }, + { + title: "QR Codes", + query: "image containing a black and white QR code", + minScore: 0.2, + showVideo: false, + textEmbedding: [ + 0.015853409, -0.00093012175, -0.0027214673, 0.019863265, + 0.0101142125, -0.007943928, -0.012732333, 0.009018735, 0.025230411, + -0.011078783, -0.028310148, -0.055221666, 0.06398548, -0.017796328, + -0.008426214, 0.47144082, -0.039871216, -0.013400643, 0.035992265, + -0.029688105, -0.043867294, 0.028875113, -0.039981455, 0.031603467, + -0.024672339, -0.025271751, 0.017314043, -0.0022254023, 0.007020696, + 0.03579935, -0.027214672, -0.010982326, 0.016197896, -0.014565017, + -0.007034476, -0.03582002, 0.018443968, 0.029054245, -0.015557148, + -0.028992238, -0.059217747, -0.0077923527, -0.013297294, + 0.009218539, 0.009370115, 0.0027972548, -0.019890824, 0.01249119, + 0.019387871, -0.011002995, 0.0040374175, -0.029881021, 0.026498135, + 0.00985929, -0.01610144, -0.012883908, 0.08034873, -0.0068829004, + -0.021764847, -0.021716619, -0.004740176, -0.036322974, + -0.0058425423, -0.07806133, 0.07493336, 0.05982405, -0.011912447, + 0.033835758, -0.024203833, -0.010782522, 0.0034724541, 0.008109283, + -0.0010265788, 0.04795983, -0.008253969, -0.03578557, + -0.00039960784, 0.00953547, -0.0060216766, 0.035020802, -0.01247052, + -0.003582691, 0.03843125, 0.023742218, -0.009721494, 0.05814983, + 0.007020696, 0.008095504, 0.0031072954, 0.0007096484, 0.05289292, + 0.004567931, 0.024210723, -0.0012263827, 0.011065003, -0.012945916, + 0.24611709, -0.014220527, 0.0026112306, 0.041834805, 0.015357343, + -0.0015708721, 0.014875057, 0.07415482, 0.023390837, 0.023452846, + -0.06579749, 0.036260966, 0.007068925, 0.017072901, -0.014089622, + -0.02133768, 0.18148397, 0.020228425, 0.028255029, -0.02333572, + 0.017837668, -0.0213239, 0.007999047, -0.007117153, 0.009122083, + -0.025554232, 0.010506931, -0.004230331, 0.0016259905, -0.01247052, + 0.008054165, -0.029915469, -0.016129, -0.02722845, 0.035020802, + 0.012566977, 0.015212657, 0.01280812, 0.010562048, -0.04532104, + 0.00394096, 0.041262954, 0.017169356, -0.032285556, -0.015522697, + 0.011030554, 0.017582744, 0.008570899, -0.00592522, 0.003720487, + -0.022467606, 0.0020049291, 0.03905133, 0.034035567, -0.013834699, + 0.00054429343, 0.054091744, -0.011953786, -0.0096939355, + -0.015667383, -0.019491218, 0.025333758, -0.018003022, 0.022508945, + 0.016935105, 0.03550309, -0.0041821026, 0.021916423, 0.03122453, + 0.067340806, -0.014172299, -0.47525772, -0.012787451, -0.025450885, + 0.0055187223, 0.063461855, 0.0062903785, 0.008315977, -0.02627077, + -0.019532556, 0.0018464639, 0.020359332, 0.012305166, 0.09251611, + 0.015880967, 0.018561095, -0.010024644, 0.010713624, -0.024017809, + -0.0059321094, -0.037976522, 0.026436124, -0.023707768, 0.012711663, + -0.009501021, 0.019126058, -0.019794367, 0.007937038, -0.018223496, + -0.046127148, -0.02594006, -0.075181395, 0.0066555375, 0.038582824, + 0.033153668, -0.00790259, -0.042551342, 0.006986248, -0.017651644, + 0.023129027, 0.015949864, 0.006676207, -0.03054244, 0.015701832, + -0.015908526, -0.0379903, 0.04526592, -0.03584069, 0.014861278, + -0.03122453, 0.055035647, -0.0049744286, -0.053016935, -0.007585659, + -0.020269763, -0.037673377, -0.013118161, 0.031658586, -0.032671385, + 0.039499167, -0.026877074, -3.444895e-5, 0.0027145774, -0.022033548, + -0.012732333, -0.03127965, 0.0053326976, -0.015667383, -0.037349552, + -0.036205847, -0.0006683096, 0.021744179, 0.009749053, 0.010768742, + 0.006049236, -0.019291412, 0.011857329, -0.0180857, 0.10372579, + -0.017858338, 0.021640832, -0.04204839, -0.048180304, -0.0007372076, + -5.5118322e-5, -0.0051328936, -0.029274719, 0.010754962, + 0.057075027, 0.010989215, -0.02365265, 0.018692, 0.038610384, + -0.031438112, 0.01745873, -0.009721494, 0.0022529615, -0.011402603, + -0.025437107, 0.014516789, -0.0117953215, 0.0030108383, + 0.0052362406, -0.050825983, -0.0103277955, 0.0013297294, + 0.022901662, 0.017679201, 0.028530622, 0.017768769, -0.025044387, + 0.018871136, 0.023852453, -0.027455814, -0.007937038, -0.048421446, + -0.0052362406, 0.020889845, 0.016066993, -0.004071866, 0.0017775659, + 0.024121156, 0.014716592, 0.0048504123, -0.014689034, 0.013931155, + -0.022453828, 0.02431407, -0.025416438, -0.0015639823, 0.015343564, + 0.07087527, -0.043185204, 0.0003651589, 0.008267748, 0.032058194, + -0.00024114265, 0.020545354, 0.017947905, -0.0086535765, 0.01478549, + 0.03147945, -0.008171291, 0.0098041715, -0.008839601, -0.005560061, + -0.00086811354, -0.025120176, 0.020869175, -0.09649841, -0.06394414, + 0.022784537, -0.059093732, 0.0018533536, -0.032602485, -0.033319023, + -0.046147816, -0.051129136, 0.03454541, 0.008088614, -0.017982353, + 0.051762994, -0.014268757, -0.0011161461, 0.027131995, -0.022095557, + -0.029192042, -0.007082704, -0.016225455, 0.009735274, 0.032058194, + 0.023893794, 0.01021067, 0.037921406, 0.002659459, 0.022577843, + 0.028702868, -0.030094605, -0.019429209, -0.00035826908, + 0.009025626, -0.010472481, -0.13410288, 0.03019795, -0.008908499, + 0.034979466, 0.024410527, 0.022570953, 0.0057529747, 0.014957734, + -0.09229563, -0.041676342, -0.0055807303, -0.020758938, + -0.047456875, 0.005001988, 0.014854388, -0.0030935158, 0.03641254, + 0.03418025, 0.00918409, -0.0005374036, 0.0016811087, -0.003382887, + 0.009473462, 0.027007978, 0.028310148, -0.018829796, -0.06887034, + 0.046292502, -0.021606382, -0.025499115, -0.14208815, -0.018926254, + -0.0044370247, 0.0048779715, -0.0059321094, -0.0071516023, + 0.014957734, -0.013125051, 0.009039405, 0.025237301, 0.01836818, + 0.04567931, -0.0051742326, -0.03025996, 0.026401676, 0.015488249, + -0.019229405, -0.032995205, 0.006793333, 0.13470918, 0.019567003, + -0.01904338, -0.0060010073, 0.011292366, -0.010830751, 0.025740257, + 0.0016811087, 0.0153711215, -0.043936193, 0.080059364, 0.017941015, + -0.048021838, 0.062290594, 0.034648754, 0.008164401, 0.008288417, + -0.008508892, 0.029371176, -0.002328749, 0.020111298, 0.002459655, + -0.01804436, 0.0047884043, -0.051749215, 0.035344627, 0.041145828, + -0.001839574, -0.0282757, 0.011175239, -0.029757006, -0.020131966, + 0.02559557, -0.023728438, -0.010424253, 0.01380025, 0.017369162, + 0.025561122, 0.02195776, 0.01411029, -0.037707824, 0.016135888, + 0.024555212, 0.028075894, -0.0007992157, 0.016253015, 0.039003104, + 0.029701887, -0.02822747, 0.014055173, 0.008350426, 0.0015157539, + -0.016053211, 0.023101468, 0.031100513, -0.0139931645, 0.0089636175, + -0.0102244485, 0.00016535497, -0.025113286, 0.025416438, + 0.022943001, 0.028255029, -0.003968519, -0.016066993, 0.040263932, + 0.049496252, -0.01442722, 0.06689986, 0.025712699, -0.006448844, + -0.02135146, 0.019525666, -0.021110317, -0.042930283, -0.016204787, + -0.0072549493, 0.014599466, 0.025223523, 0.0012401623, + -0.00016535497, -0.013497099, 0.038775742, -0.017672313, + -0.04007102, 0.0064695133, -0.014399661, 0.02266052, -0.046650767, + -0.14170921, 0.04127673, -0.028503062, 0.0011574848, -0.0051328936, + 0.05357501, -0.005126004, -0.015067972, 0.032285556, -0.00075787696, + 0.03803164, 0.02565069, 0.0095148, 0.020173306, -0.008825822, + -0.03055622, 0.0050571063, 0.001743117, -0.008681135, 0.0226054, + 0.043839738, 0.006648647, -0.014716592, 0.0054222653, 0.00787503, + 0.029991256, + ], + }, + { + title: "Receipts", + query: "receipts, bills and invoices", + minScore: 0.2, + showVideo: true, + textEmbedding: [ + -0.012871421, 0.017175015, 0.03869955, 0.015771098, -0.05597953, + 0.014459028, -0.014563994, 0.036213174, -0.006153615, -0.007446006, + -0.03248689, -0.002643824, 0.0044151205, 0.0035229118, -0.027901199, + 0.5036518, -0.015305313, 0.03251969, -0.01211698, 0.0145049505, + -0.01817219, 0.051118307, -0.021583576, 0.015462763, 0.02681874, + 0.02679906, -0.045587923, 0.0022502027, 0.014570554, -0.014728002, + -0.013041991, -0.0065800385, 0.010509692, 0.0016860118, 0.046558857, + -0.00495307, 0.0046381727, 0.041638587, -0.016118798, 0.0161844, + -0.0025913413, 0.009007371, -0.01946458, -0.009079535, 0.0058321576, + 0.045601044, 0.013967001, -0.036475588, -0.008049558, 0.031883337, + -0.032250717, 0.018605174, -0.020625765, -0.031089535, 0.009565001, + -0.015941668, -0.023276148, 0.00030177642, -0.01926777, + 0.0124843605, 0.0060355286, 0.0040871026, 0.01542996, -0.029993955, + -0.11969373, 0.013429052, 0.004270793, -0.00055107, 0.013245362, + 0.00033457825, -0.0078658685, 0.008961448, -0.016604265, + -0.0012727093, -0.032631215, -0.042675126, -0.03104361, + -0.034113858, 0.020114055, -0.011854566, 0.07291837, -0.015764538, + 0.04847448, -0.0028603158, -0.016210644, -0.023413915, 0.014826409, + 0.03197518, 0.024798151, 0.0003280179, 0.04501061, 0.0044872845, + 0.024155235, 0.025408264, -0.010680262, -0.0029652815, 0.08523872, + 0.031686526, 0.031594682, -0.014649278, 0.031161698, -0.0121957045, + -0.0040477403, -0.03986729, -0.009374751, 0.017647361, -0.046735983, + 0.021360524, -0.013855475, -0.024988402, -0.026700657, -0.027573183, + 0.2742295, -0.00052482856, 0.022016559, -0.008567827, 0.031876776, + 0.002256763, 0.00934851, -0.01761456, -0.021524534, -0.039985377, + -0.017194696, -0.04042492, 0.03219823, 0.019517064, -0.046230838, + 0.024922797, -0.034061372, -0.027671589, -0.0075312904, + -0.0008594068, -0.015193787, 0.018998794, 0.004579129, 0.025368901, + -0.010188235, 0.031620923, -0.023105578, -0.018998794, 0.030125162, + -0.01835588, 0.0077281008, 0.010785228, 0.031470034, -0.016250005, + -0.0031292904, 0.013284723, 0.040031303, 0.019963168, -0.0216295, + 0.0061929775, -0.065537974, 0.020835694, -0.024443893, 0.0059568044, + -0.028334184, -0.043055624, -0.019930365, -0.0017319344, + -0.0075050485, 0.021852551, -0.017876973, 0.014596796, 0.021058748, + -0.09324892, 0.025926534, -0.50703037, 0.052213885, -0.016919162, + 0.013914518, 0.05725224, 0.021878792, -0.016768275, -0.014340941, + -0.0055500623, -0.015305313, -0.0289771, -0.016381213, -0.006678444, + -0.016873239, 0.010214476, 0.017706405, -0.016105678, -0.0027487897, + 0.019051278, 0.05134136, 0.013461853, -0.00051826827, -0.008948327, + -0.012057937, -0.018211551, -0.040497087, 0.026457923, + 0.00082660496, 0.0056812693, 0.000990614, -0.011815203, 0.038135357, + 0.020501116, -0.05576304, -0.03269026, -0.034015454, -0.024732547, + 0.021773826, 0.0017122533, 0.02718612, 0.017115973, -0.031542197, + 0.00606177, 0.0040411805, 0.03806319, -0.05327666, 0.031424113, + 0.0047628195, 0.01229411, -0.014373742, -0.004480724, 0.008416939, + 0.011959531, -0.048185825, 0.08870259, 0.013422491, -0.005963365, + -0.0063110637, 0.060663626, -0.04789061, -0.00017712965, + -0.007557532, -0.012543404, -0.020455195, 0.033877686, -0.042091254, + -0.019661391, 0.006363547, -0.03137163, -0.0044151205, 0.053355385, + 0.051282313, -0.011113245, -0.014406545, -0.025985574, 0.06363547, + -0.0034835497, 0.004454483, -0.026825303, -0.022239612, + 0.0050383545, -0.05691766, -0.003103049, -0.0040805424, + -0.030623749, -0.04771348, -0.0477594, 0.009604364, 0.00990614, + 0.009223863, 0.05252878, 0.016105678, 0.026549768, 0.021950955, + -0.020422393, -0.008147963, -0.01633529, 0.0117036775, 0.024496375, + -0.05052131, -0.06555765, -0.008830241, -0.023276148, -0.0106671415, + -0.0070392634, 0.03454684, -0.02954785, -0.030282611, 0.0022502027, + 0.0023551683, 0.001725374, -0.0024338926, -0.0019484262, + -0.024122434, -0.013101033, -0.04315403, -0.030545022, 0.033497185, + 0.029692179, -0.0019418658, 0.0065603578, 0.00862687, -0.014668958, + -0.01817875, 0.009118897, -0.046558857, -0.005812477, -0.018559251, + -0.03874547, 0.014045725, -0.056819256, 0.018250914, 0.011119806, + 0.06538052, 0.026333276, 0.030571265, -0.009256665, 0.022606991, + 0.012969826, 0.011342858, -0.016243445, 0.00028865572, -0.045089334, + 0.030945206, 0.021544214, -0.017234059, -0.023171183, -0.027251724, + -0.06330745, -0.039952576, 0.02607742, -0.06366171, -0.026897466, + 0.0010627779, -0.048487604, -0.049491335, -0.029626574, + -0.054824907, 0.0038640504, 0.040831663, -0.0034769892, -0.02408307, + -0.032421287, 0.031666845, 0.009755251, -0.05476586, 0.016682988, + -0.005969925, 0.013002628, 0.0056091053, -0.028196417, -0.026320154, + 0.015462763, -0.06549205, -0.01064746, -0.040333077, 0.0202059, + -0.041264646, -0.008922086, 0.018441165, -0.011683996, -0.012740213, + 0.008908966, -0.009984864, 0.0030243248, -0.035767067, 0.027993044, + 0.00642259, 0.0023158062, -0.033818644, 0.037125062, -0.011611832, + 0.013980121, -0.016190961, 0.00532045, -0.008226688, -0.029869309, + 0.033156045, 0.0003411386, 0.0054647774, -0.03877827, 0.0073935227, + 0.0488681, -0.04382319, 0.011815203, -0.007983955, -0.0040871026, + -0.047090247, 0.068214595, -0.04290474, -0.0006625961, -0.028524432, + 0.015994152, 0.00787899, -0.017161895, -0.0048874663, 0.027205803, + 0.013324087, -0.00972245, -0.028334184, -0.006809651, 0.018296836, + 0.012825498, -0.020291185, -0.06573478, -0.031522516, 0.045909382, + -0.04077262, -0.015731737, -0.008403818, 0.061024442, 0.021386765, + -0.04957662, -0.043094985, -0.0006494754, 0.0028012726, 0.051728416, + 0.0064094695, 0.015443081, 0.025467308, -0.0057403124, -0.007091746, + -0.018808546, 0.017129093, -0.037886064, -0.010929555, -0.020579841, + -0.034973267, -0.01928745, 0.0031292904, -0.02075041, -0.0057140714, + -0.035865474, -0.0935179, -0.0006888375, 0.024686623, -0.023991227, + 0.051485684, 0.00825293, -0.006088012, 0.018447725, 0.0111854095, + 0.004677535, -0.015206908, -0.050042406, 0.025834689, -0.027481338, + -0.0031686525, 0.028471952, -0.012615567, -0.007446006, 0.013232241, + -0.009079535, -0.0003542593, 0.016112236, 0.0076165753, + -0.0070851855, 0.007931472, -0.027061474, -0.010745865, -0.04331148, + -0.010345683, 0.053033933, -0.00072819967, 0.023413915, + -0.020868497, 0.04771348, -0.023702571, -0.022534829, -0.04109408, + -0.020848814, 0.0028603158, -0.027999604, -0.050750926, + -0.005313889, 0.017647361, 0.0139538795, 0.018237794, 0.042124055, + 0.019562986, 0.032381922, 0.001928745, -0.016663307, 0.023735372, + 0.040155947, -0.04385599, -0.030577825, -0.01928745, 0.029462563, + -0.020507677, 0.017686725, 0.011598712, 0.024115874, -0.026825303, + -0.015521805, -0.025946213, -0.0032342563, 0.015475882, 0.037505563, + -0.009965182, 0.0316078, 0.0035294723, -0.035124153, -0.021891912, + -0.038469933, -0.04850072, -0.049661905, -0.040818542, -0.02206248, + 0.021084988, 0.030223567, -0.028242338, -0.049242042, 0.018605174, + -0.038141917, -0.0006691565, 0.07440758, -0.0012071057, + -0.0008594068, 0.0381616, -0.0003542593, 0.0026372636, -0.042458635, + 0.045384552, 0.0015351236, + ], + }, + { + title: "Notes", + query: "screenshots of digital notes, photos of notebook and whiteboards", + minScore: 0.2, + showVideo: true, + textEmbedding: [ + 0.03148959, -0.012310127, -0.008262599, -0.040193092, 0.008668235, + 0.006904606, -0.014038484, 0.012133764, -0.027979966, -0.048931874, + -0.010167318, -0.01888846, -0.0037388923, 0.0039328914, + -0.040907364, 0.5144681, 0.017318832, 0.024770161, -0.01295385, + 0.03631311, -0.005193886, 0.004259163, 0.0110932235, 0.014303027, + 0.01637529, 0.04230063, -0.04334117, 0.03342076, 0.020925451, + -0.0005026341, -0.002733624, -0.03218622, 0.0015960837, + 0.0084389625, 0.021622084, -0.0022662627, -0.0011463584, + -0.008262599, -0.04159518, -0.016692745, -0.0064107897, + 0.0019047188, 0.023059443, 0.03879101, -0.014497027, -0.023570895, + -0.023950074, -0.004276799, -0.012468853, -0.0035184387, + -0.022521535, 0.009955683, -0.0011992673, -0.008139146, 0.016622199, + 0.00522034, 0.00021163542, -0.0035272571, 0.025502069, + -0.0014990842, -0.008103873, -0.038773373, -0.0061903363, + 0.00384471, -0.072079495, 0.038641103, 0.012133764, -0.01362403, + -7.054514e-5, -0.048490964, -0.0067899697, -0.0031128044, + -0.0281916, 0.00028218055, -0.010176136, -0.031507224, -0.048755508, + -0.0140473, -0.02521107, -0.017265923, 0.031480767, 0.002036991, + -0.012645217, 0.038076736, -0.008659416, 0.06813779, -0.013500575, + 0.013826846, -0.04963732, -0.03164831, 0.023553258, 0.00029981683, + -0.049143508, 0.00820969, -0.029875867, 0.013756301, 0.1874296, + 0.016683925, -0.013333031, 0.00892396, -0.005088068, -0.017724466, + -0.026366245, 0.008694689, -0.016366472, -0.0023368078, -0.05606575, + 0.0061727, -0.012310127, -0.01442648, -0.02644561, -0.011084405, + 0.24448301, -0.019726183, -0.012856851, -0.0027600785, -0.017486377, + 0.036242567, 0.0301845, -0.047600333, 0.024055893, 0.032979853, + 0.02568725, -0.028764782, 0.027803604, -0.011878038, 0.0064107897, + -0.0057317927, -0.030660681, -0.052591402, 0.019391095, + -0.018200645, -0.03531666, 0.010176136, -0.053773034, -0.014735116, + -0.014117845, 1.7636285e-5, 0.022486264, 0.013456484, -0.014673389, + -0.011825128, -0.034152664, 0.012072037, -0.011375403, + -0.0012345399, 0.015669838, -0.0007142695, 0.0281916, 0.010043864, + 0.013641666, 0.012504126, -0.010193773, -0.023412168, 0.023473896, + 0.014303027, -0.0066488795, -0.028076967, 0.0065077893, 0.027591966, + 0.07500712, -0.014400026, -0.0022750807, -0.019302914, 0.015008478, + -0.012327763, -0.0049381596, -0.5187449, -0.007724693, -0.05559839, + 0.013571121, 0.061083272, -0.049161144, 0.001798901, -0.002354444, + -0.0065077893, 0.0091708675, -0.028500237, -0.03745947, + -0.025934156, -0.004814706, 0.008835779, -0.012380673, 0.007468967, + 0.01637529, -0.0018429917, 0.0284826, -0.002407353, 0.014408845, + 0.00768942, -0.017257104, -0.00033508943, 0.0071779676, + -0.027256878, -0.0019399914, -0.040810365, -0.015352386, + -0.05730029, 0.00308635, 0.043949623, -0.030131591, -0.010528862, + -0.0077335113, 0.025202252, 0.04715061, -0.058305558, 0.038896825, + -0.015026114, 0.016675107, -0.015590476, -0.0016666289, + 0.0130508505, -0.046771426, -0.03849119, -0.0051850677, 0.073922485, + -0.016754469, 0.0037565285, -0.015934383, -0.065183714, 0.04718588, + -0.0029011688, 0.0042856173, 0.013377122, -0.006825242, 0.055474933, + 0.021401633, -0.02235399, -0.016410563, -0.0142765725, 0.004267981, + -0.0039064367, -0.055748295, 0.012363035, 0.01167522, -0.021860175, + 0.0043032533, 0.024514437, -0.0027071696, -0.017398193, + 0.0043120715, -0.009047414, 0.0065430617, -0.039293643, + -0.011472403, 0.030978136, -0.031401407, -0.04954914, 0.027759513, + -0.0468067, 0.014893843, -0.0483146, -0.024435071, 0.02340335, + -0.00043208894, 0.00882696, -0.01979673, 0.025325704, -0.044461075, + 0.006886969, -0.008818143, 0.0049117054, -0.024099983, -0.005167431, + -0.0066576973, -0.011075586, 0.018130101, 0.0008377235, 0.01533475, + 0.0070897867, -0.063702255, -0.03418794, -0.009267868, -0.021613266, + -0.00061726995, -0.00322744, -0.025369795, -0.02207181, 0.013236032, + 0.017098378, 0.027574332, 0.0009964501, 0.022036538, -0.030484319, + -0.016886743, -0.018200645, -0.0067546973, 0.030299138, + 0.0009611775, -0.009770501, 0.014567572, 0.023526805, -0.053676035, + 0.04506071, 0.0074777845, 0.030501954, -0.009682321, -0.02372962, + -0.012424762, 0.035766385, -0.013553485, -0.005925792, 0.008359599, + 0.018835552, 0.0036859834, 0.016728016, -0.001172813, 0.027406786, + 0.029214505, -0.016287109, 0.0064107897, -0.0056612473, -0.0385441, + 0.024029437, -0.040713366, -0.031886403, -0.045254707, 0.005714156, + -0.0349463, 0.0052291583, 0.0066665155, 0.016736835, 0.004356162, + -0.002345626, 0.03151604, 0.019338185, 0.015775656, -0.09950392, + -0.007001605, -0.011833947, 0.042803265, 0.008562416, -0.039425917, + -0.08338436, -0.037062652, -0.033068035, 0.018121284, 0.015176023, + 0.02525516, 0.0064107897, -0.013818029, -0.009073868, -0.026163427, + -0.0151054775, -0.035202023, 0.023764893, 0.020475727, -0.027988784, + 0.00892396, -0.016551653, 0.028597236, 0.04570443, 0.0148056615, + 0.07217649, 0.013923847, -0.019867275, -0.14952925, 0.058367286, + 0.012424762, 0.06264409, 0.010811042, 0.0054760664, 0.026983516, + -0.016181292, 0.005767065, -0.04343817, -0.031551313, -0.021692632, + 0.013950301, 0.008703507, 0.015493477, -0.012971488, -0.01855337, + 0.0066224253, -0.027882965, 0.05995455, -0.0007936328, 0.05747665, + -0.03848237, -0.039796278, -0.03830601, 0.03682456, -0.025193432, + -0.00768942, -0.022159992, 0.018738551, 0.0032538946, -0.005828792, + 0.031683587, 0.08285526, -0.043085445, -0.048199967, 0.003597802, + 0.024981797, 0.017618649, 0.0036859834, 0.017239468, 0.035898656, + -0.022459809, -0.05572184, 0.014814478, -0.0070456956, -0.007830511, + -0.023182897, 0.029752413, -0.008253781, -0.0042062537, -0.0182271, + 0.018068373, -0.016287109, 0.0030069866, -0.0033244397, + -0.033050396, 0.037247833, -0.028447328, -0.015687475, + -0.0066665155, 0.00223099, -0.019920183, 0.001543175, 0.04078391, + 0.012380673, 0.016595745, -0.01828001, 0.03284758, -0.007839329, + 0.007407239, -0.02316526, -0.0013491758, -0.015087841, 0.023844257, + -0.049143508, -0.021771993, -0.013994392, 0.0029893504, + -0.021868993, -0.033782303, 0.0069663324, 0.019338185, 0.037380103, + 0.003641893, 0.013086123, -0.018271191, 0.018712098, -0.03128677, + 0.021348724, -0.0063226083, -0.01865919, -0.0118868565, + -0.022371627, -0.00044972525, 0.03457594, -0.04105727, 0.010149682, + 0.015255386, 0.008236145, -0.021181177, -0.011128495, 0.0011375403, + -0.024576163, 0.00731024, -0.04319126, -0.008527143, -0.028835325, + -0.00028218055, 0.031110404, 0.03949646, 0.044452257, -0.013712212, + -0.041797996, 0.02093427, 0.028403237, -0.015255386, 0.00731024, + -0.047547426, -0.0005643611, 0.016163655, -0.0033244397, + 0.012027946, -0.011710493, -0.0025131707, -0.004144527, 0.010714043, + 0.003976982, -0.010528862, -0.03949646, -0.08931015, 0.026401518, + -0.009647048, 0.035969205, 0.0021692629, 0.014770388, -0.0054319757, + 0.0042062537, -0.00721324, 0.01518484, -0.010846316, -0.0065430617, + 0.015087841, 0.0039858003, -0.0071603316, -0.019205915, + -0.008244963, 0.038985007, -0.026983516, -0.011428312, 0.006128609, + 0.013271305, -0.030546045, -0.020193547, 0.008941596, 0.05889637, + ], + }, + { + title: "Memes", + query: "memes", + minScore: 0.18, + showVideo: true, + textEmbedding: [ + 0.024733832, 0.02873802, -0.004280583, -0.048865262, 0.0050743334, + 0.007377628, 0.007221712, 0.0043231053, 0.05938246, -0.04826995, + 0.034428928, 0.002204075, 0.0025371667, 0.003791576, 0.0009284046, + 0.52122474, -0.025187403, -0.031905934, -0.0012969316, + -0.0030332608, 0.057624865, 0.010906982, -0.0064137876, 0.006725618, + -0.009695095, -0.015981317, -0.011906258, 0.01809326, -0.011303858, + -0.011211725, 0.017916081, -0.007306757, -0.014471772, 0.016895546, + -0.007604413, -0.04029701, 0.00023387291, -0.0073634526, + -0.025541758, 0.001346541, 0.010573891, -0.0027143431, 0.026902473, + 0.032359507, 0.026491422, -0.029255377, -0.014464686, 0.027859226, + 0.04314601, 0.004401063, 0.0140182, -0.0009921881, 0.03883708, + 0.05593815, 0.028865587, 0.02632842, -0.06859563, 0.026725296, + 0.012905533, -0.010609326, 0.008135943, 0.0043231053, -0.023004591, + 0.032324072, -0.061728273, 0.034372233, -0.00027639527, 0.009284046, + 0.033408392, 0.016023839, 0.011133768, -0.009113957, 0.0074343234, + -0.004642023, 0.023203027, -0.022359667, 0.015988404, 0.009503745, + -0.048666827, -0.0018851574, 0.016796328, -0.06535684, 0.01110542, + 0.021360392, -0.046115484, 0.0014953692, 0.031388577, 0.023465248, + 0.0019843762, -0.04834082, 0.009305307, -0.038496897, 0.018674398, + 0.014195377, -0.011870822, -0.010580977, 0.13177675, -0.039184343, + 0.04032536, -0.06712861, -0.0038766204, -0.030892486, -0.0062720464, + 0.027200127, 0.013266971, 0.033642262, -0.047660466, -0.01650576, + 0.01250157, 0.0125582665, 0.00073705404, -0.044598855, 0.14651075, + -0.01353628, 0.009964403, 0.053946685, -0.0039687525, -0.017916081, + -0.01284175, -0.020651687, 0.017278247, -0.03042474, -0.018532656, + -0.012480309, 0.0067823143, 0.018086173, 0.05493887, -0.015017476, + 0.009220262, 0.0036214867, -0.010432149, 0.0007654023, 0.024875574, + -0.013699283, -0.019468147, 0.012374003, 0.008887171, 0.014648949, + -0.042047516, -0.026753645, -0.0068177497, -0.0016371105, + 0.008306031, -0.01579705, -0.019446887, 0.0078028506, -0.0058397357, + -0.014415076, 0.013387453, -0.03543529, 0.0050955946, 0.032508336, + -0.044322457, -0.027214302, -0.03252251, 0.015549005, -0.010751067, + -0.0049892887, -0.0067327046, 0.025803978, 0.059467502, 0.023543207, + 0.009624224, 0.028220663, -0.016895546, 0.02160844, -0.031806715, + -0.5259943, -0.0060665216, -0.0310484, -0.020935168, 0.06222437, + -0.02355738, 0.039900135, 0.0019347668, -0.014450511, 0.0020410726, + -0.0069594905, -0.018801965, 0.000106305866, 0.011048723, + 0.0074910205, 0.020949343, 0.021700572, 0.063492954, -0.03522268, + 0.014974954, -0.025889022, 0.0240393, -0.000616574, 0.03145945, + -0.041756947, -0.02143835, -0.03777402, -0.009525006, 0.029347507, + 0.006520093, -0.008206814, 0.014656036, 0.012600789, 0.019787066, + -0.0164278, -0.022869937, 0.015145043, -0.055392448, -0.0055987756, + -0.019021664, 0.011310944, -0.013266971, -0.0017150679, + 0.0066618347, 0.0062791333, -0.023160506, 0.009078521, + -0.0023316422, -0.023940083, -0.09124587, -0.0089367805, + -0.0024946444, 0.010347105, 0.06452766, -0.07113989, 0.020113071, + -0.0065838764, 0.008001288, -0.0124661345, -0.04612966, + 0.0021828138, 0.04277748, -0.025577193, 0.007937505, -0.014634774, + -0.013047273, -0.048631392, -0.010481758, -0.014025288, + -0.020155592, 0.0029127807, 0.0016512844, -0.0065130065, + -0.022246275, -0.016052186, -0.028000964, 0.0070091, -0.02741274, + 0.015641136, 0.017164854, -0.0039829263, -0.055392448, -0.008695819, + -0.012898446, -0.050835468, -0.01534348, 0.10378288, -0.023188854, + 0.01031167, 0.020587903, -0.015003301, 0.008674559, -0.014776516, + -0.006236611, -0.028915197, -0.015371829, -0.014174115, 0.0448469, + 0.04323814, 0.0011339292, -0.042359345, 0.017816864, 0.021282436, + 0.016094709, 0.021672223, 0.021211565, -0.014450511, 0.018752355, + 0.027426915, 0.020807602, 0.0008929693, 0.018568091, -0.0074201496, + 0.02531497, 0.014797776, -0.0051948135, -0.004415237, -0.0017930256, + -0.039318997, 0.013642587, 0.014266248, 0.0065271803, 0.024613352, + 0.010942417, 0.019978417, -0.04790851, -0.01700185, 0.032458726, + 0.014174115, 0.027589917, 0.027575742, 0.012990577, 0.008894257, + -0.015471048, -0.009872272, 0.011084158, 0.030488525, -0.0033734394, + -0.02495353, 0.010396714, 0.024124345, 0.042097125, -0.03175002, + -0.026292983, -0.008178465, -0.010134493, -0.039106388, 0.04943223, + 0.019609889, -0.06208971, 0.03557703, -0.08570379, -0.0124661345, + 0.05873045, -0.033592656, -0.01963115, 0.039928485, -0.002941129, + -0.0027568655, -0.043004267, 0.038433116, 0.044811465, -0.008894257, + -0.027667874, -0.01206926, 0.0035789642, 0.01387646, -0.048191994, + -0.010906982, -0.0030332608, 0.04902118, -0.022083273, -0.010389627, + 0.0079658525, -0.034393493, -0.018057823, 0.03714327, -0.02440074, + 0.024939355, -0.017930256, 0.01201965, -0.024514133, 0.0072996696, + 0.0053436416, 0.0141315935, 0.003068696, -0.036703873, -0.03748345, + -0.03376274, -0.04908496, -0.009404526, 0.0015449787, 0.054350648, + -0.010155753, -0.0018709833, 0.0033805268, -0.020821776, + 0.0054712086, -0.0085328175, 0.013578803, 0.040176533, 0.007710719, + 0.010772328, 0.020389466, -0.026144156, -0.014025288, 0.035612464, + -0.02476218, -0.047596682, 0.0051877266, -0.035725858, -0.090976566, + -0.029021503, -0.012785052, -0.0106235, 0.002955303, 0.0332383, + -0.023940083, 0.0064988323, -0.0044790204, 0.024003865, 0.038893774, + 0.0543223, 0.011403076, -0.022097446, 0.007398888, 0.020807602, + -0.06272755, -0.0075902385, -0.021948619, 0.056760244, 0.010729806, + -0.19647451, 0.03880164, 0.00046774582, -0.003791576, 0.017646775, + 0.03071531, 0.022295883, -0.01589627, -0.0041388418, 0.016193926, + 0.030332608, 0.022657324, -0.04556978, 0.0057972134, -0.00049609406, + 0.0049255053, -0.015818313, -0.02384795, 0.027526133, 0.011615688, + -0.02782379, 0.06167158, 0.04608714, 0.013968592, -0.011133768, + -0.050204717, 0.004053797, -0.019517757, 0.006123218, -0.005308206, + 0.03252251, -0.047724247, 0.01818539, 0.0067823143, -0.016633324, + 9.9218814e-5, -0.0116794715, -0.043911412, -0.027561568, + -0.024173955, 0.028362405, -0.008802126, -0.0065271803, 0.00725006, + -0.043394055, 0.022175403, -0.02570476, 0.010446324, -0.014330031, + 0.0042238864, -0.02873802, -0.07274156, -0.04075058, 0.030963358, + -0.013621326, 0.008015462, -0.034216315, -0.021289522, -0.033330433, + -0.008702907, -0.024351131, 0.0047058063, -0.011360554, 0.008880083, + -0.019914633, 0.028121445, 0.008901345, -0.038128372, -0.027115082, + 0.004259322, -0.03988596, 0.03615108, 0.0054641217, -0.025931545, + 0.018369654, 0.020991866, 0.019213013, -0.0070445356, 0.008950954, + -0.006590964, 0.024939355, 0.0034443103, -0.019964242, -0.009602964, + 0.030438915, 0.0110203745, 0.029999517, -0.07240139, 0.0074484977, + -0.0033380042, 0.014266248, 0.023982603, -0.033139084, 0.014244986, + 0.00029765643, -0.013798501, 0.02211162, -0.013422888, 0.025874848, + -0.009595877, 0.041650638, 0.08546283, -0.0069736647, 0.006569702, + -0.013337842, 0.006243698, -0.014755256, 0.012005476, 0.03403205, + -0.024173955, 0.024492873, -0.011183377, 0.024960618, + ], + }, + { + title: "Visiting Cards", + query: "business card", + minScore: 0.2, + showVideo: false, + textEmbedding: [ + 0.0029489065, -0.009914337, 0.008136701, 0.013697645, 0.027322732, + -0.019538447, -0.019248221, 0.020445405, -0.0040579853, 0.020186275, + 0.02225932, 0.016019454, 0.013091278, 0.008665328, -0.021046588, + 0.50537205, -0.03424152, -0.05618988, -0.007680631, 0.007110544, + 0.017449856, 0.014583871, -0.029193653, 0.0013008356, 0.029888125, + -0.011790443, -0.011386199, 0.010857572, 0.0028400715, 0.039740272, + 0.033521138, 0.009370162, -0.022176398, -0.037345905, -0.0037211154, + 0.033816542, -0.007317848, -0.005400282, 0.004275655, 0.019217126, + -0.046778258, -0.0075717964, 0.0063901604, -0.0028037932, + -0.00079812226, -0.012811417, -0.02258064, 0.0021404189, + 0.047353525, 0.014521679, -0.04745718, -0.016491072, -0.03712305, + 0.030546315, -0.07583198, -0.0063331523, 0.048996415, -0.010888668, + -0.0035760025, -0.016869402, -0.012583383, -0.0021248711, + 0.03339675, -0.03584813, -0.07071674, 0.037713867, -0.0074525964, + -0.032723013, 0.0317435, 0.016993785, -0.025710938, -0.04251297, + -0.044834778, 0.013568078, -0.009007379, 0.01095086, -0.052111164, + 0.011831904, 0.0012230964, 0.002337358, 0.03213738, 0.014526862, + 0.028245235, 0.024783252, 0.008421745, -0.012438269, 0.0185693, + -0.0464673, 0.027270906, -0.012407173, -0.021792883, 0.0036692894, + -0.017708985, 0.011847451, 0.006089569, -0.0054987515, 0.19703254, + -0.015724046, 0.012915069, 0.008732702, 0.008768979, -0.019232674, + -0.020512778, -0.012707765, -0.0034671675, -0.022891598, + -0.057112385, -0.022456259, -0.013692461, 0.021202065, 0.01960064, + -0.04331627, 0.24108475, 0.030681064, -0.037097137, 0.062424563, + 0.024897268, -0.038081832, 0.004410403, 0.039486323, -0.008805258, + 0.015822515, 0.026752643, -0.018579664, -0.0065404563, 0.015656672, + -0.03855345, -0.034184508, 0.06779375, -0.010121642, -0.0024513756, + -0.0041357246, -0.0024358276, -0.001803549, -0.013998236, + -0.016019454, 0.010588077, 0.023725998, -0.05693618, -0.02987776, + -0.0074525964, 0.022513267, -0.013490339, -0.020087805, 0.00878971, + 0.035640825, -2.073045e-5, 0.01727883, 0.0146823395, -0.022269685, + 0.016796846, 0.0018709231, -0.014905193, 0.00049753074, + -0.029851845, 0.007359309, -0.03926347, 0.015718862, 0.002710506, + 0.016807212, 0.0074992394, 0.0026534975, 0.00048198292, + -0.0057941605, 0.0066337436, 0.0014200357, -0.005970369, + -0.50913984, 0.004182368, -0.022435527, -0.015553019, 0.0639638, + 0.042305663, 0.04190142, -0.007561431, -0.0012179138, -0.019792397, + 0.0012386444, 0.0001762088, 0.0035863677, 0.043466568, 0.016143836, + 0.003529359, 0.004425951, -0.0038973242, 0.011417294, -0.0047835507, + 0.027213896, 0.04382935, 0.019372603, -0.011868183, -0.009453085, + 0.012764773, -0.031831603, -0.0061154827, -0.03019908, -0.021637406, + 0.019408882, 0.0019590273, -0.021870622, 0.031753864, -0.054510716, + -0.0465865, -0.00498049, -0.044928063, 0.030240541, 0.031095672, + 0.015148776, 0.026286209, -0.00472136, -0.008877815, 0.01420554, + -0.076547176, -0.0077946484, 0.034101587, -0.04702184, 0.043451022, + 0.028162314, -0.028462905, 0.0050685946, 0.0014511314, 0.018626308, + -0.006975796, -0.032650456, -0.010251207, 0.009732946, 0.021855075, + 0.005586856, 0.011013051, -0.06129994, -0.018439734, 0.0056438646, + -0.021476744, 0.013080913, 0.013459245, 0.02295379, 0.008297361, + -0.0020004883, -0.04693892, -0.007551066, -0.00392842, 0.01938297, + 0.0010261572, -0.011318824, 0.039512236, -0.028841237, -0.007152005, + -0.002430645, 0.14335105, 0.008515031, -0.018097682, 0.02194318, + -0.026265478, -0.03372844, 0.031458456, -0.0007566614, 0.023554971, + 0.0071001784, 0.0072349263, 0.01585361, -0.0015340532, -0.008281815, + 0.00416682, -0.03297178, 0.009292424, -0.004052803, 0.0068047694, + -0.07608075, -0.006887692, 0.009670755, 0.004612525, -0.045669176, + 0.0019538447, -0.031935256, -0.039097626, -0.010588077, 0.031795327, + 0.02345132, -0.048913494, -0.0077687358, -0.013516253, 0.026654173, + -0.017900743, -0.023466866, 0.022611737, -0.0030421934, + -0.013593991, -0.026120367, 0.11547897, -0.00031095673, + 0.0024461928, 0.04492288, -0.053588208, -0.02704287, -0.034345172, + -0.016019454, 0.0131431045, -0.085704856, 0.014925922, 0.015915802, + 0.014837818, 0.03104903, 0.018579664, 0.011691973, -0.019942692, + -0.0064627174, 0.007613257, 0.011868183, 0.01936224, 0.03600361, + 0.025643567, 0.023104087, -0.006296874, -0.007950127, 0.020004883, + -0.046498396, -0.063222684, -0.017107802, -0.014713435, + 0.0045399684, -0.0034049763, -0.08453877, -0.02008262, -0.005306995, + 0.006866961, 0.04562253, -0.06647218, -0.03928938, 0.023777824, + 0.022145301, 0.004944212, 0.015267976, 0.004296385, -0.010183833, + -0.015464915, 0.014101888, 0.020631978, -0.024736607, 0.014863731, + 0.021528572, 0.023586068, -0.01144839, 0.000114017465, + -0.0072608395, -0.04225902, 0.026654173, -0.04146608, 0.011225538, + -0.020979214, 0.014936289, -0.013982688, -0.061455414, 0.008069327, + 0.013360774, -0.017973298, 0.018781785, -0.10951895, -0.010468877, + 0.016755385, -0.0023425405, 0.021062136, 0.009748493, -0.008919275, + -0.040642045, -0.02446193, 0.03081063, -0.0010261572, -0.005213708, + -0.0103341285, 0.0024617407, 0.013075731, 0.010645086, 0.0020212189, + -0.0071416395, 0.0070898137, 0.045524064, -0.035189938, + -0.012821783, -0.0982675, 0.0015236881, -0.010137189, -0.026125548, + 0.0019020187, -0.0017154446, -0.012930617, 0.012427905, 0.030644784, + -0.015775872, 0.015604846, 0.014817089, 0.020549057, -0.03503446, + 0.049872275, 0.019331142, -0.02659198, -0.014713435, -0.0070898137, + 0.05996282, 0.00022285232, -0.02797574, -0.01950217, -0.014252184, + 0.02234224, -0.005260351, -0.01075392, 0.006815135, -0.023565337, + -0.040538393, 0.0065663694, 0.0056231343, -0.019346692, + -0.033624787, -0.015806967, -0.0064523523, -0.037418462, + -0.006400526, -0.020181092, -0.023078172, 0.013127556, -0.06507288, + -0.069566205, 0.054458886, -0.020409126, 0.027965376, 0.025788678, + -0.024632957, 0.0109560415, 0.02715689, 3.6278285e-5, 0.005379551, + -0.042507786, 0.011598687, 0.00924578, 0.011780077, 0.00484056, + -0.02635358, 0.0052085253, -0.018009577, 0.013060183, 0.032292854, + 0.015765507, 0.01236053, 0.01107006, -0.047187682, 0.014122617, + -0.026607532, -0.00565423, 0.05966223, -0.017071525, 0.056734055, + 0.01607128, 0.02081337, 0.026970314, -0.018087316, 0.03444882, + 0.0071001784, -0.011438025, -0.0012490095, 0.054132383, + -0.021741059, 0.013246756, -0.012044391, 0.01835163, -0.0059185433, + -0.008126336, -0.0072193784, -0.024114693, -0.002632767, + -0.00064782647, -0.0048198295, -0.004006159, -0.024285719, + 0.052007515, -0.012287973, 0.0088259885, 0.025482906, -0.0002954089, + 0.011142616, -0.0033738804, -0.0059548216, 0.0020523143, + -0.014760079, -0.027198348, -0.008758615, 0.038138844, -0.035837762, + -0.10033019, 0.03762058, -0.0065715523, -0.0008706788, -0.024788434, + 0.013070548, 0.010930129, -0.019232674, 0.022653198, 0.025902696, + 0.017366933, -0.0015651488, 0.0071571874, -0.0034775329, + -0.0015444184, 0.020901475, 0.019061647, 0.010556981, 0.0018501924, + 0.016693193, 0.018346447, -0.006561187, -0.017138898, -0.025902696, + 0.025799042, 0.02654534, + ], + }, + { + title: "Babies", + query: "photo of toddler", + minScore: 0.2, + showVideo: true, + textEmbedding: [ + -0.037668705, 0.00064175244, -0.023759263, 0.025042767, + 0.0014709831, 0.015027101, -0.018747823, 0.018531503, -0.021221096, + 0.014623303, 0.040452033, 0.011789497, 0.06196877, -0.01336864, + 0.025980156, 0.55503654, -0.00842931, 0.0072323335, -0.006705953, + -0.07496245, 0.0052782334, -0.024631757, -0.04707146, -0.016930727, + -0.01767343, -0.02683823, -0.015690487, -0.0075856578, 0.021934954, + 0.014457457, -0.012885524, -0.03559923, -0.014536774, -0.010960266, + 0.01573375, 0.021466259, 0.010058929, 0.0024949026, 0.009402756, + 0.0049104877, -0.003208762, -0.0012186086, 0.0063814707, + -0.009265752, 0.017680641, -0.026355116, -0.027436718, -0.027804464, + 0.015329951, 0.0061939927, 0.00616515, 0.04787906, -0.0113496445, + -0.010246406, -0.007008802, 0.012777363, -0.016916307, 0.0013916654, + 0.017622955, 0.026124371, 0.0368539, -0.01377244, -0.0018243075, + 0.03783455, 0.001521458, 0.03125118, 0.0034395044, -0.019815007, + -0.015762594, 0.089218006, 0.0068213237, -0.02516535, -0.007787558, + 0.025792679, 0.014089711, -0.021963798, -0.066093296, 0.0018964144, + 0.05002064, -0.070527874, 0.010131036, -0.039413694, 0.009474862, + 0.018675717, -0.009186434, 0.0127917845, -0.025331194, 0.011674126, + -0.016635088, -0.0004326421, -0.007845243, 0.02827316, 0.007080909, + 0.038959417, -0.0016007758, 0.021862846, 0.034106616, 0.011313591, + 0.007621712, 0.02106246, 0.0036125614, 0.0018531504, -0.00077875576, + -0.03751728, -0.029729724, -0.0143420845, -0.07822169, 0.006979959, + 0.0038721466, 0.013498433, -0.015265056, -0.0040740464, 0.056019936, + -0.057426028, 0.002610274, -0.06806181, 0.003403451, 0.011890447, + 0.0072323335, 0.0034683477, -0.03414988, 0.015366005, 0.01470983, + -0.005847879, -0.06406708, 0.0052061267, -0.036046296, -0.07235218, + -0.0017810432, 0.012013029, 0.06053384, 0.017868118, 0.00915038, + -0.05428937, -0.030883435, -0.016195236, 0.0074919187, -0.007419812, + -0.046905614, 0.039031528, -0.028201055, 0.033673976, 0.0070881196, + 0.00023074244, 0.012964842, 0.011609229, 0.019613108, -0.0147170415, + 0.0021415784, 0.009532548, -0.011666915, -0.04653787, -0.08192078, + 0.03661594, 0.018675717, -0.04123079, 0.02106967, 0.007448655, + 0.03256353, -0.051859368, -0.042269133, 0.0046509025, -0.010505992, + 0.0009950768, 0.04112984, 0.018625243, -0.017868118, -0.5596875, + -0.030256104, -0.039925657, -0.016606247, 0.09181386, -0.03318365, + -0.019180465, 0.0014060867, -0.008551892, 0.015070367, 0.003042916, + 0.006208414, 0.029974885, 0.00031006016, 0.03846909, 0.0027400665, + 0.0059055644, -0.023283355, 0.013628227, 0.014810781, -0.022893978, + 0.059697397, 0.028114527, -0.0072828084, -0.0042903675, 0.060483366, + -0.07522925, -0.024141429, -0.034236412, 0.019115571, -0.04477846, + 0.057909146, 0.0089412695, 0.037337013, -0.03711348, -0.002098314, + -0.030147944, -0.0020694714, -0.010830474, 0.026549803, 0.03926948, + 0.0018387288, 0.04306952, -0.009309015, 0.015596747, 0.015755383, + -0.03440226, 0.008645631, 0.021134567, 0.026412798, 0.0041245213, + 0.007996668, -0.026066687, 0.01881272, 0.01953379, -0.002350689, + 0.05210453, 0.003958675, 0.010087771, 0.013887811, -0.02608832, + -0.00090854836, -0.044403497, 0.023283355, -0.0044850563, + -0.016880251, -0.11022999, 0.011111692, -0.04955194, -0.0036197721, + 0.029664826, -0.009323437, 0.0034683477, 0.015380426, -0.023103088, + 0.0016873041, 0.027205976, -0.02619648, -0.01953379, 0.009705604, + 0.066735044, -0.07096773, 0.004636481, -0.003331344, 0.02949177, + 0.004492267, 0.06757148, -0.02508603, -0.0012907155, -0.038368143, + -0.019461684, 0.019129992, -0.020939877, -0.031099755, -0.021776319, + 0.023687156, -0.014096921, -0.008256254, -0.014428614, -0.010008453, + -0.027941467, 0.004636481, 0.0039082007, 0.008277886, 0.0074342336, + 0.012510567, 0.006186782, 0.027169922, 0.016808145, 0.0158347, + 0.0071962797, 0.008724948, -0.01120543, 0.043119997, 0.03249863, + 0.001788254, 0.043949224, 0.002538167, 0.02918892, -0.03585161, + 0.0010094981, 0.030688744, -0.041266844, -0.031070912, -0.015243424, + 0.0005263812, 0.009705604, -0.0072828084, -0.001694515, 0.004506688, + 0.042002335, 0.023391517, 0.010909791, -0.024754338, 0.003475558, + 0.014089711, -0.009575811, -0.012373564, -0.00015142473, + 0.004326421, -0.01438535, 0.040401563, 0.017449897, -0.04408623, + -0.0035548757, 0.004492267, -0.038288828, -0.021855637, + -0.039392065, -0.066655725, -0.017211946, -0.009272962, 0.028785119, + -0.019613108, -0.0060137254, 0.021055248, 0.044569347, + -0.0054657115, 0.0063309963, 0.015697697, 0.01951216, 0.008458153, + 0.00063454174, 0.002026207, -0.02291561, -0.032743797, 0.008623999, + -0.041879755, -0.00057685614, 0.015611168, 0.002653538, 0.003136655, + 0.006410314, -0.05578199, -0.011573176, 0.05498881, 0.020132279, + -0.021906111, -0.013700333, 0.013008106, -0.024213536, -0.020427918, + 0.011017952, 0.01490452, 0.0023290566, 0.022901189, -0.017594112, + 0.0047879056, 0.0012258192, -0.027090607, -0.048181906, + 0.0030357053, 0.03383982, -0.08334129, -0.01972127, -0.0069150627, + -0.025158137, -0.025230244, 0.029859515, 0.03844746, -0.036630366, + 0.010902581, 0.004809538, 0.021797951, 0.005919986, -0.009035009, + -0.03577229, 0.024588494, -0.061795715, 0.01254662, 0.02598737, + -0.05224153, -0.002804963, -0.013080212, 0.01294321, -0.04575911, + -0.015236213, 0.029816251, 0.0014205083, 0.00034611367, 0.003893779, + 0.029044706, -0.02569894, -0.013628227, -0.010924213, 0.008458153, + -0.033529762, 0.024213536, 0.019598687, -0.027595354, 0.002487692, + -0.031597294, -0.022771396, -0.029044706, 0.005775772, 0.017226366, + 0.0022569494, 0.04748968, -0.004362474, 0.014024814, 0.0047518522, + 0.0046653235, 0.027595354, -0.0006778059, 0.028713014, 0.012049083, + -0.014190662, -0.013145109, -0.015654434, 0.03022726, -0.022634393, + -0.020637028, -0.006179571, 0.052169424, 0.031215128, 0.044028543, + 0.0073621264, -0.013202795, 0.008400467, -0.0041100997, + -0.013736386, -0.021148989, -0.032513052, 0.012690835, 0.02477597, + -0.029037496, 0.0057685613, 0.0036558257, 0.017723905, 0.013260481, + 0.034596946, 0.020737978, -0.0069655376, -0.025504252, 0.028352479, + 0.02076682, 0.024653388, -0.0044345814, -0.0012835049, + 0.00035332437, 0.016180813, -0.01181834, -0.021091303, -0.02538167, + 0.011313591, -0.0062156245, 0.015452534, -0.02642001, 0.024516385, + -0.0024732705, -0.03708464, -0.028496692, -0.00493212, -0.041591328, + 0.036745735, 0.0012690835, -0.00585509, 0.034899794, -0.0050835446, + 0.013296533, 0.017781591, 0.023413148, 0.016887464, -0.0015286687, + -0.01572654, 0.011248694, 0.014017604, -0.0136498585, 0.002538167, + 0.027516035, 0.008133672, -0.00493212, 0.054000944, -0.033241335, + 0.01574096, -0.032556318, -0.030133521, -0.009842608, -0.050943606, + -0.053589933, -0.012236561, -0.0029491768, 0.009777712, + -0.028244318, 0.0030357053, 0.014515142, 0.07838033, 0.019093938, + -0.018344024, -0.058197573, -0.009777712, 0.00585509, 0.029852306, + 0.019461684, 0.022576706, 0.0042831567, -0.020824507, 0.011068427, + 0.0042110495, 0.018228654, -0.03414988, -0.013339797, -0.02361505, + 0.028648118, 0.00873937, + ], + }, + { + title: "Pets", + query: "photos of pets", + minScore: 0.2, + showVideo: true, + textEmbedding: [ + 0.015499058, 0.038108576, 0.0033366026, -0.024849616, 0.011684836, + -0.0008677857, -0.036285553, -0.00020853766, -0.00016144852, + 0.0013319502, 0.03972306, 0.0017893877, -0.0593996, -0.04062448, + -0.005630517, 0.5492949, -0.022737332, -0.0070768264, 0.023954922, + -0.06362417, 0.015277065, 0.04124337, 0.01919219, -0.03480561, + 0.0033635108, -0.04083302, -0.0294509, 0.016461022, 0.041976616, + 0.0049174526, 0.029154912, -0.0050990824, -0.0110794045, + 0.032767322, -0.0073997234, -0.03424054, -0.0033029674, 0.009142023, + 0.017564254, 0.0022737333, 0.020423237, -0.017362442, -0.019118194, + -0.035148688, -0.011671382, -0.04937634, -0.021701371, -0.013097511, + 0.03356111, -0.014732176, 0.04385345, 0.02055105, -0.010541243, + 0.013588583, 0.0069759213, 0.0060677733, -0.05872017, -0.020786496, + -0.0064848484, -0.015162706, -0.0145034585, 0.037805863, + 0.002166101, 0.056722246, -0.04638954, 0.029928518, -0.007749529, + 0.021593738, 0.035700303, 0.010837232, 0.005529612, 0.013938388, + 0.0011906828, -0.03349384, -0.004379291, -0.007971521, -0.11617566, + 0.014160381, -0.0051058093, -0.012249906, 0.03877455, 0.004769458, + 0.02775569, -0.00030944298, -0.02714353, 0.019246008, + -0.00057852385, 0.019071106, -0.008758582, -0.0080858795, + 0.037038982, 0.0074199047, -0.039379984, 0.032464605, -0.01099868, + 0.00442638, -0.031879354, 0.019905256, 0.032464605, -0.027681693, + -0.035337046, 0.026928267, 0.0038142211, 0.047533132, 0.0017153904, + 0.01219609, -0.061458066, 0.0035585945, 0.019461274, -0.011906828, + 0.0022602794, -0.005583428, 0.09441374, -0.0012983151, 0.010070351, + -0.008227147, -0.019521816, 0.021075757, 0.0072920914, 0.019044198, + -0.027681693, -0.0016212121, 0.06122935, -0.0110794045, + 0.0096869115, -0.014597637, 0.0067673833, -0.004708915, 0.011637747, + 0.029444173, 0.024291275, 0.049255252, 0.015129071, 0.019992707, + -0.02283151, -0.03871401, -0.0042582043, -0.010319252, -0.036124103, + 0.02380693, -0.02461417, -0.008543317, 0.016931914, 0.0041438453, + 0.029417265, 0.002805168, -0.0067808377, -0.00090142083, + -0.0064310324, -0.016676286, 0.006659751, -0.02201754, -0.036413368, + -0.020584686, -0.007305545, -0.01131485, -0.005085628, 0.00833478, + -0.0059197787, 0.024789074, -0.04277713, 0.016790645, 0.0060543194, + 0.032108072, 0.01810914, 0.028058408, -0.007581353, -0.5537482, + 0.016582107, -0.0077293473, 0.014732176, 0.073230356, -0.026168115, + 0.043483466, -0.010581604, 0.04939652, 0.010722872, 0.030251415, + -0.02236062, -0.051972967, -0.014658179, 0.010278889, 0.005771784, + -0.010278889, 0.002832076, 0.005381617, 0.007023011, -0.02176864, + 0.014416007, -0.030446498, 0.0064377594, -0.045400668, 0.0179746, + -0.020470327, -0.018687665, -0.023995286, 0.0030137056, + -0.030688671, 0.025266692, -0.004749277, 0.04636936, -0.046470266, + 0.01176556, -0.0087384, -0.003087703, 0.0022535522, -0.02700899, + 0.026417013, -0.029471083, 0.0011974098, 0.010029988, -0.024264365, + -0.027204074, -0.0325386, 0.0016750284, 0.06352999, -0.04159317, + -0.03649409, -0.012317176, -0.012216271, 0.023248585, 0.041061737, + 0.019595813, -0.014247832, -0.004439834, 0.00085433177, 0.004332202, + 0.01865403, -0.0039554886, -0.08877651, -0.006040865, 0.032464605, + -0.028132405, -0.08509682, -0.026201747, -0.025555953, -0.032646235, + 0.019703444, 0.041902617, -0.024721805, -0.03310367, -0.011570477, + -0.015801774, -0.010796869, -0.032733686, -0.030009242, 0.014981077, + -0.040241044, -0.080683894, -0.022905508, 0.010036716, + -0.0045541935, -0.0073728156, 0.054576326, -0.03471816, + 0.0070297373, -0.01978417, -0.035855025, 0.0007466994, 0.014133472, + -0.042541683, -6.7270216e-6, 0.022703698, -0.002865711, 0.037651137, + 0.056163903, 0.03125374, -0.050311394, 0.00961964, 0.010783415, + -0.010238527, 0.0041842074, -0.003107884, 0.020154156, + -0.0052807117, 0.019420912, 0.035686847, 0.030856848, -0.0030406138, + -0.01662247, 0.014920533, 0.039359804, 0.023786748, -0.0474255, + 0.017510436, 0.008240601, -0.010420156, 0.0060745003, -0.0180486, + 0.004029486, -0.032330066, 0.017100088, 0.012841884, 0.017833333, + 0.016461022, -0.011052496, 0.007366088, 0.053822897, -0.023558028, + -0.01187992, -0.021580286, -0.009175657, 0.00978109, -0.00096869114, + 0.02488998, -0.0069893748, 0.045434304, 0.0036594996, 0.035350498, + 0.008254056, 0.032215703, 0.027230984, 0.021943543, -0.006269584, + -0.008045517, -0.042212058, -0.057394948, -0.0020181064, + -0.018344587, 0.002172828, 0.020261789, 0.0054623415, -0.006040865, + 0.035646487, -0.06451887, -0.014691815, 0.006249403, -0.034543257, + 0.010265434, -0.016575381, -0.012061549, 0.01477254, -0.02380693, + 0.06783528, -0.022024268, 0.02039633, -0.012754433, -0.012835157, + 0.017086634, 0.00713737, -0.012969697, 0.014785993, -0.025825035, + 0.005448887, -0.012492078, 0.0017153904, 0.0024217279, -0.021674464, + 0.0074468125, -0.0071306424, -0.02226644, 0.015633598, -0.01878857, + -0.01104577, 0.02123048, -0.009673457, 0.04532667, -0.04788294, + -0.028737836, 0.05532975, 0.049181253, -0.002858984, -0.0058726897, + -0.0058054198, 0.039992142, 0.021297751, -0.029282724, -0.004419653, + -0.010581604, 0.028885832, 0.020181064, -0.0029195272, 0.05319056, + -0.026376652, 0.016595563, -0.03611738, -0.0125862565, -0.04194298, + -0.08883704, -0.011489753, -0.0068682893, 0.0015337608, + -0.022340437, -0.02640356, 0.024183642, 0.0024015466, -0.015546147, + -0.022730606, 0.04638954, -0.0015001257, -0.04510468, -0.04361801, + -0.0037200428, 0.011489753, -0.002542814, -0.0074199047, + 0.029598894, 0.00905457, -0.042595502, -0.058309823, 0.028724382, + -0.006424306, -0.027506793, -0.015418333, 0.04352383, 0.027109895, + 0.038444925, -0.041001197, 0.05113882, 0.034617253, -0.0030137056, + -0.022844965, 0.029894883, 0.01176556, -0.023255313, -0.0006121589, + -0.029767068, -0.012498806, -0.0061888597, -0.03656809, 0.058121465, + -0.088890865, 0.05002886, 0.02850239, 0.015875772, -0.023820383, + -0.0067404755, 0.017079907, 0.011920282, -0.023571484, -0.03588866, + 0.03321803, 0.017712247, -0.04509795, -0.024150006, -0.0108708665, + -0.020409781, 0.038788006, -0.0015270339, 0.017853515, -0.07197913, + -0.0029397083, 0.000941783, 0.004688734, 0.008872941, -0.009599459, + 0.0090411175, -0.03679681, 0.042299513, -0.011671382, -0.062742926, + 0.050264303, -0.004567648, 0.004930907, 0.020490507, -0.04079266, + -0.02689463, -0.018122597, 0.0011099585, -0.034697976, + 0.00029598895, -0.029262545, -0.017766064, 0.02682736, 0.005852509, + 0.0021593738, 0.049147617, -0.011644474, -0.007406451, 0.037294604, + -0.030910663, 9.41783e-5, -0.01976399, -0.0072920914, 0.016198669, + 0.0049510878, 0.0189702, 0.007480448, 0.006209041, -0.03825657, + -0.023833837, -0.0065992083, -0.0002488998, 0.006249403, + -0.042138062, -0.025206149, -0.07841689, -0.002811895, 0.0017221175, + 0.021102667, 0.020490507, 0.032377154, 0.014537094, -0.019999435, + 0.020786496, 0.012404627, 0.015485603, -0.010776688, -0.006760657, + -0.0032087893, 0.025152333, -0.015821954, 0.0032760596, -0.01903747, + -0.018990383, 0.0089402115, -0.015014712, -0.0043120207, 0.03105193, + 0.00023544575, 0.018963473, -0.03123356, + ], + }, + { + title: "Selfies", + query: "selfie", + minScore: 0.18, + showVideo: true, + textEmbedding: [ + -0.0011263192, 0.040301856, -0.02074704, -0.11898843, 0.03114752, + -0.038252916, 0.0115447715, -0.0012581225, 0.029248353, 0.06059357, + 0.04173372, -0.017290197, -0.024018157, 0.010490346, -0.013917232, + 0.4287502, 0.05462049, -0.034514494, 0.07627218, -0.03058436, + 0.02120236, 0.03208212, 0.033400156, -0.028265819, 0.010819853, + 0.017481912, -0.017937232, -0.03074612, 0.061078854, -0.02775658, + 0.0045052767, -0.040799115, 0.0027618785, 0.0077644135, 0.008447395, + -0.038318817, 0.018470436, -0.015582745, 0.025066594, -0.0488511, + -0.058676437, 0.032261856, -0.041805614, 0.075205766, 0.040613394, + -0.011484861, -0.008195769, -0.002935619, 0.036275864, 0.007171299, + 0.005332043, 0.04541223, 0.0035227428, 0.026061108, -0.062486753, + -0.044771187, 0.03306466, 0.0012760957, -0.008657081, -0.01927923, + 0.0028517444, -0.00859717, -0.038079172, -0.0409429, -0.086433016, + 0.030069128, 0.006236693, 0.05291903, 0.034460574, 0.072869256, + 0.01470206, 0.031824507, -0.033382185, 0.014372553, 0.011161344, + 0.003025485, -0.06787271, -0.011029541, 0.086672656, 0.0046191067, + 0.034748144, -0.021633716, -0.018116964, 0.031518962, 0.02063321, + -0.0005451864, -0.037054703, 0.030308772, 0.022011153, -0.020004148, + 0.023838425, -0.004810821, -0.0072551733, 0.020111987, + -0.0012641135, 0.0024443523, 0.01323425, 0.021118484, -0.050055303, + -0.03371768, -0.013461911, 0.021328172, 0.009340062, -0.018021107, + -0.004379465, 0.032759115, -0.060647495, 0.015151391, -0.034400664, + 0.02549795, -0.022035118, -0.015391031, 0.12818472, -0.08661874, + -0.024036132, 0.071635105, -0.02102862, -0.087325685, -0.0015876308, + -0.018626204, -0.035227433, -0.021406054, 0.037348267, -0.009933176, + 0.02047145, 0.0014797917, 0.039013777, -0.04082308, -0.024868889, + -0.0091004195, 0.0050864094, 0.023161437, -0.018338634, -0.01685285, + 0.0063684965, -0.008279645, 0.025414074, 0.004307572, 0.015492881, + -0.020657172, -0.024437532, -0.007752431, 0.01153878, -0.02080695, + -0.008495322, -0.036785107, -0.021519886, -0.029014701, 0.00661413, + -0.047509104, 0.024563344, 0.027708651, -0.035485048, 0.0050684363, + 0.03207613, -0.019375086, -0.009999079, -0.002120835, -0.07661966, + -0.016739022, 0.027522927, 0.055093784, -0.016181853, 0.008165814, + 0.0416139, -0.075697035, -0.019261258, -0.43341124, 0.052601505, + 0.035790592, -0.010694042, 0.07215632, -0.009094428, -0.01707452, + 0.022250796, -0.045370292, 0.013072493, 0.0041877506, 0.0365874, + 0.04126642, -0.01696668, -0.015115443, 0.027061615, -0.028930828, + 0.04953408, 0.05953915, 0.044375777, -0.04473524, 0.058460757, + -0.060240105, 0.014426471, -0.0213761, 0.009124383, -0.03601226, + -0.04032582, -0.009963132, 0.003217199, 0.02758284, -0.03369372, + -0.009795383, 0.077338584, 0.021975206, -0.01600212, -0.02601318, + -0.045885526, 0.0046071247, 0.025827456, 0.061444305, 0.049929492, + 0.044603437, 0.007171299, -0.004331536, -0.027798515, 0.01712245, + 0.0054458734, -0.008069957, -0.06565003, 0.008788884, 0.048713304, + 0.0040679295, 0.009519793, -0.015187336, 0.026091065, -0.03308862, + 0.001815291, 0.024293747, -0.034975808, 0.02951196, 0.057034887, + -0.01295267, -0.033507995, -0.0050085257, 0.02380248, -0.09486843, + 0.00876492, 0.012766948, 0.029493988, -0.03177658, 0.019225309, + -0.025875386, 0.01176045, -0.023149453, -0.031932347, 0.015486889, + -0.025743583, 0.030470528, 0.043944422, -0.04116457, -0.075020045, + 0.0061468272, 0.0092741605, -0.009376008, 0.022640213, 0.04531038, + -0.022580303, 0.019992165, -0.017889302, -0.050444722, -0.009987096, + 0.0007488825, -0.050366838, 0.012677082, -0.014228767, 0.0014797917, + 0.051708832, -0.024185907, -0.011604683, -0.054290984, 0.058394857, + -0.028601317, 0.004427393, 0.016367575, 0.056339923, -0.043602932, + -0.007818333, 0.002324531, -0.0070275124, 0.042081203, -0.01714042, + 0.005308079, 0.04495691, -0.0101069175, -0.025258306, -0.043680813, + 0.030488504, 0.0003414904, -0.026384626, 0.0055776765, 0.015175354, + 0.041913453, -0.0587663, 0.008477349, -0.04929444, 0.04883912, + -0.043507077, -0.013995115, -0.0037983318, 0.09000968, + -0.0077883773, -0.024335682, 0.0062187198, -0.040194016, -0.0264685, + 0.03414904, -0.05403337, -0.018051064, 0.00040739204, 0.036281858, + -0.020117978, -0.050085258, -0.0228499, -0.04105074, -0.028199919, + 0.021957234, 0.029877415, 0.027277295, -0.059275545, 0.084264256, + -0.03755196, 0.009250196, 0.00046730263, -0.026606295, -0.029032674, + 0.011736486, -0.0040799114, -0.03198627, -0.011808378, 0.013282179, + 0.008471359, 0.007105396, 0.025360154, -0.01599613, 0.010058989, + 0.06759114, -0.020070048, -0.05330845, -0.018506382, 0.008681046, + -0.075097926, -0.016810913, -0.048587494, -0.0036845016, + -0.04048758, -0.040301856, 0.046892025, -0.0206392, -0.015696576, + -0.01752984, -0.11967141, 0.01940504, -0.012060003, 0.007542744, + 0.024389602, 0.012617171, 0.0044873036, 0.00024563345, -0.06508687, + -0.0065002996, -0.05510577, 0.043351308, -0.073037006, + -0.0032531454, -0.0022286742, -0.015558782, 0.032717176, + -0.01012489, 0.07116181, 0.06544633, 0.0058592563, -0.013881286, + 0.013995115, -0.03325637, 0.02165768, 0.009813355, 0.014252732, + -0.042201024, 0.0027379142, -0.059413336, 0.005302088, 0.014468409, + -0.038768146, 0.0030674224, 0.02618692, -0.05269736, 0.028092079, + -0.043327343, -0.00565556, -0.013449929, 0.012413475, -0.087115996, + 0.03048251, -0.086552836, -0.007500807, 0.024808979, -0.013575741, + 0.058406837, -0.010214756, -0.015211299, -0.025743583, -0.11988111, + 0.0047029816, -0.06276234, 0.02550394, 0.013431956, 0.031015715, + 0.020651182, 0.077284664, -0.0051942486, 0.009603669, 0.015001614, + -0.014294668, 0.0037084657, 0.007590672, 0.024802987, -0.016373565, + 0.06445181, -0.036209963, 0.005206231, -0.03969676, -0.031524956, + -0.009747454, -0.056807227, -0.02108853, 0.022700125, 0.01832665, + 0.0025941287, 0.012407484, 0.032213926, 0.045268446, 0.027774552, + 0.020375593, -0.0032291813, -0.030308772, 0.007452878, -0.012245725, + -0.05944329, -0.008387484, -0.024581317, 0.027936311, -0.019482926, + -0.01814692, -0.062564634, -0.048108205, -0.057358403, -0.023323195, + -0.049623948, -0.05750219, -0.01786534, 0.0009705517, -0.012754966, + -0.050294947, 0.02753491, 0.09078852, -0.019434998, -0.008974607, + -0.03320245, 0.059245586, -0.048689343, -0.0057094796, 0.007291119, + -0.0055716857, 0.025456011, -0.0036964838, 0.03136919, -0.023419052, + 0.057238586, -0.0349818, -0.01328817, -0.032765105, -0.032363705, + -0.0046610446, -0.018093, 0.016774967, 0.003930135, 0.010718006, + -0.0008806857, 0.044243973, 0.04494493, 0.0025641737, 0.010975622, + 0.00605097, -0.053146686, -0.043033782, 0.028775059, 0.050420757, + -0.0063325497, -0.00938799, 0.034634314, -0.019177383, 0.023928292, + 0.040739205, -0.047119685, -0.024785014, 0.026708145, 0.0089566335, + 0.0398705, 0.025605788, -0.014947694, -0.0062906127, -0.014342598, + 0.0042776163, -0.0048707314, 0.015846353, -0.022646205, + -0.014324623, -0.0026121018, 0.01780543, 0.037641827, 0.027127517, + -0.009939168, -0.050019357, -0.013030554, + ], + }, + { + title: "Wallpapers", + query: "high resolution hd wallpaper", + minScore: 0.18, + showVideo: false, + textEmbedding: [ + 0.032093976, -0.022385579, -0.00020051774, 0.005241311, -0.05675209, + 0.012348551, 0.039089818, -0.040042277, 0.020720165, 0.101189055, + -0.02948168, -0.02221291, -0.04773436, 0.0058929934, -0.014816034, + 0.49386406, 0.009563583, -0.0051299124, 0.03526327, 0.04364046, + -0.015127949, 0.013312151, 0.02672456, -0.014086371, 4.45595e-5, + -0.005029653, 0.005230171, -0.0012810855, 0.012348551, -0.032238796, + 0.034138143, -0.016425746, -0.034266256, -0.029414838, 0.004812426, + 0.011462931, -0.03292947, -0.050123867, 0.0028295282, 0.018514471, + 0.03807609, 0.014337019, 0.031481285, 0.037268452, -0.009312935, + -0.012209302, -0.033285942, 0.01945579, 0.0030244759, 0.020174311, + -0.007558405, 0.0059876824, 0.009658271, 0.007614104, -0.0013757745, + -0.0177681, 0.02490319, -0.018787399, -0.047478143, 0.0024452023, + 0.022107081, -0.019316543, -0.022942571, 0.064511016, -0.03565874, + 0.04229253, -0.022859022, -0.027036475, -0.0076363836, 0.05596116, + 0.03557519, 0.05645131, -0.013735466, 0.0093407845, -0.0032249936, + -0.004556209, -0.072086126, -0.043874394, 0.0070292605, 0.068293, + -0.007491566, -0.049021017, -0.0035703299, -0.006043382, + -0.023365887, 0.062132653, 0.044966105, -0.0012532359, -0.008750372, + -0.006082372, -0.05675209, -0.0057815947, -0.033274807, + 0.0031525844, -0.024752801, -0.0045116493, -0.003096885, + -0.030021962, -0.019355532, -0.027827406, 0.014743623, 0.021789595, + -0.002049737, 0.08173326, 0.009351925, -0.042838387, -0.06335803, + -0.0041551734, 0.031230638, 0.013223032, 0.041451473, 0.0005569937, + 0.12252748, -0.04196391, -0.0048291357, 0.1443115, -0.011964225, + 0.0019884675, -0.014242329, -0.028829996, 0.035369102, 0.030551108, + 0.037920132, 0.0424652, 0.015478856, 0.006862163, -0.07678716, + -0.027147874, 0.020631047, -0.02044724, 0.020057343, 0.027465362, + -0.002216835, -0.017244525, -0.017996468, -0.0032138538, + 0.011368242, 0.03188789, -0.041234244, -0.014426137, 0.005625637, + -0.019299833, -0.019344391, -0.010265394, -0.008143248, + 0.0090010185, 0.026362512, -0.0127774365, -0.011869536, + -0.019349962, 0.00793716, 0.0035536198, 0.05486945, 0.0034477913, + -0.023722362, -0.034010038, 0.0047678663, 0.017055148, -0.021388559, + -0.015840901, -0.025120417, -0.015378597, 0.013451398, 0.014877303, + 0.018330663, -0.06190985, 0.012226013, -0.49835342, 0.033302654, + -0.040287357, -0.010204125, 0.0561951, -0.024117827, -0.009056718, + 0.0024674824, 0.012365261, 0.0026958496, 0.005920843, 0.010744409, + 0.058796257, 0.018341804, -0.014559816, -0.0038766763, 0.015373027, + 0.026323525, 0.0065836655, 0.0025176115, 0.021388559, 0.007820192, + 0.009530162, 0.01413093, -0.017038438, -0.044648618, 0.011490781, + -0.015573544, -0.045528665, 0.028206162, 0.049700547, 0.047884747, + 0.005085353, 0.057281237, -0.052491087, 0.00707382, 0.0379257, + 0.024262648, -0.016414605, -0.046776332, -0.010967206, -0.033831798, + -0.01248223, -0.038616374, 0.030734912, -0.016726522, 0.030902011, + -0.003013336, 0.04066611, -0.046553534, -0.0009134697, 0.027309401, + 0.009947908, -0.024635833, -0.02480293, 0.017523022, -0.029392557, + -0.026100725, -0.05679665, 0.056423463, 0.006962422, -0.014119791, + -0.029024944, 0.034477912, 0.031642813, -0.038772333, -0.0042164424, + 0.010989486, -0.008934179, -0.03391535, -0.022831174, -0.0058874236, + 0.00013367849, -0.028022354, -0.013612926, -0.07437537, + -0.016481444, -0.012866555, -0.026407074, -0.011563189, + -0.047622964, -0.06853808, 0.01962289, -0.00980866, 0.017110847, + -0.0231041, 0.005408409, -0.032127395, 0.00531372, -0.0134124085, + -0.032461595, -0.002851808, 0.011680159, -6.126931e-5, 0.020023923, + 0.014905152, -0.018965635, 0.009936769, -0.023482855, -0.039886322, + -0.028880125, -0.007591825, 0.017127557, 0.0039379457, -0.016726522, + -0.020848276, 0.0020107473, 0.0014871733, 0.0025398915, + -0.018319523, -0.0135182375, -0.06772486, -0.043985795, -0.03196587, + 0.009106847, 0.031213928, -0.014699064, 0.05031881, -0.0011084175, + 0.010354513, 0.026340235, 0.02676355, 0.019968225, -0.014905152, + 0.0074302964, -0.050497055, -0.0018157995, -0.018792968, + -0.020068483, -0.01665968, -0.008299206, 0.06564171, -0.0021444259, + 0.021020943, -0.014437278, 0.024986738, 0.025009017, -0.004327841, + 0.024652543, -0.031169368, 0.009530162, 0.024524432, 0.00013924843, + 0.08978739, 0.022574956, -0.0054418286, -0.0062717493, -0.004506079, + 0.013445828, -0.044553928, 0.020113043, -0.06200454, 0.0068510226, + -0.04795159, -0.06583666, -0.031770922, 0.0006294029, -0.050491482, + -0.0017601001, 0.053048085, -0.07610762, 0.0155122755, -0.008254647, + -0.027320541, 0.028551497, -0.009491173, 0.0056479163, + -0.0071016704, 0.016425746, 0.0070571103, -0.008388326, 0.012810856, + 0.035318974, 0.018397503, -0.0025788809, 0.059659597, -0.0021945552, + -0.02670228, -0.023644384, 0.005831724, -0.025471324, 0.023700083, + -0.025070287, -0.0097640995, -0.007302188, -0.022491407, + -0.014648935, 0.010387933, -0.00892304, 0.06877758, 0.010527181, + -0.008187808, -0.0020998663, 0.0022056953, -0.0030801753, + -0.006800893, 0.0020218873, 0.04506636, 0.042843957, -0.0009970188, + -0.043228284, -0.038315598, -0.009608141, 0.02392845, 0.03625472, + -0.02561614, 0.012120184, -0.0014927431, -0.02485306, 0.0248085, + -0.008182238, -0.11733073, 0.04351235, 0.011930806, 0.0059876824, + 0.024413036, -0.01686577, 0.0051466217, 0.019628458, -0.019155014, + 0.009591431, -0.01784608, 0.036978815, -0.023388166, -0.07350647, + 0.03270667, 0.005096493, 0.011746998, 0.015679374, 0.03409359, + -0.04006456, 0.0019996073, -0.10097182, -0.060623195, + -0.00049572444, 0.016442455, -0.06061763, -0.02830085, -0.007190789, + -0.054824892, 0.04801843, -0.011992075, -0.07789, 0.03106911, + -0.0056367763, 0.003737428, 0.008544284, 0.016392326, -0.013150621, + -0.05674095, 0.013702045, 0.00797615, -0.027977794, -0.032316778, + 0.057698984, -0.02484192, -0.007903741, 0.015712794, -0.052897695, + 0.008327057, -0.0178795, -0.014743623, -0.004701027, -0.013757746, + 0.06597034, 0.03799811, 0.060962964, 0.019004626, -0.024385184, + 0.006087941, -0.01336228, -0.033982188, 0.03980277, 0.03016121, + -0.0328125, 0.032311205, 0.013835724, 0.012593628, -0.016564993, + -0.015412017, -0.0017601001, -0.06927331, -0.0041050436, + 0.065608285, 0.00060155324, -0.010120575, 0.009658271, -0.029370278, + 0.0031470144, -0.00444481, -0.025866788, 0.017378204, 0.008087549, + -0.0195839, -0.002049737, 0.016208518, -0.012103474, -0.017606571, + 0.06707318, -0.012576918, 0.020202164, 0.021717185, 0.0134959575, + 0.03835459, 0.023315758, -0.006750764, 0.009847648, -0.00021722756, + 0.010794538, -0.012844276, 0.012604767, 0.00971954, -0.05551557, + -0.0033642421, -0.07514402, 0.03486781, 0.013401269, 0.08324828, + -0.042598877, -0.10107208, 0.04008684, -0.0031637242, 0.0026234405, + -0.009212676, -0.031731933, -0.028601626, -0.0345169, 0.036728166, + -0.00064054277, 0.014816034, -0.045924135, 0.0036037494, + -0.011674589, 0.004272142, -0.036165603, 0.041791238, -0.032366905, + 0.023571974, -0.022095941, -0.027716009, 0.03546936, -0.0002840668, + 0.0011418371, 0.0044949395, -0.0117804175, + ], + }, + { + title: "Food", + query: "photo of food", + minScore: 0.2, + showVideo: true, + textEmbedding: [ + -0.012083651, 0.016745469, -0.007044219, -0.033847954, -0.020761909, + 0.01960847, 0.0015722477, -0.0116099175, -0.018303985, -0.039670076, + 0.027504038, 0.026577167, 0.017823385, -0.006968696, -0.00054925686, + 0.5646292, 0.018441299, -0.0029522558, -0.016690543, -0.043597262, + 0.015530238, -0.021654453, 0.007071682, -0.021935945, -0.016676812, + -0.0141914245, -0.040198736, 0.023013864, 0.011094988, -0.006103617, + -0.009577666, -0.0047854003, 0.016072629, -0.014088438, -0.01623054, + 0.03081331, 0.0015035907, -0.013195896, 0.033078995, -0.003844798, + 0.046000265, -0.0061997366, -0.013662764, -0.0039065895, + -0.024757752, -0.052172538, -0.02633, 0.00061791396, -0.0044009206, + -0.0244282, 0.012056189, 0.0056161513, 0.016347257, -0.006831382, + -0.030909428, 0.01824906, -0.046158176, -0.015420386, -0.010133789, + 0.042663526, -0.0068451134, -0.008781244, 0.0029385241, -0.0399859, + -0.054431356, 0.04163367, -0.00893229, -0.0073463107, 0.04817669, + 0.049240876, 0.032866158, -0.008389899, -0.02981778, -0.019670261, + 0.017260397, 0.002279416, -0.07170548, 0.017596817, 0.012275891, + 0.0028904642, -0.009742443, -0.009357964, 0.011129317, 0.026790004, + -0.006069288, 0.013786348, 0.0063713794, -0.0044215177, + 0.0052110744, -0.028774194, 0.054905087, 0.029206732, -0.006851979, + 0.0040233065, -0.010978271, -0.0033436012, -0.052337315, + 0.028478969, -0.029364644, -0.017280994, -0.020720715, -0.00521794, + 0.03899037, 0.035736024, -0.0045245034, 0.0014417992, -0.07982762, + -0.011671708, 0.023109982, 0.0014143364, 0.0009886624, -0.019471155, + 0.12529236, -0.04319905, 0.0256091, -0.03793305, 0.013271418, + -0.0105457315, -0.03239929, 0.012028726, -0.03534468, -0.009200052, + 0.0021215046, -0.0411668, -0.025355069, 0.011836486, 0.034452137, + 0.0034191238, -0.022368485, 0.006790188, -0.001139708, -0.021702511, + 0.0074630277, 0.0008650796, -0.013031119, -0.017157411, 0.02173684, + 0.023295356, -0.005286597, 0.014143364, -0.013978587, 0.0003844798, + 0.009941549, -0.015001577, -0.0030209126, -0.003130764, + -0.0036182296, 0.01395799, 0.00801915, -0.0167592, -0.023432672, + 0.024853874, 0.03587334, -0.027600158, 0.002162699, 0.013443061, + 0.02809449, 0.009893489, 0.006618545, -0.010943944, 0.026549704, + -0.016608154, 0.025739549, 0.021427885, 0.043020543, -0.054191053, + -0.030023754, -0.5698471, -0.008657661, -0.0045931605, 0.016052032, + 0.08418048, 0.00749049, -0.01946429, -0.017603682, 0.0021695646, + 0.009763041, 0.01592845, -0.009701249, -0.0036250954, 0.0007964224, + 0.011637379, 0.005471972, 0.00083761674, -0.011760962, 0.019951755, + -0.027016573, -0.030566145, 0.045526527, -0.0017988163, + -0.005135552, -0.050092224, -0.02023325, 0.0066460078, -0.015180087, + -0.03463751, -0.018812047, -0.00787497, -0.0068725767, 0.040164407, + 0.031596, 0.00275315, 0.0007620939, 0.009783638, 0.003412258, + 0.020013547, 0.050600287, -0.0040988293, 0.021448482, -0.03798111, + 0.0060280943, -0.0066666054, -0.0037418124, -0.0048334603, + -0.0017095619, 0.027050901, -0.06111169, 0.0013319479, -0.013855004, + 0.021805497, -0.007497356, 0.061125424, -0.012399473, 0.014417993, + 0.0022382217, -0.009632592, 0.013930527, -0.04257427, -0.007085413, + -0.06352842, -0.020864895, 0.030854503, -0.014589635, -0.10236088, + -0.014170826, -0.01000334, -0.02037743, 0.004867789, 0.007984822, + -0.0056916745, -0.016360989, -0.0009886624, -0.0090078125, + 0.026247611, 0.016601289, 0.0041949493, 0.028760463, -0.009344232, + -0.059395265, -0.0018949362, -0.008369301, -0.004908983, + -0.017562488, 0.07001652, -0.01661502, -0.025162831, -0.004016441, + 0.02028131, 0.0069549647, 0.009714981, -0.0013250822, -0.015646955, + -0.009522741, -0.000116717085, 0.0065155593, 0.0032406154, + 0.007222728, -0.05040118, -0.0046412204, -0.017795922, -0.009234381, + 0.0075866105, 0.024174167, 0.02633, 0.0009955281, -0.00046000263, + 0.008204524, 0.016944574, 0.0029453898, -0.02254013, 0.029440168, + 0.014301275, 0.025444325, -0.020356832, 0.029755991, 0.03331243, + -0.031190922, -0.010875286, 0.020658923, 0.002300013, -0.03776141, + -0.00080328813, -0.040535156, -0.03139003, -0.0040027094, + 0.0072295936, -0.006577351, 0.0057260026, 0.029913902, 0.0082731815, + 0.025739549, 0.0049295803, 0.03047689, -0.016381586, 0.009817966, + -0.0132920155, 0.016827857, -0.0010710509, 0.02703717, + -0.0020459818, -0.013518585, -0.010037669, 0.022581322, + -0.0118227545, -1.3731422e-5, -0.02024698, -0.070950255, + 0.0020253847, -0.031781375, 0.0048334603, -0.026240747, 0.032859292, + -0.043803234, 0.045354884, -0.0061173486, 0.0026295672, + 0.0123926075, -0.036422595, 0.023878943, -0.0005904511, 0.011802156, + -0.003844798, -0.019581007, 0.07515894, -0.020713849, + -0.00033641982, -0.00565048, 0.015386058, -0.031753913, 0.02733926, + -0.010477074, 0.018880704, 0.044565327, 0.010971406, 0.026288806, + -0.022066394, 0.023652375, -0.016409049, -0.080521055, 0.020775642, + 0.005643614, 0.011156781, -0.022306696, 0.017583085, -0.027085228, + 0.009426621, 0.017823385, -0.0046343547, 0.011033197, -0.05370359, + 0.033641983, -0.00083761674, -0.021908483, -0.004085098, + 0.014163961, 0.0060418253, -0.008657661, 0.011115585, 0.005266, + -0.004270472, 0.002732553, -0.033113323, 0.007003025, 0.0036182296, + 0.035811547, -0.041606206, 0.008300644, -0.009584532, -0.115179166, + -0.017054426, -0.009872892, -0.012557385, 0.0027600157, + -0.024084914, 0.022045797, 0.019629067, -0.014562172, 0.0021901617, + 0.023103116, 0.034761094, -0.023830881, 0.04333637, 0.0147612775, + -0.0057809283, 0.012447534, -0.022203708, 0.013374405, 0.011136183, + -0.00029522556, 0.048327737, 0.037747677, 0.0027874787, 0.008410496, + 0.0029522558, 0.009344232, 0.007284519, 0.023597447, 0.042491883, + 0.0014761279, -0.010133789, -0.013182165, -0.01261231, -0.011335288, + 0.003748678, -0.02024698, -0.023995658, -0.030167934, -0.025217757, + -0.023501327, 0.0018880705, 0.023782821, -0.0058701825, 0.010662449, + -0.012866342, 0.013731421, 0.012481863, -0.008554676, -0.0046137576, + 0.013024253, 0.019443693, -0.007318848, 0.011706037, -0.006440037, + -0.034040194, -0.008589004, 0.029460765, 0.011383349, 0.035179902, + 0.009103932, -0.01989683, -0.04038411, -0.028039563, -0.017514428, + 0.004936446, -0.0047579375, 0.0049433117, 0.019965487, 0.014012916, + 0.013868735, 0.01487113, -0.07837895, 0.017610548, -0.0445104, + 0.020658923, -0.0017919504, 0.0121797705, -0.02048728, + -0.0024304616, -0.008197659, -0.02671448, -0.010683046, + -0.009783638, -0.019972352, -0.009591398, 0.009790503, 0.058392867, + 0.0005767197, -0.0031925554, -0.019574141, -0.010133789, 0.01763801, + 0.055049267, 0.0038173355, -0.024078049, 0.014315006, 0.013092911, + -0.024325214, -0.010751703, 0.02333655, -0.011912008, -0.009145127, + -0.006611679, 0.00260897, -0.05136238, -0.13511032, -0.01999295, + -0.1386805, 0.017802788, 0.004442115, 0.0072089965, -0.0012701565, + 0.025018651, -0.009996475, 0.0027668814, -0.00061791396, + -0.007044219, -0.010696777, -0.013312614, 0.029604943, -0.014424859, + 0.0148779955, -0.041283518, 0.02476462, -0.005197343, 0.00067283964, + -0.010030803, -0.024922531, -0.00048059976, -0.029584348, + 0.014397396, 0.007689596, -0.040693067, + ], + }, + { + title: "Celebrations", + query: "celebration with friends", + minScore: 0.2, + showVideo: true, + textEmbedding: [ + 0.026342282, 0.01797094, -0.003183855, -0.0350773, 0.01633098, + -0.010189708, -0.041939057, -0.044670034, -0.029567307, + -0.034555804, -0.013538245, -0.00022643796, -0.00014409689, + 0.016029064, -0.008419375, 0.4912194, 0.031111203, 0.007348941, + 0.039407067, -0.010114229, 0.025381636, -0.0009126136, -0.007877297, + 0.0039386484, -0.0057089813, 0.001605651, -0.0016674068, + 0.017922906, -0.014828255, -0.018464986, 0.006614733, -0.05048194, + -0.026932394, -0.025264986, 0.020564685, -0.05590959, 0.017476894, + -0.032106157, 0.081654906, 0.0328335, 0.032806057, -0.01785429, + -0.037245613, 0.016962262, -0.023384865, -0.027028458, -0.02778325, + -0.018533604, 0.0028064582, 0.02000202, 0.0020173562, -0.002312412, + 0.016193746, 0.104593754, 0.01206983, -0.0011870839, 0.024791526, + 0.017168114, 0.0046042386, -0.05118184, -0.03469304, 0.0052217967, + 0.009215339, -0.036737844, -0.10656994, 0.012268821, 0.015946722, + 0.009229062, -0.017607268, 0.014917458, 0.0107729575, -0.018101314, + -0.019288396, 0.04593946, 0.013133402, 0.008151767, -0.029025229, + -0.010169122, 0.034761656, 0.075623415, -0.0019556005, 0.015946722, + 0.009078104, 0.038041577, -0.0042611505, 0.0004871847, 0.022925127, + 0.030610293, -0.029450657, 0.028538045, -0.034500908, 0.016454492, + 0.006504945, -0.006511807, 0.023062363, 0.03228456, -0.109788105, + -0.0023673058, -0.013895056, -0.062325332, 0.020530375, 0.000535217, + -0.007842988, -0.003801413, 0.006786277, 0.025621798, -0.042364486, + -0.012241374, 0.0052355197, 0.0058256313, -0.023179013, 0.010498487, + 0.11106439, -0.041486178, -0.033855908, 0.005715843, 0.0059560044, + -0.04880081, -0.027344098, 0.042289004, 0.027186278, -0.027028458, + 0.052567914, -0.034500908, 0.022726137, -0.02632856, 0.03277861, + -0.07358547, -0.008240969, -0.035084162, -0.024647428, -0.003609284, + -0.005180626, 0.027947934, -0.024503332, -0.0032318872, + -0.027323514, -0.012543291, -0.017819982, -0.019665794, 0.004274874, + -0.025964886, -0.016674068, -0.047373563, -0.0012694249, 0.0296222, + -0.01195318, 0.004590515, -0.017785672, -0.0040415744, 0.012783452, + -0.010697478, -0.088667616, 0.01750434, 0.020448033, -0.015493846, + -0.0060726544, 0.013414733, -0.035845812, 0.023858327, -0.029066399, + -0.010539657, 0.022568317, 0.020667609, 0.021758629, -0.040388297, + 0.018005248, -0.49567956, -0.019370737, -0.028901717, 0.008920283, + 0.05122987, -0.04781272, 0.058935624, 0.045706157, 0.008879113, + 0.024777802, 0.0050159437, 0.000102926344, -0.07427851, + -0.021093039, -0.014004845, 0.014327347, -0.022925127, 0.010011302, + -0.030335825, 0.018355198, 0.009002624, 0.016159438, -0.030877903, + 0.031934615, 0.0019693242, -0.044217158, 0.012255097, -0.009606459, + -0.015384058, 0.0067725535, 0.00030877904, -0.032092433, + 0.036586884, -0.014533199, 0.024510194, -0.011829668, -0.0058462163, + -0.04714713, 0.016694652, -0.04055298, 0.033505954, -0.004020989, + -0.0053658937, -0.015363472, 0.00406216, 0.03762301, -0.063533, + 0.0048238146, -0.022355603, -0.0031975785, -0.015987892, + -0.009771141, -0.0056746723, -0.04106075, 0.016118266, 0.025765896, + 0.035084162, 0.01633098, 0.03136509, -0.05513421, 0.039324727, + -0.012900102, 0.0027858731, -0.010663169, 0.018663976, -0.005633502, + 0.004638547, -0.020331383, -0.032476693, 0.009359436, -0.032408074, + -0.01578204, 0.005715843, -0.032593343, 0.028757619, -0.049871244, + 0.046824623, -0.07769567, 0.029601617, 0.008899698, 0.040038347, + -0.05852392, 0.04053239, -0.04418971, -0.02055096, -0.021971343, + 0.029423213, 0.007040162, 0.037986685, 0.014601817, -0.013133402, + -0.03594188, 0.03199637, -0.033931382, -0.028421395, -0.04235076, + 0.008227246, 0.0550107, -0.031241575, -0.0038219984, -0.047455907, + 0.035056714, 0.01930212, -0.02812634, 0.0036024218, 0.055353787, + -0.010670031, 0.040882345, 0.018787488, 0.029594755, -0.0191443, + -0.034713626, -0.019583452, 0.004508174, 0.019748135, 0.043084968, + -0.0004871847, 0.002332997, 0.037163273, -0.053816754, 0.0052149347, + 0.031145513, 0.039805047, -0.027851868, -0.017030878, -0.063210495, + -0.012364885, 0.02750192, -0.019164884, 0.013421595, 0.027460746, + -0.0044326945, -0.030376995, 0.018547326, -0.022787893, 0.00906438, + 0.004199395, 0.02437982, 0.02605409, 0.023233907, 0.006882341, + 0.031166095, -0.02895661, -0.0039386484, 0.013565692, -0.018190516, + 0.016042786, -0.031474873, -0.020873463, -0.03955116, 0.029196773, + 0.044409286, 0.016831888, 0.0009194754, -0.0063814335, -0.06200283, + 0.041740064, -0.04781958, -0.01933643, 0.011781636, 0.013764682, + -0.014903734, -0.028146923, 0.06463775, -0.0031358227, 0.038501315, + 0.059381638, -0.025999194, -0.023508377, 0.049452677, -0.029018367, + 0.018471848, -0.00203108, -0.02977316, -0.030829871, 0.07470394, + 0.01781312, 0.014862564, 0.026534412, -0.040470637, 0.025985472, + -0.06216065, -0.026589306, -0.02934087, 0.019357014, -0.022863371, + -0.025930578, -0.084303536, 0.038309183, -0.09027327, 0.0066764886, + 0.016749546, -0.033396166, 0.01418325, 0.0047414736, -0.044553384, + -0.031769935, 0.043942686, -0.013798991, 0.02378971, 0.031241575, + 0.012913825, 0.029231083, 0.025223818, -0.022609487, 0.006312816, + 0.049795765, 0.0027035319, 0.020221597, 0.026616752, -0.03675157, + -0.073194355, -0.009407467, 0.0020653887, -0.006779415, + -0.0150684165, -0.061556816, -0.011507166, -0.023096673, + -0.025367914, 0.0039592334, -0.0063265394, 0.025539456, 0.027097076, + -0.09998951, 0.017689608, 0.03093966, 0.005969728, -0.0029162464, + 0.016619174, 0.008405651, 0.028421395, -0.1283766, 0.04167831, + -0.018238548, -0.0108552985, -0.03868658, -0.044937644, 0.022046823, + 0.009098689, -0.0033553988, -0.02437982, -0.018705148, + -0.0015576186, 0.018684562, -0.023796571, 0.01890414, 0.043544706, + -0.030075077, -0.030720085, -0.022142887, -0.0152468225, + 0.006902927, 0.0066559035, 0.010361252, 0.017133806, -0.011136631, + 0.011479719, 0.015617357, 0.027323514, 0.010656307, -0.017964078, + -0.012687388, -0.03907084, 0.004624824, 0.023062363, 0.0024222, + -0.045157216, 0.0034171545, -0.0027035319, -0.041520488, + -0.02848315, -0.030322101, 0.010368113, -0.034720488, -0.014725328, + 0.034816552, 0.047256913, 0.03532432, -0.009640767, -0.030150557, + 0.017600406, -0.019631485, -0.00691665, 0.006079516, 0.05363149, + 0.01277659, 0.008954592, 0.007822402, 0.02562866, -0.017524926, + -0.0154183665, -0.018389506, -0.025361052, -0.043819174, + 0.0038631689, -0.00929768, -0.026472656, 0.007486176, -0.045610093, + 0.02891544, -0.046941273, -0.0001578204, 0.033341274, -0.047188297, + -0.016186884, -0.007897881, 0.014766499, 0.014979214, -0.015672252, + 0.066078715, 0.031022001, 0.009798588, -0.028297883, -0.06480929, + -0.007122503, -0.018272856, 0.003924925, -0.02610212, -0.102102935, + 0.075760655, 0.012275682, 0.006779415, 0.008590919, 0.0196109, + 0.02055096, 0.0050433907, -0.007211706, 0.0054207873, 0.021168519, + -0.016536832, -0.02660303, 0.040422607, -0.006902927, -0.0059354194, + -0.06325853, -0.0034994958, -0.019528558, -0.021223411, + -0.003520081, 0.017998386, 0.06544057, -0.04656388, -0.021017559, + -0.10281656, + ], + }, + { + title: "Sunset", + query: "photo of setting sun", + minScore: 0.2, + showVideo: true, + textEmbedding: [ + 0.0338652, 0.003039766, 0.02503756, -0.016871026, 0.031680975, + -0.0032925399, -0.04358078, 0.026450502, 0.039121594, 0.020092271, + 0.01947654, -0.027597705, 0.010940565, -0.031583752, 0.02112281, + 0.522237, 0.003124024, -0.024194982, 0.032102264, -0.012509059, + 0.005405469, 0.014570137, -0.02804492, 0.035323508, 0.0032990212, + -0.028531026, 0.02337833, 0.008775786, -0.027306044, -0.053276923, + 0.032380965, -0.0024434794, -0.038914192, -0.013954407, + -0.039717883, 0.027856963, -0.0070063695, 0.007142478, -0.004958254, + 0.014233105, 0.033670757, 0.018335817, -0.02804492, -0.015600677, + 0.0031499495, -0.024065355, 0.0024629235, 0.016721955, + -0.0022879262, 0.0064878594, 0.006604524, 0.04352245, -0.050684374, + 0.0024629235, -0.025199596, 0.02790233, 0.012787758, -0.005509171, + -0.014719209, 0.02385147, -0.004932328, 0.011847959, -0.0054702824, + -0.042926166, 0.018873772, 0.036341086, 0.047871456, -0.013157196, + 0.013228492, 0.0053989873, 0.01867933, 0.028414361, -0.037818838, + -0.046212222, 0.016611772, 0.010324835, -0.039542884, 0.03268559, + 0.025588479, -0.035498504, -0.03513555, -0.0043165977, 0.0059499047, + 0.005424913, -0.019178396, -0.06280455, -0.010506313, -0.039763253, + -0.0074341404, 0.0057295375, 0.042854868, 0.018880254, -0.007310994, + 0.0327504, -0.023067223, -0.015600677, 0.014894205, -0.015983077, + 0.027131047, 0.0049647354, -0.011958141, -0.013183122, 0.006960999, + -0.058941647, 0.038071614, 0.015516418, -0.06952573, 0.054495424, + -0.003778643, -0.015179386, 0.0267292, -0.01641733, 0.04549927, + 0.027228266, -0.013422933, 0.0068767415, 0.0048351074, 0.0075637675, + -0.022503344, 0.0068184095, 0.015224756, -0.032717995, -0.027351415, + -0.01595715, 0.0047314055, 0.02521256, 0.033087432, -0.016683066, + -0.00031110612, 0.020785777, 0.059492562, -0.020403376, + -0.038486417, 0.033625387, -0.042226177, -0.024032949, -0.044416882, + -0.0011796107, 0.020766335, -0.0061897156, 0.023592213, + -0.0055804662, 0.0041416003, -0.031830046, 0.060205515, -0.05368525, + -0.0046277037, -0.0109146405, -0.040631756, 0.014090515, + -0.03293836, -0.016533993, 0.0020481155, -0.0076156184, + -0.0033249469, -0.011050749, 0.02821992, -0.052797303, -0.026411613, + -0.022030203, -0.0120423995, -0.024305167, 0.032316152, + 0.0032147632, -0.018808957, -0.057327785, -0.016339552, -0.5266703, + 0.0148164295, 0.013397008, -0.013947925, 0.068773896, 0.02880324, + -0.056407433, -0.049057547, 0.028336585, -0.012722944, -0.026469946, + -0.0066823, -0.025847733, 0.00931374, -0.0016981208, 0.042621538, + 0.05301767, -0.023339441, 0.0072785867, -0.011958141, -0.05072326, + 0.02134966, 0.027980108, 0.005800833, -0.014284956, 0.020882998, + -0.037818838, -0.0012573872, 0.0034740183, -0.018925622, + -0.08952727, -0.020545967, -0.012411838, 0.029768968, -0.03866142, + 0.033281874, 0.007498954, 0.0067471145, 0.04041139, 0.040566944, + -0.0043360414, -0.026366243, -0.011614629, 0.025387555, 0.018510815, + -0.029328233, 0.007356364, 0.015775673, -0.018329335, 0.0134294145, + -0.020818185, 0.0025536628, -0.023352403, -0.042271543, 0.034850366, + 0.031402275, -0.008944301, 0.007712839, -0.0109081585, -0.032030966, + 0.013072939, 0.039322518, 0.003311984, 0.01735713, 0.036768854, + -0.031784676, -0.0821385, 0.0012833127, 0.022898708, -0.00058332394, + 0.040949345, 0.06251288, -0.0040573427, 0.030339329, 0.016734917, + -0.02187465, 0.028893983, -0.0639712, -0.004932328, -0.05363988, + -0.013202567, -0.08604029, -0.013422933, 0.020857073, -0.02982082, + -0.002780511, 0.0313245, -0.031972636, 0.017823787, 0.018335817, + -0.013649781, 0.0057619447, 0.008976708, -0.01735065, -0.025076449, + 0.0066369306, -0.022282977, 0.03600405, 0.05933053, -0.0054638013, + -0.0038953077, 0.0067730392, 0.030352293, 0.04864922, -0.029866187, + 0.008555418, -0.015542344, 0.00014259032, 0.03542721, 0.029269902, + -0.020740408, -0.005321211, -0.028531026, 0.0011472038, -0.03497999, + 0.00058980536, 0.0020481155, 0.022483898, 0.034467965, -0.018620998, + 0.041565076, 0.000978688, 0.037241995, -0.030015262, 0.013584967, + 0.021414472, 0.019048769, -0.008043389, 0.043652076, -0.016864544, + 0.106294595, -0.013986813, 0.020708002, -0.014615507, -0.028686577, + -0.042880796, -0.024966266, -0.013085901, 0.022010759, 0.017551571, + 0.0266255, -0.042407654, 0.015451604, 0.072863646, -0.014388657, + 0.0135460785, 0.0051267697, 0.020345045, 0.0078035784, -0.05250564, + 0.034111492, -0.07254606, 0.013338676, -0.0032666142, -0.016871026, + 0.012483133, 0.039212335, -0.08336996, 0.01086927, 0.029982852, + 0.0541195, -0.0033573536, 0.061786972, -0.008354496, 0.016948802, + 0.0044915946, 0.07134052, -0.002061078, 0.056990754, -0.01693584, + 0.020526523, 0.079727426, 0.026269022, -0.0011536853, 0.02305426, + 0.015419197, -0.003124024, -0.0075767306, 0.02381258, 0.01952839, + 0.020008013, -0.023397774, 0.02135614, 0.010947047, 0.026094027, + 0.013850704, -0.010616496, 0.006202678, 0.006591561, -0.02983378, + -0.024188502, -0.005885091, -0.006131383, -0.009475774, 0.018549703, + -0.004977698, -0.03251707, 0.035978127, 0.028654171, 0.041053046, + 0.038123462, 0.02432461, -0.015730303, -1.9444133e-5, 0.009799843, + -0.0071554407, -0.015892338, 0.041416004, 0.004925847, 0.017344166, + -0.01985246, -0.07217662, 0.0030138404, 0.039400294, 0.0063063805, + -0.03166153, -0.0071943286, -0.0023916285, 0.031830046, + 0.0074082147, -0.01495902, 0.004809182, -0.012794239, 0.093824424, + -0.07693396, 0.043347456, -0.05726945, -0.020014493, -0.029243976, + 0.02413017, -0.02530978, 0.030462474, -0.01547753, 0.04019102, + -0.010979454, -0.014952539, -0.013844223, -0.005710094, 0.01820619, + -0.04529835, 0.00936559, -0.02991804, 0.00564528, -0.040385462, + -0.031253204, 0.0035064255, -0.03284762, -0.0063323057, 0.019521909, + 0.051118627, 0.007129515, -0.009825769, 0.013254417, 0.0068637785, + -0.011685924, -0.02597736, -0.0013481266, 0.02465516, 0.008367458, + -0.01900988, 0.035731833, 0.005431394, 0.02028023, -0.00093331834, + 0.0024953305, -0.030637473, -0.016812693, -0.07578675, 0.046607587, + 0.015885856, 0.037831802, -0.018964512, -0.033599462, -0.005632317, + -0.002262001, -0.036548488, -0.05151399, -0.018867292, 0.008717453, + -0.0008879488, 0.06401008, -0.00020740408, -0.013260898, + -0.00014907168, 0.027215306, -0.009702622, 0.015399754, 0.011109081, + -0.017804345, -0.0013934962, 0.046918694, 0.039154, 0.014498841, + -0.034610555, 0.0033573536, -0.043891888, 0.017700642, 0.012729425, + -0.0057943515, -0.04294561, 0.0020999664, -0.019599685, -0.01068131, + -0.022976482, 0.013643299, 0.006060088, 0.0070452574, 0.00073887705, + 0.027824553, -0.033100393, 0.034429077, 0.069435, 0.002061078, + -0.010979454, -0.028621763, -0.003227726, -0.014790503, 0.010571127, + -0.052797303, -0.12194064, 0.024557939, -0.028381951, 0.0019508946, + -0.042362284, 0.047404796, -0.020980218, -0.0071619223, 0.010733161, + 0.00837394, -0.033787422, -0.014647913, -0.017940454, -0.038486417, + -0.016916396, 0.0007129515, -0.008944301, -0.022004277, + -0.010849826, -0.05809259, -0.004277709, -0.012321099, -0.06377028, + 0.03492166, 0.021304287, 0.040677123, + ], + }, + { + title: "Hills", + query: "photo of mountains or hills", + minScore: 0.2, + showVideo: true, + textEmbedding: [ + 0.020085756, 0.022995783, 0.01930537, 0.011414018, 0.013222759, + -0.004777113, 0.0046385406, 0.001684753, 0.053211935, 0.024593016, + 0.0017430993, -0.03710103, 0.030733977, -0.022033067, 0.017905058, + 0.55763865, -0.01830619, 7.2933026e-6, 0.015527441, -0.013266518, + -0.02212788, 0.0019473118, -0.03455567, -0.009794906, 0.0030996536, + -0.022310212, -0.022601943, 0.0072568366, -0.016847529, 0.0481358, + 0.04608638, -0.0040915427, -0.036094554, 0.025074374, -0.01258824, + 0.033009488, -0.011523418, -0.0044051544, -0.008576924, + -0.022405026, 0.039989177, 0.018824015, -0.015753534, -0.034015965, + -0.0068848776, -0.012814333, -0.035693422, -0.024636777, + 0.029836902, -0.025563026, 0.0007147436, 0.020836966, -0.03551838, + 0.03173316, -0.005083432, -0.00560855, -0.012960199, -0.008905123, + 0.0091312155, 0.037925176, 0.00010210624, 0.014484499, 0.0044416212, + -0.013259225, 0.027882297, 0.040229857, -0.03148519, -0.015410749, + 0.048332717, 0.018036338, 0.019932596, 0.06964374, -0.0011231686, + -0.020501474, 0.013499903, 0.00014586606, -0.06148254, 0.015425335, + 0.041608293, 0.039551582, 0.0029100277, 0.0031288266, 0.02645281, + -0.00644728, -0.011166046, -0.049368367, -0.013084185, -0.01258824, + -0.016191132, 0.008350831, 0.046932403, 0.03791788, -0.008146619, + 0.0033913855, -0.0022755102, 0.022003895, 0.018539574, 0.052592, + -0.020012822, 0.001976485, -0.0065420927, -0.002625589, 0.0251546, + -0.06844765, 0.018597921, 0.012004776, -0.07617125, 0.026715368, + 0.025511973, -0.005061552, 0.001662873, -0.045335166, 0.012391321, + -0.0109253675, 0.021755923, -0.015717067, 0.007774661, 0.0122746285, + -0.011311912, -0.0040186094, -0.00472606, -0.04054347, 0.024461737, + -0.0031652933, -0.013069599, 0.00013127945, 0.102376096, + -0.055509325, 0.009174975, -0.0010575289, 0.0133759165, + -0.021894494, -0.010801381, -0.0005105312, 0.013178998, + -0.0062211873, -0.03951511, 0.009488587, -0.050068524, -0.008810309, + 0.009306254, 0.0073881154, 0.0036758247, 0.009145801, 0.009109335, + -0.055655196, 0.002049418, -0.018933414, -0.01413442, + -0.00037195845, -0.04357019, 0.00238491, 0.022536304, 0.02457843, + 0.009853251, -0.04352643, 0.014520965, 0.016526623, -0.023484435, + -0.009226028, -0.003515372, 0.031507067, 0.028604334, -0.02230292, + -0.039471354, -0.042738754, -0.037881415, -0.5623866, 0.022383146, + -0.026489275, -0.005688776, 0.08086814, -0.0054408037, -0.021668402, + -0.005645016, -0.015177363, -0.013893741, -0.01534511, -0.030551644, + -0.040069405, -0.0073224762, -0.0053168177, -0.034774467, + 0.021063058, -0.026007917, 0.02034102, -0.017664379, -0.006162841, + 0.02097554, 0.016417224, -0.038238786, -0.006316, -0.02627777, + -0.0813495, -0.005783589, -0.02592769, 0.0005032379, -0.0567346, + 0.012500721, 0.017153848, -0.01035649, -0.0148054045, 0.039289024, + 0.0024870161, 0.009182268, 0.019385599, 0.01903552, 0.011187926, + -0.032710463, 0.008204965, 0.028808545, -0.0016993396, 0.030281793, + -0.008540457, -0.016657902, -0.012092296, 0.020742152, -0.014499086, + -0.0016993396, -0.019378304, 0.025234828, 0.014338633, 0.034701534, + 0.0016920463, -0.010283557, 0.029253436, 0.03559861, -0.015016911, + -0.042585593, -0.10697816, -0.04332951, 0.00068557047, 0.018845895, + -0.070176154, -0.0068994644, -0.09547663, -0.0364811, -0.011516125, + 0.035861168, 0.0062576532, 0.021836147, 0.010261677, -0.05277434, + -0.012019362, -0.01183703, 0.046830297, -0.03769179, -0.03413995, + -0.09999117, 0.012777866, -0.0005178245, -0.0024943093, + -0.034417093, 0.007789247, -0.02926073, -0.0102325035, 0.018656267, + 0.0035883049, 0.022740517, 0.008234139, -0.026423635, -0.021427725, + 0.009692799, 0.029975474, -0.016074438, 0.017365353, -0.019123038, + -0.006439986, 0.015403455, -0.013222759, 0.011114993, 0.027437406, + 0.012427788, 0.016067147, 0.0063451733, 0.063641354, 0.03654674, + 0.01879484, 0.03968286, -0.026263181, 0.010268969, -0.018029043, + 0.021901788, -0.028093802, 0.010575289, 0.0008970762, -0.0022098708, + 0.015257589, 0.026372582, 0.00781842, 0.0033840926, -0.012216282, + 0.010983714, -0.018670855, -0.012697641, -0.019159507, 0.0055574966, + 0.041972958, 0.0055210297, 0.002961081, -0.02345526, 0.019961769, + 0.0208005, 0.0055210297, 0.011676578, 0.009401067, -0.01316441, + -0.018225962, -0.00242867, -0.009153095, 0.10668643, -0.0030923602, + -0.00940836, 0.0028298015, 0.033133473, 0.038246077, -0.0537954, + -0.0039237966, -0.016687077, 0.017037155, -0.0012106881, + -0.018780256, -0.0005907575, 0.046757363, -0.05332863, 0.021194337, + -0.044284936, 0.040047526, 0.015476389, -0.013930209, -0.019086573, + 0.024877455, -0.023156237, 0.047800303, -0.020567114, 0.018138442, + -0.035671543, -0.015841054, 0.014878337, 0.007249543, 0.013966675, + 0.0070161573, -0.009379188, 0.0036831177, -0.006782771, 0.01821867, + 0.02239044, 0.030945482, -0.020902606, -0.0072203698, 0.0013638476, + -0.024024138, 0.0011596351, -0.035277706, 0.014623072, -0.02981502, + -0.025511973, 0.05313171, 0.0173143, 0.028451173, 0.050491538, + -0.004711474, -0.01806551, -0.031040296, -0.03389927, -0.019801317, + -0.010743035, -0.00065639726, -0.021653816, 0.030879842, + -0.017868591, 0.0014149008, -0.011078526, -0.016008798, 0.03063187, + -0.012325682, 0.015666014, 0.008832189, -0.090925604, 0.0021734042, + -0.015126309, 0.011705751, -0.017321594, 0.0017285127, + 0.00019691917, -0.014396979, 0.0023703235, -0.026576793, + -0.0042228224, 0.0009189562, -0.07509914, 0.008380005, -0.0139375, + 0.0012909146, 0.0020567114, -0.01942936, -0.010268969, + -0.0022025772, 0.0010648222, -0.013499903, 0.0117422175, + -0.011034767, 7.2933026e-6, 0.026343409, 0.026846647, 0.020027408, + -0.049470473, 0.019990943, -0.0018597922, 0.008671737, -0.04440892, + -0.064574905, -0.013543664, -0.020691099, -0.004244702, + -0.012318389, 0.015841054, 0.0017868591, -0.0004959446, 0.011844324, + 0.013507197, -0.0019400185, -0.07307889, 0.020253502, -0.03182797, + -0.030515177, -0.008066393, 0.01636617, -0.028451173, 0.038895182, + 0.020122223, -0.0075996215, -0.010808675, 0.012566361, + -0.0017868591, 0.017438287, -0.020836966, 0.033439793, -0.009860545, + -0.02696334, -0.0075266883, 0.004361395, 0.021799682, 0.013354036, + 0.036772832, 0.006323293, 0.020370195, -0.07023451, 0.027065445, + -0.006264947, -0.049601752, 0.0049959123, -0.013076892, -0.02799899, + -0.004959446, -0.0074756355, 0.009889718, 0.07115346, 0.010152278, + -0.013798929, -0.004594781, -0.009379188, 0.012296508, 0.014010434, + 0.007125557, 0.059010115, 0.011567178, 0.0019473118, 0.0128435055, + -0.013003958, -0.0022828039, 0.05829537, -0.042366795, 0.007935113, + -0.053824574, 0.021901788, -0.0014076075, -0.019108452, + -0.004047783, 0.010837848, -0.014520965, -0.0011669284, + -0.0054335105, 0.018700028, 0.01035649, 0.016971515, -0.0676235, + -0.013018546, 0.008423764, -0.011465072, -0.015272176, + -0.0016701663, -0.011348379, 0.0039821435, -0.007876767, + -0.009328133, -0.035831995, -0.022930143, -0.0023776167, + -0.0005907575, -0.012704934, 0.008606098, -0.021763215, -0.02034102, + 0.017875886, -0.005995095, 0.016220305, -0.023907445, -0.0023922033, + -0.013550956, 0.016074438, 0.036313355, + ], + }, + { + title: "Greenery", + query: "outdoor photo of trees, leaves, flowers", + minScore: 0.2, + showVideo: true, + textEmbedding: [ + -0.007156132, -0.01483226, 0.0016425262, -0.0041434583, + 0.0028806115, 0.0021790299, -0.0032767986, 0.014353533, 0.02069253, + -0.05503701, -0.0010482455, 0.011753554, 0.06092204, 0.004613931, + 0.012496406, 0.49475536, -0.032041643, -0.0028558497, 0.031637203, + -0.016020821, -0.01912429, -0.027213112, -0.0074285106, + -0.043093614, -0.0060253474, -0.014312264, -0.025884233, + -0.008567549, -0.011307844, 0.018959211, -0.001411417, + -0.0012380852, 0.0058602695, -0.008121839, 0.0022120455, + 0.0027898184, 0.03533495, 0.0071808943, 0.0017993505, 0.019751586, + 0.021129988, -0.012323074, 0.0057529686, -0.044612333, 0.022607435, + -0.0024761704, -0.0050101182, 0.00935167, -0.008303424, + -0.0023110923, 0.035681613, 0.026585815, 0.005719953, -0.0041104425, + -0.019603014, 0.006751691, -0.040006656, -0.0006768199, + -0.024448056, 0.04070824, 0.02441504, -0.0032767986, -0.008749135, + -0.039148252, 0.019685555, 0.007255179, -0.020717291, -0.007445019, + 0.0030126737, 0.017696364, 0.05585415, -0.0048202784, 0.0039536185, + -0.030663243, 0.024134407, 0.0136602055, -0.1504686, 0.010086267, + 0.00266601, -0.04824405, -0.040988874, 0.0027320413, 0.020981416, + 0.0074202567, -0.0038463178, 0.02020555, -0.015789712, 0.0027733108, + -0.021105224, 0.018397946, 0.05896587, 0.012801801, -0.01747351, + 0.020238565, 0.009624049, 0.021633474, -0.14422867, 0.0062151873, + -0.041211728, 0.01747351, -0.012983386, 0.007981523, -0.0022202993, + -0.08149902, -0.012174504, 0.0051586884, -0.04341552, -0.0069580385, + -0.0065453434, -0.0015764951, 0.008798659, -0.040931094, -0.1643847, + 0.013907823, 0.003598701, 0.034435276, -0.008773897, -0.007898984, + 0.020956654, 0.016070344, -0.009739603, -0.0008006284, 0.00897199, + -0.01637574, -0.037992705, 0.018785879, 0.08574152, 0.007015816, + 0.019008733, 0.0040196497, -0.0010647532, 0.03245434, -0.04447202, + -0.016029077, 0.046386924, 0.027097557, -0.0012628469, 0.031397842, + 0.011118005, 0.011959903, 0.012091965, 0.001411417, 0.024084883, + 0.030423878, 0.0011142766, -0.00014031632, -0.0067104213, + -0.0011060227, -0.03654002, 0.049374834, 0.012281804, 0.007197402, + 0.052734174, -0.014749722, 0.0006768199, 0.0041434583, 0.025083605, + -0.029548965, -0.013635444, 0.009491987, 0.07705017, 0.005885031, + 0.0018158582, 0.021534426, -0.07587811, 0.077892065, -0.020898879, + -0.49878326, 0.02576868, -0.025182651, -0.0033015604, 0.06256457, + -0.005340274, 0.030168008, -0.012686246, -0.023655681, -0.022549657, + -0.008584057, 0.02725438, 0.028327389, 0.020552212, 0.040122215, + 0.0161859, 0.011687524, -0.0055053523, 0.002996166, -0.008798659, + 0.007981523, 0.03664732, -0.004878056, 0.0059758243, -0.026263913, + 0.068342306, -0.012248789, -0.01070531, -0.020618245, 0.030729273, + -0.0508853, 0.052527826, 0.017993504, 0.03138133, -0.048929125, + 0.041640934, -0.019099526, -0.022062676, -0.005076149, 0.012554184, + 0.009525002, -0.021237286, 0.017490016, 0.048103735, 0.0060253474, + 0.03127403, 0.0103338845, -0.016986528, 0.05153736, 0.10163853, + -0.03116673, 0.00072634325, -0.005530114, -0.008707866, 0.031389583, + 0.016532563, 0.01961127, -0.02354838, -0.002344108, -0.009624049, + -0.046543747, -0.0030291816, -0.11197242, -0.0019644285, 0.03652351, + -0.033188935, 0.03323846, 0.0069910544, -0.012851324, -0.014658929, + 0.011687524, -0.018934447, 0.015682412, -0.0070570856, -0.017712872, + -0.0020304597, -0.03505432, 0.04561931, -0.0029879122, -0.02274775, + -0.079873, -0.14792642, -0.05039832, 0.0027402951, -0.013354812, + 0.015682412, 0.020527452, -0.0041847276, -0.031554665, 0.011737047, + -0.086418346, 0.0014774483, 0.014386549, 0.026239151, -0.025537571, + 0.009888174, -0.008064061, 0.004498376, -0.021806806, -0.010267853, + 0.0031282285, 0.016053837, -0.01802652, -0.009962458, 0.010573247, + 0.023820758, -0.016062092, -0.0025834711, 0.042441558, 0.011604984, + -0.014898291, 0.028046755, -0.053410996, -0.014039885, -0.025034081, + -0.0015930028, 0.019371906, -0.014782737, 0.008064061, -0.03370068, + -0.008245647, 0.052808456, -0.010573247, -0.03406385, -0.031117208, + 0.109141335, -0.006759945, -0.012091965, -0.0025504555, -0.0355413, + -0.023350285, 0.05615954, 0.0042837746, 0.014328773, -0.029301347, + -0.016425263, -0.0052329735, 0.012892594, 0.007403749, 0.004671708, + 0.008039299, 0.00878215, -0.018084297, 0.066237554, -0.0022037916, + 0.011770062, -0.002344108, 0.017754141, 0.010465947, -0.028005486, + 0.0054640826, 0.0233833, -0.012438629, 0.081771396, 0.022186486, + -0.04242505, 0.03782763, -0.07913015, 0.007081847, 0.028682305, + -0.023507109, 0.034988288, -0.022136962, -0.005579637, 0.004646946, + -0.0242252, 0.015500827, -0.0072799404, 0.016590342, 0.027559776, + 0.016499547, -0.011976411, -0.0848501, -0.018686833, -0.010862134, + 0.002979658, -0.019338889, -0.0014361787, -0.023895044, + -0.027163588, 0.016400501, 0.06977848, -0.008352947, 0.0015682412, + 0.010482455, -0.019107781, 0.053138614, -0.005736461, -0.038199052, + 0.06096331, 0.053221155, -0.008740881, -0.0403038, -0.0022368073, + -0.006900261, -0.040254276, 0.01100245, 0.010688801, -0.0033263222, + -0.0025999788, -0.02467091, -0.023556633, 0.008130093, 0.052882746, + -0.019726822, 0.012752278, 0.007445019, 0.0031199746, 0.023564888, + -0.005356782, 0.035739392, 0.020453166, -0.036168594, 0.006446297, + 0.016565578, -0.01453512, -0.038669527, 0.013264019, -0.0073789875, + -0.0015187177, 0.004192982, 0.014741467, -0.010408169, -0.030919112, + -0.053881466, -4.9523405e-5, -0.011753554, 0.007395495, + -0.023663934, -0.020477928, -0.045487247, 0.020230312, 0.009665318, + 0.00039618724, 0.05012594, -0.029152777, -0.01074658, -0.06847437, + 0.010845627, -0.011547208, -0.0008914213, -0.002509186, -0.03701049, + 0.035962246, -0.0059345546, -0.0015022099, -0.01889318, + -0.008410726, -0.0030374355, -0.016499547, -0.011737047, + 0.009104053, -0.00052824966, 0.06893658, -0.05145482, 0.026263913, + 0.04291203, 0.07261782, -0.00014857022, 0.0032933066, -0.008641834, + 0.0005695192, -0.0022285532, 0.008097077, 0.03805874, -0.0139986165, + 0.011794825, -0.015872251, 0.017052561, -0.041319028, 0.057645246, + -0.025908995, -0.028236596, -0.043242186, -0.013808776, -0.0138583, + -0.008641834, -0.031686727, 0.008468502, 0.008518025, 0.022632197, + 0.031554665, -0.004267267, -0.03546701, -0.045751374, -0.054417968, + 0.010185314, 0.009450716, -0.008220885, -0.00070983544, -0.06394297, + 0.0075935894, -0.013594175, -0.005406305, -0.016152885, 0.012422121, + 0.05346877, 0.02166649, 0.005431067, 0.004770755, 0.02445631, + 0.0012133234, 0.003937111, -0.00054475747, 0.010251345, 0.020172535, + 0.0040279035, -0.04656851, -0.012224028, 0.01836493, -0.029623251, + -0.00047872626, -0.027411206, -0.00184062, 0.0011307844, 0.03762128, + 0.00024761702, 0.01772938, 0.02786517, -0.12921482, 0.037456203, + -0.009830396, 0.012644976, -0.0031447362, 0.0035821933, + 0.0043745674, 0.0148652755, -0.010994197, -0.016342724, + -0.021889346, -0.0096818255, 0.023069654, 0.0021872837, 0.019495714, + -0.027328666, 0.000998722, 0.016458279, 0.008229139, -0.027535014, + 0.011695778, 8.2539016e-5, -0.0024431548, 0.020832846, 0.02932611, + 0.10528676, + ], + }, +]; diff --git a/src/clip-keywords.ts b/src/clip-keywords.ts new file mode 100644 index 0000000..f8c3a37 --- /dev/null +++ b/src/clip-keywords.ts @@ -0,0 +1,57 @@ +// Keywords for a file from its CLIP embedding. +// +// Ente stores no keywords. A file's ML data holds a CLIP embedding, the 512 +// numbers Ente's image model gave the photo, and Ente's photos app sorts files +// into the categories of its Discover section with it: a file falls in a +// category when the cosine similarity of its CLIP embedding and the text +// embedding of the category's query is at least the category's `minScore`, +// unless the file is a video and the category leaves out videos. A file's +// keywords are the titles of the categories it falls in. The categories and +// their text embeddings are in `clip-keyword-vectors.ts`. + +import { DISCOVER_CATEGORIES } from "./clip-keyword-vectors.js"; +import { clipEmbedding } from "./library/mldata.js"; +import type { MLData } from "./mldata-fetch.js"; + +// How many numbers a CLIP embedding from Ente's image model has. +const EMBEDDING_LENGTH = 512; + +const norm = (vector: number[]): number => Math.hypot(...vector); + +const cosineSimilarity = (a: number[], b: number[]): number => { + let dot = 0; + a.forEach((value, i) => { + dot += value * (b[i] ?? 0); + }); + return dot / (norm(a) * norm(b)); +}; + +// The titles of the Discover categories a file falls in, highest score first, +// or undefined when its ML data has no CLIP embedding of 512 finite numbers +// that are not all zero. Tests pass their own `categories`. +export const clipKeywords = ( + mlData: MLData, + isVideo: boolean, + categories = DISCOVER_CATEGORIES, +): string[] | undefined => { + const embedding = clipEmbedding(mlData); + if ( + !embedding || + embedding.length !== EMBEDDING_LENGTH || + norm(embedding) === 0 + ) { + return undefined; + } + + const matches: { title: string; score: number }[] = []; + for (const category of categories) { + if (isVideo && !category.showVideo) continue; + const score = cosineSimilarity(embedding, category.textEmbedding); + if (score >= category.minScore) { + matches.push({ title: category.title, score }); + } + } + return matches + .sort((a, b) => b.score - a.score) + .map((match) => match.title); +}; diff --git a/src/library/mldata.ts b/src/library/mldata.ts index 06307d8..fcc624a 100644 --- a/src/library/mldata.ts +++ b/src/library/mldata.ts @@ -61,7 +61,7 @@ export interface MLIndex { // Pull the CLIP embedding out of a payload, or undefined when it is absent or // misshapen. Kept strict so a bad payload is skipped rather than corrupting the // packed index. -const clipEmbedding = (payload: MLData): number[] | undefined => { +export const clipEmbedding = (payload: MLData): number[] | undefined => { const clip = payload.clip; if (typeof clip !== "object" || clip === null) return undefined; const embedding = (clip as { embedding?: unknown }).embedding; diff --git a/src/metadata-backup.ts b/src/metadata-backup.ts index 35e13e2..fb07840 100644 --- a/src/metadata-backup.ts +++ b/src/metadata-backup.ts @@ -1,6 +1,7 @@ import { mkdirSync, readFileSync, writeFileSync } from "node:fs"; import { join } from "node:path"; import * as jpeg from "jpeg-js"; +import { clipKeywords } from "./clip-keywords.js"; import type { Client } from "./client.js"; import { describeError } from "./errors.js"; import { readExifTags } from "./exif.js"; @@ -97,7 +98,8 @@ const extractExif = async ( // Dump every decrypted metadata layer the account holds into a directory tree // of plain JSON: account, per-collection, and per-file records including the -// private and public magic metadata and (by default) the ML data. Collections +// private and public magic metadata and the ML data, with the keywords of its +// CLIP embedding in place of the embedding (see clip-keywords.ts). Collections // and files are enumerated from the library's cache, which the caller refreshes // first. Returns how many files have no ML data because their ML data request // failed or their ML data did not decrypt, gunzip or parse; they are still @@ -230,8 +232,16 @@ export const runMetadataBackup = async ( if (file.pubMagicMetadata) fileMeta.pubMagicMetadata = file.pubMagicMetadata; + // The ML data without the CLIP embedding, and the keywords that + // embedding gives. const ml = mlDataMap.get(file.id); - if (ml) fileMeta.mlData = ml; + if (ml) { + const { clip: _clip, ...mlDataWithoutClip } = ml; + fileMeta.mlData = mlDataWithoutClip; + const isVideo = file.metadata.fileType === "video"; + const keywords = clipKeywords(ml, isVideo); + if (keywords) fileMeta.keywords = keywords; + } const mlError = mlDataErrors.get(file.id); if (mlError) fileMeta.mlDataError = mlError; diff --git a/test/cli/backup-end-to-end.test.ts b/test/cli/backup-end-to-end.test.ts index 46895e3..8f075f7 100644 --- a/test/cli/backup-end-to-end.test.ts +++ b/test/cli/backup-end-to-end.test.ts @@ -86,10 +86,17 @@ const DAY = "2024/2024-03/2024-03-01"; const TOAST = new TextEncoder().encode("a few seconds of the wedding toast"); // Ente's ML data for beach.jpg: a face it found, and the photo's CLIP -// embedding. +// embedding, here that of a photo of a sunset (see clip-keywords.test.ts). const ML_DATA = { face: { faces: [{ faceID: "100_1", score: 0.98 }] }, - clip: { embedding: [0.5, 0.6, 0.7] }, + clip: { + embedding: JSON.parse( + readFileSync( + new URL("../clip-keywords-sunset.json", import.meta.url), + "utf-8", + ), + ).embedding, + }, }; // Each file's ID, its owner, its original as uploaded, and the metadata its @@ -486,7 +493,10 @@ describe("the first backup of the account", () => { }, updationTime: TAKEN, }); - expect(beach.mlData).toEqual(ML_DATA); + // The JSON holds the ML data without its CLIP embedding, and the + // keywords the embedding gives in its place. + expect(beach.mlData).toEqual({ face: ML_DATA.face }); + expect(beach.keywords).toEqual(["Sunset"]); expect(beach.imageMetadata.exif.Make.description).toBe("Canon"); expect(beach.imageMetadata.exif.Model.description).toBe("EOS R5"); @@ -497,6 +507,7 @@ describe("the first backup of the account", () => { metadata: { title: "toast.mp4", fileType: "video" }, }); expect(toast).not.toHaveProperty("mlData"); + expect(toast).not.toHaveProperty("keywords"); expect(toast).not.toHaveProperty("imageMetadata"); expect(toast).not.toHaveProperty("imageMetadataError"); diff --git a/test/cli/backup.test.ts b/test/cli/backup.test.ts index c6258dd..df203c7 100644 --- a/test/cli/backup.test.ts +++ b/test/cli/backup.test.ts @@ -52,6 +52,7 @@ import { tmpdir } from "node:os"; import { describe, it, expect, beforeEach, afterEach, vi } from "vitest"; import { runBackup, type BackupLibrary } from "../../src/backup.js"; +import { DISCOVER_CATEGORIES } from "../../src/clip-keyword-vectors.js"; import { Library } from "../../src/library/index.js"; import type { ContentSource } from "../../src/library/content.js"; import type { CollectionsPage, FilesPage } from "../../src/client.js"; @@ -946,9 +947,19 @@ describe("the backup lock", () => { }); }); -// Also serves ML data: file 100 has `ML_PAYLOAD`, the other two have none. -// While `mlError` is set, every ML data request fails with it. -const ML_PAYLOAD = { face: { faces: [] }, clip: { embedding: [0.5, 0.25] } }; +// Also serves ML data: file 100 has `ML_PAYLOAD`, whose CLIP embedding is that +// of a photo of a sunset (see clip-keywords.test.ts), and the other two have +// none. While `mlError` is set, every ML data request fails with it. +const SUNSET_EMBEDDING: number[] = JSON.parse( + readFileSync( + new URL("../clip-keywords-sunset.json", import.meta.url), + "utf-8", + ), +).embedding; +const ML_PAYLOAD = { + face: { faces: [] }, + clip: { embedding: SUNSET_EMBEDDING }, +}; class MLClient extends MockClient { mlError?: string; @@ -968,16 +979,99 @@ const fileJSON = (outDir: string, fileID: number): Record<string, unknown> => JSON.parse(readFileSync(saved(outDir, `${fileID}.json`), "utf-8")); describe("ML data in each file's JSON", () => { - it("writes a file's ML data, and no ML field for a file that has none", async () => { + // The JSON holds the ML data without its CLIP embedding, and in its place + // the keywords the embedding gives. + it("writes a file's ML data and keywords, and no ML field for a file that has none", async () => { const lib = await openLibrary(stubSource(), new MLClient()); const outDir = join(root, "backup"); const result = await lib.backup({ downloadDirectory: outDir }); expect(result.failed).toBe(0); - expect(fileJSON(outDir, 100).mlData).toEqual(ML_PAYLOAD); + expect(fileJSON(outDir, 100).mlData).toEqual({ face: { faces: [] } }); + expect(fileJSON(outDir, 100).keywords).toEqual(["Sunset"]); expect(fileJSON(outDir, 101)).not.toHaveProperty("mlData"); expect(fileJSON(outDir, 101)).not.toHaveProperty("mlDataError"); + expect(fileJSON(outDir, 101)).not.toHaveProperty("keywords"); + await lib.close(); + }); + + // A file's ML data can hold faces and no CLIP embedding. Its JSON then has + // the ML data and no keywords. + it("writes no keywords for ML data without a CLIP embedding", async () => { + const faces = { face: { faces: [{ faceID: "101_1", score: 0.9 }] } }; + + // Gives file 101 ML data holding only faces. + class FacesClient extends MockClient { + async fetchMLData(args: { fileIDs: number[] }): Promise<{ + payloads: Map<number, MLData>; + errors: Map<number, unknown>; + }> { + const payloads = new Map<number, MLData>(); + if (args.fileIDs.includes(101)) payloads.set(101, faces); + return { payloads, errors: new Map() }; + } + } + const lib = await openLibrary(stubSource(), new FacesClient()); + const outDir = join(root, "backup"); + + const result = await lib.backup({ downloadDirectory: outDir }); + + expect(result.failed).toBe(0); + expect(fileJSON(outDir, 101).mlData).toEqual(faces); + expect(fileJSON(outDir, 101)).not.toHaveProperty("keywords"); + await lib.close(); + }); + + // Ente's photos app keeps videos out of some Discover categories, Identity + // among them. Here a photo and a video share one CLIP embedding, Identity's + // own text embedding, so the photo gets the keyword Identity and the video + // does not. + it("keeps a video out of a category that leaves out videos", async () => { + const identity = DISCOVER_CATEGORIES.find( + (c) => c.title === "Identity", + )!; + expect(identity.showVideo).toBe(false); + const toast = file(102, 1, "toast.mp4"); + const video = { + ...toast, + metadata: { ...toast.metadata, fileType: "video" as const }, + }; + + // Serves beach.jpg and the video in the Vacation album, and gives + // each ML data holding only that embedding. + class VideoClient extends MockClient { + override async filesSince(args: { + collectionID: number; + }): Promise<FilesPage> { + const files = + args.collectionID === 1 + ? [file(100, 1, "beach.jpg"), video] + : []; + return { files, deleted: [], cursor: 1 }; + } + async fetchMLData(args: { fileIDs: number[] }): Promise<{ + payloads: Map<number, MLData>; + errors: Map<number, unknown>; + }> { + const payloads = new Map<number, MLData>(); + for (const id of args.fileIDs) { + payloads.set(id, { + clip: { embedding: identity.textEmbedding }, + }); + } + return { payloads, errors: new Map() }; + } + } + const lib = await openLibrary(stubSource(), new VideoClient()); + const outDir = join(root, "backup"); + + const result = await lib.backup({ downloadDirectory: outDir }); + + expect(result.failed).toBe(0); + expect(fileJSON(outDir, 100).keywords).toContain("Identity"); + expect(fileJSON(outDir, 102)).toHaveProperty("keywords"); + expect(fileJSON(outDir, 102).keywords).not.toContain("Identity"); await lib.close(); }); @@ -993,7 +1087,7 @@ describe("ML data in each file's JSON", () => { const result = await lib.backup({ downloadDirectory: outDir }); // File 100's ML data was cached by the first run and is kept. - expect(fileJSON(outDir, 100).mlData).toEqual(ML_PAYLOAD); + expect(fileJSON(outDir, 100).mlData).toEqual({ face: { faces: [] } }); expect(fileJSON(outDir, 100)).not.toHaveProperty("mlDataError"); for (const fileID of [101, 200]) { expect(fileJSON(outDir, fileID).mlDataError).toBe( @@ -1026,7 +1120,7 @@ describe("ML data in each file's JSON", () => { expect(result.failed).toBe(0); expect(result.skipped).toBe(3); - expect(fileJSON(outDir, 100).mlData).toEqual(ML_PAYLOAD); + expect(fileJSON(outDir, 100).mlData).toEqual({ face: { faces: [] } }); for (const fileID of [100, 101, 200]) { expect(fileJSON(outDir, fileID)).not.toHaveProperty("mlDataError"); } diff --git a/test/cli/metadata-backup.test.ts b/test/cli/metadata-backup.test.ts index 0b75804..c66c799 100644 --- a/test/cli/metadata-backup.test.ts +++ b/test/cli/metadata-backup.test.ts @@ -41,6 +41,7 @@ import { beforeAll, afterAll, describe, expect, it, vi } from "vitest"; import { init, toBase64, encryptBlob } from "../../src/crypto/index.js"; import * as jpegJs from "jpeg-js"; import { Client } from "../../src/client.js"; +import { DISCOVER_CATEGORIES } from "../../src/clip-keyword-vectors.js"; import { Library } from "../../src/library/index.js"; import { runMetadataBackup, @@ -56,8 +57,8 @@ import { type MetaMockState, } from "../ente-server.js"; -// One file per ML data request, so the two files of the mock account are -// fetched in two requests and one of them can fail on its own. +// One file per ML data request, so each file of the mock account is fetched in +// a request of its own and one of them can fail on its own. vi.mock("../../src/mldata-fetch.js", async (importOriginal) => ({ ...(await importOriginal<typeof import("../../src/mldata-fetch.js")>()), MLDATA_BATCH_SIZE: 1, @@ -66,6 +67,21 @@ vi.mock("../../src/mldata-fetch.js", async (importOriginal) => ({ const TEST_EMAIL = "metabackup@example.com"; const TEST_PASSWORD = "metapass"; +// The CLIP embedding of a photo of a sunset (see clip-keywords.test.ts), which +// the ML data of beach.jpg holds. +const SUNSET_EMBEDDING: number[] = JSON.parse( + readFileSync( + new URL("../clip-keywords-sunset.json", import.meta.url), + "utf-8", + ), +).embedding; + +// Identity, a Discover category of Ente's photos app that leaves out videos. +const IDENTITY = DISCOVER_CATEGORIES.find((c) => c.title === "Identity")!; + +// ML data holding faces and no CLIP embedding, which file 302 has. +const FACES_ONLY = { face: { faces: [{ faceID: "face-def", score: 0.9 }] } }; + let mock: MetaMockState; let testDir: string; @@ -204,6 +220,65 @@ const buildMetaMock = async (): Promise<MetaMockState> => { updationTime: 1710000000000000, }; + // A file in coll 10 with the ML data given. `fileType` is Ente's number: 0 + // an image, 1 a video. + const vacationFile = ( + id: number, + title: string, + fileType: number, + mlData: object, + ) => { + const fileKey = sodium.crypto_secretstream_xchacha20poly1305_keygen(); + const { ciphertext: encKey, nonce } = encryptSecretbox(fileKey, ck1); + const meta = encryptStreamBlob( + new TextEncoder().encode( + JSON.stringify({ + title, + fileType, + creationTime: 1700000000000000, + modificationTime: 1700000000000000, + }), + ), + fileKey, + ); + const ml = encryptBlob( + new Uint8Array(gzipSync(Buffer.from(JSON.stringify(mlData)))), + fileKey, + ); + return { + raw: { + id, + collectionID: 10, + ownerID: 42, + encryptedKey: toBase64(encKey), + keyDecryptionNonce: toBase64(nonce), + metadata: { + encryptedData: toBase64(meta.ciphertext), + decryptionHeader: toBase64(meta.header), + }, + file: { + decryptionHeader: toBase64(sodium.randombytes_buf(24)), + }, + thumbnail: { + decryptionHeader: toBase64(sodium.randombytes_buf(24)), + }, + updationTime: 1700000000000000, + }, + mlData: { + encryptedData: toBase64(ml.ciphertext), + decryptionHeader: toBase64(ml.header), + }, + }; + }; + // Files 300 and 301 in coll 10: a photo and a video whose ML data holds + // only a CLIP embedding, Identity's own text embedding. + const identityML = { clip: { embedding: IDENTITY.textEmbedding } }; + const identityPhoto = vacationFile(300, "passport.jpg", 0, identityML); + const identityVideo = vacationFile(301, "passport.mp4", 1, identityML); + // File 302 in coll 10: a photo whose ML data holds faces and no CLIP + // embedding. + const facesPhoto = vacationFile(302, "party.jpg", 0, FACES_ONLY); + // Encrypt ML data for file 100 (gzipped JSON, encrypted with file key) const mlPayload = JSON.stringify({ face: { @@ -230,7 +305,7 @@ const buildMetaMock = async (): Promise<MetaMockState> => { clip: { version: 1, client: "test", - embedding: [0.5, 0.6, 0.7], + embedding: SUNSET_EMBEDDING, }, }); const gzipped = gzipSync(Buffer.from(mlPayload)); @@ -270,12 +345,23 @@ const buildMetaMock = async (): Promise<MetaMockState> => { keyAttributes: account.keyAttributes, encryptedToken: account.encryptedToken, collections: [rawColl1, rawColl2], - filesByCollection: { 10: [rawFile1], 20: [rawFile2] }, + filesByCollection: { + 10: [ + rawFile1, + identityPhoto.raw, + identityVideo.raw, + facesPhoto.raw, + ], + 20: [rawFile2], + }, encryptedMLData: { 100: { encryptedData: toBase64(mlCiphertext), decryptionHeader: toBase64(mlHeader), }, + 300: identityPhoto.mlData, + 301: identityVideo.mlData, + 302: facesPhoto.mlData, }, fileCiphertexts: { 100: encFileBody1 }, }; @@ -432,7 +518,9 @@ describe("quak backup-metadata", () => { expect(account.email).toBe(TEST_EMAIL); }); - it("fetches and decrypts ML data by default", async () => { + // The CLIP embedding is left out of the ML data, and the keywords it gives + // are written in its place. + it("fetches and decrypts ML data by default, with keywords for the CLIP embedding", async () => { const outDir = join(testDir, "ml-data"); await runBackup(outDir); @@ -452,8 +540,41 @@ describe("quak backup-metadata", () => { expect(fileMeta.mlData.face.faces[0].faceID).toBe("face-abc"); expect(fileMeta.mlData.face.faces[0].score).toBeCloseTo(0.98); expect(fileMeta.mlData.face.faces[0].detection.box.x).toBeCloseTo(0.1); - expect(fileMeta.mlData.clip).toBeDefined(); - expect(fileMeta.mlData.clip.embedding).toEqual([0.5, 0.6, 0.7]); + expect(fileMeta.mlData).not.toHaveProperty("clip"); + expect(fileMeta.keywords).toEqual(["Sunset"]); + }); + + // Ente's photos app keeps videos out of some Discover categories, Identity + // among them. Files 300 and 301, a photo and a video, share one CLIP + // embedding, Identity's own text embedding, so the photo gets the keyword + // Identity and the video does not. + it("keeps a video out of a category that leaves out videos", async () => { + expect(IDENTITY.showVideo).toBe(false); + const outDir = join(testDir, "video-keywords"); + await runBackup(outDir); + + const vacation = join(outDir, "collections", "10-Vacation"); + const fileJSON = (fileID: number) => + JSON.parse(readFileSync(join(vacation, `${fileID}.json`), "utf-8")); + expect(fileJSON(300).keywords).toContain("Identity"); + expect(fileJSON(301)).toHaveProperty("keywords"); + expect(fileJSON(301).keywords).not.toContain("Identity"); + }); + + // A file's ML data can hold faces and no CLIP embedding. File 302's JSON + // then has the ML data and no keywords. + it("writes no keywords for ML data without a CLIP embedding", async () => { + const outDir = join(testDir, "no-clip"); + await runBackup(outDir); + + const fileMeta = JSON.parse( + readFileSync( + join(outDir, "collections", "10-Vacation", "302.json"), + "utf-8", + ), + ); + expect(fileMeta.mlData).toEqual(FACES_ONLY); + expect(fileMeta).not.toHaveProperty("keywords"); }); it("extracts EXIF from downloaded files when --exif is set", async () => { @@ -541,7 +662,7 @@ describe("quak backup-metadata when a file's ML data cannot be read", () => { "utf-8", ), ); - expect(ok.mlData.clip.embedding).toEqual([0.5, 0.6, 0.7]); + expect(ok.keywords).toEqual(["Sunset"]); expect(ok.mlDataError).toBeUndefined(); const failed = JSON.parse( @@ -599,7 +720,7 @@ describe("quak backup-metadata when a file's ML data cannot be read", () => { "utf-8", ), ); - expect(ok.mlData.clip.embedding).toEqual([0.5, 0.6, 0.7]); + expect(ok.keywords).toEqual(["Sunset"]); }); it("exits 0 when every ML data request succeeds", async () => { diff --git a/test/clip-keywords-sunset.json b/test/clip-keywords-sunset.json new file mode 100644 index 0000000..499071f --- /dev/null +++ b/test/clip-keywords-sunset.json @@ -0,0 +1,109 @@ +{ + "photo": "https://upload.wikimedia.org/wikipedia/commons/d/db/Sunset_off_the_Massachusetts_coast_-_NOAA.jpg", + "sha256": "be62f3289c9bf7b75af68cd6978021d54e8a531d5d4cc493f6e9fbf5e1ab74d3", + "embedding": [ + -0.0021123015, -0.04437615, -0.062354308, 0.006249992, 0.022141185, + 0.011159019, 0.01227392, -0.00030532264, 0.03186437, 0.09196922, + 0.039111234, -0.037084445, -0.032962628, -0.019820206, -0.013141825, + 0.0047183544, 0.017137334, 0.046581306, 0.0047622235, 0.04982521, + 0.023128154, 0.032080866, 0.037709303, 0.019241406, 0.0032100384, + -0.014275843, 0.049337693, 0.025953151, 0.014263995, -0.0062328726, + -0.040504098, -0.015083303, -0.008123875, -0.006233639, -0.05352683, + -0.0048083956, 0.0763486, -0.013155067, 0.025114346, 0.022016674, + 0.0001588023, -0.010341572, -0.0037518279, -0.019181008, 0.04549337, + -0.0010725886, 0.004636866, 0.017177915, -0.0117317885, 0.0026097358, + 0.030738998, 0.013158572, -0.08856403, -0.0121391015, 0.011477229, + -0.0033104012, 0.035524756, -0.018607879, -0.0071739843, -0.031906743, + 0.0056713386, -0.0044195866, 0.00292471, 0.00061686034, -0.043075223, + 0.09040427, 0.001522537, -0.011445652, 0.009609886, -0.0053201183, + -0.0057452377, 0.00972853, -0.0024397348, -0.021255083, 0.015893811, + -0.006020548, -0.09428023, 0.014277583, -0.017975299, -0.011878315, + 0.011825521, 0.00014203975, -0.0020437292, -0.05305343, -0.024840152, + -0.011336872, -0.002796541, -0.00036171873, 0.004530499, 0.044486605, + 0.021602718, -0.01292857, 0.024530916, 0.019559948, -0.039928272, + -0.0681174, 0.049101, 0.02461532, 0.0365531, -0.04070358, -0.011011939, + -0.0027188526, -0.020370876, 0.012904856, 0.043157287, -0.051658224, + -0.3726012, 0.019549074, -0.0499252, -0.04800559, -0.011493859, + 0.02893256, 0.060341954, 0.0013929688, -0.051283866, -0.024286803, + 0.044124287, -0.005995806, -0.062817656, 0.007369614, -0.021470634, + -0.028147545, 0.00411659, 0.04234773, 0.03787647, 0.008579682, + -0.038568925, -0.027498025, -0.028416378, 0.0022012168, 0.010906191, + -0.013745787, 0.018267838, 0.013531097, -0.061559845, 0.004682517, + -0.005993303, 0.0028113858, -0.0028286583, 0.0026029332, -0.06693403, + -0.0072381413, -0.02419584, 0.0037009937, -0.0011296495, -0.030310456, + 0.0005999609, 0.009945354, 0.024915686, 0.038672045, 0.023009432, + -0.05093051, -0.09101409, -0.06686167, -0.06211481, -0.022600474, + 0.0024902367, -0.0067171026, -0.016769778, -0.0032803628, 0.008623083, + -0.0016110836, 0.014925073, 0.033811048, 0.012829449, -0.052575592, + 0.03728281, -0.0051201633, 0.021238064, -0.031086605, -0.0043402985, + 0.25054494, -0.028396713, -0.010725523, -0.023445267, 0.021751683, + 0.003130971, 0.012270548, -0.014836475, 0.014466885, 0.0021478906, + -0.088153616, 0.015356315, 0.039639276, -0.040283255, 0.014923924, + -0.029303031, -0.004475201, 0.047299225, -0.06726505, -0.012491676, + -0.002080582, 0.03506341, 0.04146718, 0.0074114986, -0.009582633, + 0.006623749, -0.05454898, -0.0034387247, -0.014780821, 0.03698328, + -0.034441486, 0.22032379, -0.021792699, 0.008001907, 0.0341019, + 0.0025784625, -0.0071689896, -0.014151225, -0.050547987, -0.030372301, + -0.03288388, -0.042987447, -0.0033656165, 0.021502906, -0.0030374376, + -0.024413, -0.03506603, 0.030891743, 0.0069994032, 0.004887366, + 0.030707076, 0.0065348223, 0.0008240555, 0.016809713, -0.024151647, + -0.0040458064, 0.044567548, 0.03908212, -0.061782435, 0.0026002831, + 0.02003242, -0.0115451, -0.045135964, -0.0663542, 0.005240975, + 0.0010777109, -0.009276002, -0.011919081, 0.027654119, -0.013634551, + 0.0129154315, -0.013195512, 0.040571272, -0.099290505, -0.04656693, + -0.20979854, 0.04181004, -0.06769166, -0.04121946, 0.018260444, + 0.005617506, -0.15694222, 0.032472793, 0.014189598, 0.008345103, + -0.041118417, 0.0074527003, -0.011494104, 0.004906474, -0.0027100458, + 0.0009994594, -0.041061398, -0.045274835, 0.015187793, 0.034275036, + -0.02813289, 0.02098019, -0.015043198, 0.03716667, 0.013542642, + -0.008104835, 0.014965486, -0.027978558, 0.021163663, -0.006942737, + -0.034969814, -0.038961593, 0.016802894, -0.012318175, -0.047821593, + -0.01933216, 0.0064432374, 0.0322358, -0.020437784, 0.02659447, + -0.03022022, -0.019443465, -0.094178624, -0.017009107, 0.03860671, + -0.00023695534, -0.033089854, -0.014677462, 0.012356399, 0.018040499, + -0.19360113, 0.011710817, -0.019871691, 0.020124774, 0.024948105, + -0.024573864, -0.028349072, -0.041731454, 0.032646254, 0.015098165, + 0.0018447012, -0.01129152, -0.02481687, 0.036048, 0.20797811, + 0.04317757, -0.017239755, 0.033431605, -0.011384422, 0.0099174455, + -0.19739223, -0.008901062, -0.056426458, 0.014591864, 0.0044474266, + 0.008209326, -0.012241819, 0.042553645, -0.021552453, 0.008611734, + 0.048944127, -0.013872049, 0.019179989, 0.008183264, -0.053223096, + -0.027327595, -0.0033542586, -0.012353858, -0.010941209, 0.010212546, + 0.029714134, -0.011385593, -0.0013529479, 0.014505385, 0.01148663, + 0.012613636, -0.0074430504, -0.014008101, -0.028146038, -0.042778917, + 0.02143165, 0.016812252, -0.06286215, 0.02085532, -0.035128277, + 0.0241282, 0.01397908, -0.034992814, 0.013153766, 0.0068236147, + 0.013594184, -0.06859393, 0.051849406, 0.029675119, -0.0037699796, + -0.00013216575, 0.02916647, 0.031859048, -0.015006051, 0.0001439966, + 0.0026584554, -0.0046948595, 0.039754897, 0.034873154, 0.0012870439, + -0.011516082, -0.074089095, 0.019241294, 0.00076195505, 0.0027334464, + -0.036206458, -0.0011109262, -0.048838835, -0.00204685, 0.01179991, + 0.0068136845, 0.009852495, 0.017308988, 0.03159545, 0.058803353, + 0.038116634, 0.008836143, -0.010460637, -0.010994099, -0.0012926942, + -0.048370842, 0.037696566, -0.08093239, -0.0041242186, -0.005866373, + 0.02048607, -0.12549639, 0.011705739, -0.0021425558, 0.018078592, + -0.038228087, 0.004245971, -0.11560507, 0.01381203, -0.02734893, + 0.029595908, 0.101082794, -0.042842254, -0.08397301, -0.1546237, + -0.037624564, 0.0069606556, -0.008020479, 0.0043268185, 0.016504597, + 0.044554908, -0.05240209, 0.020519335, -0.050994243, -0.019861093, + 0.022066997, -0.0188911, 0.012919696, -0.022872692, -0.012961374, + -0.020071112, 0.0074553066, 0.06402014, 0.04065211, 0.014139608, + 0.06221546, -0.02574805, 0.010114545, 0.020619567, 0.032264773, + -0.009148828, 0.022849955, -0.016593235, -0.032456633, -0.006737303, + -0.0037493357, -0.065912865, -0.0024181998, -0.0037048983, -0.019304488, + 0.009085037, 0.057529982, -0.0378338, 0.01792042, 0.01885012, + -0.0043600523, 0.017382368, -0.027792772, 0.006749118, -0.0053297766, + -0.011628587, 0.015736576, 0.024801968, 0.011543913, -0.01159873, + -0.000759976, 0.042183932, 0.05764386, -0.06422416, 0.010577374, + -0.02261249, 0.02282793, -0.05141075, -0.11287078, -0.020670144, + -0.020953743, 0.030885734, -0.012597732, -0.121472664, 0.028447151, + -0.0415664, -0.010061708, 0.06441338, 0.0271207, -0.0016693748, + -0.0045694243, 0.020342957, 0.030528478, 0.007129517, -0.0029600926, + -0.007061397, 0.06036486, -0.004736018, -0.0090786135, -0.07601116, + -0.039556272, -0.051269546, -0.019473685, 0.029858151, 0.028464556, + 0.012424476, 0.013742254, 0.008742927, 0.006065751, -0.009892663, + 0.07699999, -0.003443197, 0.0027863425, 0.029197995, 0.0211013, + 0.01098379, 0.016684106, -0.028917195, 0.008122037, 0.0265241, + -0.04461223 + ] +} diff --git a/test/clip-keywords.test.ts b/test/clip-keywords.test.ts new file mode 100644 index 0000000..13a6a5d --- /dev/null +++ b/test/clip-keywords.test.ts @@ -0,0 +1,129 @@ +// `clipKeywords`: the keywords quak writes for a file in place of its CLIP +// embedding. +// +// Ente's ML data for a file holds a CLIP embedding, 512 numbers its image +// model gave the photo. Ente's photos app puts a file in a category of its +// Discover section when the cosine similarity of that embedding and the text +// embedding of the category's query is at least the category's minimum score, +// and keeps videos out of some categories. `clipKeywords` applies that rule +// and returns the titles of the categories a file falls in, highest score +// first; `quak backup` and `quak backup-metadata` write them as `keywords`. + +import { readFileSync } from "node:fs"; +import { describe, expect, it } from "vitest"; + +import { DISCOVER_CATEGORIES } from "../src/clip-keyword-vectors.js"; +import { clipKeywords } from "../src/clip-keywords.js"; + +// 512 numbers: the ones given, then zeros. +const vector = (...start: number[]): number[] => [ + ...start, + ...new Array<number>(512 - start.length).fill(0), +]; + +const withEmbedding = (embedding: unknown) => ({ + face: { faces: [] }, + clip: { embedding }, +}); + +// Two made-up categories, each with a text embedding along one axis. The +// embedding [3, 4] has a cosine similarity of 3/5 with "Across" and 4/5 with +// "Up". Videos are kept out of "Across". +const ACROSS = { + title: "Across", + query: "across", + minScore: 0.6, + showVideo: false, + textEmbedding: vector(1), +}; +const UP = { + title: "Up", + query: "up", + minScore: 0.5, + showVideo: true, + textEmbedding: vector(0, 1), +}; + +describe("clipKeywords", () => { + it("gives each category whose minimum score the similarity reaches, highest score first", () => { + expect( + clipKeywords(withEmbedding(vector(3, 4)), false, [ACROSS, UP]), + ).toEqual(["Up", "Across"]); + }); + + it("leaves out a category when the similarity is just below its minimum score", () => { + const stricter = { ...ACROSS, minScore: 0.6 + 1e-9 }; + + expect( + clipKeywords(withEmbedding(vector(3, 4)), false, [stricter, UP]), + ).toEqual(["Up"]); + }); + + it("keeps a video out of a category that leaves out videos", () => { + expect( + clipKeywords(withEmbedding(vector(3, 4)), true, [ACROSS, UP]), + ).toEqual(["Up"]); + }); + + it("gives an empty list for an embedding that falls in no category", () => { + expect( + clipKeywords(withEmbedding(vector(0, 0, 1)), false, [ACROSS, UP]), + ).toEqual([]); + }); + + it.each([ + ["no CLIP embedding", { face: { faces: [] } }], + ["a CLIP part that is not an object", { clip: null }], + ["a CLIP part without an embedding", { clip: {} }], + ["an embedding of 511 numbers", withEmbedding(vector(3, 4).slice(1))], + ["an embedding with NaN in it", withEmbedding(vector(3, NaN))], + ["an embedding with Infinity in it", withEmbedding(vector(Infinity))], + ["an embedding of strings", withEmbedding(vector(3, 4).map(String))], + ["an embedding of zeros", withEmbedding(vector())], + ])("gives no keywords for ML data with %s", (_, mlData) => { + expect(clipKeywords(mlData, false, [ACROSS, UP])).toBeUndefined(); + }); +}); + +// The CLIP embedding Ente's image model gives a public-domain NOAA photo of a +// sunset, made by bin/clip-keyword-vectors from the photo at `photo`. +const SUNSET: { photo: string; sha256: string; embedding: number[] } = + JSON.parse( + readFileSync( + new URL("clip-keywords-sunset.json", import.meta.url), + "utf-8", + ), + ); + +describe("Ente's Discover categories", () => { + it("are the 16 of Ente's photos app, each with a text embedding of norm 1", () => { + expect(DISCOVER_CATEGORIES.map((c) => c.title)).toEqual([ + "Identity", + "Screenshots", + "QR Codes", + "Receipts", + "Notes", + "Memes", + "Visiting Cards", + "Babies", + "Pets", + "Selfies", + "Wallpapers", + "Food", + "Celebrations", + "Sunset", + "Hills", + "Greenery", + ]); + for (const category of DISCOVER_CATEGORIES) { + expect(category.textEmbedding).toHaveLength(512); + expect(Math.hypot(...category.textEmbedding)).toBeCloseTo(1, 5); + } + }); + + it("give a photo of a sunset the keyword Sunset and no other", () => { + expect( + clipKeywords({ clip: { embedding: SUNSET.embedding } }, false), + ).toEqual(["Sunset"]); + }); +}); -- 2.54.0 From af1491966c146d0ae147aa19cfcbe374edb2ffce Mon Sep 17 00:00:00 2001 From: clawbot <35+clawbot@noreply.example.org> Date: Mon, 26 Oct 2026 09:04:39 +0100 Subject: [PATCH 2/7] quak files COLLECTION fetches that collection's files alone (closes #217) `quak files` with a collection ID used to wait for a full refresh, which on an empty cache fetched the files of every collection in the account. It now reads the collection list, then that one collection's files through the new `Library.freshFiles()`, and stores nothing. The files print in the server's order, and no refresh progress lines appear. Each file prints once, as the latest copy the server sent. `quak files` without a collection ID still waits for a refresh. Some tests ran `quak files` with a collection ID to fill the cache, to show a refresh's progress or to show what `-q` hides. They now run a command that still refreshes. Model: opus-5-5 --- README.md | 31 ++++-- TODO.md | 7 ++ src/cli-commands.ts | 23 ++-- src/cli-read.ts | 53 ++++----- src/library/index.ts | 25 ++++- test/cli/commands.test.ts | 228 ++++++++++++++++++++++++++------------ test/cli/log.test.ts | 4 +- test/cli/read.test.ts | 58 ++++++---- 8 files changed, 282 insertions(+), 147 deletions(-) diff --git a/README.md b/README.md index 8e34535..640417a 100644 --- a/README.md +++ b/README.md @@ -685,17 +685,20 @@ library. The read commands — `collections`, `files`, `get`, `get-thumb`, `helper fix-missing-thumbnails` — force a fresh server round-trip before they answer, so they report current account state rather than whatever the cache last held. `collections` asks the server for the collection list alone, in one -request, and neither fetches any collection's files nor changes the cache. The -others wait for a refresh. With an empty cache, as on a first run, that refresh -fetches the files of every collection, one collection after another, and stores -nothing until it has them all, which on a large account takes a while. Meanwhile -the command logs a line on stderr for each collection whose files it has -fetched, such as `Fetched the files of 3 of 120 collections`, with `--json` too; -`backup` does the same while the refresh it starts with runs. stdout holds only -the command's output. If that round-trip fails, the command prints the error on -one line and exits 1, or 3 when the server no longer accepts the saved session -(see "Session handling"). An album or file quak cannot decrypt does not fail it; -the command answers from what did decrypt (see "Default reads vs. fresh reads"). +request, and neither fetches any collection's files nor changes the cache. +`files` with a collection ID asks for the same list, then for that one +collection's files, and does not change the cache either. The others wait for a +refresh, and so does `files` without a collection ID. With an empty cache, as on +a first run, that refresh fetches the files of every collection, one collection +after another, and stores nothing until it has them all, which on a large +account takes a while. Meanwhile the command logs a line on stderr for each +collection whose files it has fetched, such as +`Fetched the files of 3 of 120 collections`, with `--json` too; `backup` does +the same while the refresh it starts with runs. stdout holds only the command's +output. If that round-trip fails, the command prints the error on one line and +exits 1, or 3 when the server no longer accepts the saved session (see "Session +handling"). An album or file quak cannot decrypt does not fail it; the command +answers from what did decrypt (see "Default reads vs. fresh reads"). `--cache-dir` overrides where the cache lives; without it each account gets its own directory under the per-user cache path. @@ -1180,6 +1183,12 @@ since the stored copy, so a collection stored without its files would never get them. `quak collections` uses it (issue https://git.eeqj.de/sneak/quak/issues/210). +`await lib.freshFiles(collection)` is the fresh read of one collection's files. +It answers with every live file in that collection that decrypts, in the +server's order. `collection` is a record that `freshCollections()` returned. The +method fetches no other collection's files and stores nothing. `quak files` with +a collection ID uses it. + An album or file in the server's answer that quak cannot decrypt, or whose metadata is not JSON, does not fail the refresh. The refresh leaves it out, an album with all its files, and drops any older copy the cache held, so reads diff --git a/TODO.md b/TODO.md index 99ae803..cbf043d 100644 --- a/TODO.md +++ b/TODO.md @@ -26,6 +26,13 @@ declares one. # Completed Steps +- 2026-10-26: `quak files` with a collection ID fetches that collection's files + alone (https://git.eeqj.de/sneak/quak/issues/217). It used to wait for a + refresh, which on an empty cache fetched the files of every collection. It now + reads the collection list through `lib.freshCollections()`, then the one + collection's files through the new `lib.freshFiles()`. It stores nothing, and + the files print in the server's order. + - 2026-10-22: The JSON `quak backup` and `quak backup-metadata` write for a file holds `keywords` in place of the CLIP embedding in its ML data (https://git.eeqj.de/sneak/quak/issues/218). The keywords are the titles of diff --git a/src/cli-commands.ts b/src/cli-commands.ts index 52f1b05..9373717 100644 --- a/src/cli-commands.ts +++ b/src/cli-commands.ts @@ -128,12 +128,13 @@ const readLibraryClient = (client: Client): LibraryClient => ({ contentSource: () => client.contentSource(), }); -// Open a library for a single point command: the aggressive background precache -// (issue #48) is off — a one-shot `collections` or `get` must not start -// downloading the whole account — and no refresh starts as it opens or on a -// timer: each command asks for the one it needs, and `collections` needs none -// (issue #210). While a refresh runs, each collection whose files it has -// fetched is logged, so a large account shows progress rather than nothing. +// Open a library for a single point command. The aggressive background +// precache (issue #48) is off, because a one-shot `collections` or `get` must +// not start downloading the whole account. No refresh starts as the library +// opens or on a timer. Each command asks for the refresh it needs. +// `collections` and `files` with a collection ID need none (issues #210 and +// #217). While a refresh runs, each collection whose files it has fetched is +// logged, so a large account shows progress rather than nothing. const openReadLibrary = (ctx: CliContext, client: Client): Promise<Library> => Library.open({ client: readLibraryClient(client), @@ -282,11 +283,11 @@ export const filesCommand = async ( const lib = await openReadLibrary(ctx, client); try { - // Force a server round-trip and list in enumeration order (issue #36 - // amendment, issue #52). Each file prints from its own decrypted - // metadata (raw title, microsecond creationTime) via cli-output, and in - // the pre-library CLI's enumeration order, not the projection's - // newest-first order. + // Ask the server for current state (issue #36 amendment, issue #52). + // Each file prints from its own decrypted metadata (raw title, + // microsecond creationTime) via cli-output. The files print in the + // order `freshFiles` returns them, not the projection's newest-first + // order. const files = await freshFiles(lib, collectionID); if (!files) { ctx.log("fatal", `Collection ${collectionID} not found`); diff --git a/src/cli-read.ts b/src/cli-read.ts index 70b1f21..a9f57f1 100644 --- a/src/cli-read.ts +++ b/src/cli-read.ts @@ -2,34 +2,35 @@ // // `collections`, `files`, `get`, and `get-thumb` must answer for // the account's state at the moment the command runs, not for whatever the -// local cache last happened to hold (owner amendment, issue #36). `files`, -// `get` and `get-thumb` therefore force a server round-trip through -// `Library.fresh()` and only then read — so a collection, file, or metadata -// change made elsewhere is visible. `collections` asks the library for the -// collection list alone through `Library.freshCollections()` (issue #210): a -// refresh would also fetch the files of every collection, which on a large -// account takes long and which `collections` does not print. +// local cache last happened to hold (owner amendment, issue #36). `get`, +// `get-thumb` and `files` without a collection ID therefore force a server +// round-trip through `Library.fresh()` and only then read, so a change made +// elsewhere is visible. `collections` asks the library for the collection list +// alone through `Library.freshCollections()` (issue #210): a refresh would also +// fetch the files of every collection, which on a large account takes long and +// which `collections` does not print. `files` with a collection ID reads that +// list too, then that one collection's files through `Library.freshFiles()` +// (issue #217). // -// `collections` lists in the server's order and `files` in the library's own -// enumeration order (`listFiles()`), the orders the pre-library CLI printed, -// rather than the `albums`/`photos` projection's newest-first order, which -// re-sorts the rows. The field values still come from each record's raw -// metadata via `cli-output.ts`. `files` without a collection ID lists every -// file in the account once (issue #211): the library keeps a record of a file -// for each collection that holds it, and only the first collection's record is -// listed. +// `collections` and `files` with a collection ID list in the server's order. +// `files` without one lists in the library's own enumeration order +// (`listFiles()`). Neither command uses the `albums`/`photos` projection's +// newest-first order, which re-sorts the rows. The field values still come +// from each record's raw metadata via `cli-output.ts`. `files` without a +// collection ID lists every file in the account once (issue #211): the library +// keeps a record of a file for each collection that holds it, and only the +// first collection's record is listed. import type { Collection, EnteFile } from "./model/types.js"; import type { Photo, PhotosAPI } from "./library/index.js"; // The slice of `Library` these helpers read. `Library` satisfies it -// structurally; a test can drive them with a stand-in that records the -// `fresh()` and `freshCollections()` calls and serves records in a known -// enumeration order. +// structurally; a test can drive them with a stand-in that records each fresh +// read and serves records in a known enumeration order. export interface FreshReadLibrary { fresh(): Promise<unknown>; freshCollections(): Promise<Collection[]>; - getCollection(id: number): Collection | undefined; + freshFiles(collection: Collection): Promise<EnteFile[]>; listCollections(): Collection[]; listFiles(collectionID: number): EnteFile[]; getFileByID(fileID: number): EnteFile | undefined; @@ -42,21 +43,21 @@ export const freshCollections = ( lib: FreshReadLibrary, ): Promise<Collection[]> => lib.freshCollections(); -// The files of one collection, current as of a forced refresh, in enumeration +// The files of one collection, as the server holds them now, in the server's // order. `undefined` (not an empty list) when the collection does not exist, so // the caller can tell "no such collection" from "an empty collection". Without -// a collection ID, every file in the account, each once, as held by the first -// collection that holds it. +// a collection ID, every file in the account as of a forced refresh, each once, +// as held by the first collection that holds it. export const freshFiles = async ( lib: FreshReadLibrary, collectionID?: number, ): Promise<EnteFile[] | undefined> => { - await lib.fresh(); if (collectionID !== undefined) { - return lib.getCollection(collectionID) - ? lib.listFiles(collectionID) - : undefined; + const collections = await lib.freshCollections(); + const collection = collections.find((c) => c.id === collectionID); + return collection ? lib.freshFiles(collection) : undefined; } + await lib.fresh(); const listed = new Set<number>(); const files: EnteFile[] = []; for (const collection of lib.listCollections()) { diff --git a/src/library/index.ts b/src/library/index.ts index a3bed63..5980fbc 100644 --- a/src/library/index.ts +++ b/src/library/index.ts @@ -20,9 +20,10 @@ // // `freshCollections()` is the fresh read of the collection list alone (issue // #210): one collections request, no collection's files, and nothing stored, -// for a caller such as `quak collections` that needs no more. A caller that -// asks for each refresh it needs can also open the library with -// `refreshOnOpen: false`, so that `open()` starts none. +// for a caller such as `quak collections` that needs no more. `freshFiles()` +// is the fresh read of one collection's files, and stores nothing either +// (issue #217). A caller that asks for each refresh it needs can also open the +// library with `refreshOnOpen: false`, so that `open()` starts none. // // A refresh reports through `onProgress` each collection whose files it has // fetched, so a large account, whose first refresh fetches the files of every @@ -625,6 +626,24 @@ export class Library { return collections; } + // The fresh read of one collection's files (issue #217): every live file + // in it that decrypts, in the server's order. `collection` is a record + // `freshCollections()` returned, whose key decrypts the files. It fetches + // no other collection's files. Like `freshCollections()`, it stores + // nothing. It returns each file once, as the latest copy the server sent, + // where the first copy was. + async freshFiles(collection: Collection): Promise<EnteFile[]> { + if (this.closed) throw new Error("the library is closed"); + const { files } = await this.client.filesSince({ + collectionID: collection.id, + collectionKey: collection.key, + sinceTime: 0, + }); + const byID = new Map<number, EnteFile>(); + for (const file of files) byID.set(file.id, file); + return [...byID.values()]; + } + // Back up every in-scope file to `opts.downloadDirectory`, or else the // library's, each original at its save path, with a durable failure // ledger (issue #51). Takes the lock in that directory first, unless the diff --git a/test/cli/commands.test.ts b/test/cli/commands.test.ts index 3269fdb..3822da7 100644 --- a/test/cli/commands.test.ts +++ b/test/cli/commands.test.ts @@ -535,9 +535,7 @@ describe("files", () => { it("exits 1 for an unknown collection", async () => { expect(await filesCommand(context(), "9", {})).toBe(1); - expect(stderr.text).toBe( - FETCHED_BOTH_ALBUMS + "Collection 9 not found\n", - ); + expect(stderr.text).toBe("Collection 9 not found\n"); }); // Without a collection ID it lists every file in the account once (issue @@ -1390,11 +1388,11 @@ describe("backup-metadata --exif", () => { }); // Each test first runs `files` so the cache holds the account as it was, then -// changes the server under it. (`collections` would not do: it reads the -// collection list alone and stores nothing.) +// changes the server under it. (`collections` and `files` with a collection ID +// would not do: they store nothing.) describe("backup-metadata and the thumbnail helpers refresh first", () => { beforeEach(async () => { - expect(await filesCommand(context(), "1", {})).toBe(0); + expect(await filesCommand(context(), undefined, {})).toBe(0); stdout.text = ""; stderr.text = ""; }); @@ -1577,20 +1575,73 @@ describe("an ID given on the command line", () => { ); }); -// An account with many albums (issue #210). On an empty cache `collections` -// used to wait for a refresh that fetched and decrypted the files of every -// album, one album after another, before it printed anything, so on such an -// account it seemed to hang. The account here has 40 albums of one photo each, -// encrypted as Ente's clients encrypt them, on the fake Ente server of -// `test/ente-server.ts`, which also records the path of each request quak -// sends. The session is the one logging in to that server saves. +// An account with many albums (issues #210 and #217). On an empty cache +// `collections` and `files` with a collection ID used to wait for a refresh +// that fetched and decrypted the files of every album, one album after another. +// On such an account they seemed to hang. The account here has 40 albums of one +// photo each, encrypted as Ente's clients encrypt them, on the fake Ente server +// of `test/ente-server.ts`. The tests record the path and query of each +// request quak sends. The session is the one logging in to that server saves. describe("an account with many albums", () => { const ALBUMS = 40; const PASSWORD = "many-albums"; - const paths: string[] = []; + const requests: string[] = []; let server: typeof globalThis.fetch; let snapshot: ClientSnapshot; + // What `files` without a collection ID prints on an empty cache: every + // album's photo on stdout, and on stderr each album whose files its + // refresh fetched. + let everyPhoto = ""; + let progress = ""; + for (let n = 1; n <= ALBUMS; n++) { + everyPhoto += `${n * 100}\timage\tphoto-${n}.jpg\n`; + progress += `Fetched the files of ${n} of ${ALBUMS} collections\n`; + } + + // The key of each album, by album ID. + const albumKeys = new Map<number, Uint8Array>(); + + // A photo in album `collectionID` as the server sends it. These tests + // fetch no original and no thumbnail. + const photoRow = (photo: { + id: number; + collectionID: number; + title: string; + updationTime: number; + }): Record<string, unknown> => { + const photoKey = sodium.crypto_secretstream_xchacha20poly1305_keygen(); + const photoKeyBox = encryptSecretbox( + photoKey, + albumKeys.get(photo.collectionID)!, + ); + const metadata = encryptStreamBlob( + new TextEncoder().encode( + JSON.stringify({ + title: photo.title, + fileType: 0, + creationTime: 1000, + modificationTime: 1000, + }), + ), + photoKey, + ); + return { + id: photo.id, + collectionID: photo.collectionID, + ownerID: USER_ID, + encryptedKey: toBase64(photoKeyBox.ciphertext), + keyDecryptionNonce: toBase64(photoKeyBox.nonce), + metadata: { + encryptedData: toBase64(metadata.ciphertext), + decryptionHeader: toBase64(metadata.header), + }, + file: { decryptionHeader: "" }, + thumbnail: { decryptionHeader: "" }, + updationTime: photo.updationTime, + }; + }; + // Album n is named `Album n` and holds one photo, n * 100, titled // `photo-n.jpg`. beforeAll(async () => { @@ -1600,6 +1651,7 @@ describe("an account with many albums", () => { const filesByCollection: Record<number, Record<string, unknown>[]> = {}; for (let id = 1; id <= ALBUMS; id++) { const albumKey = sodium.crypto_secretbox_keygen(); + albumKeys.set(id, albumKey); const albumKeyBox = encryptSecretbox(albumKey, account.masterKey); const name = encryptSecretbox( new TextEncoder().encode(`Album ${id}`), @@ -1615,36 +1667,13 @@ describe("an account with many albums", () => { type: "album", updationTime: 1, }); - const photoKey = - sodium.crypto_secretstream_xchacha20poly1305_keygen(); - const photoKeyBox = encryptSecretbox(photoKey, albumKey); - const metadata = encryptStreamBlob( - new TextEncoder().encode( - JSON.stringify({ - title: `photo-${id}.jpg`, - fileType: 0, - creationTime: 1000, - modificationTime: 1000, - }), - ), - photoKey, - ); filesByCollection[id] = [ - { + photoRow({ id: id * 100, collectionID: id, - ownerID: USER_ID, - encryptedKey: toBase64(photoKeyBox.ciphertext), - keyDecryptionNonce: toBase64(photoKeyBox.nonce), - metadata: { - encryptedData: toBase64(metadata.ciphertext), - decryptionHeader: toBase64(metadata.header), - }, - // These tests fetch no original and no thumbnail. - file: { decryptionHeader: "" }, - thumbnail: { decryptionHeader: "" }, + title: `photo-${id}.jpg`, updationTime: 1, - }, + }), ]; } const ente = buildMetaFetch({ @@ -1664,7 +1693,8 @@ describe("an account with many albums", () => { : input instanceof URL ? input.href : input.url; - paths.push(new URL(url).pathname); + const { pathname, search } = new URL(url); + requests.push(pathname + search); return ente(input, init); }) as typeof globalThis.fetch; const client = await Client.login({ @@ -1676,16 +1706,17 @@ describe("an account with many albums", () => { }); beforeEach(() => { - paths.length = 0; + requests.length = 0; }); - // A context whose saved session sends its requests to the fake server. - const accountContext = (): CliContext => { + // A context whose saved session sends its requests to `fetch`, by default + // the fake server. + const accountContext = (fetch = server): CliContext => { const ctx = context(); saveSession(ctx.sessionDir, snapshot); return { ...ctx, - loadSession: (path) => loadSession(path, { fetch: server }), + loadSession: (path) => loadSession(path, { fetch }), }; }; @@ -1698,46 +1729,103 @@ describe("an account with many albums", () => { albums += `${id}\talbum\tAlbum ${id}\n`; expect(stdout.text).toBe(albums); expect(stderr.text).toBe(""); - expect(paths).toEqual(["/collections/v2"]); + expect(requests).toEqual(["/collections/v2?sinceTime=0"]); // It stored nothing that would make a refresh skip an album's files: - // `files` after it still finds the album's photo. + // `files` after it still lists every album's photo. stdout.text = ""; - expect(await filesCommand(ctx, "7", {})).toBe(0); - expect(stdout.text).toBe("700\timage\tphoto-7.jpg\n"); + expect(await filesCommand(ctx, undefined, {})).toBe(0); + expect(stdout.text).toBe(everyPhoto); }); - it("files counts on stderr each album whose files its refresh fetched", async () => { + it("files with a collection ID asks for the album list and that album's files alone", async () => { expect(await filesCommand(accountContext(), "7", {})).toBe(0); expect(stdout.text).toBe("700\timage\tphoto-7.jpg\n"); - let progress = ""; - for (let n = 1; n <= ALBUMS; n++) { - progress += `Fetched the files of ${n} of ${ALBUMS} collections\n`; - } + expect(stderr.text).toBe(""); + expect(requests).toEqual([ + "/collections/v2?sinceTime=0", + "/collections/v2/diff?collectionID=7&sinceTime=0", + ]); + }); + + // Here the server sends album 7's files in two pages. Photo 700 changed + // between them, so both pages hold a copy of it. + it("files with a collection ID lists only the later copy of a photo sent in two pages", async () => { + const firstPage = [ + photoRow({ + id: 700, + collectionID: 7, + title: "photo-7.jpg", + updationTime: 1, + }), + photoRow({ + id: 701, + collectionID: 7, + title: "photo-701.jpg", + updationTime: 1, + }), + ]; + const secondPage = [ + photoRow({ + id: 700, + collectionID: 7, + title: "photo-7-edited.jpg", + updationTime: 2, + }), + ]; + const paged = (async (input: RequestInfo | URL, init?: RequestInit) => { + const url = new URL(input instanceof Request ? input.url : input); + if ( + url.pathname !== "/collections/v2/diff" || + url.searchParams.get("collectionID") !== "7" + ) { + return server(input, init); + } + const first = url.searchParams.get("sinceTime") === "0"; + return new Response( + JSON.stringify({ + diff: first ? firstPage : secondPage, + hasMore: first, + }), + ); + }) as typeof globalThis.fetch; + + expect(await filesCommand(accountContext(paged), "7", {})).toBe(0); + + // Photo 700 is listed once, with the later title, before photo 701. + expect(stdout.text).toBe( + "700\timage\tphoto-7-edited.jpg\n701\timage\tphoto-701.jpg\n", + ); + }); + + // The cache stays empty, so a later `files` without a collection ID still + // fetches the files of all 40 albums. + it("files with a collection ID stores nothing in the cache", async () => { + const ctx = accountContext(); + expect(await filesCommand(ctx, "7", {})).toBe(0); + expect(existsSync(join(root, "cache", "metadata.json"))).toBe(false); + + stdout.text = ""; + expect(await filesCommand(ctx, undefined, {})).toBe(0); + expect(stdout.text).toBe(everyPhoto); + expect(stderr.text).toBe(progress); + }); + + it("files counts on stderr each album whose files its refresh fetched", async () => { + expect(await filesCommand(accountContext(), undefined, {})).toBe(0); + + expect(stdout.text).toBe(everyPhoto); expect(stderr.text).toBe(progress); }); // `--json` shapes stdout only, so the refresh's progress is still counted // on stderr. it("files --json counts on stderr each album whose files its refresh fetched", async () => { - expect(await filesCommand(accountContext(), "7", { json: true })).toBe( - 0, - ); + const opts = { json: true }; + expect(await filesCommand(accountContext(), undefined, opts)).toBe(0); - expect(JSON.parse(stdout.text)).toEqual([ - { - id: 700, - title: "photo-7.jpg", - fileType: "image", - creationTime: 1000, - collectionID: 7, - }, - ]); - let progress = ""; - for (let n = 1; n <= ALBUMS; n++) { - progress += `Fetched the files of ${n} of ${ALBUMS} collections\n`; - } + expect(JSON.parse(stdout.text)).toHaveLength(ALBUMS); expect(stderr.text).toBe(progress); }); }); diff --git a/test/cli/log.test.ts b/test/cli/log.test.ts index 137c983..e30970e 100644 --- a/test/cli/log.test.ts +++ b/test/cli/log.test.ts @@ -237,10 +237,10 @@ describe("-q", () => { // The refresh's progress is left out; the line saying why the command // failed is the fatal one, and it is printed. it("prints only the line a failing command exits with", async () => { - expect(await quak("-q", "files", "9")).toEqual({ + expect(await quak("-q", "get", "999")).toEqual({ code: 1, stdout: "", - stderr: "Collection 9 not found\n", + stderr: "File 999 not found\n", }); }); diff --git a/test/cli/read.test.ts b/test/cli/read.test.ts index 66ff369..cb8dc1d 100644 --- a/test/cli/read.test.ts +++ b/test/cli/read.test.ts @@ -3,20 +3,21 @@ * issue #36, issue #52). * * The `collections`, `files`, `get`, and `get-thumb` commands must answer for - * current server state, not the local cache. `files`, `get` and `get-thumb` - * force a `Library.fresh()` round-trip before they read; the stand-in library - * below serves nothing until `fresh()` has been awaited, so a helper that read - * without refreshing would come back empty and fail here. `collections` asks - * the library for the collection list alone (`freshCollections()`, issue - * #210), and must not wait for a refresh, which would fetch the files of every + * current server state, not the local cache. `get`, `get-thumb` and `files` + * without a collection ID force a `Library.fresh()` round-trip before they + * read. The stand-in library below serves no cached record until `fresh()` has + * been awaited, so a helper that read without refreshing would come back empty + * and fail here. `collections` asks the library for the collection list alone + * (`freshCollections()`, issue #210). `files` with a collection ID asks for + * that list, then for that one collection's files (`freshFiles()`, issue + * #217). Neither may wait for a refresh, which would fetch the files of every * collection. * - * `collections` and `files` also list in the library's enumeration order - * (`listCollections`/`listFiles`) — the order the pre-library CLI printed — not + * `collections` and `files` also list in the order the library gives them, not * the albums/photos projection's newest-first order. The fixtures are seeded in - * an enumeration order that a newest-first sort would rearrange, so a - * regression to the projection order would fail here too. Field values still - * come from the raw metadata via `cli-output.ts`. + * an order that a newest-first sort would rearrange, so a regression to the + * projection order would fail here too. Field values still come from the raw + * metadata via `cli-output.ts`. */ import { describe, it, expect } from "vitest"; @@ -62,14 +63,16 @@ const file = ( updationTime: creationTime, }); -// A library that reveals its records only after `fresh()` has been awaited, and -// serves them in the enumeration order it was given. `freshCollections()` -// answers the collection list in that order without a refresh. `photos.byID` -// returns a stand-in `Photo` carrying just the fileID the helper passes -// through. +// A library that reveals its cached records only after `fresh()` has been +// awaited, and serves them in the enumeration order it was given. +// `freshCollections()` and `freshFiles()` answer in that order without a +// refresh. `photos.byID` returns a stand-in `Photo` carrying just the fileID +// the helper passes through. class FakeLibrary implements FreshReadLibrary { freshCalls = 0; freshCollectionsCalls = 0; + // The ID of each collection `freshFiles()` was asked for. + freshFilesCalls: number[] = []; private refreshed = false; constructor( @@ -88,9 +91,9 @@ class FakeLibrary implements FreshReadLibrary { return this.collections; } - getCollection(id: number): Collection | undefined { - if (!this.refreshed) return undefined; - return this.collections.find((c) => c.id === id); + async freshFiles(collection: Collection): Promise<EnteFile[]> { + this.freshFilesCalls.push(collection.id); + return this.files.filter((f) => f.collectionID === collection.id); } listCollections(): Collection[] { @@ -138,18 +141,24 @@ describe("CLI read helpers (issue #36 amendment, issue #52)", () => { expect(rows.map((c) => c.id)).not.toEqual(newestFirst); }); - it("freshFiles refreshes first, lists in enumeration order, keeps raw fields", async () => { - // Enumeration order by id 10, 11, 12; creationTimes ascending, so a + it("freshFiles with a collection ID reads that collection's files alone and keeps their order and raw fields", async () => { + // Given in the order 10, 11, 12; creationTimes ascending, so a // newest-first sort would reverse them. const files = [ file(10, 1, 1_700_000_000_000_000), file(11, 1, 1_700_000_000_000_001), file(12, 1, 1_700_000_000_000_002), + file(20, 2, 1_700_000_000_000_003), ]; - const lib = new FakeLibrary([collection(1, 100)], files); + const lib = new FakeLibrary( + [collection(1, 100), collection(2, 200)], + files, + ); const rows = await freshFiles(lib, 1); - expect(lib.freshCalls).toBe(1); + expect(lib.freshCalls).toBe(0); + expect(lib.freshCollectionsCalls).toBe(1); + expect(lib.freshFilesCalls).toEqual([1]); expect(rows?.map((f) => f.id)).toEqual([10, 11, 12]); // Field values come from raw metadata: microsecond creationTime and the @@ -206,7 +215,8 @@ describe("CLI read helpers (issue #36 amendment, issue #52)", () => { it("freshFiles returns undefined for an unknown collection", async () => { const lib = new FakeLibrary([collection(1, 100)], []); const rows = await freshFiles(lib, 999); - expect(lib.freshCalls).toBe(1); + expect(lib.freshCalls).toBe(0); + expect(lib.freshFilesCalls).toEqual([]); expect(rows).toBeUndefined(); }); -- 2.54.0 From f65c237e2cbce43cce8f24c31b9cab0c08eabcd3 Mon Sep 17 00:00:00 2001 From: clawbot <35+clawbot@noreply.example.org> Date: Mon, 26 Oct 2026 10:18:28 +0100 Subject: [PATCH 3/7] quak get and get-thumb on an empty cache stop fetching collections once they find the file (closes #222) `quak get` and `quak get-thumb` used to wait for a refresh, which on an empty cache fetched the files of every collection. Now, on an empty cache, `freshFile` reads the collection list, then one collection's files at a time, and stops at the first that holds the file. That search stores nothing. Once the cache holds the account's collections, `freshFile` refreshes as before and looks the file up in the cache. The commands fetch the content through the new `Library.originalOf()` and `Library.thumbnailOf()`, which take the file record itself, since the store may not hold it. On an empty cache they no longer print a refresh's progress lines. The `-q` test that relied on `get` refreshing now runs `helper fix-missing-thumbnails`. Model: opus-5-5 --- README.md | 25 ++++-- TODO.md | 9 ++ src/cli-commands.ts | 19 ++--- src/cli-read.ts | 31 ++++--- src/library/content.ts | 49 ++++++++--- src/library/index.ts | 32 +++++++ test/cli/commands.test.ts | 121 ++++++++++++++++++--------- test/cli/log.test.ts | 8 +- test/cli/read.test.ts | 68 +++++++++------ test/library/content-library.test.ts | 6 ++ test/library/content.test.ts | 19 +++++ 11 files changed, 278 insertions(+), 109 deletions(-) diff --git a/README.md b/README.md index 640417a..97d8e56 100644 --- a/README.md +++ b/README.md @@ -687,12 +687,16 @@ answer, so they report current account state rather than whatever the cache last held. `collections` asks the server for the collection list alone, in one request, and neither fetches any collection's files nor changes the cache. `files` with a collection ID asks for the same list, then for that one -collection's files, and does not change the cache either. The others wait for a -refresh, and so does `files` without a collection ID. With an empty cache, as on -a first run, that refresh fetches the files of every collection, one collection -after another, and stores nothing until it has them all, which on a large -account takes a while. Meanwhile the command logs a line on stderr for each -collection whose files it has fetched, such as +collection's files, and does not change the cache either. On an empty cache, +`get` and `get-thumb` ask for the same list, then for one collection's files at +a time, until they find the file. They store none of the collections or file +records they read. The original or thumbnail they fetch is kept in the cache. +Once the cache holds the account's collections, they wait for a refresh instead. +The others wait for a refresh, and so does `files` without a collection ID. With +an empty cache, as on a first run, that refresh fetches the files of every +collection, one collection after another, and stores nothing until it has them +all, which on a large account takes a while. Meanwhile the command logs a line +on stderr for each collection whose files it has fetched, such as `Fetched the files of 3 of 120 collections`, with `--json` too; `backup` does the same while the refresh it starts with runs. stdout holds only the command's output. If that round-trip fails, the command prints the error on one line and @@ -1187,7 +1191,8 @@ https://git.eeqj.de/sneak/quak/issues/210). It answers with every live file in that collection that decrypts, in the server's order. `collection` is a record that `freshCollections()` returned. The method fetches no other collection's files and stores nothing. `quak files` with -a collection ID uses it. +a collection ID uses it, and so do `quak get` and `quak get-thumb` on an empty +cache. An album or file in the server's answer that quak cannot decrypt, or whose metadata is not JSON, does not fail the refresh. The refresh leaves it out, an @@ -1270,6 +1275,12 @@ source. An original that `original()`, `content()` or `exif()` downloads lands in the cache, which does not make `isLocal` true; only `download()` and `lib.backup()` do. +`await lib.originalOf(file, opts?)` and `await lib.thumbnailOf(file, opts?)` +fetch and cache as `photo.original()` and `photo.thumbnail()` do. `file` is an +`EnteFile` the caller holds, and the library need not hold it. On an empty +cache, `quak get` and `quak get-thumb` pass one that `lib.freshFiles()` +returned. + Lower-level accessors that return decrypted model objects (which hold key material) are also available: `listCollections()`, `getCollection(id)`, `listFiles(collectionID)`, `getFile(collectionID, fileID)`, and diff --git a/TODO.md b/TODO.md index cbf043d..197ab3d 100644 --- a/TODO.md +++ b/TODO.md @@ -26,6 +26,15 @@ declares one. # Completed Steps +- 2026-10-26: On an empty cache, `quak get` and `quak get-thumb` stop fetching + collections' files once they find the file + (https://git.eeqj.de/sneak/quak/issues/222). They used to wait for a refresh, + which on an empty cache fetched the files of every collection. Now they read + the collection list, then one collection's files at a time. They store none of + the collections or file records they read, and fetch the original or thumbnail + through the new `lib.originalOf()` and `lib.thumbnailOf()`. Once the cache + holds the account's collections, they still wait for a refresh. + - 2026-10-26: `quak files` with a collection ID fetches that collection's files alone (https://git.eeqj.de/sneak/quak/issues/217). It used to wait for a refresh, which on an empty cache fetched the files of every collection. It now diff --git a/src/cli-commands.ts b/src/cli-commands.ts index 9373717..1bb7d13 100644 --- a/src/cli-commands.ts +++ b/src/cli-commands.ts @@ -133,8 +133,9 @@ const readLibraryClient = (client: Client): LibraryClient => ({ // not start downloading the whole account. No refresh starts as the library // opens or on a timer. Each command asks for the refresh it needs. // `collections` and `files` with a collection ID need none (issues #210 and -// #217). While a refresh runs, each collection whose files it has fetched is -// logged, so a large account shows progress rather than nothing. +// #217). Nor do `get` and `get-thumb` on an empty cache (issue #222). While a +// refresh runs, each collection whose files it has fetched is logged, so a +// large account shows progress rather than nothing. const openReadLibrary = (ctx: CliContext, client: Client): Promise<Library> => Library.open({ client: readLibraryClient(client), @@ -324,14 +325,13 @@ export const getCommand = async ( try { // Force a server round-trip so the file resolves against current state // (issue #36 amendment, issue #52). - const resolved = await freshFile(lib, fileID); - if (!resolved) { + const file = await freshFile(lib, fileID); + if (!file) { ctx.log("fatal", `File ${fileID} not found`); return 1; } - const { photo, file } = resolved; - const result = await photo.original(); + const result = await lib.originalOf(file); // Default name is the file's own title, as the pre-library CLI used // (not the editedName-preferring projection title) (issue #52). if (result.videoPath === undefined) { @@ -382,14 +382,13 @@ export const getThumbCommand = async ( try { // Force a server round-trip so the file resolves against current state // (issue #36 amendment, issue #52). - const resolved = await freshFile(lib, fileID); - if (!resolved) { + const file = await freshFile(lib, fileID); + if (!file) { ctx.log("fatal", `File ${fileID} not found`); return 1; } - const { photo, file } = resolved; - const result = await photo.thumbnail(); + const result = await lib.thumbnailOf(file); // Default name is thumb_<file's own title>, as the pre-library CLI // used (not the projection title) (issue #52). const outPath = opts.out ?? thumbnailName(file); diff --git a/src/cli-read.ts b/src/cli-read.ts index a9f57f1..e5234fa 100644 --- a/src/cli-read.ts +++ b/src/cli-read.ts @@ -10,7 +10,8 @@ // fetch the files of every collection, which on a large account takes long and // which `collections` does not print. `files` with a collection ID reads that // list too, then that one collection's files through `Library.freshFiles()` -// (issue #217). +// (issue #217). On an empty cache, `get` and `get-thumb` read that list, then +// one collection's files at a time, until they find the file (issue #222). // // `collections` and `files` with a collection ID list in the server's order. // `files` without one lists in the library's own enumeration order @@ -22,7 +23,6 @@ // first collection's record is listed. import type { Collection, EnteFile } from "./model/types.js"; -import type { Photo, PhotosAPI } from "./library/index.js"; // The slice of `Library` these helpers read. `Library` satisfies it // structurally; a test can drive them with a stand-in that records each fresh @@ -34,7 +34,6 @@ export interface FreshReadLibrary { listCollections(): Collection[]; listFiles(collectionID: number): EnteFile[]; getFileByID(fileID: number): EnteFile | undefined; - photos: Pick<PhotosAPI, "byID">; } // Every live collection, current as of one collections request, in the @@ -70,16 +69,24 @@ export const freshFiles = async ( return files; }; -// One file, current as of a forced refresh, resolved to both its content -// handle (`Photo`, for fetching bytes) and its raw record (`EnteFile`, for the -// default output name and field values). `undefined` when the file is unknown. +// One file as the server holds it now, or `undefined` when no collection holds +// it. When the cache holds the account's collections, it refreshes and looks +// the file up in the cache. On an empty cache, it reads the collection list, +// then one collection's files at a time, and stops at the first collection +// that holds the file. That search stores nothing. export const freshFile = async ( lib: FreshReadLibrary, fileID: number, -): Promise<{ photo: Photo; file: EnteFile } | undefined> => { - await lib.fresh(); - const photo = lib.photos.byID({ fileID }); - const file = lib.getFileByID(fileID); - if (!photo || !file) return undefined; - return { photo, file }; +): Promise<EnteFile | undefined> => { + // Only a refresh stores collections, so a cache holding any holds them all. + if (lib.listCollections().length > 0) { + await lib.fresh(); + return lib.getFileByID(fileID); + } + for (const collection of await lib.freshCollections()) { + const files = await lib.freshFiles(collection); + const file = files.find((f) => f.id === fileID); + if (file) return file; + } + return undefined; }; diff --git a/src/library/content.ts b/src/library/content.ts index 9f2d6e6..906c0d6 100644 --- a/src/library/content.ts +++ b/src/library/content.ts @@ -581,14 +581,34 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI { fileID: number, opts?: ContentOptions, ): Promise<ContentResult> { - return this.get(fileID, "original", "on-demand", opts?.onProgress); + const file = this.fileOrThrow(fileID); + return this.get(file, "original", "on-demand", opts?.onProgress); } async thumbnail( fileID: number, opts?: ContentOptions, ): Promise<ContentResult> { - return this.get(fileID, "thumbnail", "on-demand", opts?.onProgress); + const file = this.fileOrThrow(fileID); + return this.get(file, "thumbnail", "on-demand", opts?.onProgress); + } + + // The original of `file`, fetched and cached as `original` does. `file` is + // used as given, so `getFile` need not know it. + async originalOf( + file: EnteFile, + opts?: ContentOptions, + ): Promise<ContentResult> { + return this.get(file, "original", "on-demand", opts?.onProgress); + } + + // The thumbnail of `file`, fetched and cached as `thumbnail` does. `file` + // is used as given, so `getFile` need not know it. + async thumbnailOf( + file: EnteFile, + opts?: ContentOptions, + ): Promise<ContentResult> { + return this.get(file, "thumbnail", "on-demand", opts?.onProgress); } // Put the original at the save path of `file` under the download directory @@ -615,7 +635,7 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI { destination: string, ): Promise<ContentResult> { const result = await this.acquire( - fileID, + this.fileOrThrow(fileID), "original", "on-demand", undefined, @@ -638,8 +658,7 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI { fileID: number, destination: string, ): Promise<{ path: string; videoPath?: string }> { - const file = this.getFile(fileID); - if (!file) throw new Error(`content cache: unknown file ${fileID}`); + const file = this.fileOrThrow(fileID); return this.pools.content.run( async () => { const fetched = await this.fetchInto( @@ -721,7 +740,8 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI { onProgress: ((event: EnsureEvent) => void) | undefined, ): Promise<EnsureResult> { try { - const result = await this.acquire(fileID, kind, priority, signal); + const file = this.fileOrThrow(fileID); + const result = await this.acquire(file, kind, priority, signal); const status = result.cached ? "skipped" : "done"; onProgress?.({ fileID, status, path: result.path }); return { fileID, path: result.path }; @@ -736,8 +756,16 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI { } } + // The record `getFile` gives for `fileID`. It throws for a file `getFile` + // does not know. + private fileOrThrow(fileID: number): EnteFile { + const file = this.getFile(fileID); + if (!file) throw new Error(`content cache: unknown file ${fileID}`); + return file; + } + private async get( - fileID: number, + file: EnteFile, kind: Kind, priority: Priority, onProgress: ((event: ContentEvent) => void) | undefined, @@ -745,7 +773,7 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI { const onByte: ProgressCallback | undefined = onProgress ? (bytesDone) => onProgress({ status: "downloading", bytesDone }) : undefined; - const result = await this.acquire(fileID, kind, priority, undefined, { + const result = await this.acquire(file, kind, priority, undefined, { onByte, }); onProgress?.( @@ -767,7 +795,7 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI { // photo's original is present only with its video, and is returned with // it. private async acquire( - fileID: number, + file: EnteFile, kind: Kind, priority: Priority, signal: AbortSignal | undefined, @@ -778,8 +806,7 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI { videoPath?: string; cached: boolean; }> { - const file = this.getFile(fileID); - if (!file) throw new Error(`content cache: unknown file ${fileID}`); + const fileID = file.id; const isLivePhoto = kind === "original" && file.metadata.fileType === "livePhoto"; diff --git a/src/library/index.ts b/src/library/index.ts index 5980fbc..23239e9 100644 --- a/src/library/index.ts +++ b/src/library/index.ts @@ -66,6 +66,8 @@ import { ContentCache, savePath, storedAtSavePath, + type ContentOptions, + type ContentResult, type ContentSource, type ThumbnailsAPI, type EnsureOptions, @@ -644,6 +646,36 @@ export class Library { return [...byID.values()]; } + // The original of `file`, fetched and cached as `Photo.original()` does. + // `file` is used as given, so the store need not hold it. On an empty + // cache, `quak get` passes a file that `freshFiles()` returned. + async originalOf( + file: EnteFile, + opts?: ContentOptions, + ): Promise<ContentResult> { + if (!this.cache) { + throw new Error( + "originalOf requires a library opened with a content cache", + ); + } + return this.cache.originalOf(file, opts); + } + + // The thumbnail of `file`, fetched and cached as `Photo.thumbnail()` does. + // `file` is used as given, so the store need not hold it. On an empty + // cache, `quak get-thumb` passes a file that `freshFiles()` returned. + async thumbnailOf( + file: EnteFile, + opts?: ContentOptions, + ): Promise<ContentResult> { + if (!this.cache) { + throw new Error( + "thumbnailOf requires a library opened with a content cache", + ); + } + return this.cache.thumbnailOf(file, opts); + } + // Back up every in-scope file to `opts.downloadDirectory`, or else the // library's, each original at its save path, with a durable failure // ledger (issue #51). Takes the lock in that directory first, unless the diff --git a/test/cli/commands.test.ts b/test/cli/commands.test.ts index 3822da7..78f99fa 100644 --- a/test/cli/commands.test.ts +++ b/test/cli/commands.test.ts @@ -194,9 +194,6 @@ const FETCHED_BOTH_ALBUMS = "Fetched the files of 1 of 2 collections\n" + "Fetched the files of 2 of 2 collections\n"; -// The same for an account with one album. -const FETCHED_ONE_ALBUM = "Fetched the files of 1 of 1 collections\n"; - // Collects everything written to it. class Output { text = ""; @@ -576,27 +573,27 @@ describe("get and get-thumb", () => { const out = join(root, "diagram.png"); expect(await getCommand(context(), "200", { out })).toBe(0); expect(readFileSync(out)).toEqual(Buffer.alloc(7, 200)); - expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS + `7 bytes -> ${out}\n`); + expect(stderr.text).toBe(`7 bytes -> ${out}\n`); }); it("get-thumb finds a file in any album without --collection", async () => { const out = join(root, "thumb.jpg"); expect(await getThumbCommand(context(), "200", { out })).toBe(0); expect(readFileSync(out)).toEqual(Buffer.alloc(3, 200)); - expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS + `3 bytes -> ${out}\n`); + expect(stderr.text).toBe(`3 bytes -> ${out}\n`); }); it("get exits 1 when no album has the file", async () => { const out = join(root, "x"); expect(await getCommand(context(), "999", { out })).toBe(1); - expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS + "File 999 not found\n"); + expect(stderr.text).toBe("File 999 not found\n"); expect(existsSync(out)).toBe(false); }); it("get-thumb exits 1 when no album has the file", async () => { const out = join(root, "x"); expect(await getThumbCommand(context(), "999", { out })).toBe(1); - expect(stderr.text).toBe(FETCHED_BOTH_ALBUMS + "File 999 not found\n"); + expect(stderr.text).toBe("File 999 not found\n"); expect(existsSync(out)).toBe(false); }); }); @@ -644,7 +641,7 @@ describe("names from an album name or a title over 255 bytes", () => { expect(Buffer.byteLength(name)).toBe(255); expect(code).toBe(0); expect(readdirSync(dir)).toEqual([name]); - expect(stderr.text).toBe(FETCHED_ONE_ALBUM + `7 bytes -> ${name}\n`); + expect(stderr.text).toBe(`7 bytes -> ${name}\n`); }); it("get-thumb cuts the title short so that thumb_ and it fit in 255 bytes, and exits 0", async () => { @@ -656,7 +653,7 @@ describe("names from an album name or a title over 255 bytes", () => { expect(Buffer.byteLength(name)).toBe(255); expect(code).toBe(0); expect(readdirSync(dir)).toEqual([name]); - expect(stderr.text).toBe(FETCHED_ONE_ALBUM + `3 bytes -> ${name}\n`); + expect(stderr.text).toBe(`3 bytes -> ${name}\n`); }); it("backup-metadata cuts the album name short so that its folder's name fits in 255 bytes, and exits 0", async () => { @@ -745,8 +742,7 @@ describe("a live photo", () => { Buffer.from(VIDEO), ); expect(stderr.text).toBe( - FETCHED_ONE_ALBUM + - `${IMAGE.length} bytes -> IMG_0300.heic\n` + + `${IMAGE.length} bytes -> IMG_0300.heic\n` + `${VIDEO.length} bytes -> IMG_0300.mov\n`, ); }); @@ -800,8 +796,7 @@ describe("a live photo", () => { expect(readFileSync(out)).toEqual(Buffer.from(IMAGE)); expect(readFileSync(video)).toEqual(Buffer.from(VIDEO)); expect(stderr.text).toBe( - FETCHED_ONE_ALBUM + - `${IMAGE.length} bytes -> ${out}\n${VIDEO.length} bytes -> ${video}\n`, + `${IMAGE.length} bytes -> ${out}\n${VIDEO.length} bytes -> ${video}\n`, ); }); @@ -815,8 +810,7 @@ describe("a live photo", () => { expect(existsSync(out)).toBe(false); expect(existsSync(join(root, "photo.mov"))).toBe(false); expect(stderr.text).toBe( - FETCHED_ONE_ALBUM + - `File 300 is a live photo, and its video would also be written to ${out}\n`, + `File 300 is a live photo, and its video would also be written to ${out}\n`, ); }); @@ -1575,13 +1569,14 @@ describe("an ID given on the command line", () => { ); }); -// An account with many albums (issues #210 and #217). On an empty cache -// `collections` and `files` with a collection ID used to wait for a refresh -// that fetched and decrypted the files of every album, one album after another. -// On such an account they seemed to hang. The account here has 40 albums of one -// photo each, encrypted as Ente's clients encrypt them, on the fake Ente server -// of `test/ente-server.ts`. The tests record the path and query of each -// request quak sends. The session is the one logging in to that server saves. +// An account with many albums (issues #210, #217 and #222). On an empty cache +// `collections`, `files` with a collection ID and `get` used to wait for a +// refresh that fetched and decrypted the files of every album, one album after +// another. On such an account they seemed to hang. The account here has 40 +// albums of one photo each, encrypted as Ente's clients encrypt them, on the +// fake Ente server of `test/ente-server.ts`. The tests record the path and query +// of each request quak sends. The session is the one logging in to that server +// saves. describe("an account with many albums", () => { const ALBUMS = 40; const PASSWORD = "many-albums"; @@ -1602,14 +1597,15 @@ describe("an account with many albums", () => { // The key of each album, by album ID. const albumKeys = new Map<number, Uint8Array>(); - // A photo in album `collectionID` as the server sends it. These tests - // fetch no original and no thumbnail. + // A photo in album `collectionID` as the server sends it, and its original + // encrypted as the server stores it. The original's bytes are the title. + // These tests fetch no thumbnail. const photoRow = (photo: { id: number; collectionID: number; title: string; updationTime: number; - }): Record<string, unknown> => { + }): { row: Record<string, unknown>; original: Uint8Array } => { const photoKey = sodium.crypto_secretstream_xchacha20poly1305_keygen(); const photoKeyBox = encryptSecretbox( photoKey, @@ -1626,7 +1622,11 @@ describe("an account with many albums", () => { ), photoKey, ); - return { + const original = encryptStreamBlob( + new TextEncoder().encode(photo.title), + photoKey, + ); + const row = { id: photo.id, collectionID: photo.collectionID, ownerID: USER_ID, @@ -1636,10 +1636,11 @@ describe("an account with many albums", () => { encryptedData: toBase64(metadata.ciphertext), decryptionHeader: toBase64(metadata.header), }, - file: { decryptionHeader: "" }, + file: { decryptionHeader: toBase64(original.header) }, thumbnail: { decryptionHeader: "" }, updationTime: photo.updationTime, }; + return { row, original: original.ciphertext }; }; // Album n is named `Album n` and holds one photo, n * 100, titled @@ -1649,6 +1650,7 @@ describe("an account with many albums", () => { const account = await buildAccount(PASSWORD); const collections: Record<string, unknown>[] = []; const filesByCollection: Record<number, Record<string, unknown>[]> = {}; + const fileCiphertexts: Record<number, Uint8Array> = {}; for (let id = 1; id <= ALBUMS; id++) { const albumKey = sodium.crypto_secretbox_keygen(); albumKeys.set(id, albumKey); @@ -1667,14 +1669,14 @@ describe("an account with many albums", () => { type: "album", updationTime: 1, }); - filesByCollection[id] = [ - photoRow({ - id: id * 100, - collectionID: id, - title: `photo-${id}.jpg`, - updationTime: 1, - }), - ]; + const { row, original } = photoRow({ + id: id * 100, + collectionID: id, + title: `photo-${id}.jpg`, + updationTime: 1, + }); + filesByCollection[id] = [row]; + fileCiphertexts[id * 100] = original; } const ente = buildMetaFetch({ verifier: account.verifier, @@ -1684,7 +1686,7 @@ describe("an account with many albums", () => { collections, filesByCollection, encryptedMLData: {}, - fileCiphertexts: {}, + fileCiphertexts, }); server = (async (input: RequestInfo | URL, init?: RequestInit) => { const url = @@ -1758,13 +1760,13 @@ describe("an account with many albums", () => { collectionID: 7, title: "photo-7.jpg", updationTime: 1, - }), + }).row, photoRow({ id: 701, collectionID: 7, title: "photo-701.jpg", updationTime: 1, - }), + }).row, ]; const secondPage = [ photoRow({ @@ -1772,7 +1774,7 @@ describe("an account with many albums", () => { collectionID: 7, title: "photo-7-edited.jpg", updationTime: 2, - }), + }).row, ]; const paged = (async (input: RequestInfo | URL, init?: RequestInit) => { const url = new URL(input instanceof Request ? input.url : input); @@ -1812,6 +1814,47 @@ describe("an account with many albums", () => { expect(stderr.text).toBe(progress); }); + // Photo 200 is in album 2, so `get` asks for no file of album 3 or later. + // The last request downloads the photo's original. + it("get on an empty cache asks for the albums' files only until one holds the photo", async () => { + const out = join(root, "photo.jpg"); + expect(await getCommand(accountContext(), "200", { out })).toBe(0); + + expect(readFileSync(out, "utf-8")).toBe("photo-2.jpg"); + expect(stderr.text).toBe(`11 bytes -> ${out}\n`); + expect(requests).toEqual([ + "/collections/v2?sinceTime=0", + "/collections/v2/diff?collectionID=1&sinceTime=0", + "/collections/v2/diff?collectionID=2&sinceTime=0", + "/?fileID=200", + ]); + expect(existsSync(join(root, "cache", "metadata.json"))).toBe(false); + }); + + // `files` first fills the cache, so `get` waits for a refresh. No album has + // changed since, so the refresh asks for no album's files. + it("get on a cache that holds the account asks for no unchanged album's files, also for a photo in no album", async () => { + const ctx = accountContext(); + expect(await filesCommand(ctx, undefined, {})).toBe(0); + requests.length = 0; + stderr.text = ""; + + const out = join(root, "photo.jpg"); + expect(await getCommand(ctx, "4000", { out })).toBe(0); + expect(readFileSync(out, "utf-8")).toBe("photo-40.jpg"); + expect(stderr.text).toBe(`12 bytes -> ${out}\n`); + expect(requests).toEqual([ + "/collections/v2?sinceTime=1", + "/?fileID=4000", + ]); + + requests.length = 0; + stderr.text = ""; + expect(await getCommand(ctx, "999", { out })).toBe(1); + expect(stderr.text).toBe("File 999 not found\n"); + expect(requests).toEqual(["/collections/v2?sinceTime=1"]); + }); + it("files counts on stderr each album whose files its refresh fetched", async () => { expect(await filesCommand(accountContext(), undefined, {})).toBe(0); diff --git a/test/cli/log.test.ts b/test/cli/log.test.ts index e30970e..1bc1fa6 100644 --- a/test/cli/log.test.ts +++ b/test/cli/log.test.ts @@ -234,13 +234,15 @@ describe("-q", () => { }); }); - // The refresh's progress is left out; the line saying why the command + // No album holds file 999, so fixing it fails. The refresh's progress and + // the list of failed files are left out. The line saying why the command // failed is the fatal one, and it is printed. it("prints only the line a failing command exits with", async () => { - expect(await quak("-q", "get", "999")).toEqual({ + const fix = ["helper", "fix-missing-thumbnails", "--file", "999"]; + expect(await quak("-q", ...fix)).toEqual({ code: 1, stdout: "", - stderr: "File 999 not found\n", + stderr: `quak: 1 file(s) failed; see the "Failed files:" list, or the JSON with --json\n`, }); }); diff --git a/test/cli/read.test.ts b/test/cli/read.test.ts index cb8dc1d..b60b068 100644 --- a/test/cli/read.test.ts +++ b/test/cli/read.test.ts @@ -11,7 +11,8 @@ * (`freshCollections()`, issue #210). `files` with a collection ID asks for * that list, then for that one collection's files (`freshFiles()`, issue * #217). Neither may wait for a refresh, which would fetch the files of every - * collection. + * collection. On an empty cache, `get` and `get-thumb` ask for that list, then + * for one collection's files at a time, until they find the file (issue #222). * * `collections` and `files` also list in the order the library gives them, not * the albums/photos projection's newest-first order. The fixtures are seeded in @@ -29,7 +30,6 @@ import { type FreshReadLibrary, } from "../../src/cli-read.js"; import { fileListRow } from "../../src/cli-output.js"; -import type { Photo } from "../../src/library/index.js"; import type { Collection, EnteFile } from "../../src/model/types.js"; const collection = (id: number, updationTime: number): Collection => ({ @@ -66,8 +66,7 @@ const file = ( // A library that reveals its cached records only after `fresh()` has been // awaited, and serves them in the enumeration order it was given. // `freshCollections()` and `freshFiles()` answer in that order without a -// refresh. `photos.byID` returns a stand-in `Photo` carrying just the fileID -// the helper passes through. +// refresh. class FakeLibrary implements FreshReadLibrary { freshCalls = 0; freshCollectionsCalls = 0; @@ -110,14 +109,6 @@ class FakeLibrary implements FreshReadLibrary { if (!this.refreshed) return undefined; return this.files.find((f) => f.id === fileID); } - - photos = { - byID: ({ fileID }: { fileID: number }): Photo | undefined => { - if (!this.refreshed) return undefined; - if (!this.files.some((f) => f.id === fileID)) return undefined; - return { fileID } as unknown as Photo; - }, - }; } describe("CLI read helpers (issue #36 amendment, issue #52)", () => { @@ -220,23 +211,46 @@ describe("CLI read helpers (issue #36 amendment, issue #52)", () => { expect(rows).toBeUndefined(); }); - it("freshFile refreshes first, then resolves the photo and its raw record", async () => { - const f = file(10, 1, 1_700_000_000_000_000); - const lib = new FakeLibrary([collection(1, 100)], [f]); - - const resolved = await freshFile(lib, 10); - expect(lib.freshCalls).toBe(1); - expect(resolved?.photo.fileID).toBe(10); - expect(resolved?.file.metadata.title).toBe("file-10.jpg"); - expect(resolved?.file.metadata.creationTime).toBe( - 1_700_000_000_000_000, + // Three collections, each holding one file. + const threeCollections = (): FakeLibrary => + new FakeLibrary( + [collection(1, 100), collection(2, 200), collection(3, 300)], + [ + file(10, 1, 1_700_000_000_000_000), + file(20, 2, 1_700_000_000_000_001), + file(30, 3, 1_700_000_000_000_002), + ], ); + + it("freshFile on an empty cache reads each collection's files in the server's order until one holds the file", async () => { + const lib = threeCollections(); + + const found = await freshFile(lib, 20); + expect(lib.freshCalls).toBe(0); + expect(lib.freshCollectionsCalls).toBe(1); + expect(lib.freshFilesCalls).toEqual([1, 2]); + expect(found?.metadata.title).toBe("file-20.jpg"); + expect(found?.metadata.creationTime).toBe(1_700_000_000_000_001); }); - it("freshFile returns undefined for an unknown file", async () => { - const lib = new FakeLibrary([collection(1, 100)], []); - const resolved = await freshFile(lib, 404); - expect(lib.freshCalls).toBe(1); - expect(resolved).toBeUndefined(); + // The stand-in's cache holds what the server holds once `fresh()` has run. + it("freshFile on a cache that holds the collections refreshes, then looks the file up in the cache", async () => { + const lib = threeCollections(); + await lib.fresh(); + + const found = await freshFile(lib, 30); + expect(lib.freshCalls).toBe(2); + expect(lib.freshCollectionsCalls).toBe(0); + expect(lib.freshFilesCalls).toEqual([]); + expect(found?.collectionID).toBe(3); + }); + + it("freshFile on an empty cache returns undefined for an unknown file, after reading every collection's files", async () => { + const lib = threeCollections(); + + const found = await freshFile(lib, 404); + expect(lib.freshCalls).toBe(0); + expect(lib.freshFilesCalls).toEqual([1, 2, 3]); + expect(found).toBeUndefined(); }); }); diff --git a/test/library/content-library.test.ts b/test/library/content-library.test.ts index 48a1094..92e0985 100644 --- a/test/library/content-library.test.ts +++ b/test/library/content-library.test.ts @@ -253,6 +253,12 @@ describe("Library content wiring", () => { await expect( lib.thumbnails.ensure({ fileIDs: [1], priority: "visible" }), ).rejects.toThrow(/content cache/i); + await expect(lib.originalOf(file(1, 1))).rejects.toThrow( + /content cache/i, + ); + await expect(lib.thumbnailOf(file(1, 1))).rejects.toThrow( + /content cache/i, + ); await lib.close(); }); }); diff --git a/test/library/content.test.ts b/test/library/content.test.ts index b461742..1809116 100644 --- a/test/library/content.test.ts +++ b/test/library/content.test.ts @@ -375,6 +375,25 @@ describe("ContentCache.original / thumbnail", () => { await cache.open(); await expect(cache.original(999)).rejects.toThrow(/unknown file/i); }); + + // `getFile` knows no file here, so the cache can fetch only the file it is + // given. + it("originalOf and thumbnailOf fetch and cache the file they are given", async () => { + const { cache, source } = buildCache({ files: [] }); + await cache.open(); + + const original = await cache.originalOf(file(5)); + const thumbnail = await cache.thumbnailOf(file(5)); + + expect(source.originalCalls).toEqual([5]); + expect(source.thumbnailCalls).toEqual([5]); + expect(original.path).toBe(join(cacheDir, "originals", "5.jpg")); + expect(thumbnail.path).toBe(join(cacheDir, "thumbnails", "5.jpg")); + expect(cache.pathsFor(5)).toEqual({ + originalPath: original.path, + thumbnailPath: thumbnail.path, + }); + }); }); describe("ContentCache.ensureThumbnails", () => { -- 2.54.0 From a1ea47f9a89755d0922f2ffbd395883c7f67f118 Mon Sep 17 00:00:00 2001 From: clawbot <35+clawbot@noreply.example.org> Date: Mon, 26 Oct 2026 11:25:10 +0100 Subject: [PATCH 4/7] quak helper fix-missing-thumbnails --file on an empty cache stops fetching collections once it finds every file (closes #223) `helper fix-missing-thumbnails --file` used to wait for a refresh, which on an empty cache fetched the files of every collection. It now finds the named files through the new `freshFilesByID` in `src/cli-read.ts`, by the rule `freshFile` follows. On an empty cache it reads the collection list, then one collection's files at a time, and stores none of the collections or file records it reads. It searches the most recently updated collection first. So a file in several albums is still named after the first of them that `Library.albums.list()` lists. `fixMissingThumbnails` now takes the records the command found, and fetches each original through `lib.originalOf()`. Model: opus-5-5 --- README.md | 20 +++--- TODO.md | 9 +++ src/cli-commands.ts | 27 ++++++-- src/cli-read.ts | 58 ++++++++++++++++ src/thumbnails.ts | 47 +++---------- test/cli/commands.test.ts | 107 +++++++++++++++++++++++++++-- test/cli/log.test.ts | 6 +- test/cli/read.test.ts | 70 +++++++++++++++++++ test/thumbnails/thumbnails.test.ts | 48 ++++++++++--- 9 files changed, 322 insertions(+), 70 deletions(-) diff --git a/README.md b/README.md index 97d8e56..926ba26 100644 --- a/README.md +++ b/README.md @@ -689,14 +689,15 @@ request, and neither fetches any collection's files nor changes the cache. `files` with a collection ID asks for the same list, then for that one collection's files, and does not change the cache either. On an empty cache, `get` and `get-thumb` ask for the same list, then for one collection's files at -a time, until they find the file. They store none of the collections or file -records they read. The original or thumbnail they fetch is kept in the cache. -Once the cache holds the account's collections, they wait for a refresh instead. -The others wait for a refresh, and so does `files` without a collection ID. With -an empty cache, as on a first run, that refresh fetches the files of every -collection, one collection after another, and stores nothing until it has them -all, which on a large account takes a while. Meanwhile the command logs a line -on stderr for each collection whose files it has fetched, such as +a time, until they find the file. `helper fix-missing-thumbnails --file` does +the same until it finds every file it names. They store none of the collections +or file records they read. The original or thumbnail they fetch is kept in the +cache. Once the cache holds the account's collections, they wait for a refresh +instead. The others wait for a refresh, and so does `files` without a collection +ID. With an empty cache, as on a first run, that refresh fetches the files of +every collection, one collection after another, and stores nothing until it has +them all, which on a large account takes a while. Meanwhile the command logs a +line on stderr for each collection whose files it has fetched, such as `Fetched the files of 3 of 120 collections`, with `--json` too; `backup` does the same while the refresh it starts with runs. stdout holds only the command's output. If that round-trip fails, the command prints the error on one line and @@ -1192,7 +1193,8 @@ It answers with every live file in that collection that decrypts, in the server's order. `collection` is a record that `freshCollections()` returned. The method fetches no other collection's files and stores nothing. `quak files` with a collection ID uses it, and so do `quak get` and `quak get-thumb` on an empty -cache. +cache. `quak helper fix-missing-thumbnails --file` uses it on an empty cache +too. An album or file in the server's answer that quak cannot decrypt, or whose metadata is not JSON, does not fail the refresh. The refresh leaves it out, an diff --git a/TODO.md b/TODO.md index 197ab3d..854ff37 100644 --- a/TODO.md +++ b/TODO.md @@ -26,6 +26,15 @@ declares one. # Completed Steps +- 2026-10-26: On an empty cache, `quak helper fix-missing-thumbnails --file` + stops fetching collections' files once it finds every file it names + (https://git.eeqj.de/sneak/quak/issues/223). It used to wait for a refresh, + which on an empty cache fetched the files of every collection. Now it reads + the collection list, then one collection's files at a time, the most recently + updated first. It stores none of the collections or file records it reads. It + fetches each original through `lib.originalOf()`, which keeps it in the cache. + Once the cache holds the account's collections, it still waits for a refresh. + - 2026-10-26: On an empty cache, `quak get` and `quak get-thumb` stop fetching collections' files once they find the file (https://git.eeqj.de/sneak/quak/issues/222). They used to wait for a refresh, diff --git a/src/cli-commands.ts b/src/cli-commands.ts index 1bb7d13..6089b9b 100644 --- a/src/cli-commands.ts +++ b/src/cli-commands.ts @@ -37,9 +37,16 @@ import { thumbnailName, } from "./cli-output.js"; import { type Log, logRefresh } from "./cli-log.js"; -import { freshCollections, freshFiles, freshFile } from "./cli-read.js"; +import { + cachedFilesByID, + freshCollections, + freshFiles, + freshFile, + freshFilesByID, +} from "./cli-read.js"; import { withExtension } from "./filename.js"; import { runMetadataBackup } from "./metadata-backup.js"; +import type { Collection, EnteFile } from "./model/types.js"; import { listMissingThumbnails, fixMissingThumbnails } from "./thumbnails.js"; export interface CliContext { @@ -133,8 +140,9 @@ const readLibraryClient = (client: Client): LibraryClient => ({ // not start downloading the whole account. No refresh starts as the library // opens or on a timer. Each command asks for the refresh it needs. // `collections` and `files` with a collection ID need none (issues #210 and -// #217). Nor do `get` and `get-thumb` on an empty cache (issue #222). While a -// refresh runs, each collection whose files it has fetched is logged, so a +// #217). Nor do `get` and `get-thumb` on an empty cache (issue #222). Nor does +// `helper fix-missing-thumbnails --file` on an empty cache (issue #223). While +// a refresh runs, each collection whose files it has fetched is logged, so a // large account shows progress rather than nothing. const openReadLibrary = (ctx: CliContext, client: Client): Promise<Library> => Library.open({ @@ -565,10 +573,13 @@ export const fixMissingThumbnailsCommand = async ( } const lib = await openReadLibrary(ctx, client); try { - // Refresh first so files added since the cache was written are found; - // a failed refresh throws. - await lib.fresh(); - if (fileIDs.length === 0) { + let found: Map<number, { file: EnteFile; collection: Collection }>; + if (fileIDs.length > 0) { + found = await freshFilesByID(lib, fileIDs); + } else { + // Refresh first so files added since the cache was written are + // checked; a failed refresh throws. + await lib.fresh(); ctx.log("info", "Scanning for missing thumbnails..."); const missing = await listMissingThumbnails( lib, @@ -581,12 +592,14 @@ export const fixMissingThumbnailsCommand = async ( return 0; } ctx.log("info", `Found ${fileIDs.length} file(s) to fix.`); + found = cachedFilesByID(lib, fileIDs); } const results = await fixMissingThumbnails( lib, client, fileIDs, + found, (msg, level) => ctx.log(level ?? "info", msg), ); diff --git a/src/cli-read.ts b/src/cli-read.ts index e5234fa..55b2d99 100644 --- a/src/cli-read.ts +++ b/src/cli-read.ts @@ -12,6 +12,8 @@ // list too, then that one collection's files through `Library.freshFiles()` // (issue #217). On an empty cache, `get` and `get-thumb` read that list, then // one collection's files at a time, until they find the file (issue #222). +// `helper fix-missing-thumbnails --file` does the same until it finds every +// file it names (issue #223). // // `collections` and `files` with a collection ID list in the server's order. // `files` without one lists in the library's own enumeration order @@ -90,3 +92,59 @@ export const freshFile = async ( } return undefined; }; + +// The most recently updated collection first, and the higher ID first on a +// tie. `Library.albums.list()` lists albums in this order. +const byNewestCollection = (a: Collection, b: Collection): number => + b.updationTime - a.updationTime || b.id - a.id; + +// Each file named by `fileIDs` that the cache holds, mapped to its record and +// the collection it was found in. Collections are searched in +// `byNewestCollection` order, and a file is found in the first that holds it. +// So `helper fix-missing-thumbnails` names a file after the same album as +// `helper list-missing-thumbnails`. It reads the cache alone, so the caller +// refreshes first. +export const cachedFilesByID = ( + lib: FreshReadLibrary, + fileIDs: number[], +): Map<number, { file: EnteFile; collection: Collection }> => { + const wanted = new Set(fileIDs); + const found = new Map<number, { file: EnteFile; collection: Collection }>(); + const collections = [...lib.listCollections()].sort(byNewestCollection); + for (const collection of collections) { + for (const file of lib.listFiles(collection.id)) { + if (wanted.has(file.id) && !found.has(file.id)) { + found.set(file.id, { file, collection }); + } + } + } + return found; +}; + +// Each file named by `fileIDs` as the server holds it now, mapped to its record +// and collection as `cachedFilesByID` maps it. When the cache holds the +// account's collections, it refreshes and looks the files up in the cache. On +// an empty cache, it reads the collection list, then one collection's files at +// a time, and stops once it has found every file. That search stores nothing. +export const freshFilesByID = async ( + lib: FreshReadLibrary, + fileIDs: number[], +): Promise<Map<number, { file: EnteFile; collection: Collection }>> => { + // Only a refresh stores collections, so a cache holding any holds them all. + if (lib.listCollections().length > 0) { + await lib.fresh(); + return cachedFilesByID(lib, fileIDs); + } + const wanted = new Set(fileIDs); + const found = new Map<number, { file: EnteFile; collection: Collection }>(); + const collections = await lib.freshCollections(); + for (const collection of [...collections].sort(byNewestCollection)) { + if (found.size === wanted.size) break; + for (const file of await lib.freshFiles(collection)) { + if (wanted.has(file.id) && !found.has(file.id)) { + found.set(file.id, { file, collection }); + } + } + } + return found; +}; diff --git a/src/thumbnails.ts b/src/thumbnails.ts index 0205296..471866b 100644 --- a/src/thumbnails.ts +++ b/src/thumbnails.ts @@ -5,7 +5,7 @@ import type { Client } from "./client.js"; import type { Library } from "./library/index.js"; import { ApiError } from "./api/client.js"; import { encryptBlob, toBase64 } from "./crypto/index.js"; -import type { EnteFile } from "./model/types.js"; +import type { Collection, EnteFile } from "./model/types.js"; // The server refuses a thumbnail larger than the one it already records for the // file (`thumbnail.size`, the encrypted size), so these encodings are tried @@ -226,16 +226,18 @@ const encryptThumbnailWithin = ( return undefined; }; -// Regenerate and upload a thumbnail for each requested file. Originals are read -// through the library's content cache (`photo.original()`); the generated -// thumbnail is JPEG-encoded, encrypted under the file's own key, and registered -// with the server — the encrypt-and-upload path is unchanged. Each file is +// Regenerate and upload a thumbnail for each requested file. `found` holds the +// record of each requested file that a collection holds, and the collection it +// was found in. Each original is read through the library's content cache, by +// `lib.originalOf(file)`. The new thumbnail is a JPEG, encrypted under the +// file's own key. It is uploaded, then registered with the server. Each file is // resolved to one outcome (fixed / skipped / failed) and a failure on one file // never stops the others. export const fixMissingThumbnails = async ( lib: Library, client: Client, fileIDs: number[], + found: Map<number, { file: EnteFile; collection: Collection }>, onProgress?: ProgressCallback, ): Promise<ThumbnailFixResult[]> => { const log = onProgress ?? (() => {}); @@ -243,31 +245,8 @@ export const fixMissingThumbnails = async ( const api = client.getApiClient(); const { userID } = client.whoami(); - // Resolve each requested fileID to its file record and owning album by - // enumerating the library, each file taken from the first album that holds - // it. The raw `EnteFile` carries the per-file key the thumbnail is - // encrypted under, which the projected records deliberately do not. - const wanted = new Set(fileIDs); - const fileMap = new Map< - number, - { file: EnteFile; collectionName: string } - >(); - for (const album of lib.albums.list()) { - for (const photo of album.photos.list()) { - if (!wanted.has(photo.fileID) || fileMap.has(photo.fileID)) - continue; - const file = lib.getFile(album.collectionID, photo.fileID); - if (file) { - fileMap.set(photo.fileID, { - file, - collectionName: album.name, - }); - } - } - } - for (const fileID of fileIDs) { - const entry = fileMap.get(fileID); + const entry = found.get(fileID); if (!entry) { results.push({ fileID, @@ -279,7 +258,8 @@ export const fixMissingThumbnails = async ( continue; } - const { file, collectionName } = entry; + const { file } = entry; + const collectionName = entry.collection.name; const title = file.metadata.title; const skipReason = reasonToSkip(file, userID); @@ -297,15 +277,10 @@ export const fixMissingThumbnails = async ( const maxSize = file.thumbnail.size!; try { - const photo = lib.photos.byID({ fileID }); - if (!photo) { - throw new Error("file not present in the library cache"); - } - log( `[${collectionName}] Downloading ${title} for thumbnail generation...`, ); - const { path } = await photo.original(); + const { path } = await lib.originalOf(file); const fileBytes = new Uint8Array(readFileSync(path)); if (!isJpeg(fileBytes)) { diff --git a/test/cli/commands.test.ts b/test/cli/commands.test.ts index 78f99fa..2bd5e53 100644 --- a/test/cli/commands.test.ts +++ b/test/cli/commands.test.ts @@ -1353,6 +1353,23 @@ describe("helper fix-missing-thumbnails", () => { ).toBe(1); expect(stderr.text).toBe(last); }); + + // beach.jpg (100) is in Vacation (1) and Work (2), both updated at the same + // time. The fixer names a file after the first album holding it in the + // order of `Library.albums.list()`. On a tie, the album with the higher ID + // comes first, so beach.jpg is named after Work. + it("names a photo in two albums after the same album on an empty cache as on a cache that holds the account", async () => { + const ctx = context(fakeClient({ beachInWork: true })); + const opts = { file: ["100"], json: true }; + const inWork = [{ fileID: 100, collection: "Work", status: "skipped" }]; + expect(await fixMissingThumbnailsCommand(ctx, opts)).toBe(0); + expect(JSON.parse(stdout.text)).toMatchObject(inWork); + + expect(await filesCommand(ctx, undefined, {})).toBe(0); + stdout.text = ""; + expect(await fixMissingThumbnailsCommand(ctx, opts)).toBe(0); + expect(JSON.parse(stdout.text)).toMatchObject(inWork); + }); }); describe("backup-metadata --exif", () => { @@ -1572,11 +1589,11 @@ describe("an ID given on the command line", () => { // An account with many albums (issues #210, #217 and #222). On an empty cache // `collections`, `files` with a collection ID and `get` used to wait for a // refresh that fetched and decrypted the files of every album, one album after -// another. On such an account they seemed to hang. The account here has 40 -// albums of one photo each, encrypted as Ente's clients encrypt them, on the -// fake Ente server of `test/ente-server.ts`. The tests record the path and query -// of each request quak sends. The session is the one logging in to that server -// saves. +// another. So did `helper fix-missing-thumbnails --file` (issue #223). On such +// an account they seemed to hang. The account here has 40 albums of one photo +// each, encrypted as Ente's clients encrypt them, on the fake Ente server of +// `test/ente-server.ts`. The tests record the path and query of each request +// quak sends. The session is the one logging in to that server saves. describe("an account with many albums", () => { const ALBUMS = 40; const PASSWORD = "many-albums"; @@ -1599,7 +1616,9 @@ describe("an account with many albums", () => { // A photo in album `collectionID` as the server sends it, and its original // encrypted as the server stores it. The original's bytes are the title. - // These tests fetch no thumbnail. + // These tests fetch no thumbnail. The server records a thumbnail size, so + // `helper fix-missing-thumbnails` downloads the original before it skips + // the photo as not a JPEG. const photoRow = (photo: { id: number; collectionID: number; @@ -1638,6 +1657,7 @@ describe("an account with many albums", () => { }, file: { decryptionHeader: toBase64(original.header) }, thumbnail: { decryptionHeader: "" }, + info: { thumbSize: 1000 }, updationTime: photo.updationTime, }; return { row, original: original.ciphertext }; @@ -1855,6 +1875,81 @@ describe("an account with many albums", () => { expect(requests).toEqual(["/collections/v2?sinceTime=1"]); }); + // The helper searches the most recently updated album first. All 40 were + // updated at the same time, and on a tie the higher ID comes first. Photo + // 3900 is in album 39, so the helper asks for no file of album 38 or + // earlier. The last request downloads the photo's original. + it("fix-missing-thumbnails --file on an empty cache asks for the albums' files only until one holds the photo", async () => { + const opts = { file: ["3900"], json: true }; + expect(await fixMissingThumbnailsCommand(accountContext(), opts)).toBe( + 0, + ); + + const reason = + "unsupported image format (only baseline JPEG can be regenerated)"; + expect(JSON.parse(stdout.text)).toEqual([ + { + fileID: 3900, + title: "photo-39.jpg", + collection: "Album 39", + status: "skipped", + reason, + }, + ]); + expect(stderr.text).toBe( + "[Album 39] Downloading photo-39.jpg for thumbnail generation...\n" + + `[Album 39] Skipping photo-39.jpg: ${reason}\n`, + ); + expect(requests).toEqual([ + "/collections/v2?sinceTime=0", + "/collections/v2/diff?collectionID=40&sinceTime=0", + "/collections/v2/diff?collectionID=39&sinceTime=0", + "/?fileID=3900", + ]); + expect(existsSync(join(root, "cache", "metadata.json"))).toBe(false); + }); + + // `files` first fills the cache, so the helper waits for a refresh. No + // album has changed since, so the refresh asks for no album's files. + it("fix-missing-thumbnails --file on a cache that holds the account asks for no unchanged album's files, also for a photo in no album", async () => { + const ctx = accountContext(); + expect(await filesCommand(ctx, undefined, {})).toBe(0); + requests.length = 0; + stdout.text = ""; + stderr.text = ""; + + const opts = { file: ["3900"], json: true }; + expect(await fixMissingThumbnailsCommand(ctx, opts)).toBe(0); + const reason = + "unsupported image format (only baseline JPEG can be regenerated)"; + expect(JSON.parse(stdout.text)).toEqual([ + { + fileID: 3900, + title: "photo-39.jpg", + collection: "Album 39", + status: "skipped", + reason, + }, + ]); + expect(stderr.text).toBe( + "[Album 39] Downloading photo-39.jpg for thumbnail generation...\n" + + `[Album 39] Skipping photo-39.jpg: ${reason}\n`, + ); + expect(requests).toEqual([ + "/collections/v2?sinceTime=1", + "/?fileID=3900", + ]); + + requests.length = 0; + stdout.text = ""; + const notFound = { file: ["999"], json: true }; + expect(await fixMissingThumbnailsCommand(ctx, notFound)).toBe(1); + expect(JSON.parse(stdout.text)).toMatchObject([ + { fileID: 999, reason: "file not found in any collection" }, + ]); + expect(requests).toEqual(["/collections/v2?sinceTime=1"]); + }); + it("files counts on stderr each album whose files its refresh fetched", async () => { expect(await filesCommand(accountContext(), undefined, {})).toBe(0); diff --git a/test/cli/log.test.ts b/test/cli/log.test.ts index 1bc1fa6..d89d929 100644 --- a/test/cli/log.test.ts +++ b/test/cli/log.test.ts @@ -234,9 +234,9 @@ describe("-q", () => { }); }); - // No album holds file 999, so fixing it fails. The refresh's progress and - // the list of failed files are left out. The line saying why the command - // failed is the fatal one, and it is printed. + // No album holds file 999, so fixing it fails. The summary from + // `--- Done ---` to the end of the `Failed files:` list is left out. The + // line saying why the command failed is the fatal one, and it is printed. it("prints only the line a failing command exits with", async () => { const fix = ["helper", "fix-missing-thumbnails", "--file", "999"]; expect(await quak("-q", ...fix)).toEqual({ diff --git a/test/cli/read.test.ts b/test/cli/read.test.ts index b60b068..d292041 100644 --- a/test/cli/read.test.ts +++ b/test/cli/read.test.ts @@ -13,6 +13,9 @@ * #217). Neither may wait for a refresh, which would fetch the files of every * collection. On an empty cache, `get` and `get-thumb` ask for that list, then * for one collection's files at a time, until they find the file (issue #222). + * `helper fix-missing-thumbnails --file` does the same until it finds every + * file it names, reading the most recently updated collection first (issue + * #223). * * `collections` and `files` also list in the order the library gives them, not * the albums/photos projection's newest-first order. The fixtures are seeded in @@ -27,6 +30,7 @@ import { freshCollections, freshFiles, freshFile, + freshFilesByID, type FreshReadLibrary, } from "../../src/cli-read.js"; import { fileListRow } from "../../src/cli-output.js"; @@ -253,4 +257,70 @@ describe("CLI read helpers (issue #36 amendment, issue #52)", () => { expect(lib.freshFilesCalls).toEqual([1, 2, 3]); expect(found).toBeUndefined(); }); + + // The server lists collections 1, 2 and 3, and collection 3 was updated + // most recently. File 11 is in collections 1 and 2. + const fileInTwoCollections = (): FakeLibrary => + new FakeLibrary( + [collection(1, 100), collection(2, 200), collection(3, 300)], + [ + file(10, 1, 1_700_000_000_000_000), + file(11, 1, 1_700_000_000_000_001), + file(11, 2, 1_700_000_000_000_001), + file(20, 2, 1_700_000_000_000_002), + file(30, 3, 1_700_000_000_000_003), + ], + ); + + it("freshFilesByID on an empty cache reads the most recently updated collection's files first, until it has found every file", async () => { + const lib = fileInTwoCollections(); + + const found = await freshFilesByID(lib, [11, 20]); + expect(lib.freshCalls).toBe(0); + expect(lib.freshCollectionsCalls).toBe(1); + expect(lib.freshFilesCalls).toEqual([3, 2]); + expect(found.get(11)?.collection.id).toBe(2); + expect(found.get(11)?.file.collectionID).toBe(2); + expect(found.get(20)?.collection.id).toBe(2); + }); + + // Collection 1 was updated last, though collections 2 and 3 have higher + // IDs. File 11 is in collections 1 and 2. + it("freshFilesByID on an empty cache reads the most recently updated collection first, though its ID is the lowest", async () => { + const lib = new FakeLibrary( + [collection(1, 400), collection(2, 200), collection(3, 300)], + [ + file(11, 1, 1_700_000_000_000_001), + file(11, 2, 1_700_000_000_000_001), + ], + ); + + const found = await freshFilesByID(lib, [11]); + expect(lib.freshFilesCalls).toEqual([1]); + expect(found.get(11)?.collection.id).toBe(1); + expect(found.get(11)?.file.collectionID).toBe(1); + }); + + // The stand-in's cache holds what the server holds once `fresh()` has run. + it("freshFilesByID on a cache that holds the collections refreshes, then finds each file in the most recently updated collection that holds it", async () => { + const lib = fileInTwoCollections(); + await lib.fresh(); + + const found = await freshFilesByID(lib, [11, 404]); + expect(lib.freshCalls).toBe(2); + expect(lib.freshCollectionsCalls).toBe(0); + expect(lib.freshFilesCalls).toEqual([]); + expect(found.get(11)?.collection.id).toBe(2); + expect(found.get(11)?.file.collectionID).toBe(2); + expect(found.has(404)).toBe(false); + }); + + it("freshFilesByID on an empty cache reads every collection's files when a file is in none of them", async () => { + const lib = fileInTwoCollections(); + + const found = await freshFilesByID(lib, [10, 404]); + expect(lib.freshFilesCalls).toEqual([3, 2, 1]); + expect(found.get(10)?.collection.id).toBe(1); + expect(found.has(404)).toBe(false); + }); }); diff --git a/test/thumbnails/thumbnails.test.ts b/test/thumbnails/thumbnails.test.ts index bf2be5e..9db0e48 100644 --- a/test/thumbnails/thumbnails.test.ts +++ b/test/thumbnails/thumbnails.test.ts @@ -6,10 +6,12 @@ * working thumbnails, others return 404 or empty bodies. The tests * verify that the detection and repair logic handles each case correctly. * - * As of issue #52 both helpers take an open `Library` for enumeration and the - * `Client` for the API operations that stay unchanged (the thumbnail existence - * check, and the encrypt-and-upload path). `fixMissingThumbnails` reads each - * original through the library's content cache (`photo.original()`). + * `listMissingThumbnails` takes an open `Library` to enumerate the files. Both + * helpers take the `Client` for the API operations: the thumbnail existence + * check, and the encrypt-and-upload path. `fixMissingThumbnails` takes the + * record of each file, which these tests look up with `cachedFilesByID`. It + * reads each original through the library's content cache, by + * `lib.originalOf(file)`. * * `fixMissingThumbnails` is the most complex function in quak: it * downloads the original file, generates a JPEG thumbnail with jpeg-js, @@ -41,6 +43,7 @@ import { import { SRP, SrpServer } from "fast-srp-hap"; import { Client } from "../../src/client.js"; import { Library } from "../../src/library/index.js"; +import { cachedFilesByID } from "../../src/cli-read.js"; import { listMissingThumbnails, fixMissingThumbnails, @@ -656,7 +659,12 @@ describe("fixMissingThumbnails", () => { const client = await login(buildThumbFetch(fixMock)); const lib = await openLib(client); - const results = await fixMissingThumbnails(lib, client, [101]); + const results = await fixMissingThumbnails( + lib, + client, + [101], + cachedFilesByID(lib, [101]), + ); lib.close(); expect(results.length).toBe(1); @@ -693,7 +701,12 @@ describe("fixMissingThumbnails", () => { const client = await login(buildThumbFetch(fixMock)); const lib = await openLib(client); - const results = await fixMissingThumbnails(lib, client, [999]); + const results = await fixMissingThumbnails( + lib, + client, + [999], + cachedFilesByID(lib, [999]), + ); lib.close(); expect(results.length).toBe(1); @@ -715,6 +728,7 @@ describe("fixMissingThumbnails", () => { lib, client, [101, 102], + cachedFilesByID(lib, [101, 102]), (msg, level) => { if (level === "error") errors.push(msg); }, @@ -745,6 +759,7 @@ describe("fixMissingThumbnails", () => { lib, client, [101, 103, 104], + cachedFilesByID(lib, [101, 103, 104]), ); lib.close(); @@ -779,7 +794,12 @@ describe("fixMissingThumbnails", () => { const client = await login(counted.fetch); const lib = await openLib(client); - const results = await fixMissingThumbnails(lib, client, [101]); + const results = await fixMissingThumbnails( + lib, + client, + [101], + cachedFilesByID(lib, [101]), + ); lib.close(); expect(results[0]!.status).toBe("skipped"); @@ -800,7 +820,12 @@ describe("fixMissingThumbnails", () => { const client = await login(counted.fetch); const lib = await openLib(client); - const results = await fixMissingThumbnails(lib, client, [101]); + const results = await fixMissingThumbnails( + lib, + client, + [101], + cachedFilesByID(lib, [101]), + ); lib.close(); expect(results[0]!.status).toBe("skipped"); @@ -841,7 +866,12 @@ describe("fixMissingThumbnails", () => { const client = await login(buildThumbFetch(fixMock)); const lib = await openLib(client); - const results = await fixMissingThumbnails(lib, client, [101]); + const results = await fixMissingThumbnails( + lib, + client, + [101], + cachedFilesByID(lib, [101]), + ); lib.close(); expect(results[0]!.status).toBe("fixed"); -- 2.54.0 From b5fcbf440022ea8ad77d701deaee5b392d2f27ac Mon Sep 17 00:00:00 2001 From: clawbot <35+clawbot@noreply.example.org> Date: Mon, 26 Oct 2026 17:48:25 +0100 Subject: [PATCH 5/7] README.md rewritten sentence by sentence (closes #219) The old text served as the list of what to cover. The new text leads with what quak is, what it is for and how to use it. Every command, option, exit code and file layout follows in reference sections, then the library reference, then design, entrypoints and development. Every statement was checked against the code on next; the pull request lists each disagreement. Sentences stay under about 30 words, tables are lists, and no issue links or project history remain in text for users. Two code comments now name the new README sections, and the one in `src/cli-log.ts` matches "What quak prints" on stderr, stdout and exit status; a third no longer cites the README. Model: fable-5-1 --- README.md | 2942 ++++++++++++++++------------ src/cli-log.ts | 17 +- test/cli/backup-end-to-end.test.ts | 2 +- test/cli/log.test.ts | 2 +- 4 files changed, 1721 insertions(+), 1242 deletions(-) diff --git a/README.md b/README.md index 926ba26..4c51a1e 100644 --- a/README.md +++ b/README.md @@ -1,28 +1,37 @@ # quak -quak is a WTFPL-licensed TypeScript client library and CLI by -[@sneak](https://sneak.berlin) for the [Ente](https://ente.io) end-to-end -encrypted photo hosting service. It logs in, enumerates collections and files, -and downloads individual images while decrypting them on the way to disk. +quak is a WTFPL-licensed TypeScript library and command-line tool by +[@sneak](https://sneak.berlin) for reading and backing up an +[Ente](https://ente.io) photo account. -quak also includes a resilient backup command that downloads every file in the -account into a deduplicated local directory tree, skipping files that already -exist on disk and continuing past individual download failures instead of -crashing. For each file it persists the basic metadata fields quak keeps (title, -file type, creation and modification time, latitude, longitude, content hash), -its update time, and the private and public magic metadata in full. It keeps any -ML data Ente has for the file, except the CLIP embedding, and writes the file's -keywords in place of the embedding. For an image (for a live photo, its image), -it keeps its original's EXIF and XMP, with its dimensions for a JPEG only. For a -video it keeps none of these three. It runs unattended from cron (see "Running -the backup from cron"). A helper subcommand can detect and regenerate missing -thumbnails, encrypting and uploading them back to the server. +Ente is a photo service that encrypts photos end to end. The server never sees a +photo, its name or its date unencrypted. Only a client that holds the account's +keys can read them. quak is such a client, built for scripts and for jobs that +run unattended. It logs in with the account's email and password, derives the +keys, and keeps a decrypted copy of the account's album and file lists in a +local cache. + +With the command line you can: + +- list the account's albums and the files in them; +- download and decrypt one file, or its thumbnail; +- back up every file in the account into a directory tree that a photo viewer + can open, and keep that backup current from cron; +- write the metadata quak keeps about the account, its albums and its files as + JSON; +- find files whose thumbnails are missing on the server, and repair them. + +The library underneath lets a program of your own read and back up the account. +It keeps the account's album and file lists on disk and answers from that copy +without asking the server. + +A backup run never prompts and never stops on one bad file. It skips files that +are already on disk, records each failure in a file, and tries the failed files +again on the next run. Every download is decrypted on its way to disk. ## Getting Started -The commands below need git, node 22 or later, and yarn. `make install`, which -builds the single binary that "Running the backup from cron" uses, also needs -[nix](https://nixos.org/download). +The commands below need `git`, Node 22 or later and `yarn`. ```bash git clone https://git.eeqj.de/sneak/quak.git @@ -30,43 +39,64 @@ cd quak yarn install yarn build -# Log in (prompts for email, password, and OTP/TOTP if required). +# Log in. quak asks for the email and the password, and for a code when +# the account needs one. yarn quak login -# List the user's collections (albums). +# List the account's collections. Ente calls an album a collection. yarn quak collections # List every file in the account. yarn quak files -# List the files in a collection. +# List the files in one collection. yarn quak files 12345 -# Download and decrypt a single file. +# Download and decrypt one file. yarn quak get 67890 --out ./photo.jpg # Back up every file in the account. yarn quak backup ./my-backup ``` -For library use, the primary surface is the cache-backed `Library`: +`yarn quak` runs the command line that `yarn build` wrote to `dist/`. To run +quak from cron, build it as one binary instead; see "Running the backup from +cron". + +### Using the library + +The library keeps a local copy of the account's album and file lists, the cache. +It brings the cache up to date with the server in rounds called refreshes. + +quak is not published to a package registry: `package.json` marks it `private`, +and `dist/` is not committed. Clone and build it as above, then add the clone to +your own program's `package.json` by its path, for example +`yarn add file:../quak`. The package's `main`, `types` and `exports` fields +point into `dist/`, so the clone must be built first. The package is an ES +module only, so a CommonJS program cannot `require` it. ```ts import { Client, Library } from "quak"; -// Log in once; the client satisfies the library's client interface. +// Log in once. The client holds the account's keys in memory. const client = await Client.login({ email: "you@example.com", password: "your-password", }); -// Open a cache-backed library. On an empty cache this awaits one server -// refresh; on an existing cache it returns immediately and refreshes in the -// background. Later refreshes start `refreshIntervalSeconds` (default 3) after -// the previous one ends. -const lib = await Library.open({ client }); +// Open the library. On an empty cache this waits for one refresh. On an +// existing cache it returns at once and refreshes in the background. +const lib = await Library.open({ + client, + // Without these two, open() also starts downloading every thumbnail in + // the account, and the originals of the favorites album and the latest + // week, into the cache. It fetches machine-learning data after each + // refresh either way. + precacheThumbnails: false, + precacheOriginals: false, +}); -// Default reads answer synchronously from the local cache — no network. +// Default reads answer from the cache and never touch the network. for (const album of lib.albums.list()) { console.log(album.collectionID, album.name); for (const photo of album.photos.list()) { @@ -74,11 +104,11 @@ for (const album of lib.albums.list()) { } } -// Fresh reads await a server round-trip, joining one already running. +// A fresh read waits for a refresh first. const { albums } = await lib.fresh(); console.log(`${albums.list().length} albums as of now`); -// Fetch (and cache) one photo's full-resolution bytes. +// Fetch one photo's original. The bytes land in the cache. const photo = lib.photos.byID({ fileID: 12345 }); if (photo) { const { path } = await photo.original(); @@ -88,46 +118,59 @@ if (photo) { await lib.close(); ``` -The lower-level `Client` (login, session serialization, and the raw -enumeration/download calls) is exported too and documented under Design below. +`Library.open({ client })` with no other options does more than the example asks +for. It refreshes every 3 seconds until `close()`. It also downloads, in the +background, every thumbnail in the account and the originals of the favorites +album and of the latest week (see "Opening a library"). The example turns those +two downloads off. After each refresh that succeeds, the library also fetches +each file's machine-learning data into the cache. That is the faces and the +other data Ente's apps compute for a photo (see "The JSON beside each +original"). No option in `LibraryOptions` turns this off. A `client` without a +`fetchMLData` method fetches none, which is how quak's own read commands open +their library. + +The `Client` under the library handles login, saved sessions and the raw +requests. The "Library reference" below covers both. ## Running the backup from cron -`quak backup` never prompts, so cron can run it. `make install` builds quak as a -single binary and copies it to `~/bin/quak`. Besides what Getting Started needs, -it needs [nix](https://nixos.org/download), even where bun is already installed: -`make install` runs `make build-bin`, which bundles the built `dist/` with bun -run through `nix-shell -p bun`. Log in once with `~/bin/quak`, as the user the -cron job will run as; the session is saved in that user's data directory (see -"Session handling"): +`quak backup` never prompts, so cron can run it. `make install` builds quak as +one binary and copies it to `~/bin/quak`. Besides what Getting Started needs, +`make install` needs `make` and [nix](https://nixos.org/download). It uses bun, +a JavaScript runtime, to bundle the built `dist/` into one executable. It runs +bun through `nix-shell -p bun`. + +Log in once with `~/bin/quak`, as the user the cron job will run as: ```bash ~/bin/quak login ``` +The session is saved in that user's data directory (see "The session file"). Then add the backup to that user's crontab with `crontab -e`. These two lines -back up the account to `~/photos-backup` at 03:30 every night, with `--verify` -on Sundays, and append all output to `~/quak-backup.log`: +back up the account to `~/photos-backup` at 03:30 every night and append all +output to `~/quak-backup.log`. Sunday's run adds `--verify`: ``` 30 3 * * 1-6 $HOME/bin/quak backup $HOME/photos-backup >> $HOME/quak-backup.log 2>&1 30 3 * * 0 $HOME/bin/quak backup --verify $HOME/photos-backup >> $HOME/quak-backup.log 2>&1 ``` -A run with `--verify` does all a plain run does, and also checks each original -already in the backup against the content hash Ente records for it, and replaces -any that do not match (see "Backup layout"). Sunday's run is the `--verify` one, -not a second job that night, because a backup that starts while another backup -of the same directory is running exits 2 without backing anything up. +A run with `--verify` does all a plain run does. It also hashes each original +already in the backup and compares the result with the content hash. The content +hash is the hash of the file's bytes that Ente recorded at upload. The run +fetches a new copy of each original that does not match, and replaces the +original once the copy matches. Sunday's run is the `--verify` run, rather than +a second job that night, because two backups of one directory cannot run at +once. The second one exits 2 without doing anything. Cron runs the job with a short `PATH`, usually `/usr/bin:/bin`, so the crontab -names quak by its full path. To find the saved session, quak needs the same -`HOME` as when you logged in, which cron sets from the password file, and on -Linux the same `XDG_DATA_HOME`: quak looks for the session in -`$XDG_DATA_HOME/quak`, or in `~/.local/share/quak` when that is not set. Cron -does not set `XDG_DATA_HOME`, so if your login shell does, set it at the top of -the crontab too. Cron does not expand variables in such a line, so give the full -path: +names quak by its full path. quak finds the saved session through `HOME`, which +cron sets from `/etc/passwd`. On Linux it also reads `XDG_DATA_HOME`: the +session is in `$XDG_DATA_HOME/quak`, or in `~/.local/share/quak` when that +variable is not set. Cron does not set `XDG_DATA_HOME`. If your login shell sets +it, set it at the top of the crontab too, as a full path, because cron does not +expand variables there: ``` XDG_DATA_HOME=/home/you/.data @@ -136,747 +179,555 @@ XDG_DATA_HOME=/home/you/.data On macOS the session is in `~/Library/Application Support/quak`, and only `HOME` matters. +A backup that ends with failures exits 1, and its last line says how many files +and albums failed. The next run fetches each missing original again and rebuilds +the album links. The exit codes are listed under "Exit codes", and the backup's +files under "The backup directory". + +## Command reference + +Every command has the form `quak [options] <command> [arguments]`. The options +below belong to quak itself, and go before or after the command's name. + +### Global options + +- `--cache-dir <path>`: where the local cache lives. Without it, each account + gets its own directory under the per-user cache directory (see "The cache"). +- `-v`, `--verbose`: log more on stderr. `-v` adds debug lines, and `-vv` adds + trace lines too (see "What quak prints"). +- `-q`, `--quiet`: print nothing on stderr but the reason a failing command + exits. `-q` with `-v` is refused with one line, + `error: option '-q, --quiet' cannot be used with option '-v, --verbose'`, and + exit status 1. +- `-V`, `--version`: print the version and exit (see "Version"). +- `-h`, `--help`: print the usage and exit. + +### The commands + +``` +quak login +quak whoami +quak logout +quak collections [--json] +quak files [collectionID] [--json] +quak get <fileID> [--out <path>] +quak get-thumb <fileID> [--out <path>] +quak backup <dir> [--json] [--verify] +quak backup-metadata <dir> [--exif] +quak helper list-missing-thumbnails [--json] +quak helper fix-missing-thumbnails [--file <id>...] [--json] +``` + +Every command but `login` and `logout` needs a saved session. Every command but +`login`, `whoami` and `logout` opens the library. How each one gets current data +from the server is described under "How a command gets current data". + +#### `quak login` + +Logs in to an account and saves the session. It reads the email from +`QUAK_EMAIL` and the password from `QUAK_PASSWORD`. For each one that is not +set, it asks on the terminal. The password shows as `*` characters while it is +typed. + +When the account has a second factor, quak asks for the code. It asks +`TOTP code: ` for a time-based code from an authenticator app. An account whose +only second factor is a passkey cannot log in with quak. When the account has +email verification on, quak asks `Email verification code: ` for the code sent +by email. The password is then used only to unlock the keys. After the email +code quak expects the login to complete. When the server asks for a second +factor at that point, login fails. + +On success it logs `Logged in as <email> (user <id>)` and +`Session saved to <path>`. On failure it logs `Login failed: <reason>`, and the +exit status is 1. `login` is the only command that prompts, and its prompts go +to stdout. + +#### `quak whoami` + +Prints the saved session's account on stdout as one line of JSON: +`{"email":"you@example.com","userID":12345}`. It reads the session file only and +sends no request, so it succeeds even when the server no longer accepts the +session. + +#### `quak logout` + +Ends the session on the server and deletes the session file. Ending it on the +server makes the saved token stop working everywhere, even in a copy of the +file. With no session file, the command logs `No session found.` and exits 0. + +When the server call fails, or the session file is corrupt, the file is still +deleted. The command then logs +`Could not end the session on the server: <reason>` and exits 1. + +`logout` does not delete the cache. It warns that the cache directory still +holds decrypted data. It knows the directory from `--cache-dir` or from a +session file it can read. Delete that directory yourself if you want the data +gone. + +#### `quak collections [--json]` + +Lists every collection in the account, in the server's order, after one request +to the server. + +Without `--json`, each line holds the collection's ID, its type and its name, +separated by tabs. A collection another account owns gets ` (shared)` after its +name. A collection's type is one of `album`, `folder`, `favorites`, +`uncategorized` and `unknown`. + +With `--json`, the output is a JSON array with one object per collection. Each +object holds `id`, `name`, `type`, `ownerID`, `isShared` and `updationTime`. +Update times are in microseconds, as Ente records them. + +#### `quak files [collectionID] [--json]` + +With a collection ID, lists the files in that collection, in the server's order. +Without one, lists every file in the account once, collection by collection. A +file that is in several collections is listed under the first collection that +holds it. + +Without `--json`, each line holds the file's ID, its type and its title, +separated by tabs. A file's type is one of `image`, `video`, `livePhoto` and +`unknown`. With `--json`, each object holds `id`, `title`, `fileType`, +`creationTime` in microseconds and `collectionID`. + +A collection ID that does not exist gives `Collection <id> not found` and exit +status 1. + +#### `quak get <fileID> [--out <path>]` + +Downloads and decrypts one file's original, the full-size file as uploaded. +Without `--out`, it writes the file to the working directory under the file's +title, with unsafe characters replaced (see "File names from the server"). With +`--out`, it writes to that path as given. It logs `<bytes> bytes -> <path>`. The +original is kept in the cache as well. + +A file ID that no collection holds gives `File <id> not found`, and the exit +status is 1. The option `--collection <id>` is accepted and ignored. + +A live photo is a still image and a short video that Ente stores as one file. +`get` writes both, each named after the title with its own extension, as Ente's +own clients name them: `IMG_0001.heic` and `IMG_0001.mov`. With `--out <path>`, +the image goes to `<path>` and the video beside it, with the same name and the +video's extension. A `<path>` that already ends in the video's extension is +refused, because both files would land on it. + +#### `quak get-thumb <fileID> [--out <path>]` + +Downloads and decrypts one file's thumbnail, the small JPEG Ente shows in a +grid. It writes one file, also for a live photo. Without `--out`, the name is +`thumb_` followed by the title. In all else it works as `get` does. + +#### `quak backup <dir> [--json] [--verify]` + +Backs up every file in the account to `<dir>`. The layout is described under +"The backup directory". The command downloads only the originals that are not +yet in the backup, and it goes on past a file that fails. A file is downloaded +once, however many collections hold it. An original the cache already holds is +copied from there instead. + +Progress goes to stderr: one line per original fetched, and a `FAILED` line for +each original or album that failed. The run ends with a summary of counts and, +when any file failed, a `Failed files:` list of every failed file. With +`--json`, the summary and the list are replaced by one JSON object on stdout. +Its fields are: + +- `totalFiles`: how many distinct files the run covered, however many + collections hold each; +- `downloaded`: how many originals this run fetched or copied from the cache; +- `skipped`: how many originals were already in the backup and left alone; +- `verified`, `mismatched` and `unchecked`: the counts a `--verify` run makes + (see "`--verify`"), all 0 without it; +- `failed`: how many files are in `failures.json` after this run; +- `errors`: this run's failures, one object per file, with `fileID`, `title`, + `collection` and `error`; +- `undecryptableAlbums`: the IDs of the collections quak could not decrypt; +- `failedAlbums`: the IDs of the collections whose directory or JSON could not + be written. + +With `--verify`, the run also hashes each original already in the backup and +replaces any whose bytes do not match the content hash Ente records. "The backup +directory" describes what it checks and what it changes. + +A backup refreshes the library as it starts and not again while it runs. Each +request it sends retries for longer than other commands' requests do (see +"Retries and timeouts"). + +The exit code is 0 when every original is in the backup and nothing failed. It +is 1 when files or albums failed, 2 when another backup of `<dir>` is running, +and 3 when there is no usable session (see "Exit codes"). + +#### `quak backup-metadata <dir> [--exif]` + +Writes the metadata quak keeps about the account, its collections and its files +as JSON files under `<dir>`. The layout is described under "The metadata dump +directory". Photo content is not written. With `--exif`, or its alias `--all`, +the command also fetches each file's original and records the image's own +metadata from it (see "The metadata dump directory"). Each original it fetches +stays in the cache. + +Ente's apps compute machine-learning data for each photo, such as the faces in +it (see "The JSON beside each original"). The command fetches it in requests of +up to 200 files. When a request fails, each of its files is written with the +reason in `mlDataError` instead of `mlData`, and the dump goes on. A file whose +data does not decrypt, decompress or parse also gets `mlDataError`. When any +file was written with `mlDataError`, the last line says how many and the exit +status is 1. + +A collection or file that quak cannot decrypt is left out of the dump without a +word, and does not change the exit status. + +#### `quak helper list-missing-thumbnails [--json]` + +Finds the files whose thumbnail is missing on the server. It downloads the +thumbnail of every file you own, one collection at a time, newest updated first, +so on a large account it takes a while. A thumbnail counts as missing when the +server answers with an empty body or with HTTP 404. Any other failure is logged +as `[<collection>] Could not check <title>: <reason> (not reported as missing)`, +and the file is left out of the list. The command exits 0 even when some files +could not be checked. + +The server lets only a file's owner replace its thumbnail. A file another +account owns, in a collection shared with you, is skipped and not checked. + +Without `--json`, the command logs `<n> file(s) with missing thumbnails:` and +prints one line per file on stdout. Each line holds the file's ID, its title, +its collection and the reason, separated by tabs. With nothing missing it logs +`No missing thumbnails found.` With `--json`, the output is a JSON array; each +object holds `fileID`, `title`, `collection` and `reason`. + +#### `quak helper fix-missing-thumbnails [--file <id>...] [--json]` + +Makes a new thumbnail for each file that lacks one, and uploads it. Without +`--file`, it first finds the files as `list-missing-thumbnails` does. With +`--file`, it works on the files named and checks nothing else. + +For each file, the helper: + +1. downloads the original; +2. makes a JPEG thumbnail from it; +3. encrypts the thumbnail under the file's own key; +4. uploads it. + +The server refuses a thumbnail larger than the one it already records for the +file. The helper therefore tries five encodings in turn and uploads the first +that fits. They run from 720 pixels on the long side at JPEG quality 50 down to +160 pixels at quality 20. + +Each file ends in one of three states: + +- `fixed`: a thumbnail was uploaded. +- `skipped`: the helper cannot make a thumbnail for the file, or the server + would refuse it. This covers: + - a file another account owns; + - a file that is not an image; + - an image that is not a JPEG; + - a file whose recorded thumbnail size is 0 or unknown; + - a file for which even the smallest encoding is too large. +- `failed`: the download, the encoding or the upload failed, or an ID given with + `--file` is held by no collection. + +Without `--json`, the command ends with counts of fixed, skipped and failed +files. A `Skipped:` list and a `Failed files:` list follow when they have +entries. With `--json`, one JSON array on stdout replaces them; each object +holds `fileID`, `title`, `collection`, `status` and, for a skipped or failed +file, `reason`. When any file failed, the exit status is 1. A skipped file does +not change it. When nothing needs fixing, the command logs +`No missing thumbnails found.` and exits 0. With `--json` it then writes nothing +on stdout. + +quak refuses the whole command, before any request, when one `--file` ID is +invalid (see "IDs"). + +### How a command gets current data + +Every command that reads the account answers for the account as it is now, not +for whatever the cache last held. Each one gets current data this way: + +- `collections` asks for the collection list alone, in one request. It fetches + no collection's files and changes nothing in the cache. +- `files` with a collection ID asks for the same list, then for that one + collection's files. It changes nothing in the cache either. +- `get`, `get-thumb` and `fix-missing-thumbnails --file` look the file up in the + cache after a refresh. On an empty cache, they ask for the collection list + instead, then for one collection's files at a time, and stop at the collection + that holds the file. `fix-missing-thumbnails --file` stops once it has found + every file named. These searches store nothing in the cache. The original or + thumbnail fetched is kept in the cache. +- `files` without a collection ID, `backup`, `backup-metadata` and the two + thumbnail helpers without `--file` wait for a refresh. + +A refresh asks the server which collections changed since the last refresh and +fetches the files of each one, one collection after another. On an empty cache +that is every collection. On a large account this takes a while. Nothing is +stored until every collection is in. Meanwhile the command logs a line for each +collection fetched, such as `Fetched the files of 3 of 120 collections`. + +When a refresh or a search fails, the command logs the error on one line and +exits 1, or 3 when the server no longer accepts the saved session. A collection +or file that quak cannot decrypt does not stop a refresh or a search: the +command leaves it out and answers from what did decrypt (see "Default reads and +fresh reads"). `backup` counts it as failed (see "What quak cannot decrypt"). + +### IDs + +A collection ID or file ID given to `files`, `get`, `get-thumb` or +`fix-missing-thumbnails --file` is written in decimal digits and is at +most 9007199254740991. Anything else, such as `0x10`, `1e3` or `-1`, is refused +before any request is sent. The command logs `Invalid file ID: <arg>` or +`Invalid collection ID: <arg>` and exits 1. + +### What quak prints + +stdout holds a command's output, such as the listing or the JSON, and the +prompts of `login`. Every other line goes to stderr as a log line at one of six +levels. From the quietest, the levels are fatal, error, warning, info, debug and +trace. A run prints the lines at its own level and at every quieter one. The +level is info unless `-v`, `-vv` or `-q` says otherwise. + +- Fatal: why a command exits non-zero. A command that runs and exits non-zero + logs at least one fatal line. A command that exits 1 because files or albums + failed logs it last, saying how many failed and where to look. +- Error: something that failed without ending the run. These lines are: + - the `FAILED` lines of `backup` and `fix-missing-thumbnails`; + - their `Failed files:` lists; + - the `ML data` lines of `backup-metadata`; + - `[<collection>] Could not check <title>` from the thumbnail helpers. +- Warning: something that did not fail but needs your attention. The only + warning is the one `logout` prints: the cache directory still holds decrypted + data. +- Info: everything else quak prints by default, such as a refresh's progress, + each original a backup fetches, and the summaries. +- Debug, with `-v`: each retry of a request, with why the attempt failed and how + long quak waits. Also the start and end of each refresh and of each fetch of + machine-learning data, with the error of one that failed. +- Trace, with `-vv`: each request quak sends, as its method, its URL and the + HTTP status of the answer or the error it failed with. The URL of a thumbnail + upload is logged without its query, which holds the upload's signature. + +`-q` lowers the level to fatal: a run that exits 0 prints nothing on stderr, and +one that exits non-zero prints only its fatal lines. The prompts of `login` are +not log lines, and `-q` leaves them alone. + +Two things on stderr are not log lines. A command line that quak cannot parse, +such as a missing argument, an unknown option or `-q` with `-v`, is refused with +an `error: …` message, and quak exits 1. `-q` does not hide the message. `quak` +with no command prints the usage on stderr and exits 1. + +`yarn quak` prints yarn's own lines on stdout as well. To pipe a command's +output, run `yarn --silent quak files --json` or +`node dist/bin/quak.js files --json` instead. + +With `--json`, a command's JSON on stdout takes the place of its summary on +stderr. For `backup`, that is the `--- Backup complete ---` line, the counts and +the `Failed files:` list. For `list-missing-thumbnails`, it is +`No missing thumbnails found.` or `<n> file(s) with missing thumbnails:`. For +`fix-missing-thumbnails`, it is the `--- Done ---` line, the counts, the +`Skipped:` list and the `Failed files:` list. Every other line is logged as +without `--json`. + ### Exit codes -| Code | Meaning | -| ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `0` | The backup is complete: every original is at its save path, and no file failed. | -| `1` | The run finished with files in `failures.json` or albums it could not decrypt or write (see "Backup layout"), and its last line says how many, or it stopped on another error, printed as one line `quak: <message>`. The next run tries again. | -| `2` | Another backup of the same directory is running. This run sent no request and changed nothing. | -| `3` | There is no usable session: none is saved, the saved one is corrupt, or the server no longer accepts it. Run `quak login` as the user the cron job runs as. | +- `0`: the command succeeded. For `backup`, every original is in the backup and + no file or album failed. +- `1`: the command failed. For `backup`, the run finished with files in + `failures.json` or with albums it could not decrypt or write, and its last + line says how many. The next backup run tries again. An unexpected error is + printed as one line, `quak: <message>` (see "Error messages"). +- `2`: another backup of the same directory is running. This run sent no request + and changed nothing. Only `backup` exits 2. +- `3`: there is no usable session. None is saved, the saved one is corrupt, or + the server no longer accepts it. Run `quak login` as the user the command runs + as. -`<message>` is the error's message, followed by the message of each error that -caused it. Under Node, as `yarn quak` runs quak, `fetch` reports every failure -below HTTP as just `fetch failed` and gives the reason as its cause, so a run -that could not reach the server prints, for example, +### Error messages + +An unexpected error is printed as one line, `quak: <message>`, with no stack +trace. `<message>` is the error's own message, followed by the message of each +error that caused it, separated by `: `. Some lines a command logs itself, such +as `File <id> not found` and `Login failed: <reason>`, have no prefix. + +`yarn quak` runs quak under Node. Node's `fetch` reports every network failure +as `fetch failed`, with the reason as its cause. A run that could not reach the +server therefore prints, for example, `quak: fetch failed: connect ECONNREFUSED 1.2.3.4:443`. When the server has -several addresses and none of them could be reached, the line gives the reason -for each, separated by `, `. A file whose download failed that way is in -`failures.json` with that error, as a transient failure. So is each file with no -ML data cached when the ML data fetch failed that way, its error starting -`ML data: `. The `~/bin/quak` that `make install` builds runs under bun, and -prints bun's own wording for such a failure, with no reason after it, such as -`quak: Unable to connect. Is the computer able to access the url?` for a refused -connection. It records such a failure in `failures.json` as transient too. +several addresses and none could be reached, the line gives the reason for each +one, separated by `, `. -After a `1`, the next run fetches each missing original again, fetches the ML -data that is not cached, and rebuilds the album links. An album quak could not -decrypt, or a file in `failures.json` because quak could not decrypt it, keeps -the exit code at `1` until the server sends a copy that decrypts, or deletes it -(see "Backup layout"). An original that a `--verify` run could not read, or -could not replace with a download that matches, stays at its save path as it -was. Only a later `--verify` run checks it again: a run without `--verify` -leaves it as it is and takes the file out of `failures.json`, so that run can -exit 0. +The `~/bin/quak` that `make install` builds runs under bun. Bun gives its own +wording for such a failure, with no reason after it. A refused connection, for +example, prints +`quak: Unable to connect. Is the computer able to access the url?`. -## Examples +A file whose download failed that way goes into the backup's `failures.json` as +a transient failure, with that error. So does each file whose machine-learning +data could not be fetched that way, with an error starting `ML data: `. -`examples/download-albums.ts` downloads every album's photos and their metadata -into a directory, `photos` in the working directory unless you name another. The -build compiles it; run it after `yarn install`: +## Files quak writes -```bash -yarn build -QUAK_EMAIL=… QUAK_PASSWORD=… node dist/examples/download-albums.js [dir] -``` +### The session file -It opens the library with `precacheThumbnails` and `precacheOriginals` off, as -`quak backup` does, so the only file content it fetches is the originals it -saves. It asks on the terminal for a two-factor or email code when the account -requires one, and writes: +`quak login` writes the session to `session.json` in quak's data directory. On +Linux that is `$XDG_DATA_HOME/quak/session.json`, or +`~/.local/share/quak/session.json` when the variable is not set. On macOS it is +`~/Library/Application Support/quak/session.json`. quak creates the file with +mode `0600`, in a directory it creates with mode `0700`. -- each photo's original at its save path under `dir`, as `photo.download()` - writes it: `YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>`, and for a live - photo its image, its video and the `.livephoto.json` file naming them -- beside each original, a JSON file named after it with `.json` added, for - example `2026-03-01.12345.jpg.json`: the photo's record (`photo.record()`) - without its cache paths, and every EXIF tag of the photo (`photo.exif()`) - under `exif` -- `albums/<collectionID>.json` for each album: its `collectionID`, its `name`, - and under `savePaths` the save paths of its photos relative to `dir`, newest - first +The file holds the account's email, its user ID, the session token and the +account's three keys, in base64 and unencrypted. Treat it as you would treat the +password. A missing file means "not logged in". A file that cannot be read back +is reported as corrupt, with the reason. Run `quak logout`, then `quak login`. -A photo in several albums is downloaded once. A second run downloads nothing and -rewrites only the JSON files whose content changed. A failed download stops the -run; running it again carries on, since every photo already saved is skipped. +A command that needs a session exits 3 when the file is missing or corrupt. It +also exits 3 when the server no longer accepts the session, and prints +`quak: the saved session is no longer valid; run "quak login"`. `quak backup` +learns this at its first refresh. By then it has written only its lock. When the +session stops working partway through a backup, each remaining file goes into +`failures.json`. That run exits 1, and the next run stops at its refresh with +status 3. -## Entrypoints +### The cache -This repository adheres to the -[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all) -standard: normalized scripts in `script/` are the entrypoints for the -development workflow, and most Makefile targets are thin shims that call them. -The scripts are POSIX sh (not bash) so they run in minimal containers such as -alpine. We provide: +Without `--cache-dir`, each account has its own cache directory: -- `script/bootstrap` — install all dependencies (node/yarn if missing, then - `yarn install --frozen-lockfile`) -- `script/setup` — set up the repo for development after a fresh clone: runs - `script/bootstrap`, then `script/install-precommit` -- `script/projectname` — output the project name (our own extension); used by - `script/docker` for the image tag -- `script/build` — compile the TypeScript sources into `dist/`, stamp the - version into `dist/package.json`, then verify that the entrypoints - `package.json` declares (`main`, `types`, `bin`) are among the files the - compiler wrote, and make the CLI executable (our own extension) -- `script/version` — print the version `script/build` stamps (our own - extension); see Version below -- `script/test` — run the test suite, by building the `test` phase of the - `Dockerfile` (vitest, 90s timeout, verbose rerun on failure); requires docker -- `script/lint` — run eslint and a prettier check, by building the `lint` phase - of the `Dockerfile`; requires docker (see Linting and testing below) -- `script/fmt` — format all files with prettier (writes) -- `script/fmt-check` — check formatting on the host (read-only) -- `script/check` — run all checks: `test`, `lint`, `fmt-check` (our own - extension) -- `script/docker` — build the image, tagged via `script/projectname` -- `script/cibuild` — what CI runs: `script/bootstrap`, then `script/check`, then - the image build -- `script/precommit` — run by the git pre-commit hook (our own extension); runs - `script/lint` and `script/fmt-check` but deliberately not the tests, so the - TDD red-phase commit can land -- `script/install-precommit` — installs the git pre-commit hook (our own - extension); `make hooks` shims to it +- on Linux, `$XDG_CACHE_HOME/quak/<userID>`, or `~/.cache/quak/<userID>` when + the variable is not set; +- on macOS, `~/Library/Caches/quak/<userID>`. -`make hooks` installs the pre-commit hook that runs `script/precommit`. - -### Linting and testing - -Linting and testing are phases of the `Dockerfile`. The `lint` phase copies the -repo into a digest-pinned node image and runs eslint and `prettier --check .`; -the `test` phase does the same with the suite. `script/lint` and `script/test` -each build one phase with `docker build --no-cache --target <phase>`. Neither -script runs the tools on the host: docker is required, and that also works where -the docker daemon is remote and bind mounts are impossible. - -The last stage of the `Dockerfile` compiles the package, and it copies a file -from each phase, so it cannot be built unless lint and the tests pass. The image -build in `script/cibuild` therefore runs lint and the tests a second time, after -`script/check` has run them. - -Every `docker build` in `script/` passes `--no-cache`. On an unchanged tree -Docker would otherwise serve the lint and test steps from cache, nothing would -run, and the build would still exit 0. - -`script/fmt-check` runs prettier on the host. Its verdict matches the `lint` -phase's: prettier is pinned to an exact version, installed from `yarn.lock` -under `--frozen-lockfile` in both places, and reads `.gitignore` as its default -ignore file — which is why `.dockerignore` keeps `.gitignore` in the build -context. - -### Version - -`quak --version` reports the `version` of `dist/package.json`, which -`script/build` writes after compiling; the repo's own `package.json` keeps -`0.0.0`, and that is what the tests, which run from source, report. -`script/version` decides what is written: - -- the `VERSION` environment variable, or the `Dockerfile`'s `VERSION` build arg - (`--build-arg VERSION=...`), when one is given and not empty; -- otherwise, in a checkout with `.git`, `git describe --tags --always`: the tag - on a tagged commit; the tag, the commits since it and the short commit on a - later commit (`v1.2.3-4-gabc1234`); the short commit when no tag is reachable; -- otherwise, as in a source tarball, the version `package.json` declares. - -The build fails if the checkout has `.git` and the version still comes out -empty, `dev` or `unknown`: such a build could not be traced back to its commit. - -`.dockerignore` therefore does not leave out `.git`, so any `docker build .` of -a clone stamps the commit it was built from; a shallow clone stamps a tag only -when the cloned commit itself carries one, and otherwise the short commit. It -leaves out `.git/config`, which holds the clone's remote URL and any credential -in it, so the image carries `.git` without its config; `git describe` does not -need that file. `script/docker` (and so `make docker`) and `script/cibuild` pass -the version they resolve on the host, with `--dirty`, as the build arg, which -takes precedence. The image's `org.opencontainers.image.version` label carries -that build arg only, so a build given none leaves it empty. `make build-bin` -bundles the built `dist/`, so the single binary reports the stamped version too. - -## Rationale - -Ente is one of very few photo services with a credible end-to-end encryption -story. The shipping clients (mobile Flutter, web React, desktop Electron, and Go -CLI) work, but they are slow, buggy, and difficult to script against. The -Flutter app fails to sync reliably. The web app is heavy. The desktop app is the -web app inside a slow Electron wrapper. The Go CLI is the closest thing to a -usable tool, but it is awkward to integrate from anything that is not a shell. -The Go CLI's backup mode crashes entirely when a single file download fails, -which makes it useless as an actual backup tool. - -quak fixes these problems. This repo ships a correct, well-tested implementation -of Ente's cryptographic protocol and API surface, plus a CLI that proves the -library is enough to do real work without a UI. The backup command is resilient -by design: per-file errors are logged and the run continues. - -The longer-term goal of this project is a simple desktop client for Ente, built -on this library in Electron (or a comparable runtime), with two priorities above -everything else: correctness and stability. Performance and simplicity follow -from those. Features will be added only after the protocol layer is correct, the -local cache is reliable, and the UI is responsive on a five-year-old laptop. - -## Development workflow - -All work on quak is test-driven. No exceptions. - -1. Every change starts on a feature branch off `next`, and its pull request - targets `next`. -2. The first commit on the branch is the test suite for what is being added or - changed. Those tests must fail at that commit; the branch is red until the - implementation lands. -3. Subsequent commits add the implementation and any refactors needed to make - the tests pass. -4. A pull request can only be merged into `next` when `make check` is green. - Once it has passed review, the repository manager squash-merges it into - `next`. Only sneak merges `next` into `main`. `main` and `next` are always - green. CI runs `script/cibuild`, which builds the `Dockerfile`: its `lint` - and `test` phases, then the compile, so neither a red branch nor one that - does not compile can pass CI. -5. Tests are the canonical API documentation for this library. Every test file - is commented thoroughly enough that a reader who has never seen quak can - learn how to use it from the tests alone. Comments explain why a behavior - matters, not just what the assertion checks. -6. Test fixtures (cryptographic vectors, recorded HTTP responses, sample files) - are committed alongside their tests. Where possible they are generated by - deterministic helpers in the `test/` tree so any reviewer can reproduce them - by running the helper. -7. `git rebase -i` is allowed on a feature branch before merge to clean up the - test-then-implementation sequence into reviewable commits, but the final - history must still show tests landing before (or with) the matching - implementation. -8. The pre-commit hook installed by `make hooks` runs `script/precommit`, which - runs `script/lint` and `script/fmt-check` but not the tests, and so not the - full `make check`. This is deliberate so the TDD red-phase commit (failing - tests, no implementation yet) can land. CI executes `script/cibuild`, which - runs the tests, so a red branch still cannot reach `next`. - -## Design - -quak is a TypeScript library with a thin CLI wrapper. The library does the work; -the CLI is for humans. - -### Layout +The directory is readable by its owner only (mode `0700`), because it holds +decrypted data, file keys included. ``` -quak/ - src/ - crypto/ libsodium primitives (boxes, secretstreams, KDF, hash) - api/ HTTP client (ApiClient class) - auth/ login flow (SRP + email OTP + TOTP), key unwrap - model/ decrypted Collection, File, Metadata types + decrypt fns - download/ streaming file/thumbnail download + decryption - library/ the cache-backed Library: metadata store, read - surface, records, content cache, precache, ML data - and search, request pools - backup.ts resilient full-account backup with dedup - metadata-backup.ts - backup-metadata: the metadata quak keeps, as JSON - exif.ts EXIF read from an image's bytes with exifreader - mldata-fetch.ts fetch + decrypt per-file ML data - clip-keywords.ts - a file's keywords from its CLIP embedding - clip-keyword-vectors.ts - Ente's Discover categories with their text - embeddings, written by bin/clip-keyword-vectors - filename.ts safe file names from server metadata - errors.ts error types shared across layers, and how an error - is described - retry.ts retry classifier + exponential backoff with jitter - thumbnails.ts detect + regenerate missing thumbnails - client.ts high-level Client class assembled from the above - cli-commands.ts the CLI's commands as functions returning exit codes - cli-log.ts the CLI's log level, and the one function the - commands write stderr lines through - cli-output.ts how the CLI prints a file's title and time - cli-read.ts fresh reads for the CLI's read commands - cli-run.ts run a command, print its error, exit with its code - cli-session.ts read the saved session file back into a Client - index.ts public library exports - bin/ - quak.ts CLI entrypoint (commander.js) - clip-keyword-vectors, clip-keyword-vectors.py - write src/clip-keyword-vectors.ts and a test - fixture (see Source attribution) - examples/ - download-albums.ts - download every album's photos and metadata - test/ unit + integration tests (vitest) - Makefile - Dockerfile lint phase, test phase, compile - package.json - tsconfig.json +<cacheDirectory>/ + metadata.json the decrypted collection and file records, + the refresh cursor, and the list of what + did not decrypt + originals/<fileID>.<ext> cached originals + originals/<fileID>.livephoto.json + which of a live photo's two files is which + thumbnails/<fileID>.jpg cached thumbnails + mldata/ + <fileID>.json one decrypted machine-learning payload per + file + clip.f32, clip.json the packed CLIP index and its list of IDs + fetched.json per-file fetch bookkeeping ``` -`make build` compiles that tree into `dist/`, preserving its shape: the library -lands in `dist/src/`, the examples in `dist/examples/`, and the CLI in -`dist/bin/quak.js`, which is what `package.json` points `main`, `types` and -`bin` at. The compiler's `rootDir` is the repository root rather than `src/`, -because `bin/` is compiled too and `rootDir` has to contain everything that is -compiled. +When `metadata.json` belongs to another account than the one logged in, opening +the library deletes it and `mldata/` and starts from an empty cache. Cached +originals and thumbnails are kept. They are reached only through the records of +the current account, so another account's files are never served. -### Cryptography +A file appears in the cache only once it is complete: each one is written to a +temporary file and renamed into place. Ente's clients record a hash of each +file's bytes at upload, the content hash. Every original whose metadata records +one is hashed as it is written. One that does not match is not stored: the +download fails with an error naming the file, so `quak get` exits 1. The hash is +Ente's: unkeyed BLAKE2b with a 64-byte output, in standard base64. A live photo +arrives as a ZIP of its image and its video and is unpacked as it is written. +Its two files are hashed separately and compared as `<imageHash>:<videoHash>`, +and neither is stored unless both match. quak refuses a live photo ZIP that +holds a second image or video. It also refuses one whose files unpack to more +than 20 times its size plus 16 MiB. An original with no recorded hash, uploaded +by a very old client, is stored unchecked. -All cryptography is done by `libsodium-wrappers-sumo` (the "sumo" build is -required for `crypto_pwhash` / Argon2id), except the SRP handshake, which uses -`fast-srp-hap`, and the MD5 checksum sent with a thumbnail upload, which uses -Node's built-in `node:crypto`. No hand-rolled crypto. +The cache has a size limit for originals (see "Opening a library"). Thumbnails +have none. A live photo's image and video leave the cache together. -The key hierarchy, derived during login, is: +### The backup directory -1. The user enters their password. -2. Argon2id (`crypto_pwhash`) over the password and a server-issued `kekSalt`, - with server-issued `memLimit` and `opsLimit`, produces a 32-byte Key - Encryption Key (KEK). -3. SRP login: `crypto_kdf_derive_from_key` (BLAKE2b) derives a 32-byte subkey - from the KEK with subkey id 1 and context `loginctx`. Its first 16 bytes are - the SRP password. -4. When SRP completes, the server returns a blob of "key attributes" plus an - encrypted auth token, or first asks for a second factor. quak answers a TOTP - request with the code (`POST /users/two-factor/verify`), after which the - server returns them, and cannot answer a passkey request. When the account - has email MFA on (`isEmailMFAEnabled`), an email OTP replaces SRP and the - server returns them after it. -5. `crypto_secretbox_open_easy` over the encrypted master key with the KEK - yields the 32-byte master key. -6. `crypto_secretbox_open_easy` over the encrypted secret key with the master - key yields the user's X25519 private key. The matching public key is - delivered in cleartext. -7. `crypto_box_seal_open` over the encrypted token with the user's keypair - yields the auth token's bytes. Encoded as URL-safe base64 with padding, they - are the `X-Auth-Token` value for all subsequent calls. - -Per-collection keys are decrypted with `crypto_secretbox_open_easy` using the -master key (for owned collections). Per-file keys are decrypted with -`crypto_secretbox_open_easy` using the collection key. File metadata is a -secretstream blob (single chunk, TAG_FINAL) under the file key. File content is -a chunked `crypto_secretstream_xchacha20poly1305` stream under the file key, -with a 4 MiB plaintext chunk size and a 17-byte authentication overhead per -chunk. Thumbnails use the same secretstream blob format as metadata. - -For upload (thumbnail repair), `encryptBlob` performs the push side: a single -secretstream chunk with TAG_FINAL, returning the header and ciphertext. - -### HTTP API - -Production endpoints: - -- API: `https://api.ente.io` -- File download CDN: `https://files.ente.io/?fileID=<id>` -- Thumbnail CDN: `https://thumbnails.ente.io/?fileID=<id>` - -A custom API endpoint is configurable for self-hosted servers via the -constructor option `apiOrigin`. When set, file downloads route through -`<apiOrigin>/files/download/<id>` instead of the dedicated CDN host. - -The constructor option `onRequest` is called once for each request sent, a -retried request once for each attempt, with its method, its URL and the status -of the answer or the error the request failed with. The URL of a presigned -upload is given without its query, which holds the upload's signature. The CLI -logs each request with `-vv` (see "CLI surface"). - -Required request headers on every authenticated call: - -- `X-Auth-Token`: the decrypted auth token from login. -- `X-Client-Package`: identifies the client. quak uses `berlin.sneak.quak`. - -Endpoints used: - -- `GET /users/srp/attributes?email=<email>`: fetch SRP and KDF parameters. -- `POST /users/srp/create-session`: begin SRP handshake. -- `POST /users/srp/verify-session`: complete SRP, receive 2FA challenge or the - encrypted token plus key attributes. -- `POST /users/ott` and `POST /users/verify-email`: email OTP, used instead of - SRP when the SRP attributes have `isEmailMFAEnabled` set. -- `POST /users/two-factor/verify`: TOTP second factor. -- `POST /users/logout`: end the calling token's session (`quak logout`). -- `GET /collections/v2?sinceTime=<usec>`: list collections changed since - microsecond timestamp; pass 0 for a full enumeration. -- `GET /collections/v2/diff?collectionID=<id>&sinceTime=<usec>`: list files in a - collection; paginate while `hasMore` is true. -- `GET https://files.ente.io/?fileID=<id>`: download encrypted file bytes. -- `POST /files/data/fetch`: fetch encrypted ML data for a batch of files. -- `POST /files/upload-url`: mint a presigned upload URL (for thumbnail repair). -- `PUT /files/thumbnail`: register an uploaded thumbnail's object key. - -### Retries and timeouts - -Every request in the library goes through one policy, in `src/retry.ts`. A -request is repeated only when repeating it could produce a different answer: - -- `ApiError` with a 5xx status: retried. So are `408` and `429`, the two 4xx - codes that are statements about timing rather than about the request. -- Every other 4xx: not retried. A 404 in particular is an answer, and - `listMissingThumbnails` depends on getting it promptly and once. -- Transport failures — a `fetch` rejection, `ECONNRESET`, `ETIMEDOUT`, a DNS or - TLS failure — and deadline aborts: retried. Node's `fetch` rejects with a - plain `TypeError`, so every `TypeError` is retried. The errno is looked for in - the error's `cause` chain, because that is where Node's `fetch` puts it. -- A truncated download: retried. -- Anything else, including a secretstream authentication failure that is not - truncation: not retried. The default answer is no. For a backup tool, retrying - a permanent failure spends round trips and delays every remaining file, while - declining to retry a transient one costs a single file that the next run picks - up. - -The `~/bin/quak` that `make install` builds runs under bun, whose `fetch` -rejects with a plain `Error` that has no `cause` and carries a code of bun's -own. quak retries a refused connection, a name that does not resolve, a reset -connection and a deadline under bun as it does under Node. Bun reports a refused -connection, and a TLS handshake that failed, as `ConnectionRefused`. For a name -that does not resolve it gives `ConnectionRefused` on some requests and -`FailedToOpenSocket` on others. A connection reset or closed before the response -ended is `ECONNRESET`, as under Node, and a deadline is the same `TimeoutError`. -A certificate bun rejects, reported with a code such as -`DEPTH_ZERO_SELF_SIGNED_CERT`, is not retried. - -Backoff is exponential with full jitter: the delay before retry _n_ is -`random() * min(maxDelayMs, baseDelayMs * 2 ** (n - 1))`. The exponential term -is the ceiling and the wait is drawn below it, so a client that lost many -parallel downloads to one CDN blip does not send them all again at the same -instant. The numbers, configurable through `ApiClientOptions.retry`: - -| Option | Default | `quak backup` | Meaning | -| ------------- | ------- | ------------- | ----------------------------------- | -| `attempts` | `4` | `10` | total calls, not retries | -| `baseDelayMs` | `500` | `1000` | ceiling for the first retry's delay | -| `maxDelayMs` | `10000` | `60000` | upper bound on that ceiling | - -With the defaults a file that is going to fail gives up after at most three and -a half seconds of waiting. `quak backup` usually runs from cron with nobody -watching, so every request it makes uses the `quak backup` column instead, -exported as `UNATTENDED_RETRY_OPTIONS`: a request that keeps failing gives up -after at most 243 seconds of waiting, and usually after about half that, since -each wait is drawn at random below its ceiling. Every other command uses the -defaults. A library user gets the same budget by passing -`UNATTENDED_RETRY_OPTIONS` as `ApiClientOptions.retry`. `sleep` and `random` are -injectable through the same option, which is how the test suite exercises the -whole policy without waiting. `onRetry`, given there too, is called before each -wait with the error the attempt failed with, the attempt's number and the wait; -the CLI logs it with `-v` (see "CLI surface"). - -Two deadlines, renewed for each attempt: - -| Option | Default | Applies to | Kind | -| ------------------- | ------- | ------------------------------------------- | ------------------------------------- | -| `requestTimeoutMs` | `30000` | `getJSON`, `postJSON`, `putJSON`, `putFile` | the whole request | -| `downloadTimeoutMs` | `60000` | file and thumbnail downloads | idle: no bytes received for this long | - -They are different kinds because a download's length depends on the file and the -link: a whole-transfer deadline short enough to catch a hung connection would -cancel a large video on a slow link that is still making progress. The download -deadline restarts every time bytes arrive, so a slow download runs as long as it -keeps moving, and one that stalls is aborted after 60 seconds of silence. It -covers the wait for the headers and the body — `getFileStream` returns as soon -as headers arrive, so a deadline that only guarded the initial request would -leave the same hang one layer down. There is no limit on the total length of a -download. - -**Non-idempotent requests are not blindly replayed.** `postJSON` and `putJSON` -send every `POST` and `PUT` in the endpoint list above; some of them change -server state, and `/users/two-factor/verify` consumes one of a small number of -second-factor attempts. They are retried only when every errno in the error's -`cause` chain is one of the three that establish no TCP connection to the server -ever existed, so no request byte can have been transmitted: `ENOTFOUND` and -`EAI_AGAIN` (name resolution produced no address) and `ECONNREFUSED` (the peer -refused the connection). A 5xx, a mid-flight reset and a deadline are all left -to the caller, because each of them can happen after the server has already -acted. These two do not follow redirects either: a redirect means the server -already received the request, so it is reported as an error and not retried. The -routing errnos `EHOSTUNREACH`, `ENETUNREACH` and `ENETDOWN` are excluded for the -same reason, despite looking like connect-time failures: on Linux an ICMP -unreachable arriving mid-flight, or a local interface going down after the -request was written, delivers them on an already-established socket. They stay -retryable for the idempotent calls. So do bun's `ConnectionRefused` and -`FailedToOpenSocket`, on which a `POST` or `PUT` is not replayed either. When a -connection bun kept open from an earlier request closes before any answer, bun -sends the request again on a new connection, and if that one fails it gives one -of these codes although the server may already have the request. `putFile` is -exempt: a presigned PUT stores one whole object at one key in one request, so -replaying it has no partial state to damage. - -A download is retried as a whole — request, stream consumption, and decryption — -because a socket reset after the response headers have arrived surfaces in the -download layer rather than in `ApiClient`, and that is the common failure for -multi-megabyte photos over a CDN. The secretstream pull state is not resumable -and these endpoints have no Range support, so a retry starts the file over. The -atomic write is part of the retried unit: each attempt writes its own temporary -files, one for most files and two for a live photo (its image and its video), -and removes them if it fails. Only the attempt that completes renames anything -into place. The retry sits below `runBackup` and `runMetadataBackup`. -`runBackup` logs, counts and steps over a file that still fails after its -retries; `runMetadataBackup`, which downloads only with `--exif`, records the -error in that file's JSON as `imageMetadataError` and goes on. - -One imprecision is deliberate and worth knowing about. When a body ends part-way -through a secretstream chunk, Poly1305 fails and carries no framing signal, so a -cut connection and genuinely corrupt bytes are indistinguishable. quak reports -that as truncation, which means it is retried. For a body of more than one chunk -the distinction is real — a chunk that failed while the stream carried on past -it stays an authentication failure and is not retried — but for a single-chunk -body, which is most thumbnails and every small file, a wrong key, server-side -corruption and a mid-chunk cutoff all present alike and all get retried. The -cost is bounded by the attempt count, and it buys never silently keeping a -truncated file. - -### Session handling - -The `Client` class holds the auth token, master key, secret key, and public key -in memory. There is no on-disk session store in the library; the consumer -decides how to persist sessions. - -`client.toJSON()` returns a `ClientSnapshot` (a plain serializable object with -base64-encoded keys) that the consumer can write to disk, a database, or -whatever else fits their use case. `Client.fromJSON(snapshot)` restores a -working client from that snapshot without re-authenticating; it checks every -field and each key's length first, and throws an error naming the bad field. -`client.logout()` clears the token and zeroes the key buffers in place; after -it, every other method on that client throws. It does not contact the server, so -the token stays valid there and in any saved snapshot; -`await client.logoutOnServer()` first ends the session on the server -(`POST /users/logout`). - -The CLI stores the snapshot at the platform-appropriate data directory via -`env-paths`: `~/Library/Application Support/quak/session.json` on macOS, -`$XDG_DATA_HOME/quak/session.json` on Linux. The file is written with mode -`0600`. The key material is stored in cleartext in the JSON; treat this file as -you would treat the password itself. A missing file is reported as "not logged -in"; a file that exists but is corrupt is reported as such, naming the bad -field. When the refresh a command starts with gets HTTP 401 from the server, -because it no longer accepts the saved session's token, the command prints one -line, `quak: the saved session is no longer valid; run "quak login"`, with no -stack trace. All three exit with status 3, which means the user must run -`quak login` again. `quak logout` is the exception: with no file it says there -is no session and exits 0, and it handles a corrupt file or a failed server call -as described below. `quak backup` meets an expired session on the refresh that -starts every run, before it touches any file but its lock (see "Backup layout"). -A session that stops working partway through a backup instead fails each -remaining file into `failures.json`, so that run exits 1 and the next one stops -at its refresh with status 3. No command but `quak login` ever prompts. - -`quak logout` ends the session on the server, so the token in `session.json` -stops working even in a copy of the file, and then deletes the file. If the -server call fails (or the file is corrupt), the file is still deleted, the -command says the server session could not be ended, and it exits with status 1. -It does not delete the cache. When it knows the cache directory, from -`--cache-dir` or from a session file it could read, it prints it and says it -still holds decrypted data (file keys in `metadata.json`, cached originals and -thumbnails), for the user to delete if they want it gone. - -### CLI surface - -``` -quak [--cache-dir <path>] <command> global: local metadata/content cache location -quak [-v | -vv | -q] <command> global: log more on stderr, or only fatal errors -quak login interactive or QUAK_EMAIL/QUAK_PASSWORD -quak whoami print logged-in account as JSON -quak logout end the session, delete it -quak collections [--json] list all collections -quak files [collectionID] [--json] list files in a collection, or every file -quak get <fileID> [--out path] [--collection] download and decrypt a file -quak get-thumb <fileID> [--out] [--collection] download and decrypt a thumbnail -quak backup <dir> [--json] [--verify] full incremental backup -quak backup-metadata <dir> [--exif] dump the metadata quak keeps as JSON -quak helper list-missing-thumbnails [--json] find files with missing thumbnails -quak helper fix-missing-thumbnails [--file ids] [--json] generate + upload missing thumbnails -``` - -Every command except `login`, `whoami` and `logout` runs on the cache-backed -library. The read commands — `collections`, `files`, `get`, `get-thumb`, -`backup-metadata`, `helper list-missing-thumbnails` and -`helper fix-missing-thumbnails` — force a fresh server round-trip before they -answer, so they report current account state rather than whatever the cache last -held. `collections` asks the server for the collection list alone, in one -request, and neither fetches any collection's files nor changes the cache. -`files` with a collection ID asks for the same list, then for that one -collection's files, and does not change the cache either. On an empty cache, -`get` and `get-thumb` ask for the same list, then for one collection's files at -a time, until they find the file. `helper fix-missing-thumbnails --file` does -the same until it finds every file it names. They store none of the collections -or file records they read. The original or thumbnail they fetch is kept in the -cache. Once the cache holds the account's collections, they wait for a refresh -instead. The others wait for a refresh, and so does `files` without a collection -ID. With an empty cache, as on a first run, that refresh fetches the files of -every collection, one collection after another, and stores nothing until it has -them all, which on a large account takes a while. Meanwhile the command logs a -line on stderr for each collection whose files it has fetched, such as -`Fetched the files of 3 of 120 collections`, with `--json` too; `backup` does -the same while the refresh it starts with runs. stdout holds only the command's -output. If that round-trip fails, the command prints the error on one line and -exits 1, or 3 when the server no longer accepts the saved session (see "Session -handling"). An album or file quak cannot decrypt does not fail it; the command -answers from what did decrypt (see "Default reads vs. fresh reads"). -`--cache-dir` overrides where the cache lives; without it each account gets its -own directory under the per-user cache path. - -`-v` (`--verbose`) and `-q` (`--quiet`) set how much quak logs on stderr. Like -`--cache-dir`, they go before or after any command's name. Every line quak -prints on stderr is a log line at one of six levels, from the quietest: fatal, -error, warning, info, debug and trace. A run prints the lines at its own level -and at every quieter one. stdout holds only the command's output, at every -level. - -- Every command that exits non-zero logs a fatal line saying why, such as - `quak: <message>`, `Not logged in`, an invalid ID or a collection that does - not exist. A command that exits 1 because files or albums failed logs it last, - saying how many failed and where to look: `backup` points to `failures.json` - and its `FAILED` lines, `backup-metadata` to the `mlDataError` in the JSON of - each such file, and `helper fix-missing-thumbnails` to its `Failed files:` - list, or to its JSON with `--json`. -- An error line reports something that failed: the `FAILED` lines that `backup` - and `helper fix-missing-thumbnails` log as they work, - `ML data of file <id> could not be read` and - `ML data request for <n> file(s) failed` from `backup-metadata`, - `Could not check <title>` from both thumbnail helpers, and the `Failed files:` - list at the end of `backup` and of `helper fix-missing-thumbnails`. -- A warning reports something that did not fail but needs your attention: after - `logout`, that the cache directory still holds decrypted data. -- Every other line quak logs by default is info. - -With `--json`, a command's result on stdout takes the place of its summary on -stderr: for `backup`, the lines from `--- Backup complete ---` to the end of its -`Failed files:` list; for `helper list-missing-thumbnails`, -`No missing thumbnails found.` or `<n> file(s) with missing thumbnails:`; and -for `helper fix-missing-thumbnails`, the lines from `--- Done ---` to the end of -its `Failed files:` list. Every other line is logged as it is without `--json`. - -- With neither option the level is info: the lines quak prints as it works, such - as a refresh's progress, the files a backup fetches, its summary and the files - that failed. -- `-v` raises the level to debug, which adds each retry of a request, with why - the attempt failed and how long quak waits before the next, and the start and - end of each refresh and each ML data fetch, with the error of one that failed. -- `-vv` raises it to trace, which adds each request quak sends: its method, its - URL and the status of the answer, or the error the request failed with. The - URL of an upload of a thumbnail is logged without its query, which holds the - upload's signature. More repeats stay at trace, and `--verbose --verbose` is - `-vv`. -- `-q` lowers it to fatal: a run that exits 0 prints nothing on stderr, and one - that exits non-zero prints only its fatal lines. The prompts of `login` are - not log lines, and `-q` leaves them as they are. - -`-q` with `-v` is refused with one line, -`error: option '-q, --quiet' cannot be used with option '-v, --verbose'`, and -exit status 1. - -`files <collectionID>` lists the files in that collection. `files` with no -argument lists every file in the account, collection by collection, each file -once: a file in several collections is listed with the first of them, which is -its `collectionID` in `--json`. - -An ID given to `files`, `get`, `get-thumb` or -`helper fix-missing-thumbnails --file` is written in decimal digits, no larger -than 9007199254740991. Anything else, such as `0x10`, `1e3` or `-1`, is refused -before any request with one line, `Invalid file ID: <arg>` or -`Invalid collection ID: <arg>`, and exit status 1. One bad `--file` ID refuses -the whole command, so nothing is fixed. - -Without `--out`, `get` writes the file to the working directory under its title, -and `get-thumb` writes the thumbnail there under `thumb_` followed by the title, -each with unsafe characters replaced. A name over 255 bytes, which most file -systems refuse, is cut short between two characters before its extension, so the -extension is kept whenever it fits. An extension too long to leave room for the -name's first character is cut short too, so the name never starts with a dot. - -`get` and `get-thumb` resolve the file by ID directly, so `--collection` is -accepted for backward compatibility but ignored. For a live photo, `get` writes -its image and its video, each named after the title with its own extension, as -Ente's clients name them (`IMG_0001.heic` and `IMG_0001.mov`), and each name is -cut short as above once its own extension is in it. With `--out PATH`, the image -is written to `PATH` and the video beside it, with `PATH`'s name and the video's -extension; a `PATH` with the video's extension is refused. -`backup-metadata --exif` (alias `--all`) additionally fetches each file's -original through the cache and records, from it or a live photo's image, its XMP -metadata, its EXIF metadata and, for a JPEG, its dimensions. EXIF is read with -[exifreader](https://github.com/mattiasw/ExifReader) from any image format it -reads, JPEG, HEIC/HEIF, AVIF, PNG and WebP among them. The record's `exif` field -is exifreader's EXIF tag output: each tag by name, with its `value`, -`description` and `computed` value. An EXIF block exifreader finds but reads no -tag from is recorded, base64, as `exifRaw`, with the reason in `exifError`. -`collections`, `files`, `backup`, `helper list-missing-thumbnails` and -`helper fix-missing-thumbnails` take `--json` for machine-readable output. - -`backup --verify` also hashes the originals already in the backup and replaces -any that do not match the content hash Ente records; one it cannot replace stays -as it was and goes into `failures.json` (see "Backup layout"). - -`backup-metadata <dir>` writes `account.json` and, for each collection, a -directory `collections/<id>-<name>/` holding `_collection.json` and a -`<fileID>.json` for each of its files. Each `<fileID>.json` holds the file's ML -data and keywords as `quak backup` writes them (see "Backup layout"). The -collection's name has unsafe characters replaced and is cut short between two -characters so that the directory's name fits in 255 bytes; the ID keeps the -directories apart. - -`backup-metadata` fetches ML data in requests of up to 200 files. When a request -fails, the error is logged, each of its files is written with the reason in an -`mlDataError` field instead of `mlData`, and the dump goes on. A file whose ML -data does not decrypt, gunzip or parse is logged and written the same way, with -why in `mlDataError`. The exit code is non-zero if any file was written with -`mlDataError`, and the last line then says how many were. - -`helper fix-missing-thumbnails` regenerates thumbnails for JPEG images only, -because the bundled decoder (`jpeg-js`) decodes only JPEG. A non-JPEG image -(PNG, HEIC) or a video is reported as `skipped` (unsupported format), kept -distinct from a `failed` repair, and does not affect the exit code; a genuine -failure still exits non-zero. The server accepts a new thumbnail only from the -file's owner and only when it is no larger than the thumbnail size it records -for the file. So a file another account owns, in an album shared with you, is -skipped by both thumbnail helpers without being fetched, and the fixer skips a -file whose recorded thumbnail size is 0 or unknown. Otherwise the fixer lowers -the quality and size of the thumbnail until it fits, and skips the file if even -the smallest does not. - -### Backup layout - -`quak backup <dir>` produces: +`quak backup <dir>` produces this tree: ``` <dir>/ YYYY/YYYY-MM/YYYY-MM-DD/ - YYYY-MM-DD.<fileID>.<ext> actual file content, at its save path (one - per unique file, two for a live photo: see - below) - YYYY-MM-DD.<fileID>.json the file's basic metadata fields quak - keeps, its update time, its private and - public magic metadata, its ML data and - keywords, and, for an image (for a live - photo, its image), its original's EXIF - and XMP, with dimensions for a JPEG - only; none of these three for a video + YYYY-MM-DD.<fileID>.<ext> the original, at its save path (one per + file, two for a live photo) + YYYY-MM-DD.<fileID>.json the metadata quak keeps about the file, + plus its machine-learning data, its + keywords and its image metadata YYYY-MM-DD.<fileID>.livephoto.json which of a live photo's two files is which collections/ <name>/ <title> -> ../../YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext> - (symlink) - <name>.json collection metadata + file list + a symlink per file + <name>.json the collection's metadata and file list account.json the account's email and user ID - backup.lock the lock a running backup holds (see below) - failures.json files that failed and have not yet succeeded + backup.lock the lock a running backup holds + failures.json the files that failed and have not yet + succeeded ``` -Each original is saved at its save path, the same path `photo.savePath` gives -and `photo.download()` writes (see Read surface below). The date is the photo's -`takenAt` (the date set in Ente if it was edited, else its creation time) in the -time zone of the machine running quak. The extension is the one in the file's -name as uploaded, case kept, or `.bin` when it has none, it holds anything but -letters and digits, or it is more than 16 of them long. When the date or the -time zone changes, the next run saves the original at its new path and leaves -the old copy where it is. +#### Save paths -`account.json` holds the account's `email` and `userID`, as `backup-metadata` -writes it. A collection's JSON holds its `id`, `name`, `type`, `ownerID`, -`isShared` and `updationTime`, its `magicMetadata`, `pubMagicMetadata` and -`sharedMagicMetadata` when it has them, as `backup-metadata`'s -`_collection.json` does, and `files`, each file's `id` and `metadata`. A file's -JSON holds its `id`, `collectionID`, `ownerID`, `metadata` and `updationTime`, -and its `magicMetadata` and `pubMagicMetadata` when it has them. Update times -are in microseconds, as Ente records them. +Each original is stored once, at its save path: +`YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>` under `<dir>`. The date is +the day the photo was taken, in the time zone of the machine running quak. That +is the date set in Ente when the date was edited, and otherwise the file's +creation time. The extension is the one in the file's title, with its case kept. +It is `.bin` when the title has no extension, when the extension holds anything +but letters and digits, or when it is longer than 16 characters. -A file's JSON holds Ente's ML data for it as `mlData`, such as the faces Ente -found, without its CLIP embedding. It holds `keywords` in place of the -embedding. +When the date or the time zone changes, the next run stores the original at its +new path and leaves the old copy where it is. -The Discover section of Ente's photos app has 16 categories: +The library gives the same path as `photo.savePath` and writes it with +`photo.download()` (see "A photo's content"). + +#### The JSON beside each original + +The JSON file beside an original holds: + +- `id`, `collectionID`, `ownerID` and `updationTime`; +- `metadata`: the fields quak keeps from the file's own metadata. They are + `title`, `fileType`, `creationTime`, `modificationTime`, `latitude`, + `longitude` and `hash`. The last three are present only when the file has + them. +- `magicMetadata` and `pubMagicMetadata`, when the file has them. The magic + metadata is Ente's name for the fields its clients add later, such as an + edited name, an edited date or a caption. + +Update times are in microseconds, as Ente records them. + +The JSON also holds the machine-learning data Ente computed for the photo, as +`mlData`. Ente's apps detect faces in each photo and compute a CLIP embedding +for it, a list of 512 numbers that describes what the photo shows. `mlData` +holds everything but that embedding. In its place the JSON holds `keywords`, +derived from the embedding as described under "Keywords". A file with no +machine-learning data has neither field. A file whose data holds no usable +embedding has no `keywords`. + +The backup waits for the library's fetch of machine-learning data to finish +before it writes the JSON files. When a file's data cannot be fetched, +decrypted, decompressed or parsed, the JSON holds the reason in `mlDataError` +instead of `mlData`. Such a file counts as failed and goes into `failures.json` +with an error starting `ML data: `. Nothing is cached for it, so the next run +asks for its data again. The JSON files are rewritten on every run, so data that +arrived since the last run appears. + +Every file but a video also holds `imageMetadata`: what the original itself +records. That is its `format`, `width` and `height` for a JPEG, its EXIF tags as +`exif`, and its XMP packet as `xmp`. EXIF and XMP are the two blocks of metadata +a camera or an editor writes inside an image file. For a live photo, these come +from its image. An original with none of them gets `{}`. When the original +cannot be read, the reason is in `imageMetadataError` instead, and neither the +file nor the run fails. + +EXIF is read with [exifreader](https://github.com/mattiasw/ExifReader) from any +image format it reads, such as JPEG, HEIC, AVIF, PNG and WebP. `exif` holds each +tag by name, with its `value`, `description` and `computed` value. An EXIF block +that exifreader finds but reads no tag from is kept in base64 as `exifRaw`, with +the reason in `exifError`. + +The original is read for this when the run stores it, or when the JSON beside it +has neither `imageMetadata` nor `imageMetadataError`. Otherwise the two fields +are copied from the JSON being rewritten, so a run does not read every stored +original again. + +#### Keywords + +The Discover section of Ente's photos app sorts photos into 16 categories. quak +derives a file's keywords the way the app sorts it. The categories are: - Identity (no videos) - Screenshots (no videos) @@ -895,507 +746,775 @@ The Discover section of Ente's photos app has 16 categories: - Hills - Greenery -A file's keywords are the titles of the categories it falls in. A file's score -for a category is the cosine similarity of its CLIP embedding and the text -embedding of the category's query. As in Ente's app, the file falls in the -category when that score is at least the category's minimum score. A video never -falls in a category marked "no videos". The keywords are listed highest score -first. +Each category has a query, and a text embedding of that query from Ente's CLIP +text model. A file's score for a category is the cosine similarity of the file's +embedding and the category's: a number from -1 to 1, higher the more alike the +two embeddings are. The file falls in the category when its score reaches the +category's minimum score. The keywords are the titles of the categories the file +falls in, highest score first. A video never falls in a category marked "no +videos", as in the app. -A file with no ML data has neither `mlData` nor `keywords`. A file whose ML data -holds no usable CLIP embedding has no `keywords`. `backup-metadata` writes both -fields the same way. +#### Collection directories -The backup waits for the library's ML data fetch to finish before it writes the -JSON files. If that fetch fails, each file whose ML data is not in the cache -gets the reason in `mlDataError` instead and counts as failed, and the next run -fetches it again. The same goes for a file with no ML data in the cache whose ML -data the server sent but quak could not decrypt, gunzip or parse. Its -`mlDataError` says why, and `failures.json` records it with an error starting -`ML data: `. quak keeps that reason in memory only and caches nothing for the -file, so the next run asks for its ML data again. An ML data fetch that fails, -such as one the library's refresh starts while the backup writes the JSON files, -leaves the reasons as the last fetch that succeeded left them. The JSON files -are rewritten on every run, so ML data that arrived since the last run appears. +Each collection gets a directory under `collections/`, named after the +collection, and a JSON file beside it, `<name>.json`. The directory holds one +symlink per file, named after the file's title, pointing at the original's save +path. A live photo gets two symlinks, one to its image and one to its video, +each named after the title with that file's extension. -A file's JSON holds, as `imageMetadata`, what `backup-metadata --exif` records -from its original, or from a live photo's image: `format`, `width` and `height` -for a JPEG, `exif` (or `exifRaw` and `exifError`), and `xmp`. An original with -none of these gets `{}`. A video gets no `imageMetadata`: reading a whole video -to look for tags is not worth it. When the original cannot be read, the reason -is in `imageMetadataError` instead; neither the file nor the run fails. The -original is read when the run stores it, or when the JSON beside it has neither -field, as one an earlier version wrote. Otherwise the field is taken from that -JSON when it is rewritten, so a run does not read every stored original again. +The JSON file holds: -`failures.json` records each failed file with the kind of failure, how many -times it has been tried and when it was last tried. A file leaves it once it -succeeds, or once it is no longer in the library or in the backup's scope. The -library's `lib.backup({ includeThumbnails: true })` also writes -`thumbnails/<fileID>.jpg` beside `collections/`; `quak backup` does not. +- `id`, `name`, `type`, `ownerID`, `isShared` and `updationTime`; +- `magicMetadata`, `pubMagicMetadata` and `sharedMagicMetadata`, when the + collection has them; +- `files`: each file's `id` and `metadata`. -A file in the backup's scope that the library could not decrypt (see "Default -reads vs. fresh reads") goes into `failures.json` with an error that starts -`could not be decrypted:` and gives the reason. If a copy of it in another album -decrypted, it does not: its original is stored from that copy, and the album -where it does not decrypt neither links nor lists it. An album the library could -not decrypt has no name quak can read and no files it can list. `quak backup` -prints one line for it on stderr, -`FAILED album <id>: could not be decrypted: <reason>`, and `--json` and the -`BackupResult` list its ID in `undecryptableAlbums`. Such a file in -`failures.json`, or such an album, keeps the exit code at 1, run after run, -until the server sends a copy that decrypts or deletes it; the next run then -drops it. +Names come from the server, so unsafe characters in them are replaced (see "File +names from the server"). When two collections would get the same directory name, +ignoring case, each gets its ID added: two albums named `Trip` become +`Trip (10)/` and `Trip (11)/`. Two files in one collection whose titles match, +ignoring case, get their IDs added before the extension: `IMG_0001 (12345).JPG` +and `IMG_0001 (12346).JPG`. A collection named like another collection's JSON +file, such as `Trip.json` beside `Trip`, also gets its ID added. A name too long +for a file name, which most file systems limit to 255 bytes, gets its ID added +too. It is then cut short before the ID, just far enough to fit. IDs never +change, so a name stays the same from run to run until such a clash appears or +goes away. + +Each run removes every symlink in a collection's directory that points into the +date folders and is not one of its current links, yours included. A file or +symlink of yours at the name of one of its links is replaced by the link. A +directory of yours at that name stays, and the photo whose link it blocks goes +into `failures.json`. Anything else you put under `collections/` stays. A run +removes the directory and JSON of a collection that was deleted or renamed, once +the directory holds nothing else. One that still holds something of yours stays, +with its JSON. + +A collection whose directory or JSON cannot be written does not stop the run. +For example, a file you put there may already have the directory's name. The run +logs `FAILED album <name>: <reason>`, lists the collection's ID in +`failedAlbums`, and exits 1. The other collections and `failures.json` are still +written. + +#### What quak cannot decrypt + +A collection or file that quak cannot decrypt is left out of the library (see +"Default reads and fresh reads"). The backup treats the two cases this way: + +- A file quak cannot decrypt goes into `failures.json` with an error starting + `could not be decrypted:` and the reason. It stays out of `failures.json` when + its copy in another collection decrypts. The original is then stored from that + copy, and the collection where it does not decrypt neither links nor lists it. +- A collection quak cannot decrypt has no name quak can read and no files it can + list. The run logs `FAILED album <id>: could not be decrypted: <reason>` and + lists the ID in `undecryptableAlbums`. + +Every run exits 1 until the server either sends a copy that decrypts or deletes +the record. The run after that drops the entry. A collection that stops +decrypting loses its directory and JSON as a deleted one does, and a file that +stops decrypting loses its symlink in that collection. The originals stay at +their save paths. + +#### Live photos + +Ente stores a live photo as one ZIP of its image and its video. The backup +stores the two files, which a photo viewer can open: each is +`YYYY-MM-DD.<fileID>.<ext>` with the extension it has inside the ZIP, such as +`2026-03-01.12345.heic` and `2026-03-01.12345.mov`. The file +`YYYY-MM-DD.<fileID>.livephoto.json` beside them names which is which. The live +photo counts as stored only when its `.livephoto.json` and both files it names +are present and not empty. + +#### `failures.json` + +`failures.json` records each file that failed and has not succeeded since. It +holds `version`, which is 1, and `files`, one entry per file keyed by its ID. +Each entry holds: + +- `fileID` and `title`; +- `classification`: the kind of failure; +- `attempts`: how many runs in a row have failed the file; +- `lastTriedAt`: when it was last tried, as a Unix time in milliseconds; +- `error`: the error, as quak printed it. + +The kinds are: + +- `transient`: a network or server problem; +- `permanent`: a refusal, or a local file-system or decryption error; +- `unknown`: anything else. + +A file leaves the list once it succeeds, or once it is no longer in the library. +A library caller that backs up only some albums (`onlyAlbumNames`) also drops a +file outside them. `failures.json` is deleted when the list is empty. + +#### `backup.lock` `backup.lock` keeps two backups of the same directory from running at once, such as a cron run that starts while the previous one is still going. A backup -creates `<dir>` if it is missing and takes the lock before its refresh, and -removes the lock when it ends, whether it succeeds or fails, unless it lost the -lock (see below); `quak backup` takes it before it opens its library, so a -refused run sends no request. The lock is a directory that -[proper-lockfile](https://github.com/moxystudio/node-proper-lockfile) creates -and keeps touching while the backup runs. A second backup of the directory, from -another process or the same one, fails at once: `quak backup` prints -`quak: another backup of <dir> is running` and exits with status 2. A run -stopped with Ctrl-C or `kill` removes the lock as it exits. Only a run that -cannot, such as one killed with SIGKILL or cut off by a crash or power loss, -leaves it behind; once it has gone 10 seconds untouched, the next run takes it -over, so nobody has to remove it. The lock is outside the date folders and -`collections/`, so it is never taken for an original, and the removal of old -album directories never touches it. +creates `<dir>` if it is missing and takes the lock before its refresh, so a +refused run sends no request. A run that still holds the lock removes it when it +ends, whether it succeeded or failed. -A running backup loses its lock when someone removes `backup.lock`, or when -another backup takes it over because the running one left it untouched for 10 -seconds, as a stopped or stalled process does. The backup finds this out the -next time it goes to touch the lock, which it does every 5 seconds, and then -stops before it starts another file. The originals it is downloading at that -moment are still stored. It leaves `failures.json` and `backup.lock` as they -are, since another backup may be using them. `quak backup` prints -`quak: lost the lock on <dir>: <reason>` as one line and exits 1, and -`lib.backup()` rejects with `lost the lock on <dir>: <reason>`. +The lock is a directory that +[proper-lockfile](https://github.com/moxystudio/node-proper-lockfile) creates. +While the backup runs, it updates the modification time of `backup.lock` every 5 +seconds. A second backup of `<dir>` fails at once: it prints +`quak: another backup of <dir> is running` and exits 2. A run that is killed, or +cut off by a crash, can leave the lock behind. Once the lock has gone 10 seconds +without an update, the next run takes it over, so nobody has to remove it by +hand. -A collection's directory and JSON are named after the collection, and a symlink -after the file's title, both with unsafe characters replaced. When two -collections would get the same name, or two symlinks in one collection the same -name (ignoring case in both), each of them gets its ID added: two albums named -`Trip` become `Trip (10)/` and `Trip (11)/`, and two files titled `IMG_0001.JPG` -become `IMG_0001 (12345).JPG` and `IMG_0001 (12346).JPG`. A collection whose -directory would have the name of another collection's JSON, again ignoring case, -gets its ID added as well: beside an album `x`, whose JSON is `x.json`, an album -named `x.json` gets the directory `x.json (12)/`. A name too long for a file -name, which most file systems limit to 255 bytes, gets its ID added too. A name -with its ID added is cut short between two characters in the part before the ID, -just far enough that it fits. When a symlink's extension leaves no room for the -ID and the first character of the name, the extension is cut short, and the part -before the ID keeps that character and any bytes the cut leaves over. A -collection's directory name also leaves room for the `.json` its JSON adds. IDs -never change, so a name stays the same from run to run until such a clash -appears or goes away. +A running backup loses its lock when someone removes `backup.lock`. Another +backup takes the lock over when this one has not updated it for 10 seconds, for +example because this process was stopped. The backup notices at its next update +of the lock, and stops before it starts another file. The originals it is +downloading at that moment are still stored. It leaves `failures.json` and +`backup.lock` as they are, since another backup may be using them. It prints +`quak: lost the lock on <dir>: <reason>` and exits 1. -A collection whose directory or JSON cannot be written, such as one whose -directory name is taken by a file you put in `collections/`, does not stop the -run. `quak backup` prints `FAILED album <name>: <reason>` for it on stderr, -`--json` and the `BackupResult` list its ID in `failedAlbums`, and the exit code -is 1. The other collections and `failures.json` are still written. +#### `--verify` -A live photo, which Ente stores as one ZIP of its image and its video, is stored -as those two files, which a photo viewer can open: each is -`YYYY-MM-DD.<fileID>.<ext>` with the extension it has inside the ZIP (for -example `2026-03-01.12345.heic` and `2026-03-01.12345.mov`), and -`YYYY-MM-DD.<fileID>.livephoto.json` names the two. The live photo counts as -stored only when both files are present and not empty. Its album folder links -both, each named after the title with that file's extension (`IMG_0001.heic` and -`IMG_0001.mov`). +With `--verify`, a run hashes each original already at its save path the way a +download is checked. For a live photo it hashes the image and the video and +joins the two as `<imageHash>:<videoHash>`. An original that matches the content +hash in its metadata is left as it is. One that does not is logged on one line +naming the file, downloaded again in the same run, and replaced once the new +copy matches. The new copy is downloaded even when the cache holds the original, +because a copy from the cache is not checked as a download is. -Each run removes the symlinks into the date folders that no longer belong in -their collection's directory, and the directories (and JSON) of collections that -were deleted or renamed. An album quak can no longer decrypt loses its directory -and JSON the same way, and a file whose copy in an album quak can no longer -decrypt loses its symlink there and its entry in that album's JSON; the -originals stay at their save paths. Nothing else in `collections/` is touched: a -file or a symlink you put there stays, and a directory that still holds one -after its symlinks are removed stays too, with its JSON. +The new copy is written to a temporary file beside the original, hashed the same +way, and renamed over the original only when it matches. A live photo's image +and video are both hashed before either is renamed. Until then the original +stays as it was. When the new copy fails or does not match: -Each file is downloaded exactly once regardless of how many collections it -appears in, and written once: straight to its save path, with no copy left in -the cache. An original the cache already held is copied from there instead. On -subsequent runs, existing originals are skipped. If a download fails, the error -is logged and the backup continues with the next file. The exit code is non-zero -if any files failed. `quak backup` opens its library with the thumbnail and -originals precache off, so the only file content it fetches is the originals the -backup stores. It also opens it with no background timer, so it asks the server -for changes only as it starts, not again while it downloads. +- the temporary files are deleted; +- the original keeps its bytes and its album links; +- the file goes into `failures.json`. -With `--verify`, or `lib.backup({ verify: true })`, a run also hashes each -original already at its save path the way a download is checked (see "On-disk -cache layout" below): its bytes, read in chunks, or a live photo's image and -video, joined as `<imageHash>:<videoHash>`. An original that matches the content -hash its metadata records is left as it is. One that does not is logged on one -line naming the file, and replaced once a matching copy is downloaded, in the -same run. The copy is downloaded even when the cache holds the original, because -a copy from the cache is not checked as a download is. It is written to a -temporary file beside the original, a live photo's image and video each to its -own, hashed the same way, and renamed over the original only when it matches; a -live photo's image and video are both hashed before either is renamed. Until -then the original stays as it was: when the download fails, or what it brings -does not match either, the temporary files are deleted, the original keeps its -bytes and its album links, and the file goes into `failures.json`. A run that -stops between renaming a live photo's new image and its new video leaves the new -image beside the old video, which a later `--verify` run replaces if the video -was the part that did not match. A file whose metadata records no hash is left -as it is and counted as unchecked. A stored original that cannot be read is left -as it is and counts as failed. The summary and `--json` add the counts -`verified`, `mismatched` and `unchecked`, all of originals that were already -stored; one first downloaded in this run is in none of them. A mismatch that was -replaced with matching bytes does not make the exit code non-zero. Without -`--verify` nothing is hashed, the summary is unchanged, and the three counts are -0 in `--json`. +A run that stops between renaming a live photo's new image and its new video +leaves the new image beside the old video. A later `--verify` run then replaces +the video, if the video was the part that did not match. + +A file whose metadata records no hash is left as it is and counted as unchecked. +A stored original that cannot be read is left as it is and counted as failed. +The summary and the `--json` output add the counts `verified`, `mismatched` and +`unchecked`. Each counts only originals that were already stored. A mismatch +that was replaced with matching bytes does not make the exit code non-zero. + +A run without `--verify` drops two kinds of failure from `failures.json`: + +- a mismatched original whose new copy failed; +- a stored original that cannot be read. + +That run finds the original in place, counts it as skipped and drops the entry, +so it can exit 0. Only a `--verify` run hashes the original again. + +Without `--verify`, no stored original is hashed, the summary is unchanged, and +the three counts are 0 in the JSON. + +#### Temporary files and crash safety Each original is written to a temporary file in the same directory, synced to -disk, and renamed into place, so an original is either complete or absent, even -after a power cut. A downloaded original's temporary file is named -`.quak-<pid>-<random>.tmp`, one copied from the cache -`.quak-backup-YYYY-MM-DD.<fileID>.<ext>-<pid>-<random>.tmp`. A copy that -`--verify` downloads to replace an original is written to such a -`.quak-<pid>-<random>.tmp` too, a live photo's image and video to -`.quak-<pid>-<random>.<ext>`, each with its own extension. A run that is killed -can leave one of these temporary files behind; the next backup deletes those -whose process is no longer running. The content cache uses the same scheme, and -opening a library deletes the temporary files in the cache whose process is no -longer running, so a download another process has in progress in the same cache -is left alone. The rename replaces whatever was at the destination rather than -writing through it: a symlink there is replaced, not followed, and the new file -has the temporary file's permissions, not those of the file it replaced. +disk, and renamed into place. So an original is either complete or absent, even +after a power cut. The temporary files are named: -## TODO +- `.quak-<pid>-<random>.tmp` for a downloaded original, and for the copy that + `--verify` downloads; +- `.quak-backup-YYYY-MM-DD.<fileID>.<ext>-<pid>-<random>.tmp` for an original + copied from the cache; +- `.quak-<pid>-<random>.<ext>` for the image and the video of a live photo that + `--verify` downloads, each with its own extension. -- [x] Retry policy: no retry on 4xx (except `408` and `429`), exponential - backoff on 5xx and network errors -- [x] Update the API reference section below to match the current implementation -- [x] `make docker` green -- [x] Store live photos in a form a photo viewer can open - (https://git.eeqj.de/sneak/quak/issues/107): unpacked into the image and - the video +A run that is killed can leave one of these files behind. The next backup +deletes those whose process is no longer running. The cache uses the same +scheme. Opening a library deletes such leftovers in the cache's `originals/` and +`thumbnails/`. A download another process has in progress is left alone. -Tagging and releases are decided by sneak alone, and happen only when he -declares one. +The rename replaces whatever was at the destination rather than writing through +it. A symlink there is replaced, not followed, and the new file has the +temporary file's permissions, not those of the file it replaced. Each original +is written with mode `0600`, readable by its owner only. Everything else the +backup writes, its folders and JSON files included, gets the mode your umask +allows, which usually lets every user on the machine read it. The JSON beside an +original holds the file's location and faces. The `backup-metadata` tree is +written the same way. -Future (desktop client, separate repo): +### The metadata dump directory -- [ ] Electron app skeleton consuming this library -- [ ] Local cache (SQLite) keyed on `(collectionID, fileID, updationTime)` -- [ ] Background sync worker that streams new files into the cache -- [ ] Gallery UI: thumbnails, full-image view, basic search -- [ ] Upload, delete, and share operations in the library +`quak backup-metadata <dir>` produces this tree: -## API reference +``` +<dir>/ + account.json the account's email and user ID + collections/ + <id>-<name>/ + _collection.json the collection's metadata + <fileID>.json one file's metadata, one per file in + the collection +``` -The library's primary surface is the cache-backed `Library`; the lower-level -`Client` sits underneath it and is covered by the Design sections above. The -test suite is the canonical, executable documentation — `test/library/` and -`test/client/usage.test.ts` walk most operations, `test/cli/backup.test.ts` -walks `lib.backup()`, `test/cli/backup-end-to-end.test.ts` runs `quak backup` -against a fake of the Ente HTTP API, and `yarn test` verifies them. +Each run writes into `<dir>` and deletes nothing. `account.json` holds `email` +and `userID`. In `<id>-<name>`, the name has unsafe characters replaced. It is +cut short so the directory name fits in 255 bytes. The ID keeps two collections +of the same name apart. + +`_collection.json` holds the fields a backup's collection JSON holds, except +`files`. Each `<fileID>.json` holds the fields a backup's JSON beside an +original holds, with these differences: + +- `imageMetadata` is written only with `--exif`. The original of every file, + videos included, is then downloaded and read for its EXIF, XMP and JPEG + dimensions. +- A file whose original holds no image metadata gets no `imageMetadata` field. +- An original that cannot be downloaded or read gives `imageMetadataError` + instead. +- A file in several collections has its original read once; its image metadata + is written with the first of them. + +### File names from the server + +A file's title and a collection's name come from the server, so a name taken +from them is made safe before it is used on disk. Each path separator, each +character Windows forbids in a file name and each control character becomes `_`. +A leading run of dots becomes one `_`. A name Windows reserves for a device, +such as `CON`, gets a leading `_`. An empty title becomes `file-<id>`. An empty +collection name becomes `collection-<id>` in a backup and `unnamed` in the +metadata dump. + +A file name over 255 bytes is shortened before its extension, without splitting +a character, so the extension is kept when it fits. An extension too long to +leave room for the name's first character is cut short too, so the name never +starts with a dot. A collection's directory name is cut at its end. A path you +give with `--out` is used as it is. + +## Library reference + +The library's main class is `Library`. The `Client` sits underneath it and +handles login, saved sessions and the raw requests. The test suite is the +executable documentation: + +- `test/library/` shows most of what `Library` does; +- `test/client/usage.test.ts` shows the `Client`; +- `test/cli/backup.test.ts` shows `lib.backup()`; +- `test/cli/backup-end-to-end.test.ts` runs `quak backup` against a fake of the + Ente HTTP API. ### Opening a library -`Library.open(options)` loads the on-disk cache, starts the background refresh -loop, and resolves to a `Library`. On an empty cache it awaits the first -refresh, so it opens onto the account's data whenever the server is reachable; -if that refresh fails, it opens with no data and records the error in -`lib.status()`. On an existing cache it returns immediately and refreshes in the -background, so an unreachable server does not block opening. With -`refreshOnOpen: false` it starts no refresh and returns at once onto what the -cache holds, empty or not; a refresh then starts only when `fresh()` or -`lib.backup()` asks, or the background timer fires. The CLI's read commands open -it so (see "CLI surface"). +`Library.open(options)` loads the on-disk cache and resolves to a `Library`. On +an empty cache it waits for the first refresh, so it opens onto the account's +data whenever the server is reachable. If that refresh fails, it opens with no +data and records the error in `lib.status()`. On an existing cache it returns at +once and refreshes in the background, so an unreachable server does not block +opening. With `refreshOnOpen: false` it starts no refresh and returns at once +onto what the cache holds, empty or not. A refresh then starts only when +`fresh()` or `lib.backup()` asks, or when the background timer fires. `LibraryOptions`: -| Option | Default | Meaning | -| ------------------------ | -------------------------------------------------- | -------------------------------------------------------------------- | -| `client` | required | the account client (a `Client`, or any `LibraryClient`) | -| `cacheDirectory` | `quak/<userID>` under the per-user cache directory | where `metadata.json` and the content cache live | -| `downloadDirectory` | `photos` in the working directory | root of the save paths; an original stored there counts as cached | -| `refreshIntervalSeconds` | `3` | background refresh cadence; `Infinity` for no background timer | -| `refreshOnOpen` | `true` | start a refresh as the library opens | -| `precacheThumbnails` | `true` | prefetch every thumbnail, newest first | -| `precacheOriginals` | `true` | prefetch the favorites album and the latest-window originals | -| `precacheOriginalsDays` | `7` | length in days of that latest window | -| `cacheOriginalsMaxBytes` | 100 GiB | size limit on the originals cache; pinned originals can exceed it | -| `freeBelowBytes` | 50 GiB | free space to protect on the volume; the effective limit adapts down | -| `isOriginalPinned` | none | extra predicate for originals that must never be evicted | -| `pools` | fresh `RequestPools` | the bounded request pools (sets concurrency) | -| `onProgress` | none | refresh/ML/precache progress callback (`RefreshEvent`) | -| `contentSource` | the client's own | override the byte source (mainly for tests) | +- `client`, required: the account's `Client`, or anything that satisfies + `LibraryClient`. +- `cacheDirectory`: where `metadata.json` and the content cache live. Default: + `quak/<userID>` under the per-user cache directory (see "The cache"). +- `downloadDirectory`: the root of the save paths. Default: `photos` in the + working directory, resolved once when the library opens; + `lib.downloadDirectory` holds the result. An original stored at its save path + counts as cached. +- `refreshIntervalSeconds`: how long after one refresh ends the background timer + starts the next. Default `3`. `Infinity` starts no timer. +- `refreshOnOpen`: whether `open()` starts a refresh. Default `true`. +- `precacheThumbnails`: fetch every thumbnail in the account in the background, + newest first, until all are on disk. Default `true`. +- `precacheOriginals`: fetch, in the background, the originals of the favorites + album and of every file taken in the latest days (see the next option). These + originals are pinned: they are never evicted from the cache. Default `true`. +- `precacheOriginalsDays`: how many days count as the latest, counted back from + the newest file in the account. Default `7`. +- `cacheOriginalsMaxBytes`: the size limit on the originals cache. Default 100 + GiB. The limit in force is this, or less when the disk is fuller (see the next + option). Pinned originals can push the cache over it. +- `freeBelowBytes`: the free space to keep on the cache's volume. Default 50 + GiB. When the volume cannot hold the limit above and this much free space too, + the limit in force shrinks to what fits. `lib.status().originalsLimitBytes` + reports the limit in force. +- `isOriginalPinned`: a function that names more originals to pin. +- `pools`: the request pools, which set how many requests run at once (see + "Request pools"). Default: a fresh `RequestPools`. +- `onProgress`: a callback for the progress of refreshes, machine-learning + fetches and the background downloads, called with a `RefreshEvent`. +- `contentSource`: a replacement for the client's own byte source. Mainly for + tests. -The default `downloadDirectory` is resolved against the working directory once, -when the library opens; `lib.downloadDirectory` holds the result. +In the content and thumbnail pools, your own requests take the next free slot +ahead of the two background downloads. A file already in the cache is not +fetched again. When the originals cache grows past its limit, the least recently +used originals are evicted, but never a pinned one, and never one whose fetch is +still in flight. -`onProgress` receives a `RefreshEvent` with status `"started"` as a refresh, an -ML data fetch or a precache sweep starts, and `"done"` or `"failed"` as it ends. -In between, a refresh sends `"progress"` after each collection whose files it -fetched, with `fetched`, how many collections it has fetched the files of so -far, and `total`, how many it will. A refresh fetches the files of each -collection that is new or changed since the cache's copy, so on an empty cache -it fetches those of every collection. +`onProgress` receives a `RefreshEvent`. Its `operation` names the work: +`"refresh"`, `"fetchMLData"` for a machine-learning fetch, and +`"precacheThumbnails"` or `"precacheOriginals"` for the two background +downloads. Its `status` is `"started"` as the work starts and `"done"` as it +ends. A refresh or a machine-learning fetch that fails ends with `"failed"` +instead, with the message in `error`. In between, a refresh sends `"progress"` +after each collection whose files it fetched: `fetched` is the number of +collections fetched so far, and `total` the number it will fetch. A refresh +fetches the files of each collection that is new or changed since the cache's +copy, so on an empty cache it fetches those of every collection. The `"done"` +event of a machine-learning fetch, and of each background download, also carries +`fetched`: how many items it newly stored. -Concurrency is set through `pools`: construct -`new RequestPools({ metadataConcurrency, contentConcurrency, thumbnailConcurrency })` -and pass it. The three pools default to 10 / 5 / 25 (see Request pools below). +`lib.status()` returns a `LibraryStatus`. It holds the collection and file +counts. `lastRefreshAt` is the time of the last refresh that succeeded, and +`lastError` the message of the latest refresh, set only while that refresh +failed. `lastMLFetchAt` and `lastMLError` do the same for machine-learning +fetches. It also holds the originals cache's usage and limit in force, the +background downloads' progress and `closed`. `lib.status().mlDataErrors` lists +each file whose data arrived in the last successful fetch but could not be +decrypted, decompressed or parsed. Each entry is `{ fileID, error }`. The +`fetchMLData` event with status `"done"` carries the same list. Such data is not +cached, so the next fetch asks for it again. The reasons are kept in memory +only. -`lib.status()` returns a `LibraryStatus` (collection/file counts, last -refresh/ML times and errors, originals usage and effective limit, precache -progress, and `closed`). `lib.close()` stops the background timer; it is -idempotent, and an in-flight refresh is left to finish. The promise it returns -resolves once that refresh (including its cache write), the ML data fetch and -the precache fetches already running have all finished, so the cache directory -can then be removed. +`lib.close()` stops the background timer. It can be called more than once. A +refresh in flight is left to finish, with its write to the cache. The promise +`close()` returns resolves once that refresh, the machine-learning fetch and the +background downloads already running have all finished. The cache directory can +then be removed. After `close()`, `fresh()`, `freshCollections()`, +`freshFiles()` and `lib.backup()` reject with `the library is closed`. -`lib.status().mlDataErrors` lists, as `{ fileID, error }`, each file whose ML -data the last ML data fetch that succeeded got but could not decrypt, gunzip or -parse; the `fetchMLData` event with status `"done"` carries the same list. A -fetch that is still running leaves the list as it is until it ends, and a fetch -that fails leaves it as it was. Such ML data is not cached, so the next ML data -fetch asks for it again. The reasons are kept in memory only, never in the -cache. +### Default reads and fresh reads -### Default reads vs. fresh reads +Default reads, through `lib.albums`, `lib.photos` and `lib.timeline`, answer +synchronously from the copy held in memory and never touch the network. A +refresh replaces that copy only once all its server requests have succeeded. So +a default read is immediate, but only as current as the last refresh whose +requests all succeeded. Right after opening an existing cache, it is the copy on +disk. -Default reads — `lib.albums`, `lib.photos`, `lib.timeline` — answer -synchronously from the copy held in RAM and never touch the network. A refresh -changes that copy only once all its server requests have succeeded, and the -background timer starts the next refresh `refreshIntervalSeconds` after the -previous one ends. So a default read is immediate, but only as current as the -last refresh whose requests all succeeded; right after opening an existing -cache, it is the copy on disk. - -`await lib.fresh()` waits for a refresh to complete and persist, and returns the -same `{ albums, photos, timeline }` namespaces, which then reflect a completed -server round-trip. When a refresh is already running, background or not, -`fresh()` waits for that one, so its answer can come from requests made before -the call; only when none is running does it start one. A refresh that fails -rejects the caller (default reads stay silent and keep serving the last good -copy). The CLI's read commands use fresh reads (issue -https://git.eeqj.de/sneak/quak/issues/75). +`await lib.fresh()` waits for a refresh to complete and persist, then returns +the same `{ albums, photos, timeline }`, which then reflect a completed server +round-trip. When a refresh is already running, `fresh()` waits for that one, so +its answer can come from requests made before the call. Only when none is +running does it start one. When the refresh fails, `fresh()` rejects. Default +reads stay silent and keep serving the last good copy. `await lib.freshCollections()` is the fresh read of the collection list alone: -one collections request, answered with every live collection that decrypts, as -the `Collection` objects `listCollections()` returns, in the server's order. It -fetches no collection's files and stores nothing, not even the list: a refresh -fetches a collection's files only when the collection is new or has changed -since the stored copy, so a collection stored without its files would never get -them. `quak collections` uses it (issue -https://git.eeqj.de/sneak/quak/issues/210). +one request, answered with every collection that is not deleted and decrypts, as +`Collection` objects, in the server's order. It fetches no collection's files +and stores nothing, not even the list. It stores nothing because a refresh +fetches a collection's files only when the collection is new or changed. A +collection stored without its files would never get them. -`await lib.freshFiles(collection)` is the fresh read of one collection's files. -It answers with every live file in that collection that decrypts, in the -server's order. `collection` is a record that `freshCollections()` returned. The -method fetches no other collection's files and stores nothing. `quak files` with -a collection ID uses it, and so do `quak get` and `quak get-thumb` on an empty -cache. `quak helper fix-missing-thumbnails --file` uses it on an empty cache -too. +`await lib.freshFiles(collection)` is the fresh read of one collection's files, +as `EnteFile` objects, in the server's order, each file once. `collection` is an +object that `freshCollections()` returned. The method fetches no other +collection's files and stores nothing. -An album or file in the server's answer that quak cannot decrypt, or whose -metadata is not JSON, does not fail the refresh. The refresh leaves it out, an -album with all its files, and drops any older copy the cache held, so reads -answer from what did decrypt. The library keeps a list of what it left out, with -the reason, in `metadata.json`: the next refresh asks the server only for what -changed, so it would not see the record again. A record stays on that list until -a later refresh gets a copy that decrypts or learns that the server deleted it. -`lib.backup()` reports each such album, and each such file in its scope that has -no copy that decrypted (see "Backup layout"). +The refresh does not fail on a record it cannot decrypt, or whose metadata is +not JSON. It leaves the record out; a collection left out takes its files with +it. It also drops any older copy the cache held, so reads answer from what did +decrypt. The library keeps a list of what it left out, with the reason, in +`metadata.json`. It needs the list because the next refresh asks only for what +changed, and would not see the record again. A record stays on that list until a +later refresh gets a copy that decrypts or learns that the server deleted it. -### Read surface +### Albums, photos and the timeline -- `lib.albums.list()` → `Album[]`, newest-updated first. - `lib.albums.byID({ collectionID })` and `byName({ albumName })` → - `Album | undefined`. -- `lib.photos.byID({ fileID })` → `Photo | undefined`. - `lib.photos.records({ fileIDs })` → `PhotoRecord[]` in the requested order, - each id once, unknown ids dropped. -- `lib.timeline.groups({ groupBy, filter? })` → `TimelineGroup[]`, grouped by - `"day" | "week" | "month"` (keys `YYYY-MM-DD`, ISO `YYYY-Www`, `YYYY-MM`), - newest group first. A `PhotoFilter` combines `albumID`, `text` - (title/caption/album-name substring), `fileTypes`, `hasLocation`, and - `includeArchived`; hidden photos are always excluded. +- `lib.albums.list()` returns `Album[]`, newest updated first. + `lib.albums.byID({ collectionID })` and `lib.albums.byName({ albumName })` + return `Album | undefined`. Names are not unique in Ente, so `byName` picks + the newest updated album of that name. +- `lib.photos.byID({ fileID })` returns `Photo | undefined`. + `lib.photos.records({ fileIDs })` returns `PhotoRecord[]` in the order asked + for, each ID once, unknown IDs dropped. +- `lib.timeline.groups({ groupBy, filter? })` returns `TimelineGroup[]`, newest + group first. `groupBy` is `"day"`, `"week"` or `"month"`, and the group keys + are `YYYY-MM-DD`, `YYYY-Www` (the ISO week) and `YYYY-MM`. Each group holds + `key`, `startsAt` and `fileIDs`. `key` is that group key. `startsAt` is the + start of the period, in local-time milliseconds. `fileIDs` lists its files, + newest first. Weeks start on Monday. -An `Album` exposes its record fields and `album.photos.list()` → `Photo[]` -(newest first). A `Photo` exposes its record fields other than `thumbnailPath` -and `originalPath`, and `photo.year`, the local-time year of `takenAt`, all as -synchronous getters read from RAM; `photo.record()` → `PhotoRecord`. Two more -synchronous getters look at the disk and never touch the network: +A `PhotoFilter` narrows the timeline. Its fields combine with AND: -- `photo.savePath` → `string` — where `photo.download()` and `lib.backup()` put - the original, whether or not it is there yet: - `YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>` under the library's - `downloadDirectory` (see Backup layout above for the date and the extension). - For a live photo already stored, its image. For a live photo not yet stored, - it carries the title's extension, and the image may be stored under a - different one, found inside the live photo. It needs no content source. -- `photo.isLocal` → `boolean` — whether the whole original is at `savePath`. A - copy only in the cache does not count. +- `albumID`: only photos in that album. +- `text`: a case-insensitive substring of the title, the caption or the name of + an album the photo is in. +- `fileTypes`: only photos of one of these types. +- `hasLocation`: `true` keeps only photos with coordinates, `false` only those + without. +- `includeArchived`: archived photos are left out unless this is `true`. -These async methods may download: +Hidden photos are always left out. -- `await photo.original(opts?)` → `{ path, bytes, videoPath? }` — the - full-resolution file. For a live photo, `path` and `bytes` are its image's and - `videoPath` is its video. -- `await photo.download()` → `{ path, bytes, videoPath? }`, as `original()` — - puts the original at `savePath`, creating its folders. When it is already - there, nothing is written. When the cache holds it, it is copied from the - cache; otherwise it is fetched straight to `savePath`, with no copy left in - the cache. Afterwards `isLocal` is true. -- `await photo.thumbnail(opts?)` → `{ path, bytes }`. -- `await photo.content(opts?)` → `Uint8Array` — the original's bytes, read - through `original()`; for a live photo, its image's. -- `await photo.exif(opts?)` → `ExifTags` — every EXIF tag in the file, keyed by - tag name, each as exifreader decodes it, with its `id`, `value`, `description` - and `computed` value: for example `Make` is - `{ id: 271, value: ["Canon"], description: "Canon", computed: "Canon" }`. A - tag exifreader has no name for is keyed `undefined-<tag number>`. The embedded - thumbnail's tags are under `Thumbnail`, so they cannot hide the main image's - tags of the same name; the thumbnail image itself is left out. EXIF is read - from any image format exifreader reads (such as JPEG, HEIC/HEIF, AVIF, PNG, - WebP and TIFF), a live photo's image included. Any other original gives `{}`, - and a video gives `{}` without being downloaded. -- `await photo.make(opts?)`, and likewise `model()`, `lensModel()`, - `dateTimeOriginal()`, `offsetTimeOriginal()`, `exposureTime()`, `fNumber()`, - `iso()`, `focalLength()`, `orientation()`, `gpsLatitude()`, `gpsLongitude()` - and `gpsAltitude()` → one common field each, picked from the tags `exif()` - returns and typed as in `PhotoExif`, or `undefined` when the file lacks it. - GPS values are signed decimal degrees and metres. `dateTimeOriginal()` is the - camera's clock reading held in the `Date`'s UTC fields; - `offsetTimeOriginal()`, when present, is that clock's offset from UTC. Each - calls `exif()` with its `opts`, so each call reads the original again. +An `Album` exposes its record's fields and `album.photos.list()`, which returns +`Photo[]`, newest first. A `Photo` exposes its record's fields except the cache +paths, plus `photo.year`, the local-time year of `takenAt`. All of these are +synchronous getters read from memory. `photo.record()` returns the +`PhotoRecord`. -They serve from the on-disk content cache when the bytes are present and -otherwise fetch through the pools; `original()`, `content()` and `exif()` also -serve an original already stored at its save path. `opts.onProgress` reports -per-file progress. They throw when the library was opened without a content -source. An original that `original()`, `content()` or `exif()` downloads lands -in the cache, which does not make `isLocal` true; only `download()` and -`lib.backup()` do. +Two more getters look at the disk and never touch the network: + +- `photo.savePath`: where `photo.download()` and `lib.backup()` put the + original, whether or not it is there yet (see "Save paths"). For a live photo + already stored, it is the image. For a live photo not yet stored, it carries + the title's extension, and the image may be stored under another one. +- `photo.isLocal`: whether the whole original is at `savePath`. A copy only in + the cache does not count. + +### A photo's content + +These methods may download. They serve from the cache when the bytes are present +and otherwise fetch through the pools. They throw when the library was opened +without a content source. `opts.onProgress`, where accepted, reports the +progress of that one file. + +- `await photo.original(opts?)` returns `{ path, bytes, videoPath? }`, the + full-size file. `bytes` is the file's size. For a live photo, `path` and + `bytes` are its image's and `videoPath` is its video. `original()` also serves + an original already stored at its save path. +- `await photo.download()` returns the same shape and puts the original at + `savePath`, creating its folders. When it is already there, nothing is + written. When the cache holds it, it is copied from the cache. Otherwise it is + fetched straight to `savePath`, with no copy left in the cache. Afterwards + `isLocal` is true. +- `await photo.thumbnail(opts?)` returns `{ path, bytes }`. +- `await photo.content(opts?)` returns the original's bytes as a `Uint8Array`, + read through `original()`. For a live photo, its image's. +- `await photo.exif(opts?)` returns `ExifTags`: every EXIF tag in the file, + keyed by tag name, each as exifreader decodes it with its `id`, `value`, + `description` and `computed` value. For example, `Make` is + `{ id: 271, value: ["Example"], description: "Example", computed: "Example" }`. + A tag that exifreader has no name for is keyed `undefined-<tag number>`. The + tags of the thumbnail embedded in the file are under `Thumbnail`, so they + cannot hide the main image's tags of the same name. The thumbnail image itself + is left out. Any file exifreader cannot read gives `{}`, and a video gives + `{}` without being downloaded. +- `await photo.make(opts?)` returns one common EXIF field, picked from the tags + `exif()` returns and typed as in `PhotoExif`, or `undefined` when the file + lacks it. These methods do the same for their fields: + - `model()` and `lensModel()`; + - `dateTimeOriginal()` and `offsetTimeOriginal()`; + - `exposureTime()`, `fNumber()`, `iso()` and `focalLength()`; + - `orientation()`; + - `gpsLatitude()`, `gpsLongitude()` and `gpsAltitude()`, as signed decimal + degrees and metres. + +`dateTimeOriginal()` is the camera's clock reading, held in the `Date`'s UTC +fields. `offsetTimeOriginal()`, when present, is that clock's offset from UTC. +Each of these methods calls `exif()`, so each call reads the original again. + +An original that `original()`, `content()` or `exif()` downloads lands in the +cache, which does not make `isLocal` true. Only `download()` and `lib.backup()` +do. `await lib.originalOf(file, opts?)` and `await lib.thumbnailOf(file, opts?)` fetch and cache as `photo.original()` and `photo.thumbnail()` do. `file` is an -`EnteFile` the caller holds, and the library need not hold it. On an empty -cache, `quak get` and `quak get-thumb` pass one that `lib.freshFiles()` -returned. +`EnteFile` the caller holds, and the library need not hold it. `quak get` and +`quak get-thumb` use them; on an empty cache they pass a file that +`lib.freshFiles()` returned. -Lower-level accessors that return decrypted model objects (which hold key -material) are also available: `listCollections()`, `getCollection(id)`, -`listFiles(collectionID)`, `getFile(collectionID, fileID)`, and -`getFileByID(fileID)`. +Lower-level reads return the decrypted model objects, which hold key material: +`lib.listCollections()`, `lib.getCollection(id)`, `lib.listFiles(collectionID)`, +`lib.getFile(collectionID, fileID)` and `lib.getFileByID(fileID)`. ### Records and change notifications -The GUI-facing records hold no key material and no binary, so they survive -`structuredClone`/JSON across the Electron IPC boundary: +The records hold no key material and no binary, so they survive +`structuredClone` and JSON, for example between processes: -- `PhotoRecord`: `fileID`, `albumIDs`, `title`, `takenAt` and `modifiedAt` - (milliseconds), `fileType`, optional `caption` / `width` / `height` / - `latitude` / `longitude`, optional `hash` (the content hash recorded at - upload; very old files have none), `isArchived`, `isHidden`, and - `thumbnailPath` / `originalPath` once the bytes are cached (for a live photo, - `originalPath` is its image). -- `AlbumRecord`: `collectionID`, `name`, `type`, `isShared`, `updationTime`, and - `fileIDs` (newest first). -- `LibrarySnapshot`: `{ albums, photos, takenAt }`. +- `PhotoRecord` holds: + - `fileID` and `albumIDs`; + - `title`; + - `takenAt` and `modifiedAt`, in milliseconds; + - `fileType`, `isArchived` and `isHidden`; + - optional: `caption`, `width`, `height`, `latitude` and `longitude`; + - optional: `hash`, the content hash recorded at upload, which very old + files lack; + - optional: `thumbnailPath` and `originalPath`, once the bytes are cached. A + live photo's `originalPath` is its image. +- `AlbumRecord` holds `collectionID`, `name`, `type`, `isShared`, `updationTime` + and `fileIDs`, newest first. +- `LibrarySnapshot` is `{ albums, photos, takenAt }`. Here `takenAt` is when the + snapshot was taken, in milliseconds. -`lib.snapshot()` returns a `LibrarySnapshot` (albums newest-updated first, -photos newest first). `lib.subscribe({ onChange })` delivers a `LibraryChange` -(`albumsChanged`, `photosChanged`, `fileIDsRemoved`, `albumIDsRemoved`, -`refreshedAt`) whenever a refresh alters the projection, and returns -`{ unsubscribe }`; a refresh that changes nothing delivers nothing. +`lib.snapshot()` returns a `LibrarySnapshot`, albums newest updated first and +photos newest first. `lib.subscribe({ onChange })` delivers a `LibraryChange` +whenever a refresh changes the records, and returns `{ unsubscribe }`. A +`LibraryChange` holds `albumsChanged`, `photosChanged`, `fileIDsRemoved`, +`albumIDsRemoved` and `refreshedAt`. A refresh that changes nothing delivers +nothing. -### Thumbnails, ML search, and backup +### Thumbnails, machine-learning data and backup -- `lib.thumbnails.ensure({ fileIDs, priority, signal?, onProgress? })` - prefetches thumbnails through the thumbnail pool, deduped by fileID, returning - one `EnsureResult` (`{ fileID, path?, error? }`) per file. `priority` is - `"visible" | "ahead" | "background"`; only `"visible"` preempts background - work. -- `lib.mldata` searches the CLIP index built from Ente's per-file ML data: - `forFile({ fileID })` → `Promise<MLData | undefined>` (the whole stored - payload — face boxes, landmarks, embedding — read from disk on demand); - `similar({ fileID, limit? })` and `searchByEmbedding({ embedding, limit? })` → - `SimilarResult[]` (`{ fileID, score }`, cosine similarity, most similar first, - default limit 20). quak bundles no text encoder, so `searchByEmbedding` takes - a query vector the caller produced elsewhere. -- `await lib.backup(opts?)` → `BackupResult`. It takes the lock in the download - directory, and fails at once with an error whose `code` is `ELOCKED` while - another backup of it runs. A caller can instead take the lock itself, before - it opens its library, as `quak backup` does: `await lockBackupDirectory(dir)` - takes it, failing the same way, and returns a `BackupLock`, whose `release()` - removes the lock unless it was lost and whose `lost` holds why it was lost. - The caller passes it to the backup as `lock`, and releases it once the backup - ends. It waits for a refresh as `fresh()` does, puts every in-scope original - not already at its save path there as `photo.download()` does (and, with - `includeThumbnails`, fetches thumbnails) through the content cache, waits for - an ML data fetch, and rebuilds the on-disk backup tree with a durable failure - ledger. Each file's JSON holds its ML data and keywords. An image's JSON also - holds the EXIF and XMP of its original, and its dimensions if it is a JPEG. A - live photo's JSON holds these for its image. A video's JSON holds no EXIF, XMP - or dimensions. A fetched original is written straight to its save path and not - into the cache, which then counts it as present; one the cache already held is - copied from there. `BackupOptions`: `downloadDirectory` (falls back to the - library's), `includeOriginals` (default `true`), `includeThumbnails` (default - `false`), `onlyAlbumNames`, `verify` (default `false`), `onProgress`, and - `lock`. See Backup layout above for the tree it writes, and for the albums - `BackupResult` lists in `undecryptableAlbums` and `failedAlbums`. Whether the - backup took the lock or its caller did, once the lock is lost the backup - rejects with `lost the lock on <dir>: <reason>` before it starts another file. +`lib.thumbnails.ensure({ fileIDs, priority, signal?, onProgress? })` fetches +thumbnails ahead of need through the thumbnail pool, each file once, and returns +one `EnsureResult` per file: `{ fileID, path?, error? }`. `priority` is +`"visible"`, `"ahead"` or `"background"`. Only `"visible"` goes ahead of +background work; `"ahead"` is treated as `"background"`. + +`lib.mldata` searches the CLIP index built from Ente's per-file machine-learning +data: + +- `forFile({ fileID })` returns `Promise<MLData | undefined>`, the whole stored + payload (faces, landmarks and embedding), read from disk on demand. `MLData` + is a plain object type and is not exported by name. +- `similar({ fileID, limit? })` and `searchByEmbedding({ embedding, limit? })` + return `SimilarResult[]`, each `{ fileID, score }`, most similar first. The + score is the cosine similarity, from -1 to 1 (see "Keywords"). `limit` + defaults to 20. quak bundles no text encoder, so `searchByEmbedding` takes a + query vector the caller produced elsewhere. + +`await lib.backup(opts?)` returns a `BackupResult` and writes the tree described +under "The backup directory". It takes the lock in the download directory and +fails at once, with an error whose `code` is `ELOCKED`, while another backup of +it runs. Then it waits for a refresh as `fresh()` does. It puts the original of +every file in the albums it backs up at its save path, as `photo.download()` +does. It waits for a machine-learning fetch, and rebuilds the JSON files and +symlinks. Once the lock is lost, it rejects with +`lost the lock on <dir>: <reason>` before it starts another file. + +`BackupOptions`: + +- `downloadDirectory`: where the tree lives. Default: the library's. +- `includeOriginals`: fetch and store originals. Default `true`. With `false`, + the backup also fetches no machine-learning data, and writes no JSON beside + the originals and no symlinks. +- `includeThumbnails`: also store thumbnails as `thumbnails/<fileID>.jpg` beside + `collections/`. Default `false`. `quak backup` does not set it. +- `onlyAlbumNames`: back up only the albums with these names. +- `verify`: as `--verify`. Default `false`. +- `onProgress`: called with each progress line and, for a line that reports a + failure, the level `"error"`. +- `lock`: a lock the caller took itself. + +A caller can take the lock before it opens its library, as `quak backup` does: +`await lockBackupDirectory(dir)` takes it, failing the same way, and returns a +`BackupLock`. Its `release()` removes the lock unless it was lost, and its +`lost` holds why it was lost. The caller passes it to the backup as `lock` and +releases it once the backup ends. ### Request pools -`RequestPools` holds three independent bounded pools — metadata (10), content -(5), thumbnails (25) — because Ente meters these traffic classes differently. -Each pool orders on-demand work ahead of background/precache work and dedups -in-flight fetches by key, and an idle pool never lends its slots to a busy one. +`RequestPools` holds three independent pools, each with its own limit on how +many requests run at once, because Ente meters these three kinds of traffic +differently: -`lib.backup()`, and so `quak backup`, works on up to the content pool's limit of -originals at once, 5 unless the library was opened with other `pools`. The limit -covers everything it does for one original, hashing it for `verify` included, so -however large the account is, no more originals than that are open at once. +- metadata, with 10 slots: only the machine-learning data requests; +- content, with 5: originals; +- thumbnails, with 25: thumbnails. -### On-disk cache layout +Only the requests named above go through the pools. A refresh, +`freshCollections()` and `freshFiles()` send theirs outside them. Construct one +with +`new RequestPools({ metadataConcurrency, contentConcurrency, thumbnailConcurrency })` +and pass it as `pools`. Each pool serves on-demand work ahead of background +work. Two tasks given to `BoundedPool.run()` with the same `key` run once while +the first is pending, and both get its result. An idle pool never lends its +slots to a busy one. -Under `cacheDirectory`: +`lib.backup()` works on as many originals at once as the content pool allows, 5 +by default. `quak backup` uses the default. The limit covers everything it does +for one original, hashing for `verify` included. -``` -<cacheDirectory>/ - metadata.json decrypted account state + refresh cursor - originals/<fileID>.<ext> cached full-resolution files - originals/<fileID>.livephoto.json - which of a live photo's two files is which - thumbnails/<fileID>.jpg cached thumbnails - mldata/ - <fileID>.json one decrypted ML payload per file - clip.f32, clip.json the packed CLIP index and its id list - fetched.json per-file fetch bookkeeping -``` +### Retries and timeouts -When `metadata.json` belongs to a different account than the client's, -`Library.open` deletes it and `mldata/` and starts from an empty cache. Cached -originals and thumbnails are kept; they are reached only through the files the -current account's records name. +Every request in the library goes through one policy. A request is repeated only +when repeating it could produce a different answer: -A live photo's original is cached as at its save path: its image and its video, -each `originals/<fileID>.<ext>` with its own extension, and -`originals/<fileID>.livephoto.json` naming them; the two are evicted together. +- An `ApiError` with a 5xx status is retried. So are `408` and `429`, the two + 4xx codes that are statements about timing rather than about the request. +- Every other 4xx is not retried. A 404 in particular is an answer, and + `list-missing-thumbnails` depends on getting it promptly and once. +- A transport failure is retried: a rejected `fetch`, `ECONNRESET`, `ETIMEDOUT`, + a DNS or TLS failure, and one of quak's own deadlines. Node's `fetch` rejects + with a plain `TypeError`, so every `TypeError` is retried. The error code is + looked for along the error's `cause` chain, which is where Node's `fetch` puts + it. +- A truncated download is retried. +- Anything else is not retried. This includes an authentication failure in the + encrypted stream that is not truncation. -A stored file appears only via an atomic temp-then-rename, so its presence means -it is complete. Every downloaded original (by `quak get`, the cache, or -`backup`) whose metadata records a content hash (`FileMetadata.hash`) is hashed -as it is written: unkeyed BLAKE2b with a 64-byte output, standard base64. A live -photo arrives as a ZIP and is unpacked as it is written; its image and its video -are hashed separately and joined as `<imageHash>:<videoHash>`, and neither is -stored unless both are complete and match. A live photo ZIP holding a second -image or video, or whose two files together come to more than 20 times its size -plus 16 MiB, is refused. A mismatch or a refusal stores nothing and fails the -download with an error naming the file ID. An original with no recorded hash, -from a very old client, is stored unchecked. +The binary that `make install` builds runs under bun. Bun's `fetch` rejects with +a plain `Error` that has no `cause`. That error carries a code of bun's own. +quak retries a refused connection, a name that does not resolve, a reset +connection and a deadline under bun as it does under Node. Bun reports a refused +connection, and a failed TLS handshake, as `ConnectionRefused`. For a name that +does not resolve it gives `ConnectionRefused` on some requests and +`FailedToOpenSocket` on others. A connection reset or closed before the response +ended is `ECONNRESET`, as under Node, and a deadline is the same `TimeoutError`. +A certificate bun rejects, with a code such as `DEPTH_ZERO_SELF_SIGNED_CERT`, is +not retried. + +Backoff is exponential with full jitter. The delay before retry _n_ is +`random() * min(maxDelayMs, baseDelayMs * 2 ** (n - 1))`. The exponential term +is the ceiling, and the wait is drawn at random below it. So a client that lost +many parallel downloads to one blip does not send them all again at the same +instant. The numbers are set through `ApiClientOptions.retry`: + +- `attempts`: total calls, not retries. Default `4`; `quak backup` uses `10`. +- `baseDelayMs`: the ceiling for the first retry's delay. Default `500`; + `quak backup` uses `1000`. +- `maxDelayMs`: the upper bound on that ceiling. Default `10000`; `quak backup` + uses `60000`. + +With the defaults, a request that keeps failing gives up after at most three and +a half seconds of waiting. `quak backup` usually runs from cron with nobody +watching, so every request it makes uses its own numbers, exported as +`UNATTENDED_RETRY_OPTIONS`. Under them, a request that keeps failing gives up +after at most 243 seconds of waiting. It usually gives up after about half that, +since each wait is drawn at random below its ceiling. Every other command uses +the defaults. A library user gets the backup's budget by passing +`UNATTENDED_RETRY_OPTIONS` as `ApiClientOptions.retry`. `sleep` and `random` can +be injected through the same option, which is how the test suite exercises the +policy without waiting. `onRetry`, given there too, is called before each wait +with the error, the attempt's number and the wait. + +Two deadlines apply, each renewed for every attempt: + +- `requestTimeoutMs`, default `30000`, bounds a whole JSON request or upload. +- `downloadTimeoutMs`, default `60000`, is an idle deadline on a file or + thumbnail download: the download is aborted when no bytes have arrived for + that long. + +They differ because a download's length depends on the file and the link. A +whole-transfer deadline short enough to catch a hung connection would cancel a +large video on a slow link that is still making progress. The download deadline +restarts each time bytes arrive, so a slow download runs as long as it keeps +moving, and one that stalls is aborted after 60 seconds of silence. The download +deadline covers the wait for the headers as well as the body. There is no limit +on the total length of a download. + +A request that changes state is not blindly replayed. Every `POST` and `PUT` in +the endpoint list under "HTTP API" goes through this rule. Verifying a second +factor, for one, uses up one of a small number of attempts. Such a request is +retried only when every error code in its `cause` chain proves that no TCP +connection to the server ever existed. Three codes prove that: + +- `ENOTFOUND`: name resolution found no address; +- `EAI_AGAIN`: name resolution failed for the moment; +- `ECONNREFUSED`: the peer refused the connection. + +A 5xx, a reset mid-flight and a deadline are all left to the caller, because +each of them can happen after the server has already acted. These requests do +not follow redirects either: a redirect means the server already received the +request, so it is reported as an error and not retried. + +The routing errors `EHOSTUNREACH`, `ENETUNREACH` and `ENETDOWN` are left out of +that rule although they look like connect-time failures. On Linux they are also +delivered on an established socket, after the request may have been written. +They stay retryable for requests that change nothing. Bun's `ConnectionRefused` +and `FailedToOpenSocket` are treated the same way: a request that changes +nothing is retried on them, and a `POST` or `PUT` is not replayed on them. Bun +reuses a connection it kept open from an earlier request. When that connection +closes before any answer, bun sends the request again on a new one. If that one +fails, bun gives one of these codes although the server may already have the +request. The upload of a thumbnail to its presigned URL is exempt from the rule. +It stores one whole object at one key in one request, so replaying it has no +partial state to damage. + +A download is retried as a whole: the request, the reading of the body and the +decryption. A socket reset after the response headers have arrived surfaces in +the download layer rather than in the HTTP client. That is the common failure +for a large file over a CDN. The encrypted stream cannot be resumed and the +endpoints have no Range support, so a retry starts the file over. Each attempt +writes its own temporary files and removes them if it fails. Only the attempt +that completes renames anything into place. + +One imprecision is deliberate. When a body ends partway through an encrypted +chunk, the authentication check fails and carries no sign of why, so a cut +connection and corrupt bytes look the same. quak reports that as truncation, +which means it is retried. quak can tell the two apart only in a body of more +than one chunk. There, a chunk that fails while the stream carries on past it +stays an authentication failure and is not retried. Most thumbnails and every +small file are one chunk. There a wrong key, corrupt bytes and a cut connection +look alike, and all are retried. The attempt count limits the extra retries. In +return, quak never keeps a truncated file. + +### The `Client` + +`Client.login({ email, password, totp?, emailOTP?, apiOptions? })` logs in and +resolves to a `Client`. `totp` and `emailOTP` are callbacks that return the code +when the account asks for one. Without the callback the account needs, login +throws. A passkey challenge always throws. After an email code, login expects +the key attributes and the token. When the server asks for a second factor +instead, login throws. `apiOptions` configures the HTTP client (see below). + +The `Client` holds the session token, the master key, the secret key and the +public key in memory. There is no on-disk session store in the library; the +caller decides how to keep a session. `client.toJSON()` returns a +`ClientSnapshot`, a plain object with the keys in base64, which the caller can +write wherever it likes. `Client.fromJSON(snapshot, apiOptions?)` restores a +working client from that snapshot without logging in again. It checks every +field and each key's length first, and throws an error naming the bad field. + +`Client.fromJSON` works only once libsodium, the cryptography library quak uses, +is loaded. `Client.login` loads it, and `Client.fromJSON` does not. A program +that restores a session in a new process first runs `await sodium.ready`, with +`sodium` imported from `libsodium-wrappers-sumo`, which quak depends on. The +exported `beginLogin`, `unwrapAuth`, `decryptCollection`, `decryptFile`, +`downloadFile` and `downloadThumbnail` need the same. + +`client.logout()` clears the token and zeroes the key buffers in place. After +it, every other method on that client throws. `logout()` does not contact the +server, so the token stays valid there and in any saved snapshot. Call +`await client.logoutOnServer()` before `logout()` to end the session on the +server. + +`client.whoami()` returns `{ email, userID }`. + +`client.collectionsSince({ sinceTime })` and +`client.filesSince({ collectionID, collectionKey, sinceTime })` enumerate the +changes since a microsecond timestamp, with `0` for everything. Each returns: + +- `collections` or `files`: the decrypted records that are not deleted; +- `deleted`: the IDs of deleted records; +- `undecryptable`: each record that did not decrypt, as `{ id, error }`; +- `cursor`: the timestamp to pass next time. + +`client.listCollections()` and `client.listFiles(collectionID, collectionKey)` +return only the decrypted records that are not deleted, from `sinceTime` `0`. +`client.fetchMLData({ fileIDs, fileKeys })` fetches and decrypts the +machine-learning data of up to 200 files. `client.downloadFile(file, outPath?)` +and `client.downloadThumbnail(file, outPath?)` download and decrypt one file to +`outPath`. For a live photo, `downloadFile` writes the image and the video, each +named `outPath` with its own extension. `quak get --out` instead puts the image +at the path itself. Without `outPath`, they write to the working directory under +the names `quak get` and `quak get-thumb` would use. + +`ApiClientOptions`: + +- `apiOrigin`: the API server. Default `https://api.ente.io`. A self-hosted + server is set here; file and thumbnail downloads then go through it too. +- `filesOrigin` and `thumbsOrigin`: the download hosts used with the default API + server. +- `authToken`: a session token to send from the start. +- `fetch`: a replacement for the global `fetch`. +- `retry`, `requestTimeoutMs` and `downloadTimeoutMs`: see "Retries and + timeouts". +- `onRequest`: called once for each request sent, and once for each attempt of a + retried one. It gets the method, the URL and the status of the answer or the + error. The URL of a thumbnail upload is given without its query, which holds + the upload's signature. + +The package also exports the pieces the `Client` and the `Library` are built +from, and a few constants: + +- the HTTP client: `ApiClient`, `ApiError` and `TruncatedStreamError`, with + `DEFAULT_REQUEST_TIMEOUT_MS` and `DEFAULT_DOWNLOAD_TIMEOUT_MS`; +- the retry policy: `withRetry`, `isRetryable`, `isSafeToReplay`, + `resolveRetryOptions`, `DEFAULT_RETRY_OPTIONS` and `UNATTENDED_RETRY_OPTIONS`; +- the login steps: `beginLogin`, `submitTOTP`, `requestEmailOTP`, + `submitEmailOTP` and `unwrapAuth`; +- the record decryption: `decryptCollection` and `decryptFile`; +- the downloads: `downloadFile` and `downloadThumbnail`; +- the backup: `runBackup` and `lockBackupDirectory`; +- the pools: `RequestPools` and `BoundedPool`, with + `DEFAULT_METADATA_CONCURRENCY`, `DEFAULT_CONTENT_CONCURRENCY` and + `DEFAULT_THUMBNAIL_CONCURRENCY`; +- the library: `Library`, `Album`, `Photo` and + `DEFAULT_REFRESH_INTERVAL_SECONDS`; +- `VERSION`: the version string `quak --version` prints. ### Key types by source file @@ -1408,8 +1527,8 @@ from a very old client, is stored unchecked. - `src/library/records.ts`: `PhotoRecord`, `AlbumRecord`, `LibrarySnapshot`, `LibraryChange` - `src/library/mlsearch.ts`: `MLDataAPI`, `SimilarResult` -- `src/exif.ts`: `ExifTags`, `PhotoExif` - `src/library/pools.ts`: `RequestPools`, `RequestPoolsOptions`, `BoundedPool` +- `src/exif.ts`: `ExifTags`, `PhotoExif` - `src/backup.ts`: `BackupLock`, `BackupOptions`, `BackupResult`, `BackupError` - `src/client.ts`: `Client`, `LoginOptions`, `ClientSnapshot` - `src/api/client.ts`: `ApiClient`, `ApiClientOptions`, `StreamOptions` @@ -1417,62 +1536,419 @@ from a very old client, is stored unchecked. - `src/retry.ts`: `withRetry`, `isRetryable`, `isSafeToReplay`, `RetryOptions` - `src/model/types.ts`: `Collection`, `EnteFile`, `FileMetadata`, `FileType`, `CollectionType`, `RawCollection`, `RawEnteFile` -- `src/thumbnails.ts`: `MissingThumbnailInfo`, `ThumbnailFixResult` + +`src/index.ts` lists what the package exports. `MLDataAPI` and `SimilarResult` +are the types of `lib.mldata` and of its results, and `MLData` is the type of a +stored payload. None of the three is exported by name. + +## Examples + +`examples/download-albums.ts` downloads every album's photos and their metadata +into a directory, `photos` in the working directory unless you name another. It +uses only the library's public API. The build compiles it; run it after +`yarn install`: + +```bash +yarn build +QUAK_EMAIL=… QUAK_PASSWORD=… node dist/examples/download-albums.js [dir] +``` + +It opens the library with `precacheThumbnails` and `precacheOriginals` off, as +`quak backup` does, so the only file content it fetches is the originals it +saves. It asks on the terminal for a second-factor code or an email code when +the account requires one. It writes: + +- each photo's original at its save path under `dir`, as `photo.download()` + writes it, and for a live photo its image, its video and the `.livephoto.json` + file naming them; +- beside each original, a JSON file named after it with `.json` added, such as + `2026-03-01.12345.jpg.json`. It holds the photo's record without its cache + paths, and under `exif` every EXIF tag of the photo; +- `albums/<collectionID>.json` for each album, holding its `collectionID`, its + `name` and, under `savePaths`, the save paths of its photos relative to `dir`, + newest first. + +A photo in several albums is downloaded once. A second run downloads nothing and +rewrites only the JSON files whose content changed. A failed download stops the +run; running it again carries on, since every photo already saved is skipped. + +## Rationale + +Ente encrypts photos end to end, so a backup of an Ente account cannot be made +from the server's side. Only a client that holds the account's keys can read the +photos. Ente's own clients are apps and a command-line tool; none is a library a +program can use. quak is the client for scripts and for cron. + +quak is a library first, so that a program of your own can read an account the +way quak's command line does. The backup it writes is plain files in a tree a +photo viewer can open, with no quak needed to read it. One failed file never +ends a run; it is recorded and tried again next time. + +## Design + +quak is a TypeScript library with a thin command-line wrapper. The library does +the work; the command line is for people and cron. Login uses SRP, a protocol +that proves the password to the server without sending it. + +### Layout + +``` +quak/ + src/ + crypto/ libsodium primitives: boxes, streams, key derivation, + hashing + api/ the HTTP client (ApiClient) + auth/ the login flow (SRP, email code, TOTP) and key unwrap + model/ the decrypted Collection, EnteFile and FileMetadata + types, and the functions that decrypt them + download/ streaming download and decryption of files and + thumbnails + library/ the Library over the cache: metadata store, reads, + records, content cache, background downloads, + machine-learning data and search, request pools + backup.ts the full-account backup + metadata-backup.ts + backup-metadata: the metadata quak keeps, as JSON + exif.ts EXIF read from an image's bytes with exifreader + mldata-fetch.ts fetch and decrypt per-file machine-learning data + clip-keywords.ts + a file's keywords from its CLIP embedding + clip-keyword-vectors.ts + Ente's Discover categories with their text + embeddings, written by bin/clip-keyword-vectors + filename.ts safe file names from server metadata + errors.ts error types shared across layers, and how an error + is described + retry.ts the retry classifier and the backoff loop + thumbnails.ts find and regenerate missing thumbnails + client.ts the Client class assembled from the above + cli-commands.ts the command line's commands as functions returning + exit codes + cli-log.ts the command line's log level and the one function + its commands write stderr lines through + cli-output.ts how the command line prints a file's title and time + cli-read.ts how the read commands get current data + cli-run.ts run a command, print its error, exit with its code + cli-session.ts read the saved session file back into a Client + index.ts the package's exports + bin/ + quak.ts the command-line entrypoint (commander.js) + clip-keyword-vectors, clip-keyword-vectors.py + write src/clip-keyword-vectors.ts and a test + fixture (see "Source attribution") + examples/ + download-albums.ts + download every album's photos and metadata + test/ unit and integration tests (vitest) + script/ the entrypoints (see "Entrypoints") + Makefile + Dockerfile lint phase, test phase, compile + package.json + tsconfig.json +``` + +`make build` compiles `src/`, `bin/` and `examples/` into `dist/`, keeping their +shape: the library lands in `dist/src/`, the examples in `dist/examples/` and +the command line in `dist/bin/quak.js`. `package.json` points `main` and `types` +at `dist/src/index.js` and `dist/src/index.d.ts`, and `bin` at +`dist/bin/quak.js`. The compiler's `rootDir` is the repository root rather than +`src/`, because `bin/` and `examples/` are compiled too. + +### Cryptography + +All cryptography is done by `libsodium-wrappers-sumo`, with two exceptions. The +SRP handshake uses `fast-srp-hap`, and the MD5 checksum sent with a thumbnail +upload uses Node's `node:crypto`. The "sumo" build is required for +`crypto_pwhash`, the Argon2id function. No cryptography is written by hand. + +Login derives the keys in this order: + +1. The user enters the password. +2. Argon2id (`crypto_pwhash`) derives a 32-byte key encryption key, the KEK, + from the password. It uses the server-issued `kekSalt`, `memLimit` and + `opsLimit`. +3. For SRP login, `crypto_kdf_derive_from_key` (BLAKE2b) derives a 32-byte + subkey from the KEK with subkey ID 1 and context `loginctx`. Its first 16 + bytes are the SRP password. +4. When SRP completes, the server returns the key attributes and an encrypted + token, or first asks for a second factor. quak answers a TOTP request with + the code (`POST /users/two-factor/verify`); the server then returns the key + attributes and the token. quak cannot answer a passkey request. When the + account has email verification on (`isEmailMFAEnabled`), an email code + replaces SRP. quak then expects the key attributes and the token in the + answer, and fails the login when the server asks for a second factor instead. +5. `crypto_secretbox_open_easy` over the encrypted master key with the KEK + yields the 32-byte master key. +6. `crypto_secretbox_open_easy` over the encrypted secret key with the master + key yields the user's X25519 private key. The matching public key arrives in + cleartext. +7. `crypto_box_seal_open` over the encrypted token with the user's key pair + yields the token's bytes. Encoded as URL-safe base64 with padding, they are + the `X-Auth-Token` value for every later call. + +A collection the account owns carries its key as a secretbox under the master +key. A collection shared with the account carries it as a sealed box to the +account's public key. Each file's key is a secretbox under its collection's key. +File metadata is a secretstream blob, a single chunk with the final tag, under +the file key. File content is a chunked `crypto_secretstream_xchacha20poly1305` +stream under the file key, with 4 MiB plaintext chunks and 17 bytes of +authentication overhead per chunk. A thumbnail is such a stream too. For a +thumbnail upload, `encryptBlob` does the push side: one chunk with the final +tag, returning the header and the ciphertext. + +### HTTP API + +The production endpoints are: + +- API: `https://api.ente.io` +- File downloads: `https://files.ente.io/?fileID=<id>` +- Thumbnail downloads: `https://thumbnails.ente.io/?fileID=<id>` + +A self-hosted server is set through the `apiOrigin` option. File downloads then +go through `<apiOrigin>/files/download/<id>` and thumbnail downloads through +`<apiOrigin>/files/preview/<id>`. + +Every authenticated call carries two headers: `X-Auth-Token`, the token from +login, and `X-Client-Package`, which names the client as `berlin.sneak.quak`. + +Endpoints used: + +- `GET /users/srp/attributes?email=<email>`: fetch the SRP and key-derivation + parameters. +- `POST /users/srp/create-session`: begin the SRP handshake. +- `POST /users/srp/verify-session`: complete SRP; receive the second-factor + challenge or the encrypted token with the key attributes. +- `POST /users/ott` and `POST /users/verify-email`: the email code, used instead + of SRP when the SRP attributes have `isEmailMFAEnabled` set. +- `POST /users/two-factor/verify`: the TOTP second factor. +- `POST /users/logout`: end the calling token's session (`quak logout`). +- `GET /collections/v2?sinceTime=<usec>`: list the collections changed since a + microsecond timestamp; `0` lists them all. +- `GET /collections/v2/diff?collectionID=<id>&sinceTime=<usec>`: list the files + changed in a collection; quak pages while `hasMore` is true. +- `GET https://files.ente.io/?fileID=<id>`: download a file's encrypted bytes. +- `GET https://thumbnails.ente.io/?fileID=<id>`: download a thumbnail's + encrypted bytes. +- `POST /files/data/fetch`: fetch the encrypted machine-learning data of a batch + of files. +- `POST /files/upload-url`: get a presigned upload URL, for thumbnail repair. +- `PUT /files/thumbnail`: register an uploaded thumbnail's object key. ## Source attribution -The cryptographic protocol and wire format implemented here are Ente's, taken -from the Ente open source clients at <https://github.com/ente-io/ente>. No code -is imported or vendored from those projects; any reference code that is copied -is rewritten in TypeScript in this repository. Protocol fidelity is verified -against the upstream implementations in `web/packages/base/`, -`mobile/apps/photos/lib/`, and `cli/`. +The cryptographic protocol and wire format are Ente's, read from the Ente +clients at <https://github.com/ente-io/ente>. No code from those projects is +included; quak's implementation is written in TypeScript from reading them. The keywords come from Ente too. `src/clip-keyword-vectors.ts` holds the Discover categories of Ente's photos app, from -`mobile/apps/photos/assets/discover.json`, each with its query's text embedding -from Ente's CLIP text model. `bin/clip-keyword-vectors` writes that file and the -CLIP embedding of a public-domain photo that the tests use. It runs Ente's -models in Docker, with every download pinned by sha256. It is run by hand only, -followed by `make fmt`. +`mobile/apps/photos/assets/discover.json`, each with the text embedding of its +query from Ente's CLIP text model. `bin/clip-keyword-vectors` writes that file, +and the CLIP embedding of a public-domain photo that the tests use. It runs +Ente's models in Docker, with every download pinned by SHA-256 hash. It is run +by hand only, followed by `make fmt`. -## For LLMs +## Entrypoints -If you are an LLM agent working on this repository, read and follow these -documents: +This repository adheres to the +[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all) +standard. Normalized scripts in `script/` are the entrypoints for the +development workflow, and most Makefile targets are thin shims that call them. +The scripts are POSIX `sh`, not bash, so they run in minimal containers such as +Alpine. Of these, only `bootstrap`, `setup`, `test` and `cibuild` come from the +standard. The others are additions to it. -- **`REPO_POLICIES.md`** in the repo root. It is copied from +- `script/bootstrap`: install all dependencies. It installs `make`, `git`, Node + and `yarn` when they are missing, then runs `yarn install --frozen-lockfile`. +- `script/setup`: set up the repo for development after a fresh clone. It runs + `script/bootstrap`, then `script/install-precommit`. +- `script/projectname`: print the project name. `script/docker` uses it for the + image tag. +- `script/build`: compile the TypeScript sources into `dist/` and stamp the + version into `dist/package.json`. Then check that the entrypoints + `package.json` declares (`main`, `types`, `bin`) were written, and make the + command line executable. +- `script/version`: print the version `script/build` stamps (see "Version"). +- `script/test`: run the test suite, by building the `test` phase of the + `Dockerfile`. Requires Docker. +- `script/lint`: run eslint and a prettier check, by building the `lint` phase + of the `Dockerfile`. Requires Docker. +- `script/fmt`: format all files with prettier. It rewrites files in place. +- `script/fmt-check`: check formatting on the host. It is read-only. +- `script/check`: run all checks: `test`, `lint`, `fmt-check`. +- `script/docker`: build the image, tagged with `script/projectname`. +- `script/cibuild`: what CI runs: `script/bootstrap`, then `script/check`, then + the image build. +- `script/precommit`: run by the git pre-commit hook. It runs `script/lint` and + `script/fmt-check`, and not the tests, so that a commit of failing tests can + land (see "Development workflow"). +- `script/install-precommit`: install the git pre-commit hook. `make hooks` + calls it. + +The Makefile targets are: + +- `make bootstrap`: runs `script/bootstrap`. +- `make setup`: runs `script/setup`. +- `make test`: runs `script/test`. +- `make lint`: runs `script/lint`. +- `make fmt`: runs `script/fmt`. +- `make fmt-check`: runs `script/fmt-check`. +- `make check`: runs `script/check`. +- `make build`: runs `script/build`. +- `make docker`: runs `script/docker`. +- `make hooks`: runs `script/install-precommit`. +- `make build-bin`: bundles the built `dist/` into one binary, `bin/quak`. It + runs bun through `nix-shell -p bun`. +- `make install`: runs `make build-bin` and copies the binary to `~/bin/quak`. +- `make dev`: runs the TypeScript compiler in watch mode. +- `make clean`: removes `dist/`, `coverage/`, `.vitest-cache/` and the + compiler's `*.tsbuildinfo` files. + +### Linting and testing + +Linting and testing are phases of the `Dockerfile`. The `lint` phase copies the +repo into a node image pinned by digest and runs eslint and +`prettier --check .`. The `test` phase runs the test suite the same way, as the +image's non-root `node` user. A 90-second `timeout` stops a hung test. A suite +that fails is run again with verbose output. `script/lint` and `script/test` +each build one phase with `docker build --no-cache --target <phase>`. Neither +script runs the tools on the host, so Docker is required. The scripts also work +where the Docker daemon is remote, because they copy the repo into the image +instead of sharing a directory with the container. + +The last stage of the `Dockerfile` compiles the package. It copies a file from +each phase, so it cannot be built unless lint and the tests pass. The image +build in `script/cibuild` therefore runs lint and the tests a second time, after +`script/check` has run them. + +Every `docker build` in `script/` passes `--no-cache`. On an unchanged tree +Docker would otherwise serve the lint and test steps from its cache, nothing +would run, and the build would still exit 0. + +`script/fmt-check` runs prettier on the host. Its verdict matches the `lint` +phase's, because prettier is pinned to one version and installed from +`yarn.lock` in both places. Prettier reads `.gitignore` as its default ignore +file, which is why `.dockerignore` keeps `.gitignore` in the build context. + +### Version + +`quak --version` reports the `version` in `dist/package.json`, which +`script/build` writes after compiling. The repo's own `package.json` keeps +`0.0.0`. The tests run from source, so they report `0.0.0`. `script/version` +decides what is written: + +- the `VERSION` environment variable, or the `Dockerfile`'s `VERSION` build + argument (`--build-arg VERSION=...`), when one is given and not empty; +- otherwise, in a checkout with `.git`, what `git describe --tags --always` + prints. That is the tag on a tagged commit. On a later commit it is the tag, + the number of commits since it and the short commit, as in + `v1.2.3-4-gabc1234`. When no tag is reachable it is the short commit; +- otherwise, as in a source tarball, the version `package.json` declares. + +The build fails if the checkout has `.git` and the version still comes out +empty, `dev` or `unknown`, because such a build could not be traced to its +commit. + +`.dockerignore` therefore does not leave out `.git`, so any `docker build .` of +a clone stamps the commit it was built from. A clone of depth 1 stamps a tag +only when its commit carries one, and otherwise the short commit. +`.dockerignore` does leave out `.git/config`, which holds the clone's remote URL +and any credential in it; `git describe` does not need that file. +`script/docker` and `script/cibuild` work out the version on the host, with +`--dirty`, and pass it as the `VERSION` build argument. That argument takes +precedence. The image's `org.opencontainers.image.version` label carries that +build argument only, so a build given none leaves it empty. `make build-bin` +bundles the built `dist/`, so the single binary reports the stamped version too. + +## Development workflow + +All work on quak is test-driven. + +1. Every change starts on a feature branch off `next`, and its pull request + targets `next`. +2. The first commit on the branch is the tests for what is being added or + changed. Those tests fail at that commit; the branch is red until the + implementation lands. +3. Later commits add the implementation and any refactors needed to make the + tests pass. +4. A pull request is merged into `next` only when `make check` is green. Once it + has passed review, a maintainer squash-merges it into `next`. Only sneak + merges `next` into `main`. `main` and `next` are always green. CI runs + `script/cibuild`, which builds the `Dockerfile`: its `lint` and `test` + phases, then the compile. So neither a red branch nor one that does not + compile can pass CI. +5. Tests are the canonical API documentation for this library. Every test file + is commented well enough that a reader who has never seen quak can learn how + to use it from the tests alone. Comments explain why a behavior matters, not + just what the assertion checks. +6. Test fixtures, such as cryptographic vectors, recorded HTTP responses and + sample files, are committed beside their tests. Where possible they are + generated by deterministic helpers in `test/`, so a reviewer can reproduce + them by running the helper. +7. `git rebase -i` is allowed on a feature branch before merge, to clean the + tests-then-implementation sequence into reviewable commits. The final history + must still show tests landing before, or with, the matching implementation. +8. The pre-commit hook that `make hooks` installs runs `script/precommit`, which + runs `script/lint` and `script/fmt-check` and not the tests. This is + deliberate, so that the commit of failing tests can land. CI runs the tests, + so a red branch still cannot reach `next`. + +### Rules for contributors + +These apply to people and to coding agents alike. + +- Read and follow `REPO_POLICIES.md` in the repo root. It is copied from <https://git.eeqj.de/sneak/prompts> and covers repository structure, tooling, - Makefile targets, Dockerfile conventions, dependency pinning, and commit - hygiene. All external dependencies must be pinned by cryptographic hash in - `yarn.lock`. Never `git add -A`. Never force-push to main. + Makefile targets, Dockerfile conventions, dependency pinning and commit + hygiene. Never `git add -A`. Never force-push `main`. +- Dependencies are pinned: + - npm packages by hash, in `yarn.lock`; + - container images by digest; + - the nvm archive `script/bootstrap` downloads, and the Python packages + `bin/clip-keyword-vectors` installs, by SHA-256 hash; + - Node and yarn, when `script/bootstrap` installs them, by version. +- Not pinned: + - bun, which `make build-bin` takes from `nix-shell -p bun`; + - `make` and `git`, which `script/bootstrap` installs from the system + package manager when they are missing, as every `Dockerfile` stage does; + - `bash` and `curl`, which `script/bootstrap` installs the same way when it + has to install Node. +- Follow the workflow above: branches off `next`, tests first and committed + failing, and `next` merged into `main` by sneak alone. +- Before every commit, `make lint` and `make fmt-check` must pass. `make lint` + builds the `lint` phase of the `Dockerfile`, so it needs Docker. The + pre-commit hook enforces exactly that. `make check`, which also runs the + tests, must pass before a merge into `next`. Never invoke eslint or prettier + directly; linting runs in the container only. +- Format with `make fmt`: prettier with 4-space indents and `proseWrap: always` + for markdown. Use `yarn`, not `npm`. +- Tests use vitest and live in `test/`, mirroring `src/`. Use `mkdtempSync` for + temporary directories, never a path built from a timestamp. `make test` must + finish in under 60 seconds. The 90-second `timeout` in the `test` phase of the + `Dockerfile` only catches a hung test. +- Code style: `const` for everything, `let` only when a variable is reassigned, + never `var`. Avoid comments that restate the code. No cryptography written by + hand. `LLM_PROSE_TELLS.md` in the prompts repo applies to any prose written in + this repository: the README, comments and commit messages. -- **The "Development workflow" section above.** All changes go on feature - branches off `next`, and every pull request targets `next`; only sneak merges - `next` into `main`. Tests are written first and committed in a failing state - before the implementation. Tests are the canonical API documentation and must - be commented thoroughly. `main` and `next` are always green. +## TODO -- **Required checks before every commit:** `make lint` and `make fmt-check` must - pass. `make lint` is eslint plus the prettier check, and it builds the `lint` - phase of the `Dockerfile`, so it needs docker. The pre-commit hook enforces - exactly that. `make check` (which also runs the tests) must pass before - merging into `next`. Never invoke eslint or prettier directly; linting runs in - the container only. +`TODO.md` in the repo root is the work log: the next step, the steps done and +the steps planned. Open now: -- **Formatting:** prettier with 4-space indents and `proseWrap: always` for - markdown. Use `make fmt` to format. Use `yarn` not `npm`. +- [ ] Check quak against a live Ente account. This needs an account only sneak + has. -- **Testing:** vitest. Tests go in `test/` mirroring the `src/` structure. - `make test` must finish in under 60 seconds (the hard cap) and should finish - in under 20. The 90-second `timeout` in the `test` phase of the `Dockerfile` - is a backstop that catches a hung test, not the time limit. Use `mkdtempSync` - for temporary directories, never manual timestamp paths. +Planned for a desktop client, in a separate repository, on top of this library: -- **Code style:** `const` for everything, `let` if reassignment is needed, never - `var`. Avoid unnecessary comments. No hand-rolled crypto. The - `LLM_PROSE_TELLS.md` document in the prompts repo applies to any prose written - in this repository (README, comments, commit messages). +- [ ] An Electron app skeleton that uses this library. +- [ ] A local cache in SQLite keyed on `(collectionID, fileID, updationTime)`. +- [ ] A background sync worker that streams new files into the cache. +- [ ] A gallery UI: thumbnails, full-image view, basic search. +- [ ] Upload, delete and share operations in the library. + +Tagging and releases are decided by sneak alone, and happen only when he +declares one. ## License diff --git a/src/cli-log.ts b/src/cli-log.ts index 88916c2..a790c6b 100644 --- a/src/cli-log.ts +++ b/src/cli-log.ts @@ -1,15 +1,18 @@ // The CLI's log level, and the one function the commands write stderr lines // through. // -// Every line quak prints on stderr is a log line at one of six levels, from -// the quietest: fatal, error, warning, info, debug, trace. A run prints the -// lines at its own level and at every quieter one. Its level is info unless -// `-v` or `-q` asks for another (see "CLI surface" in the README). Every -// command that exits non-zero logs a fatal line saying why, so even `-q`, which -// prints only fatal lines, never leaves a non-zero exit unexplained. An error +// Every line quak logs goes to stderr, at one of six levels, from the +// quietest: fatal, error, warning, info, debug, trace. A run prints the lines +// at its own level and at every quieter one. Its level is info unless `-v` or +// `-q` asks for another (see "What quak prints" in the README). A command that +// runs and exits non-zero logs a fatal line saying why, so even `-q`, which +// prints only fatal lines, never leaves such an exit unexplained. An error // line reports something that failed, such as a file or an album, and a // warning something that did not fail but needs the user's attention. stdout -// carries only a command's output and is not a log. +// carries a command's output and the prompts of `login`, and is not a log. +// Two things on stderr are not log lines: the `error:` message for a +// command line that quak cannot parse, and the usage printed when no command +// is given. // // The library itself writes no lines: its layers report through callbacks, and // the CLI logs what they report. `logRequests` gives the API client options diff --git a/test/cli/backup-end-to-end.test.ts b/test/cli/backup-end-to-end.test.ts index 8f075f7..6b2e432 100644 --- a/test/cli/backup-end-to-end.test.ts +++ b/test/cli/backup-end-to-end.test.ts @@ -418,7 +418,7 @@ describe("the first backup of the account", () => { expect(downloads.sort()).toEqual([100, 200, 300]); }); - // The tree "Backup layout" in the README describes. Each original is at + // The tree under "The backup directory" in the README. Each original is at // its save path, `YYYY/YYYY-MM/YYYY-MM-DD/YYYY-MM-DD.<fileID>.<ext>`, with // the extension of its title; the live photo is its image and its video, // each with the extension it had inside the ZIP, and a `.livephoto.json` diff --git a/test/cli/log.test.ts b/test/cli/log.test.ts index d89d929..a407293 100644 --- a/test/cli/log.test.ts +++ b/test/cli/log.test.ts @@ -328,7 +328,7 @@ describe("what the library reports", () => { ]); }); - // Each operation is named in plain words, as the README names it. + // Each operation is named in plain words. it("logs a refresh's progress at info, and its start and end at debug", () => { const { log, lines } = collect(); const onProgress = logRefresh(log); -- 2.54.0 From 67794bf5f825ebbfb21e21aea5a29f7983a25c22 Mon Sep 17 00:00:00 2001 From: clawbot <35+clawbot@noreply.example.org> Date: Mon, 26 Oct 2026 18:59:28 +0100 Subject: [PATCH 6/7] README: four sentences fixed for the documents gate (closes #229) The documents gate on `next` failed five README sentences (https://git.eeqj.de/sneak/quak/issues/219#issuecomment-156230). Four are rewritten after a check against the code: - the keywords score sentence is split in two; - `mlData` drops the whole `clip` block, as `src/backup.ts` and `src/metadata-backup.ts` do; - a file name gets its ID past 255 bytes and a collection name past 250, and either is then cut short, as `uniqueNames` does; - `lastTriedAt` is taken once per run, after its refresh. The IDs sentence keeps `-1`: the pinned option parser passes a negative number to a command with no subcommands as an argument, so `quak get -1` reaches the ID check. Nothing else changes; `make fmt` reflowed the touched paragraphs. Model: fable-5-1 (change); opus-5-5 (landing message) --- README.md | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 4c51a1e..8fa5f17 100644 --- a/README.md +++ b/README.md @@ -692,10 +692,10 @@ Update times are in microseconds, as Ente records them. The JSON also holds the machine-learning data Ente computed for the photo, as `mlData`. Ente's apps detect faces in each photo and compute a CLIP embedding for it, a list of 512 numbers that describes what the photo shows. `mlData` -holds everything but that embedding. In its place the JSON holds `keywords`, -derived from the embedding as described under "Keywords". A file with no -machine-learning data has neither field. A file whose data holds no usable -embedding has no `keywords`. +holds everything but the `clip` block, which holds that embedding. In its place +the JSON holds `keywords`, derived from the embedding as described under +"Keywords". A file with no machine-learning data has neither field. A file whose +data holds no usable embedding has no `keywords`. The backup waits for the library's fetch of machine-learning data to finish before it writes the JSON files. When a file's data cannot be fetched, @@ -748,11 +748,11 @@ derives a file's keywords the way the app sorts it. The categories are: Each category has a query, and a text embedding of that query from Ente's CLIP text model. A file's score for a category is the cosine similarity of the file's -embedding and the category's: a number from -1 to 1, higher the more alike the -two embeddings are. The file falls in the category when its score reaches the -category's minimum score. The keywords are the titles of the categories the file -falls in, highest score first. A video never falls in a category marked "no -videos", as in the app. +embedding and the category's. That is a number from -1 to 1; the more alike the +two embeddings are, the higher it is. The file falls in the category when its +score reaches the category's minimum score. The keywords are the titles of the +categories the file falls in, highest score first. A video never falls in a +category marked "no videos", as in the app. #### Collection directories @@ -775,9 +775,10 @@ ignoring case, each gets its ID added: two albums named `Trip` become `Trip (10)/` and `Trip (11)/`. Two files in one collection whose titles match, ignoring case, get their IDs added before the extension: `IMG_0001 (12345).JPG` and `IMG_0001 (12346).JPG`. A collection named like another collection's JSON -file, such as `Trip.json` beside `Trip`, also gets its ID added. A name too long -for a file name, which most file systems limit to 255 bytes, gets its ID added -too. It is then cut short before the ID, just far enough to fit. IDs never +file, such as `Trip.json` beside `Trip`, also gets its ID added. A file name +longer than 255 bytes, the limit on most file systems, gets its ID added too. So +does a collection name longer than 250 bytes, which leaves room for `.json`. +Either name is then cut short before the ID, just far enough to fit. IDs never change, so a name stays the same from run to run until such a clash appears or goes away. @@ -834,7 +835,8 @@ Each entry holds: - `fileID` and `title`; - `classification`: the kind of failure; - `attempts`: how many runs in a row have failed the file; -- `lastTriedAt`: when it was last tried, as a Unix time in milliseconds; +- `lastTriedAt`: when the run that last tried it finished its refresh, as a Unix + time in milliseconds; - `error`: the error, as quak printed it. The kinds are: -- 2.54.0 From aeb275e1375346fc29af890170466010185393b9 Mon Sep 17 00:00:00 2001 From: clawbot <35+clawbot@noreply.example.org> Date: Mon, 26 Oct 2026 19:32:03 +0100 Subject: [PATCH 7/7] README: the close() sentence split in two (closes #231) The sentence in "Opening a library" had a relative clause with its "that" left out, so "returns resolves" read as two verbs in a row. It is now two sentences; the behaviour described is unchanged and matches `close()` in `src/library/index.ts`. Model: fable-5-1 (change); opus-5-5 (landing message) --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 8fa5f17..57ae839 100644 --- a/README.md +++ b/README.md @@ -1079,10 +1079,10 @@ cached, so the next fetch asks for it again. The reasons are kept in memory only. `lib.close()` stops the background timer. It can be called more than once. A -refresh in flight is left to finish, with its write to the cache. The promise -`close()` returns resolves once that refresh, the machine-learning fetch and the -background downloads already running have all finished. The cache directory can -then be removed. After `close()`, `fresh()`, `freshCollections()`, +refresh in flight is left to finish, with its write to the cache. `close()` +returns a promise. It resolves once that refresh, the machine-learning fetch and +the background downloads already running have all finished. The cache directory +can then be removed. After `close()`, `fresh()`, `freshCollections()`, `freshFiles()` and `lib.backup()` reject with `the library is closed`. ### Default reads and fresh reads -- 2.54.0