Compare commits

..
1 Commits
Author SHA1 Message Date
clawbot 3c8377cb2c docker build . stamps the git tag or short commit, not dev (closes #154)
check / check (push) Failing after 56s
script/build writes the version script/version prints into
dist/package.json, which quak --version reports: the VERSION environment
variable or build arg when one is given, otherwise git describe --tags
--always, otherwise package.json's version. A checkout with .git whose
version comes out empty, dev or unknown fails the build.

.dockerignore no longer leaves out .git, and ARG VERSION has no default,
so a plain docker build . of a clone stamps its commit. script/docker and
script/cibuild still pass the host's version. make build-bin bundles the
built dist/, so the single binary reports the same version.

Model: opus-5-5
2026-10-02 00:44:56 +00:00
11 changed files with 110 additions and 91 deletions
+1 -4
View File
@@ -3,10 +3,7 @@
# and dropping it would change what the lint phase's prettier check sees. # and dropping it would change what the lint phase's prettier check sees.
# #
# .git is deliberately NOT excluded: the build derives the version it stamps # .git is deliberately NOT excluded: the build derives the version it stamps
# from it (script/version). It is sent without its config, which holds the # from it (script/version).
# clone's remote URL and any credential in it, and which the build stage, the
# final image, would otherwise carry. git describe does not need it.
.git/config
# OS # OS
.DS_Store .DS_Store
+11 -10
View File
@@ -195,15 +195,13 @@ The build fails if the checkout has `.git` and the version still comes out
empty, `dev` or `unknown`: such a build could not be traced back to its commit. empty, `dev` or `unknown`: such a build could not be traced back to its commit.
`.dockerignore` therefore does not leave out `.git`, so any `docker build .` of `.dockerignore` therefore does not leave out `.git`, so any `docker build .` of
a clone stamps the commit it was built from; a shallow clone stamps a tag only a clone stamps the commit it was built from; a shallow clone of one branch has
when the cloned commit itself carries one, and otherwise the short commit. It no tags and stamps the short commit. `script/docker` (and so `make docker`) and
leaves out `.git/config`, which holds the clone's remote URL and any credential `script/cibuild` pass the version they resolve on the host, with `--dirty`, as
in it, so the image carries `.git` without its config; `git describe` does not the build arg, which takes precedence. The image's
need that file. `script/docker` (and so `make docker`) and `script/cibuild` pass `org.opencontainers.image.version` label carries that build arg only, so a build
the version they resolve on the host, with `--dirty`, as the build arg, which given none leaves it empty. `make build-bin` bundles the built `dist/`, so the
takes precedence. The image's `org.opencontainers.image.version` label carries single binary reports the stamped version too.
that build arg only, so a build given none leaves it empty. `make build-bin`
bundles the built `dist/`, so the single binary reports the stamped version too.
## Rationale ## Rationale
@@ -909,7 +907,10 @@ current account's records name.
A live photo's original is cached as at its save path: its image and its video, A live photo's original is cached as at its save path: its image and its video,
each `originals/<fileID>.<ext>` with its own extension, and each `originals/<fileID>.<ext>` with its own extension, and
`originals/<fileID>.livephoto.json` naming them; the two are evicted together. `originals/<fileID>.livephoto.json` naming them; the two are evicted together. A
live photo that an earlier version cached as its ZIP is not served: the library
removes the ZIP when it opens the cache, and fetches the two files when the
photo is next read or precached.
A stored file appears only via an atomic temp-then-rename, so its presence means A stored file appears only via an atomic temp-then-rename, so its presence means
it is complete. Every downloaded original (by `quak get`, the cache, or it is complete. Every downloaded original (by `quak get`, the cache, or
-4
View File
@@ -34,10 +34,6 @@ declares one.
`make build-bin` bundles the built `dist/`, so the single binary reports the `make build-bin` bundles the built `dist/`, so the single binary reports the
same version. same version.
- 2026-10-01: The content cache no longer looks for a live photo that an earlier
version cached as one ZIP (issue 151). When the cache opens, a live photo's
file that no JSON file names is now always left alone.
- 2026-10-01: `examples/download-albums.ts` logs in, opens the library, and for - 2026-10-01: `examples/download-albums.ts` logs in, opens the library, and for
every album downloads each photo to its save path, writes the photo's record every album downloads each photo to its save path, writes the photo's record
and EXIF fields to a JSON file beside it, and writes the album's photos to and EXIF fields to a JSON file beside it, and writes the album's photos to
+5 -3
View File
@@ -361,8 +361,8 @@ const openPart = async (
// written unpacked: each part is named `destination` with the extension // written unpacked: each part is named `destination` with the extension
// replaced by its own entry's, and the two must differ ignoring case. When the // replaced by its own entry's, and the two must differ ignoring case. When the
// file records a hash, `<imageHash>:<videoHash>` must match it, each over that // file records a hash, `<imageHash>:<videoHash>` must match it, each over that
// part's own bytes. Only then are the image, then the video, renamed into // part's own bytes. Only then is whatever was at `destination` removed and the
// place; on any failure neither is stored. // image, then the video, renamed into place; on any failure neither is stored.
// //
// The ZIP is chosen by its uploader and may expand enormously, so each part is // The ZIP is chosen by its uploader and may expand enormously, so each part is
// written as it decompresses and never held, and the ZIP is refused once the // written as it decompresses and never held, and the ZIP is refused once the
@@ -486,6 +486,7 @@ const decryptLivePhoto = async (
await part.handle.sync(); await part.handle.sync();
await part.handle.close(); await part.handle.close();
} }
await rm(destination, { force: true });
await rename(image.tmpPath, path); await rename(image.tmpPath, path);
try { try {
await rename(video.tmpPath, videoPath); await rename(video.tmpPath, videoPath);
@@ -569,7 +570,8 @@ const fetchAndDecrypt = async (
}, api.getRetryOptions()); }, api.getRetryOptions());
// Write `file`'s original to `outPath`. A live photo is written as its image // Write `file`'s original to `outPath`. A live photo is written as its image
// and its video beside `outPath` instead (see `decryptLivePhoto`). // and its video beside `outPath` instead, and whatever was at `outPath` is
// removed (see `decryptLivePhoto`).
export const downloadFile = async ( export const downloadFile = async (
api: ApiClient, api: ApiClient,
file: EnteFile, file: EnteFile,
+2 -4
View File
@@ -1,7 +1,5 @@
// A build reports the version script/build stamps into dist/package.json; // package.json is the one place the version is written. tsc copies it to
// package.json's own version is reported only when running from source. tsc // dist/package.json, so this path resolves from source and from dist/src/.
// copies package.json to dist/package.json, so this path resolves from source
// and from dist/src/.
import pkg from "../package.json" with { type: "json" }; import pkg from "../package.json" with { type: "json" };
export const VERSION: string = pkg.version; export const VERSION: string = pkg.version;
+38 -8
View File
@@ -29,7 +29,14 @@
// the cache does not count as saved there, but is copied there rather than // the cache does not count as saved there, but is copied there rather than
// fetched again. // fetched again.
import { existsSync, readFileSync, statSync } from "node:fs"; import {
closeSync,
existsSync,
openSync,
readFileSync,
readSync,
statSync,
} from "node:fs";
import { import {
chmod, chmod,
copyFile, copyFile,
@@ -274,6 +281,25 @@ const fileSize = (path: string): number | undefined => {
const hasContent = (path: string | undefined): boolean => const hasContent = (path: string | undefined): boolean =>
path !== undefined && (fileSize(path) ?? 0) > 0; path !== undefined && (fileSize(path) ?? 0) > 0;
// Whether the file at `path` begins as a ZIP does, with `PK\x03\x04`. False
// when it cannot be read.
const isZip = (path: string): boolean => {
try {
const fd = openSync(path, "r");
try {
const head = Buffer.alloc(4);
return (
readSync(fd, head, 0, 4, 0) === 4 &&
head.toString("latin1") === "PK\x03\x04"
);
} finally {
closeSync(fd);
}
} catch {
return false;
}
};
// A live photo's image and video are named with the extensions from inside its // A live photo's image and video are named with the extensions from inside its
// ZIP, so their names alone do not say which is which. Wherever the cache or a // ZIP, so their names alone do not say which is which. Wherever the cache or a
// save path stores one, a JSON file of this name beside them names both. // save path stores one, a JSON file of this name beside them names both.
@@ -687,10 +713,8 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
await this.touch(cached.path); await this.touch(cached.path);
return { ...cached, bytes: size, cached: true }; return { ...cached, bytes: size, cached: true };
} }
// A recorded file that has since gone re-fetches below. So does a // A recorded file that has since gone, or a live photo an earlier
// live photo recorded with no video: the cache opened before the // version stored as one ZIP, re-fetches below.
// library's records said it is a live photo, while its image and
// video had no JSON file beside them yet.
known.delete(fileID); known.delete(fileID);
} }
@@ -959,14 +983,20 @@ export class ContentCache implements PhotoContent, ThumbnailsAPI {
if (id === undefined || !existsSync(path)) continue; if (id === undefined || !existsSync(path)) continue;
// A live photo's image and video are one entry, as the JSON file // A live photo's image and video are one entry, as the JSON file
// beside them names them. A live photo's file with no such JSON // beside them names them. A live photo's file with no such JSON
// file is not its original and is left alone: another process may // file is not its original. If it is a ZIP, it is the one an
// have just stored it and not yet written the JSON file. // earlier version stored under the image's name, and is removed.
// Any other is left alone: another process may have just stored
// it and not yet written the JSON file.
const livePhoto = names.has(livePhotoJSONName(String(id))) const livePhoto = names.has(livePhotoJSONName(String(id)))
? readLivePhotoJSON(dir, String(id)) ? readLivePhotoJSON(dir, String(id))
: undefined; : undefined;
if (livePhoto !== undefined) { if (livePhoto !== undefined) {
into.set(id, livePhoto); into.set(id, livePhoto);
} else if (!isLivePhoto(id)) { } else if (isLivePhoto(id)) {
if (isZip(path)) {
await rm(path, { force: true }).catch(() => undefined);
}
} else {
into.set(id, { path }); into.set(id, { path });
} }
} }
+9
View File
@@ -1883,6 +1883,15 @@ describe("downloadFile live photos", () => {
expect(readdirSync(t.dir).sort()).toEqual(["f.JPG", "f.bin"]); expect(readdirSync(t.dir).sort()).toEqual(["f.JPG", "f.bin"]);
}); });
it("replaces what was at the destination, such as an earlier ZIP of the two", async () => {
const t = setup(livePhotoZip(), livePhoto);
writeFileSync(t.outPath, livePhotoZip());
await t.run();
expect(readdirSync(t.dir).sort()).toEqual(["f.heic", "f.mov"]);
});
it("renames the image and then the video into place, each from its own temp file", async () => { it("renames the image and then the video into place, each from its own temp file", async () => {
const t = setup(livePhotoZip(), livePhoto); const t = setup(livePhotoZip(), livePhoto);
+3 -17
View File
@@ -176,7 +176,7 @@ describe("Library content wiring", () => {
await lib.close(); await lib.close();
}); });
it("does not take a live photo's image or video with no JSON file as its original when it opens, and precaches both", async () => { it("removes a live photo's ZIP an earlier version cached when it opens, and precaches its image and video", async () => {
const { file: live, body } = await asLivePhoto(file(1, 1)); const { file: live, body } = await asLivePhoto(file(1, 1));
class LiveClient extends MockClient { class LiveClient extends MockClient {
override async filesSince(): Promise<FilesPage> { override async filesSince(): Promise<FilesPage> {
@@ -197,8 +197,7 @@ describe("Library content wiring", () => {
// A first run records the library, so the next one knows that file 1 // A first run records the library, so the next one knows that file 1
// is a live photo when it opens the cache. // is a live photo when it opens the cache.
await (await open({})).close(); await (await open({})).close();
writeFileSync(join(originals, "1.heic"), "an image"); writeFileSync(join(originals, "1.jpg"), livePhotoZip());
writeFileSync(join(originals, "1.mov"), "a video");
let precached!: () => void; let precached!: () => void;
const done = new Promise<void>((r) => (precached = r)); const done = new Promise<void>((r) => (precached = r));
@@ -209,9 +208,7 @@ describe("Library content wiring", () => {
precached(); precached();
}, },
}); });
expect( expect(existsSync(join(originals, "1.jpg"))).toBe(false);
lib.photos.byID({ fileID: 1 })!.record().originalPath,
).toBeUndefined();
await done; await done;
expect(readdirSync(originals).sort()).toEqual([ expect(readdirSync(originals).sort()).toEqual([
@@ -219,17 +216,6 @@ describe("Library content wiring", () => {
"1.livephoto.json", "1.livephoto.json",
"1.mov", "1.mov",
]); ]);
expect(readFileSync(join(originals, "1.heic"))).toEqual(
Buffer.from(IMAGE),
);
expect(readFileSync(join(originals, "1.mov"))).toEqual(
Buffer.from(VIDEO),
);
expect(
JSON.parse(
readFileSync(join(originals, "1.livephoto.json"), "utf-8"),
),
).toEqual({ image: "1.heic", video: "1.mov" });
expect(lib.photos.byID({ fileID: 1 })!.record().originalPath).toBe( expect(lib.photos.byID({ fileID: 1 })!.record().originalPath).toBe(
join(originals, "1.heic"), join(originals, "1.heic"),
); );
+37 -30
View File
@@ -566,6 +566,43 @@ describe("ContentCache live photos", () => {
expect(events).toEqual(["skipped"]); expect(events).toEqual(["skipped"]);
}); });
it("replaces a live photo an earlier version stored as a ZIP under the image's name", async () => {
const { file: live, body } = await asLivePhoto(file(5, "IMG_5.HEIC"));
mkdirSync(originals(), { recursive: true });
writeFileSync(join(originals(), "5.HEIC"), livePhotoZip());
const cache = cacheOf([live], new Map([[5, body]]));
// Opened without being told that file 5 is a live photo, the cache
// records the ZIP, and does not serve it.
await cache.open();
const result = await cache.original(5);
expect(result.videoPath).toBe(join(originals(), "5.mov"));
expect(readdirSync(originals()).sort()).toEqual([
"5.heic",
"5.livephoto.json",
"5.mov",
]);
});
it("removes a live photo's ZIP an earlier version stored when it opens, so the precache fetches the image and video", async () => {
const { file: live, body } = await asLivePhoto(file(5, "IMG_5.HEIC"));
mkdirSync(originals(), { recursive: true });
writeFileSync(join(originals(), "5.HEIC"), livePhotoZip());
const cache = cacheOf([live], new Map([[5, body]]));
await cache.open((fileID) => fileID === 5);
expect(readdirSync(originals())).toEqual([]);
expect(cache.pathsFor(5)).toEqual({});
const [fetched] = await cache.ensureOriginals({ fileIDs: [5] });
expect(fetched).toEqual({
fileID: 5,
path: join(originals(), "5.heic"),
});
expect(cache.pathsFor(5)).toEqual({ originalPath: fetched!.path });
});
it("leaves the image and video another process has just stored when it opens before their JSON file is written", async () => { it("leaves the image and video another process has just stored when it opens before their JSON file is written", async () => {
const { file: live, body } = await asLivePhoto(file(5, "IMG_5.HEIC")); const { file: live, body } = await asLivePhoto(file(5, "IMG_5.HEIC"));
const server = cdnSource(new Map([[5, body]])); const server = cdnSource(new Map([[5, body]]));
@@ -596,36 +633,6 @@ describe("ContentCache live photos", () => {
expect(second!.pathsFor(5)).toEqual({}); expect(second!.pathsFor(5)).toEqual({});
}); });
it("fetches a live photo's image and video again when the cache opened before knowing it is a live photo and no JSON file names them", async () => {
const { file: live, body } = await asLivePhoto(file(5, "IMG_5.HEIC"));
mkdirSync(originals(), { recursive: true });
writeFileSync(join(originals(), "5.heic"), "an image");
writeFileSync(join(originals(), "5.mov"), "a video");
const cache = cacheOf([live], new Map([[5, body]]));
// Opened without being told that file 5 is a live photo, the cache
// records one of the two files as its original, with no video.
await cache.open();
const events: string[] = [];
const result = await cache.original(5, {
onProgress: (e) => events.push(e.status),
});
expect(events.at(-1)).toBe("done");
expect(result).toEqual({
path: join(originals(), "5.heic"),
videoPath: join(originals(), "5.mov"),
bytes: IMAGE.length,
});
expect(readFileSync(result.path)).toEqual(Buffer.from(IMAGE));
expect(readFileSync(result.videoPath!)).toEqual(Buffer.from(VIDEO));
expect(
JSON.parse(
readFileSync(join(originals(), "5.livephoto.json"), "utf-8"),
),
).toEqual({ image: "5.heic", video: "5.mov" });
});
it.each(["missing", "empty"])( it.each(["missing", "empty"])(
"fetches a live photo again when the video its JSON file names is %s", "fetches a live photo again when the video its JSON file names is %s",
async (state) => { async (state) => {
-6
View File
@@ -54,12 +54,6 @@ describe(".dockerignore", () => {
expect(dockerignore).not.toContain(".git/"); expect(dockerignore).not.toContain(".git/");
}); });
// The build stage is the final image, so a .git/config sent in would
// ship the clone's remote URL and any credential in it.
it("sends .git without its config", () => {
expect(dockerignore).toContain(".git/config");
});
// BuildKit lets a `Dockerfile.dockerignore` shadow the root one; such a // BuildKit lets a `Dockerfile.dockerignore` shadow the root one; such a
// file would silently give the build a different, unreviewed context — // file would silently give the build a different, unreviewed context —
// and eslint's flat config does not ignore dot-directories, so a stray // and eslint's flat config does not ignore dot-directories, so a stray
+4 -5
View File
@@ -787,13 +787,12 @@ describe("fixMissingThumbnails", () => {
}); });
it("re-encodes smaller until the thumbnail fits the recorded size", async () => { it("re-encodes smaller until the thumbnail fits the recorded size", async () => {
// A noisy 64x48 JPEG, which the default encoding (quality 50, not // A noisy 400x300 JPEG, which the default encoding (quality 50, not
// resized because it is under 720 px) cannot compress below the size // resized because it is under 720 px) cannot compress below the size
// recorded here: one byte less than that encoding's ciphertext. It is // recorded here: one byte less than that encoding's ciphertext.
// small so that each encode is quick even on a busy host.
const fixMock = await buildThumbMock(); const fixMock = await buildThumbMock();
const w = 64; const w = 400;
const h = 48; const h = 300;
const noisy = new Uint8Array( const noisy = new Uint8Array(
jpegJs.encode( jpegJs.encode(
{ {