Author SHA1 Message Date
sneak 279d3f6974 make build-bin: on Linux, build with bun's pinned release build (closes #240)
check / check (push) Successful in 2m52s
nix's bun loads through a glibc in the nix store, and bun build --compile
copied that loader into bin/quak, so nix-collect-garbage could break the
cron backup. The build moves into script/build-bin. On Linux it downloads
bun 1.3.13's release archive for x86-64 (baseline) or arm64 from GitHub,
checks its SHA-256 before running it, and compiles with that bun, so the
binary loads through the system loader. macOS still runs bun through
nix-shell.

Model: opus-5-5
2026-11-11 21:07:08 +00:00
clawbot 8e120a4f49 Remove the no-op --collection option from get and get-thumb (closes #241)
check / check (push) Successful in 3m6s
Both commands listed `--collection <id>` as accepted and ignored. It now
is refused as an unknown option, and the README no longer mentions it.

Model: opus-5-5
2026-11-11 21:59:55 +01:00
11 changed files with 95 additions and 58 deletions

No files matched your search

+2 -4
View File
@@ -26,10 +26,8 @@ check:
build: build:
@script/build @script/build
# Bundles the built dist/, so the binary reports the version script/build build-bin:
# stamped. @script/build-bin
build-bin: build
nix-shell -p bun --run "bun build dist/bin/quak.js --compile --outfile bin/quak"
install: build-bin install: build-bin
mkdir -p ~/bin mkdir -p ~/bin
+12 -8
View File
@@ -135,10 +135,12 @@ requests. The "Library reference" below covers both.
## Running the backup from cron ## Running the backup from cron
`quak backup` never prompts, so cron can run it. `make install` builds quak as `quak backup` never prompts, so cron can run it. `make install` builds quak as
one binary and copies it to `~/bin/quak`. Besides what Getting Started needs, one binary and copies it to `~/bin/quak`. It uses bun, a JavaScript runtime, to
`make install` needs `make` and [nix](https://nixos.org/download). It uses bun, bundle the built `dist/` into one executable. Besides what Getting Started
a JavaScript runtime, to bundle the built `dist/` into one executable. It runs needs, `make install` needs `make`. On Linux it downloads bun from GitHub with
bun through `nix-shell -p bun`. `curl` and unpacks it with `unzip`. It checks the download against a SHA-256
hash in the repo. On macOS it runs bun through `nix-shell -p bun`, so it needs
[nix](https://nixos.org/download).
Log in once with `~/bin/quak`, as the user the cron job will run as: Log in once with `~/bin/quak`, as the user the cron job will run as:
@@ -302,7 +304,7 @@ title, with unsafe characters replaced (see "File names from the server"). With
original is kept in the cache as well. original is kept in the cache as well.
A file ID that no collection holds gives `File <id> not found`, and the exit A file ID that no collection holds gives `File <id> not found`, and the exit
status is 1. The option `--collection <id>` is accepted and ignored. status is 1.
A live photo is a still image and a short video that Ente stores as one file. A live photo is a still image and a short video that Ente stores as one file.
`get` writes both, each named after the title with its own extension, as Ente's `get` writes both, each named after the title with its own extension, as Ente's
@@ -1771,6 +1773,8 @@ standard. The others are additions to it.
`package.json` declares (`main`, `types`, `bin`) were written, and make the `package.json` declares (`main`, `types`, `bin`) were written, and make the
command line executable. command line executable.
- `script/version`: print the version `script/build` stamps (see "Version"). - `script/version`: print the version `script/build` stamps (see "Version").
- `script/build-bin`: run `script/build`, then bundle the built `dist/` into one
binary, `bin/quak`, with bun (see "Running the backup from cron").
- `script/test`: run the test suite, by building the `test` phase of the - `script/test`: run the test suite, by building the `test` phase of the
`Dockerfile`. Requires Docker. `Dockerfile`. Requires Docker.
- `script/lint`: run eslint and a prettier check, by building the `lint` phase - `script/lint`: run eslint and a prettier check, by building the `lint` phase
@@ -1799,8 +1803,7 @@ The Makefile targets are:
- `make build`: runs `script/build`. - `make build`: runs `script/build`.
- `make docker`: runs `script/docker`. - `make docker`: runs `script/docker`.
- `make hooks`: runs `script/install-precommit`. - `make hooks`: runs `script/install-precommit`.
- `make build-bin`: bundles the built `dist/` into one binary, `bin/quak`. It - `make build-bin`: runs `script/build-bin`.
runs bun through `nix-shell -p bun`.
- `make install`: runs `make build-bin` and copies the binary to `~/bin/quak`. - `make install`: runs `make build-bin` and copies the binary to `~/bin/quak`.
- `make dev`: runs the TypeScript compiler in watch mode. - `make dev`: runs the TypeScript compiler in watch mode.
- `make clean`: removes `dist/`, `coverage/`, `.vitest-cache/` and the - `make clean`: removes `dist/`, `coverage/`, `.vitest-cache/` and the
@@ -1908,9 +1911,10 @@ These apply to people and to coding agents alike.
- container images by digest; - container images by digest;
- the nvm archive `script/bootstrap` downloads, and the Python packages - the nvm archive `script/bootstrap` downloads, and the Python packages
`bin/clip-keyword-vectors` installs, by SHA-256 hash; `bin/clip-keyword-vectors` installs, by SHA-256 hash;
- bun, which `script/build-bin` downloads on Linux, by SHA-256 hash;
- Node and yarn, when `script/bootstrap` installs them, by version. - Node and yarn, when `script/bootstrap` installs them, by version.
- Not pinned: - Not pinned:
- bun, which `make build-bin` takes from `nix-shell -p bun`; - bun on macOS, which `script/build-bin` takes from `nix-shell -p bun`;
- `make` and `git`, which `script/bootstrap` installs from the system - `make` and `git`, which `script/bootstrap` installs from the system
package manager when they are missing, as every `Dockerfile` stage does; package manager when they are missing, as every `Dockerfile` stage does;
- `bash` and `curl`, which `script/bootstrap` installs the same way when it - `bash` and `curl`, which `script/bootstrap` installs the same way when it
-2
View File
@@ -123,7 +123,6 @@ program
.description("Download and decrypt a single file") .description("Download and decrypt a single file")
.argument("<fileID>", "File ID (from `quak files`)") .argument("<fileID>", "File ID (from `quak files`)")
.option("--out <path>", "Output file path") .option("--out <path>", "Output file path")
.option("--collection <id>", "Accepted for compatibility; ignored")
.action((fileID: string, opts: { out?: string }) => .action((fileID: string, opts: { out?: string }) =>
run(getCommand(context(), fileID, opts)), run(getCommand(context(), fileID, opts)),
); );
@@ -133,7 +132,6 @@ program
.description("Download and decrypt a thumbnail") .description("Download and decrypt a thumbnail")
.argument("<fileID>", "File ID (from `quak files`)") .argument("<fileID>", "File ID (from `quak files`)")
.option("--out <path>", "Output file path") .option("--out <path>", "Output file path")
.option("--collection <id>", "Accepted for compatibility; ignored")
.action((fileID: string, opts: { out?: string }) => .action((fileID: string, opts: { out?: string }) =>
run(getThumbCommand(context(), fileID, opts)), run(getThumbCommand(context(), fileID, opts)),
); );
+56
View File
@@ -0,0 +1,56 @@
#!/bin/sh
# script/build-bin: run script/build, then bundle the built dist/ into one
# executable, bin/quak, which reports the version script/build stamped. Our own
# extension to scripts-to-rule-them-all.
#
# bun compiles the binary from a copy of its own executable. On Linux, nix's
# bun loads through a C library in the nix store, and so does a binary compiled
# from it. nix-collect-garbage can delete that library, and the binary then no
# longer starts. On Linux this script therefore compiles with bun's own release
# build, which loads through the system's C library. It downloads that build
# and checks it against the hash below before running it.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# bun 1.3.13, 2026-11-11, and the SHA-256 of each release archive below. The
# x86-64 archive is the baseline build, which also runs on CPUs without AVX2.
BUN_VERSION="1.3.13"
BUN_LINUX_X64_BASELINE_SHA256="9d8a24292a7068090205daac0a5a223f5f69736f5287e37bf88d3b4031edc750"
BUN_LINUX_AARCH64_SHA256="70bae41b3908b0a120e1e58c5c8af30e74afae3b8d11b0d3fdd8e787ddfb4b22"
# compile_with_downloaded_bun <archive-name> <sha256>
compile_with_downloaded_bun() {
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT
curl -fsSL -o "$tmp/$1.zip" \
"https://github.com/oven-sh/bun/releases/download/bun-v$BUN_VERSION/$1.zip"
echo "$2 $tmp/$1.zip" | sha256sum -c -
unzip -q "$tmp/$1.zip" -d "$tmp"
"$tmp/$1/bun" build dist/bin/quak.js --compile --outfile bin/quak
}
main() {
cd "$ROOT"
"$ROOT/script/build"
case "$(uname -s)-$(uname -m)" in
Linux-x86_64)
compile_with_downloaded_bun bun-linux-x64-baseline \
"$BUN_LINUX_X64_BASELINE_SHA256"
;;
Linux-aarch64)
compile_with_downloaded_bun bun-linux-aarch64 \
"$BUN_LINUX_AARCH64_SHA256"
;;
Linux-*)
echo "build-bin: bun has no release build for Linux on $(uname -m)" >&2
exit 1
;;
*)
nix-shell -p bun --run \
"bun build dist/bin/quak.js --compile --outfile bin/quak"
;;
esac
}
main "$@"
+1 -8
View File
@@ -354,13 +354,7 @@ export class ApiClient {
}); });
} }
// `contentMD5` is the value given to `getUploadURL`. The server signs it async putFile(presignedURL: string, data: Uint8Array): Promise<void> {
// into the URL, and storage refuses a PUT without that `Content-MD5`.
async putFile(
presignedURL: string,
data: Uint8Array,
contentMD5: string,
): Promise<void> {
// Idempotent despite being a write: a presigned PUT stores one whole // Idempotent despite being a write: a presigned PUT stores one whole
// object at one key in one request, so replaying it either overwrites // object at one key in one request, so replaying it either overwrites
// the same bytes or lands them for the first time. There is no partial // the same bytes or lands them for the first time. There is no partial
@@ -376,7 +370,6 @@ export class ApiClient {
headers: { headers: {
"Content-Type": "application/octet-stream", "Content-Type": "application/octet-stream",
"Content-Length": String(data.length), "Content-Length": String(data.length),
"Content-MD5": contentMD5,
}, },
body: data, body: data,
signal: AbortSignal.timeout(this.requestTimeoutMs), signal: AbortSignal.timeout(this.requestTimeoutMs),
+1 -1
View File
@@ -325,7 +325,7 @@ export const fixMissingThumbnails = async (
ciphertext.length, ciphertext.length,
md5, md5,
); );
await api.putFile(url, ciphertext, md5); await api.putFile(url, ciphertext);
await api.updateThumbnail(file.id, objectKey, toBase64(header)); await api.updateThumbnail(file.id, objectKey, toBase64(header));
log(`[${collectionName}] Thumbnail uploaded for ${title}`); log(`[${collectionName}] Thumbnail uploaded for ${title}`);
+4 -10
View File
@@ -780,7 +780,7 @@ describe("ApiClient timeouts", () => {
await client.getJSON("/a"); await client.getJSON("/a");
await client.postJSON("/b", {}); await client.postJSON("/b", {});
await client.putFile("https://s3.example/x", new Uint8Array([1]), "m"); await client.putFile("https://s3.example/x", new Uint8Array([1]));
await client.putJSON("/c", {}); await client.putJSON("/c", {});
for (const call of calls) { for (const call of calls) {
@@ -825,8 +825,7 @@ describe("ApiClient timeouts", () => {
[ [
"putFile", "putFile",
() => new Response(null, { status: 200 }), () => new Response(null, { status: 200 }),
(c) => (c) => c.putFile("https://s3.example/x", new Uint8Array([1])),
c.putFile("https://s3.example/x", new Uint8Array([1]), "m"),
], ],
[ [
"getFileStream", "getFileStream",
@@ -1050,7 +1049,7 @@ describe("ApiClient error typing", () => {
const client = new ApiClient({ fetch, retry: noWait }); const client = new ApiClient({ fetch, retry: noWait });
const err: unknown = await client const err: unknown = await client
.putFile("https://s3.example/obj", new Uint8Array([1, 2]), "m") .putFile("https://s3.example/obj", new Uint8Array([1, 2]))
.catch((e: unknown) => e); .catch((e: unknown) => e);
expect(err).toBeInstanceOf(ApiError); expect(err).toBeInstanceOf(ApiError);
@@ -1068,11 +1067,7 @@ describe("ApiClient error typing", () => {
); );
const client = new ApiClient({ fetch, retry: noWait }); const client = new ApiClient({ fetch, retry: noWait });
await client.putFile( await client.putFile("https://s3.example/obj", new Uint8Array([1, 2]));
"https://s3.example/obj",
new Uint8Array([1, 2]),
"m",
);
expect(calls).toHaveLength(2); expect(calls).toHaveLength(2);
}); });
@@ -1282,7 +1277,6 @@ describe("ApiClient onRequest", () => {
await client.putFile( await client.putFile(
"https://s3.example/obj?X-Amz-Signature=abc", "https://s3.example/obj?X-Amz-Signature=abc",
new Uint8Array([1, 2]), new Uint8Array([1, 2]),
"m",
); );
expect(requests).toEqual([ expect(requests).toEqual([
+2 -13
View File
@@ -80,20 +80,13 @@ describe("ApiClient.putFile", () => {
const client = new ApiClient({ fetch, authToken: "secret-tok" }); const client = new ApiClient({ fetch, authToken: "secret-tok" });
const data = new Uint8Array([1, 2, 3, 4, 5]); const data = new Uint8Array([1, 2, 3, 4, 5]);
await client.putFile( await client.putFile("https://s3.example.com/presigned?sig=abc", data);
"https://s3.example.com/presigned?sig=abc",
data,
"md5base64==",
);
expect(calls[0]!.url).toBe("https://s3.example.com/presigned?sig=abc"); expect(calls[0]!.url).toBe("https://s3.example.com/presigned?sig=abc");
expect(calls[0]!.init?.method).toBe("PUT"); expect(calls[0]!.init?.method).toBe("PUT");
const headers = new Headers(calls[0]!.init?.headers as HeadersInit); const headers = new Headers(calls[0]!.init?.headers as HeadersInit);
expect(headers.get("Content-Type")).toBe("application/octet-stream"); expect(headers.get("Content-Type")).toBe("application/octet-stream");
expect(headers.get("Content-Length")).toBe("5"); expect(headers.get("Content-Length")).toBe("5");
// The server signs the MD5 into the URL; storage refuses the PUT
// without it.
expect(headers.get("Content-MD5")).toBe("md5base64==");
// Must NOT leak auth headers to S3 // Must NOT leak auth headers to S3
expect(headers.has("X-Auth-Token")).toBe(false); expect(headers.has("X-Auth-Token")).toBe(false);
expect(headers.has("X-Client-Package")).toBe(false); expect(headers.has("X-Client-Package")).toBe(false);
@@ -106,11 +99,7 @@ describe("ApiClient.putFile", () => {
const client = new ApiClient({ fetch }); const client = new ApiClient({ fetch });
await expect( await expect(
client.putFile( client.putFile("https://s3.example.com/bad", new Uint8Array(10)),
"https://s3.example.com/bad",
new Uint8Array(10),
"md5base64==",
),
).rejects.toThrow(/403/); ).rejects.toThrow(/403/);
}); });
}); });
+14 -2
View File
@@ -1,8 +1,9 @@
/** /**
* Tests for the retry options `bin/quak.ts` loads the saved session with * Tests for the retry options `bin/quak.ts` loads the saved session with
* (`quak backup` gets the unattended ones, every other command the default), * (`quak backup` gets the unattended ones, every other command the default),
* each with the callbacks that log its requests and retries, and for * each with the callbacks that log its requests and retries, for `quak files`
* `quak files` taking its collection ID as an optional argument. * taking its collection ID as an optional argument, and for `quak get` and
* `quak get-thumb` refusing `--collection`.
* *
* Each test runs `bin/quak.ts`, as the smoke test does, with its session loader * Each test runs `bin/quak.ts`, as the smoke test does, with its session loader
* replaced by one that records the options it is given and reports no session, * replaced by one that records the options it is given and reports no session,
@@ -105,3 +106,14 @@ describe("bin/quak.ts files", () => {
expect(await quak("files", "7")).toMatch(/^Not logged in/); expect(await quak("files", "7")).toMatch(/^Not logged in/);
}); });
}); });
describe("bin/quak.ts get and get-thumb", () => {
it.each(["get", "get-thumb"])(
"%s refuses --collection as an unknown option",
async (command) => {
expect(await quak(command, "1", "--collection", "2")).toMatch(
/^error: unknown option '--collection'/,
);
},
);
});
+2 -2
View File
@@ -569,14 +569,14 @@ describe("files", () => {
}); });
describe("get and get-thumb", () => { describe("get and get-thumb", () => {
it("get finds a file in any album without --collection", async () => { it("get finds a file in any album", async () => {
const out = join(root, "diagram.png"); const out = join(root, "diagram.png");
expect(await getCommand(context(), "200", { out })).toBe(0); expect(await getCommand(context(), "200", { out })).toBe(0);
expect(readFileSync(out)).toEqual(Buffer.alloc(7, 200)); expect(readFileSync(out)).toEqual(Buffer.alloc(7, 200));
expect(stderr.text).toBe(`7 bytes -> ${out}\n`); expect(stderr.text).toBe(`7 bytes -> ${out}\n`);
}); });
it("get-thumb finds a file in any album without --collection", async () => { it("get-thumb finds a file in any album", async () => {
const out = join(root, "thumb.jpg"); const out = join(root, "thumb.jpg");
expect(await getThumbCommand(context(), "200", { out })).toBe(0); expect(await getThumbCommand(context(), "200", { out })).toBe(0);
expect(readFileSync(out)).toEqual(Buffer.alloc(3, 200)); expect(readFileSync(out)).toEqual(Buffer.alloc(3, 200));
+1 -8
View File
@@ -273,7 +273,6 @@ const buildThumbMock = async (opts?: {
const buildThumbFetch = (m: ThumbMockState) => { const buildThumbFetch = (m: ThumbMockState) => {
let srpServer: SrpServer; let srpServer: SrpServer;
let signedMD5: string | undefined;
return (async ( return (async (
input: RequestInfo | URL, input: RequestInfo | URL,
@@ -366,9 +365,8 @@ const buildThumbFetch = (m: ThumbMockState) => {
return new Response("not found", { status: 404 }); return new Response("not found", { status: 404 });
} }
// Upload URL minting. The real server signs the MD5 into the URL. // Upload URL minting
if (path === "/files/upload-url") { if (path === "/files/upload-url") {
signedMD5 = JSON.parse(init?.body as string).contentMD5;
return json({ return json({
objectKey: `42/thumb-${Date.now()}`, objectKey: `42/thumb-${Date.now()}`,
url: "https://s3.mock.test/presigned-put", url: "https://s3.mock.test/presigned-put",
@@ -377,11 +375,6 @@ const buildThumbFetch = (m: ThumbMockState) => {
// Presigned PUT (S3 upload) // Presigned PUT (S3 upload)
if (url.startsWith("https://s3.mock.test/")) { if (url.startsWith("https://s3.mock.test/")) {
// Storage refuses a PUT that lacks the signed `Content-MD5`.
const headers = new Headers(init?.headers);
if (headers.get("Content-MD5") !== signedMD5) {
return new Response("AccessDenied", { status: 400 });
}
const body = init?.body; const body = init?.body;
// Store the uploaded bytes for later inspection // Store the uploaded bytes for later inspection
if (body instanceof Uint8Array) { if (body instanceof Uint8Array) {