Author SHA1 Message Date
sneak b6020aca4e quak login --server: log in to a self-hosted Ente server (closes #243)
check / check (push) Successful in 4m25s
`quak login` takes the server's API address from `--server` or
`QUAK_SERVER`, refusing anything that is not an http or https URL. The
client's snapshot now keeps a non-default `apiOrigin`, so the session
file names the server and every later command uses it. A snapshot's
`apiOrigin` wins over the one in `fromJSON`'s `apiOptions`, since the
token belongs to that server.

Model: opus-5-5
2026-11-13 18:48:55 +00:00
clawbot 9bd7e8f099 make build-bin: on macOS, build with bun's pinned release build (closes #251)
check / check (push) Successful in 2m57s
nixpkgs points its macOS bun at an ICU in the nix store, and bun build
--compile copied that reference into bin/quak, so nix-collect-garbage could
break the cron backup. script/build-bin now downloads bun 1.3.13's release
archive on macOS too, Intel (baseline) or Apple silicon, checks its SHA-256,
and compiles with it. Those builds load only /usr/lib libraries. nix-shell is
no longer used; any system without a bun release build stops with an error.
The hash check falls back to shasum where sha256sum is missing.

Model: opus-5-5
2026-11-12 02:17:25 +01:00
clawbot 50e2381cad make build-bin: on Linux, build with bun's pinned release build (closes #240)
check / check (push) Successful in 3m12s
nix's bun loads through a glibc in the nix store, and bun build --compile
copied that loader into bin/quak, so nix-collect-garbage could break the
cron backup. The build moves into script/build-bin. On Linux it downloads
bun 1.3.13's release archive for the machine from GitHub: x86-64
(baseline) or arm64, glibc or musl. It checks the archive's SHA-256
before running it, and compiles with that bun, so the binary loads
through the system loader. macOS still runs bun through nix-shell.

Model: opus-5-5
2026-11-12 00:28:48 +01:00
clawbot 37cdbc6b9a Node 22.13 minimum, and the library linked to its clone (closes #239)
check / check (push) Successful in 3m6s
`engines.node` and the README now state the minimum that quak's
`yarn.lock` needs: Node 22.13, or 23.5 on Node 23. `@inquirer/prompts`
and its packages refuse 23.0 to 23.4.

"Using the library" now links the clone with `yarn link` instead of
`yarn add file:../quak`. A linked quak runs on the dependencies the clone
installed from `yarn.lock`; `file:` resolved them afresh, so a new
upstream release could raise the Node minimum without any change here.

Model: opus-5-5
2026-11-11 23:29:01 +01:00
clawbot 1bf18baf00 Thumbnail upload PUT sends the signed Content-MD5 (closes #238)
check / check (push) Successful in 5m34s
`/files/upload-url` signs the content MD5 into the presigned URL, so
storage refused the PUT that `putFile` sent without it. `putFile` now
takes the MD5 given to `getUploadURL` and sends it. The thumbnail test's
fake storage refuses a PUT without the signed header.

Model: opus-5-5
2026-11-11 22:59:10 +01:00
clawbot 7878ff3812 README: what a backup does with a file deleted in Ente (closes #242)
check / check (push) Successful in 3m24s
A file deleted in Ente keeps its original at its save path, and the JSON
beside it. The next run removes its symlinks from the collection
directories.

Model: opus-5-5
2026-11-11 22:29:14 +01:00
clawbot 8e120a4f49 Remove the no-op --collection option from get and get-thumb (closes #241)
check / check (push) Successful in 3m6s
Both commands listed `--collection <id>` as accepted and ignored. It now
is refused as an unknown option, and the README no longer mentions it.

Model: opus-5-5
2026-11-11 21:59:55 +01:00
clawbot 30fc349b37 TODO.md: open work only (closes #235)
check / check (push) Successful in 2m49s
Removes the Completed Steps section: dated accounts of finished work,
review and merge history, and notes on past decisions. Git keeps the
history. Status, Next Step and Future Steps are unchanged.

The Workflow step "move Next Step to the top of Completed Steps" now
reads "delete Next Step from `TODO.md`", the standard wording. The
`README.md` TODO section no longer calls `TODO.md` the work log; it
says the file holds the next step and the steps planned.

Model: opus-5-5
2026-11-11 14:59:31 +01:00
16 changed files with 666 additions and 47 deletions

No files matched your search

+2 -4
View File
@@ -26,10 +26,8 @@ check:
build: build:
@script/build @script/build
# Bundles the built dist/, so the binary reports the version script/build build-bin:
# stamped. @script/build-bin
build-bin: build
nix-shell -p bun --run "bun build dist/bin/quak.js --compile --outfile bin/quak"
install: build-bin install: build-bin
mkdir -p ~/bin mkdir -p ~/bin
+67 -19
View File
@@ -31,7 +31,8 @@ again on the next run. Every download is decrypted on its way to disk.
## Getting Started ## Getting Started
The commands below need `git`, Node 22 or later and `yarn`. The commands below need `git`, `yarn` and Node 22.13 or later. Node 23 needs
23.5 or later.
```bash ```bash
git clone https://git.eeqj.de/sneak/quak.git git clone https://git.eeqj.de/sneak/quak.git
@@ -69,11 +70,29 @@ The library keeps a local copy of the account's album and file lists, the cache.
It brings the cache up to date with the server in rounds called refreshes. It brings the cache up to date with the server in rounds called refreshes.
quak is not published to a package registry: `package.json` marks it `private`, quak is not published to a package registry: `package.json` marks it `private`,
and `dist/` is not committed. Clone and build it as above, then add the clone to and `dist/` is not committed. Clone and build it as above, then link the clone
your own program's `package.json` by its path, for example into your own program:
`yarn add file:../quak`. The package's `main`, `types` and `exports` fields
point into `dist/`, so the clone must be built first. The package is an ES ```bash
module only, so a CommonJS program cannot `require` it. # In the quak clone:
yarn link
# In your program's directory:
yarn link quak
```
`yarn link` also installs the `quak` command in yarn's global bin directory, and
`yarn unlink` in the clone removes the link and the command.
A linked quak uses the dependencies that `yarn install` put in the clone, at the
versions in quak's `yarn.lock`. Avoid `yarn add file:../quak`: it resolves
quak's dependencies again, and their newer releases can need a newer Node.
`yarn link quak` does not add quak to your `package.json`, so run it again in
each new checkout of your program.
The package's `main`, `types` and `exports` fields point into `dist/`, so the
clone must be built first. The package is an ES module only, so a CommonJS
program cannot `require` it.
```ts ```ts
import { Client, Library } from "quak"; import { Client, Library } from "quak";
@@ -135,10 +154,15 @@ requests. The "Library reference" below covers both.
## Running the backup from cron ## Running the backup from cron
`quak backup` never prompts, so cron can run it. `make install` builds quak as `quak backup` never prompts, so cron can run it. `make install` builds quak as
one binary and copies it to `~/bin/quak`. Besides what Getting Started needs, one binary and copies it to `~/bin/quak`. It uses bun, a JavaScript runtime, to
`make install` needs `make` and [nix](https://nixos.org/download). It uses bun, bundle the built `dist/` into one executable. Besides what Getting Started
a JavaScript runtime, to bundle the built `dist/` into one executable. It runs needs, `make install` needs `make`. It downloads bun from GitHub with `curl` and
bun through `nix-shell -p bun`. unpacks it with `unzip`. It checks the download against a SHA-256 hash in the
repo. `make install` works on Linux and macOS, on x86-64 and arm64.
On Linux systems that use the musl C library, such as Alpine, `make install`
downloads bun's musl build. On Alpine, both that bun and `~/bin/quak` need the
`libstdc++` and `libgcc` packages.
Log in once with `~/bin/quak`, as the user the cron job will run as: Log in once with `~/bin/quak`, as the user the cron job will run as:
@@ -205,7 +229,7 @@ below belong to quak itself, and go before or after the command's name.
### The commands ### The commands
``` ```
quak login quak login [--server <url>]
quak whoami quak whoami
quak logout quak logout
quak collections [--json] quak collections [--json]
@@ -222,13 +246,26 @@ Every command but `login` and `logout` needs a saved session. Every command but
`login`, `whoami` and `logout` opens the library. How each one gets current data `login`, `whoami` and `logout` opens the library. How each one gets current data
from the server is described under "How a command gets current data". from the server is described under "How a command gets current data".
#### `quak login` #### `quak login [--server <url>]`
Logs in to an account and saves the session. It reads the email from Logs in to an account and saves the session. It reads the email from
`QUAK_EMAIL` and the password from `QUAK_PASSWORD`. For each one that is not `QUAK_EMAIL` and the password from `QUAK_PASSWORD`. For each one that is not
set, it asks on the terminal. The password shows as `*` characters while it is set, it asks on the terminal. The password shows as `*` characters while it is
typed. typed.
quak logs in to Ente's hosted service unless you name a self-hosted Ente server.
Name it with `--server`, or with `QUAK_SERVER` when `--server` is not given.
Give the address of the server's API, for example
`quak login --server https://ente.example.com`. The address is an http or https
URL. It may include a path, but no query or fragment. quak refuses any other
address before it asks for anything. It logs
`Invalid server address "<url>": it must be an http or https URL with no query or fragment`,
and the exit status is 1.
The session file keeps the server's address. Every later command, cron runs
included, uses that server without further setting. To change server, log in
again.
When the account has a second factor, quak asks for the code. It asks When the account has a second factor, quak asks for the code. It asks
`TOTP code: ` for a time-based code from an authenticator app. An account whose `TOTP code: ` for a time-based code from an authenticator app. An account whose
only second factor is a passkey cannot log in with quak. When the account has only second factor is a passkey cannot log in with quak. When the account has
@@ -302,7 +339,7 @@ title, with unsafe characters replaced (see "File names from the server"). With
original is kept in the cache as well. original is kept in the cache as well.
A file ID that no collection holds gives `File <id> not found`, and the exit A file ID that no collection holds gives `File <id> not found`, and the exit
status is 1. The option `--collection <id>` is accepted and ignored. status is 1.
A live photo is a still image and a short video that Ente stores as one file. A live photo is a still image and a short video that Ente stores as one file.
`get` writes both, each named after the title with its own extension, as Ente's `get` writes both, each named after the title with its own extension, as Ente's
@@ -573,9 +610,11 @@ Linux that is `$XDG_DATA_HOME/quak/session.json`, or
mode `0600`, in a directory it creates with mode `0700`. mode `0600`, in a directory it creates with mode `0700`.
The file holds the account's email, its user ID, the session token and the The file holds the account's email, its user ID, the session token and the
account's three keys, in base64 and unencrypted. Treat it as you would treat the account's three keys, in base64 and unencrypted. After a login to a self-hosted
password. A missing file means "not logged in". A file that cannot be read back server, it also holds the server's address as `apiOrigin`. Treat it as you would
is reported as corrupt, with the reason. Run `quak logout`, then `quak login`. treat the password. A missing file means "not logged in". A file that cannot be
read back is reported as corrupt, with the reason. Run `quak logout`, then
`quak login`.
A command that needs a session exits 3 when the file is missing or corrupt. It A command that needs a session exits 3 when the file is missing or corrupt. It
also exits 3 when the server no longer accepts the session, and prints also exits 3 when the server no longer accepts the session, and prints
@@ -671,6 +710,9 @@ but letters and digits, or when it is longer than 16 characters.
When the date or the time zone changes, the next run stores the original at its When the date or the time zone changes, the next run stores the original at its
new path and leaves the old copy where it is. new path and leaves the old copy where it is.
A file deleted in Ente keeps its original at its save path, and the JSON beside
it. The next run removes its symlinks from the collection directories.
The library gives the same path as `photo.savePath` and writes it with The library gives the same path as `photo.savePath` and writes it with
`photo.download()` (see "A photo's content"). `photo.download()` (see "A photo's content").
@@ -1450,6 +1492,11 @@ write wherever it likes. `Client.fromJSON(snapshot, apiOptions?)` restores a
working client from that snapshot without logging in again. It checks every working client from that snapshot without logging in again. It checks every
field and each key's length first, and throws an error naming the bad field. field and each key's length first, and throws an error naming the bad field.
A client whose `apiOptions` named an `apiOrigin` puts it in its snapshot. The
restored client then uses that server, even when the `apiOptions` given to
`Client.fromJSON` name another. A snapshot without `apiOrigin` uses the
`apiOrigin` of those `apiOptions`, or the default server.
`Client.fromJSON` works only once libsodium, the cryptography library quak uses, `Client.fromJSON` works only once libsodium, the cryptography library quak uses,
is loaded. `Client.login` loads it, and `Client.fromJSON` does not. A program is loaded. `Client.login` loads it, and `Client.fromJSON` does not. A program
that restores a session in a new process first runs `await sodium.ready`, with that restores a session in a new process first runs `await sodium.ready`, with
@@ -1771,6 +1818,8 @@ standard. The others are additions to it.
`package.json` declares (`main`, `types`, `bin`) were written, and make the `package.json` declares (`main`, `types`, `bin`) were written, and make the
command line executable. command line executable.
- `script/version`: print the version `script/build` stamps (see "Version"). - `script/version`: print the version `script/build` stamps (see "Version").
- `script/build-bin`: run `script/build`, then bundle the built `dist/` into one
binary, `bin/quak`, with bun (see "Running the backup from cron").
- `script/test`: run the test suite, by building the `test` phase of the - `script/test`: run the test suite, by building the `test` phase of the
`Dockerfile`. Requires Docker. `Dockerfile`. Requires Docker.
- `script/lint`: run eslint and a prettier check, by building the `lint` phase - `script/lint`: run eslint and a prettier check, by building the `lint` phase
@@ -1799,8 +1848,7 @@ The Makefile targets are:
- `make build`: runs `script/build`. - `make build`: runs `script/build`.
- `make docker`: runs `script/docker`. - `make docker`: runs `script/docker`.
- `make hooks`: runs `script/install-precommit`. - `make hooks`: runs `script/install-precommit`.
- `make build-bin`: bundles the built `dist/` into one binary, `bin/quak`. It - `make build-bin`: runs `script/build-bin`.
runs bun through `nix-shell -p bun`.
- `make install`: runs `make build-bin` and copies the binary to `~/bin/quak`. - `make install`: runs `make build-bin` and copies the binary to `~/bin/quak`.
- `make dev`: runs the TypeScript compiler in watch mode. - `make dev`: runs the TypeScript compiler in watch mode.
- `make clean`: removes `dist/`, `coverage/`, `.vitest-cache/` and the - `make clean`: removes `dist/`, `coverage/`, `.vitest-cache/` and the
@@ -1908,9 +1956,9 @@ These apply to people and to coding agents alike.
- container images by digest; - container images by digest;
- the nvm archive `script/bootstrap` downloads, and the Python packages - the nvm archive `script/bootstrap` downloads, and the Python packages
`bin/clip-keyword-vectors` installs, by SHA-256 hash; `bin/clip-keyword-vectors` installs, by SHA-256 hash;
- the bun builds `script/build-bin` downloads, by SHA-256 hash;
- Node and yarn, when `script/bootstrap` installs them, by version. - Node and yarn, when `script/bootstrap` installs them, by version.
- Not pinned: - Not pinned:
- bun, which `make build-bin` takes from `nix-shell -p bun`;
- `make` and `git`, which `script/bootstrap` installs from the system - `make` and `git`, which `script/bootstrap` installs from the system
package manager when they are missing, as every `Dockerfile` stage does; package manager when they are missing, as every `Dockerfile` stage does;
- `bash` and `curl`, which `script/bootstrap` installs the same way when it - `bash` and `curl`, which `script/bootstrap` installs the same way when it
+11 -4
View File
@@ -74,7 +74,11 @@ const context = (): CliContext => ({
sessionDir: paths.data, sessionDir: paths.data,
cacheDir: program.opts<ProgramOptions>().cacheDir, cacheDir: program.opts<ProgramOptions>().cacheDir,
loadSession: (path) => loadSession(path, logRequests(log())), loadSession: (path) => loadSession(path, logRequests(log())),
login: (opts) => Client.login({ ...opts, apiOptions: logRequests(log()) }), login: (opts) =>
Client.login({
...opts,
apiOptions: { ...opts.apiOptions, ...logRequests(log()) },
}),
prompt: (message) => input({ message }), prompt: (message) => input({ message }),
promptSecret: (message) => password({ message, mask: true }), promptSecret: (message) => password({ message, mask: true }),
}); });
@@ -85,7 +89,12 @@ const run = (command: Promise<number>): Promise<void> =>
program program
.command("login") .command("login")
.description("Log in to an Ente account and save the session") .description("Log in to an Ente account and save the session")
.action(() => run(loginCommand(context()))); .option(
"--server <url>",
"Address of a self-hosted Ente server's API, saved with the session " +
"(default: QUAK_SERVER, then Ente's hosted service)",
)
.action((opts: { server?: string }) => run(loginCommand(context(), opts)));
program program
.command("whoami") .command("whoami")
@@ -123,7 +132,6 @@ program
.description("Download and decrypt a single file") .description("Download and decrypt a single file")
.argument("<fileID>", "File ID (from `quak files`)") .argument("<fileID>", "File ID (from `quak files`)")
.option("--out <path>", "Output file path") .option("--out <path>", "Output file path")
.option("--collection <id>", "Accepted for compatibility; ignored")
.action((fileID: string, opts: { out?: string }) => .action((fileID: string, opts: { out?: string }) =>
run(getCommand(context(), fileID, opts)), run(getCommand(context(), fileID, opts)),
); );
@@ -133,7 +141,6 @@ program
.description("Download and decrypt a thumbnail") .description("Download and decrypt a thumbnail")
.argument("<fileID>", "File ID (from `quak files`)") .argument("<fileID>", "File ID (from `quak files`)")
.option("--out <path>", "Output file path") .option("--out <path>", "Output file path")
.option("--collection <id>", "Accepted for compatibility; ignored")
.action((fileID: string, opts: { out?: string }) => .action((fileID: string, opts: { out?: string }) =>
run(getThumbCommand(context(), fileID, opts)), run(getThumbCommand(context(), fileID, opts)),
); );
+1 -1
View File
@@ -12,7 +12,7 @@
"private": true, "private": true,
"type": "module", "type": "module",
"engines": { "engines": {
"node": ">=22" "node": "^22.13.0 || >=23.5.0"
}, },
"main": "./dist/src/index.js", "main": "./dist/src/index.js",
"types": "./dist/src/index.d.ts", "types": "./dist/src/index.d.ts",
+85
View File
@@ -0,0 +1,85 @@
#!/bin/sh
# script/build-bin: run script/build, then bundle the built dist/ into one
# executable, bin/quak, which reports the version script/build stamped. Our own
# extension to scripts-to-rule-them-all.
#
# bun compiles the binary from a copy of its own executable, so the binary
# loads the same libraries as the bun that built it. nix's bun loads libraries
# from the nix store: on Linux its C library, on macOS its ICU. A binary
# compiled from it loads them too. nix-collect-garbage can delete them, and the
# binary then no longer starts. This script therefore compiles with bun's own
# release build, which loads only the system's libraries. It downloads that
# build and checks it against the hash below before running it.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# bun 1.3.13, 2026-11-11, and the SHA-256 of each release archive below. The
# x86-64 archives are the baseline builds, which also run on CPUs without AVX2.
# The musl archives are for Linux systems that use the musl C library, such as
# Alpine.
BUN_VERSION="1.3.13"
BUN_LINUX_X64_BASELINE_SHA256="9d8a24292a7068090205daac0a5a223f5f69736f5287e37bf88d3b4031edc750"
BUN_LINUX_X64_MUSL_BASELINE_SHA256="88ca7c7ad235b498f549eea2f770f434e9f0f5e9ba95168a2d3a1f235184c394"
BUN_LINUX_AARCH64_SHA256="70bae41b3908b0a120e1e58c5c8af30e74afae3b8d11b0d3fdd8e787ddfb4b22"
BUN_LINUX_AARCH64_MUSL_SHA256="5385e978107ce4934298d8d6afe9bfbb898683f6cc23e6753a0da60bc60c5b81"
BUN_DARWIN_X64_BASELINE_SHA256="a98ba6a480f22fda9b343626b906a4e26aa53618bf85d2bc5928ecf2ba45f0ed"
BUN_DARWIN_AARCH64_SHA256="5467e3f65dba526b9fea98f0cce04efafc0c63e169733ec27b876a3ad32da190"
# compile_with_downloaded_bun <archive-name> <sha256>
compile_with_downloaded_bun() {
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT
curl -fsSL -o "$tmp/$1.zip" \
"https://github.com/oven-sh/bun/releases/download/bun-v$BUN_VERSION/$1.zip"
# macOS always has shasum, but not every macOS has sha256sum.
if command -v sha256sum >/dev/null 2>&1; then
echo "$2 $tmp/$1.zip" | sha256sum -c -
else
echo "$2 $tmp/$1.zip" | shasum -a 256 -c -
fi
unzip -q "$tmp/$1.zip" -d "$tmp"
"$tmp/$1/bun" build dist/bin/quak.js --compile --outfile bin/quak
}
main() {
cd "$ROOT"
"$ROOT/script/build"
platform="$(uname -s)-$(uname -m)"
# musl's ldd begins its version output with "musl libc"; glibc's does not.
if ldd --version 2>&1 | grep -q '^musl libc'; then
platform="$platform-musl"
fi
case "$platform" in
Linux-x86_64)
compile_with_downloaded_bun bun-linux-x64-baseline \
"$BUN_LINUX_X64_BASELINE_SHA256"
;;
Linux-x86_64-musl)
compile_with_downloaded_bun bun-linux-x64-musl-baseline \
"$BUN_LINUX_X64_MUSL_BASELINE_SHA256"
;;
Linux-aarch64)
compile_with_downloaded_bun bun-linux-aarch64 \
"$BUN_LINUX_AARCH64_SHA256"
;;
Linux-aarch64-musl)
compile_with_downloaded_bun bun-linux-aarch64-musl \
"$BUN_LINUX_AARCH64_MUSL_SHA256"
;;
Darwin-x86_64)
compile_with_downloaded_bun bun-darwin-x64-baseline \
"$BUN_DARWIN_X64_BASELINE_SHA256"
;;
Darwin-arm64)
compile_with_downloaded_bun bun-darwin-aarch64 \
"$BUN_DARWIN_AARCH64_SHA256"
;;
*)
echo "build-bin: bun has no release build for $(uname -s) on $(uname -m)" >&2
exit 1
;;
esac
}
main "$@"
+8 -1
View File
@@ -354,7 +354,13 @@ export class ApiClient {
}); });
} }
async putFile(presignedURL: string, data: Uint8Array): Promise<void> { // `contentMD5` is the value given to `getUploadURL`. The server signs it
// into the URL, and storage refuses a PUT without that `Content-MD5`.
async putFile(
presignedURL: string,
data: Uint8Array,
contentMD5: string,
): Promise<void> {
// Idempotent despite being a write: a presigned PUT stores one whole // Idempotent despite being a write: a presigned PUT stores one whole
// object at one key in one request, so replaying it either overwrites // object at one key in one request, so replaying it either overwrites
// the same bytes or lands them for the first time. There is no partial // the same bytes or lands them for the first time. There is no partial
@@ -370,6 +376,7 @@ export class ApiClient {
headers: { headers: {
"Content-Type": "application/octet-stream", "Content-Type": "application/octet-stream",
"Content-Length": String(data.length), "Content-Length": String(data.length),
"Content-MD5": contentMD5,
}, },
body: data, body: data,
signal: AbortSignal.timeout(this.requestTimeoutMs), signal: AbortSignal.timeout(this.requestTimeoutMs),
+26 -1
View File
@@ -155,8 +155,31 @@ const openReadLibrary = (ctx: CliContext, client: Client): Promise<Library> =>
onProgress: logRefresh(ctx.log), onProgress: logRefresh(ctx.log),
}); });
export const loginCommand = async (ctx: CliContext): Promise<number> => { // Logs in to the server `--server` or `QUAK_SERVER` names, the flag first, or
// to Ente's hosted service when neither is set. The saved session keeps the
// server, so every later command uses it.
// Whether `text` can be an Ente server's address: an http or https URL, which
// may have a path, and no query or fragment, which request URLs could not
// carry.
const isServerAddress = (text: string): boolean =>
URL.canParse(text) &&
["http:", "https:"].includes(new URL(text).protocol) &&
!/[?#]/.test(text);
export const loginCommand = async (
ctx: CliContext,
opts: { server?: string },
): Promise<number> => {
await init(); await init();
const server = opts.server ?? process.env.QUAK_SERVER;
if (server !== undefined && !isServerAddress(server)) {
ctx.log(
"fatal",
`Invalid server address "${server}": it must be an http or ` +
`https URL with no query or fragment`,
);
return 1;
}
const email = process.env.QUAK_EMAIL ?? (await ctx.prompt("Email")); const email = process.env.QUAK_EMAIL ?? (await ctx.prompt("Email"));
const password = const password =
process.env.QUAK_PASSWORD ?? (await ctx.promptSecret("Password")); process.env.QUAK_PASSWORD ?? (await ctx.promptSecret("Password"));
@@ -168,6 +191,8 @@ export const loginCommand = async (ctx: CliContext): Promise<number> => {
password, password,
totp: async () => ctx.prompt("TOTP code: "), totp: async () => ctx.prompt("TOTP code: "),
emailOTP: async () => ctx.prompt("Email verification code: "), emailOTP: async () => ctx.prompt("Email verification code: "),
apiOptions:
server === undefined ? undefined : { apiOrigin: server },
}); });
saveSession(ctx.sessionDir, client.toJSON()); saveSession(ctx.sessionDir, client.toJSON());
+23 -1
View File
@@ -40,6 +40,9 @@ export interface ClientSnapshot {
masterKey: string; masterKey: string;
secretKey: string; secretKey: string;
publicKey: string; publicKey: string;
// The `apiOrigin` the client was given, so a restored client talks to
// the server that issued its token. Absent means the default server.
apiOrigin?: string;
} }
// The result of a resumable enumeration. Live decrypted records and deleted // The result of a resumable enumeration. Live decrypted records and deleted
@@ -73,6 +76,7 @@ export class Client {
private readonly masterKey: Uint8Array; private readonly masterKey: Uint8Array;
private readonly secretKey: Uint8Array; private readonly secretKey: Uint8Array;
private readonly publicKey: Uint8Array; private readonly publicKey: Uint8Array;
private readonly apiOrigin: string | undefined;
private loggedOut = false; private loggedOut = false;
private constructor( private constructor(
@@ -82,6 +86,7 @@ export class Client {
masterKey: Uint8Array, masterKey: Uint8Array,
secretKey: Uint8Array, secretKey: Uint8Array,
publicKey: Uint8Array, publicKey: Uint8Array,
apiOrigin: string | undefined,
) { ) {
this.api = api; this.api = api;
this.email = email; this.email = email;
@@ -89,6 +94,7 @@ export class Client {
this.masterKey = masterKey; this.masterKey = masterKey;
this.secretKey = secretKey; this.secretKey = secretKey;
this.publicKey = publicKey; this.publicKey = publicKey;
this.apiOrigin = apiOrigin;
} }
static async login(opts: LoginOptions): Promise<Client> { static async login(opts: LoginOptions): Promise<Client> {
@@ -130,12 +136,15 @@ export class Client {
unwrapped.masterKey, unwrapped.masterKey,
unwrapped.secretKey, unwrapped.secretKey,
unwrapped.publicKey, unwrapped.publicKey,
opts.apiOptions?.apiOrigin,
); );
} }
// Restore a client from a `toJSON()` snapshot. The snapshot usually comes // Restore a client from a `toJSON()` snapshot. The snapshot usually comes
// straight from `JSON.parse` of a file on disk, so every field is checked // straight from `JSON.parse` of a file on disk, so every field is checked
// before use; a bad one throws an error naming it. Needs `init()` first. // before use; a bad one throws an error naming it. The snapshot's
// `apiOrigin`, when it has one, is used instead of `apiOptions.apiOrigin`.
// Needs `init()` first.
static fromJSON(snapshot: unknown, apiOptions?: ApiClientOptions): Client { static fromJSON(snapshot: unknown, apiOptions?: ApiClientOptions): Client {
const invalid = (field: string, problem: string): Error => const invalid = (field: string, problem: string): Error =>
new Error(`Invalid session data: ${field} ${problem}`); new Error(`Invalid session data: ${field} ${problem}`);
@@ -152,6 +161,14 @@ export class Client {
if (!Number.isInteger(s.userID)) { if (!Number.isInteger(s.userID)) {
throw invalid("userID", "must be an integer"); throw invalid("userID", "must be an integer");
} }
if (
s.apiOrigin !== undefined &&
(typeof s.apiOrigin !== "string" || s.apiOrigin === "")
) {
throw invalid("apiOrigin", "must be a non-empty string");
}
const apiOrigin =
(s.apiOrigin as string | undefined) ?? apiOptions?.apiOrigin;
const key = (field: string): Uint8Array => { const key = (field: string): Uint8Array => {
const value = s[field]; const value = s[field];
if (typeof value !== "string") { if (typeof value !== "string") {
@@ -175,6 +192,7 @@ export class Client {
const api = new ApiClient({ const api = new ApiClient({
...apiOptions, ...apiOptions,
apiOrigin,
authToken: s.token as string, authToken: s.token as string,
}); });
return new Client( return new Client(
@@ -184,6 +202,7 @@ export class Client {
key("masterKey"), key("masterKey"),
key("secretKey"), key("secretKey"),
key("publicKey"), key("publicKey"),
apiOrigin,
); );
} }
@@ -222,6 +241,9 @@ export class Client {
masterKey: toBase64(this.masterKey), masterKey: toBase64(this.masterKey),
secretKey: toBase64(this.secretKey), secretKey: toBase64(this.secretKey),
publicKey: toBase64(this.publicKey), publicKey: toBase64(this.publicKey),
...(this.apiOrigin === undefined
? {}
: { apiOrigin: this.apiOrigin }),
}; };
} }
+1 -1
View File
@@ -325,7 +325,7 @@ export const fixMissingThumbnails = async (
ciphertext.length, ciphertext.length,
md5, md5,
); );
await api.putFile(url, ciphertext); await api.putFile(url, ciphertext, md5);
await api.updateThumbnail(file.id, objectKey, toBase64(header)); await api.updateThumbnail(file.id, objectKey, toBase64(header));
log(`[${collectionName}] Thumbnail uploaded for ${title}`); log(`[${collectionName}] Thumbnail uploaded for ${title}`);
+10 -4
View File
@@ -780,7 +780,7 @@ describe("ApiClient timeouts", () => {
await client.getJSON("/a"); await client.getJSON("/a");
await client.postJSON("/b", {}); await client.postJSON("/b", {});
await client.putFile("https://s3.example/x", new Uint8Array([1])); await client.putFile("https://s3.example/x", new Uint8Array([1]), "m");
await client.putJSON("/c", {}); await client.putJSON("/c", {});
for (const call of calls) { for (const call of calls) {
@@ -825,7 +825,8 @@ describe("ApiClient timeouts", () => {
[ [
"putFile", "putFile",
() => new Response(null, { status: 200 }), () => new Response(null, { status: 200 }),
(c) => c.putFile("https://s3.example/x", new Uint8Array([1])), (c) =>
c.putFile("https://s3.example/x", new Uint8Array([1]), "m"),
], ],
[ [
"getFileStream", "getFileStream",
@@ -1049,7 +1050,7 @@ describe("ApiClient error typing", () => {
const client = new ApiClient({ fetch, retry: noWait }); const client = new ApiClient({ fetch, retry: noWait });
const err: unknown = await client const err: unknown = await client
.putFile("https://s3.example/obj", new Uint8Array([1, 2])) .putFile("https://s3.example/obj", new Uint8Array([1, 2]), "m")
.catch((e: unknown) => e); .catch((e: unknown) => e);
expect(err).toBeInstanceOf(ApiError); expect(err).toBeInstanceOf(ApiError);
@@ -1067,7 +1068,11 @@ describe("ApiClient error typing", () => {
); );
const client = new ApiClient({ fetch, retry: noWait }); const client = new ApiClient({ fetch, retry: noWait });
await client.putFile("https://s3.example/obj", new Uint8Array([1, 2])); await client.putFile(
"https://s3.example/obj",
new Uint8Array([1, 2]),
"m",
);
expect(calls).toHaveLength(2); expect(calls).toHaveLength(2);
}); });
@@ -1277,6 +1282,7 @@ describe("ApiClient onRequest", () => {
await client.putFile( await client.putFile(
"https://s3.example/obj?X-Amz-Signature=abc", "https://s3.example/obj?X-Amz-Signature=abc",
new Uint8Array([1, 2]), new Uint8Array([1, 2]),
"m",
); );
expect(requests).toEqual([ expect(requests).toEqual([
+13 -2
View File
@@ -80,13 +80,20 @@ describe("ApiClient.putFile", () => {
const client = new ApiClient({ fetch, authToken: "secret-tok" }); const client = new ApiClient({ fetch, authToken: "secret-tok" });
const data = new Uint8Array([1, 2, 3, 4, 5]); const data = new Uint8Array([1, 2, 3, 4, 5]);
await client.putFile("https://s3.example.com/presigned?sig=abc", data); await client.putFile(
"https://s3.example.com/presigned?sig=abc",
data,
"md5base64==",
);
expect(calls[0]!.url).toBe("https://s3.example.com/presigned?sig=abc"); expect(calls[0]!.url).toBe("https://s3.example.com/presigned?sig=abc");
expect(calls[0]!.init?.method).toBe("PUT"); expect(calls[0]!.init?.method).toBe("PUT");
const headers = new Headers(calls[0]!.init?.headers as HeadersInit); const headers = new Headers(calls[0]!.init?.headers as HeadersInit);
expect(headers.get("Content-Type")).toBe("application/octet-stream"); expect(headers.get("Content-Type")).toBe("application/octet-stream");
expect(headers.get("Content-Length")).toBe("5"); expect(headers.get("Content-Length")).toBe("5");
// The server signs the MD5 into the URL; storage refuses the PUT
// without it.
expect(headers.get("Content-MD5")).toBe("md5base64==");
// Must NOT leak auth headers to S3 // Must NOT leak auth headers to S3
expect(headers.has("X-Auth-Token")).toBe(false); expect(headers.has("X-Auth-Token")).toBe(false);
expect(headers.has("X-Client-Package")).toBe(false); expect(headers.has("X-Client-Package")).toBe(false);
@@ -99,7 +106,11 @@ describe("ApiClient.putFile", () => {
const client = new ApiClient({ fetch }); const client = new ApiClient({ fetch });
await expect( await expect(
client.putFile("https://s3.example.com/bad", new Uint8Array(10)), client.putFile(
"https://s3.example.com/bad",
new Uint8Array(10),
"md5base64==",
),
).rejects.toThrow(/403/); ).rejects.toThrow(/403/);
}); });
}); });
+25 -2
View File
@@ -1,8 +1,10 @@
/** /**
* Tests for the retry options `bin/quak.ts` loads the saved session with * Tests for the retry options `bin/quak.ts` loads the saved session with
* (`quak backup` gets the unattended ones, every other command the default), * (`quak backup` gets the unattended ones, every other command the default),
* each with the callbacks that log its requests and retries, and for * each with the callbacks that log its requests and retries, for `quak files`
* `quak files` taking its collection ID as an optional argument. * taking its collection ID as an optional argument, for `quak get` and
* `quak get-thumb` refusing `--collection`, and for `quak login` taking
* `--server`.
* *
* Each test runs `bin/quak.ts`, as the smoke test does, with its session loader * Each test runs `bin/quak.ts`, as the smoke test does, with its session loader
* replaced by one that records the options it is given and reports no session, * replaced by one that records the options it is given and reports no session,
@@ -105,3 +107,24 @@ describe("bin/quak.ts files", () => {
expect(await quak("files", "7")).toMatch(/^Not logged in/); expect(await quak("files", "7")).toMatch(/^Not logged in/);
}); });
}); });
describe("bin/quak.ts get and get-thumb", () => {
it.each(["get", "get-thumb"])(
"%s refuses --collection as an unknown option",
async (command) => {
expect(await quak(command, "1", "--collection", "2")).toMatch(
/^error: unknown option '--collection'/,
);
},
);
});
// The command refuses an address that is not a URL before it asks for anything
// or sends any request.
describe("bin/quak.ts login", () => {
it("hands --server to the command", async () => {
expect(await quak("login", "--server", "ente.example.com")).toMatch(
/^Invalid server address "ente\.example\.com"/,
);
});
});
+105 -6
View File
@@ -328,6 +328,7 @@ describe("login", () => {
prompts = []; prompts = [];
vi.stubEnv("QUAK_EMAIL", "cli@example.com"); vi.stubEnv("QUAK_EMAIL", "cli@example.com");
vi.stubEnv("QUAK_PASSWORD", "hunter2"); vi.stubEnv("QUAK_PASSWORD", "hunter2");
vi.stubEnv("QUAK_SERVER", undefined);
}); });
afterEach(() => { afterEach(() => {
@@ -340,12 +341,14 @@ describe("login", () => {
calls.push(opts); calls.push(opts);
return loggedIn; return loggedIn;
}); });
expect(await loginCommand(ctx)).toBe(0); expect(await loginCommand(ctx, {})).toBe(0);
expect(prompts).toEqual([]); expect(prompts).toEqual([]);
expect(calls).toHaveLength(1); expect(calls).toHaveLength(1);
expect(calls[0]!.email).toBe("cli@example.com"); expect(calls[0]!.email).toBe("cli@example.com");
expect(calls[0]!.password).toBe("hunter2"); expect(calls[0]!.password).toBe("hunter2");
// No server named: the client uses Ente's hosted service.
expect(calls[0]!.apiOptions).toBeUndefined();
const path = join(ctx.sessionDir, "session.json"); const path = join(ctx.sessionDir, "session.json");
expect(stderr.text).toBe( expect(stderr.text).toBe(
"Authenticating...\n" + "Authenticating...\n" +
@@ -356,7 +359,7 @@ describe("login", () => {
it("saves the session with mode 0600 in a directory with mode 0700", async () => { it("saves the session with mode 0600 in a directory with mode 0700", async () => {
const ctx = loginContext(async () => loggedIn); const ctx = loginContext(async () => loggedIn);
expect(await loginCommand(ctx)).toBe(0); expect(await loginCommand(ctx, {})).toBe(0);
expect(statSync(ctx.sessionDir).mode & 0o777).toBe(0o700); expect(statSync(ctx.sessionDir).mode & 0o777).toBe(0o700);
const path = join(ctx.sessionDir, "session.json"); const path = join(ctx.sessionDir, "session.json");
@@ -370,7 +373,7 @@ describe("login", () => {
code = await opts.totp!(); code = await opts.totp!();
return loggedIn; return loggedIn;
}); });
expect(await loginCommand(ctx)).toBe(0); expect(await loginCommand(ctx, {})).toBe(0);
expect(prompts).toEqual(["TOTP code: "]); expect(prompts).toEqual(["TOTP code: "]);
expect(code).toBe("123456"); expect(code).toBe("123456");
@@ -380,13 +383,68 @@ describe("login", () => {
const ctx = loginContext(async () => { const ctx = loginContext(async () => {
throw new Error("HTTP 401 from server"); throw new Error("HTTP 401 from server");
}); });
expect(await loginCommand(ctx)).toBe(1); expect(await loginCommand(ctx, {})).toBe(1);
expect(stderr.text).toBe( expect(stderr.text).toBe(
"Authenticating...\nLogin failed: HTTP 401 from server\n", "Authenticating...\nLogin failed: HTTP 401 from server\n",
); );
expect(existsSync(join(ctx.sessionDir, "session.json"))).toBe(false); expect(existsSync(join(ctx.sessionDir, "session.json"))).toBe(false);
}); });
it("logs in to the server --server names, ahead of QUAK_SERVER", async () => {
vi.stubEnv("QUAK_SERVER", "https://other.example.com");
const calls: LoginOptions[] = [];
const ctx = loginContext(async (opts) => {
calls.push(opts);
return loggedIn;
});
expect(
await loginCommand(ctx, { server: "https://ente.example.com" }),
).toBe(0);
expect(calls[0]!.apiOptions).toEqual({
apiOrigin: "https://ente.example.com",
});
});
it("logs in to the server QUAK_SERVER names without --server", async () => {
vi.stubEnv("QUAK_SERVER", "http://192.0.2.1:8080/ente");
const calls: LoginOptions[] = [];
const ctx = loginContext(async (opts) => {
calls.push(opts);
return loggedIn;
});
expect(await loginCommand(ctx, {})).toBe(0);
expect(calls[0]!.apiOptions).toEqual({
apiOrigin: "http://192.0.2.1:8080/ente",
});
});
it.each([
"",
"ente.example.com",
"ftp://ente.example.com",
"https://ente.example.com/?region=eu",
"https://ente.example.com/#top",
])(
"refuses the server address %j before asking for anything",
async (server) => {
const ctx = loginContext(async () => {
throw new Error("login must not be called");
});
expect(await loginCommand(ctx, { server })).toBe(1);
expect(prompts).toEqual([]);
expect(stderr.text).toBe(
`Invalid server address "${server}": it must be an http or ` +
`https URL with no query or fragment\n`,
);
expect(existsSync(join(ctx.sessionDir, "session.json"))).toBe(
false,
);
},
);
}); });
// These use a real client read from the session file, over a fake API that // These use a real client read from the session file, over a fake API that
@@ -569,14 +627,14 @@ describe("files", () => {
}); });
describe("get and get-thumb", () => { describe("get and get-thumb", () => {
it("get finds a file in any album without --collection", async () => { it("get finds a file in any album", async () => {
const out = join(root, "diagram.png"); const out = join(root, "diagram.png");
expect(await getCommand(context(), "200", { out })).toBe(0); expect(await getCommand(context(), "200", { out })).toBe(0);
expect(readFileSync(out)).toEqual(Buffer.alloc(7, 200)); expect(readFileSync(out)).toEqual(Buffer.alloc(7, 200));
expect(stderr.text).toBe(`7 bytes -> ${out}\n`); expect(stderr.text).toBe(`7 bytes -> ${out}\n`);
}); });
it("get-thumb finds a file in any album without --collection", async () => { it("get-thumb finds a file in any album", async () => {
const out = join(root, "thumb.jpg"); const out = join(root, "thumb.jpg");
expect(await getThumbCommand(context(), "200", { out })).toBe(0); expect(await getThumbCommand(context(), "200", { out })).toBe(0);
expect(readFileSync(out)).toEqual(Buffer.alloc(3, 200)); expect(readFileSync(out)).toEqual(Buffer.alloc(3, 200));
@@ -1851,6 +1909,47 @@ describe("an account with many albums", () => {
expect(existsSync(join(root, "cache", "metadata.json"))).toBe(false); expect(existsSync(join(root, "cache", "metadata.json"))).toBe(false);
}); });
// A self-hosted server serves downloads under its own address, at
// `/files/download/<id>`, so no request goes to Ente's hosted service.
it("login --server saves the server, and a later get sends every request to it", async () => {
vi.stubEnv("QUAK_EMAIL", "many@example.com");
vi.stubEnv("QUAK_PASSWORD", PASSWORD);
vi.stubEnv("QUAK_SERVER", undefined);
const hosts = new Set<string>();
const fetch = (async (input: RequestInfo | URL, init?: RequestInit) => {
hosts.add(
new URL(input instanceof Request ? input.url : input).host,
);
return server(input, init);
}) as typeof globalThis.fetch;
const ctx: CliContext = {
...context(),
login: (opts) =>
Client.login({
...opts,
apiOptions: { ...opts.apiOptions, fetch },
}),
loadSession: (path) => loadSession(path, { fetch }),
};
const out = join(root, "photo.jpg");
try {
const opts = { server: "https://ente.example.com" };
expect(await loginCommand(ctx, opts)).toBe(0);
requests.length = 0;
expect(await getCommand(ctx, "200", { out })).toBe(0);
} finally {
vi.unstubAllEnvs();
}
const path = join(ctx.sessionDir, "session.json");
expect(JSON.parse(readFileSync(path, "utf-8")).apiOrigin).toBe(
"https://ente.example.com",
);
expect(readFileSync(out, "utf-8")).toBe("photo-2.jpg");
expect(requests.at(-1)).toBe("/files/download/200");
expect([...hosts]).toEqual(["ente.example.com"]);
});
// `files` first fills the cache, so `get` waits for a refresh. No album has // `files` first fills the cache, so `get` waits for a refresh. No album has
// changed since, so the refresh asks for no album's files. // changed since, so the refresh asks for no album's files.
it("get on a cache that holds the account asks for no unchanged album's files, also for a photo in no album", async () => { it("get on a cache that holds the account asks for no unchanged album's files, also for a photo in no album", async () => {
+40
View File
@@ -47,6 +47,20 @@ describe("Client.toJSON", () => {
expect(Client.fromJSON(snapshot).toJSON()).toEqual(snapshot); expect(Client.fromJSON(snapshot).toJSON()).toEqual(snapshot);
}); });
it("round-trips a snapshot that names a server", () => {
const snapshot = {
...validSnapshot(),
apiOrigin: "https://ente.example.com",
};
expect(Client.fromJSON(snapshot).toJSON()).toEqual(snapshot);
});
it("names the server apiOptions gave fromJSON", () => {
const apiOrigin = "https://ente.example.com";
const client = Client.fromJSON(validSnapshot(), { apiOrigin });
expect(client.toJSON().apiOrigin).toBe(apiOrigin);
});
it("throws instead of emitting a snapshot without a token", () => { it("throws instead of emitting a snapshot without a token", () => {
const client = Client.fromJSON(validSnapshot()); const client = Client.fromJSON(validSnapshot());
client.getApiClient().clearAuthToken(); client.getApiClient().clearAuthToken();
@@ -77,6 +91,8 @@ describe("Client.fromJSON", () => {
["publicKey", undefined], ["publicKey", undefined],
["publicKey", "not base64!"], ["publicKey", "not base64!"],
["publicKey", shortKey], ["publicKey", shortKey],
["apiOrigin", 7],
["apiOrigin", ""],
])("rejects %s = %j, naming the field", (field, value) => { ])("rejects %s = %j, naming the field", (field, value) => {
const snapshot: Record<string, unknown> = { ...validSnapshot() }; const snapshot: Record<string, unknown> = { ...validSnapshot() };
snapshot[field] = value; snapshot[field] = value;
@@ -85,6 +101,30 @@ describe("Client.fromJSON", () => {
); );
}); });
// The token belongs to the snapshot's server, so that server wins over
// the one in `apiOptions`.
it("sends requests to the snapshot's server", async () => {
const urls: string[] = [];
const client = Client.fromJSON(
{ ...validSnapshot(), apiOrigin: "https://ente.example.com/" },
{
apiOrigin: "https://other.example.com",
fetch: async (input) => {
urls.push(String(input));
return new Response(JSON.stringify({ collections: [] }), {
status: 200,
headers: { "content-type": "application/json" },
});
},
},
);
await client.listCollections();
expect(urls).toEqual([
"https://ente.example.com/collections/v2?sinceTime=0",
]);
});
it.each([null, "a string", 42])("rejects a non-object %j", (value) => { it.each([null, "a string", 42])("rejects a non-object %j", (value) => {
expect(() => Client.fromJSON(value)).toThrow( expect(() => Client.fromJSON(value)).toThrow(
/^Invalid session data: not a JSON object/, /^Invalid session data: not a JSON object/,
+241
View File
@@ -0,0 +1,241 @@
// `script/build-bin` compiles `bin/quak`, the single binary that the README's
// cron setup runs. bun compiles it from a copy of its own executable, so the
// binary loads the same libraries as the bun that built it. nix's bun loads
// libraries from the nix store, which garbage collection can delete: on Linux
// its C library, on macOS its ICU. The script therefore downloads bun's own
// release build, which loads only the system's libraries.
//
// What the script promises, and why each test matters:
//
// - Each system, machine and C library that bun publishes a build for gets its
// own archive: Linux on x86-64 or arm64, with glibc or with musl, as on
// Alpine, and macOS on Intel or Apple silicon. A bun built for another
// system, machine or C library does not start, and `make install` fails.
// - The archive's SHA-256 must match the one in the script before anything in
// the archive runs. Without that check, a tampered download would build the
// binary that cron runs.
// - Any other system or machine stops with an error, and never falls back to
// nix's bun. Building with nix's bun would bring back the dependency on the
// nix store.
//
// Each test copies the script into a fresh directory, which the script then
// treats as the checkout, and runs it there. Stubs first on PATH stand in for
// `uname`, `ldd`, `curl`, `unzip`, `nix-shell` and `shasum`, so no test
// downloads anything or runs a real bun.
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { execFileSync, spawnSync } from "node:child_process";
import {
chmodSync,
copyFileSync,
existsSync,
mkdirSync,
mkdtempSync,
readFileSync,
rmSync,
symlinkSync,
writeFileSync,
} from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
const repoRoot = fileURLToPath(new URL("../../", import.meta.url));
// The stubs, keyed by the command each replaces. `uname` and `ldd` describe the
// system that runBuildBin passes in the environment. `curl`, `unzip`,
// `nix-shell` and `shasum` write the arguments they were given, one per line,
// to a file named after the command in $STUB_LOG_DIR, which also shows that
// they ran.
const stubs: Record<string, string> = {
uname: `
case "$1" in
-s) echo "$STUB_UNAME_S" ;;
-m) echo "$STUB_UNAME_M" ;;
esac`,
// The script looks only at the start of the line: "musl libc" or not.
ldd: `
if [ "$STUB_LIBC" = musl ]; then
echo "musl libc (x86_64)"
else
echo "ldd (GNU libc) 2.36"
fi`,
// Writes a file that is not the archive to the path after -o.
curl: `
printf '%s\\n' "$@" > "$STUB_LOG_DIR/curl"
while [ $# -gt 1 ]; do
if [ "$1" = -o ]; then echo "not the archive" > "$2"; fi
shift
done`,
// Unpacks a stub bun where the real archive puts bun: in a directory named
// after the archive, inside the directory after -d. That bun writes
// bin/quak, as a real compile would.
unzip: `
printf '%s\\n' "$@" > "$STUB_LOG_DIR/unzip"
for arg in "$@"; do
case "$arg" in *.zip) archive="$arg" ;; esac
if [ "$previous" = -d ]; then directory="$arg"; fi
previous="$arg"
done
bun="$directory/$(basename "$archive" .zip)/bun"
mkdir -p "$(dirname "$bun")"
echo '#!/bin/sh' > "$bun"
echo 'mkdir -p bin && echo binary > bin/quak' >> "$bun"
chmod +x "$bun"`,
"nix-shell": `
printf '%s\\n' "$@" > "$STUB_LOG_DIR/nix-shell"`,
// The script checks the hash with `shasum` where there is no `sha256sum`.
// The stub fails, as `shasum` does when the hash does not match.
shasum: `
printf '%s\\n' "$@" > "$STUB_LOG_DIR/shasum"
exit 1`,
};
let checkout = "";
const writeScript = (path: string, body: string): void => {
writeFileSync(path, `#!/bin/sh\n${body}\n`);
chmodSync(path, 0o755);
};
// A checkout holding the script, a stub script/build that does nothing, the
// stubs, and the directory they log to.
beforeEach(() => {
checkout = mkdtempSync(join(tmpdir(), "quak-build-bin-"));
mkdirSync(join(checkout, "script"));
copyFileSync(
join(repoRoot, "script/build-bin"),
join(checkout, "script/build-bin"),
);
chmodSync(join(checkout, "script/build-bin"), 0o755);
writeScript(join(checkout, "script/build"), "exit 0");
mkdirSync(join(checkout, "stubs"));
for (const [command, body] of Object.entries(stubs)) {
writeScript(join(checkout, "stubs", command), body);
}
mkdirSync(join(checkout, "log"));
});
afterEach(() => {
rmSync(checkout, { recursive: true, force: true });
});
// Runs the script with nothing in its environment but the stubs' settings and a
// PATH. Unless the caller passes one, the PATH finds the stubs first and then
// the host's commands.
const runBuildBin = (
system: string,
machine: string,
{
libc = "glibc",
path = `${join(checkout, "stubs")}:${process.env.PATH}`,
} = {},
) =>
spawnSync(join(checkout, "script/build-bin"), {
cwd: checkout,
encoding: "utf-8",
env: {
PATH: path,
STUB_UNAME_S: system,
STUB_UNAME_M: machine,
STUB_LIBC: libc,
STUB_LOG_DIR: join(checkout, "log"),
},
});
// The real commands that the script and the stubs run.
const hostCommands = [
"basename",
"chmod",
"dirname",
"grep",
"mkdir",
"mktemp",
"rm",
];
// A Mac has no `ldd`, and not every Mac has `sha256sum`. This makes a PATH that
// finds neither: the stubs other than `ldd`, then links to hostCommands.
const makeMacPath = (): string => {
rmSync(join(checkout, "stubs", "ldd"));
mkdirSync(join(checkout, "host-commands"));
for (const command of hostCommands) {
const target = execFileSync("sh", ["-c", `command -v ${command}`], {
encoding: "utf-8",
}).trim();
symlinkSync(target, join(checkout, "host-commands", command));
}
return `${join(checkout, "stubs")}:${join(checkout, "host-commands")}`;
};
const ran = (command: string): boolean =>
existsSync(join(checkout, "log", command));
const argumentsOf = (command: string): string[] =>
readFileSync(join(checkout, "log", command), "utf-8")
.trimEnd()
.split("\n");
describe("script/build-bin", () => {
// The stub curl writes a file that is not the archive, so the hash check
// must stop the script before unzip runs. A script without the check would
// run unzip, and the stub bun it unpacks would write bin/quak. On Linux the
// check must use `sha256sum`, since a stock Alpine has no `shasum`.
it.each([
["Linux", "x86_64", "glibc", "bun-linux-x64-baseline.zip"],
["Linux", "x86_64", "musl", "bun-linux-x64-musl-baseline.zip"],
["Linux", "aarch64", "glibc", "bun-linux-aarch64.zip"],
["Linux", "aarch64", "musl", "bun-linux-aarch64-musl.zip"],
])(
"on %s %s with %s, downloads %s and stops when its hash does not match",
(system, machine, libc, archive) => {
const result = runBuildBin(system, machine, { libc });
expect(argumentsOf("curl")).toContain(
"https://github.com/oven-sh/bun/releases/download/bun-v1.3.13/" +
archive,
);
expect(ran("shasum")).toBe(false);
expect(result.status).not.toBe(0);
expect(ran("unzip")).toBe(false);
expect(ran("nix-shell")).toBe(false);
expect(existsSync(join(checkout, "bin/quak"))).toBe(false);
},
);
// On the PATH from makeMacPath the script finds no `sha256sum` and checks
// the hash with the stub `shasum`. That stub fails, so the script must stop
// before unzip runs.
// On Apple silicon `uname -m` says arm64, where bun's archive says aarch64.
it.each([
["Darwin", "x86_64", "bun-darwin-x64-baseline.zip"],
["Darwin", "arm64", "bun-darwin-aarch64.zip"],
])(
"on %s %s, downloads %s and stops when shasum rejects its hash",
(system, machine, archive) => {
const result = runBuildBin(system, machine, {
path: makeMacPath(),
});
expect(argumentsOf("curl")).toContain(
"https://github.com/oven-sh/bun/releases/download/bun-v1.3.13/" +
archive,
);
expect(argumentsOf("shasum")).toEqual(["-a", "256", "-c", "-"]);
expect(result.status).not.toBe(0);
expect(ran("unzip")).toBe(false);
expect(ran("nix-shell")).toBe(false);
expect(existsSync(join(checkout, "bin/quak"))).toBe(false);
},
);
it.each([
["Linux", "riscv64"],
["FreeBSD", "amd64"],
])(
"stops on %s %s, which bun publishes no build for",
(system, machine) => {
const result = runBuildBin(system, machine);
expect(result.status).not.toBe(0);
expect(ran("curl")).toBe(false);
expect(ran("nix-shell")).toBe(false);
},
);
});
+8 -1
View File
@@ -273,6 +273,7 @@ const buildThumbMock = async (opts?: {
const buildThumbFetch = (m: ThumbMockState) => { const buildThumbFetch = (m: ThumbMockState) => {
let srpServer: SrpServer; let srpServer: SrpServer;
let signedMD5: string | undefined;
return (async ( return (async (
input: RequestInfo | URL, input: RequestInfo | URL,
@@ -365,8 +366,9 @@ const buildThumbFetch = (m: ThumbMockState) => {
return new Response("not found", { status: 404 }); return new Response("not found", { status: 404 });
} }
// Upload URL minting // Upload URL minting. The real server signs the MD5 into the URL.
if (path === "/files/upload-url") { if (path === "/files/upload-url") {
signedMD5 = JSON.parse(init?.body as string).contentMD5;
return json({ return json({
objectKey: `42/thumb-${Date.now()}`, objectKey: `42/thumb-${Date.now()}`,
url: "https://s3.mock.test/presigned-put", url: "https://s3.mock.test/presigned-put",
@@ -375,6 +377,11 @@ const buildThumbFetch = (m: ThumbMockState) => {
// Presigned PUT (S3 upload) // Presigned PUT (S3 upload)
if (url.startsWith("https://s3.mock.test/")) { if (url.startsWith("https://s3.mock.test/")) {
// Storage refuses a PUT that lacks the signed `Content-MD5`.
const headers = new Headers(init?.headers);
if (headers.get("Content-MD5") !== signedMD5) {
return new Response("AccessDenied", { status: 400 });
}
const body = init?.body; const body = init?.body;
// Store the uploaded bytes for later inspection // Store the uploaded bytes for later inspection
if (body instanceof Uint8Array) { if (body instanceof Uint8Array) {