5 Commits

Author SHA1 Message Date
156fe871e8 Make the image build multi-stage and cache-proof (closes #4)
All checks were successful
check / check (push) Successful in 23s
The image build reported a green it had not earned. `script/cibuild` is a
bare `docker build .`, and with `COPY . .` followed by `RUN make check`,
an unchanged tree served that layer from cache: the suite never ran and
the build still exited 0, while the script's header comment asserted the
opposite.

CHECK_EPOCH, passed by `script/cibuild` and `script/docker`, changes the
cache key of the check and build layers on every invocation. It is
guarded, because an unset ARG is the empty string and therefore a stable
key: without the guard a plain `docker build .` — the command the policy
names, and the one anyone debugging types — would still get the false
green. A missing argument is now a hard failure rather than a silent
degradation to the behaviour the epoch was added to prevent.

The Dockerfile is now two stages: `fmt-check` and `lint` run first, and
the check stage takes a `COPY --from=lint` dependency on them, so a
formatting mistake fails the build in seconds instead of racing the
suite to the finish. Both stages stay pinned to the same digest.

The remaining fixes are one-liners that had made the target unusable:
`script/projectname` still printed the pre-rename name, so `make docker`
tagged its image after a name this project dropped in May;
`script/bootstrap` installed without fetching apt's package lists, which
cannot work on a Debian base; and `.dockerignore` had drifted far enough
from `.gitignore` to ship a ~100 MB compiled binary and any agent
worktree under `.claude/` into the build context. The second of those is
a correctness problem, not a size one — vitest globs a copied worktree's
tests alongside the real ones and runs the suite twice over. `.gitignore`
itself stays in the context, because prettier reads it as a default
ignore file and dropping it would change what `make fmt-check` sees.
2026-08-09 14:38:32 +00:00
118f8e22c3 Add failing tests for the project name and build context
script/projectname still prints the pre-rename name, so make docker tags
its image quack, and .dockerignore has drifted from .gitignore: a compiled
bin/quak, the vitest and tsc caches, the CLI's runtime directory and the
local worktree directory all reach the build context. Neither failure is
visible in a build that exits 0.

Red until the fixes land.
2026-08-09 14:27:50 +00:00
69bd6d1539 Compile bin/ alongside src/ so the package can be built (closes #3)
All checks were successful
check / check (push) Successful in 5s
rootDir was ./src while include also matched bin/**/*, which is TS6059: tsc
refuses to emit at all when a compiled file sits outside rootDir. rootDir is
now the repository root, which is the smallest change that makes the two
agree and leaves the source layout the README documents alone. Output keeps
the shape of the source tree, so main and types move to dist/src/index.js and
dist/src/index.d.ts while bin.quak stays at dist/bin/quak.js. The alternative,
moving the CLI body into src/ behind a shim in bin/, would hold main at
dist/index.js at the cost of churning the CLI and contradicting the layout
diagram in the README.

Clearing TS6059 exposed two type errors that had never been reached, because
the config error aborts before checking: StateAddress was read as a namespace
member off the default import, and the secretstream pull was called without
the additional-data argument, which libsodium does not make optional. The type
is now taken from the module's named export and the pull passes null for ad,
matching the null already passed on the push side in encryptBlob. Neither
changes what runs.

noEmitOnError stops a failed build from leaving output behind. It emitted
despite the error before, which is how a stale bin/quak.js came to sit next to
bin/quak.ts in a working tree, where eslint then read it and failed make check
on a generated file.

script/build compiles and then checks that the files package.json advertises
are among the ones the compiler wrote, since tsc knows nothing about the
manifest and a green build could still ship a package whose main resolves to
nothing. It also sets the executable bit on the bin entries, which tsc does
not carry over from the source even though it does copy the shebang. The
Makefile target is now a shim over it, as the other targets are, and
package.json's build script points at it so yarn build gets the same checks.

The Dockerfile runs make build after make check, so a branch that does not
compile cannot reach main. What make check itself runs is unchanged.

package.json gains a quak script, so the yarn quak commands the README's
Getting Started block has always listed resolve to the built CLI.
2026-08-09 10:06:31 +00:00
d79ed83f4d Add failing tests for the package entrypoint contract
The manifests disagreed and nothing noticed. tsconfig.json set rootDir to
./src while include also matched bin/**/*, which is TS6059, so no build had
succeeded; package.json meanwhile advertised main, types and a bin that a
successful build would have to produce. make check runs test, lint and
fmt-check, so neither half was ever exercised.

These tests read tsconfig.json and package.json and assert the contract
between them without invoking a compiler, which keeps them in the fast unit
suite: every include pattern must root under rootDir, and main, types and
bin.quak must equal the paths tsc will emit for src/index.ts and bin/quak.ts.
They also require a quak script pointing at the built CLI, which the README's
Getting Started block has always told the reader to run.

Three of them fail at this commit.
2026-08-09 10:01:32 +00:00
348f23bac9 Narrow the replay errno set to failures that prove no connection existed
All checks were successful
check / check (push) Successful in 4s
`CONNECT_CODES` drives `isSafeToReplay`, which is the only thing standing
between a transport failure and a replayed `POST /users/two-factor/verify`.
It included `EHOSTUNREACH`, `ENETUNREACH` and `ENETDOWN` on the stated
grounds that those errnos can only be reported before any request byte was
written. That is not true on Linux: an ICMP destination-unreachable
delivered on an already-established connection sets the socket error and
the next read or write returns `EHOSTUNREACH` or `ENETUNREACH`, and a local
interface going down after the request was fully written surfaces as
`ENETDOWN` the same way. In each case the server may already have received
and acted on the request -- exactly the ambiguity the rule exists to
exclude, on the paths that consume a second-factor attempt or register a
thumbnail.

The three are dropped from `CONNECT_CODES` and stay in `TRANSPORT_CODES`,
so they remain retryable for the idempotent calls; only replay eligibility
narrows. What is left -- `ENOTFOUND`, `EAI_AGAIN`, `ECONNREFUSED` -- means
no TCP connection to the server ever existed, so no request byte can have
been transmitted.

The justification is corrected everywhere it was stated: the comment on
`CONNECT_CODES`, the one on `isSafeToReplay`, the `postJSON` call site, the
README's idempotency section and the `client.test.ts` docblock. All of them
now describe what the narrowed set actually establishes rather than
claiming a proof it did not support.

The narrowing is enforced by the suite rather than asserted in a comment:
the three errnos join `ECONNRESET`/`EPIPE`/`ETIMEDOUT` in the
`isSafeToReplay`-returns-false test, with companion `isRetryable` assertions
so a future edit cannot make them non-retryable by accident. Putting the
three back into `CONNECT_CODES` turns that test red (1 failure, verified).
2026-08-09 05:41:59 +00:00
21 changed files with 478 additions and 68 deletions

View File

@@ -1,8 +1,50 @@
# Mirrors .gitignore, with one deliberate exception: .gitignore itself stays
# in the build context, because prettier 3 reads it as a default ignore file
# and dropping it would change what `make fmt-check` sees inside the image.
# VCS
.git .git
# OS
.DS_Store
Thumbs.db
# Editors
*.swp
*.swo
*~
*.bak
.idea/
.vscode/
*.sublime-*
# Node
node_modules node_modules
# TypeScript / build artifacts
dist dist
build build
*.tsbuildinfo
coverage coverage
.DS_Store .nyc_output/
# Vitest
.vitest-cache/
# Environment / secrets
.env .env
.env.* .env.*
*.pem
*.key
# Compiled binary (built by make build-bin); around 100 MB
bin/quak
# quak runtime data (in case anyone runs the CLI from inside the repo)
.quak/
# Local per-developer tool state, including agent worktrees. Correctness,
# not context size: a worktree copied in here has its own test/ tree, which
# vitest globs alongside the real one, so the containerised suite runs N+1
# times over and still reports success.
.claude/

3
.gitignore vendored
View File

@@ -36,5 +36,6 @@ bin/quak
# quak runtime data (in case anyone runs the CLI from inside the repo) # quak runtime data (in case anyone runs the CLI from inside the repo)
.quak/ .quak/
# Local Claude Code settings (per-developer) # Local per-developer tool settings and scratch state, including the
# worktrees agents check out under this directory
.claude/ .claude/

View File

@@ -1,10 +1,37 @@
# node 22-alpine, 2026-02-22 # Lint stage — fast feedback on formatting and lint issues
FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 # node 22.22.0 on Alpine 3.23.3 (node:22-alpine), 2026-08-09
FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS lint
WORKDIR /app WORKDIR /app
COPY script/ script/ COPY script/ script/
COPY package.json yarn.lock ./ COPY package.json yarn.lock ./
RUN script/bootstrap RUN script/bootstrap
COPY . . COPY . .
RUN make fmt-check
RUN make lint
# Check stage — the full suite and the build
# node 22.22.0 on Alpine 3.23.3 (node:22-alpine), 2026-08-09
FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS check
WORKDIR /app
# Force BuildKit to run the lint stage before proceeding. Without this the
# two stages run in parallel and a lint failure can lose the race.
COPY --from=lint /app/yarn.lock /dev/null
COPY script/ script/
COPY package.json yarn.lock ./
RUN script/bootstrap
COPY . .
# CHECK_EPOCH is a cache buster: without it Docker serves `make check` from
# cache on an unchanged tree, the suite never executes, and the build still
# exits 0. The guard makes an absent argument a hard failure — an unset ARG
# is the empty string, which is a perfectly stable cache key, so a plain
# `docker build .` would otherwise still get the false green. Fail closed.
ARG CHECK_EPOCH
RUN [ -n "$CHECK_EPOCH" ] || exit 1
RUN make check RUN make check
ARG CHECK_EPOCH
RUN [ -n "$CHECK_EPOCH" ] || exit 1
RUN make build

View File

@@ -24,7 +24,7 @@ check:
@script/check @script/check
build: build:
@$(YARN) tsc @script/build
build-bin: build-bin:
nix-shell -p bun --run "bun build bin/quak.ts --compile --outfile bin/quak" nix-shell -p bun --run "bun build bin/quak.ts --compile --outfile bin/quak"

View File

@@ -81,6 +81,9 @@ alpine. We provide:
`script/bootstrap`, then `script/install-precommit` `script/bootstrap`, then `script/install-precommit`
- `script/projectname` — output the project name (our own extension); used by - `script/projectname` — output the project name (our own extension); used by
`script/docker` for the image tag `script/docker` for the image tag
- `script/build` — compile the TypeScript sources into `dist/`, then verify that
the entrypoints `package.json` declares (`main`, `types`, `bin`) are among the
files the compiler wrote, and make the CLI executable (our own extension)
- `script/test` — run the test suite (vitest, hard-capped at 30s where `timeout` - `script/test` — run the test suite (vitest, hard-capped at 30s where `timeout`
is available, verbose rerun on failure) is available, verbose rerun on failure)
- `script/lint` — run eslint and a prettier check - `script/lint` — run eslint and a prettier check
@@ -89,9 +92,9 @@ alpine. We provide:
- `script/check` — run all checks: `test`, `lint`, `fmt-check` (our own - `script/check` — run all checks: `test`, `lint`, `fmt-check` (our own
extension) extension)
- `script/docker` — build the Docker image, tagged via `script/projectname` - `script/docker` — build the Docker image, tagged via `script/projectname`
(byte-identical across repos) - `script/cibuild` — cd to the repo root and run the image build (what CI runs;
- `script/cibuild` — cd to the repo root and `docker build .` (what CI runs; the the build runs `make fmt-check` and `make lint` in a first stage, then
image build runs `make check`) `make check` and `make build` in a second)
- `script/precommit` — run by the git pre-commit hook (our own extension); runs - `script/precommit` — run by the git pre-commit hook (our own extension); runs
`script/lint` and `script/fmt-check` but deliberately not the tests, so the `script/lint` and `script/fmt-check` but deliberately not the tests, so the
TDD red-phase commit can land TDD red-phase commit can land
@@ -100,6 +103,15 @@ alpine. We provide:
`make hooks` installs the pre-commit hook that runs `script/precommit`. `make hooks` installs the pre-commit hook that runs `script/precommit`.
Both `script/docker` and `script/cibuild` pass
`--build-arg CHECK_EPOCH="$(date +%s)"`. The Dockerfile refuses to build without
it. This is deliberate: on an unchanged tree Docker would otherwise serve the
`make check` layer from cache, so the suite would never run and the build would
still exit 0. A changing epoch invalidates the check and build layers on every
invocation while leaving the dependency layers below them cached, and the
missing-argument guard means a bare `docker build .` fails loudly instead of
quietly reporting a green it did not earn.
## Rationale ## Rationale
Ente is one of very few photo services with a credible end-to-end encryption Ente is one of very few photo services with a credible end-to-end encryption
@@ -133,8 +145,8 @@ All work on quak is test-driven. No exceptions.
3. Subsequent commits add the implementation and any refactors needed to make 3. Subsequent commits add the implementation and any refactors needed to make
the tests pass. the tests pass.
4. A feature branch can only be merged into `main` when `make check` is green. 4. A feature branch can only be merged into `main` when `make check` is green.
`main` is always green. The Dockerfile runs `make check`, so a red branch `main` is always green. The Dockerfile runs `make check` and `make build`, so
cannot pass CI. neither a red branch nor one that does not compile can pass CI.
5. Tests are the canonical API documentation for this library. Every test file 5. Tests are the canonical API documentation for this library. Every test file
is commented thoroughly enough that a reader who has never seen quak can is commented thoroughly enough that a reader who has never seen quak can
learn how to use it from the tests alone. Comments explain why a behavior learn how to use it from the tests alone. Comments explain why a behavior
@@ -150,8 +162,9 @@ All work on quak is test-driven. No exceptions.
8. The pre-commit hook installed by `make hooks` runs `script/precommit`, which 8. The pre-commit hook installed by `make hooks` runs `script/precommit`, which
runs the lint and format checks but not the full `make check`. This is runs the lint and format checks but not the full `make check`. This is
deliberate so the TDD red-phase commit (failing tests, no implementation yet) deliberate so the TDD red-phase commit (failing tests, no implementation yet)
can land. The full `make check` runs as part of `docker build .`, which is can land. The full `make check` runs as part of the image build, which is
what CI executes, so a red branch still cannot reach `main`. what CI executes via `script/cibuild`, so a red branch still cannot reach
`main`.
## Design ## Design
@@ -183,6 +196,12 @@ quak/
tsconfig.json tsconfig.json
``` ```
`make build` compiles that tree into `dist/`, preserving its shape: the library
lands in `dist/src/` and the CLI in `dist/bin/quak.js`, which is what
`package.json` points `main`, `types` and `bin` at. The compiler's `rootDir` is
the repository root rather than `src/`, because `bin/` is compiled too and
`rootDir` has to contain everything that is compiled.
### Cryptography ### Cryptography
All cryptography is done by `libsodium-wrappers-sumo` (the "sumo" build is All cryptography is done by `libsodium-wrappers-sumo` (the "sumo" build is
@@ -305,12 +324,18 @@ only guarded the initial request would leave the same hang one layer down.
**Non-idempotent requests are not blindly replayed.** `postJSON` and `putJSON` **Non-idempotent requests are not blindly replayed.** `postJSON` and `putJSON`
reach `/users/srp/create-session`, `/users/two-factor/verify` — which consumes reach `/users/srp/create-session`, `/users/two-factor/verify` — which consumes
one of a small number of second-factor attempts — and `/files/thumbnail`. They one of a small number of second-factor attempts — and `/files/thumbnail`. They
are retried only on failures that prove no request byte reached the server, are retried only on the three failures that establish no TCP connection to the
which means the connection was never established (`ECONNREFUSED`, `ENOTFOUND`, server ever existed, so no request byte can have been transmitted: `ENOTFOUND`
and the like). A 5xx, a mid-flight reset and a deadline are all left to the and `EAI_AGAIN` (name resolution produced no address) and `ECONNREFUSED` (the
caller, because each of them can happen after the server has already acted. peer refused the connection). A 5xx, a mid-flight reset and a deadline are all
`putFile` is exempt: a presigned PUT stores one whole object at one key in one left to the caller, because each of them can happen after the server has already
request, so replaying it has no partial state to damage. acted. The routing errnos `EHOSTUNREACH`, `ENETUNREACH` and `ENETDOWN` are
excluded for the same reason, despite looking like connect-time failures: on
Linux an ICMP unreachable arriving mid-flight, or a local interface going down
after the request was written, delivers them on an already-established socket.
They stay retryable for the idempotent calls. `putFile` is exempt: a presigned
PUT stores one whole object at one key in one request, so replaying it has no
partial state to damage.
A download is retried as a whole — request, stream consumption, and decryption — A download is retried as a whole — request, stream consumption, and decryption —
because a socket reset after the response headers have arrived surfaces in the because a socket reset after the response headers have arrived surfaces in the
@@ -394,7 +419,7 @@ code is non-zero if any files failed.
- [x] Retry policy: no retry on 4xx, exponential backoff on 5xx and network - [x] Retry policy: no retry on 4xx, exponential backoff on 5xx and network
errors errors
- [ ] Update the API reference section below to match the current implementation - [ ] Update the API reference section below to match the current implementation
- [ ] `make docker` green - [x] `make docker` green
- [ ] Tag `v1.0.0` - [ ] Tag `v1.0.0`
Future (desktop client, separate repo): Future (desktop client, separate repo):

18
TODO.md
View File

@@ -18,6 +18,23 @@ Update the README API reference section to match the current implementation.
# Completed Steps # Completed Steps
- 2026-08-09: Made `make docker` green and policy-conformant. Multi-stage
Dockerfile: a lint stage runs `make fmt-check` and `make lint`, and the check
stage takes a `COPY --from=lint` dependency on it before running `make check`
and `make build`. `CHECK_EPOCH` and a fail-closed guard stop Docker serving
those two layers from cache, which is what let a build report success without
running the suite. `script/projectname` says `quak`, so the image is tagged
`quak`; `script/bootstrap` updates apt lists before installing, so a Debian
base works; `.dockerignore` no longer ships the compiled binary, the caches or
agent worktrees into the build context, and keeps `.gitignore` in it for
prettier.
- 2026-08-09: Fixed the TypeScript build. `rootDir` is the repo root, so `bin/`
compiles alongside `src/` instead of failing with TS6059; output is
`dist/src/` and `dist/bin/`, which is where `main`, `types` and `bin.quak` now
point. `script/build` verifies the declared entrypoints exist after the
compiler runs and makes the CLI executable, the Dockerfile runs `make build`
as well as `make check`, and a `quak` script makes the README's
`yarn quak <command>` examples work.
- 2026-08-09: Retry policy: no retry on 4xx (except `408` and `429`), - 2026-08-09: Retry policy: no retry on 4xx (except `408` and `429`),
exponential backoff with full jitter on 5xx, transport failures and truncated exponential backoff with full jitter on 5xx, transport failures and truncated
transfers, under per-attempt deadlines that cover the response body as well as transfers, under per-attempt deadlines that cover the response body as well as
@@ -49,7 +66,6 @@ Update the README API reference section to match the current implementation.
# Future Steps # Future Steps
- Make `make docker` green.
- Tag v1.0.0. - Tag v1.0.0.
- Future desktop client, separate repo: - Future desktop client, separate repo:
- Electron app skeleton consuming this library. - Electron app skeleton consuming this library.

View File

@@ -10,8 +10,8 @@
"url": "https://git.eeqj.de/sneak/quak.git" "url": "https://git.eeqj.de/sneak/quak.git"
}, },
"type": "module", "type": "module",
"main": "./dist/index.js", "main": "./dist/src/index.js",
"types": "./dist/index.d.ts", "types": "./dist/src/index.d.ts",
"bin": { "bin": {
"quak": "./dist/bin/quak.js" "quak": "./dist/bin/quak.js"
}, },
@@ -21,7 +21,8 @@
"LICENSE" "LICENSE"
], ],
"scripts": { "scripts": {
"build": "tsc", "build": "script/build",
"quak": "node ./dist/bin/quak.js",
"test": "vitest run", "test": "vitest run",
"lint": "eslint .", "lint": "eslint .",
"fmt": "prettier --write .", "fmt": "prettier --write .",

View File

@@ -19,6 +19,7 @@ YARN_VERSION="1.22.22"
PKGMGR="" PKGMGR=""
SUDO="" SUDO=""
APT_UPDATED=""
detect_pkgmgr() { detect_pkgmgr() {
[ -n "$PKGMGR" ] && return 0 [ -n "$PKGMGR" ] && return 0
@@ -42,12 +43,24 @@ detect_pkgmgr() {
fi fi
} }
# A fresh Debian image ships no package lists at all, so apt-get install
# fails with "E: Unable to locate package make" until they are fetched.
# Done once per run, since the lists do not go stale mid-bootstrap.
apt_update_once() {
[ -n "$APT_UPDATED" ] && return 0
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get update
APT_UPDATED="yes"
}
# pkg_install <nix-attr> <apt-pkg> <brew-formula> <apk-pkg> # pkg_install <nix-attr> <apt-pkg> <brew-formula> <apk-pkg>
pkg_install() { pkg_install() {
detect_pkgmgr detect_pkgmgr
case "$PKGMGR" in case "$PKGMGR" in
nix) nix-env -iA "nixpkgs.$1" ;; nix) nix-env -iA "nixpkgs.$1" ;;
apt) $SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2" ;; apt)
apt_update_once
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2"
;;
brew) brew install "$3" ;; brew) brew install "$3" ;;
apk) apk add --no-cache "$4" ;; apk) apk add --no-cache "$4" ;;
esac esac

54
script/build Executable file
View File

@@ -0,0 +1,54 @@
#!/bin/sh
# script/build: compile the TypeScript sources into dist/, then verify that
# the artifacts package.json advertises are among the files the compiler
# actually wrote. tsc reports success by exit status alone and knows nothing
# about the manifest, so without this step a green build can still ship a
# package whose main, types or bin resolve to nothing. Our own extension to
# scripts-to-rule-them-all.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Reads package.json, requires every declared entrypoint to exist, requires
# each bin entry to have kept its shebang, and makes the bin entries
# executable: tsc copies the shebang through but not the mode bits, and an
# installed CLI has to be runnable.
verify_entrypoints() {
node -e '
const { readFileSync, statSync, chmodSync } = require("node:fs");
const pkg = JSON.parse(readFileSync("package.json", "utf-8"));
const bins = Object.values(pkg.bin ?? {});
const fail = (message) => {
console.error("build: " + message);
process.exit(1);
};
for (const declared of [pkg.main, pkg.types, ...bins]) {
if (!declared) continue;
try {
statSync(declared);
} catch {
fail("package.json declares " + declared + ", which the build did not produce");
}
console.log("build: verified " + declared);
}
for (const bin of bins) {
const firstLine = readFileSync(bin, "utf-8").split("\n")[0];
if (!firstLine.startsWith("#!")) {
fail(bin + " lost its shebang, so it cannot be executed directly");
}
chmodSync(bin, 0o755);
console.log("build: " + bin + " is executable (" + firstLine + ")");
}
'
}
main() {
cd "$ROOT"
yarn run tsc
verify_entrypoints
}
main "$@"

View File

@@ -1,13 +1,17 @@
#!/bin/sh #!/bin/sh
# script/cibuild: run the CI build. The Dockerfile runs script/check, so # script/cibuild: run the CI build. The Dockerfile runs script/check and
# a successful build implies all checks pass. # script/build, and CHECK_EPOCH differs on every invocation, so those two
# layers cannot be served from Docker's cache: a green build here means
# the checks ran now, not that a previous run was remembered. The layers
# below the epoch (bootstrap, yarn install) are unaffected and stay
# cached. A build that omits the argument fails by design.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
docker build . docker build --build-arg CHECK_EPOCH="$(date +%s)" .
} }
main "$@" main "$@"

View File

@@ -1,6 +1,9 @@
#!/bin/sh #!/bin/sh
# script/docker: build the Docker image tagged with the project name. # script/docker: build the Docker image tagged with the project name.
# Identical in all repos; the tag comes from script/projectname. # Identical in all repos; the tag comes from script/projectname.
# CHECK_EPOCH is passed for the same reason script/cibuild passes it: the
# Dockerfile refuses to build without it, so that no path to an image can
# quietly serve the check and build layers from cache.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -8,7 +11,8 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
docker build -t "$("$SCRIPT_DIR/projectname")" . docker build --build-arg CHECK_EPOCH="$(date +%s)" \
-t "$("$SCRIPT_DIR/projectname")" .
} }
main "$@" main "$@"

View File

@@ -6,7 +6,7 @@
set -eu set -eu
main() { main() {
echo "quack" echo "quak"
} }
main "$@" main "$@"

View File

@@ -227,11 +227,12 @@ export class ApiClient {
// Idempotency: this reaches `/users/srp/create-session`, // Idempotency: this reaches `/users/srp/create-session`,
// `/users/two-factor/verify` and `/users/ott`, all of which change // `/users/two-factor/verify` and `/users/ott`, all of which change
// server state — verifying a second factor consumes one of a small // server state — verifying a second factor consumes one of a small
// number of attempts. So a POST is replayed only when the failure // number of attempts. So a POST is replayed only on a failure that
// proves the request never reached the server, which in practice means // establishes no TCP connection to the server ever existed: DNS
// the connection was never established. A 5xx, a mid-flight reset and // produced no address, or the peer refused the connection. A 5xx, a
// a timeout are all left to the caller, because each of them can occur // mid-flight reset, a routing errno (which Linux also delivers on an
// after the server has already acted. // established socket) and a timeout are all left to the caller,
// because each of them can occur after the server has already acted.
return withRetry( return withRetry(
async () => { async () => {
const resp = await this._fetch(url, { const resp = await this._fetch(url, {

View File

@@ -1,4 +1,4 @@
import sodium from "libsodium-wrappers-sumo"; import sodium, { type StateAddress } from "libsodium-wrappers-sumo";
// Plaintext chunk size used by Ente for file content streams. Hard-coded by // Plaintext chunk size used by Ente for file content streams. Hard-coded by
// the server; clients must match. // the server; clients must match.
@@ -47,8 +47,10 @@ export const encryptBlob = (
}; };
// Opaque handle to libsodium's secretstream pull state. Threaded through // Opaque handle to libsodium's secretstream pull state. Threaded through
// successive pullStreamChunk calls. // successive pullStreamChunk calls. The type comes from the named export;
export type StreamPullState = sodium.StateAddress; // the default import is the module's value side and has no type namespace
// under it.
export type StreamPullState = StateAddress;
// Initialise a pull stream from the per-file decryption header and the // Initialise a pull stream from the per-file decryption header and the
// per-file key. // per-file key.
@@ -84,9 +86,13 @@ export const pullStreamChunk = (
state: StreamPullState, state: StreamPullState,
ciphertext: Uint8Array, ciphertext: Uint8Array,
): { plaintext: Uint8Array; tag: number } => { ): { plaintext: Uint8Array; tag: number } => {
// The additional-data argument is not optional in libsodium's signature.
// null is "no additional data", matching the null passed on the push side
// in encryptBlob; Ente's file streams carry none.
const result = sodium.crypto_secretstream_xchacha20poly1305_pull( const result = sodium.crypto_secretstream_xchacha20poly1305_pull(
state, state,
ciphertext, ciphertext,
null,
); );
if (result === false) { if (result === false) {
throw new Error("secretstream chunk authentication failed"); throw new Error("secretstream chunk authentication failed");

View File

@@ -62,17 +62,22 @@ const TRANSPORT_CODES = new Set([
"ENETDOWN", "ENETDOWN",
]); ]);
// The subset of the above that can only happen before any request byte was // The subset of the above that can only be reported before a TCP connection
// written: name resolution failed, or the connection was refused or never // exists, and therefore before any request byte could have been written: name
// routed. See `isSafeToReplay`. // resolution produced no address (`ENOTFOUND`, `EAI_AGAIN`) or the peer
const CONNECT_CODES = new Set([ // refused the connection with an RST to the SYN (`ECONNREFUSED`).
"ENOTFOUND", //
"EAI_AGAIN", // The routing errnos — `EHOSTUNREACH`, `ENETUNREACH`, `ENETDOWN` — are
"ECONNREFUSED", // deliberately absent even though they look like connect-time failures. On
"EHOSTUNREACH", // Linux they are also delivered on an already-established socket: an ICMP
"ENETUNREACH", // destination-unreachable arriving mid-flight sets the socket error and the
"ENETDOWN", // next read or write returns it, and a local interface going down after the
]); // request was fully written surfaces the same way. In those cases the server
// may already have received and acted on the request, which is exactly the
// ambiguity this set exists to exclude. They stay in `TRANSPORT_CODES`, so
// they remain retryable for idempotent calls; only replay eligibility is
// narrowed. See `isSafeToReplay`.
const CONNECT_CODES = new Set(["ENOTFOUND", "EAI_AGAIN", "ECONNREFUSED"]);
// `cause` is an arbitrary user-settable property and nothing prevents it from // `cause` is an arbitrary user-settable property and nothing prevents it from
// forming a cycle, so the walk is bounded. Hanging the process would be a // forming a cycle, so the walk is bounded. Hanging the process would be a
@@ -148,13 +153,16 @@ export const isRetryable = (err: unknown): boolean => {
// `isRetryable` is the wrong question for a request that changes state. // `isRetryable` is the wrong question for a request that changes state.
// quak's non-idempotent calls are `/users/srp/create-session`, // quak's non-idempotent calls are `/users/srp/create-session`,
// `/users/two-factor/verify` — which consumes one of a small number of 2FA // `/users/two-factor/verify` — which consumes one of a small number of 2FA
// attempts — and `/files/thumbnail`. They are replayed only when the failure // attempts — and `/files/thumbnail`. They are replayed only on the failures in
// proves no request byte reached the server, which means the connection was // `CONNECT_CODES`, which establish that no TCP connection to the server ever
// never established. // existed: there was no address to connect to, or the peer refused the
// connection outright. A request byte cannot have been transmitted, so the
// server cannot have acted.
// //
// Everything else is ambiguous. A 5xx proves the server did process the // Everything else is ambiguous. A 5xx proves the server did process the
// request. A reset or a broken pipe can arrive after it was fully sent and // request. A reset or a broken pipe can arrive after it was fully sent and
// acted on. A deadline says nothing at all about the server's state. // acted on. A routing errno can be delivered on an established socket. A
// deadline says nothing at all about the server's state.
export const isSafeToReplay = (err: unknown): boolean => export const isSafeToReplay = (err: unknown): boolean =>
isRetryable(err) && causeCodes(err).some((code) => CONNECT_CODES.has(code)); isRetryable(err) && causeCodes(err).some((code) => CONNECT_CODES.has(code));

View File

@@ -824,10 +824,11 @@ describe("ApiClient non-idempotent requests", () => {
* state: `/users/srp/create-session`, `/users/two-factor/verify` — which * state: `/users/srp/create-session`, `/users/two-factor/verify` — which
* consumes one of a small number of 2FA attempts — and `/files/thumbnail`. * consumes one of a small number of 2FA attempts — and `/files/thumbnail`.
* *
* They are retried only when the failure proves the request never reached * They are retried only on a failure that establishes no TCP connection to
* the server, which in practice means the connection was never * the server ever existed — DNS produced no address, or the peer refused
* established. Everything else is ambiguous: a 5xx proves the server did * the connection — so no request byte can have been transmitted.
* process the request, and a reset or a timeout can arrive after it did. * Everything else is ambiguous: a 5xx proves the server did process the
* request, and a reset or a timeout can arrive after it did.
* Replaying under that ambiguity can burn a 2FA attempt or register a * Replaying under that ambiguity can burn a 2FA attempt or register a
* thumbnail twice, and neither is worth the round trip it saves. * thumbnail twice, and neither is worth the round trip it saves.
*/ */

View File

@@ -0,0 +1,49 @@
// The Docker build context is load-bearing in two directions, and both
// failures are silent.
//
// Excluding too little: a worktree left under `.claude/` is copied into the
// image, vitest globs its `test/` tree as well as the real one, and the
// containerised `make check` runs the whole suite twice over while reporting
// success. A compiled `bin/quak` is ~100 MB of context nobody needs.
//
// Excluding too much: Prettier 3 reads `.gitignore` as a default ignore file,
// so dropping it from the context silently changes which files
// `make fmt-check` looks at inside the image compared to the host.
//
// Neither shows up as a build failure, so they are asserted here.
import { describe, expect, it } from "vitest";
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { join } from "node:path";
const repoRoot = fileURLToPath(new URL("../../", import.meta.url));
const patterns = (name: string): string[] =>
readFileSync(join(repoRoot, name), "utf-8")
.split("\n")
.map((line) => line.trim())
.filter((line) => line !== "" && !line.startsWith("#"));
const dockerignore = patterns(".dockerignore");
describe(".dockerignore", () => {
// Everything here is either generated, enormous, or secret. `.claude/` is
// the correctness one: see the header comment and issue #25.
it.each([
".claude/",
".quak/",
"bin/quak",
"node_modules",
"coverage",
"dist",
".vitest-cache/",
".nyc_output/",
"*.tsbuildinfo",
])("keeps %s out of the build context", (pattern) => {
expect(dockerignore).toContain(pattern);
});
it("leaves .gitignore in the build context for prettier", () => {
expect(dockerignore).not.toContain(".gitignore");
});
});

View File

@@ -0,0 +1,116 @@
// The package manifest promises three files that only exist after a build:
// `main`, `types`, and the `quak` binary. Nothing in the test suite used to
// look at them, and `make check` runs test, lint and fmt-check but never the
// build, so `tsconfig.json` and `package.json` were free to drift apart. They
// did: `rootDir` was `./src` while `include` also pulled in `bin/**/*`, which
// is TS6059, and no build had succeeded for as long as that was true.
//
// These tests read both files and check the contract between them, without
// running a build, so they stay in the fast unit suite. The complementary
// check — that the files really landed on disk — is in `script/build`, which
// runs after the compiler and is the only place that can honestly answer it.
import { describe, expect, it } from "vitest";
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { join, posix } from "node:path";
interface TsConfig {
compilerOptions: {
outDir: string;
rootDir: string;
};
include: string[];
}
interface PackageJson {
main: string;
types: string;
bin: Record<string, string>;
scripts: Record<string, string>;
}
const repoRoot = fileURLToPath(new URL("../../", import.meta.url));
const readJSON = <T>(name: string): T =>
JSON.parse(readFileSync(join(repoRoot, name), "utf-8")) as T;
const tsconfig = readJSON<TsConfig>("tsconfig.json");
const pkg = readJSON<PackageJson>("package.json");
// Paths in the two manifests are written with a leading "./"; normalize so
// they can be compared and joined. Everything here is POSIX-style because
// that is what both JSON files contain, regardless of the host OS.
const clean = (p: string): string => posix.normalize(p);
const outDir = clean(tsconfig.compilerOptions.outDir);
const rootDir = clean(tsconfig.compilerOptions.rootDir);
// The directory prefix of a glob: the part before the first segment
// containing a wildcard. "src/**/*" -> "src", "bin/**/*" -> "bin".
const globRoot = (pattern: string): string => {
const segments = clean(pattern).split("/");
const wildcard = segments.findIndex((s) => s.includes("*"));
return (
segments
.slice(0, wildcard === -1 ? segments.length : wildcard)
.join("/") || "."
);
};
// Where tsc will write the output for a source file: the path relative to
// rootDir, re-rooted under outDir, with the extension swapped.
const emitted = (source: string, extension: string): string =>
"./" +
posix
.join(outDir, posix.relative(rootDir, clean(source)))
.replace(/\.ts$/, extension);
describe("tsconfig include and rootDir", () => {
// TS6059 is not a style complaint: tsc refuses to emit anything at all
// when a compiled file sits outside rootDir, so this single mismatch
// took out both the library and the CLI artifacts.
it("compiles only files that live under rootDir", () => {
for (const pattern of tsconfig.include) {
const root = globRoot(pattern);
const relative = posix.relative(rootDir, root);
expect(
relative === "" || !relative.startsWith(".."),
`include pattern ${pattern} matches files outside rootDir ` +
`${rootDir}; tsc rejects that with TS6059`,
).toBe(true);
}
});
});
describe("package.json entrypoints", () => {
// Each of these is the path a consumer resolves — `import { Client } from
// "quak"` for main, the editor for types, `npx quak` for the bin — so a
// wrong value is a broken package even when the build itself is green.
it("names the file tsc emits for src/index.ts as main", () => {
expect(pkg.main).toBe(emitted("src/index.ts", ".js"));
});
it("names the declaration tsc emits for src/index.ts as types", () => {
expect(pkg.types).toBe(emitted("src/index.ts", ".d.ts"));
});
it("names the file tsc emits for bin/quak.ts as the quak binary", () => {
expect(pkg.bin.quak).toBe(emitted("bin/quak.ts", ".js"));
});
// The README's Getting Started block tells the reader to run
// `yarn quak login` straight after `yarn build`. That only works if a
// `quak` script exists and points at the built CLI, not at the source.
it("runs the built CLI from the quak script", () => {
expect(pkg.scripts.quak).toBeDefined();
expect(pkg.scripts.quak).toContain(pkg.bin.quak);
});
});
describe("bin/quak.ts", () => {
// tsc copies the shebang into the emitted file, so it has to be in the
// source for the installed binary to be directly executable.
it("starts with a node shebang", () => {
const source = readFileSync(join(repoRoot, "bin/quak.ts"), "utf-8");
expect(source.split("\n")[0]).toBe("#!/usr/bin/env node");
});
});

View File

@@ -0,0 +1,30 @@
// `script/projectname` is the single source of the project's name for every
// script that needs one — `script/docker` builds its image tag from it, which
// is the whole reason that file exists. Nothing checked that it agreed with
// `package.json`, and after the repo was renamed it did not: the script still
// said "quack", so `make docker` produced an image tagged after a name this
// project has not used since May.
//
// The script is executed rather than read, because what matters is the string
// it prints, not the source it prints it from.
import { describe, expect, it } from "vitest";
import { execFileSync } from "node:child_process";
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { join } from "node:path";
const repoRoot = fileURLToPath(new URL("../../", import.meta.url));
const pkg = JSON.parse(
readFileSync(join(repoRoot, "package.json"), "utf-8"),
) as { name: string };
describe("script/projectname", () => {
it("prints the name package.json declares", () => {
const printed = execFileSync(join(repoRoot, "script/projectname"), {
cwd: repoRoot,
encoding: "utf-8",
}).trim();
expect(printed).toBe(pkg.name);
});
});

View File

@@ -282,18 +282,13 @@ describe("isSafeToReplay", () => {
* quak's non-idempotent calls are `/users/srp/create-session`, * quak's non-idempotent calls are `/users/srp/create-session`,
* `/users/two-factor/verify` (which consumes one of a limited number of * `/users/two-factor/verify` (which consumes one of a limited number of
* 2FA attempts) and `/files/thumbnail`. A blind replay of any of them can * 2FA attempts) and `/files/thumbnail`. A blind replay of any of them can
* do real damage, so they retry only on failures that prove no request * do real damage, so they retry only on the failures that establish no TCP
* byte ever reached the server — which means the connection was never * connection to the server ever existed — DNS produced no address, or the
* established. * peer refused the connection — and therefore that no request byte can
* have been transmitted.
*/ */
it("replays only failures where the connection was never established", () => { it("replays only failures where the connection was never established", () => {
for (const code of [ for (const code of ["ENOTFOUND", "EAI_AGAIN", "ECONNREFUSED"]) {
"ENOTFOUND",
"EAI_AGAIN",
"ECONNREFUSED",
"EHOSTUNREACH",
"ENETUNREACH",
]) {
expect(isSafeToReplay(errnoError(code))).toBe(true); expect(isSafeToReplay(errnoError(code))).toBe(true);
} }
// Also when undici has buried it, which is how it actually arrives. // Also when undici has buried it, which is how it actually arrives.
@@ -317,6 +312,22 @@ describe("isSafeToReplay", () => {
expect(isSafeToReplay(errnoError("ECONNRESET"))).toBe(false); expect(isSafeToReplay(errnoError("ECONNRESET"))).toBe(false);
expect(isSafeToReplay(errnoError("EPIPE"))).toBe(false); expect(isSafeToReplay(errnoError("EPIPE"))).toBe(false);
expect(isSafeToReplay(errnoError("ETIMEDOUT"))).toBe(false); expect(isSafeToReplay(errnoError("ETIMEDOUT"))).toBe(false);
// The routing errnos look like connect-time failures but are not. On
// Linux an ICMP destination-unreachable delivered on an established
// connection sets the socket error, and the next read or write returns
// `EHOSTUNREACH` or `ENETUNREACH`; a local interface going down after
// the request was fully written surfaces as `ENETDOWN` the same way.
// In each case the server may already have consumed the request — a
// replayed `/users/two-factor/verify` would burn a second attempt.
// They remain retryable for the idempotent calls; this asserts only
// that they are not replayable.
expect(isSafeToReplay(errnoError("EHOSTUNREACH"))).toBe(false);
expect(isSafeToReplay(errnoError("ENETUNREACH"))).toBe(false);
expect(isSafeToReplay(errnoError("ENETDOWN"))).toBe(false);
// ...and that the narrowing did not make them non-retryable.
expect(isRetryable(errnoError("EHOSTUNREACH"))).toBe(true);
expect(isRetryable(errnoError("ENETUNREACH"))).toBe(true);
expect(isRetryable(errnoError("ENETDOWN"))).toBe(true);
expect( expect(
isSafeToReplay(new DOMException("timed out", "TimeoutError")), isSafeToReplay(new DOMException("timed out", "TimeoutError")),
).toBe(false); ).toBe(false);

View File

@@ -5,7 +5,8 @@
"moduleResolution": "NodeNext", "moduleResolution": "NodeNext",
"lib": ["ES2022"], "lib": ["ES2022"],
"outDir": "./dist", "outDir": "./dist",
"rootDir": "./src", "rootDir": ".",
"noEmitOnError": true,
"strict": true, "strict": true,
"noImplicitOverride": true, "noImplicitOverride": true,
"noUncheckedIndexedAccess": true, "noUncheckedIndexedAccess": true,