Compile bin/ alongside src/ so the package can be built (closes #3) #26

Merged
clawbot merged 2 commits from fix-ts-build-rootdir into main 2026-08-09 12:21:47 +02:00
9 changed files with 206 additions and 11 deletions

View File

@@ -8,3 +8,4 @@ RUN script/bootstrap
COPY . . COPY . .
RUN make check RUN make check
RUN make build

View File

@@ -24,7 +24,7 @@ check:
@script/check @script/check
build: build:
@$(YARN) tsc @script/build
build-bin: build-bin:
nix-shell -p bun --run "bun build bin/quak.ts --compile --outfile bin/quak" nix-shell -p bun --run "bun build bin/quak.ts --compile --outfile bin/quak"

View File

@@ -81,6 +81,9 @@ alpine. We provide:
`script/bootstrap`, then `script/install-precommit` `script/bootstrap`, then `script/install-precommit`
- `script/projectname` — output the project name (our own extension); used by - `script/projectname` — output the project name (our own extension); used by
`script/docker` for the image tag `script/docker` for the image tag
- `script/build` — compile the TypeScript sources into `dist/`, then verify that
the entrypoints `package.json` declares (`main`, `types`, `bin`) are among the
files the compiler wrote, and make the CLI executable (our own extension)
- `script/test` — run the test suite (vitest, hard-capped at 30s where `timeout` - `script/test` — run the test suite (vitest, hard-capped at 30s where `timeout`
is available, verbose rerun on failure) is available, verbose rerun on failure)
- `script/lint` — run eslint and a prettier check - `script/lint` — run eslint and a prettier check
@@ -91,7 +94,7 @@ alpine. We provide:
- `script/docker` — build the Docker image, tagged via `script/projectname` - `script/docker` — build the Docker image, tagged via `script/projectname`
(byte-identical across repos) (byte-identical across repos)
- `script/cibuild` — cd to the repo root and `docker build .` (what CI runs; the - `script/cibuild` — cd to the repo root and `docker build .` (what CI runs; the
image build runs `make check`) image build runs `make check` and `make build`)
- `script/precommit` — run by the git pre-commit hook (our own extension); runs - `script/precommit` — run by the git pre-commit hook (our own extension); runs
`script/lint` and `script/fmt-check` but deliberately not the tests, so the `script/lint` and `script/fmt-check` but deliberately not the tests, so the
TDD red-phase commit can land TDD red-phase commit can land
@@ -133,8 +136,8 @@ All work on quak is test-driven. No exceptions.
3. Subsequent commits add the implementation and any refactors needed to make 3. Subsequent commits add the implementation and any refactors needed to make
the tests pass. the tests pass.
4. A feature branch can only be merged into `main` when `make check` is green. 4. A feature branch can only be merged into `main` when `make check` is green.
`main` is always green. The Dockerfile runs `make check`, so a red branch `main` is always green. The Dockerfile runs `make check` and `make build`, so
cannot pass CI. neither a red branch nor one that does not compile can pass CI.
5. Tests are the canonical API documentation for this library. Every test file 5. Tests are the canonical API documentation for this library. Every test file
is commented thoroughly enough that a reader who has never seen quak can is commented thoroughly enough that a reader who has never seen quak can
learn how to use it from the tests alone. Comments explain why a behavior learn how to use it from the tests alone. Comments explain why a behavior
@@ -183,6 +186,12 @@ quak/
tsconfig.json tsconfig.json
``` ```
`make build` compiles that tree into `dist/`, preserving its shape: the library
lands in `dist/src/` and the CLI in `dist/bin/quak.js`, which is what
`package.json` points `main`, `types` and `bin` at. The compiler's `rootDir` is
the repository root rather than `src/`, because `bin/` is compiled too and
`rootDir` has to contain everything that is compiled.
### Cryptography ### Cryptography
All cryptography is done by `libsodium-wrappers-sumo` (the "sumo" build is All cryptography is done by `libsodium-wrappers-sumo` (the "sumo" build is

View File

@@ -18,6 +18,13 @@ Update the README API reference section to match the current implementation.
# Completed Steps # Completed Steps
- 2026-08-09: Fixed the TypeScript build. `rootDir` is the repo root, so `bin/`
compiles alongside `src/` instead of failing with TS6059; output is
`dist/src/` and `dist/bin/`, which is where `main`, `types` and `bin.quak` now
point. `script/build` verifies the declared entrypoints exist after the
compiler runs and makes the CLI executable, the Dockerfile runs `make build`
as well as `make check`, and a `quak` script makes the README's
`yarn quak <command>` examples work.
- 2026-08-09: Retry policy: no retry on 4xx (except `408` and `429`), - 2026-08-09: Retry policy: no retry on 4xx (except `408` and `429`),
exponential backoff with full jitter on 5xx, transport failures and truncated exponential backoff with full jitter on 5xx, transport failures and truncated
transfers, under per-attempt deadlines that cover the response body as well as transfers, under per-attempt deadlines that cover the response body as well as

View File

@@ -10,8 +10,8 @@
"url": "https://git.eeqj.de/sneak/quak.git" "url": "https://git.eeqj.de/sneak/quak.git"
}, },
"type": "module", "type": "module",
"main": "./dist/index.js", "main": "./dist/src/index.js",
"types": "./dist/index.d.ts", "types": "./dist/src/index.d.ts",
"bin": { "bin": {
"quak": "./dist/bin/quak.js" "quak": "./dist/bin/quak.js"
}, },
@@ -21,7 +21,8 @@
"LICENSE" "LICENSE"
], ],
"scripts": { "scripts": {
"build": "tsc", "build": "script/build",
"quak": "node ./dist/bin/quak.js",
"test": "vitest run", "test": "vitest run",
"lint": "eslint .", "lint": "eslint .",
"fmt": "prettier --write .", "fmt": "prettier --write .",

54
script/build Executable file
View File

@@ -0,0 +1,54 @@
#!/bin/sh
# script/build: compile the TypeScript sources into dist/, then verify that
# the artifacts package.json advertises are among the files the compiler
# actually wrote. tsc reports success by exit status alone and knows nothing
# about the manifest, so without this step a green build can still ship a
# package whose main, types or bin resolve to nothing. Our own extension to
# scripts-to-rule-them-all.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Reads package.json, requires every declared entrypoint to exist, requires
# each bin entry to have kept its shebang, and makes the bin entries
# executable: tsc copies the shebang through but not the mode bits, and an
# installed CLI has to be runnable.
verify_entrypoints() {
node -e '
const { readFileSync, statSync, chmodSync } = require("node:fs");
const pkg = JSON.parse(readFileSync("package.json", "utf-8"));
const bins = Object.values(pkg.bin ?? {});
const fail = (message) => {
console.error("build: " + message);
process.exit(1);
};
for (const declared of [pkg.main, pkg.types, ...bins]) {
if (!declared) continue;
try {
statSync(declared);
} catch {
fail("package.json declares " + declared + ", which the build did not produce");
}
console.log("build: verified " + declared);
}
for (const bin of bins) {
const firstLine = readFileSync(bin, "utf-8").split("\n")[0];
if (!firstLine.startsWith("#!")) {
fail(bin + " lost its shebang, so it cannot be executed directly");
}
chmodSync(bin, 0o755);
console.log("build: " + bin + " is executable (" + firstLine + ")");
}
'
}
main() {
cd "$ROOT"
yarn run tsc
verify_entrypoints
}
main "$@"

View File

@@ -1,4 +1,4 @@
import sodium from "libsodium-wrappers-sumo"; import sodium, { type StateAddress } from "libsodium-wrappers-sumo";
// Plaintext chunk size used by Ente for file content streams. Hard-coded by // Plaintext chunk size used by Ente for file content streams. Hard-coded by
// the server; clients must match. // the server; clients must match.
@@ -47,8 +47,10 @@ export const encryptBlob = (
}; };
// Opaque handle to libsodium's secretstream pull state. Threaded through // Opaque handle to libsodium's secretstream pull state. Threaded through
// successive pullStreamChunk calls. // successive pullStreamChunk calls. The type comes from the named export;
export type StreamPullState = sodium.StateAddress; // the default import is the module's value side and has no type namespace
// under it.
export type StreamPullState = StateAddress;
// Initialise a pull stream from the per-file decryption header and the // Initialise a pull stream from the per-file decryption header and the
// per-file key. // per-file key.
@@ -84,9 +86,13 @@ export const pullStreamChunk = (
state: StreamPullState, state: StreamPullState,
ciphertext: Uint8Array, ciphertext: Uint8Array,
): { plaintext: Uint8Array; tag: number } => { ): { plaintext: Uint8Array; tag: number } => {
// The additional-data argument is not optional in libsodium's signature.
// null is "no additional data", matching the null passed on the push side
// in encryptBlob; Ente's file streams carry none.
const result = sodium.crypto_secretstream_xchacha20poly1305_pull( const result = sodium.crypto_secretstream_xchacha20poly1305_pull(
state, state,
ciphertext, ciphertext,
null,
); );
if (result === false) { if (result === false) {
throw new Error("secretstream chunk authentication failed"); throw new Error("secretstream chunk authentication failed");

View File

@@ -0,0 +1,116 @@
// The package manifest promises three files that only exist after a build:
// `main`, `types`, and the `quak` binary. Nothing in the test suite used to
// look at them, and `make check` runs test, lint and fmt-check but never the
// build, so `tsconfig.json` and `package.json` were free to drift apart. They
// did: `rootDir` was `./src` while `include` also pulled in `bin/**/*`, which
// is TS6059, and no build had succeeded for as long as that was true.
//
// These tests read both files and check the contract between them, without
// running a build, so they stay in the fast unit suite. The complementary
// check — that the files really landed on disk — is in `script/build`, which
// runs after the compiler and is the only place that can honestly answer it.
import { describe, expect, it } from "vitest";
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { join, posix } from "node:path";
interface TsConfig {
compilerOptions: {
outDir: string;
rootDir: string;
};
include: string[];
}
interface PackageJson {
main: string;
types: string;
bin: Record<string, string>;
scripts: Record<string, string>;
}
const repoRoot = fileURLToPath(new URL("../../", import.meta.url));
const readJSON = <T>(name: string): T =>
JSON.parse(readFileSync(join(repoRoot, name), "utf-8")) as T;
const tsconfig = readJSON<TsConfig>("tsconfig.json");
const pkg = readJSON<PackageJson>("package.json");
// Paths in the two manifests are written with a leading "./"; normalize so
// they can be compared and joined. Everything here is POSIX-style because
// that is what both JSON files contain, regardless of the host OS.
const clean = (p: string): string => posix.normalize(p);
const outDir = clean(tsconfig.compilerOptions.outDir);
const rootDir = clean(tsconfig.compilerOptions.rootDir);
// The directory prefix of a glob: the part before the first segment
// containing a wildcard. "src/**/*" -> "src", "bin/**/*" -> "bin".
const globRoot = (pattern: string): string => {
const segments = clean(pattern).split("/");
const wildcard = segments.findIndex((s) => s.includes("*"));
return (
segments
.slice(0, wildcard === -1 ? segments.length : wildcard)
.join("/") || "."
);
};
// Where tsc will write the output for a source file: the path relative to
// rootDir, re-rooted under outDir, with the extension swapped.
const emitted = (source: string, extension: string): string =>
"./" +
posix
.join(outDir, posix.relative(rootDir, clean(source)))
.replace(/\.ts$/, extension);
describe("tsconfig include and rootDir", () => {
// TS6059 is not a style complaint: tsc refuses to emit anything at all
// when a compiled file sits outside rootDir, so this single mismatch
// took out both the library and the CLI artifacts.
it("compiles only files that live under rootDir", () => {
for (const pattern of tsconfig.include) {
const root = globRoot(pattern);
const relative = posix.relative(rootDir, root);
expect(
relative === "" || !relative.startsWith(".."),
`include pattern ${pattern} matches files outside rootDir ` +
`${rootDir}; tsc rejects that with TS6059`,
).toBe(true);
}
});
});
describe("package.json entrypoints", () => {
// Each of these is the path a consumer resolves — `import { Client } from
// "quak"` for main, the editor for types, `npx quak` for the bin — so a
// wrong value is a broken package even when the build itself is green.
it("names the file tsc emits for src/index.ts as main", () => {
expect(pkg.main).toBe(emitted("src/index.ts", ".js"));
});
it("names the declaration tsc emits for src/index.ts as types", () => {
expect(pkg.types).toBe(emitted("src/index.ts", ".d.ts"));
});
it("names the file tsc emits for bin/quak.ts as the quak binary", () => {
expect(pkg.bin.quak).toBe(emitted("bin/quak.ts", ".js"));
});
// The README's Getting Started block tells the reader to run
// `yarn quak login` straight after `yarn build`. That only works if a
// `quak` script exists and points at the built CLI, not at the source.
it("runs the built CLI from the quak script", () => {
expect(pkg.scripts.quak).toBeDefined();
expect(pkg.scripts.quak).toContain(pkg.bin.quak);
});
});
describe("bin/quak.ts", () => {
// tsc copies the shebang into the emitted file, so it has to be in the
// source for the installed binary to be directly executable.
it("starts with a node shebang", () => {
const source = readFileSync(join(repoRoot, "bin/quak.ts"), "utf-8");
expect(source.split("\n")[0]).toBe("#!/usr/bin/env node");
});
});

View File

@@ -5,7 +5,8 @@
"moduleResolution": "NodeNext", "moduleResolution": "NodeNext",
"lib": ["ES2022"], "lib": ["ES2022"],
"outDir": "./dist", "outDir": "./dist",
"rootDir": "./src", "rootDir": ".",
"noEmitOnError": true,
"strict": true, "strict": true,
"noImplicitOverride": true, "noImplicitOverride": true,
"noUncheckedIndexedAccess": true, "noUncheckedIndexedAccess": true,