Standing rule from the same hour (sneak/project-management#20): main and next are green on every managed repo at all times, and any divergence is top priority. This comes before all other upaas work.
Definition of done:
Reproduce on a fresh clone of the next head (docker build . in the repo root, plus script/cibuild or make check), and post the failing step and its cause here in a few lines.
Fix it at the cause, through a PR to next with an independent review, rebased onto next right before the review, then squashed. Never weaken a test.
After the squash, docker build . and the check pass on a fresh clone of the new next head. One line here says so, with the SHA.
Say whether any PR merged today (for example #256) caused it, and why its review did not catch it.
model: opus-5-5
Owner (chat, 2026-10-01 ~23:03 UTC):
> "docker build ." in upaas root on next fails
Standing rule from the same hour (https://git.eeqj.de/sneak/project-management/issues/20): `main` and `next` are green on every managed repo at all times, and any divergence is top priority. This comes before all other upaas work.
Definition of done:
- Reproduce on a fresh clone of the `next` head (`docker build .` in the repo root, plus `script/cibuild` or `make check`), and post the failing step and its cause here in a few lines.
- Fix it at the cause, through a PR to `next` with an independent review, rebased onto `next` right before the review, then squashed. Never weaken a test.
- After the squash, `docker build .` and the check pass on a fresh clone of the new `next` head. One line here says so, with the SHA.
- Say whether any PR merged today (for example https://git.eeqj.de/sneak/upaas/pulls/256) caused it, and why its review did not catch it.
model: opus-5-5
Not reproduced yet. On a fresh clone of next at 5ca1bce, docker build . succeeds here (Docker Engine 29.8, linux/amd64), and make check passes on the same tree. The merge commit 5ca1bce changes no files compared with 9488e2f, so it is not the cause.
So the failure depends on the machine or the working copy it runs in. The leading suspect: since #236 (a48d90f), .dockerignore sends the working copy's whole .git directory and every git-ignored file except .env, bin/, .vscode/, .idea/ and *.test into the build. A fresh clone has none of those local files, and the review of 236 gated only fresh clones, which is why it passed.
@sneak: which step fails, and with what error? The last 20 or so lines of docker build --progress=plain ., plus the machine (OS, CPU) and docker version, are enough. Until then a worker is checking the build against a working copy with typical local files.
Model: opus-5-5
Not reproduced yet. On a fresh clone of `next` at `5ca1bce`, `docker build .` succeeds here (Docker Engine 29.8, linux/amd64), and `make check` passes on the same tree. The merge commit `5ca1bce` changes no files compared with `9488e2f`, so it is not the cause.
So the failure depends on the machine or the working copy it runs in. The leading suspect: since https://git.eeqj.de/sneak/upaas/issues/236 (`a48d90f`), `.dockerignore` sends the working copy's whole `.git` directory and every git-ignored file except `.env`, `bin/`, `.vscode/`, `.idea/` and `*.test` into the build. A fresh clone has none of those local files, and the review of 236 gated only fresh clones, which is why it passed.
@sneak: which step fails, and with what error? The last 20 or so lines of `docker build --progress=plain .`, plus the machine (OS, CPU) and `docker version`, are enough. Until then a worker is checking the build against a working copy with typical local files.
Model: opus-5-5
sneak
was assigned by clawbot2026-10-02 01:11:01 +02:00
Still not reproduced, on next at 7685812. Tried here:
docker build . and make docker from a working copy set up like a developer's: make bootstrap run (so node_modules/ exists), git-ignored local files (.env.local, a *.key, a *.out, .DS_Store), the data/ directory upaasd creates when run from the checkout, and an uncommitted change. The build does not fail.
make test, run as the build runs it, on one CPU and on two: every package stays far inside the 30-second limit, so a slower machine is not the cause.
A .git that is a file pointing outside the checkout, and core.fsmonitor set in .git/config as on macOS: neither breaks the build.
Not tested: an arm64 machine such as an Apple Silicon Mac (no emulation on this host).
None of the 2026-10-01 merges, #256 included, changes the Dockerfile, .dockerignore, the Makefile, script/ or the dependencies. The only change on next that makes the build depend on the working copy is #236 (a48d90f, 2026-09-29).
Found on the way, not this failure: since #236 the build stages receive the working copy's git-ignored secrets (.env.local, *.key, data/session.key); the final image holds only the binary. data/, the default data directory, is not git-ignored.
No fix opened: there is no known cause to fix. @sneak: still needed are the last 20 or so lines of docker build --progress=plain . from the copy where it fails, and the machine (OS, CPU).
Model: opus-5-5
Still not reproduced, on `next` at `7685812`. Tried here:
- `docker build .` and `make docker` from a working copy set up like a developer's: `make bootstrap` run (so `node_modules/` exists), git-ignored local files (`.env.local`, a `*.key`, a `*.out`, `.DS_Store`), the `data/` directory upaasd creates when run from the checkout, and an uncommitted change. The build does not fail.
- `make test`, run as the build runs it, on one CPU and on two: every package stays far inside the 30-second limit, so a slower machine is not the cause.
- A `.git` that is a file pointing outside the checkout, and `core.fsmonitor` set in `.git/config` as on macOS: neither breaks the build.
Not tested: an arm64 machine such as an Apple Silicon Mac (no emulation on this host).
None of the 2026-10-01 merges, https://git.eeqj.de/sneak/upaas/pulls/256 included, changes the `Dockerfile`, `.dockerignore`, the `Makefile`, `script/` or the dependencies. The only change on `next` that makes the build depend on the working copy is https://git.eeqj.de/sneak/upaas/issues/236 (`a48d90f`, 2026-09-29).
Found on the way, not this failure: since https://git.eeqj.de/sneak/upaas/issues/236 the build stages receive the working copy's git-ignored secrets (`.env.local`, `*.key`, `data/session.key`); the final image holds only the binary. `data/`, the default data directory, is not git-ignored.
No fix opened: there is no known cause to fix. @sneak: still needed are the last 20 or so lines of `docker build --progress=plain .` from the copy where it fails, and the machine (OS, CPU).
Model: opus-5-5
next at d3b9c6f passes docker build . and make check on a fresh clone here. Since the last report two changes narrowed what the build receives from a working copy: git-ignored files and data/ stay out (#266) and so does .git/config (#269). If the failure came from local files in your copy, it may be gone.
@sneak: please run docker build --progress=plain . again on d3b9c6f where it failed; if it still fails, the last 20 or so lines and the machine (OS, CPU) are what is needed.
Model: opus-5-5
`next` at `d3b9c6f` passes `docker build .` and `make check` on a fresh clone here. Since the last report two changes narrowed what the build receives from a working copy: git-ignored files and `data/` stay out (https://git.eeqj.de/sneak/upaas/issues/266) and so does `.git/config` (https://git.eeqj.de/sneak/upaas/issues/269). If the failure came from local files in your copy, it may be gone.
@sneak: please run `docker build --progress=plain .` again on `d3b9c6f` where it failed; if it still fails, the last 20 or so lines and the machine (OS, CPU) are what is needed.
Model: opus-5-5
Likely cause: there is no --- FAIL line, so no test assertion failed. Something outside the test killed the process, most likely the out-of-memory killer in the Lima VM, whose default size is 4 GiB. The auth tests run argon2id (64 MiB per hash, 4 threads) in parallel subtests: TestVerifyPassword, TestCreateUserRaceCondition and others were running when it was killed. script/test runs go test -race, and the race detector multiplies memory use several times over. Other packages' tests run at the same time. That also explains why it never failed here: sandcastle has far more memory.
The build session failed ... unable to upgrade to h2c lines in the log come from TestBuildImageLogsBuildErrorBeforeDeployError, which passed. They are not the failure.
Next (upaas manager):
Measure the peak memory of make test, with the auth package's share, on a fresh clone. docker run --memory fails on sandcastle, so use a measurement such as /usr/bin/time -v, not a container limit.
Fix it so the peak fits well inside a 4 GiB builder, and so docker build . passes there. Production password hashing must not get weaker.
Land the fix on next with an independent review, then close this issue and 264.
Model: opus-5-5
sneak sent the failing build log in chat (2026-10-03 00:1x UTC). It is `docker build .` on `next`, run 2026-10-02 on his Mac (`heraklion`, builder `docker:lima-default`). Full log, gzipped: https://git.eeqj.de/attachments/91faff25-fbb8-4c33-a411-4983b49a354e
**What fails:** `RUN make test` (Dockerfile line 33). Every package passes except `internal/service/auth`, whose test binary is killed after 4.6 s:
```
42.47 signal: killed
42.47 FAIL sneak.berlin/go/upaas/internal/service/auth 4.602s
```
**Likely cause:** there is no `--- FAIL` line, so no test assertion failed. Something outside the test killed the process, most likely the out-of-memory killer in the Lima VM, whose default size is 4 GiB. The auth tests run argon2id (64 MiB per hash, 4 threads) in parallel subtests: `TestVerifyPassword`, `TestCreateUserRaceCondition` and others were running when it was killed. `script/test` runs `go test -race`, and the race detector multiplies memory use several times over. Other packages' tests run at the same time. That also explains why it never failed here: sandcastle has far more memory.
The `build session failed ... unable to upgrade to h2c` lines in the log come from `TestBuildImageLogsBuildErrorBeforeDeployError`, which passed. They are not the failure.
**Next (upaas manager):**
- Measure the peak memory of `make test`, with the auth package's share, on a fresh clone. `docker run --memory` fails on sandcastle, so use a measurement such as `/usr/bin/time -v`, not a container limit.
- Fix it so the peak fits well inside a 4 GiB builder, and so `docker build .` passes there. Production password hashing must not get weaker.
- Land the fix on `next` with an independent review, then close this issue and 264.
Model: opus-5-5
sneak
was unassigned by clawbot2026-10-03 02:16:17 +02:00
Ruling from sneak (chat, 2026-10-03 00:2x UTC): running the test suite must not take more memory than his Lima VM has, which is 4 GB.
Definition of done for this issue, updated: the peak memory of make test, and so of docker build ., stays well under 4 GB, measured on a fresh clone and reported in the PR. Production password hashing stays as strong as it is now.
Model: opus-5-5
**Ruling from sneak** (chat, 2026-10-03 00:2x UTC): running the test suite must not take more memory than his Lima VM has, which is 4 GB.
Definition of done for this issue, updated: the peak memory of `make test`, and so of `docker build .`, stays well under 4 GB, measured on a fresh clone and reported in the PR. Production password hashing stays as strong as it is now.
Model: opus-5-5
Implementer's brief for #261 (docker build . fails on next on sneak's Mac).
What is known. sneak's log (https://git.eeqj.de/attachments/91faff25-fbb8-4c33-a411-4983b49a354e, gzipped; summary in #261 (comment)): on his Mac's Lima VM builder (default 4 CPUs, 4 GiB, arm64), RUN make test fails because the internal/service/auth test binary is killed (signal: killed, no --- FAIL line) after 4.6 s, almost certainly by the out-of-memory killer. HashPassword and VerifyPassword run argon2id at 64 MiB per hash, and the auth tests call them in many parallel subtests; internal/handlers tests also create users. script/test runs go test -race, and other packages' tests run at the same time.
Steps.
Fresh clone of next in your own mktemp -d /srv/code/tmp/... directory. Measure the total peak memory of make test as the build runs it on a 4-CPU machine: GOMAXPROCS=4 make test (the go command and the test binaries both take their parallelism from it). Measure the sum over all processes at once, not one process: GNU time is not installed, and its maximum resident size is the largest single process anyway. A cgroup's memory.peak (for example under systemd-run --user --scope, if that works here) or sampling the resident size of the whole process tree from /proc are both fine. Also measure the auth package alone. Do not use container memory limits (docker run --memory fails here).
Fix the cause so the peak of GOMAXPROCS=4 make test stays well inside 4 GiB (aim for about half of it or less), choosing from what the measurement shows. Constraints:
Production password hashing stays exactly as strong: the argon2id parameters upaasd uses are unchanged.
Never weaken a test: every test keeps checking what it checks now, and at least one test still hashes and verifies with the production parameters.
Small and plain. Acceptable shapes, if the numbers support them: running the hash-heavy auth subtests one at a time instead of in parallel; tests using a cheaper argon2id cost through one obvious, plainly named setting that production code never changes; capping how many packages script/test runs at once. No new helper layers or checking machinery.
Re-measure. The PR body states the peak before and after in one or two lines (sneak asked for the measurement in the PR), plus the method in one line.
make check, and docker build . once on the final head (under the shared lock, below). Remove any image or container you create.
Branch and PR. Branch off current next; PR base next, never main. Pull next before starting, and pull-rebase-resolve again right before pushing, re-running make check after any resolution. Commit title ends (closes #261). Label the PR needs-review, assign clawbot. Push every commit at once.
Rules.
Gate only through make targets or script/ (make check, make test, make lint, make fmt); never raw go test/go vet/golangci-lint. All linting runs in Docker through make lint.
No scripted edits: never change a file with sed -i, perl -pi, awk, a python replace or a shell splice. Read and edit by hand; formatting only through make fmt.
Comprehensibility is a merge criterion: a newcomer must follow the change in one reading; no coined identifiers or terms.
Shared host (~18 sessions): run docker build, make lint and your one authoritative make check under flock -w 1800 /srv/code/tmp/docker-gate.lock ...; flock -w exits 1 without running the command on timeout, so tell that apart from a red gate. Measurement runs stay outside the lock, one at a time, as few as needed. Reap only processes you started (record PIDs, trap/wait); never pkill, killall or anything chosen from ps. Name any exit-status file by head SHA. Apart from the sanctioned measurement runs, stay under 2 to 3 GiB.
Tracker text: plain language, no jargon, full links (https://git.eeqj.de/sneak/upaas/issues/N or /pulls/N, never a bare #N), PR body at most about 250 words, no logs or pass/fail notes (the before/after peak line is the one exception), escape </> as </>, no 4-byte emoji. Every comment, the PR body and the commit body end with Model: opus-5-5. Never mention any company or product name, no Co-Authored-By or session trailers.
Never ask an interactive question; nobody is at the terminal. Decide from this brief and the issue; a point only sneak can settle goes on the PR as a question with your recommendation, then stop.
upaas has one installation (sneak's); no migrations or compatibility code.
Cleanup at the end: two separate Bash calls, first only cd /srv/code/tmp, then only rm -rf <your temp dir>.
Final report: PR link, the before/after peak, any disclosure, then a line done.
Model: opus-5-5
Implementer's brief for https://git.eeqj.de/sneak/upaas/issues/261 (`docker build .` fails on `next` on sneak's Mac).
**What is known.** sneak's log (https://git.eeqj.de/attachments/91faff25-fbb8-4c33-a411-4983b49a354e, gzipped; summary in https://git.eeqj.de/sneak/upaas/issues/261#issuecomment-114957): on his Mac's Lima VM builder (default 4 CPUs, 4 GiB, arm64), `RUN make test` fails because the `internal/service/auth` test binary is killed (`signal: killed`, no `--- FAIL` line) after 4.6 s, almost certainly by the out-of-memory killer. `HashPassword` and `VerifyPassword` run argon2id at 64 MiB per hash, and the auth tests call them in many parallel subtests; `internal/handlers` tests also create users. `script/test` runs `go test -race`, and other packages' tests run at the same time.
**Steps.**
1. Fresh clone of `next` in your own `mktemp -d /srv/code/tmp/...` directory. Measure the total peak memory of `make test` as the build runs it on a 4-CPU machine: `GOMAXPROCS=4 make test` (the go command and the test binaries both take their parallelism from it). Measure the sum over all processes at once, not one process: GNU `time` is not installed, and its maximum resident size is the largest single process anyway. A cgroup's `memory.peak` (for example under `systemd-run --user --scope`, if that works here) or sampling the resident size of the whole process tree from `/proc` are both fine. Also measure the auth package alone. Do not use container memory limits (`docker run --memory` fails here).
2. Fix the cause so the peak of `GOMAXPROCS=4 make test` stays well inside 4 GiB (aim for about half of it or less), choosing from what the measurement shows. Constraints:
- Production password hashing stays exactly as strong: the argon2id parameters `upaasd` uses are unchanged.
- Never weaken a test: every test keeps checking what it checks now, and at least one test still hashes and verifies with the production parameters.
- Small and plain. Acceptable shapes, if the numbers support them: running the hash-heavy auth subtests one at a time instead of in parallel; tests using a cheaper argon2id cost through one obvious, plainly named setting that production code never changes; capping how many packages `script/test` runs at once. No new helper layers or checking machinery.
3. Re-measure. The PR body states the peak before and after in one or two lines (sneak asked for the measurement in the PR), plus the method in one line.
4. `make check`, and `docker build .` once on the final head (under the shared lock, below). Remove any image or container you create.
**Branch and PR.** Branch off current `next`; PR base `next`, never `main`. Pull `next` before starting, and pull-rebase-resolve again right before pushing, re-running `make check` after any resolution. Commit title ends ` (closes #261)`. Label the PR `needs-review`, assign `clawbot`. Push every commit at once.
**Rules.**
- Gate only through `make` targets or `script/` (`make check`, `make test`, `make lint`, `make fmt`); never raw `go test`/`go vet`/`golangci-lint`. All linting runs in Docker through `make lint`.
- No scripted edits: never change a file with `sed -i`, `perl -pi`, `awk`, a python replace or a shell splice. Read and edit by hand; formatting only through `make fmt`.
- Comprehensibility is a merge criterion: a newcomer must follow the change in one reading; no coined identifiers or terms.
- Shared host (~18 sessions): run `docker build`, `make lint` and your one authoritative `make check` under `flock -w 1800 /srv/code/tmp/docker-gate.lock ...`; `flock -w` exits 1 without running the command on timeout, so tell that apart from a red gate. Measurement runs stay outside the lock, one at a time, as few as needed. Reap only processes you started (record PIDs, `trap`/`wait`); never `pkill`, `killall` or anything chosen from `ps`. Name any exit-status file by head SHA. Apart from the sanctioned measurement runs, stay under 2 to 3 GiB.
- Tracker text: plain language, no jargon, full links (`https://git.eeqj.de/sneak/upaas/issues/N` or `/pulls/N`, never a bare `#N`), PR body at most about 250 words, no logs or pass/fail notes (the before/after peak line is the one exception), escape `<`/`>` as `<`/`>`, no 4-byte emoji. Every comment, the PR body and the commit body end with `Model: opus-5-5`. Never mention any company or product name, no `Co-Authored-By` or session trailers.
- Never ask an interactive question; nobody is at the terminal. Decide from this brief and the issue; a point only sneak can settle goes on the PR as a question with your recommendation, then stop.
- upaas has one installation (sneak's); no migrations or compatibility code.
- Cleanup at the end: two separate Bash calls, first only `cd /srv/code/tmp`, then only `rm -rf <your temp dir>`.
- Final report: PR link, the before/after peak, any disclosure, then a line `done`.
Model: opus-5-5
Cause: upaasd hashes passwords with argon2id at 64 MiB per hash, the auth and handlers tests run up to 13 and 4 such hashes at once, and the race detector multiplies what each one takes. In the Dockerfile's Go image, GOMAXPROCS=4 make test on next peaks at 3.1 GiB, 2.6 GiB of it the auth test binary. A 4 GiB machine also needs memory for its own system, so that binary is killed. The PR lowers the tests' hash memory to 1 MiB and keeps upaasd's 64 MiB.
No PR merged on 2026-10-01 caused it, #256 included: the tests that hash many passwords at once date from February. Reviews build on a host with far more than 4 GiB of memory, where nothing is killed.
Model: opus-5-5
PR: https://git.eeqj.de/sneak/upaas/pulls/276
Cause: upaasd hashes passwords with argon2id at 64 MiB per hash, the auth and handlers tests run up to 13 and 4 such hashes at once, and the race detector multiplies what each one takes. In the Dockerfile's Go image, `GOMAXPROCS=4 make test` on `next` peaks at 3.1 GiB, 2.6 GiB of it the auth test binary. A 4 GiB machine also needs memory for its own system, so that binary is killed. The PR lowers the tests' hash memory to 1 MiB and keeps upaasd's 64 MiB.
No PR merged on 2026-10-01 caused it, https://git.eeqj.de/sneak/upaas/pulls/256 included: the tests that hash many passwords at once date from February. Reviews build on a host with far more than 4 GiB of memory, where nothing is killed.
Model: opus-5-5
The fix is on next as 2a2c520 (#276), and milestone PR #244 carries it. Tests now hash passwords with 1 MiB instead of 64 MiB, which brings the peak memory of the test run in the build from about 3.1 GiB to 1.0 GiB on 4 CPUs. upaasd still hashes with 64 MiB.
No PR merged on 2026-10-01 caused it. The test of concurrent setup requests, which runs ten 64 MiB hashes at once under the race detector, dates from February (763e722). Reviews ran on a machine with far more memory and never measured the peak against a 4 GiB builder.
Not tried on a 4 GiB arm64 machine like yours.
Model: opus-5-5
The fix is on `next` as `2a2c520` (https://git.eeqj.de/sneak/upaas/pulls/276), and milestone PR https://git.eeqj.de/sneak/upaas/pulls/244 carries it. Tests now hash passwords with 1 MiB instead of 64 MiB, which brings the peak memory of the test run in the build from about 3.1 GiB to 1.0 GiB on 4 CPUs. upaasd still hashes with 64 MiB.
No PR merged on 2026-10-01 caused it. The test of concurrent setup requests, which runs ten 64 MiB hashes at once under the race detector, dates from February (`763e722`). Reviews ran on a machine with far more memory and never measured the peak against a 4 GiB builder.
Not tried on a 4 GiB arm64 machine like yours.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Owner (chat, 2026-10-01 ~23:03 UTC):
Standing rule from the same hour (sneak/project-management#20):
mainandnextare green on every managed repo at all times, and any divergence is top priority. This comes before all other upaas work.Definition of done:
nexthead (docker build .in the repo root, plusscript/cibuildormake check), and post the failing step and its cause here in a few lines.nextwith an independent review, rebased ontonextright before the review, then squashed. Never weaken a test.docker build .and the check pass on a fresh clone of the newnexthead. One line here says so, with the SHA.model: opus-5-5
Not reproduced yet. On a fresh clone of
nextat5ca1bce,docker build .succeeds here (Docker Engine 29.8, linux/amd64), andmake checkpasses on the same tree. The merge commit5ca1bcechanges no files compared with9488e2f, so it is not the cause.So the failure depends on the machine or the working copy it runs in. The leading suspect: since #236 (
a48d90f),.dockerignoresends the working copy's whole.gitdirectory and every git-ignored file except.env,bin/,.vscode/,.idea/and*.testinto the build. A fresh clone has none of those local files, and the review of 236 gated only fresh clones, which is why it passed.@sneak: which step fails, and with what error? The last 20 or so lines of
docker build --progress=plain ., plus the machine (OS, CPU) anddocker version, are enough. Until then a worker is checking the build against a working copy with typical local files.Model: opus-5-5
Still not reproduced, on
nextat7685812. Tried here:docker build .andmake dockerfrom a working copy set up like a developer's:make bootstraprun (sonode_modules/exists), git-ignored local files (.env.local, a*.key, a*.out,.DS_Store), thedata/directory upaasd creates when run from the checkout, and an uncommitted change. The build does not fail.make test, run as the build runs it, on one CPU and on two: every package stays far inside the 30-second limit, so a slower machine is not the cause..gitthat is a file pointing outside the checkout, andcore.fsmonitorset in.git/configas on macOS: neither breaks the build.Not tested: an arm64 machine such as an Apple Silicon Mac (no emulation on this host).
None of the 2026-10-01 merges, #256 included, changes the
Dockerfile,.dockerignore, theMakefile,script/or the dependencies. The only change onnextthat makes the build depend on the working copy is #236 (a48d90f, 2026-09-29).Found on the way, not this failure: since #236 the build stages receive the working copy's git-ignored secrets (
.env.local,*.key,data/session.key); the final image holds only the binary.data/, the default data directory, is not git-ignored.No fix opened: there is no known cause to fix. @sneak: still needed are the last 20 or so lines of
docker build --progress=plain .from the copy where it fails, and the machine (OS, CPU).Model: opus-5-5
nextatd3b9c6fpassesdocker build .andmake checkon a fresh clone here. Since the last report two changes narrowed what the build receives from a working copy: git-ignored files anddata/stay out (#266) and so does.git/config(#269). If the failure came from local files in your copy, it may be gone.@sneak: please run
docker build --progress=plain .again ond3b9c6fwhere it failed; if it still fails, the last 20 or so lines and the machine (OS, CPU) are what is needed.Model: opus-5-5
sneak sent the failing build log in chat (2026-10-03 00:1x UTC). It is
docker build .onnext, run 2026-10-02 on his Mac (heraklion, builderdocker:lima-default). Full log, gzipped: https://git.eeqj.de/attachments/91faff25-fbb8-4c33-a411-4983b49a354eWhat fails:
RUN make test(Dockerfile line 33). Every package passes exceptinternal/service/auth, whose test binary is killed after 4.6 s:Likely cause: there is no
--- FAILline, so no test assertion failed. Something outside the test killed the process, most likely the out-of-memory killer in the Lima VM, whose default size is 4 GiB. The auth tests run argon2id (64 MiB per hash, 4 threads) in parallel subtests:TestVerifyPassword,TestCreateUserRaceConditionand others were running when it was killed.script/testrunsgo test -race, and the race detector multiplies memory use several times over. Other packages' tests run at the same time. That also explains why it never failed here: sandcastle has far more memory.The
build session failed ... unable to upgrade to h2clines in the log come fromTestBuildImageLogsBuildErrorBeforeDeployError, which passed. They are not the failure.Next (upaas manager):
make test, with the auth package's share, on a fresh clone.docker run --memoryfails on sandcastle, so use a measurement such as/usr/bin/time -v, not a container limit.docker build .passes there. Production password hashing must not get weaker.nextwith an independent review, then close this issue and 264.Model: opus-5-5
Ruling from sneak (chat, 2026-10-03 00:2x UTC): running the test suite must not take more memory than his Lima VM has, which is 4 GB.
Definition of done for this issue, updated: the peak memory of
make test, and so ofdocker build ., stays well under 4 GB, measured on a fresh clone and reported in the PR. Production password hashing stays as strong as it is now.Model: opus-5-5
Implementer's brief for #261 (
docker build .fails onnexton sneak's Mac).What is known. sneak's log (https://git.eeqj.de/attachments/91faff25-fbb8-4c33-a411-4983b49a354e, gzipped; summary in #261 (comment)): on his Mac's Lima VM builder (default 4 CPUs, 4 GiB, arm64),
RUN make testfails because theinternal/service/authtest binary is killed (signal: killed, no--- FAILline) after 4.6 s, almost certainly by the out-of-memory killer.HashPasswordandVerifyPasswordrun argon2id at 64 MiB per hash, and the auth tests call them in many parallel subtests;internal/handlerstests also create users.script/testrunsgo test -race, and other packages' tests run at the same time.Steps.
nextin your ownmktemp -d /srv/code/tmp/...directory. Measure the total peak memory ofmake testas the build runs it on a 4-CPU machine:GOMAXPROCS=4 make test(the go command and the test binaries both take their parallelism from it). Measure the sum over all processes at once, not one process: GNUtimeis not installed, and its maximum resident size is the largest single process anyway. A cgroup'smemory.peak(for example undersystemd-run --user --scope, if that works here) or sampling the resident size of the whole process tree from/procare both fine. Also measure the auth package alone. Do not use container memory limits (docker run --memoryfails here).GOMAXPROCS=4 make teststays well inside 4 GiB (aim for about half of it or less), choosing from what the measurement shows. Constraints:upaasduses are unchanged.script/testruns at once. No new helper layers or checking machinery.make check, anddocker build .once on the final head (under the shared lock, below). Remove any image or container you create.Branch and PR. Branch off current
next; PR basenext, nevermain. Pullnextbefore starting, and pull-rebase-resolve again right before pushing, re-runningmake checkafter any resolution. Commit title ends(closes #261). Label the PRneeds-review, assignclawbot. Push every commit at once.Rules.
maketargets orscript/(make check,make test,make lint,make fmt); never rawgo test/go vet/golangci-lint. All linting runs in Docker throughmake lint.sed -i,perl -pi,awk, a python replace or a shell splice. Read and edit by hand; formatting only throughmake fmt.docker build,make lintand your one authoritativemake checkunderflock -w 1800 /srv/code/tmp/docker-gate.lock ...;flock -wexits 1 without running the command on timeout, so tell that apart from a red gate. Measurement runs stay outside the lock, one at a time, as few as needed. Reap only processes you started (record PIDs,trap/wait); neverpkill,killallor anything chosen fromps. Name any exit-status file by head SHA. Apart from the sanctioned measurement runs, stay under 2 to 3 GiB.https://git.eeqj.de/sneak/upaas/issues/Nor/pulls/N, never a bare#N), PR body at most about 250 words, no logs or pass/fail notes (the before/after peak line is the one exception), escape</>as</>, no 4-byte emoji. Every comment, the PR body and the commit body end withModel: opus-5-5. Never mention any company or product name, noCo-Authored-Byor session trailers.cd /srv/code/tmp, then onlyrm -rf <your temp dir>.done.Model: opus-5-5
PR: #276
Cause: upaasd hashes passwords with argon2id at 64 MiB per hash, the auth and handlers tests run up to 13 and 4 such hashes at once, and the race detector multiplies what each one takes. In the Dockerfile's Go image,
GOMAXPROCS=4 make testonnextpeaks at 3.1 GiB, 2.6 GiB of it the auth test binary. A 4 GiB machine also needs memory for its own system, so that binary is killed. The PR lowers the tests' hash memory to 1 MiB and keeps upaasd's 64 MiB.No PR merged on 2026-10-01 caused it, #256 included: the tests that hash many passwords at once date from February. Reviews build on a host with far more than 4 GiB of memory, where nothing is killed.
Model: opus-5-5
The fix is on
nextas2a2c520(#276), and milestone PR #244 carries it. Tests now hash passwords with 1 MiB instead of 64 MiB, which brings the peak memory of the test run in the build from about 3.1 GiB to 1.0 GiB on 4 CPUs. upaasd still hashes with 64 MiB.No PR merged on 2026-10-01 caused it. The test of concurrent setup requests, which runs ten 64 MiB hashes at once under the race detector, dates from February (
763e722). Reviews ran on a machine with far more memory and never measured the peak against a 4 GiB builder.Not tried on a 4 GiB arm64 machine like yours.
Model: opus-5-5