make test takes over 20 seconds: every run compiles everything with -race #129

Open
opened 2026-10-07 04:25:36 +02:00 by clawbot · 6 comments
Collaborator

Since #128, script/test builds the test phase of the Dockerfile with --no-cache, as REPO_POLICIES.md requires. On this host --no-cache also starts Go's build cache mount empty, so every make test compiles the standard library and every dependency with -race before any test runs; that is about 40 s of a run. make test takes 51 to 60 s here, and about 90 s when the host is loaded: over the policy's 20-second target, and over its 60-second cap under load.

Definition of done

  • make test with -race takes under 20 seconds in script/cibuild on this host, measured twice, within what sneak rules may be cached (question on #128).

Model: opus-5-5

Since https://git.eeqj.de/sneak/secret/pulls/128, `script/test` builds the test phase of the `Dockerfile` with `--no-cache`, as `REPO_POLICIES.md` requires. On this host `--no-cache` also starts Go's build cache mount empty, so every `make test` compiles the standard library and every dependency with `-race` before any test runs; that is about 40 s of a run. `make test` takes 51 to 60 s here, and about 90 s when the host is loaded: over the policy's 20-second target, and over its 60-second cap under load. ## Definition of done - `make test` with `-race` takes under 20 seconds in `script/cibuild` on this host, measured twice, within what sneak rules may be cached (question on https://git.eeqj.de/sneak/secret/pulls/128). Model: opus-5-5
Author
Collaborator

Question for sneak: the policy at sneak/prompts dd4027b, now vendored here (#128), requires --no-cache on every build in script/ and caps make test at 60 s. On this host --no-cache also empties Go's build cache mount, so each make test compiles the standard library and every dependency with -race first: 51 to 60 s quiet, about 90 s loaded, against the 20-second target. Before this, with the cache mount kept, it was about 20 s. This will hit every Go repository that runs -race.

Options:

  1. Let check builds keep Go's build cache mount (recommended): script/test and script/lint rebuild every layer without --no-cache emptying the mount, for example by busting the layer cache with a build argument as CHECK_EPOCH did. -count=1 keeps test results from being reused. Needs a change to REPO_POLICIES.md in sneak/prompts, then a re-vendor here.
  2. Rebuild only the test stage (--no-cache-filter=test) on top of a cached stage that compiles the dependencies; a departure from the policy's --no-cache for this repository only. Unverified whether that keeps the mount.
  3. Keep the policy as written and accept make test over the target, and over the cap under load.

Model: opus-5-5

**Question for sneak:** the policy at `sneak/prompts` `dd4027b`, now vendored here (https://git.eeqj.de/sneak/secret/pulls/128), requires `--no-cache` on every build in `script/` and caps `make test` at 60 s. On this host `--no-cache` also empties Go's build cache mount, so each `make test` compiles the standard library and every dependency with `-race` first: 51 to 60 s quiet, about 90 s loaded, against the 20-second target. Before this, with the cache mount kept, it was about 20 s. This will hit every Go repository that runs `-race`. Options: 1. Let check builds keep Go's build cache mount (recommended): `script/test` and `script/lint` rebuild every layer without `--no-cache` emptying the mount, for example by busting the layer cache with a build argument as `CHECK_EPOCH` did. `-count=1` keeps test results from being reused. Needs a change to `REPO_POLICIES.md` in `sneak/prompts`, then a re-vendor here. 2. Rebuild only the test stage (`--no-cache-filter=test`) on top of a cached stage that compiles the dependencies; a departure from the policy's `--no-cache` for this repository only. Unverified whether that keeps the mount. 3. Keep the policy as written and accept `make test` over the target, and over the cap under load. Model: opus-5-5
sneak was assigned by clawbot 2026-10-07 05:43:13 +02:00
Author
Collaborator

Ruling (sneak, 2026-10-15, in chat): option 1 approved. Go's compile cache may survive between check builds; test results are still never reused (-count=1). The change lands upstream first, in REPO_POLICIES.md in sneak/prompts (tracked on sneak/prompts#113), then this repo re-vendors it and make test is measured again against the 20-second target.

Model: opus-5-5

**Ruling (sneak, 2026-10-15, in chat): option 1 approved.** Go's compile cache may survive between check builds; test results are still never reused (`-count=1`). The change lands upstream first, in `REPO_POLICIES.md` in `sneak/prompts` (tracked on https://git.eeqj.de/sneak/prompts/issues/113), then this repo re-vendors it and `make test` is measured again against the 20-second target. Model: opus-5-5
sneak was unassigned by clawbot 2026-10-15 10:32:07 +02:00
clawbot self-assigned this 2026-10-15 10:32:07 +02:00
Author
Collaborator

Plan. One PR on next re-vendors the canonical files from sneak/prompts next at e3f79c4, the branch sneak/prompts#129 merged into (main there does not have the change yet). The last re-vendor here was dd4027b; everything that changed between the two comes in.

  • Every file vendored at dd4027b is brought to e3f79c4, with this repository's own entries kept after the canonical content: REPO_POLICIES.md, the script/ files, .gitea/workflows/check.yml, .gitignore, .editorconfig, .golangci.yml, and the README where it describes them.
  • script/lint, script/test, script/cibuild and script/docker pass a new CACHEBUST value in place of --no-cache; the two gate builds write no image.
  • Dockerfile: every stage, the runtime stage included, declares ARG CACHEBUST as its first line after FROM. Go's build cache is mounted the way the canonical Go Dockerfile mounts it, on every step that compiles (both go vet runs, both golangci-lint run runs, both go test runs, the build) and nothing else. Both go test runs keep -count=1. The macOS checks, cgo and make build stay.
  • One Completed Steps entry at the top of TODO.md.

Done when:

  • On the second of two back-to-back script/cibuild runs on an unchanged tree, no RUN step of any stage reports CACHED.
  • The test build inside script/cibuild takes under 20 seconds with -race, on both of two script/cibuild runs in a fresh clone. If it does not, the issue says by how much and why.

Model: opus-5-5

**Plan.** One PR on `next` re-vendors the canonical files from `sneak/prompts` `next` at `e3f79c4`, the branch https://git.eeqj.de/sneak/prompts/pulls/129 merged into (`main` there does not have the change yet). The last re-vendor here was `dd4027b`; everything that changed between the two comes in. - Every file vendored at `dd4027b` is brought to `e3f79c4`, with this repository's own entries kept after the canonical content: `REPO_POLICIES.md`, the `script/` files, `.gitea/workflows/check.yml`, `.gitignore`, `.editorconfig`, `.golangci.yml`, and the README where it describes them. - `script/lint`, `script/test`, `script/cibuild` and `script/docker` pass a new `CACHEBUST` value in place of `--no-cache`; the two gate builds write no image. - `Dockerfile`: every stage, the runtime stage included, declares `ARG CACHEBUST` as its first line after `FROM`. Go's build cache is mounted the way the canonical Go `Dockerfile` mounts it, on every step that compiles (both `go vet` runs, both `golangci-lint run` runs, both `go test` runs, the build) and nothing else. Both `go test` runs keep `-count=1`. The macOS checks, cgo and `make build` stay. - One Completed Steps entry at the top of `TODO.md`. Done when: - On the second of two back-to-back `script/cibuild` runs on an unchanged tree, no `RUN` step of any stage reports `CACHED`. - The test build inside `script/cibuild` takes under 20 seconds with `-race`, on both of two `script/cibuild` runs in a fresh clone. If it does not, the issue says by how much and why. Model: opus-5-5
Author
Collaborator

Built in #193, as planned above. The 20-second target is not met: on two fresh-clone runs the test build was 3 s and 21 s over. Compiling now takes under a second; what remains is downloading the Go modules again on every build (by policy no module cache is kept) and the internal/cli tests under -race, both slower when the shared host is busy.

Model: opus-5-5

Built in https://git.eeqj.de/sneak/secret/pulls/193, as planned above. The 20-second target is not met: on two fresh-clone runs the test build was 3 s and 21 s over. Compiling now takes under a second; what remains is downloading the Go modules again on every build (by policy no module cache is kept) and the `internal/cli` tests under `-race`, both slower when the shared host is busy. Model: opus-5-5
Author
Collaborator

Measured on next at 26a4739, after #193 merged, with Go's compile cache warm: two script/cibuild runs in a fresh clone, after one run to warm the cache. The test build took 33 s and 29 s, 13 s and 9 s over the 20-second target.

  • Compiling no longer costs anything noticeable.
  • go mod download takes about 7 s of every build, since no module cache is kept.
  • go test -race takes 21 to 26 s, and the internal/cli tests are 17 to 21 s of that.

A kept module cache, asked on sneak/prompts#130, would save the 7 s and leave about 22 to 26 s: still over. What remains is how long the internal/cli tests themselves run under -race.

Model: opus-5-5

Measured on `next` at `26a4739`, after https://git.eeqj.de/sneak/secret/pulls/193 merged, with Go's compile cache warm: two `script/cibuild` runs in a fresh clone, after one run to warm the cache. The test build took 33 s and 29 s, 13 s and 9 s over the 20-second target. - Compiling no longer costs anything noticeable. - `go mod download` takes about 7 s of every build, since no module cache is kept. - `go test -race` takes 21 to 26 s, and the `internal/cli` tests are 17 to 21 s of that. A kept module cache, asked on https://git.eeqj.de/sneak/prompts/issues/130, would save the 7 s and leave about 22 to 26 s: still over. What remains is how long the `internal/cli` tests themselves run under `-race`. Model: opus-5-5
clawbot reopened this issue 2026-10-15 20:27:02 +02:00
Author
Collaborator

Reopened: the re-vendor merged (#193) but the definition of done above, make test under 20 seconds, is not met: 33 s and 29 s with the cache warm (#129 (comment)). What is left is about 7 s of module download, which waits on sneak/prompts#130, and 17-21 s in internal/cli's tests, which is work in this repo when it next has a manager. Not a bug in behaviour, so it does not affect v1.0.0-rc4.

Model: opus-5-5

Reopened: the re-vendor merged (https://git.eeqj.de/sneak/secret/pulls/193) but the definition of done above, `make test` under 20 seconds, is not met: 33 s and 29 s with the cache warm (https://git.eeqj.de/sneak/secret/issues/129#issuecomment-143605). What is left is about 7 s of module download, which waits on https://git.eeqj.de/sneak/prompts/issues/130, and 17-21 s in `internal/cli`'s tests, which is work in this repo when it next has a manager. Not a bug in behaviour, so it does not affect `v1.0.0-rc4`. Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/secret#129