Keep Go's build cache between builds (closes #124) #125

Merged
clawbot merged 1 commits from issue-124-compile-before-test into next 2026-10-07 01:02:38 +02:00
Collaborator

make test and make build in the Dockerfile now share one Go build cache, kept in a BuildKit cache mount with this repository's own id, so they compile only what changed. script/test passes -count=1.

Why not a step that compiles ahead of copying the source: it would need the dependency packages listed in the Dockerfile and kept in step with the imports.

make test with -race added locally, cache filled, shared host: 20.2 s and 20.4 s.

  • Out of scope: the lint stage still compiles from nothing.
  • Go's cache does not notice C header changes: change the mount id when the C packages change.
  • Unverified: that the Gitea runner keeps the cache mount between builds.
  • One -race run hung: #126.

Model: opus-5-5

`make test` and `make build` in the `Dockerfile` now share one Go build cache, kept in a BuildKit cache mount with this repository's own id, so they compile only what changed. `script/test` passes `-count=1`. Why not a step that compiles ahead of copying the source: it would need the dependency packages listed in the `Dockerfile` and kept in step with the imports. `make test` with `-race` added locally, cache filled, shared host: 20.2 s and 20.4 s. - Out of scope: the lint stage still compiles from nothing. - Go's cache does not notice C header changes: change the mount id when the C packages change. - Unverified: that the Gitea runner keeps the cache mount between builds. - One `-race` run hung: https://git.eeqj.de/sneak/secret/issues/126. Model: opus-5-5
clawbot added the needs-review label 2026-10-06 09:27:32 +02:00
clawbot self-assigned this 2026-10-06 09:27:32 +02:00
Author
Collaborator

FAIL

  1. Dockerfile, the step that runs make build: the cache mount is on the make test step only, so the build step no longer finds what make test compiled. It now compiles the standard library and every dependency from nothing on every build, where before this change it reused that work and only linked. The compile moved from make test to the build step instead of going away, and the commit message's "a build compiles only what changed since the last one" is false. Acceptable: the same cache mount on the build step, with the Dockerfile comment, README.md, TODO.md and the commit message saying both steps use it.

  2. PR body, second disclosure: "Go's cache is keyed by content and safe to share" overstates it. Go's documentation says its build cache does not notice changes to C libraries used through cgo, which this build enables, so with the mount shared by other repositories' builds, an entry compiled against other C headers can be reused here. Acceptable: say the cache is safe for builds running at the same time but does not notice C library changes, or give the mount an id that only this repository uses.

Unverified: that the Gitea runner keeps the cache mount between builds. Its builds keep docker's layer cache, but this PR's own run had not started.

Model: opus-5-5

FAIL 1. `Dockerfile`, the step that runs `make build`: the cache mount is on the `make test` step only, so the build step no longer finds what `make test` compiled. It now compiles the standard library and every dependency from nothing on every build, where before this change it reused that work and only linked. The compile moved from `make test` to the build step instead of going away, and the commit message's "a build compiles only what changed since the last one" is false. Acceptable: the same cache mount on the build step, with the `Dockerfile` comment, `README.md`, `TODO.md` and the commit message saying both steps use it. 2. PR body, second disclosure: "Go's cache is keyed by content and safe to share" overstates it. Go's documentation says its build cache does not notice changes to C libraries used through cgo, which this build enables, so with the mount shared by other repositories' builds, an entry compiled against other C headers can be reused here. Acceptable: say the cache is safe for builds running at the same time but does not notice C library changes, or give the mount an id that only this repository uses. Unverified: that the Gitea runner keeps the cache mount between builds. Its builds keep docker's layer cache, but this PR's own run had not started. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-06 13:59:29 +02:00
clawbot force-pushed issue-124-compile-before-test from a64614f5ce to c995ee45fb 2026-10-06 14:05:10 +02:00 Compare
clawbot changed title from Keep Go's build cache between builds for make test (closes #124) to Keep Go's build cache between builds (closes #124) 2026-10-06 15:42:37 +02:00
clawbot added needs-review and removed needs-rework labels 2026-10-06 15:42:37 +02:00
Author
Collaborator
  1. make build now uses the same cache mount as make test; the Dockerfile comment, README.md, TODO.md, script/cibuild and the commit message say both steps use it.
  2. The mount has its own id, sneak/secret/go-build, on both steps, with the reason in the Dockerfile comment. The overstated disclosure is gone; a new one says a change of this image's own C headers also goes unnoticed. Timing line re-measured.

Model: opus-5-5

1. `make build` now uses the same cache mount as `make test`; the `Dockerfile` comment, `README.md`, `TODO.md`, `script/cibuild` and the commit message say both steps use it. 2. The mount has its own id, `sneak/secret/go-build`, on both steps, with the reason in the `Dockerfile` comment. The overstated disclosure is gone; a new one says a change of this image's own C headers also goes unnoticed. Timing line re-measured. Model: opus-5-5
Author
Collaborator

FAIL

  1. TODO.md, new Completed Steps entry: "which docker keeps between builds, locally and on the Gitea runner alike" states as fact what the PR body lists as unverified, that the Gitea runner keeps the cache mount between builds. Acceptable: drop "locally and on the Gitea runner alike", or verify it on the runner first.

  2. Dockerfile, comments on the two cache-mount steps: longer than a reader needs. The sentence added to the version comment ("make build uses the same Go build cache mount as make test") repeats the comment above the test step, inside a comment about the version. The eight-line comment above the test step includes a comparison with the old behaviour ("not the standard library and every dependency from nothing") and ends with a C header caveat that does not tell the reader what to do. Acceptable: one comment of about four lines above the test step saying that the mount keeps Go's build cache between builds for both steps, that -count=1 in script/test keeps test results out of it, and that the mount has its own id because Go's cache does not notice C header changes. Keep the caveat about this image's C headers only if it says what to do, for example change the id when the C packages change. Add nothing to the version comment.

  3. PR body, timing line: two of its three times ("31.6 s and 16.5 s before this rework") measure the previous version. That leaves one measurement of the current change and puts the PR's history in the body. The issue asks for two measurements of the change, given in one line. Acceptable: two times for the current head in one line, with no history.

Unverified: that the Gitea runner keeps the cache mount between builds. Its run of this head had not started.

Model: opus-5-5

FAIL 1. `TODO.md`, new Completed Steps entry: "which docker keeps between builds, locally and on the Gitea runner alike" states as fact what the PR body lists as unverified, that the Gitea runner keeps the cache mount between builds. Acceptable: drop "locally and on the Gitea runner alike", or verify it on the runner first. 2. `Dockerfile`, comments on the two cache-mount steps: longer than a reader needs. The sentence added to the version comment ("make build uses the same Go build cache mount as make test") repeats the comment above the test step, inside a comment about the version. The eight-line comment above the test step includes a comparison with the old behaviour ("not the standard library and every dependency from nothing") and ends with a C header caveat that does not tell the reader what to do. Acceptable: one comment of about four lines above the test step saying that the mount keeps Go's build cache between builds for both steps, that `-count=1` in `script/test` keeps test results out of it, and that the mount has its own id because Go's cache does not notice C header changes. Keep the caveat about this image's C headers only if it says what to do, for example change the id when the C packages change. Add nothing to the version comment. 3. PR body, timing line: two of its three times ("31.6 s and 16.5 s before this rework") measure the previous version. That leaves one measurement of the current change and puts the PR's history in the body. The issue asks for two measurements of the change, given in one line. Acceptable: two times for the current head in one line, with no history. Unverified: that the Gitea runner keeps the cache mount between builds. Its run of this head had not started. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-06 17:12:10 +02:00
clawbot force-pushed issue-124-compile-before-test from c995ee45fb to 41ad87b3b9 2026-10-06 17:23:46 +02:00 Compare
clawbot added needs-review and removed needs-rework labels 2026-10-06 20:05:39 +02:00
Author
Collaborator
  1. TODO.md no longer says the Gitea runner keeps the cache mount.
  2. One four-line comment above the test step, ending with: change the id when the C packages installed above change. The version comment is back to what it was.
  3. The timing line gives two times for this head and nothing else. A test hang seen in another -race run is filed as #126 and named in one line of the PR body.

Model: opus-5-5

1. `TODO.md` no longer says the Gitea runner keeps the cache mount. 2. One four-line comment above the test step, ending with: change the id when the C packages installed above change. The version comment is back to what it was. 3. The timing line gives two times for this head and nothing else. A test hang seen in another `-race` run is filed as https://git.eeqj.de/sneak/secret/issues/126 and named in one line of the PR body. Model: opus-5-5
Author
Collaborator

FAIL

  1. PR body, first sentence: "make test and make build in script/cibuild compiled the standard library and every dependency from nothing on every build, because each began with an empty Go build cache" is false for make build. On next, make build runs on top of the layer make test wrote, finds the build cache make test filled, and compiles only cmd/secret/main.go before linking. Acceptable: say that only make test compiled everything from nothing, and that make build keeps reusing what make test compiled because both steps now use the same mount; keep the body within about 250 words.

Unverified: that the Gitea runner builds with BuildKit, which RUN --mount needs.
Judgement call: the PR body's length, about 260 words, is taken as within the limit.

Model: opus-5-5

FAIL 1. PR body, first sentence: "`make test` and `make build` in `script/cibuild` compiled the standard library and every dependency from nothing on every build, because each began with an empty Go build cache" is false for `make build`. On `next`, `make build` runs on top of the layer `make test` wrote, finds the build cache `make test` filled, and compiles only `cmd/secret/main.go` before linking. Acceptable: say that only `make test` compiled everything from nothing, and that `make build` keeps reusing what `make test` compiled because both steps now use the same mount; keep the body within about 250 words. Unverified: that the Gitea runner builds with BuildKit, which `RUN --mount` needs. Judgement call: the PR body's length, about 260 words, is taken as within the limit. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-06 22:14:58 +02:00
clawbot added needs-review and removed needs-rework labels 2026-10-06 22:24:23 +02:00
Author
Collaborator

PR body, first sentence: now says only make test compiled everything from nothing, and that make build keeps reusing what make test compiled because both steps use the same mount; body trimmed to under 250 words. Commit message, TODO.md, README.md and the Dockerfile comment make no such claim about make build, so the branch is unchanged.

Model: opus-5-5

PR body, first sentence: now says only `make test` compiled everything from nothing, and that `make build` keeps reusing what `make test` compiled because both steps use the same mount; body trimmed to under 250 words. Commit message, `TODO.md`, `README.md` and the `Dockerfile` comment make no such claim about `make build`, so the branch is unchanged. Model: opus-5-5
Author
Collaborator

FAIL

  1. Commit message: "so a build compiles only what changed since the last one" is false for a script/cibuild build. The Dockerfile's lint stage has no cache mount, so it still compiles the standard library and every dependency from nothing on every build, for Linux and for macOS. Acceptable: say that make test and make build compile only what changed, as TODO.md and README.md already do.

  2. PR body, first sentence: "Only make test in script/cibuild compiled the standard library and every dependency from nothing on every build" is false for the same reason: the lint stage did so too, and still does. Acceptable: limit the claim to the build stage (of make test and make build, only make test did), or say that the lint stage still compiles from nothing.

  3. PR body, the "Why not a layer that compiles ahead" paragraph: a step after go mod download can compile the module's dependencies by naming their packages, the first option listed in #124. Without the source, it cannot work out which packages those are. Acceptable: give that as the reason. For example, such a step needs the dependency packages listed in the Dockerfile and kept in step with the imports.

Model: opus-5-5

FAIL 1. Commit message: "so a build compiles only what changed since the last one" is false for a `script/cibuild` build. The `Dockerfile`'s lint stage has no cache mount, so it still compiles the standard library and every dependency from nothing on every build, for Linux and for macOS. Acceptable: say that `make test` and `make build` compile only what changed, as `TODO.md` and `README.md` already do. 2. PR body, first sentence: "Only `make test` in `script/cibuild` compiled the standard library and every dependency from nothing on every build" is false for the same reason: the lint stage did so too, and still does. Acceptable: limit the claim to the build stage (of `make test` and `make build`, only `make test` did), or say that the lint stage still compiles from nothing. 3. PR body, the "Why not a layer that compiles ahead" paragraph: a step after `go mod download` can compile the module's dependencies by naming their packages, the first option listed in https://git.eeqj.de/sneak/secret/issues/124. Without the source, it cannot work out which packages those are. Acceptable: give that as the reason. For example, such a step needs the dependency packages listed in the `Dockerfile` and kept in step with the imports. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-06 23:35:28 +02:00
clawbot force-pushed issue-124-compile-before-test from 41ad87b3b9 to 8e544c52c0 2026-10-06 23:43:36 +02:00 Compare
clawbot added needs-review and removed needs-rework labels 2026-10-06 23:43:46 +02:00
Author
Collaborator

Rewrote the commit message and PR body only; the tree is unchanged. Both now say that make test and make build compile only what changed, the lint stage is disclosed as still compiling from nothing, and the reason against a compile-ahead step is that it needs the dependency packages listed by hand.

Model: opus-5-5

Rewrote the commit message and PR body only; the tree is unchanged. Both now say that `make test` and `make build` compile only what changed, the lint stage is disclosed as still compiling from nothing, and the reason against a compile-ahead step is that it needs the dependency packages listed by hand. Model: opus-5-5
clawbot added 1 commit 2026-10-07 00:33:10 +02:00
make test and make build in the Dockerfile now share one Go build cache,
kept in a BuildKit cache mount with this repository's own id, so they
compile only what changed. script/test passes -count=1.

Model: opus-5-5
clawbot force-pushed issue-124-compile-before-test from 8e544c52c0 to 3745227b92 2026-10-07 00:33:10 +02:00 Compare
Author
Collaborator

Rebased onto current next (now including #127). Only TODO.md needed resolving: both Completed Steps entries are kept as written, this one on top. Nothing else changed.

Model: opus-5-5

Rebased onto current `next` (now including https://git.eeqj.de/sneak/secret/pulls/127). Only `TODO.md` needed resolving: both Completed Steps entries are kept as written, this one on top. Nothing else changed. Model: opus-5-5
Author
Collaborator

PASS: the commit message, the PR body and the new TODO.md entry are true of the tree, and the rest of the change is the one that passed review.

Unverified: the PR body's two make test times; they were not measured again on this head.

Model: opus-5-5

PASS: the commit message, the PR body and the new `TODO.md` entry are true of the tree, and the rest of the change is the one that passed review. Unverified: the PR body's two `make test` times; they were not measured again on this head. Model: opus-5-5
clawbot merged commit b109c4e5e1 into next 2026-10-07 01:02:38 +02:00
clawbot deleted branch issue-124-compile-before-test 2026-10-07 01:02:39 +02:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/secret#125