May Go's module download cache survive between check builds too? #130

Closed
opened 2026-10-15 18:18:14 +02:00 by clawbot · 9 comments
Collaborator

On 2026-10-15 sneak ruled that Go's compile cache may survive between check builds (sneak/secret#129 (comment)); the policy change is merged here (#129). Measured since, with the compile cache kept:

  • homoicon: make check takes about 75 seconds on the re-vendored branch (sneak/homoicon#1505), against his requirement that it finish in under 60 seconds even on a loaded host (sneak/homoicon#895). It was 29 seconds before the re-vendor.
  • secret: make test takes 23 to 41 seconds, against the 20-second target (sneak/secret#129).

Most of what remains is go mod download in every Go stage, 6 to 16 seconds each, because the policy still keeps no module cache.

Question for sneak: may Go's module download cache survive between check builds too, the way the compile cache now does?

Recommendation: yes. The downloaded modules are checked against go.sum on every build, so a kept cache cannot let a wrong dependency through, and tests still never reuse results. If no, homoicon's make check stays over its 60 seconds, and the re-vendor there waits for another way under it.

Model: opus-5-5

On 2026-10-15 sneak ruled that Go's compile cache may survive between check builds (https://git.eeqj.de/sneak/secret/issues/129#issuecomment-142366); the policy change is merged here (https://git.eeqj.de/sneak/prompts/pulls/129). Measured since, with the compile cache kept: - `homoicon`: `make check` takes about 75 seconds on the re-vendored branch (https://git.eeqj.de/sneak/homoicon/pulls/1505), against his requirement that it finish in under 60 seconds even on a loaded host (https://git.eeqj.de/sneak/homoicon/issues/895). It was 29 seconds before the re-vendor. - `secret`: `make test` takes 23 to 41 seconds, against the 20-second target (https://git.eeqj.de/sneak/secret/issues/129). Most of what remains is `go mod download` in every Go stage, 6 to 16 seconds each, because the policy still keeps no module cache. **Question for sneak:** may Go's module download cache survive between check builds too, the way the compile cache now does? **Recommendation:** yes. The downloaded modules are checked against `go.sum` on every build, so a kept cache cannot let a wrong dependency through, and tests still never reuse results. If no, homoicon's `make check` stays over its 60 seconds, and the re-vendor there waits for another way under it. Model: opus-5-5
sneak was assigned by clawbot 2026-10-15 18:18:14 +02:00
Author
Collaborator

Correction to the figures above: the host's own build-cache purges (every 3 hours, and the 12-hourly reclaim) were removing the kept compile cache between builds, so some of those runs compiled everything from scratch. Fixed at 16:45 UTC today; the purges now keep the compile-cache mounts (sneak/sfdupes#109 (comment)). The homoicon and secret times are being measured again with the cache actually present; the question stands until those numbers are in, and I will post them here.

Model: opus-5-5

Correction to the figures above: the host's own build-cache purges (every 3 hours, and the 12-hourly reclaim) were removing the kept compile cache between builds, so some of those runs compiled everything from scratch. Fixed at 16:45 UTC today; the purges now keep the compile-cache mounts (https://git.eeqj.de/sneak/sfdupes/issues/109#issuecomment-143356). The homoicon and secret times are being measured again with the cache actually present; the question stands until those numbers are in, and I will post them here. Model: opus-5-5
Author
Collaborator

homoicon, measured again since the 16:45 UTC fix, with Go's build cache kept and the module cache not kept: make check on sneak/homoicon#1505 (head fa6f5163), run three times in a row under the shared build lock.

  • Warm-up run: 65 s (host load 35 to 49 on 48 cores).
  • Measured run 1: 90 s (load 49 to 65).
  • Measured run 2: 68 s (load 65 to 72).

Every test package ran in both measured runs, and no test result came from a cache. The longest single step is golangci-lint run (16 to 29 s). Each Go phase also downloads the modules again (about 5 s each). Both measured runs are over the 60 seconds sneak/homoicon#895 requires, so sneak/homoicon#1505 keeps waiting for the answer here.

Model: opus-5-5

homoicon, measured again since the 16:45 UTC fix, with Go's build cache kept and the module cache not kept: `make check` on https://git.eeqj.de/sneak/homoicon/pulls/1505 (head `fa6f5163`), run three times in a row under the shared build lock. - Warm-up run: 65 s (host load 35 to 49 on 48 cores). - Measured run 1: 90 s (load 49 to 65). - Measured run 2: 68 s (load 65 to 72). Every test package ran in both measured runs, and no test result came from a cache. The longest single step is `golangci-lint run` (16 to 29 s). Each Go phase also downloads the modules again (about 5 s each). Both measured runs are over the 60 seconds https://git.eeqj.de/sneak/homoicon/issues/895 requires, so https://git.eeqj.de/sneak/homoicon/pulls/1505 keeps waiting for the answer here. Model: opus-5-5
Author
Collaborator

Re-measured with the compile cache really kept (#130 (comment)): homoicon's make check still takes 65 to 90 seconds at the host's current load, over its 60. The two biggest remaining steps are golangci-lint run (16 to 29 seconds, its own analysis cache is not kept either) and go mod download in each Go stage.

The question, sharpened: may both Go's module download cache and golangci-lint's analysis cache survive between check builds, as the compile cache now does?

Recommendation: yes to both. Modules are checked against go.sum on every build, and golangci-lint's cache is keyed on file contents, so neither can hand back a stale result for changed code; every test still runs fresh (-count=1). Keeping only the module cache would probably still leave homoicon over 60 seconds at this load.

Model: opus-5-5

Re-measured with the compile cache really kept (https://git.eeqj.de/sneak/prompts/issues/130#issuecomment-143402): `homoicon`'s `make check` still takes 65 to 90 seconds at the host's current load, over its 60. The two biggest remaining steps are `golangci-lint run` (16 to 29 seconds, its own analysis cache is not kept either) and `go mod download` in each Go stage. **The question, sharpened:** may both Go's module download cache and golangci-lint's analysis cache survive between check builds, as the compile cache now does? **Recommendation:** yes to both. Modules are checked against `go.sum` on every build, and golangci-lint's cache is keyed on file contents, so neither can hand back a stale result for changed code; every test still runs fresh (`-count=1`). Keeping only the module cache would probably still leave `homoicon` over 60 seconds at this load. Model: opus-5-5
Author
Collaborator

A third case, from pixa (#113 (comment)): its make test takes about 95 seconds with Go's cache kept, over the 60-second cap. Twelve seconds are the build and the tests; the rest is installing the Debian libvips-dev package, which reruns on every build because it comes after the ARG CACHEBUST line.

The question, whole: may check builds keep these three as well as Go's compile cache: Go's downloaded modules, golangci-lint's analysis cache, and the layer that installs system packages (by placing that install before the ARG CACHEBUST line)?

Recommendation: yes to all three. None of them can hand back a stale result for changed code (modules are checked against go.sum, the lint cache is keyed on file contents, and the package list is fixed in the Dockerfile), and every test still runs fresh. Without them homoicon stays at 65-90 seconds against its 60 and pixa at about 95 against the 60-second cap.

Model: opus-5-5

A third case, from `pixa` (https://git.eeqj.de/sneak/prompts/issues/113#issuecomment-143586): its `make test` takes about 95 seconds with Go's cache kept, over the 60-second cap. Twelve seconds are the build and the tests; the rest is installing the Debian `libvips-dev` package, which reruns on every build because it comes after the `ARG CACHEBUST` line. **The question, whole:** may check builds keep these three as well as Go's compile cache: Go's downloaded modules, golangci-lint's analysis cache, and the layer that installs system packages (by placing that install before the `ARG CACHEBUST` line)? **Recommendation:** yes to all three. None of them can hand back a stale result for changed code (modules are checked against `go.sum`, the lint cache is keyed on file contents, and the package list is fixed in the `Dockerfile`), and every test still runs fresh. Without them `homoicon` stays at 65-90 seconds against its 60 and `pixa` at about 95 against the 60-second cap. Model: opus-5-5
Author
Collaborator

secret measured again with Go's compile cache warm, after the host's build-cache purges were fixed (sneak/secret#129 (comment)): the test build takes 29 to 33 s. go mod download is about 7 s of that; the tests themselves are 21 to 26 s. A kept module cache would save those 7 s but would not, on its own, bring secret under its 20-second target.

Model: opus-5-5

`secret` measured again with Go's compile cache warm, after the host's build-cache purges were fixed (https://git.eeqj.de/sneak/secret/issues/129#issuecomment-143605): the test build takes 29 to 33 s. `go mod download` is about 7 s of that; the tests themselves are 21 to 26 s. A kept module cache would save those 7 s but would not, on its own, bring `secret` under its 20-second target. Model: opus-5-5
Owner

yes to all 3

yes to all 3
Author
Collaborator

Ruling (sneak, 2026-10-16): yes to all three. Besides Go's compile cache, check builds may keep Go's downloaded modules, golangci-lint's analysis cache, and the layer that installs system packages (placed before the ARG CACHEBUST line). Every test still runs fresh.

Next: a prompts session changes REPO_POLICIES.md and the templates; the repos then re-vendor it (homoicon PR 1505 and pixa wait on it). The fleet is paused at the primary account's weekly limit; this starts first when it resumes on 18 October 13:00 UTC.

Done when: the change is merged here and its commit named on this issue.

Model: opus-5-5

**Ruling (sneak, 2026-10-16):** yes to all three. Besides Go's compile cache, check builds may keep Go's downloaded modules, golangci-lint's analysis cache, and the layer that installs system packages (placed before the `ARG CACHEBUST` line). Every test still runs fresh. **Next:** a `prompts` session changes `REPO_POLICIES.md` and the templates; the repos then re-vendor it (homoicon PR 1505 and pixa wait on it). The fleet is paused at the primary account's weekly limit; this starts first when it resumes on 18 October 13:00 UTC. **Done when:** the change is merged here and its commit named on this issue. Model: opus-5-5
sneak was unassigned by clawbot 2026-10-16 09:18:24 +02:00
clawbot self-assigned this 2026-10-16 09:18:24 +02:00
Author
Collaborator

Plan, on sneak's ruling above: one PR on next, its title ending (closes #130).

The canonical Go Dockerfile in prompts/REPO_POLICIES.md changes in three places:

  1. System packages. Each stage's package install (apk add --no-cache git in the build stage, any apt-get install of libraries in the lint or test phase) moves above ARG CACHEBUST. Docker keeps that layer until the base image or the install line changes.
  2. Go's modules. In each Go stage, WORKDIR /src, COPY go.mod go.sum ./ and RUN go mod download move above ARG CACHEBUST. Docker keeps that layer until go.mod or go.sum changes. Go checks every module against go.sum when it downloads it, and no later build can change a kept layer. Judgement call: the modules are kept as a layer, not a cache mount. A cache mount is shared by every build on the host, and Go does not check its files again (only go mod verify does), which is the reason the current text gives for not keeping them.
  3. golangci-lint's analysis cache. The golangci-lint run step also mounts the directory golangci-lint keeps its cache in, in the pinned image, next to Go's build cache.

Everything after ARG CACHEBUST is unchanged: COPY . . and every lint, test and build step rerun on every check build, and both go test runs keep -count=1.

The text then says the same everywhere it states the rule: in each stage ARG CACHEBUST comes right after the steps that install system packages and download Go's modules, or right after FROM in a stage with neither, and every step that lints, tests or compiles comes after it. That covers prompts/REPO_POLICIES.md (the CACHEBUST rule, the list under the Dockerfile, the lint-phase apt-get example, the Makefile test section), both checklists, prompts/CODE_STYLEGUIDE_GO.md, README.md, and a TODO.md entry. This repo's own Dockerfile is a node one, and its single script/bootstrap step installs system and node packages together. The ruling does not cover node packages, so that step stays below ARG CACHEBUST.

Done when: merged to next, with each check shown by planting a defect in a scratch Go module built from the canonical Dockerfile while every cache is warm. A new lint finding, a changed .golangci.yml setting and a failing test each fail the next check build. The package and module layers come from the cache while go.mod and go.sum are unchanged.

Model: opus-5-5

**Plan**, on sneak's ruling above: one PR on `next`, its title ending `(closes #130)`. The canonical Go `Dockerfile` in `prompts/REPO_POLICIES.md` changes in three places: 1. **System packages.** Each stage's package install (`apk add --no-cache git` in the build stage, any `apt-get install` of libraries in the lint or test phase) moves above `ARG CACHEBUST`. Docker keeps that layer until the base image or the install line changes. 2. **Go's modules.** In each Go stage, `WORKDIR /src`, `COPY go.mod go.sum ./` and `RUN go mod download` move above `ARG CACHEBUST`. Docker keeps that layer until `go.mod` or `go.sum` changes. Go checks every module against `go.sum` when it downloads it, and no later build can change a kept layer. Judgement call: the modules are kept as a layer, not a cache mount. A cache mount is shared by every build on the host, and Go does not check its files again (only `go mod verify` does), which is the reason the current text gives for not keeping them. 3. **golangci-lint's analysis cache.** The `golangci-lint run` step also mounts the directory golangci-lint keeps its cache in, in the pinned image, next to Go's build cache. Everything after `ARG CACHEBUST` is unchanged: `COPY . .` and every lint, test and build step rerun on every check build, and both `go test` runs keep `-count=1`. The text then says the same everywhere it states the rule: in each stage `ARG CACHEBUST` comes right after the steps that install system packages and download Go's modules, or right after `FROM` in a stage with neither, and every step that lints, tests or compiles comes after it. That covers `prompts/REPO_POLICIES.md` (the `CACHEBUST` rule, the list under the `Dockerfile`, the lint-phase `apt-get` example, the `Makefile` test section), both checklists, `prompts/CODE_STYLEGUIDE_GO.md`, `README.md`, and a `TODO.md` entry. This repo's own `Dockerfile` is a node one, and its single `script/bootstrap` step installs system and node packages together. The ruling does not cover node packages, so that step stays below `ARG CACHEBUST`. **Done when:** merged to `next`, with each check shown by planting a defect in a scratch Go module built from the canonical `Dockerfile` while every cache is warm. A new lint finding, a changed `.golangci.yml` setting and a failing test each fail the next check build. The package and module layers come from the cache while `go.mod` and `go.sum` are unchanged. Model: opus-5-5
Author
Collaborator

Merged to next as 0653579 (065357920a), from #131. Repositories can re-vendor from it: each Go stage's package install and module download move above ARG CACHEBUST, and golangci-lint run mounts /root/.cache/golangci-lint.

Model: opus-5-5

Merged to `next` as `0653579` (https://git.eeqj.de/sneak/prompts/commit/065357920a0a09bd736bf33d4cb5ce935fdc2328), from https://git.eeqj.de/sneak/prompts/pulls/131. Repositories can re-vendor from it: each Go stage's package install and module download move above `ARG CACHEBUST`, and `golangci-lint run` mounts `/root/.cache/golangci-lint`. Model: opus-5-5
Sign in to join this conversation.
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/prompts#130