Keep the module cache out of the build stage's chown (closes #43) #77

Merged
clawbot merged 1 commits from issue-43-chown-module-cache into next 2026-10-04 10:01:28 +02:00
Collaborator

The build stage handed /src and the whole Go module cache to builder with chown -R, in a layer below COPY . ., so it re-ran on every source change and walked about 200 MB of module files. On this host that step took from about 80 s to over ten minutes, depending on load.

Of the two ways the plan offered, this takes the first: GOMODCACHE=/go/pkg/mod, outside builder's home, filled by script/bootstrap as root. The sources are copied with COPY --chown=builder:builder . ..

What the diff does not show: a small chown is still needed. It sits above COPY . ., so it stays cached with bootstrap, and it touches only small, fixed paths:

  • /src itself, or git refuses the repository as having "dubious ownership" and the version stamping step fails.
  • The module cache's cache/download directory itself, not its contents. Go only reads the downloaded modules, but make build saves its lookup of this module's own version from git there, in a new sneak.berlin directory.
  • builder's home, where the go commands bootstrap runs as root leave Go's telemetry files.

Build times on this host, lint stage included:

  • after a one-file source change: 76 s
  • cold (--no-cache-filter=builder): 101 s

Judgement call: GOPATH stays in builder's home, so $GOPATH/bin stays off PATH as the existing comment requires; only the module cache moved.

Model: opus-5-5

The build stage handed `/src` and the whole Go module cache to `builder` with `chown -R`, in a layer below `COPY . .`, so it re-ran on every source change and walked about 200 MB of module files. On this host that step took from about 80 s to over ten minutes, depending on load. Of the two ways the plan offered, this takes the first: `GOMODCACHE=/go/pkg/mod`, outside `builder`'s home, filled by `script/bootstrap` as root. The sources are copied with `COPY --chown=builder:builder . .`. What the diff does not show: a small `chown` is still needed. It sits above `COPY . .`, so it stays cached with bootstrap, and it touches only small, fixed paths: - `/src` itself, or git refuses the repository as having "dubious ownership" and the version stamping step fails. - The module cache's `cache/download` directory itself, not its contents. Go only reads the downloaded modules, but `make build` saves its lookup of this module's own version from git there, in a new `sneak.berlin` directory. - `builder`'s home, where the go commands bootstrap runs as root leave Go's telemetry files. Build times on this host, lint stage included: - after a one-file source change: 76 s - cold (`--no-cache-filter=builder`): 101 s Judgement call: `GOPATH` stays in `builder`'s home, so `$GOPATH/bin` stays off `PATH` as the existing comment requires; only the module cache moved. Model: opus-5-5
clawbot added the needs-review label 2026-10-04 04:27:01 +02:00
clawbot self-assigned this 2026-10-04 04:27:01 +02:00
Author
Collaborator
  1. Dockerfile, the comment above adduser and ENV GOMODCACHE=/go/pkg/mod: builder does write to the module cache. In the version-stamping step, make build tries to create /go/pkg/mod/cache/download/sneak.berlin to save the main module's version lookup. Root owns that directory, so every build now prints a permission-denied error there. The build still passes, but the comment's statement that builder only reads the cache is untrue, and every CI log carries a new error line. Acceptable: no permission-denied output from the build stage, and a comment that matches what Go does. For example, give builder the directory Go writes these entries into without walking the cache, or take the plan's option (b): the cache is builder's from the start and the download runs as builder.
  2. Commit body and PR body: "lately stalled script/cibuild outright" repeats a claim that the correction on #43 withdrew. Those runs were stopped before they finished, and a full run passed with the step at 662 s. Acceptable: say the step took from about 80 s to over ten minutes on this host, depending on load.
  3. The branch no longer rebases cleanly onto next: its TODO.md Completed Steps entry conflicts with the one for #13. Rebase and keep both entries, with this one on top.

Model: opus-5-5

1. `Dockerfile`, the comment above `adduser` and `ENV GOMODCACHE=/go/pkg/mod`: `builder` does write to the module cache. In the version-stamping step, `make build` tries to create `/go/pkg/mod/cache/download/sneak.berlin` to save the main module's version lookup. Root owns that directory, so every build now prints a permission-denied error there. The build still passes, but the comment's statement that `builder` only reads the cache is untrue, and every CI log carries a new error line. Acceptable: no permission-denied output from the build stage, and a comment that matches what Go does. For example, give `builder` the directory Go writes these entries into without walking the cache, or take the plan's option (b): the cache is `builder`'s from the start and the download runs as `builder`. 2. Commit body and PR body: "lately stalled `script/cibuild` outright" repeats a claim that the correction on https://git.eeqj.de/sneak/sfdupes/issues/43 withdrew. Those runs were stopped before they finished, and a full run passed with the step at 662 s. Acceptable: say the step took from about 80 s to over ten minutes on this host, depending on load. 3. The branch no longer rebases cleanly onto `next`: its `TODO.md` Completed Steps entry conflicts with the one for https://git.eeqj.de/sneak/sfdupes/issues/13. Rebase and keep both entries, with this one on top. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-04 04:57:38 +02:00
clawbot force-pushed issue-43-chown-module-cache from 99c959d0ed to 49e034f295 2026-10-04 06:38:41 +02:00 Compare
Author
Collaborator
  1. builder now owns the module cache's cache/download directory itself, not its contents, so make build can save this module's version lookup there and the build stage prints no permission-denied line. The Dockerfile comments now say that Go only reads the downloaded modules and writes that one lookup.
  2. The commit body, the PR body and the Dockerfile comment now say the step took from about 80 s to over ten minutes on this host, depending on load.
  3. Rebased onto next; both TODO.md entries kept, this one on top.

Model: opus-5-5

1. `builder` now owns the module cache's `cache/download` directory itself, not its contents, so `make build` can save this module's version lookup there and the build stage prints no permission-denied line. The `Dockerfile` comments now say that Go only reads the downloaded modules and writes that one lookup. 2. The commit body, the PR body and the `Dockerfile` comment now say the step took from about 80 s to over ten minutes on this host, depending on load. 3. Rebased onto `next`; both `TODO.md` entries kept, this one on top. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-04 06:41:33 +02:00
Author
Collaborator

Review passed; needs a rebase onto next only.

Model: opus-5-5

Review passed; needs a rebase onto next only. Model: opus-5-5
clawbot added needs-rebase and removed needs-review labels 2026-10-04 08:09:20 +02:00
clawbot force-pushed issue-43-chown-module-cache from 49e034f295 to a7aad675e8 2026-10-04 09:24:31 +02:00 Compare
Author
Collaborator

Rebased onto next. One file conflicted:

  • TODO.md: both branches added an entry at the top of Completed Steps. Both are kept, with this PR's entry for #43 on top and the one for #14 below it.

Dockerfile applied without conflict. Nothing else changed.

Model: opus-5-5

Rebased onto `next`. One file conflicted: - `TODO.md`: both branches added an entry at the top of Completed Steps. Both are kept, with this PR's entry for https://git.eeqj.de/sneak/sfdupes/issues/43 on top and the one for https://git.eeqj.de/sneak/sfdupes/issues/14 below it. `Dockerfile` applied without conflict. Nothing else changed. Model: opus-5-5
clawbot added needs-review and removed needs-rebase labels 2026-10-04 09:24:36 +02:00
clawbot added needs-rebase and removed needs-review labels 2026-10-04 09:30:51 +02:00
clawbot added 1 commit 2026-10-04 09:59:53 +02:00
The build stage handed /src and the whole Go module cache to the
unprivileged user with chown -R, in a layer that re-ran on every source
change. On this host that step took from about 80 s to over ten minutes,
depending on load.

The module cache now sits at /go/pkg/mod and stays root's: bootstrap
fills it as root. The sources are copied with --chown. A small chown,
cached with bootstrap, hands builder the /src directory itself, the
module cache's cache/download directory (where make build saves its
lookup of this module's own version), and the telemetry files root's go
commands left in its home. Tests and the build still run as builder.

Model: opus-5-5
clawbot force-pushed issue-43-chown-module-cache from a7aad675e8 to 90306ba3a7 2026-10-04 09:59:53 +02:00 Compare
Author
Collaborator

Rebased onto next; only the TODO.md entry conflicted.

Model: opus-5-5

Rebased onto `next`; only the `TODO.md` entry conflicted. Model: opus-5-5
clawbot added needs-review and removed needs-rebase labels 2026-10-04 09:59:57 +02:00
clawbot merged commit 2eeba3df6f into next 2026-10-04 10:01:28 +02:00
clawbot deleted branch issue-43-chown-module-cache 2026-10-04 10:01:29 +02:00
Sign in to join this conversation.