Does the 60-second hard cap on make test include building the test phase?
Your ruling on #41 (2026-08-10) set it: "org wide. the hard cap is 60 for ci/green, but over 20s should be filed as an improvement bug." Tests then ran on the host, with Go's build cache. make test is now an uncached docker build --no-cache --target test, so it also compiles everything from scratch with -race and saves the image. In dnswatcher (sneak/dnswatcher#259) that takes about 63 s for the test step plus 25 s to save the image, while its slowest test package takes 11.4 s. Read literally, most Go repositories now fail the cap without a slow test.
Options:
The cap covers the test run (the time go test reports); building the test phase counts against the existing 5-minute Docker build limit.
The cap covers all of make test, build included. Most Go repositories with -race exceed it.
Raise the cap for the uncached build.
Recommendation: option 1. It keeps what the cap measured when you set it, fast tests, and the 5-minute limit already bounds the build. The policy text gets one sentence saying so.
Model: opus-5-5
Does the 60-second hard cap on `make test` include building the test phase?
Your ruling on https://git.eeqj.de/sneak/prompts/issues/41 (2026-08-10) set it: "org wide. the hard cap is 60 for ci/green, but over 20s should be filed as an improvement bug." Tests then ran on the host, with Go's build cache. `make test` is now an uncached `docker build --no-cache --target test`, so it also compiles everything from scratch with `-race` and saves the image. In `dnswatcher` (https://git.eeqj.de/sneak/dnswatcher/pulls/259) that takes about 63 s for the test step plus 25 s to save the image, while its slowest test package takes 11.4 s. Read literally, most Go repositories now fail the cap without a slow test.
Options:
1. The cap covers the test run (the time `go test` reports); building the test phase counts against the existing 5-minute Docker build limit.
2. The cap covers all of `make test`, build included. Most Go repositories with `-race` exceed it.
3. Raise the cap for the uncached build.
Recommendation: option 1. It keeps what the cap measured when you set it, fast tests, and the 5-minute limit already bounds the build. The policy text gets one sentence saying so.
Model: opus-5-5
sneak
was assigned by clawbot2026-10-06 04:30:29 +02:00
Same question for the 5-minute Docker build limit, from #112 (closed: Go build cache mounts are caching, which you ruled out on #40, and --no-cache empties them anyway). On bsfirehose, after its re-vendor, on the shared build host at a load of about 100–150: make test 158 s, script/cibuild 454 s. Every compiling step starts from an empty Go build cache, and script/cibuild runs lint and test twice. If option 1 is taken for the test cap, the 5-minute limit would bound these builds alone, and bsfirehose breaks it too.
Model: opus-5-5
Same question for the 5-minute Docker build limit, from https://git.eeqj.de/sneak/prompts/issues/112 (closed: Go build cache mounts are caching, which you ruled out on https://git.eeqj.de/sneak/prompts/issues/40, and `--no-cache` empties them anyway). On `bsfirehose`, after its re-vendor, on the shared build host at a load of about 100–150: `make test` 158 s, `script/cibuild` 454 s. Every compiling step starts from an empty Go build cache, and `script/cibuild` runs lint and test twice. If option 1 is taken for the test cap, the 5-minute limit would bound these builds alone, and `bsfirehose` breaks it too.
Model: opus-5-5
Restated with a second ruling that the question above leaves out. This replaces it.
Two of your rulings cannot both hold for larger Go repositories on this host:
#40 (2026-08-10): lint and test run as Docker build phases "using no caching".
#41 (2026-08-10): make test has a 60-second hard cap, org wide; and sneak/homoicon#895 (2026-09-02): homoicon's make check finishes in under 60 seconds on this host.
With no cache, every make test compiles the module and all its dependencies from scratch before a test runs. That alone puts make test at about three minutes in homoicon (sneak/homoicon#1505) and 158 s in bsfirehose. #123 saves about 16 s per build, which is not enough.
Which gives way?
No caching stays. The 60-second limits count only the time the tests take to run; compiling counts against the 5-minute build limit. homoicon's make check limit is read the same way.
The 60-second limits stay as written. The lint and test builds keep Go's compiled packages between runs; every lint and test step still runs every time and no test result is reused. This partly reverses the no-caching ruling.
Raise the limits.
Recommendation: 2. It is the only option that brings homoicon's make check back toward 60 seconds, which is what sneak/homoicon#895 asked for; #112 measured a test step at 8 s with the compiled packages kept against 127 s without. This changes the recommendation above.
Model: opus-5-5
Restated with a second ruling that the question above leaves out. This replaces it.
Two of your rulings cannot both hold for larger Go repositories on this host:
- https://git.eeqj.de/sneak/prompts/issues/40 (2026-08-10): lint and test run as Docker build phases "using no caching".
- https://git.eeqj.de/sneak/prompts/issues/41 (2026-08-10): `make test` has a 60-second hard cap, org wide; and https://git.eeqj.de/sneak/homoicon/issues/895 (2026-09-02): homoicon's `make check` finishes in under 60 seconds on this host.
With no cache, every `make test` compiles the module and all its dependencies from scratch before a test runs. That alone puts `make test` at about three minutes in homoicon (https://git.eeqj.de/sneak/homoicon/pulls/1505) and 158 s in bsfirehose. https://git.eeqj.de/sneak/prompts/issues/123 saves about 16 s per build, which is not enough.
Which gives way?
1. No caching stays. The 60-second limits count only the time the tests take to run; compiling counts against the 5-minute build limit. homoicon's `make check` limit is read the same way.
2. The 60-second limits stay as written. The lint and test builds keep Go's compiled packages between runs; every lint and test step still runs every time and no test result is reused. This partly reverses the no-caching ruling.
3. Raise the limits.
Recommendation: 2. It is the only option that brings homoicon's `make check` back toward 60 seconds, which is what https://git.eeqj.de/sneak/homoicon/issues/895 asked for; https://git.eeqj.de/sneak/prompts/issues/112 measured a test step at 8 s with the compiled packages kept against 127 s without. This changes the recommendation above.
Model: opus-5-5
Correction to the bsfirehose figures above: since sneak/bsfirehose#96 replaced its cgo SQLite driver, script/cibuild there takes 259 s, under the 5-minute limit, and make test 68 s, with its slowest test package at 3.6 s. Under option 1 it meets both limits.
Model: opus-5-5
Correction to the `bsfirehose` figures above: since https://git.eeqj.de/sneak/bsfirehose/issues/96 replaced its cgo SQLite driver, `script/cibuild` there takes 259 s, under the 5-minute limit, and `make test` 68 s, with its slowest test package at 3.6 s. Under option 1 it meets both limits.
Model: opus-5-5
Ruling (sneak, 2026-10-15, in chat, on sneak/secret#129): Go's compile cache may survive between check builds. So REPO_POLICIES.md changes: --no-cache still rebuilds every layer of a check build, but Go's build cache mount is kept (for example by busting the layer cache with a build argument instead of emptying the mount), and tests still never reuse results (-count=1).
With the cache kept, building the test phase costs seconds, so the question this issue asked (whether the 60-second cap includes that build) no longer decides any repo's result; if a repo's make test still exceeds 60 seconds with the cache, it comes back to sneak here.
Next: one PR on next here with the policy change, then each repo re-vendors (homoicon PR 1505 and issue 1484, sfdupes PR 97, pixa issue 221, secret issue 129).
Model: opus-5-5
**Ruling (sneak, 2026-10-15, in chat, on https://git.eeqj.de/sneak/secret/issues/129): Go's compile cache may survive between check builds.** So `REPO_POLICIES.md` changes: `--no-cache` still rebuilds every layer of a check build, but Go's build cache mount is kept (for example by busting the layer cache with a build argument instead of emptying the mount), and tests still never reuse results (`-count=1`).
With the cache kept, building the test phase costs seconds, so the question this issue asked (whether the 60-second cap includes that build) no longer decides any repo's result; if a repo's `make test` still exceeds 60 seconds with the cache, it comes back to sneak here.
Next: one PR on `next` here with the policy change, then each repo re-vendors (homoicon PR 1505 and issue 1484, sfdupes PR 97, pixa issue 221, secret issue 129).
Model: opus-5-5
sneak
was unassigned by clawbot2026-10-15 10:32:10 +02:00
clawbot
self-assigned this 2026-10-15 10:32:10 +02:00
Plan: keep Go's build cache in check builds, rerun every step, reuse no test result. One PR on next, title ending (closes #113).
Checked on this host: docker build --no-cache gives a RUN --mount=type=cache step an empty mount and replaces the stored one, so no cache mount can survive it. A build argument whose value changes on every run reruns every RUN step after its ARG line and keeps the mount.
Scripts (script/lint, script/test, script/cibuild, script/docker): replace --no-cache with --build-arg CACHEBUST="$(date +%s)-$$" (CACHEBUST is the usual name for this in Dockerfiles; the value is the time and the script's process id, so no two builds share it, including the three in one script/cibuild). Rewrite each script's comment to say why in one or two lines.
Dockerfiles: every stage declares ARG CACHEBUST as its first line after FROM, so every RUN step in it runs on every check build. COPY steps are compared by content and need nothing. This repo's own Dockerfile (three stages) and the canonical Go Dockerfile in REPO_POLICIES.md. In the Go one, the steps that compile (golangci-lint run, both go test runs, go build) get --mount=type=cache,target=/root/.cache/go-build: Go's build cache only, never golangci-lint's own cache (#30) and never the module cache. Both go test runs in the test phase get -count=1.
Text: rewrite the --no-cache rule in REPO_POLICIES.md (both copies, kept byte-identical): check builds pass a new CACHEBUST value each run; every stage declares it first; why not --no-cache (it empties the cache mounts); a stage missing the ARG is served from cache, so a re-vendor takes the scripts and the ARG lines together. Fix the gate snippets, the version snippet, the "image build is uncached" sentence, and the paragraph that says the test phase needs no -count=1. Same in both checklists, README.md, CODE_STYLEGUIDE_GO.md. Bump last_modified; add a TODO.md entry.
Done when: two script/test runs in a row on an unchanged tree each rerun every RUN step; on a scratch Go module built from the new canonical Dockerfile, the second test build reuses compiled packages, prints no (cached) test, and a planted failing test and a planted lint finding each fail script/cibuild; script/cibuild green on a fresh clone.
Model: opus-5-5
**Plan: keep Go's build cache in check builds, rerun every step, reuse no test result.** One PR on `next`, title ending `(closes #113)`.
Checked on this host: `docker build --no-cache` gives a `RUN --mount=type=cache` step an empty mount and replaces the stored one, so no cache mount can survive it. A build argument whose value changes on every run reruns every `RUN` step after its `ARG` line and keeps the mount.
**Scripts** (`script/lint`, `script/test`, `script/cibuild`, `script/docker`): replace `--no-cache` with `--build-arg CACHEBUST="$(date +%s)-$$"` (`CACHEBUST` is the usual name for this in Dockerfiles; the value is the time and the script's process id, so no two builds share it, including the three in one `script/cibuild`). Rewrite each script's comment to say why in one or two lines.
**Dockerfiles**: every stage declares `ARG CACHEBUST` as its first line after `FROM`, so every `RUN` step in it runs on every check build. `COPY` steps are compared by content and need nothing. This repo's own `Dockerfile` (three stages) and the canonical Go `Dockerfile` in `REPO_POLICIES.md`. In the Go one, the steps that compile (`golangci-lint run`, both `go test` runs, `go build`) get `--mount=type=cache,target=/root/.cache/go-build`: Go's build cache only, never golangci-lint's own cache (https://git.eeqj.de/sneak/prompts/issues/30) and never the module cache. Both `go test` runs in the test phase get `-count=1`.
**Text**: rewrite the `--no-cache` rule in `REPO_POLICIES.md` (both copies, kept byte-identical): check builds pass a new `CACHEBUST` value each run; every stage declares it first; why not `--no-cache` (it empties the cache mounts); a stage missing the `ARG` is served from cache, so a re-vendor takes the scripts and the `ARG` lines together. Fix the gate snippets, the version snippet, the "image build is uncached" sentence, and the paragraph that says the test phase needs no `-count=1`. Same in both checklists, `README.md`, `CODE_STYLEGUIDE_GO.md`. Bump `last_modified`; add a `TODO.md` entry.
**Done when**: two `script/test` runs in a row on an unchanged tree each rerun every `RUN` step; on a scratch Go module built from the new canonical `Dockerfile`, the second test build reuses compiled packages, prints no `(cached)` test, and a planted failing test and a planted lint finding each fail `script/cibuild`; `script/cibuild` green on a fresh clone.
Model: opus-5-5
Built in #129. The script/ builds pass a new CACHEBUST value in place of --no-cache, every Dockerfile stage declares ARG CACHEBUST first, and the canonical Go Dockerfile mounts Go's build cache on the steps that compile, with -count=1 on both go test runs. Repositories pick it up at their next re-vendor, taking the scripts and the ARG lines in one commit.
Model: opus-5-5
Built in https://git.eeqj.de/sneak/prompts/pulls/129. The `script/` builds pass a new `CACHEBUST` value in place of `--no-cache`, every `Dockerfile` stage declares `ARG CACHEBUST` first, and the canonical Go `Dockerfile` mounts Go's build cache on the steps that compile, with `-count=1` on both `go test` runs. Repositories pick it up at their next re-vendor, taking the scripts and the `ARG` lines in one commit.
Model: opus-5-5
Merged to next as e3f79c49da (from #129). To re-vendor, a repository takes the four script/ builds, an ARG CACHEBUST line first in every Dockerfile stage and, for Go, the build cache mounts and -count=1 from the canonical Dockerfile, all in one commit.
Model: opus-5-5
Merged to `next` as https://git.eeqj.de/sneak/prompts/commit/e3f79c49daf7cc5b29134427db4ae7298a4a63c0 (from https://git.eeqj.de/sneak/prompts/pulls/129). To re-vendor, a repository takes the four `script/` builds, an `ARG CACHEBUST` line first in every `Dockerfile` stage and, for Go, the build cache mounts and `-count=1` from the canonical `Dockerfile`, all in one commit.
Model: opus-5-5
Question from pixa: its make test is still over 60 seconds with Go's build cache kept (sneak/pixa#276), about 95 seconds on a second run. Building and running the tests takes about 12 of them. The rest is the test phase installing Debian's libvips-dev and the 289 packages it pulls in, which reruns on every build because it comes after the ARG CACHEBUST line. Before the re-vendor, make test took about as long, then mostly compiling.
Options:
A stage may install its system packages before its ARG CACHEBUST line, so that step is reused like the base image and every step that builds or checks the code still reruns. pixa's make test would be about the 12 seconds of build and tests.
Accept a make test over 60 seconds for repositories whose test phase installs system libraries.
Keep pixa's test phase on the Alpine Go image, where the same libraries install in about 20 seconds, against the policy's Debian test image.
Recommendation: 1. The package step installs what the build needs and checks nothing, so reusing it skips no check, as reusing the base image skips none.
Model: opus-5-5
Question from `pixa`: its `make test` is still over 60 seconds with Go's build cache kept (https://git.eeqj.de/sneak/pixa/pulls/276), about 95 seconds on a second run. Building and running the tests takes about 12 of them. The rest is the test phase installing Debian's `libvips-dev` and the 289 packages it pulls in, which reruns on every build because it comes after the `ARG CACHEBUST` line. Before the re-vendor, `make test` took about as long, then mostly compiling.
Options:
1. A stage may install its system packages before its `ARG CACHEBUST` line, so that step is reused like the base image and every step that builds or checks the code still reruns. `pixa`'s `make test` would be about the 12 seconds of build and tests.
2. Accept a `make test` over 60 seconds for repositories whose test phase installs system libraries.
3. Keep `pixa`'s test phase on the Alpine Go image, where the same libraries install in about 20 seconds, against the policy's Debian test image.
Recommendation: 1. The package step installs what the build needs and checks nothing, so reusing it skips no check, as reusing the base image skips none.
Model: opus-5-5
The question above is carried, with the other caches it belongs with, on the open #130 (comment) (assigned to sneak), so it is asked in one place.
Model: opus-5-5
The question above is carried, with the other caches it belongs with, on the open https://git.eeqj.de/sneak/prompts/issues/130#issuecomment-143600 (assigned to sneak), so it is asked in one place.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Does the 60-second hard cap on
make testinclude building the test phase?Your ruling on #41 (2026-08-10) set it: "org wide. the hard cap is 60 for ci/green, but over 20s should be filed as an improvement bug." Tests then ran on the host, with Go's build cache.
make testis now an uncacheddocker build --no-cache --target test, so it also compiles everything from scratch with-raceand saves the image. Indnswatcher(sneak/dnswatcher#259) that takes about 63 s for the test step plus 25 s to save the image, while its slowest test package takes 11.4 s. Read literally, most Go repositories now fail the cap without a slow test.Options:
go testreports); building the test phase counts against the existing 5-minute Docker build limit.make test, build included. Most Go repositories with-raceexceed it.Recommendation: option 1. It keeps what the cap measured when you set it, fast tests, and the 5-minute limit already bounds the build. The policy text gets one sentence saying so.
Model: opus-5-5
Same question for the 5-minute Docker build limit, from #112 (closed: Go build cache mounts are caching, which you ruled out on #40, and
--no-cacheempties them anyway). Onbsfirehose, after its re-vendor, on the shared build host at a load of about 100–150:make test158 s,script/cibuild454 s. Every compiling step starts from an empty Go build cache, andscript/cibuildruns lint and test twice. If option 1 is taken for the test cap, the 5-minute limit would bound these builds alone, andbsfirehosebreaks it too.Model: opus-5-5
Restated with a second ruling that the question above leaves out. This replaces it.
Two of your rulings cannot both hold for larger Go repositories on this host:
make testhas a 60-second hard cap, org wide; and sneak/homoicon#895 (2026-09-02): homoicon'smake checkfinishes in under 60 seconds on this host.With no cache, every
make testcompiles the module and all its dependencies from scratch before a test runs. That alone putsmake testat about three minutes in homoicon (sneak/homoicon#1505) and 158 s in bsfirehose. #123 saves about 16 s per build, which is not enough.Which gives way?
make checklimit is read the same way.Recommendation: 2. It is the only option that brings homoicon's
make checkback toward 60 seconds, which is what sneak/homoicon#895 asked for; #112 measured a test step at 8 s with the compiled packages kept against 127 s without. This changes the recommendation above.Model: opus-5-5
Correction to the
bsfirehosefigures above: since sneak/bsfirehose#96 replaced its cgo SQLite driver,script/cibuildthere takes 259 s, under the 5-minute limit, andmake test68 s, with its slowest test package at 3.6 s. Under option 1 it meets both limits.Model: opus-5-5
clawbot referenced this issue from sneak/routewatch2026-10-14 22:03:11 +02:00
Ruling (sneak, 2026-10-15, in chat, on sneak/secret#129): Go's compile cache may survive between check builds. So
REPO_POLICIES.mdchanges:--no-cachestill rebuilds every layer of a check build, but Go's build cache mount is kept (for example by busting the layer cache with a build argument instead of emptying the mount), and tests still never reuse results (-count=1).With the cache kept, building the test phase costs seconds, so the question this issue asked (whether the 60-second cap includes that build) no longer decides any repo's result; if a repo's
make teststill exceeds 60 seconds with the cache, it comes back to sneak here.Next: one PR on
nexthere with the policy change, then each repo re-vendors (homoicon PR 1505 and issue 1484, sfdupes PR 97, pixa issue 221, secret issue 129).Model: opus-5-5
Plan: keep Go's build cache in check builds, rerun every step, reuse no test result. One PR on
next, title ending(closes #113).Checked on this host:
docker build --no-cachegives aRUN --mount=type=cachestep an empty mount and replaces the stored one, so no cache mount can survive it. A build argument whose value changes on every run reruns everyRUNstep after itsARGline and keeps the mount.Scripts (
script/lint,script/test,script/cibuild,script/docker): replace--no-cachewith--build-arg CACHEBUST="$(date +%s)-$$"(CACHEBUSTis the usual name for this in Dockerfiles; the value is the time and the script's process id, so no two builds share it, including the three in onescript/cibuild). Rewrite each script's comment to say why in one or two lines.Dockerfiles: every stage declares
ARG CACHEBUSTas its first line afterFROM, so everyRUNstep in it runs on every check build.COPYsteps are compared by content and need nothing. This repo's ownDockerfile(three stages) and the canonical GoDockerfileinREPO_POLICIES.md. In the Go one, the steps that compile (golangci-lint run, bothgo testruns,go build) get--mount=type=cache,target=/root/.cache/go-build: Go's build cache only, never golangci-lint's own cache (#30) and never the module cache. Bothgo testruns in the test phase get-count=1.Text: rewrite the
--no-cacherule inREPO_POLICIES.md(both copies, kept byte-identical): check builds pass a newCACHEBUSTvalue each run; every stage declares it first; why not--no-cache(it empties the cache mounts); a stage missing theARGis served from cache, so a re-vendor takes the scripts and theARGlines together. Fix the gate snippets, the version snippet, the "image build is uncached" sentence, and the paragraph that says the test phase needs no-count=1. Same in both checklists,README.md,CODE_STYLEGUIDE_GO.md. Bumplast_modified; add aTODO.mdentry.Done when: two
script/testruns in a row on an unchanged tree each rerun everyRUNstep; on a scratch Go module built from the new canonicalDockerfile, the second test build reuses compiled packages, prints no(cached)test, and a planted failing test and a planted lint finding each failscript/cibuild;script/cibuildgreen on a fresh clone.Model: opus-5-5
Built in #129. The
script/builds pass a newCACHEBUSTvalue in place of--no-cache, everyDockerfilestage declaresARG CACHEBUSTfirst, and the canonical GoDockerfilemounts Go's build cache on the steps that compile, with-count=1on bothgo testruns. Repositories pick it up at their next re-vendor, taking the scripts and theARGlines in one commit.Model: opus-5-5
Merged to
nextase3f79c49da(from #129). To re-vendor, a repository takes the fourscript/builds, anARG CACHEBUSTline first in everyDockerfilestage and, for Go, the build cache mounts and-count=1from the canonicalDockerfile, all in one commit.Model: opus-5-5
Question from
pixa: itsmake testis still over 60 seconds with Go's build cache kept (sneak/pixa#276), about 95 seconds on a second run. Building and running the tests takes about 12 of them. The rest is the test phase installing Debian'slibvips-devand the 289 packages it pulls in, which reruns on every build because it comes after theARG CACHEBUSTline. Before the re-vendor,make testtook about as long, then mostly compiling.Options:
ARG CACHEBUSTline, so that step is reused like the base image and every step that builds or checks the code still reruns.pixa'smake testwould be about the 12 seconds of build and tests.make testover 60 seconds for repositories whose test phase installs system libraries.pixa's test phase on the Alpine Go image, where the same libraries install in about 20 seconds, against the policy's Debian test image.Recommendation: 1. The package step installs what the build needs and checks nothing, so reusing it skips no check, as reusing the base image skips none.
Model: opus-5-5
The question above is carried, with the other caches it belongs with, on the open #130 (comment) (assigned to sneak), so it is asked in one place.
Model: opus-5-5