make test goes over 60 seconds when the kept Go build cache has been removed #109

Open
opened 2026-10-15 18:35:06 +02:00 by clawbot · 2 comments
Collaborator

REPO_POLICIES.md caps make test at 60 seconds, sets 20 seconds as the target, and says time between 20 and 60 seconds is filed against the repository. Since #97 the test phase keeps Go's build cache between builds, as sneak ruled on sneak/prompts#113.

Timed on this host as script/cibuild runs it, on fresh clones, make test took between 19 and 77 seconds in five runs. None met the 20-second target, and two went over 60 seconds, the last one on next after the merge.

Where the time goes:

  • Compiling. With the kept build cache it takes about 2 seconds; without it, 30 to 36 seconds, mostly modernc.org/sqlite and modernc.org/libc under the race detector. Both runs over 60 seconds compiled everything from scratch. Every cache mount on this host, in every repository, had been created in the five minutes before it was looked at, so something removes them between builds. Docker's own cleanup limit for them is about 106 GiB, far above what they use, so Docker's automatic cleanup is not what removes them. This is not specific to sfdupes.
  • go mod download runs on every build and fetches the modules over the network: usually 7 to 30 seconds, once 64. The policy does not allow keeping the module cache.
  • The tests themselves take 13 to 18 seconds.

Definition of done

  1. make test, run as script/cibuild runs it on this host, finishes in under 60 seconds every time.
  2. Any time over 20 seconds that the policy makes unavoidable (such as the module download) is stated on this issue, with what it would take to remove it.
  3. Nothing the policy rules out is built (keeping the module cache, dropping -race or -count=1); if under 60 seconds needs one of those, it goes to sneak as one question with a recommendation.

Model: opus-5-5

`REPO_POLICIES.md` caps `make test` at 60 seconds, sets 20 seconds as the target, and says time between 20 and 60 seconds is filed against the repository. Since https://git.eeqj.de/sneak/sfdupes/pulls/97 the test phase keeps Go's build cache between builds, as sneak ruled on https://git.eeqj.de/sneak/prompts/issues/113. Timed on this host as `script/cibuild` runs it, on fresh clones, `make test` took between 19 and 77 seconds in five runs. None met the 20-second target, and two went over 60 seconds, the last one on `next` after the merge. Where the time goes: - Compiling. With the kept build cache it takes about 2 seconds; without it, 30 to 36 seconds, mostly `modernc.org/sqlite` and `modernc.org/libc` under the race detector. Both runs over 60 seconds compiled everything from scratch. Every cache mount on this host, in every repository, had been created in the five minutes before it was looked at, so something removes them between builds. Docker's own cleanup limit for them is about 106 GiB, far above what they use, so Docker's automatic cleanup is not what removes them. This is not specific to sfdupes. - `go mod download` runs on every build and fetches the modules over the network: usually 7 to 30 seconds, once 64. The policy does not allow keeping the module cache. - The tests themselves take 13 to 18 seconds. ## Definition of done 1. `make test`, run as `script/cibuild` runs it on this host, finishes in under 60 seconds every time. 2. Any time over 20 seconds that the policy makes unavoidable (such as the module download) is stated on this issue, with what it would take to remove it. 3. Nothing the policy rules out is built (keeping the module cache, dropping `-race` or `-count=1`); if under 60 seconds needs one of those, it goes to sneak as one question with a recommendation. Model: opus-5-5
clawbot self-assigned this 2026-10-15 18:35:06 +02:00
Author
Collaborator

Cause found, on the host: both build-cache purges ran an unfiltered docker builder prune -a -f — the system cron every 3 hours (/etc/cron.d/docker-builder-prune, 15:23 and 18:23 UTC slots) and the top manager's 12-hourly reclaim (16:3x UTC today) — and that removes the BuildKit cache mounts holding Go's compile cache along with the layer cache. Both now run docker builder prune -a -f --filter 'type!=exec.cachemount', which keeps the cache mounts and still purges the layer cache (project-management crons/cron.d/docker-builder-prune and crons/docker-reclaim.txt, installed at 16:45 UTC). Closing; reopen if make test stays over 60 seconds with the cache present.

Model: opus-5-5

Cause found, on the host: both build-cache purges ran an unfiltered `docker builder prune -a -f` — the system cron every 3 hours (`/etc/cron.d/docker-builder-prune`, 15:23 and 18:23 UTC slots) and the top manager's 12-hourly reclaim (16:3x UTC today) — and that removes the BuildKit cache mounts holding Go's compile cache along with the layer cache. Both now run `docker builder prune -a -f --filter 'type!=exec.cachemount'`, which keeps the cache mounts and still purges the layer cache (project-management `crons/cron.d/docker-builder-prune` and `crons/docker-reclaim.txt`, installed at 16:45 UTC). Closing; reopen if `make test` stays over 60 seconds with the cache present. Model: opus-5-5
clawbot reopened this issue 2026-10-15 18:36:27 +02:00
Author
Collaborator

Reopened: closing was early. The definition of done above wants make test measured under 60 seconds every time with the cache now kept, and the remaining time over 20 seconds stated here; that measurement is still to be done.

Model: opus-5-5

Reopened: closing was early. The definition of done above wants `make test` measured under 60 seconds every time with the cache now kept, and the remaining time over 20 seconds stated here; that measurement is still to be done. Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/sfdupes#109