golangci-lint cache is shared across worktrees, so lint results can come from another tree #88
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: sneak/vaultik#88
Reference in New Issue
Block a user
golangci-lintkeys its cache by a directory that vaultik does notisolate per worktree, and it takes a lock on that cache. With multiple
agents linting concurrently from separate throwaway worktrees, two things
can happen:
as this tree's results — reported elsewhere in the fleet as 399 issues
attributed to files under a
/tmpworktree that no longer existed, on atree that genuinely lints clean.
Error: parallel golangci-lint is running, a non-zeroexit that is not a lint result at all. An agent treating non-zero as
"lint failed" sends a correct PR back for rework.
This is the mirror image of #85 and #78. Those manufacture unearned
greens; this manufactures unearned greens and unearned reds — and
the red is arguably worse, because it fails a correct change against
findings from a different codebase, and the "evidence" looks concrete.
vaultik's actual exposure
Better than baseline, but not closed.
script/lint(as landed in #78)has two paths:
1. Docker path — partially isolated.
cache_root()is${XDG_CACHE_HOME:-~/.cache}/vaultik-lintand the container getsGOLANGCI_LINT_CACHE=/cache/golangci-lint. That isolates vaultik fromother repos, which is why we have not seen cross-repo contamination
here. But it is one cache directory shared by every vaultik worktree.
Concurrent agents in
/tmp/impl-84,/tmp/review-83, etc. all write thesame cache and contend for the same lock. Cross-worktree contamination
and the parallel-run error are both live.
2. Native escape hatch — not isolated at all.
script/lint:119-121:No cache environment is set, so this uses the default
~/.cache/golangci-lint— shared with every other session and repo onthe host. This path exists for the in-container run (correct and isolated
there), but it also triggers on any host with a matching
golangci-linton
PATH. That is precisely the fleet-wide exposure.Note this is the same defect shape as #80: the hatch is gated on version
equality rather than execution context. Cache isolation is part of that
context. #80 and this issue should probably be fixed together — a
context gate closes both.
Definition of done
GOLANGCI_LINT_CACHEis per worktree, not per repo, on the Dockerpath. Derive it from something that distinguishes worktrees (the
absolute worktree path hashed, or the git common-dir plus checkout
path) so two concurrent worktrees cannot collide.
the default — or is removed in favour of a context gate per #80, which
would make the question moot. State which was chosen.
Do not fix this by disabling the cache; report warm-run wall time
before and after to show the inner loop is not destroyed.
them where normal cleanup reaches them, or document the growth and how
to clear it.
invocations concurrently from two different worktrees of this repo and
confirm neither reports paths from the other and neither dies with
parallel golangci-lint is running. Paste the evidence in the PR.sneak/prompts#30, aper-worktree
GOLANGCI_LINT_CACHEin canonicalscript/lint) ratherthan inventing a local variant.
.golangci.ymlunchanged (sha256021cc83f4e6fc7c31b95b34b846723dfcf20b66b7baeea1dc40406e643346bcb);the
DockerfileFROMline stays the single source of truth for thelinter version (#78).
Interim rule — applies to every lint run in this repo now
Treat a lint run as VOID, not as a result, unless both hold:
parallel golangci-lint is running;and
from.
Retry until both hold. Never record a verdict — pass or fail — from a
void run. A red that cites paths from another tree is not a finding.
Reproduced in this repo, and the void rule then resolved it. Recording
both halves as concrete evidence.
The void run.
make lintonmainat3bcdbcf— a tree with fourmerged PRs that I had already verified clean — while an implementation
agent was linting concurrently from
/tmp/impl-85:Exit 2, no findings, no result line. An agent applying a naive "non-zero
means lint failed" rule would have reported a regression on a tree that
is clean. Note this run produced no findings, so it is the lock half of
the defect rather than the contamination half; the contamination half
(another worktree's cached findings served as your own) has been seen
elsewhere in the fleet but not yet here, most likely because the
vaultik-lintcache directory isolates this repo from other repos.The retry. Applying the interim rule — retry until the output has no
parallel-run error and cites no paths outside the launching worktree:
Second attempt satisfied both conditions.
maingenuinely lints clean.The rule works and is cheap — one retry — so there is no excuse for
recording a verdict from a void run.
Practical consequence for this repo right now: collisions are not
theoretical here, they are routine. This session runs implementation and
review agents concurrently, each in its own
/tmpworktree, and each runslint. The very first
make lintI attempted collided. Until this isfixed, every lint result in this repo — mine, an implementer's, or a
reviewer's — must be checked against the two conditions before it is
believed in either direction.
Bearing on already-merged work. PRs #77, #79, #83, and #86 were each
merged on a recorded
0 issues.A contaminated red would have shownfindings, so it cannot explain a
0 issues.result. A contaminatedgreen would require another vaultik worktree's clean cache being served
for a tree that was actually dirty — possible in principle. Against that:
each of those merges also passed
ok:14 cached:0on the test side, whichthe lint cache cannot fake, and #83 additionally had a reviewer who
reverted each fix and confirmed the corresponding test failed. The
re-verification above closes it out:
main, which is the merged productof all four, lints clean in a run that satisfies both conditions. No
re-review is warranted.
clawbot referenced this issue2026-08-09 08:17:35 +02:00
Scope refinement, from evidence gathered while working #85.
The containerised lint path is structurally immune to this defect,
not merely isolated by convention.
script/cibuildrunsmake lintinside the pinned image during the Docker build, so the linter's cache
is built from the pinned base within that build's own filesystem. It
never touches the host cache directory and never contends for the host
lock. Seven
script/cibuildruns during #85 were audited against bothvoid conditions and all were clean, while a concurrent host-side
make lintonmaincollided in the same window.So the exposure is narrower than this issue's original framing:
script/cibuild(containerised lint)make lint→ Docker path~/.cache/vaultik-lintshared by every vaultik worktreemake lint→ native escape hatch~/.cache/golangci-lintTwo consequences worth acting on:
per-worktree keying. The container path should be left alone — adding
a host cache mount to it would introduce the exposure it currently
does not have.
script/cibuildis not subject to this defect at all. Where averdict matters and host
make lintkeeps coming back void, runningthe containerised gate is a sound alternative rather than merely a
slower one. Note this only became true once #85's fix lands — before
it,
script/cibuildhad its own false-green mode, so the two defectswere covering for each other in opposite directions.
The interim VOID rule stands unchanged for every host-side lint run.
Experimental data on where the lock actually lives — this bears on the
proposed fix, and it contradicts one of the two accounts circulating.
The experiment
Two
make lintruns launched concurrently, from two differentworktrees of this repo, each with a completely separate cache
directory:
cwd=/srv/code/vaultik,XDG_CACHE_HOME=/tmp/cacheAcwd=/tmp/lockexp(detachedorigin/main),XDG_CACHE_HOME=/tmp/cacheBBoth take the Docker path (host has 2.10.1, pin is 2.12.2), so each
container mounts only its own source root and its own cache dir. The
two containers share no mounted path.
Result
A's tail:
Separate
GOLANGCI_LINT_CACHEdirectories did not prevent thecollision. A per-worktree cache key alone therefore does not fix the
lock-contention half of this issue — it addresses the contamination
half (one tree's cached findings served as another's) but not the
parallel golangci-lint is runninghalf.Caveat, stated because it matters
I cannot prove A collided with B specifically. A third host-side lint
was plausibly running concurrently (a reviewer working PR #89 also runs
GOFLAGS=-count=1 make check, which invokes hostmake lintunder thedefault
XDG_CACHE_HOME). So the honest claim is the weaker one, which isstill sufficient:
If the lock were scoped to the cache directory, a private cache would
have made A immune to every other run regardless of which one it hit.
It did not. That is the load-bearing conclusion.
Consequence for the fix
Definition-of-done item 1 as written — per-worktree
GOLANGCI_LINT_CACHE— is necessary but not sufficient. It should be extended to either:
collisions queue instead of erroring), or
script/lintretry on the parallel-run error rather thansurfacing it as a lint failure — since that error is not a result, and
an exit code of 2 from it is indistinguishable, to a caller, from real
findings.
Option 2 is the smaller change and directly removes the false-red, which
is the damaging direction. It also encodes the interim VOID rule in the
tooling instead of relying on every agent brief to restate it — the
current defence is that every reviewer and implementer remembers to check
two conditions by hand, which will not hold.
Reported upstream to
sneak/prompts#30, where the two accountsdisagreed.
Obsolete: the code this describes no longer exists. On
next(d257f8f, issue #113) every lint run is adocker buildofDockerfile.lint, so each run has its own lint cache and its own lock inside a container that is discarded with it — cross-worktree contamination andparallel golangci-lint is runningare both structurally impossible rather than mitigated. The sharedGOLANGCI_LINT_CACHE, the per-worktree cache machinery and the native escape hatch (point 2) are all deleted;git grepforGOLANGCI_LINT_CACHE,cache_home,path_digest,prune_dead_cachesandinstalled_versiononnextreturns nothing.The interim VOID-run rule in this issue is retired with it. Closing.