PROPAGATION: roll the Docker build-context trio (#26 CHECK_EPOCH, #29 .dockerignore secrets, #27 .claude/) into every consuming repo — as ONE commit per repo #35

Open
opened 2026-08-09 16:49:30 +02:00 by clawbot · 4 comments
Collaborator

Follow-up required by the definitions of done in #26, #29 and #27. Filed as a single tracking issue for all three deliberately — see the ordering rule below, which is the whole reason this is one issue and not three.

Do not start a repo until the canonical change it depends on has landed here. Status is tracked at the top; the checklist is at the bottom.

Canonical status

  • #26 CHECK_EPOCH cache-bust — landed on next (PR #34, commit 22a5a37), under review
  • #29 .dockerignore secret patterns with **/ prefixes
  • #27 .claude/ exclusion + --build-arg VERSION guidance

THE ORDERING RULE — read before doing any repo

All three changes must land in a single commit per repo. Never .dockerignore-first.

This is not tidiness. Any path in the build context that churns between builds accidentally protects a repo, because it invalidates COPY . . and forces the check layers to re-run. .git churns on nearly every git operation; .claude/worktrees/ churns wherever agents run. Tightening .dockerignore removes that protection. So adding the canonical .dockerignore without the CHECK_EPOCH fix is a strict regression: it converts a repo whose gate works by accident into one that reliably reports greens it did not earn.

The fleet-level version of the same trap: if .dockerignore and CHECK_EPOCH are run as two separate passes across the roster, the window between the passes is exactly when CI stops meaning anything in every repo the first pass has reached and the second has not. Hence: one commit, per repo, all three.

Absence of a .dockerignore is NOT a screening test for whether a repo is affected. A repo whose .dockerignore has always excluded .git never had the protection and is exposed right now — cattbox is the worked example. Every repo needs the cache-bust regardless of its .dockerignore state. A manager who checks for the file, finds one, and concludes the repo is fine has it exactly backwards.

Per-repo definition of done

  1. Apply all three canonical changes in one commit. Copy the canonical files rather than re-deriving them; consuming repos must stay byte-identical to canonical.
  2. ARG CHECK_EPOCH + the [ -n "$CHECK_EPOCH" ] || exit 1 guard + the expanded RUN echo "check epoch: ${CHECK_EPOCH}" && make check form in every stage containing a check-running RUN. ARG is stage-scoped; most repos have check steps in two stages (make fmt-check/make lint in lint, make test in builder). A fix written against a single-stage repo silently leaves the other stage frozen and reviews as complete.
  3. Both script/cibuild and script/docker updated. A warm local make docker is the likelier deception today, and once only cibuild is fixed the two entrypoints silently disagree about whether the tree is green.
  4. **/-prefix every depth-independent .dockerignore pattern. Do NOT apply **/ to .gitignore — different semantics; prefixing there produces a file that is wrong in a way that looks careful.
  5. If the repo embeds a version via git describe, thread it in with --build-arg VERSION=... computed on the host. .dockerignore excludes .git, so git describe in a build stage yields an empty version without erroring.
  6. Verify by negative control, never by inspection:
    • two consecutive script/cibuild runs on an unchanged tree, both executing the checks;
    • the script/bootstrap (or go mod download) layer still CACHED in run 2 — this is the validity control proving no concurrent prune landed mid-pair, not merely a performance check;
    • bare docker build . fails on the guard;
    • a planted defect fails the build with its predicted error.
  7. Confirm the build context shrank; enumerate what reaches the image rather than reading the ignore file. Plant secrets at least two directories deep — a root-only probe passes a broken .dockerignore.
  8. Check whether any existing image layer already contains a .env, *.pem or *.key. The exposure is not necessarily only prospective.

Method warnings

  • A cache experiment can itself be served from cache. Vary probes with a per-run nonce or use the A/B/C/D liveness design, or you will measure your own history and conclude the mechanism does not work. This nearly produced a false result twice already.
  • Wall-clock is triage; CACHED-vs-executed on the specific layer is what settles a question.
  • Never run docker builder prune or docker system prune — this host is shared with ~18 sessions and a prune destroyed ~41 GB on 2026-08-09. Use --no-cache-filter=<stage> for scoped invalidation. The two-run protocol warms its own cache, so no prune is needed for a valid measurement.
  • Do not present reasoning about BuildKit as a result. A correct fix carrying an incorrect explanation is worse than a wrong fix: it passes review on its results and then misleads whoever maintains it next. One PR elsewhere was failed on exactly this, with correct code.

Repos to sweep

Each manager: confirm applicability first. A repo with no Dockerfile ticks with a note; do not invent one.

  • attrsum
  • AutistMask
  • bsfirehose
  • cattbox
  • clawhook
  • clawpub
  • dnswatcher
  • gohttpserver
  • homoicon
  • jekyllsitebuilder
  • lora.vegas
  • mfer
  • neoirc
  • netwatch
  • pixa
  • quak
  • rfscan
  • rgoue
  • routewatch
  • secret
  • sfdupes
  • simplelog
  • smartconfig
  • sneak.berlin
  • upaas
  • vaultik
  • webhooker

Repos that already landed a variant of the CHECK_EPOCH fix ahead of canonical (rfscan, dnswatcher, cattbox, vaultik at least) are not broken and need no urgent rework, but should re-run the negative control against the canonical form on their own machine rather than assuming their earlier verification transfers — several used the bare unreferenced-ARG form, which works but leaves the miss dependent on BuildKit behaviour rather than contractual.

#25 changes .golangci.yml, so its sha256 moves and every repo verifying by hash will mismatch. #32 (-count=1) and #28 (bootstrap version enforcement) are Go-only. Those are separate sweeps with different scopes and must not be folded into this commit.

Follow-up required by the definitions of done in #26, #29 and #27. Filed as a single tracking issue for all three deliberately — see the ordering rule below, which is the whole reason this is one issue and not three. Do not start a repo until the canonical change it depends on has landed here. Status is tracked at the top; the checklist is at the bottom. ## Canonical status - [x] #26 `CHECK_EPOCH` cache-bust — landed on `next` (PR #34, commit `22a5a37`), under review - [ ] #29 `.dockerignore` secret patterns with `**/` prefixes - [ ] #27 `.claude/` exclusion + `--build-arg VERSION` guidance ## THE ORDERING RULE — read before doing any repo **All three changes must land in a single commit per repo. Never `.dockerignore`-first.** This is not tidiness. Any path in the build context that churns between builds accidentally protects a repo, because it invalidates `COPY . .` and forces the check layers to re-run. `.git` churns on nearly every git operation; `.claude/worktrees/` churns wherever agents run. Tightening `.dockerignore` removes that protection. So adding the canonical `.dockerignore` **without** the `CHECK_EPOCH` fix is a strict regression: it converts a repo whose gate works by accident into one that reliably reports greens it did not earn. The fleet-level version of the same trap: if `.dockerignore` and `CHECK_EPOCH` are run as two separate passes across the roster, **the window between the passes is exactly when CI stops meaning anything** in every repo the first pass has reached and the second has not. Hence: one commit, per repo, all three. **Absence of a `.dockerignore` is NOT a screening test for whether a repo is affected.** A repo whose `.dockerignore` has always excluded `.git` never had the protection and is exposed right now — cattbox is the worked example. Every repo needs the cache-bust regardless of its `.dockerignore` state. A manager who checks for the file, finds one, and concludes the repo is fine has it exactly backwards. ## Per-repo definition of done 1. Apply all three canonical changes in one commit. Copy the canonical files rather than re-deriving them; consuming repos must stay byte-identical to canonical. 2. `ARG CHECK_EPOCH` + the `[ -n "$CHECK_EPOCH" ] || exit 1` guard + the expanded `RUN echo "check epoch: ${CHECK_EPOCH}" && make check` form in **every stage containing a check-running `RUN`**. `ARG` is stage-scoped; most repos have check steps in two stages (`make fmt-check`/`make lint` in lint, `make test` in builder). A fix written against a single-stage repo silently leaves the other stage frozen and reviews as complete. 3. Both `script/cibuild` **and** `script/docker` updated. A warm local `make docker` is the likelier deception today, and once only cibuild is fixed the two entrypoints silently disagree about whether the tree is green. 4. `**/`-prefix every depth-independent `.dockerignore` pattern. Do **NOT** apply `**/` to `.gitignore` — different semantics; prefixing there produces a file that is wrong in a way that looks careful. 5. If the repo embeds a version via `git describe`, thread it in with `--build-arg VERSION=...` computed on the host. `.dockerignore` excludes `.git`, so `git describe` in a build stage yields an empty version **without erroring**. 6. Verify by **negative control, never by inspection**: - two consecutive `script/cibuild` runs on an unchanged tree, both executing the checks; - the `script/bootstrap` (or `go mod download`) layer still `CACHED` in run 2 — this is the validity control proving no concurrent prune landed mid-pair, not merely a performance check; - bare `docker build .` fails on the guard; - a planted defect fails the build with its predicted error. 7. Confirm the build context shrank; enumerate what reaches the image rather than reading the ignore file. Plant secrets **at least two directories deep** — a root-only probe passes a broken `.dockerignore`. 8. Check whether any **existing** image layer already contains a `.env`, `*.pem` or `*.key`. The exposure is not necessarily only prospective. ## Method warnings - **A cache experiment can itself be served from cache.** Vary probes with a per-run nonce or use the A/B/C/D liveness design, or you will measure your own history and conclude the mechanism does not work. This nearly produced a false result twice already. - Wall-clock is triage; `CACHED`-vs-executed on the specific layer is what settles a question. - **Never run `docker builder prune` or `docker system prune`** — this host is shared with ~18 sessions and a prune destroyed ~41 GB on 2026-08-09. Use `--no-cache-filter=<stage>` for scoped invalidation. The two-run protocol warms its own cache, so no prune is needed for a valid measurement. - Do not present reasoning about BuildKit as a result. A correct fix carrying an incorrect explanation is worse than a wrong fix: it passes review on its results and then misleads whoever maintains it next. One PR elsewhere was failed on exactly this, with correct code. ## Repos to sweep Each manager: confirm applicability first. A repo with no `Dockerfile` ticks with a note; do not invent one. - [ ] attrsum - [ ] AutistMask - [ ] bsfirehose - [ ] cattbox - [ ] clawhook - [ ] clawpub - [ ] dnswatcher - [ ] gohttpserver - [ ] homoicon - [ ] jekyllsitebuilder - [ ] lora.vegas - [ ] mfer - [ ] neoirc - [ ] netwatch - [ ] pixa - [ ] quak - [ ] rfscan - [ ] rgoue - [ ] routewatch - [ ] secret - [ ] sfdupes - [ ] simplelog - [ ] smartconfig - [ ] sneak.berlin - [ ] upaas - [ ] vaultik - [ ] webhooker Repos that already landed a variant of the `CHECK_EPOCH` fix ahead of canonical (rfscan, dnswatcher, cattbox, vaultik at least) are not broken and need no urgent rework, but should re-run the negative control against the canonical form on their own machine rather than assuming their earlier verification transfers — several used the bare unreferenced-`ARG` form, which works but leaves the miss dependent on BuildKit behaviour rather than contractual. ## Related propagation, tracked separately #25 changes `.golangci.yml`, so its sha256 moves and every repo verifying by hash will mismatch. #32 (`-count=1`) and #28 (bootstrap version enforcement) are Go-only. Those are separate sweeps with different scopes and must not be folded into this commit.
Author
Collaborator

Scope addition, from the PR #34 review: the sweep is not three files per repo. It is a grep.

The canonical fix passed code review and failed on scope — five sites across three org-canonical documents still asserted the old guarantee, beyond the Dockerfile / script/cibuild / script/docker / REPO_POLICIES.md set. Consuming repos vendor those documents too, so the same sites exist downstream.

Two are actively harmful rather than merely stale:

  • EXISTING_REPO_CHECKLIST.md carried a final acceptance item "docker build succeeds". The [ -n "$CHECK_EPOCH" ] guard makes that unsatisfiable by design. An agent working the fixup checklist hits a guard failure at the last box, and the cheapest way to tick it is to delete the guard. The fix disarming itself through its own documentation. Any consuming repo that vendored this checklist has a live instruction to undo the change it is about to receive.
  • NEW_REPO_CHECKLIST.md instructed writing the pre-fix script/cibuild verbatim, so a new repo created from it is born with the false green even after the sweep completes.

Also CODE_STYLEGUIDE_GO.md, which restated "docker build . ... makes sure the code is compiled, linted and tests run" — both halves now wrong.

Amended per-repo done-criterion, replacing "update these files":

> A repo-wide grep for docker build returns only sites that describe it as failing closed by design, or that route through script/cibuild / script/docker.

Check vendored checklists, styleguides, README.md, CONTRIBUTING, runbooks, and any .gitea/workflows/ step that invokes docker build directly. A file list will miss sites; the grep will not. The canonical brief named two sites, the implementer found three, the reviewer found five — assume your repo's list is longer than you expect.

**Scope addition, from the PR #34 review: the sweep is not three files per repo. It is a grep.** The canonical fix passed code review and failed on scope — **five** sites across three org-canonical documents still asserted the old guarantee, beyond the `Dockerfile` / `script/cibuild` / `script/docker` / `REPO_POLICIES.md` set. Consuming repos vendor those documents too, so the same sites exist downstream. Two are actively harmful rather than merely stale: - **`EXISTING_REPO_CHECKLIST.md`** carried a final acceptance item "`docker build` succeeds". The `[ -n "$CHECK_EPOCH" ]` guard makes that **unsatisfiable by design**. An agent working the fixup checklist hits a guard failure at the last box, and the cheapest way to tick it is to delete the guard. **The fix disarming itself through its own documentation.** Any consuming repo that vendored this checklist has a live instruction to undo the change it is about to receive. - **`NEW_REPO_CHECKLIST.md`** instructed writing the pre-fix `script/cibuild` verbatim, so a new repo created from it is born with the false green even after the sweep completes. Also `CODE_STYLEGUIDE_GO.md`, which restated "`docker build .` ... makes sure the code is compiled, linted and tests run" — both halves now wrong. **Amended per-repo done-criterion, replacing "update these files":** > A repo-wide grep for `docker build` returns only sites that describe it as failing closed by design, or that route through `script/cibuild` / `script/docker`. Check vendored checklists, styleguides, `README.md`, `CONTRIBUTING`, runbooks, and any `.gitea/workflows/` step that invokes `docker build` directly. A file list will miss sites; the grep will not. The canonical brief named two sites, the implementer found three, the reviewer found five — assume your repo's list is longer than you expect.
Author
Collaborator

The .dockerignore verification method in circulation cannot detect the defect. Anyone about to run this sweep must not use it.

From the #29 implementation. transferring context is a BuildKit delta, not a total — it reports what this build had to send given what the daemon already holds, not what is in the context.

Measured: the naive .dockerignore form (patterns without **/ prefixes) reported 2.18kB transferred while 43 files, five of them secrets, were present in the image.

That matters beyond a method note, because the definition of done in #29 as originally written required exactly that measurement — "verified absent from the build context by measuring transferring context size before and after, not by reading the file". A sweep following it would have compared two small numbers, seen them differ, and recorded the broken form as fixed. The instruction intended to prevent verification-by-reading instead prescribed verification-by-artefact.

Use image enumeration instead, per repo:

  1. Plant secrets at least two directories deep, plus root-level copies, so the test distinguishes "root works" from "all depths work". Root-only probes pass a broken file — that is how the naive form got through elsewhere.
  2. Build and find inside the image. That is the measurement.
  3. Include a positive control — an ordinary source file that must still arrive. A .dockerignore excluding everything otherwise passes every negative test.
  4. Run the naive form as a control and confirm nested secrets still reach the image. Without it you cannot show your test detects the failure mode at all.
  5. Plant on the filesystem and verify removal on the filesystem. git status never sees these files — .gitignore covers them, which is the exact property that made the exposure invisible.

Also confirmed by enumeration rather than assumed: **/foo does match at the context root, so **/.env covers a root .env and no separate unprefixed entry is needed.

Two standing warnings, restated because this sweep is where they bite: do not apply **/ to .gitignore — different semantics, and the result is wrong in a way that looks careful — and check the consuming repo's checklists, which instruct agents to "extend" these files and are where the naive shape actually gets written.

**The `.dockerignore` verification method in circulation cannot detect the defect. Anyone about to run this sweep must not use it.** From the #29 implementation. `transferring context` is a **BuildKit delta, not a total** — it reports what this build had to send given what the daemon already holds, not what is in the context. Measured: the naive `.dockerignore` form (patterns without `**/` prefixes) reported **2.18kB transferred while 43 files, five of them secrets, were present in the image**. That matters beyond a method note, because **the definition of done in #29 as originally written required exactly that measurement** — "verified absent from the build context by measuring `transferring context` size before and after, not by reading the file". A sweep following it would have compared two small numbers, seen them differ, and recorded the broken form as fixed. The instruction intended to prevent verification-by-reading instead prescribed verification-by-artefact. **Use image enumeration instead**, per repo: 1. Plant secrets **at least two directories deep**, plus root-level copies, so the test distinguishes "root works" from "all depths work". Root-only probes pass a broken file — that is how the naive form got through elsewhere. 2. Build and **`find` inside the image**. That is the measurement. 3. Include a **positive control** — an ordinary source file that must still arrive. A `.dockerignore` excluding everything otherwise passes every negative test. 4. **Run the naive form as a control** and confirm nested secrets still reach the image. Without it you cannot show your test detects the failure mode at all. 5. Plant on the filesystem and verify removal on the filesystem. `git status` never sees these files — `.gitignore` covers them, which is the exact property that made the exposure invisible. Also confirmed by enumeration rather than assumed: **`**/foo` does match at the context root**, so `**/.env` covers a root `.env` and no separate unprefixed entry is needed. Two standing warnings, restated because this sweep is where they bite: **do not apply `**/` to `.gitignore`** — different semantics, and the result is wrong in a way that looks careful — and **check the consuming repo's checklists**, which instruct agents to "extend" these files and are where the naive shape actually gets written.
Author
Collaborator

The git describe consequence is already written INTO the canonical corpus, so the sweep has to fix Makefiles too, not just ignore files.

From the #27 implementation. prompts/CODE_STYLEGUIDE_GO.md and prompts/GO_HTTP_SERVER_CONVENTIONS.md both carry:

VERSION := $(shell git describe --tags --always --dirty)

and the canonical Dockerfile runs make inside the build. .dockerignore excludes .git, so that $(shell ...) produces an empty string, with no error — the binary simply reports no version. The defect this issue warns about was therefore not merely undocumented, it was prescribed, and any repo that copied the styleguide Makefile has it today.

Fixed canonically by changing := to ?= so an ARG VERSION passed into the build wins via the environment, plus documenting that the value is computed on the host and threaded in with --build-arg VERSION=....

Add to the per-repo checklist in this sweep:

  • grep -rn 'git describe' . in the consuming repo. Any occurrence inside a Makefile, Dockerfile, or build script that runs during a Docker build is broken the moment .git is excluded — and it fails silently, which is why nobody has noticed.
  • Convert VERSION := to VERSION ?= in the Makefile so a build arg can win.
  • Compute the version on the host and pass --build-arg VERSION=... from script/cibuild and script/docker. Assign it on its own line, never inline in the argument: a failing command substitution inside an argument does not trip set -e (confirmed in dash), so the inline form degrades silently to an empty constant — the same trap CHECK_EPOCH has.
  • CI caveat: the standard checkout action clones shallow with no tags, so git describe --tags degrades to a bare hash there. A repo embedding a tag-derived version needs full history and tags fetched; a repo embedding no version needs nothing.

Sequencing note: this belongs in the same single per-repo commit as the .dockerignore tightening and the CHECK_EPOCH bust. A repo that tightens .dockerignore without it starts emitting unversioned binaries, and it will not announce itself.

**The `git describe` consequence is already written INTO the canonical corpus, so the sweep has to fix Makefiles too, not just ignore files.** From the #27 implementation. `prompts/CODE_STYLEGUIDE_GO.md` and `prompts/GO_HTTP_SERVER_CONVENTIONS.md` both carry: ```makefile VERSION := $(shell git describe --tags --always --dirty) ``` and the canonical Dockerfile runs `make` inside the build. `.dockerignore` excludes `.git`, so that `$(shell ...)` produces an **empty string, with no error** — the binary simply reports no version. The defect this issue warns about was therefore not merely undocumented, it was **prescribed**, and any repo that copied the styleguide Makefile has it today. Fixed canonically by changing `:=` to `?=` so an `ARG VERSION` passed into the build wins via the environment, plus documenting that the value is computed on the **host** and threaded in with `--build-arg VERSION=...`. **Add to the per-repo checklist in this sweep:** - `grep -rn 'git describe' .` in the consuming repo. Any occurrence inside a Makefile, Dockerfile, or build script that runs during a Docker build is broken the moment `.git` is excluded — and it fails **silently**, which is why nobody has noticed. - Convert `VERSION :=` to `VERSION ?=` in the Makefile so a build arg can win. - Compute the version on the host and pass `--build-arg VERSION=...` from `script/cibuild` and `script/docker`. Assign it **on its own line**, never inline in the argument: a failing command substitution inside an argument does not trip `set -e` (confirmed in `dash`), so the inline form degrades silently to an empty constant — the same trap `CHECK_EPOCH` has. - **CI caveat**: the standard checkout action clones shallow with no tags, so `git describe --tags` degrades to a bare hash there. A repo embedding a tag-derived version needs full history and tags fetched; a repo embedding no version needs nothing. **Sequencing note:** this belongs in the same single per-repo commit as the `.dockerignore` tightening and the `CHECK_EPOCH` bust. A repo that tightens `.dockerignore` without it starts emitting unversioned binaries, and it will not announce itself.
Author
Collaborator

The single question that would have caught every defect in this batch, for whoever runs this sweep: "what does this do when the guard itself fails?"

Five canonical fixes landed. Across their review cycles, every defect found — in the original code and in the remedies — had the same shape: a mechanism that failed into a plausible result rather than an obvious error. Listing them because the pattern is more useful than any individual fix:

  • script/cibuild reported a green without running the suite.
  • $(date +%s) inline in an argument does not trip set -e, so it degrades to an empty constant — a stable cache key, and the false green returns.
  • An unset ARG is empty, and empty is a stable cache key: a bare docker build . stayed broken while the scripted path was fixed.
  • script/bootstrap printed "installed" while a shadowing binary stayed on PATH, changing nothing any caller sees.
  • A version parser truncating at the first - made 2.12.2-rc1 compare equal to a 2.12.2 pin and skip the install.
  • A canonical block defined two functions and never called one: exit 0, zero output, adopted verbatim.
  • A .dockerignore with the right names but no **/ prefixes protects only the context root, and reads as solved.
  • transferring context is a delta, not a total — 2.18kB reported while 43 files including five secrets were in the image.
  • A lint wrapper writing to fixed paths printed another concurrent run's 0 issues. while its own linter had a real finding.
  • A signal trap that deletes and resumes returned the findings exit status on a run that analysed nothing.

Three of those were the prescribed check rather than the code, including two definitions of done in these very issues: #29's "measure transferring context before and after" cannot detect the broken form, and #30's negative control as written passes against the unisolated form because differing content never collides in a content-keyed cache.

What to actually do differently

  1. Pair every control with the broken form in the same environment. Not "the fix works" — "the fix works AND the pre-fix form fails here, now". Every real defect above was caught by that pairing, and several partial fixes passed until someone built the half-fix and measured it failing too.
  2. State the expected failure before running the control. One control mis-generated itself into a syntax error and would have read as confirmation; it was caught only because the predicted failure had been written down first.
  3. Exercise the artifact as adopted, not the function. Paste the block into a file shaped like the real script and run that file. A block that defines functions and never calls one passes every control when the harness calls the function by hand.
  4. Ask what the guard does when it fails, and require the answer to be "loudly". Every remedy above that broke, broke into a plausible-looking success.
  5. A correct fix carrying a wrong explanation is worse than a wrong fix — it passes review on its results and misleads whoever maintains it next. Several were caught here, including two where the conclusion was right and the stated mechanism was not.

Vendored prose is part of the artifact. Consuming repos receive files, not pull requests: a caveat recorded only in a PR body is not a caveat, and a checklist item an agent can satisfy by deleting a guard is a live instruction to disarm the fix.

**The single question that would have caught every defect in this batch, for whoever runs this sweep: "what does this do when the guard itself fails?"** Five canonical fixes landed. Across their review cycles, **every** defect found — in the original code and in the remedies — had the same shape: a mechanism that failed into a plausible result rather than an obvious error. Listing them because the pattern is more useful than any individual fix: - `script/cibuild` reported a green without running the suite. - `$(date +%s)` inline in an argument does not trip `set -e`, so it degrades to an empty constant — a stable cache key, and the false green returns. - An unset `ARG` is empty, and empty is a stable cache key: a bare `docker build .` stayed broken while the scripted path was fixed. - `script/bootstrap` printed "installed" while a shadowing binary stayed on `PATH`, changing nothing any caller sees. - A version parser truncating at the first `-` made `2.12.2-rc1` compare **equal** to a `2.12.2` pin and skip the install. - A canonical block defined two functions and never called one: exit 0, zero output, adopted verbatim. - A `.dockerignore` with the right names but no `**/` prefixes protects only the context root, and **reads as solved**. - `transferring context` is a delta, not a total — 2.18kB reported while 43 files including five secrets were in the image. - A lint wrapper writing to fixed paths printed another concurrent run's `0 issues.` while its own linter had a real finding. - A signal trap that deletes and resumes returned the *findings* exit status on a run that analysed nothing. **Three of those were the prescribed check rather than the code**, including two definitions of done in these very issues: #29's "measure `transferring context` before and after" cannot detect the broken form, and #30's negative control as written passes against the unisolated form because differing content never collides in a content-keyed cache. ## What to actually do differently 1. **Pair every control with the broken form in the same environment.** Not "the fix works" — "the fix works AND the pre-fix form fails here, now". Every real defect above was caught by that pairing, and several partial fixes passed until someone built the half-fix and measured it failing too. 2. **State the expected failure before running the control.** One control mis-generated itself into a syntax error and would have read as confirmation; it was caught only because the predicted failure had been written down first. 3. **Exercise the artifact as adopted**, not the function. Paste the block into a file shaped like the real script and run that file. A block that defines functions and never calls one passes every control when the harness calls the function by hand. 4. **Ask what the guard does when it fails**, and require the answer to be "loudly". Every remedy above that broke, broke into a plausible-looking success. 5. **A correct fix carrying a wrong explanation is worse than a wrong fix** — it passes review on its results and misleads whoever maintains it next. Several were caught here, including two where the conclusion was right and the stated mechanism was not. Vendored prose is part of the artifact. Consuming repos receive files, not pull requests: a caveat recorded only in a PR body is not a caveat, and a checklist item an agent can satisfy by deleting a guard is a live instruction to disarm the fix.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/prompts#35