List the steps above ARG CACHEBUST and split the long Python install sentences (closes #133)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
The ARG CACHEBUST rule and the CACHED expectation now list their steps as bullets, the same way in the policy, the README and both checklists. The checklists' Python item and the policy's Python point use shorter sentences with the verb early. The TODO entry no longer counts "no Python" as an install or says "except" where only a difference is meant. Model: opus-5-5
This commit is contained in:
@@ -144,17 +144,22 @@ alpine. We provide:
|
||||
|
||||
`make hooks` installs the pre-commit hook that runs `script/precommit`.
|
||||
|
||||
In every stage of a `Dockerfile`, `ARG CACHEBUST` comes right after the steps
|
||||
that install system packages, download Go's modules or run `script/bootstrap` to
|
||||
install a Python `test` phase's packages from `Pipfile.lock`, or right after
|
||||
`FROM` in a stage with none of these, and every step that lints, tests or
|
||||
compiles comes after it. The `script/` builds pass a new `CACHEBUST` value on
|
||||
every build, which reruns every `RUN` step after that line and leaves the layers
|
||||
above it and the cache mounts in place; `--no-cache` would build those layers
|
||||
again and empty the mounts. This repo's own `Dockerfile` declares
|
||||
`ARG CACHEBUST` right after `FROM` in every stage: each stage installs system
|
||||
packages and node packages together in one `script/bootstrap` step, and check
|
||||
builds may keep system packages but not node packages.
|
||||
In every stage of a `Dockerfile`, every step that lints, tests or compiles comes
|
||||
after `ARG CACHEBUST`. `ARG CACHEBUST` comes right after these steps:
|
||||
|
||||
- the steps that install system packages
|
||||
- the steps that download Go's modules
|
||||
- the step that runs `script/bootstrap` to install a Python `test` phase's
|
||||
packages from `Pipfile.lock`
|
||||
|
||||
A stage with none of these steps declares `ARG CACHEBUST` right after `FROM`.
|
||||
The `script/` builds pass a new `CACHEBUST` value on every build, which reruns
|
||||
every `RUN` step after that line and leaves the layers above it and the cache
|
||||
mounts in place; `--no-cache` would build those layers again and empty the
|
||||
mounts. This repo's own `Dockerfile` declares `ARG CACHEBUST` right after `FROM`
|
||||
in every stage: each stage installs system packages and node packages together
|
||||
in one `script/bootstrap` step, and check builds may keep system packages but
|
||||
not node packages.
|
||||
|
||||
## Rationale
|
||||
|
||||
|
||||
@@ -27,17 +27,16 @@ fmt-check, and commit.
|
||||
comes before `ARG CACHEBUST`: `COPY script/ script/`,
|
||||
`COPY Pipfile Pipfile.lock ./`, then `RUN script/bootstrap`. In that step
|
||||
`script/bootstrap` installs nothing beyond system packages, pipenv and its
|
||||
dependencies by hash, and the packages from the lockfile, and no Python, since
|
||||
the tests use the base image's. `--deploy` installs only what the lockfile
|
||||
names, checks every package against its hashes, and fails when the lockfile is
|
||||
out of date with `Pipfile`. A kept layer holds nothing a new install would
|
||||
refuse, except that a package built from source and the system packages may
|
||||
differ from a new install. The packages are kept as a layer, not a cache
|
||||
mount, for the reason Go's modules are. The tests still run after
|
||||
`ARG CACHEBUST`, and every other stage still installs its Python packages
|
||||
after it. `REPO_POLICIES.md` gains a short Python `test` phase example, and
|
||||
both checklists and the README state the rule the same way. rfscan picks this
|
||||
up on its next re-vendor.
|
||||
dependencies by hash, and the packages from the lockfile. It installs no
|
||||
Python, because the tests use the base image's. `--deploy` installs only what
|
||||
the lockfile names, checks every package against its hashes, and fails when
|
||||
the lockfile is out of date with `Pipfile`. A kept layer holds nothing a new
|
||||
install would refuse. A package built from source and the system packages may
|
||||
differ in a new install. As with Go's modules, the packages are kept in a
|
||||
layer, not a cache mount. The tests still run after `ARG CACHEBUST`, and every
|
||||
other stage still installs its Python packages after it. `REPO_POLICIES.md`
|
||||
gains a short Python `test` phase example, and both checklists and the README
|
||||
state the rule the same way. rfscan picks this up on its next re-vendor.
|
||||
- 2026-10-21: The canonical `.golangci.yml` now accepts a local `replace` of
|
||||
`github.com/grafana/sobek` in a repository's root `go.mod` (issue 134), for
|
||||
homoicon, which keeps a local copy of sobek on sneak's decision in
|
||||
|
||||
@@ -49,18 +49,21 @@ with your task.
|
||||
`--target` names it, so a lost `COPY --from=` edge leaves `docker build .`
|
||||
passing while the gate never runs. Confirm by planting a violation, not by
|
||||
reading the file.
|
||||
- [ ] In every stage `ARG CACHEBUST` comes right after the steps that install
|
||||
system packages, download Go's modules or run `script/bootstrap` to
|
||||
install a Python `test` phase's packages from `Pipfile.lock`, or right
|
||||
after `FROM` in a stage with none of these, and every step that lints,
|
||||
tests or compiles comes after it. A step that installs anything else as
|
||||
well, such as a `script/bootstrap` that also installs node packages, comes
|
||||
after it too. The `script/` builds pass a new `CACHEBUST` value on every
|
||||
build, which reruns every `RUN` step after that line and leaves the layers
|
||||
above it and the cache mounts in place; `--no-cache` would build those
|
||||
layers again and empty the mounts. A stage without the `ARG` line is
|
||||
served from cache on an unchanged tree, so the scripts and the `ARG` lines
|
||||
are re-vendored in one commit.
|
||||
- [ ] In every stage, every step that lints, tests or compiles comes after
|
||||
`ARG CACHEBUST`. `ARG CACHEBUST` comes right after these steps:
|
||||
- the steps that install system packages
|
||||
- the steps that download Go's modules
|
||||
- the step that runs `script/bootstrap` to install a Python `test` phase's
|
||||
packages from `Pipfile.lock`
|
||||
- [ ] A stage with none of these steps declares `ARG CACHEBUST` right after
|
||||
`FROM`. A step that installs anything else as well, such as a
|
||||
`script/bootstrap` that also installs node packages, comes after it too.
|
||||
The `script/` builds pass a new `CACHEBUST` value on every build, which
|
||||
reruns every `RUN` step after that line and leaves the layers above it and
|
||||
the cache mounts in place; `--no-cache` would build those layers again and
|
||||
empty the mounts. A stage without the `ARG` line is served from cache on
|
||||
an unchanged tree, so the scripts and the `ARG` lines are re-vendored in
|
||||
one commit.
|
||||
- [ ] Go: each Go stage downloads the modules (`COPY go.mod go.sum ./`,
|
||||
`RUN go mod download`) above `ARG CACHEBUST`; the steps that compile
|
||||
(`golangci-lint run`, `go test`, `go build`) mount Go's build cache
|
||||
@@ -71,15 +74,16 @@ with your task.
|
||||
canonical Go `Dockerfile` in `REPO_POLICIES.md`
|
||||
- [ ] Python: a `test` phase that installs its packages from `Pipfile.lock` does
|
||||
it above `ARG CACHEBUST`, with `COPY script/ script/`,
|
||||
`COPY Pipfile Pipfile.lock ./` and `RUN script/bootstrap`.
|
||||
`script/bootstrap`, when it runs in that step, installs nothing beyond
|
||||
system packages, pipenv and its dependencies by hash
|
||||
(`pip install --require-hashes`), and the packages from the lockfile
|
||||
(`pipenv install --deploy`, with `--dev` when the tests need the
|
||||
development packages). It installs no Python there, because the tests use
|
||||
the base image's. The tests run after `ARG CACHEBUST`, and every other
|
||||
stage installs its Python packages after it, as in the Python `test` phase
|
||||
example in `REPO_POLICIES.md`
|
||||
`COPY Pipfile Pipfile.lock ./` and `RUN script/bootstrap`. When
|
||||
`script/bootstrap` runs in that step, it installs nothing beyond system
|
||||
packages, pipenv and its dependencies, and the packages from
|
||||
`Pipfile.lock`. It installs pipenv and its dependencies by hash, with
|
||||
`pip install --require-hashes`. It installs the packages from
|
||||
`Pipfile.lock` with `pipenv install --deploy`, plus `--dev` when the tests
|
||||
need the development packages. It installs no Python there, because the
|
||||
tests use the base image's. The tests run after `ARG CACHEBUST`, and every
|
||||
other stage installs its Python packages after it, as in the Python `test`
|
||||
phase example in `REPO_POLICIES.md`
|
||||
- [ ] The gate phases invoke their tools directly, never through `make lint` or
|
||||
`script/test` — those are themselves a `docker build` and would recurse
|
||||
inside a build step
|
||||
@@ -245,9 +249,11 @@ with your task.
|
||||
git and node 24, with no yarn on `PATH`), and demonstrably executed the
|
||||
checks — a sub-second build, or `CACHED` on a step that lints or tests,
|
||||
one of the `RUN` steps below `ARG CACHEBUST`, means nothing ran. Above
|
||||
that line, the steps that install system packages, download Go's modules
|
||||
or run `script/bootstrap` to install a Python `test` phase's packages from
|
||||
`Pipfile.lock` are expected to show `CACHED`.
|
||||
that line, these steps are expected to show `CACHED`:
|
||||
- the steps that install system packages
|
||||
- the steps that download Go's modules
|
||||
- the step that runs `script/bootstrap` to install a Python `test` phase's
|
||||
packages from `Pipfile.lock`
|
||||
- [ ] A planted lint violation fails both `make lint` and a plain
|
||||
`docker build .`; revert it afterwards
|
||||
- [ ] Commit and merge fixes before starting your actual task
|
||||
|
||||
@@ -108,18 +108,21 @@ Template files can be fetched from:
|
||||
stage carries a `COPY --from=` of a harmless file from each so the image
|
||||
cannot be built unless both passed. Keep the final stage last: a stage
|
||||
nothing depends on is built only when `--target` names it.
|
||||
- In every stage `ARG CACHEBUST` comes right after the steps that install
|
||||
system packages, download Go's modules or run `script/bootstrap` to
|
||||
install a Python `test` phase's packages from `Pipfile.lock`, or right
|
||||
after `FROM` in a stage with none of these, and every step that lints,
|
||||
tests or compiles comes after it. A step that installs anything else as
|
||||
well, such as a `script/bootstrap` that also installs node packages, comes
|
||||
after it too. The `script/` builds pass a new `CACHEBUST` value on every
|
||||
build, which reruns every `RUN` step after that line and leaves the layers
|
||||
above it and the cache mounts in place; `--no-cache` would build those
|
||||
layers again and empty the mounts. A stage without the `ARG` line is
|
||||
served from cache on an unchanged tree, so the scripts and every stage's
|
||||
`ARG CACHEBUST` line are added in the same commit.
|
||||
- In every stage, every step that lints, tests or compiles comes after
|
||||
`ARG CACHEBUST`. `ARG CACHEBUST` comes right after these steps:
|
||||
- the steps that install system packages
|
||||
- the steps that download Go's modules
|
||||
- the step that runs `script/bootstrap` to install a Python `test`
|
||||
phase's packages from `Pipfile.lock`
|
||||
- A stage with none of these steps declares `ARG CACHEBUST` right after
|
||||
`FROM`. A step that installs anything else as well, such as a
|
||||
`script/bootstrap` that also installs node packages, comes after it too.
|
||||
The `script/` builds pass a new `CACHEBUST` value on every build, which
|
||||
reruns every `RUN` step after that line and leaves the layers above it and
|
||||
the cache mounts in place; `--no-cache` would build those layers again and
|
||||
empty the mounts. A stage without the `ARG` line is served from cache on
|
||||
an unchanged tree, so the scripts and every stage's `ARG CACHEBUST` line
|
||||
are added in the same commit.
|
||||
- Go: each Go stage downloads the modules (`COPY go.mod go.sum ./`,
|
||||
`RUN go mod download`) above `ARG CACHEBUST`; the steps that compile
|
||||
(`golangci-lint run`, `go test`, `go build`) mount Go's build cache
|
||||
@@ -130,15 +133,16 @@ Template files can be fetched from:
|
||||
canonical Go `Dockerfile` in `REPO_POLICIES.md`
|
||||
- Python: a `test` phase that installs its packages from `Pipfile.lock` does
|
||||
it above `ARG CACHEBUST`, with `COPY script/ script/`,
|
||||
`COPY Pipfile Pipfile.lock ./` and `RUN script/bootstrap`.
|
||||
`script/bootstrap`, when it runs in that step, installs nothing beyond
|
||||
system packages, pipenv and its dependencies by hash
|
||||
(`pip install --require-hashes`), and the packages from the lockfile
|
||||
(`pipenv install --deploy`, with `--dev` when the tests need the
|
||||
development packages). It installs no Python there, because the tests use
|
||||
the base image's. The tests run after `ARG CACHEBUST`, and every other
|
||||
stage installs its Python packages after it, as in the Python `test` phase
|
||||
example in `REPO_POLICIES.md`
|
||||
`COPY Pipfile Pipfile.lock ./` and `RUN script/bootstrap`. When
|
||||
`script/bootstrap` runs in that step, it installs nothing beyond system
|
||||
packages, pipenv and its dependencies, and the packages from
|
||||
`Pipfile.lock`. It installs pipenv and its dependencies by hash, with
|
||||
`pip install --require-hashes`. It installs the packages from
|
||||
`Pipfile.lock` with `pipenv install --deploy`, plus `--dev` when the tests
|
||||
need the development packages. It installs no Python there, because the
|
||||
tests use the base image's. The tests run after `ARG CACHEBUST`, and every
|
||||
other stage installs its Python packages after it, as in the Python `test`
|
||||
phase example in `REPO_POLICIES.md`
|
||||
- Server: the final stage builds and runs the application
|
||||
- Non-server: the final stage brings up the dev environment
|
||||
- Image pinned by sha256 hash with version/date comment
|
||||
@@ -228,9 +232,11 @@ are thin shims calling them. Model scripts:
|
||||
git and node 24, with no yarn on `PATH`), and demonstrably executed the
|
||||
checks — a sub-second build, or `CACHED` on a step that lints or tests,
|
||||
one of the `RUN` steps below `ARG CACHEBUST`, means nothing ran. Above
|
||||
that line, the steps that install system packages, download Go's modules
|
||||
or run `script/bootstrap` to install a Python `test` phase's packages from
|
||||
`Pipfile.lock` are expected to show `CACHED`.
|
||||
that line, these steps are expected to show `CACHED`:
|
||||
- the steps that install system packages
|
||||
- the steps that download Go's modules
|
||||
- the step that runs `script/bootstrap` to install a Python `test` phase's
|
||||
packages from `Pipfile.lock`
|
||||
- [ ] Plant a lint violation and confirm both `make lint` and a plain
|
||||
`docker build .` fail on it; revert. A plain build that passes proves the
|
||||
final stage is missing its `COPY --from=` edge to the gate phases.
|
||||
|
||||
+36
-32
@@ -174,28 +174,32 @@ style conventions are in separate documents:
|
||||
`script/cibuild`. `script/lint`, `script/test`, `script/cibuild` and
|
||||
`script/docker` are the four, and there is no fifth — `script/check` runs the
|
||||
two gate phases and `script/fmt-check`, and builds no image of its own. In
|
||||
every stage of the `Dockerfile`, `ARG CACHEBUST` comes right after the steps
|
||||
that install system packages, download Go's modules or run `script/bootstrap`
|
||||
to install a Python `test` phase's packages from `Pipfile.lock`, or right
|
||||
after `FROM` in a stage with none of these, and every step that lints, tests
|
||||
or compiles comes after it. A step that installs anything else as well, such
|
||||
as a `script/bootstrap` that also installs node packages, comes after it too.
|
||||
A new value reruns every `RUN` step after that line; `COPY` steps are compared
|
||||
by content and need nothing. Docker keeps the layers above the line until the
|
||||
base image, their own lines or the files they copy change. A stage without the
|
||||
`ARG` line is served from cache on an unchanged tree, so a repository
|
||||
re-vendors the scripts and adds the `ARG` lines in one commit. The scripts do
|
||||
not pass `--no-cache`: it reruns every step too, but it also builds the layers
|
||||
above the line again and empties every cache mount, so Go's build cache and
|
||||
golangci-lint's, which the canonical Go `Dockerfile` below mounts so that they
|
||||
survive between check builds, would start empty every time. A bare
|
||||
`docker build .` is not evidence that anything ran: a sub-second build
|
||||
reporting success is a cache hit, not a result. Never invalidate by pruning —
|
||||
`docker builder prune` and friends destroy a build cache shared with every
|
||||
other build on the host. When a check is added or changed, prove it works by
|
||||
planting a defect it must catch and watching the run fail on it, then revert
|
||||
the defect. A green run alone shows neither that the check ran nor that it
|
||||
covers what it should.
|
||||
every stage of the `Dockerfile`, every step that lints, tests or compiles
|
||||
comes after `ARG CACHEBUST`. `ARG CACHEBUST` comes right after these steps:
|
||||
- the steps that install system packages
|
||||
- the steps that download Go's modules
|
||||
- the step that runs `script/bootstrap` to install a Python `test` phase's
|
||||
packages from `Pipfile.lock`
|
||||
|
||||
A stage with none of these steps declares `ARG CACHEBUST` right after
|
||||
`FROM`. A step that installs anything else as well, such as a
|
||||
`script/bootstrap` that also installs node packages, comes after it too. A
|
||||
new value reruns every `RUN` step after that line; `COPY` steps are compared
|
||||
by content and need nothing. Docker keeps the layers above the line until
|
||||
the base image, their own lines or the files they copy change. A stage
|
||||
without the `ARG` line is served from cache on an unchanged tree, so a
|
||||
repository re-vendors the scripts and adds the `ARG` lines in one commit.
|
||||
The scripts do not pass `--no-cache`: it reruns every step too, but it also
|
||||
builds the layers above the line again and empties every cache mount, so
|
||||
Go's build cache and golangci-lint's, which the canonical Go `Dockerfile`
|
||||
below mounts so that they survive between check builds, would start empty
|
||||
every time. A bare `docker build .` is not evidence that anything ran: a
|
||||
sub-second build reporting success is a cache hit, not a result. Never
|
||||
invalidate by pruning — `docker builder prune` and friends destroy a build
|
||||
cache shared with every other build on the host. When a check is added or
|
||||
changed, prove it works by planting a defect it must catch and watching the
|
||||
run fail on it, then revert the defect. A green run alone shows neither that
|
||||
the check ran nor that it covers what it should.
|
||||
|
||||
- **The gate phases are separate stages, and the build stage depends on both.**
|
||||
The lint phase is based on the `golangci/golangci-lint` image (pinned by
|
||||
@@ -380,8 +384,8 @@ style conventions are in separate documents:
|
||||
`Pipfile.lock`. It installs no Python there: the tests use the base
|
||||
image's Python, which the image's hash pins. If it installs anything else
|
||||
there, the step goes after `ARG CACHEBUST`.
|
||||
- It installs pipenv and its dependencies by hash, as the rule that every
|
||||
external reference is pinned by hash requires. It does so with
|
||||
- Every external reference must be pinned by hash, so `script/bootstrap`
|
||||
installs pipenv and its dependencies by hash. It does so with
|
||||
`pip install --require-hashes -r` and a requirements file in `script/`
|
||||
that lists each of them with `==` and its `--hash`. pip refuses to install
|
||||
a dependency the file does not list.
|
||||
@@ -394,14 +398,14 @@ style conventions are in separate documents:
|
||||
their hashes. The packages `virtualenv` puts into the virtualenv that
|
||||
pipenv creates come from wheels the `virtualenv` package ships, which its
|
||||
hash covers. Two things in the layer are not fixed by a hash. A package
|
||||
with no ready-built wheel for the image is built from source, with build
|
||||
tools pip downloads without a hash check, so a kept layer's build of it
|
||||
may differ from a new one. And system packages keep the versions the
|
||||
package manager gave when the layer was built; a new install may get newer
|
||||
ones.
|
||||
- The packages are kept in a layer rather than a cache mount for the reason
|
||||
Go's modules are: a cache mount is shared by every build on the host, and
|
||||
nothing checks its files again.
|
||||
with no ready-built wheel for the image is built from source. pip
|
||||
downloads the build tools without a hash check, so a kept layer's build of
|
||||
such a package may differ from a new one. And system packages keep the
|
||||
versions the package manager gave when the layer was built; a new install
|
||||
may get newer ones.
|
||||
- As with Go's modules, the packages are kept in a layer, not a cache mount.
|
||||
A cache mount is shared by every build on the host, and nothing checks its
|
||||
files again.
|
||||
|
||||
In this example, `script/bootstrap` finds `make` and `git` already
|
||||
installed, installs pipenv and its dependencies by hash, and runs
|
||||
|
||||
Reference in New Issue
Block a user