Keep a Python test phase's lockfile install in check builds (closes #133)
check / check (push) Canceled after 0s

On sneak's ruling in
sneak/rfscan#80 (comment), a Python
test phase installs its packages from Pipfile.lock above ARG CACHEBUST,
so Docker keeps them until the base image, Pipfile, Pipfile.lock or a
line up to the install changes. The step is COPY Pipfile Pipfile.lock ./
and one RUN of pipenv install --deploy, or a script/bootstrap that
installs nothing beyond system packages, the pinned pipenv and the
lockfile's packages. --deploy installs only what the lockfile names and
checks its hashes. The tests, and every other stage's Python installs,
stay after the line. REPO_POLICIES.md gains a short example; both
checklists and the README state the rule the same way.

Model: opus-5-5
This commit is contained in:
2026-10-21 15:11:57 +00:00
parent aef30c320a
commit 83f148f3b2
5 changed files with 126 additions and 52 deletions
+62 -21
View File
@@ -1,6 +1,6 @@
---
title: Repository Policies
last_modified: 2026-10-18
last_modified: 2026-10-21
---
This document covers repository structure, tooling, and workflow standards. Code
@@ -175,26 +175,26 @@ style conventions are in separate documents:
`script/docker` are the four, and there is no fifth — `script/check` runs the
two gate phases and `script/fmt-check`, and builds no image of its own. In
every stage of the `Dockerfile`, `ARG CACHEBUST` comes right after the steps
that install system packages and download Go's modules, or right after `FROM`
in a stage with neither, and every step that lints, tests or compiles comes
after it. A step that installs anything else as well, such as a
`script/bootstrap` that also installs node packages, comes after it too. A new
value reruns every `RUN` step after that line; `COPY` steps are compared by
content and need nothing. Docker keeps the layers above the line until the
base image, their own lines or the files they copy change. A stage without the
`ARG` line is served from cache on an unchanged tree, so a repository
re-vendors the scripts and adds the `ARG` lines in one commit. The scripts do
not pass `--no-cache`: it reruns every step too, but it also builds the layers
above the line again and empties every cache mount, so Go's build cache and
golangci-lint's, which the canonical Go `Dockerfile` below mounts so that they
survive between check builds, would start empty every time. A bare
`docker build .` is not evidence that anything ran: a sub-second build
reporting success is a cache hit, not a result. Never invalidate by pruning —
`docker builder prune` and friends destroy a build cache shared with every
other build on the host. When a check is added or changed, prove it works by
planting a defect it must catch and watching the run fail on it, then revert
the defect. A green run alone shows neither that the check ran nor that it
covers what it should.
that install system packages, download Go's modules or install a `test`
phase's Python packages from `Pipfile.lock`, or right after `FROM` in a stage
with none of these, and every step that lints, tests or compiles comes after
it. A step that installs anything else as well, such as a `script/bootstrap`
that also installs node packages, comes after it too. A new value reruns every
`RUN` step after that line; `COPY` steps are compared by content and need
nothing. Docker keeps the layers above the line until the base image, their
own lines or the files they copy change. A stage without the `ARG` line is
served from cache on an unchanged tree, so a repository re-vendors the scripts
and adds the `ARG` lines in one commit. The scripts do not pass `--no-cache`:
it reruns every step too, but it also builds the layers above the line again
and empties every cache mount, so Go's build cache and golangci-lint's, which
the canonical Go `Dockerfile` below mounts so that they survive between check
builds, would start empty every time. A bare `docker build .` is not evidence
that anything ran: a sub-second build reporting success is a cache hit, not a
result. Never invalidate by pruning — `docker builder prune` and friends
destroy a build cache shared with every other build on the host. When a check
is added or changed, prove it works by planting a defect it must catch and
watching the run fail on it, then revert the defect. A green run alone shows
neither that the check ran nor that it covers what it should.
- **The gate phases are separate stages, and the build stage depends on both.**
The lint phase is based on the `golangci/golangci-lint` image (pinned by
@@ -309,6 +309,8 @@ style conventions are in separate documents:
a package's findings only when the package's files and those of every
package it imports, `go.mod`, the linters and their settings in
`.golangci.yml`, and the golangci-lint version are all unchanged.
- In a Python `test` phase, the packages installed from `Pipfile.lock`,
as described for that phase below.
- If the project uses `//go:embed` directives that reference build artifacts
(e.g. a web frontend compiled in a separate stage), the lint phase must
create placeholder files so the embed directives resolve. Example:
@@ -363,6 +365,45 @@ style conventions are in separate documents:
`--build-arg VERSION=...`, as `script/docker` and `script/cibuild` already
do.
- **A Python `test` phase installs its packages from `Pipfile.lock` above
`ARG CACHEBUST`, so check builds keep them.** The install has steps of its
own: `COPY Pipfile Pipfile.lock ./`, then one `RUN` that runs
`pipenv install --deploy`, with `--dev` when the tests need the development
packages. Docker keeps what that `RUN` installed until the base image,
`Pipfile`, `Pipfile.lock` or a line up to and including the `RUN` changes. If
the `RUN` runs `script/bootstrap` instead, as in the example below,
`COPY script/ script/` comes before it, so Docker installs again when
`script/` changes too. That `script/bootstrap` may install nothing beyond
system packages, the pinned pipenv and the packages from the lockfile; one
that installs anything else comes after `ARG CACHEBUST`. A kept layer holds
the same packages a new install would: `--deploy` installs only what
`Pipfile.lock` names, checks every package against the lockfile's hashes, and
fails when the lockfile is out of date with `Pipfile`. The packages are kept
in a layer rather than a cache mount for the reason Go's modules are: a cache
mount is shared by every build on the host, and nothing checks its files
again. The tests run after `ARG CACHEBUST`, on every check build. In every
other stage, a step that installs Python packages comes after `ARG CACHEBUST`.
An example, with a `script/bootstrap` that installs the pinned pipenv and then
runs `pipenv install --dev --deploy`:
```dockerfile
# Test phase. The 90-second timeout covers both pytest runs.
# python:3.x-slim, YYYY-MM-DD
FROM python@sha256:... AS test
RUN apt-get update \
&& apt-get install -y --no-install-recommends make git \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY script/ script/
COPY Pipfile Pipfile.lock ./
RUN script/bootstrap
ARG CACHEBUST
COPY . .
RUN timeout 90 sh -c 'pipenv run pytest || { \
echo "--- Rerunning with -v for details ---"; \
pipenv run pytest -v; exit 1; }'
```
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
runs `script/cibuild` on push, and checks out the repo as its only other step,
with `persist-credentials: false`: `script/cibuild` needs no token, and