2 Commits
Author SHA1 Message Date
clawbot 86c7617f86 Install node, yarn and python from the distribution (closes #146)
check / check (push) Waiting to run
Follows sneak's ruling at
#146 (comment). Runtimes
and the installer tools bundled with python come from the distribution,
with no version and no hash. Third-party packages stay pinned by hash.

script/bootstrap drops nvm and corepack. It installs node, yarn and the
node packages only when yarn.lock is in the tree, and stops there when
yarn's major version is not 1. This repository's Dockerfile starts from
Alpine 3.24. The Python example starts from debian:trixie-slim, keeps
pipenv at $HOME/.pipenv on a host and in an image, and sets
PIP_ONLY_BINARY=:all:, so pip downloads no build tools without a hash
check.

Model: opus-5-5
2026-11-04 09:55:26 +01:00
clawbotandsneak b36ddacee1 Rewrite REPO_POLICIES.md sentence by sentence (closes #136) (#147)
check / check (push) Canceled after 0s
Rewrites every sentence of `prompts/REPO_POLICIES.md` against the sentence rules, for #136. No rule's meaning, `must`/`should` or order changed. Code blocks, digests, URLs and commands are byte-identical. Both checklists carry the same rewrite of the paragraph they quote.

## For sneak

Kept as they are: each looks false or contradicts another rule. Line numbers are this branch's.

- Lines 196-197: the final stage carries the `COPY --from=` lines; in the `Dockerfile` (345-346) the build stage does, the runtime stage none.
- Lines 629-641 and 670-675: `make test` runs `go test` and `pytest` on the host; 80-81 make targets shims, 183-185 make `script/test` a Docker build.
- Lines 23-37 pin by hash; 103-112 install node (nvm) and yarn (corepack) by version. Tracked at #146.
- Line 131 counts four extensions; `script/lint`, `script/fmt`, `script/fmt-check`, `script/docker` (85-89) are not in the standard either.
- Lines 149-151: the Makefile is the single source of truth; 80-81 (implementation in `script/`), 211 (tool invoked in a phase), 616 (test command in the `Dockerfile`).
- Lines 64-68 require `make test` and `make lint`, which 183-185 make Docker builds; line 167 only says a repo should have a `Dockerfile`.
- Line 759 says byte-identical; the block at 767 has `tag="$(script/projectname)"`, both scripts have `tag="$("$SCRIPT_DIR/projectname")"`.
- Lines 834-839: `script/bootstrap` never tests presence, 860 excepts only node; `ensure_yarn` in the canonical `script/bootstrap` skips the install when yarn is present, without a version check.

## Disclosures

- Judgement call: short name lists stay inline under about 15 words.

Model: fable-5-1
Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #147
Co-authored-by: clawbot <35+clawbot@noreply.example.org>
2026-11-04 05:17:16 +01:00
10 changed files with 827 additions and 648 deletions
+16 -10
View File
@@ -2,11 +2,15 @@
# lint` or `script/lint`, which are themselves a docker build and would
# recurse into a daemon that does not exist in a build step.
#
# Every stage declares ARG CACHEBUST first: script/ passes a new value on
# every build, which reruns every RUN step after it.
# In every stage, ARG CACHEBUST comes right after the step that installs
# system packages. script/ passes a new value on every build. That value
# reruns every RUN step below the ARG line, and the install step keeps
# its layer. node and yarn come from the distribution, which is trusted,
# so they need no version and no hash.
#
# node 22-alpine, 2026-02-22
FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS lint
# alpine 3.24.3, 2026-11-04
FROM alpine@sha256:be2656061b536c1883c4034790e21d0a09c5aaaf51968b89368a3c0371ea2c9a AS lint
RUN apk add --no-cache git make nodejs yarn
ARG CACHEBUST
WORKDIR /app
@@ -21,8 +25,9 @@ RUN yarn run prettier --check '**/*.md' --tab-width 4 --prose-wrap always
# Test phase, same shape and for the same reason.
#
# node 22-alpine, 2026-02-22
FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS test
# alpine 3.24.3, 2026-11-04
FROM alpine@sha256:be2656061b536c1883c4034790e21d0a09c5aaaf51968b89368a3c0371ea2c9a AS test
RUN apk add --no-cache git make nodejs yarn
ARG CACHEBUST
WORKDIR /app
@@ -41,8 +46,9 @@ RUN echo "No tests defined."
# this image cannot be produced unless lint and test passed. A stage
# appended after this one would drop all three out of a plain build.
#
# node 22-alpine, 2026-02-22
FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34
# alpine 3.24.3, 2026-11-04
FROM alpine@sha256:be2656061b536c1883c4034790e21d0a09c5aaaf51968b89368a3c0371ea2c9a
RUN apk add --no-cache git make nodejs yarn
ARG CACHEBUST
WORKDIR /app
@@ -50,8 +56,8 @@ WORKDIR /app
COPY --from=lint /app/package.json /dev/null
COPY --from=test /app/package.json /dev/null
# script/bootstrap installs all prerequisites. Manifests are copied
# first so that layer stays cached until dependencies change.
# script/bootstrap finds the system packages above and installs the node
# packages.
COPY script/ script/
COPY package.json yarn.lock ./
RUN script/bootstrap
+3 -3
View File
@@ -156,9 +156,9 @@ A stage with none of these steps declares `ARG CACHEBUST` right after `FROM`.
The `script/` builds pass a new `CACHEBUST` value on every build, which reruns
every `RUN` step after that line and leaves the layers above it and the cache
mounts in place; `--no-cache` would build those layers again and empty the
mounts. This repo's own `Dockerfile` declares `ARG CACHEBUST` right after `FROM`
in every stage: each stage installs system packages and node packages together
in one `script/bootstrap` step, and check builds may keep system packages but
mounts. This repo's own `Dockerfile` installs git, make, node and yarn right
after `FROM` in every stage, above `ARG CACHEBUST`. `script/bootstrap` installs
the node packages below that line, because check builds keep system packages but
not node packages.
## Rationale
+24
View File
@@ -21,6 +21,30 @@ fmt-check, and commit.
# Completed Steps
- 2026-11-04: node, yarn, npm, python, pip and venv now come from the
distribution with no version and no hash
(https://git.eeqj.de/sneak/prompts/issues/146), on sneak's ruling at
https://git.eeqj.de/sneak/prompts/issues/146#issuecomment-166033. Third-party
packages stay pinned by hash. These changed:
- `script/bootstrap` no longer uses nvm or corepack, and stops when yarn's
major version is not 1.
- `script/bootstrap` installs node, yarn and the node packages only when
`yarn.lock` is in the tree.
- `script/bootstrap`, `script/fmt` and `script/fmt-check` run `yarn` or
`yarnpkg`, whichever is present.
- This repository's `Dockerfile` starts from Alpine 3.24.
- The Python `test` phase example starts from `debian:trixie-slim`.
- A Python repository's `script/bootstrap` creates its virtual environment
at `$HOME/.pipenv`, so it works on a host and in a `Dockerfile`.
- `PIP_ONLY_BINARY=:all:` stops a package with no wheel, so pip downloads no
build tools without a hash check.
- 2026-11-04: Rewrote `prompts/REPO_POLICIES.md` sentence by sentence
(https://git.eeqj.de/sneak/prompts/issues/136), on sneak's yes at
https://git.eeqj.de/sneak/prompts/issues/136#issuecomment-165193. Every rule
keeps its meaning and its place, and `must` and `should` stay as they were.
Long sentences are split, comma-chained lists are now bulleted lists, and
every reason stays. Code blocks, digests, URLs, file names, commands and
quoted strings are unchanged.
- 2026-11-03: The canonical golangci-lint is now v2.15.0, still built with
go1.27.0, so the `go` directive rule is unchanged (issue 142). Repositories
pick this up on their next re-vendor. Each item in the canonical
+3 -2
View File
@@ -1,6 +1,6 @@
---
title: Code Styleguide — JavaScript
last_modified: 2026-02-22
last_modified: 2026-11-04
---
1. Use `const` for everything. If you need to reassign, use `let`. Never use
@@ -8,7 +8,8 @@ last_modified: 2026-02-22
1. Use yarn for package management, avoid using npm.
1. Use LTS node versions.
1. Install node from the distribution. If its version is too old, use a newer
distribution.
1. Use `prettier` for code formatting, with four spaces for indentation.
+91 -64
View File
@@ -1,6 +1,6 @@
---
title: Existing Repo Checklist
last_modified: 2026-11-03
last_modified: 2026-11-04
---
Use this checklist when beginning work in a repo that may not yet conform to our
@@ -56,14 +56,14 @@ with your task.
- the step that runs `script/bootstrap` to install a Python `test` phase's
packages from `Pipfile.lock`
- [ ] A stage with none of these steps declares `ARG CACHEBUST` right after
`FROM`. A step that installs anything else as well, such as a
`script/bootstrap` that also installs node packages, comes after it too.
The `script/` builds pass a new `CACHEBUST` value on every build, which
reruns every `RUN` step after that line and leaves the layers above it and
the cache mounts in place; `--no-cache` would build those layers again and
empty the mounts. A stage without the `ARG` line is served from cache on
an unchanged tree, so the scripts and the `ARG` lines are re-vendored in
one commit.
`FROM`. A step that also installs anything else comes after
`ARG CACHEBUST` too. A `script/bootstrap` that also installs node packages
is one. The `script/` builds pass a new `CACHEBUST` value on every build,
which reruns every `RUN` step after that line and leaves the layers above
it and the cache mounts in place; `--no-cache` would build those layers
again and empty the mounts. A stage without the `ARG` line is served from
cache on an unchanged tree, so the scripts and the `ARG` lines are
re-vendored in one commit.
- [ ] Go: each Go stage downloads the modules (`COPY go.mod go.sum ./`,
`RUN go mod download`) above `ARG CACHEBUST`; the steps that compile
(`golangci-lint run`, `go test`, `go build`) mount Go's build cache
@@ -76,15 +76,25 @@ with your task.
- [ ] Python: a `test` phase that installs its packages from `Pipfile.lock` does
it above `ARG CACHEBUST`, with `COPY script/ script/`,
`COPY Pipfile Pipfile.lock ./` and `RUN script/bootstrap`. When
`script/bootstrap` runs in that step, it installs nothing beyond system
packages, pipenv and its dependencies, and the packages from
`Pipfile.lock`. It installs pipenv and its dependencies by hash, with
`pip install --require-hashes`. It installs the packages from
`Pipfile.lock` with `pipenv install --deploy`, plus `--dev` when the tests
need the development packages. It installs no Python there, because the
tests use the base image's. The tests run after `ARG CACHEBUST`, and every
other stage installs its Python packages after it, as in the Python `test`
phase example in `REPO_POLICIES.md`
`script/bootstrap` runs in that step, it installs nothing beyond these:
- system packages
- pipenv and its dependencies
- the packages from `Pipfile.lock`
It creates a virtual environment at `$HOME/.pipenv` with `python3 -m venv`
when it is missing, then uses that environment's `pip` and `pipenv`. On a
host, it first installs the distribution's python and venv packages when
they are missing. It installs pipenv and its dependencies by hash, with
`pip install --require-hashes`. It installs the packages from `Pipfile.lock`
with `pipenv install --deploy`, plus `--dev` when the tests need the
development packages. It sets `PIP_ONLY_BINARY=:all:` for both installs, so
a package with no wheel for the image stops the install. In that step it
installs no Python, because a step above it already did. `Pipfile` names no
`python_version`, because `--deploy` stops when the distribution's Python
differs from it. The tests run after `ARG CACHEBUST`, and every other stage
installs its Python packages after it, as in the Python `test` phase example
in `REPO_POLICIES.md`
- [ ] The gate phases invoke their tools directly, never through `make lint` or
`script/test` — those are themselves a `docker build` and would recurse
inside a build step
@@ -105,37 +115,43 @@ with your task.
`services/api/.claude/`: add anchored entries for those directories.
- [ ] If the repo embeds a version in a binary: `.dockerignore` lets `.git` into
the build context. It keeps out every git `config` at any depth
(`**/.git/config`, `**/.git/modules/**/config`): the repository's own,
each submodule's under `.git/modules/`, and that of a submodule keeping
its own `.git` directory. `git describe` does not need them, and each can
hold a credential: a password in a remote URL, or the token the CI
checkout step stores there. A submodule whose name has a `config` segment
(`config`, `deploy/config`, `config/lib`) loses its whole git directory to
`**/.git/modules/**/config`, and Go's version stamping then fails the
build: give it a name without that segment (`git submodule add --name`).
The stage that compiles has `git` (the Debian Go image has it; an alpine
one needs `apk add --no-cache git`) and takes the version from the
`VERSION` build argument when one is given, otherwise from
`git describe --tags --always`. That gives the tag on a tagged commit; on
a later commit, the tag, the number of commits since it and the short
commit (`v1.2.3-4-gabc1234`); and the short commit when no tag is
reachable. The stage that compiles also marks its working directory safe
for git (`git config --system --add safe.directory /src`): a context sent
as a tar stream keeps the sender's file owners, and git refuses a checkout
owned by another user, so the version would come out empty. `ARG VERSION`
has no default, and the build fails if the context carries `.git` and the
version still comes out empty, `dev` or `unknown`. A plain
`docker build .` with no build arguments must succeed; a Dockerfile that
refuses an empty build argument drops that refusal and keeps the argument.
A checkout whose `.git` is a file (a linked worktree, or a repository
checked out as a submodule) is the exception: that file points to a git
directory outside the build context, so the build cannot read the version
and a plain `docker build .` fails; pass the version with
`--build-arg VERSION=...`. `script/docker` and `script/cibuild` already
pass the version they compute on the host; it takes precedence. The
canonical `.gitea/workflows/check.yml` sets `fetch-depth: 0` on its
checkout step, which otherwise clones shallow and fetches no tags, so a CI
build finds the tag too.
(`**/.git/config`, `**/.git/modules/**/config`):
- the repository's own
- each submodule's under `.git/modules/`
- that of a submodule keeping its own `.git` directory
`git describe` does not need them, and each can hold a credential: a
password in a remote URL, or the token the CI checkout step stores there.
Some submodule names have a `config` segment, as in `config`,
`deploy/config` or `config/lib`. Such a submodule loses its whole git
directory to `**/.git/modules/**/config`, and Go's version stamping then
fails the build. Give it a name without that segment
(`git submodule add --name`). The stage that compiles has `git`: the Debian
Go image has it, and an alpine one needs `apk add --no-cache git`. That
stage takes the version from the `VERSION` build argument when one is given,
otherwise from `git describe --tags --always`. That gives:
- the tag, on a tagged commit
- the tag, the number of commits since it and the short commit
(`v1.2.3-4-gabc1234`), on a later commit
- the short commit, when no tag is reachable
The stage that compiles also marks its working directory safe for git
(`git config --system --add safe.directory /src`). A context sent as a tar
stream keeps the sender's file owners, and git refuses a checkout owned by
another user, so the version would come out empty. `ARG VERSION` has no
default. The build fails if the context carries `.git` and the version still
comes out empty, `dev` or `unknown`. A plain `docker build .` with no build
arguments must succeed; a Dockerfile that refuses an empty build argument
drops that refusal and keeps the argument. A checkout whose `.git` is a file
is the exception: a linked worktree, or a repository checked out as a
submodule. That file points to a git directory outside the build context, so
the build cannot read the version and a plain `docker build .` fails. Pass
the version with `--build-arg VERSION=...`. `script/docker` and
`script/cibuild` already pass the version they compute on the host; it takes
precedence. The canonical `.gitea/workflows/check.yml` sets `fetch-depth: 0`
on its checkout step, which otherwise clones shallow and fetches no tags, so
a CI build finds the tag too.
- [ ] Gitea Actions workflow in `.gitea/workflows/` runs `script/cibuild` on
push, checks out with `persist-credentials: false` and with
`fetch-depth: 0` (which fetches the tags `git describe` needs), carries
@@ -151,7 +167,9 @@ with your task.
- [ ] JS: `package.json`, `yarn.lock`, `.prettierrc`, `.prettierignore`
(fetch from
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.prettierrc` and
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.prettierignore`)
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.prettierignore`).
Make the first `yarn.lock` with yarn 1, from a distribution that
packages it, such as Alpine (`apk add nodejs yarn`).
- [ ] Python: `pyproject.toml`
- [ ] Docs/writing: `.prettierrc`, `.prettierignore` (same URLs as above)
@@ -186,13 +204,17 @@ with your task.
`docker build` in `script/`. Without the bootstrap the CI run dies in
`script/fmt-check`, which runs the formatter on the host and finds nothing
installed.
- [ ] `script/fmt` and `script/fmt-check` pick their `yarn` with the same test
as `script/bootstrap`: the `yarn` on `PATH` when the node on `PATH` has
the pinned major version, and otherwise yarn under the pinned node, with
nvm loaded through `$HOME/.nvm/nvm.sh`. `script/bootstrap` leaves the node
and yarn it installs under nvm off the `PATH` of the shell that called it,
so a bare `yarn` exits 127 on a runner carrying nothing but docker and
git, or runs under another node.
- [ ] `script/bootstrap` installs node, yarn and the node packages only when
`yarn.lock` is in the tree. node and yarn come from the distribution when
they are missing. With `yarn.lock` in the tree, `script/bootstrap` stops
with a one-line message when yarn's major version is not 1.
- [ ] A `Dockerfile` stage copies `yarn.lock` before it runs `script/bootstrap`
only if that stage needs yarn. A stage that needs yarn starts from an
image whose distribution packages yarn 1, such as Alpine.
- [ ] `script/bootstrap`, `script/fmt` and `script/fmt-check` run yarn by
whichever name is present: `yarn`, or `yarnpkg` on Debian and Ubuntu. With
neither name on `PATH`, `script/fmt` and `script/fmt-check` print one line
saying to run `script/bootstrap`, and exit 1.
- [ ] `script/bootstrap` installs no linter of its own — delete the block, its
version variables and its call site. A JS repo's `yarn install` stays; it
brings a linter along with every other dependency, and no verdict is taken
@@ -220,8 +242,12 @@ with your task.
is a package from the distribution's own package index, which needs no
version and no hash. Pulling that index is also allowed. In a
`Dockerfile`, the distribution is the pinned base image's. On a host, it
is the host's. A package source that a `Dockerfile` adds itself does not
get this exception
is the host's, and the host's package manager stands for it, nix and brew
included. A package source that a `Dockerfile` adds itself does not get
this exception
- [ ] node, yarn, npm, python, pip and venv come from the distribution wherever
possible, never from nvm, pyenv, corepack or `npm install -g yarn`. If the
distribution's version is too old, use a newer distribution
- [ ] Using `yarn`, not `npm` (JS projects)
# Directory Structure
@@ -250,11 +276,12 @@ with your task.
# Final
- [ ] `make check` passes
- [ ] `script/cibuild` succeeds in a fresh clone on a host carrying what CI has
(the runner image `docker.gitea.com/runner-images:ubuntu-latest`: docker,
git and node 24, with no yarn on `PATH`), and demonstrably executed the
checks — a sub-second build, or `CACHED` on a step that lints or tests,
one of the `RUN` steps below `ARG CACHEBUST`, means nothing ran. Above
- [ ] `script/cibuild` succeeds in a fresh clone on a host carrying what CI has.
That host is the runner image
`docker.gitea.com/runner-images:ubuntu-latest`, which has docker, git and
node 24. Take any yarn off `PATH` first. The run must show that the checks
executed. A sub-second build means nothing ran. So does `CACHED` on a step
that lints or tests, one of the `RUN` steps below `ARG CACHEBUST`. Above
that line, these steps are expected to show `CACHED`:
- the steps that install system packages
- the steps that download Go's modules
+99 -68
View File
@@ -1,6 +1,6 @@
---
title: New Repo Checklist
last_modified: 2026-11-03
last_modified: 2026-11-04
---
Use this checklist when creating a new repository from scratch. Follow the steps
@@ -75,34 +75,42 @@ Template files can be fetched from:
anchored entry.
- If the image embeds a version in a binary: `.dockerignore` lets `.git`
into the build context. It keeps out every git `config` at any depth
(`**/.git/config`, `**/.git/modules/**/config`): the repository's own,
each submodule's under `.git/modules/`, and that of a submodule keeping
its own `.git` directory. `git describe` does not need them, and each can
hold a credential: a password in a remote URL, or the token the CI
checkout step stores there. A submodule whose name has a `config` segment
(`config`, `deploy/config`, `config/lib`) loses its whole git directory to
`**/.git/modules/**/config`, and Go's version stamping then fails the
build: give it a name without that segment (`git submodule add --name`).
The stage that compiles has `git` (the Debian Go image has it; an alpine
one needs `apk add --no-cache git`) and takes the version from the
`VERSION` build argument when one is given, otherwise from
`git describe --tags --always`. That gives the tag on a tagged commit; on
a later commit, the tag, the number of commits since it and the short
commit (`v1.2.3-4-gabc1234`); and the short commit when no tag is
reachable. The stage that compiles also marks its working directory safe
for git (`git config --system --add safe.directory /src`): a context sent
as a tar stream keeps the sender's file owners, and git refuses a checkout
owned by another user, so the version would come out empty. `ARG VERSION`
has no default, and the build fails if the context carries `.git` and the
version still comes out empty, `dev` or `unknown`. A plain
`docker build .` with no build arguments must succeed; a Dockerfile that
refuses an empty build argument drops that refusal and keeps the argument.
A checkout whose `.git` is a file (a linked worktree, or a repository
checked out as a submodule) is the exception: that file points to a git
directory outside the build context, so the build cannot read the version
and a plain `docker build .` fails; pass the version with
`--build-arg VERSION=...`, as `script/docker` and `script/cibuild` already
do.
(`**/.git/config`, `**/.git/modules/**/config`):
- the repository's own
- each submodule's under `.git/modules/`
- that of a submodule keeping its own `.git` directory
`git describe` does not need them, and each can hold a credential: a
password in a remote URL, or the token the CI checkout step stores
there. Some submodule names have a `config` segment, as in `config`,
`deploy/config` or `config/lib`. Such a submodule loses its whole git
directory to `**/.git/modules/**/config`, and Go's version stamping then
fails the build. Give it a name without that segment
(`git submodule add --name`). The stage that compiles has `git`: the
Debian Go image has it, and an alpine one needs
`apk add --no-cache git`. That stage takes the version from the
`VERSION` build argument when one is given, otherwise from
`git describe --tags --always`. That gives:
- the tag, on a tagged commit
- the tag, the number of commits since it and the short commit
(`v1.2.3-4-gabc1234`), on a later commit
- the short commit, when no tag is reachable
The stage that compiles also marks its working directory safe for git
(`git config --system --add safe.directory /src`). A context sent as a
tar stream keeps the sender's file owners, and git refuses a checkout
owned by another user, so the version would come out empty.
`ARG VERSION` has no default. The build fails if the context carries
`.git` and the version still comes out empty, `dev` or `unknown`. A
plain `docker build .` with no build arguments must succeed; a
Dockerfile that refuses an empty build argument drops that refusal and
keeps the argument. A checkout whose `.git` is a file is the exception:
a linked worktree, or a repository checked out as a submodule. That file
points to a git directory outside the build context, so the build cannot
read the version and a plain `docker build .` fails. Pass the version
with `--build-arg VERSION=...`, as `script/docker` and `script/cibuild`
already do.
- The Dockerfile carries a `lint` phase and a `test` phase, each invoking
its tool directly rather than through `make` or `script/`, and the final
stage carries a `COPY --from=` of a harmless file from each so the image
@@ -115,14 +123,14 @@ Template files can be fetched from:
- the step that runs `script/bootstrap` to install a Python `test`
phase's packages from `Pipfile.lock`
- A stage with none of these steps declares `ARG CACHEBUST` right after
`FROM`. A step that installs anything else as well, such as a
`script/bootstrap` that also installs node packages, comes after it too.
The `script/` builds pass a new `CACHEBUST` value on every build, which
reruns every `RUN` step after that line and leaves the layers above it and
the cache mounts in place; `--no-cache` would build those layers again and
empty the mounts. A stage without the `ARG` line is served from cache on
an unchanged tree, so the scripts and every stage's `ARG CACHEBUST` line
are added in the same commit.
`FROM`. A step that also installs anything else comes after
`ARG CACHEBUST` too. A `script/bootstrap` that also installs node packages
is one. The `script/` builds pass a new `CACHEBUST` value on every build,
which reruns every `RUN` step after that line and leaves the layers above
it and the cache mounts in place; `--no-cache` would build those layers
again and empty the mounts. A stage without the `ARG` line is served from
cache on an unchanged tree, so the scripts and every stage's
`ARG CACHEBUST` line are added in the same commit.
- Go: each Go stage downloads the modules (`COPY go.mod go.sum ./`,
`RUN go mod download`) above `ARG CACHEBUST`; the steps that compile
(`golangci-lint run`, `go test`, `go build`) mount Go's build cache
@@ -135,18 +143,30 @@ Template files can be fetched from:
- Python: a `test` phase that installs its packages from `Pipfile.lock` does
it above `ARG CACHEBUST`, with `COPY script/ script/`,
`COPY Pipfile Pipfile.lock ./` and `RUN script/bootstrap`. When
`script/bootstrap` runs in that step, it installs nothing beyond system
packages, pipenv and its dependencies, and the packages from
`Pipfile.lock`. It installs pipenv and its dependencies by hash, with
`pip install --require-hashes`. It installs the packages from
`Pipfile.lock` with `pipenv install --deploy`, plus `--dev` when the tests
need the development packages. It installs no Python there, because the
tests use the base image's. The tests run after `ARG CACHEBUST`, and every
other stage installs its Python packages after it, as in the Python `test`
phase example in `REPO_POLICIES.md`
`script/bootstrap` runs in that step, it installs nothing beyond these:
- system packages
- pipenv and its dependencies
- the packages from `Pipfile.lock`
It creates a virtual environment at `$HOME/.pipenv` with
`python3 -m venv` when it is missing, then uses that environment's `pip`
and `pipenv`. On a host, it first installs the distribution's python and
venv packages when they are missing. It installs pipenv and its
dependencies by hash, with `pip install --require-hashes`. It installs
the packages from `Pipfile.lock` with `pipenv install --deploy`, plus
`--dev` when the tests need the development packages. It sets
`PIP_ONLY_BINARY=:all:` for both installs, so a package with no wheel
for the image stops the install. In that step it installs no Python,
because a step above it already did. `Pipfile` names no
`python_version`, because `--deploy` stops when the distribution's
Python differs from it. The tests run after `ARG CACHEBUST`, and every
other stage installs its Python packages after it, as in the Python
`test` phase example in `REPO_POLICIES.md`
- Server: the final stage builds and runs the application
- Non-server: the final stage brings up the dev environment
- Image pinned by sha256 hash with version/date comment
- [ ] Gitea Actions workflow at `.gitea/workflows/check.yml` that runs
`script/cibuild` on push, checks out with `persist-credentials: false` and
with `fetch-depth: 0` (which fetches the tags `git describe` needs),
@@ -159,7 +179,9 @@ Template files can be fetched from:
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.golangci.yml` and,
in the same commit, set the lint phase digest to the one named in the
`.golangci.yml` paragraph of `REPO_POLICIES.md`)
- [ ] JS: `yarn init`, `yarn add --dev prettier`
- [ ] JS: `yarn init`, `yarn add --dev prettier`. `script/bootstrap`
installs no yarn until `yarn.lock` exists, so use yarn 1 from a
distribution that packages it, such as Alpine (`apk add nodejs yarn`).
- [ ] Python: `pyproject.toml`
## Configure script/ Entrypoints and Makefile
@@ -170,14 +192,21 @@ are thin shims calling them. Model scripts:
- [ ] scripts are POSIX sh (`#!/bin/sh`, `set -eu`, no bashisms) so they run on
alpine images without bash
- [ ] `script/bootstrap` / `make bootstrap` — installs all dependencies,
idempotently, assuming nothing (pkg manager detection nix/apt/brew/apk;
node used if its major version is the pinned one, else pinned version via
nvm from a hash-verified archive, with yarn and the packages installed
under it; pinned yarn via corepack); a non-server repo's development
environment stage runs it instead of inline installs; a gate phase or the
build stage installs what its base image lacks either inline or by running
it
- [ ] `script/bootstrap` / `make bootstrap` installs all dependencies,
idempotently, and assumes nothing is present:
- It detects the package manager: nix, apt, brew or apk.
- It installs node, yarn and the node packages only when `yarn.lock` is in
the tree. node and yarn come from that package manager when they are
missing.
- With `yarn.lock` in the tree, it stops with a one-line message when yarn's
major version is not 1.
- A non-server repo's development environment stage runs it, with only
system packages installed inline above it.
- A gate phase or the build stage installs what its base image lacks, either
inline or by running it.
- [ ] A `Dockerfile` stage copies `yarn.lock` before it runs `script/bootstrap`
only if that stage needs yarn. A stage that needs yarn starts from an
image whose distribution packages yarn 1, such as Alpine.
- [ ] `script/setup` / `make setup` — readies a fresh clone: runs `bootstrap`,
then `install-precommit`, plus repo-specific init
- [ ] `script/test` / `make test` —
@@ -207,13 +236,10 @@ are thin shims calling them. Model scripts:
from `script/projectname`, each assigned on its own line before the build.
The bootstrap is required: CI checks out and runs this alone, and
`script/fmt-check` runs the formatter on the host.
- [ ] `script/fmt` and `script/fmt-check` pick their `yarn` with the same test
as `script/bootstrap`: the `yarn` on `PATH` when the node on `PATH` has
the pinned major version, and otherwise yarn under the pinned node, with
nvm loaded through `$HOME/.nvm/nvm.sh`. `script/bootstrap` leaves the node
and yarn it installs under nvm off the `PATH` of the shell that called it,
so a bare `yarn` exits 127 on a runner carrying nothing but docker and
git, or runs under another node.
- [ ] `script/bootstrap`, `script/fmt` and `script/fmt-check` run yarn by
whichever name is present: `yarn`, or `yarnpkg` on Debian and Ubuntu. With
neither name on `PATH`, `script/fmt` and `script/fmt-check` print one line
saying to run `script/bootstrap`, and exit 1.
- [ ] No `docker build` in `script/` leaves a dangling image behind:
`script/lint` and `script/test` write no image, and `script/docker` and
`script/cibuild` tag theirs
@@ -228,11 +254,12 @@ are thin shims calling them. Model scripts:
- [ ] `make check` passes
- [ ] `make docker` succeeds
- [ ] `script/cibuild` succeeds in a fresh clone on a host carrying what CI has
(the runner image `docker.gitea.com/runner-images:ubuntu-latest`: docker,
git and node 24, with no yarn on `PATH`), and demonstrably executed the
checks — a sub-second build, or `CACHED` on a step that lints or tests,
one of the `RUN` steps below `ARG CACHEBUST`, means nothing ran. Above
- [ ] `script/cibuild` succeeds in a fresh clone on a host carrying what CI has.
That host is the runner image
`docker.gitea.com/runner-images:ubuntu-latest`, which has docker, git and
node 24. Take any yarn off `PATH` first. The run must show that the checks
executed. A sub-second build means nothing ran. So does `CACHED` on a step
that lints or tests, one of the `RUN` steps below `ARG CACHEBUST`. Above
that line, these steps are expected to show `CACHED`:
- the steps that install system packages
- the steps that download Go's modules
@@ -246,8 +273,12 @@ are thin shims calling them. Model scripts:
- [ ] No mutable image/package references. The one exception is a package from
the distribution's own package index, which needs no version and no hash.
Pulling that index is also allowed. In a `Dockerfile`, the distribution is
the pinned base image's. On a host, it is the host's. A package source
the pinned base image's. On a host, it is the host's, and the host's
package manager stands for it, nix and brew included. A package source
that a `Dockerfile` adds itself does not get this exception
- [ ] node, yarn, npm, python, pip and venv come from the distribution wherever
possible, never from nvm, pyenv, corepack or `npm install -g yarn`. If the
distribution's version is too old, use a newer distribution
- [ ] No unnecessary files in repo root
- [ ] All dates written as YYYY-MM-DD
+547 -357
View File
File diff suppressed because it is too large Load Diff
+30 -96
View File
@@ -1,24 +1,15 @@
#!/bin/sh
# script/bootstrap: install all dependencies needed to build and develop
# this repo. Idempotent: every install is guarded by a check so already
# installed tools are skipped. Base tooling comes from nix, apt, brew,
# or apk (detected in that order); assumes nothing is present. The
# installed node is used only when its major version is the pinned one;
# otherwise (node missing, or another major) the pinned version is
# installed via nvm (installing nvm itself first, from a hash-verified
# release archive, never curl | sh), and yarn and the packages are
# installed under it.
# installed tools are skipped. Base tools, node and yarn come from nix,
# apt, brew, or apk (detected in that order), with no version and no
# hash, because the distribution is trusted. It installs node, yarn and
# the node packages only when yarn.lock is in the tree. Assumes nothing
# is present.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Pinned versions, 2026-07-06
NODE_VERSION="22.17.0"
NVM_VERSION="0.40.3"
# sha256 of https://github.com/nvm-sh/nvm/archive/refs/tags/v0.40.3.tar.gz
NVM_SHA256="5f4d6aaa04a177dc93c985e31dbc411ab6b8c6e1e21d8015dbc1372625fcd1d0"
YARN_VERSION="1.22.22"
PKGMGR=""
SUDO=""
APT_UPDATED=""
@@ -67,99 +58,42 @@ missing() {
! command -v "$1" >/dev/null 2>&1
}
# verify_sha256 <file> <expected-hash>
verify_sha256() {
if command -v sha256sum >/dev/null 2>&1; then
actual="$(sha256sum "$1" | cut -d' ' -f1)"
# Debian and Ubuntu name the yarn command yarnpkg.
run_yarn() {
if command -v yarn >/dev/null 2>&1; then
yarn "$@"
elif command -v yarnpkg >/dev/null 2>&1; then
yarnpkg "$@"
else
actual="$(shasum -a 256 "$1" | cut -d' ' -f1)"
fi
if [ "$actual" != "$2" ]; then
echo "bootstrap: sha256 mismatch for $1" >&2
echo " expected: $2" >&2
echo " actual: $actual" >&2
echo "no yarn; run script/bootstrap first" >&2
exit 1
fi
}
# nvm is a bash script; run a command in a bash with nvm loaded
nvm_sh() {
bash -c ". \"\$HOME/.nvm/nvm.sh\" && $*"
}
ensure_nvm() {
[ -s "$HOME/.nvm/nvm.sh" ] && return 0
# nvm prerequisites; nvm itself requires bash
if missing bash; then pkg_install bash bash bash bash; fi
if missing curl; then pkg_install curl curl curl curl; fi
if missing git; then pkg_install git git git git; fi
tmp="$(mktemp -d)"
curl -fsSL -o "$tmp/nvm.tar.gz" \
"https://github.com/nvm-sh/nvm/archive/refs/tags/v${NVM_VERSION}.tar.gz"
verify_sha256 "$tmp/nvm.tar.gz" "$NVM_SHA256"
mkdir -p "$HOME/.nvm"
tar -xzf "$tmp/nvm.tar.gz" -C "$HOME/.nvm" --strip-components=1
rm -rf "$tmp"
}
# True when the node on PATH has the pinned major version. Only the
# major is compared: the Dockerfile stages start from a node 22 alpine
# image whose exact version is not the pin, and nvm cannot install a
# prebuilt node on alpine. Another major is not used: the pinned yarn 1
# prints a deprecation warning under node 24. script/fmt and
# script/fmt-check carry this function unchanged, to run the yarn
# installed here.
node_is_pinned_major() {
if ! command -v node >/dev/null 2>&1; then return 1; fi
installed="$(node --version)"
installed="${installed#v}"
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
}
ensure_node() {
if node_is_pinned_major; then
echo "bootstrap: using node $(node --version)"
return 0
fi
ensure_nvm
nvm_sh "nvm install $NODE_VERSION"
nvm_sh "nvm use $NODE_VERSION >/dev/null && \
echo \"bootstrap: using node \$(node --version)\""
}
ensure_yarn() {
if ! node_is_pinned_major; then
nvm_sh "nvm use $NODE_VERSION >/dev/null && corepack enable && \
corepack prepare yarn@$YARN_VERSION --activate"
return 0
fi
if ! missing yarn; then return 0; fi
if ! missing corepack; then
corepack enable
corepack prepare "yarn@$YARN_VERSION" --activate
else
npm install -g "yarn@$YARN_VERSION"
fi
}
install_js_deps() {
if node_is_pinned_major; then
yarn install --frozen-lockfile
else
nvm_sh "nvm use $NODE_VERSION >/dev/null && cd \"$ROOT\" && \
yarn install --frozen-lockfile"
fi
}
main() {
cd "$ROOT"
if missing make; then pkg_install gnumake make make make; fi
if missing git; then pkg_install git git git git; fi
ensure_node
ensure_yarn
install_js_deps
# A Dockerfile stage that needs no yarn copies no yarn.lock.
if [ -f yarn.lock ]; then
if missing node; then pkg_install nodejs nodejs node nodejs; fi
if missing yarn && missing yarnpkg; then
pkg_install yarn yarnpkg yarn yarn
fi
# Debian 13 and Ubuntu 26.04 package yarn 4, which refuses a yarn 1
# yarn.lock, or rewrites it in its own format.
yarn_version="$(run_yarn --version)"
if [ "${yarn_version%%.*}" != "1" ]; then
echo "bootstrap: yarn $yarn_version refuses this yarn 1 yarn.lock;" \
"use a distribution that packages yarn 1" >&2
exit 1
fi
run_yarn install --frozen-lockfile
fi
echo "bootstrap complete"
}
+7 -24
View File
@@ -4,33 +4,16 @@ set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Must match the pin in script/bootstrap.
NODE_VERSION="22.17.0"
# True when the node on PATH has the pinned major version: the same
# function as in script/bootstrap, which uses it to decide where it
# installs yarn.
node_is_pinned_major() {
if ! command -v node >/dev/null 2>&1; then return 1; fi
installed="$(node --version)"
installed="${installed#v}"
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
}
# Run the yarn script/bootstrap installed: the yarn on PATH when the
# node on PATH has the pinned major version, and otherwise yarn under
# the pinned node in nvm, which bootstrap leaves off the PATH of the
# shell that called it. nvm is a bash script, hence the subshell.
# Debian and Ubuntu name the yarn command yarnpkg.
run_yarn() {
if node_is_pinned_major && command -v yarn >/dev/null 2>&1; then
exec yarn "$@"
fi
if node_is_pinned_major || [ ! -s "$HOME/.nvm/nvm.sh" ]; then
echo "fmt: no yarn; run script/bootstrap first" >&2
if command -v yarn >/dev/null 2>&1; then
yarn "$@"
elif command -v yarnpkg >/dev/null 2>&1; then
yarnpkg "$@"
else
echo "no yarn; run script/bootstrap first" >&2
exit 1
fi
exec bash -c '. "$HOME/.nvm/nvm.sh" && nvm use "$1" >/dev/null &&
shift && exec yarn "$@"' bash "$NODE_VERSION" "$@"
}
main() {
+7 -24
View File
@@ -4,33 +4,16 @@ set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Must match the pin in script/bootstrap.
NODE_VERSION="22.17.0"
# True when the node on PATH has the pinned major version: the same
# function as in script/bootstrap, which uses it to decide where it
# installs yarn.
node_is_pinned_major() {
if ! command -v node >/dev/null 2>&1; then return 1; fi
installed="$(node --version)"
installed="${installed#v}"
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
}
# Run the yarn script/bootstrap installed: the yarn on PATH when the
# node on PATH has the pinned major version, and otherwise yarn under
# the pinned node in nvm, which bootstrap leaves off the PATH of the
# shell that called it. nvm is a bash script, hence the subshell.
# Debian and Ubuntu name the yarn command yarnpkg.
run_yarn() {
if node_is_pinned_major && command -v yarn >/dev/null 2>&1; then
exec yarn "$@"
fi
if node_is_pinned_major || [ ! -s "$HOME/.nvm/nvm.sh" ]; then
echo "fmt-check: no yarn; run script/bootstrap first" >&2
if command -v yarn >/dev/null 2>&1; then
yarn "$@"
elif command -v yarnpkg >/dev/null 2>&1; then
yarnpkg "$@"
else
echo "no yarn; run script/bootstrap first" >&2
exit 1
fi
exec bash -c '. "$HOME/.nvm/nvm.sh" && nvm use "$1" >/dev/null &&
shift && exec yarn "$@"' bash "$NODE_VERSION" "$@"
}
main() {