Moves whole bullets, and whole paragraphs inside them, into the order of the sections that the next commits add. No line changes: git show --color-moved shows every changed line as moved, and the sorted lines of the file are identical before and after. Paragraphs that later join another section sit under the nearest bullet until that section's commit. Model: opus-5-5
This commit is contained in:
1 parent
48c7faf557
commit
4cce747e00
1 file changed
+360
-360
+360
-360
@@ -16,10 +16,6 @@ style conventions are in separate documents:
|
||||
|
||||
---
|
||||
|
||||
- Cross-project documentation (such as this file) must include
|
||||
`last_modified: YYYY-MM-DD` in the YAML front matter so it can be kept in sync
|
||||
with the authoritative source as policies evolve.
|
||||
|
||||
- **External references must be pinned by cryptographic hash.** This is the
|
||||
single most important rule in this document. It covers what we download and
|
||||
install ourselves in scripts and Dockerfiles:
|
||||
@@ -83,6 +79,30 @@ style conventions are in separate documents:
|
||||
- the npm registry
|
||||
- `go get`
|
||||
|
||||
- When pinning images or packages by hash, add a comment above the reference
|
||||
with the version and date (YYYY-MM-DD).
|
||||
|
||||
- Never commit secrets. `.env` files, credentials, API keys, and private keys
|
||||
must be in `.gitignore`. No exceptions.
|
||||
|
||||
- Make all changes on a feature branch. You can do whatever you want on a
|
||||
feature branch.
|
||||
|
||||
- Never force-push to `main`.
|
||||
|
||||
- Never use `git add -A` or `git add .`. Always stage files explicitly by name.
|
||||
|
||||
- **No build artifacts in version control.** Code-derived data must never be
|
||||
committed to the repository if it can be avoided. Code-derived data includes:
|
||||
- compiled bundles
|
||||
- minified output
|
||||
- generated assets
|
||||
|
||||
The build process (e.g. Dockerfile, Makefile) should generate these at build
|
||||
time. Notable exception: Go protobuf generated files (`.pb.go`) ARE
|
||||
committed because repos need to work with `go get`, which downloads code but
|
||||
does not execute code generation.
|
||||
|
||||
- Every repo with software must have a root `Makefile` with these targets:
|
||||
- `make bootstrap`
|
||||
- `make setup`
|
||||
@@ -113,22 +133,137 @@ style conventions are in separate documents:
|
||||
- `script/docker`
|
||||
- `script/update-policy`
|
||||
|
||||
The scripts must be POSIX sh (`#!/bin/sh`, `set -eu`, no bashisms) so they
|
||||
run in minimal containers; alpine images, for example, have no bash. Locate
|
||||
the repo root with `$(cd "$(dirname "$0")/.." && pwd -P)`, and `cd` there
|
||||
before acting.
|
||||
Five further scripts are our own extensions to the standard:
|
||||
- `script/check` runs `script/test`, `script/lint` and `script/fmt-check`.
|
||||
- `script/precommit` is what the git pre-commit hook runs, and it calls
|
||||
`script/check`.
|
||||
- `script/install-precommit` installs the git pre-commit hook. The
|
||||
`make hooks` target shims to it.
|
||||
- `script/update-policy` fetches the vendored policy files from the
|
||||
`prompts` repo, commits them and pushes. The `make update-policy` target
|
||||
shims to it.
|
||||
- `script/projectname` (literally that filename) simply outputs the
|
||||
project's name.
|
||||
|
||||
Scripts that need the name call `script/projectname`, so those scripts stay
|
||||
byte-identical across all repos. `script/docker`, for example, assembles its
|
||||
image tag from it. Repo-type-specific pre-commit extras belong in
|
||||
`script/precommit`, not in the hook itself; `go mod tidy` verification in Go
|
||||
repos is one. Model scripts are at
|
||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The
|
||||
README must document the provided scripts in an **Entrypoints** section (see
|
||||
the README requirements below).
|
||||
|
||||
From the standard's canonical set we use `bootstrap`, `setup`, `test`, and
|
||||
`cibuild`. `setup` makes the repo ready for development after a fresh clone:
|
||||
it runs `bootstrap`, then `install-precommit`, plus any repo-specific
|
||||
initialization.
|
||||
|
||||
The scripts must be POSIX sh (`#!/bin/sh`, `set -eu`, no bashisms) so they
|
||||
run in minimal containers; alpine images, for example, have no bash. Locate
|
||||
the repo root with `$(cd "$(dirname "$0")/.." && pwd -P)`, and `cd` there
|
||||
before acting.
|
||||
|
||||
- Pre-commit hook: runs `script/precommit`, which calls `script/check`. If local
|
||||
testing is not possible in the repo, `script/precommit` may skip `script/test`
|
||||
and run only `script/lint` and `script/fmt-check`. `script/install-precommit`
|
||||
installs the hook; the Makefile must provide a `make hooks` target that shims
|
||||
to it.
|
||||
|
||||
- Run the repo's own entrypoints, never the underlying tools directly. The
|
||||
`script/` scripts are the single source of truth for how each operation runs.
|
||||
Calling them directly is fine, and so is calling the Makefile targets. The
|
||||
Makefile targets must still exist, as the thin shims described above.
|
||||
|
||||
An agent whose work is confined to one package may run `go test` directly on
|
||||
that package's suite, or on individual tests in it, while iterating. This is
|
||||
the only exception to this rule. `go build`, `go vet` and the linter still
|
||||
go through the entrypoints. A run while iterating is not a gate, so the full
|
||||
`make check` is still required before every push.
|
||||
|
||||
- The Makefile is authoritative documentation for how the repo is used. Beyond
|
||||
the required targets above, it should have targets for every common operation:
|
||||
- running a local development server (`make run`, `make dev`)
|
||||
- re-initializing or migrating the database (`make db-reset`,
|
||||
`make migrate`)
|
||||
- building artifacts (`make build`)
|
||||
- generating code
|
||||
- seeding data
|
||||
- anything else a developer would do regularly
|
||||
|
||||
If someone checks out the repo and types `make<tab>`, they should see every
|
||||
meaningful operation available. A new contributor should be able to
|
||||
understand the entire development workflow by reading the Makefile.
|
||||
|
||||
- A repo updates its policy files with one command: `make update-policy`. It
|
||||
runs `script/update-policy`, which fetches each vendored file the repo carries
|
||||
from the `prompts` repo, commits them and pushes. It uses the head of `main`,
|
||||
or the commit given by a full hash: `script/update-policy <hash>`. The list of
|
||||
vendored files is at the top of that script. The script replaces each file and
|
||||
keeps only the repo's own entries:
|
||||
- in `.gitignore`, `.editorconfig` and `.dockerignore`: the lines below the
|
||||
comment that ends the canonical file
|
||||
- in `.golangci.yml`: the entries the repo added to the `deny` list of the
|
||||
`test-support` depguard rule
|
||||
|
||||
- All repos with software must have tests that run via the platform-standard
|
||||
test framework (`go test`, `pytest`, `jest`/`vitest`, etc.). If no meaningful
|
||||
tests exist yet, add the most minimal test possible: importing the module
|
||||
under test to verify it compiles/parses, for example. There is no excuse for
|
||||
`make test` to be a no-op.
|
||||
|
||||
- `make test` must complete in under 60 seconds. That is the hard cap, and a
|
||||
suite that exceeds it fails. Under 20 seconds is the target. A suite between
|
||||
20 and 60 seconds is still green, but the overage must be filed as an
|
||||
improvement bug against that repo. Add a 90-second timeout to the test
|
||||
invocation (`go test -timeout 90s`). The backstop deliberately sits above the
|
||||
hard cap so that it catches a genuinely hung test rather than a merely slow
|
||||
one. A hung package uses the backstop once, because a first run that timed out
|
||||
gets no rerun. `-timeout` applies to each package, so several hung packages
|
||||
can each use it.
|
||||
|
||||
- Docker builds must complete in under 5 minutes. A hung package uses the
|
||||
90-second test backstop once, because a first run that timed out gets no
|
||||
rerun.
|
||||
|
||||
- `make check` must not modify any files in the repo. Tests may use temporary
|
||||
directories.
|
||||
|
||||
- `main` must always pass `make check`, no exceptions.
|
||||
|
||||
- **The test command should use the conditional verbose rerun pattern.** Run
|
||||
tests without `-v` (verbose) first. If tests fail, automatically rerun with
|
||||
`-v` to show full output. This keeps CI logs and `docker build` output clean
|
||||
on success, with just package/suite summaries. On failure it provides full
|
||||
diagnostic detail: every test case, every assertion. A first run that timed
|
||||
out gets no rerun, because the rerun would only wait for the same hang. The
|
||||
command lives in the `test` phase of the `Dockerfile`, since `script/test`
|
||||
builds that phase. The Makefile form below is the same pattern for any
|
||||
repo-local invocation:
|
||||
|
||||
```makefile
|
||||
test:
|
||||
@<test-command> || \
|
||||
{ echo "--- Rerunning with -v for details ---"; \
|
||||
<test-command-with-v>; exit 1; }
|
||||
```
|
||||
|
||||
The `exit 1` ensures the target always fails after a rerun. The first run
|
||||
already proved the tests are broken, so the build must not pass even if a
|
||||
flaky test happens to succeed on the second attempt. The rerun exists solely
|
||||
for diagnostic output.
|
||||
|
||||
`script/bootstrap` installs all dependencies idempotently and assumes
|
||||
nothing is present. Base tools, node and yarn come from nix, apt, brew, or
|
||||
apk, detected in that order; apt runs noninteractive. Each one is installed
|
||||
only when it is missing. A tool from outside the distribution is never
|
||||
installed as "latest" or "lts"; it is always an exact version.
|
||||
|
||||
A tool from the distribution has no pin, so `script/bootstrap` installs it
|
||||
when it is missing. make, git, node and yarn are such tools.
|
||||
|
||||
- Use `yarn`, not `npm`.
|
||||
|
||||
`script/bootstrap` installs node, yarn and the node packages only when
|
||||
`yarn.lock` is in the tree. With no `yarn.lock`, it installs no node and no
|
||||
yarn. It installs the node packages with `yarn install --immutable`. That
|
||||
@@ -178,59 +313,38 @@ style conventions are in separate documents:
|
||||
`examples/python-test-phase.Dockerfile` need no yarn and copy no
|
||||
`yarn.lock`, so `script/bootstrap` installs no node and no yarn there.
|
||||
|
||||
`script/cibuild` runs the CI build, and the Gitea workflow calls it. The
|
||||
script changes to the repo root, runs `script/bootstrap`, runs
|
||||
`script/check`, and builds the image with the version. **`script/cibuild`
|
||||
runs `script/bootstrap` first**, because the workflow checks out the repo
|
||||
and runs nothing else, while `script/fmt-check` runs the formatter on the
|
||||
host. On a pristine checkout with nothing installed, the run dies there,
|
||||
after the containerised gates have passed.
|
||||
- **`script/bootstrap` installs a pinned tool by comparing versions, never by
|
||||
testing presence.** An `if ! command -v <tool>; then install; fi` guard tests
|
||||
`PATH` only. On an already-provisioned machine the pin is then inert, and a
|
||||
version bump is a silent no-op. The Dockerfile, installing into a clean image,
|
||||
gets the pinned version, so a local `make check` and `make docker` can
|
||||
disagree about what the tool even is. The canonical form:
|
||||
- compares the installed version against the pin over the **whole** version
|
||||
token. A parser that stops at the first `-` reports `2.12.2` for a host
|
||||
running `2.12.2-rc1` and skips the install.
|
||||
- treats `--version` output as a mismatch when it is any of these:
|
||||
- absent
|
||||
- non-zero
|
||||
- empty
|
||||
- unrecognised
|
||||
|
||||
Five further scripts are our own extensions to the standard:
|
||||
- `script/check` runs `script/test`, `script/lint` and `script/fmt-check`.
|
||||
- `script/precommit` is what the git pre-commit hook runs, and it calls
|
||||
`script/check`.
|
||||
- `script/install-precommit` installs the git pre-commit hook. The
|
||||
`make hooks` target shims to it.
|
||||
- `script/update-policy` fetches the vendored policy files from the
|
||||
`prompts` repo, commits them and pushes. The `make update-policy` target
|
||||
shims to it.
|
||||
- `script/projectname` (literally that filename) simply outputs the
|
||||
project's name.
|
||||
So a failure causes a redundant install, never a skipped one.
|
||||
|
||||
Scripts that need the name call `script/projectname`, so those scripts stay
|
||||
byte-identical across all repos. `script/docker`, for example, assembles its
|
||||
image tag from it. Repo-type-specific pre-commit extras belong in
|
||||
`script/precommit`, not in the hook itself; `go mod tidy` verification in Go
|
||||
repos is one. Model scripts are at
|
||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The
|
||||
README must document the provided scripts in an **Entrypoints** section (see
|
||||
the README requirements below).
|
||||
- after installing, re-resolves the binary the way callers do, with
|
||||
`hash -r` and then through `PATH`, not through the directory the installer
|
||||
wrote to. A shadowing binary can hide an install, which then succeeds
|
||||
while no caller sees any change. In that case it fails and names the
|
||||
resolved path.
|
||||
- is actually called, and prints the version on both success paths. A
|
||||
function defined and never invoked has the same exit status and the same
|
||||
empty output as one that worked.
|
||||
|
||||
- Run the repo's own entrypoints, never the underlying tools directly. The
|
||||
`script/` scripts are the single source of truth for how each operation runs.
|
||||
Calling them directly is fine, and so is calling the Makefile targets. The
|
||||
Makefile targets must still exist, as the thin shims described above.
|
||||
Keep it POSIX sh: no arrays, no `[[`, no `grep -P`.
|
||||
|
||||
An agent whose work is confined to one package may run `go test` directly on
|
||||
that package's suite, or on individual tests in it, while iterating. This is
|
||||
the only exception to this rule. `go build`, `go vet` and the linter still
|
||||
go through the entrypoints. A run while iterating is not a gate, so the full
|
||||
`make check` is still required before every push.
|
||||
|
||||
- The Makefile is authoritative documentation for how the repo is used. Beyond
|
||||
the required targets above, it should have targets for every common operation:
|
||||
- running a local development server (`make run`, `make dev`)
|
||||
- re-initializing or migrating the database (`make db-reset`,
|
||||
`make migrate`)
|
||||
- building artifacts (`make build`)
|
||||
- generating code
|
||||
- seeding data
|
||||
- anything else a developer would do regularly
|
||||
|
||||
If someone checks out the repo and types `make<tab>`, they should see every
|
||||
meaningful operation available. A new contributor should be able to
|
||||
understand the entire development workflow by reading the Makefile.
|
||||
A Go tool a repo needs on the host is installed with `go install` pinned to
|
||||
a commit hash (`go install <package>@<commit hash>`). It is never tracked as
|
||||
a `go.mod` tool dependency or through a `tools.go` file, either of which
|
||||
pulls the tool's own dependencies into the repo's `go.mod` and `go.sum`.
|
||||
|
||||
- Every repo should have a `Dockerfile`, and it carries the repo's gates: a
|
||||
`lint` phase and a `test` phase. The final stage depends on both, so the image
|
||||
@@ -372,42 +486,46 @@ style conventions are in separate documents:
|
||||
base image or the install line changes. The first lines of a lint phase
|
||||
that installs system libraries are at
|
||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/examples/go-lint-system-libraries.Dockerfile`.
|
||||
- `.dockerignore` lets `.git` into the build context. It keeps out every git
|
||||
`config` at any depth (`**/.git/config`, `**/.git/modules/**/config`):
|
||||
- the repository's own
|
||||
- each submodule's under `.git/modules/`
|
||||
- that of a submodule keeping its own `.git` directory
|
||||
|
||||
`git describe` does not need them, and each can hold a credential: a
|
||||
password in a remote URL, or the token the CI checkout step stores
|
||||
there. Some submodule names have a `config` segment, as in `config`,
|
||||
`deploy/config` or `config/lib`. Such a submodule loses its whole git
|
||||
directory to `**/.git/modules/**/config`, and Go's version stamping then
|
||||
fails the build. Give it a name without that segment
|
||||
(`git submodule add --name`). The stage that compiles has `git`: the
|
||||
Debian Go image has it, and an alpine one needs
|
||||
`apk add --no-cache git`. That stage takes the version from the
|
||||
`VERSION` build argument when one is given, otherwise from
|
||||
`git describe --tags --always`. That gives:
|
||||
- the tag, on a tagged commit
|
||||
- the tag, the number of commits since it and the short commit
|
||||
(`v1.2.3-4-gabc1234`), on a later commit
|
||||
- the short commit, when no tag is reachable
|
||||
Go example:
|
||||
|
||||
The stage that compiles also marks its working directory safe for git
|
||||
(`git config --system --add safe.directory /src`). A context sent as a
|
||||
tar stream keeps the sender's file owners, and git refuses a checkout
|
||||
owned by another user, so the version would come out empty.
|
||||
`ARG VERSION` has no default. The build fails if the context carries
|
||||
`.git` and the version still comes out empty, `dev` or `unknown`. A
|
||||
plain `docker build .` with no build arguments must succeed; a
|
||||
Dockerfile that refuses an empty build argument drops that refusal and
|
||||
keeps the argument. A checkout whose `.git` is a file is the exception:
|
||||
a linked worktree, or a repository checked out as a submodule. That file
|
||||
points to a git directory outside the build context, so the build cannot
|
||||
read the version and a plain `docker build .` fails. Pass the version
|
||||
with `--build-arg VERSION=...`, as `script/docker` and `script/cibuild`
|
||||
already do.
|
||||
```makefile
|
||||
test:
|
||||
@status=0; \
|
||||
output=$$(go test -count=1 -timeout 90s -race -cover ./... 2>&1) || status=$$?; \
|
||||
printf '%s\n' "$$output"; \
|
||||
test "$$status" -eq 0 && exit 0; \
|
||||
case "$$output" in \
|
||||
*"panic: test timed out"*|*"*** Test killed"*) exit 1 ;; \
|
||||
esac; \
|
||||
echo "--- Rerunning with -v for details ---"; \
|
||||
go test -count=1 -timeout 90s -race -v ./...; \
|
||||
exit 1
|
||||
```
|
||||
|
||||
The first run's output is kept so that the recipe can look in it for what Go
|
||||
prints for a hang:
|
||||
- `panic: test timed out` means the test's own timer fired.
|
||||
- `*** Test killed` means `go test` stopped a test that ignored that timer,
|
||||
about a minute later.
|
||||
|
||||
Go prints the goroutine stacks with either line, as long as the test can
|
||||
still respond. A rerun would only wait for the same hang, so the recipe
|
||||
exits 1. A failure with neither line in its output reruns with `-v`. The
|
||||
kept output prints when the first run ends, not while it runs.
|
||||
|
||||
`-count=1` is required on both invocations. It defeats Go's test _result_
|
||||
cache, so neither run can report a stored pass in place of running the
|
||||
tests. It leaves the build cache alone, so it costs the runtime of the suite
|
||||
and no recompilation.
|
||||
|
||||
That cache is Go's own, separate from Docker's layer cache. Go stores a
|
||||
passing result in its cache directory (`GOCACHE`). When the same tests run
|
||||
again on unchanged code, it prints that result, marked `(cached)`, without
|
||||
running them. The directory lasts from one run to the next on a developer's
|
||||
machine, where this target runs. The directory also lasts in the `test`
|
||||
phase of the `Dockerfile`, which mounts it to keep compiled packages between
|
||||
check builds, so the phase's `go test` runs carry `-count=1` too.
|
||||
|
||||
- **A Python `test` phase that installs its packages from `Pipfile.lock` does it
|
||||
before `COPY . .`, so check builds keep them.** The steps are:
|
||||
@@ -494,6 +612,88 @@ style conventions are in separate documents:
|
||||
example is at
|
||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/examples/python-test-phase.Dockerfile`.
|
||||
|
||||
Python example:
|
||||
|
||||
```makefile
|
||||
test:
|
||||
@timeout 90 sh -c 'pipenv run pytest || { \
|
||||
echo "--- Rerunning with -v for details ---"; \
|
||||
pipenv run pytest -v; exit 1; }'
|
||||
```
|
||||
|
||||
Both pytest runs share that one 90-second timeout, as in the `prompts`
|
||||
repo's `examples/python-test-phase.Dockerfile`, so a hang uses it once and
|
||||
gets no rerun. Ctrl-C does not stop a running `make test`, which goes on
|
||||
until the tests end or the 90 seconds run out. On a Mac without `timeout`,
|
||||
the target fails until it is installed. On a host, the target needs
|
||||
`$HOME/.pipenv/bin` on `PATH`.
|
||||
|
||||
- **A plain `docker build .` of a clone stamps the version that
|
||||
`git describe --tags --always` gives.** The version is derived from the `.git`
|
||||
in the build context, as the `prompts` repo's `examples/go.Dockerfile` shows.
|
||||
Without its failure check, a missing `git` or an unreadable checkout would
|
||||
leave `-X main.Version=` empty, and the build would still exit 0.
|
||||
`script/docker` and `script/cibuild` pass the version they compute on the
|
||||
host; it takes precedence. They do this byte-identically across repos:
|
||||
|
||||
```sh
|
||||
# The version and the tag each get their own line: a failing command
|
||||
# substitution inside an argument does not trip `set -e`, so the inline
|
||||
# form degrades to an empty constant.
|
||||
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
||||
[ -n "$version" ] || version="unknown"
|
||||
tag="$(script/projectname)"
|
||||
docker build --build-arg VERSION="$version" -t "$tag" .
|
||||
```
|
||||
|
||||
`--always` makes an untagged repo yield an abbreviated commit hash rather
|
||||
than failing. The `[ -n "$version" ]` line is the single place the fallback
|
||||
is applied. That line is a live check that fires on a build from an export
|
||||
with no `.git`, and on a repository with no commits yet. Do not fold it into
|
||||
the substitution as `|| echo unknown`, which makes the guard unreachable.
|
||||
The Dockerfile's side is `ARG VERSION` in the stage that compiles, declared
|
||||
there because `ARG` is stage-scoped. Passing `VERSION` to a repo whose
|
||||
Dockerfile declares no such `ARG` is ignored and costs nothing, which is why
|
||||
the scripts stay byte-identical. One consequence for CI: the standard
|
||||
checkout action clones shallow and fetches no tags, so the canonical
|
||||
`.gitea/workflows/check.yml` sets `fetch-depth: 0` on its checkout step.
|
||||
- `.dockerignore` lets `.git` into the build context. It keeps out every git
|
||||
`config` at any depth (`**/.git/config`, `**/.git/modules/**/config`):
|
||||
- the repository's own
|
||||
- each submodule's under `.git/modules/`
|
||||
- that of a submodule keeping its own `.git` directory
|
||||
|
||||
`git describe` does not need them, and each can hold a credential: a
|
||||
password in a remote URL, or the token the CI checkout step stores
|
||||
there. Some submodule names have a `config` segment, as in `config`,
|
||||
`deploy/config` or `config/lib`. Such a submodule loses its whole git
|
||||
directory to `**/.git/modules/**/config`, and Go's version stamping then
|
||||
fails the build. Give it a name without that segment
|
||||
(`git submodule add --name`). The stage that compiles has `git`: the
|
||||
Debian Go image has it, and an alpine one needs
|
||||
`apk add --no-cache git`. That stage takes the version from the
|
||||
`VERSION` build argument when one is given, otherwise from
|
||||
`git describe --tags --always`. That gives:
|
||||
- the tag, on a tagged commit
|
||||
- the tag, the number of commits since it and the short commit
|
||||
(`v1.2.3-4-gabc1234`), on a later commit
|
||||
- the short commit, when no tag is reachable
|
||||
|
||||
The stage that compiles also marks its working directory safe for git
|
||||
(`git config --system --add safe.directory /src`). A context sent as a
|
||||
tar stream keeps the sender's file owners, and git refuses a checkout
|
||||
owned by another user, so the version would come out empty.
|
||||
`ARG VERSION` has no default. The build fails if the context carries
|
||||
`.git` and the version still comes out empty, `dev` or `unknown`. A
|
||||
plain `docker build .` with no build arguments must succeed; a
|
||||
Dockerfile that refuses an empty build argument drops that refusal and
|
||||
keeps the argument. A checkout whose `.git` is a file is the exception:
|
||||
a linked worktree, or a repository checked out as a submodule. That file
|
||||
points to a git directory outside the build context, so the build cannot
|
||||
read the version and a plain `docker build .` fails. Pass the version
|
||||
with `--build-arg VERSION=...`, as `script/docker` and `script/cibuild`
|
||||
already do.
|
||||
|
||||
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
|
||||
runs `script/cibuild` on push. The workflow's only other step checks out the
|
||||
repo, with `persist-credentials: false`. `script/cibuild` needs no token, and
|
||||
@@ -518,149 +718,13 @@ style conventions are in separate documents:
|
||||
publishes behind `if: github.ref_name == 'main'`, so the run from the feature
|
||||
branch publishes nothing.
|
||||
|
||||
- Use platform-standard formatters:
|
||||
- `black` for Python
|
||||
- `prettier` for JS/CSS/Markdown/HTML
|
||||
- `gofmt`, then `golines -m 77`, for Go
|
||||
|
||||
Always use default configuration, with three exceptions:
|
||||
- four-space indents (except Go)
|
||||
- `proseWrap: always` for Markdown (hard-wrap at 80 columns)
|
||||
- a 77-column limit for Go, set by `golines -m 77`
|
||||
|
||||
`script/fmt` runs `gofmt` and then `golines -m 77` on Go code.
|
||||
`script/fmt-check` fails when either of them would change a file. `golines`
|
||||
comes from `github.com/segmentio/golines`. `script/bootstrap` installs it
|
||||
with `go install`, pinned to a commit hash as below:
|
||||
|
||||
```sh
|
||||
# golines v0.13.0, 2025-08-21
|
||||
go install github.com/segmentio/golines@8f32f0f7e89c30f572c7f2cd3b2a48016b9d8bbf
|
||||
```
|
||||
|
||||
`script/bootstrap` reads the installed version from the `mod` line of
|
||||
`go version -m "$(command -v golines)"`, which reports `v0.13.0`. It does
|
||||
not use `golines --version`, which prints `golines vdev` for a binary that
|
||||
`go install` built.
|
||||
|
||||
`golines` cannot shorten every line: long string literals and comments stay
|
||||
as they are. golangci-lint's `lll` limit therefore stays at 88 columns, so
|
||||
it fails only lines far past the formatter's limit.
|
||||
|
||||
Documentation and writing repos (Markdown, HTML, CSS) should also have
|
||||
`.prettierrc` and `.prettierignore`.
|
||||
|
||||
- Pre-commit hook: runs `script/precommit`, which calls `script/check`. If local
|
||||
testing is not possible in the repo, `script/precommit` may skip `script/test`
|
||||
and run only `script/lint` and `script/fmt-check`. `script/install-precommit`
|
||||
installs the hook; the Makefile must provide a `make hooks` target that shims
|
||||
to it.
|
||||
|
||||
- All repos with software must have tests that run via the platform-standard
|
||||
test framework (`go test`, `pytest`, `jest`/`vitest`, etc.). If no meaningful
|
||||
tests exist yet, add the most minimal test possible: importing the module
|
||||
under test to verify it compiles/parses, for example. There is no excuse for
|
||||
`make test` to be a no-op.
|
||||
|
||||
- `make test` must complete in under 60 seconds. That is the hard cap, and a
|
||||
suite that exceeds it fails. Under 20 seconds is the target. A suite between
|
||||
20 and 60 seconds is still green, but the overage must be filed as an
|
||||
improvement bug against that repo. Add a 90-second timeout to the test
|
||||
invocation (`go test -timeout 90s`). The backstop deliberately sits above the
|
||||
hard cap so that it catches a genuinely hung test rather than a merely slow
|
||||
one. A hung package uses the backstop once, because a first run that timed out
|
||||
gets no rerun. `-timeout` applies to each package, so several hung packages
|
||||
can each use it.
|
||||
|
||||
- **The test command should use the conditional verbose rerun pattern.** Run
|
||||
tests without `-v` (verbose) first. If tests fail, automatically rerun with
|
||||
`-v` to show full output. This keeps CI logs and `docker build` output clean
|
||||
on success, with just package/suite summaries. On failure it provides full
|
||||
diagnostic detail: every test case, every assertion. A first run that timed
|
||||
out gets no rerun, because the rerun would only wait for the same hang. The
|
||||
command lives in the `test` phase of the `Dockerfile`, since `script/test`
|
||||
builds that phase. The Makefile form below is the same pattern for any
|
||||
repo-local invocation:
|
||||
|
||||
```makefile
|
||||
test:
|
||||
@<test-command> || \
|
||||
{ echo "--- Rerunning with -v for details ---"; \
|
||||
<test-command-with-v>; exit 1; }
|
||||
```
|
||||
|
||||
Go example:
|
||||
|
||||
```makefile
|
||||
test:
|
||||
@status=0; \
|
||||
output=$$(go test -count=1 -timeout 90s -race -cover ./... 2>&1) || status=$$?; \
|
||||
printf '%s\n' "$$output"; \
|
||||
test "$$status" -eq 0 && exit 0; \
|
||||
case "$$output" in \
|
||||
*"panic: test timed out"*|*"*** Test killed"*) exit 1 ;; \
|
||||
esac; \
|
||||
echo "--- Rerunning with -v for details ---"; \
|
||||
go test -count=1 -timeout 90s -race -v ./...; \
|
||||
exit 1
|
||||
```
|
||||
|
||||
The first run's output is kept so that the recipe can look in it for what Go
|
||||
prints for a hang:
|
||||
- `panic: test timed out` means the test's own timer fired.
|
||||
- `*** Test killed` means `go test` stopped a test that ignored that timer,
|
||||
about a minute later.
|
||||
|
||||
Go prints the goroutine stacks with either line, as long as the test can
|
||||
still respond. A rerun would only wait for the same hang, so the recipe
|
||||
exits 1. A failure with neither line in its output reruns with `-v`. The
|
||||
kept output prints when the first run ends, not while it runs.
|
||||
|
||||
`-count=1` is required on both invocations. It defeats Go's test _result_
|
||||
cache, so neither run can report a stored pass in place of running the
|
||||
tests. It leaves the build cache alone, so it costs the runtime of the suite
|
||||
and no recompilation.
|
||||
|
||||
That cache is Go's own, separate from Docker's layer cache. Go stores a
|
||||
passing result in its cache directory (`GOCACHE`). When the same tests run
|
||||
again on unchanged code, it prints that result, marked `(cached)`, without
|
||||
running them. The directory lasts from one run to the next on a developer's
|
||||
machine, where this target runs. The directory also lasts in the `test`
|
||||
phase of the `Dockerfile`, which mounts it to keep compiled packages between
|
||||
check builds, so the phase's `go test` runs carry `-count=1` too.
|
||||
|
||||
Python example:
|
||||
|
||||
```makefile
|
||||
test:
|
||||
@timeout 90 sh -c 'pipenv run pytest || { \
|
||||
echo "--- Rerunning with -v for details ---"; \
|
||||
pipenv run pytest -v; exit 1; }'
|
||||
```
|
||||
|
||||
Both pytest runs share that one 90-second timeout, as in the `prompts`
|
||||
repo's `examples/python-test-phase.Dockerfile`, so a hang uses it once and
|
||||
gets no rerun. Ctrl-C does not stop a running `make test`, which goes on
|
||||
until the tests end or the 90 seconds run out. On a Mac without `timeout`,
|
||||
the target fails until it is installed. On a host, the target needs
|
||||
`$HOME/.pipenv/bin` on `PATH`.
|
||||
|
||||
The `exit 1` ensures the target always fails after a rerun. The first run
|
||||
already proved the tests are broken, so the build must not pass even if a
|
||||
flaky test happens to succeed on the second attempt. The rerun exists solely
|
||||
for diagnostic output.
|
||||
|
||||
- Docker builds must complete in under 5 minutes. A hung package uses the
|
||||
90-second test backstop once, because a first run that timed out gets no
|
||||
rerun.
|
||||
|
||||
- `make check` must not modify any files in the repo. Tests may use temporary
|
||||
directories.
|
||||
|
||||
- `main` must always pass `make check`, no exceptions.
|
||||
|
||||
- Never commit secrets. `.env` files, credentials, API keys, and private keys
|
||||
must be in `.gitignore`. No exceptions.
|
||||
`script/cibuild` runs the CI build, and the Gitea workflow calls it. The
|
||||
script changes to the repo root, runs `script/bootstrap`, runs
|
||||
`script/check`, and builds the image with the version. **`script/cibuild`
|
||||
runs `script/bootstrap` first**, because the workflow checks out the repo
|
||||
and runs nothing else, while `script/fmt-check` runs the formatter on the
|
||||
host. On a pristine checkout with nothing installed, the run dies there,
|
||||
after the containerised gates have passed.
|
||||
|
||||
- `.gitignore` should be comprehensive from the start:
|
||||
- OS files (`.DS_Store`)
|
||||
@@ -716,36 +780,6 @@ style conventions are in separate documents:
|
||||
therefore still ships `services/api/.claude/` and must add its own anchored
|
||||
entry there.
|
||||
|
||||
- **A plain `docker build .` of a clone stamps the version that
|
||||
`git describe --tags --always` gives.** The version is derived from the `.git`
|
||||
in the build context, as the `prompts` repo's `examples/go.Dockerfile` shows.
|
||||
Without its failure check, a missing `git` or an unreadable checkout would
|
||||
leave `-X main.Version=` empty, and the build would still exit 0.
|
||||
`script/docker` and `script/cibuild` pass the version they compute on the
|
||||
host; it takes precedence. They do this byte-identically across repos:
|
||||
|
||||
```sh
|
||||
# The version and the tag each get their own line: a failing command
|
||||
# substitution inside an argument does not trip `set -e`, so the inline
|
||||
# form degrades to an empty constant.
|
||||
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
||||
[ -n "$version" ] || version="unknown"
|
||||
tag="$(script/projectname)"
|
||||
docker build --build-arg VERSION="$version" -t "$tag" .
|
||||
```
|
||||
|
||||
`--always` makes an untagged repo yield an abbreviated commit hash rather
|
||||
than failing. The `[ -n "$version" ]` line is the single place the fallback
|
||||
is applied. That line is a live check that fires on a build from an export
|
||||
with no `.git`, and on a repository with no commits yet. Do not fold it into
|
||||
the substitution as `|| echo unknown`, which makes the guard unreachable.
|
||||
The Dockerfile's side is `ARG VERSION` in the stage that compiles, declared
|
||||
there because `ARG` is stage-scoped. Passing `VERSION` to a repo whose
|
||||
Dockerfile declares no such `ARG` is ignored and costs nothing, which is why
|
||||
the scripts stay byte-identical. One consequence for CI: the standard
|
||||
checkout action clones shallow and fetches no tags, so the canonical
|
||||
`.gitea/workflows/check.yml` sets `fetch-depth: 0` on its checkout step.
|
||||
|
||||
- **Verify `.dockerignore` by enumerating the image, not by reading the
|
||||
patterns.** Plant files at the root _and_ at least two directories deep. Build
|
||||
a probe image that does `COPY . .`, and list what actually landed
|
||||
@@ -753,35 +787,6 @@ style conventions are in separate documents:
|
||||
size is not a substitute: a nested secret is a few bytes, and BuildKit
|
||||
transfers only the delta from the previous build.
|
||||
|
||||
- **No build artifacts in version control.** Code-derived data must never be
|
||||
committed to the repository if it can be avoided. Code-derived data includes:
|
||||
- compiled bundles
|
||||
- minified output
|
||||
- generated assets
|
||||
|
||||
The build process (e.g. Dockerfile, Makefile) should generate these at build
|
||||
time. Notable exception: Go protobuf generated files (`.pb.go`) ARE
|
||||
committed because repos need to work with `go get`, which downloads code but
|
||||
does not execute code generation.
|
||||
|
||||
- Never use `git add -A` or `git add .`. Always stage files explicitly by name.
|
||||
|
||||
- Never force-push to `main`.
|
||||
|
||||
- Make all changes on a feature branch. You can do whatever you want on a
|
||||
feature branch.
|
||||
|
||||
- A repo updates its policy files with one command: `make update-policy`. It
|
||||
runs `script/update-policy`, which fetches each vendored file the repo carries
|
||||
from the `prompts` repo, commits them and pushes. It uses the head of `main`,
|
||||
or the commit given by a full hash: `script/update-policy <hash>`. The list of
|
||||
vendored files is at the top of that script. The script replaces each file and
|
||||
keeps only the repo's own entries:
|
||||
- in `.gitignore`, `.editorconfig` and `.dockerignore`: the lines below the
|
||||
comment that ends the canonical file
|
||||
- in `.golangci.yml`: the entries the repo added to the `deny` list of the
|
||||
`test-support` depguard rule
|
||||
|
||||
- `.golangci.yml` is standardized. The vendored copy in a consuming repo must
|
||||
_NEVER_ be modified by an agent. Fetch it from
|
||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.golangci.yml` and keep it
|
||||
@@ -806,50 +811,44 @@ style conventions are in separate documents:
|
||||
golangci-lint rejects. A newer golangci-lint can add linters that
|
||||
`default: all` switches on until the canonical copy disables them.
|
||||
|
||||
- **`script/bootstrap` installs a pinned tool by comparing versions, never by
|
||||
testing presence.** An `if ! command -v <tool>; then install; fi` guard tests
|
||||
`PATH` only. On an already-provisioned machine the pin is then inert, and a
|
||||
version bump is a silent no-op. The Dockerfile, installing into a clean image,
|
||||
gets the pinned version, so a local `make check` and `make docker` can
|
||||
disagree about what the tool even is. The canonical form:
|
||||
- compares the installed version against the pin over the **whole** version
|
||||
token. A parser that stops at the first `-` reports `2.12.2` for a host
|
||||
running `2.12.2-rc1` and skips the install.
|
||||
- treats `--version` output as a mismatch when it is any of these:
|
||||
- absent
|
||||
- non-zero
|
||||
- empty
|
||||
- unrecognised
|
||||
- Use platform-standard formatters:
|
||||
- `black` for Python
|
||||
- `prettier` for JS/CSS/Markdown/HTML
|
||||
- `gofmt`, then `golines -m 77`, for Go
|
||||
|
||||
So a failure causes a redundant install, never a skipped one.
|
||||
Always use default configuration, with three exceptions:
|
||||
- four-space indents (except Go)
|
||||
- `proseWrap: always` for Markdown (hard-wrap at 80 columns)
|
||||
- a 77-column limit for Go, set by `golines -m 77`
|
||||
|
||||
- after installing, re-resolves the binary the way callers do, with
|
||||
`hash -r` and then through `PATH`, not through the directory the installer
|
||||
wrote to. A shadowing binary can hide an install, which then succeeds
|
||||
while no caller sees any change. In that case it fails and names the
|
||||
resolved path.
|
||||
- is actually called, and prints the version on both success paths. A
|
||||
function defined and never invoked has the same exit status and the same
|
||||
empty output as one that worked.
|
||||
`script/fmt` runs `gofmt` and then `golines -m 77` on Go code.
|
||||
`script/fmt-check` fails when either of them would change a file. `golines`
|
||||
comes from `github.com/segmentio/golines`. `script/bootstrap` installs it
|
||||
with `go install`, pinned to a commit hash as below:
|
||||
|
||||
A tool from the distribution has no pin, so `script/bootstrap` installs it
|
||||
when it is missing. make, git, node and yarn are such tools.
|
||||
```sh
|
||||
# golines v0.13.0, 2025-08-21
|
||||
go install github.com/segmentio/golines@8f32f0f7e89c30f572c7f2cd3b2a48016b9d8bbf
|
||||
```
|
||||
|
||||
Keep it POSIX sh: no arrays, no `[[`, no `grep -P`.
|
||||
`script/bootstrap` reads the installed version from the `mod` line of
|
||||
`go version -m "$(command -v golines)"`, which reports `v0.13.0`. It does
|
||||
not use `golines --version`, which prints `golines vdev` for a binary that
|
||||
`go install` built.
|
||||
|
||||
A Go tool a repo needs on the host is installed with `go install` pinned to
|
||||
a commit hash (`go install <package>@<commit hash>`). It is never tracked as
|
||||
a `go.mod` tool dependency or through a `tools.go` file, either of which
|
||||
pulls the tool's own dependencies into the repo's `go.mod` and `go.sum`.
|
||||
`golines` cannot shorten every line: long string literals and comments stay
|
||||
as they are. golangci-lint's `lll` limit therefore stays at 88 columns, so
|
||||
it fails only lines far past the formatter's limit.
|
||||
|
||||
- When pinning images or packages by hash, add a comment above the reference
|
||||
with the version and date (YYYY-MM-DD).
|
||||
Documentation and writing repos (Markdown, HTML, CSS) should also have
|
||||
`.prettierrc` and `.prettierignore`.
|
||||
|
||||
- Use `yarn`, not `npm`.
|
||||
|
||||
- Write all dates as YYYY-MM-DD (ISO 8601).
|
||||
|
||||
- Simple projects should be configured with environment variables.
|
||||
- All repos should have an `.editorconfig` enforcing the project's indentation
|
||||
settings. It is the standard file from
|
||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.editorconfig`, followed by
|
||||
the repo's own sections, such as one for another language it uses. The
|
||||
standard file sets tabs for `Makefile` and Go files. A re-vendor replaces the
|
||||
standard part and keeps those sections.
|
||||
|
||||
- Dockerized web services listen on port 8080 by default, overridable with
|
||||
`PORT`.
|
||||
@@ -920,6 +919,16 @@ style conventions are in separate documents:
|
||||
hardening measure exists for HTTP services and is not listed here, it is
|
||||
still expected. When in doubt, harden.
|
||||
|
||||
- Database migrations live in `internal/db/migrations/` and must be embedded in
|
||||
the binary.
|
||||
- `000_migration.sql` — contains ONLY the creation of the migrations
|
||||
tracking table itself. Nothing else.
|
||||
- `001_schema.sql` — the full application schema.
|
||||
- **Pre-1.0.0:** never add additional migration files (002, 003, etc.).
|
||||
There is no installed base to migrate. Edit `001_schema.sql` directly.
|
||||
- **Post-1.0.0:** add new numbered migration files for each schema change.
|
||||
Never edit existing migrations after release.
|
||||
|
||||
- `README.md` is the primary documentation. Required sections:
|
||||
- **Description**: First line must include:
|
||||
- the project name
|
||||
@@ -946,30 +955,6 @@ style conventions are in separate documents:
|
||||
`LICENSE` file in the repo root and a License section in the README.
|
||||
- **Author**: [@sneak](https://sneak.berlin).
|
||||
|
||||
- First commit of a new repo should contain only `README.md`.
|
||||
|
||||
- Go module root: `sneak.berlin/go/<name>`. Always run `go mod tidy` before
|
||||
committing.
|
||||
|
||||
- Use SemVer.
|
||||
|
||||
- Database migrations live in `internal/db/migrations/` and must be embedded in
|
||||
the binary.
|
||||
- `000_migration.sql` — contains ONLY the creation of the migrations
|
||||
tracking table itself. Nothing else.
|
||||
- `001_schema.sql` — the full application schema.
|
||||
- **Pre-1.0.0:** never add additional migration files (002, 003, etc.).
|
||||
There is no installed base to migrate. Edit `001_schema.sql` directly.
|
||||
- **Post-1.0.0:** add new numbered migration files for each schema change.
|
||||
Never edit existing migrations after release.
|
||||
|
||||
- All repos should have an `.editorconfig` enforcing the project's indentation
|
||||
settings. It is the standard file from
|
||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.editorconfig`, followed by
|
||||
the repo's own sections, such as one for another language it uses. The
|
||||
standard file sets tabs for `Makefile` and Go files. A re-vendor replaces the
|
||||
standard part and keeps those sections.
|
||||
|
||||
- Avoid putting files in the repo root unless necessary. Root should contain
|
||||
only project-level config files:
|
||||
- `README.md`
|
||||
@@ -997,9 +982,9 @@ style conventions are in separate documents:
|
||||
- `static/` — static assets (images, fonts, etc.)
|
||||
- `web/` — web frontend source
|
||||
|
||||
- When setting up a new repo, files from the `prompts` repo may be used as
|
||||
templates. Fetch them from
|
||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/<path>`.
|
||||
- Guidance for coding agents lives in one `AGENTS.md` at the repository root. It
|
||||
is never committed under a file or directory named after one agent tool, such
|
||||
as `CLAUDE.md` or `.claude/`, and never split into separate memory files.
|
||||
|
||||
- New repos must contain at minimum:
|
||||
- `README.md`, `.git`, `.gitignore`, `.editorconfig`
|
||||
@@ -1015,6 +1000,21 @@ style conventions are in separate documents:
|
||||
`.gitattributes`, `.prettierrc`, `.prettierignore`
|
||||
- Python: `pyproject.toml`
|
||||
|
||||
- Guidance for coding agents lives in one `AGENTS.md` at the repository root. It
|
||||
is never committed under a file or directory named after one agent tool, such
|
||||
as `CLAUDE.md` or `.claude/`, and never split into separate memory files.
|
||||
- When setting up a new repo, files from the `prompts` repo may be used as
|
||||
templates. Fetch them from
|
||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/<path>`.
|
||||
|
||||
- First commit of a new repo should contain only `README.md`.
|
||||
|
||||
- Write all dates as YYYY-MM-DD (ISO 8601).
|
||||
|
||||
- Cross-project documentation (such as this file) must include
|
||||
`last_modified: YYYY-MM-DD` in the YAML front matter so it can be kept in sync
|
||||
with the authoritative source as policies evolve.
|
||||
|
||||
- Use SemVer.
|
||||
|
||||
- Go module root: `sneak.berlin/go/<name>`. Always run `go mod tidy` before
|
||||
committing.
|
||||
|
||||
- Simple projects should be configured with environment variables.
|
||||
Reference in new issue
Block a user