From 6383275411207437a129b3aab6284b60684dac4a Mon Sep 17 00:00:00 2001 From: sneak Date: Tue, 7 Jul 2026 01:53:42 +0200 Subject: [PATCH] Refresh vendored REPO_POLICIES.md --- REPO_POLICIES.md | 468 ++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 384 insertions(+), 84 deletions(-) diff --git a/REPO_POLICIES.md b/REPO_POLICIES.md index 2d40cb4..bc2f161 100644 --- a/REPO_POLICIES.md +++ b/REPO_POLICIES.md @@ -1,108 +1,408 @@ -# Development Policies +--- +title: Repository Policies +last_modified: 2026-07-06 +--- -- Docker image references by tag are server-mutable, therefore using them is an - RCE vulnerability. All docker image references must use cryptographic hashes - to securely specify the exact image that is expected. +This document covers repository structure, tooling, and workflow standards. Code +style conventions are in separate documents: -- Correspondingly, `go install` commands using things like '@latest' are also - dangerous RCE. Whenever writing scripts or tools, ALWAYS specify go install - targets using commit hashes which are cryptographically secure. +- [Code Styleguide](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/CODE_STYLEGUIDE.md) + (general, bash, Docker) +- [Go](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/CODE_STYLEGUIDE_GO.md) +- [JavaScript](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/CODE_STYLEGUIDE_JS.md) +- [Python](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/CODE_STYLEGUIDE_PYTHON.md) +- [Go HTTP Server Conventions](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/GO_HTTP_SERVER_CONVENTIONS.md) -- Every repo with software in it must have a Makefile in the root. Each such - Makefile should support `make test` (runs the project-specific tests), - `make lint`, `make fmt` (writes), `make fmt-check` (readonly), and - `make check` (has `test`, `lint`, and `fmt-check` as prereqs), `make docker` - (builds docker image). +--- -- Every repo should have a Dockerfile. If the repo contains non-server software, - the Dockerfile should bring up a development environment and `make check` - (i.e. the docker build should fail if the branch is not green). +- Cross-project documentation (such as this file) must include + `last_modified: YYYY-MM-DD` in the YAML front matter so it can be kept in sync + with the authoritative source as policies evolve. -- Platform-specific standard formatting should be used. `black` for python, - `prettier` for js/css/etc, `go fmt` for go. The only changes to default - settings should be to specify four-space indents where applicable (i.e. - everything except `go fmt`). +- **ALL external references must be pinned by cryptographic hash.** This + includes Docker base images, Go modules, npm packages, GitHub Actions, and + anything else fetched from a remote source. Version tags (`@v4`, `@latest`, + `:3.21`, etc.) are server-mutable and therefore remote code execution + vulnerabilities. The ONLY acceptable way to reference an external dependency + is by its content hash (Docker `@sha256:...`, Go module hash in `go.sum`, npm + integrity hash in lockfile, GitHub Actions `@`). No exceptions. + This also means never `curl | bash` to install tools like pyenv, nvm, rustup, + etc. Instead, download a specific release archive from GitHub, verify its hash + (hardcoded in the Dockerfile or script), and only then install. Unverified + install scripts are arbitrary remote code execution. This is the single most + important rule in this document. Double-check every external reference in + every file before committing. There are zero exceptions to this rule. -- If local testing is possible (it is not always), `make check` should be a - pre-commit hook. If it is not possible, `make lint && make fmt-check` should - be a pre-commit hook. +- Every repo with software must have a root `Makefile` with these targets: + `make bootstrap`, `make setup`, `make test`, `make lint`, `make fmt` (writes), + `make fmt-check` (read-only), `make check` (runs `test`, `lint`, `fmt-check`), + `make docker`, and `make hooks` (installs pre-commit hook). A model Makefile + is at `https://git.eeqj.de/sneak/prompts/raw/branch/main/Makefile`. -- If a working `make test` takes more than 20 seconds, that's a bug that needs - fixing. In fact, there should be a timeout specified in the `Makefile` that - fails it automatically if it takes >30s. +- Repos follow the + [Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all) + pattern: the implementation of each Makefile target lives in an executable + script in `script/` (`script/bootstrap`, `script/setup`, `script/test`, + `script/lint`, `script/fmt`, `script/fmt-check`, `script/check`, + `script/docker`), and the Makefile targets are thin shims that call them. The + scripts must be POSIX sh (`#!/bin/sh`, `set -eu`, no bashisms) so they run in + minimal containers (e.g. alpine images have no bash); locate the repo root + with `$(cd "$(dirname "$0")/.." && pwd -P)` and `cd` there before acting. From + the standard's canonical set we use `bootstrap`, `setup` (make the repo ready + for development after a fresh clone: runs `bootstrap`, then + `install-precommit`, plus any repo-specific initialization), `test`, and + `cibuild`. `script/bootstrap` installs all dependencies idempotently and + assumes nothing is present: base tools come from nix, apt, brew, or apk + (detected in that order; apt runs noninteractive). For node it uses the + installed node if present; otherwise it installs a PINNED node version via + nvm, first installing nvm itself if missing — from a hash-verified GitHub + release archive (never `curl | sh`), with bash installed as an explicit + prerequisite since nvm requires bash. yarn is then pinned via + `corepack prepare yarn@ --activate`. Never install "latest" or "lts"; + always exact versions. `script/cibuild` runs the CI build: it changes to the + repo root and runs `docker build .`; the Gitea workflow calls it. Four further + scripts are our own extensions to the standard: `script/check` runs + `script/test`, `script/lint`, and `script/fmt-check`; `script/precommit` is + what the git pre-commit hook runs, and it calls `script/check`; + `script/install-precommit` installs the git pre-commit hook (the `make hooks` + target shims to it); and `script/projectname` (literally that filename) simply + outputs the project's name. Scripts that need the name call + `script/projectname` — e.g. `script/docker` assembles its image tag from it — + so those scripts stay byte-identical across all repos. Repo-type-specific + pre-commit extras (e.g. `go mod tidy` verification in Go repos) belong in + `script/precommit`, not in the hook itself. Model scripts are at + `https://git.eeqj.de/sneak/prompts/raw/branch/main/script/`. The README + must document the provided scripts in an **Entrypoints** section (see the + README requirements below). -- Docker builds should time out in 5 minutes or less. +- Always use Makefile targets (`make fmt`, `make test`, `make lint`, etc.) + instead of invoking the underlying tools directly. The Makefile is the single + source of truth for how these operations are run. + +- The Makefile is authoritative documentation for how the repo is used. Beyond + the required targets above, it should have targets for every common operation: + running a local development server (`make run`, `make dev`), re-initializing + or migrating the database (`make db-reset`, `make migrate`), building + artifacts (`make build`), generating code, seeding data, or anything else a + developer would do regularly. If someone checks out the repo and types + `make`, they should see every meaningful operation available. A new + contributor should be able to understand the entire development workflow by + reading the Makefile. + +- Every repo should have a `Dockerfile`. All Dockerfiles must run `make check` + as a build step so the build fails if the branch is not green. For non-server + repos, the Dockerfile should bring up a development environment and run + `make check`. For server repos, `make check` should run as an early build + stage before the final image is assembled. Dockerfiles install development + prerequisites by running `script/bootstrap` rather than duplicating installs + inline; COPY `script/` and the dependency manifests (`package.json` + + `yarn.lock`, `go.mod` + `go.sum`, etc.) before running it so the bootstrap + layer stays cached until dependencies change. + +- **Dockerfiles must use a separate lint stage for fail-fast feedback.** Go + repos use a multistage build where linting runs in an independent stage based + on the `golangci/golangci-lint` image (pinned by hash). This stage runs + `make fmt-check` and `make lint` before the full build begins. The build stage + then declares an explicit dependency on the lint stage via + `COPY --from=lint /src/go.sum /dev/null`, which forces BuildKit to complete + linting before proceeding to compilation and tests. This ensures lint failures + surface in seconds rather than minutes, without blocking on dependency + download or compilation in the build stage. + + The standard pattern for a Go repo Dockerfile is: + + ```dockerfile + # Lint stage — fast feedback on formatting and lint issues + # golangci/golangci-lint:v2.x.x, YYYY-MM-DD + FROM golangci/golangci-lint@sha256:... AS lint + WORKDIR /src + COPY go.mod go.sum ./ + RUN go mod download + COPY . . + RUN make fmt-check + RUN make lint + + # Build stage + # golang:1.x-alpine, YYYY-MM-DD + FROM golang@sha256:... AS builder + WORKDIR /src + + # Force BuildKit to run the lint stage before proceeding + COPY --from=lint /src/go.sum /dev/null + + COPY go.mod go.sum ./ + RUN go mod download + COPY . . + RUN make test + + ARG VERSION=dev + RUN CGO_ENABLED=0 go build -trimpath \ + -ldflags="-s -w -X main.Version=${VERSION}" \ + -o /app ./cmd/app/ + + # Runtime stage + FROM alpine@sha256:... + COPY --from=builder /app /usr/local/bin/app + ENTRYPOINT ["app"] + ``` + + Key points: + - The lint stage uses the `golangci/golangci-lint` image directly (it + includes both Go and the linter), so there is no need to install the + linter separately. + - `COPY --from=lint /src/go.sum /dev/null` is a no-op file copy that creates + a stage dependency. BuildKit runs stages in parallel by default; without + this line, the build stage would not wait for lint to finish and a lint + failure might not fail the overall build. + - If the project uses `//go:embed` directives that reference build artifacts + (e.g. a web frontend compiled in a separate stage), the lint stage must + create placeholder files so the embed directives resolve. Example: + `RUN mkdir -p web/dist && touch web/dist/index.html web/dist/style.css`. + The lint stage should not depend on the actual build output — it exists to + fail fast. + - If the project requires CGO or system libraries for linting (e.g. + `vips-dev`), install them in the lint stage with `apk add`. + - The build stage runs `make test` after compilation setup. Tests run in the + build stage, not the lint stage, because they may require compiled + artifacts or heavier dependencies. + +- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that + runs `script/cibuild` (which runs `docker build .`) on push. Since the + Dockerfile already runs `make check`, a successful build implies all checks + pass. + +- Use platform-standard formatters: `black` for Python, `prettier` for + JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with + two exceptions: four-space indents (except Go), and `proseWrap: always` for + Markdown (hard-wrap at 80 columns). Documentation and writing repos (Markdown, + HTML, CSS) should also have `.prettierrc` and `.prettierignore`. + +- Pre-commit hook: runs `script/precommit`, which calls `script/check`. If local + testing is not possible in the repo, `script/precommit` may skip `script/test` + and run only `script/lint` and `script/fmt-check`. The hook is installed by + `script/install-precommit`; the Makefile must provide a `make hooks` target + that shims to it. + +- All repos with software must have tests that run via the platform-standard + test framework (`go test`, `pytest`, `jest`/`vitest`, etc.). If no meaningful + tests exist yet, add the most minimal test possible — e.g. importing the + module under test to verify it compiles/parses. There is no excuse for + `make test` to be a no-op. + +- `make test` must complete in under 20 seconds. Add a 30-second timeout in the + Makefile. + +- **`make test` should use the conditional verbose rerun pattern.** Run tests + without `-v` (verbose) first. If tests fail, automatically rerun with `-v` to + show full output. This keeps CI logs and `docker build` output clean on + success (just package/suite summaries) while providing full diagnostic detail + on failure (every test case, every assertion). The general shell pattern: + + ```makefile + test: + @ || \ + { echo "--- Rerunning with -v for details ---"; \ + ; exit 1; } + ``` + + Go example: + + ```makefile + test: + @go test -timeout 30s -race -cover ./... || \ + { echo "--- Rerunning with -v for details ---"; \ + go test -timeout 30s -race -v ./...; exit 1; } + ``` + + Python example: + + ```makefile + test: + @python -m pytest || \ + { echo "--- Rerunning with -v for details ---"; \ + python -m pytest -v; exit 1; } + ``` + + The `exit 1` ensures the target always fails after a rerun — the first run + already proved the tests are broken, so the build must not pass even if a + flaky test happens to succeed on the second attempt. The rerun exists solely + for diagnostic output. + +- Docker builds must complete in under 5 minutes. + +- `make check` must not modify any files in the repo. Tests may use temporary + directories. - `main` must always pass `make check`, no exceptions. -- Do all changes on a feature branch. You can do whatever you want on a feature - branch. +- Never commit secrets. `.env` files, credentials, API keys, and private keys + must be in `.gitignore`. No exceptions. -- We have a standardized `.golangci.yml` which we reuse and is _NEVER_ to be - modified by an agent, only manually by the user. It can be copied from - `~/dev/upaas/.golangci.yml` if it exists at that location. +- `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`), + editor files (`.swp`, `*~`), language build artifacts, and `node_modules/`. + Fetch the standard `.gitignore` from + `https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when setting up + a new repo. -- When specifying images or packages by hash in Dockerfiles or - `docker-compose.yml`, put a comment above the line and show the version and - date at which it was current. +- **No build artifacts in version control.** Code-derived data (compiled + bundles, minified output, generated assets) must never be committed to the + repository if it can be avoided. The build process (e.g. Dockerfile, Makefile) + should generate these at build time. Notable exception: Go protobuf generated + files (`.pb.go`) ARE committed because repos need to work with `go get`, which + downloads code but does not execute code generation. -- For javascript, always use `yarn` over `npm`. +- Never use `git add -A` or `git add .`. Always stage files explicitly by name. -- Whenever writing dates, ALWAYS write YYYY-MM-DD (ISO 8601). +- Never force-push to `main`. -- Simple projects should be configured with environment variables, as is - standard for Dockerized applications. +- Make all changes on a feature branch. You can do whatever you want on a + feature branch. -- Dockerized web services should listen on the default HTTP port of 8080 unless - overridden with the `PORT` environment variable. +- `.golangci.yml` is standardized and must _NEVER_ be modified by an agent, only + manually by the user. Fetch from + `https://git.eeqj.de/sneak/prompts/raw/branch/main/.golangci.yml`. -- The `README.md` is a project's primary documentation. It should contain at a - minimum the following sections: - - Description - - Include a short and complete description of the functionality and - purpose of the software as the first line in the readme. It must - include: - - the name - - the purpose - - the category (web server, SPA, command line tool, etc) - - the license - - the author - - eg: "µPaaS is an MIT-licensed Go web application by @sneak that - receives git-frontend webhooks and interacts with a Docker server - to build and deploy applications in realtime as certain branches - are updated." - - Getting Started - - a code block with copy-pasteable installation/use sections - - Rationale - - why does this exist? - - Design - - how is the program structured? - - TODO - - This is your TODO list for the project - update it meticulously, even - in between commits. Whenever planning, put your todo list in the - README so that a separate agent with new context can pick up where you - left off. - - License - - GPL or MIT or WTFPL - ask the user when beginning a new project and - include a LICENSE file in the root and in a section in the README. - - Author - - @sneak (link `@sneak` to `https://sneak.berlin`). +- When pinning images or packages by hash, add a comment above the reference + with the version and date (YYYY-MM-DD). -- When beginning a new project, initialize a git repo and make the first commit - simply the first version of the README.md in the root of the repo. +- Use `yarn`, not `npm`. -- For Go packages, the module root is `sneak.berlin/go/...`, such as - `sneak.berlin/go/dnswatcher`. +- Write all dates as YYYY-MM-DD (ISO 8601). -- We use SemVer always. +- Simple projects should be configured with environment variables. -- If no tag `1.0.0` or greater exists in the repository, modify the existing - migrations and assume no installed base or existing databases. If `>=1.0.0`, - database changes add new migration files. +- Dockerized web services listen on port 8080 by default, overridable with + `PORT`. -- New repos must have at a minimum the following files: - - `README.md`, `.git`, `.gitignore` - - `POLICIES.md` (copy from `~/Documents/_PROMPTS/POLICIES.md`) +- **HTTP/web services must be hardened for production internet exposure before + tagging 1.0.** This means full compliance with security best practices + including, without limitation, all of the following: + - **Security headers** on every response: + - `Strict-Transport-Security` (HSTS) with `max-age` of at least one year + and `includeSubDomains`. + - `Content-Security-Policy` (CSP) with a restrictive default policy + (`default-src 'self'` as a baseline, tightened per-resource as + needed). Never use `unsafe-inline` or `unsafe-eval` unless + unavoidable, and document the reason. + - `X-Frame-Options: DENY` (or `SAMEORIGIN` if framing is required). + Prefer the `frame-ancestors` CSP directive as the primary control. + - `X-Content-Type-Options: nosniff`. + - `Referrer-Policy: strict-origin-when-cross-origin` (or stricter). + - `Permissions-Policy` restricting access to browser features the + application does not use (camera, microphone, geolocation, etc.). + - **Request and response limits:** + - Maximum request body size enforced on all endpoints (e.g. Go + `http.MaxBytesReader`). Choose a sane default per-route; never accept + unbounded input. + - Maximum response body size where applicable (e.g. paginated APIs). + - `ReadTimeout` and `ReadHeaderTimeout` on the `http.Server` to defend + against slowloris attacks. + - `WriteTimeout` on the `http.Server`. + - `IdleTimeout` on the `http.Server`. + - Per-handler execution time limits via `context.WithTimeout` or + chi/stdlib `middleware.Timeout`. + - **Authentication and session security:** + - Rate limiting on password-based authentication endpoints. API keys are + high-entropy and not susceptible to brute force, so they are exempt. + - CSRF tokens on all state-mutating HTML forms. API endpoints + authenticated via `Authorization` header (Bearer token, API key) are + exempt because the browser does not attach these automatically. + - Passwords stored using bcrypt, scrypt, or argon2 — never plain-text, + MD5, or SHA. + - Session cookies set with `HttpOnly`, `Secure`, and `SameSite=Lax` (or + `Strict`) attributes. + - **Reverse proxy awareness:** + - True client IP detection when behind a reverse proxy + (`X-Forwarded-For`, `X-Real-IP`). The application must accept + forwarded headers only from a configured set of trusted proxy + addresses — never trust `X-Forwarded-For` unconditionally. + - **CORS:** + - Authenticated endpoints must restrict `Access-Control-Allow-Origin` to + an explicit allowlist of known origins. Wildcard (`*`) is acceptable + only for public, unauthenticated read-only APIs. + - **Error handling:** + - Internal errors must never leak stack traces, SQL queries, file paths, + or other implementation details to the client. Return generic error + messages in production; detailed errors only when `DEBUG` is enabled. + - **TLS:** + - Services never terminate TLS directly. They are always deployed behind + a TLS-terminating reverse proxy. The service itself listens on plain + HTTP. However, HSTS headers and `Secure` cookie flags must still be + set by the application so that the browser enforces HTTPS end-to-end. + + This list is non-exhaustive. Apply defense-in-depth: if a standard security + hardening measure exists for HTTP services and is not listed here, it is + still expected. When in doubt, harden. + +- `README.md` is the primary documentation. Required sections: + - **Description**: First line must include the project name, purpose, + category (web server, SPA, CLI tool, etc.), license, and author. Example: + "µPaaS is an MIT-licensed Go web application by @sneak that receives + git-frontend webhooks and deploys applications via Docker in realtime." + - **Getting Started**: Copy-pasteable install/usage code block. + - **Entrypoints**: Opens by stating that the repo adheres to the + [Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all) + standard (with that link), then documents each provided `script/` + entrypoint and its purpose. + - **Rationale**: Why does this exist? + - **Design**: How is the program structured? + - **TODO**: Update meticulously, even between commits. When planning, put + the todo list in the README so a new agent can pick up where the last one + left off. + - **License**: MIT, GPL, or WTFPL. Ask the user for new projects. Include a + `LICENSE` file in the repo root and a License section in the README. + - **Author**: [@sneak](https://sneak.berlin). + +- First commit of a new repo should contain only `README.md`. + +- Go module root: `sneak.berlin/go/`. Always run `go mod tidy` before + committing. + +- Use SemVer. + +- Database migrations live in `internal/db/migrations/` and must be embedded in + the binary. + - `000_migration.sql` — contains ONLY the creation of the migrations + tracking table itself. Nothing else. + - `001_schema.sql` — the full application schema. + - **Pre-1.0.0:** never add additional migration files (002, 003, etc.). + There is no installed base to migrate. Edit `001_schema.sql` directly. + - **Post-1.0.0:** add new numbered migration files for each schema change. + Never edit existing migrations after release. + +- All repos should have an `.editorconfig` enforcing the project's indentation + settings. + +- Avoid putting files in the repo root unless necessary. Root should contain + only project-level config files (`README.md`, `Makefile`, `Dockerfile`, + `LICENSE`, `.gitignore`, `.editorconfig`, `REPO_POLICIES.md`, and + language-specific config). Everything else goes in a subdirectory. Canonical + subdirectory names: + - `bin/` — executable scripts and tools + - `cmd/` — Go command entrypoints + - `configs/` — configuration templates and examples + - `deploy/` — deployment manifests (k8s, compose, terraform) + - `docs/` — documentation and markdown (README.md stays in root) + - `internal/` — Go internal packages + - `internal/db/migrations/` — database migrations + - `pkg/` — Go library packages + - `share/` — systemd units, data files + - `static/` — static assets (images, fonts, etc.) + - `web/` — web frontend source + +- When setting up a new repo, files from the `prompts` repo may be used as + templates. Fetch them from + `https://git.eeqj.de/sneak/prompts/raw/branch/main/`. + +- New repos must contain at minimum: + - `README.md`, `.git`, `.gitignore`, `.editorconfig` + - `LICENSE`, `REPO_POLICIES.md` (copy from the `prompts` repo) + - `Makefile` + - `script/` entrypoints (`bootstrap`, `setup`, `projectname`, `test`, + `lint`, `fmt`, `fmt-check`, `check`, `docker`, `cibuild`, `precommit`, + `install-precommit`) - `Dockerfile`, `.dockerignore` - - for go: `go.mod`, `go.sum`, `.golangci.yml` - - for js: `package.json` + - `.gitea/workflows/check.yml` + - Go: `go.mod`, `go.sum`, `.golangci.yml` + - JS: `package.json`, `yarn.lock`, `.prettierrc`, `.prettierignore` + - Python: `pyproject.toml`