Compare commits
5
Commits
478ef0856a
...
next
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
128eb1b644 | ||
|
|
3de345fe6f | ||
|
|
a9d77e20d7 | ||
|
|
faf3da9a75 | ||
|
|
fa6a9ed4dc |
+83
-29
@@ -1,30 +1,84 @@
|
|||||||
# .git is sent so the build can derive the version it stamps into the binary
|
# .dockerignore does NOT use .gitignore semantics. Docker matches with
|
||||||
# (script/version). Its config, which can hold a remote URL carrying a
|
# moby/patternmatcher: filepath.Match plus `**`, so `*` does not cross
|
||||||
# credential and which `git describe` does not need, is left out of a
|
# `/` and an unprefixed pattern is anchored at the context root. Every
|
||||||
# directory context. A context sent as a tar is not filtered by this file, so
|
# depth-independent pattern therefore needs `**/`, or `config/.env` and
|
||||||
# it carries .git/config unless its sender leaves it out.
|
# `certs/server.key` still ship while this file reads as solved. Only
|
||||||
.git/config
|
# genuinely root-anchored entries go unprefixed. Never transplant these
|
||||||
|
# into .gitignore, where `**/` is wrong.
|
||||||
# No tracked file may be listed here: git in the build would see it as
|
|
||||||
# deleted and mark the version -dirty.
|
|
||||||
#
|
#
|
||||||
# .ci-fingerprint is deliberately NOT excluded: it is the CI cache barrier
|
# Matching is case-sensitive, so secrets use character ranges rather
|
||||||
# that keeps the check stages from replaying a cached pass. See the lint
|
# than an ALL-CAPS twin, which would still miss `Server.Key`.
|
||||||
# stage of the Dockerfile.
|
#
|
||||||
bin/
|
# Extend with this repo's own host-built artifacts, written anchored:
|
||||||
# Extracted from 3p/ by `make assets` inside the build; a host copy is not
|
# `/myapp`, never `**/myapp`, which also matches `cmd/myapp/` and
|
||||||
# needed. The tarball in 3p/ must stay in the context.
|
# deletes the package directory from the context.
|
||||||
static/js/alpine.min.js
|
|
||||||
# The js-deps stage installs ESLint and prettier; a host copy would overwrite
|
# .git is sent without its config. Without a VERSION build argument the
|
||||||
# them at the `COPY . .` of the stages built on it.
|
# stage that compiles runs `git describe --tags --always` on .git, which
|
||||||
node_modules/
|
# does not need .git/config; that file can hold a credential, such as a
|
||||||
.env
|
# password in a remote URL or the token the CI checkout step stores there.
|
||||||
.env.*
|
# Each submodule keeps a config with the same exposure in its git directory
|
||||||
*.db
|
# under .git/modules/, nested again for a submodule's own submodules, or in
|
||||||
*.sqlite
|
# its own .git directory when it keeps one.
|
||||||
*.sqlite3
|
# KNOWN GAP: a submodule whose name has a `config` segment (`config`,
|
||||||
.DS_Store
|
# `deploy/config`, `config/lib`) loses its whole git directory, because
|
||||||
.idea/
|
# `**/.git/modules/**/config` also matches that segment's directory
|
||||||
.vscode/
|
# under .git/modules/. Go's version stamping then fails the build;
|
||||||
tmp/
|
# nothing leaks. Name such a submodule without that segment:
|
||||||
temp/
|
# `git submodule add --name`.
|
||||||
|
**/.git/config
|
||||||
|
**/.git/modules/**/config
|
||||||
|
|
||||||
|
# Agent scratch: one full checkout of the repo per in-flight agent.
|
||||||
|
# Anchored because it occurs once where agents run at the repo root.
|
||||||
|
# KNOWN GAP: a repo running agents in subdirectories still ships
|
||||||
|
# `services/api/.claude/` and must add its own anchored entry.
|
||||||
|
.claude
|
||||||
|
|
||||||
|
# Environment files. `*.env` covers bare `.env` and the `prod.env`
|
||||||
|
# convention. Re-include a committed template with a negation if the
|
||||||
|
# build needs one: `!docs/example.env`.
|
||||||
|
**/*.[eE][nN][vV]
|
||||||
|
**/.[eE][nN][vV].*
|
||||||
|
**/.[eE][nN][vV][rR][cC]
|
||||||
|
|
||||||
|
# Private keys and the bundles carrying them. Public certificates
|
||||||
|
# (*.crt, *.cer) are deliberately absent: they are legitimate inputs.
|
||||||
|
**/*.[pP][eE][mM]
|
||||||
|
**/*.[kK][eE][yY]
|
||||||
|
**/*.[pP]12
|
||||||
|
**/*.[pP][fF][xX]
|
||||||
|
**/[iI][dD]_[rR][sS][aA]
|
||||||
|
**/[iI][dD]_[dD][sS][aA]
|
||||||
|
**/[iI][dD]_[eE][cC][dD][sS][aA]
|
||||||
|
**/[iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK]
|
||||||
|
**/[iI][dD]_[eE][dD]25519
|
||||||
|
**/[iI][dD]_[eE][dD]25519_[sS][kK]
|
||||||
|
|
||||||
|
# Dependencies: restored inside the image, never copied in.
|
||||||
|
**/node_modules
|
||||||
|
|
||||||
|
# OS metadata.
|
||||||
|
**/.DS_Store
|
||||||
|
**/Thumbs.db
|
||||||
|
|
||||||
|
# Editor state: never a build input, and it churns COPY.
|
||||||
|
**/*.swp
|
||||||
|
**/*.swo
|
||||||
|
**/*~
|
||||||
|
**/*.bak
|
||||||
|
**/.idea
|
||||||
|
**/.vscode
|
||||||
|
**/*.sublime-*
|
||||||
|
|
||||||
|
# This repository's own host-built artifacts: the binary `make build`
|
||||||
|
# writes, and the Alpine.js file `make assets` extracts from 3p/ (the
|
||||||
|
# build extracts its own).
|
||||||
|
/bin
|
||||||
|
/static/js/alpine.min.js
|
||||||
|
|
||||||
|
# SQLite databases, which hold the session key and webhook payloads, at
|
||||||
|
# any depth.
|
||||||
|
**/*.db
|
||||||
|
**/*.sqlite
|
||||||
|
**/*.sqlite3
|
||||||
|
|||||||
@@ -10,3 +10,6 @@ insert_final_newline = true
|
|||||||
|
|
||||||
[Makefile]
|
[Makefile]
|
||||||
indent_style = tab
|
indent_style = tab
|
||||||
|
|
||||||
|
[*.go]
|
||||||
|
indent_style = tab
|
||||||
|
|||||||
@@ -1,37 +1,9 @@
|
|||||||
name: check
|
name: check
|
||||||
|
on: [push]
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches:
|
|
||||||
- '**'
|
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
check:
|
check:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
steps:
|
steps:
|
||||||
- name: Checkout
|
# actions/checkout v4.2.2, 2026-02-22
|
||||||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 2024-10-23
|
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
|
||||||
with:
|
- run: script/cibuild
|
||||||
# The superseded-status step needs history to walk ancestors (it
|
|
||||||
# aborts on a shallow clone).
|
|
||||||
fetch-depth: 0
|
|
||||||
|
|
||||||
- name: Mark superseded run statuses
|
|
||||||
# Gitea cancels the in-flight run when another commit is pushed to the
|
|
||||||
# same branch and records the cancellation as `failure`, so a commit
|
|
||||||
# that was never tested reads as a test result. The script rewrites
|
|
||||||
# those statuses to say what happened. See its header for why the
|
|
||||||
# state stays `failure` and not `skipped`.
|
|
||||||
env:
|
|
||||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
|
||||||
run: script/ci-mark-superseded
|
|
||||||
|
|
||||||
- name: Fingerprint the build context
|
|
||||||
# Writes the hash of the commit being checked into the context, which
|
|
||||||
# invalidates the `COPY . .` layer of every check stage: a commit
|
|
||||||
# that was never linted, format-checked, stylesheet-checked, tested
|
|
||||||
# and built cannot report success from cache.
|
|
||||||
run: git rev-parse HEAD > .ci-fingerprint
|
|
||||||
|
|
||||||
- name: Build Docker image (runs the gofmt check, golangci-lint, the stylesheet check, ESLint, the Markdown check, make test, make build)
|
|
||||||
run: script/cibuild
|
|
||||||
|
|||||||
+50
-21
@@ -1,3 +1,53 @@
|
|||||||
|
# OS
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
|
|
||||||
|
# Editors
|
||||||
|
*.swp
|
||||||
|
*.swo
|
||||||
|
*~
|
||||||
|
*.bak
|
||||||
|
.idea/
|
||||||
|
.vscode/
|
||||||
|
*.sublime-*
|
||||||
|
|
||||||
|
# Agent scratch (worktrees of this repo, created and destroyed by
|
||||||
|
# in-flight tooling). Unanchored: .gitignore patterns already match at
|
||||||
|
# every depth, so no prefix is wanted here. This is not a .dockerignore
|
||||||
|
# entry and must not be given a `**/` prefix on the way into one.
|
||||||
|
.claude/
|
||||||
|
|
||||||
|
# Node
|
||||||
|
node_modules/
|
||||||
|
|
||||||
|
# Secrets. Unanchored like every entry above, so each matches at every
|
||||||
|
# depth. Matching is case-sensitive on Linux, so names use character
|
||||||
|
# ranges rather than a lowercase form that misses `Server.Key`.
|
||||||
|
|
||||||
|
# Environment files. `*.env` covers bare `.env` and the `prod.env`
|
||||||
|
# convention. Only the templates `example.env` and `sample.env` are
|
||||||
|
# re-included below. A repository that commits any other template adds
|
||||||
|
# its own negation after these lines, for example `!.env.example`.
|
||||||
|
*.[eE][nN][vV]
|
||||||
|
.[eE][nN][vV].*
|
||||||
|
.[eE][nN][vV][rR][cC]
|
||||||
|
!example.env
|
||||||
|
!sample.env
|
||||||
|
|
||||||
|
# Private keys and the bundles carrying them.
|
||||||
|
*.[pP][eE][mM]
|
||||||
|
*.[kK][eE][yY]
|
||||||
|
*.[pP]12
|
||||||
|
*.[pP][fF][xX]
|
||||||
|
[iI][dD]_[rR][sS][aA]
|
||||||
|
[iI][dD]_[dD][sS][aA]
|
||||||
|
[iI][dD]_[eE][cC][dD][sS][aA]
|
||||||
|
[iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK]
|
||||||
|
[iI][dD]_[eE][dD]25519
|
||||||
|
[iI][dD]_[eE][dD]25519_[sS][kK]
|
||||||
|
|
||||||
|
# This repository's own entries, after the shared content above.
|
||||||
|
|
||||||
# Binaries
|
# Binaries
|
||||||
*.exe
|
*.exe
|
||||||
*.dll
|
*.dll
|
||||||
@@ -15,24 +65,6 @@ bin/
|
|||||||
# Go vendor directory
|
# Go vendor directory
|
||||||
vendor/
|
vendor/
|
||||||
|
|
||||||
# ESLint, prettier and their dependencies, installed from yarn.lock
|
|
||||||
node_modules/
|
|
||||||
|
|
||||||
# IDE specific files
|
|
||||||
.idea/
|
|
||||||
*.swp
|
|
||||||
*.swo
|
|
||||||
*~
|
|
||||||
.vscode/
|
|
||||||
|
|
||||||
# OS specific files
|
|
||||||
.DS_Store
|
|
||||||
Thumbs.db
|
|
||||||
|
|
||||||
# Environment and config files
|
|
||||||
.env
|
|
||||||
.env.local
|
|
||||||
|
|
||||||
# Data directory (SQLite databases)
|
# Data directory (SQLite databases)
|
||||||
data/
|
data/
|
||||||
*.db
|
*.db
|
||||||
@@ -46,9 +78,6 @@ data/
|
|||||||
tmp/
|
tmp/
|
||||||
temp/
|
temp/
|
||||||
|
|
||||||
# CI cache barrier, written into the build context by the check workflow
|
|
||||||
.ci-fingerprint
|
|
||||||
|
|
||||||
# Alpine.js, extracted by `make assets` from its tarball in 3p/, which is
|
# Alpine.js, extracted by `make assets` from its tarball in 3p/, which is
|
||||||
# what is committed.
|
# what is committed.
|
||||||
/static/js/alpine.min.js
|
/static/js/alpine.min.js
|
||||||
|
|||||||
+73
-2
@@ -10,14 +10,21 @@ run:
|
|||||||
|
|
||||||
linters:
|
linters:
|
||||||
default: all
|
default: all
|
||||||
|
enable:
|
||||||
|
# Successor to the deprecated gomodguard. Named explicitly, rather than
|
||||||
|
# left to `default: all`, because it carries the module policy below.
|
||||||
|
- gomodguard_v2
|
||||||
disable:
|
disable:
|
||||||
# Genuinely incompatible with project patterns
|
# Genuinely incompatible with project patterns
|
||||||
- exhaustruct # Requires all struct fields
|
- exhaustruct # Requires all struct fields
|
||||||
- depguard # Dependency allow/block lists
|
- exhaustruct_v5 # Requires all struct fields (successor to exhaustruct)
|
||||||
- godot # Requires comments to end with periods
|
- godot # Requires comments to end with periods
|
||||||
- wsl # Deprecated, replaced by wsl_v5
|
|
||||||
- wrapcheck # Too verbose for internal packages
|
- wrapcheck # Too verbose for internal packages
|
||||||
- varnamelen # Short names like db, id are idiomatic Go
|
- varnamelen # Short names like db, id are idiomatic Go
|
||||||
|
# Deprecated: the warning is attached to the old name, so it is
|
||||||
|
# silenced by disabling that name, not by enabling the successor.
|
||||||
|
- wsl # Deprecated, replaced by wsl_v5
|
||||||
|
- gomodguard # Deprecated, replaced by gomodguard_v2
|
||||||
settings:
|
settings:
|
||||||
lll:
|
lll:
|
||||||
line-length: 88
|
line-length: 88
|
||||||
@@ -28,6 +35,70 @@ linters:
|
|||||||
max-complexity: 15
|
max-complexity: 15
|
||||||
dupl:
|
dupl:
|
||||||
threshold: 100
|
threshold: 100
|
||||||
|
depguard:
|
||||||
|
# Test-support code must not be compiled into the shipped binary. A
|
||||||
|
# test-support package exists to hand a test privileges the program
|
||||||
|
# itself must never have, so a file that is not a test must not import
|
||||||
|
# one. Test files, and the files inside a package whose directory name
|
||||||
|
# ends in `test`, are where that code belongs, and are exempt.
|
||||||
|
#
|
||||||
|
# The deny list below is the one part of this file a repository is
|
||||||
|
# expected to extend, and the only part it may. depguard matches an
|
||||||
|
# import path against a list of prefixes, so it cannot be told "any path
|
||||||
|
# whose last segment ends in test"; a repository's own test-support
|
||||||
|
# packages have to be named here one at a time, by full import path,
|
||||||
|
# under a module path that differs from repository to repository. Add
|
||||||
|
# them; change nothing else.
|
||||||
|
rules:
|
||||||
|
test-support:
|
||||||
|
list-mode: lax
|
||||||
|
files:
|
||||||
|
- "$all"
|
||||||
|
- "!$test"
|
||||||
|
- "!**/*test/**"
|
||||||
|
deny:
|
||||||
|
- pkg: net/http/httptest
|
||||||
|
desc: >-
|
||||||
|
Test-support code belongs in test files and in packages whose
|
||||||
|
directory name ends in test, not in the shipped binary.
|
||||||
|
- pkg: sneak.berlin/go/webhooker/internal/config/configtest
|
||||||
|
desc: test support; a file that is not a test must not import it
|
||||||
|
- pkg: sneak.berlin/go/webhooker/internal/database/databasetest
|
||||||
|
desc: test support; a file that is not a test must not import it
|
||||||
|
- pkg: sneak.berlin/go/webhooker/internal/middleware/middlewaretest
|
||||||
|
desc: test support; a file that is not a test must not import it
|
||||||
|
# Only decisions already recorded in the Go package defaults are
|
||||||
|
# listed here. Every entry matches the module path exactly.
|
||||||
|
gomodguard_v2:
|
||||||
|
blocked:
|
||||||
|
- module: github.com/rs/zerolog
|
||||||
|
recommendations:
|
||||||
|
- log/slog
|
||||||
|
reason: "Structured logging is stdlib log/slog."
|
||||||
|
# One entry per pre-fork module path, because the later releases
|
||||||
|
# are separate paths. A prefix match would be shorter but would
|
||||||
|
# also reach github.com/go-redis/redismock, the test double for
|
||||||
|
# the successor these entries recommend.
|
||||||
|
- module: github.com/go-redis/redis
|
||||||
|
recommendations:
|
||||||
|
- github.com/redis/go-redis/v9
|
||||||
|
reason: "Pre-fork module; use the maintained go-redis v9."
|
||||||
|
- module: github.com/go-redis/redis/v7
|
||||||
|
recommendations:
|
||||||
|
- github.com/redis/go-redis/v9
|
||||||
|
reason: "Pre-fork module; use the maintained go-redis v9."
|
||||||
|
- module: github.com/go-redis/redis/v8
|
||||||
|
recommendations:
|
||||||
|
- github.com/redis/go-redis/v9
|
||||||
|
reason: "Pre-fork module; use the maintained go-redis v9."
|
||||||
|
- module: github.com/sergi/go-diff
|
||||||
|
recommendations:
|
||||||
|
- github.com/aymanbagabas/go-udiff
|
||||||
|
reason: "No unified diff output; use go-udiff."
|
||||||
|
- module: github.com/hexops/gotextdiff
|
||||||
|
recommendations:
|
||||||
|
- github.com/aymanbagabas/go-udiff
|
||||||
|
reason: "Unmaintained fork; use go-udiff."
|
||||||
|
|
||||||
issues:
|
issues:
|
||||||
max-issues-per-linter: 0
|
max-issues-per-linter: 0
|
||||||
|
|||||||
@@ -0,0 +1,2 @@
|
|||||||
|
node_modules/
|
||||||
|
yarn.lock
|
||||||
+113
-58
@@ -1,34 +1,3 @@
|
|||||||
# Lint stage
|
|
||||||
# golangci/golangci-lint:v2.12.2 (Debian-based), 2026-08-07
|
|
||||||
# Using Debian-based image because mattn/go-sqlite3 (CGO) does not
|
|
||||||
# compile on Alpine musl (off64_t is a glibc type).
|
|
||||||
FROM golangci/golangci-lint:v2.12.2@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240 AS lint
|
|
||||||
|
|
||||||
WORKDIR /src
|
|
||||||
|
|
||||||
# Copy go mod files first for better layer caching
|
|
||||||
COPY go.mod go.sum ./
|
|
||||||
RUN go mod download
|
|
||||||
|
|
||||||
# Copy source code. In CI the context also carries .ci-fingerprint, which
|
|
||||||
# holds the hash of the commit being checked (see
|
|
||||||
# .gitea/workflows/check.yml). That invalidates this layer, so the checks
|
|
||||||
# below cannot report success by replaying a cached pass. Do not add it to
|
|
||||||
# .dockerignore.
|
|
||||||
COPY . .
|
|
||||||
|
|
||||||
# Run the Go formatting check and the linter. gofmt and golangci-lint are
|
|
||||||
# invoked directly rather than through `make fmt-check` and `make lint`: this
|
|
||||||
# stage is already the pinned linter image, and both scripts build docker
|
|
||||||
# stages, so calling them here would need a docker daemon inside the build.
|
|
||||||
# The Markdown half of `make fmt-check` is the markdown-check stage below.
|
|
||||||
# Keep the golangci-lint steps in step with Dockerfile.lint, including
|
|
||||||
# --network=none (see its header for why).
|
|
||||||
RUN if [ -n "$(gofmt -s -l .)" ]; then echo "gofmt needed on:"; gofmt -s -l .; exit 1; fi
|
|
||||||
RUN script/assets
|
|
||||||
RUN --network=none golangci-lint config verify --config .golangci.yml
|
|
||||||
RUN --network=none golangci-lint run --config .golangci.yml --build-tags browser ./...
|
|
||||||
|
|
||||||
# Stylesheet stages. static/css/tailwind.css is generated, by this pinned
|
# Stylesheet stages. static/css/tailwind.css is generated, by this pinned
|
||||||
# tailwindcss, from static/css/input.css and the files its @source lines
|
# tailwindcss, from static/css/input.css and the files its @source lines
|
||||||
# name. `make css` (script/css) writes it out from the css-output stage.
|
# name. `make css` (script/css) writes it out from the css-output stage.
|
||||||
@@ -67,10 +36,7 @@ RUN sed 's/}/}\n/g' static/css/tailwind.css > /tmp/committed.css \
|
|||||||
|
|
||||||
# JavaScript lint stages: ESLint, at the version package.json and yarn.lock
|
# JavaScript lint stages: ESLint, at the version package.json and yarn.lock
|
||||||
# pin, checks static/js/ against eslint.config.mjs. js-deps installs it, and
|
# pin, checks static/js/ against eslint.config.mjs. js-deps installs it, and
|
||||||
# prettier for the Markdown stages below, and stays cached until package.json,
|
# prettier for the Markdown stages below. The lint phase below runs js-lint.
|
||||||
# yarn.lock or .yarnrc.yml changes. script/lint forces only js-lint to re-run,
|
|
||||||
# and the build stage below runs it too. COPY . . brings in the CI cache
|
|
||||||
# barrier described in the lint stage above.
|
|
||||||
#
|
#
|
||||||
# The image's own corepack runs the yarn that package.json's packageManager
|
# The image's own corepack runs the yarn that package.json's packageManager
|
||||||
# field names, yarn 4.18.1 (released 2026-09-24), and checks it against the
|
# field names, yarn 4.18.1 (released 2026-09-24), and checks it against the
|
||||||
@@ -104,23 +70,116 @@ FROM js-deps AS markdown-check
|
|||||||
COPY . .
|
COPY . .
|
||||||
RUN --network=none node_modules/.bin/prettier --check '**/*.md'
|
RUN --network=none node_modules/.bin/prettier --check '**/*.md'
|
||||||
|
|
||||||
|
# Lint phase: the Go formatting check and golangci-lint over the Go code,
|
||||||
|
# and ESLint over static/js/ through the copy from js-lint at the end.
|
||||||
|
# `make lint` (script/lint) builds this stage alone; the build stage below
|
||||||
|
# depends on it.
|
||||||
|
#
|
||||||
|
# golangci/golangci-lint:v2.14.0 (Debian-based), 2026-09-24
|
||||||
|
# Using Debian-based image because mattn/go-sqlite3 (CGO) does not
|
||||||
|
# compile on Alpine musl (off64_t is a glibc type).
|
||||||
|
FROM golangci/golangci-lint:v2.14.0@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint
|
||||||
|
|
||||||
|
WORKDIR /src
|
||||||
|
|
||||||
|
# Copy go mod files first for better layer caching
|
||||||
|
COPY go.mod go.sum ./
|
||||||
|
RUN go mod download
|
||||||
|
|
||||||
|
COPY . .
|
||||||
|
|
||||||
|
# gofmt and golangci-lint are invoked directly rather than through `make
|
||||||
|
# fmt-check` and `make lint`, which are themselves docker builds and would
|
||||||
|
# need a docker daemon inside this one. The Markdown half of `make
|
||||||
|
# fmt-check` is the markdown-check stage above.
|
||||||
|
RUN if [ -n "$(gofmt -s -l .)" ]; then echo "gofmt needed on:"; gofmt -s -l .; exit 1; fi
|
||||||
|
|
||||||
|
# static/static.go embeds the Alpine.js file this extracts from 3p/; without
|
||||||
|
# it the static package does not compile and cannot be linted.
|
||||||
|
RUN script/assets
|
||||||
|
|
||||||
|
# The golangci-lint steps run with --network=none. `golangci-lint config
|
||||||
|
# verify` is documented as fetching its JSON schema over HTTPS; this pinned
|
||||||
|
# image resolves the schema without any network, and --network=none enforces
|
||||||
|
# that. It also proves no linter reaches out at analysis time.
|
||||||
|
#
|
||||||
|
# `run` silently ignores config keys it does not recognize, so a typo would
|
||||||
|
# disable a setting without a word. `config verify` is what catches that.
|
||||||
|
RUN --network=none golangci-lint config verify --config .golangci.yml
|
||||||
|
# --build-tags browser also lints the browser test, which is built only with
|
||||||
|
# that tag (make test-browser).
|
||||||
|
RUN --network=none golangci-lint run --config .golangci.yml --build-tags browser ./...
|
||||||
|
|
||||||
|
# Nothing is wanted from js-lint; the copy is what makes this phase run it.
|
||||||
|
COPY --from=js-lint /src/yarn.lock /dev/null
|
||||||
|
|
||||||
|
# Test phase. -race needs cgo and so a C compiler, which the Debian Go image
|
||||||
|
# ships and the alpine one does not. `make test` (script/test) builds this
|
||||||
|
# stage alone; the build stage below depends on it.
|
||||||
|
#
|
||||||
|
# golang:1.26.1-bookworm (Debian-based), 2026-03-17
|
||||||
|
FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4 AS test
|
||||||
|
|
||||||
|
WORKDIR /src
|
||||||
|
|
||||||
|
COPY go.mod go.sum ./
|
||||||
|
RUN go mod download
|
||||||
|
|
||||||
|
COPY . .
|
||||||
|
|
||||||
|
# static/static.go embeds the Alpine.js file this extracts from 3p/.
|
||||||
|
RUN script/assets
|
||||||
|
|
||||||
|
# -timeout applies to each package on its own, so 90s has only to clear the
|
||||||
|
# slowest one. -p 4 -parallel 8 keep the run under 2 GB of memory: at most
|
||||||
|
# four test binaries build or run at once, each with at most eight parallel
|
||||||
|
# tests. Under -race every test binary and every link costs a few hundred MB,
|
||||||
|
# so the defaults (one per core) add up to several GB on a many-core host.
|
||||||
|
#
|
||||||
|
# The first run has no -v: go test then prints one result line per package,
|
||||||
|
# with its coverage, and for a package that fails, everything its tests
|
||||||
|
# wrote. Verbose output from the whole suite passes the 2 MiB at which the
|
||||||
|
# Docker build cuts off a step's log, so on a failure only the tests that
|
||||||
|
# failed run again, with -v. go test reports a failed test as a line starting
|
||||||
|
# "--- FAIL: TestName" (a failed subtest's line is indented, and reruns with
|
||||||
|
# its parent) and a failed package as "FAIL<tab>package/path<tab>...". A
|
||||||
|
# failure that names no test, such as a build error or a timeout, is already
|
||||||
|
# shown in full, so there is nothing to rerun. The step fails after the rerun
|
||||||
|
# whatever its result: the first run already showed the suite is broken.
|
||||||
|
#
|
||||||
|
# TMPDIR, where the tests keep their SQLite databases, is a tmpfs: SQLite
|
||||||
|
# waits for the disk at every commit, and on a busy host that waiting was
|
||||||
|
# about 40% of the slowest package's run time. GOTMPDIR keeps go's own
|
||||||
|
# build files, the test binaries among them, on disk.
|
||||||
|
#
|
||||||
|
# bash with pipefail, so that the first run's status is go test's, not tee's.
|
||||||
|
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
|
||||||
|
RUN --mount=type=tmpfs,target=/tmp/tests,size=512m \
|
||||||
|
export TMPDIR=/tmp/tests GOTMPDIR=/tmp; \
|
||||||
|
go test -race -cover -p 4 -parallel 8 -timeout 90s ./... 2>&1 | tee /tmp/go-test.log && exit 0; \
|
||||||
|
tests="$(awk '/^--- FAIL: / { print $3 }' /tmp/go-test.log | paste -s -d '|' -)"; \
|
||||||
|
packages="$(awk '/^FAIL\t/ { print $2 }' /tmp/go-test.log)"; \
|
||||||
|
if [ -n "$tests" ]; then \
|
||||||
|
echo "--- Rerunning the failed tests with -v for details ---"; \
|
||||||
|
go test -race -v -p 4 -parallel 8 -timeout 90s -run "^($tests)\$" $packages; \
|
||||||
|
fi; \
|
||||||
|
exit 1
|
||||||
|
|
||||||
# Build stage
|
# Build stage
|
||||||
# golang:1.26.1-bookworm (Debian-based), 2026-03-17
|
# golang:1.26.1-bookworm (Debian-based), 2026-03-17
|
||||||
# Using Debian-based image because gorm.io/driver/sqlite pulls in
|
# Using Debian-based image because gorm.io/driver/sqlite pulls in
|
||||||
# mattn/go-sqlite3 (CGO), which does not compile on Alpine musl.
|
# mattn/go-sqlite3 (CGO), which does not compile on Alpine musl. The image
|
||||||
|
# ships git and make, which the version step below uses.
|
||||||
FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4 AS builder
|
FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4 AS builder
|
||||||
|
|
||||||
# Depend on the lint, stylesheet check, JavaScript lint and Markdown check
|
# Nothing is wanted from the lint and test phases or from the stylesheet and
|
||||||
# stages passing
|
# Markdown checks; the copies are what make BuildKit build them first, so
|
||||||
|
# this stage cannot run unless they all passed.
|
||||||
COPY --from=lint /src/go.sum /dev/null
|
COPY --from=lint /src/go.sum /dev/null
|
||||||
|
COPY --from=test /src/go.sum /dev/null
|
||||||
COPY --from=css-check /out/tailwind.css /dev/null
|
COPY --from=css-check /out/tailwind.css /dev/null
|
||||||
COPY --from=js-lint /src/yarn.lock /dev/null
|
|
||||||
COPY --from=markdown-check /src/yarn.lock /dev/null
|
COPY --from=markdown-check /src/yarn.lock /dev/null
|
||||||
|
|
||||||
# jq is a runtime dependency of script/ci-mark-superseded, which the test
|
|
||||||
# suite executes. git is what script/version derives the version with.
|
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends make curl ca-certificates jq git && rm -rf /var/lib/apt/lists/*
|
|
||||||
|
|
||||||
# A build context sent as a tar archive keeps its files' owners, and git
|
# A build context sent as a tar archive keeps its files' owners, and git
|
||||||
# refuses to read a checkout owned by another user. Trust this one
|
# refuses to read a checkout owned by another user. Trust this one
|
||||||
# whoever owns it.
|
# whoever owns it.
|
||||||
@@ -132,31 +191,27 @@ WORKDIR /build
|
|||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN go mod download
|
RUN go mod download
|
||||||
|
|
||||||
# Copy source code, including the .ci-fingerprint cache barrier described in
|
|
||||||
# the lint stage above.
|
|
||||||
COPY . .
|
COPY . .
|
||||||
|
|
||||||
# Run tests and build. Both first run script/assets, which extracts Alpine.js
|
|
||||||
# from its tarball in 3p/.
|
|
||||||
RUN make test
|
|
||||||
|
|
||||||
# Version stamped into the binary: the VERSION build arg when one is
|
# Version stamped into the binary: the VERSION build arg when one is
|
||||||
# given, otherwise what script/version derives from the .git the build
|
# given, otherwise what script/version derives from the .git the build
|
||||||
# context carries, so any `docker build .` of a clone stamps its commit.
|
# context carries, so any `docker build .` of a clone stamps its commit.
|
||||||
# With neither, as from a source tarball, it is "unknown".
|
# With neither, as from a source tarball, it is "unknown".
|
||||||
#
|
|
||||||
# Declared here, below the test step, so a changed version does not
|
|
||||||
# invalidate its cached layer.
|
|
||||||
ARG VERSION
|
ARG VERSION
|
||||||
|
|
||||||
# A context that carries .git must not stamp "unknown": that means git is
|
# A context that carries .git must not stamp an empty version, "dev" or
|
||||||
# missing here or could not read the checkout, and the image could not be
|
# "unknown": that means git is missing here or could not read the
|
||||||
# traced back to its commit.
|
# checkout, and the image could not be traced back to its commit.
|
||||||
RUN if [ -d .git ] && [ "$(make version VERSION="$VERSION")" = unknown ]; then \
|
RUN version="$(make version VERSION="$VERSION")"; \
|
||||||
echo "version is unknown although the build context carries .git" >&2; \
|
if [ -e .git ]; then \
|
||||||
exit 1; \
|
case "$version" in ""|dev|unknown) \
|
||||||
|
echo "version is '$version' although .git is present" >&2; \
|
||||||
|
exit 1 ;; \
|
||||||
|
esac; \
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# Builds through the Makefile's build target, which runs script/assets
|
||||||
|
# (Alpine.js, extracted from its tarball in 3p/) first.
|
||||||
RUN make build VERSION="$VERSION"
|
RUN make build VERSION="$VERSION"
|
||||||
|
|
||||||
# Rebuild with static linking for Alpine runtime.
|
# Rebuild with static linking for Alpine runtime.
|
||||||
|
|||||||
+1
-1
@@ -16,7 +16,7 @@ COPY . .
|
|||||||
|
|
||||||
# The test binary embeds the templates and static files, so the browser
|
# The test binary embeds the templates and static files, so the browser
|
||||||
# stage needs nothing else. -p 4 keeps the compile's memory down, as in
|
# stage needs nothing else. -p 4 keeps the compile's memory down, as in
|
||||||
# script/test.
|
# the test phase of Dockerfile.
|
||||||
RUN make assets && go test -c -p 4 -tags browser -o /browser.test ./internal/server
|
RUN make assets && go test -c -p 4 -tags browser -o /browser.test ./internal/server
|
||||||
|
|
||||||
# chromedp/headless-shell:151.0.7922.109 (Debian trixie), 2026-08-11. The
|
# chromedp/headless-shell:151.0.7922.109 (Debian trixie), 2026-08-11. The
|
||||||
|
|||||||
@@ -1,43 +0,0 @@
|
|||||||
# Lint-only image, built by script/lint. golangci-lint is never installed on
|
|
||||||
# the host: the repo is COPYed into the pinned image and linted as a build
|
|
||||||
# step, so a successful build IS a clean lint. This works even when the docker
|
|
||||||
# daemon is remote and bind mounts are impossible.
|
|
||||||
#
|
|
||||||
# script/lint passes --no-cache-filter=lint. Without it an unchanged tree
|
|
||||||
# replays the lint stage from cache and the build succeeds in under a second
|
|
||||||
# having run no linter at all. Do not drop that flag.
|
|
||||||
#
|
|
||||||
# The lint steps run with --network=none. `golangci-lint config verify` is
|
|
||||||
# documented as fetching its JSON schema over HTTPS, which would make linting
|
|
||||||
# depend on an unpinned remote artifact; this pinned image resolves the schema
|
|
||||||
# without any network, and --network=none enforces that rather than trusting
|
|
||||||
# it. It also proves no linter reaches out at analysis time. If a future image
|
|
||||||
# bump makes either step need the network, this build fails loudly instead of
|
|
||||||
# quietly acquiring an unpinned dependency.
|
|
||||||
|
|
||||||
# golangci/golangci-lint:v2.12.2 (Debian-based), 2026-08-07
|
|
||||||
# Using Debian-based image because mattn/go-sqlite3 (CGO) does not
|
|
||||||
# compile on Alpine musl (off64_t is a glibc type).
|
|
||||||
FROM golangci/golangci-lint:v2.12.2@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240 AS deps
|
|
||||||
|
|
||||||
WORKDIR /src
|
|
||||||
|
|
||||||
# Copy go mod files first for better layer caching. This stage is cacheable;
|
|
||||||
# only the lint stage below is forced to re-execute.
|
|
||||||
COPY go.mod go.sum ./
|
|
||||||
RUN go mod download
|
|
||||||
|
|
||||||
FROM deps AS lint
|
|
||||||
|
|
||||||
COPY . .
|
|
||||||
|
|
||||||
# static/static.go embeds the Alpine.js file this extracts from 3p/; without
|
|
||||||
# it the static package does not compile and cannot be linted.
|
|
||||||
RUN script/assets
|
|
||||||
|
|
||||||
# `run` silently ignores config keys it does not recognize, so a typo would
|
|
||||||
# disable a setting without a word. `config verify` is what catches that.
|
|
||||||
RUN --network=none golangci-lint config verify --config .golangci.yml
|
|
||||||
# --build-tags browser also lints the browser test, which is built only with
|
|
||||||
# that tag (make test-browser).
|
|
||||||
RUN --network=none golangci-lint run --config .golangci.yml --build-tags browser ./...
|
|
||||||
@@ -19,6 +19,10 @@ override VERSION := $(or $(strip $(VERSION)),$(shell script/version))
|
|||||||
# Extra linker flags for the build target. The static relink in the
|
# Extra linker flags for the build target. The static relink in the
|
||||||
# Dockerfile adds -extldflags here rather than passing its own -ldflags,
|
# Dockerfile adds -extldflags here rather than passing its own -ldflags,
|
||||||
# so composing flags cannot drop the version stamp.
|
# so composing flags cannot drop the version stamp.
|
||||||
|
#
|
||||||
|
# The build target itself always passes -trimpath and -s -w, as the Go
|
||||||
|
# Dockerfile in REPO_POLICIES.md does: no build paths, symbol table or
|
||||||
|
# debug information in the binary.
|
||||||
GO_LDFLAGS ?=
|
GO_LDFLAGS ?=
|
||||||
|
|
||||||
bootstrap:
|
bootstrap:
|
||||||
@@ -49,7 +53,7 @@ check:
|
|||||||
@script/check
|
@script/check
|
||||||
|
|
||||||
build: assets
|
build: assets
|
||||||
go build -ldflags '$(strip -X main.version=$(VERSION) $(GO_LDFLAGS))' -o bin/webhooker ./cmd/webhooker
|
go build -trimpath -ldflags '$(strip -s -w -X main.version=$(VERSION) $(GO_LDFLAGS))' -o bin/webhooker ./cmd/webhooker
|
||||||
|
|
||||||
run: build
|
run: build
|
||||||
./bin/webhooker
|
./bin/webhooker
|
||||||
|
|||||||
@@ -17,14 +17,14 @@ before deploying one.
|
|||||||
### Prerequisites
|
### Prerequisites
|
||||||
|
|
||||||
- Go 1.26.1+ (the version in `go.mod`)
|
- Go 1.26.1+ (the version in `go.mod`)
|
||||||
- Docker (for `make lint`, `make fmt` and `make css`, and so for `make check`,
|
- Docker (for `make test`, `make lint`, `make fmt` and `make css`, and so for
|
||||||
for the browser test in `make test-browser`, for the CI gate, and for
|
`make check`, for the browser test in `make test-browser`, for the CI gate,
|
||||||
containerized deployment)
|
and for containerized deployment)
|
||||||
|
|
||||||
golangci-lint is not a prerequisite and must not be installed on the host:
|
golangci-lint is not a prerequisite and must not be installed on the host:
|
||||||
`script/bootstrap` does not install it, and `make lint` runs the digest-pinned
|
`script/bootstrap` does not install it, and `make lint` runs the digest-pinned
|
||||||
linter image via `Dockerfile.lint`. The same holds for tailwindcss (see
|
linter image in the Dockerfile's `lint` phase. The same holds for tailwindcss
|
||||||
[Stylesheet](#stylesheet)). ESLint, prettier, node and yarn are not
|
(see [Stylesheet](#stylesheet)). ESLint, prettier, node and yarn are not
|
||||||
prerequisites either, and `make lint` and `make fmt` never use a host copy of
|
prerequisites either, and `make lint` and `make fmt` never use a host copy of
|
||||||
them (see [Linting](#linting)).
|
them (see [Linting](#linting)).
|
||||||
|
|
||||||
@@ -43,8 +43,9 @@ make check
|
|||||||
|
|
||||||
# Run the server from the clone. DATA_DIR defaults to
|
# Run the server from the clone. DATA_DIR defaults to
|
||||||
# /var/lib/webhooker in every environment, so set it (in .env or the
|
# /var/lib/webhooker in every environment, so set it (in .env or the
|
||||||
# shell) to a writable directory.
|
# shell) to a writable directory outside the clone: the databases hold
|
||||||
DATA_DIR=./data make dev
|
# the session key.
|
||||||
|
DATA_DIR=../webhooker-data make dev
|
||||||
|
|
||||||
# Build Docker image
|
# Build Docker image
|
||||||
make docker
|
make docker
|
||||||
@@ -55,11 +56,11 @@ make docker
|
|||||||
```bash
|
```bash
|
||||||
make bootstrap # Install all dependencies (idempotent)
|
make bootstrap # Install all dependencies (idempotent)
|
||||||
make setup # Bootstrap + install git pre-commit hook
|
make setup # Bootstrap + install git pre-commit hook
|
||||||
make assets # Extract Alpine.js from 3p/ (test, check, build, dev run it)
|
make assets # Extract Alpine.js from 3p/ (build and dev run it)
|
||||||
make fmt # Format Go (gofmt + goimports) and Markdown (prettier, in Docker)
|
make fmt # Format Go (gofmt + goimports) and Markdown (prettier, in Docker)
|
||||||
make fmt-check # Fail if gofmt or prettier would change anything (writes nothing)
|
make fmt-check # Fail if gofmt or prettier would change anything (writes nothing)
|
||||||
make lint # Run golangci-lint and ESLint in Docker
|
make lint # Run golangci-lint and ESLint in Docker
|
||||||
make test # Run tests with race detection
|
make test # Run tests with race detection, in Docker
|
||||||
make test-browser # Run the browser test in Docker (Dockerfile.browser)
|
make test-browser # Run the browser test in Docker (Dockerfile.browser)
|
||||||
make check # test + lint + fmt-check + css-check (CI gate)
|
make check # test + lint + fmt-check + css-check (CI gate)
|
||||||
make build # Build binary to bin/webhooker (version-stamped)
|
make build # Build binary to bin/webhooker (version-stamped)
|
||||||
@@ -1095,7 +1096,9 @@ field), in the UI footer, and in the startup log line (`msg=starting`,
|
|||||||
The value is stamped in at build time by the linker; it is not read from a file
|
The value is stamped in at build time by the linker; it is not read from a file
|
||||||
at runtime, so it identifies the build itself.
|
at runtime, so it identifies the build itself.
|
||||||
|
|
||||||
`script/version` produces the value and both build paths use it:
|
`script/version` produces the value for `make build`, and the image build runs
|
||||||
|
`make build` too; `script/docker` and `script/cibuild` run the same
|
||||||
|
`git describe --tags --always --dirty` on the host. All of them report:
|
||||||
|
|
||||||
| Build | What it reports |
|
| Build | What it reports |
|
||||||
| ----------------------- | --------------------------------------------------- |
|
| ----------------------- | --------------------------------------------------- |
|
||||||
@@ -1110,18 +1113,19 @@ carries, so any `docker build .` of a clone, with no build arguments, stamps the
|
|||||||
commit it was built from; a shallow clone of one branch has no tags and stamps
|
commit it was built from; a shallow clone of one branch has no tags and stamps
|
||||||
the short SHA. `.dockerignore` must therefore leave out neither `.git` nor any
|
the short SHA. `.dockerignore` must therefore leave out neither `.git` nor any
|
||||||
tracked file, which git in the build would see as deleted, marking the version
|
tracked file, which git in the build would see as deleted, marking the version
|
||||||
`-dirty`. It does leave `.git/config`, which can hold a remote URL carrying a
|
`-dirty`. It does leave out every git `config` (`**/.git/config`,
|
||||||
credential and which `git describe` does not need, out of a directory context. A
|
`**/.git/modules/**/config`), which can hold a remote URL carrying a credential
|
||||||
context sent as a tar is not filtered by `.dockerignore`, so it carries
|
and which `git describe` does not need, from a directory context. A context sent
|
||||||
`.git/config` unless its sender leaves it out; for upaas, that is
|
as a tar is not filtered by `.dockerignore`, so it carries `.git/config` unless
|
||||||
|
its sender leaves it out; for upaas, that is
|
||||||
https://git.eeqj.de/sneak/upaas/issues/274. git in the build reads the checkout
|
https://git.eeqj.de/sneak/upaas/issues/274. git in the build reads the checkout
|
||||||
whoever owns its files, since a context sent as a tar archive keeps the sender's
|
whoever owns its files, since a context sent as a tar archive keeps the sender's
|
||||||
owners and git otherwise refuses a checkout owned by another user. A `VERSION`
|
owners and git otherwise refuses a checkout owned by another user. A `VERSION`
|
||||||
build arg (`--build-arg VERSION=...`) takes precedence; `script/docker` (and so
|
build arg (`--build-arg VERSION=...`) takes precedence; `script/docker` (and so
|
||||||
`make docker`) passes the one `script/version` resolves on the host. The image
|
`make docker`) and `script/cibuild` pass the one they resolve on the host, or
|
||||||
build fails if its context carries `.git` and the version still comes out
|
`unknown` where git gives none. The image build fails if its context carries
|
||||||
`unknown`, which means git is missing from the build or could not read the
|
`.git` and the version still comes out empty, `dev` or `unknown`, which means
|
||||||
checkout.
|
git is missing from the build or could not read the checkout.
|
||||||
|
|
||||||
`unknown` is what a source tarball, or a `docker build` with no `.git` in its
|
`unknown` is what a source tarball, or a `docker build` with no `.git` in its
|
||||||
context and no `VERSION` build arg, reports. A build that reports `unknown` is a
|
context and no `VERSION` build arg, reports. A build that reports `unknown` is a
|
||||||
@@ -1135,7 +1139,9 @@ to a commit.
|
|||||||
|
|
||||||
Nothing that varies between two builds of the same commit is stamped — no
|
Nothing that varies between two builds of the same commit is stamped — no
|
||||||
timestamp, no hostname, no builder identity — so two builds of one commit still
|
timestamp, no hostname, no builder identity — so two builds of one commit still
|
||||||
produce a byte-identical binary.
|
produce a byte-identical binary. `make build` passes `-trimpath`, so the
|
||||||
|
directory it builds in is not recorded either, and `-s -w`, which leave out the
|
||||||
|
symbol table and debug information.
|
||||||
|
|
||||||
### Backups contain secrets
|
### Backups contain secrets
|
||||||
|
|
||||||
@@ -1213,11 +1219,14 @@ commands with no script behind them, though `build`, `run` and `dev` first run
|
|||||||
`script/assets`, and `build` and `version` both take their value from
|
`script/assets`, and `build` and `version` both take their value from
|
||||||
`script/version`.
|
`script/version`.
|
||||||
|
|
||||||
`script/test`, `make build` and `make dev` each run `script/assets` first, which
|
`make build` and `make dev` each run `script/assets` first, which writes the
|
||||||
writes the ignored `static/js/alpine.min.js` (see
|
uncommitted `static/js/alpine.min.js` (see
|
||||||
[Third-party browser assets](#third-party-browser-assets)), so `make test`,
|
[Third-party browser assets](#third-party-browser-assets)), so they work on a
|
||||||
`make check` and the pre-commit hook work on a fresh clone without a separate
|
fresh clone without a separate step. The Docker stages that compile the code run
|
||||||
step.
|
it themselves.
|
||||||
|
|
||||||
|
Every `docker build` in `script/` passes `--no-cache`: a check served from the
|
||||||
|
build cache is a check that did not run.
|
||||||
|
|
||||||
We provide:
|
We provide:
|
||||||
|
|
||||||
@@ -1227,10 +1236,12 @@ We provide:
|
|||||||
- `script/projectname` — output the project name ("webhooker")
|
- `script/projectname` — output the project name ("webhooker")
|
||||||
- `script/assets` — extract Alpine.js from its tarball in `3p/` (see
|
- `script/assets` — extract Alpine.js from its tarball in `3p/` (see
|
||||||
[Third-party browser assets](#third-party-browser-assets))
|
[Third-party browser assets](#third-party-browser-assets))
|
||||||
- `script/test` — run the test suite
|
- `script/test` — run the test suite: builds the Dockerfile's `test` phase,
|
||||||
|
tagged `webhooker-test`
|
||||||
- `script/test-browser` — run the browser test in Docker (see
|
- `script/test-browser` — run the browser test in Docker (see
|
||||||
[Third-party browser assets](#third-party-browser-assets))
|
[Third-party browser assets](#third-party-browser-assets))
|
||||||
- `script/lint` — run golangci-lint and ESLint in Docker (see Linting below)
|
- `script/lint` — run the `gofmt` check, golangci-lint and ESLint: builds the
|
||||||
|
Dockerfile's `lint` phase, tagged `webhooker-lint` (see Linting below)
|
||||||
- `script/fmt` — format the Go code and, in Docker, the Markdown (writes)
|
- `script/fmt` — format the Go code and, in Docker, the Markdown (writes)
|
||||||
- `script/fmt-check` — check formatting (read-only)
|
- `script/fmt-check` — check formatting (read-only)
|
||||||
- `script/css` — regenerate `static/css/tailwind.css` in Docker (writes; see
|
- `script/css` — regenerate `static/css/tailwind.css` in Docker (writes; see
|
||||||
@@ -1241,11 +1252,11 @@ We provide:
|
|||||||
- `script/version` — output the version to stamp into the binary (see
|
- `script/version` — output the version to stamp into the binary (see
|
||||||
[Version stamping](#version-stamping))
|
[Version stamping](#version-stamping))
|
||||||
- `script/docker` — build the Docker image tagged via `script/projectname`,
|
- `script/docker` — build the Docker image tagged via `script/projectname`,
|
||||||
passing `script/version`'s output in as the `VERSION` build arg
|
passing the version `git describe` gives on the host in as the `VERSION` build
|
||||||
- `script/cibuild` — CI entrypoint: `docker build .` (the Dockerfile runs the
|
arg
|
||||||
checks, so a green build implies a green repo)
|
- `script/cibuild` — CI entrypoint: `script/bootstrap`, then `script/check`,
|
||||||
- `script/ci-mark-superseded` — CI helper: mark the commits whose run a newer
|
then the same image build as `script/docker`, whose gate phases run again (see
|
||||||
push cancelled (see [CI gate honesty](#ci-gate-honesty))
|
[CI gate honesty](#ci-gate-honesty))
|
||||||
- `script/precommit` — pre-commit checks (`go mod tidy` guard, then
|
- `script/precommit` — pre-commit checks (`go mod tidy` guard, then
|
||||||
`script/check`)
|
`script/check`)
|
||||||
- `script/install-precommit` — install the git pre-commit hook that runs
|
- `script/install-precommit` — install the git pre-commit hook that runs
|
||||||
@@ -1298,12 +1309,12 @@ apply. The directory is `3p/` rather than `vendor/` because Go treats a root
|
|||||||
|
|
||||||
`script/assets` (`make assets`) extracts the browser build,
|
`script/assets` (`make assets`) extracts the browser build,
|
||||||
`package/dist/cdn.min.js`, from the tarball to `static/js/alpine.min.js`, where
|
`package/dist/cdn.min.js`, from the tarball to `static/js/alpine.min.js`, where
|
||||||
`go:embed` picks it up. `script/test`, `make build` and `make dev` run it first,
|
`go:embed` picks it up. `make build` and `make dev` run it first, and so do the
|
||||||
and the Dockerfile builds through `make test` and `make build`, so nothing
|
Dockerfile's lint, test and build stages, so nothing downloads Alpine.js. The
|
||||||
downloads Alpine.js. The extracted file is not committed, and `.dockerignore`
|
extracted file is not committed, and `.dockerignore` keeps any host copy out of
|
||||||
keeps any host copy out of the build context. `static/static.go` names every
|
the build context. `static/static.go` names every file it embeds, so a build
|
||||||
file it embeds, so a build that skips the extraction, such as a bare `go build`,
|
that skips the extraction, such as a bare `go build`, fails with an error naming
|
||||||
fails with an error naming `js/alpine.min.js`.
|
`js/alpine.min.js`.
|
||||||
|
|
||||||
To move to a new version: download
|
To move to a new version: download
|
||||||
`https://registry.npmjs.org/@alpinejs/csp/-/csp-<version>.tgz`, check it against
|
`https://registry.npmjs.org/@alpinejs/csp/-/csp-<version>.tgz`, check it against
|
||||||
@@ -2969,13 +2980,11 @@ webhooker/
|
|||||||
├── internal/
|
├── internal/
|
||||||
│ ├── banner/
|
│ ├── banner/
|
||||||
│ │ └── banner.go # Ruled block for the one credential shown in the clear
|
│ │ └── banner.go # Ruled block for the one credential shown in the clear
|
||||||
│ ├── ciscript/
|
|
||||||
│ │ └── doc.go # Tests for the CI shell scripts in script/; no runtime code
|
|
||||||
│ ├── resetpw/
|
│ ├── resetpw/
|
||||||
│ │ └── resetpw.go # `webhooker resetpw`: set an account's password, stopped deployments only
|
│ │ └── resetpw.go # `webhooker resetpw`: set an account's password, stopped deployments only
|
||||||
│ ├── config/
|
│ ├── config/
|
||||||
│ │ ├── config.go # Configuration loading from environment variables
|
│ │ ├── config.go # Configuration loading from environment variables
|
||||||
│ │ └── testing.go # ClearEnvForTest: an empty environment for one test
|
│ │ └── configtest/ # Test support: ClearEnv, an empty environment for one test
|
||||||
│ ├── database/
|
│ ├── database/
|
||||||
│ │ ├── base_model.go # BaseModel with UUID primary keys
|
│ │ ├── base_model.go # BaseModel with UUID primary keys
|
||||||
│ │ ├── database.go # GORM connection, migrations, admin seed
|
│ │ ├── database.go # GORM connection, migrations, admin seed
|
||||||
@@ -2992,8 +3001,8 @@ webhooker/
|
|||||||
│ │ ├── model_apikey.go # APIKey entity
|
│ │ ├── model_apikey.go # APIKey entity
|
||||||
│ │ ├── password.go # Argon2id hashing and verification
|
│ │ ├── password.go # Argon2id hashing and verification
|
||||||
│ │ ├── retention.go # Retention reaper (per-webhook event expiry)
|
│ │ ├── retention.go # Retention reaper (per-webhook event expiry)
|
||||||
│ │ ├── testing.go # NewTestDatabase: wrapper for tests, no fx lifecycle
|
│ │ ├── webhook_db_manager.go # Per-webhook DB lifecycle manager
|
||||||
│ │ └── webhook_db_manager.go # Per-webhook DB lifecycle manager
|
│ │ └── databasetest/ # Test support: a WebhookDBManager for tests in other packages
|
||||||
│ ├── datadir/
|
│ ├── datadir/
|
||||||
│ │ └── lock.go # Exclusive advisory lock on DATA_DIR (one instance)
|
│ │ └── lock.go # Exclusive advisory lock on DATA_DIR (one instance)
|
||||||
│ ├── globals/
|
│ ├── globals/
|
||||||
@@ -3054,7 +3063,7 @@ webhooker/
|
|||||||
│ │ ├── csrf.go # CSRF protection middleware (gorilla/csrf)
|
│ │ ├── csrf.go # CSRF protection middleware (gorilla/csrf)
|
||||||
│ │ ├── ratelimit.go # Per-IP rate limiting middleware (go-chi/httprate)
|
│ │ ├── ratelimit.go # Per-IP rate limiting middleware (go-chi/httprate)
|
||||||
│ │ ├── loginguard.go # Login failure counters and the Argon2id verification semaphore
|
│ │ ├── loginguard.go # Login failure counters and the Argon2id verification semaphore
|
||||||
│ │ └── testing.go # NewForTest: Middleware without the fx lifecycle
|
│ │ └── middlewaretest/ # Test support: a Middleware for tests in other packages
|
||||||
│ ├── reqtls/
|
│ ├── reqtls/
|
||||||
│ │ └── reqtls.go # IsTLS: the one TLS predicate, r.TLS or X-Forwarded-Proto
|
│ │ └── reqtls.go # IsTLS: the one TLS predicate, r.TLS or X-Forwarded-Proto
|
||||||
│ ├── server/
|
│ ├── server/
|
||||||
@@ -3062,8 +3071,7 @@ webhooker/
|
|||||||
│ │ ├── http.go # HTTP server setup with timeouts
|
│ │ ├── http.go # HTTP server setup with timeouts
|
||||||
│ │ └── routes.go # All route definitions
|
│ │ └── routes.go # All route definitions
|
||||||
│ ├── session/
|
│ ├── session/
|
||||||
│ │ ├── session.go # Cookie-based session management
|
│ │ └── session.go # Cookie-based session management
|
||||||
│ │ └── testing.go # NewForTest: Session without the fx lifecycle
|
|
||||||
│ └── versionscript/
|
│ └── versionscript/
|
||||||
│ └── doc.go # Tests for script/version and the build files that use it
|
│ └── doc.go # Tests for script/version and the build files that use it
|
||||||
├── static/
|
├── static/
|
||||||
@@ -3075,8 +3083,7 @@ webhooker/
|
|||||||
│ └── js/alpine.min.js # Alpine.js CSP build, extracted from 3p/ by make assets, not committed
|
│ └── js/alpine.min.js # Alpine.js CSP build, extracted from 3p/ by make assets, not committed
|
||||||
├── templates/ # Go HTML templates (base, login, sources, etc.)
|
├── templates/ # Go HTML templates (base, login, sources, etc.)
|
||||||
├── script/ # Scripts to Rule Them All entrypoints
|
├── script/ # Scripts to Rule Them All entrypoints
|
||||||
├── Dockerfile # Stages: lint, stylesheet, JavaScript lint, Markdown, test+build, Alpine runtime
|
├── Dockerfile # Stages: stylesheet, JavaScript lint, Markdown, lint, test, build, Alpine runtime
|
||||||
├── Dockerfile.lint # Lint-only image built by script/lint
|
|
||||||
├── Dockerfile.browser # Browser test image built by script/test-browser
|
├── Dockerfile.browser # Browser test image built by script/test-browser
|
||||||
├── Makefile # 13 of 19 targets shim script/; 6 are inline
|
├── Makefile # 13 of 19 targets shim script/; 6 are inline
|
||||||
├── go.mod / go.sum
|
├── go.mod / go.sum
|
||||||
@@ -3084,6 +3091,7 @@ webhooker/
|
|||||||
├── .yarnrc.yml # yarn settings: install into node_modules/
|
├── .yarnrc.yml # yarn settings: install into node_modules/
|
||||||
├── eslint.config.mjs # ESLint configuration for static/js/
|
├── eslint.config.mjs # ESLint configuration for static/js/
|
||||||
├── .prettierrc # prettier settings for the Markdown
|
├── .prettierrc # prettier settings for the Markdown
|
||||||
|
├── .prettierignore # Files prettier skips
|
||||||
└── .golangci.yml # golangci-lint configuration
|
└── .golangci.yml # golangci-lint configuration
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -3339,47 +3347,36 @@ Two operational consequences follow from bounding the sequence:
|
|||||||
|
|
||||||
### Linting
|
### Linting
|
||||||
|
|
||||||
golangci-lint never runs on the host. `script/lint` builds `Dockerfile.lint`,
|
golangci-lint never runs on the host. `script/lint` builds the Dockerfile's
|
||||||
which copies the repo into the digest-pinned golangci-lint image and lints as a
|
`lint` phase, which copies the repo into the digest-pinned golangci-lint image
|
||||||
build step, so a successful build is a clean lint. A host binary would share one
|
and lints as a build step, so a successful build is a clean lint. A host binary
|
||||||
cache and one lock with every other checkout on the machine, which has produced
|
would share one cache and one lock with every other checkout on the machine,
|
||||||
both invented findings attributed to other worktrees and unearned passes.
|
which has produced both invented findings attributed to other worktrees and
|
||||||
|
unearned passes.
|
||||||
|
|
||||||
Three properties are load-bearing:
|
Two properties are load-bearing:
|
||||||
|
|
||||||
- `script/lint` passes `--no-cache-filter=lint`. Without it an unchanged tree
|
- `script/lint` passes `--no-cache`. Without it an unchanged tree replays the
|
||||||
replays the lint layer from cache and the build exits 0 in under a second
|
lint layer from cache and the build exits 0 in under a second having linted
|
||||||
having linted nothing. The `deps` stage stays cacheable, so module downloads
|
nothing. Never prune the shared build cache instead.
|
||||||
are not repeated. Invalidation is scoped to the one stage; never prune the
|
- Both golangci-lint steps use `RUN --network=none`.
|
||||||
shared build cache.
|
`golangci-lint config verify` is documented as fetching its JSON schema over
|
||||||
- `script/lint` does not trust that flag. Docker silently ignores
|
HTTPS, which would be an unpinned remote dependency; the pinned image resolves
|
||||||
`--no-cache-filter` for a stage name that does not match, so a stage rename or
|
the schema without network access, and `--network=none` enforces that instead
|
||||||
a one-character typo would restore the cached false green with no warning and
|
of trusting it. Verify is worth keeping because `golangci-lint run` silently
|
||||||
a fast exit 0. The script therefore tees the build output and treats a run as
|
ignores config keys it does not recognize, so a typo would disable a setting
|
||||||
a pass only if golangci-lint's own summary line (`N issues.` / `N issues:`)
|
with no warning.
|
||||||
appears in it: no summary, no lint, whatever the exit code says.
|
|
||||||
- Both lint steps use `RUN --network=none`. `golangci-lint config verify` is
|
|
||||||
documented as fetching its JSON schema over HTTPS, which would be an unpinned
|
|
||||||
remote dependency; the pinned image resolves the schema without network
|
|
||||||
access, and `--network=none` enforces that instead of trusting it. Verify is
|
|
||||||
worth keeping because `golangci-lint run` silently ignores config keys it does
|
|
||||||
not recognize, so a typo would disable a setting with no warning.
|
|
||||||
|
|
||||||
ESLint never runs on the host either. It lints `static/js/` (not the extracted
|
ESLint never runs on the host either. It lints `static/js/` (not the extracted
|
||||||
Alpine.js) in the Dockerfile's `js-lint` stage, which `script/lint` builds after
|
Alpine.js) in the Dockerfile's `js-lint` stage. The `lint` phase copies a file
|
||||||
`Dockerfile.lint` and the image build runs before the builder stage. Its version
|
from it, so `make lint` and the image build both run ESLint. Its version is
|
||||||
is pinned in `package.json` and every package's hash in `yarn.lock`. The
|
pinned in `package.json` and every package's hash in `yarn.lock`. The `js-deps`
|
||||||
`js-deps` stage before it installs ESLint with `yarn install --immutable`, which
|
stage before it installs ESLint with `yarn install --immutable`, which fails
|
||||||
fails rather than change `yarn.lock`. The yarn it runs is the one the
|
rather than change `yarn.lock`. The yarn it runs is the one the `packageManager`
|
||||||
`packageManager` field in `package.json` pins by version and hash, which the
|
field in `package.json` pins by version and hash, which the node image's own
|
||||||
node image's own corepack fetches and checks. The stage stays cached until
|
corepack fetches and checks. `eslint.config.mjs` turns on the rules of the
|
||||||
`package.json`, `yarn.lock` or `.yarnrc.yml` changes, so only the lint step
|
JavaScript styleguide `REPO_POLICIES.md` links to that a linter can check:
|
||||||
re-runs and ESLint is not downloaded again. `eslint.config.mjs` turns on the
|
`no-var` and `prefer-const`.
|
||||||
rules of the JavaScript styleguide `REPO_POLICIES.md` links to that a linter can
|
|
||||||
check: `no-var` and `prefer-const`. ESLint prints nothing on a pass, so
|
|
||||||
`script/lint` has no summary line to look for; it names the stage once for both
|
|
||||||
`--target` and `--no-cache-filter`, and `--target` fails on a name that matches
|
|
||||||
no stage.
|
|
||||||
|
|
||||||
prettier formats the Markdown, and it never runs on the host either. It is
|
prettier formats the Markdown, and it never runs on the host either. It is
|
||||||
pinned in `package.json` and `yarn.lock` beside ESLint, installed by the same
|
pinned in `package.json` and `yarn.lock` beside ESLint, installed by the same
|
||||||
@@ -3391,103 +3388,81 @@ on any Markdown file prettier would change.
|
|||||||
### Docker
|
### Docker
|
||||||
|
|
||||||
The Dockerfile uses a multi-stage build. Each stage is pinned by digest, and the
|
The Dockerfile uses a multi-stage build. Each stage is pinned by digest, and the
|
||||||
lint and builder stages are separate images so the linter's version is fixed
|
lint phase and the Go stages are separate images so the linter's version is
|
||||||
independently of the compiler's:
|
fixed independently of the compiler's:
|
||||||
|
|
||||||
1. **Lint stage** (`golangci/golangci-lint:v2.12.2`, Debian-based) — downloads
|
1. **Stylesheet stages** (`debian:bookworm-slim`, with the Tailwind standalone
|
||||||
dependencies, copies the source, and runs the `gofmt` check, then
|
|
||||||
`script/assets` to extract Alpine.js from `3p/`, then
|
|
||||||
`golangci-lint config verify` and `golangci-lint run`, both with
|
|
||||||
`--network=none`.
|
|
||||||
2. **Stylesheet stages** (`debian:bookworm-slim`, with the Tailwind standalone
|
|
||||||
CLI pinned by version and sha256, one binary per architecture) — generate
|
CLI pinned by version and sha256, one binary per architecture) — generate
|
||||||
`static/css/tailwind.css` from `static/css/input.css` and the files its
|
`static/css/tailwind.css` from `static/css/input.css` and the files its
|
||||||
`@source` lines name. `css-check` fails when the committed file differs from
|
`@source` lines name. `css-check` fails when the committed file differs from
|
||||||
the generated one, and `make css` writes the generated file out from
|
the generated one, and `make css` writes the generated file out from
|
||||||
`css-output` (see [Stylesheet](#stylesheet)).
|
`css-output` (see [Stylesheet](#stylesheet)).
|
||||||
3. **JavaScript lint stages** (`node:24.21.0-alpine`, with the yarn
|
2. **JavaScript lint stages** (`node:24.21.0-alpine`, with the yarn
|
||||||
`package.json` pins, run through the image's corepack) — `js-deps` installs
|
`package.json` pins, run through the image's corepack) — `js-deps` installs
|
||||||
ESLint and prettier from `yarn.lock` and `js-lint` runs ESLint over
|
ESLint and prettier from `yarn.lock` and `js-lint` runs ESLint over
|
||||||
`static/js/` (see [Linting](#linting)).
|
`static/js/` (see [Linting](#linting)).
|
||||||
4. **Markdown stages** (on `js-deps`) — `markdown-check` runs prettier over the
|
3. **Markdown stages** (on `js-deps`) — `markdown-check` runs prettier over the
|
||||||
Markdown and fails on any file it would change, and `make fmt` writes the
|
Markdown and fails on any file it would change, and `make fmt` writes the
|
||||||
formatted files out from `markdown-output`.
|
formatted files out from `markdown-output`.
|
||||||
5. **Builder stage** (`golang:1.26.1-bookworm`) — depends on the lint,
|
4. **Lint phase** (`lint`, `golangci/golangci-lint:v2.14.0`, Debian-based) —
|
||||||
`css-check`, `js-lint` and `markdown-check` stages passing (it copies a file
|
downloads dependencies, copies the source, and runs the `gofmt` check, then
|
||||||
from each), runs `make test` and `make build` (both extract Alpine.js from
|
`script/assets` to extract Alpine.js from `3p/`, then
|
||||||
`3p/` first), and finally rebuilds the binary with `CGO_ENABLED=1` and static
|
`golangci-lint config verify` and `golangci-lint run`, both with
|
||||||
linking so it runs on musl. Both builds go through `make build`, the relink
|
`--network=none`, and depends on `js-lint` (it copies a file from it).
|
||||||
adding its `-extldflags` via `GO_LDFLAGS`, so neither can drop the `-X` that
|
`make lint` builds this stage alone.
|
||||||
stamps the version. The version is the `VERSION` build arg if one is given,
|
5. **Test phase** (`test`, `golang:1.26.1-bookworm`, whose C compiler `-race`
|
||||||
|
needs) — extracts Alpine.js, then runs `go test -race -cover` at most four
|
||||||
|
packages and eight tests at a time, with a 90-second timeout per package. On
|
||||||
|
a failure it runs only the failed tests again with `-v`, since verbose output
|
||||||
|
from the whole suite would pass the 2 MiB at which the Docker build cuts off
|
||||||
|
a step's log, and then fails. `make test` builds this stage alone.
|
||||||
|
6. **Builder stage** (`golang:1.26.1-bookworm`) — depends on the lint and test
|
||||||
|
phases and the `css-check` and `markdown-check` stages passing (it copies a
|
||||||
|
file from each), runs `make build` (which extracts Alpine.js from `3p/`
|
||||||
|
first), and then rebuilds the binary with `CGO_ENABLED=1` and static linking
|
||||||
|
so it runs on musl. Both builds go through `make build`, the relink adding
|
||||||
|
its `-extldflags` via `GO_LDFLAGS`, so neither can drop the `-X` that stamps
|
||||||
|
the version. The version is the `VERSION` build arg if one is given,
|
||||||
otherwise derived from the `.git` in the context, and the stage fails if a
|
otherwise derived from the `.git` in the context, and the stage fails if a
|
||||||
context with `.git` would stamp `unknown` (see
|
context with `.git` would stamp an empty version, `dev` or `unknown` (see
|
||||||
[Version stamping](#version-stamping)).
|
[Version stamping](#version-stamping)).
|
||||||
6. **Runtime stage** (`alpine:3.21`) — copies the static binary and
|
7. **Runtime stage** (`alpine:3.21`) — copies the static binary and
|
||||||
`deploy/docker-entrypoint.sh`, creates the `/var/lib/webhooker` directory for
|
`deploy/docker-entrypoint.sh`, creates the `/var/lib/webhooker` directory for
|
||||||
all SQLite databases, exposes port 8080, and includes a health check against
|
all SQLite databases, exposes port 8080, and includes a health check against
|
||||||
`/.well-known/healthcheck`. It sets no `USER`: the `ENTRYPOINT` script starts
|
`/.well-known/healthcheck`. It sets no `USER`: the `ENTRYPOINT` script starts
|
||||||
as root, sets the data directory's owner and mode, and runs the app as the
|
as root, sets the data directory's owner and mode, and runs the app as the
|
||||||
non-root `webhooker` user (UID 1000) through `su-exec`.
|
non-root `webhooker` user (UID 1000) through `su-exec`.
|
||||||
|
|
||||||
The lint stage invokes `gofmt` and `golangci-lint` directly rather than
|
The lint and test phases invoke `gofmt`, `golangci-lint` and `go test` directly
|
||||||
`make fmt-check` and `make lint`: it is already the pinned linter image, and
|
rather than `make fmt-check`, `make lint` and `make test`: those targets build
|
||||||
both targets build docker stages, which would need a docker daemon inside this
|
docker stages, which would need a docker daemon inside this build.
|
||||||
build.
|
|
||||||
|
|
||||||
The lint and builder stages use Debian rather than Alpine because
|
The lint phase and the Go stages use Debian rather than Alpine because
|
||||||
`gorm.io/driver/sqlite` pulls in `mattn/go-sqlite3`, which needs CGO and does
|
`gorm.io/driver/sqlite` pulls in `mattn/go-sqlite3`, which needs CGO and does
|
||||||
not compile against musl. Only the final binary is statically linked, which is
|
not compile against musl. Only the final binary is statically linked, which is
|
||||||
what lets it run on the Alpine runtime image.
|
what lets it run on the Alpine runtime image.
|
||||||
|
|
||||||
`script/cibuild` — `docker build .` — is the CI gate: the checks run inside the
|
`script/cibuild` is the CI gate: it runs `script/bootstrap`, then
|
||||||
image, so a build that succeeds is a repo that is formatted, linted, tested and
|
`script/check`, then builds the image, whose build runs the lint and test phases
|
||||||
compiled, with a current stylesheet. `script/lint` also uses Docker
|
and the stylesheet and Markdown checks again. A build that succeeds is a repo
|
||||||
(`Dockerfile.lint` and the `js-lint` stage, see Linting above), so `make lint`
|
that is formatted, linted, tested and compiled, with a current stylesheet.
|
||||||
and `make check` run the same pinned linter versions the gate does; of the steps
|
`make check` runs the same stages the gate does; of its steps, only the `gofmt`
|
||||||
`make check` runs, only `script/test` and the `gofmt` check in
|
check in `script/fmt-check` runs on the host.
|
||||||
`script/fmt-check` run on the host.
|
|
||||||
|
|
||||||
#### CI gate honesty
|
#### CI gate honesty
|
||||||
|
|
||||||
A layer cache lets `docker build .` exit 0 in seconds with the lint and test
|
A layer cache lets `docker build .` exit 0 in seconds with the lint and test
|
||||||
stages replayed rather than executed, which would make a green check
|
stages replayed rather than executed, which would make a green check
|
||||||
meaningless. The `check` workflow therefore writes `.ci-fingerprint` into the
|
meaningless. Every `docker build` in `script/` therefore passes `--no-cache`, so
|
||||||
build context before building. Its value is the hash of the commit being
|
on every run the `gofmt` check, `golangci-lint`, ESLint, the stylesheet check,
|
||||||
checked, so every commit, docs-only ones and a squash merge whose tree matches
|
the Markdown check, `go test` and `make build` really execute. A run that
|
||||||
an already-built branch included, gets a new fingerprint, invalidates the
|
reports success ran them. A bare `docker build .` carries no such guarantee.
|
||||||
`COPY . .` layer of every check stage, and really runs the `gofmt` check,
|
|
||||||
`golangci-lint`, the stylesheet check, ESLint, the Markdown check, `make test`,
|
|
||||||
and `make build`. A run that reports success ran them.
|
|
||||||
|
|
||||||
The module download layer sits above `COPY . .` and stays cached.
|
The `check` workflow is the shared one from `REPO_POLICIES.md`: it checks out
|
||||||
|
the repository and runs `script/cibuild`, nothing else. Gitea cancels an
|
||||||
A separate workflow step, run before the fingerprint is written, covers a second
|
in-flight run when a newer commit lands on the same branch and records that as
|
||||||
way the gate lied: Gitea cancels an in-flight run when a newer commit lands on
|
`failure` / `Has been cancelled`: nothing was verified about that commit, so
|
||||||
the same branch and records that cancellation as a `failure` status, so a commit
|
test the commit itself before concluding anything about it.
|
||||||
nothing ever tested reads as a test result. Cancellation is unconditional
|
|
||||||
server-side for push events, so the superseding run calls
|
|
||||||
`script/ci-mark-superseded`, which rewrites that exact status to `failure` /
|
|
||||||
`Superseded by a newer commit; never tested`.
|
|
||||||
|
|
||||||
The state stays `failure` on purpose: Gitea's combined status folds `skipped`
|
|
||||||
into `success`, so marking a never-tested commit `skipped` made the status API
|
|
||||||
report green for it, indistinguishable from a commit that passed. Reading a
|
|
||||||
commit's status on this repo therefore goes:
|
|
||||||
|
|
||||||
- `success` / `Successful in ...` — the checks ran and passed.
|
|
||||||
- `failure` / `Failing after ...` — the checks ran and failed.
|
|
||||||
- `failure` / `Superseded by a newer commit; never tested` — the run was
|
|
||||||
cancelled, by a newer push or by hand, and nothing was verified about this
|
|
||||||
commit. Test the commit itself before concluding anything about it.
|
|
||||||
|
|
||||||
Genuine failures and successes are never touched, and no status is left
|
|
||||||
`pending`, which would block the commit indefinitely. The step derives its
|
|
||||||
context string from the workflow name, the job **id** and the event. That is
|
|
||||||
deliberately not byte-identical to Gitea's own rule, which uses the job's
|
|
||||||
display `name:` where the runner exports the id, so giving the job a `name:` —
|
|
||||||
or renaming the workflow — makes the derived context stop matching. The step
|
|
||||||
fails loudly when no status on the commit carries that context, so no rename can
|
|
||||||
silently disable the rewrite.
|
|
||||||
|
|
||||||
## TODO
|
## TODO
|
||||||
|
|
||||||
|
|||||||
+353
-90
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
title: Repository Policies
|
title: Repository Policies
|
||||||
last_modified: 2026-08-07
|
last_modified: 2026-10-04
|
||||||
---
|
---
|
||||||
|
|
||||||
This document covers repository structure, tooling, and workflow standards. Code
|
This document covers repository structure, tooling, and workflow standards. Code
|
||||||
@@ -60,17 +60,28 @@ style conventions are in separate documents:
|
|||||||
prerequisite since nvm requires bash. yarn is then pinned via
|
prerequisite since nvm requires bash. yarn is then pinned via
|
||||||
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts";
|
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts";
|
||||||
always exact versions. `script/cibuild` runs the CI build: it changes to the
|
always exact versions. `script/cibuild` runs the CI build: it changes to the
|
||||||
repo root and runs `docker build .`; the Gitea workflow calls it. Four further
|
repo root, runs `script/bootstrap`, runs `script/check`, and builds the image
|
||||||
scripts are our own extensions to the standard: `script/check` runs
|
with the version; the Gitea workflow calls it. **`script/cibuild` runs
|
||||||
`script/test`, `script/lint`, and `script/fmt-check`; `script/precommit` is
|
`script/bootstrap` first**, because the workflow checks out the repo and runs
|
||||||
what the git pre-commit hook runs, and it calls `script/check`;
|
nothing else, while `script/fmt-check` runs the formatter on the host: on a
|
||||||
`script/install-precommit` installs the git pre-commit hook (the `make hooks`
|
pristine checkout with nothing installed the run dies there, after the
|
||||||
target shims to it); and `script/projectname` (literally that filename) simply
|
containerised gates have passed. **The bootstrap alone is not enough**:
|
||||||
outputs the project's name. Scripts that need the name call
|
`script/bootstrap` installs node and yarn under nvm and leaves neither on the
|
||||||
`script/projectname` — e.g. `script/docker` assembles its image tag from it —
|
`PATH` of the shell that called it, so a bare `yarn` still exits 127. The host
|
||||||
so those scripts stay byte-identical across all repos. Repo-type-specific
|
entrypoints that need yarn — `script/fmt` and `script/fmt-check` — therefore
|
||||||
pre-commit extras (e.g. `go mod tidy` verification in Go repos) belong in
|
source nvm for the pinned node version before invoking it, exactly as
|
||||||
`script/precommit`, not in the hook itself. Model scripts are at
|
`script/bootstrap`'s own install step does. A runner carrying nothing but
|
||||||
|
docker and git then gets through `script/check`. Four further scripts are our
|
||||||
|
own extensions to the standard: `script/check` runs `script/test`,
|
||||||
|
`script/lint` and `script/fmt-check`; `script/precommit` is what the git
|
||||||
|
pre-commit hook runs, and it calls `script/check`; `script/install-precommit`
|
||||||
|
installs the git pre-commit hook (the `make hooks` target shims to it); and
|
||||||
|
`script/projectname` (literally that filename) simply outputs the project's
|
||||||
|
name. Scripts that need the name call `script/projectname` — e.g.
|
||||||
|
`script/docker` assembles its image tag from it — so those scripts stay
|
||||||
|
byte-identical across all repos. Repo-type-specific pre-commit extras (e.g.
|
||||||
|
`go mod tidy` verification in Go repos) belong in `script/precommit`, not in
|
||||||
|
the hook itself. Model scripts are at
|
||||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
|
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
|
||||||
must document the provided scripts in an **Entrypoints** section (see the
|
must document the provided scripts in an **Entrypoints** section (see the
|
||||||
README requirements below).
|
README requirements below).
|
||||||
@@ -89,87 +100,198 @@ style conventions are in separate documents:
|
|||||||
contributor should be able to understand the entire development workflow by
|
contributor should be able to understand the entire development workflow by
|
||||||
reading the Makefile.
|
reading the Makefile.
|
||||||
|
|
||||||
- Every repo should have a `Dockerfile`. All Dockerfiles must run `make check`
|
- Every repo should have a `Dockerfile`, and it carries the repo's gates: a
|
||||||
as a build step so the build fails if the branch is not green. For non-server
|
`lint` phase and a `test` phase, with the final stage depending on both so the
|
||||||
repos, the Dockerfile should bring up a development environment and run
|
image cannot be built unless they pass. For non-server repos the final stage
|
||||||
`make check`. For server repos, `make check` should run as an early build
|
brings up a development environment; for server repos it is the runtime image.
|
||||||
stage before the final image is assembled. Dockerfiles install development
|
The gate phases and the build stage start from their pinned base images and
|
||||||
prerequisites by running `script/bootstrap` rather than duplicating installs
|
install what those images lack either inline, as the canonical Go `Dockerfile`
|
||||||
inline; COPY `script/` and the dependency manifests (`package.json` +
|
below does for `git`, or by running `script/bootstrap`, as the `prompts`
|
||||||
`yarn.lock`, `go.mod` + `go.sum`, etc.) before running it so the bootstrap
|
repo's own `Dockerfile` does for its yarn packages. The development
|
||||||
layer stays cached until dependencies change.
|
environment stage installs development prerequisites by running
|
||||||
|
`script/bootstrap` rather than duplicating its installs inline. A stage that
|
||||||
|
runs `script/bootstrap` COPYs `script/` and the dependency manifests
|
||||||
|
(`package.json` + `yarn.lock`, `go.mod` + `go.sum`, etc.) before running it.
|
||||||
|
|
||||||
- **Dockerfiles must use a separate lint stage for fail-fast feedback.** Go
|
- **Linting and testing run in Docker, as phases of the `Dockerfile`.** There is
|
||||||
repos use a multistage build where linting runs in an independent stage based
|
no separate lint file. `script/lint` and `script/test` each build one phase
|
||||||
on the `golangci/golangci-lint` image (pinned by hash). This stage runs
|
and nothing else:
|
||||||
`make fmt-check` and `make lint` before the full build begins. The build stage
|
|
||||||
then declares an explicit dependency on the lint stage via
|
|
||||||
`COPY --from=lint /src/go.sum /dev/null`, which forces BuildKit to complete
|
|
||||||
linting before proceeding to compilation and tests. This ensures lint failures
|
|
||||||
surface in seconds rather than minutes, without blocking on dependency
|
|
||||||
download or compilation in the build stage.
|
|
||||||
|
|
||||||
The standard pattern for a Go repo Dockerfile is:
|
```sh
|
||||||
|
docker build --no-cache --target lint -t "$(script/projectname)-lint" .
|
||||||
|
docker build --no-cache --target test -t "$(script/projectname)-test" .
|
||||||
|
```
|
||||||
|
|
||||||
|
**A stage that is not the last one in the file is built only when the final
|
||||||
|
stage's chain depends on it, or when `--target` names it.** That is why the
|
||||||
|
two gates are always invoked by name here, and why the final stage carries a
|
||||||
|
`COPY --from=` of a harmless file from each of them: without that edge a
|
||||||
|
plain `docker build .` builds the last stage alone and exits 0 having linted
|
||||||
|
and tested nothing.
|
||||||
|
|
||||||
|
**Every `docker build` in `script/` is tagged**, here and in
|
||||||
|
`script/cibuild` and `script/docker`. An untagged build leaves a dangling
|
||||||
|
image behind on every invocation, on every developer host and every CI
|
||||||
|
runner; a tagged one replaces the previous image.
|
||||||
|
|
||||||
|
Inside a phase the tool is invoked directly — `golangci-lint`, `go test`,
|
||||||
|
`eslint`, `prettier` — never through `make lint` or `script/test`, which are
|
||||||
|
themselves a `docker build` and would recurse into a daemon that does not
|
||||||
|
exist in a build step. Formatting is the exception and stays on the host:
|
||||||
|
`script/fmt` writes the working tree, and `script/fmt-check` is its
|
||||||
|
read-only twin.
|
||||||
|
|
||||||
|
**No lint verdict may come from a host invocation of the linter.** On a
|
||||||
|
shared host golangci-lint reads a result cache keyed on file content rather
|
||||||
|
than location, so a second checkout of the same content is served the first
|
||||||
|
one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs
|
||||||
|
exit non-zero with `parallel golangci-lint is running` — a status a caller
|
||||||
|
cannot tell from real findings. Both have produced wrong verdicts in this
|
||||||
|
org, in both directions. A container has its own cache, its own `TMPDIR` and
|
||||||
|
a digest-pinned binary, so neither is reachable.
|
||||||
|
|
||||||
|
- **Any build that runs checks is built with `--no-cache`.** Docker invalidates
|
||||||
|
a `COPY` layer only when the copied content changes, so on an unchanged tree
|
||||||
|
the check `RUN` is served from cache, nothing executes, and the build still
|
||||||
|
exits 0. Every `docker build` in `script/` therefore passes `--no-cache`:
|
||||||
|
`script/lint`, `script/test`, `script/cibuild` and `script/docker` are the
|
||||||
|
four, and there is no fifth — `script/check` runs the two gate phases and
|
||||||
|
`script/fmt-check`, and builds no image of its own. A bare `docker build .` is
|
||||||
|
not evidence that anything ran: a sub-second build reporting success is a
|
||||||
|
cache hit, not a result. Never invalidate by pruning — `docker builder prune`
|
||||||
|
and friends destroy a build cache shared with every other build on the host.
|
||||||
|
When a check is added or changed, prove it works by planting a defect it must
|
||||||
|
catch and watching the run fail on it, then revert the defect. A green run
|
||||||
|
alone shows neither that the check ran nor that it covers what it should.
|
||||||
|
|
||||||
|
- **The gate phases are separate stages, and the build stage depends on both.**
|
||||||
|
The lint phase is based on the `golangci/golangci-lint` image (pinned by
|
||||||
|
hash), so lint failures surface in seconds rather than after a full compile,
|
||||||
|
and the test phase is based on the Debian Go image. The canonical Go repo
|
||||||
|
`Dockerfile`:
|
||||||
|
|
||||||
```dockerfile
|
```dockerfile
|
||||||
# Lint stage — fast feedback on formatting and lint issues
|
# Lint phase
|
||||||
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
|
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
|
||||||
FROM golangci/golangci-lint@sha256:... AS lint
|
FROM golangci/golangci-lint@sha256:... AS lint
|
||||||
WORKDIR /src
|
WORKDIR /src
|
||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN go mod download
|
RUN go mod download
|
||||||
COPY . .
|
COPY . .
|
||||||
RUN make fmt-check
|
RUN golangci-lint run --config .golangci.yml ./...
|
||||||
RUN make lint
|
|
||||||
|
|
||||||
# Build stage
|
# Test phase. -race needs cgo and so a C compiler, which the Debian Go
|
||||||
# golang:1.x-alpine, YYYY-MM-DD
|
# image ships and the alpine one does not.
|
||||||
FROM golang@sha256:... AS builder
|
# golang:1.x, YYYY-MM-DD
|
||||||
|
FROM golang@sha256:... AS test
|
||||||
WORKDIR /src
|
WORKDIR /src
|
||||||
|
|
||||||
# Force BuildKit to run the lint stage before proceeding
|
|
||||||
COPY --from=lint /src/go.sum /dev/null
|
|
||||||
|
|
||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN go mod download
|
RUN go mod download
|
||||||
COPY . .
|
COPY . .
|
||||||
RUN make test
|
RUN go test -timeout 90s -race -cover ./... || \
|
||||||
|
{ echo "--- Rerunning with -v for details ---"; \
|
||||||
|
go test -timeout 90s -race -v ./...; exit 1; }
|
||||||
|
|
||||||
ARG VERSION=dev
|
# Build stage. Nothing is wanted from either phase above; the copies
|
||||||
RUN CGO_ENABLED=0 go build -trimpath \
|
# are what make BuildKit build them first, so this stage cannot run
|
||||||
-ldflags="-s -w -X main.Version=${VERSION}" \
|
# unless lint and test passed.
|
||||||
-o /app ./cmd/app/
|
# golang:1.x-alpine, YYYY-MM-DD
|
||||||
|
FROM golang@sha256:... AS builder
|
||||||
|
COPY --from=lint /src/go.sum /dev/null
|
||||||
|
COPY --from=test /src/go.sum /dev/null
|
||||||
|
RUN apk add --no-cache git
|
||||||
|
# A tar-stream context keeps the sender's file owners, which git refuses.
|
||||||
|
RUN git config --system --add safe.directory /src
|
||||||
|
WORKDIR /src
|
||||||
|
COPY go.mod go.sum ./
|
||||||
|
RUN go mod download
|
||||||
|
COPY . .
|
||||||
|
|
||||||
# Runtime stage
|
# The VERSION build arg when one is given, otherwise
|
||||||
|
# `git describe --tags --always` on the .git in the build context. With
|
||||||
|
# .git present, a version that is still empty, dev or unknown fails the
|
||||||
|
# build: git is missing or could not read the checkout.
|
||||||
|
ARG VERSION
|
||||||
|
RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \
|
||||||
|
if [ -e .git ]; then \
|
||||||
|
case "$VERSION" in ""|dev|unknown) \
|
||||||
|
echo "version is '$VERSION' although .git is present" >&2; \
|
||||||
|
exit 1 ;; \
|
||||||
|
esac; \
|
||||||
|
fi; \
|
||||||
|
CGO_ENABLED=0 go build -trimpath \
|
||||||
|
-ldflags="-s -w -X main.Version=${VERSION}" \
|
||||||
|
-o /app ./cmd/app/
|
||||||
|
|
||||||
|
# Runtime stage, and the last one
|
||||||
FROM alpine@sha256:...
|
FROM alpine@sha256:...
|
||||||
COPY --from=builder /app /usr/local/bin/app
|
COPY --from=builder /app /usr/local/bin/app
|
||||||
ENTRYPOINT ["app"]
|
ENTRYPOINT ["app"]
|
||||||
```
|
```
|
||||||
|
|
||||||
Key points:
|
Key points:
|
||||||
- The lint stage uses the `golangci/golangci-lint` image directly (it
|
- The lint phase uses the `golangci/golangci-lint` image directly (it has
|
||||||
includes both Go and the linter), so there is no need to install the
|
both Go and the linter), so nothing needs installing.
|
||||||
linter separately.
|
- `COPY --from=<phase> /src/go.sum /dev/null` is a no-op copy whose only
|
||||||
- `COPY --from=lint /src/go.sum /dev/null` is a no-op file copy that creates
|
purpose is the ordering edge. BuildKit runs stages in parallel by default,
|
||||||
a stage dependency. BuildKit runs stages in parallel by default; without
|
and a stage nothing depends on is not built at all, so without these two
|
||||||
this line, the build stage would not wait for lint to finish and a lint
|
lines a red gate would not fail the build.
|
||||||
failure might not fail the overall build.
|
- Keep the runtime stage last, and if you add a stage after it, give it the
|
||||||
|
same two copies. A plain `docker build .` builds the last stage's chain
|
||||||
|
and nothing else.
|
||||||
- If the project uses `//go:embed` directives that reference build artifacts
|
- If the project uses `//go:embed` directives that reference build artifacts
|
||||||
(e.g. a web frontend compiled in a separate stage), the lint stage must
|
(e.g. a web frontend compiled in a separate stage), the lint phase must
|
||||||
create placeholder files so the embed directives resolve. Example:
|
create placeholder files so the embed directives resolve. Example:
|
||||||
`RUN mkdir -p web/dist && touch web/dist/index.html web/dist/style.css`.
|
`RUN mkdir -p web/dist && touch web/dist/index.html web/dist/style.css`.
|
||||||
The lint stage should not depend on the actual build output — it exists to
|
- If the project requires CGO or system libraries for linting, install them
|
||||||
fail fast.
|
in the lint phase. The `golangci/golangci-lint` image is Debian-based and
|
||||||
- If the project requires CGO or system libraries for linting (e.g.
|
has no `apk`, so install with `apt-get` under the Debian package name
|
||||||
`vips-dev`), install them in the lint stage with `apk add`.
|
(`libvips-dev`, where alpine says `vips-dev`), and delete the package
|
||||||
- The build stage runs `make test` after compilation setup. Tests run in the
|
lists in the same `RUN`, so the layer does not keep them:
|
||||||
build stage, not the lint stage, because they may require compiled
|
|
||||||
artifacts or heavier dependencies.
|
```dockerfile
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends libvips-dev \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
```
|
||||||
|
|
||||||
|
- `.dockerignore` lets `.git` into the build context. It keeps out every git
|
||||||
|
`config` at any depth (`**/.git/config`, `**/.git/modules/**/config`): the
|
||||||
|
repository's own, each submodule's under `.git/modules/`, and that of a
|
||||||
|
submodule keeping its own `.git` directory. `git describe` does not need
|
||||||
|
them, and each can hold a credential: a password in a remote URL, or the
|
||||||
|
token the CI checkout step stores there. A submodule whose name has a
|
||||||
|
`config` segment (`config`, `deploy/config`, `config/lib`) loses its whole
|
||||||
|
git directory to `**/.git/modules/**/config`, and Go's version stamping
|
||||||
|
then fails the build: give it a name without that segment
|
||||||
|
(`git submodule add --name`). The stage that compiles has `git` (the
|
||||||
|
Debian Go image has it; an alpine one needs `apk add --no-cache git`) and
|
||||||
|
takes the version from the `VERSION` build argument when one is given,
|
||||||
|
otherwise from `git describe --tags --always`. That gives the tag on a
|
||||||
|
tagged commit; on a later commit, the tag, the number of commits since it
|
||||||
|
and the short commit (`v1.2.3-4-gabc1234`); and the short commit when no
|
||||||
|
tag is reachable. The stage that compiles also marks its working directory
|
||||||
|
safe for git (`git config --system --add safe.directory /src`): a context
|
||||||
|
sent as a tar stream keeps the sender's file owners, and git refuses a
|
||||||
|
checkout owned by another user, so the version would come out empty.
|
||||||
|
`ARG VERSION` has no default, and the build fails if the context carries
|
||||||
|
`.git` and the version still comes out empty, `dev` or `unknown`. A plain
|
||||||
|
`docker build .` with no build arguments must succeed; a Dockerfile that
|
||||||
|
refuses an empty build argument drops that refusal and keeps the argument.
|
||||||
|
|
||||||
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
|
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
|
||||||
runs `script/cibuild` (which runs `docker build .`) on push. Since the
|
runs `script/cibuild` on push, and checks out the repo as its only other step.
|
||||||
Dockerfile already runs `make check`, a successful build implies all checks
|
That script bootstraps, runs the gate phases, and then builds the image, so a
|
||||||
pass.
|
successful run means every check passed; a bare `docker build .` does not
|
||||||
|
carry the same guarantee, because its gate phases may come from the cache. The
|
||||||
|
image build is uncached and so runs the gate phases a second time. That is the
|
||||||
|
price of the rule above, and it is worth paying: the image that ships is built
|
||||||
|
from a run of its own gates rather than from a cache entry. A separate
|
||||||
|
workflow limited to `main` by a `branches` list under `on: push` cannot be
|
||||||
|
checked by review: to try a change to it, add the feature branch to that list
|
||||||
|
and push, then remove the branch from the list again before merging. Keep any
|
||||||
|
job in it that publishes behind `if: github.ref_name == 'main'`, so the run
|
||||||
|
from the feature branch publishes nothing.
|
||||||
|
|
||||||
- Use platform-standard formatters: `black` for Python, `prettier` for
|
- Use platform-standard formatters: `black` for Python, `prettier` for
|
||||||
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
|
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
|
||||||
@@ -193,15 +315,17 @@ style conventions are in separate documents:
|
|||||||
suite that exceeds it fails. Under 20 seconds is the target. A suite between
|
suite that exceeds it fails. Under 20 seconds is the target. A suite between
|
||||||
20 and 60 seconds is still green, but the overage must be filed as an
|
20 and 60 seconds is still green, but the overage must be filed as an
|
||||||
improvement bug against that repo. Add a 90-second timeout to the test
|
improvement bug against that repo. Add a 90-second timeout to the test
|
||||||
invocation in the Makefile (`go test -timeout 90s`). The backstop deliberately
|
invocation (`go test -timeout 90s`). The backstop deliberately sits above the
|
||||||
sits above the hard cap so that it catches a genuinely hung test rather than a
|
hard cap so that it catches a genuinely hung test rather than a merely slow
|
||||||
merely slow one.
|
one.
|
||||||
|
|
||||||
- **`make test` should use the conditional verbose rerun pattern.** Run tests
|
- **The test command should use the conditional verbose rerun pattern.** Run
|
||||||
without `-v` (verbose) first. If tests fail, automatically rerun with `-v` to
|
tests without `-v` (verbose) first. If tests fail, automatically rerun with
|
||||||
show full output. This keeps CI logs and `docker build` output clean on
|
`-v` to show full output. This keeps CI logs and `docker build` output clean
|
||||||
success (just package/suite summaries) while providing full diagnostic detail
|
on success (just package/suite summaries) while providing full diagnostic
|
||||||
on failure (every test case, every assertion). The general shell pattern:
|
detail on failure (every test case, every assertion). The command lives in the
|
||||||
|
`test` phase of the `Dockerfile`, since `script/test` builds that phase; the
|
||||||
|
Makefile form below is the same pattern for any repo-local invocation:
|
||||||
|
|
||||||
```makefile
|
```makefile
|
||||||
test:
|
test:
|
||||||
@@ -214,11 +338,26 @@ style conventions are in separate documents:
|
|||||||
|
|
||||||
```makefile
|
```makefile
|
||||||
test:
|
test:
|
||||||
@go test -timeout 90s -race -cover ./... || \
|
@go test -count=1 -timeout 90s -race -cover ./... || \
|
||||||
{ echo "--- Rerunning with -v for details ---"; \
|
{ echo "--- Rerunning with -v for details ---"; \
|
||||||
go test -timeout 90s -race -v ./...; exit 1; }
|
go test -count=1 -timeout 90s -race -v ./...; exit 1; }
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`-count=1` is required on both invocations: it defeats Go's test _result_
|
||||||
|
cache, so neither run can report a stored pass in place of running the
|
||||||
|
tests. It leaves the build cache alone, so it costs the runtime of the suite
|
||||||
|
and no recompilation.
|
||||||
|
|
||||||
|
That cache is Go's own, separate from Docker's layer cache. Go stores a
|
||||||
|
passing result in its cache directory (`GOCACHE`), and when the same tests
|
||||||
|
run again on unchanged code it prints that result, marked `(cached)`,
|
||||||
|
without running them. That matters on a developer's machine, where this
|
||||||
|
target runs and the directory lasts from one run to the next. The `test`
|
||||||
|
phase of the `Dockerfile` needs no `-count=1`: its base image holds no
|
||||||
|
result for this repo's tests and nothing before its `go test` step runs a
|
||||||
|
test, so there is nothing to replay. `--no-cache` (above) is what makes that
|
||||||
|
step run on an unchanged tree.
|
||||||
|
|
||||||
Python example:
|
Python example:
|
||||||
|
|
||||||
```makefile
|
```makefile
|
||||||
@@ -244,10 +383,84 @@ style conventions are in separate documents:
|
|||||||
must be in `.gitignore`. No exceptions.
|
must be in `.gitignore`. No exceptions.
|
||||||
|
|
||||||
- `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`),
|
- `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`),
|
||||||
editor files (`.swp`, `*~`), language build artifacts, and `node_modules/`.
|
editor files (`.swp`, `*~`), in-repo agent scratch directories (`.claude/`),
|
||||||
Fetch the standard `.gitignore` from
|
language build artifacts, and `node_modules/`. Fetch the standard `.gitignore`
|
||||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when setting up
|
from `https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when
|
||||||
a new repo.
|
setting up a new repo. These patterns are written to `.gitignore`'s own
|
||||||
|
semantics, in which an unanchored pattern already matches at every depth; they
|
||||||
|
are not a `.dockerignore` and must not be transplanted into one unmodified.
|
||||||
|
|
||||||
|
- **`.dockerignore` does not use `.gitignore` semantics, and copying patterns
|
||||||
|
across unmodified leaves secrets in the build context.** Docker matches with
|
||||||
|
`moby/patternmatcher`: `filepath.Match` semantics plus a `**` extension, so
|
||||||
|
`*` does not cross `/` and a pattern without a leading `**/` is anchored at
|
||||||
|
the build-context root. A `.dockerignore` listing `.env`, `*.pem` and `*.key`
|
||||||
|
therefore excludes only the copies at the repository root, while `config/.env`
|
||||||
|
and `certs/server.key` still reach the context and can land in an image layer
|
||||||
|
— which is more dangerous than a short file with no secret patterns at all,
|
||||||
|
because it reads as solved and stops anyone looking. Give every
|
||||||
|
depth-independent pattern the `**/` prefix and leave only genuinely
|
||||||
|
root-anchored entries unprefixed: `.claude`, and the repo's own host-built
|
||||||
|
binary, written `/myapp` and never `**/myapp`, which would also match
|
||||||
|
`cmd/myapp/` and delete the package directory from the context. Matching is
|
||||||
|
case-sensitive, and an ALL-CAPS twin per pattern still misses `Server.Key`, so
|
||||||
|
secret names use character ranges — `**/*.[kK][eE][yY]`, `**/*.[pP][eE][mM]`,
|
||||||
|
and likewise for `.envrc` and the extensionless SSH keys. Where such a pattern
|
||||||
|
also catches something the build needs, re-include it with a negation
|
||||||
|
(`!docs/example.env`); deleting the pattern reopens the exposure for every
|
||||||
|
other file it covers. Fetch the standard `.dockerignore` from
|
||||||
|
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.dockerignore` and extend
|
||||||
|
it with the repo's own artifacts.
|
||||||
|
|
||||||
|
- **In-repo agent scratch belongs in both files, written to each file's own
|
||||||
|
semantics.** `.claude/` holds one worktree per in-flight agent — an entire
|
||||||
|
additional checkout of the repo — so under `COPY . .` the build context
|
||||||
|
inflates by a multiple of the repo and another session's unreviewed work can
|
||||||
|
be copied into an image layer. In `.gitignore` the entry is `.claude/`,
|
||||||
|
unanchored. In `.dockerignore` it is `.claude`, anchored and with **no** `**/`
|
||||||
|
prefix, because the prefixed form would also delete any nested directory of
|
||||||
|
that name from the build. Anchoring carries a known gap that the canonical
|
||||||
|
`.dockerignore` states in its own comment, since consuming repos receive the
|
||||||
|
file and not the tracker: the directory is created in the agent's working
|
||||||
|
directory, so a repo running agents in subdirectories still ships
|
||||||
|
`services/api/.claude/` and must add its own anchored entry there.
|
||||||
|
|
||||||
|
- **A plain `docker build .` of a clone stamps the version that
|
||||||
|
`git describe --tags --always` gives**, derived from the `.git` in the build
|
||||||
|
context as the canonical `Dockerfile` above shows. Without its failure check,
|
||||||
|
a missing `git` or an unreadable checkout would leave `-X main.Version=` empty
|
||||||
|
and the build would still exit 0. `script/docker` and `script/cibuild` pass
|
||||||
|
the version they compute on the host; it takes precedence. They do this
|
||||||
|
byte-identically across repos:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# Own line: a failing command substitution inside an argument does not
|
||||||
|
# trip `set -e`, so the inline form degrades to an empty constant.
|
||||||
|
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
||||||
|
[ -n "$version" ] || version="unknown"
|
||||||
|
docker build --no-cache \
|
||||||
|
--build-arg VERSION="$version" \
|
||||||
|
-t "$(script/projectname)" .
|
||||||
|
```
|
||||||
|
|
||||||
|
`--always` makes an untagged repo yield an abbreviated commit hash rather
|
||||||
|
than failing, and the `[ -n "$version" ]` line is the single place the
|
||||||
|
fallback is applied — a live check that fires on a build from an export with
|
||||||
|
no `.git` and on a repository with no commits yet. Do not fold it into the
|
||||||
|
substitution as `|| echo unknown`, which makes the guard unreachable. The
|
||||||
|
Dockerfile's side is `ARG VERSION` in the stage that compiles, declared
|
||||||
|
there because `ARG` is stage-scoped; passing `VERSION` to a repo whose
|
||||||
|
Dockerfile declares no such `ARG` is ignored and costs nothing, which is why
|
||||||
|
the scripts stay byte-identical. One consequence for CI: the standard
|
||||||
|
checkout action clones shallow and fetches no tags, so a repo that embeds a
|
||||||
|
tag-derived version must set `fetch-depth: 0` on its checkout step.
|
||||||
|
|
||||||
|
- **Verify `.dockerignore` by enumerating the image, not by reading the
|
||||||
|
patterns.** Plant files at the root _and_ at least two directories deep, build
|
||||||
|
a probe image that does `COPY . .`, and list what actually landed
|
||||||
|
(`docker run --rm --entrypoint find IMAGE /app`). The `transferring context`
|
||||||
|
size is not a substitute: a nested secret is a few bytes, and BuildKit
|
||||||
|
transfers only the delta from the previous build.
|
||||||
|
|
||||||
- **No build artifacts in version control.** Code-derived data (compiled
|
- **No build artifacts in version control.** Code-derived data (compiled
|
||||||
bundles, minified output, generated assets) must never be committed to the
|
bundles, minified output, generated assets) must never be committed to the
|
||||||
@@ -263,12 +476,56 @@ style conventions are in separate documents:
|
|||||||
- Make all changes on a feature branch. You can do whatever you want on a
|
- Make all changes on a feature branch. You can do whatever you want on a
|
||||||
feature branch.
|
feature branch.
|
||||||
|
|
||||||
- `.golangci.yml` is standardized and must _NEVER_ be modified by an agent, only
|
- `.golangci.yml` is standardized. The vendored copy in a consuming repo must
|
||||||
manually by the user. Fetch from
|
_NEVER_ be modified by an agent: fetch it from
|
||||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.golangci.yml`. The
|
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.golangci.yml` and keep it
|
||||||
canonical golangci-lint version is v2.12.2 (released 2026-05-06), installed
|
byte-identical, so that no repo can quietly loosen its own linting. Linter
|
||||||
commit-pinned via
|
configuration changes are made to the canonical copy in the `prompts` repo and
|
||||||
`go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@c0d3ddc9cf3faa61a4e378e879ece580256d76e5`.
|
reach consuming repos by re-vendoring; an agent may open a PR against
|
||||||
|
canonical, which only the user merges. One list is exempt from byte-identity,
|
||||||
|
because it cannot be written once for every repo: the `deny` list of the
|
||||||
|
`test-support` depguard rule, where a repo names its own test-support packages
|
||||||
|
by full import path. A repo adds entries there and changes nothing else, and a
|
||||||
|
re-vendor carries its entries forward. The canonical golangci-lint version is
|
||||||
|
v2.14.0 (released 2026-09-24), pinned as the digest of the lint phase's base
|
||||||
|
image
|
||||||
|
(`golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f`,
|
||||||
|
which reports `2.14.0 built with go1.27.0 from 114493f9`). A module's `go`
|
||||||
|
directive must not name a newer Go minor version than the one golangci-lint
|
||||||
|
was built with, or golangci-lint refuses to lint it: this release lints
|
||||||
|
`go 1.27.1` but not `go 1.28`. That digest is the only pin, since no repo
|
||||||
|
installs golangci-lint on the host. A repo sets the lint phase digest to the
|
||||||
|
one named here and re-vendors `.golangci.yml` in the same commit, whichever of
|
||||||
|
the two prompted the change: the canonical copy can name linters that an older
|
||||||
|
golangci-lint rejects, and a newer golangci-lint can add linters that
|
||||||
|
`default: all` switches on until the canonical copy disables them.
|
||||||
|
|
||||||
|
- **`script/bootstrap` installs a pinned tool by comparing versions, never by
|
||||||
|
testing presence.** An `if ! command -v <tool>; then install; fi` guard tests
|
||||||
|
`PATH` only, so on an already-provisioned machine the pin is inert and a
|
||||||
|
version bump is a silent no-op — while the Dockerfile, installing into a clean
|
||||||
|
image, gets the pinned version, so a local `make check` and `make docker` can
|
||||||
|
disagree about what the tool even is. The canonical form:
|
||||||
|
- compares the installed version against the pin over the **whole** version
|
||||||
|
token; a parser that stops at the first `-` reports `2.12.2` for a host
|
||||||
|
running `2.12.2-rc1` and skips the install;
|
||||||
|
- treats absent, non-zero, empty or unrecognised `--version` output as a
|
||||||
|
mismatch, so the failure direction is a redundant install and never a
|
||||||
|
skipped one;
|
||||||
|
- after installing, re-resolves the binary the way callers do — `hash -r`,
|
||||||
|
then through `PATH`, not through the directory the installer wrote to —
|
||||||
|
and fails naming the resolved path, since an install that a shadowing
|
||||||
|
binary hides succeeds while changing nothing any caller sees;
|
||||||
|
- is actually called, and prints the version on both success paths: a
|
||||||
|
function defined and never invoked has the same exit status and the same
|
||||||
|
empty output as one that worked.
|
||||||
|
|
||||||
|
Keep it POSIX sh: no arrays, no `[[`, no `grep -P`.
|
||||||
|
|
||||||
|
A Go tool a repo needs on the host is installed with `go install` pinned to
|
||||||
|
a commit hash (`go install <package>@<commit hash>`). It is never tracked as
|
||||||
|
a `go.mod` tool dependency or through a `tools.go` file, either of which
|
||||||
|
pulls the tool's own dependencies into the repo's `go.mod` and `go.sum`.
|
||||||
|
|
||||||
- When pinning images or packages by hash, add a comment above the reference
|
- When pinning images or packages by hash, add a comment above the reference
|
||||||
with the version and date (YYYY-MM-DD).
|
with the version and date (YYYY-MM-DD).
|
||||||
@@ -382,12 +639,14 @@ style conventions are in separate documents:
|
|||||||
settings.
|
settings.
|
||||||
|
|
||||||
- Avoid putting files in the repo root unless necessary. Root should contain
|
- Avoid putting files in the repo root unless necessary. Root should contain
|
||||||
only project-level config files (`README.md`, `Makefile`, `Dockerfile`,
|
only project-level config files (`README.md`, `AGENTS.md`, `Makefile`,
|
||||||
`LICENSE`, `.gitignore`, `.editorconfig`, `REPO_POLICIES.md`, and
|
`Dockerfile`, `LICENSE`, `.gitignore`, `.editorconfig`, `REPO_POLICIES.md`,
|
||||||
language-specific config). Everything else goes in a subdirectory. Canonical
|
and language-specific config). Everything else goes in a subdirectory.
|
||||||
subdirectory names:
|
Canonical subdirectory names:
|
||||||
- `bin/` — executable scripts and tools
|
- `bin/` — executable scripts and tools
|
||||||
- `cmd/` — Go command entrypoints
|
- `cmd/` — Go command entrypoints; thin only: one `main.go` per binary whose
|
||||||
|
body is a single call into `internal/` or `pkg/`, no project logic in
|
||||||
|
`cmd/`
|
||||||
- `configs/` — configuration templates and examples
|
- `configs/` — configuration templates and examples
|
||||||
- `deploy/` — deployment manifests (k8s, compose, terraform)
|
- `deploy/` — deployment manifests (k8s, compose, terraform)
|
||||||
- `docs/` — documentation and markdown (README.md stays in root)
|
- `docs/` — documentation and markdown (README.md stays in root)
|
||||||
@@ -414,3 +673,7 @@ style conventions are in separate documents:
|
|||||||
- Go: `go.mod`, `go.sum`, `.golangci.yml`
|
- Go: `go.mod`, `go.sum`, `.golangci.yml`
|
||||||
- JS: `package.json`, `yarn.lock`, `.prettierrc`, `.prettierignore`
|
- JS: `package.json`, `yarn.lock`, `.prettierrc`, `.prettierignore`
|
||||||
- Python: `pyproject.toml`
|
- Python: `pyproject.toml`
|
||||||
|
|
||||||
|
- Guidance for coding agents lives in one `AGENTS.md` at the repository root. It
|
||||||
|
is never committed under a file or directory named after one agent tool, such
|
||||||
|
as `CLAUDE.md` or `.claude/`, and never split into separate memory files.
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ package main
|
|||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
|
"log/slog"
|
||||||
"os"
|
"os"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
@@ -187,6 +188,9 @@ func newApp() *fx.App {
|
|||||||
fx.Provide(
|
fx.Provide(
|
||||||
globals.New,
|
globals.New,
|
||||||
logger.New,
|
logger.New,
|
||||||
|
// The plain logger the session, the middleware and the
|
||||||
|
// webhook database manager take.
|
||||||
|
func(l *logger.Logger) *slog.Logger { return l.Get() },
|
||||||
config.New,
|
config.New,
|
||||||
database.New,
|
database.New,
|
||||||
database.NewWebhookDBManager,
|
database.NewWebhookDBManager,
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ import (
|
|||||||
|
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config/configtest"
|
||||||
"sneak.berlin/go/webhooker/internal/datadir"
|
"sneak.berlin/go/webhooker/internal/datadir"
|
||||||
"sneak.berlin/go/webhooker/internal/resetpw"
|
"sneak.berlin/go/webhooker/internal/resetpw"
|
||||||
"sneak.berlin/go/webhooker/internal/server"
|
"sneak.berlin/go/webhooker/internal/server"
|
||||||
@@ -37,7 +37,7 @@ const dockerStopGrace = 10 * time.Second
|
|||||||
// fx.New applies options before it executes invokes, so the timeout
|
// fx.New applies options before it executes invokes, so the timeout
|
||||||
// is set whether or not the graph itself can be constructed here.
|
// is set whether or not the graph itself can be constructed here.
|
||||||
func TestNewApp_StopTimeout(t *testing.T) {
|
func TestNewApp_StopTimeout(t *testing.T) {
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
t.Setenv("DATA_DIR", t.TempDir())
|
t.Setenv("DATA_DIR", t.TempDir())
|
||||||
|
|
||||||
got := newApp().StopTimeout()
|
got := newApp().StopTimeout()
|
||||||
@@ -75,7 +75,7 @@ func freePort(t *testing.T) int {
|
|||||||
// anything is built, and the run of logger.New, which happens before
|
// anything is built, and the run of logger.New, which happens before
|
||||||
// the configuration sets the level.
|
// the configuration sets the level.
|
||||||
func TestNewApp_SendsFxEventsToTheLogger(t *testing.T) {
|
func TestNewApp_SendsFxEventsToTheLogger(t *testing.T) {
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
t.Setenv("DATA_DIR", t.TempDir())
|
t.Setenv("DATA_DIR", t.TempDir())
|
||||||
t.Setenv("PORT", strconv.Itoa(freePort(t)))
|
t.Setenv("PORT", strconv.Itoa(freePort(t)))
|
||||||
t.Setenv("DEBUG", "true")
|
t.Setenv("DEBUG", "true")
|
||||||
|
|||||||
@@ -22,7 +22,6 @@ require (
|
|||||||
github.com/stretchr/testify v1.11.1
|
github.com/stretchr/testify v1.11.1
|
||||||
go.uber.org/fx v1.24.0
|
go.uber.org/fx v1.24.0
|
||||||
golang.org/x/crypto v0.38.0
|
golang.org/x/crypto v0.38.0
|
||||||
gopkg.in/yaml.v3 v3.0.1
|
|
||||||
gorm.io/driver/sqlite v1.5.4
|
gorm.io/driver/sqlite v1.5.4
|
||||||
gorm.io/gorm v1.25.5
|
gorm.io/gorm v1.25.5
|
||||||
modernc.org/sqlite v1.28.0
|
modernc.org/sqlite v1.28.0
|
||||||
@@ -59,6 +58,7 @@ require (
|
|||||||
golang.org/x/text v0.25.0 // indirect
|
golang.org/x/text v0.25.0 // indirect
|
||||||
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
|
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
|
||||||
google.golang.org/protobuf v1.31.0 // indirect
|
google.golang.org/protobuf v1.31.0 // indirect
|
||||||
|
gopkg.in/yaml.v3 v3.0.1 // indirect
|
||||||
lukechampine.com/uint128 v1.2.0 // indirect
|
lukechampine.com/uint128 v1.2.0 // indirect
|
||||||
modernc.org/cc/v3 v3.40.0 // indirect
|
modernc.org/cc/v3 v3.40.0 // indirect
|
||||||
modernc.org/ccgo/v3 v3.16.13 // indirect
|
modernc.org/ccgo/v3 v3.16.13 // indirect
|
||||||
|
|||||||
@@ -1,387 +0,0 @@
|
|||||||
package ciscript_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"maps"
|
|
||||||
"os"
|
|
||||||
"os/exec"
|
|
||||||
"path/filepath"
|
|
||||||
"slices"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
"gopkg.in/yaml.v3"
|
|
||||||
)
|
|
||||||
|
|
||||||
const (
|
|
||||||
// supersededDesc is the description script/ci-mark-superseded
|
|
||||||
// writes, and the one an earlier revision of it wrote alongside a
|
|
||||||
// `skipped` state.
|
|
||||||
supersededDesc = "Superseded by a newer commit; never tested"
|
|
||||||
|
|
||||||
// liveContext is the commit-status context Gitea uses for this
|
|
||||||
// repository's runs, as seen in its API. The script derives it from
|
|
||||||
// the workflow and job names rather than hardcoding it; the
|
|
||||||
// derivation is checked against this value below.
|
|
||||||
liveContext = "check / check (push)"
|
|
||||||
|
|
||||||
scriptPath = "../../script/ci-mark-superseded"
|
|
||||||
workflow = "../../.gitea/workflows/check.yml"
|
|
||||||
|
|
||||||
// failure is the only state that neither folds into a combined
|
|
||||||
// `success` (as `skipped` does) nor blocks the commit forever (as
|
|
||||||
// `pending` does).
|
|
||||||
failure = "failure"
|
|
||||||
)
|
|
||||||
|
|
||||||
// repo is a throwaway git history: parent is the commit a run would be
|
|
||||||
// cancelled on, head the commit that superseded it.
|
|
||||||
type repo struct {
|
|
||||||
dir string
|
|
||||||
head string
|
|
||||||
parent string
|
|
||||||
}
|
|
||||||
|
|
||||||
// scriptEnv is the run identity the Gitea runner exports and the script
|
|
||||||
// builds its context string from.
|
|
||||||
type scriptEnv struct {
|
|
||||||
workflow string
|
|
||||||
job string
|
|
||||||
event string
|
|
||||||
}
|
|
||||||
|
|
||||||
func defaultEnv() scriptEnv {
|
|
||||||
return scriptEnv{workflow: "check", job: "check", event: "push"}
|
|
||||||
}
|
|
||||||
|
|
||||||
func cancelled() commitStatus {
|
|
||||||
return commitStatus{
|
|
||||||
Context: liveContext,
|
|
||||||
Status: failure,
|
|
||||||
Description: "Has been cancelled",
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func running() commitStatus {
|
|
||||||
return commitStatus{
|
|
||||||
Context: liveContext,
|
|
||||||
Status: "pending",
|
|
||||||
Description: "Has started running",
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestMarkSuperseded(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
cases := map[string]struct {
|
|
||||||
parent commitStatus
|
|
||||||
wantMark bool
|
|
||||||
}{
|
|
||||||
"a cancelled run is marked": {
|
|
||||||
parent: cancelled(),
|
|
||||||
wantMark: true,
|
|
||||||
},
|
|
||||||
"a laundered skipped status is marked": {
|
|
||||||
parent: commitStatus{
|
|
||||||
Context: liveContext,
|
|
||||||
Status: "skipped",
|
|
||||||
Description: supersededDesc,
|
|
||||||
},
|
|
||||||
wantMark: true,
|
|
||||||
},
|
|
||||||
"a genuine failure is left alone": {
|
|
||||||
parent: commitStatus{
|
|
||||||
Context: liveContext,
|
|
||||||
Status: failure,
|
|
||||||
Description: "Failing after 3m1s",
|
|
||||||
},
|
|
||||||
wantMark: false,
|
|
||||||
},
|
|
||||||
"a passing run is left alone": {
|
|
||||||
parent: commitStatus{
|
|
||||||
Context: liveContext,
|
|
||||||
Status: "success",
|
|
||||||
Description: "Successful in 2m52s",
|
|
||||||
},
|
|
||||||
wantMark: false,
|
|
||||||
},
|
|
||||||
"another context is left alone": {
|
|
||||||
parent: commitStatus{
|
|
||||||
Context: "other / other (push)",
|
|
||||||
Status: failure,
|
|
||||||
Description: "Has been cancelled",
|
|
||||||
},
|
|
||||||
wantMark: false,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
for name, tc := range cases {
|
|
||||||
t.Run(name, func(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
requireTools(t)
|
|
||||||
|
|
||||||
history := newRepo(t)
|
|
||||||
fake, api := newFakeGitea(t)
|
|
||||||
fake.setStatus(history.head, running())
|
|
||||||
fake.setStatus(history.parent, tc.parent)
|
|
||||||
|
|
||||||
out, err := runScript(t, history, api, defaultEnv())
|
|
||||||
require.NoError(t, err, out)
|
|
||||||
|
|
||||||
posted := fake.postedFor(history.parent)
|
|
||||||
if !tc.wantMark {
|
|
||||||
require.Empty(t, posted)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
require.Equal(t, []postedStatus{{
|
|
||||||
Context: liveContext,
|
|
||||||
// Not `skipped`: Gitea's combined status folds
|
|
||||||
// that into `success`, which is what made a
|
|
||||||
// never-tested commit read green.
|
|
||||||
State: failure,
|
|
||||||
Description: supersededDesc,
|
|
||||||
}}, posted)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// A second run must not rewrite what the first one wrote, or every
|
|
||||||
// later push would post a duplicate status.
|
|
||||||
func TestMarkSupersededIsIdempotent(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
requireTools(t)
|
|
||||||
|
|
||||||
history := newRepo(t)
|
|
||||||
fake, api := newFakeGitea(t)
|
|
||||||
fake.setStatus(history.head, running())
|
|
||||||
fake.setStatus(history.parent, cancelled())
|
|
||||||
|
|
||||||
for range 2 {
|
|
||||||
out, err := runScript(t, history, api, defaultEnv())
|
|
||||||
require.NoError(t, err, out)
|
|
||||||
}
|
|
||||||
|
|
||||||
require.Len(t, fake.postedFor(history.parent), 1)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Renaming the workflow or the job changes the context string Gitea
|
|
||||||
// uses. The script must say so instead of quietly matching nothing.
|
|
||||||
func TestMarkSupersededRejectsAnUnknownContext(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
requireTools(t)
|
|
||||||
|
|
||||||
history := newRepo(t)
|
|
||||||
fake, api := newFakeGitea(t)
|
|
||||||
fake.setStatus(history.head, running())
|
|
||||||
fake.setStatus(history.parent, cancelled())
|
|
||||||
|
|
||||||
env := defaultEnv()
|
|
||||||
env.job = "renamed"
|
|
||||||
|
|
||||||
out, err := runScript(t, history, api, env)
|
|
||||||
require.Error(t, err)
|
|
||||||
require.Contains(t, out, "renamed")
|
|
||||||
require.Contains(t, out, liveContext)
|
|
||||||
require.Empty(t, fake.postedFor(history.parent))
|
|
||||||
}
|
|
||||||
|
|
||||||
// ANCESTOR_LIMIT is a documented knob. A value that is set but unusable
|
|
||||||
// must abort: handing it to git and discarding the exit status left the
|
|
||||||
// walk empty and the step green, marking nothing.
|
|
||||||
func TestMarkSupersededRejectsAnUnparseableAncestorLimit(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
requireTools(t)
|
|
||||||
|
|
||||||
history := newRepo(t)
|
|
||||||
fake, api := newFakeGitea(t)
|
|
||||||
fake.setStatus(history.head, running())
|
|
||||||
fake.setStatus(history.parent, cancelled())
|
|
||||||
|
|
||||||
out, err := runScript(
|
|
||||||
t, history, api, defaultEnv(), "ANCESTOR_LIMIT=twenty",
|
|
||||||
)
|
|
||||||
require.Error(t, err)
|
|
||||||
require.Contains(t, out, "ANCESTOR_LIMIT")
|
|
||||||
require.Contains(t, out, "twenty")
|
|
||||||
require.Empty(t, fake.postedFor(history.parent))
|
|
||||||
}
|
|
||||||
|
|
||||||
// A status read that fails is not the same as a commit with nothing to
|
|
||||||
// do. Losing curl's exit status through a pipe made the two identical
|
|
||||||
// and left a laundered commit laundered with no signal.
|
|
||||||
func TestMarkSupersededFailsOnAnUnreadableAncestorStatus(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
requireTools(t)
|
|
||||||
|
|
||||||
history := newRepo(t)
|
|
||||||
fake, api := newFakeGitea(t)
|
|
||||||
fake.setStatus(history.head, running())
|
|
||||||
fake.setStatus(history.parent, cancelled())
|
|
||||||
fake.failStatusRead(history.parent)
|
|
||||||
|
|
||||||
out, err := runScript(t, history, api, defaultEnv())
|
|
||||||
require.Error(t, err)
|
|
||||||
require.Contains(t, out, history.parent)
|
|
||||||
require.Contains(t, out, "cannot read commit statuses")
|
|
||||||
require.Empty(t, fake.postedFor(history.parent))
|
|
||||||
}
|
|
||||||
|
|
||||||
// A shallow clone cannot resolve the parent, so it is indistinguishable
|
|
||||||
// from a root commit to rev-parse and the walk would exit 0 having
|
|
||||||
// marked nothing. It must abort instead: dropping `fetch-depth: 0` from
|
|
||||||
// the checkout step is one edit, and a silent no-op there restores the
|
|
||||||
// false-green bug this script exists to prevent.
|
|
||||||
func TestMarkSupersededRejectsAShallowRepository(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
requireTools(t)
|
|
||||||
|
|
||||||
history := shallowClone(t, newRepo(t))
|
|
||||||
fake, api := newFakeGitea(t)
|
|
||||||
fake.setStatus(history.head, running())
|
|
||||||
fake.setStatus(history.parent, cancelled())
|
|
||||||
|
|
||||||
out, err := runScript(t, history, api, defaultEnv())
|
|
||||||
require.Error(t, err)
|
|
||||||
require.Contains(t, out, "shallow repository")
|
|
||||||
require.Empty(t, fake.postedFor(history.parent))
|
|
||||||
require.Empty(t, fake.postedFor(history.head))
|
|
||||||
}
|
|
||||||
|
|
||||||
// shallowClone returns the same history as a depth-1 clone. The `file://`
|
|
||||||
// URL is required: git ignores --depth for a plain local path.
|
|
||||||
func shallowClone(t *testing.T, history repo) repo {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
dir := t.TempDir()
|
|
||||||
|
|
||||||
//nolint:gosec // fixed argv, arguments are test-local paths
|
|
||||||
cmd := exec.CommandContext(t.Context(), "git", "clone", "-q",
|
|
||||||
"--depth=1", "file://"+history.dir, dir)
|
|
||||||
|
|
||||||
out, err := cmd.CombinedOutput()
|
|
||||||
require.NoError(t, err, string(out))
|
|
||||||
|
|
||||||
return repo{dir: dir, head: history.head, parent: history.parent}
|
|
||||||
}
|
|
||||||
|
|
||||||
// The derived context must equal the one Gitea actually uses, which is
|
|
||||||
// built from the same workflow and job names.
|
|
||||||
func TestDerivedContextMatchesGitea(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
requireTools(t)
|
|
||||||
|
|
||||||
name, job := workflowIdentity(t)
|
|
||||||
|
|
||||||
history := newRepo(t)
|
|
||||||
fake, api := newFakeGitea(t)
|
|
||||||
fake.setStatus(history.head, running())
|
|
||||||
fake.setStatus(history.parent, cancelled())
|
|
||||||
|
|
||||||
out, err := runScript(t, history, api, scriptEnv{
|
|
||||||
workflow: name,
|
|
||||||
job: job,
|
|
||||||
event: "push",
|
|
||||||
})
|
|
||||||
require.NoError(t, err, out)
|
|
||||||
|
|
||||||
posted := fake.postedFor(history.parent)
|
|
||||||
require.Len(t, posted, 1)
|
|
||||||
require.Equal(t, liveContext, posted[0].Context)
|
|
||||||
}
|
|
||||||
|
|
||||||
// workflowIdentity reads the workflow name and its single job id out of
|
|
||||||
// the checked-in workflow file.
|
|
||||||
func workflowIdentity(t *testing.T) (string, string) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
raw, err := os.ReadFile(workflow)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
var parsed struct {
|
|
||||||
Name string `yaml:"name"`
|
|
||||||
Jobs map[string]any `yaml:"jobs"`
|
|
||||||
}
|
|
||||||
|
|
||||||
require.NoError(t, yaml.Unmarshal(raw, &parsed))
|
|
||||||
|
|
||||||
jobs := slices.Collect(maps.Keys(parsed.Jobs))
|
|
||||||
require.Len(t, jobs, 1)
|
|
||||||
|
|
||||||
return parsed.Name, jobs[0]
|
|
||||||
}
|
|
||||||
|
|
||||||
func runScript(
|
|
||||||
t *testing.T, history repo, api string, env scriptEnv,
|
|
||||||
extra ...string,
|
|
||||||
) (string, error) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
script, err := filepath.Abs(scriptPath)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
//nolint:gosec // fixed argv, repo-local script under test
|
|
||||||
cmd := exec.CommandContext(t.Context(), "sh", script)
|
|
||||||
cmd.Dir = history.dir
|
|
||||||
cmd.Env = append(os.Environ(),
|
|
||||||
"GITHUB_API_URL="+api,
|
|
||||||
"GITHUB_REPOSITORY=sneak/webhooker",
|
|
||||||
"GITHUB_SHA="+history.head,
|
|
||||||
"GITHUB_WORKFLOW="+env.workflow,
|
|
||||||
"GITHUB_JOB="+env.job,
|
|
||||||
"GITHUB_EVENT_NAME="+env.event,
|
|
||||||
"GITEA_TOKEN=test-token",
|
|
||||||
)
|
|
||||||
cmd.Env = append(cmd.Env, extra...)
|
|
||||||
|
|
||||||
out, err := cmd.CombinedOutput()
|
|
||||||
|
|
||||||
return string(out), err
|
|
||||||
}
|
|
||||||
|
|
||||||
func newRepo(t *testing.T) repo {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
dir := t.TempDir()
|
|
||||||
|
|
||||||
git := func(args ...string) string {
|
|
||||||
//nolint:gosec // fixed argv, arguments are test constants
|
|
||||||
cmd := exec.CommandContext(t.Context(), "git", args...)
|
|
||||||
cmd.Dir = dir
|
|
||||||
|
|
||||||
out, err := cmd.CombinedOutput()
|
|
||||||
require.NoError(t, err, string(out))
|
|
||||||
|
|
||||||
return strings.TrimSpace(string(out))
|
|
||||||
}
|
|
||||||
|
|
||||||
commit := func(message string) string {
|
|
||||||
git(
|
|
||||||
"-c", "user.email=ci@example.invalid",
|
|
||||||
"-c", "user.name=ci",
|
|
||||||
"-c", "commit.gpgsign=false",
|
|
||||||
"commit", "-q", "--allow-empty", "-m", message,
|
|
||||||
)
|
|
||||||
|
|
||||||
return git("rev-parse", "HEAD")
|
|
||||||
}
|
|
||||||
|
|
||||||
git("init", "-q", "-b", "main")
|
|
||||||
|
|
||||||
parent := commit("parent")
|
|
||||||
head := commit("head")
|
|
||||||
|
|
||||||
return repo{dir: dir, head: head, parent: parent}
|
|
||||||
}
|
|
||||||
|
|
||||||
func requireTools(t *testing.T) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
for _, tool := range []string{"sh", "git", "curl", "jq"} {
|
|
||||||
_, err := exec.LookPath(tool)
|
|
||||||
if err != nil {
|
|
||||||
t.Skipf("%s is not installed: %v", tool, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,10 +0,0 @@
|
|||||||
// Package ciscript holds the tests for the repository's CI shell
|
|
||||||
// scripts in script/. It carries no runtime code: the scripts run on
|
|
||||||
// the CI runner, not inside the binary, but their behaviour still has
|
|
||||||
// to be verified by the test suite.
|
|
||||||
//
|
|
||||||
// The scripts under test are outside the Go build graph, so `go test`'s
|
|
||||||
// result cache serves a stale PASS when only a script changed: run the
|
|
||||||
// container build, or GOFLAGS=-count=1, to trust a result here after
|
|
||||||
// editing script/.
|
|
||||||
package ciscript
|
|
||||||
@@ -1,162 +0,0 @@
|
|||||||
package ciscript_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"net/http"
|
|
||||||
"net/http/httptest"
|
|
||||||
"sync"
|
|
||||||
"testing"
|
|
||||||
)
|
|
||||||
|
|
||||||
// commitStatus is the part of an entry in Gitea's combined-status
|
|
||||||
// response that script/ci-mark-superseded reads.
|
|
||||||
type commitStatus struct {
|
|
||||||
Context string `json:"context"`
|
|
||||||
Status string `json:"status"`
|
|
||||||
Description string `json:"description"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// postedStatus is the part of a create-status request body the script
|
|
||||||
// writes.
|
|
||||||
type postedStatus struct {
|
|
||||||
Context string `json:"context"`
|
|
||||||
State string `json:"state"`
|
|
||||||
Description string `json:"description"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// fakeGitea serves the two endpoints the script talks to. Like Gitea,
|
|
||||||
// the newest status for a context replaces the previous one, so a
|
|
||||||
// second run of the script sees what the first one wrote.
|
|
||||||
type fakeGitea struct {
|
|
||||||
mu sync.Mutex
|
|
||||||
statuses map[string][]commitStatus
|
|
||||||
posted map[string][]postedStatus
|
|
||||||
// failRead is a commit whose combined-status read answers HTTP
|
|
||||||
// 500, standing in for a status API that is down.
|
|
||||||
failRead string
|
|
||||||
}
|
|
||||||
|
|
||||||
// newFakeGitea returns the fake and the base URL to hand the script as
|
|
||||||
// GITHUB_API_URL.
|
|
||||||
func newFakeGitea(t *testing.T) (*fakeGitea, string) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
fake := &fakeGitea{
|
|
||||||
mu: sync.Mutex{},
|
|
||||||
statuses: map[string][]commitStatus{},
|
|
||||||
posted: map[string][]postedStatus{},
|
|
||||||
failRead: "",
|
|
||||||
}
|
|
||||||
|
|
||||||
srv := httptest.NewServer(fake.routes())
|
|
||||||
t.Cleanup(srv.Close)
|
|
||||||
|
|
||||||
return fake, srv.URL
|
|
||||||
}
|
|
||||||
|
|
||||||
func (f *fakeGitea) routes() http.Handler {
|
|
||||||
mux := http.NewServeMux()
|
|
||||||
|
|
||||||
mux.HandleFunc(
|
|
||||||
"GET /repos/{owner}/{repo}/commits/{sha}/status",
|
|
||||||
f.handleCombined,
|
|
||||||
)
|
|
||||||
mux.HandleFunc(
|
|
||||||
"POST /repos/{owner}/{repo}/statuses/{sha}",
|
|
||||||
f.handleCreate,
|
|
||||||
)
|
|
||||||
|
|
||||||
return mux
|
|
||||||
}
|
|
||||||
|
|
||||||
func (f *fakeGitea) handleCombined(
|
|
||||||
w http.ResponseWriter, r *http.Request,
|
|
||||||
) {
|
|
||||||
f.mu.Lock()
|
|
||||||
defer f.mu.Unlock()
|
|
||||||
|
|
||||||
sha := r.PathValue("sha")
|
|
||||||
if f.failRead != "" && f.failRead == sha {
|
|
||||||
http.Error(w, "boom", http.StatusInternalServerError)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
body := struct {
|
|
||||||
Statuses []commitStatus `json:"statuses"`
|
|
||||||
}{Statuses: f.statuses[sha]}
|
|
||||||
|
|
||||||
payload, err := json.Marshal(body)
|
|
||||||
if err != nil {
|
|
||||||
http.Error(w, err.Error(), http.StatusInternalServerError)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
w.Header().Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
_, _ = w.Write(payload)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (f *fakeGitea) handleCreate(w http.ResponseWriter, r *http.Request) {
|
|
||||||
var got postedStatus
|
|
||||||
|
|
||||||
err := json.NewDecoder(r.Body).Decode(&got)
|
|
||||||
if err != nil {
|
|
||||||
http.Error(w, err.Error(), http.StatusBadRequest)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
sha := r.PathValue("sha")
|
|
||||||
|
|
||||||
f.mu.Lock()
|
|
||||||
defer f.mu.Unlock()
|
|
||||||
|
|
||||||
f.posted[sha] = append(f.posted[sha], got)
|
|
||||||
f.replaceLocked(sha, commitStatus{
|
|
||||||
Context: got.Context,
|
|
||||||
Status: got.State,
|
|
||||||
Description: got.Description,
|
|
||||||
})
|
|
||||||
|
|
||||||
w.WriteHeader(http.StatusCreated)
|
|
||||||
}
|
|
||||||
|
|
||||||
// failStatusRead makes the combined-status read for one commit answer
|
|
||||||
// HTTP 500.
|
|
||||||
func (f *fakeGitea) failStatusRead(sha string) {
|
|
||||||
f.mu.Lock()
|
|
||||||
defer f.mu.Unlock()
|
|
||||||
|
|
||||||
f.failRead = sha
|
|
||||||
}
|
|
||||||
|
|
||||||
// setStatus gives a commit its latest status for a context.
|
|
||||||
func (f *fakeGitea) setStatus(sha string, status commitStatus) {
|
|
||||||
f.mu.Lock()
|
|
||||||
defer f.mu.Unlock()
|
|
||||||
|
|
||||||
f.replaceLocked(sha, status)
|
|
||||||
}
|
|
||||||
|
|
||||||
// postedFor returns the statuses the script created for a commit.
|
|
||||||
func (f *fakeGitea) postedFor(sha string) []postedStatus {
|
|
||||||
f.mu.Lock()
|
|
||||||
defer f.mu.Unlock()
|
|
||||||
|
|
||||||
return append([]postedStatus(nil), f.posted[sha]...)
|
|
||||||
}
|
|
||||||
|
|
||||||
// replaceLocked requires f.mu.
|
|
||||||
func (f *fakeGitea) replaceLocked(sha string, status commitStatus) {
|
|
||||||
for i, existing := range f.statuses[sha] {
|
|
||||||
if existing.Context == status.Context {
|
|
||||||
f.statuses[sha][i] = status
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
f.statuses[sha] = append(f.statuses[sha], status)
|
|
||||||
}
|
|
||||||
@@ -7,21 +7,22 @@ import (
|
|||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
|
"sneak.berlin/go/webhooker/internal/config/configtest"
|
||||||
)
|
)
|
||||||
|
|
||||||
// TestClearEnvForTest_RemovesAddedVariables pins that a variable set
|
// TestClearEnv_RemovesAddedVariables pins that a variable set
|
||||||
// after the clear other than through t.Setenv, as a test's .env file
|
// after the clear other than through t.Setenv, as a test's .env file
|
||||||
// sets one, is gone once the test ends, so it cannot reach the tests
|
// sets one, is gone once the test ends, so it cannot reach the tests
|
||||||
// that run after it.
|
// that run after it.
|
||||||
//
|
//
|
||||||
//nolint:paralleltest // ClearEnvForTest uses t.Setenv.
|
//nolint:paralleltest // ClearEnv uses t.Setenv.
|
||||||
func TestClearEnvForTest_RemovesAddedVariables(t *testing.T) {
|
func TestClearEnv_RemovesAddedVariables(t *testing.T) {
|
||||||
// The outer clear keeps a value of the key exported in the shell
|
// The outer clear keeps a value of the key exported in the shell
|
||||||
// from making it a variable the inner clear has to put back.
|
// from making it a variable the inner clear has to put back.
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
|
|
||||||
t.Run("loads a .env file after the clear", func(t *testing.T) {
|
t.Run("loads a .env file after the clear", func(t *testing.T) {
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
|
|
||||||
path := writeDotEnv(t, dotEnvKey+"=from-dot-env\n")
|
path := writeDotEnv(t, dotEnvKey+"=from-dot-env\n")
|
||||||
require.NoError(t, config.LoadDotEnvFileForTest(path))
|
require.NoError(t, config.LoadDotEnvFileForTest(path))
|
||||||
@@ -11,6 +11,7 @@ import (
|
|||||||
"go.uber.org/fx"
|
"go.uber.org/fx"
|
||||||
"go.uber.org/fx/fxtest"
|
"go.uber.org/fx/fxtest"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
|
"sneak.berlin/go/webhooker/internal/config/configtest"
|
||||||
"sneak.berlin/go/webhooker/internal/globals"
|
"sneak.berlin/go/webhooker/internal/globals"
|
||||||
"sneak.berlin/go/webhooker/internal/logger"
|
"sneak.berlin/go/webhooker/internal/logger"
|
||||||
)
|
)
|
||||||
@@ -70,7 +71,7 @@ func TestEnvironmentConfig(t *testing.T) {
|
|||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
// Cannot use t.Parallel() here because t.Setenv
|
// Cannot use t.Parallel() here because t.Setenv
|
||||||
// is incompatible with parallel subtests.
|
// is incompatible with parallel subtests.
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
|
|
||||||
if tt.envValue != "" {
|
if tt.envValue != "" {
|
||||||
t.Setenv(
|
t.Setenv(
|
||||||
@@ -196,7 +197,7 @@ func TestRetentionSweepInterval(t *testing.T) {
|
|||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
// Cannot use t.Parallel() here because t.Setenv
|
// Cannot use t.Parallel() here because t.Setenv
|
||||||
// is incompatible with parallel subtests.
|
// is incompatible with parallel subtests.
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
|
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
|
||||||
|
|
||||||
if tt.set {
|
if tt.set {
|
||||||
@@ -335,7 +336,7 @@ func TestSessionIdleTimeout(t *testing.T) {
|
|||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
// Cannot use t.Parallel() here because t.Setenv
|
// Cannot use t.Parallel() here because t.Setenv
|
||||||
// is incompatible with parallel subtests.
|
// is incompatible with parallel subtests.
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
|
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
|
||||||
|
|
||||||
if tt.set {
|
if tt.set {
|
||||||
@@ -388,7 +389,7 @@ func TestDefaultDataDir(t *testing.T) {
|
|||||||
t.Run("env="+name, func(t *testing.T) {
|
t.Run("env="+name, func(t *testing.T) {
|
||||||
// Cannot use t.Parallel() here because t.Setenv
|
// Cannot use t.Parallel() here because t.Setenv
|
||||||
// is incompatible with parallel subtests.
|
// is incompatible with parallel subtests.
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
|
|
||||||
if env != "" {
|
if env != "" {
|
||||||
t.Setenv("WEBHOOKER_ENVIRONMENT", env)
|
t.Setenv("WEBHOOKER_ENVIRONMENT", env)
|
||||||
@@ -433,7 +434,7 @@ func TestDataDirHelper(t *testing.T) {
|
|||||||
t.Run(name, func(t *testing.T) {
|
t.Run(name, func(t *testing.T) {
|
||||||
// Cannot use t.Parallel() here because t.Setenv
|
// Cannot use t.Parallel() here because t.Setenv
|
||||||
// is incompatible with parallel subtests.
|
// is incompatible with parallel subtests.
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
|
|
||||||
if set != "" {
|
if set != "" {
|
||||||
t.Setenv("DATA_DIR", set)
|
t.Setenv("DATA_DIR", set)
|
||||||
@@ -498,7 +499,7 @@ func TestReceiverRateLimit(t *testing.T) {
|
|||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
// Cannot use t.Parallel() here because t.Setenv
|
// Cannot use t.Parallel() here because t.Setenv
|
||||||
// is incompatible with parallel subtests.
|
// is incompatible with parallel subtests.
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
|
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
|
||||||
|
|
||||||
if tt.set {
|
if tt.set {
|
||||||
@@ -614,7 +615,7 @@ func TestTrustedProxies(t *testing.T) {
|
|||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
// Cannot use t.Parallel() here because t.Setenv
|
// Cannot use t.Parallel() here because t.Setenv
|
||||||
// is incompatible with parallel subtests.
|
// is incompatible with parallel subtests.
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
|
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
|
||||||
|
|
||||||
if tt.set {
|
if tt.set {
|
||||||
@@ -725,7 +726,7 @@ func TestAllowedEgressCIDRs(t *testing.T) {
|
|||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
// Cannot use t.Parallel() here because t.Setenv
|
// Cannot use t.Parallel() here because t.Setenv
|
||||||
// is incompatible with parallel subtests.
|
// is incompatible with parallel subtests.
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
|
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
|
||||||
|
|
||||||
if tt.set {
|
if tt.set {
|
||||||
@@ -797,7 +798,7 @@ func TestEgressAllowlistWarning(t *testing.T) {
|
|||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
// Cannot use t.Parallel() here because t.Setenv
|
// Cannot use t.Parallel() here because t.Setenv
|
||||||
// is incompatible with parallel subtests.
|
// is incompatible with parallel subtests.
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
t.Setenv("WEBHOOKER_ENVIRONMENT", config.EnvironmentDev)
|
t.Setenv("WEBHOOKER_ENVIRONMENT", config.EnvironmentDev)
|
||||||
|
|
||||||
if tt.allowed != "" {
|
if tt.allowed != "" {
|
||||||
@@ -933,7 +934,7 @@ func TestMetricsAuthConfig(t *testing.T) {
|
|||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
// Cannot use t.Parallel() here because t.Setenv
|
// Cannot use t.Parallel() here because t.Setenv
|
||||||
// is incompatible with parallel subtests.
|
// is incompatible with parallel subtests.
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
|
|
||||||
if tt.username.set {
|
if tt.username.set {
|
||||||
t.Setenv("METRICS_USERNAME", tt.username.value)
|
t.Setenv("METRICS_USERNAME", tt.username.value)
|
||||||
|
|||||||
@@ -1,4 +1,6 @@
|
|||||||
package config
|
// Package configtest holds test support for code that reads the
|
||||||
|
// process environment.
|
||||||
|
package configtest
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"os"
|
"os"
|
||||||
@@ -6,12 +8,12 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
)
|
)
|
||||||
|
|
||||||
// ClearEnvForTest unsets every variable in the process environment
|
// ClearEnv unsets every variable in the process environment
|
||||||
// for the rest of the test, so a test sees only the variables it sets
|
// for the rest of the test, so a test sees only the variables it sets
|
||||||
// itself, not whatever the developer's shell exports. When the test
|
// itself, not whatever the developer's shell exports. When the test
|
||||||
// ends it leaves the environment exactly as it found it: each variable
|
// ends it leaves the environment exactly as it found it: each variable
|
||||||
// it unset is put back, and any variable added since is removed.
|
// it unset is put back, and any variable added since is removed.
|
||||||
func ClearEnvForTest(t *testing.T) {
|
func ClearEnv(t *testing.T) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
present := make(map[string]bool)
|
present := make(map[string]bool)
|
||||||
@@ -8,6 +8,7 @@ import (
|
|||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
|
"sneak.berlin/go/webhooker/internal/config/configtest"
|
||||||
)
|
)
|
||||||
|
|
||||||
// dotEnvKey is a throwaway variable name the .env tests write and
|
// dotEnvKey is a throwaway variable name the .env tests write and
|
||||||
@@ -39,9 +40,9 @@ func writeDotEnv(t *testing.T, contents string) string {
|
|||||||
// normally rather than be refused for a file it was never meant to
|
// normally rather than be refused for a file it was never meant to
|
||||||
// have.
|
// have.
|
||||||
//
|
//
|
||||||
//nolint:paralleltest // ClearEnvForTest uses t.Setenv.
|
//nolint:paralleltest // ClearEnv uses t.Setenv.
|
||||||
func TestLoadDotEnv_MissingFileIsFine(t *testing.T) {
|
func TestLoadDotEnv_MissingFileIsFine(t *testing.T) {
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
|
|
||||||
absent := filepath.Join(t.TempDir(), config.DotEnvPath)
|
absent := filepath.Join(t.TempDir(), config.DotEnvPath)
|
||||||
require.NoError(t, config.LoadDotEnvFileForTest(absent))
|
require.NoError(t, config.LoadDotEnvFileForTest(absent))
|
||||||
@@ -54,9 +55,9 @@ func TestLoadDotEnv_MissingFileIsFine(t *testing.T) {
|
|||||||
// reaches the environment, which is the whole reason the file is read
|
// reaches the environment, which is the whole reason the file is read
|
||||||
// at all.
|
// at all.
|
||||||
//
|
//
|
||||||
//nolint:paralleltest // ClearEnvForTest uses t.Setenv.
|
//nolint:paralleltest // ClearEnv uses t.Setenv.
|
||||||
func TestLoadDotEnv_AppliesValues(t *testing.T) {
|
func TestLoadDotEnv_AppliesValues(t *testing.T) {
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
|
|
||||||
path := writeDotEnv(t, "# a comment\n"+dotEnvKey+"=from-dot-env\n")
|
path := writeDotEnv(t, "# a comment\n"+dotEnvKey+"=from-dot-env\n")
|
||||||
|
|
||||||
@@ -82,9 +83,9 @@ func TestLoadDotEnv_RealEnvironmentWins(t *testing.T) {
|
|||||||
// reverts to its default; the process used to start that way with no
|
// reverts to its default; the process used to start that way with no
|
||||||
// log line naming the file at all.
|
// log line naming the file at all.
|
||||||
//
|
//
|
||||||
//nolint:paralleltest // ClearEnvForTest uses t.Setenv.
|
//nolint:paralleltest // ClearEnv uses t.Setenv.
|
||||||
func TestLoadDotEnv_MalformedFileAborts(t *testing.T) {
|
func TestLoadDotEnv_MalformedFileAborts(t *testing.T) {
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
|
|
||||||
path := writeDotEnv(
|
path := writeDotEnv(
|
||||||
t, malformedDotEnv+dotEnvKey+"=from-dot-env\n",
|
t, malformedDotEnv+dotEnvKey+"=from-dot-env\n",
|
||||||
@@ -132,7 +133,7 @@ func TestLoadDotEnv_UnreadableFileAborts(t *testing.T) {
|
|||||||
//
|
//
|
||||||
//nolint:paralleltest // t.Chdir moves the whole process.
|
//nolint:paralleltest // t.Chdir moves the whole process.
|
||||||
func TestLoadDotEnv_ReadsTheWorkingDirectory(t *testing.T) {
|
func TestLoadDotEnv_ReadsTheWorkingDirectory(t *testing.T) {
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
|
|
||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
require.NoError(t, os.WriteFile(
|
require.NoError(t, os.WriteFile(
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"go.uber.org/fx"
|
"go.uber.org/fx"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
|
"sneak.berlin/go/webhooker/internal/config/configtest"
|
||||||
"sneak.berlin/go/webhooker/internal/globals"
|
"sneak.berlin/go/webhooker/internal/globals"
|
||||||
"sneak.berlin/go/webhooker/internal/logger"
|
"sneak.berlin/go/webhooker/internal/logger"
|
||||||
)
|
)
|
||||||
@@ -120,7 +121,7 @@ func TestEnvBool(t *testing.T) {
|
|||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
// Cannot use t.Parallel() here because t.Setenv
|
// Cannot use t.Parallel() here because t.Setenv
|
||||||
// is incompatible with parallel subtests.
|
// is incompatible with parallel subtests.
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
|
|
||||||
if tt.set {
|
if tt.set {
|
||||||
t.Setenv(testEnvKey, tt.value)
|
t.Setenv(testEnvKey, tt.value)
|
||||||
@@ -169,7 +170,7 @@ func runEnvIntCases(
|
|||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
// Cannot use t.Parallel() here because t.Setenv
|
// Cannot use t.Parallel() here because t.Setenv
|
||||||
// is incompatible with parallel subtests.
|
// is incompatible with parallel subtests.
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
|
|
||||||
if tt.set {
|
if tt.set {
|
||||||
t.Setenv(testEnvKey, tt.value)
|
t.Setenv(testEnvKey, tt.value)
|
||||||
@@ -310,7 +311,7 @@ func TestEnvBindAddress(t *testing.T) {
|
|||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
// Cannot use t.Parallel() here because t.Setenv
|
// Cannot use t.Parallel() here because t.Setenv
|
||||||
// is incompatible with parallel subtests.
|
// is incompatible with parallel subtests.
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
|
|
||||||
if tt.set {
|
if tt.set {
|
||||||
t.Setenv(testEnvKey, tt.value)
|
t.Setenv(testEnvKey, tt.value)
|
||||||
@@ -476,7 +477,7 @@ func TestNewRejectsBadEnvValues(t *testing.T) {
|
|||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
// Cannot use t.Parallel() here because t.Setenv
|
// Cannot use t.Parallel() here because t.Setenv
|
||||||
// is incompatible with parallel subtests.
|
// is incompatible with parallel subtests.
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
|
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
|
||||||
t.Setenv(tt.key, tt.value)
|
t.Setenv(tt.key, tt.value)
|
||||||
|
|
||||||
@@ -638,7 +639,7 @@ func sentryEnvValueCases() []badEnvValueCase {
|
|||||||
// break the legitimate unset case: absent variables still get their
|
// break the legitimate unset case: absent variables still get their
|
||||||
// documented defaults.
|
// documented defaults.
|
||||||
func TestNewUsesDefaultsWhenUnset(t *testing.T) {
|
func TestNewUsesDefaultsWhenUnset(t *testing.T) {
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
|
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
|
||||||
|
|
||||||
cfg, err := buildConfig(t)
|
cfg, err := buildConfig(t)
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import (
|
|||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
|
"sneak.berlin/go/webhooker/internal/config/configtest"
|
||||||
)
|
)
|
||||||
|
|
||||||
// envKeySentryDSN is the variable envSentryDSN reads in production.
|
// envKeySentryDSN is the variable envSentryDSN reads in production.
|
||||||
@@ -100,7 +101,7 @@ func TestEnvSentryDSN(t *testing.T) {
|
|||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
// Cannot use t.Parallel() here because t.Setenv
|
// Cannot use t.Parallel() here because t.Setenv
|
||||||
// is incompatible with parallel subtests.
|
// is incompatible with parallel subtests.
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
|
|
||||||
if tt.set {
|
if tt.set {
|
||||||
t.Setenv(envKeySentryDSN, tt.value)
|
t.Setenv(envKeySentryDSN, tt.value)
|
||||||
|
|||||||
@@ -0,0 +1,55 @@
|
|||||||
|
// Package databasetest builds a WebhookDBManager for tests in other
|
||||||
|
// packages.
|
||||||
|
package databasetest
|
||||||
|
|
||||||
|
import (
|
||||||
|
"log/slog"
|
||||||
|
"os"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"go.uber.org/fx/fxtest"
|
||||||
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
)
|
||||||
|
|
||||||
|
// NewWebhookDBManager creates a WebhookDBManager backed by the given
|
||||||
|
// data directory, logging at DEBUG to standard error.
|
||||||
|
func NewWebhookDBManager(
|
||||||
|
t *testing.T, dataDir string,
|
||||||
|
) *database.WebhookDBManager {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
return NewWebhookDBManagerWithLogger(
|
||||||
|
t,
|
||||||
|
dataDir,
|
||||||
|
slog.New(slog.NewTextHandler(
|
||||||
|
os.Stderr,
|
||||||
|
&slog.HandlerOptions{Level: slog.LevelDebug},
|
||||||
|
)),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewWebhookDBManagerWithLogger is NewWebhookDBManager with the
|
||||||
|
// logger supplied by the caller. The per-webhook databases this manager
|
||||||
|
// opens hand that logger to gormlog, so a test that needs to see the SQL
|
||||||
|
// the service emits can capture it.
|
||||||
|
//
|
||||||
|
// It is built through database.NewWebhookDBManager on a lifecycle that
|
||||||
|
// is never started, so nothing closes its databases but the caller.
|
||||||
|
func NewWebhookDBManagerWithLogger(
|
||||||
|
t *testing.T, dataDir string, log *slog.Logger,
|
||||||
|
) *database.WebhookDBManager {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
mgr, err := database.NewWebhookDBManager(
|
||||||
|
fxtest.NewLifecycle(t),
|
||||||
|
database.WebhookDBManagerParams{
|
||||||
|
Config: &config.Config{DataDir: dataDir},
|
||||||
|
Logger: log,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
return mgr
|
||||||
|
}
|
||||||
@@ -13,6 +13,7 @@ import (
|
|||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
_ "modernc.org/sqlite"
|
_ "modernc.org/sqlite"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database/databasetest"
|
||||||
)
|
)
|
||||||
|
|
||||||
// testDataDirPerm is the mode the test data directory is created
|
// testDataDirPerm is the mode the test data directory is created
|
||||||
@@ -133,7 +134,7 @@ func TestOpenPurgesLeakedTargetRows(t *testing.T) {
|
|||||||
// Create the file the way the application does, so the targets
|
// Create the file the way the application does, so the targets
|
||||||
// table has exactly the shape AutoMigrate gives it, then write
|
// table has exactly the shape AutoMigrate gives it, then write
|
||||||
// a leaked row into it the way the association upsert did.
|
// a leaked row into it the way the association upsert did.
|
||||||
initial := database.NewTestWebhookDBManager(dataDir)
|
initial := databasetest.NewWebhookDBManager(t, dataDir)
|
||||||
|
|
||||||
_, err := initial.GetDB(webhookID)
|
_, err := initial.GetDB(webhookID)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
@@ -156,7 +157,7 @@ func TestOpenPurgesLeakedTargetRows(t *testing.T) {
|
|||||||
clearEventDBSweptMarker(t, seed)
|
clearEventDBSweptMarker(t, seed)
|
||||||
require.NoError(t, seed.Close())
|
require.NoError(t, seed.Close())
|
||||||
|
|
||||||
mgr := database.NewTestWebhookDBManager(dataDir)
|
mgr := databasetest.NewWebhookDBManager(t, dataDir)
|
||||||
|
|
||||||
_, err = mgr.GetDB(webhookID)
|
_, err = mgr.GetDB(webhookID)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
@@ -172,7 +173,7 @@ func TestOpenPurgesLeakedTargetRows(t *testing.T) {
|
|||||||
|
|
||||||
// Idempotent: a second open leaves it at zero and does not
|
// Idempotent: a second open leaves it at zero and does not
|
||||||
// error.
|
// error.
|
||||||
again := database.NewTestWebhookDBManager(dataDir)
|
again := databasetest.NewWebhookDBManager(t, dataDir)
|
||||||
|
|
||||||
_, err = again.GetDB(webhookID)
|
_, err = again.GetDB(webhookID)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
@@ -195,7 +196,7 @@ func TestOpenPurgeRemovesCredentialBytes(t *testing.T) {
|
|||||||
webhookID := uuid.New().String()
|
webhookID := uuid.New().String()
|
||||||
credential := "T00000000/B00000000/" + uuid.New().String()
|
credential := "T00000000/B00000000/" + uuid.New().String()
|
||||||
|
|
||||||
initial := database.NewTestWebhookDBManager(dataDir)
|
initial := databasetest.NewWebhookDBManager(t, dataDir)
|
||||||
|
|
||||||
_, err := initial.GetDB(webhookID)
|
_, err := initial.GetDB(webhookID)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
@@ -230,7 +231,7 @@ func TestOpenPurgeRemovesCredentialBytes(t *testing.T) {
|
|||||||
"seeded credential is not in the file, so this test proves nothing",
|
"seeded credential is not in the file, so this test proves nothing",
|
||||||
)
|
)
|
||||||
|
|
||||||
mgr := database.NewTestWebhookDBManager(dataDir)
|
mgr := databasetest.NewWebhookDBManager(t, dataDir)
|
||||||
|
|
||||||
_, err = mgr.GetDB(webhookID)
|
_, err = mgr.GetDB(webhookID)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
@@ -258,7 +259,7 @@ func TestOpenRevacuumsAfterIncompleteSweep(t *testing.T) {
|
|||||||
webhookID := uuid.New().String()
|
webhookID := uuid.New().String()
|
||||||
credential := "T00000000/B00000000/" + uuid.New().String()
|
credential := "T00000000/B00000000/" + uuid.New().String()
|
||||||
|
|
||||||
initial := database.NewTestWebhookDBManager(dataDir)
|
initial := databasetest.NewWebhookDBManager(t, dataDir)
|
||||||
|
|
||||||
_, err := initial.GetDB(webhookID)
|
_, err := initial.GetDB(webhookID)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
@@ -298,7 +299,7 @@ func TestOpenRevacuumsAfterIncompleteSweep(t *testing.T) {
|
|||||||
"test proves nothing",
|
"test proves nothing",
|
||||||
)
|
)
|
||||||
|
|
||||||
mgr := database.NewTestWebhookDBManager(dataDir)
|
mgr := databasetest.NewWebhookDBManager(t, dataDir)
|
||||||
|
|
||||||
_, err = mgr.GetDB(webhookID)
|
_, err = mgr.GetDB(webhookID)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
@@ -325,7 +326,7 @@ func TestOpenSkipsSweptDatabase(t *testing.T) {
|
|||||||
dataDir := eventDBDataDir(t)
|
dataDir := eventDBDataDir(t)
|
||||||
webhookID := uuid.New().String()
|
webhookID := uuid.New().String()
|
||||||
|
|
||||||
mgr := database.NewTestWebhookDBManager(dataDir)
|
mgr := databasetest.NewWebhookDBManager(t, dataDir)
|
||||||
|
|
||||||
_, err := mgr.GetDB(webhookID)
|
_, err := mgr.GetDB(webhookID)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
@@ -347,7 +348,7 @@ func TestOpenSkipsSweptDatabase(t *testing.T) {
|
|||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
require.NoError(t, marked.Close())
|
require.NoError(t, marked.Close())
|
||||||
|
|
||||||
again := database.NewTestWebhookDBManager(dataDir)
|
again := databasetest.NewWebhookDBManager(t, dataDir)
|
||||||
|
|
||||||
_, err = again.GetDB(webhookID)
|
_, err = again.GetDB(webhookID)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
@@ -378,7 +379,7 @@ func TestOpenSucceedsWithoutTargetsTable(t *testing.T) {
|
|||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
require.NoError(t, seed.Close())
|
require.NoError(t, seed.Close())
|
||||||
|
|
||||||
mgr := database.NewTestWebhookDBManager(dataDir)
|
mgr := databasetest.NewWebhookDBManager(t, dataDir)
|
||||||
|
|
||||||
db, err := mgr.GetDB(webhookID)
|
db, err := mgr.GetDB(webhookID)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
@@ -396,7 +397,7 @@ func TestEventDBCreateOmitsAssociations(t *testing.T) {
|
|||||||
dataDir := eventDBDataDir(t)
|
dataDir := eventDBDataDir(t)
|
||||||
webhookID := uuid.New().String()
|
webhookID := uuid.New().String()
|
||||||
|
|
||||||
mgr := database.NewTestWebhookDBManager(dataDir)
|
mgr := databasetest.NewWebhookDBManager(t, dataDir)
|
||||||
|
|
||||||
db, err := mgr.GetDB(webhookID)
|
db, err := mgr.GetDB(webhookID)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
|
|||||||
@@ -51,7 +51,7 @@ func setupRetentionTest(t *testing.T) *retentionTestEnv {
|
|||||||
|
|
||||||
mgr, err := database.NewWebhookDBManager(
|
mgr, err := database.NewWebhookDBManager(
|
||||||
lc,
|
lc,
|
||||||
database.WebhookDBManagerParams{Config: cfg, Logger: l},
|
database.WebhookDBManagerParams{Config: cfg, Logger: l.Get()},
|
||||||
)
|
)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
|||||||
@@ -1,47 +0,0 @@
|
|||||||
package database
|
|
||||||
|
|
||||||
import (
|
|
||||||
"log/slog"
|
|
||||||
"os"
|
|
||||||
|
|
||||||
"gorm.io/gorm"
|
|
||||||
)
|
|
||||||
|
|
||||||
// NewTestDatabase creates a Database wrapper around a pre-opened *gorm.DB.
|
|
||||||
// Intended for use in tests that need a *database.Database without the
|
|
||||||
// full fx lifecycle. The caller is responsible for closing the underlying
|
|
||||||
// sql.DB connection.
|
|
||||||
func NewTestDatabase(db *gorm.DB) *Database {
|
|
||||||
return &Database{
|
|
||||||
db: db,
|
|
||||||
log: slog.New(slog.NewTextHandler(
|
|
||||||
os.Stderr,
|
|
||||||
&slog.HandlerOptions{Level: slog.LevelDebug},
|
|
||||||
)),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// NewTestWebhookDBManager creates a WebhookDBManager backed by the given
|
|
||||||
// data directory. Intended for use in tests without the fx lifecycle.
|
|
||||||
func NewTestWebhookDBManager(dataDir string) *WebhookDBManager {
|
|
||||||
return NewTestWebhookDBManagerWithLogger(
|
|
||||||
dataDir,
|
|
||||||
slog.New(slog.NewTextHandler(
|
|
||||||
os.Stderr,
|
|
||||||
&slog.HandlerOptions{Level: slog.LevelDebug},
|
|
||||||
)),
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
// NewTestWebhookDBManagerWithLogger is NewTestWebhookDBManager with the
|
|
||||||
// logger supplied by the caller. The per-webhook databases this manager
|
|
||||||
// opens hand that logger to gormlog, so a test that needs to see the SQL
|
|
||||||
// the service emits can capture it.
|
|
||||||
func NewTestWebhookDBManagerWithLogger(
|
|
||||||
dataDir string, log *slog.Logger,
|
|
||||||
) *WebhookDBManager {
|
|
||||||
return &WebhookDBManager{
|
|
||||||
dataDir: dataDir,
|
|
||||||
log: log,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -15,7 +15,6 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
"sneak.berlin/go/webhooker/internal/datadir"
|
"sneak.berlin/go/webhooker/internal/datadir"
|
||||||
"sneak.berlin/go/webhooker/internal/gormlog"
|
"sneak.berlin/go/webhooker/internal/gormlog"
|
||||||
"sneak.berlin/go/webhooker/internal/logger"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// WebhookDBManagerParams holds the fx dependencies for
|
// WebhookDBManagerParams holds the fx dependencies for
|
||||||
@@ -24,7 +23,7 @@ type WebhookDBManagerParams struct {
|
|||||||
fx.In
|
fx.In
|
||||||
|
|
||||||
Config *config.Config
|
Config *config.Config
|
||||||
Logger *logger.Logger
|
Logger *slog.Logger
|
||||||
}
|
}
|
||||||
|
|
||||||
// errInvalidCachedDBType indicates a type assertion failure
|
// errInvalidCachedDBType indicates a type assertion failure
|
||||||
@@ -70,7 +69,7 @@ func NewWebhookDBManager(
|
|||||||
) (*WebhookDBManager, error) {
|
) (*WebhookDBManager, error) {
|
||||||
m := &WebhookDBManager{
|
m := &WebhookDBManager{
|
||||||
dataDir: params.Config.DataDir,
|
dataDir: params.Config.DataDir,
|
||||||
log: params.Logger.Get(),
|
log: params.Logger,
|
||||||
}
|
}
|
||||||
|
|
||||||
// Create data directory if it doesn't exist. datadir.DirPerm is the
|
// Create data directory if it doesn't exist. datadir.DirPerm is the
|
||||||
|
|||||||
@@ -18,6 +18,7 @@ import (
|
|||||||
"gorm.io/gorm"
|
"gorm.io/gorm"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database/databasetest"
|
||||||
"sneak.berlin/go/webhooker/internal/globals"
|
"sneak.berlin/go/webhooker/internal/globals"
|
||||||
"sneak.berlin/go/webhooker/internal/logger"
|
"sneak.berlin/go/webhooker/internal/logger"
|
||||||
)
|
)
|
||||||
@@ -50,7 +51,7 @@ func setupTestWebhookDBManager(
|
|||||||
lc,
|
lc,
|
||||||
database.WebhookDBManagerParams{
|
database.WebhookDBManagerParams{
|
||||||
Config: cfg,
|
Config: cfg,
|
||||||
Logger: l,
|
Logger: l.Get(),
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
@@ -117,7 +118,8 @@ func TestWebhookDBManager_ConcurrentFirstTouchOpensOnce(t *testing.T) {
|
|||||||
|
|
||||||
var logs bytes.Buffer
|
var logs bytes.Buffer
|
||||||
|
|
||||||
mgr := database.NewTestWebhookDBManagerWithLogger(
|
mgr := databasetest.NewWebhookDBManagerWithLogger(
|
||||||
|
t,
|
||||||
t.TempDir(),
|
t.TempDir(),
|
||||||
slog.New(slog.NewTextHandler(&logs, nil)),
|
slog.New(slog.NewTextHandler(&logs, nil)),
|
||||||
)
|
)
|
||||||
@@ -307,7 +309,8 @@ func TestWebhookDBManager_LostDatabaseIsLogged(t *testing.T) {
|
|||||||
|
|
||||||
var logs bytes.Buffer
|
var logs bytes.Buffer
|
||||||
|
|
||||||
mgr := database.NewTestWebhookDBManagerWithLogger(
|
mgr := databasetest.NewWebhookDBManagerWithLogger(
|
||||||
|
t,
|
||||||
t.TempDir(),
|
t.TempDir(),
|
||||||
slog.New(slog.NewTextHandler(&logs, nil)),
|
slog.New(slog.NewTextHandler(&logs, nil)),
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ import (
|
|||||||
"gorm.io/gorm/clause"
|
"gorm.io/gorm/clause"
|
||||||
_ "modernc.org/sqlite" // Pure Go SQLite driver.
|
_ "modernc.org/sqlite" // Pure Go SQLite driver.
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database/databasetest"
|
||||||
"sneak.berlin/go/webhooker/internal/delivery"
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
"sneak.berlin/go/webhooker/internal/gormlog"
|
"sneak.berlin/go/webhooker/internal/gormlog"
|
||||||
)
|
)
|
||||||
@@ -59,29 +60,14 @@ func setupArchiveTest(t *testing.T) *archiveEnv {
|
|||||||
dataDir := t.TempDir()
|
dataDir := t.TempDir()
|
||||||
log := archiveTestLogger()
|
log := archiveTestLogger()
|
||||||
|
|
||||||
sqlDB, err := sql.Open(
|
mainDB, err := database.Open(dataDir, slog.New(slog.DiscardHandler))
|
||||||
"sqlite",
|
|
||||||
fmt.Sprintf(
|
|
||||||
"file:%s?mode=rwc",
|
|
||||||
filepath.Join(dataDir, "main.db"),
|
|
||||||
),
|
|
||||||
)
|
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
|
|
||||||
t.Cleanup(func() { _ = sqlDB.Close() })
|
t.Cleanup(func() { _ = mainDB.Close() })
|
||||||
|
|
||||||
gdb, err := gorm.Open(
|
|
||||||
sqlite.Dialector{Conn: sqlDB},
|
|
||||||
&gorm.Config{Logger: gormlog.New(slog.New(slog.DiscardHandler))},
|
|
||||||
)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
mainDB := database.NewTestDatabase(gdb)
|
|
||||||
require.NoError(t, mainDB.Migrate())
|
|
||||||
|
|
||||||
eng := delivery.NewTestEngineWithDB(
|
eng := delivery.NewTestEngineWithDB(
|
||||||
mainDB,
|
mainDB,
|
||||||
database.NewTestWebhookDBManager(dataDir),
|
databasetest.NewWebhookDBManager(t, dataDir),
|
||||||
log,
|
log,
|
||||||
&http.Client{Timeout: 5 * time.Second},
|
&http.Client{Timeout: 5 * time.Second},
|
||||||
1,
|
1,
|
||||||
|
|||||||
@@ -10,7 +10,6 @@ import (
|
|||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
"strings"
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
"testing"
|
"testing"
|
||||||
@@ -19,12 +18,11 @@ import (
|
|||||||
"github.com/google/uuid"
|
"github.com/google/uuid"
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"gorm.io/driver/sqlite"
|
|
||||||
"gorm.io/gorm"
|
"gorm.io/gorm"
|
||||||
_ "modernc.org/sqlite"
|
_ "modernc.org/sqlite"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database/databasetest"
|
||||||
"sneak.berlin/go/webhooker/internal/delivery"
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
"sneak.berlin/go/webhooker/internal/gormlog"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// iSetup holds common integration test dependencies.
|
// iSetup holds common integration test dependencies.
|
||||||
@@ -45,12 +43,12 @@ func newISetup(t *testing.T) iSetup {
|
|||||||
wDB := iSeedWebhookDB(t, dbMgr, wID)
|
wDB := iSeedWebhookDB(t, dbMgr, wID)
|
||||||
|
|
||||||
return iSetup{
|
return iSetup{
|
||||||
MainDB: mainDB,
|
MainDB: mainDB.DB(),
|
||||||
DBMgr: dbMgr,
|
DBMgr: dbMgr,
|
||||||
WebhookID: wID,
|
WebhookID: wID,
|
||||||
WebhookDB: wDB,
|
WebhookDB: wDB,
|
||||||
Engine: delivery.NewTestEngineWithDB(
|
Engine: delivery.NewTestEngineWithDB(
|
||||||
database.NewTestDatabase(mainDB),
|
mainDB,
|
||||||
dbMgr,
|
dbMgr,
|
||||||
slog.New(slog.NewTextHandler(
|
slog.New(slog.NewTextHandler(
|
||||||
os.Stderr,
|
os.Stderr,
|
||||||
@@ -64,35 +62,16 @@ func newISetup(t *testing.T) iSetup {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func iMainDB(t *testing.T) *gorm.DB {
|
// iMainDB opens a main database through database.Open, the way the
|
||||||
|
// service opens it, so these tests cannot pass against journal and
|
||||||
|
// locking settings production does not use.
|
||||||
|
func iMainDB(t *testing.T) *database.Database {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
dbPath := filepath.Join(
|
db, err := database.Open(t.TempDir(), slog.New(slog.DiscardHandler))
|
||||||
t.TempDir(), "main-test.db",
|
|
||||||
)
|
|
||||||
|
|
||||||
// Opened the way the service opens the main database, so these
|
|
||||||
// tests cannot pass against journal and locking settings
|
|
||||||
// production does not use.
|
|
||||||
sqlDB, err := database.OpenSQLite(
|
|
||||||
dbPath, database.SQLiteModeCreate,
|
|
||||||
)
|
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
|
|
||||||
t.Cleanup(func() { _ = sqlDB.Close() })
|
t.Cleanup(func() { _ = db.Close() })
|
||||||
|
|
||||||
db, err := gorm.Open(
|
|
||||||
sqlite.Dialector{Conn: sqlDB},
|
|
||||||
&gorm.Config{Logger: gormlog.New(slog.New(slog.DiscardHandler))},
|
|
||||||
)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
require.NoError(t, db.AutoMigrate(
|
|
||||||
&database.Webhook{},
|
|
||||||
&database.Target{},
|
|
||||||
&database.User{},
|
|
||||||
&database.Setting{},
|
|
||||||
))
|
|
||||||
|
|
||||||
return db
|
return db
|
||||||
}
|
}
|
||||||
@@ -102,7 +81,7 @@ func iDBManager(
|
|||||||
) *database.WebhookDBManager {
|
) *database.WebhookDBManager {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
return database.NewTestWebhookDBManager(t.TempDir())
|
return databasetest.NewWebhookDBManager(t, t.TempDir())
|
||||||
}
|
}
|
||||||
|
|
||||||
func iSeedWebhookDB(
|
func iSeedWebhookDB(
|
||||||
@@ -1154,17 +1133,17 @@ func TestRecoverInFlight_ReportsAMissingWebhookDatabase(t *testing.T) {
|
|||||||
|
|
||||||
mainDB := iMainDB(t)
|
mainDB := iMainDB(t)
|
||||||
webhookID := uuid.New().String()
|
webhookID := uuid.New().String()
|
||||||
iCreateWebhook(t, mainDB, webhookID, "lost-database")
|
iCreateWebhook(t, mainDB.DB(), webhookID, "lost-database")
|
||||||
|
|
||||||
var logs bytes.Buffer
|
var logs bytes.Buffer
|
||||||
|
|
||||||
dbMgr := database.NewTestWebhookDBManagerWithLogger(
|
dbMgr := databasetest.NewWebhookDBManagerWithLogger(
|
||||||
t.TempDir(), slog.New(slog.NewTextHandler(&logs, nil)),
|
t, t.TempDir(), slog.New(slog.NewTextHandler(&logs, nil)),
|
||||||
)
|
)
|
||||||
t.Cleanup(func() { _ = dbMgr.CloseAll() })
|
t.Cleanup(func() { _ = dbMgr.CloseAll() })
|
||||||
|
|
||||||
engine := delivery.NewTestEngineWithDB(
|
engine := delivery.NewTestEngineWithDB(
|
||||||
database.NewTestDatabase(mainDB), dbMgr,
|
mainDB, dbMgr,
|
||||||
slog.New(slog.DiscardHandler),
|
slog.New(slog.DiscardHandler),
|
||||||
&http.Client{Timeout: 5 * time.Second}, 1,
|
&http.Client{Timeout: 5 * time.Second}, 1,
|
||||||
)
|
)
|
||||||
@@ -1188,14 +1167,14 @@ func TestRecoverInFlight_SkipsAWebhookDeletedAfterTheListIsRead(
|
|||||||
|
|
||||||
mainDB := iMainDB(t)
|
mainDB := iMainDB(t)
|
||||||
webhookID := uuid.New().String()
|
webhookID := uuid.New().String()
|
||||||
iCreateWebhook(t, mainDB, webhookID, "deleted-during-recovery")
|
iCreateWebhook(t, mainDB.DB(), webhookID, "deleted-during-recovery")
|
||||||
|
|
||||||
// The first query to return is recovery's read of the list of
|
// The first query to return is recovery's read of the list of
|
||||||
// webhooks. Deleting the webhook right after it puts the delete
|
// webhooks. Deleting the webhook right after it puts the delete
|
||||||
// between that read and the opening of the webhook's database.
|
// between that read and the opening of the webhook's database.
|
||||||
deleted := false
|
deleted := false
|
||||||
|
|
||||||
require.NoError(t, mainDB.Callback().Query().After("gorm:query").
|
require.NoError(t, mainDB.DB().Callback().Query().After("gorm:query").
|
||||||
Register("delete-after-list", func(*gorm.DB) {
|
Register("delete-after-list", func(*gorm.DB) {
|
||||||
if deleted {
|
if deleted {
|
||||||
return
|
return
|
||||||
@@ -1203,16 +1182,16 @@ func TestRecoverInFlight_SkipsAWebhookDeletedAfterTheListIsRead(
|
|||||||
|
|
||||||
deleted = true
|
deleted = true
|
||||||
|
|
||||||
require.NoError(t, mainDB.Delete(
|
require.NoError(t, mainDB.DB().Delete(
|
||||||
&database.Webhook{}, "id = ?", webhookID,
|
&database.Webhook{}, "id = ?", webhookID,
|
||||||
).Error)
|
).Error)
|
||||||
}))
|
}))
|
||||||
|
|
||||||
dbMgr := database.NewTestWebhookDBManager(t.TempDir())
|
dbMgr := databasetest.NewWebhookDBManager(t, t.TempDir())
|
||||||
t.Cleanup(func() { _ = dbMgr.CloseAll() })
|
t.Cleanup(func() { _ = dbMgr.CloseAll() })
|
||||||
|
|
||||||
engine := delivery.NewTestEngineWithDB(
|
engine := delivery.NewTestEngineWithDB(
|
||||||
database.NewTestDatabase(mainDB), dbMgr,
|
mainDB, dbMgr,
|
||||||
slog.New(slog.DiscardHandler),
|
slog.New(slog.DiscardHandler),
|
||||||
&http.Client{Timeout: 5 * time.Second}, 1,
|
&http.Client{Timeout: 5 * time.Second}, 1,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -48,8 +48,9 @@ func fSweepSetup(
|
|||||||
//
|
//
|
||||||
// Every caller drives the dispatch paths synchronously and has already
|
// Every caller drives the dispatch paths synchronously and has already
|
||||||
// waited for them to return, so anything they queued is in the channel
|
// waited for them to return, so anything they queued is in the channel
|
||||||
// by now. The short grace covers nothing but scheduler jitter, and is
|
// by now, and nothing is waited for. A timer here would race the queued
|
||||||
// kept small because one of these tests runs the drain forty times.
|
// tasks: on a busy host it can be due by the time select looks, and
|
||||||
|
// select picks at random among the cases that are ready.
|
||||||
func fDrain(e *delivery.Engine) []delivery.Task {
|
func fDrain(e *delivery.Engine) []delivery.Task {
|
||||||
var out []delivery.Task
|
var out []delivery.Task
|
||||||
|
|
||||||
@@ -59,7 +60,7 @@ func fDrain(e *delivery.Engine) []delivery.Task {
|
|||||||
out = append(out, task)
|
out = append(out, task)
|
||||||
case task := <-e.ExportRetryCh():
|
case task := <-e.ExportRetryCh():
|
||||||
out = append(out, task)
|
out = append(out, task)
|
||||||
case <-time.After(25 * time.Millisecond):
|
default:
|
||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -5,7 +5,6 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net/http"
|
"net/http"
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
"strings"
|
||||||
"sync"
|
"sync"
|
||||||
"testing"
|
"testing"
|
||||||
@@ -14,11 +13,9 @@ import (
|
|||||||
"github.com/google/uuid"
|
"github.com/google/uuid"
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"gorm.io/driver/sqlite"
|
|
||||||
"gorm.io/gorm"
|
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database/databasetest"
|
||||||
"sneak.berlin/go/webhooker/internal/delivery"
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
"sneak.berlin/go/webhooker/internal/gormlog"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// qdAggregateMarker identifies the queue-depth aggregate in the
|
// qdAggregateMarker identifies the queue-depth aggregate in the
|
||||||
@@ -49,27 +46,13 @@ func (q *qdSyncBuf) String() string {
|
|||||||
|
|
||||||
// qdMainDB opens a main database whose GORM logger is the service's
|
// qdMainDB opens a main database whose GORM logger is the service's
|
||||||
// adapter, writing through log.
|
// adapter, writing through log.
|
||||||
func qdMainDB(t *testing.T, log *slog.Logger) *gorm.DB {
|
func qdMainDB(t *testing.T, log *slog.Logger) *database.Database {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
sqlDB, err := database.OpenSQLite(
|
db, err := database.Open(t.TempDir(), log)
|
||||||
filepath.Join(t.TempDir(), "main-gormlog.db"),
|
|
||||||
database.SQLiteModeCreate,
|
|
||||||
)
|
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
|
|
||||||
t.Cleanup(func() { _ = sqlDB.Close() })
|
t.Cleanup(func() { _ = db.Close() })
|
||||||
|
|
||||||
db, err := gorm.Open(
|
|
||||||
sqlite.Dialector{Conn: sqlDB},
|
|
||||||
&gorm.Config{Logger: gormlog.New(log)},
|
|
||||||
)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
require.NoError(t, db.AutoMigrate(
|
|
||||||
&database.Webhook{},
|
|
||||||
&database.Target{},
|
|
||||||
))
|
|
||||||
|
|
||||||
return db
|
return db
|
||||||
}
|
}
|
||||||
@@ -106,18 +89,18 @@ func TestQueueDepthSample_LogsNoBoundValue(t *testing.T) {
|
|||||||
))
|
))
|
||||||
|
|
||||||
mainDB := qdMainDB(t, log)
|
mainDB := qdMainDB(t, log)
|
||||||
dbMgr := database.NewTestWebhookDBManagerWithLogger(
|
dbMgr := databasetest.NewWebhookDBManagerWithLogger(
|
||||||
t.TempDir(), log,
|
t, t.TempDir(), log,
|
||||||
)
|
)
|
||||||
|
|
||||||
webhookID := uuid.New().String()
|
webhookID := uuid.New().String()
|
||||||
webhookDB := iSeedWebhookDB(t, dbMgr, webhookID)
|
webhookDB := iSeedWebhookDB(t, dbMgr, webhookID)
|
||||||
|
|
||||||
iCreateWebhook(t, mainDB, webhookID, "queue-depth-gormlog")
|
iCreateWebhook(t, mainDB.DB(), webhookID, "queue-depth-gormlog")
|
||||||
|
|
||||||
targetID := uuid.New().String()
|
targetID := uuid.New().String()
|
||||||
|
|
||||||
iCreateTarget(t, mainDB, targetID, webhookID,
|
iCreateTarget(t, mainDB.DB(), targetID, webhookID,
|
||||||
"queue-depth-gormlog-target", database.TargetTypeHTTP,
|
"queue-depth-gormlog-target", database.TargetTypeHTTP,
|
||||||
iHTTPConfig("https://example.com/hook"), 3,
|
iHTTPConfig("https://example.com/hook"), 3,
|
||||||
)
|
)
|
||||||
@@ -136,7 +119,7 @@ func TestQueueDepthSample_LogsNoBoundValue(t *testing.T) {
|
|||||||
)
|
)
|
||||||
|
|
||||||
engine := delivery.NewTestEngineWithDB(
|
engine := delivery.NewTestEngineWithDB(
|
||||||
database.NewTestDatabase(mainDB),
|
mainDB,
|
||||||
dbMgr,
|
dbMgr,
|
||||||
log,
|
log,
|
||||||
&http.Client{Timeout: 5 * time.Second},
|
&http.Client{Timeout: 5 * time.Second},
|
||||||
|
|||||||
@@ -460,8 +460,13 @@ func TestArchiveExport_OneFileOpenAtATime(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// heapPeak is an io.Writer that discards what it is given and records
|
// heapPeak is an io.Writer that discards what it is given and records
|
||||||
// the largest heap it saw at a write. It collects garbage before each
|
// the largest heap it saw at a write. It collects garbage twice before
|
||||||
// reading, so the heap it reads is what is still held.
|
// each reading, so the heap it reads is what is still held. Once is not
|
||||||
|
// enough: the libraries the export calls (regexp, under GORM's table
|
||||||
|
// names, and encoding/json among them) cache buffers in a sync.Pool,
|
||||||
|
// which keeps them through one collection, so after one the reading
|
||||||
|
// counts however many happen to be cached. That varies from run to run
|
||||||
|
// by about as much as the limit in TestArchiveExport_Streams.
|
||||||
type heapPeak struct {
|
type heapPeak struct {
|
||||||
max uint64
|
max uint64
|
||||||
}
|
}
|
||||||
@@ -469,6 +474,7 @@ type heapPeak struct {
|
|||||||
func (p *heapPeak) Write(b []byte) (int, error) {
|
func (p *heapPeak) Write(b []byte) (int, error) {
|
||||||
var m runtime.MemStats
|
var m runtime.MemStats
|
||||||
|
|
||||||
|
runtime.GC()
|
||||||
runtime.GC()
|
runtime.GC()
|
||||||
runtime.ReadMemStats(&m)
|
runtime.ReadMemStats(&m)
|
||||||
p.max = max(p.max, m.HeapAlloc)
|
p.max = max(p.max, m.HeapAlloc)
|
||||||
@@ -498,6 +504,8 @@ func exportHeapGrowth(t *testing.T, rows, bodySize int) uint64 {
|
|||||||
|
|
||||||
export := listExport(t, path)
|
export := listExport(t, path)
|
||||||
|
|
||||||
|
// Twice, for the reason heapPeak gives.
|
||||||
|
runtime.GC()
|
||||||
runtime.GC()
|
runtime.GC()
|
||||||
|
|
||||||
var start runtime.MemStats
|
var start runtime.MemStats
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ import (
|
|||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database/databasetest"
|
||||||
"sneak.berlin/go/webhooker/internal/delivery"
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -335,7 +336,7 @@ func TestArchivePathAt(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
dataDir := t.TempDir()
|
dataDir := t.TempDir()
|
||||||
dbMgr := database.NewTestWebhookDBManager(dataDir)
|
dbMgr := databasetest.NewWebhookDBManager(t, dataDir)
|
||||||
webhook := &database.Webhook{
|
webhook := &database.Webhook{
|
||||||
BaseModel: database.BaseModel{ID: "wh-id"}, Name: "Orders",
|
BaseModel: database.BaseModel{ID: "wh-id"}, Name: "Orders",
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ package gormlog_test
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"database/sql"
|
"database/sql"
|
||||||
|
"log/slog"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"testing"
|
"testing"
|
||||||
@@ -13,6 +14,7 @@ import (
|
|||||||
"go.uber.org/fx/fxtest"
|
"go.uber.org/fx/fxtest"
|
||||||
_ "modernc.org/sqlite" // Pure Go SQLite driver.
|
_ "modernc.org/sqlite" // Pure Go SQLite driver.
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
|
"sneak.berlin/go/webhooker/internal/config/configtest"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
"sneak.berlin/go/webhooker/internal/globals"
|
"sneak.berlin/go/webhooker/internal/globals"
|
||||||
"sneak.berlin/go/webhooker/internal/logger"
|
"sneak.berlin/go/webhooker/internal/logger"
|
||||||
@@ -128,7 +130,7 @@ func readFirstBootSecrets(
|
|||||||
func bootAtDebug(t *testing.T, dataDir string) string {
|
func bootAtDebug(t *testing.T, dataDir string) string {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
config.ClearEnvForTest(t)
|
configtest.ClearEnv(t)
|
||||||
t.Setenv("DEBUG", "true")
|
t.Setenv("DEBUG", "true")
|
||||||
t.Setenv("DATA_DIR", dataDir)
|
t.Setenv("DATA_DIR", dataDir)
|
||||||
|
|
||||||
@@ -145,6 +147,7 @@ func bootAtDebug(t *testing.T, dataDir string) string {
|
|||||||
fx.Provide(
|
fx.Provide(
|
||||||
globals.New,
|
globals.New,
|
||||||
logger.New,
|
logger.New,
|
||||||
|
func(l *logger.Logger) *slog.Logger { return l.Get() },
|
||||||
config.New,
|
config.New,
|
||||||
database.New,
|
database.New,
|
||||||
session.New,
|
session.New,
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"html/template"
|
"html/template"
|
||||||
|
"log/slog"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"sync"
|
"sync"
|
||||||
@@ -249,6 +250,7 @@ func newTestAppWithConfig(
|
|||||||
fx.Provide(
|
fx.Provide(
|
||||||
globals.New,
|
globals.New,
|
||||||
logger.New,
|
logger.New,
|
||||||
|
func(l *logger.Logger) *slog.Logger { return l.Get() },
|
||||||
func() *config.Config { return cfg },
|
func() *config.Config { return cfg },
|
||||||
database.New,
|
database.New,
|
||||||
database.NewWebhookDBManager,
|
database.NewWebhookDBManager,
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
"sneak.berlin/go/webhooker/internal/handlers"
|
"sneak.berlin/go/webhooker/internal/handlers"
|
||||||
"sneak.berlin/go/webhooker/internal/logger"
|
"sneak.berlin/go/webhooker/internal/logger"
|
||||||
"sneak.berlin/go/webhooker/internal/middleware"
|
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
|
||||||
"sneak.berlin/go/webhooker/internal/session"
|
"sneak.berlin/go/webhooker/internal/session"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -135,7 +135,7 @@ func TestUserRoute_Unauthenticated_RedirectedByMiddleware(t *testing.T) {
|
|||||||
|
|
||||||
t.Cleanup(app.RequireStop)
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
mw := middleware.NewForTest(log.Get(), cfg, sess)
|
mw := middlewaretest.New(t, log.Get(), cfg, sess)
|
||||||
|
|
||||||
var handlerReached bool
|
var handlerReached bool
|
||||||
|
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ import (
|
|||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
"sneak.berlin/go/webhooker/internal/middleware"
|
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
|
||||||
)
|
)
|
||||||
|
|
||||||
// targetSecretSegments are the path segments of an incoming-webhook
|
// targetSecretSegments are the path segments of an incoming-webhook
|
||||||
@@ -65,7 +65,8 @@ func postTargetCreate(
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
logBuf := new(bytes.Buffer)
|
logBuf := new(bytes.Buffer)
|
||||||
mw := middleware.NewForTest(
|
mw := middlewaretest.New(
|
||||||
|
t,
|
||||||
slog.New(slog.NewJSONHandler(
|
slog.New(slog.NewJSONHandler(
|
||||||
logBuf, &slog.HandlerOptions{Level: slog.LevelInfo},
|
logBuf, &slog.HandlerOptions{Level: slog.LevelInfo},
|
||||||
)),
|
)),
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
"sneak.berlin/go/webhooker/internal/delivery"
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
"sneak.berlin/go/webhooker/internal/middleware"
|
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
|
||||||
)
|
)
|
||||||
|
|
||||||
// errClientGone is the write failure of a client that has gone away.
|
// errClientGone is the write failure of a client that has gone away.
|
||||||
@@ -310,7 +310,8 @@ func limitedServer(
|
|||||||
const sendBuffer = 4 << 10
|
const sendBuffer = 4 << 10
|
||||||
|
|
||||||
logBuf := new(bytes.Buffer)
|
logBuf := new(bytes.Buffer)
|
||||||
mw := middleware.NewForTest(
|
mw := middlewaretest.New(
|
||||||
|
t,
|
||||||
slog.New(slog.NewJSONHandler(logBuf, nil)),
|
slog.New(slog.NewJSONHandler(logBuf, nil)),
|
||||||
&config.Config{Environment: config.EnvironmentDev},
|
&config.Config{Environment: config.EnvironmentDev},
|
||||||
nil,
|
nil,
|
||||||
|
|||||||
@@ -16,6 +16,7 @@ import (
|
|||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
"sneak.berlin/go/webhooker/internal/middleware"
|
"sneak.berlin/go/webhooker/internal/middleware"
|
||||||
|
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
|
||||||
)
|
)
|
||||||
|
|
||||||
// floodRequests is the number of distinct invented paths each flood
|
// floodRequests is the number of distinct invented paths each flood
|
||||||
@@ -83,7 +84,7 @@ func capturingMiddleware(t *testing.T) (*middleware.Middleware, *bytes.Buffer) {
|
|||||||
TrustedProxies: trustedProxies("192.0.2.1/32"),
|
TrustedProxies: trustedProxies("192.0.2.1/32"),
|
||||||
}
|
}
|
||||||
|
|
||||||
return middleware.NewForTest(log, cfg, nil), buf
|
return middlewaretest.New(t, log, cfg, nil), buf
|
||||||
}
|
}
|
||||||
|
|
||||||
// capturingTextMiddleware is capturingMiddleware for the other handler
|
// capturingTextMiddleware is capturingMiddleware for the other handler
|
||||||
@@ -107,7 +108,7 @@ func capturingTextMiddleware(
|
|||||||
TrustedProxies: trustedProxies("192.0.2.1/32"),
|
TrustedProxies: trustedProxies("192.0.2.1/32"),
|
||||||
}
|
}
|
||||||
|
|
||||||
return middleware.NewForTest(log, cfg, nil), buf
|
return middlewaretest.New(t, log, cfg, nil), buf
|
||||||
}
|
}
|
||||||
|
|
||||||
// accessLogRouter mirrors the production route shapes that an
|
// accessLogRouter mirrors the production route shapes that an
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ import (
|
|||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
"sneak.berlin/go/webhooker/internal/middleware"
|
"sneak.berlin/go/webhooker/internal/middleware"
|
||||||
|
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -133,8 +134,8 @@ func clientLogLines(
|
|||||||
TrustedProxies: trustedProxies(trustedProxyCIDR),
|
TrustedProxies: trustedProxies(trustedProxyCIDR),
|
||||||
}
|
}
|
||||||
|
|
||||||
m := middleware.NewForTest(
|
m := middlewaretest.New(
|
||||||
log, cfg, newTestSessionManager(cfg, log, nil),
|
t, log, cfg, newTestSessionManager(t, cfg),
|
||||||
)
|
)
|
||||||
handler := m.Logging()(site.build(m))
|
handler := m.Logging()(site.build(m))
|
||||||
|
|
||||||
|
|||||||
@@ -41,6 +41,7 @@ import (
|
|||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
"sneak.berlin/go/webhooker/internal/middleware"
|
"sneak.berlin/go/webhooker/internal/middleware"
|
||||||
|
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
|
||||||
)
|
)
|
||||||
|
|
||||||
// bodyLimitBytes is the MaxBodySize cap these tests install. Any
|
// bodyLimitBytes is the MaxBodySize cap these tests install. Any
|
||||||
@@ -155,9 +156,9 @@ func capturingBoundMiddleware(
|
|||||||
ReceiverRateLimit: receiverLimitPerMinute,
|
ReceiverRateLimit: receiverLimitPerMinute,
|
||||||
}
|
}
|
||||||
|
|
||||||
sess := newTestSessionManager(cfg, log, nil)
|
sess := newTestSessionManager(t, cfg)
|
||||||
|
|
||||||
return middleware.NewForTest(log, cfg, sess), buf
|
return middlewaretest.New(t, log, cfg, sess), buf
|
||||||
}
|
}
|
||||||
|
|
||||||
// unreachable is a next-handler that fails the test if the middleware
|
// unreachable is a next-handler that fails the test if the middleware
|
||||||
|
|||||||
@@ -236,9 +236,9 @@ func TestLoginGuard_SemaphoreBoundsConcurrentVerifications(
|
|||||||
|
|
||||||
// rendezvousDeadlock is the deadlock guard described below.
|
// rendezvousDeadlock is the deadlock guard described below.
|
||||||
// It is orders of magnitude longer than any scheduling delay,
|
// It is orders of magnitude longer than any scheduling delay,
|
||||||
// so it never decides the result, and well inside script/test's
|
// so it never decides the result, and well inside the 90s
|
||||||
// 30s timeout, so a wedge fails on the assertion instead of
|
// package timeout of the Dockerfile's test phase, so a wedge
|
||||||
// blowing the package timeout.
|
// fails on the assertion instead of blowing that timeout.
|
||||||
rendezvousDeadlock = 5 * time.Second
|
rendezvousDeadlock = 5 * time.Second
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -151,7 +151,7 @@ var _ httpmetrics.Recorder = boundedLabelRecorder{}
|
|||||||
|
|
||||||
// Metrics returns middleware that records Prometheus HTTP metrics
|
// Metrics returns middleware that records Prometheus HTTP metrics
|
||||||
// with the Middleware's one recorder, which New builds on the registry
|
// with the Middleware's one recorder, which New builds on the registry
|
||||||
// the /metrics route serves and NewForTest on a registry of its own.
|
// it is given: in the application, the one the /metrics route serves.
|
||||||
// Every call reuses that recorder, so any number of routers can
|
// Every call reuses that recorder, so any number of routers can
|
||||||
// install it.
|
// install it.
|
||||||
func (s *Middleware) Metrics() func(http.Handler) http.Handler {
|
func (s *Middleware) Metrics() func(http.Handler) http.Handler {
|
||||||
|
|||||||
@@ -16,6 +16,7 @@ import (
|
|||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
"sneak.berlin/go/webhooker/internal/middleware"
|
"sneak.berlin/go/webhooker/internal/middleware"
|
||||||
|
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -70,8 +71,8 @@ func metricsTestRouter(
|
|||||||
Environment: "prod",
|
Environment: "prod",
|
||||||
ReceiverRateLimit: receiverLimit,
|
ReceiverRateLimit: receiverLimit,
|
||||||
}
|
}
|
||||||
m := middleware.NewForTest(
|
m := middlewaretest.New(
|
||||||
log, cfg, newTestSessionManager(cfg, log, nil),
|
t, log, cfg, newTestSessionManager(t, cfg),
|
||||||
)
|
)
|
||||||
|
|
||||||
reg := prometheus.NewRegistry()
|
reg := prometheus.NewRegistry()
|
||||||
@@ -455,11 +456,11 @@ func TestMetrics_StatusAndSizeStillRecorded(t *testing.T) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestMetrics_WorksOnNewForTestMiddleware pins that a Middleware built
|
// TestMetrics_WorksOnMiddlewaretestNew pins that a Middleware built
|
||||||
// by NewForTest has a recorder of its own: its Metrics() serves a
|
// by middlewaretest.New has a recorder of its own: its Metrics()
|
||||||
// request instead of panicking, and a second one does not collide
|
// serves a request instead of panicking, and a second one does not
|
||||||
// with the first.
|
// collide with the first.
|
||||||
func TestMetrics_WorksOnNewForTestMiddleware(t *testing.T) {
|
func TestMetrics_WorksOnMiddlewaretestNew(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
log := slog.New(slog.DiscardHandler)
|
log := slog.New(slog.DiscardHandler)
|
||||||
@@ -469,7 +470,7 @@ func TestMetrics_WorksOnNewForTestMiddleware(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
for range 2 {
|
for range 2 {
|
||||||
h := middleware.NewForTest(log, cfg, nil).Metrics()(ok)
|
h := middlewaretest.New(t, log, cfg, nil).Metrics()(ok)
|
||||||
|
|
||||||
req := httptest.NewRequestWithContext(
|
req := httptest.NewRequestWithContext(
|
||||||
t.Context(), http.MethodGet, okRoute, nil,
|
t.Context(), http.MethodGet, okRoute, nil,
|
||||||
|
|||||||
@@ -22,7 +22,6 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
"sneak.berlin/go/webhooker/internal/globals"
|
"sneak.berlin/go/webhooker/internal/globals"
|
||||||
"sneak.berlin/go/webhooker/internal/logfield"
|
"sneak.berlin/go/webhooker/internal/logfield"
|
||||||
"sneak.berlin/go/webhooker/internal/logger"
|
|
||||||
"sneak.berlin/go/webhooker/internal/session"
|
"sneak.berlin/go/webhooker/internal/session"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -155,7 +154,7 @@ const (
|
|||||||
type MiddlewareParams struct {
|
type MiddlewareParams struct {
|
||||||
fx.In
|
fx.In
|
||||||
|
|
||||||
Logger *logger.Logger
|
Logger *slog.Logger
|
||||||
Globals *globals.Globals
|
Globals *globals.Globals
|
||||||
Config *config.Config
|
Config *config.Config
|
||||||
Session *session.Session
|
Session *session.Session
|
||||||
@@ -169,12 +168,10 @@ type Middleware struct {
|
|||||||
params *MiddlewareParams
|
params *MiddlewareParams
|
||||||
session *session.Session
|
session *session.Session
|
||||||
|
|
||||||
// metricsRecorder records the inbound HTTP metrics. New builds
|
// metricsRecorder records the inbound HTTP metrics on
|
||||||
// it on the registry /metrics serves, NewForTest on a registry
|
// params.Registry. It is built once per Middleware and Metrics
|
||||||
// of its own. Either way it is built once per Middleware and
|
// reuses it, because building it registers its collectors, and a
|
||||||
// Metrics reuses it, because building it registers its
|
// second registration on the same registry panics.
|
||||||
// collectors, and a second registration on the same registry
|
|
||||||
// panics.
|
|
||||||
metricsRecorder httpmetrics.Recorder
|
metricsRecorder httpmetrics.Recorder
|
||||||
|
|
||||||
// loginGuard counts failed credential verifications and bounds
|
// loginGuard counts failed credential verifications and bounds
|
||||||
@@ -193,7 +190,7 @@ func New(
|
|||||||
) (*Middleware, error) {
|
) (*Middleware, error) {
|
||||||
s := new(Middleware)
|
s := new(Middleware)
|
||||||
s.params = ¶ms
|
s.params = ¶ms
|
||||||
s.log = params.Logger.Get()
|
s.log = params.Logger
|
||||||
s.session = params.Session
|
s.session = params.Session
|
||||||
s.metricsRecorder = prommetrics.NewRecorder(
|
s.metricsRecorder = prommetrics.NewRecorder(
|
||||||
prommetrics.Config{Registry: params.Registry},
|
prommetrics.Config{Registry: params.Registry},
|
||||||
|
|||||||
@@ -14,36 +14,34 @@ import (
|
|||||||
|
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
|
"go.uber.org/fx/fxtest"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
"sneak.berlin/go/webhooker/internal/middleware"
|
"sneak.berlin/go/webhooker/internal/middleware"
|
||||||
|
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
|
||||||
"sneak.berlin/go/webhooker/internal/session"
|
"sneak.berlin/go/webhooker/internal/session"
|
||||||
)
|
)
|
||||||
|
|
||||||
const testKeySize = 32
|
const testKeySize = 32
|
||||||
|
|
||||||
// testMiddleware creates a Middleware with minimal dependencies
|
// testMiddleware creates a Middleware with minimal dependencies
|
||||||
// for testing. It uses a real session.Session backed by an
|
// for testing. It uses a real session.Session.
|
||||||
// in-memory cookie store.
|
|
||||||
func testMiddleware(
|
func testMiddleware(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
env string,
|
env string,
|
||||||
) (*middleware.Middleware, *session.Session) {
|
) (*middleware.Middleware, *session.Session) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
m, s, _ := testMiddlewareWithSessionClock(t, env, 0, nil)
|
return testMiddlewareWithIdleTimeout(t, env, 0)
|
||||||
|
|
||||||
return m, s
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// testMiddlewareWithSessionClock is testMiddleware with a
|
// testMiddlewareWithIdleTimeout is testMiddleware with a
|
||||||
// configurable session idle timeout and a manually advanced clock,
|
// configurable session idle timeout, for the session-expiry tests.
|
||||||
// for the session-expiry tests. A nil clock uses the real one.
|
func testMiddlewareWithIdleTimeout(
|
||||||
func testMiddlewareWithSessionClock(
|
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
env string,
|
env string,
|
||||||
idleTimeout time.Duration,
|
idleTimeout time.Duration,
|
||||||
clock *fakeClock,
|
) (*middleware.Middleware, *session.Session) {
|
||||||
) (*middleware.Middleware, *session.Session, *fakeClock) {
|
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
log := slog.New(slog.NewTextHandler(
|
log := slog.New(slog.NewTextHandler(
|
||||||
@@ -56,59 +54,44 @@ func testMiddlewareWithSessionClock(
|
|||||||
SessionIdleTimeout: idleTimeout,
|
SessionIdleTimeout: idleTimeout,
|
||||||
}
|
}
|
||||||
|
|
||||||
sessManager := newTestSessionManager(cfg, log, clock)
|
sessManager := newTestSessionManager(t, cfg)
|
||||||
|
|
||||||
m := middleware.NewForTest(log, cfg, sessManager)
|
m := middlewaretest.New(t, log, cfg, sessManager)
|
||||||
|
|
||||||
return m, sessManager, clock
|
return m, sessManager
|
||||||
}
|
}
|
||||||
|
|
||||||
// newTestSessionManager builds the real session.Session the
|
// newTestSessionManager builds the real session.Session the
|
||||||
// middleware tests run against: an in-memory cookie store with a
|
// middleware tests run against, through session.New, with its key
|
||||||
// known key, and optionally a manually advanced clock.
|
// in a main database of its own.
|
||||||
func newTestSessionManager(
|
func newTestSessionManager(
|
||||||
|
t *testing.T,
|
||||||
cfg *config.Config,
|
cfg *config.Config,
|
||||||
log *slog.Logger,
|
|
||||||
clock *fakeClock,
|
|
||||||
) *session.Session {
|
) *session.Session {
|
||||||
key := make([]byte, testKeySize)
|
t.Helper()
|
||||||
|
|
||||||
for i := range key {
|
discard := slog.New(slog.DiscardHandler)
|
||||||
key[i] = byte(i)
|
|
||||||
}
|
|
||||||
|
|
||||||
store := session.NewStore(key)
|
db, err := database.Open(t.TempDir(), discard)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
var now func() time.Time
|
t.Cleanup(func() { _ = db.Close() })
|
||||||
|
|
||||||
if clock != nil {
|
lc := fxtest.NewLifecycle(t)
|
||||||
now = clock.Now
|
|
||||||
}
|
|
||||||
|
|
||||||
return session.NewForTest(store, cfg, log, key, now)
|
sessManager, err := session.New(lc, session.Params{
|
||||||
}
|
Config: cfg,
|
||||||
|
Database: db,
|
||||||
|
Logger: discard,
|
||||||
|
})
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
// fakeClock is a manually advanced clock, so session expiry can be
|
// The start hook reads the key from db and builds the cookie
|
||||||
// tested without sleeping.
|
// store.
|
||||||
type fakeClock struct {
|
lc.RequireStart()
|
||||||
t time.Time
|
t.Cleanup(lc.RequireStop)
|
||||||
}
|
|
||||||
|
|
||||||
func (c *fakeClock) Now() time.Time {
|
return sessManager
|
||||||
return c.t
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *fakeClock) Advance(d time.Duration) {
|
|
||||||
c.t = c.t.Add(d)
|
|
||||||
}
|
|
||||||
|
|
||||||
// newFakeClock returns a clock started at a fixed instant.
|
|
||||||
func newFakeClock() *fakeClock {
|
|
||||||
return &fakeClock{
|
|
||||||
t: time.Date(
|
|
||||||
2026, time.January, 2, 3, 4, 5, 0, time.UTC,
|
|
||||||
),
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- Logging Middleware Tests ---
|
// --- Logging Middleware Tests ---
|
||||||
@@ -583,6 +566,40 @@ func sessionCookies(
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// aged re-issues the session cookie in cookies with both of its
|
||||||
|
// timestamps moved back by d: the cookie as it stands once d has
|
||||||
|
// passed, so session expiry can be tested without sleeping.
|
||||||
|
func aged(
|
||||||
|
t *testing.T,
|
||||||
|
sessManager *session.Session,
|
||||||
|
cookies []*http.Cookie,
|
||||||
|
d time.Duration,
|
||||||
|
) []*http.Cookie {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
req := httptest.NewRequestWithContext(
|
||||||
|
context.Background(), http.MethodGet, "/", nil)
|
||||||
|
|
||||||
|
for _, c := range cookies {
|
||||||
|
req.AddCookie(c)
|
||||||
|
}
|
||||||
|
|
||||||
|
sess, err := sessManager.Get(req)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
for _, key := range []string{session.CreatedAtKey, session.LastSeenKey} {
|
||||||
|
at, ok := sess.Values[key].(int64)
|
||||||
|
require.True(t, ok, "the session has no %s", key)
|
||||||
|
|
||||||
|
sess.Values[key] = at - int64(d/time.Second)
|
||||||
|
}
|
||||||
|
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
require.NoError(t, sessManager.Save(req, w, sess))
|
||||||
|
|
||||||
|
return sessionCookies(w)
|
||||||
|
}
|
||||||
|
|
||||||
func TestRequireAuth_IdleExpiredSession_RedirectsToLogin(
|
func TestRequireAuth_IdleExpiredSession_RedirectsToLogin(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
) {
|
) {
|
||||||
@@ -590,13 +607,11 @@ func TestRequireAuth_IdleExpiredSession_RedirectsToLogin(
|
|||||||
|
|
||||||
idle := time.Hour
|
idle := time.Hour
|
||||||
|
|
||||||
m, sessManager, clock := testMiddlewareWithSessionClock(
|
m, sessManager := testMiddlewareWithIdleTimeout(
|
||||||
t, config.EnvironmentDev, idle, newFakeClock(),
|
t, config.EnvironmentDev, idle,
|
||||||
)
|
)
|
||||||
|
|
||||||
cookies := loginCookies(t, sessManager)
|
cookies := aged(t, sessManager, loginCookies(t, sessManager), idle)
|
||||||
|
|
||||||
clock.Advance(idle)
|
|
||||||
|
|
||||||
called, w := runAuthed(t, m, cookies)
|
called, w := runAuthed(t, m, cookies)
|
||||||
|
|
||||||
@@ -621,14 +636,12 @@ func TestRequireAuth_RefreshesIdleDeadlineOnActivity(
|
|||||||
|
|
||||||
idle := time.Hour
|
idle := time.Hour
|
||||||
|
|
||||||
m, sessManager, clock := testMiddlewareWithSessionClock(
|
m, sessManager := testMiddlewareWithIdleTimeout(
|
||||||
t, config.EnvironmentDev, idle, newFakeClock(),
|
t, config.EnvironmentDev, idle,
|
||||||
)
|
)
|
||||||
|
|
||||||
cookies := loginCookies(t, sessManager)
|
|
||||||
|
|
||||||
// Activity halfway through the idle window.
|
// Activity halfway through the idle window.
|
||||||
clock.Advance(idle / 2)
|
cookies := aged(t, sessManager, loginCookies(t, sessManager), idle/2)
|
||||||
|
|
||||||
called, w := runAuthed(t, m, cookies)
|
called, w := runAuthed(t, m, cookies)
|
||||||
require.True(t, called, "handler should run while valid")
|
require.True(t, called, "handler should run while valid")
|
||||||
@@ -640,16 +653,22 @@ func TestRequireAuth_RefreshesIdleDeadlineOnActivity(
|
|||||||
)
|
)
|
||||||
|
|
||||||
// Past the original deadline. The refreshed cookie is still
|
// Past the original deadline. The refreshed cookie is still
|
||||||
// good; the original one is not.
|
// good; the original one is not. A minute short of the idle
|
||||||
clock.Advance(idle - time.Second)
|
// window leaves room for the real clock, which the session
|
||||||
|
// reads, to tick on while the test runs.
|
||||||
|
later := idle - time.Minute
|
||||||
|
|
||||||
calledRefreshed, _ := runAuthed(t, m, refreshed)
|
calledRefreshed, _ := runAuthed(
|
||||||
|
t, m, aged(t, sessManager, refreshed, later),
|
||||||
|
)
|
||||||
assert.True(
|
assert.True(
|
||||||
t, calledRefreshed,
|
t, calledRefreshed,
|
||||||
"refreshed session should outlive the original deadline",
|
"refreshed session should outlive the original deadline",
|
||||||
)
|
)
|
||||||
|
|
||||||
calledStale, staleW := runAuthed(t, m, cookies)
|
calledStale, staleW := runAuthed(
|
||||||
|
t, m, aged(t, sessManager, cookies, later),
|
||||||
|
)
|
||||||
assert.False(
|
assert.False(
|
||||||
t, calledStale,
|
t, calledStale,
|
||||||
"the pre-refresh cookie carries the old idle deadline",
|
"the pre-refresh cookie carries the old idle deadline",
|
||||||
@@ -662,8 +681,8 @@ func TestRequireAuth_UnauthenticatedRequestDoesNotRefresh(
|
|||||||
) {
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
m, sessManager, _ := testMiddlewareWithSessionClock(
|
m, sessManager := testMiddlewareWithIdleTimeout(
|
||||||
t, config.EnvironmentDev, time.Hour, newFakeClock(),
|
t, config.EnvironmentDev, time.Hour,
|
||||||
)
|
)
|
||||||
|
|
||||||
// A session cookie that exists but was never authenticated.
|
// A session cookie that exists but was never authenticated.
|
||||||
@@ -924,12 +943,9 @@ func metricsAuthMiddleware(
|
|||||||
MetricsPassword: "secret",
|
MetricsPassword: "secret",
|
||||||
}
|
}
|
||||||
|
|
||||||
key := make([]byte, testKeySize)
|
return middlewaretest.New(
|
||||||
store := session.NewStore(key)
|
t, log, cfg, newTestSessionManager(t, cfg),
|
||||||
|
)
|
||||||
sessManager := session.NewForTest(store, cfg, log, key, nil)
|
|
||||||
|
|
||||||
return middleware.NewForTest(log, cfg, sessManager)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// runMetricsAuthRequest sends a GET /metrics request with the
|
// runMetricsAuthRequest sends a GET /metrics request with the
|
||||||
|
|||||||
@@ -0,0 +1,42 @@
|
|||||||
|
// Package middlewaretest builds a Middleware for tests in other
|
||||||
|
// packages.
|
||||||
|
package middlewaretest
|
||||||
|
|
||||||
|
import (
|
||||||
|
"log/slog"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/prometheus/client_golang/prometheus"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"go.uber.org/fx/fxtest"
|
||||||
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
|
"sneak.berlin/go/webhooker/internal/middleware"
|
||||||
|
"sneak.berlin/go/webhooker/internal/session"
|
||||||
|
)
|
||||||
|
|
||||||
|
// New builds a Middleware through middleware.New, on a
|
||||||
|
// lifecycle that is never started.
|
||||||
|
//
|
||||||
|
// Its metrics recorder writes to a fresh registry of its own, so
|
||||||
|
// Metrics() works on it and two of them never collide.
|
||||||
|
func New(
|
||||||
|
t *testing.T,
|
||||||
|
log *slog.Logger,
|
||||||
|
cfg *config.Config,
|
||||||
|
sess *session.Session,
|
||||||
|
) *middleware.Middleware {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
m, err := middleware.New(
|
||||||
|
fxtest.NewLifecycle(t),
|
||||||
|
middleware.MiddlewareParams{
|
||||||
|
Logger: log,
|
||||||
|
Config: cfg,
|
||||||
|
Session: sess,
|
||||||
|
Registry: prometheus.NewRegistry(),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
return m
|
||||||
|
}
|
||||||
@@ -18,6 +18,7 @@ import (
|
|||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
"sneak.berlin/go/webhooker/internal/middleware"
|
"sneak.berlin/go/webhooker/internal/middleware"
|
||||||
|
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestPostRateLimit_AllowsGET(t *testing.T) {
|
func TestPostRateLimit_AllowsGET(t *testing.T) {
|
||||||
@@ -198,7 +199,7 @@ func rateLimitMiddleware(
|
|||||||
&slog.HandlerOptions{Level: slog.LevelDebug},
|
&slog.HandlerOptions{Level: slog.LevelDebug},
|
||||||
))
|
))
|
||||||
|
|
||||||
return middleware.NewForTest(log, cfg, nil)
|
return middlewaretest.New(t, log, cfg, nil)
|
||||||
}
|
}
|
||||||
|
|
||||||
// trustedProxies parses CIDR strings for a test Config.
|
// trustedProxies parses CIDR strings for a test Config.
|
||||||
|
|||||||
@@ -1,32 +0,0 @@
|
|||||||
package middleware
|
|
||||||
|
|
||||||
import (
|
|
||||||
"log/slog"
|
|
||||||
|
|
||||||
"github.com/prometheus/client_golang/prometheus"
|
|
||||||
prommetrics "github.com/slok/go-http-metrics/metrics/prometheus"
|
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
|
||||||
"sneak.berlin/go/webhooker/internal/session"
|
|
||||||
)
|
|
||||||
|
|
||||||
// NewForTest creates a Middleware with the minimum dependencies
|
|
||||||
// needed for testing. This bypasses the fx lifecycle.
|
|
||||||
//
|
|
||||||
// Its metrics recorder writes to a fresh registry of its own, so
|
|
||||||
// Metrics() works on it and two of them never collide.
|
|
||||||
func NewForTest(
|
|
||||||
log *slog.Logger,
|
|
||||||
cfg *config.Config,
|
|
||||||
sess *session.Session,
|
|
||||||
) *Middleware {
|
|
||||||
return &Middleware{
|
|
||||||
log: log,
|
|
||||||
params: &MiddlewareParams{
|
|
||||||
Config: cfg,
|
|
||||||
},
|
|
||||||
session: sess,
|
|
||||||
metricsRecorder: prommetrics.NewRecorder(
|
|
||||||
prommetrics.Config{Registry: prometheus.NewRegistry()},
|
|
||||||
),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -174,6 +174,7 @@ func newServerApp(
|
|||||||
fx.Provide(
|
fx.Provide(
|
||||||
globals.New,
|
globals.New,
|
||||||
logger.New,
|
logger.New,
|
||||||
|
func(l *logger.Logger) *slog.Logger { return l.Get() },
|
||||||
func() *config.Config {
|
func() *config.Config {
|
||||||
return &config.Config{DataDir: dir}
|
return &config.Config{DataDir: dir}
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -280,6 +280,23 @@ func click(ctx context.Context, t *testing.T, xpath string) {
|
|||||||
))
|
))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// clickAndLoad clicks the link or button matching an XPath expression
|
||||||
|
// and waits, as loadPage does, for the page the click opens to load and
|
||||||
|
// for Alpine.js to start on it. Reading earlier, a check can find an
|
||||||
|
// element of the page being left, gone by the time its value is read;
|
||||||
|
// and the wait in shown is too short for a page load on a busy host.
|
||||||
|
func clickAndLoad(ctx context.Context, t *testing.T, xpath string) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
_, err := chromedp.RunResponse(
|
||||||
|
ctx, chromedp.Click(xpath, chromedp.BySearch),
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.NoError(t, chromedp.Run(
|
||||||
|
ctx, chromedp.WaitNotPresent("[x-cloak]", chromedp.ByQuery),
|
||||||
|
))
|
||||||
|
}
|
||||||
|
|
||||||
// checkAddEntrypoint loads a webhook page and checks that the add
|
// checkAddEntrypoint loads a webhook page and checks that the add
|
||||||
// entrypoint form stays hidden until the Add button beside its heading
|
// entrypoint form stays hidden until the Add button beside its heading
|
||||||
// is clicked. The click looks for a button element there, so it also
|
// is clicked. The click looks for a button element there, so it also
|
||||||
@@ -422,7 +439,7 @@ func checkAddTarget(
|
|||||||
)))
|
)))
|
||||||
}
|
}
|
||||||
|
|
||||||
click(ctx, t, saveButton)
|
clickAndLoad(ctx, t, saveButton)
|
||||||
assert.Truef(t, shown(ctx, `//span[text()="`+name+
|
assert.Truef(t, shown(ctx, `//span[text()="`+name+
|
||||||
`"]/following-sibling::div/span[text()="`+badge+`"]`),
|
`"]/following-sibling::div/span[text()="`+badge+`"]`),
|
||||||
"%s: the added target is not listed as %s", targetType, badge)
|
"%s: the added target is not listed as %s", targetType, badge)
|
||||||
@@ -485,10 +502,9 @@ func checkArchiveChoices(ctx context.Context, t *testing.T, url string) {
|
|||||||
`/following-sibling::span[text()="daily"]`),
|
`/following-sibling::span[text()="daily"]`),
|
||||||
"a database target added with daily is not listed as daily")
|
"a database target added with daily is not listed as daily")
|
||||||
|
|
||||||
click(ctx, t, row+`//a[text()="Edit"]`)
|
clickAndLoad(ctx, t, row+`//a[text()="Edit"]`)
|
||||||
require.NoError(t, chromedp.Run(
|
require.NoError(t, chromedp.Run(
|
||||||
ctx,
|
ctx,
|
||||||
chromedp.WaitReady("#expiry", chromedp.ByQuery),
|
|
||||||
chromedp.Value("#expiry", &editedExpiry, chromedp.ByQuery),
|
chromedp.Value("#expiry", &editedExpiry, chromedp.ByQuery),
|
||||||
chromedp.Value("#rotation", &editedRotation, chromedp.ByQuery),
|
chromedp.Value("#rotation", &editedRotation, chromedp.ByQuery),
|
||||||
))
|
))
|
||||||
@@ -523,7 +539,7 @@ func checkRefusedTarget(ctx context.Context, t *testing.T, url string) {
|
|||||||
chromedp.Click(forwardQuery, chromedp.ByQuery),
|
chromedp.Click(forwardQuery, chromedp.ByQuery),
|
||||||
))
|
))
|
||||||
|
|
||||||
click(ctx, t, saveButton)
|
clickAndLoad(ctx, t, saveButton)
|
||||||
|
|
||||||
assert.True(t, shown(ctx, reason),
|
assert.True(t, shown(ctx, reason),
|
||||||
"a refused target does not show the reason")
|
"a refused target does not show the reason")
|
||||||
@@ -636,7 +652,7 @@ func checkRefusedEdits(
|
|||||||
))
|
))
|
||||||
}
|
}
|
||||||
|
|
||||||
click(ctx, t, `//button[text()="Save Changes"]`)
|
clickAndLoad(ctx, t, `//button[text()="Save Changes"]`)
|
||||||
|
|
||||||
assert.Truef(t, shown(ctx, reason),
|
assert.Truef(t, shown(ctx, reason),
|
||||||
"%s: a refused save does not show the reason", edit.url)
|
"%s: a refused save does not show the reason", edit.url)
|
||||||
@@ -760,7 +776,7 @@ func checkEntrypointEdit(
|
|||||||
require.NoError(t, chromedp.Run(
|
require.NoError(t, chromedp.Run(
|
||||||
ctx, chromedp.SendKeys(input, "Billing sender", chromedp.ByQuery),
|
ctx, chromedp.SendKeys(input, "Billing sender", chromedp.ByQuery),
|
||||||
))
|
))
|
||||||
click(ctx, t, saveEdit)
|
clickAndLoad(ctx, t, saveEdit)
|
||||||
|
|
||||||
assert.True(t, shown(ctx, `//span[text()="Billing sender"]`),
|
assert.True(t, shown(ctx, `//span[text()="Billing sender"]`),
|
||||||
"saving the edit form does not change the description")
|
"saving the edit form does not change the description")
|
||||||
@@ -798,7 +814,7 @@ func checkRecentEvents(ctx context.Context, t *testing.T, url string) {
|
|||||||
"clicking the newest event does not collapse it")
|
"clicking the newest event does not collapse it")
|
||||||
|
|
||||||
require.NoError(t, chromedp.Run(ctx, loadPage(url)))
|
require.NoError(t, chromedp.Run(ctx, loadPage(url)))
|
||||||
click(ctx, t, newest+`/ancestor::div[@x-data][1]//a[text()="Open"]`)
|
clickAndLoad(ctx, t, newest+`/ancestor::div[@x-data][1]//a[text()="Open"]`)
|
||||||
|
|
||||||
assert.True(t, shown(ctx, `//h2[text()="Body"]`),
|
assert.True(t, shown(ctx, `//h2[text()="Body"]`),
|
||||||
"Open does not lead to the event's own page")
|
"Open does not lead to the event's own page")
|
||||||
@@ -1199,7 +1215,7 @@ func checkNewWebhookTargets(
|
|||||||
`","rotation":"none"}`
|
`","rotation":"none"}`
|
||||||
}
|
}
|
||||||
|
|
||||||
click(ctx, t, createButton)
|
clickAndLoad(ctx, t, createButton)
|
||||||
require.Truef(t, shown(ctx, `//h1[text()="`+name+`"]`),
|
require.Truef(t, shown(ctx, `//h1[text()="`+name+`"]`),
|
||||||
"%s: the new webhook's page does not open", name)
|
"%s: the new webhook's page does not open", name)
|
||||||
|
|
||||||
@@ -1260,7 +1276,7 @@ func checkRefusedNewWebhook(ctx context.Context, t *testing.T, url string) {
|
|||||||
chromedp.SetValue(pruningChoice, "2160h", chromedp.BySearch),
|
chromedp.SetValue(pruningChoice, "2160h", chromedp.BySearch),
|
||||||
chromedp.SetValue("#archive_rotation", "monthly", chromedp.ByQuery),
|
chromedp.SetValue("#archive_rotation", "monthly", chromedp.ByQuery),
|
||||||
))
|
))
|
||||||
click(ctx, t, createButton)
|
clickAndLoad(ctx, t, createButton)
|
||||||
|
|
||||||
assert.True(t, shown(ctx, `//div[@class="alert-error"]`),
|
assert.True(t, shown(ctx, `//div[@class="alert-error"]`),
|
||||||
"a refused webhook does not show the reason")
|
"a refused webhook does not show the reason")
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ package server_test
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"html"
|
"html"
|
||||||
|
"log/slog"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"net/url"
|
"net/url"
|
||||||
@@ -130,6 +131,7 @@ func newTestEnvWithConfig(
|
|||||||
fx.Provide(
|
fx.Provide(
|
||||||
globals.New,
|
globals.New,
|
||||||
logger.New,
|
logger.New,
|
||||||
|
func(l *logger.Logger) *slog.Logger { return l.Get() },
|
||||||
func() *config.Config { return cfg },
|
func() *config.Config { return cfg },
|
||||||
database.New,
|
database.New,
|
||||||
database.NewWebhookDBManager,
|
database.NewWebhookDBManager,
|
||||||
|
|||||||
@@ -16,8 +16,7 @@ func NewStore(key []byte) *sessions.CookieStore {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// NewForTest creates a Session with a pre-configured cookie store for use
|
// NewForTest creates a Session with a pre-configured cookie store for use
|
||||||
// in tests. This bypasses the fx lifecycle and database dependency, allowing
|
// in tests. This bypasses the fx lifecycle and database dependency. The key
|
||||||
// middleware and handler tests to use real session functionality. The key
|
|
||||||
// parameter is the raw 32-byte authentication key used for session encryption
|
// parameter is the raw 32-byte authentication key used for session encryption
|
||||||
// and CSRF cookie signing.
|
// and CSRF cookie signing.
|
||||||
//
|
//
|
||||||
@@ -16,7 +16,6 @@ import (
|
|||||||
"go.uber.org/fx"
|
"go.uber.org/fx"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
"sneak.berlin/go/webhooker/internal/logger"
|
|
||||||
"sneak.berlin/go/webhooker/internal/reqtls"
|
"sneak.berlin/go/webhooker/internal/reqtls"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -80,7 +79,7 @@ type Params struct {
|
|||||||
|
|
||||||
Config *config.Config
|
Config *config.Config
|
||||||
Database *database.Database
|
Database *database.Database
|
||||||
Logger *logger.Logger
|
Logger *slog.Logger
|
||||||
}
|
}
|
||||||
|
|
||||||
// Session manages encrypted session storage.
|
// Session manages encrypted session storage.
|
||||||
@@ -180,7 +179,7 @@ func New(
|
|||||||
params Params,
|
params Params,
|
||||||
) (*Session, error) {
|
) (*Session, error) {
|
||||||
s := &Session{
|
s := &Session{
|
||||||
log: params.Logger.Get(),
|
log: params.Logger,
|
||||||
idleTimeout: params.Config.SessionIdleTimeout,
|
idleTimeout: params.Config.SessionIdleTimeout,
|
||||||
now: time.Now,
|
now: time.Now,
|
||||||
}
|
}
|
||||||
|
|||||||
+2
-2
@@ -2,8 +2,8 @@
|
|||||||
# script/assets: extract Alpine.js from its npm package tarball, committed
|
# script/assets: extract Alpine.js from its npm package tarball, committed
|
||||||
# in 3p/, to static/js/alpine.min.js, where go:embed reads it. The package
|
# in 3p/, to static/js/alpine.min.js, where go:embed reads it. The package
|
||||||
# is @alpinejs/csp, Alpine's build for pages whose Content-Security-Policy
|
# is @alpinejs/csp, Alpine's build for pages whose Content-Security-Policy
|
||||||
# forbids eval. The extracted file is not committed. script/test, make
|
# forbids eval. The extracted file is not committed. make build, make dev
|
||||||
# build and make dev run this first.
|
# and the Dockerfile's lint, test and build stages run this first.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|||||||
+12
-4
@@ -11,6 +11,7 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
|||||||
|
|
||||||
PKGMGR=""
|
PKGMGR=""
|
||||||
SUDO=""
|
SUDO=""
|
||||||
|
APT_UPDATED=""
|
||||||
|
|
||||||
detect_pkgmgr() {
|
detect_pkgmgr() {
|
||||||
[ -n "$PKGMGR" ] && return 0
|
[ -n "$PKGMGR" ] && return 0
|
||||||
@@ -39,7 +40,14 @@ pkg_install() {
|
|||||||
detect_pkgmgr
|
detect_pkgmgr
|
||||||
case "$PKGMGR" in
|
case "$PKGMGR" in
|
||||||
nix) nix-env -iA "nixpkgs.$1" ;;
|
nix) nix-env -iA "nixpkgs.$1" ;;
|
||||||
apt) $SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2" ;;
|
apt)
|
||||||
|
# Package lists may be empty (fresh images); refresh once per run.
|
||||||
|
if [ -z "$APT_UPDATED" ]; then
|
||||||
|
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get update
|
||||||
|
APT_UPDATED=1
|
||||||
|
fi
|
||||||
|
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2"
|
||||||
|
;;
|
||||||
brew) brew install "$3" ;;
|
brew) brew install "$3" ;;
|
||||||
apk) apk add --no-cache "$4" ;;
|
apk) apk add --no-cache "$4" ;;
|
||||||
esac
|
esac
|
||||||
@@ -60,10 +68,10 @@ main() {
|
|||||||
if missing go; then pkg_install go golang go go; fi
|
if missing go; then pkg_install go golang go go; fi
|
||||||
|
|
||||||
# Not installed here: docker is platform-specific and out of scope for a
|
# Not installed here: docker is platform-specific and out of scope for a
|
||||||
# package-manager bootstrap, but script/lint, script/fmt and script/css
|
# package-manager bootstrap, but script/test, script/lint, script/fmt and
|
||||||
# need it.
|
# script/css need it.
|
||||||
if missing docker; then
|
if missing docker; then
|
||||||
echo "bootstrap: docker not found; script/lint, script/fmt and script/css require it" >&2
|
echo "bootstrap: docker not found; script/test, script/lint, script/fmt and script/css require it" >&2
|
||||||
fi
|
fi
|
||||||
|
|
||||||
go mod download
|
go mod download
|
||||||
|
|||||||
+2
-2
@@ -1,7 +1,7 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/check: run all checks (test, lint, fmt-check, css-check). Our own
|
# script/check: run all checks (test, lint, fmt-check, css-check). Our own
|
||||||
# extension to scripts-to-rule-them-all.
|
# extension to scripts-to-rule-them-all. test, lint and css-check are
|
||||||
# Writes only the ignored static/js/alpine.min.js, through script/test.
|
# Docker builds; fmt-check runs gofmt on the host. Writes nothing.
|
||||||
# Generic, apart from css-check.
|
# Generic, apart from css-check.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
|
|||||||
@@ -1,152 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
# script/ci-mark-superseded: record an honest status on commits whose CI
|
|
||||||
# run Gitea cancelled because a newer commit landed on the same branch.
|
|
||||||
# Gitea writes `failure` / "Has been cancelled" for such a run, which
|
|
||||||
# reads as a test result on a commit nothing ever tested. Cancellation is
|
|
||||||
# unconditional server-side for push events, so the superseding run
|
|
||||||
# rewrites those statuses to `failure` with a description that says the
|
|
||||||
# commit was never tested. `skipped` cannot be used: Gitea's combined
|
|
||||||
# status folds `skipped` into `success`, so a never-tested commit would
|
|
||||||
# report green. Genuine failures and successes are never touched.
|
|
||||||
#
|
|
||||||
# Called by the Gitea Actions workflow, which supplies GITHUB_API_URL,
|
|
||||||
# GITHUB_REPOSITORY, GITHUB_SHA, GITHUB_WORKFLOW, GITHUB_JOB,
|
|
||||||
# GITHUB_EVENT_NAME and GITEA_TOKEN. ANCESTOR_LIMIT (default 20) caps how
|
|
||||||
# far back the walk looks; a value that is set but not a positive integer
|
|
||||||
# aborts rather than silently disabling the walk.
|
|
||||||
set -eu
|
|
||||||
|
|
||||||
SUPERSEDED_DESC='Superseded by a newer commit; never tested'
|
|
||||||
|
|
||||||
# Gitea builds the commit-status context as
|
|
||||||
# "<workflow name> / <job name> (<event>)", so derive it rather than
|
|
||||||
# hardcoding the result.
|
|
||||||
#
|
|
||||||
# The derivation is deliberately not byte-exact with Gitea's own rule and
|
|
||||||
# must not be "fixed" into a silent fallback. Gitea uses the job's `name:`
|
|
||||||
# (falling back to the job id) and the workflow's `name:` (falling back to
|
|
||||||
# the workflow filename), while the runner exports GITHUB_JOB as the job
|
|
||||||
# *id* and GITHUB_WORKFLOW as the parsed workflow `name:`. So giving the
|
|
||||||
# job a display `name:`, or dropping the workflow's `name:`, makes the
|
|
||||||
# derived context stop matching --- and require_own_context below then
|
|
||||||
# turns every push red with a message. That loud failure is the point
|
|
||||||
# (https://git.eeqj.de/sneak/webhooker/issues/147 item 2); guessing at a
|
|
||||||
# fallback would restore the silent no-op it replaced.
|
|
||||||
context() {
|
|
||||||
printf '%s / %s (%s)' \
|
|
||||||
"$GITHUB_WORKFLOW" "$GITHUB_JOB" "$GITHUB_EVENT_NAME"
|
|
||||||
}
|
|
||||||
|
|
||||||
# ANCESTOR_LIMIT is a documented knob, so a value that is set but
|
|
||||||
# unusable must fail loudly instead of defaulting
|
|
||||||
# (https://git.eeqj.de/sneak/webhooker/issues/80). Passing it straight to
|
|
||||||
# git would print `fatal: not an integer` into a discarded exit status
|
|
||||||
# and mark nothing.
|
|
||||||
ancestor_limit() {
|
|
||||||
# `-` and not `:-`: an explicitly empty value is set-but-unusable
|
|
||||||
# config, so it aborts like any other bad value rather than silently
|
|
||||||
# running at the default.
|
|
||||||
_limit="${ANCESTOR_LIMIT-20}"
|
|
||||||
case "$_limit" in
|
|
||||||
'' | *[!0-9]* | 0*)
|
|
||||||
echo "ANCESTOR_LIMIT must be a positive integer," \
|
|
||||||
"got '${_limit}'" >&2
|
|
||||||
|
|
||||||
return 1
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
printf '%s' "$_limit"
|
|
||||||
}
|
|
||||||
|
|
||||||
# The status Gitea created for this very job proves which context string
|
|
||||||
# it uses. If the derived one is missing, the workflow or the job was
|
|
||||||
# renamed and the match below would silently stop firing, restoring the
|
|
||||||
# false-red bug with no signal. Fail loudly instead.
|
|
||||||
require_own_context() {
|
|
||||||
if ! _body="$(curl -sf --retry 3 --retry-delay 2 --max-time 30 \
|
|
||||||
"${1}/commits/${GITHUB_SHA}/status")"; then
|
|
||||||
echo "cannot read commit statuses for ${GITHUB_SHA}" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
_found="$(printf '%s' "$_body" | jq -r '(.statuses // [])[].context')"
|
|
||||||
if printf '%s\n' "$_found" | grep -qxF "$2"; then
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
echo "no commit status with context '${2}' on ${GITHUB_SHA}:" >&2
|
|
||||||
echo "workflow or job renamed? contexts present:" >&2
|
|
||||||
printf '%s\n' "$_found" >&2
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
|
|
||||||
# Latest status for our context on a commit, as "state|description".
|
|
||||||
# The read is retried and bounded, and a read that still fails aborts the
|
|
||||||
# step: a laundered commit that cannot be read is not the same as one
|
|
||||||
# with nothing to do, and piping curl into jq would discard the
|
|
||||||
# difference.
|
|
||||||
status_of() {
|
|
||||||
if ! _sbody="$(curl -sf --retry 3 --retry-delay 2 --max-time 30 \
|
|
||||||
"${1}/commits/${2}/status")"; then
|
|
||||||
echo "cannot read commit statuses for ${2}" >&2
|
|
||||||
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
printf '%s' "$_sbody" | jq -r --arg c "$3" \
|
|
||||||
'[(.statuses // [])[] | select(.context == $c)][0] // empty
|
|
||||||
| "\(.status)|\(.description)"'
|
|
||||||
}
|
|
||||||
|
|
||||||
mark_superseded() {
|
|
||||||
curl -sf -X POST "${1}/statuses/${2}" \
|
|
||||||
-H "Authorization: token ${GITEA_TOKEN}" \
|
|
||||||
-H 'Content-Type: application/json' \
|
|
||||||
-d "$(jq -nc --arg c "$3" --arg d "$SUPERSEDED_DESC" \
|
|
||||||
'{context: $c, state: "failure", description: $d}')" \
|
|
||||||
>/dev/null
|
|
||||||
}
|
|
||||||
|
|
||||||
main() {
|
|
||||||
_api="${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}"
|
|
||||||
_ctx="$(context)"
|
|
||||||
|
|
||||||
_limit="$(ancestor_limit)"
|
|
||||||
|
|
||||||
require_own_context "$_api" "$_ctx"
|
|
||||||
|
|
||||||
# A shallow clone cannot resolve the parent, so it looks exactly like
|
|
||||||
# a root commit to rev-parse below and would exit 0 having walked
|
|
||||||
# nothing (or, at depth > 1, only the ancestors that happen to be
|
|
||||||
# present). The workflow checks out with `fetch-depth: 0`; verify
|
|
||||||
# that here rather than depend on it silently.
|
|
||||||
if [ "$(git rev-parse --is-shallow-repository)" = 'true' ]; then
|
|
||||||
echo "shallow repository: the ancestor walk needs full history" >&2
|
|
||||||
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# A root commit legitimately has no ancestors and is not an error.
|
|
||||||
# A SHA this repository does not have lands here too, since its
|
|
||||||
# parent is equally unresolvable, but require_own_context above has
|
|
||||||
# already aborted on the 404 for it. The walk itself carries no
|
|
||||||
# `|| true`, so a rev-list failure aborts.
|
|
||||||
if ! git rev-parse -q --verify "${GITHUB_SHA}^" >/dev/null; then
|
|
||||||
echo "no ancestor of ${GITHUB_SHA} to check"
|
|
||||||
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
_walk="$(git rev-list --max-count="$_limit" "${GITHUB_SHA}^")"
|
|
||||||
|
|
||||||
for _sha in $_walk; do
|
|
||||||
_latest="$(status_of "$_api" "$_sha" "$_ctx")"
|
|
||||||
# A run that was cancelled, or one an earlier revision of this
|
|
||||||
# script laundered into `skipped`. Anything else stands.
|
|
||||||
case "$_latest" in
|
|
||||||
'failure|Has been cancelled' | "skipped|${SUPERSEDED_DESC}") ;;
|
|
||||||
*) continue ;;
|
|
||||||
esac
|
|
||||||
mark_superseded "$_api" "$_sha" "$_ctx"
|
|
||||||
echo "marked superseded: ${_sha}"
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
main "$@"
|
|
||||||
+19
-6
@@ -1,15 +1,28 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/cibuild: run the CI build. The Dockerfile runs the checks (the
|
# script/cibuild: run the CI build. It bootstraps first: a CI runner
|
||||||
# gofmt check, golangci-lint, the stylesheet check, ESLint, the Markdown
|
# checks out and runs this and nothing else, and script/fmt-check runs
|
||||||
# check, make test), so a successful build implies a green repo. Generic:
|
# the formatter on the host, which a pristine checkout cannot do.
|
||||||
# needs no adaptation. The Gitea workflow runs this on push.
|
# --no-cache for the same reason as script/docker: the gate phases the
|
||||||
|
# final stage depends on are RUN steps, and a cached one is a check that
|
||||||
|
# did not run.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
|
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build .
|
"$SCRIPT_DIR/bootstrap"
|
||||||
|
"$SCRIPT_DIR/check"
|
||||||
|
# Own line: a failing command substitution inside an argument does
|
||||||
|
# not trip `set -e`, so the inline form degrades silently to an
|
||||||
|
# empty constant. The VERSION build argument takes precedence over
|
||||||
|
# the version a build stage derives from the .git in the context.
|
||||||
|
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
||||||
|
[ -n "$version" ] || version="unknown"
|
||||||
|
docker build --no-cache \
|
||||||
|
--build-arg VERSION="$version" \
|
||||||
|
-t "$("$SCRIPT_DIR/projectname")" .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+4
-2
@@ -2,14 +2,16 @@
|
|||||||
# script/css: regenerate static/css/tailwind.css (writes). tailwindcss is
|
# script/css: regenerate static/css/tailwind.css (writes). tailwindcss is
|
||||||
# never installed locally: it runs in docker, at the version and sha256
|
# never installed locally: it runs in docker, at the version and sha256
|
||||||
# pinned in the Dockerfile's stylesheet stages, which also say what the
|
# pinned in the Dockerfile's stylesheet stages, which also say what the
|
||||||
# stylesheet is generated from.
|
# stylesheet is generated from. --no-cache, as on every docker build in
|
||||||
|
# script/, so the stylesheet is generated rather than taken from the cache.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build --target css-output --output type=local,dest=static/css .
|
docker build --no-cache \
|
||||||
|
--target css-output --output type=local,dest=static/css .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+3
-2
@@ -1,14 +1,15 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/css-check: fail when static/css/tailwind.css differs from what
|
# script/css-check: fail when static/css/tailwind.css differs from what
|
||||||
# script/css would generate (read-only). The comparison is the Dockerfile's
|
# script/css would generate (read-only). The comparison is the Dockerfile's
|
||||||
# css-check stage, which the image build runs too.
|
# css-check stage, which the image build runs too. --no-cache because a
|
||||||
|
# cached check is a check that did not run.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build --target css-check --output type=cacheonly .
|
docker build --no-cache --target css-check --output type=cacheonly .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+11
-7
@@ -1,10 +1,8 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/docker: build the Docker image tagged with the project name.
|
# script/docker: build the Docker image tagged with the project name.
|
||||||
# The tag comes from script/projectname.
|
# Identical in all repos; the tag comes from script/projectname.
|
||||||
#
|
# --no-cache because the gate phases the final stage depends on are RUN
|
||||||
# The version script/version resolves here goes in as the VERSION build
|
# steps, and a cached one is a check that did not run.
|
||||||
# arg, which takes precedence over what the build would derive from the
|
|
||||||
# .git in its context.
|
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
@@ -12,8 +10,14 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
|||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build \
|
# Own line: a failing command substitution inside an argument does
|
||||||
--build-arg VERSION="$("$SCRIPT_DIR/version")" \
|
# not trip `set -e`, so the inline form degrades silently to an
|
||||||
|
# empty constant. The VERSION build argument takes precedence over
|
||||||
|
# the version a build stage derives from the .git in the context.
|
||||||
|
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
||||||
|
[ -n "$version" ] || version="unknown"
|
||||||
|
docker build --no-cache \
|
||||||
|
--build-arg VERSION="$version" \
|
||||||
-t "$("$SCRIPT_DIR/projectname")" .
|
-t "$("$SCRIPT_DIR/projectname")" .
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+4
-2
@@ -1,7 +1,8 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/fmt: format all files (writes): the Go code with gofmt and
|
# script/fmt: format all files (writes): the Go code with gofmt and
|
||||||
# goimports, the Markdown with prettier. prettier is never installed
|
# goimports, the Markdown with prettier. prettier is never installed
|
||||||
# locally: it runs in docker, in the Dockerfile's Markdown stages.
|
# locally: it runs in docker, in the Dockerfile's Markdown stages, built
|
||||||
|
# with --no-cache like every docker build in script/.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
@@ -12,7 +13,8 @@ main() {
|
|||||||
if command -v goimports >/dev/null 2>&1; then
|
if command -v goimports >/dev/null 2>&1; then
|
||||||
goimports -w .
|
goimports -w .
|
||||||
fi
|
fi
|
||||||
docker build --target markdown-output --output type=local,dest=. .
|
docker build --no-cache \
|
||||||
|
--target markdown-output --output type=local,dest=. .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+3
-2
@@ -1,6 +1,7 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/fmt-check: check formatting (read-only). Same scope as
|
# script/fmt-check: check formatting (read-only). Same scope as
|
||||||
# script/fmt, but fails instead of writing.
|
# script/fmt, but fails instead of writing. --no-cache because a cached
|
||||||
|
# check is a check that did not run.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
@@ -12,7 +13,7 @@ main() {
|
|||||||
gofmt -s -l .
|
gofmt -s -l .
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
docker build --target markdown-check --output type=cacheonly .
|
docker build --no-cache --target markdown-check --output type=cacheonly .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+13
-61
@@ -1,71 +1,23 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/lint: run the linters, golangci-lint over the Go code and then
|
# script/lint: run the linter. Linting is a phase of the Dockerfile and
|
||||||
# ESLint over static/js/. Neither is ever installed locally.
|
# this builds that phase alone; the linter is never installed or run on
|
||||||
|
# a developer host, where a shared result cache and a host-global lock
|
||||||
|
# make its answer untrustworthy.
|
||||||
#
|
#
|
||||||
# golangci-lint runs via docker only, one way, everywhere — script/lint builds
|
# The phase is not the last stage in the file, so it is built only when
|
||||||
# Dockerfile.lint, which COPYs the repo into the pinned golangci-lint image
|
# --target names it. --no-cache because a cached lint layer is a lint
|
||||||
# and lints as a build step. This works even when the docker daemon is remote
|
# that did not run. The tag makes each build replace the previous image
|
||||||
# and bind mounts are impossible, and it removes the host linter's shared
|
# instead of leaving a dangling one behind.
|
||||||
# cache, which has attributed other checkouts' findings to this one.
|
|
||||||
#
|
|
||||||
# --no-cache-filter=lint forces the lint stage to re-execute on every run; a
|
|
||||||
# cached lint stage exits 0 in under a second having linted nothing. The deps
|
|
||||||
# stage keeps its cache, so module downloads are not repeated.
|
|
||||||
# --progress=plain keeps the linter's own output visible on success, so a
|
|
||||||
# passing run shows the issue count rather than nothing.
|
|
||||||
# --output=type=cacheonly leaves no image behind to clean up.
|
|
||||||
#
|
|
||||||
# docker silently ignores --no-cache-filter for a stage name that does not
|
|
||||||
# match, so a rename or a typo would restore the cached false green with no
|
|
||||||
# warning and a fast exit 0. The flag is therefore not trusted: the build
|
|
||||||
# output is teed to a log and a run is only a pass if golangci-lint's own
|
|
||||||
# summary line ("N issues." / "N issues:") is in it. No summary, no lint,
|
|
||||||
# whatever the exit code says.
|
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
|
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
|
docker build --no-cache \
|
||||||
log="$(mktemp -t webhooker-lint.XXXXXXXX)"
|
--target lint \
|
||||||
rcfile="$(mktemp -t webhooker-lint-rc.XXXXXXXX)"
|
-t "$("$SCRIPT_DIR/projectname")-lint" .
|
||||||
trap 'rm -f "$log" "$rcfile"' EXIT INT TERM
|
|
||||||
|
|
||||||
# The pipeline's status is tee's, and POSIX sh has no pipefail, so the
|
|
||||||
# build's status travels via a file. Output still streams live.
|
|
||||||
{
|
|
||||||
docker build \
|
|
||||||
-f Dockerfile.lint \
|
|
||||||
--no-cache-filter=lint \
|
|
||||||
--progress=plain \
|
|
||||||
--output=type=cacheonly \
|
|
||||||
. 2>&1 && echo 0 >"$rcfile" || echo $? >"$rcfile"
|
|
||||||
} | tee "$log" >&2
|
|
||||||
|
|
||||||
rc="$(cat "$rcfile")"
|
|
||||||
[ "$rc" -eq 0 ] || exit "$rc"
|
|
||||||
|
|
||||||
if ! grep -qE '[0-9]+ issues[.:]' "$log"; then
|
|
||||||
echo "script/lint: golangci-lint printed no summary line; the linter" >&2
|
|
||||||
echo " did not run. Check that the stage named in --no-cache-filter" >&2
|
|
||||||
echo " still matches a stage in Dockerfile.lint." >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# ESLint runs in the Dockerfile's js-lint stage, which the image build
|
|
||||||
# runs too. It prints nothing on a pass, so there is no summary to look
|
|
||||||
# for. Instead the stage is named once, for both flags: --target fails
|
|
||||||
# on a name that matches no stage, so a rename cannot leave
|
|
||||||
# --no-cache-filter silently ignored. The js-deps stage, which installs
|
|
||||||
# ESLint, keeps its cache, so ESLint is not downloaded again.
|
|
||||||
js_stage=js-lint
|
|
||||||
docker build \
|
|
||||||
--target "$js_stage" \
|
|
||||||
--no-cache-filter="$js_stage" \
|
|
||||||
--progress=plain \
|
|
||||||
--output=type=cacheonly \
|
|
||||||
.
|
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+10
-75
@@ -1,84 +1,19 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/test: run the test suite.
|
# script/test: run the test suite. Testing is a phase of the Dockerfile
|
||||||
#
|
# and this builds that phase alone, on the same terms as script/lint:
|
||||||
# -timeout is applied by `go test` per package, not to the run as a whole, so
|
# --target because a phase that is not the last stage is built only when
|
||||||
# it only has to clear the slowest single package. When this budget was set
|
# named, --no-cache because a cached test layer is a test that did not
|
||||||
# that was internal/handlers, measured in a cache-defeated builder stage on the
|
# run, and a tag so each build replaces the previous image.
|
||||||
# 48-core shared build host (2026-08-18); load- and host-dependent, not
|
|
||||||
# invariants:
|
|
||||||
#
|
|
||||||
# 16.9s host load 5-20, GOMAXPROCS 48
|
|
||||||
# 45.9s / 47.3s / 49.0s three runs at deliberate host load 31-73
|
|
||||||
# 30.6s / 39.7s host load 5-20, GOMAXPROCS 6 / 4
|
|
||||||
# 67.3s / 97.5s host load 5-20, GOMAXPROCS 2 / 1
|
|
||||||
# 67.3s GOMAXPROCS 4 at deliberate host load 52-68
|
|
||||||
#
|
|
||||||
# The old 30s budget was breached by every loaded run and by every GOMAXPROCS
|
|
||||||
# at or below 6; at GOMAXPROCS 4 it failed outright ("panic: test timed out
|
|
||||||
# after 30s"), reproduced on 33e4fa4 with no other change.
|
|
||||||
#
|
|
||||||
# 90s matches the org-wide backstop in REPO_POLICIES.md and is sized here
|
|
||||||
# against the figures above: the worst case under native parallelism is 49.0s,
|
|
||||||
# and the compound GOMAXPROCS-4-under-load case at 67.3s sits at 75% of it.
|
|
||||||
# The one figure above 90s is GOMAXPROCS 1, a synthetic core floor rather than
|
|
||||||
# a condition CI runs under. If a CPU-limited runner ever puts a real run near
|
|
||||||
# 67s, that is the datum to revisit the org figure with.
|
|
||||||
#
|
|
||||||
# Those figures predate tests hashing the admin password at 1 MB instead of
|
|
||||||
# 64 MB (https://git.eeqj.de/sneak/webhooker/pulls/404). After that change, in
|
|
||||||
# a cache-defeated build at host load 44-109 (2026-10-02), internal/handlers
|
|
||||||
# took 8.5s and the slowest package was internal/database at 15.8s. Once its
|
|
||||||
# retention tests seeded 50 rows per insert instead of 500
|
|
||||||
# (https://git.eeqj.de/sneak/webhooker/issues/198), internal/database took
|
|
||||||
# 7.3s and the slowest package was internal/handlers at 8.1s to 10.0s, at host
|
|
||||||
# load 25-48 (2026-10-02).
|
|
||||||
#
|
|
||||||
# -p 4 -parallel 8 keep the run under 2 GB of memory: at most four test
|
|
||||||
# binaries build or run at once, each with at most eight parallel tests. Under
|
|
||||||
# -race every test binary and every link costs a few hundred MB, so the
|
|
||||||
# defaults (one per core) add up to several GB on a many-core host.
|
|
||||||
#
|
|
||||||
# The first run has no -v: go test then prints one result line per package,
|
|
||||||
# with its coverage, and for a package that fails, everything its tests wrote,
|
|
||||||
# application log lines included. Verbose output from the whole suite passes
|
|
||||||
# the 2 MiB at which the Docker build cuts off each step's log, so on a failure
|
|
||||||
# only the tests that failed run again, with -v. The script exits 1 after that
|
|
||||||
# rerun whatever its result: the first run already showed the suite is broken.
|
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
|
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
"$ROOT/script/assets"
|
docker build --no-cache \
|
||||||
|
--target test \
|
||||||
log="$(mktemp -t webhooker-test.XXXXXXXX)"
|
-t "$("$SCRIPT_DIR/projectname")-test" .
|
||||||
rcfile="$(mktemp -t webhooker-test-rc.XXXXXXXX)"
|
|
||||||
trap 'rm -f "$log" "$rcfile"' EXIT INT TERM
|
|
||||||
|
|
||||||
# The pipeline's status is tee's, and POSIX sh has no pipefail, so go
|
|
||||||
# test's status travels via a file. Output still streams live.
|
|
||||||
{
|
|
||||||
go test -race -cover -p 4 -parallel 8 -timeout 90s ./... 2>&1 \
|
|
||||||
&& echo 0 >"$rcfile" || echo $? >"$rcfile"
|
|
||||||
} | tee "$log"
|
|
||||||
if [ "$(cat "$rcfile")" -eq 0 ]; then
|
|
||||||
return
|
|
||||||
fi
|
|
||||||
|
|
||||||
# go test reports a failed test as a line starting "--- FAIL: TestName"
|
|
||||||
# (a failed subtest's line is indented, and reruns with its parent), and
|
|
||||||
# a failed package as "FAIL<tab>package/path<tab>...". A failure that
|
|
||||||
# names no test, such as a build error or a timeout, is already shown in
|
|
||||||
# full above, so there is nothing to rerun.
|
|
||||||
tests="$(awk '/^--- FAIL: / { print $3 }' "$log" | paste -s -d '|' -)"
|
|
||||||
packages="$(awk '/^FAIL\t/ { print $2 }' "$log")"
|
|
||||||
if [ -n "$tests" ]; then
|
|
||||||
echo "--- Rerunning the failed tests with -v for details ---"
|
|
||||||
go test -race -v -p 4 -parallel 8 -timeout 90s \
|
|
||||||
-run "^($tests)\$" $packages || true
|
|
||||||
fi
|
|
||||||
exit 1
|
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+2
-3
@@ -3,8 +3,7 @@
|
|||||||
# Docker: Dockerfile.browser builds the test and runs it in a digest-pinned
|
# Docker: Dockerfile.browser builds the test and runs it in a digest-pinned
|
||||||
# headless browser image, so the host needs no browser.
|
# headless browser image, so the host needs no browser.
|
||||||
#
|
#
|
||||||
# --no-cache-filter=browser runs the test again even when nothing changed;
|
# --no-cache because a cached test layer is a test that did not run.
|
||||||
# it must name the stage in Dockerfile.browser that runs it.
|
|
||||||
# --output=type=cacheonly leaves no image behind to clean up.
|
# --output=type=cacheonly leaves no image behind to clean up.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
@@ -14,7 +13,7 @@ main() {
|
|||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build \
|
docker build \
|
||||||
-f Dockerfile.browser \
|
-f Dockerfile.browser \
|
||||||
--no-cache-filter=browser \
|
--no-cache \
|
||||||
--progress=plain \
|
--progress=plain \
|
||||||
--output=type=cacheonly \
|
--output=type=cacheonly \
|
||||||
.
|
.
|
||||||
|
|||||||
+5
-3
@@ -1,9 +1,11 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/version: output the version string the binary is stamped with.
|
# script/version: output the version string the binary is stamped with.
|
||||||
# Our own extension to scripts-to-rule-them-all. The Makefile's build
|
# Our own extension to scripts-to-rule-them-all. The Makefile's build
|
||||||
# target and script/docker both take the value from here, so a `make
|
# and version targets take the value from here, and the Dockerfile's
|
||||||
# build` binary and a `make docker` image built from the same checkout
|
# build stage calls them. script/docker and script/cibuild run the same
|
||||||
# report the same thing.
|
# `git describe` on the host and pass the result in as $VERSION, so a
|
||||||
|
# `make build` binary and a `make docker` image built from the same
|
||||||
|
# checkout report the same thing.
|
||||||
#
|
#
|
||||||
# Order of precedence:
|
# Order of precedence:
|
||||||
#
|
#
|
||||||
|
|||||||
Reference in New Issue
Block a user