Serve /metrics from a registry of its own (closes #227)

The HTTP metrics recorder, the delivery collectors and the Go and
process collectors now register on one prometheus.Registry that fx
provides, instead of Prometheus's global default registry, and
/metrics serves that registry. A second metrics-enabled router in one
process, or the server tests run with -count=2, no longer panics on a
duplicate registration.

The middleware builds its recorder once, in New, so installing
Metrics() on more than one router over the same graph is also safe;
NewForTest gives its Middleware a recorder on a fresh registry. The
scrape keeps the same series and labels, including go_*, process_*
and promhttp_metric_handler_*.

Model: opus-5-5

.dockerignore

@@ -1,18 +1,14 @@
-# .git is sent, without its config, so the build can derive the version it
-# stamps into the binary (script/version). The config can hold a remote URL
-# carrying a credential, and `git describe` does not need it.
-.git/config
-
-# No tracked file may be listed here: git in the build would see it as
-# deleted and mark the version -dirty.
-#
 # .ci-fingerprint is deliberately NOT excluded: it is the CI cache barrier
 # that keeps the check stages from replaying a cached pass. See the lint
 # stage of the Dockerfile.
+.git/
 bin/
 # Extracted from 3p/ by `make assets` inside the build; a host copy is not
 # needed. The tarball in 3p/ must stay in the context.
 static/js/alpine.min.js
+*.md
+LICENSE
+.editorconfig
 .env
 .env.*
 *.db

.gitea/workflows/check.yml

@@ -12,8 +12,9 @@ jobs:
       - name: Checkout
         uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 2024-10-23
         with:
-          # The superseded-status step needs history to walk ancestors (it
+          # The fingerprint step below needs history to find the last commit
-          # aborts on a shallow clone).
+          # that touched the Docker build context, and the superseded-status
+          # step needs it to walk ancestors (it aborts on a shallow clone).
           fetch-depth: 0
 
       - name: Mark superseded run statuses
@@ -27,11 +28,16 @@ jobs:
         run: script/ci-mark-superseded
 
       - name: Fingerprint the build context
-        # Writes the hash of the commit being checked into the context, which
+        # `.dockerignore` keeps docs out of the build context, so a docs-only
-        # invalidates the `COPY . .` layer of both check stages: a commit
+        # commit legitimately replays the whole image from cache and stays
-        # that was never linted, format-checked, tested and built cannot
+        # cheap. Every other commit writes a new fingerprint into the context,
-        # report success from cache.
+        # which invalidates the `COPY . .` layer of both check stages: a
-        run: git rev-parse HEAD > .ci-fingerprint
+        # commit that was never linted, formatted-checked, tested and built
+        # cannot report success from cache.
+        run: |
+          set -eu
+          fp="$(git log -1 --format=%H -- . ':!*.md' ':!LICENSE' ':!.editorconfig')"
+          printf '%s\n' "${fp:-$GITHUB_SHA}" > .ci-fingerprint
 
       - name: Build Docker image (runs make check)
         run: script/cibuild

Dockerfile

@@ -12,8 +12,8 @@ WORKDIR /src
 COPY go.mod go.sum ./
 RUN go mod download
 
-# Copy source code. In CI the context also carries .ci-fingerprint, which
+# Copy source code. In CI the context also carries .ci-fingerprint, whose
-# holds the hash of the commit being checked (see
+# value changes with every commit that touches the build context (see
 # .gitea/workflows/check.yml). That invalidates this layer, so the checks
 # below cannot report success by replaying a cached pass. Do not add it to
 # .dockerignore.
@@ -38,13 +38,8 @@ FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a349228
 COPY --from=lint /src/go.sum /dev/null
 
 # jq is a runtime dependency of script/ci-mark-superseded, which the test
-# suite executes. git is what script/version derives the version with.
+# suite executes.
-RUN apt-get update && apt-get install -y --no-install-recommends make curl ca-certificates jq git && rm -rf /var/lib/apt/lists/*
+RUN apt-get update && apt-get install -y --no-install-recommends make curl ca-certificates jq && rm -rf /var/lib/apt/lists/*
-
-# A build context sent as a tar archive keeps its files' owners, and git
-# refuses to read a checkout owned by another user. Trust this one
-# whoever owns it.
-RUN git config --system --add safe.directory /build
 
 WORKDIR /build
 
@@ -60,22 +55,14 @@ COPY . .
 # from its tarball in 3p/.
 RUN make test
 
-# Version stamped into the binary: the VERSION build arg when one is
+# Version stamped into the binary. .dockerignore excludes .git/, so
-# given, otherwise what script/version derives from the .git the build
+# nothing in this stage can derive it: script/docker resolves it on the
-# context carries, so any `docker build .` of a clone stamps its commit.
+# host and passes it in. The default is what a bare `docker build .`
-# With neither, as from a source tarball, it is "unknown".
+# with no --build-arg gets, and it names no tag the tree may not be at.
 #
 # Declared here, below the test step, so a changed version does not
 # invalidate its cached layer.
-ARG VERSION
+ARG VERSION=unknown
-
-# A context that carries .git must not stamp "unknown": that means git is
-# missing here or could not read the checkout, and the image could not be
-# traced back to its commit.
-RUN if [ -d .git ] && [ "$(make version VERSION="$VERSION")" = unknown ]; then \
-        echo "version is unknown although the build context carries .git" >&2; \
-        exit 1; \
-    fi
 
 RUN make build VERSION="$VERSION"
 

Makefile

@@ -4,12 +4,12 @@
 .DEFAULT_GOAL := check
 
 # Version stamped into the binary. Derived from git by script/version;
-# override it (`make build VERSION=v1.2.3`) to stamp a given value, which is
+# override it (`make build VERSION=v1.2.3`) where git metadata is
-# how the Dockerfile passes its build arg in.
+# unavailable, which is how the Dockerfile passes its build arg in.
 VERSION ?= $(shell script/version)
 
-# An empty override (`make build VERSION=`, or the Dockerfile's `make build
+# An empty override (`make build VERSION=`, or a `--build-arg VERSION=`
-# VERSION="$VERSION"` when no VERSION build arg was given) means unset,
+# landing on the Dockerfile's `make build VERSION="$VERSION"`) means unset,
 # exactly as it does in script/version -- stamping "" would leave the binary
 # reporting no version and the footer back on its "dev" fallback. `override`
 # is required: a plain assignment loses to the command-line definition it

README.md

@@ -1133,26 +1133,13 @@ build itself.
 | Uncommitted changes | the above with a `-dirty` suffix |
 | No git metadata | `unknown` |
 
-The image derives it the same way, from the `.git` that the build
+`unknown` is what a source tarball or a `docker build .` with no
-context carries, so any `docker build .` of a clone, with no build
+`--build-arg VERSION=...` reports. `.dockerignore` excludes `.git/`, so
-arguments, stamps the commit it was built from; a shallow clone of one
+the build context carries no git metadata and the image cannot derive
-branch has no tags and stamps the short SHA. `.dockerignore` must
+the version itself: `script/docker` (and so `make docker`) resolves it
-therefore leave out neither `.git` nor any tracked file, which git in
+on the host and passes it in as the `VERSION` build arg. A build that
-the build would see as deleted, marking the version `-dirty`. It does
+reports `unknown` is a build nobody told what it was; it is not a
-leave out `.git/config`, which can hold a remote URL carrying a
+failure, but it cannot be traced back to a commit.
-credential and which `git describe` does not need. git in the build
-reads the checkout whoever owns its files, since a context sent as a tar
-archive keeps the sender's owners and git otherwise refuses a checkout
-owned by another user. A `VERSION` build arg (`--build-arg VERSION=...`)
-takes precedence; `script/docker` (and so `make docker`) passes the one
-`script/version` resolves on the host. The image build fails if its
-context carries `.git` and the version still comes out `unknown`, which
-means git is missing from the build or could not read the checkout.
-
-`unknown` is what a source tarball, or a `docker build` with no `.git`
-in its context and no `VERSION` build arg, reports. A build that reports
-`unknown` is a build nobody told what it was; it is not a failure, but
-it cannot be traced back to a commit.
 
 `make version` prints what the current checkout would stamp, and
 `make build VERSION=v1.2.3` overrides it. An empty override — from
@@ -2896,13 +2883,15 @@ Components are wired via Uber fx in this order:
 7. `healthcheck.New` — Health check service
 8. `session.New` — Cookie-based session manager (key from database)
 9. `handlers.New` — HTTP handlers
-10. `middleware.New` — HTTP middleware
+10. `metrics.NewRegistry` — The registry `/metrics` serves
-11. `delivery.New` — Event-driven delivery engine
+11. `metrics.New` — The delivery collectors, registered on that registry
-12. `delivery.NewArchiveSweeper` — Periodic pruning of idle archives
+12. `middleware.New` — HTTP middleware
-13. `delivery.Engine` → `delivery.Notifier` — interface bridge
+13. `delivery.New` — Event-driven delivery engine
-14. `delivery.Engine` → `delivery.WebhookEvictor` — interface bridge so
+14. `delivery.NewArchiveSweeper` — Periodic pruning of idle archives
+15. `delivery.Engine` → `delivery.Notifier` — interface bridge
+16. `delivery.Engine` → `delivery.WebhookEvictor` — interface bridge so
     deleting a webhook releases its archive writer
-15. `server.New` — HTTP server and router
+17. `server.New` — HTTP server and router
 
 The server starts via `fx.Invoke(func(*server.Server, *delivery.Engine,
 *database.RetentionReaper, *delivery.ArchiveSweeper) {})`, which
@@ -3187,9 +3176,8 @@ version is fixed independently of the compiler's:
    rebuilds the binary with `CGO_ENABLED=1` and static linking so it
    runs on musl. Both builds go through `make build`, the relink adding
    its `-extldflags` via `GO_LDFLAGS`, so neither can drop the `-X` that
-   stamps the version. The version is the `VERSION` build arg if one is
+   stamps the version. The version arrives as the `VERSION` build arg,
-   given, otherwise derived from the `.git` in the context, and the
+   since the context has no `.git` (see
-   stage fails if a context with `.git` would stamp `unknown` (see
    [Version stamping](#version-stamping)).
 3. **Runtime stage** (`alpine:3.21`) — copies the static binary and
    `deploy/docker-entrypoint.sh`, creates the `/var/lib/webhooker`
@@ -3221,13 +3209,19 @@ A layer cache lets `docker build .` exit 0 in seconds with the lint and
 test stages replayed rather than executed, which would make a green
 check meaningless. The `check` workflow therefore writes
 `.ci-fingerprint` into the build context before building. Its value is
-the hash of the commit being checked, so every commit, docs-only ones
+the hash of the last commit that touched the build context, so:
-and a squash merge whose tree matches an already-built branch included,
-gets a new fingerprint, invalidates the `COPY . .` layer of both check
-stages, and really runs `make fmt-check`, `golangci-lint`, `make test`,
-and `make build`. A run that reports success ran them.
 
-The module download layer sits above `COPY . .` and stays cached.
+- Any commit that changes code (including a squash merge whose tree
+  matches an already-built branch) gets a new fingerprint, invalidates
+  the `COPY . .` layer of both check stages, and really runs
+  `make fmt-check`, `golangci-lint`, `make test`, and `make build`. A
+  run that reports success ran them.
+- A docs-only commit leaves the fingerprint unchanged — `.dockerignore`
+  excludes `*.md`, `LICENSE` and `.editorconfig` from the context
+  anyway — so the image replays from cache and costs seconds.
+
+The module download layer sits above `COPY . .` and stays cached either
+way.
 
 A separate workflow step, run before the fingerprint is written, covers
 a second way the gate lied: Gitea cancels an in-flight run when a newer

TODO.md

@@ -387,7 +387,7 @@ point of the branch.
 - 2026-03-05 security headers middleware, session regeneration on
   login, request body size limits (#41)
 - 2026-03-04 tests for delivery, middleware, and session packages
-  (#32); removed the build-architecture global (#31)
+  (#32); removed globals.Buildarch (#31)
 - 2026-03-04 1.0 MVP merge: Webhook/Entrypoint/Target rename, core
   delivery engine with bounded worker pool and circuit breaker,
   parallel fan-out, per-webhook event databases, management UI (#16)

cmd/webhooker/main.go

@@ -16,6 +16,7 @@ import (
 	"sneak.berlin/go/webhooker/internal/handlers"
 	"sneak.berlin/go/webhooker/internal/healthcheck"
 	"sneak.berlin/go/webhooker/internal/logger"
+	"sneak.berlin/go/webhooker/internal/metrics"
 	"sneak.berlin/go/webhooker/internal/middleware"
 	"sneak.berlin/go/webhooker/internal/resetpw"
 	"sneak.berlin/go/webhooker/internal/server"
@@ -177,6 +178,10 @@ func newApp() *fx.App {
 			healthcheck.New,
 			session.New,
 			handlers.New,
+			// The registry /metrics serves, and the delivery
+			// collectors registered on it.
+			metrics.NewRegistry,
+			metrics.New,
 			middleware.New,
 			// The one SSRF guard both target-creation validation
 			// and the delivery dialer consult, so they cannot

internal/database/export_test.go

@@ -5,7 +5,6 @@ import (
 	"io"
 	"log/slog"
 	"os"
-	"testing"
 	"time"
 
 	"go.uber.org/fx"
@@ -80,14 +79,3 @@ func (d *Database) ExportSetBannerOut(w io.Writer) {
 func DummyPasswordHashForTest() string {
 	return dummyPasswordHash()
 }
-
-// HashAtShippedCostForTest makes HashPassword hash at the shipped
-// memory cost until t ends. t must not run in parallel with other
-// tests, which would hash at that cost alongside it.
-func HashAtShippedCostForTest(t *testing.T) {
-	t.Helper()
-
-	hashAtShippedCostInTest = true
-
-	t.Cleanup(func() { hashAtShippedCostInTest = false })
-}

internal/database/password.go

@@ -9,7 +9,6 @@ import (
 	"math/big"
 	"strings"
 	"sync"
-	"testing"
 
 	"golang.org/x/crypto/argon2"
 )
@@ -64,30 +63,10 @@ func DefaultPasswordConfig() *PasswordConfig {
 	}
 }
 
-// testArgon2Memory is the Argon2id memory cost, in KiB, that a test
+// HashPassword generates an Argon2id hash of the password
-// binary hashes with: 1 MB instead of the shipped 64 MB. Every test
-// that starts a database hashes the bootstrap admin password, dozens
-// of them run in parallel, and under the race detector each 64 MB hash
-// holds about 150 MB. VerifyPassword reads the cost from the hash it
-// checks, so verification follows.
-const testArgon2Memory = 1024
-
-// hashAtShippedCostInTest makes a test binary hash at the shipped
-// memory cost. Only TestHashPassword_ShippedParameters sets it.
-//
-//nolint:gochecknoglobals // set by one test, see above
-var hashAtShippedCostInTest bool
-
-// HashPassword generates an Argon2id hash of the password. A binary
-// built by go test hashes at testArgon2Memory; one built by go build
-// always hashes at the defaults.
 func HashPassword(password string) (string, error) {
 	config := DefaultPasswordConfig()
 
-	if testing.Testing() && !hashAtShippedCostInTest {
-		config.Memory = testArgon2Memory
-	}
-
 	// Generate a salt
 	salt := make([]byte, config.SaltLen)
 

internal/database/password_test.go

@@ -192,39 +192,6 @@ func TestHashPasswordUniqueness(t *testing.T) {
 	}
 }
 
-// TestHashPassword_ShippedParameters hashes and verifies through
-// HashPassword at the shipped Argon2id parameters. Every other test
-// hashes at the lower memory cost a test binary uses, so this is the
-// one that keeps production hashing covered. One hash and one
-// verification: each costs 64 MB.
-//
-//nolint:paralleltest // changes the hashing cost for the whole binary
-func TestHashPassword_ShippedParameters(t *testing.T) {
-	database.HashAtShippedCostForTest(t)
-
-	password := "correct horse battery staple"
-
-	hash, err := database.HashPassword(password)
-	if err != nil {
-		t.Fatalf("hashing with the shipped parameters: %v", err)
-	}
-
-	const shipped = "$argon2id$v=19$m=65536,t=1,p=4$"
-
-	if !strings.HasPrefix(hash, shipped) {
-		t.Errorf("hash = %q, want prefix %q", hash, shipped)
-	}
-
-	valid, err := database.VerifyPassword(password, hash)
-	if err != nil {
-		t.Fatalf("VerifyPassword() error = %v", err)
-	}
-
-	if !valid {
-		t.Error("VerifyPassword() returned false for correct password")
-	}
-}
-
 // TestVerifyDummyPassword_DoesRealWork covers the anti-enumeration
 // path. Login charges an unknown username a verification against a
 // dummy hash so that a nonexistent account is not answered in

internal/delivery/engine.go

@@ -148,6 +148,7 @@ type EngineParams struct {
 	DBManager *database.WebhookDBManager
 	Logger    *logger.Logger
 	SSRFGuard *Guard
+	Metrics   *metrics.Set
 }
 
 // Engine processes queued deliveries in the background
@@ -167,10 +168,10 @@ type Engine struct {
 	retryCh    chan Task
 	workers    int
 
-	// mtr is the delivery metric set. Production wires the
+	// mtr is the delivery metric set. Production wires the one
-	// process-wide one; a test can substitute a set registered on
+	// registered on the registry /metrics serves; a test can
-	// a private registry so its assertions are not disturbed by
+	// substitute a set registered on a registry it holds, so it can
-	// deliveries other tests are making at the same time.
+	// gather what its own deliveries recorded.
 	mtr *metrics.Set
 
 	// targets maps each target type to its implementation.
@@ -204,7 +205,7 @@ func New(
 		deliveryCh: make(chan Task, deliveryChannelSize),
 		retryCh:    make(chan Task, retryChannelSize),
 		workers:    defaultWorkers,
-		mtr:        metrics.Default(),
+		mtr:        params.Metrics,
 	}
 
 	e.initTargets(&http.Client{

internal/delivery/export_test.go

@@ -9,6 +9,7 @@ import (
 	"net/url"
 	"time"
 
+	"github.com/prometheus/client_golang/prometheus"
 	"go.uber.org/fx"
 	"gorm.io/gorm"
 	"sneak.berlin/go/webhooker/internal/database"
@@ -389,7 +390,7 @@ func NewTestEngine(
 		deliveryCh: make(chan Task, deliveryChannelSize),
 		retryCh:    make(chan Task, retryChannelSize),
 		workers:    workers,
-		mtr:        metrics.Default(),
+		mtr:        metrics.New(prometheus.NewRegistry()),
 	}
 	e.initTargets(client)
 
@@ -404,7 +405,7 @@ func NewTestEngineSmallRetry(
 	e := &Engine{
 		log:     log,
 		retryCh: make(chan Task, 1),
-		mtr:     metrics.Default(),
+		mtr:     metrics.New(prometheus.NewRegistry()),
 	}
 	e.initTargets(nil)
 
@@ -427,7 +428,7 @@ func NewTestEngineWithDB(
 		deliveryCh: make(chan Task, deliveryChannelSize),
 		retryCh:    make(chan Task, retryChannelSize),
 		workers:    workers,
-		mtr:        metrics.Default(),
+		mtr:        metrics.New(prometheus.NewRegistry()),
 	}
 	e.initTargets(client)
 
@@ -435,8 +436,7 @@ func NewTestEngineWithDB(
 }
 
 // ExportSetMetrics substitutes the engine's metric set, so a test can
-// assert on collectors registered on a private registry instead of
+// assert on collectors registered on a registry it holds.
-// the process-wide ones every other test is also moving.
 func (e *Engine) ExportSetMetrics(mtr *metrics.Set) {
 	e.mtr = mtr
 }

internal/delivery/metrics_test.go

@@ -35,9 +35,8 @@ const (
 )
 
 // mIsolate gives the setup's engine a metric set registered on a
-// private registry. The process-wide collectors are moved by every
+// registry this test holds, so its exact assertions can gather from
-// other delivery test running in parallel, so exact assertions are
+// it.
-// only possible against a registry this test owns.
 func mIsolate(
 	t *testing.T, s iSetup,
 ) *prometheus.Registry {

internal/handlers/handlers.go

@@ -12,6 +12,7 @@ import (
 	"net/http"
 	"sync/atomic"
 
+	"github.com/prometheus/client_golang/prometheus"
 	"go.uber.org/fx"
 	"sneak.berlin/go/webhooker/internal/database"
 	"sneak.berlin/go/webhooker/internal/delivery"
@@ -61,6 +62,8 @@ type HandlersParams struct {
 	Notifier     delivery.Notifier
 	Evictor      delivery.WebhookEvictor
 	SSRFGuard    *delivery.Guard
+	Metrics      *metrics.Set
+	Registry     *prometheus.Registry
 }
 
 // Handlers provides HTTP handler methods for all application
@@ -122,7 +125,7 @@ func New(
 	s.mw = params.Middleware
 	s.notifier = params.Notifier
 	s.evictor = params.Evictor
-	s.mtr = metrics.Default()
+	s.mtr = params.Metrics
 	s.ssrf = params.SSRFGuard
 
 	// Parse all page templates once at startup

internal/handlers/handlers_test.go

@@ -20,6 +20,7 @@ import (
 	"sneak.berlin/go/webhooker/internal/handlers"
 	"sneak.berlin/go/webhooker/internal/healthcheck"
 	"sneak.berlin/go/webhooker/internal/logger"
+	"sneak.berlin/go/webhooker/internal/metrics"
 	"sneak.berlin/go/webhooker/internal/middleware"
 	"sneak.berlin/go/webhooker/internal/session"
 )
@@ -109,6 +110,8 @@ func newTestApp(
 			func(r *recordingEvictor) delivery.WebhookEvictor {
 				return r
 			},
+			metrics.NewRegistry,
+			metrics.New,
 			middleware.New,
 			delivery.NewGuard,
 			handlers.New,

internal/handlers/metrics.go

@@ -0,0 +1,21 @@
+package handlers
+
+import (
+	"net/http"
+
+	"github.com/prometheus/client_golang/prometheus/promhttp"
+)
+
+// HandleMetrics returns the Prometheus scrape handler for the
+// registry built by metrics.NewRegistry, which the HTTP, delivery, Go
+// runtime and process collectors register on. It is what
+// promhttp.Handler builds for the global default registry, including
+// the promhttp_metric_handler_* series that count scrapes, pointed at
+// that registry instead.
+func (s *Handlers) HandleMetrics() http.HandlerFunc {
+	reg := s.params.Registry
+
+	return promhttp.InstrumentMetricHandler(
+		reg, promhttp.HandlerFor(reg, promhttp.HandlerOpts{}),
+	).ServeHTTP
+}

internal/metrics/metrics.go

@@ -3,17 +3,18 @@
 // deliveries are attempted, how they end, how long they take, how
 // deep the queues are, and how many circuit breakers are open.
 //
-// The inbound HTTP metrics come from the go-http-metrics recorder in
+// It also builds the registry the authenticated /metrics route
-// internal/middleware and land on prometheus.DefaultRegisterer. These
+// serves. In production, these collectors, the inbound HTTP metrics
-// collectors register there too, so both surfaces are gathered by the
+// recorded in internal/middleware, and the Go runtime and process
-// one promhttp handler mounted on the authenticated /metrics route.
+// collectors all register on that one registry, never on Prometheus's
+// global default.
 package metrics
 
 import (
-	"sync"
 	"time"
 
 	"github.com/prometheus/client_golang/prometheus"
+	"github.com/prometheus/client_golang/prometheus/collectors"
 	"github.com/prometheus/client_golang/prometheus/promauto"
 	"sneak.berlin/go/webhooker/internal/database"
 )
@@ -57,25 +58,31 @@ var knownTargetTypes = []database.TargetType{
 	database.TargetTypeSlack,
 }
 
-// defaultSet is the process-wide metric set, registered on the same
+// NewRegistry returns the registry /metrics serves, carrying the Go
-// registry the HTTP middleware and the /metrics handler already use.
+// runtime and process collectors that Prometheus's global default
-// It is built on first use rather than in an init so that a test
+// registry carries, so the go_* and process_* series stay in the
-// binary that never touches metrics never registers them.
+// scrape.
 //
-//nolint:gochecknoglobals // one process-wide registration, by design
+// A registry of its own, rather than the global default, is what lets
-var defaultSet = sync.OnceValue(func() *Set {
+// two dependency graphs in one process — two tests, say — each
-	return New(prometheus.DefaultRegisterer)
+// register their collectors without the second registration
-})
+// panicking.
+func NewRegistry() *prometheus.Registry {
+	reg := prometheus.NewRegistry()
+	reg.MustRegister(
+		collectors.NewGoCollector(),
+		collectors.NewProcessCollector(
+			collectors.ProcessCollectorOpts{},
+		),
+	)
 
-// Default returns the process-wide metric set.
+	return reg
-func Default() *Set {
-	return defaultSet()
 }
 
 // Set is one registered group of webhooker's delivery collectors.
-// Production uses the single Default set; tests build their own
+// Production builds one on the registry /metrics serves; tests build
-// against a private registry so assertions are not disturbed by
+// one on a registry of their own so they can gather what their own
-// deliveries other tests are making concurrently.
+// deliveries recorded.
 type Set struct {
 	eventsReceived      prometheus.Counter
 	deliveryAttempts    *prometheus.CounterVec
@@ -93,7 +100,7 @@ type Set struct {
 // New registers a full set of delivery collectors on reg and returns
 // it. It panics if reg already holds them, which is the intended
 // behaviour for a duplicate registration.
-func New(reg prometheus.Registerer) *Set {
+func New(reg *prometheus.Registry) *Set {
 	factory := promauto.With(reg)
 
 	s := &Set{

internal/middleware/export_test.go

@@ -10,8 +10,7 @@ import (
 
 // MetricsMiddlewareForTest builds the metrics recording middleware
 // against a caller-supplied recorder, so a test can gather from its
-// own Prometheus registry rather than the process-wide default one
+// own Prometheus registry without building a whole Middleware.
-// that Middleware.Metrics uses.
 func MetricsMiddlewareForTest(
 	rec httpmetrics.Recorder,
 ) func(http.Handler) http.Handler {

internal/middleware/metrics.go

@@ -7,7 +7,6 @@ import (
 
 	"github.com/go-chi/chi"
 	httpmetrics "github.com/slok/go-http-metrics/metrics"
-	prommetrics "github.com/slok/go-http-metrics/metrics/prometheus"
 	ghmm "github.com/slok/go-http-metrics/middleware"
 	"github.com/slok/go-http-metrics/middleware/std"
 )
@@ -151,17 +150,17 @@ func (r boundedLabelRecorder) AddInflightRequests(
 
 var _ httpmetrics.Recorder = boundedLabelRecorder{}
 
-// Metrics returns middleware that records Prometheus HTTP metrics on
+// Metrics returns middleware that records Prometheus HTTP metrics
-// the default registry, which is the one the /metrics route gathers.
+// with the Middleware's one recorder, which New builds on the registry
+// the /metrics route serves and NewForTest on a registry of its own.
+// Every call reuses that recorder, so any number of routers can
+// install it.
 func (s *Middleware) Metrics() func(http.Handler) http.Handler {
-	return metricsMiddleware(
+	return metricsMiddleware(s.metricsRecorder)
-		prommetrics.NewRecorder(prommetrics.Config{}),
-	)
 }
 
 // metricsMiddleware builds the recording middleware against a given
-// recorder, so tests can gather from a registry of their own instead
+// recorder, so tests can gather from a registry of their own.
-// of the process-wide default.
 func metricsMiddleware(
 	rec httpmetrics.Recorder,
 ) func(http.Handler) http.Handler {

internal/middleware/metrics_test.go

@@ -57,9 +57,8 @@ const (
 // Server.setupWebhookRoutes inside it. That ordering is the whole
 // defect, so a test that flattens it would prove nothing.
 //
-// The recorder writes to a registry of the test's own rather than the
+// The recorder writes to a registry of the test's own, so each test
-// process-wide default one, so each test observes only its own
+// observes only its own traffic.
-// traffic.
 func metricsTestRouter(
 	t *testing.T,
 	receiverLimit int,
@@ -455,3 +454,29 @@ func TestMetrics_StatusAndSizeStillRecorded(t *testing.T) {
 		"the interceptor must still count written bytes",
 	)
 }
+
+// TestMetrics_WorksOnNewForTestMiddleware pins that a Middleware built
+// by NewForTest has a recorder of its own: its Metrics() serves a
+// request instead of panicking, and a second one does not collide
+// with the first.
+func TestMetrics_WorksOnNewForTestMiddleware(t *testing.T) {
+	t.Parallel()
+
+	log := slog.New(slog.DiscardHandler)
+	cfg := &config.Config{Environment: "prod"}
+	ok := http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
+		_, _ = w.Write([]byte(okBody))
+	})
+
+	for range 2 {
+		h := middleware.NewForTest(log, cfg, nil).Metrics()(ok)
+
+		req := httptest.NewRequestWithContext(
+			t.Context(), http.MethodGet, okRoute, nil,
+		)
+		w := httptest.NewRecorder()
+		h.ServeHTTP(w, req)
+
+		assert.Equal(t, http.StatusOK, w.Code)
+	}
+}

internal/middleware/middleware.go

@@ -13,6 +13,9 @@ import (
 	"github.com/go-chi/chi"
 	"github.com/go-chi/chi/middleware"
 	"github.com/go-chi/cors"
+	"github.com/prometheus/client_golang/prometheus"
+	httpmetrics "github.com/slok/go-http-metrics/metrics"
+	prommetrics "github.com/slok/go-http-metrics/metrics/prometheus"
 	"go.uber.org/fx"
 	"sneak.berlin/go/webhooker/internal/config"
 	"sneak.berlin/go/webhooker/internal/globals"
@@ -148,10 +151,11 @@ const (
 type MiddlewareParams struct {
 	fx.In
 
-	Logger  *logger.Logger
+	Logger   *logger.Logger
-	Globals *globals.Globals
+	Globals  *globals.Globals
-	Config  *config.Config
+	Config   *config.Config
-	Session *session.Session
+	Session  *session.Session
+	Registry *prometheus.Registry
 }
 
 // Middleware provides HTTP middleware for logging, CORS, auth, and
@@ -161,6 +165,14 @@ type Middleware struct {
 	params  *MiddlewareParams
 	session *session.Session
 
+	// metricsRecorder records the inbound HTTP metrics. New builds
+	// it on the registry /metrics serves, NewForTest on a registry
+	// of its own. Either way it is built once per Middleware and
+	// Metrics reuses it, because building it registers its
+	// collectors, and a second registration on the same registry
+	// panics.
+	metricsRecorder httpmetrics.Recorder
+
 	// loginGuard counts failed credential verifications and bounds
 	// concurrent password hashing. It is built on first use so that
 	// every construction path gets one; see guard().
@@ -179,6 +191,9 @@ func New(
 	s.params = &params
 	s.log = params.Logger.Get()
 	s.session = params.Session
+	s.metricsRecorder = prommetrics.NewRecorder(
+		prommetrics.Config{Registry: params.Registry},
+	)
 
 	return s, nil
 }

internal/middleware/testing.go

@@ -3,12 +3,17 @@ package middleware
 import (
 	"log/slog"
 
+	"github.com/prometheus/client_golang/prometheus"
+	prommetrics "github.com/slok/go-http-metrics/metrics/prometheus"
 	"sneak.berlin/go/webhooker/internal/config"
 	"sneak.berlin/go/webhooker/internal/session"
 )
 
 // NewForTest creates a Middleware with the minimum dependencies
 // needed for testing. This bypasses the fx lifecycle.
+//
+// Its metrics recorder writes to a fresh registry of its own, so
+// Metrics() works on it and two of them never collide.
 func NewForTest(
 	log *slog.Logger,
 	cfg *config.Config,
@@ -20,5 +25,8 @@ func NewForTest(
 			Config: cfg,
 		},
 		session: sess,
+		metricsRecorder: prommetrics.NewRecorder(
+			prommetrics.Config{Registry: prometheus.NewRegistry()},
+		),
 	}
 }

internal/resetpw/resetpw_test.go

@@ -24,6 +24,7 @@ import (
 	"sneak.berlin/go/webhooker/internal/handlers"
 	"sneak.berlin/go/webhooker/internal/healthcheck"
 	"sneak.berlin/go/webhooker/internal/logger"
+	"sneak.berlin/go/webhooker/internal/metrics"
 	"sneak.berlin/go/webhooker/internal/middleware"
 	"sneak.berlin/go/webhooker/internal/resetpw"
 	"sneak.berlin/go/webhooker/internal/session"
@@ -140,7 +141,7 @@ func (n *noopEvictor) EvictWebhook(string) {}
 // and the database, exactly as internal/handlers builds them.
 //
 // One application per test function, not per case: every start that
-// finds no account seeds one with an Argon2id hash, and this package's
+// finds no account seeds one at 64 MB of Argon2id, and this package's
 // budget is not the place to spend that repeatedly.
 func newServerApp(
 	t *testing.T, dir string,
@@ -163,6 +164,8 @@ func newServerApp(
 			session.New,
 			func() delivery.Notifier { return &noopNotifier{} },
 			func() delivery.WebhookEvictor { return &noopEvictor{} },
+			metrics.NewRegistry,
+			metrics.New,
 			middleware.New,
 			delivery.NewGuard,
 			handlers.New,

internal/server/routes.go

@@ -7,7 +7,6 @@ import (
 	sentryhttp "github.com/getsentry/sentry-go/http"
 	"github.com/go-chi/chi"
 	"github.com/go-chi/chi/middleware"
-	"github.com/prometheus/client_golang/prometheus/promhttp"
 	"sneak.berlin/go/webhooker/static"
 )
 
@@ -130,12 +129,7 @@ func (s *Server) setupRoutes() {
 	if s.params.Config.MetricsAuthEnabled() {
 		s.router.Group(func(r chi.Router) {
 			r.Use(s.mw.MetricsAuth())
-			r.Get(
+			r.Get("/metrics", s.h.HandleMetrics())
-				"/metrics",
-				http.HandlerFunc(
-					promhttp.Handler().ServeHTTP,
-				),
-			)
 		})
 	}
 

internal/server/routes_test.go

@@ -24,6 +24,7 @@ import (
 	"sneak.berlin/go/webhooker/internal/handlers"
 	"sneak.berlin/go/webhooker/internal/healthcheck"
 	"sneak.berlin/go/webhooker/internal/logger"
+	"sneak.berlin/go/webhooker/internal/metrics"
 	"sneak.berlin/go/webhooker/internal/middleware"
 	"sneak.berlin/go/webhooker/internal/server"
 	"sneak.berlin/go/webhooker/internal/session"
@@ -113,6 +114,8 @@ func newTestEnvWithConfig(
 			session.New,
 			func() delivery.Notifier { return &noopNotifier{} },
 			func() delivery.WebhookEvictor { return &noopEvictor{} },
+			metrics.NewRegistry,
+			metrics.New,
 			middleware.New,
 			delivery.NewGuard,
 			handlers.New,
@@ -1027,3 +1030,46 @@ func TestMetricsRouteUnmountedOnHalfSetConfig(t *testing.T) {
 		})
 	}
 }
+
+// TestTwoMetricsRoutersInOneProcess pins
+// https://git.eeqj.de/sneak/webhooker/issues/227: a second
+// metrics-enabled router in one process used to panic, because the
+// HTTP metrics registered on Prometheus's global default registry.
+// Two routers are built over separate dependency graphs and a third
+// over the first graph again, and each must still serve the HTTP,
+// delivery, Go runtime and process series, and the series counting
+// scrapes of /metrics itself.
+func TestTwoMetricsRoutersInOneProcess(t *testing.T) {
+	t.Parallel()
+
+	first := newTestEnvWithConfig(
+		t, metricsConfig(t, metricsUser, metricsAuthValue),
+	)
+	second := newTestEnvWithConfig(
+		t, metricsConfig(t, metricsUser, metricsAuthValue),
+	)
+	third := &testEnv{
+		router: server.NewRouterForTest(
+			first.log.Get(), first.cfg, first.mw, first.hnd,
+		),
+	}
+
+	for _, env := range []*testEnv{first, second, third} {
+		env.get("/", nil)
+
+		scrape := env.metricsRequest(metricsUser, metricsAuthValue)
+		require.Equal(t, http.StatusOK, scrape.Code)
+
+		for _, series := range []string{
+			"http_request_duration_seconds",
+			"http_response_size_bytes",
+			"http_requests_inflight",
+			"webhooker_events_received_total",
+			"go_goroutines",
+			"process_start_time_seconds",
+			"promhttp_metric_handler_requests_total",
+		} {
+			assert.Contains(t, scrape.Body.String(), series)
+		}
+	}
+}

internal/versionscript/version_script_test.go

@@ -115,8 +115,8 @@ func TestVersion_EnclosingRepositoryIsNotUsed(t *testing.T) {
 	require.Equal(t, unknown, runScript(t, inner, nil))
 }
 
-// An explicit VERSION, such as the Dockerfile's build arg, wins over
+// The Docker build has no git metadata, so the version arrives as an
-// anything derivable.
+// environment override. It wins over anything derivable.
 func TestVersion_EnvironmentOverrideWins(t *testing.T) {
 	t.Parallel()
 
@@ -128,8 +128,8 @@ func TestVersion_EnvironmentOverrideWins(t *testing.T) {
 }
 
 // An empty VERSION is treated as unset rather than stamping an empty
-// string: a caller exporting VERSION= must not produce a binary
+// string: the Dockerfile's build arg has a non-empty default, but a
-// reporting "".
+// caller exporting VERSION= must not produce a binary reporting "".
 func TestVersion_EmptyOverrideFallsBackToGit(t *testing.T) {
 	t.Parallel()
 
@@ -168,8 +168,8 @@ func TestMakefile_BuildComposesVersionAndExtraFlags(t *testing.T) {
 }
 
 // A caller can define VERSION as the empty string -- `make build
-// VERSION=`, or the Dockerfile's `make build VERSION="$VERSION"` when no
+// VERSION=`, or a `--build-arg VERSION=` reaching the Dockerfile's `make
-// VERSION build arg was given. script/version's own guard does not cover
+// build VERSION="$VERSION"`. script/version's own guard does not cover
 // that: the value never passes through the script. Stamping "" would
 // leave the binary reporting no version and the footer on "dev", which
 // is the defect this package exists for.
@@ -231,7 +231,7 @@ func TestDockerfile_BuildsThroughTheMakeTarget(t *testing.T) {
 
 	require.NotContains(t, dockerfile, "go build",
 		"a raw go build bypasses the Makefile's -X flag")
-	require.Contains(t, dockerfile, "ARG VERSION")
+	require.Contains(t, dockerfile, "ARG VERSION=")
 	require.Contains(t, dockerfile,
 		`make build VERSION="$VERSION" GO_LDFLAGS='-extldflags "-static"'`)
 }

script/docker

@@ -2,9 +2,9 @@
 # script/docker: build the Docker image tagged with the project name.
 # The tag comes from script/projectname.
 #
-# The version script/version resolves here goes in as the VERSION build
+# .dockerignore excludes .git/, so the builder stage cannot derive the
-# arg, which takes precedence over what the build would derive from the
+# version itself. It is resolved here, where the checkout is, and passed
-# .git in its context.
+# in as a build arg; without it the image would stamp itself "unknown".
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"

script/test

@@ -22,11 +22,6 @@
 # The one figure above 90s is GOMAXPROCS 1, a synthetic core floor rather than
 # a condition CI runs under. If a CPU-limited runner ever puts a real run near
 # 67s, that is the datum to revisit the org figure with.
-#
-# -p 4 -parallel 8 keep the run under 2 GB of memory: at most four test
-# binaries build or run at once, each with at most eight parallel tests. Under
-# -race every test binary and every link costs a few hundred MB, so the
-# defaults (one per core) add up to several GB on a many-core host.
 set -eu
 
 ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
@@ -34,7 +29,7 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
 main() {
     cd "$ROOT"
     "$ROOT/script/assets"
-    go test -v -race -p 4 -parallel 8 -timeout 90s ./...
+    go test -v -race -timeout 90s ./...
 }
 
 main "$@"

script/version

@@ -7,16 +7,18 @@
 #
 # Order of precedence:
 #
-#   1. $VERSION, if set and non-empty: an explicit value, such as the
+#   1. $VERSION, if set and non-empty. This is how the value reaches a
-#      Dockerfile's VERSION build arg.
+#      build that cannot derive it: .dockerignore excludes .git/, so the
+#      builder stage has no git metadata and the Dockerfile takes the
+#      value as a build arg instead.
 #   2. `git describe --tags --always --dirty` against this checkout. At
 #      a clean tagged commit that is exactly the tag; otherwise it
 #      carries the short SHA, the commit distance when a tag is
 #      reachable, and a -dirty suffix for uncommitted changes.
 #   3. "unknown", for a tree with no git metadata and no $VERSION -- a
-#      source tarball, or a `docker build` with no .git in its context
+#      source tarball, or `docker build .` with no --build-arg. That
-#      and no VERSION build arg. That case must not fail the build and
+#      case must not fail the build and must not name a tag the tree may
-#      must not name a tag the tree may not be at, so it names nothing.
+#      not be at, so it names nothing.
 #
 # The git step insists the enclosing repository is this checkout, not
 # merely some repository above it: an unpacked tarball sitting inside an