5 Commits
Author SHA1 Message Date
clawbot 128eb1b644 Keep test helpers out of the shipped binary (closes #506)
check / check (push) Waiting to run
The test helpers lived in ordinary `testing.go` files inside the config, database, middleware and session packages, so they were built into the binary and the shared `test-support` lint rule could not see them. The four files are gone: the session's helpers move into its own `_test.go` file, and the rest into `configtest`, `databasetest` and `middlewaretest`, which the `depguard` deny list now names, so a non-test file importing them fails lint. The test-support packages build through the production constructors.

Judgement call: the session, the middleware and the webhook database manager now take the plain logger they log through, which the application wiring provides.
Judgement call: two idle-expiry tests move the stored timestamps back instead of advancing a fake clock.

Model: opus-5-5
2026-10-06 14:29:47 +02:00
clawbot 3de345fe6f Read the export test's heap only after pools drop their caches (closes #511)
check / check (push) Waiting to run
`TestArchiveExport_Streams` read the heap after one garbage collection, but the libraries the export calls (`regexp` under GORM's table names, `encoding/json`, GORM's row scanning) keep spare buffers in a `sync.Pool`, which keeps them through one collection. Each reading therefore counted however many happened to be cached, which varied by about as much as the test's limit and failed `go test` on `next`. The test now collects twice before each reading, so a reading is the memory the export holds. The limit, the row counts and the claim are unchanged, and an export that does not stream still fails it.

Unverified: the branch's CI run waits for the runner outage to clear.

Model: opus-5-5
2026-10-06 10:56:50 +02:00
clawbot a9d77e20d7 Make the load-sensitive tests wait for what they check (closes #507)
check / check (push) Successful in 9m48s
Three tests failed at random on a busy host. The browser test now waits for each page a click opens to load, and for Alpine.js to start on it, before reading it. The delivery tests' drain takes what is already queued instead of racing a 25 ms timer, since both dispatch paths queue before they return. The test phase keeps the tests' temporary directories on a tmpfs, because SQLite waiting for the disk made `internal/handlers` slow under load; the timeout and the parallel cap are unchanged.

Deviation: the delivery tests do not wait with a deadline; dispatch has finished when they read.
Unverified: `internal/handlers` at a host load of 260 to 290.

Model: opus-5-5
2026-10-06 08:51:35 +02:00
clawbot faf3da9a75 Refresh the package lists before installing in script/bootstrap (closes #508)
check / check (push) Successful in 9m44s
The CI runner's image starts with empty package lists, so `script/bootstrap`, which `script/cibuild` now runs first, failed to install Go and every CI run stopped there. The apt branch of `pkg_install` now runs `apt-get update` once per run, before its first `apt-get install`, with the same code as the pending shared fix in sneak/prompts#116.

Deviation: `script/bootstrap` differs from the vendored copy until the next re-vendor.
Unverified: proven by running the CI job in the runner's image; the branch's own CI run waits for the runner outage to clear.

Model: opus-5-5
2026-10-06 07:21:30 +02:00
clawbot fa6a9ed4dc Re-vendor the shared files from sneak/prompts at dd4027b (closes #504)
check / check (push) Failing after 5s
The shared workflow, lint config, prettier settings and policies are the copies at `sneak/prompts` commit `dd4027b`. `.gitignore`, `.editorconfig` and `.dockerignore` are the shared copy followed by this repository's own entries. Linting is the image build's lint phase on golangci-lint v2.14.0, and tests run in their own test phase. Every scripted `docker build` passes `--no-cache`, so the CI fingerprint step and the superseded-run script are gone. The binary is built with `-trimpath -s -w`, and a build that has `.git` but no version fails. The development run keeps its databases outside the checkout.

Deviation: `.dockerignore` also leaves out SQLite databases at any depth.

Model: opus-5-5
2026-10-06 06:05:42 +02:00
47 changed files with 444 additions and 347 deletions
+6
View File
@@ -76,3 +76,9 @@
# build extracts its own). # build extracts its own).
/bin /bin
/static/js/alpine.min.js /static/js/alpine.min.js
# SQLite databases, which hold the session key and webhook payloads, at
# any depth.
**/*.db
**/*.sqlite
**/*.sqlite3
+3
View File
@@ -10,3 +10,6 @@ insert_final_newline = true
[Makefile] [Makefile]
indent_style = tab indent_style = tab
[*.go]
indent_style = tab
+36
View File
@@ -45,3 +45,39 @@ node_modules/
[iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK] [iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK]
[iI][dD]_[eE][dD]25519 [iI][dD]_[eE][dD]25519
[iI][dD]_[eE][dD]25519_[sS][kK] [iI][dD]_[eE][dD]25519_[sS][kK]
# This repository's own entries, after the shared content above.
# Binaries
*.exe
*.dll
*.so
*.dylib
bin/
/webhooker
# Test binary, built with `go test -c`
*.test
# Output of the go coverage tool
*.out
# Go vendor directory
vendor/
# Data directory (SQLite databases)
data/
*.db
*.sqlite
*.sqlite3
# Log files
*.log
# Temporary files
tmp/
temp/
# Alpine.js, extracted by `make assets` from its tarball in 3p/, which is
# what is committed.
/static/js/alpine.min.js
+6
View File
@@ -61,6 +61,12 @@ linters:
desc: >- desc: >-
Test-support code belongs in test files and in packages whose Test-support code belongs in test files and in packages whose
directory name ends in test, not in the shipped binary. directory name ends in test, not in the shipped binary.
- pkg: sneak.berlin/go/webhooker/internal/config/configtest
desc: test support; a file that is not a test must not import it
- pkg: sneak.berlin/go/webhooker/internal/database/databasetest
desc: test support; a file that is not a test must not import it
- pkg: sneak.berlin/go/webhooker/internal/middleware/middlewaretest
desc: test support; a file that is not a test must not import it
# Only decisions already recorded in the Go package defaults are # Only decisions already recorded in the Go package defaults are
# listed here. Every entry matches the module path exactly. # listed here. Every entry matches the module path exactly.
gomodguard_v2: gomodguard_v2:
+8 -1
View File
@@ -147,9 +147,16 @@ RUN script/assets
# shown in full, so there is nothing to rerun. The step fails after the rerun # shown in full, so there is nothing to rerun. The step fails after the rerun
# whatever its result: the first run already showed the suite is broken. # whatever its result: the first run already showed the suite is broken.
# #
# TMPDIR, where the tests keep their SQLite databases, is a tmpfs: SQLite
# waits for the disk at every commit, and on a busy host that waiting was
# about 40% of the slowest package's run time. GOTMPDIR keeps go's own
# build files, the test binaries among them, on disk.
#
# bash with pipefail, so that the first run's status is go test's, not tee's. # bash with pipefail, so that the first run's status is go test's, not tee's.
SHELL ["/bin/bash", "-o", "pipefail", "-c"] SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN go test -race -cover -p 4 -parallel 8 -timeout 90s ./... 2>&1 | tee /tmp/go-test.log && exit 0; \ RUN --mount=type=tmpfs,target=/tmp/tests,size=512m \
export TMPDIR=/tmp/tests GOTMPDIR=/tmp; \
go test -race -cover -p 4 -parallel 8 -timeout 90s ./... 2>&1 | tee /tmp/go-test.log && exit 0; \
tests="$(awk '/^--- FAIL: / { print $3 }' /tmp/go-test.log | paste -s -d '|' -)"; \ tests="$(awk '/^--- FAIL: / { print $3 }' /tmp/go-test.log | paste -s -d '|' -)"; \
packages="$(awk '/^FAIL\t/ { print $2 }' /tmp/go-test.log)"; \ packages="$(awk '/^FAIL\t/ { print $2 }' /tmp/go-test.log)"; \
if [ -n "$tests" ]; then \ if [ -n "$tests" ]; then \
+6 -7
View File
@@ -44,7 +44,7 @@ make check
# Run the server from the clone. DATA_DIR defaults to # Run the server from the clone. DATA_DIR defaults to
# /var/lib/webhooker in every environment, so set it (in .env or the # /var/lib/webhooker in every environment, so set it (in .env or the
# shell) to a writable directory outside the clone: the databases hold # shell) to a writable directory outside the clone: the databases hold
# the session key, and git does not ignore them. # the session key.
DATA_DIR=../webhooker-data make dev DATA_DIR=../webhooker-data make dev
# Build Docker image # Build Docker image
@@ -2984,7 +2984,7 @@ webhooker/
│ │ └── resetpw.go # `webhooker resetpw`: set an account's password, stopped deployments only │ │ └── resetpw.go # `webhooker resetpw`: set an account's password, stopped deployments only
│ ├── config/ │ ├── config/
│ │ ├── config.go # Configuration loading from environment variables │ │ ├── config.go # Configuration loading from environment variables
│ │ └── testing.go # ClearEnvForTest: an empty environment for one test │ │ └── configtest/ # Test support: ClearEnv, an empty environment for one test
│ ├── database/ │ ├── database/
│ │ ├── base_model.go # BaseModel with UUID primary keys │ │ ├── base_model.go # BaseModel with UUID primary keys
│ │ ├── database.go # GORM connection, migrations, admin seed │ │ ├── database.go # GORM connection, migrations, admin seed
@@ -3001,8 +3001,8 @@ webhooker/
│ │ ├── model_apikey.go # APIKey entity │ │ ├── model_apikey.go # APIKey entity
│ │ ├── password.go # Argon2id hashing and verification │ │ ├── password.go # Argon2id hashing and verification
│ │ ├── retention.go # Retention reaper (per-webhook event expiry) │ │ ├── retention.go # Retention reaper (per-webhook event expiry)
│ │ ├── testing.go # NewTestDatabase: wrapper for tests, no fx lifecycle │ │ ├── webhook_db_manager.go # Per-webhook DB lifecycle manager
│ │ └── webhook_db_manager.go # Per-webhook DB lifecycle manager │ │ └── databasetest/ # Test support: a WebhookDBManager for tests in other packages
│ ├── datadir/ │ ├── datadir/
│ │ └── lock.go # Exclusive advisory lock on DATA_DIR (one instance) │ │ └── lock.go # Exclusive advisory lock on DATA_DIR (one instance)
│ ├── globals/ │ ├── globals/
@@ -3063,7 +3063,7 @@ webhooker/
│ │ ├── csrf.go # CSRF protection middleware (gorilla/csrf) │ │ ├── csrf.go # CSRF protection middleware (gorilla/csrf)
│ │ ├── ratelimit.go # Per-IP rate limiting middleware (go-chi/httprate) │ │ ├── ratelimit.go # Per-IP rate limiting middleware (go-chi/httprate)
│ │ ├── loginguard.go # Login failure counters and the Argon2id verification semaphore │ │ ├── loginguard.go # Login failure counters and the Argon2id verification semaphore
│ │ └── testing.go # NewForTest: Middleware without the fx lifecycle │ │ └── middlewaretest/ # Test support: a Middleware for tests in other packages
│ ├── reqtls/ │ ├── reqtls/
│ │ └── reqtls.go # IsTLS: the one TLS predicate, r.TLS or X-Forwarded-Proto │ │ └── reqtls.go # IsTLS: the one TLS predicate, r.TLS or X-Forwarded-Proto
│ ├── server/ │ ├── server/
@@ -3071,8 +3071,7 @@ webhooker/
│ │ ├── http.go # HTTP server setup with timeouts │ │ ├── http.go # HTTP server setup with timeouts
│ │ └── routes.go # All route definitions │ │ └── routes.go # All route definitions
│ ├── session/ │ ├── session/
│ │ ├── session.go # Cookie-based session management │ │ └── session.go # Cookie-based session management
│ │ └── testing.go # NewForTest: Session without the fx lifecycle
│ └── versionscript/ │ └── versionscript/
│ └── doc.go # Tests for script/version and the build files that use it │ └── doc.go # Tests for script/version and the build files that use it
├── static/ ├── static/
+4
View File
@@ -4,6 +4,7 @@ package main
import ( import (
"fmt" "fmt"
"io" "io"
"log/slog"
"os" "os"
"time" "time"
@@ -187,6 +188,9 @@ func newApp() *fx.App {
fx.Provide( fx.Provide(
globals.New, globals.New,
logger.New, logger.New,
// The plain logger the session, the middleware and the
// webhook database manager take.
func(l *logger.Logger) *slog.Logger { return l.Get() },
config.New, config.New,
database.New, database.New,
database.NewWebhookDBManager, database.NewWebhookDBManager,
+3 -3
View File
@@ -14,7 +14,7 @@ import (
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config/configtest"
"sneak.berlin/go/webhooker/internal/datadir" "sneak.berlin/go/webhooker/internal/datadir"
"sneak.berlin/go/webhooker/internal/resetpw" "sneak.berlin/go/webhooker/internal/resetpw"
"sneak.berlin/go/webhooker/internal/server" "sneak.berlin/go/webhooker/internal/server"
@@ -37,7 +37,7 @@ const dockerStopGrace = 10 * time.Second
// fx.New applies options before it executes invokes, so the timeout // fx.New applies options before it executes invokes, so the timeout
// is set whether or not the graph itself can be constructed here. // is set whether or not the graph itself can be constructed here.
func TestNewApp_StopTimeout(t *testing.T) { func TestNewApp_StopTimeout(t *testing.T) {
config.ClearEnvForTest(t) configtest.ClearEnv(t)
t.Setenv("DATA_DIR", t.TempDir()) t.Setenv("DATA_DIR", t.TempDir())
got := newApp().StopTimeout() got := newApp().StopTimeout()
@@ -75,7 +75,7 @@ func freePort(t *testing.T) int {
// anything is built, and the run of logger.New, which happens before // anything is built, and the run of logger.New, which happens before
// the configuration sets the level. // the configuration sets the level.
func TestNewApp_SendsFxEventsToTheLogger(t *testing.T) { func TestNewApp_SendsFxEventsToTheLogger(t *testing.T) {
config.ClearEnvForTest(t) configtest.ClearEnv(t)
t.Setenv("DATA_DIR", t.TempDir()) t.Setenv("DATA_DIR", t.TempDir())
t.Setenv("PORT", strconv.Itoa(freePort(t))) t.Setenv("PORT", strconv.Itoa(freePort(t)))
t.Setenv("DEBUG", "true") t.Setenv("DEBUG", "true")
@@ -7,21 +7,22 @@ import (
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/config/configtest"
) )
// TestClearEnvForTest_RemovesAddedVariables pins that a variable set // TestClearEnv_RemovesAddedVariables pins that a variable set
// after the clear other than through t.Setenv, as a test's .env file // after the clear other than through t.Setenv, as a test's .env file
// sets one, is gone once the test ends, so it cannot reach the tests // sets one, is gone once the test ends, so it cannot reach the tests
// that run after it. // that run after it.
// //
//nolint:paralleltest // ClearEnvForTest uses t.Setenv. //nolint:paralleltest // ClearEnv uses t.Setenv.
func TestClearEnvForTest_RemovesAddedVariables(t *testing.T) { func TestClearEnv_RemovesAddedVariables(t *testing.T) {
// The outer clear keeps a value of the key exported in the shell // The outer clear keeps a value of the key exported in the shell
// from making it a variable the inner clear has to put back. // from making it a variable the inner clear has to put back.
config.ClearEnvForTest(t) configtest.ClearEnv(t)
t.Run("loads a .env file after the clear", func(t *testing.T) { t.Run("loads a .env file after the clear", func(t *testing.T) {
config.ClearEnvForTest(t) configtest.ClearEnv(t)
path := writeDotEnv(t, dotEnvKey+"=from-dot-env\n") path := writeDotEnv(t, dotEnvKey+"=from-dot-env\n")
require.NoError(t, config.LoadDotEnvFileForTest(path)) require.NoError(t, config.LoadDotEnvFileForTest(path))
+11 -10
View File
@@ -11,6 +11,7 @@ import (
"go.uber.org/fx" "go.uber.org/fx"
"go.uber.org/fx/fxtest" "go.uber.org/fx/fxtest"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/config/configtest"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
) )
@@ -70,7 +71,7 @@ func TestEnvironmentConfig(t *testing.T) {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv // Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests. // is incompatible with parallel subtests.
config.ClearEnvForTest(t) configtest.ClearEnv(t)
if tt.envValue != "" { if tt.envValue != "" {
t.Setenv( t.Setenv(
@@ -196,7 +197,7 @@ func TestRetentionSweepInterval(t *testing.T) {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv // Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests. // is incompatible with parallel subtests.
config.ClearEnvForTest(t) configtest.ClearEnv(t)
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev") t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
if tt.set { if tt.set {
@@ -335,7 +336,7 @@ func TestSessionIdleTimeout(t *testing.T) {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv // Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests. // is incompatible with parallel subtests.
config.ClearEnvForTest(t) configtest.ClearEnv(t)
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev") t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
if tt.set { if tt.set {
@@ -388,7 +389,7 @@ func TestDefaultDataDir(t *testing.T) {
t.Run("env="+name, func(t *testing.T) { t.Run("env="+name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv // Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests. // is incompatible with parallel subtests.
config.ClearEnvForTest(t) configtest.ClearEnv(t)
if env != "" { if env != "" {
t.Setenv("WEBHOOKER_ENVIRONMENT", env) t.Setenv("WEBHOOKER_ENVIRONMENT", env)
@@ -433,7 +434,7 @@ func TestDataDirHelper(t *testing.T) {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv // Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests. // is incompatible with parallel subtests.
config.ClearEnvForTest(t) configtest.ClearEnv(t)
if set != "" { if set != "" {
t.Setenv("DATA_DIR", set) t.Setenv("DATA_DIR", set)
@@ -498,7 +499,7 @@ func TestReceiverRateLimit(t *testing.T) {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv // Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests. // is incompatible with parallel subtests.
config.ClearEnvForTest(t) configtest.ClearEnv(t)
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev") t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
if tt.set { if tt.set {
@@ -614,7 +615,7 @@ func TestTrustedProxies(t *testing.T) {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv // Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests. // is incompatible with parallel subtests.
config.ClearEnvForTest(t) configtest.ClearEnv(t)
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev") t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
if tt.set { if tt.set {
@@ -725,7 +726,7 @@ func TestAllowedEgressCIDRs(t *testing.T) {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv // Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests. // is incompatible with parallel subtests.
config.ClearEnvForTest(t) configtest.ClearEnv(t)
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev") t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
if tt.set { if tt.set {
@@ -797,7 +798,7 @@ func TestEgressAllowlistWarning(t *testing.T) {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv // Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests. // is incompatible with parallel subtests.
config.ClearEnvForTest(t) configtest.ClearEnv(t)
t.Setenv("WEBHOOKER_ENVIRONMENT", config.EnvironmentDev) t.Setenv("WEBHOOKER_ENVIRONMENT", config.EnvironmentDev)
if tt.allowed != "" { if tt.allowed != "" {
@@ -933,7 +934,7 @@ func TestMetricsAuthConfig(t *testing.T) {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv // Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests. // is incompatible with parallel subtests.
config.ClearEnvForTest(t) configtest.ClearEnv(t)
if tt.username.set { if tt.username.set {
t.Setenv("METRICS_USERNAME", tt.username.value) t.Setenv("METRICS_USERNAME", tt.username.value)
@@ -1,4 +1,6 @@
package config // Package configtest holds test support for code that reads the
// process environment.
package configtest
import ( import (
"os" "os"
@@ -6,12 +8,12 @@ import (
"testing" "testing"
) )
// ClearEnvForTest unsets every variable in the process environment // ClearEnv unsets every variable in the process environment
// for the rest of the test, so a test sees only the variables it sets // for the rest of the test, so a test sees only the variables it sets
// itself, not whatever the developer's shell exports. When the test // itself, not whatever the developer's shell exports. When the test
// ends it leaves the environment exactly as it found it: each variable // ends it leaves the environment exactly as it found it: each variable
// it unset is put back, and any variable added since is removed. // it unset is put back, and any variable added since is removed.
func ClearEnvForTest(t *testing.T) { func ClearEnv(t *testing.T) {
t.Helper() t.Helper()
present := make(map[string]bool) present := make(map[string]bool)
+8 -7
View File
@@ -8,6 +8,7 @@ import (
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/config/configtest"
) )
// dotEnvKey is a throwaway variable name the .env tests write and // dotEnvKey is a throwaway variable name the .env tests write and
@@ -39,9 +40,9 @@ func writeDotEnv(t *testing.T, contents string) string {
// normally rather than be refused for a file it was never meant to // normally rather than be refused for a file it was never meant to
// have. // have.
// //
//nolint:paralleltest // ClearEnvForTest uses t.Setenv. //nolint:paralleltest // ClearEnv uses t.Setenv.
func TestLoadDotEnv_MissingFileIsFine(t *testing.T) { func TestLoadDotEnv_MissingFileIsFine(t *testing.T) {
config.ClearEnvForTest(t) configtest.ClearEnv(t)
absent := filepath.Join(t.TempDir(), config.DotEnvPath) absent := filepath.Join(t.TempDir(), config.DotEnvPath)
require.NoError(t, config.LoadDotEnvFileForTest(absent)) require.NoError(t, config.LoadDotEnvFileForTest(absent))
@@ -54,9 +55,9 @@ func TestLoadDotEnv_MissingFileIsFine(t *testing.T) {
// reaches the environment, which is the whole reason the file is read // reaches the environment, which is the whole reason the file is read
// at all. // at all.
// //
//nolint:paralleltest // ClearEnvForTest uses t.Setenv. //nolint:paralleltest // ClearEnv uses t.Setenv.
func TestLoadDotEnv_AppliesValues(t *testing.T) { func TestLoadDotEnv_AppliesValues(t *testing.T) {
config.ClearEnvForTest(t) configtest.ClearEnv(t)
path := writeDotEnv(t, "# a comment\n"+dotEnvKey+"=from-dot-env\n") path := writeDotEnv(t, "# a comment\n"+dotEnvKey+"=from-dot-env\n")
@@ -82,9 +83,9 @@ func TestLoadDotEnv_RealEnvironmentWins(t *testing.T) {
// reverts to its default; the process used to start that way with no // reverts to its default; the process used to start that way with no
// log line naming the file at all. // log line naming the file at all.
// //
//nolint:paralleltest // ClearEnvForTest uses t.Setenv. //nolint:paralleltest // ClearEnv uses t.Setenv.
func TestLoadDotEnv_MalformedFileAborts(t *testing.T) { func TestLoadDotEnv_MalformedFileAborts(t *testing.T) {
config.ClearEnvForTest(t) configtest.ClearEnv(t)
path := writeDotEnv( path := writeDotEnv(
t, malformedDotEnv+dotEnvKey+"=from-dot-env\n", t, malformedDotEnv+dotEnvKey+"=from-dot-env\n",
@@ -132,7 +133,7 @@ func TestLoadDotEnv_UnreadableFileAborts(t *testing.T) {
// //
//nolint:paralleltest // t.Chdir moves the whole process. //nolint:paralleltest // t.Chdir moves the whole process.
func TestLoadDotEnv_ReadsTheWorkingDirectory(t *testing.T) { func TestLoadDotEnv_ReadsTheWorkingDirectory(t *testing.T) {
config.ClearEnvForTest(t) configtest.ClearEnv(t)
dir := t.TempDir() dir := t.TempDir()
require.NoError(t, os.WriteFile( require.NoError(t, os.WriteFile(
+6 -5
View File
@@ -7,6 +7,7 @@ import (
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"go.uber.org/fx" "go.uber.org/fx"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/config/configtest"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
) )
@@ -120,7 +121,7 @@ func TestEnvBool(t *testing.T) {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv // Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests. // is incompatible with parallel subtests.
config.ClearEnvForTest(t) configtest.ClearEnv(t)
if tt.set { if tt.set {
t.Setenv(testEnvKey, tt.value) t.Setenv(testEnvKey, tt.value)
@@ -169,7 +170,7 @@ func runEnvIntCases(
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv // Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests. // is incompatible with parallel subtests.
config.ClearEnvForTest(t) configtest.ClearEnv(t)
if tt.set { if tt.set {
t.Setenv(testEnvKey, tt.value) t.Setenv(testEnvKey, tt.value)
@@ -310,7 +311,7 @@ func TestEnvBindAddress(t *testing.T) {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv // Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests. // is incompatible with parallel subtests.
config.ClearEnvForTest(t) configtest.ClearEnv(t)
if tt.set { if tt.set {
t.Setenv(testEnvKey, tt.value) t.Setenv(testEnvKey, tt.value)
@@ -476,7 +477,7 @@ func TestNewRejectsBadEnvValues(t *testing.T) {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv // Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests. // is incompatible with parallel subtests.
config.ClearEnvForTest(t) configtest.ClearEnv(t)
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev") t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
t.Setenv(tt.key, tt.value) t.Setenv(tt.key, tt.value)
@@ -638,7 +639,7 @@ func sentryEnvValueCases() []badEnvValueCase {
// break the legitimate unset case: absent variables still get their // break the legitimate unset case: absent variables still get their
// documented defaults. // documented defaults.
func TestNewUsesDefaultsWhenUnset(t *testing.T) { func TestNewUsesDefaultsWhenUnset(t *testing.T) {
config.ClearEnvForTest(t) configtest.ClearEnv(t)
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev") t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
cfg, err := buildConfig(t) cfg, err := buildConfig(t)
+2 -1
View File
@@ -6,6 +6,7 @@ import (
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/config/configtest"
) )
// envKeySentryDSN is the variable envSentryDSN reads in production. // envKeySentryDSN is the variable envSentryDSN reads in production.
@@ -100,7 +101,7 @@ func TestEnvSentryDSN(t *testing.T) {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv // Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests. // is incompatible with parallel subtests.
config.ClearEnvForTest(t) configtest.ClearEnv(t)
if tt.set { if tt.set {
t.Setenv(envKeySentryDSN, tt.value) t.Setenv(envKeySentryDSN, tt.value)
@@ -0,0 +1,55 @@
// Package databasetest builds a WebhookDBManager for tests in other
// packages.
package databasetest
import (
"log/slog"
"os"
"testing"
"github.com/stretchr/testify/require"
"go.uber.org/fx/fxtest"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database"
)
// NewWebhookDBManager creates a WebhookDBManager backed by the given
// data directory, logging at DEBUG to standard error.
func NewWebhookDBManager(
t *testing.T, dataDir string,
) *database.WebhookDBManager {
t.Helper()
return NewWebhookDBManagerWithLogger(
t,
dataDir,
slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
)),
)
}
// NewWebhookDBManagerWithLogger is NewWebhookDBManager with the
// logger supplied by the caller. The per-webhook databases this manager
// opens hand that logger to gormlog, so a test that needs to see the SQL
// the service emits can capture it.
//
// It is built through database.NewWebhookDBManager on a lifecycle that
// is never started, so nothing closes its databases but the caller.
func NewWebhookDBManagerWithLogger(
t *testing.T, dataDir string, log *slog.Logger,
) *database.WebhookDBManager {
t.Helper()
mgr, err := database.NewWebhookDBManager(
fxtest.NewLifecycle(t),
database.WebhookDBManagerParams{
Config: &config.Config{DataDir: dataDir},
Logger: log,
},
)
require.NoError(t, err)
return mgr
}
+12 -11
View File
@@ -13,6 +13,7 @@ import (
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
_ "modernc.org/sqlite" _ "modernc.org/sqlite"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/database/databasetest"
) )
// testDataDirPerm is the mode the test data directory is created // testDataDirPerm is the mode the test data directory is created
@@ -133,7 +134,7 @@ func TestOpenPurgesLeakedTargetRows(t *testing.T) {
// Create the file the way the application does, so the targets // Create the file the way the application does, so the targets
// table has exactly the shape AutoMigrate gives it, then write // table has exactly the shape AutoMigrate gives it, then write
// a leaked row into it the way the association upsert did. // a leaked row into it the way the association upsert did.
initial := database.NewTestWebhookDBManager(dataDir) initial := databasetest.NewWebhookDBManager(t, dataDir)
_, err := initial.GetDB(webhookID) _, err := initial.GetDB(webhookID)
require.NoError(t, err) require.NoError(t, err)
@@ -156,7 +157,7 @@ func TestOpenPurgesLeakedTargetRows(t *testing.T) {
clearEventDBSweptMarker(t, seed) clearEventDBSweptMarker(t, seed)
require.NoError(t, seed.Close()) require.NoError(t, seed.Close())
mgr := database.NewTestWebhookDBManager(dataDir) mgr := databasetest.NewWebhookDBManager(t, dataDir)
_, err = mgr.GetDB(webhookID) _, err = mgr.GetDB(webhookID)
require.NoError(t, err) require.NoError(t, err)
@@ -172,7 +173,7 @@ func TestOpenPurgesLeakedTargetRows(t *testing.T) {
// Idempotent: a second open leaves it at zero and does not // Idempotent: a second open leaves it at zero and does not
// error. // error.
again := database.NewTestWebhookDBManager(dataDir) again := databasetest.NewWebhookDBManager(t, dataDir)
_, err = again.GetDB(webhookID) _, err = again.GetDB(webhookID)
require.NoError(t, err) require.NoError(t, err)
@@ -195,7 +196,7 @@ func TestOpenPurgeRemovesCredentialBytes(t *testing.T) {
webhookID := uuid.New().String() webhookID := uuid.New().String()
credential := "T00000000/B00000000/" + uuid.New().String() credential := "T00000000/B00000000/" + uuid.New().String()
initial := database.NewTestWebhookDBManager(dataDir) initial := databasetest.NewWebhookDBManager(t, dataDir)
_, err := initial.GetDB(webhookID) _, err := initial.GetDB(webhookID)
require.NoError(t, err) require.NoError(t, err)
@@ -230,7 +231,7 @@ func TestOpenPurgeRemovesCredentialBytes(t *testing.T) {
"seeded credential is not in the file, so this test proves nothing", "seeded credential is not in the file, so this test proves nothing",
) )
mgr := database.NewTestWebhookDBManager(dataDir) mgr := databasetest.NewWebhookDBManager(t, dataDir)
_, err = mgr.GetDB(webhookID) _, err = mgr.GetDB(webhookID)
require.NoError(t, err) require.NoError(t, err)
@@ -258,7 +259,7 @@ func TestOpenRevacuumsAfterIncompleteSweep(t *testing.T) {
webhookID := uuid.New().String() webhookID := uuid.New().String()
credential := "T00000000/B00000000/" + uuid.New().String() credential := "T00000000/B00000000/" + uuid.New().String()
initial := database.NewTestWebhookDBManager(dataDir) initial := databasetest.NewWebhookDBManager(t, dataDir)
_, err := initial.GetDB(webhookID) _, err := initial.GetDB(webhookID)
require.NoError(t, err) require.NoError(t, err)
@@ -298,7 +299,7 @@ func TestOpenRevacuumsAfterIncompleteSweep(t *testing.T) {
"test proves nothing", "test proves nothing",
) )
mgr := database.NewTestWebhookDBManager(dataDir) mgr := databasetest.NewWebhookDBManager(t, dataDir)
_, err = mgr.GetDB(webhookID) _, err = mgr.GetDB(webhookID)
require.NoError(t, err) require.NoError(t, err)
@@ -325,7 +326,7 @@ func TestOpenSkipsSweptDatabase(t *testing.T) {
dataDir := eventDBDataDir(t) dataDir := eventDBDataDir(t)
webhookID := uuid.New().String() webhookID := uuid.New().String()
mgr := database.NewTestWebhookDBManager(dataDir) mgr := databasetest.NewWebhookDBManager(t, dataDir)
_, err := mgr.GetDB(webhookID) _, err := mgr.GetDB(webhookID)
require.NoError(t, err) require.NoError(t, err)
@@ -347,7 +348,7 @@ func TestOpenSkipsSweptDatabase(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.NoError(t, marked.Close()) require.NoError(t, marked.Close())
again := database.NewTestWebhookDBManager(dataDir) again := databasetest.NewWebhookDBManager(t, dataDir)
_, err = again.GetDB(webhookID) _, err = again.GetDB(webhookID)
require.NoError(t, err) require.NoError(t, err)
@@ -378,7 +379,7 @@ func TestOpenSucceedsWithoutTargetsTable(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.NoError(t, seed.Close()) require.NoError(t, seed.Close())
mgr := database.NewTestWebhookDBManager(dataDir) mgr := databasetest.NewWebhookDBManager(t, dataDir)
db, err := mgr.GetDB(webhookID) db, err := mgr.GetDB(webhookID)
require.NoError(t, err) require.NoError(t, err)
@@ -396,7 +397,7 @@ func TestEventDBCreateOmitsAssociations(t *testing.T) {
dataDir := eventDBDataDir(t) dataDir := eventDBDataDir(t)
webhookID := uuid.New().String() webhookID := uuid.New().String()
mgr := database.NewTestWebhookDBManager(dataDir) mgr := databasetest.NewWebhookDBManager(t, dataDir)
db, err := mgr.GetDB(webhookID) db, err := mgr.GetDB(webhookID)
require.NoError(t, err) require.NoError(t, err)
+1 -1
View File
@@ -51,7 +51,7 @@ func setupRetentionTest(t *testing.T) *retentionTestEnv {
mgr, err := database.NewWebhookDBManager( mgr, err := database.NewWebhookDBManager(
lc, lc,
database.WebhookDBManagerParams{Config: cfg, Logger: l}, database.WebhookDBManagerParams{Config: cfg, Logger: l.Get()},
) )
require.NoError(t, err) require.NoError(t, err)
-47
View File
@@ -1,47 +0,0 @@
package database
import (
"log/slog"
"os"
"gorm.io/gorm"
)
// NewTestDatabase creates a Database wrapper around a pre-opened *gorm.DB.
// Intended for use in tests that need a *database.Database without the
// full fx lifecycle. The caller is responsible for closing the underlying
// sql.DB connection.
func NewTestDatabase(db *gorm.DB) *Database {
return &Database{
db: db,
log: slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
)),
}
}
// NewTestWebhookDBManager creates a WebhookDBManager backed by the given
// data directory. Intended for use in tests without the fx lifecycle.
func NewTestWebhookDBManager(dataDir string) *WebhookDBManager {
return NewTestWebhookDBManagerWithLogger(
dataDir,
slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
)),
)
}
// NewTestWebhookDBManagerWithLogger is NewTestWebhookDBManager with the
// logger supplied by the caller. The per-webhook databases this manager
// opens hand that logger to gormlog, so a test that needs to see the SQL
// the service emits can capture it.
func NewTestWebhookDBManagerWithLogger(
dataDir string, log *slog.Logger,
) *WebhookDBManager {
return &WebhookDBManager{
dataDir: dataDir,
log: log,
}
}
+2 -3
View File
@@ -15,7 +15,6 @@ import (
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/datadir" "sneak.berlin/go/webhooker/internal/datadir"
"sneak.berlin/go/webhooker/internal/gormlog" "sneak.berlin/go/webhooker/internal/gormlog"
"sneak.berlin/go/webhooker/internal/logger"
) )
// WebhookDBManagerParams holds the fx dependencies for // WebhookDBManagerParams holds the fx dependencies for
@@ -24,7 +23,7 @@ type WebhookDBManagerParams struct {
fx.In fx.In
Config *config.Config Config *config.Config
Logger *logger.Logger Logger *slog.Logger
} }
// errInvalidCachedDBType indicates a type assertion failure // errInvalidCachedDBType indicates a type assertion failure
@@ -70,7 +69,7 @@ func NewWebhookDBManager(
) (*WebhookDBManager, error) { ) (*WebhookDBManager, error) {
m := &WebhookDBManager{ m := &WebhookDBManager{
dataDir: params.Config.DataDir, dataDir: params.Config.DataDir,
log: params.Logger.Get(), log: params.Logger,
} }
// Create data directory if it doesn't exist. datadir.DirPerm is the // Create data directory if it doesn't exist. datadir.DirPerm is the
+6 -3
View File
@@ -18,6 +18,7 @@ import (
"gorm.io/gorm" "gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/database/databasetest"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
) )
@@ -50,7 +51,7 @@ func setupTestWebhookDBManager(
lc, lc,
database.WebhookDBManagerParams{ database.WebhookDBManagerParams{
Config: cfg, Config: cfg,
Logger: l, Logger: l.Get(),
}, },
) )
require.NoError(t, err) require.NoError(t, err)
@@ -117,7 +118,8 @@ func TestWebhookDBManager_ConcurrentFirstTouchOpensOnce(t *testing.T) {
var logs bytes.Buffer var logs bytes.Buffer
mgr := database.NewTestWebhookDBManagerWithLogger( mgr := databasetest.NewWebhookDBManagerWithLogger(
t,
t.TempDir(), t.TempDir(),
slog.New(slog.NewTextHandler(&logs, nil)), slog.New(slog.NewTextHandler(&logs, nil)),
) )
@@ -307,7 +309,8 @@ func TestWebhookDBManager_LostDatabaseIsLogged(t *testing.T) {
var logs bytes.Buffer var logs bytes.Buffer
mgr := database.NewTestWebhookDBManagerWithLogger( mgr := databasetest.NewWebhookDBManagerWithLogger(
t,
t.TempDir(), t.TempDir(),
slog.New(slog.NewTextHandler(&logs, nil)), slog.New(slog.NewTextHandler(&logs, nil)),
) )
+4 -18
View File
@@ -21,6 +21,7 @@ import (
"gorm.io/gorm/clause" "gorm.io/gorm/clause"
_ "modernc.org/sqlite" // Pure Go SQLite driver. _ "modernc.org/sqlite" // Pure Go SQLite driver.
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/database/databasetest"
"sneak.berlin/go/webhooker/internal/delivery" "sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/gormlog" "sneak.berlin/go/webhooker/internal/gormlog"
) )
@@ -59,29 +60,14 @@ func setupArchiveTest(t *testing.T) *archiveEnv {
dataDir := t.TempDir() dataDir := t.TempDir()
log := archiveTestLogger() log := archiveTestLogger()
sqlDB, err := sql.Open( mainDB, err := database.Open(dataDir, slog.New(slog.DiscardHandler))
"sqlite",
fmt.Sprintf(
"file:%s?mode=rwc",
filepath.Join(dataDir, "main.db"),
),
)
require.NoError(t, err) require.NoError(t, err)
t.Cleanup(func() { _ = sqlDB.Close() }) t.Cleanup(func() { _ = mainDB.Close() })
gdb, err := gorm.Open(
sqlite.Dialector{Conn: sqlDB},
&gorm.Config{Logger: gormlog.New(slog.New(slog.DiscardHandler))},
)
require.NoError(t, err)
mainDB := database.NewTestDatabase(gdb)
require.NoError(t, mainDB.Migrate())
eng := delivery.NewTestEngineWithDB( eng := delivery.NewTestEngineWithDB(
mainDB, mainDB,
database.NewTestWebhookDBManager(dataDir), databasetest.NewWebhookDBManager(t, dataDir),
log, log,
&http.Client{Timeout: 5 * time.Second}, &http.Client{Timeout: 5 * time.Second},
1, 1,
+19 -40
View File
@@ -10,7 +10,6 @@ import (
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"os" "os"
"path/filepath"
"strings" "strings"
"sync/atomic" "sync/atomic"
"testing" "testing"
@@ -19,12 +18,11 @@ import (
"github.com/google/uuid" "github.com/google/uuid"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"gorm.io/driver/sqlite"
"gorm.io/gorm" "gorm.io/gorm"
_ "modernc.org/sqlite" _ "modernc.org/sqlite"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/database/databasetest"
"sneak.berlin/go/webhooker/internal/delivery" "sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/gormlog"
) )
// iSetup holds common integration test dependencies. // iSetup holds common integration test dependencies.
@@ -45,12 +43,12 @@ func newISetup(t *testing.T) iSetup {
wDB := iSeedWebhookDB(t, dbMgr, wID) wDB := iSeedWebhookDB(t, dbMgr, wID)
return iSetup{ return iSetup{
MainDB: mainDB, MainDB: mainDB.DB(),
DBMgr: dbMgr, DBMgr: dbMgr,
WebhookID: wID, WebhookID: wID,
WebhookDB: wDB, WebhookDB: wDB,
Engine: delivery.NewTestEngineWithDB( Engine: delivery.NewTestEngineWithDB(
database.NewTestDatabase(mainDB), mainDB,
dbMgr, dbMgr,
slog.New(slog.NewTextHandler( slog.New(slog.NewTextHandler(
os.Stderr, os.Stderr,
@@ -64,35 +62,16 @@ func newISetup(t *testing.T) iSetup {
} }
} }
func iMainDB(t *testing.T) *gorm.DB { // iMainDB opens a main database through database.Open, the way the
// service opens it, so these tests cannot pass against journal and
// locking settings production does not use.
func iMainDB(t *testing.T) *database.Database {
t.Helper() t.Helper()
dbPath := filepath.Join( db, err := database.Open(t.TempDir(), slog.New(slog.DiscardHandler))
t.TempDir(), "main-test.db",
)
// Opened the way the service opens the main database, so these
// tests cannot pass against journal and locking settings
// production does not use.
sqlDB, err := database.OpenSQLite(
dbPath, database.SQLiteModeCreate,
)
require.NoError(t, err) require.NoError(t, err)
t.Cleanup(func() { _ = sqlDB.Close() }) t.Cleanup(func() { _ = db.Close() })
db, err := gorm.Open(
sqlite.Dialector{Conn: sqlDB},
&gorm.Config{Logger: gormlog.New(slog.New(slog.DiscardHandler))},
)
require.NoError(t, err)
require.NoError(t, db.AutoMigrate(
&database.Webhook{},
&database.Target{},
&database.User{},
&database.Setting{},
))
return db return db
} }
@@ -102,7 +81,7 @@ func iDBManager(
) *database.WebhookDBManager { ) *database.WebhookDBManager {
t.Helper() t.Helper()
return database.NewTestWebhookDBManager(t.TempDir()) return databasetest.NewWebhookDBManager(t, t.TempDir())
} }
func iSeedWebhookDB( func iSeedWebhookDB(
@@ -1154,17 +1133,17 @@ func TestRecoverInFlight_ReportsAMissingWebhookDatabase(t *testing.T) {
mainDB := iMainDB(t) mainDB := iMainDB(t)
webhookID := uuid.New().String() webhookID := uuid.New().String()
iCreateWebhook(t, mainDB, webhookID, "lost-database") iCreateWebhook(t, mainDB.DB(), webhookID, "lost-database")
var logs bytes.Buffer var logs bytes.Buffer
dbMgr := database.NewTestWebhookDBManagerWithLogger( dbMgr := databasetest.NewWebhookDBManagerWithLogger(
t.TempDir(), slog.New(slog.NewTextHandler(&logs, nil)), t, t.TempDir(), slog.New(slog.NewTextHandler(&logs, nil)),
) )
t.Cleanup(func() { _ = dbMgr.CloseAll() }) t.Cleanup(func() { _ = dbMgr.CloseAll() })
engine := delivery.NewTestEngineWithDB( engine := delivery.NewTestEngineWithDB(
database.NewTestDatabase(mainDB), dbMgr, mainDB, dbMgr,
slog.New(slog.DiscardHandler), slog.New(slog.DiscardHandler),
&http.Client{Timeout: 5 * time.Second}, 1, &http.Client{Timeout: 5 * time.Second}, 1,
) )
@@ -1188,14 +1167,14 @@ func TestRecoverInFlight_SkipsAWebhookDeletedAfterTheListIsRead(
mainDB := iMainDB(t) mainDB := iMainDB(t)
webhookID := uuid.New().String() webhookID := uuid.New().String()
iCreateWebhook(t, mainDB, webhookID, "deleted-during-recovery") iCreateWebhook(t, mainDB.DB(), webhookID, "deleted-during-recovery")
// The first query to return is recovery's read of the list of // The first query to return is recovery's read of the list of
// webhooks. Deleting the webhook right after it puts the delete // webhooks. Deleting the webhook right after it puts the delete
// between that read and the opening of the webhook's database. // between that read and the opening of the webhook's database.
deleted := false deleted := false
require.NoError(t, mainDB.Callback().Query().After("gorm:query"). require.NoError(t, mainDB.DB().Callback().Query().After("gorm:query").
Register("delete-after-list", func(*gorm.DB) { Register("delete-after-list", func(*gorm.DB) {
if deleted { if deleted {
return return
@@ -1203,16 +1182,16 @@ func TestRecoverInFlight_SkipsAWebhookDeletedAfterTheListIsRead(
deleted = true deleted = true
require.NoError(t, mainDB.Delete( require.NoError(t, mainDB.DB().Delete(
&database.Webhook{}, "id = ?", webhookID, &database.Webhook{}, "id = ?", webhookID,
).Error) ).Error)
})) }))
dbMgr := database.NewTestWebhookDBManager(t.TempDir()) dbMgr := databasetest.NewWebhookDBManager(t, t.TempDir())
t.Cleanup(func() { _ = dbMgr.CloseAll() }) t.Cleanup(func() { _ = dbMgr.CloseAll() })
engine := delivery.NewTestEngineWithDB( engine := delivery.NewTestEngineWithDB(
database.NewTestDatabase(mainDB), dbMgr, mainDB, dbMgr,
slog.New(slog.DiscardHandler), slog.New(slog.DiscardHandler),
&http.Client{Timeout: 5 * time.Second}, 1, &http.Client{Timeout: 5 * time.Second}, 1,
) )
+4 -3
View File
@@ -48,8 +48,9 @@ func fSweepSetup(
// //
// Every caller drives the dispatch paths synchronously and has already // Every caller drives the dispatch paths synchronously and has already
// waited for them to return, so anything they queued is in the channel // waited for them to return, so anything they queued is in the channel
// by now. The short grace covers nothing but scheduler jitter, and is // by now, and nothing is waited for. A timer here would race the queued
// kept small because one of these tests runs the drain forty times. // tasks: on a busy host it can be due by the time select looks, and
// select picks at random among the cases that are ready.
func fDrain(e *delivery.Engine) []delivery.Task { func fDrain(e *delivery.Engine) []delivery.Task {
var out []delivery.Task var out []delivery.Task
@@ -59,7 +60,7 @@ func fDrain(e *delivery.Engine) []delivery.Task {
out = append(out, task) out = append(out, task)
case task := <-e.ExportRetryCh(): case task := <-e.ExportRetryCh():
out = append(out, task) out = append(out, task)
case <-time.After(25 * time.Millisecond): default:
return out return out
} }
} }
+9 -26
View File
@@ -5,7 +5,6 @@ import (
"context" "context"
"log/slog" "log/slog"
"net/http" "net/http"
"path/filepath"
"strings" "strings"
"sync" "sync"
"testing" "testing"
@@ -14,11 +13,9 @@ import (
"github.com/google/uuid" "github.com/google/uuid"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/database/databasetest"
"sneak.berlin/go/webhooker/internal/delivery" "sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/gormlog"
) )
// qdAggregateMarker identifies the queue-depth aggregate in the // qdAggregateMarker identifies the queue-depth aggregate in the
@@ -49,27 +46,13 @@ func (q *qdSyncBuf) String() string {
// qdMainDB opens a main database whose GORM logger is the service's // qdMainDB opens a main database whose GORM logger is the service's
// adapter, writing through log. // adapter, writing through log.
func qdMainDB(t *testing.T, log *slog.Logger) *gorm.DB { func qdMainDB(t *testing.T, log *slog.Logger) *database.Database {
t.Helper() t.Helper()
sqlDB, err := database.OpenSQLite( db, err := database.Open(t.TempDir(), log)
filepath.Join(t.TempDir(), "main-gormlog.db"),
database.SQLiteModeCreate,
)
require.NoError(t, err) require.NoError(t, err)
t.Cleanup(func() { _ = sqlDB.Close() }) t.Cleanup(func() { _ = db.Close() })
db, err := gorm.Open(
sqlite.Dialector{Conn: sqlDB},
&gorm.Config{Logger: gormlog.New(log)},
)
require.NoError(t, err)
require.NoError(t, db.AutoMigrate(
&database.Webhook{},
&database.Target{},
))
return db return db
} }
@@ -106,18 +89,18 @@ func TestQueueDepthSample_LogsNoBoundValue(t *testing.T) {
)) ))
mainDB := qdMainDB(t, log) mainDB := qdMainDB(t, log)
dbMgr := database.NewTestWebhookDBManagerWithLogger( dbMgr := databasetest.NewWebhookDBManagerWithLogger(
t.TempDir(), log, t, t.TempDir(), log,
) )
webhookID := uuid.New().String() webhookID := uuid.New().String()
webhookDB := iSeedWebhookDB(t, dbMgr, webhookID) webhookDB := iSeedWebhookDB(t, dbMgr, webhookID)
iCreateWebhook(t, mainDB, webhookID, "queue-depth-gormlog") iCreateWebhook(t, mainDB.DB(), webhookID, "queue-depth-gormlog")
targetID := uuid.New().String() targetID := uuid.New().String()
iCreateTarget(t, mainDB, targetID, webhookID, iCreateTarget(t, mainDB.DB(), targetID, webhookID,
"queue-depth-gormlog-target", database.TargetTypeHTTP, "queue-depth-gormlog-target", database.TargetTypeHTTP,
iHTTPConfig("https://example.com/hook"), 3, iHTTPConfig("https://example.com/hook"), 3,
) )
@@ -136,7 +119,7 @@ func TestQueueDepthSample_LogsNoBoundValue(t *testing.T) {
) )
engine := delivery.NewTestEngineWithDB( engine := delivery.NewTestEngineWithDB(
database.NewTestDatabase(mainDB), mainDB,
dbMgr, dbMgr,
log, log,
&http.Client{Timeout: 5 * time.Second}, &http.Client{Timeout: 5 * time.Second},
@@ -460,8 +460,13 @@ func TestArchiveExport_OneFileOpenAtATime(t *testing.T) {
} }
// heapPeak is an io.Writer that discards what it is given and records // heapPeak is an io.Writer that discards what it is given and records
// the largest heap it saw at a write. It collects garbage before each // the largest heap it saw at a write. It collects garbage twice before
// reading, so the heap it reads is what is still held. // each reading, so the heap it reads is what is still held. Once is not
// enough: the libraries the export calls (regexp, under GORM's table
// names, and encoding/json among them) cache buffers in a sync.Pool,
// which keeps them through one collection, so after one the reading
// counts however many happen to be cached. That varies from run to run
// by about as much as the limit in TestArchiveExport_Streams.
type heapPeak struct { type heapPeak struct {
max uint64 max uint64
} }
@@ -469,6 +474,7 @@ type heapPeak struct {
func (p *heapPeak) Write(b []byte) (int, error) { func (p *heapPeak) Write(b []byte) (int, error) {
var m runtime.MemStats var m runtime.MemStats
runtime.GC()
runtime.GC() runtime.GC()
runtime.ReadMemStats(&m) runtime.ReadMemStats(&m)
p.max = max(p.max, m.HeapAlloc) p.max = max(p.max, m.HeapAlloc)
@@ -498,6 +504,8 @@ func exportHeapGrowth(t *testing.T, rows, bodySize int) uint64 {
export := listExport(t, path) export := listExport(t, path)
// Twice, for the reason heapPeak gives.
runtime.GC()
runtime.GC() runtime.GC()
var start runtime.MemStats var start runtime.MemStats
@@ -10,6 +10,7 @@ import (
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/database/databasetest"
"sneak.berlin/go/webhooker/internal/delivery" "sneak.berlin/go/webhooker/internal/delivery"
) )
@@ -335,7 +336,7 @@ func TestArchivePathAt(t *testing.T) {
t.Parallel() t.Parallel()
dataDir := t.TempDir() dataDir := t.TempDir()
dbMgr := database.NewTestWebhookDBManager(dataDir) dbMgr := databasetest.NewWebhookDBManager(t, dataDir)
webhook := &database.Webhook{ webhook := &database.Webhook{
BaseModel: database.BaseModel{ID: "wh-id"}, Name: "Orders", BaseModel: database.BaseModel{ID: "wh-id"}, Name: "Orders",
} }
+4 -1
View File
@@ -3,6 +3,7 @@ package gormlog_test
import ( import (
"context" "context"
"database/sql" "database/sql"
"log/slog"
"os" "os"
"path/filepath" "path/filepath"
"testing" "testing"
@@ -13,6 +14,7 @@ import (
"go.uber.org/fx/fxtest" "go.uber.org/fx/fxtest"
_ "modernc.org/sqlite" // Pure Go SQLite driver. _ "modernc.org/sqlite" // Pure Go SQLite driver.
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/config/configtest"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
@@ -128,7 +130,7 @@ func readFirstBootSecrets(
func bootAtDebug(t *testing.T, dataDir string) string { func bootAtDebug(t *testing.T, dataDir string) string {
t.Helper() t.Helper()
config.ClearEnvForTest(t) configtest.ClearEnv(t)
t.Setenv("DEBUG", "true") t.Setenv("DEBUG", "true")
t.Setenv("DATA_DIR", dataDir) t.Setenv("DATA_DIR", dataDir)
@@ -145,6 +147,7 @@ func bootAtDebug(t *testing.T, dataDir string) string {
fx.Provide( fx.Provide(
globals.New, globals.New,
logger.New, logger.New,
func(l *logger.Logger) *slog.Logger { return l.Get() },
config.New, config.New,
database.New, database.New,
session.New, session.New,
+2
View File
@@ -5,6 +5,7 @@ import (
"errors" "errors"
"fmt" "fmt"
"html/template" "html/template"
"log/slog"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"sync" "sync"
@@ -249,6 +250,7 @@ func newTestAppWithConfig(
fx.Provide( fx.Provide(
globals.New, globals.New,
logger.New, logger.New,
func(l *logger.Logger) *slog.Logger { return l.Get() },
func() *config.Config { return cfg }, func() *config.Config { return cfg },
database.New, database.New,
database.NewWebhookDBManager, database.NewWebhookDBManager,
+2 -2
View File
@@ -15,7 +15,7 @@ import (
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/handlers" "sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
"sneak.berlin/go/webhooker/internal/middleware" "sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
"sneak.berlin/go/webhooker/internal/session" "sneak.berlin/go/webhooker/internal/session"
) )
@@ -135,7 +135,7 @@ func TestUserRoute_Unauthenticated_RedirectedByMiddleware(t *testing.T) {
t.Cleanup(app.RequireStop) t.Cleanup(app.RequireStop)
mw := middleware.NewForTest(log.Get(), cfg, sess) mw := middlewaretest.New(t, log.Get(), cfg, sess)
var handlerReached bool var handlerReached bool
@@ -15,7 +15,7 @@ import (
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/middleware" "sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
) )
// targetSecretSegments are the path segments of an incoming-webhook // targetSecretSegments are the path segments of an incoming-webhook
@@ -65,7 +65,8 @@ func postTargetCreate(
t.Helper() t.Helper()
logBuf := new(bytes.Buffer) logBuf := new(bytes.Buffer)
mw := middleware.NewForTest( mw := middlewaretest.New(
t,
slog.New(slog.NewJSONHandler( slog.New(slog.NewJSONHandler(
logBuf, &slog.HandlerOptions{Level: slog.LevelInfo}, logBuf, &slog.HandlerOptions{Level: slog.LevelInfo},
)), )),
+3 -2
View File
@@ -24,7 +24,7 @@ import (
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery" "sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/middleware" "sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
) )
// errClientGone is the write failure of a client that has gone away. // errClientGone is the write failure of a client that has gone away.
@@ -310,7 +310,8 @@ func limitedServer(
const sendBuffer = 4 << 10 const sendBuffer = 4 << 10
logBuf := new(bytes.Buffer) logBuf := new(bytes.Buffer)
mw := middleware.NewForTest( mw := middlewaretest.New(
t,
slog.New(slog.NewJSONHandler(logBuf, nil)), slog.New(slog.NewJSONHandler(logBuf, nil)),
&config.Config{Environment: config.EnvironmentDev}, &config.Config{Environment: config.EnvironmentDev},
nil, nil,
+3 -2
View File
@@ -16,6 +16,7 @@ import (
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/middleware" "sneak.berlin/go/webhooker/internal/middleware"
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
) )
// floodRequests is the number of distinct invented paths each flood // floodRequests is the number of distinct invented paths each flood
@@ -83,7 +84,7 @@ func capturingMiddleware(t *testing.T) (*middleware.Middleware, *bytes.Buffer) {
TrustedProxies: trustedProxies("192.0.2.1/32"), TrustedProxies: trustedProxies("192.0.2.1/32"),
} }
return middleware.NewForTest(log, cfg, nil), buf return middlewaretest.New(t, log, cfg, nil), buf
} }
// capturingTextMiddleware is capturingMiddleware for the other handler // capturingTextMiddleware is capturingMiddleware for the other handler
@@ -107,7 +108,7 @@ func capturingTextMiddleware(
TrustedProxies: trustedProxies("192.0.2.1/32"), TrustedProxies: trustedProxies("192.0.2.1/32"),
} }
return middleware.NewForTest(log, cfg, nil), buf return middlewaretest.New(t, log, cfg, nil), buf
} }
// accessLogRouter mirrors the production route shapes that an // accessLogRouter mirrors the production route shapes that an
+3 -2
View File
@@ -12,6 +12,7 @@ import (
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/middleware" "sneak.berlin/go/webhooker/internal/middleware"
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
) )
const ( const (
@@ -133,8 +134,8 @@ func clientLogLines(
TrustedProxies: trustedProxies(trustedProxyCIDR), TrustedProxies: trustedProxies(trustedProxyCIDR),
} }
m := middleware.NewForTest( m := middlewaretest.New(
log, cfg, newTestSessionManager(cfg, log, nil), t, log, cfg, newTestSessionManager(t, cfg),
) )
handler := m.Logging()(site.build(m)) handler := m.Logging()(site.build(m))
+3 -2
View File
@@ -41,6 +41,7 @@ import (
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/middleware" "sneak.berlin/go/webhooker/internal/middleware"
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
) )
// bodyLimitBytes is the MaxBodySize cap these tests install. Any // bodyLimitBytes is the MaxBodySize cap these tests install. Any
@@ -155,9 +156,9 @@ func capturingBoundMiddleware(
ReceiverRateLimit: receiverLimitPerMinute, ReceiverRateLimit: receiverLimitPerMinute,
} }
sess := newTestSessionManager(cfg, log, nil) sess := newTestSessionManager(t, cfg)
return middleware.NewForTest(log, cfg, sess), buf return middlewaretest.New(t, log, cfg, sess), buf
} }
// unreachable is a next-handler that fails the test if the middleware // unreachable is a next-handler that fails the test if the middleware
+1 -1
View File
@@ -151,7 +151,7 @@ var _ httpmetrics.Recorder = boundedLabelRecorder{}
// Metrics returns middleware that records Prometheus HTTP metrics // Metrics returns middleware that records Prometheus HTTP metrics
// with the Middleware's one recorder, which New builds on the registry // with the Middleware's one recorder, which New builds on the registry
// the /metrics route serves and NewForTest on a registry of its own. // it is given: in the application, the one the /metrics route serves.
// Every call reuses that recorder, so any number of routers can // Every call reuses that recorder, so any number of routers can
// install it. // install it.
func (s *Middleware) Metrics() func(http.Handler) http.Handler { func (s *Middleware) Metrics() func(http.Handler) http.Handler {
+9 -8
View File
@@ -16,6 +16,7 @@ import (
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/middleware" "sneak.berlin/go/webhooker/internal/middleware"
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
) )
const ( const (
@@ -70,8 +71,8 @@ func metricsTestRouter(
Environment: "prod", Environment: "prod",
ReceiverRateLimit: receiverLimit, ReceiverRateLimit: receiverLimit,
} }
m := middleware.NewForTest( m := middlewaretest.New(
log, cfg, newTestSessionManager(cfg, log, nil), t, log, cfg, newTestSessionManager(t, cfg),
) )
reg := prometheus.NewRegistry() reg := prometheus.NewRegistry()
@@ -455,11 +456,11 @@ func TestMetrics_StatusAndSizeStillRecorded(t *testing.T) {
) )
} }
// TestMetrics_WorksOnNewForTestMiddleware pins that a Middleware built // TestMetrics_WorksOnMiddlewaretestNew pins that a Middleware built
// by NewForTest has a recorder of its own: its Metrics() serves a // by middlewaretest.New has a recorder of its own: its Metrics()
// request instead of panicking, and a second one does not collide // serves a request instead of panicking, and a second one does not
// with the first. // collide with the first.
func TestMetrics_WorksOnNewForTestMiddleware(t *testing.T) { func TestMetrics_WorksOnMiddlewaretestNew(t *testing.T) {
t.Parallel() t.Parallel()
log := slog.New(slog.DiscardHandler) log := slog.New(slog.DiscardHandler)
@@ -469,7 +470,7 @@ func TestMetrics_WorksOnNewForTestMiddleware(t *testing.T) {
}) })
for range 2 { for range 2 {
h := middleware.NewForTest(log, cfg, nil).Metrics()(ok) h := middlewaretest.New(t, log, cfg, nil).Metrics()(ok)
req := httptest.NewRequestWithContext( req := httptest.NewRequestWithContext(
t.Context(), http.MethodGet, okRoute, nil, t.Context(), http.MethodGet, okRoute, nil,
+6 -9
View File
@@ -22,7 +22,6 @@ import (
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/logfield" "sneak.berlin/go/webhooker/internal/logfield"
"sneak.berlin/go/webhooker/internal/logger"
"sneak.berlin/go/webhooker/internal/session" "sneak.berlin/go/webhooker/internal/session"
) )
@@ -155,7 +154,7 @@ const (
type MiddlewareParams struct { type MiddlewareParams struct {
fx.In fx.In
Logger *logger.Logger Logger *slog.Logger
Globals *globals.Globals Globals *globals.Globals
Config *config.Config Config *config.Config
Session *session.Session Session *session.Session
@@ -169,12 +168,10 @@ type Middleware struct {
params *MiddlewareParams params *MiddlewareParams
session *session.Session session *session.Session
// metricsRecorder records the inbound HTTP metrics. New builds // metricsRecorder records the inbound HTTP metrics on
// it on the registry /metrics serves, NewForTest on a registry // params.Registry. It is built once per Middleware and Metrics
// of its own. Either way it is built once per Middleware and // reuses it, because building it registers its collectors, and a
// Metrics reuses it, because building it registers its // second registration on the same registry panics.
// collectors, and a second registration on the same registry
// panics.
metricsRecorder httpmetrics.Recorder metricsRecorder httpmetrics.Recorder
// loginGuard counts failed credential verifications and bounds // loginGuard counts failed credential verifications and bounds
@@ -193,7 +190,7 @@ func New(
) (*Middleware, error) { ) (*Middleware, error) {
s := new(Middleware) s := new(Middleware)
s.params = &params s.params = &params
s.log = params.Logger.Get() s.log = params.Logger
s.session = params.Session s.session = params.Session
s.metricsRecorder = prommetrics.NewRecorder( s.metricsRecorder = prommetrics.NewRecorder(
prommetrics.Config{Registry: params.Registry}, prommetrics.Config{Registry: params.Registry},
+87 -71
View File
@@ -14,36 +14,34 @@ import (
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"go.uber.org/fx/fxtest"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/middleware" "sneak.berlin/go/webhooker/internal/middleware"
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
"sneak.berlin/go/webhooker/internal/session" "sneak.berlin/go/webhooker/internal/session"
) )
const testKeySize = 32 const testKeySize = 32
// testMiddleware creates a Middleware with minimal dependencies // testMiddleware creates a Middleware with minimal dependencies
// for testing. It uses a real session.Session backed by an // for testing. It uses a real session.Session.
// in-memory cookie store.
func testMiddleware( func testMiddleware(
t *testing.T, t *testing.T,
env string, env string,
) (*middleware.Middleware, *session.Session) { ) (*middleware.Middleware, *session.Session) {
t.Helper() t.Helper()
m, s, _ := testMiddlewareWithSessionClock(t, env, 0, nil) return testMiddlewareWithIdleTimeout(t, env, 0)
return m, s
} }
// testMiddlewareWithSessionClock is testMiddleware with a // testMiddlewareWithIdleTimeout is testMiddleware with a
// configurable session idle timeout and a manually advanced clock, // configurable session idle timeout, for the session-expiry tests.
// for the session-expiry tests. A nil clock uses the real one. func testMiddlewareWithIdleTimeout(
func testMiddlewareWithSessionClock(
t *testing.T, t *testing.T,
env string, env string,
idleTimeout time.Duration, idleTimeout time.Duration,
clock *fakeClock, ) (*middleware.Middleware, *session.Session) {
) (*middleware.Middleware, *session.Session, *fakeClock) {
t.Helper() t.Helper()
log := slog.New(slog.NewTextHandler( log := slog.New(slog.NewTextHandler(
@@ -56,59 +54,44 @@ func testMiddlewareWithSessionClock(
SessionIdleTimeout: idleTimeout, SessionIdleTimeout: idleTimeout,
} }
sessManager := newTestSessionManager(cfg, log, clock) sessManager := newTestSessionManager(t, cfg)
m := middleware.NewForTest(log, cfg, sessManager) m := middlewaretest.New(t, log, cfg, sessManager)
return m, sessManager, clock return m, sessManager
} }
// newTestSessionManager builds the real session.Session the // newTestSessionManager builds the real session.Session the
// middleware tests run against: an in-memory cookie store with a // middleware tests run against, through session.New, with its key
// known key, and optionally a manually advanced clock. // in a main database of its own.
func newTestSessionManager( func newTestSessionManager(
t *testing.T,
cfg *config.Config, cfg *config.Config,
log *slog.Logger,
clock *fakeClock,
) *session.Session { ) *session.Session {
key := make([]byte, testKeySize) t.Helper()
for i := range key { discard := slog.New(slog.DiscardHandler)
key[i] = byte(i)
}
store := session.NewStore(key) db, err := database.Open(t.TempDir(), discard)
require.NoError(t, err)
var now func() time.Time t.Cleanup(func() { _ = db.Close() })
if clock != nil { lc := fxtest.NewLifecycle(t)
now = clock.Now
}
return session.NewForTest(store, cfg, log, key, now) sessManager, err := session.New(lc, session.Params{
} Config: cfg,
Database: db,
Logger: discard,
})
require.NoError(t, err)
// fakeClock is a manually advanced clock, so session expiry can be // The start hook reads the key from db and builds the cookie
// tested without sleeping. // store.
type fakeClock struct { lc.RequireStart()
t time.Time t.Cleanup(lc.RequireStop)
}
func (c *fakeClock) Now() time.Time { return sessManager
return c.t
}
func (c *fakeClock) Advance(d time.Duration) {
c.t = c.t.Add(d)
}
// newFakeClock returns a clock started at a fixed instant.
func newFakeClock() *fakeClock {
return &fakeClock{
t: time.Date(
2026, time.January, 2, 3, 4, 5, 0, time.UTC,
),
}
} }
// --- Logging Middleware Tests --- // --- Logging Middleware Tests ---
@@ -583,6 +566,40 @@ func sessionCookies(
return out return out
} }
// aged re-issues the session cookie in cookies with both of its
// timestamps moved back by d: the cookie as it stands once d has
// passed, so session expiry can be tested without sleeping.
func aged(
t *testing.T,
sessManager *session.Session,
cookies []*http.Cookie,
d time.Duration,
) []*http.Cookie {
t.Helper()
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
for _, c := range cookies {
req.AddCookie(c)
}
sess, err := sessManager.Get(req)
require.NoError(t, err)
for _, key := range []string{session.CreatedAtKey, session.LastSeenKey} {
at, ok := sess.Values[key].(int64)
require.True(t, ok, "the session has no %s", key)
sess.Values[key] = at - int64(d/time.Second)
}
w := httptest.NewRecorder()
require.NoError(t, sessManager.Save(req, w, sess))
return sessionCookies(w)
}
func TestRequireAuth_IdleExpiredSession_RedirectsToLogin( func TestRequireAuth_IdleExpiredSession_RedirectsToLogin(
t *testing.T, t *testing.T,
) { ) {
@@ -590,13 +607,11 @@ func TestRequireAuth_IdleExpiredSession_RedirectsToLogin(
idle := time.Hour idle := time.Hour
m, sessManager, clock := testMiddlewareWithSessionClock( m, sessManager := testMiddlewareWithIdleTimeout(
t, config.EnvironmentDev, idle, newFakeClock(), t, config.EnvironmentDev, idle,
) )
cookies := loginCookies(t, sessManager) cookies := aged(t, sessManager, loginCookies(t, sessManager), idle)
clock.Advance(idle)
called, w := runAuthed(t, m, cookies) called, w := runAuthed(t, m, cookies)
@@ -621,14 +636,12 @@ func TestRequireAuth_RefreshesIdleDeadlineOnActivity(
idle := time.Hour idle := time.Hour
m, sessManager, clock := testMiddlewareWithSessionClock( m, sessManager := testMiddlewareWithIdleTimeout(
t, config.EnvironmentDev, idle, newFakeClock(), t, config.EnvironmentDev, idle,
) )
cookies := loginCookies(t, sessManager)
// Activity halfway through the idle window. // Activity halfway through the idle window.
clock.Advance(idle / 2) cookies := aged(t, sessManager, loginCookies(t, sessManager), idle/2)
called, w := runAuthed(t, m, cookies) called, w := runAuthed(t, m, cookies)
require.True(t, called, "handler should run while valid") require.True(t, called, "handler should run while valid")
@@ -640,16 +653,22 @@ func TestRequireAuth_RefreshesIdleDeadlineOnActivity(
) )
// Past the original deadline. The refreshed cookie is still // Past the original deadline. The refreshed cookie is still
// good; the original one is not. // good; the original one is not. A minute short of the idle
clock.Advance(idle - time.Second) // window leaves room for the real clock, which the session
// reads, to tick on while the test runs.
later := idle - time.Minute
calledRefreshed, _ := runAuthed(t, m, refreshed) calledRefreshed, _ := runAuthed(
t, m, aged(t, sessManager, refreshed, later),
)
assert.True( assert.True(
t, calledRefreshed, t, calledRefreshed,
"refreshed session should outlive the original deadline", "refreshed session should outlive the original deadline",
) )
calledStale, staleW := runAuthed(t, m, cookies) calledStale, staleW := runAuthed(
t, m, aged(t, sessManager, cookies, later),
)
assert.False( assert.False(
t, calledStale, t, calledStale,
"the pre-refresh cookie carries the old idle deadline", "the pre-refresh cookie carries the old idle deadline",
@@ -662,8 +681,8 @@ func TestRequireAuth_UnauthenticatedRequestDoesNotRefresh(
) { ) {
t.Parallel() t.Parallel()
m, sessManager, _ := testMiddlewareWithSessionClock( m, sessManager := testMiddlewareWithIdleTimeout(
t, config.EnvironmentDev, time.Hour, newFakeClock(), t, config.EnvironmentDev, time.Hour,
) )
// A session cookie that exists but was never authenticated. // A session cookie that exists but was never authenticated.
@@ -924,12 +943,9 @@ func metricsAuthMiddleware(
MetricsPassword: "secret", MetricsPassword: "secret",
} }
key := make([]byte, testKeySize) return middlewaretest.New(
store := session.NewStore(key) t, log, cfg, newTestSessionManager(t, cfg),
)
sessManager := session.NewForTest(store, cfg, log, key, nil)
return middleware.NewForTest(log, cfg, sessManager)
} }
// runMetricsAuthRequest sends a GET /metrics request with the // runMetricsAuthRequest sends a GET /metrics request with the
@@ -0,0 +1,42 @@
// Package middlewaretest builds a Middleware for tests in other
// packages.
package middlewaretest
import (
"log/slog"
"testing"
"github.com/prometheus/client_golang/prometheus"
"github.com/stretchr/testify/require"
"go.uber.org/fx/fxtest"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/middleware"
"sneak.berlin/go/webhooker/internal/session"
)
// New builds a Middleware through middleware.New, on a
// lifecycle that is never started.
//
// Its metrics recorder writes to a fresh registry of its own, so
// Metrics() works on it and two of them never collide.
func New(
t *testing.T,
log *slog.Logger,
cfg *config.Config,
sess *session.Session,
) *middleware.Middleware {
t.Helper()
m, err := middleware.New(
fxtest.NewLifecycle(t),
middleware.MiddlewareParams{
Logger: log,
Config: cfg,
Session: sess,
Registry: prometheus.NewRegistry(),
},
)
require.NoError(t, err)
return m
}
+2 -1
View File
@@ -18,6 +18,7 @@ import (
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/middleware" "sneak.berlin/go/webhooker/internal/middleware"
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
) )
func TestPostRateLimit_AllowsGET(t *testing.T) { func TestPostRateLimit_AllowsGET(t *testing.T) {
@@ -198,7 +199,7 @@ func rateLimitMiddleware(
&slog.HandlerOptions{Level: slog.LevelDebug}, &slog.HandlerOptions{Level: slog.LevelDebug},
)) ))
return middleware.NewForTest(log, cfg, nil) return middlewaretest.New(t, log, cfg, nil)
} }
// trustedProxies parses CIDR strings for a test Config. // trustedProxies parses CIDR strings for a test Config.
-32
View File
@@ -1,32 +0,0 @@
package middleware
import (
"log/slog"
"github.com/prometheus/client_golang/prometheus"
prommetrics "github.com/slok/go-http-metrics/metrics/prometheus"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/session"
)
// NewForTest creates a Middleware with the minimum dependencies
// needed for testing. This bypasses the fx lifecycle.
//
// Its metrics recorder writes to a fresh registry of its own, so
// Metrics() works on it and two of them never collide.
func NewForTest(
log *slog.Logger,
cfg *config.Config,
sess *session.Session,
) *Middleware {
return &Middleware{
log: log,
params: &MiddlewareParams{
Config: cfg,
},
session: sess,
metricsRecorder: prommetrics.NewRecorder(
prommetrics.Config{Registry: prometheus.NewRegistry()},
),
}
}
+1
View File
@@ -174,6 +174,7 @@ func newServerApp(
fx.Provide( fx.Provide(
globals.New, globals.New,
logger.New, logger.New,
func(l *logger.Logger) *slog.Logger { return l.Get() },
func() *config.Config { func() *config.Config {
return &config.Config{DataDir: dir} return &config.Config{DataDir: dir}
}, },
+25 -9
View File
@@ -280,6 +280,23 @@ func click(ctx context.Context, t *testing.T, xpath string) {
)) ))
} }
// clickAndLoad clicks the link or button matching an XPath expression
// and waits, as loadPage does, for the page the click opens to load and
// for Alpine.js to start on it. Reading earlier, a check can find an
// element of the page being left, gone by the time its value is read;
// and the wait in shown is too short for a page load on a busy host.
func clickAndLoad(ctx context.Context, t *testing.T, xpath string) {
t.Helper()
_, err := chromedp.RunResponse(
ctx, chromedp.Click(xpath, chromedp.BySearch),
)
require.NoError(t, err)
require.NoError(t, chromedp.Run(
ctx, chromedp.WaitNotPresent("[x-cloak]", chromedp.ByQuery),
))
}
// checkAddEntrypoint loads a webhook page and checks that the add // checkAddEntrypoint loads a webhook page and checks that the add
// entrypoint form stays hidden until the Add button beside its heading // entrypoint form stays hidden until the Add button beside its heading
// is clicked. The click looks for a button element there, so it also // is clicked. The click looks for a button element there, so it also
@@ -422,7 +439,7 @@ func checkAddTarget(
))) )))
} }
click(ctx, t, saveButton) clickAndLoad(ctx, t, saveButton)
assert.Truef(t, shown(ctx, `//span[text()="`+name+ assert.Truef(t, shown(ctx, `//span[text()="`+name+
`"]/following-sibling::div/span[text()="`+badge+`"]`), `"]/following-sibling::div/span[text()="`+badge+`"]`),
"%s: the added target is not listed as %s", targetType, badge) "%s: the added target is not listed as %s", targetType, badge)
@@ -485,10 +502,9 @@ func checkArchiveChoices(ctx context.Context, t *testing.T, url string) {
`/following-sibling::span[text()="daily"]`), `/following-sibling::span[text()="daily"]`),
"a database target added with daily is not listed as daily") "a database target added with daily is not listed as daily")
click(ctx, t, row+`//a[text()="Edit"]`) clickAndLoad(ctx, t, row+`//a[text()="Edit"]`)
require.NoError(t, chromedp.Run( require.NoError(t, chromedp.Run(
ctx, ctx,
chromedp.WaitReady("#expiry", chromedp.ByQuery),
chromedp.Value("#expiry", &editedExpiry, chromedp.ByQuery), chromedp.Value("#expiry", &editedExpiry, chromedp.ByQuery),
chromedp.Value("#rotation", &editedRotation, chromedp.ByQuery), chromedp.Value("#rotation", &editedRotation, chromedp.ByQuery),
)) ))
@@ -523,7 +539,7 @@ func checkRefusedTarget(ctx context.Context, t *testing.T, url string) {
chromedp.Click(forwardQuery, chromedp.ByQuery), chromedp.Click(forwardQuery, chromedp.ByQuery),
)) ))
click(ctx, t, saveButton) clickAndLoad(ctx, t, saveButton)
assert.True(t, shown(ctx, reason), assert.True(t, shown(ctx, reason),
"a refused target does not show the reason") "a refused target does not show the reason")
@@ -636,7 +652,7 @@ func checkRefusedEdits(
)) ))
} }
click(ctx, t, `//button[text()="Save Changes"]`) clickAndLoad(ctx, t, `//button[text()="Save Changes"]`)
assert.Truef(t, shown(ctx, reason), assert.Truef(t, shown(ctx, reason),
"%s: a refused save does not show the reason", edit.url) "%s: a refused save does not show the reason", edit.url)
@@ -760,7 +776,7 @@ func checkEntrypointEdit(
require.NoError(t, chromedp.Run( require.NoError(t, chromedp.Run(
ctx, chromedp.SendKeys(input, "Billing sender", chromedp.ByQuery), ctx, chromedp.SendKeys(input, "Billing sender", chromedp.ByQuery),
)) ))
click(ctx, t, saveEdit) clickAndLoad(ctx, t, saveEdit)
assert.True(t, shown(ctx, `//span[text()="Billing sender"]`), assert.True(t, shown(ctx, `//span[text()="Billing sender"]`),
"saving the edit form does not change the description") "saving the edit form does not change the description")
@@ -798,7 +814,7 @@ func checkRecentEvents(ctx context.Context, t *testing.T, url string) {
"clicking the newest event does not collapse it") "clicking the newest event does not collapse it")
require.NoError(t, chromedp.Run(ctx, loadPage(url))) require.NoError(t, chromedp.Run(ctx, loadPage(url)))
click(ctx, t, newest+`/ancestor::div[@x-data][1]//a[text()="Open"]`) clickAndLoad(ctx, t, newest+`/ancestor::div[@x-data][1]//a[text()="Open"]`)
assert.True(t, shown(ctx, `//h2[text()="Body"]`), assert.True(t, shown(ctx, `//h2[text()="Body"]`),
"Open does not lead to the event's own page") "Open does not lead to the event's own page")
@@ -1199,7 +1215,7 @@ func checkNewWebhookTargets(
`","rotation":"none"}` `","rotation":"none"}`
} }
click(ctx, t, createButton) clickAndLoad(ctx, t, createButton)
require.Truef(t, shown(ctx, `//h1[text()="`+name+`"]`), require.Truef(t, shown(ctx, `//h1[text()="`+name+`"]`),
"%s: the new webhook's page does not open", name) "%s: the new webhook's page does not open", name)
@@ -1260,7 +1276,7 @@ func checkRefusedNewWebhook(ctx context.Context, t *testing.T, url string) {
chromedp.SetValue(pruningChoice, "2160h", chromedp.BySearch), chromedp.SetValue(pruningChoice, "2160h", chromedp.BySearch),
chromedp.SetValue("#archive_rotation", "monthly", chromedp.ByQuery), chromedp.SetValue("#archive_rotation", "monthly", chromedp.ByQuery),
)) ))
click(ctx, t, createButton) clickAndLoad(ctx, t, createButton)
assert.True(t, shown(ctx, `//div[@class="alert-error"]`), assert.True(t, shown(ctx, `//div[@class="alert-error"]`),
"a refused webhook does not show the reason") "a refused webhook does not show the reason")
+2
View File
@@ -3,6 +3,7 @@ package server_test
import ( import (
"context" "context"
"html" "html"
"log/slog"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"net/url" "net/url"
@@ -130,6 +131,7 @@ func newTestEnvWithConfig(
fx.Provide( fx.Provide(
globals.New, globals.New,
logger.New, logger.New,
func(l *logger.Logger) *slog.Logger { return l.Get() },
func() *config.Config { return cfg }, func() *config.Config { return cfg },
database.New, database.New,
database.NewWebhookDBManager, database.NewWebhookDBManager,
@@ -16,8 +16,7 @@ func NewStore(key []byte) *sessions.CookieStore {
} }
// NewForTest creates a Session with a pre-configured cookie store for use // NewForTest creates a Session with a pre-configured cookie store for use
// in tests. This bypasses the fx lifecycle and database dependency, allowing // in tests. This bypasses the fx lifecycle and database dependency. The key
// middleware and handler tests to use real session functionality. The key
// parameter is the raw 32-byte authentication key used for session encryption // parameter is the raw 32-byte authentication key used for session encryption
// and CSRF cookie signing. // and CSRF cookie signing.
// //
+2 -3
View File
@@ -16,7 +16,6 @@ import (
"go.uber.org/fx" "go.uber.org/fx"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/logger"
"sneak.berlin/go/webhooker/internal/reqtls" "sneak.berlin/go/webhooker/internal/reqtls"
) )
@@ -80,7 +79,7 @@ type Params struct {
Config *config.Config Config *config.Config
Database *database.Database Database *database.Database
Logger *logger.Logger Logger *slog.Logger
} }
// Session manages encrypted session storage. // Session manages encrypted session storage.
@@ -180,7 +179,7 @@ func New(
params Params, params Params,
) (*Session, error) { ) (*Session, error) {
s := &Session{ s := &Session{
log: params.Logger.Get(), log: params.Logger,
idleTimeout: params.Config.SessionIdleTimeout, idleTimeout: params.Config.SessionIdleTimeout,
now: time.Now, now: time.Now,
} }
+9 -1
View File
@@ -11,6 +11,7 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
PKGMGR="" PKGMGR=""
SUDO="" SUDO=""
APT_UPDATED=""
detect_pkgmgr() { detect_pkgmgr() {
[ -n "$PKGMGR" ] && return 0 [ -n "$PKGMGR" ] && return 0
@@ -39,7 +40,14 @@ pkg_install() {
detect_pkgmgr detect_pkgmgr
case "$PKGMGR" in case "$PKGMGR" in
nix) nix-env -iA "nixpkgs.$1" ;; nix) nix-env -iA "nixpkgs.$1" ;;
apt) $SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2" ;; apt)
# Package lists may be empty (fresh images); refresh once per run.
if [ -z "$APT_UPDATED" ]; then
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get update
APT_UPDATED=1
fi
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2"
;;
brew) brew install "$3" ;; brew) brew install "$3" ;;
apk) apk add --no-cache "$4" ;; apk) apk add --no-cache "$4" ;;
esac esac