deploy (#365)

Reviewed-on: #365

Dockerfile

@@ -88,7 +88,9 @@ RUN CGO_ENABLED=1 make build VERSION="$VERSION" GO_LDFLAGS='-extldflags "-static
 # alpine:3.21, 2026-03-17
 FROM alpine:3.21@sha256:c3f8e73fdb79deaebaa2037150150191b9dcbfba68b4a46d70103204c53f4709
 
-RUN apk --no-cache add ca-certificates
+# su-exec 0.2-r3 (Alpine 3.21), 2026-09-29: the entrypoint runs the app
+# as webhooker with it.
+RUN apk --no-cache add ca-certificates su-exec=0.2-r3
 
 # Create non-root user
 RUN addgroup -g 1000 -S webhooker && \
@@ -99,13 +101,17 @@ WORKDIR /app
 # Copy binary from builder
 COPY --from=builder /build/bin/webhooker /app/webhooker
 
+# Not under /app, which belongs to webhooker: this script runs as root.
+COPY deploy/docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
+
 # Create data directory for all SQLite databases (main app DB +
 # per-webhook event DBs). DATA_DIR defaults to /var/lib/webhooker.
 RUN mkdir -p /var/lib/webhooker
 
 RUN chown -R webhooker:webhooker /app /var/lib/webhooker
 
-USER webhooker
+# No USER: the entrypoint starts as root to make the data directory
+# webhooker's, then runs the app as webhooker.
 
 EXPOSE 8080
 
@@ -124,4 +130,5 @@ ENV BIND_ADDRESS=0.0.0.0
 HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
     CMD wget --no-verbose --tries=1 --spider http://localhost:8080/.well-known/healthcheck || exit 1
 
+ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]
 CMD ["/app/webhooker"]

README.md

@@ -538,6 +538,12 @@ its Argon2id hash. There is no second account and no forgot-password
 flow, so the banner and the reset command below are the only two ways
 in.
 
+A start that finds no `webhooker.db` in `DATA_DIR` also logs
+`created a new, empty database` at `WARN`, with the file's path,
+shortly before the banner. On a deployment that has run before, that
+line means `DATA_DIR` was empty, most often because its volume is not
+mounted.
+
 #### Recovering a lost admin password
 
 `webhooker resetpw` sets an existing account's password from the
@@ -552,8 +558,9 @@ printf '%s' "$NEW_PASSWORD" | \
     DATA_DIR=/var/lib/webhooker webhooker resetpw admin
 ```
 
-In a container it is the same binary, which the image sets as `CMD`
-rather than `ENTRYPOINT`, so the whole command has to be given:
+In a container it is the same binary. The image's `CMD` is
+`/app/webhooker`, and a command given to `docker run` replaces all of
+it, so the whole command has to be given:
 
 ```bash
 docker run --rm -v webhooker-data:/var/lib/webhooker \
@@ -690,38 +697,22 @@ those three values rather than trusting the figure. Measured at 65s on
 Docker 29.7.2.) A container `unhealthy` with `connection refused` in
 its health log, or a published port that resets connections, is this.
 
-The container runs as a non-root user (`webhooker`, UID 1000), exposes
-port 8080, and includes a health check against
-`/.well-known/healthcheck`. The `/var/lib/webhooker` volume holds all
-SQLite databases: the main application database (`webhooker.db`), the
-per-webhook event databases (`events-{uuid}.db`), and any archive
-databases written by `database` targets (`archive-{uuid}.db`). Mount
-this as a persistent volume to preserve data across container
-restarts.
+The app runs as a non-root user (`webhooker`, UID 1000), exposes port
+8080, and includes a health check against `/.well-known/healthcheck`.
+The `/var/lib/webhooker` volume holds all SQLite databases: the main
+application database (`webhooker.db`), the per-webhook event databases
+(`events-{uuid}.db`), and any archive databases written by `database`
+targets (`archive-{uuid}.db`). Mount this as a persistent volume to
+preserve data across container restarts.
 
-**The bind-mounted directory must be owned by UID 1000, or the
-container does not start.** Docker creates a `-v` source path that
-does not exist yet as `root:root`, and the process runs as UID 1000,
-so it cannot take its `DATA_DIR` lock:
-
-```
-webhooker: locking data directory /var/lib/webhooker: open
-/var/lib/webhooker/webhooker.lock: permission denied
-```
-
-It exits non-zero at that point, before opening any database. Create
-the directory ahead of the first `docker run`:
-
-```bash
-mkdir -p /path/to/data
-chown 1000:1000 /path/to/data
-chmod 750 /path/to/data
-```
-
-The same `chown` is what a restore needs — see step 4 of
-[Restore](#restore). A **named volume** does not have this problem:
-Docker copies the image's ownership onto a volume it initializes, and
-the image creates `/var/lib/webhooker` owned by `webhooker`.
+**The container sets its data directory's owner and mode itself
+before the app starts**, so a host directory can be mounted as it is,
+whoever owns it. The image's `ENTRYPOINT`,
+`deploy/docker-entrypoint.sh`, starts as root, creates `DATA_DIR` if
+it is missing, gives the directory and anything in it that belongs to
+another user to `webhooker`, sets the directory to `0750`, and only
+then runs the app as `webhooker`. Started with `--user`, it changes
+nothing and runs the app as that user.
 
 **The file modes are not yours to set, and do not depend on the
 directory.** `webhooker.db` holds target configuration in plaintext —
@@ -729,13 +720,10 @@ bearer tokens, API keys, Slack webhook URLs — along with the session
 encryption key, so webhooker creates every SQLite file it owns `0600`:
 each database and both of its `-wal` and `-shm` sidecars, across all
 three tiers. Files an earlier build left `0644` are tightened when
-they are opened. A `DATA_DIR` webhooker creates itself is `0750`, but
-a bind mount supplies its own directory and Docker's default for one
-it creates is `0755`; the `0600` files hold there regardless. The
-`chmod 750` above is defence in depth — it stops other local users
-listing the directory and learning your webhook UUIDs from the
-`events-{uuid}.db` filenames — not the barrier protecting the
-credentials.
+they are opened. The directory's `0750` is defence in depth — it stops
+other local users listing the directory and learning your webhook
+UUIDs from the `events-{uuid}.db` filenames — not the barrier
+protecting the credentials.
 
 ### Running under upaas
 
@@ -751,17 +739,6 @@ repository's `Dockerfile` and runs it. The app needs:
   app name, port `8080`. Leave `PORT` unset: the image's health check
   probes `8080`.
 - **Volume:** one host directory mounted at `/var/lib/webhooker`.
-  upaas bind-mounts the host path it is given and does not create it,
-  and the container does not start unless UID 1000 owns it (see
-  [Running with Docker](#running-with-docker)). Create it before the
-  first deploy:
-
-    ```bash
-    mkdir -p /path/to/data
-    chown 1000:1000 /path/to/data
-    chmod 750 /path/to/data
-    ```
-
 - **Environment variables:**
     - `WEBHOOKER_ENVIRONMENT=prod`
     - `TRUSTED_PROXIES`: your reverse proxy's address on that Docker
@@ -1020,12 +997,12 @@ done
 `.backup` reads through the WAL and writes a single consistent file with
 no sidecars of its own, so the destination is complete as it stands.
 Two caveats. First, the runtime image is `alpine:3.21` with only
-`ca-certificates` added — the `sqlite3` CLI is **not** in it, so run
-this on the host against the volume path, or from a throwaway container
-that mounts the volume. Second, each file is captured at its own
-instant, so a webhook created or an event delivered between two files
-being copied lands in one and not the other. If you need the whole set
-coherent as of a single moment, stop the service.
+`ca-certificates` and `su-exec` added — the `sqlite3` CLI is **not** in
+it, so run this on the host against the volume path, or from a
+throwaway container that mounts the volume. Second, each file is
+captured at its own instant, so a webhook created or an event delivered
+between two files being copied lands in one and not the other. If you
+need the whole set coherent as of a single moment, stop the service.
 
 Note that `sqlite3 <db> .dump` is **not** one of these procedures: it is
 an export, it holds a read transaction open for as long as it runs, and
@@ -1079,21 +1056,11 @@ with any `-wal`/`-shm` beside it, or wait until there are none.
    archive not opened since a crash. A copy salvaged from a crashed
    instance has them for everything, and needs all of them.
 
-4. **Fix ownership.** The container runs as the non-root `webhooker`
-   user, UID 1000 / GID 1000. Restored files must be owned by (or
-   writable by) that UID, and so must the directory itself — SQLite
-   creates the `-wal` and `-shm` sidecars beside the database, so a
-   writable file inside a directory it cannot write is not enough:
-
-    ```bash
-    chown -R 1000:1000 /path/to/data
-    ```
-
-    Restoring as `root` on the host and forgetting this step is the
-    usual way a restore fails.
-
-5. Start the service. `AutoMigrate` runs against each restored database
-   as it is opened.
+4. Start the service. The container gives the directory and the
+   restored files to the `webhooker` user before the app starts,
+   whoever restored them (see
+   [Running with Docker](#running-with-docker)). `AutoMigrate` runs
+   against each restored database as it is opened.
 
 ### Upgrades
 
@@ -3071,7 +3038,11 @@ check, see [The login endpoint](#the-login-endpoint).
 - Prometheus metrics behind basic auth
 - Static assets embedded in binary (no filesystem access needed at
   runtime)
-- Container runs as non-root user (UID 1000)
+- The app runs as the non-root `webhooker` user (UID 1000) in the
+  container. The image sets no `USER`, so these run as root: the
+  `ENTRYPOINT` script, which sets the data directory's owner and mode
+  before the app starts; the image's health check; and `docker exec`,
+  unless given `--user`
 - GORM soft deletes on every entity that carries `BaseModel`, which is
   all of them but `Setting` (data preserved for audit)
 
@@ -3198,10 +3169,13 @@ version is fixed independently of the compiler's:
    `GO_LDFLAGS`, so neither can drop the `-X` that stamps the version.
    The version arrives as the `VERSION` build arg, since the context
    has no `.git` (see [Version stamping](#version-stamping)).
-3. **Runtime stage** (`alpine:3.21`) — copies the static binary,
-   creates the `/var/lib/webhooker` directory for all SQLite databases,
-   runs as the non-root `webhooker` user (UID 1000), exposes port 8080,
-   and includes a health check against `/.well-known/healthcheck`.
+3. **Runtime stage** (`alpine:3.21`) — copies the static binary and
+   `deploy/docker-entrypoint.sh`, creates the `/var/lib/webhooker`
+   directory for all SQLite databases, exposes port 8080, and includes
+   a health check against `/.well-known/healthcheck`. It sets no
+   `USER`: the `ENTRYPOINT` script starts as root, sets the data
+   directory's owner and mode, and runs the app as the non-root
+   `webhooker` user (UID 1000) through `su-exec`.
 
 The lint stage invokes `golangci-lint` directly rather than `make lint`:
 it is already the pinned linter image, and `make lint` builds

deploy/docker-entrypoint.sh

@@ -0,0 +1,22 @@
+#!/bin/sh
+# deploy/docker-entrypoint.sh: the image's ENTRYPOINT. A bind-mounted
+# data directory keeps its owner from the host, often root, and the app
+# could not write to it. Started as root, this creates DATA_DIR if
+# needed, gives it and everything in it to webhooker, sets its mode, and
+# runs the command as webhooker, so the app never runs as root. Started
+# as another user, it only runs the command.
+set -eu
+
+main() {
+    if [ "$(id -u)" != 0 ]; then
+        exec "$@"
+    fi
+
+    dir="${DATA_DIR:-/var/lib/webhooker}"
+    mkdir -p "$dir"
+    find "$dir" ! -user webhooker -exec chown -h webhooker:webhooker {} +
+    chmod 750 "$dir"
+    exec su-exec webhooker "$@"
+}
+
+main "$@"

go.mod

@@ -4,7 +4,6 @@ go 1.26.1
 
 require (
 	github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8
-	github.com/dustin/go-humanize v1.0.1
 	github.com/getsentry/sentry-go v0.25.0
 	github.com/go-chi/chi v1.5.5
 	github.com/go-chi/cors v1.2.1
@@ -30,6 +29,7 @@ require (
 	github.com/beorn7/perks v1.0.1 // indirect
 	github.com/cespare/xxhash/v2 v2.2.0 // indirect
 	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
+	github.com/dustin/go-humanize v1.0.1 // indirect
 	github.com/gorilla/securecookie v1.1.2 // indirect
 	github.com/jinzhu/inflection v1.0.0 // indirect
 	github.com/jinzhu/now v1.1.5 // indirect

internal/database/bootstrap_banner_test.go

@@ -3,6 +3,8 @@ package database_test
 import (
 	"bytes"
 	"context"
+	"log/slog"
+	"path/filepath"
 	"strings"
 	"testing"
 
@@ -83,3 +85,37 @@ func TestFirstBoot_PrintsTheAdminPasswordAsABanner(t *testing.T) {
 		t, ok, "the printed password must open the seeded account",
 	)
 }
+
+// TestNewDatabase_IsLoggedWithItsPath is the log half of
+// https://git.eeqj.de/sneak/webhooker/issues/359. A DATA_DIR that is
+// unexpectedly empty boots exactly like a first start, so the start
+// that creates the database must say so, and where. Opening that
+// database again must not.
+func TestNewDatabase_IsLoggedWithItsPath(t *testing.T) {
+	t.Parallel()
+
+	dir := t.TempDir()
+
+	open := func() string {
+		var out bytes.Buffer
+
+		db, err := database.Open(dir, slog.New(slog.NewTextHandler(&out, nil)))
+		require.NoError(t, err)
+		require.NoError(t, db.Close())
+
+		return out.String()
+	}
+
+	const created = `level=WARN msg="created a new, empty database"`
+
+	first := open()
+	second := open()
+
+	assert.Contains(
+		t, first,
+		created+" path="+filepath.Join(dir, database.MainDBFileName),
+	)
+	assert.NotContains(
+		t, second, created, "an existing database is not new",
+	)
+}

internal/database/database.go

@@ -8,6 +8,7 @@ import (
 	"errors"
 	"fmt"
 	"io"
+	"io/fs"
 	"log/slog"
 	"os"
 	"path/filepath"
@@ -199,6 +200,12 @@ func (d *Database) connectTo(dataDir string) error {
 	// Construct the main application database path inside DATA_DIR.
 	dbPath := filepath.Join(dataDir, MainDBFileName)
 
+	// Checked before opening, which creates the file. A DATA_DIR that
+	// is unexpectedly empty -- its volume not mounted, say -- looks
+	// exactly like a first start, so a new database is a warning.
+	_, statErr := os.Stat(dbPath)
+	created := errors.Is(statErr, fs.ErrNotExist)
+
 	// Opened through OpenSQLite so this handle carries the same WAL
 	// journaling, busy timeout, immediate-transaction locking, and pool
 	// bounds as every other database file. See sqlite_open.go.
@@ -229,7 +236,12 @@ func (d *Database) connectTo(dataDir string) error {
 	}
 
 	d.db = db
-	d.log.Info("connected to database", "path", dbPath)
+
+	if created {
+		d.log.Warn("created a new, empty database", "path", dbPath)
+	} else {
+		d.log.Info("connected to database", "path", dbPath)
+	}
 
 	// Run migrations
 	return d.migrate()

internal/handlers/delivery_result_view.go

@@ -1,8 +1,6 @@
 package handlers
 
 import (
-	"time"
-
 	"sneak.berlin/go/webhooker/internal/delivery"
 )
 
@@ -27,7 +25,7 @@ const maxRenderedResponseBytes = 4096
 // cut, so an oversized stored response never becomes a Go
 // string at all.
 const deliveryResultColumns = "delivery_id, attempt_num, success, " +
-	"status_code, error, duration, created_at, " +
+	"status_code, error, duration, " +
 	"substr(cast(response_body as blob), 1, ?) AS response_body, " +
 	"length(cast(response_body as blob)) AS response_bytes"
 
@@ -108,10 +106,6 @@ type deliveryResultRow struct {
 	Duration      int64
 	ResponseBody  []byte
 	ResponseBytes int64
-
-	// CreatedAt is when the attempt's result was recorded, which
-	// is when the attempt finished.
-	CreatedAt time.Time
 }
 
 // view projects a loaded row for rendering, stripping the

internal/handlers/handlers.go

@@ -28,7 +28,7 @@ const (
 	// maxBodyShift is the bit shift for 1 MB body limit.
 	maxBodyShift = 20
 	// recentEventLimit is the number of recent events to show.
-	recentEventLimit = 50
+	recentEventLimit = 20
 	// paginationPerPage is the number of items per page.
 	paginationPerPage = 25
 

internal/handlers/recent_events.go

@@ -1,248 +0,0 @@
-package handlers
-
-import (
-	"net/http"
-	"strconv"
-	"time"
-
-	"github.com/dustin/go-humanize"
-	"gorm.io/gorm"
-	"sneak.berlin/go/webhooker/internal/database"
-)
-
-// recentEventColumns is the recent events list's projection. It
-// reads the body's size and never the body itself, for the reason
-// maxRenderedBodyBytes gives; the cast to blob makes length count
-// bytes rather than characters.
-const recentEventColumns = "id, created_at, method, content_type, " +
-	"resubmitted_from_id, length(cast(body as blob)) AS body_bytes"
-
-// RecentEventView is one row of the recent events list on a
-// webhook's page.
-type RecentEventView struct {
-	Method      string
-	ContentType string
-
-	// ResubmittedFromID names the event this one was copied from,
-	// empty for an event that arrived on the receiver.
-	ResubmittedFromID string
-
-	// Received is how long ago the event arrived, and ReceivedUTC
-	// the full timestamp the page shows on hover.
-	Received    string
-	ReceivedUTC string
-
-	// Size is the size of the stored body.
-	Size string
-
-	// ProcessingTime is how long the event's slowest delivery
-	// took; see processingTime.
-	ProcessingTime string
-
-	// Status is what the webhook's HTTP target answered, and
-	// StatusClass its colour; see targetStatus. Both are empty
-	// unless the webhook has exactly one HTTP target.
-	Status      string
-	StatusClass string
-}
-
-// recentEventRow is one row of recentEventColumns.
-type recentEventRow struct {
-	ID                string
-	CreatedAt         time.Time
-	Method            string
-	ContentType       string
-	ResubmittedFromID *string
-	BodyBytes         uint64
-}
-
-// singleHTTPTargetID returns the ID of the webhook's HTTP target
-// when it has exactly one, and "" when it has none or several.
-func singleHTTPTargetID(targets []database.Target) string {
-	id := ""
-	count := 0
-
-	for i := range targets {
-		if targets[i].Type == database.TargetTypeHTTP {
-			id = targets[i].ID
-			count++
-		}
-	}
-
-	if count != 1 {
-		return ""
-	}
-
-	return id
-}
-
-// loadRecentEvents loads the webhook's recentEventLimit newest
-// events for its page, newest first. statusTargetID is the
-// webhook's only HTTP target, or "" when the list shows no status.
-func (h *Handlers) loadRecentEvents(
-	webhookDB *gorm.DB, webhookID, statusTargetID string,
-) ([]RecentEventView, error) {
-	var rows []recentEventRow
-
-	err := webhookDB.Model(&database.Event{}).
-		Select(recentEventColumns).
-		Where("webhook_id = ?", webhookID).
-		Order("created_at DESC").
-		Limit(recentEventLimit).
-		Find(&rows).Error
-	if err != nil {
-		return nil, err
-	}
-
-	eventIDs := make([]string, len(rows))
-	for i := range rows {
-		eventIDs[i] = rows[i].ID
-	}
-
-	// Oldest first, so an event's last delivery to a target is its
-	// newest: a replay adds a delivery rather than changing the
-	// earlier one.
-	var deliveries []database.Delivery
-
-	err = webhookDB.
-		Select("id, event_id, target_id, status, created_at").
-		Where("event_id IN ?", eventIDs).
-		Order("created_at ASC").
-		Find(&deliveries).Error
-	if err != nil {
-		return nil, err
-	}
-
-	byEvent := make(map[string][]database.Delivery, len(rows))
-	deliveryIDs := make([]string, len(deliveries))
-
-	for i := range deliveries {
-		eventID := deliveries[i].EventID
-		byEvent[eventID] = append(byEvent[eventID], deliveries[i])
-		deliveryIDs[i] = deliveries[i].ID
-	}
-
-	attempts, err := h.loadDeliveryResults(webhookDB, deliveryIDs)
-	if err != nil {
-		return nil, err
-	}
-
-	views := make([]RecentEventView, len(rows))
-	for i := range rows {
-		views[i] = rows[i].view(
-			byEvent[rows[i].ID], attempts, statusTargetID,
-		)
-	}
-
-	return views, nil
-}
-
-// view projects a loaded row for rendering. deliveries is the
-// event's deliveries, oldest first, and attempts their recorded
-// attempts keyed by delivery ID.
-func (r *recentEventRow) view(
-	deliveries []database.Delivery,
-	attempts map[string][]deliveryResultRow,
-	statusTargetID string,
-) RecentEventView {
-	v := RecentEventView{
-		Method:         r.Method,
-		ContentType:    r.ContentType,
-		Received:       humanize.Time(r.CreatedAt),
-		ReceivedUTC:    r.CreatedAt.UTC().Format(time.DateTime) + " UTC",
-		Size:           humanize.Bytes(r.BodyBytes),
-		ProcessingTime: processingTime(deliveries, attempts),
-	}
-
-	if r.ResubmittedFromID != nil {
-		v.ResubmittedFromID = *r.ResubmittedFromID
-	}
-
-	if statusTargetID != "" {
-		v.Status, v.StatusClass = targetStatus(
-			deliveries, attempts, statusTargetID,
-		)
-	}
-
-	return v
-}
-
-// processingTime is how long the event's slowest delivery took,
-// from being queued to its last recorded attempt, time spent
-// waiting between retries included. A delivery is queued when its
-// event is received, or when an operator replays it, so a replay
-// is timed from the replay rather than from the event's arrival.
-// It is "in progress" while any delivery is pending or retrying,
-// and empty for an event with no deliveries.
-func processingTime(
-	deliveries []database.Delivery,
-	attempts map[string][]deliveryResultRow,
-) string {
-	if len(deliveries) == 0 {
-		return ""
-	}
-
-	var slowest time.Duration
-
-	for i := range deliveries {
-		if !deliveries[i].Status.Terminal() {
-			return "in progress"
-		}
-
-		tries := attempts[deliveries[i].ID]
-		if len(tries) == 0 {
-			continue
-		}
-
-		last := tries[len(tries)-1].CreatedAt
-		slowest = max(slowest, last.Sub(deliveries[i].CreatedAt))
-	}
-
-	return slowest.Round(time.Millisecond).String()
-}
-
-// targetStatus is what the target answered for the event, and the
-// colour to show it in: the HTTP status code of the last attempt of
-// the event's newest delivery to the target. Without a code it is
-// "no response" when that attempt failed before a response
-// arrived, the delivery's status ("pending") before any attempt,
-// and "not sent" when the event has no delivery to the target.
-func targetStatus(
-	deliveries []database.Delivery,
-	attempts map[string][]deliveryResultRow,
-	targetID string,
-) (string, string) {
-	newest := -1
-
-	for i := range deliveries {
-		if deliveries[i].TargetID == targetID {
-			newest = i
-		}
-	}
-
-	if newest < 0 {
-		return "not sent", "text-gray-400"
-	}
-
-	tries := attempts[deliveries[newest].ID]
-	if len(tries) == 0 {
-		return string(deliveries[newest].Status), "text-gray-400"
-	}
-
-	code := tries[len(tries)-1].StatusCode
-
-	switch {
-	case code == 0:
-		return "no response", "text-red-600"
-	case code >= http.StatusInternalServerError:
-		return strconv.Itoa(code), "text-red-600"
-	case code >= http.StatusBadRequest:
-		return strconv.Itoa(code), "text-yellow-600"
-	case code >= http.StatusMultipleChoices:
-		return strconv.Itoa(code), "text-gray-500"
-	case code >= http.StatusOK:
-		return strconv.Itoa(code), "text-green-600"
-	default:
-		return strconv.Itoa(code), "text-gray-500"
-	}
-}

internal/handlers/recent_events_test.go

@@ -1,295 +0,0 @@
-package handlers_test
-
-import (
-	"fmt"
-	"net/http"
-	"strings"
-	"testing"
-	"time"
-
-	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
-	"gorm.io/gorm"
-	"gorm.io/gorm/clause"
-	"sneak.berlin/go/webhooker/internal/database"
-	"sneak.berlin/go/webhooker/internal/handlers"
-	"sneak.berlin/go/webhooker/internal/session"
-)
-
-// statusTitle marks the status column's cell in a recent events
-// row; it is absent from the page when the column is not shown.
-const statusTitle = `title="HTTP status from the HTTP target"`
-
-// recentEventsFixture is one started app and a webhook whose
-// recent events list a test fills.
-type recentEventsFixture struct {
-	h         *handlers.Handlers
-	sess      *session.Session
-	db        *database.Database
-	webhook   *database.Webhook
-	webhookDB *gorm.DB
-}
-
-func newRecentEventsFixture(t *testing.T) *recentEventsFixture {
-	t.Helper()
-
-	f := &recentEventsFixture{}
-
-	var dbMgr *database.WebhookDBManager
-
-	app := newTestApp(t, &f.h, &f.sess, &f.db, &dbMgr)
-	app.RequireStart()
-
-	t.Cleanup(app.RequireStop)
-
-	f.webhook = seedWebhook(t, f.db)
-
-	webhookDB, err := dbMgr.GetDB(f.webhook.ID)
-	require.NoError(t, err)
-
-	f.webhookDB = webhookDB
-
-	return f
-}
-
-func (f *recentEventsFixture) render(t *testing.T) string {
-	t.Helper()
-
-	return renderSourceDetailPage(t, f.h, f.sess, f.webhook.ID)
-}
-
-// event records an event received at receivedAt.
-func (f *recentEventsFixture) event(
-	t *testing.T, contentType, body string, receivedAt time.Time,
-) *database.Event {
-	t.Helper()
-
-	event := &database.Event{
-		WebhookID:   f.webhook.ID,
-		Method:      http.MethodPost,
-		Body:        body,
-		ContentType: contentType,
-	}
-	event.CreatedAt = receivedAt
-
-	require.NoError(t, f.webhookDB.Omit(
-		clause.Associations,
-	).Create(event).Error)
-
-	return event
-}
-
-// delivery records a delivery of the event to the target, queued
-// when the event was received.
-func (f *recentEventsFixture) delivery(
-	t *testing.T,
-	event *database.Event,
-	targetID string,
-	status database.DeliveryStatus,
-) *database.Delivery {
-	t.Helper()
-
-	return f.deliveryQueuedAt(
-		t, event, targetID, status, event.CreatedAt,
-	)
-}
-
-// deliveryQueuedAt records a delivery of the event to the target,
-// queued at queuedAt, as a replay is.
-func (f *recentEventsFixture) deliveryQueuedAt(
-	t *testing.T,
-	event *database.Event,
-	targetID string,
-	status database.DeliveryStatus,
-	queuedAt time.Time,
-) *database.Delivery {
-	t.Helper()
-
-	dlv := &database.Delivery{
-		EventID:  event.ID,
-		TargetID: targetID,
-		Status:   status,
-	}
-	dlv.CreatedAt = queuedAt
-
-	require.NoError(t, f.webhookDB.Omit(
-		clause.Associations,
-	).Create(dlv).Error)
-
-	return dlv
-}
-
-// attempt records one attempt of the delivery that finished took
-// after the delivery was queued, with HTTP status code (0 for no
-// response).
-func (f *recentEventsFixture) attempt(
-	t *testing.T, dlv *database.Delivery, code int, took time.Duration,
-) {
-	t.Helper()
-
-	result := &database.DeliveryResult{
-		DeliveryID: dlv.ID,
-		AttemptNum: 1,
-		StatusCode: code,
-	}
-	result.CreatedAt = dlv.CreatedAt.Add(took)
-
-	require.NoError(t, f.webhookDB.Omit(
-		clause.Associations,
-	).Create(result).Error)
-}
-
-// statusCell is the status column's cell as the page renders it.
-func statusCell(class, text string) string {
-	return `<span class="font-medium ` + class + `" ` + statusTitle +
-		`>` + text + `</span>`
-}
-
-// TestHandleSourceDetail_ShowsFiftyNewestEvents proves the list
-// holds the 50 newest events, newest first, and not one more.
-func TestHandleSourceDetail_ShowsFiftyNewestEvents(t *testing.T) {
-	t.Parallel()
-
-	f := newRecentEventsFixture(t)
-	base := time.Now().Add(-time.Hour)
-
-	for i := range 51 {
-		f.event(
-			t, fmt.Sprintf("application/x-recent-%02d", i), "{}",
-			base.Add(time.Duration(i)*time.Second),
-		)
-	}
-
-	body := f.render(t)
-
-	assert.Equal(t, 50, strings.Count(body, `title="Body size"`))
-	assert.NotContains(t, body, "application/x-recent-00")
-	assert.Contains(t, body, "application/x-recent-01")
-	assert.Less(
-		t,
-		strings.Index(body, "application/x-recent-50"),
-		strings.Index(body, "application/x-recent-49"),
-	)
-}
-
-// TestHandleSourceDetail_RecentEventColumns proves a row shows its
-// time relative with the UTC timestamp on hover, its body size,
-// and its processing time once every delivery has finished.
-func TestHandleSourceDetail_RecentEventColumns(t *testing.T) {
-	t.Parallel()
-
-	f := newRecentEventsFixture(t)
-	logTarget := seedTarget(t, f.db, f.webhook.ID, database.TargetTypeLog)
-
-	receivedAt := time.Now().Add(-210 * time.Second).
-		UTC().Truncate(time.Second)
-
-	done := f.event(
-		t, contentTypeJSON, strings.Repeat("x", 2048), receivedAt,
-	)
-	f.attempt(
-		t,
-		f.delivery(t, done, logTarget.ID, database.DeliveryStatusDelivered),
-		0, 1500*time.Millisecond,
-	)
-
-	waiting := f.event(t, "text/plain", "{}", receivedAt)
-	f.delivery(t, waiting, logTarget.ID, database.DeliveryStatusPending)
-
-	body := f.render(t)
-
-	assert.Contains(
-		t, body,
-		`<span title="`+receivedAt.Format(time.DateTime)+
-			` UTC">3 minutes ago</span>`,
-	)
-	assert.Contains(t, body, `<span title="Body size">2.0 kB</span>`)
-	assert.Contains(t, body, ">1.5s</span>")
-	assert.Contains(t, body, ">in progress</span>")
-}
-
-// TestHandleSourceDetail_StatusWithSingleHTTPTarget proves that a
-// webhook with exactly one HTTP target shows, colour-coded, what
-// that target answered for each event. The log target beside it
-// does not count against "exactly one".
-func TestHandleSourceDetail_StatusWithSingleHTTPTarget(t *testing.T) {
-	t.Parallel()
-
-	f := newRecentEventsFixture(t)
-	target := seedTarget(t, f.db, f.webhook.ID, database.TargetTypeHTTP)
-	seedTarget(t, f.db, f.webhook.ID, database.TargetTypeLog)
-
-	now := time.Now()
-
-	for _, code := range []int{204, 302, 404, 503, 0} {
-		dlv := f.delivery(
-			t, f.event(t, contentTypeJSON, "{}", now), target.ID,
-			database.DeliveryStatusDelivered,
-		)
-		f.attempt(t, dlv, code, time.Second)
-	}
-
-	f.delivery(
-		t, f.event(t, contentTypeJSON, "{}", now), target.ID,
-		database.DeliveryStatusPending,
-	)
-	f.event(t, contentTypeJSON, "{}", now)
-
-	// A replay is a newer delivery, and its answer is the one shown.
-	replayed := f.event(t, contentTypeJSON, "{}", now)
-	f.attempt(t, f.delivery(
-		t, replayed, target.ID, database.DeliveryStatusFailed,
-	), 502, time.Second)
-	f.attempt(t, f.deliveryQueuedAt(
-		t, replayed, target.ID, database.DeliveryStatusDelivered,
-		now.Add(time.Minute),
-	), 200, time.Second)
-
-	body := f.render(t)
-
-	assert.Contains(t, body, statusCell("text-green-600", "204"))
-	assert.Contains(t, body, statusCell("text-gray-500", "302"))
-	assert.Contains(t, body, statusCell("text-yellow-600", "404"))
-	assert.Contains(t, body, statusCell("text-red-600", "503"))
-	assert.Contains(t, body, statusCell("text-red-600", "no response"))
-	assert.Contains(t, body, statusCell("text-gray-400", "pending"))
-	assert.Contains(t, body, statusCell("text-gray-400", "not sent"))
-	assert.Contains(t, body, statusCell("text-green-600", "200"))
-	assert.NotContains(t, body, ">502<")
-}
-
-// TestHandleSourceDetail_NoStatusWithoutSingleHTTPTarget proves the
-// status column is absent when the webhook has no HTTP target or
-// more than one.
-func TestHandleSourceDetail_NoStatusWithoutSingleHTTPTarget(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	cases := map[string][]database.TargetType{
-		"none":    {database.TargetTypeLog},
-		"several": {database.TargetTypeHTTP, database.TargetTypeHTTP},
-	}
-
-	for name, types := range cases {
-		t.Run(name, func(t *testing.T) {
-			t.Parallel()
-
-			f := newRecentEventsFixture(t)
-			event := f.event(t, contentTypeJSON, "{}", time.Now())
-
-			for _, tt := range types {
-				target := seedTarget(t, f.db, f.webhook.ID, tt)
-				f.attempt(t, f.delivery(
-					t, event, target.ID,
-					database.DeliveryStatusDelivered,
-				), 200, time.Second)
-			}
-
-			body := f.render(t)
-
-			assert.Contains(t, body, `title="Body size"`)
-			assert.NotContains(t, body, statusTitle)
-		})
-	}
-}

internal/handlers/source_management.go

@@ -415,21 +415,16 @@ func (h *Handlers) renderSourceDetail(
 		"webhook_id = ?", webhook.ID,
 	).Find(&targets)
 
-	var events []RecentEventView
+	var events []database.Event
 
 	if h.dbMgr.DBExists(webhook.ID) {
 		webhookDB, dbErr := h.dbMgr.GetDB(webhook.ID)
 		if dbErr == nil {
-			events, dbErr = h.loadRecentEvents(
-				webhookDB, webhook.ID, singleHTTPTargetID(targets),
-			)
-		}
-
-		if dbErr != nil {
-			h.log.Error(
-				"failed to load recent events",
-				"webhook_id", webhook.ID, "error", dbErr,
-			)
+			webhookDB.Where(
+				"webhook_id = ?", webhook.ID,
+			).Order("created_at DESC").Limit(
+				recentEventLimit,
+			).Find(&events)
 		}
 	}
 

internal/server/routes_test.go

@@ -7,6 +7,7 @@ import (
 	"net/http/httptest"
 	"net/url"
 	"regexp"
+	"slices"
 	"strconv"
 	"strings"
 	"testing"
@@ -220,9 +221,21 @@ func (e *testEnv) csrfFrom(
 	// out of the markup has to be unescaped before it is submitted.
 	token := html.UnescapeString(match[1])
 
-	combined := make([]*http.Cookie, 0, len(cookies))
-	combined = append(combined, cookies...)
-	combined = append(combined, w.Result().Cookies()...)
+	// A cookie the page sets replaces the one of the same name, as in
+	// a browser. Sent both, the server would read the first, older one.
+	set := w.Result().Cookies()
+	combined := make([]*http.Cookie, 0, len(cookies)+len(set))
+
+	for _, c := range cookies {
+		replaced := slices.ContainsFunc(set, func(n *http.Cookie) bool {
+			return n.Name == c.Name
+		})
+		if !replaced {
+			combined = append(combined, c)
+		}
+	}
+
+	combined = append(combined, set...)
 
 	return token, combined
 }
@@ -614,6 +627,59 @@ func TestPagesLogin_CorrectPasswordSurvivesASpentBudget(
 	)
 }
 
+// TestPagesLogin_CookiesFromAnEarlierDatabase is
+// https://git.eeqj.de/sneak/webhooker/issues/359. A new database
+// brings a new session key, and the operator's browser still holds
+// the session and CSRF cookies signed with the old one. Logging in
+// must work as from a fresh browser and leave cookies the new key
+// accepts.
+func TestPagesLogin_CookiesFromAnEarlierDatabase(t *testing.T) {
+	t.Parallel()
+
+	const (
+		username = "operator"
+		password = "correct-horse-battery-staple"
+	)
+
+	earlier := newTestEnv(t)
+	earlierID, _ := earlier.seedUser(t, username, password)
+	_, stale := earlier.csrfFrom(t, "/pages/login", nil)
+	stale = append(stale, earlier.authCookies(t, earlierID, username)...)
+
+	env := newTestEnv(t)
+	env.seedUser(t, username, password)
+
+	token, cookies := env.csrfFrom(t, "/pages/login", stale)
+
+	form := url.Values{}
+	form.Set("csrf_token", token)
+	form.Set("username", username)
+	form.Set("password", password)
+
+	w := env.post("/pages/login", form, cookies)
+	require.Equal(
+		t, http.StatusSeeOther, w.Code,
+		"a session cookie from another key must not fail the login",
+	)
+
+	// The response deletes the old session cookie and then sets the
+	// new one; a browser keeps the last.
+	var fresh *http.Cookie
+
+	for _, c := range w.Result().Cookies() {
+		if c.Name == session.SessionName {
+			fresh = c
+		}
+	}
+
+	require.NotNil(t, fresh, "login must set a session cookie")
+	assert.Equal(
+		t, "/sources",
+		env.get("/", []*http.Cookie{fresh}).Header().Get("Location"),
+		"the new session cookie must authenticate",
+	)
+}
+
 // --- /user/{username} group ---
 
 // TestPasswordChange_OversizeBody_RejectedAndPasswordUnchanged

internal/session/codec_test.go

@@ -19,8 +19,8 @@ import (
 )
 
 // The tests below exercise the securecookie codecs underneath the
-// store and nothing else: Session.Get only decodes, so no server-side
-// expiry check takes part in the result. They exist because
+// store and nothing else: they decode through the store itself, so no
+// server-side expiry check takes part in the result. They exist because
 // NewCookieStore gives its codecs a 30-day max age that assigning
 // store.Options does not override, which would let the codec accept a
 // cookie weeks past the cap the cookie attribute advertises.
@@ -75,10 +75,11 @@ func restamp(
 	return base64.URLEncoding.EncodeToString(payload)
 }
 
-// decodeCookie feeds value back through the store's decode path.
+// decodeCookie feeds value back through the store's decode path. It
+// asks the store rather than Session.Get, which treats a cookie that
+// does not decode as absent and so hides the codec's reason.
 func decodeCookie(
 	t *testing.T,
-	s *session.Session,
 	value string,
 ) (*sessions.Session, error) {
 	t.Helper()
@@ -94,7 +95,7 @@ func decodeCookie(
 		SameSite: http.SameSiteLaxMode,
 	})
 
-	sess, err := s.Get(req)
+	sess, err := session.NewStore(testKey()).Get(req, session.SessionName)
 	require.NotNil(t, sess)
 
 	return sess, err
@@ -105,7 +106,7 @@ func TestCodec_AcceptsCookieInsideAbsoluteCap(t *testing.T) {
 
 	s := testSession(t)
 
-	sess, err := decodeCookie(t, s, restamp(
+	sess, err := decodeCookie(t, restamp(
 		t,
 		issuedCookie(t, s),
 		time.Now().Add(-(testAbsoluteMaxAge-time.Hour)),
@@ -126,7 +127,7 @@ func TestCodec_RejectsCookiePastAbsoluteCap(t *testing.T) {
 
 	s := testSession(t)
 
-	sess, err := decodeCookie(t, s, restamp(
+	sess, err := decodeCookie(t, restamp(
 		t,
 		issuedCookie(t, s),
 		time.Now().Add(-(testAbsoluteMaxAge+time.Hour)),

internal/session/session.go

@@ -224,10 +224,22 @@ func New(
 }
 
 // Get retrieves a session for the request.
+//
+// A session cookie that does not decode -- one signed with an earlier
+// session key, say, because the database was made anew -- is treated
+// as absent: the caller gets a new, empty session and no error, and
+// the next save replaces the cookie.
 func (s *Session) Get(
 	r *http.Request,
 ) (*sessions.Session, error) {
-	return s.store.Get(r, SessionName)
+	sess, err := s.store.Get(r, SessionName)
+	if sess == nil {
+		return nil, err
+	}
+
+	// For a cookie that does not decode, gorilla/sessions returns a
+	// new, empty session alongside the error that is dropped here.
+	return sess, nil
 }
 
 // GetKey returns the raw 32-byte authentication key used for

templates/source_detail.html

@@ -187,24 +187,12 @@
         <div class="divide-y divide-gray-100">
             {{range .Events}}
             <div class="p-4">
-                <div class="flex flex-wrap items-center justify-between gap-3">
-                    <div class="flex flex-wrap items-center gap-3">
+                <div class="flex items-center justify-between">
+                    <div class="flex items-center gap-3">
                         <span class="badge-info">{{.Method}}</span>
-                        <span class="text-sm text-gray-500 break-all">{{.ContentType}}</span>
-                        {{if .ResubmittedFromID}}
-                        <span class="text-xs text-gray-500" title="This event is a copy of {{.ResubmittedFromID}}">resubmitted copy</span>
-                        {{end}}
-                    </div>
-                    <div class="flex flex-wrap items-center gap-3 text-xs text-gray-400">
-                        <span title="Body size">{{.Size}}</span>
-                        {{if .ProcessingTime}}
-                        <span title="Processing time: how long the slowest delivery took, from being queued to its last attempt">{{.ProcessingTime}}</span>
-                        {{end}}
-                        {{if .Status}}
-                        <span class="font-medium {{.StatusClass}}" title="HTTP status from the HTTP target">{{.Status}}</span>
-                        {{end}}
-                        <span title="{{.ReceivedUTC}}">{{.Received}}</span>
+                        <span class="text-sm text-gray-500">{{.ContentType}}</span>
                     </div>
+                    <span class="text-xs text-gray-400">{{.CreatedAt.Format "2006-01-02 15:04:05 UTC"}}</span>
                 </div>
             </div>
             {{else}}