Align DeliveryResult tags and make the index table test-local

Re-align the json tag column in DeliveryResult after the wider
DeliveryID gorm tag, and move the test's index table out of package
scope into the test function.

Model: opus-5-5

README.md

@@ -731,6 +731,66 @@ listing the directory and learning your webhook UUIDs from the
 `events-{uuid}.db` filenames — not the barrier protecting the
 credentials.
 
+### Running under upaas
+
+[upaas](https://git.eeqj.de/sneak/upaas) builds the image from this
+repository's `Dockerfile` and runs it. The app needs:
+
+- **Network and port:** add no port mapping in upaas. upaas publishes
+  every mapped port on all interfaces of the host
+  ([upaas issue 113](https://git.eeqj.de/sneak/upaas/issues/113)),
+  which would put the plain-HTTP admin UI and receiver there. Instead,
+  set the app's Docker Network in upaas to your reverse proxy's Docker
+  network; the proxy then reaches the app at `upaas-` followed by the
+  app name, port `8080`. Leave `PORT` unset: the image's health check
+  probes `8080`.
+- **Volume:** one host directory mounted at `/var/lib/webhooker`.
+  upaas bind-mounts the host path it is given and does not create it,
+  and the container does not start unless UID 1000 owns it (see
+  [Running with Docker](#running-with-docker)). Create it before the
+  first deploy:
+
+    ```bash
+    mkdir -p /path/to/data
+    chown 1000:1000 /path/to/data
+    chmod 750 /path/to/data
+    ```
+
+- **Environment variables:**
+    - `WEBHOOKER_ENVIRONMENT=prod`
+    - `TRUSTED_PROXIES`: your reverse proxy's address on that Docker
+      network. The `remoteIP` field of the `http request` log line for a
+      request that came through the proxy shows it; the health check's
+      own lines show `::1`. See [Trusted proxies](#trusted-proxies).
+    - Leave `BIND_ADDRESS` and `DATA_DIR` unset: the image sets
+      `BIND_ADDRESS` to `0.0.0.0`, and `DATA_DIR` defaults to
+      `/var/lib/webhooker`.
+    - Everything else is optional; see [Configuration](#configuration).
+- **Health check:** the image's own, which requests
+  `/.well-known/healthcheck`. upaas reads the container's health 60
+  seconds after a deploy and marks the deploy failed unless it is
+  `healthy`.
+- **First run:** the first start prints the `admin` password once, in
+  the banner described under [The admin account](#the-admin-account),
+  to the container's log. upaas names the container `upaas-` followed
+  by the app name, so for an app named `webhooker`:
+
+    ```bash
+    docker logs upaas-webhooker
+    ```
+
+  If the password is lost, stop the container, set a new password with
+  the app's own image and volume, and start it again (see
+  [Recovering a lost admin password](#recovering-a-lost-admin-password)):
+
+    ```bash
+    docker stop upaas-webhooker
+    docker run --rm --volumes-from upaas-webhooker \
+        "$(docker inspect -f '{{.Image}}' upaas-webhooker)" \
+        /app/webhooker resetpw -generate admin
+    docker start upaas-webhooker
+    ```
+
 ## Deployment behind a reverse proxy
 
 webhooker terminates no TLS of its own. It serves plaintext HTTP and
@@ -1507,7 +1567,7 @@ events should be forwarded.
 | `type`           | TargetType | One of: `http`, `slack`, `database`, `log` |
 | `active`         | boolean    | Whether deliveries are enabled (default: true) |
 | `config`         | JSON text  | Type-specific configuration |
-| `max_retries`    | integer    | Maximum retry attempts for `http` and `slack` targets (0 = fire-and-forget, >0 = retries with backoff and a circuit breaker). Ignored by `database` and `log` targets |
+| `max_retries`    | integer    | Total delivery attempts for `http` and `slack` targets, not retries on top of the first: 0 is a single fire-and-forget attempt with no retries and no circuit breaker, and a value of N makes N attempts in all, with exponential backoff and a per-target circuit breaker. Ignored by `database` and `log` targets |
 | `max_queue_size` | integer    | Stored and shown on the target's detail view, but not enforced anywhere yet: nothing in the delivery engine consults it. Queue depth is set by the two fixed 10,000-entry channels |
 
 **Relations:** Belongs to Webhook. Has many Deliveries.
@@ -1515,12 +1575,12 @@ events should be forwarded.
 **Target types:**
 
 - **`http`** — Forward the event as an HTTP POST to a configured URL.
-  Behavior depends on `max_retries`: when `max_retries` is 0 (the
+  `max_retries` is the total number of delivery attempts, not retries on
-  default), the target operates in fire-and-forget mode — a single
+  top of the first: when `max_retries` is 0 (the default), the target
-  attempt with no retries and no circuit breaker. When `max_retries` is
+  operates in fire-and-forget mode, a single attempt with no retries and
-  greater than 0, failed deliveries are retried with exponential backoff
+  no circuit breaker; a value of N makes up to N attempts in all,
-  up to `max_retries` attempts, protected by a per-target circuit
+  retrying failed deliveries with exponential backoff and protecting them
-  breaker.
+  with a per-target circuit breaker.
 - **`slack`** — Post the event as a formatted message to a
   Slack-compatible incoming webhook URL (`webhookUrl` in `config`). It
   is built on the same HTTP core as `http` and honours `max_retries`
@@ -1700,6 +1760,22 @@ retries) is individually logged for full observability.
 
 **Relations:** Belongs to Delivery.
 
+#### Event-tier indexes
+
+Beyond the primary keys, the per-webhook event databases carry secondary
+indexes on the columns the background work reads by, each created by
+`AutoMigrate` on a fresh and on an existing database:
+
+| Column                         | Serves |
+| ------------------------------ | ------ |
+| `deliveries.status`            | The recovery and sweep queries that select deliveries by status once a minute |
+| `deliveries.event_id`          | Loading a page of the event log, which reads deliveries by event |
+| `delivery_results.delivery_id` | Loading a page of the event log, which reads results by delivery |
+| `events.created_at`            | Retention, which deletes events by age |
+
+The `events.resubmitted_from_id` column is also indexed, to resolve the
+resubmit relationship both ways in the event log.
+
 #### Common Fields
 
 Every entity except `Setting` includes these fields from `BaseModel`.

internal/database/database.go

@@ -17,12 +17,12 @@ import (
 	"gorm.io/gorm"
 	"sneak.berlin/go/webhooker/internal/banner"
 	"sneak.berlin/go/webhooker/internal/config"
+	"sneak.berlin/go/webhooker/internal/datadir"
 	"sneak.berlin/go/webhooker/internal/gormlog"
 	"sneak.berlin/go/webhooker/internal/logger"
 )
 
 const (
-	dataDirPerm       = 0750
 	randomPasswordLen = 16
 	sessionKeyLen     = 32
 )
@@ -185,7 +185,9 @@ func (d *Database) connect() error {
 // caller's decision.
 func (d *Database) connectTo(dataDir string) error {
 	// Ensure the data directory exists before opening the database.
-	err := os.MkdirAll(dataDir, dataDirPerm)
+	// datadir.DirPerm is the single source of the directory mode; this
+	// package creates the directory too, since either may run first.
+	err := os.MkdirAll(dataDir, datadir.DirPerm)
 	if err != nil {
 		return fmt.Errorf(
 			"creating data directory %s: %w",

internal/database/event_tier_indexes_test.go

@@ -0,0 +1,72 @@
+package database_test
+
+import (
+	"context"
+	"testing"
+
+	"github.com/google/uuid"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"sneak.berlin/go/webhooker/internal/database"
+)
+
+// indexedColumn names a secondary index by the model and struct field
+// GORM derives the index name from.
+type indexedColumn struct {
+	model any
+	field string
+}
+
+// TestWebhookDBManager_OpenAddsEventTierIndexes verifies that opening a
+// per-webhook database that predates these indexes creates them, so the
+// queries that read by those columns stop scanning whole tables. It
+// stands in for an older database file by dropping the indexes
+// AutoMigrate just created, then reopening the same file.
+func TestWebhookDBManager_OpenAddsEventTierIndexes(t *testing.T) {
+	t.Parallel()
+
+	// The columns the background work reads by: the recovery and sweep
+	// queries (status), the event log (event_id and delivery_id) and
+	// retention (created_at).
+	eventTierIndexes := []indexedColumn{
+		{&database.Delivery{}, "Status"},
+		{&database.Delivery{}, "EventID"},
+		{&database.DeliveryResult{}, "DeliveryID"},
+		{&database.Event{}, "CreatedAt"},
+	}
+
+	mgr, lc := setupTestWebhookDBManager(t)
+	ctx := context.Background()
+	require.NoError(t, lc.Start(ctx))
+
+	defer func() { require.NoError(t, lc.Stop(ctx)) }()
+
+	webhookID := uuid.New().String()
+
+	db, err := mgr.GetDB(webhookID)
+	require.NoError(t, err)
+
+	// A fresh database has them.
+	for _, ix := range eventTierIndexes {
+		require.True(t, db.Migrator().HasIndex(ix.model, ix.field))
+	}
+
+	// Stand in for a database file created before the indexes existed.
+	for _, ix := range eventTierIndexes {
+		require.NoError(t, db.Migrator().DropIndex(ix.model, ix.field))
+		require.False(t, db.Migrator().HasIndex(ix.model, ix.field))
+	}
+
+	// Drop the cached connection so the next open reopens the file and
+	// runs AutoMigrate against it, as a restart would.
+	require.NoError(t, mgr.CloseAll())
+
+	db, err = mgr.GetDB(webhookID)
+	require.NoError(t, err)
+
+	for _, ix := range eventTierIndexes {
+		assert.True(t, db.Migrator().HasIndex(ix.model, ix.field),
+			"opening the existing database should create the index on %s",
+			ix.field)
+	}
+}

internal/database/model_delivery.go

@@ -32,9 +32,9 @@ func (s DeliveryStatus) Terminal() bool {
 type Delivery struct {
 	BaseModel
 
-	EventID  string         `gorm:"type:uuid;not null"         json:"eventId"`
+	EventID  string         `gorm:"type:uuid;not null;index"         json:"eventId"`
-	TargetID string         `gorm:"type:uuid;not null"         json:"targetId"`
+	TargetID string         `gorm:"type:uuid;not null"               json:"targetId"`
-	Status   DeliveryStatus `gorm:"not null;default:'pending'" json:"status"`
+	Status   DeliveryStatus `gorm:"not null;default:'pending';index" json:"status"`
 
 	// Relations
 	Event           Event            `json:"event,omitzero"`

internal/database/model_delivery_result.go

@@ -4,11 +4,11 @@ package database
 type DeliveryResult struct {
 	BaseModel
 
-	DeliveryID   string `gorm:"type:uuid;not null"   json:"deliveryId"`
+	DeliveryID   string `gorm:"type:uuid;not null;index" json:"deliveryId"`
-	AttemptNum   int    `gorm:"not null"             json:"attemptNum"`
+	AttemptNum   int    `gorm:"not null"                 json:"attemptNum"`
 	Success      bool   `json:"success"`
 	StatusCode   int    `json:"statusCode,omitempty"`
-	ResponseBody string `gorm:"type:text"            json:"responseBody,omitempty"`
+	ResponseBody string `gorm:"type:text"                json:"responseBody,omitempty"`
 	Error        string `json:"error,omitempty"`
 	Duration     int64  `json:"durationMs"` // Duration in milliseconds
 

internal/database/model_event.go

@@ -1,9 +1,17 @@
 package database
 
+import "time"
+
 // Event represents a captured webhook event
 type Event struct {
 	BaseModel
 
+	// CreatedAt overrides BaseModel.CreatedAt only to add an index:
+	// retention deletes events by age, so events.created_at is queried
+	// on every sweep. The other tables keep the unindexed BaseModel
+	// field.
+	CreatedAt time.Time `gorm:"index" json:"createdAt"`
+
 	WebhookID    string `gorm:"type:uuid;not null" json:"webhookId"`
 	EntrypointID string `gorm:"type:uuid;not null" json:"entrypointId"`
 

internal/database/webhook_db_manager.go

@@ -13,6 +13,7 @@ import (
 	"gorm.io/driver/sqlite"
 	"gorm.io/gorm"
 	"sneak.berlin/go/webhooker/internal/config"
+	"sneak.berlin/go/webhooker/internal/datadir"
 	"sneak.berlin/go/webhooker/internal/gormlog"
 	"sneak.berlin/go/webhooker/internal/logger"
 )
@@ -53,8 +54,9 @@ func NewWebhookDBManager(
 		log:     params.Logger.Get(),
 	}
 
-	// Create data directory if it doesn't exist
+	// Create data directory if it doesn't exist. datadir.DirPerm is the
-	err := os.MkdirAll(m.dataDir, dataDirPerm)
+	// single source of the directory mode; either package may run first.
+	err := os.MkdirAll(m.dataDir, datadir.DirPerm)
 	if err != nil {
 		return nil, fmt.Errorf(
 			"creating data directory %s: %w",

internal/datadir/lock.go

@@ -29,9 +29,11 @@ import (
 // process that was killed with SIGKILL blocks nothing.
 const LockFileName = "webhooker.lock"
 
-// dirPerm is the mode Acquire creates DATA_DIR with. It matches what
+// DirPerm is the mode DATA_DIR is created with. It is the single
-// internal/database uses, since whichever runs first creates it.
+// source of that mode: internal/database consumes it rather than
-const dirPerm = 0o750
+// keeping its own copy, so the two packages that both create the
+// directory cannot drift into disagreeing about its permissions.
+const DirPerm = 0o750
 
 // ErrLocked reports that another live process holds the data
 // directory. Callers that need to know whether a deployment is running
@@ -64,7 +66,7 @@ func Acquire(dir string) (*Lock, error) {
 		return nil, ErrNoDir
 	}
 
-	err := os.MkdirAll(dir, dirPerm)
+	err := os.MkdirAll(dir, DirPerm)
 	if err != nil {
 		return nil, fmt.Errorf(
 			"creating data directory %s: %w", dir, err,

internal/handlers/ui_copy_test.go

@@ -300,3 +300,80 @@ func TestEntrypointCopyButtonIsProgressiveEnhancement(t *testing.T) {
 		"the page must render to completion, not abort partway",
 	)
 }
+
+// maxRetriesHelp is the wording both target forms must carry. The
+// delivery core makes max_retries attempts in total, not that many
+// retries on top of a first try (a fresh delivery starts at attempt 1
+// and target_http gives up once the attempt number reaches
+// max_retries), and 0 is special-cased to a single fire-and-forget
+// attempt with no circuit breaker.
+const maxRetriesHelp = "This is the total number of delivery attempts, " +
+	"not retries on top of the first: a value of 3 makes three attempts " +
+	"in all. 0 means a single attempt with no retries and no circuit " +
+	"breaker."
+
+// TestTargetFormMaxRetriesCopyMatchesBehaviour pins the max_retries
+// help text on both the create form (the add-target form on the webhook
+// detail page) and the edit form, so the copy cannot drift back to
+// calling the number a retry count.
+func TestTargetFormMaxRetriesCopyMatchesBehaviour(t *testing.T) {
+	t.Parallel()
+
+	var h *handlers.Handlers
+
+	var sess *session.Session
+
+	app := newTestApp(t, &h, &sess)
+	app.RequireStart()
+
+	t.Cleanup(app.RequireStop)
+
+	webhook := &database.Webhook{Name: "wh", RetentionDays: 14}
+	webhook.ID = testWebhookID
+
+	entrypoint := database.Entrypoint{Path: "abc123"}
+	entrypoint.ID = "ep-1"
+
+	createBody := renderPage(
+		t, h, sess, "source_detail.html", map[string]any{
+			dataKeyWebhook: webhook,
+			"Entrypoints": handlers.NewEntrypointViews(
+				[]database.Entrypoint{entrypoint},
+			),
+			"Targets": delivery.NewTargetViews(nil),
+			"Events":  []database.Event{},
+			"BaseURL": "https://hooks.example.com",
+		},
+	)
+
+	assert.Contains(
+		t, createBody, maxRetriesHelp,
+		"the add-target form must explain max_retries as total attempts",
+	)
+
+	// A slack target exercises the same max_retries field while needing
+	// only Config.URL from the edit template, so the test data stays
+	// minimal. The Target key mirrors the field names the template reads
+	// off the handler's view value.
+	editBody := renderPage(
+		t, h, sess, "target_edit.html", map[string]any{
+			dataKeyWebhook: webhook,
+			"Target": map[string]any{
+				"ID":         "tg-1",
+				"Name":       "t",
+				"Type":       "slack",
+				"Active":     true,
+				"MaxRetries": 3,
+				"Config": map[string]any{
+					"URL": "https://hooks.slack.com/services/x",
+				},
+			},
+			dataKeyError: "",
+		},
+	)
+
+	assert.Contains(
+		t, editBody, maxRetriesHelp,
+		"the target edit form must explain max_retries as total attempts",
+	)
+}

templates/source_detail.html

@@ -120,9 +120,12 @@
                         <label class="text-sm text-gray-700">Timeout (seconds, blank = default):</label>
                         <input type="number" name="timeout" min="0" max="300" :disabled="targetType !== 'http'" class="input text-sm w-24">
                     </div>
-                    <div x-show="targetType === 'http'" class="flex gap-2 items-center">
+                    <div x-show="targetType === 'http'">
-                        <label class="text-sm text-gray-700">Max retries (0 = fire-and-forget):</label>
+                        <div class="flex gap-2 items-center">
-                        <input type="number" name="max_retries" value="0" min="0" max="20" class="input text-sm w-24">
+                            <label class="text-sm text-gray-700">Max retries:</label>
+                            <input type="number" name="max_retries" value="0" min="0" max="20" class="input text-sm w-24">
+                        </div>
+                        <p class="text-xs text-gray-500 mt-1">This is the total number of delivery attempts, not retries on top of the first: a value of 3 makes three attempts in all. 0 means a single attempt with no retries and no circuit breaker.</p>
                     </div>
                     <div x-show="targetType === 'slack'">
                         <input type="url" name="url" placeholder="https://hooks.slack.com/services/..." :disabled="targetType !== 'slack'" class="input text-sm">

templates/target_edit.html

@@ -69,7 +69,7 @@
             <div class="form-group">
                 <label for="max_retries" class="label">Max retries</label>
                 <input type="number" id="max_retries" name="max_retries" value="{{.Target.MaxRetries}}" min="0" max="20" class="input">
-                <p class="text-xs text-gray-500 mt-1">0 is fire-and-forget: one attempt, no circuit breaker.</p>
+                <p class="text-xs text-gray-500 mt-1">This is the total number of delivery attempts, not retries on top of the first: a value of 3 makes three attempts in all. 0 means a single attempt with no retries and no circuit breaker.</p>
             </div>
             {{end}}