New-webhook page: optional HTTP target URL and archive with pruning (closes #373)

The new-webhook page gains an optional HTTP target URL, which creates an http target named HTTP, and an archive checkbox whose pruning choice (never, 1h, 12h, 24h, 30d, 90d, 365d) creates a database target named Archive with that expiry. Both are validated by the add target form's own validation, and the webhook, its entrypoint and its targets are created in one transaction or not at all. A refused form comes back with the reason and every value entered, retention included. The targets can be renamed on the webhook page like any other.

Model: opus-5-5

README.md

@@ -1580,6 +1580,13 @@ more entrypoints (receiver URLs) and one or more targets (delivery
 destinations) into a logical unit. A user creates a webhook to set up
 event routing.
 
+The new webhook form can also give the webhook its first targets: an
+optional HTTP target URL creates an `http` target named `HTTP`, and the
+archive checkbox creates a `database` target named `Archive` whose
+`expiry` is the pruning chosen beside it (never, 1h, 12h, 24h, 30d, 90d
+or 365d). Both are validated as on the add target form, and the webhook
+and its targets are created together or not at all.
+
 | Field            | Type    | Description |
 | ---------------- | ------- | ----------- |
 | `id`             | UUID    | Primary key |
@@ -1913,6 +1920,12 @@ counted with the pane's query. It opens each webhook's event database once
 with the number of webhooks and, for each, with the deliveries that
 finished in the last 24 hours, never with the events stored.
 
+The target list on the webhook page shows, for each target, its
+`delivered` and `failed` totals, which retention does not reduce, and its
+deliveries that became `delivered` and `failed` in the last 24 hours,
+counted with the pane's query. Deliveries still `pending` or `retrying`
+count in neither.
+
 #### Event-tier indexes
 
 These indexes on the per-webhook event databases are declared in the model
@@ -1920,7 +1933,7 @@ tags, so `AutoMigrate` creates them on a fresh database:
 
 | Table              | Columns                     | Serves |
 | ------------------ | --------------------------- | ------ |
-| `deliveries`       | `status`, `deleted_at`, `finished_at`, `target_id` | Startup recovery, the retry and pending sweeps every 60 seconds and the queue-depth sampler every 30 seconds, which select deliveries by status, and the webhook page's statistics and the webhook list, which count each target's deliveries by status and when they finished |
+| `deliveries`       | `status`, `deleted_at`, `finished_at`, `target_id` | Startup recovery, the retry and pending sweeps every 60 seconds and the queue-depth sampler every 30 seconds, which select deliveries by status, and the webhook page's statistics and target list and the webhook list, which count each target's deliveries by status and when they finished |
 | `deliveries`       | `event_id`, `deleted_at`    | The event log, which loads each event's deliveries, and retention, which counts and deletes the deliveries of expired events |
 | `delivery_results` | `delivery_id`, `deleted_at` | The event log, which loads the attempts of a page's deliveries, and retention, which deletes the attempts of expired events |
 | `events`           | `deleted_at`, `created_at`  | The webhook page's statistics, which count recent events |

internal/delivery/target_config_view.go

@@ -97,7 +97,7 @@ func targetConfigFields(
 ) []ConfigField {
 	switch t.Type {
 	case database.TargetTypeSlack:
-		return slackConfigFields(t.Config)
+		return slackConfigFields(t)
 	case database.TargetTypeHTTP:
 		return httpConfigFields(t)
 	case database.TargetTypeDatabase:
@@ -119,10 +119,11 @@ func unavailableConfigFields() []ConfigField {
 	}}
 }
 
-// slackConfigFields describes a Slack target. Only the masked
-// webhook URL is shown; the full URL is the credential.
-func slackConfigFields(configJSON string) []ConfigField {
-	cfg, err := parseSlackConfig(configJSON)
+// slackConfigFields describes a Slack target: its masked
+// webhook URL and its retry count. Only the masked URL is
+// shown; the full URL is the credential.
+func slackConfigFields(t *database.Target) []ConfigField {
+	cfg, err := parseSlackConfig(t.Config)
 	if err != nil {
 		return unavailableConfigFields()
 	}
@@ -130,7 +131,7 @@ func slackConfigFields(configJSON string) []ConfigField {
 	return []ConfigField{{
 		Label: "Webhook URL",
 		Value: cfg.MaskedWebhookURL(),
-	}}
+	}, maxRetriesField(t)}
 }
 
 // httpConfigFields describes an HTTP target: its destination
@@ -170,21 +171,7 @@ func httpConfigFields(t *database.Target) []ConfigField {
 		})
 	}
 
-	return append(fields, retryFields(t)...)
-}
-
-// retryFields describes a target's retry settings, which live
-// on the target row rather than in its configuration blob.
-func retryFields(t *database.Target) []ConfigField {
-	retries := strconv.Itoa(t.MaxRetries)
-	if t.MaxRetries == 0 {
-		retries += " (fire-and-forget)"
-	}
-
-	fields := []ConfigField{{
-		Label: "Max Retries",
-		Value: retries,
-	}}
+	fields = append(fields, maxRetriesField(t))
 
 	if t.MaxQueueSize > 0 {
 		fields = append(fields, ConfigField{
@@ -196,6 +183,20 @@ func retryFields(t *database.Target) []ConfigField {
 	return fields
 }
 
+// maxRetriesField describes a target's retry count, which lives
+// on the target row rather than in its configuration blob.
+func maxRetriesField(t *database.Target) ConfigField {
+	retries := strconv.Itoa(t.MaxRetries)
+	if t.MaxRetries == 0 {
+		retries += " (fire-and-forget)"
+	}
+
+	return ConfigField{
+		Label: "Max Retries",
+		Value: retries,
+	}
+}
+
 // databaseConfigFields describes an archive target. Its
 // configuration is optional, and an absent or empty expiry
 // means the archive is kept forever. An expiry that is set

internal/delivery/target_config_view_test.go

@@ -32,6 +32,7 @@ const (
 	viewMaskedOrigin  = viewExampleOrigin + "/..."
 	viewUnavailable   = "(unavailable)"
 	viewExpiryNever   = "never"
+	viewMaxRetries    = "Max Retries"
 )
 
 func TestMaskedWebhookURL(t *testing.T) {
@@ -157,9 +158,7 @@ func TestNewTargetViews_DeletedTarget(t *testing.T) {
 		t, slackTargetName+" (deleted)", view.DisplayName(),
 	)
 	assert.Equal(
-		t,
-		map[string]string{"Webhook URL": slackMaskedURL},
-		fieldMap(view.Config),
+		t, viewFor(t, slackTarget()).Config, view.Config,
 	)
 }
 
@@ -189,7 +188,32 @@ func TestNewTargetViews_Slack(t *testing.T) {
 
 	assert.Equal(
 		t,
-		map[string]string{"Webhook URL": slackMaskedURL},
+		map[string]string{
+			"Webhook URL":  slackMaskedURL,
+			viewMaxRetries: "0 (fire-and-forget)",
+		},
+		fieldMap(view.Config),
+	)
+}
+
+// TestNewTargetViews_SlackRetries proves a Slack target shows
+// its retry count the same way an HTTP target does, and no
+// queue size even when one is stored: delivery never reads it.
+func TestNewTargetViews_SlackRetries(t *testing.T) {
+	t.Parallel()
+
+	target := slackTarget()
+	target.MaxRetries = 2
+	target.MaxQueueSize = 100
+
+	view := viewFor(t, target)
+
+	assert.Equal(
+		t,
+		map[string]string{
+			"Webhook URL":  slackMaskedURL,
+			viewMaxRetries: "2",
+		},
 		fieldMap(view.Config),
 	)
 }
@@ -214,7 +238,7 @@ func TestNewTargetViews_HTTP(t *testing.T) {
 			"Destination URL": viewMaskedOrigin,
 			"Timeout":         "30s",
 			"Headers":         "1 configured",
-			"Max Retries":     "5",
+			viewMaxRetries:    "5",
 			"Max Queue Size":  "100",
 		},
 		fields,
@@ -238,7 +262,7 @@ func TestNewTargetViews_HTTPFireAndForget(t *testing.T) {
 		t,
 		map[string]string{
 			"Destination URL": viewMaskedOrigin,
-			"Max Retries":     "0 (fire-and-forget)",
+			viewMaxRetries:    "0 (fire-and-forget)",
 		},
 		fieldMap(view.Config),
 	)

internal/handlers/source_create_targets_test.go

@@ -0,0 +1,207 @@
+package handlers_test
+
+import (
+	"errors"
+	"html"
+	"net/http"
+	"net/http/httptest"
+	"net/url"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"gorm.io/gorm"
+	"sneak.berlin/go/webhooker/internal/database"
+)
+
+// submitCreateForm posts the new webhook form and returns the
+// recorder.
+func submitCreateForm(
+	env *sourceTestEnv, form url.Values,
+) *httptest.ResponseRecorder {
+	req := formRequest("/hooks/new", env.cookies, form, nil)
+	w := httptest.NewRecorder()
+
+	env.handlers.HandleSourceCreateSubmit().ServeHTTP(w, req)
+
+	return w
+}
+
+// assertNothingCreated checks that the main database holds no webhook,
+// entrypoint or target.
+func assertNothingCreated(t *testing.T, db *database.Database) {
+	t.Helper()
+
+	for _, model := range []any{
+		&database.Webhook{}, &database.Entrypoint{}, &database.Target{},
+	} {
+		var count int64
+
+		require.NoError(t, db.DB().Model(model).Count(&count).Error)
+		assert.Zerof(t, count, "%T rows were created", model)
+	}
+}
+
+// TestHandleSourceCreateSubmit_CreatesRequestedTargets submits the new
+// webhook form with the HTTP target URL filled in or empty, and with
+// the archive checkbox off or on with each pruning choice. The webhook
+// gets an HTTP target only for a URL and a database target only for a
+// checked archive. The pruning choice is always submitted, as the
+// browser submits it while it is hidden, and is ignored when archive
+// is off.
+func TestHandleSourceCreateSubmit_CreatesRequestedTargets(t *testing.T) {
+	t.Parallel()
+
+	env := setupSourceTest(t)
+
+	// Each value the archive pruning choice submits, after an empty
+	// one that stands for the archive checkbox left off.
+	expiries := []string{
+		"", "never", "1h", "12h", "24h", "720h", "2160h", "8760h",
+	}
+
+	for _, httpURL := range []string{"", editOriginalURL} {
+		for _, expiry := range expiries {
+			name := "url=" + httpURL + " archive=" + expiry
+
+			t.Run(name, func(t *testing.T) {
+				t.Parallel()
+
+				form := url.Values{}
+				form.Set("name", name)
+				form.Set("http_url", httpURL)
+				form.Set("archive_expiry", "720h")
+
+				if expiry != "" {
+					form.Set("archive", "on")
+					form.Set("archive_expiry", expiry)
+				}
+
+				w := submitCreateForm(env, form)
+				require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
+
+				var webhook database.Webhook
+
+				require.NoError(t, env.db.DB().
+					Where("name = ?", name).First(&webhook).Error)
+
+				byType := map[database.TargetType]database.Target{}
+				for _, target := range targetsForWebhook(t, env.db, webhook.ID) {
+					byType[target.Type] = target
+				}
+
+				wantCount := 0
+
+				if httpURL != "" {
+					wantCount++
+
+					assert.Equal(t, "HTTP", byType[database.TargetTypeHTTP].Name)
+					assert.JSONEq(t, `{"url":"`+httpURL+`"}`,
+						byType[database.TargetTypeHTTP].Config)
+				}
+
+				if expiry != "" {
+					wantCount++
+
+					assert.Equal(t, "Archive",
+						byType[database.TargetTypeDatabase].Name)
+					assert.JSONEq(t, `{"expiry":"`+expiry+`"}`,
+						byType[database.TargetTypeDatabase].Config)
+				}
+
+				assert.Len(t, byType, wantCount)
+			})
+		}
+	}
+}
+
+// TestHandleSourceCreateSubmit_RefusedFormKeepsEveryValue refuses the
+// new webhook form for an invalid HTTP target URL and for an invalid
+// retention, each with archive on. Nothing is created, and the form
+// comes back with the reason and every value entered: name,
+// description, retention, URL, the checked archive box and the pruning
+// choice.
+func TestHandleSourceCreateSubmit_RefusedFormKeepsEveryValue(
+	t *testing.T,
+) {
+	t.Parallel()
+
+	const badURL = "Invalid target URL"
+
+	cases := []struct {
+		name      string
+		retention string
+		httpURL   string
+		reason    string
+	}{
+		{"blocked url", "7", editBlockedURL, badURL},
+		{"unsupported scheme", "7", "ftp://93.184.216.34/hook", badURL},
+		{"bad retention", "-5", editOriginalURL, "Retention must be"},
+	}
+
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+
+			env := setupSourceTest(t)
+
+			form := url.Values{}
+			form.Set("name", "kept name")
+			form.Set("description", "kept description")
+			form.Set("retention_days", tc.retention)
+			form.Set("http_url", tc.httpURL)
+			form.Set("archive", "on")
+			form.Set("archive_expiry", "2160h")
+
+			w := submitCreateForm(env, form)
+			require.Equal(t, http.StatusBadRequest, w.Code)
+
+			page := w.Body.String()
+			assert.Contains(t, page, tc.reason)
+			assert.Contains(t, page, `value="kept name"`)
+			assert.Contains(t, page, `>kept description</textarea>`)
+			assert.Contains(t, page, `value="`+tc.retention+`"`)
+			assert.Contains(t, page,
+				`value="`+html.EscapeString(tc.httpURL)+`"`)
+			assert.Contains(t, page, `name="archive" value="on" checked`)
+			assert.Contains(t, page, `x-data="collapsible" data-open`)
+			assert.Contains(t, page, `<option value="2160h" selected>`)
+
+			assertNothingCreated(t, env.db)
+		})
+	}
+}
+
+// errInjectedTargetCreate is the failure a test makes the insert of a
+// target report.
+var errInjectedTargetCreate = errors.New("injected target create failure")
+
+// TestHandleSourceCreateSubmit_FailedTargetInsertCreatesNothing makes
+// inserting a target fail after the webhook and its entrypoint were
+// inserted, and checks that neither is left behind.
+func TestHandleSourceCreateSubmit_FailedTargetInsertCreatesNothing(
+	t *testing.T,
+) {
+	t.Parallel()
+
+	env := setupSourceTest(t)
+
+	require.NoError(t, env.db.DB().Callback().Create().
+		Before("gorm:create").
+		Register("test:fail_target_create", func(tx *gorm.DB) {
+			if tx.Statement.Table == "targets" {
+				_ = tx.AddError(errInjectedTargetCreate)
+			}
+		}),
+	)
+
+	form := url.Values{}
+	form.Set("name", "rolled back")
+	form.Set("archive", "on")
+	form.Set("archive_expiry", "never")
+
+	w := submitCreateForm(env, form)
+	require.Equal(t, http.StatusInternalServerError, w.Code)
+
+	assertNothingCreated(t, env.db)
+}

internal/handlers/source_management.go

@@ -276,27 +276,41 @@ func (h *Handlers) HandleSourceCreate() http.HandlerFunc {
 	return func(w http.ResponseWriter, r *http.Request) {
 		h.renderTemplate(
 			w, r, "sources_new.html",
-			newSourceFormData("", "", ""),
+			newSourceFormData("", sourceFormInput{
+				RetentionDays: strconv.Itoa(
+					database.DefaultRetentionDays,
+				),
+			}),
 		)
 	}
 }
 
+// sourceFormInput carries the raw values of the new webhook form. A
+// refused submission is shown again from it, so every value entered
+// comes back, retention included.
+type sourceFormInput struct {
+	Name          string
+	Description   string
+	RetentionDays string
+	// HTTPURL, when not empty, asks for an HTTP target with this
+	// destination.
+	HTTPURL string
+	// Archive asks for a database (archive) target, whose rows expire
+	// after ArchiveExpiry.
+	Archive       bool
+	ArchiveExpiry string
+}
+
 // newSourceFormData builds the template data for the webhook creation
-// form.
-//
-// It carries the retention default so the pre-filled value comes from
-// database.DefaultRetentionDays rather than being a third hardcoded
-// copy of the same policy, and it carries the submitted name and
-// description so that re-rendering the form after a validation failure
-// gives the user their input back instead of a blank form. The edit
-// form already behaves that way; create now matches it.
+// form. It carries the retention default, which the form's help text
+// names, from database.DefaultRetentionDays rather than a hardcoded
+// copy of the same policy.
 func newSourceFormData(
-	errMsg, name, description string,
+	errMsg string, in sourceFormInput,
 ) map[string]any {
 	return map[string]any{
 		tmplKeyError:           errMsg,
-		"Name":                 name,
-		"Description":          description,
+		"Form":                 in,
 		"DefaultRetentionDays": database.DefaultRetentionDays,
 	}
 }
@@ -323,57 +337,112 @@ func (h *Handlers) HandleSourceCreateSubmit() http.HandlerFunc {
 			return
 		}
 
-		name := r.PostFormValue("name")
-		description := r.PostFormValue("description")
-		retentionStr := r.PostFormValue("retention_days")
+		in := sourceFormInput{
+			Name:          r.PostFormValue("name"),
+			Description:   r.PostFormValue("description"),
+			RetentionDays: r.PostFormValue("retention_days"),
+			HTTPURL:       r.PostFormValue("http_url"),
+			Archive:       r.PostFormValue("archive") != "",
+			ArchiveExpiry: r.PostFormValue("archive_expiry"),
+		}
 
-		if name == "" {
+		refuse := func(errMsg string) {
 			h.renderTemplateStatus(
 				w, r, "sources_new.html",
-				newSourceFormData(
-					"Name is required", name, description,
-				),
+				newSourceFormData(errMsg, in),
 				http.StatusBadRequest,
 			)
+		}
+
+		if in.Name == "" {
+			refuse("Name is required")
 
 			return
 		}
 
 		retentionDays, errMsg := parseRetentionDays(
-			retentionStr, database.DefaultRetentionDays,
+			in.RetentionDays, database.DefaultRetentionDays,
 		)
 		if errMsg != "" {
-			h.renderTemplateStatus(
-				w, r, "sources_new.html",
-				newSourceFormData(errMsg, name, description),
-				http.StatusBadRequest,
-			)
+			refuse(errMsg)
 
 			return
 		}
 
-		h.createWebhookWithEntrypoint(
-			w, r, userID, name, description, retentionDays,
-		)
+		targets, errMsg, err := h.newWebhookTargets(r.Context(), in)
+		if err != nil {
+			h.serverError(w, r, "failed to encode target config", err)
+
+			return
+		}
+
+		if errMsg != "" {
+			refuse(errMsg)
+
+			return
+		}
+
+		h.createWebhookWithEntrypoint(w, r, &database.Webhook{
+			UserID:        userID,
+			Name:          in.Name,
+			Description:   in.Description,
+			RetentionDays: retentionDays,
+		}, targets)
 	}
 }
 
-// createWebhookWithEntrypoint creates a webhook and its default
-// entrypoint in a transaction.
+// newWebhookTargets validates the targets the new webhook form asks
+// for and returns the rows to create with the webhook, or the message
+// the form shows for the first one it refuses. A filled-in HTTP URL
+// asks for an HTTP target named "HTTP", and the archive checkbox for a
+// database target named "Archive". Each goes through newTarget, as on
+// the webhook page's add target form. The rows have no WebhookID yet:
+// the webhook has no ID until it is created.
+func (h *Handlers) newWebhookTargets(
+	ctx context.Context,
+	in sourceFormInput,
+) ([]*database.Target, string, error) {
+	var requested []targetFormInput
+
+	if in.HTTPURL != "" {
+		requested = append(requested, targetFormInput{
+			Name: "HTTP",
+			Type: database.TargetTypeHTTP,
+			URL:  in.HTTPURL,
+		})
+	}
+
+	if in.Archive {
+		requested = append(requested, targetFormInput{
+			Name:   "Archive",
+			Type:   database.TargetTypeDatabase,
+			Expiry: in.ArchiveExpiry,
+		})
+	}
+
+	targets := make([]*database.Target, 0, len(requested))
+
+	for _, form := range requested {
+		target, errMsg, err := h.newTarget(ctx, "", form)
+		if err != nil || errMsg != "" {
+			return nil, errMsg, err
+		}
+
+		targets = append(targets, target)
+	}
+
+	return targets, "", nil
+}
+
+// createWebhookWithEntrypoint creates a webhook, its default
+// entrypoint and the given targets in a transaction.
 func (h *Handlers) createWebhookWithEntrypoint(
 	w http.ResponseWriter,
 	r *http.Request,
-	userID, name, description string,
-	retentionDays int,
+	webhook *database.Webhook,
+	targets []*database.Target,
 ) {
-	webhook := &database.Webhook{
-		UserID:        userID,
-		Name:          name,
-		Description:   description,
-		RetentionDays: retentionDays,
-	}
-
-	err := h.commitWebhook(webhook)
+	err := h.commitWebhook(webhook, targets)
 	if err != nil {
 		h.serverError(w, r, "failed to create webhook", err)
 
@@ -390,7 +459,7 @@ func (h *Handlers) createWebhookWithEntrypoint(
 
 	h.log.Info("webhook created",
 		"webhook_id", webhook.ID,
-		"name", name, "user_id", userID,
+		"name", webhook.Name, "user_id", webhook.UserID,
 	)
 
 	http.Redirect(
@@ -399,10 +468,12 @@ func (h *Handlers) createWebhookWithEntrypoint(
 	)
 }
 
-// commitWebhook creates a webhook and default entrypoint in
-// a transaction. Returns an error on failure (rolls back).
+// commitWebhook creates a webhook, its default entrypoint and the
+// given targets in a transaction. Returns an error on failure (rolls
+// back).
 func (h *Handlers) commitWebhook(
 	webhook *database.Webhook,
+	targets []*database.Target,
 ) error {
 	tx := h.db.DB().Begin()
 	if tx.Error != nil {
@@ -430,6 +501,17 @@ func (h *Handlers) commitWebhook(
 		return err
 	}
 
+	for _, target := range targets {
+		target.WebhookID = webhook.ID
+
+		err = tx.Create(target).Error
+		if err != nil {
+			tx.Rollback()
+
+			return err
+		}
+	}
+
 	return tx.Commit().Error
 }
 

internal/handlers/target_list.go

@@ -2,11 +2,13 @@ package handlers
 
 import (
 	"errors"
+	"fmt"
 	"io/fs"
 	"path/filepath"
 	"time"
 
 	"github.com/dustin/go-humanize"
+	"gorm.io/gorm"
 	"sneak.berlin/go/webhooker/internal/database"
 	"sneak.berlin/go/webhooker/internal/delivery"
 )
@@ -15,11 +17,27 @@ import (
 type TargetRowView struct {
 	delivery.TargetView
 
+	// Deliveries counts the target's delivered and failed deliveries,
+	// and is nil when the webhook's event database could not be read.
+	Deliveries *TargetDeliveries
+
 	// Archive is a database target's archive file, and nil for a target
 	// of any other type.
 	Archive *ArchiveFileView
 }
 
+// TargetDeliveries is how many of a target's deliveries became
+// delivered and how many failed: in total, which retention does not
+// reduce, and in the last 24 hours. Deliveries still pending or
+// retrying count in neither.
+type TargetDeliveries struct {
+	Delivered int64
+	Failed    int64
+
+	DeliveredLast24Hours int64
+	FailedLast24Hours    int64
+}
+
 // ArchiveFileView is what a database target's row shows about its
 // archive file.
 type ArchiveFileView struct {
@@ -45,10 +63,24 @@ func (h *Handlers) targetRows(
 	views := delivery.NewTargetViews(targets)
 	rows := make([]TargetRowView, len(views))
 
+	deliveries, err := h.loadTargetDeliveries(webhook.ID)
+	if err != nil {
+		h.log.Error(
+			"failed to read target delivery counts",
+			"webhook_id", webhook.ID,
+			"error", err,
+		)
+	}
+
 	// NewTargetViews returns one view per target, in order.
 	for i := range views {
 		rows[i].TargetView = views[i]
 
+		if err == nil {
+			counts := deliveries[targets[i].ID]
+			rows[i].Deliveries = &counts
+		}
+
 		if targets[i].Type == database.TargetTypeDatabase {
 			rows[i].Archive = h.archiveFileView(webhook, &targets[i])
 		}
@@ -57,6 +89,64 @@ func (h *Handlers) targetRows(
 	return rows
 }
 
+// loadTargetDeliveries reads the delivery counts of a webhook's targets
+// from its event database, keyed by target. A target with no deliveries
+// is left out, and so is every target when the event database does not
+// exist yet, since opening it would create it.
+func (h *Handlers) loadTargetDeliveries(
+	webhookID string,
+) (map[string]TargetDeliveries, error) {
+	if !h.dbMgr.DBExists(webhookID) {
+		return map[string]TargetDeliveries{}, nil
+	}
+
+	webhookDB, err := h.dbMgr.GetDB(webhookID)
+	if err != nil {
+		return nil, err
+	}
+
+	return readTargetDeliveries(webhookDB, time.Now())
+}
+
+// readTargetDeliveries counts each target's deliveries that became
+// delivered and those that failed: in total from the targets' running
+// totals, and in the 24 hours before now from the deliveries' status
+// index. Each is one query for all the targets, and neither reads every
+// stored delivery.
+func readTargetDeliveries(
+	db *gorm.DB, now time.Time,
+) (map[string]TargetDeliveries, error) {
+	var totals []database.TargetTotals
+
+	err := db.Find(&totals).Error
+	if err != nil {
+		return nil, fmt.Errorf("reading target totals: %w", err)
+	}
+
+	lastDay, err := finishedByTarget(db, now.Add(-longWindow))
+	if err != nil {
+		return nil, err
+	}
+
+	byTarget := make(map[string]TargetDeliveries, len(totals))
+
+	for _, total := range totals {
+		byTarget[total.TargetID] = TargetDeliveries{
+			Delivered: total.Delivered,
+			Failed:    total.Failed,
+		}
+	}
+
+	for _, finished := range lastDay {
+		counts := byTarget[finished.TargetID]
+		counts.DeliveredLast24Hours = finished.Delivered
+		counts.FailedLast24Hours = finished.Failed
+		byTarget[finished.TargetID] = counts
+	}
+
+	return byTarget, nil
+}
+
 // archiveFileView describes a database target's archive file from the
 // file's metadata alone; the archive is never opened. The file is found
 // by the name the archive writer uses, so it follows a rename of the

internal/handlers/target_list_test.go

@@ -3,6 +3,7 @@ package handlers_test
 import (
 	"os"
 	"path/filepath"
+	"regexp"
 	"strings"
 	"testing"
 	"time"
@@ -12,6 +13,7 @@ import (
 	"sneak.berlin/go/webhooker/internal/database"
 	"sneak.berlin/go/webhooker/internal/delivery"
 	"sneak.berlin/go/webhooker/internal/handlers"
+	"sneak.berlin/go/webhooker/internal/logger"
 	"sneak.berlin/go/webhooker/internal/session"
 )
 
@@ -69,3 +71,104 @@ func TestHandleSourceDetail_ShowsArchiveFile(t *testing.T) {
 	assert.Contains(t, body, "not created yet")
 	assert.NotContains(t, body, "Archive Size:")
 }
+
+// targetList returns the text of the targets section in a rendered
+// webhook page, from its heading to the next heading, with the markup
+// taken out and each run of space made one space. Each target's row
+// then reads as its name, type, state and buttons, followed by the
+// lines below them.
+func targetList(t *testing.T, page string) string {
+	t.Helper()
+
+	_, list, found := strings.Cut(page, ">Targets</h2>")
+	require.True(t, found, "the page has no targets section")
+
+	list, _, _ = strings.Cut(list, "<h2")
+	list = regexp.MustCompile(`<[^>]*>`).ReplaceAllString(list, " ")
+
+	return strings.Join(strings.Fields(list), " ")
+}
+
+// TestHandleSourceDetail_ShowsTargetDeliveries checks each target row's
+// delivered and failed deliveries, in total and in the last 24 hours,
+// for the history seedStatsHistory builds, before and after the real
+// retention reaper removes the oldest event. The http target has one
+// delivered, one of them in the last 24 hours, and three failed, one of
+// them in the last 24 hours and one of them the oldest event's, which
+// retention removes without changing the total. The active log target
+// has two failed, both in the last 24 hours, and its pending and
+// retrying deliveries count in neither. The four inactive log targets
+// have none.
+func TestHandleSourceDetail_ShowsTargetDeliveries(t *testing.T) {
+	t.Parallel()
+
+	var (
+		h     *handlers.Handlers
+		sess  *session.Session
+		db    *database.Database
+		dbMgr *database.WebhookDBManager
+		log   *logger.Logger
+	)
+
+	app := newTestApp(t, &h, &sess, &db, &dbMgr, &log)
+	app.RequireStart()
+
+	t.Cleanup(app.RequireStop)
+
+	hist := seedStatsHistory(t, h, sess, db, dbMgr)
+
+	const (
+		httpRow = "Delivered: 1 in total, 1 in the last 24 hours " +
+			"Failed: 3 in total, 1 in the last 24 hours"
+		activeLogRow = "t-log log Active Edit Deactivate Delete " +
+			"Delivered: 0 in total, 0 in the last 24 hours " +
+			"Failed: 2 in total, 2 in the last 24 hours"
+		inactiveLogRow = "t-log log Inactive Edit Activate Delete " +
+			"Delivered: 0 in total, 0 in the last 24 hours " +
+			"Failed: 0 in total, 0 in the last 24 hours"
+	)
+
+	list := targetList(t, renderSourceDetailPage(t, h, sess, hist.webhook.ID))
+	assert.Equal(t, 1, strings.Count(list, httpRow))
+	assert.Equal(t, 1, strings.Count(list, activeLogRow))
+	assert.Equal(t, 4, strings.Count(list, inactiveLogRow))
+
+	statsPrune(t, db, dbMgr, log, hist.webhookDB)
+
+	list = targetList(t, renderSourceDetailPage(t, h, sess, hist.webhook.ID))
+	assert.Equal(t, 1, strings.Count(list, httpRow))
+	assert.Equal(t, 1, strings.Count(list, activeLogRow))
+	assert.Equal(t, 4, strings.Count(list, inactiveLogRow))
+}
+
+// TestHandleSourceDetail_TargetDeliveriesUnreadable checks that when the
+// webhook's event database cannot be read, each target's row says so
+// instead of showing zeros.
+func TestHandleSourceDetail_TargetDeliveriesUnreadable(t *testing.T) {
+	t.Parallel()
+
+	var (
+		h     *handlers.Handlers
+		sess  *session.Session
+		db    *database.Database
+		dbMgr *database.WebhookDBManager
+	)
+
+	app := newTestApp(t, &h, &sess, &db, &dbMgr)
+	app.RequireStart()
+
+	t.Cleanup(app.RequireStop)
+
+	wh := seedWebhook(t, db)
+	seedTarget(t, db, wh.ID, database.TargetTypeLog)
+
+	webhookDB, err := dbMgr.GetDB(wh.ID)
+	require.NoError(t, err)
+	require.NoError(t,
+		webhookDB.Migrator().DropTable(&database.TargetTotals{}))
+
+	list := targetList(t, renderSourceDetailPage(t, h, sess, wh.ID))
+	assert.Contains(t, list, "t-log log Active Edit Deactivate Delete "+
+		"The delivery counts could not be read.")
+	assert.NotContains(t, list, "Delivered:")
+}

internal/handlers/ui_copy_test.go

@@ -196,8 +196,6 @@ func TestCreateFormRetentionCopyMatchesBehaviour(t *testing.T) {
 	t.Cleanup(app.RequireStop)
 
 	body := renderPage(t, h, sess, "sources_new.html", map[string]any{
-		"Name":                 "",
-		"Description":          "",
 		"DefaultRetentionDays": database.DefaultRetentionDays,
 		dataKeyError:           "",
 	})

internal/server/alpine_browser_test.go

@@ -47,7 +47,8 @@ const (
 // TestAlpineRunsUnderTheSecurityPolicy loads the webhook page and the
 // event log in a headless browser, served by the real router and so
 // under the real Content-Security-Policy, and checks that the pages'
-// Alpine.js directives and the copy control work.
+// Alpine.js directives and the copy control work, and that a target's
+// row shows its delivery counts.
 func TestAlpineRunsUnderTheSecurityPolicy(t *testing.T) {
 	t.Parallel()
 
@@ -118,10 +119,16 @@ func TestAlpineRunsUnderTheSecurityPolicy(t *testing.T) {
 	}
 
 	checkRefusedTarget(ctx, t, page)
+	checkTargetDeliveries(ctx, t, page, target.Name,
+		"0 in total, 0 in the last 24 hours",
+		"1 in total, 1 in the last 24 hours")
 	checkCopy(ctx, t, page)
 	checkEntrypointEdit(ctx, t, page, page+"/events")
 	checkRecentEvents(ctx, t, page)
 	checkEventLog(ctx, t, page+"/events", event.ID, target.Name)
+	checkArchiveChoice(ctx, t, srv.URL+"/hooks/new", page)
+	checkNewWebhookTargets(ctx, t, env, srv.URL+"/hooks/new")
+	checkRefusedNewWebhook(ctx, t, srv.URL+"/hooks/new")
 	checkMobileMenu(ctx, t, page)
 
 	assert.Empty(t, problems(), "the browser reported problems")
@@ -452,6 +459,28 @@ func checkRefusedTarget(ctx context.Context, t *testing.T, url string) {
 	assert.Empty(t, typed, "after Cancel, the next Add keeps the url entered")
 }
 
+// checkTargetDeliveries loads a webhook page and checks that the row of
+// the target named name shows delivered and failed beside its
+// "Delivered:" and "Failed:" labels.
+func checkTargetDeliveries(
+	ctx context.Context, t *testing.T, url, name, delivered, failed string,
+) {
+	t.Helper()
+
+	row := `//span[text()="` + name + `"]/ancestor::div[@class="p-4"][1]`
+	figure := func(label, value string) string {
+		return row + `//span[text()="` + label +
+			`"]/following-sibling::span[text()="` + value + `"]`
+	}
+
+	require.NoError(t, chromedp.Run(ctx, loadPage(url)))
+
+	assert.Truef(t, shown(ctx, figure("Delivered:", delivered)),
+		"the row of %s does not show %q delivered", name, delivered)
+	assert.Truef(t, shown(ctx, figure("Failed:", failed)),
+		"the row of %s does not show %q failed", name, failed)
+}
+
 // checkCopy loads a webhook page and checks that the Copy control beside
 // its entrypoint's URL is a button, and that clicking it copies the URL
 // and says so: the button reads "Copied" only once the copy succeeded.
@@ -642,6 +671,199 @@ func checkEventLog(
 		"clicking the event again does not collapse it")
 }
 
+// The parts of the new webhook page the checks below find and click.
+const (
+	archiveBox    = `//input[@name="archive"]`
+	archiveIsOn   = `document.querySelector('input[name="archive"]').checked`
+	pruningChoice = `//select[@name="archive_expiry"]`
+	createButton  = `//button[text()="Create Webhook"]`
+)
+
+// checkArchiveChoice loads the new webhook page and checks that the
+// archive pruning choice stays hidden until the archive box is checked
+// and hides again when it is unchecked; and that after checking it,
+// opening the page at elsewhere and going back, the page again shows
+// the box unchecked and the choice hidden.
+func checkArchiveChoice(
+	ctx context.Context, t *testing.T, url, elsewhere string,
+) {
+	t.Helper()
+
+	require.NoError(t, chromedp.Run(ctx, loadPage(url)))
+
+	assert.True(t, hidden(ctx, pruningChoice),
+		"the pruning choice shows before archive is checked")
+
+	click(ctx, t, archiveBox)
+	assert.True(t, shown(ctx, pruningChoice),
+		"checking archive does not show the pruning choice")
+
+	click(ctx, t, archiveBox)
+	assert.True(t, hidden(ctx, pruningChoice),
+		"unchecking archive does not hide the pruning choice")
+
+	var (
+		loaded  string
+		checked bool
+	)
+
+	click(ctx, t, archiveBox)
+	require.NoError(t, chromedp.Run(
+		ctx,
+		loadPage(elsewhere),
+		chromedp.NavigateBack(),
+		chromedp.WaitNotPresent("[x-cloak]", chromedp.ByQuery),
+		chromedp.Evaluate(
+			`performance.getEntriesByType("navigation")[0].type`, &loaded,
+		),
+		chromedp.Evaluate(archiveIsOn, &checked),
+	))
+	require.Equal(
+		t, "back_forward", loaded,
+		"going back, the browser did not load the page again",
+	)
+
+	assert.False(t, checked, "going back leaves archive checked")
+	assert.True(t, hidden(ctx, pruningChoice),
+		"going back shows the pruning choice")
+}
+
+// checkNewWebhookTargets submits the new webhook page with the HTTP
+// target URL filled in or empty, and with archive left off or checked
+// with each pruning choice, and checks that each webhook is created
+// with exactly the targets asked for.
+func checkNewWebhookTargets(
+	ctx context.Context, t *testing.T, env *testEnv, url string,
+) {
+	t.Helper()
+
+	// Each value the pruning choice submits, after an empty one that
+	// stands for archive left off.
+	expiries := []string{
+		"", "never", "1h", "12h", "24h", "720h", "2160h", "8760h",
+	}
+
+	for _, httpURL := range []string{"", publicTargetURL} {
+		for _, expiry := range expiries {
+			name := "url=" + httpURL + " archive=" + expiry
+			want := map[database.TargetType]string{}
+
+			require.NoError(t, chromedp.Run(
+				ctx,
+				loadPage(url),
+				chromedp.SetValue("#name", name, chromedp.ByQuery),
+			))
+
+			if httpURL != "" {
+				require.NoError(t, chromedp.Run(ctx, chromedp.SetValue(
+					"#http_url", httpURL, chromedp.ByQuery,
+				)))
+
+				want[database.TargetTypeHTTP] = `{"url":"` + httpURL + `"}`
+			}
+
+			if expiry != "" {
+				// The choice showing moves Create down, so it is
+				// waited for before Create is clicked.
+				click(ctx, t, archiveBox)
+				require.Truef(t, shown(ctx, pruningChoice),
+					"%s: checking archive does not show the pruning choice",
+					name)
+				require.NoError(t, chromedp.Run(ctx, chromedp.SetValue(
+					pruningChoice, expiry, chromedp.BySearch,
+				)))
+
+				want[database.TargetTypeDatabase] = `{"expiry":"` + expiry + `"}`
+			}
+
+			click(ctx, t, createButton)
+			require.Truef(t, shown(ctx, `//h1[text()="`+name+`"]`),
+				"%s: the new webhook's page does not open", name)
+
+			assert.Equalf(t, want, targetConfigs(t, env, name),
+				"%s: the webhook does not have the targets asked for", name)
+		}
+	}
+}
+
+// targetConfigs reads the targets of the webhook named name, and
+// returns each one's stored configuration by its type.
+func targetConfigs(
+	t *testing.T, env *testEnv, name string,
+) map[database.TargetType]string {
+	t.Helper()
+
+	var (
+		webhook database.Webhook
+		targets []database.Target
+	)
+
+	require.NoError(t, env.db.DB().
+		Where("name = ?", name).First(&webhook).Error)
+	require.NoError(t, env.db.DB().
+		Where("webhook_id = ?", webhook.ID).Find(&targets).Error)
+
+	configs := map[database.TargetType]string{}
+	for _, target := range targets {
+		configs[target.Type] = target.Config
+	}
+
+	return configs
+}
+
+// checkRefusedNewWebhook submits the new webhook page with archive
+// checked and an HTTP target URL the server refuses, a loopback
+// destination, and checks that the page comes back with the reason and
+// every value entered, archive still checked and its pruning choice
+// showing.
+func checkRefusedNewWebhook(ctx context.Context, t *testing.T, url string) {
+	t.Helper()
+
+	const refusedURL = "http://127.0.0.1/hook"
+
+	require.NoError(t, chromedp.Run(
+		ctx,
+		loadPage(url),
+		chromedp.SetValue("#name", "refused", chromedp.ByQuery),
+		chromedp.SetValue("#description", "kept", chromedp.ByQuery),
+		chromedp.SetValue("#retention_days", "7", chromedp.ByQuery),
+		chromedp.SetValue("#http_url", refusedURL, chromedp.ByQuery),
+	))
+	click(ctx, t, archiveBox)
+	require.True(t, shown(ctx, pruningChoice),
+		"checking archive does not show the pruning choice")
+	require.NoError(t, chromedp.Run(ctx, chromedp.SetValue(
+		pruningChoice, "2160h", chromedp.BySearch,
+	)))
+	click(ctx, t, createButton)
+
+	assert.True(t, shown(ctx, `//div[@class="alert-error"]`),
+		"a refused webhook does not show the reason")
+
+	var (
+		name, description, retention, typed, expiry string
+		checked                                     bool
+	)
+
+	require.NoError(t, chromedp.Run(
+		ctx,
+		chromedp.Value("#name", &name, chromedp.ByQuery),
+		chromedp.Value("#description", &description, chromedp.ByQuery),
+		chromedp.Value("#retention_days", &retention, chromedp.ByQuery),
+		chromedp.Value("#http_url", &typed, chromedp.ByQuery),
+		chromedp.Value("#archive_expiry", &expiry, chromedp.ByQuery),
+		chromedp.Evaluate(archiveIsOn, &checked),
+	))
+
+	assert.Equal(t, "refused", name, "the name entered is lost")
+	assert.Equal(t, "kept", description, "the description entered is lost")
+	assert.Equal(t, "7", retention, "the retention entered is lost")
+	assert.Equal(t, refusedURL, typed, "the url entered is lost")
+	assert.True(t, checked, "archive is no longer checked")
+	assert.True(t, shown(ctx, pruningChoice), "the pruning choice is hidden")
+	assert.Equal(t, "2160h", expiry, "the pruning chosen is lost")
+}
+
 // checkMobileMenu loads a page in a phone-sized window and checks that
 // the menu button opens and closes the mobile menu.
 func checkMobileMenu(ctx context.Context, t *testing.T, url string) {

internal/server/routes_test.go

@@ -11,6 +11,7 @@ import (
 	"strconv"
 	"strings"
 	"testing"
+	"time"
 
 	"github.com/google/uuid"
 	"github.com/stretchr/testify/assert"
@@ -439,7 +440,8 @@ func (e *testEnv) storedEntrypoint(
 }
 
 // seedFailedDelivery records a terminally failed delivery of an event
-// to a target in the webhook's own database.
+// to a target in the webhook's own database, as the delivery engine
+// leaves one: finished now, and counted in its target's totals.
 func (e *testEnv) seedFailedDelivery(
 	t *testing.T,
 	webhookID, eventID, targetID string,
@@ -449,16 +451,21 @@ func (e *testEnv) seedFailedDelivery(
 	webhookDB, err := e.dbMgr.GetDB(webhookID)
 	require.NoError(t, err)
 
+	finishedAt := time.Now()
 	dlv := &database.Delivery{
 		EventID:    eventID,
 		TargetID:   targetID,
 		Status:     database.DeliveryStatusFailed,
+		FinishedAt: &finishedAt,
 	}
 
 	require.NoError(
 		t,
 		webhookDB.Omit(clause.Associations).Create(dlv).Error,
 	)
+	require.NoError(t, database.AddTargetTotals(webhookDB,
+		database.TargetTotals{TargetID: targetID, Deliveries: 1, Failed: 1},
+	))
 
 	return dlv
 }

static/js/app.js

@@ -71,8 +71,9 @@ document.addEventListener("alpine:init", function () {
 
     // Something a click shows and hides: the mobile menu, an add form,
     // an entrypoint's edit form, an event in the event log or in the
-    // recent events, a delivery's attempts. It starts hidden, or shown
-    // when its element has the data-open attribute.
+    // recent events, a delivery's attempts, the new webhook page's
+    // archive pruning choice. It starts hidden, or shown when its
+    // element has the data-open attribute.
     window.Alpine.data("collapsible", function () {
         return {
             open: false,

templates/source_detail.html

@@ -271,6 +271,18 @@
                     </div>
                     {{end}}
                     {{end}}
+                    {{with .Deliveries}}
+                    <div class="text-xs text-gray-500 mt-1">
+                        <span class="font-medium text-gray-700">Delivered:</span>
+                        <span>{{.Delivered}} in total, {{.DeliveredLast24Hours}} in the last 24 hours</span>
+                    </div>
+                    <div class="text-xs text-gray-500 mt-1">
+                        <span class="font-medium text-gray-700">Failed:</span>
+                        <span>{{.Failed}} in total, {{.FailedLast24Hours}} in the last 24 hours</span>
+                    </div>
+                    {{else}}
+                    <div class="text-xs text-gray-500 mt-1">The delivery counts could not be read.</div>
+                    {{end}}
                 </div>
                 {{else}}
                 <div class="p-4 text-sm text-gray-500">No targets configured.</div>

templates/sources_new.html

@@ -18,20 +18,50 @@
             <input type="hidden" name="csrf_token" value="{{.CSRFToken}}">
             <div class="form-group">
                 <label for="name" class="label">Name</label>
-                <input type="text" id="name" name="name" value="{{.Name}}" required autofocus placeholder="My Webhook" class="input">
+                <input type="text" id="name" name="name" value="{{.Form.Name}}" required autofocus placeholder="My Webhook" class="input">
             </div>
 
             <div class="form-group">
                 <label for="description" class="label">Description</label>
-                <textarea id="description" name="description" rows="3" placeholder="Optional description" class="input">{{.Description}}</textarea>
+                <textarea id="description" name="description" rows="3" placeholder="Optional description" class="input">{{.Form.Description}}</textarea>
             </div>
 
             <div class="form-group">
                 <label for="retention_days" class="label">Retention (days)</label>
-                <input type="number" id="retention_days" name="retention_days" value="{{.DefaultRetentionDays}}" min="0" class="input">
+                <input type="number" id="retention_days" name="retention_days" value="{{.Form.RetentionDays}}" min="0" class="input">
                 <p class="text-xs text-gray-500 mt-1">A periodic cleanup permanently deletes events older than this, along with their delivery records. Enter 0 to retain events forever; leave blank to use the default of {{.DefaultRetentionDays}} days.</p>
             </div>
 
+            <div class="form-group">
+                <label for="http_url" class="label">HTTP target URL</label>
+                <input type="url" id="http_url" name="http_url" value="{{.Form.HTTPURL}}" placeholder="https://example.com/webhook" class="input">
+                <p class="text-xs text-gray-500 mt-1">Optional. When filled in, the webhook is created with an HTTP target that delivers each event to this URL.</p>
+            </div>
+
+            <!-- The checkbox shows the pruning choice while checked. With
+                 autocomplete="off", going back to the page does not
+                 check the box again with the choice hidden. -->
+            <div class="form-group" x-data="collapsible"{{if .Form.Archive}} data-open{{end}}>
+                <label class="flex items-center gap-2 text-sm font-medium text-gray-700">
+                    <input type="checkbox" name="archive" value="on"{{if .Form.Archive}} checked{{end}} autocomplete="off" @change="toggle" class="h-4 w-4">
+                    Archive
+                </label>
+                <p class="text-xs text-gray-500 mt-1">When checked, the webhook is created with a database target that keeps a copy of every event.</p>
+                <div x-show="open" x-cloak class="mt-3">
+                    <label for="archive_expiry" class="label">Archive pruning</label>
+                    <select id="archive_expiry" name="archive_expiry" class="input">
+                        <option value="never"{{if eq .Form.ArchiveExpiry "never"}} selected{{end}}>never</option>
+                        <option value="1h"{{if eq .Form.ArchiveExpiry "1h"}} selected{{end}}>1h</option>
+                        <option value="12h"{{if eq .Form.ArchiveExpiry "12h"}} selected{{end}}>12h</option>
+                        <option value="24h"{{if eq .Form.ArchiveExpiry "24h"}} selected{{end}}>24h</option>
+                        <option value="720h"{{if eq .Form.ArchiveExpiry "720h"}} selected{{end}}>30d</option>
+                        <option value="2160h"{{if eq .Form.ArchiveExpiry "2160h"}} selected{{end}}>90d</option>
+                        <option value="8760h"{{if eq .Form.ArchiveExpiry "8760h"}} selected{{end}}>365d</option>
+                    </select>
+                    <p class="text-xs text-gray-500 mt-1">Archived events older than this are deleted from the archive; never keeps them all.</p>
+                </div>
+            </div>
+
             <div class="flex gap-3">
                 <button type="submit" class="btn-primary">Create Webhook</button>
                 <a href="/hooks" class="btn-secondary">Cancel</a>