Author SHA1 Message Date
clawbot 820d9391ff Index the event log's resubmit count with deleted_at (closes #325)
check / check (push) Waiting to run
The event log's resubmit count, run on every page load over up to 25 event ids, read every live event in the webhook's database: GORM adds deleted_at IS NULL, and SQLite, keeping no statistics there, chose the deleted_at index over the resubmitted_from_id one. deleted_at is now the second column of idx_events_resubmitted_from_id, so the count is answered from that index for a whole page of events. A test checks SQLite's plan for the statement as GORM builds it, with a full page of ids. The README's event-tier indexes table lists the index. Pre-1.0: the index changes in the schema in place, with nothing for older databases.

Model: opus-5-5
2026-10-02 21:23:33 +02:00
clawbot 2967c475a1 Show a database target's archive file, size and last write in the target list (closes #397)
check / check (push) Waiting to run
For a database target, the target list showed only its expiry, so the archive file the README's backup and move-away advice depend on could only be found from a shell on the host. Each database target now shows its archive file's name, its size on disk (the file and its -wal together) and when it was last written, relative with the full UTC time on hover, all from the files' metadata without opening the archive. Before the first write, and after the file has been moved away, it shows the name and "not created yet". Tests cover all three states.

Model: opus-5-5
2026-10-02 21:22:33 +02:00
clawbot 35d2f28c67 Name each embedded static file so a missing Alpine.js fails the build (closes #166)
check / check (push) Waiting to run
static/static.go embedded the css and js directories, so a build without the extracted Alpine.js file compiled and produced a binary whose admin pages silently had no Alpine. It now names the four files the pages load, so such a build fails naming js/alpine.min.js; make build extracts the file first, so the failure shows when that step is skipped. Both lint stages extract Alpine.js before linting, since the static package no longer compiles without it, and the README's lint stage says so. static/css/input.css, the Tailwind source no page loads, is no longer embedded or served.

Model: opus-5-5
2026-10-02 20:56:35 +02:00
20 changed files with 720 additions and 714 deletions
+1
View File
@@ -25,6 +25,7 @@ COPY . .
# would need a docker daemon inside the build. Keep these steps in step with
# Dockerfile.lint, including --network=none (see its header for why).
RUN make fmt-check
RUN script/assets
RUN --network=none golangci-lint config verify --config .golangci.yml
RUN --network=none golangci-lint run --config .golangci.yml --build-tags browser ./...
+4
View File
@@ -31,6 +31,10 @@ FROM deps AS lint
COPY . .
# static/static.go embeds the Alpine.js file this extracts from 3p/; without
# it the static package does not compile and cannot be linted.
RUN script/assets
# `run` silently ignores config keys it does not recognize, so a typo would
# disable a setting without a word. `config verify` is what catches that.
RUN --network=none golangci-lint config verify --config .golangci.yml
+34 -22
View File
@@ -1050,7 +1050,8 @@ Archive databases are the one exception the service is built for: the
archive writer closes and reopens its handle around writes (debounced
to at most one reopen per second), so an operator can move an
`archive-….db` away for offline retention while the service runs,
and it is recreated on the next write. See
and it is recreated on the next write. The webhook page names each
`database` target's archive file. See
[Database Architecture](#database-architecture). That is a
move-the-file-away workflow, not a substitute for the backup procedures
above.
@@ -1323,22 +1324,19 @@ markup. The CSP build runs no expressions, so every Alpine directive in
`x-data="{ open: false }"` or `@click="open = !open"`.
A browser test in `internal/server` loads the webhook page and the event log
under the real policy and checks that: the add entrypoint form stays hidden
until Add is clicked; for every target type, the targets section's Add shows
only a choice of type with Next and Cancel, Next shows only that type's fields
(no url field for `database` or `log`), Cancel at either step closes the form,
and saving adds the target; a refused target comes back with its form open, the
values entered and the reason, and after Cancel the next Add starts with an
empty form and no reason; the Copy button beside an entrypoint URL reads
under the real policy and checks that: both add forms stay hidden until Add is
clicked; choosing Slack in the add target form leaves the HTTP fields out of
what it submits, also after leaving the page and going back to it, when the
browser restores the choice; the Copy button beside an entrypoint URL reads
"Copied" once clicked; an event expands and collapses, and so do a delivery's
attempts inside it; and at phone width the menu button opens and closes the
mobile menu. It also fails if the browser reports a console warning or error, an
uncaught exception, or anything the policy refused. `make check` and the image
build lint it but do not run it, and `make test` leaves it out (its file is
built only with the `browser` build tag). Run it with `make test-browser` after
changing `templates/` or `static/js/`: that builds `Dockerfile.browser`, which
runs the test in a digest-pinned headless browser image, so the host needs no
browser.
mobile menu. It also fails if the browser reports a console warning or error,
an uncaught exception, or anything the policy refused. `make check` and the
image build lint it but do not run it, and `make test` leaves it out (its file
is built only with the `browser` build tag). Run it with `make test-browser`
after changing `templates/` or `static/js/`: that builds `Dockerfile.browser`,
which runs the test in a digest-pinned headless browser image, so the host
needs no browser.
The package's tarball is committed as `3p/alpinejs-csp-3.14.9.tgz`, byte for
byte as the npm registry publishes it. It is a dependency, not this repo's build
@@ -1351,7 +1349,9 @@ apply. The directory is `3p/` rather than `vendor/` because Go treats a root
where `go:embed` picks it up. `script/test`, `make build` and `make dev` run
it first, and the Dockerfile builds through `make test` and `make build`, so
nothing downloads Alpine.js. The extracted file is not committed, and
`.dockerignore` keeps any host copy out of the build context.
`.dockerignore` keeps any host copy out of the build context. `static/static.go`
names every file it embeds, so a build that skips the extraction, such as a
bare `go build`, fails with an error naming `js/alpine.min.js`.
To move to a new version: download
`https://registry.npmjs.org/@alpinejs/csp/-/csp-<version>.tgz`, check it against
@@ -1879,16 +1879,19 @@ tags, so `AutoMigrate` creates them on a fresh database:
| `deliveries` | `event_id`, `deleted_at` | The event log, which loads each event's deliveries, and retention, which counts and deletes the deliveries of expired events |
| `delivery_results` | `delivery_id`, `deleted_at` | The event log, which loads the attempts of a page's deliveries, and retention, which deletes the attempts of expired events |
| `events` | `deleted_at`, `created_at` | The webhook page's statistics, which count recent events |
| `events` | `resubmitted_from_id`, `deleted_at` | The event log, which counts the events resubmitted from each event on a page |
| `events` | `created_at` | Retention, which selects expired events by age |
GORM's soft delete adds `deleted_at IS NULL` to these queries; retention
leaves it out. SQLite keeps no statistics on these tables, and without them it
rates the `deleted_at` index, which every live row matches, above an index on
a column matched against several values or compared with a range. So every
index but the last also covers `deleted_at`. It comes second, so that
retention can use the index without it, except in `events`, where the
statistics compare `created_at` with a range (`>=`) and SQLite narrows by a
range only on the last column it uses.
index but the last also covers `deleted_at`. It comes second in the `event_id`
and `delivery_id` indexes, so that retention can use them without it. The event
log's count, the one query on the `resubmitted_from_id` index, always carries
`deleted_at IS NULL` and uses both columns. In the statistics' `events` index
`deleted_at` comes first, because they compare `created_at` with a range (`>=`)
and SQLite narrows by a range only on the last column it uses.
#### Common Fields
@@ -2034,6 +2037,14 @@ Because each `database` target has its own archive file, a target's
webhook with different expiries keep two archives, each pruned on its
own schedule.
The webhook page shows, for each `database` target, its archive file's
name, its size on disk and when it was last written. The size counts
the `.db` and its `-wal` together, and the last write is the later of
their two modification times, since a write lands in the `-wal` first.
Both are read from the files' metadata; the archive is never opened.
Before the first write, and after the file has been moved away, the page
shows `not created yet` beside the name.
Each `database` target on the webhook page has a **Download** button,
which returns its archive as one gzipped JSON file,
`archive-{webhook_name}-{target_name}-{YYYYMMDDTHHMMSSZ}.json.gz`, the
@@ -3382,8 +3393,9 @@ version is fixed independently of the compiler's:
1. **Lint stage** (`golangci/golangci-lint:v2.12.2`, Debian-based) —
installs `make`, downloads dependencies, copies the source, and runs
`make fmt-check`, then `golangci-lint config verify` and
`golangci-lint run`, both with `--network=none`.
`make fmt-check`, then `script/assets` to extract Alpine.js from
`3p/`, then `golangci-lint config verify` and `golangci-lint run`,
both with `--network=none`.
2. **Builder stage** (`golang:1.26.1-bookworm`) — depends on the lint
stage passing (it copies a file from it), runs `make test` and
`make build` (both extract Alpine.js from `3p/` first), and finally
@@ -199,6 +199,40 @@ func TestStatisticsQueriesUseTheirIndexes(t *testing.T) {
"(deleted_at=? AND created_at>?)")
}
// TestResubmitCountUsesItsIndex does the same for the event log's count
// of the events resubmitted from each of a page's events (resubmitCounts
// in the handlers). It passes a full page of 25 ids: with an index on
// resubmitted_from_id alone, SQLite uses it for three ids and turns to
// the deleted_at index from five.
func TestResubmitCountUsesItsIndex(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
defer func() { require.NoError(t, lc.Stop(ctx)) }()
db, err := mgr.GetDB(uuid.New().String())
require.NoError(t, err)
dry := db.Session(&gorm.Session{DryRun: true})
page := make([]string, 25)
for i := range page {
page[i] = uuid.New().String()
}
var counts []struct{ Total int }
assertPlanUses(t, db, dry.Model(&database.Event{}).
Select("resubmitted_from_id, count(*) AS total").
Where("resubmitted_from_id IN ?", page).
Group("resubmitted_from_id").Find(&counts),
"idx_events_resubmitted_from_id "+
"(resubmitted_from_id=? AND deleted_at=?)")
}
// assertPlanUses asserts that SQLite's plan for a statement GORM built
// in a dry run, run with the same SQL and arguments GORM would send,
// names each of the given indexes.
+5 -3
View File
@@ -19,8 +19,10 @@ type Event struct {
// narrows by a < only on the last column it uses. Its final delete
// has no deleted_at condition and uses the index on created_at
// alone. The other tables keep the unindexed BaseModel created_at.
CreatedAt time.Time `gorm:"index;index:idx_events_deleted_at_created_at,priority:2" json:"createdAt"`
DeletedAt gorm.DeletedAt `gorm:"index:idx_events_deleted_at_created_at,priority:1" json:"deletedAt,omitzero"`
// DeletedAt is also the second column of the resubmitted_from_id
// index, for the reason DeliveryResult gives.
CreatedAt time.Time `gorm:"index;index:idx_events_deleted_at_created_at,priority:2" json:"createdAt"`
DeletedAt gorm.DeletedAt `gorm:"index:idx_events_deleted_at_created_at,priority:1;index:idx_events_resubmitted_from_id,priority:2" json:"deletedAt,omitzero"`
WebhookID string `gorm:"type:uuid;not null" json:"webhookId"`
EntrypointID string `gorm:"type:uuid;not null" json:"entrypointId"`
@@ -42,7 +44,7 @@ type Event struct {
// existed. It is not a foreign key: the source event can be
// reaped by retention while its copies remain, and the id is
// kept as the record of where the copy came from either way.
ResubmittedFromID *string `gorm:"type:uuid;index" json:"resubmittedFromId,omitempty"`
ResubmittedFromID *string `gorm:"type:uuid;index:idx_events_resubmitted_from_id,priority:1" json:"resubmittedFromId,omitempty"`
// Relations. No model marshals the record it belongs to, so
// Webhook and Entrypoint are left out of the JSON.
@@ -515,6 +515,47 @@ func (w *archiveWriter) prune(expiry time.Duration) {
}
}
// ArchiveFileInfo is what the metadata of a database target's archive
// file says about it.
type ArchiveFileInfo struct {
// Size is the bytes on disk of the file and its -wal together.
Size int64
// Written is when the file or its -wal was last modified, whichever
// is later: a write lands in the -wal first.
Written time.Time
}
// StatArchive reads the metadata of the archive file at path and of
// its -wal, without opening the archive. With no file at path, which is
// so before the first write and after the operator moved it away, the
// error wraps fs.ErrNotExist.
func StatArchive(path string) (ArchiveFileInfo, error) {
file, err := os.Stat(path)
if err != nil {
return ArchiveFileInfo{}, err
}
info := ArchiveFileInfo{Size: file.Size(), Written: file.ModTime()}
wal, err := os.Stat(path + "-wal")
if errors.Is(err, fs.ErrNotExist) {
return info, nil
}
if err != nil {
return ArchiveFileInfo{}, err
}
info.Size += wal.Size()
if wal.ModTime().After(info.Written) {
info.Written = wal.ModTime()
}
return info, nil
}
// fileExists reports whether a path currently exists.
func fileExists(path string) bool {
_, err := os.Stat(path)
+44
View File
@@ -3,6 +3,7 @@ package delivery_test
import (
"database/sql"
"fmt"
"io/fs"
"log/slog"
"os"
"path/filepath"
@@ -183,6 +184,49 @@ func TestArchiveWriter_RecreatesAfterRemoval(
assert.Equal(t, "b", got[0].EventID)
}
// TestStatArchive proves StatArchive finds no file before the first
// write; after a write still held in the -wal, counts the -wal in the
// size and takes its later time as the last write; and finds no file
// again once the file has been moved away.
func TestStatArchive(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive-wh.db")
_, err := delivery.StatArchive(path)
require.ErrorIs(t, err, fs.ErrNotExist)
// With the clock stopped, the reopen debounce never passes, so
// the handle stays open after the write.
stopped := time.Now()
w := delivery.NewExportArchiveWriter(path, archiveTestLogger(), 0)
w.SetNow(func() time.Time { return stopped })
require.NoError(t, w.Write(delivery.ExportArchivedEvent{EventID: "a"}, 0))
written := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
earlier := written.Add(-time.Hour)
require.NoError(t, os.Chtimes(path, earlier, earlier))
require.NoError(t, os.Chtimes(path+"-wal", written, written))
file, err := os.Stat(path)
require.NoError(t, err)
wal, err := os.Stat(path + "-wal")
require.NoError(t, err)
require.Positive(t, wal.Size())
got, err := delivery.StatArchive(path)
require.NoError(t, err)
assert.Equal(t, file.Size()+wal.Size(), got.Size)
assert.True(t, written.Equal(got.Written), got.Written)
removeArchiveFiles(t, path)
_, err = delivery.StatArchive(path)
require.ErrorIs(t, err, fs.ErrNotExist)
}
func TestArchiveWriter_ReopenDebounce(t *testing.T) {
t.Parallel()
+12 -9
View File
@@ -137,20 +137,22 @@ func (s *Handlers) RenderTemplateForTest(
// BuildSlackTargetConfigForTest exposes
// buildSlackTargetConfig for use in the handlers_test package.
func (s *Handlers) BuildSlackTargetConfigForTest(
ctx context.Context,
w http.ResponseWriter,
r *http.Request,
targetURL string,
) (string, string, error) {
return s.buildSlackTargetConfig(ctx, targetURL)
) (string, error) {
return s.buildSlackTargetConfig(w, r, targetURL)
}
// BuildHTTPTargetConfigForTest exposes buildHTTPTargetConfig
// for use in the handlers_test package, taking the form fields
// an HTTP target's configuration is built from.
func (s *Handlers) BuildHTTPTargetConfigForTest(
ctx context.Context,
w http.ResponseWriter,
r *http.Request,
targetURL, headers, timeout string,
) (string, string, error) {
return s.buildHTTPTargetConfig(ctx, targetFormInput{
) (string, error) {
return s.buildHTTPTargetConfig(w, r, targetFormInput{
URL: targetURL,
Headers: headers,
Timeout: timeout,
@@ -160,8 +162,9 @@ func (s *Handlers) BuildHTTPTargetConfigForTest(
// BuildDatabaseTargetConfigForTest exposes
// buildDatabaseTargetConfig for use in the handlers_test
// package.
func BuildDatabaseTargetConfigForTest(
func (s *Handlers) BuildDatabaseTargetConfigForTest(
w http.ResponseWriter,
expiry string,
) (string, string, error) {
return buildDatabaseTargetConfig(expiry)
) (string, error) {
return s.buildDatabaseTargetConfig(w, newRequestForTest(), expiry)
}
+43 -20
View File
@@ -314,12 +314,16 @@ func TestBuildSlackTargetConfig_AcceptsPublicURL(t *testing.T) {
t.Cleanup(app.RequireStop)
cfg, errMsg, err := h.BuildSlackTargetConfigForTest(
t.Context(), "http://93.184.216.34/services/T00/B00/xxx",
req := httptest.NewRequestWithContext(
context.Background(), http.MethodPost, "/", nil)
w := httptest.NewRecorder()
cfg, err := h.BuildSlackTargetConfigForTest(
w, req, "http://93.184.216.34/services/T00/B00/xxx",
)
require.NoError(t, err)
assert.Empty(t, errMsg)
assert.Equal(t, http.StatusOK, w.Code)
assert.Contains(t, cfg, "webhookUrl")
}
@@ -333,13 +337,17 @@ func TestBuildSlackTargetConfig_RejectsReservedURL(t *testing.T) {
t.Cleanup(app.RequireStop)
cfg, errMsg, err := h.BuildSlackTargetConfigForTest(
t.Context(), "http://169.254.169.254/latest/meta-data/",
req := httptest.NewRequestWithContext(
context.Background(), http.MethodPost, "/", nil)
w := httptest.NewRecorder()
cfg, err := h.BuildSlackTargetConfigForTest(
w, req, "http://169.254.169.254/latest/meta-data/",
)
require.NoError(t, err)
assert.Contains(t, errMsg, "Invalid target URL")
require.Error(t, err)
assert.Empty(t, cfg)
assert.Equal(t, http.StatusBadRequest, w.Code)
}
func TestRenderTemplate(t *testing.T) {
@@ -436,22 +444,29 @@ func TestRenderTemplateMidRenderErrorSendsNoPartialBody(t *testing.T) {
func TestBuildDatabaseTargetConfig_Valid(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
app := newTestApp(t, &h)
app.RequireStart()
t.Cleanup(app.RequireStop)
// Empty expiry: the keep-forever default, empty config.
cfg, errMsg, err := handlers.BuildDatabaseTargetConfigForTest("")
w := httptest.NewRecorder()
cfg, err := h.BuildDatabaseTargetConfigForTest(w, "")
require.NoError(t, err)
assert.Empty(t, errMsg)
assert.Empty(t, cfg)
// Explicit never is stored as config.
cfg, errMsg, err = handlers.BuildDatabaseTargetConfigForTest("never")
w = httptest.NewRecorder()
cfg, err = h.BuildDatabaseTargetConfigForTest(w, "never")
require.NoError(t, err)
assert.Empty(t, errMsg)
assert.JSONEq(t, `{"expiry":"never"}`, cfg)
// A positive duration is stored as config.
cfg, errMsg, err = handlers.BuildDatabaseTargetConfigForTest("720h")
w = httptest.NewRecorder()
cfg, err = h.BuildDatabaseTargetConfigForTest(w, "720h")
require.NoError(t, err)
assert.Empty(t, errMsg)
assert.JSONEq(t, `{"expiry":"720h"}`, cfg)
}
@@ -460,14 +475,22 @@ func TestBuildDatabaseTargetConfig_RejectsBadExpiry(
) {
t.Parallel()
for _, bad := range []string{"nonsense", "7d", "-5h"} {
cfg, errMsg, err := handlers.BuildDatabaseTargetConfigForTest(bad)
var h *handlers.Handlers
require.NoError(t, err)
assert.Contains(
t, errMsg, "Invalid archive expiry",
"expiry %q should be refused", bad,
)
app := newTestApp(t, &h)
app.RequireStart()
t.Cleanup(app.RequireStop)
for _, bad := range []string{"nonsense", "7d", "-5h"} {
w := httptest.NewRecorder()
cfg, err := h.BuildDatabaseTargetConfigForTest(w, bad)
require.Error(t, err, "expiry %q", bad)
assert.Empty(t, cfg)
assert.Equal(
t, http.StatusBadRequest, w.Code,
"expiry %q should be rejected with 400", bad,
)
}
}
+159 -143
View File
@@ -1,7 +1,6 @@
package handlers
import (
"context"
"encoding/json"
"errors"
"fmt"
@@ -39,6 +38,9 @@ type WebhookListItem struct {
EventsUnreadable bool
}
// errMissingURL signals that a required URL was not provided.
var errMissingURL = errors.New("missing URL")
// parseRetentionDays interprets a retention_days form value. It
// returns the number of days, or, for a value it refuses, the message
// the create and edit forms show; the message is empty when the value
@@ -458,20 +460,15 @@ func (h *Handlers) HandleSourceDetail() http.HandlerFunc {
return
}
h.renderSourceDetail(w, r, webhook, targetFormInput{}, "")
h.renderSourceDetail(w, r, webhook)
}
}
// renderSourceDetail loads and renders a source detail page. With a
// targetErr, it is the page shown again for a refused add target
// form: it answers 400, and the form opens on targetForm's type with
// its values and the message.
// renderSourceDetail loads and renders a source detail page.
func (h *Handlers) renderSourceDetail(
w http.ResponseWriter,
r *http.Request,
webhook database.Webhook,
targetForm targetFormInput,
targetErr string,
) {
var entrypoints []database.Entrypoint
@@ -524,20 +521,13 @@ func (h *Handlers) renderSourceDetail(
// target's stored config blob holds a credential, and it
// must never reach a template.
"Entrypoints": NewEntrypointViews(entrypoints),
"Targets": delivery.NewTargetViews(targets),
"Targets": h.targetRows(&webhook, targets),
"Events": events,
"BaseURL": baseURL,
"Stats": h.loadWebhookStats(webhook.ID, entrypoints, targets),
"TargetForm": targetForm,
"TargetError": targetErr,
}
status := http.StatusOK
if targetErr != "" {
status = http.StatusBadRequest
}
h.renderTemplateStatus(w, r, "source_detail.html", data, status)
h.renderTemplate(w, r, "source_detail.html", data)
}
// HandleSourceEdit shows the form to edit a webhook.
@@ -1447,29 +1437,66 @@ func (h *Handlers) HandleTargetCreate() http.HandlerFunc {
}
}
// processTargetCreate validates and creates a new target. A refused
// submission shows the webhook page again, with the add target form
// open on the chosen type, the values entered, and the reason.
// processTargetCreate validates and creates a new target.
func (h *Handlers) processTargetCreate(
w http.ResponseWriter,
r *http.Request,
webhook database.Webhook,
) {
in := targetFormInputFrom(r)
// The body size cap is enforced by the MaxBodySize middleware,
// which runs before CSRF parses the form.
//
// Every field here is read with PostFormValue, not FormValue.
// FormValue falls back to the query string, which would let
// `POST /hook/{id}/targets?url=https://hooks.slack.com/...`
// configure a target from a value the request line carries — and
// the request line, unlike the body, is what logs, proxies,
// Referer headers and error trackers record.
name := r.PostFormValue("name")
targetType := database.TargetType(r.PostFormValue("type"))
target, errMsg, err := h.newTarget(r.Context(), webhook.ID, in)
if err != nil {
h.serverError(w, r, "failed to encode target config", err)
if name == "" {
http.Error(
w, "Name is required", http.StatusBadRequest,
)
return
}
if errMsg != "" {
h.renderSourceDetail(w, r, webhook, in, errMsg)
if !isValidTargetType(targetType) {
http.Error(
w, "Invalid target type",
http.StatusBadRequest,
)
return
}
configJSON, err := h.buildTargetConfig(
w, r, targetType, targetFormInputFrom(r),
)
if err != nil {
return
}
// A new target has no stored retry count, so an absent field
// takes the fire-and-forget default. A field the operator filled
// in with something invalid is rejected rather than becoming
// that default.
maxRetries, ok := targetMaxRetries(w, r, 0)
if !ok {
return
}
target := &database.Target{
WebhookID: webhook.ID,
Name: name,
Type: targetType,
Active: true,
Config: configJSON,
MaxRetries: maxRetries,
}
err = h.db.DB().Create(target).Error
if err != nil {
h.serverError(w, r, "failed to create target", err)
@@ -1483,49 +1510,6 @@ func (h *Handlers) processTargetCreate(
)
}
// newTarget validates a new target for a webhook and returns the row
// to create, or, when it refuses the target, the message the form
// shows. An error is the server's fault, not a refusal: the accepted
// configuration could not be encoded. Every form that creates a
// target goes through here, so they all accept and refuse the same
// things.
func (h *Handlers) newTarget(
ctx context.Context,
webhookID string,
in targetFormInput,
) (*database.Target, string, error) {
if in.Name == "" {
return nil, "Name is required", nil
}
if !isValidTargetType(in.Type) {
return nil, "Invalid target type", nil
}
configJSON, errMsg, err := h.buildTargetConfig(ctx, in.Type, in)
if err != nil || errMsg != "" {
return nil, errMsg, err
}
// A new target has no stored retry count, so an absent field
// takes the fire-and-forget default. A field the operator filled
// in with something invalid is refused rather than becoming
// that default.
maxRetries, err := parseMaxRetries(in.MaxRetries, 0)
if err != nil {
return nil, "Invalid max retries: " + retriesErrorMessage(err), nil
}
return &database.Target{
WebhookID: webhookID,
Name: in.Name,
Type: in.Type,
Active: true,
Config: configJSON,
MaxRetries: maxRetries,
}, "", nil
}
// isValidTargetType checks whether the target type is supported.
func isValidTargetType(tt database.TargetType) bool {
switch tt {
@@ -1557,16 +1541,11 @@ func pageOrFirst(s string) int {
return v
}
// targetFormInput carries the raw values of a target form. Both the
// create and the edit path fill one and hand it to buildTargetConfig,
// so neither can come to validate a destination differently from the
// other. A refused add target form is shown again from it.
// targetFormInput carries the raw form values describing a target's
// configuration. Both the create and the edit path fill one and hand
// it to buildTargetConfig, so neither can come to validate a
// destination differently from the other.
type targetFormInput struct {
// Name is the target's name.
Name string
// Type is the type chosen on the add target form. The edit form
// has none: a target's stored type decides.
Type database.TargetType
// URL is the destination for an HTTP target and the webhook URL
// for a Slack target.
URL string
@@ -1575,15 +1554,13 @@ type targetFormInput struct {
Headers string
// Timeout is an HTTP target's per-request timeout in seconds.
Timeout string
// MaxRetries is an HTTP or Slack target's max_retries.
MaxRetries string
// Expiry is a database (archive) target's row expiry.
Expiry string
}
// targetFormInputFrom reads a target form from a request body. The
// body size cap is enforced by the MaxBodySize middleware, which runs
// before CSRF parses the form.
// targetFormInputFrom reads the configuration fields from a request
// body. The body size cap is enforced by the MaxBodySize middleware,
// which runs before CSRF parses the form.
//
// Every field is read with PostFormValue, not FormValue. FormValue
// falls back to the query string, which would let
@@ -1595,38 +1572,38 @@ type targetFormInput struct {
// tokens.
func targetFormInputFrom(r *http.Request) targetFormInput {
return targetFormInput{
Name: r.PostFormValue("name"),
Type: database.TargetType(r.PostFormValue("type")),
URL: r.PostFormValue("url"),
Headers: r.PostFormValue("headers"),
Timeout: r.PostFormValue("timeout"),
MaxRetries: r.PostFormValue("max_retries"),
Expiry: r.PostFormValue("expiry"),
URL: r.PostFormValue("url"),
Headers: r.PostFormValue("headers"),
Timeout: r.PostFormValue("timeout"),
Expiry: r.PostFormValue("expiry"),
}
}
// buildTargetConfig builds the JSON config string for a target from
// the submitted form values, or returns the message the form shows
// for a value it refuses. An error is the server's fault, not a
// refusal: the accepted configuration could not be encoded. Which
// fields of in apply depends on the target type; a type without a URL
// ignores any URL submitted.
// the submitted form values, writing its own 4xx response on
// rejection. Which fields of in apply depends on the target type.
func (h *Handlers) buildTargetConfig(
ctx context.Context,
w http.ResponseWriter,
r *http.Request,
targetType database.TargetType,
in targetFormInput,
) (string, string, error) {
) (string, error) {
switch targetType {
case database.TargetTypeHTTP:
return h.buildHTTPTargetConfig(ctx, in)
return h.buildHTTPTargetConfig(w, r, in)
case database.TargetTypeSlack:
return h.buildSlackTargetConfig(ctx, in.URL)
return h.buildSlackTargetConfig(w, r, in.URL)
case database.TargetTypeDatabase:
return buildDatabaseTargetConfig(in.Expiry)
return h.buildDatabaseTargetConfig(w, r, in.Expiry)
case database.TargetTypeLog:
return "", "", nil
return "", nil
default:
return "", "Invalid target type", nil
http.Error(
w, "Invalid target type",
http.StatusBadRequest,
)
return "", errMissingURL
}
}
@@ -1634,73 +1611,92 @@ func (h *Handlers) buildTargetConfig(
// SSRF-validated destination plus the optional headers and timeout
// the delivery path honours.
func (h *Handlers) buildHTTPTargetConfig(
ctx context.Context,
w http.ResponseWriter,
r *http.Request,
in targetFormInput,
) (string, string, error) {
errMsg := h.validateTargetURL(
ctx, in.URL, "URL is required for HTTP targets",
) (string, error) {
err := h.validateTargetURL(
w, r, in.URL, "URL is required for HTTP targets",
)
if errMsg != "" {
return "", errMsg, nil
if err != nil {
return "", err
}
headers, err := delivery.ParseTargetHeaders(in.Headers)
if err != nil {
return "", fmt.Sprintf("Invalid headers: %v", err), nil
http.Error(
w,
"Invalid headers: "+err.Error(),
http.StatusBadRequest,
)
return "", err
}
timeout, err := delivery.ParseTargetTimeout(in.Timeout)
if err != nil {
return "", fmt.Sprintf("Invalid timeout: %v", err), nil
http.Error(
w,
"Invalid timeout: "+err.Error(),
http.StatusBadRequest,
)
return "", err
}
configJSON, err := marshalTargetConfig(delivery.HTTPTargetConfig{
return h.marshalTargetConfig(w, r, delivery.HTTPTargetConfig{
URL: in.URL,
Headers: headers,
Timeout: timeout,
})
return configJSON, "", err
}
// buildSlackTargetConfig builds config JSON for a Slack target,
// whose whole configuration is one SSRF-validated webhook URL.
func (h *Handlers) buildSlackTargetConfig(
ctx context.Context,
w http.ResponseWriter,
r *http.Request,
targetURL string,
) (string, string, error) {
errMsg := h.validateTargetURL(
ctx, targetURL,
) (string, error) {
err := h.validateTargetURL(
w, r, targetURL,
"Webhook URL is required for Slack targets",
)
if errMsg != "" {
return "", errMsg, nil
if err != nil {
return "", err
}
configJSON, err := marshalTargetConfig(delivery.SlackTargetConfig{
return h.marshalTargetConfig(w, r, delivery.SlackTargetConfig{
WebhookURL: targetURL,
})
return configJSON, "", err
}
// validateTargetURL refuses an empty or SSRF-blocked destination,
// returning the message the form shows, or "" when the destination
// is accepted. missingMsg is the message for no URL at all.
// validateTargetURL rejects an empty or SSRF-blocked destination,
// writing the 400 itself. missingMsg is the error shown when no URL
// is given.
//
// It is the single point at which a user-supplied destination enters
// the SSRF guard, on create and on edit alike. An edit path that
// reached storage without passing through here would reopen the hole
// the guard closes.
func (h *Handlers) validateTargetURL(
ctx context.Context,
w http.ResponseWriter,
r *http.Request,
targetURL, missingMsg string,
) string {
) error {
if targetURL == "" {
return missingMsg
http.Error(
w,
missingMsg,
http.StatusBadRequest,
)
return errMissingURL
}
err := h.ssrf.ValidateTargetURL(ctx, targetURL)
err := h.ssrf.ValidateTargetURL(
r.Context(), targetURL,
)
if err != nil {
// The submitted URL can be a credential (a Slack
// incoming webhook URL is a bearer token), so the log
@@ -1726,16 +1722,25 @@ func (h *Handlers) validateTargetURL(
"egress to your own network\" in the README)."
}
return msg
http.Error(w, msg, http.StatusBadRequest)
return err
}
return ""
return nil
}
// marshalTargetConfig serialises a target configuration for storage.
func marshalTargetConfig(cfg any) (string, error) {
// marshalTargetConfig serialises a target configuration for storage,
// writing a 500 itself if it cannot.
func (h *Handlers) marshalTargetConfig(
w http.ResponseWriter,
r *http.Request,
cfg any,
) (string, error) {
configBytes, err := json.Marshal(cfg)
if err != nil {
h.serverError(w, r, "failed to encode target config", err)
return "", err
}
@@ -1743,24 +1748,35 @@ func marshalTargetConfig(cfg any) (string, error) {
}
// buildDatabaseTargetConfig builds config JSON for a database
// (archive) target. The optional expiry is validated here, at
// creation time, so an unparseable value is refused instead of
// failing every subsequent delivery. An empty expiry yields an
// empty config (the keep-forever default).
func buildDatabaseTargetConfig(expiry string) (string, string, error) {
// (archive) target. The optional expiry (a form value read by
// the caller, which bounds the request body) is validated here,
// at creation time, so an unparseable value is rejected with a
// 400 instead of failing every subsequent delivery. An empty
// expiry yields an empty config (the keep-forever default).
func (h *Handlers) buildDatabaseTargetConfig(
w http.ResponseWriter,
r *http.Request,
expiry string,
) (string, error) {
expiry = strings.TrimSpace(expiry)
if expiry == "" {
return "", "", nil
return "", nil
}
err := delivery.ValidateArchiveExpiry(expiry)
if err != nil {
return "", fmt.Sprintf("Invalid archive expiry: %v", err), nil
http.Error(
w,
"Invalid archive expiry: "+err.Error(),
http.StatusBadRequest,
)
return "", err
}
configJSON, err := marshalTargetConfig(map[string]any{"expiry": expiry})
return configJSON, "", err
return h.marshalTargetConfig(
w, r, map[string]any{"expiry": expiry},
)
}
// HandleEntrypointDelete handles deleting an entrypoint.
-154
View File
@@ -1,154 +0,0 @@
package handlers_test
import (
"html"
"net/http"
"net/url"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database"
)
// TestHandleTargetCreate_EveryType adds a target of each type. Each
// submission carries a url: only the http and slack types store one.
func TestHandleTargetCreate_EveryType(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
// fields is the rest of each submission, as a query string.
cases := []struct {
targetType database.TargetType
fields string
wantConfig string
wantRetries int
}{
{
database.TargetTypeHTTP, "timeout=12&max_retries=3",
`{"url":"` + editOriginalURL + `","timeout":12}`, 3,
},
{
database.TargetTypeSlack, "max_retries=4",
`{"webhookUrl":"` + editOriginalURL + `"}`, 4,
},
{
database.TargetTypeDatabase, "expiry=720h",
`{"expiry":"720h"}`, 0,
},
{database.TargetTypeLog, "", "", 0},
}
for _, tc := range cases {
t.Run(string(tc.targetType), func(t *testing.T) {
t.Parallel()
webhook := seedWebhookWithRetention(t, env.db, 30)
form, err := url.ParseQuery(tc.fields)
require.NoError(t, err)
form.Set("name", "every-type")
form.Set("type", string(tc.targetType))
form.Set("url", editOriginalURL)
w := serveTarget(
env, http.MethodPost,
"/hook/"+webhook.ID+"/targets", form,
)
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1)
assert.Equal(t, tc.targetType, targets[0].Type)
assert.Equal(t, tc.wantRetries, targets[0].MaxRetries)
if tc.wantConfig == "" {
assert.Empty(t, targets[0].Config)
} else {
assert.JSONEq(t, tc.wantConfig, targets[0].Config)
}
})
}
}
// TestHandleTargetCreate_RefusedFormComesBack refuses a target of each
// type and checks that the webhook page comes back with the add target
// form open on that type, the values entered, and the reason.
func TestHandleTargetCreate_RefusedFormComesBack(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
// fields is what the operator typed, as a query string.
cases := []struct {
targetType database.TargetType
fields string
reason string
}{
{
database.TargetTypeHTTP,
"name=private&url=" + editBlockedURL +
"&timeout=12&max_retries=3",
"Invalid target URL",
},
{
database.TargetTypeSlack, "name=no-url&max_retries=4",
"Webhook URL is required for Slack targets",
},
{
database.TargetTypeDatabase, "name=archive&expiry=7d",
"Invalid archive expiry",
},
{database.TargetTypeLog, "name=", "Name is required"},
}
for _, tc := range cases {
t.Run(string(tc.targetType), func(t *testing.T) {
t.Parallel()
webhook := seedWebhookWithRetention(t, env.db, 30)
typed, err := url.ParseQuery(tc.fields)
require.NoError(t, err)
form := url.Values{}
form.Set("type", string(tc.targetType))
for field := range typed {
form.Set(field, typed.Get(field))
}
w := serveTarget(
env, http.MethodPost,
"/hook/"+webhook.ID+"/targets", form,
)
assert.Equal(t, http.StatusBadRequest, w.Code)
page := w.Body.String()
assert.Contains(
t, page, `data-type="`+string(tc.targetType)+`"`,
)
assert.Contains(t, page, html.EscapeString(tc.reason))
// Each value comes back in a data attribute of the targets
// section named after its field (max_retries as
// data-max-retries), except url, which comes back in
// data-destination; templates/source_detail.html says why.
for field := range typed {
attr := "data-" + strings.ReplaceAll(field, "_", "-")
if field == "url" {
attr = "data-destination"
}
assert.Contains(
t, page, attr+`="`+
html.EscapeString(typed.Get(field))+`"`,
)
}
assert.Empty(t, targetsForWebhook(t, env.db, webhook.ID))
})
}
}
+6 -11
View File
@@ -120,23 +120,18 @@ func (h *Handlers) applyTargetEdit(
) {
name := r.PostFormValue("name")
if name == "" {
http.Error(w, "Name is required", http.StatusBadRequest)
http.Error(
w, "Name is required", http.StatusBadRequest,
)
return
}
configJSON, errMsg, err := h.buildTargetConfig(
r.Context(), target.Type, targetFormInputFrom(r),
configJSON, err := h.buildTargetConfig(
w, r, target.Type, targetFormInputFrom(r),
)
if err != nil {
h.serverError(w, r, "failed to encode target config", err)
return
}
if errMsg != "" {
http.Error(w, errMsg, http.StatusBadRequest)
// buildTargetConfig has already written the response.
return
}
+90
View File
@@ -0,0 +1,90 @@
package handlers
import (
"errors"
"io/fs"
"path/filepath"
"time"
"github.com/dustin/go-humanize"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
)
// TargetRowView is one row of the target list on a webhook's page.
type TargetRowView struct {
delivery.TargetView
// Archive is a database target's archive file, and nil for a target
// of any other type.
Archive *ArchiveFileView
}
// ArchiveFileView is what a database target's row shows about its
// archive file.
type ArchiveFileView struct {
Name string
// Note stands in for the size and the last write when there are
// none to show, and is empty when there are.
Note string
// Size is the size on disk. Written is how long ago the file was
// last written, and WrittenUTC the full time the page shows on
// hover.
Size string
Written string
WrittenUTC string
}
// targetRows projects a webhook's targets for the target list on its
// page.
func (h *Handlers) targetRows(
webhook *database.Webhook, targets []database.Target,
) []TargetRowView {
views := delivery.NewTargetViews(targets)
rows := make([]TargetRowView, len(views))
// NewTargetViews returns one view per target, in order.
for i := range views {
rows[i].TargetView = views[i]
if targets[i].Type == database.TargetTypeDatabase {
rows[i].Archive = h.archiveFileView(webhook, &targets[i])
}
}
return rows
}
// archiveFileView describes a database target's archive file from the
// file's metadata alone; the archive is never opened. The file is found
// by the name the archive writer uses, so it follows a rename of the
// webhook or the target.
func (h *Handlers) archiveFileView(
webhook *database.Webhook, target *database.Target,
) *ArchiveFileView {
path := delivery.ArchivePath(h.dbMgr, webhook, target)
view := &ArchiveFileView{Name: filepath.Base(path)}
file, err := delivery.StatArchive(path)
switch {
case errors.Is(err, fs.ErrNotExist):
view.Note = "not created yet"
case err != nil:
h.log.Error(
"failed to read archive file metadata",
"target_id", target.ID,
"error", err,
)
view.Note = "could not be read"
default:
view.Size = humanize.Bytes(uint64(file.Size)) //nolint:gosec // never negative
view.Written = humanize.Time(file.Written)
view.WrittenUTC = file.Written.UTC().Format(time.DateTime) + " UTC"
}
return view
}
+71
View File
@@ -0,0 +1,71 @@
package handlers_test
import (
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/session"
)
// TestHandleSourceDetail_ShowsArchiveFile proves a database target's
// row names its archive file and says "not created yet" before the
// first write, adds the file's size and last write once it has one row,
// and says "not created yet" again once the file has been moved away.
// A target of another type shows no archive file.
func TestHandleSourceDetail_ShowsArchiveFile(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
archive := seedTarget(t, db, wh.ID, database.TargetTypeDatabase)
seedTarget(t, db, wh.ID, database.TargetTypeLog)
path := delivery.ArchivePath(dbMgr, wh, archive)
body := renderSourceDetailPage(t, h, sess, wh.ID)
assert.Equal(t, 1, strings.Count(body, "Archive File:"))
assert.Contains(t, body, filepath.Base(path))
assert.Contains(t, body, "not created yet")
assert.NotContains(t, body, "Archive Size:")
seedArchive(t, path, 1, 100)
file, err := os.Stat(path)
require.NoError(t, err)
body = renderSourceDetailPage(t, h, sess, wh.ID)
assert.Contains(t, body, filepath.Base(path))
assert.NotContains(t, body, "not created yet")
assert.Regexp(t,
`Archive Size:</span>\s*<span>[1-9][0-9.]* [kM]?B</span>`, body,
)
assert.Contains(t, body,
`title="`+file.ModTime().UTC().Format(time.DateTime)+` UTC"`,
)
require.NoError(t, os.Rename(path, filepath.Join(t.TempDir(), "moved.db")))
body = renderSourceDetailPage(t, h, sess, wh.ID)
assert.Contains(t, body, filepath.Base(path))
assert.Contains(t, body, "not created yet")
assert.NotContains(t, body, "Archive Size:")
}
@@ -1,7 +1,6 @@
package handlers_test
import (
"html"
"net/http"
"net/url"
"testing"
@@ -44,16 +43,12 @@ func TestTargetRefusal_PrivateDestinationSaysHowToAllowIt(
form.Set("type", string(targetType))
form.Set("url", editBlockedURL)
// A refused add shows the webhook page again, where
// the hint is HTML-escaped; a refused edit answers in
// plain text.
added := serveTarget(
env, http.MethodPost, targetsPath, form,
)
assert.Equal(t, http.StatusBadRequest, added.Code)
assert.Contains(
t, added.Body.String(),
html.EscapeString(privateRefusalHint),
t, added.Body.String(), privateRefusalHint,
)
form.Set("url", editOriginalURL)
+4 -5
View File
@@ -90,14 +90,13 @@ func retriesErrorMessage(err error) string {
", or 0 for fire-and-forget"
}
// targetMaxRetries reads and validates max_retries from a target edit
// targetMaxRetries reads and validates max_retries from a target form
// submission, answering the request with a 400 and reporting false
// when the value is set but invalid.
//
// It and the create path (newTarget) both use parseMaxRetries and
// retriesErrorMessage, so the two cannot come to disagree about what a
// valid retry count is. The wording matches the timeout control on
// the same submission.
// Both the create and the edit path go through here, so the two
// cannot come to disagree about what a valid retry count is. The
// wording matches the timeout control on the same submission.
func targetMaxRetries(
w http.ResponseWriter,
r *http.Request,
+86 -194
View File
@@ -83,37 +83,8 @@ func TestAlpineRunsUnderTheSecurityPolicy(t *testing.T) {
page := srv.URL + "/hook/" + webhook.ID
checkAddEntrypoint(ctx, t, page)
// Each target type, with the fields its add target form submits, in
// page order. Only http and slack have a url field.
targetTypes := []struct {
name string
fields string
values map[string]string
}{
{
"http", "csrf_token name type url headers timeout max_retries",
map[string]string{"url": publicTargetURL},
},
{
"slack", "csrf_token name type url max_retries",
map[string]string{"url": publicTargetURL},
},
{
"database", "csrf_token name type expiry",
map[string]string{"expiry": "720h"},
},
{"log", "csrf_token name type", nil},
}
for _, tt := range targetTypes {
checkAddTarget(
ctx, t, page, tt.name, strings.Fields(tt.fields), tt.values,
)
}
checkRefusedTarget(ctx, t, page)
checkAddForms(ctx, t, page)
checkTargetType(ctx, t, page+"/events")
checkCopy(ctx, t, page)
checkEventLog(ctx, t, page+"/events", event.ID, target.Name)
checkMobileMenu(ctx, t, page)
@@ -256,194 +227,115 @@ func click(ctx context.Context, t *testing.T, xpath string) {
))
}
// checkAddEntrypoint loads a webhook page and checks that the add
// entrypoint form stays hidden until the Add button beside its heading
// is clicked. The click looks for a button element there, so it also
// checks that Add is one.
func checkAddEntrypoint(ctx context.Context, t *testing.T, url string) {
t.Helper()
form := `form[action$="/entrypoints"]`
require.NoError(t, chromedp.Run(ctx, loadPage(url)))
assert.True(t, hidden(ctx, form),
"the add entrypoint form shows before Add is clicked")
click(ctx, t, `//h2[text()="Entrypoints"]/following-sibling::button`)
assert.True(t, shown(ctx, form),
"the add entrypoint form stays hidden when Add is clicked")
}
// publicTargetURL is a destination the server accepts for an http or
// slack target. It is a literal public address, so accepting it needs
// no DNS.
const publicTargetURL = "https://93.184.216.34/hook"
// The parts of the targets section's add target form the checks below
// find and click. Add is the button beside the Targets heading; each
// Cancel is found from the button beside it, since both are on the
// page at once.
const (
addTarget = `//h2[text()="Targets"]/following-sibling::button`
typeSelect = `//select[@aria-label="Target type"]`
nextButton = `//button[text()="Next"]`
cancelChoice = nextButton + `/following-sibling::button[text()="Cancel"]`
saveButton = `//form[contains(@action, "/targets")]//button[text()="Save"]`
cancelFields = saveButton + `/following-sibling::button[text()="Cancel"]`
targetName = `form[action$="/targets"] input[name="name"]`
submittedKeys = `[...new FormData(
document.querySelector('form[action$="/targets"]')).keys()]`
)
// checkAddTarget loads a webhook page and walks the add target form for
// one target type. The form shows nothing until Add is clicked; Add
// shows only the type choice; Cancel there closes it; Next shows the
// type's own fields in place of the choice, and the form then submits
// exactly fields, so a field another type uses, such as url, is absent;
// Cancel closes it again. It then adds a target of the type, filling in
// values, and checks that the section lists it with that type.
func checkAddTarget(
ctx context.Context,
t *testing.T,
url, targetType string,
fields []string,
values map[string]string,
) {
// checkAddForms loads a webhook page and checks that each section's add
// form stays hidden until the Add button beside its heading is clicked.
// The click looks for a button element there, so it also checks that
// Add is one.
func checkAddForms(ctx context.Context, t *testing.T, url string) {
t.Helper()
require.NoError(t, chromedp.Run(ctx, loadPage(url)))
assert.Truef(t, hidden(ctx, typeSelect),
"%s: the type choice shows before Add is clicked", targetType)
assert.Truef(t, hidden(ctx, targetName),
"%s: the fields show before Add is clicked", targetType)
click(ctx, t, addTarget)
assert.Truef(t, shown(ctx, typeSelect),
"%s: Add does not show the type choice", targetType)
assert.Truef(t, hidden(ctx, targetName),
"%s: Add shows the fields before Next", targetType)
click(ctx, t, cancelChoice)
assert.Truef(t, hidden(ctx, typeSelect),
"%s: Cancel does not close the type choice", targetType)
chooseTargetType(ctx, t, targetType)
var submitted []string
require.NoError(t, chromedp.Run(
ctx, chromedp.Evaluate(submittedKeys, &submitted),
))
assert.Equalf(t, fields, submitted,
"%s: the form does not submit exactly the type's fields", targetType)
click(ctx, t, cancelFields)
assert.Truef(t, hidden(ctx, targetName),
"%s: Cancel does not close the fields", targetType)
assert.Truef(t, shown(ctx, addTarget),
"%s: Add does not come back after Cancel", targetType)
name := "added-" + targetType
chooseTargetType(ctx, t, targetType)
require.NoError(t, chromedp.Run(
ctx, chromedp.SetValue(targetName, name, chromedp.ByQuery),
))
for field, value := range values {
require.NoError(t, chromedp.Run(ctx, chromedp.SetValue(
`form[action$="/targets"] [name="`+field+`"]`, value,
chromedp.ByQuery,
)))
sections := []struct{ heading, form string }{
{"Entrypoints", `form[action$="/entrypoints"]`},
{"Targets", `form[action$="/targets"]`},
}
click(ctx, t, saveButton)
assert.Truef(t, shown(ctx, `//span[text()="`+name+
`"]/following-sibling::div/span[text()="`+targetType+`"]`),
"%s: the added target is not listed with its type", targetType)
for _, s := range sections {
assert.Truef(
t, hidden(ctx, s.form),
"%s: the add form shows before Add is clicked", s.heading,
)
click(ctx, t, `//h2[text()="`+s.heading+
`"]/following-sibling::button`)
assert.Truef(
t, shown(ctx, s.form),
"%s: the add form stays hidden when Add is clicked", s.heading,
)
}
}
// chooseTargetType clicks Add, picks targetType and clicks Next, and
// checks that the type's fields then show in place of the type choice.
func chooseTargetType(ctx context.Context, t *testing.T, targetType string) {
t.Helper()
click(ctx, t, addTarget)
require.NoError(t, chromedp.Run(
ctx, chromedp.SetValue(typeSelect, targetType, chromedp.BySearch),
))
click(ctx, t, nextButton)
assert.Truef(t, shown(ctx, targetName),
"%s: Next does not show the fields", targetType)
assert.Truef(t, hidden(ctx, typeSelect),
"%s: Next leaves the type choice showing", targetType)
assert.Truef(t, hidden(ctx, addTarget),
"%s: Add still shows while the form is open", targetType)
}
// checkRefusedTarget submits an http target the server refuses, a
// loopback destination, and checks that the page comes back with the
// form open on the http fields, the values entered and the reason, and
// that after Cancel the next Add starts with an empty form and no
// reason.
func checkRefusedTarget(ctx context.Context, t *testing.T, url string) {
// checkTargetType chooses Slack in the open add target form and checks
// what the form would then submit: one url field, the Slack one, and
// not the HTTP url, headers or timeout, which are hidden and disabled.
//
// It then opens the page at elsewhere and goes back. The browser loads
// the webhook page again and restores the form as it was left, Slack
// chosen, without a change event; the form must again show and submit
// Slack's fields, not the HTTP ones.
func checkTargetType(ctx context.Context, t *testing.T, elsewhere string) {
t.Helper()
const (
refusedURL = "http://127.0.0.1/hook"
urlField = `form[action$="/targets"] input[name="url"]`
reason = `//div[@class="alert-error"]`
chooseSlack = `(() => {
const type = document.querySelector('select[name="type"]');
type.value = "slack";
type.dispatchEvent(new Event("change"));
})()`
chosen = `document.querySelector('select[name="type"]').value`
howLoaded = `performance.getEntriesByType("navigation")[0].type`
submitted = `[...new FormData(
document.querySelector('form[action$="/targets"]')).keys()]`
slackURL = `input[placeholder^="https://hooks.slack.com/"]`
httpURL = `input[placeholder="https://example.com/webhook"]`
)
require.NoError(t, chromedp.Run(ctx, loadPage(url)))
slackFields := strings.Fields("csrf_token name type max_retries url")
chooseTargetType(ctx, t, "http")
require.NoError(t, chromedp.Run(
ctx,
chromedp.SetValue(targetName, "refused", chromedp.ByQuery),
chromedp.SetValue(urlField, refusedURL, chromedp.ByQuery),
))
click(ctx, t, saveButton)
assert.True(t, shown(ctx, reason),
"a refused target does not show the reason")
var name, typed string
var fields []string
require.NoError(t, chromedp.Run(
ctx,
chromedp.Value(targetName, &name, chromedp.ByQuery),
chromedp.Value(urlField, &typed, chromedp.ByQuery),
chromedp.Evaluate(chooseSlack, nil),
chromedp.Evaluate(submitted, &fields),
))
assert.Equal(t, "refused", name,
"a refused target does not keep the name entered")
assert.Equal(t, refusedURL, typed,
"a refused target does not keep the url entered")
assert.True(t, shown(ctx, targetName),
"a refused target does not come back with the form open")
assert.True(t, hidden(ctx, typeSelect),
"a refused target comes back on the type choice")
assert.Equal(
t, slackFields, fields,
"with Slack chosen, the HTTP fields must not be submitted",
)
click(ctx, t, cancelFields)
chooseTargetType(ctx, t, "http")
assert.True(t, hidden(ctx, reason),
"after Cancel, the next Add still shows the reason")
var loaded, restored string
// Going back waits for the load event, after which the browser has
// restored the form.
require.NoError(t, chromedp.Run(
ctx,
chromedp.Value(targetName, &name, chromedp.ByQuery),
chromedp.Value(urlField, &typed, chromedp.ByQuery),
loadPage(elsewhere),
chromedp.NavigateBack(),
chromedp.WaitNotPresent("[x-cloak]", chromedp.ByQuery),
chromedp.Evaluate(howLoaded, &loaded),
chromedp.Evaluate(chosen, &restored),
))
assert.Empty(t, name, "after Cancel, the next Add keeps the name entered")
assert.Empty(t, typed, "after Cancel, the next Add keeps the url entered")
// A page the browser kept in memory and showed again as it was
// would prove nothing here.
require.Equal(
t, "back_forward", loaded,
"going back, the browser did not load the page again",
)
require.Equal(
t, "slack", restored,
"going back, the browser did not restore the chosen type",
)
click(ctx, t, `//h2[text()="Targets"]/following-sibling::button`)
assert.True(t, shown(ctx, slackURL),
"going back with Slack chosen, the Slack fields are not shown")
assert.True(t, hidden(ctx, httpURL),
"going back with Slack chosen, the HTTP fields are shown")
require.NoError(t, chromedp.Run(
ctx, chromedp.Evaluate(submitted, &fields),
))
assert.Equal(
t, slackFields, fields,
"going back with Slack chosen, the HTTP fields must not be submitted",
)
}
// checkCopy loads a webhook page and checks that the Copy control beside
+20 -55
View File
@@ -87,65 +87,24 @@ document.addEventListener("alpine:init", function () {
};
});
// The targets section's add target form, in three steps: closed,
// choosing a type, then filling in that type's fields. targetType
// is empty until Next takes it from the type select.
// The add target form. Only the chosen type's fields show, and the
// others are disabled so that the form does not submit them.
//
// The reason and the fields' values come from the properties below
// rather than from the markup, because each type's fields are made
// afresh from the markup whenever that type is chosen. A refused
// submission comes back with its type, reason and values in the
// section's data attributes, and starts on that type's fields with
// them. Cancel empties these properties and resets the form, which
// holds whatever was typed, so the next Add starts with an empty
// form and no reason.
// The type is read from the type select when Alpine starts, when the
// select changes, and on pageshow. Going back to the page, the
// browser restores the type chosen before without a change event,
// in some browsers only after Alpine has started, but always before
// pageshow.
window.Alpine.data("targetForm", function () {
return {
choosing: false,
targetType: "",
reason: "",
name: "",
url: "",
headers: "",
timeout: "",
maxRetries: "",
expiry: "",
init() {
const refused = this.$root.dataset;
this.targetType = refused.type;
this.reason = refused.reason;
this.name = refused.name;
this.url = refused.destination;
this.headers = refused.headers;
this.timeout = refused.timeout;
this.maxRetries = refused.maxRetries;
this.expiry = refused.expiry;
this.readType();
},
add() {
this.choosing = true;
},
next() {
this.targetType = this.$refs.type.value;
this.choosing = false;
},
cancel() {
this.choosing = false;
this.targetType = "";
this.reason = "";
this.name = "";
this.url = "";
this.headers = "";
this.timeout = "";
this.maxRetries = "";
this.expiry = "";
this.$refs.form.reset();
},
get filling() {
return this.targetType !== "";
},
get closed() {
return !this.choosing && !this.filling;
readType() {
this.targetType = this.$root.querySelector(
'select[name="type"]'
).value;
},
get isHttp() {
return this.targetType === "http";
@@ -156,8 +115,14 @@ document.addEventListener("alpine:init", function () {
get isDatabase() {
return this.targetType === "database";
},
get isLog() {
return this.targetType === "log";
get notHttp() {
return !this.isHttp;
},
get notSlack() {
return !this.isSlack;
},
get notDatabase() {
return !this.isDatabase;
},
};
});
+5 -2
View File
@@ -5,7 +5,10 @@ import (
"embed"
)
// Static holds the embedded CSS and JavaScript files for the web UI.
// Static holds the CSS and JavaScript files the web UI's pages load. They
// are named one by one so that a missing js/alpine.min.js, which make
// assets extracts and git does not track, fails the build instead of
// leaving the pages without Alpine.js.
//
//go:embed css js
//go:embed css/tailwind.css css/style.css js/app.js js/alpine.min.js
var Static embed.FS
+60 -90
View File
@@ -90,23 +90,11 @@
</div>
</div>
<!-- Targets. The data attributes carry a refused add target
submission's type, reason and values back to the form. The
URL is data-destination, not data-url: html/template treats
an attribute named like a URL as a link and would rewrite
a refused ftp: or javascript: value. -->
<div class="card" x-data="targetForm"
data-type="{{.TargetForm.Type}}"
data-reason="{{.TargetError}}"
data-name="{{.TargetForm.Name}}"
data-destination="{{.TargetForm.URL}}"
data-headers="{{.TargetForm.Headers}}"
data-timeout="{{.TargetForm.Timeout}}"
data-max-retries="{{.TargetForm.MaxRetries}}"
data-expiry="{{.TargetForm.Expiry}}">
<!-- Targets -->
<div class="card" x-data="collapsible">
<div class="p-4 border-b border-gray-200 flex justify-between items-center">
<h2 class="text-lg font-medium text-gray-900">Targets</h2>
<button type="button" @click="add" x-show="closed" class="btn-small">
<button type="button" @click="toggle" class="btn-small">
<svg class="w-3 h-3 mr-1" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 4v16m8-8H4"/>
</svg>
@@ -114,83 +102,48 @@
</button>
</div>
<!-- Add target form. Add shows the type choice; Next replaces
it with the chosen type's fields. Each type's fields,
and the hidden type field submitted with them, exist
only while that type is chosen. A refused submission
comes back open on its type, with the values entered;
Cancel empties the form. -->
<form method="POST" action="/hook/{{.Webhook.ID}}/targets" x-ref="form">
<input type="hidden" name="csrf_token" value="{{.CSRFToken}}">
<div x-show="choosing" x-cloak class="p-4 bg-gray-50 border-b border-gray-200 flex flex-wrap gap-2">
<select x-ref="type" aria-label="Target type" class="input text-sm w-40">
<option value="http">HTTP</option>
<option value="slack">Slack</option>
<option value="database">Database</option>
<option value="log">Log</option>
</select>
<button type="button" @click="next" class="btn-primary text-sm">Next</button>
<button type="button" @click="cancel" class="btn-secondary text-sm">Cancel</button>
</div>
<div x-show="filling" x-cloak class="p-4 bg-gray-50 border-b border-gray-200 space-y-3">
<div x-show="reason" x-text="reason" class="alert-error"></div>
<input type="text" name="name" :value="name" placeholder="Target name" required class="input text-sm">
<template x-if="isHttp">
<div class="space-y-3">
<input type="hidden" name="type" value="http">
<input type="url" name="url" :value="url" placeholder="https://example.com/webhook" class="input text-sm">
<div>
<textarea name="headers" rows="3" :value="headers" placeholder="Authorization: Bearer ..." class="input text-sm"></textarea>
<p class="text-xs text-gray-500 mt-1">Optional request headers, one <code>Name: value</code> per line, sent with every delivery.</p>
</div>
<div class="flex gap-2 items-center">
<label class="text-sm text-gray-700">Timeout (seconds, blank = default):</label>
<input type="number" name="timeout" :value="timeout" min="0" max="300" class="input text-sm w-24">
</div>
<div>
<div class="flex gap-2 items-center">
<label class="text-sm text-gray-700">Max retries:</label>
<input type="number" name="max_retries" :value="maxRetries" placeholder="0" min="0" max="20" class="input text-sm w-24">
</div>
<p class="text-xs text-gray-500 mt-1">This is the total number of delivery attempts, not retries on top of the first: a value of 3 makes three attempts in all. 0 means a single attempt with no retries and no circuit breaker.</p>
</div>
</div>
</template>
<template x-if="isSlack">
<div class="space-y-3">
<input type="hidden" name="type" value="slack">
<div>
<input type="url" name="url" :value="url" placeholder="https://hooks.slack.com/services/..." class="input text-sm">
<p class="text-xs text-gray-500 mt-1">Slack or Mattermost incoming webhook URL. Payloads are pretty-printed in code blocks.</p>
</div>
<div>
<div class="flex gap-2 items-center">
<label class="text-sm text-gray-700">Max retries:</label>
<input type="number" name="max_retries" :value="maxRetries" placeholder="0" min="0" max="20" class="input text-sm w-24">
</div>
<p class="text-xs text-gray-500 mt-1">This is the total number of delivery attempts, not retries on top of the first: a value of 3 makes three attempts in all. 0 means a single attempt with no retries and no circuit breaker.</p>
</div>
</div>
</template>
<template x-if="isDatabase">
<div>
<input type="hidden" name="type" value="database">
<input type="text" name="expiry" :value="expiry" placeholder="never" class="input text-sm">
<p class="text-xs text-gray-500 mt-1">Archive expiry: "never" (default) keeps rows forever, or a duration like "720h" prunes older rows.</p>
</div>
</template>
<template x-if="isLog">
<div>
<input type="hidden" name="type" value="log">
<p class="text-xs text-gray-500">A log target writes each event to the application log. It has no settings beyond its name.</p>
</div>
</template>
<!-- Add target form -->
<div x-show="open" x-cloak class="p-4 bg-gray-50 border-b border-gray-200">
<form method="POST" action="/hook/{{.Webhook.ID}}/targets" x-data="targetForm" @pageshow.window="readType" class="space-y-3">
<input type="hidden" name="csrf_token" value="{{.CSRFToken}}">
<div class="flex gap-2">
<button type="submit" class="btn-primary text-sm">Save</button>
<button type="button" @click="cancel" class="btn-secondary text-sm">Cancel</button>
<input type="text" name="name" placeholder="Target name" required class="input text-sm flex-1">
<select name="type" @change="readType" class="input text-sm w-32">
<option value="http">HTTP</option>
<option value="slack">Slack</option>
<option value="database">Database</option>
<option value="log">Log</option>
</select>
</div>
</div>
</form>
<div x-show="isHttp">
<input type="url" name="url" placeholder="https://example.com/webhook" :disabled="notHttp" class="input text-sm">
</div>
<div x-show="isHttp">
<textarea name="headers" rows="3" placeholder="Authorization: Bearer ..." :disabled="notHttp" class="input text-sm"></textarea>
<p class="text-xs text-gray-500 mt-1">Optional request headers, one <code>Name: value</code> per line, sent with every delivery.</p>
</div>
<div x-show="isHttp" class="flex gap-2 items-center">
<label class="text-sm text-gray-700">Timeout (seconds, blank = default):</label>
<input type="number" name="timeout" min="0" max="300" :disabled="notHttp" class="input text-sm w-24">
</div>
<div x-show="isHttp">
<div class="flex gap-2 items-center">
<label class="text-sm text-gray-700">Max retries:</label>
<input type="number" name="max_retries" value="0" min="0" max="20" class="input text-sm w-24">
</div>
<p class="text-xs text-gray-500 mt-1">This is the total number of delivery attempts, not retries on top of the first: a value of 3 makes three attempts in all. 0 means a single attempt with no retries and no circuit breaker.</p>
</div>
<div x-show="isSlack">
<input type="url" name="url" placeholder="https://hooks.slack.com/services/..." :disabled="notSlack" class="input text-sm">
<p class="text-xs text-gray-500 mt-1">Slack or Mattermost incoming webhook URL. Payloads are pretty-printed in code blocks.</p>
</div>
<div x-show="isDatabase">
<input type="text" name="expiry" placeholder="never" :disabled="notDatabase" class="input text-sm">
<p class="text-xs text-gray-500 mt-1">Archive expiry: "never" (default) keeps rows forever, or a duration like "720h" prunes older rows.</p>
</div>
<button type="submit" class="btn-primary text-sm">Add Target</button>
</form>
</div>
<div class="divide-y divide-gray-100">
{{range .Targets}}
@@ -226,6 +179,23 @@
<span>{{.Value}}</span>
</div>
{{end}}
{{with .Archive}}
<div class="text-xs text-gray-500 mt-1">
<span class="font-medium text-gray-700">Archive File:</span>
<span class="break-all">{{.Name}}</span>
{{with .Note}}<span>({{.}})</span>{{end}}
</div>
{{if .Size}}
<div class="text-xs text-gray-500 mt-1">
<span class="font-medium text-gray-700">Archive Size:</span>
<span>{{.Size}}</span>
</div>
<div class="text-xs text-gray-500 mt-1">
<span class="font-medium text-gray-700">Last Written:</span>
<span title="{{.WrittenUTC}}">{{.Written}}</span>
</div>
{{end}}
{{end}}
</div>
{{else}}
<div class="p-4 text-sm text-gray-500">No targets configured.</div>