Author SHA1 Message Date
sneak 81bac1aa3e Load Alpine's CSP build so the UI's directives run (closes #371)
check / check (push) Waiting to run
The pages' Content-Security-Policy forbids eval, which the standard
Alpine.js build needs, so no directive ran: add forms showed open and
events never collapsed. 3p/ now holds the @alpinejs/csp 3.14.9 tarball
instead, and every directive in templates/ names a property or method
of a component registered in static/js/app.js, as that build requires.
The policy is unchanged.

A headless Chromium test in internal/server loads the webhook page and
the event log under the real headers. The Dockerfile's test stage
installs chromium; where it is missing the test skips.

Model: opus-5-5
2026-10-01 21:21:12 +00:00
34 changed files with 534 additions and 1694 deletions
Binary file not shown.
Binary file not shown.
+3 -2
View File
@@ -38,8 +38,9 @@ FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a349228
COPY --from=lint /src/go.sum /dev/null
# jq is a runtime dependency of script/ci-mark-superseded, which the test
# suite executes.
RUN apt-get update && apt-get install -y --no-install-recommends make curl ca-certificates jq && rm -rf /var/lib/apt/lists/*
# suite executes. chromium runs the browser test in internal/server, which
# skips where it is not installed.
RUN apt-get update && apt-get install -y --no-install-recommends make curl ca-certificates jq chromium && rm -rf /var/lib/apt/lists/*
WORKDIR /build
+39 -70
View File
@@ -1065,7 +1065,7 @@ unconditionally against whatever files it finds:
- the main database on connect — `Setting`, `User`, `APIKey`, `Webhook`,
`Entrypoint`, `Target`
- each event database when it is lazily opened — `Event`, `Delivery`,
`DeliveryResult`, `EventTotals`, `TargetTotals`
`DeliveryResult`
- each archive database on every open and reopen
There is no schema version table, no migration ledger, and no down
@@ -1255,9 +1255,20 @@ We provide:
## Third-party browser assets
The web UI serves one third-party script, Alpine.js. Its npm package tarball
is committed as `3p/alpinejs-3.14.9.tgz`, byte for byte as the npm registry
publishes it. It is a dependency, not this repo's build output, so
The web UI serves one third-party script, Alpine.js, in its CSP build: the npm
package `@alpinejs/csp`. The pages' Content-Security-Policy forbids eval, which
the standard `alpinejs` build needs to run the expressions written in the
markup. The CSP build runs no expressions, so every Alpine directive in
`templates/` only names a property or method of a component registered in
`static/js/app.js`: `x-data="collapsible"` and `@click="toggle"`, never
`x-data="{ open: false }"` or `@click="open = !open"`. A browser test in
`internal/server` loads the pages under the real policy and fails on any
directive that does not work. It needs `chromium` on `PATH` and skips without
it; the Dockerfile installs it, so the image build always runs it.
The package's tarball is committed as `3p/alpinejs-csp-3.14.9.tgz`, byte for
byte as the npm registry publishes it. It is a dependency, not this repo's
build output, so
`REPO_POLICIES.md`'s rule against committed build artifacts does not apply.
The directory is `3p/` rather than `vendor/` because Go treats a root
`vendor/` directory as its module vendor directory.
@@ -1270,10 +1281,11 @@ nothing downloads Alpine.js. The extracted file is not committed, and
`.dockerignore` keeps any host copy out of the build context.
To move to a new version: download
`https://registry.npmjs.org/alpinejs/-/alpinejs-<version>.tgz`, check it
`https://registry.npmjs.org/@alpinejs/csp/-/csp-<version>.tgz`, check it
against the `dist.integrity` hash listed at
`https://registry.npmjs.org/alpinejs/<version>`, replace the tarball in `3p/`
with it, update its file name in `script/assets`, and run `make check`.
`https://registry.npmjs.org/@alpinejs/csp/<version>`, replace the tarball in
`3p/` with it as `alpinejs-csp-<version>.tgz`, update its file name in
`script/assets`, and run `make check`.
## Rationale
@@ -1381,7 +1393,7 @@ The codebase uses consistent naming throughout (rename completed in
### Data Model
webhooker's data model has eleven entities organized into two tiers: the
webhooker's data model has nine entities organized into two tiers: the
**application tier** (user and webhook configuration) and the **event
tier** (event ingestion, delivery, and logging).
@@ -1410,13 +1422,6 @@ tier** (event ingestion, delivery, and logging).
│ ┌──────────┐ ┌──────────┐ ┌─────────────────┐ │
│ │ Event │──1:N──│ Delivery │──1:N──│ DeliveryResult │ │
│ └──────────┘ └──────────┘ └─────────────────┘ │
│ │
│ ┌──────────────┐ (one row: running counts of events) │
│ │ EventTotals │ │
│ └──────────────┘ │
│ ┌──────────────┐ (one row per target: running counts │
│ │ TargetTotals │ of its deliveries) │
│ └──────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
@@ -1669,7 +1674,6 @@ status across potentially multiple attempts.
| `event_id` | UUID | Foreign key → Event |
| `target_id`| UUID | Foreign key → Target |
| `status` | DeliveryStatus | One of: `pending`, `delivered`, `failed`, `retrying` |
| `finished_at` | timestamp | When the delivery became `delivered` or `failed` (nullable; empty while `pending` or `retrying`) |
**Relations:** Belongs to Event. Belongs to Target. Has many
DeliveryResults.
@@ -1737,65 +1741,33 @@ retries) is individually logged for full observability.
**Relations:** Belongs to Delivery.
#### EventTotals and TargetTotals
Running counts in each event database, read by the statistics pane at the
top of the webhook page. `EventTotals` is one row:
| Field | Type | Description |
| ---------------- | ------- | ----------- |
| `events` | integer | Events ever stored, resubmitted copies included |
| `events_removed` | integer | Events retention has deleted |
`TargetTotals` is one row per target, created by the first delivery to it:
| Field | Type | Description |
| -------------------- | ------- | ----------- |
| `target_id` | UUID | The target (primary key) |
| `deliveries` | integer | Deliveries to it ever created, replays included |
| `delivered` | integer | Of those, how many became `delivered` |
| `failed` | integer | Of those, how many became `failed` |
| `deliveries_removed` | integer | Its deliveries retention has deleted |
| `failed_removed` | integer | Its failed deliveries retention has deleted |
Each count changes in the transaction that writes or deletes the rows it
counts. The pane's lifetime events are `events`, and its lifetime
deliveries and failures are `deliveries` and `failed` summed over the
targets; each figure within retention is the same less what retention
removed, so neither needs the rows themselves. Its last-10-minutes and
last-24-hours figures are counted from the `events` and `deliveries`
indexes over just that window, the deliveries in one query grouped by
target. Its failure percentage for a window is the deliveries that became
`failed` in it out of all that became `delivered` or `failed` in it, and
a dash when none did.
#### Event-tier indexes
These indexes on the per-webhook event databases are declared in the model
tags, so `AutoMigrate` creates them on a fresh database:
tags, so `AutoMigrate` creates them on a fresh and on an existing database:
| Table | Columns | Serves |
| ------------------ | --------------------------- | ------ |
| `deliveries` | `status`, `deleted_at`, `finished_at`, `target_id` | Startup recovery, the retry and pending sweeps every 60 seconds and the queue-depth sampler every 30 seconds, which select deliveries by status, and the webhook page's statistics, which count each target's deliveries by status and when they finished |
| `deliveries` | `event_id`, `deleted_at` | The event log, which loads each event's deliveries, and retention, which counts and deletes the deliveries of expired events |
| `deliveries` | `status`, `deleted_at` | Startup recovery, the retry and pending sweeps every 60 seconds and the queue-depth sampler every 30 seconds, which select deliveries by status |
| `deliveries` | `event_id`, `deleted_at` | The event log, which loads each event's deliveries, and retention, which selects and deletes the deliveries of expired events |
| `delivery_results` | `delivery_id`, `deleted_at` | The event log, which loads the attempts of a page's deliveries, and retention, which deletes the attempts of expired events |
| `events` | `deleted_at`, `created_at` | The webhook page's statistics, which count recent events and find the newest |
| `events` | `created_at` | Retention, which selects expired events by age |
| `events` | `deleted_at`, `created_at` | Retention, which selects expired events by age |
| `events` | `created_at` | Retention's delete of the expired events themselves |
GORM's soft delete adds `deleted_at IS NULL` to these queries; retention
leaves it out. SQLite keeps no statistics on these tables, and without them it
rates the `deleted_at` index, which every live row matches, above an index on
a column matched against several values or compared with `<`. So every index
but the last also covers `deleted_at`. It comes second, so that retention can
use the index without it, except in `events`, where `created_at` is compared
with `<` and SQLite narrows by a `<` only on the last column it uses.
GORM's soft delete adds `deleted_at IS NULL` to these queries; retention's
deletes leave it out, but their lookups of expired rows keep it. SQLite keeps
no statistics on these tables, and without them it rates the `deleted_at`
index, which every live row matches, above an index on a column matched
against several values or compared with `<`. So every index but the last also
covers `deleted_at`. It comes second, so that retention's deletes can use the
index without it, except in `events`, where `created_at` is compared with `<`
and SQLite narrows by a `<` only on the last column it uses.
#### Common Fields
Every entity except `Setting`, `EventTotals` and `TargetTotals` includes
these fields from `BaseModel`. `Setting` is a bare key-value row with no
`id`, no timestamps and no soft delete, and the two totals tables hold
only counts, keyed by a numeric `id` and by `target_id`:
Every entity except `Setting` includes these fields from `BaseModel`.
`Setting` is a bare key-value row with no `id`, no timestamps and no
soft delete:
| Field | Type | Description |
| ------------ | --------- | ----------- |
@@ -1837,8 +1809,6 @@ encryption key is generated and stored, and an `admin` user is created.
- **Events** — captured incoming webhook payloads
- **Deliveries** — event-to-target pairings and their status
- **DeliveryResults** — individual delivery attempt logs
- **EventTotals** and **TargetTotals** — running counts of the above,
the deliveries per target, kept through retention
Per-webhook databases are created automatically when a webhook is
created (and lazily on first access for webhooks that predate this
@@ -2795,7 +2765,7 @@ imports. The entry point is `cmd/webhooker/main.go`.
```
webhooker/
├── 3p/
│ └── alpinejs-3.14.9.tgz # Alpine.js npm package, extracted by make assets
│ └── alpinejs-csp-3.14.9.tgz # Alpine.js CSP build npm package, extracted by make assets
├── cmd/webhooker/
│ └── main.go # Entry point: subcommand dispatch; no args locks DATA_DIR and wires fx
├── internal/
@@ -2819,7 +2789,6 @@ webhooker/
│ │ ├── model_event.go # Event entity (per-webhook DB)
│ │ ├── model_delivery.go # Delivery entity (per-webhook DB)
│ │ ├── model_delivery_result.go # DeliveryResult entity (per-webhook DB)
│ │ ├── model_totals.go # EventTotals and TargetTotals (per-webhook DB)
│ │ ├── model_apikey.go # APIKey entity
│ │ ├── password.go # Argon2id hashing and verification
│ │ ├── retention.go # Retention reaper (per-webhook event expiry)
@@ -2889,8 +2858,8 @@ webhooker/
│ ├── css/input.css # Tailwind input, source for tailwind.css (make css)
│ ├── css/tailwind.css # Generated stylesheet the pages load
│ ├── css/style.css # Older hand-written stylesheet, no longer loaded
│ ├── js/app.js # Progressive-enhancement copy-to-clipboard
│ └── js/alpine.min.js # Alpine.js, extracted from 3p/ by make assets, not committed
│ ├── js/app.js # Copy-to-clipboard, and the Alpine.js components
│ └── js/alpine.min.js # Alpine.js CSP build, extracted from 3p/ by make assets, not committed
├── templates/ # Go HTML templates (base, login, sources, etc.)
├── script/ # Scripts to Rule Them All entrypoints
├── Dockerfile # Three stages: lint, test+build, Alpine runtime
+8 -1
View File
@@ -4,6 +4,8 @@ go 1.26.1
require (
github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8
github.com/chromedp/cdproto v0.0.0-20260922220944-a19bff23514f
github.com/chromedp/chromedp v0.16.0
github.com/getsentry/sentry-go v0.25.0
github.com/go-chi/chi v1.5.5
github.com/go-chi/cors v1.2.1
@@ -28,8 +30,13 @@ require (
require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/chromedp/sysutil v1.1.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/go-json-experiment/json v0.0.0-20260820222146-c27c302e5fc3 // indirect
github.com/gobwas/httphead v0.1.0 // indirect
github.com/gobwas/pool v0.2.1 // indirect
github.com/gobwas/ws v1.4.0 // indirect
github.com/gorilla/securecookie v1.1.2 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect
@@ -50,7 +57,7 @@ require (
go.uber.org/zap v1.23.0 // indirect
golang.org/x/mod v0.17.0 // indirect
golang.org/x/sync v0.14.0 // indirect
golang.org/x/sys v0.37.0 // indirect
golang.org/x/sys v0.47.0 // indirect
golang.org/x/text v0.25.0 // indirect
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
google.golang.org/protobuf v1.31.0 // indirect
+20 -2
View File
@@ -6,6 +6,12 @@ github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/chromedp/cdproto v0.0.0-20260922220944-a19bff23514f h1:8PK9FM4bE0C8GMoWBW5lVsef3U7sPICjDg6JqngyYhk=
github.com/chromedp/cdproto v0.0.0-20260922220944-a19bff23514f/go.mod h1:3v4FIp5njIUyPDvqXsxEOxnB34lijG0up98/5kM1KaE=
github.com/chromedp/chromedp v0.16.0 h1:rOO4deOm4CbZgBCa8mD9g2rDyIoNs0BkgvNrlbp5ouk=
github.com/chromedp/chromedp v0.16.0/go.mod h1:rbuGKFT1vMcFcFqKfPIO1GpX/N+2s8onm2qMxZLbU5U=
github.com/chromedp/sysutil v1.1.0 h1:PUFNv5EcprjqXZD9nJb9b/c9ibAbxiYo4exNWZyipwM=
github.com/chromedp/sysutil v1.1.0/go.mod h1:WiThHUdltqCNKGc4gaU50XgYjwjYIhKWoHGPTUfWTJ8=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -23,6 +29,14 @@ github.com/go-chi/httprate v0.15.0 h1:j54xcWV9KGmPf/X4H32/aTH+wBlrvxL7P+SdnRqxh5
github.com/go-chi/httprate v0.15.0/go.mod h1:rzGHhVrsBn3IMLYDOZQsSU4fJNWcjui4fWKJcCId1R4=
github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=
github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og=
github.com/go-json-experiment/json v0.0.0-20260820222146-c27c302e5fc3 h1:UADEEmDKgfXbtnGJZ97beY5XLo9ZechG1nlU4KnRrkE=
github.com/go-json-experiment/json v0.0.0-20260820222146-c27c302e5fc3/go.mod h1:tphK2c80bpPhMOI4v6bIc2xWywPfbqi1Z06+RcrMkDg=
github.com/gobwas/httphead v0.1.0 h1:exrUm0f4YX0L7EBwZHuCF4GDp8aJfVeBrlLQrs6NqWU=
github.com/gobwas/httphead v0.1.0/go.mod h1:O/RXo79gxV8G+RqlR/otEwx4Q36zl9rqC5u12GKvMCM=
github.com/gobwas/pool v0.2.1 h1:xfeeEhW7pwmX8nuLVlqbzVc7udMDrwetjEv+TZIz1og=
github.com/gobwas/pool v0.2.1/go.mod h1:q8bcK0KcYlCgd9e7WYLm9LpyS+YeLd8JVDW6WezmKEw=
github.com/gobwas/ws v1.4.0 h1:CTaoG1tojrh4ucGPcoJFiAQUAsEWekEWvLy7GsVNqGs=
github.com/gobwas/ws v1.4.0/go.mod h1:G3gNqMNtPppf5XUz7O4shetPpcZ1VJ7zt18dlUeakrc=
github.com/gofrs/flock v0.13.0 h1:95JolYOvGMqeH31+FC7D2+uULf6mG61mEZ/A8dRYMzw=
github.com/gofrs/flock v0.13.0/go.mod h1:jxeyy9R1auM5S6JYDBhDt+E2TCo7DkratH4Pgi8P+Z0=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
@@ -55,12 +69,16 @@ github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80 h1:6Yzfa6GP0rIo/kULo2bwGEkFvCePZ3qHDDTC3/J9Swo=
github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80/go.mod h1:imJHygn/1yfhB7XSJJKlFZKl/J+dCPAknuiaGOshXAs=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-sqlite3 v1.14.17 h1:mCRHCLDUBXgpKAqIKsaAaAsrAlbkeomtRFKXh2L6YIM=
github.com/mattn/go-sqlite3 v1.14.17/go.mod h1:2eHXhiwb8IkHr+BDWZGa96P6+rkvnG63S2DGjv9HUNg=
github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 h1:jWpvCLoY8Z/e3VKvlsiIGKtc+UG6U5vzxaoagmhXfyg=
github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0/go.mod h1:QUyp042oQthUoa9bqDv0ER0wrtXnBruoNd7aNjkbP+k=
github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde h1:x0TT0RDC7UhAVbbWWBzr41ElhJx5tXPWkIHA2HWPRuw=
github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde/go.mod h1:nZgzbfBr3hhjoZnS66nKrHmduYNpc34ny7RK4z5/HM0=
github.com/pingcap/errors v0.11.4 h1:lFuQV/oaUMGcD2tqt+01ROSmJs75VG1ToEOkZIZ4nE4=
github.com/pingcap/errors v0.11.4/go.mod h1:Oi8TUi2kEtXXLMJk9l1cGmz20kV3TaQ0usTwv5KuLY8=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
@@ -111,8 +129,8 @@ golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/sync v0.14.0 h1:woo0S4Yywslg6hp4eUFjTVOyKt0RookbpAHG4c1HmhQ=
golang.org/x/sync v0.14.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ=
golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/text v0.25.0 h1:qVyWApTSYLk/drJRO5mDlNYskwQznZmkpV2c8q9zls4=
golang.org/x/text v0.25.0/go.mod h1:WEdwpYrmk1qmdHvhkSTNPm3app7v4rsT8F2UD6+VHIA=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg=
+21 -94
View File
@@ -93,11 +93,11 @@ func TestEventTierQueriesUseTheirIndexes(t *testing.T) {
deliveries []database.Delivery
results []database.DeliveryResult
depths []struct{ Depth int }
removed []database.TargetTotals
)
byStatus := "idx_deliveries_status (status=? AND deleted_at=?)"
byEvent := "idx_deliveries_event_id (event_id=? AND deleted_at=?)"
byAge := "idx_events_deleted_at_created_at (deleted_at=? AND created_at<?)"
// The delivery engine: recovery and the retry sweep, the sweep for
// stranded pending deliveries, and the queue depth count.
@@ -123,89 +123,25 @@ func TestEventTierQueriesUseTheirIndexes(t *testing.T) {
Order("attempt_num ASC").Find(&results),
"idx_delivery_results_delivery_id (delivery_id=? AND deleted_at=?)")
// Retention (reapExpired, deleteEvents): one batch of expired
// events, then their attempts, deliveries and the events.
var expired []string
// Retention's three deletes (reapExpired), whose subqueries are built
// afresh for each statement as it builds them.
expiredEventIDs := func() *gorm.DB {
return dry.Model(&database.Event{}).Select("id").
Where("created_at < ?", cutoff)
}
assertPlanUses(t, db, dry.Unscoped().Model(&database.Event{}).
Where("created_at < ?", cutoff).
Limit(database.ExportReapBatchSize).Pluck("id", &expired),
"idx_events_created_at (created_at<?)")
assertPlanUses(t, db, dry.Unscoped().Where(
"delivery_id IN (?)", dry.Unscoped().Model(&database.Delivery{}).
Select("id").Where("event_id IN ?", ids),
"delivery_id IN (?)", dry.Model(&database.Delivery{}).
Select("id").Where("event_id IN (?)", expiredEventIDs()),
).Delete(&database.DeliveryResult{}),
"idx_delivery_results_delivery_id (delivery_id=?)",
"idx_deliveries_event_id (event_id=?)")
assertPlanUses(t, db, dry.Unscoped().Model(&database.Delivery{}).
Select("target_id, count(*) AS deliveries_removed, "+
"count(CASE WHEN status = ? THEN 1 END) AS failed_removed",
database.DeliveryStatusFailed).
Where("event_id IN ?", ids).Group("target_id").Find(&removed),
"idx_deliveries_event_id (event_id=?)")
assertPlanUses(t, db, dry.Unscoped().Where("event_id IN ?", ids).
Delete(&database.Delivery{}), "idx_deliveries_event_id (event_id=?)")
assertPlanUses(t, db, dry.Unscoped().Where("id IN ?", ids).
Delete(&database.Event{}), "sqlite_autoindex_events_1 (id=?)")
}
// TestStatisticsQueriesUseTheirIndexes does the same for the webhook
// page's statistics (readEventStats in the handlers): deliveries in
// progress, each target's deliveries finished since a time, which must
// come from the index alone, events received since a time, and the
// newest event, which must come straight off an index rather than from
// sorting every event.
func TestStatisticsQueriesUseTheirIndexes(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
defer func() { require.NoError(t, lc.Stop(ctx)) }()
db, err := mgr.GetDB(uuid.New().String())
require.NoError(t, err)
dry := db.Session(&gorm.Session{DryRun: true})
since := time.Now()
var (
count int64
newest []time.Time
byTarget []struct{ TargetID string }
)
assertPlanUses(t, db, dry.Model(&database.Delivery{}).
Where("status IN ?", []database.DeliveryStatus{
database.DeliveryStatusPending,
database.DeliveryStatusRetrying,
}).Count(&count),
"idx_deliveries_status (status=? AND deleted_at=?)")
assertPlanUses(t, db, dry.Model(&database.Delivery{}).
Select("target_id, "+
"count(CASE WHEN status = ? THEN 1 END) AS delivered, "+
"count(CASE WHEN status = ? THEN 1 END) AS failed",
database.DeliveryStatusDelivered,
database.DeliveryStatusFailed).
Where("status IN ? AND finished_at >= ?",
[]database.DeliveryStatus{
database.DeliveryStatusDelivered,
database.DeliveryStatusFailed,
}, since).
Group("target_id").Find(&byTarget),
"COVERING INDEX idx_deliveries_status "+
"(status=? AND deleted_at=? AND finished_at>?)")
assertPlanUses(t, db, dry.Model(&database.Event{}).
Where("created_at >= ?", since).Count(&count),
"idx_events_deleted_at_created_at "+
"(deleted_at=? AND created_at>?)")
newestEvent := dry.Model(&database.Event{}).
Order("created_at DESC").Limit(1).Pluck("created_at", &newest)
assertPlanUses(t, db, newestEvent,
"idx_events_deleted_at_created_at (deleted_at=?)")
assert.NotContains(t, queryPlan(t, db, newestEvent), "TEMP B-TREE")
"idx_delivery_results_delivery_id (delivery_id=?)", byEvent, byAge)
assertPlanUses(t, db, dry.Unscoped().Where(
"event_id IN (?)", expiredEventIDs(),
).Delete(&database.Delivery{}),
"idx_deliveries_event_id (event_id=?)", byAge)
assertPlanUses(t, db, dry.Unscoped().Where(
"created_at < ?", cutoff,
).Delete(&database.Event{}), "idx_events_created_at (created_at<?)")
}
// assertPlanUses asserts that SQLite's plan for a statement GORM built
@@ -216,18 +152,6 @@ func assertPlanUses(
) {
t.Helper()
plan := queryPlan(t, db, built)
for _, index := range indexes {
assert.Contains(t, plan, index, built.Statement.SQL.String())
}
}
// queryPlan returns SQLite's plan for a statement GORM built in a dry
// run, run with the same SQL and arguments GORM would send.
func queryPlan(t *testing.T, db, built *gorm.DB) string {
t.Helper()
var plan []struct{ Detail string }
require.NoError(t, db.Raw(
@@ -235,5 +159,8 @@ func queryPlan(t *testing.T, db, built *gorm.DB) string {
built.Statement.Vars...,
).Scan(&plan).Error)
return fmt.Sprint(plan)
for _, index := range indexes {
assert.Contains(t, fmt.Sprint(plan), index,
built.Statement.SQL.String())
}
}
-4
View File
@@ -28,10 +28,6 @@ func NewTestRetentionReaper(
}
}
// ExportReapBatchSize exposes how many expired events one retention
// transaction deletes.
const ExportReapBatchSize = reapBatchSize
// ExportSweep runs a single retention sweep synchronously for tests.
func (r *RetentionReaper) ExportSweep(ctx context.Context) {
r.sweep(ctx)
+2 -13
View File
@@ -1,10 +1,6 @@
package database
import (
"time"
"gorm.io/gorm"
)
import "gorm.io/gorm"
// DeliveryStatus represents the status of a delivery
type DeliveryStatus string
@@ -41,7 +37,7 @@ type Delivery struct {
BaseModel
EventID string `gorm:"type:uuid;not null;index:idx_deliveries_event_id,priority:1" json:"eventId"`
TargetID string `gorm:"type:uuid;not null;index:idx_deliveries_status,priority:4" json:"targetId"`
TargetID string `gorm:"type:uuid;not null" json:"targetId"`
Status DeliveryStatus `gorm:"not null;default:'pending';index:idx_deliveries_status,priority:1" json:"status"`
// DeletedAt repeats the BaseModel field only to be the second column
@@ -49,13 +45,6 @@ type Delivery struct {
// gives.
DeletedAt gorm.DeletedAt `gorm:"index:idx_deliveries_event_id,priority:2;index:idx_deliveries_status,priority:2" json:"deletedAt,omitzero"`
// FinishedAt is when the delivery became delivered or failed, and
// nil while it is pending or retrying. It and then TargetID end the
// status index, so the webhook page counts each target's deliveries
// that finished in a recent window by reading just that window from
// the index.
FinishedAt *time.Time `gorm:"index:idx_deliveries_status,priority:3" json:"finishedAt,omitempty"`
// Relations
Event Event `json:"event,omitzero"`
Target Target `json:"target,omitzero"`
-91
View File
@@ -1,91 +0,0 @@
package database
import (
"fmt"
"gorm.io/gorm"
)
// The running totals in a webhook's event database keep the webhook
// page's lifetime figures right after retention has removed the rows
// they count, and let the page show them without counting every row.
// Each total changes in the transaction that writes or deletes the
// rows it counts.
// EventTotals is the single row counting a webhook's events: every
// event ever stored, and how many of them retention has deleted.
type EventTotals struct {
ID int64 `gorm:"primaryKey"`
Events int64 `gorm:"not null"`
EventsRemoved int64 `gorm:"not null"`
}
// TableName names the table AddEventTotals updates.
func (EventTotals) TableName() string {
return "event_totals"
}
// TargetTotals is one row per target counting its deliveries: every
// delivery ever created, how many became delivered and how many
// failed, and how many deliveries and failed deliveries retention has
// deleted. The webhook's delivery figures are these rows summed.
type TargetTotals struct {
TargetID string `gorm:"type:uuid;primaryKey"`
Deliveries int64 `gorm:"not null"`
Delivered int64 `gorm:"not null"`
Failed int64 `gorm:"not null"`
DeliveriesRemoved int64 `gorm:"not null"`
FailedRemoved int64 `gorm:"not null"`
}
// TableName names the table AddTargetTotals updates.
func (TargetTotals) TableName() string {
return "target_totals"
}
// AddEventTotals adds each count in add to the webhook's event totals.
// Call it on the transaction that writes or deletes the events it
// counts.
func AddEventTotals(tx *gorm.DB, add EventTotals) error {
err := tx.Exec(
`UPDATE event_totals SET
events = events + ?,
events_removed = events_removed + ?`,
add.Events, add.EventsRemoved,
).Error
if err != nil {
return fmt.Errorf("adding to event totals: %w", err)
}
return nil
}
// AddTargetTotals adds each count in add to the totals of the target
// add.TargetID names, creating its row the first time. Call it on the
// transaction that writes or deletes the deliveries it counts.
func AddTargetTotals(tx *gorm.DB, add TargetTotals) error {
err := tx.Exec(
`INSERT INTO target_totals (target_id, deliveries, delivered,
failed, deliveries_removed, failed_removed)
VALUES (?, ?, ?, ?, ?, ?)
ON CONFLICT (target_id) DO UPDATE SET
deliveries = deliveries + excluded.deliveries,
delivered = delivered + excluded.delivered,
failed = failed + excluded.failed,
deliveries_removed =
deliveries_removed + excluded.deliveries_removed,
failed_removed = failed_removed + excluded.failed_removed`,
add.TargetID, add.Deliveries, add.Delivered,
add.Failed, add.DeliveriesRemoved, add.FailedRemoved,
).Error
if err != nil {
return fmt.Errorf(
"adding to totals of target %s: %w", add.TargetID, err,
)
}
return nil
}
+1 -2
View File
@@ -2,8 +2,7 @@ package database
// Migrate runs database migrations for the main application database.
// Only configuration-tier models are stored in the main database.
// Event-tier models (Event, Delivery, DeliveryResult, EventTotals,
// TargetTotals) live in
// Event-tier models (Event, Delivery, DeliveryResult) live in
// per-webhook dedicated databases managed by WebhookDBManager.
func (d *Database) Migrate() error {
return d.db.AutoMigrate(
+41 -88
View File
@@ -18,13 +18,6 @@ import (
// computation.
const hoursPerDay = 24
// reapBatchSize is how many expired events one retention transaction
// deletes. A transaction holds the event database's write lock, which
// the receiver and the delivery workers wait for, so a large prune is
// split into transactions each short enough to finish well inside the
// busy timeout.
const reapBatchSize = 1000
// RetentionReaperParams holds the fx dependencies for the
// RetentionReaper.
type RetentionReaperParams struct {
@@ -272,97 +265,57 @@ func retentionCutoff(
), true
}
// reapExpired hard-deletes the events older than cutoff, with their
// deliveries and delivery results, reapBatchSize events per
// transaction until none is left. It returns the number of events
// reapExpired hard-deletes, in foreign-key-safe order, the delivery
// results, deliveries, and events associated with events older than
// cutoff. Deletes are unscoped so rows are physically removed rather
// than soft-deleted, reclaiming disk. It returns the number of events
// deleted.
func reapExpired(db *gorm.DB, cutoff time.Time) (int64, error) {
var total int64
for {
var eventIDs []string
err := db.Transaction(func(tx *gorm.DB) error {
err := tx.Unscoped().Model(&Event{}).
Where("created_at < ?", cutoff).
Limit(reapBatchSize).
Pluck("id", &eventIDs).Error
if err != nil {
return fmt.Errorf("selecting expired events: %w", err)
}
if len(eventIDs) == 0 {
return nil
}
return deleteEvents(tx, eventIDs)
})
if err != nil {
return total, err
}
total += int64(len(eventIDs))
if len(eventIDs) < reapBatchSize {
return total, nil
}
}
}
// deleteEvents hard-deletes the given events and, in foreign-key-safe
// order before them, their delivery results and deliveries, then adds
// what it deleted to the running totals. It runs on reapExpired's
// transaction, so the totals change exactly when the rows do. Deletes
// are unscoped so rows are physically removed rather than
// soft-deleted, reclaiming disk.
func deleteEvents(tx *gorm.DB, eventIDs []string) error {
// 1. The delivery results of the events' deliveries.
err := tx.Unscoped().
Where("delivery_id IN (?)", tx.Unscoped().Model(&Delivery{}).
// Fresh subqueries are built per statement to avoid reusing a
// mutated builder across executions.
expiredEventIDs := func() *gorm.DB {
return db.Model(&Event{}).
Select("id").
Where("event_id IN ?", eventIDs)).
Delete(&DeliveryResult{}).Error
if err != nil {
return fmt.Errorf("deleting expired delivery results: %w", err)
Where("created_at < ?", cutoff)
}
expiredDeliveryIDs := func() *gorm.DB {
return db.Model(&Delivery{}).
Select("id").
Where("event_id IN (?)", expiredEventIDs())
}
// 2. The events' deliveries, after counting them, and the failed
// ones among them, per target. The status is tested in the select
// list rather than the WHERE clause: there, SQLite would read every
// failed delivery the webhook has through the status index,
// instead of only these through the event_id index.
var removed []TargetTotals
err = tx.Unscoped().Model(&Delivery{}).
Select("target_id, count(*) AS deliveries_removed, "+
"count(CASE WHEN status = ? THEN 1 END) AS failed_removed",
DeliveryStatusFailed).
Where("event_id IN ?", eventIDs).
Group("target_id").
Find(&removed).Error
if err != nil {
return fmt.Errorf("counting expired deliveries: %w", err)
// 1. Delivery results whose delivery belongs to an expired event.
res := db.Unscoped().
Where("delivery_id IN (?)", expiredDeliveryIDs()).
Delete(&DeliveryResult{})
if res.Error != nil {
return 0, fmt.Errorf(
"deleting expired delivery results: %w",
res.Error,
)
}
err = tx.Unscoped().
Where("event_id IN ?", eventIDs).
Delete(&Delivery{}).Error
if err != nil {
return fmt.Errorf("deleting expired deliveries: %w", err)
// 2. Deliveries belonging to an expired event.
del := db.Unscoped().
Where("event_id IN (?)", expiredEventIDs()).
Delete(&Delivery{})
if del.Error != nil {
return 0, fmt.Errorf(
"deleting expired deliveries: %w",
del.Error,
)
}
// 3. The events themselves.
ev := tx.Unscoped().Where("id IN ?", eventIDs).Delete(&Event{})
// 3. The expired events themselves.
ev := db.Unscoped().
Where("created_at < ?", cutoff).
Delete(&Event{})
if ev.Error != nil {
return fmt.Errorf("deleting expired events: %w", ev.Error)
return 0, fmt.Errorf(
"deleting expired events: %w",
ev.Error,
)
}
for i := range removed {
err = AddTargetTotals(tx, removed[i])
if err != nil {
return err
}
}
return AddEventTotals(tx, EventTotals{EventsRemoved: ev.RowsAffected})
return ev.RowsAffected, nil
}
-228
View File
@@ -1,228 +0,0 @@
package database_test
import (
"context"
"net/http"
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// readEventTotals reads a webhook database's row of event totals,
// asserting that it has exactly one.
func readEventTotals(t *testing.T, db *gorm.DB) database.EventTotals {
t.Helper()
var rows []database.EventTotals
require.NoError(t, db.Find(&rows).Error)
require.Len(t, rows, 1)
return rows[0]
}
// readTargetTotals reads a webhook database's target totals, keyed by
// target.
func readTargetTotals(
t *testing.T, db *gorm.DB,
) map[string]database.TargetTotals {
t.Helper()
var rows []database.TargetTotals
require.NoError(t, db.Find(&rows).Error)
byTarget := make(map[string]database.TargetTotals, len(rows))
for _, row := range rows {
byTarget[row.TargetID] = row
}
return byTarget
}
// TestWebhookDBManager_TotalsSurviveReopen verifies that a new event
// database starts with one row of zero event totals and no target
// totals, that adding to a target twice adds to the one row, and that
// opening the database again keeps everything added.
func TestWebhookDBManager_TotalsSurviveReopen(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
defer func() { require.NoError(t, lc.Stop(ctx)) }()
webhookID := uuid.New().String()
db, err := mgr.GetDB(webhookID)
require.NoError(t, err)
fresh := readEventTotals(t, db)
assert.Equal(t, database.EventTotals{ID: fresh.ID}, fresh)
assert.Empty(t, readTargetTotals(t, db))
first, second := uuid.New().String(), uuid.New().String()
require.NoError(t, database.AddEventTotals(db, database.EventTotals{
Events: 2,
}))
require.NoError(t, database.AddTargetTotals(db, database.TargetTotals{
TargetID: first, Deliveries: 2, Delivered: 1,
}))
require.NoError(t, database.AddTargetTotals(db, database.TargetTotals{
TargetID: first, Failed: 1,
}))
require.NoError(t, database.AddTargetTotals(db, database.TargetTotals{
TargetID: second, Deliveries: 1,
}))
// Drop the cached connection so the next open reopens the file,
// as a restart would.
require.NoError(t, mgr.CloseAll())
db, err = mgr.GetDB(webhookID)
require.NoError(t, err)
assert.Equal(t, database.EventTotals{ID: fresh.ID, Events: 2},
readEventTotals(t, db))
assert.Equal(t, map[string]database.TargetTotals{
first: {
TargetID: first, Deliveries: 2, Delivered: 1, Failed: 1,
},
second: {TargetID: second, Deliveries: 1},
}, readTargetTotals(t, db))
}
// seedExpiredEvents stores count events created at the given time,
// each with a delivered delivery to one target and a failed delivery
// to the other, and one attempt for each delivery.
func seedExpiredEvents(
t *testing.T,
db *gorm.DB,
webhookID string,
count int,
createdAt time.Time,
delivered, failed string,
) {
t.Helper()
events := make([]database.Event, count)
deliveries := make([]database.Delivery, 0, 2*count)
for i := range events {
events[i] = database.Event{
WebhookID: webhookID,
EntrypointID: uuid.New().String(),
Method: http.MethodPost,
}
events[i].ID = uuid.New().String()
events[i].CreatedAt = createdAt
deliveries = append(deliveries,
database.Delivery{
EventID: events[i].ID,
TargetID: delivered,
Status: database.DeliveryStatusDelivered,
},
database.Delivery{
EventID: events[i].ID,
TargetID: failed,
Status: database.DeliveryStatusFailed,
},
)
}
require.NoError(t, db.CreateInBatches(events, 500).Error)
require.NoError(t, db.CreateInBatches(deliveries, 500).Error)
results := make([]database.DeliveryResult, len(deliveries))
for i := range deliveries {
results[i] = database.DeliveryResult{
DeliveryID: deliveries[i].ID, AttemptNum: 1,
}
}
require.NoError(t, db.CreateInBatches(results, 500).Error)
}
// TestRetentionReaper_PrunesMoreThanOneBatch verifies that a prune
// larger than one transaction's batch removes every expired event with
// its deliveries and delivery results, keeps the recent event, and
// adds what it removed to the event and target totals, so the totals
// within retention match the rows still stored.
func TestRetentionReaper_PrunesMoreThanOneBatch(t *testing.T) {
t.Parallel()
env := setupRetentionTest(t)
webhookID := createWebhook(t, env.mainDB.DB(), 30)
db, err := env.mgr.GetDB(webhookID)
require.NoError(t, err)
expired := database.ExportReapBatchSize + 1
delivered, failed := uuid.New().String(), uuid.New().String()
seedExpiredEvents(t, db, webhookID, expired,
time.Now().Add(-40*24*time.Hour), delivered, failed)
// One recent event, delivered to the first target.
recent := seedEventChain(t, db, webhookID, time.Now())
require.NoError(t, db.Model(&database.Delivery{}).
Where("id = ?", recent.deliveryID).
Update("target_id", delivered).Error)
// The totals storing those rows would have left.
n := int64(expired)
require.NoError(t, database.AddEventTotals(db, database.EventTotals{
Events: n + 1,
}))
require.NoError(t, database.AddTargetTotals(db, database.TargetTotals{
TargetID: delivered, Deliveries: n + 1, Delivered: n + 1,
}))
require.NoError(t, database.AddTargetTotals(db, database.TargetTotals{
TargetID: failed, Deliveries: n, Failed: n,
}))
env.reaper.ExportSweep(context.Background())
// Only the recent event's rows are left.
for _, model := range []any{
&database.Event{}, &database.Delivery{}, &database.DeliveryResult{},
} {
var count int64
require.NoError(t, db.Model(model).Count(&count).Error)
assert.Equal(t, int64(1), count, "%T rows left", model)
}
assertChainPresent(t, db, recent)
eventTotals := readEventTotals(t, db)
assert.Equal(t, database.EventTotals{
ID: eventTotals.ID, Events: n + 1, EventsRemoved: n,
}, eventTotals)
targetTotals := readTargetTotals(t, db)
assert.Equal(t, map[string]database.TargetTotals{
delivered: {
TargetID: delivered, Deliveries: n + 1, Delivered: n + 1,
DeliveriesRemoved: n,
},
failed: {
TargetID: failed, Deliveries: n, Failed: n,
DeliveriesRemoved: n, FailedRemoved: n,
},
}, targetTotals)
// A sweep with nothing left to remove changes nothing.
env.reaper.ExportSweep(context.Background())
assert.Equal(t, eventTotals, readEventTotals(t, db))
assert.Equal(t, targetTotals, readTargetTotals(t, db))
}
+1 -15
View File
@@ -35,8 +35,7 @@ var errInvalidCachedDBType = errors.New(
// WebhookDBManager manages per-webhook SQLite database files
// for event storage. Each webhook gets its own dedicated
// database containing Events, Deliveries, DeliveryResults and the
// running totals of them (EventTotals, TargetTotals).
// database containing Events, Deliveries, and DeliveryResults.
// Database connections are opened lazily and cached.
type WebhookDBManager struct {
dataDir string
@@ -296,7 +295,6 @@ func (m *WebhookDBManager) openDB(
// Run migrations for event-tier models only
err = db.AutoMigrate(
&Event{}, &Delivery{}, &DeliveryResult{},
&EventTotals{}, &TargetTotals{},
)
if err != nil {
_ = sqlDB.Close()
@@ -307,18 +305,6 @@ func (m *WebhookDBManager) openDB(
)
}
// A new database gets its row of event totals, all zero. Target
// totals rows are created by the first delivery to each target.
err = db.FirstOrCreate(&EventTotals{}).Error
if err != nil {
_ = sqlDB.Close()
return nil, fmt.Errorf(
"creating event totals for webhook database %s: %w",
webhookID, err,
)
}
m.log.Info(
"opened per-webhook database",
"webhook_id", webhookID,
-116
View File
@@ -1,116 +0,0 @@
package delivery_test
import (
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// targetTotals reads one target's totals from a webhook database, all
// zero when it has no row.
func targetTotals(
t *testing.T, db *gorm.DB, targetID string,
) database.TargetTotals {
t.Helper()
var rows []database.TargetTotals
require.NoError(t, db.Where("target_id = ?", targetID).
Find(&rows).Error)
if len(rows) == 0 {
return database.TargetTotals{TargetID: targetID}
}
return rows[0]
}
// TestUpdateDeliveryStatus_FinishTimeAndTargetTotals pins what a status
// write records for the webhook page's statistics: the time a delivery
// finished, set only when it becomes delivered or failed, and one more
// on its target's delivered or failed total.
func TestUpdateDeliveryStatus_FinishTimeAndTargetTotals(t *testing.T) {
t.Parallel()
tests := []struct {
status database.DeliveryStatus
finished bool
delivered int64
failed int64
}{
{database.DeliveryStatusRetrying, false, 0, 0},
{database.DeliveryStatusDelivered, true, 1, 0},
{database.DeliveryStatusFailed, true, 0, 1},
}
for _, tt := range tests {
t.Run(string(tt.status), func(t *testing.T) {
t.Parallel()
db := testWebhookDB(t)
e := testEngine(t, 1)
event := seedEvent(t, db, `{}`)
targetID := uuid.New().String()
d := seedDelivery(
t, db, event.ID, targetID,
database.DeliveryStatusPending,
)
before := time.Now()
require.NoError(t, e.ExportUpdateDeliveryStatus(
db, &d, tt.status,
))
var stored database.Delivery
require.NoError(t, db.First(&stored, "id = ?", d.ID).Error)
assert.Equal(t, tt.status, stored.Status)
if tt.finished {
require.NotNil(t, stored.FinishedAt)
assert.False(t, stored.FinishedAt.Before(before))
} else {
assert.Nil(t, stored.FinishedAt)
}
assert.Equal(t, database.TargetTotals{
TargetID: targetID,
Delivered: tt.delivered,
Failed: tt.failed,
}, targetTotals(t, db, targetID))
})
}
}
// TestUpdateDeliveryStatus_DeletedDeliveryIsNotCounted covers a
// delivery retention deleted while the engine still held it. Failing
// it afterwards writes no row, so it adds no failure either: retention
// has already counted what it removed.
func TestUpdateDeliveryStatus_DeletedDeliveryIsNotCounted(t *testing.T) {
t.Parallel()
db := testWebhookDB(t)
e := testEngine(t, 1)
event := seedEvent(t, db, `{}`)
targetID := uuid.New().String()
d := seedDelivery(
t, db, event.ID, targetID,
database.DeliveryStatusRetrying,
)
require.NoError(t, db.Unscoped().
Delete(&database.Delivery{}, "id = ?", d.ID).Error)
require.NoError(t, e.ExportUpdateDeliveryStatus(
db, &d, database.DeliveryStatusFailed,
))
assert.Equal(t, database.TargetTotals{TargetID: targetID},
targetTotals(t, db, targetID))
}
+2 -33
View File
@@ -1554,9 +1554,8 @@ func (e *Engine) updateDeliveryStatus(
targetType database.TargetType,
status database.DeliveryStatus,
) error {
err := webhookDB.Transaction(func(tx *gorm.DB) error {
return writeDeliveryStatus(tx, d, status)
})
err := webhookDB.Model(d).
Update("status", status).Error
if err != nil {
return fmt.Errorf(
"updating delivery %s to status %s: %w",
@@ -1575,36 +1574,6 @@ func (e *Engine) updateDeliveryStatus(
return nil
}
// writeDeliveryStatus writes a delivery's new status. A delivery that
// becomes delivered or failed also gets the time it finished, and is
// added to its target's delivered or failed total. It is counted only
// if the row was still there to update: retention may have deleted it
// while the engine was working on it.
func writeDeliveryStatus(
tx *gorm.DB,
d *database.Delivery,
status database.DeliveryStatus,
) error {
if !status.Terminal() {
return tx.Model(d).Update("status", status).Error
}
res := tx.Model(d).Updates(map[string]any{
"status": status,
"finished_at": time.Now(),
})
if res.Error != nil || res.RowsAffected == 0 {
return res.Error
}
add := database.TargetTotals{TargetID: d.TargetID, Delivered: 1}
if status == database.DeliveryStatusFailed {
add = database.TargetTotals{TargetID: d.TargetID, Failed: 1}
}
return database.AddTargetTotals(tx, add)
}
// settleStatus moves a delivery to its outcome status and reports a
// failed write through bookkeepingFailed, which leaves the row
// recoverable. It exists so the target call sites read as one
-3
View File
@@ -57,10 +57,7 @@ func testWebhookDB(t *testing.T) *gorm.DB {
&database.Event{},
&database.Delivery{},
&database.DeliveryResult{},
&database.EventTotals{},
&database.TargetTotals{},
))
require.NoError(t, db.Create(&database.EventTotals{}).Error)
return db
}
-10
View File
@@ -150,16 +150,6 @@ func (e *Engine) ExportDeliverSlack(
)
}
// ExportUpdateDeliveryStatus exposes updateDeliveryStatus. It passes no
// target type, so no metric moves.
func (e *Engine) ExportUpdateDeliveryStatus(
webhookDB *gorm.DB,
d *database.Delivery,
status database.DeliveryStatus,
) error {
return e.updateDeliveryStatus(webhookDB, d, "", status)
}
// ExportProcessNewTask exposes processNewTask.
func (e *Engine) ExportProcessNewTask(
ctx context.Context, task *Task,
+3 -13
View File
@@ -299,9 +299,8 @@ func countInFlightDeliveries(
return count, err
}
// createReplayDelivery writes the new pending delivery row, adds it to
// its target's totals in the same transaction, and returns the task
// that carries it to the delivery engine.
// createReplayDelivery writes the new pending delivery row and returns
// the task that carries it to the delivery engine.
//
// The row is written with associations omitted, and neither Event nor
// Target is populated on it: GORM's SaveBeforeAssociations would
@@ -320,16 +319,7 @@ func createReplayDelivery(
Status: database.DeliveryStatusPending,
}
err := webhookDB.Transaction(func(tx *gorm.DB) error {
err := tx.Omit(clause.Associations).Create(dlv).Error
if err != nil {
return err
}
return database.AddTargetTotals(tx, database.TargetTotals{
TargetID: dlv.TargetID, Deliveries: 1,
})
})
err := webhookDB.Omit(clause.Associations).Create(dlv).Error
if err != nil {
return delivery.Task{}, err
}
-25
View File
@@ -4,9 +4,7 @@ import (
"html/template"
"log/slog"
"net/http"
"time"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
@@ -71,29 +69,6 @@ func (s *Handlers) LoadEventLogViewsForTest(
return views
}
// WebhookStatsForTest returns the figures the statistics pane on a
// webhook's page shows, from the webhook's entrypoints and targets
// loaded as that page loads them.
func (s *Handlers) WebhookStatsForTest(webhookID string) *WebhookStats {
var entrypoints []database.Entrypoint
s.db.DB().Where("webhook_id = ?", webhookID).Find(&entrypoints)
var targets []database.Target
s.db.DB().Where("webhook_id = ?", webhookID).Find(&targets)
return s.loadWebhookStats(webhookID, entrypoints, targets)
}
// FinishedByTargetForTest exposes finishedByTarget for use in the
// handlers_test package.
func FinishedByTargetForTest(
webhookDB *gorm.DB, since time.Time,
) ([]TargetFinished, error) {
return finishedByTarget(webhookDB, since)
}
// AddTemplateForTest registers a template under a page name so that
// the handlers_test package can drive the render path with a
// template of its own.
+12 -16
View File
@@ -91,22 +91,18 @@ type Handlers struct {
// parsePageTemplate parses a page-specific template set from the
// embedded FS. Each page template is combined with the shared
// base, htmlheader, and navbar templates, and with any further files
// the page includes. The page file must be listed first so that its
// root action ({{template "base" .}}) becomes the template set's entry
// point.
func parsePageTemplate(
pageFile string, included ...string,
) *template.Template {
files := append([]string{
pageFile,
"base.html",
"htmlheader.html",
"navbar.html",
}, included...)
// base, htmlheader, and navbar templates. The page file must be
// listed first so that its root action ({{template "base" .}})
// becomes the template set's entry point.
func parsePageTemplate(pageFile string) *template.Template {
return template.Must(
template.ParseFS(templates.Templates, files...),
template.ParseFS(
templates.Templates,
pageFile,
"base.html",
"htmlheader.html",
"navbar.html",
),
)
}
@@ -135,7 +131,7 @@ func New(
"profile.html": parsePageTemplate("profile.html"),
"sources_list.html": parsePageTemplate("sources_list.html"),
"sources_new.html": parsePageTemplate("sources_new.html"),
"source_detail.html": parsePageTemplate("source_detail.html", "webhook_stats.html"),
"source_detail.html": parsePageTemplate("source_detail.html"),
"source_edit.html": parsePageTemplate("source_edit.html"),
"source_logs.html": parsePageTemplate("source_logs.html"),
"target_edit.html": parsePageTemplate("target_edit.html"),
-2
View File
@@ -88,8 +88,6 @@ func TestHandleProfile_OwnProfile_OK(t *testing.T) {
h.HandleProfile().ServeHTTP(w, req)
assert.Equal(t, http.StatusOK, w.Code)
assert.Contains(t, w.Body.String(), "Account Information")
assert.NotContains(t, w.Body.String(), "Account Type")
}
func TestHandleProfile_OtherProfile_Forbidden(t *testing.T) {
-1
View File
@@ -450,7 +450,6 @@ func (h *Handlers) renderSourceDetail(
"Targets": delivery.NewTargetViews(targets),
"Events": events,
"BaseURL": baseURL,
"Stats": h.loadWebhookStats(webhook.ID, entrypoints, targets),
}
h.renderTemplate(w, r, "source_detail.html", data)
+7 -23
View File
@@ -252,12 +252,11 @@ func requestEventSource(
}
}
// createAndFanOut writes the event and one pending delivery per target,
// and adds them to the webhook's running totals, in a single
// transaction, then hands the tasks to the delivery engine. It is the
// only path by which an event and its deliveries are created, so a
// resubmitted event is retried, SSRF-guarded and circuit-broken
// exactly as a received one is.
// createAndFanOut writes the event and one pending delivery per target
// in a single transaction, then hands the tasks to the delivery
// engine. It is the only path by which an event and its deliveries are
// created, so a resubmitted event is retried, SSRF-guarded and
// circuit-broken exactly as a received one is.
//
// The tasks are returned as well as queued, so a caller can report how
// many targets the event went to.
@@ -297,13 +296,6 @@ func (h *Handlers) createAndFanOut(
return nil, nil, err
}
err = database.AddEventTotals(tx, database.EventTotals{Events: 1})
if err != nil {
tx.Rollback()
return nil, nil, err
}
err = tx.Commit().Error
if err != nil {
return nil, nil, fmt.Errorf(
@@ -362,9 +354,8 @@ func (h *Handlers) finishWebhookResponse(
}
// buildDeliveryTasks creates one pending delivery per target in the
// transaction, adds each to its target's totals, and returns the tasks
// for the delivery engine. The caller owns the transaction and rolls
// it back on error.
// transaction and returns the tasks for the delivery engine. The
// caller owns the transaction and rolls it back on error.
func buildDeliveryTasks(
tx *gorm.DB,
event *database.Event,
@@ -388,13 +379,6 @@ func buildDeliveryTasks(
)
}
err = database.AddTargetTotals(tx, database.TargetTotals{
TargetID: targets[i].ID, Deliveries: 1,
})
if err != nil {
return nil, err
}
tasks = append(tasks, delivery.Task{
DeliveryID: dlv.ID,
EventID: event.ID,
-271
View File
@@ -1,271 +0,0 @@
package handlers
import (
"fmt"
"time"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// The spans of the two recent windows the statistics pane reports on:
// the last 10 minutes and the last 24 hours.
const (
shortWindow = 10 * time.Minute
longWindow = 24 * time.Hour
)
// percent turns a fraction into a percentage.
const percent = 100
// WebhookStats holds the figures in the statistics pane at the top of
// the webhook page.
type WebhookStats struct {
Entrypoints int
ActiveEntrypoints int
Targets int
ActiveTargets int
// Lifetime counts every event, delivery and failure the webhook
// has had, and WithinRetention those still stored.
Lifetime Counts
WithinRetention Counts
// InProgress counts the deliveries still pending or retrying.
InProgress int64
// LastEventAt is when the newest stored event arrived, or nil when
// none is stored.
LastEventAt *time.Time
Last10Minutes RecentWindow
Last24Hours RecentWindow
}
// Counts holds a number of events, of deliveries and of failed
// deliveries.
type Counts struct {
Events int64
Deliveries int64
Failures int64
}
// RecentWindow holds what happened in one recent window: the events
// received in it, and the deliveries that became delivered or failed in
// it.
type RecentWindow struct {
Events int64
Delivered int64
Failed int64
}
// TargetFinished is how many of one target's deliveries became
// delivered, and how many failed, in a recent window.
type TargetFinished struct {
TargetID string
Delivered int64
Failed int64
}
// FailurePercent is the share of the deliveries finished in the window
// that failed, or a dash when none finished. Deliveries still pending
// or retrying are not counted either way.
func (w RecentWindow) FailurePercent() string {
finished := w.Delivered + w.Failed
if finished == 0 {
return "—"
}
return fmt.Sprintf(
"%.1f%%", percent*float64(w.Failed)/float64(finished),
)
}
// loadWebhookStats gathers the figures for the statistics pane from the
// webhook's entrypoints and targets, as the page has already loaded
// them, and from its event database. It returns nil, and logs why, when
// the event database cannot be read.
func (h *Handlers) loadWebhookStats(
webhookID string,
entrypoints []database.Entrypoint,
targets []database.Target,
) *WebhookStats {
stats := &WebhookStats{
Entrypoints: len(entrypoints),
Targets: len(targets),
}
for i := range entrypoints {
if entrypoints[i].Active {
stats.ActiveEntrypoints++
}
}
for i := range targets {
if targets[i].Active {
stats.ActiveTargets++
}
}
// Opening an event database that does not exist would create it,
// and it would hold nothing to count.
if !h.dbMgr.DBExists(webhookID) {
return stats
}
webhookDB, err := h.dbMgr.GetDB(webhookID)
if err == nil {
err = readEventStats(webhookDB, time.Now(), stats)
}
if err != nil {
h.log.Error(
"failed to read webhook statistics",
"webhook_id", webhookID,
"error", err,
)
return nil
}
return stats
}
// readEventStats fills in the figures that come from the webhook's
// event database. None of them reads every stored row: the totals are
// one row for the events and one per target for the deliveries, and
// every other figure is read from an index, over only the rows it
// counts.
func readEventStats(
db *gorm.DB, now time.Time, stats *WebhookStats,
) error {
err := readTotals(db, stats)
if err != nil {
return err
}
err = db.Model(&database.Delivery{}).
Where("status IN ?", []database.DeliveryStatus{
database.DeliveryStatusPending,
database.DeliveryStatusRetrying,
}).
Count(&stats.InProgress).Error
if err != nil {
return fmt.Errorf("counting deliveries in progress: %w", err)
}
var newest []time.Time
err = db.Model(&database.Event{}).
Order("created_at DESC").
Limit(1).
Pluck("created_at", &newest).Error
if err != nil {
return fmt.Errorf("reading newest event time: %w", err)
}
if len(newest) > 0 {
stats.LastEventAt = &newest[0]
}
stats.Last10Minutes, err = readRecentWindow(
db, now.Add(-shortWindow),
)
if err != nil {
return err
}
stats.Last24Hours, err = readRecentWindow(
db, now.Add(-longWindow),
)
return err
}
// readTotals fills in the lifetime and within-retention figures from
// the running totals: the events' row, and the targets' rows summed.
func readTotals(db *gorm.DB, stats *WebhookStats) error {
var events database.EventTotals
err := db.Take(&events).Error
if err != nil {
return fmt.Errorf("reading event totals: %w", err)
}
var targets []database.TargetTotals
err = db.Find(&targets).Error
if err != nil {
return fmt.Errorf("reading target totals: %w", err)
}
stats.Lifetime.Events = events.Events
stats.WithinRetention.Events = events.Events - events.EventsRemoved
for _, t := range targets {
stats.Lifetime.Deliveries += t.Deliveries
stats.Lifetime.Failures += t.Failed
stats.WithinRetention.Deliveries += t.Deliveries - t.DeliveriesRemoved
stats.WithinRetention.Failures += t.Failed - t.FailedRemoved
}
return nil
}
// readRecentWindow counts the events received, and the deliveries that
// became delivered or failed, since the given time.
func readRecentWindow(
db *gorm.DB, since time.Time,
) (RecentWindow, error) {
var w RecentWindow
err := db.Model(&database.Event{}).
Where("created_at >= ?", since).
Count(&w.Events).Error
if err != nil {
return w, fmt.Errorf("counting recent events: %w", err)
}
byTarget, err := finishedByTarget(db, since)
if err != nil {
return w, err
}
for _, f := range byTarget {
w.Delivered += f.Delivered
w.Failed += f.Failed
}
return w, nil
}
// finishedByTarget counts, for each target, the deliveries that became
// delivered and those that failed since the given time, in one query
// over just that window of the deliveries' status index. A target with
// neither is left out.
func finishedByTarget(
db *gorm.DB, since time.Time,
) ([]TargetFinished, error) {
var byTarget []TargetFinished
err := db.Model(&database.Delivery{}).
Select("target_id, "+
"count(CASE WHEN status = ? THEN 1 END) AS delivered, "+
"count(CASE WHEN status = ? THEN 1 END) AS failed",
database.DeliveryStatusDelivered,
database.DeliveryStatusFailed).
Where("status IN ? AND finished_at >= ?",
[]database.DeliveryStatus{
database.DeliveryStatusDelivered,
database.DeliveryStatusFailed,
}, since).
Group("target_id").
Find(&byTarget).Error
if err != nil {
return nil, fmt.Errorf(
"counting deliveries finished by target: %w", err,
)
}
return byTarget, nil
}
-442
View File
@@ -1,442 +0,0 @@
package handlers_test
import (
"net/http"
"strings"
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/fx/fxtest"
"gorm.io/gorm"
"gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/logger"
"sneak.berlin/go/webhooker/internal/session"
)
// statsEntrypoint adds an entrypoint to a webhook and returns its path.
func statsEntrypoint(
t *testing.T, db *database.Database, webhookID string, active bool,
) string {
t.Helper()
ep := &database.Entrypoint{
WebhookID: webhookID,
Path: uuid.New().String(),
}
require.NoError(t, db.DB().Omit(clause.Associations).Create(ep).Error)
require.NoError(t, db.DB().Model(ep).Update("active", active).Error)
return ep.Path
}
// statsDelivery returns an event's delivery to a target.
func statsDelivery(
t *testing.T, webhookDB *gorm.DB, eventID, targetID string,
) database.Delivery {
t.Helper()
var d database.Delivery
require.NoError(t, webhookDB.Where(
"event_id = ? AND target_id = ?", eventID, targetID,
).First(&d).Error)
return d
}
// statsFinish settles a delivery as the delivery engine does: its
// final status and the time it finished, and one more on its target's
// delivered or failed total, in one transaction.
func statsFinish(
t *testing.T,
webhookDB *gorm.DB,
d database.Delivery,
status database.DeliveryStatus,
at time.Time,
) {
t.Helper()
add := database.TargetTotals{TargetID: d.TargetID, Delivered: 1}
if status == database.DeliveryStatusFailed {
add = database.TargetTotals{TargetID: d.TargetID, Failed: 1}
}
require.NoError(t, webhookDB.Transaction(func(tx *gorm.DB) error {
err := tx.Model(&database.Delivery{}).
Where("id = ?", d.ID).
Updates(map[string]any{"status": status, "finished_at": at}).
Error
if err != nil {
return err
}
return database.AddTargetTotals(tx, add)
}))
}
// statsAge moves an event's arrival back to the given time.
func statsAge(
t *testing.T, webhookDB *gorm.DB, eventID string, at time.Time,
) {
t.Helper()
require.NoError(t, webhookDB.Model(&database.Event{}).
Where("id = ?", eventID).
Update("created_at", at).Error)
}
// statsTargetTotals reads a webhook database's target totals, keyed by
// target.
func statsTargetTotals(
t *testing.T, webhookDB *gorm.DB,
) map[string]database.TargetTotals {
t.Helper()
var rows []database.TargetTotals
require.NoError(t, webhookDB.Find(&rows).Error)
byTarget := make(map[string]database.TargetTotals, len(rows))
for _, row := range rows {
byTarget[row.TargetID] = row
}
return byTarget
}
// statsHistory is the webhook seedStatsHistory builds: its event
// database, its newest event, and its two active targets.
type statsHistory struct {
webhook *database.Webhook
webhookDB *gorm.DB
newest database.Event
first, second string
}
// seedStatsHistory builds the webhook the statistics test checks: one
// day of retention, two entrypoints (one inactive) and three targets
// (one inactive). Three events arrive through the receiver, and so
// each has a delivery to the two active targets. The oldest event is
// past retention, the middle one six hours old, the newest just in.
// Their deliveries are settled as the delivery engine would, and a
// replay adds a pending delivery to the oldest event.
func seedStatsHistory(
t *testing.T,
h *handlers.Handlers,
sess *session.Session,
db *database.Database,
dbMgr *database.WebhookDBManager,
) statsHistory {
t.Helper()
wh := &database.Webhook{
UserID: deleteTestUserID, Name: "stats", RetentionDays: 1,
}
require.NoError(t, db.DB().Omit(clause.Associations).Create(wh).Error)
path := statsEntrypoint(t, db, wh.ID, true)
statsEntrypoint(t, db, wh.ID, false)
first := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+replayTargetURL+`"}`,
)
second := seedTarget(t, db, wh.ID, database.TargetTypeLog)
inactive := seedTarget(t, db, wh.ID, database.TargetTypeLog)
require.NoError(t, db.DB().Model(inactive).
Update("active", false).Error)
router := receiverRouter(h)
for range 3 {
require.Equal(t, http.StatusOK, postReceiver(t, router, path))
}
webhookDB, err := dbMgr.GetDB(wh.ID)
require.NoError(t, err)
events := listEvents(t, webhookDB)
require.Len(t, events, 3)
oldest, middle, newest := events[0], events[1], events[2]
now := time.Now()
statsAge(t, webhookDB, oldest.ID, now.Add(-50*time.Hour))
statsAge(t, webhookDB, middle.ID, now.Add(-6*time.Hour))
oldestFailure := statsDelivery(t, webhookDB, oldest.ID, first.ID)
statsFinish(t, webhookDB, oldestFailure,
database.DeliveryStatusFailed, now.Add(-49*time.Hour))
statsFinish(t, webhookDB,
statsDelivery(t, webhookDB, oldest.ID, second.ID),
database.DeliveryStatusDelivered, now.Add(-49*time.Hour))
statsFinish(t, webhookDB,
statsDelivery(t, webhookDB, middle.ID, first.ID),
database.DeliveryStatusFailed, now.Add(-5*time.Hour))
statsFinish(t, webhookDB,
statsDelivery(t, webhookDB, middle.ID, second.ID),
database.DeliveryStatusFailed, now.Add(-time.Minute))
statsFinish(t, webhookDB,
statsDelivery(t, webhookDB, newest.ID, first.ID),
database.DeliveryStatusDelivered, now.Add(-2*time.Minute))
require.Equal(t, http.StatusSeeOther,
postReplay(t, h, sess, wh.ID, oldestFailure.ID).Code)
return statsHistory{
webhook: wh,
webhookDB: webhookDB,
newest: newest,
first: first.ID,
second: second.ID,
}
}
// statsPrune runs the real retention reaper until it has removed one
// event from the webhook's database, then stops it.
func statsPrune(
t *testing.T,
db *database.Database,
dbMgr *database.WebhookDBManager,
log *logger.Logger,
webhookDB *gorm.DB,
) {
t.Helper()
lc := fxtest.NewLifecycle(t)
database.NewRetentionReaper(lc, database.RetentionReaperParams{
Config: &config.Config{
RetentionSweepInterval: 10 * time.Millisecond,
},
Database: db,
DBManager: dbMgr,
Logger: log,
})
lc.RequireStart()
require.Eventually(t, func() bool {
var totals database.EventTotals
err := webhookDB.Take(&totals).Error
return err == nil && totals.EventsRemoved == 1
}, 10*time.Second, 10*time.Millisecond)
lc.RequireStop()
}
// statsPane returns the statistics pane from a rendered webhook page:
// everything from its heading to the next heading on the page.
func statsPane(t *testing.T, page string) string {
t.Helper()
_, pane, found := strings.Cut(page, ">Statistics</h2>")
require.True(t, found, "the page has no statistics pane")
pane, _, _ = strings.Cut(pane, "<h2")
return pane
}
// assertStatsTargets checks, for the history seedStatsHistory builds,
// each target's totals and its deliveries finished in the last 24
// hours. The first target has three deliveries and the replay, the
// second three; the inactive target has none and so no row.
func assertStatsTargets(t *testing.T, hist statsHistory) {
t.Helper()
first, second := hist.first, hist.second
assert.Equal(t, map[string]database.TargetTotals{
first: {TargetID: first, Deliveries: 4, Delivered: 1, Failed: 2},
second: {
TargetID: second, Deliveries: 3, Delivered: 1, Failed: 1,
},
}, statsTargetTotals(t, hist.webhookDB))
lastDay, err := handlers.FinishedByTargetForTest(
hist.webhookDB, time.Now().Add(-24*time.Hour),
)
require.NoError(t, err)
assert.ElementsMatch(t, []handlers.TargetFinished{
{TargetID: first, Delivered: 1, Failed: 1},
{TargetID: second, Failed: 1},
}, lastDay)
}
// TestWebhookStats_EveryFigureAcrossRetentionPrune checks every figure
// the statistics pane shows for the history seedStatsHistory builds,
// and each target's totals and recent figures, before and after the
// real retention reaper removes the oldest event.
func TestWebhookStats_EveryFigureAcrossRetentionPrune(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
log *logger.Logger
)
app := newTestApp(t, &h, &sess, &db, &dbMgr, &log)
app.RequireStart()
t.Cleanup(app.RequireStop)
hist := seedStatsHistory(t, h, sess, db, dbMgr)
first, second := hist.first, hist.second
stats := h.WebhookStatsForTest(hist.webhook.ID)
require.NotNil(t, stats)
assert.Equal(t, 2, stats.Entrypoints)
assert.Equal(t, 1, stats.ActiveEntrypoints)
assert.Equal(t, 3, stats.Targets)
assert.Equal(t, 2, stats.ActiveTargets)
assert.Equal(t, handlers.Counts{Events: 3, Deliveries: 7, Failures: 3},
stats.Lifetime)
assert.Equal(t, stats.Lifetime, stats.WithinRetention)
assert.Equal(t, int64(2), stats.InProgress)
require.NotNil(t, stats.LastEventAt)
assert.True(t, hist.newest.CreatedAt.Equal(*stats.LastEventAt))
assert.Equal(t, handlers.RecentWindow{
Events: 1, Delivered: 1, Failed: 1,
}, stats.Last10Minutes)
assert.Equal(t, handlers.RecentWindow{
Events: 2, Delivered: 1, Failed: 2,
}, stats.Last24Hours)
assert.Equal(t, "50.0%", stats.Last10Minutes.FailurePercent())
assert.Equal(t, "66.7%", stats.Last24Hours.FailurePercent())
assertStatsTargets(t, hist)
// Retention removes the oldest event with its three deliveries:
// the first target's failed one and the pending replay, and the
// second target's delivered one.
statsPrune(t, db, dbMgr, log, hist.webhookDB)
after := h.WebhookStatsForTest(hist.webhook.ID)
require.NotNil(t, after)
assert.Equal(t, stats.Lifetime, after.Lifetime)
assert.Equal(t, handlers.Counts{Events: 2, Deliveries: 4, Failures: 2},
after.WithinRetention)
assert.Equal(t, int64(1), after.InProgress)
assert.Equal(t, stats.LastEventAt, after.LastEventAt)
assert.Equal(t, stats.Last10Minutes, after.Last10Minutes)
assert.Equal(t, stats.Last24Hours, after.Last24Hours)
assert.Equal(t, map[string]database.TargetTotals{
first: {
TargetID: first, Deliveries: 4, Delivered: 1, Failed: 2,
DeliveriesRemoved: 2, FailedRemoved: 1,
},
second: {
TargetID: second, Deliveries: 3, Delivered: 1, Failed: 1,
DeliveriesRemoved: 1,
},
}, statsTargetTotals(t, hist.webhookDB))
pane := statsPane(t, renderSourceDetailPage(t, h, sess, hist.webhook.ID))
assert.Contains(t, pane, "Within retention")
assert.Contains(t, pane, "50.0%")
assert.Contains(t, pane, "66.7%")
}
// TestWebhookStats_PaneShowsRetentionPeriod checks that the statistics
// pane itself, not only the line at the foot of the page, shows the
// webhook's retention period, for a finite one and for forever.
func TestWebhookStats_PaneShowsRetentionPeriod(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
)
app := newTestApp(t, &h, &sess, &db)
app.RequireStart()
t.Cleanup(app.RequireStop)
tests := []struct {
retentionDays int
want string
}{
{30, "30 days"},
{database.RetentionForeverDays, "forever"},
}
for _, tt := range tests {
wh := &database.Webhook{
UserID: deleteTestUserID,
Name: "retention",
RetentionDays: tt.retentionDays,
}
require.NoError(t,
db.DB().Omit(clause.Associations).Create(wh).Error)
pane := statsPane(t, renderSourceDetailPage(t, h, sess, wh.ID))
assert.Contains(t, pane, "Retention", tt.want)
assert.Contains(t, pane, tt.want)
}
}
// TestWebhookStats_WebhookWithNoEvents covers a webhook whose event
// database has never been opened: every count is zero, the
// percentages are a dash, and showing the page does not create the
// database.
func TestWebhookStats_WebhookWithNoEvents(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
assert.Equal(t, &handlers.WebhookStats{}, h.WebhookStatsForTest(wh.ID))
assert.Equal(t, "—", handlers.RecentWindow{}.FailurePercent())
statsPane(t, renderSourceDetailPage(t, h, sess, wh.ID))
assert.False(t, dbMgr.DBExists(wh.ID))
}
// TestRecentWindow_FailurePercent pins the percentage: failed
// deliveries out of all that finished in the window.
func TestRecentWindow_FailurePercent(t *testing.T) {
t.Parallel()
tests := []struct {
window handlers.RecentWindow
want string
}{
{handlers.RecentWindow{}, "—"},
{handlers.RecentWindow{Events: 4}, "—"},
{handlers.RecentWindow{Delivered: 3, Failed: 1}, "25.0%"},
{handlers.RecentWindow{Failed: 2}, "100.0%"},
{handlers.RecentWindow{Delivered: 2}, "0.0%"},
}
for _, tt := range tests {
assert.Equal(t, tt.want, tt.window.FailurePercent(), tt.window)
}
}
+276
View File
@@ -0,0 +1,276 @@
package server_test
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"os/exec"
"slices"
"strings"
"sync"
"testing"
"time"
"github.com/chromedp/cdproto/network"
"github.com/chromedp/cdproto/runtime"
"github.com/chromedp/chromedp"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const (
// browserTimeout bounds everything one test does in the browser.
browserTimeout = 60 * time.Second
// settleTimeout bounds the wait for an element to show or hide.
settleTimeout = 5 * time.Second
)
// TestAlpineRunsUnderTheSecurityPolicy loads the webhook page and the
// event log in headless Chromium, served by the real router and so
// under the real Content-Security-Policy, and checks that the pages'
// Alpine.js directives work.
func TestAlpineRunsUnderTheSecurityPolicy(t *testing.T) {
t.Parallel()
ctx, consoleProblems := startBrowser(t)
env := newTestEnv(t)
srv := httptest.NewServer(env.router)
t.Cleanup(srv.Close)
userID, _ := env.seedUser(t, "browser", "browser-password")
webhook := env.seedWebhook(t, userID)
event := env.seedEvent(t, webhook.ID, `{"hello":"browser"}`)
require.NoError(t, chromedp.Run(
ctx, setCookies(srv.URL, env.authCookies(t, userID, "browser")),
))
checkAddForms(ctx, t, srv.URL+"/source/"+webhook.ID)
checkTargetType(ctx, t)
checkEventToggle(
ctx, t, srv.URL+"/source/"+webhook.ID+"/logs", event.ID,
)
assert.Empty(
t, consoleProblems(),
"the pages printed console warnings, errors or exceptions",
)
}
// startBrowser starts headless Chromium for one test. It returns the
// context that drives it, and a function listing every console warning
// or error and every uncaught exception its pages raised: that is how
// Alpine.js reports an expression it cannot run.
//
// The test is skipped when chromium is not on PATH. The Dockerfile's
// test stage installs it, so the image build always runs this check.
func startBrowser(t *testing.T) (context.Context, func() []string) {
t.Helper()
path, err := exec.LookPath("chromium")
if err != nil {
t.Skipf("chromium is not installed: %v", err)
}
allocCtx, cancelAlloc := chromedp.NewExecAllocator(
t.Context(),
append(
chromedp.DefaultExecAllocatorOptions[:],
chromedp.ExecPath(path),
// The image build runs tests as root, where Chromium's
// sandbox cannot start.
chromedp.NoSandbox,
)...,
)
t.Cleanup(cancelAlloc)
ctx, cancel := chromedp.NewContext(allocCtx)
t.Cleanup(cancel)
ctx, cancelTimeout := context.WithTimeout(ctx, browserTimeout)
t.Cleanup(cancelTimeout)
var (
mu sync.Mutex
problems []string
)
chromedp.ListenTarget(ctx, func(ev any) {
var problem string
switch ev := ev.(type) {
case *runtime.EventConsoleAPICalled:
if ev.Type != runtime.APITypeWarning &&
ev.Type != runtime.APITypeError {
return
}
args := make([]string, 0, len(ev.Args))
for _, arg := range ev.Args {
args = append(args, string(arg.Value))
}
problem = strings.Join(args, " ")
case *runtime.EventExceptionThrown:
problem = ev.ExceptionDetails.Error()
default:
return
}
mu.Lock()
defer mu.Unlock()
problems = append(problems, problem)
})
return ctx, func() []string {
mu.Lock()
defer mu.Unlock()
return slices.Clone(problems)
}
}
// setCookies gives the browser the cookies for the server at base.
func setCookies(base string, cookies []*http.Cookie) chromedp.ActionFunc {
return chromedp.ActionFunc(func(ctx context.Context) error {
for _, c := range cookies {
err := network.SetCookie(c.Name, c.Value).
WithURL(base).
Do(ctx)
if err != nil {
return fmt.Errorf("set cookie %s: %w", c.Name, err)
}
}
return nil
})
}
// loadPage opens url and waits for Alpine.js to start, which it does
// by removing every x-cloak attribute. Until then x-cloak hides the
// elements Alpine would hide, so a check made earlier proves nothing.
func loadPage(url string) chromedp.Tasks {
return chromedp.Tasks{
chromedp.Navigate(url),
chromedp.WaitNotPresent("[x-cloak]", chromedp.ByQuery),
}
}
// shown waits up to settleTimeout for the first element matching a CSS
// selector to be rendered, and reports whether it was. The wait is
// needed because Alpine.js shows an element on the next animation
// frame, not at once.
func shown(ctx context.Context, selector string) bool {
ctx, cancel := context.WithTimeout(ctx, settleTimeout)
defer cancel()
return chromedp.Run(
ctx, chromedp.WaitVisible(selector, chromedp.ByQuery),
) == nil
}
// hidden is shown's opposite: it waits for the element to be hidden.
func hidden(ctx context.Context, selector string) bool {
ctx, cancel := context.WithTimeout(ctx, settleTimeout)
defer cancel()
return chromedp.Run(
ctx, chromedp.WaitNotVisible(selector, chromedp.ByQuery),
) == nil
}
// click clicks the first element matching an XPath expression.
func click(ctx context.Context, t *testing.T, xpath string) {
t.Helper()
require.NoError(t, chromedp.Run(
ctx, chromedp.Click(xpath, chromedp.BySearch),
))
}
// checkAddForms loads a webhook page and checks that each section's add
// form stays hidden until the Add button beside its heading is clicked.
func checkAddForms(ctx context.Context, t *testing.T, url string) {
t.Helper()
require.NoError(t, chromedp.Run(ctx, loadPage(url)))
sections := []struct{ heading, form string }{
{"Entrypoints", `form[action$="/entrypoints"]`},
{"Targets", `form[action$="/targets"]`},
}
for _, s := range sections {
assert.Truef(
t, hidden(ctx, s.form),
"%s: the add form shows before Add is clicked", s.heading,
)
click(ctx, t, `//h2[text()="`+s.heading+
`"]/following-sibling::button`)
assert.Truef(
t, shown(ctx, s.form),
"%s: the add form stays hidden when Add is clicked", s.heading,
)
}
}
// checkTargetType chooses Slack in the open add target form and checks
// what the form would then submit: one url field, the Slack one, and
// not the HTTP url, headers or timeout, which are hidden and disabled.
func checkTargetType(ctx context.Context, t *testing.T) {
t.Helper()
const (
chooseSlack = `(() => {
const type = document.querySelector('select[name="type"]');
type.value = "slack";
type.dispatchEvent(new Event("change"));
})()`
submitted = `[...new FormData(
document.querySelector('form[action$="/targets"]')).keys()]`
)
var fields []string
require.NoError(t, chromedp.Run(
ctx,
chromedp.Evaluate(chooseSlack, nil),
chromedp.Evaluate(submitted, &fields),
))
assert.Equal(
t,
[]string{"csrf_token", "name", "type", "max_retries", "url"},
fields,
"with Slack chosen, the HTTP fields must not be submitted",
)
}
// checkEventToggle loads the event log and checks that clicking an
// event's row expands it and clicking again collapses it.
func checkEventToggle(
ctx context.Context, t *testing.T, url, eventID string,
) {
t.Helper()
// The row shows the event's ID; its Resubmit form is in the part
// that expands.
row := `//span[text()="` + eventID + `"]`
expanded := `form[action$="/resubmit"]`
require.NoError(t, chromedp.Run(ctx, loadPage(url)))
assert.True(t, hidden(ctx, expanded), "the event starts expanded")
click(ctx, t, row)
assert.True(t, shown(ctx, expanded), "clicking the event does not expand it")
click(ctx, t, row)
assert.True(t, hidden(ctx, expanded), "clicking it again does not collapse it")
}
+5 -4
View File
@@ -1,15 +1,16 @@
#!/bin/sh
# script/assets: extract Alpine.js from its npm package tarball, committed
# in 3p/, to static/js/alpine.min.js, where go:embed reads it. The
# extracted file is not committed. script/test, make build and make dev run
# this first.
# in 3p/, to static/js/alpine.min.js, where go:embed reads it. The package
# is @alpinejs/csp, Alpine's build for pages whose Content-Security-Policy
# forbids eval. The extracted file is not committed. script/test, make
# build and make dev run this first.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
tar -xzOf 3p/alpinejs-3.14.9.tgz package/dist/cdn.min.js \
tar -xzOf 3p/alpinejs-csp-3.14.9.tgz package/dist/cdn.min.js \
>static/js/alpine.min.js
}
+59
View File
@@ -57,3 +57,62 @@
init();
}
})();
// Alpine.js components.
//
// The pages' Content-Security-Policy forbids eval, so the UI loads
// Alpine's CSP build, which cannot run expressions written in the
// markup: a directive in templates/ may only name a property or method,
// and each x-data names a component registered here. This script runs
// before Alpine, whose script tag is deferred, so this listener is in
// place when Alpine starts.
document.addEventListener("alpine:init", function () {
"use strict";
// Something a click shows and hides: the mobile menu, an add form,
// an event in the event log, a delivery's attempts.
window.Alpine.data("collapsible", function () {
return {
open: false,
toggle() {
this.open = !this.open;
},
get closed() {
return !this.open;
},
// Turns a downward caret up while open.
get caretClass() {
return { "rotate-180": this.open };
},
};
});
// The add target form. Only the chosen type's fields show, and the
// others are disabled so that the form does not submit them.
window.Alpine.data("targetForm", function () {
return {
targetType: "http",
chooseType(event) {
this.targetType = event.target.value;
},
get isHttp() {
return this.targetType === "http";
},
get isSlack() {
return this.targetType === "slack";
},
get isDatabase() {
return this.targetType === "database";
},
get notHttp() {
return !this.isHttp;
},
get notSlack() {
return !this.isSlack;
},
get notDatabase() {
return !this.isDatabase;
},
};
});
});
+3 -3
View File
@@ -1,14 +1,14 @@
{{define "navbar"}}
<nav class="app-bar" x-data="{ open: false }">
<nav class="app-bar" x-data="collapsible">
<div class="max-w-6xl mx-auto flex justify-between items-center">
<div class="flex items-center gap-3">
<a href="/" class="text-xl font-medium text-gray-900 hover:text-primary-600 transition-colors">Webhooker</a>
</div>
<!-- Mobile menu button -->
<button @click="open = !open" class="md:hidden p-2 rounded-md text-gray-500 hover:bg-gray-100">
<button @click="toggle" class="md:hidden p-2 rounded-md text-gray-500 hover:bg-gray-100">
<svg class="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path x-show="!open" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 6h16M4 12h16M4 18h16"/>
<path x-show="closed" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 6h16M4 12h16M4 18h16"/>
<path x-show="open" x-cloak stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M6 18L18 6M6 6l12 12"/>
</svg>
</button>
+4
View File
@@ -41,6 +41,10 @@
<dt class="w-32 text-sm font-medium text-gray-500">Username</dt>
<dd class="text-sm text-gray-900">{{.User.Username}}</dd>
</div>
<div class="flex">
<dt class="w-32 text-sm font-medium text-gray-500">Account Type</dt>
<dd class="text-sm text-gray-900">Standard User</dd>
</div>
</dl>
</div>
</div>
+20 -22
View File
@@ -3,7 +3,7 @@
{{define "title"}}{{.Webhook.Name}} - Webhooker{{end}}
{{define "content"}}
<div class="max-w-6xl mx-auto px-6 py-8" x-data="{ showAddEntrypoint: false, showAddTarget: false }">
<div class="max-w-6xl mx-auto px-6 py-8">
<div class="mb-6">
<a href="/sources" class="text-sm text-primary-600 hover:text-primary-700">&larr; Back to webhooks</a>
<div class="flex justify-between items-center mt-2">
@@ -24,14 +24,12 @@
</div>
</div>
{{template "webhook_stats" .}}
<div class="grid grid-cols-1 lg:grid-cols-2 gap-6">
<!-- Entrypoints -->
<div class="card">
<div class="card" x-data="collapsible">
<div class="p-4 border-b border-gray-200 flex justify-between items-center">
<h2 class="text-lg font-medium text-gray-900">Entrypoints</h2>
<button @click="showAddEntrypoint = !showAddEntrypoint" class="btn-text text-sm">
<button @click="toggle" class="btn-text text-sm">
<svg class="w-4 h-4 mr-1" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 4v16m8-8H4"/>
</svg>
@@ -40,7 +38,7 @@
</div>
<!-- Add entrypoint form -->
<div x-show="showAddEntrypoint" x-cloak class="p-4 bg-gray-50 border-b border-gray-200">
<div x-show="open" x-cloak class="p-4 bg-gray-50 border-b border-gray-200">
<form method="POST" action="/source/{{.Webhook.ID}}/entrypoints" class="flex gap-2">
<input type="hidden" name="csrf_token" value="{{.CSRFToken}}">
<input type="text" name="description" placeholder="Description (optional)" class="input text-sm flex-1">
@@ -87,10 +85,10 @@
</div>
<!-- Targets -->
<div class="card">
<div class="card" x-data="collapsible">
<div class="p-4 border-b border-gray-200 flex justify-between items-center">
<h2 class="text-lg font-medium text-gray-900">Targets</h2>
<button @click="showAddTarget = !showAddTarget" class="btn-text text-sm">
<button @click="toggle" class="btn-text text-sm">
<svg class="w-4 h-4 mr-1" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 4v16m8-8H4"/>
</svg>
@@ -99,42 +97,42 @@
</div>
<!-- Add target form -->
<div x-show="showAddTarget" x-cloak class="p-4 bg-gray-50 border-b border-gray-200">
<form method="POST" action="/source/{{.Webhook.ID}}/targets" x-data="{ targetType: 'http' }" class="space-y-3">
<div x-show="open" x-cloak class="p-4 bg-gray-50 border-b border-gray-200">
<form method="POST" action="/source/{{.Webhook.ID}}/targets" x-data="targetForm" class="space-y-3">
<input type="hidden" name="csrf_token" value="{{.CSRFToken}}">
<div class="flex gap-2">
<input type="text" name="name" placeholder="Target name" required class="input text-sm flex-1">
<select name="type" x-model="targetType" class="input text-sm w-32">
<select name="type" @change="chooseType" class="input text-sm w-32">
<option value="http">HTTP</option>
<option value="slack">Slack</option>
<option value="database">Database</option>
<option value="log">Log</option>
</select>
</div>
<div x-show="targetType === 'http'">
<input type="url" name="url" placeholder="https://example.com/webhook" :disabled="targetType !== 'http'" class="input text-sm">
<div x-show="isHttp">
<input type="url" name="url" placeholder="https://example.com/webhook" :disabled="notHttp" class="input text-sm">
</div>
<div x-show="targetType === 'http'">
<textarea name="headers" rows="3" placeholder="Authorization: Bearer ..." :disabled="targetType !== 'http'" class="input text-sm"></textarea>
<div x-show="isHttp">
<textarea name="headers" rows="3" placeholder="Authorization: Bearer ..." :disabled="notHttp" class="input text-sm"></textarea>
<p class="text-xs text-gray-500 mt-1">Optional request headers, one <code>Name: value</code> per line, sent with every delivery.</p>
</div>
<div x-show="targetType === 'http'" class="flex gap-2 items-center">
<div x-show="isHttp" class="flex gap-2 items-center">
<label class="text-sm text-gray-700">Timeout (seconds, blank = default):</label>
<input type="number" name="timeout" min="0" max="300" :disabled="targetType !== 'http'" class="input text-sm w-24">
<input type="number" name="timeout" min="0" max="300" :disabled="notHttp" class="input text-sm w-24">
</div>
<div x-show="targetType === 'http'">
<div x-show="isHttp">
<div class="flex gap-2 items-center">
<label class="text-sm text-gray-700">Max retries:</label>
<input type="number" name="max_retries" value="0" min="0" max="20" class="input text-sm w-24">
</div>
<p class="text-xs text-gray-500 mt-1">This is the total number of delivery attempts, not retries on top of the first: a value of 3 makes three attempts in all. 0 means a single attempt with no retries and no circuit breaker.</p>
</div>
<div x-show="targetType === 'slack'">
<input type="url" name="url" placeholder="https://hooks.slack.com/services/..." :disabled="targetType !== 'slack'" class="input text-sm">
<div x-show="isSlack">
<input type="url" name="url" placeholder="https://hooks.slack.com/services/..." :disabled="notSlack" class="input text-sm">
<p class="text-xs text-gray-500 mt-1">Slack or Mattermost incoming webhook URL. Payloads are pretty-printed in code blocks.</p>
</div>
<div x-show="targetType === 'database'">
<input type="text" name="expiry" placeholder="never" :disabled="targetType !== 'database'" class="input text-sm">
<div x-show="isDatabase">
<input type="text" name="expiry" placeholder="never" :disabled="notDatabase" class="input text-sm">
<p class="text-xs text-gray-500 mt-1">Archive expiry: "never" (default) keeps rows forever, or a duration like "720h" prunes older rows.</p>
</div>
<button type="submit" class="btn-primary text-sm">Add Target</button>
+7 -7
View File
@@ -23,8 +23,8 @@
<div class="card">
<div class="divide-y divide-gray-100">
{{range .Events}}
<div class="p-4" x-data="{ open: false }">
<div class="flex items-center justify-between cursor-pointer" @click="open = !open">
<div class="p-4" x-data="collapsible">
<div class="flex items-center justify-between cursor-pointer" @click="toggle">
<div class="flex items-center gap-3">
<span class="badge-info">{{.Method}}</span>
<span class="text-sm font-mono text-gray-700">{{.ID}}</span>
@@ -43,7 +43,7 @@
</span>
{{end}}
<span class="text-xs text-gray-400">{{.CreatedAt.Format "2006-01-02 15:04:05"}}</span>
<svg class="w-4 h-4 text-gray-400 transition-transform" :class="{ 'rotate-180': open }" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<svg class="w-4 h-4 text-gray-400 transition-transform" :class="caretClass" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M19 9l-7 7-7-7"/>
</svg>
</div>
@@ -71,8 +71,8 @@
<h3 class="text-xs font-medium uppercase tracking-wide text-gray-500">Deliveries</h3>
<div class="mt-2 divide-y divide-gray-200">
{{range .Deliveries}}
<div class="py-2" x-data="{ attempts: false }">
<div class="flex items-center justify-between cursor-pointer" @click="attempts = !attempts">
<div class="py-2" x-data="collapsible">
<div class="flex items-center justify-between cursor-pointer" @click="toggle">
<div class="flex items-center gap-3">
<span class="text-sm text-gray-700">{{.Target.DisplayName}}</span>
<span class="text-xs {{if eq .Status "delivered"}}text-green-600{{else if eq .Status "failed"}}text-red-600{{else if eq .Status "retrying"}}text-yellow-600{{else}}text-gray-400{{end}}">{{.Status}}</span>
@@ -86,13 +86,13 @@
</form>
{{end}}
<span class="text-xs text-gray-400">{{.AttemptCount}} attempt{{if ne .AttemptCount 1}}s{{end}}</span>
<svg class="w-3 h-3 text-gray-400 transition-transform" :class="{ 'rotate-180': attempts }" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<svg class="w-3 h-3 text-gray-400 transition-transform" :class="caretClass" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M19 9l-7 7-7-7"/>
</svg>
</div>
</div>
<div x-show="attempts" x-cloak class="mt-2 space-y-2">
<div x-show="open" x-cloak class="mt-2 space-y-2">
{{if .AttemptsOmitted}}
<p class="text-xs text-gray-500">{{.AttemptsOmitted}} attempt{{if ne .AttemptsOmitted 1}}s{{end}} omitted between the first and last shown.</p>
{{end}}
-93
View File
@@ -1,93 +0,0 @@
{{define "webhook_stats"}}
<!-- Statistics pane at the top of the webhook page. -->
<div class="card mb-6">
<div class="p-4 border-b border-gray-200">
<h2 class="text-lg font-medium text-gray-900">Statistics</h2>
</div>
{{with .Stats}}
<div class="p-4 flex flex-wrap gap-6 text-sm border-b border-gray-200">
<div>
<span class="text-gray-500">Entrypoints</span>
<span class="font-medium text-gray-900">{{.Entrypoints}}</span>
<span class="text-gray-500">({{.ActiveEntrypoints}} active)</span>
</div>
<div>
<span class="text-gray-500">Targets</span>
<span class="font-medium text-gray-900">{{.Targets}}</span>
<span class="text-gray-500">({{.ActiveTargets}} active)</span>
</div>
<div>
<span class="text-gray-500">Deliveries in progress</span>
<span class="font-medium text-gray-900">{{.InProgress}}</span>
</div>
<div>
<span class="text-gray-500">Last event</span>
<span class="font-medium text-gray-900">{{with .LastEventAt}}{{.Format "2006-01-02 15:04:05 UTC"}}{{else}}none{{end}}</span>
</div>
<div>
<span class="text-gray-500">Retention</span>
<span class="font-medium text-gray-900">{{$.Webhook.RetentionLabel}}</span>
</div>
</div>
<div class="p-4 grid grid-cols-1 lg:grid-cols-2 gap-6 text-sm">
<table class="w-full text-center text-gray-900">
<thead>
<tr class="border-b border-gray-200 text-xs text-gray-500 uppercase tracking-wide">
<th></th>
<th class="py-2 font-medium">Lifetime</th>
<th class="py-2 font-medium">Within retention</th>
</tr>
</thead>
<tbody>
<tr>
<td class="py-2 text-left text-gray-600">Events</td>
<td class="py-2">{{.Lifetime.Events}}</td>
<td class="py-2">{{.WithinRetention.Events}}</td>
</tr>
<tr>
<td class="py-2 text-left text-gray-600">Deliveries</td>
<td class="py-2">{{.Lifetime.Deliveries}}</td>
<td class="py-2">{{.WithinRetention.Deliveries}}</td>
</tr>
<tr>
<td class="py-2 text-left text-gray-600">Failures</td>
<td class="py-2">{{.Lifetime.Failures}}</td>
<td class="py-2">{{.WithinRetention.Failures}}</td>
</tr>
</tbody>
</table>
<div>
<table class="w-full text-center text-gray-900">
<thead>
<tr class="border-b border-gray-200 text-xs text-gray-500 uppercase tracking-wide">
<th></th>
<th class="py-2 font-medium">Last 10 minutes</th>
<th class="py-2 font-medium">Last 24 hours</th>
</tr>
</thead>
<tbody>
<tr>
<td class="py-2 text-left text-gray-600">Events</td>
<td class="py-2">{{.Last10Minutes.Events}}</td>
<td class="py-2">{{.Last24Hours.Events}}</td>
</tr>
<tr>
<td class="py-2 text-left text-gray-600">Failures</td>
<td class="py-2">{{.Last10Minutes.Failed}}</td>
<td class="py-2">{{.Last24Hours.Failed}}</td>
</tr>
<tr>
<td class="py-2 text-left text-gray-600">Failure percentage</td>
<td class="py-2">{{.Last10Minutes.FailurePercent}}</td>
<td class="py-2">{{.Last24Hours.FailurePercent}}</td>
</tr>
</tbody>
</table>
<p class="mt-2 text-xs text-gray-500">Failure percentage is the failed deliveries out of all deliveries that finished in the window. Deliveries still pending or retrying are not counted.</p>
</div>
</div>
{{else}}
<div class="p-4 text-sm text-gray-500">The statistics could not be read.</div>
{{end}}
</div>
{{end}}