Compare commits

6 Commits
Author SHA1 Message Date
sneak d66292df43 Show a target paused by its circuit breaker (closes #385)
check / check (push) Successful in 3m21s
While an http or slack target's circuit breaker is open, the target's
row on the webhook page says its deliveries are paused until the
cooldown ends, in UTC and from now. Each of its retrying deliveries
shows as waiting in the event log and on the event's page, until the
later of the cooldown's end and the end of its own backoff. While the
breaker is half-open, the row says deliveries are held while one
delivery tests the target, with no time, and deliveries keep their
plain status.

The engine gains one read, StateAndCooldown(targetID), taking a
breaker's state and remaining cooldown under one lock; the handlers
reach it through a one-method interface wired like Archives.

Model: opus-5-5
2026-10-02 23:16:44 +00:00
clawbot 3489d6909a Offer archive expiry choices on the target forms, show plain units (closes #396)
check / check (push) Successful in 3m11s
A database target's archive expiry was typed by hand as never or a raw duration such as 720h, and the target list showed it back raw. Adding or editing a database target now offers the new-webhook page's list of choices (never, 1h, 12h, 24h, 30d, 90d, 365d), defined once and shared by all three forms. The edit form starts on the stored expiry, or on the submitted one after a refused save; a stored value outside the choices is listed under its own value, so saving unchanged keeps it. The target list shows the expiry in plain units: "30 days", "12 hours", "never".

Model: opus-5-5
2026-10-03 01:16:14 +02:00
clawbot f282c6363d Keep what was typed when a target or webhook edit is refused (closes #381)
check / check (push) Successful in 3m17s
A refused save on the target edit page answered with a bare text page, losing the form and everything typed, and the webhook edit page came back with the stored values instead of the submitted ones. A refused target edit now shows the edit form again with the reason above it and every value submitted, with the same status codes as before; a refused webhook edit keeps the submitted name, description and retention. Target edits use the same validation as new targets, with no second copy; an encoding or database failure stays a logged 500. The browser test covers a refused save on both pages, and its main function is now a plain list of checks.

Model: opus-5-5
2026-10-03 00:51:56 +02:00
clawbot 61371d388e Pin tailwindcss and check the committed stylesheet against it (closes #231)
check / check (push) Successful in 3m25s
make css ran whatever tailwindcss binary was on the host's PATH, so the committed stylesheet depended on the machine that built it, and nothing noticed when a template used a class the stylesheet lacked. make css now runs the standalone tailwindcss v4.2.1, pinned by sha256, in a Dockerfile stage, and a check stage, run by make check and required by the image build, fails when the committed static/css/tailwind.css differs from what the templates need, showing the differing rules. input.css names its sources. The unused .btn-text is removed and the stylesheet regenerated, dropping only unused rules. The README has a Stylesheet section.

Model: opus-5-5
2026-10-03 00:30:57 +02:00
clawbot 19a6705c63 New-webhook page: optional HTTP target URL and archive with pruning (closes #373)
check / check (push) Successful in 3m19s
The new-webhook page gains an optional HTTP target URL, which creates an http target named HTTP, and an archive checkbox whose pruning choice (never, 1h, 12h, 24h, 30d, 90d, 365d) creates a database target named Archive with that expiry. Both are validated by the add target form's own validation, and the webhook, its entrypoint and its targets are created in one transaction or not at all. A refused form comes back with the reason and every value entered, retention included. The targets can be renamed on the webhook page like any other.

Model: opus-5-5
2026-10-03 00:21:56 +02:00
clawbot 93911f28f9 Show a slack target's retry setting in the target list (closes #395)
check / check (push) Successful in 3m24s
A slack target's edit page offers Max Retries and the delivery engine honours it, but the target list showed only its masked webhook URL, so setting retries changed nothing visible. The list now shows a slack target's Max Retries line exactly as an http target's, from the one function both use, so the label and the "0 (fire-and-forget)" wording cannot drift apart. The Max Queue Size line stays on http targets only. Tests cover a slack target with retries set, and one with a queue size stored that shows no queue-size line.

Model: opus-5-5
2026-10-03 00:19:13 +02:00
41 changed files with 1849 additions and 533 deletions
+4 -4
View File
@@ -28,10 +28,10 @@ jobs:
- name: Fingerprint the build context
# Writes the hash of the commit being checked into the context, which
# invalidates the `COPY . .` layer of both check stages: a commit
# that was never linted, format-checked, tested and built cannot
# report success from cache.
# invalidates the `COPY . .` layer of every check stage: a commit
# that was never linted, format-checked, stylesheet-checked, tested
# and built cannot report success from cache.
run: git rev-parse HEAD > .ci-fingerprint
- name: Build Docker image (runs make fmt-check, golangci-lint, make test, make build)
- name: Build Docker image (runs make fmt-check, golangci-lint, the stylesheet check, make test, make build)
run: script/cibuild
+38 -1
View File
@@ -29,14 +29,51 @@ RUN script/assets
RUN --network=none golangci-lint config verify --config .golangci.yml
RUN --network=none golangci-lint run --config .golangci.yml --build-tags browser ./...
# Stylesheet stages. static/css/tailwind.css is generated, by this pinned
# tailwindcss, from static/css/input.css and the files its @source lines
# name. `make css` (script/css) writes it out from the css-output stage.
# The css-check stage fails when the committed file differs from what is
# generated; `make check` runs it, and so does the build stage below.
#
# tailwindcss v4.2.1 standalone CLI, released 2026-02-23: one binary per
# architecture, each pinned by its sha256 from the release's sha256sums.txt.
# debian:bookworm-slim, 2026-10-02: the binary needs glibc.
FROM debian:bookworm-slim@sha256:3783cc01769c7b2b1b83a5c5ad96c815348e28ed7da68e2e3687004faa906251 AS tailwind-amd64
ADD --checksum=sha256:39e8d4e24b3c83b0a6e69e100a972fbc75d5fef8dce47b3ddac3cf92dea81fe3 --chmod=755 \
https://github.com/tailwindlabs/tailwindcss/releases/download/v4.2.1/tailwindcss-linux-x64 /usr/local/bin/tailwindcss
FROM debian:bookworm-slim@sha256:3783cc01769c7b2b1b83a5c5ad96c815348e28ed7da68e2e3687004faa906251 AS tailwind-arm64
ADD --checksum=sha256:d87e6486bb3f70b04ef1dcaacc4ee6548a5a15fbf521b31bc24d2c774f68a951 --chmod=755 \
https://github.com/tailwindlabs/tailwindcss/releases/download/v4.2.1/tailwindcss-linux-arm64 /usr/local/bin/tailwindcss
# TARGETARCH, set by docker, is the architecture being built for.
FROM tailwind-${TARGETARCH} AS css
WORKDIR /src
COPY . .
RUN tailwindcss -i static/css/input.css -o /out/tailwind.css --minify
FROM scratch AS css-output
COPY --from=css /out/tailwind.css /
# Both files are split after each "}", one rule per line, so that when they
# differ the diff shows the rules that differ.
FROM css AS css-check
RUN sed 's/}/}\n/g' static/css/tailwind.css > /tmp/committed.css \
&& sed 's/}/}\n/g' /out/tailwind.css > /tmp/generated.css \
&& diff -U0 /tmp/committed.css /tmp/generated.css || { \
echo "static/css/tailwind.css is not what make css generates; run make css" >&2; \
exit 1; \
}
# Build stage
# golang:1.26.1-bookworm (Debian-based), 2026-03-17
# Using Debian-based image because gorm.io/driver/sqlite pulls in
# mattn/go-sqlite3 (CGO), which does not compile on Alpine musl.
FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4 AS builder
# Depend on lint stage passing
# Depend on the lint and stylesheet check stages passing
COPY --from=lint /src/go.sum /dev/null
COPY --from=css-check /out/tailwind.css /dev/null
# jq is a runtime dependency of script/ci-mark-superseded, which the test
# suite executes. git is what script/version derives the version with.
+5 -2
View File
@@ -1,4 +1,4 @@
.PHONY: bootstrap setup assets test test-browser lint fmt fmt-check check build run dev deps docker clean hooks css version
.PHONY: bootstrap setup assets test test-browser lint fmt fmt-check check build run dev deps docker clean hooks css css-check version
# Default target
.DEFAULT_GOAL := check
@@ -74,4 +74,7 @@ hooks:
@script/install-precommit
css:
tailwindcss -i static/css/input.css -o static/css/tailwind.css --minify
@script/css
css-check:
@script/css-check
+103 -56
View File
@@ -19,12 +19,14 @@ before deploying one.
### Prerequisites
- Go 1.26.1+ (the version in `go.mod`)
- Docker (for `make lint` and so for `make check`, for the browser test in
`make test-browser`, for the CI gate, and for containerized deployment)
- Docker (for `make lint` and `make css`, and so for `make check`, for the
browser test in `make test-browser`, for the CI gate, and for
containerized deployment)
golangci-lint is not a prerequisite and must not be installed on the
host: `script/bootstrap` does not install it, and `make lint` runs the
digest-pinned linter image via `Dockerfile.lint`.
digest-pinned linter image via `Dockerfile.lint`. The same holds for
tailwindcss (see [Stylesheet](#stylesheet)).
### Quick Start
@@ -36,7 +38,7 @@ cd webhooker
# Install the Go toolchain if missing, and the Go dependencies
make bootstrap
# Run all checks (test, lint, format check)
# Run all checks (test, lint, format check, stylesheet check)
make check
# Run the server from the clone. DATA_DIR defaults to
@@ -59,7 +61,7 @@ make fmt-check # Fail if gofmt would change anything (writes nothing)
make lint # Run golangci-lint in Docker (Dockerfile.lint)
make test # Run tests with race detection
make test-browser # Run the browser test in Docker (Dockerfile.browser)
make check # test + lint + fmt-check (CI gate)
make check # test + lint + fmt-check + css-check (CI gate)
make build # Build binary to bin/webhooker (version-stamped)
make version # Print the version this checkout would stamp
make run # build, then run ./bin/webhooker
@@ -67,7 +69,8 @@ make dev # go run ./cmd/webhooker
make deps # go mod download + go mod tidy
make docker # Build Docker image
make hooks # Install git pre-commit hook that runs script/precommit
make css # Regenerate static/css/tailwind.css (needs tailwindcss)
make css # Regenerate static/css/tailwind.css (tailwindcss in Docker)
make css-check # Fail if static/css/tailwind.css is stale (writes nothing)
make clean # Remove bin/
```
@@ -1292,11 +1295,11 @@ What that means for an operator:
This repository adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard: normalized scripts in `script/` are the entrypoints for the
development workflow. Eleven of the Makefile's eighteen targets are thin
shims that call them; `build`, `run`, `dev`, `deps`, `clean`, `css` and
`version` are inline commands with no script behind them, though `build`,
`run` and `dev` first run `script/assets`, and `build` and `version` both
take their value from `script/version`.
development workflow. Thirteen of the Makefile's nineteen targets are thin
shims that call them; `build`, `run`, `dev`, `deps`, `clean` and `version`
are inline commands with no script behind them, though `build`, `run` and
`dev` first run `script/assets`, and `build` and `version` both take their
value from `script/version`.
`script/test`, `make build` and `make dev` each run `script/assets`
first, which writes the ignored `static/js/alpine.min.js` (see
@@ -1318,7 +1321,11 @@ We provide:
- `script/lint` — run golangci-lint in Docker (see Linting below)
- `script/fmt` — format all code (writes)
- `script/fmt-check` — check formatting (read-only)
- `script/check` — run test, lint, and fmt-check
- `script/css` — regenerate `static/css/tailwind.css` in Docker (writes;
see [Stylesheet](#stylesheet))
- `script/css-check` — fail if `static/css/tailwind.css` differs from what
`script/css` would generate (read-only)
- `script/check` — run test, lint, fmt-check, and css-check
- `script/version` — output the version to stamp into the binary (see
[Version stamping](#version-stamping))
- `script/docker` — build the Docker image tagged via
@@ -1343,23 +1350,25 @@ markup. The CSP build runs no expressions, so every Alpine directive in
`static/js/app.js`: `x-data="collapsible"` and `@click="toggle"`, never
`x-data="{ open: false }"` or `@click="open = !open"`.
A browser test in `internal/server` loads the webhook page and the event log
under the real policy and checks that: the add entrypoint form stays hidden
until Add is clicked; for every target type, the targets section's Add shows
only a choice of type with Next and Cancel, Next shows only that type's fields
(no url field for `database` or `log`), Cancel at either step closes the form,
and saving adds the target; a refused target comes back with its form open, the
values entered and the reason, and after Cancel the next Add starts with an
empty form and no reason; the Copy button beside an entrypoint URL reads
"Copied" once clicked; an entrypoint's Edit button shows its edit form in place
of its description and hides until the form closes, Cancel hides the form and
drops what was typed, as does leaving the page and going back to it, and Save
changes the description; of the recent events on the webhook page only the
newest starts expanded, each expands and collapses, and Open leads to the
event's own page; an event in the event log expands and collapses, and so do a
delivery's attempts inside it; and at phone width the menu button opens and
closes the mobile menu. It also fails if the browser reports a console warning
or error, an uncaught exception, or anything the policy refused. `make check`
A browser test in `internal/server` loads the webhook page, its edit pages and
the event log under the real policy and checks that: the add entrypoint form
stays hidden until Add is clicked; for every target type, the targets section's
Add shows only a choice of type with Next and Cancel, Next shows only that
type's fields (no url field for `database` or `log`), Cancel at either step
closes the form, and saving adds the target; a refused target comes back with
its form open, the values entered and the reason, and after Cancel the next Add
starts with an empty form and no reason; a refused save on the target edit page
and on the webhook edit page comes back with the reason and every value
entered; the Copy button beside an entrypoint URL reads "Copied" once clicked;
an entrypoint's Edit button shows its edit form in place of its description and
hides until the form closes, Cancel hides the form and drops what was typed, as
does leaving the page and going back to it, and Save changes the description;
of the recent events on the webhook page only the newest starts expanded, each
expands and collapses, and Open leads to the event's own page; an event in the
event log expands and collapses, and so do a delivery's attempts inside it; and
at phone width the menu button opens and closes the mobile menu. It also fails
if the browser reports a console warning or error, an uncaught exception, or
anything the policy refused. `make check`
and the image build lint it but do not run it, and `make test` leaves it out
(its file is built only with the `browser` build tag). Run it with
`make test-browser` after changing `templates/` or `static/js/`: that builds
@@ -1388,6 +1397,23 @@ the `dist.integrity` hash listed at
`3p/` with it as `alpinejs-csp-<version>.tgz`, update its file name in
`script/assets`, and run `make check` and `make test-browser`.
## Stylesheet
`static/css/tailwind.css` is generated by Tailwind and committed. To change the
styles, edit the templates, `static/js/app.js`,
`internal/handlers/recent_events.go` or `static/css/input.css`, run `make css`,
and commit the regenerated file with the change. Tailwind takes classes only
from the files that `input.css` names in its `@source` lines; a class written in
any other file is not generated until that file is named there too. `make check`
and the image build fail when the committed file differs from what `make css`
generates. `static/css/style.css` is hand-written and is not generated.
`make css` runs the Tailwind standalone CLI in Docker, at the version and sha256
pinned in the Dockerfile's stylesheet stages; it is never installed on the host.
To move to a new version, change the version in both download URLs and both
sha256 sums, taken from the release's `sha256sums.txt`, then run `make css` and
commit the result.
## Rationale
Webhook integrations between services are inherently fragile. The
@@ -1580,6 +1606,13 @@ more entrypoints (receiver URLs) and one or more targets (delivery
destinations) into a logical unit. A user creates a webhook to set up
event routing.
The new webhook form can also give the webhook its first targets: an
optional HTTP target URL creates an `http` target named `HTTP`, and the
archive checkbox creates a `database` target named `Archive` whose
`expiry` is the pruning chosen beside it (never, 1h, 12h, 24h, 30d, 90d
or 365d). Both are validated as on the add target form, and the webhook
and its targets are created together or not at all.
| Field | Type | Description |
| ---------------- | ------- | ----------- |
| `id` | UUID | Primary key |
@@ -1688,8 +1721,11 @@ events should be forwarded.
own archive database
(`archive-{webhook_name}-{target_name}-{target_uuid}.db`) for long-term
retention, with an optional creation-validated expiry (default: keep
forever). No external delivery and no retries; an archive write
failure fails the delivery. See the database target section under
forever). The new webhook form, the add target form and the target edit
form all offer the same expiries: never, 1h, 12h, 24h, 30d, 90d or 365d.
The target list shows the expiry in plain units, such as "30 days". No
external delivery and no retries; an archive write failure fails the
delivery. See the database target section under
"Per-Webhook Event Databases" for the full semantics.
- **`log`** — Write the event to the application log (stdout). Useful
for debugging.
@@ -2331,13 +2367,15 @@ just delayed until the target is healthy again. A delivery already in
`retrying` keeps that status without another database write each time
the breaker turns it away.
While a target's breaker is turning its deliveries away, the target's row
on the webhook page says its deliveries are paused and until when, in UTC
and as a time from now, and each of its `retrying` deliveries shows as
waiting until then in the event log and on the event's page. That time is
when the cooldown ends; while a half-open breaker's probe delivery is in
flight, it is one whole cooldown from now, the delay the breaker gives the
other deliveries.
While a target's breaker is open, the target's row on the webhook page
says its deliveries are paused until the cooldown ends, in UTC and as a
time from now. Each of its `retrying` deliveries shows as waiting in the
event log and on the event's page, with the time it will be tried next:
the later of the cooldown's end and the end of its own backoff after its
last attempt. While the breaker is half-open, the row says instead that
deliveries are held while one delivery tests whether the target has
recovered, with no time, and the target's deliveries show their plain
status, since any of them may be the one being sent.
### Metrics
@@ -3148,10 +3186,10 @@ webhooker/
│ └── js/alpine.min.js # Alpine.js CSP build, extracted from 3p/ by make assets, not committed
├── templates/ # Go HTML templates (base, login, sources, etc.)
├── script/ # Scripts to Rule Them All entrypoints
├── Dockerfile # Three stages: lint, test+build, Alpine runtime
├── Dockerfile # Stages: lint, stylesheet, test+build, Alpine runtime
├── Dockerfile.lint # Lint-only image built by script/lint
├── Dockerfile.browser # Browser test image built by script/test-browser
├── Makefile # 11 of 18 targets shim script/; 7 are inline
├── Makefile # 13 of 19 targets shim script/; 6 are inline
├── go.mod / go.sum
└── .golangci.yml # Linter configuration
```
@@ -3465,18 +3503,26 @@ Three properties are load-bearing:
### Docker
The Dockerfile uses a three-stage build. Each stage is pinned by
digest, and the two check stages are separate images so the linter's
version is fixed independently of the compiler's:
The Dockerfile uses a multi-stage build. Each stage is pinned by
digest, and the lint and builder stages are separate images so the
linter's version is fixed independently of the compiler's:
1. **Lint stage** (`golangci/golangci-lint:v2.12.2`, Debian-based) —
installs `make`, downloads dependencies, copies the source, and runs
`make fmt-check`, then `script/assets` to extract Alpine.js from
`3p/`, then `golangci-lint config verify` and `golangci-lint run`,
both with `--network=none`.
2. **Builder stage** (`golang:1.26.1-bookworm`) — depends on the lint
stage passing (it copies a file from it), runs `make test` and
`make build` (both extract Alpine.js from `3p/` first), and finally
2. **Stylesheet stages** (`debian:bookworm-slim`, with the Tailwind
standalone CLI pinned by version and sha256, one binary per
architecture) — generate `static/css/tailwind.css` from
`static/css/input.css` and the files its `@source` lines name.
`css-check` fails when the committed file differs from the generated
one, and `make css` writes the generated file out from `css-output`
(see [Stylesheet](#stylesheet)).
3. **Builder stage** (`golang:1.26.1-bookworm`) — depends on the lint
and `css-check` stages passing (it copies a file from each), runs
`make test` and `make build` (both extract Alpine.js from `3p/`
first), and finally
rebuilds the binary with `CGO_ENABLED=1` and static linking so it
runs on musl. Both builds go through `make build`, the relink adding
its `-extldflags` via `GO_LDFLAGS`, so neither can drop the `-X` that
@@ -3484,7 +3530,7 @@ version is fixed independently of the compiler's:
given, otherwise derived from the `.git` in the context, and the
stage fails if a context with `.git` would stamp `unknown` (see
[Version stamping](#version-stamping)).
3. **Runtime stage** (`alpine:3.21`) — copies the static binary and
4. **Runtime stage** (`alpine:3.21`) — copies the static binary and
`deploy/docker-entrypoint.sh`, creates the `/var/lib/webhooker`
directory for all SQLite databases, exposes port 8080, and includes
a health check against `/.well-known/healthcheck`. It sets no
@@ -3496,18 +3542,18 @@ The lint stage invokes `golangci-lint` directly rather than `make lint`:
it is already the pinned linter image, and `make lint` builds
`Dockerfile.lint`, which would need a docker daemon inside this build.
Both check stages use Debian rather than Alpine because
The lint and builder stages use Debian rather than Alpine because
`gorm.io/driver/sqlite` pulls in `mattn/go-sqlite3`, which needs CGO
and does not compile against musl. Only the final binary is statically
linked, which is what lets it run on the Alpine runtime image.
`script/cibuild` — `docker build .` — is the CI gate: the checks run
inside the image, so a build that succeeds is a repo that is formatted,
linted, tested and compiled. `script/lint` also uses Docker
(`Dockerfile.lint`, see Linting above), so `make lint` and `make check`
run the same pinned linter version the gate does; of the steps
`make check` runs, only `script/test` and `script/fmt-check` run on the
host.
linted, tested and compiled, with a current stylesheet. `script/lint`
also uses Docker (`Dockerfile.lint`, see Linting above), so `make lint`
and `make check` run the same pinned linter version the gate does; of
the steps `make check` runs, only `script/test` and `script/fmt-check`
run on the host.
#### CI gate honesty
@@ -3517,9 +3563,10 @@ check meaningless. The `check` workflow therefore writes
`.ci-fingerprint` into the build context before building. Its value is
the hash of the commit being checked, so every commit, docs-only ones
and a squash merge whose tree matches an already-built branch included,
gets a new fingerprint, invalidates the `COPY . .` layer of both check
stages, and really runs `make fmt-check`, `golangci-lint`, `make test`,
and `make build`. A run that reports success ran them.
gets a new fingerprint, invalidates the `COPY . .` layer of every check
stage, and really runs `make fmt-check`, `golangci-lint`, the stylesheet
check, `make test`, and `make build`. A run that reports success ran
them.
The module download layer sits above `COPY . .` and stays cached.
+14
View File
@@ -102,6 +102,20 @@ func (cb *CircuitBreaker) CooldownRemaining() time.Duration {
return remaining
}
// StateAndCooldown returns the circuit state and, while the circuit is
// open, what is left of the cooldown, or zero once that has passed.
// Both are read under one lock, so they always agree.
func (cb *CircuitBreaker) StateAndCooldown() (CircuitState, time.Duration) {
cb.mu.Lock()
defer cb.mu.Unlock()
if cb.state != CircuitOpen {
return cb.state, 0
}
return cb.state, max(cb.cooldown-time.Since(cb.lastFailure), 0)
}
// RecordSuccess records a successful delivery and resets
// the circuit breaker to closed state.
func (cb *CircuitBreaker) RecordSuccess() {
+14 -10
View File
@@ -149,7 +149,7 @@ type Archives interface {
// it keeps the handlers free of the engine's internals and is
// trivially faked in tests.
type CircuitBreakers interface {
CooldownRemaining(targetID string) time.Duration
StateAndCooldown(targetID string) (CircuitState, time.Duration)
}
// EngineParams are the fx dependencies for the delivery
@@ -195,7 +195,7 @@ type Engine struct {
// targets maps each target type to its implementation.
targets map[database.TargetType]Target
// httpTarget and slackTarget are retained so CooldownRemaining
// httpTarget and slackTarget are retained so StateAndCooldown
// can read their circuit breakers, and so tests can reach the
// HTTP target's shared client.
httpTarget *httpTarget
@@ -311,12 +311,16 @@ func (e *Engine) Rename(
return e.dbTarget.rename(targetID, webhookName, targetName)
}
// CooldownRemaining implements CircuitBreakers. It is
// CircuitBreaker.CooldownRemaining for the target's breaker: how long
// the breaker will keep turning the target's deliveries away, and
// zero when it lets them through. A target with no breaker gets zero,
// and reading never creates one.
func (e *Engine) CooldownRemaining(targetID string) time.Duration {
// StateAndCooldown implements CircuitBreakers. It is
// CircuitBreaker.StateAndCooldown for the target's breaker. While the
// breaker is open, the pages show the target's deliveries as paused
// until its cooldown ends; while it is half-open, they show them as
// held, with no time, while one delivery tests whether the target has
// recovered. A target with no breaker reads as closed, and reading
// never creates one.
func (e *Engine) StateAndCooldown(
targetID string,
) (CircuitState, time.Duration) {
for _, core := range []*httpCore{
e.httpTarget.httpCore, e.slackTarget.httpCore,
} {
@@ -324,11 +328,11 @@ func (e *Engine) CooldownRemaining(targetID string) time.Duration {
if ok {
cb, _ := val.(*CircuitBreaker)
return cb.CooldownRemaining()
return cb.StateAndCooldown()
}
}
return 0
return CircuitClosed, 0
}
// ScheduleRetry schedules a task to be re-enqueued onto the
+31 -11
View File
@@ -1018,11 +1018,11 @@ func TestGetCircuitBreaker_CreatesOnDemand(t *testing.T) {
)
}
// TestCooldownRemaining_ReadsHTTPAndSlackBreakers proves the engine
// reads the cooldown of an http or a slack target's circuit breaker
// while it is open, and zero for a target with no breaker and once the
// breaker closes.
func TestCooldownRemaining_ReadsHTTPAndSlackBreakers(t *testing.T) {
// TestStateAndCooldown_ReadsHTTPAndSlackBreakers proves the engine
// reads the state of an http or a slack target's circuit breaker, with
// what is left of its cooldown while it is open, and no cooldown while
// it is half-open, once it closes, or for a target with no breaker.
func TestStateAndCooldown_ReadsHTTPAndSlackBreakers(t *testing.T) {
t.Parallel()
e := testEngine(t, 1)
@@ -1030,7 +1030,9 @@ func TestCooldownRemaining_ReadsHTTPAndSlackBreakers(t *testing.T) {
httpID := uuid.New().String()
slackID := uuid.New().String()
assert.Zero(t, e.CooldownRemaining(httpID), "no breaker")
state, cooldown := e.StateAndCooldown(httpID)
assert.Equal(t, delivery.CircuitClosed, state, "no breaker")
assert.Zero(t, cooldown, "no breaker")
httpCB := delivery.NewTestCircuitBreaker(1, time.Hour)
e.ExportSetCircuitBreaker(httpID, httpCB)
@@ -1042,16 +1044,34 @@ func TestCooldownRemaining_ReadsHTTPAndSlackBreakers(t *testing.T) {
slackCB.RecordFailure()
for _, id := range []string{httpID, slackID} {
remaining := e.CooldownRemaining(id)
assert.Greater(t, remaining, 59*time.Minute)
assert.LessOrEqual(t, remaining, time.Hour)
state, cooldown := e.StateAndCooldown(id)
assert.Equal(t, delivery.CircuitOpen, state)
assert.Greater(t, cooldown, 59*time.Minute)
assert.LessOrEqual(t, cooldown, time.Hour)
}
httpCB.RecordSuccess()
slackCB.RecordSuccess()
assert.Zero(t, e.CooldownRemaining(httpID), "closed http breaker")
assert.Zero(t, e.CooldownRemaining(slackID), "closed slack breaker")
for _, id := range []string{httpID, slackID} {
state, cooldown := e.StateAndCooldown(id)
assert.Equal(t, delivery.CircuitClosed, state, "closed")
assert.Zero(t, cooldown, "closed")
}
// A breaker with no cooldown goes half-open on the first Allow
// after it trips, letting that one delivery through to test the
// target.
halfOpenID := uuid.New().String()
halfOpenCB := delivery.NewTestCircuitBreaker(1, 0)
e.ExportSetCircuitBreaker(halfOpenID, halfOpenCB)
halfOpenCB.RecordFailure()
require.True(t, halfOpenCB.Allow())
state, cooldown = e.StateAndCooldown(halfOpenID)
assert.Equal(t, delivery.CircuitHalfOpen, state)
assert.Zero(t, cooldown, "half-open")
}
func TestParseHTTPConfig_Valid(t *testing.T) {
+3 -3
View File
@@ -86,9 +86,9 @@ func NewTargetConfigForm(
}
// databaseConfigForm parses an archive target's optional expiry.
// An absent or empty configuration is the keep-forever default and
// yields an empty field, so re-saving the form unchanged stores the
// same empty configuration it started with. An expiry that is set
// An absent, empty or never expiry yields an empty expiry, on which
// the edit form starts at never; saving it unchanged stores never,
// which means the same as an empty expiry. An expiry that is set
// but not a valid duration is an error, not a blank field.
func databaseConfigForm(
configJSON string,
+69 -44
View File
@@ -1,9 +1,9 @@
package delivery
import (
"encoding/json"
"fmt"
"strconv"
"time"
"sneak.berlin/go/webhooker/internal/database"
)
@@ -97,7 +97,7 @@ func targetConfigFields(
) []ConfigField {
switch t.Type {
case database.TargetTypeSlack:
return slackConfigFields(t.Config)
return slackConfigFields(t)
case database.TargetTypeHTTP:
return httpConfigFields(t)
case database.TargetTypeDatabase:
@@ -119,10 +119,11 @@ func unavailableConfigFields() []ConfigField {
}}
}
// slackConfigFields describes a Slack target. Only the masked
// webhook URL is shown; the full URL is the credential.
func slackConfigFields(configJSON string) []ConfigField {
cfg, err := parseSlackConfig(configJSON)
// slackConfigFields describes a Slack target: its masked
// webhook URL and its retry count. Only the masked URL is
// shown; the full URL is the credential.
func slackConfigFields(t *database.Target) []ConfigField {
cfg, err := parseSlackConfig(t.Config)
if err != nil {
return unavailableConfigFields()
}
@@ -130,7 +131,7 @@ func slackConfigFields(configJSON string) []ConfigField {
return []ConfigField{{
Label: "Webhook URL",
Value: cfg.MaskedWebhookURL(),
}}
}, maxRetriesField(t)}
}
// httpConfigFields describes an HTTP target: its destination
@@ -170,21 +171,7 @@ func httpConfigFields(t *database.Target) []ConfigField {
})
}
return append(fields, retryFields(t)...)
}
// retryFields describes a target's retry settings, which live
// on the target row rather than in its configuration blob.
func retryFields(t *database.Target) []ConfigField {
retries := strconv.Itoa(t.MaxRetries)
if t.MaxRetries == 0 {
retries += " (fire-and-forget)"
}
fields := []ConfigField{{
Label: "Max Retries",
Value: retries,
}}
fields = append(fields, maxRetriesField(t))
if t.MaxQueueSize > 0 {
fields = append(fields, ConfigField{
@@ -196,37 +183,75 @@ func retryFields(t *database.Target) []ConfigField {
return fields
}
// databaseConfigFields describes an archive target. Its
// configuration is optional, and an absent or empty expiry
// means the archive is kept forever. An expiry that is set
// but not a valid duration is reported as unavailable rather
// than echoed back.
// maxRetriesField describes a target's retry count, which lives
// on the target row rather than in its configuration blob.
func maxRetriesField(t *database.Target) ConfigField {
retries := strconv.Itoa(t.MaxRetries)
if t.MaxRetries == 0 {
retries += " (fire-and-forget)"
}
return ConfigField{
Label: "Max Retries",
Value: retries,
}
}
// databaseConfigFields describes an archive target by its
// expiry in plain units, such as "30 days", or "never" when
// the archive is kept forever. An expiry that is set but not
// a valid duration is reported as unavailable rather than
// echoed back.
func databaseConfigFields(configJSON string) []ConfigField {
expiry := archiveExpiryNever
expiry, err := parseArchiveExpiry(configJSON)
if err != nil {
return unavailableConfigFields()
}
if configJSON != "" {
var cfg databaseTargetConfig
err := json.Unmarshal([]byte(configJSON), &cfg)
if err != nil {
return unavailableConfigFields()
}
if cfg.Expiry != "" {
if ValidateArchiveExpiry(cfg.Expiry) != nil {
return unavailableConfigFields()
}
expiry = cfg.Expiry
}
value := archiveExpiryNever
if expiry > 0 {
value = plainDuration(expiry)
}
return []ConfigField{{
Label: "Archive Expiry",
Value: expiry,
Value: value,
}}
}
// plainDuration writes a positive duration as a count of the
// largest whole unit it divides into: "30 days", "12 hours",
// "1 minute". A duration with a fraction of a second is
// written as Go writes it.
func plainDuration(d time.Duration) string {
const day = 24 * time.Hour
units := []struct {
size time.Duration
name string
}{
{day, "day"},
{time.Hour, "hour"},
{time.Minute, "minute"},
{time.Second, "second"},
}
for _, unit := range units {
if d%unit.size != 0 {
continue
}
count := int64(d / unit.size)
if count == 1 {
return "1 " + unit.name
}
return fmt.Sprintf("%d %ss", count, unit.name)
}
return d.String()
}
// MaskedWebhookURL returns the Slack webhook URL reduced to
// its scheme and host, with the path, query and any userinfo
// elided. The path segments are the credential, so none of
+40 -10
View File
@@ -32,6 +32,7 @@ const (
viewMaskedOrigin = viewExampleOrigin + "/..."
viewUnavailable = "(unavailable)"
viewExpiryNever = "never"
viewMaxRetries = "Max Retries"
)
func TestMaskedWebhookURL(t *testing.T) {
@@ -157,9 +158,7 @@ func TestNewTargetViews_DeletedTarget(t *testing.T) {
t, slackTargetName+" (deleted)", view.DisplayName(),
)
assert.Equal(
t,
map[string]string{"Webhook URL": slackMaskedURL},
fieldMap(view.Config),
t, viewFor(t, slackTarget()).Config, view.Config,
)
}
@@ -189,7 +188,32 @@ func TestNewTargetViews_Slack(t *testing.T) {
assert.Equal(
t,
map[string]string{"Webhook URL": slackMaskedURL},
map[string]string{
"Webhook URL": slackMaskedURL,
viewMaxRetries: "0 (fire-and-forget)",
},
fieldMap(view.Config),
)
}
// TestNewTargetViews_SlackRetries proves a Slack target shows
// its retry count the same way an HTTP target does, and no
// queue size even when one is stored: delivery never reads it.
func TestNewTargetViews_SlackRetries(t *testing.T) {
t.Parallel()
target := slackTarget()
target.MaxRetries = 2
target.MaxQueueSize = 100
view := viewFor(t, target)
assert.Equal(
t,
map[string]string{
"Webhook URL": slackMaskedURL,
viewMaxRetries: "2",
},
fieldMap(view.Config),
)
}
@@ -214,7 +238,7 @@ func TestNewTargetViews_HTTP(t *testing.T) {
"Destination URL": viewMaskedOrigin,
"Timeout": "30s",
"Headers": "1 configured",
"Max Retries": "5",
viewMaxRetries: "5",
"Max Queue Size": "100",
},
fields,
@@ -238,7 +262,7 @@ func TestNewTargetViews_HTTPFireAndForget(t *testing.T) {
t,
map[string]string{
"Destination URL": viewMaskedOrigin,
"Max Retries": "0 (fire-and-forget)",
viewMaxRetries: "0 (fire-and-forget)",
},
fieldMap(view.Config),
)
@@ -281,14 +305,20 @@ func TestNewTargetViews_Database(t *testing.T) {
}{
"empty config": {config: "", want: viewExpiryNever},
"empty expiry": {config: `{}`, want: viewExpiryNever},
"explicit": {
config: `{"expiry":"720h"}`,
want: "720h",
},
"never literal": {
config: `{"expiry":"` + viewExpiryNever + `"}`,
want: viewExpiryNever,
},
"1h": {config: `{"expiry":"1h"}`, want: "1 hour"},
"12h": {config: `{"expiry":"12h"}`, want: "12 hours"},
"24h": {config: `{"expiry":"24h"}`, want: "1 day"},
"720h": {config: `{"expiry":"720h"}`, want: "30 days"},
"2160h": {config: `{"expiry":"2160h"}`, want: "90 days"},
"8760h": {config: `{"expiry":"8760h"}`, want: "365 days"},
"36h": {config: `{"expiry":"36h"}`, want: "36 hours"},
"1h30m": {config: `{"expiry":"1h30m"}`, want: "90 minutes"},
"45s": {config: `{"expiry":"45s"}`, want: "45 seconds"},
"1.5s": {config: `{"expiry":"1.5s"}`, want: "1.5s"},
}
for name, tc := range tests {
+3 -2
View File
@@ -216,8 +216,9 @@ func TestNewTargetConfigForm(t *testing.T) {
assert.Empty(t, form.URL)
}
// A keep-forever archive target must pre-fill as an empty field, so
// saving the form back unchanged stores the same empty config.
// A keep-forever archive target yields an empty expiry, so the edit
// form starts on never; saving it unchanged stores never, which means
// the same as an empty expiry.
func TestNewTargetConfigForm_DatabaseNeverIsBlank(t *testing.T) {
t.Parallel()
+6 -4
View File
@@ -234,7 +234,7 @@ func (c *httpCore) handleRetry(
database.DeliveryStatusRetrying,
)
backoff := calcBackoff(attemptNum)
backoff := Backoff(attemptNum)
retryTask := *task
retryTask.AttemptNum = attemptNum + 1
@@ -301,7 +301,7 @@ func (c *httpCore) remainingBackoff(
return 0
}
backoff := calcBackoff(attemptNum)
backoff := Backoff(attemptNum)
elapsed := time.Since(lastResult.CreatedAt)
remaining := backoff - elapsed
@@ -326,12 +326,14 @@ func (c *httpCore) backoffElapsed(
return true
}
backoff := calcBackoff(attemptNum)
backoff := Backoff(attemptNum)
return time.Since(lastResult.CreatedAt) >= backoff
}
func calcBackoff(attemptNum int) time.Duration {
// Backoff is how long an http or slack target with retries waits after
// a delivery's failed attempt attemptNum before trying it again.
func Backoff(attemptNum int) time.Duration {
shift := max(attemptNum-1, 0)
shift = min(shift, maxBackoffShift)
+58
View File
@@ -0,0 +1,58 @@
package handlers
const (
// archiveExpiryNever is the archive expiry that keeps archived
// events forever. A stored empty expiry means the same.
archiveExpiryNever = "never"
// tmplKeyArchiveExpiryChoices is the template data key for the
// entries of a page's archive expiry select.
tmplKeyArchiveExpiryChoices = "ArchiveExpiryChoices"
)
// archiveExpiryChoice is one entry of a database target's archive
// expiry select: the expiry stored, the label shown, and whether the
// select starts on it.
type archiveExpiryChoice struct {
Value string
Label string
Selected bool
}
// archiveExpiryChoices lists the archive expiries offered by the new
// webhook page, the add target form and the target edit form.
func archiveExpiryChoices() []archiveExpiryChoice {
return []archiveExpiryChoice{
{Value: archiveExpiryNever, Label: archiveExpiryNever},
{Value: "1h", Label: "1h"},
{Value: "12h", Label: "12h"},
{Value: "24h", Label: "24h"},
{Value: "720h", Label: "30d"},
{Value: "2160h", Label: "90d"},
{Value: "8760h", Label: "365d"},
}
}
// archiveExpiryOptions returns the choices with expiry selected; an
// empty expiry selects never. An expiry that is not one of the
// choices comes first as its own selected entry, so saving the form
// unchanged keeps it.
func archiveExpiryOptions(expiry string) []archiveExpiryChoice {
if expiry == "" {
expiry = archiveExpiryNever
}
options := archiveExpiryChoices()
for i := range options {
if options[i].Value == expiry {
options[i].Selected = true
return options
}
}
own := archiveExpiryChoice{Value: expiry, Label: expiry, Selected: true}
return append([]archiveExpiryChoice{own}, options...)
}
+166
View File
@@ -0,0 +1,166 @@
package handlers_test
import (
"net/http"
"net/http/httptest"
"net/url"
"regexp"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database"
)
// expiryNever is the archive expiry that keeps archived events
// forever.
const expiryNever = "never"
// matched returns what the one group of pattern matched in page, at
// each match.
func matched(pattern, page string) []string {
matches := regexp.MustCompile(pattern).FindAllStringSubmatch(page, -1)
groups := make([]string, 0, len(matches))
for _, m := range matches {
groups = append(groups, m[1])
}
return groups
}
// expiryShown returns the archive expiries the webhook page's target
// list shows.
func expiryShown(
t *testing.T, env *sourceTestEnv, webhookID string,
) []string {
t.Helper()
w := httptest.NewRecorder()
env.handlers.HandleSourceDetail().ServeHTTP(w, getRequest(
t, "/hook/"+webhookID, env.cookies,
map[string]string{sourceIDParam: webhookID},
))
require.Equal(t, http.StatusOK, w.Code)
return matched(
`Archive Expiry:</span>\s*<span>([^<]*)</span>`, w.Body.String(),
)
}
// expirySelected returns the target edit page and the expiries its
// select starts on.
func expirySelected(
t *testing.T, env *sourceTestEnv, webhookID, targetID string,
) (string, []string) {
t.Helper()
w := serveTarget(
env, http.MethodGet,
"/hook/"+webhookID+"/targets/"+targetID+"/edit", nil,
)
require.Equal(t, http.StatusOK, w.Code)
page := w.Body.String()
return page, matched(`<option value="([^"]*)" selected>`, page)
}
// TestArchiveExpiryChoices adds a database target with each archive
// expiry the forms offer, and checks that it is stored as chosen,
// shown in plain units in the target list, and that the target edit
// form starts on it.
func TestArchiveExpiryChoices(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
choices := []struct{ value, shown string }{
{expiryNever, expiryNever},
{"1h", "1 hour"},
{"12h", "12 hours"},
{"24h", "1 day"},
{"720h", "30 days"},
{"2160h", "90 days"},
{"8760h", "365 days"},
}
for _, choice := range choices {
t.Run(choice.value, func(t *testing.T) {
t.Parallel()
webhook := seedWebhookWithRetention(t, env.db, 30)
form := url.Values{}
form.Set("name", "archive")
form.Set("type", string(database.TargetTypeDatabase))
form.Set("expiry", choice.value)
w := serveTarget(
env, http.MethodPost, "/hook/"+webhook.ID+"/targets", form,
)
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1)
assert.JSONEq(
t, `{"expiry":"`+choice.value+`"}`, targets[0].Config,
)
assert.Equal(
t, []string{choice.shown},
expiryShown(t, env, webhook.ID),
)
_, selected := expirySelected(t, env, webhook.ID, targets[0].ID)
assert.Equal(t, []string{choice.value}, selected)
})
}
}
// TestArchiveExpiryEditStartsOnStoredValue checks the edit form of a
// database target whose stored expiry is empty, which selects never,
// and of one whose expiry is not one of the choices, which is listed
// first as its own selected entry and saved unchanged.
func TestArchiveExpiryEditStartsOnStoredValue(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
webhook := seedWebhookWithRetention(t, env.db, 30)
empty := seedConfiguredTarget(
t, env.db, webhook.ID, database.TargetTypeDatabase, "",
)
_, selected := expirySelected(t, env, webhook.ID, empty.ID)
assert.Equal(t, []string{expiryNever}, selected)
webhook = seedWebhookWithRetention(t, env.db, 30)
unlisted := seedConfiguredTarget(
t, env.db, webhook.ID, database.TargetTypeDatabase,
`{"expiry":"36h"}`,
)
assert.Equal(t, []string{"36 hours"}, expiryShown(t, env, webhook.ID))
page, selected := expirySelected(t, env, webhook.ID, unlisted.ID)
assert.Equal(t, []string{"36h"}, selected)
assert.Regexp(
t,
`<select id="expiry" name="expiry" class="input">\s*`+
`<option value="36h" selected>36h</option>\s*`+
`<option value="never">never</option>`,
page,
)
assert.Contains(t, page, `<option value="8760h">365d</option>`)
form := url.Values{}
form.Set("name", unlisted.Name)
form.Set("expiry", "36h")
w := submitTargetEdit(env, webhook.ID, unlisted.ID, form)
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
assert.JSONEq(
t, `{"expiry":"36h"}`, storedTarget(t, env, unlisted.ID).Config,
)
}
+5 -2
View File
@@ -1,6 +1,8 @@
package handlers
import (
"time"
"sneak.berlin/go/webhooker/internal/delivery"
)
@@ -24,8 +26,8 @@ const maxRenderedResponseBytes = 4096
// bytes rather than characters, and they make SQLite do the
// cut, so an oversized stored response never becomes a Go
// string at all.
const deliveryResultColumns = "delivery_id, attempt_num, success, " +
"status_code, error, duration, " +
const deliveryResultColumns = "delivery_id, attempt_num, created_at, " +
"success, status_code, error, duration, " +
"substr(cast(response_body as blob), 1, ?) AS response_body, " +
"length(cast(response_body as blob)) AS response_bytes"
@@ -100,6 +102,7 @@ func (v DeliveryResultView) HasStatusCode() bool {
type deliveryResultRow struct {
DeliveryID string
AttemptNum int
CreatedAt time.Time
Success bool
StatusCode int
Error string
+18 -25
View File
@@ -182,45 +182,38 @@ func (r *recordingArchives) Renames() []archiveRename {
return out
}
// testCircuitBreakers is a delivery.CircuitBreakers over real circuit
// breakers, one per target a test asks for, so a test can trip a
// target's breaker with RecordFailure and close it with RecordSuccess.
// testCircuitBreakers is a delivery.CircuitBreakers that reports, for
// each target, the circuit state and cooldown a test gave it with Set,
// and a closed breaker for any other target.
type testCircuitBreakers struct {
mu sync.Mutex
breakers map[string]*delivery.CircuitBreaker
mu sync.Mutex
states map[string]delivery.CircuitState
cooldowns map[string]time.Duration
}
// Breaker returns the target's circuit breaker, making it on first
// use.
func (b *testCircuitBreakers) Breaker(
targetID string,
) *delivery.CircuitBreaker {
// Set makes the target's breaker read as state, with cooldown left.
func (b *testCircuitBreakers) Set(
targetID string, state delivery.CircuitState, cooldown time.Duration,
) {
b.mu.Lock()
defer b.mu.Unlock()
if b.breakers == nil {
b.breakers = map[string]*delivery.CircuitBreaker{}
if b.states == nil {
b.states = map[string]delivery.CircuitState{}
b.cooldowns = map[string]time.Duration{}
}
if b.breakers[targetID] == nil {
b.breakers[targetID] = delivery.NewCircuitBreaker()
}
return b.breakers[targetID]
b.states[targetID] = state
b.cooldowns[targetID] = cooldown
}
func (b *testCircuitBreakers) CooldownRemaining(
func (b *testCircuitBreakers) StateAndCooldown(
targetID string,
) time.Duration {
) (delivery.CircuitState, time.Duration) {
b.mu.Lock()
defer b.mu.Unlock()
cb := b.breakers[targetID]
if cb == nil {
return 0
}
return cb.CooldownRemaining()
return b.states[targetID], b.cooldowns[targetID]
}
// newTestApp returns an app whose RequireStart fails the test when
@@ -0,0 +1,207 @@
package handlers_test
import (
"errors"
"html"
"net/http"
"net/http/httptest"
"net/url"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// submitCreateForm posts the new webhook form and returns the
// recorder.
func submitCreateForm(
env *sourceTestEnv, form url.Values,
) *httptest.ResponseRecorder {
req := formRequest("/hooks/new", env.cookies, form, nil)
w := httptest.NewRecorder()
env.handlers.HandleSourceCreateSubmit().ServeHTTP(w, req)
return w
}
// assertNothingCreated checks that the main database holds no webhook,
// entrypoint or target.
func assertNothingCreated(t *testing.T, db *database.Database) {
t.Helper()
for _, model := range []any{
&database.Webhook{}, &database.Entrypoint{}, &database.Target{},
} {
var count int64
require.NoError(t, db.DB().Model(model).Count(&count).Error)
assert.Zerof(t, count, "%T rows were created", model)
}
}
// TestHandleSourceCreateSubmit_CreatesRequestedTargets submits the new
// webhook form with the HTTP target URL filled in or empty, and with
// the archive checkbox off or on with each pruning choice. The webhook
// gets an HTTP target only for a URL and a database target only for a
// checked archive. The pruning choice is always submitted, as the
// browser submits it while it is hidden, and is ignored when archive
// is off.
func TestHandleSourceCreateSubmit_CreatesRequestedTargets(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
// Each value the archive pruning choice submits, after an empty
// one that stands for the archive checkbox left off.
expiries := []string{
"", "never", "1h", "12h", "24h", "720h", "2160h", "8760h",
}
for _, httpURL := range []string{"", editOriginalURL} {
for _, expiry := range expiries {
name := "url=" + httpURL + " archive=" + expiry
t.Run(name, func(t *testing.T) {
t.Parallel()
form := url.Values{}
form.Set("name", name)
form.Set("http_url", httpURL)
form.Set("archive_expiry", "720h")
if expiry != "" {
form.Set("archive", "on")
form.Set("archive_expiry", expiry)
}
w := submitCreateForm(env, form)
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
var webhook database.Webhook
require.NoError(t, env.db.DB().
Where("name = ?", name).First(&webhook).Error)
byType := map[database.TargetType]database.Target{}
for _, target := range targetsForWebhook(t, env.db, webhook.ID) {
byType[target.Type] = target
}
wantCount := 0
if httpURL != "" {
wantCount++
assert.Equal(t, "HTTP", byType[database.TargetTypeHTTP].Name)
assert.JSONEq(t, `{"url":"`+httpURL+`"}`,
byType[database.TargetTypeHTTP].Config)
}
if expiry != "" {
wantCount++
assert.Equal(t, "Archive",
byType[database.TargetTypeDatabase].Name)
assert.JSONEq(t, `{"expiry":"`+expiry+`"}`,
byType[database.TargetTypeDatabase].Config)
}
assert.Len(t, byType, wantCount)
})
}
}
}
// TestHandleSourceCreateSubmit_RefusedFormKeepsEveryValue refuses the
// new webhook form for an invalid HTTP target URL and for an invalid
// retention, each with archive on. Nothing is created, and the form
// comes back with the reason and every value entered: name,
// description, retention, URL, the checked archive box and the pruning
// choice.
func TestHandleSourceCreateSubmit_RefusedFormKeepsEveryValue(
t *testing.T,
) {
t.Parallel()
const badURL = "Invalid target URL"
cases := []struct {
name string
retention string
httpURL string
reason string
}{
{"blocked url", "7", editBlockedURL, badURL},
{"unsupported scheme", "7", "ftp://93.184.216.34/hook", badURL},
{"bad retention", "-5", editOriginalURL, "Retention must be"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
form := url.Values{}
form.Set("name", "kept name")
form.Set("description", "kept description")
form.Set("retention_days", tc.retention)
form.Set("http_url", tc.httpURL)
form.Set("archive", "on")
form.Set("archive_expiry", "2160h")
w := submitCreateForm(env, form)
require.Equal(t, http.StatusBadRequest, w.Code)
page := w.Body.String()
assert.Contains(t, page, tc.reason)
assert.Contains(t, page, `value="kept name"`)
assert.Contains(t, page, `>kept description</textarea>`)
assert.Contains(t, page, `value="`+tc.retention+`"`)
assert.Contains(t, page,
`value="`+html.EscapeString(tc.httpURL)+`"`)
assert.Contains(t, page, `name="archive" value="on" checked`)
assert.Contains(t, page, `x-data="collapsible" data-open`)
assert.Contains(t, page, `<option value="2160h" selected>`)
assertNothingCreated(t, env.db)
})
}
}
// errInjectedTargetCreate is the failure a test makes the insert of a
// target report.
var errInjectedTargetCreate = errors.New("injected target create failure")
// TestHandleSourceCreateSubmit_FailedTargetInsertCreatesNothing makes
// inserting a target fail after the webhook and its entrypoint were
// inserted, and checks that neither is left behind.
func TestHandleSourceCreateSubmit_FailedTargetInsertCreatesNothing(
t *testing.T,
) {
t.Parallel()
env := setupSourceTest(t)
require.NoError(t, env.db.DB().Callback().Create().
Before("gorm:create").
Register("test:fail_target_create", func(tx *gorm.DB) {
if tx.Statement.Table == "targets" {
_ = tx.AddError(errInjectedTargetCreate)
}
}),
)
form := url.Values{}
form.Set("name", "rolled back")
form.Set("archive", "on")
form.Set("archive_expiry", "never")
w := submitCreateForm(env, form)
require.Equal(t, http.StatusInternalServerError, w.Code)
assertNothingCreated(t, env.db)
}
+1 -1
View File
@@ -234,7 +234,7 @@ func TestHandleSourceDetail_RendersNamedTargetFields(
assert.NotContains(t, body, "sekrit")
assert.Contains(t, body, "Archive Expiry")
assert.Contains(t, body, "720h")
assert.Contains(t, body, "30 days")
// An unknown type gets the neutral placeholder, never the
// stored blob.
+245 -126
View File
@@ -111,8 +111,7 @@ type DeliveryView struct {
AttemptsOmitted int
// Paused is set while the delivery is retrying and its
// target's circuit breaker is turning deliveries away, and
// nil otherwise.
// target's circuit breaker is open, and nil otherwise.
Paused *PausedView
}
@@ -281,28 +280,45 @@ func (h *Handlers) HandleSourceCreate() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
h.renderTemplate(
w, r, "sources_new.html",
newSourceFormData("", "", ""),
newSourceFormData("", sourceFormInput{
RetentionDays: strconv.Itoa(
database.DefaultRetentionDays,
),
}),
)
}
}
// sourceFormInput carries the raw values of the new webhook form. A
// refused submission is shown again from it, so every value entered
// comes back, retention included.
type sourceFormInput struct {
Name string
Description string
RetentionDays string
// HTTPURL, when not empty, asks for an HTTP target with this
// destination.
HTTPURL string
// Archive asks for a database (archive) target, whose rows expire
// after ArchiveExpiry.
Archive bool
ArchiveExpiry string
}
// newSourceFormData builds the template data for the webhook creation
// form.
//
// It carries the retention default so the pre-filled value comes from
// database.DefaultRetentionDays rather than being a third hardcoded
// copy of the same policy, and it carries the submitted name and
// description so that re-rendering the form after a validation failure
// gives the user their input back instead of a blank form. The edit
// form already behaves that way; create now matches it.
// form. It carries the retention default, which the form's help text
// names, from database.DefaultRetentionDays rather than a hardcoded
// copy of the same policy.
func newSourceFormData(
errMsg, name, description string,
errMsg string, in sourceFormInput,
) map[string]any {
return map[string]any{
tmplKeyError: errMsg,
"Name": name,
"Description": description,
"Form": in,
"DefaultRetentionDays": database.DefaultRetentionDays,
tmplKeyArchiveExpiryChoices: archiveExpiryOptions(
in.ArchiveExpiry,
),
}
}
@@ -328,57 +344,112 @@ func (h *Handlers) HandleSourceCreateSubmit() http.HandlerFunc {
return
}
name := r.PostFormValue("name")
description := r.PostFormValue("description")
retentionStr := r.PostFormValue("retention_days")
in := sourceFormInput{
Name: r.PostFormValue("name"),
Description: r.PostFormValue("description"),
RetentionDays: r.PostFormValue("retention_days"),
HTTPURL: r.PostFormValue("http_url"),
Archive: r.PostFormValue("archive") != "",
ArchiveExpiry: r.PostFormValue("archive_expiry"),
}
if name == "" {
refuse := func(errMsg string) {
h.renderTemplateStatus(
w, r, "sources_new.html",
newSourceFormData(
"Name is required", name, description,
),
newSourceFormData(errMsg, in),
http.StatusBadRequest,
)
}
if in.Name == "" {
refuse("Name is required")
return
}
retentionDays, errMsg := parseRetentionDays(
retentionStr, database.DefaultRetentionDays,
in.RetentionDays, database.DefaultRetentionDays,
)
if errMsg != "" {
h.renderTemplateStatus(
w, r, "sources_new.html",
newSourceFormData(errMsg, name, description),
http.StatusBadRequest,
)
refuse(errMsg)
return
}
h.createWebhookWithEntrypoint(
w, r, userID, name, description, retentionDays,
)
targets, errMsg, err := h.newWebhookTargets(r.Context(), in)
if err != nil {
h.serverError(w, r, "failed to encode target config", err)
return
}
if errMsg != "" {
refuse(errMsg)
return
}
h.createWebhookWithEntrypoint(w, r, &database.Webhook{
UserID: userID,
Name: in.Name,
Description: in.Description,
RetentionDays: retentionDays,
}, targets)
}
}
// createWebhookWithEntrypoint creates a webhook and its default
// entrypoint in a transaction.
// newWebhookTargets validates the targets the new webhook form asks
// for and returns the rows to create with the webhook, or the message
// the form shows for the first one it refuses. A filled-in HTTP URL
// asks for an HTTP target named "HTTP", and the archive checkbox for a
// database target named "Archive". Each goes through newTarget, as on
// the webhook page's add target form. The rows have no WebhookID yet:
// the webhook has no ID until it is created.
func (h *Handlers) newWebhookTargets(
ctx context.Context,
in sourceFormInput,
) ([]*database.Target, string, error) {
var requested []targetFormInput
if in.HTTPURL != "" {
requested = append(requested, targetFormInput{
Name: "HTTP",
Type: database.TargetTypeHTTP,
URL: in.HTTPURL,
})
}
if in.Archive {
requested = append(requested, targetFormInput{
Name: "Archive",
Type: database.TargetTypeDatabase,
Expiry: in.ArchiveExpiry,
})
}
targets := make([]*database.Target, 0, len(requested))
for _, form := range requested {
target, errMsg, err := h.newTarget(ctx, "", form)
if err != nil || errMsg != "" {
return nil, errMsg, err
}
targets = append(targets, target)
}
return targets, "", nil
}
// createWebhookWithEntrypoint creates a webhook, its default
// entrypoint and the given targets in a transaction.
func (h *Handlers) createWebhookWithEntrypoint(
w http.ResponseWriter,
r *http.Request,
userID, name, description string,
retentionDays int,
webhook *database.Webhook,
targets []*database.Target,
) {
webhook := &database.Webhook{
UserID: userID,
Name: name,
Description: description,
RetentionDays: retentionDays,
}
err := h.commitWebhook(webhook)
err := h.commitWebhook(webhook, targets)
if err != nil {
h.serverError(w, r, "failed to create webhook", err)
@@ -395,7 +466,7 @@ func (h *Handlers) createWebhookWithEntrypoint(
h.log.Info("webhook created",
"webhook_id", webhook.ID,
"name", name, "user_id", userID,
"name", webhook.Name, "user_id", webhook.UserID,
)
http.Redirect(
@@ -404,10 +475,12 @@ func (h *Handlers) createWebhookWithEntrypoint(
)
}
// commitWebhook creates a webhook and default entrypoint in
// a transaction. Returns an error on failure (rolls back).
// commitWebhook creates a webhook, its default entrypoint and the
// given targets in a transaction. Returns an error on failure (rolls
// back).
func (h *Handlers) commitWebhook(
webhook *database.Webhook,
targets []*database.Target,
) error {
tx := h.db.DB().Begin()
if tx.Error != nil {
@@ -435,6 +508,17 @@ func (h *Handlers) commitWebhook(
return err
}
for _, target := range targets {
target.WebhookID = webhook.ID
err = tx.Create(target).Error
if err != nil {
tx.Rollback()
return err
}
}
return tx.Commit().Error
}
@@ -539,13 +623,16 @@ func (h *Handlers) renderSourceDetail(
// Targets are projected to a display-safe view: a
// target's stored config blob holds a credential, and it
// must never reach a template.
"Entrypoints": entrypointViews,
"Targets": h.targetRows(&webhook, targets),
"Events": events,
"BaseURL": baseURL,
"Stats": h.loadWebhookStats(webhook.ID, entrypoints, targets),
"TargetForm": targetForm,
"TargetError": targetErr,
"Entrypoints": entrypointViews,
"Targets": h.targetRows(&webhook, targets),
"Events": events,
"BaseURL": baseURL,
"Stats": h.loadWebhookStats(webhook.ID, entrypoints, targets),
tmplKeyTargetForm: targetForm,
"TargetError": targetErr,
// The add target form's select starts on its expiry
// through Alpine, so no choice is selected here.
tmplKeyArchiveExpiryChoices: archiveExpiryChoices(),
}
status := http.StatusOK
@@ -581,12 +668,12 @@ func (h *Handlers) HandleSourceEdit() http.HandlerFunc {
return
}
data := map[string]any{
tmplKeyWebhook: &webhook,
tmplKeyError: "",
}
h.renderTemplate(w, r, "source_edit.html", data)
h.renderWebhookEdit(
w, r, &webhook,
webhook.Name, webhook.Description,
strconv.Itoa(webhook.RetentionDays),
"", http.StatusOK,
)
}
}
@@ -632,7 +719,8 @@ func (h *Handlers) HandleSourceEditSubmit() http.HandlerFunc {
}
}
// applyWebhookEdit validates and saves webhook edits.
// applyWebhookEdit validates and saves webhook edits. A refused save
// shows the edit form again with the values submitted and the reason.
func (h *Handlers) applyWebhookEdit(
w http.ResponseWriter,
r *http.Request,
@@ -641,52 +729,52 @@ func (h *Handlers) applyWebhookEdit(
// The body size cap is enforced by the MaxBodySize middleware,
// which runs before CSRF parses the form.
name := r.PostFormValue("name")
if name == "" {
data := map[string]any{
tmplKeyWebhook: webhook,
tmplKeyError: "Name is required",
}
description := r.PostFormValue("description")
retention := r.PostFormValue("retention_days")
h.renderTemplateStatus(w, r, "source_edit.html", data, http.StatusBadRequest)
if name == "" {
h.renderWebhookEdit(
w, r, webhook, name, description, retention,
"Name is required", http.StatusBadRequest,
)
return
}
oldName := webhook.Name
webhook.Name = name
webhook.Description = r.PostFormValue("description")
// An empty field falls back to the stored value, so submitting the
// form without touching retention leaves the policy alone.
retentionDays, errMsg := parseRetentionDays(
r.PostFormValue("retention_days"), webhook.RetentionDays,
retention, webhook.RetentionDays,
)
if errMsg != "" {
data := map[string]any{
tmplKeyWebhook: webhook,
tmplKeyError: errMsg,
}
h.renderTemplateStatus(w, r, "source_edit.html", data, http.StatusBadRequest)
h.renderWebhookEdit(
w, r, webhook, name, description, retention,
errMsg, http.StatusBadRequest,
)
return
}
webhook.RetentionDays = retentionDays
// edited is the webhook as the submission leaves it; webhook stays
// as stored, for the page shown again when the save is refused.
edited := *webhook
edited.Name = name
edited.Description = description
edited.RetentionDays = retentionDays
// A new name renames the archive files before it is saved (see
// delivery.Engine.Rename). If either step fails, the same targets'
// archives go back to the name that is still stored, without
// reading the main database again.
targets, err := h.renameWebhookArchives(
webhook.ID, oldName, webhook.Name,
webhook.ID, webhook.Name, edited.Name,
)
if err == nil {
err = h.db.DB().Save(webhook).Error
err = h.db.DB().Save(&edited).Error
}
if err != nil {
restoreErr := h.renameArchives(targets, oldName)
restoreErr := h.renameArchives(targets, webhook.Name)
if restoreErr != nil {
h.log.Error(
"failed to rename archives back",
@@ -696,15 +784,14 @@ func (h *Handlers) applyWebhookEdit(
}
if errors.Is(err, delivery.ErrArchiveNameTaken) {
data := map[string]any{
tmplKeyWebhook: webhook,
tmplKeyError: "Not saved: " + err.Error() +
". Move that archive out of the data directory, " +
"its .db together with any -wal and -shm beside " +
h.renderWebhookEdit(
w, r, webhook, name, description, retention,
"Not saved: "+err.Error()+
". Move that archive out of the data directory, "+
"its .db together with any -wal and -shm beside "+
"it, then save again.",
}
h.renderTemplateStatus(w, r, "source_edit.html", data, http.StatusConflict)
http.StatusConflict,
)
return
}
@@ -720,6 +807,27 @@ func (h *Handlers) applyWebhookEdit(
)
}
// renderWebhookEdit renders the webhook edit page for the webhook as
// stored, its form showing name, description and retentionDays, with
// an optional error message above it.
func (h *Handlers) renderWebhookEdit(
w http.ResponseWriter,
r *http.Request,
webhook *database.Webhook,
name, description, retentionDays, errMsg string,
status int,
) {
data := map[string]any{
tmplKeyWebhook: webhook,
tmplKeyError: errMsg,
"Name": name,
"Description": description,
"RetentionDays": retentionDays,
}
h.renderTemplateStatus(w, r, "source_edit.html", data, status)
}
// HandleSourceDelete handles webhook deletion.
func (h *Handlers) HandleSourceDelete() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
@@ -1337,7 +1445,9 @@ func (h *Handlers) newDeliveryViews(
}
if deliveries[i].Status == database.DeliveryStatusRetrying {
views[i].Paused = h.pausedView(deliveries[i].TargetID)
views[i].Paused = h.deliveryPausedView(
deliveries[i].TargetID, rows,
)
}
}
@@ -1596,49 +1706,57 @@ func (h *Handlers) newTarget(
webhookID string,
in targetFormInput,
) (*database.Target, string, error) {
if in.Name == "" {
return nil, "Name is required", nil
target := &database.Target{
WebhookID: webhookID,
Type: in.Type,
Active: true,
}
if !isValidTargetType(in.Type) {
return nil, "Invalid target type", nil
}
configJSON, errMsg, err := h.buildTargetConfig(ctx, in.Type, in)
errMsg, err := h.setTargetFromForm(ctx, target, in)
if err != nil || errMsg != "" {
return nil, errMsg, err
}
// A new target has no stored retry count, so an absent field
// takes the fire-and-forget default. A field the operator filled
// in with something invalid is refused rather than becoming
// that default.
maxRetries, err := parseMaxRetries(in.MaxRetries, 0)
if err != nil {
return nil, "Invalid max retries: " + retriesErrorMessage(err), nil
}
return &database.Target{
WebhookID: webhookID,
Name: in.Name,
Type: in.Type,
Active: true,
Config: configJSON,
MaxRetries: maxRetries,
}, "", nil
return target, "", nil
}
// isValidTargetType checks whether the target type is supported.
func isValidTargetType(tt database.TargetType) bool {
switch tt {
case database.TargetTypeHTTP,
database.TargetTypeDatabase,
database.TargetTypeLog,
database.TargetTypeSlack:
return true
default:
return false
// setTargetFromForm validates a target form against the target's type
// and, when it accepts it, sets the target's name, configuration and
// retry count from it. It returns the message the form shows for
// anything it refuses, an unknown type among them, and then leaves the
// target unchanged; an error is the server's fault, as for newTarget.
// The add target form and the target edit form both go through here,
// so the two cannot come to disagree about what a target may be.
func (h *Handlers) setTargetFromForm(
ctx context.Context,
target *database.Target,
in targetFormInput,
) (string, error) {
if in.Name == "" {
return "Name is required", nil
}
configJSON, errMsg, err := h.buildTargetConfig(ctx, target.Type, in)
if err != nil || errMsg != "" {
return errMsg, err
}
// An empty max_retries keeps the target's count: the
// fire-and-forget default of 0 for a new target, and the stored
// count for an edited one, since the forms for target types that
// do not retry have no such field. A value that is filled in but
// invalid is refused rather than becoming that count, so a typo
// cannot destroy the count a target is delivering with.
maxRetries, err := parseMaxRetries(in.MaxRetries, target.MaxRetries)
if err != nil {
return "Invalid max retries: " + retriesErrorMessage(err), nil
}
target.Name = in.Name
target.Config = configJSON
target.MaxRetries = maxRetries
return "", nil
}
// pageOrFirst parses a paginated page number, answering 1 for
@@ -1660,9 +1778,10 @@ func pageOrFirst(s string) int {
}
// targetFormInput carries the raw values of a target form. Both the
// create and the edit path fill one and hand it to buildTargetConfig,
// so neither can come to validate a destination differently from the
// other. A refused add target form is shown again from it.
// create and the edit path fill one and hand it to setTargetFromForm,
// so neither can come to validate a target differently from the
// other. Both forms are filled from one: the edit form with the
// stored values, and a refused form with the values submitted.
type targetFormInput struct {
// Name is the target's name.
Name string
@@ -509,6 +509,51 @@ func TestHandleSourceEditSubmit_InvalidRetentionIsRejected(
)
}
// TestHandleSourceEditSubmit_RefusedFormComesBack refuses an edit for
// each reason the form can give and checks that the form comes back
// with the reason and the name, description and retention submitted,
// that the page still reports the stored retention, and that nothing
// is saved.
func TestHandleSourceEditSubmit_RefusedFormComesBack(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
refused := func(name, retention, reason string) {
t.Helper()
wh := seedWebhookWithRetention(t, env.db, 30)
submitted := wh
submitted.Name = name
submitted.Description = "a description worth keeping"
w := submitEdit(t, env, submitted, retention)
assert.Equal(t, http.StatusBadRequest, w.Code)
page := w.Body.String()
assert.Contains(t, page, `class="alert-error">`+reason)
assert.Contains(t, page, `name="name" value="`+name+`"`)
assert.Contains(t, page, ">a description worth keeping</textarea>")
assert.Contains(
t, page, `name="retention_days" value="`+retention+`"`,
)
assert.Contains(t, page, "Currently 30 days.")
var stored database.Webhook
require.NoError(
t, env.db.DB().First(&stored, "id = ?", wh.ID).Error,
)
assert.Equal(t, wh.Name, stored.Name)
assert.Empty(t, stored.Description)
assert.Equal(t, 30, stored.RetentionDays)
}
refused("", "45", "Name is required")
refused("kept-name", "nonsense", "Retention must be")
}
func TestHandleSourceEditSubmit_EmptyRetentionLeavesValueUnchanged(
t *testing.T,
) {
@@ -809,6 +854,10 @@ func TestHandleSourceEditSubmit_ArchiveNameTaken(t *testing.T) {
w := submitEdit(t, env, wh, "")
require.Equal(t, http.StatusConflict, w.Code)
assert.Contains(t, w.Body.String(), "archive-taken.db")
assert.Contains(
t, w.Body.String(), `name="name" value="`+renamedWebhookName+`"`,
"the form comes back with the name submitted",
)
var stored database.Webhook
+63 -68
View File
@@ -3,6 +3,7 @@ package handlers
import (
"errors"
"net/http"
"strconv"
"github.com/go-chi/chi"
"sneak.berlin/go/webhooker/internal/database"
@@ -13,10 +14,13 @@ import (
const targetEditTemplate = "target_edit.html"
// tmplKeyTarget is the template data key for the target being
// edited, and tmplKeyMaxTimeout for the timeout ceiling the form
// tells the user about.
// edited, tmplKeyTargetForm for the values its form shows, and
// tmplKeyMaxTimeout for the timeout ceiling the form tells the user
// about. The add target form on the webhook page takes its values
// under the same key as the edit form.
const (
tmplKeyTarget = "Target"
tmplKeyTargetForm = "TargetForm"
tmplKeyMaxTimeout = "MaxTimeout"
)
@@ -28,20 +32,19 @@ const configUnreadableMessage = "The stored configuration for this " +
"target could not be read. Enter the values below; saving " +
"replaces the stored configuration."
// targetEditView is the display model for the target edit page.
// targetEditView is the display model for the target edit page: the
// target's row fields as stored. The values the form shows, the
// UNMASKED configuration among them, come separately, as a
// targetFormInput.
//
// It carries the target's row fields alongside its UNMASKED
// configuration, and deliberately omits database.Target's raw
// Config blob: the form renders named fields, and giving the
// template the blob as well would put an unreviewed second path to
// the credential on the page.
// It deliberately omits database.Target's raw Config blob: the form
// renders named fields, and giving the template the blob as well
// would put an unreviewed second path to the credential on the page.
type targetEditView struct {
ID string
Name string
Type database.TargetType
Active bool
MaxRetries int
Config delivery.TargetConfigForm
ID string
Name string
Type database.TargetType
Active bool
}
// HandleTargetEdit shows the form to edit a target.
@@ -73,7 +76,18 @@ func (h *Handlers) HandleTargetEdit() http.HandlerFunc {
msg = configUnreadableMessage
}
h.renderTargetEdit(w, r, webhook, target, cfg, msg)
form := targetFormInput{
Name: target.Name,
URL: cfg.URL,
Headers: cfg.Headers,
Timeout: cfg.Timeout,
MaxRetries: strconv.Itoa(target.MaxRetries),
Expiry: cfg.Expiry,
}
h.renderTargetEdit(
w, r, webhook, target, form, msg, http.StatusOK,
)
}
}
@@ -101,11 +115,12 @@ func (h *Handlers) HandleTargetEditSubmit() http.HandlerFunc {
}
}
// applyTargetEdit validates and saves target edits.
// applyTargetEdit validates and saves target edits. A refused save
// shows the edit form again with the values submitted and the reason.
//
// The submitted configuration goes through buildTargetConfig, the
// same builder the create path uses, so an edited destination is
// SSRF-validated exactly as a new one is.
// The submission goes through setTargetFromForm, as a new target
// does, so an edited destination is SSRF-validated exactly as a new
// one is.
//
// The target's type is not editable. Each type stores a different
// configuration shape and its delivery history is recorded against
@@ -118,16 +133,13 @@ func (h *Handlers) applyTargetEdit(
webhook database.Webhook,
target *database.Target,
) {
name := r.PostFormValue("name")
if name == "" {
http.Error(w, "Name is required", http.StatusBadRequest)
in := targetFormInputFrom(r)
return
}
// edited is the target as the submission leaves it; target stays
// as stored, for the page shown again when the save is refused.
edited := *target
configJSON, errMsg, err := h.buildTargetConfig(
r.Context(), target.Type, targetFormInputFrom(r),
)
errMsg, err := h.setTargetFromForm(r.Context(), &edited, in)
if err != nil {
h.serverError(w, r, "failed to encode target config", err)
@@ -135,45 +147,26 @@ func (h *Handlers) applyTargetEdit(
}
if errMsg != "" {
http.Error(w, errMsg, http.StatusBadRequest)
h.renderTargetEdit(
w, r, webhook, target, in, errMsg, http.StatusBadRequest,
)
return
}
// Retries are offered only by the forms for target types that
// retry, so an absent field means "this form does not edit
// retries" rather than "set them to zero". Reading it
// unconditionally would silently disable retries on any target
// saved from a form that does not render the input.
//
// A field that IS submitted but does not parse is a 400, through
// the same validator the create path uses. It is rejected before
// anything is written, so a typo cannot destroy the retry count
// the target is already delivering with.
if r.PostForm.Has("max_retries") {
retries, ok := targetMaxRetries(w, r, target.MaxRetries)
if !ok {
return
}
target.MaxRetries = retries
}
oldName := target.Name
target.Name = name
target.Config = configJSON
// A new name renames the archive file before it is saved (see
// delivery.Engine.Rename). If either step fails, it goes back to
// the name that is still stored.
err = h.renameTargetArchive(target, webhook.Name, oldName, name)
err = h.renameTargetArchive(
target, webhook.Name, target.Name, edited.Name,
)
if err == nil {
err = h.db.DB().Save(target).Error
err = h.db.DB().Save(&edited).Error
}
if err != nil {
restoreErr := h.renameTargetArchive(
target, webhook.Name, name, oldName,
target, webhook.Name, edited.Name, target.Name,
)
if restoreErr != nil {
h.log.Error(
@@ -184,8 +177,8 @@ func (h *Handlers) applyTargetEdit(
}
if errors.Is(err, delivery.ErrArchiveNameTaken) {
http.Error(
w,
h.renderTargetEdit(
w, r, webhook, target, in,
"Not saved: "+err.Error()+
". Move that archive out of the data directory, "+
"its .db together with any -wal and -shm beside "+
@@ -222,15 +215,17 @@ func (h *Handlers) renameTargetArchive(
return h.archives.Rename(target.ID, webhookName, newName)
}
// renderTargetEdit renders the target edit page with an optional
// error message.
// renderTargetEdit renders the target edit page for the target as
// stored, its form showing form's values, with an optional error
// message above it.
func (h *Handlers) renderTargetEdit(
w http.ResponseWriter,
r *http.Request,
webhook database.Webhook,
target *database.Target,
cfg delivery.TargetConfigForm,
form targetFormInput,
errMsg string,
status int,
) {
// The template calls Webhook methods, which take pointer
// receivers; html/template cannot address a value stored in a
@@ -238,18 +233,18 @@ func (h *Handlers) renderTargetEdit(
data := map[string]any{
tmplKeyWebhook: &webhook,
tmplKeyTarget: targetEditView{
ID: target.ID,
Name: target.Name,
Type: target.Type,
Active: target.Active,
MaxRetries: target.MaxRetries,
Config: cfg,
ID: target.ID,
Name: target.Name,
Type: target.Type,
Active: target.Active,
},
tmplKeyMaxTimeout: delivery.MaxTargetTimeoutSeconds,
tmplKeyError: errMsg,
tmplKeyTargetForm: form,
tmplKeyMaxTimeout: delivery.MaxTargetTimeoutSeconds,
tmplKeyError: errMsg,
tmplKeyArchiveExpiryChoices: archiveExpiryOptions(form.Expiry),
}
h.renderTemplate(w, r, targetEditTemplate, data)
h.renderTemplateStatus(w, r, targetEditTemplate, data, status)
}
// ownedTarget resolves the request's sourceID and targetID
+75
View File
@@ -565,6 +565,77 @@ func assertEditRejectsTimeout(
)
}
// TestHandleTargetEditSubmit_RefusedFormComesBack refuses an edit of
// a target of each type and checks that the edit form comes back with
// the reason and every value submitted, and that nothing is saved.
func TestHandleTargetEditSubmit_RefusedFormComesBack(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
// fields is what the operator submitted, as a query string.
cases := []struct {
targetType database.TargetType
fields string
reason string
}{
{
database.TargetTypeHTTP,
"name=edited&url=" + editBlockedURL +
"&headers=X-Edited:+kept&timeout=12&max_retries=3",
"Invalid target URL",
},
{
database.TargetTypeSlack,
"name=edited&url=" + editOriginalURL + "&max_retries=25",
"Invalid max retries",
},
{
database.TargetTypeDatabase, "name=edited&expiry=7d",
"Invalid archive expiry",
},
{database.TargetTypeLog, "name=", "Name is required"},
}
for _, tc := range cases {
t.Run(string(tc.targetType), func(t *testing.T) {
t.Parallel()
webhook := seedWebhookWithRetention(t, env.db, 30)
target := seedTarget(t, env.db, webhook.ID, tc.targetType)
form, err := url.ParseQuery(tc.fields)
require.NoError(t, err)
w := submitTargetEdit(env, webhook.ID, target.ID, form)
assert.Equal(t, http.StatusBadRequest, w.Code)
page := w.Body.String()
assert.Contains(t, page, `class="alert-error">`+tc.reason)
// headers is the form's one textarea and expiry its one
// select; every other field is an input.
for field := range form {
shown := `name="` + field + `" value="` + form.Get(field) + `"`
switch field {
case "headers":
shown = ">" + form.Get(field) + "</textarea>"
case "expiry":
shown = `<option value="` + form.Get(field) + `" selected>`
}
assert.Contains(t, page, shown)
}
assert.Equal(
t, target.Name, storedTarget(t, env, target.ID).Name,
"a refused edit must save nothing",
)
})
}
}
// TestHandleTargetEdit_Scoping keeps the edit routes scoped the way
// the delete and toggle routes are: ownership is decided by the
// webhook, and the target is then scoped to it.
@@ -700,6 +771,10 @@ func TestHandleTargetEditSubmit_RenamesArchive(t *testing.T) {
w = submitTargetEdit(env, wh.ID, archive.ID, again)
require.Equal(t, http.StatusConflict, w.Code)
assert.Contains(t, w.Body.String(), "archive-taken.db")
assert.Contains(
t, w.Body.String(), `name="name" value="Again"`,
"the form comes back with the name submitted",
)
assert.Equal(
t, renamedTargetName, storedTarget(t, env, archive.ID).Name,
)
+45 -8
View File
@@ -30,24 +30,61 @@ type TargetRowView struct {
Paused *PausedView
}
// PausedView is when a target's circuit breaker will let deliveries to
// it through again: Until is the time in UTC, and Relative how long
// that is from now.
// PausedView is a target's circuit breaker turning deliveries away.
// While the breaker is open, Until is when they resume, in UTC, and
// Relative how long that is from now. While it is half-open both are
// empty: the cooldown has ended, and the target's deliveries are held
// while one delivery tests whether the target has recovered.
type PausedView struct {
Until string
Relative string
}
// pausedView reads the target's circuit breaker, and returns nil when
// the breaker is not turning the target's deliveries away.
// pausedView reads the target's circuit breaker for its row, and
// returns nil when the breaker lets the target's deliveries through.
func (h *Handlers) pausedView(targetID string) *PausedView {
remaining := h.breakers.CooldownRemaining(targetID)
if remaining <= 0 {
state, cooldown := h.breakers.StateAndCooldown(targetID)
switch {
case state == delivery.CircuitHalfOpen:
return &PausedView{}
case state == delivery.CircuitOpen && cooldown > 0:
return newPausedView(time.Now().Add(cooldown))
default:
return nil
}
}
// deliveryPausedView reads the circuit breaker of a retrying delivery's
// target. While it is open, it says when the delivery will be tried
// next: the later of the cooldown's end and the end of the delivery's
// own backoff after its last attempt. Otherwise it returns nil, half-open
// included, since the delivery may then be the one being sent to test
// the target.
func (h *Handlers) deliveryPausedView(
targetID string, attempts []deliveryResultRow,
) *PausedView {
state, cooldown := h.breakers.StateAndCooldown(targetID)
if state != delivery.CircuitOpen || cooldown <= 0 {
return nil
}
until := time.Now().Add(remaining)
next := time.Now().Add(cooldown)
if len(attempts) > 0 {
last := attempts[len(attempts)-1]
backoffEnd := last.CreatedAt.Add(delivery.Backoff(last.AttemptNum))
if backoffEnd.After(next) {
next = backoffEnd
}
}
return newPausedView(next)
}
// newPausedView is a PausedView of deliveries that resume at until.
func newPausedView(until time.Time) *PausedView {
return &PausedView{
Until: until.UTC().Format(time.TimeOnly) + " UTC",
Relative: humanize.Time(until),
+102 -26
View File
@@ -4,6 +4,7 @@ import (
"net/http"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -15,15 +16,21 @@ import (
)
// resumesAt is how the pages write when a paused target's deliveries
// resume: the time in UTC, then how long that is from now.
// resume at the end of its breaker's cooldown: the time in UTC, then
// how long that is from now.
const resumesAt = `\d\d:\d\d:\d\d UTC \(\d+ seconds from now\)`
// TestPausedTarget_ShownUntilBreakerCloses trips the circuit breaker of
// an http target, then checks that its row on the webhook page says its
// deliveries are paused and until when, and that its retrying delivery
// says it is waiting and why in the event log and on the event's page,
// while its delivered delivery and the log target are shown as before.
// Once the breaker closes, none of the pages says so any more.
// TestPausedTarget_ShownUntilBreakerCloses takes an http target's
// circuit breaker from open through half-open to closed.
//
// Open, the target's row on the webhook page says its deliveries are
// paused and until when, and each retrying delivery says it is waiting
// and why in the event log and on the event's page, until the later of
// the cooldown's end and the end of its own backoff. Half-open, the row
// says deliveries are held while one delivery tests the target, with no
// time, and no delivery says it is waiting. Closed, the pages say
// neither. The delivered delivery and the log target are shown as
// before throughout.
func TestPausedTarget_ShownUntilBreakerCloses(t *testing.T) {
t.Parallel()
@@ -52,13 +59,22 @@ func TestPausedTarget_ShownUntilBreakerCloses(t *testing.T) {
addDelivery(t, dbMgr, wh.ID, delivered.ID, target.ID,
database.DeliveryStatusDelivered)
cb := breakers.Breaker(target.ID)
for cb.State() != delivery.CircuitOpen {
cb.RecordFailure()
}
// This delivery's 13th attempt failed a minute ago, so its own
// backoff ends over an hour from now, long after the cooldown.
backedOff := seedStoredEvent(t, dbMgr, wh.ID, `{"n":3}`)
backedOffID := addDelivery(t, dbMgr, wh.ID, backedOff.ID, target.ID,
database.DeliveryStatusRetrying)
failedAt := time.Now().Add(-time.Minute).Truncate(time.Second)
addFailedAttempt(t, dbMgr, wh.ID, backedOffID, 13, failedAt)
backoffEnds := failedAt.Add(delivery.Backoff(13)).UTC().
Format(time.TimeOnly) + " UTC (1 hour from now)"
const waiting = "waiting: target paused after repeated failures, " +
"resumes " + resumesAt
"resumes "
breakers.Set(target.ID, delivery.CircuitOpen, 30*time.Second)
list := targetList(t, renderSourceDetailPage(t, h, sess, wh.ID))
assert.Regexp(t, "t-http http Active Edit Deactivate Delete "+
@@ -67,36 +83,95 @@ func TestPausedTarget_ShownUntilBreakerCloses(t *testing.T) {
assert.Equal(t, 1, strings.Count(list, "Paused"))
log := renderSourceLogsPage(t, h, sess, wh.ID)
assert.Contains(t, log, "t-http: waiting")
assert.Equal(t, 2, strings.Count(log, "t-http: waiting"))
assert.Contains(t, log, "t-http: delivered")
assert.Regexp(t, waiting, log)
assert.Regexp(t, waiting+resumesAt, log)
assert.Contains(t, log, waiting+backoffEnds)
assert.NotContains(t, log, "retrying")
page := eventPage(t, h, sess, wh.ID, retrying.ID)
assert.Regexp(t, waiting, page)
assert.Regexp(t, waiting+resumesAt, page)
assert.NotContains(t, page, "retrying")
cb.RecordSuccess()
page = eventPage(t, h, sess, wh.ID, backedOff.ID)
assert.Contains(t, page, waiting+backoffEnds)
assert.NotContains(t, page, "seconds from now")
breakers.Set(target.ID, delivery.CircuitHalfOpen, 0)
list = targetList(t, renderSourceDetailPage(t, h, sess, wh.ID))
assert.Contains(t, list, "t-http http Active Edit Deactivate Delete "+
"Deliveries Paused: held while one delivery tests whether the "+
"target has recovered")
assert.NotContains(t, list, "UTC")
assertRetryingNotWaiting(t, h, sess, wh.ID, retrying, backedOff)
breakers.Set(target.ID, delivery.CircuitClosed, 0)
list = targetList(t, renderSourceDetailPage(t, h, sess, wh.ID))
assert.NotContains(t, list, "Paused")
log = renderSourceLogsPage(t, h, sess, wh.ID)
assert.Contains(t, log, "t-http: retrying")
assertRetryingNotWaiting(t, h, sess, wh.ID, retrying, backedOff)
}
// assertRetryingNotWaiting checks that the event log and each event's
// page show the http target's delivery of the event as retrying, and
// none of them as waiting.
func assertRetryingNotWaiting(
t *testing.T,
h *handlers.Handlers,
sess *session.Session,
webhookID string,
events ...*database.Event,
) {
t.Helper()
log := renderSourceLogsPage(t, h, sess, webhookID)
assert.Equal(t, len(events), strings.Count(log, "t-http: retrying"))
assert.NotContains(t, log, "waiting")
page = eventPage(t, h, sess, wh.ID, retrying.ID)
assert.Contains(t, page, ">retrying</span>")
assert.NotContains(t, page, "waiting")
for _, event := range events {
page := eventPage(t, h, sess, webhookID, event.ID)
assert.Contains(t, page, ">retrying</span>")
assert.NotContains(t, page, "waiting")
}
}
// addDelivery records a delivery of the event to the target, with the
// given status, in the webhook's own database.
// given status, in the webhook's own database, and returns its ID.
func addDelivery(
t *testing.T,
dbMgr *database.WebhookDBManager,
webhookID, eventID, targetID string,
status database.DeliveryStatus,
) string {
t.Helper()
webhookDB, err := dbMgr.GetDB(webhookID)
require.NoError(t, err)
dlv := &database.Delivery{
EventID: eventID,
TargetID: targetID,
Status: status,
}
require.NoError(t, webhookDB.Omit(clause.Associations).Create(
dlv,
).Error)
return dlv.ID
}
// addFailedAttempt records the delivery's failed attempt attemptNum,
// made at the given time.
func addFailedAttempt(
t *testing.T,
dbMgr *database.WebhookDBManager,
webhookID, deliveryID string,
attemptNum int,
at time.Time,
) {
t.Helper()
@@ -104,10 +179,11 @@ func addDelivery(
require.NoError(t, err)
require.NoError(t, webhookDB.Omit(clause.Associations).Create(
&database.Delivery{
EventID: eventID,
TargetID: targetID,
Status: status,
&database.DeliveryResult{
BaseModel: database.BaseModel{CreatedAt: at},
DeliveryID: deliveryID,
AttemptNum: attemptNum,
Error: "connection refused",
},
).Error)
}
@@ -44,9 +44,9 @@ func TestTargetRefusal_PrivateDestinationSaysHowToAllowIt(
form.Set("type", string(targetType))
form.Set("url", editBlockedURL)
// A refused add shows the webhook page again, where
// the hint is HTML-escaped; a refused edit answers in
// plain text.
// A refused add shows the webhook page again, and a
// refused edit the edit page, where the hint is
// HTML-escaped.
added := serveTarget(
env, http.MethodPost, targetsPath, form,
)
@@ -76,7 +76,8 @@ func TestTargetRefusal_PrivateDestinationSaysHowToAllowIt(
)
assert.Equal(t, http.StatusBadRequest, edited.Code)
assert.Contains(
t, edited.Body.String(), privateRefusalHint,
t, edited.Body.String(),
html.EscapeString(privateRefusalHint),
)
})
}
-30
View File
@@ -2,7 +2,6 @@ package handlers
import (
"errors"
"net/http"
"strconv"
"strings"
)
@@ -89,32 +88,3 @@ func retriesErrorMessage(err error) string {
return errRetriesInvalid.Error() +
", or 0 for fire-and-forget"
}
// targetMaxRetries reads and validates max_retries from a target edit
// submission, answering the request with a 400 and reporting false
// when the value is set but invalid.
//
// It and the create path (newTarget) both use parseMaxRetries and
// retriesErrorMessage, so the two cannot come to disagree about what a
// valid retry count is. The wording matches the timeout control on
// the same submission.
func targetMaxRetries(
w http.ResponseWriter,
r *http.Request,
fallback int,
) (int, bool) {
retries, err := parseMaxRetries(
r.PostFormValue("max_retries"), fallback,
)
if err != nil {
http.Error(
w,
"Invalid max retries: "+retriesErrorMessage(err),
http.StatusBadRequest,
)
return 0, false
}
return retries, true
}
+11 -13
View File
@@ -196,8 +196,6 @@ func TestCreateFormRetentionCopyMatchesBehaviour(t *testing.T) {
t.Cleanup(app.RequireStop)
body := renderPage(t, h, sess, "sources_new.html", map[string]any{
"Name": "",
"Description": "",
"DefaultRetentionDays": database.DefaultRetentionDays,
dataKeyError: "",
})
@@ -398,21 +396,21 @@ func TestTargetFormMaxRetriesCopyMatchesBehaviour(t *testing.T) {
)
// A slack target exercises the same max_retries field while needing
// only Config.URL from the edit template, so the test data stays
// minimal. The Target key mirrors the field names the template reads
// off the handler's view value.
// only a URL from the edit template, so the test data stays
// minimal. The Target and TargetForm keys mirror the field names
// the template reads off the handler's values.
editBody := renderPage(
t, h, sess, "target_edit.html", map[string]any{
dataKeyWebhook: webhook,
"Target": map[string]any{
"ID": "tg-1",
"Name": "t",
"Type": "slack",
"Active": true,
"MaxRetries": 3,
"Config": map[string]any{
"URL": "https://hooks.slack.com/services/x",
},
"ID": "tg-1",
"Name": "t",
"Type": "slack",
"Active": true,
},
"TargetForm": map[string]any{
"URL": "https://hooks.slack.com/services/x",
"MaxRetries": "3",
},
dataKeyError: "",
},
+4 -2
View File
@@ -145,8 +145,10 @@ func (n *noopArchives) Rename(_, _, _ string) error {
type noopCircuitBreakers struct{}
func (n *noopCircuitBreakers) CooldownRemaining(string) time.Duration {
return 0
func (n *noopCircuitBreakers) StateAndCooldown(
string,
) (delivery.CircuitState, time.Duration) {
return delivery.CircuitClosed, 0
}
// newServerApp starts the real login path against dir: the handlers,
+362 -47
View File
@@ -59,6 +59,45 @@ func TestAlpineRunsUnderTheSecurityPolicy(t *testing.T) {
t.Cleanup(srv.Close)
userID, _ := env.seedUser(t, "browser", "browser-password")
webhook, event, target := seedBrowserWebhook(t, env, userID)
require.NoError(t, chromedp.Run(
ctx, setCookies(srv.URL, env.authCookies(t, userID, "browser")),
))
page := srv.URL + "/hook/" + webhook.ID
// The checks share one browser tab, so they run one at a time, in
// this order. A new check is one more line here.
checkAddEntrypoint(ctx, t, page)
checkAddEachTargetType(ctx, t, page)
checkArchiveExpiry(ctx, t, page)
checkRefusedTarget(ctx, t, page)
checkTargetDeliveries(ctx, t, page, target.Name,
"0 in total, 0 in the last 24 hours",
"1 in total, 1 in the last 24 hours")
checkRefusedEdits(ctx, t, page, target.ID)
checkCopy(ctx, t, page)
checkEntrypointEdit(ctx, t, page, page+"/events")
checkRecentEvents(ctx, t, page)
checkEventLog(ctx, t, page+"/events", event.ID, target.Name)
checkArchiveChoice(ctx, t, srv.URL+"/hooks/new", page)
checkNewWebhookTargets(ctx, t, env, srv.URL+"/hooks/new")
checkRefusedNewWebhook(ctx, t, srv.URL+"/hooks/new")
checkMobileMenu(ctx, t, page)
assert.Empty(t, problems(), "the browser reported problems")
}
// seedBrowserWebhook seeds the webhook the browser test loads, owned by
// userID: an entrypoint, two events, and a target whose delivery of the
// newer event failed once with a 502. It returns the webhook, the newer
// event and the target.
func seedBrowserWebhook(
t *testing.T, env *testEnv, userID string,
) (*database.Webhook, *database.Event, *database.Target) {
t.Helper()
webhook := env.seedWebhook(t, userID)
require.NoError(t, env.db.DB().Omit(clause.Associations).Create(
&database.Entrypoint{
@@ -82,53 +121,7 @@ func TestAlpineRunsUnderTheSecurityPolicy(t *testing.T) {
},
).Error)
require.NoError(t, chromedp.Run(
ctx, setCookies(srv.URL, env.authCookies(t, userID, "browser")),
))
page := srv.URL + "/hook/" + webhook.ID
checkAddEntrypoint(ctx, t, page)
// Each target type, with the fields its add target form submits, in
// page order. Only http and slack have a url field.
targetTypes := []struct {
name string
fields string
values map[string]string
}{
{
"http", "csrf_token name type url headers timeout max_retries",
map[string]string{"url": publicTargetURL},
},
{
"slack", "csrf_token name type url max_retries",
map[string]string{"url": publicTargetURL},
},
{
"database", "csrf_token name type expiry",
map[string]string{"expiry": "720h"},
},
{"log", "csrf_token name type", nil},
}
for _, tt := range targetTypes {
checkAddTarget(
ctx, t, page, tt.name, strings.Fields(tt.fields), tt.values,
)
}
checkRefusedTarget(ctx, t, page)
checkTargetDeliveries(ctx, t, page, target.Name,
"0 in total, 0 in the last 24 hours",
"1 in total, 1 in the last 24 hours")
checkCopy(ctx, t, page)
checkEntrypointEdit(ctx, t, page, page+"/events")
checkRecentEvents(ctx, t, page)
checkEventLog(ctx, t, page+"/events", event.ID, target.Name)
checkMobileMenu(ctx, t, page)
assert.Empty(t, problems(), "the browser reported problems")
return webhook, event, target
}
// startBrowser starts a headless browser for one test. It returns the
@@ -307,6 +300,40 @@ const (
document.querySelector('form[action$="/targets"]')).keys()]`
)
// checkAddEachTargetType runs checkAddTarget on a webhook page for each
// target type, in page order.
func checkAddEachTargetType(ctx context.Context, t *testing.T, url string) {
t.Helper()
// Each target type, with the fields its add target form submits, in
// page order. Only http and slack have a url field.
targetTypes := []struct {
name string
fields string
values map[string]string
}{
{
"http", "csrf_token name type url headers timeout max_retries",
map[string]string{"url": publicTargetURL},
},
{
"slack", "csrf_token name type url max_retries",
map[string]string{"url": publicTargetURL},
},
{
"database", "csrf_token name type expiry",
map[string]string{"expiry": "720h"},
},
{"log", "csrf_token name type", nil},
}
for _, tt := range targetTypes {
checkAddTarget(
ctx, t, url, tt.name, strings.Fields(tt.fields), tt.values,
)
}
}
// checkAddTarget loads a webhook page and walks the add target form for
// one target type. The form shows nothing until Add is clicked; Add
// shows only the type choice; Cancel there closes it; Next shows the
@@ -395,6 +422,43 @@ func chooseTargetType(ctx context.Context, t *testing.T, targetType string) {
"%s: Add still shows while the form is open", targetType)
}
// checkArchiveExpiry loads a webhook page and checks that the add
// target form's archive expiry starts on never, that the database
// target checkAddTarget added with 720h is listed as 30 days, and that
// its edit form starts on 720h.
func checkArchiveExpiry(ctx context.Context, t *testing.T, url string) {
t.Helper()
const expiry = `form[action$="/targets"] select[name="expiry"]`
row := `//span[text()="added-database"]/ancestor::div[@class="p-4"][1]`
require.NoError(t, chromedp.Run(ctx, loadPage(url)))
chooseTargetType(ctx, t, "database")
var start, edited string
require.NoError(t, chromedp.Run(
ctx, chromedp.Value(expiry, &start, chromedp.ByQuery),
))
assert.Equal(t, "never", start,
"the add target form's archive expiry does not start on never")
assert.True(t, shown(ctx, row+`//span[text()="Archive Expiry:"]`+
`/following-sibling::span[text()="30 days"]`),
"a database target added with 720h is not listed as 30 days")
click(ctx, t, row+`//a[text()="Edit"]`)
require.NoError(t, chromedp.Run(
ctx,
chromedp.WaitReady("#expiry", chromedp.ByQuery),
chromedp.Value("#expiry", &edited, chromedp.ByQuery),
))
assert.Equal(t, "720h", edited,
"the edit form does not start on the stored archive expiry")
}
// checkRefusedTarget submits an http target the server refuses, a
// loopback destination, and checks that the page comes back with the
// form open on the http fields, the values entered and the reason, and
@@ -478,6 +542,64 @@ func checkTargetDeliveries(
"the row of %s does not show %q failed", name, failed)
}
// checkRefusedEdits fills in the target edit page and the webhook edit
// page of a webhook page with values the server refuses, a loopback
// destination and a retention above the longest finite one, which the
// browser lets through. It saves each and checks that the page comes
// back with the reason and every value still in its field. The values
// are keyed by the id of their field.
func checkRefusedEdits(
ctx context.Context, t *testing.T, page, targetID string,
) {
t.Helper()
const reason = `//div[@class="alert-error"]`
edits := []struct {
url string
values map[string]string
}{
{page + "/targets/" + targetID + "/edit", map[string]string{
"#name": "edited-target",
"#url": "http://127.0.0.1/hook",
"#headers": "X-Edited: kept",
"#timeout": "12",
"#max_retries": "3",
}},
{page + "/edit", map[string]string{
"#name": "edited-webhook",
"#description": "kept description",
"#retention_days": "200000",
}},
}
for _, edit := range edits {
require.NoError(t, chromedp.Run(ctx, loadPage(edit.url)))
for field, value := range edit.values {
require.NoError(t, chromedp.Run(
ctx, chromedp.SetValue(field, value, chromedp.ByQuery),
))
}
click(ctx, t, `//button[text()="Save Changes"]`)
assert.Truef(t, shown(ctx, reason),
"%s: a refused save does not show the reason", edit.url)
for field, value := range edit.values {
var kept string
require.NoError(t, chromedp.Run(
ctx, chromedp.Value(field, &kept, chromedp.ByQuery),
))
assert.Equalf(t, value, kept,
"%s: a refused save does not keep the %s entered",
edit.url, field)
}
}
}
// checkCopy loads a webhook page and checks that the Copy control beside
// its entrypoint's URL is a button, and that clicking it copies the URL
// and says so: the button reads "Copied" only once the copy succeeded.
@@ -668,6 +790,199 @@ func checkEventLog(
"clicking the event again does not collapse it")
}
// The parts of the new webhook page the checks below find and click.
const (
archiveBox = `//input[@name="archive"]`
archiveIsOn = `document.querySelector('input[name="archive"]').checked`
pruningChoice = `//select[@name="archive_expiry"]`
createButton = `//button[text()="Create Webhook"]`
)
// checkArchiveChoice loads the new webhook page and checks that the
// archive pruning choice stays hidden until the archive box is checked
// and hides again when it is unchecked; and that after checking it,
// opening the page at elsewhere and going back, the page again shows
// the box unchecked and the choice hidden.
func checkArchiveChoice(
ctx context.Context, t *testing.T, url, elsewhere string,
) {
t.Helper()
require.NoError(t, chromedp.Run(ctx, loadPage(url)))
assert.True(t, hidden(ctx, pruningChoice),
"the pruning choice shows before archive is checked")
click(ctx, t, archiveBox)
assert.True(t, shown(ctx, pruningChoice),
"checking archive does not show the pruning choice")
click(ctx, t, archiveBox)
assert.True(t, hidden(ctx, pruningChoice),
"unchecking archive does not hide the pruning choice")
var (
loaded string
checked bool
)
click(ctx, t, archiveBox)
require.NoError(t, chromedp.Run(
ctx,
loadPage(elsewhere),
chromedp.NavigateBack(),
chromedp.WaitNotPresent("[x-cloak]", chromedp.ByQuery),
chromedp.Evaluate(
`performance.getEntriesByType("navigation")[0].type`, &loaded,
),
chromedp.Evaluate(archiveIsOn, &checked),
))
require.Equal(
t, "back_forward", loaded,
"going back, the browser did not load the page again",
)
assert.False(t, checked, "going back leaves archive checked")
assert.True(t, hidden(ctx, pruningChoice),
"going back shows the pruning choice")
}
// checkNewWebhookTargets submits the new webhook page with the HTTP
// target URL filled in or empty, and with archive left off or checked
// with each pruning choice, and checks that each webhook is created
// with exactly the targets asked for.
func checkNewWebhookTargets(
ctx context.Context, t *testing.T, env *testEnv, url string,
) {
t.Helper()
// Each value the pruning choice submits, after an empty one that
// stands for archive left off.
expiries := []string{
"", "never", "1h", "12h", "24h", "720h", "2160h", "8760h",
}
for _, httpURL := range []string{"", publicTargetURL} {
for _, expiry := range expiries {
name := "url=" + httpURL + " archive=" + expiry
want := map[database.TargetType]string{}
require.NoError(t, chromedp.Run(
ctx,
loadPage(url),
chromedp.SetValue("#name", name, chromedp.ByQuery),
))
if httpURL != "" {
require.NoError(t, chromedp.Run(ctx, chromedp.SetValue(
"#http_url", httpURL, chromedp.ByQuery,
)))
want[database.TargetTypeHTTP] = `{"url":"` + httpURL + `"}`
}
if expiry != "" {
// The choice showing moves Create down, so it is
// waited for before Create is clicked.
click(ctx, t, archiveBox)
require.Truef(t, shown(ctx, pruningChoice),
"%s: checking archive does not show the pruning choice",
name)
require.NoError(t, chromedp.Run(ctx, chromedp.SetValue(
pruningChoice, expiry, chromedp.BySearch,
)))
want[database.TargetTypeDatabase] = `{"expiry":"` + expiry + `"}`
}
click(ctx, t, createButton)
require.Truef(t, shown(ctx, `//h1[text()="`+name+`"]`),
"%s: the new webhook's page does not open", name)
assert.Equalf(t, want, targetConfigs(t, env, name),
"%s: the webhook does not have the targets asked for", name)
}
}
}
// targetConfigs reads the targets of the webhook named name, and
// returns each one's stored configuration by its type.
func targetConfigs(
t *testing.T, env *testEnv, name string,
) map[database.TargetType]string {
t.Helper()
var (
webhook database.Webhook
targets []database.Target
)
require.NoError(t, env.db.DB().
Where("name = ?", name).First(&webhook).Error)
require.NoError(t, env.db.DB().
Where("webhook_id = ?", webhook.ID).Find(&targets).Error)
configs := map[database.TargetType]string{}
for _, target := range targets {
configs[target.Type] = target.Config
}
return configs
}
// checkRefusedNewWebhook submits the new webhook page with archive
// checked and an HTTP target URL the server refuses, a loopback
// destination, and checks that the page comes back with the reason and
// every value entered, archive still checked and its pruning choice
// showing.
func checkRefusedNewWebhook(ctx context.Context, t *testing.T, url string) {
t.Helper()
const refusedURL = "http://127.0.0.1/hook"
require.NoError(t, chromedp.Run(
ctx,
loadPage(url),
chromedp.SetValue("#name", "refused", chromedp.ByQuery),
chromedp.SetValue("#description", "kept", chromedp.ByQuery),
chromedp.SetValue("#retention_days", "7", chromedp.ByQuery),
chromedp.SetValue("#http_url", refusedURL, chromedp.ByQuery),
))
click(ctx, t, archiveBox)
require.True(t, shown(ctx, pruningChoice),
"checking archive does not show the pruning choice")
require.NoError(t, chromedp.Run(ctx, chromedp.SetValue(
pruningChoice, "2160h", chromedp.BySearch,
)))
click(ctx, t, createButton)
assert.True(t, shown(ctx, `//div[@class="alert-error"]`),
"a refused webhook does not show the reason")
var (
name, description, retention, typed, expiry string
checked bool
)
require.NoError(t, chromedp.Run(
ctx,
chromedp.Value("#name", &name, chromedp.ByQuery),
chromedp.Value("#description", &description, chromedp.ByQuery),
chromedp.Value("#retention_days", &retention, chromedp.ByQuery),
chromedp.Value("#http_url", &typed, chromedp.ByQuery),
chromedp.Value("#archive_expiry", &expiry, chromedp.ByQuery),
chromedp.Evaluate(archiveIsOn, &checked),
))
assert.Equal(t, "refused", name, "the name entered is lost")
assert.Equal(t, "kept", description, "the description entered is lost")
assert.Equal(t, "7", retention, "the retention entered is lost")
assert.Equal(t, refusedURL, typed, "the url entered is lost")
assert.True(t, checked, "archive is no longer checked")
assert.True(t, shown(ctx, pruningChoice), "the pruning choice is hidden")
assert.Equal(t, "2160h", expiry, "the pruning chosen is lost")
}
// checkMobileMenu loads a page in a phone-sized window and checks that
// the menu button opens and closes the mobile menu.
func checkMobileMenu(ctx context.Context, t *testing.T, url string) {
+4 -2
View File
@@ -67,8 +67,10 @@ func (e *noopArchives) Rename(_, _, _ string) error {
// paused.
type noopCircuitBreakers struct{}
func (b *noopCircuitBreakers) CooldownRemaining(string) time.Duration {
return 0
func (b *noopCircuitBreakers) StateAndCooldown(
string,
) (delivery.CircuitState, time.Duration) {
return delivery.CircuitClosed, 0
}
// testEnv is the real router from routes.go plus the collaborators
+2 -2
View File
@@ -60,9 +60,9 @@ main() {
if missing go; then pkg_install go golang go go; fi
# Not installed here: docker is platform-specific and out of scope for a
# package-manager bootstrap, but script/lint needs it.
# package-manager bootstrap, but script/lint and script/css need it.
if missing docker; then
echo "bootstrap: docker not found; script/lint requires it" >&2
echo "bootstrap: docker not found; script/lint and script/css require it" >&2
fi
go mod download
+3 -2
View File
@@ -1,8 +1,8 @@
#!/bin/sh
# script/check: run all checks (test, lint, fmt-check). Our own
# script/check: run all checks (test, lint, fmt-check, css-check). Our own
# extension to scripts-to-rule-them-all.
# Writes only the ignored static/js/alpine.min.js, through script/test.
# Generic: usually needs no adaptation.
# Generic, apart from css-check.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -11,6 +11,7 @@ main() {
"$SCRIPT_DIR/test"
"$SCRIPT_DIR/lint"
"$SCRIPT_DIR/fmt-check"
"$SCRIPT_DIR/css-check"
}
main "$@"
Executable
+15
View File
@@ -0,0 +1,15 @@
#!/bin/sh
# script/css: regenerate static/css/tailwind.css (writes). tailwindcss is
# never installed locally: it runs in docker, at the version and sha256
# pinned in the Dockerfile's stylesheet stages, which also say what the
# stylesheet is generated from.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
docker build --target css-output --output type=local,dest=static/css .
}
main "$@"
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
# script/css-check: fail when static/css/tailwind.css differs from what
# script/css would generate (read-only). The comparison is the Dockerfile's
# css-check stage, which the image build runs too.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
docker build --target css-check --output type=cacheonly .
}
main "$@"
+6 -7
View File
@@ -1,7 +1,10 @@
@import "tailwindcss";
/* Source the templates */
/* Classes are taken only from the files named below. A class written in
any other file is not generated: name that file here too. */
@import "tailwindcss" source(none);
@source "../../templates/**/*.html";
@source "../js/app.js";
/* targetStatus picks a target's status colour class */
@source "../../internal/handlers/recent_events.go";
/* Material Design inspired theme customization */
@theme {
@@ -53,10 +56,6 @@
@apply inline-flex items-center justify-center px-4 py-2 rounded-md font-medium text-sm transition-all duration-200 focus:outline-none focus:ring-2 focus:ring-offset-2 disabled:opacity-50 disabled:cursor-not-allowed bg-error-500 text-white hover:bg-error-700 active:bg-red-800 focus:ring-red-500 shadow-elevation-1 hover:shadow-elevation-2;
}
.btn-text {
@apply inline-flex items-center justify-center px-4 py-2 rounded-md font-medium text-sm transition-all duration-200 focus:outline-none focus:ring-2 focus:ring-offset-2 disabled:opacity-50 disabled:cursor-not-allowed text-primary-600 hover:bg-primary-50 active:bg-primary-100;
}
/* Cards */
.card {
@apply bg-white rounded-lg shadow-elevation-1 overflow-hidden;
File diff suppressed because one or more lines are too long
+3 -2
View File
@@ -71,8 +71,9 @@ document.addEventListener("alpine:init", function () {
// Something a click shows and hides: the mobile menu, an add form,
// an entrypoint's edit form, an event in the event log or in the
// recent events, a delivery's attempts. It starts hidden, or shown
// when its element has the data-open attribute.
// recent events, a delivery's attempts, the new webhook page's
// archive pruning choice. It starts hidden, or shown when its
// element has the data-open attribute.
window.Alpine.data("collapsible", function () {
return {
open: false,
+10 -3
View File
@@ -203,8 +203,15 @@
<template x-if="isDatabase">
<div>
<input type="hidden" name="type" value="database">
<input type="text" name="expiry" :value="expiry" placeholder="never" class="input text-sm">
<p class="text-xs text-gray-500 mt-1">Archive expiry: "never" (default) keeps rows forever, or a duration like "720h" prunes older rows.</p>
<div class="flex gap-2 items-center">
<label class="text-sm text-gray-700">Archive expiry:</label>
<select name="expiry" :value="expiry" class="input text-sm w-24">
{{range .ArchiveExpiryChoices}}
<option value="{{.Value}}">{{.Label}}</option>
{{end}}
</select>
</div>
<p class="text-xs text-gray-500 mt-1">Archived events older than this are deleted from the archive; never keeps them all.</p>
</div>
</template>
<template x-if="isLog">
@@ -251,7 +258,7 @@
{{with .Paused}}
<div class="text-xs text-yellow-600 mt-1">
<span class="font-medium">Deliveries Paused:</span>
<span>after repeated failures, until {{.Until}} ({{.Relative}})</span>
<span>{{if .Until}}after repeated failures, until {{.Until}} ({{.Relative}}){{else}}held while one delivery tests whether the target has recovered{{end}}</span>
</div>
{{end}}
{{range .Config}}
+3 -3
View File
@@ -18,17 +18,17 @@
<input type="hidden" name="csrf_token" value="{{.CSRFToken}}">
<div class="form-group">
<label for="name" class="label">Name</label>
<input type="text" id="name" name="name" value="{{.Webhook.Name}}" required class="input">
<input type="text" id="name" name="name" value="{{.Name}}" required class="input">
</div>
<div class="form-group">
<label for="description" class="label">Description</label>
<textarea id="description" name="description" rows="3" class="input">{{.Webhook.Description}}</textarea>
<textarea id="description" name="description" rows="3" class="input">{{.Description}}</textarea>
</div>
<div class="form-group">
<label for="retention_days" class="label">Retention (days)</label>
<input type="number" id="retention_days" name="retention_days" value="{{.Webhook.RetentionDays}}" min="0" class="input">
<input type="number" id="retention_days" name="retention_days" value="{{.RetentionDays}}" min="0" class="input">
<p class="text-xs text-gray-500 mt-1">Currently {{.Webhook.RetentionLabel}}.{{if .Webhook.RetainsForever}} No events are deleted while retention is set to forever.{{else}} A periodic cleanup permanently deletes events older than this, along with their delivery records.{{end}} Enter 0 to retain events forever; leave blank to keep the current setting.</p>
</div>
+29 -3
View File
@@ -18,20 +18,46 @@
<input type="hidden" name="csrf_token" value="{{.CSRFToken}}">
<div class="form-group">
<label for="name" class="label">Name</label>
<input type="text" id="name" name="name" value="{{.Name}}" required autofocus placeholder="My Webhook" class="input">
<input type="text" id="name" name="name" value="{{.Form.Name}}" required autofocus placeholder="My Webhook" class="input">
</div>
<div class="form-group">
<label for="description" class="label">Description</label>
<textarea id="description" name="description" rows="3" placeholder="Optional description" class="input">{{.Description}}</textarea>
<textarea id="description" name="description" rows="3" placeholder="Optional description" class="input">{{.Form.Description}}</textarea>
</div>
<div class="form-group">
<label for="retention_days" class="label">Retention (days)</label>
<input type="number" id="retention_days" name="retention_days" value="{{.DefaultRetentionDays}}" min="0" class="input">
<input type="number" id="retention_days" name="retention_days" value="{{.Form.RetentionDays}}" min="0" class="input">
<p class="text-xs text-gray-500 mt-1">A periodic cleanup permanently deletes events older than this, along with their delivery records. Enter 0 to retain events forever; leave blank to use the default of {{.DefaultRetentionDays}} days.</p>
</div>
<div class="form-group">
<label for="http_url" class="label">HTTP target URL</label>
<input type="url" id="http_url" name="http_url" value="{{.Form.HTTPURL}}" placeholder="https://example.com/webhook" class="input">
<p class="text-xs text-gray-500 mt-1">Optional. When filled in, the webhook is created with an HTTP target that delivers each event to this URL.</p>
</div>
<!-- The checkbox shows the pruning choice while checked. With
autocomplete="off", going back to the page does not
check the box again with the choice hidden. -->
<div class="form-group" x-data="collapsible"{{if .Form.Archive}} data-open{{end}}>
<label class="flex items-center gap-2 text-sm font-medium text-gray-700">
<input type="checkbox" name="archive" value="on"{{if .Form.Archive}} checked{{end}} autocomplete="off" @change="toggle" class="h-4 w-4">
Archive
</label>
<p class="text-xs text-gray-500 mt-1">When checked, the webhook is created with a database target that keeps a copy of every event.</p>
<div x-show="open" x-cloak class="mt-3">
<label for="archive_expiry" class="label">Archive pruning</label>
<select id="archive_expiry" name="archive_expiry" class="input">
{{range .ArchiveExpiryChoices}}
<option value="{{.Value}}"{{if .Selected}} selected{{end}}>{{.Label}}</option>
{{end}}
</select>
<p class="text-xs text-gray-500 mt-1">Archived events older than this are deleted from the archive; never keeps them all.</p>
</div>
</div>
<div class="flex gap-3">
<button type="submit" class="btn-primary">Create Webhook</button>
<a href="/hooks" class="btn-secondary">Cancel</a>
+13 -9
View File
@@ -17,7 +17,7 @@
{{if or (eq .Target.Type "http") (eq .Target.Type "slack")}}
<div class="mb-6 rounded-md bg-gray-50 p-4 text-sm text-gray-700">
This form shows the target's stored destination in full, including any credential carried in its URL or headers. It is the only page that does; everywhere else the value is masked.
This form shows the target's destination in full, including any credential carried in its URL or headers. It is the only page that does; everywhere else the value is masked.
</div>
{{end}}
@@ -26,25 +26,25 @@
<div class="form-group">
<label for="name" class="label">Name</label>
<input type="text" id="name" name="name" value="{{.Target.Name}}" required class="input">
<input type="text" id="name" name="name" value="{{.TargetForm.Name}}" required class="input">
</div>
{{if eq .Target.Type "http"}}
<div class="form-group">
<label for="url" class="label">Destination URL</label>
<input type="url" id="url" name="url" value="{{.Target.Config.URL}}" required class="input">
<input type="url" id="url" name="url" value="{{.TargetForm.URL}}" required class="input">
<p class="text-xs text-gray-500 mt-1">Revalidated on save; destinations that resolve to private or link-local addresses are rejected.</p>
</div>
<div class="form-group">
<label for="headers" class="label">Headers</label>
<textarea id="headers" name="headers" rows="4" class="input" placeholder="Authorization: Bearer ...">{{.Target.Config.Headers}}</textarea>
<textarea id="headers" name="headers" rows="4" class="input" placeholder="Authorization: Bearer ...">{{.TargetForm.Headers}}</textarea>
<p class="text-xs text-gray-500 mt-1">One <code>Name: value</code> per line, sent with every delivery. Leave blank for none. <code>Host</code>, <code>Content-Length</code>, <code>Transfer-Encoding</code>, <code>Connection</code>, <code>Trailer</code> and <code>User-Agent</code> are set by the delivery engine and are rejected here rather than silently ignored. Headers set here are dropped if a redirect leaves the destination's own origin, so a credential cannot follow one to another host.</p>
</div>
<div class="form-group">
<label for="timeout" class="label">Timeout (seconds)</label>
<input type="number" id="timeout" name="timeout" value="{{.Target.Config.Timeout}}" min="0" max="{{.MaxTimeout}}" class="input">
<input type="number" id="timeout" name="timeout" value="{{.TargetForm.Timeout}}" min="0" max="{{.MaxTimeout}}" class="input">
<p class="text-xs text-gray-500 mt-1">Per-request timeout, at most {{.MaxTimeout}} seconds. Leave blank to use the default.</p>
</div>
{{end}}
@@ -52,7 +52,7 @@
{{if eq .Target.Type "slack"}}
<div class="form-group">
<label for="url" class="label">Webhook URL</label>
<input type="url" id="url" name="url" value="{{.Target.Config.URL}}" required class="input">
<input type="url" id="url" name="url" value="{{.TargetForm.URL}}" required class="input">
<p class="text-xs text-gray-500 mt-1">Slack or Mattermost incoming webhook URL. Revalidated on save.</p>
</div>
{{end}}
@@ -60,15 +60,19 @@
{{if eq .Target.Type "database"}}
<div class="form-group">
<label for="expiry" class="label">Archive Expiry</label>
<input type="text" id="expiry" name="expiry" value="{{.Target.Config.Expiry}}" placeholder="never" class="input">
<p class="text-xs text-gray-500 mt-1">"never" (the default when blank) keeps archived rows forever, or a Go duration like "720h" prunes older rows.</p>
<select id="expiry" name="expiry" class="input">
{{range .ArchiveExpiryChoices}}
<option value="{{.Value}}"{{if .Selected}} selected{{end}}>{{.Label}}</option>
{{end}}
</select>
<p class="text-xs text-gray-500 mt-1">Archived events older than this are deleted from the archive; never keeps them all.</p>
</div>
{{end}}
{{if or (eq .Target.Type "http") (eq .Target.Type "slack")}}
<div class="form-group">
<label for="max_retries" class="label">Max retries</label>
<input type="number" id="max_retries" name="max_retries" value="{{.Target.MaxRetries}}" min="0" max="20" class="input">
<input type="number" id="max_retries" name="max_retries" value="{{.TargetForm.MaxRetries}}" min="0" max="20" class="input">
<p class="text-xs text-gray-500 mt-1">This is the total number of delivery attempts, not retries on top of the first: a value of 3 makes three attempts in all. 0 means a single attempt with no retries and no circuit breaker.</p>
</div>
{{end}}