16 Commits
Author SHA1 Message Date
clawbot 8406c77949 Store a target's settings as columns (closes #543)
check / check (push) Waiting to run
A target's settings move from the per-type JSON string in
targets.config to columns: url, headers, timeout_seconds,
forward_query, archive_expiry and archive_rotation. A column a type
does not use stays empty. headers holds the header map as JSON text
through GORM's JSON serializer, and archive_expiry holds a
time.Duration, 0 meaning never. The forms check each setting once,
with the same refusals and messages, and the edit form is filled from
the columns, so the per-type config structs, their parsers and the
"could not be read" branches are gone. A delivery task carries the
six settings in place of the JSON string; tests check that each copy
into a task, and back out of one, keeps all six.

Model: opus-5-5
2026-10-15 23:02:39 +02:00
clawbot ded39aa158 Use one recovery routine at start and on every sweep tick (closes #542)
check / check (push) Waiting to run
Start-up recovery and the 60-second sweep were two parallel sets of
functions doing the same job. recoverDeliveries now does it, run once at
start and then on every tick from one goroutine. A retrying delivery is
scheduled after what is left of its backoff, now if nothing is left, so
backoffElapsed is gone. The run at start still takes every pending
delivery and opens a webhook database whose file is missing; a tick
still takes only pending deliveries untouched for 15 minutes, at most
500, and skips a missing file. A retrying delivery is taken before its
event is read. The README describes one routine.

Model: opus-5-5
2026-10-15 21:57:05 +02:00
clawbot 0cdda43e6c Move the archive files into their own package (closes #541)
check / check (push) Waiting to run
The SQLite archive files the database target writes are now managed by
internal/archive: file names, rotation, the writer of each target,
pruning, renames, the gzipped JSON export, the idle sweep, and the
rename lock, which was Handlers.renameMu. The database target stays in
internal/delivery and writes through archive.Writers; the engine still
closes the writers when it stops, so the stop order is unchanged. The
handlers take the rename lock from archive.Archives.RenameLock, and
rename and evict through archive.Archives.

Exported names drop their Archive prefix in the new package. The tests
moved with the code; the two that check a delivery's recorded attempt
stay in delivery.

Model: opus-5-5
2026-10-15 21:03:43 +02:00
clawbot f10440b7cc Store a webhook's retention as a plain day count, 0 meaning forever (closes #539)
check / check (push) Waiting to run
retention_days loses its column default of 30, and Webhook.RetentionDays
its default tag, so a 0 is stored as 0. The 365000-day stand-in
RetentionForeverDays and the BeforeSave hook that wrote it are gone. The
reaper and the "forever" label read 0 as forever, and the forms accept 0
or 1 to MaxFiniteRetentionDays; a blank field still gives 30 on the new
webhook form and keeps the stored value on the edit form. A negative
value, which the forms refuse, is still read as forever so it cannot put
the reaper's cutoff in the future.

Model: opus-5-5
2026-10-15 20:03:33 +02:00
clawbot 40e8ecaab4 Store a target's number of delivery attempts, at least one (closes #540)
check / check (push) Waiting to run
Target.MaxRetries (max_retries) becomes Target.MaxAttempts
(max_attempts), and the form field, parser and ceiling are renamed to
match. The column is NOT NULL and a check constraint refuses a value
below 1, so every target, whatever its type, is stored with at least
1. 1 is a single attempt with no circuit breaker; above 1 retries
behind the breaker up to that many attempts. Both target forms refuse
0 and less; a blank field on the add target form gives 1, and on the
edit form keeps the stored value. The target page shows the plain
number, and the README's two paragraphs on 0 against 1 become one
sentence.

Model: opus-5-5
2026-10-15 19:38:14 +02:00
clawbot 147d58dd96 Say the query logger drops bound values (closes #535)
check / check (push) Waiting to run
The comment on MaxAccessLogLineBytes said the GORM adapter logs each
statement with the client's parameter written into it. Its
ParamsFilter drops every bound value, so the statement is logged with
its placeholders. The comment now says so and keeps the parts that
still hold: the widest line spends two budgets and the adapter's test
checks it against this constant.

The assertBounded doc no longer claims each line was cut, and the
postReceiver doc no longer explains itself by a log size test that is
gone.

Model: opus-5-5
2026-10-15 18:57:58 +02:00
clawbot c7c4ea3340 Send only the Referer's scheme and host to Sentry (closes #548)
check / check (push) Waiting to run
A Sentry event kept the Referer header verbatim, and a Referer can be
an entrypoint URL: a sender that follows the README's http:// to
https:// redirect sends the URL it was redirected from, and a browser
sends the 404 page's URL under /h/ when it fetches that page's
stylesheet. The Sentry hook now keeps only the Referer's scheme and
host. The access log's reduction moves, unchanged, into a new package,
internal/referer, as SchemeAndHost; the access log and the Sentry hook
both call it there. The comment above sentryKeepsHeader and the
README's Sentry passage say so, and the README's Package Layout lists
the new package.

Model: opus-5-5
2026-10-15 18:07:21 +02:00
clawbot 8c05ec4378 Build the binary without cgo (closes #545)
check / check (push) Waiting to run
make build now builds with CGO_ENABLED=0, and the image builds through
it. The binary's only cgo code was mattn/go-sqlite3, which
gorm.io/driver/sqlite imports and webhooker never opens: every SQLite
file is opened with modernc.org/sqlite and the connection handed to
gorm. Without cgo, mattn's package compiles to a stub nothing calls.

The Dockerfile builds the binary once. The static relink and its
netgo, osusergo and sqlite_omit_load_extension tags are gone, and with
them the Makefile's GO_LDFLAGS and GO_TAGS, which only the relink used.
The test phase keeps cgo for -race. No README or Dockerfile sentence
says the binary needs cgo any more.

Model: opus-5-5
2026-10-15 17:36:35 +02:00
clawbot 872e4eb4a1 Drop 'unsafe-inline' from the Content-Security-Policy (closes #530)
check / check (push) Waiting to run
The delete prompts on the webhook page were inline onsubmit handlers.
Each delete form now carries its question in a data-confirm attribute,
which one submit listener in static/js/app.js asks. The three style
attributes became Tailwind classes, with tailwind.css regenerated, and
the x-cloak rule moved to style.css. The policy now sends
script-src 'self' and style-src 'self'.

The browser test now answers a target's delete prompt no and then yes,
and also loads the sign-in page, the webhook list, the settings and
profile pages and the 404 error page under the policy. The README
states the policy as sent.

Model: opus-5-5
2026-10-15 16:18:26 +02:00
clawbot dfe8f3138f Bind 0.0.0.0 to IPv4 only (closes #528)
check / check (push) Waiting to run
Go listens on any unspecified address with one socket that answers
IPv4 and IPv6, so BIND_ADDRESS=0.0.0.0 also bound every IPv6 address,
which the README says it does not. The listener now uses tcp4 for an
IPv4 address, written plain or IPv4-mapped; :: still binds both. The
server's start hook suppresses contextcheck, as the other start hooks
do.

The image's health check asked for localhost, which wget reached over
::1 only. It now asks for the container's own address: its IPv4
address, or its IPv6 address on an IPv6-only network. A loopback bind
answers neither. The README says such a network needs ::, drops the
bindv6only condition from :: (Go sets that socket option itself) and
names an ss check.

Model: opus-5-5
2026-10-15 15:44:42 +02:00
clawbot 0d38aea12c Keep the entrypoint UUID out of webhooker's own logs (closes #531)
check / check (push) Waiting to run
On the receiver route the access log and the per-entrypoint 429 line
name the route pattern whatever the status. A request chi never routed,
such as a dev-mode CORS preflight to the receiver, logs (unmatched).
The access log keeps only the scheme and host of a Referer. The
received line names the entrypoint and webhook IDs, and the
unknown-entrypoint line logs no path. The pattern is
middleware.ReceiverRoute, which routes.go registers the receiver with.

The README says the proxy's access log holds the UUID and must be
protected like the URL, and that a log target writes the request's
headers as received. Its access-log, rate-limit, Sentry and log-size
passages now match the code.

Model: opus-5-5
2026-10-15 14:58:35 +02:00
clawbot c0f7cc29fd Leave a delivery cut off by a stop for the next start (closes #527)
check / check (push) Waiting to run
Stopping the engine cancels the context every http and slack request is
sent with. An attempt cut off that way was recorded as failed, which for
a target with no retries, or on a last retry, ended the delivery failed
and nothing sent it again.

The shared retry core now records nothing for a failed attempt when the
engine's context is cancelled: no result, no circuit breaker outcome, no
status change, no count in the attempt metrics. The delivery stays
pending or retrying and startup recovery sends it again. A target's own
timeout is still a failed attempt. The README says what happens to an
attempt in flight at a stop and no longer claims every attempt is
recorded.

Model: opus-5-5
2026-10-15 12:29:17 +02:00
clawbot f14fe02011 Store delivery and archive times without the monotonic reading (closes #529)
check / check (push) Waiting to run
finished_at, archived_at and the updated_at stamp a re-dispatch
writes came from a bare time.Now(). The SQLite driver stores a time
as time.Time.String(), which appends the monotonic clock reading
(" m=+…") to a time read from time.Now(). All three now come from
GORM's own clock, NowFunc(), which stamps created_at, so they are
stored in the same form. The tests for each write read the stored
text back.

Model: opus-5-5
2026-10-15 11:15:35 +02:00
clawbot 573f7f986a Build the image's static binary with netgo, osusergo and sqlite_omit_load_extension (closes #525)
check / check (push) Successful in 7m24s
The image's static relink linked glibc's getaddrinfo (from Go's cgo DNS
resolver) and dlopen (from SQLite extension loading). A static binary
still needs glibc's shared libraries at run time for both, and the
Alpine runtime has none. The relink now passes the standard build tags
netgo, osusergo and sqlite_omit_load_extension through a GO_TAGS
variable, which the Makefile's build target hands to go build as -tags
next to GO_LDFLAGS, so the version stamp is composed as before.

Model: opus-5-5
2026-10-13 10:43:47 +02:00
clawbot 373a2f828d Describe code in its own terms, not by an earlier build's data (closes #522)
check / check (push) Successful in 8m12s
Several comments justified code by what an earlier build would have
left in a database or on disk. Each now says what the code does:
reserveSQLiteFile sets the mode on every open whoever created the
file, and expired treats a session lacking a deadline's timestamp as
expired. The comments on retention, resubmitted events, archive
pruning, target headers, the redactor and the delivery result view
drop their mentions of older rows. parseMaxRetries loses its
paragraph on targets saved before the ceiling existed, and neither
BeforeSave nor retentionCutoff cites a migration. Comments only.

Model: opus-5-5
2026-10-13 09:57:24 +02:00
clawbot 267d5b12f2 Escape the database path in the SQLite connection string (closes #518)
check / check (push) Successful in 7m13s
SQLite reads a file: URI's path only up to the first ? or #, and
decodes %XX escapes in it. With a data directory such as
/srv/hooks#1, every database opened the one file /srv/hooks. The
connection string is now built with url.URL, which escapes the path,
and OmitHost keeps a relative path relative.

Tests open databases in a directory whose name holds #, ? and %, and
at a relative path, and check through PRAGMA database_list that each
opened the file asked for. A handlers test comment that blamed # in
the temporary directory's path is dropped, as that is no longer true.

Model: opus-5-5
2026-10-13 09:30:24 +02:00
142 changed files with 6753 additions and 5865 deletions
+14 -18
View File
@@ -76,8 +76,6 @@ RUN --network=none node_modules/.bin/prettier --check '**/*.md'
# depends on it. # depends on it.
# #
# golangci/golangci-lint:v2.14.0 (Debian-based), 2026-09-24 # golangci/golangci-lint:v2.14.0 (Debian-based), 2026-09-24
# Using Debian-based image because mattn/go-sqlite3 (CGO) does not
# compile on Alpine musl (off64_t is a glibc type).
FROM golangci/golangci-lint:v2.14.0@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint FROM golangci/golangci-lint:v2.14.0@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint
WORKDIR /src WORKDIR /src
@@ -167,9 +165,7 @@ RUN --mount=type=tmpfs,target=/tmp/tests,size=512m \
# Build stage # Build stage
# golang:1.26.1-bookworm (Debian-based), 2026-03-17 # golang:1.26.1-bookworm (Debian-based), 2026-03-17
# Using Debian-based image because gorm.io/driver/sqlite pulls in # The image ships git and make, which the version step below uses.
# mattn/go-sqlite3 (CGO), which does not compile on Alpine musl. The image
# ships git and make, which the version step below uses.
FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4 AS builder FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4 AS builder
# Nothing is wanted from the lint and test phases or from the stylesheet and # Nothing is wanted from the lint and test phases or from the stylesheet and
@@ -211,20 +207,11 @@ RUN version="$(make version VERSION="$VERSION")"; \
fi fi
# Builds through the Makefile's build target, which runs script/assets # Builds through the Makefile's build target, which runs script/assets
# (Alpine.js, extracted from its tarball in 3p/) first. # (Alpine.js, extracted from its tarball in 3p/) first. The target builds
# with cgo off, so the binary is statically linked and runs on the Alpine
# runtime image below as it is.
RUN make build VERSION="$VERSION" RUN make build VERSION="$VERSION"
# Rebuild with static linking for Alpine runtime.
# make build already verified compilation.
# The CGO binary from `make build` is dynamically linked against glibc,
# which doesn't exist on Alpine (musl). Rebuild with static linking so
# the binary runs on Alpine without glibc.
#
# The static flags go in through GO_LDFLAGS rather than a -ldflags of
# their own: the build target composes them with the -X that stamps the
# version, so this relink cannot silently drop the stamp.
RUN CGO_ENABLED=1 make build VERSION="$VERSION" GO_LDFLAGS='-extldflags "-static"'
# Runtime stage # Runtime stage
# alpine:3.21, 2026-03-17 # alpine:3.21, 2026-03-17
FROM alpine:3.21@sha256:c3f8e73fdb79deaebaa2037150150191b9dcbfba68b4a46d70103204c53f4709 FROM alpine:3.21@sha256:c3f8e73fdb79deaebaa2037150150191b9dcbfba68b4a46d70103204c53f4709
@@ -268,8 +255,17 @@ EXPOSE 8080
# port. # port.
ENV BIND_ADDRESS=0.0.0.0 ENV BIND_ADDRESS=0.0.0.0
# The probe requests the container's own address, not loopback, so a
# listener that answers only on loopback (an empty or loopback
# BIND_ADDRESS) fails it. On a network with IPv4, hostname -i prints the
# container's IPv4 address, which 0.0.0.0 and :: both answer. On an
# IPv6-only network it prints none, and the probe requests the IPv6
# address /etc/hosts lists under the container's hostname instead, which
# only :: answers.
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \ HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://localhost:8080/.well-known/healthcheck || exit 1 CMD addr="$(hostname -i 2>/dev/null | cut -d ' ' -f 1)"; \
[ -n "$addr" ] || addr="[$(getent hosts "$(hostname)" | cut -d ' ' -f 1)]"; \
wget --no-verbose --tries=1 --spider "http://$addr:8080/.well-known/healthcheck" || exit 1
ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]
CMD ["/app/webhooker"] CMD ["/app/webhooker"]
+5 -10
View File
@@ -16,15 +16,6 @@ VERSION ?= $(shell script/version)
# exists to correct. # exists to correct.
override VERSION := $(or $(strip $(VERSION)),$(shell script/version)) override VERSION := $(or $(strip $(VERSION)),$(shell script/version))
# Extra linker flags for the build target. The static relink in the
# Dockerfile adds -extldflags here rather than passing its own -ldflags,
# so composing flags cannot drop the version stamp.
#
# The build target itself always passes -trimpath and -s -w, as the Go
# Dockerfile in REPO_POLICIES.md does: no build paths, symbol table or
# debug information in the binary.
GO_LDFLAGS ?=
bootstrap: bootstrap:
@script/bootstrap @script/bootstrap
@@ -52,8 +43,12 @@ fmt-check:
check: check:
@script/check @script/check
# Built with cgo off, -trimpath and -s -w, as the Go Dockerfile in
# REPO_POLICIES.md does: the binary carries no build paths, symbol table or
# debug information, and on Linux, the image's included, it is statically
# linked.
build: assets build: assets
go build -trimpath -ldflags '$(strip -s -w -X main.version=$(VERSION) $(GO_LDFLAGS))' -o bin/webhooker ./cmd/webhooker CGO_ENABLED=0 go build -trimpath -ldflags '-s -w -X main.version=$(VERSION)' -o bin/webhooker ./cmd/webhooker
run: build run: build
./bin/webhooker ./bin/webhooker
+305 -255
View File
@@ -310,16 +310,17 @@ the namespace, whatever the process bound. Exposure is decided at the publish
flag instead — `-p 127.0.0.1:8080:8080` rather than `-p 8080:8080` — which is flag instead — `-p 127.0.0.1:8080:8080` rather than `-p 8080:8080` — which is
the operator's to choose and is what [Running with Docker](#running-with-docker) the operator's to choose and is what [Running with Docker](#running-with-docker)
shows. A loopback bind inside a container buys nothing and makes the process shows. A loopback bind inside a container buys nothing and makes the process
unreachable through its own published port. unreachable through its own published port. A container on an IPv6-only Docker
network has no IPv4 address for `0.0.0.0` to answer on, so there it needs
`BIND_ADDRESS=::`.
The value must be an IP address literal: The value must be an IP address literal:
- `127.0.0.1` — loopback only (the binary's default). Use this with a reverse - `127.0.0.1` — loopback only (the binary's default). Use this with a reverse
proxy on the same host. proxy on the same host.
- `0.0.0.0` — every IPv4 address. The image's default; on a bare host, only - `0.0.0.0` — every IPv4 address and no IPv6 address. The image's default; on a
behind a firewall on the port. bare host, only behind a firewall on the port.
- `::` — every address, IPv6 and (on Linux, with the default - `::` — every IPv6 address and every IPv4 address.
`net.ipv6.bindv6only=0`) IPv4 as well.
- A specific address such as `10.0.0.5` — that interface only. - A specific address such as `10.0.0.5` — that interface only.
An **empty** value is treated as unset, as everywhere else here, and takes the An **empty** value is treated as unset, as everywhere else here, and takes the
@@ -340,7 +341,8 @@ machine that has no such interface — is a valid literal, so it reaches the
listener and fails there. The process logs the bind error and exits non-zero listener and fails there. The process logs the bind error and exits non-zero
rather than staying up with nothing listening. The effective value is in the rather than staying up with nothing listening. The effective value is in the
`bindAddress` field of the startup log line, which is the way to check what a `bindAddress` field of the startup log line, which is the way to check what a
running deployment actually bound. running deployment actually bound. On a bare host, `ss -ltn` lists the socket
itself as `0.0.0.0:8080` for a `0.0.0.0` bind and as `*:8080` for `::`.
#### Metrics credentials #### Metrics credentials
@@ -668,9 +670,9 @@ Either set a literal or leave the variable out entirely.
Overriding `BIND_ADDRESS` to a loopback address in a container — by that route Overriding `BIND_ADDRESS` to a loopback address in a container — by that route
or deliberately — makes the container unreachable from outside its namespace or deliberately — makes the container unreachable from outside its namespace
even with `-p`. The published port answers nothing, and the health check fails even with `-p`. The published port answers nothing, and the health check fails
too: it requests `http://localhost:8080`, `localhost` resolves to `::1` first, too: it requests the container's own address (its IPv4 address, or its IPv6
and a `127.0.0.1` bind is not listening there. The container then goes address on an IPv6-only network), where a loopback bind is not listening. The
`unhealthy` about **65 seconds** after start — from container then goes `unhealthy` about **65 seconds** after start — from
`HEALTHCHECK --start-period=5s --interval=30s --retries=3`, so failing probes at `HEALTHCHECK --start-period=5s --interval=30s --retries=3`, so failing probes at
5s, 35s and 65s, and `unhealthy` on the third. (Docker's probe cadence during 5s, 35s and 65s, and `unhealthy` on the third. (Docker's probe cadence during
the start period has changed between versions; re-derive from those three values the start period has changed between versions; re-derive from those three values
@@ -726,8 +728,8 @@ the credentials.
address (directly, or through anything that can rewrite source addresses, address (directly, or through anything that can rewrite source addresses,
such as NAT or a published container port). The `remoteIP` field of the such as NAT or a published container port). The `remoteIP` field of the
`http request` log line for a request that came through the proxy shows `http request` log line for a request that came through the proxy shows
that address; the health check's own lines show `::1`. See that address; the health check's own lines show the container's own
[Trusted proxies](#trusted-proxies). address. See [Trusted proxies](#trusted-proxies).
- Leave `BIND_ADDRESS` and `DATA_DIR` unset: the image sets `BIND_ADDRESS` - Leave `BIND_ADDRESS` and `DATA_DIR` unset: the image sets `BIND_ADDRESS`
to `0.0.0.0`, and `DATA_DIR` defaults to `/var/lib/webhooker`. to `0.0.0.0`, and `DATA_DIR` defaults to `/var/lib/webhooker`.
- Everything else is optional; see [Configuration](#configuration). - Everything else is optional; see [Configuration](#configuration).
@@ -802,7 +804,12 @@ comes up, serves pages, and is broken in a way nothing reports.
proxy's log names it regardless. nginx's default `combined` format already proxy's log names it regardless. nginx's default `combined` format already
logs `$remote_addr`; do not replace it with one that drops the client logs `$remote_addr`; do not replace it with one that drops the client
address, and retain those logs as long as you would want to answer a question address, and retain those logs as long as you would want to answer a question
about traffic. about traffic. That log also records the full request path, so it holds the
entrypoint UUID of every event a sender posts. webhooker's own log does not,
apart from the line a `log` target writes. Protect the proxy's log as you
would the entrypoint URLs themselves: anyone who can read it can post events
to every entrypoint in it (see
[The entrypoint URL is the authentication secret](#the-entrypoint-url-is-the-authentication-secret)).
### nginx ### nginx
@@ -828,7 +835,9 @@ server {
client_max_body_size 1m; client_max_body_size 1m;
# $remote_addr is the client. webhooker's own log names it, as # $remote_addr is the client. webhooker's own log names it, as
# clientIP, only while TRUSTED_PROXIES covers this proxy. # clientIP, only while TRUSTED_PROXIES covers this proxy. The
# request path in this log holds the entrypoint UUID, so protect
# the file like the entrypoint URL itself.
access_log /var/log/nginx/webhooker.access.log combined; access_log /var/log/nginx/webhooker.access.log combined;
location / { location / {
@@ -1160,7 +1169,16 @@ What that means for an operator:
- **The URL is a capability, so treat it as a secret.** Keep it out of logs, - **The URL is a capability, so treat it as a secret.** Keep it out of logs,
ticket bodies, chat messages and screenshots. Anyone who reads it anywhere can ticket bodies, chat messages and screenshots. Anyone who reads it anywhere can
post events as that sender. post events as that sender. Apart from a `log` target's line, webhooker's own
log lines never carry the UUID: they name a request to the receiver by the
route pattern `/h/{uuid}`, the `webhook request received` line names the
entrypoint by its `entrypoint_id`, and the access log keeps only the scheme
and host of a `Referer`. A `log` target writes the request's headers as
received, so a `Referer` naming the entrypoint URL appears in its line, as it
does when a sender has followed a redirect from `http://` to `https://`. The
reverse proxy's access log does carry the UUID, in the request path, so it
must be protected like the URL itself (see
[Deployment behind a reverse proxy](#deployment-behind-a-reverse-proxy)).
- **Rotating means minting a new entrypoint, not changing a key.** There is no - **Rotating means minting a new entrypoint, not changing a key.** There is no
way to rotate the UUID in place. To retire one, delete the entrypoint (or way to rotate the UUID in place. To retire one, delete the entrypoint (or
deactivate it, which answers `410`) and create a new one, then point the deactivate it, which answers `410`) and create a new one, then point the
@@ -1235,34 +1253,37 @@ markup. The CSP build runs no expressions, so every Alpine directive in
`static/js/app.js`: `x-data="collapsible"` and `@click="toggle"`, never `static/js/app.js`: `x-data="collapsible"` and `@click="toggle"`, never
`x-data="{ open: false }"` or `@click="open = !open"`. `x-data="{ open: false }"` or `@click="open = !open"`.
A browser test in `internal/server` loads the webhook page, its edit pages and A browser test in `internal/server` loads the sign-in page, the webhook page,
the event log under the real policy and checks that: the add entrypoint form its edit pages, the event log, the webhook list, the settings page, the profile
stays hidden until Add is clicked; for every target type, the targets section's page and the `404` error page shown for a path that does not exist, under the
Add shows only a choice of type with Next and Cancel, Next shows only that real policy and checks that: the add entrypoint form stays hidden until Add is
type's fields (no url field for `database` or `log`), Cancel at either step clicked; for every target type, the targets section's Add shows only a choice of
closes the form, and saving adds the target; a refused target comes back with type with Next and Cancel, Next shows only that type's fields (no url field for
its form open, the values entered and the reason, and after Cancel the next Add `database` or `log`), Cancel at either step closes the form, and saving adds the
starts with an empty form and no reason; a refused save on the target edit page target; a refused target comes back with its form open, the values entered and
and on the webhook edit page comes back with the reason and every value entered; the reason, and after Cancel the next Add starts with an empty form and no
the Copy button beside an entrypoint URL reads "Copied" once clicked; an reason; a refused save on the target edit page and on the webhook edit page
entrypoint's Edit button shows its edit form in place of its description and comes back with the reason and every value entered; the Copy button beside an
hides until the form closes, Cancel hides the form and drops what was typed, as entrypoint URL reads "Copied" once clicked; an entrypoint's Edit button shows
does leaving the page and going back to it, and Save changes the description; of its edit form in place of its description and hides until the form closes,
the recent events on the webhook page only the newest starts expanded, each Cancel hides the form and drops what was typed, as does leaving the page and
expands and collapses, and Open leads to the event's own page; of the events in going back to it, and Save changes the description; of the recent events on the
the event log only the newest starts expanded, and an event there expands and webhook page only the newest starts expanded, each expands and collapses, and
collapses when its row's caret or its ID is clicked, and from the keyboard, but Open leads to the event's own page; of the events in the event log only the
not when its ID is selected with the mouse, and a delivery's attempts inside it newest starts expanded, and an event there expands and collapses when its row's
expand and collapse; and at phone width the menu button opens and closes the caret or its ID is clicked, and from the keyboard, but not when its ID is
mobile menu, and neither the webhook page nor the event log, with a delivery's selected with the mouse, and a delivery's attempts inside it expand and
attempts open, scrolls sideways or cuts anything off at the page's or a card's collapse; at phone width the menu button opens and closes the mobile menu, and
edge. It also fails if the browser reports a console warning or error, an neither the webhook page nor the event log, with a delivery's attempts open,
uncaught exception, or anything the policy refused. `make check` and the image scrolls sideways or cuts anything off at the page's or a card's edge; and a
build lint it but do not run it, and `make test` leaves it out (its file is target's Delete asks first, naming the target, sends nothing when the answer is
built only with the `browser` build tag). Run it with `make test-browser` after no and deletes the target when it is yes. It also fails if the browser reports a
changing `templates/` or `static/js/`: that builds `Dockerfile.browser`, which console warning or error, an uncaught exception, or anything the policy refused.
runs the test in a digest-pinned headless browser image, so the host needs no `make check` and the image build lint it but do not run it, and `make test`
browser. leaves it out (its file is built only with the `browser` build tag). Run it with
`make test-browser` after changing `templates/` or `static/js/`: that builds
`Dockerfile.browser`, which runs the test in a digest-pinned headless browser
image, so the host needs no browser.
The package's tarball is committed as `3p/alpinejs-csp-3.14.9.tgz`, byte for The package's tarball is committed as `3p/alpinejs-csp-3.14.9.tgz`, byte for
byte as the npm registry publishes it. It is a dependency, not this repo's build byte as the npm registry publishes it. It is a dependency, not this repo's build
@@ -1295,7 +1316,9 @@ and commit the regenerated file with the change. Tailwind takes classes only
from the files that `input.css` names in its `@source` lines; a class written in from the files that `input.css` names in its `@source` lines; a class written in
any other file is not generated until that file is named there too. `make check` any other file is not generated until that file is named there too. `make check`
and the image build fail when the committed file differs from what `make css` and the image build fail when the committed file differs from what `make css`
generates. `static/css/style.css` is hand-written and is not generated. generates. `static/css/style.css` is hand-written and is not generated. The
pages' Content-Security-Policy refuses inline styles, so a template styles an
element with a class, never a `style` attribute.
`make css` runs the Tailwind standalone CLI in Docker, at the version and sha256 `make css` runs the Tailwind standalone CLI in Docker, at the version and sha256
pinned in the Dockerfile's stylesheet stages; it is never installed on the host. pinned in the Dockerfile's stylesheet stages; it is never installed on the host.
@@ -1317,20 +1340,22 @@ webhooker solves this by acting as a durable intermediary:
nothing is lost if downstream targets are unavailable. nothing is lost if downstream targets are unavailable.
2. **Guaranteed delivery** — Events are queued for delivery to each configured 2. **Guaranteed delivery** — Events are queued for delivery to each configured
target. Failed deliveries are retried with configurable backoff. Every target. Failed deliveries are retried with configurable backoff. Delivery
delivery attempt is logged with status codes, response bodies, and timing. attempts are logged with status codes, response bodies, and timing.
**That guarantee is at-least-once, not exactly-once.** When a send reaches **That guarantee is at-least-once, not exactly-once.** When a send reaches
its target but the write recording that outcome fails, the delivery is its target but the write recording that outcome fails, the delivery is
deliberately left in a recoverable state rather than marked done — losing a deliberately left in a recoverable state rather than marked done — losing a
delivery is the worse failure — so the pending sweep picks it up about delivery is the worse failure — so the sweep picks it up about fifteen
fifteen minutes later, or the next restart does, and the target receives a minutes later, or the next restart does, and the target receives a payload
payload it already got. webhooker adds no delivery identifier of its own to it already got. A send still in flight when the service stops is left the
an outbound request, so **make your receiver idempotent** against whatever same way: its attempt is not recorded, and the next start sends it again
the payload itself carries. (see Shutdown). webhooker adds no delivery identifier of its own to an
outbound request, so **make your receiver idempotent** against whatever the
payload itself carries.
3. **Observability** — Full request/response logging for every webhook received 3. **Observability** — Full request/response logging for every webhook received
and every delivery attempted. Prometheus metrics expose volume, latency, and and for delivery attempts. Prometheus metrics expose volume, latency, and
error rates. The web UI provides real-time visibility into event flow. error rates. The web UI provides real-time visibility into event flow.
4. **Fan-out** — A single incoming webhook can be delivered to multiple targets 4. **Fan-out** — A single incoming webhook can be delivered to multiple targets
@@ -1368,10 +1393,10 @@ It uses:
management management
- **[go-chi](https://github.com/go-chi/chi)** for HTTP routing - **[go-chi](https://github.com/go-chi/chi)** for HTTP routing
- **[GORM](https://gorm.io)** for database access with - **[GORM](https://gorm.io)** for database access with
**[modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite)** as the runtime **[modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite)**, which is pure
SQLite driver. Note: `gorm.io/driver/sqlite` transitively depends on Go, as the SQLite driver. `gorm.io/driver/sqlite` also imports
`mattn/go-sqlite3`, which requires CGO at build time (see [Docker](#docker) `mattn/go-sqlite3`, whose stub for builds without cgo is linked into the
section) binary but never used
- **[slog](https://pkg.go.dev/log/slog)** (stdlib) for structured logging with - **[slog](https://pkg.go.dev/log/slog)** (stdlib) for structured logging with
TTY detection (text for dev, JSON for prod) TTY detection (text for dev, JSON for prod)
- **[gorilla/sessions](https://github.com/gorilla/sessions)** for encrypted - **[gorilla/sessions](https://github.com/gorilla/sessions)** for encrypted
@@ -1494,47 +1519,35 @@ the rotation chosen below that (none, monthly, daily or hourly). Both are
validated as on the add target form, and the webhook and its targets are created validated as on the add target form, and the webhook and its targets are created
together or not at all. together or not at all.
| Field | Type | Description | | Field | Type | Description |
| ---------------- | ------- | ----------------------------------------------------------- | | ---------------- | ------- | --------------------------------------------- |
| `id` | UUID | Primary key | | `id` | UUID | Primary key |
| `user_id` | UUID | Foreign key → User | | `user_id` | UUID | Foreign key → User |
| `name` | string | Human-readable name | | `name` | string | Human-readable name |
| `description` | string | Optional description | | `description` | string | Optional description |
| `retention_days` | integer | Days to retain events (default: 30; 0 means retain forever) | | `retention_days` | integer | Days to retain events; 0 means retain forever |
**Relations:** Belongs to User. Has many Entrypoints. Has many Targets. **Relations:** Belongs to User. Has many Entrypoints. Has many Targets.
The `retention_days` field controls how long event data is kept in the webhook's The `retention_days` field controls how long event data is kept in the webhook's
dedicated database before automatic cleanup. dedicated database before automatic cleanup.
Setting `retention_days` to `0` means "retain events forever". Because the The value is stored as entered. Setting `retention_days` to `0` means "retain
column carries a default of 30, a literal zero cannot survive an insert, so a events forever": the retention reaper skips the webhook entirely, and the web UI
zero is rewritten on save to a sentinel of `365 * 1000` days displays its retention as "forever" rather than as a day count.
(`database.RetentionForeverDays`). The retention reaper recognises that sentinel
and skips the webhook entirely, and the web UI displays such a webhook's
retention as "forever" rather than as a day count.
Submitted `retention_days` values therefore fall into three bands, not two: The new webhook and edit forms accept `0`, or a whole number of days from `1` up
to `database.MaxFiniteRetentionDays` (106751 days, about 292 years). Anything
- `1` up to `database.MaxFiniteRetentionDays` (106751 days, about 292 years) is else, a negative value or one above that cap included, is rejected with a 400. A
accepted as a finite retention. blank field gives the default of 30 days (`database.DefaultRetentionDays`) on
- Above that ceiling but below the retain-forever sentinel of 365000 the new webhook form, and leaves the stored value unchanged on the edit form.
(`database.RetentionForeverDays`) is rejected with a 400. This is the band the
cap exists for.
- `0`, and `365000` or above, are accepted and mean retain forever, collapsing
to the sentinel — `0` in `Webhook.BeforeSave`, the large values in
`parseRetentionDays`. The large values are not out of range: the edit form
pre-fills the sentinel for a retain-forever webhook, so submitting that form
back unchanged has to keep meaning "forever".
A negative value is in none of the three: `parseRetentionDays` rejects it with a
400 before `BeforeSave` ever sees it.
The cap is not arbitrary: the reaper computes its cutoff as a `time.Duration`, The cap is not arbitrary: the reaper computes its cutoff as a `time.Duration`,
an int64 nanosecond count, and a longer period overflows it. An overflowed an int64 nanosecond count, and a longer period overflows it. An overflowed
cutoff lands in the future, where it matches every row, so the sweep would cutoff lands in the future, where it matches every row, so the sweep would
delete every event the webhook has instead of none. The reaper also clamps the delete every event the webhook has instead of none. The reaper also clamps the
value it reads to the cap, so no stored value can trigger that either. value it reads to the cap, and reads a negative value as forever, so no stored
value can trigger that either.
#### Entrypoint #### Entrypoint
@@ -1566,31 +1579,34 @@ did not arrive on the URL and counts in neither.
A delivery destination for events. Each target defines where and how events A delivery destination for events. Each target defines where and how events
should be forwarded. should be forwarded.
| Field | Type | Description | | Field | Type | Description |
| ------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ------------------ | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id` | UUID | Primary key | | `id` | UUID | Primary key |
| `webhook_id` | UUID | Foreign key → Webhook | | `webhook_id` | UUID | Foreign key → Webhook |
| `name` | string | Human-readable name | | `name` | string | Human-readable name |
| `type` | TargetType | One of: `http`, `slack`, `database`, `log` | | `type` | TargetType | One of: `http`, `slack`, `database`, `log` |
| `active` | boolean | Whether deliveries are enabled (default: true) | | `active` | boolean | Whether deliveries are enabled (default: true) |
| `config` | JSON text | Type-specific configuration | | `url` | string | Where an `http` or `slack` target sends each event. It is a credential: the target list shows only its scheme and host, and only the target edit form shows it whole |
| `max_retries` | integer | Total delivery attempts for `http` and `slack` targets, not retries on top of the first: 0 is a single fire-and-forget attempt with no retries and no circuit breaker, and a value of N makes N attempts in all, with exponential backoff and a per-target circuit breaker. Ignored by `database` and `log` targets. The web UI labels it Delivery attempts | | `headers` | JSON text | An `http` target's extra request headers, a JSON object of name to value |
| `timeout_seconds` | integer | An `http` target's per-request timeout; 0 uses the shared 30-second client timeout |
| `forward_query` | boolean | Whether an `http` target is sent the query string each event arrived with |
| `archive_expiry` | integer | How long a `database` target keeps each archived event, as a Go `time.Duration` (nanoseconds); 0 keeps them forever |
| `archive_rotation` | string | How often a `database` target starts a new archive file: `none`, `monthly`, `daily` or `hourly` |
| `max_attempts` | integer | How many times an `http` or `slack` target attempts each delivery in all: at least 1, and 1 by default. Ignored by `database` and `log` targets. The web UI labels it Delivery attempts |
**Relations:** Belongs to Webhook. Has many Deliveries. **Relations:** Belongs to Webhook. Has many Deliveries.
**Target types:** **Target types:**
- **`http`** — Forward the event as an HTTP POST to a configured URL. - **`http`** — Forward the event as an HTTP POST to a configured URL. With
`max_retries` is the total number of delivery attempts, not retries on top of `max_attempts` at 1 the target makes a single attempt; above 1 it retries a
the first: when `max_retries` is 0 (the default), the target operates in failed delivery with exponential backoff, up to that many attempts in all,
fire-and-forget mode, a single attempt with no retries and no circuit breaker; behind a per-target circuit breaker.
a value of N makes up to N attempts in all, retrying failed deliveries with
exponential backoff and protecting them with a per-target circuit breaker.
- **`slack`** — Post the event as a formatted message to a Slack-compatible - **`slack`** — Post the event as a formatted message to a Slack-compatible
incoming webhook URL (`webhookUrl` in `config`). It is built on the same HTTP incoming webhook URL, its `url`. It is built on the same HTTP core as `http`
core as `http` and honours `max_retries` identically, circuit breaker and honours `max_attempts` identically, circuit breaker included. See the
included. See the Slack target section under "Per-Webhook Event Databases" for Slack target section under "Per-Webhook Event Databases" for the message
the message format. format.
- **`database`** — Archive the full event as a row into the target's own archive - **`database`** — Archive the full event as a row into the target's own archive
database (`archive-{webhook_name}-{target_name}-{target_uuid}.db`) for database (`archive-{webhook_name}-{target_name}-{target_uuid}.db`) for
long-term retention, with an optional creation-validated expiry (default: keep long-term retention, with an optional creation-validated expiry (default: keep
@@ -1606,29 +1622,28 @@ should be forwarded.
- **`log`** — Write the event to the application log (stdout). Useful for - **`log`** — Write the event to the application log (stdout). Useful for
debugging. debugging.
The `config` field stores type-specific configuration as JSON (e.g., destination Each type uses only its own settings, and the ones it does not use stay empty:
URL, custom headers, timeout settings). `http` uses `url`, `headers`, `timeout_seconds` and `forward_query`; `slack`
uses `url`; `database` uses `archive_expiry` and `archive_rotation`; `log` uses
none. The add target form, the new webhook form and the target edit form check
every setting before it is saved, and each setting is a column of its own, so
nothing that reads a target parses its settings or has to handle one it cannot
read.
**`http` target configuration:** An `http` target POSTs the event to its `url`. Its `headers` are applied last,
so they win over the event's own forwarded headers.
| Key | Type | Description | `forward_query` is off by default, and the target URL is then sent exactly as
| -------------- | ------------- | ----------------------------------------------------------------------------------------- |
| `url` | string | Destination the event is POSTed to |
| `headers` | object | Extra request headers, applied last so they win over the event's own forwarded headers |
| `timeout` | integer (sec) | Per-target request timeout; unset (or 0) uses the shared 30-second client timeout |
| `forwardQuery` | boolean | Pass the query string each event arrived with on to the target; unset (or false) does not |
`forwardQuery` is off by default, and the target URL is then sent exactly as
configured. On, each delivery appends the event's query string to the target configured. On, each delivery appends the event's query string to the target
URL, joined with `&` when the URL already has a query string of its own; a URL, joined with `&` when the URL already has a query string of its own; a
replayed delivery and a resubmitted event's deliveries do the same. Both target replayed delivery and a resubmitted event's deliveries do the same. Both target
forms offer it as "Pass the query string on to this target", and the target list forms offer it as "Pass the query string on to this target", and the target list
shows it when it is on. shows it when it is on.
`timeout` is capped at **300 seconds**, and the form rejects anything above it `timeout_seconds` is capped at **300 seconds**, and the form rejects anything
rather than substituting the cap. A delivery attempt holds one of the bounded above it rather than substituting the cap. A delivery attempt holds one of the
pool's workers for its whole duration, so an unbounded timeout would let a bounded pool's workers for its whole duration, so an unbounded timeout would let
single unresponsive destination stall the queue. a single unresponsive destination stall the queue.
`headers` rejects the names the delivery path or `net/http` writes regardless of `headers` rejects the names the delivery path or `net/http` writes regardless of
what is configured: `Host`, `Content-Length`, `Transfer-Encoding`, `Connection`, what is configured: `Host`, `Content-Length`, `Transfer-Encoding`, `Connection`,
@@ -1702,7 +1717,7 @@ The event log and the event's own page show the query string with the rest of
the request. The event log leaves out one larger than 32 KiB, as it does request the request. The event log leaves out one larger than 32 KiB, as it does request
headers, and links to the event's page, which shows it whole. The `database` and headers, and links to the event's page, which shows it whole. The `database` and
`log` targets carry it with the rest of the event. An `http` target receives it `log` targets carry it with the rest of the event. An `http` target receives it
only when its `forwardQuery` setting is on. only when its `forward_query` setting is on.
#### Delivery #### Delivery
@@ -1768,7 +1783,7 @@ is what bounds it.
#### DeliveryResult #### DeliveryResult
The result of a single delivery attempt. Every attempt (including retries) is The result of a single delivery attempt. Every attempt (including retries) is
individually logged for full observability. individually logged, except one cut off by a stop (see Shutdown).
| Field | Type | Description | | Field | Type | Description |
| --------------- | ------- | ----------------------------------------- | | --------------- | ------- | ----------------------------------------- |
@@ -1854,15 +1869,15 @@ These indexes on the per-webhook event databases are created by
`internal/db/migrations/events/001_schema.sql`, and declared in the model tags `internal/db/migrations/events/001_schema.sql`, and declared in the model tags
to match: to match:
| Table | Columns | Serves | | Table | Columns | Serves |
| ------------------ | ------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ------------------ | ------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `deliveries` | `status`, `deleted_at`, `finished_at`, `target_id` | Startup recovery, the retry and pending sweeps every 60 seconds and the queue-depth sampler every 30 seconds, which select deliveries by status, the webhook page's statistics and target list and the webhook list, which count each target's deliveries by status and when they finished, and the event log, which lists and counts the events with a failed delivery or one pending or retrying | | `deliveries` | `status`, `deleted_at`, `finished_at`, `target_id` | Delivery recovery at start and every 60 seconds and the queue-depth sampler every 30 seconds, which select deliveries by status, the webhook page's statistics and target list and the webhook list, which count each target's deliveries by status and when they finished, and the event log, which lists and counts the events with a failed delivery or one pending or retrying |
| `deliveries` | `event_id`, `deleted_at` | The event log, which loads each event's deliveries, and retention, which counts and deletes the deliveries of expired events | | `deliveries` | `event_id`, `deleted_at` | The event log, which loads each event's deliveries, and retention, which counts and deletes the deliveries of expired events |
| `delivery_results` | `delivery_id`, `deleted_at` | The event log, which loads the attempts of a page's deliveries, and retention, which deletes the attempts of expired events | | `delivery_results` | `delivery_id`, `deleted_at` | The event log, which loads the attempts of a page's deliveries, and retention, which deletes the attempts of expired events |
| `events` | `deleted_at`, `created_at` | The webhook page's statistics, which count recent events | | `events` | `deleted_at`, `created_at` | The webhook page's statistics, which count recent events |
| `events` | `resubmitted_from_id`, `deleted_at` | The event log, which counts the events resubmitted from each event on a page | | `events` | `resubmitted_from_id`, `deleted_at` | The event log, which counts the events resubmitted from each event on a page |
| `events` | `entrypoint_id`, `deleted_at`, `resubmitted_from_id`, `created_at` | The webhook page's entrypoint list, which counts the events that arrived on each entrypoint's URL within the retention period | | `events` | `entrypoint_id`, `deleted_at`, `resubmitted_from_id`, `created_at` | The webhook page's entrypoint list, which counts the events that arrived on each entrypoint's URL within the retention period |
| `events` | `created_at` | Retention, which selects expired events by age | | `events` | `created_at` | Retention, which selects expired events by age |
GORM's soft delete adds `deleted_at IS NULL` to these queries; retention leaves GORM's soft delete adds `deleted_at IS NULL` to these queries; retention leaves
it out. SQLite keeps no statistics on these tables, and without them it rates it out. SQLite keeps no statistics on these tables, and without them it rates
@@ -1981,11 +1996,10 @@ target UUID keeps the file name unique. A webhook named `Orders (EU)` with a
target named `Long-term archive` archives into target named `Long-term archive` archives into
`archive-orders-eu-long-term-archive-{target_uuid}.db`. `archive-orders-eu-long-term-archive-{target_uuid}.db`.
An optional `rotation` in the target's config JSON (e.g. `{"rotation":"daily"}`) A target's `archive_rotation` is `none`, the default, which keeps the one file,
is `none`, the default, which keeps the one file, or `monthly`, `daily` or or `monthly`, `daily` or `hourly`. A target that rotates writes each event to a
`hourly`. A target that rotates writes each event to a file named for the period file named for the period of the event's receive time, in UTC, put before the
of the event's receive time, in UTC, put before the `.db`: `.db`: `archive-orders-eu-long-term-archive-{target_uuid}-2026-10.db` monthly,
`archive-orders-eu-long-term-archive-{target_uuid}-2026-10.db` monthly,
`…-2026-10-01.db` daily and `…-2026-10-01-19.db` hourly. Each file holds exactly `…-2026-10-01.db` daily and `…-2026-10-01-19.db` hourly. Each file holds exactly
its period's events, and the first event of a new period starts the next file, its period's events, and the first event of a new period starts the next file,
so a finished period's file can be moved away like any archive. A changed so a finished period's file can be moved away like any archive. A changed
@@ -2020,12 +2034,11 @@ archiving without stopping the service; a moved or removed archive file is
recreated automatically on the next write, from the SQL files in recreated automatically on the next write, from the SQL files in
`internal/db/migrations/archive/`. A zero-length archive file is written to as a `internal/db/migrations/archive/`. A zero-length archive file is written to as a
new archive: SQLite opens it as an empty database, so it holds nothing to lose. new archive: SQLite opens it as an empty database, so it holds nothing to lose.
An optional `expiry` in the target's config JSON (e.g. `{"expiry":"720h"}`) is A target's `archive_expiry` is checked by the form that saves it — the default,
validated when the target is created — the default (unset or the literal `never` on the forms and 0 in the column, keeps rows forever — and rows older
`never`) keeps rows forever — and rows older than the expiry are pruned each than the expiry are pruned each time the archive is (re)opened. An archive write
time the archive is (re)opened. An archive write failure is never silent failure is never silent success: the delivery records a failed attempt with the
success: the delivery records a failed attempt with the error and is marked error and is marked failed.
failed.
Because reopens only happen on writes, an archive whose target has stopped Because reopens only happen on writes, an archive whose target has stopped
receiving events would never be pruned. A background **archive sweeper** closes receiving events would never be pruned. A background **archive sweeper** closes
@@ -2036,8 +2049,7 @@ traffic. The sweep never creates an archive — a target whose archive file does
not yet exist is skipped, not initialised — it takes the same per-target lock not yet exist is skipped, not initialised — it takes the same per-target lock
the write path uses, so it can never interleave with a write, and it leaves the the write path uses, so it can never interleave with a write, and it leaves the
archive closed afterwards so the move-the-file-away workflow keeps working. archive closed afterwards so the move-the-file-away workflow keeps working.
Archives with no expiry, or the expiry `never`, are not touched by the sweep at Archives with the expiry `never` are not touched by the sweep at all.
all.
For a target with several files, the write path prunes only the file it writes For a target with several files, the write path prunes only the file it writes
to, and the sweep prunes every one of them, taking the target's lock for one to, and the sweep prunes every one of them, taking the target's lock for one
@@ -2089,15 +2101,15 @@ writes go on meanwhile, since under WAL a reader never blocks a writer. Until
the open file is closed its `-wal` cannot be checkpointed past what the download the open file is closed its `-wal` cannot be checkpointed past what the download
reads, so a long download lets that `-wal` grow. reads, so a long download lets that `-wal` grow.
Deleting a webhook releases its archives: the delivery engine's cached archive Deleting a webhook releases its archives: its cached archive writers are dropped
writers are dropped and their file handles closed, so nothing lingers after the and their file handles closed, so nothing lingers after the webhook is gone. The
webhook is gone. The archive **files themselves are deliberately left on disk**. archive **files themselves are deliberately left on disk**. Unlike the event
Unlike the event database — per-webhook working storage that is hard-deleted database — per-webhook working storage that is hard-deleted with the webhook —
with the webhook — an archive is long-term storage an operator may still want to an archive is long-term storage an operator may still want to keep or move away
keep or move away for offline retention, and destroying it as a side effect of for offline retention, and destroying it as a side effect of deleting a webhook
deleting a webhook would be unrecoverable. Removing an `archive-….db` is the would be unrecoverable. Removing an `archive-….db` is the operator's call.
operator's call. Deleting a `database` target releases its writer the same way, Deleting a `database` target releases its writer the same way, and for the same
and for the same reason leaves its file alone. reason leaves its file alone.
The **Slack target type** sends webhook events as formatted messages to any The **Slack target type** sends webhook events as formatted messages to any
Slack-compatible incoming webhook URL (works with Slack, Mattermost, and other Slack-compatible incoming webhook URL (works with Slack, Mattermost, and other
@@ -2105,14 +2117,11 @@ compatible services). Each message includes event metadata (HTTP method, content
type, timestamp, body size) and the payload pretty-printed in a code block. JSON type, timestamp, body size) and the payload pretty-printed in a code block. JSON
payloads are automatically formatted with indentation for readability; non-JSON payloads are automatically formatted with indentation for readability; non-JSON
payloads are shown as raw text. Large payloads are truncated to keep messages payloads are shown as raw text. Large payloads are truncated to keep messages
reasonable. Config stores `webhookUrl` — the Slack/Mattermost incoming webhook reasonable. The target's `url` is the Slack/Mattermost incoming webhook
endpoint. That is the JSON key; the error text for a missing one reads endpoint.
`webhook_url is required`, which is the message, not the key.
The database uses the The database uses the
[modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) driver at runtime, [modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) driver.
though CGO is required at build time due to the transitive `mattn/go-sqlite3`
dependency from `gorm.io/driver/sqlite`.
#### Schema #### Schema
@@ -2147,7 +2156,7 @@ External Service
3. Capture full request as Event 3. Capture full request as Event
4. Create Delivery records for each active Target 4. Create Delivery records for each active Target
5. Build self-contained delivery.Task structs 5. Build self-contained delivery.Task structs
(target config + event data inline for (target settings + event data inline for
bodies < 16 KiB) bodies < 16 KiB)
6. Notify Engine via channel (no DB read needed) 6. Notify Engine via channel (no DB read needed)
│ │
@@ -2163,11 +2172,11 @@ External Service
▼ ▼ ▼ ▼ ▼ ▼
┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐
│ HTTP Target│ │ HTTP Target│ │ Log Target │ │ HTTP Target│ │ HTTP Target│ │ Log Target │
│(max_retries│ │(max_retries│ │ (stdout) │ │max_attempts│ │max_attempts│ │ (stdout) │
│ == 0) │ │ > 0, │ └────────────┘ │ == 1, │ │ > 1, │ └────────────┘
│ fire+forget│ │ backoff + │ │ one attempt│ │ backoff + │
└────────────┘ │ circuit │ └────────────┘ │ circuit │
│ breaker) │ │ breaker │
└────────────┘ └────────────┘
``` ```
@@ -2190,7 +2199,7 @@ preventing goroutine explosions regardless of queue depth.
Workers are the ONLY goroutines doing actual HTTP delivery. Workers are the ONLY goroutines doing actual HTTP delivery.
- **Retry backpressure with DB fallback:** When a retry timer fires and the - **Retry backpressure with DB fallback:** When a retry timer fires and the
retry channel is full, the timer is dropped — the delivery stays in `retrying` retry channel is full, the timer is dropped — the delivery stays in `retrying`
status in the database. A periodic sweep (every 60s) scans for these status in the database. The sweep (every 60s, see Recovery below) finds these
"orphaned" retries and re-queues them. No blocked goroutines, no unbounded "orphaned" retries and re-queues them. No blocked goroutines, no unbounded
timer chains. timer chains.
- **Bounded concurrency:** At most N deliveries (N = number of workers) are - **Bounded concurrency:** At most N deliveries (N = number of workers) are
@@ -2206,31 +2215,47 @@ This means:
buffer rather than spawning more goroutines. buffer rather than spawning more goroutines.
- **Independent results** — each worker records its own delivery result in the - **Independent results** — each worker records its own delivery result in the
per-webhook database without coordination. per-webhook database without coordination.
- **Graceful shutdown** — cancel the context, workers finish their current task - **Graceful shutdown** — the stop hook cancels the workers' context. A worker
and exit. The stop hook waits for the pool via `lifecycle.WaitForShutdown`, may still take tasks already queued before it exits. An `http` or `slack`
which bounds that wait by fx's stop timeout rather than blocking forever on a request in flight is cut off, and one started after the cancel fails at once.
wedged worker. On timeout it logs at `ERROR` and returns an error, and the Neither is recorded or counted in the metrics, and the next start sends the
goroutines that did not finish are still running — an unclean shutdown is delivery again (see Shutdown). A `database` or `log` task runs as normal. The
reported rather than hidden. stop hook waits for the pool via `lifecycle.WaitForShutdown`, which bounds
that wait by fx's stop timeout rather than blocking forever on a wedged
worker. On timeout it logs at `ERROR` and returns an error, and the goroutines
that did not finish are still running — an unclean shutdown is reported rather
than hidden.
**Recovery paths:** **Recovery:** one routine finds the deliveries the engine is not working on and
hands them back. It runs once when the engine starts, and then every 60 seconds;
that periodic run is the sweep. For every webhook it reads the `pending` and
`retrying` deliveries. One that already has a successful attempt is marked
`delivered` and not sent again. A `retrying` delivery gets a backoff timer for
what is left of its backoff, and is queued at once if nothing is left. A
`pending` delivery is queued on the delivery channel. A delivery the engine has
queued, is attempting, or holds a backoff timer for is left alone, at start as
on every sweep. This is what catches "orphaned" retries, whose in-memory timer
was dropped because the retry channel was full, and deliveries a stop or a crash
cut off: the database is the durable fallback that ensures no delivery is
permanently lost, even under extreme backpressure.
1. **Startup recovery:** When the engine starts, it scans all per-webhook The run at start and the sweep differ in two ways:
databases for `pending` and `retrying` deliveries. Pending deliveries are
sent to the delivery channel; retrying deliveries get backoff timers - At start every `pending` delivery is taken, however recent, so one a stop cut
scheduled. off moments ago is sent again at once (see Shutdown). The sweep takes only
2. **Periodic retry sweep (DB-mediated fallback):** Every 60 seconds the engine `pending` deliveries untouched for 15 minutes, at most 500 per webhook each
scans for `retrying` deliveries whose backoff period has elapsed. This time, so a delivery whose outcome the database will not record is sent again
catches "orphaned" retries — ones whose in-memory timer was dropped because every 15 minutes rather than every minute.
the retry channel was full. The database is the durable fallback that ensures - At start every webhook's database is opened, so that a missing one is reported
no retry is permanently lost, even under extreme backpressure. then (see Per-Webhook Event Databases). The sweep skips a webhook whose
database file is missing rather than create it.
**Changing a target's type does not migrate in-flight deliveries.** Only `http` **Changing a target's type does not migrate in-flight deliveries.** Only `http`
and `slack` targets own durable retries; `database` and `log` targets are and `slack` targets own durable retries; `database` and `log` targets are
fire-and-forget and never produce a `retrying` delivery. If a target's `type` is fire-and-forget and never produce a `retrying` delivery. If a target's `type` is
edited from a retrying type to a non-retrying (or unknown) one while one of its edited from a retrying type to a non-retrying (or unknown) one while one of its
deliveries is still `retrying`, both recovery paths above terminally mark that deliveries is still `retrying`, the recovery routine above terminally marks that
delivery `failed` and record a `DeliveryResult` naming the current target type delivery `failed` and records a `DeliveryResult` naming the current target type
as the reason, logging it at warn level. The delivery is not re-dispatched under as the reason, logging it at warn level. The delivery is not re-dispatched under
the new type — the operator never asked for that delivery — and the event itself the new type — the operator never asked for that delivery — and the event itself
remains stored in the per-webhook event database. Nothing is lost: the delivery remains stored in the per-webhook event database. Nothing is lost: the delivery
@@ -2240,11 +2265,11 @@ re-inject it to every currently active target.
### Circuit Breaker (HTTP and Slack Targets with Retries) ### Circuit Breaker (HTTP and Slack Targets with Retries)
`http` and `slack` targets with `max_retries` > 0 are protected by a `http` and `slack` targets with `max_attempts` > 1 are protected by a
**per-target circuit breaker** that prevents hammering a down target with **per-target circuit breaker** that prevents hammering a down target with
repeated failed delivery attempts. The circuit breaker is in-memory only and repeated failed delivery attempts. The circuit breaker is in-memory only and
resets on restart (which is fine — startup recovery rescans the database resets on restart (which is fine — the recovery routine rescans the database at
anyway). start anyway).
**States:** **States:**
@@ -2281,10 +2306,10 @@ anyway).
- **Cooldown:** 30 seconds in open state before probing - **Cooldown:** 30 seconds in open state before probing
**Scope:** Circuit breakers apply to **`http` and `slack` targets with **Scope:** Circuit breakers apply to **`http` and `slack` targets with
`max_retries` > 0**. The Slack target is built on the same HTTP core and hands `max_attempts` > 1**. The Slack target is built on the same HTTP core and hands
its own `max_retries` to the same retry path, so it gets a breaker with the same its own `max_attempts` to the same retry path, so it gets a breaker with the
5-failure / 30-second defaults. Fire-and-forget targets of either type same 5-failure / 30-second defaults. Single-attempt targets of either type
(`max_retries` == 0), database targets (local operations), and log targets (`max_attempts` == 1), database targets (local operations), and log targets
(stdout) do not use circuit breakers. (stdout) do not use circuit breakers.
When a circuit is open and a new delivery arrives, the engine marks the delivery When a circuit is open and a new delivery arrives, the engine marks the delivery
@@ -2317,7 +2342,7 @@ getting anywhere.
| Metric | Type | Meaning | | Metric | Type | Meaning |
| -------------------------------------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | -------------------------------------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `webhooker_events_received_total` | counter | Events received and durably stored. Compare against the delivery counters on one dashboard | | `webhooker_events_received_total` | counter | Events received and durably stored. Compare against the delivery counters on one dashboard |
| `webhooker_delivery_attempts_total` | counter | Delivery attempts actually dispatched to a target. A delivery a circuit breaker refused is not one: it is counted as a retry instead, but only when the refusal moves it into `retrying` | | `webhooker_delivery_attempts_total` | counter | Delivery attempts actually dispatched to a target. A delivery a circuit breaker refused is not one: it is counted as a retry instead, but only when the refusal moves it into `retrying`. An attempt cut off by a stop is not counted (see Shutdown) |
| `webhooker_deliveries_succeeded_total` | counter | Deliveries that reached `delivered` | | `webhooker_deliveries_succeeded_total` | counter | Deliveries that reached `delivered` |
| `webhooker_deliveries_failed_total` | counter | Deliveries that failed terminally and will not be retried | | `webhooker_deliveries_failed_total` | counter | Deliveries that failed terminally and will not be retried |
| `webhooker_delivery_retries_total` | counter | Deliveries put back into `retrying` | | `webhooker_delivery_retries_total` | counter | Deliveries put back into `retrying` |
@@ -2416,27 +2441,32 @@ log volume it caps rather than eliminates. A path that names no entrypoint is
recorded by the handler at `DEBUG`, and the aggregate limiter logs its own recorded by the handler at `DEBUG`, and the aggregate limiter logs its own
rejections at `DEBUG` and without the path, so neither appears at all under the rejections at `DEBUG` and without the path, so neither appears at all under the
default level. The per-entrypoint limiter is the loud one: it logs every default level. The per-entrypoint limiter is the loud one: it logs every
rejection at `WARN` with the request path, which on this route is rejection at `WARN`, naming the route pattern `/h/{uuid}` rather than the
attacker-controlled text. A client hammering a single invented path is served request path. A client hammering a single invented path is served
`RECEIVER_RATE_LIMIT` requests and has the rest of its aggregate budget rejected `RECEIVER_RATE_LIMIT` requests and has the rest of its aggregate budget rejected
there, so the aggregate limit is what bounds the _number_ of those `WARN` lines there, so the aggregate limit is what bounds the _number_ of those `WARN` lines
— to under ten times `RECEIVER_RATE_LIMIT` per minute per client IP, 1080 at the — to under ten times `RECEIVER_RATE_LIMIT` per minute per client IP, 1080 at the
defaults, where before it there was no bound at all. Their _width_ is bounded by defaults, where before it there was no bound at all. The access log is bounded
the field budgets below, the same ones the access log spends. The access log is by neither limit: every request is recorded once at `INFO`, served or rejected
bounded by neither limit: every request is recorded once at `INFO`, served or alike.
rejected alike.
What the access log does bound is the _content_ of those lines. A 3xx or 4xx What the access log does bound is the _content_ of those lines. On the receiver
response logs the chi route pattern — `/h/{uuid}`, `/user/{username}//`, or the route it logs the route pattern `/h/{uuid}` whatever the status, because the
literal `(unmatched)` when the request hit no route at all — in place of the concrete path carries the entrypoint UUID, which is the entrypoint's credential
concrete URL. Those are the outcomes an unauthenticated client can drive for (see
free: 404 and 429 on any invented receiver path, a login redirect on any [The entrypoint URL is the authentication secret](#the-entrypoint-url-is-the-authentication-secret)).
invented profile path. Logging the URL there would let a flood write text of its A request chi never routed, because it matched no route or because a middleware
own choosing, at a length of its own choosing, into the log. 2xx and 5xx answered it first, logs the literal `(unmatched)` whatever the status. In the
responses keep the concrete path — a success resolved against a static route or `dev` environment the CORS middleware answers a preflight that way, so a
against the operator's own data (on the receiver, against a stored entrypoint preflight to the receiver does not log its UUID either. On every other route, a
UUID), and a 5xx is a bug in this service, where the exact path is the evidence 3xx or 4xx response logs the chi route pattern, such as `/user/{username}//`, in
and no client can provoke one at will. place of the concrete URL. Those are the outcomes an unauthenticated client can
drive for free: a 404 on any invented path, a login redirect on any invented
profile path. Logging the URL there would let a flood write text of its own
choosing, at a length of its own choosing, into the log. 2xx and 5xx responses
keep the concrete path — a success resolved against a static route or against
the operator's own data, and a 5xx is a bug in this service, where the exact
path is the evidence and no client can provoke one at will.
The query string is never logged; it is replaced by the fixed marker The query string is never logged; it is replaced by the fixed marker
`?(redacted)`. It is client-chosen on every route, and `?(redacted)`. It is client-chosen on every route, and
@@ -2448,6 +2478,11 @@ to, `notice`, which names the line a page shows after an action, and the event
log's `show`, which picks the events it lists. A query string sent to an log's `show`, which picks the events it lists. A query string sent to an
entrypoint is not lost either: the event stores it, and the event log shows it. entrypoint is not lost either: the event stores it, and the event log shows it.
The `referer` field keeps only the scheme and host of the request's `Referer`,
and logs `(redacted)` for one without them, because its path can be an
entrypoint URL: a sender that follows the `http://` to `https://` redirect in
the nginx example sends the URL it was redirected from.
Client-supplied request content does not leave the host by the other route Client-supplied request content does not leave the host by the other route
either. The Sentry SDK attaches the request to every event it captures, either. The Sentry SDK attaches the request to every event it captures,
independently of the access log, and `SendDefaultPII=false` does not cover all independently of the access log, and `SendDefaultPII=false` does not cover all
@@ -2457,14 +2492,18 @@ are both taken unconditionally, the body precisely because these handlers call
body with `(redacted)`, drops cookies and the remote-address environment, and body with `(redacted)`, drops cookies and the remote-address environment, and
reduces the headers to a fixed allowlist — `Accept`, `Content-Length`, reduces the headers to a fixed allowlist — `Accept`, `Content-Length`,
`Content-Type`, `Host`, `Origin`, `Referer`, `User-Agent` and `X-Request-Id`. `Content-Type`, `Host`, `Origin`, `Referer`, `User-Agent` and `X-Request-Id`.
`Referer` is sent as its scheme and host only, because its path can be an
entrypoint URL; `internal/referer.SchemeAndHost` does that reduction for both
the access log's `referer` field and this hook.
The same hook rewrites the request URL. The SDK builds it as The same hook rewrites the request URL. The SDK builds it as
`scheme://host/path` from the concrete path, which on the receiver route is `scheme://host/path` from the concrete path, which on the receiver route is
`/h/<uuid>` in full — and that UUID is a write capability, not an identifier: `/h/<uuid>` in full — and that UUID is a write capability, not an identifier:
anyone holding it can post events this service accepts and its targets then anyone holding it can post events this service accepts and its targets then
deliver. A tracker has its own retention, access control and deletion policy, so deliver. What is sent is the chi route pattern instead: `http://host/h/{uuid}`.
the rule the access log follows above does not carry across that boundary. What A tracker has its own retention, access control and deletion policy, so the
is sent is the chi route pattern instead: `http://host/h/{uuid}`. pattern is sent on every route, including the 2xx and 5xx responses outside the
receiver for which the access log keeps the concrete path.
The scheme and the host are kept, and everything else in the URL is discarded The scheme and the host are kept, and everything else in the URL is discarded
rather than edited, so a future SDK version that starts appending a query string rather than edited, so a future SDK version that starts appending a query string
@@ -2526,8 +2565,9 @@ receiver route. What survives still names the failing route — scheme, host,
route pattern, method — and `X-Request-Id` ties the event to the local access route pattern, method — and `X-Request-Id` ties the event to the local access
log line that holds the rest. Nothing dropped is needed for the likeliest use, log line that holds the rest. Nothing dropped is needed for the likeliest use,
debugging a CSRF rejection. Its three inputs are the TLS decision, `Origin` and debugging a CSRF rejection. Its three inputs are the TLS decision, `Origin` and
`Referer`; the latter two are kept, and the first is the scheme of the retained `Referer`; the latter two are kept (`Referer` as its scheme and host, which is
URL, because the SDK derives that scheme from all the check compares), and the first is the scheme of the retained URL,
because the SDK derives that scheme from
`r.TLS != nil || r.Header.Get("X-Forwarded-Proto") == "https"`. That predicate `r.TLS != nil || r.Header.Get("X-Forwarded-Proto") == "https"`. That predicate
is the SDK's own and is stricter than `internal/reqtls.IsTLS`, which this is the SDK's own and is stricter than `internal/reqtls.IsTLS`, which this
service now uses everywhere it decides transport: the SDK reports `http` for the service now uses everywhere it decides transport: the SDK reports `http` for the
@@ -2593,9 +2633,7 @@ than it:
| ------------------------------------------ | ------- | ------------------- | ------------------------------------------ | | ------------------------------------------ | ------- | ------------------- | ------------------------------------------ |
| `request body exceeds limit` (413) | `WARN` | path, method | yes — `MaxBodySize` precedes `RequireAuth` | | `request body exceeds limit` (413) | `WARN` | path, method | yes — `MaxBodySize` precedes `RequireAuth` |
| `csrf: token validation failed` (403) | `WARN` | path, method | yes — `CSRF` precedes `RequireAuth` | | `csrf: token validation failed` (403) | `WARN` | path, method | yes — `CSRF` precedes `RequireAuth` |
| `... rate limit exceeded` (429) | `WARN` | path | yes, on the receiver |
| `auth middleware: unauthenticated request` | `DEBUG` | path, method | yes, by definition | | `auth middleware: unauthenticated request` | `DEBUG` | path, method | yes, by definition |
| `entrypoint not found` | `DEBUG` | entrypoint UUID | yes, on the receiver |
| `user not found` / `invalid password` | `DEBUG` | username | yes, on the sign-in form | | `user not found` / `invalid password` | `DEBUG` | username | yes, on the sign-in form |
| `login failure limit exceeded` (429) | `WARN` | path | yes, on the sign-in form | | `login failure limit exceeded` (429) | `WARN` | path | yes, on the sign-in form |
| `password verification capacity exhausted` | `WARN` | path | yes, on the sign-in form | | `password verification capacity exhausted` | `WARN` | path | yes, on the sign-in form |
@@ -2623,13 +2661,12 @@ wider of the two handlers spends more on than the client spent sending them.
Every case holds each line to the 2,560-byte ceiling. That per-line ceiling is Every case holds each line to the 2,560-byte ceiling. That per-line ceiling is
what the figure above states, and every row establishes it. what the figure above states, and every row establishes it.
Three of the sites go further and bound the whole flood's output — the total Two of the sites go further and bound the whole flood's output — the total bytes
bytes a run of distinct invented values wrote, which is the shape an operator a run of distinct invented values wrote, which is the shape an operator sizing
sizing storage cares about. They are `request body exceeds limit` storage cares about. They are `request body exceeds limit`
(`TestMaxBodySize_FloodOfOversizePathsDoesNotGrowTheLog`), (`TestMaxBodySize_FloodOfOversizePathsDoesNotGrowTheLog`) and `user not found`
`entrypoint not found` and `user not found` (the last two through (through `assertBoundedFlood`). The other rows carry no aggregate assertion; the
`assertBoundedFlood`). The other rows carry no aggregate assertion; the per-line per-line ceiling is what they establish.
ceiling is what they establish.
`internal/logfield/logfield_test.go` measures the per-rune charge against what `internal/logfield/logfield_test.go` measures the per-rune charge against what
the handlers really emit, over roughly 3,000 code points on each, so an the handlers really emit, over roughly 3,000 code points on each, so an
@@ -2647,7 +2684,7 @@ everything else, so its lines take the level the operator set and the handler
`internal/logger` selected, and every value it emits is spent through the same `internal/logger` selected, and every value it emits is spent through the same
`internal/logfield` budget. A record-not-found is not logged as an error: it is `internal/logfield` budget. A record-not-found is not logged as an error: it is
the expected outcome on both of those paths, and each handler already records the expected outcome on both of those paths, and each handler already records
its own miss at `DEBUG` — bounded, per the table above — without the SQL. Slow its own miss at `DEBUG`, without the SQL and within the ceiling above. Slow
statements are kept, at `WARN`, above the same 200 ms threshold GORM used and statements are kept, at `WARN`, above the same 200 ms threshold GORM used and
with the statement bounded, because that report is the one thing GORM's logger with the statement bounded, because that report is the one thing GORM's logger
gave an operator that nothing else here does. The adapter orders its cases gave an operator that nothing else here does. The adapter orders its cases
@@ -2972,6 +3009,12 @@ webhooker/
├── cmd/webhooker/ ├── cmd/webhooker/
│ └── main.go # Entry point: subcommand dispatch; no args locks DATA_DIR and wires fx │ └── main.go # Entry point: subcommand dispatch; no args locks DATA_DIR and wires fx
├── internal/ ├── internal/
│ ├── archive/
│ │ ├── archive.go # The archive writer of each database target, eviction, rename, and the rename lock
│ │ ├── writer.go # Archive file lifecycle and pruning
│ │ ├── rotation.go # Archive rotation and file names
│ │ ├── download.go # Archive download as gzipped JSON
│ │ └── sweeper.go # Periodic pruning of idle archives
│ ├── banner/ │ ├── banner/
│ │ └── banner.go # Ruled block for the one credential shown in the clear │ │ └── banner.go # Ruled block for the one credential shown in the clear
│ ├── resetpw/ │ ├── resetpw/
@@ -3015,20 +3058,16 @@ webhooker/
│ │ ├── target.go # Target interface, Task, Scheduler │ │ ├── target.go # Target interface, Task, Scheduler
│ │ ├── target_http.go # HTTP target (retries, circuit breaker) │ │ ├── target_http.go # HTTP target (retries, circuit breaker)
│ │ ├── target_slack.go # Slack/Mattermost incoming-webhook target │ │ ├── target_slack.go # Slack/Mattermost incoming-webhook target
│ │ ├── target_database.go # Database archive target │ │ ├── target_database.go # Database archive target, writing through internal/archive
│ │ ├── target_database_archive.go # Archive file lifecycle and pruning
│ │ ├── target_database_rotation.go # Archive rotation and file names
│ │ ├── target_database_export.go # Archive download as gzipped JSON
│ │ ├── target_log.go # Log target (stdout) │ │ ├── target_log.go # Log target (stdout)
│ │ ├── target_config_view.go # Masked target config for templates │ │ ├── target_config_view.go # Masked target settings for templates
│ │ ├── archive_sweeper.go # Periodic pruning of idle archives
│ │ ├── queue_depth.go # Periodic sampler behind the queue-depth gauges │ │ ├── queue_depth.go # Periodic sampler behind the queue-depth gauges
│ │ ├── url_mask.go # Strips credentials from *url.Error │ │ ├── url_mask.go # Strips credentials from *url.Error
│ │ └── ssrf.go # SSRF prevention (IP validation, safe HTTP transport) │ │ └── ssrf.go # SSRF prevention (IP validation, safe HTTP transport)
│ ├── handlers/ │ ├── handlers/
│ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering │ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering
│ │ ├── auth.go # Login, logout handlers │ │ ├── auth.go # Login, logout handlers
│ │ ├── delivery_replay.go # Per-delivery replay: new delivery, current target config │ │ ├── delivery_replay.go # Per-delivery replay: new delivery, current target settings
│ │ ├── event_resubmit.go # Event resubmit: new event, all currently active targets │ │ ├── event_resubmit.go # Event resubmit: new event, all currently active targets
│ │ ├── entrypoint_view.go # Masked entrypoint view for templates │ │ ├── entrypoint_view.go # Masked entrypoint view for templates
│ │ ├── event_log_view.go # Event log projection, byte-capped in SQL │ │ ├── event_log_view.go # Event log projection, byte-capped in SQL
@@ -3043,7 +3082,7 @@ webhooker/
│ │ ├── webhook_delete.go # Webhook delete, event database and archive writer removal │ │ ├── webhook_delete.go # Webhook delete, event database and archive writer removal
│ │ ├── event_log.go # Event log page: loaders, filters, delivery views │ │ ├── event_log.go # Event log page: loaders, filters, delivery views
│ │ ├── entrypoint.go # Entrypoint create, edit, delete and toggle │ │ ├── entrypoint.go # Entrypoint create, edit, delete and toggle
│ │ ├── target_create.go # Target create, per-type config builders │ │ ├── target_create.go # Target create, each type's settings from the form
│ │ ├── target_delete.go # Target delete │ │ ├── target_delete.go # Target delete
│ │ ├── target_toggle.go # Target toggle │ │ ├── target_toggle.go # Target toggle
│ │ ├── shared.go # Helpers shared by several handlers │ │ ├── shared.go # Helpers shared by several handlers
@@ -3062,6 +3101,8 @@ webhooker/
│ │ ├── ratelimit.go # Per-IP rate limiting middleware (go-chi/httprate) │ │ ├── ratelimit.go # Per-IP rate limiting middleware (go-chi/httprate)
│ │ ├── loginguard.go # Login failure counters and the Argon2id verification semaphore │ │ ├── loginguard.go # Login failure counters and the Argon2id verification semaphore
│ │ └── middlewaretest/ # Test support: a Middleware for tests in other packages │ │ └── middlewaretest/ # Test support: a Middleware for tests in other packages
│ ├── referer/
│ │ └── referer.go # SchemeAndHost: cuts a Referer down to its scheme and host, for the access log and the Sentry hook
│ ├── reqtls/ │ ├── reqtls/
│ │ └── reqtls.go # IsTLS: the one TLS predicate, r.TLS or X-Forwarded-Proto │ │ └── reqtls.go # IsTLS: the one TLS predicate, r.TLS or X-Forwarded-Proto
│ ├── server/ │ ├── server/
@@ -3076,8 +3117,8 @@ webhooker/
│ ├── static.go # //go:embed directive │ ├── static.go # //go:embed directive
│ ├── css/input.css # Tailwind input, source for tailwind.css (make css) │ ├── css/input.css # Tailwind input, source for tailwind.css (make css)
│ ├── css/tailwind.css # Generated stylesheet the pages load │ ├── css/tailwind.css # Generated stylesheet the pages load
│ ├── css/style.css # Hand-written, loaded after tailwind.css: btn-small, the pointer cursor for input.css's buttons, the webhook list cards' focus outline │ ├── css/style.css # Hand-written, loaded after tailwind.css: btn-small, the pointer cursor for input.css's buttons, the webhook list cards' focus outline, the rule hiding x-cloak elements
│ ├── js/app.js # Copy-to-clipboard, and the Alpine.js components │ ├── js/app.js # Copy-to-clipboard, the delete prompts, and the Alpine.js components
│ └── js/alpine.min.js # Alpine.js CSP build, extracted from 3p/ by make assets, not committed │ └── js/alpine.min.js # Alpine.js CSP build, extracted from 3p/ by make assets, not committed
├── templates/ # Go HTML templates (base, login, sources, etc.) ├── templates/ # Go HTML templates (base, login, sources, etc.)
├── script/ # Scripts to Rule Them All entrypoints ├── script/ # Scripts to Rule Them All entrypoints
@@ -3110,18 +3151,20 @@ Components are wired via Uber fx in this order:
11. `metrics.New` — The delivery collectors, registered on that registry 11. `metrics.New` — The delivery collectors, registered on that registry
12. `middleware.New` — HTTP middleware 12. `middleware.New` — HTTP middleware
13. `delivery.New` — Event-driven delivery engine 13. `delivery.New` — Event-driven delivery engine
14. `delivery.NewArchiveSweeper` — Periodic pruning of idle archives 14. `archive.NewWriters` — The database targets' archive writers, and the lock
15. `delivery.Engine` → `delivery.Notifier` — interface bridge every rename of their files runs under
16. `delivery.Engine` → `delivery.Archives` — interface bridge so deleting or 15. `archive.NewSweeper` — Periodic pruning of idle archives
16. `delivery.Engine` → `delivery.Notifier` — interface bridge
17. `archive.Writers` → `archive.Archives` — interface bridge so deleting or
renaming a webhook or target reaches its archive files renaming a webhook or target reaches its archive files
17. `server.New` — HTTP server and router 18. `server.New` — HTTP server and router
The server starts via The server starts via
`fx.Invoke(func(*server.Server, *delivery.Engine, *database.RetentionReaper, *delivery.ArchiveSweeper) {})`, `fx.Invoke(func(*server.Server, *delivery.Engine, *database.RetentionReaper, *archive.Sweeper) {})`,
which triggers the fx lifecycle hooks in dependency order. The which triggers the fx lifecycle hooks in dependency order. The
`delivery.Notifier` interface allows the webhook handler to send self-contained `delivery.Notifier` interface allows the webhook handler to send self-contained
`delivery.Task` slices to the engine without a direct package dependency. Each `delivery.Task` slices to the engine without a direct package dependency. Each
task carries all target config and event data inline (for bodies under 16 KiB, task carries all target settings and event data inline (for bodies under 16 KiB,
`delivery.MaxInlineBodySize`), so the engine can deliver without reading from `delivery.MaxInlineBodySize`), so the engine can deliver without reading from
any database — it only writes to record results. any database — it only writes to record results.
@@ -3131,8 +3174,9 @@ Applied to all routes in this order:
1. **RequestID** — Generate unique request IDs (chi built-in) 1. **RequestID** — Generate unique request IDs (chi built-in)
2. **SecurityHeaders** — Production security headers on every response (HSTS 2. **SecurityHeaders** — Production security headers on every response (HSTS
unless `HSTS_MAX_AGE` is `0`, X-Content-Type-Options, X-Frame-Options, CSP, unless `HSTS_MAX_AGE` is `0`, X-Content-Type-Options, X-Frame-Options, CSP
Referrer-Policy, Permissions-Policy) `default-src 'self'; script-src 'self'; style-src 'self'`, Referrer-Policy,
Permissions-Policy)
3. **Logging** — Structured request logging (method, URL, status, latency, 3. **Logging** — Structured request logging (method, URL, status, latency,
remote IP, client IP, user agent, request ID) remote IP, client IP, user agent, request ID)
4. **Metrics** — Prometheus HTTP metrics (if `METRICS_USERNAME` and 4. **Metrics** — Prometheus HTTP metrics (if `METRICS_USERNAME` and
@@ -3217,6 +3261,12 @@ credential check, see [The login endpoint](#the-login-endpoint).
- Production security headers on all responses: HSTS (unless `HSTS_MAX_AGE` is - Production security headers on all responses: HSTS (unless `HSTS_MAX_AGE` is
`0`), X-Content-Type-Options (`nosniff`), X-Frame-Options (`DENY`), `0`), X-Content-Type-Options (`nosniff`), X-Frame-Options (`DENY`),
Content-Security-Policy, Referrer-Policy, and Permissions-Policy Content-Security-Policy, Referrer-Policy, and Permissions-Policy
- The Content-Security-Policy is
`default-src 'self'; script-src 'self'; style-src 'self'`: a page takes
scripts and styles only from files webhooker serves, and the browser runs no
inline script or event handler and ignores any `<style>` element or `style`
attribute in a page. The pages need none: their scripts are in `static/js/`
and their styles in `static/css/`
- Request body size limits (1 MB) on all form POST endpoints, enforced by - Request body size limits (1 MB) on all form POST endpoints, enforced by
middleware that runs before CSRF parses the form middleware that runs before CSRF parses the form
- **CSRF protection** via [gorilla/csrf](https://github.com/gorilla/csrf) on all - **CSRF protection** via [gorilla/csrf](https://github.com/gorilla/csrf) on all
@@ -3279,12 +3329,13 @@ order under a **5 second budget** (`fx.StopTimeout` in `cmd/webhooker/main.go`).
That budget covers the whole sequence, not each hook. The order, read off the fx That budget covers the whole sequence, not each hook. The order, read off the fx
stop-hook log: stop-hook log:
1. `ArchiveSweeper` 1. `archive.Sweeper`
2. `RetentionReaper` 2. `RetentionReaper`
3. `server` — the HTTP drain, bounded by `server.ShutdownTimeout` (**3 3. `server` — the HTTP drain, bounded by `server.ShutdownTimeout` (**3
seconds**) and by what the hooks before it left, then a Sentry flush if seconds**) and by what the hooks before it left, then a Sentry flush if
`SENTRY_DSN` is set `SENTRY_DSN` is set
4. `delivery.Engine` — waits for its workers, then closes the archive databases 4. `delivery.Engine` — cuts off any `http` or `slack` request in flight, waits
for its workers, then closes the archive databases
5. `healthcheck` 5. `healthcheck`
6. `WebhookDBManager` 6. `WebhookDBManager`
7. the database close 7. the database close
@@ -3295,6 +3346,12 @@ by the stop context, so a wedge there consumes the 5 seconds before the HTTP
server hook is ever entered. The hooks after the server are microsecond-scale in server hook is ever entered. The hooks after the server are microsecond-scale in
normal operation. normal operation.
The delivery engine cancels a delivery request in flight. That attempt is not
recorded or counted in the metrics, and the delivery keeps its `pending` or
`retrying` status, so recovery at the next start sends it again (see Recovery
under Bounded Worker Pool). If the receiver already had it, it receives it
twice: the at-least-once duplicate described under Rationale.
The HTTP drain budget is deliberately **shorter** than the sequence budget. Were The HTTP drain budget is deliberately **shorter** than the sequence budget. Were
the two equal, a drain that used its whole budget would exhaust the sequence the two equal, a drain that used its whole budget would exhaust the sequence
budget at the instant it finished, and every later hook — the delivery engine, budget at the instant it finished, and every later hook — the delivery engine,
@@ -3315,7 +3372,7 @@ skipped when that leaves too little to be worth attempting — so a full-length
drain means Sentry events are dropped rather than the database close being drain means Sentry events are dropped rather than the database close being
skipped. skipped.
This does not make the database close unconditional. A slow `ArchiveSweeper` or This does not make the database close unconditional. A slow `archive.Sweeper` or
`RetentionReaper` is enough to cut the shutdown short, not only one that `RetentionReaper` is enough to cut the shutdown short, not only one that
consumes the whole budget: what they spend comes out of the drain first, so consumes the whole budget: what they spend comes out of the drain first, so
after 2 seconds of theirs a request still in flight gets 1 second to finish, and after 2 seconds of theirs a request still in flight gets 1 second to finish, and
@@ -3417,14 +3474,12 @@ fixed independently of the compiler's:
a step's log, and then fails. `make test` builds this stage alone. a step's log, and then fails. `make test` builds this stage alone.
6. **Builder stage** (`golang:1.26.1-bookworm`) — depends on the lint and test 6. **Builder stage** (`golang:1.26.1-bookworm`) — depends on the lint and test
phases and the `css-check` and `markdown-check` stages passing (it copies a phases and the `css-check` and `markdown-check` stages passing (it copies a
file from each), runs `make build` (which extracts Alpine.js from `3p/` file from each), and runs `make build` (which extracts Alpine.js from `3p/`
first), and then rebuilds the binary with `CGO_ENABLED=1` and static linking first). `make build` compiles with cgo off, so the binary is statically
so it runs on musl. Both builds go through `make build`, the relink adding linked and runs on the Alpine runtime image. The version is the `VERSION`
its `-extldflags` via `GO_LDFLAGS`, so neither can drop the `-X` that stamps build arg if one is given, otherwise derived from the `.git` in the context,
the version. The version is the `VERSION` build arg if one is given, and the stage fails if a context with `.git` would stamp an empty version,
otherwise derived from the `.git` in the context, and the stage fails if a `dev` or `unknown` (see [Version stamping](#version-stamping)).
context with `.git` would stamp an empty version, `dev` or `unknown` (see
[Version stamping](#version-stamping)).
7. **Runtime stage** (`alpine:3.21`) — copies the static binary and 7. **Runtime stage** (`alpine:3.21`) — copies the static binary and
`deploy/docker-entrypoint.sh`, creates the `/var/lib/webhooker` directory for `deploy/docker-entrypoint.sh`, creates the `/var/lib/webhooker` directory for
all SQLite databases, exposes port 8080, and includes a health check against all SQLite databases, exposes port 8080, and includes a health check against
@@ -3436,11 +3491,6 @@ The lint and test phases invoke `gofmt`, `golangci-lint` and `go test` directly
rather than `make fmt-check`, `make lint` and `make test`: those targets build rather than `make fmt-check`, `make lint` and `make test`: those targets build
docker stages, which would need a docker daemon inside this build. docker stages, which would need a docker daemon inside this build.
The lint phase and the Go stages use Debian rather than Alpine because
`gorm.io/driver/sqlite` pulls in `mattn/go-sqlite3`, which needs CGO and does
not compile against musl. Only the final binary is statically linked, which is
what lets it run on the Alpine runtime image.
`script/cibuild` is the CI gate: it runs `script/bootstrap`, then `script/cibuild` is the CI gate: it runs `script/bootstrap`, then
`script/check`, then builds the image, whose build runs the lint and test phases `script/check`, then builds the image, whose build runs the lint and test phases
and the stylesheet and Markdown checks again. A build that succeeds is a repo and the stylesheet and Markdown checks again. A build that succeeds is a repo
+9 -7
View File
@@ -10,6 +10,7 @@ import (
"go.uber.org/fx" "go.uber.org/fx"
"go.uber.org/fx/fxevent" "go.uber.org/fx/fxevent"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/datadir" "sneak.berlin/go/webhooker/internal/datadir"
@@ -49,7 +50,7 @@ import (
// before the server may already have spent. // before the server may already have spent.
// //
// This does not make the database close unconditional: the // This does not make the database close unconditional: the
// ArchiveSweeper and RetentionReaper hooks run before the server. // archive.Sweeper and RetentionReaper hooks run before the server.
// What they spend comes out of the drain first, but past 3s it comes // What they spend comes out of the drain first, but past 3s it comes
// out of the reserve, and they can consume the whole budget. // out of the reserve, and they can consume the whole budget.
const stopTimeout = 5 * time.Second const stopTimeout = 5 * time.Second
@@ -188,8 +189,8 @@ func newApp() *fx.App {
fx.Provide( fx.Provide(
globals.New, globals.New,
logger.New, logger.New,
// The plain logger the session, the middleware and the // The plain logger the session, the middleware, the
// webhook database manager take. // webhook database manager and the archive writers take.
func(l *logger.Logger) *slog.Logger { return l.Get() }, func(l *logger.Logger) *slog.Logger { return l.Get() },
config.New, config.New,
database.New, database.New,
@@ -208,14 +209,15 @@ func newApp() *fx.App {
// disagree about a destination. // disagree about a destination.
delivery.NewGuard, delivery.NewGuard,
delivery.New, delivery.New,
delivery.NewArchiveSweeper, archive.NewWriters,
archive.NewSweeper,
// Wire *delivery.Engine as delivery.Notifier so the // Wire *delivery.Engine as delivery.Notifier so the
// webhook handler can notify the engine of new deliveries. // webhook handler can notify the engine of new deliveries.
func(e *delivery.Engine) delivery.Notifier { return e }, func(e *delivery.Engine) delivery.Notifier { return e },
// Wire *delivery.Engine as delivery.Archives so deleting // Wire *archive.Writers as archive.Archives so deleting
// or renaming a webhook or target reaches its archive // or renaming a webhook or target reaches its archive
// files. // files.
func(e *delivery.Engine) delivery.Archives { return e }, func(w *archive.Writers) archive.Archives { return w },
// Wire *delivery.Engine as delivery.CircuitBreakers so // Wire *delivery.Engine as delivery.CircuitBreakers so
// the pages can show a target whose deliveries are // the pages can show a target whose deliveries are
// paused. // paused.
@@ -227,7 +229,7 @@ func newApp() *fx.App {
*server.Server, *server.Server,
*delivery.Engine, *delivery.Engine,
*database.RetentionReaper, *database.RetentionReaper,
*delivery.ArchiveSweeper, *archive.Sweeper,
) { ) {
}, },
), ),
+1 -1
View File
@@ -252,7 +252,7 @@ const tailHeadroom = 2 * time.Second
// can produce, since a shorter drain leaves the flush more room and // can produce, since a shorter drain leaves the flush more room and
// the worst case is not necessarily at either extreme. // the worst case is not necessarily at either extreme.
// //
// Nor does the hook start on a full budget: the ArchiveSweeper and // Nor does the hook start on a full budget: the archive.Sweeper and
// RetentionReaper hooks run before it, and whatever they spent is // RetentionReaper hooks run before it, and whatever they spent is
// gone. The outer sweep walks every amount they can spend. Once they // gone. The outer sweep walks every amount they can spend. Once they
// have eaten into the headroom themselves, the hook must spend // have eaten into the headroom themselves, the hook must spend
+446
View File
@@ -0,0 +1,446 @@
// Package archive manages the SQLite archive files the database target
// writes: how they are named and rotated, the writers that open, write,
// close and reopen them, pruning the rows past a target's expiry,
// renaming a target's files and moving them back when that fails, the
// gzipped JSON export, the idle sweep, and the locks that keep a rename
// apart from archive writes, from other edits and from exports.
package archive
import (
"fmt"
"log/slog"
"strings"
"sync"
"time"
"go.uber.org/fx"
"sneak.berlin/go/webhooker/internal/database"
)
// archiveNameMaxLen is how many characters of a webhook or target
// name an archive file name keeps.
const archiveNameMaxLen = 40
// Archives is how the handlers keep the database targets' archive
// files in step with the configuration. Deleting a webhook or a
// target releases the cached archive writers, whose open file
// handles would otherwise outlive them; renaming one renames the
// archive files, which are named for the webhook and the target
// (see FileName). RenameLock returns the lock every rename runs under
// (see Writers.RenameLock).
//
// It is an interface so that the handlers' tests can stand in for
// Writers.
//
// Neither eviction deletes an archive file. Both are idempotent
// and are no-ops for a webhook or target with no writer.
type Archives interface {
RenameLock() *sync.Mutex
EvictWebhook(webhookID string)
EvictTarget(targetID string)
Rename(targetID, webhookName, targetName string) error
}
// WritersParams holds the fx dependencies for Writers.
type WritersParams struct {
fx.In
Database *database.Database
DBManager *database.WebhookDBManager
Logger *slog.Logger
}
// Writers holds the archive writer of each database target, and the
// lock every rename of archive files runs under. The database target
// writes an event through it, the handlers rename and evict through
// it, the archive sweeper prunes through it, and the delivery engine
// evicts every writer when it stops. See archiveWriter for the
// close/reopen, auto-recreate, and expiry semantics.
type Writers struct {
db *database.Database
dbMgr *database.WebhookDBManager
log *slog.Logger
// writers holds one archive writer per database target, keyed
// by target ID.
mu sync.Mutex
writers map[string]*archiveWriter
// renameMu makes the webhook edit, the target edit and target
// creation run one at a time, each held from loading the stored
// names through the archive rename, the save and any move back.
// Interleaved, one could rename an archive between another's
// rename and save, leaving the file named for one edit and the
// stored names from the other. An archive download holds it while
// it reads the stored names and lists the files they give, and
// again for each file while it finds the file under the names
// stored then and opens it. RenameLock returns it.
renameMu sync.Mutex
}
// NewWriters builds the archive writers. It opens no archive file:
// each target's writer is made on its first use.
func NewWriters(params WritersParams) *Writers {
return &Writers{
db: params.Database,
dbMgr: params.DBManager,
log: params.Logger,
}
}
// RenameLock implements Archives. It returns the lock every rename of
// archive files runs under: see renameMu for who holds it and for how
// long. Archive writes and Rename itself do not take it.
func (ws *Writers) RenameLock() *sync.Mutex {
return &ws.renameMu
}
// FileName returns the file name of a database target's
// archive: archive-WEBHOOKNAME-TARGETNAME-TARGETID.db, with both
// names passed through archiveNamePart. The target ID keeps the
// name unique when two targets' names come out the same.
func FileName(webhookName, targetName, targetID string) string {
return "archive-" + archiveNamePart(webhookName) + "-" +
archiveNamePart(targetName) + "-" + targetID + ".db"
}
// archiveNamePart makes a webhook or target name safe to put in a
// file name. It is lowercased; ASCII letters and digits are kept,
// every other run of characters becomes a single "-", and no "-" is
// left at either end. It is cut to archiveNameMaxLen characters, and
// a name with nothing left is "unnamed".
func archiveNamePart(name string) string {
var b strings.Builder
dash := false
for _, r := range strings.ToLower(name) {
if (r < 'a' || r > 'z') && (r < '0' || r > '9') {
dash = b.Len() > 0
continue
}
if dash {
b.WriteByte('-')
dash = false
}
b.WriteRune(r)
}
part := b.String()
if len(part) > archiveNameMaxLen {
part = strings.TrimRight(part[:archiveNameMaxLen], "-")
}
if part == "" {
return "unnamed"
}
return part
}
// Write writes the full event of a delivery to a database target as
// a row into the target's archive database, honouring the target's
// ArchiveExpiry and ArchiveRotation. With rotation, the event goes to
// the file for the period of its receive time.
func (ws *Writers) Write(d *database.Delivery) error {
webhookID := d.Event.WebhookID
if webhookID == "" {
return errArchiveMissingWebhookID
}
// An event whose stored row was gone before its delivery ran has
// no receive time (see delivery's Engine.hydrateEvent), and goes
// to the file for now.
receivedAt := d.Event.CreatedAt
if receivedAt.IsZero() {
receivedAt = time.Now()
}
w, err := ws.writerFor(d.TargetID)
if err != nil {
return err
}
row := archivedEvent{
EventID: d.Event.ID,
WebhookID: webhookID,
EntrypointID: d.Event.EntrypointID,
Method: d.Event.Method,
RawQuery: d.Event.RawQuery,
Headers: d.Event.Headers,
Body: d.Event.Body,
ContentType: d.Event.ContentType,
}
return w.write(
row, d.Target.ArchiveExpiry,
archivePeriod(d.Target.ArchiveRotation, receivedAt),
)
}
// Rename implements Archives. It renames every one of a database
// target's archive files to FileName(webhookName, targetName,
// targetID), each keeping the period in its name. It goes through the
// target's writer, so the move holds the lock that the target's
// archive writes and the idle sweep take, and later writes use the
// new name. It never replaces a file: if one already has a new name,
// the error is ErrNameTaken.
//
// The writer is created if there is none, and it stays cached. The
// handlers rename before they save the new name, so until the save
// the main database still has the old one; a delivery in that window
// must find this writer rather than build one from the old name.
func (ws *Writers) Rename(
targetID, webhookName, targetName string,
) error {
w, err := ws.writerFor(targetID)
if err != nil {
return err
}
return w.rename(FileName(webhookName, targetName, targetID))
}
// EvictTarget implements Archives. It drops a database target's
// archive writer from the registry and closes its handle, so a
// deleted target does not leave a writer (and an open archive handle
// within its debounce window) alive for the process lifetime.
//
// The map entry is removed under the registry lock, which is
// then released before the handle is closed under the writer's
// own lock: that ordering keeps the registry available to other
// targets while an in-flight write on this one drains, and
// closing under the writer's lock means eviction can never race
// a write.
//
// Eviction is idempotent and silent for a target with no writer,
// which is the common case: only a database target that has
// received an event or been renamed has one. It never deletes the
// archive file.
func (ws *Writers) EvictTarget(targetID string) {
ws.mu.Lock()
w, ok := ws.writers[targetID]
if ok {
delete(ws.writers, targetID)
}
ws.mu.Unlock()
if !ok {
return
}
w.evict()
ws.log.Info(
"evicted archive writer",
"target_id", targetID,
"path", w.path,
)
}
// EvictWebhook implements Archives. It evicts, exactly as EvictTarget
// does, the writer of every database target of a webhook. The archive
// files themselves are left on disk — they are long-term storage the
// operator owns.
func (ws *Writers) EvictWebhook(webhookID string) {
ws.mu.Lock()
var gone []*archiveWriter
for targetID, w := range ws.writers {
if w.webhookID == webhookID {
delete(ws.writers, targetID)
gone = append(gone, w)
}
}
ws.mu.Unlock()
for _, w := range gone {
w.evict()
ws.log.Info(
"evicted archive writer",
"webhook_id", webhookID,
"path", w.path,
)
}
}
// EvictAll evicts every cached archive writer, exactly as EvictTarget
// does for one target. The delivery engine calls it at shutdown, once
// its workers have returned. Closing the last handle on an archive
// moves the contents of its -wal into the .db and removes the
// -wal, so a clean stop leaves each archive as a single file.
func (ws *Writers) EvictAll() {
ws.mu.Lock()
writers := ws.writers
ws.writers = nil
ws.mu.Unlock()
for _, w := range writers {
w.evict()
}
}
// writerFor returns the archive writer for a database target,
// creating and caching it on first use. Each target has one writer
// so its close/reopen debounce state is shared across concurrent
// deliveries, and so a rename and the idle sweep take the same lock
// as its writes.
func (ws *Writers) writerFor(
targetID string,
) (*archiveWriter, error) {
ws.mu.Lock()
defer ws.mu.Unlock()
w, ok := ws.writers[targetID]
if !ok {
var err error
w, err = ws.newWriter(targetID)
if err != nil {
return nil, err
}
if ws.writers == nil {
ws.writers = make(map[string]*archiveWriter)
}
ws.writers[targetID] = w
}
// A delivery claims the entry: even if the idle sweep created
// it moments ago, it now belongs to the registry proper and
// the sweep must leave it in place when it finishes.
w.sweepOwned = false
return w, nil
}
// sweepWriterFor returns the archive writer the idle sweep should
// prune a target's archive through, together with whether the sweep
// itself created the registry entry.
//
// The sweep must route its prune through the registered writer so
// the writer's mutex orders it against concurrent writes, but it
// must never leave a registry entry behind: a sweep that ran
// concurrently with the target's deletion would otherwise
// re-create an entry that nothing will ever evict again, which is
// exactly the leak eviction exists to prevent. An entry the sweep
// creates is therefore marked sweep-owned and handed back to
// releaseSweepWriter when the sweep is done.
func (ws *Writers) sweepWriterFor(
targetID string,
) (*archiveWriter, bool, error) {
ws.mu.Lock()
defer ws.mu.Unlock()
w, ok := ws.writers[targetID]
if ok {
return w, false, nil
}
w, err := ws.newWriter(targetID)
if err != nil {
return nil, false, err
}
if ws.writers == nil {
ws.writers = make(map[string]*archiveWriter)
}
w.sweepOwned = true
ws.writers[targetID] = w
return w, true, nil
}
// releaseSweepWriter drops a registry entry that the idle sweep
// created, so a sweep leaves the registry exactly as it found it.
//
// The entry is removed only if it is still the very writer the
// sweep installed and no delivery has claimed it in the meantime
// (writerFor clears sweepOwned when it hands a writer to the
// write path). Both conditions are evaluated under the registry
// lock, so an eviction that raced the sweep — which removes the
// entry outright — simply finds nothing left to do here, and a
// delivery that adopted the writer keeps a registered, evictable
// one.
func (ws *Writers) releaseSweepWriter(
targetID string, w *archiveWriter,
) {
ws.mu.Lock()
defer ws.mu.Unlock()
cur, ok := ws.writers[targetID]
if !ok || cur != w || !cur.sweepOwned {
return
}
delete(ws.writers, targetID)
}
// newWriter builds the writer for a database target's archive. Its
// path is the one Path gives for the webhook and the target as
// the main database names them now; from then on only rename changes
// the name the writer uses. It does not touch the archive files.
func (ws *Writers) newWriter(
targetID string,
) (*archiveWriter, error) {
if ws.dbMgr == nil {
return nil, errArchiveNoDataDir
}
var target database.Target
err := ws.db.DB().
Preload("Webhook").
First(&target, "id = ?", targetID).Error
if err != nil {
return nil, fmt.Errorf(
"loading database target %s: %w", targetID, err,
)
}
w := newArchiveWriter(
Path(ws.dbMgr, &target.Webhook, &target),
ws.log,
)
w.webhookID = target.WebhookID
return w, nil
}
// sweepArchive prunes one database target's archive of rows older
// than expiry, without requiring a write. A missing archive file is
// left missing (see sweepExpired), so a sweep never creates an
// archive for a target that has never received an event.
//
// It also never leaves a registry entry behind: an entry it had
// to create to reach the writer's mutex is released again once
// the prune is done, so a sweep racing a target deletion cannot
// resurrect the writer the eviction just dropped.
func (ws *Writers) sweepArchive(
targetID string, expiry time.Duration,
) error {
w, created, err := ws.sweepWriterFor(targetID)
if err != nil {
return err
}
if created {
defer ws.releaseSweepWriter(targetID, w)
}
return w.sweepExpired(expiry)
}
+757
View File
@@ -0,0 +1,757 @@
package archive_test
import (
"database/sql"
"fmt"
"io/fs"
"log/slog"
"net/http"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
_ "modernc.org/sqlite" // Pure Go SQLite driver.
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/gormlog"
)
const (
// testContentType is the event content type used in tests.
testContentType = "application/json"
// eventQuery is the query string an archived event carries.
eventQuery = "a=1&b=2"
)
func archiveTestLogger() *slog.Logger {
return slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
))
}
// openArchiveDBForRead opens an archive file read-only so a
// test can inspect the rows the writer persisted.
func openArchiveDBForRead(
t *testing.T, path string,
) *gorm.DB {
t.Helper()
sqlDB, err := sql.Open(
"sqlite",
fmt.Sprintf("file:%s?mode=ro", path),
)
require.NoError(t, err)
t.Cleanup(func() { _ = sqlDB.Close() })
gdb, err := gorm.Open(
sqlite.Dialector{Conn: sqlDB},
&gorm.Config{Logger: gormlog.New(slog.New(slog.DiscardHandler))},
)
require.NoError(t, err)
return gdb
}
// archiveFileSuffixes returns the archive file itself and the
// SQLite sidecars that accompany an open database. A test that
// asserts no archive was created has to check all of them.
func archiveFileSuffixes() []string {
return []string{"", "-wal", "-shm"}
}
// removeArchiveFiles simulates an operator moving the archive
// away by deleting the SQLite file and its sidecar files.
func removeArchiveFiles(t *testing.T, path string) {
t.Helper()
for _, suffix := range []string{
"", "-wal", "-shm", "-journal",
} {
err := os.Remove(path + suffix)
if err != nil && !os.IsNotExist(err) {
t.Fatalf("removing %s%s: %v", path, suffix, err)
}
}
}
// assertStoredTime asserts that text, a timestamp as SQLite holds it,
// is in the form GORM stores created_at in: the layout of
// time.Time.String, with no " m=+…" monotonic clock reading after it.
func assertStoredTime(t *testing.T, text string) {
t.Helper()
_, err := time.Parse(
"2006-01-02 15:04:05.999999999 -0700 MST", text,
)
assert.NoError(t, err, "stored as %q", text)
}
// newEvent returns an event received now with the given body, as the
// webhook handler stores it.
func newEvent(body string) database.Event {
return database.Event{
BaseModel: database.BaseModel{
ID: uuid.New().String(),
CreatedAt: time.Now(),
},
WebhookID: uuid.New().String(),
EntrypointID: uuid.New().String(),
Method: http.MethodPost,
Headers: `{"Content-Type":["application/json"]}`,
Body: body,
ContentType: testContentType,
}
}
// databaseTargetDelivery returns a delivery of event to a database
// target, as the database target hands it to Writers.Write.
func databaseTargetDelivery(
event database.Event, tgt *database.Target,
) *database.Delivery {
return &database.Delivery{
EventID: event.ID,
TargetID: tgt.ID,
Event: event,
Target: database.Target{
Name: tgt.Name,
Type: database.TargetTypeDatabase,
ArchiveExpiry: tgt.ArchiveExpiry,
ArchiveRotation: tgt.ArchiveRotation,
},
}
}
func TestArchiveWriter_WritesRow(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive-wh.db")
w := archive.NewExportArchiveWriter(
path, archiveTestLogger(), 0,
)
row := archive.ExportArchivedEvent{
EventID: "ev-1",
WebhookID: "wh-1",
EntrypointID: "ep-1",
Method: "POST",
Headers: `{"X":"Y"}`,
Body: `{"hello":"world"}`,
ContentType: "application/json",
}
require.NoError(t, w.Write(row, 0))
assert.FileExists(t, path)
var got []archive.ExportArchivedEvent
require.NoError(t, w.DB().Find(&got).Error)
require.Len(t, got, 1)
assert.Equal(t, "ev-1", got[0].EventID)
assert.Equal(t, "wh-1", got[0].WebhookID)
assert.Equal(t, "ep-1", got[0].EntrypointID)
assert.Equal(t, row.Method, got[0].Method)
assert.Equal(t, row.ContentType, got[0].ContentType)
assert.JSONEq(t, `{"hello":"world"}`, got[0].Body)
assert.False(t, got[0].ArchivedAt.IsZero())
var archivedAt string
require.NoError(t, w.DB().Raw(
"SELECT CAST(archived_at AS TEXT) FROM archived_events",
).Row().Scan(&archivedAt))
assertStoredTime(t, archivedAt)
}
func TestArchiveWriter_RecreatesAfterRemoval(
t *testing.T,
) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive-wh.db")
w := archive.NewExportArchiveWriter(
path, archiveTestLogger(), 0,
)
require.NoError(t, w.Write(
archive.ExportArchivedEvent{EventID: "a"}, 0,
))
assert.FileExists(t, path)
// The operator moves the archive away while the handle is
// still open.
removeArchiveFiles(t, path)
require.NoFileExists(t, path)
// The next write recreates the file with a fresh schema and
// only the new row.
require.NoError(t, w.Write(
archive.ExportArchivedEvent{EventID: "b"}, 0,
))
assert.FileExists(t, path)
var got []archive.ExportArchivedEvent
require.NoError(t, w.DB().Find(&got).Error)
require.Len(t, got, 1)
assert.Equal(t, "b", got[0].EventID)
}
// TestArchiveHandleIsWAL closes the last gap in the durability
// evidence: the main and per-webhook tiers each assert their journal
// mode on a live handle, and the archive tier gets its settings from
// the same code path but nothing checked the running file.
func TestArchiveHandleIsWAL(t *testing.T) {
t.Parallel()
w := archive.NewExportArchiveWriter(
filepath.Join(t.TempDir(), "archive-wal.db"),
archiveTestLogger(), 0,
)
require.NoError(t, w.Open(0))
var mode string
row := w.DB().Raw("pragma journal_mode").Row()
require.NoError(t, row.Scan(&mode))
assert.Equal(t, "wal", strings.ToLower(mode))
var busy string
row = w.DB().Raw("pragma busy_timeout").Row()
require.NoError(t, row.Scan(&busy))
assert.Equal(t, "10000", busy)
}
// TestStatArchive proves Stat finds no file before the first
// write; after a write still held in the -wal, counts the -wal in the
// size and takes its later time as the last write; and finds no file
// again once the file has been moved away.
func TestStatArchive(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive-wh.db")
_, err := archive.Stat(path)
require.ErrorIs(t, err, fs.ErrNotExist)
// With the clock stopped, the reopen debounce never passes, so
// the handle stays open after the write.
stopped := time.Now()
w := archive.NewExportArchiveWriter(path, archiveTestLogger(), 0)
w.SetNow(func() time.Time { return stopped })
require.NoError(t, w.Write(archive.ExportArchivedEvent{EventID: "a"}, 0))
written := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
earlier := written.Add(-time.Hour)
require.NoError(t, os.Chtimes(path, earlier, earlier))
require.NoError(t, os.Chtimes(path+"-wal", written, written))
file, err := os.Stat(path)
require.NoError(t, err)
wal, err := os.Stat(path + "-wal")
require.NoError(t, err)
require.Positive(t, wal.Size())
got, err := archive.Stat(path)
require.NoError(t, err)
assert.Equal(t, 1, got.Files)
assert.Equal(t, file.Size()+wal.Size(), got.Size)
assert.True(t, written.Equal(got.Written), got.Written)
removeArchiveFiles(t, path)
_, err = archive.Stat(path)
require.ErrorIs(t, err, fs.ErrNotExist)
}
func TestArchiveWriter_ReopenDebounce(t *testing.T) {
t.Parallel()
const debounce = 2 * time.Second
path := filepath.Join(t.TempDir(), "archive-wh.db")
w := archive.NewExportArchiveWriter(
path, archiveTestLogger(), debounce,
)
// The writer measures its reopen debounce on this clock, which
// only the test moves, so how long the host takes between
// writes cannot change the result.
now := time.Now()
w.SetNow(func() time.Time { return now })
require.NoError(t, w.Write(
archive.ExportArchivedEvent{EventID: "a"}, 0,
))
require.NoError(t, w.Write(
archive.ExportArchivedEvent{EventID: "b"}, 0,
))
// Two writes inside the debounce window trigger only the
// initial open — no extra close/reopen.
assert.Equal(t, 1, w.Reopens())
now = now.Add(debounce)
require.NoError(t, w.Write(
archive.ExportArchivedEvent{EventID: "c"}, 0,
))
// A write after the window elapses closes and reopens once.
assert.Equal(t, 2, w.Reopens())
}
func TestArchiveWriter_ExpiryPrune(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive-wh.db")
w := archive.NewExportArchiveWriter(
path, archiveTestLogger(), 0,
)
require.NoError(t, w.Open(0))
old := archive.ExportArchivedEvent{
EventID: "old",
ArchivedAt: time.Now().Add(-2 * time.Hour),
}
fresh := archive.ExportArchivedEvent{
EventID: "fresh",
ArchivedAt: time.Now(),
}
require.NoError(t, w.DB().Create(&old).Error)
require.NoError(t, w.DB().Create(&fresh).Error)
// Reopening with a one-hour expiry prunes the old row.
require.NoError(t, w.Reopen(time.Hour))
var got []archive.ExportArchivedEvent
require.NoError(t, w.DB().Find(&got).Error)
require.Len(t, got, 1)
assert.Equal(t, "fresh", got[0].EventID)
}
func TestParseArchiveExpiry(t *testing.T) {
t.Parallel()
cases := []struct {
name string
in string
want time.Duration
wantErr bool
}{
{"empty", "", 0, false},
{"explicit never", "never", 0, false},
{"hour", "1h", time.Hour, false},
{"thirty days", "720h", 720 * time.Hour, false},
{"minutes", "30m", 30 * time.Minute, false},
{"unparseable", "nonsense", 0, true},
{"days are not a Go unit", "7d", 0, true},
{"zero duration", "0s", 0, true},
{"bare zero", "0", 0, true},
{"negative duration", "-5h", 0, true},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
got, err := archive.ParseExpiry(tc.in)
if tc.wantErr {
require.Error(t, err)
return
}
require.NoError(t, err)
assert.Equal(t, tc.want, got)
})
}
}
// TestArchiveFileName pins the archive file name and the rules
// that make a webhook or target name safe to put in it.
func TestArchiveFileName(t *testing.T) {
t.Parallel()
const id = "3f2a1c9e-8d4b-4c1a-9e2f-0a1b2c3d4e5f"
cases := []struct {
name string
webhook string
target string
want string
}{
{
"plain names", "orders", "archive",
"archive-orders-archive-" + id + ".db",
},
{
"lowercased", "Orders", "Main Archive",
"archive-orders-main-archive-" + id + ".db",
},
{
"a run of other characters is one dash",
`a /\..b`, "c__--d",
"archive-a-b-c-d-" + id + ".db",
},
{
"no dash at either end", " --orders!! ", "(archive)",
"archive-orders-archive-" + id + ".db",
},
{
"path separators", "../../etc/passwd", "a/b",
"archive-etc-passwd-a-b-" + id + ".db",
},
{
"letters outside ASCII are dropped",
"Bestellungen Größe", "café",
"archive-bestellungen-gr-e-caf-" + id + ".db",
},
{
"nothing left is unnamed", "", "!!!",
"archive-unnamed-unnamed-" + id + ".db",
},
{
"cut to 40 characters", strings.Repeat("a", 50), "x",
"archive-" + strings.Repeat("a", 40) + "-x-" + id + ".db",
},
{
"no dash left by the cut",
strings.Repeat("a", 39) + " b", "x",
"archive-" + strings.Repeat("a", 39) + "-x-" + id + ".db",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
assert.Equal(
t, tc.want,
archive.FileName(tc.webhook, tc.target, id),
)
})
}
}
// TestWrite_EachTargetHasItsOwnArchive proves two
// database targets of one webhook archive into separate files.
func TestWrite_EachTargetHasItsOwnArchive(t *testing.T) {
t.Parallel()
env := setupArchiveTest(t)
first := env.seedDatabaseTarget(t, 0, "")
second := env.addDatabaseTarget(t, first.WebhookID, 0, "")
event := newEvent(`{"n":1}`)
for _, tgt := range []*database.Target{first, second} {
require.NoError(t, env.writers.Write(
databaseTargetDelivery(event, tgt),
))
}
require.NotEqual(
t, env.archivePath(first), env.archivePath(second),
)
assert.Equal(
t, []string{event.ID},
archivedEventIDs(t, env.archivePath(first)),
)
assert.Equal(
t, []string{event.ID},
archivedEventIDs(t, env.archivePath(second)),
)
}
// TestRename_MovesTheFile proves a rename moves the archive, rows
// and all, and that later writes go to the new name.
func TestRename_MovesTheFile(t *testing.T) {
t.Parallel()
env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, "")
oldPath := env.archivePath(tgt)
first := newEvent(`{"n":1}`)
require.NoError(t, env.writers.Write(databaseTargetDelivery(first, tgt)))
require.FileExists(t, oldPath)
require.NoError(
t, env.writers.Rename(tgt.ID, "Orders", "Long Term"),
)
newPath := filepath.Join(
env.dataDir, "archive-orders-long-term-"+tgt.ID+".db",
)
assert.NoFileExists(t, oldPath)
assert.Equal(t, []string{first.ID}, archivedEventIDs(t, newPath))
second := newEvent(`{"n":2}`)
require.NoError(t, env.writers.Write(databaseTargetDelivery(second, tgt)))
assert.ElementsMatch(
t, []string{first.ID, second.ID},
archivedEventIDs(t, newPath),
)
assert.NoFileExists(
t, oldPath, "a write after the rename must use the new name",
)
}
// TestRename_NeverReplacesAFile plants a file at the new name, once
// the .db alone, once a lone -wal and once a lone -shm, and proves
// each time that the rename is refused, the planted file survives,
// and the archive keeps its name and its rows.
func TestRename_NeverReplacesAFile(t *testing.T) {
t.Parallel()
for _, suffix := range archiveFileSuffixes() {
t.Run("planted .db"+suffix, func(t *testing.T) {
t.Parallel()
env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, "")
oldPath := env.archivePath(tgt)
first := newEvent(`{"n":1}`)
require.NoError(
t, env.writers.Write(databaseTargetDelivery(first, tgt)),
)
newPath := filepath.Join(
env.dataDir, "archive-orders-long-term-"+tgt.ID+".db",
)
plantedPath := newPath + suffix
require.NoError(
t, os.WriteFile(plantedPath, []byte("planted"), 0o600),
)
require.ErrorIs(
t, env.writers.Rename(tgt.ID, "Orders", "Long Term"),
archive.ErrNameTaken,
)
//nolint:gosec // reads the file the test planted under t.TempDir()
planted, err := os.ReadFile(plantedPath)
require.NoError(t, err)
assert.Equal(t, "planted", string(planted))
second := newEvent(`{"n":2}`)
require.NoError(
t, env.writers.Write(databaseTargetDelivery(second, tgt)),
)
assert.ElementsMatch(
t, []string{first.ID, second.ID},
archivedEventIDs(t, oldPath),
)
})
}
}
// TestRename_BeforeTheNameIsSaved covers the order the handlers
// use: they rename before they save the new name, so a delivery in
// between must write under the new name although the main database
// still has the old one. It also shows that renaming an archive that
// does not exist yet is not an error.
func TestRename_BeforeTheNameIsSaved(t *testing.T) {
t.Parallel()
env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, "")
require.NoError(
t, env.writers.Rename(tgt.ID, "Orders", "Archive"),
)
event := newEvent(`{"n":1}`)
require.NoError(t, env.writers.Write(databaseTargetDelivery(event, tgt)))
assert.FileExists(
t,
filepath.Join(
env.dataDir, "archive-orders-archive-"+tgt.ID+".db",
),
)
assert.NoFileExists(t, env.archivePath(tgt))
}
// TestArchiveWriter_RenameMovesSidecars proves a rename carries
// the -wal and -shm a crash can leave beside an archive no handle
// has opened since. SQLite finds them by name, so a -wal left
// behind would lose the transactions it holds.
func TestArchiveWriter_RenameMovesSidecars(t *testing.T) {
t.Parallel()
dir := t.TempDir()
oldPath := filepath.Join(dir, "archive-old.db")
newPath := filepath.Join(dir, "archive-new.db")
for _, suffix := range archiveFileSuffixes() {
require.NoError(
t, os.WriteFile(oldPath+suffix, []byte(suffix), 0o600),
)
}
w := archive.NewExportArchiveWriter(
oldPath, archiveTestLogger(), 0,
)
require.NoError(t, w.Rename("archive-new.db"))
for _, suffix := range archiveFileSuffixes() {
assert.NoFileExists(t, oldPath+suffix)
assert.FileExists(t, newPath+suffix)
}
assert.Equal(t, newPath, w.Path())
}
// TestArchiveWriter_RenameMovesBackOnFailure makes the -wal fail to
// move after the .db has moved, and proves the .db is moved back, so
// the archive is never split across two names. The new name is 255
// bytes, the longest a file name may be, so the .db can take it but
// the -wal, four bytes longer, cannot.
func TestArchiveWriter_RenameMovesBackOnFailure(t *testing.T) {
t.Parallel()
dir := t.TempDir()
oldPath := filepath.Join(dir, "archive-old.db")
newName := strings.Repeat("a", 252) + ".db"
for _, suffix := range archiveFileSuffixes() {
require.NoError(
t, os.WriteFile(oldPath+suffix, []byte(suffix), 0o600),
)
}
w := archive.NewExportArchiveWriter(
oldPath, archiveTestLogger(), 0,
)
require.Error(t, w.Rename(newName))
for _, suffix := range archiveFileSuffixes() {
assert.FileExists(t, oldPath+suffix)
}
assert.NoFileExists(t, filepath.Join(dir, newName))
assert.Equal(t, oldPath, w.Path())
}
// TestArchiveWriter_RenameMovesBackEveryFile renames a target with a
// file without a period and a file for a month, and proves that when
// the month's file fails to move, the file already moved is moved back:
// both files are under the old name with their rows, and nothing is
// under the new name. The new name is 251 bytes, so the file without a
// period, with its -wal and -shm, can take it, but the month's file,
// eight bytes longer, cannot.
func TestArchiveWriter_RenameMovesBackEveryFile(t *testing.T) {
t.Parallel()
dir := t.TempDir()
oldPath := filepath.Join(dir, "archive-old.db")
monthPath := filepath.Join(dir, "archive-old-2026-03.db")
w := archive.NewExportArchiveWriter(oldPath, archiveTestLogger(), 0)
require.NoError(t, w.WritePeriod(
archive.ExportArchivedEvent{EventID: "in-none"}, 0, "",
))
require.NoError(t, w.WritePeriod(
archive.ExportArchivedEvent{EventID: "in-month"}, 0, "2026-03",
))
require.Error(t, w.Rename(strings.Repeat("a", 248)+".db"))
assert.Equal(t, []string{"in-none"}, archivedEventIDs(t, oldPath))
assert.Equal(t, []string{"in-month"}, archivedEventIDs(t, monthPath))
entries, err := os.ReadDir(dir)
require.NoError(t, err)
for _, entry := range entries {
assert.True(t, strings.HasPrefix(entry.Name(), "archive-old"),
"%s is not under the old name", entry.Name())
}
assert.Equal(t, oldPath, w.Path())
}
// TestRenameLock_HeldByOneEditAtATime holds the rename lock, as an edit
// does from loading the stored names until it has saved them. While it
// is held, the edit's rename and an archive write still finish, since
// neither takes it, and the event ends up under the new name whichever
// ran first. A second edit waits for the lock until the first lets go.
func TestRenameLock_HeldByOneEditAtATime(t *testing.T) {
t.Parallel()
env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, "")
event := newEvent(`{"n":1}`)
env.writers.RenameLock().Lock()
finished := make(chan error, 2)
go func() {
finished <- env.writers.Rename(tgt.ID, "Orders", "Long Term")
}()
go func() {
finished <- env.writers.Write(databaseTargetDelivery(event, tgt))
}()
for range 2 {
select {
case err := <-finished:
require.NoError(t, err)
case <-time.After(10 * time.Second):
t.Fatal("a rename or a write waited for the rename lock")
}
}
newPath := filepath.Join(
env.dataDir, "archive-orders-long-term-"+tgt.ID+".db",
)
assert.Equal(t, []string{event.ID}, archivedEventIDs(t, newPath))
secondEdit := make(chan struct{})
go func() {
env.writers.RenameLock().Lock()
close(secondEdit)
env.writers.RenameLock().Unlock()
}()
select {
case <-secondEdit:
t.Fatal("a second edit took the rename lock while it was held")
case <-time.After(100 * time.Millisecond):
}
env.writers.RenameLock().Unlock()
select {
case <-secondEdit:
case <-time.After(10 * time.Second):
t.Fatal("a second edit did not get the rename lock once released")
}
}
@@ -1,4 +1,4 @@
package delivery package archive
import ( import (
"compress/gzip" "compress/gzip"
@@ -28,25 +28,25 @@ import (
const archiveTableQuery = "SELECT count(*) FROM sqlite_master " + const archiveTableQuery = "SELECT count(*) FROM sqlite_master " +
"WHERE type = 'table' AND name = 'archived_events'" "WHERE type = 'table' AND name = 'archived_events'"
// ArchivePath returns where a database target's archive file is: in // Path returns where a database target's archive file is: in
// the data directory, beside the webhook's event database, under the // the data directory, beside the webhook's event database, under the
// name ArchiveFileName gives it. // name FileName gives it.
func ArchivePath( func Path(
dbMgr *database.WebhookDBManager, dbMgr *database.WebhookDBManager,
webhook *database.Webhook, webhook *database.Webhook,
target *database.Target, target *database.Target,
) string { ) string {
return filepath.Join( return filepath.Join(
filepath.Dir(dbMgr.DBPath(webhook.ID)), filepath.Dir(dbMgr.DBPath(webhook.ID)),
ArchiveFileName(webhook.Name, target.Name, target.ID), FileName(webhook.Name, target.Name, target.ID),
) )
} }
// ArchiveExportFileName returns the name a database target's archive // ExportFileName returns the name a database target's archive
// downloads under: // downloads under:
// archive-WEBHOOKNAME-TARGETNAME-YYYYMMDDTHHMMSSZ.json.gz, the names // archive-WEBHOOKNAME-TARGETNAME-YYYYMMDDTHHMMSSZ.json.gz, the names
// made safe as in ArchiveFileName and the time in UTC. // made safe as in FileName and the time in UTC.
func ArchiveExportFileName( func ExportFileName(
webhookName, targetName string, at time.Time, webhookName, targetName string, at time.Time,
) string { ) string {
return "archive-" + archiveNamePart(webhookName) + "-" + return "archive-" + archiveNamePart(webhookName) + "-" +
@@ -54,7 +54,7 @@ func ArchiveExportFileName(
at.UTC().Format("20060102T150405Z") + ".json.gz" at.UTC().Format("20060102T150405Z") + ".json.gz"
} }
// ArchiveExport is a database target's archive listed for download. It // Export is a database target's archive listed for download. It
// opens one of the target's files at a time, only when its rows are // opens one of the target's files at a time, only when its rows are
// about to be written out, and closes it before it opens the next, so // about to be written out, and closes it before it opens the next, so
// an export holds at most one file open however many the target has. // an export holds at most one file open however many the target has.
@@ -66,7 +66,7 @@ func ArchiveExportFileName(
// is open, and the export does not see them. SQLite cannot checkpoint // is open, and the export does not see them. SQLite cannot checkpoint
// a -wal past an open snapshot, so the open file's -wal grows until the // a -wal past an open snapshot, so the open file's -wal grows until the
// export has written that file out. // export has written that file out.
type ArchiveExport struct { type Export struct {
// periods are the periods of the target's files when the export // periods are the periods of the target's files when the export
// was listed, "" for the file without one, in the order // was listed, "" for the file without one, in the order
// archiveFiles lists them. // archiveFiles lists them.
@@ -77,7 +77,7 @@ type ArchiveExport struct {
// between. // between.
lock sync.Locker lock sync.Locker
// currentPath returns the path ArchivePath gives the target under // currentPath returns the path Path gives the target under
// the names stored for it now, which a rename may have changed since // the names stored for it now, which a rename may have changed since
// the export was listed. // the export was listed.
currentPath func() (string, error) currentPath func() (string, error)
@@ -103,11 +103,12 @@ type exportedName struct {
Name string `json:"name"` Name string `json:"name"`
} }
// NewArchiveExport lists a database target's archive files for export, // NewExport lists a database target's archive files for export,
// given the path ArchivePath gives it (see archiveFiles). It opens none // given the path Path gives it (see archiveFiles). It opens none
// of them. Its caller holds lock, which every rename of the target's // of them. Its caller holds lock, which every rename of the target's
// files runs under, from reading the names path is made of until it // files runs under (Writers.RenameLock returns it), from reading the names
// returns, so the files it lists are the ones those names give. // path is made of until it returns, so the files it lists are the ones
// those names give.
// //
// WriteGzipJSON, called without lock held, finds each file again by its // WriteGzipJSON, called without lock held, finds each file again by its
// period under the path currentPath gives, holding lock while it does // period under the path currentPath gives, holding lock while it does
@@ -115,18 +116,18 @@ type exportedName struct {
// file. A file that is gone by then, emptied by the sweep or moved // file. A file that is gone by then, emptied by the sweep or moved
// away, is skipped. The export never creates a file: with no files, it // away, is skipped. The export never creates a file: with no files, it
// has no rows. // has no rows.
func NewArchiveExport( func NewExport(
path string, path string,
lock sync.Locker, lock sync.Locker,
currentPath func() (string, error), currentPath func() (string, error),
log *slog.Logger, log *slog.Logger,
) (*ArchiveExport, error) { ) (*Export, error) {
files, err := archiveFiles(path) files, err := archiveFiles(path)
if err != nil { if err != nil {
return nil, err return nil, err
} }
x := &ArchiveExport{lock: lock, currentPath: currentPath, log: log} x := &Export{lock: lock, currentPath: currentPath, log: log}
for _, file := range files { for _, file := range files {
x.periods = append(x.periods, file.period) x.periods = append(x.periods, file.period)
@@ -199,7 +200,7 @@ func openExportFile(
// closed once its rows are written, before the next is opened. When it // closed once its rows are written, before the next is opened. When it
// returns, no file is open. After an error the gzip stream is left // returns, no file is open. After an error the gzip stream is left
// unfinished, so what was written does not decompress as a whole file. // unfinished, so what was written does not decompress as a whole file.
func (x *ArchiveExport) WriteGzipJSON( func (x *Export) WriteGzipJSON(
ctx context.Context, ctx context.Context,
w io.Writer, w io.Writer,
webhook *database.Webhook, webhook *database.Webhook,
@@ -228,7 +229,7 @@ func (x *ArchiveExport) WriteGzipJSON(
// openFile finds the target's archive file for period under the path // openFile finds the target's archive file for period under the path
// currentPath gives now, and opens it for the export, holding x.lock // currentPath gives now, and opens it for the export, holding x.lock
// for both. For a file that is gone, the error wraps fs.ErrNotExist. // for both. For a file that is gone, the error wraps fs.ErrNotExist.
func (x *ArchiveExport) openFile( func (x *Export) openFile(
ctx context.Context, period string, ctx context.Context, period string,
) (*exportFile, error) { ) (*exportFile, error) {
x.lock.Lock() x.lock.Lock()
@@ -258,7 +259,7 @@ func (f *exportFile) close() error {
// writeJSON writes head with archived_events added as its last key, // writeJSON writes head with archived_events added as its last key,
// the rows going into it one at a time. // the rows going into it one at a time.
func (x *ArchiveExport) writeJSON( func (x *Export) writeJSON(
ctx context.Context, w io.Writer, head []byte, ctx context.Context, w io.Writer, head []byte,
) error { ) error {
// head goes out without its closing brace, so that // head goes out without its closing brace, so that
@@ -286,7 +287,7 @@ func (x *ArchiveExport) writeJSON(
// writeRows writes the archived rows of each file to w, one per line, // writeRows writes the archived rows of each file to w, one per line,
// separated by commas, opening each file in turn and closing it once // separated by commas, opening each file in turn and closing it once
// its rows are written. // its rows are written.
func (x *ArchiveExport) writeRows(ctx context.Context, w io.Writer) error { func (x *Export) writeRows(ctx context.Context, w io.Writer) error {
sep := "\n" sep := "\n"
for _, period := range x.periods { for _, period := range x.periods {
@@ -1,4 +1,4 @@
package delivery_test package archive_test
import ( import (
"bufio" "bufio"
@@ -19,8 +19,8 @@ import (
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
) )
// The webhook and the target the export tests' archives belong to. // The webhook and the target the export tests' archives belong to.
@@ -37,7 +37,7 @@ const binaryBody = "\xff\xfe\x00\x01binary\x80"
// writeExportTo writes export to w as the archive of the export tests' // writeExportTo writes export to w as the archive of the export tests'
// webhook and target, exported at 2026-10-02T12:03:04Z. // webhook and target, exported at 2026-10-02T12:03:04Z.
func writeExportTo( func writeExportTo(
t *testing.T, export *delivery.ArchiveExport, w io.Writer, t *testing.T, export *archive.Export, w io.Writer,
) error { ) error {
t.Helper() t.Helper()
@@ -57,7 +57,7 @@ func writeExportTo(
// listExport lists the archive at path for export, as the archive of a // listExport lists the archive at path for export, as the archive of a
// target whose names do not change. // target whose names do not change.
func listExport(t *testing.T, path string) *delivery.ArchiveExport { func listExport(t *testing.T, path string) *archive.Export {
t.Helper() t.Helper()
return newExport(t, path, &sync.Mutex{}, func() (string, error) { return newExport(t, path, &sync.Mutex{}, func() (string, error) {
@@ -73,10 +73,10 @@ func newExport(
path string, path string,
lock sync.Locker, lock sync.Locker,
currentPath func() (string, error), currentPath func() (string, error),
) *delivery.ArchiveExport { ) *archive.Export {
t.Helper() t.Helper()
export, err := delivery.NewArchiveExport( export, err := archive.NewExport(
path, lock, currentPath, archiveTestLogger(), path, lock, currentPath, archiveTestLogger(),
) )
require.NoError(t, err) require.NoError(t, err)
@@ -96,7 +96,7 @@ func exportArchive(t *testing.T, path string) map[string]any {
// decompressed and parsed. Reading to the end makes the gzip reader // decompressed and parsed. Reading to the end makes the gzip reader
// check that the stream was finished. // check that the stream was finished.
func writeExport( func writeExport(
t *testing.T, export *delivery.ArchiveExport, t *testing.T, export *archive.Export,
) map[string]any { ) map[string]any {
t.Helper() t.Helper()
@@ -157,11 +157,11 @@ func TestArchiveExport_MatchesStoredRows(t *testing.T) {
t.Parallel() t.Parallel()
path := filepath.Join(t.TempDir(), "archive.db") path := filepath.Join(t.TempDir(), "archive.db")
w := delivery.NewExportArchiveWriter(path, archiveTestLogger(), 0) w := archive.NewExportArchiveWriter(path, archiveTestLogger(), 0)
bodies := []string{`{"order":1}`, "plain text", "", binaryBody} bodies := []string{`{"order":1}`, "plain text", "", binaryBody}
for i, body := range bodies { for i, body := range bodies {
require.NoError(t, w.Write(delivery.ExportArchivedEvent{ require.NoError(t, w.Write(archive.ExportArchivedEvent{
EventID: fmt.Sprintf("ev-%d", i), EventID: fmt.Sprintf("ev-%d", i),
WebhookID: exportWebhookID, WebhookID: exportWebhookID,
EntrypointID: "ep-1", EntrypointID: "ep-1",
@@ -173,7 +173,7 @@ func TestArchiveExport_MatchesStoredRows(t *testing.T) {
}, 0)) }, 0))
} }
var stored []delivery.ExportArchivedEvent var stored []archive.ExportArchivedEvent
require.NoError(t, openArchiveDBForRead(t, path). require.NoError(t, openArchiveDBForRead(t, path).
Order("id").Find(&stored).Error) Order("id").Find(&stored).Error)
@@ -201,7 +201,7 @@ func TestArchiveExport_MatchesStoredRows(t *testing.T) {
// assertExportedRow checks that ev, from an export, holds every column // assertExportedRow checks that ev, from an export, holds every column
// of the stored row. // of the stored row.
func assertExportedRow( func assertExportedRow(
t *testing.T, row delivery.ExportArchivedEvent, ev map[string]any, t *testing.T, row archive.ExportArchivedEvent, ev map[string]any,
) { ) {
t.Helper() t.Helper()
@@ -248,7 +248,7 @@ func TestArchiveExport_Empty(t *testing.T) {
require.NoError(t, os.WriteFile(noTable, nil, 0o600)) require.NoError(t, os.WriteFile(noTable, nil, 0o600))
require.NoError(t, require.NoError(t,
delivery.NewExportArchiveWriter(noRows, archiveTestLogger(), 0). archive.NewExportArchiveWriter(noRows, archiveTestLogger(), 0).
Open(0), Open(0),
) )
@@ -284,18 +284,18 @@ func TestArchiveExport_ReadsOneSnapshot(t *testing.T) {
t.Parallel() t.Parallel()
path := filepath.Join(t.TempDir(), "archive.db") path := filepath.Join(t.TempDir(), "archive.db")
w := delivery.NewExportArchiveWriter(path, archiveTestLogger(), 0) w := archive.NewExportArchiveWriter(path, archiveTestLogger(), 0)
require.NoError(t, w.Write(delivery.ExportArchivedEvent{EventID: "listed"}, 0)) require.NoError(t, w.Write(archive.ExportArchivedEvent{EventID: "listed"}, 0))
opened := &unlockHook{fn: func() { opened := &unlockHook{fn: func() {
require.NoError(t, w.Write(delivery.ExportArchivedEvent{EventID: "during"}, 0)) require.NoError(t, w.Write(archive.ExportArchivedEvent{EventID: "during"}, 0))
}} }}
export := newExport(t, path, opened, func() (string, error) { export := newExport(t, path, opened, func() (string, error) {
return path, nil return path, nil
}) })
require.NoError(t, w.Write(delivery.ExportArchivedEvent{EventID: "before-open"}, 0)) require.NoError(t, w.Write(archive.ExportArchivedEvent{EventID: "before-open"}, 0))
assert.Equal(t, assert.Equal(t,
[]string{"listed", "before-open"}, []string{"listed", "before-open"},
@@ -305,7 +305,7 @@ func TestArchiveExport_ReadsOneSnapshot(t *testing.T) {
var stored int64 var stored int64
require.NoError(t, openArchiveDBForRead(t, path). require.NoError(t, openArchiveDBForRead(t, path).
Model(&delivery.ExportArchivedEvent{}).Count(&stored).Error) Model(&archive.ExportArchivedEvent{}).Count(&stored).Error)
assert.Equal(t, int64(3), stored) assert.Equal(t, int64(3), stored)
} }
@@ -318,11 +318,11 @@ func TestArchiveExport_FindsFilesAfterRename(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
path := filepath.Join(dir, "archive-old.db") path := filepath.Join(dir, "archive-old.db")
w := delivery.NewExportArchiveWriter(path, archiveTestLogger(), 0) w := archive.NewExportArchiveWriter(path, archiveTestLogger(), 0)
for _, period := range []string{"", dayPeriod, hourPeriod} { for _, period := range []string{"", dayPeriod, hourPeriod} {
require.NoError(t, w.WritePeriod( require.NoError(t, w.WritePeriod(
delivery.ExportArchivedEvent{EventID: "in-" + period}, 0, period, archive.ExportArchivedEvent{EventID: "in-" + period}, 0, period,
)) ))
} }
@@ -353,19 +353,19 @@ func TestArchiveExport_EveryFileOldestFirst(t *testing.T) {
t.Parallel() t.Parallel()
path := filepath.Join(t.TempDir(), "archive-wh.db") path := filepath.Join(t.TempDir(), "archive-wh.db")
w := delivery.NewExportArchiveWriter(path, archiveTestLogger(), 0) w := archive.NewExportArchiveWriter(path, archiveTestLogger(), 0)
// Written in an order that is not the export's. // Written in an order that is not the export's.
for _, period := range []string{nextDayPeriod, "", hourPeriod, "2026-03"} { for _, period := range []string{nextDayPeriod, "", hourPeriod, "2026-03"} {
require.NoError(t, w.WritePeriod( require.NoError(t, w.WritePeriod(
delivery.ExportArchivedEvent{EventID: "in-" + period}, 0, period, archive.ExportArchivedEvent{EventID: "in-" + period}, 0, period,
)) ))
} }
export := listExport(t, path) export := listExport(t, path)
require.NoError(t, w.WritePeriod( require.NoError(t, w.WritePeriod(
delivery.ExportArchivedEvent{EventID: "later"}, 0, "2026-03-06", archive.ExportArchivedEvent{EventID: "later"}, 0, "2026-03-06",
)) ))
events := exportedEvents(t, writeExport(t, export)) events := exportedEvents(t, writeExport(t, export))
@@ -434,13 +434,13 @@ func TestArchiveExport_OneFileOpenAtATime(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
path := filepath.Join(dir, "archive-wh.db") path := filepath.Join(dir, "archive-wh.db")
w := delivery.NewExportArchiveWriter(path, archiveTestLogger(), 0) w := archive.NewExportArchiveWriter(path, archiveTestLogger(), 0)
random := make([]byte, 36<<10) random := make([]byte, 36<<10)
for hour := range 24 { for hour := range 24 {
_, _ = rand.Read(random) _, _ = rand.Read(random)
require.NoError(t, w.WritePeriod(delivery.ExportArchivedEvent{ require.NoError(t, w.WritePeriod(archive.ExportArchivedEvent{
Body: base64.StdEncoding.EncodeToString(random), Body: base64.StdEncoding.EncodeToString(random),
}, 0, fmt.Sprintf("2026-10-01-%02d", hour))) }, 0, fmt.Sprintf("2026-10-01-%02d", hour)))
} }
@@ -489,7 +489,7 @@ func exportHeapGrowth(t *testing.T, rows, bodySize int) uint64 {
t.Helper() t.Helper()
path := filepath.Join(t.TempDir(), "archive.db") path := filepath.Join(t.TempDir(), "archive.db")
w := delivery.NewExportArchiveWriter(path, archiveTestLogger(), 0) w := archive.NewExportArchiveWriter(path, archiveTestLogger(), 0)
// Base64 makes four characters of every three bytes. // Base64 makes four characters of every three bytes.
random := make([]byte, bodySize/4*3) random := make([]byte, bodySize/4*3)
@@ -497,7 +497,7 @@ func exportHeapGrowth(t *testing.T, rows, bodySize int) uint64 {
for range rows { for range rows {
_, _ = rand.Read(random) _, _ = rand.Read(random)
require.NoError(t, w.Write(delivery.ExportArchivedEvent{ require.NoError(t, w.Write(archive.ExportArchivedEvent{
Body: base64.StdEncoding.EncodeToString(random), Body: base64.StdEncoding.EncodeToString(random),
}, 0)) }, 0))
} }
@@ -565,7 +565,7 @@ func TestArchiveExportFileName(t *testing.T) {
assert.Equal(t, assert.Equal(t,
"archive-orders-eu-long-term-archive-20261002T120304Z.json.gz", "archive-orders-eu-long-term-archive-20261002T120304Z.json.gz",
delivery.ArchiveExportFileName( archive.ExportFileName(
exportWebhookName, exportTargetName, exportWebhookName, exportTargetName,
time.Date(2026, 10, 2, 14, 3, 4, 0, cest), time.Date(2026, 10, 2, 14, 3, 4, 0, cest),
), ),
@@ -1,7 +1,6 @@
package delivery_test package archive_test
import ( import (
"context"
"errors" "errors"
"fmt" "fmt"
"net/http" "net/http"
@@ -13,8 +12,8 @@ import (
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
) )
// deliverTo archives one event to a database target, leaving the // deliverTo archives one event to a database target, leaving the
@@ -24,12 +23,9 @@ func deliverTo(
) { ) {
t.Helper() t.Helper()
webhookDB := testWebhookDB(t) event := newEvent(`{"archived":true}`)
event := seedEvent(t, webhookDB, `{"archived":true}`)
env.eng.ExportDeliverDatabase( require.NoError(t, env.writers.Write(databaseTargetDelivery(event, tgt)))
webhookDB, seedDatabaseTargetDelivery(t, webhookDB, event, tgt),
)
} }
// TestEvictWebhook_ClosesAndRemovesWriter proves that evicting // TestEvictWebhook_ClosesAndRemovesWriter proves that evicting
@@ -41,28 +37,28 @@ func TestEvictWebhook_ClosesAndRemovesWriter(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
first := env.seedDatabaseTarget(t, "") first := env.seedDatabaseTarget(t, 0, "")
second := env.addDatabaseTarget(t, first.WebhookID, "") second := env.addDatabaseTarget(t, first.WebhookID, 0, "")
other := env.seedDatabaseTarget(t, "") other := env.seedDatabaseTarget(t, 0, "")
for _, tgt := range []*database.Target{first, second, other} { for _, tgt := range []*database.Target{first, second, other} {
deliverTo(t, env, tgt) deliverTo(t, env, tgt)
require.True( require.True(
t, env.eng.ExportArchiveHandleOpen(tgt.ID), t, env.writers.ExportArchiveHandleOpen(tgt.ID),
"the writer should hold an open handle after a write", "the writer should hold an open handle after a write",
) )
} }
env.eng.EvictWebhook(first.WebhookID) env.writers.EvictWebhook(first.WebhookID)
for _, tgt := range []*database.Target{first, second} { for _, tgt := range []*database.Target{first, second} {
assert.False( assert.False(
t, env.eng.ExportHasArchiveWriter(tgt.ID), t, env.writers.ExportHasArchiveWriter(tgt.ID),
"eviction should remove the registry entry", "eviction should remove the registry entry",
) )
assert.False( assert.False(
t, env.eng.ExportArchiveHandleOpen(tgt.ID), t, env.writers.ExportArchiveHandleOpen(tgt.ID),
"eviction should close the archive handle", "eviction should close the archive handle",
) )
assert.FileExists( assert.FileExists(
@@ -72,7 +68,7 @@ func TestEvictWebhook_ClosesAndRemovesWriter(t *testing.T) {
} }
assert.True( assert.True(
t, env.eng.ExportArchiveHandleOpen(other.ID), t, env.writers.ExportArchiveHandleOpen(other.ID),
"another webhook's writer must be left alone", "another webhook's writer must be left alone",
) )
} }
@@ -84,21 +80,21 @@ func TestEvictTarget_LeavesOtherTargets(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
doomed := env.seedDatabaseTarget(t, "") doomed := env.seedDatabaseTarget(t, 0, "")
kept := env.addDatabaseTarget(t, doomed.WebhookID, "") kept := env.addDatabaseTarget(t, doomed.WebhookID, 0, "")
deliverTo(t, env, doomed) deliverTo(t, env, doomed)
deliverTo(t, env, kept) deliverTo(t, env, kept)
env.eng.EvictTarget(doomed.ID) env.writers.EvictTarget(doomed.ID)
assert.False(t, env.eng.ExportHasArchiveWriter(doomed.ID)) assert.False(t, env.writers.ExportHasArchiveWriter(doomed.ID))
assert.FileExists( assert.FileExists(
t, env.archivePath(doomed), t, env.archivePath(doomed),
"eviction must not delete the archive file", "eviction must not delete the archive file",
) )
assert.True( assert.True(
t, env.eng.ExportArchiveHandleOpen(kept.ID), t, env.writers.ExportArchiveHandleOpen(kept.ID),
"the other target's writer must be left alone", "the other target's writer must be left alone",
) )
} }
@@ -112,21 +108,21 @@ func TestEvictWebhook_UnknownWebhookIsNoOp(t *testing.T) {
env := setupArchiveTest(t) env := setupArchiveTest(t)
assert.NotPanics(t, func() { assert.NotPanics(t, func() {
env.eng.EvictWebhook("no-such-webhook") env.writers.EvictWebhook("no-such-webhook")
env.eng.EvictWebhook("no-such-webhook") env.writers.EvictWebhook("no-such-webhook")
env.eng.EvictTarget("no-such-target") env.writers.EvictTarget("no-such-target")
env.eng.EvictTarget("no-such-target") env.writers.EvictTarget("no-such-target")
}) })
assert.False( assert.False(
t, env.eng.ExportHasArchiveWriter("no-such-target"), t, env.writers.ExportHasArchiveWriter("no-such-target"),
"eviction must not create a writer", "eviction must not create a writer",
) )
} }
// evictTestRow builds an archive row for the eviction tests. // evictTestRow builds an archive row for the eviction tests.
func evictTestRow(eventID string) delivery.ExportArchivedEvent { func evictTestRow(eventID string) archive.ExportArchivedEvent {
return delivery.ExportArchivedEvent{ return archive.ExportArchivedEvent{
EventID: eventID, EventID: eventID,
WebhookID: "wh-evict", WebhookID: "wh-evict",
Method: http.MethodPost, Method: http.MethodPost,
@@ -147,7 +143,7 @@ func TestEvictedWriter_WriteDoesNotReopenFile(t *testing.T) {
path := filepath.Join(t.TempDir(), "archive-evicted.db") path := filepath.Join(t.TempDir(), "archive-evicted.db")
w := delivery.NewExportArchiveWriter( w := archive.NewExportArchiveWriter(
path, archiveTestLogger(), 0, path, archiveTestLogger(), 0,
) )
@@ -164,7 +160,7 @@ func TestEvictedWriter_WriteDoesNotReopenFile(t *testing.T) {
err := w.Write(evictTestRow("ev-2"), 0) err := w.Write(evictTestRow("ev-2"), 0)
require.ErrorIs( require.ErrorIs(
t, err, delivery.ErrExportArchiveWriterEvicted, t, err, archive.ErrExportArchiveWriterEvicted,
"an evicted writer must refuse writes", "an evicted writer must refuse writes",
) )
assert.NoFileExists( assert.NoFileExists(
@@ -187,7 +183,7 @@ func TestEvictedWriter_SweepDoesNotReopenFile(t *testing.T) {
path := filepath.Join(t.TempDir(), "archive-evicted.db") path := filepath.Join(t.TempDir(), "archive-evicted.db")
w := delivery.NewExportArchiveWriter( w := archive.NewExportArchiveWriter(
path, archiveTestLogger(), 0, path, archiveTestLogger(), 0,
) )
@@ -199,7 +195,7 @@ func TestEvictedWriter_SweepDoesNotReopenFile(t *testing.T) {
err := w.SweepExpired(time.Hour) err := w.SweepExpired(time.Hour)
require.ErrorIs( require.ErrorIs(
t, err, delivery.ErrExportArchiveWriterEvicted, t, err, archive.ErrExportArchiveWriterEvicted,
"an evicted writer must refuse an idle sweep", "an evicted writer must refuse an idle sweep",
) )
assert.False( assert.False(
@@ -230,7 +226,7 @@ const racingWriteGoroutines = 4
// against a hang, not a timing assumption: the first write after // against a hang, not a timing assumption: the first write after
// the eviction is refused. // the eviction is refused.
func startRacingWrites( func startRacingWrites(
w *delivery.ExportArchiveWriter, w *archive.ExportArchiveWriter,
) *racingWrites { ) *racingWrites {
r := &racingWrites{ r := &racingWrites{
started: make(chan struct{}, racingWriteGoroutines), started: make(chan struct{}, racingWriteGoroutines),
@@ -276,7 +272,7 @@ func (r *racingWrites) record(err error) {
r.mu.Lock() r.mu.Lock()
defer r.mu.Unlock() defer r.mu.Unlock()
if errors.Is(err, delivery.ErrExportArchiveWriterEvicted) { if errors.Is(err, archive.ErrExportArchiveWriterEvicted) {
r.sawEvicted = true r.sawEvicted = true
return return
@@ -314,13 +310,13 @@ func TestEvictWebhook_RacingWriteDoesNotReopenHandle(
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, "") tgt := env.seedDatabaseTarget(t, 0, "")
// Prime the registry so the test can hold the very writer the // Prime the registry so the test can hold the very writer the
// eviction is about to detach. // eviction is about to detach.
deliverTo(t, env, tgt) deliverTo(t, env, tgt)
w := env.eng.ExportArchiveWriterFor(tgt.ID) w := env.writers.ExportArchiveWriterFor(tgt.ID)
require.NotNil(t, w) require.NotNil(t, w)
require.True(t, w.HandleOpen()) require.True(t, w.HandleOpen())
@@ -330,7 +326,7 @@ func TestEvictWebhook_RacingWriteDoesNotReopenHandle(
// eviction has to contend for the writer's mutex. // eviction has to contend for the writer's mutex.
race.awaitFirstWrite() race.awaitFirstWrite()
env.eng.EvictWebhook(tgt.WebhookID) env.writers.EvictWebhook(tgt.WebhookID)
sawEvicted, otherErr := race.wait() sawEvicted, otherErr := race.wait()
@@ -345,7 +341,7 @@ func TestEvictWebhook_RacingWriteDoesNotReopenHandle(
"been evicted", "been evicted",
) )
assert.False( assert.False(
t, env.eng.ExportHasArchiveWriter(tgt.ID), t, env.writers.ExportHasArchiveWriter(tgt.ID),
"the registry entry must stay gone", "the registry entry must stay gone",
) )
} }
@@ -359,45 +355,45 @@ func TestEvictWebhook_LaterDeliveryRecreatesWriter(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, "") tgt := env.seedDatabaseTarget(t, 0, "")
deliverTo(t, env, tgt) deliverTo(t, env, tgt)
require.True(t, env.eng.ExportHasArchiveWriter(tgt.ID)) require.True(t, env.writers.ExportHasArchiveWriter(tgt.ID))
env.eng.EvictWebhook(tgt.WebhookID) env.writers.EvictWebhook(tgt.WebhookID)
// A fresh delivery for the same target gets a brand new // A fresh delivery for the same target gets a brand new
// writer from the registry, so archiving keeps working. // writer from the registry, so archiving keeps working.
deliverTo(t, env, tgt) deliverTo(t, env, tgt)
assert.True( assert.True(
t, env.eng.ExportHasArchiveWriter(tgt.ID), t, env.writers.ExportHasArchiveWriter(tgt.ID),
"a later delivery should recreate the writer", "a later delivery should recreate the writer",
) )
} }
// TestEngineStop_WriteAfterStopIsRefused proves the engine's stop // TestEvictAll_WriteAfterIsRefused proves that EvictAll, which the
// closes each archive writer the way deleting its webhook does: a // delivery engine calls when it stops, closes each archive writer the
// write that reaches a writer after the stop is refused, reopens // way deleting its webhook does: a write that reaches a writer after
// nothing and adds no row. // it is refused, reopens nothing and adds no row.
func TestEngineStop_WriteAfterStopIsRefused(t *testing.T) { func TestEvictAll_WriteAfterIsRefused(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, "") tgt := env.seedDatabaseTarget(t, 0, "")
deliverTo(t, env, tgt) deliverTo(t, env, tgt)
w := env.eng.ExportArchiveWriterFor(tgt.ID) w := env.writers.ExportArchiveWriterFor(tgt.ID)
require.NotNil(t, w) require.NotNil(t, w)
require.True(t, w.HandleOpen()) require.True(t, w.HandleOpen())
require.NoError(t, env.eng.ExportStop(context.Background())) env.writers.EvictAll()
err := w.Write(evictTestRow("ev-after-stop"), 0) err := w.Write(evictTestRow("ev-after-stop"), 0)
require.ErrorIs( require.ErrorIs(
t, err, delivery.ErrExportArchiveWriterEvicted, t, err, archive.ErrExportArchiveWriterEvicted,
"a write after the stop must be refused", "a write after the stop must be refused",
) )
assert.False( assert.False(
@@ -405,7 +401,7 @@ func TestEngineStop_WriteAfterStopIsRefused(t *testing.T) {
"a refused write must not reopen the archive", "a refused write must not reopen the archive",
) )
assert.False( assert.False(
t, env.eng.ExportHasArchiveWriter(tgt.ID), t, env.writers.ExportHasArchiveWriter(tgt.ID),
"the stop should empty the registry", "the stop should empty the registry",
) )
+280
View File
@@ -0,0 +1,280 @@
package archive
import (
"context"
"log/slog"
"time"
"go.uber.org/fx"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// ErrExportArchiveWriterEvicted exposes the sentinel returned by
// an evicted archive writer. It carries the Err prefix rather
// than this file's usual Export one because it is a sentinel
// error.
var ErrExportArchiveWriterEvicted = errArchiveWriterEvicted
// ExportArchivedEvent aliases the archive row type so black-box
// tests can construct and read archive rows.
type ExportArchivedEvent = archivedEvent
// ExportArchiveWriter wraps an archiveWriter so black-box tests
// can exercise the archive file mechanics.
type ExportArchiveWriter struct {
w *archiveWriter
}
// NewExportArchiveWriter builds an archive writer for tests,
// optionally overriding the reopen debounce (a non-positive
// debounce keeps the production default).
func NewExportArchiveWriter(
path string, log *slog.Logger, debounce time.Duration,
) *ExportArchiveWriter {
w := newArchiveWriter(path, log)
if debounce > 0 {
w.debounce = debounce
}
return &ExportArchiveWriter{w: w}
}
// Write archives a row through the writer, into the file named
// without a period.
func (e *ExportArchiveWriter) Write(
row ExportArchivedEvent, expiry time.Duration,
) error {
return e.w.write(row, expiry, "")
}
// WritePeriod archives a row through the writer, into the file for
// period.
func (e *ExportArchiveWriter) WritePeriod(
row ExportArchivedEvent, expiry time.Duration, period string,
) error {
return e.w.write(row, expiry, period)
}
// Open opens the archive file, pruning when expiry is positive.
func (e *ExportArchiveWriter) Open(expiry time.Duration) error {
return e.w.open(e.w.path, expiry)
}
// Reopen closes and reopens the archive file.
func (e *ExportArchiveWriter) Reopen(
expiry time.Duration,
) error {
return e.w.reopen(e.w.path, expiry)
}
// SetNow replaces the clock the writer measures its reopen
// debounce on.
func (e *ExportArchiveWriter) SetNow(now func() time.Time) {
e.w.now = now
}
// Reopens reports how many times the file has been opened.
func (e *ExportArchiveWriter) Reopens() int {
return e.w.reopens
}
// DB returns the writer's current open handle for row
// inspection in tests.
func (e *ExportArchiveWriter) DB() *gorm.DB {
return e.w.db
}
// Path returns the archive file the writer owns.
func (e *ExportArchiveWriter) Path() string {
return e.w.path
}
// OpenExisting opens the archive without permitting creation,
// the way the idle sweep does.
func (e *ExportArchiveWriter) OpenExisting(
expiry time.Duration,
) error {
return e.w.openMode(e.w.path, archiveModeExisting, expiry)
}
// SweepExpired runs an idle sweep of the archive.
func (e *ExportArchiveWriter) SweepExpired(
expiry time.Duration,
) error {
return e.w.sweepExpired(expiry)
}
// Evict marks the writer evicted and closes its handle, exactly
// as leaving the registry does.
func (e *ExportArchiveWriter) Evict() {
e.w.evict()
}
// Rename gives the archive file a new name in the same directory,
// as a rename of the webhook or target does.
func (e *ExportArchiveWriter) Rename(name string) error {
return e.w.rename(name)
}
// HandleOpen reports whether the writer currently holds an open
// archive handle.
func (e *ExportArchiveWriter) HandleOpen() bool {
e.w.mu.Lock()
defer e.w.mu.Unlock()
return e.w.db != nil
}
// Same reports whether both wrappers refer to the very same
// underlying archive writer, so a test can prove a registry entry
// is the writer it was handed rather than a replacement.
func (e *ExportArchiveWriter) Same(
other *ExportArchiveWriter,
) bool {
return other != nil && e.w == other.w
}
// ExportArchiveWriterFor returns the archive writer the registry
// currently caches for a database target, or nil when none is
// cached. It never creates one, so a test can hold a reference to
// the very writer an eviction is about to detach.
func (ws *Writers) ExportArchiveWriterFor(
targetID string,
) *ExportArchiveWriter {
ws.mu.Lock()
defer ws.mu.Unlock()
w, ok := ws.writers[targetID]
if !ok {
return nil
}
return &ExportArchiveWriter{w: w}
}
// ExportHasArchiveWriter reports whether the registry currently
// caches an archive writer for a target.
func (ws *Writers) ExportHasArchiveWriter(
targetID string,
) bool {
ws.mu.Lock()
defer ws.mu.Unlock()
_, ok := ws.writers[targetID]
return ok
}
// ExportArchiveHandleOpen reports whether the cached archive
// writer for a target holds an open database handle. It
// returns false when no writer is cached.
func (ws *Writers) ExportArchiveHandleOpen(
targetID string,
) bool {
ws.mu.Lock()
w, ok := ws.writers[targetID]
ws.mu.Unlock()
if !ok {
return false
}
w.mu.Lock()
defer w.mu.Unlock()
return w.db != nil
}
// ExportEnsureArchiveWriter creates (if needed) and returns the
// archive file path of the cached writer for a target, so a
// test can prime the registry the way a delivery would.
func (ws *Writers) ExportEnsureArchiveWriter(
targetID string,
) (string, error) {
w, err := ws.writerFor(targetID)
if err != nil {
return "", err
}
return w.path, nil
}
// ExportSweepWriterFor takes a target's registry writer exactly
// as the idle sweep does, reporting whether the sweep had to
// create the entry. It lets a test drive the registry through the
// sweep's own entry point instead of choreographing goroutines.
func (ws *Writers) ExportSweepWriterFor(
targetID string,
) (*ExportArchiveWriter, bool, error) {
w, created, err := ws.sweepWriterFor(targetID)
if err != nil {
return nil, false, err
}
return &ExportArchiveWriter{w: w}, created, nil
}
// ExportReleaseSweepWriter releases a sweep-created registry entry
// exactly as a finished sweep does.
func (ws *Writers) ExportReleaseSweepWriter(
targetID string, w *ExportArchiveWriter,
) {
ws.releaseSweepWriter(targetID, w.w)
}
// NewTestSweeper builds a Sweeper backed by the given main
// database and archive writers, without the fx lifecycle.
// Intended for tests.
func NewTestSweeper(
db *database.Database,
writers *Writers,
log *slog.Logger,
) *Sweeper {
return &Sweeper{
db: db,
writers: writers,
log: log,
interval: time.Hour,
}
}
// ExportSweep runs a single archive sweep synchronously for
// tests.
func (s *Sweeper) ExportSweep(ctx context.Context) {
s.sweep(ctx)
}
// ExportStart starts the sweeper's background loop for tests.
func (s *Sweeper) ExportStart() {
s.start()
}
// ExportRegisterHooks registers the sweeper's real fx lifecycle
// hooks on a lifecycle supplied by a test, so a test can drive
// the exact OnStart/OnStop functions the application runs and
// hand OnStart the kind of context fx actually supplies.
func (s *Sweeper) ExportRegisterHooks(lc fx.Lifecycle) {
s.registerHooks(lc)
}
// ExportStop stops the sweeper's background loop for tests.
func (s *Sweeper) ExportStop(ctx context.Context) error {
return s.stop(ctx)
}
// ExportWedgeLoop adds a goroutine to the sweeper's WaitGroup
// that never observes cancellation and returns only when release
// is closed. It stands in for a prune stuck on a locked archive.
func (s *Sweeper) ExportWedgeLoop(
release <-chan struct{},
) {
s.wg.Go(func() {
<-release
})
}
// ExportSetInterval overrides the sweep interval for tests.
func (s *Sweeper) ExportSetInterval(d time.Duration) {
s.interval = d
}
@@ -1,7 +1,6 @@
package delivery package archive
import ( import (
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"os" "os"
@@ -41,10 +40,10 @@ type archiveFile struct {
period string period string
} }
// ValidateArchiveRotation checks a user-supplied archive rotation for // ValidateRotation checks a user-supplied archive rotation for
// a database target: empty or none (both meaning one file), monthly, // a database target: empty or none (both meaning one file), monthly,
// daily or hourly. // daily or hourly.
func ValidateArchiveRotation(rotation string) error { func ValidateRotation(rotation string) error {
switch rotation { switch rotation {
case "", archiveRotationNone, archiveRotationMonthly, case "", archiveRotationNone, archiveRotationMonthly,
archiveRotationDaily, archiveRotationHourly: archiveRotationDaily, archiveRotationHourly:
@@ -54,35 +53,10 @@ func ValidateArchiveRotation(rotation string) error {
} }
} }
// parseArchiveRotation reads the rotation from a database target's
// config JSON. An empty config or an empty rotation is none.
func parseArchiveRotation(configJSON string) (string, error) {
if configJSON == "" {
return archiveRotationNone, nil
}
var cfg databaseTargetConfig
err := json.Unmarshal([]byte(configJSON), &cfg)
if err != nil {
return "", fmt.Errorf("parsing database target config: %w", err)
}
err = ValidateArchiveRotation(cfg.Rotation)
if err != nil {
return "", err
}
if cfg.Rotation == "" {
return archiveRotationNone, nil
}
return cfg.Rotation, nil
}
// archivePeriod returns the period, in UTC, that a rotation puts an // archivePeriod returns the period, in UTC, that a rotation puts an
// event received at receivedAt in: "2026-10" for monthly, // event received at receivedAt in: "2026-10" for monthly,
// "2026-10-01" for daily, "2026-10-01-19" for hourly, and "" for none. // "2026-10-01" for daily, "2026-10-01-19" for hourly, and "" for none
// or an empty rotation.
func archivePeriod(rotation string, receivedAt time.Time) string { func archivePeriod(rotation string, receivedAt time.Time) string {
switch rotation { switch rotation {
case archiveRotationMonthly: case archiveRotationMonthly:
@@ -97,7 +71,7 @@ func archivePeriod(rotation string, receivedAt time.Time) string {
} }
// archivePeriodPath returns the path of a database target's archive // archivePeriodPath returns the path of a database target's archive
// file for a period: path, as ArchivePath gives it, with "-" and the // file for a period: path, as Path gives it, with "-" and the
// period put before its ".db". The period "" gives path itself. // period put before its ".db". The period "" gives path itself.
func archivePeriodPath(path, period string) string { func archivePeriodPath(path, period string) string {
if period == "" { if period == "" {
@@ -107,28 +81,23 @@ func archivePeriodPath(path, period string) string {
return strings.TrimSuffix(path, ".db") + "-" + period + ".db" return strings.TrimSuffix(path, ".db") + "-" + period + ".db"
} }
// ArchivePathAt returns the archive file a database target writes an // PathAt returns the archive file a database target writes an
// event received at receivedAt to: ArchivePath's file, with the // event received at receivedAt to: Path's file, with the
// period in its name when the target rotates. // period in its name when the target rotates.
func ArchivePathAt( func PathAt(
dbMgr *database.WebhookDBManager, dbMgr *database.WebhookDBManager,
webhook *database.Webhook, webhook *database.Webhook,
target *database.Target, target *database.Target,
receivedAt time.Time, receivedAt time.Time,
) (string, error) { ) string {
rotation, err := parseArchiveRotation(target.Config)
if err != nil {
return "", err
}
return archivePeriodPath( return archivePeriodPath(
ArchivePath(dbMgr, webhook, target), Path(dbMgr, webhook, target),
archivePeriod(rotation, receivedAt), archivePeriod(target.ArchiveRotation, receivedAt),
), nil )
} }
// archiveFiles lists the database target's archive files that exist, // archiveFiles lists the database target's archive files that exist,
// given the path ArchivePath gives it: the file at path, then each // given the path Path gives it: the file at path, then each
// file archivePeriodPath names for path and a period, oldest period // file archivePeriodPath names for path and a period, oldest period
// first. Which rotation wrote a file does not matter, so the files of // first. Which rotation wrote a file does not matter, so the files of
// an earlier rotation setting are listed too. // an earlier rotation setting are listed too.
@@ -1,4 +1,4 @@
package delivery_test package archive_test
import ( import (
"os" "os"
@@ -9,21 +9,17 @@ import (
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/database/databasetest" "sneak.berlin/go/webhooker/internal/database/databasetest"
"sneak.berlin/go/webhooker/internal/delivery"
) )
// The archive rotations, and the configs of a daily target and of one // The archive rotations.
// whose rotation is not one of the four.
const ( const (
rotationNone = "none" rotationNone = "none"
rotationMonthly = "monthly" rotationMonthly = "monthly"
rotationDaily = "daily" rotationDaily = "daily"
rotationHourly = "hourly" rotationHourly = "hourly"
dailyConfig = `{"rotation":"daily"}`
weeklyConfig = `{"rotation":"weekly"}`
) )
// The periods the tests archive into most: two days, and an hour of // The periods the tests archive into most: two days, and an hour of
@@ -40,32 +36,30 @@ func periodPath(path, period string) string {
return strings.TrimSuffix(path, ".db") + "-" + period + ".db" return strings.TrimSuffix(path, ".db") + "-" + period + ".db"
} }
// deliverReceivedAt delivers to a database target an event whose // deliverReceivedAt archives, as a delivery to a database target
// receive time is receivedAt, and returns the event's id. The receive // does, an event whose receive time is receivedAt, and returns the
// time is what decides a rotated archive's file, so setting it is how // event's id. The receive time is what decides a rotated archive's
// these tests move the clock across a period boundary. // file, so setting it is how these tests move the clock across a
// period boundary.
func (env *archiveEnv) deliverReceivedAt( func (env *archiveEnv) deliverReceivedAt(
t *testing.T, tgt *database.Target, receivedAt time.Time, t *testing.T, tgt *database.Target, receivedAt time.Time,
) string { ) string {
t.Helper() t.Helper()
webhookDB := testWebhookDB(t) event := newEvent(`{"n":1}`)
event := seedEvent(t, webhookDB, `{"n":1}`)
event.CreatedAt = receivedAt event.CreatedAt = receivedAt
env.eng.ExportDeliverDatabase( require.NoError(t, env.writers.Write(databaseTargetDelivery(event, tgt)))
webhookDB, seedDatabaseTargetDelivery(t, webhookDB, event, tgt),
)
return event.ID return event.ID
} }
// TestDeliverDatabase_RotatesAtEachPeriodBoundary delivers, for each // TestWrite_RotatesAtEachPeriodBoundary writes, for each
// rotation, an event received in the last second of a period and one // rotation, an event received in the last second of a period and one
// received in the first second of the next, and checks each lands in // received in the first second of the next, and checks each lands in
// the file named for its own period, in UTC. Rotation none keeps both // the file named for its own period, in UTC. Rotation none keeps both
// in the one file. // in the one file.
func TestDeliverDatabase_RotatesAtEachPeriodBoundary(t *testing.T) { func TestWrite_RotatesAtEachPeriodBoundary(t *testing.T) {
t.Parallel() t.Parallel()
berlin := time.FixedZone("CEST", 2*60*60) berlin := time.FixedZone("CEST", 2*60*60)
@@ -110,7 +104,7 @@ func TestDeliverDatabase_RotatesAtEachPeriodBoundary(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"rotation":"`+tc.rotation+`"}`) tgt := env.seedDatabaseTarget(t, 0, tc.rotation)
path := env.archivePath(tgt) path := env.archivePath(tgt)
first := env.deliverReceivedAt(t, tgt, tc.before) first := env.deliverReceivedAt(t, tgt, tc.before)
@@ -129,7 +123,7 @@ func TestDeliverDatabase_RotatesAtEachPeriodBoundary(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"rotation":"`+rotationNone+`"}`) tgt := env.seedDatabaseTarget(t, 0, rotationNone)
first := env.deliverReceivedAt(t, tgt, cases[0].before) first := env.deliverReceivedAt(t, tgt, cases[0].before)
second := env.deliverReceivedAt(t, tgt, cases[0].after) second := env.deliverReceivedAt(t, tgt, cases[0].after)
@@ -139,14 +133,14 @@ func TestDeliverDatabase_RotatesAtEachPeriodBoundary(t *testing.T) {
}) })
} }
// TestDeliverDatabase_EventWithoutReceiveTime proves an event whose // TestWrite_EventWithoutReceiveTime proves an event whose
// receive time is not known goes to the file for the time it is // receive time is not known goes to the file for the time it is
// archived, rather than to one for the year 1. // archived, rather than to one for the year 1.
func TestDeliverDatabase_EventWithoutReceiveTime(t *testing.T) { func TestWrite_EventWithoutReceiveTime(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"rotation":"`+rotationMonthly+`"}`) tgt := env.seedDatabaseTarget(t, 0, rotationMonthly)
path := env.archivePath(tgt) path := env.archivePath(tgt)
before := time.Now().UTC().Format("2006-01") before := time.Now().UTC().Format("2006-01")
@@ -164,20 +158,20 @@ func TestDeliverDatabase_EventWithoutReceiveTime(t *testing.T) {
assert.NoFileExists(t, periodPath(path, "0001-01")) assert.NoFileExists(t, periodPath(path, "0001-01"))
} }
// TestDeliverDatabase_RotationChangeKeepsOldFiles changes a target's // TestWrite_RotationChangeKeepsOldFiles changes a target's
// rotation from none to daily between two events, and checks the // rotation from none to daily between two events, and checks the
// second goes to the daily file while the first stays where it was. // second goes to the daily file while the first stays where it was.
func TestDeliverDatabase_RotationChangeKeepsOldFiles(t *testing.T) { func TestWrite_RotationChangeKeepsOldFiles(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, "") tgt := env.seedDatabaseTarget(t, 0, rotationNone)
path := env.archivePath(tgt) path := env.archivePath(tgt)
at := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC) at := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC)
first := env.deliverReceivedAt(t, tgt, at) first := env.deliverReceivedAt(t, tgt, at)
tgt.Config = dailyConfig tgt.ArchiveRotation = rotationDaily
second := env.deliverReceivedAt(t, tgt, at) second := env.deliverReceivedAt(t, tgt, at)
assert.Equal(t, []string{first}, archivedEventIDs(t, path)) assert.Equal(t, []string{first}, archivedEventIDs(t, path))
@@ -195,9 +189,7 @@ func TestArchiveSweep_PrunesEveryFile(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget( tgt := env.seedDatabaseTarget(t, time.Hour, rotationDaily)
t, `{"expiry":"1h","rotation":"`+rotationDaily+`"}`,
)
path := env.archivePath(tgt) path := env.archivePath(tgt)
emptied := periodPath(path, dayPeriod) emptied := periodPath(path, dayPeriod)
kept := periodPath(path, nextDayPeriod) kept := periodPath(path, nextDayPeriod)
@@ -227,17 +219,17 @@ func TestRename_MovesEveryFile(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, "") tgt := env.seedDatabaseTarget(t, 0, rotationNone)
oldPath := env.archivePath(tgt) oldPath := env.archivePath(tgt)
day := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC) day := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC)
unrotated := env.deliverReceivedAt(t, tgt, day) unrotated := env.deliverReceivedAt(t, tgt, day)
tgt.Config = dailyConfig tgt.ArchiveRotation = rotationDaily
first := env.deliverReceivedAt(t, tgt, day) first := env.deliverReceivedAt(t, tgt, day)
second := env.deliverReceivedAt(t, tgt, day.Add(24*time.Hour)) second := env.deliverReceivedAt(t, tgt, day.Add(24*time.Hour))
require.NoError(t, env.eng.Rename(tgt.ID, "Orders", "Long Term")) require.NoError(t, env.writers.Rename(tgt.ID, "Orders", "Long Term"))
newPath := filepath.Join( newPath := filepath.Join(
env.dataDir, "archive-orders-long-term-"+tgt.ID+".db", env.dataDir, "archive-orders-long-term-"+tgt.ID+".db",
@@ -269,7 +261,7 @@ func TestRename_NeverReplacesARotatedFile(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, dailyConfig) tgt := env.seedDatabaseTarget(t, 0, rotationDaily)
oldPath := env.archivePath(tgt) oldPath := env.archivePath(tgt)
day := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC) day := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC)
@@ -283,8 +275,8 @@ func TestRename_NeverReplacesARotatedFile(t *testing.T) {
require.NoError(t, os.WriteFile(planted, []byte("planted"), 0o600)) require.NoError(t, os.WriteFile(planted, []byte("planted"), 0o600))
require.ErrorIs( require.ErrorIs(
t, env.eng.Rename(tgt.ID, "Orders", "Long Term"), t, env.writers.Rename(tgt.ID, "Orders", "Long Term"),
delivery.ErrArchiveNameTaken, archive.ErrNameTaken,
) )
assert.FileExists(t, periodPath(oldPath, dayPeriod)) assert.FileExists(t, periodPath(oldPath, dayPeriod))
@@ -292,7 +284,7 @@ func TestRename_NeverReplacesARotatedFile(t *testing.T) {
assert.NoFileExists(t, periodPath(newPath, dayPeriod)) assert.NoFileExists(t, periodPath(newPath, dayPeriod))
} }
// TestStatArchive_EveryFile proves StatArchive counts and adds up every // TestStatArchive_EveryFile proves Stat counts and adds up every
// one of a target's files, takes the latest write of any of them, and // one of a target's files, takes the latest write of any of them, and
// leaves out files whose names only look like the target's. // leaves out files whose names only look like the target's.
func TestStatArchive_EveryFile(t *testing.T) { func TestStatArchive_EveryFile(t *testing.T) {
@@ -324,7 +316,7 @@ func TestStatArchive_EveryFile(t *testing.T) {
os.WriteFile(filepath.Join(dir, other), []byte("x"), 0o600)) os.WriteFile(filepath.Join(dir, other), []byte("x"), 0o600))
} }
got, err := delivery.StatArchive(path) got, err := archive.Stat(path)
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, len(files), got.Files) assert.Equal(t, len(files), got.Files)
assert.Equal(t, size, got.Size) assert.Equal(t, size, got.Size)
@@ -343,34 +335,27 @@ func TestArchivePathAt(t *testing.T) {
at := time.Date(2026, 10, 1, 19, 30, 0, 0, time.UTC) at := time.Date(2026, 10, 1, 19, 30, 0, 0, time.UTC)
cases := map[string]string{ cases := map[string]string{
"": "", "": "",
`{"rotation":"` + rotationNone + `"}`: "", rotationNone: "",
`{"rotation":"` + rotationMonthly + `"}`: "-2026-10", rotationMonthly: "-2026-10",
dailyConfig: "-2026-10-01", rotationDaily: "-2026-10-01",
`{"rotation":"` + rotationHourly + `"}`: "-2026-10-01-19", rotationHourly: "-2026-10-01-19",
} }
for config, period := range cases { for rotation, period := range cases {
target := &database.Target{ target := &database.Target{
BaseModel: database.BaseModel{ID: "tgt-id"}, BaseModel: database.BaseModel{ID: "tgt-id"},
Name: "Archive", Name: "Archive",
Config: config, ArchiveRotation: rotation,
} }
got, err := delivery.ArchivePathAt(dbMgr, webhook, target, at)
require.NoError(t, err, config)
assert.Equal(t, assert.Equal(t,
filepath.Join( filepath.Join(
dataDir, "archive-orders-archive-tgt-id"+period+".db", dataDir, "archive-orders-archive-tgt-id"+period+".db",
), ),
got, config, archive.PathAt(dbMgr, webhook, target, at), rotation,
) )
} }
_, err := delivery.ArchivePathAt(dbMgr, webhook, &database.Target{
Config: weeklyConfig,
}, at)
require.Error(t, err)
} }
// TestValidateArchiveRotation accepts the four rotations, and empty, // TestValidateArchiveRotation accepts the four rotations, and empty,
@@ -381,10 +366,10 @@ func TestValidateArchiveRotation(t *testing.T) {
for _, ok := range []string{ for _, ok := range []string{
"", rotationNone, rotationMonthly, rotationDaily, rotationHourly, "", rotationNone, rotationMonthly, rotationDaily, rotationHourly,
} { } {
require.NoError(t, delivery.ValidateArchiveRotation(ok), ok) require.NoError(t, archive.ValidateRotation(ok), ok)
} }
for _, bad := range []string{"weekly", "Daily", "hourly "} { for _, bad := range []string{"weekly", "Daily", "hourly "} {
require.Error(t, delivery.ValidateArchiveRotation(bad), bad) require.Error(t, archive.ValidateRotation(bad), bad)
} }
} }
@@ -1,4 +1,4 @@
package delivery package archive
import ( import (
"context" "context"
@@ -15,18 +15,18 @@ import (
"sneak.berlin/go/webhooker/internal/logger" "sneak.berlin/go/webhooker/internal/logger"
) )
// ArchiveSweeperParams holds the fx dependencies for the // SweeperParams holds the fx dependencies for the
// ArchiveSweeper. // Sweeper.
type ArchiveSweeperParams struct { type SweeperParams struct {
fx.In fx.In
Config *config.Config Config *config.Config
Database *database.Database Database *database.Database
Engine *Engine Writers *Writers
Logger *logger.Logger Logger *logger.Logger
} }
// ArchiveSweeper periodically prunes expired rows from the // Sweeper periodically prunes expired rows from the
// archive databases of database targets that carry a positive // archive databases of database targets that carry a positive
// expiry. // expiry.
// //
@@ -40,9 +40,9 @@ type ArchiveSweeperParams struct {
// It reuses Config.RetentionSweepInterval rather than // It reuses Config.RetentionSweepInterval rather than
// introducing a second interval: this is a retention sweep with // introducing a second interval: this is a retention sweep with
// the same semantics as the event retention reaper. // the same semantics as the event retention reaper.
type ArchiveSweeper struct { type Sweeper struct {
db *database.Database db *database.Database
eng *Engine writers *Writers
log *slog.Logger log *slog.Logger
interval time.Duration interval time.Duration
@@ -54,16 +54,16 @@ type ArchiveSweeper struct {
wg sync.WaitGroup wg sync.WaitGroup
} }
// NewArchiveSweeper creates the archive sweeper and registers // NewSweeper creates the archive sweeper and registers
// its fx lifecycle hooks. The background sweep loop starts on // its fx lifecycle hooks. The background sweep loop starts on
// OnStart and stops cleanly on OnStop via context cancellation. // OnStart and stops cleanly on OnStop via context cancellation.
func NewArchiveSweeper( func NewSweeper(
lc fx.Lifecycle, lc fx.Lifecycle,
params ArchiveSweeperParams, params SweeperParams,
) *ArchiveSweeper { ) *Sweeper {
s := &ArchiveSweeper{ s := &Sweeper{
db: params.Database, db: params.Database,
eng: params.Engine, writers: params.Writers,
log: params.Logger.Get(), log: params.Logger.Get(),
interval: params.Config.RetentionSweepInterval, interval: params.Config.RetentionSweepInterval,
} }
@@ -77,7 +77,7 @@ func NewArchiveSweeper(
// lifecycle. The start hook's context is deliberately ignored // lifecycle. The start hook's context is deliberately ignored
// (see start for why the background loop must not inherit it); // (see start for why the background loop must not inherit it);
// the stop hook's context is honoured (see stop). // the stop hook's context is honoured (see stop).
func (s *ArchiveSweeper) registerHooks(lc fx.Lifecycle) { func (s *Sweeper) registerHooks(lc fx.Lifecycle) {
lc.Append(fx.Hook{ lc.Append(fx.Hook{
//nolint:contextcheck // Not passing the hook context is //nolint:contextcheck // Not passing the hook context is
// the point: see start. // the point: see start.
@@ -103,7 +103,7 @@ func (s *ArchiveSweeper) registerHooks(lc fx.Lifecycle) {
// goroutine must outlive the startup phase, so its lifetime is // goroutine must outlive the startup phase, so its lifetime is
// bounded by OnStop instead: stop cancels this context and waits // bounded by OnStop instead: stop cancels this context and waits
// on the WaitGroup. // on the WaitGroup.
func (s *ArchiveSweeper) start() { func (s *Sweeper) start() {
ctx, cancel := context.WithCancel(context.Background()) ctx, cancel := context.WithCancel(context.Background())
s.cancel = cancel s.cancel = cancel
@@ -121,7 +121,7 @@ func (s *ArchiveSweeper) start() {
// exit, bounded by the stop hook's context: a prune wedged on a // exit, bounded by the stop hook's context: a prune wedged on a
// locked archive must not hang the process past fx's stop // locked archive must not hang the process past fx's stop
// timeout. // timeout.
func (s *ArchiveSweeper) stop(ctx context.Context) error { func (s *Sweeper) stop(ctx context.Context) error {
s.log.Info("archive sweeper stopping") s.log.Info("archive sweeper stopping")
if s.cancel != nil { if s.cancel != nil {
@@ -140,7 +140,7 @@ func (s *ArchiveSweeper) stop(ctx context.Context) error {
return nil return nil
} }
func (s *ArchiveSweeper) run(ctx context.Context) { func (s *Sweeper) run(ctx context.Context) {
defer s.wg.Done() defer s.wg.Done()
ticker := time.NewTicker(s.interval) ticker := time.NewTicker(s.interval)
@@ -164,7 +164,7 @@ func (s *ArchiveSweeper) run(ctx context.Context) {
// A failure for one target is logged and the sweep continues, // A failure for one target is logged and the sweep continues,
// matching how the write path already treats a prune error as // matching how the write path already treats a prune error as
// non-fatal. // non-fatal.
func (s *ArchiveSweeper) sweep(ctx context.Context) { func (s *Sweeper) sweep(ctx context.Context) {
var targets []database.Target var targets []database.Target
err := s.db.DB(). err := s.db.DB().
@@ -199,32 +199,20 @@ func (s *ArchiveSweeper) sweep(ctx context.Context) {
} }
} }
// sweepTarget prunes the archive of a single database target. // sweepTarget prunes the archive of a single database target. A
// A missing, empty, or "never" expiry parses as a zero duration // target with an ArchiveExpiry of 0 keeps its archive forever and is
// and is skipped entirely, so those archives keep exactly the // skipped entirely, so that archive is never pruned.
// behaviour they had before the sweep existed. func (s *Sweeper) sweepTarget(target *database.Target) {
func (s *ArchiveSweeper) sweepTarget(target *database.Target) { expiry := target.ArchiveExpiry
expiry, err := parseArchiveExpiry(target.Config)
if err != nil {
s.log.Error(
"archive sweep: invalid database target config",
"webhook_id", target.WebhookID,
"target_id", target.ID,
"error", err,
)
return
}
if expiry <= 0 { if expiry <= 0 {
return return
} }
if s.eng == nil || s.eng.dbTarget == nil { if s.writers == nil {
return return
} }
err = s.eng.dbTarget.sweepArchive(target.ID, expiry) err := s.writers.sweepArchive(target.ID, expiry)
if err == nil { if err == nil {
return return
} }
@@ -1,4 +1,4 @@
package delivery_test package archive_test
import ( import (
"bytes" "bytes"
@@ -16,14 +16,15 @@ import (
"github.com/google/uuid" "github.com/google/uuid"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"go.uber.org/fx"
"gorm.io/driver/sqlite" "gorm.io/driver/sqlite"
"gorm.io/gorm" "gorm.io/gorm"
"gorm.io/gorm/clause" "gorm.io/gorm/clause"
_ "modernc.org/sqlite" // Pure Go SQLite driver. _ "modernc.org/sqlite" // Pure Go SQLite driver.
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/database/databasetest" "sneak.berlin/go/webhooker/internal/database/databasetest"
"sneak.berlin/go/webhooker/internal/db/migrations" "sneak.berlin/go/webhooker/internal/db/migrations"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/gormlog" "sneak.berlin/go/webhooker/internal/gormlog"
) )
@@ -37,6 +38,18 @@ const (
// sweepConcurrentWrites is how many deliveries the // sweepConcurrentWrites is how many deliveries the
// concurrent write-plus-sweep test races against the sweep. // concurrent write-plus-sweep test races against the sweep.
sweepConcurrentWrites = 20 sweepConcurrentWrites = 20
// hookStopTimeout bounds how long a lifecycle test waits for
// the sweeper's OnStop hook to return before declaring the
// shutdown hung.
hookStopTimeout = 10 * time.Second
// wedgeStopTimeout is the stop timeout a wedged-shutdown test
// hands OnStop, standing in for fx's StopTimeout. The test
// asserts only that the hook returns at all, and allows it
// hookStopTimeout — forty times this budget — to do so, so no
// assertion here races the wall clock.
wedgeStopTimeout = 250 * time.Millisecond
) )
// archiveTestWebhookName is the name of every webhook // archiveTestWebhookName is the name of every webhook
@@ -45,12 +58,12 @@ const (
const archiveTestWebhookName = "Sweep Test!" const archiveTestWebhookName = "Sweep Test!"
// archiveEnv bundles the pieces an archive test drives: a main // archiveEnv bundles the pieces an archive test drives: a main
// configuration database holding webhooks and targets, a delivery // configuration database holding webhooks and targets, the archive
// engine owning the archive writer registry, the archive sweeper, // writers, the archive sweeper, and the data directory the archive
// and the data directory the archive files live in. // files live in.
type archiveEnv struct { type archiveEnv struct {
sweeper *delivery.ArchiveSweeper sweeper *archive.Sweeper
eng *delivery.Engine writers *archive.Writers
mainDB *database.Database mainDB *database.Database
dataDir string dataDir string
} }
@@ -66,39 +79,37 @@ func setupArchiveTest(t *testing.T) *archiveEnv {
t.Cleanup(func() { _ = mainDB.Close() }) t.Cleanup(func() { _ = mainDB.Close() })
eng := delivery.NewTestEngineWithDB( writers := archive.NewWriters(archive.WritersParams{
mainDB, Database: mainDB,
databasetest.NewWebhookDBManager(t, dataDir), DBManager: databasetest.NewWebhookDBManager(t, dataDir),
log, Logger: log,
&http.Client{Timeout: 5 * time.Second}, })
1,
)
return &archiveEnv{ return &archiveEnv{
sweeper: delivery.NewTestArchiveSweeper( sweeper: archive.NewTestSweeper(
mainDB, eng, log, mainDB, writers, log,
), ),
eng: eng, writers: writers,
mainDB: mainDB, mainDB: mainDB,
dataDir: dataDir, dataDir: dataDir,
} }
} }
// archivePath returns where the engine keeps a database target's // archivePath returns where the writers keep a database target's
// archive file, for the names seedDatabaseTarget gave it. // archive file, for the names seedDatabaseTarget gave it.
func (env *archiveEnv) archivePath(tgt *database.Target) string { func (env *archiveEnv) archivePath(tgt *database.Target) string {
return filepath.Join( return filepath.Join(
env.dataDir, env.dataDir,
delivery.ArchiveFileName( archive.FileName(
archiveTestWebhookName, tgt.Name, tgt.ID, archiveTestWebhookName, tgt.Name, tgt.ID,
), ),
) )
} }
// seedDatabaseTarget creates a webhook with one database target // seedDatabaseTarget creates a webhook with one database target with
// carrying the given target config JSON, and returns the target. // the given archive expiry and rotation, and returns the target.
func (env *archiveEnv) seedDatabaseTarget( func (env *archiveEnv) seedDatabaseTarget(
t *testing.T, configJSON string, t *testing.T, expiry time.Duration, rotation string,
) *database.Target { ) *database.Target {
t.Helper() t.Helper()
@@ -113,22 +124,27 @@ func (env *archiveEnv) seedDatabaseTarget(
Create(wh).Error, Create(wh).Error,
) )
return env.addDatabaseTarget(t, wh.ID, configJSON) return env.addDatabaseTarget(t, wh.ID, expiry, rotation)
} }
// addDatabaseTarget creates one more database target on an // addDatabaseTarget creates one more database target on an
// existing webhook and returns it. // existing webhook and returns it.
func (env *archiveEnv) addDatabaseTarget( func (env *archiveEnv) addDatabaseTarget(
t *testing.T, webhookID, configJSON string, t *testing.T,
webhookID string,
expiry time.Duration,
rotation string,
) *database.Target { ) *database.Target {
t.Helper() t.Helper()
tgt := &database.Target{ tgt := &database.Target{
WebhookID: webhookID, WebhookID: webhookID,
Name: "Archive", Name: "Archive",
Type: database.TargetTypeDatabase, Type: database.TargetTypeDatabase,
Active: true, Active: true,
Config: configJSON, ArchiveExpiry: expiry,
ArchiveRotation: rotation,
MaxAttempts: 1,
} }
require.NoError( require.NoError(
t, t,
@@ -176,7 +192,7 @@ func seedArchiveFile(
require.NoError(t, migrations.Apply(gdb, migrations.Archive)) require.NoError(t, migrations.Apply(gdb, migrations.Archive))
for i, at := range archivedAt { for i, at := range archivedAt {
row := delivery.ExportArchivedEvent{ row := archive.ExportArchivedEvent{
EventID: fmt.Sprintf("ev-%d", i), EventID: fmt.Sprintf("ev-%d", i),
WebhookID: webhookID, WebhookID: webhookID,
Method: http.MethodPost, Method: http.MethodPost,
@@ -196,7 +212,7 @@ func archivedEventIDs(
) []string { ) []string {
t.Helper() t.Helper()
var rows []delivery.ExportArchivedEvent var rows []archive.ExportArchivedEvent
rdb := openArchiveDBForRead(t, path) rdb := openArchiveDBForRead(t, path)
require.NoError(t, rdb.Order("event_id").Find(&rows).Error) require.NoError(t, rdb.Order("event_id").Find(&rows).Error)
@@ -233,7 +249,7 @@ func countArchivedRows(path string) (int64, error) {
var count int64 var count int64
err = gdb.Model(&delivery.ExportArchivedEvent{}). err = gdb.Model(&archive.ExportArchivedEvent{}).
Count(&count).Error Count(&count).Error
if err != nil { if err != nil {
return 0, err return 0, err
@@ -242,6 +258,55 @@ func countArchivedRows(path string) (int64, error) {
return count, nil return count, nil
} }
// recordingLifecycle is a minimal fx.Lifecycle that records the
// hooks a component registers, so a test can invoke the real
// OnStart/OnStop functions with a context of its choosing.
type recordingLifecycle struct {
hooks []fx.Hook
}
func (l *recordingLifecycle) Append(h fx.Hook) {
l.hooks = append(l.hooks, h)
}
// requireStopHookExpires drives hook.OnStop with a stop context
// that expires while a wedged goroutine is still running, and
// requires the hook to return the deadline error naming
// component instead of blocking on the WaitGroup forever.
func requireStopHookExpires(
t *testing.T, hook fx.Hook, component string,
) {
t.Helper()
stopCtx, cancel := context.WithTimeout(
context.Background(), wedgeStopTimeout,
)
defer cancel()
var stopErr error
stopped := make(chan struct{})
go func() {
defer close(stopped)
stopErr = hook.OnStop(stopCtx)
}()
select {
case <-stopped:
case <-time.After(hookStopTimeout):
t.Fatal(
"OnStop did not return: it discarded the stop " +
"context and is waiting on a wedged goroutine " +
"that will never observe cancellation",
)
}
require.ErrorIs(t, stopErr, context.DeadlineExceeded)
require.ErrorContains(t, stopErr, component)
}
// TestArchiveSweeper_LoopOutlivesStartHookContext is the // TestArchiveSweeper_LoopOutlivesStartHookContext is the
// regression test for a sweeper that never swept. fx calls // regression test for a sweeper that never swept. fx calls
// OnStart with a context carrying the application's start // OnStart with a context carrying the application's start
@@ -262,7 +327,7 @@ func TestArchiveSweeper_LoopOutlivesStartHookContext(
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`) tgt := env.seedDatabaseTarget(t, time.Hour, "")
now := time.Now() now := time.Now()
path := env.seedArchiveRows( path := env.seedArchiveRows(
@@ -316,7 +381,7 @@ func TestArchiveSweep_DoesNotResurrectEvictedWriter(
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`) tgt := env.seedDatabaseTarget(t, time.Hour, "")
env.seedArchiveRows( env.seedArchiveRows(
t, tgt, time.Now().Add(-48*time.Hour), t, tgt, time.Now().Add(-48*time.Hour),
) )
@@ -325,16 +390,16 @@ func TestArchiveSweep_DoesNotResurrectEvictedWriter(
// the deletion path does. The target row is deliberately left // the deletion path does. The target row is deliberately left
// in place: this is the tick that listed the target before // in place: this is the tick that listed the target before
// the deletion committed. // the deletion committed.
_, err := env.eng.ExportEnsureArchiveWriter(tgt.ID) _, err := env.writers.ExportEnsureArchiveWriter(tgt.ID)
require.NoError(t, err) require.NoError(t, err)
env.eng.EvictTarget(tgt.ID) env.writers.EvictTarget(tgt.ID)
require.False(t, env.eng.ExportHasArchiveWriter(tgt.ID)) require.False(t, env.writers.ExportHasArchiveWriter(tgt.ID))
env.sweeper.ExportSweep(context.Background()) env.sweeper.ExportSweep(context.Background())
assert.False( assert.False(
t, env.eng.ExportHasArchiveWriter(tgt.ID), t, env.writers.ExportHasArchiveWriter(tgt.ID),
"a sweep must never re-register a writer for a target "+ "a sweep must never re-register a writer for a target "+
"whose registry entry has already been released", "whose registry entry has already been released",
) )
@@ -350,14 +415,14 @@ func TestArchiveSweep_LeavesNoRegistryEntry(t *testing.T) {
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`) tgt := env.seedDatabaseTarget(t, time.Hour, "")
path := env.seedArchiveRows( path := env.seedArchiveRows(
t, tgt, t, tgt,
time.Now().Add(-48*time.Hour), time.Now().Add(-48*time.Hour),
time.Now().Add(-time.Minute), time.Now().Add(-time.Minute),
) )
require.False(t, env.eng.ExportHasArchiveWriter(tgt.ID)) require.False(t, env.writers.ExportHasArchiveWriter(tgt.ID))
env.sweeper.ExportSweep(context.Background()) env.sweeper.ExportSweep(context.Background())
@@ -366,7 +431,7 @@ func TestArchiveSweep_LeavesNoRegistryEntry(t *testing.T) {
"the sweep must still prune an idle archive", "the sweep must still prune an idle archive",
) )
assert.False( assert.False(
t, env.eng.ExportHasArchiveWriter(tgt.ID), t, env.writers.ExportHasArchiveWriter(tgt.ID),
"the sweep must release the registry entry it created", "the sweep must release the registry entry it created",
) )
} }
@@ -383,29 +448,27 @@ func TestArchiveSweep_KeepsWriterAdoptedByDelivery(
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`) tgt := env.seedDatabaseTarget(t, time.Hour, "")
env.seedArchiveRows( env.seedArchiveRows(
t, tgt, time.Now().Add(-48*time.Hour), t, tgt, time.Now().Add(-48*time.Hour),
) )
webhookDB := testWebhookDB(t) d := databaseTargetDelivery(newEvent(`{"n":1}`), tgt)
event := seedEvent(t, webhookDB, `{"n":1}`)
d := seedDatabaseTargetDelivery(t, webhookDB, event, tgt)
env.sweeper.ExportSweep(context.Background()) env.sweeper.ExportSweep(context.Background())
require.False(t, env.eng.ExportHasArchiveWriter(tgt.ID)) require.False(t, env.writers.ExportHasArchiveWriter(tgt.ID))
env.eng.ExportDeliverDatabase(webhookDB, d) require.NoError(t, env.writers.Write(d))
assert.True( assert.True(
t, env.eng.ExportHasArchiveWriter(tgt.ID), t, env.writers.ExportHasArchiveWriter(tgt.ID),
"a delivery's writer must stay registered", "a delivery's writer must stay registered",
) )
env.sweeper.ExportSweep(context.Background()) env.sweeper.ExportSweep(context.Background())
assert.True( assert.True(
t, env.eng.ExportHasArchiveWriter(tgt.ID), t, env.writers.ExportHasArchiveWriter(tgt.ID),
"a sweep must not drop a writer a delivery owns", "a sweep must not drop a writer a delivery owns",
) )
} }
@@ -439,12 +502,12 @@ func TestArchiveSweep_KeepsWriterAdoptedDuringSweep(
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`) tgt := env.seedDatabaseTarget(t, time.Hour, "")
env.seedArchiveRows( env.seedArchiveRows(
t, tgt, time.Now().Add(-48*time.Hour), t, tgt, time.Now().Add(-48*time.Hour),
) )
sweepWriter, created, err := env.eng.ExportSweepWriterFor( sweepWriter, created, err := env.writers.ExportSweepWriterFor(
tgt.ID, tgt.ID,
) )
require.NoError(t, err) require.NoError(t, err)
@@ -454,36 +517,34 @@ func TestArchiveSweep_KeepsWriterAdoptedDuringSweep(
) )
// The delivery lands mid-sweep and adopts the entry. // The delivery lands mid-sweep and adopts the entry.
webhookDB := testWebhookDB(t) d := databaseTargetDelivery(newEvent(`{"n":1}`), tgt)
event := seedEvent(t, webhookDB, `{"n":1}`)
d := seedDatabaseTargetDelivery(t, webhookDB, event, tgt)
env.eng.ExportDeliverDatabase(webhookDB, d) require.NoError(t, env.writers.Write(d))
adopted := env.eng.ExportArchiveWriterFor(tgt.ID) adopted := env.writers.ExportArchiveWriterFor(tgt.ID)
require.NotNil(t, adopted) require.NotNil(t, adopted)
require.True( require.True(
t, sweepWriter.Same(adopted), t, sweepWriter.Same(adopted),
"the delivery must have adopted the sweep's writer", "the delivery must have adopted the sweep's writer",
) )
require.True( require.True(
t, env.eng.ExportArchiveHandleOpen(tgt.ID), t, env.writers.ExportArchiveHandleOpen(tgt.ID),
"the delivery leaves the archive handle open", "the delivery leaves the archive handle open",
) )
// The sweep finishes. // The sweep finishes.
env.eng.ExportReleaseSweepWriter(tgt.ID, sweepWriter) env.writers.ExportReleaseSweepWriter(tgt.ID, sweepWriter)
require.True( require.True(
t, env.eng.ExportHasArchiveWriter(tgt.ID), t, env.writers.ExportHasArchiveWriter(tgt.ID),
"a writer adopted by a delivery during a sweep must "+ "a writer adopted by a delivery during a sweep must "+
"stay registered, or its open handle is unreachable", "stay registered, or its open handle is unreachable",
) )
env.eng.EvictTarget(tgt.ID) env.writers.EvictTarget(tgt.ID)
assert.False( assert.False(
t, env.eng.ExportHasArchiveWriter(tgt.ID), t, env.writers.ExportHasArchiveWriter(tgt.ID),
"the adopted writer must still be evictable", "the adopted writer must still be evictable",
) )
assert.False( assert.False(
@@ -494,8 +555,8 @@ func TestArchiveSweep_KeepsWriterAdoptedDuringSweep(
// TestArchiveSweep_ContinuesAfterPerTargetFailure proves a // TestArchiveSweep_ContinuesAfterPerTargetFailure proves a
// failure for one target does not abort the sweep for the // failure for one target does not abort the sweep for the
// others: an unparseable expiry and an unreadable archive both // others: an unreadable archive has to be logged and stepped
// have to be logged and stepped over. // over.
func TestArchiveSweep_ContinuesAfterPerTargetFailure( func TestArchiveSweep_ContinuesAfterPerTargetFailure(
t *testing.T, t *testing.T,
) { ) {
@@ -503,21 +564,16 @@ func TestArchiveSweep_ContinuesAfterPerTargetFailure(
env := setupArchiveTest(t) env := setupArchiveTest(t)
// Seeded first so the sweep reaches them before the healthy // Seeded first so the sweep reaches it before the healthy
// target: targets come back in insertion order. // target: targets come back in insertion order.
badConfig := env.seedDatabaseTarget(t, `{"expiry":"!!!"}`) corrupt := env.seedDatabaseTarget(t, time.Hour, "")
env.seedArchiveRows(
t, badConfig, time.Now().Add(-48*time.Hour),
)
corrupt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
require.NoError(t, os.WriteFile( require.NoError(t, os.WriteFile(
env.archivePath(corrupt), env.archivePath(corrupt),
[]byte("this is not a sqlite database"), []byte("this is not a sqlite database"),
0o600, 0o600,
)) ))
healthy := env.seedDatabaseTarget(t, `{"expiry":"1h"}`) healthy := env.seedDatabaseTarget(t, time.Hour, "")
healthyPath := env.seedArchiveRows( healthyPath := env.seedArchiveRows(
t, healthy, t, healthy,
time.Now().Add(-48*time.Hour), time.Now().Add(-48*time.Hour),
@@ -547,7 +603,7 @@ func TestArchiveSweep_OpenExistingDoesNotCreateFile(
dir := t.TempDir() dir := t.TempDir()
path := filepath.Join(dir, "archive-absent.db") path := filepath.Join(dir, "archive-absent.db")
w := delivery.NewExportArchiveWriter( w := archive.NewExportArchiveWriter(
path, archiveTestLogger(), 0, path, archiveTestLogger(), 0,
) )
@@ -574,7 +630,7 @@ func TestArchiveSweep_PrunesIdleArchive(t *testing.T) {
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`) tgt := env.seedDatabaseTarget(t, time.Hour, "")
now := time.Now() now := time.Now()
path := env.seedArchiveRows( path := env.seedArchiveRows(
@@ -613,12 +669,12 @@ func TestArchiveSweep_LeavesArchiveClosed(t *testing.T) {
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`) tgt := env.seedDatabaseTarget(t, time.Hour, "")
path := env.seedArchiveRows( path := env.seedArchiveRows(
t, tgt, time.Now().Add(-48*time.Hour), t, tgt, time.Now().Add(-48*time.Hour),
) )
w := delivery.NewExportArchiveWriter( w := archive.NewExportArchiveWriter(
path, archiveTestLogger(), 0, path, archiveTestLogger(), 0,
) )
@@ -653,30 +709,28 @@ func TestArchiveSweep_ClosesHandleOfRegisteredWriter(
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`) tgt := env.seedDatabaseTarget(t, time.Hour, "")
env.seedArchiveRows( env.seedArchiveRows(
t, tgt, time.Now().Add(-48*time.Hour), t, tgt, time.Now().Add(-48*time.Hour),
) )
webhookDB := testWebhookDB(t) d := databaseTargetDelivery(newEvent(`{"n":1}`), tgt)
event := seedEvent(t, webhookDB, `{"n":1}`)
d := seedDatabaseTargetDelivery(t, webhookDB, event, tgt)
env.eng.ExportDeliverDatabase(webhookDB, d) require.NoError(t, env.writers.Write(d))
require.True( require.True(
t, env.eng.ExportArchiveHandleOpen(tgt.ID), t, env.writers.ExportArchiveHandleOpen(tgt.ID),
"the delivery must leave the archive handle open", "the delivery must leave the archive handle open",
) )
env.sweeper.ExportSweep(context.Background()) env.sweeper.ExportSweep(context.Background())
require.True( require.True(
t, env.eng.ExportHasArchiveWriter(tgt.ID), t, env.writers.ExportHasArchiveWriter(tgt.ID),
"the delivery's registry entry must survive the sweep", "the delivery's registry entry must survive the sweep",
) )
assert.False( assert.False(
t, env.eng.ExportArchiveHandleOpen(tgt.ID), t, env.writers.ExportArchiveHandleOpen(tgt.ID),
"the sweep must leave the archive closed", "the sweep must leave the archive closed",
) )
} }
@@ -692,7 +746,7 @@ func TestArchiveSweep_ClosesHandleBeforeReopening(t *testing.T) {
path := filepath.Join(t.TempDir(), "archive.db") path := filepath.Join(t.TempDir(), "archive.db")
w := delivery.NewExportArchiveWriter( w := archive.NewExportArchiveWriter(
path, archiveTestLogger(), 0, path, archiveTestLogger(), 0,
) )
@@ -720,8 +774,8 @@ func TestArchiveSweep_CancelledSweepLogsNoError(t *testing.T) {
var errorLines bytes.Buffer var errorLines bytes.Buffer
sweeper := delivery.NewTestArchiveSweeper( sweeper := archive.NewTestSweeper(
env.mainDB, env.eng, env.mainDB, env.writers,
slog.New(slog.NewTextHandler( slog.New(slog.NewTextHandler(
&errorLines, &errorLines,
&slog.HandlerOptions{Level: slog.LevelError}, &slog.HandlerOptions{Level: slog.LevelError},
@@ -740,36 +794,29 @@ func TestArchiveSweep_CancelledSweepLogsNoError(t *testing.T) {
} }
// TestArchiveSweep_NeverExpiryUntouched proves the sweep is a // TestArchiveSweep_NeverExpiryUntouched proves the sweep is a
// no-op for the default retention policy, so archives with no // no-op for the default retention policy, so an archive with an
// expiry (or the literal "never") behave exactly as before. // expiry of 0, which keeps it forever, is left alone.
func TestArchiveSweep_NeverExpiryUntouched(t *testing.T) { func TestArchiveSweep_NeverExpiryUntouched(t *testing.T) {
t.Parallel() t.Parallel()
for _, configJSON := range []string{ env := setupArchiveTest(t)
`{"expiry":"never"}`,
`{"expiry":""}`,
"",
} {
env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, configJSON) tgt := env.seedDatabaseTarget(t, 0, "")
path := env.seedArchiveRows( path := env.seedArchiveRows(
t, tgt, t, tgt,
time.Now().Add(-10000*time.Hour), time.Now().Add(-10000*time.Hour),
) )
env.sweeper.ExportSweep(context.Background()) env.sweeper.ExportSweep(context.Background())
assert.Equal( assert.Equal(
t, []string{sweepRowOld}, archivedEventIDs(t, path), t, []string{sweepRowOld}, archivedEventIDs(t, path),
"config %q must keep rows forever", configJSON, "an expiry of 0 must keep rows forever",
) )
assert.False( assert.False(
t, env.eng.ExportHasArchiveWriter(tgt.ID), t, env.writers.ExportHasArchiveWriter(tgt.ID),
"config %q must leave no registry entry behind", "an expiry of 0 must leave no registry entry behind",
configJSON, )
)
}
} }
// TestArchiveSweep_NeverExpirySkipsBeforeOpening pins the // TestArchiveSweep_NeverExpirySkipsBeforeOpening pins the
@@ -790,7 +837,7 @@ func TestArchiveSweep_NeverExpirySkipsBeforeOpening(
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"expiry":"never"}`) tgt := env.seedDatabaseTarget(t, 0, "")
path := env.archivePath(tgt) path := env.archivePath(tgt)
seedUnmigratedArchive(t, path) seedUnmigratedArchive(t, path)
@@ -830,7 +877,7 @@ func archiveTableExists(t *testing.T, path string) bool {
return openArchiveDBForRead(t, path). return openArchiveDBForRead(t, path).
Migrator(). Migrator().
HasTable(&delivery.ExportArchivedEvent{}) HasTable(&archive.ExportArchivedEvent{})
} }
// TestArchiveSweep_DoesNotCreateArchiveFile proves the sweep // TestArchiveSweep_DoesNotCreateArchiveFile proves the sweep
@@ -842,7 +889,7 @@ func TestArchiveSweep_DoesNotCreateArchiveFile(t *testing.T) {
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`) tgt := env.seedDatabaseTarget(t, time.Hour, "")
path := env.archivePath(tgt) path := env.archivePath(tgt)
require.NoFileExists(t, path) require.NoFileExists(t, path)
@@ -857,7 +904,7 @@ func TestArchiveSweep_DoesNotCreateArchiveFile(t *testing.T) {
} }
assert.False( assert.False(
t, env.eng.ExportHasArchiveWriter(tgt.ID), t, env.writers.ExportHasArchiveWriter(tgt.ID),
"the sweep must leave no registry entry behind", "the sweep must leave no registry entry behind",
) )
} }
@@ -873,9 +920,9 @@ func TestArchiveSweep_DoesNotCreateAfterWriterExists(
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`) tgt := env.seedDatabaseTarget(t, time.Hour, "")
path, err := env.eng.ExportEnsureArchiveWriter(tgt.ID) path, err := env.writers.ExportEnsureArchiveWriter(tgt.ID)
require.NoError(t, err) require.NoError(t, err)
require.NoFileExists(t, path) require.NoFileExists(t, path)
@@ -892,7 +939,7 @@ func TestArchiveSweep_SkipsDeletedWebhookTargets(t *testing.T) {
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`) tgt := env.seedDatabaseTarget(t, time.Hour, "")
path := env.seedArchiveRows( path := env.seedArchiveRows(
t, tgt, time.Now().Add(-48*time.Hour), t, tgt, time.Now().Add(-48*time.Hour),
) )
@@ -920,23 +967,16 @@ func TestArchiveSweep_ConcurrentWrites(t *testing.T) {
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`) tgt := env.seedDatabaseTarget(t, time.Hour, "")
webhookDB := testWebhookDB(t)
// The deliveries are seeded up front, on the test's own
// goroutine: the seed helpers assert, and testify assertions
// must not run off the test goroutine.
deliveries := make( deliveries := make(
[]*database.Delivery, 0, sweepConcurrentWrites, []*database.Delivery, 0, sweepConcurrentWrites,
) )
for range sweepConcurrentWrites { for range sweepConcurrentWrites {
event := seedEvent(t, webhookDB, `{"n":1}`)
deliveries = append( deliveries = append(
deliveries, deliveries,
seedDatabaseTargetDelivery(t, webhookDB, event, tgt), databaseTargetDelivery(newEvent(`{"n":1}`), tgt),
) )
} }
@@ -948,7 +988,7 @@ func TestArchiveSweep_ConcurrentWrites(t *testing.T) {
defer wg.Done() defer wg.Done()
for _, d := range deliveries { for _, d := range deliveries {
env.eng.ExportDeliverDatabase(webhookDB, d) _ = env.writers.Write(d)
} }
}() }()
@@ -972,7 +1012,7 @@ func TestArchiveSweeper_StopsCleanly(t *testing.T) {
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`) tgt := env.seedDatabaseTarget(t, time.Hour, "")
env.seedArchiveRows( env.seedArchiveRows(
t, tgt, time.Now().Add(-48*time.Hour), t, tgt, time.Now().Add(-48*time.Hour),
) )
@@ -1,7 +1,6 @@
package delivery package archive
import ( import (
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"io/fs" "io/fs"
@@ -18,9 +17,10 @@ import (
"sneak.berlin/go/webhooker/internal/gormlog" "sneak.berlin/go/webhooker/internal/gormlog"
) )
// archiveExpiryNever is the expiry sentinel (and default) that // ExpiryNever is the archive expiry, as the target forms take it,
// disables pruning so archived rows are kept forever. // that disables pruning so archived rows are kept forever. It is
const archiveExpiryNever = "never" // stored as an ArchiveExpiry of 0.
const ExpiryNever = "never"
// archiveReopenDebounce bounds how often an archive file is // archiveReopenDebounce bounds how often an archive file is
// closed and reopened. After each write the handle is closed // closed and reopened. After each write the handle is closed
@@ -71,27 +71,14 @@ var (
"archive writer has been evicted", "archive writer has been evicted",
) )
// ErrArchiveNameTaken is returned when an archive cannot be // ErrNameTaken is returned when an archive cannot be
// renamed because a file already has the new name. That file may // renamed because a file already has the new name. That file may
// be an archive with rows of its own, so it is never replaced. // be an archive with rows of its own, so it is never replaced.
ErrArchiveNameTaken = errors.New( ErrNameTaken = errors.New(
"a file already has the archive's new name", "a file already has the archive's new name",
) )
) )
// databaseTargetConfig is the optional per-target JSON config
// for a database (archive) target.
type databaseTargetConfig struct {
// Expiry is a Go duration (e.g. "720h") after which
// archived rows are pruned, or "never" (the default) to
// keep them forever.
Expiry string `json:"expiry"`
// Rotation is none (the default), monthly, daily or hourly: see
// archivePeriod.
Rotation string `json:"rotation"`
}
// archivedEvent is one fully captured webhook event stored in a // archivedEvent is one fully captured webhook event stored in a
// database target's archive for long-term retention. It is a // database target's archive for long-term retention. It is a
// self-contained copy — independent of the per-webhook event // self-contained copy — independent of the per-webhook event
@@ -112,76 +99,33 @@ type archivedEvent struct {
ArchivedAt time.Time `gorm:"index"` ArchivedAt time.Time `gorm:"index"`
} }
// parseArchiveExpiry reads the optional expiry from a database // ParseExpiry reads a user-supplied archive expiry for a database
// target's config JSON. An empty config, an empty expiry, or // target, as the target forms submit it, into the ArchiveExpiry the
// the literal "never" all mean keep forever, returned as a zero // target stores. Empty and "never" both mean keep forever, which is
// duration. Any other value must parse as a positive Go // 0; anything else must be a positive Go duration such as "720h". Any
// duration; a set-but-invalid value (unparseable, zero, or // other value is an error, so a bad expiry is refused at the form
// negative) is an error rather than a silent default, matching // rather than stored.
// ValidateArchiveExpiry at target creation. func ParseExpiry(expiry string) (time.Duration, error) {
func parseArchiveExpiry( if expiry == "" || expiry == ExpiryNever {
configJSON string,
) (time.Duration, error) {
if configJSON == "" {
return 0, nil return 0, nil
} }
var cfg databaseTargetConfig
err := json.Unmarshal([]byte(configJSON), &cfg)
if err != nil {
return 0, fmt.Errorf(
"parsing database target config: %w", err,
)
}
if cfg.Expiry == "" || cfg.Expiry == archiveExpiryNever {
return 0, nil
}
dur, err := time.ParseDuration(cfg.Expiry)
if err != nil {
return 0, fmt.Errorf(
"parsing archive expiry %q: %w", cfg.Expiry, err,
)
}
if dur <= 0 {
return 0, fmt.Errorf(
"%w: %q", errArchiveExpiryNotPositive, cfg.Expiry,
)
}
return dur, nil
}
// ValidateArchiveExpiry checks a user-supplied archive expiry
// for a database target at configuration time. Valid values are
// empty, "never" (both meaning keep forever), or a positive Go
// duration such as "720h". Anything else is an error, so a bad
// expiry is rejected when the target is created rather than
// failing every subsequent delivery.
func ValidateArchiveExpiry(expiry string) error {
if expiry == "" || expiry == archiveExpiryNever {
return nil
}
dur, err := time.ParseDuration(expiry) dur, err := time.ParseDuration(expiry)
if err != nil { if err != nil {
return fmt.Errorf( return 0, fmt.Errorf(
"expiry must be %q or a Go duration "+ "expiry must be %q or a Go duration "+
"such as \"720h\": %w", "such as \"720h\": %w",
archiveExpiryNever, err, ExpiryNever, err,
) )
} }
if dur <= 0 { if dur <= 0 {
return fmt.Errorf( return 0, fmt.Errorf(
"%w: %q", errArchiveExpiryNotPositive, expiry, "%w: %q", errArchiveExpiryNotPositive, expiry,
) )
} }
return nil return dur, nil
} }
// archiveWriter owns one database target's archive SQLite files. // archiveWriter owns one database target's archive SQLite files.
@@ -194,7 +138,7 @@ func ValidateArchiveExpiry(expiry string) error {
type archiveWriter struct { type archiveWriter struct {
mu sync.Mutex mu sync.Mutex
// path is the target's archive file as ArchivePath names it. A // path is the target's archive file as Path names it. A
// target that rotates writes to the files archivePeriodPath names // target that rotates writes to the files archivePeriodPath names
// for path and a period instead. // for path and a period instead.
path string path string
@@ -232,7 +176,7 @@ type archiveWriter struct {
// registry proper. // registry proper.
// //
// Unlike every other field here it is guarded by // Unlike every other field here it is guarded by
// databaseTarget.mu, not by this writer's mu: it describes the // Writers.mu, not by this writer's mu: it describes the
// registry entry rather than the file. // registry entry rather than the file.
sweepOwned bool sweepOwned bool
} }
@@ -277,7 +221,11 @@ func (w *archiveWriter) write(
} }
} }
row.ArchivedAt = time.Now() // GORM's clock, the one that stamps created_at in the other
// databases, so this is stored in the same form. A bare time.Now()
// carries a monotonic clock reading, which the driver would store
// as an " m=+…" suffix.
row.ArchivedAt = w.db.NowFunc()
err := w.db.Create(&row).Error err := w.db.Create(&row).Error
if err != nil { if err != nil {
@@ -495,7 +443,7 @@ func (w *archiveWriter) sweepFile(
// the next write creates its file under the new name. // the next write creates its file under the new name.
// //
// If a file already has one of the new names, nothing is moved and // If a file already has one of the new names, nothing is moved and
// the error is ErrArchiveNameTaken. If one file fails to move, those // the error is ErrNameTaken. If one file fails to move, those
// already moved are moved back before the error is returned, so the // already moved are moved back before the error is returned, so the
// archive is never split across two names. // archive is never split across two names.
func (w *archiveWriter) rename(name string) error { func (w *archiveWriter) rename(name string) error {
@@ -533,7 +481,7 @@ func (w *archiveWriter) rename(name string) error {
for _, taken := range to { for _, taken := range to {
if fileExists(taken) { if fileExists(taken) {
return fmt.Errorf( return fmt.Errorf(
"%w: %s", ErrArchiveNameTaken, filepath.Base(taken), "%w: %s", ErrNameTaken, filepath.Base(taken),
) )
} }
} }
@@ -586,7 +534,7 @@ func (w *archiveWriter) evict() {
// each row's archived time. It runs on every (re)open, so a // each row's archived time. It runs on every (re)open, so a
// steadily written archive is swept by its own write traffic. An // steadily written archive is swept by its own write traffic. An
// archive that goes idle receives no further reopens, which is // archive that goes idle receives no further reopens, which is
// why ArchiveSweeper exists to drive sweepExpired on a timer. // why Sweeper exists to drive sweepExpired on a timer.
// Failures are logged, not fatal: a prune error must not stop // Failures are logged, not fatal: a prune error must not stop
// archiving. // archiving.
func (w *archiveWriter) prune(expiry time.Duration) { func (w *archiveWriter) prune(expiry time.Duration) {
@@ -613,9 +561,9 @@ func (w *archiveWriter) prune(expiry time.Duration) {
} }
} }
// ArchiveFileInfo is what the metadata of a database target's archive // FileInfo is what the metadata of a database target's archive
// files says about them. // files says about them.
type ArchiveFileInfo struct { type FileInfo struct {
// Files counts the files. // Files counts the files.
Files int Files int
@@ -627,18 +575,18 @@ type ArchiveFileInfo struct {
Written time.Time Written time.Time
} }
// StatArchive reads the metadata of a database target's archive // Stat reads the metadata of a database target's archive
// files, given the path ArchivePath gives it (see archiveFiles), and // files, given the path Path gives it (see archiveFiles), and
// of their -wal, without opening them. With no files, which is so // of their -wal, without opening them. With no files, which is so
// before the first write and after the operator moved them away, the // before the first write and after the operator moved them away, the
// error wraps fs.ErrNotExist. // error wraps fs.ErrNotExist.
func StatArchive(path string) (ArchiveFileInfo, error) { func Stat(path string) (FileInfo, error) {
files, err := archiveFiles(path) files, err := archiveFiles(path)
if err != nil { if err != nil {
return ArchiveFileInfo{}, err return FileInfo{}, err
} }
var info ArchiveFileInfo var info FileInfo
for _, file := range files { for _, file := range files {
db, err := os.Stat(file.path) db, err := os.Stat(file.path)
@@ -647,7 +595,7 @@ func StatArchive(path string) (ArchiveFileInfo, error) {
} }
if err != nil { if err != nil {
return ArchiveFileInfo{}, err return FileInfo{}, err
} }
info.Files++ info.Files++
@@ -663,7 +611,7 @@ func StatArchive(path string) (ArchiveFileInfo, error) {
} }
if err != nil { if err != nil {
return ArchiveFileInfo{}, err return FileInfo{}, err
} }
info.Size += wal.Size() info.Size += wal.Size()
@@ -674,7 +622,7 @@ func StatArchive(path string) (ArchiveFileInfo, error) {
} }
if info.Files == 0 { if info.Files == 0 {
return ArchiveFileInfo{}, fmt.Errorf( return FileInfo{}, fmt.Errorf(
"no archive file for %s: %w", path, fs.ErrNotExist, "no archive file for %s: %w", path, fs.ErrNotExist,
) )
} }
@@ -1,4 +1,4 @@
package delivery_test package archive_test
import ( import (
"bytes" "bytes"
@@ -14,7 +14,7 @@ import (
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"gorm.io/gorm" "gorm.io/gorm"
gormlogger "gorm.io/gorm/logger" gormlogger "gorm.io/gorm/logger"
"sneak.berlin/go/webhooker/internal/delivery" "sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/middleware" "sneak.berlin/go/webhooker/internal/middleware"
) )
@@ -57,7 +57,7 @@ func (g *gormDefaultBuf) String() string {
// without exporting production code to carry it, and a logging // without exporting production code to carry it, and a logging
// detector is not worth a production symbol. What it detects is the // detector is not worth a production symbol. What it detects is the
// third gorm.Open in this service, at // third gorm.Open in this service, at
// internal/delivery/target_database_archive.go — the archive writer, // internal/archive/writer.go — the archive writer,
// whose type is unexported, so nothing outside this package can drive // whose type is unexported, so nothing outside this package can drive
// it. // it.
func captureArchiveGORMDefault(t *testing.T) *gormDefaultBuf { func captureArchiveGORMDefault(t *testing.T) *gormDefaultBuf {
@@ -85,7 +85,7 @@ func captureArchiveGORMDefault(t *testing.T) *gormDefaultBuf {
// writer's gorm.Open to the adapter. // writer's gorm.Open to the adapter.
// //
// Restore a bare &gorm.Config{} at // Restore a bare &gorm.Config{} at
// internal/delivery/target_database_archive.go and this fails: the // internal/archive/writer.go and this fails: the
// default logger prints the fully interpolated SELECT on every // default logger prints the fully interpolated SELECT on every
// ErrRecordNotFound, so the client-chosen event id below arrives whole // ErrRecordNotFound, so the client-chosen event id below arrives whole
// and unbounded on stdout, answering to no level the operator set. // and unbounded on stdout, answering to no level the operator set.
@@ -100,7 +100,7 @@ func TestArchiveWriter_NeverUsesGORMsDefaultLogger(t *testing.T) {
gormDefault := captureArchiveGORMDefault(t) gormDefault := captureArchiveGORMDefault(t)
w := delivery.NewExportArchiveWriter( w := archive.NewExportArchiveWriter(
filepath.Join(t.TempDir(), "archive.db"), filepath.Join(t.TempDir(), "archive.db"),
slog.New(slog.NewTextHandler( slog.New(slog.NewTextHandler(
&captured, &slog.HandlerOptions{Level: slog.LevelDebug}, &captured, &slog.HandlerOptions{Level: slog.LevelDebug},
@@ -118,7 +118,7 @@ func TestArchiveWriter_NeverUsesGORMsDefaultLogger(t *testing.T) {
value := strings.Repeat("\x01", archiveGORMFillBytes) + value := strings.Repeat("\x01", archiveGORMFillBytes) +
archiveGORMTailMarker archiveGORMTailMarker
var row delivery.ExportArchivedEvent var row archive.ExportArchivedEvent
err := w.DB().Where("event_id = ?", value).First(&row).Error err := w.DB().Where("event_id = ?", value).First(&row).Error
require.ErrorIs(t, err, gorm.ErrRecordNotFound) require.ErrorIs(t, err, gorm.ErrRecordNotFound)
+5 -4
View File
@@ -25,10 +25,11 @@ func TestEventDBCreateOmitsAssociations(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
target := database.Target{ target := database.Target{
WebhookID: webhookID, WebhookID: webhookID,
Name: "leaky-target", Name: "leaky-target",
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
Config: `{"webhookUrl":"https://hooks.example/secret"}`, URL: "https://hooks.example/secret",
MaxAttempts: 1,
} }
target.ID = uuid.New().String() target.ID = uuid.New().String()
+3 -2
View File
@@ -79,8 +79,9 @@ func TestEventTierQueriesUseTheirIndexes(t *testing.T) {
byStatus := "idx_deliveries_status (status=? AND deleted_at=?)" byStatus := "idx_deliveries_status (status=? AND deleted_at=?)"
byEvent := "idx_deliveries_event_id (event_id=? AND deleted_at=?)" byEvent := "idx_deliveries_event_id (event_id=? AND deleted_at=?)"
// The delivery engine: recovery and the retry sweep, the sweep for // The delivery engine: the recovery routine, which reads the
// stranded pending deliveries, and the queue depth count. // retrying deliveries and, on a tick of the sweep, the pending ones
// untouched for a while; and the queue depth count.
assertPlanUses(t, db, dry.Where( assertPlanUses(t, db, dry.Where(
"status = ?", database.DeliveryStatusRetrying, "status = ?", database.DeliveryStatusRetrying,
).Find(&deliveries), byStatus) ).Find(&deliveries), byStatus)
+1 -2
View File
@@ -45,8 +45,7 @@ type Event struct {
// ResubmittedFromID names the event this one was copied from by // ResubmittedFromID names the event this one was copied from by
// an operator resubmit. It is nil for an event that arrived on // an operator resubmit. It is nil for an event that arrived on
// the receiver, which is every event created before the column // the receiver. It is not a foreign key: the source event can be
// existed. It is not a foreign key: the source event can be
// reaped by retention while its copies remain, and the id is // reaped by retention while its copies remain, and the id is
// kept as the record of where the copy came from either way. // kept as the record of where the copy came from either way.
ResubmittedFromID *string `gorm:"type:uuid;index:idx_events_resubmitted_from_id,priority:1;index:idx_events_entrypoint_id,priority:3" json:"resubmittedFromId,omitempty"` ResubmittedFromID *string `gorm:"type:uuid;index:idx_events_resubmitted_from_id,priority:1;index:idx_events_entrypoint_id,priority:3" json:"resubmittedFromId,omitempty"`
+3 -2
View File
@@ -23,8 +23,9 @@ func TestPreloadedModelsMarshalWithoutTheirParent(t *testing.T) {
Name: testWebhookName, Name: testWebhookName,
Entrypoints: []database.Entrypoint{{Path: uuid.New().String()}}, Entrypoints: []database.Entrypoint{{Path: uuid.New().String()}},
Targets: []database.Target{{ Targets: []database.Target{{
Name: "log", Name: "log",
Type: database.TargetTypeLog, Type: database.TargetTypeLog,
MaxAttempts: 1,
}}, }},
} }
require.NoError(t, db.Create(&stored).Error) require.NoError(t, db.Create(&stored).Error)
+23 -10
View File
@@ -29,8 +29,10 @@ func marshalModel(t *testing.T, v any) string {
// these tags the first handler that marshals a model serialises the // these tags the first handler that marshals a model serialises the
// secret with it. Each field below is a live credential: // secret with it. Each field below is a live credential:
// //
// - Target.Config holds an incoming-webhook URL whose path segments // - Target.URL is an incoming-webhook URL whose path segments are
// are the bearer token. // the bearer token.
// - Target.Headers holds header values, routinely authorization
// tokens.
// - APIKey.Key is a bearer token outright. // - APIKey.Key is a bearer token outright.
// - Setting.Value holds the session encryption key. // - Setting.Value holds the session encryption key.
// - User.Password holds the Argon2 hash, and was already tagged. // - User.Password holds the Argon2 hash, and was already tagged.
@@ -44,11 +46,21 @@ func TestModelsDoNotMarshalTheirSecrets(t *testing.T) {
model any model any
}{ }{
{ {
name: "target config", name: "target url",
model: database.Target{ model: database.Target{
Name: keptField, Name: keptField,
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
Config: `{"webhookUrl":"https://h/s/` + marker + `"}`, URL: "https://h/s/" + marker,
MaxAttempts: 1,
},
},
{
name: "target headers",
model: database.Target{
Name: keptField,
Type: database.TargetTypeHTTP,
Headers: map[string]string{"Authorization": marker},
MaxAttempts: 1,
}, },
}, },
{ {
@@ -86,10 +98,10 @@ func TestModelsDoNotMarshalTheirSecrets(t *testing.T) {
} }
} }
// TestWebhookMarshalsNoTargetConfig covers the nested case: a webhook // TestWebhookMarshalsNoTargetURL covers the nested case: a webhook
// marshalled with its targets preloaded must not carry the credential // marshalled with its targets preloaded must not carry the credential
// through the association either. // through the association either.
func TestWebhookMarshalsNoTargetConfig(t *testing.T) { func TestWebhookMarshalsNoTargetURL(t *testing.T) {
t.Parallel() t.Parallel()
const marker = "QQNESTEDMARKERQQ" const marker = "QQNESTEDMARKERQQ"
@@ -97,8 +109,9 @@ func TestWebhookMarshalsNoTargetConfig(t *testing.T) {
encoded := marshalModel(t, database.Webhook{ encoded := marshalModel(t, database.Webhook{
Name: keptField, Name: keptField,
Targets: []database.Target{{ Targets: []database.Target{{
Name: "slack", Name: "slack",
Config: `{"webhookUrl":"https://h/s/` + marker + `"}`, URL: "https://h/s/" + marker,
MaxAttempts: 1,
}}, }},
}) })
+39 -11
View File
@@ -1,5 +1,7 @@
package database package database
import "time"
// TargetType represents the type of delivery target // TargetType represents the type of delivery target
type TargetType string type TargetType string
@@ -11,7 +13,9 @@ const (
TargetTypeSlack TargetType = "slack" TargetTypeSlack TargetType = "slack"
) )
// Target represents a delivery target for a webhook // Target represents a delivery target for a webhook. Its settings
// are the fields from URL to ArchiveRotation; each type uses some of
// them, and the ones a type does not use stay empty.
type Target struct { type Target struct {
BaseModel BaseModel
@@ -20,18 +24,42 @@ type Target struct {
Type TargetType `gorm:"not null" json:"type"` Type TargetType `gorm:"not null" json:"type"`
Active bool `gorm:"default:true" json:"active"` Active bool `gorm:"default:true" json:"active"`
// Configuration fields (JSON stored based on type). // URL is where an http or slack target sends each event.
// //
// json:"-" because the blob holds the target's credential — a // json:"-" because the URL is the target's credential: a Slack
// Slack incoming-webhook URL, or an http destination whose path // incoming-webhook URL, or an http destination whose path segments
// segments are the secret. delivery.TargetView is the masking // are the secret. delivery.TargetView is the masking barrier for
// barrier for the HTML path; this tag is the barrier for any // the HTML path; this tag is the barrier for any handler that
// handler that marshals the model itself. // marshals the model itself.
Config string `gorm:"type:text" json:"-"` // JSON configuration URL string `json:"-"`
// For HTTP targets (max_retries=0 means fire-and-forget, // Headers is an http target's extra request headers, name to
// >0 enables retries with backoff) // value, stored as JSON text. json:"-" for the same reason as URL:
MaxRetries int `json:"maxRetries,omitempty"` // a header value is routinely an authorization token.
Headers map[string]string `gorm:"serializer:json" json:"-"`
// TimeoutSeconds is an http target's per-request timeout; 0 uses
// the delivery engine's shared client timeout.
TimeoutSeconds int `json:"timeoutSeconds,omitempty"`
// ForwardQuery passes each event's query string on to an http
// target, appended to URL. Off, URL is sent exactly as stored.
ForwardQuery bool `json:"forwardQuery,omitempty"`
// ArchiveExpiry is how long a database target keeps each archived
// event; 0 keeps them forever.
ArchiveExpiry time.Duration `json:"archiveExpiry,omitempty"`
// ArchiveRotation is how often a database target starts a new
// archive file: none, monthly, daily or hourly.
ArchiveRotation string `json:"archiveRotation,omitempty"`
// MaxAttempts is how many times an HTTP or Slack target attempts
// each delivery in all, at least 1. 1 is a single attempt with no
// circuit breaker; above 1, a failed attempt is retried with
// backoff behind the breaker. The table's check constraint refuses
// a target saved with less than 1, whatever its type.
MaxAttempts int `gorm:"not null;check:max_attempts >= 1" json:"maxAttempts,omitempty"`
// Relations. No model marshals the record it belongs to: // Relations. No model marshals the record it belongs to:
// Webhook.Targets leads back here, and the JSON could loop. // Webhook.Targets leads back here, and the JSON could loop.
+63
View File
@@ -0,0 +1,63 @@
package database_test
import (
"testing"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database"
)
// newHTTPTarget returns an http target that makes maxAttempts attempts
// at each delivery.
func newHTTPTarget(maxAttempts int) *database.Target {
return &database.Target{
WebhookID: uuid.New().String(),
Name: "target",
Type: database.TargetTypeHTTP,
URL: "https://example.com/",
MaxAttempts: maxAttempts,
}
}
// TestTargetCreate_RefusesZeroAttempts relies on the targets table
// alone: nothing in the model checks the number, so the check
// constraint is what keeps a target with 0 attempts out of the
// database.
func TestTargetCreate_RefusesZeroAttempts(t *testing.T) {
t.Parallel()
db := startedTestDB(t)
err := db.Create(newHTTPTarget(0)).Error
require.Error(t, err)
assert.Contains(t, err.Error(), "CHECK constraint failed")
}
func TestTargetCreate_AcceptsOneAttempt(t *testing.T) {
t.Parallel()
db := startedTestDB(t)
require.NoError(t, db.Create(newHTTPTarget(1)).Error)
}
// TestTargetHeaders_StoredAsJSON reads the headers column back as
// text, to show it holds the header map as JSON.
func TestTargetHeaders_StoredAsJSON(t *testing.T) {
t.Parallel()
db := startedTestDB(t)
target := newHTTPTarget(1)
target.Headers = map[string]string{"Authorization": "Bearer x"}
require.NoError(t, db.Create(target).Error)
var headers string
require.NoError(t, db.Raw(
"SELECT headers FROM targets WHERE id = ?", target.ID,
).Scan(&headers).Error)
assert.JSONEq(t, `{"Authorization":"Bearer x"}`, headers)
}
+19 -56
View File
@@ -4,27 +4,13 @@ import (
"math" "math"
"strconv" "strconv"
"time" "time"
"gorm.io/gorm"
) )
const ( const (
// DefaultRetentionDays is the event retention period applied to a // DefaultRetentionDays is the event retention period the new
// webhook created without an explicit retention value. It is the // webhook form applies when its retention field is left blank.
// single source of truth for that policy and must stay in sync
// with the `gorm:"default:30"` column default on
// Webhook.RetentionDays below; a struct tag cannot reference a
// constant, so a test asserts the two agree.
DefaultRetentionDays = 30 DefaultRetentionDays = 30
// RetentionForeverDays is the sentinel RetentionDays value meaning
// "retain events forever". Users express that intent as 0, which
// Webhook.BeforeSave rewrites to this value: the column default
// substitutes DefaultRetentionDays for a zero value at insert
// time, so a zero can never survive a round trip to the database.
// Nothing outside this file may hardcode the number.
RetentionForeverDays = 365 * 1000
// MaxFiniteRetentionDays is the largest finite retention period the // MaxFiniteRetentionDays is the largest finite retention period the
// reaper's cutoff arithmetic can represent, and therefore the // reaper's cutoff arithmetic can represent, and therefore the
// largest one a caller may request. It is derived from that // largest one a caller may request. It is derived from that
@@ -38,9 +24,7 @@ const (
// turns the cutoff into a timestamp in the far future that matches // turns the cutoff into a timestamp in the far future that matches
// every row in the webhook's database. That is why this bound is // every row in the webhook's database. That is why this bound is
// enforced on input and why retentionCutoff saturates underneath // enforced on input and why retentionCutoff saturates underneath
// it. Note that RetentionForeverDays deliberately sits above this // it.
// ceiling: such webhooks are skipped before any cutoff is
// computed, and never reach the arithmetic at all.
MaxFiniteRetentionDays = int( MaxFiniteRetentionDays = int(
math.MaxInt64 / int64(time.Hour) / hoursPerDay, math.MaxInt64 / int64(time.Hour) / hoursPerDay,
) )
@@ -48,12 +32,10 @@ const (
// Webhook represents a webhook processing unit that groups entrypoints and targets // Webhook represents a webhook processing unit that groups entrypoints and targets
// //
// Every method below takes a pointer receiver. BeforeSave has to, // Every method below takes a pointer receiver, so handlers put a
// because it mutates the record and GORM only invokes hooks declared // *Webhook into template data: html/template cannot call a pointer
// that way; the display helpers follow suit so the receiver kinds do // method on a value held in a map, because a map element is not
// not mix. Handlers therefore put a *Webhook into template data: // addressable.
// html/template cannot call a pointer method on a value held in a map,
// because a map element is not addressable.
type Webhook struct { type Webhook struct {
BaseModel BaseModel
@@ -61,10 +43,9 @@ type Webhook struct {
Name string `gorm:"not null" json:"name"` Name string `gorm:"not null" json:"name"`
Description string `json:"description"` Description string `json:"description"`
// RetentionDays is the number of days to retain events. A value of // RetentionDays is the number of days to retain events, and 0
// RetentionForeverDays means retain forever. The column default // means retain them forever.
// must equal DefaultRetentionDays. RetentionDays int `json:"retentionDays"`
RetentionDays int `gorm:"default:30" json:"retentionDays"`
// Relations. No model marshals the record it belongs to: // Relations. No model marshals the record it belongs to:
// User.Webhooks leads back here, and the JSON could loop. // User.Webhooks leads back here, and the JSON could loop.
@@ -73,36 +54,17 @@ type Webhook struct {
Targets []Target `json:"targets,omitempty"` Targets []Target `json:"targets,omitempty"`
} }
// BeforeSave normalises RetentionDays on every insert and update. A // retainsForever reports whether a RetentionDays value means "keep
// non-positive value is the user's way of asking for "retain forever", // events indefinitely", which 0 does. It is the single definition of
// which is stored as the RetentionForeverDays sentinel. // that question, shared by Webhook.RetainsForever and by the reaper's
//
// This has to happen in a hook rather than at the call sites. GORM
// substitutes the column default (DefaultRetentionDays) for a zero
// value while building the insert statement, which runs after
// BeforeSave; rewriting any later than this loses that race and the
// row lands at 30 days. Living on the model also means a future call
// site — a REST API, a fixture, a migration — cannot bypass it.
func (w *Webhook) BeforeSave(_ *gorm.DB) error {
if w.RetentionDays <= 0 {
w.RetentionDays = RetentionForeverDays
}
return nil
}
// retainsForever reports whether a stored RetentionDays value means
// "keep events indefinitely". It is the single definition of that
// question, shared by Webhook.RetainsForever and by the reaper's
// cutoff computation so the two cannot disagree about which webhooks // cutoff computation so the two cannot disagree about which webhooks
// are exempt from reaping. // are exempt from reaping.
// //
// It accepts the RetentionForeverDays sentinel written by BeforeSave // A negative value, which the forms refuse, is read the same way: as a
// and, defensively, the non-positive values that rows written before // day count it would put the cutoff in the future, where it matches
// the sentinel existed may still carry. // every event the webhook has.
func retainsForever(retentionDays int) bool { func retainsForever(retentionDays int) bool {
return retentionDays <= 0 || return retentionDays <= 0
retentionDays >= RetentionForeverDays
} }
// RetainsForever reports whether this webhook's events are kept // RetainsForever reports whether this webhook's events are kept
@@ -119,7 +81,8 @@ func (w *Webhook) RetentionCutoff(now time.Time) (time.Time, bool) {
} }
// RetentionLabel returns the webhook's retention policy as display // RetentionLabel returns the webhook's retention policy as display
// text, so that no template has to know about the sentinel value. // text: "forever" rather than "0 days" for a webhook that keeps its
// events.
func (w *Webhook) RetentionLabel() string { func (w *Webhook) RetentionLabel() string {
if w.RetainsForever() { if w.RetainsForever() {
return "forever" return "forever"
+10 -72
View File
@@ -2,8 +2,6 @@ package database_test
import ( import (
"context" "context"
"reflect"
"strconv"
"testing" "testing"
"time" "time"
@@ -47,8 +45,7 @@ func storedRetention(t *testing.T, db *gorm.DB, id string) int {
} }
// newWebhookWithRetention creates a webhook through the ordinary Create // newWebhookWithRetention creates a webhook through the ordinary Create
// path, so the BeforeSave hook and the GORM column default both apply // path, as the new webhook form does.
// exactly as they do in production.
func newWebhookWithRetention( func newWebhookWithRetention(
t *testing.T, t *testing.T,
db *gorm.DB, db *gorm.DB,
@@ -67,7 +64,7 @@ func newWebhookWithRetention(
return wh.ID return wh.ID
} }
func TestWebhookBeforeSave_ZeroBecomesForeverSentinel(t *testing.T) { func TestWebhookRetentionDays_ZeroIsStoredAsZero(t *testing.T) {
t.Parallel() t.Parallel()
db := startedTestDB(t) db := startedTestDB(t)
@@ -75,31 +72,10 @@ func TestWebhookBeforeSave_ZeroBecomesForeverSentinel(t *testing.T) {
wh := &database.Webhook{RetentionDays: 0} wh := &database.Webhook{RetentionDays: 0}
id := newWebhookWithRetention(t, db, wh) id := newWebhookWithRetention(t, db, wh)
assert.Equal( assert.Equal(t, 0, storedRetention(t, db, id))
t,
database.RetentionForeverDays,
storedRetention(t, db, id),
"a zero retention must be stored as the sentinel, "+
"not replaced by the column default",
)
} }
func TestWebhookBeforeSave_NegativeBecomesForeverSentinel(t *testing.T) { func TestWebhookRetentionDays_DayCountIsStored(t *testing.T) {
t.Parallel()
db := startedTestDB(t)
wh := &database.Webhook{RetentionDays: -5}
id := newWebhookWithRetention(t, db, wh)
assert.Equal(
t,
database.RetentionForeverDays,
storedRetention(t, db, id),
)
}
func TestWebhookBeforeSave_PositiveIsPreserved(t *testing.T) {
t.Parallel() t.Parallel()
db := startedTestDB(t) db := startedTestDB(t)
@@ -110,10 +86,9 @@ func TestWebhookBeforeSave_PositiveIsPreserved(t *testing.T) {
assert.Equal(t, 7, storedRetention(t, db, id)) assert.Equal(t, 7, storedRetention(t, db, id))
} }
// TestWebhookBeforeSave_UpdateToZeroBecomesSentinel proves the hook // TestWebhookRetentionDays_UpdateToZeroIsStoredAsZero changes a day
// fires on update as well as insert, via the same Save call the edit // count to 0 through the same Save call the edit handler makes.
// handler makes. func TestWebhookRetentionDays_UpdateToZeroIsStoredAsZero(t *testing.T) {
func TestWebhookBeforeSave_UpdateToZeroBecomesSentinel(t *testing.T) {
t.Parallel() t.Parallel()
db := startedTestDB(t) db := startedTestDB(t)
@@ -125,28 +100,7 @@ func TestWebhookBeforeSave_UpdateToZeroBecomesSentinel(t *testing.T) {
wh.RetentionDays = 0 wh.RetentionDays = 0
require.NoError(t, db.Omit(clause.Associations).Save(wh).Error) require.NoError(t, db.Omit(clause.Associations).Save(wh).Error)
assert.Equal( assert.Equal(t, 0, storedRetention(t, db, id))
t,
database.RetentionForeverDays,
storedRetention(t, db, id),
)
}
// TestWebhookRetentionColumnDefaultMatchesConstant guards the one place
// the default lives twice: a struct tag cannot reference a constant, so
// this asserts the tag and DefaultRetentionDays agree.
func TestWebhookRetentionColumnDefaultMatchesConstant(t *testing.T) {
t.Parallel()
field, ok := reflect.TypeFor[database.Webhook]().
FieldByName("RetentionDays")
require.True(t, ok, "Webhook.RetentionDays must exist")
assert.Equal(
t,
"default:"+strconv.Itoa(database.DefaultRetentionDays),
field.Tag.Get("gorm"),
)
} }
// TestMaxFiniteRetentionDaysIsTheOverflowCeiling asserts that the // TestMaxFiniteRetentionDaysIsTheOverflowCeiling asserts that the
@@ -176,14 +130,6 @@ func TestMaxFiniteRetentionDaysIsTheOverflowCeiling(t *testing.T) {
time.Duration(overCeiling*hoursPerDay)*time.Hour, time.Duration(overCeiling*hoursPerDay)*time.Hour,
"one day past the ceiling must overflow", "one day past the ceiling must overflow",
) )
assert.Less(
t,
database.MaxFiniteRetentionDays,
database.RetentionForeverDays,
"the sentinel sits above the ceiling and is only safe "+
"because retain-forever webhooks skip the arithmetic",
)
} }
func TestWebhookRetainsForeverAndLabel(t *testing.T) { func TestWebhookRetainsForeverAndLabel(t *testing.T) {
@@ -195,16 +141,8 @@ func TestWebhookRetainsForeverAndLabel(t *testing.T) {
forever bool forever bool
label string label string
}{ }{
{ {"zero", 0, true, testForeverLabel},
"sentinel", {"negative", -1, true, testForeverLabel},
database.RetentionForeverDays, true, testForeverLabel,
},
{
"above sentinel",
database.RetentionForeverDays + 1, true, testForeverLabel,
},
{"legacy zero", 0, true, testForeverLabel},
{"legacy negative", -1, true, testForeverLabel},
{"default", database.DefaultRetentionDays, false, "30 days"}, {"default", database.DefaultRetentionDays, false, "30 days"},
{"one day", 1, false, "1 day"}, {"one day", 1, false, "1 day"},
} }
+6 -4
View File
@@ -184,8 +184,9 @@ func (r *RetentionReaper) sweep(ctx context.Context) {
wh := webhooks[i] wh := webhooks[i]
// A missing database has nothing to reap. Restart recovery // A missing database has nothing to reap. The delivery
// reports a lost one (see WebhookDBManager.GetDB). // engine's recovery routine reports a lost one at start (see
// WebhookDBManager.GetDB).
if !r.dbManager.DBExists(wh.ID) { if !r.dbManager.DBExists(wh.ID) {
continue continue
} }
@@ -243,7 +244,8 @@ func (r *RetentionReaper) reapWebhook(
// retentionCutoff returns the timestamp before which a webhook's // retentionCutoff returns the timestamp before which a webhook's
// events have expired, and whether any cutoff applies at all. It // events have expired, and whether any cutoff applies at all. It
// reports false for a retain-forever policy, so no DELETE is issued. // reports false for a retention of 0, which means retain forever, so
// no DELETE is issued.
// //
// The day count is clamped to MaxFiniteRetentionDays first. This is // The day count is clamped to MaxFiniteRetentionDays first. This is
// defense in depth rather than decoration: a time.Duration is an int64 // defense in depth rather than decoration: a time.Duration is an int64
@@ -253,7 +255,7 @@ func (r *RetentionReaper) reapWebhook(
// row in the database: the sweep then deletes every event, delivery, // row in the database: the sweep then deletes every event, delivery,
// and delivery result, including ones created seconds ago. Rejecting // and delivery result, including ones created seconds ago. Rejecting
// out-of-range input at the form is the primary guard; saturating here // out-of-range input at the form is the primary guard; saturating here
// means an old row, a migration, or a future call site cannot turn a // means a value that reaches the row by any other path cannot turn a
// too-large retention into total data loss. // too-large retention into total data loss.
func retentionCutoff( func retentionCutoff(
now time.Time, now time.Time,
+19 -65
View File
@@ -85,41 +85,6 @@ func createWebhook(
db.Omit(clause.Associations).Create(wh).Error, db.Omit(clause.Associations).Create(wh).Error,
) )
// Webhook.BeforeSave rewrites a non-positive RetentionDays to the
// retain-forever sentinel, and the column's GORM default would
// otherwise substitute 30. Force the requested value with a
// column-level update so tests can plant legacy rows that predate
// the sentinel and still carry a literal 0 or negative value.
require.NoError(
t,
db.Model(wh).
Update("retention_days", retentionDays).Error,
)
return wh.ID
}
// createWebhookNormally inserts a webhook through the ordinary Create
// path, with no column-level forcing, so Webhook.BeforeSave applies
// exactly as it does in production. Passing 0 therefore yields a row
// holding the RetentionForeverDays sentinel.
func createWebhookNormally(
t *testing.T,
db *gorm.DB,
retentionDays int,
) string {
t.Helper()
wh := &database.Webhook{
UserID: uuid.New().String(),
Name: testWebhookName,
RetentionDays: retentionDays,
}
require.NoError(
t,
db.Omit(clause.Associations).Create(wh).Error,
)
return wh.ID return wh.ID
} }
@@ -281,19 +246,19 @@ func TestRetentionReaper_ReapsExpiredKeepsRecent(t *testing.T) {
assertChainPresent(t, db, recent) assertChainPresent(t, db, recent)
} }
// TestRetentionReaper_SkipsSentinelReapsFiniteInSameSweep covers the // TestRetentionReaper_SkipsZeroReapsFiniteInSameSweep covers the
// end-to-end retain-forever path: a webhook created the normal way with // end-to-end retain-forever path: a webhook created with a retention
// a requested retention of 0 lands on the RetentionForeverDays // of 0 reads back 0, and the reaper leaves its ancient events alone
// sentinel, and the reaper leaves its ancient events alone while still // while still reaping a finite-retention webhook in the very same
// reaping a finite-retention webhook in the very same sweep. // sweep.
func TestRetentionReaper_SkipsSentinelReapsFiniteInSameSweep( func TestRetentionReaper_SkipsZeroReapsFiniteInSameSweep(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
env := setupRetentionTest(t) env := setupRetentionTest(t)
foreverID := createWebhookNormally(t, env.mainDB.DB(), 0) foreverID := createWebhook(t, env.mainDB.DB(), 0)
var stored database.Webhook var stored database.Webhook
@@ -302,14 +267,12 @@ func TestRetentionReaper_SkipsSentinelReapsFiniteInSameSweep(
env.mainDB.DB().Where("id = ?", foreverID). env.mainDB.DB().Where("id = ?", foreverID).
First(&stored).Error, First(&stored).Error,
) )
require.Equal( require.Zero(
t, t, stored.RetentionDays,
database.RetentionForeverDays, "a retention of 0 must be stored as 0",
stored.RetentionDays,
"a requested retention of 0 must persist as the sentinel",
) )
finiteID := createWebhookNormally(t, env.mainDB.DB(), 30) finiteID := createWebhook(t, env.mainDB.DB(), 30)
foreverDB, err := env.mgr.GetDB(foreverID) foreverDB, err := env.mgr.GetDB(foreverID)
require.NoError(t, err) require.NoError(t, err)
@@ -335,14 +298,10 @@ func TestRetentionReaper_SkipsSentinelReapsFiniteInSameSweep(
// negative, so subtracting it moves the cutoff into the far future, // negative, so subtracting it moves the cutoff into the far future,
// where "created_at < cutoff" matches every row: an event created a // where "created_at < cutoff" matches every row: an event created a
// moment ago, and its delivery and delivery result, were all deleted // moment ago, and its delivery and delivery result, were all deleted
// on the first sweep. 200000 is inside that band and below the // on the first sweep. 200000 is inside that band.
// retain-forever sentinel, so it is treated as a finite policy and
// really does reach the arithmetic.
// //
// The row is planted at the column level because such a value can no // The form refuses such a value; the point of the test is that a row
// longer be submitted through the form; the point of the test is that // written by any other path still cannot trigger the wipe.
// a row from an older version, or a future call site, still cannot
// trigger the wipe.
func TestRetentionReaper_HugeFiniteRetentionRetainsRecentEvents( func TestRetentionReaper_HugeFiniteRetentionRetainsRecentEvents(
t *testing.T, t *testing.T,
) { ) {
@@ -358,12 +317,6 @@ func TestRetentionReaper_HugeFiniteRetentionRetainsRecentEvents(
database.MaxFiniteRetentionDays, database.MaxFiniteRetentionDays,
"the test value must exceed what the cutoff can represent", "the test value must exceed what the cutoff can represent",
) )
require.Less(
t,
overflowingRetentionDays,
database.RetentionForeverDays,
"the test value must not be treated as retain-forever",
)
webhookID := createWebhook( webhookID := createWebhook(
t, env.mainDB.DB(), overflowingRetentionDays, t, env.mainDB.DB(), overflowingRetentionDays,
@@ -379,14 +332,15 @@ func TestRetentionReaper_HugeFiniteRetentionRetainsRecentEvents(
assertChainPresent(t, db, fresh) assertChainPresent(t, db, fresh)
} }
func TestRetentionReaper_RetainsForeverWhenNonPositive(t *testing.T) { // TestRetentionReaper_RetainsForeverWhenNegative checks that a negative
// retention, which the forms refuse, keeps every event rather than
// putting the cutoff in the future, where it would match them all.
func TestRetentionReaper_RetainsForeverWhenNegative(t *testing.T) {
t.Parallel() t.Parallel()
env := setupRetentionTest(t) env := setupRetentionTest(t)
// A legacy row written before the sentinel existed still carries a webhookID := createWebhook(t, env.mainDB.DB(), -1)
// literal 0; the <= 0 guard must keep honouring it.
webhookID := createWebhook(t, env.mainDB.DB(), 0)
db, err := env.mgr.GetDB(webhookID) db, err := env.mgr.GetDB(webhookID)
require.NoError(t, err) require.NoError(t, err)
+12 -7
View File
@@ -87,13 +87,18 @@ func TestMainSchemaHoldsEveryModel(t *testing.T) {
Active: true, Active: true,
}) })
assertRoundTrip(t, db, &database.Target{ assertRoundTrip(t, db, &database.Target{
BaseModel: newBaseModel(), BaseModel: newBaseModel(),
WebhookID: uuid.New().String(), WebhookID: uuid.New().String(),
Name: "target", Name: "target",
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Active: true, Active: true,
Config: `{"url":"https://example.com/"}`, URL: "https://example.com/",
MaxRetries: 3, Headers: map[string]string{"X-Api-Key": "key"},
TimeoutSeconds: 30,
ForwardQuery: true,
ArchiveExpiry: 720 * time.Hour,
ArchiveRotation: "daily",
MaxAttempts: 3,
}) })
} }
+1 -1
View File
@@ -137,7 +137,7 @@ func TestPerWebhookEventDatabaseFilesAreOwnerOnly(t *testing.T) {
} }
// TestArchiveDatabaseFilesAreOwnerOnly covers the archive-*.db tier. // TestArchiveDatabaseFilesAreOwnerOnly covers the archive-*.db tier.
// internal/delivery builds that path and opens it through OpenSQLite, // internal/archive builds that path and opens it through OpenSQLite,
// the same single open path exercised here, so the mode is settled for // the same single open path exercised here, so the mode is settled for
// all three tiers in one place. // all three tiers in one place.
func TestArchiveDatabaseFilesAreOwnerOnly(t *testing.T) { func TestArchiveDatabaseFilesAreOwnerOnly(t *testing.T) {
+15 -4
View File
@@ -118,9 +118,8 @@ const SQLiteFilePerm fs.FileMode = 0o600
// SQLite still reports the absence rather than this function // SQLite still reports the absence rather than this function
// materializing an empty database the caller asked not to create. // materializing an empty database the caller asked not to create.
// //
// Chmod of a file that already exists is what tightens a data // The mode is set on every open, whoever created the file, so a
// directory an earlier build left at 0644 — including a developer's // database or sidecar found at a wider mode is tightened too.
// own scratch directory — without any migration machinery.
func reserveSQLiteFile(path string, create bool) error { func reserveSQLiteFile(path string, create bool) error {
if create { if create {
// gosec G304: the path is the database file the caller asked // gosec G304: the path is the database file the caller asked
@@ -242,7 +241,19 @@ func SQLiteDSN(path, mode string) string {
) )
q.Add("_pragma", "journal_mode(WAL)") q.Add("_pragma", "journal_mode(WAL)")
return "file:" + path + "?" + q.Encode() // SQLite reads a file: URI's path only up to the first ? or #, and
// decodes %XX escapes in it, so url.URL escapes the path. OmitHost
// keeps a relative path as "file:data/x.db"; written
// "file://data/x.db", SQLite would reject "data" as the URI's
// authority.
dsn := url.URL{
Scheme: "file",
Path: path,
OmitHost: true,
RawQuery: q.Encode(),
}
return dsn.String()
} }
// OpenSQLite opens the SQLite file at path with the service's // OpenSQLite opens the SQLite file at path with the service's
+75
View File
@@ -2,6 +2,8 @@ package database_test
import ( import (
"context" "context"
"database/sql"
"os"
"path/filepath" "path/filepath"
"strings" "strings"
"testing" "testing"
@@ -112,6 +114,79 @@ func TestMainDBAppliesPragmasOnALiveHandle(t *testing.T) {
assert.Equal(t, "10000", busy) assert.Equal(t, "10000", busy)
} }
// TestOpenSQLiteOpensThePathAskedFor uses a data directory whose name
// holds ?, # and %. SQLite reads a file: URI's path only up to a ? or
// #, and decodes %41 to A, so an unescaped path sent every database in
// that directory to one file outside it.
func TestOpenSQLiteOpensThePathAskedFor(t *testing.T) {
t.Parallel()
dir := filepath.Join(t.TempDir(), "hooks#1?x%41")
require.NoError(t, os.Mkdir(dir, 0o700))
for _, name := range []string{
database.MainDBFileName,
"events-" + uuid.New().String() + ".db",
} {
path := filepath.Join(dir, name)
sqlDB, err := database.OpenSQLite(
path, database.SQLiteModeCreate,
)
require.NoError(t, err)
t.Cleanup(func() { assert.NoError(t, sqlDB.Close()) })
assertOpenedFile(t, sqlDB, path)
}
}
// TestOpenSQLiteOpensARelativePath opens a database at a path relative
// to the working directory, which is what a relative DATA_DIR such as
// the README's DATA_DIR=../webhooker-data gives OpenSQLite.
//
//nolint:paralleltest // t.Chdir moves the whole process.
func TestOpenSQLiteOpensARelativePath(t *testing.T) {
t.Chdir(t.TempDir())
require.NoError(t, os.Mkdir("data", 0o700))
path := filepath.Join("data", database.MainDBFileName)
sqlDB, err := database.OpenSQLite(path, database.SQLiteModeCreate)
require.NoError(t, err)
defer func() { require.NoError(t, sqlDB.Close()) }()
assertOpenedFile(t, sqlDB, path)
}
// assertOpenedFile checks that sqlDB is the SQLite file at path.
// database_list names the file SQLite actually opened, as an absolute
// path with symlinks resolved, so the files are compared, not the names.
func assertOpenedFile(t *testing.T, sqlDB *sql.DB, path string) {
t.Helper()
var (
seq int
schema, file string
)
require.NoError(t, sqlDB.
QueryRowContext(t.Context(), "pragma database_list").
Scan(&seq, &schema, &file))
asked, err := os.Stat(path)
require.NoError(t, err)
opened, err := os.Stat(file)
require.NoError(t, err)
assert.True(
t, os.SameFile(asked, opened),
"asked for %s, SQLite opened %s", path, file,
)
}
// TestConcurrentReaderDoesNotBlockWrites is the unit-scale form of the // TestConcurrentReaderDoesNotBlockWrites is the unit-scale form of the
// reproduction in // reproduction in
// https://git.eeqj.de/sneak/webhooker/issues/256: an operator's // https://git.eeqj.de/sneak/webhooker/issues/256: an operator's
@@ -1,5 +1,5 @@
-- Migration 001: the schema of a database target's archive file. The -- Migration 001: the schema of a database target's archive file. The
-- archivedEvent model in internal/delivery must agree with it. -- archivedEvent model in internal/archive must agree with it.
CREATE TABLE archived_events ( CREATE TABLE archived_events (
id integer PRIMARY KEY AUTOINCREMENT, id integer PRIMARY KEY AUTOINCREMENT,
+10 -4
View File
@@ -46,7 +46,7 @@ CREATE TABLE webhooks (
user_id uuid NOT NULL, user_id uuid NOT NULL,
name text NOT NULL, name text NOT NULL,
description text, description text,
retention_days integer DEFAULT 30, retention_days integer,
PRIMARY KEY (id), PRIMARY KEY (id),
CONSTRAINT fk_users_webhooks FOREIGN KEY (user_id) REFERENCES users (id) CONSTRAINT fk_users_webhooks FOREIGN KEY (user_id) REFERENCES users (id)
); );
@@ -79,11 +79,17 @@ CREATE TABLE targets (
name text NOT NULL, name text NOT NULL,
type text NOT NULL, type text NOT NULL,
active numeric DEFAULT true, active numeric DEFAULT true,
config text, url text,
max_retries integer, headers text,
timeout_seconds integer,
forward_query numeric,
archive_expiry integer,
archive_rotation text,
max_attempts integer NOT NULL,
PRIMARY KEY (id), PRIMARY KEY (id),
CONSTRAINT fk_webhooks_targets CONSTRAINT fk_webhooks_targets
FOREIGN KEY (webhook_id) REFERENCES webhooks (id) FOREIGN KEY (webhook_id) REFERENCES webhooks (id),
CONSTRAINT chk_targets_max_attempts CHECK (max_attempts >= 1)
); );
CREATE INDEX idx_targets_deleted_at ON targets (deleted_at); CREATE INDEX idx_targets_deleted_at ON targets (deleted_at);
+3 -19
View File
@@ -46,12 +46,7 @@ func TestClientForRequest_TimeoutKeepsSSRFGuard(t *testing.T) {
t.Run(target, func(t *testing.T) { t.Run(target, func(t *testing.T) {
t.Parallel() t.Parallel()
cfg := &delivery.HTTPTargetConfig{ client := engine.ExportClientForRequest(5, nil)
URL: target,
Timeout: 5,
}
client := engine.ExportClientForRequest(cfg, nil)
require.NotSame(t, engine.ExportClient(), client, require.NotSame(t, engine.ExportClient(), client,
"a per-target timeout must yield a "+ "a per-target timeout must yield a "+
@@ -101,11 +96,7 @@ func TestClientForRequest_NoTimeoutUnchanged(t *testing.T) {
engine := newSSRFTestEngine() engine := newSSRFTestEngine()
cfg := &delivery.HTTPTargetConfig{ client := engine.ExportClientForRequest(0, nil)
URL: "https://example.com/hook",
}
client := engine.ExportClientForRequest(cfg, nil)
assert.Same(t, engine.ExportClient(), client, assert.Same(t, engine.ExportClient(), client,
"without a per-target timeout the shared client "+ "without a per-target timeout the shared client "+
@@ -123,15 +114,8 @@ func TestClientForRequest_HeadersKeepSSRFGuard(t *testing.T) {
engine := newSSRFTestEngine() engine := newSSRFTestEngine()
cfg := &delivery.HTTPTargetConfig{
URL: "https://example.com/with-headers",
Headers: map[string]string{
"X-Api-Key": "configured",
},
}
client := engine.ExportClientForRequest( client := engine.ExportClientForRequest(
cfg, []string{"X-Api-Key"}, 0, []string{probeHeaderName},
) )
require.NotNil(t, client.CheckRedirect, require.NotNil(t, client.CheckRedirect,
+13 -2
View File
@@ -32,8 +32,9 @@ func targetTotals(
// TestUpdateDeliveryStatus_FinishTimeAndTargetTotals pins what a status // TestUpdateDeliveryStatus_FinishTimeAndTargetTotals pins what a status
// write records for the webhook page's statistics: the time a delivery // write records for the webhook page's statistics: the time a delivery
// finished, set only when it becomes delivered or failed, and one more // finished, set only when it becomes delivered or failed and stored in
// on its target's delivered or failed total. // the same form as created_at, and one more on its target's delivered
// or failed total.
func TestUpdateDeliveryStatus_FinishTimeAndTargetTotals(t *testing.T) { func TestUpdateDeliveryStatus_FinishTimeAndTargetTotals(t *testing.T) {
t.Parallel() t.Parallel()
@@ -75,6 +76,16 @@ func TestUpdateDeliveryStatus_FinishTimeAndTargetTotals(t *testing.T) {
if tt.finished { if tt.finished {
require.NotNil(t, stored.FinishedAt) require.NotNil(t, stored.FinishedAt)
assert.False(t, stored.FinishedAt.Before(before)) assert.False(t, stored.FinishedAt.Before(before))
var createdAt, finishedAt string
require.NoError(t, db.Raw(
`SELECT CAST(created_at AS TEXT),
CAST(finished_at AS TEXT)
FROM deliveries WHERE id = ?`, d.ID,
).Row().Scan(&createdAt, &finishedAt))
assertStoredTime(t, createdAt)
assertStoredTime(t, finishedAt)
} else { } else {
assert.Nil(t, stored.FinishedAt) assert.Nil(t, stored.FinishedAt)
} }
+220 -472
View File
@@ -13,6 +13,7 @@ import (
"go.uber.org/fx" "go.uber.org/fx"
"gorm.io/gorm" "gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/globals" "sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/lifecycle" "sneak.berlin/go/webhooker/internal/lifecycle"
@@ -39,9 +40,10 @@ const (
// explosions regardless of queue depth. // explosions regardless of queue depth.
defaultWorkers = 10 defaultWorkers = 10
// retrySweepInterval is how often the periodic retry // sweepInterval is how often the recovery routine runs
// sweep runs. // once it has run at start. Each of those runs is a tick of
retrySweepInterval = 60 * time.Second // the sweep.
sweepInterval = 60 * time.Second
// pendingSweepMinAge is how long a delivery must have sat // pendingSweepMinAge is how long a delivery must have sat
// untouched at pending before the sweep will look at it. // untouched at pending before the sweep will look at it.
@@ -62,10 +64,10 @@ const (
pendingSweepMinAge = 15 * time.Minute pendingSweepMinAge = 15 * time.Minute
// pendingSweepBatch bounds how many stranded pending deliveries // pendingSweepBatch bounds how many stranded pending deliveries
// one sweep of one webhook re-dispatches. The sweep runs every // one tick of the sweep re-dispatches for one webhook. The sweep
// retrySweepInterval, so a larger backlog drains across // ticks every sweepInterval, so a larger backlog drains
// successive sweeps instead of arriving as one burst against a // across successive ticks instead of arriving as one burst against
// database that was already struggling to accept writes. // a database that was already struggling to accept writes.
pendingSweepBatch = 500 pendingSweepBatch = 500
// MaxInlineBodySize is the maximum event body size that // MaxInlineBodySize is the maximum event body size that
@@ -103,12 +105,19 @@ type Task struct {
WebhookID string WebhookID string
EntrypointID string EntrypointID string
TargetID string // The target's fields, as they were when the task was made.
TargetName string TargetID string
TargetType database.TargetType TargetName string
TargetConfig string TargetType database.TargetType
MaxRetries int TargetURL string
TargetHeaders map[string]string
TimeoutSeconds int
ForwardQuery bool
ArchiveExpiry time.Duration
ArchiveRotation string
MaxAttempts int
// The event's fields.
Method string Method string
RawQuery string RawQuery string
Headers string Headers string
@@ -124,31 +133,11 @@ type Notifier interface {
Notify(tasks []Task) Notify(tasks []Task)
} }
// Archives is how the handlers keep the database targets' archive
// files in step with the configuration. Deleting a webhook or a
// target releases the cached archive writers, whose open file
// handles would otherwise outlive them; renaming one renames the
// archive files, which are named for the webhook and the target
// (see ArchiveFileName).
//
// It is deliberately separate from Notifier: archiving lifecycle
// is not notification, and a small interface keeps the handlers
// package free of any dependency on the engine's internals while
// staying trivially fakeable in tests.
//
// Neither eviction deletes an archive file. Both are idempotent
// and are no-ops for a webhook or target with no engine state.
type Archives interface {
EvictWebhook(webhookID string)
EvictTarget(targetID string)
Rename(targetID, webhookName, targetName string) error
}
// CircuitBreakers is how the handlers read a target's circuit // CircuitBreakers is how the handlers read a target's circuit
// breaker, so the webhook page and the event log can say that // breaker, so the webhook page and the event log can say that
// deliveries to the target are paused and until when. Like Archives, // deliveries to the target are paused and until when. It keeps the
// it keeps the handlers free of the engine's internals and is // handlers free of the engine's internals and is trivially faked in
// trivially faked in tests. // tests.
type CircuitBreakers interface { type CircuitBreakers interface {
StateAndCooldown(targetID string) (CircuitState, time.Duration) StateAndCooldown(targetID string) (CircuitState, time.Duration)
} }
@@ -164,12 +153,13 @@ type EngineParams struct {
Logger *logger.Logger Logger *logger.Logger
SSRFGuard *Guard SSRFGuard *Guard
Metrics *metrics.Set Metrics *metrics.Set
Archives *archive.Writers
} }
// Engine processes queued deliveries in the background // Engine processes queued deliveries in the background
// using a bounded worker pool architecture. It owns only // using a bounded worker pool architecture. It owns only
// the cross-target machinery: the worker pool, the queue // the cross-target machinery: the worker pool, the queue
// and retry channels, restart recovery, and the persistence // and retry channels, the recovery routine, and the persistence
// helpers and Scheduler that individual targets rely on. // helpers and Scheduler that individual targets rely on.
// Each target type owns its own delivery, including retries, // Each target type owns its own delivery, including retries,
// backoff, and circuit breaking. // backoff, and circuit breaking.
@@ -202,12 +192,13 @@ type Engine struct {
httpTarget *httpTarget httpTarget *httpTarget
slackTarget *slackTarget slackTarget *slackTarget
// dbTarget is retained so the engine can reach the archive // archives is the database targets' archive writers. The
// writer registry for eviction, renames and the idle sweep. // database target writes each event through them, and stop
dbTarget *databaseTarget // closes them.
archives *archive.Writers
// inflight is the set of deliveries this engine currently owns. // inflight is the set of deliveries this engine currently owns.
// Recovery and the sweeps re-dispatch only what it does not // The recovery routine re-dispatches only what it does not
// hold. Held by value: its zero value works, so no constructor // hold. Held by value: its zero value works, so no constructor
// can leave it out. See inflight.go. // can leave it out. See inflight.go.
inflight inflightSet inflight inflightSet
@@ -228,6 +219,7 @@ func New(
workers: defaultWorkers, workers: defaultWorkers,
version: params.Globals.Version, version: params.Globals.Version,
mtr: params.Metrics, mtr: params.Metrics,
archives: params.Archives,
} }
e.initTargets(&http.Client{ e.initTargets(&http.Client{
@@ -272,47 +264,6 @@ func (e *Engine) Notify(tasks []Task) {
} }
} }
// EvictWebhook implements Archives. The cached archive writer of
// every database target of the webhook is dropped from the
// registry and its file handle closed. The archive files
// themselves are left on disk — they are long-term storage the
// operator owns.
func (e *Engine) EvictWebhook(webhookID string) {
if e.dbTarget == nil {
return
}
e.dbTarget.evictWebhook(webhookID)
}
// EvictTarget implements Archives. It is EvictWebhook for a single
// database target, and leaves the archive file on disk the same
// way.
func (e *Engine) EvictTarget(targetID string) {
if e.dbTarget == nil {
return
}
e.dbTarget.evict(targetID)
}
// Rename implements Archives. It renames every one of a database
// target's archive files to ArchiveFileName(webhookName, targetName,
// targetID), each keeping the period in its name, under the lock the
// target's archive writes and the idle sweep take. It never replaces
// a file: if one already has a new name, the error is
// ErrArchiveNameTaken. The caller renames before it saves the new
// name: see databaseTarget.rename.
func (e *Engine) Rename(
targetID, webhookName, targetName string,
) error {
if e.dbTarget == nil {
return nil
}
return e.dbTarget.rename(targetID, webhookName, targetName)
}
// StateAndCooldown implements CircuitBreakers. It returns the state of // StateAndCooldown implements CircuitBreakers. It returns the state of
// the target's circuit breaker and, while the breaker is open, what is // the target's circuit breaker and, while the breaker is open, what is
// left of its cooldown; the cooldown is zero once that has passed and // left of its cooldown; the cooldown is zero once that has passed and
@@ -399,8 +350,8 @@ func (e *Engine) registerHooks(lc fx.Lifecycle) {
}) })
} }
// start launches the worker pool, restart recovery, and the // start launches the worker pool and the recovery routine, which
// periodic retry sweep. // runs now and then on every tick of the sweep (see runRecovery).
// //
// Their context is derived from context.Background(), NOT from // Their context is derived from context.Background(), NOT from
// the fx OnStart hook context. The hook context carries fx's // the fx OnStart hook context. The hook context carries fx's
@@ -423,11 +374,7 @@ func (e *Engine) start() {
e.wg.Add(1) e.wg.Add(1)
go e.recoverPending(ctx) go e.runRecovery(ctx)
e.wg.Add(1)
go e.retrySweep(ctx)
e.wg.Add(1) e.wg.Add(1)
@@ -466,7 +413,7 @@ func (e *Engine) stop(ctx context.Context) error {
return err return err
} }
e.dbTarget.evictAll() e.archives.EvictAll()
e.log.Info("delivery engine stopped") e.log.Info("delivery engine stopped")
@@ -510,12 +457,6 @@ func (e *Engine) runTask(
run(ctx, task) run(ctx, task)
} }
func (e *Engine) recoverPending(ctx context.Context) {
defer e.wg.Done()
e.recoverInFlight(ctx)
}
func (e *Engine) processNewTask( func (e *Engine) processNewTask(
ctx context.Context, task *Task, ctx context.Context, task *Task,
) { ) {
@@ -530,9 +471,10 @@ func (e *Engine) processNewTask(
return return
} }
// Restart recovery can send and release this delivery before the // The recovery routine's run at start can send and release this
// receiver's Notify queues it. Ownership cannot refuse a delivery // delivery before the receiver's Notify queues it. Ownership cannot
// nobody holds, so the row decides whether it still needs sending. // refuse a delivery nobody holds, so the row decides whether it
// still needs sending.
row, err := e.loadDelivery(webhookDB, task.DeliveryID) row, err := e.loadDelivery(webhookDB, task.DeliveryID)
if err != nil { if err != nil {
e.log.Error( e.log.Error(
@@ -662,9 +604,9 @@ func (e *Engine) processRetryTask(
// else on this path reads the target row. Without this check a // else on this path reads the target row. Without this check a
// deletion stops nothing: the timer keeps firing and keeps sending to // deletion stops nothing: the timer keeps firing and keeps sending to
// the destination the operator removed, for the whole remaining // the destination the operator removed, for the whole remaining
// backoff chain. Terminalising in the recovery and sweep paths alone // backoff chain. Terminalising in the recovery routine alone is not
// is not enough, because those only see the delivery once nothing // enough, because it only sees the delivery once nothing holds it in
// holds it in memory — which is to say after a restart. // memory — which is to say after a restart.
// //
// The worker already owns this delivery, so the terminal write happens // The worker already owns this delivery, so the terminal write happens
// here directly, exactly as a target's own Deliver fails one. Claiming // here directly, exactly as a target's own Deliver fails one. Claiming
@@ -712,7 +654,47 @@ func (e *Engine) abandonRetryForMissingTarget(
return true return true
} }
func (e *Engine) recoverInFlight(ctx context.Context) { // runRecovery runs the recovery routine, recoverDeliveries, once when
// the engine starts and then on every tick of the sweep, every
// sweepInterval, until the engine stops.
func (e *Engine) runRecovery(ctx context.Context) {
defer e.wg.Done()
ticker := time.NewTicker(sweepInterval)
defer ticker.Stop()
e.recoverDeliveries(ctx, true)
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
e.recoverDeliveries(ctx, false)
}
}
}
// recoverDeliveries is the engine's one recovery routine: atStart is
// true for its run when the engine starts and false on a tick of the
// sweep.
//
// For every webhook it finds the pending and retrying deliveries the
// engine is not working on, such as those a stop or a crash cut off, a
// full channel dropped, or a failed bookkeeping write left behind. One
// that already has a successful attempt is marked delivered. A retrying
// one is scheduled after what is left of its backoff, or now when
// nothing is left. A pending one goes to a worker now.
//
// Ownership keeps it off a delivery the engine is working on, has
// queued, or holds a retry timer for, the same at start as on a tick:
// every delivery it acts on is taken through the inflight set first.
// See inflight.go.
//
// A run at start differs from a tick in which webhooks it opens (see
// recoverWebhookDeliveries) and which pending deliveries it takes (see
// recoverPendingDeliveries).
func (e *Engine) recoverDeliveries(ctx context.Context, atStart bool) {
var webhookIDs []string var webhookIDs []string
err := e.database.DB(). err := e.database.DB().
@@ -734,18 +716,27 @@ func (e *Engine) recoverInFlight(ctx context.Context) {
default: default:
} }
// Opened even when its file is missing, so that a lost e.recoverWebhookDeliveries(ctx, webhookID, atStart)
// database is reported at start, not when the webhook next
// receives an event, which for a quiet webhook may be never.
e.recoverWebhookDeliveries(ctx, webhookID)
} }
} }
// recoverWebhookDeliveries runs the recovery routine over one webhook.
//
// At start the webhook's database is opened even when its file is
// missing, so that a lost database is reported at start, not when the
// webhook next receives an event, which for a quiet webhook may be
// never. A tick leaves a missing file alone rather than create one: a
// webhook being created has its row saved a moment before its file,
// and a tick in between would report that file as lost.
func (e *Engine) recoverWebhookDeliveries( func (e *Engine) recoverWebhookDeliveries(
ctx context.Context, webhookID string, ctx context.Context, webhookID string, atStart bool,
) { ) {
// The web interface is already serving, so the webhook may have if !atStart && !e.dbManager.DBExists(webhookID) {
// been deleted since the list was read. Opening its database then return
}
// The web interface is serving, so the webhook may have been
// deleted since the list was read. Opening its database then
// would create the file again after the delete removed it. // would create the file again after the delete removed it.
stillExists := func() (bool, error) { stillExists := func() (bool, error) {
var count int64 var count int64
@@ -777,16 +768,18 @@ func (e *Engine) recoverWebhookDeliveries(
} }
e.recoverPendingDeliveries( e.recoverPendingDeliveries(
ctx, webhookDB, webhookID, ctx, webhookDB, webhookID, atStart,
) )
e.recoverRetryingDeliveries( e.recoverRetryingDeliveries(
webhookDB, webhookID, ctx, webhookDB, webhookID,
) )
} }
// recoverRetryingDeliveries recovers every retrying delivery of one
// webhook, at start and on a tick alike.
func (e *Engine) recoverRetryingDeliveries( func (e *Engine) recoverRetryingDeliveries(
webhookDB *gorm.DB, webhookID string, ctx context.Context, webhookDB *gorm.DB, webhookID string,
) { ) {
var retrying []database.Delivery var retrying []database.Delivery
@@ -813,6 +806,12 @@ func (e *Engine) recoverRetryingDeliveries(
) )
for i := range retrying { for i := range retrying {
select {
case <-ctx.Done():
return
default:
}
if _, ok := settled[retrying[i].ID]; ok { if _, ok := settled[retrying[i].ID]; ok {
continue continue
} }
@@ -824,8 +823,9 @@ func (e *Engine) recoverRetryingDeliveries(
} }
// recoverSingleRetry hands an orphaned retrying delivery back // recoverSingleRetry hands an orphaned retrying delivery back
// to its target to recompute the remaining backoff, then // to its target to work out what is left of its backoff, then
// reschedules it. Targets that do not own durable retries // schedules its next attempt after that, or now when nothing is
// left. Targets that do not own durable retries
// (fire-and-forget) never produce retrying deliveries, so a // (fire-and-forget) never produce retrying deliveries, so a
// delivery found in that state has had its target's type // delivery found in that state has had its target's type
// changed underneath it and is terminally failed. // changed underneath it and is terminally failed.
@@ -873,6 +873,19 @@ func (e *Engine) recoverSingleRetry(
webhookDB, d.ID, attemptNum, webhookDB, d.ID, attemptNum,
) )
// Taken before the event is read, so that a delivery the engine
// already holds, most often one waiting out its backoff on a
// timer, costs a tick no read of its body.
if !e.takeForRedispatch(
webhookDB, d.ID, database.DeliveryStatusRetrying,
) {
return
}
// ScheduleRetry takes a reference of its own before it returns,
// and that one keeps the delivery owned through the backoff.
defer e.inflight.release(d.ID)
event, err := e.loadEvent(webhookDB, d.EventID) event, err := e.loadEvent(webhookDB, d.EventID)
if err != nil { if err != nil {
e.log.Error( e.log.Error(
@@ -886,13 +899,9 @@ func (e *Engine) recoverSingleRetry(
return return
} }
task := buildRecoveryTask( e.ScheduleRetry(buildRecoveryTask(
d, webhookID, &event, &target, attemptNum+1, d, webhookID, &event, &target, attemptNum+1,
) ), remaining)
if !e.rescheduleRecovered(webhookDB, task, remaining) {
return
}
e.log.Info( e.log.Info(
"recovering retrying delivery", "recovering retrying delivery",
@@ -903,28 +912,54 @@ func (e *Engine) recoverSingleRetry(
) )
} }
// recoverPendingDeliveries recovers the pending deliveries of one
// webhook: it marks delivered those that already have a successful
// attempt and sends the rest.
//
// A delivery is created pending and leaves that state only when its
// outcome is written, so a pending row the engine does not own is one
// it stopped working on before then: for example one whose bookkeeping
// write failed, whose attempt a stop cut off, that was still queued
// when the engine last stopped, whose task was dropped because the
// delivery channel was full (by Notify or by queueRecovered), or whose
// worker could not open the webhook's database.
//
// At start every pending delivery is taken, however recent, so one a
// stop cut off moments ago is sent again now. A tick takes only those
// untouched for pendingSweepMinAge, at most pendingSweepBatch of them.
// The age bound is not what keeps a tick off live work: a delivery
// waiting in deliveryCh is pending and arbitrarily old. Ownership is,
// checked per delivery in takeForRedispatch. See inflight.go.
func (e *Engine) recoverPendingDeliveries( func (e *Engine) recoverPendingDeliveries(
ctx context.Context, ctx context.Context,
webhookDB *gorm.DB, webhookDB *gorm.DB,
webhookID string, webhookID string,
atStart bool,
) { ) {
query := webhookDB.Where(
"status = ?", database.DeliveryStatusPending,
)
if !atStart {
query = query.
Where(
"updated_at < ?",
time.Now().Add(-pendingSweepMinAge),
).
Limit(pendingSweepBatch)
}
var deliveries []database.Delivery var deliveries []database.Delivery
// No Preload: event bodies are read one at a time in // No Preload: event bodies are read one at a time in
// sendRecoveredDeliveries, and only for the deliveries actually // sendRecoveredDeliveries, and only for the deliveries actually
// being sent. // being sent.
result := webhookDB. err := query.Find(&deliveries).Error
Where( if err != nil {
"status = ?",
database.DeliveryStatusPending,
).
Find(&deliveries)
if result.Error != nil {
e.log.Error( e.log.Error(
"failed to query pending deliveries", "failed to query pending deliveries for recovery",
"webhook_id", webhookID, "webhook_id", webhookID,
"error", result.Error, "error", err,
) )
return return
@@ -940,21 +975,6 @@ func (e *Engine) recoverPendingDeliveries(
"count", len(deliveries), "count", len(deliveries),
) )
e.recoverPendingBatch(
ctx, webhookDB, webhookID, deliveries,
)
}
// recoverPendingBatch settles every delivery in the batch that was
// already delivered, and re-dispatches only the rest. Both the
// restart-time recovery and the periodic sweep go through it, so a
// pending delivery is treated the same however it was found.
func (e *Engine) recoverPendingBatch(
ctx context.Context,
webhookDB *gorm.DB,
webhookID string,
deliveries []database.Delivery,
) {
targetMap := e.loadTargetMap(deliveries) targetMap := e.loadTargetMap(deliveries)
settled := e.reconcileDelivered( settled := e.reconcileDelivered(
@@ -982,11 +1002,11 @@ func (e *Engine) recoverPendingBatch(
// wire I/O happened and was recorded, and all that is missing is the // wire I/O happened and was recorded, and all that is missing is the
// status. // status.
// //
// Every recovery path runs this, not only the pending one. A delivery // The recovery routine runs this for retrying deliveries as well as
// abandoned at retrying can hold a successful result just as a pending // pending ones. A delivery abandoned at retrying can hold a successful
// one can — a second attempt that reached the receiver and whose status // result just as a pending one can — a second attempt that reached the
// write then failed sits at retrying with success recorded — and // receiver and whose status write then failed sits at retrying with
// re-sending it is the same duplicate. // success recorded — and re-sending it is the same duplicate.
// //
// Deliveries whose result row itself never landed are not in the // Deliveries whose result row itself never landed are not in the
// returned set and are re-sent, recorded as the further attempt they // returned set and are re-sent, recorded as the further attempt they
@@ -1073,247 +1093,8 @@ func (e *Engine) reconcileDelivered(
return settled return settled
} }
func (e *Engine) retrySweep(ctx context.Context) {
defer e.wg.Done()
ticker := time.NewTicker(retrySweepInterval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
e.sweepOrphanedRetries(ctx)
}
}
}
func (e *Engine) sweepOrphanedRetries(
ctx context.Context,
) {
var webhookIDs []string
err := e.database.DB().
Model(&database.Webhook{}).
Pluck("id", &webhookIDs).Error
if err != nil {
e.log.Error(
"retry sweep: failed to query webhook IDs",
"error", err,
)
return
}
for _, webhookID := range webhookIDs {
select {
case <-ctx.Done():
return
default:
}
if !e.dbManager.DBExists(webhookID) {
continue
}
e.sweepWebhookRetries(ctx, webhookID)
}
}
func (e *Engine) sweepWebhookRetries(
ctx context.Context, webhookID string,
) {
webhookDB, err := e.dbManager.GetDB(webhookID)
if err != nil {
e.log.Error(
"retry sweep: failed to get webhook database",
"webhook_id", webhookID,
"error", err,
)
return
}
var retrying []database.Delivery
err = webhookDB.
Where(
"status = ?",
database.DeliveryStatusRetrying,
).
Find(&retrying).Error
if err != nil {
e.log.Error(
"retry sweep: "+
"failed to query retrying deliveries",
"webhook_id", webhookID,
"error", err,
)
return
}
settled := e.reconcileDelivered(
webhookDB, webhookID, retrying,
e.loadTargetMap(retrying),
)
for i := range retrying {
select {
case <-ctx.Done():
return
default:
}
if _, ok := settled[retrying[i].ID]; ok {
continue
}
e.sweepSingleRetry(
webhookDB, webhookID, &retrying[i],
)
}
e.sweepWebhookPending(ctx, webhookDB, webhookID)
}
// sweepWebhookPending recovers deliveries stranded at pending.
//
// A delivery is created pending and leaves that state only when its
// outcome is written, so a pending row the engine does not own is one
// whose bookkeeping write failed — the state that used to sit there
// until a restart, and then produce a duplicate at the receiver. The
// sweep gives it the same reconcile-then-dispatch treatment restart
// recovery gets, so it costs a minute rather than an operator
// noticing.
//
// What keeps the sweep off live work is ownership, checked per
// delivery in takeForRedispatch, not the age bound in this query.
// A delivery waiting in deliveryCh is pending and arbitrarily old —
// the channel holds 10000 tasks and 10 workers drain it — so
// reasoning from the row's age alone re-sends it. See inflight.go.
func (e *Engine) sweepWebhookPending(
ctx context.Context,
webhookDB *gorm.DB,
webhookID string,
) {
var pending []database.Delivery
err := webhookDB.
Where(
"status = ? AND updated_at < ?",
database.DeliveryStatusPending,
time.Now().Add(-pendingSweepMinAge),
).
Limit(pendingSweepBatch).
Find(&pending).Error
if err != nil {
e.log.Error(
"retry sweep: "+
"failed to query pending deliveries",
"webhook_id", webhookID,
"error", err,
)
return
}
if len(pending) == 0 {
return
}
e.log.Info(
"retry sweep: recovering stranded pending deliveries",
"webhook_id", webhookID,
"count", len(pending),
)
e.recoverPendingBatch(ctx, webhookDB, webhookID, pending)
}
// sweepSingleRetry re-enqueues an orphaned retrying delivery
// whose backoff window has elapsed, delegating the backoff
// decision to the delivery's target. A delivery whose target
// no longer owns durable retries is terminally failed.
func (e *Engine) sweepSingleRetry(
webhookDB *gorm.DB,
webhookID string,
d *database.Delivery,
) {
target, err := e.loadTarget(d.TargetID)
if err != nil {
// Deleted is terminal, unreadable is not; see
// recoverSingleRetry.
if errors.Is(err, gorm.ErrRecordNotFound) {
e.failMissingTarget(webhookDB, webhookID, d)
return
}
e.log.Error(
"retry sweep: failed to load target",
"delivery_id", d.ID,
"target_id", d.TargetID,
"error", err,
)
return
}
rs, ok := e.targets[target.Type].(rescheduler)
if !ok {
e.failUnretryableRetry(
webhookDB, webhookID, d, &target,
)
return
}
attemptNum := e.countAttempts(webhookDB, d.ID)
if !rs.backoffElapsed(
webhookDB, d.ID, attemptNum,
) {
return
}
event, err := e.loadEvent(webhookDB, d.EventID)
if err != nil {
e.log.Error(
"retry sweep: failed to load event",
"delivery_id", d.ID,
"event_id", d.EventID,
"error", err,
)
return
}
task := buildRecoveryTask(
d, webhookID, &event, &target, attemptNum+1,
)
if !e.redispatch(
e.retryCh, webhookDB, task,
database.DeliveryStatusRetrying,
) {
return
}
e.log.Info(
"retry sweep: "+
"recovered orphaned retrying delivery",
"delivery_id", d.ID,
"webhook_id", webhookID,
"attempt", attemptNum+1,
)
}
// failUnretryableRetry terminally fails an orphaned retrying // failUnretryableRetry terminally fails an orphaned retrying
// delivery whose target type no longer supports retries. Both // delivery whose target type no longer supports retries.
// restart recovery and the periodic sweep call it, so the
// terminal transition exists once.
// //
// This is only reachable when a target's type has been changed // This is only reachable when a target's type has been changed
// out from under an in-flight retrying delivery (or the type is // out from under an in-flight retrying delivery (or the type is
@@ -1361,8 +1142,8 @@ func (e *Engine) failUnretryableRetry(
} }
// failMissingTarget terminally fails a recovered delivery, pending or // failMissingTarget terminally fails a recovered delivery, pending or
// retrying, whose target row is gone. Restart recovery and the periodic // retrying, whose target row is gone. The recovery routine calls it for
// sweep call it for both statuses, so the transition exists once. // both statuses.
// //
// Until it existed those paths logged the failed lookup and moved on, // Until it existed those paths logged the failed lookup and moved on,
// which left the delivery where it was for the life of the database and // which left the delivery where it was for the life of the database and
@@ -1701,8 +1482,11 @@ func writeDeliveryStatus(
database.DeliveryStatusFailed, database.DeliveryStatusFailed,
}). }).
Updates(map[string]any{ Updates(map[string]any{
"status": status, "status": status,
"finished_at": time.Now(), // GORM's clock, the one that stamps created_at. A bare
// time.Now() carries a monotonic clock reading, which the
// driver would store as an " m=+…" suffix.
"finished_at": tx.NowFunc(),
}) })
if res.Error != nil || res.RowsAffected == 0 { if res.Error != nil || res.RowsAffected == 0 {
return res.Error return res.Error
@@ -1766,10 +1550,15 @@ func buildEventFromTask(task *Task) database.Event {
func buildTargetFromTask(task *Task) database.Target { func buildTargetFromTask(task *Task) database.Target {
target := database.Target{ target := database.Target{
Name: task.TargetName, Name: task.TargetName,
Type: task.TargetType, Type: task.TargetType,
Config: task.TargetConfig, URL: task.TargetURL,
MaxRetries: task.MaxRetries, Headers: task.TargetHeaders,
TimeoutSeconds: task.TimeoutSeconds,
ForwardQuery: task.ForwardQuery,
ArchiveExpiry: task.ArchiveExpiry,
ArchiveRotation: task.ArchiveRotation,
MaxAttempts: task.MaxAttempts,
} }
target.ID = task.TargetID target.ID = task.TargetID
@@ -1884,11 +1673,11 @@ func (e *Engine) countAttempts(
// answer cannot go stale between asking and acting. // answer cannot go stale between asking and acting.
// //
// Stamping updated_at is the same statement, and it is a cadence // Stamping updated_at is the same statement, and it is a cadence
// control rather than a claim: the pending sweep selects on that // control rather than a claim: a tick of the sweep selects pending
// column, so a delivery handed out now is not selected again on the // deliveries on that column, so one handed out now is not selected
// next tick a minute later but after pendingSweepMinAge. A delivery // again on the next tick a minute later but after pendingSweepMinAge.
// the database refuses to settle is therefore retried on that // A delivery the database refuses to settle is therefore retried on
// interval instead of every tick. // that interval instead of every tick.
// //
// A failed write is a refusal. It means the database is not accepting // A failed write is a refusal. It means the database is not accepting
// writes, which is the condition that stranded the delivery in the // writes, which is the condition that stranded the delivery in the
@@ -1906,12 +1695,13 @@ func (e *Engine) takeForRedispatch(
return false return false
} }
// GORM's clock, as writeDeliveryStatus explains.
res := webhookDB. res := webhookDB.
Model(&database.Delivery{}). Model(&database.Delivery{}).
Where( Where(
"id = ? AND status = ?", deliveryID, eligible, "id = ? AND status = ?", deliveryID, eligible,
). ).
UpdateColumn("updated_at", time.Now()) UpdateColumn("updated_at", webhookDB.NowFunc())
if res.Error != nil { if res.Error != nil {
e.log.Error( e.log.Error(
@@ -1935,14 +1725,11 @@ func (e *Engine) takeForRedispatch(
return true return true
} }
// queueRecovered puts an owned delivery's task on a worker channel, // queueRecovered puts an owned pending delivery's task on the delivery
// dropping the ownership the gate took if it does not fit. // channel, dropping the ownership the gate took if it does not fit.
func (e *Engine) queueRecovered( func (e *Engine) queueRecovered(task Task) {
ch chan<- Task, task Task,
) bool {
select { select {
case ch <- task: case e.deliveryCh <- task:
return true
default: default:
e.inflight.release(task.DeliveryID) e.inflight.release(task.DeliveryID)
e.log.Warn( e.log.Warn(
@@ -1951,53 +1738,9 @@ func (e *Engine) queueRecovered(
"delivery_id", task.DeliveryID, "delivery_id", task.DeliveryID,
"webhook_id", task.WebhookID, "webhook_id", task.WebhookID,
) )
return false
} }
} }
// redispatch hands a recovered delivery to a worker channel through
// takeForRedispatch, and reports whether the task was queued.
func (e *Engine) redispatch(
ch chan<- Task,
webhookDB *gorm.DB,
task Task,
eligible database.DeliveryStatus,
) bool {
if !e.takeForRedispatch(
webhookDB, task.DeliveryID, eligible,
) {
return false
}
return e.queueRecovered(ch, task)
}
// rescheduleRecovered hands an orphaned retrying delivery back to the
// retry timer, through the same gate. It reports whether the delivery
// was rescheduled.
//
// The reference taken by the gate is dropped as soon as ScheduleRetry
// has taken its own, which it does before returning: what keeps the
// delivery owned through the backoff window is ScheduleRetry's
// reference, not this one.
func (e *Engine) rescheduleRecovered(
webhookDB *gorm.DB, task Task, delay time.Duration,
) bool {
if !e.takeForRedispatch(
webhookDB, task.DeliveryID,
database.DeliveryStatusRetrying,
) {
return false
}
defer e.inflight.release(task.DeliveryID)
e.ScheduleRetry(task, delay)
return true
}
// countAttemptsBatch counts the recorded attempts of every delivery // countAttemptsBatch counts the recorded attempts of every delivery
// in a batch with one grouped query, keyed by delivery id. Deliveries // in a batch with one grouped query, keyed by delivery id. Deliveries
// with no attempts are simply absent from the result, which reads back // with no attempts are simply absent from the result, which reads back
@@ -2024,7 +1767,7 @@ func (e *Engine) countAttemptsBatch(
// One delivery id per recorded attempt, tallied here rather than // One delivery id per recorded attempt, tallied here rather than
// grouped in SQL: internal/gormlog forbids (*gorm.DB).Scan, which // grouped in SQL: internal/gormlog forbids (*gorm.DB).Scan, which
// a GROUP BY into a struct would need, and an attempt row per // a GROUP BY into a struct would need, and an attempt row per
// delivery is bounded by the target's MaxRetries. // delivery is bounded by the target's MaxAttempts.
var attemptIDs []string var attemptIDs []string
err := webhookDB. err := webhookDB.
@@ -2094,21 +1837,26 @@ func buildRecoveryTask(
} }
return Task{ return Task{
DeliveryID: d.ID, DeliveryID: d.ID,
EventID: d.EventID, EventID: d.EventID,
WebhookID: webhookID, WebhookID: webhookID,
EntrypointID: event.EntrypointID, EntrypointID: event.EntrypointID,
TargetID: target.ID, TargetID: target.ID,
TargetName: target.Name, TargetName: target.Name,
TargetType: target.Type, TargetType: target.Type,
TargetConfig: target.Config, TargetURL: target.URL,
MaxRetries: target.MaxRetries, TargetHeaders: target.Headers,
Method: event.Method, TimeoutSeconds: target.TimeoutSeconds,
RawQuery: event.RawQuery, ForwardQuery: target.ForwardQuery,
Headers: event.Headers, ArchiveExpiry: target.ArchiveExpiry,
ContentType: event.ContentType, ArchiveRotation: target.ArchiveRotation,
Body: bodyPtr, MaxAttempts: target.MaxAttempts,
AttemptNum: attemptNum, Method: event.Method,
RawQuery: event.RawQuery,
Headers: event.Headers,
ContentType: event.ContentType,
Body: bodyPtr,
AttemptNum: attemptNum,
} }
} }
@@ -2255,6 +2003,6 @@ func (e *Engine) sendRecoveredDeliveries(
attempts[deliveries[i].ID]+1, attempts[deliveries[i].ID]+1,
) )
e.queueRecovered(e.deliveryCh, task) e.queueRecovered(task)
} }
} }
+130 -180
View File
@@ -3,7 +3,6 @@ package delivery_test
import ( import (
"bytes" "bytes"
"context" "context"
"encoding/json"
"fmt" "fmt"
"io" "io"
"log/slog" "log/slog"
@@ -97,17 +96,6 @@ func iSeedWebhookDB(
return db return db
} }
func iHTTPConfig(url string) string {
cfg := delivery.HTTPTargetConfig{URL: url}
data, err := json.Marshal(cfg)
if err != nil {
panic("failed to marshal HTTPTargetConfig")
}
return string(data)
}
func iEngine( func iEngine(
t *testing.T, workers int, t *testing.T, workers int,
) *delivery.Engine { ) *delivery.Engine {
@@ -169,24 +157,24 @@ func iSeedDelivery(
func iTask( func iTask(
d database.Delivery, d database.Delivery,
event database.Event, event database.Event,
webhookID, targetID, name, config string, webhookID, targetID, name, targetURL string,
maxRetries, attemptNum int, maxAttempts, attemptNum int,
body *string, body *string,
) delivery.Task { ) delivery.Task {
return delivery.Task{ return delivery.Task{
DeliveryID: d.ID, DeliveryID: d.ID,
EventID: event.ID, EventID: event.ID,
WebhookID: webhookID, WebhookID: webhookID,
TargetID: targetID, TargetID: targetID,
TargetName: name, TargetName: name,
TargetType: database.TargetTypeHTTP, TargetType: database.TargetTypeHTTP,
TargetConfig: config, TargetURL: targetURL,
MaxRetries: maxRetries, MaxAttempts: maxAttempts,
Method: event.Method, Method: event.Method,
Headers: event.Headers, Headers: event.Headers,
ContentType: event.ContentType, ContentType: event.ContentType,
Body: body, Body: body,
AttemptNum: attemptNum, AttemptNum: attemptNum,
} }
} }
@@ -239,10 +227,9 @@ func TestProcessNewTask_InlineBody(t *testing.T) {
) )
bodyStr := event.Body bodyStr := event.Body
cfg := iHTTPConfig(ts.URL)
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"test-target", cfg, 0, 1, &bodyStr, "test-target", ts.URL, 1, 1, &bodyStr,
) )
s.Engine.ExportProcessNewTask( s.Engine.ExportProcessNewTask(
@@ -290,10 +277,9 @@ func TestProcessNewTask_LargeBody_FetchFromDB(
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
cfg := iHTTPConfig(ts.URL)
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"test-large", cfg, 0, 1, nil, "test-large", ts.URL, 1, 1, nil,
) )
s.Engine.ExportProcessNewTask( s.Engine.ExportProcessNewTask(
@@ -313,16 +299,16 @@ func TestProcessNewTask_InvalidWebhookID(t *testing.T) {
s := newISetup(t) s := newISetup(t)
task := delivery.Task{ task := delivery.Task{
DeliveryID: uuid.New().String(), DeliveryID: uuid.New().String(),
EventID: uuid.New().String(), EventID: uuid.New().String(),
WebhookID: uuid.New().String(), WebhookID: uuid.New().String(),
TargetID: uuid.New().String(), TargetID: uuid.New().String(),
TargetName: "test", TargetName: "test",
TargetType: database.TargetTypeHTTP, TargetType: database.TargetTypeHTTP,
TargetConfig: iHTTPConfig("http://localhost:9999"), TargetURL: "http://localhost:9999",
MaxRetries: 0, MaxAttempts: 1,
Body: nil, Body: nil,
AttemptNum: 1, AttemptNum: 1,
} }
s.Engine.ExportProcessNewTask( s.Engine.ExportProcessNewTask(
@@ -363,7 +349,6 @@ func TestProcessRetryTask_SuccessfulRetry(t *testing.T) {
) )
bodyStr := event.Body bodyStr := event.Body
cfg := iHTTPConfig(ts.URL)
// The target row exists because the engine confirms a scheduled // The target row exists because the engine confirms a scheduled
// retry's target has not been deleted before it runs it. A retry // retry's target has not been deleted before it runs it. A retry
@@ -372,12 +357,12 @@ func TestProcessRetryTask_SuccessfulRetry(t *testing.T) {
// task. See https://git.eeqj.de/sneak/webhooker/issues/107. // task. See https://git.eeqj.de/sneak/webhooker/issues/107.
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "retry-target", t, s.MainDB, targetID, s.WebhookID, "retry-target",
database.TargetTypeHTTP, cfg, 5, database.TargetTypeHTTP, ts.URL, 5,
) )
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"retry-target", cfg, 5, 2, &bodyStr, "retry-target", ts.URL, 5, 2, &bodyStr,
) )
s.Engine.ExportProcessRetryTask( s.Engine.ExportProcessRetryTask(
@@ -410,10 +395,9 @@ func TestProcessRetryTask_SkipsNonRetryingDelivery(
) )
bodyStr := event.Body bodyStr := event.Body
cfg := iHTTPConfig("http://localhost:1")
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"skip-target", cfg, 5, 2, &bodyStr, "skip-target", "http://localhost:1", 5, 2, &bodyStr,
) )
s.Engine.ExportProcessRetryTask( s.Engine.ExportProcessRetryTask(
@@ -460,16 +444,14 @@ func TestProcessRetryTask_LargeBody_FetchFromDB(
database.DeliveryStatusRetrying, database.DeliveryStatusRetrying,
) )
cfg := iHTTPConfig(ts.URL)
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "retry-large", t, s.MainDB, targetID, s.WebhookID, "retry-large",
database.TargetTypeHTTP, cfg, 5, database.TargetTypeHTTP, ts.URL, 5,
) )
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"retry-large", cfg, 5, 2, nil, "retry-large", ts.URL, 5, 2, nil,
) )
s.Engine.ExportProcessRetryTask( s.Engine.ExportProcessRetryTask(
@@ -506,7 +488,7 @@ func TestWorkerLifecycle_StartStop(t *testing.T) {
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"lifecycle-test", "", "lifecycle-test", "",
0, 1, &bodyStr, 1, 1, &bodyStr,
) )
task.TargetType = database.TargetTypeLog task.TargetType = database.TargetTypeLog
@@ -570,16 +552,15 @@ func TestWorkerLifecycle_ProcessesRetryChannel(
s.Engine.ExportStart() s.Engine.ExportStart()
bodyStr := event.Body bodyStr := event.Body
cfg := iHTTPConfig(ts.URL)
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "retry-chan-test", t, s.MainDB, targetID, s.WebhookID, "retry-chan-test",
database.TargetTypeHTTP, cfg, 5, database.TargetTypeHTTP, ts.URL, 5,
) )
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"retry-chan-test", cfg, 5, 2, &bodyStr, "retry-chan-test", ts.URL, 5, 2, &bodyStr,
) )
s.Engine.ExportRetryCh() <- task s.Engine.ExportRetryCh() <- task
@@ -645,7 +626,7 @@ func TestRecoverPendingDeliveries(t *testing.T) {
iCreateTarget(t, s.MainDB, targetID, iCreateTarget(t, s.MainDB, targetID,
s.WebhookID, "recovery-target", s.WebhookID, "recovery-target",
database.TargetTypeLog, "", 0, database.TargetTypeLog, "", 1,
) )
iSeedPendingDeliveries( iSeedPendingDeliveries(
@@ -659,7 +640,7 @@ func TestRecoverPendingDeliveries(t *testing.T) {
s.Engine.ExportRecoverPendingDeliveries( s.Engine.ExportRecoverPendingDeliveries(
context.Background(), s.WebhookDB, context.Background(), s.WebhookDB,
s.WebhookID, s.WebhookID, atStart,
) )
for i := range 3 { for i := range 3 {
@@ -685,18 +666,18 @@ func iCreateTarget(
mainDB *gorm.DB, mainDB *gorm.DB,
targetID, webhookID, name string, targetID, webhookID, name string,
targetType database.TargetType, targetType database.TargetType,
config string, targetURL string,
maxRetries int, maxAttempts int,
) { ) {
t.Helper() t.Helper()
target := database.Target{ target := database.Target{
WebhookID: webhookID, WebhookID: webhookID,
Name: name, Name: name,
Type: targetType, Type: targetType,
Active: true, Active: true,
Config: config, URL: targetURL,
MaxRetries: maxRetries, MaxAttempts: maxAttempts,
} }
target.ID = targetID target.ID = targetID
@@ -736,7 +717,7 @@ func TestRecoverWebhookDeliveries_RetryingDeliveries(
iCreateTarget(t, s.MainDB, targetID, iCreateTarget(t, s.MainDB, targetID,
s.WebhookID, "retry-recovery", s.WebhookID, "retry-recovery",
database.TargetTypeHTTP, database.TargetTypeHTTP,
iHTTPConfig("http://example.com/hook"), 5, "http://example.com/hook", 5,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -756,7 +737,7 @@ func TestRecoverWebhookDeliveries_RetryingDeliveries(
) )
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, atStart,
) )
select { select {
@@ -796,7 +777,7 @@ func iSeedRetryingWithType(
iCreateTarget(t, s.MainDB, targetID, iCreateTarget(t, s.MainDB, targetID,
s.WebhookID, "mutated-target", targetType, s.WebhookID, "mutated-target", targetType,
iHTTPConfig("http://example.com/hook"), 5, "http://example.com/hook", 5,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -880,7 +861,7 @@ func TestRecoverSingleRetry_TypeNoLongerRetries(
) )
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, atStart,
) )
iAssertTerminallyFailed( iAssertTerminallyFailed(
@@ -888,7 +869,7 @@ func TestRecoverSingleRetry_TypeNoLongerRetries(
) )
} }
func TestSweepSingleRetry_TypeNoLongerRetries( func TestRecoverSingleRetry_OnTick_TypeNoLongerRetries(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
@@ -903,8 +884,8 @@ func TestSweepSingleRetry_TypeNoLongerRetries(
t, s, database.TargetTypeDatabase, t, s, database.TargetTypeDatabase,
) )
s.Engine.ExportSweepWebhookRetries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, onTick,
) )
iAssertTerminallyFailed( iAssertTerminallyFailed(
@@ -934,14 +915,14 @@ func TestFailUnretryableRetry_WritesNoTargetRow(
targetID := uuid.New().String() targetID := uuid.New().String()
// A Slack incoming-webhook URL: the target config IS the // A Slack incoming-webhook URL: the target's URL IS the
// credential, which is what makes a leaked target row a // credential, which is what makes a leaked target row a
// disclosure rather than a curiosity. // disclosure rather than a curiosity.
hookURL := "https://hooks.slack.com/services/T00/B00/x" hookURL := "https://hooks.slack.com/services/T00/B00/x"
iCreateTarget(t, s.MainDB, targetID, iCreateTarget(t, s.MainDB, targetID,
s.WebhookID, "credential-bearing", s.WebhookID, "credential-bearing",
database.TargetTypeLog, iHTTPConfig(hookURL), 5, database.TargetTypeLog, hookURL, 5,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -955,8 +936,8 @@ func TestFailUnretryableRetry_WritesNoTargetRow(
iSeedFailedResult(t, s.WebhookDB, d.ID) iSeedFailedResult(t, s.WebhookDB, d.ID)
s.Engine.ExportSweepWebhookRetries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, onTick,
) )
iAssertStatus(t, s.WebhookDB, d.ID, iAssertStatus(t, s.WebhookDB, d.ID,
@@ -987,13 +968,13 @@ func TestRecoverSingleRetry_UnknownTargetType(
deliveryID := iSeedRetryingWithType(t, s, unknown) deliveryID := iSeedRetryingWithType(t, s, unknown)
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, atStart,
) )
iAssertTerminallyFailed(t, s, deliveryID, unknown) iAssertTerminallyFailed(t, s, deliveryID, unknown)
} }
func TestSweepSingleRetry_UnknownTargetType( func TestRecoverSingleRetry_OnTick_UnknownTargetType(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
@@ -1008,8 +989,8 @@ func TestSweepSingleRetry_UnknownTargetType(
deliveryID := iSeedRetryingWithType(t, s, unknown) deliveryID := iSeedRetryingWithType(t, s, unknown)
s.Engine.ExportSweepWebhookRetries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, onTick,
) )
iAssertTerminallyFailed(t, s, deliveryID, unknown) iAssertTerminallyFailed(t, s, deliveryID, unknown)
@@ -1051,19 +1032,19 @@ func iCreateWebhook(
require.NoError(t, mainDB.Create(&webhook).Error) require.NoError(t, mainDB.Create(&webhook).Error)
} }
// --- recoverInFlight Tests --- // --- recoverDeliveries Tests ---
func TestRecoverInFlight_NoWebhooks(t *testing.T) { func TestRecoverDeliveries_NoWebhooks(t *testing.T) {
t.Parallel() t.Parallel()
s := newISetup(t) s := newISetup(t)
s.Engine.ExportRecoverInFlight( s.Engine.ExportRecoverDeliveries(
context.Background(), context.Background(), atStart,
) )
} }
func TestRecoverInFlight_WithPendingDeliveries( func TestRecoverDeliveries_WithPendingDeliveries(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
@@ -1077,7 +1058,7 @@ func TestRecoverInFlight_WithPendingDeliveries(
iCreateTarget(t, s.MainDB, targetID, iCreateTarget(t, s.MainDB, targetID,
s.WebhookID, "recover-target", s.WebhookID, "recover-target",
database.TargetTypeLog, "", 0, database.TargetTypeLog, "", 1,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -1090,8 +1071,8 @@ func TestRecoverInFlight_WithPendingDeliveries(
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
s.Engine.ExportRecoverInFlight( s.Engine.ExportRecoverDeliveries(
context.Background(), context.Background(), atStart,
) )
select { select {
@@ -1102,16 +1083,16 @@ func TestRecoverInFlight_WithPendingDeliveries(
database.TargetTypeLog, task.TargetType, database.TargetTypeLog, task.TargetType,
) )
case <-time.After(2 * time.Second): case <-time.After(2 * time.Second):
t.Fatal("expected task from recoverInFlight") t.Fatal("expected task from recovery at start")
} }
} }
// TestRecoverInFlight_ReportsAMissingWebhookDatabase covers a webhook // TestRecoverDeliveries_ReportsAMissingWebhookDatabase covers a webhook
// whose database file is gone, after a partial restore say. Restart // whose database file is gone, after a partial restore say. Recovery at
// recovery opens every webhook's database, so the empty one made in its // start opens every webhook's database, so the empty one made in its
// place is reported at start, naming the file // place is reported at start, naming the file
// (https://git.eeqj.de/sneak/webhooker/issues/290). // (https://git.eeqj.de/sneak/webhooker/issues/290).
func TestRecoverInFlight_ReportsAMissingWebhookDatabase(t *testing.T) { func TestRecoverDeliveries_ReportsAMissingWebhookDatabase(t *testing.T) {
t.Parallel() t.Parallel()
mainDB := iMainDB(t) mainDB := iMainDB(t)
@@ -1131,7 +1112,7 @@ func TestRecoverInFlight_ReportsAMissingWebhookDatabase(t *testing.T) {
&http.Client{Timeout: 5 * time.Second}, 1, &http.Client{Timeout: 5 * time.Second}, 1,
) )
engine.ExportRecoverInFlight(context.Background()) engine.ExportRecoverDeliveries(context.Background(), atStart)
assert.Contains( assert.Contains(
t, logs.String(), t, logs.String(),
@@ -1140,10 +1121,11 @@ func TestRecoverInFlight_ReportsAMissingWebhookDatabase(t *testing.T) {
) )
} }
// TestRecoverInFlight_SkipsAWebhookDeletedAfterTheListIsRead covers a // TestRecoverDeliveries_SkipsAWebhookDeletedAfterTheListIsRead covers a
// webhook deleted from the web interface while restart recovery runs. // webhook deleted from the web interface while the recovery routine
// Its database file is gone, and recovery must not create it again. // runs. Its database file is gone, and recovery must not create it
func TestRecoverInFlight_SkipsAWebhookDeletedAfterTheListIsRead( // again.
func TestRecoverDeliveries_SkipsAWebhookDeletedAfterTheListIsRead(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
@@ -1179,13 +1161,13 @@ func TestRecoverInFlight_SkipsAWebhookDeletedAfterTheListIsRead(
&http.Client{Timeout: 5 * time.Second}, 1, &http.Client{Timeout: 5 * time.Second}, 1,
) )
engine.ExportRecoverInFlight(context.Background()) engine.ExportRecoverDeliveries(context.Background(), atStart)
require.True(t, deleted) require.True(t, deleted)
assert.False(t, dbMgr.DBExists(webhookID)) assert.False(t, dbMgr.DBExists(webhookID))
} }
// --- HTTP Config with custom headers --- // --- HTTP target with custom headers ---
func TestDeliverHTTP_CustomTargetHeaders(t *testing.T) { func TestDeliverHTTP_CustomTargetHeaders(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1207,16 +1189,6 @@ func TestDeliverHTTP_CustomTargetHeaders(t *testing.T) {
) )
defer ts.Close() defer ts.Close()
cfg := delivery.HTTPTargetConfig{
URL: ts.URL,
Headers: map[string]string{
"Authorization": "Bearer secret-token",
},
}
cfgJSON, err := json.Marshal(cfg)
require.NoError(t, err)
event := iSeedEvent( event := iSeedEvent(
t, s.WebhookDB, s.WebhookID, t, s.WebhookDB, s.WebhookID,
`{"auth":"test"}`, `{"auth":"test"}`,
@@ -1231,9 +1203,12 @@ func TestDeliverHTTP_CustomTargetHeaders(t *testing.T) {
bodyStr := event.Body bodyStr := event.Body
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"auth-target", string(cfgJSON), "auth-target", ts.URL,
0, 1, &bodyStr, 1, 1, &bodyStr,
) )
task.TargetHeaders = map[string]string{
headerAuthorization: "Bearer secret-token",
}
s.Engine.ExportProcessNewTask( s.Engine.ExportProcessNewTask(
context.TODO(), &task, context.TODO(), &task,
@@ -1246,11 +1221,14 @@ func TestDeliverHTTP_CustomTargetHeaders(t *testing.T) {
// --- HTTP delivery with custom timeout --- // --- HTTP delivery with custom timeout ---
// TestDeliverHTTP_TargetTimeout proves a task's TimeoutSeconds reaches
// the delivery through the engine. The destination answers after two
// seconds, inside the engine's own five-second client timeout, so only
// the task's one second can fail the delivery.
func TestDeliverHTTP_TargetTimeout(t *testing.T) { func TestDeliverHTTP_TargetTimeout(t *testing.T) {
t.Parallel() t.Parallel()
db := testWebhookDB(t) s := newISetup(t)
e := iEngine(t, 1)
ts := httptest.NewServer( ts := httptest.NewServer(
http.HandlerFunc( http.HandlerFunc(
@@ -1262,31 +1240,30 @@ func TestDeliverHTTP_TargetTimeout(t *testing.T) {
) )
defer ts.Close() defer ts.Close()
cfg := delivery.HTTPTargetConfig{ event := iSeedEvent(
URL: ts.URL, t, s.WebhookDB, s.WebhookID, `{"timeout":"test"}`,
Timeout: 1, )
} targetID := uuid.New().String()
cfgJSON, err := json.Marshal(cfg) d := iSeedDelivery(
require.NoError(t, err) t, s.WebhookDB, event.ID, targetID,
database.DeliveryStatusPending,
event, del := iSeedEventAndDelivery(
t, db, `{"timeout":"test"}`,
string(cfgJSON),
) )
task, d := iHTTPTaskAndDelivery( bodyStr := event.Body
event, del, "timeout-target", task := iTask(
string(cfgJSON), 0, 1, d, event, s.WebhookID, targetID,
"timeout-target", ts.URL, 1, 1, &bodyStr,
) )
task.TimeoutSeconds = 1
e.ExportDeliverHTTP(context.TODO(), db, d, task) s.Engine.ExportProcessNewTask(context.TODO(), &task)
iAssertStatus(t, db, del.ID, iAssertStatus(t, s.WebhookDB, d.ID,
database.DeliveryStatusFailed, database.DeliveryStatusFailed,
) )
iAssertResultFailed(t, db, del.ID) iAssertResultFailed(t, s.WebhookDB, d.ID)
} }
// TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog pins the size // TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog pins the size
@@ -1324,8 +1301,6 @@ func TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog(
)) ))
defer ts.Close() defer ts.Close()
cfgJSON := iHTTPConfig(ts.URL)
event := iSeedEvent( event := iSeedEvent(
t, s.WebhookDB, s.WebhookID, `{"cut":"test"}`, t, s.WebhookDB, s.WebhookID, `{"cut":"test"}`,
) )
@@ -1339,7 +1314,7 @@ func TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog(
bodyStr := event.Body bodyStr := event.Body
task := iTask( task := iTask(
del, event, s.WebhookID, targetID, del, event, s.WebhookID, targetID,
"cut-target", cfgJSON, 0, 1, &bodyStr, "cut-target", ts.URL, 1, 1, &bodyStr,
) )
s.Engine.ExportProcessNewTask(context.TODO(), &task) s.Engine.ExportProcessNewTask(context.TODO(), &task)
@@ -1371,7 +1346,7 @@ func TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog(
func iSeedEventAndDelivery( func iSeedEventAndDelivery(
t *testing.T, t *testing.T,
db *gorm.DB, db *gorm.DB,
body, _ string, body string,
) (database.Event, database.Delivery) { ) (database.Event, database.Delivery) {
t.Helper() t.Helper()
@@ -1402,19 +1377,19 @@ func iSeedEventAndDelivery(
func iHTTPTaskAndDelivery( func iHTTPTaskAndDelivery(
event database.Event, event database.Event,
del database.Delivery, del database.Delivery,
name, config string, name, targetURL string,
maxRetries, attemptNum int, maxAttempts, attemptNum int,
) (*delivery.Task, *database.Delivery) { ) (*delivery.Task, *database.Delivery) {
task := &delivery.Task{ task := &delivery.Task{
DeliveryID: del.ID, DeliveryID: del.ID,
EventID: event.ID, EventID: event.ID,
WebhookID: event.WebhookID, WebhookID: event.WebhookID,
TargetID: del.TargetID, TargetID: del.TargetID,
TargetName: name, TargetName: name,
TargetType: database.TargetTypeHTTP, TargetType: database.TargetTypeHTTP,
TargetConfig: config, TargetURL: targetURL,
MaxRetries: maxRetries, MaxAttempts: maxAttempts,
AttemptNum: attemptNum, AttemptNum: attemptNum,
} }
d := &database.Delivery{ d := &database.Delivery{
@@ -1423,9 +1398,10 @@ func iHTTPTaskAndDelivery(
Status: database.DeliveryStatusPending, Status: database.DeliveryStatusPending,
Event: event, Event: event,
Target: database.Target{ Target: database.Target{
Name: name, Name: name,
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Config: config, URL: targetURL,
MaxAttempts: maxAttempts,
}, },
} }
d.ID = del.ID d.ID = del.ID
@@ -1453,53 +1429,27 @@ func iAssertResultFailed(
assert.NotEmpty(t, result.Error) assert.NotEmpty(t, result.Error)
} }
// --- HTTP request with invalid config --- // --- HTTP request with no URL ---
func TestDeliverHTTP_InvalidConfig(t *testing.T) { // TestDeliverHTTP_NoURL proves an http target without a URL, which
// the forms never store, fails its delivery and records why.
func TestDeliverHTTP_NoURL(t *testing.T) {
t.Parallel() t.Parallel()
db := testWebhookDB(t) db := testWebhookDB(t)
e := iEngine(t, 1) e := iEngine(t, 1)
event, del := iSeedEventAndDelivery( event, del := iSeedEventAndDelivery(t, db, `{"url":"none"}`)
t, db, `{"config":"invalid"}`, "",
)
task, d := iHTTPTaskAndDelivery( task, d := iHTTPTaskAndDelivery(event, del, "no-url", "", 1, 1)
event, del, "bad-config", `not-json`, 0, 1,
)
e.ExportDeliverHTTP(context.TODO(), db, d, task) e.ExportDeliverHTTP(context.TODO(), db, d, task)
iAssertStatus(t, db, del.ID, iAssertStatus(t, db, del.ID,
database.DeliveryStatusFailed, database.DeliveryStatusFailed,
) )
}
// TestDeliverHTTP_InvalidConfigUnrecordedStaysPending: a delivery is iAssertResultFailed(t, db, del.ID)
// failed for an invalid config only once the reason is recorded.
// Unrecorded, it stays pending, where the sweep finds it again.
func TestDeliverHTTP_InvalidConfigUnrecordedStaysPending(t *testing.T) {
t.Parallel()
db := testWebhookDB(t)
e := testEngine(t, 1)
event, del := iSeedEventAndDelivery(
t, db, `{"config":"invalid"}`, "",
)
task, d := iHTTPTaskAndDelivery(
event, del, "bad-config", `not-json`, 0, 1,
)
require.NoError(t, db.Exec("drop table delivery_results").Error)
e.ExportDeliverHTTP(context.TODO(), db, d, task)
iAssertStatus(t, db, del.ID,
database.DeliveryStatusPending,
)
} }
// --- Notify batching --- // --- Notify batching ---
+172 -16
View File
@@ -2,11 +2,16 @@ package delivery_test
import ( import (
"context" "context"
"io"
"net/http"
"net/http/httptest"
"path/filepath" "path/filepath"
"sync"
"testing" "testing"
"time" "time"
"github.com/google/uuid" "github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"go.uber.org/fx" "go.uber.org/fx"
"gorm.io/gorm/clause" "gorm.io/gorm/clause"
@@ -94,7 +99,7 @@ func requireStopHookExpires(
// the caller can drive OnStop too. // the caller can drive OnStop too.
// //
// Callers must not seed pending or retrying deliveries before // Callers must not seed pending or retrying deliveries before
// calling this: restart recovery enqueues those during startup, // calling this: the recovery routine's run at start enqueues those,
// which would put work in the queue while the pool is still // which would put work in the queue while the pool is still
// racing its first select. // racing its first select.
func startEngineViaHook( func startEngineViaHook(
@@ -144,7 +149,7 @@ func seedLogTask(
bodyStr := event.Body bodyStr := event.Body
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"hook-context-test", "", 0, 1, &bodyStr, "hook-context-test", "", 1, 1, &bodyStr,
) )
task.TargetType = database.TargetTypeLog task.TargetType = database.TargetTypeLog
@@ -173,8 +178,8 @@ func TestEngine_WorkersOutliveStartHookContext(t *testing.T) {
_ = lc.hooks[0].OnStop(context.Background()) _ = lc.hooks[0].OnStop(context.Background())
}) })
// Seeded only after the pool has settled, so restart recovery // Seeded only after the pool has settled, so the recovery
// cannot enqueue it during startup. // routine's run at start cannot enqueue it.
deliveryID, task := seedLogTask(t, s) deliveryID, task := seedLogTask(t, s)
s.Engine.Notify([]delivery.Task{task}) s.Engine.Notify([]delivery.Task{task})
@@ -272,19 +277,168 @@ func TestEngine_StopHookHonoursStopTimeout(t *testing.T) {
requireStopHookExpires(t, lc.hooks[0], "delivery engine") requireStopHookExpires(t, lc.hooks[0], "delivery engine")
} }
// seedHeldDelivery gives the setup's webhook an http target with
// maxAttempts, pointed at a receiver that keeps every request open until
// the client gives up on it, and seeds one delivery to that target at
// status. It returns the task that sends attempt attemptNum of the
// delivery, and a channel that is closed when a request reaches the
// receiver.
func seedHeldDelivery(
t *testing.T,
s iSetup,
maxAttempts int,
status database.DeliveryStatus,
attemptNum int,
) (delivery.Task, <-chan struct{}) {
t.Helper()
arrived := make(chan struct{})
var once sync.Once
ts := httptest.NewServer(http.HandlerFunc(
func(_ http.ResponseWriter, r *http.Request) {
// The server notices the client hanging up only once the
// body has been read; until then r.Context() never ends.
_, _ = io.Copy(io.Discard, r.Body)
once.Do(func() { close(arrived) })
<-r.Context().Done()
},
))
t.Cleanup(ts.Close)
targetID := uuid.New().String()
iCreateWebhook(t, s.MainDB, s.WebhookID, "hook")
iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "held",
database.TargetTypeHTTP, ts.URL, maxAttempts,
)
event := iSeedEvent(t, s.WebhookDB, s.WebhookID, `{"held":true}`)
d := iSeedDelivery(t, s.WebhookDB, event.ID, targetID, status)
bodyStr := event.Body
return iTask(
d, event, s.WebhookID, targetID, "held", ts.URL,
maxAttempts, attemptNum, &bodyStr,
), arrived
}
// stopDuringAttempt waits until the receiver holds an attempt open, then
// stops the engine through its real stop hook.
func stopDuringAttempt(
t *testing.T, lc *recordingLifecycle, arrived <-chan struct{},
) {
t.Helper()
select {
case <-arrived:
case <-time.After(5 * time.Second):
t.Fatal("the delivery attempt never reached the receiver")
}
require.NoError(t, lc.hooks[0].OnStop(context.Background()))
}
// requireRequeued runs the recovery routine as at start on the stopped
// engine and requires it to queue task's attempt again on ch.
func requireRequeued(
t *testing.T, s iSetup, ch <-chan delivery.Task, task delivery.Task,
) {
t.Helper()
s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, atStart,
)
select {
case got := <-ch:
assert.Equal(t, task.DeliveryID, got.DeliveryID)
assert.Equal(t, task.AttemptNum, got.AttemptNum)
case <-time.After(5 * time.Second):
t.Fatal("recovery at start did not queue the delivery again")
}
}
// TestEngine_StopLeavesFireAndForgetAttemptPending is the regression
// test for a stop that turned an attempt in flight into a terminal
// failure. Stopping the engine cancels the request; for a target with
// no retries that was the delivery's only attempt, recorded as failed
// with "context canceled", and nothing ever sent it again. The
// delivery must instead stay pending, with no result and no count in
// the attempt metrics, for the recovery routine to send at the next
// start.
func TestEngine_StopLeavesFireAndForgetAttemptPending(t *testing.T) {
t.Parallel()
s := newISetup(t)
reg := mIsolate(t, s)
lc := startEngineViaHook(t, s.Engine)
task, arrived := seedHeldDelivery(
t, s, 1, database.DeliveryStatusPending, 1,
)
s.Engine.Notify([]delivery.Task{task})
stopDuringAttempt(t, lc, arrived)
iAssertStatus(
t, s.WebhookDB, task.DeliveryID, database.DeliveryStatusPending,
)
assert.Empty(t, iResults(t, s.WebhookDB, task.DeliveryID))
assert.Zero(t, mCounter(t, reg, mAttempts, mTypeHTTP))
assert.Zero(t, mHTTPDurations(t, reg))
requireRequeued(t, s, s.Engine.ExportDeliveryCh(), task)
}
// TestEngine_StopLeavesLastRetryAttemptRetrying is the same regression
// for a target with retries, on its last attempt: the cancelled attempt
// used to exhaust the retries and end the delivery failed. It must stay
// retrying with only its earlier attempt recorded, and the recovery
// routine at the next start must send that last attempt again.
func TestEngine_StopLeavesLastRetryAttemptRetrying(t *testing.T) {
t.Parallel()
const maxAttempts = 2
s := newISetup(t)
lc := startEngineViaHook(t, s.Engine)
task, arrived := seedHeldDelivery(
t, s, maxAttempts, database.DeliveryStatusRetrying, maxAttempts,
)
iSeedFailedResult(t, s.WebhookDB, task.DeliveryID)
s.Engine.ExportScheduleRetry(task, 0)
stopDuringAttempt(t, lc, arrived)
iAssertStatus(
t, s.WebhookDB, task.DeliveryID, database.DeliveryStatusRetrying,
)
assert.Len(t, iResults(t, s.WebhookDB, task.DeliveryID), 1)
requireRequeued(t, s, s.Engine.ExportRetryCh(), task)
}
// deliverToArchive gives the setup's webhook a database target, // deliverToArchive gives the setup's webhook a database target,
// runs one delivery to it through the running engine, and returns // runs one delivery to it through the running engine, and returns
// the target's ID and archive file path. The archive writer holds // the target's archive file path. The archive writer holds the file
// the file open afterwards. // open afterwards.
func deliverToArchive(t *testing.T, s iSetup) (string, string) { func deliverToArchive(t *testing.T, s iSetup) string {
t.Helper() t.Helper()
iCreateWebhook(t, s.MainDB, s.WebhookID, "hook") iCreateWebhook(t, s.MainDB, s.WebhookID, "hook")
tgt := &database.Target{ tgt := &database.Target{
WebhookID: s.WebhookID, WebhookID: s.WebhookID,
Name: "archive", Name: "archive",
Type: database.TargetTypeDatabase, Type: database.TargetTypeDatabase,
MaxAttempts: 1,
} }
require.NoError( require.NoError(
t, s.MainDB.Omit(clause.Associations).Create(tgt).Error, t, s.MainDB.Omit(clause.Associations).Create(tgt).Error,
@@ -298,7 +452,7 @@ func deliverToArchive(t *testing.T, s iSetup) (string, string) {
iWaitForDelivered(t, s.WebhookDB, deliveryID) iWaitForDelivered(t, s.WebhookDB, deliveryID)
return tgt.ID, filepath.Join( return filepath.Join(
filepath.Dir(s.DBMgr.DBPath(s.WebhookID)), filepath.Dir(s.DBMgr.DBPath(s.WebhookID)),
"archive-hook-archive-"+tgt.ID+".db", "archive-hook-archive-"+tgt.ID+".db",
) )
@@ -317,7 +471,7 @@ func TestEngine_StopHookClosesArchives(t *testing.T) {
lc := startEngineViaHook(t, s.Engine) lc := startEngineViaHook(t, s.Engine)
_, path := deliverToArchive(t, s) path := deliverToArchive(t, s)
require.FileExists( require.FileExists(
t, path+"-wal", t, path+"-wal",
"an open archive should have a -wal for the stop to remove", "an open archive should have a -wal for the stop to remove",
@@ -351,21 +505,23 @@ func TestEngine_StopHookTimeoutLeavesArchivesOpen(t *testing.T) {
lc := startEngineViaHook(t, s.Engine) lc := startEngineViaHook(t, s.Engine)
targetID, _ := deliverToArchive(t, s) path := deliverToArchive(t, s)
release := make(chan struct{}) release := make(chan struct{})
t.Cleanup(func() { t.Cleanup(func() {
close(release) close(release)
s.Engine.EvictWebhook(s.WebhookID) s.Engine.ExportArchives().EvictWebhook(s.WebhookID)
}) })
s.Engine.ExportWedgeWorker(release) s.Engine.ExportWedgeWorker(release)
requireStopHookExpires(t, lc.hooks[0], "delivery engine") requireStopHookExpires(t, lc.hooks[0], "delivery engine")
require.True( // Closing the last handle on an archive removes its -wal, so a
t, s.Engine.ExportArchiveHandleOpen(targetID), // -wal still there is a writer still open.
require.FileExists(
t, path+"-wal",
"a stop that timed out must not close archive writers", "a stop that timed out must not close archive writers",
) )
} }
+185 -235
View File
@@ -65,20 +65,6 @@ func testEngine(
) )
} }
func newHTTPTargetConfig(url string) string {
cfg := delivery.HTTPTargetConfig{URL: url}
data, err := json.Marshal(cfg)
if err != nil {
panic(
"failed to marshal HTTPTargetConfig: " +
err.Error(),
)
}
return string(data)
}
func seedEvent( func seedEvent(
t *testing.T, db *gorm.DB, body string, t *testing.T, db *gorm.DB, body string,
) database.Event { ) database.Event {
@@ -117,6 +103,18 @@ func seedDelivery(
return d return d
} }
// assertStoredTime asserts that text, a timestamp as SQLite holds it,
// is in the form GORM stores created_at in: the layout of
// time.Time.String, with no " m=+…" monotonic clock reading after it.
func assertStoredTime(t *testing.T, text string) {
t.Helper()
_, err := time.Parse(
"2006-01-02 15:04:05.999999999 -0700 MST", text,
)
assert.NoError(t, err, "stored as %q", text)
}
// httpDeliveryFixture holds the task and delivery needed // httpDeliveryFixture holds the task and delivery needed
// for HTTP delivery tests. // for HTTP delivery tests.
type httpDeliveryFixture struct { type httpDeliveryFixture struct {
@@ -129,19 +127,19 @@ type httpDeliveryFixture struct {
func buildHTTPFixture( func buildHTTPFixture(
dlv database.Delivery, dlv database.Delivery,
event database.Event, event database.Event,
targetID, name, config string, targetID, name, targetURL string,
maxRetries, attemptNum int, maxAttempts, attemptNum int,
) httpDeliveryFixture { ) httpDeliveryFixture {
task := &delivery.Task{ task := &delivery.Task{
DeliveryID: dlv.ID, DeliveryID: dlv.ID,
EventID: event.ID, EventID: event.ID,
WebhookID: event.WebhookID, WebhookID: event.WebhookID,
TargetID: targetID, TargetID: targetID,
TargetName: name, TargetName: name,
TargetType: database.TargetTypeHTTP, TargetType: database.TargetTypeHTTP,
TargetConfig: config, TargetURL: targetURL,
MaxRetries: maxRetries, MaxAttempts: maxAttempts,
AttemptNum: attemptNum, AttemptNum: attemptNum,
} }
d := &database.Delivery{ d := &database.Delivery{
@@ -150,10 +148,10 @@ func buildHTTPFixture(
Status: dlv.Status, Status: dlv.Status,
Event: event, Event: event,
Target: database.Target{ Target: database.Target{
Name: name, Name: name,
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Config: config, URL: targetURL,
MaxRetries: maxRetries, MaxAttempts: maxAttempts,
}, },
} }
d.ID = dlv.ID d.ID = dlv.ID
@@ -263,9 +261,8 @@ func TestDeliverHTTP_Success(t *testing.T) {
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
cfg := newHTTPTargetConfig(ts.URL)
fix := buildHTTPFixture( fix := buildHTTPFixture(
dlv, event, targetID, "test-http", cfg, 0, 1, dlv, event, targetID, "test-http", ts.URL, 1, 1,
) )
e.ExportDeliverHTTP( e.ExportDeliverHTTP(
@@ -309,10 +306,9 @@ func TestDeliverHTTP_Failure(t *testing.T) {
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
cfg := newHTTPTargetConfig(ts.URL)
fix := buildHTTPFixture( fix := buildHTTPFixture(
dlv, event, targetID, dlv, event, targetID,
"test-http-fail", cfg, 0, 1, "test-http-fail", ts.URL, 1, 1,
) )
e.ExportDeliverHTTP( e.ExportDeliverHTTP(
@@ -339,7 +335,7 @@ func TestDeliverDatabase_ImmediateSuccess(
// The database target archives for real, so the engine needs // The database target archives for real, so the engine needs
// the target in the main database and a data directory. // the target in the main database and a data directory.
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, "") tgt := env.seedDatabaseTarget(t)
event := seedEvent(t, db, `{"db":"target"}`) event := seedEvent(t, db, `{"db":"target"}`)
d := seedDatabaseTargetDelivery(t, db, event, tgt) d := seedDatabaseTargetDelivery(t, db, event, tgt)
@@ -440,10 +436,9 @@ func TestDeliverHTTP_WithRetries_Success(t *testing.T) {
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
cfg := newHTTPTargetConfig(ts.URL)
fix := buildHTTPFixture( fix := buildHTTPFixture(
dlv, event, targetID, dlv, event, targetID,
"test-http-retry", cfg, 5, 1, "test-http-retry", ts.URL, 5, 1,
) )
e.ExportDeliverHTTP( e.ExportDeliverHTTP(
@@ -461,14 +456,23 @@ func TestDeliverHTTP_WithRetries_Success(t *testing.T) {
) )
} }
func TestDeliverHTTP_MaxRetriesExhausted(t *testing.T) { // TestDeliverHTTP_MaxAttemptsExhausted proves a target with three
// delivery attempts makes at most three: the first two failures are
// retried, the third is final, and each attempt goes through the
// target's circuit breaker.
func TestDeliverHTTP_MaxAttemptsExhausted(t *testing.T) {
t.Parallel() t.Parallel()
const maxAttempts = 3
db := testWebhookDB(t) db := testWebhookDB(t)
var requests atomic.Int32
ts := httptest.NewServer( ts := httptest.NewServer(
http.HandlerFunc( http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) { func(w http.ResponseWriter, _ *http.Request) {
requests.Add(1)
w.WriteHeader(http.StatusBadGateway) w.WriteHeader(http.StatusBadGateway)
}, },
), ),
@@ -477,26 +481,101 @@ func TestDeliverHTTP_MaxRetriesExhausted(t *testing.T) {
e := testEngine(t, 1) e := testEngine(t, 1)
targetID := uuid.New().String() targetID := uuid.New().String()
// It opens on the last failure only if every attempt reached it.
cb := delivery.NewTestCircuitBreaker(maxAttempts, time.Minute)
e.ExportSetCircuitBreaker(targetID, cb)
event := seedEvent(t, db, `{"retry":"exhaust"}`) event := seedEvent(t, db, `{"retry":"exhaust"}`)
dlv := seedDelivery( dlv := seedDelivery(
t, db, event.ID, targetID, t, db, event.ID, targetID,
database.DeliveryStatusRetrying, database.DeliveryStatusPending,
) )
cfg := newHTTPTargetConfig(ts.URL) sched := &recordingScheduler{}
fix := buildHTTPFixture(
dlv, event, targetID,
"test-http-exhaust", cfg, 3, 3,
)
e.ExportDeliverHTTP( for attempt := 1; attempt <= maxAttempts; attempt++ {
context.TODO(), db, fix.Delivery, fix.Task, fix := buildHTTPFixture(
) dlv, event, targetID,
"test-http-exhaust", ts.URL, maxAttempts, attempt,
)
e.ExportDeliverHTTPWithScheduler(
context.TODO(), db, fix.Delivery, fix.Task, sched,
)
}
assert.Equal(t, int32(maxAttempts), requests.Load())
assert.Len(t, sched.delays, maxAttempts-1,
"every failure but the last should be retried",
)
assertDeliveryStatus(t, db, dlv.ID, assertDeliveryStatus(t, db, dlv.ID,
database.DeliveryStatusFailed, database.DeliveryStatusFailed,
) )
assert.Equal(t, delivery.CircuitOpen, cb.State(),
"every attempt should be counted by the circuit breaker",
)
}
// TestDeliverHTTP_SingleAttemptSkipsCircuitBreaker proves a target
// with one delivery attempt makes exactly that attempt, even while
// its circuit breaker is open, and that a failure is final.
func TestDeliverHTTP_SingleAttemptSkipsCircuitBreaker(t *testing.T) {
t.Parallel()
db := testWebhookDB(t)
var requests atomic.Int32
ts := httptest.NewServer(
http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
requests.Add(1)
w.WriteHeader(http.StatusBadGateway)
},
),
)
defer ts.Close()
e := testEngine(t, 1)
targetID := uuid.New().String()
cb := e.ExportGetCircuitBreaker(targetID)
for range delivery.ExportDefaultFailureThreshold {
cb.RecordFailure()
}
require.Equal(t, delivery.CircuitOpen, cb.State())
event := seedEvent(t, db, `{"retry":"none"}`)
dlv := seedDelivery(
t, db, event.ID, targetID,
database.DeliveryStatusPending,
)
fix := buildHTTPFixture(
dlv, event, targetID,
"test-http-single", ts.URL, 1, 1,
)
sched := &recordingScheduler{}
e.ExportDeliverHTTPWithScheduler(
context.TODO(), db, fix.Delivery, fix.Task, sched,
)
assert.Equal(t, int32(1), requests.Load(),
"the open circuit breaker should not hold the attempt back",
)
assert.Empty(t, sched.delays, "no retry should be scheduled")
assertDeliveryStatus(t, db, dlv.ID,
database.DeliveryStatusFailed,
)
assertDeliveryResult(
t, db, dlv.ID, false, http.StatusBadGateway,
)
} }
func TestDeliverHTTP_SchedulesRetryOnFailure( func TestDeliverHTTP_SchedulesRetryOnFailure(
@@ -526,10 +605,9 @@ func TestDeliverHTTP_SchedulesRetryOnFailure(
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
cfg := newHTTPTargetConfig(ts.URL)
fix := buildHTTPFixture( fix := buildHTTPFixture(
dlv, event, targetID, dlv, event, targetID,
"test-http-schedule", cfg, 5, 1, "test-http-schedule", ts.URL, 5, 1,
) )
e.ExportDeliverHTTP( e.ExportDeliverHTTP(
@@ -691,7 +769,7 @@ func seedConcurrencyTasks(
t *testing.T, t *testing.T,
db *gorm.DB, db *gorm.DB,
n int, n int,
targetCfg string, targetURL string,
) ([]database.Delivery, []delivery.Task) { ) ([]database.Delivery, []delivery.Task) {
t.Helper() t.Helper()
@@ -715,22 +793,23 @@ func seedConcurrencyTasks(
Status: database.DeliveryStatusPending, Status: database.DeliveryStatusPending,
Event: event, Event: event,
Target: database.Target{ Target: database.Target{
Name: fmt.Sprintf("task-%d", i), Name: fmt.Sprintf("task-%d", i),
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Config: targetCfg, URL: targetURL,
MaxAttempts: 1,
}, },
} }
tasks[i].ID = dlv.ID tasks[i].ID = dlv.ID
dtasks[i] = delivery.Task{ dtasks[i] = delivery.Task{
DeliveryID: tasks[i].ID, DeliveryID: tasks[i].ID,
EventID: tasks[i].EventID, EventID: tasks[i].EventID,
TargetID: tasks[i].TargetID, TargetID: tasks[i].TargetID,
TargetName: tasks[i].Target.Name, TargetName: tasks[i].Target.Name,
TargetType: tasks[i].Target.Type, TargetType: tasks[i].Target.Type,
TargetConfig: tasks[i].Target.Config, TargetURL: tasks[i].Target.URL,
MaxRetries: 0, MaxAttempts: 1,
AttemptNum: 1, AttemptNum: 1,
} }
} }
@@ -766,9 +845,8 @@ func TestWorkerPool_BoundedConcurrency(t *testing.T) {
defer ts.Close() defer ts.Close()
e := testEngine(t, numWorkers) e := testEngine(t, numWorkers)
cfg := newHTTPTargetConfig(ts.URL)
tasks, dtasks := seedConcurrencyTasks( tasks, dtasks := seedConcurrencyTasks(
t, db, numTasks, cfg, t, db, numTasks, ts.URL,
) )
runConcurrencyWorkers( runConcurrencyWorkers(
@@ -849,12 +927,9 @@ func TestDeliverHTTP_CircuitBreakerBlocks(t *testing.T) {
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
cfg := newHTTPTargetConfig(
"http://will-not-be-called.invalid",
)
fix := buildHTTPFixture( fix := buildHTTPFixture(
dlv, event, targetID, dlv, event, targetID,
"test-cb-block", cfg, 5, 1, "test-cb-block", "http://will-not-be-called.invalid", 5, 1,
) )
e.ExportDeliverHTTP( e.ExportDeliverHTTP(
@@ -917,9 +992,8 @@ func TestDeliverHTTP_HalfOpenBreakerDelaysQueuedTasks(t *testing.T) {
require.True(t, cb.Allow(), "the probe delivery should go through") require.True(t, cb.Allow(), "the probe delivery should go through")
require.Equal(t, delivery.CircuitHalfOpen, cb.State()) require.Equal(t, delivery.CircuitHalfOpen, cb.State())
cfg := newHTTPTargetConfig( const targetURL = "http://will-not-be-called.invalid"
"http://will-not-be-called.invalid",
)
sched := &recordingScheduler{} sched := &recordingScheduler{}
const queued, passes = 3, 4 const queued, passes = 3, 4
@@ -941,7 +1015,7 @@ func TestDeliverHTTP_HalfOpenBreakerDelaysQueuedTasks(t *testing.T) {
fix := buildHTTPFixture( fix := buildHTTPFixture(
row, event, targetID, row, event, targetID,
"test-cb-half-open", cfg, 5, 1, "test-cb-half-open", targetURL, 5, 1,
) )
e.ExportDeliverHTTPWithScheduler( e.ExportDeliverHTTPWithScheduler(
@@ -1056,62 +1130,6 @@ func TestStateAndCooldown_ReadsHTTPAndSlackBreakers(t *testing.T) {
assert.Zero(t, cooldown, "half-open") assert.Zero(t, cooldown, "half-open")
} }
func TestParseHTTPConfig_Valid(t *testing.T) {
t.Parallel()
e := testEngine(t, 1)
cfg, err := e.ExportParseHTTPConfig(
`{"url":"https://example.com/hook",` +
`"headers":{"X-Token":"secret"}}`,
)
require.NoError(t, err)
assert.Equal(t, "https://example.com/hook", cfg.URL)
assert.Equal(t, "secret", cfg.Headers["X-Token"])
}
func TestParseHTTPConfig_Empty(t *testing.T) {
t.Parallel()
e := testEngine(t, 1)
_, err := e.ExportParseHTTPConfig("")
assert.Error(t, err,
"empty config should return error",
)
}
func TestParseHTTPConfig_MissingURL(t *testing.T) {
t.Parallel()
e := testEngine(t, 1)
_, err := e.ExportParseHTTPConfig(
`{"headers":{"X-Token":"secret"}}`,
)
assert.Error(t, err,
"config without URL should return error",
)
}
func TestParseHTTPConfig_Undecodable(t *testing.T) {
t.Parallel()
e := testEngine(t, 1)
_, err := e.ExportParseHTTPConfig(
`{"url":"https://example.com/hook","timeout":"soon"}`,
)
assert.Error(t, err,
"config that does not decode should return error, "+
"even when the part that did names a URL",
)
}
func TestScheduleRetry_SendsToRetryChannel( func TestScheduleRetry_SendsToRetryChannel(
t *testing.T, t *testing.T,
) { ) {
@@ -1259,7 +1277,7 @@ func TestDoHTTPRequest_ForwardsHeaders(t *testing.T) {
e := testEngine(t, 1) e := testEngine(t, 1)
cfg := &delivery.HTTPTargetConfig{ target := &database.Target{
URL: ts.URL, URL: ts.URL,
Headers: map[string]string{ Headers: map[string]string{
"X-Target-Auth": "bearer xyz", "X-Target-Auth": "bearer xyz",
@@ -1274,7 +1292,7 @@ func TestDoHTTPRequest_ForwardsHeaders(t *testing.T) {
} }
statusCode, _, _, err := e.ExportDoHTTPRequest( statusCode, _, _, err := e.ExportDoHTTPRequest(
context.TODO(), cfg, event, context.TODO(), target, event,
) )
require.NoError(t, err) require.NoError(t, err)
@@ -1316,7 +1334,7 @@ func TestDoHTTPRequest_CutShortResponseIsAnError(t *testing.T) {
_, body, _, err := e.ExportDoHTTPRequest( _, body, _, err := e.ExportDoHTTPRequest(
context.TODO(), context.TODO(),
&delivery.HTTPTargetConfig{URL: ts.URL}, &database.Target{URL: ts.URL},
&database.Event{}, &database.Event{},
) )
@@ -1368,9 +1386,9 @@ func TestApplyRequestHeaders_SendsOneContentType(t *testing.T) {
}) })
require.NoError(t, err) require.NoError(t, err)
cfg := &delivery.HTTPTargetConfig{} var configured map[string]string
if tc.configured != "" { if tc.configured != "" {
cfg.Headers = map[string]string{ configured = map[string]string{
headerContentType: tc.configured, headerContentType: tc.configured,
} }
} }
@@ -1389,7 +1407,7 @@ func TestApplyRequestHeaders_SendsOneContentType(t *testing.T) {
Headers: string(inbound), Headers: string(inbound),
ContentType: tc.event, ContentType: tc.event,
}, },
cfg, configured,
"webhooker/dev", "webhooker/dev",
) )
@@ -1420,7 +1438,7 @@ func TestApplyRequestHeaders_UndecodableInboundForwardsNothing(
&database.Event{ &database.Event{
Headers: `{"X-Custom":["value1"],"X-Broken":"not a list"}`, Headers: `{"X-Custom":["value1"],"X-Broken":"not a list"}`,
}, },
&delivery.HTTPTargetConfig{}, nil,
"webhooker/dev", "webhooker/dev",
) )
@@ -1438,7 +1456,7 @@ func TestProcessDelivery_RoutesToCorrectHandler(
// The database target archives for real, so the engine needs // The database target archives for real, so the engine needs
// the target in the main database and a data directory. // the target in the main database and a data directory.
env := setupArchiveTest(t) env := setupArchiveTest(t)
archive := env.seedDatabaseTarget(t, "") archive := env.seedDatabaseTarget(t)
tests := []struct { tests := []struct {
name string name string
@@ -1523,51 +1541,6 @@ func TestMaxInlineBodySize_Constant(t *testing.T) {
) )
} }
func TestParseSlackConfig_Valid(t *testing.T) {
t.Parallel()
e := testEngine(t, 1)
cfg, err := e.ExportParseSlackConfig(
`{"webhookUrl":"https://hooks.slack.com/services/T00/B00/xxx"}`,
)
require.NoError(t, err)
assert.Equal(t,
"https://hooks.slack.com/services/T00/B00/xxx",
cfg.WebhookURL,
)
}
func TestParseSlackConfig_Empty(t *testing.T) {
t.Parallel()
e := testEngine(t, 1)
_, err := e.ExportParseSlackConfig("")
assert.Error(t, err,
"empty config should return error",
)
}
func TestParseSlackConfig_MissingWebhookURL(
t *testing.T,
) {
t.Parallel()
e := testEngine(t, 1)
_, err := e.ExportParseSlackConfig(
`{"other":"field"}`,
)
assert.Error(t, err,
"config without webhook_url should return error",
)
}
func TestFormatSlackMessage_JSONBody(t *testing.T) { func TestFormatSlackMessage_JSONBody(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1664,11 +1637,11 @@ func TestFormatSlackMessage_LargeJSONTruncated(
} }
// buildSlackDelivery creates a database.Delivery // buildSlackDelivery creates a database.Delivery
// with a Slack target config for testing. // to a Slack target with the given webhook URL for testing.
func buildSlackDelivery( func buildSlackDelivery(
dlv database.Delivery, dlv database.Delivery,
event database.Event, event database.Event,
targetID, name, slackCfg string, targetID, name, webhookURL string,
) *database.Delivery { ) *database.Delivery {
d := &database.Delivery{ d := &database.Delivery{
EventID: event.ID, EventID: event.ID,
@@ -1676,9 +1649,10 @@ func buildSlackDelivery(
Status: dlv.Status, Status: dlv.Status,
Event: event, Event: event,
Target: database.Target{ Target: database.Target{
Name: name, Name: name,
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
Config: slackCfg, URL: webhookURL,
MaxAttempts: 1,
}, },
} }
d.ID = dlv.ID d.ID = dlv.ID
@@ -1715,11 +1689,10 @@ func TestDeliverSlack_Success(t *testing.T) {
) )
defer ts.Close() defer ts.Close()
e, targetID, slackCfg, event, dlv := e, targetID, event, dlv := setupSlackTest(t, db)
setupSlackTest(t, db, ts.URL)
d := buildSlackDelivery( d := buildSlackDelivery(
dlv, event, targetID, "test-slack", slackCfg, dlv, event, targetID, "test-slack", ts.URL,
) )
e.ExportDeliverSlack(context.TODO(), db, d) e.ExportDeliverSlack(context.TODO(), db, d)
@@ -1738,9 +1711,8 @@ func TestDeliverSlack_Success(t *testing.T) {
func setupSlackTest( func setupSlackTest(
t *testing.T, t *testing.T,
db *gorm.DB, db *gorm.DB,
serverURL string,
) ( ) (
*delivery.Engine, string, string, *delivery.Engine, string,
database.Event, database.Delivery, database.Event, database.Delivery,
) { ) {
t.Helper() t.Helper()
@@ -1748,13 +1720,6 @@ func setupSlackTest(
e := testEngine(t, 1) e := testEngine(t, 1)
targetID := uuid.New().String() targetID := uuid.New().String()
cfgBytes, err := json.Marshal(
delivery.SlackTargetConfig{
WebhookURL: serverURL,
},
)
require.NoError(t, err)
event := seedEvent( event := seedEvent(
t, db, `{"action":"test","data":"value"}`, t, db, `{"action":"test","data":"value"}`,
) )
@@ -1764,7 +1729,7 @@ func setupSlackTest(
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
return e, targetID, string(cfgBytes), event, dlv return e, targetID, event, dlv
} }
func assertSlackPayload( func assertSlackPayload(
@@ -1813,13 +1778,6 @@ func TestDeliverSlack_Failure(t *testing.T) {
e := testEngine(t, 1) e := testEngine(t, 1)
targetID := uuid.New().String() targetID := uuid.New().String()
slackCfg, err := json.Marshal(
delivery.SlackTargetConfig{
WebhookURL: ts.URL,
},
)
require.NoError(t, err)
event := seedEvent(t, db, `{"test":true}`) event := seedEvent(t, db, `{"test":true}`)
dlv := seedDelivery( dlv := seedDelivery(
@@ -1829,7 +1787,7 @@ func TestDeliverSlack_Failure(t *testing.T) {
d := buildSlackDelivery( d := buildSlackDelivery(
dlv, event, targetID, dlv, event, targetID,
"test-slack-fail", string(slackCfg), "test-slack-fail", ts.URL,
) )
e.ExportDeliverSlack(context.TODO(), db, d) e.ExportDeliverSlack(context.TODO(), db, d)
@@ -1843,7 +1801,10 @@ func TestDeliverSlack_Failure(t *testing.T) {
) )
} }
func TestDeliverSlack_InvalidConfig(t *testing.T) { // TestDeliverSlack_NoURL proves a Slack target without a URL, which
// the forms never store, fails its delivery rather than reporting it
// delivered.
func TestDeliverSlack_NoURL(t *testing.T) {
t.Parallel() t.Parallel()
db := testWebhookDB(t) db := testWebhookDB(t)
@@ -1862,9 +1823,9 @@ func TestDeliverSlack_InvalidConfig(t *testing.T) {
Status: database.DeliveryStatusPending, Status: database.DeliveryStatusPending,
Event: event, Event: event,
Target: database.Target{ Target: database.Target{
Name: "test-slack-bad", Name: "test-slack-bad",
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
Config: `{"not_webhook_url":"missing"}`, MaxAttempts: 1,
}, },
} }
d.ID = dlv.ID d.ID = dlv.ID
@@ -1901,13 +1862,6 @@ func TestProcessDelivery_RoutesToSlack(t *testing.T) {
e := testEngine(t, 1) e := testEngine(t, 1)
slackCfg, err := json.Marshal(
delivery.SlackTargetConfig{
WebhookURL: ts.URL,
},
)
require.NoError(t, err)
event := seedEvent(t, db, `{"route":"slack"}`) event := seedEvent(t, db, `{"route":"slack"}`)
dlv := seedDelivery( dlv := seedDelivery(
@@ -1917,12 +1871,13 @@ func TestProcessDelivery_RoutesToSlack(t *testing.T) {
d := buildSlackDelivery( d := buildSlackDelivery(
dlv, event, dlv.TargetID, dlv, event, dlv.TargetID,
"test-slack-route", string(slackCfg), "test-slack-route", ts.URL,
) )
task := &delivery.Task{ task := &delivery.Task{
DeliveryID: dlv.ID, DeliveryID: dlv.ID,
TargetType: database.TargetTypeSlack, TargetType: database.TargetTypeSlack,
MaxAttempts: 1,
} }
e.ExportProcessDelivery( e.ExportProcessDelivery(
@@ -2031,7 +1986,7 @@ func TestDeliverLog_LogsFullContent(t *testing.T) {
func buildSlackRetryDelivery( func buildSlackRetryDelivery(
dlv database.Delivery, dlv database.Delivery,
event database.Event, event database.Event,
targetID, cfg string, targetID, webhookURL string,
) *database.Delivery { ) *database.Delivery {
d := &database.Delivery{ d := &database.Delivery{
EventID: event.ID, EventID: event.ID,
@@ -2039,10 +1994,10 @@ func buildSlackRetryDelivery(
Status: database.DeliveryStatusPending, Status: database.DeliveryStatusPending,
Event: event, Event: event,
Target: database.Target{ Target: database.Target{
Name: "test-slack-retry", Name: "test-slack-retry",
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
Config: cfg, URL: webhookURL,
MaxRetries: 5, MaxAttempts: 5,
}, },
} }
d.ID = dlv.ID d.ID = dlv.ID
@@ -2060,11 +2015,6 @@ func TestDeliverSlack_WithRetries_SchedulesRetry(
e := testEngine(t, 1) e := testEngine(t, 1)
targetID := uuid.New().String() targetID := uuid.New().String()
slackCfg, err := json.Marshal(
delivery.SlackTargetConfig{WebhookURL: ts.URL},
)
require.NoError(t, err)
event := seedEvent(t, db, `{"slack":"retry"}`) event := seedEvent(t, db, `{"slack":"retry"}`)
dlv := seedDelivery( dlv := seedDelivery(
@@ -2073,15 +2023,15 @@ func TestDeliverSlack_WithRetries_SchedulesRetry(
) )
d := buildSlackRetryDelivery( d := buildSlackRetryDelivery(
dlv, event, targetID, string(slackCfg), dlv, event, targetID, ts.URL,
) )
task := &delivery.Task{ task := &delivery.Task{
DeliveryID: dlv.ID, DeliveryID: dlv.ID,
TargetID: targetID, TargetID: targetID,
TargetType: database.TargetTypeSlack, TargetType: database.TargetTypeSlack,
MaxRetries: 5, MaxAttempts: 5,
AttemptNum: 1, AttemptNum: 1,
} }
e.ExportProcessDelivery(context.TODO(), db, d, task) e.ExportProcessDelivery(context.TODO(), db, d, task)
+4 -6
View File
@@ -69,7 +69,6 @@ func TestEventDBHoldsNoTargetRows(t *testing.T) {
)) ))
defer ts.Close() defer ts.Close()
cfg := iHTTPConfig(ts.URL)
targetID := uuid.New().String() targetID := uuid.New().String()
dbPath := s.DBMgr.DBPath(s.WebhookID) dbPath := s.DBMgr.DBPath(s.WebhookID)
@@ -85,7 +84,7 @@ func TestEventDBHoldsNoTargetRows(t *testing.T) {
) )
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"leaky-target", cfg, 5, 1, &body, "leaky-target", ts.URL, 5, 1, &body,
) )
s.Engine.ExportProcessNewTask(context.TODO(), &task) s.Engine.ExportProcessNewTask(context.TODO(), &task)
@@ -102,7 +101,7 @@ func TestEventDBHoldsNoTargetRows(t *testing.T) {
// https://git.eeqj.de/sneak/webhooker/issues/107. // https://git.eeqj.de/sneak/webhooker/issues/107.
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "leaky-target", t, s.MainDB, targetID, s.WebhookID, "leaky-target",
database.TargetTypeHTTP, cfg, 5, database.TargetTypeHTTP, ts.URL, 5,
) )
rd := iSeedDelivery( rd := iSeedDelivery(
@@ -111,7 +110,7 @@ func TestEventDBHoldsNoTargetRows(t *testing.T) {
) )
rTask := iTask( rTask := iTask(
rd, event, s.WebhookID, targetID, rd, event, s.WebhookID, targetID,
"leaky-target", cfg, 5, 2, &body, "leaky-target", ts.URL, 5, 2, &body,
) )
s.Engine.ExportProcessRetryTask(context.TODO(), &rTask) s.Engine.ExportProcessRetryTask(context.TODO(), &rTask)
@@ -138,7 +137,6 @@ func TestEventDBHoldsNoTargetRowsOnFailedDelivery(t *testing.T) {
)) ))
defer ts.Close() defer ts.Close()
cfg := iHTTPConfig(ts.URL)
targetID := uuid.New().String() targetID := uuid.New().String()
event := iSeedEvent( event := iSeedEvent(
@@ -152,7 +150,7 @@ func TestEventDBHoldsNoTargetRowsOnFailedDelivery(t *testing.T) {
) )
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"leaky-target", cfg, 0, 1, &body, "leaky-target", ts.URL, 1, 1, &body,
) )
s.Engine.ExportProcessNewTask(context.TODO(), &task) s.Engine.ExportProcessNewTask(context.TODO(), &task)
+12 -25
View File
@@ -93,17 +93,6 @@ func (s *tsSink) text(t *testing.T) string {
} }
} }
func tsSlackConfig(t *testing.T, url string) string {
t.Helper()
data, err := json.Marshal(
delivery.SlackTargetConfig{WebhookURL: url},
)
require.NoError(t, err)
return string(data)
}
// tsSeedEvent writes an event whose CreatedAt is tsEventCreatedAt // tsSeedEvent writes an event whose CreatedAt is tsEventCreatedAt
// rather than the write time, so an assertion on the rendered // rather than the write time, so an assertion on the rendered
// timestamp cannot pass by accident against "roughly now". // timestamp cannot pass by accident against "roughly now".
@@ -142,16 +131,17 @@ func tsSeedEvent(
// tsSeedTarget writes the slack target row into the main database. // tsSeedTarget writes the slack target row into the main database.
// The retry path confirms the target still exists before sending. // The retry path confirms the target still exists before sending.
func tsSeedTarget( func tsSeedTarget(
t *testing.T, mainDB *gorm.DB, webhookID, config string, t *testing.T, mainDB *gorm.DB, webhookID, webhookURL string,
) database.Target { ) database.Target {
t.Helper() t.Helper()
target := database.Target{ target := database.Target{
WebhookID: webhookID, WebhookID: webhookID,
Name: "slack-sink", Name: "slack-sink",
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
Config: config, URL: webhookURL,
Active: true, Active: true,
MaxAttempts: 1,
} }
require.NoError(t, mainDB.Create(&target).Error) require.NoError(t, mainDB.Create(&target).Error)
@@ -175,8 +165,8 @@ func tsTask(
TargetID: target.ID, TargetID: target.ID,
TargetName: target.Name, TargetName: target.Name,
TargetType: database.TargetTypeSlack, TargetType: database.TargetTypeSlack,
TargetConfig: target.Config, TargetURL: target.URL,
MaxRetries: 0, MaxAttempts: 1,
Method: event.Method, Method: event.Method,
Headers: event.Headers, Headers: event.Headers,
ContentType: event.ContentType, ContentType: event.ContentType,
@@ -225,8 +215,7 @@ func (c tsCase) run(t *testing.T) (iSetup, database.Delivery, string) {
s := newISetup(t) s := newISetup(t)
sink := newTSSink(t) sink := newTSSink(t)
cfg := tsSlackConfig(t, sink.URL) target := tsSeedTarget(t, s.MainDB, s.WebhookID, sink.URL)
target := tsSeedTarget(t, s.MainDB, s.WebhookID, cfg)
event := tsSeedEvent(t, s.WebhookDB, s.WebhookID) event := tsSeedEvent(t, s.WebhookDB, s.WebhookID)
d := iSeedDelivery( d := iSeedDelivery(
@@ -337,8 +326,7 @@ func TestFormatSlackMessageOverTaskReconstructedEvent(
s := newISetup(t) s := newISetup(t)
cfg := tsSlackConfig(t, tsUndeliverableHook) target := tsSeedTarget(t, s.MainDB, s.WebhookID, tsUndeliverableHook)
target := tsSeedTarget(t, s.MainDB, s.WebhookID, cfg)
event := tsSeedEvent(t, s.WebhookDB, s.WebhookID) event := tsSeedEvent(t, s.WebhookDB, s.WebhookID)
d := iSeedDelivery( d := iSeedDelivery(
@@ -409,8 +397,7 @@ func TestEventReconstructionSurvivesAReapedRow(t *testing.T) {
s := newISetup(t) s := newISetup(t)
cfg := tsSlackConfig(t, tsUndeliverableHook) target := tsSeedTarget(t, s.MainDB, s.WebhookID, tsUndeliverableHook)
target := tsSeedTarget(t, s.MainDB, s.WebhookID, cfg)
event := tsSeedEvent(t, s.WebhookDB, s.WebhookID) event := tsSeedEvent(t, s.WebhookDB, s.WebhookID)
d := iSeedDelivery( d := iSeedDelivery(
+41 -320
View File
@@ -12,16 +12,11 @@ import (
"github.com/prometheus/client_golang/prometheus" "github.com/prometheus/client_golang/prometheus"
"go.uber.org/fx" "go.uber.org/fx"
"gorm.io/gorm" "gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/metrics" "sneak.berlin/go/webhooker/internal/metrics"
) )
// ErrExportArchiveWriterEvicted exposes the sentinel returned by
// an evicted archive writer. It carries the Err prefix rather
// than this file's usual Export one because it is a sentinel
// error.
var ErrExportArchiveWriterEvicted = errArchiveWriterEvicted
// Exported constants for test access. // Exported constants for test access.
const ( const (
ExportDeliveryChannelSize = deliveryChannelSize ExportDeliveryChannelSize = deliveryChannelSize
@@ -82,10 +77,10 @@ func ExportIsForwardableHeader(name string) bool {
func ExportApplyRequestHeaders( func ExportApplyRequestHeaders(
req *http.Request, req *http.Request,
event *database.Event, event *database.Event,
cfg *HTTPTargetConfig, configured map[string]string,
userAgent string, userAgent string,
) []string { ) []string {
return applyRequestHeaders(req, event, cfg, userAgent) return applyRequestHeaders(req, event, configured, userAgent)
} }
// ExportTruncate exposes truncate for testing. // ExportTruncate exposes truncate for testing.
@@ -142,9 +137,10 @@ func (e *Engine) ExportDeliverSlack(
d *database.Delivery, d *database.Delivery,
) { ) {
task := &Task{ task := &Task{
DeliveryID: d.ID, DeliveryID: d.ID,
TargetID: d.TargetID, TargetID: d.TargetID,
AttemptNum: 1, MaxAttempts: d.Target.MaxAttempts,
AttemptNum: 1,
} }
e.targets[database.TargetTypeSlack].Deliver( e.targets[database.TargetTypeSlack].Deliver(
@@ -220,37 +216,23 @@ func (e *Engine) ExportSetSlackCircuitBreaker(
e.slackTarget.circuitBreakers.Store(targetID, cb) e.slackTarget.circuitBreakers.Store(targetID, cb)
} }
// ExportParseHTTPConfig exposes parseHTTPConfig.
func (e *Engine) ExportParseHTTPConfig(
configJSON string,
) (*HTTPTargetConfig, error) {
return parseHTTPConfig(configJSON)
}
// ExportParseSlackConfig exposes parseSlackConfig.
func (e *Engine) ExportParseSlackConfig(
configJSON string,
) (*SlackTargetConfig, error) {
return parseSlackConfig(configJSON)
}
// ExportDoHTTPRequest exposes the http target's // ExportDoHTTPRequest exposes the http target's
// doHTTPRequest. // doHTTPRequest.
func (e *Engine) ExportDoHTTPRequest( func (e *Engine) ExportDoHTTPRequest(
ctx context.Context, ctx context.Context,
cfg *HTTPTargetConfig, target *database.Target,
event *database.Event, event *database.Event,
) (int, string, int64, error) { ) (int, string, int64, error) {
return e.httpTarget.doHTTPRequest(ctx, cfg, event) return e.httpTarget.doHTTPRequest(ctx, target, event)
} }
// ExportClientForRequest exposes the http target's // ExportClientForRequest exposes the http target's
// clientForRequest. // clientForRequest.
func (e *Engine) ExportClientForRequest( func (e *Engine) ExportClientForRequest(
cfg *HTTPTargetConfig, timeoutSeconds int,
originScoped []string, originScoped []string,
) *http.Client { ) *http.Client {
return e.httpTarget.clientForRequest(cfg, originScoped) return e.httpTarget.clientForRequest(timeoutSeconds, originScoped)
} }
// ErrExportTooManyRedirects exposes the sentinel the redirect // ErrExportTooManyRedirects exposes the sentinel the redirect
@@ -285,32 +267,27 @@ func (e *Engine) ExportRecoverPendingDeliveries(
ctx context.Context, ctx context.Context,
webhookDB *gorm.DB, webhookDB *gorm.DB,
webhookID string, webhookID string,
atStart bool,
) { ) {
e.recoverPendingDeliveries( e.recoverPendingDeliveries(
ctx, webhookDB, webhookID, ctx, webhookDB, webhookID, atStart,
) )
} }
// ExportRecoverWebhookDeliveries exposes // ExportRecoverWebhookDeliveries exposes
// recoverWebhookDeliveries. // recoverWebhookDeliveries.
func (e *Engine) ExportRecoverWebhookDeliveries( func (e *Engine) ExportRecoverWebhookDeliveries(
ctx context.Context, webhookID string, ctx context.Context, webhookID string, atStart bool,
) { ) {
e.recoverWebhookDeliveries(ctx, webhookID) e.recoverWebhookDeliveries(ctx, webhookID, atStart)
} }
// ExportRecoverInFlight exposes recoverInFlight. // ExportRecoverDeliveries exposes recoverDeliveries, the recovery
func (e *Engine) ExportRecoverInFlight( // routine.
ctx context.Context, func (e *Engine) ExportRecoverDeliveries(
ctx context.Context, atStart bool,
) { ) {
e.recoverInFlight(ctx) e.recoverDeliveries(ctx, atStart)
}
// ExportSweepWebhookRetries exposes sweepWebhookRetries.
func (e *Engine) ExportSweepWebhookRetries(
ctx context.Context, webhookID string,
) {
e.sweepWebhookRetries(ctx, webhookID)
} }
// ExportStart exposes start for testing. // ExportStart exposes start for testing.
@@ -357,9 +334,9 @@ func (e *Engine) ExportRetainDelivery(deliveryID string) bool {
// ExportRecoverRetryingDeliveries exposes recoverRetryingDeliveries. // ExportRecoverRetryingDeliveries exposes recoverRetryingDeliveries.
func (e *Engine) ExportRecoverRetryingDeliveries( func (e *Engine) ExportRecoverRetryingDeliveries(
webhookDB *gorm.DB, webhookID string, ctx context.Context, webhookDB *gorm.DB, webhookID string,
) { ) {
e.recoverRetryingDeliveries(webhookDB, webhookID) e.recoverRetryingDeliveries(ctx, webhookDB, webhookID)
} }
// ExportFailMissingTarget exposes failMissingTarget, so a test can hand // ExportFailMissingTarget exposes failMissingTarget, so a test can hand
@@ -398,7 +375,8 @@ func (e *Engine) ExportRetryCh() chan Task {
} }
// NewTestEngine creates an Engine for unit tests without // NewTestEngine creates an Engine for unit tests without
// database dependencies. // database dependencies. Its archive writers have no data
// directory, so a delivery to a database target fails.
func NewTestEngine( func NewTestEngine(
log *slog.Logger, log *slog.Logger,
client *http.Client, client *http.Client,
@@ -410,6 +388,7 @@ func NewTestEngine(
retryCh: make(chan Task, retryChannelSize), retryCh: make(chan Task, retryChannelSize),
workers: workers, workers: workers,
mtr: metrics.New(prometheus.NewRegistry()), mtr: metrics.New(prometheus.NewRegistry()),
archives: archive.NewWriters(archive.WritersParams{Logger: log}),
} }
e.initTargets(client) e.initTargets(client)
@@ -422,9 +401,10 @@ func NewTestEngineSmallRetry(
log *slog.Logger, log *slog.Logger,
) *Engine { ) *Engine {
e := &Engine{ e := &Engine{
log: log, log: log,
retryCh: make(chan Task, 1), retryCh: make(chan Task, 1),
mtr: metrics.New(prometheus.NewRegistry()), mtr: metrics.New(prometheus.NewRegistry()),
archives: archive.NewWriters(archive.WritersParams{Logger: log}),
} }
e.initTargets(nil) e.initTargets(nil)
@@ -432,7 +412,8 @@ func NewTestEngineSmallRetry(
} }
// NewTestEngineWithDB creates an Engine with a real // NewTestEngineWithDB creates an Engine with a real
// database and dbManager for integration tests. // database and dbManager for integration tests, and archive
// writers of its own over the same two.
func NewTestEngineWithDB( func NewTestEngineWithDB(
db *database.Database, db *database.Database,
dbMgr *database.WebhookDBManager, dbMgr *database.WebhookDBManager,
@@ -448,12 +429,22 @@ func NewTestEngineWithDB(
retryCh: make(chan Task, retryChannelSize), retryCh: make(chan Task, retryChannelSize),
workers: workers, workers: workers,
mtr: metrics.New(prometheus.NewRegistry()), mtr: metrics.New(prometheus.NewRegistry()),
archives: archive.NewWriters(archive.WritersParams{
Database: db,
DBManager: dbMgr,
Logger: log,
}),
} }
e.initTargets(client) e.initTargets(client)
return e return e
} }
// ExportArchives returns the engine's archive writers.
func (e *Engine) ExportArchives() *archive.Writers {
return e.archives
}
// ExportSetMetrics substitutes the engine's metric set, so a test can // ExportSetMetrics substitutes the engine's metric set, so a test can
// assert on collectors registered on a registry it holds. // assert on collectors registered on a registry it holds.
func (e *Engine) ExportSetMetrics(mtr *metrics.Set) { func (e *Engine) ExportSetMetrics(mtr *metrics.Set) {
@@ -476,273 +467,3 @@ func NewTestCircuitBreaker(
cooldown: cooldown, cooldown: cooldown,
} }
} }
// ExportArchivedEvent aliases the archive row type so black-box
// tests can construct and read archive rows.
type ExportArchivedEvent = archivedEvent
// ExportArchiveWriter wraps an archiveWriter so black-box tests
// can exercise the archive file mechanics.
type ExportArchiveWriter struct {
w *archiveWriter
}
// NewExportArchiveWriter builds an archive writer for tests,
// optionally overriding the reopen debounce (a non-positive
// debounce keeps the production default).
func NewExportArchiveWriter(
path string, log *slog.Logger, debounce time.Duration,
) *ExportArchiveWriter {
w := newArchiveWriter(path, log)
if debounce > 0 {
w.debounce = debounce
}
return &ExportArchiveWriter{w: w}
}
// Write archives a row through the writer, into the file named
// without a period.
func (e *ExportArchiveWriter) Write(
row ExportArchivedEvent, expiry time.Duration,
) error {
return e.w.write(row, expiry, "")
}
// WritePeriod archives a row through the writer, into the file for
// period.
func (e *ExportArchiveWriter) WritePeriod(
row ExportArchivedEvent, expiry time.Duration, period string,
) error {
return e.w.write(row, expiry, period)
}
// Open opens the archive file, pruning when expiry is positive.
func (e *ExportArchiveWriter) Open(expiry time.Duration) error {
return e.w.open(e.w.path, expiry)
}
// Reopen closes and reopens the archive file.
func (e *ExportArchiveWriter) Reopen(
expiry time.Duration,
) error {
return e.w.reopen(e.w.path, expiry)
}
// SetNow replaces the clock the writer measures its reopen
// debounce on.
func (e *ExportArchiveWriter) SetNow(now func() time.Time) {
e.w.now = now
}
// Reopens reports how many times the file has been opened.
func (e *ExportArchiveWriter) Reopens() int {
return e.w.reopens
}
// DB returns the writer's current open handle for row
// inspection in tests.
func (e *ExportArchiveWriter) DB() *gorm.DB {
return e.w.db
}
// Path returns the archive file the writer owns.
func (e *ExportArchiveWriter) Path() string {
return e.w.path
}
// OpenExisting opens the archive without permitting creation,
// the way the idle sweep does.
func (e *ExportArchiveWriter) OpenExisting(
expiry time.Duration,
) error {
return e.w.openMode(e.w.path, archiveModeExisting, expiry)
}
// SweepExpired runs an idle sweep of the archive.
func (e *ExportArchiveWriter) SweepExpired(
expiry time.Duration,
) error {
return e.w.sweepExpired(expiry)
}
// Evict marks the writer evicted and closes its handle, exactly
// as leaving the registry does.
func (e *ExportArchiveWriter) Evict() {
e.w.evict()
}
// Rename gives the archive file a new name in the same directory,
// as a rename of the webhook or target does.
func (e *ExportArchiveWriter) Rename(name string) error {
return e.w.rename(name)
}
// HandleOpen reports whether the writer currently holds an open
// archive handle.
func (e *ExportArchiveWriter) HandleOpen() bool {
e.w.mu.Lock()
defer e.w.mu.Unlock()
return e.w.db != nil
}
// Same reports whether both wrappers refer to the very same
// underlying archive writer, so a test can prove a registry entry
// is the writer it was handed rather than a replacement.
func (e *ExportArchiveWriter) Same(
other *ExportArchiveWriter,
) bool {
return other != nil && e.w == other.w
}
// ExportArchiveWriterFor returns the archive writer the registry
// currently caches for a database target, or nil when none is
// cached. It never creates one, so a test can hold a reference to
// the very writer an eviction is about to detach.
func (e *Engine) ExportArchiveWriterFor(
targetID string,
) *ExportArchiveWriter {
e.dbTarget.mu.Lock()
defer e.dbTarget.mu.Unlock()
w, ok := e.dbTarget.writers[targetID]
if !ok {
return nil
}
return &ExportArchiveWriter{w: w}
}
// ExportHasArchiveWriter reports whether the database target
// type currently caches an archive writer for a target.
func (e *Engine) ExportHasArchiveWriter(
targetID string,
) bool {
e.dbTarget.mu.Lock()
defer e.dbTarget.mu.Unlock()
_, ok := e.dbTarget.writers[targetID]
return ok
}
// ExportArchiveHandleOpen reports whether the cached archive
// writer for a target holds an open database handle. It
// returns false when no writer is cached.
func (e *Engine) ExportArchiveHandleOpen(
targetID string,
) bool {
e.dbTarget.mu.Lock()
w, ok := e.dbTarget.writers[targetID]
e.dbTarget.mu.Unlock()
if !ok {
return false
}
w.mu.Lock()
defer w.mu.Unlock()
return w.db != nil
}
// ExportEnsureArchiveWriter creates (if needed) and returns the
// archive file path of the cached writer for a target, so a
// test can prime the registry the way a delivery would.
func (e *Engine) ExportEnsureArchiveWriter(
targetID string,
) (string, error) {
w, err := e.dbTarget.writerFor(targetID)
if err != nil {
return "", err
}
return w.path, nil
}
// ExportSweepWriterFor takes a target's registry writer exactly
// as the idle sweep does, reporting whether the sweep had to
// create the entry. It lets a test drive the registry through the
// sweep's own entry point instead of choreographing goroutines.
func (e *Engine) ExportSweepWriterFor(
targetID string,
) (*ExportArchiveWriter, bool, error) {
w, created, err := e.dbTarget.sweepWriterFor(targetID)
if err != nil {
return nil, false, err
}
return &ExportArchiveWriter{w: w}, created, nil
}
// ExportReleaseSweepWriter releases a sweep-created registry entry
// exactly as a finished sweep does.
func (e *Engine) ExportReleaseSweepWriter(
targetID string, w *ExportArchiveWriter,
) {
e.dbTarget.releaseSweepWriter(targetID, w.w)
}
// NewTestArchiveSweeper builds an ArchiveSweeper backed by the
// given main database and engine, without the fx lifecycle.
// Intended for tests.
func NewTestArchiveSweeper(
db *database.Database,
eng *Engine,
log *slog.Logger,
) *ArchiveSweeper {
return &ArchiveSweeper{
db: db,
eng: eng,
log: log,
interval: time.Hour,
}
}
// ExportSweep runs a single archive sweep synchronously for
// tests.
func (s *ArchiveSweeper) ExportSweep(ctx context.Context) {
s.sweep(ctx)
}
// ExportStart starts the sweeper's background loop for tests.
func (s *ArchiveSweeper) ExportStart() {
s.start()
}
// ExportRegisterHooks registers the sweeper's real fx lifecycle
// hooks on a lifecycle supplied by a test, so a test can drive
// the exact OnStart/OnStop functions the application runs and
// hand OnStart the kind of context fx actually supplies.
func (s *ArchiveSweeper) ExportRegisterHooks(lc fx.Lifecycle) {
s.registerHooks(lc)
}
// ExportStop stops the sweeper's background loop for tests.
func (s *ArchiveSweeper) ExportStop(ctx context.Context) error {
return s.stop(ctx)
}
// ExportWedgeLoop adds a goroutine to the sweeper's WaitGroup
// that never observes cancellation and returns only when release
// is closed. It stands in for a prune stuck on a locked archive.
func (s *ArchiveSweeper) ExportWedgeLoop(
release <-chan struct{},
) {
s.wg.Go(func() {
<-release
})
}
// ExportSetInterval overrides the sweep interval for tests.
func (s *ArchiveSweeper) ExportSetInterval(d time.Duration) {
s.interval = d
}
// ExportParseArchiveExpiry exposes parseArchiveExpiry.
func ExportParseArchiveExpiry(
configJSON string,
) (time.Duration, error) {
return parseArchiveExpiry(configJSON)
}
+9 -9
View File
@@ -6,10 +6,10 @@ import "sync"
// //
// A delivery is owned from the moment a task for it is handed to a // A delivery is owned from the moment a task for it is handed to a
// channel or to a retry timer until the engine has no further plan for // channel or to a retry timer until the engine has no further plan for
// it in memory. Restart recovery and both arms of the periodic sweep // it in memory. The recovery routine, at start and on every tick of the
// re-dispatch only deliveries the set does not hold, which is what // sweep, re-dispatches only deliveries the set does not hold, which is
// makes them exact rather than a guess about how long a row has sat at // what makes it exact rather than a guess about how long a row has sat
// pending. // at pending.
// //
// This replaces reasoning from timestamps. A delivery's row says // This replaces reasoning from timestamps. A delivery's row says
// pending from creation until its outcome is written, which covers // pending from creation until its outcome is written, which covers
@@ -25,8 +25,8 @@ import "sync"
// In-memory state is sufficient because a data directory admits one // In-memory state is sufficient because a data directory admits one
// process: internal/datadir takes an flock on it at startup and a // process: internal/datadir takes an flock on it at startup and a
// second instance refuses to run. Deliveries owned by a process that // second instance refuses to run. Deliveries owned by a process that
// died are not in any successor's set, and restart recovery is what // died are not in any successor's set, and the recovery routine's run
// picks those up. // at start is what picks those up.
// //
// References are counted rather than held as a plain set because // References are counted rather than held as a plain set because
// ownership outlives the worker that took it. A target that schedules // ownership outlives the worker that took it. A target that schedules
@@ -61,9 +61,9 @@ func (s *inflightSet) retain(deliveryID string) {
// retainIdle takes the first reference on a delivery, and reports // retainIdle takes the first reference on a delivery, and reports
// whether it got it. It fails when the engine already owns the // whether it got it. It fails when the engine already owns the
// delivery, which is what makes two claimants — restart recovery and // delivery, which is what makes two claimants — Notify and the
// the sweep run concurrently, or two sweep arms — mutually exclusive // recovery routine, say — mutually exclusive rather than merely
// rather than merely atomic. // atomic.
func (s *inflightSet) retainIdle(deliveryID string) bool { func (s *inflightSet) retainIdle(deliveryID string) bool {
s.mu.Lock() s.mu.Lock()
defer s.mu.Unlock() defer s.mu.Unlock()
+37 -52
View File
@@ -2,8 +2,6 @@ package delivery_test
import ( import (
"context" "context"
"path/filepath"
"strings"
"sync" "sync"
"testing" "testing"
"time" "time"
@@ -32,7 +30,7 @@ func fSweepSetup(
iCreateTarget(t, s.MainDB, targetID, iCreateTarget(t, s.MainDB, targetID,
s.WebhookID, name, s.WebhookID, name,
database.TargetTypeLog, "", 0, database.TargetTypeLog, "", 1,
) )
require.NoError(t, s.MainDB.Create(&database.Webhook{ require.NoError(t, s.MainDB.Create(&database.Webhook{
@@ -66,33 +64,6 @@ func fDrain(e *delivery.Engine) []delivery.Task {
} }
} }
// TestArchiveHandleIsWAL closes the last gap in the durability
// evidence: the main and per-webhook tiers each assert their journal
// mode on a live handle, and the archive tier gets its settings from
// the same code path but nothing checked the running file.
func TestArchiveHandleIsWAL(t *testing.T) {
t.Parallel()
w := delivery.NewExportArchiveWriter(
filepath.Join(t.TempDir(), "archive-wal.db"),
archiveTestLogger(), 0,
)
require.NoError(t, w.Open(0))
var mode string
row := w.DB().Raw("pragma journal_mode").Row()
require.NoError(t, row.Scan(&mode))
assert.Equal(t, "wal", strings.ToLower(mode))
var busy string
row = w.DB().Raw("pragma busy_timeout").Row()
require.NoError(t, row.Scan(&busy))
assert.Equal(t, "10000", busy)
}
// TestSweepLeavesAQueuedDeliveryAlone is the case the age bound cannot // TestSweepLeavesAQueuedDeliveryAlone is the case the age bound cannot
// see. The delivery is queued and untouched, so its row is arbitrarily // see. The delivery is queued and untouched, so its row is arbitrarily
// old and still perfectly healthy; only ownership distinguishes it // old and still perfectly healthy; only ownership distinguishes it
@@ -124,8 +95,8 @@ func TestSweepLeavesAQueuedDeliveryAlone(t *testing.T) {
require.Equal(t, 1, s.Engine.ExportInflightHeld()) require.Equal(t, 1, s.Engine.ExportInflightHeld())
s.Engine.ExportSweepWebhookRetries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, onTick,
) )
tasks := fDrain(s.Engine) tasks := fDrain(s.Engine)
@@ -136,9 +107,11 @@ func TestSweepLeavesAQueuedDeliveryAlone(t *testing.T) {
) )
} }
// TestRecoveryAndSweepDoNotDoubleDispatch drives the two entry points // TestRecoveryAndSweepDoNotDoubleDispatch drives the recovery routine's
// the engine starts concurrently against one aged pending row. Before // run at start and a tick of the sweep at once against one aged pending
// ownership they both dispatched it. // row. The engine runs them one after the other, but Notify can race
// either, and ownership is what keeps any two claimants from both
// sending. Before ownership the two runs both dispatched the row.
func TestRecoveryAndSweepDoNotDoubleDispatch(t *testing.T) { func TestRecoveryAndSweepDoNotDoubleDispatch(t *testing.T) {
t.Parallel() t.Parallel()
@@ -162,12 +135,12 @@ func TestRecoveryAndSweepDoNotDoubleDispatch(t *testing.T) {
wg.Go(func() { wg.Go(func() {
s.Engine.ExportRecoverPendingDeliveries( s.Engine.ExportRecoverPendingDeliveries(
ctx, s.WebhookDB, s.WebhookID, ctx, s.WebhookDB, s.WebhookID, atStart,
) )
}) })
wg.Go(func() { wg.Go(func() {
s.Engine.ExportSweepWebhookRetries( s.Engine.ExportRecoverWebhookDeliveries(
ctx, s.WebhookID, ctx, s.WebhookID, onTick,
) )
}) })
wg.Wait() wg.Wait()
@@ -229,7 +202,7 @@ func TestOwnershipIsReleasedAfterDelivery(t *testing.T) {
iCreateTarget(t, s.MainDB, targetID, iCreateTarget(t, s.MainDB, targetID,
s.WebhookID, "released", s.WebhookID, "released",
database.TargetTypeLog, "", 0, database.TargetTypeLog, "", 1,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -276,9 +249,10 @@ func TestOwnershipIsReleasedAfterDelivery(t *testing.T) {
// TestNotifyAfterRecoveryDoesNotSendAgain is the startup race of // TestNotifyAfterRecoveryDoesNotSendAgain is the startup race of
// https://git.eeqj.de/sneak/webhooker/issues/299. The receiver has // https://git.eeqj.de/sneak/webhooker/issues/299. The receiver has
// written a delivery, restart recovery finds it pending, sends it and // written a delivery, the recovery routine's run at start finds it
// releases it, and only then does the receiver's Notify for it arrive. // pending, sends it and releases it, and only then does the receiver's
// Nothing owns the delivery by then, so Notify takes it. // Notify for it arrive. Nothing owns the delivery by then, so Notify
// takes it.
func TestNotifyAfterRecoveryDoesNotSendAgain(t *testing.T) { func TestNotifyAfterRecoveryDoesNotSendAgain(t *testing.T) {
t.Parallel() t.Parallel()
@@ -302,7 +276,8 @@ func TestNotifyAfterRecoveryDoesNotSendAgain(t *testing.T) {
) )
}() }()
// Restart recovery sends the delivery and lets it go. // The recovery routine's run at start sends the delivery and lets
// it go.
iWaitForDelivered(t, s.WebhookDB, d.ID) iWaitForDelivered(t, s.WebhookDB, d.ID)
require.Eventually( require.Eventually(
t, t,
@@ -363,7 +338,7 @@ func TestRetryingRecoverySkipsASuccessfulResult(t *testing.T) {
rSeedResult(t, s.WebhookDB, d.ID, 2, true) rSeedResult(t, s.WebhookDB, d.ID, 2, true)
s.Engine.ExportRecoverRetryingDeliveries( s.Engine.ExportRecoverRetryingDeliveries(
s.WebhookDB, s.WebhookID, context.Background(), s.WebhookDB, s.WebhookID,
) )
assert.Empty( assert.Empty(
@@ -378,8 +353,8 @@ func TestRetryingRecoverySkipsASuccessfulResult(t *testing.T) {
) )
} }
// TestRetryingSweepSkipsASuccessfulResult is the same rule on the // TestRetryingSweepSkipsASuccessfulResult is the same rule on a tick of
// periodic sweep's retrying arm. // the sweep.
func TestRetryingSweepSkipsASuccessfulResult(t *testing.T) { func TestRetryingSweepSkipsASuccessfulResult(t *testing.T) {
t.Parallel() t.Parallel()
@@ -397,8 +372,8 @@ func TestRetryingSweepSkipsASuccessfulResult(t *testing.T) {
rSeedResult(t, s.WebhookDB, d.ID, 1, false) rSeedResult(t, s.WebhookDB, d.ID, 1, false)
rSeedResult(t, s.WebhookDB, d.ID, 2, true) rSeedResult(t, s.WebhookDB, d.ID, 2, true)
s.Engine.ExportSweepWebhookRetries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, onTick,
) )
assert.Empty(t, fDrain(s.Engine)) assert.Empty(t, fDrain(s.Engine))
@@ -449,8 +424,8 @@ func TestScheduledRetryIsNotSweptDuringBackoff(t *testing.T) {
require.Equal(t, 1, s.Engine.ExportInflightHeld()) require.Equal(t, 1, s.Engine.ExportInflightHeld())
s.Engine.ExportSweepWebhookRetries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, onTick,
) )
assert.Empty( assert.Empty(
@@ -462,7 +437,8 @@ func TestScheduledRetryIsNotSweptDuringBackoff(t *testing.T) {
// TestRedispatchStampsTheRow pins the cadence control: a stranded // TestRedispatchStampsTheRow pins the cadence control: a stranded
// delivery that has just been handed out is not selected again by the // delivery that has just been handed out is not selected again by the
// next tick a minute later. // next tick a minute later. The stamp is stored in the same form as
// created_at.
func TestRedispatchStampsTheRow(t *testing.T) { func TestRedispatchStampsTheRow(t *testing.T) {
t.Parallel() t.Parallel()
@@ -481,7 +457,7 @@ func TestRedispatchStampsTheRow(t *testing.T) {
ctx := context.Background() ctx := context.Background()
s.Engine.ExportSweepWebhookRetries(ctx, s.WebhookID) s.Engine.ExportRecoverWebhookDeliveries(ctx, s.WebhookID, onTick)
require.Len(t, fDrain(s.Engine), 1) require.Len(t, fDrain(s.Engine), 1)
var row database.Delivery var row database.Delivery
@@ -493,4 +469,13 @@ func TestRedispatchStampsTheRow(t *testing.T) {
"a re-dispatched delivery must be stamped so the "+ "a re-dispatched delivery must be stamped so the "+
"next tick does not select it again", "next tick does not select it again",
) )
var createdAt, updatedAt string
require.NoError(t, s.WebhookDB.Raw(
`SELECT CAST(created_at AS TEXT), CAST(updated_at AS TEXT)
FROM deliveries WHERE id = ?`, d.ID,
).Row().Scan(&createdAt, &updatedAt))
assertStoredTime(t, createdAt)
assertStoredTime(t, updatedAt)
} }
+17 -20
View File
@@ -192,7 +192,7 @@ func mDeliverOK(t *testing.T, s iSetup) {
body := event.Body body := event.Body
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"metrics-ok", iHTTPConfig(ts.URL), 3, 1, &body, "metrics-ok", ts.URL, 3, 1, &body,
) )
s.Engine.ExportProcessNewTask(context.TODO(), &task) s.Engine.ExportProcessNewTask(context.TODO(), &task)
@@ -226,18 +226,17 @@ func mExhaustRetries(t *testing.T, s iSetup) {
) )
body := event.Body body := event.Body
cfg := iHTTPConfig(ts.URL)
// The retry below is only run if its target still exists; see // The retry below is only run if its target still exists; see
// https://git.eeqj.de/sneak/webhooker/issues/107. // https://git.eeqj.de/sneak/webhooker/issues/107.
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "metrics-fail", t, s.MainDB, targetID, s.WebhookID, "metrics-fail",
database.TargetTypeHTTP, cfg, 2, database.TargetTypeHTTP, ts.URL, 2,
) )
first := iTask( first := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"metrics-fail", cfg, 2, 1, &body, "metrics-fail", ts.URL, 2, 1, &body,
) )
s.Engine.ExportProcessNewTask(context.TODO(), &first) s.Engine.ExportProcessNewTask(context.TODO(), &first)
@@ -251,7 +250,7 @@ func mExhaustRetries(t *testing.T, s iSetup) {
// deterministic and off the wall clock. // deterministic and off the wall clock.
second := iTask( second := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"metrics-fail", cfg, 2, 2, &body, "metrics-fail", ts.URL, 2, 2, &body,
) )
s.Engine.ExportProcessRetryTask( s.Engine.ExportProcessRetryTask(
@@ -289,22 +288,21 @@ func TestDeliveryMetrics_CircuitBreakerGauge(t *testing.T) {
) )
body := event.Body body := event.Body
cfg := iHTTPConfig(ts.URL)
// A retry budget above the failure threshold, so the breaker // An attempt budget above the failure threshold, so the breaker
// rather than the budget is what stops the delivery. // rather than the budget is what stops the delivery.
maxRetries := delivery.ExportDefaultFailureThreshold + 5 maxAttempts := delivery.ExportDefaultFailureThreshold + 5
// The retries below are only run if their target still exists; // The retries below are only run if their target still exists;
// see https://git.eeqj.de/sneak/webhooker/issues/107. // see https://git.eeqj.de/sneak/webhooker/issues/107.
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "metrics-trip", t, s.MainDB, targetID, s.WebhookID, "metrics-trip",
database.TargetTypeHTTP, cfg, maxRetries, database.TargetTypeHTTP, ts.URL, maxAttempts,
) )
first := iTask( first := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"metrics-trip", cfg, maxRetries, 1, &body, "metrics-trip", ts.URL, maxAttempts, 1, &body,
) )
s.Engine.ExportProcessNewTask(context.TODO(), &first) s.Engine.ExportProcessNewTask(context.TODO(), &first)
@@ -316,7 +314,7 @@ func TestDeliveryMetrics_CircuitBreakerGauge(t *testing.T) {
ExportDefaultFailureThreshold; attempt++ { ExportDefaultFailureThreshold; attempt++ {
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"metrics-trip", cfg, maxRetries, attempt, &body, "metrics-trip", ts.URL, maxAttempts, attempt, &body,
) )
s.Engine.ExportProcessRetryTask( s.Engine.ExportProcessRetryTask(
@@ -363,17 +361,16 @@ func TestDeliveryMetrics_BreakerBlockedIsNotAnAttempt(
) )
body := event.Body body := event.Body
cfg := iHTTPConfig(ts.URL) maxAttempts := delivery.ExportDefaultFailureThreshold + 5
maxRetries := delivery.ExportDefaultFailureThreshold + 5
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "metrics-blocked", t, s.MainDB, targetID, s.WebhookID, "metrics-blocked",
database.TargetTypeHTTP, cfg, maxRetries, database.TargetTypeHTTP, ts.URL, maxAttempts,
) )
first := iTask( first := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"metrics-blocked", cfg, maxRetries, 1, &body, "metrics-blocked", ts.URL, maxAttempts, 1, &body,
) )
s.Engine.ExportProcessNewTask(context.TODO(), &first) s.Engine.ExportProcessNewTask(context.TODO(), &first)
@@ -382,7 +379,7 @@ func TestDeliveryMetrics_BreakerBlockedIsNotAnAttempt(
ExportDefaultFailureThreshold; attempt++ { ExportDefaultFailureThreshold; attempt++ {
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"metrics-blocked", cfg, maxRetries, attempt, &body, "metrics-blocked", ts.URL, maxAttempts, attempt, &body,
) )
s.Engine.ExportProcessRetryTask(context.TODO(), &task) s.Engine.ExportProcessRetryTask(context.TODO(), &task)
@@ -405,7 +402,7 @@ func TestDeliveryMetrics_BreakerBlockedIsNotAnAttempt(
blocked := iTask( blocked := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"metrics-blocked", cfg, maxRetries, "metrics-blocked", ts.URL, maxAttempts,
delivery.ExportDefaultFailureThreshold+1, &body, delivery.ExportDefaultFailureThreshold+1, &body,
) )
@@ -444,8 +441,8 @@ func TestDeliveryMetrics_OrphanedRetryFailureLabelled(
t, s, database.TargetTypeLog, t, s, database.TargetTypeLog,
) )
s.Engine.ExportSweepWebhookRetries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, onTick,
) )
iAssertStatus(t, s.WebhookDB, deliveryID, iAssertStatus(t, s.WebhookDB, deliveryID,
@@ -474,7 +471,7 @@ func TestDeliveryMetrics_QueueDepthGauges(t *testing.T) {
iCreateTarget(t, s.MainDB, targetID, s.WebhookID, iCreateTarget(t, s.MainDB, targetID, s.WebhookID,
"queue-depth-target", database.TargetTypeHTTP, "queue-depth-target", database.TargetTypeHTTP,
iHTTPConfig("https://example.com/hook"), 3, "https://example.com/hook", 3,
) )
event := iSeedEvent( event := iSeedEvent(
+2 -1
View File
@@ -20,7 +20,8 @@ const queueDepthSampleInterval = 30 * time.Second
// to be seeded correctly at startup from rows written by a previous // to be seeded correctly at startup from rows written by a previous
// process, and would drift permanently on any transition that failed // process, and would drift permanently on any transition that failed
// to persist. Counting is the measurement that cannot go wrong, and // to persist. Counting is the measurement that cannot go wrong, and
// it is the same whole-database walk the retry sweep already makes. // it is the same whole-database walk each tick of the sweep already
// makes.
func (e *Engine) queueDepthSampler(ctx context.Context) { func (e *Engine) queueDepthSampler(ctx context.Context) {
defer e.wg.Done() defer e.wg.Done()
@@ -102,7 +102,7 @@ func TestQueueDepthSample_LogsNoBoundValue(t *testing.T) {
iCreateTarget(t, mainDB.DB(), targetID, webhookID, iCreateTarget(t, mainDB.DB(), targetID, webhookID,
"queue-depth-gormlog-target", database.TargetTypeHTTP, "queue-depth-gormlog-target", database.TargetTypeHTTP,
iHTTPConfig("https://example.com/hook"), 3, "https://example.com/hook", 3,
) )
event := iSeedEvent( event := iSeedEvent(
+27 -26
View File
@@ -66,7 +66,7 @@ func TestRecoverySkipsPendingWithSuccessfulResult(
iCreateTarget(t, s.MainDB, targetID, iCreateTarget(t, s.MainDB, targetID,
s.WebhookID, "already-delivered", s.WebhookID, "already-delivered",
database.TargetTypeLog, "", 0, database.TargetTypeLog, "", 1,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -88,7 +88,7 @@ func TestRecoverySkipsPendingWithSuccessfulResult(
) )
s.Engine.ExportRecoverPendingDeliveries( s.Engine.ExportRecoverPendingDeliveries(
context.Background(), s.WebhookDB, s.WebhookID, context.Background(), s.WebhookDB, s.WebhookID, atStart,
) )
select { select {
@@ -129,7 +129,7 @@ func TestRecoveryContinuesTheAttemptNumbering(t *testing.T) {
iCreateTarget(t, s.MainDB, targetID, iCreateTarget(t, s.MainDB, targetID,
s.WebhookID, "numbering", s.WebhookID, "numbering",
database.TargetTypeLog, "", 0, database.TargetTypeLog, "", 1,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -145,7 +145,7 @@ func TestRecoveryContinuesTheAttemptNumbering(t *testing.T) {
rSeedResult(t, s.WebhookDB, d.ID, 2, false) rSeedResult(t, s.WebhookDB, d.ID, 2, false)
s.Engine.ExportRecoverPendingDeliveries( s.Engine.ExportRecoverPendingDeliveries(
context.Background(), s.WebhookDB, s.WebhookID, context.Background(), s.WebhookDB, s.WebhookID, atStart,
) )
select { select {
@@ -183,8 +183,8 @@ func TestSweepRecoversStrandedPending(t *testing.T) {
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
s.Engine.ExportSweepWebhookRetries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, onTick,
) )
select { select {
@@ -230,7 +230,7 @@ func TestSweepClaimsAStrandedDeliveryOnlyOnce(t *testing.T) {
ctx := context.Background() ctx := context.Background()
s.Engine.ExportSweepWebhookRetries(ctx, s.WebhookID) s.Engine.ExportRecoverWebhookDeliveries(ctx, s.WebhookID, onTick)
select { select {
case task := <-s.Engine.ExportDeliveryCh(): case task := <-s.Engine.ExportDeliveryCh():
@@ -246,7 +246,7 @@ func TestSweepClaimsAStrandedDeliveryOnlyOnce(t *testing.T) {
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
s.Engine.ExportSweepWebhookRetries(ctx, s.WebhookID) s.Engine.ExportRecoverWebhookDeliveries(ctx, s.WebhookID, onTick)
select { select {
case task := <-s.Engine.ExportDeliveryCh(): case task := <-s.Engine.ExportDeliveryCh():
@@ -258,9 +258,9 @@ func TestSweepClaimsAStrandedDeliveryOnlyOnce(t *testing.T) {
} }
} }
// TestSweepSettlesStrandedPendingWithoutResending is the sweep's own // TestSweepSettlesStrandedPendingWithoutResending is the reconcile on a
// version of the reconcile: a stranded delivery holding a successful // tick of the sweep: a stranded delivery holding a successful result is
// result is settled where it stands, and the receiver hears nothing. // settled where it stands, and the receiver hears nothing.
func TestSweepSettlesStrandedPendingWithoutResending( func TestSweepSettlesStrandedPendingWithoutResending(
t *testing.T, t *testing.T,
) { ) {
@@ -280,8 +280,8 @@ func TestSweepSettlesStrandedPendingWithoutResending(
rSeedResult(t, s.WebhookDB, d.ID, 1, true) rSeedResult(t, s.WebhookDB, d.ID, 1, true)
rAgePending(t, s.WebhookDB, d.ID) rAgePending(t, s.WebhookDB, d.ID)
s.Engine.ExportSweepWebhookRetries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, onTick,
) )
select { select {
@@ -312,8 +312,9 @@ func TestSweepSettlesStrandedPendingWithoutResending(
// TestFailedResultWriteLeavesDeliveryRecoverable is the rule the // TestFailedResultWriteLeavesDeliveryRecoverable is the rule the
// targets now follow: a bookkeeping write that fails must not advance // targets now follow: a bookkeeping write that fails must not advance
// the status, because pending and retrying are the states the sweeps // the status, because pending and retrying are the states the recovery
// recover and delivered is a claim the database refused to record. // routine recovers and delivered is a claim the database refused to
// record.
func TestFailedResultWriteLeavesDeliveryRecoverable( func TestFailedResultWriteLeavesDeliveryRecoverable(
t *testing.T, t *testing.T,
) { ) {
@@ -354,16 +355,17 @@ func TestFailedResultWriteLeavesDeliveryRecoverable(
Status: database.DeliveryStatusPending, Status: database.DeliveryStatusPending,
Event: event, Event: event,
Target: database.Target{ Target: database.Target{
Name: "unwritable", Name: "unwritable",
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Config: iHTTPConfig(ts.URL), URL: ts.URL,
MaxAttempts: 1,
}, },
} }
full.ID = d.ID full.ID = d.ID
s.Engine.ExportDeliverHTTP( s.Engine.ExportDeliverHTTP(
context.Background(), s.WebhookDB, full, context.Background(), s.WebhookDB, full,
&delivery.Task{DeliveryID: d.ID, AttemptNum: 1}, &delivery.Task{DeliveryID: d.ID, MaxAttempts: 1, AttemptNum: 1},
) )
assert.Equal( assert.Equal(
@@ -444,10 +446,10 @@ func TestFailedResultWriteWithRetriesLeavesDeliveryRecoverable(
Status: database.DeliveryStatusPending, Status: database.DeliveryStatusPending,
Event: event, Event: event,
Target: database.Target{ Target: database.Target{
Name: "unwritable", Name: "unwritable",
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Config: iHTTPConfig(ts.URL), URL: ts.URL,
MaxRetries: 3, MaxAttempts: 3,
}, },
} }
full.ID = d.ID full.ID = d.ID
@@ -457,9 +459,8 @@ func TestFailedResultWriteWithRetriesLeavesDeliveryRecoverable(
s.Engine.ExportDeliverHTTPWithScheduler( s.Engine.ExportDeliverHTTPWithScheduler(
context.Background(), s.WebhookDB, full, context.Background(), s.WebhookDB, full,
&delivery.Task{ &delivery.Task{
DeliveryID: d.ID, DeliveryID: d.ID, TargetID: targetID,
TargetID: targetID, MaxAttempts: 3, AttemptNum: 1,
AttemptNum: 1,
}, },
sched, sched,
) )
+389
View File
@@ -0,0 +1,389 @@
package delivery_test
import (
"context"
"log/slog"
"net/http"
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/database/databasetest"
"sneak.berlin/go/webhooker/internal/delivery"
)
// These tests cover the recovery routine, which the engine runs once
// when it starts and then on every tick of the sweep
// (https://git.eeqj.de/sneak/webhooker/issues/542). Every case runs it
// both ways, each on a webhook of its own.
// atStart and onTick are the two values of the recovery routine's
// atStart parameter: its run when the engine starts, and a tick of the
// sweep.
const (
atStart = true
onTick = false
)
// recoveryRun names one of the two runs of the recovery routine.
type recoveryRun struct {
name string
atStart bool
}
// recoveryRuns returns the routine's run at start and a tick.
func recoveryRuns() []recoveryRun {
return []recoveryRun{
{"at start", atStart},
{"on a tick", onTick},
}
}
// recoverySetup gives a fresh webhook its main-database row and an http
// target with retries, and returns the setup and the target's ID. No
// worker runs, so whatever the routine queues stays in the channels for
// the test to read, and nothing is sent to the target.
func recoverySetup(t *testing.T) (iSetup, string) {
t.Helper()
s := newISetup(t)
targetID := uuid.New().String()
iCreateWebhook(t, s.MainDB, s.WebhookID, "recovery")
iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "recovery",
database.TargetTypeHTTP,
"http://example.com/hook", 20,
)
return s, targetID
}
// seedRecoveryDelivery seeds a delivery at status with the given
// recorded attempts, failed but for the last when lastSucceeded is set,
// all recorded now.
func seedRecoveryDelivery(
t *testing.T,
s iSetup,
targetID string,
status database.DeliveryStatus,
attempts int,
lastSucceeded bool,
) string {
t.Helper()
event := iSeedEvent(t, s.WebhookDB, s.WebhookID, `{"recovery":true}`)
d := iSeedDelivery(t, s.WebhookDB, event.ID, targetID, status)
for n := 1; n <= attempts; n++ {
rSeedResult(
t, s.WebhookDB, d.ID, n, lastSucceeded && n == attempts,
)
}
return d.ID
}
// backdateAttempts moves every recorded attempt of a delivery an hour
// into the past, long past the backoff after its first attempt.
func backdateAttempts(t *testing.T, s iSetup, deliveryID string) {
t.Helper()
require.NoError(t, s.WebhookDB.Model(&database.DeliveryResult{}).
Where("delivery_id = ?", deliveryID).
UpdateColumn("created_at", s.WebhookDB.NowFunc().Add(-time.Hour)).
Error)
}
// requireQueued waits for a task on ch. A retrying delivery reaches the
// retry channel through a timer, even one due now, so it is waited for.
func requireQueued(
t *testing.T, ch <-chan delivery.Task,
) delivery.Task {
t.Helper()
select {
case task := <-ch:
return task
case <-time.After(5 * time.Second):
t.Fatal("the recovery routine queued nothing")
return delivery.Task{}
}
}
// assertNothingQueued asserts the engine queues nothing, giving a retry
// timer that is due now the moment it needs to fire.
func assertNothingQueued(t *testing.T, e *delivery.Engine) {
t.Helper()
select {
case task := <-e.ExportDeliveryCh():
t.Errorf("queued pending delivery %s", task.DeliveryID)
case task := <-e.ExportRetryCh():
t.Errorf("queued retrying delivery %s", task.DeliveryID)
case <-time.After(200 * time.Millisecond):
}
}
// TestRecovery_RetryingBackoffPassed: a retrying delivery whose backoff
// is over is queued for its next attempt at once.
func TestRecovery_RetryingBackoffPassed(t *testing.T) {
t.Parallel()
for _, run := range recoveryRuns() {
t.Run(run.name, func(t *testing.T) {
t.Parallel()
s, targetID := recoverySetup(t)
id := seedRecoveryDelivery(
t, s, targetID, database.DeliveryStatusRetrying, 1, false,
)
backdateAttempts(t, s, id)
s.Engine.ExportRecoverDeliveries(context.Background(), run.atStart)
task := requireQueued(t, s.Engine.ExportRetryCh())
assert.Equal(t, id, task.DeliveryID)
assert.Equal(t, 2, task.AttemptNum)
iAssertStatus(t, s.WebhookDB, id, database.DeliveryStatusRetrying)
})
}
}
// TestRecovery_RetryingBackoffNotPassed: a retrying delivery whose last
// attempt was just now waits out the rest of its backoff, ten attempts
// in, eight and a half minutes. It is not sent now, and the engine holds
// it, so a later tick leaves it to the timer.
func TestRecovery_RetryingBackoffNotPassed(t *testing.T) {
t.Parallel()
for _, run := range recoveryRuns() {
t.Run(run.name, func(t *testing.T) {
t.Parallel()
s, targetID := recoverySetup(t)
id := seedRecoveryDelivery(
t, s, targetID, database.DeliveryStatusRetrying, 10, false,
)
s.Engine.ExportRecoverDeliveries(context.Background(), run.atStart)
assertNothingQueued(t, s.Engine)
iAssertStatus(t, s.WebhookDB, id, database.DeliveryStatusRetrying)
assert.Equal(t, 1, s.Engine.ExportInflightHeld(),
"no retry timer holds the delivery",
)
})
}
}
// TestRecovery_PendingOlderThanTheAgeBound: a pending delivery untouched
// for longer than the sweep's age bound is sent by either run.
func TestRecovery_PendingOlderThanTheAgeBound(t *testing.T) {
t.Parallel()
for _, run := range recoveryRuns() {
t.Run(run.name, func(t *testing.T) {
t.Parallel()
s, targetID := recoverySetup(t)
id := seedRecoveryDelivery(
t, s, targetID, database.DeliveryStatusPending, 0, false,
)
rAgePending(t, s.WebhookDB, id)
s.Engine.ExportRecoverDeliveries(context.Background(), run.atStart)
task := requireQueued(t, s.Engine.ExportDeliveryCh())
assert.Equal(t, id, task.DeliveryID)
assert.Equal(t, 1, task.AttemptNum)
})
}
}
// TestRecovery_TaskCarriesEveryTargetSetting: the task the recovery
// routine queues carries every one of the target's settings, all six
// set here.
// They are copied into the task one field at a time, and a field left
// out is sent empty without any error.
func TestRecovery_TaskCarriesEveryTargetSetting(t *testing.T) {
t.Parallel()
for _, run := range recoveryRuns() {
t.Run(run.name, func(t *testing.T) {
t.Parallel()
s := newISetup(t)
iCreateWebhook(t, s.MainDB, s.WebhookID, "recovery")
target := database.Target{
WebhookID: s.WebhookID,
Name: "recovery",
Type: database.TargetTypeHTTP,
Active: true,
URL: "http://example.com/hook",
Headers: map[string]string{headerAuthorization: "Bearer t"},
TimeoutSeconds: 7,
ForwardQuery: true,
ArchiveExpiry: time.Hour,
ArchiveRotation: "daily",
MaxAttempts: 1,
}
require.NoError(t, s.MainDB.Create(&target).Error)
id := seedRecoveryDelivery(
t, s, target.ID, database.DeliveryStatusPending, 0, false,
)
rAgePending(t, s.WebhookDB, id)
s.Engine.ExportRecoverDeliveries(context.Background(), run.atStart)
task := requireQueued(t, s.Engine.ExportDeliveryCh())
assert.Equal(t, target.ID, task.TargetID)
assert.Equal(t, target.URL, task.TargetURL)
assert.Equal(t, target.Headers, task.TargetHeaders)
assert.Equal(t, target.TimeoutSeconds, task.TimeoutSeconds)
assert.Equal(t, target.ForwardQuery, task.ForwardQuery)
assert.Equal(t, target.ArchiveExpiry, task.ArchiveExpiry)
assert.Equal(t, target.ArchiveRotation, task.ArchiveRotation)
})
}
}
// TestRecovery_PendingYoungerThanTheAgeBound: a pending delivery touched
// a moment ago, as one a stop cut off is, is sent at start, which is how
// it reaches its target on the next start
// (https://git.eeqj.de/sneak/webhooker/issues/527). A tick leaves it.
func TestRecovery_PendingYoungerThanTheAgeBound(t *testing.T) {
t.Parallel()
for _, run := range recoveryRuns() {
t.Run(run.name, func(t *testing.T) {
t.Parallel()
s, targetID := recoverySetup(t)
id := seedRecoveryDelivery(
t, s, targetID, database.DeliveryStatusPending, 0, false,
)
s.Engine.ExportRecoverDeliveries(context.Background(), run.atStart)
if run.atStart {
task := requireQueued(t, s.Engine.ExportDeliveryCh())
assert.Equal(t, id, task.DeliveryID)
return
}
assertNothingQueued(t, s.Engine)
iAssertStatus(t, s.WebhookDB, id, database.DeliveryStatusPending)
assert.Zero(t, s.Engine.ExportInflightHeld())
})
}
}
// TestRecovery_AlreadySucceeded: a pending or retrying delivery whose
// success is recorded but whose status write was lost is marked
// delivered where it stands, and not sent again.
func TestRecovery_AlreadySucceeded(t *testing.T) {
t.Parallel()
for _, run := range recoveryRuns() {
t.Run(run.name, func(t *testing.T) {
t.Parallel()
s, targetID := recoverySetup(t)
pending := seedRecoveryDelivery(
t, s, targetID, database.DeliveryStatusPending, 1, true,
)
rAgePending(t, s.WebhookDB, pending)
retrying := seedRecoveryDelivery(
t, s, targetID, database.DeliveryStatusRetrying, 2, true,
)
backdateAttempts(t, s, retrying)
s.Engine.ExportRecoverDeliveries(context.Background(), run.atStart)
assertNothingQueued(t, s.Engine)
for id, attempts := range map[string]int{pending: 1, retrying: 2} {
iAssertStatus(
t, s.WebhookDB, id, database.DeliveryStatusDelivered,
)
assert.Len(t, iResults(t, s.WebhookDB, id), attempts,
"marking the delivery delivered recorded an attempt",
)
}
})
}
}
// TestRecovery_MissingDatabase: a tick leaves a webhook whose database
// file is missing alone, and the run at start opens it, which creates an
// empty one and reports the loss (see
// TestRecoverDeliveries_ReportsAMissingWebhookDatabase).
func TestRecovery_MissingDatabase(t *testing.T) {
t.Parallel()
mainDB := iMainDB(t)
webhookID := uuid.New().String()
iCreateWebhook(t, mainDB.DB(), webhookID, "missing-database")
dbMgr := databasetest.NewWebhookDBManager(t, t.TempDir())
t.Cleanup(func() { _ = dbMgr.CloseAll() })
engine := delivery.NewTestEngineWithDB(
mainDB, dbMgr,
slog.New(slog.DiscardHandler),
&http.Client{Timeout: 5 * time.Second}, 1,
)
engine.ExportRecoverDeliveries(context.Background(), onTick)
assert.False(t, dbMgr.DBExists(webhookID),
"a tick created the missing database",
)
engine.ExportRecoverDeliveries(context.Background(), atStart)
assert.True(t, dbMgr.DBExists(webhookID),
"the run at start did not open the missing database",
)
}
// TestRecovery_LeavesADeliveryTheEngineHolds: a delivery the engine is
// working on, has queued, or holds a retry timer for is left alone by
// either run, pending and retrying alike, however old.
func TestRecovery_LeavesADeliveryTheEngineHolds(t *testing.T) {
t.Parallel()
for _, run := range recoveryRuns() {
t.Run(run.name, func(t *testing.T) {
t.Parallel()
s, targetID := recoverySetup(t)
pending := seedRecoveryDelivery(
t, s, targetID, database.DeliveryStatusPending, 0, false,
)
rAgePending(t, s.WebhookDB, pending)
retrying := seedRecoveryDelivery(
t, s, targetID, database.DeliveryStatusRetrying, 1, false,
)
backdateAttempts(t, s, retrying)
require.True(t, s.Engine.ExportRetainDelivery(pending))
require.True(t, s.Engine.ExportRetainDelivery(retrying))
s.Engine.ExportRecoverDeliveries(context.Background(), run.atStart)
assertNothingQueued(t, s.Engine)
iAssertStatus(t, s.WebhookDB, pending, database.DeliveryStatusPending)
iAssertStatus(t, s.WebhookDB, retrying, database.DeliveryStatusRetrying)
assert.Equal(t, 2, s.Engine.ExportInflightHeld())
})
}
}
+9 -17
View File
@@ -85,19 +85,14 @@ func deliverWithProbeHeaders(
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
cfg, err := json.Marshal(delivery.HTTPTargetConfig{
URL: targetURL,
Headers: map[string]string{
probeHeaderName: probeHeaderValue,
},
})
require.NoError(t, err)
body := event.Body body := event.Body
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"redirect-target", string(cfg), 0, 1, &body, "redirect-target", targetURL, 1, 1, &body,
) )
task.TargetHeaders = map[string]string{
probeHeaderName: probeHeaderValue,
}
s.Engine.ExportProcessNewTask(context.TODO(), &task) s.Engine.ExportProcessNewTask(context.TODO(), &task)
@@ -309,8 +304,7 @@ func TestRedirectPolicy_StopsAtHopCap(t *testing.T) {
) )
client := engine.ExportClientForRequest( client := engine.ExportClientForRequest(
&delivery.HTTPTargetConfig{URL: srv.URL}, 0, []string{probeHeaderName},
[]string{probeHeaderName},
) )
require.NotNil(t, client.CheckRedirect) require.NotNil(t, client.CheckRedirect)
@@ -350,7 +344,7 @@ func TestApplyRequestHeaders_ReportsOriginScopedNames(t *testing.T) {
inbound, err := json.Marshal(map[string][]string{ inbound, err := json.Marshal(map[string][]string{
inboundHeaderName: {inboundHeaderValue}, inboundHeaderName: {inboundHeaderValue},
"Content-Type": {testContentType}, "Content-Type": {testContentType},
"User-Agent": {"curl/8.7.1"}, headerUserAgent: {"curl/8.7.1"},
"Host": {"inbound.example.com"}, "Host": {"inbound.example.com"},
}) })
require.NoError(t, err) require.NoError(t, err)
@@ -369,11 +363,9 @@ func TestApplyRequestHeaders_ReportsOriginScopedNames(t *testing.T) {
Headers: string(inbound), Headers: string(inbound),
ContentType: testContentType, ContentType: testContentType,
}, },
&delivery.HTTPTargetConfig{ map[string]string{
Headers: map[string]string{ probeHeaderName: probeHeaderValue,
probeHeaderName: probeHeaderValue, "Content-Type": testContentType,
"Content-Type": testContentType,
},
}, },
"webhooker/dev", "webhooker/dev",
) )
+12 -23
View File
@@ -13,8 +13,8 @@ import (
// The engine provides one to each target so a target can own // The engine provides one to each target so a target can own
// its retries durably: it records the attempt, marks the // its retries durably: it records the attempt, marks the
// delivery retrying, and asks the Scheduler to deliver the // delivery retrying, and asks the Scheduler to deliver the
// next attempt after delay — exactly what the engine does for // next attempt after delay — exactly what the engine's
// its own restart recovery. // recovery routine does for a retrying delivery.
type Scheduler interface { type Scheduler interface {
ScheduleRetry(task Task, delay time.Duration) ScheduleRetry(task Task, delay time.Duration)
} }
@@ -32,7 +32,8 @@ type Scheduler interface {
// Engine.observeAttempt, alongside the DeliveryResult it records for // Engine.observeAttempt, alongside the DeliveryResult it records for
// it. Deliver is also entered for attempts that never happen — an // it. Deliver is also entered for attempts that never happen — an
// open circuit breaker refuses one — so the count cannot be taken // open circuit breaker refuses one — so the count cannot be taken
// from around this call. // from around this call. An attempt cut off by a stop is neither
// counted nor recorded.
type Target interface { type Target interface {
Deliver( Deliver(
ctx context.Context, ctx context.Context,
@@ -44,29 +45,20 @@ type Target interface {
} }
// rescheduler is implemented by targets that own durable // rescheduler is implemented by targets that own durable
// retries. The engine's restart recovery and periodic sweep // retries. The engine's recovery routine uses it to let the
// use it to let the target recompute the schedule for an // target recompute the schedule for an orphaned retrying
// orphaned retrying delivery, keeping the retry schedule // delivery, keeping the retry schedule target-owned.
// target-owned. Fire-and-forget targets do not implement it // Fire-and-forget targets do not implement it, and a retrying
// and their (never-occurring) retrying deliveries are // delivery found for one is terminally failed.
// skipped.
type rescheduler interface { type rescheduler interface {
// remainingBackoff returns how long to wait before the // remainingBackoff returns how long to wait before the
// next attempt of a recovered retrying delivery. // next attempt of a recovered retrying delivery: zero once
// its backoff has passed.
remainingBackoff( remainingBackoff(
webhookDB *gorm.DB, webhookDB *gorm.DB,
deliveryID string, deliveryID string,
attemptNum int, attemptNum int,
) time.Duration ) time.Duration
// backoffElapsed reports whether the backoff window for
// the last attempt has already passed, so the periodic
// sweep can re-enqueue the delivery now.
backoffElapsed(
webhookDB *gorm.DB,
deliveryID string,
attemptNum int,
) bool
} }
// attemptResult is the outcome of a single delivery attempt, // attemptResult is the outcome of a single delivery attempt,
@@ -102,16 +94,13 @@ func (e *Engine) initTargets(client *http.Client) {
client: client, client: client,
} }
dbT := &databaseTarget{eng: e}
e.httpTarget = httpT e.httpTarget = httpT
e.slackTarget = slackT e.slackTarget = slackT
e.dbTarget = dbT
e.targets = map[database.TargetType]Target{ e.targets = map[database.TargetType]Target{
database.TargetTypeHTTP: httpT, database.TargetTypeHTTP: httpT,
database.TargetTypeSlack: slackT, database.TargetTypeSlack: slackT,
database.TargetTypeDatabase: dbT, database.TargetTypeDatabase: &databaseTarget{eng: e},
database.TargetTypeLog: &logTarget{eng: e}, database.TargetTypeLog: &logTarget{eng: e},
} }
} }
-136
View File
@@ -1,136 +0,0 @@
package delivery
import (
"encoding/json"
"errors"
"fmt"
"sneak.berlin/go/webhooker/internal/database"
)
// errUnknownTargetTypeForEdit is returned when a stored target has a
// type the edit form has no field set for.
var errUnknownTargetTypeForEdit = errors.New(
"unknown target type",
)
// TargetConfigForm is the UNMASKED projection of a target's stored
// configuration, for pre-filling the target edit form.
//
// It is the deliberate exception to the rule TargetView enforces
// everywhere else: TargetView exists so that no template can render
// a target's stored blob, because a destination URL's path segments
// and a header value are both routinely the credential. An operator
// cannot correct a value they cannot see, so the edit form — and
// only the edit form — is shown the full value.
//
// Everything that keeps that exception narrow lives at the call
// site: the route is behind RequireAuth and the webhook's ownership
// check, and its group sets NoCache so the rendered secret is not
// written to a shared cache. Do not reach for this type from any
// other page.
type TargetConfigForm struct {
// URL is the destination for an HTTP target and the webhook
// URL for a Slack target.
URL string
// Headers is the HTTP target's configured headers in the
// textarea representation, one "Name: value" per line.
Headers string
// Timeout is the HTTP target's per-request timeout in seconds,
// empty when unset.
Timeout string
// ForwardQuery is the HTTP target's setting that passes each
// event's query string on to it.
ForwardQuery bool
// Expiry is the database (archive) target's row expiry.
Expiry string
// Rotation is the database (archive) target's rotation.
Rotation string
}
// NewTargetConfigForm parses a target's stored configuration into
// the edit form's fields.
//
// A configuration that does not parse is an error rather than a
// zero-valued form that silently looks like a target with no
// settings. The caller shows the operator that the stored value
// could not be read, so that saving the form is understood as
// replacing it rather than preserving it.
func NewTargetConfigForm(
t *database.Target,
) (TargetConfigForm, error) {
switch t.Type {
case database.TargetTypeHTTP:
cfg, err := parseHTTPConfig(t.Config)
if err != nil {
return TargetConfigForm{}, err
}
return TargetConfigForm{
URL: cfg.URL,
Headers: FormatTargetHeaders(cfg.Headers),
Timeout: FormatTargetTimeout(cfg.Timeout),
ForwardQuery: cfg.ForwardQuery,
}, nil
case database.TargetTypeSlack:
cfg, err := parseSlackConfig(t.Config)
if err != nil {
return TargetConfigForm{}, err
}
return TargetConfigForm{URL: cfg.WebhookURL}, nil
case database.TargetTypeDatabase:
return databaseConfigForm(t.Config)
case database.TargetTypeLog:
// The log target takes no configuration.
return TargetConfigForm{}, nil
default:
return TargetConfigForm{}, fmt.Errorf(
"%w: %q", errUnknownTargetTypeForEdit, t.Type,
)
}
}
// databaseConfigForm parses an archive target's optional expiry and
// rotation. An absent, empty or never expiry yields an empty expiry,
// on which the edit form starts at never; saving it unchanged stores
// never, which means the same as an empty expiry. An absent rotation
// is empty too, and the form starts at none. An expiry that is set
// but not a valid duration, or a rotation that is not one of the
// four, is an error, not a blank field.
func databaseConfigForm(
configJSON string,
) (TargetConfigForm, error) {
if configJSON == "" {
return TargetConfigForm{}, nil
}
var cfg databaseTargetConfig
err := json.Unmarshal([]byte(configJSON), &cfg)
if err != nil {
return TargetConfigForm{}, fmt.Errorf(
"parsing config JSON: %w", err,
)
}
err = ValidateArchiveRotation(cfg.Rotation)
if err != nil {
return TargetConfigForm{}, err
}
form := TargetConfigForm{Rotation: cfg.Rotation}
if cfg.Expiry == "" || cfg.Expiry == archiveExpiryNever {
return form, nil
}
err = ValidateArchiveExpiry(cfg.Expiry)
if err != nil {
return TargetConfigForm{}, err
}
form.Expiry = cfg.Expiry
return form, nil
}
+42 -87
View File
@@ -5,19 +5,21 @@ import (
"strconv" "strconv"
"time" "time"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
) )
// configUnavailable is what a target's configuration renders // configUnavailable is what a target's settings render as when
// as when it is absent, of an unknown type, or does not // the target is of an unknown type, and what its URL renders as
// parse. The stored blob is never shown as a fallback: it can // when it does not parse into a scheme and host. The stored URL is
// hold a credential (a Slack incoming webhook URL is a bearer // never shown as a fallback: it can be a credential (a Slack
// token) and a UI that prints it leaks that credential into // incoming webhook URL is a bearer token) and a UI that prints it
// browser history, screenshots and screen shares. // leaks that credential into browser history, screenshots and
// screen shares.
const configUnavailable = "(unavailable)" const configUnavailable = "(unavailable)"
// ConfigField is one labelled, display-safe value derived // ConfigField is one labelled, display-safe value derived
// from a target's stored configuration. // from a target's settings.
type ConfigField struct { type ConfigField struct {
Label string Label string
Value string Value string
@@ -31,9 +33,9 @@ type ConfigField struct {
const deletedNameSuffix = " (deleted)" const deletedNameSuffix = " (deleted)"
// TargetView is the display-safe projection of a target for // TargetView is the display-safe projection of a target for
// the UI. It deliberately has no raw configuration field, so // the UI. It deliberately has no URL or header field, so no
// no template — present or future — can render the stored // template — present or future — can render the stored
// blob. // credential.
type TargetView struct { type TargetView struct {
ID string ID string
Name string Name string
@@ -60,8 +62,7 @@ func (v TargetView) DisplayName() string {
} }
// NewTargetViews projects targets for rendering, replacing // NewTargetViews projects targets for rendering, replacing
// each stored configuration blob with named, display-safe // each target's settings with named, display-safe fields.
// fields.
// //
// A soft-deleted row projects exactly as a live one does, minus // A soft-deleted row projects exactly as a live one does, minus
// the deleted marker on its name: masking is a property of the // the deleted marker on its name: masking is a property of the
@@ -90,8 +91,8 @@ func NewTargetViews(
} }
// targetConfigFields returns the display-safe fields for a // targetConfigFields returns the display-safe fields for a
// target's configuration. Anything it cannot parse becomes // target's settings. A target of an unknown type gets the
// the neutral placeholder. // neutral placeholder.
func targetConfigFields( func targetConfigFields(
t *database.Target, t *database.Target,
) []ConfigField { ) []ConfigField {
@@ -101,37 +102,26 @@ func targetConfigFields(
case database.TargetTypeHTTP: case database.TargetTypeHTTP:
return httpConfigFields(t) return httpConfigFields(t)
case database.TargetTypeDatabase: case database.TargetTypeDatabase:
return databaseConfigFields(t.Config) return databaseConfigFields(t)
case database.TargetTypeLog: case database.TargetTypeLog:
// The log target takes no configuration. // The log target takes no settings.
return nil return nil
default: default:
return unavailableConfigFields() return []ConfigField{{
Label: "Configuration",
Value: configUnavailable,
}}
} }
} }
// unavailableConfigFields is the neutral placeholder shown
// for a configuration that could not be presented.
func unavailableConfigFields() []ConfigField {
return []ConfigField{{
Label: "Configuration",
Value: configUnavailable,
}}
}
// slackConfigFields describes a Slack target: its masked // slackConfigFields describes a Slack target: its masked
// webhook URL and its retry count. Only the masked URL is // webhook URL and its number of delivery attempts. Only the
// shown; the full URL is the credential. // masked URL is shown; the full URL is the credential.
func slackConfigFields(t *database.Target) []ConfigField { func slackConfigFields(t *database.Target) []ConfigField {
cfg, err := parseSlackConfig(t.Config)
if err != nil {
return unavailableConfigFields()
}
return []ConfigField{{ return []ConfigField{{
Label: "Webhook URL", Label: "Webhook URL",
Value: cfg.MaskedWebhookURL(), Value: MaskURL(t.URL),
}, maxRetriesField(t)} }, maxAttemptsField(t)}
} }
// httpConfigFields describes an HTTP target: its destination // httpConfigFields describes an HTTP target: its destination
@@ -145,79 +135,55 @@ func slackConfigFields(t *database.Target) []ConfigField {
// whose path segments are the credential, and the field takes // whose path segments are the credential, and the field takes
// an arbitrary URL, so no segment can be assumed non-secret. // an arbitrary URL, so no segment can be assumed non-secret.
func httpConfigFields(t *database.Target) []ConfigField { func httpConfigFields(t *database.Target) []ConfigField {
cfg, err := parseHTTPConfig(t.Config)
if err != nil {
return unavailableConfigFields()
}
fields := []ConfigField{{ fields := []ConfigField{{
Label: "Destination URL", Label: "Destination URL",
Value: MaskURL(cfg.URL), Value: MaskURL(t.URL),
}} }}
if cfg.Timeout > 0 { if t.TimeoutSeconds > 0 {
fields = append(fields, ConfigField{ fields = append(fields, ConfigField{
Label: "Timeout", Label: "Timeout",
Value: strconv.Itoa(cfg.Timeout) + "s", Value: strconv.Itoa(t.TimeoutSeconds) + "s",
}) })
} }
if len(cfg.Headers) > 0 { if len(t.Headers) > 0 {
fields = append(fields, ConfigField{ fields = append(fields, ConfigField{
Label: "Headers", Label: "Headers",
Value: fmt.Sprintf( Value: fmt.Sprintf(
"%d configured", len(cfg.Headers), "%d configured", len(t.Headers),
), ),
}) })
} }
if cfg.ForwardQuery { if t.ForwardQuery {
fields = append(fields, ConfigField{ fields = append(fields, ConfigField{
Label: "Query string", Label: "Query string",
Value: "passed on to this target", Value: "passed on to this target",
}) })
} }
fields = append(fields, maxRetriesField(t)) fields = append(fields, maxAttemptsField(t))
return fields return fields
} }
// maxRetriesField describes a target's retry count, which lives // maxAttemptsField describes a target's number of delivery
// on the target row rather than in its configuration blob. A // attempts.
// stored 0 makes a single attempt, so it is shown as 1. func maxAttemptsField(t *database.Target) ConfigField {
func maxRetriesField(t *database.Target) ConfigField {
attempts := strconv.Itoa(t.MaxRetries)
if t.MaxRetries == 0 {
attempts = "1 (fire-and-forget: no retries, no circuit breaker)"
}
return ConfigField{ return ConfigField{
Label: "Delivery attempts", Label: "Delivery attempts",
Value: attempts, Value: strconv.Itoa(t.MaxAttempts),
} }
} }
// databaseConfigFields describes an archive target by its // databaseConfigFields describes an archive target by its
// expiry in plain units, such as "30 days", or "never" when // expiry in plain units, such as "30 days", or "never" when
// the archive is kept forever, and by its rotation. An expiry // the archive is kept forever, and by its rotation.
// that is set but not a valid duration, or a rotation that is func databaseConfigFields(t *database.Target) []ConfigField {
// not one of the four, is reported as unavailable rather than value := archive.ExpiryNever
// echoed back. if t.ArchiveExpiry > 0 {
func databaseConfigFields(configJSON string) []ConfigField { value = plainDuration(t.ArchiveExpiry)
expiry, err := parseArchiveExpiry(configJSON)
if err != nil {
return unavailableConfigFields()
}
rotation, err := parseArchiveRotation(configJSON)
if err != nil {
return unavailableConfigFields()
}
value := archiveExpiryNever
if expiry > 0 {
value = plainDuration(expiry)
} }
return []ConfigField{{ return []ConfigField{{
@@ -225,7 +191,7 @@ func databaseConfigFields(configJSON string) []ConfigField {
Value: value, Value: value,
}, { }, {
Label: "Archive rotation", Label: "Archive rotation",
Value: rotation, Value: t.ArchiveRotation,
}} }}
} }
@@ -261,14 +227,3 @@ func plainDuration(d time.Duration) string {
return d.String() return d.String()
} }
// MaskedWebhookURL returns the Slack webhook URL reduced to
// its scheme and host, with the path, query and any userinfo
// elided. The path segments are the credential, so none of
// them is shown: the field accepts an arbitrary URL, so no
// segment can be assumed non-secret. A URL that does not
// parse into a scheme and host yields the neutral
// placeholder, never the raw string.
func (c *SlackTargetConfig) MaskedWebhookURL() string {
return MaskURL(c.WebhookURL)
}
+85 -128
View File
@@ -32,10 +32,20 @@ const (
viewMaskedOrigin = viewExampleOrigin + "/..." viewMaskedOrigin = viewExampleOrigin + "/..."
viewUnavailable = "(unavailable)" viewUnavailable = "(unavailable)"
viewExpiryNever = "never" viewExpiryNever = "never"
viewMaxRetries = "Delivery attempts" viewMaxAttempts = "Delivery attempts"
) )
func TestMaskedWebhookURL(t *testing.T) { // The archive rotations a database target's settings can name.
const (
rotationNone = "none"
rotationMonthly = "monthly"
rotationDaily = "daily"
rotationHourly = "hourly"
)
// TestNewTargetViews_SlackMasksWebhookURL shows what a Slack target's
// URL renders as on the target list.
func TestNewTargetViews_SlackMasksWebhookURL(t *testing.T) {
t.Parallel() t.Parallel()
tests := map[string]struct { tests := map[string]struct {
@@ -79,28 +89,24 @@ func TestMaskedWebhookURL(t *testing.T) {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
t.Parallel() t.Parallel()
cfg := &delivery.SlackTargetConfig{ target := slackTarget()
WebhookURL: tc.url, target.URL = tc.url
}
assert.Equal( assert.Equal(
t, tc.want, cfg.MaskedWebhookURL(), t, tc.want,
fieldMap(viewFor(t, target).Config)["Webhook URL"],
) )
}) })
} }
} }
// TestMaskedWebhookURL_NeverLeaksPath is the direct // TestNewTargetViews_SlackNeverLeaksPath is the direct
// expression of the rule: whatever the input, the masked // expression of the rule: whatever the input, the masked
// value never contains a path segment of it. // value never contains a path segment of it.
func TestMaskedWebhookURL_NeverLeaksPath(t *testing.T) { func TestNewTargetViews_SlackNeverLeaksPath(t *testing.T) {
t.Parallel() t.Parallel()
cfg := &delivery.SlackTargetConfig{ masked := fieldMap(viewFor(t, slackTarget()).Config)["Webhook URL"]
WebhookURL: slackWebhookURL,
}
masked := cfg.MaskedWebhookURL()
assert.NotContains(t, masked, "T00000000") assert.NotContains(t, masked, "T00000000")
assert.NotContains(t, masked, "B00000000") assert.NotContains(t, masked, "B00000000")
@@ -166,11 +172,11 @@ func TestNewTargetViews_DeletedTarget(t *testing.T) {
// share. // share.
func slackTarget() database.Target { func slackTarget() database.Target {
return database.Target{ return database.Target{
Name: slackTargetName, Name: slackTargetName,
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
Active: true, Active: true,
Config: `{"webhookUrl":"` + URL: slackWebhookURL,
slackWebhookURL + `"}`, MaxAttempts: 1,
} }
} }
@@ -189,28 +195,29 @@ func TestNewTargetViews_Slack(t *testing.T) {
assert.Equal( assert.Equal(
t, t,
map[string]string{ map[string]string{
"Webhook URL": slackMaskedURL, "Webhook URL": slackMaskedURL,
viewMaxRetries: "1 (fire-and-forget: no retries, no circuit breaker)", viewMaxAttempts: "1",
}, },
fieldMap(view.Config), fieldMap(view.Config),
) )
} }
// TestNewTargetViews_SlackRetries proves a Slack target shows // TestNewTargetViews_SlackAttempts proves a Slack target shows
// its retry count the same way an HTTP target does. // its number of delivery attempts the same way an HTTP target
func TestNewTargetViews_SlackRetries(t *testing.T) { // does.
func TestNewTargetViews_SlackAttempts(t *testing.T) {
t.Parallel() t.Parallel()
target := slackTarget() target := slackTarget()
target.MaxRetries = 2 target.MaxAttempts = 2
view := viewFor(t, target) view := viewFor(t, target)
assert.Equal( assert.Equal(
t, t,
map[string]string{ map[string]string{
"Webhook URL": slackMaskedURL, "Webhook URL": slackMaskedURL,
viewMaxRetries: "2", viewMaxAttempts: "2",
}, },
fieldMap(view.Config), fieldMap(view.Config),
) )
@@ -220,12 +227,12 @@ func TestNewTargetViews_HTTP(t *testing.T) {
t.Parallel() t.Parallel()
view := viewFor(t, database.Target{ view := viewFor(t, database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Config: `{"url":"` + viewExampleHook + `",` + URL: viewExampleHook,
`"timeout":30,` + TimeoutSeconds: 30,
`"headers":{"Authorization":"Bearer sekrit"},` + Headers: map[string]string{headerAuthorization: "Bearer sekrit"},
`"forwardQuery":true}`, ForwardQuery: true,
MaxRetries: 5, MaxAttempts: 5,
}) })
fields := fieldMap(view.Config) fields := fieldMap(view.Config)
@@ -237,7 +244,7 @@ func TestNewTargetViews_HTTP(t *testing.T) {
"Timeout": "30s", "Timeout": "30s",
"Headers": "1 configured", "Headers": "1 configured",
"Query string": "passed on to this target", "Query string": "passed on to this target",
viewMaxRetries: "5", viewMaxAttempts: "5",
}, },
fields, fields,
) )
@@ -248,19 +255,22 @@ func TestNewTargetViews_HTTP(t *testing.T) {
} }
} }
func TestNewTargetViews_HTTPFireAndForget(t *testing.T) { // TestNewTargetViews_HTTPSingleAttempt proves a single attempt is
// shown as the plain number, like any other.
func TestNewTargetViews_HTTPSingleAttempt(t *testing.T) {
t.Parallel() t.Parallel()
view := viewFor(t, database.Target{ view := viewFor(t, database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Config: `{"url":"` + viewExampleHook + `"}`, URL: viewExampleHook,
MaxAttempts: 1,
}) })
assert.Equal( assert.Equal(
t, t,
map[string]string{ map[string]string{
"Destination URL": viewMaskedOrigin, "Destination URL": viewMaskedOrigin,
viewMaxRetries: "1 (fire-and-forget: no retries, no circuit breaker)", viewMaxAttempts: "1",
}, },
fieldMap(view.Config), fieldMap(view.Config),
) )
@@ -274,8 +284,9 @@ func TestNewTargetViews_HTTPMasksDestinationURL(t *testing.T) {
t.Parallel() t.Parallel()
view := viewFor(t, database.Target{ view := viewFor(t, database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Config: `{"url":"` + slackWebhookURL + `"}`, URL: slackWebhookURL,
MaxAttempts: 1,
}) })
fields := fieldMap(view.Config) fields := fieldMap(view.Config)
@@ -298,25 +309,20 @@ func TestNewTargetViews_Database(t *testing.T) {
t.Parallel() t.Parallel()
tests := map[string]struct { tests := map[string]struct {
config string expiry time.Duration
want string want string
}{ }{
"empty config": {config: "", want: viewExpiryNever}, "never": {expiry: 0, want: viewExpiryNever},
"empty expiry": {config: `{}`, want: viewExpiryNever}, "1h": {expiry: time.Hour, want: "1 hour"},
"never literal": { "12h": {expiry: 12 * time.Hour, want: "12 hours"},
config: `{"expiry":"` + viewExpiryNever + `"}`, "24h": {expiry: 24 * time.Hour, want: "1 day"},
want: viewExpiryNever, "720h": {expiry: 720 * time.Hour, want: "30 days"},
}, "2160h": {expiry: 2160 * time.Hour, want: "90 days"},
"1h": {config: `{"expiry":"1h"}`, want: "1 hour"}, "8760h": {expiry: 8760 * time.Hour, want: "365 days"},
"12h": {config: `{"expiry":"12h"}`, want: "12 hours"}, "36h": {expiry: 36 * time.Hour, want: "36 hours"},
"24h": {config: `{"expiry":"24h"}`, want: "1 day"}, "1h30m": {expiry: 90 * time.Minute, want: "90 minutes"},
"720h": {config: `{"expiry":"720h"}`, want: "30 days"}, "45s": {expiry: 45 * time.Second, want: "45 seconds"},
"2160h": {config: `{"expiry":"2160h"}`, want: "90 days"}, "1.5s": {expiry: 1500 * time.Millisecond, want: "1.5s"},
"8760h": {config: `{"expiry":"8760h"}`, want: "365 days"},
"36h": {config: `{"expiry":"36h"}`, want: "36 hours"},
"1h30m": {config: `{"expiry":"1h30m"}`, want: "90 minutes"},
"45s": {config: `{"expiry":"45s"}`, want: "45 seconds"},
"1.5s": {config: `{"expiry":"1.5s"}`, want: "1.5s"},
} }
for name, tc := range tests { for name, tc := range tests {
@@ -324,8 +330,9 @@ func TestNewTargetViews_Database(t *testing.T) {
t.Parallel() t.Parallel()
view := viewFor(t, database.Target{ view := viewFor(t, database.Target{
Type: database.TargetTypeDatabase, Type: database.TargetTypeDatabase,
Config: tc.config, ArchiveExpiry: tc.expiry,
ArchiveRotation: rotationNone,
}) })
assert.Equal( assert.Equal(
@@ -341,33 +348,23 @@ func TestNewTargetViews_Database(t *testing.T) {
} }
// TestNewTargetViews_DatabaseRotation proves the target list shows a // TestNewTargetViews_DatabaseRotation proves the target list shows a
// database target's rotation, none when it has none stored. // database target's rotation.
func TestNewTargetViews_DatabaseRotation(t *testing.T) { func TestNewTargetViews_DatabaseRotation(t *testing.T) {
t.Parallel() t.Parallel()
// Each stored config, and the rotation the list shows for it.
tests := map[string]string{
"": rotationNone,
`{"rotation":""}`: rotationNone,
}
for _, rotation := range []string{ for _, rotation := range []string{
rotationNone, rotationMonthly, rotationDaily, rotationHourly, rotationNone, rotationMonthly, rotationDaily, rotationHourly,
} { } {
tests[`{"rotation":"`+rotation+`"}`] = rotation t.Run(rotation, func(t *testing.T) {
}
for config, want := range tests {
t.Run(config, func(t *testing.T) {
t.Parallel() t.Parallel()
view := viewFor(t, database.Target{ view := viewFor(t, database.Target{
Type: database.TargetTypeDatabase, Type: database.TargetTypeDatabase,
Config: config, ArchiveRotation: rotation,
}) })
assert.Equal( assert.Equal(
t, want, fieldMap(view.Config)["Archive rotation"], t, rotation, fieldMap(view.Config)["Archive rotation"],
) )
}) })
} }
@@ -377,68 +374,28 @@ func TestNewTargetViews_Log(t *testing.T) {
t.Parallel() t.Parallel()
view := viewFor(t, database.Target{ view := viewFor(t, database.Target{
Type: database.TargetTypeLog, Type: database.TargetTypeLog,
Config: "",
}) })
assert.Empty(t, view.Config) assert.Empty(t, view.Config)
} }
// TestNewTargetViews_Unpresentable proves that no config the // TestNewTargetViews_UnknownType proves that a target of a type the
// view cannot present falls back to the stored blob. // view does not know shows the neutral placeholder, and never its
func TestNewTargetViews_Unpresentable(t *testing.T) { // URL.
func TestNewTargetViews_UnknownType(t *testing.T) {
t.Parallel() t.Parallel()
const blob = `{"webhookUrl":"https://hooks.slack.com` + view := viewFor(t, database.Target{
slackSecretPath + `"` Type: database.TargetType("carrier-pigeon"),
URL: slackWebhookURL,
})
tests := map[string]database.Target{ assert.Equal(
"unknown target type": { t,
Type: database.TargetType("carrier-pigeon"), map[string]string{
Config: blob, "Configuration": viewUnavailable,
}, },
"unparseable json": { fieldMap(view.Config),
Type: database.TargetTypeSlack, )
Config: blob,
},
"empty slack config": {
Type: database.TargetTypeSlack,
},
"slack config without url": {
Type: database.TargetTypeSlack,
Config: `{}`,
},
"unparseable http json": {
Type: database.TargetTypeHTTP,
Config: `{"url":`,
},
"unparseable archive json": {
Type: database.TargetTypeDatabase,
Config: `{"expiry":`,
},
"invalid archive expiry": {
Type: database.TargetTypeDatabase,
Config: `{"expiry":"a fortnight"}`,
},
"invalid archive rotation": {
Type: database.TargetTypeDatabase,
Config: weeklyConfig,
},
}
for name, target := range tests {
t.Run(name, func(t *testing.T) {
t.Parallel()
view := viewFor(t, target)
assert.Equal(
t,
map[string]string{
"Configuration": viewUnavailable,
},
fieldMap(view.Config),
)
})
}
} }
+4 -369
View File
@@ -2,83 +2,23 @@ package delivery
import ( import (
"context" "context"
"fmt"
"strings"
"sync"
"time" "time"
"gorm.io/gorm" "gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
) )
// archiveNameMaxLen is how many characters of a webhook or target
// name an archive file name keeps.
const archiveNameMaxLen = 40
// databaseTarget is a no-retry target that archives the full // databaseTarget is a no-retry target that archives the full
// inbound event into the target's own archive SQLite file, separate // inbound event into the target's own archive SQLite file, separate
// from the per-webhook event database. The event is already // from the per-webhook event database. The event is already
// persisted in the per-webhook event DB by the time delivery runs; // persisted in the per-webhook event DB by the time delivery runs;
// the database target additionally writes a durable long-term copy // the database target additionally writes a durable long-term copy
// into the file ArchiveFileName names, with a period added when the // through the engine's archive.Writers, and then records a single
// target rotates (see archivePeriodPath), and then records a single
// attempt whose outcome reflects whether the archive write // attempt whose outcome reflects whether the archive write
// succeeded. See archiveWriter for the close/reopen, auto-recreate, // succeeded. Package archive holds everything about the files
// and expiry semantics. // themselves.
type databaseTarget struct { type databaseTarget struct {
eng *Engine eng *Engine
// writers holds one archive writer per database target, keyed
// by target ID.
mu sync.Mutex
writers map[string]*archiveWriter
}
// ArchiveFileName returns the file name of a database target's
// archive: archive-WEBHOOKNAME-TARGETNAME-TARGETID.db, with both
// names passed through archiveNamePart. The target ID keeps the
// name unique when two targets' names come out the same.
func ArchiveFileName(webhookName, targetName, targetID string) string {
return "archive-" + archiveNamePart(webhookName) + "-" +
archiveNamePart(targetName) + "-" + targetID + ".db"
}
// archiveNamePart makes a webhook or target name safe to put in a
// file name. It is lowercased; ASCII letters and digits are kept,
// every other run of characters becomes a single "-", and no "-" is
// left at either end. It is cut to archiveNameMaxLen characters, and
// a name with nothing left is "unnamed".
func archiveNamePart(name string) string {
var b strings.Builder
dash := false
for _, r := range strings.ToLower(name) {
if (r < 'a' || r > 'z') && (r < '0' || r > '9') {
dash = b.Len() > 0
continue
}
if dash {
b.WriteByte('-')
dash = false
}
b.WriteRune(r)
}
part := b.String()
if len(part) > archiveNameMaxLen {
part = strings.TrimRight(part[:archiveNameMaxLen], "-")
}
if part == "" {
return "unnamed"
}
return part
} }
// Deliver implements Target. It archives the event, then // Deliver implements Target. It archives the event, then
@@ -98,7 +38,7 @@ func (t *databaseTarget) Deliver(
) { ) {
start := time.Now() start := time.Now()
err := t.archive(d) err := t.eng.archives.Write(d)
elapsed := time.Since(start) elapsed := time.Since(start)
@@ -145,308 +85,3 @@ func (t *databaseTarget) Deliver(
database.DeliveryStatusDelivered, database.DeliveryStatusDelivered,
) )
} }
// archive writes the full event as a row into the target's
// archive database, honouring the optional per-target expiry and
// rotation parsed from the target config JSON. With rotation, the
// event goes to the file for the period of its receive time.
func (t *databaseTarget) archive(d *database.Delivery) error {
webhookID := d.Event.WebhookID
if webhookID == "" {
return errArchiveMissingWebhookID
}
expiry, err := parseArchiveExpiry(d.Target.Config)
if err != nil {
return err
}
rotation, err := parseArchiveRotation(d.Target.Config)
if err != nil {
return err
}
// An event whose stored row was gone before its delivery ran has
// no receive time (see Engine.hydrateEvent), and goes to the
// file for now.
receivedAt := d.Event.CreatedAt
if receivedAt.IsZero() {
receivedAt = time.Now()
}
w, err := t.writerFor(d.TargetID)
if err != nil {
return err
}
row := archivedEvent{
EventID: d.Event.ID,
WebhookID: webhookID,
EntrypointID: d.Event.EntrypointID,
Method: d.Event.Method,
RawQuery: d.Event.RawQuery,
Headers: d.Event.Headers,
Body: d.Event.Body,
ContentType: d.Event.ContentType,
}
return w.write(row, expiry, archivePeriod(rotation, receivedAt))
}
// writerFor returns the archive writer for a database target,
// creating and caching it on first use. Each target has one writer
// so its close/reopen debounce state is shared across concurrent
// deliveries, and so a rename and the idle sweep take the same lock
// as its writes.
func (t *databaseTarget) writerFor(
targetID string,
) (*archiveWriter, error) {
t.mu.Lock()
defer t.mu.Unlock()
w, ok := t.writers[targetID]
if !ok {
var err error
w, err = t.newWriter(targetID)
if err != nil {
return nil, err
}
if t.writers == nil {
t.writers = make(map[string]*archiveWriter)
}
t.writers[targetID] = w
}
// A delivery claims the entry: even if the idle sweep created
// it moments ago, it now belongs to the registry proper and
// the sweep must leave it in place when it finishes.
w.sweepOwned = false
return w, nil
}
// sweepWriterFor returns the archive writer the idle sweep should
// prune a target's archive through, together with whether the sweep
// itself created the registry entry.
//
// The sweep must route its prune through the registered writer so
// the writer's mutex orders it against concurrent writes, but it
// must never leave a registry entry behind: a sweep that ran
// concurrently with the target's deletion would otherwise
// re-create an entry that nothing will ever evict again, which is
// exactly the leak eviction exists to prevent. An entry the sweep
// creates is therefore marked sweep-owned and handed back to
// releaseSweepWriter when the sweep is done.
func (t *databaseTarget) sweepWriterFor(
targetID string,
) (*archiveWriter, bool, error) {
t.mu.Lock()
defer t.mu.Unlock()
w, ok := t.writers[targetID]
if ok {
return w, false, nil
}
w, err := t.newWriter(targetID)
if err != nil {
return nil, false, err
}
if t.writers == nil {
t.writers = make(map[string]*archiveWriter)
}
w.sweepOwned = true
t.writers[targetID] = w
return w, true, nil
}
// releaseSweepWriter drops a registry entry that the idle sweep
// created, so a sweep leaves the registry exactly as it found it.
//
// The entry is removed only if it is still the very writer the
// sweep installed and no delivery has claimed it in the meantime
// (writerFor clears sweepOwned when it hands a writer to the
// write path). Both conditions are evaluated under the registry
// lock, so an eviction that raced the sweep — which removes the
// entry outright — simply finds nothing left to do here, and a
// delivery that adopted the writer keeps a registered, evictable
// one.
func (t *databaseTarget) releaseSweepWriter(
targetID string, w *archiveWriter,
) {
t.mu.Lock()
defer t.mu.Unlock()
cur, ok := t.writers[targetID]
if !ok || cur != w || !cur.sweepOwned {
return
}
delete(t.writers, targetID)
}
// newWriter builds the writer for a database target's archive. Its
// path is the one ArchivePath gives for the webhook and the target as
// the main database names them now; from then on only rename changes
// the name the writer uses. It does not touch the archive files.
func (t *databaseTarget) newWriter(
targetID string,
) (*archiveWriter, error) {
if t.eng.dbManager == nil {
return nil, errArchiveNoDataDir
}
var target database.Target
err := t.eng.database.DB().
Preload("Webhook").
First(&target, "id = ?", targetID).Error
if err != nil {
return nil, fmt.Errorf(
"loading database target %s: %w", targetID, err,
)
}
w := newArchiveWriter(
ArchivePath(t.eng.dbManager, &target.Webhook, &target),
t.eng.log,
)
w.webhookID = target.WebhookID
return w, nil
}
// rename moves every one of a database target's archive files to the
// name for webhookName and targetName. It goes through the target's
// writer, so the move holds the lock that writes and the idle sweep
// take, and later writes use the new name.
//
// The writer is created if there is none, and it stays cached. The
// handlers rename before they save the new name, so until the save
// the main database still has the old one; a delivery in that window
// must find this writer rather than build one from the old name.
func (t *databaseTarget) rename(
targetID, webhookName, targetName string,
) error {
w, err := t.writerFor(targetID)
if err != nil {
return err
}
return w.rename(ArchiveFileName(webhookName, targetName, targetID))
}
// evict drops a database target's archive writer from the registry
// and closes its handle, so a deleted target does not leave a
// writer (and an open archive handle within its debounce window)
// alive for the process lifetime.
//
// The map entry is removed under the registry lock, which is
// then released before the handle is closed under the writer's
// own lock: that ordering keeps the registry available to other
// targets while an in-flight write on this one drains, and
// closing under the writer's lock means eviction can never race
// a write.
//
// Eviction is idempotent and silent for a target with no writer,
// which is the common case: only a database target that has
// received an event or been renamed has one. It never deletes the
// archive file.
func (t *databaseTarget) evict(targetID string) {
t.mu.Lock()
w, ok := t.writers[targetID]
if ok {
delete(t.writers, targetID)
}
t.mu.Unlock()
if !ok {
return
}
w.evict()
t.eng.log.Info(
"evicted archive writer",
"target_id", targetID,
"path", w.path,
)
}
// evictWebhook evicts, exactly as evict does, the writer of every
// database target of a webhook.
func (t *databaseTarget) evictWebhook(webhookID string) {
t.mu.Lock()
var gone []*archiveWriter
for targetID, w := range t.writers {
if w.webhookID == webhookID {
delete(t.writers, targetID)
gone = append(gone, w)
}
}
t.mu.Unlock()
for _, w := range gone {
w.evict()
t.eng.log.Info(
"evicted archive writer",
"webhook_id", webhookID,
"path", w.path,
)
}
}
// evictAll evicts every cached archive writer, exactly as evict
// does for one target. The engine calls it at shutdown, once its
// workers have returned. Closing the last handle on an archive
// moves the contents of its -wal into the .db and removes the
// -wal, so a clean stop leaves each archive as a single file.
func (t *databaseTarget) evictAll() {
t.mu.Lock()
writers := t.writers
t.writers = nil
t.mu.Unlock()
for _, w := range writers {
w.evict()
}
}
// sweepArchive prunes one database target's archive of rows older
// than expiry, without requiring a write. A missing archive file is
// left missing (see sweepExpired), so a sweep never creates an
// archive for a target that has never received an event.
//
// It also never leaves a registry entry behind: an entry it had
// to create to reach the writer's mutex is released again once
// the prune is done, so a sweep racing a target deletion cannot
// resurrect the writer the eviction just dropped.
func (t *databaseTarget) sweepArchive(
targetID string, expiry time.Duration,
) error {
w, created, err := t.sweepWriterFor(targetID)
if err != nil {
return err
}
if created {
defer t.releaseSweepWriter(targetID, w)
}
return w.sweepExpired(expiry)
}
+249 -617
View File
@@ -1,26 +1,69 @@
package delivery_test package delivery_test
import ( import (
"context"
"database/sql" "database/sql"
"fmt" "fmt"
"io/fs"
"log/slog" "log/slog"
"net/http"
"os" "os"
"path/filepath" "path/filepath"
"strings"
"testing" "testing"
"time" "time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"gorm.io/driver/sqlite" "gorm.io/driver/sqlite"
"gorm.io/gorm" "gorm.io/gorm"
"gorm.io/gorm/clause"
_ "modernc.org/sqlite" // Pure Go SQLite driver. _ "modernc.org/sqlite" // Pure Go SQLite driver.
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/database/databasetest"
"sneak.berlin/go/webhooker/internal/db/migrations"
"sneak.berlin/go/webhooker/internal/delivery" "sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/gormlog" "sneak.berlin/go/webhooker/internal/gormlog"
) )
// archiveTestWebhookName is the name of every webhook
// seedDatabaseTarget creates. It is not safe in a file name as it
// stands, so every archive file name made from it is made safe first.
const archiveTestWebhookName = "Sweep Test!"
// archiveEnv bundles the pieces a database target test drives: a main
// configuration database holding webhooks and targets, a delivery
// engine whose archive writers write into the data directory, and that
// directory.
type archiveEnv struct {
eng *delivery.Engine
mainDB *database.Database
dataDir string
}
func setupArchiveTest(t *testing.T) *archiveEnv {
t.Helper()
dataDir := t.TempDir()
mainDB, err := database.Open(dataDir, slog.New(slog.DiscardHandler))
require.NoError(t, err)
t.Cleanup(func() { _ = mainDB.Close() })
return &archiveEnv{
eng: delivery.NewTestEngineWithDB(
mainDB,
databasetest.NewWebhookDBManager(t, dataDir),
archiveTestLogger(),
&http.Client{Timeout: 5 * time.Second},
1,
),
mainDB: mainDB,
dataDir: dataDir,
}
}
func archiveTestLogger() *slog.Logger { func archiveTestLogger() *slog.Logger {
return slog.New(slog.NewTextHandler( return slog.New(slog.NewTextHandler(
os.Stderr, os.Stderr,
@@ -28,8 +71,87 @@ func archiveTestLogger() *slog.Logger {
)) ))
} }
// archivePath returns where the engine keeps a database target's
// archive file, for the names seedDatabaseTarget gave it.
func (env *archiveEnv) archivePath(tgt *database.Target) string {
return filepath.Join(
env.dataDir,
archive.FileName(archiveTestWebhookName, tgt.Name, tgt.ID),
)
}
// seedDatabaseTarget creates a webhook with one database target,
// which keeps its archive forever in one file, and returns the
// target.
func (env *archiveEnv) seedDatabaseTarget(
t *testing.T,
) *database.Target {
t.Helper()
wh := &database.Webhook{
UserID: uuid.New().String(),
Name: archiveTestWebhookName,
}
require.NoError(
t,
env.mainDB.DB().
Omit(clause.Associations).
Create(wh).Error,
)
tgt := &database.Target{
WebhookID: wh.ID,
Name: "Archive",
Type: database.TargetTypeDatabase,
Active: true,
ArchiveRotation: "none",
MaxAttempts: 1,
}
require.NoError(
t,
env.mainDB.DB().
Omit(clause.Associations).
Create(tgt).Error,
)
return tgt
}
// seedDatabaseTargetDelivery seeds a pending delivery of an event
// to a database target and returns the in-memory delivery the
// target handler is invoked with.
func seedDatabaseTargetDelivery(
t *testing.T,
webhookDB *gorm.DB,
event database.Event,
tgt *database.Target,
) *database.Delivery {
t.Helper()
dlv := seedDelivery(
t, webhookDB, event.ID, tgt.ID,
database.DeliveryStatusPending,
)
d := &database.Delivery{
EventID: event.ID,
TargetID: tgt.ID,
Status: database.DeliveryStatusPending,
Event: event,
Target: database.Target{
Name: tgt.Name,
Type: database.TargetTypeDatabase,
ArchiveExpiry: tgt.ArchiveExpiry,
ArchiveRotation: tgt.ArchiveRotation,
},
}
d.ID = dlv.ID
return d
}
// openArchiveDBForRead opens an archive file read-only so a // openArchiveDBForRead opens an archive file read-only so a
// test can inspect the rows the writer persisted. // test can inspect the rows the database target archived.
func openArchiveDBForRead( func openArchiveDBForRead(
t *testing.T, path string, t *testing.T, path string,
) *gorm.DB { ) *gorm.DB {
@@ -52,26 +174,45 @@ func openArchiveDBForRead(
return gdb return gdb
} }
// archiveFileSuffixes returns the archive file itself and the // countArchivedRows counts the rows in an archive file without
// SQLite sidecars that accompany an open database. A test that // asserting anything, so it is safe to poll from an
// asserts no archive was created has to check all of them. // assert.Eventually condition (which runs off the test
func archiveFileSuffixes() []string { // goroutine, where testify assertions must not be used).
return []string{"", "-wal", "-shm"} func countArchivedRows(path string) (int64, error) {
sqlDB, err := sql.Open(
"sqlite", fmt.Sprintf("file:%s?mode=ro", path),
)
if err != nil {
return 0, err
}
defer func() { _ = sqlDB.Close() }()
gdb, err := gorm.Open(
sqlite.Dialector{Conn: sqlDB},
&gorm.Config{Logger: gormlog.New(slog.New(slog.DiscardHandler))},
)
if err != nil {
return 0, err
}
var count int64
err = gdb.Table("archived_events").Count(&count).Error
if err != nil {
return 0, err
}
return count, nil
} }
// removeArchiveFiles simulates an operator moving the archive // archivedRow is the columns of an archive row these tests read.
// away by deleting the SQLite file and its sidecar files. type archivedRow struct {
func removeArchiveFiles(t *testing.T, path string) { EventID string
t.Helper() WebhookID string
Method string
for _, suffix := range []string{ RawQuery string
"", "-wal", "-shm", "-journal", Body string
} {
err := os.Remove(path + suffix)
if err != nil && !os.IsNotExist(err) {
t.Fatalf("removing %s%s: %v", path, suffix, err)
}
}
} }
// TestDeliverDatabase_ArchivesEvent verifies that delivering to // TestDeliverDatabase_ArchivesEvent verifies that delivering to
@@ -81,7 +222,7 @@ func TestDeliverDatabase_ArchivesEvent(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, "") tgt := env.seedDatabaseTarget(t)
webhookDB := testWebhookDB(t) webhookDB := testWebhookDB(t)
event := seedEvent(t, webhookDB, `{"archived":true}`) event := seedEvent(t, webhookDB, `{"archived":true}`)
@@ -107,9 +248,9 @@ func TestDeliverDatabase_ArchivesEvent(t *testing.T) {
rdb := openArchiveDBForRead(t, archivePath) rdb := openArchiveDBForRead(t, archivePath)
var rows []delivery.ExportArchivedEvent var rows []archivedRow
require.NoError(t, rdb.Find(&rows).Error) require.NoError(t, rdb.Table("archived_events").Find(&rows).Error)
require.Len(t, rows, 1) require.Len(t, rows, 1)
assert.Equal(t, event.ID, rows[0].EventID) assert.Equal(t, event.ID, rows[0].EventID)
assert.Equal(t, event.WebhookID, rows[0].WebhookID) assert.Equal(t, event.WebhookID, rows[0].WebhookID)
@@ -118,259 +259,10 @@ func TestDeliverDatabase_ArchivesEvent(t *testing.T) {
assert.JSONEq(t, `{"archived":true}`, rows[0].Body) assert.JSONEq(t, `{"archived":true}`, rows[0].Body)
} }
func TestArchiveWriter_WritesRow(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive-wh.db")
w := delivery.NewExportArchiveWriter(
path, archiveTestLogger(), 0,
)
row := delivery.ExportArchivedEvent{
EventID: "ev-1",
WebhookID: "wh-1",
EntrypointID: "ep-1",
Method: "POST",
Headers: `{"X":"Y"}`,
Body: `{"hello":"world"}`,
ContentType: "application/json",
}
require.NoError(t, w.Write(row, 0))
assert.FileExists(t, path)
var got []delivery.ExportArchivedEvent
require.NoError(t, w.DB().Find(&got).Error)
require.Len(t, got, 1)
assert.Equal(t, "ev-1", got[0].EventID)
assert.Equal(t, "wh-1", got[0].WebhookID)
assert.Equal(t, "ep-1", got[0].EntrypointID)
assert.Equal(t, row.Method, got[0].Method)
assert.Equal(t, row.ContentType, got[0].ContentType)
assert.JSONEq(t, `{"hello":"world"}`, got[0].Body)
assert.False(t, got[0].ArchivedAt.IsZero())
}
func TestArchiveWriter_RecreatesAfterRemoval(
t *testing.T,
) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive-wh.db")
w := delivery.NewExportArchiveWriter(
path, archiveTestLogger(), 0,
)
require.NoError(t, w.Write(
delivery.ExportArchivedEvent{EventID: "a"}, 0,
))
assert.FileExists(t, path)
// The operator moves the archive away while the handle is
// still open.
removeArchiveFiles(t, path)
require.NoFileExists(t, path)
// The next write recreates the file with a fresh schema and
// only the new row.
require.NoError(t, w.Write(
delivery.ExportArchivedEvent{EventID: "b"}, 0,
))
assert.FileExists(t, path)
var got []delivery.ExportArchivedEvent
require.NoError(t, w.DB().Find(&got).Error)
require.Len(t, got, 1)
assert.Equal(t, "b", got[0].EventID)
}
// TestStatArchive proves StatArchive finds no file before the first
// write; after a write still held in the -wal, counts the -wal in the
// size and takes its later time as the last write; and finds no file
// again once the file has been moved away.
func TestStatArchive(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive-wh.db")
_, err := delivery.StatArchive(path)
require.ErrorIs(t, err, fs.ErrNotExist)
// With the clock stopped, the reopen debounce never passes, so
// the handle stays open after the write.
stopped := time.Now()
w := delivery.NewExportArchiveWriter(path, archiveTestLogger(), 0)
w.SetNow(func() time.Time { return stopped })
require.NoError(t, w.Write(delivery.ExportArchivedEvent{EventID: "a"}, 0))
written := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
earlier := written.Add(-time.Hour)
require.NoError(t, os.Chtimes(path, earlier, earlier))
require.NoError(t, os.Chtimes(path+"-wal", written, written))
file, err := os.Stat(path)
require.NoError(t, err)
wal, err := os.Stat(path + "-wal")
require.NoError(t, err)
require.Positive(t, wal.Size())
got, err := delivery.StatArchive(path)
require.NoError(t, err)
assert.Equal(t, 1, got.Files)
assert.Equal(t, file.Size()+wal.Size(), got.Size)
assert.True(t, written.Equal(got.Written), got.Written)
removeArchiveFiles(t, path)
_, err = delivery.StatArchive(path)
require.ErrorIs(t, err, fs.ErrNotExist)
}
func TestArchiveWriter_ReopenDebounce(t *testing.T) {
t.Parallel()
const debounce = 2 * time.Second
path := filepath.Join(t.TempDir(), "archive-wh.db")
w := delivery.NewExportArchiveWriter(
path, archiveTestLogger(), debounce,
)
// The writer measures its reopen debounce on this clock, which
// only the test moves, so how long the host takes between
// writes cannot change the result.
now := time.Now()
w.SetNow(func() time.Time { return now })
require.NoError(t, w.Write(
delivery.ExportArchivedEvent{EventID: "a"}, 0,
))
require.NoError(t, w.Write(
delivery.ExportArchivedEvent{EventID: "b"}, 0,
))
// Two writes inside the debounce window trigger only the
// initial open — no extra close/reopen.
assert.Equal(t, 1, w.Reopens())
now = now.Add(debounce)
require.NoError(t, w.Write(
delivery.ExportArchivedEvent{EventID: "c"}, 0,
))
// A write after the window elapses closes and reopens once.
assert.Equal(t, 2, w.Reopens())
}
func TestArchiveWriter_ExpiryPrune(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive-wh.db")
w := delivery.NewExportArchiveWriter(
path, archiveTestLogger(), 0,
)
require.NoError(t, w.Open(0))
old := delivery.ExportArchivedEvent{
EventID: "old",
ArchivedAt: time.Now().Add(-2 * time.Hour),
}
fresh := delivery.ExportArchivedEvent{
EventID: "fresh",
ArchivedAt: time.Now(),
}
require.NoError(t, w.DB().Create(&old).Error)
require.NoError(t, w.DB().Create(&fresh).Error)
// Reopening with a one-hour expiry prunes the old row.
require.NoError(t, w.Reopen(time.Hour))
var got []delivery.ExportArchivedEvent
require.NoError(t, w.DB().Find(&got).Error)
require.Len(t, got, 1)
assert.Equal(t, "fresh", got[0].EventID)
}
func TestParseArchiveExpiry(t *testing.T) {
t.Parallel()
cases := []struct {
name string
in string
want time.Duration
wantErr bool
}{
{"empty config", "", 0, false},
{"explicit never", `{"expiry":"never"}`, 0, false},
{"empty expiry", `{"expiry":""}`, 0, false},
{"duration", `{"expiry":"1h"}`, time.Hour, false},
{"unparseable", `{"expiry":"nonsense"}`, 0, true},
{"zero duration", `{"expiry":"0s"}`, 0, true},
{"negative duration", `{"expiry":"-5h"}`, 0, true},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
got, err := delivery.ExportParseArchiveExpiry(tc.in)
if tc.wantErr {
require.Error(t, err)
return
}
require.NoError(t, err)
assert.Equal(t, tc.want, got)
})
}
}
// seedDatabaseTargetDelivery seeds a pending delivery of an event
// to a database target and returns the in-memory delivery the
// target handler is invoked with.
func seedDatabaseTargetDelivery(
t *testing.T,
webhookDB *gorm.DB,
event database.Event,
tgt *database.Target,
) *database.Delivery {
t.Helper()
dlv := seedDelivery(
t, webhookDB, event.ID, tgt.ID,
database.DeliveryStatusPending,
)
d := &database.Delivery{
EventID: event.ID,
TargetID: tgt.ID,
Status: database.DeliveryStatusPending,
Event: event,
Target: database.Target{
Name: tgt.Name,
Type: database.TargetTypeDatabase,
Config: tgt.Config,
},
}
d.ID = dlv.ID
return d
}
// TestDeliverDatabase_ArchiveFailureFailsDelivery verifies that // TestDeliverDatabase_ArchiveFailureFailsDelivery verifies that
// an archive error (here: an unparseable expiry in the target // an archive error (here: an event with no webhook id to record)
// config) fails the delivery loudly: the attempt is recorded as // fails the delivery loudly: the attempt is recorded as failed
// failed with the error and the delivery is marked failed, not // with the error and the delivery is marked failed, not
// delivered. // delivered.
func TestDeliverDatabase_ArchiveFailureFailsDelivery( func TestDeliverDatabase_ArchiveFailureFailsDelivery(
t *testing.T, t *testing.T,
@@ -378,11 +270,12 @@ func TestDeliverDatabase_ArchiveFailureFailsDelivery(
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, `{"expiry":"nonsense"}`) tgt := env.seedDatabaseTarget(t)
webhookDB := testWebhookDB(t) webhookDB := testWebhookDB(t)
event := seedEvent(t, webhookDB, `{"archived":false}`) event := seedEvent(t, webhookDB, `{"archived":false}`)
d := seedDatabaseTargetDelivery(t, webhookDB, event, tgt) d := seedDatabaseTargetDelivery(t, webhookDB, event, tgt)
d.Event.WebhookID = ""
env.eng.ExportDeliverDatabase(webhookDB, d) env.eng.ExportDeliverDatabase(webhookDB, d)
@@ -407,357 +300,96 @@ func TestDeliverDatabase_ArchiveFailureFailsDelivery(
"the attempt must be recorded as failed", "the attempt must be recorded as failed",
) )
assert.Contains(t, assert.Contains(t,
results[0].Error, "nonsense", results[0].Error, "without a webhook id",
"the archive error must be recorded on the attempt", "the archive error must be recorded on the attempt",
) )
assert.NoFileExists(t, assert.NoFileExists(t,
env.archivePath(tgt), env.archivePath(tgt),
"no archive file should exist for a failed config", "no archive file should exist for a failed write",
) )
} }
func TestValidateArchiveExpiry(t *testing.T) { // seedArchivedRow creates the archive file at path holding one row, for
t.Parallel() // eventID, archived at archivedAt.
func seedArchivedRow(
t *testing.T, path, eventID string, archivedAt time.Time,
) {
t.Helper()
valid := []string{"", "never", "1h", "720h", "30m"} sqlDB, err := sql.Open(
for _, in := range valid { "sqlite", fmt.Sprintf("file:%s?mode=rwc", path),
require.NoError(t,
delivery.ValidateArchiveExpiry(in),
"expiry %q should be accepted", in,
)
}
invalid := []string{"nonsense", "7d", "-5h", "0s", "0"}
for _, in := range invalid {
require.Error(t,
delivery.ValidateArchiveExpiry(in),
"expiry %q should be rejected", in,
)
}
}
// TestArchiveFileName pins the archive file name and the rules
// that make a webhook or target name safe to put in it.
func TestArchiveFileName(t *testing.T) {
t.Parallel()
const id = "3f2a1c9e-8d4b-4c1a-9e2f-0a1b2c3d4e5f"
cases := []struct {
name string
webhook string
target string
want string
}{
{
"plain names", "orders", "archive",
"archive-orders-archive-" + id + ".db",
},
{
"lowercased", "Orders", "Main Archive",
"archive-orders-main-archive-" + id + ".db",
},
{
"a run of other characters is one dash",
`a /\..b`, "c__--d",
"archive-a-b-c-d-" + id + ".db",
},
{
"no dash at either end", " --orders!! ", "(archive)",
"archive-orders-archive-" + id + ".db",
},
{
"path separators", "../../etc/passwd", "a/b",
"archive-etc-passwd-a-b-" + id + ".db",
},
{
"letters outside ASCII are dropped",
"Bestellungen Größe", "café",
"archive-bestellungen-gr-e-caf-" + id + ".db",
},
{
"nothing left is unnamed", "", "!!!",
"archive-unnamed-unnamed-" + id + ".db",
},
{
"cut to 40 characters", strings.Repeat("a", 50), "x",
"archive-" + strings.Repeat("a", 40) + "-x-" + id + ".db",
},
{
"no dash left by the cut",
strings.Repeat("a", 39) + " b", "x",
"archive-" + strings.Repeat("a", 39) + "-x-" + id + ".db",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
assert.Equal(
t, tc.want,
delivery.ArchiveFileName(tc.webhook, tc.target, id),
)
})
}
}
// TestDeliverDatabase_EachTargetHasItsOwnArchive proves two
// database targets of one webhook archive into separate files.
func TestDeliverDatabase_EachTargetHasItsOwnArchive(t *testing.T) {
t.Parallel()
env := setupArchiveTest(t)
first := env.seedDatabaseTarget(t, "")
second := env.addDatabaseTarget(t, first.WebhookID, "")
webhookDB := testWebhookDB(t)
event := seedEvent(t, webhookDB, `{"n":1}`)
for _, tgt := range []*database.Target{first, second} {
env.eng.ExportDeliverDatabase(
webhookDB,
seedDatabaseTargetDelivery(t, webhookDB, event, tgt),
)
}
require.NotEqual(
t, env.archivePath(first), env.archivePath(second),
) )
assert.Equal(
t, []string{event.ID},
archivedEventIDs(t, env.archivePath(first)),
)
assert.Equal(
t, []string{event.ID},
archivedEventIDs(t, env.archivePath(second)),
)
}
// TestRename_MovesTheFile proves a rename moves the archive, rows
// and all, and that later writes go to the new name.
func TestRename_MovesTheFile(t *testing.T) {
t.Parallel()
env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, "")
oldPath := env.archivePath(tgt)
webhookDB := testWebhookDB(t)
first := seedEvent(t, webhookDB, `{"n":1}`)
env.eng.ExportDeliverDatabase(
webhookDB, seedDatabaseTargetDelivery(t, webhookDB, first, tgt),
)
require.FileExists(t, oldPath)
require.NoError(
t, env.eng.Rename(tgt.ID, "Orders", "Long Term"),
)
newPath := filepath.Join(
env.dataDir, "archive-orders-long-term-"+tgt.ID+".db",
)
assert.NoFileExists(t, oldPath)
assert.Equal(t, []string{first.ID}, archivedEventIDs(t, newPath))
second := seedEvent(t, webhookDB, `{"n":2}`)
env.eng.ExportDeliverDatabase(
webhookDB,
seedDatabaseTargetDelivery(t, webhookDB, second, tgt),
)
assert.ElementsMatch(
t, []string{first.ID, second.ID},
archivedEventIDs(t, newPath),
)
assert.NoFileExists(
t, oldPath, "a write after the rename must use the new name",
)
}
// TestRename_NeverReplacesAFile plants a file at the new name, once
// the .db alone, once a lone -wal and once a lone -shm, and proves
// each time that the rename is refused, the planted file survives,
// and the archive keeps its name and its rows.
func TestRename_NeverReplacesAFile(t *testing.T) {
t.Parallel()
for _, suffix := range archiveFileSuffixes() {
t.Run("planted .db"+suffix, func(t *testing.T) {
t.Parallel()
env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, "")
oldPath := env.archivePath(tgt)
webhookDB := testWebhookDB(t)
first := seedEvent(t, webhookDB, `{"n":1}`)
env.eng.ExportDeliverDatabase(
webhookDB,
seedDatabaseTargetDelivery(t, webhookDB, first, tgt),
)
newPath := filepath.Join(
env.dataDir, "archive-orders-long-term-"+tgt.ID+".db",
)
plantedPath := newPath + suffix
require.NoError(
t, os.WriteFile(plantedPath, []byte("planted"), 0o600),
)
require.ErrorIs(
t, env.eng.Rename(tgt.ID, "Orders", "Long Term"),
delivery.ErrArchiveNameTaken,
)
//nolint:gosec // reads the file the test planted under t.TempDir()
planted, err := os.ReadFile(plantedPath)
require.NoError(t, err)
assert.Equal(t, "planted", string(planted))
second := seedEvent(t, webhookDB, `{"n":2}`)
env.eng.ExportDeliverDatabase(
webhookDB,
seedDatabaseTargetDelivery(t, webhookDB, second, tgt),
)
assert.ElementsMatch(
t, []string{first.ID, second.ID},
archivedEventIDs(t, oldPath),
)
})
}
}
// TestRename_BeforeTheNameIsSaved covers the order the handlers
// use: they rename before they save the new name, so a delivery in
// between must write under the new name although the main database
// still has the old one. It also shows that renaming an archive that
// does not exist yet is not an error.
func TestRename_BeforeTheNameIsSaved(t *testing.T) {
t.Parallel()
env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, "")
require.NoError(
t, env.eng.Rename(tgt.ID, "Orders", "Archive"),
)
webhookDB := testWebhookDB(t)
event := seedEvent(t, webhookDB, `{"n":1}`)
env.eng.ExportDeliverDatabase(
webhookDB, seedDatabaseTargetDelivery(t, webhookDB, event, tgt),
)
assert.FileExists(
t,
filepath.Join(
env.dataDir, "archive-orders-archive-"+tgt.ID+".db",
),
)
assert.NoFileExists(t, env.archivePath(tgt))
}
// TestArchiveWriter_RenameMovesSidecars proves a rename carries
// the -wal and -shm a crash can leave beside an archive no handle
// has opened since. SQLite finds them by name, so a -wal left
// behind would lose the transactions it holds.
func TestArchiveWriter_RenameMovesSidecars(t *testing.T) {
t.Parallel()
dir := t.TempDir()
oldPath := filepath.Join(dir, "archive-old.db")
newPath := filepath.Join(dir, "archive-new.db")
for _, suffix := range archiveFileSuffixes() {
require.NoError(
t, os.WriteFile(oldPath+suffix, []byte(suffix), 0o600),
)
}
w := delivery.NewExportArchiveWriter(
oldPath, archiveTestLogger(), 0,
)
require.NoError(t, w.Rename("archive-new.db"))
for _, suffix := range archiveFileSuffixes() {
assert.NoFileExists(t, oldPath+suffix)
assert.FileExists(t, newPath+suffix)
}
assert.Equal(t, newPath, w.Path())
}
// TestArchiveWriter_RenameMovesBackOnFailure makes the -wal fail to
// move after the .db has moved, and proves the .db is moved back, so
// the archive is never split across two names. The new name is 255
// bytes, the longest a file name may be, so the .db can take it but
// the -wal, four bytes longer, cannot.
func TestArchiveWriter_RenameMovesBackOnFailure(t *testing.T) {
t.Parallel()
dir := t.TempDir()
oldPath := filepath.Join(dir, "archive-old.db")
newName := strings.Repeat("a", 252) + ".db"
for _, suffix := range archiveFileSuffixes() {
require.NoError(
t, os.WriteFile(oldPath+suffix, []byte(suffix), 0o600),
)
}
w := delivery.NewExportArchiveWriter(
oldPath, archiveTestLogger(), 0,
)
require.Error(t, w.Rename(newName))
for _, suffix := range archiveFileSuffixes() {
assert.FileExists(t, oldPath+suffix)
}
assert.NoFileExists(t, filepath.Join(dir, newName))
assert.Equal(t, oldPath, w.Path())
}
// TestArchiveWriter_RenameMovesBackEveryFile renames a target with a
// file without a period and a file for a month, and proves that when
// the month's file fails to move, the file already moved is moved back:
// both files are under the old name with their rows, and nothing is
// under the new name. The new name is 251 bytes, so the file without a
// period, with its -wal and -shm, can take it, but the month's file,
// eight bytes longer, cannot.
func TestArchiveWriter_RenameMovesBackEveryFile(t *testing.T) {
t.Parallel()
dir := t.TempDir()
oldPath := filepath.Join(dir, "archive-old.db")
monthPath := filepath.Join(dir, "archive-old-2026-03.db")
w := delivery.NewExportArchiveWriter(oldPath, archiveTestLogger(), 0)
require.NoError(t, w.WritePeriod(
delivery.ExportArchivedEvent{EventID: "in-none"}, 0, "",
))
require.NoError(t, w.WritePeriod(
delivery.ExportArchivedEvent{EventID: "in-month"}, 0, "2026-03",
))
require.Error(t, w.Rename(strings.Repeat("a", 248)+".db"))
assert.Equal(t, []string{"in-none"}, archivedEventIDs(t, oldPath))
assert.Equal(t, []string{"in-month"}, archivedEventIDs(t, monthPath))
entries, err := os.ReadDir(dir)
require.NoError(t, err) require.NoError(t, err)
for _, entry := range entries { defer func() { _ = sqlDB.Close() }()
assert.True(t, strings.HasPrefix(entry.Name(), "archive-old"),
"%s is not under the old name", entry.Name())
}
assert.Equal(t, oldPath, w.Path()) gdb, err := gorm.Open(
sqlite.Dialector{Conn: sqlDB},
&gorm.Config{Logger: gormlog.New(slog.New(slog.DiscardHandler))},
)
require.NoError(t, err)
require.NoError(t, migrations.Apply(gdb, migrations.Archive))
require.NoError(t, gdb.Exec(
"INSERT INTO archived_events (event_id, archived_at) VALUES (?, ?)",
eventID, archivedAt,
).Error)
}
// TestDeliverDatabase_TaskArchiveSettings proves a task's ArchiveExpiry
// and ArchiveRotation reach the archive through the engine. The
// target's row has neither, so only the task can bring them: with daily
// rotation the event goes to the file for its day, and opening that
// file with a one-hour expiry prunes the row it already held, archived
// two hours ago.
func TestDeliverDatabase_TaskArchiveSettings(t *testing.T) {
t.Parallel()
s := newISetup(t)
webhook := database.Webhook{UserID: uuid.NewString(), Name: "settings"}
webhook.ID = s.WebhookID
require.NoError(t, s.MainDB.Create(&webhook).Error)
target := database.Target{
WebhookID: s.WebhookID,
Name: "archive",
Type: database.TargetTypeDatabase,
Active: true,
MaxAttempts: 1,
}
require.NoError(t, s.MainDB.Create(&target).Error)
event := iSeedEvent(t, s.WebhookDB, s.WebhookID, `{"archived":true}`)
d := iSeedDelivery(
t, s.WebhookDB, event.ID, target.ID,
database.DeliveryStatusPending,
)
task := iTask(
d, event, s.WebhookID, target.ID, target.Name, "", 1, 1,
&event.Body,
)
task.TargetType = database.TargetTypeDatabase
task.ArchiveExpiry = time.Hour
task.ArchiveRotation = "daily"
daily := target
daily.ArchiveRotation = task.ArchiveRotation
dailyFile := archive.PathAt(s.DBMgr, &webhook, &daily, event.CreatedAt)
seedArchivedRow(t, dailyFile, "expired", time.Now().Add(-2*time.Hour))
s.Engine.ExportProcessNewTask(context.TODO(), &task)
iAssertStatus(t, s.WebhookDB, d.ID, database.DeliveryStatusDelivered)
var rows []archivedRow
require.NoError(t, openArchiveDBForRead(t, dailyFile).
Table("archived_events").Find(&rows).Error)
require.Len(t, rows, 1)
assert.Equal(t, event.ID, rows[0].EventID,
"the event goes to its day's file and the expired row is pruned",
)
} }
+7 -9
View File
@@ -17,7 +17,7 @@ import (
const MaxTargetTimeoutSeconds = 300 const MaxTargetTimeoutSeconds = 300
// Errors returned when a target's header or timeout form input // Errors returned when a target's header or timeout form input
// cannot be turned into a configuration. // cannot be turned into the target's setting.
// //
// None of these ever quotes a header VALUE. A target header value // None of these ever quotes a header VALUE. A target header value
// is routinely an authorization token, and these messages are shown // is routinely an authorization token, and these messages are shown
@@ -48,8 +48,8 @@ var (
) )
// isReservedTargetHeader reports whether name (canonicalised) is a // isReservedTargetHeader reports whether name (canonicalised) is a
// header a target configuration may not set, because the delivery // header a target may not set, because the delivery path or
// path or net/http itself writes it regardless. // net/http itself writes it regardless.
// //
// These are rejected rather than accepted-and-ignored. Storing a // These are rejected rather than accepted-and-ignored. Storing a
// header that provably never reaches the wire tells the operator // header that provably never reaches the wire tells the operator
@@ -77,13 +77,11 @@ func isReservedTargetHeader(name string) bool {
// ParseTargetHeaders turns the target form's headers field — one // ParseTargetHeaders turns the target form's headers field — one
// "Name: value" pair per line, blank lines ignored — into the map // "Name: value" pair per line, blank lines ignored — into the map
// stored in HTTPTargetConfig.Headers. Names are canonicalised, so a // stored in database.Target.Headers. Names are canonicalised, so a
// name repeated in a different case is still a duplicate rather than // name repeated in a different case is still a duplicate rather than
// one pair silently overwriting the other. // one pair silently overwriting the other.
// //
// An input with no pairs yields an empty map, which omitempty drops // An input with no pairs yields an empty map.
// from the stored config: a target configured with no headers keeps
// the same config JSON it had before this field existed.
func ParseTargetHeaders(raw string) (map[string]string, error) { func ParseTargetHeaders(raw string) (map[string]string, error) {
headers := make(map[string]string) headers := make(map[string]string)
@@ -223,8 +221,8 @@ func FormatTargetHeaders(headers map[string]string) string {
// ParseTargetTimeout interprets the target form's timeout field as // ParseTargetTimeout interprets the target form's timeout field as
// a whole number of seconds. An empty field means "unset" and yields // a whole number of seconds. An empty field means "unset" and yields
// 0, which omitempty drops from the stored config and which the // 0, which the delivery path reads as "use the shared client's
// delivery path reads as "use the shared client's timeout". // timeout".
// //
// Anything else that is not a whole number in range is an error, not // Anything else that is not a whole number in range is an error, not
// a silently substituted default: a target whose timeout was typed // a silently substituted default: a target whose timeout was typed
+4 -97
View File
@@ -1,23 +1,21 @@
package delivery_test package delivery_test
import ( import (
"encoding/json"
"strconv" "strconv"
"testing" "testing"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery" "sneak.berlin/go/webhooker/internal/delivery"
) )
// Literals these tests repeat, named so that the header names and the // Literals these tests repeat, named so that each header name has one
// keep-forever archive config each have one definition. // definition.
const ( const (
headerAuthorization = "Authorization" headerAuthorization = "Authorization"
headerContentType = "Content-Type" headerContentType = "Content-Type"
headerUserAgent = "User-Agent"
bearerValue = "Bearer abc" bearerValue = "Bearer abc"
archiveConfigNever = "{\"expiry\":\"never\"}"
) )
func TestParseTargetHeaders_AcceptsPairs(t *testing.T) { func TestParseTargetHeaders_AcceptsPairs(t *testing.T) {
@@ -41,23 +39,13 @@ func TestParseTargetHeaders_AcceptsPairs(t *testing.T) {
) )
} }
// A configuration with no headers must stay indistinguishable from // An input of blank lines is no headers, not an error.
// one written before the field existed, so omitempty drops the key.
func TestParseTargetHeaders_EmptyInputYieldsNoHeaders(t *testing.T) { func TestParseTargetHeaders_EmptyInputYieldsNoHeaders(t *testing.T) {
t.Parallel() t.Parallel()
got, err := delivery.ParseTargetHeaders("\n \n") got, err := delivery.ParseTargetHeaders("\n \n")
require.NoError(t, err) require.NoError(t, err)
assert.Empty(t, got) assert.Empty(t, got)
encoded, err := json.Marshal(delivery.HTTPTargetConfig{
URL: "https://example.com/h",
Headers: got,
})
require.NoError(t, err)
assert.JSONEq(
t, `{"url":"https://example.com/h"}`, string(encoded),
)
} }
func TestParseTargetHeaders_Rejects(t *testing.T) { func TestParseTargetHeaders_Rejects(t *testing.T) {
@@ -181,87 +169,6 @@ func TestFormatTargetTimeout(t *testing.T) {
assert.Empty(t, delivery.FormatTargetTimeout(-1)) assert.Empty(t, delivery.FormatTargetTimeout(-1))
} }
func TestNewTargetConfigForm(t *testing.T) {
t.Parallel()
form, err := delivery.NewTargetConfigForm(&database.Target{
Type: database.TargetTypeHTTP,
Config: `{"url":"https://example.com/h",` +
`"headers":{"Authorization":"Bearer abc"},` +
`"timeout":9}`,
})
require.NoError(t, err)
assert.Equal(t, "https://example.com/h", form.URL)
assert.Equal(t, "Authorization: Bearer abc\n", form.Headers)
assert.Equal(t, "9", form.Timeout)
form, err = delivery.NewTargetConfigForm(&database.Target{
Type: database.TargetTypeSlack,
Config: `{"webhookUrl":"https://hooks.example/s"}`,
})
require.NoError(t, err)
assert.Equal(t, "https://hooks.example/s", form.URL)
form, err = delivery.NewTargetConfigForm(&database.Target{
Type: database.TargetTypeDatabase,
Config: `{"expiry":"720h","rotation":"daily"}`,
})
require.NoError(t, err)
assert.Equal(t, "720h", form.Expiry)
assert.Equal(t, rotationDaily, form.Rotation)
form, err = delivery.NewTargetConfigForm(&database.Target{
Type: database.TargetTypeLog,
})
require.NoError(t, err)
assert.Empty(t, form.URL)
}
// A keep-forever archive target yields an empty expiry, so the edit
// form starts on never; saving it unchanged stores never, which means
// the same as an empty expiry.
func TestNewTargetConfigForm_DatabaseNeverIsBlank(t *testing.T) {
t.Parallel()
for _, cfg := range []string{"", `{}`, archiveConfigNever} {
form, err := delivery.NewTargetConfigForm(
&database.Target{
Type: database.TargetTypeDatabase,
Config: cfg,
},
)
require.NoError(t, err, cfg)
assert.Empty(t, form.Expiry, cfg)
}
}
// An unreadable stored config is an error rather than a blank form
// that looks like a target with no settings, so the caller can tell
// the operator that saving replaces the stored value.
func TestNewTargetConfigForm_UnreadableConfigErrors(t *testing.T) {
t.Parallel()
cases := []*database.Target{
{Type: database.TargetTypeHTTP, Config: "not json"},
{Type: database.TargetTypeHTTP, Config: `{}`},
{Type: database.TargetTypeSlack, Config: ""},
{
Type: database.TargetTypeDatabase,
Config: `{"expiry":"soon"}`,
},
{
Type: database.TargetTypeDatabase,
Config: weeklyConfig,
},
{Type: database.TargetType("nope")},
}
for _, target := range cases {
_, err := delivery.NewTargetConfigForm(target)
require.Error(t, err, target.Type)
}
}
// The ceiling exists so one misconfigured target cannot hold a // The ceiling exists so one misconfigured target cannot hold a
// delivery worker indefinitely, and it is inclusive. // delivery worker indefinitely, and it is inclusive.
func TestParseTargetTimeout_CeilingIsInclusive(t *testing.T) { func TestParseTargetTimeout_CeilingIsInclusive(t *testing.T) {
+82 -135
View File
@@ -4,7 +4,6 @@ import (
"bytes" "bytes"
"context" "context"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
@@ -17,29 +16,6 @@ import (
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
) )
// Sentinel errors returned by the config parsers.
var (
errEmptyTargetConfig = errors.New(
"empty target config",
)
errMissingTargetURL = errors.New(
"target URL is required",
)
)
// HTTPTargetConfig holds configuration for http target
// types.
type HTTPTargetConfig struct {
URL string `json:"url"`
Headers map[string]string `json:"headers,omitempty"`
Timeout int `json:"timeout,omitempty"`
// ForwardQuery passes each event's query string on to the target,
// appended to URL. Off, the target URL is sent exactly as
// configured.
ForwardQuery bool `json:"forwardQuery,omitempty"`
}
// httpCore holds the retry, backoff, and circuit-breaker // httpCore holds the retry, backoff, and circuit-breaker
// machinery shared by the HTTP and Slack targets. Each of // machinery shared by the HTTP and Slack targets. Each of
// those targets owns its own httpCore instance (and thus its // those targets owns its own httpCore instance (and thus its
@@ -53,34 +29,72 @@ type httpCore struct {
} }
// deliver runs one delivery attempt through the retry core. // deliver runs one delivery attempt through the retry core.
// A maxRetries of 0 is fire-and-forget: a single attempt is // A maxAttempts of 1 is a single attempt, recorded without
// recorded and no circuit breaker is consulted. A positive // consulting a circuit breaker. A higher maxAttempts gates the
// maxRetries gates the attempt on the circuit breaker and // attempt on the circuit breaker and schedules a backed-off retry
// schedules a backed-off retry on failure. // on failure, until maxAttempts attempts have been made.
//
// ctx is the context attempt sends its request with. Stopping the
// engine cancels it; see interruptedByStop.
func (c *httpCore) deliver( func (c *httpCore) deliver(
ctx context.Context,
webhookDB *gorm.DB, webhookDB *gorm.DB,
d *database.Delivery, d *database.Delivery,
task *Task, task *Task,
sched Scheduler, sched Scheduler,
maxRetries int, maxAttempts int,
attempt func() attemptResult, attempt func() attemptResult,
) { ) {
if maxRetries == 0 { if maxAttempts == 1 {
c.fireAndForget(webhookDB, d, attempt()) c.fireAndForget(ctx, webhookDB, d, attempt())
return return
} }
c.withRetry( c.withRetry(
webhookDB, d, task, sched, maxRetries, attempt, ctx, webhookDB, d, task, sched, maxAttempts, attempt,
) )
} }
// interruptedByStop reports whether an attempt failed because the
// engine is stopping, which cancels the context its request is made
// with. Such an attempt is not recorded: no DeliveryResult, no circuit
// breaker outcome, no status change, no count in the attempt metrics.
// The delivery stays pending or retrying, and the recovery routine
// sends it again on the next start.
//
// A target's own timeout leaves that context alone, so its attempt is
// still recorded as the failure it is.
func (c *httpCore) interruptedByStop(
ctx context.Context,
d *database.Delivery,
res attemptResult,
) bool {
if res.success || ctx.Err() == nil {
return false
}
c.eng.log.Warn(
"delivery attempt interrupted by shutdown; not recorded, "+
"it is sent again on the next start",
"delivery_id", d.ID,
"event_id", d.EventID,
"target_id", d.TargetID,
)
return true
}
func (c *httpCore) fireAndForget( func (c *httpCore) fireAndForget(
ctx context.Context,
webhookDB *gorm.DB, webhookDB *gorm.DB,
d *database.Delivery, d *database.Delivery,
res attemptResult, res attemptResult,
) { ) {
if c.interruptedByStop(ctx, d, res) {
return
}
c.eng.observeAttempt(d.Target.Type, res.elapsed()) c.eng.observeAttempt(d.Target.Type, res.elapsed())
err := c.eng.recordResult( err := c.eng.recordResult(
@@ -110,11 +124,12 @@ func (c *httpCore) fireAndForget(
} }
func (c *httpCore) withRetry( func (c *httpCore) withRetry(
ctx context.Context,
webhookDB *gorm.DB, webhookDB *gorm.DB,
d *database.Delivery, d *database.Delivery,
task *Task, task *Task,
sched Scheduler, sched Scheduler,
maxRetries int, maxAttempts int,
attempt func() attemptResult, attempt func() attemptResult,
) { ) {
cb := c.getCircuitBreaker(task.TargetID) cb := c.getCircuitBreaker(task.TargetID)
@@ -131,6 +146,10 @@ func (c *httpCore) withRetry(
res := attempt() res := attempt()
if c.interruptedByStop(ctx, d, res) {
return
}
c.eng.observeAttempt(d.Target.Type, res.elapsed()) c.eng.observeAttempt(d.Target.Type, res.elapsed())
err := c.eng.recordResult( err := c.eng.recordResult(
@@ -162,7 +181,7 @@ func (c *httpCore) withRetry(
cb.RecordFailure() cb.RecordFailure()
c.handleRetry( c.handleRetry(
webhookDB, d, task, sched, maxRetries, attemptNum, webhookDB, d, task, sched, maxAttempts, attemptNum,
) )
} }
@@ -223,10 +242,10 @@ func (c *httpCore) handleRetry(
d *database.Delivery, d *database.Delivery,
task *Task, task *Task,
sched Scheduler, sched Scheduler,
maxRetries int, maxAttempts int,
attemptNum int, attemptNum int,
) { ) {
if attemptNum >= maxRetries { if attemptNum >= maxAttempts {
c.eng.settleStatus( c.eng.settleStatus(
webhookDB, d, d.Target.Type, webhookDB, d, d.Target.Type,
database.DeliveryStatusFailed, database.DeliveryStatusFailed,
@@ -291,7 +310,8 @@ func (c *httpCore) publishCircuitState(
// remainingBackoff returns how long remains of the backoff // remainingBackoff returns how long remains of the backoff
// window for the last attempt of a recovered retrying // window for the last attempt of a recovered retrying
// delivery. It implements rescheduler. // delivery: zero once it has passed, or when the last attempt
// cannot be read. It implements rescheduler.
func (c *httpCore) remainingBackoff( func (c *httpCore) remainingBackoff(
webhookDB *gorm.DB, webhookDB *gorm.DB,
deliveryID string, deliveryID string,
@@ -314,29 +334,6 @@ func (c *httpCore) remainingBackoff(
return max(remaining, 0) return max(remaining, 0)
} }
// backoffElapsed reports whether the backoff window for the
// last attempt of a retrying delivery has passed. It
// implements rescheduler.
func (c *httpCore) backoffElapsed(
webhookDB *gorm.DB,
deliveryID string,
attemptNum int,
) bool {
var lastResult database.DeliveryResult
err := webhookDB.
Where("delivery_id = ?", deliveryID).
Order("created_at DESC").
First(&lastResult).Error
if err != nil {
return true
}
backoff := Backoff(attemptNum)
return time.Since(lastResult.CreatedAt) >= backoff
}
// Backoff is how long an http or slack target with retries waits after // Backoff is how long an http or slack target with retries waits after
// a delivery's failed attempt attemptNum before trying it again. // a delivery's failed attempt attemptNum before trying it again.
func Backoff(attemptNum int) time.Duration { func Backoff(attemptNum int) time.Duration {
@@ -364,39 +361,13 @@ func (t *httpTarget) Deliver(
task *Task, task *Task,
sched Scheduler, sched Scheduler,
) { ) {
cfg, err := parseHTTPConfig(d.Target.Config)
if err != nil {
t.eng.log.Error(
"invalid HTTP target config",
"target_id", d.TargetID,
"error", err,
)
recErr := t.eng.recordResult(
webhookDB, d, task.AttemptNum,
false, 0, "", err.Error(), 0,
)
if recErr != nil {
t.eng.bookkeepingFailed(d, recErr)
return
}
t.eng.settleStatus(
webhookDB, d, d.Target.Type,
database.DeliveryStatusFailed,
)
return
}
attempt := func() attemptResult { attempt := func() attemptResult {
return t.attempt(ctx, cfg, &d.Event) return t.attempt(ctx, &d.Target, &d.Event)
} }
t.deliver( t.deliver(
webhookDB, d, task, sched, ctx, webhookDB, d, task, sched,
d.Target.MaxRetries, attempt, d.Target.MaxAttempts, attempt,
) )
} }
@@ -406,11 +377,11 @@ func (t *httpTarget) Deliver(
// string; only a transport-level error does. // string; only a transport-level error does.
func (t *httpTarget) attempt( func (t *httpTarget) attempt(
ctx context.Context, ctx context.Context,
cfg *HTTPTargetConfig, target *database.Target,
event *database.Event, event *database.Event,
) attemptResult { ) attemptResult {
statusCode, respBody, duration, reqErr := statusCode, respBody, duration, reqErr :=
t.doHTTPRequest(ctx, cfg, event) t.doHTTPRequest(ctx, target, event)
success := reqErr == nil && success := reqErr == nil &&
statusCode >= httpSuccessMin && statusCode >= httpSuccessMin &&
@@ -432,7 +403,7 @@ func (t *httpTarget) attempt(
func (t *httpTarget) doHTTPRequest( func (t *httpTarget) doHTTPRequest(
ctx context.Context, ctx context.Context,
cfg *HTTPTargetConfig, target *database.Target,
event *database.Event, event *database.Event,
) (int, string, int64, error) { ) (int, string, int64, error) {
start := time.Now() start := time.Now()
@@ -440,7 +411,7 @@ func (t *httpTarget) doHTTPRequest(
req, reqErr := http.NewRequestWithContext( req, reqErr := http.NewRequestWithContext(
ctx, ctx,
http.MethodPost, http.MethodPost,
cfg.URL, target.URL,
bytes.NewReader([]byte(event.Body)), bytes.NewReader([]byte(event.Body)),
) )
if reqErr != nil { if reqErr != nil {
@@ -450,15 +421,15 @@ func (t *httpTarget) doHTTPRequest(
) )
} }
if cfg.ForwardQuery { if target.ForwardQuery {
appendQuery(req.URL, event.RawQuery) appendQuery(req.URL, event.RawQuery)
} }
originScoped := applyRequestHeaders( originScoped := applyRequestHeaders(
req, event, cfg, t.eng.userAgent(), req, event, target.Headers, t.eng.userAgent(),
) )
client := t.clientForRequest(cfg, originScoped) client := t.clientForRequest(target.TimeoutSeconds, originScoped)
resp, doErr := executeHTTPRequest(client, req) resp, doErr := executeHTTPRequest(client, req)
@@ -498,17 +469,17 @@ func appendQuery(u *url.URL, rawQuery string) {
} }
// clientForRequest returns the client for one delivery attempt. // clientForRequest returns the client for one delivery attempt.
// originScoped is the header set applyRequestHeaders built for that // timeoutSeconds is the target's TimeoutSeconds, and originScoped is
// attempt; a request with neither a per-target timeout nor an // the header set applyRequestHeaders built for that attempt; a
// origin-scoped header gets the shared client, because there is // request with neither a per-target timeout nor an origin-scoped
// then nothing for the redirect policy to strip and net/http's // header gets the shared client, because there is then nothing for
// default policy already withholds Authorization and Cookie across // the redirect policy to strip and net/http's default policy already
// hosts. // withholds Authorization and Cookie across hosts.
func (t *httpTarget) clientForRequest( func (t *httpTarget) clientForRequest(
cfg *HTTPTargetConfig, timeoutSeconds int,
originScoped []string, originScoped []string,
) *http.Client { ) *http.Client {
if cfg.Timeout <= 0 && len(originScoped) == 0 { if timeoutSeconds <= 0 && len(originScoped) == 0 {
return t.client return t.client
} }
@@ -521,9 +492,9 @@ func (t *httpTarget) clientForRequest(
Transport: t.client.Transport, Transport: t.client.Transport,
} }
if cfg.Timeout > 0 { if timeoutSeconds > 0 {
client.Timeout = time.Duration( client.Timeout = time.Duration(
cfg.Timeout, timeoutSeconds,
) * time.Second ) * time.Second
} }
@@ -534,31 +505,6 @@ func (t *httpTarget) clientForRequest(
return client return client
} }
func parseHTTPConfig(
configJSON string,
) (*HTTPTargetConfig, error) {
if configJSON == "" {
return nil, errEmptyTargetConfig
}
var cfg HTTPTargetConfig
err := json.Unmarshal(
[]byte(configJSON), &cfg,
)
if err != nil {
return nil, fmt.Errorf(
"parsing config JSON: %w", err,
)
}
if cfg.URL == "" {
return nil, errMissingTargetURL
}
return &cfg, nil
}
// isForwardableHeader returns true if the header should // isForwardableHeader returns true if the header should
// be forwarded to targets. // be forwarded to targets.
func isForwardableHeader(name string) bool { func isForwardableHeader(name string) bool {
@@ -590,11 +536,12 @@ func isForwardableHeader(name string) bool {
// wins, otherwise the event's ContentType, otherwise none. The inbound // wins, otherwise the event's ContentType, otherwise none. The inbound
// Content-Type in the event's headers is never forwarded. // Content-Type in the event's headers is never forwarded.
// //
// userAgent is set last, over any configured or inbound User-Agent. // configured is the target's Headers. userAgent is set last, over
// any configured or inbound User-Agent.
func applyRequestHeaders( func applyRequestHeaders(
req *http.Request, req *http.Request,
event *database.Event, event *database.Event,
cfg *HTTPTargetConfig, configured map[string]string,
userAgent string, userAgent string,
) []string { ) []string {
if event.ContentType != "" { if event.ContentType != "" {
@@ -605,7 +552,7 @@ func applyRequestHeaders(
originScoped := forwardEventHeaders(req, event) originScoped := forwardEventHeaders(req, event)
for k, v := range cfg.Headers { for k, v := range configured {
req.Header.Set(k, v) req.Header.Set(k, v)
originScoped[http.CanonicalHeaderKey(k)] = struct{}{} originScoped[http.CanonicalHeaderKey(k)] = struct{}{}
} }
@@ -670,7 +617,7 @@ func forwardEventHeaders(
} }
// executeHTTPRequest sends an HTTP request using the provided // executeHTTPRequest sends an HTTP request using the provided
// client. URLs are validated by the config parsers and the // client. URLs are validated by the target forms and the
// SSRF-safe transport before reaching here. // SSRF-safe transport before reaching here.
// //
// Transport failures are masked here, at the single point // Transport failures are masked here, at the single point
+18 -27
View File
@@ -2,7 +2,6 @@ package delivery_test
import ( import (
"context" "context"
"encoding/json"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"testing" "testing"
@@ -18,25 +17,13 @@ import (
// with. // with.
const eventQuery = "a=1&b=2" const eventQuery = "a=1&b=2"
// httpTargetConfig is the stored configuration of an HTTP target at
// targetURL.
func httpTargetConfig(
t *testing.T, targetURL string, forwardQuery bool,
) string {
t.Helper()
cfg, err := json.Marshal(delivery.HTTPTargetConfig{
URL: targetURL, ForwardQuery: forwardQuery,
})
require.NoError(t, err)
return string(cfg)
}
// deliverWithQuery sends one event that arrived with eventQuery to an // deliverWithQuery sends one event that arrived with eventQuery to an
// HTTP target configured with cfg, through the path a received event's // HTTP target at targetURL, with its ForwardQuery setting as given,
// delivery takes, and returns the attempt it recorded. // through the path a received event's delivery takes, and returns the
func deliverWithQuery(t *testing.T, cfg string) database.DeliveryResult { // attempt it recorded.
func deliverWithQuery(
t *testing.T, targetURL string, forwardQuery bool,
) database.DeliveryResult {
t.Helper() t.Helper()
s := newISetup(t) s := newISetup(t)
@@ -47,8 +34,10 @@ func deliverWithQuery(t *testing.T, cfg string) database.DeliveryResult {
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
task := iTask( task := iTask(
d, event, s.WebhookID, d.TargetID, "query", cfg, 0, 1, &event.Body, d, event, s.WebhookID, d.TargetID, "query", targetURL, 1, 1,
&event.Body,
) )
task.ForwardQuery = forwardQuery
task.RawQuery = eventQuery task.RawQuery = eventQuery
s.Engine.ExportProcessNewTask(context.TODO(), &task) s.Engine.ExportProcessNewTask(context.TODO(), &task)
@@ -111,9 +100,9 @@ func TestDeliverHTTP_ForwardQuery(t *testing.T) {
)) ))
t.Cleanup(ts.Close) t.Cleanup(ts.Close)
result := deliverWithQuery(t, httpTargetConfig( result := deliverWithQuery(
t, ts.URL+tc.path, tc.forwardQuery, t, ts.URL+tc.path, tc.forwardQuery,
)) )
assert.True(t, result.Success) assert.True(t, result.Success)
require.Len(t, received, 1) require.Len(t, received, 1)
@@ -144,11 +133,13 @@ func TestDeliverHTTP_ForwardedQueryKeepsTheTargetURLMasked(t *testing.T) {
t.Cleanup(ts.Close) t.Cleanup(ts.Close)
target := &database.Target{ target := &database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Config: httpTargetConfig(t, ts.URL+"/in?token="+secret, true), URL: ts.URL + "/in?token=" + secret,
ForwardQuery: true,
MaxAttempts: 1,
} }
deliverWithQuery(t, target.Config) deliverWithQuery(t, target.URL, target.ForwardQuery)
require.Len(t, received, 1) require.Len(t, received, 1)
sent := <-received sent := <-received
@@ -163,9 +154,9 @@ func TestDeliverHTTP_ForwardedQueryKeepsTheTargetURLMasked(t *testing.T) {
} }
// Nothing listens on port 1. // Nothing listens on port 1.
failed := deliverWithQuery(t, httpTargetConfig( failed := deliverWithQuery(
t, "http://127.0.0.1:1/in?token="+secret, true, t, "http://127.0.0.1:1/in?token="+secret, true,
)) )
require.NotEmpty(t, failed.Error) require.NotEmpty(t, failed.Error)
assert.NotContains(t, failed.Error, secret) assert.NotContains(t, failed.Error, secret)
assert.NotContains(t, failed.Error, eventQuery) assert.NotContains(t, failed.Error, eventQuery)
+9 -30
View File
@@ -13,12 +13,12 @@ import (
const RedactionMarker = "(redacted)" const RedactionMarker = "(redacted)"
// Redactor removes one target's own credential material from // Redactor removes one target's own credential material from
// text that target's remote peer chose: a delivery response // a delivery response body, whose text the target's remote
// body, or a delivery error stored before the delivery path // peer chose, and from a stored delivery error as a second
// learned to mask the URLs it embeds. // line behind the delivery path's own URL masking.
// //
// It removes byte-identical echoes of strings taken from the // It removes byte-identical echoes of strings taken from the
// target's stored configuration, and nothing else. Anything // target's URL and headers, and nothing else. Anything
// the remote re-encodes survives: JSON "\/" escaping (what // the remote re-encodes survives: JSON "\/" escaping (what
// PHP's json_encode emits by default), percent-encoding, HTML // PHP's json_encode emits by default), percent-encoding, HTML
// entities, and an echo of only part of a path. It cannot // entities, and an echo of only part of a path. It cannot
@@ -115,7 +115,7 @@ func (r Redactor) secretPrefixSuffix(s string) int {
} }
// targetSecrets returns the credential-bearing strings a // targetSecrets returns the credential-bearing strings a
// target's configuration carries. // target's settings carry.
// //
// The destination URL contributes. Its path, query and // The destination URL contributes. Its path, query and
// userinfo are the credential for both target types that have // userinfo are the credential for both target types that have
@@ -129,36 +129,15 @@ func (r Redactor) secretPrefixSuffix(s string) int {
// picks out. That is the same class-based rule applied to // picks out. That is the same class-based rule applied to
// URLs: an echoed Accept or User-Agent still renders, an // URLs: an echoed Accept or User-Agent still renders, an
// echoed Authorization does not. // echoed Authorization does not.
//
// A setting a type does not use is empty and contributes
// nothing, so a database or log target has nothing to redact.
func targetSecrets(t *database.Target) []string { func targetSecrets(t *database.Target) []string {
if t == nil { if t == nil {
return nil return nil
} }
switch t.Type { return append(urlSecrets(t.URL), headerSecrets(t.Headers)...)
case database.TargetTypeSlack:
cfg, err := parseSlackConfig(t.Config)
if err != nil {
return nil
}
return urlSecrets(cfg.WebhookURL)
case database.TargetTypeHTTP:
cfg, err := parseHTTPConfig(t.Config)
if err != nil {
return nil
}
return append(
urlSecrets(cfg.URL),
headerSecrets(cfg.Headers)...,
)
case database.TargetTypeDatabase, database.TargetTypeLog:
// Neither has a destination URL, so neither has
// anything to redact.
return nil
default:
return nil
}
} }
// urlSecrets returns the substrings of a destination URL that // urlSecrets returns the substrings of a destination URL that
+45 -36
View File
@@ -22,8 +22,9 @@ func TestRedactor_RemovesSlackWebhookURL(t *testing.T) {
t.Parallel() t.Parallel()
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
Config: `{"webhookUrl":"` + redactWebhookURL + `"}`, URL: redactWebhookURL,
MaxAttempts: 1,
}) })
got := r.Redact("no_service for " + redactWebhookURL) got := r.Redact("no_service for " + redactWebhookURL)
@@ -52,8 +53,9 @@ func TestRedactor_RemovesSecretSeveredByACut(t *testing.T) {
t.Parallel() t.Parallel()
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
Config: `{"webhookUrl":"` + redactWebhookURL + `"}`, URL: redactWebhookURL,
MaxAttempts: 1,
}) })
// Every cut position inside the credential, not just a // Every cut position inside the credential, not just a
@@ -84,18 +86,20 @@ func TestRedactor_RedactsCredentialShapedHeaderValues(
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Config: `{"url":"https://example.com/in",` + URL: "https://example.com/in",
`"headers":{` + Headers: map[string]string{
`"Authorization":"Bearer AAAAAAAAAAAA",` + headerAuthorization: "Bearer AAAAAAAAAAAA",
`"Cookie":"session=BBBBBBBBBBBB",` + "Cookie": "session=BBBBBBBBBBBB",
`"X-Api-Key":"CCCCCCCCCCCC",` + probeHeaderName: "CCCCCCCCCCCC",
`"X-Hub-Signature":"sha256=DDDDDDDDDDDD",` + "X-Hub-Signature": "sha256=DDDDDDDDDDDD",
`"X-Sig":"EEEEEEEEEEEE",` + "X-Sig": "EEEEEEEEEEEE",
`"X-Pass":"FFFFFFFFFFFF",` + "X-Pass": "FFFFFFFFFFFF",
`"X-HMAC":"GGGGGGGGGGGG",` + "X-HMAC": "GGGGGGGGGGGG",
`"X-Credential":"HHHHHHHHHHHH",` + "X-Credential": "HHHHHHHHHHHH",
`"Accept":"application/json",` + "Accept": "application/json",
`"User-Agent":"webhooker/1.0"}}`, headerUserAgent: "webhooker/1.0",
},
MaxAttempts: 1,
}) })
for _, secret := range []string{ for _, secret := range []string{
@@ -133,9 +137,10 @@ func TestRedactor_IgnoresVeryShortHeaderValues(t *testing.T) {
t.Parallel() t.Parallel()
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Config: `{"url":"https://example.com/in",` + URL: "https://example.com/in",
`"headers":{"X-Api-Key":"ab"}}`, Headers: map[string]string{probeHeaderName: "ab"},
MaxAttempts: 1,
}) })
const response = "rabbit" const response = "rabbit"
@@ -150,8 +155,9 @@ func TestRedactor_RemovesBarePath(t *testing.T) {
t.Parallel() t.Parallel()
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
Config: `{"webhookUrl":"` + redactWebhookURL + `"}`, URL: redactWebhookURL,
MaxAttempts: 1,
}) })
got := r.Redact("POST " + redactSecretPath + " 404") got := r.Redact("POST " + redactSecretPath + " 404")
@@ -183,8 +189,9 @@ func TestRedactor_RemovesHTTPURLQueryAndUserinfo(t *testing.T) {
raw := dest.String() raw := dest.String()
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Config: `{"url":"` + raw + `"}`, URL: raw,
MaxAttempts: 1,
}) })
for _, echoed := range []string{ for _, echoed := range []string{
@@ -232,8 +239,9 @@ func TestRedactor_RemovesEchoedQueryOfURLWithoutPath(t *testing.T) {
"https://example.com?token=" + secret, "https://example.com?token=" + secret,
} { } {
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Config: `{"url":"` + dest + `"}`, URL: dest,
MaxAttempts: 1,
}) })
for echoed, want := range echoes { for echoed, want := range echoes {
@@ -253,8 +261,9 @@ func TestRedactor_LeavesUnrelatedTextAlone(t *testing.T) {
const response = "ok=false error=channel_not_found" const response = "ok=false error=channel_not_found"
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
Config: `{"webhookUrl":"` + redactWebhookURL + `"}`, URL: redactWebhookURL,
MaxAttempts: 1,
}) })
assert.Equal(t, response, r.Redact(response)) assert.Equal(t, response, r.Redact(response))
@@ -279,8 +288,9 @@ func TestRedactor_EmptyUserinfoDoesNotShredTheBody(t *testing.T) {
require.Empty(t, parsed.User.String()) require.Empty(t, parsed.User.String())
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Config: `{"url":"` + dest + `"}`, URL: dest,
MaxAttempts: 1,
}) })
const body = "ok=false error=channel_not_found" const body = "ok=false error=channel_not_found"
@@ -297,11 +307,10 @@ func TestRedactor_EmptyUserinfoDoesNotShredTheBody(t *testing.T) {
) )
} }
// TestRedactor_ZeroValueAndConfiglessTargets pins that a // TestRedactor_ZeroValueAndURLlessTargets pins that a caller
// caller with no target, an unparseable config, or a target // with no target, or a target with no destination URL, gets a
// type with no destination URL gets a redactor that changes // redactor that changes nothing rather than one that panics.
// nothing rather than one that panics. func TestRedactor_ZeroValueAndURLlessTargets(t *testing.T) {
func TestRedactor_ZeroValueAndConfiglessTargets(t *testing.T) {
t.Parallel() t.Parallel()
const text = "some response body" const text = "some response body"
@@ -314,8 +323,8 @@ func TestRedactor_ZeroValueAndConfiglessTargets(t *testing.T) {
for _, tgt := range []database.Target{ for _, tgt := range []database.Target{
{Type: database.TargetTypeLog}, {Type: database.TargetTypeLog},
{Type: database.TargetTypeDatabase}, {Type: database.TargetTypeDatabase},
{Type: database.TargetTypeSlack, Config: "not json"}, {Type: database.TargetTypeSlack, MaxAttempts: 1},
{Type: database.TargetTypeHTTP, Config: ""}, {Type: database.TargetTypeHTTP, MaxAttempts: 1},
} { } {
assert.Equal( assert.Equal(
t, text, t, text,
+23 -86
View File
@@ -4,7 +4,6 @@ import (
"bytes" "bytes"
"context" "context"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
@@ -15,24 +14,11 @@ import (
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
) )
// errMissingWebhookURL is returned when a Slack target config
// omits its webhook URL.
var errMissingWebhookURL = errors.New(
"webhook_url is required",
)
// SlackTargetConfig holds configuration for slack target
// types.
type SlackTargetConfig struct {
WebhookURL string `json:"webhookUrl"`
}
// slackTarget delivers events to Slack incoming webhooks. It // slackTarget delivers events to Slack incoming webhooks. It
// formats the event into a Slack message and posts it as // formats the event into a Slack message and posts it as
// JSON. It shares the retry core with the HTTP target: a // JSON. It shares the retry core with the HTTP target: a
// MaxRetries of 0 stays single-attempt fire-and-forget // MaxAttempts of 1 is a single attempt, while a higher one
// (preserving existing Slack targets), while a positive // adds backoff and circuit breaking.
// MaxRetries adds backoff and circuit breaking.
type slackTarget struct { type slackTarget struct {
*httpCore *httpCore
@@ -47,19 +33,6 @@ func (t *slackTarget) Deliver(
task *Task, task *Task,
sched Scheduler, sched Scheduler,
) { ) {
cfg, err := parseSlackConfig(d.Target.Config)
if err != nil {
t.eng.log.Error(
"invalid Slack target config",
"target_id", d.TargetID,
"error", err,
)
t.failConfig(webhookDB, d, err)
return
}
msg := FormatSlackMessage(&d.Event) msg := FormatSlackMessage(&d.Event)
payload, err := json.Marshal( payload, err := json.Marshal(
@@ -72,52 +45,41 @@ func (t *slackTarget) Deliver(
"error", err, "error", err,
) )
t.failConfig(webhookDB, d, err) recErr := t.eng.recordResult(
webhookDB, d, 1,
false, 0, "", err.Error(), 0,
)
if recErr != nil {
t.eng.bookkeepingFailed(d, recErr)
return
}
t.eng.settleStatus(
webhookDB, d, d.Target.Type,
database.DeliveryStatusFailed,
)
return return
} }
attempt := func() attemptResult { attempt := func() attemptResult {
return t.attempt(ctx, cfg, payload) return t.attempt(ctx, d.Target.URL, payload)
} }
t.deliver( t.deliver(
webhookDB, d, task, sched, ctx, webhookDB, d, task, sched,
d.Target.MaxRetries, attempt, d.Target.MaxAttempts, attempt,
) )
} }
// failConfig records a first-attempt failure for a delivery // attempt performs a single Slack POST to targetURL and derives
// that could not be prepared (bad config or unmarshalable // its outcome, preserving the engine's original semantics: a
// payload) and marks it failed.
func (t *slackTarget) failConfig(
webhookDB *gorm.DB,
d *database.Delivery,
err error,
) {
recErr := t.eng.recordResult(
webhookDB, d, 1,
false, 0, "", err.Error(), 0,
)
if recErr != nil {
t.eng.bookkeepingFailed(d, recErr)
return
}
t.eng.settleStatus(
webhookDB, d, d.Target.Type,
database.DeliveryStatusFailed,
)
}
// attempt performs a single Slack POST and derives its
// outcome, preserving the engine's original semantics: a
// non-2xx response records an "HTTP <code>" error string and // non-2xx response records an "HTTP <code>" error string and
// a transport error records a "sending request" error. // a transport error records a "sending request" error.
func (t *slackTarget) attempt( func (t *slackTarget) attempt(
ctx context.Context, ctx context.Context,
cfg *SlackTargetConfig, targetURL string,
payload []byte, payload []byte,
) attemptResult { ) attemptResult {
start := time.Now() start := time.Now()
@@ -125,7 +87,7 @@ func (t *slackTarget) attempt(
req, err := http.NewRequestWithContext( req, err := http.NewRequestWithContext(
ctx, ctx,
http.MethodPost, http.MethodPost,
cfg.WebhookURL, targetURL,
bytes.NewReader(payload), bytes.NewReader(payload),
) )
if err != nil { if err != nil {
@@ -187,31 +149,6 @@ func (t *slackTarget) readSlackResponse(
} }
} }
func parseSlackConfig(
configJSON string,
) (*SlackTargetConfig, error) {
if configJSON == "" {
return nil, errEmptyTargetConfig
}
var cfg SlackTargetConfig
err := json.Unmarshal(
[]byte(configJSON), &cfg,
)
if err != nil {
return nil, fmt.Errorf(
"parsing config JSON: %w", err,
)
}
if cfg.WebhookURL == "" {
return nil, errMissingWebhookURL
}
return &cfg, nil
}
// FormatSlackMessage builds a Slack-compatible message // FormatSlackMessage builds a Slack-compatible message
// string from a webhook event. // string from a webhook event.
func FormatSlackMessage( func FormatSlackMessage(
+48 -48
View File
@@ -45,7 +45,7 @@ func tSeedDeletedTarget(
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, name, t, s.MainDB, targetID, s.WebhookID, name,
database.TargetTypeHTTP, iHTTPConfig(url), 5, database.TargetTypeHTTP, url, 5,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -122,9 +122,9 @@ func TestProcessDelivery_UnknownTargetType_RecordsWhy(
) )
target := database.Target{ target := database.Target{
Name: "mystery", Name: "mystery",
Type: tUnknownType, Type: tUnknownType,
Config: iHTTPConfig("http://example.com/hook"), URL: "http://example.com/hook",
} }
target.ID = targetID target.ID = targetID
@@ -140,7 +140,7 @@ func TestProcessDelivery_UnknownTargetType_RecordsWhy(
body := event.Body body := event.Body
task := iTask( task := iTask(
seeded, event, s.WebhookID, targetID, "mystery", seeded, event, s.WebhookID, targetID, "mystery",
target.Config, 0, 1, &body, target.URL, 0, 1, &body,
) )
task.TargetType = tUnknownType task.TargetType = tUnknownType
@@ -178,7 +178,7 @@ func TestRecoverSingleRetry_TargetDeleted(t *testing.T) {
) )
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, atStart,
) )
iAssertStatus( iAssertStatus(
@@ -201,7 +201,7 @@ func TestRecoverSingleRetry_TargetDeleted(t *testing.T) {
) )
} }
func TestSweepSingleRetry_TargetDeleted(t *testing.T) { func TestRecoverSingleRetry_OnTick_TargetDeleted(t *testing.T) {
t.Parallel() t.Parallel()
s := newISetup(t) s := newISetup(t)
@@ -218,11 +218,11 @@ func TestSweepSingleRetry_TargetDeleted(t *testing.T) {
// Twice, because the bug was an error the sweep repeated every // Twice, because the bug was an error the sweep repeated every
// minute for the life of the database: the second sweep must // minute for the life of the database: the second sweep must
// find nothing left to do. // find nothing left to do.
s.Engine.ExportSweepWebhookRetries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, onTick,
) )
s.Engine.ExportSweepWebhookRetries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, onTick,
) )
iAssertStatus( iAssertStatus(
@@ -239,10 +239,11 @@ func TestSweepSingleRetry_TargetDeleted(t *testing.T) {
assert.Zero(t, s.Engine.ExportInflightHeld()) assert.Zero(t, s.Engine.ExportInflightHeld())
} }
// TestSweepSingleRetry_TargetNeverExisted covers the other half of the // TestRecoverSingleRetry_OnTick_TargetNeverExisted covers the other
// soft-delete distinction: an id with no row at all, deleted or // half of the soft-delete distinction: an id with no row at all,
// otherwise, must not be reported as something the operator deleted. // deleted or otherwise, must not be reported as something the operator
func TestSweepSingleRetry_TargetNeverExisted(t *testing.T) { // deleted.
func TestRecoverSingleRetry_OnTick_TargetNeverExisted(t *testing.T) {
t.Parallel() t.Parallel()
s := newISetup(t) s := newISetup(t)
@@ -264,8 +265,8 @@ func TestSweepSingleRetry_TargetNeverExisted(t *testing.T) {
iSeedFailedResult(t, s.WebhookDB, d.ID) iSeedFailedResult(t, s.WebhookDB, d.ID)
s.Engine.ExportSweepWebhookRetries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, onTick,
) )
iAssertStatus( iAssertStatus(
@@ -303,8 +304,8 @@ func TestFailMissingTarget_WritesNoTargetRow(
database.DeliveryStatusRetrying, database.DeliveryStatusRetrying,
) )
s.Engine.ExportSweepWebhookRetries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, onTick,
) )
iAssertStatus( iAssertStatus(
@@ -345,11 +346,10 @@ func tRetryChainSetup(
iCreateWebhook(t, s.MainDB, s.WebhookID, name) iCreateWebhook(t, s.MainDB, s.WebhookID, name)
targetID := uuid.New().String() targetID := uuid.New().String()
cfg := iHTTPConfig(ts.URL)
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, name, t, s.MainDB, targetID, s.WebhookID, name,
database.TargetTypeHTTP, cfg, 5, database.TargetTypeHTTP, ts.URL, 5,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -366,7 +366,7 @@ func tRetryChainSetup(
body := event.Body body := event.Body
return iTask( return iTask(
d, event, s.WebhookID, targetID, name, cfg, 5, 2, &body, d, event, s.WebhookID, targetID, name, ts.URL, 5, 2, &body,
), targetID ), targetID
} }
@@ -527,7 +527,7 @@ func TestRecoverSingleRetry_TargetUnreadable_LeavesDeliveryAlone(
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "healthy", t, s.MainDB, targetID, s.WebhookID, "healthy",
database.TargetTypeHTTP, database.TargetTypeHTTP,
iHTTPConfig("http://example.com/hook"), 5, "http://example.com/hook", 5,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -546,7 +546,7 @@ func TestRecoverSingleRetry_TargetUnreadable_LeavesDeliveryAlone(
require.NoError(t, sqlDB.Close()) require.NoError(t, sqlDB.Close())
s.Engine.ExportRecoverRetryingDeliveries( s.Engine.ExportRecoverRetryingDeliveries(
s.WebhookDB, s.WebhookID, context.Background(), s.WebhookDB, s.WebhookID,
) )
iAssertStatus( iAssertStatus(
@@ -564,7 +564,7 @@ func TestRecoverSingleRetry_TargetUnreadable_LeavesDeliveryAlone(
// --- 4. A pending delivery whose target is gone --- // --- 4. A pending delivery whose target is gone ---
func TestRecoverPending_TargetDeleted(t *testing.T) { func TestRecoverPendingDeliveries_TargetDeleted(t *testing.T) {
t.Parallel() t.Parallel()
s := newISetup(t) s := newISetup(t)
@@ -577,7 +577,7 @@ func TestRecoverPending_TargetDeleted(t *testing.T) {
) )
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, atStart,
) )
iAssertStatus( iAssertStatus(
@@ -600,10 +600,10 @@ func TestRecoverPending_TargetDeleted(t *testing.T) {
) )
} }
// TestRecoverPending_TargetDeleted_LeavesAnOwnedDeliveryAlone: the // TestRecoverPendingDeliveries_TargetDeleted_LeavesAnOwnedDeliveryAlone:
// terminal write takes ownership like every other recovery write, so a // the terminal write takes ownership like every other recovery write, so
// delivery the engine still holds is not failed underneath its worker. // a delivery the engine still holds is not failed underneath its worker.
func TestRecoverPending_TargetDeleted_LeavesAnOwnedDeliveryAlone( func TestRecoverPendingDeliveries_TargetDeleted_LeavesAnOwnedDeliveryAlone(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
@@ -620,7 +620,7 @@ func TestRecoverPending_TargetDeleted_LeavesAnOwnedDeliveryAlone(
require.True(t, s.Engine.ExportRetainDelivery(deliveryID)) require.True(t, s.Engine.ExportRetainDelivery(deliveryID))
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, context.Background(), s.WebhookID, atStart,
) )
iAssertStatus( iAssertStatus(
@@ -675,18 +675,18 @@ func TestFailMissingTarget_LeavesASettledDeliveryAlone(
assert.Zero(t, s.Engine.ExportInflightHeld()) assert.Zero(t, s.Engine.ExportInflightHeld())
} }
// TestSweepPending_TargetDeleted sweeps twice over a batch that also // TestRecoverPendingDeliveries_OnTick_TargetDeleted sweeps twice over a
// holds a healthy stranded delivery. The one whose target is gone is // batch that also holds a healthy stranded delivery. The one whose
// failed once and then left alone; the healthy one is queued by the // target is gone is failed once and then left alone; the healthy one is
// first sweep and not again by the second. // queued by the first sweep and not again by the second.
func TestSweepPending_TargetDeleted(t *testing.T) { func TestRecoverPendingDeliveries_OnTick_TargetDeleted(t *testing.T) {
t.Parallel() t.Parallel()
liveTargetID := uuid.New().String() liveTargetID := uuid.New().String()
s := fSweepSetup(t, liveTargetID, "still-there") s := fSweepSetup(t, liveTargetID, "still-there")
deliveryID := tSeedDeletedTarget( deliveryID := tSeedDeletedTarget(
t, s, "gone-on-pending-sweep", "http://example.com/hook", t, s, "gone-while-pending-on-tick", "http://example.com/hook",
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
rAgePending(t, s.WebhookDB, deliveryID) rAgePending(t, s.WebhookDB, deliveryID)
@@ -703,7 +703,7 @@ func TestSweepPending_TargetDeleted(t *testing.T) {
ctx := context.Background() ctx := context.Background()
s.Engine.ExportSweepWebhookRetries(ctx, s.WebhookID) s.Engine.ExportRecoverWebhookDeliveries(ctx, s.WebhookID, onTick)
tasks := fDrain(s.Engine) tasks := fDrain(s.Engine)
require.Len(t, tasks, 1, require.Len(t, tasks, 1,
@@ -711,7 +711,7 @@ func TestSweepPending_TargetDeleted(t *testing.T) {
) )
assert.Equal(t, healthy.ID, tasks[0].DeliveryID) assert.Equal(t, healthy.ID, tasks[0].DeliveryID)
s.Engine.ExportSweepWebhookRetries(ctx, s.WebhookID) s.Engine.ExportRecoverWebhookDeliveries(ctx, s.WebhookID, onTick)
assert.Empty(t, fDrain(s.Engine), assert.Empty(t, fDrain(s.Engine),
"the second sweep queued a delivery again", "the second sweep queued a delivery again",
@@ -724,7 +724,7 @@ func TestSweepPending_TargetDeleted(t *testing.T) {
last := tLastResult(t, s, deliveryID, 2) last := tLastResult(t, s, deliveryID, 2)
assert.Contains(t, last.Error, "gone-on-pending-sweep") assert.Contains(t, last.Error, "gone-while-pending-on-tick")
assert.Contains(t, last.Error, "was deleted") assert.Contains(t, last.Error, "was deleted")
} }
@@ -743,7 +743,7 @@ func TestSendRecoveredDeliveries_TargetMissingFromMap(
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "found-on-lookup", t, s.MainDB, targetID, s.WebhookID, "found-on-lookup",
database.TargetTypeLog, "", 0, database.TargetTypeLog, "", 1,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -775,11 +775,11 @@ func TestSendRecoveredDeliveries_TargetMissingFromMap(
) )
} }
// TestRecoverPending_TargetUnreadable_LeavesDeliveryAlone: a failed // TestRecoverPendingDeliveries_TargetUnreadable_LeavesDeliveryAlone: a
// read of the main database is not a deleted target. Restart recovery // failed read of the main database is not a deleted target. At start the
// holds every pending delivery of the webhook in one batch, so failing // recovery routine holds every pending delivery of the webhook in one
// on this would fail all of them. // batch, so failing on this would fail all of them.
func TestRecoverPending_TargetUnreadable_LeavesDeliveryAlone( func TestRecoverPendingDeliveries_TargetUnreadable_LeavesDeliveryAlone(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
@@ -790,7 +790,7 @@ func TestRecoverPending_TargetUnreadable_LeavesDeliveryAlone(
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "healthy", t, s.MainDB, targetID, s.WebhookID, "healthy",
database.TargetTypeLog, "", 0, database.TargetTypeLog, "", 1,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -807,7 +807,7 @@ func TestRecoverPending_TargetUnreadable_LeavesDeliveryAlone(
require.NoError(t, sqlDB.Close()) require.NoError(t, sqlDB.Close())
s.Engine.ExportRecoverPendingDeliveries( s.Engine.ExportRecoverPendingDeliveries(
context.Background(), s.WebhookDB, s.WebhookID, context.Background(), s.WebhookDB, s.WebhookID, atStart,
) )
iAssertStatus( iAssertStatus(
+4 -16
View File
@@ -2,7 +2,6 @@ package delivery_test
import ( import (
"context" "context"
"encoding/json"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"testing" "testing"
@@ -71,13 +70,6 @@ func deliverSlackTo(
e := testEngine(t, 1) e := testEngine(t, 1)
targetID := uuid.New().String() targetID := uuid.New().String()
slackCfg, err := json.Marshal(
delivery.SlackTargetConfig{
WebhookURL: webhookURL,
},
)
require.NoError(t, err)
event := seedEvent(t, db, `{"test":true}`) event := seedEvent(t, db, `{"test":true}`)
dlv := seedDelivery( dlv := seedDelivery(
@@ -87,7 +79,7 @@ func deliverSlackTo(
d := buildSlackDelivery( d := buildSlackDelivery(
dlv, event, targetID, dlv, event, targetID,
"test-slack-mask", string(slackCfg), "test-slack-mask", webhookURL,
) )
e.ExportDeliverSlack(context.TODO(), db, d) e.ExportDeliverSlack(context.TODO(), db, d)
@@ -160,13 +152,9 @@ func TestDoHTTPRequest_TransportErrorMasksURL(t *testing.T) {
e := testEngine(t, 1) e := testEngine(t, 1)
cfg, err := e.ExportParseHTTPConfig(
newHTTPTargetConfig(host + maskSecretPath),
)
require.NoError(t, err)
statusCode, _, _, reqErr := e.ExportDoHTTPRequest( statusCode, _, _, reqErr := e.ExportDoHTTPRequest(
context.TODO(), cfg, context.TODO(),
&database.Target{URL: host + maskSecretPath},
&database.Event{Body: `{"test":true}`}, &database.Event{Body: `{"test":true}`},
) )
require.Error(t, reqErr) require.Error(t, reqErr)
@@ -188,7 +176,7 @@ func TestDoHTTPRequest_UnparsableURLIsMasked(t *testing.T) {
statusCode, _, _, reqErr := e.ExportDoHTTPRequest( statusCode, _, _, reqErr := e.ExportDoHTTPRequest(
context.TODO(), context.TODO(),
&delivery.HTTPTargetConfig{ &database.Target{
URL: "https://hooks.example.com" + maskSecretPath + "\n", URL: "https://hooks.example.com" + maskSecretPath + "\n",
}, },
&database.Event{}, &database.Event{},
+3 -9
View File
@@ -2,7 +2,6 @@ package delivery_test
import ( import (
"context" "context"
"encoding/json"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"net/netip" "net/netip"
@@ -59,9 +58,9 @@ func TestUserAgent_IsTheBuildVersion(t *testing.T) {
statusCode, _, _, err := e.ExportDoHTTPRequest( statusCode, _, _, err := e.ExportDoHTTPRequest(
context.Background(), context.Background(),
&delivery.HTTPTargetConfig{ &database.Target{
URL: ts.URL, URL: ts.URL,
Headers: map[string]string{"User-Agent": "configured/1"}, Headers: map[string]string{headerUserAgent: "configured/1"},
}, },
&database.Event{Headers: `{"User-Agent":["curl/8"]}`}, &database.Event{Headers: `{"User-Agent":["curl/8"]}`},
) )
@@ -73,18 +72,13 @@ func TestUserAgent_IsTheBuildVersion(t *testing.T) {
db := testWebhookDB(t) db := testWebhookDB(t)
targetID := uuid.New().String() targetID := uuid.New().String()
slackCfg, err := json.Marshal(
delivery.SlackTargetConfig{WebhookURL: ts.URL},
)
require.NoError(t, err)
event := seedEvent(t, db, `{"action":"test"}`) event := seedEvent(t, db, `{"action":"test"}`)
dlv := seedDelivery( dlv := seedDelivery(
t, db, event.ID, targetID, database.DeliveryStatusPending, t, db, event.ID, targetID, database.DeliveryStatusPending,
) )
e.ExportDeliverSlack(context.Background(), db, buildSlackDelivery( e.ExportDeliverSlack(context.Background(), db, buildSlackDelivery(
dlv, event, targetID, "test-slack", string(slackCfg), dlv, event, targetID, "test-slack", ts.URL,
)) ))
require.Len(t, userAgents, 1, "the slack target sent no request") require.Len(t, userAgents, 1, "the slack target sent no request")
assert.Equal(t, want, <-userAgents, "slack target") assert.Equal(t, want, <-userAgents, "slack target")
+1 -1
View File
@@ -146,7 +146,7 @@ func openDB(
} }
// assertBounded holds every line the adapter wrote to the stated // assertBounded holds every line the adapter wrote to the stated
// ceiling and proves each was cut rather than merely short. // ceiling and proves no line carried a whole client value.
func assertBounded(t *testing.T, out string) { func assertBounded(t *testing.T, out string) {
t.Helper() t.Helper()
+23 -1
View File
@@ -1,8 +1,14 @@
package handlers package handlers
import (
"strconv"
"time"
)
const ( const (
// archiveExpiryNever is the archive expiry that keeps archived // archiveExpiryNever is the archive expiry that keeps archived
// events forever. A stored empty expiry means the same. // events forever. It is stored as an ArchiveExpiry of 0, and an
// empty expiry on a form means the same.
archiveExpiryNever = "never" archiveExpiryNever = "never"
// tmplKeyArchiveExpiryChoices is the template data key for the // tmplKeyArchiveExpiryChoices is the template data key for the
@@ -56,3 +62,19 @@ func archiveExpiryOptions(expiry string) []archiveChoice {
return append([]archiveChoice{own}, options...) return append([]archiveChoice{own}, options...)
} }
// formatArchiveExpiry writes a stored ArchiveExpiry the way the
// expiry select spells its values, so the target edit form starts on
// the stored choice: never for 0, a whole number of hours as "720h",
// and anything else as Go writes a duration, which the form accepts
// back unchanged.
func formatArchiveExpiry(expiry time.Duration) string {
switch {
case expiry == 0:
return archiveExpiryNever
case expiry%time.Hour == 0:
return strconv.FormatInt(int64(expiry/time.Hour), 10) + "h"
default:
return expiry.String()
}
}
+20 -16
View File
@@ -7,6 +7,7 @@ import (
"regexp" "regexp"
"strings" "strings"
"testing" "testing"
"time"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
@@ -94,14 +95,18 @@ func TestArchiveExpiryChoices(t *testing.T) {
env := setupSourceTest(t) env := setupSourceTest(t)
choices := []struct{ value, shown string }{ choices := []struct {
{expiryNever, expiryNever}, value string
{"1h", "1 hour"}, stored time.Duration
{"12h", "12 hours"}, shown string
{"24h", "1 day"}, }{
{"720h", "30 days"}, {expiryNever, 0, expiryNever},
{"2160h", "90 days"}, {"1h", time.Hour, "1 hour"},
{"8760h", "365 days"}, {"12h", 12 * time.Hour, "12 hours"},
{"24h", 24 * time.Hour, "1 day"},
{"720h", 720 * time.Hour, "30 days"},
{"2160h", 2160 * time.Hour, "90 days"},
{"8760h", 8760 * time.Hour, "365 days"},
} }
for _, choice := range choices { for _, choice := range choices {
@@ -122,9 +127,7 @@ func TestArchiveExpiryChoices(t *testing.T) {
targets := targetsForWebhook(t, env.db, webhook.ID) targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1) require.Len(t, targets, 1)
assert.JSONEq( assert.Equal(t, choice.stored, targets[0].ArchiveExpiry)
t, `{"expiry":"`+choice.value+`"}`, targets[0].Config,
)
assert.Equal( assert.Equal(
t, []string{choice.shown}, t, []string{choice.shown},
@@ -138,7 +141,7 @@ func TestArchiveExpiryChoices(t *testing.T) {
} }
// TestArchiveExpiryEditStartsOnStoredValue checks the edit form of a // TestArchiveExpiryEditStartsOnStoredValue checks the edit form of a
// database target whose stored expiry is empty, which selects never, // database target whose stored expiry is 0, which selects never,
// and of one whose expiry is not one of the choices, which is listed // and of one whose expiry is not one of the choices, which is listed
// first as its own selected entry and saved unchanged. // first as its own selected entry and saved unchanged.
func TestArchiveExpiryEditStartsOnStoredValue(t *testing.T) { func TestArchiveExpiryEditStartsOnStoredValue(t *testing.T) {
@@ -148,7 +151,8 @@ func TestArchiveExpiryEditStartsOnStoredValue(t *testing.T) {
webhook := seedWebhookWithRetention(t, env.db, 30) webhook := seedWebhookWithRetention(t, env.db, 30)
empty := seedConfiguredTarget( empty := seedConfiguredTarget(
t, env.db, webhook.ID, database.TargetTypeDatabase, "", t, env.db, webhook.ID, database.TargetTypeDatabase,
database.Target{},
) )
_, selected := expirySelected(t, env, webhook.ID, empty.ID) _, selected := expirySelected(t, env, webhook.ID, empty.ID)
@@ -157,7 +161,7 @@ func TestArchiveExpiryEditStartsOnStoredValue(t *testing.T) {
webhook = seedWebhookWithRetention(t, env.db, 30) webhook = seedWebhookWithRetention(t, env.db, 30)
unlisted := seedConfiguredTarget( unlisted := seedConfiguredTarget(
t, env.db, webhook.ID, database.TargetTypeDatabase, t, env.db, webhook.ID, database.TargetTypeDatabase,
`{"expiry":"36h"}`, database.Target{ArchiveExpiry: 36 * time.Hour},
) )
assert.Equal(t, []string{"36 hours"}, expiryShown(t, env, webhook.ID)) assert.Equal(t, []string{"36 hours"}, expiryShown(t, env, webhook.ID))
@@ -179,7 +183,7 @@ func TestArchiveExpiryEditStartsOnStoredValue(t *testing.T) {
w := submitTargetEdit(env, webhook.ID, unlisted.ID, form) w := submitTargetEdit(env, webhook.ID, unlisted.ID, form)
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String()) require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
assert.JSONEq( assert.Equal(
t, `{"expiry":"36h"}`, storedTarget(t, env, unlisted.ID).Config, t, 36*time.Hour, storedTarget(t, env, unlisted.ID).ArchiveExpiry,
) )
} }
+40 -26
View File
@@ -12,12 +12,16 @@ import (
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
) )
// rotationNone is the archive rotation that keeps one file. // The archive rotations these tests store: none keeps one file.
const rotationNone = "none" const (
rotationNone = "none"
rotationDaily = "daily"
rotationHourly = "hourly"
)
// rotationShown returns the archive rotations the webhook page's // rotationShown returns the archive rotations the webhook page's
// target list shows. // target list shows.
@@ -41,7 +45,9 @@ func TestArchiveRotationChoices(t *testing.T) {
env := setupSourceTest(t) env := setupSourceTest(t)
for _, rotation := range []string{rotationNone, "monthly", "daily", "hourly"} { for _, rotation := range []string{
rotationNone, "monthly", rotationDaily, rotationHourly,
} {
t.Run(rotation, func(t *testing.T) { t.Run(rotation, func(t *testing.T) {
t.Parallel() t.Parallel()
@@ -60,10 +66,8 @@ func TestArchiveRotationChoices(t *testing.T) {
targets := targetsForWebhook(t, env.db, webhook.ID) targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1) require.Len(t, targets, 1)
assert.JSONEq(t, assert.Equal(t, 720*time.Hour, targets[0].ArchiveExpiry)
`{"expiry":"720h","rotation":"`+rotation+`"}`, assert.Equal(t, rotation, targets[0].ArchiveRotation)
targets[0].Config,
)
assert.Equal(t, assert.Equal(t,
[]string{rotation}, rotationShown(t, env, webhook.ID)) []string{rotation}, rotationShown(t, env, webhook.ID))
@@ -78,26 +82,37 @@ func TestArchiveRotationChoices(t *testing.T) {
w = submitTargetEdit(env, webhook.ID, targets[0].ID, form) w = submitTargetEdit(env, webhook.ID, targets[0].ID, form)
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String()) require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
assert.JSONEq(t,
`{"expiry":"720h","rotation":"hourly"}`, stored := storedTarget(t, env, targets[0].ID)
storedTarget(t, env, targets[0].ID).Config, assert.Equal(t, 720*time.Hour, stored.ArchiveExpiry)
) assert.Equal(t, rotationHourly, stored.ArchiveRotation)
}) })
} }
} }
// TestArchiveRotationEditStartsOnNone checks the edit form of a // TestArchiveRotationEditStartsOnNone checks that a database target
// database target with no rotation stored starts on none. // added with no rotation is stored with none and shown with none, and
// that its edit form starts on none.
func TestArchiveRotationEditStartsOnNone(t *testing.T) { func TestArchiveRotationEditStartsOnNone(t *testing.T) {
t.Parallel() t.Parallel()
env := setupSourceTest(t) env := setupSourceTest(t)
webhook := seedWebhookWithRetention(t, env.db, 30) webhook := seedWebhookWithRetention(t, env.db, 30)
target := seedConfiguredTarget(
t, env.db, webhook.ID, database.TargetTypeDatabase, "",
)
page := targetEditPage(t, env, webhook.ID, target.ID) form := url.Values{}
form.Set("name", "archive")
form.Set("type", string(database.TargetTypeDatabase))
w := serveTarget(
env, http.MethodPost, "/hook/"+webhook.ID+"/targets", form,
)
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1)
assert.Equal(t, rotationNone, targets[0].ArchiveRotation)
page := targetEditPage(t, env, webhook.ID, targets[0].ID)
assert.Equal(t, []string{rotationNone}, selectedIn(page, "rotation")) assert.Equal(t, []string{rotationNone}, selectedIn(page, "rotation"))
assert.Equal(t, []string{rotationNone}, rotationShown(t, env, webhook.ID)) assert.Equal(t, []string{rotationNone}, rotationShown(t, env, webhook.ID))
} }
@@ -125,7 +140,7 @@ func TestArchiveRotationRefused(t *testing.T) {
target := seedConfiguredTarget( target := seedConfiguredTarget(
t, env.db, webhook.ID, database.TargetTypeDatabase, t, env.db, webhook.ID, database.TargetTypeDatabase,
`{"rotation":"daily"}`, database.Target{ArchiveRotation: rotationDaily},
) )
form.Del("type") form.Del("type")
@@ -133,8 +148,8 @@ func TestArchiveRotationRefused(t *testing.T) {
w = submitTargetEdit(env, webhook.ID, target.ID, form) w = submitTargetEdit(env, webhook.ID, target.ID, form)
assert.Equal(t, http.StatusBadRequest, w.Code) assert.Equal(t, http.StatusBadRequest, w.Code)
assert.Contains(t, w.Body.String(), "Invalid archive rotation") assert.Contains(t, w.Body.String(), "Invalid archive rotation")
assert.JSONEq(t, assert.Equal(
`{"rotation":"daily"}`, storedTarget(t, env, target.ID).Config, t, rotationDaily, storedTarget(t, env, target.ID).ArchiveRotation,
) )
} }
@@ -161,9 +176,8 @@ func TestHandleSourceCreateSubmit_ArchiveRotation(t *testing.T) {
targets := targetsForWebhook(t, env.db, webhook.ID) targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1) require.Len(t, targets, 1)
assert.JSONEq(t, assert.Equal(t, 720*time.Hour, targets[0].ArchiveExpiry)
`{"expiry":"720h","rotation":"daily"}`, targets[0].Config, assert.Equal(t, rotationDaily, targets[0].ArchiveRotation)
)
} }
// TestArchiveFileView_Rotated describes a daily target's archive files // TestArchiveFileView_Rotated describes a daily target's archive files
@@ -178,10 +192,10 @@ func TestArchiveFileView_Rotated(t *testing.T) {
webhook := seedWebhookWithRetention(t, env.db, 30) webhook := seedWebhookWithRetention(t, env.db, 30)
target := seedConfiguredTarget( target := seedConfiguredTarget(
t, env.db, webhook.ID, database.TargetTypeDatabase, t, env.db, webhook.ID, database.TargetTypeDatabase,
`{"rotation":"daily"}`, database.Target{ArchiveRotation: rotationDaily},
) )
path := delivery.ArchivePath(env.dbMgr, &webhook, target) path := archive.Path(env.dbMgr, &webhook, target)
stem := strings.TrimSuffix(path, ".db") stem := strings.TrimSuffix(path, ".db")
written := time.Date(2026, 10, 2, 9, 0, 0, 0, time.UTC) written := time.Date(2026, 10, 2, 9, 0, 0, 0, time.UTC)
@@ -42,9 +42,6 @@ func TestDeliveryAttempts_ReadInTheTargetTypesOwnTerms(t *testing.T) {
{database.TargetTypeLog, true, 0, "", "written to the log", ""}, {database.TargetTypeLog, true, 0, "", "written to the log", ""},
} }
// Named for type and outcome, so that no two cases share a name:
// Go adds "#01" to a repeated one, and a "#" in the temporary
// directory's path cuts the path SQLiteDSN gives SQLite short.
for _, tc := range cases { for _, tc := range cases {
t.Run(string(tc.targetType)+" "+tc.outcome, func(t *testing.T) { t.Run(string(tc.targetType)+" "+tc.outcome, func(t *testing.T) {
t.Parallel() t.Parallel()
+20 -15
View File
@@ -300,21 +300,26 @@ func createReplayDelivery(
} }
return delivery.Task{ return delivery.Task{
DeliveryID: dlv.ID, DeliveryID: dlv.ID,
EventID: event.ID, EventID: event.ID,
WebhookID: webhookID, WebhookID: webhookID,
EntrypointID: event.EntrypointID, EntrypointID: event.EntrypointID,
TargetID: target.ID, TargetID: target.ID,
TargetName: target.Name, TargetName: target.Name,
TargetType: target.Type, TargetType: target.Type,
TargetConfig: target.Config, TargetURL: target.URL,
MaxRetries: target.MaxRetries, TargetHeaders: target.Headers,
Method: event.Method, TimeoutSeconds: target.TimeoutSeconds,
RawQuery: event.RawQuery, ForwardQuery: target.ForwardQuery,
Headers: event.Headers, ArchiveExpiry: target.ArchiveExpiry,
ContentType: event.ContentType, ArchiveRotation: target.ArchiveRotation,
Body: replayBody(event.Body), MaxAttempts: target.MaxAttempts,
AttemptNum: 1, Method: event.Method,
RawQuery: event.RawQuery,
Headers: event.Headers,
ContentType: event.ContentType,
Body: replayBody(event.Body),
AttemptNum: 1,
}, nil }, nil
} }
+54 -17
View File
@@ -192,7 +192,7 @@ func TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal(
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+replayTargetURL+`"}`, database.Target{URL: replayTargetURL},
) )
event, original := seedFailedDelivery( event, original := seedFailedDelivery(
@@ -205,13 +205,13 @@ func TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal(
before := loadDelivery(t, webhookDB, original.ID) before := loadDelivery(t, webhookDB, original.ID)
// The operator fixes the destination, which is the whole reason // The operator fixes the destination, which is the whole reason
// to replay. The replay must use this, not the config the // to replay. The replay must use this, not the URL the original
// original delivery ran against. // delivery ran against.
const fixedConfig = `{"url":"http://93.184.216.34/fixed"}` const fixedURL = "http://93.184.216.34/fixed"
require.NoError(t, db.DB().Model(&database.Target{}). require.NoError(t, db.DB().Model(&database.Target{}).
Where("id = ?", tgt.ID). Where("id = ?", tgt.ID).
Update("config", fixedConfig).Error) Update("url", fixedURL).Error)
w := postReplay(t, h, sess, wh.ID, original.ID) w := postReplay(t, h, sess, wh.ID, original.ID)
@@ -240,7 +240,7 @@ func TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal(
tasks := notif.Tasks() tasks := notif.Tasks()
require.Len(t, tasks, 1) require.Len(t, tasks, 1)
assertReplayTask( assertReplayTask(
t, tasks[0], wh.ID, event, tgt, replayed.ID, fixedConfig, t, tasks[0], wh.ID, event, tgt, replayed.ID, fixedURL,
) )
assertNoLeakedTarget(t, webhookDB) assertNoLeakedTarget(t, webhookDB)
} }
@@ -277,15 +277,15 @@ func assertDeliveryUntouched(
// assertReplayTask proves the task handed to the delivery engine is // assertReplayTask proves the task handed to the delivery engine is
// the one the receiver would build for this event and this target, and // the one the receiver would build for this event and this target, and
// that it carries wantConfig — the target's configuration as it stands // that it carries wantURL — the target's URL as it stands now rather
// now rather than as the original delivery ran against it. // than as the original delivery ran against it.
func assertReplayTask( func assertReplayTask(
t *testing.T, t *testing.T,
task delivery.Task, task delivery.Task,
webhookID string, webhookID string,
event *database.Event, event *database.Event,
target *database.Target, target *database.Target,
wantDeliveryID, wantConfig string, wantDeliveryID, wantURL string,
) { ) {
t.Helper() t.Helper()
@@ -295,9 +295,9 @@ func assertReplayTask(
assert.Equal(t, event.EntrypointID, task.EntrypointID) assert.Equal(t, event.EntrypointID, task.EntrypointID)
assert.Equal(t, target.ID, task.TargetID) assert.Equal(t, target.ID, task.TargetID)
assert.Equal(t, target.Type, task.TargetType) assert.Equal(t, target.Type, task.TargetType)
assert.JSONEq( assert.Equal(
t, wantConfig, task.TargetConfig, t, wantURL, task.TargetURL,
"replay must use the target's current configuration", "replay must use the target's current URL",
) )
assert.Equal(t, event.Method, task.Method) assert.Equal(t, event.Method, task.Method)
assert.Equal( assert.Equal(
@@ -327,6 +327,43 @@ func assertNoLeakedTarget(t *testing.T, webhookDB *gorm.DB) {
) )
} }
// TestHandleDeliveryReplay_TaskCarriesEveryTargetSetting proves the
// task a replay hands the delivery engine carries every one of the
// target's settings.
func TestHandleDeliveryReplay_TaskCarriesEveryTargetSetting(
t *testing.T,
) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
notif *recordingNotifier
)
app := newTestApp(t, &h, &sess, &db, &dbMgr, &notif)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP,
targetWithEverySetting(),
)
_, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
w := postReplay(t, h, sess, wh.ID, original.ID)
require.Equal(t, http.StatusSeeOther, w.Code)
tasks := notif.Tasks()
require.Len(t, tasks, 1)
assertTaskCarriesTargetSettings(t, tasks[0], tgt)
}
// TestHandleDeliveryReplay_RefusesDeletedTarget proves the required // TestHandleDeliveryReplay_RefusesDeletedTarget proves the required
// refusal: a target deleted since the delivery ran is reported as // refusal: a target deleted since the delivery ran is reported as
// deleted rather than erroring, and nothing is created or queued. // deleted rather than erroring, and nothing is created or queued.
@@ -349,7 +386,7 @@ func TestHandleDeliveryReplay_RefusesDeletedTarget(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+replayTargetURL+`"}`, database.Target{URL: replayTargetURL},
) )
event, original := seedFailedDelivery( event, original := seedFailedDelivery(
@@ -424,7 +461,7 @@ func TestHandleDeliveryReplay_RefusesWhileEarlierReplayInFlight(
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+replayTargetURL+`"}`, database.Target{URL: replayTargetURL},
) )
event, original := seedFailedDelivery( event, original := seedFailedDelivery(
@@ -498,7 +535,7 @@ func TestHandleDeliveryReplay_ReturnsToTheListItWasPressedIn(
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+replayTargetURL+`"}`, database.Target{URL: replayTargetURL},
) )
_, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID) _, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
@@ -556,7 +593,7 @@ func TestHandleSourceLogs_RendersReplayControlAndBanner(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+replayTargetURL+`"}`, database.Target{URL: replayTargetURL},
) )
_, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID) _, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
@@ -615,7 +652,7 @@ func TestHandleDeliveryReplay_LabelsTheReplay(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+replayTargetURL+`"}`, database.Target{URL: replayTargetURL},
) )
event, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID) event, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
+6 -8
View File
@@ -12,13 +12,12 @@ import (
// //
// The bound is enforced here and in SQL, because this page's // The bound is enforced here and in SQL, because this page's
// memory profile must not depend on a constant in another // memory profile must not depend on a constant in another
// package staying where it is, and because rows predating the // package staying where it is, or on every stored row having
// delivery engine's own cap or restored from an archive are // been written by the delivery engine under its own cap.
// not covered by it at all.
// //
// It happens to equal that engine cap, so a row written by the // It happens to equal that engine cap, so a row written by the
// current engine reaches this one exactly and is never cut // engine reaches this one exactly and is never cut twice.
// twice. Nothing here may assume the two differ: see view. // Nothing here may assume the two differ: see view.
const maxRenderedResponseBytes = 4096 const maxRenderedResponseBytes = 4096
// deliveryResultColumns is the delivery attempt projection. // deliveryResultColumns is the delivery attempt projection.
@@ -40,9 +39,8 @@ const deliveryResultColumns = "delivery_id, attempt_num, created_at, " +
// //
// Both Error and ResponseBody have been through the target's // Both Error and ResponseBody have been through the target's
// Redactor. The engine already masks the URL out of the // Redactor. The engine already masks the URL out of the
// errors it stores, so for errors this is a second line // errors it stores, so for errors this is a second line; for
// covering rows written before it did; for response bodies it // response bodies it is the only line, and its reach is what
// is the only line, and its reach is what
// delivery.Redactor documents. // delivery.Redactor documents.
type DeliveryResultView struct { type DeliveryResultView struct {
AttemptNum int AttemptNum int
+13 -14
View File
@@ -86,12 +86,12 @@ func seedFailedDeliveryWithResponse(
} }
// seedFailureAndRender seeds a failed delivery against a // seedFailureAndRender seeds a failed delivery against a
// target of the given type and config, and returns the // target of the given type and URL, and returns the
// rendered event log page. // rendered event log page.
func seedFailureAndRender( func seedFailureAndRender(
t *testing.T, t *testing.T,
targetType database.TargetType, targetType database.TargetType,
config, responseBody string, targetURL, responseBody string,
) string { ) string {
t.Helper() t.Helper()
@@ -109,7 +109,7 @@ func seedFailureAndRender(
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, targetType, config, t, db, wh.ID, targetType, database.Target{URL: targetURL},
) )
seedFailedDeliveryWithResponse( seedFailedDeliveryWithResponse(
@@ -129,7 +129,7 @@ func TestHandleSourceLogs_RendersFailedAttempt(t *testing.T) {
body := seedFailureAndRender( body := seedFailureAndRender(
t, t,
database.TargetTypeHTTP, database.TargetTypeHTTP,
`{"url":"https://example.com/hook/abc"}`, "https://example.com/hook/abc",
"upstream exploded", "upstream exploded",
) )
@@ -158,8 +158,7 @@ func TestHandleSourceLogs_RendersFailedAttempt(t *testing.T) {
// TestHandleSourceLogs_EscapesResponseBody proves the // TestHandleSourceLogs_EscapesResponseBody proves the
// response body is treated as the untrusted remote content it // response body is treated as the untrusted remote content it
// is. The remote chooses these bytes and the page is rendered // is. The remote chooses these bytes and the page is rendered
// inside the operator's authenticated origin, where the // inside the operator's authenticated origin.
// application's own CSP allows inline script from 'self'.
func TestHandleSourceLogs_EscapesResponseBody(t *testing.T) { func TestHandleSourceLogs_EscapesResponseBody(t *testing.T) {
t.Parallel() t.Parallel()
@@ -168,7 +167,7 @@ func TestHandleSourceLogs_EscapesResponseBody(t *testing.T) {
body := seedFailureAndRender( body := seedFailureAndRender(
t, t,
database.TargetTypeHTTP, database.TargetTypeHTTP,
`{"url":"https://example.com/hook/abc"}`, "https://example.com/hook/abc",
payload, payload,
) )
@@ -190,7 +189,7 @@ func TestHandleSourceLogs_RedactsCredentialEchoedInResponse(
body := seedFailureAndRender( body := seedFailureAndRender(
t, t,
database.TargetTypeSlack, database.TargetTypeSlack,
`{"webhookUrl":"`+slackWebhookURL+`"}`, slackWebhookURL,
"no_service: "+slackWebhookURL, "no_service: "+slackWebhookURL,
) )
@@ -229,7 +228,7 @@ func TestHandleSourceLogs_RedactsCredentialEchoedInError(
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, t, db, wh.ID,
database.TargetTypeSlack, database.TargetTypeSlack,
`{"webhookUrl":"`+slackWebhookURL+`"}`, database.Target{URL: slackWebhookURL},
) )
dlv := seedFailedDeliveryWithResponse(t, dbMgr, wh.ID, tgt.ID, "") dlv := seedFailedDeliveryWithResponse(t, dbMgr, wh.ID, tgt.ID, "")
@@ -301,7 +300,7 @@ func TestHandleSourceLogs_RedactsCredentialSeveredByTheEngineCut(
body := seedFailureAndRender( body := seedFailureAndRender(
t, t,
database.TargetTypeSlack, database.TargetTypeSlack,
`{"webhookUrl":"`+slackWebhookURL+`"}`, slackWebhookURL,
stored, stored,
) )
@@ -336,7 +335,7 @@ func TestHandleSourceLogs_RedactsCredentialSeveredBySQLCut(
body := seedFailureAndRender( body := seedFailureAndRender(
t, t,
database.TargetTypeSlack, database.TargetTypeSlack,
`{"webhookUrl":"`+slackWebhookURL+`"}`, slackWebhookURL,
stored, stored,
) )
@@ -374,7 +373,7 @@ func TestHandleSourceLogs_RedactsForSoftDeletedTarget(
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, t, db, wh.ID,
database.TargetTypeSlack, database.TargetTypeSlack,
`{"webhookUrl":"`+slackWebhookURL+`"}`, database.Target{URL: slackWebhookURL},
) )
seedFailedDeliveryWithResponse( seedFailedDeliveryWithResponse(
@@ -414,7 +413,7 @@ func TestHandleSourceLogs_BoundsRenderedAttempts(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeLog, "", t, db, wh.ID, database.TargetTypeLog, database.Target{},
) )
dlv := seedFailedDeliveryWithResponse(t, dbMgr, wh.ID, tgt.ID, "") dlv := seedFailedDeliveryWithResponse(t, dbMgr, wh.ID, tgt.ID, "")
@@ -481,7 +480,7 @@ func TestHandleSourceLogs_BoundsOversizeResponse(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeLog, "", t, db, wh.ID, database.TargetTypeLog, database.Target{},
) )
stored := strings.Repeat("A", responseCap*4) + tail stored := strings.Repeat("A", responseCap*4) + tail
+4 -5
View File
@@ -35,11 +35,10 @@ const eventBodyQuery = "SELECT cast(body as blob) " +
// response is deliberately not renderable. Content-Disposition // response is deliberately not renderable. Content-Disposition
// makes the browser download rather than display it, and the // makes the browser download rather than display it, and the
// octet-stream type plus nosniff stop it being interpreted as // octet-stream type plus nosniff stop it being interpreted as
// HTML or script. Without those a stored payload would execute // HTML or script. Without those a stored payload would be shown
// as the logged-in operator. The application's CSP does not // as a page of the operator's own origin. The application's CSP
// help here: script-src allows 'unsafe-inline' from 'self', so // is a second defence behind them: it allows no inline script,
// a document served from this origin could run its own inline // so such a page could not run script of its own.
// script.
func (h *Handlers) HandleEventBodyDownload() http.HandlerFunc { func (h *Handlers) HandleEventBodyDownload() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) { return func(w http.ResponseWriter, r *http.Request) {
webhook, ok := h.ownedWebhook(w, r) webhook, ok := h.ownedWebhook(w, r)
+1 -1
View File
@@ -84,7 +84,7 @@ func TestHandleEventDetail_ShowsEventWholeWithDeliveries(t *testing.T) {
assert.Contains(t, page, strconv.Itoa(len(body))+" bytes") assert.Contains(t, page, strconv.Itoa(len(body))+" bytes")
assert.Contains(t, page, "{\n &#34;pad&#34;: &#34;xxx") assert.Contains(t, page, "{\n &#34;pad&#34;: &#34;xxx")
assert.Contains(t, page, "&#34;tail&#34;: &#34;"+sentinel+"&#34;\n}") assert.Contains(t, page, "&#34;tail&#34;: &#34;"+sentinel+"&#34;\n}")
assert.Contains(t, page, `style="max-height: 32rem; overflow-y: auto"`) assert.Contains(t, page, `break-all max-h-[32rem] overflow-y-auto"`)
assert.NotContains(t, page, "Showing the first") assert.NotContains(t, page, "Showing the first")
assert.Contains(t, page, target.Name) assert.Contains(t, page, target.Name)
assert.Contains(t, page, ">failed</span>") assert.Contains(t, page, ">failed</span>")
+4 -4
View File
@@ -13,8 +13,8 @@ import (
// DeliveryView is the display-safe projection of a delivery // DeliveryView is the display-safe projection of a delivery
// for the event log page. Its target is a TargetView, so the // for the event log page. Its target is a TargetView, so the
// stored configuration blob — which holds the target's // target's URL and headers — which hold the target's
// credential — has no path to the template. // credential — have no path to the template.
type DeliveryView struct { type DeliveryView struct {
ID string ID string
Status database.DeliveryStatus Status database.DeliveryStatus
@@ -126,7 +126,7 @@ func (h *Handlers) HandleSourceLogs() http.HandlerFunc {
// loadTargetMap loads targets into a map of display-safe // loadTargetMap loads targets into a map of display-safe
// views keyed by target ID, each paired with its redactor. // views keyed by target ID, each paired with its redactor.
// The projection happens here so that no caller can hand a // The projection happens here so that no caller can hand a
// raw target, configuration blob and all, to a template: the // raw target, URL and headers and all, to a template: the
// raw rows do not leave this function. // raw rows do not leave this function.
// //
// The load is Unscoped because deleting a target only soft // The load is Unscoped because deleting a target only soft
@@ -496,7 +496,7 @@ const deliveryIDChunkSize = 500
// //
// Each response body is cut by SQLite rather than in Go, for // Each response body is cut by SQLite rather than in Go, for
// the reason deliveryResultColumns gives. How many attempts a // the reason deliveryResultColumns gives. How many attempts a
// delivery has is the target's MaxRetries, which the // delivery has is bounded by the target's MaxAttempts, which the
// authenticated operator sets; how many of them reach the page // authenticated operator sets; how many of them reach the page
// is bounded again by maxRenderedAttempts. // is bounded again by maxRenderedAttempts.
func (h *Handlers) loadDeliveryResults( func (h *Handlers) loadDeliveryResults(
+7 -7
View File
@@ -151,7 +151,7 @@ func TestHandleEventResubmit_DeliversToTargetCreatedAfterTheEvent(
// captured. It has no prior delivery of anything. // captured. It has no prior delivery of anything.
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+resubmitTargetURL+`"}`, database.Target{URL: resubmitTargetURL},
) )
w := postResubmit(t, h, sess, wh.ID, original.ID) w := postResubmit(t, h, sess, wh.ID, original.ID)
@@ -274,7 +274,7 @@ func TestHandleEventResubmit_IsRepeatable(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+resubmitTargetURL+`"}`, database.Target{URL: resubmitTargetURL},
) )
original := seedStoredEvent(t, dbMgr, wh.ID, `{"fire":"again"}`) original := seedStoredEvent(t, dbMgr, wh.ID, `{"fire":"again"}`)
@@ -368,7 +368,7 @@ func TestHandleEventResubmit_OversizeBodySurvivesIntact(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
seedConfiguredTarget( seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+resubmitTargetURL+`"}`, database.Target{URL: resubmitTargetURL},
) )
original := seedStoredEvent(t, dbMgr, wh.ID, stored) original := seedStoredEvent(t, dbMgr, wh.ID, stored)
@@ -425,11 +425,11 @@ func TestHandleEventResubmit_SkipsInactiveTarget(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
active := seedConfiguredTarget( active := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+resubmitTargetURL+`"}`, database.Target{URL: resubmitTargetURL},
) )
off := seedConfiguredTarget( off := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+resubmitTargetURL+`/off"}`, database.Target{URL: resubmitTargetURL + "/off"},
) )
require.NoError(t, db.DB().Model(&database.Target{}). require.NoError(t, db.DB().Model(&database.Target{}).
@@ -528,7 +528,7 @@ func TestHandleEventResubmit_RefusesEventOfAnotherWebhook(
mine := seedWebhook(t, db) mine := seedWebhook(t, db)
seedConfiguredTarget( seedConfiguredTarget(
t, db, mine.ID, database.TargetTypeHTTP, t, db, mine.ID, database.TargetTypeHTTP,
`{"url":"`+resubmitTargetURL+`"}`, database.Target{URL: resubmitTargetURL},
) )
seedStoredEvent(t, dbMgr, mine.ID, `{"mine":true}`) seedStoredEvent(t, dbMgr, mine.ID, `{"mine":true}`)
@@ -575,7 +575,7 @@ func TestHandleSourceLogs_ShowsResubmitProvenance(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
seedConfiguredTarget( seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+resubmitTargetURL+`"}`, database.Target{URL: resubmitTargetURL},
) )
original := seedStoredEvent(t, dbMgr, wh.ID, `{"trace":"me"}`) original := seedStoredEvent(t, dbMgr, wh.ID, `{"trace":"me"}`)
+19 -27
View File
@@ -37,10 +37,10 @@ const MaxRenderedResponseBytesForTest = maxRenderedResponseBytes
// per-delivery attempt ceiling to the handlers_test package. // per-delivery attempt ceiling to the handlers_test package.
const MaxRenderedAttemptsForTest = maxRenderedAttempts const MaxRenderedAttemptsForTest = maxRenderedAttempts
// MaxTargetRetriesForTest exposes the target max_retries ceiling to // MaxTargetAttemptsForTest exposes the target max_attempts ceiling to
// the handlers_test package, so the tests assert against the constant // the handlers_test package, so the tests assert against the constant
// the handlers enforce rather than a number copied beside it. // the handlers enforce rather than a number copied beside it.
const MaxTargetRetriesForTest = maxTargetRetries const MaxTargetAttemptsForTest = maxTargetAttempts
// EventDBLeftMsgForTest and SidecarLeftMsgForTest expose the two // EventDBLeftMsgForTest and SidecarLeftMsgForTest expose the two
// messages the webhook delete handler logs when a file of the event // messages the webhook delete handler logs when a file of the event
@@ -133,36 +133,28 @@ func (s *Handlers) RenderTemplateForTest(
s.renderTemplate(w, r, pageTemplate, data) s.renderTemplate(w, r, pageTemplate, data)
} }
// BuildSlackTargetConfigForTest exposes // SetSlackTargetSettingsForTest exposes setSlackTargetSettings for use
// buildSlackTargetConfig for use in the handlers_test package. // in the handlers_test package.
func (s *Handlers) BuildSlackTargetConfigForTest( func (s *Handlers) SetSlackTargetSettingsForTest(
ctx context.Context, ctx context.Context,
target *database.Target,
targetURL string, targetURL string,
) (string, string, error) { ) string {
return s.buildSlackTargetConfig(ctx, targetURL) return s.setSlackTargetSettings(ctx, target, targetURL)
} }
// BuildHTTPTargetConfigForTest exposes buildHTTPTargetConfig // SetDatabaseTargetSettingsForTest exposes setDatabaseTargetSettings
// for use in the handlers_test package, taking the form fields // for use in the handlers_test package.
// an HTTP target's configuration is built from. func SetDatabaseTargetSettingsForTest(
func (s *Handlers) BuildHTTPTargetConfigForTest( target *database.Target,
ctx context.Context,
targetURL, headers, timeout string,
) (string, string, error) {
return s.buildHTTPTargetConfig(ctx, targetFormInput{
URL: targetURL,
Headers: headers,
Timeout: timeout,
})
}
// BuildDatabaseTargetConfigForTest exposes
// buildDatabaseTargetConfig for use in the handlers_test
// package.
func BuildDatabaseTargetConfigForTest(
expiry, rotation string, expiry, rotation string,
) (string, string, error) { ) string {
return buildDatabaseTargetConfig(expiry, rotation) return setDatabaseTargetSettings(target, expiry, rotation)
}
// FormatArchiveExpiryForTest exposes formatArchiveExpiry.
func FormatArchiveExpiryForTest(expiry time.Duration) string {
return formatArchiveExpiry(expiry)
} }
// ArchiveFileViewForTest exposes archiveFileView, which describes a // ArchiveFileViewForTest exposes archiveFileView, which describes a
+1 -1
View File
@@ -374,7 +374,7 @@ func assertFloodBounded(t *testing.T, label, out string) {
// this service's three gorm.Open sites: the main database behind the // this service's three gorm.Open sites: the main database behind the
// two unauthenticated lookups, and the per-webhook database the // two unauthenticated lookups, and the per-webhook database the
// WebhookDBManager opens. The third, the archive writer, is pinned in // WebhookDBManager opens. The third, the archive writer, is pinned in
// internal/delivery, where its type lives. // internal/archive, where its type lives.
// //
// What each assertion is worth, since two of the three would pass // What each assertion is worth, since two of the three would pass
// against a service that had never been fixed if the capture were set // against a service that had never been fixed if the capture were set
+3 -14
View File
@@ -10,11 +10,11 @@ import (
"html/template" "html/template"
"log/slog" "log/slog"
"net/http" "net/http"
"sync"
"sync/atomic" "sync/atomic"
"github.com/prometheus/client_golang/prometheus" "github.com/prometheus/client_golang/prometheus"
"go.uber.org/fx" "go.uber.org/fx"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery" "sneak.berlin/go/webhooker/internal/delivery"
@@ -65,7 +65,7 @@ type HandlersParams struct {
Session *session.Session Session *session.Session
Middleware *middleware.Middleware Middleware *middleware.Middleware
Notifier delivery.Notifier Notifier delivery.Notifier
Archives delivery.Archives Archives archive.Archives
CircuitBreakers delivery.CircuitBreakers CircuitBreakers delivery.CircuitBreakers
SSRFGuard *delivery.Guard SSRFGuard *delivery.Guard
Metrics *metrics.Set Metrics *metrics.Set
@@ -83,7 +83,7 @@ type Handlers struct {
session *session.Session session *session.Session
mw *middleware.Middleware mw *middleware.Middleware
notifier delivery.Notifier notifier delivery.Notifier
archives delivery.Archives archives archive.Archives
breakers delivery.CircuitBreakers breakers delivery.CircuitBreakers
mtr *metrics.Set mtr *metrics.Set
templates map[string]*template.Template templates map[string]*template.Template
@@ -93,17 +93,6 @@ type Handlers struct {
// is one delivery will actually attempt. // is one delivery will actually attempt.
ssrf *delivery.Guard ssrf *delivery.Guard
// renameMu makes the webhook edit, the target edit and target
// creation run one at a time, each held from loading the stored
// names through the archive rename, the save and any move back.
// Interleaved, one could rename an archive between another's
// rename and save, leaving the file named for one edit and the
// stored names from the other. An archive download holds it while
// it reads the stored names and lists the files they give, and
// again for each file while it finds the file under the names
// stored then and opens it.
renameMu sync.Mutex
// dummyVerifications counts the equivalent-cost verifications // dummyVerifications counts the equivalent-cost verifications
// charged for usernames that do not exist. It exists so a test // charged for usernames that do not exist. It exists so a test
// can prove that path runs without measuring wall-clock time. // can prove that path runs without measuring wall-clock time.
+98 -60
View File
@@ -8,6 +8,7 @@ import (
"log/slog" "log/slog"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"strconv"
"sync" "sync"
"testing" "testing"
"time" "time"
@@ -16,6 +17,7 @@ import (
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"go.uber.org/fx" "go.uber.org/fx"
"go.uber.org/fx/fxtest" "go.uber.org/fx/fxtest"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery" "sneak.berlin/go/webhooker/internal/delivery"
@@ -55,12 +57,14 @@ func (n *recordingNotifier) Tasks() []delivery.Task {
return out return out
} }
// recordingArchives is a delivery.Archives that records what it // recordingArchives is an archive.Archives that records what it
// was asked to do, so a test can prove that a deletion or rename // was asked to do, so a test can prove that a deletion or rename
// path reached the delivery engine. After FailRenames, every // path reached the archive writers. After FailRenames, every
// rename of that target fails with the given error. After // rename of that target fails with the given error. After
// BlockNextRename, the next rename is recorded and then waits. // BlockNextRename, the next rename is recorded and then waits.
// RenameLock returns a plain mutex, as in archive.Writers.
type recordingArchives struct { type recordingArchives struct {
renameMu sync.Mutex
mu sync.Mutex mu sync.Mutex
evicted []string evicted []string
evictedTargets []string evictedTargets []string
@@ -73,10 +77,10 @@ type recordingArchives struct {
// errInjectedRename is the failure a test hands FailRenames. // errInjectedRename is the failure a test hands FailRenames.
var errInjectedRename = errors.New("injected rename failure") var errInjectedRename = errors.New("injected rename failure")
// errNameTaken is what the delivery engine returns when a file // errNameTaken is what the archive writers return when a file
// already has an archive's new name, here archive-taken.db. // already has an archive's new name, here archive-taken.db.
var errNameTaken = fmt.Errorf( var errNameTaken = fmt.Errorf(
"%w: archive-taken.db", delivery.ErrArchiveNameTaken, "%w: archive-taken.db", archive.ErrNameTaken,
) )
// archiveRename is one recorded Rename call. // archiveRename is one recorded Rename call.
@@ -86,6 +90,10 @@ type archiveRename struct {
TargetName string TargetName string
} }
func (r *recordingArchives) RenameLock() *sync.Mutex {
return &r.renameMu
}
func (r *recordingArchives) EvictWebhook(webhookID string) { func (r *recordingArchives) EvictWebhook(webhookID string) {
r.mu.Lock() r.mu.Lock()
defer r.mu.Unlock() defer r.mu.Unlock()
@@ -265,7 +273,7 @@ func newTestAppWithConfig(
func() *recordingArchives { func() *recordingArchives {
return &recordingArchives{} return &recordingArchives{}
}, },
func(r *recordingArchives) delivery.Archives { func(r *recordingArchives) archive.Archives {
return r return r
}, },
func() *testCircuitBreakers { func() *testCircuitBreakers {
@@ -347,7 +355,7 @@ func TestHandleIndex_Authenticated(t *testing.T) {
) )
} }
func TestBuildSlackTargetConfig_AcceptsPublicURL(t *testing.T) { func TestSetSlackTargetSettings_AcceptsPublicURL(t *testing.T) {
t.Parallel() t.Parallel()
var h *handlers.Handlers var h *handlers.Handlers
@@ -357,16 +365,17 @@ func TestBuildSlackTargetConfig_AcceptsPublicURL(t *testing.T) {
t.Cleanup(app.RequireStop) t.Cleanup(app.RequireStop)
cfg, errMsg, err := h.BuildSlackTargetConfigForTest( const hookURL = "http://93.184.216.34/services/T00/B00/xxx"
t.Context(), "http://93.184.216.34/services/T00/B00/xxx",
) var target database.Target
errMsg := h.SetSlackTargetSettingsForTest(t.Context(), &target, hookURL)
require.NoError(t, err)
assert.Empty(t, errMsg) assert.Empty(t, errMsg)
assert.Contains(t, cfg, "webhookUrl") assert.Equal(t, hookURL, target.URL)
} }
func TestBuildSlackTargetConfig_RejectsReservedURL(t *testing.T) { func TestSetSlackTargetSettings_RejectsReservedURL(t *testing.T) {
t.Parallel() t.Parallel()
var h *handlers.Handlers var h *handlers.Handlers
@@ -376,13 +385,14 @@ func TestBuildSlackTargetConfig_RejectsReservedURL(t *testing.T) {
t.Cleanup(app.RequireStop) t.Cleanup(app.RequireStop)
cfg, errMsg, err := h.BuildSlackTargetConfigForTest( var target database.Target
t.Context(), "http://169.254.169.254/latest/meta-data/",
errMsg := h.SetSlackTargetSettingsForTest(
t.Context(), &target, "http://169.254.169.254/latest/meta-data/",
) )
require.NoError(t, err)
assert.Contains(t, errMsg, "Invalid target URL") assert.Contains(t, errMsg, "Invalid target URL")
assert.Empty(t, cfg) assert.Empty(t, target.URL)
} }
func TestRenderTemplate(t *testing.T) { func TestRenderTemplate(t *testing.T) {
@@ -476,72 +486,100 @@ func TestRenderTemplateMidRenderErrorSendsNoPartialBody(t *testing.T) {
) )
} }
func TestBuildDatabaseTargetConfig_Valid(t *testing.T) { func TestSetDatabaseTargetSettings_Valid(t *testing.T) {
t.Parallel() t.Parallel()
// Empty expiry and rotation: the keep-forever, one-file default, cases := []struct {
// empty config. expiry, rotation string
cfg, errMsg, err := handlers.BuildDatabaseTargetConfigForTest("", "") wantExpiry time.Duration
require.NoError(t, err) wantRotation string
assert.Empty(t, errMsg) }{
assert.Empty(t, cfg) // Empty expiry and rotation: the keep-forever, one-file
// default.
{"", "", 0, rotationNone},
{expiryNever, "", 0, rotationNone},
{" 36h ", "", 36 * time.Hour, rotationNone},
{"", rotationDaily, 0, rotationDaily},
{"90m", rotationHourly, 90 * time.Minute, rotationHourly},
}
// Explicit never is stored as config. for _, tc := range cases {
cfg, errMsg, err = handlers.BuildDatabaseTargetConfigForTest("never", "") var target database.Target
require.NoError(t, err)
assert.Empty(t, errMsg)
assert.JSONEq(t, `{"expiry":"never"}`, cfg)
// A positive duration is stored as config. errMsg := handlers.SetDatabaseTargetSettingsForTest(
cfg, errMsg, err = handlers.BuildDatabaseTargetConfigForTest("720h", "") &target, tc.expiry, tc.rotation,
require.NoError(t, err) )
assert.Empty(t, errMsg)
assert.JSONEq(t, `{"expiry":"720h"}`, cfg)
// A rotation is stored as config, with or without an expiry. assert.Empty(t, errMsg, tc)
cfg, errMsg, err = handlers.BuildDatabaseTargetConfigForTest("", "daily") assert.Equal(t, tc.wantExpiry, target.ArchiveExpiry, tc)
require.NoError(t, err) assert.Equal(t, tc.wantRotation, target.ArchiveRotation, tc)
assert.Empty(t, errMsg) }
assert.JSONEq(t, `{"rotation":"daily"}`, cfg)
cfg, errMsg, err = handlers.BuildDatabaseTargetConfigForTest(
"720h", "hourly",
)
require.NoError(t, err)
assert.Empty(t, errMsg)
assert.JSONEq(t, `{"expiry":"720h","rotation":"hourly"}`, cfg)
} }
func TestBuildDatabaseTargetConfig_RejectsBadExpiry( // TestSetDatabaseTargetSettings_RejectsBadExpiry pins each refusal's
// whole message, as the form shows it.
func TestSetDatabaseTargetSettings_RejectsBadExpiry(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
for _, bad := range []string{"nonsense", "7d", "-5h"} { cases := map[string]string{
cfg, errMsg, err := handlers.BuildDatabaseTargetConfigForTest(bad, "") "nonsense": `Invalid archive expiry: expiry must be "never" or ` +
`a Go duration such as "720h": time: invalid duration ` +
`"nonsense"`,
"7d": `Invalid archive expiry: expiry must be "never" or a Go ` +
`duration such as "720h": time: unknown unit "d" in ` +
`duration "7d"`,
"-5h": `Invalid archive expiry: expiry must be a positive ` +
`duration or "never": "-5h"`,
}
require.NoError(t, err) for bad, want := range cases {
assert.Contains( var target database.Target
t, errMsg, "Invalid archive expiry",
"expiry %q should be refused", bad, errMsg := handlers.SetDatabaseTargetSettingsForTest(
&target, bad, "",
) )
assert.Empty(t, cfg)
assert.Equal(t, want, errMsg, "expiry %q", bad)
} }
} }
func TestBuildDatabaseTargetConfig_RejectsBadRotation( func TestSetDatabaseTargetSettings_RejectsBadRotation(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
for _, bad := range []string{"weekly", "Daily", " none"} { for _, bad := range []string{"weekly", "Daily", " none"} {
cfg, errMsg, err := handlers.BuildDatabaseTargetConfigForTest("", bad) var target database.Target
require.NoError(t, err) errMsg := handlers.SetDatabaseTargetSettingsForTest(
assert.Contains( &target, "", bad,
t, errMsg, "Invalid archive rotation", )
"rotation %q should be refused", bad,
assert.Equal(
t,
"Invalid archive rotation: rotation must be none, "+
"monthly, daily or hourly: "+strconv.Quote(bad),
errMsg,
) )
assert.Empty(t, cfg) }
}
// TestFormatArchiveExpiry covers the expiries the expiry choices do
// not: TestArchiveExpiryChoices shows the edit form starting on each
// choice. An expiry in whole hours is written in hours, and any other
// as Go writes it, which the form accepts back unchanged.
func TestFormatArchiveExpiry(t *testing.T) {
t.Parallel()
cases := map[time.Duration]string{
0: expiryNever,
36 * time.Hour: "36h",
90 * time.Minute: "1h30m0s",
}
for expiry, want := range cases {
assert.Equal(t, want, handlers.FormatArchiveExpiryForTest(expiry))
} }
} }
+11 -53
View File
@@ -1,19 +1,15 @@
package handlers_test package handlers_test
// The handler-side half of the log-field audit. Two slog calls in // The handler-side half of the log-field audit. The slog calls in
// this package reach a value an UNAUTHENTICATED client picks outright // this package that reach a value an UNAUTHENTICATED client picks
// and of a length it picks outright: // outright, and of a length it picks outright, are the failed-login
// DEBUG lines, whose username is a form field.
// //
// - the unknown-entrypoint DEBUG line on /h/{uuid}, whose // They are at DEBUG, which is off in production by default. That is
// path segment matched no stored entrypoint and so is bounded by
// nothing;
// - the failed-login DEBUG lines, whose username is a form field.
//
// Both are at DEBUG, which is off in production by default. That is
// not a bound: an operator turning DEBUG on to diagnose a flood must // not a bound: an operator turning DEBUG on to diagnose a flood must
// not thereby hand the flood an unbounded write. Both spend the same // not thereby hand the flood an unbounded write. They spend the same
// internal/logfield budget as the access log, and both are held here // internal/logfield budget as the access log, and are held here to
// to middleware.MaxAccessLogLineBytes. // middleware.MaxAccessLogLineBytes.
// //
// The two login lines past the username lookup — "invalid password" // The two login lines past the username lookup — "invalid password"
// and "user logged in" — carry the same cap without needing it, since // and "user logged in" — carry the same cap without needing it, since
@@ -195,13 +191,12 @@ func receiverRouter(h *handlers.Handlers) *chi.Mux {
return router return router
} }
// postReceiver sends one POST at /h/<segment>. // postReceiver sends one POST with an empty body at /h/<segment>
// through router and returns the response status.
// //
// RawPath is cleared after parsing so chi routes on the decoded path // RawPath is cleared after parsing so chi routes on the decoded path
// and the handler sees the raw bytes rather than their percent-escaped // and the handler sees the raw bytes rather than their percent-escaped
// spelling. That is the harder case for the budget: the escaped // spelling.
// spelling is plain ASCII, which costs one byte per byte, while the
// decoded bytes are what the log handler has to escape.
func postReceiver( func postReceiver(
t *testing.T, router *chi.Mux, segment string, t *testing.T, router *chi.Mux, segment string,
) int { ) int {
@@ -260,43 +255,6 @@ func postLoginWithPassword(
return w.Code return w.Code
} }
// TestUnknownEntrypoint_LogLineDoesNotTrackPathSize drives 8 KB of
// client-chosen path at the unauthenticated receiver's
// unknown-entrypoint DEBUG line and holds it to the same ceiling the
// access log states.
func TestUnknownEntrypoint_LogLineDoesNotTrackPathSize(t *testing.T) {
t.Parallel()
for handlerName, newHandler := range logHandlers() {
for fillName, fill := range escapeFills() {
t.Run(handlerName+"/"+fillName, func(t *testing.T) {
t.Parallel()
h, buf := capturingHandlers(t, newHandler)
router := receiverRouter(h)
for i := range floodRequests {
assert.Equal(
t,
http.StatusNotFound,
postReceiver(
t, router,
oversizedFill(fill)+
strings.Repeat("y", i),
),
)
}
lines := logLines(t, buf)
require.Len(t, lines, floodRequests)
assertNoClientText(t, buf)
assertBoundedFlood(t, buf.Len())
})
}
}
}
// TestFailedLogin_LogLineDoesNotTrackUsernameSize drives 8 KB of // TestFailedLogin_LogLineDoesNotTrackUsernameSize drives 8 KB of
// client-chosen username at the unauthenticated login endpoint's // client-chosen username at the unauthenticated login endpoint's
// DEBUG line and holds it to the same ceiling. // DEBUG line and holds it to the same ceiling.
+6 -15
View File
@@ -16,21 +16,16 @@ import (
// //
// An empty value yields fallback, which lets the create path apply the // An empty value yields fallback, which lets the create path apply the
// default and the edit path leave the stored value unchanged. A value // default and the edit path leave the stored value unchanged. A value
// of 0 is returned as 0 and is rewritten to the retain-forever // of 0 means retain events forever and is returned as it is. Anything
// sentinel by database.Webhook's BeforeSave hook. Anything unparseable // unparseable or negative is refused rather than silently given a
// or negative is refused rather than silently given a default. // default.
// //
// The upper bound is not cosmetic. The reaper computes its cutoff as a // The upper bound is not cosmetic. The reaper computes its cutoff as a
// time.Duration, an int64 nanosecond count, so a day count above // time.Duration, an int64 nanosecond count, so a day count above
// database.MaxFiniteRetentionDays overflows, puts the cutoff in the // database.MaxFiniteRetentionDays overflows, puts the cutoff in the
// future, and deletes every event the webhook has. A finite value // future, and deletes every event the webhook has. A value above that
// above that ceiling is therefore refused, and the message names the // ceiling is therefore refused, and the message names the ceiling
// ceiling rather than implying the input was not a number. // rather than implying the input was not a number.
//
// A value at or above the retain-forever sentinel is not out of range:
// it is what the edit form pre-fills for a retain-forever webhook, so
// submitting the form back unchanged has to keep meaning "forever"
// rather than being rejected.
func parseRetentionDays(raw string, fallback int) (int, string) { func parseRetentionDays(raw string, fallback int) (int, string) {
raw = strings.TrimSpace(raw) raw = strings.TrimSpace(raw)
if raw == "" { if raw == "" {
@@ -43,10 +38,6 @@ func parseRetentionDays(raw string, fallback int) (int, string) {
"retain events forever." "retain events forever."
} }
if v >= database.RetentionForeverDays {
return database.RetentionForeverDays, ""
}
if v > database.MaxFiniteRetentionDays { if v > database.MaxFiniteRetentionDays {
return 0, "Retention must be at most " + return 0, "Retention must be at most " +
strconv.Itoa(database.MaxFiniteRetentionDays) + strconv.Itoa(database.MaxFiniteRetentionDays) +
+23 -12
View File
@@ -7,6 +7,7 @@ import (
"net/http/httptest" "net/http/httptest"
"net/url" "net/url"
"testing" "testing"
"time"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
@@ -54,15 +55,26 @@ func TestHandleSourceCreateSubmit_CreatesRequestedTargets(t *testing.T) {
env := setupSourceTest(t) env := setupSourceTest(t)
// Each value the archive pruning choice submits, after an empty // Each value the archive pruning choice submits and the expiry it
// one that stands for the archive checkbox left off. // stores, after an empty one that stands for the archive checkbox
expiries := []string{ // left off.
"", "never", "1h", "12h", "24h", "720h", "2160h", "8760h", expiries := []struct {
value string
stored time.Duration
}{
{"", 0},
{expiryNever, 0},
{"1h", time.Hour},
{"12h", 12 * time.Hour},
{"24h", 24 * time.Hour},
{"720h", 720 * time.Hour},
{"2160h", 2160 * time.Hour},
{"8760h", 8760 * time.Hour},
} }
for _, httpURL := range []string{"", editOriginalURL} { for _, httpURL := range []string{"", editOriginalURL} {
for _, expiry := range expiries { for _, expiry := range expiries {
name := "url=" + httpURL + " archive=" + expiry name := "url=" + httpURL + " archive=" + expiry.value
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
t.Parallel() t.Parallel()
@@ -72,9 +84,9 @@ func TestHandleSourceCreateSubmit_CreatesRequestedTargets(t *testing.T) {
form.Set("http_url", httpURL) form.Set("http_url", httpURL)
form.Set("archive_expiry", "720h") form.Set("archive_expiry", "720h")
if expiry != "" { if expiry.value != "" {
form.Set("archive", "on") form.Set("archive", "on")
form.Set("archive_expiry", expiry) form.Set("archive_expiry", expiry.value)
} }
w := submitCreateForm(env, form) w := submitCreateForm(env, form)
@@ -96,17 +108,16 @@ func TestHandleSourceCreateSubmit_CreatesRequestedTargets(t *testing.T) {
wantCount++ wantCount++
assert.Equal(t, "HTTP", byType[database.TargetTypeHTTP].Name) assert.Equal(t, "HTTP", byType[database.TargetTypeHTTP].Name)
assert.JSONEq(t, `{"url":"`+httpURL+`"}`, assert.Equal(t, httpURL, byType[database.TargetTypeHTTP].URL)
byType[database.TargetTypeHTTP].Config)
} }
if expiry != "" { if expiry.value != "" {
wantCount++ wantCount++
assert.Equal(t, "Archive", assert.Equal(t, "Archive",
byType[database.TargetTypeDatabase].Name) byType[database.TargetTypeDatabase].Name)
assert.JSONEq(t, `{"expiry":"`+expiry+`"}`, assert.Equal(t, expiry.stored,
byType[database.TargetTypeDatabase].Config) byType[database.TargetTypeDatabase].ArchiveExpiry)
} }
assert.Len(t, byType, wantCount) assert.Len(t, byType, wantCount)
+12 -11
View File
@@ -16,8 +16,8 @@ import (
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"gorm.io/gorm" "gorm.io/gorm"
"gorm.io/gorm/clause" "gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/handlers" "sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/session" "sneak.berlin/go/webhooker/internal/session"
) )
@@ -64,10 +64,11 @@ func seedTarget(
t.Helper() t.Helper()
tgt := &database.Target{ tgt := &database.Target{
WebhookID: webhookID, WebhookID: webhookID,
Name: "t-" + string(targetType), Name: "t-" + string(targetType),
Type: targetType, Type: targetType,
Active: true, Active: true,
MaxAttempts: 1,
} }
require.NoError( require.NoError(
@@ -179,7 +180,7 @@ func failSaveOnTable(
} }
// archivePathFor returns the archive database path the // archivePathFor returns the archive database path the
// delivery engine would use for a database target: beside the // archive writers would use for a database target: beside the
// webhook's event database in the data directory. // webhook's event database in the data directory.
func archivePathFor( func archivePathFor(
t *testing.T, t *testing.T,
@@ -191,7 +192,7 @@ func archivePathFor(
return filepath.Join( return filepath.Join(
filepath.Dir(mgr.DBPath(wh.ID)), filepath.Dir(mgr.DBPath(wh.ID)),
delivery.ArchiveFileName(wh.Name, tgt.Name, tgt.ID), archive.FileName(wh.Name, tgt.Name, tgt.ID),
) )
} }
@@ -227,9 +228,9 @@ func postRequest(
} }
// TestHandleSourceDelete_EvictsArchiveWriter proves that // TestHandleSourceDelete_EvictsArchiveWriter proves that
// deleting a webhook reaches the delivery engine and releases // deleting a webhook releases the webhook's archive writers,
// the webhook's archive writers, exercised through the real // exercised through the real deletion handler rather than by
// deletion handler rather than by calling the engine directly. // calling the archive writers directly.
func TestHandleSourceDelete_EvictsArchiveWriter(t *testing.T) { func TestHandleSourceDelete_EvictsArchiveWriter(t *testing.T) {
t.Parallel() t.Parallel()
@@ -289,7 +290,7 @@ func TestHandleSourceDelete_KeepsArchiveFile(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedTarget(t, db, wh.ID, database.TargetTypeDatabase) tgt := seedTarget(t, db, wh.ID, database.TargetTypeDatabase)
// Place an archive file where the delivery engine would. // Place an archive file where the archive writers would.
archivePath := archivePathFor(t, mgr, wh, tgt) archivePath := archivePathFor(t, mgr, wh, tgt)
require.NoError( require.NoError(
t, t,
+79 -103
View File
@@ -5,6 +5,7 @@ import (
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"testing" "testing"
"time"
"github.com/go-chi/chi" "github.com/go-chi/chi"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
@@ -25,24 +26,24 @@ const (
slackSecretPath slackSecretPath
) )
// seedConfiguredTarget inserts a target with a stored config // seedConfiguredTarget inserts an active target of targetType with
// blob and returns it. // the settings (URL to ArchiveRotation) that settings carries, and
// returns it.
func seedConfiguredTarget( func seedConfiguredTarget(
t *testing.T, t *testing.T,
db *database.Database, db *database.Database,
webhookID string, webhookID string,
targetType database.TargetType, targetType database.TargetType,
config string, settings database.Target,
) *database.Target { ) *database.Target {
t.Helper() t.Helper()
tgt := &database.Target{ tgt := &settings
WebhookID: webhookID, tgt.WebhookID = webhookID
Name: "t-" + string(targetType), tgt.Name = "t-" + string(targetType)
Type: targetType, tgt.Type = targetType
Active: true, tgt.Active = true
Config: config, tgt.MaxAttempts = 1
}
require.NoError( require.NoError(
t, t,
@@ -107,89 +108,59 @@ func serveSourceDetailPage(
return w return w
} }
// TestHandleSourceDetail_MasksSlackWebhookURL is the // TestHandleSourceDetail_MasksTargetURL is the load-bearing
// load-bearing regression test for the credential leak: the // regression test for the credential leak: the rendered page must
// rendered page must show the Slack target without any of the // show a Slack target, and an http target, without any of the secret
// secret path segments of its webhook URL. // path segments of its URL. An http target's destination is routinely
func TestHandleSourceDetail_MasksSlackWebhookURL(t *testing.T) { // an incoming-webhook endpoint too, so the same Slack URL serves for
// both.
func TestHandleSourceDetail_MasksTargetURL(t *testing.T) {
t.Parallel() t.Parallel()
var ( labels := map[database.TargetType]string{
h *handlers.Handlers database.TargetTypeSlack: "Webhook URL",
sess *session.Session database.TargetTypeHTTP: "Destination URL",
db *database.Database }
)
app := newTestApp(t, &h, &sess, &db) for targetType, label := range labels {
app.RequireStart() t.Run(string(targetType), func(t *testing.T) {
t.Parallel()
t.Cleanup(app.RequireStop) var (
h *handlers.Handlers
sess *session.Session
db *database.Database
)
wh := seedWebhook(t, db) app := newTestApp(t, &h, &sess, &db)
seedConfiguredTarget( app.RequireStart()
t, db, wh.ID,
database.TargetTypeSlack,
`{"webhookUrl":"`+slackWebhookURL+`"}`,
)
body := renderSourceDetailPage(t, h, sess, wh.ID) t.Cleanup(app.RequireStop)
assert.NotContains(t, body, slackSecretPath) wh := seedWebhook(t, db)
assert.NotContains(t, body, "T00000000") seedConfiguredTarget(
assert.NotContains(t, body, "B00000000") t, db, wh.ID, targetType,
assert.NotContains( database.Target{URL: slackWebhookURL},
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX", )
)
assert.NotContains(t, body, "webhookUrl")
assert.Contains(t, body, "Webhook URL") body := renderSourceDetailPage(t, h, sess, wh.ID)
assert.Contains(t, body, "https://hooks.slack.com/...")
}
// TestHandleSourceDetail_MasksHTTPDestinationURL is the assert.NotContains(t, body, slackSecretPath)
// regression test for the same leak reached through the http assert.NotContains(t, body, "T00000000")
// target: its destination is routinely an incoming-webhook assert.NotContains(t, body, "B00000000")
// endpoint whose path segments are the credential, so the assert.NotContains(
// rendered page must not contain them. t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
func TestHandleSourceDetail_MasksHTTPDestinationURL( )
t *testing.T,
) {
t.Parallel()
var ( assert.Contains(t, body, label)
h *handlers.Handlers assert.Contains(t, body, "https://hooks.slack.com/...")
sess *session.Session })
db *database.Database }
)
app := newTestApp(t, &h, &sess, &db)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
seedConfiguredTarget(
t, db, wh.ID,
database.TargetTypeHTTP,
`{"url":"`+slackWebhookURL+`"}`,
)
body := renderSourceDetailPage(t, h, sess, wh.ID)
assert.NotContains(t, body, slackSecretPath)
assert.NotContains(t, body, "T00000000")
assert.NotContains(t, body, "B00000000")
assert.NotContains(
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
)
assert.Contains(t, body, "Destination URL")
assert.Contains(t, body, "https://hooks.slack.com/...")
} }
// TestHandleSourceDetail_RendersNamedTargetFields proves the // TestHandleSourceDetail_RendersNamedTargetFields proves the
// other target types render labelled fields rather than the // other target types render labelled fields rather than the
// stored blob. // stored values.
func TestHandleSourceDetail_RendersNamedTargetFields( func TestHandleSourceDetail_RendersNamedTargetFields(
t *testing.T, t *testing.T,
) { ) {
@@ -211,18 +182,24 @@ func TestHandleSourceDetail_RendersNamedTargetFields(
seedConfiguredTarget( seedConfiguredTarget(
t, db, wh.ID, t, db, wh.ID,
database.TargetTypeHTTP, database.TargetTypeHTTP,
`{"url":"https://example.com/hook","timeout":30,`+ database.Target{
`"headers":{"Authorization":"Bearer sekrit"}}`, URL: "https://example.com/hook",
TimeoutSeconds: 30,
Headers: map[string]string{"X-Api-Key": "sekrit"},
},
) )
seedConfiguredTarget( seedConfiguredTarget(
t, db, wh.ID, t, db, wh.ID,
database.TargetTypeDatabase, database.TargetTypeDatabase,
`{"expiry":"720h"}`, database.Target{
ArchiveExpiry: 720 * time.Hour,
ArchiveRotation: rotationNone,
},
) )
seedConfiguredTarget( seedConfiguredTarget(
t, db, wh.ID, t, db, wh.ID,
database.TargetType("carrier-pigeon"), database.TargetType("carrier-pigeon"),
`{"beak":"sharp"}`, database.Target{URL: "https://pigeon.example/beak"},
) )
body := renderSourceDetailPage(t, h, sess, wh.ID) body := renderSourceDetailPage(t, h, sess, wh.ID)
@@ -241,8 +218,8 @@ func TestHandleSourceDetail_RendersNamedTargetFields(
assert.Contains(t, list, "Archive rotation: none") assert.Contains(t, list, "Archive rotation: none")
assert.Contains(t, body, `<option value="database">Archive</option>`) assert.Contains(t, body, `<option value="database">Archive</option>`)
// An unknown type gets the neutral placeholder, never the // An unknown type gets the neutral placeholder, never its
// stored blob. // stored URL.
assert.Contains(t, body, "(unavailable)") assert.Contains(t, body, "(unavailable)")
assert.NotContains(t, body, "beak") assert.NotContains(t, body, "beak")
} }
@@ -273,7 +250,7 @@ func TestHandleSourceDetail_FitsWideAndNarrowWindows(t *testing.T) {
assert.Contains( assert.Contains(
t, body, t, body,
`<div class="mx-auto px-6 py-8" style="max-width: 108rem"`, `<div class="max-w-[108rem] mx-auto px-6 py-8">`,
) )
assert.Contains( assert.Contains(
t, body, t, body,
@@ -286,8 +263,8 @@ func TestHandleSourceDetail_FitsWideAndNarrowWindows(t *testing.T) {
// target and says what deleting it loses, that the webhook's gives its // target and says what deleting it loses, that the webhook's gives its
// number of stored events (5 received, 2 removed by retention, so 3, // number of stored events (5 received, 2 removed by retention, so 3,
// the statistics pane's "Within retention" figure), and that an // the statistics pane's "Within retention" figure), and that an
// entrypoint with no description is named by its URL. The template // entrypoint with no description is named by its URL. Each prompt is
// writes the slashes after http: as \/, which the browser reads as /. // its form's data-confirm attribute, where &#10; is a line break.
func TestHandleSourceDetail_DeletePromptsNameWhatIsLost(t *testing.T) { func TestHandleSourceDetail_DeletePromptsNameWhatIsLost(t *testing.T) {
t.Parallel() t.Parallel()
@@ -325,29 +302,28 @@ func TestHandleSourceDetail_DeletePromptsNameWhatIsLost(t *testing.T) {
body := renderSourceDetailPage(t, h, sess, wh.ID) body := renderSourceDetailPage(t, h, sess, wh.ID)
assert.Contains(t, body, assert.Contains(t, body,
`Delete webhook &quot;delete-me&quot;?\n\n`+ `data-confirm="Delete webhook &quot;delete-me&quot;?&#10;&#10;`+
`This deletes its stored events (3) and their deliveries. `+ `This deletes its stored events (3) and their deliveries. `+
`Any archive files it wrote are kept.`) `Any archive files it wrote are kept."`)
assert.Contains(t, body, assert.Contains(t, body,
`Delete entrypoint &quot;Stripe&quot;?\n\n`+ `data-confirm="Delete entrypoint &quot;Stripe&quot;?&#10;&#10;`+
`Senders using its URL get an error from now on, `+ `Senders using its URL get an error from now on, `+
`and the URL cannot be restored.`) `and the URL cannot be restored."`)
assert.Contains(t, body, assert.Contains(t, body,
`Delete entrypoint &quot;http:\/\/example.com/h/`+ `data-confirm="Delete entrypoint &quot;http://example.com/h/`+
unnamed.Path+`&quot;?`) unnamed.Path+`&quot;?&#10;&#10;`)
assert.Contains(t, body, assert.Contains(t, body,
`Delete target &quot;t-log&quot;?\n\n`+ `data-confirm="Delete target &quot;t-log&quot;?&#10;&#10;`+
`Nothing more is delivered to it. `+ `Nothing more is delivered to it. `+
`Its past deliveries stay in the event log.`) `Its past deliveries stay in the event log."`)
} }
// TestHandleSourceDetail_DeletePromptKeepsQuotesInName checks that a // TestHandleSourceDetail_DeletePromptKeepsQuotesInName checks that a
// webhook name with quotes, a backslash, a closing script tag and a // webhook name with quotes, a backslash, a closing script tag and a
// newline reaches its delete prompt escaped for the script, which the // newline reaches its delete prompt escaped for the attribute, which
// browser reads back as the name typed: each quote and angle bracket // the browser reads back as the name typed: each quote and angle
// as a \u escape, the slash as \/, the newline as \n and the backslash // bracket as a character reference, the rest as typed. An unescaped
// doubled. An unescaped newline would break the prompt's script, and // double quote would end the attribute and cut the prompt short.
// the form would then submit without asking.
func TestHandleSourceDetail_DeletePromptKeepsQuotesInName(t *testing.T) { func TestHandleSourceDetail_DeletePromptKeepsQuotesInName(t *testing.T) {
t.Parallel() t.Parallel()
@@ -373,6 +349,6 @@ func TestHandleSourceDetail_DeletePromptKeepsQuotesInName(t *testing.T) {
body := renderSourceDetailPage(t, h, sess, wh.ID) body := renderSourceDetailPage(t, h, sess, wh.ID)
assert.Contains(t, body, assert.Contains(t, body,
"Delete webhook &quot;Bob\\u0027s \\u0022best\\u0022 \\\\ hook"+ "Delete webhook &quot;Bob&#39;s &#34;best&#34; \\ hook"+
"\\u003c\\/script\\u003e\\nline two&quot;?") "&lt;/script&gt;\nline two&quot;?")
} }
@@ -161,7 +161,7 @@ func TestHandleSourceLogs_MasksDeletedTargetConfig(t *testing.T) {
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, t, db, wh.ID,
database.TargetTypeSlack, database.TargetTypeSlack,
`{"webhookUrl":"`+slackWebhookURL+`"}`, database.Target{URL: slackWebhookURL},
) )
seedDeliveredEvent(t, dbMgr, wh.ID, tgt.ID) seedDeliveredEvent(t, dbMgr, wh.ID, tgt.ID)
@@ -175,7 +175,6 @@ func TestHandleSourceLogs_MasksDeletedTargetConfig(t *testing.T) {
assert.NotContains( assert.NotContains(
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX", t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
) )
assert.NotContains(t, body, "webhookUrl")
// The name is there; only the credential is not. // The name is there; only the credential is not.
assert.Contains(t, body, tgt.Name+deletedMarker) assert.Contains(t, body, tgt.Name+deletedMarker)
+1 -2
View File
@@ -138,7 +138,7 @@ func TestHandleSourceLogs_MasksSlackWebhookURL(t *testing.T) {
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, t, db, wh.ID,
database.TargetTypeSlack, database.TargetTypeSlack,
`{"webhookUrl":"`+slackWebhookURL+`"}`, database.Target{URL: slackWebhookURL},
) )
seedDeliveredEvent(t, dbMgr, wh.ID, tgt.ID) seedDeliveredEvent(t, dbMgr, wh.ID, tgt.ID)
@@ -151,7 +151,6 @@ func TestHandleSourceLogs_MasksSlackWebhookURL(t *testing.T) {
assert.NotContains( assert.NotContains(
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX", t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
) )
assert.NotContains(t, body, "webhookUrl")
// The page still identifies the delivery's target. // The page still identifies the delivery's target.
assert.Contains(t, body, tgt.Name) assert.Contains(t, body, tgt.Name)
+64 -103
View File
@@ -137,8 +137,7 @@ func onlyWebhook(
} }
// seedWebhookWithRetention inserts a webhook owned by the test user // seedWebhookWithRetention inserts a webhook owned by the test user
// with an exact stored retention value, bypassing Webhook.BeforeSave // with the given retention.
// via a column-level update so that legacy rows can be planted too.
func seedWebhookWithRetention( func seedWebhookWithRetention(
t *testing.T, t *testing.T,
db *database.Database, db *database.Database,
@@ -155,13 +154,6 @@ func seedWebhookWithRetention(
t, t,
db.DB().Omit(clause.Associations).Create(wh).Error, db.DB().Omit(clause.Associations).Create(wh).Error,
) )
require.NoError(
t,
db.DB().Model(wh).
Update("retention_days", retentionDays).Error,
)
wh.RetentionDays = retentionDays
return *wh return *wh
} }
@@ -225,10 +217,9 @@ func setupSourceTest(t *testing.T) *sourceTestEnv {
} }
} }
// TestHandleSourceCreateSubmit_ZeroRetentionPersistsForever is the core // TestHandleSourceCreateSubmit_ZeroRetentionPersistsForever checks
// regression test for the bug: the create form's 0 must reach the // that the create form's 0 reaches the database as 0, which means
// database as the retain-forever sentinel rather than being replaced by // retain forever.
// the column's default of 30.
func TestHandleSourceCreateSubmit_ZeroRetentionPersistsForever( func TestHandleSourceCreateSubmit_ZeroRetentionPersistsForever(
t *testing.T, t *testing.T,
) { ) {
@@ -241,35 +232,60 @@ func TestHandleSourceCreateSubmit_ZeroRetentionPersistsForever(
require.Equal(t, http.StatusSeeOther, w.Code) require.Equal(t, http.StatusSeeOther, w.Code)
wh := onlyWebhook(t, env.db) wh := onlyWebhook(t, env.db)
assert.Equal( assert.Equal(t, 0, storedRetentionDays(t, env.db, wh.ID))
t,
database.RetentionForeverDays,
storedRetentionDays(t, env.db, wh.ID),
)
assert.True(t, wh.RetainsForever()) assert.True(t, wh.RetainsForever())
} }
func TestHandleSourceCreateSubmit_OmittedRetentionUsesDefault( func TestHandleSourceCreateSubmit_DayCountIsStored(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
days := "45"
w := submitCreate(t, env.handlers, env.cookies, "finite", &days)
require.Equal(t, http.StatusSeeOther, w.Code)
wh := onlyWebhook(t, env.db)
assert.Equal(t, 45, storedRetentionDays(t, env.db, wh.ID))
}
// TestHandleSourceCreateSubmit_BlankRetentionUsesDefault covers a
// retention field left empty and one missing from the form: both
// give the default, not forever.
func TestHandleSourceCreateSubmit_BlankRetentionUsesDefault(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
env := setupSourceTest(t) blank := ""
w := submitCreate(t, env.handlers, env.cookies, "defaulted", nil) for name, retention := range map[string]*string{
require.Equal(t, http.StatusSeeOther, w.Code) "blank": &blank,
"omitted": nil,
} {
t.Run(name, func(t *testing.T) {
t.Parallel()
wh := onlyWebhook(t, env.db) env := setupSourceTest(t)
assert.Equal(
t, w := submitCreate(
database.DefaultRetentionDays, t, env.handlers, env.cookies, "defaulted", retention,
storedRetentionDays(t, env.db, wh.ID), )
) require.Equal(t, http.StatusSeeOther, w.Code)
wh := onlyWebhook(t, env.db)
assert.Equal(
t,
database.DefaultRetentionDays,
storedRetentionDays(t, env.db, wh.ID),
)
})
}
} }
// TestHandleSourceCreate_PrefillsDefaultFromConstant keeps the create // TestHandleSourceCreate_PrefillsDefaultFromConstant checks that the
// form's pre-filled retention from becoming a third hardcoded copy of // create form pre-fills its retention from database.DefaultRetentionDays
// the 30-day policy. // rather than from a hardcoded 30.
func TestHandleSourceCreate_PrefillsDefaultFromConstant(t *testing.T) { func TestHandleSourceCreate_PrefillsDefaultFromConstant(t *testing.T) {
t.Parallel() t.Parallel()
@@ -290,7 +306,7 @@ func TestHandleSourceCreate_PrefillsDefaultFromConstant(t *testing.T) {
) )
assert.NotContains( assert.NotContains(
t, body, `max="365"`, t, body, `max="365"`,
"a max below the sentinel would block retain-forever", "a max of 365 would block any longer retention",
) )
assert.Contains(t, body, `min="0"`) assert.Contains(t, body, `min="0"`)
} }
@@ -368,40 +384,6 @@ func TestHandleSourceCreateSubmit_OverflowingRetentionIsRejected(
) )
} }
// TestHandleSourceCreateSubmit_SentinelIsAcceptedAsForever guards the
// boundary between "too large to represent" and "retain forever": the
// sentinel is above MaxFiniteRetentionDays, but it is the value the
// edit form pre-fills, so it must be accepted rather than rejected as
// out of range. A value above the sentinel is stored as the sentinel.
func TestHandleSourceCreateSubmit_SentinelIsAcceptedAsForever(
t *testing.T,
) {
t.Parallel()
for _, days := range []int{
database.RetentionForeverDays,
database.RetentionForeverDays + 1,
} {
raw := strconv.Itoa(days)
t.Run(raw, func(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
w := submitCreate(t, env.handlers, env.cookies, "forever", &raw)
require.Equal(t, http.StatusSeeOther, w.Code)
wh := onlyWebhook(t, env.db)
assert.Equal(
t,
database.RetentionForeverDays,
storedRetentionDays(t, env.db, wh.ID),
)
})
}
}
// TestHandleSourceCreateSubmit_RejectedFormKeepsUserInput checks that a // TestHandleSourceCreateSubmit_RejectedFormKeepsUserInput checks that a
// validation failure hands the user's typing back, matching what the // validation failure hands the user's typing back, matching what the
// edit form already does. Losing a long description to a mistyped // edit form already does. Losing a long description to a mistyped
@@ -480,11 +462,7 @@ func TestHandleSourceEditSubmit_ZeroRetentionPersistsForever(
w := submitEdit(t, env, wh, "0") w := submitEdit(t, env, wh, "0")
require.Equal(t, http.StatusSeeOther, w.Code) require.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal( assert.Equal(t, 0, storedRetentionDays(t, env.db, wh.ID))
t,
database.RetentionForeverDays,
storedRetentionDays(t, env.db, wh.ID),
)
} }
func TestHandleSourceEditSubmit_InvalidRetentionIsRejected( func TestHandleSourceEditSubmit_InvalidRetentionIsRejected(
@@ -867,18 +845,15 @@ func TestHandleSourceEditSubmit_ArchiveNameTaken(t *testing.T) {
assert.Equal(t, oldName, stored.Name) assert.Equal(t, oldName, stored.Name)
} }
// TestSourceEditForm_ForeverWebhookRoundTrips walks the exact path that // TestSourceEditForm_ForeverWebhookRoundTrips renders the edit form for
// the removed max="365" cap used to break: render the edit form for a // a retain-forever webhook, confirms it pre-fills 0, then submits that
// retain-forever webhook, confirm the pre-filled sentinel is not capped // pre-filled value straight back and confirms the retention policy
// by browser validation, then submit that pre-filled value straight // survives untouched.
// back and confirm the retention policy survives untouched.
func TestSourceEditForm_ForeverWebhookRoundTrips(t *testing.T) { func TestSourceEditForm_ForeverWebhookRoundTrips(t *testing.T) {
t.Parallel() t.Parallel()
env := setupSourceTest(t) env := setupSourceTest(t)
wh := seedWebhookWithRetention( wh := seedWebhookWithRetention(t, env.db, 0)
t, env.db, database.RetentionForeverDays,
)
req := getRequest( req := getRequest(
t, "/hook/"+wh.ID+"/edit", env.cookies, t, "/hook/"+wh.ID+"/edit", env.cookies,
@@ -889,17 +864,12 @@ func TestSourceEditForm_ForeverWebhookRoundTrips(t *testing.T) {
require.Equal(t, http.StatusOK, w.Code) require.Equal(t, http.StatusOK, w.Code)
sentinel := strconv.Itoa(database.RetentionForeverDays)
body := w.Body.String() body := w.Body.String()
assert.Contains( assert.Contains(
t, body, `value="`+sentinel+`"`, t, body, `name="retention_days" value="0"`,
"the edit form pre-fills the stored retention", "the edit form pre-fills the stored retention",
) )
assert.NotContains(
t, body, `max="365"`,
"a max below the sentinel would block saving any edit",
)
// "Currently forever." is the rendered RetentionLabel, not the // "Currently forever." is the rendered RetentionLabel, not the
// static hint below the input, which says "Enter 0 to retain events // static hint below the input, which says "Enter 0 to retain events
// forever." A bare Contains of "forever" would pass for any // forever." A bare Contains of "forever" would pass for any
@@ -910,29 +880,20 @@ func TestSourceEditForm_ForeverWebhookRoundTrips(t *testing.T) {
) )
// Submit the pre-filled value back, exactly as a browser would. // Submit the pre-filled value back, exactly as a browser would.
post := submitEdit(t, env, wh, sentinel) post := submitEdit(t, env, wh, "0")
require.Equal(t, http.StatusSeeOther, post.Code) require.Equal(t, http.StatusSeeOther, post.Code)
assert.Equal( assert.Equal(t, 0, storedRetentionDays(t, env.db, wh.ID))
t,
database.RetentionForeverDays,
storedRetentionDays(t, env.db, wh.ID),
)
} }
// TestSourceListAndDetail_ShowForeverNotTheSentinelNumber checks that // TestSourceListAndDetail_ShowForeverNotZeroDays checks that a
// the retain-forever value is never rendered to the user as a raw day // retention of 0 is shown as forever, never as a day count, on either
// count on either read-only view. // read-only view.
func TestSourceListAndDetail_ShowForeverNotTheSentinelNumber( func TestSourceListAndDetail_ShowForeverNotZeroDays(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
env := setupSourceTest(t) env := setupSourceTest(t)
wh := seedWebhookWithRetention( wh := seedWebhookWithRetention(t, env.db, 0)
t, env.db, database.RetentionForeverDays,
)
sentinel := strconv.Itoa(database.RetentionForeverDays)
listW := httptest.NewRecorder() listW := httptest.NewRecorder()
env.handlers.HandleSourceList().ServeHTTP( env.handlers.HandleSourceList().ServeHTTP(
@@ -941,7 +902,7 @@ func TestSourceListAndDetail_ShowForeverNotTheSentinelNumber(
require.Equal(t, http.StatusOK, listW.Code) require.Equal(t, http.StatusOK, listW.Code)
assert.Contains(t, listW.Body.String(), "Retention: forever") assert.Contains(t, listW.Body.String(), "Retention: forever")
assert.NotContains(t, listW.Body.String(), sentinel) assert.NotContains(t, listW.Body.String(), "Retention: 0 days")
detailW := httptest.NewRecorder() detailW := httptest.NewRecorder()
env.handlers.HandleSourceDetail().ServeHTTP( env.handlers.HandleSourceDetail().ServeHTTP(
@@ -954,5 +915,5 @@ func TestSourceListAndDetail_ShowForeverNotTheSentinelNumber(
require.Equal(t, http.StatusOK, detailW.Code) require.Equal(t, http.StatusOK, detailW.Code)
assert.Contains(t, detailW.Body.String(), "Retention: forever") assert.Contains(t, detailW.Body.String(), "Retention: forever")
assert.NotContains(t, detailW.Body.String(), sentinel) assert.NotContains(t, detailW.Body.String(), "Retention: 0 days")
} }
+81
View File
@@ -0,0 +1,81 @@
package handlers
import (
"errors"
"strconv"
"strings"
)
// maxTargetAttempts bounds a target's max_attempts.
//
// Both target forms already declare max="20" on the input, so this
// enforces server-side what the UI has always advertised rather than
// introducing a new limit.
//
// The number is not cosmetic. Every attempt writes a delivery_results
// row that the event log then loads and renders, and the engine backs
// off by 2^(n-1) seconds, so attempt 20 is already about six days
// after the first. A value beyond this buys no additional durability
// and only costs rows.
const maxTargetAttempts = 20
// Errors returned when a max_attempts form value cannot be turned into
// a number of attempts.
var (
// errAttemptsInvalid signals a max_attempts form value that is not
// a whole number of at least 1.
errAttemptsInvalid = errors.New(
"must be a whole number, at least 1",
)
// errAttemptsTooLarge signals a max_attempts form value that is a
// whole number but above maxTargetAttempts. It is distinguished
// from errAttemptsInvalid so the message can name the ceiling
// instead of implying the input was not a number.
errAttemptsTooLarge = errors.New("out of range")
)
// parseMaxAttempts interprets a max_attempts form value.
//
// An ABSENT value — the field empty or not submitted — yields
// fallback, which lets the create path apply its default and the edit
// path leave the stored value alone. A value that is SET BUT INVALID
// is an error: unparseable, below 1, or above maxTargetAttempts.
//
// The distinction is the whole point of this function. Returning a
// default for input the operator typed but that did not parse would
// silently turn retries off on a store-and-forward proxy — and on the
// edit path destroy a working configuration over a typo. A default
// answers a question that was not asked; it never answers one that
// was asked badly.
func parseMaxAttempts(raw string, fallback int) (int, error) {
raw = strings.TrimSpace(raw)
if raw == "" {
return fallback, nil
}
v, err := strconv.Atoi(raw)
if err != nil || v < 1 {
return 0, errAttemptsInvalid
}
if v > maxTargetAttempts {
return 0, errAttemptsTooLarge
}
return v, nil
}
// attemptsErrorMessage returns the message the create and edit forms
// show for a rejected max_attempts value. Any error other than
// errAttemptsTooLarge falls back to the generic wording, so an
// unrecognised parse failure still produces a sensible 400.
func attemptsErrorMessage(err error) string {
if errors.Is(err, errAttemptsTooLarge) {
return errAttemptsTooLarge.Error() +
": at most " + strconv.Itoa(maxTargetAttempts) +
" attempts"
}
return errAttemptsInvalid.Error()
}
+429
View File
@@ -0,0 +1,429 @@
package handlers_test
import (
"net/http"
"net/url"
"strconv"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/handlers"
)
// attemptsTargetURL is the destination the attempts-validation targets
// point at. It is a literal public address rather than a hostname so
// the SSRF check resolves nothing and a sandbox without DNS cannot
// make these cases pass or fail for the wrong reason.
const attemptsTargetURL = "https://93.184.216.34/hooks/attempts"
const (
// wayAboveCeiling is the typo'd-extra-zero case from the report.
wayAboveCeiling = "999999999"
// notANumber is the plainest garbage an operator can type, and
// the value the report submitted on the edit form.
notANumber = "abc"
// workingAttempts is the number of attempts a seeded target is
// already delivering with, which a rejected submission must not
// disturb.
workingAttempts = 2
)
// aboveCeiling is the smallest rejected whole number.
func aboveCeiling() string {
return strconv.Itoa(handlers.MaxTargetAttemptsForTest + 1)
}
// overCeilingAttempts is whole-number input past the limit, which is
// rejected with the limit named.
func overCeilingAttempts() []string {
return []string{aboveCeiling(), wayAboveCeiling}
}
// belowOneAttempts is whole-number input under the floor of one
// attempt, which is rejected with the floor named.
func belowOneAttempts() []string {
return []string{"0", "-5"}
}
// unparseableAttempts is input an operator can type into the field
// that is not a number of attempts. Each must be REJECTED: silently
// reading any of them as a default turns retries off on a
// store-and-forward proxy without saying so.
//
// The twenty-digit case is here because it parses as digits but
// overflows int, which is the one failure the field's own min/max
// attributes cannot describe.
func unparseableAttempts() []string {
return []string{
notANumber,
"2.7",
"12345678901234567890",
"1e3",
}
}
// createAttemptsForm is a complete, otherwise-valid HTTP target
// creation, so the only thing any case below varies is max_attempts.
func createAttemptsForm(attempts string) url.Values {
form := url.Values{}
form.Set("name", "attempts-target")
form.Set("type", string(database.TargetTypeHTTP))
form.Set("url", attemptsTargetURL)
if attempts != absentField {
form.Set("max_attempts", attempts)
}
return form
}
// absentField marks a field the form does not submit at all, which is
// the case that legitimately takes a default and must stay distinct
// from a field submitted with garbage in it.
const absentField = "\x00absent"
// absentAttempts is every way of saying "the operator did not set
// this", each of which takes the default rather than a 400. Blank and
// whitespace-only count as absent here because they do in the timeout
// and retention controls on the same forms; a rule the fields do not
// share would be its own surprise.
func absentAttempts() []string {
return []string{absentField, "", " "}
}
// createWithAttempts posts the target create form for a fresh webhook
// and returns the webhook and the response.
func createWithAttempts(
t *testing.T,
env *sourceTestEnv,
attempts string,
) (database.Webhook, int, string) {
t.Helper()
webhook := seedWebhookWithRetention(t, env.db, 30)
w := serveTarget(
env, http.MethodPost,
"/hook/"+webhook.ID+"/targets",
createAttemptsForm(attempts),
)
return webhook, w.Code, w.Body.String()
}
// TestTargetCreate_AttemptsAboveCeilingRejected proves the create form
// enforces a ceiling at all, and that the 400 names it — a rejection
// that does not say what the limit is leaves the operator guessing.
func TestTargetCreate_AttemptsAboveCeilingRejected(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
ceiling := strconv.Itoa(handlers.MaxTargetAttemptsForTest)
for _, attempts := range overCeilingAttempts() {
webhook, code, body := createWithAttempts(t, env, attempts)
assert.Equal(t, http.StatusBadRequest, code,
"max_attempts=%s should be rejected", attempts)
assert.Contains(t, body, ceiling,
"the rejection for %s should name the ceiling",
attempts)
assert.Empty(t,
targetsForWebhook(t, env.db, webhook.ID),
"no target should be created for %s", attempts)
}
}
// TestTargetCreate_AttemptsBelowOneRejected proves the create form
// refuses 0 and less, naming the floor: every delivery is attempted
// at least once.
func TestTargetCreate_AttemptsBelowOneRejected(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
for _, attempts := range belowOneAttempts() {
webhook, code, body := createWithAttempts(t, env, attempts)
assert.Equal(t, http.StatusBadRequest, code,
"max_attempts=%s should be rejected", attempts)
assert.Contains(t, body, "at least 1",
"the rejection for %s should name the floor",
attempts)
assert.Empty(t,
targetsForWebhook(t, env.db, webhook.ID),
"no target should be created for %s", attempts)
}
}
// TestTargetCreate_UnparseableAttemptsRejected is the core of the
// defect: each of these was accepted with HTTP 200 and stored as a
// default.
func TestTargetCreate_UnparseableAttemptsRejected(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
for _, attempts := range unparseableAttempts() {
webhook, code, body := createWithAttempts(t, env, attempts)
assert.Equal(t, http.StatusBadRequest, code,
"max_attempts=%q should be rejected, not coerced",
attempts)
assert.Contains(t, body, "whole number",
"the rejection for %q should say why", attempts)
assert.Empty(t,
targetsForWebhook(t, env.db, webhook.ID),
"no target should be created for %q", attempts)
}
}
// TestTargetCreate_ValidAttemptsStored covers the accepting half,
// including both bounds: a bound that rejects its own limit would
// make the advertised range unreachable.
func TestTargetCreate_ValidAttemptsStored(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
for _, want := range []int{1, 3, handlers.MaxTargetAttemptsForTest} {
webhook, code, body := createWithAttempts(
t, env, strconv.Itoa(want),
)
require.Equal(t, http.StatusSeeOther, code, body)
targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1)
assert.Equal(t, want, targets[0].MaxAttempts)
}
}
// TestTargetCreate_AbsentAttemptsTakesDefault keeps the two cases
// distinct. An omitted field is not an operator asking for something
// invalid, so it still gets the default of one attempt rather than a
// 400 — otherwise the fix above would make the form unusable.
func TestTargetCreate_AbsentAttemptsTakesDefault(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
for _, attempts := range absentAttempts() {
webhook, code, body := createWithAttempts(t, env, attempts)
require.Equal(t, http.StatusSeeOther, code, body)
targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1)
assert.Equal(t, 1, targets[0].MaxAttempts,
"an absent max_attempts should take the default")
}
}
// seedAttemptsTarget creates an HTTP target already delivering with
// workingAttempts attempts, through the real create handler.
func seedAttemptsTarget(
t *testing.T,
env *sourceTestEnv,
) (database.Webhook, database.Target) {
t.Helper()
webhook, code, body := createWithAttempts(
t, env, strconv.Itoa(workingAttempts),
)
require.Equal(t, http.StatusSeeOther, code, body)
targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1)
require.Equal(t, workingAttempts, targets[0].MaxAttempts)
return webhook, targets[0]
}
// editAttemptsForm is a complete edit submission that changes the
// target's name as well, so a rejected submission can be shown to
// have written nothing at all rather than merely to have left
// max_attempts alone.
func editAttemptsForm(attempts string) url.Values {
form := url.Values{}
form.Set("name", "renamed-by-edit")
form.Set("url", attemptsTargetURL)
if attempts != absentField {
form.Set("max_attempts", attempts)
}
return form
}
// assertEditRejectedAndUnchanged submits an edit expected to fail and
// checks both halves of the requirement: the 400 explains itself, and
// the target it was submitted against is untouched.
func assertEditRejectedAndUnchanged(
t *testing.T,
env *sourceTestEnv,
attempts, wantReason string,
) {
t.Helper()
webhook, target := seedAttemptsTarget(t, env)
w := submitTargetEdit(
env, webhook.ID, target.ID, editAttemptsForm(attempts),
)
assert.Equal(t, http.StatusBadRequest, w.Code,
"max_attempts=%q should be rejected on edit", attempts)
assert.Contains(t, w.Body.String(), wantReason,
"the rejection for %q should say why", attempts)
stored := storedTarget(t, env, target.ID)
assert.Equal(t, workingAttempts, stored.MaxAttempts,
"a rejected edit must not destroy the working number "+
"of attempts with %q", attempts)
assert.Equal(t, "attempts-target", stored.Name,
"a rejected edit must write nothing at all")
}
// TestTargetEdit_UnparseableAttemptsRejected is the damaging half of
// the defect. A target delivering with retries, re-saved with a typo
// in the field, returned 200 and was left with retries disabled and
// nothing said.
func TestTargetEdit_UnparseableAttemptsRejected(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
for _, attempts := range unparseableAttempts() {
assertEditRejectedAndUnchanged(
t, env, attempts, "whole number",
)
}
}
// TestTargetEdit_AttemptsBelowOneRejected proves the floor applies
// to the edit path too.
func TestTargetEdit_AttemptsBelowOneRejected(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
for _, attempts := range belowOneAttempts() {
assertEditRejectedAndUnchanged(t, env, attempts, "at least 1")
}
}
// TestTargetEdit_AttemptsAboveCeilingRejected proves the ceiling
// applies to the edit path too, naming itself, so the two paths
// cannot disagree about what is storable.
func TestTargetEdit_AttemptsAboveCeilingRejected(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
ceiling := strconv.Itoa(handlers.MaxTargetAttemptsForTest)
for _, attempts := range overCeilingAttempts() {
assertEditRejectedAndUnchanged(t, env, attempts, ceiling)
}
}
// TestTargetEdit_ValidAttemptsStored covers the accepting half of the
// edit path, so the bounds cannot be enforced by simply refusing
// every submission.
func TestTargetEdit_ValidAttemptsStored(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
for _, want := range []int{1, 9, handlers.MaxTargetAttemptsForTest} {
webhook, target := seedAttemptsTarget(t, env)
w := submitTargetEdit(
env, webhook.ID, target.ID,
editAttemptsForm(strconv.Itoa(want)),
)
require.Equal(t,
http.StatusSeeOther, w.Code, w.Body.String(),
)
assert.Equal(t, want,
storedTarget(t, env, target.ID).MaxAttempts)
}
}
// TestTargetEdit_AbsentAttemptsLeavesStoredValue is the edit path's
// absent-versus-invalid case. The field is only offered by the forms
// for types that retry, so a submission without it must leave the
// stored count alone rather than be rejected or reset.
func TestTargetEdit_AbsentAttemptsLeavesStoredValue(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
for _, attempts := range absentAttempts() {
webhook, target := seedAttemptsTarget(t, env)
w := submitTargetEdit(
env, webhook.ID, target.ID,
editAttemptsForm(attempts),
)
require.Equal(t,
http.StatusSeeOther, w.Code, w.Body.String(),
)
assert.Equal(t, workingAttempts,
storedTarget(t, env, target.ID).MaxAttempts,
"an absent max_attempts must leave the stored "+
"count alone (%q)", attempts)
}
}
// TestTargetAttempts_CreateAndEditAgreeOnEveryCase proves the two
// paths cannot disagree, which is what let the create form and the
// edit form drift apart in the first place. Every input is submitted
// to both and the accept/reject verdicts are compared.
func TestTargetAttempts_CreateAndEditAgreeOnEveryCase(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
accepted := []string{
"1", "2",
strconv.Itoa(handlers.MaxTargetAttemptsForTest),
}
overCeiling := overCeilingAttempts()
belowOne := belowOneAttempts()
unparseable := unparseableAttempts()
cases := make(
[]string, 0,
len(accepted)+len(overCeiling)+len(belowOne)+
len(unparseable),
)
cases = append(cases, accepted...)
cases = append(cases, overCeiling...)
cases = append(cases, belowOne...)
cases = append(cases, unparseable...)
for _, attempts := range cases {
_, createCode, _ := createWithAttempts(t, env, attempts)
webhook, target := seedAttemptsTarget(t, env)
editCode := submitTargetEdit(
env, webhook.ID, target.ID,
editAttemptsForm(attempts),
).Code
assert.Equal(t,
createCode == http.StatusBadRequest,
editCode == http.StatusBadRequest,
"create and edit must agree on max_attempts=%q "+
"(create %d, edit %d)",
attempts, createCode, editCode,
)
}
}
+91 -123
View File
@@ -2,13 +2,13 @@ package handlers
import ( import (
"context" "context"
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"net/http" "net/http"
"strings" "strings"
"github.com/go-chi/chi" "github.com/go-chi/chi"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery" "sneak.berlin/go/webhooker/internal/delivery"
) )
@@ -27,8 +27,8 @@ func (h *Handlers) HandleTargetCreate() http.HandlerFunc {
sourceID := chi.URLParam(r, "sourceID") sourceID := chi.URLParam(r, "sourceID")
h.renameMu.Lock() h.archives.RenameLock().Lock()
defer h.renameMu.Unlock() defer h.archives.RenameLock().Unlock()
var webhook database.Webhook var webhook database.Webhook
@@ -64,20 +64,14 @@ func (h *Handlers) processTargetCreate(
) { ) {
in := targetFormInputFrom(r) in := targetFormInputFrom(r)
target, errMsg, err := h.newTarget(r.Context(), webhook.ID, in) target, errMsg := h.newTarget(r.Context(), webhook.ID, in)
if err != nil {
h.serverError(w, r, "failed to encode target config", err)
return
}
if errMsg != "" { if errMsg != "" {
h.renderSourceDetail(w, r, webhook, in, errMsg) h.renderSourceDetail(w, r, webhook, in, errMsg)
return return
} }
err = h.db.DB().Create(target).Error err := h.db.DB().Create(target).Error
if err != nil { if err != nil {
h.serverError(w, r, "failed to create target", err) h.serverError(w, r, "failed to create target", err)
@@ -92,66 +86,64 @@ func (h *Handlers) processTargetCreate(
// newTarget validates a new target for a webhook and returns the row // newTarget validates a new target for a webhook and returns the row
// to create, or, when it refuses the target, the message the form // to create, or, when it refuses the target, the message the form
// shows. An error is the server's fault, not a refusal: the accepted // shows. Every form that creates a target goes through here, so they
// configuration could not be encoded. Every form that creates a // all accept and refuse the same things.
// target goes through here, so they all accept and refuse the same
// things.
func (h *Handlers) newTarget( func (h *Handlers) newTarget(
ctx context.Context, ctx context.Context,
webhookID string, webhookID string,
in targetFormInput, in targetFormInput,
) (*database.Target, string, error) { ) (*database.Target, string) {
target := &database.Target{ target := &database.Target{
WebhookID: webhookID, WebhookID: webhookID,
Type: in.Type, Type: in.Type,
Active: true, Active: true,
MaxAttempts: 1,
} }
errMsg, err := h.setTargetFromForm(ctx, target, in) errMsg := h.setTargetFromForm(ctx, target, in)
if err != nil || errMsg != "" { if errMsg != "" {
return nil, errMsg, err return nil, errMsg
} }
return target, "", nil return target, ""
} }
// setTargetFromForm validates a target form against the target's type // setTargetFromForm validates a target form against the target's type
// and, when it accepts it, sets the target's name, configuration and // and sets the target's name, settings and number of delivery attempts
// retry count from it. It returns the message the form shows for // from it. It returns the message the form shows for anything it
// anything it refuses, an unknown type among them, and then leaves the // refuses, an unknown type among them; the caller then discards the
// target unchanged; an error is the server's fault, as for newTarget. // target, which may be partly set.
// The add target form and the target edit form both go through here, // The add target form and the target edit form both go through here,
// so the two cannot come to disagree about what a target may be. // so the two cannot come to disagree about what a target may be.
func (h *Handlers) setTargetFromForm( func (h *Handlers) setTargetFromForm(
ctx context.Context, ctx context.Context,
target *database.Target, target *database.Target,
in targetFormInput, in targetFormInput,
) (string, error) { ) string {
if in.Name == "" { if in.Name == "" {
return "Name is required", nil return "Name is required"
} }
configJSON, errMsg, err := h.buildTargetConfig(ctx, target.Type, in) errMsg := h.setTargetSettings(ctx, target, in)
if err != nil || errMsg != "" { if errMsg != "" {
return errMsg, err return errMsg
} }
// An empty max_retries keeps the target's count: the // An empty max_attempts keeps the target's count: the default of
// fire-and-forget default of 0 for a new target, and the stored // 1 for a new target, and the stored count for an edited one,
// count for an edited one, since the forms for target types that // since the forms for target types that do not retry have no such
// do not retry have no such field. A value that is filled in but // field. A value that is filled in but invalid is refused rather
// invalid is refused rather than becoming that count, so a typo // than becoming that count, so a typo cannot destroy the count a
// cannot destroy the count a target is delivering with. // target is delivering with.
maxRetries, err := parseMaxRetries(in.MaxRetries, target.MaxRetries) maxAttempts, err := parseMaxAttempts(in.MaxAttempts, target.MaxAttempts)
if err != nil { if err != nil {
return "Invalid delivery attempts: " + retriesErrorMessage(err), nil return "Invalid delivery attempts: " + attemptsErrorMessage(err)
} }
target.Name = in.Name target.Name = in.Name
target.Config = configJSON target.MaxAttempts = maxAttempts
target.MaxRetries = maxRetries
return "", nil return ""
} }
// targetFormInput carries the raw values of a target form. Both the // targetFormInput carries the raw values of a target form. Both the
@@ -176,8 +168,8 @@ type targetFormInput struct {
// ForwardQuery is an HTTP target's checkbox that passes each // ForwardQuery is an HTTP target's checkbox that passes each
// event's query string on to it. // event's query string on to it.
ForwardQuery bool ForwardQuery bool
// MaxRetries is an HTTP or Slack target's max_retries. // MaxAttempts is an HTTP or Slack target's max_attempts.
MaxRetries string MaxAttempts string
// Expiry is a database (archive) target's row expiry. // Expiry is a database (archive) target's row expiry.
Expiry string Expiry string
// Rotation is a database (archive) target's rotation. // Rotation is a database (archive) target's rotation.
@@ -204,90 +196,87 @@ func targetFormInputFrom(r *http.Request) targetFormInput {
Headers: r.PostFormValue("headers"), Headers: r.PostFormValue("headers"),
Timeout: r.PostFormValue("timeout"), Timeout: r.PostFormValue("timeout"),
ForwardQuery: r.PostFormValue("forward_query") != "", ForwardQuery: r.PostFormValue("forward_query") != "",
MaxRetries: r.PostFormValue("max_retries"), MaxAttempts: r.PostFormValue("max_attempts"),
Expiry: r.PostFormValue("expiry"), Expiry: r.PostFormValue("expiry"),
Rotation: r.PostFormValue("rotation"), Rotation: r.PostFormValue("rotation"),
} }
} }
// buildTargetConfig builds the JSON config string for a target from // setTargetSettings sets a target's settings from the submitted form
// the submitted form values, or returns the message the form shows // values, or returns the message the form shows for a value it
// for a value it refuses. An error is the server's fault, not a // refuses. Which fields of in apply depends on the target type; a
// refusal: the accepted configuration could not be encoded. Which // type without a URL ignores any URL submitted, and the settings a
// fields of in apply depends on the target type; a type without a URL // type does not use are left empty.
// ignores any URL submitted. func (h *Handlers) setTargetSettings(
func (h *Handlers) buildTargetConfig(
ctx context.Context, ctx context.Context,
targetType database.TargetType, target *database.Target,
in targetFormInput, in targetFormInput,
) (string, string, error) { ) string {
switch targetType { switch target.Type {
case database.TargetTypeHTTP: case database.TargetTypeHTTP:
return h.buildHTTPTargetConfig(ctx, in) return h.setHTTPTargetSettings(ctx, target, in)
case database.TargetTypeSlack: case database.TargetTypeSlack:
return h.buildSlackTargetConfig(ctx, in.URL) return h.setSlackTargetSettings(ctx, target, in.URL)
case database.TargetTypeDatabase: case database.TargetTypeDatabase:
return buildDatabaseTargetConfig(in.Expiry, in.Rotation) return setDatabaseTargetSettings(target, in.Expiry, in.Rotation)
case database.TargetTypeLog: case database.TargetTypeLog:
return "", "", nil return ""
default: default:
return "", "Invalid target type", nil return "Invalid target type"
} }
} }
// buildHTTPTargetConfig builds config JSON for an HTTP target: an // setHTTPTargetSettings sets an HTTP target's SSRF-validated
// SSRF-validated destination plus the optional headers, timeout and // destination plus the optional headers, timeout and query string
// query string setting the delivery path honours. // setting the delivery path honours.
func (h *Handlers) buildHTTPTargetConfig( func (h *Handlers) setHTTPTargetSettings(
ctx context.Context, ctx context.Context,
target *database.Target,
in targetFormInput, in targetFormInput,
) (string, string, error) { ) string {
errMsg := h.validateTargetURL( errMsg := h.validateTargetURL(
ctx, in.URL, "URL is required for HTTP targets", ctx, in.URL, "URL is required for HTTP targets",
) )
if errMsg != "" { if errMsg != "" {
return "", errMsg, nil return errMsg
} }
headers, err := delivery.ParseTargetHeaders(in.Headers) headers, err := delivery.ParseTargetHeaders(in.Headers)
if err != nil { if err != nil {
return "", fmt.Sprintf("Invalid headers: %v", err), nil return fmt.Sprintf("Invalid headers: %v", err)
} }
timeout, err := delivery.ParseTargetTimeout(in.Timeout) timeout, err := delivery.ParseTargetTimeout(in.Timeout)
if err != nil { if err != nil {
return "", fmt.Sprintf("Invalid timeout: %v", err), nil return fmt.Sprintf("Invalid timeout: %v", err)
} }
configJSON, err := marshalTargetConfig(delivery.HTTPTargetConfig{ target.URL = in.URL
URL: in.URL, target.Headers = headers
Headers: headers, target.TimeoutSeconds = timeout
Timeout: timeout, target.ForwardQuery = in.ForwardQuery
ForwardQuery: in.ForwardQuery,
})
return configJSON, "", err return ""
} }
// buildSlackTargetConfig builds config JSON for a Slack target, // setSlackTargetSettings sets a Slack target's one setting, its
// whose whole configuration is one SSRF-validated webhook URL. // SSRF-validated webhook URL.
func (h *Handlers) buildSlackTargetConfig( func (h *Handlers) setSlackTargetSettings(
ctx context.Context, ctx context.Context,
target *database.Target,
targetURL string, targetURL string,
) (string, string, error) { ) string {
errMsg := h.validateTargetURL( errMsg := h.validateTargetURL(
ctx, targetURL, ctx, targetURL,
"Webhook URL is required for Slack targets", "Webhook URL is required for Slack targets",
) )
if errMsg != "" { if errMsg != "" {
return "", errMsg, nil return errMsg
} }
configJSON, err := marshalTargetConfig(delivery.SlackTargetConfig{ target.URL = targetURL
WebhookURL: targetURL,
})
return configJSON, "", err return ""
} }
// validateTargetURL refuses an empty or SSRF-blocked destination, // validateTargetURL refuses an empty or SSRF-blocked destination,
@@ -338,52 +327,31 @@ func (h *Handlers) validateTargetURL(
return "" return ""
} }
// marshalTargetConfig serialises a target configuration for storage. // setDatabaseTargetSettings sets a database (archive) target's
func marshalTargetConfig(cfg any) (string, error) { // expiry and rotation. Both are optional: an empty or never expiry
configBytes, err := json.Marshal(cfg) // keeps the archive forever, and an empty rotation is stored as none,
if err != nil { // one file. A bad value is refused here, at the form, instead of
return "", err // being stored.
} func setDatabaseTargetSettings(
target *database.Target,
return string(configBytes), nil
}
// buildDatabaseTargetConfig builds config JSON for a database
// (archive) target. The optional expiry and rotation are validated
// here, at creation time, so a bad value is refused instead of
// failing every subsequent delivery. Each is stored only when set,
// and with neither the config is empty (the keep-forever, one-file
// default).
func buildDatabaseTargetConfig(
expiry, rotation string, expiry, rotation string,
) (string, string, error) { ) string {
expiry = strings.TrimSpace(expiry) archiveExpiry, err := archive.ParseExpiry(strings.TrimSpace(expiry))
err := delivery.ValidateArchiveExpiry(expiry)
if err != nil { if err != nil {
return "", fmt.Sprintf("Invalid archive expiry: %v", err), nil return fmt.Sprintf("Invalid archive expiry: %v", err)
} }
err = delivery.ValidateArchiveRotation(rotation) err = archive.ValidateRotation(rotation)
if err != nil { if err != nil {
return "", fmt.Sprintf("Invalid archive rotation: %v", err), nil return fmt.Sprintf("Invalid archive rotation: %v", err)
} }
cfg := map[string]any{} if rotation == "" {
rotation = archiveRotationNone
if expiry != "" {
cfg["expiry"] = expiry
} }
if rotation != "" { target.ArchiveExpiry = archiveExpiry
cfg["rotation"] = rotation target.ArchiveRotation = rotation
}
if len(cfg) == 0 { return ""
return "", "", nil
}
configJSON, err := marshalTargetConfig(cfg)
return configJSON, "", err
} }
@@ -176,7 +176,7 @@ func TestHandleTargetCreate_BodyURLStillCreatesTheTarget(t *testing.T) {
targets := targetsForWebhook(t, env.db, webhook.ID) targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1) require.Len(t, targets, 1)
assert.Contains(t, targets[0].Config, targetSecretSegments) assert.Equal(t, targetSecretURL, targets[0].URL)
// The body carried the credential, so the access log must still // The body carried the credential, so the access log must still
// not have it: the log records the request line only. // not have it: the log records the request line only.
@@ -196,7 +196,7 @@ func TestHandleTargetCreate_QueryStringCannotSupplyNameOrType(
w, _ := postTargetCreate( w, _ := postTargetCreate(
t, env, webhook.ID, t, env, webhook.ID,
"name=leaky&type=slack&max_retries=9&expiry=30d&url="+ "name=leaky&type=slack&max_attempts=9&expiry=30d&url="+
url.QueryEscape(targetSecretURL), url.QueryEscape(targetSecretURL),
url.Values{}, url.Values{},
) )
+82 -41
View File
@@ -6,73 +6,114 @@ import (
"net/url" "net/url"
"strings" "strings"
"testing" "testing"
"time"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
) )
// TestHandleTargetCreate_EveryType adds a target of each type. Each // TestHandleTargetCreate_EveryType adds a target of each type and
// submission carries a url: only the http and slack types store one. // checks the settings it stores. Each submission carries a url and an
// expiry: only the http and slack types store the url, only the
// database type the expiry, and every setting a type does not use
// stays empty.
func TestHandleTargetCreate_EveryType(t *testing.T) { func TestHandleTargetCreate_EveryType(t *testing.T) {
t.Parallel() t.Parallel()
env := setupSourceTest(t) env := setupSourceTest(t)
// fields is the rest of each submission, as a query string. // fields is the rest of each submission, as a query string, and
// want holds the settings stored.
cases := []struct { cases := []struct {
targetType database.TargetType targetType database.TargetType
fields string fields string
wantConfig string want database.Target
wantRetries int wantAttempts int
}{ }{
{ {
database.TargetTypeHTTP, "timeout=12&max_retries=3", database.TargetTypeHTTP,
`{"url":"` + editOriginalURL + `","timeout":12}`, 3, "timeout=12&max_attempts=3&forward_query=on",
database.Target{
URL: editOriginalURL,
Headers: map[string]string{},
TimeoutSeconds: 12,
ForwardQuery: true,
}, 3,
}, },
{ {
database.TargetTypeSlack, "max_retries=4", database.TargetTypeSlack, "max_attempts=4",
`{"webhookUrl":"` + editOriginalURL + `"}`, 4, database.Target{URL: editOriginalURL}, 4,
}, },
{ {
database.TargetTypeDatabase, "expiry=720h", database.TargetTypeDatabase, "rotation=daily",
`{"expiry":"720h"}`, 0, database.Target{
ArchiveExpiry: 720 * time.Hour,
ArchiveRotation: rotationDaily,
}, 1,
}, },
{database.TargetTypeLog, "", "", 0}, {database.TargetTypeLog, "", database.Target{}, 1},
} }
for _, tc := range cases { for _, tc := range cases {
t.Run(string(tc.targetType), func(t *testing.T) { t.Run(string(tc.targetType), func(t *testing.T) {
t.Parallel() t.Parallel()
webhook := seedWebhookWithRetention(t, env.db, 30) _, got := addTargetThroughForm(
t, env, tc.targetType,
form, err := url.ParseQuery(tc.fields) "url="+editOriginalURL+"&expiry=720h&"+tc.fields,
require.NoError(t, err)
form.Set("name", "every-type")
form.Set("type", string(tc.targetType))
form.Set("url", editOriginalURL)
w := serveTarget(
env, http.MethodPost,
"/hook/"+webhook.ID+"/targets", form,
) )
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String()) assert.Equal(t, tc.targetType, got.Type)
assert.Equal(t, tc.wantAttempts, got.MaxAttempts)
targets := targetsForWebhook(t, env.db, webhook.ID) assert.Equal(t, tc.want, targetSettingsOf(got))
require.Len(t, targets, 1)
assert.Equal(t, tc.targetType, targets[0].Type)
assert.Equal(t, tc.wantRetries, targets[0].MaxRetries)
if tc.wantConfig == "" {
assert.Empty(t, targets[0].Config)
} else {
assert.JSONEq(t, tc.wantConfig, targets[0].Config)
}
}) })
} }
} }
// addTargetThroughForm adds a target of targetType named every-type to
// a new webhook through the add target form, with the rest of the form
// given as a query string, and returns the webhook and the target the
// form stored.
func addTargetThroughForm(
t *testing.T,
env *sourceTestEnv,
targetType database.TargetType,
fields string,
) (database.Webhook, database.Target) {
t.Helper()
webhook := seedWebhookWithRetention(t, env.db, 30)
form, err := url.ParseQuery(fields)
require.NoError(t, err)
form.Set("name", "every-type")
form.Set("type", string(targetType))
w := serveTarget(
env, http.MethodPost, "/hook/"+webhook.ID+"/targets", form,
)
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1)
return webhook, targets[0]
}
// targetSettingsOf returns a target's settings, URL to
// ArchiveRotation, and nothing else, for comparing with the settings a
// test expects.
func targetSettingsOf(target database.Target) database.Target {
return database.Target{
URL: target.URL,
Headers: target.Headers,
TimeoutSeconds: target.TimeoutSeconds,
ForwardQuery: target.ForwardQuery,
ArchiveExpiry: target.ArchiveExpiry,
ArchiveRotation: target.ArchiveRotation,
}
}
// TestHandleTargetCreate_RefusedFormComesBack refuses a target of each // TestHandleTargetCreate_RefusedFormComesBack refuses a target of each
// type and checks that the webhook page comes back with the add target // type and checks that the webhook page comes back with the add target
// form open on that type, the values entered, and the reason. // form open on that type, the values entered, and the reason.
@@ -90,11 +131,11 @@ func TestHandleTargetCreate_RefusedFormComesBack(t *testing.T) {
{ {
database.TargetTypeHTTP, database.TargetTypeHTTP,
"name=private&url=" + editBlockedURL + "name=private&url=" + editBlockedURL +
"&timeout=12&max_retries=3", "&timeout=12&max_attempts=3",
"Invalid target URL", "Invalid target URL",
}, },
{ {
database.TargetTypeSlack, "name=no-url&max_retries=4", database.TargetTypeSlack, "name=no-url&max_attempts=4",
"Webhook URL is required for Slack targets", "Webhook URL is required for Slack targets",
}, },
{ {
@@ -133,8 +174,8 @@ func TestHandleTargetCreate_RefusedFormComesBack(t *testing.T) {
assert.Contains(t, page, html.EscapeString(tc.reason)) assert.Contains(t, page, html.EscapeString(tc.reason))
// Each value comes back in a data attribute of the targets // Each value comes back in a data attribute of the targets
// section named after its field (max_retries as // section named after its field (max_attempts as
// data-max-retries), except url, which comes back in // data-max-attempts), except url, which comes back in
// data-destination; templates/source_detail.html says why. // data-destination; templates/source_detail.html says why.
for field := range typed { for field := range typed {
attr := "data-" + strings.ReplaceAll(field, "_", "-") attr := "data-" + strings.ReplaceAll(field, "_", "-")
+16 -16
View File
@@ -6,8 +6,8 @@ import (
"net/http" "net/http"
"time" "time"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
) )
// downloadWriteTimeout is how long one write of a download may wait // downloadWriteTimeout is how long one write of a download may wait
@@ -16,7 +16,7 @@ const downloadWriteTimeout = 60 * time.Second
// HandleTargetDownload serves a database target's archive as one // HandleTargetDownload serves a database target's archive as one
// gzipped JSON file, named for the webhook, the target and the time; // gzipped JSON file, named for the webhook, the target and the time;
// see delivery.ArchiveExport.WriteGzipJSON for what it holds. Other // see archive.Export.WriteGzipJSON for what it holds. Other
// target types have no archive and are a 404. // target types have no archive and are a 404.
// //
// A download runs for as long as the client keeps reading: it reads // A download runs for as long as the client keeps reading: it reads
@@ -37,7 +37,7 @@ func (h *Handlers) HandleTargetDownload() http.HandlerFunc {
w.Header().Set("Content-Type", "application/gzip") w.Header().Set("Content-Type", "application/gzip")
w.Header().Set( w.Header().Set(
"Content-Disposition", "Content-Disposition",
`attachment; filename="`+delivery.ArchiveExportFileName( `attachment; filename="`+archive.ExportFileName(
webhook.Name, target.Name, now, webhook.Name, target.Name, now,
)+`"`, )+`"`,
) )
@@ -82,17 +82,17 @@ func (d downloadWriter) Write(b []byte) (int, error) {
// listTargetArchive lists the archive files of the request's database // listTargetArchive lists the archive files of the request's database
// target for export. It reports false once it has written the response. // target for export. It reports false once it has written the response.
// //
// It holds renameMu, which every archive rename runs under, while it // It holds the rename lock, which every archive rename runs under,
// reads the stored names and lists the files, so the files it lists // while it reads the stored names and lists the files, so the files it
// are the ones those names give. It lets go before the export is // lists are the ones those names give. It lets go before the export is
// streamed, which takes renameMu again for each file only while it // streamed, which takes the lock again for each file only while it
// finds the file under the names stored then and opens it. // finds the file under the names stored then and opens it.
func (h *Handlers) listTargetArchive( func (h *Handlers) listTargetArchive(
w http.ResponseWriter, w http.ResponseWriter,
r *http.Request, r *http.Request,
) (database.Webhook, *database.Target, *delivery.ArchiveExport, bool) { ) (database.Webhook, *database.Target, *archive.Export, bool) {
h.renameMu.Lock() h.archives.RenameLock().Lock()
defer h.renameMu.Unlock() defer h.archives.RenameLock().Unlock()
webhook, target, ok := h.ownedTarget(w, r) webhook, target, ok := h.ownedTarget(w, r)
if !ok { if !ok {
@@ -105,9 +105,9 @@ func (h *Handlers) listTargetArchive(
return database.Webhook{}, nil, nil, false return database.Webhook{}, nil, nil, false
} }
export, err := delivery.NewArchiveExport( export, err := archive.NewExport(
delivery.ArchivePath(h.dbMgr, &webhook, target), archive.Path(h.dbMgr, &webhook, target),
&h.renameMu, h.archives.RenameLock(),
func() (string, error) { func() (string, error) {
return h.storedArchivePath(webhook.ID, target.ID) return h.storedArchivePath(webhook.ID, target.ID)
}, },
@@ -122,9 +122,9 @@ func (h *Handlers) listTargetArchive(
return webhook, target, export, true return webhook, target, export, true
} }
// storedArchivePath returns the path delivery.ArchivePath gives a // storedArchivePath returns the path archive.Path gives a
// database target under the names stored for it and its webhook now. // database target under the names stored for it and its webhook now.
// Its caller holds renameMu. A webhook or target deleted since is still // Its caller holds the rename lock. A webhook or target deleted since is still
// found, since deleting one leaves its archive files under their names. // found, since deleting one leaves its archive files under their names.
func (h *Handlers) storedArchivePath( func (h *Handlers) storedArchivePath(
webhookID, targetID string, webhookID, targetID string,
@@ -143,5 +143,5 @@ func (h *Handlers) storedArchivePath(
return "", err return "", err
} }
return delivery.ArchivePath(h.dbMgr, &webhook, &target), nil return archive.Path(h.dbMgr, &webhook, &target), nil
} }
+11 -11
View File
@@ -21,9 +21,9 @@ import (
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/config" "sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest" "sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
) )
@@ -223,8 +223,8 @@ func TestHandleTargetDownload_FindsFilesAfterRename(t *testing.T) {
env := setupSourceTest(t) env := setupSourceTest(t)
wh := seedWebhookWithRetention(t, env.db, 7) wh := seedWebhookWithRetention(t, env.db, 7)
archive := seedTarget(t, env.db, wh.ID, database.TargetTypeDatabase) target := seedTarget(t, env.db, wh.ID, database.TargetTypeDatabase)
oldPath := delivery.ArchivePath(env.dbMgr, &wh, archive) oldPath := archive.Path(env.dbMgr, &wh, target)
month := func(path string) string { month := func(path string) string {
return strings.TrimSuffix(path, ".db") + "-2026-10.db" return strings.TrimSuffix(path, ".db") + "-2026-10.db"
@@ -233,19 +233,19 @@ func TestHandleTargetDownload_FindsFilesAfterRename(t *testing.T) {
seedArchive(t, oldPath, 1, 16) seedArchive(t, oldPath, 1, 16)
seedArchive(t, month(oldPath), 1, 16) seedArchive(t, month(oldPath), 1, 16)
sw, downloaded := startStalledDownload(t, env, wh.ID, archive.ID) sw, downloaded := startStalledDownload(t, env, wh.ID, target.ID)
require.Equal(t, require.Equal(t,
http.StatusSeeOther, renameTarget(env, wh.ID, archive.ID).Code, http.StatusSeeOther, renameTarget(env, wh.ID, target.ID).Code,
) )
// The test's archives record a rename without moving any file, so // The test's archives record a rename without moving any file, so
// the files are moved here, as the delivery engine moves them. // the files are moved here, as the archive writers move them.
var renamed database.Target var renamed database.Target
require.NoError(t, env.db.DB().First(&renamed, "id = ?", archive.ID).Error) require.NoError(t, env.db.DB().First(&renamed, "id = ?", target.ID).Error)
newPath := delivery.ArchivePath(env.dbMgr, &wh, &renamed) newPath := archive.Path(env.dbMgr, &wh, &renamed)
require.NoError(t, os.Rename(oldPath, newPath)) require.NoError(t, os.Rename(oldPath, newPath))
require.NoError(t, os.Rename(month(oldPath), month(newPath))) require.NoError(t, os.Rename(month(oldPath), month(newPath)))
@@ -355,16 +355,16 @@ func TestHandleTargetDownload_OutlastsTheRequestLimit(t *testing.T) {
env := setupSourceTest(t) env := setupSourceTest(t)
wh := seedWebhookWithRetention(t, env.db, 7) wh := seedWebhookWithRetention(t, env.db, 7)
archive := seedTarget(t, env.db, wh.ID, database.TargetTypeDatabase) target := seedTarget(t, env.db, wh.ID, database.TargetTypeDatabase)
seedArchive( seedArchive(
t, delivery.ArchivePath(env.dbMgr, &wh, archive), rows, bodySize, t, archive.Path(env.dbMgr, &wh, target), rows, bodySize,
) )
srv, accessLog := limitedServer(t, env, limit) srv, accessLog := limitedServer(t, env, limit)
req, err := http.NewRequestWithContext( req, err := http.NewRequestWithContext(
t.Context(), http.MethodGet, t.Context(), http.MethodGet,
srv.URL+downloadPath(wh.ID, archive.ID), nil, srv.URL+downloadPath(wh.ID, target.ID), nil,
) )
require.NoError(t, err) require.NoError(t, err)
+33 -57
View File
@@ -6,6 +6,7 @@ import (
"strconv" "strconv"
"github.com/go-chi/chi" "github.com/go-chi/chi"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery" "sneak.berlin/go/webhooker/internal/delivery"
) )
@@ -24,22 +25,15 @@ const (
tmplKeyMaxTimeout = "MaxTimeout" tmplKeyMaxTimeout = "MaxTimeout"
) )
// configUnreadableMessage is shown when a target's stored
// configuration does not parse. It says plainly that saving replaces
// the stored value rather than preserving it, because the form
// cannot pre-fill what it could not read.
const configUnreadableMessage = "The stored configuration for this " +
"target could not be read. Enter the values below; saving " +
"replaces the stored configuration."
// targetEditView is the display model for the target edit page: the // targetEditView is the display model for the target edit page: the
// target's row fields as stored. The values the form shows, the // target's row fields as stored. The values the form shows, the
// UNMASKED configuration among them, come separately, as a // UNMASKED URL and headers among them, come separately, as a
// targetFormInput. // targetFormInput.
// //
// It deliberately omits database.Target's raw Config blob: the form // It deliberately omits database.Target's settings: the form renders
// renders named fields, and giving the template the blob as well // them from the targetFormInput, and giving the template the target's
// would put an unreviewed second path to the credential on the page. // URL and headers as well would put an unreviewed second path to the
// credential on the page.
type targetEditView struct { type targetEditView struct {
ID string ID string
Name string Name string
@@ -47,13 +41,16 @@ type targetEditView struct {
Active bool Active bool
} }
// HandleTargetEdit shows the form to edit a target. // HandleTargetEdit shows the form to edit a target, filled with the
// target's settings as stored.
// //
// This page is the one place the full destination URL and header // This page is the one place the full destination URL and header
// values are shown. It is reachable only through the // values are shown: an operator cannot correct a value they cannot
// /hook/{sourceID} route group, which supplies RequireAuth and // see. It is reachable only through the /hook/{sourceID} route
// NoCache, and only for a target of a webhook the session's user // group, which supplies RequireAuth and NoCache, and only for a
// owns; masking (delivery.TargetView) is unchanged everywhere else. // target of a webhook the session's user owns; masking
// (delivery.TargetView) is unchanged everywhere else. Do not fill a
// form like this one on any other page.
func (h *Handlers) HandleTargetEdit() http.HandlerFunc { func (h *Handlers) HandleTargetEdit() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) { return func(w http.ResponseWriter, r *http.Request) {
webhook, target, ok := h.ownedTarget(w, r) webhook, target, ok := h.ownedTarget(w, r)
@@ -61,34 +58,19 @@ func (h *Handlers) HandleTargetEdit() http.HandlerFunc {
return return
} }
cfg, err := delivery.NewTargetConfigForm(target)
msg := ""
if err != nil {
// The error carries the parse failure, never the
// blob, so it is safe to log against the target id.
h.log.Warn(
"stored target config could not be read for editing",
"target_id", target.ID,
"error", err,
)
msg = configUnreadableMessage
}
form := targetFormInput{ form := targetFormInput{
Name: target.Name, Name: target.Name,
URL: cfg.URL, URL: target.URL,
Headers: cfg.Headers, Headers: delivery.FormatTargetHeaders(target.Headers),
Timeout: cfg.Timeout, Timeout: delivery.FormatTargetTimeout(target.TimeoutSeconds),
ForwardQuery: cfg.ForwardQuery, ForwardQuery: target.ForwardQuery,
MaxRetries: strconv.Itoa(target.MaxRetries), MaxAttempts: strconv.Itoa(target.MaxAttempts),
Expiry: cfg.Expiry, Expiry: formatArchiveExpiry(target.ArchiveExpiry),
Rotation: cfg.Rotation, Rotation: target.ArchiveRotation,
} }
h.renderTargetEdit( h.renderTargetEdit(
w, r, webhook, target, form, msg, http.StatusOK, w, r, webhook, target, form, "", http.StatusOK,
) )
} }
} }
@@ -96,8 +78,8 @@ func (h *Handlers) HandleTargetEdit() http.HandlerFunc {
// HandleTargetEditSubmit handles the target edit form submission. // HandleTargetEditSubmit handles the target edit form submission.
func (h *Handlers) HandleTargetEditSubmit() http.HandlerFunc { func (h *Handlers) HandleTargetEditSubmit() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) { return func(w http.ResponseWriter, r *http.Request) {
h.renameMu.Lock() h.archives.RenameLock().Lock()
defer h.renameMu.Unlock() defer h.archives.RenameLock().Unlock()
webhook, target, ok := h.ownedTarget(w, r) webhook, target, ok := h.ownedTarget(w, r)
if !ok { if !ok {
@@ -124,11 +106,11 @@ func (h *Handlers) HandleTargetEditSubmit() http.HandlerFunc {
// does, so an edited destination is SSRF-validated exactly as a new // does, so an edited destination is SSRF-validated exactly as a new
// one is. // one is.
// //
// The target's type is not editable. Each type stores a different // The target's type is not editable. Each type uses different
// configuration shape and its delivery history is recorded against // settings and its delivery history is recorded against the target
// the target row, so changing the type of an existing target is // row, so changing the type of an existing target is really the
// really the creation of a different one. The stored type decides // creation of a different one. The stored type decides which fields
// which fields the form offers and which builder runs. // the form offers and which settings are set.
func (h *Handlers) applyTargetEdit( func (h *Handlers) applyTargetEdit(
w http.ResponseWriter, w http.ResponseWriter,
r *http.Request, r *http.Request,
@@ -141,13 +123,7 @@ func (h *Handlers) applyTargetEdit(
// as stored, for the page shown again when the save is refused. // as stored, for the page shown again when the save is refused.
edited := *target edited := *target
errMsg, err := h.setTargetFromForm(r.Context(), &edited, in) errMsg := h.setTargetFromForm(r.Context(), &edited, in)
if err != nil {
h.serverError(w, r, "failed to encode target config", err)
return
}
if errMsg != "" { if errMsg != "" {
h.renderTargetEdit( h.renderTargetEdit(
w, r, webhook, target, in, errMsg, http.StatusBadRequest, w, r, webhook, target, in, errMsg, http.StatusBadRequest,
@@ -157,9 +133,9 @@ func (h *Handlers) applyTargetEdit(
} }
// A new name renames the archive file before it is saved (see // A new name renames the archive file before it is saved (see
// delivery.Engine.Rename). If either step fails, it goes back to // archive.Writers.Rename). If either step fails, it goes back to
// the name that is still stored. // the name that is still stored.
err = h.renameTargetArchive( err := h.renameTargetArchive(
target, webhook.Name, target.Name, edited.Name, target, webhook.Name, target.Name, edited.Name,
) )
if err == nil { if err == nil {
@@ -178,7 +154,7 @@ func (h *Handlers) applyTargetEdit(
) )
} }
if errors.Is(err, delivery.ErrArchiveNameTaken) { if errors.Is(err, archive.ErrNameTaken) {
h.renderTargetEdit( h.renderTargetEdit(
w, r, webhook, target, in, w, r, webhook, target, in,
"Not saved: "+err.Error()+ "Not saved: "+err.Error()+

Some files were not shown because too many files have changed in this diff Show More