sneak/webhooker
1
0
Fork 0
Code Issues 5 Pull Requests 1 Actions 11 Packages Projects Releases Wiki Activity

Compare commits

base: sneak/webhooker:next
Branches Tags
sneak/webhooker:main sneak/webhooker:next sneak/webhooker:prod sneak/webhooker:issue-70-log-target
sneak/webhooker:v1.0.0-rc2 sneak/webhooker:v1.0.0-rc1
..
compare: sneak/webhooker:ab7eec4b3ef3a65d629933b2d68cb349d7bbdc90
Branches Tags
sneak/webhooker:next sneak/webhooker:main sneak/webhooker:prod sneak/webhooker:issue-70-log-target
sneak/webhooker:v1.0.0-rc2 sneak/webhooker:v1.0.0-rc1
1 Commits
next .. ab7eec4b3e
Author SHA1 Message Date
sneak ab7eec4b3e Move the archive files into their own package (closes #541)
check / check (push) Canceled after 0s
Details 
The SQLite archive files the database target writes are now managed by
internal/archive: file names, rotation, the writer of each target,
pruning, renames, the gzipped JSON export, the idle sweep, and the
rename lock, which was Handlers.renameMu. The database target stays in
internal/delivery and writes through archive.Writers; the engine still
closes the writers when it stops, so the stop order is unchanged. The
handlers take the rename lock from archive.Archives.RenameLock, and
rename and evict through archive.Archives.

Exported names drop their Archive prefix in the new package. The tests
moved with the code; the two that check a delivery's recorded attempt
stay in delivery.

Model: opus-5-5
 2026-10-15 18:42:29 +00:00
83 changed files with 3416 additions and 3156 deletions
Expand all files Collapse all files
README.md
+97 -123
View File
@@ -1346,11 +1346,11 @@ webhooker solves this by acting as a durable intermediary:
**That guarantee is at-least-once, not exactly-once.** When a send reaches **That guarantee is at-least-once, not exactly-once.** When a send reaches
its target but the write recording that outcome fails, the delivery is its target but the write recording that outcome fails, the delivery is
deliberately left in a recoverable state rather than marked done — losing a deliberately left in a recoverable state rather than marked done — losing a
delivery is the worse failure — so the sweep picks it up about fifteen delivery is the worse failure — so the pending sweep picks it up about
minutes later, or the next restart does, and the target receives a payload fifteen minutes later, or the next restart does, and the target receives a
it already got. A send still in flight when the service stops is left the payload it already got. A send still in flight when the service stops is
same way: its attempt is not recorded, and the next start sends it again left the same way: its attempt is not recorded, and the next start sends it
(see Shutdown). webhooker adds no delivery identifier of its own to an again (see Shutdown). webhooker adds no delivery identifier of its own to an
outbound request, so **make your receiver idempotent** against whatever the outbound request, so **make your receiver idempotent** against whatever the
payload itself carries. payload itself carries.
@@ -1579,20 +1579,15 @@ did not arrive on the URL and counts in neither.
A delivery destination for events. Each target defines where and how events A delivery destination for events. Each target defines where and how events
should be forwarded. should be forwarded.
| Field | Type | Description | | Field | Type | Description |
| ------------------ | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | -------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id` | UUID | Primary key | | `id` | UUID | Primary key |
| `webhook_id` | UUID | Foreign key → Webhook | | `webhook_id` | UUID | Foreign key → Webhook |
| `name` | string | Human-readable name | | `name` | string | Human-readable name |
| `type` | TargetType | One of: `http`, `slack`, `database`, `log` | | `type` | TargetType | One of: `http`, `slack`, `database`, `log` |
| `active` | boolean | Whether deliveries are enabled (default: true) | | `active` | boolean | Whether deliveries are enabled (default: true) |
| `url` | string | Where an `http` or `slack` target sends each event. It is a credential: the target list shows only its scheme and host, and only the target edit form shows it whole | | `config` | JSON text | Type-specific configuration |
| `headers` | JSON text | An `http` target's extra request headers, a JSON object of name to value | | `max_attempts` | integer | How many times an `http` or `slack` target attempts each delivery in all: at least 1, and 1 by default. Ignored by `database` and `log` targets. The web UI labels it Delivery attempts |
| `timeout_seconds` | integer | An `http` target's per-request timeout; 0 uses the shared 30-second client timeout |
| `forward_query` | boolean | Whether an `http` target is sent the query string each event arrived with |
| `archive_expiry` | integer | How long a `database` target keeps each archived event, as a Go `time.Duration` (nanoseconds); 0 keeps them forever |
| `archive_rotation` | string | How often a `database` target starts a new archive file: `none`, `monthly`, `daily` or `hourly` |
| `max_attempts` | integer | How many times an `http` or `slack` target attempts each delivery in all: at least 1, and 1 by default. Ignored by `database` and `log` targets. The web UI labels it Delivery attempts |
**Relations:** Belongs to Webhook. Has many Deliveries. **Relations:** Belongs to Webhook. Has many Deliveries.
@@ -1603,10 +1598,10 @@ should be forwarded.
failed delivery with exponential backoff, up to that many attempts in all, failed delivery with exponential backoff, up to that many attempts in all,
behind a per-target circuit breaker. behind a per-target circuit breaker.
- **`slack`** — Post the event as a formatted message to a Slack-compatible - **`slack`** — Post the event as a formatted message to a Slack-compatible
incoming webhook URL, its `url`. It is built on the same HTTP core as `http` incoming webhook URL (`webhookUrl` in `config`). It is built on the same HTTP
and honours `max_attempts` identically, circuit breaker included. See the core as `http` and honours `max_attempts` identically, circuit breaker
Slack target section under "Per-Webhook Event Databases" for the message included. See the Slack target section under "Per-Webhook Event Databases" for
format. the message format.
- **`database`** — Archive the full event as a row into the target's own archive - **`database`** — Archive the full event as a row into the target's own archive
database (`archive-{webhook_name}-{target_name}-{target_uuid}.db`) for database (`archive-{webhook_name}-{target_name}-{target_uuid}.db`) for
long-term retention, with an optional creation-validated expiry (default: keep long-term retention, with an optional creation-validated expiry (default: keep
@@ -1622,28 +1617,29 @@ should be forwarded.
- **`log`** — Write the event to the application log (stdout). Useful for - **`log`** — Write the event to the application log (stdout). Useful for
debugging. debugging.
Each type uses only its own settings, and the ones it does not use stay empty: The `config` field stores type-specific configuration as JSON (e.g., destination
`http` uses `url`, `headers`, `timeout_seconds` and `forward_query`; `slack` URL, custom headers, timeout settings).
uses `url`; `database` uses `archive_expiry` and `archive_rotation`; `log` uses
none. The add target form, the new webhook form and the target edit form check
every setting before it is saved, and each setting is a column of its own, so
nothing that reads a target parses its settings or has to handle one it cannot
read.
An `http` target POSTs the event to its `url`. Its `headers` are applied last, **`http` target configuration:**
so they win over the event's own forwarded headers.
`forward_query` is off by default, and the target URL is then sent exactly as | Key | Type | Description |
| -------------- | ------------- | ----------------------------------------------------------------------------------------- |
| `url` | string | Destination the event is POSTed to |
| `headers` | object | Extra request headers, applied last so they win over the event's own forwarded headers |
| `timeout` | integer (sec) | Per-target request timeout; unset (or 0) uses the shared 30-second client timeout |
| `forwardQuery` | boolean | Pass the query string each event arrived with on to the target; unset (or false) does not |
`forwardQuery` is off by default, and the target URL is then sent exactly as
configured. On, each delivery appends the event's query string to the target configured. On, each delivery appends the event's query string to the target
URL, joined with `&` when the URL already has a query string of its own; a URL, joined with `&` when the URL already has a query string of its own; a
replayed delivery and a resubmitted event's deliveries do the same. Both target replayed delivery and a resubmitted event's deliveries do the same. Both target
forms offer it as "Pass the query string on to this target", and the target list forms offer it as "Pass the query string on to this target", and the target list
shows it when it is on. shows it when it is on.
`timeout_seconds` is capped at **300 seconds**, and the form rejects anything `timeout` is capped at **300 seconds**, and the form rejects anything above it
above it rather than substituting the cap. A delivery attempt holds one of the rather than substituting the cap. A delivery attempt holds one of the bounded
bounded pool's workers for its whole duration, so an unbounded timeout would let pool's workers for its whole duration, so an unbounded timeout would let a
a single unresponsive destination stall the queue. single unresponsive destination stall the queue.
`headers` rejects the names the delivery path or `net/http` writes regardless of `headers` rejects the names the delivery path or `net/http` writes regardless of
what is configured: `Host`, `Content-Length`, `Transfer-Encoding`, `Connection`, what is configured: `Host`, `Content-Length`, `Transfer-Encoding`, `Connection`,
@@ -1717,7 +1713,7 @@ The event log and the event's own page show the query string with the rest of
the request. The event log leaves out one larger than 32 KiB, as it does request the request. The event log leaves out one larger than 32 KiB, as it does request
headers, and links to the event's page, which shows it whole. The `database` and headers, and links to the event's page, which shows it whole. The `database` and
`log` targets carry it with the rest of the event. An `http` target receives it `log` targets carry it with the rest of the event. An `http` target receives it
only when its `forward_query` setting is on. only when its `forwardQuery` setting is on.
#### Delivery #### Delivery
@@ -1869,15 +1865,15 @@ These indexes on the per-webhook event databases are created by
`internal/db/migrations/events/001_schema.sql`, and declared in the model tags `internal/db/migrations/events/001_schema.sql`, and declared in the model tags
to match: to match:
| Table | Columns | Serves | | Table | Columns | Serves |
| ------------------ | ------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ------------------ | ------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `deliveries` | `status`, `deleted_at`, `finished_at`, `target_id` | Delivery recovery at start and every 60 seconds and the queue-depth sampler every 30 seconds, which select deliveries by status, the webhook page's statistics and target list and the webhook list, which count each target's deliveries by status and when they finished, and the event log, which lists and counts the events with a failed delivery or one pending or retrying | | `deliveries` | `status`, `deleted_at`, `finished_at`, `target_id` | Startup recovery, the retry and pending sweeps every 60 seconds and the queue-depth sampler every 30 seconds, which select deliveries by status, the webhook page's statistics and target list and the webhook list, which count each target's deliveries by status and when they finished, and the event log, which lists and counts the events with a failed delivery or one pending or retrying |
| `deliveries` | `event_id`, `deleted_at` | The event log, which loads each event's deliveries, and retention, which counts and deletes the deliveries of expired events | | `deliveries` | `event_id`, `deleted_at` | The event log, which loads each event's deliveries, and retention, which counts and deletes the deliveries of expired events |
| `delivery_results` | `delivery_id`, `deleted_at` | The event log, which loads the attempts of a page's deliveries, and retention, which deletes the attempts of expired events | | `delivery_results` | `delivery_id`, `deleted_at` | The event log, which loads the attempts of a page's deliveries, and retention, which deletes the attempts of expired events |
| `events` | `deleted_at`, `created_at` | The webhook page's statistics, which count recent events | | `events` | `deleted_at`, `created_at` | The webhook page's statistics, which count recent events |
| `events` | `resubmitted_from_id`, `deleted_at` | The event log, which counts the events resubmitted from each event on a page | | `events` | `resubmitted_from_id`, `deleted_at` | The event log, which counts the events resubmitted from each event on a page |
| `events` | `entrypoint_id`, `deleted_at`, `resubmitted_from_id`, `created_at` | The webhook page's entrypoint list, which counts the events that arrived on each entrypoint's URL within the retention period | | `events` | `entrypoint_id`, `deleted_at`, `resubmitted_from_id`, `created_at` | The webhook page's entrypoint list, which counts the events that arrived on each entrypoint's URL within the retention period |
| `events` | `created_at` | Retention, which selects expired events by age | | `events` | `created_at` | Retention, which selects expired events by age |
GORM's soft delete adds `deleted_at IS NULL` to these queries; retention leaves GORM's soft delete adds `deleted_at IS NULL` to these queries; retention leaves
it out. SQLite keeps no statistics on these tables, and without them it rates it out. SQLite keeps no statistics on these tables, and without them it rates
@@ -1996,10 +1992,11 @@ target UUID keeps the file name unique. A webhook named `Orders (EU)` with a
target named `Long-term archive` archives into target named `Long-term archive` archives into
`archive-orders-eu-long-term-archive-{target_uuid}.db`. `archive-orders-eu-long-term-archive-{target_uuid}.db`.
A target's `archive_rotation` is `none`, the default, which keeps the one file, An optional `rotation` in the target's config JSON (e.g. `{"rotation":"daily"}`)
or `monthly`, `daily` or `hourly`. A target that rotates writes each event to a is `none`, the default, which keeps the one file, or `monthly`, `daily` or
file named for the period of the event's receive time, in UTC, put before the `hourly`. A target that rotates writes each event to a file named for the period
`.db`: `archive-orders-eu-long-term-archive-{target_uuid}-2026-10.db` monthly, of the event's receive time, in UTC, put before the `.db`:
`archive-orders-eu-long-term-archive-{target_uuid}-2026-10.db` monthly,
`…-2026-10-01.db` daily and `…-2026-10-01-19.db` hourly. Each file holds exactly `…-2026-10-01.db` daily and `…-2026-10-01-19.db` hourly. Each file holds exactly
its period's events, and the first event of a new period starts the next file, its period's events, and the first event of a new period starts the next file,
so a finished period's file can be moved away like any archive. A changed so a finished period's file can be moved away like any archive. A changed
@@ -2034,11 +2031,12 @@ archiving without stopping the service; a moved or removed archive file is
recreated automatically on the next write, from the SQL files in recreated automatically on the next write, from the SQL files in
`internal/db/migrations/archive/`. A zero-length archive file is written to as a `internal/db/migrations/archive/`. A zero-length archive file is written to as a
new archive: SQLite opens it as an empty database, so it holds nothing to lose. new archive: SQLite opens it as an empty database, so it holds nothing to lose.
A target's `archive_expiry` is checked by the form that saves it — the default, An optional `expiry` in the target's config JSON (e.g. `{"expiry":"720h"}`) is
`never` on the forms and 0 in the column, keeps rows forever — and rows older validated when the target is created — the default (unset or the literal
than the expiry are pruned each time the archive is (re)opened. An archive write `never`) keeps rows forever — and rows older than the expiry are pruned each
failure is never silent success: the delivery records a failed attempt with the time the archive is (re)opened. An archive write failure is never silent
error and is marked failed. success: the delivery records a failed attempt with the error and is marked
failed.
Because reopens only happen on writes, an archive whose target has stopped Because reopens only happen on writes, an archive whose target has stopped
receiving events would never be pruned. A background **archive sweeper** closes receiving events would never be pruned. A background **archive sweeper** closes
@@ -2049,7 +2047,8 @@ traffic. The sweep never creates an archive — a target whose archive file does
not yet exist is skipped, not initialised — it takes the same per-target lock not yet exist is skipped, not initialised — it takes the same per-target lock
the write path uses, so it can never interleave with a write, and it leaves the the write path uses, so it can never interleave with a write, and it leaves the
archive closed afterwards so the move-the-file-away workflow keeps working. archive closed afterwards so the move-the-file-away workflow keeps working.
Archives with the expiry `never` are not touched by the sweep at all. Archives with no expiry, or the expiry `never`, are not touched by the sweep at
all.
For a target with several files, the write path prunes only the file it writes For a target with several files, the write path prunes only the file it writes
to, and the sweep prunes every one of them, taking the target's lock for one to, and the sweep prunes every one of them, taking the target's lock for one
@@ -2117,8 +2116,9 @@ compatible services). Each message includes event metadata (HTTP method, content
type, timestamp, body size) and the payload pretty-printed in a code block. JSON type, timestamp, body size) and the payload pretty-printed in a code block. JSON
payloads are automatically formatted with indentation for readability; non-JSON payloads are automatically formatted with indentation for readability; non-JSON
payloads are shown as raw text. Large payloads are truncated to keep messages payloads are shown as raw text. Large payloads are truncated to keep messages
reasonable. The target's `url` is the Slack/Mattermost incoming webhook reasonable. Config stores `webhookUrl` — the Slack/Mattermost incoming webhook
endpoint. endpoint. That is the JSON key; the error text for a missing one reads
`webhook_url is required`, which is the message, not the key.
The database uses the The database uses the
[modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) driver. [modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) driver.
@@ -2153,8 +2153,12 @@ External Service
1. Look up Entrypoint by UUID — 404 if unknown, 1. Look up Entrypoint by UUID — 404 if unknown,
410 if inactive 410 if inactive
2. Read the body under the 1 MB cap 2. Read the body under the 1 MB cap
3. Hand the full request, as an Event, to 3. Capture full request as Event
Engine.StoreEvent 4. Create Delivery records for each active Target
5. Build self-contained delivery.Task structs
(target config + event data inline for
bodies < 16 KiB)
6. Notify Engine via channel (no DB read needed)
│ │
▼ ▼
┌──────────────┐ ┌──────────────┐
@@ -2163,13 +2167,6 @@ External Service
│ (worker │ │ (worker │
│ pool) │ │ pool) │
└──────┬───────┘ └──────┬───────┘
4. Store the Event, a pending Delivery for each
active Target, and the totals, in one
transaction
5. Build a delivery.Task for each Delivery
(target settings + event data, the body
inline when it is under 16 KiB)
6. Queue the tasks on the delivery channel
│ │
┌── bounded worker pool (N workers) ──┐ ┌── bounded worker pool (N workers) ──┐
▼ ▼ ▼ ▼ ▼ ▼
@@ -2183,12 +2180,6 @@ External Service
└────────────┘ └────────────┘
``` ```
A **Resubmit** stores its copy of an event through the same `Engine.StoreEvent`,
and a **Replay** adds one delivery of a stored event through
`Engine.CreateReplay`, which writes the delivery and its target's totals in one
transaction and queues it. Those two methods are the only code that creates a
delivery; the handlers read and check the request, then make the one call.
### Bounded Worker Pool ### Bounded Worker Pool
The delivery engine uses a **fixed-size worker pool** (default: 10 workers) to The delivery engine uses a **fixed-size worker pool** (default: 10 workers) to
@@ -2198,9 +2189,8 @@ preventing goroutine explosions regardless of queue depth.
**Architecture:** **Architecture:**
- **Channels as queues:** Two buffered channels serve as bounded queues: a - **Channels as queues:** Two buffered channels serve as bounded queues: a
delivery channel (the tasks of newly stored deliveries and of the `pending` delivery channel (new tasks from the webhook handler) and a retry channel
deliveries the recovery routine hands back) and a retry channel (tasks from (tasks from backoff timers). Both are buffered to 10,000.
backoff timers). Both are buffered to 10,000.
- **Fan-out via channel, not goroutines:** When an event arrives with multiple - **Fan-out via channel, not goroutines:** When an event arrives with multiple
targets, each `delivery.Task` is sent to the delivery channel. Workers pick targets, each `delivery.Task` is sent to the delivery channel. Workers pick
them up and process them — no goroutine-per-target. them up and process them — no goroutine-per-target.
@@ -2209,7 +2199,7 @@ preventing goroutine explosions regardless of queue depth.
Workers are the ONLY goroutines doing actual HTTP delivery. Workers are the ONLY goroutines doing actual HTTP delivery.
- **Retry backpressure with DB fallback:** When a retry timer fires and the - **Retry backpressure with DB fallback:** When a retry timer fires and the
retry channel is full, the timer is dropped — the delivery stays in `retrying` retry channel is full, the timer is dropped — the delivery stays in `retrying`
status in the database. The sweep (every 60s, see Recovery below) finds these status in the database. A periodic sweep (every 60s) scans for these
"orphaned" retries and re-queues them. No blocked goroutines, no unbounded "orphaned" retries and re-queues them. No blocked goroutines, no unbounded
timer chains. timer chains.
- **Bounded concurrency:** At most N deliveries (N = number of workers) are - **Bounded concurrency:** At most N deliveries (N = number of workers) are
@@ -2236,36 +2226,24 @@ This means:
that did not finish are still running — an unclean shutdown is reported rather that did not finish are still running — an unclean shutdown is reported rather
than hidden. than hidden.
**Recovery:** one routine finds the deliveries the engine is not working on and **Recovery paths:**
hands them back. It runs once when the engine starts, and then every 60 seconds;
that periodic run is the sweep. For every webhook it reads the `pending` and
`retrying` deliveries. One that already has a successful attempt is marked
`delivered` and not sent again. A `retrying` delivery gets a backoff timer for
what is left of its backoff, and is queued at once if nothing is left. A
`pending` delivery is queued on the delivery channel. A delivery the engine has
queued, is attempting, or holds a backoff timer for is left alone, at start as
on every sweep. This is what catches "orphaned" retries, whose in-memory timer
was dropped because the retry channel was full, and deliveries a stop or a crash
cut off: the database is the durable fallback that ensures no delivery is
permanently lost, even under extreme backpressure.
The run at start and the sweep differ in two ways: 1. **Startup recovery:** When the engine starts, it scans all per-webhook
databases for `pending` and `retrying` deliveries. Pending deliveries are
- At start every `pending` delivery is taken, however recent, so one a stop cut sent to the delivery channel; retrying deliveries get backoff timers
off moments ago is sent again at once (see Shutdown). The sweep takes only scheduled.
`pending` deliveries untouched for 15 minutes, at most 500 per webhook each 2. **Periodic retry sweep (DB-mediated fallback):** Every 60 seconds the engine
time, so a delivery whose outcome the database will not record is sent again scans for `retrying` deliveries whose backoff period has elapsed. This
every 15 minutes rather than every minute. catches "orphaned" retries — ones whose in-memory timer was dropped because
- At start every webhook's database is opened, so that a missing one is reported the retry channel was full. The database is the durable fallback that ensures
then (see Per-Webhook Event Databases). The sweep skips a webhook whose no retry is permanently lost, even under extreme backpressure.
database file is missing rather than create it.
**Changing a target's type does not migrate in-flight deliveries.** Only `http` **Changing a target's type does not migrate in-flight deliveries.** Only `http`
and `slack` targets own durable retries; `database` and `log` targets are and `slack` targets own durable retries; `database` and `log` targets are
fire-and-forget and never produce a `retrying` delivery. If a target's `type` is fire-and-forget and never produce a `retrying` delivery. If a target's `type` is
edited from a retrying type to a non-retrying (or unknown) one while one of its edited from a retrying type to a non-retrying (or unknown) one while one of its
deliveries is still `retrying`, the recovery routine above terminally marks that deliveries is still `retrying`, both recovery paths above terminally mark that
delivery `failed` and records a `DeliveryResult` naming the current target type delivery `failed` and record a `DeliveryResult` naming the current target type
as the reason, logging it at warn level. The delivery is not re-dispatched under as the reason, logging it at warn level. The delivery is not re-dispatched under
the new type — the operator never asked for that delivery — and the event itself the new type — the operator never asked for that delivery — and the event itself
remains stored in the per-webhook event database. Nothing is lost: the delivery remains stored in the per-webhook event database. Nothing is lost: the delivery
@@ -2278,8 +2256,8 @@ re-inject it to every currently active target.
`http` and `slack` targets with `max_attempts` > 1 are protected by a `http` and `slack` targets with `max_attempts` > 1 are protected by a
**per-target circuit breaker** that prevents hammering a down target with **per-target circuit breaker** that prevents hammering a down target with
repeated failed delivery attempts. The circuit breaker is in-memory only and repeated failed delivery attempts. The circuit breaker is in-memory only and
resets on restart (which is fine — the recovery routine rescans the database at resets on restart (which is fine — startup recovery rescans the database
start anyway). anyway).
**States:** **States:**
@@ -3064,21 +3042,20 @@ webhooker/
│ │ └── logfield.go # Encoded-byte budget for client-supplied log values │ │ └── logfield.go # Encoded-byte budget for client-supplied log values
│ ├── delivery/ │ ├── delivery/
│ │ ├── engine.go # Event-driven delivery engine (channel + timer based) │ │ ├── engine.go # Event-driven delivery engine (channel + timer based)
│ │ ├── store.go # Storing an event with its deliveries, and a replay, then queueing them
│ │ ├── circuit_breaker.go # Per-target circuit breaker for http/slack targets with retries │ │ ├── circuit_breaker.go # Per-target circuit breaker for http/slack targets with retries
│ │ ├── target.go # Target interface, Task, Scheduler │ │ ├── target.go # Target interface, Task, Scheduler
│ │ ├── target_http.go # HTTP target (retries, circuit breaker) │ │ ├── target_http.go # HTTP target (retries, circuit breaker)
│ │ ├── target_slack.go # Slack/Mattermost incoming-webhook target │ │ ├── target_slack.go # Slack/Mattermost incoming-webhook target
│ │ ├── target_database.go # Database archive target, writing through internal/archive │ │ ├── target_database.go # Database archive target, writing through internal/archive
│ │ ├── target_log.go # Log target (stdout) │ │ ├── target_log.go # Log target (stdout)
│ │ ├── target_config_view.go # Masked target settings for templates │ │ ├── target_config_view.go # Masked target config for templates
│ │ ├── queue_depth.go # Periodic sampler behind the queue-depth gauges │ │ ├── queue_depth.go # Periodic sampler behind the queue-depth gauges
│ │ ├── url_mask.go # Strips credentials from *url.Error │ │ ├── url_mask.go # Strips credentials from *url.Error
│ │ └── ssrf.go # SSRF prevention (IP validation, safe HTTP transport) │ │ └── ssrf.go # SSRF prevention (IP validation, safe HTTP transport)
│ ├── handlers/ │ ├── handlers/
│ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering │ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering
│ │ ├── auth.go # Login, logout handlers │ │ ├── auth.go # Login, logout handlers
│ │ ├── delivery_replay.go # Per-delivery replay: new delivery, current target settings │ │ ├── delivery_replay.go # Per-delivery replay: new delivery, current target config
│ │ ├── event_resubmit.go # Event resubmit: new event, all currently active targets │ │ ├── event_resubmit.go # Event resubmit: new event, all currently active targets
│ │ ├── entrypoint_view.go # Masked entrypoint view for templates │ │ ├── entrypoint_view.go # Masked entrypoint view for templates
│ │ ├── event_log_view.go # Event log projection, byte-capped in SQL │ │ ├── event_log_view.go # Event log projection, byte-capped in SQL
@@ -3093,7 +3070,7 @@ webhooker/
│ │ ├── webhook_delete.go # Webhook delete, event database and archive writer removal │ │ ├── webhook_delete.go # Webhook delete, event database and archive writer removal
│ │ ├── event_log.go # Event log page: loaders, filters, delivery views │ │ ├── event_log.go # Event log page: loaders, filters, delivery views
│ │ ├── entrypoint.go # Entrypoint create, edit, delete and toggle │ │ ├── entrypoint.go # Entrypoint create, edit, delete and toggle
│ │ ├── target_create.go # Target create, each type's settings from the form │ │ ├── target_create.go # Target create, per-type config builders
│ │ ├── target_delete.go # Target delete │ │ ├── target_delete.go # Target delete
│ │ ├── target_toggle.go # Target toggle │ │ ├── target_toggle.go # Target toggle
│ │ ├── shared.go # Helpers shared by several handlers │ │ ├── shared.go # Helpers shared by several handlers
@@ -3161,26 +3138,23 @@ Components are wired via Uber fx in this order:
10. `metrics.NewRegistry` — The registry `/metrics` serves 10. `metrics.NewRegistry` — The registry `/metrics` serves
11. `metrics.New` — The delivery collectors, registered on that registry 11. `metrics.New` — The delivery collectors, registered on that registry
12. `middleware.New` — HTTP middleware 12. `middleware.New` — HTTP middleware
13. `delivery.New` — Event-driven delivery engine, which also stores each event 13. `delivery.New` — Event-driven delivery engine
with its deliveries and creates replay deliveries
14. `archive.NewWriters` — The database targets' archive writers, and the lock 14. `archive.NewWriters` — The database targets' archive writers, and the lock
every rename of their files runs under every rename of their files runs under
15. `archive.NewSweeper` — Periodic pruning of idle archives 15. `archive.NewSweeper` — Periodic pruning of idle archives
16. `archive.Writers` → `archive.Archives` — interface bridge so deleting or 16. `delivery.Engine` → `delivery.Notifier` — interface bridge
17. `archive.Writers` → `archive.Archives` — interface bridge so deleting or
renaming a webhook or target reaches its archive files renaming a webhook or target reaches its archive files
17. `delivery.Engine` → `delivery.CircuitBreakers` — interface bridge so the
pages can show a target whose deliveries are paused
18. `server.New` — HTTP server and router 18. `server.New` — HTTP server and router
The server starts via The server starts via
`fx.Invoke(func(*server.Server, *delivery.Engine, *database.RetentionReaper, *archive.Sweeper) {})`, `fx.Invoke(func(*server.Server, *delivery.Engine, *database.RetentionReaper, *archive.Sweeper) {})`,
which triggers the fx lifecycle hooks in dependency order. The handlers take which triggers the fx lifecycle hooks in dependency order. The
`*delivery.Engine` itself: the receiver and resubmit handlers store an event `delivery.Notifier` interface allows the webhook handler to send self-contained
through `StoreEvent`, and the replay handler adds a delivery through `delivery.Task` slices to the engine without a direct package dependency. Each
`CreateReplay`. Each task the engine queues carries the target's settings and task carries all target config and event data inline (for bodies under 16 KiB,
the event's data, and the body too when it is under 16 KiB `delivery.MaxInlineBodySize`), so the engine can deliver without reading from
(`delivery.MaxInlineBodySize`); a larger body is read from the event row when any database — it only writes to record results.
the attempt runs.
### Middleware Stack ### Middleware Stack
@@ -3362,9 +3336,9 @@ normal operation.
The delivery engine cancels a delivery request in flight. That attempt is not The delivery engine cancels a delivery request in flight. That attempt is not
recorded or counted in the metrics, and the delivery keeps its `pending` or recorded or counted in the metrics, and the delivery keeps its `pending` or
`retrying` status, so recovery at the next start sends it again (see Recovery `retrying` status, so startup recovery sends it again on the next start (see
under Bounded Worker Pool). If the receiver already had it, it receives it Recovery paths under Bounded Worker Pool). If the receiver already had it, it
twice: the at-least-once duplicate described under Rationale. receives it twice: the at-least-once duplicate described under Rationale.
The HTTP drain budget is deliberately **shorter** than the sequence budget. Were The HTTP drain budget is deliberately **shorter** than the sequence budget. Were
the two equal, a drain that used its whole budget would exhaust the sequence the two equal, a drain that used its whole budget would exhaust the sequence
cmd/webhooker/main.go
+3
View File
@@ -211,6 +211,9 @@ func newApp() *fx.App {
delivery.New, delivery.New,
archive.NewWriters, archive.NewWriters,
archive.NewSweeper, archive.NewSweeper,
// Wire *delivery.Engine as delivery.Notifier so the
// webhook handler can notify the engine of new deliveries.
func(e *delivery.Engine) delivery.Notifier { return e },
// Wire *archive.Writers as archive.Archives so deleting // Wire *archive.Writers as archive.Archives so deleting
// or renaming a webhook or target reaches its archive // or renaming a webhook or target reaches its archive
// files. // files.
internal/archive/archive.go
+15 -7
View File
@@ -143,15 +143,26 @@ func archiveNamePart(name string) string {
} }
// Write writes the full event of a delivery to a database target as // Write writes the full event of a delivery to a database target as
// a row into the target's archive database, honouring the target's // a row into the target's archive database, honouring the optional
// ArchiveExpiry and ArchiveRotation. With rotation, the event goes to // per-target expiry and rotation parsed from the target config JSON.
// the file for the period of its receive time. // With rotation, the event goes to the file for the period of its
// receive time.
func (ws *Writers) Write(d *database.Delivery) error { func (ws *Writers) Write(d *database.Delivery) error {
webhookID := d.Event.WebhookID webhookID := d.Event.WebhookID
if webhookID == "" { if webhookID == "" {
return errArchiveMissingWebhookID return errArchiveMissingWebhookID
} }
expiry, err := ParseExpiry(d.Target.Config)
if err != nil {
return err
}
rotation, err := ParseRotation(d.Target.Config)
if err != nil {
return err
}
// An event whose stored row was gone before its delivery ran has // An event whose stored row was gone before its delivery ran has
// no receive time (see delivery's Engine.hydrateEvent), and goes // no receive time (see delivery's Engine.hydrateEvent), and goes
// to the file for now. // to the file for now.
@@ -176,10 +187,7 @@ func (ws *Writers) Write(d *database.Delivery) error {
ContentType: d.Event.ContentType, ContentType: d.Event.ContentType,
} }
return w.write( return w.write(row, expiry, archivePeriod(rotation, receivedAt))
row, d.Target.ArchiveExpiry,
archivePeriod(d.Target.ArchiveRotation, receivedAt),
)
} }
// Rename implements Archives. It renames every one of a database // Rename implements Archives. It renames every one of a database
internal/archive/archive_test.go
+36 -20
View File
@@ -123,10 +123,9 @@ func databaseTargetDelivery(
TargetID: tgt.ID, TargetID: tgt.ID,
Event: event, Event: event,
Target: database.Target{ Target: database.Target{
Name: tgt.Name, Name: tgt.Name,
Type: database.TargetTypeDatabase, Type: database.TargetTypeDatabase,
ArchiveExpiry: tgt.ArchiveExpiry, Config: tgt.Config,
ArchiveRotation: tgt.ArchiveRotation,
}, },
} }
} }
@@ -356,16 +355,13 @@ func TestParseArchiveExpiry(t *testing.T) {
want time.Duration want time.Duration
wantErr bool wantErr bool
}{ }{
{"empty", "", 0, false}, {"empty config", "", 0, false},
{"explicit never", "never", 0, false}, {"explicit never", `{"expiry":"never"}`, 0, false},
{"hour", "1h", time.Hour, false}, {"empty expiry", `{"expiry":""}`, 0, false},
{"thirty days", "720h", 720 * time.Hour, false}, {"duration", `{"expiry":"1h"}`, time.Hour, false},
{"minutes", "30m", 30 * time.Minute, false}, {"unparseable", `{"expiry":"nonsense"}`, 0, true},
{"unparseable", "nonsense", 0, true}, {"zero duration", `{"expiry":"0s"}`, 0, true},
{"days are not a Go unit", "7d", 0, true}, {"negative duration", `{"expiry":"-5h"}`, 0, true},
{"zero duration", "0s", 0, true},
{"bare zero", "0", 0, true},
{"negative duration", "-5h", 0, true},
} }
for _, tc := range cases { for _, tc := range cases {
@@ -385,6 +381,26 @@ func TestParseArchiveExpiry(t *testing.T) {
} }
} }
func TestValidateArchiveExpiry(t *testing.T) {
t.Parallel()
valid := []string{"", "never", "1h", "720h", "30m"}
for _, in := range valid {
require.NoError(t,
archive.ValidateExpiry(in),
"expiry %q should be accepted", in,
)
}
invalid := []string{"nonsense", "7d", "-5h", "0s", "0"}
for _, in := range invalid {
require.Error(t,
archive.ValidateExpiry(in),
"expiry %q should be rejected", in,
)
}
}
// TestArchiveFileName pins the archive file name and the rules // TestArchiveFileName pins the archive file name and the rules
// that make a webhook or target name safe to put in it. // that make a webhook or target name safe to put in it.
func TestArchiveFileName(t *testing.T) { func TestArchiveFileName(t *testing.T) {
@@ -457,8 +473,8 @@ func TestWrite_EachTargetHasItsOwnArchive(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
first := env.seedDatabaseTarget(t, 0, "") first := env.seedDatabaseTarget(t, "")
second := env.addDatabaseTarget(t, first.WebhookID, 0, "") second := env.addDatabaseTarget(t, first.WebhookID, "")
event := newEvent(`{"n":1}`) event := newEvent(`{"n":1}`)
@@ -487,7 +503,7 @@ func TestRename_MovesTheFile(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, "") tgt := env.seedDatabaseTarget(t, "")
oldPath := env.archivePath(tgt) oldPath := env.archivePath(tgt)
first := newEvent(`{"n":1}`) first := newEvent(`{"n":1}`)
@@ -529,7 +545,7 @@ func TestRename_NeverReplacesAFile(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, "") tgt := env.seedDatabaseTarget(t, "")
oldPath := env.archivePath(tgt) oldPath := env.archivePath(tgt)
first := newEvent(`{"n":1}`) first := newEvent(`{"n":1}`)
@@ -577,7 +593,7 @@ func TestRename_BeforeTheNameIsSaved(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, "") tgt := env.seedDatabaseTarget(t, "")
require.NoError( require.NoError(
t, env.writers.Rename(tgt.ID, "Orders", "Archive"), t, env.writers.Rename(tgt.ID, "Orders", "Archive"),
@@ -705,7 +721,7 @@ func TestRenameLock_HeldByOneEditAtATime(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, "") tgt := env.seedDatabaseTarget(t, "")
event := newEvent(`{"n":1}`) event := newEvent(`{"n":1}`)
env.writers.RenameLock().Lock() env.writers.RenameLock().Lock()
internal/archive/evict_test.go
+8 -8
View File
@@ -37,9 +37,9 @@ func TestEvictWebhook_ClosesAndRemovesWriter(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
first := env.seedDatabaseTarget(t, 0, "") first := env.seedDatabaseTarget(t, "")
second := env.addDatabaseTarget(t, first.WebhookID, 0, "") second := env.addDatabaseTarget(t, first.WebhookID, "")
other := env.seedDatabaseTarget(t, 0, "") other := env.seedDatabaseTarget(t, "")
for _, tgt := range []*database.Target{first, second, other} { for _, tgt := range []*database.Target{first, second, other} {
deliverTo(t, env, tgt) deliverTo(t, env, tgt)
@@ -80,8 +80,8 @@ func TestEvictTarget_LeavesOtherTargets(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
doomed := env.seedDatabaseTarget(t, 0, "") doomed := env.seedDatabaseTarget(t, "")
kept := env.addDatabaseTarget(t, doomed.WebhookID, 0, "") kept := env.addDatabaseTarget(t, doomed.WebhookID, "")
deliverTo(t, env, doomed) deliverTo(t, env, doomed)
deliverTo(t, env, kept) deliverTo(t, env, kept)
@@ -310,7 +310,7 @@ func TestEvictWebhook_RacingWriteDoesNotReopenHandle(
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, "") tgt := env.seedDatabaseTarget(t, "")
// Prime the registry so the test can hold the very writer the // Prime the registry so the test can hold the very writer the
// eviction is about to detach. // eviction is about to detach.
@@ -355,7 +355,7 @@ func TestEvictWebhook_LaterDeliveryRecreatesWriter(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, "") tgt := env.seedDatabaseTarget(t, "")
deliverTo(t, env, tgt) deliverTo(t, env, tgt)
require.True(t, env.writers.ExportHasArchiveWriter(tgt.ID)) require.True(t, env.writers.ExportHasArchiveWriter(tgt.ID))
@@ -380,7 +380,7 @@ func TestEvictAll_WriteAfterIsRefused(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, "") tgt := env.seedDatabaseTarget(t, "")
deliverTo(t, env, tgt) deliverTo(t, env, tgt)
internal/archive/rotation.go
+36 -5
View File
@@ -1,6 +1,7 @@
package archive package archive
import ( import (
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"os" "os"
@@ -53,10 +54,35 @@ func ValidateRotation(rotation string) error {
} }
} }
// ParseRotation reads the rotation from a database target's
// config JSON. An empty config or an empty rotation is none.
func ParseRotation(configJSON string) (string, error) {
if configJSON == "" {
return archiveRotationNone, nil
}
var cfg DatabaseTargetConfig
err := json.Unmarshal([]byte(configJSON), &cfg)
if err != nil {
return "", fmt.Errorf("parsing database target config: %w", err)
}
err = ValidateRotation(cfg.Rotation)
if err != nil {
return "", err
}
if cfg.Rotation == "" {
return archiveRotationNone, nil
}
return cfg.Rotation, nil
}
// archivePeriod returns the period, in UTC, that a rotation puts an // archivePeriod returns the period, in UTC, that a rotation puts an
// event received at receivedAt in: "2026-10" for monthly, // event received at receivedAt in: "2026-10" for monthly,
// "2026-10-01" for daily, "2026-10-01-19" for hourly, and "" for none // "2026-10-01" for daily, "2026-10-01-19" for hourly, and "" for none.
// or an empty rotation.
func archivePeriod(rotation string, receivedAt time.Time) string { func archivePeriod(rotation string, receivedAt time.Time) string {
switch rotation { switch rotation {
case archiveRotationMonthly: case archiveRotationMonthly:
@@ -89,11 +115,16 @@ func PathAt(
webhook *database.Webhook, webhook *database.Webhook,
target *database.Target, target *database.Target,
receivedAt time.Time, receivedAt time.Time,
) string { ) (string, error) {
rotation, err := ParseRotation(target.Config)
if err != nil {
return "", err
}
return archivePeriodPath( return archivePeriodPath(
Path(dbMgr, webhook, target), Path(dbMgr, webhook, target),
archivePeriod(target.ArchiveRotation, receivedAt), archivePeriod(rotation, receivedAt),
) ), nil
} }
// archiveFiles lists the database target's archive files that exist, // archiveFiles lists the database target's archive files that exist,
internal/archive/rotation_test.go
+33 -20
View File
@@ -14,12 +14,16 @@ import (
"sneak.berlin/go/webhooker/internal/database/databasetest" "sneak.berlin/go/webhooker/internal/database/databasetest"
) )
// The archive rotations. // The archive rotations, and the configs of a daily target and of one
// whose rotation is not one of the four.
const ( const (
rotationNone = "none" rotationNone = "none"
rotationMonthly = "monthly" rotationMonthly = "monthly"
rotationDaily = "daily" rotationDaily = "daily"
rotationHourly = "hourly" rotationHourly = "hourly"
dailyConfig = `{"rotation":"daily"}`
weeklyConfig = `{"rotation":"weekly"}`
) )
// The periods the tests archive into most: two days, and an hour of // The periods the tests archive into most: two days, and an hour of
@@ -104,7 +108,7 @@ func TestWrite_RotatesAtEachPeriodBoundary(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, tc.rotation) tgt := env.seedDatabaseTarget(t, `{"rotation":"`+tc.rotation+`"}`)
path := env.archivePath(tgt) path := env.archivePath(tgt)
first := env.deliverReceivedAt(t, tgt, tc.before) first := env.deliverReceivedAt(t, tgt, tc.before)
@@ -123,7 +127,7 @@ func TestWrite_RotatesAtEachPeriodBoundary(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, rotationNone) tgt := env.seedDatabaseTarget(t, `{"rotation":"`+rotationNone+`"}`)
first := env.deliverReceivedAt(t, tgt, cases[0].before) first := env.deliverReceivedAt(t, tgt, cases[0].before)
second := env.deliverReceivedAt(t, tgt, cases[0].after) second := env.deliverReceivedAt(t, tgt, cases[0].after)
@@ -140,7 +144,7 @@ func TestWrite_EventWithoutReceiveTime(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, rotationMonthly) tgt := env.seedDatabaseTarget(t, `{"rotation":"`+rotationMonthly+`"}`)
path := env.archivePath(tgt) path := env.archivePath(tgt)
before := time.Now().UTC().Format("2006-01") before := time.Now().UTC().Format("2006-01")
@@ -165,13 +169,13 @@ func TestWrite_RotationChangeKeepsOldFiles(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, rotationNone) tgt := env.seedDatabaseTarget(t, "")
path := env.archivePath(tgt) path := env.archivePath(tgt)
at := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC) at := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC)
first := env.deliverReceivedAt(t, tgt, at) first := env.deliverReceivedAt(t, tgt, at)
tgt.ArchiveRotation = rotationDaily tgt.Config = dailyConfig
second := env.deliverReceivedAt(t, tgt, at) second := env.deliverReceivedAt(t, tgt, at)
assert.Equal(t, []string{first}, archivedEventIDs(t, path)) assert.Equal(t, []string{first}, archivedEventIDs(t, path))
@@ -189,7 +193,9 @@ func TestArchiveSweep_PrunesEveryFile(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, time.Hour, rotationDaily) tgt := env.seedDatabaseTarget(
t, `{"expiry":"1h","rotation":"`+rotationDaily+`"}`,
)
path := env.archivePath(tgt) path := env.archivePath(tgt)
emptied := periodPath(path, dayPeriod) emptied := periodPath(path, dayPeriod)
kept := periodPath(path, nextDayPeriod) kept := periodPath(path, nextDayPeriod)
@@ -219,13 +225,13 @@ func TestRename_MovesEveryFile(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, rotationNone) tgt := env.seedDatabaseTarget(t, "")
oldPath := env.archivePath(tgt) oldPath := env.archivePath(tgt)
day := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC) day := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC)
unrotated := env.deliverReceivedAt(t, tgt, day) unrotated := env.deliverReceivedAt(t, tgt, day)
tgt.ArchiveRotation = rotationDaily tgt.Config = dailyConfig
first := env.deliverReceivedAt(t, tgt, day) first := env.deliverReceivedAt(t, tgt, day)
second := env.deliverReceivedAt(t, tgt, day.Add(24*time.Hour)) second := env.deliverReceivedAt(t, tgt, day.Add(24*time.Hour))
@@ -261,7 +267,7 @@ func TestRename_NeverReplacesARotatedFile(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, rotationDaily) tgt := env.seedDatabaseTarget(t, dailyConfig)
oldPath := env.archivePath(tgt) oldPath := env.archivePath(tgt)
day := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC) day := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC)
@@ -335,27 +341,34 @@ func TestArchivePathAt(t *testing.T) {
at := time.Date(2026, 10, 1, 19, 30, 0, 0, time.UTC) at := time.Date(2026, 10, 1, 19, 30, 0, 0, time.UTC)
cases := map[string]string{ cases := map[string]string{
"": "", "": "",
rotationNone: "", `{"rotation":"` + rotationNone + `"}`: "",
rotationMonthly: "-2026-10", `{"rotation":"` + rotationMonthly + `"}`: "-2026-10",
rotationDaily: "-2026-10-01", dailyConfig: "-2026-10-01",
rotationHourly: "-2026-10-01-19", `{"rotation":"` + rotationHourly + `"}`: "-2026-10-01-19",
} }
for rotation, period := range cases { for config, period := range cases {
target := &database.Target{ target := &database.Target{
BaseModel: database.BaseModel{ID: "tgt-id"}, BaseModel: database.BaseModel{ID: "tgt-id"},
Name: "Archive", Name: "Archive",
ArchiveRotation: rotation, Config: config,
} }
got, err := archive.PathAt(dbMgr, webhook, target, at)
require.NoError(t, err, config)
assert.Equal(t, assert.Equal(t,
filepath.Join( filepath.Join(
dataDir, "archive-orders-archive-tgt-id"+period+".db", dataDir, "archive-orders-archive-tgt-id"+period+".db",
), ),
archive.PathAt(dbMgr, webhook, target, at), rotation, got, config,
) )
} }
_, err := archive.PathAt(dbMgr, webhook, &database.Target{
Config: weeklyConfig,
}, at)
require.Error(t, err)
} }
// TestValidateArchiveRotation accepts the four rotations, and empty, // TestValidateArchiveRotation accepts the four rotations, and empty,
internal/archive/sweeper.go
+16 -5
View File
@@ -199,11 +199,22 @@ func (s *Sweeper) sweep(ctx context.Context) {
} }
} }
// sweepTarget prunes the archive of a single database target. A // sweepTarget prunes the archive of a single database target.
// target with an ArchiveExpiry of 0 keeps its archive forever and is // A missing, empty, or "never" expiry parses as a zero duration
// skipped entirely, so that archive is never pruned. // and is skipped entirely, so those archives are never pruned.
func (s *Sweeper) sweepTarget(target *database.Target) { func (s *Sweeper) sweepTarget(target *database.Target) {
expiry := target.ArchiveExpiry expiry, err := ParseExpiry(target.Config)
if err != nil {
s.log.Error(
"archive sweep: invalid database target config",
"webhook_id", target.WebhookID,
"target_id", target.ID,
"error", err,
)
return
}
if expiry <= 0 { if expiry <= 0 {
return return
} }
@@ -212,7 +223,7 @@ func (s *Sweeper) sweepTarget(target *database.Target) {
return return
} }
err := s.writers.sweepArchive(target.ID, expiry) err = s.writers.sweepArchive(target.ID, expiry)
if err == nil { if err == nil {
return return
} }
internal/archive/sweeper_test.go
+59 -51
View File
@@ -106,10 +106,10 @@ func (env *archiveEnv) archivePath(tgt *database.Target) string {
) )
} }
// seedDatabaseTarget creates a webhook with one database target with // seedDatabaseTarget creates a webhook with one database target
// the given archive expiry and rotation, and returns the target. // carrying the given target config JSON, and returns the target.
func (env *archiveEnv) seedDatabaseTarget( func (env *archiveEnv) seedDatabaseTarget(
t *testing.T, expiry time.Duration, rotation string, t *testing.T, configJSON string,
) *database.Target { ) *database.Target {
t.Helper() t.Helper()
@@ -124,27 +124,23 @@ func (env *archiveEnv) seedDatabaseTarget(
Create(wh).Error, Create(wh).Error,
) )
return env.addDatabaseTarget(t, wh.ID, expiry, rotation) return env.addDatabaseTarget(t, wh.ID, configJSON)
} }
// addDatabaseTarget creates one more database target on an // addDatabaseTarget creates one more database target on an
// existing webhook and returns it. // existing webhook and returns it.
func (env *archiveEnv) addDatabaseTarget( func (env *archiveEnv) addDatabaseTarget(
t *testing.T, t *testing.T, webhookID, configJSON string,
webhookID string,
expiry time.Duration,
rotation string,
) *database.Target { ) *database.Target {
t.Helper() t.Helper()
tgt := &database.Target{ tgt := &database.Target{
WebhookID: webhookID, WebhookID: webhookID,
Name: "Archive", Name: "Archive",
Type: database.TargetTypeDatabase, Type: database.TargetTypeDatabase,
Active: true, Active: true,
ArchiveExpiry: expiry, Config: configJSON,
ArchiveRotation: rotation, MaxAttempts: 1,
MaxAttempts: 1,
} }
require.NoError( require.NoError(
t, t,
@@ -327,7 +323,7 @@ func TestArchiveSweeper_LoopOutlivesStartHookContext(
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, time.Hour, "") tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
now := time.Now() now := time.Now()
path := env.seedArchiveRows( path := env.seedArchiveRows(
@@ -381,7 +377,7 @@ func TestArchiveSweep_DoesNotResurrectEvictedWriter(
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, time.Hour, "") tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
env.seedArchiveRows( env.seedArchiveRows(
t, tgt, time.Now().Add(-48*time.Hour), t, tgt, time.Now().Add(-48*time.Hour),
) )
@@ -415,7 +411,7 @@ func TestArchiveSweep_LeavesNoRegistryEntry(t *testing.T) {
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, time.Hour, "") tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
path := env.seedArchiveRows( path := env.seedArchiveRows(
t, tgt, t, tgt,
time.Now().Add(-48*time.Hour), time.Now().Add(-48*time.Hour),
@@ -448,7 +444,7 @@ func TestArchiveSweep_KeepsWriterAdoptedByDelivery(
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, time.Hour, "") tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
env.seedArchiveRows( env.seedArchiveRows(
t, tgt, time.Now().Add(-48*time.Hour), t, tgt, time.Now().Add(-48*time.Hour),
) )
@@ -502,7 +498,7 @@ func TestArchiveSweep_KeepsWriterAdoptedDuringSweep(
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, time.Hour, "") tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
env.seedArchiveRows( env.seedArchiveRows(
t, tgt, time.Now().Add(-48*time.Hour), t, tgt, time.Now().Add(-48*time.Hour),
) )
@@ -555,8 +551,8 @@ func TestArchiveSweep_KeepsWriterAdoptedDuringSweep(
// TestArchiveSweep_ContinuesAfterPerTargetFailure proves a // TestArchiveSweep_ContinuesAfterPerTargetFailure proves a
// failure for one target does not abort the sweep for the // failure for one target does not abort the sweep for the
// others: an unreadable archive has to be logged and stepped // others: an unparseable expiry and an unreadable archive both
// over. // have to be logged and stepped over.
func TestArchiveSweep_ContinuesAfterPerTargetFailure( func TestArchiveSweep_ContinuesAfterPerTargetFailure(
t *testing.T, t *testing.T,
) { ) {
@@ -564,16 +560,21 @@ func TestArchiveSweep_ContinuesAfterPerTargetFailure(
env := setupArchiveTest(t) env := setupArchiveTest(t)
// Seeded first so the sweep reaches it before the healthy // Seeded first so the sweep reaches them before the healthy
// target: targets come back in insertion order. // target: targets come back in insertion order.
corrupt := env.seedDatabaseTarget(t, time.Hour, "") badConfig := env.seedDatabaseTarget(t, `{"expiry":"!!!"}`)
env.seedArchiveRows(
t, badConfig, time.Now().Add(-48*time.Hour),
)
corrupt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
require.NoError(t, os.WriteFile( require.NoError(t, os.WriteFile(
env.archivePath(corrupt), env.archivePath(corrupt),
[]byte("this is not a sqlite database"), []byte("this is not a sqlite database"),
0o600, 0o600,
)) ))
healthy := env.seedDatabaseTarget(t, time.Hour, "") healthy := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
healthyPath := env.seedArchiveRows( healthyPath := env.seedArchiveRows(
t, healthy, t, healthy,
time.Now().Add(-48*time.Hour), time.Now().Add(-48*time.Hour),
@@ -630,7 +631,7 @@ func TestArchiveSweep_PrunesIdleArchive(t *testing.T) {
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, time.Hour, "") tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
now := time.Now() now := time.Now()
path := env.seedArchiveRows( path := env.seedArchiveRows(
@@ -669,7 +670,7 @@ func TestArchiveSweep_LeavesArchiveClosed(t *testing.T) {
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, time.Hour, "") tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
path := env.seedArchiveRows( path := env.seedArchiveRows(
t, tgt, time.Now().Add(-48*time.Hour), t, tgt, time.Now().Add(-48*time.Hour),
) )
@@ -709,7 +710,7 @@ func TestArchiveSweep_ClosesHandleOfRegisteredWriter(
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, time.Hour, "") tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
env.seedArchiveRows( env.seedArchiveRows(
t, tgt, time.Now().Add(-48*time.Hour), t, tgt, time.Now().Add(-48*time.Hour),
) )
@@ -794,29 +795,36 @@ func TestArchiveSweep_CancelledSweepLogsNoError(t *testing.T) {
} }
// TestArchiveSweep_NeverExpiryUntouched proves the sweep is a // TestArchiveSweep_NeverExpiryUntouched proves the sweep is a
// no-op for the default retention policy, so an archive with an // no-op for the default retention policy, so archives with no
// expiry of 0, which keeps it forever, is left alone. // expiry (or the literal "never") behave exactly as before.
func TestArchiveSweep_NeverExpiryUntouched(t *testing.T) { func TestArchiveSweep_NeverExpiryUntouched(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) for _, configJSON := range []string{
`{"expiry":"never"}`,
`{"expiry":""}`,
"",
} {
env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, "") tgt := env.seedDatabaseTarget(t, configJSON)
path := env.seedArchiveRows( path := env.seedArchiveRows(
t, tgt, t, tgt,
time.Now().Add(-10000*time.Hour), time.Now().Add(-10000*time.Hour),
) )
env.sweeper.ExportSweep(context.Background()) env.sweeper.ExportSweep(context.Background())
assert.Equal( assert.Equal(
t, []string{sweepRowOld}, archivedEventIDs(t, path), t, []string{sweepRowOld}, archivedEventIDs(t, path),
"an expiry of 0 must keep rows forever", "config %q must keep rows forever", configJSON,
) )
assert.False( assert.False(
t, env.writers.ExportHasArchiveWriter(tgt.ID), t, env.writers.ExportHasArchiveWriter(tgt.ID),
"an expiry of 0 must leave no registry entry behind", "config %q must leave no registry entry behind",
) configJSON,
)
}
} }
// TestArchiveSweep_NeverExpirySkipsBeforeOpening pins the // TestArchiveSweep_NeverExpirySkipsBeforeOpening pins the
@@ -837,7 +845,7 @@ func TestArchiveSweep_NeverExpirySkipsBeforeOpening(
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, 0, "") tgt := env.seedDatabaseTarget(t, `{"expiry":"never"}`)
path := env.archivePath(tgt) path := env.archivePath(tgt)
seedUnmigratedArchive(t, path) seedUnmigratedArchive(t, path)
@@ -889,7 +897,7 @@ func TestArchiveSweep_DoesNotCreateArchiveFile(t *testing.T) {
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, time.Hour, "") tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
path := env.archivePath(tgt) path := env.archivePath(tgt)
require.NoFileExists(t, path) require.NoFileExists(t, path)
@@ -920,7 +928,7 @@ func TestArchiveSweep_DoesNotCreateAfterWriterExists(
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, time.Hour, "") tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
path, err := env.writers.ExportEnsureArchiveWriter(tgt.ID) path, err := env.writers.ExportEnsureArchiveWriter(tgt.ID)
require.NoError(t, err) require.NoError(t, err)
@@ -939,7 +947,7 @@ func TestArchiveSweep_SkipsDeletedWebhookTargets(t *testing.T) {
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, time.Hour, "") tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
path := env.seedArchiveRows( path := env.seedArchiveRows(
t, tgt, time.Now().Add(-48*time.Hour), t, tgt, time.Now().Add(-48*time.Hour),
) )
@@ -967,7 +975,7 @@ func TestArchiveSweep_ConcurrentWrites(t *testing.T) {
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, time.Hour, "") tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
deliveries := make( deliveries := make(
[]*database.Delivery, 0, sweepConcurrentWrites, []*database.Delivery, 0, sweepConcurrentWrites,
@@ -1012,7 +1020,7 @@ func TestArchiveSweeper_StopsCleanly(t *testing.T) {
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t, time.Hour, "") tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
env.seedArchiveRows( env.seedArchiveRows(
t, tgt, time.Now().Add(-48*time.Hour), t, tgt, time.Now().Add(-48*time.Hour),
) )
internal/archive/writer.go
+70 -14
View File
@@ -1,6 +1,7 @@
package archive package archive
import ( import (
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"io/fs" "io/fs"
@@ -17,9 +18,8 @@ import (
"sneak.berlin/go/webhooker/internal/gormlog" "sneak.berlin/go/webhooker/internal/gormlog"
) )
// ExpiryNever is the archive expiry, as the target forms take it, // ExpiryNever is the expiry sentinel (and default) that
// that disables pruning so archived rows are kept forever. It is // disables pruning so archived rows are kept forever.
// stored as an ArchiveExpiry of 0.
const ExpiryNever = "never" const ExpiryNever = "never"
// archiveReopenDebounce bounds how often an archive file is // archiveReopenDebounce bounds how often an archive file is
@@ -79,6 +79,19 @@ var (
) )
) )
// DatabaseTargetConfig is the optional per-target JSON config
// for a database (archive) target.
type DatabaseTargetConfig struct {
// Expiry is a Go duration (e.g. "720h") after which
// archived rows are pruned, or "never" (the default) to
// keep them forever.
Expiry string `json:"expiry"`
// Rotation is none (the default), monthly, daily or hourly: see
// archivePeriod.
Rotation string `json:"rotation"`
}
// archivedEvent is one fully captured webhook event stored in a // archivedEvent is one fully captured webhook event stored in a
// database target's archive for long-term retention. It is a // database target's archive for long-term retention. It is a
// self-contained copy — independent of the per-webhook event // self-contained copy — independent of the per-webhook event
@@ -99,20 +112,63 @@ type archivedEvent struct {
ArchivedAt time.Time `gorm:"index"` ArchivedAt time.Time `gorm:"index"`
} }
// ParseExpiry reads a user-supplied archive expiry for a database // ParseExpiry reads the optional expiry from a database
// target, as the target forms submit it, into the ArchiveExpiry the // target's config JSON. An empty config, an empty expiry, or
// target stores. Empty and "never" both mean keep forever, which is // the literal "never" all mean keep forever, returned as a zero
// 0; anything else must be a positive Go duration such as "720h". Any // duration. Any other value must parse as a positive Go
// other value is an error, so a bad expiry is refused at the form // duration; a set-but-invalid value (unparseable, zero, or
// rather than stored. // negative) is an error rather than a silent default, matching
func ParseExpiry(expiry string) (time.Duration, error) { // ValidateExpiry at target creation.
if expiry == "" || expiry == ExpiryNever { func ParseExpiry(
configJSON string,
) (time.Duration, error) {
if configJSON == "" {
return 0, nil return 0, nil
} }
var cfg DatabaseTargetConfig
err := json.Unmarshal([]byte(configJSON), &cfg)
if err != nil {
return 0, fmt.Errorf(
"parsing database target config: %w", err,
)
}
if cfg.Expiry == "" || cfg.Expiry == ExpiryNever {
return 0, nil
}
dur, err := time.ParseDuration(cfg.Expiry)
if err != nil {
return 0, fmt.Errorf(
"parsing archive expiry %q: %w", cfg.Expiry, err,
)
}
if dur <= 0 {
return 0, fmt.Errorf(
"%w: %q", errArchiveExpiryNotPositive, cfg.Expiry,
)
}
return dur, nil
}
// ValidateExpiry checks a user-supplied archive expiry
// for a database target at configuration time. Valid values are
// empty, "never" (both meaning keep forever), or a positive Go
// duration such as "720h". Anything else is an error, so a bad
// expiry is rejected when the target is created rather than
// failing every subsequent delivery.
func ValidateExpiry(expiry string) error {
if expiry == "" || expiry == ExpiryNever {
return nil
}
dur, err := time.ParseDuration(expiry) dur, err := time.ParseDuration(expiry)
if err != nil { if err != nil {
return 0, fmt.Errorf( return fmt.Errorf(
"expiry must be %q or a Go duration "+ "expiry must be %q or a Go duration "+
"such as \"720h\": %w", "such as \"720h\": %w",
ExpiryNever, err, ExpiryNever, err,
@@ -120,12 +176,12 @@ func ParseExpiry(expiry string) (time.Duration, error) {
} }
if dur <= 0 { if dur <= 0 {
return 0, fmt.Errorf( return fmt.Errorf(
"%w: %q", errArchiveExpiryNotPositive, expiry, "%w: %q", errArchiveExpiryNotPositive, expiry,
) )
} }
return dur, nil return nil
} }
// archiveWriter owns one database target's archive SQLite files. // archiveWriter owns one database target's archive SQLite files.
internal/database/event_db_isolation_test.go
+1 -1
View File
@@ -28,7 +28,7 @@ func TestEventDBCreateOmitsAssociations(t *testing.T) {
WebhookID: webhookID, WebhookID: webhookID,
Name: "leaky-target", Name: "leaky-target",
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
URL: "https://hooks.example/secret", Config: `{"webhookUrl":"https://hooks.example/secret"}`,
MaxAttempts: 1, MaxAttempts: 1,
} }
target.ID = uuid.New().String() target.ID = uuid.New().String()
internal/database/event_tier_indexes_test.go
+2 -3
View File
@@ -79,9 +79,8 @@ func TestEventTierQueriesUseTheirIndexes(t *testing.T) {
byStatus := "idx_deliveries_status (status=? AND deleted_at=?)" byStatus := "idx_deliveries_status (status=? AND deleted_at=?)"
byEvent := "idx_deliveries_event_id (event_id=? AND deleted_at=?)" byEvent := "idx_deliveries_event_id (event_id=? AND deleted_at=?)"
// The delivery engine: the recovery routine, which reads the // The delivery engine: recovery and the retry sweep, the sweep for
// retrying deliveries and, on a tick of the sweep, the pending ones // stranded pending deliveries, and the queue depth count.
// untouched for a while; and the queue depth count.
assertPlanUses(t, db, dry.Where( assertPlanUses(t, db, dry.Where(
"status = ?", database.DeliveryStatusRetrying, "status = ?", database.DeliveryStatusRetrying,
).Find(&deliveries), byStatus) ).Find(&deliveries), byStatus)
internal/database/model_secrets_test.go
+7 -18
View File
@@ -29,10 +29,8 @@ func marshalModel(t *testing.T, v any) string {
// these tags the first handler that marshals a model serialises the // these tags the first handler that marshals a model serialises the
// secret with it. Each field below is a live credential: // secret with it. Each field below is a live credential:
// //
// - Target.URL is an incoming-webhook URL whose path segments are // - Target.Config holds an incoming-webhook URL whose path segments
// the bearer token. // are the bearer token.
// - Target.Headers holds header values, routinely authorization
// tokens.
// - APIKey.Key is a bearer token outright. // - APIKey.Key is a bearer token outright.
// - Setting.Value holds the session encryption key. // - Setting.Value holds the session encryption key.
// - User.Password holds the Argon2 hash, and was already tagged. // - User.Password holds the Argon2 hash, and was already tagged.
@@ -46,20 +44,11 @@ func TestModelsDoNotMarshalTheirSecrets(t *testing.T) {
model any model any
}{ }{
{ {
name: "target url", name: "target config",
model: database.Target{ model: database.Target{
Name: keptField, Name: keptField,
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
URL: "https://h/s/" + marker, Config: `{"webhookUrl":"https://h/s/` + marker + `"}`,
MaxAttempts: 1,
},
},
{
name: "target headers",
model: database.Target{
Name: keptField,
Type: database.TargetTypeHTTP,
Headers: map[string]string{"Authorization": marker},
MaxAttempts: 1, MaxAttempts: 1,
}, },
}, },
@@ -98,10 +87,10 @@ func TestModelsDoNotMarshalTheirSecrets(t *testing.T) {
} }
} }
// TestWebhookMarshalsNoTargetURL covers the nested case: a webhook // TestWebhookMarshalsNoTargetConfig covers the nested case: a webhook
// marshalled with its targets preloaded must not carry the credential // marshalled with its targets preloaded must not carry the credential
// through the association either. // through the association either.
func TestWebhookMarshalsNoTargetURL(t *testing.T) { func TestWebhookMarshalsNoTargetConfig(t *testing.T) {
t.Parallel() t.Parallel()
const marker = "QQNESTEDMARKERQQ" const marker = "QQNESTEDMARKERQQ"
@@ -110,7 +99,7 @@ func TestWebhookMarshalsNoTargetURL(t *testing.T) {
Name: keptField, Name: keptField,
Targets: []database.Target{{ Targets: []database.Target{{
Name: "slack", Name: "slack",
URL: "https://h/s/" + marker, Config: `{"webhookUrl":"https://h/s/` + marker + `"}`,
MaxAttempts: 1, MaxAttempts: 1,
}}, }},
}) })
internal/database/model_target.go
+8 -33
View File
@@ -1,7 +1,5 @@
package database package database
import "time"
// TargetType represents the type of delivery target // TargetType represents the type of delivery target
type TargetType string type TargetType string
@@ -13,9 +11,7 @@ const (
TargetTypeSlack TargetType = "slack" TargetTypeSlack TargetType = "slack"
) )
// Target represents a delivery target for a webhook. Its settings // Target represents a delivery target for a webhook
// are the fields from URL to ArchiveRotation; each type uses some of
// them, and the ones a type does not use stay empty.
type Target struct { type Target struct {
BaseModel BaseModel
@@ -24,35 +20,14 @@ type Target struct {
Type TargetType `gorm:"not null" json:"type"` Type TargetType `gorm:"not null" json:"type"`
Active bool `gorm:"default:true" json:"active"` Active bool `gorm:"default:true" json:"active"`
// URL is where an http or slack target sends each event. // Configuration fields (JSON stored based on type).
// //
// json:"-" because the URL is the target's credential: a Slack // json:"-" because the blob holds the target's credential — a
// incoming-webhook URL, or an http destination whose path segments // Slack incoming-webhook URL, or an http destination whose path
// are the secret. delivery.TargetView is the masking barrier for // segments are the secret. delivery.TargetView is the masking
// the HTML path; this tag is the barrier for any handler that // barrier for the HTML path; this tag is the barrier for any
// marshals the model itself. // handler that marshals the model itself.
URL string `json:"-"` Config string `gorm:"type:text" json:"-"` // JSON configuration
// Headers is an http target's extra request headers, name to
// value, stored as JSON text. json:"-" for the same reason as URL:
// a header value is routinely an authorization token.
Headers map[string]string `gorm:"serializer:json" json:"-"`
// TimeoutSeconds is an http target's per-request timeout; 0 uses
// the delivery engine's shared client timeout.
TimeoutSeconds int `json:"timeoutSeconds,omitempty"`
// ForwardQuery passes each event's query string on to an http
// target, appended to URL. Off, URL is sent exactly as stored.
ForwardQuery bool `json:"forwardQuery,omitempty"`
// ArchiveExpiry is how long a database target keeps each archived
// event; 0 keeps them forever.
ArchiveExpiry time.Duration `json:"archiveExpiry,omitempty"`
// ArchiveRotation is how often a database target starts a new
// archive file: none, monthly, daily or hourly.
ArchiveRotation string `json:"archiveRotation,omitempty"`
// MaxAttempts is how many times an HTTP or Slack target attempts // MaxAttempts is how many times an HTTP or Slack target attempts
// each delivery in all, at least 1. 1 is a single attempt with no // each delivery in all, at least 1. 1 is a single attempt with no
internal/database/model_target_test.go
+1 -20
View File
@@ -16,7 +16,7 @@ func newHTTPTarget(maxAttempts int) *database.Target {
WebhookID: uuid.New().String(), WebhookID: uuid.New().String(),
Name: "target", Name: "target",
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
URL: "https://example.com/", Config: `{"url":"https://example.com/"}`,
MaxAttempts: maxAttempts, MaxAttempts: maxAttempts,
} }
} }
@@ -42,22 +42,3 @@ func TestTargetCreate_AcceptsOneAttempt(t *testing.T) {
require.NoError(t, db.Create(newHTTPTarget(1)).Error) require.NoError(t, db.Create(newHTTPTarget(1)).Error)
} }
// TestTargetHeaders_StoredAsJSON reads the headers column back as
// text, to show it holds the header map as JSON.
func TestTargetHeaders_StoredAsJSON(t *testing.T) {
t.Parallel()
db := startedTestDB(t)
target := newHTTPTarget(1)
target.Headers = map[string]string{"Authorization": "Bearer x"}
require.NoError(t, db.Create(target).Error)
var headers string
require.NoError(t, db.Raw(
"SELECT headers FROM targets WHERE id = ?", target.ID,
).Scan(&headers).Error)
assert.JSONEq(t, `{"Authorization":"Bearer x"}`, headers)
}
internal/database/retention.go
+2 -3
View File
@@ -184,9 +184,8 @@ func (r *RetentionReaper) sweep(ctx context.Context) {
wh := webhooks[i] wh := webhooks[i]
// A missing database has nothing to reap. The delivery // A missing database has nothing to reap. Restart recovery
// engine's recovery routine reports a lost one at start (see // reports a lost one (see WebhookDBManager.GetDB).
// WebhookDBManager.GetDB).
if !r.dbManager.DBExists(wh.ID) { if !r.dbManager.DBExists(wh.ID) {
continue continue
} }
internal/database/schema_test.go
+7 -12
View File
@@ -87,18 +87,13 @@ func TestMainSchemaHoldsEveryModel(t *testing.T) {
Active: true, Active: true,
}) })
assertRoundTrip(t, db, &database.Target{ assertRoundTrip(t, db, &database.Target{
BaseModel: newBaseModel(), BaseModel: newBaseModel(),
WebhookID: uuid.New().String(), WebhookID: uuid.New().String(),
Name: "target", Name: "target",
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Active: true, Active: true,
URL: "https://example.com/", Config: `{"url":"https://example.com/"}`,
Headers: map[string]string{"X-Api-Key": "key"}, MaxAttempts: 3,
TimeoutSeconds: 30,
ForwardQuery: true,
ArchiveExpiry: 720 * time.Hour,
ArchiveRotation: "daily",
MaxAttempts: 3,
}) })
} }
internal/db/migrations/main/001_schema.sql
+1 -6
View File
@@ -79,12 +79,7 @@ CREATE TABLE targets (
name text NOT NULL, name text NOT NULL,
type text NOT NULL, type text NOT NULL,
active numeric DEFAULT true, active numeric DEFAULT true,
url text, config text,
headers text,
timeout_seconds integer,
forward_query numeric,
archive_expiry integer,
archive_rotation text,
max_attempts integer NOT NULL, max_attempts integer NOT NULL,
PRIMARY KEY (id), PRIMARY KEY (id),
CONSTRAINT fk_webhooks_targets CONSTRAINT fk_webhooks_targets
internal/delivery/client_ssrf_test.go
+19 -3
View File
@@ -46,7 +46,12 @@ func TestClientForRequest_TimeoutKeepsSSRFGuard(t *testing.T) {
t.Run(target, func(t *testing.T) { t.Run(target, func(t *testing.T) {
t.Parallel() t.Parallel()
client := engine.ExportClientForRequest(5, nil) cfg := &delivery.HTTPTargetConfig{
URL: target,
Timeout: 5,
}
client := engine.ExportClientForRequest(cfg, nil)
require.NotSame(t, engine.ExportClient(), client, require.NotSame(t, engine.ExportClient(), client,
"a per-target timeout must yield a "+ "a per-target timeout must yield a "+
@@ -96,7 +101,11 @@ func TestClientForRequest_NoTimeoutUnchanged(t *testing.T) {
engine := newSSRFTestEngine() engine := newSSRFTestEngine()
client := engine.ExportClientForRequest(0, nil) cfg := &delivery.HTTPTargetConfig{
URL: "https://example.com/hook",
}
client := engine.ExportClientForRequest(cfg, nil)
assert.Same(t, engine.ExportClient(), client, assert.Same(t, engine.ExportClient(), client,
"without a per-target timeout the shared client "+ "without a per-target timeout the shared client "+
@@ -114,8 +123,15 @@ func TestClientForRequest_HeadersKeepSSRFGuard(t *testing.T) {
engine := newSSRFTestEngine() engine := newSSRFTestEngine()
cfg := &delivery.HTTPTargetConfig{
URL: "https://example.com/with-headers",
Headers: map[string]string{
"X-Api-Key": "configured",
},
}
client := engine.ExportClientForRequest( client := engine.ExportClientForRequest(
0, []string{probeHeaderName}, cfg, []string{"X-Api-Key"},
) )
require.NotNil(t, client.CheckRedirect, require.NotNil(t, client.CheckRedirect,
internal/delivery/engine.go
+454 -255
View File
@@ -1,5 +1,5 @@
// Package delivery stores each event with its deliveries and delivers // Package delivery manages asynchronous event delivery
// them, asynchronously, to configured targets. // to configured targets.
package delivery package delivery
import ( import (
@@ -23,11 +23,10 @@ import (
const ( const (
// deliveryChannelSize is the buffer size for the delivery // deliveryChannelSize is the buffer size for the delivery
// channel. The Tasks of newly stored deliveries, and of the // channel. New Tasks from the webhook handler are sent
// pending deliveries the recovery routine hands back, are // here. Workers drain this channel. Sized large enough
// sent here. Workers drain this channel. Sized large enough // that the webhook handler should never block under
// that no task should be dropped under normal load: one that // normal load.
// does not fit is left for the sweep.
deliveryChannelSize = 10000 deliveryChannelSize = 10000
// retryChannelSize is the buffer size for the retry // retryChannelSize is the buffer size for the retry
@@ -41,10 +40,9 @@ const (
// explosions regardless of queue depth. // explosions regardless of queue depth.
defaultWorkers = 10 defaultWorkers = 10
// sweepInterval is how often the recovery routine runs // retrySweepInterval is how often the periodic retry
// once it has run at start. Each of those runs is a tick of // sweep runs.
// the sweep. retrySweepInterval = 60 * time.Second
sweepInterval = 60 * time.Second
// pendingSweepMinAge is how long a delivery must have sat // pendingSweepMinAge is how long a delivery must have sat
// untouched at pending before the sweep will look at it. // untouched at pending before the sweep will look at it.
@@ -65,10 +63,10 @@ const (
pendingSweepMinAge = 15 * time.Minute pendingSweepMinAge = 15 * time.Minute
// pendingSweepBatch bounds how many stranded pending deliveries // pendingSweepBatch bounds how many stranded pending deliveries
// one tick of the sweep re-dispatches for one webhook. The sweep // one sweep of one webhook re-dispatches. The sweep runs every
// ticks every sweepInterval, so a larger backlog drains // retrySweepInterval, so a larger backlog drains across
// across successive ticks instead of arriving as one burst against // successive sweeps instead of arriving as one burst against a
// a database that was already struggling to accept writes. // database that was already struggling to accept writes.
pendingSweepBatch = 500 pendingSweepBatch = 500
// MaxInlineBodySize is the maximum event body size that // MaxInlineBodySize is the maximum event body size that
@@ -106,19 +104,12 @@ type Task struct {
WebhookID string WebhookID string
EntrypointID string EntrypointID string
// The target's fields, as they were when the task was made. TargetID string
TargetID string TargetName string
TargetName string TargetType database.TargetType
TargetType database.TargetType TargetConfig string
TargetURL string MaxAttempts int
TargetHeaders map[string]string
TimeoutSeconds int
ForwardQuery bool
ArchiveExpiry time.Duration
ArchiveRotation string
MaxAttempts int
// The event's fields.
Method string Method string
RawQuery string RawQuery string
Headers string Headers string
@@ -128,11 +119,17 @@ type Task struct {
AttemptNum int AttemptNum int
} }
// Notifier is the interface for notifying the delivery
// engine about new deliveries.
type Notifier interface {
Notify(tasks []Task)
}
// CircuitBreakers is how the handlers read a target's circuit // CircuitBreakers is how the handlers read a target's circuit
// breaker, so the webhook page and the event log can say that // breaker, so the webhook page and the event log can say that
// deliveries to the target are paused and until when. The handlers // deliveries to the target are paused and until when. It keeps the
// hold the engine itself as well; they read breakers through this // handlers free of the engine's internals and is trivially faked in
// interface so that a test can make one read as open. // tests.
type CircuitBreakers interface { type CircuitBreakers interface {
StateAndCooldown(targetID string) (CircuitState, time.Duration) StateAndCooldown(targetID string) (CircuitState, time.Duration)
} }
@@ -152,13 +149,12 @@ type EngineParams struct {
} }
// Engine processes queued deliveries in the background // Engine processes queued deliveries in the background
// using a bounded worker pool architecture. It also stores each // using a bounded worker pool architecture. It owns only
// event with its deliveries and creates replay deliveries (see // the cross-target machinery: the worker pool, the queue
// store.go). Beyond that it owns only the cross-target machinery: // and retry channels, restart recovery, and the persistence
// the worker pool, the delivery and retry channels, the recovery // helpers and Scheduler that individual targets rely on.
// routine, and the persistence helpers and Scheduler that // Each target type owns its own delivery, including retries,
// individual targets rely on. Each target type owns its own // backoff, and circuit breaking.
// delivery, including retries, backoff, and circuit breaking.
type Engine struct { type Engine struct {
database *database.Database database *database.Database
dbManager *database.WebhookDBManager dbManager *database.WebhookDBManager
@@ -194,7 +190,7 @@ type Engine struct {
archives *archive.Writers archives *archive.Writers
// inflight is the set of deliveries this engine currently owns. // inflight is the set of deliveries this engine currently owns.
// The recovery routine re-dispatches only what it does not // Recovery and the sweeps re-dispatch only what it does not
// hold. Held by value: its zero value works, so no constructor // hold. Held by value: its zero value works, so no constructor
// can leave it out. See inflight.go. // can leave it out. See inflight.go.
inflight inflightSet inflight inflightSet
@@ -228,6 +224,38 @@ func New(
return e return e
} }
// Notify signals the delivery engine that new deliveries
// are ready.
func (e *Engine) Notify(tasks []Task) {
for i := range tasks {
// Owned before it is queued, and until the worker that runs
// it returns. A task can sit in a 10000-deep channel for a
// long time on a healthy system, and nothing may re-send it
// while it waits. See inflight.go.
if !e.inflight.retainIdle(tasks[i].DeliveryID) {
e.log.Warn(
"delivery already in flight, not queued again",
"delivery_id", tasks[i].DeliveryID,
"event_id", tasks[i].EventID,
)
continue
}
select {
case e.deliveryCh <- tasks[i]:
default:
e.inflight.release(tasks[i].DeliveryID)
e.log.Warn(
"delivery channel full, "+
"task will be recovered by the sweep",
"delivery_id", tasks[i].DeliveryID,
"event_id", tasks[i].EventID,
)
}
}
}
// StateAndCooldown implements CircuitBreakers. It returns the state of // StateAndCooldown implements CircuitBreakers. It returns the state of
// the target's circuit breaker and, while the breaker is open, what is // the target's circuit breaker and, while the breaker is open, what is
// left of its cooldown; the cooldown is zero once that has passed and // left of its cooldown; the cooldown is zero once that has passed and
@@ -288,38 +316,6 @@ func (e *Engine) ScheduleRetry(
}) })
} }
// notify queues the tasks of deliveries already stored. StoreEvent and
// CreateReplay call it once their transaction has committed.
func (e *Engine) notify(tasks []Task) {
for i := range tasks {
// Owned before it is queued, and until the worker that runs
// it returns. A task can sit in a 10000-deep channel for a
// long time on a healthy system, and nothing may re-send it
// while it waits. See inflight.go.
if !e.inflight.retainIdle(tasks[i].DeliveryID) {
e.log.Warn(
"delivery already in flight, not queued again",
"delivery_id", tasks[i].DeliveryID,
"event_id", tasks[i].EventID,
)
continue
}
select {
case e.deliveryCh <- tasks[i]:
default:
e.inflight.release(tasks[i].DeliveryID)
e.log.Warn(
"delivery channel full, "+
"task will be recovered by the sweep",
"delivery_id", tasks[i].DeliveryID,
"event_id", tasks[i].EventID,
)
}
}
}
// userAgent is the User-Agent header of every http and slack // userAgent is the User-Agent header of every http and slack
// delivery request: the program name and the running build's // delivery request: the program name and the running build's
// version. // version.
@@ -346,8 +342,8 @@ func (e *Engine) registerHooks(lc fx.Lifecycle) {
}) })
} }
// start launches the worker pool and the recovery routine, which // start launches the worker pool, restart recovery, and the
// runs now and then on every tick of the sweep (see runRecovery). // periodic retry sweep.
// //
// Their context is derived from context.Background(), NOT from // Their context is derived from context.Background(), NOT from
// the fx OnStart hook context. The hook context carries fx's // the fx OnStart hook context. The hook context carries fx's
@@ -370,7 +366,11 @@ func (e *Engine) start() {
e.wg.Add(1) e.wg.Add(1)
go e.runRecovery(ctx) go e.recoverPending(ctx)
e.wg.Add(1)
go e.retrySweep(ctx)
e.wg.Add(1) e.wg.Add(1)
@@ -453,6 +453,12 @@ func (e *Engine) runTask(
run(ctx, task) run(ctx, task)
} }
func (e *Engine) recoverPending(ctx context.Context) {
defer e.wg.Done()
e.recoverInFlight(ctx)
}
func (e *Engine) processNewTask( func (e *Engine) processNewTask(
ctx context.Context, task *Task, ctx context.Context, task *Task,
) { ) {
@@ -467,10 +473,9 @@ func (e *Engine) processNewTask(
return return
} }
// The recovery routine's run at start can send and release this // Restart recovery can send and release this delivery before the
// delivery before StoreEvent or CreateReplay queues it. Ownership // receiver's Notify queues it. Ownership cannot refuse a delivery
// cannot refuse a delivery nobody holds, so the row decides whether // nobody holds, so the row decides whether it still needs sending.
// it still needs sending.
row, err := e.loadDelivery(webhookDB, task.DeliveryID) row, err := e.loadDelivery(webhookDB, task.DeliveryID)
if err != nil { if err != nil {
e.log.Error( e.log.Error(
@@ -600,9 +605,9 @@ func (e *Engine) processRetryTask(
// else on this path reads the target row. Without this check a // else on this path reads the target row. Without this check a
// deletion stops nothing: the timer keeps firing and keeps sending to // deletion stops nothing: the timer keeps firing and keeps sending to
// the destination the operator removed, for the whole remaining // the destination the operator removed, for the whole remaining
// backoff chain. Terminalising in the recovery routine alone is not // backoff chain. Terminalising in the recovery and sweep paths alone
// enough, because it only sees the delivery once nothing holds it in // is not enough, because those only see the delivery once nothing
// memory — which is to say after a restart. // holds it in memory — which is to say after a restart.
// //
// The worker already owns this delivery, so the terminal write happens // The worker already owns this delivery, so the terminal write happens
// here directly, exactly as a target's own Deliver fails one. Claiming // here directly, exactly as a target's own Deliver fails one. Claiming
@@ -650,47 +655,7 @@ func (e *Engine) abandonRetryForMissingTarget(
return true return true
} }
// runRecovery runs the recovery routine, recoverDeliveries, once when func (e *Engine) recoverInFlight(ctx context.Context) {
// the engine starts and then on every tick of the sweep, every
// sweepInterval, until the engine stops.
func (e *Engine) runRecovery(ctx context.Context) {
defer e.wg.Done()
ticker := time.NewTicker(sweepInterval)
defer ticker.Stop()
e.recoverDeliveries(ctx, true)
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
e.recoverDeliveries(ctx, false)
}
}
}
// recoverDeliveries is the engine's one recovery routine: atStart is
// true for its run when the engine starts and false on a tick of the
// sweep.
//
// For every webhook it finds the pending and retrying deliveries the
// engine is not working on, such as those a stop or a crash cut off, a
// full channel dropped, or a failed bookkeeping write left behind. One
// that already has a successful attempt is marked delivered. A retrying
// one is scheduled after what is left of its backoff, or now when
// nothing is left. A pending one goes to a worker now.
//
// Ownership keeps it off a delivery the engine is working on, has
// queued, or holds a retry timer for, the same at start as on a tick:
// every delivery it acts on is taken through the inflight set first.
// See inflight.go.
//
// A run at start differs from a tick in which webhooks it opens (see
// recoverWebhookDeliveries) and which pending deliveries it takes (see
// recoverPendingDeliveries).
func (e *Engine) recoverDeliveries(ctx context.Context, atStart bool) {
var webhookIDs []string var webhookIDs []string
err := e.database.DB(). err := e.database.DB().
@@ -712,27 +677,18 @@ func (e *Engine) recoverDeliveries(ctx context.Context, atStart bool) {
default: default:
} }
e.recoverWebhookDeliveries(ctx, webhookID, atStart) // Opened even when its file is missing, so that a lost
// database is reported at start, not when the webhook next
// receives an event, which for a quiet webhook may be never.
e.recoverWebhookDeliveries(ctx, webhookID)
} }
} }
// recoverWebhookDeliveries runs the recovery routine over one webhook.
//
// At start the webhook's database is opened even when its file is
// missing, so that a lost database is reported at start, not when the
// webhook next receives an event, which for a quiet webhook may be
// never. A tick leaves a missing file alone rather than create one: a
// webhook being created has its row saved a moment before its file,
// and a tick in between would report that file as lost.
func (e *Engine) recoverWebhookDeliveries( func (e *Engine) recoverWebhookDeliveries(
ctx context.Context, webhookID string, atStart bool, ctx context.Context, webhookID string,
) { ) {
if !atStart && !e.dbManager.DBExists(webhookID) { // The web interface is already serving, so the webhook may have
return // been deleted since the list was read. Opening its database then
}
// The web interface is serving, so the webhook may have been
// deleted since the list was read. Opening its database then
// would create the file again after the delete removed it. // would create the file again after the delete removed it.
stillExists := func() (bool, error) { stillExists := func() (bool, error) {
var count int64 var count int64
@@ -764,18 +720,16 @@ func (e *Engine) recoverWebhookDeliveries(
} }
e.recoverPendingDeliveries( e.recoverPendingDeliveries(
ctx, webhookDB, webhookID, atStart, ctx, webhookDB, webhookID,
) )
e.recoverRetryingDeliveries( e.recoverRetryingDeliveries(
ctx, webhookDB, webhookID, webhookDB, webhookID,
) )
} }
// recoverRetryingDeliveries recovers every retrying delivery of one
// webhook, at start and on a tick alike.
func (e *Engine) recoverRetryingDeliveries( func (e *Engine) recoverRetryingDeliveries(
ctx context.Context, webhookDB *gorm.DB, webhookID string, webhookDB *gorm.DB, webhookID string,
) { ) {
var retrying []database.Delivery var retrying []database.Delivery
@@ -802,12 +756,6 @@ func (e *Engine) recoverRetryingDeliveries(
) )
for i := range retrying { for i := range retrying {
select {
case <-ctx.Done():
return
default:
}
if _, ok := settled[retrying[i].ID]; ok { if _, ok := settled[retrying[i].ID]; ok {
continue continue
} }
@@ -819,9 +767,8 @@ func (e *Engine) recoverRetryingDeliveries(
} }
// recoverSingleRetry hands an orphaned retrying delivery back // recoverSingleRetry hands an orphaned retrying delivery back
// to its target to work out what is left of its backoff, then // to its target to recompute the remaining backoff, then
// schedules its next attempt after that, or now when nothing is // reschedules it. Targets that do not own durable retries
// left. Targets that do not own durable retries
// (fire-and-forget) never produce retrying deliveries, so a // (fire-and-forget) never produce retrying deliveries, so a
// delivery found in that state has had its target's type // delivery found in that state has had its target's type
// changed underneath it and is terminally failed. // changed underneath it and is terminally failed.
@@ -869,19 +816,6 @@ func (e *Engine) recoverSingleRetry(
webhookDB, d.ID, attemptNum, webhookDB, d.ID, attemptNum,
) )
// Taken before the event is read, so that a delivery the engine
// already holds, most often one waiting out its backoff on a
// timer, costs a tick no read of its body.
if !e.takeForRedispatch(
webhookDB, d.ID, database.DeliveryStatusRetrying,
) {
return
}
// ScheduleRetry takes a reference of its own before it returns,
// and that one keeps the delivery owned through the backoff.
defer e.inflight.release(d.ID)
event, err := e.loadEvent(webhookDB, d.EventID) event, err := e.loadEvent(webhookDB, d.EventID)
if err != nil { if err != nil {
e.log.Error( e.log.Error(
@@ -895,9 +829,13 @@ func (e *Engine) recoverSingleRetry(
return return
} }
e.ScheduleRetry(buildTask( task := buildRecoveryTask(
d, webhookID, &event, &target, attemptNum+1, d, webhookID, &event, &target, attemptNum+1,
), remaining) )
if !e.rescheduleRecovered(webhookDB, task, remaining) {
return
}
e.log.Info( e.log.Info(
"recovering retrying delivery", "recovering retrying delivery",
@@ -908,54 +846,28 @@ func (e *Engine) recoverSingleRetry(
) )
} }
// recoverPendingDeliveries recovers the pending deliveries of one
// webhook: it marks delivered those that already have a successful
// attempt and sends the rest.
//
// A delivery is created pending and leaves that state only when its
// outcome is written, so a pending row the engine does not own is one
// it stopped working on before then: for example one whose bookkeeping
// write failed, whose attempt a stop cut off, that was still queued
// when the engine last stopped, whose task was dropped because the
// delivery channel was full (by notify or by queueRecovered), or whose
// worker could not open the webhook's database.
//
// At start every pending delivery is taken, however recent, so one a
// stop cut off moments ago is sent again now. A tick takes only those
// untouched for pendingSweepMinAge, at most pendingSweepBatch of them.
// The age bound is not what keeps a tick off live work: a delivery
// waiting in deliveryCh is pending and arbitrarily old. Ownership is,
// checked per delivery in takeForRedispatch. See inflight.go.
func (e *Engine) recoverPendingDeliveries( func (e *Engine) recoverPendingDeliveries(
ctx context.Context, ctx context.Context,
webhookDB *gorm.DB, webhookDB *gorm.DB,
webhookID string, webhookID string,
atStart bool,
) { ) {
query := webhookDB.Where(
"status = ?", database.DeliveryStatusPending,
)
if !atStart {
query = query.
Where(
"updated_at < ?",
time.Now().Add(-pendingSweepMinAge),
).
Limit(pendingSweepBatch)
}
var deliveries []database.Delivery var deliveries []database.Delivery
// No Preload: event bodies are read one at a time in // No Preload: event bodies are read one at a time in
// sendRecoveredDeliveries, and only for the deliveries actually // sendRecoveredDeliveries, and only for the deliveries actually
// being sent. // being sent.
err := query.Find(&deliveries).Error result := webhookDB.
if err != nil { Where(
"status = ?",
database.DeliveryStatusPending,
).
Find(&deliveries)
if result.Error != nil {
e.log.Error( e.log.Error(
"failed to query pending deliveries for recovery", "failed to query pending deliveries",
"webhook_id", webhookID, "webhook_id", webhookID,
"error", err, "error", result.Error,
) )
return return
@@ -971,6 +883,21 @@ func (e *Engine) recoverPendingDeliveries(
"count", len(deliveries), "count", len(deliveries),
) )
e.recoverPendingBatch(
ctx, webhookDB, webhookID, deliveries,
)
}
// recoverPendingBatch settles every delivery in the batch that was
// already delivered, and re-dispatches only the rest. Both the
// restart-time recovery and the periodic sweep go through it, so a
// pending delivery is treated the same however it was found.
func (e *Engine) recoverPendingBatch(
ctx context.Context,
webhookDB *gorm.DB,
webhookID string,
deliveries []database.Delivery,
) {
targetMap := e.loadTargetMap(deliveries) targetMap := e.loadTargetMap(deliveries)
settled := e.reconcileDelivered( settled := e.reconcileDelivered(
@@ -998,11 +925,11 @@ func (e *Engine) recoverPendingDeliveries(
// wire I/O happened and was recorded, and all that is missing is the // wire I/O happened and was recorded, and all that is missing is the
// status. // status.
// //
// The recovery routine runs this for retrying deliveries as well as // Every recovery path runs this, not only the pending one. A delivery
// pending ones. A delivery abandoned at retrying can hold a successful // abandoned at retrying can hold a successful result just as a pending
// result just as a pending one can — a second attempt that reached the // one can — a second attempt that reached the receiver and whose status
// receiver and whose status write then failed sits at retrying with // write then failed sits at retrying with success recorded — and
// success recorded — and re-sending it is the same duplicate. // re-sending it is the same duplicate.
// //
// Deliveries whose result row itself never landed are not in the // Deliveries whose result row itself never landed are not in the
// returned set and are re-sent, recorded as the further attempt they // returned set and are re-sent, recorded as the further attempt they
@@ -1089,8 +1016,247 @@ func (e *Engine) reconcileDelivered(
return settled return settled
} }
func (e *Engine) retrySweep(ctx context.Context) {
defer e.wg.Done()
ticker := time.NewTicker(retrySweepInterval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
e.sweepOrphanedRetries(ctx)
}
}
}
func (e *Engine) sweepOrphanedRetries(
ctx context.Context,
) {
var webhookIDs []string
err := e.database.DB().
Model(&database.Webhook{}).
Pluck("id", &webhookIDs).Error
if err != nil {
e.log.Error(
"retry sweep: failed to query webhook IDs",
"error", err,
)
return
}
for _, webhookID := range webhookIDs {
select {
case <-ctx.Done():
return
default:
}
if !e.dbManager.DBExists(webhookID) {
continue
}
e.sweepWebhookRetries(ctx, webhookID)
}
}
func (e *Engine) sweepWebhookRetries(
ctx context.Context, webhookID string,
) {
webhookDB, err := e.dbManager.GetDB(webhookID)
if err != nil {
e.log.Error(
"retry sweep: failed to get webhook database",
"webhook_id", webhookID,
"error", err,
)
return
}
var retrying []database.Delivery
err = webhookDB.
Where(
"status = ?",
database.DeliveryStatusRetrying,
).
Find(&retrying).Error
if err != nil {
e.log.Error(
"retry sweep: "+
"failed to query retrying deliveries",
"webhook_id", webhookID,
"error", err,
)
return
}
settled := e.reconcileDelivered(
webhookDB, webhookID, retrying,
e.loadTargetMap(retrying),
)
for i := range retrying {
select {
case <-ctx.Done():
return
default:
}
if _, ok := settled[retrying[i].ID]; ok {
continue
}
e.sweepSingleRetry(
webhookDB, webhookID, &retrying[i],
)
}
e.sweepWebhookPending(ctx, webhookDB, webhookID)
}
// sweepWebhookPending recovers deliveries stranded at pending.
//
// A delivery is created pending and leaves that state only when its
// outcome is written, so a pending row the engine does not own is one
// whose bookkeeping write failed — the state that used to sit there
// until a restart, and then produce a duplicate at the receiver. The
// sweep gives it the same reconcile-then-dispatch treatment restart
// recovery gets, so it costs a minute rather than an operator
// noticing.
//
// What keeps the sweep off live work is ownership, checked per
// delivery in takeForRedispatch, not the age bound in this query.
// A delivery waiting in deliveryCh is pending and arbitrarily old —
// the channel holds 10000 tasks and 10 workers drain it — so
// reasoning from the row's age alone re-sends it. See inflight.go.
func (e *Engine) sweepWebhookPending(
ctx context.Context,
webhookDB *gorm.DB,
webhookID string,
) {
var pending []database.Delivery
err := webhookDB.
Where(
"status = ? AND updated_at < ?",
database.DeliveryStatusPending,
time.Now().Add(-pendingSweepMinAge),
).
Limit(pendingSweepBatch).
Find(&pending).Error
if err != nil {
e.log.Error(
"retry sweep: "+
"failed to query pending deliveries",
"webhook_id", webhookID,
"error", err,
)
return
}
if len(pending) == 0 {
return
}
e.log.Info(
"retry sweep: recovering stranded pending deliveries",
"webhook_id", webhookID,
"count", len(pending),
)
e.recoverPendingBatch(ctx, webhookDB, webhookID, pending)
}
// sweepSingleRetry re-enqueues an orphaned retrying delivery
// whose backoff window has elapsed, delegating the backoff
// decision to the delivery's target. A delivery whose target
// no longer owns durable retries is terminally failed.
func (e *Engine) sweepSingleRetry(
webhookDB *gorm.DB,
webhookID string,
d *database.Delivery,
) {
target, err := e.loadTarget(d.TargetID)
if err != nil {
// Deleted is terminal, unreadable is not; see
// recoverSingleRetry.
if errors.Is(err, gorm.ErrRecordNotFound) {
e.failMissingTarget(webhookDB, webhookID, d)
return
}
e.log.Error(
"retry sweep: failed to load target",
"delivery_id", d.ID,
"target_id", d.TargetID,
"error", err,
)
return
}
rs, ok := e.targets[target.Type].(rescheduler)
if !ok {
e.failUnretryableRetry(
webhookDB, webhookID, d, &target,
)
return
}
attemptNum := e.countAttempts(webhookDB, d.ID)
if !rs.backoffElapsed(
webhookDB, d.ID, attemptNum,
) {
return
}
event, err := e.loadEvent(webhookDB, d.EventID)
if err != nil {
e.log.Error(
"retry sweep: failed to load event",
"delivery_id", d.ID,
"event_id", d.EventID,
"error", err,
)
return
}
task := buildRecoveryTask(
d, webhookID, &event, &target, attemptNum+1,
)
if !e.redispatch(
e.retryCh, webhookDB, task,
database.DeliveryStatusRetrying,
) {
return
}
e.log.Info(
"retry sweep: "+
"recovered orphaned retrying delivery",
"delivery_id", d.ID,
"webhook_id", webhookID,
"attempt", attemptNum+1,
)
}
// failUnretryableRetry terminally fails an orphaned retrying // failUnretryableRetry terminally fails an orphaned retrying
// delivery whose target type no longer supports retries. // delivery whose target type no longer supports retries. Both
// restart recovery and the periodic sweep call it, so the
// terminal transition exists once.
// //
// This is only reachable when a target's type has been changed // This is only reachable when a target's type has been changed
// out from under an in-flight retrying delivery (or the type is // out from under an in-flight retrying delivery (or the type is
@@ -1138,8 +1304,8 @@ func (e *Engine) failUnretryableRetry(
} }
// failMissingTarget terminally fails a recovered delivery, pending or // failMissingTarget terminally fails a recovered delivery, pending or
// retrying, whose target row is gone. The recovery routine calls it for // retrying, whose target row is gone. Restart recovery and the periodic
// both statuses. // sweep call it for both statuses, so the transition exists once.
// //
// Until it existed those paths logged the failed lookup and moved on, // Until it existed those paths logged the failed lookup and moved on,
// which left the delivery where it was for the life of the database and // which left the delivery where it was for the life of the database and
@@ -1546,15 +1712,10 @@ func buildEventFromTask(task *Task) database.Event {
func buildTargetFromTask(task *Task) database.Target { func buildTargetFromTask(task *Task) database.Target {
target := database.Target{ target := database.Target{
Name: task.TargetName, Name: task.TargetName,
Type: task.TargetType, Type: task.TargetType,
URL: task.TargetURL, Config: task.TargetConfig,
Headers: task.TargetHeaders, MaxAttempts: task.MaxAttempts,
TimeoutSeconds: task.TimeoutSeconds,
ForwardQuery: task.ForwardQuery,
ArchiveExpiry: task.ArchiveExpiry,
ArchiveRotation: task.ArchiveRotation,
MaxAttempts: task.MaxAttempts,
} }
target.ID = task.TargetID target.ID = task.TargetID
@@ -1669,11 +1830,11 @@ func (e *Engine) countAttempts(
// answer cannot go stale between asking and acting. // answer cannot go stale between asking and acting.
// //
// Stamping updated_at is the same statement, and it is a cadence // Stamping updated_at is the same statement, and it is a cadence
// control rather than a claim: a tick of the sweep selects pending // control rather than a claim: the pending sweep selects on that
// deliveries on that column, so one handed out now is not selected // column, so a delivery handed out now is not selected again on the
// again on the next tick a minute later but after pendingSweepMinAge. // next tick a minute later but after pendingSweepMinAge. A delivery
// A delivery the database refuses to settle is therefore retried on // the database refuses to settle is therefore retried on that
// that interval instead of every tick. // interval instead of every tick.
// //
// A failed write is a refusal. It means the database is not accepting // A failed write is a refusal. It means the database is not accepting
// writes, which is the condition that stranded the delivery in the // writes, which is the condition that stranded the delivery in the
@@ -1721,11 +1882,14 @@ func (e *Engine) takeForRedispatch(
return true return true
} }
// queueRecovered puts an owned pending delivery's task on the delivery // queueRecovered puts an owned delivery's task on a worker channel,
// channel, dropping the ownership the gate took if it does not fit. // dropping the ownership the gate took if it does not fit.
func (e *Engine) queueRecovered(task Task) { func (e *Engine) queueRecovered(
ch chan<- Task, task Task,
) bool {
select { select {
case e.deliveryCh <- task: case ch <- task:
return true
default: default:
e.inflight.release(task.DeliveryID) e.inflight.release(task.DeliveryID)
e.log.Warn( e.log.Warn(
@@ -1734,9 +1898,53 @@ func (e *Engine) queueRecovered(task Task) {
"delivery_id", task.DeliveryID, "delivery_id", task.DeliveryID,
"webhook_id", task.WebhookID, "webhook_id", task.WebhookID,
) )
return false
} }
} }
// redispatch hands a recovered delivery to a worker channel through
// takeForRedispatch, and reports whether the task was queued.
func (e *Engine) redispatch(
ch chan<- Task,
webhookDB *gorm.DB,
task Task,
eligible database.DeliveryStatus,
) bool {
if !e.takeForRedispatch(
webhookDB, task.DeliveryID, eligible,
) {
return false
}
return e.queueRecovered(ch, task)
}
// rescheduleRecovered hands an orphaned retrying delivery back to the
// retry timer, through the same gate. It reports whether the delivery
// was rescheduled.
//
// The reference taken by the gate is dropped as soon as ScheduleRetry
// has taken its own, which it does before returning: what keeps the
// delivery owned through the backoff window is ScheduleRetry's
// reference, not this one.
func (e *Engine) rescheduleRecovered(
webhookDB *gorm.DB, task Task, delay time.Duration,
) bool {
if !e.takeForRedispatch(
webhookDB, task.DeliveryID,
database.DeliveryStatusRetrying,
) {
return false
}
defer e.inflight.release(task.DeliveryID)
e.ScheduleRetry(task, delay)
return true
}
// countAttemptsBatch counts the recorded attempts of every delivery // countAttemptsBatch counts the recorded attempts of every delivery
// in a batch with one grouped query, keyed by delivery id. Deliveries // in a batch with one grouped query, keyed by delivery id. Deliveries
// with no attempts are simply absent from the result, which reads back // with no attempts are simply absent from the result, which reads back
@@ -1818,11 +2026,7 @@ func (e *Engine) loadTarget(
return target, nil return target, nil
} }
// buildTask builds the task that carries delivery d of event to target func buildRecoveryTask(
// through the engine's channels, as attempt attemptNum. The body is
// carried only when it is under MaxInlineBodySize; a larger one is read
// from the event row when the attempt runs (see hydrateEvent).
func buildTask(
d *database.Delivery, d *database.Delivery,
webhookID string, webhookID string,
event *database.Event, event *database.Event,
@@ -1837,26 +2041,21 @@ func buildTask(
} }
return Task{ return Task{
DeliveryID: d.ID, DeliveryID: d.ID,
EventID: d.EventID, EventID: d.EventID,
WebhookID: webhookID, WebhookID: webhookID,
EntrypointID: event.EntrypointID, EntrypointID: event.EntrypointID,
TargetID: target.ID, TargetID: target.ID,
TargetName: target.Name, TargetName: target.Name,
TargetType: target.Type, TargetType: target.Type,
TargetURL: target.URL, TargetConfig: target.Config,
TargetHeaders: target.Headers, MaxAttempts: target.MaxAttempts,
TimeoutSeconds: target.TimeoutSeconds, Method: event.Method,
ForwardQuery: target.ForwardQuery, RawQuery: event.RawQuery,
ArchiveExpiry: target.ArchiveExpiry, Headers: event.Headers,
ArchiveRotation: target.ArchiveRotation, ContentType: event.ContentType,
MaxAttempts: target.MaxAttempts, Body: bodyPtr,
Method: event.Method, AttemptNum: attemptNum,
RawQuery: event.RawQuery,
Headers: event.Headers,
ContentType: event.ContentType,
Body: bodyPtr,
AttemptNum: attemptNum,
} }
} }
@@ -1998,11 +2197,11 @@ func (e *Engine) sendRecoveredDeliveries(
continue continue
} }
task := buildTask( task := buildRecoveryTask(
&deliveries[i], webhookID, &event, &target, &deliveries[i], webhookID, &event, &target,
attempts[deliveries[i].ID]+1, attempts[deliveries[i].ID]+1,
) )
e.queueRecovered(task) e.queueRecovered(e.deliveryCh, task)
} }
} }
internal/delivery/engine_integration_test.go
+170 -119
View File
@@ -3,6 +3,7 @@ package delivery_test
import ( import (
"bytes" "bytes"
"context" "context"
"encoding/json"
"fmt" "fmt"
"io" "io"
"log/slog" "log/slog"
@@ -96,6 +97,17 @@ func iSeedWebhookDB(
return db return db
} }
func iHTTPConfig(url string) string {
cfg := delivery.HTTPTargetConfig{URL: url}
data, err := json.Marshal(cfg)
if err != nil {
panic("failed to marshal HTTPTargetConfig")
}
return string(data)
}
func iEngine( func iEngine(
t *testing.T, workers int, t *testing.T, workers int,
) *delivery.Engine { ) *delivery.Engine {
@@ -157,24 +169,24 @@ func iSeedDelivery(
func iTask( func iTask(
d database.Delivery, d database.Delivery,
event database.Event, event database.Event,
webhookID, targetID, name, targetURL string, webhookID, targetID, name, config string,
maxAttempts, attemptNum int, maxAttempts, attemptNum int,
body *string, body *string,
) delivery.Task { ) delivery.Task {
return delivery.Task{ return delivery.Task{
DeliveryID: d.ID, DeliveryID: d.ID,
EventID: event.ID, EventID: event.ID,
WebhookID: webhookID, WebhookID: webhookID,
TargetID: targetID, TargetID: targetID,
TargetName: name, TargetName: name,
TargetType: database.TargetTypeHTTP, TargetType: database.TargetTypeHTTP,
TargetURL: targetURL, TargetConfig: config,
MaxAttempts: maxAttempts, MaxAttempts: maxAttempts,
Method: event.Method, Method: event.Method,
Headers: event.Headers, Headers: event.Headers,
ContentType: event.ContentType, ContentType: event.ContentType,
Body: body, Body: body,
AttemptNum: attemptNum, AttemptNum: attemptNum,
} }
} }
@@ -227,9 +239,10 @@ func TestProcessNewTask_InlineBody(t *testing.T) {
) )
bodyStr := event.Body bodyStr := event.Body
cfg := iHTTPConfig(ts.URL)
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"test-target", ts.URL, 1, 1, &bodyStr, "test-target", cfg, 1, 1, &bodyStr,
) )
s.Engine.ExportProcessNewTask( s.Engine.ExportProcessNewTask(
@@ -277,9 +290,10 @@ func TestProcessNewTask_LargeBody_FetchFromDB(
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
cfg := iHTTPConfig(ts.URL)
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"test-large", ts.URL, 1, 1, nil, "test-large", cfg, 1, 1, nil,
) )
s.Engine.ExportProcessNewTask( s.Engine.ExportProcessNewTask(
@@ -299,16 +313,16 @@ func TestProcessNewTask_InvalidWebhookID(t *testing.T) {
s := newISetup(t) s := newISetup(t)
task := delivery.Task{ task := delivery.Task{
DeliveryID: uuid.New().String(), DeliveryID: uuid.New().String(),
EventID: uuid.New().String(), EventID: uuid.New().String(),
WebhookID: uuid.New().String(), WebhookID: uuid.New().String(),
TargetID: uuid.New().String(), TargetID: uuid.New().String(),
TargetName: "test", TargetName: "test",
TargetType: database.TargetTypeHTTP, TargetType: database.TargetTypeHTTP,
TargetURL: "http://localhost:9999", TargetConfig: iHTTPConfig("http://localhost:9999"),
MaxAttempts: 1, MaxAttempts: 1,
Body: nil, Body: nil,
AttemptNum: 1, AttemptNum: 1,
} }
s.Engine.ExportProcessNewTask( s.Engine.ExportProcessNewTask(
@@ -349,6 +363,7 @@ func TestProcessRetryTask_SuccessfulRetry(t *testing.T) {
) )
bodyStr := event.Body bodyStr := event.Body
cfg := iHTTPConfig(ts.URL)
// The target row exists because the engine confirms a scheduled // The target row exists because the engine confirms a scheduled
// retry's target has not been deleted before it runs it. A retry // retry's target has not been deleted before it runs it. A retry
@@ -357,12 +372,12 @@ func TestProcessRetryTask_SuccessfulRetry(t *testing.T) {
// task. See https://git.eeqj.de/sneak/webhooker/issues/107. // task. See https://git.eeqj.de/sneak/webhooker/issues/107.
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "retry-target", t, s.MainDB, targetID, s.WebhookID, "retry-target",
database.TargetTypeHTTP, ts.URL, 5, database.TargetTypeHTTP, cfg, 5,
) )
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"retry-target", ts.URL, 5, 2, &bodyStr, "retry-target", cfg, 5, 2, &bodyStr,
) )
s.Engine.ExportProcessRetryTask( s.Engine.ExportProcessRetryTask(
@@ -395,9 +410,10 @@ func TestProcessRetryTask_SkipsNonRetryingDelivery(
) )
bodyStr := event.Body bodyStr := event.Body
cfg := iHTTPConfig("http://localhost:1")
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"skip-target", "http://localhost:1", 5, 2, &bodyStr, "skip-target", cfg, 5, 2, &bodyStr,
) )
s.Engine.ExportProcessRetryTask( s.Engine.ExportProcessRetryTask(
@@ -444,14 +460,16 @@ func TestProcessRetryTask_LargeBody_FetchFromDB(
database.DeliveryStatusRetrying, database.DeliveryStatusRetrying,
) )
cfg := iHTTPConfig(ts.URL)
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "retry-large", t, s.MainDB, targetID, s.WebhookID, "retry-large",
database.TargetTypeHTTP, ts.URL, 5, database.TargetTypeHTTP, cfg, 5,
) )
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"retry-large", ts.URL, 5, 2, nil, "retry-large", cfg, 5, 2, nil,
) )
s.Engine.ExportProcessRetryTask( s.Engine.ExportProcessRetryTask(
@@ -492,7 +510,7 @@ func TestWorkerLifecycle_StartStop(t *testing.T) {
) )
task.TargetType = database.TargetTypeLog task.TargetType = database.TargetTypeLog
s.Engine.ExportNotify([]delivery.Task{task}) s.Engine.Notify([]delivery.Task{task})
iWaitForDelivered(t, s.WebhookDB, d.ID) iWaitForDelivered(t, s.WebhookDB, d.ID)
@@ -552,15 +570,16 @@ func TestWorkerLifecycle_ProcessesRetryChannel(
s.Engine.ExportStart() s.Engine.ExportStart()
bodyStr := event.Body bodyStr := event.Body
cfg := iHTTPConfig(ts.URL)
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "retry-chan-test", t, s.MainDB, targetID, s.WebhookID, "retry-chan-test",
database.TargetTypeHTTP, ts.URL, 5, database.TargetTypeHTTP, cfg, 5,
) )
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"retry-chan-test", ts.URL, 5, 2, &bodyStr, "retry-chan-test", cfg, 5, 2, &bodyStr,
) )
s.Engine.ExportRetryCh() <- task s.Engine.ExportRetryCh() <- task
@@ -640,7 +659,7 @@ func TestRecoverPendingDeliveries(t *testing.T) {
s.Engine.ExportRecoverPendingDeliveries( s.Engine.ExportRecoverPendingDeliveries(
context.Background(), s.WebhookDB, context.Background(), s.WebhookDB,
s.WebhookID, atStart, s.WebhookID,
) )
for i := range 3 { for i := range 3 {
@@ -666,7 +685,7 @@ func iCreateTarget(
mainDB *gorm.DB, mainDB *gorm.DB,
targetID, webhookID, name string, targetID, webhookID, name string,
targetType database.TargetType, targetType database.TargetType,
targetURL string, config string,
maxAttempts int, maxAttempts int,
) { ) {
t.Helper() t.Helper()
@@ -676,7 +695,7 @@ func iCreateTarget(
Name: name, Name: name,
Type: targetType, Type: targetType,
Active: true, Active: true,
URL: targetURL, Config: config,
MaxAttempts: maxAttempts, MaxAttempts: maxAttempts,
} }
target.ID = targetID target.ID = targetID
@@ -717,7 +736,7 @@ func TestRecoverWebhookDeliveries_RetryingDeliveries(
iCreateTarget(t, s.MainDB, targetID, iCreateTarget(t, s.MainDB, targetID,
s.WebhookID, "retry-recovery", s.WebhookID, "retry-recovery",
database.TargetTypeHTTP, database.TargetTypeHTTP,
"http://example.com/hook", 5, iHTTPConfig("http://example.com/hook"), 5,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -737,7 +756,7 @@ func TestRecoverWebhookDeliveries_RetryingDeliveries(
) )
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, atStart, context.Background(), s.WebhookID,
) )
select { select {
@@ -777,7 +796,7 @@ func iSeedRetryingWithType(
iCreateTarget(t, s.MainDB, targetID, iCreateTarget(t, s.MainDB, targetID,
s.WebhookID, "mutated-target", targetType, s.WebhookID, "mutated-target", targetType,
"http://example.com/hook", 5, iHTTPConfig("http://example.com/hook"), 5,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -861,7 +880,7 @@ func TestRecoverSingleRetry_TypeNoLongerRetries(
) )
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, atStart, context.Background(), s.WebhookID,
) )
iAssertTerminallyFailed( iAssertTerminallyFailed(
@@ -869,7 +888,7 @@ func TestRecoverSingleRetry_TypeNoLongerRetries(
) )
} }
func TestRecoverSingleRetry_OnTick_TypeNoLongerRetries( func TestSweepSingleRetry_TypeNoLongerRetries(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
@@ -884,8 +903,8 @@ func TestRecoverSingleRetry_OnTick_TypeNoLongerRetries(
t, s, database.TargetTypeDatabase, t, s, database.TargetTypeDatabase,
) )
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportSweepWebhookRetries(
context.Background(), s.WebhookID, onTick, context.Background(), s.WebhookID,
) )
iAssertTerminallyFailed( iAssertTerminallyFailed(
@@ -915,14 +934,14 @@ func TestFailUnretryableRetry_WritesNoTargetRow(
targetID := uuid.New().String() targetID := uuid.New().String()
// A Slack incoming-webhook URL: the target's URL IS the // A Slack incoming-webhook URL: the target config IS the
// credential, which is what makes a leaked target row a // credential, which is what makes a leaked target row a
// disclosure rather than a curiosity. // disclosure rather than a curiosity.
hookURL := "https://hooks.slack.com/services/T00/B00/x" hookURL := "https://hooks.slack.com/services/T00/B00/x"
iCreateTarget(t, s.MainDB, targetID, iCreateTarget(t, s.MainDB, targetID,
s.WebhookID, "credential-bearing", s.WebhookID, "credential-bearing",
database.TargetTypeLog, hookURL, 5, database.TargetTypeLog, iHTTPConfig(hookURL), 5,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -936,8 +955,8 @@ func TestFailUnretryableRetry_WritesNoTargetRow(
iSeedFailedResult(t, s.WebhookDB, d.ID) iSeedFailedResult(t, s.WebhookDB, d.ID)
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportSweepWebhookRetries(
context.Background(), s.WebhookID, onTick, context.Background(), s.WebhookID,
) )
iAssertStatus(t, s.WebhookDB, d.ID, iAssertStatus(t, s.WebhookDB, d.ID,
@@ -968,13 +987,13 @@ func TestRecoverSingleRetry_UnknownTargetType(
deliveryID := iSeedRetryingWithType(t, s, unknown) deliveryID := iSeedRetryingWithType(t, s, unknown)
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, atStart, context.Background(), s.WebhookID,
) )
iAssertTerminallyFailed(t, s, deliveryID, unknown) iAssertTerminallyFailed(t, s, deliveryID, unknown)
} }
func TestRecoverSingleRetry_OnTick_UnknownTargetType( func TestSweepSingleRetry_UnknownTargetType(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
@@ -989,8 +1008,8 @@ func TestRecoverSingleRetry_OnTick_UnknownTargetType(
deliveryID := iSeedRetryingWithType(t, s, unknown) deliveryID := iSeedRetryingWithType(t, s, unknown)
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportSweepWebhookRetries(
context.Background(), s.WebhookID, onTick, context.Background(), s.WebhookID,
) )
iAssertTerminallyFailed(t, s, deliveryID, unknown) iAssertTerminallyFailed(t, s, deliveryID, unknown)
@@ -1032,19 +1051,19 @@ func iCreateWebhook(
require.NoError(t, mainDB.Create(&webhook).Error) require.NoError(t, mainDB.Create(&webhook).Error)
} }
// --- recoverDeliveries Tests --- // --- recoverInFlight Tests ---
func TestRecoverDeliveries_NoWebhooks(t *testing.T) { func TestRecoverInFlight_NoWebhooks(t *testing.T) {
t.Parallel() t.Parallel()
s := newISetup(t) s := newISetup(t)
s.Engine.ExportRecoverDeliveries( s.Engine.ExportRecoverInFlight(
context.Background(), atStart, context.Background(),
) )
} }
func TestRecoverDeliveries_WithPendingDeliveries( func TestRecoverInFlight_WithPendingDeliveries(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
@@ -1071,8 +1090,8 @@ func TestRecoverDeliveries_WithPendingDeliveries(
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
s.Engine.ExportRecoverDeliveries( s.Engine.ExportRecoverInFlight(
context.Background(), atStart, context.Background(),
) )
select { select {
@@ -1083,16 +1102,16 @@ func TestRecoverDeliveries_WithPendingDeliveries(
database.TargetTypeLog, task.TargetType, database.TargetTypeLog, task.TargetType,
) )
case <-time.After(2 * time.Second): case <-time.After(2 * time.Second):
t.Fatal("expected task from recovery at start") t.Fatal("expected task from recoverInFlight")
} }
} }
// TestRecoverDeliveries_ReportsAMissingWebhookDatabase covers a webhook // TestRecoverInFlight_ReportsAMissingWebhookDatabase covers a webhook
// whose database file is gone, after a partial restore say. Recovery at // whose database file is gone, after a partial restore say. Restart
// start opens every webhook's database, so the empty one made in its // recovery opens every webhook's database, so the empty one made in its
// place is reported at start, naming the file // place is reported at start, naming the file
// (https://git.eeqj.de/sneak/webhooker/issues/290). // (https://git.eeqj.de/sneak/webhooker/issues/290).
func TestRecoverDeliveries_ReportsAMissingWebhookDatabase(t *testing.T) { func TestRecoverInFlight_ReportsAMissingWebhookDatabase(t *testing.T) {
t.Parallel() t.Parallel()
mainDB := iMainDB(t) mainDB := iMainDB(t)
@@ -1112,7 +1131,7 @@ func TestRecoverDeliveries_ReportsAMissingWebhookDatabase(t *testing.T) {
&http.Client{Timeout: 5 * time.Second}, 1, &http.Client{Timeout: 5 * time.Second}, 1,
) )
engine.ExportRecoverDeliveries(context.Background(), atStart) engine.ExportRecoverInFlight(context.Background())
assert.Contains( assert.Contains(
t, logs.String(), t, logs.String(),
@@ -1121,11 +1140,10 @@ func TestRecoverDeliveries_ReportsAMissingWebhookDatabase(t *testing.T) {
) )
} }
// TestRecoverDeliveries_SkipsAWebhookDeletedAfterTheListIsRead covers a // TestRecoverInFlight_SkipsAWebhookDeletedAfterTheListIsRead covers a
// webhook deleted from the web interface while the recovery routine // webhook deleted from the web interface while restart recovery runs.
// runs. Its database file is gone, and recovery must not create it // Its database file is gone, and recovery must not create it again.
// again. func TestRecoverInFlight_SkipsAWebhookDeletedAfterTheListIsRead(
func TestRecoverDeliveries_SkipsAWebhookDeletedAfterTheListIsRead(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
@@ -1161,13 +1179,13 @@ func TestRecoverDeliveries_SkipsAWebhookDeletedAfterTheListIsRead(
&http.Client{Timeout: 5 * time.Second}, 1, &http.Client{Timeout: 5 * time.Second}, 1,
) )
engine.ExportRecoverDeliveries(context.Background(), atStart) engine.ExportRecoverInFlight(context.Background())
require.True(t, deleted) require.True(t, deleted)
assert.False(t, dbMgr.DBExists(webhookID)) assert.False(t, dbMgr.DBExists(webhookID))
} }
// --- HTTP target with custom headers --- // --- HTTP Config with custom headers ---
func TestDeliverHTTP_CustomTargetHeaders(t *testing.T) { func TestDeliverHTTP_CustomTargetHeaders(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1189,6 +1207,16 @@ func TestDeliverHTTP_CustomTargetHeaders(t *testing.T) {
) )
defer ts.Close() defer ts.Close()
cfg := delivery.HTTPTargetConfig{
URL: ts.URL,
Headers: map[string]string{
"Authorization": "Bearer secret-token",
},
}
cfgJSON, err := json.Marshal(cfg)
require.NoError(t, err)
event := iSeedEvent( event := iSeedEvent(
t, s.WebhookDB, s.WebhookID, t, s.WebhookDB, s.WebhookID,
`{"auth":"test"}`, `{"auth":"test"}`,
@@ -1203,12 +1231,9 @@ func TestDeliverHTTP_CustomTargetHeaders(t *testing.T) {
bodyStr := event.Body bodyStr := event.Body
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"auth-target", ts.URL, "auth-target", string(cfgJSON),
1, 1, &bodyStr, 1, 1, &bodyStr,
) )
task.TargetHeaders = map[string]string{
headerAuthorization: "Bearer secret-token",
}
s.Engine.ExportProcessNewTask( s.Engine.ExportProcessNewTask(
context.TODO(), &task, context.TODO(), &task,
@@ -1221,14 +1246,11 @@ func TestDeliverHTTP_CustomTargetHeaders(t *testing.T) {
// --- HTTP delivery with custom timeout --- // --- HTTP delivery with custom timeout ---
// TestDeliverHTTP_TargetTimeout proves a task's TimeoutSeconds reaches
// the delivery through the engine. The destination answers after two
// seconds, inside the engine's own five-second client timeout, so only
// the task's one second can fail the delivery.
func TestDeliverHTTP_TargetTimeout(t *testing.T) { func TestDeliverHTTP_TargetTimeout(t *testing.T) {
t.Parallel() t.Parallel()
s := newISetup(t) db := testWebhookDB(t)
e := iEngine(t, 1)
ts := httptest.NewServer( ts := httptest.NewServer(
http.HandlerFunc( http.HandlerFunc(
@@ -1240,30 +1262,31 @@ func TestDeliverHTTP_TargetTimeout(t *testing.T) {
) )
defer ts.Close() defer ts.Close()
event := iSeedEvent( cfg := delivery.HTTPTargetConfig{
t, s.WebhookDB, s.WebhookID, `{"timeout":"test"}`, URL: ts.URL,
) Timeout: 1,
targetID := uuid.New().String() }
d := iSeedDelivery( cfgJSON, err := json.Marshal(cfg)
t, s.WebhookDB, event.ID, targetID, require.NoError(t, err)
database.DeliveryStatusPending,
event, del := iSeedEventAndDelivery(
t, db, `{"timeout":"test"}`,
string(cfgJSON),
) )
bodyStr := event.Body task, d := iHTTPTaskAndDelivery(
task := iTask( event, del, "timeout-target",
d, event, s.WebhookID, targetID, string(cfgJSON), 1, 1,
"timeout-target", ts.URL, 1, 1, &bodyStr,
) )
task.TimeoutSeconds = 1
s.Engine.ExportProcessNewTask(context.TODO(), &task) e.ExportDeliverHTTP(context.TODO(), db, d, task)
iAssertStatus(t, s.WebhookDB, d.ID, iAssertStatus(t, db, del.ID,
database.DeliveryStatusFailed, database.DeliveryStatusFailed,
) )
iAssertResultFailed(t, s.WebhookDB, d.ID) iAssertResultFailed(t, db, del.ID)
} }
// TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog pins the size // TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog pins the size
@@ -1301,6 +1324,8 @@ func TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog(
)) ))
defer ts.Close() defer ts.Close()
cfgJSON := iHTTPConfig(ts.URL)
event := iSeedEvent( event := iSeedEvent(
t, s.WebhookDB, s.WebhookID, `{"cut":"test"}`, t, s.WebhookDB, s.WebhookID, `{"cut":"test"}`,
) )
@@ -1314,7 +1339,7 @@ func TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog(
bodyStr := event.Body bodyStr := event.Body
task := iTask( task := iTask(
del, event, s.WebhookID, targetID, del, event, s.WebhookID, targetID,
"cut-target", ts.URL, 1, 1, &bodyStr, "cut-target", cfgJSON, 1, 1, &bodyStr,
) )
s.Engine.ExportProcessNewTask(context.TODO(), &task) s.Engine.ExportProcessNewTask(context.TODO(), &task)
@@ -1346,7 +1371,7 @@ func TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog(
func iSeedEventAndDelivery( func iSeedEventAndDelivery(
t *testing.T, t *testing.T,
db *gorm.DB, db *gorm.DB,
body string, body, _ string,
) (database.Event, database.Delivery) { ) (database.Event, database.Delivery) {
t.Helper() t.Helper()
@@ -1377,19 +1402,19 @@ func iSeedEventAndDelivery(
func iHTTPTaskAndDelivery( func iHTTPTaskAndDelivery(
event database.Event, event database.Event,
del database.Delivery, del database.Delivery,
name, targetURL string, name, config string,
maxAttempts, attemptNum int, maxAttempts, attemptNum int,
) (*delivery.Task, *database.Delivery) { ) (*delivery.Task, *database.Delivery) {
task := &delivery.Task{ task := &delivery.Task{
DeliveryID: del.ID, DeliveryID: del.ID,
EventID: event.ID, EventID: event.ID,
WebhookID: event.WebhookID, WebhookID: event.WebhookID,
TargetID: del.TargetID, TargetID: del.TargetID,
TargetName: name, TargetName: name,
TargetType: database.TargetTypeHTTP, TargetType: database.TargetTypeHTTP,
TargetURL: targetURL, TargetConfig: config,
MaxAttempts: maxAttempts, MaxAttempts: maxAttempts,
AttemptNum: attemptNum, AttemptNum: attemptNum,
} }
d := &database.Delivery{ d := &database.Delivery{
@@ -1400,7 +1425,7 @@ func iHTTPTaskAndDelivery(
Target: database.Target{ Target: database.Target{
Name: name, Name: name,
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
URL: targetURL, Config: config,
MaxAttempts: maxAttempts, MaxAttempts: maxAttempts,
}, },
} }
@@ -1429,30 +1454,56 @@ func iAssertResultFailed(
assert.NotEmpty(t, result.Error) assert.NotEmpty(t, result.Error)
} }
// --- HTTP request with no URL --- // --- HTTP request with invalid config ---
// TestDeliverHTTP_NoURL proves an http target without a URL, which func TestDeliverHTTP_InvalidConfig(t *testing.T) {
// the forms never store, fails its delivery and records why.
func TestDeliverHTTP_NoURL(t *testing.T) {
t.Parallel() t.Parallel()
db := testWebhookDB(t) db := testWebhookDB(t)
e := iEngine(t, 1) e := iEngine(t, 1)
event, del := iSeedEventAndDelivery(t, db, `{"url":"none"}`) event, del := iSeedEventAndDelivery(
t, db, `{"config":"invalid"}`, "",
)
task, d := iHTTPTaskAndDelivery(event, del, "no-url", "", 1, 1) task, d := iHTTPTaskAndDelivery(
event, del, "bad-config", `not-json`, 1, 1,
)
e.ExportDeliverHTTP(context.TODO(), db, d, task) e.ExportDeliverHTTP(context.TODO(), db, d, task)
iAssertStatus(t, db, del.ID, iAssertStatus(t, db, del.ID,
database.DeliveryStatusFailed, database.DeliveryStatusFailed,
) )
iAssertResultFailed(t, db, del.ID)
} }
// --- notify batching --- // TestDeliverHTTP_InvalidConfigUnrecordedStaysPending: a delivery is
// failed for an invalid config only once the reason is recorded.
// Unrecorded, it stays pending, where the sweep finds it again.
func TestDeliverHTTP_InvalidConfigUnrecordedStaysPending(t *testing.T) {
t.Parallel()
db := testWebhookDB(t)
e := testEngine(t, 1)
event, del := iSeedEventAndDelivery(
t, db, `{"config":"invalid"}`, "",
)
task, d := iHTTPTaskAndDelivery(
event, del, "bad-config", `not-json`, 1, 1,
)
require.NoError(t, db.Exec("drop table delivery_results").Error)
e.ExportDeliverHTTP(context.TODO(), db, d, task)
iAssertStatus(t, db, del.ID,
database.DeliveryStatusPending,
)
}
// --- Notify batching ---
func TestNotify_MultipleTasks(t *testing.T) { func TestNotify_MultipleTasks(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1467,7 +1518,7 @@ func TestNotify_MultipleTasks(t *testing.T) {
} }
} }
e.ExportNotify(tasks) e.Notify(tasks)
for i := range 5 { for i := range 5 {
select { select {
internal/delivery/engine_lifecycle_test.go
+19 -19
View File
@@ -29,7 +29,7 @@ const (
// OnStart before the caller may enqueue work. A worker pool // OnStart before the caller may enqueue work. A worker pool
// wrongly rooted in the already-done hook context has nothing // wrongly rooted in the already-done hook context has nothing
// but ctx.Done() ready in its select, so it is deterministically // but ctx.Done() ready in its select, so it is deterministically
// gone by the end of this window. Without the wait, notify would // gone by the end of this window. Without the wait, Notify would
// race the pool's very first select, in which a ready ctx.Done() // race the pool's very first select, in which a ready ctx.Done()
// and a ready deliveryCh are chosen between at random and a // and a ready deliveryCh are chosen between at random and a
// doomed pool still delivers. // doomed pool still delivers.
@@ -99,7 +99,7 @@ func requireStopHookExpires(
// the caller can drive OnStop too. // the caller can drive OnStop too.
// //
// Callers must not seed pending or retrying deliveries before // Callers must not seed pending or retrying deliveries before
// calling this: the recovery routine's run at start enqueues those, // calling this: restart recovery enqueues those during startup,
// which would put work in the queue while the pool is still // which would put work in the queue while the pool is still
// racing its first select. // racing its first select.
func startEngineViaHook( func startEngineViaHook(
@@ -178,11 +178,11 @@ func TestEngine_WorkersOutliveStartHookContext(t *testing.T) {
_ = lc.hooks[0].OnStop(context.Background()) _ = lc.hooks[0].OnStop(context.Background())
}) })
// Seeded only after the pool has settled, so the recovery // Seeded only after the pool has settled, so restart recovery
// routine's run at start cannot enqueue it. // cannot enqueue it during startup.
deliveryID, task := seedLogTask(t, s) deliveryID, task := seedLogTask(t, s)
s.Engine.ExportNotify([]delivery.Task{task}) s.Engine.Notify([]delivery.Task{task})
iWaitForDelivered(t, s.WebhookDB, deliveryID) iWaitForDelivered(t, s.WebhookDB, deliveryID)
} }
@@ -202,7 +202,7 @@ func TestEngine_StopHookStopsWorkers(t *testing.T) {
// Let the pool prove it is running before stopping it, so a // Let the pool prove it is running before stopping it, so a
// fast OnStop cannot pass by stopping something already dead. // fast OnStop cannot pass by stopping something already dead.
firstID, firstTask := seedLogTask(t, s) firstID, firstTask := seedLogTask(t, s)
s.Engine.ExportNotify([]delivery.Task{firstTask}) s.Engine.Notify([]delivery.Task{firstTask})
iWaitForDelivered(t, s.WebhookDB, firstID) iWaitForDelivered(t, s.WebhookDB, firstID)
var stopErr error var stopErr error
@@ -232,7 +232,7 @@ func TestEngine_StopHookStopsWorkers(t *testing.T) {
// With every worker gone, a freshly notified task must sit // With every worker gone, a freshly notified task must sit
// untouched in the queue rather than being delivered. // untouched in the queue rather than being delivered.
secondID, secondTask := seedLogTask(t, s) secondID, secondTask := seedLogTask(t, s)
s.Engine.ExportNotify([]delivery.Task{secondTask}) s.Engine.Notify([]delivery.Task{secondTask})
time.Sleep(200 * time.Millisecond) time.Sleep(200 * time.Millisecond)
@@ -309,12 +309,13 @@ func seedHeldDelivery(
)) ))
t.Cleanup(ts.Close) t.Cleanup(ts.Close)
cfg := iHTTPConfig(ts.URL)
targetID := uuid.New().String() targetID := uuid.New().String()
iCreateWebhook(t, s.MainDB, s.WebhookID, "hook") iCreateWebhook(t, s.MainDB, s.WebhookID, "hook")
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "held", t, s.MainDB, targetID, s.WebhookID, "held",
database.TargetTypeHTTP, ts.URL, maxAttempts, database.TargetTypeHTTP, cfg, maxAttempts,
) )
event := iSeedEvent(t, s.WebhookDB, s.WebhookID, `{"held":true}`) event := iSeedEvent(t, s.WebhookDB, s.WebhookID, `{"held":true}`)
@@ -323,7 +324,7 @@ func seedHeldDelivery(
bodyStr := event.Body bodyStr := event.Body
return iTask( return iTask(
d, event, s.WebhookID, targetID, "held", ts.URL, d, event, s.WebhookID, targetID, "held", cfg,
maxAttempts, attemptNum, &bodyStr, maxAttempts, attemptNum, &bodyStr,
), arrived ), arrived
} }
@@ -344,15 +345,15 @@ func stopDuringAttempt(
require.NoError(t, lc.hooks[0].OnStop(context.Background())) require.NoError(t, lc.hooks[0].OnStop(context.Background()))
} }
// requireRequeued runs the recovery routine as at start on the stopped // requireRequeued runs restart recovery on the stopped engine and
// engine and requires it to queue task's attempt again on ch. // requires it to queue task's attempt again on ch.
func requireRequeued( func requireRequeued(
t *testing.T, s iSetup, ch <-chan delivery.Task, task delivery.Task, t *testing.T, s iSetup, ch <-chan delivery.Task, task delivery.Task,
) { ) {
t.Helper() t.Helper()
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, atStart, context.Background(), s.WebhookID,
) )
select { select {
@@ -360,7 +361,7 @@ func requireRequeued(
assert.Equal(t, task.DeliveryID, got.DeliveryID) assert.Equal(t, task.DeliveryID, got.DeliveryID)
assert.Equal(t, task.AttemptNum, got.AttemptNum) assert.Equal(t, task.AttemptNum, got.AttemptNum)
case <-time.After(5 * time.Second): case <-time.After(5 * time.Second):
t.Fatal("recovery at start did not queue the delivery again") t.Fatal("restart recovery did not queue the delivery again")
} }
} }
@@ -370,8 +371,7 @@ func requireRequeued(
// no retries that was the delivery's only attempt, recorded as failed // no retries that was the delivery's only attempt, recorded as failed
// with "context canceled", and nothing ever sent it again. The // with "context canceled", and nothing ever sent it again. The
// delivery must instead stay pending, with no result and no count in // delivery must instead stay pending, with no result and no count in
// the attempt metrics, for the recovery routine to send at the next // the attempt metrics, for restart recovery to send.
// start.
func TestEngine_StopLeavesFireAndForgetAttemptPending(t *testing.T) { func TestEngine_StopLeavesFireAndForgetAttemptPending(t *testing.T) {
t.Parallel() t.Parallel()
@@ -383,7 +383,7 @@ func TestEngine_StopLeavesFireAndForgetAttemptPending(t *testing.T) {
t, s, 1, database.DeliveryStatusPending, 1, t, s, 1, database.DeliveryStatusPending, 1,
) )
s.Engine.ExportNotify([]delivery.Task{task}) s.Engine.Notify([]delivery.Task{task})
stopDuringAttempt(t, lc, arrived) stopDuringAttempt(t, lc, arrived)
iAssertStatus( iAssertStatus(
@@ -399,8 +399,8 @@ func TestEngine_StopLeavesFireAndForgetAttemptPending(t *testing.T) {
// TestEngine_StopLeavesLastRetryAttemptRetrying is the same regression // TestEngine_StopLeavesLastRetryAttemptRetrying is the same regression
// for a target with retries, on its last attempt: the cancelled attempt // for a target with retries, on its last attempt: the cancelled attempt
// used to exhaust the retries and end the delivery failed. It must stay // used to exhaust the retries and end the delivery failed. It must stay
// retrying with only its earlier attempt recorded, and the recovery // retrying with only its earlier attempt recorded, and restart recovery
// routine at the next start must send that last attempt again. // must send that last attempt again.
func TestEngine_StopLeavesLastRetryAttemptRetrying(t *testing.T) { func TestEngine_StopLeavesLastRetryAttemptRetrying(t *testing.T) {
t.Parallel() t.Parallel()
@@ -448,7 +448,7 @@ func deliverToArchive(t *testing.T, s iSetup) string {
task.TargetID = tgt.ID task.TargetID = tgt.ID
task.TargetType = database.TargetTypeDatabase task.TargetType = database.TargetTypeDatabase
s.Engine.ExportNotify([]delivery.Task{task}) s.Engine.Notify([]delivery.Task{task})
iWaitForDelivered(t, s.WebhookDB, deliveryID) iWaitForDelivered(t, s.WebhookDB, deliveryID)
internal/delivery/engine_test.go
+211 -59
View File
@@ -65,6 +65,20 @@ func testEngine(
) )
} }
func newHTTPTargetConfig(url string) string {
cfg := delivery.HTTPTargetConfig{URL: url}
data, err := json.Marshal(cfg)
if err != nil {
panic(
"failed to marshal HTTPTargetConfig: " +
err.Error(),
)
}
return string(data)
}
func seedEvent( func seedEvent(
t *testing.T, db *gorm.DB, body string, t *testing.T, db *gorm.DB, body string,
) database.Event { ) database.Event {
@@ -127,19 +141,19 @@ type httpDeliveryFixture struct {
func buildHTTPFixture( func buildHTTPFixture(
dlv database.Delivery, dlv database.Delivery,
event database.Event, event database.Event,
targetID, name, targetURL string, targetID, name, config string,
maxAttempts, attemptNum int, maxAttempts, attemptNum int,
) httpDeliveryFixture { ) httpDeliveryFixture {
task := &delivery.Task{ task := &delivery.Task{
DeliveryID: dlv.ID, DeliveryID: dlv.ID,
EventID: event.ID, EventID: event.ID,
WebhookID: event.WebhookID, WebhookID: event.WebhookID,
TargetID: targetID, TargetID: targetID,
TargetName: name, TargetName: name,
TargetType: database.TargetTypeHTTP, TargetType: database.TargetTypeHTTP,
TargetURL: targetURL, TargetConfig: config,
MaxAttempts: maxAttempts, MaxAttempts: maxAttempts,
AttemptNum: attemptNum, AttemptNum: attemptNum,
} }
d := &database.Delivery{ d := &database.Delivery{
@@ -150,7 +164,7 @@ func buildHTTPFixture(
Target: database.Target{ Target: database.Target{
Name: name, Name: name,
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
URL: targetURL, Config: config,
MaxAttempts: maxAttempts, MaxAttempts: maxAttempts,
}, },
} }
@@ -218,7 +232,7 @@ func TestNotify_NonBlocking(t *testing.T) {
done := make(chan struct{}) done := make(chan struct{})
go func() { go func() {
e.ExportNotify([]delivery.Task{ e.Notify([]delivery.Task{
{DeliveryID: "overflow-1"}, {DeliveryID: "overflow-1"},
{DeliveryID: "overflow-2"}, {DeliveryID: "overflow-2"},
}) })
@@ -229,7 +243,7 @@ func TestNotify_NonBlocking(t *testing.T) {
case <-done: case <-done:
case <-time.After(2 * time.Second): case <-time.After(2 * time.Second):
t.Fatal( t.Fatal(
"notify blocked when delivery channel was full", "Notify blocked when delivery channel was full",
) )
} }
} }
@@ -261,8 +275,9 @@ func TestDeliverHTTP_Success(t *testing.T) {
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
cfg := newHTTPTargetConfig(ts.URL)
fix := buildHTTPFixture( fix := buildHTTPFixture(
dlv, event, targetID, "test-http", ts.URL, 1, 1, dlv, event, targetID, "test-http", cfg, 1, 1,
) )
e.ExportDeliverHTTP( e.ExportDeliverHTTP(
@@ -306,9 +321,10 @@ func TestDeliverHTTP_Failure(t *testing.T) {
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
cfg := newHTTPTargetConfig(ts.URL)
fix := buildHTTPFixture( fix := buildHTTPFixture(
dlv, event, targetID, dlv, event, targetID,
"test-http-fail", ts.URL, 1, 1, "test-http-fail", cfg, 1, 1,
) )
e.ExportDeliverHTTP( e.ExportDeliverHTTP(
@@ -335,7 +351,7 @@ func TestDeliverDatabase_ImmediateSuccess(
// The database target archives for real, so the engine needs // The database target archives for real, so the engine needs
// the target in the main database and a data directory. // the target in the main database and a data directory.
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t) tgt := env.seedDatabaseTarget(t, "")
event := seedEvent(t, db, `{"db":"target"}`) event := seedEvent(t, db, `{"db":"target"}`)
d := seedDatabaseTargetDelivery(t, db, event, tgt) d := seedDatabaseTargetDelivery(t, db, event, tgt)
@@ -436,9 +452,10 @@ func TestDeliverHTTP_WithRetries_Success(t *testing.T) {
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
cfg := newHTTPTargetConfig(ts.URL)
fix := buildHTTPFixture( fix := buildHTTPFixture(
dlv, event, targetID, dlv, event, targetID,
"test-http-retry", ts.URL, 5, 1, "test-http-retry", cfg, 5, 1,
) )
e.ExportDeliverHTTP( e.ExportDeliverHTTP(
@@ -493,12 +510,13 @@ func TestDeliverHTTP_MaxAttemptsExhausted(t *testing.T) {
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
cfg := newHTTPTargetConfig(ts.URL)
sched := &recordingScheduler{} sched := &recordingScheduler{}
for attempt := 1; attempt <= maxAttempts; attempt++ { for attempt := 1; attempt <= maxAttempts; attempt++ {
fix := buildHTTPFixture( fix := buildHTTPFixture(
dlv, event, targetID, dlv, event, targetID,
"test-http-exhaust", ts.URL, maxAttempts, attempt, "test-http-exhaust", cfg, maxAttempts, attempt,
) )
e.ExportDeliverHTTPWithScheduler( e.ExportDeliverHTTPWithScheduler(
@@ -556,9 +574,10 @@ func TestDeliverHTTP_SingleAttemptSkipsCircuitBreaker(t *testing.T) {
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
cfg := newHTTPTargetConfig(ts.URL)
fix := buildHTTPFixture( fix := buildHTTPFixture(
dlv, event, targetID, dlv, event, targetID,
"test-http-single", ts.URL, 1, 1, "test-http-single", cfg, 1, 1,
) )
sched := &recordingScheduler{} sched := &recordingScheduler{}
@@ -605,9 +624,10 @@ func TestDeliverHTTP_SchedulesRetryOnFailure(
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
cfg := newHTTPTargetConfig(ts.URL)
fix := buildHTTPFixture( fix := buildHTTPFixture(
dlv, event, targetID, dlv, event, targetID,
"test-http-schedule", ts.URL, 5, 1, "test-http-schedule", cfg, 5, 1,
) )
e.ExportDeliverHTTP( e.ExportDeliverHTTP(
@@ -769,7 +789,7 @@ func seedConcurrencyTasks(
t *testing.T, t *testing.T,
db *gorm.DB, db *gorm.DB,
n int, n int,
targetURL string, targetCfg string,
) ([]database.Delivery, []delivery.Task) { ) ([]database.Delivery, []delivery.Task) {
t.Helper() t.Helper()
@@ -795,21 +815,21 @@ func seedConcurrencyTasks(
Target: database.Target{ Target: database.Target{
Name: fmt.Sprintf("task-%d", i), Name: fmt.Sprintf("task-%d", i),
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
URL: targetURL, Config: targetCfg,
MaxAttempts: 1, MaxAttempts: 1,
}, },
} }
tasks[i].ID = dlv.ID tasks[i].ID = dlv.ID
dtasks[i] = delivery.Task{ dtasks[i] = delivery.Task{
DeliveryID: tasks[i].ID, DeliveryID: tasks[i].ID,
EventID: tasks[i].EventID, EventID: tasks[i].EventID,
TargetID: tasks[i].TargetID, TargetID: tasks[i].TargetID,
TargetName: tasks[i].Target.Name, TargetName: tasks[i].Target.Name,
TargetType: tasks[i].Target.Type, TargetType: tasks[i].Target.Type,
TargetURL: tasks[i].Target.URL, TargetConfig: tasks[i].Target.Config,
MaxAttempts: 1, MaxAttempts: 1,
AttemptNum: 1, AttemptNum: 1,
} }
} }
@@ -845,8 +865,9 @@ func TestWorkerPool_BoundedConcurrency(t *testing.T) {
defer ts.Close() defer ts.Close()
e := testEngine(t, numWorkers) e := testEngine(t, numWorkers)
cfg := newHTTPTargetConfig(ts.URL)
tasks, dtasks := seedConcurrencyTasks( tasks, dtasks := seedConcurrencyTasks(
t, db, numTasks, ts.URL, t, db, numTasks, cfg,
) )
runConcurrencyWorkers( runConcurrencyWorkers(
@@ -927,9 +948,12 @@ func TestDeliverHTTP_CircuitBreakerBlocks(t *testing.T) {
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
cfg := newHTTPTargetConfig(
"http://will-not-be-called.invalid",
)
fix := buildHTTPFixture( fix := buildHTTPFixture(
dlv, event, targetID, dlv, event, targetID,
"test-cb-block", "http://will-not-be-called.invalid", 5, 1, "test-cb-block", cfg, 5, 1,
) )
e.ExportDeliverHTTP( e.ExportDeliverHTTP(
@@ -992,8 +1016,9 @@ func TestDeliverHTTP_HalfOpenBreakerDelaysQueuedTasks(t *testing.T) {
require.True(t, cb.Allow(), "the probe delivery should go through") require.True(t, cb.Allow(), "the probe delivery should go through")
require.Equal(t, delivery.CircuitHalfOpen, cb.State()) require.Equal(t, delivery.CircuitHalfOpen, cb.State())
const targetURL = "http://will-not-be-called.invalid" cfg := newHTTPTargetConfig(
"http://will-not-be-called.invalid",
)
sched := &recordingScheduler{} sched := &recordingScheduler{}
const queued, passes = 3, 4 const queued, passes = 3, 4
@@ -1015,7 +1040,7 @@ func TestDeliverHTTP_HalfOpenBreakerDelaysQueuedTasks(t *testing.T) {
fix := buildHTTPFixture( fix := buildHTTPFixture(
row, event, targetID, row, event, targetID,
"test-cb-half-open", targetURL, 5, 1, "test-cb-half-open", cfg, 5, 1,
) )
e.ExportDeliverHTTPWithScheduler( e.ExportDeliverHTTPWithScheduler(
@@ -1130,6 +1155,62 @@ func TestStateAndCooldown_ReadsHTTPAndSlackBreakers(t *testing.T) {
assert.Zero(t, cooldown, "half-open") assert.Zero(t, cooldown, "half-open")
} }
func TestParseHTTPConfig_Valid(t *testing.T) {
t.Parallel()
e := testEngine(t, 1)
cfg, err := e.ExportParseHTTPConfig(
`{"url":"https://example.com/hook",` +
`"headers":{"X-Token":"secret"}}`,
)
require.NoError(t, err)
assert.Equal(t, "https://example.com/hook", cfg.URL)
assert.Equal(t, "secret", cfg.Headers["X-Token"])
}
func TestParseHTTPConfig_Empty(t *testing.T) {
t.Parallel()
e := testEngine(t, 1)
_, err := e.ExportParseHTTPConfig("")
assert.Error(t, err,
"empty config should return error",
)
}
func TestParseHTTPConfig_MissingURL(t *testing.T) {
t.Parallel()
e := testEngine(t, 1)
_, err := e.ExportParseHTTPConfig(
`{"headers":{"X-Token":"secret"}}`,
)
assert.Error(t, err,
"config without URL should return error",
)
}
func TestParseHTTPConfig_Undecodable(t *testing.T) {
t.Parallel()
e := testEngine(t, 1)
_, err := e.ExportParseHTTPConfig(
`{"url":"https://example.com/hook","timeout":"soon"}`,
)
assert.Error(t, err,
"config that does not decode should return error, "+
"even when the part that did names a URL",
)
}
func TestScheduleRetry_SendsToRetryChannel( func TestScheduleRetry_SendsToRetryChannel(
t *testing.T, t *testing.T,
) { ) {
@@ -1277,7 +1358,7 @@ func TestDoHTTPRequest_ForwardsHeaders(t *testing.T) {
e := testEngine(t, 1) e := testEngine(t, 1)
target := &database.Target{ cfg := &delivery.HTTPTargetConfig{
URL: ts.URL, URL: ts.URL,
Headers: map[string]string{ Headers: map[string]string{
"X-Target-Auth": "bearer xyz", "X-Target-Auth": "bearer xyz",
@@ -1292,7 +1373,7 @@ func TestDoHTTPRequest_ForwardsHeaders(t *testing.T) {
} }
statusCode, _, _, err := e.ExportDoHTTPRequest( statusCode, _, _, err := e.ExportDoHTTPRequest(
context.TODO(), target, event, context.TODO(), cfg, event,
) )
require.NoError(t, err) require.NoError(t, err)
@@ -1334,7 +1415,7 @@ func TestDoHTTPRequest_CutShortResponseIsAnError(t *testing.T) {
_, body, _, err := e.ExportDoHTTPRequest( _, body, _, err := e.ExportDoHTTPRequest(
context.TODO(), context.TODO(),
&database.Target{URL: ts.URL}, &delivery.HTTPTargetConfig{URL: ts.URL},
&database.Event{}, &database.Event{},
) )
@@ -1386,9 +1467,9 @@ func TestApplyRequestHeaders_SendsOneContentType(t *testing.T) {
}) })
require.NoError(t, err) require.NoError(t, err)
var configured map[string]string cfg := &delivery.HTTPTargetConfig{}
if tc.configured != "" { if tc.configured != "" {
configured = map[string]string{ cfg.Headers = map[string]string{
headerContentType: tc.configured, headerContentType: tc.configured,
} }
} }
@@ -1407,7 +1488,7 @@ func TestApplyRequestHeaders_SendsOneContentType(t *testing.T) {
Headers: string(inbound), Headers: string(inbound),
ContentType: tc.event, ContentType: tc.event,
}, },
configured, cfg,
"webhooker/dev", "webhooker/dev",
) )
@@ -1438,7 +1519,7 @@ func TestApplyRequestHeaders_UndecodableInboundForwardsNothing(
&database.Event{ &database.Event{
Headers: `{"X-Custom":["value1"],"X-Broken":"not a list"}`, Headers: `{"X-Custom":["value1"],"X-Broken":"not a list"}`,
}, },
nil, &delivery.HTTPTargetConfig{},
"webhooker/dev", "webhooker/dev",
) )
@@ -1456,7 +1537,7 @@ func TestProcessDelivery_RoutesToCorrectHandler(
// The database target archives for real, so the engine needs // The database target archives for real, so the engine needs
// the target in the main database and a data directory. // the target in the main database and a data directory.
env := setupArchiveTest(t) env := setupArchiveTest(t)
archive := env.seedDatabaseTarget(t) archive := env.seedDatabaseTarget(t, "")
tests := []struct { tests := []struct {
name string name string
@@ -1541,6 +1622,51 @@ func TestMaxInlineBodySize_Constant(t *testing.T) {
) )
} }
func TestParseSlackConfig_Valid(t *testing.T) {
t.Parallel()
e := testEngine(t, 1)
cfg, err := e.ExportParseSlackConfig(
`{"webhookUrl":"https://hooks.slack.com/services/T00/B00/xxx"}`,
)
require.NoError(t, err)
assert.Equal(t,
"https://hooks.slack.com/services/T00/B00/xxx",
cfg.WebhookURL,
)
}
func TestParseSlackConfig_Empty(t *testing.T) {
t.Parallel()
e := testEngine(t, 1)
_, err := e.ExportParseSlackConfig("")
assert.Error(t, err,
"empty config should return error",
)
}
func TestParseSlackConfig_MissingWebhookURL(
t *testing.T,
) {
t.Parallel()
e := testEngine(t, 1)
_, err := e.ExportParseSlackConfig(
`{"other":"field"}`,
)
assert.Error(t, err,
"config without webhook_url should return error",
)
}
func TestFormatSlackMessage_JSONBody(t *testing.T) { func TestFormatSlackMessage_JSONBody(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1637,11 +1763,11 @@ func TestFormatSlackMessage_LargeJSONTruncated(
} }
// buildSlackDelivery creates a database.Delivery // buildSlackDelivery creates a database.Delivery
// to a Slack target with the given webhook URL for testing. // with a Slack target config for testing.
func buildSlackDelivery( func buildSlackDelivery(
dlv database.Delivery, dlv database.Delivery,
event database.Event, event database.Event,
targetID, name, webhookURL string, targetID, name, slackCfg string,
) *database.Delivery { ) *database.Delivery {
d := &database.Delivery{ d := &database.Delivery{
EventID: event.ID, EventID: event.ID,
@@ -1651,7 +1777,7 @@ func buildSlackDelivery(
Target: database.Target{ Target: database.Target{
Name: name, Name: name,
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
URL: webhookURL, Config: slackCfg,
MaxAttempts: 1, MaxAttempts: 1,
}, },
} }
@@ -1689,10 +1815,11 @@ func TestDeliverSlack_Success(t *testing.T) {
) )
defer ts.Close() defer ts.Close()
e, targetID, event, dlv := setupSlackTest(t, db) e, targetID, slackCfg, event, dlv :=
setupSlackTest(t, db, ts.URL)
d := buildSlackDelivery( d := buildSlackDelivery(
dlv, event, targetID, "test-slack", ts.URL, dlv, event, targetID, "test-slack", slackCfg,
) )
e.ExportDeliverSlack(context.TODO(), db, d) e.ExportDeliverSlack(context.TODO(), db, d)
@@ -1711,8 +1838,9 @@ func TestDeliverSlack_Success(t *testing.T) {
func setupSlackTest( func setupSlackTest(
t *testing.T, t *testing.T,
db *gorm.DB, db *gorm.DB,
serverURL string,
) ( ) (
*delivery.Engine, string, *delivery.Engine, string, string,
database.Event, database.Delivery, database.Event, database.Delivery,
) { ) {
t.Helper() t.Helper()
@@ -1720,6 +1848,13 @@ func setupSlackTest(
e := testEngine(t, 1) e := testEngine(t, 1)
targetID := uuid.New().String() targetID := uuid.New().String()
cfgBytes, err := json.Marshal(
delivery.SlackTargetConfig{
WebhookURL: serverURL,
},
)
require.NoError(t, err)
event := seedEvent( event := seedEvent(
t, db, `{"action":"test","data":"value"}`, t, db, `{"action":"test","data":"value"}`,
) )
@@ -1729,7 +1864,7 @@ func setupSlackTest(
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
return e, targetID, event, dlv return e, targetID, string(cfgBytes), event, dlv
} }
func assertSlackPayload( func assertSlackPayload(
@@ -1778,6 +1913,13 @@ func TestDeliverSlack_Failure(t *testing.T) {
e := testEngine(t, 1) e := testEngine(t, 1)
targetID := uuid.New().String() targetID := uuid.New().String()
slackCfg, err := json.Marshal(
delivery.SlackTargetConfig{
WebhookURL: ts.URL,
},
)
require.NoError(t, err)
event := seedEvent(t, db, `{"test":true}`) event := seedEvent(t, db, `{"test":true}`)
dlv := seedDelivery( dlv := seedDelivery(
@@ -1787,7 +1929,7 @@ func TestDeliverSlack_Failure(t *testing.T) {
d := buildSlackDelivery( d := buildSlackDelivery(
dlv, event, targetID, dlv, event, targetID,
"test-slack-fail", ts.URL, "test-slack-fail", string(slackCfg),
) )
e.ExportDeliverSlack(context.TODO(), db, d) e.ExportDeliverSlack(context.TODO(), db, d)
@@ -1801,10 +1943,7 @@ func TestDeliverSlack_Failure(t *testing.T) {
) )
} }
// TestDeliverSlack_NoURL proves a Slack target without a URL, which func TestDeliverSlack_InvalidConfig(t *testing.T) {
// the forms never store, fails its delivery rather than reporting it
// delivered.
func TestDeliverSlack_NoURL(t *testing.T) {
t.Parallel() t.Parallel()
db := testWebhookDB(t) db := testWebhookDB(t)
@@ -1825,6 +1964,7 @@ func TestDeliverSlack_NoURL(t *testing.T) {
Target: database.Target{ Target: database.Target{
Name: "test-slack-bad", Name: "test-slack-bad",
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
Config: `{"not_webhook_url":"missing"}`,
MaxAttempts: 1, MaxAttempts: 1,
}, },
} }
@@ -1862,6 +2002,13 @@ func TestProcessDelivery_RoutesToSlack(t *testing.T) {
e := testEngine(t, 1) e := testEngine(t, 1)
slackCfg, err := json.Marshal(
delivery.SlackTargetConfig{
WebhookURL: ts.URL,
},
)
require.NoError(t, err)
event := seedEvent(t, db, `{"route":"slack"}`) event := seedEvent(t, db, `{"route":"slack"}`)
dlv := seedDelivery( dlv := seedDelivery(
@@ -1871,7 +2018,7 @@ func TestProcessDelivery_RoutesToSlack(t *testing.T) {
d := buildSlackDelivery( d := buildSlackDelivery(
dlv, event, dlv.TargetID, dlv, event, dlv.TargetID,
"test-slack-route", ts.URL, "test-slack-route", string(slackCfg),
) )
task := &delivery.Task{ task := &delivery.Task{
@@ -1986,7 +2133,7 @@ func TestDeliverLog_LogsFullContent(t *testing.T) {
func buildSlackRetryDelivery( func buildSlackRetryDelivery(
dlv database.Delivery, dlv database.Delivery,
event database.Event, event database.Event,
targetID, webhookURL string, targetID, cfg string,
) *database.Delivery { ) *database.Delivery {
d := &database.Delivery{ d := &database.Delivery{
EventID: event.ID, EventID: event.ID,
@@ -1996,7 +2143,7 @@ func buildSlackRetryDelivery(
Target: database.Target{ Target: database.Target{
Name: "test-slack-retry", Name: "test-slack-retry",
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
URL: webhookURL, Config: cfg,
MaxAttempts: 5, MaxAttempts: 5,
}, },
} }
@@ -2015,6 +2162,11 @@ func TestDeliverSlack_WithRetries_SchedulesRetry(
e := testEngine(t, 1) e := testEngine(t, 1)
targetID := uuid.New().String() targetID := uuid.New().String()
slackCfg, err := json.Marshal(
delivery.SlackTargetConfig{WebhookURL: ts.URL},
)
require.NoError(t, err)
event := seedEvent(t, db, `{"slack":"retry"}`) event := seedEvent(t, db, `{"slack":"retry"}`)
dlv := seedDelivery( dlv := seedDelivery(
@@ -2023,7 +2175,7 @@ func TestDeliverSlack_WithRetries_SchedulesRetry(
) )
d := buildSlackRetryDelivery( d := buildSlackRetryDelivery(
dlv, event, targetID, ts.URL, dlv, event, targetID, string(slackCfg),
) )
task := &delivery.Task{ task := &delivery.Task{
internal/delivery/event_db_isolation_test.go
+6 -4
View File
@@ -69,6 +69,7 @@ func TestEventDBHoldsNoTargetRows(t *testing.T) {
)) ))
defer ts.Close() defer ts.Close()
cfg := iHTTPConfig(ts.URL)
targetID := uuid.New().String() targetID := uuid.New().String()
dbPath := s.DBMgr.DBPath(s.WebhookID) dbPath := s.DBMgr.DBPath(s.WebhookID)
@@ -84,7 +85,7 @@ func TestEventDBHoldsNoTargetRows(t *testing.T) {
) )
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"leaky-target", ts.URL, 5, 1, &body, "leaky-target", cfg, 5, 1, &body,
) )
s.Engine.ExportProcessNewTask(context.TODO(), &task) s.Engine.ExportProcessNewTask(context.TODO(), &task)
@@ -101,7 +102,7 @@ func TestEventDBHoldsNoTargetRows(t *testing.T) {
// https://git.eeqj.de/sneak/webhooker/issues/107. // https://git.eeqj.de/sneak/webhooker/issues/107.
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "leaky-target", t, s.MainDB, targetID, s.WebhookID, "leaky-target",
database.TargetTypeHTTP, ts.URL, 5, database.TargetTypeHTTP, cfg, 5,
) )
rd := iSeedDelivery( rd := iSeedDelivery(
@@ -110,7 +111,7 @@ func TestEventDBHoldsNoTargetRows(t *testing.T) {
) )
rTask := iTask( rTask := iTask(
rd, event, s.WebhookID, targetID, rd, event, s.WebhookID, targetID,
"leaky-target", ts.URL, 5, 2, &body, "leaky-target", cfg, 5, 2, &body,
) )
s.Engine.ExportProcessRetryTask(context.TODO(), &rTask) s.Engine.ExportProcessRetryTask(context.TODO(), &rTask)
@@ -137,6 +138,7 @@ func TestEventDBHoldsNoTargetRowsOnFailedDelivery(t *testing.T) {
)) ))
defer ts.Close() defer ts.Close()
cfg := iHTTPConfig(ts.URL)
targetID := uuid.New().String() targetID := uuid.New().String()
event := iSeedEvent( event := iSeedEvent(
@@ -150,7 +152,7 @@ func TestEventDBHoldsNoTargetRowsOnFailedDelivery(t *testing.T) {
) )
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"leaky-target", ts.URL, 1, 1, &body, "leaky-target", cfg, 1, 1, &body,
) )
s.Engine.ExportProcessNewTask(context.TODO(), &task) s.Engine.ExportProcessNewTask(context.TODO(), &task)
internal/delivery/event_timestamp_test.go
+20 -6
View File
@@ -93,6 +93,17 @@ func (s *tsSink) text(t *testing.T) string {
} }
} }
func tsSlackConfig(t *testing.T, url string) string {
t.Helper()
data, err := json.Marshal(
delivery.SlackTargetConfig{WebhookURL: url},
)
require.NoError(t, err)
return string(data)
}
// tsSeedEvent writes an event whose CreatedAt is tsEventCreatedAt // tsSeedEvent writes an event whose CreatedAt is tsEventCreatedAt
// rather than the write time, so an assertion on the rendered // rather than the write time, so an assertion on the rendered
// timestamp cannot pass by accident against "roughly now". // timestamp cannot pass by accident against "roughly now".
@@ -131,7 +142,7 @@ func tsSeedEvent(
// tsSeedTarget writes the slack target row into the main database. // tsSeedTarget writes the slack target row into the main database.
// The retry path confirms the target still exists before sending. // The retry path confirms the target still exists before sending.
func tsSeedTarget( func tsSeedTarget(
t *testing.T, mainDB *gorm.DB, webhookID, webhookURL string, t *testing.T, mainDB *gorm.DB, webhookID, config string,
) database.Target { ) database.Target {
t.Helper() t.Helper()
@@ -139,7 +150,7 @@ func tsSeedTarget(
WebhookID: webhookID, WebhookID: webhookID,
Name: "slack-sink", Name: "slack-sink",
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
URL: webhookURL, Config: config,
Active: true, Active: true,
MaxAttempts: 1, MaxAttempts: 1,
} }
@@ -165,7 +176,7 @@ func tsTask(
TargetID: target.ID, TargetID: target.ID,
TargetName: target.Name, TargetName: target.Name,
TargetType: database.TargetTypeSlack, TargetType: database.TargetTypeSlack,
TargetURL: target.URL, TargetConfig: target.Config,
MaxAttempts: 1, MaxAttempts: 1,
Method: event.Method, Method: event.Method,
Headers: event.Headers, Headers: event.Headers,
@@ -215,7 +226,8 @@ func (c tsCase) run(t *testing.T) (iSetup, database.Delivery, string) {
s := newISetup(t) s := newISetup(t)
sink := newTSSink(t) sink := newTSSink(t)
target := tsSeedTarget(t, s.MainDB, s.WebhookID, sink.URL) cfg := tsSlackConfig(t, sink.URL)
target := tsSeedTarget(t, s.MainDB, s.WebhookID, cfg)
event := tsSeedEvent(t, s.WebhookDB, s.WebhookID) event := tsSeedEvent(t, s.WebhookDB, s.WebhookID)
d := iSeedDelivery( d := iSeedDelivery(
@@ -326,7 +338,8 @@ func TestFormatSlackMessageOverTaskReconstructedEvent(
s := newISetup(t) s := newISetup(t)
target := tsSeedTarget(t, s.MainDB, s.WebhookID, tsUndeliverableHook) cfg := tsSlackConfig(t, tsUndeliverableHook)
target := tsSeedTarget(t, s.MainDB, s.WebhookID, cfg)
event := tsSeedEvent(t, s.WebhookDB, s.WebhookID) event := tsSeedEvent(t, s.WebhookDB, s.WebhookID)
d := iSeedDelivery( d := iSeedDelivery(
@@ -397,7 +410,8 @@ func TestEventReconstructionSurvivesAReapedRow(t *testing.T) {
s := newISetup(t) s := newISetup(t)
target := tsSeedTarget(t, s.MainDB, s.WebhookID, tsUndeliverableHook) cfg := tsSlackConfig(t, tsUndeliverableHook)
target := tsSeedTarget(t, s.MainDB, s.WebhookID, cfg)
event := tsSeedEvent(t, s.WebhookDB, s.WebhookID) event := tsSeedEvent(t, s.WebhookDB, s.WebhookID)
d := iSeedDelivery( d := iSeedDelivery(
internal/delivery/export_test.go
+36 -22
View File
@@ -77,10 +77,10 @@ func ExportIsForwardableHeader(name string) bool {
func ExportApplyRequestHeaders( func ExportApplyRequestHeaders(
req *http.Request, req *http.Request,
event *database.Event, event *database.Event,
configured map[string]string, cfg *HTTPTargetConfig,
userAgent string, userAgent string,
) []string { ) []string {
return applyRequestHeaders(req, event, configured, userAgent) return applyRequestHeaders(req, event, cfg, userAgent)
} }
// ExportTruncate exposes truncate for testing. // ExportTruncate exposes truncate for testing.
@@ -158,11 +158,6 @@ func (e *Engine) ExportUpdateDeliveryStatus(
return e.updateDeliveryStatus(webhookDB, d, "", status) return e.updateDeliveryStatus(webhookDB, d, "", status)
} }
// ExportNotify exposes notify.
func (e *Engine) ExportNotify(tasks []Task) {
e.notify(tasks)
}
// ExportProcessNewTask exposes processNewTask. // ExportProcessNewTask exposes processNewTask.
func (e *Engine) ExportProcessNewTask( func (e *Engine) ExportProcessNewTask(
ctx context.Context, task *Task, ctx context.Context, task *Task,
@@ -221,23 +216,37 @@ func (e *Engine) ExportSetSlackCircuitBreaker(
e.slackTarget.circuitBreakers.Store(targetID, cb) e.slackTarget.circuitBreakers.Store(targetID, cb)
} }
// ExportParseHTTPConfig exposes parseHTTPConfig.
func (e *Engine) ExportParseHTTPConfig(
configJSON string,
) (*HTTPTargetConfig, error) {
return parseHTTPConfig(configJSON)
}
// ExportParseSlackConfig exposes parseSlackConfig.
func (e *Engine) ExportParseSlackConfig(
configJSON string,
) (*SlackTargetConfig, error) {
return parseSlackConfig(configJSON)
}
// ExportDoHTTPRequest exposes the http target's // ExportDoHTTPRequest exposes the http target's
// doHTTPRequest. // doHTTPRequest.
func (e *Engine) ExportDoHTTPRequest( func (e *Engine) ExportDoHTTPRequest(
ctx context.Context, ctx context.Context,
target *database.Target, cfg *HTTPTargetConfig,
event *database.Event, event *database.Event,
) (int, string, int64, error) { ) (int, string, int64, error) {
return e.httpTarget.doHTTPRequest(ctx, target, event) return e.httpTarget.doHTTPRequest(ctx, cfg, event)
} }
// ExportClientForRequest exposes the http target's // ExportClientForRequest exposes the http target's
// clientForRequest. // clientForRequest.
func (e *Engine) ExportClientForRequest( func (e *Engine) ExportClientForRequest(
timeoutSeconds int, cfg *HTTPTargetConfig,
originScoped []string, originScoped []string,
) *http.Client { ) *http.Client {
return e.httpTarget.clientForRequest(timeoutSeconds, originScoped) return e.httpTarget.clientForRequest(cfg, originScoped)
} }
// ErrExportTooManyRedirects exposes the sentinel the redirect // ErrExportTooManyRedirects exposes the sentinel the redirect
@@ -272,27 +281,32 @@ func (e *Engine) ExportRecoverPendingDeliveries(
ctx context.Context, ctx context.Context,
webhookDB *gorm.DB, webhookDB *gorm.DB,
webhookID string, webhookID string,
atStart bool,
) { ) {
e.recoverPendingDeliveries( e.recoverPendingDeliveries(
ctx, webhookDB, webhookID, atStart, ctx, webhookDB, webhookID,
) )
} }
// ExportRecoverWebhookDeliveries exposes // ExportRecoverWebhookDeliveries exposes
// recoverWebhookDeliveries. // recoverWebhookDeliveries.
func (e *Engine) ExportRecoverWebhookDeliveries( func (e *Engine) ExportRecoverWebhookDeliveries(
ctx context.Context, webhookID string, atStart bool, ctx context.Context, webhookID string,
) { ) {
e.recoverWebhookDeliveries(ctx, webhookID, atStart) e.recoverWebhookDeliveries(ctx, webhookID)
} }
// ExportRecoverDeliveries exposes recoverDeliveries, the recovery // ExportRecoverInFlight exposes recoverInFlight.
// routine. func (e *Engine) ExportRecoverInFlight(
func (e *Engine) ExportRecoverDeliveries( ctx context.Context,
ctx context.Context, atStart bool,
) { ) {
e.recoverDeliveries(ctx, atStart) e.recoverInFlight(ctx)
}
// ExportSweepWebhookRetries exposes sweepWebhookRetries.
func (e *Engine) ExportSweepWebhookRetries(
ctx context.Context, webhookID string,
) {
e.sweepWebhookRetries(ctx, webhookID)
} }
// ExportStart exposes start for testing. // ExportStart exposes start for testing.
@@ -339,9 +353,9 @@ func (e *Engine) ExportRetainDelivery(deliveryID string) bool {
// ExportRecoverRetryingDeliveries exposes recoverRetryingDeliveries. // ExportRecoverRetryingDeliveries exposes recoverRetryingDeliveries.
func (e *Engine) ExportRecoverRetryingDeliveries( func (e *Engine) ExportRecoverRetryingDeliveries(
ctx context.Context, webhookDB *gorm.DB, webhookID string, webhookDB *gorm.DB, webhookID string,
) { ) {
e.recoverRetryingDeliveries(ctx, webhookDB, webhookID) e.recoverRetryingDeliveries(webhookDB, webhookID)
} }
// ExportFailMissingTarget exposes failMissingTarget, so a test can hand // ExportFailMissingTarget exposes failMissingTarget, so a test can hand
internal/delivery/inflight.go
+9 -9
View File
@@ -6,10 +6,10 @@ import "sync"
// //
// A delivery is owned from the moment a task for it is handed to a // A delivery is owned from the moment a task for it is handed to a
// channel or to a retry timer until the engine has no further plan for // channel or to a retry timer until the engine has no further plan for
// it in memory. The recovery routine, at start and on every tick of the // it in memory. Restart recovery and both arms of the periodic sweep
// sweep, re-dispatches only deliveries the set does not hold, which is // re-dispatch only deliveries the set does not hold, which is what
// what makes it exact rather than a guess about how long a row has sat // makes them exact rather than a guess about how long a row has sat at
// at pending. // pending.
// //
// This replaces reasoning from timestamps. A delivery's row says // This replaces reasoning from timestamps. A delivery's row says
// pending from creation until its outcome is written, which covers // pending from creation until its outcome is written, which covers
@@ -25,8 +25,8 @@ import "sync"
// In-memory state is sufficient because a data directory admits one // In-memory state is sufficient because a data directory admits one
// process: internal/datadir takes an flock on it at startup and a // process: internal/datadir takes an flock on it at startup and a
// second instance refuses to run. Deliveries owned by a process that // second instance refuses to run. Deliveries owned by a process that
// died are not in any successor's set, and the recovery routine's run // died are not in any successor's set, and restart recovery is what
// at start is what picks those up. // picks those up.
// //
// References are counted rather than held as a plain set because // References are counted rather than held as a plain set because
// ownership outlives the worker that took it. A target that schedules // ownership outlives the worker that took it. A target that schedules
@@ -61,9 +61,9 @@ func (s *inflightSet) retain(deliveryID string) {
// retainIdle takes the first reference on a delivery, and reports // retainIdle takes the first reference on a delivery, and reports
// whether it got it. It fails when the engine already owns the // whether it got it. It fails when the engine already owns the
// delivery, which is what makes two claimants — notify and the // delivery, which is what makes two claimants — restart recovery and
// recovery routine, say — mutually exclusive rather than merely // the sweep run concurrently, or two sweep arms — mutually exclusive
// atomic. // rather than merely atomic.
func (s *inflightSet) retainIdle(deliveryID string) bool { func (s *inflightSet) retainIdle(deliveryID string) bool {
s.mu.Lock() s.mu.Lock()
defer s.mu.Unlock() defer s.mu.Unlock()
internal/delivery/inflight_test.go
+26 -30
View File
@@ -84,9 +84,9 @@ func TestSweepLeavesAQueuedDeliveryAlone(t *testing.T) {
) )
rAgePending(t, s.WebhookDB, d.ID) rAgePending(t, s.WebhookDB, d.ID)
// Queued exactly as StoreEvent queues it, and never dequeued: // Queued exactly as the receiver queues it, and never dequeued:
// no workers are running in this engine. // no workers are running in this engine.
s.Engine.ExportNotify([]delivery.Task{{ s.Engine.Notify([]delivery.Task{{
DeliveryID: d.ID, DeliveryID: d.ID,
EventID: event.ID, EventID: event.ID,
WebhookID: s.WebhookID, WebhookID: s.WebhookID,
@@ -95,8 +95,8 @@ func TestSweepLeavesAQueuedDeliveryAlone(t *testing.T) {
require.Equal(t, 1, s.Engine.ExportInflightHeld()) require.Equal(t, 1, s.Engine.ExportInflightHeld())
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportSweepWebhookRetries(
context.Background(), s.WebhookID, onTick, context.Background(), s.WebhookID,
) )
tasks := fDrain(s.Engine) tasks := fDrain(s.Engine)
@@ -107,11 +107,9 @@ func TestSweepLeavesAQueuedDeliveryAlone(t *testing.T) {
) )
} }
// TestRecoveryAndSweepDoNotDoubleDispatch drives the recovery routine's // TestRecoveryAndSweepDoNotDoubleDispatch drives the two entry points
// run at start and a tick of the sweep at once against one aged pending // the engine starts concurrently against one aged pending row. Before
// row. The engine runs them one after the other, but notify can race // ownership they both dispatched it.
// either, and ownership is what keeps any two claimants from both
// sending. Before ownership the two runs both dispatched the row.
func TestRecoveryAndSweepDoNotDoubleDispatch(t *testing.T) { func TestRecoveryAndSweepDoNotDoubleDispatch(t *testing.T) {
t.Parallel() t.Parallel()
@@ -135,12 +133,12 @@ func TestRecoveryAndSweepDoNotDoubleDispatch(t *testing.T) {
wg.Go(func() { wg.Go(func() {
s.Engine.ExportRecoverPendingDeliveries( s.Engine.ExportRecoverPendingDeliveries(
ctx, s.WebhookDB, s.WebhookID, atStart, ctx, s.WebhookDB, s.WebhookID,
) )
}) })
wg.Go(func() { wg.Go(func() {
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportSweepWebhookRetries(
ctx, s.WebhookID, onTick, ctx, s.WebhookID,
) )
}) })
wg.Wait() wg.Wait()
@@ -224,7 +222,7 @@ func TestOwnershipIsReleasedAfterDelivery(t *testing.T) {
body := `{"released":true}` body := `{"released":true}`
s.Engine.ExportNotify([]delivery.Task{{ s.Engine.Notify([]delivery.Task{{
DeliveryID: d.ID, DeliveryID: d.ID,
EventID: event.ID, EventID: event.ID,
WebhookID: s.WebhookID, WebhookID: s.WebhookID,
@@ -248,11 +246,10 @@ func TestOwnershipIsReleasedAfterDelivery(t *testing.T) {
} }
// TestNotifyAfterRecoveryDoesNotSendAgain is the startup race of // TestNotifyAfterRecoveryDoesNotSendAgain is the startup race of
// https://git.eeqj.de/sneak/webhooker/issues/299. StoreEvent has // https://git.eeqj.de/sneak/webhooker/issues/299. The receiver has
// written a delivery, the recovery routine's run at start finds it // written a delivery, restart recovery finds it pending, sends it and
// pending, sends it and releases it, and only then does StoreEvent's // releases it, and only then does the receiver's Notify for it arrive.
// notify for it arrive. Nothing owns the delivery by then, so notify // Nothing owns the delivery by then, so Notify takes it.
// takes it.
func TestNotifyAfterRecoveryDoesNotSendAgain(t *testing.T) { func TestNotifyAfterRecoveryDoesNotSendAgain(t *testing.T) {
t.Parallel() t.Parallel()
@@ -276,8 +273,7 @@ func TestNotifyAfterRecoveryDoesNotSendAgain(t *testing.T) {
) )
}() }()
// The recovery routine's run at start sends the delivery and lets // Restart recovery sends the delivery and lets it go.
// it go.
iWaitForDelivered(t, s.WebhookDB, d.ID) iWaitForDelivered(t, s.WebhookDB, d.ID)
require.Eventually( require.Eventually(
t, t,
@@ -289,7 +285,7 @@ func TestNotifyAfterRecoveryDoesNotSendAgain(t *testing.T) {
body := event.Body body := event.Body
s.Engine.ExportNotify([]delivery.Task{{ s.Engine.Notify([]delivery.Task{{
DeliveryID: d.ID, DeliveryID: d.ID,
EventID: event.ID, EventID: event.ID,
WebhookID: s.WebhookID, WebhookID: s.WebhookID,
@@ -300,7 +296,7 @@ func TestNotifyAfterRecoveryDoesNotSendAgain(t *testing.T) {
EntrypointID: event.EntrypointID, EntrypointID: event.EntrypointID,
}}) }})
// notify took the delivery, and a worker releases it once it has // Notify took the delivery, and a worker releases it once it has
// run the task. // run the task.
require.Eventually( require.Eventually(
t, t,
@@ -338,7 +334,7 @@ func TestRetryingRecoverySkipsASuccessfulResult(t *testing.T) {
rSeedResult(t, s.WebhookDB, d.ID, 2, true) rSeedResult(t, s.WebhookDB, d.ID, 2, true)
s.Engine.ExportRecoverRetryingDeliveries( s.Engine.ExportRecoverRetryingDeliveries(
context.Background(), s.WebhookDB, s.WebhookID, s.WebhookDB, s.WebhookID,
) )
assert.Empty( assert.Empty(
@@ -353,8 +349,8 @@ func TestRetryingRecoverySkipsASuccessfulResult(t *testing.T) {
) )
} }
// TestRetryingSweepSkipsASuccessfulResult is the same rule on a tick of // TestRetryingSweepSkipsASuccessfulResult is the same rule on the
// the sweep. // periodic sweep's retrying arm.
func TestRetryingSweepSkipsASuccessfulResult(t *testing.T) { func TestRetryingSweepSkipsASuccessfulResult(t *testing.T) {
t.Parallel() t.Parallel()
@@ -372,8 +368,8 @@ func TestRetryingSweepSkipsASuccessfulResult(t *testing.T) {
rSeedResult(t, s.WebhookDB, d.ID, 1, false) rSeedResult(t, s.WebhookDB, d.ID, 1, false)
rSeedResult(t, s.WebhookDB, d.ID, 2, true) rSeedResult(t, s.WebhookDB, d.ID, 2, true)
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportSweepWebhookRetries(
context.Background(), s.WebhookID, onTick, context.Background(), s.WebhookID,
) )
assert.Empty(t, fDrain(s.Engine)) assert.Empty(t, fDrain(s.Engine))
@@ -424,8 +420,8 @@ func TestScheduledRetryIsNotSweptDuringBackoff(t *testing.T) {
require.Equal(t, 1, s.Engine.ExportInflightHeld()) require.Equal(t, 1, s.Engine.ExportInflightHeld())
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportSweepWebhookRetries(
context.Background(), s.WebhookID, onTick, context.Background(), s.WebhookID,
) )
assert.Empty( assert.Empty(
@@ -457,7 +453,7 @@ func TestRedispatchStampsTheRow(t *testing.T) {
ctx := context.Background() ctx := context.Background()
s.Engine.ExportRecoverWebhookDeliveries(ctx, s.WebhookID, onTick) s.Engine.ExportSweepWebhookRetries(ctx, s.WebhookID)
require.Len(t, fDrain(s.Engine), 1) require.Len(t, fDrain(s.Engine), 1)
var row database.Delivery var row database.Delivery
internal/delivery/metrics_test.go
+17 -14
View File
@@ -192,7 +192,7 @@ func mDeliverOK(t *testing.T, s iSetup) {
body := event.Body body := event.Body
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"metrics-ok", ts.URL, 3, 1, &body, "metrics-ok", iHTTPConfig(ts.URL), 3, 1, &body,
) )
s.Engine.ExportProcessNewTask(context.TODO(), &task) s.Engine.ExportProcessNewTask(context.TODO(), &task)
@@ -226,17 +226,18 @@ func mExhaustRetries(t *testing.T, s iSetup) {
) )
body := event.Body body := event.Body
cfg := iHTTPConfig(ts.URL)
// The retry below is only run if its target still exists; see // The retry below is only run if its target still exists; see
// https://git.eeqj.de/sneak/webhooker/issues/107. // https://git.eeqj.de/sneak/webhooker/issues/107.
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "metrics-fail", t, s.MainDB, targetID, s.WebhookID, "metrics-fail",
database.TargetTypeHTTP, ts.URL, 2, database.TargetTypeHTTP, cfg, 2,
) )
first := iTask( first := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"metrics-fail", ts.URL, 2, 1, &body, "metrics-fail", cfg, 2, 1, &body,
) )
s.Engine.ExportProcessNewTask(context.TODO(), &first) s.Engine.ExportProcessNewTask(context.TODO(), &first)
@@ -250,7 +251,7 @@ func mExhaustRetries(t *testing.T, s iSetup) {
// deterministic and off the wall clock. // deterministic and off the wall clock.
second := iTask( second := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"metrics-fail", ts.URL, 2, 2, &body, "metrics-fail", cfg, 2, 2, &body,
) )
s.Engine.ExportProcessRetryTask( s.Engine.ExportProcessRetryTask(
@@ -288,6 +289,7 @@ func TestDeliveryMetrics_CircuitBreakerGauge(t *testing.T) {
) )
body := event.Body body := event.Body
cfg := iHTTPConfig(ts.URL)
// An attempt budget above the failure threshold, so the breaker // An attempt budget above the failure threshold, so the breaker
// rather than the budget is what stops the delivery. // rather than the budget is what stops the delivery.
@@ -297,12 +299,12 @@ func TestDeliveryMetrics_CircuitBreakerGauge(t *testing.T) {
// see https://git.eeqj.de/sneak/webhooker/issues/107. // see https://git.eeqj.de/sneak/webhooker/issues/107.
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "metrics-trip", t, s.MainDB, targetID, s.WebhookID, "metrics-trip",
database.TargetTypeHTTP, ts.URL, maxAttempts, database.TargetTypeHTTP, cfg, maxAttempts,
) )
first := iTask( first := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"metrics-trip", ts.URL, maxAttempts, 1, &body, "metrics-trip", cfg, maxAttempts, 1, &body,
) )
s.Engine.ExportProcessNewTask(context.TODO(), &first) s.Engine.ExportProcessNewTask(context.TODO(), &first)
@@ -314,7 +316,7 @@ func TestDeliveryMetrics_CircuitBreakerGauge(t *testing.T) {
ExportDefaultFailureThreshold; attempt++ { ExportDefaultFailureThreshold; attempt++ {
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"metrics-trip", ts.URL, maxAttempts, attempt, &body, "metrics-trip", cfg, maxAttempts, attempt, &body,
) )
s.Engine.ExportProcessRetryTask( s.Engine.ExportProcessRetryTask(
@@ -361,16 +363,17 @@ func TestDeliveryMetrics_BreakerBlockedIsNotAnAttempt(
) )
body := event.Body body := event.Body
cfg := iHTTPConfig(ts.URL)
maxAttempts := delivery.ExportDefaultFailureThreshold + 5 maxAttempts := delivery.ExportDefaultFailureThreshold + 5
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "metrics-blocked", t, s.MainDB, targetID, s.WebhookID, "metrics-blocked",
database.TargetTypeHTTP, ts.URL, maxAttempts, database.TargetTypeHTTP, cfg, maxAttempts,
) )
first := iTask( first := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"metrics-blocked", ts.URL, maxAttempts, 1, &body, "metrics-blocked", cfg, maxAttempts, 1, &body,
) )
s.Engine.ExportProcessNewTask(context.TODO(), &first) s.Engine.ExportProcessNewTask(context.TODO(), &first)
@@ -379,7 +382,7 @@ func TestDeliveryMetrics_BreakerBlockedIsNotAnAttempt(
ExportDefaultFailureThreshold; attempt++ { ExportDefaultFailureThreshold; attempt++ {
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"metrics-blocked", ts.URL, maxAttempts, attempt, &body, "metrics-blocked", cfg, maxAttempts, attempt, &body,
) )
s.Engine.ExportProcessRetryTask(context.TODO(), &task) s.Engine.ExportProcessRetryTask(context.TODO(), &task)
@@ -402,7 +405,7 @@ func TestDeliveryMetrics_BreakerBlockedIsNotAnAttempt(
blocked := iTask( blocked := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"metrics-blocked", ts.URL, maxAttempts, "metrics-blocked", cfg, maxAttempts,
delivery.ExportDefaultFailureThreshold+1, &body, delivery.ExportDefaultFailureThreshold+1, &body,
) )
@@ -441,8 +444,8 @@ func TestDeliveryMetrics_OrphanedRetryFailureLabelled(
t, s, database.TargetTypeLog, t, s, database.TargetTypeLog,
) )
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportSweepWebhookRetries(
context.Background(), s.WebhookID, onTick, context.Background(), s.WebhookID,
) )
iAssertStatus(t, s.WebhookDB, deliveryID, iAssertStatus(t, s.WebhookDB, deliveryID,
@@ -471,7 +474,7 @@ func TestDeliveryMetrics_QueueDepthGauges(t *testing.T) {
iCreateTarget(t, s.MainDB, targetID, s.WebhookID, iCreateTarget(t, s.MainDB, targetID, s.WebhookID,
"queue-depth-target", database.TargetTypeHTTP, "queue-depth-target", database.TargetTypeHTTP,
"https://example.com/hook", 3, iHTTPConfig("https://example.com/hook"), 3,
) )
event := iSeedEvent( event := iSeedEvent(
internal/delivery/queue_depth.go
+1 -2
View File
@@ -20,8 +20,7 @@ const queueDepthSampleInterval = 30 * time.Second
// to be seeded correctly at startup from rows written by a previous // to be seeded correctly at startup from rows written by a previous
// process, and would drift permanently on any transition that failed // process, and would drift permanently on any transition that failed
// to persist. Counting is the measurement that cannot go wrong, and // to persist. Counting is the measurement that cannot go wrong, and
// it is the same whole-database walk each tick of the sweep already // it is the same whole-database walk the retry sweep already makes.
// makes.
func (e *Engine) queueDepthSampler(ctx context.Context) { func (e *Engine) queueDepthSampler(ctx context.Context) {
defer e.wg.Done() defer e.wg.Done()
internal/delivery/queue_depth_gormlog_test.go
+1 -1
View File
@@ -102,7 +102,7 @@ func TestQueueDepthSample_LogsNoBoundValue(t *testing.T) {
iCreateTarget(t, mainDB.DB(), targetID, webhookID, iCreateTarget(t, mainDB.DB(), targetID, webhookID,
"queue-depth-gormlog-target", database.TargetTypeHTTP, "queue-depth-gormlog-target", database.TargetTypeHTTP,
"https://example.com/hook", 3, iHTTPConfig("https://example.com/hook"), 3,
) )
event := iSeedEvent( event := iSeedEvent(
internal/delivery/recovery_durability_test.go
+15 -16
View File
@@ -88,7 +88,7 @@ func TestRecoverySkipsPendingWithSuccessfulResult(
) )
s.Engine.ExportRecoverPendingDeliveries( s.Engine.ExportRecoverPendingDeliveries(
context.Background(), s.WebhookDB, s.WebhookID, atStart, context.Background(), s.WebhookDB, s.WebhookID,
) )
select { select {
@@ -145,7 +145,7 @@ func TestRecoveryContinuesTheAttemptNumbering(t *testing.T) {
rSeedResult(t, s.WebhookDB, d.ID, 2, false) rSeedResult(t, s.WebhookDB, d.ID, 2, false)
s.Engine.ExportRecoverPendingDeliveries( s.Engine.ExportRecoverPendingDeliveries(
context.Background(), s.WebhookDB, s.WebhookID, atStart, context.Background(), s.WebhookDB, s.WebhookID,
) )
select { select {
@@ -183,8 +183,8 @@ func TestSweepRecoversStrandedPending(t *testing.T) {
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportSweepWebhookRetries(
context.Background(), s.WebhookID, onTick, context.Background(), s.WebhookID,
) )
select { select {
@@ -230,7 +230,7 @@ func TestSweepClaimsAStrandedDeliveryOnlyOnce(t *testing.T) {
ctx := context.Background() ctx := context.Background()
s.Engine.ExportRecoverWebhookDeliveries(ctx, s.WebhookID, onTick) s.Engine.ExportSweepWebhookRetries(ctx, s.WebhookID)
select { select {
case task := <-s.Engine.ExportDeliveryCh(): case task := <-s.Engine.ExportDeliveryCh():
@@ -246,7 +246,7 @@ func TestSweepClaimsAStrandedDeliveryOnlyOnce(t *testing.T) {
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
s.Engine.ExportRecoverWebhookDeliveries(ctx, s.WebhookID, onTick) s.Engine.ExportSweepWebhookRetries(ctx, s.WebhookID)
select { select {
case task := <-s.Engine.ExportDeliveryCh(): case task := <-s.Engine.ExportDeliveryCh():
@@ -258,9 +258,9 @@ func TestSweepClaimsAStrandedDeliveryOnlyOnce(t *testing.T) {
} }
} }
// TestSweepSettlesStrandedPendingWithoutResending is the reconcile on a // TestSweepSettlesStrandedPendingWithoutResending is the sweep's own
// tick of the sweep: a stranded delivery holding a successful result is // version of the reconcile: a stranded delivery holding a successful
// settled where it stands, and the receiver hears nothing. // result is settled where it stands, and the receiver hears nothing.
func TestSweepSettlesStrandedPendingWithoutResending( func TestSweepSettlesStrandedPendingWithoutResending(
t *testing.T, t *testing.T,
) { ) {
@@ -280,8 +280,8 @@ func TestSweepSettlesStrandedPendingWithoutResending(
rSeedResult(t, s.WebhookDB, d.ID, 1, true) rSeedResult(t, s.WebhookDB, d.ID, 1, true)
rAgePending(t, s.WebhookDB, d.ID) rAgePending(t, s.WebhookDB, d.ID)
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportSweepWebhookRetries(
context.Background(), s.WebhookID, onTick, context.Background(), s.WebhookID,
) )
select { select {
@@ -312,9 +312,8 @@ func TestSweepSettlesStrandedPendingWithoutResending(
// TestFailedResultWriteLeavesDeliveryRecoverable is the rule the // TestFailedResultWriteLeavesDeliveryRecoverable is the rule the
// targets now follow: a bookkeeping write that fails must not advance // targets now follow: a bookkeeping write that fails must not advance
// the status, because pending and retrying are the states the recovery // the status, because pending and retrying are the states the sweeps
// routine recovers and delivered is a claim the database refused to // recover and delivered is a claim the database refused to record.
// record.
func TestFailedResultWriteLeavesDeliveryRecoverable( func TestFailedResultWriteLeavesDeliveryRecoverable(
t *testing.T, t *testing.T,
) { ) {
@@ -357,7 +356,7 @@ func TestFailedResultWriteLeavesDeliveryRecoverable(
Target: database.Target{ Target: database.Target{
Name: "unwritable", Name: "unwritable",
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
URL: ts.URL, Config: iHTTPConfig(ts.URL),
MaxAttempts: 1, MaxAttempts: 1,
}, },
} }
@@ -448,7 +447,7 @@ func TestFailedResultWriteWithRetriesLeavesDeliveryRecoverable(
Target: database.Target{ Target: database.Target{
Name: "unwritable", Name: "unwritable",
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
URL: ts.URL, Config: iHTTPConfig(ts.URL),
MaxAttempts: 3, MaxAttempts: 3,
}, },
} }
internal/delivery/recovery_test.go
-389
View File
@@ -1,389 +0,0 @@
package delivery_test
import (
"context"
"log/slog"
"net/http"
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/database/databasetest"
"sneak.berlin/go/webhooker/internal/delivery"
)
// These tests cover the recovery routine, which the engine runs once
// when it starts and then on every tick of the sweep
// (https://git.eeqj.de/sneak/webhooker/issues/542). Every case runs it
// both ways, each on a webhook of its own.
// atStart and onTick are the two values of the recovery routine's
// atStart parameter: its run when the engine starts, and a tick of the
// sweep.
const (
atStart = true
onTick = false
)
// recoveryRun names one of the two runs of the recovery routine.
type recoveryRun struct {
name string
atStart bool
}
// recoveryRuns returns the routine's run at start and a tick.
func recoveryRuns() []recoveryRun {
return []recoveryRun{
{"at start", atStart},
{"on a tick", onTick},
}
}
// recoverySetup gives a fresh webhook its main-database row and an http
// target with retries, and returns the setup and the target's ID. No
// worker runs, so whatever the routine queues stays in the channels for
// the test to read, and nothing is sent to the target.
func recoverySetup(t *testing.T) (iSetup, string) {
t.Helper()
s := newISetup(t)
targetID := uuid.New().String()
iCreateWebhook(t, s.MainDB, s.WebhookID, "recovery")
iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "recovery",
database.TargetTypeHTTP,
"http://example.com/hook", 20,
)
return s, targetID
}
// seedRecoveryDelivery seeds a delivery at status with the given
// recorded attempts, failed but for the last when lastSucceeded is set,
// all recorded now.
func seedRecoveryDelivery(
t *testing.T,
s iSetup,
targetID string,
status database.DeliveryStatus,
attempts int,
lastSucceeded bool,
) string {
t.Helper()
event := iSeedEvent(t, s.WebhookDB, s.WebhookID, `{"recovery":true}`)
d := iSeedDelivery(t, s.WebhookDB, event.ID, targetID, status)
for n := 1; n <= attempts; n++ {
rSeedResult(
t, s.WebhookDB, d.ID, n, lastSucceeded && n == attempts,
)
}
return d.ID
}
// backdateAttempts moves every recorded attempt of a delivery an hour
// into the past, long past the backoff after its first attempt.
func backdateAttempts(t *testing.T, s iSetup, deliveryID string) {
t.Helper()
require.NoError(t, s.WebhookDB.Model(&database.DeliveryResult{}).
Where("delivery_id = ?", deliveryID).
UpdateColumn("created_at", s.WebhookDB.NowFunc().Add(-time.Hour)).
Error)
}
// requireQueued waits for a task on ch. A retrying delivery reaches the
// retry channel through a timer, even one due now, so it is waited for.
func requireQueued(
t *testing.T, ch <-chan delivery.Task,
) delivery.Task {
t.Helper()
select {
case task := <-ch:
return task
case <-time.After(5 * time.Second):
t.Fatal("the recovery routine queued nothing")
return delivery.Task{}
}
}
// assertNothingQueued asserts the engine queues nothing, giving a retry
// timer that is due now the moment it needs to fire.
func assertNothingQueued(t *testing.T, e *delivery.Engine) {
t.Helper()
select {
case task := <-e.ExportDeliveryCh():
t.Errorf("queued pending delivery %s", task.DeliveryID)
case task := <-e.ExportRetryCh():
t.Errorf("queued retrying delivery %s", task.DeliveryID)
case <-time.After(200 * time.Millisecond):
}
}
// TestRecovery_RetryingBackoffPassed: a retrying delivery whose backoff
// is over is queued for its next attempt at once.
func TestRecovery_RetryingBackoffPassed(t *testing.T) {
t.Parallel()
for _, run := range recoveryRuns() {
t.Run(run.name, func(t *testing.T) {
t.Parallel()
s, targetID := recoverySetup(t)
id := seedRecoveryDelivery(
t, s, targetID, database.DeliveryStatusRetrying, 1, false,
)
backdateAttempts(t, s, id)
s.Engine.ExportRecoverDeliveries(context.Background(), run.atStart)
task := requireQueued(t, s.Engine.ExportRetryCh())
assert.Equal(t, id, task.DeliveryID)
assert.Equal(t, 2, task.AttemptNum)
iAssertStatus(t, s.WebhookDB, id, database.DeliveryStatusRetrying)
})
}
}
// TestRecovery_RetryingBackoffNotPassed: a retrying delivery whose last
// attempt was just now waits out the rest of its backoff, ten attempts
// in, eight and a half minutes. It is not sent now, and the engine holds
// it, so a later tick leaves it to the timer.
func TestRecovery_RetryingBackoffNotPassed(t *testing.T) {
t.Parallel()
for _, run := range recoveryRuns() {
t.Run(run.name, func(t *testing.T) {
t.Parallel()
s, targetID := recoverySetup(t)
id := seedRecoveryDelivery(
t, s, targetID, database.DeliveryStatusRetrying, 10, false,
)
s.Engine.ExportRecoverDeliveries(context.Background(), run.atStart)
assertNothingQueued(t, s.Engine)
iAssertStatus(t, s.WebhookDB, id, database.DeliveryStatusRetrying)
assert.Equal(t, 1, s.Engine.ExportInflightHeld(),
"no retry timer holds the delivery",
)
})
}
}
// TestRecovery_PendingOlderThanTheAgeBound: a pending delivery untouched
// for longer than the sweep's age bound is sent by either run.
func TestRecovery_PendingOlderThanTheAgeBound(t *testing.T) {
t.Parallel()
for _, run := range recoveryRuns() {
t.Run(run.name, func(t *testing.T) {
t.Parallel()
s, targetID := recoverySetup(t)
id := seedRecoveryDelivery(
t, s, targetID, database.DeliveryStatusPending, 0, false,
)
rAgePending(t, s.WebhookDB, id)
s.Engine.ExportRecoverDeliveries(context.Background(), run.atStart)
task := requireQueued(t, s.Engine.ExportDeliveryCh())
assert.Equal(t, id, task.DeliveryID)
assert.Equal(t, 1, task.AttemptNum)
})
}
}
// TestRecovery_TaskCarriesEveryTargetSetting: the task the recovery
// routine queues carries every one of the target's settings, all six
// set here.
// They are copied into the task one field at a time, and a field left
// out is sent empty without any error.
func TestRecovery_TaskCarriesEveryTargetSetting(t *testing.T) {
t.Parallel()
for _, run := range recoveryRuns() {
t.Run(run.name, func(t *testing.T) {
t.Parallel()
s := newISetup(t)
iCreateWebhook(t, s.MainDB, s.WebhookID, "recovery")
target := database.Target{
WebhookID: s.WebhookID,
Name: "recovery",
Type: database.TargetTypeHTTP,
Active: true,
URL: "http://example.com/hook",
Headers: map[string]string{headerAuthorization: "Bearer t"},
TimeoutSeconds: 7,
ForwardQuery: true,
ArchiveExpiry: time.Hour,
ArchiveRotation: "daily",
MaxAttempts: 1,
}
require.NoError(t, s.MainDB.Create(&target).Error)
id := seedRecoveryDelivery(
t, s, target.ID, database.DeliveryStatusPending, 0, false,
)
rAgePending(t, s.WebhookDB, id)
s.Engine.ExportRecoverDeliveries(context.Background(), run.atStart)
task := requireQueued(t, s.Engine.ExportDeliveryCh())
assert.Equal(t, target.ID, task.TargetID)
assert.Equal(t, target.URL, task.TargetURL)
assert.Equal(t, target.Headers, task.TargetHeaders)
assert.Equal(t, target.TimeoutSeconds, task.TimeoutSeconds)
assert.Equal(t, target.ForwardQuery, task.ForwardQuery)
assert.Equal(t, target.ArchiveExpiry, task.ArchiveExpiry)
assert.Equal(t, target.ArchiveRotation, task.ArchiveRotation)
})
}
}
// TestRecovery_PendingYoungerThanTheAgeBound: a pending delivery touched
// a moment ago, as one a stop cut off is, is sent at start, which is how
// it reaches its target on the next start
// (https://git.eeqj.de/sneak/webhooker/issues/527). A tick leaves it.
func TestRecovery_PendingYoungerThanTheAgeBound(t *testing.T) {
t.Parallel()
for _, run := range recoveryRuns() {
t.Run(run.name, func(t *testing.T) {
t.Parallel()
s, targetID := recoverySetup(t)
id := seedRecoveryDelivery(
t, s, targetID, database.DeliveryStatusPending, 0, false,
)
s.Engine.ExportRecoverDeliveries(context.Background(), run.atStart)
if run.atStart {
task := requireQueued(t, s.Engine.ExportDeliveryCh())
assert.Equal(t, id, task.DeliveryID)
return
}
assertNothingQueued(t, s.Engine)
iAssertStatus(t, s.WebhookDB, id, database.DeliveryStatusPending)
assert.Zero(t, s.Engine.ExportInflightHeld())
})
}
}
// TestRecovery_AlreadySucceeded: a pending or retrying delivery whose
// success is recorded but whose status write was lost is marked
// delivered where it stands, and not sent again.
func TestRecovery_AlreadySucceeded(t *testing.T) {
t.Parallel()
for _, run := range recoveryRuns() {
t.Run(run.name, func(t *testing.T) {
t.Parallel()
s, targetID := recoverySetup(t)
pending := seedRecoveryDelivery(
t, s, targetID, database.DeliveryStatusPending, 1, true,
)
rAgePending(t, s.WebhookDB, pending)
retrying := seedRecoveryDelivery(
t, s, targetID, database.DeliveryStatusRetrying, 2, true,
)
backdateAttempts(t, s, retrying)
s.Engine.ExportRecoverDeliveries(context.Background(), run.atStart)
assertNothingQueued(t, s.Engine)
for id, attempts := range map[string]int{pending: 1, retrying: 2} {
iAssertStatus(
t, s.WebhookDB, id, database.DeliveryStatusDelivered,
)
assert.Len(t, iResults(t, s.WebhookDB, id), attempts,
"marking the delivery delivered recorded an attempt",
)
}
})
}
}
// TestRecovery_MissingDatabase: a tick leaves a webhook whose database
// file is missing alone, and the run at start opens it, which creates an
// empty one and reports the loss (see
// TestRecoverDeliveries_ReportsAMissingWebhookDatabase).
func TestRecovery_MissingDatabase(t *testing.T) {
t.Parallel()
mainDB := iMainDB(t)
webhookID := uuid.New().String()
iCreateWebhook(t, mainDB.DB(), webhookID, "missing-database")
dbMgr := databasetest.NewWebhookDBManager(t, t.TempDir())
t.Cleanup(func() { _ = dbMgr.CloseAll() })
engine := delivery.NewTestEngineWithDB(
mainDB, dbMgr,
slog.New(slog.DiscardHandler),
&http.Client{Timeout: 5 * time.Second}, 1,
)
engine.ExportRecoverDeliveries(context.Background(), onTick)
assert.False(t, dbMgr.DBExists(webhookID),
"a tick created the missing database",
)
engine.ExportRecoverDeliveries(context.Background(), atStart)
assert.True(t, dbMgr.DBExists(webhookID),
"the run at start did not open the missing database",
)
}
// TestRecovery_LeavesADeliveryTheEngineHolds: a delivery the engine is
// working on, has queued, or holds a retry timer for is left alone by
// either run, pending and retrying alike, however old.
func TestRecovery_LeavesADeliveryTheEngineHolds(t *testing.T) {
t.Parallel()
for _, run := range recoveryRuns() {
t.Run(run.name, func(t *testing.T) {
t.Parallel()
s, targetID := recoverySetup(t)
pending := seedRecoveryDelivery(
t, s, targetID, database.DeliveryStatusPending, 0, false,
)
rAgePending(t, s.WebhookDB, pending)
retrying := seedRecoveryDelivery(
t, s, targetID, database.DeliveryStatusRetrying, 1, false,
)
backdateAttempts(t, s, retrying)
require.True(t, s.Engine.ExportRetainDelivery(pending))
require.True(t, s.Engine.ExportRetainDelivery(retrying))
s.Engine.ExportRecoverDeliveries(context.Background(), run.atStart)
assertNothingQueued(t, s.Engine)
iAssertStatus(t, s.WebhookDB, pending, database.DeliveryStatusPending)
iAssertStatus(t, s.WebhookDB, retrying, database.DeliveryStatusRetrying)
assert.Equal(t, 2, s.Engine.ExportInflightHeld())
})
}
}
internal/delivery/redirect_test.go
+17 -9
View File
@@ -85,14 +85,19 @@ func deliverWithProbeHeaders(
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
cfg, err := json.Marshal(delivery.HTTPTargetConfig{
URL: targetURL,
Headers: map[string]string{
probeHeaderName: probeHeaderValue,
},
})
require.NoError(t, err)
body := event.Body body := event.Body
task := iTask( task := iTask(
d, event, s.WebhookID, targetID, d, event, s.WebhookID, targetID,
"redirect-target", targetURL, 1, 1, &body, "redirect-target", string(cfg), 1, 1, &body,
) )
task.TargetHeaders = map[string]string{
probeHeaderName: probeHeaderValue,
}
s.Engine.ExportProcessNewTask(context.TODO(), &task) s.Engine.ExportProcessNewTask(context.TODO(), &task)
@@ -304,7 +309,8 @@ func TestRedirectPolicy_StopsAtHopCap(t *testing.T) {
) )
client := engine.ExportClientForRequest( client := engine.ExportClientForRequest(
0, []string{probeHeaderName}, &delivery.HTTPTargetConfig{URL: srv.URL},
[]string{probeHeaderName},
) )
require.NotNil(t, client.CheckRedirect) require.NotNil(t, client.CheckRedirect)
@@ -344,7 +350,7 @@ func TestApplyRequestHeaders_ReportsOriginScopedNames(t *testing.T) {
inbound, err := json.Marshal(map[string][]string{ inbound, err := json.Marshal(map[string][]string{
inboundHeaderName: {inboundHeaderValue}, inboundHeaderName: {inboundHeaderValue},
"Content-Type": {testContentType}, "Content-Type": {testContentType},
headerUserAgent: {"curl/8.7.1"}, "User-Agent": {"curl/8.7.1"},
"Host": {"inbound.example.com"}, "Host": {"inbound.example.com"},
}) })
require.NoError(t, err) require.NoError(t, err)
@@ -363,9 +369,11 @@ func TestApplyRequestHeaders_ReportsOriginScopedNames(t *testing.T) {
Headers: string(inbound), Headers: string(inbound),
ContentType: testContentType, ContentType: testContentType,
}, },
map[string]string{ &delivery.HTTPTargetConfig{
probeHeaderName: probeHeaderValue, Headers: map[string]string{
"Content-Type": testContentType, probeHeaderName: probeHeaderValue,
"Content-Type": testContentType,
},
}, },
"webhooker/dev", "webhooker/dev",
) )
internal/delivery/store.go
-164
View File
@@ -1,164 +0,0 @@
package delivery
import (
"fmt"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
)
// StoreEvent stores event with one pending delivery for each of its
// webhook's active targets, and adds them to the totals, in one
// transaction, then queues the deliveries. It returns how many
// deliveries it created.
//
// The receiver stores every event it accepts through here, and a
// resubmit stores its copy of a stored event, so every event is
// stored, retried, SSRF-guarded and circuit-broken the same way however
// it entered the system. CreateReplay is the only other place a
// delivery is created.
//
// The targets are the webhook's active ones as they stand now: a
// target created after a resubmitted event first arrived is included,
// and an inactive one is skipped. An event with no active target is
// stored with no deliveries.
func (e *Engine) StoreEvent(event *database.Event) (int, error) {
var targets []database.Target
err := e.database.DB().Where(
"webhook_id = ? AND active = ?", event.WebhookID, true,
).Find(&targets).Error
if err != nil {
return 0, fmt.Errorf("loading active targets: %w", err)
}
webhookDB, err := e.dbManager.GetDB(event.WebhookID)
if err != nil {
return 0, fmt.Errorf("getting webhook database: %w", err)
}
event.BodyBytes = int64(len(event.Body))
tasks := make([]Task, 0, len(targets))
err = webhookDB.Transaction(func(tx *gorm.DB) error {
err := tx.Create(event).Error
if err != nil {
return fmt.Errorf("creating event: %w", err)
}
for i := range targets {
dlv, err := createDelivery(tx, event.ID, targets[i].ID, false)
if err != nil {
return err
}
tasks = append(tasks, buildTask(
dlv, event.WebhookID, event, &targets[i], 1,
))
}
err = database.AddEventTotals(tx, database.EventTotals{
Events: 1, LastEventAt: &event.CreatedAt,
})
if err != nil {
return err
}
// A resubmitted copy did not arrive on its entrypoint's URL, so
// it leaves the entrypoint's last event as it is.
if event.ResubmittedFromID != nil {
return nil
}
return database.AddEntrypointTotals(tx, database.EntrypointTotals{
EntrypointID: event.EntrypointID, LastEventAt: event.CreatedAt,
})
})
if err != nil {
return 0, err
}
// Counted here, after the commit: an event exists once it is
// durably stored, which is what the delivery counters are compared
// against on a dashboard. A resubmitted event counts too, because
// it produces deliveries that the delivery side counts; the
// resubmit counter is what separates the two.
e.mtr.EventReceived()
e.notify(tasks)
return len(tasks), nil
}
// CreateReplay creates one more pending delivery of the stored event
// eventID to target, marked as a replay, adds it to the target's totals
// in the same transaction, and queues it. It returns the new
// delivery's id.
//
// The caller passes the target as it stands now and has already
// decided the replay may go ahead. The delivery being replayed is not
// touched.
func (e *Engine) CreateReplay(
webhookID, eventID string, target *database.Target,
) (string, error) {
webhookDB, err := e.dbManager.GetDB(webhookID)
if err != nil {
return "", fmt.Errorf("getting webhook database: %w", err)
}
event, err := e.loadEvent(webhookDB, eventID)
if err != nil {
return "", err
}
var dlv *database.Delivery
err = webhookDB.Transaction(func(tx *gorm.DB) error {
var err error
dlv, err = createDelivery(tx, event.ID, target.ID, true)
return err
})
if err != nil {
return "", err
}
e.notify([]Task{buildTask(dlv, webhookID, &event, target, 1)})
return dlv.ID, nil
}
// createDelivery writes a pending delivery of the event to the target
// on the caller's transaction, and adds it to the target's totals.
//
// Neither Event nor Target is set on the row, and the per-webhook
// database omits associations on every write in any case (see
// internal/database/event_db_isolation.go), so the target's settings
// are never copied into the event database.
func createDelivery(
tx *gorm.DB, eventID, targetID string, replay bool,
) (*database.Delivery, error) {
dlv := &database.Delivery{
EventID: eventID,
TargetID: targetID,
Status: database.DeliveryStatusPending,
Replay: replay,
}
err := tx.Create(dlv).Error
if err != nil {
return nil, fmt.Errorf(
"creating delivery for target %s: %w", targetID, err,
)
}
err = database.AddTargetTotals(tx, database.TargetTotals{
TargetID: targetID, Deliveries: 1,
})
if err != nil {
return nil, err
}
return dlv, nil
}
internal/delivery/store_test.go
-386
View File
@@ -1,386 +0,0 @@
package delivery_test
import (
"net/http"
"strings"
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
)
// storeTestTarget creates an active http target of the setup's webhook
// with every one of a target's settings set, so a task that drops one
// of them differs from the target.
func storeTestTarget(
t *testing.T, s iSetup, name string,
) database.Target {
t.Helper()
target := database.Target{
WebhookID: s.WebhookID,
Name: name,
Type: database.TargetTypeHTTP,
Active: true,
URL: "http://93.184.216.34/" + name,
Headers: map[string]string{"X-Api-Key": "k"},
TimeoutSeconds: 7,
ForwardQuery: true,
ArchiveExpiry: time.Hour,
ArchiveRotation: rotationDaily,
MaxAttempts: 3,
}
require.NoError(t, s.MainDB.Create(&target).Error)
return target
}
// newStoreTestEvent returns an event of the setup's webhook as the
// receiver builds one, not yet stored.
func newStoreTestEvent(s iSetup, body string) *database.Event {
return &database.Event{
WebhookID: s.WebhookID,
EntrypointID: uuid.New().String(),
Method: http.MethodPost,
RawQuery: "a=1&b=2",
Headers: `{"X-Test":["yes"]}`,
ContentType: testContentType,
Body: body,
}
}
// queuedTasks takes every task the engine has queued so far. The
// engine in these tests is never started, so nothing else takes them.
func queuedTasks(e *delivery.Engine) []delivery.Task {
var tasks []delivery.Task
for {
select {
case task := <-e.ExportDeliveryCh():
tasks = append(tasks, task)
default:
return tasks
}
}
}
// assertTaskFor proves task carries delivery deliveryID of event to
// target: every one of the target's settings and every event field.
// They are copied into the task one field at a time, and a field left
// out is sent empty without any error.
func assertTaskFor(
t *testing.T,
task delivery.Task,
deliveryID string,
event *database.Event,
target *database.Target,
) {
t.Helper()
assert.Equal(t, deliveryID, task.DeliveryID)
assert.Equal(t, event.ID, task.EventID)
assert.Equal(t, event.WebhookID, task.WebhookID)
assert.Equal(t, event.EntrypointID, task.EntrypointID)
assert.Equal(t, target.ID, task.TargetID)
assert.Equal(t, target.Name, task.TargetName)
assert.Equal(t, target.Type, task.TargetType)
assert.Equal(t, target.URL, task.TargetURL)
assert.Equal(t, target.Headers, task.TargetHeaders)
assert.Equal(t, target.TimeoutSeconds, task.TimeoutSeconds)
assert.Equal(t, target.ForwardQuery, task.ForwardQuery)
assert.Equal(t, target.ArchiveExpiry, task.ArchiveExpiry)
assert.Equal(t, target.ArchiveRotation, task.ArchiveRotation)
assert.Equal(t, target.MaxAttempts, task.MaxAttempts)
assert.Equal(t, event.Method, task.Method)
assert.Equal(t, event.RawQuery, task.RawQuery)
assert.Equal(t, event.Headers, task.Headers)
assert.Equal(t, event.ContentType, task.ContentType)
require.NotNil(t, task.Body)
assert.Equal(t, event.Body, *task.Body)
assert.Equal(t, 1, task.AttemptNum)
}
// storedDeliveries reads every delivery of an event.
func storedDeliveries(
t *testing.T, db *gorm.DB, eventID string,
) []database.Delivery {
t.Helper()
var deliveries []database.Delivery
require.NoError(t, db.Where("event_id = ?", eventID).
Find(&deliveries).Error)
return deliveries
}
// storedEventTotals reads the webhook's event totals.
func storedEventTotals(t *testing.T, db *gorm.DB) database.EventTotals {
t.Helper()
var totals database.EventTotals
require.NoError(t, db.First(&totals).Error)
return totals
}
// countRows counts the rows of model in db.
func countRows(t *testing.T, db *gorm.DB, model any) int64 {
t.Helper()
var n int64
require.NoError(t, db.Model(model).Count(&n).Error)
return n
}
// TestStoreEvent_StoresDeliveriesToActiveTargets proves StoreEvent
// stores the event with one pending delivery to each active target and
// none to an inactive one, adds them to the totals, and queues one task
// per delivery.
func TestStoreEvent_StoresDeliveriesToActiveTargets(t *testing.T) {
t.Parallel()
s := newISetup(t)
targets := []database.Target{
storeTestTarget(t, s, "first"),
storeTestTarget(t, s, "second"),
}
off := storeTestTarget(t, s, "off")
require.NoError(t, s.MainDB.Model(&database.Target{}).
Where("id = ?", off.ID).Update("active", false).Error)
event := newStoreTestEvent(s, `{"hello":"world"}`)
created, err := s.Engine.StoreEvent(event)
require.NoError(t, err)
assert.Equal(t, 2, created)
var stored database.Event
require.NoError(t, s.WebhookDB.First(&stored, "id = ?", event.ID).Error)
assert.Equal(t, event.Body, stored.Body)
assert.Equal(t, int64(len(event.Body)), stored.BodyBytes)
rows := storedDeliveries(t, s.WebhookDB, event.ID)
queued := queuedTasks(s.Engine)
require.Len(t, rows, 2)
require.Len(t, queued, 2)
deliveries := map[string]database.Delivery{}
for _, dlv := range rows {
deliveries[dlv.TargetID] = dlv
}
tasks := map[string]delivery.Task{}
for _, task := range queued {
tasks[task.TargetID] = task
}
for i := range targets {
dlv, ok := deliveries[targets[i].ID]
require.True(t, ok, "no delivery to %s", targets[i].Name)
assert.Equal(t, database.DeliveryStatusPending, dlv.Status)
assert.False(t, dlv.Replay)
assert.Equal(t, int64(1),
targetTotals(t, s.WebhookDB, targets[i].ID).Deliveries)
assertTaskFor(t, tasks[targets[i].ID], dlv.ID, event, &targets[i])
}
totals := storedEventTotals(t, s.WebhookDB)
assert.Equal(t, int64(1), totals.Events)
require.NotNil(t, totals.LastEventAt)
assert.True(t, totals.LastEventAt.Equal(stored.CreatedAt))
var entrypoint database.EntrypointTotals
require.NoError(t, s.WebhookDB.First(
&entrypoint, "entrypoint_id = ?", event.EntrypointID,
).Error)
assert.True(t, entrypoint.LastEventAt.Equal(stored.CreatedAt))
}
// TestStoreEvent_ResubmittedCopyLeavesTheEntrypoint proves a
// resubmitted copy is counted among the webhook's events but leaves its
// entrypoint's last event as it is: it did not arrive on that URL.
func TestStoreEvent_ResubmittedCopyLeavesTheEntrypoint(t *testing.T) {
t.Parallel()
s := newISetup(t)
storeTestTarget(t, s, "only")
sourceID := uuid.New().String()
event := newStoreTestEvent(s, `{"copy":true}`)
event.ResubmittedFromID = &sourceID
created, err := s.Engine.StoreEvent(event)
require.NoError(t, err)
assert.Equal(t, 1, created)
assert.Len(t, queuedTasks(s.Engine), 1)
assert.Equal(t, int64(1), storedEventTotals(t, s.WebhookDB).Events)
assert.Zero(t, countRows(t, s.WebhookDB, &database.EntrypointTotals{}))
}
// TestStoreEvent_NoActiveTarget proves an event of a webhook with no
// active target is stored with no delivery and nothing queued.
func TestStoreEvent_NoActiveTarget(t *testing.T) {
t.Parallel()
s := newISetup(t)
event := newStoreTestEvent(s, `{"to":"nobody"}`)
created, err := s.Engine.StoreEvent(event)
require.NoError(t, err)
assert.Zero(t, created)
assert.Equal(t, int64(1), countRows(t, s.WebhookDB, &database.Event{}))
assert.Empty(t, storedDeliveries(t, s.WebhookDB, event.ID))
assert.Empty(t, queuedTasks(s.Engine))
}
// TestStoreEvent_LargeBodyIsNotCarried proves a body at the inline
// limit is stored whole but left off the task, for the attempt to read
// from the event row.
func TestStoreEvent_LargeBodyIsNotCarried(t *testing.T) {
t.Parallel()
s := newISetup(t)
storeTestTarget(t, s, "only")
event := newStoreTestEvent(
s, strings.Repeat("x", delivery.MaxInlineBodySize),
)
_, err := s.Engine.StoreEvent(event)
require.NoError(t, err)
tasks := queuedTasks(s.Engine)
require.Len(t, tasks, 1)
assert.Nil(t, tasks[0].Body)
var stored database.Event
require.NoError(t, s.WebhookDB.First(&stored, "id = ?", event.ID).Error)
assert.Equal(t, event.Body, stored.Body)
}
// TestStoreEvent_RollsBackWhenAWriteFails proves the event, its
// deliveries and its totals are one transaction: when the entrypoint
// totals write fails, the last one, after the event, its delivery, the
// target totals and the event totals were written, none of them is left
// and nothing is queued.
func TestStoreEvent_RollsBackWhenAWriteFails(t *testing.T) {
t.Parallel()
s := newISetup(t)
storeTestTarget(t, s, "only")
require.NoError(t,
s.WebhookDB.Migrator().DropTable(&database.EntrypointTotals{}))
_, err := s.Engine.StoreEvent(newStoreTestEvent(s, `{"lost":true}`))
require.Error(t, err)
assert.Zero(t, countRows(t, s.WebhookDB, &database.Event{}))
assert.Zero(t, countRows(t, s.WebhookDB, &database.Delivery{}))
assert.Zero(t, countRows(t, s.WebhookDB, &database.TargetTotals{}))
totals := storedEventTotals(t, s.WebhookDB)
assert.Zero(t, totals.Events)
assert.Nil(t, totals.LastEventAt)
assert.Empty(t, queuedTasks(s.Engine))
assert.Zero(t, s.Engine.ExportInflightHeld())
}
// TestCreateReplay_CreatesAndQueuesAReplay proves CreateReplay adds a
// new pending delivery marked as a replay, counts it in the target's
// totals, queues its task built from the target it was given and the
// stored event, and leaves the delivery it repeats as it was. The
// stored event has every event field set, so a task that drops one of
// them differs from the event.
func TestCreateReplay_CreatesAndQueuesAReplay(t *testing.T) {
t.Parallel()
s := newISetup(t)
target := storeTestTarget(t, s, "replayed")
event := newStoreTestEvent(s, `{"replay":"me"}`)
require.NoError(t, s.WebhookDB.Create(event).Error)
original := iSeedDelivery(
t, s.WebhookDB, event.ID, target.ID,
database.DeliveryStatusFailed,
)
deliveryID, err := s.Engine.CreateReplay(s.WebhookID, event.ID, &target)
require.NoError(t, err)
replay := storedDelivery(t, s.WebhookDB, deliveryID)
assert.Equal(t, event.ID, replay.EventID)
assert.Equal(t, target.ID, replay.TargetID)
assert.Equal(t, database.DeliveryStatusPending, replay.Status)
assert.True(t, replay.Replay)
assert.Equal(t, database.DeliveryStatusFailed,
storedDelivery(t, s.WebhookDB, original.ID).Status)
assert.Equal(t, int64(1),
targetTotals(t, s.WebhookDB, target.ID).Deliveries)
tasks := queuedTasks(s.Engine)
require.Len(t, tasks, 1)
assertTaskFor(t, tasks[0], deliveryID, event, &target)
}
// TestCreateReplay_RollsBackWhenAWriteFails proves the replay delivery
// and its target totals are one transaction: when the totals write
// fails, after the delivery was written, no new delivery is left and
// nothing is queued.
func TestCreateReplay_RollsBackWhenAWriteFails(t *testing.T) {
t.Parallel()
s := newISetup(t)
target := storeTestTarget(t, s, "replayed")
event := iSeedEvent(t, s.WebhookDB, s.WebhookID, `{"replay":"me"}`)
iSeedDelivery(
t, s.WebhookDB, event.ID, target.ID,
database.DeliveryStatusFailed,
)
require.NoError(t,
s.WebhookDB.Migrator().DropTable(&database.TargetTotals{}))
_, err := s.Engine.CreateReplay(s.WebhookID, event.ID, &target)
require.Error(t, err)
assert.Len(t, storedDeliveries(t, s.WebhookDB, event.ID), 1)
assert.Empty(t, queuedTasks(s.Engine))
assert.Zero(t, s.Engine.ExportInflightHeld())
}
// storedDelivery reads one delivery back.
func storedDelivery(
t *testing.T, db *gorm.DB, deliveryID string,
) database.Delivery {
t.Helper()
var dlv database.Delivery
require.NoError(t, db.First(&dlv, "id = ?", deliveryID).Error)
return dlv
}
internal/delivery/target.go
+18 -9
View File
@@ -13,8 +13,8 @@ import (
// The engine provides one to each target so a target can own // The engine provides one to each target so a target can own
// its retries durably: it records the attempt, marks the // its retries durably: it records the attempt, marks the
// delivery retrying, and asks the Scheduler to deliver the // delivery retrying, and asks the Scheduler to deliver the
// next attempt after delay — exactly what the engine's // next attempt after delay — exactly what the engine does for
// recovery routine does for a retrying delivery. // its own restart recovery.
type Scheduler interface { type Scheduler interface {
ScheduleRetry(task Task, delay time.Duration) ScheduleRetry(task Task, delay time.Duration)
} }
@@ -45,20 +45,29 @@ type Target interface {
} }
// rescheduler is implemented by targets that own durable // rescheduler is implemented by targets that own durable
// retries. The engine's recovery routine uses it to let the // retries. The engine's restart recovery and periodic sweep
// target recompute the schedule for an orphaned retrying // use it to let the target recompute the schedule for an
// delivery, keeping the retry schedule target-owned. // orphaned retrying delivery, keeping the retry schedule
// Fire-and-forget targets do not implement it, and a retrying // target-owned. Fire-and-forget targets do not implement it
// delivery found for one is terminally failed. // and their (never-occurring) retrying deliveries are
// skipped.
type rescheduler interface { type rescheduler interface {
// remainingBackoff returns how long to wait before the // remainingBackoff returns how long to wait before the
// next attempt of a recovered retrying delivery: zero once // next attempt of a recovered retrying delivery.
// its backoff has passed.
remainingBackoff( remainingBackoff(
webhookDB *gorm.DB, webhookDB *gorm.DB,
deliveryID string, deliveryID string,
attemptNum int, attemptNum int,
) time.Duration ) time.Duration
// backoffElapsed reports whether the backoff window for
// the last attempt has already passed, so the periodic
// sweep can re-enqueue the delivery now.
backoffElapsed(
webhookDB *gorm.DB,
deliveryID string,
attemptNum int,
) bool
} }
// attemptResult is the outcome of a single delivery attempt, // attemptResult is the outcome of a single delivery attempt,
internal/delivery/target_config_edit.go
+137
View File
@@ -0,0 +1,137 @@
package delivery
import (
"encoding/json"
"errors"
"fmt"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database"
)
// errUnknownTargetTypeForEdit is returned when a stored target has a
// type the edit form has no field set for.
var errUnknownTargetTypeForEdit = errors.New(
"unknown target type",
)
// TargetConfigForm is the UNMASKED projection of a target's stored
// configuration, for pre-filling the target edit form.
//
// It is the deliberate exception to the rule TargetView enforces
// everywhere else: TargetView exists so that no template can render
// a target's stored blob, because a destination URL's path segments
// and a header value are both routinely the credential. An operator
// cannot correct a value they cannot see, so the edit form — and
// only the edit form — is shown the full value.
//
// Everything that keeps that exception narrow lives at the call
// site: the route is behind RequireAuth and the webhook's ownership
// check, and its group sets NoCache so the rendered secret is not
// written to a shared cache. Do not reach for this type from any
// other page.
type TargetConfigForm struct {
// URL is the destination for an HTTP target and the webhook
// URL for a Slack target.
URL string
// Headers is the HTTP target's configured headers in the
// textarea representation, one "Name: value" per line.
Headers string
// Timeout is the HTTP target's per-request timeout in seconds,
// empty when unset.
Timeout string
// ForwardQuery is the HTTP target's setting that passes each
// event's query string on to it.
ForwardQuery bool
// Expiry is the database (archive) target's row expiry.
Expiry string
// Rotation is the database (archive) target's rotation.
Rotation string
}
// NewTargetConfigForm parses a target's stored configuration into
// the edit form's fields.
//
// A configuration that does not parse is an error rather than a
// zero-valued form that silently looks like a target with no
// settings. The caller shows the operator that the stored value
// could not be read, so that saving the form is understood as
// replacing it rather than preserving it.
func NewTargetConfigForm(
t *database.Target,
) (TargetConfigForm, error) {
switch t.Type {
case database.TargetTypeHTTP:
cfg, err := parseHTTPConfig(t.Config)
if err != nil {
return TargetConfigForm{}, err
}
return TargetConfigForm{
URL: cfg.URL,
Headers: FormatTargetHeaders(cfg.Headers),
Timeout: FormatTargetTimeout(cfg.Timeout),
ForwardQuery: cfg.ForwardQuery,
}, nil
case database.TargetTypeSlack:
cfg, err := parseSlackConfig(t.Config)
if err != nil {
return TargetConfigForm{}, err
}
return TargetConfigForm{URL: cfg.WebhookURL}, nil
case database.TargetTypeDatabase:
return databaseConfigForm(t.Config)
case database.TargetTypeLog:
// The log target takes no configuration.
return TargetConfigForm{}, nil
default:
return TargetConfigForm{}, fmt.Errorf(
"%w: %q", errUnknownTargetTypeForEdit, t.Type,
)
}
}
// databaseConfigForm parses an archive target's optional expiry and
// rotation. An absent, empty or never expiry yields an empty expiry,
// on which the edit form starts at never; saving it unchanged stores
// never, which means the same as an empty expiry. An absent rotation
// is empty too, and the form starts at none. An expiry that is set
// but not a valid duration, or a rotation that is not one of the
// four, is an error, not a blank field.
func databaseConfigForm(
configJSON string,
) (TargetConfigForm, error) {
if configJSON == "" {
return TargetConfigForm{}, nil
}
var cfg archive.DatabaseTargetConfig
err := json.Unmarshal([]byte(configJSON), &cfg)
if err != nil {
return TargetConfigForm{}, fmt.Errorf(
"parsing config JSON: %w", err,
)
}
err = archive.ValidateRotation(cfg.Rotation)
if err != nil {
return TargetConfigForm{}, err
}
form := TargetConfigForm{Rotation: cfg.Rotation}
if cfg.Expiry == "" || cfg.Expiry == archive.ExpiryNever {
return form, nil
}
err = archive.ValidateExpiry(cfg.Expiry)
if err != nil {
return TargetConfigForm{}, err
}
form.Expiry = cfg.Expiry
return form, nil
}
internal/delivery/target_config_view.go
+74 -33
View File
@@ -9,17 +9,16 @@ import (
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
) )
// configUnavailable is what a target's settings render as when // configUnavailable is what a target's configuration renders
// the target is of an unknown type, and what its URL renders as // as when it is absent, of an unknown type, or does not
// when it does not parse into a scheme and host. The stored URL is // parse. The stored blob is never shown as a fallback: it can
// never shown as a fallback: it can be a credential (a Slack // hold a credential (a Slack incoming webhook URL is a bearer
// incoming webhook URL is a bearer token) and a UI that prints it // token) and a UI that prints it leaks that credential into
// leaks that credential into browser history, screenshots and // browser history, screenshots and screen shares.
// screen shares.
const configUnavailable = "(unavailable)" const configUnavailable = "(unavailable)"
// ConfigField is one labelled, display-safe value derived // ConfigField is one labelled, display-safe value derived
// from a target's settings. // from a target's stored configuration.
type ConfigField struct { type ConfigField struct {
Label string Label string
Value string Value string
@@ -33,9 +32,9 @@ type ConfigField struct {
const deletedNameSuffix = " (deleted)" const deletedNameSuffix = " (deleted)"
// TargetView is the display-safe projection of a target for // TargetView is the display-safe projection of a target for
// the UI. It deliberately has no URL or header field, so no // the UI. It deliberately has no raw configuration field, so
// template — present or future — can render the stored // no template — present or future — can render the stored
// credential. // blob.
type TargetView struct { type TargetView struct {
ID string ID string
Name string Name string
@@ -62,7 +61,8 @@ func (v TargetView) DisplayName() string {
} }
// NewTargetViews projects targets for rendering, replacing // NewTargetViews projects targets for rendering, replacing
// each target's settings with named, display-safe fields. // each stored configuration blob with named, display-safe
// fields.
// //
// A soft-deleted row projects exactly as a live one does, minus // A soft-deleted row projects exactly as a live one does, minus
// the deleted marker on its name: masking is a property of the // the deleted marker on its name: masking is a property of the
@@ -91,8 +91,8 @@ func NewTargetViews(
} }
// targetConfigFields returns the display-safe fields for a // targetConfigFields returns the display-safe fields for a
// target's settings. A target of an unknown type gets the // target's configuration. Anything it cannot parse becomes
// neutral placeholder. // the neutral placeholder.
func targetConfigFields( func targetConfigFields(
t *database.Target, t *database.Target,
) []ConfigField { ) []ConfigField {
@@ -102,25 +102,36 @@ func targetConfigFields(
case database.TargetTypeHTTP: case database.TargetTypeHTTP:
return httpConfigFields(t) return httpConfigFields(t)
case database.TargetTypeDatabase: case database.TargetTypeDatabase:
return databaseConfigFields(t) return databaseConfigFields(t.Config)
case database.TargetTypeLog: case database.TargetTypeLog:
// The log target takes no settings. // The log target takes no configuration.
return nil return nil
default: default:
return []ConfigField{{ return unavailableConfigFields()
Label: "Configuration",
Value: configUnavailable,
}}
} }
} }
// unavailableConfigFields is the neutral placeholder shown
// for a configuration that could not be presented.
func unavailableConfigFields() []ConfigField {
return []ConfigField{{
Label: "Configuration",
Value: configUnavailable,
}}
}
// slackConfigFields describes a Slack target: its masked // slackConfigFields describes a Slack target: its masked
// webhook URL and its number of delivery attempts. Only the // webhook URL and its number of delivery attempts. Only the
// masked URL is shown; the full URL is the credential. // masked URL is shown; the full URL is the credential.
func slackConfigFields(t *database.Target) []ConfigField { func slackConfigFields(t *database.Target) []ConfigField {
cfg, err := parseSlackConfig(t.Config)
if err != nil {
return unavailableConfigFields()
}
return []ConfigField{{ return []ConfigField{{
Label: "Webhook URL", Label: "Webhook URL",
Value: MaskURL(t.URL), Value: cfg.MaskedWebhookURL(),
}, maxAttemptsField(t)} }, maxAttemptsField(t)}
} }
@@ -135,28 +146,33 @@ func slackConfigFields(t *database.Target) []ConfigField {
// whose path segments are the credential, and the field takes // whose path segments are the credential, and the field takes
// an arbitrary URL, so no segment can be assumed non-secret. // an arbitrary URL, so no segment can be assumed non-secret.
func httpConfigFields(t *database.Target) []ConfigField { func httpConfigFields(t *database.Target) []ConfigField {
cfg, err := parseHTTPConfig(t.Config)
if err != nil {
return unavailableConfigFields()
}
fields := []ConfigField{{ fields := []ConfigField{{
Label: "Destination URL", Label: "Destination URL",
Value: MaskURL(t.URL), Value: MaskURL(cfg.URL),
}} }}
if t.TimeoutSeconds > 0 { if cfg.Timeout > 0 {
fields = append(fields, ConfigField{ fields = append(fields, ConfigField{
Label: "Timeout", Label: "Timeout",
Value: strconv.Itoa(t.TimeoutSeconds) + "s", Value: strconv.Itoa(cfg.Timeout) + "s",
}) })
} }
if len(t.Headers) > 0 { if len(cfg.Headers) > 0 {
fields = append(fields, ConfigField{ fields = append(fields, ConfigField{
Label: "Headers", Label: "Headers",
Value: fmt.Sprintf( Value: fmt.Sprintf(
"%d configured", len(t.Headers), "%d configured", len(cfg.Headers),
), ),
}) })
} }
if t.ForwardQuery { if cfg.ForwardQuery {
fields = append(fields, ConfigField{ fields = append(fields, ConfigField{
Label: "Query string", Label: "Query string",
Value: "passed on to this target", Value: "passed on to this target",
@@ -169,7 +185,8 @@ func httpConfigFields(t *database.Target) []ConfigField {
} }
// maxAttemptsField describes a target's number of delivery // maxAttemptsField describes a target's number of delivery
// attempts. // attempts, which lives on the target row rather than in its
// configuration blob.
func maxAttemptsField(t *database.Target) ConfigField { func maxAttemptsField(t *database.Target) ConfigField {
return ConfigField{ return ConfigField{
Label: "Delivery attempts", Label: "Delivery attempts",
@@ -179,11 +196,24 @@ func maxAttemptsField(t *database.Target) ConfigField {
// databaseConfigFields describes an archive target by its // databaseConfigFields describes an archive target by its
// expiry in plain units, such as "30 days", or "never" when // expiry in plain units, such as "30 days", or "never" when
// the archive is kept forever, and by its rotation. // the archive is kept forever, and by its rotation. An expiry
func databaseConfigFields(t *database.Target) []ConfigField { // that is set but not a valid duration, or a rotation that is
// not one of the four, is reported as unavailable rather than
// echoed back.
func databaseConfigFields(configJSON string) []ConfigField {
expiry, err := archive.ParseExpiry(configJSON)
if err != nil {
return unavailableConfigFields()
}
rotation, err := archive.ParseRotation(configJSON)
if err != nil {
return unavailableConfigFields()
}
value := archive.ExpiryNever value := archive.ExpiryNever
if t.ArchiveExpiry > 0 { if expiry > 0 {
value = plainDuration(t.ArchiveExpiry) value = plainDuration(expiry)
} }
return []ConfigField{{ return []ConfigField{{
@@ -191,7 +221,7 @@ func databaseConfigFields(t *database.Target) []ConfigField {
Value: value, Value: value,
}, { }, {
Label: "Archive rotation", Label: "Archive rotation",
Value: t.ArchiveRotation, Value: rotation,
}} }}
} }
@@ -227,3 +257,14 @@ func plainDuration(d time.Duration) string {
return d.String() return d.String()
} }
// MaskedWebhookURL returns the Slack webhook URL reduced to
// its scheme and host, with the path, query and any userinfo
// elided. The path segments are the credential, so none of
// them is shown: the field accepts an arbitrary URL, so no
// segment can be assumed non-secret. A URL that does not
// parse into a scheme and host yields the neutral
// placeholder, never the raw string.
func (c *SlackTargetConfig) MaskedWebhookURL() string {
return MaskURL(c.WebhookURL)
}
internal/delivery/target_config_view_test.go
+122 -58
View File
@@ -35,17 +35,18 @@ const (
viewMaxAttempts = "Delivery attempts" viewMaxAttempts = "Delivery attempts"
) )
// The archive rotations a database target's settings can name. // The archive rotations a database target's settings can name, and
// the config of one whose rotation is not one of the four.
const ( const (
rotationNone = "none" rotationNone = "none"
rotationMonthly = "monthly" rotationMonthly = "monthly"
rotationDaily = "daily" rotationDaily = "daily"
rotationHourly = "hourly" rotationHourly = "hourly"
weeklyConfig = `{"rotation":"weekly"}`
) )
// TestNewTargetViews_SlackMasksWebhookURL shows what a Slack target's func TestMaskedWebhookURL(t *testing.T) {
// URL renders as on the target list.
func TestNewTargetViews_SlackMasksWebhookURL(t *testing.T) {
t.Parallel() t.Parallel()
tests := map[string]struct { tests := map[string]struct {
@@ -89,24 +90,28 @@ func TestNewTargetViews_SlackMasksWebhookURL(t *testing.T) {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
t.Parallel() t.Parallel()
target := slackTarget() cfg := &delivery.SlackTargetConfig{
target.URL = tc.url WebhookURL: tc.url,
}
assert.Equal( assert.Equal(
t, tc.want, t, tc.want, cfg.MaskedWebhookURL(),
fieldMap(viewFor(t, target).Config)["Webhook URL"],
) )
}) })
} }
} }
// TestNewTargetViews_SlackNeverLeaksPath is the direct // TestMaskedWebhookURL_NeverLeaksPath is the direct
// expression of the rule: whatever the input, the masked // expression of the rule: whatever the input, the masked
// value never contains a path segment of it. // value never contains a path segment of it.
func TestNewTargetViews_SlackNeverLeaksPath(t *testing.T) { func TestMaskedWebhookURL_NeverLeaksPath(t *testing.T) {
t.Parallel() t.Parallel()
masked := fieldMap(viewFor(t, slackTarget()).Config)["Webhook URL"] cfg := &delivery.SlackTargetConfig{
WebhookURL: slackWebhookURL,
}
masked := cfg.MaskedWebhookURL()
assert.NotContains(t, masked, "T00000000") assert.NotContains(t, masked, "T00000000")
assert.NotContains(t, masked, "B00000000") assert.NotContains(t, masked, "B00000000")
@@ -172,10 +177,11 @@ func TestNewTargetViews_DeletedTarget(t *testing.T) {
// share. // share.
func slackTarget() database.Target { func slackTarget() database.Target {
return database.Target{ return database.Target{
Name: slackTargetName, Name: slackTargetName,
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
Active: true, Active: true,
URL: slackWebhookURL, Config: `{"webhookUrl":"` +
slackWebhookURL + `"}`,
MaxAttempts: 1, MaxAttempts: 1,
} }
} }
@@ -227,12 +233,12 @@ func TestNewTargetViews_HTTP(t *testing.T) {
t.Parallel() t.Parallel()
view := viewFor(t, database.Target{ view := viewFor(t, database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
URL: viewExampleHook, Config: `{"url":"` + viewExampleHook + `",` +
TimeoutSeconds: 30, `"timeout":30,` +
Headers: map[string]string{headerAuthorization: "Bearer sekrit"}, `"headers":{"Authorization":"Bearer sekrit"},` +
ForwardQuery: true, `"forwardQuery":true}`,
MaxAttempts: 5, MaxAttempts: 5,
}) })
fields := fieldMap(view.Config) fields := fieldMap(view.Config)
@@ -262,7 +268,7 @@ func TestNewTargetViews_HTTPSingleAttempt(t *testing.T) {
view := viewFor(t, database.Target{ view := viewFor(t, database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
URL: viewExampleHook, Config: `{"url":"` + viewExampleHook + `"}`,
MaxAttempts: 1, MaxAttempts: 1,
}) })
@@ -285,7 +291,7 @@ func TestNewTargetViews_HTTPMasksDestinationURL(t *testing.T) {
view := viewFor(t, database.Target{ view := viewFor(t, database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
URL: slackWebhookURL, Config: `{"url":"` + slackWebhookURL + `"}`,
MaxAttempts: 1, MaxAttempts: 1,
}) })
@@ -309,20 +315,25 @@ func TestNewTargetViews_Database(t *testing.T) {
t.Parallel() t.Parallel()
tests := map[string]struct { tests := map[string]struct {
expiry time.Duration config string
want string want string
}{ }{
"never": {expiry: 0, want: viewExpiryNever}, "empty config": {config: "", want: viewExpiryNever},
"1h": {expiry: time.Hour, want: "1 hour"}, "empty expiry": {config: `{}`, want: viewExpiryNever},
"12h": {expiry: 12 * time.Hour, want: "12 hours"}, "never literal": {
"24h": {expiry: 24 * time.Hour, want: "1 day"}, config: `{"expiry":"` + viewExpiryNever + `"}`,
"720h": {expiry: 720 * time.Hour, want: "30 days"}, want: viewExpiryNever,
"2160h": {expiry: 2160 * time.Hour, want: "90 days"}, },
"8760h": {expiry: 8760 * time.Hour, want: "365 days"}, "1h": {config: `{"expiry":"1h"}`, want: "1 hour"},
"36h": {expiry: 36 * time.Hour, want: "36 hours"}, "12h": {config: `{"expiry":"12h"}`, want: "12 hours"},
"1h30m": {expiry: 90 * time.Minute, want: "90 minutes"}, "24h": {config: `{"expiry":"24h"}`, want: "1 day"},
"45s": {expiry: 45 * time.Second, want: "45 seconds"}, "720h": {config: `{"expiry":"720h"}`, want: "30 days"},
"1.5s": {expiry: 1500 * time.Millisecond, want: "1.5s"}, "2160h": {config: `{"expiry":"2160h"}`, want: "90 days"},
"8760h": {config: `{"expiry":"8760h"}`, want: "365 days"},
"36h": {config: `{"expiry":"36h"}`, want: "36 hours"},
"1h30m": {config: `{"expiry":"1h30m"}`, want: "90 minutes"},
"45s": {config: `{"expiry":"45s"}`, want: "45 seconds"},
"1.5s": {config: `{"expiry":"1.5s"}`, want: "1.5s"},
} }
for name, tc := range tests { for name, tc := range tests {
@@ -330,9 +341,8 @@ func TestNewTargetViews_Database(t *testing.T) {
t.Parallel() t.Parallel()
view := viewFor(t, database.Target{ view := viewFor(t, database.Target{
Type: database.TargetTypeDatabase, Type: database.TargetTypeDatabase,
ArchiveExpiry: tc.expiry, Config: tc.config,
ArchiveRotation: rotationNone,
}) })
assert.Equal( assert.Equal(
@@ -348,23 +358,33 @@ func TestNewTargetViews_Database(t *testing.T) {
} }
// TestNewTargetViews_DatabaseRotation proves the target list shows a // TestNewTargetViews_DatabaseRotation proves the target list shows a
// database target's rotation. // database target's rotation, none when it has none stored.
func TestNewTargetViews_DatabaseRotation(t *testing.T) { func TestNewTargetViews_DatabaseRotation(t *testing.T) {
t.Parallel() t.Parallel()
// Each stored config, and the rotation the list shows for it.
tests := map[string]string{
"": rotationNone,
`{"rotation":""}`: rotationNone,
}
for _, rotation := range []string{ for _, rotation := range []string{
rotationNone, rotationMonthly, rotationDaily, rotationHourly, rotationNone, rotationMonthly, rotationDaily, rotationHourly,
} { } {
t.Run(rotation, func(t *testing.T) { tests[`{"rotation":"`+rotation+`"}`] = rotation
}
for config, want := range tests {
t.Run(config, func(t *testing.T) {
t.Parallel() t.Parallel()
view := viewFor(t, database.Target{ view := viewFor(t, database.Target{
Type: database.TargetTypeDatabase, Type: database.TargetTypeDatabase,
ArchiveRotation: rotation, Config: config,
}) })
assert.Equal( assert.Equal(
t, rotation, fieldMap(view.Config)["Archive rotation"], t, want, fieldMap(view.Config)["Archive rotation"],
) )
}) })
} }
@@ -374,28 +394,72 @@ func TestNewTargetViews_Log(t *testing.T) {
t.Parallel() t.Parallel()
view := viewFor(t, database.Target{ view := viewFor(t, database.Target{
Type: database.TargetTypeLog, Type: database.TargetTypeLog,
Config: "",
}) })
assert.Empty(t, view.Config) assert.Empty(t, view.Config)
} }
// TestNewTargetViews_UnknownType proves that a target of a type the // TestNewTargetViews_Unpresentable proves that no config the
// view does not know shows the neutral placeholder, and never its // view cannot present falls back to the stored blob.
// URL. func TestNewTargetViews_Unpresentable(t *testing.T) {
func TestNewTargetViews_UnknownType(t *testing.T) {
t.Parallel() t.Parallel()
view := viewFor(t, database.Target{ const blob = `{"webhookUrl":"https://hooks.slack.com` +
Type: database.TargetType("carrier-pigeon"), slackSecretPath + `"`
URL: slackWebhookURL,
})
assert.Equal( tests := map[string]database.Target{
t, "unknown target type": {
map[string]string{ Type: database.TargetType("carrier-pigeon"),
"Configuration": viewUnavailable, Config: blob,
}, },
fieldMap(view.Config), "unparseable json": {
) Type: database.TargetTypeSlack,
Config: blob,
MaxAttempts: 1,
},
"empty slack config": {
Type: database.TargetTypeSlack,
MaxAttempts: 1,
},
"slack config without url": {
Type: database.TargetTypeSlack,
Config: `{}`,
MaxAttempts: 1,
},
"unparseable http json": {
Type: database.TargetTypeHTTP,
Config: `{"url":`,
MaxAttempts: 1,
},
"unparseable archive json": {
Type: database.TargetTypeDatabase,
Config: `{"expiry":`,
},
"invalid archive expiry": {
Type: database.TargetTypeDatabase,
Config: `{"expiry":"a fortnight"}`,
},
"invalid archive rotation": {
Type: database.TargetTypeDatabase,
Config: weeklyConfig,
},
}
for name, target := range tests {
t.Run(name, func(t *testing.T) {
t.Parallel()
view := viewFor(t, target)
assert.Equal(
t,
map[string]string{
"Configuration": viewUnavailable,
},
fieldMap(view.Config),
)
})
}
} }
internal/delivery/target_database_test.go
+19 -108
View File
@@ -1,7 +1,6 @@
package delivery_test package delivery_test
import ( import (
"context"
"database/sql" "database/sql"
"fmt" "fmt"
"log/slog" "log/slog"
@@ -21,7 +20,6 @@ import (
"sneak.berlin/go/webhooker/internal/archive" "sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/database/databasetest" "sneak.berlin/go/webhooker/internal/database/databasetest"
"sneak.berlin/go/webhooker/internal/db/migrations"
"sneak.berlin/go/webhooker/internal/delivery" "sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/gormlog" "sneak.berlin/go/webhooker/internal/gormlog"
) )
@@ -80,11 +78,10 @@ func (env *archiveEnv) archivePath(tgt *database.Target) string {
) )
} }
// seedDatabaseTarget creates a webhook with one database target, // seedDatabaseTarget creates a webhook with one database target
// which keeps its archive forever in one file, and returns the // carrying the given target config JSON, and returns the target.
// target.
func (env *archiveEnv) seedDatabaseTarget( func (env *archiveEnv) seedDatabaseTarget(
t *testing.T, t *testing.T, configJSON string,
) *database.Target { ) *database.Target {
t.Helper() t.Helper()
@@ -100,12 +97,12 @@ func (env *archiveEnv) seedDatabaseTarget(
) )
tgt := &database.Target{ tgt := &database.Target{
WebhookID: wh.ID, WebhookID: wh.ID,
Name: "Archive", Name: "Archive",
Type: database.TargetTypeDatabase, Type: database.TargetTypeDatabase,
Active: true, Active: true,
ArchiveRotation: "none", Config: configJSON,
MaxAttempts: 1, MaxAttempts: 1,
} }
require.NoError( require.NoError(
t, t,
@@ -139,10 +136,9 @@ func seedDatabaseTargetDelivery(
Status: database.DeliveryStatusPending, Status: database.DeliveryStatusPending,
Event: event, Event: event,
Target: database.Target{ Target: database.Target{
Name: tgt.Name, Name: tgt.Name,
Type: database.TargetTypeDatabase, Type: database.TargetTypeDatabase,
ArchiveExpiry: tgt.ArchiveExpiry, Config: tgt.Config,
ArchiveRotation: tgt.ArchiveRotation,
}, },
} }
d.ID = dlv.ID d.ID = dlv.ID
@@ -222,7 +218,7 @@ func TestDeliverDatabase_ArchivesEvent(t *testing.T) {
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t) tgt := env.seedDatabaseTarget(t, "")
webhookDB := testWebhookDB(t) webhookDB := testWebhookDB(t)
event := seedEvent(t, webhookDB, `{"archived":true}`) event := seedEvent(t, webhookDB, `{"archived":true}`)
@@ -260,9 +256,9 @@ func TestDeliverDatabase_ArchivesEvent(t *testing.T) {
} }
// TestDeliverDatabase_ArchiveFailureFailsDelivery verifies that // TestDeliverDatabase_ArchiveFailureFailsDelivery verifies that
// an archive error (here: an event with no webhook id to record) // an archive error (here: an unparseable expiry in the target
// fails the delivery loudly: the attempt is recorded as failed // config) fails the delivery loudly: the attempt is recorded as
// with the error and the delivery is marked failed, not // failed with the error and the delivery is marked failed, not
// delivered. // delivered.
func TestDeliverDatabase_ArchiveFailureFailsDelivery( func TestDeliverDatabase_ArchiveFailureFailsDelivery(
t *testing.T, t *testing.T,
@@ -270,12 +266,11 @@ func TestDeliverDatabase_ArchiveFailureFailsDelivery(
t.Parallel() t.Parallel()
env := setupArchiveTest(t) env := setupArchiveTest(t)
tgt := env.seedDatabaseTarget(t) tgt := env.seedDatabaseTarget(t, `{"expiry":"nonsense"}`)
webhookDB := testWebhookDB(t) webhookDB := testWebhookDB(t)
event := seedEvent(t, webhookDB, `{"archived":false}`) event := seedEvent(t, webhookDB, `{"archived":false}`)
d := seedDatabaseTargetDelivery(t, webhookDB, event, tgt) d := seedDatabaseTargetDelivery(t, webhookDB, event, tgt)
d.Event.WebhookID = ""
env.eng.ExportDeliverDatabase(webhookDB, d) env.eng.ExportDeliverDatabase(webhookDB, d)
@@ -300,96 +295,12 @@ func TestDeliverDatabase_ArchiveFailureFailsDelivery(
"the attempt must be recorded as failed", "the attempt must be recorded as failed",
) )
assert.Contains(t, assert.Contains(t,
results[0].Error, "without a webhook id", results[0].Error, "nonsense",
"the archive error must be recorded on the attempt", "the archive error must be recorded on the attempt",
) )
assert.NoFileExists(t, assert.NoFileExists(t,
env.archivePath(tgt), env.archivePath(tgt),
"no archive file should exist for a failed write", "no archive file should exist for a failed config",
)
}
// seedArchivedRow creates the archive file at path holding one row, for
// eventID, archived at archivedAt.
func seedArchivedRow(
t *testing.T, path, eventID string, archivedAt time.Time,
) {
t.Helper()
sqlDB, err := sql.Open(
"sqlite", fmt.Sprintf("file:%s?mode=rwc", path),
)
require.NoError(t, err)
defer func() { _ = sqlDB.Close() }()
gdb, err := gorm.Open(
sqlite.Dialector{Conn: sqlDB},
&gorm.Config{Logger: gormlog.New(slog.New(slog.DiscardHandler))},
)
require.NoError(t, err)
require.NoError(t, migrations.Apply(gdb, migrations.Archive))
require.NoError(t, gdb.Exec(
"INSERT INTO archived_events (event_id, archived_at) VALUES (?, ?)",
eventID, archivedAt,
).Error)
}
// TestDeliverDatabase_TaskArchiveSettings proves a task's ArchiveExpiry
// and ArchiveRotation reach the archive through the engine. The
// target's row has neither, so only the task can bring them: with daily
// rotation the event goes to the file for its day, and opening that
// file with a one-hour expiry prunes the row it already held, archived
// two hours ago.
func TestDeliverDatabase_TaskArchiveSettings(t *testing.T) {
t.Parallel()
s := newISetup(t)
webhook := database.Webhook{UserID: uuid.NewString(), Name: "settings"}
webhook.ID = s.WebhookID
require.NoError(t, s.MainDB.Create(&webhook).Error)
target := database.Target{
WebhookID: s.WebhookID,
Name: "archive",
Type: database.TargetTypeDatabase,
Active: true,
MaxAttempts: 1,
}
require.NoError(t, s.MainDB.Create(&target).Error)
event := iSeedEvent(t, s.WebhookDB, s.WebhookID, `{"archived":true}`)
d := iSeedDelivery(
t, s.WebhookDB, event.ID, target.ID,
database.DeliveryStatusPending,
)
task := iTask(
d, event, s.WebhookID, target.ID, target.Name, "", 1, 1,
&event.Body,
)
task.TargetType = database.TargetTypeDatabase
task.ArchiveExpiry = time.Hour
task.ArchiveRotation = "daily"
daily := target
daily.ArchiveRotation = task.ArchiveRotation
dailyFile := archive.PathAt(s.DBMgr, &webhook, &daily, event.CreatedAt)
seedArchivedRow(t, dailyFile, "expired", time.Now().Add(-2*time.Hour))
s.Engine.ExportProcessNewTask(context.TODO(), &task)
iAssertStatus(t, s.WebhookDB, d.ID, database.DeliveryStatusDelivered)
var rows []archivedRow
require.NoError(t, openArchiveDBForRead(t, dailyFile).
Table("archived_events").Find(&rows).Error)
require.Len(t, rows, 1)
assert.Equal(t, event.ID, rows[0].EventID,
"the event goes to its day's file and the expired row is pruned",
) )
} }
internal/delivery/target_headers.go
+9 -7
View File
@@ -17,7 +17,7 @@ import (
const MaxTargetTimeoutSeconds = 300 const MaxTargetTimeoutSeconds = 300
// Errors returned when a target's header or timeout form input // Errors returned when a target's header or timeout form input
// cannot be turned into the target's setting. // cannot be turned into a configuration.
// //
// None of these ever quotes a header VALUE. A target header value // None of these ever quotes a header VALUE. A target header value
// is routinely an authorization token, and these messages are shown // is routinely an authorization token, and these messages are shown
@@ -48,8 +48,8 @@ var (
) )
// isReservedTargetHeader reports whether name (canonicalised) is a // isReservedTargetHeader reports whether name (canonicalised) is a
// header a target may not set, because the delivery path or // header a target configuration may not set, because the delivery
// net/http itself writes it regardless. // path or net/http itself writes it regardless.
// //
// These are rejected rather than accepted-and-ignored. Storing a // These are rejected rather than accepted-and-ignored. Storing a
// header that provably never reaches the wire tells the operator // header that provably never reaches the wire tells the operator
@@ -77,11 +77,13 @@ func isReservedTargetHeader(name string) bool {
// ParseTargetHeaders turns the target form's headers field — one // ParseTargetHeaders turns the target form's headers field — one
// "Name: value" pair per line, blank lines ignored — into the map // "Name: value" pair per line, blank lines ignored — into the map
// stored in database.Target.Headers. Names are canonicalised, so a // stored in HTTPTargetConfig.Headers. Names are canonicalised, so a
// name repeated in a different case is still a duplicate rather than // name repeated in a different case is still a duplicate rather than
// one pair silently overwriting the other. // one pair silently overwriting the other.
// //
// An input with no pairs yields an empty map. // An input with no pairs yields an empty map, which omitempty drops
// from the stored config, so a target configured with no headers
// stores no headers key at all.
func ParseTargetHeaders(raw string) (map[string]string, error) { func ParseTargetHeaders(raw string) (map[string]string, error) {
headers := make(map[string]string) headers := make(map[string]string)
@@ -221,8 +223,8 @@ func FormatTargetHeaders(headers map[string]string) string {
// ParseTargetTimeout interprets the target form's timeout field as // ParseTargetTimeout interprets the target form's timeout field as
// a whole number of seconds. An empty field means "unset" and yields // a whole number of seconds. An empty field means "unset" and yields
// 0, which the delivery path reads as "use the shared client's // 0, which omitempty drops from the stored config and which the
// timeout". // delivery path reads as "use the shared client's timeout".
// //
// Anything else that is not a whole number in range is an error, not // Anything else that is not a whole number in range is an error, not
// a silently substituted default: a target whose timeout was typed // a silently substituted default: a target whose timeout was typed
internal/delivery/target_headers_test.go
+99 -4
View File
@@ -1,21 +1,23 @@
package delivery_test package delivery_test
import ( import (
"encoding/json"
"strconv" "strconv"
"testing" "testing"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery" "sneak.berlin/go/webhooker/internal/delivery"
) )
// Literals these tests repeat, named so that each header name has one // Literals these tests repeat, named so that the header names and the
// definition. // keep-forever archive config each have one definition.
const ( const (
headerAuthorization = "Authorization" headerAuthorization = "Authorization"
headerContentType = "Content-Type" headerContentType = "Content-Type"
headerUserAgent = "User-Agent"
bearerValue = "Bearer abc" bearerValue = "Bearer abc"
archiveConfigNever = "{\"expiry\":\"never\"}"
) )
func TestParseTargetHeaders_AcceptsPairs(t *testing.T) { func TestParseTargetHeaders_AcceptsPairs(t *testing.T) {
@@ -39,13 +41,23 @@ func TestParseTargetHeaders_AcceptsPairs(t *testing.T) {
) )
} }
// An input of blank lines is no headers, not an error. // A configuration with no headers must stay indistinguishable from
// one written before the field existed, so omitempty drops the key.
func TestParseTargetHeaders_EmptyInputYieldsNoHeaders(t *testing.T) { func TestParseTargetHeaders_EmptyInputYieldsNoHeaders(t *testing.T) {
t.Parallel() t.Parallel()
got, err := delivery.ParseTargetHeaders("\n \n") got, err := delivery.ParseTargetHeaders("\n \n")
require.NoError(t, err) require.NoError(t, err)
assert.Empty(t, got) assert.Empty(t, got)
encoded, err := json.Marshal(delivery.HTTPTargetConfig{
URL: "https://example.com/h",
Headers: got,
})
require.NoError(t, err)
assert.JSONEq(
t, `{"url":"https://example.com/h"}`, string(encoded),
)
} }
func TestParseTargetHeaders_Rejects(t *testing.T) { func TestParseTargetHeaders_Rejects(t *testing.T) {
@@ -169,6 +181,89 @@ func TestFormatTargetTimeout(t *testing.T) {
assert.Empty(t, delivery.FormatTargetTimeout(-1)) assert.Empty(t, delivery.FormatTargetTimeout(-1))
} }
func TestNewTargetConfigForm(t *testing.T) {
t.Parallel()
form, err := delivery.NewTargetConfigForm(&database.Target{
Type: database.TargetTypeHTTP,
Config: `{"url":"https://example.com/h",` +
`"headers":{"Authorization":"Bearer abc"},` +
`"timeout":9}`,
MaxAttempts: 1,
})
require.NoError(t, err)
assert.Equal(t, "https://example.com/h", form.URL)
assert.Equal(t, "Authorization: Bearer abc\n", form.Headers)
assert.Equal(t, "9", form.Timeout)
form, err = delivery.NewTargetConfigForm(&database.Target{
Type: database.TargetTypeSlack,
Config: `{"webhookUrl":"https://hooks.example/s"}`,
MaxAttempts: 1,
})
require.NoError(t, err)
assert.Equal(t, "https://hooks.example/s", form.URL)
form, err = delivery.NewTargetConfigForm(&database.Target{
Type: database.TargetTypeDatabase,
Config: `{"expiry":"720h","rotation":"daily"}`,
})
require.NoError(t, err)
assert.Equal(t, "720h", form.Expiry)
assert.Equal(t, rotationDaily, form.Rotation)
form, err = delivery.NewTargetConfigForm(&database.Target{
Type: database.TargetTypeLog,
})
require.NoError(t, err)
assert.Empty(t, form.URL)
}
// A keep-forever archive target yields an empty expiry, so the edit
// form starts on never; saving it unchanged stores never, which means
// the same as an empty expiry.
func TestNewTargetConfigForm_DatabaseNeverIsBlank(t *testing.T) {
t.Parallel()
for _, cfg := range []string{"", `{}`, archiveConfigNever} {
form, err := delivery.NewTargetConfigForm(
&database.Target{
Type: database.TargetTypeDatabase,
Config: cfg,
},
)
require.NoError(t, err, cfg)
assert.Empty(t, form.Expiry, cfg)
}
}
// An unreadable stored config is an error rather than a blank form
// that looks like a target with no settings, so the caller can tell
// the operator that saving replaces the stored value.
func TestNewTargetConfigForm_UnreadableConfigErrors(t *testing.T) {
t.Parallel()
cases := []*database.Target{
{Type: database.TargetTypeHTTP, Config: "not json", MaxAttempts: 1},
{Type: database.TargetTypeHTTP, Config: `{}`, MaxAttempts: 1},
{Type: database.TargetTypeSlack, Config: "", MaxAttempts: 1},
{
Type: database.TargetTypeDatabase,
Config: `{"expiry":"soon"}`,
},
{
Type: database.TargetTypeDatabase,
Config: weeklyConfig,
},
{Type: database.TargetType("nope")},
}
for _, target := range cases {
_, err := delivery.NewTargetConfigForm(target)
require.Error(t, err, target.Type)
}
}
// The ceiling exists so one misconfigured target cannot hold a // The ceiling exists so one misconfigured target cannot hold a
// delivery worker indefinitely, and it is inclusive. // delivery worker indefinitely, and it is inclusive.
func TestParseTargetTimeout_CeilingIsInclusive(t *testing.T) { func TestParseTargetTimeout_CeilingIsInclusive(t *testing.T) {
internal/delivery/target_http.go
+123 -27
View File
@@ -4,6 +4,7 @@ import (
"bytes" "bytes"
"context" "context"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
@@ -16,6 +17,29 @@ import (
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
) )
// Sentinel errors returned by the config parsers.
var (
errEmptyTargetConfig = errors.New(
"empty target config",
)
errMissingTargetURL = errors.New(
"target URL is required",
)
)
// HTTPTargetConfig holds configuration for http target
// types.
type HTTPTargetConfig struct {
URL string `json:"url"`
Headers map[string]string `json:"headers,omitempty"`
Timeout int `json:"timeout,omitempty"`
// ForwardQuery passes each event's query string on to the target,
// appended to URL. Off, the target URL is sent exactly as
// configured.
ForwardQuery bool `json:"forwardQuery,omitempty"`
}
// httpCore holds the retry, backoff, and circuit-breaker // httpCore holds the retry, backoff, and circuit-breaker
// machinery shared by the HTTP and Slack targets. Each of // machinery shared by the HTTP and Slack targets. Each of
// those targets owns its own httpCore instance (and thus its // those targets owns its own httpCore instance (and thus its
@@ -60,8 +84,8 @@ func (c *httpCore) deliver(
// engine is stopping, which cancels the context its request is made // engine is stopping, which cancels the context its request is made
// with. Such an attempt is not recorded: no DeliveryResult, no circuit // with. Such an attempt is not recorded: no DeliveryResult, no circuit
// breaker outcome, no status change, no count in the attempt metrics. // breaker outcome, no status change, no count in the attempt metrics.
// The delivery stays pending or retrying, and the recovery routine // The delivery stays pending or retrying, and startup recovery sends
// sends it again on the next start. // it again on the next start.
// //
// A target's own timeout leaves that context alone, so its attempt is // A target's own timeout leaves that context alone, so its attempt is
// still recorded as the failure it is. // still recorded as the failure it is.
@@ -310,8 +334,7 @@ func (c *httpCore) publishCircuitState(
// remainingBackoff returns how long remains of the backoff // remainingBackoff returns how long remains of the backoff
// window for the last attempt of a recovered retrying // window for the last attempt of a recovered retrying
// delivery: zero once it has passed, or when the last attempt // delivery. It implements rescheduler.
// cannot be read. It implements rescheduler.
func (c *httpCore) remainingBackoff( func (c *httpCore) remainingBackoff(
webhookDB *gorm.DB, webhookDB *gorm.DB,
deliveryID string, deliveryID string,
@@ -334,6 +357,29 @@ func (c *httpCore) remainingBackoff(
return max(remaining, 0) return max(remaining, 0)
} }
// backoffElapsed reports whether the backoff window for the
// last attempt of a retrying delivery has passed. It
// implements rescheduler.
func (c *httpCore) backoffElapsed(
webhookDB *gorm.DB,
deliveryID string,
attemptNum int,
) bool {
var lastResult database.DeliveryResult
err := webhookDB.
Where("delivery_id = ?", deliveryID).
Order("created_at DESC").
First(&lastResult).Error
if err != nil {
return true
}
backoff := Backoff(attemptNum)
return time.Since(lastResult.CreatedAt) >= backoff
}
// Backoff is how long an http or slack target with retries waits after // Backoff is how long an http or slack target with retries waits after
// a delivery's failed attempt attemptNum before trying it again. // a delivery's failed attempt attemptNum before trying it again.
func Backoff(attemptNum int) time.Duration { func Backoff(attemptNum int) time.Duration {
@@ -361,8 +407,34 @@ func (t *httpTarget) Deliver(
task *Task, task *Task,
sched Scheduler, sched Scheduler,
) { ) {
cfg, err := parseHTTPConfig(d.Target.Config)
if err != nil {
t.eng.log.Error(
"invalid HTTP target config",
"target_id", d.TargetID,
"error", err,
)
recErr := t.eng.recordResult(
webhookDB, d, task.AttemptNum,
false, 0, "", err.Error(), 0,
)
if recErr != nil {
t.eng.bookkeepingFailed(d, recErr)
return
}
t.eng.settleStatus(
webhookDB, d, d.Target.Type,
database.DeliveryStatusFailed,
)
return
}
attempt := func() attemptResult { attempt := func() attemptResult {
return t.attempt(ctx, &d.Target, &d.Event) return t.attempt(ctx, cfg, &d.Event)
} }
t.deliver( t.deliver(
@@ -377,11 +449,11 @@ func (t *httpTarget) Deliver(
// string; only a transport-level error does. // string; only a transport-level error does.
func (t *httpTarget) attempt( func (t *httpTarget) attempt(
ctx context.Context, ctx context.Context,
target *database.Target, cfg *HTTPTargetConfig,
event *database.Event, event *database.Event,
) attemptResult { ) attemptResult {
statusCode, respBody, duration, reqErr := statusCode, respBody, duration, reqErr :=
t.doHTTPRequest(ctx, target, event) t.doHTTPRequest(ctx, cfg, event)
success := reqErr == nil && success := reqErr == nil &&
statusCode >= httpSuccessMin && statusCode >= httpSuccessMin &&
@@ -403,7 +475,7 @@ func (t *httpTarget) attempt(
func (t *httpTarget) doHTTPRequest( func (t *httpTarget) doHTTPRequest(
ctx context.Context, ctx context.Context,
target *database.Target, cfg *HTTPTargetConfig,
event *database.Event, event *database.Event,
) (int, string, int64, error) { ) (int, string, int64, error) {
start := time.Now() start := time.Now()
@@ -411,7 +483,7 @@ func (t *httpTarget) doHTTPRequest(
req, reqErr := http.NewRequestWithContext( req, reqErr := http.NewRequestWithContext(
ctx, ctx,
http.MethodPost, http.MethodPost,
target.URL, cfg.URL,
bytes.NewReader([]byte(event.Body)), bytes.NewReader([]byte(event.Body)),
) )
if reqErr != nil { if reqErr != nil {
@@ -421,15 +493,15 @@ func (t *httpTarget) doHTTPRequest(
) )
} }
if target.ForwardQuery { if cfg.ForwardQuery {
appendQuery(req.URL, event.RawQuery) appendQuery(req.URL, event.RawQuery)
} }
originScoped := applyRequestHeaders( originScoped := applyRequestHeaders(
req, event, target.Headers, t.eng.userAgent(), req, event, cfg, t.eng.userAgent(),
) )
client := t.clientForRequest(target.TimeoutSeconds, originScoped) client := t.clientForRequest(cfg, originScoped)
resp, doErr := executeHTTPRequest(client, req) resp, doErr := executeHTTPRequest(client, req)
@@ -469,17 +541,17 @@ func appendQuery(u *url.URL, rawQuery string) {
} }
// clientForRequest returns the client for one delivery attempt. // clientForRequest returns the client for one delivery attempt.
// timeoutSeconds is the target's TimeoutSeconds, and originScoped is // originScoped is the header set applyRequestHeaders built for that
// the header set applyRequestHeaders built for that attempt; a // attempt; a request with neither a per-target timeout nor an
// request with neither a per-target timeout nor an origin-scoped // origin-scoped header gets the shared client, because there is
// header gets the shared client, because there is then nothing for // then nothing for the redirect policy to strip and net/http's
// the redirect policy to strip and net/http's default policy already // default policy already withholds Authorization and Cookie across
// withholds Authorization and Cookie across hosts. // hosts.
func (t *httpTarget) clientForRequest( func (t *httpTarget) clientForRequest(
timeoutSeconds int, cfg *HTTPTargetConfig,
originScoped []string, originScoped []string,
) *http.Client { ) *http.Client {
if timeoutSeconds <= 0 && len(originScoped) == 0 { if cfg.Timeout <= 0 && len(originScoped) == 0 {
return t.client return t.client
} }
@@ -492,9 +564,9 @@ func (t *httpTarget) clientForRequest(
Transport: t.client.Transport, Transport: t.client.Transport,
} }
if timeoutSeconds > 0 { if cfg.Timeout > 0 {
client.Timeout = time.Duration( client.Timeout = time.Duration(
timeoutSeconds, cfg.Timeout,
) * time.Second ) * time.Second
} }
@@ -505,6 +577,31 @@ func (t *httpTarget) clientForRequest(
return client return client
} }
func parseHTTPConfig(
configJSON string,
) (*HTTPTargetConfig, error) {
if configJSON == "" {
return nil, errEmptyTargetConfig
}
var cfg HTTPTargetConfig
err := json.Unmarshal(
[]byte(configJSON), &cfg,
)
if err != nil {
return nil, fmt.Errorf(
"parsing config JSON: %w", err,
)
}
if cfg.URL == "" {
return nil, errMissingTargetURL
}
return &cfg, nil
}
// isForwardableHeader returns true if the header should // isForwardableHeader returns true if the header should
// be forwarded to targets. // be forwarded to targets.
func isForwardableHeader(name string) bool { func isForwardableHeader(name string) bool {
@@ -536,12 +633,11 @@ func isForwardableHeader(name string) bool {
// wins, otherwise the event's ContentType, otherwise none. The inbound // wins, otherwise the event's ContentType, otherwise none. The inbound
// Content-Type in the event's headers is never forwarded. // Content-Type in the event's headers is never forwarded.
// //
// configured is the target's Headers. userAgent is set last, over // userAgent is set last, over any configured or inbound User-Agent.
// any configured or inbound User-Agent.
func applyRequestHeaders( func applyRequestHeaders(
req *http.Request, req *http.Request,
event *database.Event, event *database.Event,
configured map[string]string, cfg *HTTPTargetConfig,
userAgent string, userAgent string,
) []string { ) []string {
if event.ContentType != "" { if event.ContentType != "" {
@@ -552,7 +648,7 @@ func applyRequestHeaders(
originScoped := forwardEventHeaders(req, event) originScoped := forwardEventHeaders(req, event)
for k, v := range configured { for k, v := range cfg.Headers {
req.Header.Set(k, v) req.Header.Set(k, v)
originScoped[http.CanonicalHeaderKey(k)] = struct{}{} originScoped[http.CanonicalHeaderKey(k)] = struct{}{}
} }
@@ -617,7 +713,7 @@ func forwardEventHeaders(
} }
// executeHTTPRequest sends an HTTP request using the provided // executeHTTPRequest sends an HTTP request using the provided
// client. URLs are validated by the target forms and the // client. URLs are validated by the config parsers and the
// SSRF-safe transport before reaching here. // SSRF-safe transport before reaching here.
// //
// Transport failures are masked here, at the single point // Transport failures are masked here, at the single point
internal/delivery/target_http_query_test.go
+28 -18
View File
@@ -2,6 +2,7 @@ package delivery_test
import ( import (
"context" "context"
"encoding/json"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"testing" "testing"
@@ -17,13 +18,25 @@ import (
// with. // with.
const eventQuery = "a=1&b=2" const eventQuery = "a=1&b=2"
// deliverWithQuery sends one event that arrived with eventQuery to an // httpTargetConfig is the stored configuration of an HTTP target at
// HTTP target at targetURL, with its ForwardQuery setting as given, // targetURL.
// through the path a received event's delivery takes, and returns the func httpTargetConfig(
// attempt it recorded.
func deliverWithQuery(
t *testing.T, targetURL string, forwardQuery bool, t *testing.T, targetURL string, forwardQuery bool,
) database.DeliveryResult { ) string {
t.Helper()
cfg, err := json.Marshal(delivery.HTTPTargetConfig{
URL: targetURL, ForwardQuery: forwardQuery,
})
require.NoError(t, err)
return string(cfg)
}
// deliverWithQuery sends one event that arrived with eventQuery to an
// HTTP target configured with cfg, through the path a received event's
// delivery takes, and returns the attempt it recorded.
func deliverWithQuery(t *testing.T, cfg string) database.DeliveryResult {
t.Helper() t.Helper()
s := newISetup(t) s := newISetup(t)
@@ -34,10 +47,8 @@ func deliverWithQuery(
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
task := iTask( task := iTask(
d, event, s.WebhookID, d.TargetID, "query", targetURL, 1, 1, d, event, s.WebhookID, d.TargetID, "query", cfg, 1, 1, &event.Body,
&event.Body,
) )
task.ForwardQuery = forwardQuery
task.RawQuery = eventQuery task.RawQuery = eventQuery
s.Engine.ExportProcessNewTask(context.TODO(), &task) s.Engine.ExportProcessNewTask(context.TODO(), &task)
@@ -100,9 +111,9 @@ func TestDeliverHTTP_ForwardQuery(t *testing.T) {
)) ))
t.Cleanup(ts.Close) t.Cleanup(ts.Close)
result := deliverWithQuery( result := deliverWithQuery(t, httpTargetConfig(
t, ts.URL+tc.path, tc.forwardQuery, t, ts.URL+tc.path, tc.forwardQuery,
) ))
assert.True(t, result.Success) assert.True(t, result.Success)
require.Len(t, received, 1) require.Len(t, received, 1)
@@ -133,13 +144,12 @@ func TestDeliverHTTP_ForwardedQueryKeepsTheTargetURLMasked(t *testing.T) {
t.Cleanup(ts.Close) t.Cleanup(ts.Close)
target := &database.Target{ target := &database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
URL: ts.URL + "/in?token=" + secret, Config: httpTargetConfig(t, ts.URL+"/in?token="+secret, true),
ForwardQuery: true, MaxAttempts: 1,
MaxAttempts: 1,
} }
deliverWithQuery(t, target.URL, target.ForwardQuery) deliverWithQuery(t, target.Config)
require.Len(t, received, 1) require.Len(t, received, 1)
sent := <-received sent := <-received
@@ -154,9 +164,9 @@ func TestDeliverHTTP_ForwardedQueryKeepsTheTargetURLMasked(t *testing.T) {
} }
// Nothing listens on port 1. // Nothing listens on port 1.
failed := deliverWithQuery( failed := deliverWithQuery(t, httpTargetConfig(
t, "http://127.0.0.1:1/in?token="+secret, true, t, "http://127.0.0.1:1/in?token="+secret, true,
) ))
require.NotEmpty(t, failed.Error) require.NotEmpty(t, failed.Error)
assert.NotContains(t, failed.Error, secret) assert.NotContains(t, failed.Error, secret)
assert.NotContains(t, failed.Error, eventQuery) assert.NotContains(t, failed.Error, eventQuery)
internal/delivery/target_redact.go
+27 -6
View File
@@ -18,7 +18,7 @@ const RedactionMarker = "(redacted)"
// line behind the delivery path's own URL masking. // line behind the delivery path's own URL masking.
// //
// It removes byte-identical echoes of strings taken from the // It removes byte-identical echoes of strings taken from the
// target's URL and headers, and nothing else. Anything // target's stored configuration, and nothing else. Anything
// the remote re-encodes survives: JSON "\/" escaping (what // the remote re-encodes survives: JSON "\/" escaping (what
// PHP's json_encode emits by default), percent-encoding, HTML // PHP's json_encode emits by default), percent-encoding, HTML
// entities, and an echo of only part of a path. It cannot // entities, and an echo of only part of a path. It cannot
@@ -115,7 +115,7 @@ func (r Redactor) secretPrefixSuffix(s string) int {
} }
// targetSecrets returns the credential-bearing strings a // targetSecrets returns the credential-bearing strings a
// target's settings carry. // target's configuration carries.
// //
// The destination URL contributes. Its path, query and // The destination URL contributes. Its path, query and
// userinfo are the credential for both target types that have // userinfo are the credential for both target types that have
@@ -129,15 +129,36 @@ func (r Redactor) secretPrefixSuffix(s string) int {
// picks out. That is the same class-based rule applied to // picks out. That is the same class-based rule applied to
// URLs: an echoed Accept or User-Agent still renders, an // URLs: an echoed Accept or User-Agent still renders, an
// echoed Authorization does not. // echoed Authorization does not.
//
// A setting a type does not use is empty and contributes
// nothing, so a database or log target has nothing to redact.
func targetSecrets(t *database.Target) []string { func targetSecrets(t *database.Target) []string {
if t == nil { if t == nil {
return nil return nil
} }
return append(urlSecrets(t.URL), headerSecrets(t.Headers)...) switch t.Type {
case database.TargetTypeSlack:
cfg, err := parseSlackConfig(t.Config)
if err != nil {
return nil
}
return urlSecrets(cfg.WebhookURL)
case database.TargetTypeHTTP:
cfg, err := parseHTTPConfig(t.Config)
if err != nil {
return nil
}
return append(
urlSecrets(cfg.URL),
headerSecrets(cfg.Headers)...,
)
case database.TargetTypeDatabase, database.TargetTypeLog:
// Neither has a destination URL, so neither has
// anything to redact.
return nil
default:
return nil
}
} }
// urlSecrets returns the substrings of a destination URL that // urlSecrets returns the substrings of a destination URL that
internal/delivery/target_redact_test.go
+29 -29
View File
@@ -23,7 +23,7 @@ func TestRedactor_RemovesSlackWebhookURL(t *testing.T) {
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
URL: redactWebhookURL, Config: `{"webhookUrl":"` + redactWebhookURL + `"}`,
MaxAttempts: 1, MaxAttempts: 1,
}) })
@@ -54,7 +54,7 @@ func TestRedactor_RemovesSecretSeveredByACut(t *testing.T) {
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
URL: redactWebhookURL, Config: `{"webhookUrl":"` + redactWebhookURL + `"}`,
MaxAttempts: 1, MaxAttempts: 1,
}) })
@@ -86,19 +86,18 @@ func TestRedactor_RedactsCredentialShapedHeaderValues(
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
URL: "https://example.com/in", Config: `{"url":"https://example.com/in",` +
Headers: map[string]string{ `"headers":{` +
headerAuthorization: "Bearer AAAAAAAAAAAA", `"Authorization":"Bearer AAAAAAAAAAAA",` +
"Cookie": "session=BBBBBBBBBBBB", `"Cookie":"session=BBBBBBBBBBBB",` +
probeHeaderName: "CCCCCCCCCCCC", `"X-Api-Key":"CCCCCCCCCCCC",` +
"X-Hub-Signature": "sha256=DDDDDDDDDDDD", `"X-Hub-Signature":"sha256=DDDDDDDDDDDD",` +
"X-Sig": "EEEEEEEEEEEE", `"X-Sig":"EEEEEEEEEEEE",` +
"X-Pass": "FFFFFFFFFFFF", `"X-Pass":"FFFFFFFFFFFF",` +
"X-HMAC": "GGGGGGGGGGGG", `"X-HMAC":"GGGGGGGGGGGG",` +
"X-Credential": "HHHHHHHHHHHH", `"X-Credential":"HHHHHHHHHHHH",` +
"Accept": "application/json", `"Accept":"application/json",` +
headerUserAgent: "webhooker/1.0", `"User-Agent":"webhooker/1.0"}}`,
},
MaxAttempts: 1, MaxAttempts: 1,
}) })
@@ -137,9 +136,9 @@ func TestRedactor_IgnoresVeryShortHeaderValues(t *testing.T) {
t.Parallel() t.Parallel()
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
URL: "https://example.com/in", Config: `{"url":"https://example.com/in",` +
Headers: map[string]string{probeHeaderName: "ab"}, `"headers":{"X-Api-Key":"ab"}}`,
MaxAttempts: 1, MaxAttempts: 1,
}) })
@@ -156,7 +155,7 @@ func TestRedactor_RemovesBarePath(t *testing.T) {
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
URL: redactWebhookURL, Config: `{"webhookUrl":"` + redactWebhookURL + `"}`,
MaxAttempts: 1, MaxAttempts: 1,
}) })
@@ -190,7 +189,7 @@ func TestRedactor_RemovesHTTPURLQueryAndUserinfo(t *testing.T) {
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
URL: raw, Config: `{"url":"` + raw + `"}`,
MaxAttempts: 1, MaxAttempts: 1,
}) })
@@ -240,7 +239,7 @@ func TestRedactor_RemovesEchoedQueryOfURLWithoutPath(t *testing.T) {
} { } {
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
URL: dest, Config: `{"url":"` + dest + `"}`,
MaxAttempts: 1, MaxAttempts: 1,
}) })
@@ -262,7 +261,7 @@ func TestRedactor_LeavesUnrelatedTextAlone(t *testing.T) {
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeSlack, Type: database.TargetTypeSlack,
URL: redactWebhookURL, Config: `{"webhookUrl":"` + redactWebhookURL + `"}`,
MaxAttempts: 1, MaxAttempts: 1,
}) })
@@ -289,7 +288,7 @@ func TestRedactor_EmptyUserinfoDoesNotShredTheBody(t *testing.T) {
r := delivery.NewRedactor(&database.Target{ r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
URL: dest, Config: `{"url":"` + dest + `"}`,
MaxAttempts: 1, MaxAttempts: 1,
}) })
@@ -307,10 +306,11 @@ func TestRedactor_EmptyUserinfoDoesNotShredTheBody(t *testing.T) {
) )
} }
// TestRedactor_ZeroValueAndURLlessTargets pins that a caller // TestRedactor_ZeroValueAndConfiglessTargets pins that a
// with no target, or a target with no destination URL, gets a // caller with no target, an unparseable config, or a target
// redactor that changes nothing rather than one that panics. // type with no destination URL gets a redactor that changes
func TestRedactor_ZeroValueAndURLlessTargets(t *testing.T) { // nothing rather than one that panics.
func TestRedactor_ZeroValueAndConfiglessTargets(t *testing.T) {
t.Parallel() t.Parallel()
const text = "some response body" const text = "some response body"
@@ -323,8 +323,8 @@ func TestRedactor_ZeroValueAndURLlessTargets(t *testing.T) {
for _, tgt := range []database.Target{ for _, tgt := range []database.Target{
{Type: database.TargetTypeLog}, {Type: database.TargetTypeLog},
{Type: database.TargetTypeDatabase}, {Type: database.TargetTypeDatabase},
{Type: database.TargetTypeSlack, MaxAttempts: 1}, {Type: database.TargetTypeSlack, Config: "not json", MaxAttempts: 1},
{Type: database.TargetTypeHTTP, MaxAttempts: 1}, {Type: database.TargetTypeHTTP, Config: "", MaxAttempts: 1},
} { } {
assert.Equal( assert.Equal(
t, text, t, text,
internal/delivery/target_slack.go
+81 -19
View File
@@ -4,6 +4,7 @@ import (
"bytes" "bytes"
"context" "context"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
@@ -14,6 +15,18 @@ import (
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
) )
// errMissingWebhookURL is returned when a Slack target config
// omits its webhook URL.
var errMissingWebhookURL = errors.New(
"webhook_url is required",
)
// SlackTargetConfig holds configuration for slack target
// types.
type SlackTargetConfig struct {
WebhookURL string `json:"webhookUrl"`
}
// slackTarget delivers events to Slack incoming webhooks. It // slackTarget delivers events to Slack incoming webhooks. It
// formats the event into a Slack message and posts it as // formats the event into a Slack message and posts it as
// JSON. It shares the retry core with the HTTP target: a // JSON. It shares the retry core with the HTTP target: a
@@ -33,6 +46,19 @@ func (t *slackTarget) Deliver(
task *Task, task *Task,
sched Scheduler, sched Scheduler,
) { ) {
cfg, err := parseSlackConfig(d.Target.Config)
if err != nil {
t.eng.log.Error(
"invalid Slack target config",
"target_id", d.TargetID,
"error", err,
)
t.failConfig(webhookDB, d, err)
return
}
msg := FormatSlackMessage(&d.Event) msg := FormatSlackMessage(&d.Event)
payload, err := json.Marshal( payload, err := json.Marshal(
@@ -45,26 +71,13 @@ func (t *slackTarget) Deliver(
"error", err, "error", err,
) )
recErr := t.eng.recordResult( t.failConfig(webhookDB, d, err)
webhookDB, d, 1,
false, 0, "", err.Error(), 0,
)
if recErr != nil {
t.eng.bookkeepingFailed(d, recErr)
return
}
t.eng.settleStatus(
webhookDB, d, d.Target.Type,
database.DeliveryStatusFailed,
)
return return
} }
attempt := func() attemptResult { attempt := func() attemptResult {
return t.attempt(ctx, d.Target.URL, payload) return t.attempt(ctx, cfg, payload)
} }
t.deliver( t.deliver(
@@ -73,13 +86,37 @@ func (t *slackTarget) Deliver(
) )
} }
// attempt performs a single Slack POST to targetURL and derives // failConfig records a first-attempt failure for a delivery
// its outcome, preserving the engine's original semantics: a // that could not be prepared (bad config or unmarshalable
// payload) and marks it failed.
func (t *slackTarget) failConfig(
webhookDB *gorm.DB,
d *database.Delivery,
err error,
) {
recErr := t.eng.recordResult(
webhookDB, d, 1,
false, 0, "", err.Error(), 0,
)
if recErr != nil {
t.eng.bookkeepingFailed(d, recErr)
return
}
t.eng.settleStatus(
webhookDB, d, d.Target.Type,
database.DeliveryStatusFailed,
)
}
// attempt performs a single Slack POST and derives its
// outcome, preserving the engine's original semantics: a
// non-2xx response records an "HTTP <code>" error string and // non-2xx response records an "HTTP <code>" error string and
// a transport error records a "sending request" error. // a transport error records a "sending request" error.
func (t *slackTarget) attempt( func (t *slackTarget) attempt(
ctx context.Context, ctx context.Context,
targetURL string, cfg *SlackTargetConfig,
payload []byte, payload []byte,
) attemptResult { ) attemptResult {
start := time.Now() start := time.Now()
@@ -87,7 +124,7 @@ func (t *slackTarget) attempt(
req, err := http.NewRequestWithContext( req, err := http.NewRequestWithContext(
ctx, ctx,
http.MethodPost, http.MethodPost,
targetURL, cfg.WebhookURL,
bytes.NewReader(payload), bytes.NewReader(payload),
) )
if err != nil { if err != nil {
@@ -149,6 +186,31 @@ func (t *slackTarget) readSlackResponse(
} }
} }
func parseSlackConfig(
configJSON string,
) (*SlackTargetConfig, error) {
if configJSON == "" {
return nil, errEmptyTargetConfig
}
var cfg SlackTargetConfig
err := json.Unmarshal(
[]byte(configJSON), &cfg,
)
if err != nil {
return nil, fmt.Errorf(
"parsing config JSON: %w", err,
)
}
if cfg.WebhookURL == "" {
return nil, errMissingWebhookURL
}
return &cfg, nil
}
// FormatSlackMessage builds a Slack-compatible message // FormatSlackMessage builds a Slack-compatible message
// string from a webhook event. // string from a webhook event.
func FormatSlackMessage( func FormatSlackMessage(
internal/delivery/terminal_state_test.go
+46 -46
View File
@@ -45,7 +45,7 @@ func tSeedDeletedTarget(
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, name, t, s.MainDB, targetID, s.WebhookID, name,
database.TargetTypeHTTP, url, 5, database.TargetTypeHTTP, iHTTPConfig(url), 5,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -122,9 +122,9 @@ func TestProcessDelivery_UnknownTargetType_RecordsWhy(
) )
target := database.Target{ target := database.Target{
Name: "mystery", Name: "mystery",
Type: tUnknownType, Type: tUnknownType,
URL: "http://example.com/hook", Config: iHTTPConfig("http://example.com/hook"),
} }
target.ID = targetID target.ID = targetID
@@ -140,7 +140,7 @@ func TestProcessDelivery_UnknownTargetType_RecordsWhy(
body := event.Body body := event.Body
task := iTask( task := iTask(
seeded, event, s.WebhookID, targetID, "mystery", seeded, event, s.WebhookID, targetID, "mystery",
target.URL, 0, 1, &body, target.Config, 0, 1, &body,
) )
task.TargetType = tUnknownType task.TargetType = tUnknownType
@@ -178,7 +178,7 @@ func TestRecoverSingleRetry_TargetDeleted(t *testing.T) {
) )
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, atStart, context.Background(), s.WebhookID,
) )
iAssertStatus( iAssertStatus(
@@ -201,7 +201,7 @@ func TestRecoverSingleRetry_TargetDeleted(t *testing.T) {
) )
} }
func TestRecoverSingleRetry_OnTick_TargetDeleted(t *testing.T) { func TestSweepSingleRetry_TargetDeleted(t *testing.T) {
t.Parallel() t.Parallel()
s := newISetup(t) s := newISetup(t)
@@ -218,11 +218,11 @@ func TestRecoverSingleRetry_OnTick_TargetDeleted(t *testing.T) {
// Twice, because the bug was an error the sweep repeated every // Twice, because the bug was an error the sweep repeated every
// minute for the life of the database: the second sweep must // minute for the life of the database: the second sweep must
// find nothing left to do. // find nothing left to do.
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportSweepWebhookRetries(
context.Background(), s.WebhookID, onTick, context.Background(), s.WebhookID,
) )
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportSweepWebhookRetries(
context.Background(), s.WebhookID, onTick, context.Background(), s.WebhookID,
) )
iAssertStatus( iAssertStatus(
@@ -239,11 +239,10 @@ func TestRecoverSingleRetry_OnTick_TargetDeleted(t *testing.T) {
assert.Zero(t, s.Engine.ExportInflightHeld()) assert.Zero(t, s.Engine.ExportInflightHeld())
} }
// TestRecoverSingleRetry_OnTick_TargetNeverExisted covers the other // TestSweepSingleRetry_TargetNeverExisted covers the other half of the
// half of the soft-delete distinction: an id with no row at all, // soft-delete distinction: an id with no row at all, deleted or
// deleted or otherwise, must not be reported as something the operator // otherwise, must not be reported as something the operator deleted.
// deleted. func TestSweepSingleRetry_TargetNeverExisted(t *testing.T) {
func TestRecoverSingleRetry_OnTick_TargetNeverExisted(t *testing.T) {
t.Parallel() t.Parallel()
s := newISetup(t) s := newISetup(t)
@@ -265,8 +264,8 @@ func TestRecoverSingleRetry_OnTick_TargetNeverExisted(t *testing.T) {
iSeedFailedResult(t, s.WebhookDB, d.ID) iSeedFailedResult(t, s.WebhookDB, d.ID)
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportSweepWebhookRetries(
context.Background(), s.WebhookID, onTick, context.Background(), s.WebhookID,
) )
iAssertStatus( iAssertStatus(
@@ -304,8 +303,8 @@ func TestFailMissingTarget_WritesNoTargetRow(
database.DeliveryStatusRetrying, database.DeliveryStatusRetrying,
) )
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportSweepWebhookRetries(
context.Background(), s.WebhookID, onTick, context.Background(), s.WebhookID,
) )
iAssertStatus( iAssertStatus(
@@ -346,10 +345,11 @@ func tRetryChainSetup(
iCreateWebhook(t, s.MainDB, s.WebhookID, name) iCreateWebhook(t, s.MainDB, s.WebhookID, name)
targetID := uuid.New().String() targetID := uuid.New().String()
cfg := iHTTPConfig(ts.URL)
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, name, t, s.MainDB, targetID, s.WebhookID, name,
database.TargetTypeHTTP, ts.URL, 5, database.TargetTypeHTTP, cfg, 5,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -366,7 +366,7 @@ func tRetryChainSetup(
body := event.Body body := event.Body
return iTask( return iTask(
d, event, s.WebhookID, targetID, name, ts.URL, 5, 2, &body, d, event, s.WebhookID, targetID, name, cfg, 5, 2, &body,
), targetID ), targetID
} }
@@ -527,7 +527,7 @@ func TestRecoverSingleRetry_TargetUnreadable_LeavesDeliveryAlone(
iCreateTarget( iCreateTarget(
t, s.MainDB, targetID, s.WebhookID, "healthy", t, s.MainDB, targetID, s.WebhookID, "healthy",
database.TargetTypeHTTP, database.TargetTypeHTTP,
"http://example.com/hook", 5, iHTTPConfig("http://example.com/hook"), 5,
) )
event := iSeedEvent( event := iSeedEvent(
@@ -546,7 +546,7 @@ func TestRecoverSingleRetry_TargetUnreadable_LeavesDeliveryAlone(
require.NoError(t, sqlDB.Close()) require.NoError(t, sqlDB.Close())
s.Engine.ExportRecoverRetryingDeliveries( s.Engine.ExportRecoverRetryingDeliveries(
context.Background(), s.WebhookDB, s.WebhookID, s.WebhookDB, s.WebhookID,
) )
iAssertStatus( iAssertStatus(
@@ -564,7 +564,7 @@ func TestRecoverSingleRetry_TargetUnreadable_LeavesDeliveryAlone(
// --- 4. A pending delivery whose target is gone --- // --- 4. A pending delivery whose target is gone ---
func TestRecoverPendingDeliveries_TargetDeleted(t *testing.T) { func TestRecoverPending_TargetDeleted(t *testing.T) {
t.Parallel() t.Parallel()
s := newISetup(t) s := newISetup(t)
@@ -577,7 +577,7 @@ func TestRecoverPendingDeliveries_TargetDeleted(t *testing.T) {
) )
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, atStart, context.Background(), s.WebhookID,
) )
iAssertStatus( iAssertStatus(
@@ -600,10 +600,10 @@ func TestRecoverPendingDeliveries_TargetDeleted(t *testing.T) {
) )
} }
// TestRecoverPendingDeliveries_TargetDeleted_LeavesAnOwnedDeliveryAlone: // TestRecoverPending_TargetDeleted_LeavesAnOwnedDeliveryAlone: the
// the terminal write takes ownership like every other recovery write, so // terminal write takes ownership like every other recovery write, so a
// a delivery the engine still holds is not failed underneath its worker. // delivery the engine still holds is not failed underneath its worker.
func TestRecoverPendingDeliveries_TargetDeleted_LeavesAnOwnedDeliveryAlone( func TestRecoverPending_TargetDeleted_LeavesAnOwnedDeliveryAlone(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
@@ -620,7 +620,7 @@ func TestRecoverPendingDeliveries_TargetDeleted_LeavesAnOwnedDeliveryAlone(
require.True(t, s.Engine.ExportRetainDelivery(deliveryID)) require.True(t, s.Engine.ExportRetainDelivery(deliveryID))
s.Engine.ExportRecoverWebhookDeliveries( s.Engine.ExportRecoverWebhookDeliveries(
context.Background(), s.WebhookID, atStart, context.Background(), s.WebhookID,
) )
iAssertStatus( iAssertStatus(
@@ -675,18 +675,18 @@ func TestFailMissingTarget_LeavesASettledDeliveryAlone(
assert.Zero(t, s.Engine.ExportInflightHeld()) assert.Zero(t, s.Engine.ExportInflightHeld())
} }
// TestRecoverPendingDeliveries_OnTick_TargetDeleted sweeps twice over a // TestSweepPending_TargetDeleted sweeps twice over a batch that also
// batch that also holds a healthy stranded delivery. The one whose // holds a healthy stranded delivery. The one whose target is gone is
// target is gone is failed once and then left alone; the healthy one is // failed once and then left alone; the healthy one is queued by the
// queued by the first sweep and not again by the second. // first sweep and not again by the second.
func TestRecoverPendingDeliveries_OnTick_TargetDeleted(t *testing.T) { func TestSweepPending_TargetDeleted(t *testing.T) {
t.Parallel() t.Parallel()
liveTargetID := uuid.New().String() liveTargetID := uuid.New().String()
s := fSweepSetup(t, liveTargetID, "still-there") s := fSweepSetup(t, liveTargetID, "still-there")
deliveryID := tSeedDeletedTarget( deliveryID := tSeedDeletedTarget(
t, s, "gone-while-pending-on-tick", "http://example.com/hook", t, s, "gone-on-pending-sweep", "http://example.com/hook",
database.DeliveryStatusPending, database.DeliveryStatusPending,
) )
rAgePending(t, s.WebhookDB, deliveryID) rAgePending(t, s.WebhookDB, deliveryID)
@@ -703,7 +703,7 @@ func TestRecoverPendingDeliveries_OnTick_TargetDeleted(t *testing.T) {
ctx := context.Background() ctx := context.Background()
s.Engine.ExportRecoverWebhookDeliveries(ctx, s.WebhookID, onTick) s.Engine.ExportSweepWebhookRetries(ctx, s.WebhookID)
tasks := fDrain(s.Engine) tasks := fDrain(s.Engine)
require.Len(t, tasks, 1, require.Len(t, tasks, 1,
@@ -711,7 +711,7 @@ func TestRecoverPendingDeliveries_OnTick_TargetDeleted(t *testing.T) {
) )
assert.Equal(t, healthy.ID, tasks[0].DeliveryID) assert.Equal(t, healthy.ID, tasks[0].DeliveryID)
s.Engine.ExportRecoverWebhookDeliveries(ctx, s.WebhookID, onTick) s.Engine.ExportSweepWebhookRetries(ctx, s.WebhookID)
assert.Empty(t, fDrain(s.Engine), assert.Empty(t, fDrain(s.Engine),
"the second sweep queued a delivery again", "the second sweep queued a delivery again",
@@ -724,7 +724,7 @@ func TestRecoverPendingDeliveries_OnTick_TargetDeleted(t *testing.T) {
last := tLastResult(t, s, deliveryID, 2) last := tLastResult(t, s, deliveryID, 2)
assert.Contains(t, last.Error, "gone-while-pending-on-tick") assert.Contains(t, last.Error, "gone-on-pending-sweep")
assert.Contains(t, last.Error, "was deleted") assert.Contains(t, last.Error, "was deleted")
} }
@@ -775,11 +775,11 @@ func TestSendRecoveredDeliveries_TargetMissingFromMap(
) )
} }
// TestRecoverPendingDeliveries_TargetUnreadable_LeavesDeliveryAlone: a // TestRecoverPending_TargetUnreadable_LeavesDeliveryAlone: a failed
// failed read of the main database is not a deleted target. At start the // read of the main database is not a deleted target. Restart recovery
// recovery routine holds every pending delivery of the webhook in one // holds every pending delivery of the webhook in one batch, so failing
// batch, so failing on this would fail all of them. // on this would fail all of them.
func TestRecoverPendingDeliveries_TargetUnreadable_LeavesDeliveryAlone( func TestRecoverPending_TargetUnreadable_LeavesDeliveryAlone(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
@@ -807,7 +807,7 @@ func TestRecoverPendingDeliveries_TargetUnreadable_LeavesDeliveryAlone(
require.NoError(t, sqlDB.Close()) require.NoError(t, sqlDB.Close())
s.Engine.ExportRecoverPendingDeliveries( s.Engine.ExportRecoverPendingDeliveries(
context.Background(), s.WebhookDB, s.WebhookID, atStart, context.Background(), s.WebhookDB, s.WebhookID,
) )
iAssertStatus( iAssertStatus(
internal/delivery/url_mask_test.go
+16 -4
View File
@@ -2,6 +2,7 @@ package delivery_test
import ( import (
"context" "context"
"encoding/json"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"testing" "testing"
@@ -70,6 +71,13 @@ func deliverSlackTo(
e := testEngine(t, 1) e := testEngine(t, 1)
targetID := uuid.New().String() targetID := uuid.New().String()
slackCfg, err := json.Marshal(
delivery.SlackTargetConfig{
WebhookURL: webhookURL,
},
)
require.NoError(t, err)
event := seedEvent(t, db, `{"test":true}`) event := seedEvent(t, db, `{"test":true}`)
dlv := seedDelivery( dlv := seedDelivery(
@@ -79,7 +87,7 @@ func deliverSlackTo(
d := buildSlackDelivery( d := buildSlackDelivery(
dlv, event, targetID, dlv, event, targetID,
"test-slack-mask", webhookURL, "test-slack-mask", string(slackCfg),
) )
e.ExportDeliverSlack(context.TODO(), db, d) e.ExportDeliverSlack(context.TODO(), db, d)
@@ -152,9 +160,13 @@ func TestDoHTTPRequest_TransportErrorMasksURL(t *testing.T) {
e := testEngine(t, 1) e := testEngine(t, 1)
cfg, err := e.ExportParseHTTPConfig(
newHTTPTargetConfig(host + maskSecretPath),
)
require.NoError(t, err)
statusCode, _, _, reqErr := e.ExportDoHTTPRequest( statusCode, _, _, reqErr := e.ExportDoHTTPRequest(
context.TODO(), context.TODO(), cfg,
&database.Target{URL: host + maskSecretPath},
&database.Event{Body: `{"test":true}`}, &database.Event{Body: `{"test":true}`},
) )
require.Error(t, reqErr) require.Error(t, reqErr)
@@ -176,7 +188,7 @@ func TestDoHTTPRequest_UnparsableURLIsMasked(t *testing.T) {
statusCode, _, _, reqErr := e.ExportDoHTTPRequest( statusCode, _, _, reqErr := e.ExportDoHTTPRequest(
context.TODO(), context.TODO(),
&database.Target{ &delivery.HTTPTargetConfig{
URL: "https://hooks.example.com" + maskSecretPath + "\n", URL: "https://hooks.example.com" + maskSecretPath + "\n",
}, },
&database.Event{}, &database.Event{},
internal/delivery/user_agent_test.go
+9 -3
View File
@@ -2,6 +2,7 @@ package delivery_test
import ( import (
"context" "context"
"encoding/json"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"net/netip" "net/netip"
@@ -58,9 +59,9 @@ func TestUserAgent_IsTheBuildVersion(t *testing.T) {
statusCode, _, _, err := e.ExportDoHTTPRequest( statusCode, _, _, err := e.ExportDoHTTPRequest(
context.Background(), context.Background(),
&database.Target{ &delivery.HTTPTargetConfig{
URL: ts.URL, URL: ts.URL,
Headers: map[string]string{headerUserAgent: "configured/1"}, Headers: map[string]string{"User-Agent": "configured/1"},
}, },
&database.Event{Headers: `{"User-Agent":["curl/8"]}`}, &database.Event{Headers: `{"User-Agent":["curl/8"]}`},
) )
@@ -72,13 +73,18 @@ func TestUserAgent_IsTheBuildVersion(t *testing.T) {
db := testWebhookDB(t) db := testWebhookDB(t)
targetID := uuid.New().String() targetID := uuid.New().String()
slackCfg, err := json.Marshal(
delivery.SlackTargetConfig{WebhookURL: ts.URL},
)
require.NoError(t, err)
event := seedEvent(t, db, `{"action":"test"}`) event := seedEvent(t, db, `{"action":"test"}`)
dlv := seedDelivery( dlv := seedDelivery(
t, db, event.ID, targetID, database.DeliveryStatusPending, t, db, event.ID, targetID, database.DeliveryStatusPending,
) )
e.ExportDeliverSlack(context.Background(), db, buildSlackDelivery( e.ExportDeliverSlack(context.Background(), db, buildSlackDelivery(
dlv, event, targetID, "test-slack", ts.URL, dlv, event, targetID, "test-slack", string(slackCfg),
)) ))
require.Len(t, userAgents, 1, "the slack target sent no request") require.Len(t, userAgents, 1, "the slack target sent no request")
assert.Equal(t, want, <-userAgents, "slack target") assert.Equal(t, want, <-userAgents, "slack target")
internal/handlers/archive_expiry.go
+1 -23
View File
@@ -1,14 +1,8 @@
package handlers package handlers
import (
"strconv"
"time"
)
const ( const (
// archiveExpiryNever is the archive expiry that keeps archived // archiveExpiryNever is the archive expiry that keeps archived
// events forever. It is stored as an ArchiveExpiry of 0, and an // events forever. A stored empty expiry means the same.
// empty expiry on a form means the same.
archiveExpiryNever = "never" archiveExpiryNever = "never"
// tmplKeyArchiveExpiryChoices is the template data key for the // tmplKeyArchiveExpiryChoices is the template data key for the
@@ -62,19 +56,3 @@ func archiveExpiryOptions(expiry string) []archiveChoice {
return append([]archiveChoice{own}, options...) return append([]archiveChoice{own}, options...)
} }
// formatArchiveExpiry writes a stored ArchiveExpiry the way the
// expiry select spells its values, so the target edit form starts on
// the stored choice: never for 0, a whole number of hours as "720h",
// and anything else as Go writes a duration, which the form accepts
// back unchanged.
func formatArchiveExpiry(expiry time.Duration) string {
switch {
case expiry == 0:
return archiveExpiryNever
case expiry%time.Hour == 0:
return strconv.FormatInt(int64(expiry/time.Hour), 10) + "h"
default:
return expiry.String()
}
}
internal/handlers/archive_expiry_test.go
+16 -20
View File
@@ -7,7 +7,6 @@ import (
"regexp" "regexp"
"strings" "strings"
"testing" "testing"
"time"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
@@ -95,18 +94,14 @@ func TestArchiveExpiryChoices(t *testing.T) {
env := setupSourceTest(t) env := setupSourceTest(t)
choices := []struct { choices := []struct{ value, shown string }{
value string {expiryNever, expiryNever},
stored time.Duration {"1h", "1 hour"},
shown string {"12h", "12 hours"},
}{ {"24h", "1 day"},
{expiryNever, 0, expiryNever}, {"720h", "30 days"},
{"1h", time.Hour, "1 hour"}, {"2160h", "90 days"},
{"12h", 12 * time.Hour, "12 hours"}, {"8760h", "365 days"},
{"24h", 24 * time.Hour, "1 day"},
{"720h", 720 * time.Hour, "30 days"},
{"2160h", 2160 * time.Hour, "90 days"},
{"8760h", 8760 * time.Hour, "365 days"},
} }
for _, choice := range choices { for _, choice := range choices {
@@ -127,7 +122,9 @@ func TestArchiveExpiryChoices(t *testing.T) {
targets := targetsForWebhook(t, env.db, webhook.ID) targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1) require.Len(t, targets, 1)
assert.Equal(t, choice.stored, targets[0].ArchiveExpiry) assert.JSONEq(
t, `{"expiry":"`+choice.value+`"}`, targets[0].Config,
)
assert.Equal( assert.Equal(
t, []string{choice.shown}, t, []string{choice.shown},
@@ -141,7 +138,7 @@ func TestArchiveExpiryChoices(t *testing.T) {
} }
// TestArchiveExpiryEditStartsOnStoredValue checks the edit form of a // TestArchiveExpiryEditStartsOnStoredValue checks the edit form of a
// database target whose stored expiry is 0, which selects never, // database target whose stored expiry is empty, which selects never,
// and of one whose expiry is not one of the choices, which is listed // and of one whose expiry is not one of the choices, which is listed
// first as its own selected entry and saved unchanged. // first as its own selected entry and saved unchanged.
func TestArchiveExpiryEditStartsOnStoredValue(t *testing.T) { func TestArchiveExpiryEditStartsOnStoredValue(t *testing.T) {
@@ -151,8 +148,7 @@ func TestArchiveExpiryEditStartsOnStoredValue(t *testing.T) {
webhook := seedWebhookWithRetention(t, env.db, 30) webhook := seedWebhookWithRetention(t, env.db, 30)
empty := seedConfiguredTarget( empty := seedConfiguredTarget(
t, env.db, webhook.ID, database.TargetTypeDatabase, t, env.db, webhook.ID, database.TargetTypeDatabase, "",
database.Target{},
) )
_, selected := expirySelected(t, env, webhook.ID, empty.ID) _, selected := expirySelected(t, env, webhook.ID, empty.ID)
@@ -161,7 +157,7 @@ func TestArchiveExpiryEditStartsOnStoredValue(t *testing.T) {
webhook = seedWebhookWithRetention(t, env.db, 30) webhook = seedWebhookWithRetention(t, env.db, 30)
unlisted := seedConfiguredTarget( unlisted := seedConfiguredTarget(
t, env.db, webhook.ID, database.TargetTypeDatabase, t, env.db, webhook.ID, database.TargetTypeDatabase,
database.Target{ArchiveExpiry: 36 * time.Hour}, `{"expiry":"36h"}`,
) )
assert.Equal(t, []string{"36 hours"}, expiryShown(t, env, webhook.ID)) assert.Equal(t, []string{"36 hours"}, expiryShown(t, env, webhook.ID))
@@ -183,7 +179,7 @@ func TestArchiveExpiryEditStartsOnStoredValue(t *testing.T) {
w := submitTargetEdit(env, webhook.ID, unlisted.ID, form) w := submitTargetEdit(env, webhook.ID, unlisted.ID, form)
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String()) require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
assert.Equal( assert.JSONEq(
t, 36*time.Hour, storedTarget(t, env, unlisted.ID).ArchiveExpiry, t, `{"expiry":"36h"}`, storedTarget(t, env, unlisted.ID).Config,
) )
} }
internal/handlers/archive_rotation_test.go
+23 -37
View File
@@ -16,12 +16,8 @@ import (
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
) )
// The archive rotations these tests store: none keeps one file. // rotationNone is the archive rotation that keeps one file.
const ( const rotationNone = "none"
rotationNone = "none"
rotationDaily = "daily"
rotationHourly = "hourly"
)
// rotationShown returns the archive rotations the webhook page's // rotationShown returns the archive rotations the webhook page's
// target list shows. // target list shows.
@@ -45,9 +41,7 @@ func TestArchiveRotationChoices(t *testing.T) {
env := setupSourceTest(t) env := setupSourceTest(t)
for _, rotation := range []string{ for _, rotation := range []string{rotationNone, "monthly", "daily", "hourly"} {
rotationNone, "monthly", rotationDaily, rotationHourly,
} {
t.Run(rotation, func(t *testing.T) { t.Run(rotation, func(t *testing.T) {
t.Parallel() t.Parallel()
@@ -66,8 +60,10 @@ func TestArchiveRotationChoices(t *testing.T) {
targets := targetsForWebhook(t, env.db, webhook.ID) targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1) require.Len(t, targets, 1)
assert.Equal(t, 720*time.Hour, targets[0].ArchiveExpiry) assert.JSONEq(t,
assert.Equal(t, rotation, targets[0].ArchiveRotation) `{"expiry":"720h","rotation":"`+rotation+`"}`,
targets[0].Config,
)
assert.Equal(t, assert.Equal(t,
[]string{rotation}, rotationShown(t, env, webhook.ID)) []string{rotation}, rotationShown(t, env, webhook.ID))
@@ -82,37 +78,26 @@ func TestArchiveRotationChoices(t *testing.T) {
w = submitTargetEdit(env, webhook.ID, targets[0].ID, form) w = submitTargetEdit(env, webhook.ID, targets[0].ID, form)
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String()) require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
assert.JSONEq(t,
stored := storedTarget(t, env, targets[0].ID) `{"expiry":"720h","rotation":"hourly"}`,
assert.Equal(t, 720*time.Hour, stored.ArchiveExpiry) storedTarget(t, env, targets[0].ID).Config,
assert.Equal(t, rotationHourly, stored.ArchiveRotation) )
}) })
} }
} }
// TestArchiveRotationEditStartsOnNone checks that a database target // TestArchiveRotationEditStartsOnNone checks the edit form of a
// added with no rotation is stored with none and shown with none, and // database target with no rotation stored starts on none.
// that its edit form starts on none.
func TestArchiveRotationEditStartsOnNone(t *testing.T) { func TestArchiveRotationEditStartsOnNone(t *testing.T) {
t.Parallel() t.Parallel()
env := setupSourceTest(t) env := setupSourceTest(t)
webhook := seedWebhookWithRetention(t, env.db, 30) webhook := seedWebhookWithRetention(t, env.db, 30)
target := seedConfiguredTarget(
form := url.Values{} t, env.db, webhook.ID, database.TargetTypeDatabase, "",
form.Set("name", "archive")
form.Set("type", string(database.TargetTypeDatabase))
w := serveTarget(
env, http.MethodPost, "/hook/"+webhook.ID+"/targets", form,
) )
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
targets := targetsForWebhook(t, env.db, webhook.ID) page := targetEditPage(t, env, webhook.ID, target.ID)
require.Len(t, targets, 1)
assert.Equal(t, rotationNone, targets[0].ArchiveRotation)
page := targetEditPage(t, env, webhook.ID, targets[0].ID)
assert.Equal(t, []string{rotationNone}, selectedIn(page, "rotation")) assert.Equal(t, []string{rotationNone}, selectedIn(page, "rotation"))
assert.Equal(t, []string{rotationNone}, rotationShown(t, env, webhook.ID)) assert.Equal(t, []string{rotationNone}, rotationShown(t, env, webhook.ID))
} }
@@ -140,7 +125,7 @@ func TestArchiveRotationRefused(t *testing.T) {
target := seedConfiguredTarget( target := seedConfiguredTarget(
t, env.db, webhook.ID, database.TargetTypeDatabase, t, env.db, webhook.ID, database.TargetTypeDatabase,
database.Target{ArchiveRotation: rotationDaily}, `{"rotation":"daily"}`,
) )
form.Del("type") form.Del("type")
@@ -148,8 +133,8 @@ func TestArchiveRotationRefused(t *testing.T) {
w = submitTargetEdit(env, webhook.ID, target.ID, form) w = submitTargetEdit(env, webhook.ID, target.ID, form)
assert.Equal(t, http.StatusBadRequest, w.Code) assert.Equal(t, http.StatusBadRequest, w.Code)
assert.Contains(t, w.Body.String(), "Invalid archive rotation") assert.Contains(t, w.Body.String(), "Invalid archive rotation")
assert.Equal( assert.JSONEq(t,
t, rotationDaily, storedTarget(t, env, target.ID).ArchiveRotation, `{"rotation":"daily"}`, storedTarget(t, env, target.ID).Config,
) )
} }
@@ -176,8 +161,9 @@ func TestHandleSourceCreateSubmit_ArchiveRotation(t *testing.T) {
targets := targetsForWebhook(t, env.db, webhook.ID) targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1) require.Len(t, targets, 1)
assert.Equal(t, 720*time.Hour, targets[0].ArchiveExpiry) assert.JSONEq(t,
assert.Equal(t, rotationDaily, targets[0].ArchiveRotation) `{"expiry":"720h","rotation":"daily"}`, targets[0].Config,
)
} }
// TestArchiveFileView_Rotated describes a daily target's archive files // TestArchiveFileView_Rotated describes a daily target's archive files
@@ -192,7 +178,7 @@ func TestArchiveFileView_Rotated(t *testing.T) {
webhook := seedWebhookWithRetention(t, env.db, 30) webhook := seedWebhookWithRetention(t, env.db, 30)
target := seedConfiguredTarget( target := seedConfiguredTarget(
t, env.db, webhook.ID, database.TargetTypeDatabase, t, env.db, webhook.ID, database.TargetTypeDatabase,
database.Target{ArchiveRotation: rotationDaily}, `{"rotation":"daily"}`,
) )
path := archive.Path(env.dbMgr, &webhook, target) path := archive.Path(env.dbMgr, &webhook, target)
internal/handlers/delivery_replay.go
+90 -12
View File
@@ -5,7 +5,9 @@ import (
"github.com/go-chi/chi" "github.com/go-chi/chi"
"gorm.io/gorm" "gorm.io/gorm"
"gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
) )
// The outcomes of a replay POST, as the notice codes its redirect // The outcomes of a replay POST, as the notice codes its redirect
@@ -44,12 +46,12 @@ const (
// HandleDeliveryReplay re-sends a finished delivery's event to its // HandleDeliveryReplay re-sends a finished delivery's event to its
// target. // target.
// //
// A replay never touches the delivery it repeats. The delivery engine // A replay never touches the delivery it repeats. It creates a NEW
// creates a NEW pending delivery row for the same event and target and // pending delivery row for the same event and target and hands it to
// queues it as it queues the receiver's, so the original's status, // the delivery engine through the same Notifier the receiver uses, so
// attempts and timestamps stand as the record of what actually // the original's status, attempts and timestamps stand as the record
// happened, and the replay is retried, SSRF-guarded and circuit-broken // of what actually happened, and the replay is retried, SSRF-guarded
// exactly as a first attempt is. // and circuit-broken exactly as a first attempt is.
// //
// What is re-sent is the stored EVENT body, never the response the // What is re-sent is the stored EVENT body, never the response the
// original delivery received. // original delivery received.
@@ -143,8 +145,7 @@ func (h *Handlers) loadReplaySource(
return &original, true return &original, true
} }
// queueReplay has the delivery engine create and queue the new // queueReplay writes the new delivery and hands it to the engine.
// delivery, unless an earlier replay is still in flight.
func (h *Handlers) queueReplay( func (h *Handlers) queueReplay(
w http.ResponseWriter, w http.ResponseWriter,
r *http.Request, r *http.Request,
@@ -170,8 +171,18 @@ func (h *Handlers) queueReplay(
return return
} }
deliveryID, err := h.engine.CreateReplay( var event database.Event
webhook.ID, original.EventID, target,
err = webhookDB.
First(&event, "id = ?", original.EventID).Error
if err != nil {
h.serverError(w, r, "failed to load event for replay", err)
return
}
task, err := createReplayDelivery(
webhookDB, webhook.ID, &event, target,
) )
if err != nil { if err != nil {
h.serverError( h.serverError(
@@ -182,14 +193,15 @@ func (h *Handlers) queueReplay(
} }
h.mtr.DeliveryReplayed(target.Type) h.mtr.DeliveryReplayed(target.Type)
h.notifier.Notify([]delivery.Task{task})
h.log.Info( h.log.Info(
"delivery replay queued", "delivery replay queued",
"webhook_id", webhook.ID, "webhook_id", webhook.ID,
"event_id", original.EventID, "event_id", event.ID,
"target_id", target.ID, "target_id", target.ID,
"replayed_delivery_id", original.ID, "replayed_delivery_id", original.ID,
"delivery_id", deliveryID, "delivery_id", task.DeliveryID,
) )
redirectToEventLog(w, r, webhook, replayQueued) redirectToEventLog(w, r, webhook, replayQueued)
@@ -251,6 +263,72 @@ func countInFlightDeliveries(
return count, err return count, err
} }
// createReplayDelivery writes the new pending delivery row, adds it to
// its target's totals in the same transaction, and returns the task
// that carries it to the delivery engine.
//
// The row is written with associations omitted, and neither Event nor
// Target is populated on it: GORM's SaveBeforeAssociations would
// otherwise upsert the whole target row — plaintext config, which for a
// Slack target is the credential — into the per-webhook event database.
// See https://git.eeqj.de/sneak/webhooker/issues/206.
func createReplayDelivery(
webhookDB *gorm.DB,
webhookID string,
event *database.Event,
target *database.Target,
) (delivery.Task, error) {
dlv := &database.Delivery{
EventID: event.ID,
TargetID: target.ID,
Status: database.DeliveryStatusPending,
Replay: true,
}
err := webhookDB.Transaction(func(tx *gorm.DB) error {
err := tx.Omit(clause.Associations).Create(dlv).Error
if err != nil {
return err
}
return database.AddTargetTotals(tx, database.TargetTotals{
TargetID: dlv.TargetID, Deliveries: 1,
})
})
if err != nil {
return delivery.Task{}, err
}
return delivery.Task{
DeliveryID: dlv.ID,
EventID: event.ID,
WebhookID: webhookID,
EntrypointID: event.EntrypointID,
TargetID: target.ID,
TargetName: target.Name,
TargetType: target.Type,
TargetConfig: target.Config,
MaxAttempts: target.MaxAttempts,
Method: event.Method,
RawQuery: event.RawQuery,
Headers: event.Headers,
ContentType: event.ContentType,
Body: replayBody(event.Body),
AttemptNum: 1,
}, nil
}
// replayBody returns the stored event body for a replay task to carry
// inline, or nil when it is large enough that the engine should fetch
// it from the per-webhook database instead.
func replayBody(body string) *string {
if len(body) >= delivery.MaxInlineBodySize {
return nil
}
return &body
}
// redirectToEventLog redirects a replay or resubmit back to the event // redirectToEventLog redirects a replay or resubmit back to the event
// log it was triggered from, carrying the outcome as its notice. A // log it was triggered from, carrying the outcome as its notice. A
// Replay form carries the list it was pressed in as show, so a replay // Replay form carries the list it was pressed in as show, so a replay
internal/handlers/delivery_replay_test.go
+85 -54
View File
@@ -12,6 +12,7 @@ import (
"gorm.io/gorm" "gorm.io/gorm"
"gorm.io/gorm/clause" "gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/handlers" "sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/session" "sneak.berlin/go/webhooker/internal/session"
) )
@@ -22,7 +23,7 @@ const paramDeliveryID = "deliveryID"
// replayTargetURL is a public destination, so a target configured with // replayTargetURL is a public destination, so a target configured with
// it is one the SSRF guard would accept. Nothing in these tests // it is one the SSRF guard would accept. Nothing in these tests
// dispatches to it: the delivery engine is never started. // dispatches to it: the notifier is recorded, not run.
const replayTargetURL = "http://93.184.216.34/hook" const replayTargetURL = "http://93.184.216.34/hook"
// replayEventQuery is the query string a seeded event arrived with. // replayEventQuery is the query string a seeded event arrived with.
@@ -162,8 +163,14 @@ func postReplay(
// TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal is the // TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal is the
// core requirement: replaying a failed delivery succeeds, appends a // core requirement: replaying a failed delivery succeeds, appends a
// new delivery marked as a replay, and leaves the original row and its // new delivery, and leaves the original row and its recorded attempt
// recorded attempt exactly as they were. // exactly as they were.
//
// It also pins the two things a replay would be wrong to get from the
// original: the task carries the target's CURRENT configuration, which
// this test changes between the failure and the replay, and it carries
// the stored EVENT body rather than anything the failed attempt
// received back.
func TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal( func TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal(
t *testing.T, t *testing.T,
) { ) {
@@ -174,9 +181,10 @@ func TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal(
sess *session.Session sess *session.Session
db *database.Database db *database.Database
dbMgr *database.WebhookDBManager dbMgr *database.WebhookDBManager
notif *recordingNotifier
) )
app := newTestApp(t, &h, &sess, &db, &dbMgr) app := newTestApp(t, &h, &sess, &db, &dbMgr, &notif)
app.RequireStart() app.RequireStart()
t.Cleanup(app.RequireStop) t.Cleanup(app.RequireStop)
@@ -184,7 +192,7 @@ func TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal(
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
database.Target{URL: replayTargetURL}, `{"url":"`+replayTargetURL+`"}`,
) )
event, original := seedFailedDelivery( event, original := seedFailedDelivery(
@@ -196,6 +204,15 @@ func TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal(
before := loadDelivery(t, webhookDB, original.ID) before := loadDelivery(t, webhookDB, original.ID)
// The operator fixes the destination, which is the whole reason
// to replay. The replay must use this, not the config the
// original delivery ran against.
const fixedConfig = `{"url":"http://93.184.216.34/fixed"}`
require.NoError(t, db.DB().Model(&database.Target{}).
Where("id = ?", tgt.ID).
Update("config", fixedConfig).Error)
w := postReplay(t, h, sess, wh.ID, original.ID) w := postReplay(t, h, sess, wh.ID, original.ID)
require.Equal(t, http.StatusSeeOther, w.Code) require.Equal(t, http.StatusSeeOther, w.Code)
@@ -217,9 +234,14 @@ func TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal(
assert.Equal( assert.Equal(
t, database.DeliveryStatusPending, replayed.Status, t, database.DeliveryStatusPending, replayed.Status,
) )
assert.True(t, replayed.Replay)
assertDeliveryUntouched(t, webhookDB, before) assertDeliveryUntouched(t, webhookDB, before)
tasks := notif.Tasks()
require.Len(t, tasks, 1)
assertReplayTask(
t, tasks[0], wh.ID, event, tgt, replayed.ID, fixedConfig,
)
assertNoLeakedTarget(t, webhookDB) assertNoLeakedTarget(t, webhookDB)
} }
@@ -253,6 +275,45 @@ func assertDeliveryUntouched(
) )
} }
// assertReplayTask proves the task handed to the delivery engine is
// the one the receiver would build for this event and this target, and
// that it carries wantConfig — the target's configuration as it stands
// now rather than as the original delivery ran against it.
func assertReplayTask(
t *testing.T,
task delivery.Task,
webhookID string,
event *database.Event,
target *database.Target,
wantDeliveryID, wantConfig string,
) {
t.Helper()
assert.Equal(t, wantDeliveryID, task.DeliveryID)
assert.Equal(t, event.ID, task.EventID)
assert.Equal(t, webhookID, task.WebhookID)
assert.Equal(t, event.EntrypointID, task.EntrypointID)
assert.Equal(t, target.ID, task.TargetID)
assert.Equal(t, target.Type, task.TargetType)
assert.JSONEq(
t, wantConfig, task.TargetConfig,
"replay must use the target's current configuration",
)
assert.Equal(t, event.Method, task.Method)
assert.Equal(
t, replayEventQuery, task.RawQuery,
"replay re-sends the stored query string",
)
assert.Equal(t, event.Headers, task.Headers)
assert.Equal(t, event.ContentType, task.ContentType)
assert.Equal(t, 1, task.AttemptNum)
require.NotNil(t, task.Body)
assert.Equal(
t, event.Body, *task.Body,
"replay re-sends the stored event body",
)
}
// assertNoLeakedTarget proves the per-webhook database holds no // assertNoLeakedTarget proves the per-webhook database holds no
// target: an association write would have upserted the whole target, // target: an association write would have upserted the whole target,
// plaintext config and all, into the event database. See // plaintext config and all, into the event database. See
@@ -266,49 +327,9 @@ func assertNoLeakedTarget(t *testing.T, webhookDB *gorm.DB) {
) )
} }
// TestHandleDeliveryReplay_FailedStoreIsAnError proves a replay the
// engine cannot store answers with the 500 error page and adds no
// delivery.
func TestHandleDeliveryReplay_FailedStoreIsAnError(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP,
database.Target{URL: replayTargetURL},
)
event, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
// Without its target totals table the webhook's database refuses
// the totals write, which comes after the delivery row.
webhookDB, err := dbMgr.GetDB(wh.ID)
require.NoError(t, err)
require.NoError(t,
webhookDB.Migrator().DropTable(&database.TargetTotals{}))
w := postReplay(t, h, sess, wh.ID, original.ID)
assert.Equal(t, http.StatusInternalServerError, w.Code)
assert.Contains(t, w.Body.String(), "500 Internal Server Error")
assert.Len(t, listDeliveries(t, webhookDB, event.ID), 1)
}
// TestHandleDeliveryReplay_RefusesDeletedTarget proves the required // TestHandleDeliveryReplay_RefusesDeletedTarget proves the required
// refusal: a target deleted since the delivery ran is reported as // refusal: a target deleted since the delivery ran is reported as
// deleted rather than erroring, and nothing is created. // deleted rather than erroring, and nothing is created or queued.
func TestHandleDeliveryReplay_RefusesDeletedTarget(t *testing.T) { func TestHandleDeliveryReplay_RefusesDeletedTarget(t *testing.T) {
t.Parallel() t.Parallel()
@@ -317,9 +338,10 @@ func TestHandleDeliveryReplay_RefusesDeletedTarget(t *testing.T) {
sess *session.Session sess *session.Session
db *database.Database db *database.Database
dbMgr *database.WebhookDBManager dbMgr *database.WebhookDBManager
notif *recordingNotifier
) )
app := newTestApp(t, &h, &sess, &db, &dbMgr) app := newTestApp(t, &h, &sess, &db, &dbMgr, &notif)
app.RequireStart() app.RequireStart()
t.Cleanup(app.RequireStop) t.Cleanup(app.RequireStop)
@@ -327,7 +349,7 @@ func TestHandleDeliveryReplay_RefusesDeletedTarget(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
database.Target{URL: replayTargetURL}, `{"url":"`+replayTargetURL+`"}`,
) )
event, original := seedFailedDelivery( event, original := seedFailedDelivery(
@@ -355,6 +377,10 @@ func TestHandleDeliveryReplay_RefusesDeletedTarget(t *testing.T) {
t, listDeliveries(t, webhookDB, event.ID), 1, t, listDeliveries(t, webhookDB, event.ID), 1,
"a refused replay must create no delivery", "a refused replay must create no delivery",
) )
assert.Empty(
t, notif.Tasks(),
"a refused replay must queue nothing",
)
// The refusal is specific, which is why the target is looked up // The refusal is specific, which is why the target is looked up
// including soft-deleted rows: an id that never named a target // including soft-deleted rows: an id that never named a target
@@ -387,9 +413,10 @@ func TestHandleDeliveryReplay_RefusesWhileEarlierReplayInFlight(
sess *session.Session sess *session.Session
db *database.Database db *database.Database
dbMgr *database.WebhookDBManager dbMgr *database.WebhookDBManager
notif *recordingNotifier
) )
app := newTestApp(t, &h, &sess, &db, &dbMgr) app := newTestApp(t, &h, &sess, &db, &dbMgr, &notif)
app.RequireStart() app.RequireStart()
t.Cleanup(app.RequireStop) t.Cleanup(app.RequireStop)
@@ -397,7 +424,7 @@ func TestHandleDeliveryReplay_RefusesWhileEarlierReplayInFlight(
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
database.Target{URL: replayTargetURL}, `{"url":"`+replayTargetURL+`"}`,
) )
event, original := seedFailedDelivery( event, original := seedFailedDelivery(
@@ -427,6 +454,10 @@ func TestHandleDeliveryReplay_RefusesWhileEarlierReplayInFlight(
t, listDeliveries(t, webhookDB, event.ID), 2, t, listDeliveries(t, webhookDB, event.ID), 2,
"the refused second replay must add nothing", "the refused second replay must add nothing",
) )
assert.Len(
t, notif.Tasks(), 1,
"only the first replay reaches the delivery engine",
)
// A delivery the engine has not finished is not replayable // A delivery the engine has not finished is not replayable
// either, which is the same rule seen from the other side. // either, which is the same rule seen from the other side.
@@ -467,7 +498,7 @@ func TestHandleDeliveryReplay_ReturnsToTheListItWasPressedIn(
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
database.Target{URL: replayTargetURL}, `{"url":"`+replayTargetURL+`"}`,
) )
_, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID) _, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
@@ -525,7 +556,7 @@ func TestHandleSourceLogs_RendersReplayControlAndBanner(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
database.Target{URL: replayTargetURL}, `{"url":"`+replayTargetURL+`"}`,
) )
_, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID) _, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
@@ -584,7 +615,7 @@ func TestHandleDeliveryReplay_LabelsTheReplay(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
database.Target{URL: replayTargetURL}, `{"url":"`+replayTargetURL+`"}`,
) )
event, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID) event, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
internal/handlers/delivery_result_view_test.go
+12 -12
View File
@@ -86,12 +86,12 @@ func seedFailedDeliveryWithResponse(
} }
// seedFailureAndRender seeds a failed delivery against a // seedFailureAndRender seeds a failed delivery against a
// target of the given type and URL, and returns the // target of the given type and config, and returns the
// rendered event log page. // rendered event log page.
func seedFailureAndRender( func seedFailureAndRender(
t *testing.T, t *testing.T,
targetType database.TargetType, targetType database.TargetType,
targetURL, responseBody string, config, responseBody string,
) string { ) string {
t.Helper() t.Helper()
@@ -109,7 +109,7 @@ func seedFailureAndRender(
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, targetType, database.Target{URL: targetURL}, t, db, wh.ID, targetType, config,
) )
seedFailedDeliveryWithResponse( seedFailedDeliveryWithResponse(
@@ -129,7 +129,7 @@ func TestHandleSourceLogs_RendersFailedAttempt(t *testing.T) {
body := seedFailureAndRender( body := seedFailureAndRender(
t, t,
database.TargetTypeHTTP, database.TargetTypeHTTP,
"https://example.com/hook/abc", `{"url":"https://example.com/hook/abc"}`,
"upstream exploded", "upstream exploded",
) )
@@ -167,7 +167,7 @@ func TestHandleSourceLogs_EscapesResponseBody(t *testing.T) {
body := seedFailureAndRender( body := seedFailureAndRender(
t, t,
database.TargetTypeHTTP, database.TargetTypeHTTP,
"https://example.com/hook/abc", `{"url":"https://example.com/hook/abc"}`,
payload, payload,
) )
@@ -189,7 +189,7 @@ func TestHandleSourceLogs_RedactsCredentialEchoedInResponse(
body := seedFailureAndRender( body := seedFailureAndRender(
t, t,
database.TargetTypeSlack, database.TargetTypeSlack,
slackWebhookURL, `{"webhookUrl":"`+slackWebhookURL+`"}`,
"no_service: "+slackWebhookURL, "no_service: "+slackWebhookURL,
) )
@@ -228,7 +228,7 @@ func TestHandleSourceLogs_RedactsCredentialEchoedInError(
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, t, db, wh.ID,
database.TargetTypeSlack, database.TargetTypeSlack,
database.Target{URL: slackWebhookURL}, `{"webhookUrl":"`+slackWebhookURL+`"}`,
) )
dlv := seedFailedDeliveryWithResponse(t, dbMgr, wh.ID, tgt.ID, "") dlv := seedFailedDeliveryWithResponse(t, dbMgr, wh.ID, tgt.ID, "")
@@ -300,7 +300,7 @@ func TestHandleSourceLogs_RedactsCredentialSeveredByTheEngineCut(
body := seedFailureAndRender( body := seedFailureAndRender(
t, t,
database.TargetTypeSlack, database.TargetTypeSlack,
slackWebhookURL, `{"webhookUrl":"`+slackWebhookURL+`"}`,
stored, stored,
) )
@@ -335,7 +335,7 @@ func TestHandleSourceLogs_RedactsCredentialSeveredBySQLCut(
body := seedFailureAndRender( body := seedFailureAndRender(
t, t,
database.TargetTypeSlack, database.TargetTypeSlack,
slackWebhookURL, `{"webhookUrl":"`+slackWebhookURL+`"}`,
stored, stored,
) )
@@ -373,7 +373,7 @@ func TestHandleSourceLogs_RedactsForSoftDeletedTarget(
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, t, db, wh.ID,
database.TargetTypeSlack, database.TargetTypeSlack,
database.Target{URL: slackWebhookURL}, `{"webhookUrl":"`+slackWebhookURL+`"}`,
) )
seedFailedDeliveryWithResponse( seedFailedDeliveryWithResponse(
@@ -413,7 +413,7 @@ func TestHandleSourceLogs_BoundsRenderedAttempts(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeLog, database.Target{}, t, db, wh.ID, database.TargetTypeLog, "",
) )
dlv := seedFailedDeliveryWithResponse(t, dbMgr, wh.ID, tgt.ID, "") dlv := seedFailedDeliveryWithResponse(t, dbMgr, wh.ID, tgt.ID, "")
@@ -480,7 +480,7 @@ func TestHandleSourceLogs_BoundsOversizeResponse(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeLog, database.Target{}, t, db, wh.ID, database.TargetTypeLog, "",
) )
stored := strings.Repeat("A", responseCap*4) + tail stored := strings.Repeat("A", responseCap*4) + tail
internal/handlers/event_log.go
+3 -3
View File
@@ -13,8 +13,8 @@ import (
// DeliveryView is the display-safe projection of a delivery // DeliveryView is the display-safe projection of a delivery
// for the event log page. Its target is a TargetView, so the // for the event log page. Its target is a TargetView, so the
// target's URL and headers — which hold the target's // stored configuration blob — which holds the target's
// credential — have no path to the template. // credential — has no path to the template.
type DeliveryView struct { type DeliveryView struct {
ID string ID string
Status database.DeliveryStatus Status database.DeliveryStatus
@@ -126,7 +126,7 @@ func (h *Handlers) HandleSourceLogs() http.HandlerFunc {
// loadTargetMap loads targets into a map of display-safe // loadTargetMap loads targets into a map of display-safe
// views keyed by target ID, each paired with its redactor. // views keyed by target ID, each paired with its redactor.
// The projection happens here so that no caller can hand a // The projection happens here so that no caller can hand a
// raw target, URL and headers and all, to a template: the // raw target, configuration blob and all, to a template: the
// raw rows do not leave this function. // raw rows do not leave this function.
// //
// The load is Unscoped because deleting a target only soft // The load is Unscoped because deleting a target only soft
internal/handlers/event_resubmit.go
+25 -17
View File
@@ -171,29 +171,37 @@ func loadResubmitSource(
return src, true, nil return src, true, nil
} }
// queueResubmit has the delivery engine store the copy, with a delivery // queueResubmit stores the copy and fans it out to the webhook's
// for each of the webhook's active targets as they stand now, through // active targets.
// the same StoreEvent the receiver uses. The one difference is that the
// copy leaves its entrypoint's last event as it is, since it did not
// arrive on that URL.
func (h *Handlers) queueResubmit( func (h *Handlers) queueResubmit(
w http.ResponseWriter, w http.ResponseWriter,
r *http.Request, r *http.Request,
webhook database.Webhook, webhook database.Webhook,
src resubmitSource, src resubmitSource,
) { ) {
event := &database.Event{ // The receiver's own query, run now: an active target created
WebhookID: webhook.ID, // after the original event arrived is included, and an
EntrypointID: src.EntrypointID, // inactive one is skipped rather than refused.
Method: src.Method, targets, err := h.loadActiveTargets(webhook.ID)
RawQuery: src.RawQuery, if err != nil {
Headers: src.Headers, h.serverError(w, r, "failed to query targets", err)
ContentType: src.ContentType,
Body: string(src.Body), return
ResubmittedFromID: &src.ID,
} }
deliveries, err := h.engine.StoreEvent(event) event, tasks, err := h.createAndFanOut(
eventSource{
WebhookID: webhook.ID,
EntrypointID: src.EntrypointID,
Method: src.Method,
RawQuery: src.RawQuery,
HeadersJSON: src.Headers,
ContentType: src.ContentType,
Body: src.Body,
ResubmittedFromID: &src.ID,
},
targets,
)
if err != nil { if err != nil {
h.serverError(w, r, "failed to store resubmitted event", err) h.serverError(w, r, "failed to store resubmitted event", err)
@@ -207,11 +215,11 @@ func (h *Handlers) queueResubmit(
"webhook_id", webhook.ID, "webhook_id", webhook.ID,
"event_id", event.ID, "event_id", event.ID,
"resubmitted_from_id", src.ID, "resubmitted_from_id", src.ID,
"target_count", deliveries, "target_count", len(tasks),
) )
code := resubmitQueued code := resubmitQueued
if deliveries == 0 { if len(tasks) == 0 {
code = resubmitNoTargets code = resubmitNoTargets
} }
internal/handlers/event_resubmit_test.go
+102 -81
View File
@@ -19,7 +19,7 @@ import (
// resubmitTargetURL is a public destination, so a target configured // resubmitTargetURL is a public destination, so a target configured
// with it is one the SSRF guard would accept. Nothing in these tests // with it is one the SSRF guard would accept. Nothing in these tests
// dispatches to it: the delivery engine is never started. // dispatches to it: the notifier is recorded, not run.
const resubmitTargetURL = "http://93.184.216.34/hook" const resubmitTargetURL = "http://93.184.216.34/hook"
// resubmitEventHeaders and resubmitEventQuery are the stored header // resubmitEventHeaders and resubmitEventQuery are the stored header
@@ -134,9 +134,10 @@ func TestHandleEventResubmit_DeliversToTargetCreatedAfterTheEvent(
sess *session.Session sess *session.Session
db *database.Database db *database.Database
dbMgr *database.WebhookDBManager dbMgr *database.WebhookDBManager
notif *recordingNotifier
) )
app := newTestApp(t, &h, &sess, &db, &dbMgr) app := newTestApp(t, &h, &sess, &db, &dbMgr, &notif)
app.RequireStart() app.RequireStart()
t.Cleanup(app.RequireStop) t.Cleanup(app.RequireStop)
@@ -150,7 +151,7 @@ func TestHandleEventResubmit_DeliversToTargetCreatedAfterTheEvent(
// captured. It has no prior delivery of anything. // captured. It has no prior delivery of anything.
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
database.Target{URL: resubmitTargetURL}, `{"url":"`+resubmitTargetURL+`"}`,
) )
w := postResubmit(t, h, sess, wh.ID, original.ID) w := postResubmit(t, h, sess, wh.ID, original.ID)
@@ -188,6 +189,11 @@ func TestHandleEventResubmit_DeliversToTargetCreatedAfterTheEvent(
t, database.DeliveryStatusPending, deliveries[0].Status, t, database.DeliveryStatusPending, deliveries[0].Status,
) )
tasks := notif.Tasks()
require.Len(t, tasks, 1)
assert.Equal(t, deliveries[0].ID, tasks[0].DeliveryID)
assertResubmitTask(t, tasks[0], wh.ID, &fresh, tgt)
assertNoLeakedTarget(t, webhookDB) assertNoLeakedTarget(t, webhookDB)
} }
@@ -219,6 +225,32 @@ func assertEventCopy(
) )
} }
// assertResubmitTask proves the task handed to the delivery engine is
// the one the receiver would build for the NEW event and this target.
func assertResubmitTask(
t *testing.T,
task delivery.Task,
webhookID string,
fresh *database.Event,
target *database.Target,
) {
t.Helper()
assert.Equal(t, fresh.ID, task.EventID)
assert.Equal(t, webhookID, task.WebhookID)
assert.Equal(t, fresh.EntrypointID, task.EntrypointID)
assert.Equal(t, target.ID, task.TargetID)
assert.Equal(t, target.Type, task.TargetType)
assert.Equal(t, fresh.Method, task.Method)
assert.Equal(t, fresh.RawQuery, task.RawQuery)
assert.Equal(t, fresh.Headers, task.Headers)
assert.Equal(t, fresh.ContentType, task.ContentType)
assert.Equal(t, 1, task.AttemptNum)
require.NotNil(t, task.Body)
assert.Equal(t, fresh.Body, *task.Body)
}
// TestHandleEventResubmit_IsRepeatable proves the requirement replay // TestHandleEventResubmit_IsRepeatable proves the requirement replay
// deliberately does not meet: firing the same captured event at a // deliberately does not meet: firing the same captured event at a
// backend over and over must work, with no in-flight refusal, and each // backend over and over must work, with no in-flight refusal, and each
@@ -231,9 +263,10 @@ func TestHandleEventResubmit_IsRepeatable(t *testing.T) {
sess *session.Session sess *session.Session
db *database.Database db *database.Database
dbMgr *database.WebhookDBManager dbMgr *database.WebhookDBManager
notif *recordingNotifier
) )
app := newTestApp(t, &h, &sess, &db, &dbMgr) app := newTestApp(t, &h, &sess, &db, &dbMgr, &notif)
app.RequireStart() app.RequireStart()
t.Cleanup(app.RequireStop) t.Cleanup(app.RequireStop)
@@ -241,7 +274,7 @@ func TestHandleEventResubmit_IsRepeatable(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
database.Target{URL: resubmitTargetURL}, `{"url":"`+resubmitTargetURL+`"}`,
) )
original := seedStoredEvent(t, dbMgr, wh.ID, `{"fire":"again"}`) original := seedStoredEvent(t, dbMgr, wh.ID, `{"fire":"again"}`)
@@ -267,18 +300,30 @@ func TestHandleEventResubmit_IsRepeatable(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
events := listEvents(t, webhookDB) events := listEvents(t, webhookDB)
require.Len( require.Len(t, events, presses+1)
t, events, presses+1, "each resubmit creates its own event",
)
assert.Empty( tasks := notif.Tasks()
t, listDeliveries(t, webhookDB, original.ID), require.Len(t, tasks, presses)
"each resubmit delivers its own new event",
)
// Every copy has its own delivery, and names the same source seen := make(map[string]struct{}, presses)
// event, so twenty presses stay traceable to the one captured
// request. for _, task := range tasks {
assert.Equal(t, tgt.ID, task.TargetID)
assert.NotEqual(
t, original.ID, task.EventID,
"each resubmit delivers its own new event",
)
_, dup := seen[task.EventID]
assert.False(t, dup, "each resubmit creates its own event")
seen[task.EventID] = struct{}{}
require.Len(t, listDeliveries(t, webhookDB, task.EventID), 1)
}
// Every copy names the same source event, so twenty presses stay
// traceable to the one captured request.
for _, e := range events { for _, e := range events {
if e.ID == original.ID { if e.ID == original.ID {
continue continue
@@ -286,10 +331,6 @@ func TestHandleEventResubmit_IsRepeatable(t *testing.T) {
require.NotNil(t, e.ResubmittedFromID) require.NotNil(t, e.ResubmittedFromID)
assert.Equal(t, original.ID, *e.ResubmittedFromID) assert.Equal(t, original.ID, *e.ResubmittedFromID)
deliveries := listDeliveries(t, webhookDB, e.ID)
require.Len(t, deliveries, 1)
assert.Equal(t, tgt.ID, deliveries[0].TargetID)
} }
} }
@@ -305,9 +346,10 @@ func TestHandleEventResubmit_OversizeBodySurvivesIntact(t *testing.T) {
sess *session.Session sess *session.Session
db *database.Database db *database.Database
dbMgr *database.WebhookDBManager dbMgr *database.WebhookDBManager
notif *recordingNotifier
) )
app := newTestApp(t, &h, &sess, &db, &dbMgr) app := newTestApp(t, &h, &sess, &db, &dbMgr, &notif)
app.RequireStart() app.RequireStart()
t.Cleanup(app.RequireStop) t.Cleanup(app.RequireStop)
@@ -326,7 +368,7 @@ func TestHandleEventResubmit_OversizeBodySurvivesIntact(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
seedConfiguredTarget( seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
database.Target{URL: resubmitTargetURL}, `{"url":"`+resubmitTargetURL+`"}`,
) )
original := seedStoredEvent(t, dbMgr, wh.ID, stored) original := seedStoredEvent(t, dbMgr, wh.ID, stored)
@@ -343,7 +385,22 @@ func TestHandleEventResubmit_OversizeBodySurvivesIntact(t *testing.T) {
t, stored, fresh.Body, t, stored, fresh.Body,
"the stored body must be copied byte for byte", "the stored body must be copied byte for byte",
) )
assert.Len(t, listDeliveries(t, webhookDB, fresh.ID), 1)
tasks := notif.Tasks()
require.Len(t, tasks, 1)
assert.Nil(
t, tasks[0].Body,
"a body over the inline limit is fetched from the new "+
"event row rather than carried on the task",
)
// The engine's own read of the body, against the new event id:
// what it would send is what was stored.
var loaded database.Event
require.NoError(t, webhookDB.Select("body").
First(&loaded, "id = ?", tasks[0].EventID).Error)
assert.Equal(t, stored, loaded.Body)
} }
// TestHandleEventResubmit_SkipsInactiveTarget proves a deactivated // TestHandleEventResubmit_SkipsInactiveTarget proves a deactivated
@@ -357,9 +414,10 @@ func TestHandleEventResubmit_SkipsInactiveTarget(t *testing.T) {
sess *session.Session sess *session.Session
db *database.Database db *database.Database
dbMgr *database.WebhookDBManager dbMgr *database.WebhookDBManager
notif *recordingNotifier
) )
app := newTestApp(t, &h, &sess, &db, &dbMgr) app := newTestApp(t, &h, &sess, &db, &dbMgr, &notif)
app.RequireStart() app.RequireStart()
t.Cleanup(app.RequireStop) t.Cleanup(app.RequireStop)
@@ -367,11 +425,11 @@ func TestHandleEventResubmit_SkipsInactiveTarget(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
active := seedConfiguredTarget( active := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
database.Target{URL: resubmitTargetURL}, `{"url":"`+resubmitTargetURL+`"}`,
) )
off := seedConfiguredTarget( off := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
database.Target{URL: resubmitTargetURL + "/off"}, `{"url":"`+resubmitTargetURL+`/off"}`,
) )
require.NoError(t, db.DB().Model(&database.Target{}). require.NoError(t, db.DB().Model(&database.Target{}).
@@ -390,19 +448,21 @@ func TestHandleEventResubmit_SkipsInactiveTarget(t *testing.T) {
"an inactive target is skipped, not an error", "an inactive target is skipped, not an error",
) )
tasks := notif.Tasks()
require.Len(t, tasks, 1)
assert.Equal(t, active.ID, tasks[0].TargetID)
webhookDB, err := dbMgr.GetDB(wh.ID) webhookDB, err := dbMgr.GetDB(wh.ID)
require.NoError(t, err) require.NoError(t, err)
fresh := theOtherEvent(t, listEvents(t, webhookDB), original.ID) fresh := theOtherEvent(t, listEvents(t, webhookDB), original.ID)
deliveries := listDeliveries(t, webhookDB, fresh.ID) require.Len(t, listDeliveries(t, webhookDB, fresh.ID), 1)
require.Len(t, deliveries, 1)
assert.Equal(t, active.ID, deliveries[0].TargetID)
} }
// TestHandleEventResubmit_NoActiveTargetsStillStoresEvent proves a // TestHandleEventResubmit_NoActiveTargetsStillStoresEvent proves a
// source with nothing to deliver to behaves as the receiver does: the // source with nothing to deliver to behaves as the receiver does: the
// event is stored with no delivery, and the operator is told so rather // event is stored, nothing is queued, and the operator is told so
// than being shown an error. // rather than being shown an error.
func TestHandleEventResubmit_NoActiveTargetsStillStoresEvent( func TestHandleEventResubmit_NoActiveTargetsStillStoresEvent(
t *testing.T, t *testing.T,
) { ) {
@@ -413,9 +473,10 @@ func TestHandleEventResubmit_NoActiveTargetsStillStoresEvent(
sess *session.Session sess *session.Session
db *database.Database db *database.Database
dbMgr *database.WebhookDBManager dbMgr *database.WebhookDBManager
notif *recordingNotifier
) )
app := newTestApp(t, &h, &sess, &db, &dbMgr) app := newTestApp(t, &h, &sess, &db, &dbMgr, &notif)
app.RequireStart() app.RequireStart()
t.Cleanup(app.RequireStop) t.Cleanup(app.RequireStop)
@@ -432,14 +493,12 @@ func TestHandleEventResubmit_NoActiveTargetsStillStoresEvent(
w.Header().Get("Location"), w.Header().Get("Location"),
) )
assert.Empty(t, notif.Tasks())
webhookDB, err := dbMgr.GetDB(wh.ID) webhookDB, err := dbMgr.GetDB(wh.ID)
require.NoError(t, err) require.NoError(t, err)
events := listEvents(t, webhookDB) assert.Len(t, listEvents(t, webhookDB), 2)
assert.Len(t, events, 2)
fresh := theOtherEvent(t, events, original.ID)
assert.Empty(t, listDeliveries(t, webhookDB, fresh.ID))
} }
// TestHandleEventResubmit_RefusesEventOfAnotherWebhook proves the // TestHandleEventResubmit_RefusesEventOfAnotherWebhook proves the
@@ -455,9 +514,10 @@ func TestHandleEventResubmit_RefusesEventOfAnotherWebhook(
sess *session.Session sess *session.Session
db *database.Database db *database.Database
dbMgr *database.WebhookDBManager dbMgr *database.WebhookDBManager
notif *recordingNotifier
) )
app := newTestApp(t, &h, &sess, &db, &dbMgr) app := newTestApp(t, &h, &sess, &db, &dbMgr, &notif)
app.RequireStart() app.RequireStart()
t.Cleanup(app.RequireStop) t.Cleanup(app.RequireStop)
@@ -468,7 +528,7 @@ func TestHandleEventResubmit_RefusesEventOfAnotherWebhook(
mine := seedWebhook(t, db) mine := seedWebhook(t, db)
seedConfiguredTarget( seedConfiguredTarget(
t, db, mine.ID, database.TargetTypeHTTP, t, db, mine.ID, database.TargetTypeHTTP,
database.Target{URL: resubmitTargetURL}, `{"url":"`+resubmitTargetURL+`"}`,
) )
seedStoredEvent(t, dbMgr, mine.ID, `{"mine":true}`) seedStoredEvent(t, dbMgr, mine.ID, `{"mine":true}`)
@@ -488,49 +548,10 @@ func TestHandleEventResubmit_RefusesEventOfAnotherWebhook(
w = postResubmit(t, h, sess, mine.ID, "not-a-uuid") w = postResubmit(t, h, sess, mine.ID, "not-a-uuid")
assert.Equal(t, http.StatusNotFound, w.Code) assert.Equal(t, http.StatusNotFound, w.Code)
for _, webhookID := range []string{theirs.ID, mine.ID} { assert.Empty(
webhookDB, err := dbMgr.GetDB(webhookID) t, notif.Tasks(),
require.NoError(t, err) "a refused resubmit must queue nothing",
assert.Len(
t, listEvents(t, webhookDB), 1,
"a refused resubmit must store nothing",
)
}
}
// TestHandleEventResubmit_FailedStoreIsAnError proves a copy the engine
// cannot store answers with the 500 error page and leaves no new event.
func TestHandleEventResubmit_FailedStoreIsAnError(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
) )
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
original := seedStoredEvent(t, dbMgr, wh.ID, `{"not":"stored"}`)
// Without its event totals table the webhook's database refuses
// the totals write, which comes after the event row.
webhookDB, err := dbMgr.GetDB(wh.ID)
require.NoError(t, err)
require.NoError(t,
webhookDB.Migrator().DropTable(&database.EventTotals{}))
w := postResubmit(t, h, sess, wh.ID, original.ID)
assert.Equal(t, http.StatusInternalServerError, w.Code)
assert.Contains(t, w.Body.String(), "500 Internal Server Error")
assert.Len(t, listEvents(t, webhookDB), 1)
} }
// TestHandleSourceLogs_ShowsResubmitProvenance proves the event log // TestHandleSourceLogs_ShowsResubmitProvenance proves the event log
@@ -554,7 +575,7 @@ func TestHandleSourceLogs_ShowsResubmitProvenance(t *testing.T) {
wh := seedWebhook(t, db) wh := seedWebhook(t, db)
seedConfiguredTarget( seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
database.Target{URL: resubmitTargetURL}, `{"url":"`+resubmitTargetURL+`"}`,
) )
original := seedStoredEvent(t, dbMgr, wh.ID, `{"trace":"me"}`) original := seedStoredEvent(t, dbMgr, wh.ID, `{"trace":"me"}`)
internal/handlers/export_test.go
+25 -17
View File
@@ -133,28 +133,36 @@ func (s *Handlers) RenderTemplateForTest(
s.renderTemplate(w, r, pageTemplate, data) s.renderTemplate(w, r, pageTemplate, data)
} }
// SetSlackTargetSettingsForTest exposes setSlackTargetSettings for use // BuildSlackTargetConfigForTest exposes
// in the handlers_test package. // buildSlackTargetConfig for use in the handlers_test package.
func (s *Handlers) SetSlackTargetSettingsForTest( func (s *Handlers) BuildSlackTargetConfigForTest(
ctx context.Context, ctx context.Context,
target *database.Target,
targetURL string, targetURL string,
) string { ) (string, string, error) {
return s.setSlackTargetSettings(ctx, target, targetURL) return s.buildSlackTargetConfig(ctx, targetURL)
} }
// SetDatabaseTargetSettingsForTest exposes setDatabaseTargetSettings // BuildHTTPTargetConfigForTest exposes buildHTTPTargetConfig
// for use in the handlers_test package. // for use in the handlers_test package, taking the form fields
func SetDatabaseTargetSettingsForTest( // an HTTP target's configuration is built from.
target *database.Target, func (s *Handlers) BuildHTTPTargetConfigForTest(
ctx context.Context,
targetURL, headers, timeout string,
) (string, string, error) {
return s.buildHTTPTargetConfig(ctx, targetFormInput{
URL: targetURL,
Headers: headers,
Timeout: timeout,
})
}
// BuildDatabaseTargetConfigForTest exposes
// buildDatabaseTargetConfig for use in the handlers_test
// package.
func BuildDatabaseTargetConfigForTest(
expiry, rotation string, expiry, rotation string,
) string { ) (string, string, error) {
return setDatabaseTargetSettings(target, expiry, rotation) return buildDatabaseTargetConfig(expiry, rotation)
}
// FormatArchiveExpiryForTest exposes formatArchiveExpiry.
func FormatArchiveExpiryForTest(expiry time.Duration) string {
return formatArchiveExpiry(expiry)
} }
// ArchiveFileViewForTest exposes archiveFileView, which describes a // ArchiveFileViewForTest exposes archiveFileView, which describes a
internal/handlers/handlers.go
+3 -3
View File
@@ -64,7 +64,7 @@ type HandlersParams struct {
Healthcheck *healthcheck.Healthcheck Healthcheck *healthcheck.Healthcheck
Session *session.Session Session *session.Session
Middleware *middleware.Middleware Middleware *middleware.Middleware
Engine *delivery.Engine Notifier delivery.Notifier
Archives archive.Archives Archives archive.Archives
CircuitBreakers delivery.CircuitBreakers CircuitBreakers delivery.CircuitBreakers
SSRFGuard *delivery.Guard SSRFGuard *delivery.Guard
@@ -82,7 +82,7 @@ type Handlers struct {
dbMgr *database.WebhookDBManager dbMgr *database.WebhookDBManager
session *session.Session session *session.Session
mw *middleware.Middleware mw *middleware.Middleware
engine *delivery.Engine notifier delivery.Notifier
archives archive.Archives archives archive.Archives
breakers delivery.CircuitBreakers breakers delivery.CircuitBreakers
mtr *metrics.Set mtr *metrics.Set
@@ -138,7 +138,7 @@ func New(
s.dbMgr = params.WebhookDBMgr s.dbMgr = params.WebhookDBMgr
s.session = params.Session s.session = params.Session
s.mw = params.Middleware s.mw = params.Middleware
s.engine = params.Engine s.notifier = params.Notifier
s.archives = params.Archives s.archives = params.Archives
s.breakers = params.CircuitBreakers s.breakers = params.CircuitBreakers
s.mtr = params.Metrics s.mtr = params.Metrics
internal/handlers/handlers_test.go
+100 -105
View File
@@ -8,7 +8,6 @@ import (
"log/slog" "log/slog"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"strconv"
"sync" "sync"
"testing" "testing"
"time" "time"
@@ -30,6 +29,33 @@ import (
"sneak.berlin/go/webhooker/internal/session" "sneak.berlin/go/webhooker/internal/session"
) )
// recordingNotifier is a delivery.Notifier that records the tasks it
// was handed, so a test can prove a handler queued the delivery it
// claims to have queued — and, on the refusal paths, that it queued
// nothing.
type recordingNotifier struct {
mu sync.Mutex
tasks []delivery.Task
}
func (n *recordingNotifier) Notify(tasks []delivery.Task) {
n.mu.Lock()
defer n.mu.Unlock()
n.tasks = append(n.tasks, tasks...)
}
// Tasks returns a copy of the recorded tasks.
func (n *recordingNotifier) Tasks() []delivery.Task {
n.mu.Lock()
defer n.mu.Unlock()
out := make([]delivery.Task, len(n.tasks))
copy(out, n.tasks)
return out
}
// recordingArchives is an archive.Archives that records what it // recordingArchives is an archive.Archives that records what it
// was asked to do, so a test can prove that a deletion or rename // was asked to do, so a test can prove that a deletion or rename
// path reached the archive writers. After FailRenames, every // path reached the archive writers. After FailRenames, every
@@ -237,12 +263,11 @@ func newTestAppWithConfig(
database.NewWebhookDBManager, database.NewWebhookDBManager,
healthcheck.New, healthcheck.New,
session.New, session.New,
archive.NewWriters, func() *recordingNotifier {
// The engine is built on a lifecycle that is never started, return &recordingNotifier{}
// so no worker runs: every delivery the engine creates for a },
// handler stays pending, where the test can read it. func(n *recordingNotifier) delivery.Notifier {
func(p delivery.EngineParams) *delivery.Engine { return n
return delivery.New(fxtest.NewLifecycle(t), p)
}, },
func() *recordingArchives { func() *recordingArchives {
return &recordingArchives{} return &recordingArchives{}
@@ -329,7 +354,7 @@ func TestHandleIndex_Authenticated(t *testing.T) {
) )
} }
func TestSetSlackTargetSettings_AcceptsPublicURL(t *testing.T) { func TestBuildSlackTargetConfig_AcceptsPublicURL(t *testing.T) {
t.Parallel() t.Parallel()
var h *handlers.Handlers var h *handlers.Handlers
@@ -339,34 +364,32 @@ func TestSetSlackTargetSettings_AcceptsPublicURL(t *testing.T) {
t.Cleanup(app.RequireStop) t.Cleanup(app.RequireStop)
const hookURL = "http://93.184.216.34/services/T00/B00/xxx" cfg, errMsg, err := h.BuildSlackTargetConfigForTest(
t.Context(), "http://93.184.216.34/services/T00/B00/xxx",
var target database.Target
errMsg := h.SetSlackTargetSettingsForTest(t.Context(), &target, hookURL)
assert.Empty(t, errMsg)
assert.Equal(t, hookURL, target.URL)
}
func TestSetSlackTargetSettings_RejectsReservedURL(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
app := newTestApp(t, &h)
app.RequireStart()
t.Cleanup(app.RequireStop)
var target database.Target
errMsg := h.SetSlackTargetSettingsForTest(
t.Context(), &target, "http://169.254.169.254/latest/meta-data/",
) )
require.NoError(t, err)
assert.Empty(t, errMsg)
assert.Contains(t, cfg, "webhookUrl")
}
func TestBuildSlackTargetConfig_RejectsReservedURL(t *testing.T) {
t.Parallel()
var h *handlers.Handlers
app := newTestApp(t, &h)
app.RequireStart()
t.Cleanup(app.RequireStop)
cfg, errMsg, err := h.BuildSlackTargetConfigForTest(
t.Context(), "http://169.254.169.254/latest/meta-data/",
)
require.NoError(t, err)
assert.Contains(t, errMsg, "Invalid target URL") assert.Contains(t, errMsg, "Invalid target URL")
assert.Empty(t, target.URL) assert.Empty(t, cfg)
} }
func TestRenderTemplate(t *testing.T) { func TestRenderTemplate(t *testing.T) {
@@ -460,100 +483,72 @@ func TestRenderTemplateMidRenderErrorSendsNoPartialBody(t *testing.T) {
) )
} }
func TestSetDatabaseTargetSettings_Valid(t *testing.T) { func TestBuildDatabaseTargetConfig_Valid(t *testing.T) {
t.Parallel() t.Parallel()
cases := []struct { // Empty expiry and rotation: the keep-forever, one-file default,
expiry, rotation string // empty config.
wantExpiry time.Duration cfg, errMsg, err := handlers.BuildDatabaseTargetConfigForTest("", "")
wantRotation string require.NoError(t, err)
}{ assert.Empty(t, errMsg)
// Empty expiry and rotation: the keep-forever, one-file assert.Empty(t, cfg)
// default.
{"", "", 0, rotationNone},
{expiryNever, "", 0, rotationNone},
{" 36h ", "", 36 * time.Hour, rotationNone},
{"", rotationDaily, 0, rotationDaily},
{"90m", rotationHourly, 90 * time.Minute, rotationHourly},
}
for _, tc := range cases { // Explicit never is stored as config.
var target database.Target cfg, errMsg, err = handlers.BuildDatabaseTargetConfigForTest("never", "")
require.NoError(t, err)
assert.Empty(t, errMsg)
assert.JSONEq(t, `{"expiry":"never"}`, cfg)
errMsg := handlers.SetDatabaseTargetSettingsForTest( // A positive duration is stored as config.
&target, tc.expiry, tc.rotation, cfg, errMsg, err = handlers.BuildDatabaseTargetConfigForTest("720h", "")
) require.NoError(t, err)
assert.Empty(t, errMsg)
assert.JSONEq(t, `{"expiry":"720h"}`, cfg)
assert.Empty(t, errMsg, tc) // A rotation is stored as config, with or without an expiry.
assert.Equal(t, tc.wantExpiry, target.ArchiveExpiry, tc) cfg, errMsg, err = handlers.BuildDatabaseTargetConfigForTest("", "daily")
assert.Equal(t, tc.wantRotation, target.ArchiveRotation, tc) require.NoError(t, err)
} assert.Empty(t, errMsg)
assert.JSONEq(t, `{"rotation":"daily"}`, cfg)
cfg, errMsg, err = handlers.BuildDatabaseTargetConfigForTest(
"720h", "hourly",
)
require.NoError(t, err)
assert.Empty(t, errMsg)
assert.JSONEq(t, `{"expiry":"720h","rotation":"hourly"}`, cfg)
} }
// TestSetDatabaseTargetSettings_RejectsBadExpiry pins each refusal's func TestBuildDatabaseTargetConfig_RejectsBadExpiry(
// whole message, as the form shows it.
func TestSetDatabaseTargetSettings_RejectsBadExpiry(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
cases := map[string]string{ for _, bad := range []string{"nonsense", "7d", "-5h"} {
"nonsense": `Invalid archive expiry: expiry must be "never" or ` + cfg, errMsg, err := handlers.BuildDatabaseTargetConfigForTest(bad, "")
`a Go duration such as "720h": time: invalid duration ` +
`"nonsense"`,
"7d": `Invalid archive expiry: expiry must be "never" or a Go ` +
`duration such as "720h": time: unknown unit "d" in ` +
`duration "7d"`,
"-5h": `Invalid archive expiry: expiry must be a positive ` +
`duration or "never": "-5h"`,
}
for bad, want := range cases { require.NoError(t, err)
var target database.Target assert.Contains(
t, errMsg, "Invalid archive expiry",
errMsg := handlers.SetDatabaseTargetSettingsForTest( "expiry %q should be refused", bad,
&target, bad, "",
) )
assert.Empty(t, cfg)
assert.Equal(t, want, errMsg, "expiry %q", bad)
} }
} }
func TestSetDatabaseTargetSettings_RejectsBadRotation( func TestBuildDatabaseTargetConfig_RejectsBadRotation(
t *testing.T, t *testing.T,
) { ) {
t.Parallel() t.Parallel()
for _, bad := range []string{"weekly", "Daily", " none"} { for _, bad := range []string{"weekly", "Daily", " none"} {
var target database.Target cfg, errMsg, err := handlers.BuildDatabaseTargetConfigForTest("", bad)
errMsg := handlers.SetDatabaseTargetSettingsForTest( require.NoError(t, err)
&target, "", bad, assert.Contains(
) t, errMsg, "Invalid archive rotation",
"rotation %q should be refused", bad,
assert.Equal(
t,
"Invalid archive rotation: rotation must be none, "+
"monthly, daily or hourly: "+strconv.Quote(bad),
errMsg,
) )
} assert.Empty(t, cfg)
}
// TestFormatArchiveExpiry covers the expiries the expiry choices do
// not: TestArchiveExpiryChoices shows the edit form starting on each
// choice. An expiry in whole hours is written in hours, and any other
// as Go writes it, which the form accepts back unchanged.
func TestFormatArchiveExpiry(t *testing.T) {
t.Parallel()
cases := map[time.Duration]string{
0: expiryNever,
36 * time.Hour: "36h",
90 * time.Minute: "1h30m0s",
}
for expiry, want := range cases {
assert.Equal(t, want, handlers.FormatArchiveExpiryForTest(expiry))
} }
} }
internal/handlers/source_create_targets_test.go
+12 -23
View File
@@ -7,7 +7,6 @@ import (
"net/http/httptest" "net/http/httptest"
"net/url" "net/url"
"testing" "testing"
"time"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
@@ -55,26 +54,15 @@ func TestHandleSourceCreateSubmit_CreatesRequestedTargets(t *testing.T) {
env := setupSourceTest(t) env := setupSourceTest(t)
// Each value the archive pruning choice submits and the expiry it // Each value the archive pruning choice submits, after an empty
// stores, after an empty one that stands for the archive checkbox // one that stands for the archive checkbox left off.
// left off. expiries := []string{
expiries := []struct { "", "never", "1h", "12h", "24h", "720h", "2160h", "8760h",
value string
stored time.Duration
}{
{"", 0},
{expiryNever, 0},
{"1h", time.Hour},
{"12h", 12 * time.Hour},
{"24h", 24 * time.Hour},
{"720h", 720 * time.Hour},
{"2160h", 2160 * time.Hour},
{"8760h", 8760 * time.Hour},
} }
for _, httpURL := range []string{"", editOriginalURL} { for _, httpURL := range []string{"", editOriginalURL} {
for _, expiry := range expiries { for _, expiry := range expiries {
name := "url=" + httpURL + " archive=" + expiry.value name := "url=" + httpURL + " archive=" + expiry
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
t.Parallel() t.Parallel()
@@ -84,9 +72,9 @@ func TestHandleSourceCreateSubmit_CreatesRequestedTargets(t *testing.T) {
form.Set("http_url", httpURL) form.Set("http_url", httpURL)
form.Set("archive_expiry", "720h") form.Set("archive_expiry", "720h")
if expiry.value != "" { if expiry != "" {
form.Set("archive", "on") form.Set("archive", "on")
form.Set("archive_expiry", expiry.value) form.Set("archive_expiry", expiry)
} }
w := submitCreateForm(env, form) w := submitCreateForm(env, form)
@@ -108,16 +96,17 @@ func TestHandleSourceCreateSubmit_CreatesRequestedTargets(t *testing.T) {
wantCount++ wantCount++
assert.Equal(t, "HTTP", byType[database.TargetTypeHTTP].Name) assert.Equal(t, "HTTP", byType[database.TargetTypeHTTP].Name)
assert.Equal(t, httpURL, byType[database.TargetTypeHTTP].URL) assert.JSONEq(t, `{"url":"`+httpURL+`"}`,
byType[database.TargetTypeHTTP].Config)
} }
if expiry.value != "" { if expiry != "" {
wantCount++ wantCount++
assert.Equal(t, "Archive", assert.Equal(t, "Archive",
byType[database.TargetTypeDatabase].Name) byType[database.TargetTypeDatabase].Name)
assert.Equal(t, expiry.stored, assert.JSONEq(t, `{"expiry":"`+expiry+`"}`,
byType[database.TargetTypeDatabase].ArchiveExpiry) byType[database.TargetTypeDatabase].Config)
} }
assert.Len(t, byType, wantCount) assert.Len(t, byType, wantCount)
internal/handlers/source_detail_test.go
+86 -62
View File
@@ -5,7 +5,6 @@ import (
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"testing" "testing"
"time"
"github.com/go-chi/chi" "github.com/go-chi/chi"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
@@ -26,24 +25,25 @@ const (
slackSecretPath slackSecretPath
) )
// seedConfiguredTarget inserts an active target of targetType with // seedConfiguredTarget inserts a target with a stored config
// the settings (URL to ArchiveRotation) that settings carries, and // blob and returns it.
// returns it.
func seedConfiguredTarget( func seedConfiguredTarget(
t *testing.T, t *testing.T,
db *database.Database, db *database.Database,
webhookID string, webhookID string,
targetType database.TargetType, targetType database.TargetType,
settings database.Target, config string,
) *database.Target { ) *database.Target {
t.Helper() t.Helper()
tgt := &settings tgt := &database.Target{
tgt.WebhookID = webhookID WebhookID: webhookID,
tgt.Name = "t-" + string(targetType) Name: "t-" + string(targetType),
tgt.Type = targetType Type: targetType,
tgt.Active = true Active: true,
tgt.MaxAttempts = 1 Config: config,
MaxAttempts: 1,
}
require.NoError( require.NoError(
t, t,
@@ -108,59 +108,89 @@ func serveSourceDetailPage(
return w return w
} }
// TestHandleSourceDetail_MasksTargetURL is the load-bearing // TestHandleSourceDetail_MasksSlackWebhookURL is the
// regression test for the credential leak: the rendered page must // load-bearing regression test for the credential leak: the
// show a Slack target, and an http target, without any of the secret // rendered page must show the Slack target without any of the
// path segments of its URL. An http target's destination is routinely // secret path segments of its webhook URL.
// an incoming-webhook endpoint too, so the same Slack URL serves for func TestHandleSourceDetail_MasksSlackWebhookURL(t *testing.T) {
// both.
func TestHandleSourceDetail_MasksTargetURL(t *testing.T) {
t.Parallel() t.Parallel()
labels := map[database.TargetType]string{ var (
database.TargetTypeSlack: "Webhook URL", h *handlers.Handlers
database.TargetTypeHTTP: "Destination URL", sess *session.Session
} db *database.Database
)
for targetType, label := range labels { app := newTestApp(t, &h, &sess, &db)
t.Run(string(targetType), func(t *testing.T) { app.RequireStart()
t.Parallel()
var ( t.Cleanup(app.RequireStop)
h *handlers.Handlers
sess *session.Session
db *database.Database
)
app := newTestApp(t, &h, &sess, &db) wh := seedWebhook(t, db)
app.RequireStart() seedConfiguredTarget(
t, db, wh.ID,
database.TargetTypeSlack,
`{"webhookUrl":"`+slackWebhookURL+`"}`,
)
t.Cleanup(app.RequireStop) body := renderSourceDetailPage(t, h, sess, wh.ID)
wh := seedWebhook(t, db) assert.NotContains(t, body, slackSecretPath)
seedConfiguredTarget( assert.NotContains(t, body, "T00000000")
t, db, wh.ID, targetType, assert.NotContains(t, body, "B00000000")
database.Target{URL: slackWebhookURL}, assert.NotContains(
) t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
)
assert.NotContains(t, body, "webhookUrl")
body := renderSourceDetailPage(t, h, sess, wh.ID) assert.Contains(t, body, "Webhook URL")
assert.Contains(t, body, "https://hooks.slack.com/...")
}
assert.NotContains(t, body, slackSecretPath) // TestHandleSourceDetail_MasksHTTPDestinationURL is the
assert.NotContains(t, body, "T00000000") // regression test for the same leak reached through the http
assert.NotContains(t, body, "B00000000") // target: its destination is routinely an incoming-webhook
assert.NotContains( // endpoint whose path segments are the credential, so the
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX", // rendered page must not contain them.
) func TestHandleSourceDetail_MasksHTTPDestinationURL(
t *testing.T,
) {
t.Parallel()
assert.Contains(t, body, label) var (
assert.Contains(t, body, "https://hooks.slack.com/...") h *handlers.Handlers
}) sess *session.Session
} db *database.Database
)
app := newTestApp(t, &h, &sess, &db)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
seedConfiguredTarget(
t, db, wh.ID,
database.TargetTypeHTTP,
`{"url":"`+slackWebhookURL+`"}`,
)
body := renderSourceDetailPage(t, h, sess, wh.ID)
assert.NotContains(t, body, slackSecretPath)
assert.NotContains(t, body, "T00000000")
assert.NotContains(t, body, "B00000000")
assert.NotContains(
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
)
assert.Contains(t, body, "Destination URL")
assert.Contains(t, body, "https://hooks.slack.com/...")
} }
// TestHandleSourceDetail_RendersNamedTargetFields proves the // TestHandleSourceDetail_RendersNamedTargetFields proves the
// other target types render labelled fields rather than the // other target types render labelled fields rather than the
// stored values. // stored blob.
func TestHandleSourceDetail_RendersNamedTargetFields( func TestHandleSourceDetail_RendersNamedTargetFields(
t *testing.T, t *testing.T,
) { ) {
@@ -182,24 +212,18 @@ func TestHandleSourceDetail_RendersNamedTargetFields(
seedConfiguredTarget( seedConfiguredTarget(
t, db, wh.ID, t, db, wh.ID,
database.TargetTypeHTTP, database.TargetTypeHTTP,
database.Target{ `{"url":"https://example.com/hook","timeout":30,`+
URL: "https://example.com/hook", `"headers":{"Authorization":"Bearer sekrit"}}`,
TimeoutSeconds: 30,
Headers: map[string]string{"X-Api-Key": "sekrit"},
},
) )
seedConfiguredTarget( seedConfiguredTarget(
t, db, wh.ID, t, db, wh.ID,
database.TargetTypeDatabase, database.TargetTypeDatabase,
database.Target{ `{"expiry":"720h"}`,
ArchiveExpiry: 720 * time.Hour,
ArchiveRotation: rotationNone,
},
) )
seedConfiguredTarget( seedConfiguredTarget(
t, db, wh.ID, t, db, wh.ID,
database.TargetType("carrier-pigeon"), database.TargetType("carrier-pigeon"),
database.Target{URL: "https://pigeon.example/beak"}, `{"beak":"sharp"}`,
) )
body := renderSourceDetailPage(t, h, sess, wh.ID) body := renderSourceDetailPage(t, h, sess, wh.ID)
@@ -218,8 +242,8 @@ func TestHandleSourceDetail_RendersNamedTargetFields(
assert.Contains(t, list, "Archive rotation: none") assert.Contains(t, list, "Archive rotation: none")
assert.Contains(t, body, `<option value="database">Archive</option>`) assert.Contains(t, body, `<option value="database">Archive</option>`)
// An unknown type gets the neutral placeholder, never its // An unknown type gets the neutral placeholder, never the
// stored URL. // stored blob.
assert.Contains(t, body, "(unavailable)") assert.Contains(t, body, "(unavailable)")
assert.NotContains(t, body, "beak") assert.NotContains(t, body, "beak")
} }
internal/handlers/source_logs_deleted_target_test.go
+2 -1
View File
@@ -161,7 +161,7 @@ func TestHandleSourceLogs_MasksDeletedTargetConfig(t *testing.T) {
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, t, db, wh.ID,
database.TargetTypeSlack, database.TargetTypeSlack,
database.Target{URL: slackWebhookURL}, `{"webhookUrl":"`+slackWebhookURL+`"}`,
) )
seedDeliveredEvent(t, dbMgr, wh.ID, tgt.ID) seedDeliveredEvent(t, dbMgr, wh.ID, tgt.ID)
@@ -175,6 +175,7 @@ func TestHandleSourceLogs_MasksDeletedTargetConfig(t *testing.T) {
assert.NotContains( assert.NotContains(
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX", t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
) )
assert.NotContains(t, body, "webhookUrl")
// The name is there; only the credential is not. // The name is there; only the credential is not.
assert.Contains(t, body, tgt.Name+deletedMarker) assert.Contains(t, body, tgt.Name+deletedMarker)
internal/handlers/source_logs_test.go
+2 -1
View File
@@ -138,7 +138,7 @@ func TestHandleSourceLogs_MasksSlackWebhookURL(t *testing.T) {
tgt := seedConfiguredTarget( tgt := seedConfiguredTarget(
t, db, wh.ID, t, db, wh.ID,
database.TargetTypeSlack, database.TargetTypeSlack,
database.Target{URL: slackWebhookURL}, `{"webhookUrl":"`+slackWebhookURL+`"}`,
) )
seedDeliveredEvent(t, dbMgr, wh.ID, tgt.ID) seedDeliveredEvent(t, dbMgr, wh.ID, tgt.ID)
@@ -151,6 +151,7 @@ func TestHandleSourceLogs_MasksSlackWebhookURL(t *testing.T) {
assert.NotContains( assert.NotContains(
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX", t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
) )
assert.NotContains(t, body, "webhookUrl")
// The page still identifies the delivery's target. // The page still identifies the delivery's target.
assert.Contains(t, body, tgt.Name) assert.Contains(t, body, tgt.Name)
internal/handlers/target_create.go
+108 -73
View File
@@ -2,6 +2,7 @@ package handlers
import ( import (
"context" "context"
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"net/http" "net/http"
@@ -64,14 +65,20 @@ func (h *Handlers) processTargetCreate(
) { ) {
in := targetFormInputFrom(r) in := targetFormInputFrom(r)
target, errMsg := h.newTarget(r.Context(), webhook.ID, in) target, errMsg, err := h.newTarget(r.Context(), webhook.ID, in)
if err != nil {
h.serverError(w, r, "failed to encode target config", err)
return
}
if errMsg != "" { if errMsg != "" {
h.renderSourceDetail(w, r, webhook, in, errMsg) h.renderSourceDetail(w, r, webhook, in, errMsg)
return return
} }
err := h.db.DB().Create(target).Error err = h.db.DB().Create(target).Error
if err != nil { if err != nil {
h.serverError(w, r, "failed to create target", err) h.serverError(w, r, "failed to create target", err)
@@ -86,13 +93,15 @@ func (h *Handlers) processTargetCreate(
// newTarget validates a new target for a webhook and returns the row // newTarget validates a new target for a webhook and returns the row
// to create, or, when it refuses the target, the message the form // to create, or, when it refuses the target, the message the form
// shows. Every form that creates a target goes through here, so they // shows. An error is the server's fault, not a refusal: the accepted
// all accept and refuse the same things. // configuration could not be encoded. Every form that creates a
// target goes through here, so they all accept and refuse the same
// things.
func (h *Handlers) newTarget( func (h *Handlers) newTarget(
ctx context.Context, ctx context.Context,
webhookID string, webhookID string,
in targetFormInput, in targetFormInput,
) (*database.Target, string) { ) (*database.Target, string, error) {
target := &database.Target{ target := &database.Target{
WebhookID: webhookID, WebhookID: webhookID,
Type: in.Type, Type: in.Type,
@@ -100,33 +109,34 @@ func (h *Handlers) newTarget(
MaxAttempts: 1, MaxAttempts: 1,
} }
errMsg := h.setTargetFromForm(ctx, target, in) errMsg, err := h.setTargetFromForm(ctx, target, in)
if errMsg != "" { if err != nil || errMsg != "" {
return nil, errMsg return nil, errMsg, err
} }
return target, "" return target, "", nil
} }
// setTargetFromForm validates a target form against the target's type // setTargetFromForm validates a target form against the target's type
// and sets the target's name, settings and number of delivery attempts // and, when it accepts it, sets the target's name, configuration and
// from it. It returns the message the form shows for anything it // number of delivery attempts from it. It returns the message the form
// refuses, an unknown type among them; the caller then discards the // shows for anything it refuses, an unknown type among them, and then
// target, which may be partly set. // leaves the target unchanged; an error is the server's fault, as for
// newTarget.
// The add target form and the target edit form both go through here, // The add target form and the target edit form both go through here,
// so the two cannot come to disagree about what a target may be. // so the two cannot come to disagree about what a target may be.
func (h *Handlers) setTargetFromForm( func (h *Handlers) setTargetFromForm(
ctx context.Context, ctx context.Context,
target *database.Target, target *database.Target,
in targetFormInput, in targetFormInput,
) string { ) (string, error) {
if in.Name == "" { if in.Name == "" {
return "Name is required" return "Name is required", nil
} }
errMsg := h.setTargetSettings(ctx, target, in) configJSON, errMsg, err := h.buildTargetConfig(ctx, target.Type, in)
if errMsg != "" { if err != nil || errMsg != "" {
return errMsg return errMsg, err
} }
// An empty max_attempts keeps the target's count: the default of // An empty max_attempts keeps the target's count: the default of
@@ -137,13 +147,14 @@ func (h *Handlers) setTargetFromForm(
// target is delivering with. // target is delivering with.
maxAttempts, err := parseMaxAttempts(in.MaxAttempts, target.MaxAttempts) maxAttempts, err := parseMaxAttempts(in.MaxAttempts, target.MaxAttempts)
if err != nil { if err != nil {
return "Invalid delivery attempts: " + attemptsErrorMessage(err) return "Invalid delivery attempts: " + attemptsErrorMessage(err), nil
} }
target.Name = in.Name target.Name = in.Name
target.Config = configJSON
target.MaxAttempts = maxAttempts target.MaxAttempts = maxAttempts
return "" return "", nil
} }
// targetFormInput carries the raw values of a target form. Both the // targetFormInput carries the raw values of a target form. Both the
@@ -202,81 +213,84 @@ func targetFormInputFrom(r *http.Request) targetFormInput {
} }
} }
// setTargetSettings sets a target's settings from the submitted form // buildTargetConfig builds the JSON config string for a target from
// values, or returns the message the form shows for a value it // the submitted form values, or returns the message the form shows
// refuses. Which fields of in apply depends on the target type; a // for a value it refuses. An error is the server's fault, not a
// type without a URL ignores any URL submitted, and the settings a // refusal: the accepted configuration could not be encoded. Which
// type does not use are left empty. // fields of in apply depends on the target type; a type without a URL
func (h *Handlers) setTargetSettings( // ignores any URL submitted.
func (h *Handlers) buildTargetConfig(
ctx context.Context, ctx context.Context,
target *database.Target, targetType database.TargetType,
in targetFormInput, in targetFormInput,
) string { ) (string, string, error) {
switch target.Type { switch targetType {
case database.TargetTypeHTTP: case database.TargetTypeHTTP:
return h.setHTTPTargetSettings(ctx, target, in) return h.buildHTTPTargetConfig(ctx, in)
case database.TargetTypeSlack: case database.TargetTypeSlack:
return h.setSlackTargetSettings(ctx, target, in.URL) return h.buildSlackTargetConfig(ctx, in.URL)
case database.TargetTypeDatabase: case database.TargetTypeDatabase:
return setDatabaseTargetSettings(target, in.Expiry, in.Rotation) return buildDatabaseTargetConfig(in.Expiry, in.Rotation)
case database.TargetTypeLog: case database.TargetTypeLog:
return "" return "", "", nil
default: default:
return "Invalid target type" return "", "Invalid target type", nil
} }
} }
// setHTTPTargetSettings sets an HTTP target's SSRF-validated // buildHTTPTargetConfig builds config JSON for an HTTP target: an
// destination plus the optional headers, timeout and query string // SSRF-validated destination plus the optional headers, timeout and
// setting the delivery path honours. // query string setting the delivery path honours.
func (h *Handlers) setHTTPTargetSettings( func (h *Handlers) buildHTTPTargetConfig(
ctx context.Context, ctx context.Context,
target *database.Target,
in targetFormInput, in targetFormInput,
) string { ) (string, string, error) {
errMsg := h.validateTargetURL( errMsg := h.validateTargetURL(
ctx, in.URL, "URL is required for HTTP targets", ctx, in.URL, "URL is required for HTTP targets",
) )
if errMsg != "" { if errMsg != "" {
return errMsg return "", errMsg, nil
} }
headers, err := delivery.ParseTargetHeaders(in.Headers) headers, err := delivery.ParseTargetHeaders(in.Headers)
if err != nil { if err != nil {
return fmt.Sprintf("Invalid headers: %v", err) return "", fmt.Sprintf("Invalid headers: %v", err), nil
} }
timeout, err := delivery.ParseTargetTimeout(in.Timeout) timeout, err := delivery.ParseTargetTimeout(in.Timeout)
if err != nil { if err != nil {
return fmt.Sprintf("Invalid timeout: %v", err) return "", fmt.Sprintf("Invalid timeout: %v", err), nil
} }
target.URL = in.URL configJSON, err := marshalTargetConfig(delivery.HTTPTargetConfig{
target.Headers = headers URL: in.URL,
target.TimeoutSeconds = timeout Headers: headers,
target.ForwardQuery = in.ForwardQuery Timeout: timeout,
ForwardQuery: in.ForwardQuery,
})
return "" return configJSON, "", err
} }
// setSlackTargetSettings sets a Slack target's one setting, its // buildSlackTargetConfig builds config JSON for a Slack target,
// SSRF-validated webhook URL. // whose whole configuration is one SSRF-validated webhook URL.
func (h *Handlers) setSlackTargetSettings( func (h *Handlers) buildSlackTargetConfig(
ctx context.Context, ctx context.Context,
target *database.Target,
targetURL string, targetURL string,
) string { ) (string, string, error) {
errMsg := h.validateTargetURL( errMsg := h.validateTargetURL(
ctx, targetURL, ctx, targetURL,
"Webhook URL is required for Slack targets", "Webhook URL is required for Slack targets",
) )
if errMsg != "" { if errMsg != "" {
return errMsg return "", errMsg, nil
} }
target.URL = targetURL configJSON, err := marshalTargetConfig(delivery.SlackTargetConfig{
WebhookURL: targetURL,
})
return "" return configJSON, "", err
} }
// validateTargetURL refuses an empty or SSRF-blocked destination, // validateTargetURL refuses an empty or SSRF-blocked destination,
@@ -327,31 +341,52 @@ func (h *Handlers) validateTargetURL(
return "" return ""
} }
// setDatabaseTargetSettings sets a database (archive) target's // marshalTargetConfig serialises a target configuration for storage.
// expiry and rotation. Both are optional: an empty or never expiry func marshalTargetConfig(cfg any) (string, error) {
// keeps the archive forever, and an empty rotation is stored as none, configBytes, err := json.Marshal(cfg)
// one file. A bad value is refused here, at the form, instead of
// being stored.
func setDatabaseTargetSettings(
target *database.Target,
expiry, rotation string,
) string {
archiveExpiry, err := archive.ParseExpiry(strings.TrimSpace(expiry))
if err != nil { if err != nil {
return fmt.Sprintf("Invalid archive expiry: %v", err) return "", err
}
return string(configBytes), nil
}
// buildDatabaseTargetConfig builds config JSON for a database
// (archive) target. The optional expiry and rotation are validated
// here, at creation time, so a bad value is refused instead of
// failing every subsequent delivery. Each is stored only when set,
// and with neither the config is empty (the keep-forever, one-file
// default).
func buildDatabaseTargetConfig(
expiry, rotation string,
) (string, string, error) {
expiry = strings.TrimSpace(expiry)
err := archive.ValidateExpiry(expiry)
if err != nil {
return "", fmt.Sprintf("Invalid archive expiry: %v", err), nil
} }
err = archive.ValidateRotation(rotation) err = archive.ValidateRotation(rotation)
if err != nil { if err != nil {
return fmt.Sprintf("Invalid archive rotation: %v", err) return "", fmt.Sprintf("Invalid archive rotation: %v", err), nil
} }
if rotation == "" { cfg := map[string]any{}
rotation = archiveRotationNone
if expiry != "" {
cfg["expiry"] = expiry
} }
target.ArchiveExpiry = archiveExpiry if rotation != "" {
target.ArchiveRotation = rotation cfg["rotation"] = rotation
}
return "" if len(cfg) == 0 {
return "", "", nil
}
configJSON, err := marshalTargetConfig(cfg)
return configJSON, "", err
} }
internal/handlers/target_create_query_test.go
+1 -1
View File
@@ -176,7 +176,7 @@ func TestHandleTargetCreate_BodyURLStillCreatesTheTarget(t *testing.T) {
targets := targetsForWebhook(t, env.db, webhook.ID) targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1) require.Len(t, targets, 1)
assert.Equal(t, targetSecretURL, targets[0].URL) assert.Contains(t, targets[0].Config, targetSecretSegments)
// The body carried the credential, so the access log must still // The body carried the credential, so the access log must still
// not have it: the log records the request line only. // not have it: the log records the request line only.
internal/handlers/target_create_test.go
+33 -74
View File
@@ -6,114 +6,73 @@ import (
"net/url" "net/url"
"strings" "strings"
"testing" "testing"
"time"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
) )
// TestHandleTargetCreate_EveryType adds a target of each type and // TestHandleTargetCreate_EveryType adds a target of each type. Each
// checks the settings it stores. Each submission carries a url and an // submission carries a url: only the http and slack types store one.
// expiry: only the http and slack types store the url, only the
// database type the expiry, and every setting a type does not use
// stays empty.
func TestHandleTargetCreate_EveryType(t *testing.T) { func TestHandleTargetCreate_EveryType(t *testing.T) {
t.Parallel() t.Parallel()
env := setupSourceTest(t) env := setupSourceTest(t)
// fields is the rest of each submission, as a query string, and // fields is the rest of each submission, as a query string.
// want holds the settings stored.
cases := []struct { cases := []struct {
targetType database.TargetType targetType database.TargetType
fields string fields string
want database.Target wantConfig string
wantAttempts int wantAttempts int
}{ }{
{ {
database.TargetTypeHTTP, database.TargetTypeHTTP, "timeout=12&max_attempts=3",
"timeout=12&max_attempts=3&forward_query=on", `{"url":"` + editOriginalURL + `","timeout":12}`, 3,
database.Target{
URL: editOriginalURL,
Headers: map[string]string{},
TimeoutSeconds: 12,
ForwardQuery: true,
}, 3,
}, },
{ {
database.TargetTypeSlack, "max_attempts=4", database.TargetTypeSlack, "max_attempts=4",
database.Target{URL: editOriginalURL}, 4, `{"webhookUrl":"` + editOriginalURL + `"}`, 4,
}, },
{ {
database.TargetTypeDatabase, "rotation=daily", database.TargetTypeDatabase, "expiry=720h",
database.Target{ `{"expiry":"720h"}`, 1,
ArchiveExpiry: 720 * time.Hour,
ArchiveRotation: rotationDaily,
}, 1,
}, },
{database.TargetTypeLog, "", database.Target{}, 1}, {database.TargetTypeLog, "", "", 1},
} }
for _, tc := range cases { for _, tc := range cases {
t.Run(string(tc.targetType), func(t *testing.T) { t.Run(string(tc.targetType), func(t *testing.T) {
t.Parallel() t.Parallel()
_, got := addTargetThroughForm( webhook := seedWebhookWithRetention(t, env.db, 30)
t, env, tc.targetType,
"url="+editOriginalURL+"&expiry=720h&"+tc.fields, form, err := url.ParseQuery(tc.fields)
require.NoError(t, err)
form.Set("name", "every-type")
form.Set("type", string(tc.targetType))
form.Set("url", editOriginalURL)
w := serveTarget(
env, http.MethodPost,
"/hook/"+webhook.ID+"/targets", form,
) )
assert.Equal(t, tc.targetType, got.Type) require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
assert.Equal(t, tc.wantAttempts, got.MaxAttempts)
assert.Equal(t, tc.want, targetSettingsOf(got)) targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1)
assert.Equal(t, tc.targetType, targets[0].Type)
assert.Equal(t, tc.wantAttempts, targets[0].MaxAttempts)
if tc.wantConfig == "" {
assert.Empty(t, targets[0].Config)
} else {
assert.JSONEq(t, tc.wantConfig, targets[0].Config)
}
}) })
} }
} }
// addTargetThroughForm adds a target of targetType named every-type to
// a new webhook through the add target form, with the rest of the form
// given as a query string, and returns the webhook and the target the
// form stored.
func addTargetThroughForm(
t *testing.T,
env *sourceTestEnv,
targetType database.TargetType,
fields string,
) (database.Webhook, database.Target) {
t.Helper()
webhook := seedWebhookWithRetention(t, env.db, 30)
form, err := url.ParseQuery(fields)
require.NoError(t, err)
form.Set("name", "every-type")
form.Set("type", string(targetType))
w := serveTarget(
env, http.MethodPost, "/hook/"+webhook.ID+"/targets", form,
)
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1)
return webhook, targets[0]
}
// targetSettingsOf returns a target's settings, URL to
// ArchiveRotation, and nothing else, for comparing with the settings a
// test expects.
func targetSettingsOf(target database.Target) database.Target {
return database.Target{
URL: target.URL,
Headers: target.Headers,
TimeoutSeconds: target.TimeoutSeconds,
ForwardQuery: target.ForwardQuery,
ArchiveExpiry: target.ArchiveExpiry,
ArchiveRotation: target.ArchiveRotation,
}
}
// TestHandleTargetCreate_RefusedFormComesBack refuses a target of each // TestHandleTargetCreate_RefusedFormComesBack refuses a target of each
// type and checks that the webhook page comes back with the add target // type and checks that the webhook page comes back with the add target
// form open on that type, the values entered, and the reason. // form open on that type, the values entered, and the reason.
internal/handlers/target_edit.go
+52 -27
View File
@@ -25,15 +25,22 @@ const (
tmplKeyMaxTimeout = "MaxTimeout" tmplKeyMaxTimeout = "MaxTimeout"
) )
// configUnreadableMessage is shown when a target's stored
// configuration does not parse. It says plainly that saving replaces
// the stored value rather than preserving it, because the form
// cannot pre-fill what it could not read.
const configUnreadableMessage = "The stored configuration for this " +
"target could not be read. Enter the values below; saving " +
"replaces the stored configuration."
// targetEditView is the display model for the target edit page: the // targetEditView is the display model for the target edit page: the
// target's row fields as stored. The values the form shows, the // target's row fields as stored. The values the form shows, the
// UNMASKED URL and headers among them, come separately, as a // UNMASKED configuration among them, come separately, as a
// targetFormInput. // targetFormInput.
// //
// It deliberately omits database.Target's settings: the form renders // It deliberately omits database.Target's raw Config blob: the form
// them from the targetFormInput, and giving the template the target's // renders named fields, and giving the template the blob as well
// URL and headers as well would put an unreviewed second path to the // would put an unreviewed second path to the credential on the page.
// credential on the page.
type targetEditView struct { type targetEditView struct {
ID string ID string
Name string Name string
@@ -41,16 +48,13 @@ type targetEditView struct {
Active bool Active bool
} }
// HandleTargetEdit shows the form to edit a target, filled with the // HandleTargetEdit shows the form to edit a target.
// target's settings as stored.
// //
// This page is the one place the full destination URL and header // This page is the one place the full destination URL and header
// values are shown: an operator cannot correct a value they cannot // values are shown. It is reachable only through the
// see. It is reachable only through the /hook/{sourceID} route // /hook/{sourceID} route group, which supplies RequireAuth and
// group, which supplies RequireAuth and NoCache, and only for a // NoCache, and only for a target of a webhook the session's user
// target of a webhook the session's user owns; masking // owns; masking (delivery.TargetView) is unchanged everywhere else.
// (delivery.TargetView) is unchanged everywhere else. Do not fill a
// form like this one on any other page.
func (h *Handlers) HandleTargetEdit() http.HandlerFunc { func (h *Handlers) HandleTargetEdit() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) { return func(w http.ResponseWriter, r *http.Request) {
webhook, target, ok := h.ownedTarget(w, r) webhook, target, ok := h.ownedTarget(w, r)
@@ -58,19 +62,34 @@ func (h *Handlers) HandleTargetEdit() http.HandlerFunc {
return return
} }
cfg, err := delivery.NewTargetConfigForm(target)
msg := ""
if err != nil {
// The error carries the parse failure, never the
// blob, so it is safe to log against the target id.
h.log.Warn(
"stored target config could not be read for editing",
"target_id", target.ID,
"error", err,
)
msg = configUnreadableMessage
}
form := targetFormInput{ form := targetFormInput{
Name: target.Name, Name: target.Name,
URL: target.URL, URL: cfg.URL,
Headers: delivery.FormatTargetHeaders(target.Headers), Headers: cfg.Headers,
Timeout: delivery.FormatTargetTimeout(target.TimeoutSeconds), Timeout: cfg.Timeout,
ForwardQuery: target.ForwardQuery, ForwardQuery: cfg.ForwardQuery,
MaxAttempts: strconv.Itoa(target.MaxAttempts), MaxAttempts: strconv.Itoa(target.MaxAttempts),
Expiry: formatArchiveExpiry(target.ArchiveExpiry), Expiry: cfg.Expiry,
Rotation: target.ArchiveRotation, Rotation: cfg.Rotation,
} }
h.renderTargetEdit( h.renderTargetEdit(
w, r, webhook, target, form, "", http.StatusOK, w, r, webhook, target, form, msg, http.StatusOK,
) )
} }
} }
@@ -106,11 +125,11 @@ func (h *Handlers) HandleTargetEditSubmit() http.HandlerFunc {
// does, so an edited destination is SSRF-validated exactly as a new // does, so an edited destination is SSRF-validated exactly as a new
// one is. // one is.
// //
// The target's type is not editable. Each type uses different // The target's type is not editable. Each type stores a different
// settings and its delivery history is recorded against the target // configuration shape and its delivery history is recorded against
// row, so changing the type of an existing target is really the // the target row, so changing the type of an existing target is
// creation of a different one. The stored type decides which fields // really the creation of a different one. The stored type decides
// the form offers and which settings are set. // which fields the form offers and which builder runs.
func (h *Handlers) applyTargetEdit( func (h *Handlers) applyTargetEdit(
w http.ResponseWriter, w http.ResponseWriter,
r *http.Request, r *http.Request,
@@ -123,7 +142,13 @@ func (h *Handlers) applyTargetEdit(
// as stored, for the page shown again when the save is refused. // as stored, for the page shown again when the save is refused.
edited := *target edited := *target
errMsg := h.setTargetFromForm(r.Context(), &edited, in) errMsg, err := h.setTargetFromForm(r.Context(), &edited, in)
if err != nil {
h.serverError(w, r, "failed to encode target config", err)
return
}
if errMsg != "" { if errMsg != "" {
h.renderTargetEdit( h.renderTargetEdit(
w, r, webhook, target, in, errMsg, http.StatusBadRequest, w, r, webhook, target, in, errMsg, http.StatusBadRequest,
@@ -135,7 +160,7 @@ func (h *Handlers) applyTargetEdit(
// A new name renames the archive file before it is saved (see // A new name renames the archive file before it is saved (see
// archive.Writers.Rename). If either step fails, it goes back to // archive.Writers.Rename). If either step fails, it goes back to
// the name that is still stored. // the name that is still stored.
err := h.renameTargetArchive( err = h.renameTargetArchive(
target, webhook.Name, target.Name, edited.Name, target, webhook.Name, target.Name, edited.Name,
) )
if err == nil { if err == nil {
internal/handlers/target_edit_test.go
+64 -116
View File
@@ -2,18 +2,19 @@ package handlers_test
import ( import (
"context" "context"
"encoding/json"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"net/url" "net/url"
"strings" "strings"
"testing" "testing"
"time"
"github.com/go-chi/chi" "github.com/go-chi/chi"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"gorm.io/gorm/clause" "gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
) )
// The destinations the target edit tests configure. Both are literal // The destinations the target edit tests configure. Both are literal
@@ -149,6 +150,27 @@ func storedTarget(
return target return target
} }
// storedHTTPConfig reloads a target and parses its stored HTTP
// configuration.
func storedHTTPConfig(
t *testing.T,
env *sourceTestEnv,
targetID string,
) delivery.HTTPTargetConfig {
t.Helper()
var cfg delivery.HTTPTargetConfig
require.NoError(
t,
json.Unmarshal(
[]byte(storedTarget(t, env, targetID).Config), &cfg,
),
)
return cfg
}
// editForm is the fully populated edit submission for an HTTP // editForm is the fully populated edit submission for an HTTP
// target. // target.
func editForm(targetURL, headers, timeout string) url.Values { func editForm(targetURL, headers, timeout string) url.Values {
@@ -175,9 +197,12 @@ func submitTargetEdit(
) )
} }
// TestHandleTargetCreate_Configuration proves the add target form // TestHandleTargetCreate_Configuration covers the half of the gap
// stores an HTTP target's headers and timeout, and leaves them empty // that is not about editing at all: HTTPTargetConfig has carried
// when the form leaves them empty. // Headers and Timeout, and the delivery path has honoured them, but
// the create form wrote {"url":...} and nothing else, so a
// destination needing an Authorization header could not be
// configured through the UI at all.
func TestHandleTargetCreate_Configuration(t *testing.T) { func TestHandleTargetCreate_Configuration(t *testing.T) {
t.Parallel() t.Parallel()
@@ -188,9 +213,9 @@ func TestHandleTargetCreate_Configuration(t *testing.T) {
assertCreateStoresHeadersAndTimeout(t, env) assertCreateStoresHeadersAndTimeout(t, env)
}) })
t.Run("without them stores only the url", func(t *testing.T) { t.Run("without them keeps a url-only config", func(t *testing.T) {
t.Parallel() t.Parallel()
assertCreateStoresOnlyTheURL(t, env) assertCreateKeepsURLOnlyConfig(t, env)
}) })
} }
@@ -203,31 +228,33 @@ func assertCreateStoresHeadersAndTimeout(
t, env, editAuthHeader+"\nX-Tenant: acme\n", "12", t, env, editAuthHeader+"\nX-Tenant: acme\n", "12",
) )
stored := storedTarget(t, env, target.ID) cfg := storedHTTPConfig(t, env, target.ID)
assert.Equal(t, editOriginalURL, stored.URL) assert.Equal(t, editOriginalURL, cfg.URL)
assert.Equal(t, 12, stored.TimeoutSeconds) assert.Equal(t, 12, cfg.Timeout)
assert.Equal( assert.Equal(
t, t,
map[string]string{ map[string]string{
"Authorization": "Bearer " + editBearerSecret, "Authorization": "Bearer " + editBearerSecret,
"X-Tenant": "acme", "X-Tenant": "acme",
}, },
stored.Headers, cfg.Headers,
) )
} }
func assertCreateStoresOnlyTheURL( // Without the new fields the stored shape must be the same
// {"url":...} the create form wrote before they existed, so no
// existing target's configuration is rewritten by this change.
func assertCreateKeepsURLOnlyConfig(
t *testing.T, env *sourceTestEnv, t *testing.T, env *sourceTestEnv,
) { ) {
t.Helper() t.Helper()
_, target := seedHTTPTarget(t, env, "", "") _, target := seedHTTPTarget(t, env, "", "")
assert.Equal(t, editOriginalURL, target.URL) assert.JSONEq(
assert.Empty(t, target.Headers) t, `{"url":"`+editOriginalURL+`"}`, target.Config,
assert.Zero(t, target.TimeoutSeconds) )
assert.False(t, target.ForwardQuery)
} }
// TestHandleTargetEditSubmit_Saves is the round trip the issue asks // TestHandleTargetEditSubmit_Saves is the round trip the issue asks
@@ -272,8 +299,13 @@ func assertEditChangesDestination(
) )
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String()) require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
assert.Equal(
t,
editReplacedURL,
storedHTTPConfig(t, env, target.ID).URL,
)
reloaded := storedTarget(t, env, target.ID) reloaded := storedTarget(t, env, target.ID)
assert.Equal(t, editReplacedURL, reloaded.URL)
assert.Equal(t, "edited-name", reloaded.Name) assert.Equal(t, "edited-name", reloaded.Name)
assert.Equal(t, 5, reloaded.MaxAttempts) assert.Equal(t, 5, reloaded.MaxAttempts)
assert.Equal( assert.Equal(
@@ -301,16 +333,16 @@ func assertEditRoundTripsHeadersAndTimeout(
) )
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String()) require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
stored := storedTarget(t, env, target.ID) cfg := storedHTTPConfig(t, env, target.ID)
assert.Equal(t, 21, stored.TimeoutSeconds) assert.Equal(t, 21, cfg.Timeout)
assert.Equal( assert.Equal(
t, t,
map[string]string{ map[string]string{
"Authorization": "Bearer rotated", "Authorization": "Bearer rotated",
"X-Trace": "on", "X-Trace": "on",
}, },
stored.Headers, cfg.Headers,
) )
} }
@@ -330,10 +362,10 @@ func assertEditClearingRemovesThem(
) )
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String()) require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
stored := storedTarget(t, env, target.ID) cfg := storedHTTPConfig(t, env, target.ID)
assert.Empty(t, stored.Headers) assert.Empty(t, cfg.Headers)
assert.Zero(t, stored.TimeoutSeconds) assert.Zero(t, cfg.Timeout)
} }
// Delivery attempts are offered only by the forms for target types // Delivery attempts are offered only by the forms for target types
@@ -410,90 +442,6 @@ func TestHandleTargetEdit_CallsTheDatabaseTypeArchive(t *testing.T) {
assert.Contains(t, page, `class="label">Archive rotation</label>`) assert.Contains(t, page, `class="label">Archive rotation</label>`)
} }
// TestHandleTargetEdit_EveryType edits a target of each type: the edit
// form starts on the settings the add target form stored, and saving
// it stores the settings submitted.
func TestHandleTargetEdit_EveryType(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
// created and edited are the add target form's and the edit
// form's fields, as query strings; shown is what the edit form
// must show of created; want is the settings edited stores.
cases := []struct {
targetType database.TargetType
created, edited string
shown []string
want database.Target
}{
{
database.TargetTypeHTTP,
"url=" + editOriginalURL +
"&headers=X-Tenant:+acme&timeout=7&forward_query=on",
"url=" + editReplacedURL + "&timeout=9",
[]string{
`value="` + editOriginalURL + `"`,
"X-Tenant: acme",
`value="7"`,
`name="forward_query" value="on" checked`,
},
database.Target{
URL: editReplacedURL,
Headers: map[string]string{},
TimeoutSeconds: 9,
},
},
{
database.TargetTypeSlack,
"url=" + editOriginalURL,
"url=" + editReplacedURL,
[]string{`value="` + editOriginalURL + `"`},
database.Target{URL: editReplacedURL},
},
{
database.TargetTypeDatabase,
"expiry=720h&rotation=daily",
"expiry=24h&rotation=hourly",
[]string{
`<option value="720h" selected>`,
`<option value="daily" selected>`,
},
database.Target{
ArchiveExpiry: 24 * time.Hour,
ArchiveRotation: rotationHourly,
},
},
{database.TargetTypeLog, "", "", nil, database.Target{}},
}
for _, tc := range cases {
t.Run(string(tc.targetType), func(t *testing.T) {
t.Parallel()
webhook, target := addTargetThroughForm(
t, env, tc.targetType, tc.created,
)
page := targetEditPage(t, env, webhook.ID, target.ID)
for _, shown := range tc.shown {
assert.Contains(t, page, shown)
}
form, err := url.ParseQuery(tc.edited)
require.NoError(t, err)
form.Set("name", "edited")
w := submitTargetEdit(env, webhook.ID, target.ID, form)
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
got := storedTarget(t, env, target.ID)
assert.Equal(t, "edited", got.Name)
assert.Equal(t, tc.want, targetSettingsOf(got))
})
}
}
// TestHandleTarget_ForwardQuery covers the HTTP target's setting that // TestHandleTarget_ForwardQuery covers the HTTP target's setting that
// passes each event's query string on to it: the add target form // passes each event's query string on to it: the add target form
// stores it checked, the edit form starts with it checked and turns it // stores it checked, the edit form starts with it checked and turns it
@@ -519,7 +467,7 @@ func TestHandleTarget_ForwardQuery(t *testing.T) {
targets := targetsForWebhook(t, env.db, webhook.ID) targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1) require.Len(t, targets, 1)
assert.True(t, storedTarget(t, env, targets[0].ID).ForwardQuery) assert.True(t, storedHTTPConfig(t, env, targets[0].ID).ForwardQuery)
w = serveTarget( w = serveTarget(
env, http.MethodGet, targetsPath+"/"+targets[0].ID+"/edit", nil, env, http.MethodGet, targetsPath+"/"+targets[0].ID+"/edit", nil,
@@ -531,7 +479,7 @@ func TestHandleTarget_ForwardQuery(t *testing.T) {
w = submitTargetEdit(env, webhook.ID, targets[0].ID, edit) w = submitTargetEdit(env, webhook.ID, targets[0].ID, edit)
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String()) require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
assert.False(t, storedTarget(t, env, targets[0].ID).ForwardQuery) assert.False(t, storedHTTPConfig(t, env, targets[0].ID).ForwardQuery)
edit.Set("url", editBlockedURL) edit.Set("url", editBlockedURL)
edit.Set("forward_query", "on") edit.Set("forward_query", "on")
@@ -620,7 +568,7 @@ func assertEditRejectsBlockedDestination(
assert.Contains(t, w.Body.String(), "Invalid target URL") assert.Contains(t, w.Body.String(), "Invalid target URL")
assert.Equal( assert.Equal(
t, editOriginalURL, t, editOriginalURL,
storedTarget(t, env, target.ID).URL, storedHTTPConfig(t, env, target.ID).URL,
"a rejected edit must leave the stored config alone", "a rejected edit must leave the stored config alone",
) )
} }
@@ -649,9 +597,9 @@ func assertEditIgnoresQueryString(
assert.Equal(t, http.StatusBadRequest, w.Code) assert.Equal(t, http.StatusBadRequest, w.Code)
stored := storedTarget(t, env, target.ID) cfg := storedHTTPConfig(t, env, target.ID)
assert.Equal(t, editOriginalURL, stored.URL) assert.Equal(t, editOriginalURL, cfg.URL)
assert.Empty(t, stored.Headers) assert.Empty(t, cfg.Headers)
} }
func assertEditRejectsHeaders( func assertEditRejectsHeaders(
@@ -669,7 +617,7 @@ func assertEditRejectsHeaders(
assert.Equal(t, http.StatusBadRequest, w.Code) assert.Equal(t, http.StatusBadRequest, w.Code)
assert.Contains(t, w.Body.String(), "Invalid headers") assert.Contains(t, w.Body.String(), "Invalid headers")
assert.Empty( assert.Empty(
t, storedTarget(t, env, target.ID).Headers, t, storedHTTPConfig(t, env, target.ID).Headers,
"a rejected header must not be stored", "a rejected header must not be stored",
) )
} }
@@ -689,7 +637,7 @@ func assertEditRejectsTimeout(
assert.Equal(t, http.StatusBadRequest, w.Code) assert.Equal(t, http.StatusBadRequest, w.Code)
assert.Contains(t, w.Body.String(), "Invalid timeout") assert.Contains(t, w.Body.String(), "Invalid timeout")
assert.Equal( assert.Equal(
t, 9, storedTarget(t, env, target.ID).TimeoutSeconds, t, 9, storedHTTPConfig(t, env, target.ID).Timeout,
"a rejected timeout must leave the stored one alone", "a rejected timeout must leave the stored one alone",
) )
} }
@@ -809,7 +757,7 @@ func assertTargetOfAnotherWebhook404s(
assert.Equal( assert.Equal(
t, editOriginalURL, t, editOriginalURL,
storedTarget(t, env, target.ID).URL, storedHTTPConfig(t, env, target.ID).URL,
) )
} }
@@ -830,7 +778,7 @@ func assertWebhookOfAnotherUser404s(
target := seedConfiguredTarget( target := seedConfiguredTarget(
t, env.db, other.ID, database.TargetTypeHTTP, t, env.db, other.ID, database.TargetTypeHTTP,
database.Target{URL: editOriginalURL}, `{"url":"`+editOriginalURL+`"}`,
) )
w := serveTarget( w := serveTarget(
internal/handlers/target_list.go
+4 -1
View File
@@ -243,7 +243,10 @@ func readTargetDeliveries(
func (h *Handlers) archiveFileView( func (h *Handlers) archiveFileView(
webhook *database.Webhook, target *database.Target, now time.Time, webhook *database.Webhook, target *database.Target, now time.Time,
) *ArchiveFileView { ) *ArchiveFileView {
current := archive.PathAt(h.dbMgr, webhook, target, now) current, err := archive.PathAt(h.dbMgr, webhook, target, now)
if err != nil {
return h.archiveUnreadable(&ArchiveFileView{}, target, err)
}
view := &ArchiveFileView{Name: filepath.Base(current)} view := &ArchiveFileView{Name: filepath.Base(current)}
internal/handlers/target_toggle_test.go
+3 -1
View File
@@ -60,5 +60,7 @@ func TestHandleTargetToggle_DoesNotUndoAnEdit(t *testing.T) {
assert.False(t, stored.Active) assert.False(t, stored.Active)
assert.Equal(t, "edited-name", stored.Name) assert.Equal(t, "edited-name", stored.Name)
assert.Equal(t, 5, stored.MaxAttempts) assert.Equal(t, 5, stored.MaxAttempts)
assert.Equal(t, editReplacedURL, stored.URL) assert.Equal(
t, editReplacedURL, storedHTTPConfig(t, env, tgt.ID).URL,
)
} }
internal/handlers/webhook.go
+259 -20
View File
@@ -2,11 +2,14 @@ package handlers
import ( import (
"encoding/json" "encoding/json"
"fmt"
"io" "io"
"net/http" "net/http"
"github.com/go-chi/chi" "github.com/go-chi/chi"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/middleware" "sneak.berlin/go/webhooker/internal/middleware"
) )
@@ -67,9 +70,8 @@ func (h *Handlers) HandleWebhook() http.HandlerFunc {
} }
} }
// processWebhookRequest reads the request into an event and has the // processWebhookRequest reads the body, serializes headers, loads
// delivery engine store it, with a delivery for each of the webhook's // targets, and delivers the event.
// active targets, and answers the sender.
// //
// Nothing about the request itself is authenticated: the entrypoint // Nothing about the request itself is authenticated: the entrypoint
// UUID in the path is the credential, and reaching here means it // UUID in the path is the credential, and reaching here means it
@@ -91,24 +93,30 @@ func (h *Handlers) processWebhookRequest(
return return
} }
event := &database.Event{ targets, err := h.loadActiveTargets(entrypoint.WebhookID)
WebhookID: entrypoint.WebhookID,
EntrypointID: entrypoint.ID,
Method: r.Method,
RawQuery: r.URL.RawQuery,
Headers: string(headersJSON),
ContentType: r.Header.Get("Content-Type"),
Body: string(body),
}
deliveries, err := h.engine.StoreEvent(event)
if err != nil { if err != nil {
h.receiverError(w, "failed to store webhook event", err) h.receiverError(w, "failed to query targets", err)
return return
} }
h.finishWebhookResponse(w, event, entrypoint, deliveries) h.createAndDeliverEvent(
w, r, entrypoint, body, headersJSON, targets,
)
}
// loadActiveTargets returns all active targets for a webhook.
func (h *Handlers) loadActiveTargets(
webhookID string,
) ([]database.Target, error) {
var targets []database.Target
err := h.db.DB().Where(
"webhook_id = ? AND active = ?",
webhookID, true,
).Find(&targets).Error
return targets, err
} }
// lookupEntrypoint finds an entrypoint by UUID path. // lookupEntrypoint finds an entrypoint by UUID path.
@@ -169,6 +177,28 @@ func (h *Handlers) readWebhookBody(
return body, true return body, true
} }
// createAndDeliverEvent stores the received event, fans it out to the
// webhook's targets, and answers the sender.
func (h *Handlers) createAndDeliverEvent(
w http.ResponseWriter,
r *http.Request,
entrypoint database.Entrypoint,
body, headersJSON []byte,
targets []database.Target,
) {
event, tasks, err := h.createAndFanOut(
requestEventSource(r, entrypoint, headersJSON, body),
targets,
)
if err != nil {
h.receiverError(w, "failed to store webhook event", err)
return
}
h.finishWebhookResponse(w, event, entrypoint, tasks)
}
// receiverError logs an error and answers the sender with a plain-text // receiverError logs an error and answers the sender with a plain-text
// 500. The receiver's answers are for programs, so it never sends the // 500. The receiver's answers are for programs, so it never sends the
// error page the web UI uses. // error page the web UI uses.
@@ -182,19 +212,172 @@ func (h *Handlers) receiverError(
) )
} }
// finishWebhookResponse logs the stored event, with how many // eventSource carries the fields a new event is built from. The
// deliveries it has, and writes the HTTP response. // receiver fills it from the live request; the resubmit handler fills
// it from a stored event. Both then go through createAndFanOut, so an
// event is constructed and fanned out in one place however it entered
// the system.
type eventSource struct {
WebhookID string
EntrypointID string
Method string
RawQuery string
HeadersJSON string
ContentType string
Body []byte
// ResubmittedFromID names the event this one copies. Only the
// resubmit path sets it.
ResubmittedFromID *string
}
// event builds the row this source stores.
func (s eventSource) event() *database.Event {
return &database.Event{
WebhookID: s.WebhookID,
EntrypointID: s.EntrypointID,
Method: s.Method,
RawQuery: s.RawQuery,
Headers: s.HeadersJSON,
Body: string(s.Body),
BodyBytes: int64(len(s.Body)),
ContentType: s.ContentType,
ResubmittedFromID: s.ResubmittedFromID,
}
}
// requestEventSource describes the event a live receiver request
// stores.
func requestEventSource(
r *http.Request,
entrypoint database.Entrypoint,
headersJSON, body []byte,
) eventSource {
return eventSource{
WebhookID: entrypoint.WebhookID,
EntrypointID: entrypoint.ID,
Method: r.Method,
RawQuery: r.URL.RawQuery,
HeadersJSON: string(headersJSON),
ContentType: r.Header.Get("Content-Type"),
Body: body,
}
}
// createAndFanOut writes the event and one pending delivery per target,
// and adds them to the webhook's running totals, in a single
// transaction, then hands the tasks to the delivery engine. Every
// event is created here, received or resubmitted, so a resubmitted
// event is retried, SSRF-guarded and circuit-broken exactly as a
// received one is. Per-delivery replay is the one other path that
// creates a delivery: it adds one to an existing event without
// coming through here.
//
// The tasks are returned as well as queued, so a caller can report how
// many targets the event went to.
func (h *Handlers) createAndFanOut(
src eventSource,
targets []database.Target,
) (*database.Event, []delivery.Task, error) {
webhookDB, err := h.dbMgr.GetDB(src.WebhookID)
if err != nil {
return nil, nil, fmt.Errorf(
"getting webhook database: %w", err,
)
}
tx := webhookDB.Begin()
if tx.Error != nil {
return nil, nil, fmt.Errorf(
"beginning transaction: %w", tx.Error,
)
}
event := src.event()
err = tx.Create(event).Error
if err != nil {
tx.Rollback()
return nil, nil, fmt.Errorf("creating event: %w", err)
}
tasks, err := buildDeliveryTasks(
tx, event, targets, inlineBody(src.Body),
)
if err != nil {
tx.Rollback()
return nil, nil, err
}
err = database.AddEventTotals(tx, database.EventTotals{
Events: 1, LastEventAt: &event.CreatedAt,
})
if err != nil {
tx.Rollback()
return nil, nil, err
}
// A resubmitted copy did not arrive on its entrypoint's URL, so it
// leaves the entrypoint's last event as it is.
if src.ResubmittedFromID == nil {
err = database.AddEntrypointTotals(tx, database.EntrypointTotals{
EntrypointID: event.EntrypointID, LastEventAt: event.CreatedAt,
})
if err != nil {
tx.Rollback()
return nil, nil, err
}
}
err = tx.Commit().Error
if err != nil {
return nil, nil, fmt.Errorf(
"committing transaction: %w", err,
)
}
// Counted here, after the commit: an event exists once it is
// durably stored, which is what the delivery counters are
// compared against on a dashboard. A resubmitted event counts
// too, because it produces deliveries that the delivery side
// counts; the resubmit counter is what separates the two.
h.mtr.EventReceived()
if len(tasks) > 0 {
h.notifier.Notify(tasks)
}
return event, tasks, nil
}
// inlineBody returns a pointer to body as a string if it fits
// within the inline size limit, or nil otherwise.
func inlineBody(body []byte) *string {
if len(body) < delivery.MaxInlineBodySize {
s := string(body)
return &s
}
return nil
}
// finishWebhookResponse logs the event and writes the HTTP response.
func (h *Handlers) finishWebhookResponse( func (h *Handlers) finishWebhookResponse(
w http.ResponseWriter, w http.ResponseWriter,
event *database.Event, event *database.Event,
entrypoint database.Entrypoint, entrypoint database.Entrypoint,
deliveries int, tasks []delivery.Task,
) { ) {
h.log.Info("webhook event created", h.log.Info("webhook event created",
"event_id", event.ID, "event_id", event.ID,
"webhook_id", entrypoint.WebhookID, "webhook_id", entrypoint.WebhookID,
"entrypoint_id", entrypoint.ID, "entrypoint_id", entrypoint.ID,
"target_count", deliveries, "target_count", len(tasks),
) )
w.WriteHeader(http.StatusOK) w.WriteHeader(http.StatusOK)
@@ -206,3 +389,59 @@ func (h *Handlers) finishWebhookResponse(
) )
} }
} }
// buildDeliveryTasks creates one pending delivery per target in the
// transaction, adds each to its target's totals, and returns the tasks
// for the delivery engine. The caller owns the transaction and rolls
// it back on error.
func buildDeliveryTasks(
tx *gorm.DB,
event *database.Event,
targets []database.Target,
bodyPtr *string,
) ([]delivery.Task, error) {
tasks := make([]delivery.Task, 0, len(targets))
for i := range targets {
dlv := &database.Delivery{
EventID: event.ID,
TargetID: targets[i].ID,
Status: database.DeliveryStatusPending,
}
err := tx.Create(dlv).Error
if err != nil {
return nil, fmt.Errorf(
"creating delivery for target %s: %w",
targets[i].ID, err,
)
}
err = database.AddTargetTotals(tx, database.TargetTotals{
TargetID: targets[i].ID, Deliveries: 1,
})
if err != nil {
return nil, err
}
tasks = append(tasks, delivery.Task{
DeliveryID: dlv.ID,
EventID: event.ID,
WebhookID: event.WebhookID,
EntrypointID: event.EntrypointID,
TargetID: targets[i].ID,
TargetName: targets[i].Name,
TargetType: targets[i].Type,
TargetConfig: targets[i].Config,
MaxAttempts: targets[i].MaxAttempts,
Method: event.Method,
RawQuery: event.RawQuery,
Headers: event.Headers,
ContentType: event.ContentType,
Body: bodyPtr,
AttemptNum: 1,
})
}
return tasks, nil
}
internal/handlers/webhook_create.go
+12 -6
View File
@@ -115,7 +115,13 @@ func (h *Handlers) HandleSourceCreateSubmit() http.HandlerFunc {
return return
} }
targets, errMsg := h.newWebhookTargets(r.Context(), in) targets, errMsg, err := h.newWebhookTargets(r.Context(), in)
if err != nil {
h.serverError(w, r, "failed to encode target config", err)
return
}
if errMsg != "" { if errMsg != "" {
refuse(errMsg) refuse(errMsg)
@@ -141,7 +147,7 @@ func (h *Handlers) HandleSourceCreateSubmit() http.HandlerFunc {
func (h *Handlers) newWebhookTargets( func (h *Handlers) newWebhookTargets(
ctx context.Context, ctx context.Context,
in sourceFormInput, in sourceFormInput,
) ([]*database.Target, string) { ) ([]*database.Target, string, error) {
var requested []targetFormInput var requested []targetFormInput
if in.HTTPURL != "" { if in.HTTPURL != "" {
@@ -164,15 +170,15 @@ func (h *Handlers) newWebhookTargets(
targets := make([]*database.Target, 0, len(requested)) targets := make([]*database.Target, 0, len(requested))
for _, form := range requested { for _, form := range requested {
target, errMsg := h.newTarget(ctx, "", form) target, errMsg, err := h.newTarget(ctx, "", form)
if errMsg != "" { if err != nil || errMsg != "" {
return nil, errMsg return nil, errMsg, err
} }
targets = append(targets, target) targets = append(targets, target)
} }
return targets, "" return targets, "", nil
} }
// createWebhookWithEntrypoint creates a webhook, its default // createWebhookWithEntrypoint creates a webhook, its default
internal/handlers/webhook_detail.go
+1 -1
View File
@@ -109,7 +109,7 @@ func (h *Handlers) renderSourceDetail(
data := map[string]any{ data := map[string]any{
tmplKeyWebhook: &webhook, tmplKeyWebhook: &webhook,
// Targets are projected to a display-safe view: a // Targets are projected to a display-safe view: a
// target's URL and headers hold a credential, and they // target's stored config blob holds a credential, and it
// must never reach a template. // must never reach a template.
"Entrypoints": entrypointViews, "Entrypoints": entrypointViews,
"Targets": h.targetRows(&webhook, targets), "Targets": h.targetRows(&webhook, targets),
internal/handlers/webhook_stats_test.go
+1 -1
View File
@@ -153,7 +153,7 @@ func seedStatsHistory(
first := seedConfiguredTarget( first := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP, t, db, wh.ID, database.TargetTypeHTTP,
database.Target{URL: replayTargetURL}, `{"url":"`+replayTargetURL+`"}`,
) )
second := seedTarget(t, db, wh.ID, database.TargetTypeLog) second := seedTarget(t, db, wh.ID, database.TargetTypeLog)
internal/handlers/webhook_test.go
-84
View File
@@ -206,87 +206,3 @@ func receivedLine(t *testing.T, buf *bytes.Buffer) map[string]any {
return found[0] return found[0]
} }
// TestHandleWebhook_StoresTheEvent proves an accepted request is
// answered with the receiver's 200 and body, and that the event is
// stored with a pending delivery to the webhook's active target.
func TestHandleWebhook_StoresTheEvent(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
webhook := seedWebhook(t, db)
entrypoint := seedEntrypoint(t, db, webhook.ID)
target := seedConfiguredTarget(
t, db, webhook.ID, database.TargetTypeHTTP,
database.Target{URL: replayTargetURL},
)
w := httptest.NewRecorder()
h.HandleWebhook().ServeHTTP(w, postRequest(
"/h/"+entrypoint.Path, nil,
map[string]string{"uuid": entrypoint.Path},
))
require.Equal(t, http.StatusOK, w.Code)
assert.JSONEq(t, `{"status":"ok"}`, w.Body.String())
webhookDB, err := dbMgr.GetDB(webhook.ID)
require.NoError(t, err)
events := listEvents(t, webhookDB)
require.Len(t, events, 1)
assert.Equal(t, entrypoint.ID, events[0].EntrypointID)
deliveries := listDeliveries(t, webhookDB, events[0].ID)
require.Len(t, deliveries, 1)
assert.Equal(t, target.ID, deliveries[0].TargetID)
assert.Equal(t, database.DeliveryStatusPending, deliveries[0].Status)
}
// TestHandleWebhook_FailedStoreAnswersPlainError proves that an event
// the engine cannot store is answered with the receiver's plain-text
// 500, and that nothing of it is left stored.
func TestHandleWebhook_FailedStoreAnswersPlainError(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
webhook := seedWebhook(t, db)
entrypoint := seedEntrypoint(t, db, webhook.ID)
// Without its event totals table the webhook's database refuses
// the totals write, which comes after the event row.
webhookDB, err := dbMgr.GetDB(webhook.ID)
require.NoError(t, err)
require.NoError(t,
webhookDB.Migrator().DropTable(&database.EventTotals{}))
w := httptest.NewRecorder()
h.HandleWebhook().ServeHTTP(w, postRequest(
"/h/"+entrypoint.Path, nil,
map[string]string{"uuid": entrypoint.Path},
))
assert.Equal(t, http.StatusInternalServerError, w.Code)
assert.Equal(t, "Internal server error\n", w.Body.String())
assert.Empty(t, listEvents(t, webhookDB))
}
internal/resetpw/resetpw_test.go
+5 -6
View File
@@ -131,6 +131,10 @@ func run(
return code, stdout.String(), stderr.String() return code, stdout.String(), stderr.String()
} }
type noopNotifier struct{}
func (n *noopNotifier) Notify([]delivery.Task) {}
type noopArchives struct { type noopArchives struct {
renameMu sync.Mutex renameMu sync.Mutex
} }
@@ -186,12 +190,7 @@ func newServerApp(
database.NewWebhookDBManager, database.NewWebhookDBManager,
healthcheck.New, healthcheck.New,
session.New, session.New,
archive.NewWriters, func() delivery.Notifier { return &noopNotifier{} },
// The engine is built on a lifecycle that is never started,
// so no worker runs.
func(p delivery.EngineParams) *delivery.Engine {
return delivery.New(fxtest.NewLifecycle(t), p)
},
func() archive.Archives { return &noopArchives{} }, func() archive.Archives { return &noopArchives{} },
func() delivery.CircuitBreakers { func() delivery.CircuitBreakers {
return &noopCircuitBreakers{} return &noopCircuitBreakers{}
internal/server/alpine_browser_test.go
+19 -39
View File
@@ -1183,26 +1183,16 @@ func checkNewWebhookTargets(
) { ) {
t.Helper() t.Helper()
// Each value the pruning choice submits and the expiry it stores, // Each value the pruning choice submits, after an empty one that
// after an empty one that stands for archive left off. // stands for archive left off.
expiries := []struct { expiries := []string{
value string "", "never", "1h", "12h", "24h", "720h", "2160h", "8760h",
stored time.Duration
}{
{"", 0},
{"never", 0},
{"1h", time.Hour},
{"12h", 12 * time.Hour},
{"24h", 24 * time.Hour},
{"720h", 720 * time.Hour},
{"2160h", 2160 * time.Hour},
{"8760h", 8760 * time.Hour},
} }
for _, httpURL := range []string{"", publicTargetURL} { for _, httpURL := range []string{"", publicTargetURL} {
for _, expiry := range expiries { for _, expiry := range expiries {
name := "url=" + httpURL + " archive=" + expiry.value name := "url=" + httpURL + " archive=" + expiry
want := map[database.TargetType]database.Target{} want := map[database.TargetType]string{}
require.NoError(t, chromedp.Run( require.NoError(t, chromedp.Run(
ctx, ctx,
@@ -1215,12 +1205,10 @@ func checkNewWebhookTargets(
"#http_url", httpURL, chromedp.ByQuery, "#http_url", httpURL, chromedp.ByQuery,
))) )))
want[database.TargetTypeHTTP] = database.Target{ want[database.TargetTypeHTTP] = `{"url":"` + httpURL + `"}`
URL: httpURL, Headers: map[string]string{},
}
} }
if expiry.value != "" { if expiry != "" {
// The choice showing moves Create down, so it is // The choice showing moves Create down, so it is
// waited for before Create is clicked. // waited for before Create is clicked.
click(ctx, t, archiveBox) click(ctx, t, archiveBox)
@@ -1228,30 +1216,29 @@ func checkNewWebhookTargets(
"%s: checking archive does not show the pruning choice", "%s: checking archive does not show the pruning choice",
name) name)
require.NoError(t, chromedp.Run(ctx, chromedp.SetValue( require.NoError(t, chromedp.Run(ctx, chromedp.SetValue(
pruningChoice, expiry.value, chromedp.BySearch, pruningChoice, expiry, chromedp.BySearch,
))) )))
// The rotation choice is left on none. // The rotation choice is left on none.
want[database.TargetTypeDatabase] = database.Target{ want[database.TargetTypeDatabase] = `{"expiry":"` + expiry +
ArchiveExpiry: expiry.stored, ArchiveRotation: "none", `","rotation":"none"}`
}
} }
clickAndLoad(ctx, t, createButton) clickAndLoad(ctx, t, createButton)
require.Truef(t, shown(ctx, `//h1[text()="`+name+`"]`), require.Truef(t, shown(ctx, `//h1[text()="`+name+`"]`),
"%s: the new webhook's page does not open", name) "%s: the new webhook's page does not open", name)
assert.Equalf(t, want, targetSettings(t, env, name), assert.Equalf(t, want, targetConfigs(t, env, name),
"%s: the webhook does not have the targets asked for", name) "%s: the webhook does not have the targets asked for", name)
} }
} }
} }
// targetSettings reads the targets of the webhook named name, and // targetConfigs reads the targets of the webhook named name, and
// returns each one's stored settings by its type. // returns each one's stored configuration by its type.
func targetSettings( func targetConfigs(
t *testing.T, env *testEnv, name string, t *testing.T, env *testEnv, name string,
) map[database.TargetType]database.Target { ) map[database.TargetType]string {
t.Helper() t.Helper()
var ( var (
@@ -1264,19 +1251,12 @@ func targetSettings(
require.NoError(t, env.db.DB(). require.NoError(t, env.db.DB().
Where("webhook_id = ?", webhook.ID).Find(&targets).Error) Where("webhook_id = ?", webhook.ID).Find(&targets).Error)
settings := map[database.TargetType]database.Target{} configs := map[database.TargetType]string{}
for _, target := range targets { for _, target := range targets {
settings[target.Type] = database.Target{ configs[target.Type] = target.Config
URL: target.URL,
Headers: target.Headers,
TimeoutSeconds: target.TimeoutSeconds,
ForwardQuery: target.ForwardQuery,
ArchiveExpiry: target.ArchiveExpiry,
ArchiveRotation: target.ArchiveRotation,
}
} }
return settings return configs
} }
// checkRefusedNewWebhook submits the new webhook page with archive // checkRefusedNewWebhook submits the new webhook page with archive
internal/server/routes.go
+1 -1
View File
@@ -311,7 +311,7 @@ func (s *Server) setupSourceRoutes() {
r.Post("/targets", s.h.HandleTargetCreate()) r.Post("/targets", s.h.HandleTargetCreate())
// The edit form is the one page that renders a target's // The edit form is the one page that renders a target's
// destination URL and header values in full; see // destination URL and header values in full; see
// Handlers.HandleTargetEdit. It belongs to this group for // delivery.TargetConfigForm. It belongs to this group for
// its RequireAuth and NoCache, which are what keep that // its RequireAuth and NoCache, which are what keep that
// exception from reaching an unauthenticated request or a // exception from reaching an unauthenticated request or a
// shared cache. // shared cache.
internal/server/routes_test.go
+6 -7
View File
@@ -48,6 +48,10 @@ const (
metricsAuthValue = "s3cret" metricsAuthValue = "s3cret"
) )
type noopNotifier struct{}
func (n *noopNotifier) Notify([]delivery.Task) {}
// noopArchives satisfies handlers.New's archive.Archives // noopArchives satisfies handlers.New's archive.Archives
// dependency. No test here checks what gets evicted or renamed, so // dependency. No test here checks what gets evicted or renamed, so
// it records nothing. Its rename lock is a plain mutex. // it records nothing. Its rename lock is a plain mutex.
@@ -141,12 +145,7 @@ func newTestEnvWithConfig(
database.NewWebhookDBManager, database.NewWebhookDBManager,
healthcheck.New, healthcheck.New,
session.New, session.New,
archive.NewWriters, func() delivery.Notifier { return &noopNotifier{} },
// The engine is built on a lifecycle that is never started,
// so no worker runs and nothing is sent anywhere.
func(p delivery.EngineParams) *delivery.Engine {
return delivery.New(fxtest.NewLifecycle(t), p)
},
func() archive.Archives { return &noopArchives{} }, func() archive.Archives { return &noopArchives{} },
func() delivery.CircuitBreakers { func() delivery.CircuitBreakers {
return &noopCircuitBreakers{} return &noopCircuitBreakers{}
@@ -415,7 +414,7 @@ func (e *testEnv) seedTarget(
Name: "routed-target", Name: "routed-target",
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Active: true, Active: true,
URL: "http://93.184.216.34/hook", Config: `{"url":"http://93.184.216.34/hook"}`,
MaxAttempts: 1, MaxAttempts: 1,
} }