A target's settings move from the per-type JSON string in targets.config to columns: url, headers, timeout_seconds, forward_query, archive_expiry and archive_rotation. A column a type does not use stays empty. headers holds the header map as JSON text through GORM's JSON serializer, and archive_expiry holds a time.Duration, 0 meaning never. The forms check each setting once, with the same refusals and messages, and the edit form is filled from the columns, so the per-type config structs, their parsers and the "could not be read" branches are gone. A delivery task carries the six settings in place of the JSON string; tests check that each copy into a task, and back out of one, keeps all six. Model: opus-5-5
This commit was merged in pull request #554.
This commit is contained in:
@@ -1579,15 +1579,20 @@ did not arrive on the URL and counts in neither.
|
|||||||
A delivery destination for events. Each target defines where and how events
|
A delivery destination for events. Each target defines where and how events
|
||||||
should be forwarded.
|
should be forwarded.
|
||||||
|
|
||||||
| Field | Type | Description |
|
| Field | Type | Description |
|
||||||
| -------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
| ------------------ | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| `id` | UUID | Primary key |
|
| `id` | UUID | Primary key |
|
||||||
| `webhook_id` | UUID | Foreign key → Webhook |
|
| `webhook_id` | UUID | Foreign key → Webhook |
|
||||||
| `name` | string | Human-readable name |
|
| `name` | string | Human-readable name |
|
||||||
| `type` | TargetType | One of: `http`, `slack`, `database`, `log` |
|
| `type` | TargetType | One of: `http`, `slack`, `database`, `log` |
|
||||||
| `active` | boolean | Whether deliveries are enabled (default: true) |
|
| `active` | boolean | Whether deliveries are enabled (default: true) |
|
||||||
| `config` | JSON text | Type-specific configuration |
|
| `url` | string | Where an `http` or `slack` target sends each event. It is a credential: the target list shows only its scheme and host, and only the target edit form shows it whole |
|
||||||
| `max_attempts` | integer | How many times an `http` or `slack` target attempts each delivery in all: at least 1, and 1 by default. Ignored by `database` and `log` targets. The web UI labels it Delivery attempts |
|
| `headers` | JSON text | An `http` target's extra request headers, a JSON object of name to value |
|
||||||
|
| `timeout_seconds` | integer | An `http` target's per-request timeout; 0 uses the shared 30-second client timeout |
|
||||||
|
| `forward_query` | boolean | Whether an `http` target is sent the query string each event arrived with |
|
||||||
|
| `archive_expiry` | integer | How long a `database` target keeps each archived event, as a Go `time.Duration` (nanoseconds); 0 keeps them forever |
|
||||||
|
| `archive_rotation` | string | How often a `database` target starts a new archive file: `none`, `monthly`, `daily` or `hourly` |
|
||||||
|
| `max_attempts` | integer | How many times an `http` or `slack` target attempts each delivery in all: at least 1, and 1 by default. Ignored by `database` and `log` targets. The web UI labels it Delivery attempts |
|
||||||
|
|
||||||
**Relations:** Belongs to Webhook. Has many Deliveries.
|
**Relations:** Belongs to Webhook. Has many Deliveries.
|
||||||
|
|
||||||
@@ -1598,10 +1603,10 @@ should be forwarded.
|
|||||||
failed delivery with exponential backoff, up to that many attempts in all,
|
failed delivery with exponential backoff, up to that many attempts in all,
|
||||||
behind a per-target circuit breaker.
|
behind a per-target circuit breaker.
|
||||||
- **`slack`** — Post the event as a formatted message to a Slack-compatible
|
- **`slack`** — Post the event as a formatted message to a Slack-compatible
|
||||||
incoming webhook URL (`webhookUrl` in `config`). It is built on the same HTTP
|
incoming webhook URL, its `url`. It is built on the same HTTP core as `http`
|
||||||
core as `http` and honours `max_attempts` identically, circuit breaker
|
and honours `max_attempts` identically, circuit breaker included. See the
|
||||||
included. See the Slack target section under "Per-Webhook Event Databases" for
|
Slack target section under "Per-Webhook Event Databases" for the message
|
||||||
the message format.
|
format.
|
||||||
- **`database`** — Archive the full event as a row into the target's own archive
|
- **`database`** — Archive the full event as a row into the target's own archive
|
||||||
database (`archive-{webhook_name}-{target_name}-{target_uuid}.db`) for
|
database (`archive-{webhook_name}-{target_name}-{target_uuid}.db`) for
|
||||||
long-term retention, with an optional creation-validated expiry (default: keep
|
long-term retention, with an optional creation-validated expiry (default: keep
|
||||||
@@ -1617,29 +1622,28 @@ should be forwarded.
|
|||||||
- **`log`** — Write the event to the application log (stdout). Useful for
|
- **`log`** — Write the event to the application log (stdout). Useful for
|
||||||
debugging.
|
debugging.
|
||||||
|
|
||||||
The `config` field stores type-specific configuration as JSON (e.g., destination
|
Each type uses only its own settings, and the ones it does not use stay empty:
|
||||||
URL, custom headers, timeout settings).
|
`http` uses `url`, `headers`, `timeout_seconds` and `forward_query`; `slack`
|
||||||
|
uses `url`; `database` uses `archive_expiry` and `archive_rotation`; `log` uses
|
||||||
|
none. The add target form, the new webhook form and the target edit form check
|
||||||
|
every setting before it is saved, and each setting is a column of its own, so
|
||||||
|
nothing that reads a target parses its settings or has to handle one it cannot
|
||||||
|
read.
|
||||||
|
|
||||||
**`http` target configuration:**
|
An `http` target POSTs the event to its `url`. Its `headers` are applied last,
|
||||||
|
so they win over the event's own forwarded headers.
|
||||||
|
|
||||||
| Key | Type | Description |
|
`forward_query` is off by default, and the target URL is then sent exactly as
|
||||||
| -------------- | ------------- | ----------------------------------------------------------------------------------------- |
|
|
||||||
| `url` | string | Destination the event is POSTed to |
|
|
||||||
| `headers` | object | Extra request headers, applied last so they win over the event's own forwarded headers |
|
|
||||||
| `timeout` | integer (sec) | Per-target request timeout; unset (or 0) uses the shared 30-second client timeout |
|
|
||||||
| `forwardQuery` | boolean | Pass the query string each event arrived with on to the target; unset (or false) does not |
|
|
||||||
|
|
||||||
`forwardQuery` is off by default, and the target URL is then sent exactly as
|
|
||||||
configured. On, each delivery appends the event's query string to the target
|
configured. On, each delivery appends the event's query string to the target
|
||||||
URL, joined with `&` when the URL already has a query string of its own; a
|
URL, joined with `&` when the URL already has a query string of its own; a
|
||||||
replayed delivery and a resubmitted event's deliveries do the same. Both target
|
replayed delivery and a resubmitted event's deliveries do the same. Both target
|
||||||
forms offer it as "Pass the query string on to this target", and the target list
|
forms offer it as "Pass the query string on to this target", and the target list
|
||||||
shows it when it is on.
|
shows it when it is on.
|
||||||
|
|
||||||
`timeout` is capped at **300 seconds**, and the form rejects anything above it
|
`timeout_seconds` is capped at **300 seconds**, and the form rejects anything
|
||||||
rather than substituting the cap. A delivery attempt holds one of the bounded
|
above it rather than substituting the cap. A delivery attempt holds one of the
|
||||||
pool's workers for its whole duration, so an unbounded timeout would let a
|
bounded pool's workers for its whole duration, so an unbounded timeout would let
|
||||||
single unresponsive destination stall the queue.
|
a single unresponsive destination stall the queue.
|
||||||
|
|
||||||
`headers` rejects the names the delivery path or `net/http` writes regardless of
|
`headers` rejects the names the delivery path or `net/http` writes regardless of
|
||||||
what is configured: `Host`, `Content-Length`, `Transfer-Encoding`, `Connection`,
|
what is configured: `Host`, `Content-Length`, `Transfer-Encoding`, `Connection`,
|
||||||
@@ -1713,7 +1717,7 @@ The event log and the event's own page show the query string with the rest of
|
|||||||
the request. The event log leaves out one larger than 32 KiB, as it does request
|
the request. The event log leaves out one larger than 32 KiB, as it does request
|
||||||
headers, and links to the event's page, which shows it whole. The `database` and
|
headers, and links to the event's page, which shows it whole. The `database` and
|
||||||
`log` targets carry it with the rest of the event. An `http` target receives it
|
`log` targets carry it with the rest of the event. An `http` target receives it
|
||||||
only when its `forwardQuery` setting is on.
|
only when its `forward_query` setting is on.
|
||||||
|
|
||||||
#### Delivery
|
#### Delivery
|
||||||
|
|
||||||
@@ -1992,11 +1996,10 @@ target UUID keeps the file name unique. A webhook named `Orders (EU)` with a
|
|||||||
target named `Long-term archive` archives into
|
target named `Long-term archive` archives into
|
||||||
`archive-orders-eu-long-term-archive-{target_uuid}.db`.
|
`archive-orders-eu-long-term-archive-{target_uuid}.db`.
|
||||||
|
|
||||||
An optional `rotation` in the target's config JSON (e.g. `{"rotation":"daily"}`)
|
A target's `archive_rotation` is `none`, the default, which keeps the one file,
|
||||||
is `none`, the default, which keeps the one file, or `monthly`, `daily` or
|
or `monthly`, `daily` or `hourly`. A target that rotates writes each event to a
|
||||||
`hourly`. A target that rotates writes each event to a file named for the period
|
file named for the period of the event's receive time, in UTC, put before the
|
||||||
of the event's receive time, in UTC, put before the `.db`:
|
`.db`: `archive-orders-eu-long-term-archive-{target_uuid}-2026-10.db` monthly,
|
||||||
`archive-orders-eu-long-term-archive-{target_uuid}-2026-10.db` monthly,
|
|
||||||
`…-2026-10-01.db` daily and `…-2026-10-01-19.db` hourly. Each file holds exactly
|
`…-2026-10-01.db` daily and `…-2026-10-01-19.db` hourly. Each file holds exactly
|
||||||
its period's events, and the first event of a new period starts the next file,
|
its period's events, and the first event of a new period starts the next file,
|
||||||
so a finished period's file can be moved away like any archive. A changed
|
so a finished period's file can be moved away like any archive. A changed
|
||||||
@@ -2031,12 +2034,11 @@ archiving without stopping the service; a moved or removed archive file is
|
|||||||
recreated automatically on the next write, from the SQL files in
|
recreated automatically on the next write, from the SQL files in
|
||||||
`internal/db/migrations/archive/`. A zero-length archive file is written to as a
|
`internal/db/migrations/archive/`. A zero-length archive file is written to as a
|
||||||
new archive: SQLite opens it as an empty database, so it holds nothing to lose.
|
new archive: SQLite opens it as an empty database, so it holds nothing to lose.
|
||||||
An optional `expiry` in the target's config JSON (e.g. `{"expiry":"720h"}`) is
|
A target's `archive_expiry` is checked by the form that saves it — the default,
|
||||||
validated when the target is created — the default (unset or the literal
|
`never` on the forms and 0 in the column, keeps rows forever — and rows older
|
||||||
`never`) keeps rows forever — and rows older than the expiry are pruned each
|
than the expiry are pruned each time the archive is (re)opened. An archive write
|
||||||
time the archive is (re)opened. An archive write failure is never silent
|
failure is never silent success: the delivery records a failed attempt with the
|
||||||
success: the delivery records a failed attempt with the error and is marked
|
error and is marked failed.
|
||||||
failed.
|
|
||||||
|
|
||||||
Because reopens only happen on writes, an archive whose target has stopped
|
Because reopens only happen on writes, an archive whose target has stopped
|
||||||
receiving events would never be pruned. A background **archive sweeper** closes
|
receiving events would never be pruned. A background **archive sweeper** closes
|
||||||
@@ -2047,8 +2049,7 @@ traffic. The sweep never creates an archive — a target whose archive file does
|
|||||||
not yet exist is skipped, not initialised — it takes the same per-target lock
|
not yet exist is skipped, not initialised — it takes the same per-target lock
|
||||||
the write path uses, so it can never interleave with a write, and it leaves the
|
the write path uses, so it can never interleave with a write, and it leaves the
|
||||||
archive closed afterwards so the move-the-file-away workflow keeps working.
|
archive closed afterwards so the move-the-file-away workflow keeps working.
|
||||||
Archives with no expiry, or the expiry `never`, are not touched by the sweep at
|
Archives with the expiry `never` are not touched by the sweep at all.
|
||||||
all.
|
|
||||||
|
|
||||||
For a target with several files, the write path prunes only the file it writes
|
For a target with several files, the write path prunes only the file it writes
|
||||||
to, and the sweep prunes every one of them, taking the target's lock for one
|
to, and the sweep prunes every one of them, taking the target's lock for one
|
||||||
@@ -2116,9 +2117,8 @@ compatible services). Each message includes event metadata (HTTP method, content
|
|||||||
type, timestamp, body size) and the payload pretty-printed in a code block. JSON
|
type, timestamp, body size) and the payload pretty-printed in a code block. JSON
|
||||||
payloads are automatically formatted with indentation for readability; non-JSON
|
payloads are automatically formatted with indentation for readability; non-JSON
|
||||||
payloads are shown as raw text. Large payloads are truncated to keep messages
|
payloads are shown as raw text. Large payloads are truncated to keep messages
|
||||||
reasonable. Config stores `webhookUrl` — the Slack/Mattermost incoming webhook
|
reasonable. The target's `url` is the Slack/Mattermost incoming webhook
|
||||||
endpoint. That is the JSON key; the error text for a missing one reads
|
endpoint.
|
||||||
`webhook_url is required`, which is the message, not the key.
|
|
||||||
|
|
||||||
The database uses the
|
The database uses the
|
||||||
[modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) driver.
|
[modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) driver.
|
||||||
@@ -2156,7 +2156,7 @@ External Service
|
|||||||
3. Capture full request as Event
|
3. Capture full request as Event
|
||||||
4. Create Delivery records for each active Target
|
4. Create Delivery records for each active Target
|
||||||
5. Build self-contained delivery.Task structs
|
5. Build self-contained delivery.Task structs
|
||||||
(target config + event data inline for
|
(target settings + event data inline for
|
||||||
bodies < 16 KiB)
|
bodies < 16 KiB)
|
||||||
6. Notify Engine via channel (no DB read needed)
|
6. Notify Engine via channel (no DB read needed)
|
||||||
│
|
│
|
||||||
@@ -3060,14 +3060,14 @@ webhooker/
|
|||||||
│ │ ├── target_slack.go # Slack/Mattermost incoming-webhook target
|
│ │ ├── target_slack.go # Slack/Mattermost incoming-webhook target
|
||||||
│ │ ├── target_database.go # Database archive target, writing through internal/archive
|
│ │ ├── target_database.go # Database archive target, writing through internal/archive
|
||||||
│ │ ├── target_log.go # Log target (stdout)
|
│ │ ├── target_log.go # Log target (stdout)
|
||||||
│ │ ├── target_config_view.go # Masked target config for templates
|
│ │ ├── target_config_view.go # Masked target settings for templates
|
||||||
│ │ ├── queue_depth.go # Periodic sampler behind the queue-depth gauges
|
│ │ ├── queue_depth.go # Periodic sampler behind the queue-depth gauges
|
||||||
│ │ ├── url_mask.go # Strips credentials from *url.Error
|
│ │ ├── url_mask.go # Strips credentials from *url.Error
|
||||||
│ │ └── ssrf.go # SSRF prevention (IP validation, safe HTTP transport)
|
│ │ └── ssrf.go # SSRF prevention (IP validation, safe HTTP transport)
|
||||||
│ ├── handlers/
|
│ ├── handlers/
|
||||||
│ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering
|
│ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering
|
||||||
│ │ ├── auth.go # Login, logout handlers
|
│ │ ├── auth.go # Login, logout handlers
|
||||||
│ │ ├── delivery_replay.go # Per-delivery replay: new delivery, current target config
|
│ │ ├── delivery_replay.go # Per-delivery replay: new delivery, current target settings
|
||||||
│ │ ├── event_resubmit.go # Event resubmit: new event, all currently active targets
|
│ │ ├── event_resubmit.go # Event resubmit: new event, all currently active targets
|
||||||
│ │ ├── entrypoint_view.go # Masked entrypoint view for templates
|
│ │ ├── entrypoint_view.go # Masked entrypoint view for templates
|
||||||
│ │ ├── event_log_view.go # Event log projection, byte-capped in SQL
|
│ │ ├── event_log_view.go # Event log projection, byte-capped in SQL
|
||||||
@@ -3082,7 +3082,7 @@ webhooker/
|
|||||||
│ │ ├── webhook_delete.go # Webhook delete, event database and archive writer removal
|
│ │ ├── webhook_delete.go # Webhook delete, event database and archive writer removal
|
||||||
│ │ ├── event_log.go # Event log page: loaders, filters, delivery views
|
│ │ ├── event_log.go # Event log page: loaders, filters, delivery views
|
||||||
│ │ ├── entrypoint.go # Entrypoint create, edit, delete and toggle
|
│ │ ├── entrypoint.go # Entrypoint create, edit, delete and toggle
|
||||||
│ │ ├── target_create.go # Target create, per-type config builders
|
│ │ ├── target_create.go # Target create, each type's settings from the form
|
||||||
│ │ ├── target_delete.go # Target delete
|
│ │ ├── target_delete.go # Target delete
|
||||||
│ │ ├── target_toggle.go # Target toggle
|
│ │ ├── target_toggle.go # Target toggle
|
||||||
│ │ ├── shared.go # Helpers shared by several handlers
|
│ │ ├── shared.go # Helpers shared by several handlers
|
||||||
@@ -3164,7 +3164,7 @@ The server starts via
|
|||||||
which triggers the fx lifecycle hooks in dependency order. The
|
which triggers the fx lifecycle hooks in dependency order. The
|
||||||
`delivery.Notifier` interface allows the webhook handler to send self-contained
|
`delivery.Notifier` interface allows the webhook handler to send self-contained
|
||||||
`delivery.Task` slices to the engine without a direct package dependency. Each
|
`delivery.Task` slices to the engine without a direct package dependency. Each
|
||||||
task carries all target config and event data inline (for bodies under 16 KiB,
|
task carries all target settings and event data inline (for bodies under 16 KiB,
|
||||||
`delivery.MaxInlineBodySize`), so the engine can deliver without reading from
|
`delivery.MaxInlineBodySize`), so the engine can deliver without reading from
|
||||||
any database — it only writes to record results.
|
any database — it only writes to record results.
|
||||||
|
|
||||||
|
|||||||
@@ -143,26 +143,15 @@ func archiveNamePart(name string) string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Write writes the full event of a delivery to a database target as
|
// Write writes the full event of a delivery to a database target as
|
||||||
// a row into the target's archive database, honouring the optional
|
// a row into the target's archive database, honouring the target's
|
||||||
// per-target expiry and rotation parsed from the target config JSON.
|
// ArchiveExpiry and ArchiveRotation. With rotation, the event goes to
|
||||||
// With rotation, the event goes to the file for the period of its
|
// the file for the period of its receive time.
|
||||||
// receive time.
|
|
||||||
func (ws *Writers) Write(d *database.Delivery) error {
|
func (ws *Writers) Write(d *database.Delivery) error {
|
||||||
webhookID := d.Event.WebhookID
|
webhookID := d.Event.WebhookID
|
||||||
if webhookID == "" {
|
if webhookID == "" {
|
||||||
return errArchiveMissingWebhookID
|
return errArchiveMissingWebhookID
|
||||||
}
|
}
|
||||||
|
|
||||||
expiry, err := ParseExpiry(d.Target.Config)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
rotation, err := ParseRotation(d.Target.Config)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// An event whose stored row was gone before its delivery ran has
|
// An event whose stored row was gone before its delivery ran has
|
||||||
// no receive time (see delivery's Engine.hydrateEvent), and goes
|
// no receive time (see delivery's Engine.hydrateEvent), and goes
|
||||||
// to the file for now.
|
// to the file for now.
|
||||||
@@ -187,7 +176,10 @@ func (ws *Writers) Write(d *database.Delivery) error {
|
|||||||
ContentType: d.Event.ContentType,
|
ContentType: d.Event.ContentType,
|
||||||
}
|
}
|
||||||
|
|
||||||
return w.write(row, expiry, archivePeriod(rotation, receivedAt))
|
return w.write(
|
||||||
|
row, d.Target.ArchiveExpiry,
|
||||||
|
archivePeriod(d.Target.ArchiveRotation, receivedAt),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Rename implements Archives. It renames every one of a database
|
// Rename implements Archives. It renames every one of a database
|
||||||
|
|||||||
@@ -123,9 +123,10 @@ func databaseTargetDelivery(
|
|||||||
TargetID: tgt.ID,
|
TargetID: tgt.ID,
|
||||||
Event: event,
|
Event: event,
|
||||||
Target: database.Target{
|
Target: database.Target{
|
||||||
Name: tgt.Name,
|
Name: tgt.Name,
|
||||||
Type: database.TargetTypeDatabase,
|
Type: database.TargetTypeDatabase,
|
||||||
Config: tgt.Config,
|
ArchiveExpiry: tgt.ArchiveExpiry,
|
||||||
|
ArchiveRotation: tgt.ArchiveRotation,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -355,13 +356,16 @@ func TestParseArchiveExpiry(t *testing.T) {
|
|||||||
want time.Duration
|
want time.Duration
|
||||||
wantErr bool
|
wantErr bool
|
||||||
}{
|
}{
|
||||||
{"empty config", "", 0, false},
|
{"empty", "", 0, false},
|
||||||
{"explicit never", `{"expiry":"never"}`, 0, false},
|
{"explicit never", "never", 0, false},
|
||||||
{"empty expiry", `{"expiry":""}`, 0, false},
|
{"hour", "1h", time.Hour, false},
|
||||||
{"duration", `{"expiry":"1h"}`, time.Hour, false},
|
{"thirty days", "720h", 720 * time.Hour, false},
|
||||||
{"unparseable", `{"expiry":"nonsense"}`, 0, true},
|
{"minutes", "30m", 30 * time.Minute, false},
|
||||||
{"zero duration", `{"expiry":"0s"}`, 0, true},
|
{"unparseable", "nonsense", 0, true},
|
||||||
{"negative duration", `{"expiry":"-5h"}`, 0, true},
|
{"days are not a Go unit", "7d", 0, true},
|
||||||
|
{"zero duration", "0s", 0, true},
|
||||||
|
{"bare zero", "0", 0, true},
|
||||||
|
{"negative duration", "-5h", 0, true},
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, tc := range cases {
|
for _, tc := range cases {
|
||||||
@@ -381,26 +385,6 @@ func TestParseArchiveExpiry(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestValidateArchiveExpiry(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
valid := []string{"", "never", "1h", "720h", "30m"}
|
|
||||||
for _, in := range valid {
|
|
||||||
require.NoError(t,
|
|
||||||
archive.ValidateExpiry(in),
|
|
||||||
"expiry %q should be accepted", in,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
invalid := []string{"nonsense", "7d", "-5h", "0s", "0"}
|
|
||||||
for _, in := range invalid {
|
|
||||||
require.Error(t,
|
|
||||||
archive.ValidateExpiry(in),
|
|
||||||
"expiry %q should be rejected", in,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestArchiveFileName pins the archive file name and the rules
|
// TestArchiveFileName pins the archive file name and the rules
|
||||||
// that make a webhook or target name safe to put in it.
|
// that make a webhook or target name safe to put in it.
|
||||||
func TestArchiveFileName(t *testing.T) {
|
func TestArchiveFileName(t *testing.T) {
|
||||||
@@ -473,8 +457,8 @@ func TestWrite_EachTargetHasItsOwnArchive(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
first := env.seedDatabaseTarget(t, "")
|
first := env.seedDatabaseTarget(t, 0, "")
|
||||||
second := env.addDatabaseTarget(t, first.WebhookID, "")
|
second := env.addDatabaseTarget(t, first.WebhookID, 0, "")
|
||||||
|
|
||||||
event := newEvent(`{"n":1}`)
|
event := newEvent(`{"n":1}`)
|
||||||
|
|
||||||
@@ -503,7 +487,7 @@ func TestRename_MovesTheFile(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(t, "")
|
tgt := env.seedDatabaseTarget(t, 0, "")
|
||||||
oldPath := env.archivePath(tgt)
|
oldPath := env.archivePath(tgt)
|
||||||
|
|
||||||
first := newEvent(`{"n":1}`)
|
first := newEvent(`{"n":1}`)
|
||||||
@@ -545,7 +529,7 @@ func TestRename_NeverReplacesAFile(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(t, "")
|
tgt := env.seedDatabaseTarget(t, 0, "")
|
||||||
oldPath := env.archivePath(tgt)
|
oldPath := env.archivePath(tgt)
|
||||||
|
|
||||||
first := newEvent(`{"n":1}`)
|
first := newEvent(`{"n":1}`)
|
||||||
@@ -593,7 +577,7 @@ func TestRename_BeforeTheNameIsSaved(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(t, "")
|
tgt := env.seedDatabaseTarget(t, 0, "")
|
||||||
|
|
||||||
require.NoError(
|
require.NoError(
|
||||||
t, env.writers.Rename(tgt.ID, "Orders", "Archive"),
|
t, env.writers.Rename(tgt.ID, "Orders", "Archive"),
|
||||||
@@ -721,7 +705,7 @@ func TestRenameLock_HeldByOneEditAtATime(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(t, "")
|
tgt := env.seedDatabaseTarget(t, 0, "")
|
||||||
event := newEvent(`{"n":1}`)
|
event := newEvent(`{"n":1}`)
|
||||||
|
|
||||||
env.writers.RenameLock().Lock()
|
env.writers.RenameLock().Lock()
|
||||||
|
|||||||
@@ -37,9 +37,9 @@ func TestEvictWebhook_ClosesAndRemovesWriter(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
first := env.seedDatabaseTarget(t, "")
|
first := env.seedDatabaseTarget(t, 0, "")
|
||||||
second := env.addDatabaseTarget(t, first.WebhookID, "")
|
second := env.addDatabaseTarget(t, first.WebhookID, 0, "")
|
||||||
other := env.seedDatabaseTarget(t, "")
|
other := env.seedDatabaseTarget(t, 0, "")
|
||||||
|
|
||||||
for _, tgt := range []*database.Target{first, second, other} {
|
for _, tgt := range []*database.Target{first, second, other} {
|
||||||
deliverTo(t, env, tgt)
|
deliverTo(t, env, tgt)
|
||||||
@@ -80,8 +80,8 @@ func TestEvictTarget_LeavesOtherTargets(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
doomed := env.seedDatabaseTarget(t, "")
|
doomed := env.seedDatabaseTarget(t, 0, "")
|
||||||
kept := env.addDatabaseTarget(t, doomed.WebhookID, "")
|
kept := env.addDatabaseTarget(t, doomed.WebhookID, 0, "")
|
||||||
|
|
||||||
deliverTo(t, env, doomed)
|
deliverTo(t, env, doomed)
|
||||||
deliverTo(t, env, kept)
|
deliverTo(t, env, kept)
|
||||||
@@ -310,7 +310,7 @@ func TestEvictWebhook_RacingWriteDoesNotReopenHandle(
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(t, "")
|
tgt := env.seedDatabaseTarget(t, 0, "")
|
||||||
|
|
||||||
// Prime the registry so the test can hold the very writer the
|
// Prime the registry so the test can hold the very writer the
|
||||||
// eviction is about to detach.
|
// eviction is about to detach.
|
||||||
@@ -355,7 +355,7 @@ func TestEvictWebhook_LaterDeliveryRecreatesWriter(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(t, "")
|
tgt := env.seedDatabaseTarget(t, 0, "")
|
||||||
|
|
||||||
deliverTo(t, env, tgt)
|
deliverTo(t, env, tgt)
|
||||||
require.True(t, env.writers.ExportHasArchiveWriter(tgt.ID))
|
require.True(t, env.writers.ExportHasArchiveWriter(tgt.ID))
|
||||||
@@ -380,7 +380,7 @@ func TestEvictAll_WriteAfterIsRefused(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(t, "")
|
tgt := env.seedDatabaseTarget(t, 0, "")
|
||||||
|
|
||||||
deliverTo(t, env, tgt)
|
deliverTo(t, env, tgt)
|
||||||
|
|
||||||
|
|||||||
@@ -1,7 +1,6 @@
|
|||||||
package archive
|
package archive
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"encoding/json"
|
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"os"
|
"os"
|
||||||
@@ -54,35 +53,10 @@ func ValidateRotation(rotation string) error {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// ParseRotation reads the rotation from a database target's
|
|
||||||
// config JSON. An empty config or an empty rotation is none.
|
|
||||||
func ParseRotation(configJSON string) (string, error) {
|
|
||||||
if configJSON == "" {
|
|
||||||
return archiveRotationNone, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
var cfg DatabaseTargetConfig
|
|
||||||
|
|
||||||
err := json.Unmarshal([]byte(configJSON), &cfg)
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("parsing database target config: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
err = ValidateRotation(cfg.Rotation)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
|
|
||||||
if cfg.Rotation == "" {
|
|
||||||
return archiveRotationNone, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
return cfg.Rotation, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// archivePeriod returns the period, in UTC, that a rotation puts an
|
// archivePeriod returns the period, in UTC, that a rotation puts an
|
||||||
// event received at receivedAt in: "2026-10" for monthly,
|
// event received at receivedAt in: "2026-10" for monthly,
|
||||||
// "2026-10-01" for daily, "2026-10-01-19" for hourly, and "" for none.
|
// "2026-10-01" for daily, "2026-10-01-19" for hourly, and "" for none
|
||||||
|
// or an empty rotation.
|
||||||
func archivePeriod(rotation string, receivedAt time.Time) string {
|
func archivePeriod(rotation string, receivedAt time.Time) string {
|
||||||
switch rotation {
|
switch rotation {
|
||||||
case archiveRotationMonthly:
|
case archiveRotationMonthly:
|
||||||
@@ -115,16 +89,11 @@ func PathAt(
|
|||||||
webhook *database.Webhook,
|
webhook *database.Webhook,
|
||||||
target *database.Target,
|
target *database.Target,
|
||||||
receivedAt time.Time,
|
receivedAt time.Time,
|
||||||
) (string, error) {
|
) string {
|
||||||
rotation, err := ParseRotation(target.Config)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
|
|
||||||
return archivePeriodPath(
|
return archivePeriodPath(
|
||||||
Path(dbMgr, webhook, target),
|
Path(dbMgr, webhook, target),
|
||||||
archivePeriod(rotation, receivedAt),
|
archivePeriod(target.ArchiveRotation, receivedAt),
|
||||||
), nil
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// archiveFiles lists the database target's archive files that exist,
|
// archiveFiles lists the database target's archive files that exist,
|
||||||
|
|||||||
@@ -14,16 +14,12 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/database/databasetest"
|
"sneak.berlin/go/webhooker/internal/database/databasetest"
|
||||||
)
|
)
|
||||||
|
|
||||||
// The archive rotations, and the configs of a daily target and of one
|
// The archive rotations.
|
||||||
// whose rotation is not one of the four.
|
|
||||||
const (
|
const (
|
||||||
rotationNone = "none"
|
rotationNone = "none"
|
||||||
rotationMonthly = "monthly"
|
rotationMonthly = "monthly"
|
||||||
rotationDaily = "daily"
|
rotationDaily = "daily"
|
||||||
rotationHourly = "hourly"
|
rotationHourly = "hourly"
|
||||||
|
|
||||||
dailyConfig = `{"rotation":"daily"}`
|
|
||||||
weeklyConfig = `{"rotation":"weekly"}`
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// The periods the tests archive into most: two days, and an hour of
|
// The periods the tests archive into most: two days, and an hour of
|
||||||
@@ -108,7 +104,7 @@ func TestWrite_RotatesAtEachPeriodBoundary(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(t, `{"rotation":"`+tc.rotation+`"}`)
|
tgt := env.seedDatabaseTarget(t, 0, tc.rotation)
|
||||||
path := env.archivePath(tgt)
|
path := env.archivePath(tgt)
|
||||||
|
|
||||||
first := env.deliverReceivedAt(t, tgt, tc.before)
|
first := env.deliverReceivedAt(t, tgt, tc.before)
|
||||||
@@ -127,7 +123,7 @@ func TestWrite_RotatesAtEachPeriodBoundary(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(t, `{"rotation":"`+rotationNone+`"}`)
|
tgt := env.seedDatabaseTarget(t, 0, rotationNone)
|
||||||
|
|
||||||
first := env.deliverReceivedAt(t, tgt, cases[0].before)
|
first := env.deliverReceivedAt(t, tgt, cases[0].before)
|
||||||
second := env.deliverReceivedAt(t, tgt, cases[0].after)
|
second := env.deliverReceivedAt(t, tgt, cases[0].after)
|
||||||
@@ -144,7 +140,7 @@ func TestWrite_EventWithoutReceiveTime(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(t, `{"rotation":"`+rotationMonthly+`"}`)
|
tgt := env.seedDatabaseTarget(t, 0, rotationMonthly)
|
||||||
path := env.archivePath(tgt)
|
path := env.archivePath(tgt)
|
||||||
|
|
||||||
before := time.Now().UTC().Format("2006-01")
|
before := time.Now().UTC().Format("2006-01")
|
||||||
@@ -169,13 +165,13 @@ func TestWrite_RotationChangeKeepsOldFiles(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(t, "")
|
tgt := env.seedDatabaseTarget(t, 0, rotationNone)
|
||||||
path := env.archivePath(tgt)
|
path := env.archivePath(tgt)
|
||||||
at := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC)
|
at := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC)
|
||||||
|
|
||||||
first := env.deliverReceivedAt(t, tgt, at)
|
first := env.deliverReceivedAt(t, tgt, at)
|
||||||
|
|
||||||
tgt.Config = dailyConfig
|
tgt.ArchiveRotation = rotationDaily
|
||||||
second := env.deliverReceivedAt(t, tgt, at)
|
second := env.deliverReceivedAt(t, tgt, at)
|
||||||
|
|
||||||
assert.Equal(t, []string{first}, archivedEventIDs(t, path))
|
assert.Equal(t, []string{first}, archivedEventIDs(t, path))
|
||||||
@@ -193,9 +189,7 @@ func TestArchiveSweep_PrunesEveryFile(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(
|
tgt := env.seedDatabaseTarget(t, time.Hour, rotationDaily)
|
||||||
t, `{"expiry":"1h","rotation":"`+rotationDaily+`"}`,
|
|
||||||
)
|
|
||||||
path := env.archivePath(tgt)
|
path := env.archivePath(tgt)
|
||||||
emptied := periodPath(path, dayPeriod)
|
emptied := periodPath(path, dayPeriod)
|
||||||
kept := periodPath(path, nextDayPeriod)
|
kept := periodPath(path, nextDayPeriod)
|
||||||
@@ -225,13 +219,13 @@ func TestRename_MovesEveryFile(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(t, "")
|
tgt := env.seedDatabaseTarget(t, 0, rotationNone)
|
||||||
oldPath := env.archivePath(tgt)
|
oldPath := env.archivePath(tgt)
|
||||||
day := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC)
|
day := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC)
|
||||||
|
|
||||||
unrotated := env.deliverReceivedAt(t, tgt, day)
|
unrotated := env.deliverReceivedAt(t, tgt, day)
|
||||||
|
|
||||||
tgt.Config = dailyConfig
|
tgt.ArchiveRotation = rotationDaily
|
||||||
first := env.deliverReceivedAt(t, tgt, day)
|
first := env.deliverReceivedAt(t, tgt, day)
|
||||||
second := env.deliverReceivedAt(t, tgt, day.Add(24*time.Hour))
|
second := env.deliverReceivedAt(t, tgt, day.Add(24*time.Hour))
|
||||||
|
|
||||||
@@ -267,7 +261,7 @@ func TestRename_NeverReplacesARotatedFile(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(t, dailyConfig)
|
tgt := env.seedDatabaseTarget(t, 0, rotationDaily)
|
||||||
oldPath := env.archivePath(tgt)
|
oldPath := env.archivePath(tgt)
|
||||||
day := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC)
|
day := time.Date(2026, 3, 4, 12, 0, 0, 0, time.UTC)
|
||||||
|
|
||||||
@@ -341,34 +335,27 @@ func TestArchivePathAt(t *testing.T) {
|
|||||||
at := time.Date(2026, 10, 1, 19, 30, 0, 0, time.UTC)
|
at := time.Date(2026, 10, 1, 19, 30, 0, 0, time.UTC)
|
||||||
|
|
||||||
cases := map[string]string{
|
cases := map[string]string{
|
||||||
"": "",
|
"": "",
|
||||||
`{"rotation":"` + rotationNone + `"}`: "",
|
rotationNone: "",
|
||||||
`{"rotation":"` + rotationMonthly + `"}`: "-2026-10",
|
rotationMonthly: "-2026-10",
|
||||||
dailyConfig: "-2026-10-01",
|
rotationDaily: "-2026-10-01",
|
||||||
`{"rotation":"` + rotationHourly + `"}`: "-2026-10-01-19",
|
rotationHourly: "-2026-10-01-19",
|
||||||
}
|
}
|
||||||
|
|
||||||
for config, period := range cases {
|
for rotation, period := range cases {
|
||||||
target := &database.Target{
|
target := &database.Target{
|
||||||
BaseModel: database.BaseModel{ID: "tgt-id"},
|
BaseModel: database.BaseModel{ID: "tgt-id"},
|
||||||
Name: "Archive",
|
Name: "Archive",
|
||||||
Config: config,
|
ArchiveRotation: rotation,
|
||||||
}
|
}
|
||||||
|
|
||||||
got, err := archive.PathAt(dbMgr, webhook, target, at)
|
|
||||||
require.NoError(t, err, config)
|
|
||||||
assert.Equal(t,
|
assert.Equal(t,
|
||||||
filepath.Join(
|
filepath.Join(
|
||||||
dataDir, "archive-orders-archive-tgt-id"+period+".db",
|
dataDir, "archive-orders-archive-tgt-id"+period+".db",
|
||||||
),
|
),
|
||||||
got, config,
|
archive.PathAt(dbMgr, webhook, target, at), rotation,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
_, err := archive.PathAt(dbMgr, webhook, &database.Target{
|
|
||||||
Config: weeklyConfig,
|
|
||||||
}, at)
|
|
||||||
require.Error(t, err)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestValidateArchiveRotation accepts the four rotations, and empty,
|
// TestValidateArchiveRotation accepts the four rotations, and empty,
|
||||||
|
|||||||
@@ -199,22 +199,11 @@ func (s *Sweeper) sweep(ctx context.Context) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// sweepTarget prunes the archive of a single database target.
|
// sweepTarget prunes the archive of a single database target. A
|
||||||
// A missing, empty, or "never" expiry parses as a zero duration
|
// target with an ArchiveExpiry of 0 keeps its archive forever and is
|
||||||
// and is skipped entirely, so those archives are never pruned.
|
// skipped entirely, so that archive is never pruned.
|
||||||
func (s *Sweeper) sweepTarget(target *database.Target) {
|
func (s *Sweeper) sweepTarget(target *database.Target) {
|
||||||
expiry, err := ParseExpiry(target.Config)
|
expiry := target.ArchiveExpiry
|
||||||
if err != nil {
|
|
||||||
s.log.Error(
|
|
||||||
"archive sweep: invalid database target config",
|
|
||||||
"webhook_id", target.WebhookID,
|
|
||||||
"target_id", target.ID,
|
|
||||||
"error", err,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
if expiry <= 0 {
|
if expiry <= 0 {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -223,7 +212,7 @@ func (s *Sweeper) sweepTarget(target *database.Target) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
err = s.writers.sweepArchive(target.ID, expiry)
|
err := s.writers.sweepArchive(target.ID, expiry)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -106,10 +106,10 @@ func (env *archiveEnv) archivePath(tgt *database.Target) string {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// seedDatabaseTarget creates a webhook with one database target
|
// seedDatabaseTarget creates a webhook with one database target with
|
||||||
// carrying the given target config JSON, and returns the target.
|
// the given archive expiry and rotation, and returns the target.
|
||||||
func (env *archiveEnv) seedDatabaseTarget(
|
func (env *archiveEnv) seedDatabaseTarget(
|
||||||
t *testing.T, configJSON string,
|
t *testing.T, expiry time.Duration, rotation string,
|
||||||
) *database.Target {
|
) *database.Target {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
@@ -124,23 +124,27 @@ func (env *archiveEnv) seedDatabaseTarget(
|
|||||||
Create(wh).Error,
|
Create(wh).Error,
|
||||||
)
|
)
|
||||||
|
|
||||||
return env.addDatabaseTarget(t, wh.ID, configJSON)
|
return env.addDatabaseTarget(t, wh.ID, expiry, rotation)
|
||||||
}
|
}
|
||||||
|
|
||||||
// addDatabaseTarget creates one more database target on an
|
// addDatabaseTarget creates one more database target on an
|
||||||
// existing webhook and returns it.
|
// existing webhook and returns it.
|
||||||
func (env *archiveEnv) addDatabaseTarget(
|
func (env *archiveEnv) addDatabaseTarget(
|
||||||
t *testing.T, webhookID, configJSON string,
|
t *testing.T,
|
||||||
|
webhookID string,
|
||||||
|
expiry time.Duration,
|
||||||
|
rotation string,
|
||||||
) *database.Target {
|
) *database.Target {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
tgt := &database.Target{
|
tgt := &database.Target{
|
||||||
WebhookID: webhookID,
|
WebhookID: webhookID,
|
||||||
Name: "Archive",
|
Name: "Archive",
|
||||||
Type: database.TargetTypeDatabase,
|
Type: database.TargetTypeDatabase,
|
||||||
Active: true,
|
Active: true,
|
||||||
Config: configJSON,
|
ArchiveExpiry: expiry,
|
||||||
MaxAttempts: 1,
|
ArchiveRotation: rotation,
|
||||||
|
MaxAttempts: 1,
|
||||||
}
|
}
|
||||||
require.NoError(
|
require.NoError(
|
||||||
t,
|
t,
|
||||||
@@ -323,7 +327,7 @@ func TestArchiveSweeper_LoopOutlivesStartHookContext(
|
|||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
|
|
||||||
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
|
tgt := env.seedDatabaseTarget(t, time.Hour, "")
|
||||||
|
|
||||||
now := time.Now()
|
now := time.Now()
|
||||||
path := env.seedArchiveRows(
|
path := env.seedArchiveRows(
|
||||||
@@ -377,7 +381,7 @@ func TestArchiveSweep_DoesNotResurrectEvictedWriter(
|
|||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
|
|
||||||
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
|
tgt := env.seedDatabaseTarget(t, time.Hour, "")
|
||||||
env.seedArchiveRows(
|
env.seedArchiveRows(
|
||||||
t, tgt, time.Now().Add(-48*time.Hour),
|
t, tgt, time.Now().Add(-48*time.Hour),
|
||||||
)
|
)
|
||||||
@@ -411,7 +415,7 @@ func TestArchiveSweep_LeavesNoRegistryEntry(t *testing.T) {
|
|||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
|
|
||||||
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
|
tgt := env.seedDatabaseTarget(t, time.Hour, "")
|
||||||
path := env.seedArchiveRows(
|
path := env.seedArchiveRows(
|
||||||
t, tgt,
|
t, tgt,
|
||||||
time.Now().Add(-48*time.Hour),
|
time.Now().Add(-48*time.Hour),
|
||||||
@@ -444,7 +448,7 @@ func TestArchiveSweep_KeepsWriterAdoptedByDelivery(
|
|||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
|
|
||||||
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
|
tgt := env.seedDatabaseTarget(t, time.Hour, "")
|
||||||
env.seedArchiveRows(
|
env.seedArchiveRows(
|
||||||
t, tgt, time.Now().Add(-48*time.Hour),
|
t, tgt, time.Now().Add(-48*time.Hour),
|
||||||
)
|
)
|
||||||
@@ -498,7 +502,7 @@ func TestArchiveSweep_KeepsWriterAdoptedDuringSweep(
|
|||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
|
|
||||||
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
|
tgt := env.seedDatabaseTarget(t, time.Hour, "")
|
||||||
env.seedArchiveRows(
|
env.seedArchiveRows(
|
||||||
t, tgt, time.Now().Add(-48*time.Hour),
|
t, tgt, time.Now().Add(-48*time.Hour),
|
||||||
)
|
)
|
||||||
@@ -551,8 +555,8 @@ func TestArchiveSweep_KeepsWriterAdoptedDuringSweep(
|
|||||||
|
|
||||||
// TestArchiveSweep_ContinuesAfterPerTargetFailure proves a
|
// TestArchiveSweep_ContinuesAfterPerTargetFailure proves a
|
||||||
// failure for one target does not abort the sweep for the
|
// failure for one target does not abort the sweep for the
|
||||||
// others: an unparseable expiry and an unreadable archive both
|
// others: an unreadable archive has to be logged and stepped
|
||||||
// have to be logged and stepped over.
|
// over.
|
||||||
func TestArchiveSweep_ContinuesAfterPerTargetFailure(
|
func TestArchiveSweep_ContinuesAfterPerTargetFailure(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
) {
|
) {
|
||||||
@@ -560,21 +564,16 @@ func TestArchiveSweep_ContinuesAfterPerTargetFailure(
|
|||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
|
|
||||||
// Seeded first so the sweep reaches them before the healthy
|
// Seeded first so the sweep reaches it before the healthy
|
||||||
// target: targets come back in insertion order.
|
// target: targets come back in insertion order.
|
||||||
badConfig := env.seedDatabaseTarget(t, `{"expiry":"!!!"}`)
|
corrupt := env.seedDatabaseTarget(t, time.Hour, "")
|
||||||
env.seedArchiveRows(
|
|
||||||
t, badConfig, time.Now().Add(-48*time.Hour),
|
|
||||||
)
|
|
||||||
|
|
||||||
corrupt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
|
|
||||||
require.NoError(t, os.WriteFile(
|
require.NoError(t, os.WriteFile(
|
||||||
env.archivePath(corrupt),
|
env.archivePath(corrupt),
|
||||||
[]byte("this is not a sqlite database"),
|
[]byte("this is not a sqlite database"),
|
||||||
0o600,
|
0o600,
|
||||||
))
|
))
|
||||||
|
|
||||||
healthy := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
|
healthy := env.seedDatabaseTarget(t, time.Hour, "")
|
||||||
healthyPath := env.seedArchiveRows(
|
healthyPath := env.seedArchiveRows(
|
||||||
t, healthy,
|
t, healthy,
|
||||||
time.Now().Add(-48*time.Hour),
|
time.Now().Add(-48*time.Hour),
|
||||||
@@ -631,7 +630,7 @@ func TestArchiveSweep_PrunesIdleArchive(t *testing.T) {
|
|||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
|
|
||||||
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
|
tgt := env.seedDatabaseTarget(t, time.Hour, "")
|
||||||
|
|
||||||
now := time.Now()
|
now := time.Now()
|
||||||
path := env.seedArchiveRows(
|
path := env.seedArchiveRows(
|
||||||
@@ -670,7 +669,7 @@ func TestArchiveSweep_LeavesArchiveClosed(t *testing.T) {
|
|||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
|
|
||||||
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
|
tgt := env.seedDatabaseTarget(t, time.Hour, "")
|
||||||
path := env.seedArchiveRows(
|
path := env.seedArchiveRows(
|
||||||
t, tgt, time.Now().Add(-48*time.Hour),
|
t, tgt, time.Now().Add(-48*time.Hour),
|
||||||
)
|
)
|
||||||
@@ -710,7 +709,7 @@ func TestArchiveSweep_ClosesHandleOfRegisteredWriter(
|
|||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
|
|
||||||
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
|
tgt := env.seedDatabaseTarget(t, time.Hour, "")
|
||||||
env.seedArchiveRows(
|
env.seedArchiveRows(
|
||||||
t, tgt, time.Now().Add(-48*time.Hour),
|
t, tgt, time.Now().Add(-48*time.Hour),
|
||||||
)
|
)
|
||||||
@@ -795,36 +794,29 @@ func TestArchiveSweep_CancelledSweepLogsNoError(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// TestArchiveSweep_NeverExpiryUntouched proves the sweep is a
|
// TestArchiveSweep_NeverExpiryUntouched proves the sweep is a
|
||||||
// no-op for the default retention policy, so archives with no
|
// no-op for the default retention policy, so an archive with an
|
||||||
// expiry (or the literal "never") behave exactly as before.
|
// expiry of 0, which keeps it forever, is left alone.
|
||||||
func TestArchiveSweep_NeverExpiryUntouched(t *testing.T) {
|
func TestArchiveSweep_NeverExpiryUntouched(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
for _, configJSON := range []string{
|
env := setupArchiveTest(t)
|
||||||
`{"expiry":"never"}`,
|
|
||||||
`{"expiry":""}`,
|
|
||||||
"",
|
|
||||||
} {
|
|
||||||
env := setupArchiveTest(t)
|
|
||||||
|
|
||||||
tgt := env.seedDatabaseTarget(t, configJSON)
|
tgt := env.seedDatabaseTarget(t, 0, "")
|
||||||
path := env.seedArchiveRows(
|
path := env.seedArchiveRows(
|
||||||
t, tgt,
|
t, tgt,
|
||||||
time.Now().Add(-10000*time.Hour),
|
time.Now().Add(-10000*time.Hour),
|
||||||
)
|
)
|
||||||
|
|
||||||
env.sweeper.ExportSweep(context.Background())
|
env.sweeper.ExportSweep(context.Background())
|
||||||
|
|
||||||
assert.Equal(
|
assert.Equal(
|
||||||
t, []string{sweepRowOld}, archivedEventIDs(t, path),
|
t, []string{sweepRowOld}, archivedEventIDs(t, path),
|
||||||
"config %q must keep rows forever", configJSON,
|
"an expiry of 0 must keep rows forever",
|
||||||
)
|
)
|
||||||
assert.False(
|
assert.False(
|
||||||
t, env.writers.ExportHasArchiveWriter(tgt.ID),
|
t, env.writers.ExportHasArchiveWriter(tgt.ID),
|
||||||
"config %q must leave no registry entry behind",
|
"an expiry of 0 must leave no registry entry behind",
|
||||||
configJSON,
|
)
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestArchiveSweep_NeverExpirySkipsBeforeOpening pins the
|
// TestArchiveSweep_NeverExpirySkipsBeforeOpening pins the
|
||||||
@@ -845,7 +837,7 @@ func TestArchiveSweep_NeverExpirySkipsBeforeOpening(
|
|||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
|
|
||||||
tgt := env.seedDatabaseTarget(t, `{"expiry":"never"}`)
|
tgt := env.seedDatabaseTarget(t, 0, "")
|
||||||
path := env.archivePath(tgt)
|
path := env.archivePath(tgt)
|
||||||
|
|
||||||
seedUnmigratedArchive(t, path)
|
seedUnmigratedArchive(t, path)
|
||||||
@@ -897,7 +889,7 @@ func TestArchiveSweep_DoesNotCreateArchiveFile(t *testing.T) {
|
|||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
|
|
||||||
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
|
tgt := env.seedDatabaseTarget(t, time.Hour, "")
|
||||||
path := env.archivePath(tgt)
|
path := env.archivePath(tgt)
|
||||||
|
|
||||||
require.NoFileExists(t, path)
|
require.NoFileExists(t, path)
|
||||||
@@ -928,7 +920,7 @@ func TestArchiveSweep_DoesNotCreateAfterWriterExists(
|
|||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
|
|
||||||
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
|
tgt := env.seedDatabaseTarget(t, time.Hour, "")
|
||||||
|
|
||||||
path, err := env.writers.ExportEnsureArchiveWriter(tgt.ID)
|
path, err := env.writers.ExportEnsureArchiveWriter(tgt.ID)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
@@ -947,7 +939,7 @@ func TestArchiveSweep_SkipsDeletedWebhookTargets(t *testing.T) {
|
|||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
|
|
||||||
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
|
tgt := env.seedDatabaseTarget(t, time.Hour, "")
|
||||||
path := env.seedArchiveRows(
|
path := env.seedArchiveRows(
|
||||||
t, tgt, time.Now().Add(-48*time.Hour),
|
t, tgt, time.Now().Add(-48*time.Hour),
|
||||||
)
|
)
|
||||||
@@ -975,7 +967,7 @@ func TestArchiveSweep_ConcurrentWrites(t *testing.T) {
|
|||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
|
|
||||||
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
|
tgt := env.seedDatabaseTarget(t, time.Hour, "")
|
||||||
|
|
||||||
deliveries := make(
|
deliveries := make(
|
||||||
[]*database.Delivery, 0, sweepConcurrentWrites,
|
[]*database.Delivery, 0, sweepConcurrentWrites,
|
||||||
@@ -1020,7 +1012,7 @@ func TestArchiveSweeper_StopsCleanly(t *testing.T) {
|
|||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
|
|
||||||
tgt := env.seedDatabaseTarget(t, `{"expiry":"1h"}`)
|
tgt := env.seedDatabaseTarget(t, time.Hour, "")
|
||||||
env.seedArchiveRows(
|
env.seedArchiveRows(
|
||||||
t, tgt, time.Now().Add(-48*time.Hour),
|
t, tgt, time.Now().Add(-48*time.Hour),
|
||||||
)
|
)
|
||||||
|
|||||||
+14
-70
@@ -1,7 +1,6 @@
|
|||||||
package archive
|
package archive
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"encoding/json"
|
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io/fs"
|
"io/fs"
|
||||||
@@ -18,8 +17,9 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/gormlog"
|
"sneak.berlin/go/webhooker/internal/gormlog"
|
||||||
)
|
)
|
||||||
|
|
||||||
// ExpiryNever is the expiry sentinel (and default) that
|
// ExpiryNever is the archive expiry, as the target forms take it,
|
||||||
// disables pruning so archived rows are kept forever.
|
// that disables pruning so archived rows are kept forever. It is
|
||||||
|
// stored as an ArchiveExpiry of 0.
|
||||||
const ExpiryNever = "never"
|
const ExpiryNever = "never"
|
||||||
|
|
||||||
// archiveReopenDebounce bounds how often an archive file is
|
// archiveReopenDebounce bounds how often an archive file is
|
||||||
@@ -79,19 +79,6 @@ var (
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
// DatabaseTargetConfig is the optional per-target JSON config
|
|
||||||
// for a database (archive) target.
|
|
||||||
type DatabaseTargetConfig struct {
|
|
||||||
// Expiry is a Go duration (e.g. "720h") after which
|
|
||||||
// archived rows are pruned, or "never" (the default) to
|
|
||||||
// keep them forever.
|
|
||||||
Expiry string `json:"expiry"`
|
|
||||||
|
|
||||||
// Rotation is none (the default), monthly, daily or hourly: see
|
|
||||||
// archivePeriod.
|
|
||||||
Rotation string `json:"rotation"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// archivedEvent is one fully captured webhook event stored in a
|
// archivedEvent is one fully captured webhook event stored in a
|
||||||
// database target's archive for long-term retention. It is a
|
// database target's archive for long-term retention. It is a
|
||||||
// self-contained copy — independent of the per-webhook event
|
// self-contained copy — independent of the per-webhook event
|
||||||
@@ -112,63 +99,20 @@ type archivedEvent struct {
|
|||||||
ArchivedAt time.Time `gorm:"index"`
|
ArchivedAt time.Time `gorm:"index"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// ParseExpiry reads the optional expiry from a database
|
// ParseExpiry reads a user-supplied archive expiry for a database
|
||||||
// target's config JSON. An empty config, an empty expiry, or
|
// target, as the target forms submit it, into the ArchiveExpiry the
|
||||||
// the literal "never" all mean keep forever, returned as a zero
|
// target stores. Empty and "never" both mean keep forever, which is
|
||||||
// duration. Any other value must parse as a positive Go
|
// 0; anything else must be a positive Go duration such as "720h". Any
|
||||||
// duration; a set-but-invalid value (unparseable, zero, or
|
// other value is an error, so a bad expiry is refused at the form
|
||||||
// negative) is an error rather than a silent default, matching
|
// rather than stored.
|
||||||
// ValidateExpiry at target creation.
|
func ParseExpiry(expiry string) (time.Duration, error) {
|
||||||
func ParseExpiry(
|
|
||||||
configJSON string,
|
|
||||||
) (time.Duration, error) {
|
|
||||||
if configJSON == "" {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
var cfg DatabaseTargetConfig
|
|
||||||
|
|
||||||
err := json.Unmarshal([]byte(configJSON), &cfg)
|
|
||||||
if err != nil {
|
|
||||||
return 0, fmt.Errorf(
|
|
||||||
"parsing database target config: %w", err,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
if cfg.Expiry == "" || cfg.Expiry == ExpiryNever {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
dur, err := time.ParseDuration(cfg.Expiry)
|
|
||||||
if err != nil {
|
|
||||||
return 0, fmt.Errorf(
|
|
||||||
"parsing archive expiry %q: %w", cfg.Expiry, err,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
if dur <= 0 {
|
|
||||||
return 0, fmt.Errorf(
|
|
||||||
"%w: %q", errArchiveExpiryNotPositive, cfg.Expiry,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
return dur, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// ValidateExpiry checks a user-supplied archive expiry
|
|
||||||
// for a database target at configuration time. Valid values are
|
|
||||||
// empty, "never" (both meaning keep forever), or a positive Go
|
|
||||||
// duration such as "720h". Anything else is an error, so a bad
|
|
||||||
// expiry is rejected when the target is created rather than
|
|
||||||
// failing every subsequent delivery.
|
|
||||||
func ValidateExpiry(expiry string) error {
|
|
||||||
if expiry == "" || expiry == ExpiryNever {
|
if expiry == "" || expiry == ExpiryNever {
|
||||||
return nil
|
return 0, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
dur, err := time.ParseDuration(expiry)
|
dur, err := time.ParseDuration(expiry)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf(
|
return 0, fmt.Errorf(
|
||||||
"expiry must be %q or a Go duration "+
|
"expiry must be %q or a Go duration "+
|
||||||
"such as \"720h\": %w",
|
"such as \"720h\": %w",
|
||||||
ExpiryNever, err,
|
ExpiryNever, err,
|
||||||
@@ -176,12 +120,12 @@ func ValidateExpiry(expiry string) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if dur <= 0 {
|
if dur <= 0 {
|
||||||
return fmt.Errorf(
|
return 0, fmt.Errorf(
|
||||||
"%w: %q", errArchiveExpiryNotPositive, expiry,
|
"%w: %q", errArchiveExpiryNotPositive, expiry,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
return nil
|
return dur, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// archiveWriter owns one database target's archive SQLite files.
|
// archiveWriter owns one database target's archive SQLite files.
|
||||||
|
|||||||
@@ -28,7 +28,7 @@ func TestEventDBCreateOmitsAssociations(t *testing.T) {
|
|||||||
WebhookID: webhookID,
|
WebhookID: webhookID,
|
||||||
Name: "leaky-target",
|
Name: "leaky-target",
|
||||||
Type: database.TargetTypeSlack,
|
Type: database.TargetTypeSlack,
|
||||||
Config: `{"webhookUrl":"https://hooks.example/secret"}`,
|
URL: "https://hooks.example/secret",
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
}
|
}
|
||||||
target.ID = uuid.New().String()
|
target.ID = uuid.New().String()
|
||||||
|
|||||||
@@ -29,8 +29,10 @@ func marshalModel(t *testing.T, v any) string {
|
|||||||
// these tags the first handler that marshals a model serialises the
|
// these tags the first handler that marshals a model serialises the
|
||||||
// secret with it. Each field below is a live credential:
|
// secret with it. Each field below is a live credential:
|
||||||
//
|
//
|
||||||
// - Target.Config holds an incoming-webhook URL whose path segments
|
// - Target.URL is an incoming-webhook URL whose path segments are
|
||||||
// are the bearer token.
|
// the bearer token.
|
||||||
|
// - Target.Headers holds header values, routinely authorization
|
||||||
|
// tokens.
|
||||||
// - APIKey.Key is a bearer token outright.
|
// - APIKey.Key is a bearer token outright.
|
||||||
// - Setting.Value holds the session encryption key.
|
// - Setting.Value holds the session encryption key.
|
||||||
// - User.Password holds the Argon2 hash, and was already tagged.
|
// - User.Password holds the Argon2 hash, and was already tagged.
|
||||||
@@ -44,11 +46,20 @@ func TestModelsDoNotMarshalTheirSecrets(t *testing.T) {
|
|||||||
model any
|
model any
|
||||||
}{
|
}{
|
||||||
{
|
{
|
||||||
name: "target config",
|
name: "target url",
|
||||||
model: database.Target{
|
model: database.Target{
|
||||||
Name: keptField,
|
Name: keptField,
|
||||||
Type: database.TargetTypeSlack,
|
Type: database.TargetTypeSlack,
|
||||||
Config: `{"webhookUrl":"https://h/s/` + marker + `"}`,
|
URL: "https://h/s/" + marker,
|
||||||
|
MaxAttempts: 1,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "target headers",
|
||||||
|
model: database.Target{
|
||||||
|
Name: keptField,
|
||||||
|
Type: database.TargetTypeHTTP,
|
||||||
|
Headers: map[string]string{"Authorization": marker},
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
@@ -87,10 +98,10 @@ func TestModelsDoNotMarshalTheirSecrets(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestWebhookMarshalsNoTargetConfig covers the nested case: a webhook
|
// TestWebhookMarshalsNoTargetURL covers the nested case: a webhook
|
||||||
// marshalled with its targets preloaded must not carry the credential
|
// marshalled with its targets preloaded must not carry the credential
|
||||||
// through the association either.
|
// through the association either.
|
||||||
func TestWebhookMarshalsNoTargetConfig(t *testing.T) {
|
func TestWebhookMarshalsNoTargetURL(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const marker = "QQNESTEDMARKERQQ"
|
const marker = "QQNESTEDMARKERQQ"
|
||||||
@@ -99,7 +110,7 @@ func TestWebhookMarshalsNoTargetConfig(t *testing.T) {
|
|||||||
Name: keptField,
|
Name: keptField,
|
||||||
Targets: []database.Target{{
|
Targets: []database.Target{{
|
||||||
Name: "slack",
|
Name: "slack",
|
||||||
Config: `{"webhookUrl":"https://h/s/` + marker + `"}`,
|
URL: "https://h/s/" + marker,
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
}},
|
}},
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -1,5 +1,7 @@
|
|||||||
package database
|
package database
|
||||||
|
|
||||||
|
import "time"
|
||||||
|
|
||||||
// TargetType represents the type of delivery target
|
// TargetType represents the type of delivery target
|
||||||
type TargetType string
|
type TargetType string
|
||||||
|
|
||||||
@@ -11,7 +13,9 @@ const (
|
|||||||
TargetTypeSlack TargetType = "slack"
|
TargetTypeSlack TargetType = "slack"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Target represents a delivery target for a webhook
|
// Target represents a delivery target for a webhook. Its settings
|
||||||
|
// are the fields from URL to ArchiveRotation; each type uses some of
|
||||||
|
// them, and the ones a type does not use stay empty.
|
||||||
type Target struct {
|
type Target struct {
|
||||||
BaseModel
|
BaseModel
|
||||||
|
|
||||||
@@ -20,14 +24,35 @@ type Target struct {
|
|||||||
Type TargetType `gorm:"not null" json:"type"`
|
Type TargetType `gorm:"not null" json:"type"`
|
||||||
Active bool `gorm:"default:true" json:"active"`
|
Active bool `gorm:"default:true" json:"active"`
|
||||||
|
|
||||||
// Configuration fields (JSON stored based on type).
|
// URL is where an http or slack target sends each event.
|
||||||
//
|
//
|
||||||
// json:"-" because the blob holds the target's credential — a
|
// json:"-" because the URL is the target's credential: a Slack
|
||||||
// Slack incoming-webhook URL, or an http destination whose path
|
// incoming-webhook URL, or an http destination whose path segments
|
||||||
// segments are the secret. delivery.TargetView is the masking
|
// are the secret. delivery.TargetView is the masking barrier for
|
||||||
// barrier for the HTML path; this tag is the barrier for any
|
// the HTML path; this tag is the barrier for any handler that
|
||||||
// handler that marshals the model itself.
|
// marshals the model itself.
|
||||||
Config string `gorm:"type:text" json:"-"` // JSON configuration
|
URL string `json:"-"`
|
||||||
|
|
||||||
|
// Headers is an http target's extra request headers, name to
|
||||||
|
// value, stored as JSON text. json:"-" for the same reason as URL:
|
||||||
|
// a header value is routinely an authorization token.
|
||||||
|
Headers map[string]string `gorm:"serializer:json" json:"-"`
|
||||||
|
|
||||||
|
// TimeoutSeconds is an http target's per-request timeout; 0 uses
|
||||||
|
// the delivery engine's shared client timeout.
|
||||||
|
TimeoutSeconds int `json:"timeoutSeconds,omitempty"`
|
||||||
|
|
||||||
|
// ForwardQuery passes each event's query string on to an http
|
||||||
|
// target, appended to URL. Off, URL is sent exactly as stored.
|
||||||
|
ForwardQuery bool `json:"forwardQuery,omitempty"`
|
||||||
|
|
||||||
|
// ArchiveExpiry is how long a database target keeps each archived
|
||||||
|
// event; 0 keeps them forever.
|
||||||
|
ArchiveExpiry time.Duration `json:"archiveExpiry,omitempty"`
|
||||||
|
|
||||||
|
// ArchiveRotation is how often a database target starts a new
|
||||||
|
// archive file: none, monthly, daily or hourly.
|
||||||
|
ArchiveRotation string `json:"archiveRotation,omitempty"`
|
||||||
|
|
||||||
// MaxAttempts is how many times an HTTP or Slack target attempts
|
// MaxAttempts is how many times an HTTP or Slack target attempts
|
||||||
// each delivery in all, at least 1. 1 is a single attempt with no
|
// each delivery in all, at least 1. 1 is a single attempt with no
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ func newHTTPTarget(maxAttempts int) *database.Target {
|
|||||||
WebhookID: uuid.New().String(),
|
WebhookID: uuid.New().String(),
|
||||||
Name: "target",
|
Name: "target",
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: `{"url":"https://example.com/"}`,
|
URL: "https://example.com/",
|
||||||
MaxAttempts: maxAttempts,
|
MaxAttempts: maxAttempts,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -42,3 +42,22 @@ func TestTargetCreate_AcceptsOneAttempt(t *testing.T) {
|
|||||||
|
|
||||||
require.NoError(t, db.Create(newHTTPTarget(1)).Error)
|
require.NoError(t, db.Create(newHTTPTarget(1)).Error)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestTargetHeaders_StoredAsJSON reads the headers column back as
|
||||||
|
// text, to show it holds the header map as JSON.
|
||||||
|
func TestTargetHeaders_StoredAsJSON(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
db := startedTestDB(t)
|
||||||
|
|
||||||
|
target := newHTTPTarget(1)
|
||||||
|
target.Headers = map[string]string{"Authorization": "Bearer x"}
|
||||||
|
require.NoError(t, db.Create(target).Error)
|
||||||
|
|
||||||
|
var headers string
|
||||||
|
|
||||||
|
require.NoError(t, db.Raw(
|
||||||
|
"SELECT headers FROM targets WHERE id = ?", target.ID,
|
||||||
|
).Scan(&headers).Error)
|
||||||
|
assert.JSONEq(t, `{"Authorization":"Bearer x"}`, headers)
|
||||||
|
}
|
||||||
|
|||||||
@@ -87,13 +87,18 @@ func TestMainSchemaHoldsEveryModel(t *testing.T) {
|
|||||||
Active: true,
|
Active: true,
|
||||||
})
|
})
|
||||||
assertRoundTrip(t, db, &database.Target{
|
assertRoundTrip(t, db, &database.Target{
|
||||||
BaseModel: newBaseModel(),
|
BaseModel: newBaseModel(),
|
||||||
WebhookID: uuid.New().String(),
|
WebhookID: uuid.New().String(),
|
||||||
Name: "target",
|
Name: "target",
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Active: true,
|
Active: true,
|
||||||
Config: `{"url":"https://example.com/"}`,
|
URL: "https://example.com/",
|
||||||
MaxAttempts: 3,
|
Headers: map[string]string{"X-Api-Key": "key"},
|
||||||
|
TimeoutSeconds: 30,
|
||||||
|
ForwardQuery: true,
|
||||||
|
ArchiveExpiry: 720 * time.Hour,
|
||||||
|
ArchiveRotation: "daily",
|
||||||
|
MaxAttempts: 3,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -79,7 +79,12 @@ CREATE TABLE targets (
|
|||||||
name text NOT NULL,
|
name text NOT NULL,
|
||||||
type text NOT NULL,
|
type text NOT NULL,
|
||||||
active numeric DEFAULT true,
|
active numeric DEFAULT true,
|
||||||
config text,
|
url text,
|
||||||
|
headers text,
|
||||||
|
timeout_seconds integer,
|
||||||
|
forward_query numeric,
|
||||||
|
archive_expiry integer,
|
||||||
|
archive_rotation text,
|
||||||
max_attempts integer NOT NULL,
|
max_attempts integer NOT NULL,
|
||||||
PRIMARY KEY (id),
|
PRIMARY KEY (id),
|
||||||
CONSTRAINT fk_webhooks_targets
|
CONSTRAINT fk_webhooks_targets
|
||||||
|
|||||||
@@ -46,12 +46,7 @@ func TestClientForRequest_TimeoutKeepsSSRFGuard(t *testing.T) {
|
|||||||
t.Run(target, func(t *testing.T) {
|
t.Run(target, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
cfg := &delivery.HTTPTargetConfig{
|
client := engine.ExportClientForRequest(5, nil)
|
||||||
URL: target,
|
|
||||||
Timeout: 5,
|
|
||||||
}
|
|
||||||
|
|
||||||
client := engine.ExportClientForRequest(cfg, nil)
|
|
||||||
|
|
||||||
require.NotSame(t, engine.ExportClient(), client,
|
require.NotSame(t, engine.ExportClient(), client,
|
||||||
"a per-target timeout must yield a "+
|
"a per-target timeout must yield a "+
|
||||||
@@ -101,11 +96,7 @@ func TestClientForRequest_NoTimeoutUnchanged(t *testing.T) {
|
|||||||
|
|
||||||
engine := newSSRFTestEngine()
|
engine := newSSRFTestEngine()
|
||||||
|
|
||||||
cfg := &delivery.HTTPTargetConfig{
|
client := engine.ExportClientForRequest(0, nil)
|
||||||
URL: "https://example.com/hook",
|
|
||||||
}
|
|
||||||
|
|
||||||
client := engine.ExportClientForRequest(cfg, nil)
|
|
||||||
|
|
||||||
assert.Same(t, engine.ExportClient(), client,
|
assert.Same(t, engine.ExportClient(), client,
|
||||||
"without a per-target timeout the shared client "+
|
"without a per-target timeout the shared client "+
|
||||||
@@ -123,15 +114,8 @@ func TestClientForRequest_HeadersKeepSSRFGuard(t *testing.T) {
|
|||||||
|
|
||||||
engine := newSSRFTestEngine()
|
engine := newSSRFTestEngine()
|
||||||
|
|
||||||
cfg := &delivery.HTTPTargetConfig{
|
|
||||||
URL: "https://example.com/with-headers",
|
|
||||||
Headers: map[string]string{
|
|
||||||
"X-Api-Key": "configured",
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
client := engine.ExportClientForRequest(
|
client := engine.ExportClientForRequest(
|
||||||
cfg, []string{"X-Api-Key"},
|
0, []string{probeHeaderName},
|
||||||
)
|
)
|
||||||
|
|
||||||
require.NotNil(t, client.CheckRedirect,
|
require.NotNil(t, client.CheckRedirect,
|
||||||
|
|||||||
+41
-24
@@ -105,12 +105,19 @@ type Task struct {
|
|||||||
WebhookID string
|
WebhookID string
|
||||||
EntrypointID string
|
EntrypointID string
|
||||||
|
|
||||||
TargetID string
|
// The target's fields, as they were when the task was made.
|
||||||
TargetName string
|
TargetID string
|
||||||
TargetType database.TargetType
|
TargetName string
|
||||||
TargetConfig string
|
TargetType database.TargetType
|
||||||
MaxAttempts int
|
TargetURL string
|
||||||
|
TargetHeaders map[string]string
|
||||||
|
TimeoutSeconds int
|
||||||
|
ForwardQuery bool
|
||||||
|
ArchiveExpiry time.Duration
|
||||||
|
ArchiveRotation string
|
||||||
|
MaxAttempts int
|
||||||
|
|
||||||
|
// The event's fields.
|
||||||
Method string
|
Method string
|
||||||
RawQuery string
|
RawQuery string
|
||||||
Headers string
|
Headers string
|
||||||
@@ -1543,10 +1550,15 @@ func buildEventFromTask(task *Task) database.Event {
|
|||||||
|
|
||||||
func buildTargetFromTask(task *Task) database.Target {
|
func buildTargetFromTask(task *Task) database.Target {
|
||||||
target := database.Target{
|
target := database.Target{
|
||||||
Name: task.TargetName,
|
Name: task.TargetName,
|
||||||
Type: task.TargetType,
|
Type: task.TargetType,
|
||||||
Config: task.TargetConfig,
|
URL: task.TargetURL,
|
||||||
MaxAttempts: task.MaxAttempts,
|
Headers: task.TargetHeaders,
|
||||||
|
TimeoutSeconds: task.TimeoutSeconds,
|
||||||
|
ForwardQuery: task.ForwardQuery,
|
||||||
|
ArchiveExpiry: task.ArchiveExpiry,
|
||||||
|
ArchiveRotation: task.ArchiveRotation,
|
||||||
|
MaxAttempts: task.MaxAttempts,
|
||||||
}
|
}
|
||||||
|
|
||||||
target.ID = task.TargetID
|
target.ID = task.TargetID
|
||||||
@@ -1825,21 +1837,26 @@ func buildRecoveryTask(
|
|||||||
}
|
}
|
||||||
|
|
||||||
return Task{
|
return Task{
|
||||||
DeliveryID: d.ID,
|
DeliveryID: d.ID,
|
||||||
EventID: d.EventID,
|
EventID: d.EventID,
|
||||||
WebhookID: webhookID,
|
WebhookID: webhookID,
|
||||||
EntrypointID: event.EntrypointID,
|
EntrypointID: event.EntrypointID,
|
||||||
TargetID: target.ID,
|
TargetID: target.ID,
|
||||||
TargetName: target.Name,
|
TargetName: target.Name,
|
||||||
TargetType: target.Type,
|
TargetType: target.Type,
|
||||||
TargetConfig: target.Config,
|
TargetURL: target.URL,
|
||||||
MaxAttempts: target.MaxAttempts,
|
TargetHeaders: target.Headers,
|
||||||
Method: event.Method,
|
TimeoutSeconds: target.TimeoutSeconds,
|
||||||
RawQuery: event.RawQuery,
|
ForwardQuery: target.ForwardQuery,
|
||||||
Headers: event.Headers,
|
ArchiveExpiry: target.ArchiveExpiry,
|
||||||
ContentType: event.ContentType,
|
ArchiveRotation: target.ArchiveRotation,
|
||||||
Body: bodyPtr,
|
MaxAttempts: target.MaxAttempts,
|
||||||
AttemptNum: attemptNum,
|
Method: event.Method,
|
||||||
|
RawQuery: event.RawQuery,
|
||||||
|
Headers: event.Headers,
|
||||||
|
ContentType: event.ContentType,
|
||||||
|
Body: bodyPtr,
|
||||||
|
AttemptNum: attemptNum,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -3,7 +3,6 @@ package delivery_test
|
|||||||
import (
|
import (
|
||||||
"bytes"
|
"bytes"
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
@@ -97,17 +96,6 @@ func iSeedWebhookDB(
|
|||||||
return db
|
return db
|
||||||
}
|
}
|
||||||
|
|
||||||
func iHTTPConfig(url string) string {
|
|
||||||
cfg := delivery.HTTPTargetConfig{URL: url}
|
|
||||||
|
|
||||||
data, err := json.Marshal(cfg)
|
|
||||||
if err != nil {
|
|
||||||
panic("failed to marshal HTTPTargetConfig")
|
|
||||||
}
|
|
||||||
|
|
||||||
return string(data)
|
|
||||||
}
|
|
||||||
|
|
||||||
func iEngine(
|
func iEngine(
|
||||||
t *testing.T, workers int,
|
t *testing.T, workers int,
|
||||||
) *delivery.Engine {
|
) *delivery.Engine {
|
||||||
@@ -169,24 +157,24 @@ func iSeedDelivery(
|
|||||||
func iTask(
|
func iTask(
|
||||||
d database.Delivery,
|
d database.Delivery,
|
||||||
event database.Event,
|
event database.Event,
|
||||||
webhookID, targetID, name, config string,
|
webhookID, targetID, name, targetURL string,
|
||||||
maxAttempts, attemptNum int,
|
maxAttempts, attemptNum int,
|
||||||
body *string,
|
body *string,
|
||||||
) delivery.Task {
|
) delivery.Task {
|
||||||
return delivery.Task{
|
return delivery.Task{
|
||||||
DeliveryID: d.ID,
|
DeliveryID: d.ID,
|
||||||
EventID: event.ID,
|
EventID: event.ID,
|
||||||
WebhookID: webhookID,
|
WebhookID: webhookID,
|
||||||
TargetID: targetID,
|
TargetID: targetID,
|
||||||
TargetName: name,
|
TargetName: name,
|
||||||
TargetType: database.TargetTypeHTTP,
|
TargetType: database.TargetTypeHTTP,
|
||||||
TargetConfig: config,
|
TargetURL: targetURL,
|
||||||
MaxAttempts: maxAttempts,
|
MaxAttempts: maxAttempts,
|
||||||
Method: event.Method,
|
Method: event.Method,
|
||||||
Headers: event.Headers,
|
Headers: event.Headers,
|
||||||
ContentType: event.ContentType,
|
ContentType: event.ContentType,
|
||||||
Body: body,
|
Body: body,
|
||||||
AttemptNum: attemptNum,
|
AttemptNum: attemptNum,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -239,10 +227,9 @@ func TestProcessNewTask_InlineBody(t *testing.T) {
|
|||||||
)
|
)
|
||||||
|
|
||||||
bodyStr := event.Body
|
bodyStr := event.Body
|
||||||
cfg := iHTTPConfig(ts.URL)
|
|
||||||
task := iTask(
|
task := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"test-target", cfg, 1, 1, &bodyStr,
|
"test-target", ts.URL, 1, 1, &bodyStr,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportProcessNewTask(
|
s.Engine.ExportProcessNewTask(
|
||||||
@@ -290,10 +277,9 @@ func TestProcessNewTask_LargeBody_FetchFromDB(
|
|||||||
database.DeliveryStatusPending,
|
database.DeliveryStatusPending,
|
||||||
)
|
)
|
||||||
|
|
||||||
cfg := iHTTPConfig(ts.URL)
|
|
||||||
task := iTask(
|
task := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"test-large", cfg, 1, 1, nil,
|
"test-large", ts.URL, 1, 1, nil,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportProcessNewTask(
|
s.Engine.ExportProcessNewTask(
|
||||||
@@ -313,16 +299,16 @@ func TestProcessNewTask_InvalidWebhookID(t *testing.T) {
|
|||||||
s := newISetup(t)
|
s := newISetup(t)
|
||||||
|
|
||||||
task := delivery.Task{
|
task := delivery.Task{
|
||||||
DeliveryID: uuid.New().String(),
|
DeliveryID: uuid.New().String(),
|
||||||
EventID: uuid.New().String(),
|
EventID: uuid.New().String(),
|
||||||
WebhookID: uuid.New().String(),
|
WebhookID: uuid.New().String(),
|
||||||
TargetID: uuid.New().String(),
|
TargetID: uuid.New().String(),
|
||||||
TargetName: "test",
|
TargetName: "test",
|
||||||
TargetType: database.TargetTypeHTTP,
|
TargetType: database.TargetTypeHTTP,
|
||||||
TargetConfig: iHTTPConfig("http://localhost:9999"),
|
TargetURL: "http://localhost:9999",
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
Body: nil,
|
Body: nil,
|
||||||
AttemptNum: 1,
|
AttemptNum: 1,
|
||||||
}
|
}
|
||||||
|
|
||||||
s.Engine.ExportProcessNewTask(
|
s.Engine.ExportProcessNewTask(
|
||||||
@@ -363,7 +349,6 @@ func TestProcessRetryTask_SuccessfulRetry(t *testing.T) {
|
|||||||
)
|
)
|
||||||
|
|
||||||
bodyStr := event.Body
|
bodyStr := event.Body
|
||||||
cfg := iHTTPConfig(ts.URL)
|
|
||||||
|
|
||||||
// The target row exists because the engine confirms a scheduled
|
// The target row exists because the engine confirms a scheduled
|
||||||
// retry's target has not been deleted before it runs it. A retry
|
// retry's target has not been deleted before it runs it. A retry
|
||||||
@@ -372,12 +357,12 @@ func TestProcessRetryTask_SuccessfulRetry(t *testing.T) {
|
|||||||
// task. See https://git.eeqj.de/sneak/webhooker/issues/107.
|
// task. See https://git.eeqj.de/sneak/webhooker/issues/107.
|
||||||
iCreateTarget(
|
iCreateTarget(
|
||||||
t, s.MainDB, targetID, s.WebhookID, "retry-target",
|
t, s.MainDB, targetID, s.WebhookID, "retry-target",
|
||||||
database.TargetTypeHTTP, cfg, 5,
|
database.TargetTypeHTTP, ts.URL, 5,
|
||||||
)
|
)
|
||||||
|
|
||||||
task := iTask(
|
task := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"retry-target", cfg, 5, 2, &bodyStr,
|
"retry-target", ts.URL, 5, 2, &bodyStr,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportProcessRetryTask(
|
s.Engine.ExportProcessRetryTask(
|
||||||
@@ -410,10 +395,9 @@ func TestProcessRetryTask_SkipsNonRetryingDelivery(
|
|||||||
)
|
)
|
||||||
|
|
||||||
bodyStr := event.Body
|
bodyStr := event.Body
|
||||||
cfg := iHTTPConfig("http://localhost:1")
|
|
||||||
task := iTask(
|
task := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"skip-target", cfg, 5, 2, &bodyStr,
|
"skip-target", "http://localhost:1", 5, 2, &bodyStr,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportProcessRetryTask(
|
s.Engine.ExportProcessRetryTask(
|
||||||
@@ -460,16 +444,14 @@ func TestProcessRetryTask_LargeBody_FetchFromDB(
|
|||||||
database.DeliveryStatusRetrying,
|
database.DeliveryStatusRetrying,
|
||||||
)
|
)
|
||||||
|
|
||||||
cfg := iHTTPConfig(ts.URL)
|
|
||||||
|
|
||||||
iCreateTarget(
|
iCreateTarget(
|
||||||
t, s.MainDB, targetID, s.WebhookID, "retry-large",
|
t, s.MainDB, targetID, s.WebhookID, "retry-large",
|
||||||
database.TargetTypeHTTP, cfg, 5,
|
database.TargetTypeHTTP, ts.URL, 5,
|
||||||
)
|
)
|
||||||
|
|
||||||
task := iTask(
|
task := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"retry-large", cfg, 5, 2, nil,
|
"retry-large", ts.URL, 5, 2, nil,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportProcessRetryTask(
|
s.Engine.ExportProcessRetryTask(
|
||||||
@@ -570,16 +552,15 @@ func TestWorkerLifecycle_ProcessesRetryChannel(
|
|||||||
s.Engine.ExportStart()
|
s.Engine.ExportStart()
|
||||||
|
|
||||||
bodyStr := event.Body
|
bodyStr := event.Body
|
||||||
cfg := iHTTPConfig(ts.URL)
|
|
||||||
|
|
||||||
iCreateTarget(
|
iCreateTarget(
|
||||||
t, s.MainDB, targetID, s.WebhookID, "retry-chan-test",
|
t, s.MainDB, targetID, s.WebhookID, "retry-chan-test",
|
||||||
database.TargetTypeHTTP, cfg, 5,
|
database.TargetTypeHTTP, ts.URL, 5,
|
||||||
)
|
)
|
||||||
|
|
||||||
task := iTask(
|
task := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"retry-chan-test", cfg, 5, 2, &bodyStr,
|
"retry-chan-test", ts.URL, 5, 2, &bodyStr,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportRetryCh() <- task
|
s.Engine.ExportRetryCh() <- task
|
||||||
@@ -685,7 +666,7 @@ func iCreateTarget(
|
|||||||
mainDB *gorm.DB,
|
mainDB *gorm.DB,
|
||||||
targetID, webhookID, name string,
|
targetID, webhookID, name string,
|
||||||
targetType database.TargetType,
|
targetType database.TargetType,
|
||||||
config string,
|
targetURL string,
|
||||||
maxAttempts int,
|
maxAttempts int,
|
||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
@@ -695,7 +676,7 @@ func iCreateTarget(
|
|||||||
Name: name,
|
Name: name,
|
||||||
Type: targetType,
|
Type: targetType,
|
||||||
Active: true,
|
Active: true,
|
||||||
Config: config,
|
URL: targetURL,
|
||||||
MaxAttempts: maxAttempts,
|
MaxAttempts: maxAttempts,
|
||||||
}
|
}
|
||||||
target.ID = targetID
|
target.ID = targetID
|
||||||
@@ -736,7 +717,7 @@ func TestRecoverWebhookDeliveries_RetryingDeliveries(
|
|||||||
iCreateTarget(t, s.MainDB, targetID,
|
iCreateTarget(t, s.MainDB, targetID,
|
||||||
s.WebhookID, "retry-recovery",
|
s.WebhookID, "retry-recovery",
|
||||||
database.TargetTypeHTTP,
|
database.TargetTypeHTTP,
|
||||||
iHTTPConfig("http://example.com/hook"), 5,
|
"http://example.com/hook", 5,
|
||||||
)
|
)
|
||||||
|
|
||||||
event := iSeedEvent(
|
event := iSeedEvent(
|
||||||
@@ -796,7 +777,7 @@ func iSeedRetryingWithType(
|
|||||||
|
|
||||||
iCreateTarget(t, s.MainDB, targetID,
|
iCreateTarget(t, s.MainDB, targetID,
|
||||||
s.WebhookID, "mutated-target", targetType,
|
s.WebhookID, "mutated-target", targetType,
|
||||||
iHTTPConfig("http://example.com/hook"), 5,
|
"http://example.com/hook", 5,
|
||||||
)
|
)
|
||||||
|
|
||||||
event := iSeedEvent(
|
event := iSeedEvent(
|
||||||
@@ -934,14 +915,14 @@ func TestFailUnretryableRetry_WritesNoTargetRow(
|
|||||||
|
|
||||||
targetID := uuid.New().String()
|
targetID := uuid.New().String()
|
||||||
|
|
||||||
// A Slack incoming-webhook URL: the target config IS the
|
// A Slack incoming-webhook URL: the target's URL IS the
|
||||||
// credential, which is what makes a leaked target row a
|
// credential, which is what makes a leaked target row a
|
||||||
// disclosure rather than a curiosity.
|
// disclosure rather than a curiosity.
|
||||||
hookURL := "https://hooks.slack.com/services/T00/B00/x"
|
hookURL := "https://hooks.slack.com/services/T00/B00/x"
|
||||||
|
|
||||||
iCreateTarget(t, s.MainDB, targetID,
|
iCreateTarget(t, s.MainDB, targetID,
|
||||||
s.WebhookID, "credential-bearing",
|
s.WebhookID, "credential-bearing",
|
||||||
database.TargetTypeLog, iHTTPConfig(hookURL), 5,
|
database.TargetTypeLog, hookURL, 5,
|
||||||
)
|
)
|
||||||
|
|
||||||
event := iSeedEvent(
|
event := iSeedEvent(
|
||||||
@@ -1186,7 +1167,7 @@ func TestRecoverDeliveries_SkipsAWebhookDeletedAfterTheListIsRead(
|
|||||||
assert.False(t, dbMgr.DBExists(webhookID))
|
assert.False(t, dbMgr.DBExists(webhookID))
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- HTTP Config with custom headers ---
|
// --- HTTP target with custom headers ---
|
||||||
|
|
||||||
func TestDeliverHTTP_CustomTargetHeaders(t *testing.T) {
|
func TestDeliverHTTP_CustomTargetHeaders(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
@@ -1208,16 +1189,6 @@ func TestDeliverHTTP_CustomTargetHeaders(t *testing.T) {
|
|||||||
)
|
)
|
||||||
defer ts.Close()
|
defer ts.Close()
|
||||||
|
|
||||||
cfg := delivery.HTTPTargetConfig{
|
|
||||||
URL: ts.URL,
|
|
||||||
Headers: map[string]string{
|
|
||||||
"Authorization": "Bearer secret-token",
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
cfgJSON, err := json.Marshal(cfg)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
event := iSeedEvent(
|
event := iSeedEvent(
|
||||||
t, s.WebhookDB, s.WebhookID,
|
t, s.WebhookDB, s.WebhookID,
|
||||||
`{"auth":"test"}`,
|
`{"auth":"test"}`,
|
||||||
@@ -1232,9 +1203,12 @@ func TestDeliverHTTP_CustomTargetHeaders(t *testing.T) {
|
|||||||
bodyStr := event.Body
|
bodyStr := event.Body
|
||||||
task := iTask(
|
task := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"auth-target", string(cfgJSON),
|
"auth-target", ts.URL,
|
||||||
1, 1, &bodyStr,
|
1, 1, &bodyStr,
|
||||||
)
|
)
|
||||||
|
task.TargetHeaders = map[string]string{
|
||||||
|
headerAuthorization: "Bearer secret-token",
|
||||||
|
}
|
||||||
|
|
||||||
s.Engine.ExportProcessNewTask(
|
s.Engine.ExportProcessNewTask(
|
||||||
context.TODO(), &task,
|
context.TODO(), &task,
|
||||||
@@ -1247,11 +1221,14 @@ func TestDeliverHTTP_CustomTargetHeaders(t *testing.T) {
|
|||||||
|
|
||||||
// --- HTTP delivery with custom timeout ---
|
// --- HTTP delivery with custom timeout ---
|
||||||
|
|
||||||
|
// TestDeliverHTTP_TargetTimeout proves a task's TimeoutSeconds reaches
|
||||||
|
// the delivery through the engine. The destination answers after two
|
||||||
|
// seconds, inside the engine's own five-second client timeout, so only
|
||||||
|
// the task's one second can fail the delivery.
|
||||||
func TestDeliverHTTP_TargetTimeout(t *testing.T) {
|
func TestDeliverHTTP_TargetTimeout(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
db := testWebhookDB(t)
|
s := newISetup(t)
|
||||||
e := iEngine(t, 1)
|
|
||||||
|
|
||||||
ts := httptest.NewServer(
|
ts := httptest.NewServer(
|
||||||
http.HandlerFunc(
|
http.HandlerFunc(
|
||||||
@@ -1263,31 +1240,30 @@ func TestDeliverHTTP_TargetTimeout(t *testing.T) {
|
|||||||
)
|
)
|
||||||
defer ts.Close()
|
defer ts.Close()
|
||||||
|
|
||||||
cfg := delivery.HTTPTargetConfig{
|
event := iSeedEvent(
|
||||||
URL: ts.URL,
|
t, s.WebhookDB, s.WebhookID, `{"timeout":"test"}`,
|
||||||
Timeout: 1,
|
)
|
||||||
}
|
targetID := uuid.New().String()
|
||||||
|
|
||||||
cfgJSON, err := json.Marshal(cfg)
|
d := iSeedDelivery(
|
||||||
require.NoError(t, err)
|
t, s.WebhookDB, event.ID, targetID,
|
||||||
|
database.DeliveryStatusPending,
|
||||||
event, del := iSeedEventAndDelivery(
|
|
||||||
t, db, `{"timeout":"test"}`,
|
|
||||||
string(cfgJSON),
|
|
||||||
)
|
)
|
||||||
|
|
||||||
task, d := iHTTPTaskAndDelivery(
|
bodyStr := event.Body
|
||||||
event, del, "timeout-target",
|
task := iTask(
|
||||||
string(cfgJSON), 1, 1,
|
d, event, s.WebhookID, targetID,
|
||||||
|
"timeout-target", ts.URL, 1, 1, &bodyStr,
|
||||||
)
|
)
|
||||||
|
task.TimeoutSeconds = 1
|
||||||
|
|
||||||
e.ExportDeliverHTTP(context.TODO(), db, d, task)
|
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
||||||
|
|
||||||
iAssertStatus(t, db, del.ID,
|
iAssertStatus(t, s.WebhookDB, d.ID,
|
||||||
database.DeliveryStatusFailed,
|
database.DeliveryStatusFailed,
|
||||||
)
|
)
|
||||||
|
|
||||||
iAssertResultFailed(t, db, del.ID)
|
iAssertResultFailed(t, s.WebhookDB, d.ID)
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog pins the size
|
// TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog pins the size
|
||||||
@@ -1325,8 +1301,6 @@ func TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog(
|
|||||||
))
|
))
|
||||||
defer ts.Close()
|
defer ts.Close()
|
||||||
|
|
||||||
cfgJSON := iHTTPConfig(ts.URL)
|
|
||||||
|
|
||||||
event := iSeedEvent(
|
event := iSeedEvent(
|
||||||
t, s.WebhookDB, s.WebhookID, `{"cut":"test"}`,
|
t, s.WebhookDB, s.WebhookID, `{"cut":"test"}`,
|
||||||
)
|
)
|
||||||
@@ -1340,7 +1314,7 @@ func TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog(
|
|||||||
bodyStr := event.Body
|
bodyStr := event.Body
|
||||||
task := iTask(
|
task := iTask(
|
||||||
del, event, s.WebhookID, targetID,
|
del, event, s.WebhookID, targetID,
|
||||||
"cut-target", cfgJSON, 1, 1, &bodyStr,
|
"cut-target", ts.URL, 1, 1, &bodyStr,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
||||||
@@ -1372,7 +1346,7 @@ func TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog(
|
|||||||
func iSeedEventAndDelivery(
|
func iSeedEventAndDelivery(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
db *gorm.DB,
|
db *gorm.DB,
|
||||||
body, _ string,
|
body string,
|
||||||
) (database.Event, database.Delivery) {
|
) (database.Event, database.Delivery) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
@@ -1403,19 +1377,19 @@ func iSeedEventAndDelivery(
|
|||||||
func iHTTPTaskAndDelivery(
|
func iHTTPTaskAndDelivery(
|
||||||
event database.Event,
|
event database.Event,
|
||||||
del database.Delivery,
|
del database.Delivery,
|
||||||
name, config string,
|
name, targetURL string,
|
||||||
maxAttempts, attemptNum int,
|
maxAttempts, attemptNum int,
|
||||||
) (*delivery.Task, *database.Delivery) {
|
) (*delivery.Task, *database.Delivery) {
|
||||||
task := &delivery.Task{
|
task := &delivery.Task{
|
||||||
DeliveryID: del.ID,
|
DeliveryID: del.ID,
|
||||||
EventID: event.ID,
|
EventID: event.ID,
|
||||||
WebhookID: event.WebhookID,
|
WebhookID: event.WebhookID,
|
||||||
TargetID: del.TargetID,
|
TargetID: del.TargetID,
|
||||||
TargetName: name,
|
TargetName: name,
|
||||||
TargetType: database.TargetTypeHTTP,
|
TargetType: database.TargetTypeHTTP,
|
||||||
TargetConfig: config,
|
TargetURL: targetURL,
|
||||||
MaxAttempts: maxAttempts,
|
MaxAttempts: maxAttempts,
|
||||||
AttemptNum: attemptNum,
|
AttemptNum: attemptNum,
|
||||||
}
|
}
|
||||||
|
|
||||||
d := &database.Delivery{
|
d := &database.Delivery{
|
||||||
@@ -1426,7 +1400,7 @@ func iHTTPTaskAndDelivery(
|
|||||||
Target: database.Target{
|
Target: database.Target{
|
||||||
Name: name,
|
Name: name,
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: config,
|
URL: targetURL,
|
||||||
MaxAttempts: maxAttempts,
|
MaxAttempts: maxAttempts,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@@ -1455,53 +1429,27 @@ func iAssertResultFailed(
|
|||||||
assert.NotEmpty(t, result.Error)
|
assert.NotEmpty(t, result.Error)
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- HTTP request with invalid config ---
|
// --- HTTP request with no URL ---
|
||||||
|
|
||||||
func TestDeliverHTTP_InvalidConfig(t *testing.T) {
|
// TestDeliverHTTP_NoURL proves an http target without a URL, which
|
||||||
|
// the forms never store, fails its delivery and records why.
|
||||||
|
func TestDeliverHTTP_NoURL(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
db := testWebhookDB(t)
|
db := testWebhookDB(t)
|
||||||
e := iEngine(t, 1)
|
e := iEngine(t, 1)
|
||||||
|
|
||||||
event, del := iSeedEventAndDelivery(
|
event, del := iSeedEventAndDelivery(t, db, `{"url":"none"}`)
|
||||||
t, db, `{"config":"invalid"}`, "",
|
|
||||||
)
|
|
||||||
|
|
||||||
task, d := iHTTPTaskAndDelivery(
|
task, d := iHTTPTaskAndDelivery(event, del, "no-url", "", 1, 1)
|
||||||
event, del, "bad-config", `not-json`, 1, 1,
|
|
||||||
)
|
|
||||||
|
|
||||||
e.ExportDeliverHTTP(context.TODO(), db, d, task)
|
e.ExportDeliverHTTP(context.TODO(), db, d, task)
|
||||||
|
|
||||||
iAssertStatus(t, db, del.ID,
|
iAssertStatus(t, db, del.ID,
|
||||||
database.DeliveryStatusFailed,
|
database.DeliveryStatusFailed,
|
||||||
)
|
)
|
||||||
}
|
|
||||||
|
|
||||||
// TestDeliverHTTP_InvalidConfigUnrecordedStaysPending: a delivery is
|
iAssertResultFailed(t, db, del.ID)
|
||||||
// failed for an invalid config only once the reason is recorded.
|
|
||||||
// Unrecorded, it stays pending, where the sweep finds it again.
|
|
||||||
func TestDeliverHTTP_InvalidConfigUnrecordedStaysPending(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
db := testWebhookDB(t)
|
|
||||||
e := testEngine(t, 1)
|
|
||||||
|
|
||||||
event, del := iSeedEventAndDelivery(
|
|
||||||
t, db, `{"config":"invalid"}`, "",
|
|
||||||
)
|
|
||||||
|
|
||||||
task, d := iHTTPTaskAndDelivery(
|
|
||||||
event, del, "bad-config", `not-json`, 1, 1,
|
|
||||||
)
|
|
||||||
|
|
||||||
require.NoError(t, db.Exec("drop table delivery_results").Error)
|
|
||||||
|
|
||||||
e.ExportDeliverHTTP(context.TODO(), db, d, task)
|
|
||||||
|
|
||||||
iAssertStatus(t, db, del.ID,
|
|
||||||
database.DeliveryStatusPending,
|
|
||||||
)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- Notify batching ---
|
// --- Notify batching ---
|
||||||
|
|||||||
@@ -309,13 +309,12 @@ func seedHeldDelivery(
|
|||||||
))
|
))
|
||||||
t.Cleanup(ts.Close)
|
t.Cleanup(ts.Close)
|
||||||
|
|
||||||
cfg := iHTTPConfig(ts.URL)
|
|
||||||
targetID := uuid.New().String()
|
targetID := uuid.New().String()
|
||||||
|
|
||||||
iCreateWebhook(t, s.MainDB, s.WebhookID, "hook")
|
iCreateWebhook(t, s.MainDB, s.WebhookID, "hook")
|
||||||
iCreateTarget(
|
iCreateTarget(
|
||||||
t, s.MainDB, targetID, s.WebhookID, "held",
|
t, s.MainDB, targetID, s.WebhookID, "held",
|
||||||
database.TargetTypeHTTP, cfg, maxAttempts,
|
database.TargetTypeHTTP, ts.URL, maxAttempts,
|
||||||
)
|
)
|
||||||
|
|
||||||
event := iSeedEvent(t, s.WebhookDB, s.WebhookID, `{"held":true}`)
|
event := iSeedEvent(t, s.WebhookDB, s.WebhookID, `{"held":true}`)
|
||||||
@@ -324,7 +323,7 @@ func seedHeldDelivery(
|
|||||||
bodyStr := event.Body
|
bodyStr := event.Body
|
||||||
|
|
||||||
return iTask(
|
return iTask(
|
||||||
d, event, s.WebhookID, targetID, "held", cfg,
|
d, event, s.WebhookID, targetID, "held", ts.URL,
|
||||||
maxAttempts, attemptNum, &bodyStr,
|
maxAttempts, attemptNum, &bodyStr,
|
||||||
), arrived
|
), arrived
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -65,20 +65,6 @@ func testEngine(
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
func newHTTPTargetConfig(url string) string {
|
|
||||||
cfg := delivery.HTTPTargetConfig{URL: url}
|
|
||||||
|
|
||||||
data, err := json.Marshal(cfg)
|
|
||||||
if err != nil {
|
|
||||||
panic(
|
|
||||||
"failed to marshal HTTPTargetConfig: " +
|
|
||||||
err.Error(),
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
return string(data)
|
|
||||||
}
|
|
||||||
|
|
||||||
func seedEvent(
|
func seedEvent(
|
||||||
t *testing.T, db *gorm.DB, body string,
|
t *testing.T, db *gorm.DB, body string,
|
||||||
) database.Event {
|
) database.Event {
|
||||||
@@ -141,19 +127,19 @@ type httpDeliveryFixture struct {
|
|||||||
func buildHTTPFixture(
|
func buildHTTPFixture(
|
||||||
dlv database.Delivery,
|
dlv database.Delivery,
|
||||||
event database.Event,
|
event database.Event,
|
||||||
targetID, name, config string,
|
targetID, name, targetURL string,
|
||||||
maxAttempts, attemptNum int,
|
maxAttempts, attemptNum int,
|
||||||
) httpDeliveryFixture {
|
) httpDeliveryFixture {
|
||||||
task := &delivery.Task{
|
task := &delivery.Task{
|
||||||
DeliveryID: dlv.ID,
|
DeliveryID: dlv.ID,
|
||||||
EventID: event.ID,
|
EventID: event.ID,
|
||||||
WebhookID: event.WebhookID,
|
WebhookID: event.WebhookID,
|
||||||
TargetID: targetID,
|
TargetID: targetID,
|
||||||
TargetName: name,
|
TargetName: name,
|
||||||
TargetType: database.TargetTypeHTTP,
|
TargetType: database.TargetTypeHTTP,
|
||||||
TargetConfig: config,
|
TargetURL: targetURL,
|
||||||
MaxAttempts: maxAttempts,
|
MaxAttempts: maxAttempts,
|
||||||
AttemptNum: attemptNum,
|
AttemptNum: attemptNum,
|
||||||
}
|
}
|
||||||
|
|
||||||
d := &database.Delivery{
|
d := &database.Delivery{
|
||||||
@@ -164,7 +150,7 @@ func buildHTTPFixture(
|
|||||||
Target: database.Target{
|
Target: database.Target{
|
||||||
Name: name,
|
Name: name,
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: config,
|
URL: targetURL,
|
||||||
MaxAttempts: maxAttempts,
|
MaxAttempts: maxAttempts,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@@ -275,9 +261,8 @@ func TestDeliverHTTP_Success(t *testing.T) {
|
|||||||
database.DeliveryStatusPending,
|
database.DeliveryStatusPending,
|
||||||
)
|
)
|
||||||
|
|
||||||
cfg := newHTTPTargetConfig(ts.URL)
|
|
||||||
fix := buildHTTPFixture(
|
fix := buildHTTPFixture(
|
||||||
dlv, event, targetID, "test-http", cfg, 1, 1,
|
dlv, event, targetID, "test-http", ts.URL, 1, 1,
|
||||||
)
|
)
|
||||||
|
|
||||||
e.ExportDeliverHTTP(
|
e.ExportDeliverHTTP(
|
||||||
@@ -321,10 +306,9 @@ func TestDeliverHTTP_Failure(t *testing.T) {
|
|||||||
database.DeliveryStatusPending,
|
database.DeliveryStatusPending,
|
||||||
)
|
)
|
||||||
|
|
||||||
cfg := newHTTPTargetConfig(ts.URL)
|
|
||||||
fix := buildHTTPFixture(
|
fix := buildHTTPFixture(
|
||||||
dlv, event, targetID,
|
dlv, event, targetID,
|
||||||
"test-http-fail", cfg, 1, 1,
|
"test-http-fail", ts.URL, 1, 1,
|
||||||
)
|
)
|
||||||
|
|
||||||
e.ExportDeliverHTTP(
|
e.ExportDeliverHTTP(
|
||||||
@@ -351,7 +335,7 @@ func TestDeliverDatabase_ImmediateSuccess(
|
|||||||
// The database target archives for real, so the engine needs
|
// The database target archives for real, so the engine needs
|
||||||
// the target in the main database and a data directory.
|
// the target in the main database and a data directory.
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(t, "")
|
tgt := env.seedDatabaseTarget(t)
|
||||||
|
|
||||||
event := seedEvent(t, db, `{"db":"target"}`)
|
event := seedEvent(t, db, `{"db":"target"}`)
|
||||||
d := seedDatabaseTargetDelivery(t, db, event, tgt)
|
d := seedDatabaseTargetDelivery(t, db, event, tgt)
|
||||||
@@ -452,10 +436,9 @@ func TestDeliverHTTP_WithRetries_Success(t *testing.T) {
|
|||||||
database.DeliveryStatusPending,
|
database.DeliveryStatusPending,
|
||||||
)
|
)
|
||||||
|
|
||||||
cfg := newHTTPTargetConfig(ts.URL)
|
|
||||||
fix := buildHTTPFixture(
|
fix := buildHTTPFixture(
|
||||||
dlv, event, targetID,
|
dlv, event, targetID,
|
||||||
"test-http-retry", cfg, 5, 1,
|
"test-http-retry", ts.URL, 5, 1,
|
||||||
)
|
)
|
||||||
|
|
||||||
e.ExportDeliverHTTP(
|
e.ExportDeliverHTTP(
|
||||||
@@ -510,13 +493,12 @@ func TestDeliverHTTP_MaxAttemptsExhausted(t *testing.T) {
|
|||||||
database.DeliveryStatusPending,
|
database.DeliveryStatusPending,
|
||||||
)
|
)
|
||||||
|
|
||||||
cfg := newHTTPTargetConfig(ts.URL)
|
|
||||||
sched := &recordingScheduler{}
|
sched := &recordingScheduler{}
|
||||||
|
|
||||||
for attempt := 1; attempt <= maxAttempts; attempt++ {
|
for attempt := 1; attempt <= maxAttempts; attempt++ {
|
||||||
fix := buildHTTPFixture(
|
fix := buildHTTPFixture(
|
||||||
dlv, event, targetID,
|
dlv, event, targetID,
|
||||||
"test-http-exhaust", cfg, maxAttempts, attempt,
|
"test-http-exhaust", ts.URL, maxAttempts, attempt,
|
||||||
)
|
)
|
||||||
|
|
||||||
e.ExportDeliverHTTPWithScheduler(
|
e.ExportDeliverHTTPWithScheduler(
|
||||||
@@ -574,10 +556,9 @@ func TestDeliverHTTP_SingleAttemptSkipsCircuitBreaker(t *testing.T) {
|
|||||||
database.DeliveryStatusPending,
|
database.DeliveryStatusPending,
|
||||||
)
|
)
|
||||||
|
|
||||||
cfg := newHTTPTargetConfig(ts.URL)
|
|
||||||
fix := buildHTTPFixture(
|
fix := buildHTTPFixture(
|
||||||
dlv, event, targetID,
|
dlv, event, targetID,
|
||||||
"test-http-single", cfg, 1, 1,
|
"test-http-single", ts.URL, 1, 1,
|
||||||
)
|
)
|
||||||
sched := &recordingScheduler{}
|
sched := &recordingScheduler{}
|
||||||
|
|
||||||
@@ -624,10 +605,9 @@ func TestDeliverHTTP_SchedulesRetryOnFailure(
|
|||||||
database.DeliveryStatusPending,
|
database.DeliveryStatusPending,
|
||||||
)
|
)
|
||||||
|
|
||||||
cfg := newHTTPTargetConfig(ts.URL)
|
|
||||||
fix := buildHTTPFixture(
|
fix := buildHTTPFixture(
|
||||||
dlv, event, targetID,
|
dlv, event, targetID,
|
||||||
"test-http-schedule", cfg, 5, 1,
|
"test-http-schedule", ts.URL, 5, 1,
|
||||||
)
|
)
|
||||||
|
|
||||||
e.ExportDeliverHTTP(
|
e.ExportDeliverHTTP(
|
||||||
@@ -789,7 +769,7 @@ func seedConcurrencyTasks(
|
|||||||
t *testing.T,
|
t *testing.T,
|
||||||
db *gorm.DB,
|
db *gorm.DB,
|
||||||
n int,
|
n int,
|
||||||
targetCfg string,
|
targetURL string,
|
||||||
) ([]database.Delivery, []delivery.Task) {
|
) ([]database.Delivery, []delivery.Task) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
@@ -815,21 +795,21 @@ func seedConcurrencyTasks(
|
|||||||
Target: database.Target{
|
Target: database.Target{
|
||||||
Name: fmt.Sprintf("task-%d", i),
|
Name: fmt.Sprintf("task-%d", i),
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: targetCfg,
|
URL: targetURL,
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
tasks[i].ID = dlv.ID
|
tasks[i].ID = dlv.ID
|
||||||
|
|
||||||
dtasks[i] = delivery.Task{
|
dtasks[i] = delivery.Task{
|
||||||
DeliveryID: tasks[i].ID,
|
DeliveryID: tasks[i].ID,
|
||||||
EventID: tasks[i].EventID,
|
EventID: tasks[i].EventID,
|
||||||
TargetID: tasks[i].TargetID,
|
TargetID: tasks[i].TargetID,
|
||||||
TargetName: tasks[i].Target.Name,
|
TargetName: tasks[i].Target.Name,
|
||||||
TargetType: tasks[i].Target.Type,
|
TargetType: tasks[i].Target.Type,
|
||||||
TargetConfig: tasks[i].Target.Config,
|
TargetURL: tasks[i].Target.URL,
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
AttemptNum: 1,
|
AttemptNum: 1,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -865,9 +845,8 @@ func TestWorkerPool_BoundedConcurrency(t *testing.T) {
|
|||||||
defer ts.Close()
|
defer ts.Close()
|
||||||
|
|
||||||
e := testEngine(t, numWorkers)
|
e := testEngine(t, numWorkers)
|
||||||
cfg := newHTTPTargetConfig(ts.URL)
|
|
||||||
tasks, dtasks := seedConcurrencyTasks(
|
tasks, dtasks := seedConcurrencyTasks(
|
||||||
t, db, numTasks, cfg,
|
t, db, numTasks, ts.URL,
|
||||||
)
|
)
|
||||||
|
|
||||||
runConcurrencyWorkers(
|
runConcurrencyWorkers(
|
||||||
@@ -948,12 +927,9 @@ func TestDeliverHTTP_CircuitBreakerBlocks(t *testing.T) {
|
|||||||
database.DeliveryStatusPending,
|
database.DeliveryStatusPending,
|
||||||
)
|
)
|
||||||
|
|
||||||
cfg := newHTTPTargetConfig(
|
|
||||||
"http://will-not-be-called.invalid",
|
|
||||||
)
|
|
||||||
fix := buildHTTPFixture(
|
fix := buildHTTPFixture(
|
||||||
dlv, event, targetID,
|
dlv, event, targetID,
|
||||||
"test-cb-block", cfg, 5, 1,
|
"test-cb-block", "http://will-not-be-called.invalid", 5, 1,
|
||||||
)
|
)
|
||||||
|
|
||||||
e.ExportDeliverHTTP(
|
e.ExportDeliverHTTP(
|
||||||
@@ -1016,9 +992,8 @@ func TestDeliverHTTP_HalfOpenBreakerDelaysQueuedTasks(t *testing.T) {
|
|||||||
require.True(t, cb.Allow(), "the probe delivery should go through")
|
require.True(t, cb.Allow(), "the probe delivery should go through")
|
||||||
require.Equal(t, delivery.CircuitHalfOpen, cb.State())
|
require.Equal(t, delivery.CircuitHalfOpen, cb.State())
|
||||||
|
|
||||||
cfg := newHTTPTargetConfig(
|
const targetURL = "http://will-not-be-called.invalid"
|
||||||
"http://will-not-be-called.invalid",
|
|
||||||
)
|
|
||||||
sched := &recordingScheduler{}
|
sched := &recordingScheduler{}
|
||||||
|
|
||||||
const queued, passes = 3, 4
|
const queued, passes = 3, 4
|
||||||
@@ -1040,7 +1015,7 @@ func TestDeliverHTTP_HalfOpenBreakerDelaysQueuedTasks(t *testing.T) {
|
|||||||
|
|
||||||
fix := buildHTTPFixture(
|
fix := buildHTTPFixture(
|
||||||
row, event, targetID,
|
row, event, targetID,
|
||||||
"test-cb-half-open", cfg, 5, 1,
|
"test-cb-half-open", targetURL, 5, 1,
|
||||||
)
|
)
|
||||||
|
|
||||||
e.ExportDeliverHTTPWithScheduler(
|
e.ExportDeliverHTTPWithScheduler(
|
||||||
@@ -1155,62 +1130,6 @@ func TestStateAndCooldown_ReadsHTTPAndSlackBreakers(t *testing.T) {
|
|||||||
assert.Zero(t, cooldown, "half-open")
|
assert.Zero(t, cooldown, "half-open")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestParseHTTPConfig_Valid(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
e := testEngine(t, 1)
|
|
||||||
|
|
||||||
cfg, err := e.ExportParseHTTPConfig(
|
|
||||||
`{"url":"https://example.com/hook",` +
|
|
||||||
`"headers":{"X-Token":"secret"}}`,
|
|
||||||
)
|
|
||||||
|
|
||||||
require.NoError(t, err)
|
|
||||||
assert.Equal(t, "https://example.com/hook", cfg.URL)
|
|
||||||
assert.Equal(t, "secret", cfg.Headers["X-Token"])
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestParseHTTPConfig_Empty(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
e := testEngine(t, 1)
|
|
||||||
|
|
||||||
_, err := e.ExportParseHTTPConfig("")
|
|
||||||
|
|
||||||
assert.Error(t, err,
|
|
||||||
"empty config should return error",
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestParseHTTPConfig_MissingURL(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
e := testEngine(t, 1)
|
|
||||||
|
|
||||||
_, err := e.ExportParseHTTPConfig(
|
|
||||||
`{"headers":{"X-Token":"secret"}}`,
|
|
||||||
)
|
|
||||||
|
|
||||||
assert.Error(t, err,
|
|
||||||
"config without URL should return error",
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestParseHTTPConfig_Undecodable(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
e := testEngine(t, 1)
|
|
||||||
|
|
||||||
_, err := e.ExportParseHTTPConfig(
|
|
||||||
`{"url":"https://example.com/hook","timeout":"soon"}`,
|
|
||||||
)
|
|
||||||
|
|
||||||
assert.Error(t, err,
|
|
||||||
"config that does not decode should return error, "+
|
|
||||||
"even when the part that did names a URL",
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestScheduleRetry_SendsToRetryChannel(
|
func TestScheduleRetry_SendsToRetryChannel(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
) {
|
) {
|
||||||
@@ -1358,7 +1277,7 @@ func TestDoHTTPRequest_ForwardsHeaders(t *testing.T) {
|
|||||||
|
|
||||||
e := testEngine(t, 1)
|
e := testEngine(t, 1)
|
||||||
|
|
||||||
cfg := &delivery.HTTPTargetConfig{
|
target := &database.Target{
|
||||||
URL: ts.URL,
|
URL: ts.URL,
|
||||||
Headers: map[string]string{
|
Headers: map[string]string{
|
||||||
"X-Target-Auth": "bearer xyz",
|
"X-Target-Auth": "bearer xyz",
|
||||||
@@ -1373,7 +1292,7 @@ func TestDoHTTPRequest_ForwardsHeaders(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
statusCode, _, _, err := e.ExportDoHTTPRequest(
|
statusCode, _, _, err := e.ExportDoHTTPRequest(
|
||||||
context.TODO(), cfg, event,
|
context.TODO(), target, event,
|
||||||
)
|
)
|
||||||
|
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
@@ -1415,7 +1334,7 @@ func TestDoHTTPRequest_CutShortResponseIsAnError(t *testing.T) {
|
|||||||
|
|
||||||
_, body, _, err := e.ExportDoHTTPRequest(
|
_, body, _, err := e.ExportDoHTTPRequest(
|
||||||
context.TODO(),
|
context.TODO(),
|
||||||
&delivery.HTTPTargetConfig{URL: ts.URL},
|
&database.Target{URL: ts.URL},
|
||||||
&database.Event{},
|
&database.Event{},
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -1467,9 +1386,9 @@ func TestApplyRequestHeaders_SendsOneContentType(t *testing.T) {
|
|||||||
})
|
})
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
|
|
||||||
cfg := &delivery.HTTPTargetConfig{}
|
var configured map[string]string
|
||||||
if tc.configured != "" {
|
if tc.configured != "" {
|
||||||
cfg.Headers = map[string]string{
|
configured = map[string]string{
|
||||||
headerContentType: tc.configured,
|
headerContentType: tc.configured,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1488,7 +1407,7 @@ func TestApplyRequestHeaders_SendsOneContentType(t *testing.T) {
|
|||||||
Headers: string(inbound),
|
Headers: string(inbound),
|
||||||
ContentType: tc.event,
|
ContentType: tc.event,
|
||||||
},
|
},
|
||||||
cfg,
|
configured,
|
||||||
"webhooker/dev",
|
"webhooker/dev",
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -1519,7 +1438,7 @@ func TestApplyRequestHeaders_UndecodableInboundForwardsNothing(
|
|||||||
&database.Event{
|
&database.Event{
|
||||||
Headers: `{"X-Custom":["value1"],"X-Broken":"not a list"}`,
|
Headers: `{"X-Custom":["value1"],"X-Broken":"not a list"}`,
|
||||||
},
|
},
|
||||||
&delivery.HTTPTargetConfig{},
|
nil,
|
||||||
"webhooker/dev",
|
"webhooker/dev",
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -1537,7 +1456,7 @@ func TestProcessDelivery_RoutesToCorrectHandler(
|
|||||||
// The database target archives for real, so the engine needs
|
// The database target archives for real, so the engine needs
|
||||||
// the target in the main database and a data directory.
|
// the target in the main database and a data directory.
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
archive := env.seedDatabaseTarget(t, "")
|
archive := env.seedDatabaseTarget(t)
|
||||||
|
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
name string
|
name string
|
||||||
@@ -1622,51 +1541,6 @@ func TestMaxInlineBodySize_Constant(t *testing.T) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestParseSlackConfig_Valid(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
e := testEngine(t, 1)
|
|
||||||
|
|
||||||
cfg, err := e.ExportParseSlackConfig(
|
|
||||||
`{"webhookUrl":"https://hooks.slack.com/services/T00/B00/xxx"}`,
|
|
||||||
)
|
|
||||||
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
assert.Equal(t,
|
|
||||||
"https://hooks.slack.com/services/T00/B00/xxx",
|
|
||||||
cfg.WebhookURL,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestParseSlackConfig_Empty(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
e := testEngine(t, 1)
|
|
||||||
|
|
||||||
_, err := e.ExportParseSlackConfig("")
|
|
||||||
|
|
||||||
assert.Error(t, err,
|
|
||||||
"empty config should return error",
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestParseSlackConfig_MissingWebhookURL(
|
|
||||||
t *testing.T,
|
|
||||||
) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
e := testEngine(t, 1)
|
|
||||||
|
|
||||||
_, err := e.ExportParseSlackConfig(
|
|
||||||
`{"other":"field"}`,
|
|
||||||
)
|
|
||||||
|
|
||||||
assert.Error(t, err,
|
|
||||||
"config without webhook_url should return error",
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestFormatSlackMessage_JSONBody(t *testing.T) {
|
func TestFormatSlackMessage_JSONBody(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -1763,11 +1637,11 @@ func TestFormatSlackMessage_LargeJSONTruncated(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// buildSlackDelivery creates a database.Delivery
|
// buildSlackDelivery creates a database.Delivery
|
||||||
// with a Slack target config for testing.
|
// to a Slack target with the given webhook URL for testing.
|
||||||
func buildSlackDelivery(
|
func buildSlackDelivery(
|
||||||
dlv database.Delivery,
|
dlv database.Delivery,
|
||||||
event database.Event,
|
event database.Event,
|
||||||
targetID, name, slackCfg string,
|
targetID, name, webhookURL string,
|
||||||
) *database.Delivery {
|
) *database.Delivery {
|
||||||
d := &database.Delivery{
|
d := &database.Delivery{
|
||||||
EventID: event.ID,
|
EventID: event.ID,
|
||||||
@@ -1777,7 +1651,7 @@ func buildSlackDelivery(
|
|||||||
Target: database.Target{
|
Target: database.Target{
|
||||||
Name: name,
|
Name: name,
|
||||||
Type: database.TargetTypeSlack,
|
Type: database.TargetTypeSlack,
|
||||||
Config: slackCfg,
|
URL: webhookURL,
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@@ -1815,11 +1689,10 @@ func TestDeliverSlack_Success(t *testing.T) {
|
|||||||
)
|
)
|
||||||
defer ts.Close()
|
defer ts.Close()
|
||||||
|
|
||||||
e, targetID, slackCfg, event, dlv :=
|
e, targetID, event, dlv := setupSlackTest(t, db)
|
||||||
setupSlackTest(t, db, ts.URL)
|
|
||||||
|
|
||||||
d := buildSlackDelivery(
|
d := buildSlackDelivery(
|
||||||
dlv, event, targetID, "test-slack", slackCfg,
|
dlv, event, targetID, "test-slack", ts.URL,
|
||||||
)
|
)
|
||||||
|
|
||||||
e.ExportDeliverSlack(context.TODO(), db, d)
|
e.ExportDeliverSlack(context.TODO(), db, d)
|
||||||
@@ -1838,9 +1711,8 @@ func TestDeliverSlack_Success(t *testing.T) {
|
|||||||
func setupSlackTest(
|
func setupSlackTest(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
db *gorm.DB,
|
db *gorm.DB,
|
||||||
serverURL string,
|
|
||||||
) (
|
) (
|
||||||
*delivery.Engine, string, string,
|
*delivery.Engine, string,
|
||||||
database.Event, database.Delivery,
|
database.Event, database.Delivery,
|
||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
@@ -1848,13 +1720,6 @@ func setupSlackTest(
|
|||||||
e := testEngine(t, 1)
|
e := testEngine(t, 1)
|
||||||
targetID := uuid.New().String()
|
targetID := uuid.New().String()
|
||||||
|
|
||||||
cfgBytes, err := json.Marshal(
|
|
||||||
delivery.SlackTargetConfig{
|
|
||||||
WebhookURL: serverURL,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
event := seedEvent(
|
event := seedEvent(
|
||||||
t, db, `{"action":"test","data":"value"}`,
|
t, db, `{"action":"test","data":"value"}`,
|
||||||
)
|
)
|
||||||
@@ -1864,7 +1729,7 @@ func setupSlackTest(
|
|||||||
database.DeliveryStatusPending,
|
database.DeliveryStatusPending,
|
||||||
)
|
)
|
||||||
|
|
||||||
return e, targetID, string(cfgBytes), event, dlv
|
return e, targetID, event, dlv
|
||||||
}
|
}
|
||||||
|
|
||||||
func assertSlackPayload(
|
func assertSlackPayload(
|
||||||
@@ -1913,13 +1778,6 @@ func TestDeliverSlack_Failure(t *testing.T) {
|
|||||||
e := testEngine(t, 1)
|
e := testEngine(t, 1)
|
||||||
targetID := uuid.New().String()
|
targetID := uuid.New().String()
|
||||||
|
|
||||||
slackCfg, err := json.Marshal(
|
|
||||||
delivery.SlackTargetConfig{
|
|
||||||
WebhookURL: ts.URL,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
event := seedEvent(t, db, `{"test":true}`)
|
event := seedEvent(t, db, `{"test":true}`)
|
||||||
|
|
||||||
dlv := seedDelivery(
|
dlv := seedDelivery(
|
||||||
@@ -1929,7 +1787,7 @@ func TestDeliverSlack_Failure(t *testing.T) {
|
|||||||
|
|
||||||
d := buildSlackDelivery(
|
d := buildSlackDelivery(
|
||||||
dlv, event, targetID,
|
dlv, event, targetID,
|
||||||
"test-slack-fail", string(slackCfg),
|
"test-slack-fail", ts.URL,
|
||||||
)
|
)
|
||||||
|
|
||||||
e.ExportDeliverSlack(context.TODO(), db, d)
|
e.ExportDeliverSlack(context.TODO(), db, d)
|
||||||
@@ -1943,7 +1801,10 @@ func TestDeliverSlack_Failure(t *testing.T) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestDeliverSlack_InvalidConfig(t *testing.T) {
|
// TestDeliverSlack_NoURL proves a Slack target without a URL, which
|
||||||
|
// the forms never store, fails its delivery rather than reporting it
|
||||||
|
// delivered.
|
||||||
|
func TestDeliverSlack_NoURL(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
db := testWebhookDB(t)
|
db := testWebhookDB(t)
|
||||||
@@ -1964,7 +1825,6 @@ func TestDeliverSlack_InvalidConfig(t *testing.T) {
|
|||||||
Target: database.Target{
|
Target: database.Target{
|
||||||
Name: "test-slack-bad",
|
Name: "test-slack-bad",
|
||||||
Type: database.TargetTypeSlack,
|
Type: database.TargetTypeSlack,
|
||||||
Config: `{"not_webhook_url":"missing"}`,
|
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@@ -2002,13 +1862,6 @@ func TestProcessDelivery_RoutesToSlack(t *testing.T) {
|
|||||||
|
|
||||||
e := testEngine(t, 1)
|
e := testEngine(t, 1)
|
||||||
|
|
||||||
slackCfg, err := json.Marshal(
|
|
||||||
delivery.SlackTargetConfig{
|
|
||||||
WebhookURL: ts.URL,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
event := seedEvent(t, db, `{"route":"slack"}`)
|
event := seedEvent(t, db, `{"route":"slack"}`)
|
||||||
|
|
||||||
dlv := seedDelivery(
|
dlv := seedDelivery(
|
||||||
@@ -2018,7 +1871,7 @@ func TestProcessDelivery_RoutesToSlack(t *testing.T) {
|
|||||||
|
|
||||||
d := buildSlackDelivery(
|
d := buildSlackDelivery(
|
||||||
dlv, event, dlv.TargetID,
|
dlv, event, dlv.TargetID,
|
||||||
"test-slack-route", string(slackCfg),
|
"test-slack-route", ts.URL,
|
||||||
)
|
)
|
||||||
|
|
||||||
task := &delivery.Task{
|
task := &delivery.Task{
|
||||||
@@ -2133,7 +1986,7 @@ func TestDeliverLog_LogsFullContent(t *testing.T) {
|
|||||||
func buildSlackRetryDelivery(
|
func buildSlackRetryDelivery(
|
||||||
dlv database.Delivery,
|
dlv database.Delivery,
|
||||||
event database.Event,
|
event database.Event,
|
||||||
targetID, cfg string,
|
targetID, webhookURL string,
|
||||||
) *database.Delivery {
|
) *database.Delivery {
|
||||||
d := &database.Delivery{
|
d := &database.Delivery{
|
||||||
EventID: event.ID,
|
EventID: event.ID,
|
||||||
@@ -2143,7 +1996,7 @@ func buildSlackRetryDelivery(
|
|||||||
Target: database.Target{
|
Target: database.Target{
|
||||||
Name: "test-slack-retry",
|
Name: "test-slack-retry",
|
||||||
Type: database.TargetTypeSlack,
|
Type: database.TargetTypeSlack,
|
||||||
Config: cfg,
|
URL: webhookURL,
|
||||||
MaxAttempts: 5,
|
MaxAttempts: 5,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@@ -2162,11 +2015,6 @@ func TestDeliverSlack_WithRetries_SchedulesRetry(
|
|||||||
e := testEngine(t, 1)
|
e := testEngine(t, 1)
|
||||||
targetID := uuid.New().String()
|
targetID := uuid.New().String()
|
||||||
|
|
||||||
slackCfg, err := json.Marshal(
|
|
||||||
delivery.SlackTargetConfig{WebhookURL: ts.URL},
|
|
||||||
)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
event := seedEvent(t, db, `{"slack":"retry"}`)
|
event := seedEvent(t, db, `{"slack":"retry"}`)
|
||||||
|
|
||||||
dlv := seedDelivery(
|
dlv := seedDelivery(
|
||||||
@@ -2175,7 +2023,7 @@ func TestDeliverSlack_WithRetries_SchedulesRetry(
|
|||||||
)
|
)
|
||||||
|
|
||||||
d := buildSlackRetryDelivery(
|
d := buildSlackRetryDelivery(
|
||||||
dlv, event, targetID, string(slackCfg),
|
dlv, event, targetID, ts.URL,
|
||||||
)
|
)
|
||||||
|
|
||||||
task := &delivery.Task{
|
task := &delivery.Task{
|
||||||
|
|||||||
@@ -69,7 +69,6 @@ func TestEventDBHoldsNoTargetRows(t *testing.T) {
|
|||||||
))
|
))
|
||||||
defer ts.Close()
|
defer ts.Close()
|
||||||
|
|
||||||
cfg := iHTTPConfig(ts.URL)
|
|
||||||
targetID := uuid.New().String()
|
targetID := uuid.New().String()
|
||||||
dbPath := s.DBMgr.DBPath(s.WebhookID)
|
dbPath := s.DBMgr.DBPath(s.WebhookID)
|
||||||
|
|
||||||
@@ -85,7 +84,7 @@ func TestEventDBHoldsNoTargetRows(t *testing.T) {
|
|||||||
)
|
)
|
||||||
task := iTask(
|
task := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"leaky-target", cfg, 5, 1, &body,
|
"leaky-target", ts.URL, 5, 1, &body,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
||||||
@@ -102,7 +101,7 @@ func TestEventDBHoldsNoTargetRows(t *testing.T) {
|
|||||||
// https://git.eeqj.de/sneak/webhooker/issues/107.
|
// https://git.eeqj.de/sneak/webhooker/issues/107.
|
||||||
iCreateTarget(
|
iCreateTarget(
|
||||||
t, s.MainDB, targetID, s.WebhookID, "leaky-target",
|
t, s.MainDB, targetID, s.WebhookID, "leaky-target",
|
||||||
database.TargetTypeHTTP, cfg, 5,
|
database.TargetTypeHTTP, ts.URL, 5,
|
||||||
)
|
)
|
||||||
|
|
||||||
rd := iSeedDelivery(
|
rd := iSeedDelivery(
|
||||||
@@ -111,7 +110,7 @@ func TestEventDBHoldsNoTargetRows(t *testing.T) {
|
|||||||
)
|
)
|
||||||
rTask := iTask(
|
rTask := iTask(
|
||||||
rd, event, s.WebhookID, targetID,
|
rd, event, s.WebhookID, targetID,
|
||||||
"leaky-target", cfg, 5, 2, &body,
|
"leaky-target", ts.URL, 5, 2, &body,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportProcessRetryTask(context.TODO(), &rTask)
|
s.Engine.ExportProcessRetryTask(context.TODO(), &rTask)
|
||||||
@@ -138,7 +137,6 @@ func TestEventDBHoldsNoTargetRowsOnFailedDelivery(t *testing.T) {
|
|||||||
))
|
))
|
||||||
defer ts.Close()
|
defer ts.Close()
|
||||||
|
|
||||||
cfg := iHTTPConfig(ts.URL)
|
|
||||||
targetID := uuid.New().String()
|
targetID := uuid.New().String()
|
||||||
|
|
||||||
event := iSeedEvent(
|
event := iSeedEvent(
|
||||||
@@ -152,7 +150,7 @@ func TestEventDBHoldsNoTargetRowsOnFailedDelivery(t *testing.T) {
|
|||||||
)
|
)
|
||||||
task := iTask(
|
task := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"leaky-target", cfg, 1, 1, &body,
|
"leaky-target", ts.URL, 1, 1, &body,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
||||||
|
|||||||
@@ -93,17 +93,6 @@ func (s *tsSink) text(t *testing.T) string {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func tsSlackConfig(t *testing.T, url string) string {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
data, err := json.Marshal(
|
|
||||||
delivery.SlackTargetConfig{WebhookURL: url},
|
|
||||||
)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
return string(data)
|
|
||||||
}
|
|
||||||
|
|
||||||
// tsSeedEvent writes an event whose CreatedAt is tsEventCreatedAt
|
// tsSeedEvent writes an event whose CreatedAt is tsEventCreatedAt
|
||||||
// rather than the write time, so an assertion on the rendered
|
// rather than the write time, so an assertion on the rendered
|
||||||
// timestamp cannot pass by accident against "roughly now".
|
// timestamp cannot pass by accident against "roughly now".
|
||||||
@@ -142,7 +131,7 @@ func tsSeedEvent(
|
|||||||
// tsSeedTarget writes the slack target row into the main database.
|
// tsSeedTarget writes the slack target row into the main database.
|
||||||
// The retry path confirms the target still exists before sending.
|
// The retry path confirms the target still exists before sending.
|
||||||
func tsSeedTarget(
|
func tsSeedTarget(
|
||||||
t *testing.T, mainDB *gorm.DB, webhookID, config string,
|
t *testing.T, mainDB *gorm.DB, webhookID, webhookURL string,
|
||||||
) database.Target {
|
) database.Target {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
@@ -150,7 +139,7 @@ func tsSeedTarget(
|
|||||||
WebhookID: webhookID,
|
WebhookID: webhookID,
|
||||||
Name: "slack-sink",
|
Name: "slack-sink",
|
||||||
Type: database.TargetTypeSlack,
|
Type: database.TargetTypeSlack,
|
||||||
Config: config,
|
URL: webhookURL,
|
||||||
Active: true,
|
Active: true,
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
}
|
}
|
||||||
@@ -176,7 +165,7 @@ func tsTask(
|
|||||||
TargetID: target.ID,
|
TargetID: target.ID,
|
||||||
TargetName: target.Name,
|
TargetName: target.Name,
|
||||||
TargetType: database.TargetTypeSlack,
|
TargetType: database.TargetTypeSlack,
|
||||||
TargetConfig: target.Config,
|
TargetURL: target.URL,
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
Method: event.Method,
|
Method: event.Method,
|
||||||
Headers: event.Headers,
|
Headers: event.Headers,
|
||||||
@@ -226,8 +215,7 @@ func (c tsCase) run(t *testing.T) (iSetup, database.Delivery, string) {
|
|||||||
s := newISetup(t)
|
s := newISetup(t)
|
||||||
sink := newTSSink(t)
|
sink := newTSSink(t)
|
||||||
|
|
||||||
cfg := tsSlackConfig(t, sink.URL)
|
target := tsSeedTarget(t, s.MainDB, s.WebhookID, sink.URL)
|
||||||
target := tsSeedTarget(t, s.MainDB, s.WebhookID, cfg)
|
|
||||||
event := tsSeedEvent(t, s.WebhookDB, s.WebhookID)
|
event := tsSeedEvent(t, s.WebhookDB, s.WebhookID)
|
||||||
|
|
||||||
d := iSeedDelivery(
|
d := iSeedDelivery(
|
||||||
@@ -338,8 +326,7 @@ func TestFormatSlackMessageOverTaskReconstructedEvent(
|
|||||||
|
|
||||||
s := newISetup(t)
|
s := newISetup(t)
|
||||||
|
|
||||||
cfg := tsSlackConfig(t, tsUndeliverableHook)
|
target := tsSeedTarget(t, s.MainDB, s.WebhookID, tsUndeliverableHook)
|
||||||
target := tsSeedTarget(t, s.MainDB, s.WebhookID, cfg)
|
|
||||||
event := tsSeedEvent(t, s.WebhookDB, s.WebhookID)
|
event := tsSeedEvent(t, s.WebhookDB, s.WebhookID)
|
||||||
|
|
||||||
d := iSeedDelivery(
|
d := iSeedDelivery(
|
||||||
@@ -410,8 +397,7 @@ func TestEventReconstructionSurvivesAReapedRow(t *testing.T) {
|
|||||||
|
|
||||||
s := newISetup(t)
|
s := newISetup(t)
|
||||||
|
|
||||||
cfg := tsSlackConfig(t, tsUndeliverableHook)
|
target := tsSeedTarget(t, s.MainDB, s.WebhookID, tsUndeliverableHook)
|
||||||
target := tsSeedTarget(t, s.MainDB, s.WebhookID, cfg)
|
|
||||||
event := tsSeedEvent(t, s.WebhookDB, s.WebhookID)
|
event := tsSeedEvent(t, s.WebhookDB, s.WebhookID)
|
||||||
|
|
||||||
d := iSeedDelivery(
|
d := iSeedDelivery(
|
||||||
|
|||||||
@@ -77,10 +77,10 @@ func ExportIsForwardableHeader(name string) bool {
|
|||||||
func ExportApplyRequestHeaders(
|
func ExportApplyRequestHeaders(
|
||||||
req *http.Request,
|
req *http.Request,
|
||||||
event *database.Event,
|
event *database.Event,
|
||||||
cfg *HTTPTargetConfig,
|
configured map[string]string,
|
||||||
userAgent string,
|
userAgent string,
|
||||||
) []string {
|
) []string {
|
||||||
return applyRequestHeaders(req, event, cfg, userAgent)
|
return applyRequestHeaders(req, event, configured, userAgent)
|
||||||
}
|
}
|
||||||
|
|
||||||
// ExportTruncate exposes truncate for testing.
|
// ExportTruncate exposes truncate for testing.
|
||||||
@@ -216,37 +216,23 @@ func (e *Engine) ExportSetSlackCircuitBreaker(
|
|||||||
e.slackTarget.circuitBreakers.Store(targetID, cb)
|
e.slackTarget.circuitBreakers.Store(targetID, cb)
|
||||||
}
|
}
|
||||||
|
|
||||||
// ExportParseHTTPConfig exposes parseHTTPConfig.
|
|
||||||
func (e *Engine) ExportParseHTTPConfig(
|
|
||||||
configJSON string,
|
|
||||||
) (*HTTPTargetConfig, error) {
|
|
||||||
return parseHTTPConfig(configJSON)
|
|
||||||
}
|
|
||||||
|
|
||||||
// ExportParseSlackConfig exposes parseSlackConfig.
|
|
||||||
func (e *Engine) ExportParseSlackConfig(
|
|
||||||
configJSON string,
|
|
||||||
) (*SlackTargetConfig, error) {
|
|
||||||
return parseSlackConfig(configJSON)
|
|
||||||
}
|
|
||||||
|
|
||||||
// ExportDoHTTPRequest exposes the http target's
|
// ExportDoHTTPRequest exposes the http target's
|
||||||
// doHTTPRequest.
|
// doHTTPRequest.
|
||||||
func (e *Engine) ExportDoHTTPRequest(
|
func (e *Engine) ExportDoHTTPRequest(
|
||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
cfg *HTTPTargetConfig,
|
target *database.Target,
|
||||||
event *database.Event,
|
event *database.Event,
|
||||||
) (int, string, int64, error) {
|
) (int, string, int64, error) {
|
||||||
return e.httpTarget.doHTTPRequest(ctx, cfg, event)
|
return e.httpTarget.doHTTPRequest(ctx, target, event)
|
||||||
}
|
}
|
||||||
|
|
||||||
// ExportClientForRequest exposes the http target's
|
// ExportClientForRequest exposes the http target's
|
||||||
// clientForRequest.
|
// clientForRequest.
|
||||||
func (e *Engine) ExportClientForRequest(
|
func (e *Engine) ExportClientForRequest(
|
||||||
cfg *HTTPTargetConfig,
|
timeoutSeconds int,
|
||||||
originScoped []string,
|
originScoped []string,
|
||||||
) *http.Client {
|
) *http.Client {
|
||||||
return e.httpTarget.clientForRequest(cfg, originScoped)
|
return e.httpTarget.clientForRequest(timeoutSeconds, originScoped)
|
||||||
}
|
}
|
||||||
|
|
||||||
// ErrExportTooManyRedirects exposes the sentinel the redirect
|
// ErrExportTooManyRedirects exposes the sentinel the redirect
|
||||||
|
|||||||
@@ -192,7 +192,7 @@ func mDeliverOK(t *testing.T, s iSetup) {
|
|||||||
body := event.Body
|
body := event.Body
|
||||||
task := iTask(
|
task := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"metrics-ok", iHTTPConfig(ts.URL), 3, 1, &body,
|
"metrics-ok", ts.URL, 3, 1, &body,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
||||||
@@ -226,18 +226,17 @@ func mExhaustRetries(t *testing.T, s iSetup) {
|
|||||||
)
|
)
|
||||||
|
|
||||||
body := event.Body
|
body := event.Body
|
||||||
cfg := iHTTPConfig(ts.URL)
|
|
||||||
|
|
||||||
// The retry below is only run if its target still exists; see
|
// The retry below is only run if its target still exists; see
|
||||||
// https://git.eeqj.de/sneak/webhooker/issues/107.
|
// https://git.eeqj.de/sneak/webhooker/issues/107.
|
||||||
iCreateTarget(
|
iCreateTarget(
|
||||||
t, s.MainDB, targetID, s.WebhookID, "metrics-fail",
|
t, s.MainDB, targetID, s.WebhookID, "metrics-fail",
|
||||||
database.TargetTypeHTTP, cfg, 2,
|
database.TargetTypeHTTP, ts.URL, 2,
|
||||||
)
|
)
|
||||||
|
|
||||||
first := iTask(
|
first := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"metrics-fail", cfg, 2, 1, &body,
|
"metrics-fail", ts.URL, 2, 1, &body,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportProcessNewTask(context.TODO(), &first)
|
s.Engine.ExportProcessNewTask(context.TODO(), &first)
|
||||||
@@ -251,7 +250,7 @@ func mExhaustRetries(t *testing.T, s iSetup) {
|
|||||||
// deterministic and off the wall clock.
|
// deterministic and off the wall clock.
|
||||||
second := iTask(
|
second := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"metrics-fail", cfg, 2, 2, &body,
|
"metrics-fail", ts.URL, 2, 2, &body,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportProcessRetryTask(
|
s.Engine.ExportProcessRetryTask(
|
||||||
@@ -289,7 +288,6 @@ func TestDeliveryMetrics_CircuitBreakerGauge(t *testing.T) {
|
|||||||
)
|
)
|
||||||
|
|
||||||
body := event.Body
|
body := event.Body
|
||||||
cfg := iHTTPConfig(ts.URL)
|
|
||||||
|
|
||||||
// An attempt budget above the failure threshold, so the breaker
|
// An attempt budget above the failure threshold, so the breaker
|
||||||
// rather than the budget is what stops the delivery.
|
// rather than the budget is what stops the delivery.
|
||||||
@@ -299,12 +297,12 @@ func TestDeliveryMetrics_CircuitBreakerGauge(t *testing.T) {
|
|||||||
// see https://git.eeqj.de/sneak/webhooker/issues/107.
|
// see https://git.eeqj.de/sneak/webhooker/issues/107.
|
||||||
iCreateTarget(
|
iCreateTarget(
|
||||||
t, s.MainDB, targetID, s.WebhookID, "metrics-trip",
|
t, s.MainDB, targetID, s.WebhookID, "metrics-trip",
|
||||||
database.TargetTypeHTTP, cfg, maxAttempts,
|
database.TargetTypeHTTP, ts.URL, maxAttempts,
|
||||||
)
|
)
|
||||||
|
|
||||||
first := iTask(
|
first := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"metrics-trip", cfg, maxAttempts, 1, &body,
|
"metrics-trip", ts.URL, maxAttempts, 1, &body,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportProcessNewTask(context.TODO(), &first)
|
s.Engine.ExportProcessNewTask(context.TODO(), &first)
|
||||||
@@ -316,7 +314,7 @@ func TestDeliveryMetrics_CircuitBreakerGauge(t *testing.T) {
|
|||||||
ExportDefaultFailureThreshold; attempt++ {
|
ExportDefaultFailureThreshold; attempt++ {
|
||||||
task := iTask(
|
task := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"metrics-trip", cfg, maxAttempts, attempt, &body,
|
"metrics-trip", ts.URL, maxAttempts, attempt, &body,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportProcessRetryTask(
|
s.Engine.ExportProcessRetryTask(
|
||||||
@@ -363,17 +361,16 @@ func TestDeliveryMetrics_BreakerBlockedIsNotAnAttempt(
|
|||||||
)
|
)
|
||||||
|
|
||||||
body := event.Body
|
body := event.Body
|
||||||
cfg := iHTTPConfig(ts.URL)
|
|
||||||
maxAttempts := delivery.ExportDefaultFailureThreshold + 5
|
maxAttempts := delivery.ExportDefaultFailureThreshold + 5
|
||||||
|
|
||||||
iCreateTarget(
|
iCreateTarget(
|
||||||
t, s.MainDB, targetID, s.WebhookID, "metrics-blocked",
|
t, s.MainDB, targetID, s.WebhookID, "metrics-blocked",
|
||||||
database.TargetTypeHTTP, cfg, maxAttempts,
|
database.TargetTypeHTTP, ts.URL, maxAttempts,
|
||||||
)
|
)
|
||||||
|
|
||||||
first := iTask(
|
first := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"metrics-blocked", cfg, maxAttempts, 1, &body,
|
"metrics-blocked", ts.URL, maxAttempts, 1, &body,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportProcessNewTask(context.TODO(), &first)
|
s.Engine.ExportProcessNewTask(context.TODO(), &first)
|
||||||
@@ -382,7 +379,7 @@ func TestDeliveryMetrics_BreakerBlockedIsNotAnAttempt(
|
|||||||
ExportDefaultFailureThreshold; attempt++ {
|
ExportDefaultFailureThreshold; attempt++ {
|
||||||
task := iTask(
|
task := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"metrics-blocked", cfg, maxAttempts, attempt, &body,
|
"metrics-blocked", ts.URL, maxAttempts, attempt, &body,
|
||||||
)
|
)
|
||||||
|
|
||||||
s.Engine.ExportProcessRetryTask(context.TODO(), &task)
|
s.Engine.ExportProcessRetryTask(context.TODO(), &task)
|
||||||
@@ -405,7 +402,7 @@ func TestDeliveryMetrics_BreakerBlockedIsNotAnAttempt(
|
|||||||
|
|
||||||
blocked := iTask(
|
blocked := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"metrics-blocked", cfg, maxAttempts,
|
"metrics-blocked", ts.URL, maxAttempts,
|
||||||
delivery.ExportDefaultFailureThreshold+1, &body,
|
delivery.ExportDefaultFailureThreshold+1, &body,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -474,7 +471,7 @@ func TestDeliveryMetrics_QueueDepthGauges(t *testing.T) {
|
|||||||
|
|
||||||
iCreateTarget(t, s.MainDB, targetID, s.WebhookID,
|
iCreateTarget(t, s.MainDB, targetID, s.WebhookID,
|
||||||
"queue-depth-target", database.TargetTypeHTTP,
|
"queue-depth-target", database.TargetTypeHTTP,
|
||||||
iHTTPConfig("https://example.com/hook"), 3,
|
"https://example.com/hook", 3,
|
||||||
)
|
)
|
||||||
|
|
||||||
event := iSeedEvent(
|
event := iSeedEvent(
|
||||||
|
|||||||
@@ -102,7 +102,7 @@ func TestQueueDepthSample_LogsNoBoundValue(t *testing.T) {
|
|||||||
|
|
||||||
iCreateTarget(t, mainDB.DB(), targetID, webhookID,
|
iCreateTarget(t, mainDB.DB(), targetID, webhookID,
|
||||||
"queue-depth-gormlog-target", database.TargetTypeHTTP,
|
"queue-depth-gormlog-target", database.TargetTypeHTTP,
|
||||||
iHTTPConfig("https://example.com/hook"), 3,
|
"https://example.com/hook", 3,
|
||||||
)
|
)
|
||||||
|
|
||||||
event := iSeedEvent(
|
event := iSeedEvent(
|
||||||
|
|||||||
@@ -357,7 +357,7 @@ func TestFailedResultWriteLeavesDeliveryRecoverable(
|
|||||||
Target: database.Target{
|
Target: database.Target{
|
||||||
Name: "unwritable",
|
Name: "unwritable",
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: iHTTPConfig(ts.URL),
|
URL: ts.URL,
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@@ -448,7 +448,7 @@ func TestFailedResultWriteWithRetriesLeavesDeliveryRecoverable(
|
|||||||
Target: database.Target{
|
Target: database.Target{
|
||||||
Name: "unwritable",
|
Name: "unwritable",
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: iHTTPConfig(ts.URL),
|
URL: ts.URL,
|
||||||
MaxAttempts: 3,
|
MaxAttempts: 3,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -56,7 +56,7 @@ func recoverySetup(t *testing.T) (iSetup, string) {
|
|||||||
iCreateTarget(
|
iCreateTarget(
|
||||||
t, s.MainDB, targetID, s.WebhookID, "recovery",
|
t, s.MainDB, targetID, s.WebhookID, "recovery",
|
||||||
database.TargetTypeHTTP,
|
database.TargetTypeHTTP,
|
||||||
iHTTPConfig("http://example.com/hook"), 20,
|
"http://example.com/hook", 20,
|
||||||
)
|
)
|
||||||
|
|
||||||
return s, targetID
|
return s, targetID
|
||||||
@@ -205,6 +205,55 @@ func TestRecovery_PendingOlderThanTheAgeBound(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestRecovery_TaskCarriesEveryTargetSetting: the task the recovery
|
||||||
|
// routine queues carries every one of the target's settings, all six
|
||||||
|
// set here.
|
||||||
|
// They are copied into the task one field at a time, and a field left
|
||||||
|
// out is sent empty without any error.
|
||||||
|
func TestRecovery_TaskCarriesEveryTargetSetting(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for _, run := range recoveryRuns() {
|
||||||
|
t.Run(run.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s := newISetup(t)
|
||||||
|
iCreateWebhook(t, s.MainDB, s.WebhookID, "recovery")
|
||||||
|
|
||||||
|
target := database.Target{
|
||||||
|
WebhookID: s.WebhookID,
|
||||||
|
Name: "recovery",
|
||||||
|
Type: database.TargetTypeHTTP,
|
||||||
|
Active: true,
|
||||||
|
URL: "http://example.com/hook",
|
||||||
|
Headers: map[string]string{headerAuthorization: "Bearer t"},
|
||||||
|
TimeoutSeconds: 7,
|
||||||
|
ForwardQuery: true,
|
||||||
|
ArchiveExpiry: time.Hour,
|
||||||
|
ArchiveRotation: "daily",
|
||||||
|
MaxAttempts: 1,
|
||||||
|
}
|
||||||
|
require.NoError(t, s.MainDB.Create(&target).Error)
|
||||||
|
|
||||||
|
id := seedRecoveryDelivery(
|
||||||
|
t, s, target.ID, database.DeliveryStatusPending, 0, false,
|
||||||
|
)
|
||||||
|
rAgePending(t, s.WebhookDB, id)
|
||||||
|
|
||||||
|
s.Engine.ExportRecoverDeliveries(context.Background(), run.atStart)
|
||||||
|
|
||||||
|
task := requireQueued(t, s.Engine.ExportDeliveryCh())
|
||||||
|
assert.Equal(t, target.ID, task.TargetID)
|
||||||
|
assert.Equal(t, target.URL, task.TargetURL)
|
||||||
|
assert.Equal(t, target.Headers, task.TargetHeaders)
|
||||||
|
assert.Equal(t, target.TimeoutSeconds, task.TimeoutSeconds)
|
||||||
|
assert.Equal(t, target.ForwardQuery, task.ForwardQuery)
|
||||||
|
assert.Equal(t, target.ArchiveExpiry, task.ArchiveExpiry)
|
||||||
|
assert.Equal(t, target.ArchiveRotation, task.ArchiveRotation)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestRecovery_PendingYoungerThanTheAgeBound: a pending delivery touched
|
// TestRecovery_PendingYoungerThanTheAgeBound: a pending delivery touched
|
||||||
// a moment ago, as one a stop cut off is, is sent at start, which is how
|
// a moment ago, as one a stop cut off is, is sent at start, which is how
|
||||||
// it reaches its target on the next start
|
// it reaches its target on the next start
|
||||||
|
|||||||
@@ -85,19 +85,14 @@ func deliverWithProbeHeaders(
|
|||||||
database.DeliveryStatusPending,
|
database.DeliveryStatusPending,
|
||||||
)
|
)
|
||||||
|
|
||||||
cfg, err := json.Marshal(delivery.HTTPTargetConfig{
|
|
||||||
URL: targetURL,
|
|
||||||
Headers: map[string]string{
|
|
||||||
probeHeaderName: probeHeaderValue,
|
|
||||||
},
|
|
||||||
})
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
body := event.Body
|
body := event.Body
|
||||||
task := iTask(
|
task := iTask(
|
||||||
d, event, s.WebhookID, targetID,
|
d, event, s.WebhookID, targetID,
|
||||||
"redirect-target", string(cfg), 1, 1, &body,
|
"redirect-target", targetURL, 1, 1, &body,
|
||||||
)
|
)
|
||||||
|
task.TargetHeaders = map[string]string{
|
||||||
|
probeHeaderName: probeHeaderValue,
|
||||||
|
}
|
||||||
|
|
||||||
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
||||||
|
|
||||||
@@ -309,8 +304,7 @@ func TestRedirectPolicy_StopsAtHopCap(t *testing.T) {
|
|||||||
)
|
)
|
||||||
|
|
||||||
client := engine.ExportClientForRequest(
|
client := engine.ExportClientForRequest(
|
||||||
&delivery.HTTPTargetConfig{URL: srv.URL},
|
0, []string{probeHeaderName},
|
||||||
[]string{probeHeaderName},
|
|
||||||
)
|
)
|
||||||
require.NotNil(t, client.CheckRedirect)
|
require.NotNil(t, client.CheckRedirect)
|
||||||
|
|
||||||
@@ -350,7 +344,7 @@ func TestApplyRequestHeaders_ReportsOriginScopedNames(t *testing.T) {
|
|||||||
inbound, err := json.Marshal(map[string][]string{
|
inbound, err := json.Marshal(map[string][]string{
|
||||||
inboundHeaderName: {inboundHeaderValue},
|
inboundHeaderName: {inboundHeaderValue},
|
||||||
"Content-Type": {testContentType},
|
"Content-Type": {testContentType},
|
||||||
"User-Agent": {"curl/8.7.1"},
|
headerUserAgent: {"curl/8.7.1"},
|
||||||
"Host": {"inbound.example.com"},
|
"Host": {"inbound.example.com"},
|
||||||
})
|
})
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
@@ -369,11 +363,9 @@ func TestApplyRequestHeaders_ReportsOriginScopedNames(t *testing.T) {
|
|||||||
Headers: string(inbound),
|
Headers: string(inbound),
|
||||||
ContentType: testContentType,
|
ContentType: testContentType,
|
||||||
},
|
},
|
||||||
&delivery.HTTPTargetConfig{
|
map[string]string{
|
||||||
Headers: map[string]string{
|
probeHeaderName: probeHeaderValue,
|
||||||
probeHeaderName: probeHeaderValue,
|
"Content-Type": testContentType,
|
||||||
"Content-Type": testContentType,
|
|
||||||
},
|
|
||||||
},
|
},
|
||||||
"webhooker/dev",
|
"webhooker/dev",
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -1,137 +0,0 @@
|
|||||||
package delivery
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"errors"
|
|
||||||
"fmt"
|
|
||||||
|
|
||||||
"sneak.berlin/go/webhooker/internal/archive"
|
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
|
||||||
)
|
|
||||||
|
|
||||||
// errUnknownTargetTypeForEdit is returned when a stored target has a
|
|
||||||
// type the edit form has no field set for.
|
|
||||||
var errUnknownTargetTypeForEdit = errors.New(
|
|
||||||
"unknown target type",
|
|
||||||
)
|
|
||||||
|
|
||||||
// TargetConfigForm is the UNMASKED projection of a target's stored
|
|
||||||
// configuration, for pre-filling the target edit form.
|
|
||||||
//
|
|
||||||
// It is the deliberate exception to the rule TargetView enforces
|
|
||||||
// everywhere else: TargetView exists so that no template can render
|
|
||||||
// a target's stored blob, because a destination URL's path segments
|
|
||||||
// and a header value are both routinely the credential. An operator
|
|
||||||
// cannot correct a value they cannot see, so the edit form — and
|
|
||||||
// only the edit form — is shown the full value.
|
|
||||||
//
|
|
||||||
// Everything that keeps that exception narrow lives at the call
|
|
||||||
// site: the route is behind RequireAuth and the webhook's ownership
|
|
||||||
// check, and its group sets NoCache so the rendered secret is not
|
|
||||||
// written to a shared cache. Do not reach for this type from any
|
|
||||||
// other page.
|
|
||||||
type TargetConfigForm struct {
|
|
||||||
// URL is the destination for an HTTP target and the webhook
|
|
||||||
// URL for a Slack target.
|
|
||||||
URL string
|
|
||||||
// Headers is the HTTP target's configured headers in the
|
|
||||||
// textarea representation, one "Name: value" per line.
|
|
||||||
Headers string
|
|
||||||
// Timeout is the HTTP target's per-request timeout in seconds,
|
|
||||||
// empty when unset.
|
|
||||||
Timeout string
|
|
||||||
// ForwardQuery is the HTTP target's setting that passes each
|
|
||||||
// event's query string on to it.
|
|
||||||
ForwardQuery bool
|
|
||||||
// Expiry is the database (archive) target's row expiry.
|
|
||||||
Expiry string
|
|
||||||
// Rotation is the database (archive) target's rotation.
|
|
||||||
Rotation string
|
|
||||||
}
|
|
||||||
|
|
||||||
// NewTargetConfigForm parses a target's stored configuration into
|
|
||||||
// the edit form's fields.
|
|
||||||
//
|
|
||||||
// A configuration that does not parse is an error rather than a
|
|
||||||
// zero-valued form that silently looks like a target with no
|
|
||||||
// settings. The caller shows the operator that the stored value
|
|
||||||
// could not be read, so that saving the form is understood as
|
|
||||||
// replacing it rather than preserving it.
|
|
||||||
func NewTargetConfigForm(
|
|
||||||
t *database.Target,
|
|
||||||
) (TargetConfigForm, error) {
|
|
||||||
switch t.Type {
|
|
||||||
case database.TargetTypeHTTP:
|
|
||||||
cfg, err := parseHTTPConfig(t.Config)
|
|
||||||
if err != nil {
|
|
||||||
return TargetConfigForm{}, err
|
|
||||||
}
|
|
||||||
|
|
||||||
return TargetConfigForm{
|
|
||||||
URL: cfg.URL,
|
|
||||||
Headers: FormatTargetHeaders(cfg.Headers),
|
|
||||||
Timeout: FormatTargetTimeout(cfg.Timeout),
|
|
||||||
ForwardQuery: cfg.ForwardQuery,
|
|
||||||
}, nil
|
|
||||||
case database.TargetTypeSlack:
|
|
||||||
cfg, err := parseSlackConfig(t.Config)
|
|
||||||
if err != nil {
|
|
||||||
return TargetConfigForm{}, err
|
|
||||||
}
|
|
||||||
|
|
||||||
return TargetConfigForm{URL: cfg.WebhookURL}, nil
|
|
||||||
case database.TargetTypeDatabase:
|
|
||||||
return databaseConfigForm(t.Config)
|
|
||||||
case database.TargetTypeLog:
|
|
||||||
// The log target takes no configuration.
|
|
||||||
return TargetConfigForm{}, nil
|
|
||||||
default:
|
|
||||||
return TargetConfigForm{}, fmt.Errorf(
|
|
||||||
"%w: %q", errUnknownTargetTypeForEdit, t.Type,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// databaseConfigForm parses an archive target's optional expiry and
|
|
||||||
// rotation. An absent, empty or never expiry yields an empty expiry,
|
|
||||||
// on which the edit form starts at never; saving it unchanged stores
|
|
||||||
// never, which means the same as an empty expiry. An absent rotation
|
|
||||||
// is empty too, and the form starts at none. An expiry that is set
|
|
||||||
// but not a valid duration, or a rotation that is not one of the
|
|
||||||
// four, is an error, not a blank field.
|
|
||||||
func databaseConfigForm(
|
|
||||||
configJSON string,
|
|
||||||
) (TargetConfigForm, error) {
|
|
||||||
if configJSON == "" {
|
|
||||||
return TargetConfigForm{}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
var cfg archive.DatabaseTargetConfig
|
|
||||||
|
|
||||||
err := json.Unmarshal([]byte(configJSON), &cfg)
|
|
||||||
if err != nil {
|
|
||||||
return TargetConfigForm{}, fmt.Errorf(
|
|
||||||
"parsing config JSON: %w", err,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
err = archive.ValidateRotation(cfg.Rotation)
|
|
||||||
if err != nil {
|
|
||||||
return TargetConfigForm{}, err
|
|
||||||
}
|
|
||||||
|
|
||||||
form := TargetConfigForm{Rotation: cfg.Rotation}
|
|
||||||
|
|
||||||
if cfg.Expiry == "" || cfg.Expiry == archive.ExpiryNever {
|
|
||||||
return form, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
err = archive.ValidateExpiry(cfg.Expiry)
|
|
||||||
if err != nil {
|
|
||||||
return TargetConfigForm{}, err
|
|
||||||
}
|
|
||||||
|
|
||||||
form.Expiry = cfg.Expiry
|
|
||||||
|
|
||||||
return form, nil
|
|
||||||
}
|
|
||||||
@@ -9,16 +9,17 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
)
|
)
|
||||||
|
|
||||||
// configUnavailable is what a target's configuration renders
|
// configUnavailable is what a target's settings render as when
|
||||||
// as when it is absent, of an unknown type, or does not
|
// the target is of an unknown type, and what its URL renders as
|
||||||
// parse. The stored blob is never shown as a fallback: it can
|
// when it does not parse into a scheme and host. The stored URL is
|
||||||
// hold a credential (a Slack incoming webhook URL is a bearer
|
// never shown as a fallback: it can be a credential (a Slack
|
||||||
// token) and a UI that prints it leaks that credential into
|
// incoming webhook URL is a bearer token) and a UI that prints it
|
||||||
// browser history, screenshots and screen shares.
|
// leaks that credential into browser history, screenshots and
|
||||||
|
// screen shares.
|
||||||
const configUnavailable = "(unavailable)"
|
const configUnavailable = "(unavailable)"
|
||||||
|
|
||||||
// ConfigField is one labelled, display-safe value derived
|
// ConfigField is one labelled, display-safe value derived
|
||||||
// from a target's stored configuration.
|
// from a target's settings.
|
||||||
type ConfigField struct {
|
type ConfigField struct {
|
||||||
Label string
|
Label string
|
||||||
Value string
|
Value string
|
||||||
@@ -32,9 +33,9 @@ type ConfigField struct {
|
|||||||
const deletedNameSuffix = " (deleted)"
|
const deletedNameSuffix = " (deleted)"
|
||||||
|
|
||||||
// TargetView is the display-safe projection of a target for
|
// TargetView is the display-safe projection of a target for
|
||||||
// the UI. It deliberately has no raw configuration field, so
|
// the UI. It deliberately has no URL or header field, so no
|
||||||
// no template — present or future — can render the stored
|
// template — present or future — can render the stored
|
||||||
// blob.
|
// credential.
|
||||||
type TargetView struct {
|
type TargetView struct {
|
||||||
ID string
|
ID string
|
||||||
Name string
|
Name string
|
||||||
@@ -61,8 +62,7 @@ func (v TargetView) DisplayName() string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// NewTargetViews projects targets for rendering, replacing
|
// NewTargetViews projects targets for rendering, replacing
|
||||||
// each stored configuration blob with named, display-safe
|
// each target's settings with named, display-safe fields.
|
||||||
// fields.
|
|
||||||
//
|
//
|
||||||
// A soft-deleted row projects exactly as a live one does, minus
|
// A soft-deleted row projects exactly as a live one does, minus
|
||||||
// the deleted marker on its name: masking is a property of the
|
// the deleted marker on its name: masking is a property of the
|
||||||
@@ -91,8 +91,8 @@ func NewTargetViews(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// targetConfigFields returns the display-safe fields for a
|
// targetConfigFields returns the display-safe fields for a
|
||||||
// target's configuration. Anything it cannot parse becomes
|
// target's settings. A target of an unknown type gets the
|
||||||
// the neutral placeholder.
|
// neutral placeholder.
|
||||||
func targetConfigFields(
|
func targetConfigFields(
|
||||||
t *database.Target,
|
t *database.Target,
|
||||||
) []ConfigField {
|
) []ConfigField {
|
||||||
@@ -102,36 +102,25 @@ func targetConfigFields(
|
|||||||
case database.TargetTypeHTTP:
|
case database.TargetTypeHTTP:
|
||||||
return httpConfigFields(t)
|
return httpConfigFields(t)
|
||||||
case database.TargetTypeDatabase:
|
case database.TargetTypeDatabase:
|
||||||
return databaseConfigFields(t.Config)
|
return databaseConfigFields(t)
|
||||||
case database.TargetTypeLog:
|
case database.TargetTypeLog:
|
||||||
// The log target takes no configuration.
|
// The log target takes no settings.
|
||||||
return nil
|
return nil
|
||||||
default:
|
default:
|
||||||
return unavailableConfigFields()
|
return []ConfigField{{
|
||||||
|
Label: "Configuration",
|
||||||
|
Value: configUnavailable,
|
||||||
|
}}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// unavailableConfigFields is the neutral placeholder shown
|
|
||||||
// for a configuration that could not be presented.
|
|
||||||
func unavailableConfigFields() []ConfigField {
|
|
||||||
return []ConfigField{{
|
|
||||||
Label: "Configuration",
|
|
||||||
Value: configUnavailable,
|
|
||||||
}}
|
|
||||||
}
|
|
||||||
|
|
||||||
// slackConfigFields describes a Slack target: its masked
|
// slackConfigFields describes a Slack target: its masked
|
||||||
// webhook URL and its number of delivery attempts. Only the
|
// webhook URL and its number of delivery attempts. Only the
|
||||||
// masked URL is shown; the full URL is the credential.
|
// masked URL is shown; the full URL is the credential.
|
||||||
func slackConfigFields(t *database.Target) []ConfigField {
|
func slackConfigFields(t *database.Target) []ConfigField {
|
||||||
cfg, err := parseSlackConfig(t.Config)
|
|
||||||
if err != nil {
|
|
||||||
return unavailableConfigFields()
|
|
||||||
}
|
|
||||||
|
|
||||||
return []ConfigField{{
|
return []ConfigField{{
|
||||||
Label: "Webhook URL",
|
Label: "Webhook URL",
|
||||||
Value: cfg.MaskedWebhookURL(),
|
Value: MaskURL(t.URL),
|
||||||
}, maxAttemptsField(t)}
|
}, maxAttemptsField(t)}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -146,33 +135,28 @@ func slackConfigFields(t *database.Target) []ConfigField {
|
|||||||
// whose path segments are the credential, and the field takes
|
// whose path segments are the credential, and the field takes
|
||||||
// an arbitrary URL, so no segment can be assumed non-secret.
|
// an arbitrary URL, so no segment can be assumed non-secret.
|
||||||
func httpConfigFields(t *database.Target) []ConfigField {
|
func httpConfigFields(t *database.Target) []ConfigField {
|
||||||
cfg, err := parseHTTPConfig(t.Config)
|
|
||||||
if err != nil {
|
|
||||||
return unavailableConfigFields()
|
|
||||||
}
|
|
||||||
|
|
||||||
fields := []ConfigField{{
|
fields := []ConfigField{{
|
||||||
Label: "Destination URL",
|
Label: "Destination URL",
|
||||||
Value: MaskURL(cfg.URL),
|
Value: MaskURL(t.URL),
|
||||||
}}
|
}}
|
||||||
|
|
||||||
if cfg.Timeout > 0 {
|
if t.TimeoutSeconds > 0 {
|
||||||
fields = append(fields, ConfigField{
|
fields = append(fields, ConfigField{
|
||||||
Label: "Timeout",
|
Label: "Timeout",
|
||||||
Value: strconv.Itoa(cfg.Timeout) + "s",
|
Value: strconv.Itoa(t.TimeoutSeconds) + "s",
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(cfg.Headers) > 0 {
|
if len(t.Headers) > 0 {
|
||||||
fields = append(fields, ConfigField{
|
fields = append(fields, ConfigField{
|
||||||
Label: "Headers",
|
Label: "Headers",
|
||||||
Value: fmt.Sprintf(
|
Value: fmt.Sprintf(
|
||||||
"%d configured", len(cfg.Headers),
|
"%d configured", len(t.Headers),
|
||||||
),
|
),
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
if cfg.ForwardQuery {
|
if t.ForwardQuery {
|
||||||
fields = append(fields, ConfigField{
|
fields = append(fields, ConfigField{
|
||||||
Label: "Query string",
|
Label: "Query string",
|
||||||
Value: "passed on to this target",
|
Value: "passed on to this target",
|
||||||
@@ -185,8 +169,7 @@ func httpConfigFields(t *database.Target) []ConfigField {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// maxAttemptsField describes a target's number of delivery
|
// maxAttemptsField describes a target's number of delivery
|
||||||
// attempts, which lives on the target row rather than in its
|
// attempts.
|
||||||
// configuration blob.
|
|
||||||
func maxAttemptsField(t *database.Target) ConfigField {
|
func maxAttemptsField(t *database.Target) ConfigField {
|
||||||
return ConfigField{
|
return ConfigField{
|
||||||
Label: "Delivery attempts",
|
Label: "Delivery attempts",
|
||||||
@@ -196,24 +179,11 @@ func maxAttemptsField(t *database.Target) ConfigField {
|
|||||||
|
|
||||||
// databaseConfigFields describes an archive target by its
|
// databaseConfigFields describes an archive target by its
|
||||||
// expiry in plain units, such as "30 days", or "never" when
|
// expiry in plain units, such as "30 days", or "never" when
|
||||||
// the archive is kept forever, and by its rotation. An expiry
|
// the archive is kept forever, and by its rotation.
|
||||||
// that is set but not a valid duration, or a rotation that is
|
func databaseConfigFields(t *database.Target) []ConfigField {
|
||||||
// not one of the four, is reported as unavailable rather than
|
|
||||||
// echoed back.
|
|
||||||
func databaseConfigFields(configJSON string) []ConfigField {
|
|
||||||
expiry, err := archive.ParseExpiry(configJSON)
|
|
||||||
if err != nil {
|
|
||||||
return unavailableConfigFields()
|
|
||||||
}
|
|
||||||
|
|
||||||
rotation, err := archive.ParseRotation(configJSON)
|
|
||||||
if err != nil {
|
|
||||||
return unavailableConfigFields()
|
|
||||||
}
|
|
||||||
|
|
||||||
value := archive.ExpiryNever
|
value := archive.ExpiryNever
|
||||||
if expiry > 0 {
|
if t.ArchiveExpiry > 0 {
|
||||||
value = plainDuration(expiry)
|
value = plainDuration(t.ArchiveExpiry)
|
||||||
}
|
}
|
||||||
|
|
||||||
return []ConfigField{{
|
return []ConfigField{{
|
||||||
@@ -221,7 +191,7 @@ func databaseConfigFields(configJSON string) []ConfigField {
|
|||||||
Value: value,
|
Value: value,
|
||||||
}, {
|
}, {
|
||||||
Label: "Archive rotation",
|
Label: "Archive rotation",
|
||||||
Value: rotation,
|
Value: t.ArchiveRotation,
|
||||||
}}
|
}}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -257,14 +227,3 @@ func plainDuration(d time.Duration) string {
|
|||||||
|
|
||||||
return d.String()
|
return d.String()
|
||||||
}
|
}
|
||||||
|
|
||||||
// MaskedWebhookURL returns the Slack webhook URL reduced to
|
|
||||||
// its scheme and host, with the path, query and any userinfo
|
|
||||||
// elided. The path segments are the credential, so none of
|
|
||||||
// them is shown: the field accepts an arbitrary URL, so no
|
|
||||||
// segment can be assumed non-secret. A URL that does not
|
|
||||||
// parse into a scheme and host yields the neutral
|
|
||||||
// placeholder, never the raw string.
|
|
||||||
func (c *SlackTargetConfig) MaskedWebhookURL() string {
|
|
||||||
return MaskURL(c.WebhookURL)
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -35,18 +35,17 @@ const (
|
|||||||
viewMaxAttempts = "Delivery attempts"
|
viewMaxAttempts = "Delivery attempts"
|
||||||
)
|
)
|
||||||
|
|
||||||
// The archive rotations a database target's settings can name, and
|
// The archive rotations a database target's settings can name.
|
||||||
// the config of one whose rotation is not one of the four.
|
|
||||||
const (
|
const (
|
||||||
rotationNone = "none"
|
rotationNone = "none"
|
||||||
rotationMonthly = "monthly"
|
rotationMonthly = "monthly"
|
||||||
rotationDaily = "daily"
|
rotationDaily = "daily"
|
||||||
rotationHourly = "hourly"
|
rotationHourly = "hourly"
|
||||||
|
|
||||||
weeklyConfig = `{"rotation":"weekly"}`
|
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestMaskedWebhookURL(t *testing.T) {
|
// TestNewTargetViews_SlackMasksWebhookURL shows what a Slack target's
|
||||||
|
// URL renders as on the target list.
|
||||||
|
func TestNewTargetViews_SlackMasksWebhookURL(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
tests := map[string]struct {
|
tests := map[string]struct {
|
||||||
@@ -90,28 +89,24 @@ func TestMaskedWebhookURL(t *testing.T) {
|
|||||||
t.Run(name, func(t *testing.T) {
|
t.Run(name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
cfg := &delivery.SlackTargetConfig{
|
target := slackTarget()
|
||||||
WebhookURL: tc.url,
|
target.URL = tc.url
|
||||||
}
|
|
||||||
|
|
||||||
assert.Equal(
|
assert.Equal(
|
||||||
t, tc.want, cfg.MaskedWebhookURL(),
|
t, tc.want,
|
||||||
|
fieldMap(viewFor(t, target).Config)["Webhook URL"],
|
||||||
)
|
)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestMaskedWebhookURL_NeverLeaksPath is the direct
|
// TestNewTargetViews_SlackNeverLeaksPath is the direct
|
||||||
// expression of the rule: whatever the input, the masked
|
// expression of the rule: whatever the input, the masked
|
||||||
// value never contains a path segment of it.
|
// value never contains a path segment of it.
|
||||||
func TestMaskedWebhookURL_NeverLeaksPath(t *testing.T) {
|
func TestNewTargetViews_SlackNeverLeaksPath(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
cfg := &delivery.SlackTargetConfig{
|
masked := fieldMap(viewFor(t, slackTarget()).Config)["Webhook URL"]
|
||||||
WebhookURL: slackWebhookURL,
|
|
||||||
}
|
|
||||||
|
|
||||||
masked := cfg.MaskedWebhookURL()
|
|
||||||
|
|
||||||
assert.NotContains(t, masked, "T00000000")
|
assert.NotContains(t, masked, "T00000000")
|
||||||
assert.NotContains(t, masked, "B00000000")
|
assert.NotContains(t, masked, "B00000000")
|
||||||
@@ -177,11 +172,10 @@ func TestNewTargetViews_DeletedTarget(t *testing.T) {
|
|||||||
// share.
|
// share.
|
||||||
func slackTarget() database.Target {
|
func slackTarget() database.Target {
|
||||||
return database.Target{
|
return database.Target{
|
||||||
Name: slackTargetName,
|
Name: slackTargetName,
|
||||||
Type: database.TargetTypeSlack,
|
Type: database.TargetTypeSlack,
|
||||||
Active: true,
|
Active: true,
|
||||||
Config: `{"webhookUrl":"` +
|
URL: slackWebhookURL,
|
||||||
slackWebhookURL + `"}`,
|
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -233,12 +227,12 @@ func TestNewTargetViews_HTTP(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
view := viewFor(t, database.Target{
|
view := viewFor(t, database.Target{
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: `{"url":"` + viewExampleHook + `",` +
|
URL: viewExampleHook,
|
||||||
`"timeout":30,` +
|
TimeoutSeconds: 30,
|
||||||
`"headers":{"Authorization":"Bearer sekrit"},` +
|
Headers: map[string]string{headerAuthorization: "Bearer sekrit"},
|
||||||
`"forwardQuery":true}`,
|
ForwardQuery: true,
|
||||||
MaxAttempts: 5,
|
MaxAttempts: 5,
|
||||||
})
|
})
|
||||||
|
|
||||||
fields := fieldMap(view.Config)
|
fields := fieldMap(view.Config)
|
||||||
@@ -268,7 +262,7 @@ func TestNewTargetViews_HTTPSingleAttempt(t *testing.T) {
|
|||||||
|
|
||||||
view := viewFor(t, database.Target{
|
view := viewFor(t, database.Target{
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: `{"url":"` + viewExampleHook + `"}`,
|
URL: viewExampleHook,
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -291,7 +285,7 @@ func TestNewTargetViews_HTTPMasksDestinationURL(t *testing.T) {
|
|||||||
|
|
||||||
view := viewFor(t, database.Target{
|
view := viewFor(t, database.Target{
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: `{"url":"` + slackWebhookURL + `"}`,
|
URL: slackWebhookURL,
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -315,25 +309,20 @@ func TestNewTargetViews_Database(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
tests := map[string]struct {
|
tests := map[string]struct {
|
||||||
config string
|
expiry time.Duration
|
||||||
want string
|
want string
|
||||||
}{
|
}{
|
||||||
"empty config": {config: "", want: viewExpiryNever},
|
"never": {expiry: 0, want: viewExpiryNever},
|
||||||
"empty expiry": {config: `{}`, want: viewExpiryNever},
|
"1h": {expiry: time.Hour, want: "1 hour"},
|
||||||
"never literal": {
|
"12h": {expiry: 12 * time.Hour, want: "12 hours"},
|
||||||
config: `{"expiry":"` + viewExpiryNever + `"}`,
|
"24h": {expiry: 24 * time.Hour, want: "1 day"},
|
||||||
want: viewExpiryNever,
|
"720h": {expiry: 720 * time.Hour, want: "30 days"},
|
||||||
},
|
"2160h": {expiry: 2160 * time.Hour, want: "90 days"},
|
||||||
"1h": {config: `{"expiry":"1h"}`, want: "1 hour"},
|
"8760h": {expiry: 8760 * time.Hour, want: "365 days"},
|
||||||
"12h": {config: `{"expiry":"12h"}`, want: "12 hours"},
|
"36h": {expiry: 36 * time.Hour, want: "36 hours"},
|
||||||
"24h": {config: `{"expiry":"24h"}`, want: "1 day"},
|
"1h30m": {expiry: 90 * time.Minute, want: "90 minutes"},
|
||||||
"720h": {config: `{"expiry":"720h"}`, want: "30 days"},
|
"45s": {expiry: 45 * time.Second, want: "45 seconds"},
|
||||||
"2160h": {config: `{"expiry":"2160h"}`, want: "90 days"},
|
"1.5s": {expiry: 1500 * time.Millisecond, want: "1.5s"},
|
||||||
"8760h": {config: `{"expiry":"8760h"}`, want: "365 days"},
|
|
||||||
"36h": {config: `{"expiry":"36h"}`, want: "36 hours"},
|
|
||||||
"1h30m": {config: `{"expiry":"1h30m"}`, want: "90 minutes"},
|
|
||||||
"45s": {config: `{"expiry":"45s"}`, want: "45 seconds"},
|
|
||||||
"1.5s": {config: `{"expiry":"1.5s"}`, want: "1.5s"},
|
|
||||||
}
|
}
|
||||||
|
|
||||||
for name, tc := range tests {
|
for name, tc := range tests {
|
||||||
@@ -341,8 +330,9 @@ func TestNewTargetViews_Database(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
view := viewFor(t, database.Target{
|
view := viewFor(t, database.Target{
|
||||||
Type: database.TargetTypeDatabase,
|
Type: database.TargetTypeDatabase,
|
||||||
Config: tc.config,
|
ArchiveExpiry: tc.expiry,
|
||||||
|
ArchiveRotation: rotationNone,
|
||||||
})
|
})
|
||||||
|
|
||||||
assert.Equal(
|
assert.Equal(
|
||||||
@@ -358,33 +348,23 @@ func TestNewTargetViews_Database(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// TestNewTargetViews_DatabaseRotation proves the target list shows a
|
// TestNewTargetViews_DatabaseRotation proves the target list shows a
|
||||||
// database target's rotation, none when it has none stored.
|
// database target's rotation.
|
||||||
func TestNewTargetViews_DatabaseRotation(t *testing.T) {
|
func TestNewTargetViews_DatabaseRotation(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
// Each stored config, and the rotation the list shows for it.
|
|
||||||
tests := map[string]string{
|
|
||||||
"": rotationNone,
|
|
||||||
`{"rotation":""}`: rotationNone,
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, rotation := range []string{
|
for _, rotation := range []string{
|
||||||
rotationNone, rotationMonthly, rotationDaily, rotationHourly,
|
rotationNone, rotationMonthly, rotationDaily, rotationHourly,
|
||||||
} {
|
} {
|
||||||
tests[`{"rotation":"`+rotation+`"}`] = rotation
|
t.Run(rotation, func(t *testing.T) {
|
||||||
}
|
|
||||||
|
|
||||||
for config, want := range tests {
|
|
||||||
t.Run(config, func(t *testing.T) {
|
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
view := viewFor(t, database.Target{
|
view := viewFor(t, database.Target{
|
||||||
Type: database.TargetTypeDatabase,
|
Type: database.TargetTypeDatabase,
|
||||||
Config: config,
|
ArchiveRotation: rotation,
|
||||||
})
|
})
|
||||||
|
|
||||||
assert.Equal(
|
assert.Equal(
|
||||||
t, want, fieldMap(view.Config)["Archive rotation"],
|
t, rotation, fieldMap(view.Config)["Archive rotation"],
|
||||||
)
|
)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -394,72 +374,28 @@ func TestNewTargetViews_Log(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
view := viewFor(t, database.Target{
|
view := viewFor(t, database.Target{
|
||||||
Type: database.TargetTypeLog,
|
Type: database.TargetTypeLog,
|
||||||
Config: "",
|
|
||||||
})
|
})
|
||||||
|
|
||||||
assert.Empty(t, view.Config)
|
assert.Empty(t, view.Config)
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestNewTargetViews_Unpresentable proves that no config the
|
// TestNewTargetViews_UnknownType proves that a target of a type the
|
||||||
// view cannot present falls back to the stored blob.
|
// view does not know shows the neutral placeholder, and never its
|
||||||
func TestNewTargetViews_Unpresentable(t *testing.T) {
|
// URL.
|
||||||
|
func TestNewTargetViews_UnknownType(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const blob = `{"webhookUrl":"https://hooks.slack.com` +
|
view := viewFor(t, database.Target{
|
||||||
slackSecretPath + `"`
|
Type: database.TargetType("carrier-pigeon"),
|
||||||
|
URL: slackWebhookURL,
|
||||||
|
})
|
||||||
|
|
||||||
tests := map[string]database.Target{
|
assert.Equal(
|
||||||
"unknown target type": {
|
t,
|
||||||
Type: database.TargetType("carrier-pigeon"),
|
map[string]string{
|
||||||
Config: blob,
|
"Configuration": viewUnavailable,
|
||||||
},
|
},
|
||||||
"unparseable json": {
|
fieldMap(view.Config),
|
||||||
Type: database.TargetTypeSlack,
|
)
|
||||||
Config: blob,
|
|
||||||
MaxAttempts: 1,
|
|
||||||
},
|
|
||||||
"empty slack config": {
|
|
||||||
Type: database.TargetTypeSlack,
|
|
||||||
MaxAttempts: 1,
|
|
||||||
},
|
|
||||||
"slack config without url": {
|
|
||||||
Type: database.TargetTypeSlack,
|
|
||||||
Config: `{}`,
|
|
||||||
MaxAttempts: 1,
|
|
||||||
},
|
|
||||||
"unparseable http json": {
|
|
||||||
Type: database.TargetTypeHTTP,
|
|
||||||
Config: `{"url":`,
|
|
||||||
MaxAttempts: 1,
|
|
||||||
},
|
|
||||||
"unparseable archive json": {
|
|
||||||
Type: database.TargetTypeDatabase,
|
|
||||||
Config: `{"expiry":`,
|
|
||||||
},
|
|
||||||
"invalid archive expiry": {
|
|
||||||
Type: database.TargetTypeDatabase,
|
|
||||||
Config: `{"expiry":"a fortnight"}`,
|
|
||||||
},
|
|
||||||
"invalid archive rotation": {
|
|
||||||
Type: database.TargetTypeDatabase,
|
|
||||||
Config: weeklyConfig,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
for name, target := range tests {
|
|
||||||
t.Run(name, func(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
view := viewFor(t, target)
|
|
||||||
|
|
||||||
assert.Equal(
|
|
||||||
t,
|
|
||||||
map[string]string{
|
|
||||||
"Configuration": viewUnavailable,
|
|
||||||
},
|
|
||||||
fieldMap(view.Config),
|
|
||||||
)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
package delivery_test
|
package delivery_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"database/sql"
|
"database/sql"
|
||||||
"fmt"
|
"fmt"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
@@ -20,6 +21,7 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/archive"
|
"sneak.berlin/go/webhooker/internal/archive"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
"sneak.berlin/go/webhooker/internal/database/databasetest"
|
"sneak.berlin/go/webhooker/internal/database/databasetest"
|
||||||
|
"sneak.berlin/go/webhooker/internal/db/migrations"
|
||||||
"sneak.berlin/go/webhooker/internal/delivery"
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
"sneak.berlin/go/webhooker/internal/gormlog"
|
"sneak.berlin/go/webhooker/internal/gormlog"
|
||||||
)
|
)
|
||||||
@@ -78,10 +80,11 @@ func (env *archiveEnv) archivePath(tgt *database.Target) string {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// seedDatabaseTarget creates a webhook with one database target
|
// seedDatabaseTarget creates a webhook with one database target,
|
||||||
// carrying the given target config JSON, and returns the target.
|
// which keeps its archive forever in one file, and returns the
|
||||||
|
// target.
|
||||||
func (env *archiveEnv) seedDatabaseTarget(
|
func (env *archiveEnv) seedDatabaseTarget(
|
||||||
t *testing.T, configJSON string,
|
t *testing.T,
|
||||||
) *database.Target {
|
) *database.Target {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
@@ -97,12 +100,12 @@ func (env *archiveEnv) seedDatabaseTarget(
|
|||||||
)
|
)
|
||||||
|
|
||||||
tgt := &database.Target{
|
tgt := &database.Target{
|
||||||
WebhookID: wh.ID,
|
WebhookID: wh.ID,
|
||||||
Name: "Archive",
|
Name: "Archive",
|
||||||
Type: database.TargetTypeDatabase,
|
Type: database.TargetTypeDatabase,
|
||||||
Active: true,
|
Active: true,
|
||||||
Config: configJSON,
|
ArchiveRotation: "none",
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
}
|
}
|
||||||
require.NoError(
|
require.NoError(
|
||||||
t,
|
t,
|
||||||
@@ -136,9 +139,10 @@ func seedDatabaseTargetDelivery(
|
|||||||
Status: database.DeliveryStatusPending,
|
Status: database.DeliveryStatusPending,
|
||||||
Event: event,
|
Event: event,
|
||||||
Target: database.Target{
|
Target: database.Target{
|
||||||
Name: tgt.Name,
|
Name: tgt.Name,
|
||||||
Type: database.TargetTypeDatabase,
|
Type: database.TargetTypeDatabase,
|
||||||
Config: tgt.Config,
|
ArchiveExpiry: tgt.ArchiveExpiry,
|
||||||
|
ArchiveRotation: tgt.ArchiveRotation,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
d.ID = dlv.ID
|
d.ID = dlv.ID
|
||||||
@@ -218,7 +222,7 @@ func TestDeliverDatabase_ArchivesEvent(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(t, "")
|
tgt := env.seedDatabaseTarget(t)
|
||||||
|
|
||||||
webhookDB := testWebhookDB(t)
|
webhookDB := testWebhookDB(t)
|
||||||
event := seedEvent(t, webhookDB, `{"archived":true}`)
|
event := seedEvent(t, webhookDB, `{"archived":true}`)
|
||||||
@@ -256,9 +260,9 @@ func TestDeliverDatabase_ArchivesEvent(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// TestDeliverDatabase_ArchiveFailureFailsDelivery verifies that
|
// TestDeliverDatabase_ArchiveFailureFailsDelivery verifies that
|
||||||
// an archive error (here: an unparseable expiry in the target
|
// an archive error (here: an event with no webhook id to record)
|
||||||
// config) fails the delivery loudly: the attempt is recorded as
|
// fails the delivery loudly: the attempt is recorded as failed
|
||||||
// failed with the error and the delivery is marked failed, not
|
// with the error and the delivery is marked failed, not
|
||||||
// delivered.
|
// delivered.
|
||||||
func TestDeliverDatabase_ArchiveFailureFailsDelivery(
|
func TestDeliverDatabase_ArchiveFailureFailsDelivery(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
@@ -266,11 +270,12 @@ func TestDeliverDatabase_ArchiveFailureFailsDelivery(
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupArchiveTest(t)
|
env := setupArchiveTest(t)
|
||||||
tgt := env.seedDatabaseTarget(t, `{"expiry":"nonsense"}`)
|
tgt := env.seedDatabaseTarget(t)
|
||||||
|
|
||||||
webhookDB := testWebhookDB(t)
|
webhookDB := testWebhookDB(t)
|
||||||
event := seedEvent(t, webhookDB, `{"archived":false}`)
|
event := seedEvent(t, webhookDB, `{"archived":false}`)
|
||||||
d := seedDatabaseTargetDelivery(t, webhookDB, event, tgt)
|
d := seedDatabaseTargetDelivery(t, webhookDB, event, tgt)
|
||||||
|
d.Event.WebhookID = ""
|
||||||
|
|
||||||
env.eng.ExportDeliverDatabase(webhookDB, d)
|
env.eng.ExportDeliverDatabase(webhookDB, d)
|
||||||
|
|
||||||
@@ -295,12 +300,96 @@ func TestDeliverDatabase_ArchiveFailureFailsDelivery(
|
|||||||
"the attempt must be recorded as failed",
|
"the attempt must be recorded as failed",
|
||||||
)
|
)
|
||||||
assert.Contains(t,
|
assert.Contains(t,
|
||||||
results[0].Error, "nonsense",
|
results[0].Error, "without a webhook id",
|
||||||
"the archive error must be recorded on the attempt",
|
"the archive error must be recorded on the attempt",
|
||||||
)
|
)
|
||||||
|
|
||||||
assert.NoFileExists(t,
|
assert.NoFileExists(t,
|
||||||
env.archivePath(tgt),
|
env.archivePath(tgt),
|
||||||
"no archive file should exist for a failed config",
|
"no archive file should exist for a failed write",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// seedArchivedRow creates the archive file at path holding one row, for
|
||||||
|
// eventID, archived at archivedAt.
|
||||||
|
func seedArchivedRow(
|
||||||
|
t *testing.T, path, eventID string, archivedAt time.Time,
|
||||||
|
) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
sqlDB, err := sql.Open(
|
||||||
|
"sqlite", fmt.Sprintf("file:%s?mode=rwc", path),
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
defer func() { _ = sqlDB.Close() }()
|
||||||
|
|
||||||
|
gdb, err := gorm.Open(
|
||||||
|
sqlite.Dialector{Conn: sqlDB},
|
||||||
|
&gorm.Config{Logger: gormlog.New(slog.New(slog.DiscardHandler))},
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
require.NoError(t, migrations.Apply(gdb, migrations.Archive))
|
||||||
|
require.NoError(t, gdb.Exec(
|
||||||
|
"INSERT INTO archived_events (event_id, archived_at) VALUES (?, ?)",
|
||||||
|
eventID, archivedAt,
|
||||||
|
).Error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestDeliverDatabase_TaskArchiveSettings proves a task's ArchiveExpiry
|
||||||
|
// and ArchiveRotation reach the archive through the engine. The
|
||||||
|
// target's row has neither, so only the task can bring them: with daily
|
||||||
|
// rotation the event goes to the file for its day, and opening that
|
||||||
|
// file with a one-hour expiry prunes the row it already held, archived
|
||||||
|
// two hours ago.
|
||||||
|
func TestDeliverDatabase_TaskArchiveSettings(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s := newISetup(t)
|
||||||
|
|
||||||
|
webhook := database.Webhook{UserID: uuid.NewString(), Name: "settings"}
|
||||||
|
webhook.ID = s.WebhookID
|
||||||
|
require.NoError(t, s.MainDB.Create(&webhook).Error)
|
||||||
|
|
||||||
|
target := database.Target{
|
||||||
|
WebhookID: s.WebhookID,
|
||||||
|
Name: "archive",
|
||||||
|
Type: database.TargetTypeDatabase,
|
||||||
|
Active: true,
|
||||||
|
MaxAttempts: 1,
|
||||||
|
}
|
||||||
|
require.NoError(t, s.MainDB.Create(&target).Error)
|
||||||
|
|
||||||
|
event := iSeedEvent(t, s.WebhookDB, s.WebhookID, `{"archived":true}`)
|
||||||
|
d := iSeedDelivery(
|
||||||
|
t, s.WebhookDB, event.ID, target.ID,
|
||||||
|
database.DeliveryStatusPending,
|
||||||
|
)
|
||||||
|
|
||||||
|
task := iTask(
|
||||||
|
d, event, s.WebhookID, target.ID, target.Name, "", 1, 1,
|
||||||
|
&event.Body,
|
||||||
|
)
|
||||||
|
task.TargetType = database.TargetTypeDatabase
|
||||||
|
task.ArchiveExpiry = time.Hour
|
||||||
|
task.ArchiveRotation = "daily"
|
||||||
|
|
||||||
|
daily := target
|
||||||
|
daily.ArchiveRotation = task.ArchiveRotation
|
||||||
|
dailyFile := archive.PathAt(s.DBMgr, &webhook, &daily, event.CreatedAt)
|
||||||
|
seedArchivedRow(t, dailyFile, "expired", time.Now().Add(-2*time.Hour))
|
||||||
|
|
||||||
|
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
||||||
|
|
||||||
|
iAssertStatus(t, s.WebhookDB, d.ID, database.DeliveryStatusDelivered)
|
||||||
|
|
||||||
|
var rows []archivedRow
|
||||||
|
|
||||||
|
require.NoError(t, openArchiveDBForRead(t, dailyFile).
|
||||||
|
Table("archived_events").Find(&rows).Error)
|
||||||
|
require.Len(t, rows, 1)
|
||||||
|
assert.Equal(t, event.ID, rows[0].EventID,
|
||||||
|
"the event goes to its day's file and the expired row is pruned",
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -17,7 +17,7 @@ import (
|
|||||||
const MaxTargetTimeoutSeconds = 300
|
const MaxTargetTimeoutSeconds = 300
|
||||||
|
|
||||||
// Errors returned when a target's header or timeout form input
|
// Errors returned when a target's header or timeout form input
|
||||||
// cannot be turned into a configuration.
|
// cannot be turned into the target's setting.
|
||||||
//
|
//
|
||||||
// None of these ever quotes a header VALUE. A target header value
|
// None of these ever quotes a header VALUE. A target header value
|
||||||
// is routinely an authorization token, and these messages are shown
|
// is routinely an authorization token, and these messages are shown
|
||||||
@@ -48,8 +48,8 @@ var (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// isReservedTargetHeader reports whether name (canonicalised) is a
|
// isReservedTargetHeader reports whether name (canonicalised) is a
|
||||||
// header a target configuration may not set, because the delivery
|
// header a target may not set, because the delivery path or
|
||||||
// path or net/http itself writes it regardless.
|
// net/http itself writes it regardless.
|
||||||
//
|
//
|
||||||
// These are rejected rather than accepted-and-ignored. Storing a
|
// These are rejected rather than accepted-and-ignored. Storing a
|
||||||
// header that provably never reaches the wire tells the operator
|
// header that provably never reaches the wire tells the operator
|
||||||
@@ -77,13 +77,11 @@ func isReservedTargetHeader(name string) bool {
|
|||||||
|
|
||||||
// ParseTargetHeaders turns the target form's headers field — one
|
// ParseTargetHeaders turns the target form's headers field — one
|
||||||
// "Name: value" pair per line, blank lines ignored — into the map
|
// "Name: value" pair per line, blank lines ignored — into the map
|
||||||
// stored in HTTPTargetConfig.Headers. Names are canonicalised, so a
|
// stored in database.Target.Headers. Names are canonicalised, so a
|
||||||
// name repeated in a different case is still a duplicate rather than
|
// name repeated in a different case is still a duplicate rather than
|
||||||
// one pair silently overwriting the other.
|
// one pair silently overwriting the other.
|
||||||
//
|
//
|
||||||
// An input with no pairs yields an empty map, which omitempty drops
|
// An input with no pairs yields an empty map.
|
||||||
// from the stored config, so a target configured with no headers
|
|
||||||
// stores no headers key at all.
|
|
||||||
func ParseTargetHeaders(raw string) (map[string]string, error) {
|
func ParseTargetHeaders(raw string) (map[string]string, error) {
|
||||||
headers := make(map[string]string)
|
headers := make(map[string]string)
|
||||||
|
|
||||||
@@ -223,8 +221,8 @@ func FormatTargetHeaders(headers map[string]string) string {
|
|||||||
|
|
||||||
// ParseTargetTimeout interprets the target form's timeout field as
|
// ParseTargetTimeout interprets the target form's timeout field as
|
||||||
// a whole number of seconds. An empty field means "unset" and yields
|
// a whole number of seconds. An empty field means "unset" and yields
|
||||||
// 0, which omitempty drops from the stored config and which the
|
// 0, which the delivery path reads as "use the shared client's
|
||||||
// delivery path reads as "use the shared client's timeout".
|
// timeout".
|
||||||
//
|
//
|
||||||
// Anything else that is not a whole number in range is an error, not
|
// Anything else that is not a whole number in range is an error, not
|
||||||
// a silently substituted default: a target whose timeout was typed
|
// a silently substituted default: a target whose timeout was typed
|
||||||
|
|||||||
@@ -1,23 +1,21 @@
|
|||||||
package delivery_test
|
package delivery_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"encoding/json"
|
|
||||||
"strconv"
|
"strconv"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
|
||||||
"sneak.berlin/go/webhooker/internal/delivery"
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Literals these tests repeat, named so that the header names and the
|
// Literals these tests repeat, named so that each header name has one
|
||||||
// keep-forever archive config each have one definition.
|
// definition.
|
||||||
const (
|
const (
|
||||||
headerAuthorization = "Authorization"
|
headerAuthorization = "Authorization"
|
||||||
headerContentType = "Content-Type"
|
headerContentType = "Content-Type"
|
||||||
|
headerUserAgent = "User-Agent"
|
||||||
bearerValue = "Bearer abc"
|
bearerValue = "Bearer abc"
|
||||||
archiveConfigNever = "{\"expiry\":\"never\"}"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestParseTargetHeaders_AcceptsPairs(t *testing.T) {
|
func TestParseTargetHeaders_AcceptsPairs(t *testing.T) {
|
||||||
@@ -41,23 +39,13 @@ func TestParseTargetHeaders_AcceptsPairs(t *testing.T) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// A configuration with no headers must stay indistinguishable from
|
// An input of blank lines is no headers, not an error.
|
||||||
// one written before the field existed, so omitempty drops the key.
|
|
||||||
func TestParseTargetHeaders_EmptyInputYieldsNoHeaders(t *testing.T) {
|
func TestParseTargetHeaders_EmptyInputYieldsNoHeaders(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
got, err := delivery.ParseTargetHeaders("\n \n")
|
got, err := delivery.ParseTargetHeaders("\n \n")
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
assert.Empty(t, got)
|
assert.Empty(t, got)
|
||||||
|
|
||||||
encoded, err := json.Marshal(delivery.HTTPTargetConfig{
|
|
||||||
URL: "https://example.com/h",
|
|
||||||
Headers: got,
|
|
||||||
})
|
|
||||||
require.NoError(t, err)
|
|
||||||
assert.JSONEq(
|
|
||||||
t, `{"url":"https://example.com/h"}`, string(encoded),
|
|
||||||
)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestParseTargetHeaders_Rejects(t *testing.T) {
|
func TestParseTargetHeaders_Rejects(t *testing.T) {
|
||||||
@@ -181,89 +169,6 @@ func TestFormatTargetTimeout(t *testing.T) {
|
|||||||
assert.Empty(t, delivery.FormatTargetTimeout(-1))
|
assert.Empty(t, delivery.FormatTargetTimeout(-1))
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestNewTargetConfigForm(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
form, err := delivery.NewTargetConfigForm(&database.Target{
|
|
||||||
Type: database.TargetTypeHTTP,
|
|
||||||
Config: `{"url":"https://example.com/h",` +
|
|
||||||
`"headers":{"Authorization":"Bearer abc"},` +
|
|
||||||
`"timeout":9}`,
|
|
||||||
MaxAttempts: 1,
|
|
||||||
})
|
|
||||||
require.NoError(t, err)
|
|
||||||
assert.Equal(t, "https://example.com/h", form.URL)
|
|
||||||
assert.Equal(t, "Authorization: Bearer abc\n", form.Headers)
|
|
||||||
assert.Equal(t, "9", form.Timeout)
|
|
||||||
|
|
||||||
form, err = delivery.NewTargetConfigForm(&database.Target{
|
|
||||||
Type: database.TargetTypeSlack,
|
|
||||||
Config: `{"webhookUrl":"https://hooks.example/s"}`,
|
|
||||||
MaxAttempts: 1,
|
|
||||||
})
|
|
||||||
require.NoError(t, err)
|
|
||||||
assert.Equal(t, "https://hooks.example/s", form.URL)
|
|
||||||
|
|
||||||
form, err = delivery.NewTargetConfigForm(&database.Target{
|
|
||||||
Type: database.TargetTypeDatabase,
|
|
||||||
Config: `{"expiry":"720h","rotation":"daily"}`,
|
|
||||||
})
|
|
||||||
require.NoError(t, err)
|
|
||||||
assert.Equal(t, "720h", form.Expiry)
|
|
||||||
assert.Equal(t, rotationDaily, form.Rotation)
|
|
||||||
|
|
||||||
form, err = delivery.NewTargetConfigForm(&database.Target{
|
|
||||||
Type: database.TargetTypeLog,
|
|
||||||
})
|
|
||||||
require.NoError(t, err)
|
|
||||||
assert.Empty(t, form.URL)
|
|
||||||
}
|
|
||||||
|
|
||||||
// A keep-forever archive target yields an empty expiry, so the edit
|
|
||||||
// form starts on never; saving it unchanged stores never, which means
|
|
||||||
// the same as an empty expiry.
|
|
||||||
func TestNewTargetConfigForm_DatabaseNeverIsBlank(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
for _, cfg := range []string{"", `{}`, archiveConfigNever} {
|
|
||||||
form, err := delivery.NewTargetConfigForm(
|
|
||||||
&database.Target{
|
|
||||||
Type: database.TargetTypeDatabase,
|
|
||||||
Config: cfg,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
require.NoError(t, err, cfg)
|
|
||||||
assert.Empty(t, form.Expiry, cfg)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// An unreadable stored config is an error rather than a blank form
|
|
||||||
// that looks like a target with no settings, so the caller can tell
|
|
||||||
// the operator that saving replaces the stored value.
|
|
||||||
func TestNewTargetConfigForm_UnreadableConfigErrors(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
cases := []*database.Target{
|
|
||||||
{Type: database.TargetTypeHTTP, Config: "not json", MaxAttempts: 1},
|
|
||||||
{Type: database.TargetTypeHTTP, Config: `{}`, MaxAttempts: 1},
|
|
||||||
{Type: database.TargetTypeSlack, Config: "", MaxAttempts: 1},
|
|
||||||
{
|
|
||||||
Type: database.TargetTypeDatabase,
|
|
||||||
Config: `{"expiry":"soon"}`,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
Type: database.TargetTypeDatabase,
|
|
||||||
Config: weeklyConfig,
|
|
||||||
},
|
|
||||||
{Type: database.TargetType("nope")},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, target := range cases {
|
|
||||||
_, err := delivery.NewTargetConfigForm(target)
|
|
||||||
require.Error(t, err, target.Type)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// The ceiling exists so one misconfigured target cannot hold a
|
// The ceiling exists so one misconfigured target cannot hold a
|
||||||
// delivery worker indefinitely, and it is inclusive.
|
// delivery worker indefinitely, and it is inclusive.
|
||||||
func TestParseTargetTimeout_CeilingIsInclusive(t *testing.T) {
|
func TestParseTargetTimeout_CeilingIsInclusive(t *testing.T) {
|
||||||
|
|||||||
@@ -4,7 +4,6 @@ import (
|
|||||||
"bytes"
|
"bytes"
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -17,29 +16,6 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Sentinel errors returned by the config parsers.
|
|
||||||
var (
|
|
||||||
errEmptyTargetConfig = errors.New(
|
|
||||||
"empty target config",
|
|
||||||
)
|
|
||||||
errMissingTargetURL = errors.New(
|
|
||||||
"target URL is required",
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
// HTTPTargetConfig holds configuration for http target
|
|
||||||
// types.
|
|
||||||
type HTTPTargetConfig struct {
|
|
||||||
URL string `json:"url"`
|
|
||||||
Headers map[string]string `json:"headers,omitempty"`
|
|
||||||
Timeout int `json:"timeout,omitempty"`
|
|
||||||
|
|
||||||
// ForwardQuery passes each event's query string on to the target,
|
|
||||||
// appended to URL. Off, the target URL is sent exactly as
|
|
||||||
// configured.
|
|
||||||
ForwardQuery bool `json:"forwardQuery,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// httpCore holds the retry, backoff, and circuit-breaker
|
// httpCore holds the retry, backoff, and circuit-breaker
|
||||||
// machinery shared by the HTTP and Slack targets. Each of
|
// machinery shared by the HTTP and Slack targets. Each of
|
||||||
// those targets owns its own httpCore instance (and thus its
|
// those targets owns its own httpCore instance (and thus its
|
||||||
@@ -385,34 +361,8 @@ func (t *httpTarget) Deliver(
|
|||||||
task *Task,
|
task *Task,
|
||||||
sched Scheduler,
|
sched Scheduler,
|
||||||
) {
|
) {
|
||||||
cfg, err := parseHTTPConfig(d.Target.Config)
|
|
||||||
if err != nil {
|
|
||||||
t.eng.log.Error(
|
|
||||||
"invalid HTTP target config",
|
|
||||||
"target_id", d.TargetID,
|
|
||||||
"error", err,
|
|
||||||
)
|
|
||||||
|
|
||||||
recErr := t.eng.recordResult(
|
|
||||||
webhookDB, d, task.AttemptNum,
|
|
||||||
false, 0, "", err.Error(), 0,
|
|
||||||
)
|
|
||||||
if recErr != nil {
|
|
||||||
t.eng.bookkeepingFailed(d, recErr)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
t.eng.settleStatus(
|
|
||||||
webhookDB, d, d.Target.Type,
|
|
||||||
database.DeliveryStatusFailed,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
attempt := func() attemptResult {
|
attempt := func() attemptResult {
|
||||||
return t.attempt(ctx, cfg, &d.Event)
|
return t.attempt(ctx, &d.Target, &d.Event)
|
||||||
}
|
}
|
||||||
|
|
||||||
t.deliver(
|
t.deliver(
|
||||||
@@ -427,11 +377,11 @@ func (t *httpTarget) Deliver(
|
|||||||
// string; only a transport-level error does.
|
// string; only a transport-level error does.
|
||||||
func (t *httpTarget) attempt(
|
func (t *httpTarget) attempt(
|
||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
cfg *HTTPTargetConfig,
|
target *database.Target,
|
||||||
event *database.Event,
|
event *database.Event,
|
||||||
) attemptResult {
|
) attemptResult {
|
||||||
statusCode, respBody, duration, reqErr :=
|
statusCode, respBody, duration, reqErr :=
|
||||||
t.doHTTPRequest(ctx, cfg, event)
|
t.doHTTPRequest(ctx, target, event)
|
||||||
|
|
||||||
success := reqErr == nil &&
|
success := reqErr == nil &&
|
||||||
statusCode >= httpSuccessMin &&
|
statusCode >= httpSuccessMin &&
|
||||||
@@ -453,7 +403,7 @@ func (t *httpTarget) attempt(
|
|||||||
|
|
||||||
func (t *httpTarget) doHTTPRequest(
|
func (t *httpTarget) doHTTPRequest(
|
||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
cfg *HTTPTargetConfig,
|
target *database.Target,
|
||||||
event *database.Event,
|
event *database.Event,
|
||||||
) (int, string, int64, error) {
|
) (int, string, int64, error) {
|
||||||
start := time.Now()
|
start := time.Now()
|
||||||
@@ -461,7 +411,7 @@ func (t *httpTarget) doHTTPRequest(
|
|||||||
req, reqErr := http.NewRequestWithContext(
|
req, reqErr := http.NewRequestWithContext(
|
||||||
ctx,
|
ctx,
|
||||||
http.MethodPost,
|
http.MethodPost,
|
||||||
cfg.URL,
|
target.URL,
|
||||||
bytes.NewReader([]byte(event.Body)),
|
bytes.NewReader([]byte(event.Body)),
|
||||||
)
|
)
|
||||||
if reqErr != nil {
|
if reqErr != nil {
|
||||||
@@ -471,15 +421,15 @@ func (t *httpTarget) doHTTPRequest(
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if cfg.ForwardQuery {
|
if target.ForwardQuery {
|
||||||
appendQuery(req.URL, event.RawQuery)
|
appendQuery(req.URL, event.RawQuery)
|
||||||
}
|
}
|
||||||
|
|
||||||
originScoped := applyRequestHeaders(
|
originScoped := applyRequestHeaders(
|
||||||
req, event, cfg, t.eng.userAgent(),
|
req, event, target.Headers, t.eng.userAgent(),
|
||||||
)
|
)
|
||||||
|
|
||||||
client := t.clientForRequest(cfg, originScoped)
|
client := t.clientForRequest(target.TimeoutSeconds, originScoped)
|
||||||
|
|
||||||
resp, doErr := executeHTTPRequest(client, req)
|
resp, doErr := executeHTTPRequest(client, req)
|
||||||
|
|
||||||
@@ -519,17 +469,17 @@ func appendQuery(u *url.URL, rawQuery string) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// clientForRequest returns the client for one delivery attempt.
|
// clientForRequest returns the client for one delivery attempt.
|
||||||
// originScoped is the header set applyRequestHeaders built for that
|
// timeoutSeconds is the target's TimeoutSeconds, and originScoped is
|
||||||
// attempt; a request with neither a per-target timeout nor an
|
// the header set applyRequestHeaders built for that attempt; a
|
||||||
// origin-scoped header gets the shared client, because there is
|
// request with neither a per-target timeout nor an origin-scoped
|
||||||
// then nothing for the redirect policy to strip and net/http's
|
// header gets the shared client, because there is then nothing for
|
||||||
// default policy already withholds Authorization and Cookie across
|
// the redirect policy to strip and net/http's default policy already
|
||||||
// hosts.
|
// withholds Authorization and Cookie across hosts.
|
||||||
func (t *httpTarget) clientForRequest(
|
func (t *httpTarget) clientForRequest(
|
||||||
cfg *HTTPTargetConfig,
|
timeoutSeconds int,
|
||||||
originScoped []string,
|
originScoped []string,
|
||||||
) *http.Client {
|
) *http.Client {
|
||||||
if cfg.Timeout <= 0 && len(originScoped) == 0 {
|
if timeoutSeconds <= 0 && len(originScoped) == 0 {
|
||||||
return t.client
|
return t.client
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -542,9 +492,9 @@ func (t *httpTarget) clientForRequest(
|
|||||||
Transport: t.client.Transport,
|
Transport: t.client.Transport,
|
||||||
}
|
}
|
||||||
|
|
||||||
if cfg.Timeout > 0 {
|
if timeoutSeconds > 0 {
|
||||||
client.Timeout = time.Duration(
|
client.Timeout = time.Duration(
|
||||||
cfg.Timeout,
|
timeoutSeconds,
|
||||||
) * time.Second
|
) * time.Second
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -555,31 +505,6 @@ func (t *httpTarget) clientForRequest(
|
|||||||
return client
|
return client
|
||||||
}
|
}
|
||||||
|
|
||||||
func parseHTTPConfig(
|
|
||||||
configJSON string,
|
|
||||||
) (*HTTPTargetConfig, error) {
|
|
||||||
if configJSON == "" {
|
|
||||||
return nil, errEmptyTargetConfig
|
|
||||||
}
|
|
||||||
|
|
||||||
var cfg HTTPTargetConfig
|
|
||||||
|
|
||||||
err := json.Unmarshal(
|
|
||||||
[]byte(configJSON), &cfg,
|
|
||||||
)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf(
|
|
||||||
"parsing config JSON: %w", err,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
if cfg.URL == "" {
|
|
||||||
return nil, errMissingTargetURL
|
|
||||||
}
|
|
||||||
|
|
||||||
return &cfg, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// isForwardableHeader returns true if the header should
|
// isForwardableHeader returns true if the header should
|
||||||
// be forwarded to targets.
|
// be forwarded to targets.
|
||||||
func isForwardableHeader(name string) bool {
|
func isForwardableHeader(name string) bool {
|
||||||
@@ -611,11 +536,12 @@ func isForwardableHeader(name string) bool {
|
|||||||
// wins, otherwise the event's ContentType, otherwise none. The inbound
|
// wins, otherwise the event's ContentType, otherwise none. The inbound
|
||||||
// Content-Type in the event's headers is never forwarded.
|
// Content-Type in the event's headers is never forwarded.
|
||||||
//
|
//
|
||||||
// userAgent is set last, over any configured or inbound User-Agent.
|
// configured is the target's Headers. userAgent is set last, over
|
||||||
|
// any configured or inbound User-Agent.
|
||||||
func applyRequestHeaders(
|
func applyRequestHeaders(
|
||||||
req *http.Request,
|
req *http.Request,
|
||||||
event *database.Event,
|
event *database.Event,
|
||||||
cfg *HTTPTargetConfig,
|
configured map[string]string,
|
||||||
userAgent string,
|
userAgent string,
|
||||||
) []string {
|
) []string {
|
||||||
if event.ContentType != "" {
|
if event.ContentType != "" {
|
||||||
@@ -626,7 +552,7 @@ func applyRequestHeaders(
|
|||||||
|
|
||||||
originScoped := forwardEventHeaders(req, event)
|
originScoped := forwardEventHeaders(req, event)
|
||||||
|
|
||||||
for k, v := range cfg.Headers {
|
for k, v := range configured {
|
||||||
req.Header.Set(k, v)
|
req.Header.Set(k, v)
|
||||||
originScoped[http.CanonicalHeaderKey(k)] = struct{}{}
|
originScoped[http.CanonicalHeaderKey(k)] = struct{}{}
|
||||||
}
|
}
|
||||||
@@ -691,7 +617,7 @@ func forwardEventHeaders(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// executeHTTPRequest sends an HTTP request using the provided
|
// executeHTTPRequest sends an HTTP request using the provided
|
||||||
// client. URLs are validated by the config parsers and the
|
// client. URLs are validated by the target forms and the
|
||||||
// SSRF-safe transport before reaching here.
|
// SSRF-safe transport before reaching here.
|
||||||
//
|
//
|
||||||
// Transport failures are masked here, at the single point
|
// Transport failures are masked here, at the single point
|
||||||
|
|||||||
@@ -2,7 +2,6 @@ package delivery_test
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"testing"
|
"testing"
|
||||||
@@ -18,25 +17,13 @@ import (
|
|||||||
// with.
|
// with.
|
||||||
const eventQuery = "a=1&b=2"
|
const eventQuery = "a=1&b=2"
|
||||||
|
|
||||||
// httpTargetConfig is the stored configuration of an HTTP target at
|
|
||||||
// targetURL.
|
|
||||||
func httpTargetConfig(
|
|
||||||
t *testing.T, targetURL string, forwardQuery bool,
|
|
||||||
) string {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
cfg, err := json.Marshal(delivery.HTTPTargetConfig{
|
|
||||||
URL: targetURL, ForwardQuery: forwardQuery,
|
|
||||||
})
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
return string(cfg)
|
|
||||||
}
|
|
||||||
|
|
||||||
// deliverWithQuery sends one event that arrived with eventQuery to an
|
// deliverWithQuery sends one event that arrived with eventQuery to an
|
||||||
// HTTP target configured with cfg, through the path a received event's
|
// HTTP target at targetURL, with its ForwardQuery setting as given,
|
||||||
// delivery takes, and returns the attempt it recorded.
|
// through the path a received event's delivery takes, and returns the
|
||||||
func deliverWithQuery(t *testing.T, cfg string) database.DeliveryResult {
|
// attempt it recorded.
|
||||||
|
func deliverWithQuery(
|
||||||
|
t *testing.T, targetURL string, forwardQuery bool,
|
||||||
|
) database.DeliveryResult {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
s := newISetup(t)
|
s := newISetup(t)
|
||||||
@@ -47,8 +34,10 @@ func deliverWithQuery(t *testing.T, cfg string) database.DeliveryResult {
|
|||||||
database.DeliveryStatusPending,
|
database.DeliveryStatusPending,
|
||||||
)
|
)
|
||||||
task := iTask(
|
task := iTask(
|
||||||
d, event, s.WebhookID, d.TargetID, "query", cfg, 1, 1, &event.Body,
|
d, event, s.WebhookID, d.TargetID, "query", targetURL, 1, 1,
|
||||||
|
&event.Body,
|
||||||
)
|
)
|
||||||
|
task.ForwardQuery = forwardQuery
|
||||||
task.RawQuery = eventQuery
|
task.RawQuery = eventQuery
|
||||||
|
|
||||||
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
||||||
@@ -111,9 +100,9 @@ func TestDeliverHTTP_ForwardQuery(t *testing.T) {
|
|||||||
))
|
))
|
||||||
t.Cleanup(ts.Close)
|
t.Cleanup(ts.Close)
|
||||||
|
|
||||||
result := deliverWithQuery(t, httpTargetConfig(
|
result := deliverWithQuery(
|
||||||
t, ts.URL+tc.path, tc.forwardQuery,
|
t, ts.URL+tc.path, tc.forwardQuery,
|
||||||
))
|
)
|
||||||
|
|
||||||
assert.True(t, result.Success)
|
assert.True(t, result.Success)
|
||||||
require.Len(t, received, 1)
|
require.Len(t, received, 1)
|
||||||
@@ -144,12 +133,13 @@ func TestDeliverHTTP_ForwardedQueryKeepsTheTargetURLMasked(t *testing.T) {
|
|||||||
t.Cleanup(ts.Close)
|
t.Cleanup(ts.Close)
|
||||||
|
|
||||||
target := &database.Target{
|
target := &database.Target{
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: httpTargetConfig(t, ts.URL+"/in?token="+secret, true),
|
URL: ts.URL + "/in?token=" + secret,
|
||||||
MaxAttempts: 1,
|
ForwardQuery: true,
|
||||||
|
MaxAttempts: 1,
|
||||||
}
|
}
|
||||||
|
|
||||||
deliverWithQuery(t, target.Config)
|
deliverWithQuery(t, target.URL, target.ForwardQuery)
|
||||||
require.Len(t, received, 1)
|
require.Len(t, received, 1)
|
||||||
|
|
||||||
sent := <-received
|
sent := <-received
|
||||||
@@ -164,9 +154,9 @@ func TestDeliverHTTP_ForwardedQueryKeepsTheTargetURLMasked(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Nothing listens on port 1.
|
// Nothing listens on port 1.
|
||||||
failed := deliverWithQuery(t, httpTargetConfig(
|
failed := deliverWithQuery(
|
||||||
t, "http://127.0.0.1:1/in?token="+secret, true,
|
t, "http://127.0.0.1:1/in?token="+secret, true,
|
||||||
))
|
)
|
||||||
require.NotEmpty(t, failed.Error)
|
require.NotEmpty(t, failed.Error)
|
||||||
assert.NotContains(t, failed.Error, secret)
|
assert.NotContains(t, failed.Error, secret)
|
||||||
assert.NotContains(t, failed.Error, eventQuery)
|
assert.NotContains(t, failed.Error, eventQuery)
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ const RedactionMarker = "(redacted)"
|
|||||||
// line behind the delivery path's own URL masking.
|
// line behind the delivery path's own URL masking.
|
||||||
//
|
//
|
||||||
// It removes byte-identical echoes of strings taken from the
|
// It removes byte-identical echoes of strings taken from the
|
||||||
// target's stored configuration, and nothing else. Anything
|
// target's URL and headers, and nothing else. Anything
|
||||||
// the remote re-encodes survives: JSON "\/" escaping (what
|
// the remote re-encodes survives: JSON "\/" escaping (what
|
||||||
// PHP's json_encode emits by default), percent-encoding, HTML
|
// PHP's json_encode emits by default), percent-encoding, HTML
|
||||||
// entities, and an echo of only part of a path. It cannot
|
// entities, and an echo of only part of a path. It cannot
|
||||||
@@ -115,7 +115,7 @@ func (r Redactor) secretPrefixSuffix(s string) int {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// targetSecrets returns the credential-bearing strings a
|
// targetSecrets returns the credential-bearing strings a
|
||||||
// target's configuration carries.
|
// target's settings carry.
|
||||||
//
|
//
|
||||||
// The destination URL contributes. Its path, query and
|
// The destination URL contributes. Its path, query and
|
||||||
// userinfo are the credential for both target types that have
|
// userinfo are the credential for both target types that have
|
||||||
@@ -129,36 +129,15 @@ func (r Redactor) secretPrefixSuffix(s string) int {
|
|||||||
// picks out. That is the same class-based rule applied to
|
// picks out. That is the same class-based rule applied to
|
||||||
// URLs: an echoed Accept or User-Agent still renders, an
|
// URLs: an echoed Accept or User-Agent still renders, an
|
||||||
// echoed Authorization does not.
|
// echoed Authorization does not.
|
||||||
|
//
|
||||||
|
// A setting a type does not use is empty and contributes
|
||||||
|
// nothing, so a database or log target has nothing to redact.
|
||||||
func targetSecrets(t *database.Target) []string {
|
func targetSecrets(t *database.Target) []string {
|
||||||
if t == nil {
|
if t == nil {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
switch t.Type {
|
return append(urlSecrets(t.URL), headerSecrets(t.Headers)...)
|
||||||
case database.TargetTypeSlack:
|
|
||||||
cfg, err := parseSlackConfig(t.Config)
|
|
||||||
if err != nil {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
return urlSecrets(cfg.WebhookURL)
|
|
||||||
case database.TargetTypeHTTP:
|
|
||||||
cfg, err := parseHTTPConfig(t.Config)
|
|
||||||
if err != nil {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
return append(
|
|
||||||
urlSecrets(cfg.URL),
|
|
||||||
headerSecrets(cfg.Headers)...,
|
|
||||||
)
|
|
||||||
case database.TargetTypeDatabase, database.TargetTypeLog:
|
|
||||||
// Neither has a destination URL, so neither has
|
|
||||||
// anything to redact.
|
|
||||||
return nil
|
|
||||||
default:
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// urlSecrets returns the substrings of a destination URL that
|
// urlSecrets returns the substrings of a destination URL that
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ func TestRedactor_RemovesSlackWebhookURL(t *testing.T) {
|
|||||||
|
|
||||||
r := delivery.NewRedactor(&database.Target{
|
r := delivery.NewRedactor(&database.Target{
|
||||||
Type: database.TargetTypeSlack,
|
Type: database.TargetTypeSlack,
|
||||||
Config: `{"webhookUrl":"` + redactWebhookURL + `"}`,
|
URL: redactWebhookURL,
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -54,7 +54,7 @@ func TestRedactor_RemovesSecretSeveredByACut(t *testing.T) {
|
|||||||
|
|
||||||
r := delivery.NewRedactor(&database.Target{
|
r := delivery.NewRedactor(&database.Target{
|
||||||
Type: database.TargetTypeSlack,
|
Type: database.TargetTypeSlack,
|
||||||
Config: `{"webhookUrl":"` + redactWebhookURL + `"}`,
|
URL: redactWebhookURL,
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -86,18 +86,19 @@ func TestRedactor_RedactsCredentialShapedHeaderValues(
|
|||||||
|
|
||||||
r := delivery.NewRedactor(&database.Target{
|
r := delivery.NewRedactor(&database.Target{
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: `{"url":"https://example.com/in",` +
|
URL: "https://example.com/in",
|
||||||
`"headers":{` +
|
Headers: map[string]string{
|
||||||
`"Authorization":"Bearer AAAAAAAAAAAA",` +
|
headerAuthorization: "Bearer AAAAAAAAAAAA",
|
||||||
`"Cookie":"session=BBBBBBBBBBBB",` +
|
"Cookie": "session=BBBBBBBBBBBB",
|
||||||
`"X-Api-Key":"CCCCCCCCCCCC",` +
|
probeHeaderName: "CCCCCCCCCCCC",
|
||||||
`"X-Hub-Signature":"sha256=DDDDDDDDDDDD",` +
|
"X-Hub-Signature": "sha256=DDDDDDDDDDDD",
|
||||||
`"X-Sig":"EEEEEEEEEEEE",` +
|
"X-Sig": "EEEEEEEEEEEE",
|
||||||
`"X-Pass":"FFFFFFFFFFFF",` +
|
"X-Pass": "FFFFFFFFFFFF",
|
||||||
`"X-HMAC":"GGGGGGGGGGGG",` +
|
"X-HMAC": "GGGGGGGGGGGG",
|
||||||
`"X-Credential":"HHHHHHHHHHHH",` +
|
"X-Credential": "HHHHHHHHHHHH",
|
||||||
`"Accept":"application/json",` +
|
"Accept": "application/json",
|
||||||
`"User-Agent":"webhooker/1.0"}}`,
|
headerUserAgent: "webhooker/1.0",
|
||||||
|
},
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -136,9 +137,9 @@ func TestRedactor_IgnoresVeryShortHeaderValues(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
r := delivery.NewRedactor(&database.Target{
|
r := delivery.NewRedactor(&database.Target{
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: `{"url":"https://example.com/in",` +
|
URL: "https://example.com/in",
|
||||||
`"headers":{"X-Api-Key":"ab"}}`,
|
Headers: map[string]string{probeHeaderName: "ab"},
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -155,7 +156,7 @@ func TestRedactor_RemovesBarePath(t *testing.T) {
|
|||||||
|
|
||||||
r := delivery.NewRedactor(&database.Target{
|
r := delivery.NewRedactor(&database.Target{
|
||||||
Type: database.TargetTypeSlack,
|
Type: database.TargetTypeSlack,
|
||||||
Config: `{"webhookUrl":"` + redactWebhookURL + `"}`,
|
URL: redactWebhookURL,
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -189,7 +190,7 @@ func TestRedactor_RemovesHTTPURLQueryAndUserinfo(t *testing.T) {
|
|||||||
|
|
||||||
r := delivery.NewRedactor(&database.Target{
|
r := delivery.NewRedactor(&database.Target{
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: `{"url":"` + raw + `"}`,
|
URL: raw,
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -239,7 +240,7 @@ func TestRedactor_RemovesEchoedQueryOfURLWithoutPath(t *testing.T) {
|
|||||||
} {
|
} {
|
||||||
r := delivery.NewRedactor(&database.Target{
|
r := delivery.NewRedactor(&database.Target{
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: `{"url":"` + dest + `"}`,
|
URL: dest,
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -261,7 +262,7 @@ func TestRedactor_LeavesUnrelatedTextAlone(t *testing.T) {
|
|||||||
|
|
||||||
r := delivery.NewRedactor(&database.Target{
|
r := delivery.NewRedactor(&database.Target{
|
||||||
Type: database.TargetTypeSlack,
|
Type: database.TargetTypeSlack,
|
||||||
Config: `{"webhookUrl":"` + redactWebhookURL + `"}`,
|
URL: redactWebhookURL,
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -288,7 +289,7 @@ func TestRedactor_EmptyUserinfoDoesNotShredTheBody(t *testing.T) {
|
|||||||
|
|
||||||
r := delivery.NewRedactor(&database.Target{
|
r := delivery.NewRedactor(&database.Target{
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: `{"url":"` + dest + `"}`,
|
URL: dest,
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -306,11 +307,10 @@ func TestRedactor_EmptyUserinfoDoesNotShredTheBody(t *testing.T) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestRedactor_ZeroValueAndConfiglessTargets pins that a
|
// TestRedactor_ZeroValueAndURLlessTargets pins that a caller
|
||||||
// caller with no target, an unparseable config, or a target
|
// with no target, or a target with no destination URL, gets a
|
||||||
// type with no destination URL gets a redactor that changes
|
// redactor that changes nothing rather than one that panics.
|
||||||
// nothing rather than one that panics.
|
func TestRedactor_ZeroValueAndURLlessTargets(t *testing.T) {
|
||||||
func TestRedactor_ZeroValueAndConfiglessTargets(t *testing.T) {
|
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const text = "some response body"
|
const text = "some response body"
|
||||||
@@ -323,8 +323,8 @@ func TestRedactor_ZeroValueAndConfiglessTargets(t *testing.T) {
|
|||||||
for _, tgt := range []database.Target{
|
for _, tgt := range []database.Target{
|
||||||
{Type: database.TargetTypeLog},
|
{Type: database.TargetTypeLog},
|
||||||
{Type: database.TargetTypeDatabase},
|
{Type: database.TargetTypeDatabase},
|
||||||
{Type: database.TargetTypeSlack, Config: "not json", MaxAttempts: 1},
|
{Type: database.TargetTypeSlack, MaxAttempts: 1},
|
||||||
{Type: database.TargetTypeHTTP, Config: "", MaxAttempts: 1},
|
{Type: database.TargetTypeHTTP, MaxAttempts: 1},
|
||||||
} {
|
} {
|
||||||
assert.Equal(
|
assert.Equal(
|
||||||
t, text,
|
t, text,
|
||||||
|
|||||||
@@ -4,7 +4,6 @@ import (
|
|||||||
"bytes"
|
"bytes"
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -15,18 +14,6 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
)
|
)
|
||||||
|
|
||||||
// errMissingWebhookURL is returned when a Slack target config
|
|
||||||
// omits its webhook URL.
|
|
||||||
var errMissingWebhookURL = errors.New(
|
|
||||||
"webhook_url is required",
|
|
||||||
)
|
|
||||||
|
|
||||||
// SlackTargetConfig holds configuration for slack target
|
|
||||||
// types.
|
|
||||||
type SlackTargetConfig struct {
|
|
||||||
WebhookURL string `json:"webhookUrl"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// slackTarget delivers events to Slack incoming webhooks. It
|
// slackTarget delivers events to Slack incoming webhooks. It
|
||||||
// formats the event into a Slack message and posts it as
|
// formats the event into a Slack message and posts it as
|
||||||
// JSON. It shares the retry core with the HTTP target: a
|
// JSON. It shares the retry core with the HTTP target: a
|
||||||
@@ -46,19 +33,6 @@ func (t *slackTarget) Deliver(
|
|||||||
task *Task,
|
task *Task,
|
||||||
sched Scheduler,
|
sched Scheduler,
|
||||||
) {
|
) {
|
||||||
cfg, err := parseSlackConfig(d.Target.Config)
|
|
||||||
if err != nil {
|
|
||||||
t.eng.log.Error(
|
|
||||||
"invalid Slack target config",
|
|
||||||
"target_id", d.TargetID,
|
|
||||||
"error", err,
|
|
||||||
)
|
|
||||||
|
|
||||||
t.failConfig(webhookDB, d, err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
msg := FormatSlackMessage(&d.Event)
|
msg := FormatSlackMessage(&d.Event)
|
||||||
|
|
||||||
payload, err := json.Marshal(
|
payload, err := json.Marshal(
|
||||||
@@ -71,13 +45,26 @@ func (t *slackTarget) Deliver(
|
|||||||
"error", err,
|
"error", err,
|
||||||
)
|
)
|
||||||
|
|
||||||
t.failConfig(webhookDB, d, err)
|
recErr := t.eng.recordResult(
|
||||||
|
webhookDB, d, 1,
|
||||||
|
false, 0, "", err.Error(), 0,
|
||||||
|
)
|
||||||
|
if recErr != nil {
|
||||||
|
t.eng.bookkeepingFailed(d, recErr)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
t.eng.settleStatus(
|
||||||
|
webhookDB, d, d.Target.Type,
|
||||||
|
database.DeliveryStatusFailed,
|
||||||
|
)
|
||||||
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
attempt := func() attemptResult {
|
attempt := func() attemptResult {
|
||||||
return t.attempt(ctx, cfg, payload)
|
return t.attempt(ctx, d.Target.URL, payload)
|
||||||
}
|
}
|
||||||
|
|
||||||
t.deliver(
|
t.deliver(
|
||||||
@@ -86,37 +73,13 @@ func (t *slackTarget) Deliver(
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// failConfig records a first-attempt failure for a delivery
|
// attempt performs a single Slack POST to targetURL and derives
|
||||||
// that could not be prepared (bad config or unmarshalable
|
// its outcome, preserving the engine's original semantics: a
|
||||||
// payload) and marks it failed.
|
|
||||||
func (t *slackTarget) failConfig(
|
|
||||||
webhookDB *gorm.DB,
|
|
||||||
d *database.Delivery,
|
|
||||||
err error,
|
|
||||||
) {
|
|
||||||
recErr := t.eng.recordResult(
|
|
||||||
webhookDB, d, 1,
|
|
||||||
false, 0, "", err.Error(), 0,
|
|
||||||
)
|
|
||||||
if recErr != nil {
|
|
||||||
t.eng.bookkeepingFailed(d, recErr)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
t.eng.settleStatus(
|
|
||||||
webhookDB, d, d.Target.Type,
|
|
||||||
database.DeliveryStatusFailed,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
// attempt performs a single Slack POST and derives its
|
|
||||||
// outcome, preserving the engine's original semantics: a
|
|
||||||
// non-2xx response records an "HTTP <code>" error string and
|
// non-2xx response records an "HTTP <code>" error string and
|
||||||
// a transport error records a "sending request" error.
|
// a transport error records a "sending request" error.
|
||||||
func (t *slackTarget) attempt(
|
func (t *slackTarget) attempt(
|
||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
cfg *SlackTargetConfig,
|
targetURL string,
|
||||||
payload []byte,
|
payload []byte,
|
||||||
) attemptResult {
|
) attemptResult {
|
||||||
start := time.Now()
|
start := time.Now()
|
||||||
@@ -124,7 +87,7 @@ func (t *slackTarget) attempt(
|
|||||||
req, err := http.NewRequestWithContext(
|
req, err := http.NewRequestWithContext(
|
||||||
ctx,
|
ctx,
|
||||||
http.MethodPost,
|
http.MethodPost,
|
||||||
cfg.WebhookURL,
|
targetURL,
|
||||||
bytes.NewReader(payload),
|
bytes.NewReader(payload),
|
||||||
)
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -186,31 +149,6 @@ func (t *slackTarget) readSlackResponse(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func parseSlackConfig(
|
|
||||||
configJSON string,
|
|
||||||
) (*SlackTargetConfig, error) {
|
|
||||||
if configJSON == "" {
|
|
||||||
return nil, errEmptyTargetConfig
|
|
||||||
}
|
|
||||||
|
|
||||||
var cfg SlackTargetConfig
|
|
||||||
|
|
||||||
err := json.Unmarshal(
|
|
||||||
[]byte(configJSON), &cfg,
|
|
||||||
)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf(
|
|
||||||
"parsing config JSON: %w", err,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
if cfg.WebhookURL == "" {
|
|
||||||
return nil, errMissingWebhookURL
|
|
||||||
}
|
|
||||||
|
|
||||||
return &cfg, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// FormatSlackMessage builds a Slack-compatible message
|
// FormatSlackMessage builds a Slack-compatible message
|
||||||
// string from a webhook event.
|
// string from a webhook event.
|
||||||
func FormatSlackMessage(
|
func FormatSlackMessage(
|
||||||
|
|||||||
@@ -45,7 +45,7 @@ func tSeedDeletedTarget(
|
|||||||
|
|
||||||
iCreateTarget(
|
iCreateTarget(
|
||||||
t, s.MainDB, targetID, s.WebhookID, name,
|
t, s.MainDB, targetID, s.WebhookID, name,
|
||||||
database.TargetTypeHTTP, iHTTPConfig(url), 5,
|
database.TargetTypeHTTP, url, 5,
|
||||||
)
|
)
|
||||||
|
|
||||||
event := iSeedEvent(
|
event := iSeedEvent(
|
||||||
@@ -122,9 +122,9 @@ func TestProcessDelivery_UnknownTargetType_RecordsWhy(
|
|||||||
)
|
)
|
||||||
|
|
||||||
target := database.Target{
|
target := database.Target{
|
||||||
Name: "mystery",
|
Name: "mystery",
|
||||||
Type: tUnknownType,
|
Type: tUnknownType,
|
||||||
Config: iHTTPConfig("http://example.com/hook"),
|
URL: "http://example.com/hook",
|
||||||
}
|
}
|
||||||
target.ID = targetID
|
target.ID = targetID
|
||||||
|
|
||||||
@@ -140,7 +140,7 @@ func TestProcessDelivery_UnknownTargetType_RecordsWhy(
|
|||||||
body := event.Body
|
body := event.Body
|
||||||
task := iTask(
|
task := iTask(
|
||||||
seeded, event, s.WebhookID, targetID, "mystery",
|
seeded, event, s.WebhookID, targetID, "mystery",
|
||||||
target.Config, 0, 1, &body,
|
target.URL, 0, 1, &body,
|
||||||
)
|
)
|
||||||
task.TargetType = tUnknownType
|
task.TargetType = tUnknownType
|
||||||
|
|
||||||
@@ -346,11 +346,10 @@ func tRetryChainSetup(
|
|||||||
iCreateWebhook(t, s.MainDB, s.WebhookID, name)
|
iCreateWebhook(t, s.MainDB, s.WebhookID, name)
|
||||||
|
|
||||||
targetID := uuid.New().String()
|
targetID := uuid.New().String()
|
||||||
cfg := iHTTPConfig(ts.URL)
|
|
||||||
|
|
||||||
iCreateTarget(
|
iCreateTarget(
|
||||||
t, s.MainDB, targetID, s.WebhookID, name,
|
t, s.MainDB, targetID, s.WebhookID, name,
|
||||||
database.TargetTypeHTTP, cfg, 5,
|
database.TargetTypeHTTP, ts.URL, 5,
|
||||||
)
|
)
|
||||||
|
|
||||||
event := iSeedEvent(
|
event := iSeedEvent(
|
||||||
@@ -367,7 +366,7 @@ func tRetryChainSetup(
|
|||||||
body := event.Body
|
body := event.Body
|
||||||
|
|
||||||
return iTask(
|
return iTask(
|
||||||
d, event, s.WebhookID, targetID, name, cfg, 5, 2, &body,
|
d, event, s.WebhookID, targetID, name, ts.URL, 5, 2, &body,
|
||||||
), targetID
|
), targetID
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -528,7 +527,7 @@ func TestRecoverSingleRetry_TargetUnreadable_LeavesDeliveryAlone(
|
|||||||
iCreateTarget(
|
iCreateTarget(
|
||||||
t, s.MainDB, targetID, s.WebhookID, "healthy",
|
t, s.MainDB, targetID, s.WebhookID, "healthy",
|
||||||
database.TargetTypeHTTP,
|
database.TargetTypeHTTP,
|
||||||
iHTTPConfig("http://example.com/hook"), 5,
|
"http://example.com/hook", 5,
|
||||||
)
|
)
|
||||||
|
|
||||||
event := iSeedEvent(
|
event := iSeedEvent(
|
||||||
|
|||||||
@@ -2,7 +2,6 @@ package delivery_test
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"testing"
|
"testing"
|
||||||
@@ -71,13 +70,6 @@ func deliverSlackTo(
|
|||||||
e := testEngine(t, 1)
|
e := testEngine(t, 1)
|
||||||
targetID := uuid.New().String()
|
targetID := uuid.New().String()
|
||||||
|
|
||||||
slackCfg, err := json.Marshal(
|
|
||||||
delivery.SlackTargetConfig{
|
|
||||||
WebhookURL: webhookURL,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
event := seedEvent(t, db, `{"test":true}`)
|
event := seedEvent(t, db, `{"test":true}`)
|
||||||
|
|
||||||
dlv := seedDelivery(
|
dlv := seedDelivery(
|
||||||
@@ -87,7 +79,7 @@ func deliverSlackTo(
|
|||||||
|
|
||||||
d := buildSlackDelivery(
|
d := buildSlackDelivery(
|
||||||
dlv, event, targetID,
|
dlv, event, targetID,
|
||||||
"test-slack-mask", string(slackCfg),
|
"test-slack-mask", webhookURL,
|
||||||
)
|
)
|
||||||
|
|
||||||
e.ExportDeliverSlack(context.TODO(), db, d)
|
e.ExportDeliverSlack(context.TODO(), db, d)
|
||||||
@@ -160,13 +152,9 @@ func TestDoHTTPRequest_TransportErrorMasksURL(t *testing.T) {
|
|||||||
|
|
||||||
e := testEngine(t, 1)
|
e := testEngine(t, 1)
|
||||||
|
|
||||||
cfg, err := e.ExportParseHTTPConfig(
|
|
||||||
newHTTPTargetConfig(host + maskSecretPath),
|
|
||||||
)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
statusCode, _, _, reqErr := e.ExportDoHTTPRequest(
|
statusCode, _, _, reqErr := e.ExportDoHTTPRequest(
|
||||||
context.TODO(), cfg,
|
context.TODO(),
|
||||||
|
&database.Target{URL: host + maskSecretPath},
|
||||||
&database.Event{Body: `{"test":true}`},
|
&database.Event{Body: `{"test":true}`},
|
||||||
)
|
)
|
||||||
require.Error(t, reqErr)
|
require.Error(t, reqErr)
|
||||||
@@ -188,7 +176,7 @@ func TestDoHTTPRequest_UnparsableURLIsMasked(t *testing.T) {
|
|||||||
|
|
||||||
statusCode, _, _, reqErr := e.ExportDoHTTPRequest(
|
statusCode, _, _, reqErr := e.ExportDoHTTPRequest(
|
||||||
context.TODO(),
|
context.TODO(),
|
||||||
&delivery.HTTPTargetConfig{
|
&database.Target{
|
||||||
URL: "https://hooks.example.com" + maskSecretPath + "\n",
|
URL: "https://hooks.example.com" + maskSecretPath + "\n",
|
||||||
},
|
},
|
||||||
&database.Event{},
|
&database.Event{},
|
||||||
|
|||||||
@@ -2,7 +2,6 @@ package delivery_test
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
@@ -59,9 +58,9 @@ func TestUserAgent_IsTheBuildVersion(t *testing.T) {
|
|||||||
|
|
||||||
statusCode, _, _, err := e.ExportDoHTTPRequest(
|
statusCode, _, _, err := e.ExportDoHTTPRequest(
|
||||||
context.Background(),
|
context.Background(),
|
||||||
&delivery.HTTPTargetConfig{
|
&database.Target{
|
||||||
URL: ts.URL,
|
URL: ts.URL,
|
||||||
Headers: map[string]string{"User-Agent": "configured/1"},
|
Headers: map[string]string{headerUserAgent: "configured/1"},
|
||||||
},
|
},
|
||||||
&database.Event{Headers: `{"User-Agent":["curl/8"]}`},
|
&database.Event{Headers: `{"User-Agent":["curl/8"]}`},
|
||||||
)
|
)
|
||||||
@@ -73,18 +72,13 @@ func TestUserAgent_IsTheBuildVersion(t *testing.T) {
|
|||||||
db := testWebhookDB(t)
|
db := testWebhookDB(t)
|
||||||
targetID := uuid.New().String()
|
targetID := uuid.New().String()
|
||||||
|
|
||||||
slackCfg, err := json.Marshal(
|
|
||||||
delivery.SlackTargetConfig{WebhookURL: ts.URL},
|
|
||||||
)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
event := seedEvent(t, db, `{"action":"test"}`)
|
event := seedEvent(t, db, `{"action":"test"}`)
|
||||||
dlv := seedDelivery(
|
dlv := seedDelivery(
|
||||||
t, db, event.ID, targetID, database.DeliveryStatusPending,
|
t, db, event.ID, targetID, database.DeliveryStatusPending,
|
||||||
)
|
)
|
||||||
|
|
||||||
e.ExportDeliverSlack(context.Background(), db, buildSlackDelivery(
|
e.ExportDeliverSlack(context.Background(), db, buildSlackDelivery(
|
||||||
dlv, event, targetID, "test-slack", string(slackCfg),
|
dlv, event, targetID, "test-slack", ts.URL,
|
||||||
))
|
))
|
||||||
require.Len(t, userAgents, 1, "the slack target sent no request")
|
require.Len(t, userAgents, 1, "the slack target sent no request")
|
||||||
assert.Equal(t, want, <-userAgents, "slack target")
|
assert.Equal(t, want, <-userAgents, "slack target")
|
||||||
|
|||||||
@@ -1,8 +1,14 @@
|
|||||||
package handlers
|
package handlers
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strconv"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
// archiveExpiryNever is the archive expiry that keeps archived
|
// archiveExpiryNever is the archive expiry that keeps archived
|
||||||
// events forever. A stored empty expiry means the same.
|
// events forever. It is stored as an ArchiveExpiry of 0, and an
|
||||||
|
// empty expiry on a form means the same.
|
||||||
archiveExpiryNever = "never"
|
archiveExpiryNever = "never"
|
||||||
|
|
||||||
// tmplKeyArchiveExpiryChoices is the template data key for the
|
// tmplKeyArchiveExpiryChoices is the template data key for the
|
||||||
@@ -56,3 +62,19 @@ func archiveExpiryOptions(expiry string) []archiveChoice {
|
|||||||
|
|
||||||
return append([]archiveChoice{own}, options...)
|
return append([]archiveChoice{own}, options...)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// formatArchiveExpiry writes a stored ArchiveExpiry the way the
|
||||||
|
// expiry select spells its values, so the target edit form starts on
|
||||||
|
// the stored choice: never for 0, a whole number of hours as "720h",
|
||||||
|
// and anything else as Go writes a duration, which the form accepts
|
||||||
|
// back unchanged.
|
||||||
|
func formatArchiveExpiry(expiry time.Duration) string {
|
||||||
|
switch {
|
||||||
|
case expiry == 0:
|
||||||
|
return archiveExpiryNever
|
||||||
|
case expiry%time.Hour == 0:
|
||||||
|
return strconv.FormatInt(int64(expiry/time.Hour), 10) + "h"
|
||||||
|
default:
|
||||||
|
return expiry.String()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
"regexp"
|
"regexp"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
@@ -94,14 +95,18 @@ func TestArchiveExpiryChoices(t *testing.T) {
|
|||||||
|
|
||||||
env := setupSourceTest(t)
|
env := setupSourceTest(t)
|
||||||
|
|
||||||
choices := []struct{ value, shown string }{
|
choices := []struct {
|
||||||
{expiryNever, expiryNever},
|
value string
|
||||||
{"1h", "1 hour"},
|
stored time.Duration
|
||||||
{"12h", "12 hours"},
|
shown string
|
||||||
{"24h", "1 day"},
|
}{
|
||||||
{"720h", "30 days"},
|
{expiryNever, 0, expiryNever},
|
||||||
{"2160h", "90 days"},
|
{"1h", time.Hour, "1 hour"},
|
||||||
{"8760h", "365 days"},
|
{"12h", 12 * time.Hour, "12 hours"},
|
||||||
|
{"24h", 24 * time.Hour, "1 day"},
|
||||||
|
{"720h", 720 * time.Hour, "30 days"},
|
||||||
|
{"2160h", 2160 * time.Hour, "90 days"},
|
||||||
|
{"8760h", 8760 * time.Hour, "365 days"},
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, choice := range choices {
|
for _, choice := range choices {
|
||||||
@@ -122,9 +127,7 @@ func TestArchiveExpiryChoices(t *testing.T) {
|
|||||||
|
|
||||||
targets := targetsForWebhook(t, env.db, webhook.ID)
|
targets := targetsForWebhook(t, env.db, webhook.ID)
|
||||||
require.Len(t, targets, 1)
|
require.Len(t, targets, 1)
|
||||||
assert.JSONEq(
|
assert.Equal(t, choice.stored, targets[0].ArchiveExpiry)
|
||||||
t, `{"expiry":"`+choice.value+`"}`, targets[0].Config,
|
|
||||||
)
|
|
||||||
|
|
||||||
assert.Equal(
|
assert.Equal(
|
||||||
t, []string{choice.shown},
|
t, []string{choice.shown},
|
||||||
@@ -138,7 +141,7 @@ func TestArchiveExpiryChoices(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// TestArchiveExpiryEditStartsOnStoredValue checks the edit form of a
|
// TestArchiveExpiryEditStartsOnStoredValue checks the edit form of a
|
||||||
// database target whose stored expiry is empty, which selects never,
|
// database target whose stored expiry is 0, which selects never,
|
||||||
// and of one whose expiry is not one of the choices, which is listed
|
// and of one whose expiry is not one of the choices, which is listed
|
||||||
// first as its own selected entry and saved unchanged.
|
// first as its own selected entry and saved unchanged.
|
||||||
func TestArchiveExpiryEditStartsOnStoredValue(t *testing.T) {
|
func TestArchiveExpiryEditStartsOnStoredValue(t *testing.T) {
|
||||||
@@ -148,7 +151,8 @@ func TestArchiveExpiryEditStartsOnStoredValue(t *testing.T) {
|
|||||||
|
|
||||||
webhook := seedWebhookWithRetention(t, env.db, 30)
|
webhook := seedWebhookWithRetention(t, env.db, 30)
|
||||||
empty := seedConfiguredTarget(
|
empty := seedConfiguredTarget(
|
||||||
t, env.db, webhook.ID, database.TargetTypeDatabase, "",
|
t, env.db, webhook.ID, database.TargetTypeDatabase,
|
||||||
|
database.Target{},
|
||||||
)
|
)
|
||||||
|
|
||||||
_, selected := expirySelected(t, env, webhook.ID, empty.ID)
|
_, selected := expirySelected(t, env, webhook.ID, empty.ID)
|
||||||
@@ -157,7 +161,7 @@ func TestArchiveExpiryEditStartsOnStoredValue(t *testing.T) {
|
|||||||
webhook = seedWebhookWithRetention(t, env.db, 30)
|
webhook = seedWebhookWithRetention(t, env.db, 30)
|
||||||
unlisted := seedConfiguredTarget(
|
unlisted := seedConfiguredTarget(
|
||||||
t, env.db, webhook.ID, database.TargetTypeDatabase,
|
t, env.db, webhook.ID, database.TargetTypeDatabase,
|
||||||
`{"expiry":"36h"}`,
|
database.Target{ArchiveExpiry: 36 * time.Hour},
|
||||||
)
|
)
|
||||||
|
|
||||||
assert.Equal(t, []string{"36 hours"}, expiryShown(t, env, webhook.ID))
|
assert.Equal(t, []string{"36 hours"}, expiryShown(t, env, webhook.ID))
|
||||||
@@ -179,7 +183,7 @@ func TestArchiveExpiryEditStartsOnStoredValue(t *testing.T) {
|
|||||||
|
|
||||||
w := submitTargetEdit(env, webhook.ID, unlisted.ID, form)
|
w := submitTargetEdit(env, webhook.ID, unlisted.ID, form)
|
||||||
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
||||||
assert.JSONEq(
|
assert.Equal(
|
||||||
t, `{"expiry":"36h"}`, storedTarget(t, env, unlisted.ID).Config,
|
t, 36*time.Hour, storedTarget(t, env, unlisted.ID).ArchiveExpiry,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -16,8 +16,12 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
)
|
)
|
||||||
|
|
||||||
// rotationNone is the archive rotation that keeps one file.
|
// The archive rotations these tests store: none keeps one file.
|
||||||
const rotationNone = "none"
|
const (
|
||||||
|
rotationNone = "none"
|
||||||
|
rotationDaily = "daily"
|
||||||
|
rotationHourly = "hourly"
|
||||||
|
)
|
||||||
|
|
||||||
// rotationShown returns the archive rotations the webhook page's
|
// rotationShown returns the archive rotations the webhook page's
|
||||||
// target list shows.
|
// target list shows.
|
||||||
@@ -41,7 +45,9 @@ func TestArchiveRotationChoices(t *testing.T) {
|
|||||||
|
|
||||||
env := setupSourceTest(t)
|
env := setupSourceTest(t)
|
||||||
|
|
||||||
for _, rotation := range []string{rotationNone, "monthly", "daily", "hourly"} {
|
for _, rotation := range []string{
|
||||||
|
rotationNone, "monthly", rotationDaily, rotationHourly,
|
||||||
|
} {
|
||||||
t.Run(rotation, func(t *testing.T) {
|
t.Run(rotation, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -60,10 +66,8 @@ func TestArchiveRotationChoices(t *testing.T) {
|
|||||||
|
|
||||||
targets := targetsForWebhook(t, env.db, webhook.ID)
|
targets := targetsForWebhook(t, env.db, webhook.ID)
|
||||||
require.Len(t, targets, 1)
|
require.Len(t, targets, 1)
|
||||||
assert.JSONEq(t,
|
assert.Equal(t, 720*time.Hour, targets[0].ArchiveExpiry)
|
||||||
`{"expiry":"720h","rotation":"`+rotation+`"}`,
|
assert.Equal(t, rotation, targets[0].ArchiveRotation)
|
||||||
targets[0].Config,
|
|
||||||
)
|
|
||||||
|
|
||||||
assert.Equal(t,
|
assert.Equal(t,
|
||||||
[]string{rotation}, rotationShown(t, env, webhook.ID))
|
[]string{rotation}, rotationShown(t, env, webhook.ID))
|
||||||
@@ -78,26 +82,37 @@ func TestArchiveRotationChoices(t *testing.T) {
|
|||||||
|
|
||||||
w = submitTargetEdit(env, webhook.ID, targets[0].ID, form)
|
w = submitTargetEdit(env, webhook.ID, targets[0].ID, form)
|
||||||
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
||||||
assert.JSONEq(t,
|
|
||||||
`{"expiry":"720h","rotation":"hourly"}`,
|
stored := storedTarget(t, env, targets[0].ID)
|
||||||
storedTarget(t, env, targets[0].ID).Config,
|
assert.Equal(t, 720*time.Hour, stored.ArchiveExpiry)
|
||||||
)
|
assert.Equal(t, rotationHourly, stored.ArchiveRotation)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestArchiveRotationEditStartsOnNone checks the edit form of a
|
// TestArchiveRotationEditStartsOnNone checks that a database target
|
||||||
// database target with no rotation stored starts on none.
|
// added with no rotation is stored with none and shown with none, and
|
||||||
|
// that its edit form starts on none.
|
||||||
func TestArchiveRotationEditStartsOnNone(t *testing.T) {
|
func TestArchiveRotationEditStartsOnNone(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupSourceTest(t)
|
env := setupSourceTest(t)
|
||||||
webhook := seedWebhookWithRetention(t, env.db, 30)
|
webhook := seedWebhookWithRetention(t, env.db, 30)
|
||||||
target := seedConfiguredTarget(
|
|
||||||
t, env.db, webhook.ID, database.TargetTypeDatabase, "",
|
|
||||||
)
|
|
||||||
|
|
||||||
page := targetEditPage(t, env, webhook.ID, target.ID)
|
form := url.Values{}
|
||||||
|
form.Set("name", "archive")
|
||||||
|
form.Set("type", string(database.TargetTypeDatabase))
|
||||||
|
|
||||||
|
w := serveTarget(
|
||||||
|
env, http.MethodPost, "/hook/"+webhook.ID+"/targets", form,
|
||||||
|
)
|
||||||
|
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
||||||
|
|
||||||
|
targets := targetsForWebhook(t, env.db, webhook.ID)
|
||||||
|
require.Len(t, targets, 1)
|
||||||
|
assert.Equal(t, rotationNone, targets[0].ArchiveRotation)
|
||||||
|
|
||||||
|
page := targetEditPage(t, env, webhook.ID, targets[0].ID)
|
||||||
assert.Equal(t, []string{rotationNone}, selectedIn(page, "rotation"))
|
assert.Equal(t, []string{rotationNone}, selectedIn(page, "rotation"))
|
||||||
assert.Equal(t, []string{rotationNone}, rotationShown(t, env, webhook.ID))
|
assert.Equal(t, []string{rotationNone}, rotationShown(t, env, webhook.ID))
|
||||||
}
|
}
|
||||||
@@ -125,7 +140,7 @@ func TestArchiveRotationRefused(t *testing.T) {
|
|||||||
|
|
||||||
target := seedConfiguredTarget(
|
target := seedConfiguredTarget(
|
||||||
t, env.db, webhook.ID, database.TargetTypeDatabase,
|
t, env.db, webhook.ID, database.TargetTypeDatabase,
|
||||||
`{"rotation":"daily"}`,
|
database.Target{ArchiveRotation: rotationDaily},
|
||||||
)
|
)
|
||||||
|
|
||||||
form.Del("type")
|
form.Del("type")
|
||||||
@@ -133,8 +148,8 @@ func TestArchiveRotationRefused(t *testing.T) {
|
|||||||
w = submitTargetEdit(env, webhook.ID, target.ID, form)
|
w = submitTargetEdit(env, webhook.ID, target.ID, form)
|
||||||
assert.Equal(t, http.StatusBadRequest, w.Code)
|
assert.Equal(t, http.StatusBadRequest, w.Code)
|
||||||
assert.Contains(t, w.Body.String(), "Invalid archive rotation")
|
assert.Contains(t, w.Body.String(), "Invalid archive rotation")
|
||||||
assert.JSONEq(t,
|
assert.Equal(
|
||||||
`{"rotation":"daily"}`, storedTarget(t, env, target.ID).Config,
|
t, rotationDaily, storedTarget(t, env, target.ID).ArchiveRotation,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -161,9 +176,8 @@ func TestHandleSourceCreateSubmit_ArchiveRotation(t *testing.T) {
|
|||||||
|
|
||||||
targets := targetsForWebhook(t, env.db, webhook.ID)
|
targets := targetsForWebhook(t, env.db, webhook.ID)
|
||||||
require.Len(t, targets, 1)
|
require.Len(t, targets, 1)
|
||||||
assert.JSONEq(t,
|
assert.Equal(t, 720*time.Hour, targets[0].ArchiveExpiry)
|
||||||
`{"expiry":"720h","rotation":"daily"}`, targets[0].Config,
|
assert.Equal(t, rotationDaily, targets[0].ArchiveRotation)
|
||||||
)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestArchiveFileView_Rotated describes a daily target's archive files
|
// TestArchiveFileView_Rotated describes a daily target's archive files
|
||||||
@@ -178,7 +192,7 @@ func TestArchiveFileView_Rotated(t *testing.T) {
|
|||||||
webhook := seedWebhookWithRetention(t, env.db, 30)
|
webhook := seedWebhookWithRetention(t, env.db, 30)
|
||||||
target := seedConfiguredTarget(
|
target := seedConfiguredTarget(
|
||||||
t, env.db, webhook.ID, database.TargetTypeDatabase,
|
t, env.db, webhook.ID, database.TargetTypeDatabase,
|
||||||
`{"rotation":"daily"}`,
|
database.Target{ArchiveRotation: rotationDaily},
|
||||||
)
|
)
|
||||||
|
|
||||||
path := archive.Path(env.dbMgr, &webhook, target)
|
path := archive.Path(env.dbMgr, &webhook, target)
|
||||||
|
|||||||
@@ -300,21 +300,26 @@ func createReplayDelivery(
|
|||||||
}
|
}
|
||||||
|
|
||||||
return delivery.Task{
|
return delivery.Task{
|
||||||
DeliveryID: dlv.ID,
|
DeliveryID: dlv.ID,
|
||||||
EventID: event.ID,
|
EventID: event.ID,
|
||||||
WebhookID: webhookID,
|
WebhookID: webhookID,
|
||||||
EntrypointID: event.EntrypointID,
|
EntrypointID: event.EntrypointID,
|
||||||
TargetID: target.ID,
|
TargetID: target.ID,
|
||||||
TargetName: target.Name,
|
TargetName: target.Name,
|
||||||
TargetType: target.Type,
|
TargetType: target.Type,
|
||||||
TargetConfig: target.Config,
|
TargetURL: target.URL,
|
||||||
MaxAttempts: target.MaxAttempts,
|
TargetHeaders: target.Headers,
|
||||||
Method: event.Method,
|
TimeoutSeconds: target.TimeoutSeconds,
|
||||||
RawQuery: event.RawQuery,
|
ForwardQuery: target.ForwardQuery,
|
||||||
Headers: event.Headers,
|
ArchiveExpiry: target.ArchiveExpiry,
|
||||||
ContentType: event.ContentType,
|
ArchiveRotation: target.ArchiveRotation,
|
||||||
Body: replayBody(event.Body),
|
MaxAttempts: target.MaxAttempts,
|
||||||
AttemptNum: 1,
|
Method: event.Method,
|
||||||
|
RawQuery: event.RawQuery,
|
||||||
|
Headers: event.Headers,
|
||||||
|
ContentType: event.ContentType,
|
||||||
|
Body: replayBody(event.Body),
|
||||||
|
AttemptNum: 1,
|
||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -192,7 +192,7 @@ func TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal(
|
|||||||
wh := seedWebhook(t, db)
|
wh := seedWebhook(t, db)
|
||||||
tgt := seedConfiguredTarget(
|
tgt := seedConfiguredTarget(
|
||||||
t, db, wh.ID, database.TargetTypeHTTP,
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
`{"url":"`+replayTargetURL+`"}`,
|
database.Target{URL: replayTargetURL},
|
||||||
)
|
)
|
||||||
|
|
||||||
event, original := seedFailedDelivery(
|
event, original := seedFailedDelivery(
|
||||||
@@ -205,13 +205,13 @@ func TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal(
|
|||||||
before := loadDelivery(t, webhookDB, original.ID)
|
before := loadDelivery(t, webhookDB, original.ID)
|
||||||
|
|
||||||
// The operator fixes the destination, which is the whole reason
|
// The operator fixes the destination, which is the whole reason
|
||||||
// to replay. The replay must use this, not the config the
|
// to replay. The replay must use this, not the URL the original
|
||||||
// original delivery ran against.
|
// delivery ran against.
|
||||||
const fixedConfig = `{"url":"http://93.184.216.34/fixed"}`
|
const fixedURL = "http://93.184.216.34/fixed"
|
||||||
|
|
||||||
require.NoError(t, db.DB().Model(&database.Target{}).
|
require.NoError(t, db.DB().Model(&database.Target{}).
|
||||||
Where("id = ?", tgt.ID).
|
Where("id = ?", tgt.ID).
|
||||||
Update("config", fixedConfig).Error)
|
Update("url", fixedURL).Error)
|
||||||
|
|
||||||
w := postReplay(t, h, sess, wh.ID, original.ID)
|
w := postReplay(t, h, sess, wh.ID, original.ID)
|
||||||
|
|
||||||
@@ -240,7 +240,7 @@ func TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal(
|
|||||||
tasks := notif.Tasks()
|
tasks := notif.Tasks()
|
||||||
require.Len(t, tasks, 1)
|
require.Len(t, tasks, 1)
|
||||||
assertReplayTask(
|
assertReplayTask(
|
||||||
t, tasks[0], wh.ID, event, tgt, replayed.ID, fixedConfig,
|
t, tasks[0], wh.ID, event, tgt, replayed.ID, fixedURL,
|
||||||
)
|
)
|
||||||
assertNoLeakedTarget(t, webhookDB)
|
assertNoLeakedTarget(t, webhookDB)
|
||||||
}
|
}
|
||||||
@@ -277,15 +277,15 @@ func assertDeliveryUntouched(
|
|||||||
|
|
||||||
// assertReplayTask proves the task handed to the delivery engine is
|
// assertReplayTask proves the task handed to the delivery engine is
|
||||||
// the one the receiver would build for this event and this target, and
|
// the one the receiver would build for this event and this target, and
|
||||||
// that it carries wantConfig — the target's configuration as it stands
|
// that it carries wantURL — the target's URL as it stands now rather
|
||||||
// now rather than as the original delivery ran against it.
|
// than as the original delivery ran against it.
|
||||||
func assertReplayTask(
|
func assertReplayTask(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
task delivery.Task,
|
task delivery.Task,
|
||||||
webhookID string,
|
webhookID string,
|
||||||
event *database.Event,
|
event *database.Event,
|
||||||
target *database.Target,
|
target *database.Target,
|
||||||
wantDeliveryID, wantConfig string,
|
wantDeliveryID, wantURL string,
|
||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
@@ -295,9 +295,9 @@ func assertReplayTask(
|
|||||||
assert.Equal(t, event.EntrypointID, task.EntrypointID)
|
assert.Equal(t, event.EntrypointID, task.EntrypointID)
|
||||||
assert.Equal(t, target.ID, task.TargetID)
|
assert.Equal(t, target.ID, task.TargetID)
|
||||||
assert.Equal(t, target.Type, task.TargetType)
|
assert.Equal(t, target.Type, task.TargetType)
|
||||||
assert.JSONEq(
|
assert.Equal(
|
||||||
t, wantConfig, task.TargetConfig,
|
t, wantURL, task.TargetURL,
|
||||||
"replay must use the target's current configuration",
|
"replay must use the target's current URL",
|
||||||
)
|
)
|
||||||
assert.Equal(t, event.Method, task.Method)
|
assert.Equal(t, event.Method, task.Method)
|
||||||
assert.Equal(
|
assert.Equal(
|
||||||
@@ -327,6 +327,43 @@ func assertNoLeakedTarget(t *testing.T, webhookDB *gorm.DB) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestHandleDeliveryReplay_TaskCarriesEveryTargetSetting proves the
|
||||||
|
// task a replay hands the delivery engine carries every one of the
|
||||||
|
// target's settings.
|
||||||
|
func TestHandleDeliveryReplay_TaskCarriesEveryTargetSetting(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
sess *session.Session
|
||||||
|
db *database.Database
|
||||||
|
dbMgr *database.WebhookDBManager
|
||||||
|
notif *recordingNotifier
|
||||||
|
)
|
||||||
|
|
||||||
|
app := newTestApp(t, &h, &sess, &db, &dbMgr, ¬if)
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
|
wh := seedWebhook(t, db)
|
||||||
|
tgt := seedConfiguredTarget(
|
||||||
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
|
targetWithEverySetting(),
|
||||||
|
)
|
||||||
|
|
||||||
|
_, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
|
||||||
|
|
||||||
|
w := postReplay(t, h, sess, wh.ID, original.ID)
|
||||||
|
require.Equal(t, http.StatusSeeOther, w.Code)
|
||||||
|
|
||||||
|
tasks := notif.Tasks()
|
||||||
|
require.Len(t, tasks, 1)
|
||||||
|
assertTaskCarriesTargetSettings(t, tasks[0], tgt)
|
||||||
|
}
|
||||||
|
|
||||||
// TestHandleDeliveryReplay_RefusesDeletedTarget proves the required
|
// TestHandleDeliveryReplay_RefusesDeletedTarget proves the required
|
||||||
// refusal: a target deleted since the delivery ran is reported as
|
// refusal: a target deleted since the delivery ran is reported as
|
||||||
// deleted rather than erroring, and nothing is created or queued.
|
// deleted rather than erroring, and nothing is created or queued.
|
||||||
@@ -349,7 +386,7 @@ func TestHandleDeliveryReplay_RefusesDeletedTarget(t *testing.T) {
|
|||||||
wh := seedWebhook(t, db)
|
wh := seedWebhook(t, db)
|
||||||
tgt := seedConfiguredTarget(
|
tgt := seedConfiguredTarget(
|
||||||
t, db, wh.ID, database.TargetTypeHTTP,
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
`{"url":"`+replayTargetURL+`"}`,
|
database.Target{URL: replayTargetURL},
|
||||||
)
|
)
|
||||||
|
|
||||||
event, original := seedFailedDelivery(
|
event, original := seedFailedDelivery(
|
||||||
@@ -424,7 +461,7 @@ func TestHandleDeliveryReplay_RefusesWhileEarlierReplayInFlight(
|
|||||||
wh := seedWebhook(t, db)
|
wh := seedWebhook(t, db)
|
||||||
tgt := seedConfiguredTarget(
|
tgt := seedConfiguredTarget(
|
||||||
t, db, wh.ID, database.TargetTypeHTTP,
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
`{"url":"`+replayTargetURL+`"}`,
|
database.Target{URL: replayTargetURL},
|
||||||
)
|
)
|
||||||
|
|
||||||
event, original := seedFailedDelivery(
|
event, original := seedFailedDelivery(
|
||||||
@@ -498,7 +535,7 @@ func TestHandleDeliveryReplay_ReturnsToTheListItWasPressedIn(
|
|||||||
wh := seedWebhook(t, db)
|
wh := seedWebhook(t, db)
|
||||||
tgt := seedConfiguredTarget(
|
tgt := seedConfiguredTarget(
|
||||||
t, db, wh.ID, database.TargetTypeHTTP,
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
`{"url":"`+replayTargetURL+`"}`,
|
database.Target{URL: replayTargetURL},
|
||||||
)
|
)
|
||||||
|
|
||||||
_, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
|
_, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
|
||||||
@@ -556,7 +593,7 @@ func TestHandleSourceLogs_RendersReplayControlAndBanner(t *testing.T) {
|
|||||||
wh := seedWebhook(t, db)
|
wh := seedWebhook(t, db)
|
||||||
tgt := seedConfiguredTarget(
|
tgt := seedConfiguredTarget(
|
||||||
t, db, wh.ID, database.TargetTypeHTTP,
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
`{"url":"`+replayTargetURL+`"}`,
|
database.Target{URL: replayTargetURL},
|
||||||
)
|
)
|
||||||
|
|
||||||
_, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
|
_, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
|
||||||
@@ -615,7 +652,7 @@ func TestHandleDeliveryReplay_LabelsTheReplay(t *testing.T) {
|
|||||||
wh := seedWebhook(t, db)
|
wh := seedWebhook(t, db)
|
||||||
tgt := seedConfiguredTarget(
|
tgt := seedConfiguredTarget(
|
||||||
t, db, wh.ID, database.TargetTypeHTTP,
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
`{"url":"`+replayTargetURL+`"}`,
|
database.Target{URL: replayTargetURL},
|
||||||
)
|
)
|
||||||
|
|
||||||
event, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
|
event, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
|
||||||
|
|||||||
@@ -86,12 +86,12 @@ func seedFailedDeliveryWithResponse(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// seedFailureAndRender seeds a failed delivery against a
|
// seedFailureAndRender seeds a failed delivery against a
|
||||||
// target of the given type and config, and returns the
|
// target of the given type and URL, and returns the
|
||||||
// rendered event log page.
|
// rendered event log page.
|
||||||
func seedFailureAndRender(
|
func seedFailureAndRender(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
targetType database.TargetType,
|
targetType database.TargetType,
|
||||||
config, responseBody string,
|
targetURL, responseBody string,
|
||||||
) string {
|
) string {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
@@ -109,7 +109,7 @@ func seedFailureAndRender(
|
|||||||
|
|
||||||
wh := seedWebhook(t, db)
|
wh := seedWebhook(t, db)
|
||||||
tgt := seedConfiguredTarget(
|
tgt := seedConfiguredTarget(
|
||||||
t, db, wh.ID, targetType, config,
|
t, db, wh.ID, targetType, database.Target{URL: targetURL},
|
||||||
)
|
)
|
||||||
|
|
||||||
seedFailedDeliveryWithResponse(
|
seedFailedDeliveryWithResponse(
|
||||||
@@ -129,7 +129,7 @@ func TestHandleSourceLogs_RendersFailedAttempt(t *testing.T) {
|
|||||||
body := seedFailureAndRender(
|
body := seedFailureAndRender(
|
||||||
t,
|
t,
|
||||||
database.TargetTypeHTTP,
|
database.TargetTypeHTTP,
|
||||||
`{"url":"https://example.com/hook/abc"}`,
|
"https://example.com/hook/abc",
|
||||||
"upstream exploded",
|
"upstream exploded",
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -167,7 +167,7 @@ func TestHandleSourceLogs_EscapesResponseBody(t *testing.T) {
|
|||||||
body := seedFailureAndRender(
|
body := seedFailureAndRender(
|
||||||
t,
|
t,
|
||||||
database.TargetTypeHTTP,
|
database.TargetTypeHTTP,
|
||||||
`{"url":"https://example.com/hook/abc"}`,
|
"https://example.com/hook/abc",
|
||||||
payload,
|
payload,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -189,7 +189,7 @@ func TestHandleSourceLogs_RedactsCredentialEchoedInResponse(
|
|||||||
body := seedFailureAndRender(
|
body := seedFailureAndRender(
|
||||||
t,
|
t,
|
||||||
database.TargetTypeSlack,
|
database.TargetTypeSlack,
|
||||||
`{"webhookUrl":"`+slackWebhookURL+`"}`,
|
slackWebhookURL,
|
||||||
"no_service: "+slackWebhookURL,
|
"no_service: "+slackWebhookURL,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -228,7 +228,7 @@ func TestHandleSourceLogs_RedactsCredentialEchoedInError(
|
|||||||
tgt := seedConfiguredTarget(
|
tgt := seedConfiguredTarget(
|
||||||
t, db, wh.ID,
|
t, db, wh.ID,
|
||||||
database.TargetTypeSlack,
|
database.TargetTypeSlack,
|
||||||
`{"webhookUrl":"`+slackWebhookURL+`"}`,
|
database.Target{URL: slackWebhookURL},
|
||||||
)
|
)
|
||||||
|
|
||||||
dlv := seedFailedDeliveryWithResponse(t, dbMgr, wh.ID, tgt.ID, "")
|
dlv := seedFailedDeliveryWithResponse(t, dbMgr, wh.ID, tgt.ID, "")
|
||||||
@@ -300,7 +300,7 @@ func TestHandleSourceLogs_RedactsCredentialSeveredByTheEngineCut(
|
|||||||
body := seedFailureAndRender(
|
body := seedFailureAndRender(
|
||||||
t,
|
t,
|
||||||
database.TargetTypeSlack,
|
database.TargetTypeSlack,
|
||||||
`{"webhookUrl":"`+slackWebhookURL+`"}`,
|
slackWebhookURL,
|
||||||
stored,
|
stored,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -335,7 +335,7 @@ func TestHandleSourceLogs_RedactsCredentialSeveredBySQLCut(
|
|||||||
body := seedFailureAndRender(
|
body := seedFailureAndRender(
|
||||||
t,
|
t,
|
||||||
database.TargetTypeSlack,
|
database.TargetTypeSlack,
|
||||||
`{"webhookUrl":"`+slackWebhookURL+`"}`,
|
slackWebhookURL,
|
||||||
stored,
|
stored,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -373,7 +373,7 @@ func TestHandleSourceLogs_RedactsForSoftDeletedTarget(
|
|||||||
tgt := seedConfiguredTarget(
|
tgt := seedConfiguredTarget(
|
||||||
t, db, wh.ID,
|
t, db, wh.ID,
|
||||||
database.TargetTypeSlack,
|
database.TargetTypeSlack,
|
||||||
`{"webhookUrl":"`+slackWebhookURL+`"}`,
|
database.Target{URL: slackWebhookURL},
|
||||||
)
|
)
|
||||||
|
|
||||||
seedFailedDeliveryWithResponse(
|
seedFailedDeliveryWithResponse(
|
||||||
@@ -413,7 +413,7 @@ func TestHandleSourceLogs_BoundsRenderedAttempts(t *testing.T) {
|
|||||||
|
|
||||||
wh := seedWebhook(t, db)
|
wh := seedWebhook(t, db)
|
||||||
tgt := seedConfiguredTarget(
|
tgt := seedConfiguredTarget(
|
||||||
t, db, wh.ID, database.TargetTypeLog, "",
|
t, db, wh.ID, database.TargetTypeLog, database.Target{},
|
||||||
)
|
)
|
||||||
|
|
||||||
dlv := seedFailedDeliveryWithResponse(t, dbMgr, wh.ID, tgt.ID, "")
|
dlv := seedFailedDeliveryWithResponse(t, dbMgr, wh.ID, tgt.ID, "")
|
||||||
@@ -480,7 +480,7 @@ func TestHandleSourceLogs_BoundsOversizeResponse(t *testing.T) {
|
|||||||
|
|
||||||
wh := seedWebhook(t, db)
|
wh := seedWebhook(t, db)
|
||||||
tgt := seedConfiguredTarget(
|
tgt := seedConfiguredTarget(
|
||||||
t, db, wh.ID, database.TargetTypeLog, "",
|
t, db, wh.ID, database.TargetTypeLog, database.Target{},
|
||||||
)
|
)
|
||||||
|
|
||||||
stored := strings.Repeat("A", responseCap*4) + tail
|
stored := strings.Repeat("A", responseCap*4) + tail
|
||||||
|
|||||||
@@ -13,8 +13,8 @@ import (
|
|||||||
|
|
||||||
// DeliveryView is the display-safe projection of a delivery
|
// DeliveryView is the display-safe projection of a delivery
|
||||||
// for the event log page. Its target is a TargetView, so the
|
// for the event log page. Its target is a TargetView, so the
|
||||||
// stored configuration blob — which holds the target's
|
// target's URL and headers — which hold the target's
|
||||||
// credential — has no path to the template.
|
// credential — have no path to the template.
|
||||||
type DeliveryView struct {
|
type DeliveryView struct {
|
||||||
ID string
|
ID string
|
||||||
Status database.DeliveryStatus
|
Status database.DeliveryStatus
|
||||||
@@ -126,7 +126,7 @@ func (h *Handlers) HandleSourceLogs() http.HandlerFunc {
|
|||||||
// loadTargetMap loads targets into a map of display-safe
|
// loadTargetMap loads targets into a map of display-safe
|
||||||
// views keyed by target ID, each paired with its redactor.
|
// views keyed by target ID, each paired with its redactor.
|
||||||
// The projection happens here so that no caller can hand a
|
// The projection happens here so that no caller can hand a
|
||||||
// raw target, configuration blob and all, to a template: the
|
// raw target, URL and headers and all, to a template: the
|
||||||
// raw rows do not leave this function.
|
// raw rows do not leave this function.
|
||||||
//
|
//
|
||||||
// The load is Unscoped because deleting a target only soft
|
// The load is Unscoped because deleting a target only soft
|
||||||
|
|||||||
@@ -151,7 +151,7 @@ func TestHandleEventResubmit_DeliversToTargetCreatedAfterTheEvent(
|
|||||||
// captured. It has no prior delivery of anything.
|
// captured. It has no prior delivery of anything.
|
||||||
tgt := seedConfiguredTarget(
|
tgt := seedConfiguredTarget(
|
||||||
t, db, wh.ID, database.TargetTypeHTTP,
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
`{"url":"`+resubmitTargetURL+`"}`,
|
database.Target{URL: resubmitTargetURL},
|
||||||
)
|
)
|
||||||
|
|
||||||
w := postResubmit(t, h, sess, wh.ID, original.ID)
|
w := postResubmit(t, h, sess, wh.ID, original.ID)
|
||||||
@@ -274,7 +274,7 @@ func TestHandleEventResubmit_IsRepeatable(t *testing.T) {
|
|||||||
wh := seedWebhook(t, db)
|
wh := seedWebhook(t, db)
|
||||||
tgt := seedConfiguredTarget(
|
tgt := seedConfiguredTarget(
|
||||||
t, db, wh.ID, database.TargetTypeHTTP,
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
`{"url":"`+resubmitTargetURL+`"}`,
|
database.Target{URL: resubmitTargetURL},
|
||||||
)
|
)
|
||||||
original := seedStoredEvent(t, dbMgr, wh.ID, `{"fire":"again"}`)
|
original := seedStoredEvent(t, dbMgr, wh.ID, `{"fire":"again"}`)
|
||||||
|
|
||||||
@@ -368,7 +368,7 @@ func TestHandleEventResubmit_OversizeBodySurvivesIntact(t *testing.T) {
|
|||||||
wh := seedWebhook(t, db)
|
wh := seedWebhook(t, db)
|
||||||
seedConfiguredTarget(
|
seedConfiguredTarget(
|
||||||
t, db, wh.ID, database.TargetTypeHTTP,
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
`{"url":"`+resubmitTargetURL+`"}`,
|
database.Target{URL: resubmitTargetURL},
|
||||||
)
|
)
|
||||||
original := seedStoredEvent(t, dbMgr, wh.ID, stored)
|
original := seedStoredEvent(t, dbMgr, wh.ID, stored)
|
||||||
|
|
||||||
@@ -425,11 +425,11 @@ func TestHandleEventResubmit_SkipsInactiveTarget(t *testing.T) {
|
|||||||
wh := seedWebhook(t, db)
|
wh := seedWebhook(t, db)
|
||||||
active := seedConfiguredTarget(
|
active := seedConfiguredTarget(
|
||||||
t, db, wh.ID, database.TargetTypeHTTP,
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
`{"url":"`+resubmitTargetURL+`"}`,
|
database.Target{URL: resubmitTargetURL},
|
||||||
)
|
)
|
||||||
off := seedConfiguredTarget(
|
off := seedConfiguredTarget(
|
||||||
t, db, wh.ID, database.TargetTypeHTTP,
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
`{"url":"`+resubmitTargetURL+`/off"}`,
|
database.Target{URL: resubmitTargetURL + "/off"},
|
||||||
)
|
)
|
||||||
|
|
||||||
require.NoError(t, db.DB().Model(&database.Target{}).
|
require.NoError(t, db.DB().Model(&database.Target{}).
|
||||||
@@ -528,7 +528,7 @@ func TestHandleEventResubmit_RefusesEventOfAnotherWebhook(
|
|||||||
mine := seedWebhook(t, db)
|
mine := seedWebhook(t, db)
|
||||||
seedConfiguredTarget(
|
seedConfiguredTarget(
|
||||||
t, db, mine.ID, database.TargetTypeHTTP,
|
t, db, mine.ID, database.TargetTypeHTTP,
|
||||||
`{"url":"`+resubmitTargetURL+`"}`,
|
database.Target{URL: resubmitTargetURL},
|
||||||
)
|
)
|
||||||
seedStoredEvent(t, dbMgr, mine.ID, `{"mine":true}`)
|
seedStoredEvent(t, dbMgr, mine.ID, `{"mine":true}`)
|
||||||
|
|
||||||
@@ -575,7 +575,7 @@ func TestHandleSourceLogs_ShowsResubmitProvenance(t *testing.T) {
|
|||||||
wh := seedWebhook(t, db)
|
wh := seedWebhook(t, db)
|
||||||
seedConfiguredTarget(
|
seedConfiguredTarget(
|
||||||
t, db, wh.ID, database.TargetTypeHTTP,
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
`{"url":"`+resubmitTargetURL+`"}`,
|
database.Target{URL: resubmitTargetURL},
|
||||||
)
|
)
|
||||||
original := seedStoredEvent(t, dbMgr, wh.ID, `{"trace":"me"}`)
|
original := seedStoredEvent(t, dbMgr, wh.ID, `{"trace":"me"}`)
|
||||||
|
|
||||||
|
|||||||
@@ -133,36 +133,28 @@ func (s *Handlers) RenderTemplateForTest(
|
|||||||
s.renderTemplate(w, r, pageTemplate, data)
|
s.renderTemplate(w, r, pageTemplate, data)
|
||||||
}
|
}
|
||||||
|
|
||||||
// BuildSlackTargetConfigForTest exposes
|
// SetSlackTargetSettingsForTest exposes setSlackTargetSettings for use
|
||||||
// buildSlackTargetConfig for use in the handlers_test package.
|
// in the handlers_test package.
|
||||||
func (s *Handlers) BuildSlackTargetConfigForTest(
|
func (s *Handlers) SetSlackTargetSettingsForTest(
|
||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
|
target *database.Target,
|
||||||
targetURL string,
|
targetURL string,
|
||||||
) (string, string, error) {
|
) string {
|
||||||
return s.buildSlackTargetConfig(ctx, targetURL)
|
return s.setSlackTargetSettings(ctx, target, targetURL)
|
||||||
}
|
}
|
||||||
|
|
||||||
// BuildHTTPTargetConfigForTest exposes buildHTTPTargetConfig
|
// SetDatabaseTargetSettingsForTest exposes setDatabaseTargetSettings
|
||||||
// for use in the handlers_test package, taking the form fields
|
// for use in the handlers_test package.
|
||||||
// an HTTP target's configuration is built from.
|
func SetDatabaseTargetSettingsForTest(
|
||||||
func (s *Handlers) BuildHTTPTargetConfigForTest(
|
target *database.Target,
|
||||||
ctx context.Context,
|
|
||||||
targetURL, headers, timeout string,
|
|
||||||
) (string, string, error) {
|
|
||||||
return s.buildHTTPTargetConfig(ctx, targetFormInput{
|
|
||||||
URL: targetURL,
|
|
||||||
Headers: headers,
|
|
||||||
Timeout: timeout,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// BuildDatabaseTargetConfigForTest exposes
|
|
||||||
// buildDatabaseTargetConfig for use in the handlers_test
|
|
||||||
// package.
|
|
||||||
func BuildDatabaseTargetConfigForTest(
|
|
||||||
expiry, rotation string,
|
expiry, rotation string,
|
||||||
) (string, string, error) {
|
) string {
|
||||||
return buildDatabaseTargetConfig(expiry, rotation)
|
return setDatabaseTargetSettings(target, expiry, rotation)
|
||||||
|
}
|
||||||
|
|
||||||
|
// FormatArchiveExpiryForTest exposes formatArchiveExpiry.
|
||||||
|
func FormatArchiveExpiryForTest(expiry time.Duration) string {
|
||||||
|
return formatArchiveExpiry(expiry)
|
||||||
}
|
}
|
||||||
|
|
||||||
// ArchiveFileViewForTest exposes archiveFileView, which describes a
|
// ArchiveFileViewForTest exposes archiveFileView, which describes a
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import (
|
|||||||
"log/slog"
|
"log/slog"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
|
"strconv"
|
||||||
"sync"
|
"sync"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
@@ -354,7 +355,7 @@ func TestHandleIndex_Authenticated(t *testing.T) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestBuildSlackTargetConfig_AcceptsPublicURL(t *testing.T) {
|
func TestSetSlackTargetSettings_AcceptsPublicURL(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
var h *handlers.Handlers
|
var h *handlers.Handlers
|
||||||
@@ -364,16 +365,17 @@ func TestBuildSlackTargetConfig_AcceptsPublicURL(t *testing.T) {
|
|||||||
|
|
||||||
t.Cleanup(app.RequireStop)
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
cfg, errMsg, err := h.BuildSlackTargetConfigForTest(
|
const hookURL = "http://93.184.216.34/services/T00/B00/xxx"
|
||||||
t.Context(), "http://93.184.216.34/services/T00/B00/xxx",
|
|
||||||
)
|
var target database.Target
|
||||||
|
|
||||||
|
errMsg := h.SetSlackTargetSettingsForTest(t.Context(), &target, hookURL)
|
||||||
|
|
||||||
require.NoError(t, err)
|
|
||||||
assert.Empty(t, errMsg)
|
assert.Empty(t, errMsg)
|
||||||
assert.Contains(t, cfg, "webhookUrl")
|
assert.Equal(t, hookURL, target.URL)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestBuildSlackTargetConfig_RejectsReservedURL(t *testing.T) {
|
func TestSetSlackTargetSettings_RejectsReservedURL(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
var h *handlers.Handlers
|
var h *handlers.Handlers
|
||||||
@@ -383,13 +385,14 @@ func TestBuildSlackTargetConfig_RejectsReservedURL(t *testing.T) {
|
|||||||
|
|
||||||
t.Cleanup(app.RequireStop)
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
cfg, errMsg, err := h.BuildSlackTargetConfigForTest(
|
var target database.Target
|
||||||
t.Context(), "http://169.254.169.254/latest/meta-data/",
|
|
||||||
|
errMsg := h.SetSlackTargetSettingsForTest(
|
||||||
|
t.Context(), &target, "http://169.254.169.254/latest/meta-data/",
|
||||||
)
|
)
|
||||||
|
|
||||||
require.NoError(t, err)
|
|
||||||
assert.Contains(t, errMsg, "Invalid target URL")
|
assert.Contains(t, errMsg, "Invalid target URL")
|
||||||
assert.Empty(t, cfg)
|
assert.Empty(t, target.URL)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRenderTemplate(t *testing.T) {
|
func TestRenderTemplate(t *testing.T) {
|
||||||
@@ -483,72 +486,100 @@ func TestRenderTemplateMidRenderErrorSendsNoPartialBody(t *testing.T) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestBuildDatabaseTargetConfig_Valid(t *testing.T) {
|
func TestSetDatabaseTargetSettings_Valid(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
// Empty expiry and rotation: the keep-forever, one-file default,
|
cases := []struct {
|
||||||
// empty config.
|
expiry, rotation string
|
||||||
cfg, errMsg, err := handlers.BuildDatabaseTargetConfigForTest("", "")
|
wantExpiry time.Duration
|
||||||
require.NoError(t, err)
|
wantRotation string
|
||||||
assert.Empty(t, errMsg)
|
}{
|
||||||
assert.Empty(t, cfg)
|
// Empty expiry and rotation: the keep-forever, one-file
|
||||||
|
// default.
|
||||||
|
{"", "", 0, rotationNone},
|
||||||
|
{expiryNever, "", 0, rotationNone},
|
||||||
|
{" 36h ", "", 36 * time.Hour, rotationNone},
|
||||||
|
{"", rotationDaily, 0, rotationDaily},
|
||||||
|
{"90m", rotationHourly, 90 * time.Minute, rotationHourly},
|
||||||
|
}
|
||||||
|
|
||||||
// Explicit never is stored as config.
|
for _, tc := range cases {
|
||||||
cfg, errMsg, err = handlers.BuildDatabaseTargetConfigForTest("never", "")
|
var target database.Target
|
||||||
require.NoError(t, err)
|
|
||||||
assert.Empty(t, errMsg)
|
|
||||||
assert.JSONEq(t, `{"expiry":"never"}`, cfg)
|
|
||||||
|
|
||||||
// A positive duration is stored as config.
|
errMsg := handlers.SetDatabaseTargetSettingsForTest(
|
||||||
cfg, errMsg, err = handlers.BuildDatabaseTargetConfigForTest("720h", "")
|
&target, tc.expiry, tc.rotation,
|
||||||
require.NoError(t, err)
|
)
|
||||||
assert.Empty(t, errMsg)
|
|
||||||
assert.JSONEq(t, `{"expiry":"720h"}`, cfg)
|
|
||||||
|
|
||||||
// A rotation is stored as config, with or without an expiry.
|
assert.Empty(t, errMsg, tc)
|
||||||
cfg, errMsg, err = handlers.BuildDatabaseTargetConfigForTest("", "daily")
|
assert.Equal(t, tc.wantExpiry, target.ArchiveExpiry, tc)
|
||||||
require.NoError(t, err)
|
assert.Equal(t, tc.wantRotation, target.ArchiveRotation, tc)
|
||||||
assert.Empty(t, errMsg)
|
}
|
||||||
assert.JSONEq(t, `{"rotation":"daily"}`, cfg)
|
|
||||||
|
|
||||||
cfg, errMsg, err = handlers.BuildDatabaseTargetConfigForTest(
|
|
||||||
"720h", "hourly",
|
|
||||||
)
|
|
||||||
require.NoError(t, err)
|
|
||||||
assert.Empty(t, errMsg)
|
|
||||||
assert.JSONEq(t, `{"expiry":"720h","rotation":"hourly"}`, cfg)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestBuildDatabaseTargetConfig_RejectsBadExpiry(
|
// TestSetDatabaseTargetSettings_RejectsBadExpiry pins each refusal's
|
||||||
|
// whole message, as the form shows it.
|
||||||
|
func TestSetDatabaseTargetSettings_RejectsBadExpiry(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
) {
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
for _, bad := range []string{"nonsense", "7d", "-5h"} {
|
cases := map[string]string{
|
||||||
cfg, errMsg, err := handlers.BuildDatabaseTargetConfigForTest(bad, "")
|
"nonsense": `Invalid archive expiry: expiry must be "never" or ` +
|
||||||
|
`a Go duration such as "720h": time: invalid duration ` +
|
||||||
|
`"nonsense"`,
|
||||||
|
"7d": `Invalid archive expiry: expiry must be "never" or a Go ` +
|
||||||
|
`duration such as "720h": time: unknown unit "d" in ` +
|
||||||
|
`duration "7d"`,
|
||||||
|
"-5h": `Invalid archive expiry: expiry must be a positive ` +
|
||||||
|
`duration or "never": "-5h"`,
|
||||||
|
}
|
||||||
|
|
||||||
require.NoError(t, err)
|
for bad, want := range cases {
|
||||||
assert.Contains(
|
var target database.Target
|
||||||
t, errMsg, "Invalid archive expiry",
|
|
||||||
"expiry %q should be refused", bad,
|
errMsg := handlers.SetDatabaseTargetSettingsForTest(
|
||||||
|
&target, bad, "",
|
||||||
)
|
)
|
||||||
assert.Empty(t, cfg)
|
|
||||||
|
assert.Equal(t, want, errMsg, "expiry %q", bad)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestBuildDatabaseTargetConfig_RejectsBadRotation(
|
func TestSetDatabaseTargetSettings_RejectsBadRotation(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
) {
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
for _, bad := range []string{"weekly", "Daily", " none"} {
|
for _, bad := range []string{"weekly", "Daily", " none"} {
|
||||||
cfg, errMsg, err := handlers.BuildDatabaseTargetConfigForTest("", bad)
|
var target database.Target
|
||||||
|
|
||||||
require.NoError(t, err)
|
errMsg := handlers.SetDatabaseTargetSettingsForTest(
|
||||||
assert.Contains(
|
&target, "", bad,
|
||||||
t, errMsg, "Invalid archive rotation",
|
)
|
||||||
"rotation %q should be refused", bad,
|
|
||||||
|
assert.Equal(
|
||||||
|
t,
|
||||||
|
"Invalid archive rotation: rotation must be none, "+
|
||||||
|
"monthly, daily or hourly: "+strconv.Quote(bad),
|
||||||
|
errMsg,
|
||||||
)
|
)
|
||||||
assert.Empty(t, cfg)
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestFormatArchiveExpiry covers the expiries the expiry choices do
|
||||||
|
// not: TestArchiveExpiryChoices shows the edit form starting on each
|
||||||
|
// choice. An expiry in whole hours is written in hours, and any other
|
||||||
|
// as Go writes it, which the form accepts back unchanged.
|
||||||
|
func TestFormatArchiveExpiry(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
cases := map[time.Duration]string{
|
||||||
|
0: expiryNever,
|
||||||
|
36 * time.Hour: "36h",
|
||||||
|
90 * time.Minute: "1h30m0s",
|
||||||
|
}
|
||||||
|
|
||||||
|
for expiry, want := range cases {
|
||||||
|
assert.Equal(t, want, handlers.FormatArchiveExpiryForTest(expiry))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"net/url"
|
"net/url"
|
||||||
"testing"
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
@@ -54,15 +55,26 @@ func TestHandleSourceCreateSubmit_CreatesRequestedTargets(t *testing.T) {
|
|||||||
|
|
||||||
env := setupSourceTest(t)
|
env := setupSourceTest(t)
|
||||||
|
|
||||||
// Each value the archive pruning choice submits, after an empty
|
// Each value the archive pruning choice submits and the expiry it
|
||||||
// one that stands for the archive checkbox left off.
|
// stores, after an empty one that stands for the archive checkbox
|
||||||
expiries := []string{
|
// left off.
|
||||||
"", "never", "1h", "12h", "24h", "720h", "2160h", "8760h",
|
expiries := []struct {
|
||||||
|
value string
|
||||||
|
stored time.Duration
|
||||||
|
}{
|
||||||
|
{"", 0},
|
||||||
|
{expiryNever, 0},
|
||||||
|
{"1h", time.Hour},
|
||||||
|
{"12h", 12 * time.Hour},
|
||||||
|
{"24h", 24 * time.Hour},
|
||||||
|
{"720h", 720 * time.Hour},
|
||||||
|
{"2160h", 2160 * time.Hour},
|
||||||
|
{"8760h", 8760 * time.Hour},
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, httpURL := range []string{"", editOriginalURL} {
|
for _, httpURL := range []string{"", editOriginalURL} {
|
||||||
for _, expiry := range expiries {
|
for _, expiry := range expiries {
|
||||||
name := "url=" + httpURL + " archive=" + expiry
|
name := "url=" + httpURL + " archive=" + expiry.value
|
||||||
|
|
||||||
t.Run(name, func(t *testing.T) {
|
t.Run(name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
@@ -72,9 +84,9 @@ func TestHandleSourceCreateSubmit_CreatesRequestedTargets(t *testing.T) {
|
|||||||
form.Set("http_url", httpURL)
|
form.Set("http_url", httpURL)
|
||||||
form.Set("archive_expiry", "720h")
|
form.Set("archive_expiry", "720h")
|
||||||
|
|
||||||
if expiry != "" {
|
if expiry.value != "" {
|
||||||
form.Set("archive", "on")
|
form.Set("archive", "on")
|
||||||
form.Set("archive_expiry", expiry)
|
form.Set("archive_expiry", expiry.value)
|
||||||
}
|
}
|
||||||
|
|
||||||
w := submitCreateForm(env, form)
|
w := submitCreateForm(env, form)
|
||||||
@@ -96,17 +108,16 @@ func TestHandleSourceCreateSubmit_CreatesRequestedTargets(t *testing.T) {
|
|||||||
wantCount++
|
wantCount++
|
||||||
|
|
||||||
assert.Equal(t, "HTTP", byType[database.TargetTypeHTTP].Name)
|
assert.Equal(t, "HTTP", byType[database.TargetTypeHTTP].Name)
|
||||||
assert.JSONEq(t, `{"url":"`+httpURL+`"}`,
|
assert.Equal(t, httpURL, byType[database.TargetTypeHTTP].URL)
|
||||||
byType[database.TargetTypeHTTP].Config)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if expiry != "" {
|
if expiry.value != "" {
|
||||||
wantCount++
|
wantCount++
|
||||||
|
|
||||||
assert.Equal(t, "Archive",
|
assert.Equal(t, "Archive",
|
||||||
byType[database.TargetTypeDatabase].Name)
|
byType[database.TargetTypeDatabase].Name)
|
||||||
assert.JSONEq(t, `{"expiry":"`+expiry+`"}`,
|
assert.Equal(t, expiry.stored,
|
||||||
byType[database.TargetTypeDatabase].Config)
|
byType[database.TargetTypeDatabase].ArchiveExpiry)
|
||||||
}
|
}
|
||||||
|
|
||||||
assert.Len(t, byType, wantCount)
|
assert.Len(t, byType, wantCount)
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import (
|
|||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"testing"
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
"github.com/go-chi/chi"
|
"github.com/go-chi/chi"
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
@@ -25,25 +26,24 @@ const (
|
|||||||
slackSecretPath
|
slackSecretPath
|
||||||
)
|
)
|
||||||
|
|
||||||
// seedConfiguredTarget inserts a target with a stored config
|
// seedConfiguredTarget inserts an active target of targetType with
|
||||||
// blob and returns it.
|
// the settings (URL to ArchiveRotation) that settings carries, and
|
||||||
|
// returns it.
|
||||||
func seedConfiguredTarget(
|
func seedConfiguredTarget(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
db *database.Database,
|
db *database.Database,
|
||||||
webhookID string,
|
webhookID string,
|
||||||
targetType database.TargetType,
|
targetType database.TargetType,
|
||||||
config string,
|
settings database.Target,
|
||||||
) *database.Target {
|
) *database.Target {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
tgt := &database.Target{
|
tgt := &settings
|
||||||
WebhookID: webhookID,
|
tgt.WebhookID = webhookID
|
||||||
Name: "t-" + string(targetType),
|
tgt.Name = "t-" + string(targetType)
|
||||||
Type: targetType,
|
tgt.Type = targetType
|
||||||
Active: true,
|
tgt.Active = true
|
||||||
Config: config,
|
tgt.MaxAttempts = 1
|
||||||
MaxAttempts: 1,
|
|
||||||
}
|
|
||||||
|
|
||||||
require.NoError(
|
require.NoError(
|
||||||
t,
|
t,
|
||||||
@@ -108,89 +108,59 @@ func serveSourceDetailPage(
|
|||||||
return w
|
return w
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestHandleSourceDetail_MasksSlackWebhookURL is the
|
// TestHandleSourceDetail_MasksTargetURL is the load-bearing
|
||||||
// load-bearing regression test for the credential leak: the
|
// regression test for the credential leak: the rendered page must
|
||||||
// rendered page must show the Slack target without any of the
|
// show a Slack target, and an http target, without any of the secret
|
||||||
// secret path segments of its webhook URL.
|
// path segments of its URL. An http target's destination is routinely
|
||||||
func TestHandleSourceDetail_MasksSlackWebhookURL(t *testing.T) {
|
// an incoming-webhook endpoint too, so the same Slack URL serves for
|
||||||
|
// both.
|
||||||
|
func TestHandleSourceDetail_MasksTargetURL(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
var (
|
labels := map[database.TargetType]string{
|
||||||
h *handlers.Handlers
|
database.TargetTypeSlack: "Webhook URL",
|
||||||
sess *session.Session
|
database.TargetTypeHTTP: "Destination URL",
|
||||||
db *database.Database
|
}
|
||||||
)
|
|
||||||
|
|
||||||
app := newTestApp(t, &h, &sess, &db)
|
for targetType, label := range labels {
|
||||||
app.RequireStart()
|
t.Run(string(targetType), func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
t.Cleanup(app.RequireStop)
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
sess *session.Session
|
||||||
|
db *database.Database
|
||||||
|
)
|
||||||
|
|
||||||
wh := seedWebhook(t, db)
|
app := newTestApp(t, &h, &sess, &db)
|
||||||
seedConfiguredTarget(
|
app.RequireStart()
|
||||||
t, db, wh.ID,
|
|
||||||
database.TargetTypeSlack,
|
|
||||||
`{"webhookUrl":"`+slackWebhookURL+`"}`,
|
|
||||||
)
|
|
||||||
|
|
||||||
body := renderSourceDetailPage(t, h, sess, wh.ID)
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
assert.NotContains(t, body, slackSecretPath)
|
wh := seedWebhook(t, db)
|
||||||
assert.NotContains(t, body, "T00000000")
|
seedConfiguredTarget(
|
||||||
assert.NotContains(t, body, "B00000000")
|
t, db, wh.ID, targetType,
|
||||||
assert.NotContains(
|
database.Target{URL: slackWebhookURL},
|
||||||
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
|
)
|
||||||
)
|
|
||||||
assert.NotContains(t, body, "webhookUrl")
|
|
||||||
|
|
||||||
assert.Contains(t, body, "Webhook URL")
|
body := renderSourceDetailPage(t, h, sess, wh.ID)
|
||||||
assert.Contains(t, body, "https://hooks.slack.com/...")
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestHandleSourceDetail_MasksHTTPDestinationURL is the
|
assert.NotContains(t, body, slackSecretPath)
|
||||||
// regression test for the same leak reached through the http
|
assert.NotContains(t, body, "T00000000")
|
||||||
// target: its destination is routinely an incoming-webhook
|
assert.NotContains(t, body, "B00000000")
|
||||||
// endpoint whose path segments are the credential, so the
|
assert.NotContains(
|
||||||
// rendered page must not contain them.
|
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
|
||||||
func TestHandleSourceDetail_MasksHTTPDestinationURL(
|
)
|
||||||
t *testing.T,
|
|
||||||
) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
var (
|
assert.Contains(t, body, label)
|
||||||
h *handlers.Handlers
|
assert.Contains(t, body, "https://hooks.slack.com/...")
|
||||||
sess *session.Session
|
})
|
||||||
db *database.Database
|
}
|
||||||
)
|
|
||||||
|
|
||||||
app := newTestApp(t, &h, &sess, &db)
|
|
||||||
app.RequireStart()
|
|
||||||
|
|
||||||
t.Cleanup(app.RequireStop)
|
|
||||||
|
|
||||||
wh := seedWebhook(t, db)
|
|
||||||
seedConfiguredTarget(
|
|
||||||
t, db, wh.ID,
|
|
||||||
database.TargetTypeHTTP,
|
|
||||||
`{"url":"`+slackWebhookURL+`"}`,
|
|
||||||
)
|
|
||||||
|
|
||||||
body := renderSourceDetailPage(t, h, sess, wh.ID)
|
|
||||||
|
|
||||||
assert.NotContains(t, body, slackSecretPath)
|
|
||||||
assert.NotContains(t, body, "T00000000")
|
|
||||||
assert.NotContains(t, body, "B00000000")
|
|
||||||
assert.NotContains(
|
|
||||||
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
|
|
||||||
)
|
|
||||||
|
|
||||||
assert.Contains(t, body, "Destination URL")
|
|
||||||
assert.Contains(t, body, "https://hooks.slack.com/...")
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestHandleSourceDetail_RendersNamedTargetFields proves the
|
// TestHandleSourceDetail_RendersNamedTargetFields proves the
|
||||||
// other target types render labelled fields rather than the
|
// other target types render labelled fields rather than the
|
||||||
// stored blob.
|
// stored values.
|
||||||
func TestHandleSourceDetail_RendersNamedTargetFields(
|
func TestHandleSourceDetail_RendersNamedTargetFields(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
) {
|
) {
|
||||||
@@ -212,18 +182,24 @@ func TestHandleSourceDetail_RendersNamedTargetFields(
|
|||||||
seedConfiguredTarget(
|
seedConfiguredTarget(
|
||||||
t, db, wh.ID,
|
t, db, wh.ID,
|
||||||
database.TargetTypeHTTP,
|
database.TargetTypeHTTP,
|
||||||
`{"url":"https://example.com/hook","timeout":30,`+
|
database.Target{
|
||||||
`"headers":{"Authorization":"Bearer sekrit"}}`,
|
URL: "https://example.com/hook",
|
||||||
|
TimeoutSeconds: 30,
|
||||||
|
Headers: map[string]string{"X-Api-Key": "sekrit"},
|
||||||
|
},
|
||||||
)
|
)
|
||||||
seedConfiguredTarget(
|
seedConfiguredTarget(
|
||||||
t, db, wh.ID,
|
t, db, wh.ID,
|
||||||
database.TargetTypeDatabase,
|
database.TargetTypeDatabase,
|
||||||
`{"expiry":"720h"}`,
|
database.Target{
|
||||||
|
ArchiveExpiry: 720 * time.Hour,
|
||||||
|
ArchiveRotation: rotationNone,
|
||||||
|
},
|
||||||
)
|
)
|
||||||
seedConfiguredTarget(
|
seedConfiguredTarget(
|
||||||
t, db, wh.ID,
|
t, db, wh.ID,
|
||||||
database.TargetType("carrier-pigeon"),
|
database.TargetType("carrier-pigeon"),
|
||||||
`{"beak":"sharp"}`,
|
database.Target{URL: "https://pigeon.example/beak"},
|
||||||
)
|
)
|
||||||
|
|
||||||
body := renderSourceDetailPage(t, h, sess, wh.ID)
|
body := renderSourceDetailPage(t, h, sess, wh.ID)
|
||||||
@@ -242,8 +218,8 @@ func TestHandleSourceDetail_RendersNamedTargetFields(
|
|||||||
assert.Contains(t, list, "Archive rotation: none")
|
assert.Contains(t, list, "Archive rotation: none")
|
||||||
assert.Contains(t, body, `<option value="database">Archive</option>`)
|
assert.Contains(t, body, `<option value="database">Archive</option>`)
|
||||||
|
|
||||||
// An unknown type gets the neutral placeholder, never the
|
// An unknown type gets the neutral placeholder, never its
|
||||||
// stored blob.
|
// stored URL.
|
||||||
assert.Contains(t, body, "(unavailable)")
|
assert.Contains(t, body, "(unavailable)")
|
||||||
assert.NotContains(t, body, "beak")
|
assert.NotContains(t, body, "beak")
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -161,7 +161,7 @@ func TestHandleSourceLogs_MasksDeletedTargetConfig(t *testing.T) {
|
|||||||
tgt := seedConfiguredTarget(
|
tgt := seedConfiguredTarget(
|
||||||
t, db, wh.ID,
|
t, db, wh.ID,
|
||||||
database.TargetTypeSlack,
|
database.TargetTypeSlack,
|
||||||
`{"webhookUrl":"`+slackWebhookURL+`"}`,
|
database.Target{URL: slackWebhookURL},
|
||||||
)
|
)
|
||||||
|
|
||||||
seedDeliveredEvent(t, dbMgr, wh.ID, tgt.ID)
|
seedDeliveredEvent(t, dbMgr, wh.ID, tgt.ID)
|
||||||
@@ -175,7 +175,6 @@ func TestHandleSourceLogs_MasksDeletedTargetConfig(t *testing.T) {
|
|||||||
assert.NotContains(
|
assert.NotContains(
|
||||||
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
|
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
|
||||||
)
|
)
|
||||||
assert.NotContains(t, body, "webhookUrl")
|
|
||||||
|
|
||||||
// The name is there; only the credential is not.
|
// The name is there; only the credential is not.
|
||||||
assert.Contains(t, body, tgt.Name+deletedMarker)
|
assert.Contains(t, body, tgt.Name+deletedMarker)
|
||||||
|
|||||||
@@ -138,7 +138,7 @@ func TestHandleSourceLogs_MasksSlackWebhookURL(t *testing.T) {
|
|||||||
tgt := seedConfiguredTarget(
|
tgt := seedConfiguredTarget(
|
||||||
t, db, wh.ID,
|
t, db, wh.ID,
|
||||||
database.TargetTypeSlack,
|
database.TargetTypeSlack,
|
||||||
`{"webhookUrl":"`+slackWebhookURL+`"}`,
|
database.Target{URL: slackWebhookURL},
|
||||||
)
|
)
|
||||||
|
|
||||||
seedDeliveredEvent(t, dbMgr, wh.ID, tgt.ID)
|
seedDeliveredEvent(t, dbMgr, wh.ID, tgt.ID)
|
||||||
@@ -151,7 +151,6 @@ func TestHandleSourceLogs_MasksSlackWebhookURL(t *testing.T) {
|
|||||||
assert.NotContains(
|
assert.NotContains(
|
||||||
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
|
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
|
||||||
)
|
)
|
||||||
assert.NotContains(t, body, "webhookUrl")
|
|
||||||
|
|
||||||
// The page still identifies the delivery's target.
|
// The page still identifies the delivery's target.
|
||||||
assert.Contains(t, body, tgt.Name)
|
assert.Contains(t, body, tgt.Name)
|
||||||
|
|||||||
@@ -2,7 +2,6 @@ package handlers
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -65,20 +64,14 @@ func (h *Handlers) processTargetCreate(
|
|||||||
) {
|
) {
|
||||||
in := targetFormInputFrom(r)
|
in := targetFormInputFrom(r)
|
||||||
|
|
||||||
target, errMsg, err := h.newTarget(r.Context(), webhook.ID, in)
|
target, errMsg := h.newTarget(r.Context(), webhook.ID, in)
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to encode target config", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
if errMsg != "" {
|
if errMsg != "" {
|
||||||
h.renderSourceDetail(w, r, webhook, in, errMsg)
|
h.renderSourceDetail(w, r, webhook, in, errMsg)
|
||||||
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
err = h.db.DB().Create(target).Error
|
err := h.db.DB().Create(target).Error
|
||||||
if err != nil {
|
if err != nil {
|
||||||
h.serverError(w, r, "failed to create target", err)
|
h.serverError(w, r, "failed to create target", err)
|
||||||
|
|
||||||
@@ -93,15 +86,13 @@ func (h *Handlers) processTargetCreate(
|
|||||||
|
|
||||||
// newTarget validates a new target for a webhook and returns the row
|
// newTarget validates a new target for a webhook and returns the row
|
||||||
// to create, or, when it refuses the target, the message the form
|
// to create, or, when it refuses the target, the message the form
|
||||||
// shows. An error is the server's fault, not a refusal: the accepted
|
// shows. Every form that creates a target goes through here, so they
|
||||||
// configuration could not be encoded. Every form that creates a
|
// all accept and refuse the same things.
|
||||||
// target goes through here, so they all accept and refuse the same
|
|
||||||
// things.
|
|
||||||
func (h *Handlers) newTarget(
|
func (h *Handlers) newTarget(
|
||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
webhookID string,
|
webhookID string,
|
||||||
in targetFormInput,
|
in targetFormInput,
|
||||||
) (*database.Target, string, error) {
|
) (*database.Target, string) {
|
||||||
target := &database.Target{
|
target := &database.Target{
|
||||||
WebhookID: webhookID,
|
WebhookID: webhookID,
|
||||||
Type: in.Type,
|
Type: in.Type,
|
||||||
@@ -109,34 +100,33 @@ func (h *Handlers) newTarget(
|
|||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
}
|
}
|
||||||
|
|
||||||
errMsg, err := h.setTargetFromForm(ctx, target, in)
|
errMsg := h.setTargetFromForm(ctx, target, in)
|
||||||
if err != nil || errMsg != "" {
|
if errMsg != "" {
|
||||||
return nil, errMsg, err
|
return nil, errMsg
|
||||||
}
|
}
|
||||||
|
|
||||||
return target, "", nil
|
return target, ""
|
||||||
}
|
}
|
||||||
|
|
||||||
// setTargetFromForm validates a target form against the target's type
|
// setTargetFromForm validates a target form against the target's type
|
||||||
// and, when it accepts it, sets the target's name, configuration and
|
// and sets the target's name, settings and number of delivery attempts
|
||||||
// number of delivery attempts from it. It returns the message the form
|
// from it. It returns the message the form shows for anything it
|
||||||
// shows for anything it refuses, an unknown type among them, and then
|
// refuses, an unknown type among them; the caller then discards the
|
||||||
// leaves the target unchanged; an error is the server's fault, as for
|
// target, which may be partly set.
|
||||||
// newTarget.
|
|
||||||
// The add target form and the target edit form both go through here,
|
// The add target form and the target edit form both go through here,
|
||||||
// so the two cannot come to disagree about what a target may be.
|
// so the two cannot come to disagree about what a target may be.
|
||||||
func (h *Handlers) setTargetFromForm(
|
func (h *Handlers) setTargetFromForm(
|
||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
target *database.Target,
|
target *database.Target,
|
||||||
in targetFormInput,
|
in targetFormInput,
|
||||||
) (string, error) {
|
) string {
|
||||||
if in.Name == "" {
|
if in.Name == "" {
|
||||||
return "Name is required", nil
|
return "Name is required"
|
||||||
}
|
}
|
||||||
|
|
||||||
configJSON, errMsg, err := h.buildTargetConfig(ctx, target.Type, in)
|
errMsg := h.setTargetSettings(ctx, target, in)
|
||||||
if err != nil || errMsg != "" {
|
if errMsg != "" {
|
||||||
return errMsg, err
|
return errMsg
|
||||||
}
|
}
|
||||||
|
|
||||||
// An empty max_attempts keeps the target's count: the default of
|
// An empty max_attempts keeps the target's count: the default of
|
||||||
@@ -147,14 +137,13 @@ func (h *Handlers) setTargetFromForm(
|
|||||||
// target is delivering with.
|
// target is delivering with.
|
||||||
maxAttempts, err := parseMaxAttempts(in.MaxAttempts, target.MaxAttempts)
|
maxAttempts, err := parseMaxAttempts(in.MaxAttempts, target.MaxAttempts)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "Invalid delivery attempts: " + attemptsErrorMessage(err), nil
|
return "Invalid delivery attempts: " + attemptsErrorMessage(err)
|
||||||
}
|
}
|
||||||
|
|
||||||
target.Name = in.Name
|
target.Name = in.Name
|
||||||
target.Config = configJSON
|
|
||||||
target.MaxAttempts = maxAttempts
|
target.MaxAttempts = maxAttempts
|
||||||
|
|
||||||
return "", nil
|
return ""
|
||||||
}
|
}
|
||||||
|
|
||||||
// targetFormInput carries the raw values of a target form. Both the
|
// targetFormInput carries the raw values of a target form. Both the
|
||||||
@@ -213,84 +202,81 @@ func targetFormInputFrom(r *http.Request) targetFormInput {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// buildTargetConfig builds the JSON config string for a target from
|
// setTargetSettings sets a target's settings from the submitted form
|
||||||
// the submitted form values, or returns the message the form shows
|
// values, or returns the message the form shows for a value it
|
||||||
// for a value it refuses. An error is the server's fault, not a
|
// refuses. Which fields of in apply depends on the target type; a
|
||||||
// refusal: the accepted configuration could not be encoded. Which
|
// type without a URL ignores any URL submitted, and the settings a
|
||||||
// fields of in apply depends on the target type; a type without a URL
|
// type does not use are left empty.
|
||||||
// ignores any URL submitted.
|
func (h *Handlers) setTargetSettings(
|
||||||
func (h *Handlers) buildTargetConfig(
|
|
||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
targetType database.TargetType,
|
target *database.Target,
|
||||||
in targetFormInput,
|
in targetFormInput,
|
||||||
) (string, string, error) {
|
) string {
|
||||||
switch targetType {
|
switch target.Type {
|
||||||
case database.TargetTypeHTTP:
|
case database.TargetTypeHTTP:
|
||||||
return h.buildHTTPTargetConfig(ctx, in)
|
return h.setHTTPTargetSettings(ctx, target, in)
|
||||||
case database.TargetTypeSlack:
|
case database.TargetTypeSlack:
|
||||||
return h.buildSlackTargetConfig(ctx, in.URL)
|
return h.setSlackTargetSettings(ctx, target, in.URL)
|
||||||
case database.TargetTypeDatabase:
|
case database.TargetTypeDatabase:
|
||||||
return buildDatabaseTargetConfig(in.Expiry, in.Rotation)
|
return setDatabaseTargetSettings(target, in.Expiry, in.Rotation)
|
||||||
case database.TargetTypeLog:
|
case database.TargetTypeLog:
|
||||||
return "", "", nil
|
return ""
|
||||||
default:
|
default:
|
||||||
return "", "Invalid target type", nil
|
return "Invalid target type"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// buildHTTPTargetConfig builds config JSON for an HTTP target: an
|
// setHTTPTargetSettings sets an HTTP target's SSRF-validated
|
||||||
// SSRF-validated destination plus the optional headers, timeout and
|
// destination plus the optional headers, timeout and query string
|
||||||
// query string setting the delivery path honours.
|
// setting the delivery path honours.
|
||||||
func (h *Handlers) buildHTTPTargetConfig(
|
func (h *Handlers) setHTTPTargetSettings(
|
||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
|
target *database.Target,
|
||||||
in targetFormInput,
|
in targetFormInput,
|
||||||
) (string, string, error) {
|
) string {
|
||||||
errMsg := h.validateTargetURL(
|
errMsg := h.validateTargetURL(
|
||||||
ctx, in.URL, "URL is required for HTTP targets",
|
ctx, in.URL, "URL is required for HTTP targets",
|
||||||
)
|
)
|
||||||
if errMsg != "" {
|
if errMsg != "" {
|
||||||
return "", errMsg, nil
|
return errMsg
|
||||||
}
|
}
|
||||||
|
|
||||||
headers, err := delivery.ParseTargetHeaders(in.Headers)
|
headers, err := delivery.ParseTargetHeaders(in.Headers)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", fmt.Sprintf("Invalid headers: %v", err), nil
|
return fmt.Sprintf("Invalid headers: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
timeout, err := delivery.ParseTargetTimeout(in.Timeout)
|
timeout, err := delivery.ParseTargetTimeout(in.Timeout)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", fmt.Sprintf("Invalid timeout: %v", err), nil
|
return fmt.Sprintf("Invalid timeout: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
configJSON, err := marshalTargetConfig(delivery.HTTPTargetConfig{
|
target.URL = in.URL
|
||||||
URL: in.URL,
|
target.Headers = headers
|
||||||
Headers: headers,
|
target.TimeoutSeconds = timeout
|
||||||
Timeout: timeout,
|
target.ForwardQuery = in.ForwardQuery
|
||||||
ForwardQuery: in.ForwardQuery,
|
|
||||||
})
|
|
||||||
|
|
||||||
return configJSON, "", err
|
return ""
|
||||||
}
|
}
|
||||||
|
|
||||||
// buildSlackTargetConfig builds config JSON for a Slack target,
|
// setSlackTargetSettings sets a Slack target's one setting, its
|
||||||
// whose whole configuration is one SSRF-validated webhook URL.
|
// SSRF-validated webhook URL.
|
||||||
func (h *Handlers) buildSlackTargetConfig(
|
func (h *Handlers) setSlackTargetSettings(
|
||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
|
target *database.Target,
|
||||||
targetURL string,
|
targetURL string,
|
||||||
) (string, string, error) {
|
) string {
|
||||||
errMsg := h.validateTargetURL(
|
errMsg := h.validateTargetURL(
|
||||||
ctx, targetURL,
|
ctx, targetURL,
|
||||||
"Webhook URL is required for Slack targets",
|
"Webhook URL is required for Slack targets",
|
||||||
)
|
)
|
||||||
if errMsg != "" {
|
if errMsg != "" {
|
||||||
return "", errMsg, nil
|
return errMsg
|
||||||
}
|
}
|
||||||
|
|
||||||
configJSON, err := marshalTargetConfig(delivery.SlackTargetConfig{
|
target.URL = targetURL
|
||||||
WebhookURL: targetURL,
|
|
||||||
})
|
|
||||||
|
|
||||||
return configJSON, "", err
|
return ""
|
||||||
}
|
}
|
||||||
|
|
||||||
// validateTargetURL refuses an empty or SSRF-blocked destination,
|
// validateTargetURL refuses an empty or SSRF-blocked destination,
|
||||||
@@ -341,52 +327,31 @@ func (h *Handlers) validateTargetURL(
|
|||||||
return ""
|
return ""
|
||||||
}
|
}
|
||||||
|
|
||||||
// marshalTargetConfig serialises a target configuration for storage.
|
// setDatabaseTargetSettings sets a database (archive) target's
|
||||||
func marshalTargetConfig(cfg any) (string, error) {
|
// expiry and rotation. Both are optional: an empty or never expiry
|
||||||
configBytes, err := json.Marshal(cfg)
|
// keeps the archive forever, and an empty rotation is stored as none,
|
||||||
if err != nil {
|
// one file. A bad value is refused here, at the form, instead of
|
||||||
return "", err
|
// being stored.
|
||||||
}
|
func setDatabaseTargetSettings(
|
||||||
|
target *database.Target,
|
||||||
return string(configBytes), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// buildDatabaseTargetConfig builds config JSON for a database
|
|
||||||
// (archive) target. The optional expiry and rotation are validated
|
|
||||||
// here, at creation time, so a bad value is refused instead of
|
|
||||||
// failing every subsequent delivery. Each is stored only when set,
|
|
||||||
// and with neither the config is empty (the keep-forever, one-file
|
|
||||||
// default).
|
|
||||||
func buildDatabaseTargetConfig(
|
|
||||||
expiry, rotation string,
|
expiry, rotation string,
|
||||||
) (string, string, error) {
|
) string {
|
||||||
expiry = strings.TrimSpace(expiry)
|
archiveExpiry, err := archive.ParseExpiry(strings.TrimSpace(expiry))
|
||||||
|
|
||||||
err := archive.ValidateExpiry(expiry)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", fmt.Sprintf("Invalid archive expiry: %v", err), nil
|
return fmt.Sprintf("Invalid archive expiry: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
err = archive.ValidateRotation(rotation)
|
err = archive.ValidateRotation(rotation)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", fmt.Sprintf("Invalid archive rotation: %v", err), nil
|
return fmt.Sprintf("Invalid archive rotation: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
cfg := map[string]any{}
|
if rotation == "" {
|
||||||
|
rotation = archiveRotationNone
|
||||||
if expiry != "" {
|
|
||||||
cfg["expiry"] = expiry
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if rotation != "" {
|
target.ArchiveExpiry = archiveExpiry
|
||||||
cfg["rotation"] = rotation
|
target.ArchiveRotation = rotation
|
||||||
}
|
|
||||||
|
|
||||||
if len(cfg) == 0 {
|
return ""
|
||||||
return "", "", nil
|
|
||||||
}
|
|
||||||
|
|
||||||
configJSON, err := marshalTargetConfig(cfg)
|
|
||||||
|
|
||||||
return configJSON, "", err
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -176,7 +176,7 @@ func TestHandleTargetCreate_BodyURLStillCreatesTheTarget(t *testing.T) {
|
|||||||
|
|
||||||
targets := targetsForWebhook(t, env.db, webhook.ID)
|
targets := targetsForWebhook(t, env.db, webhook.ID)
|
||||||
require.Len(t, targets, 1)
|
require.Len(t, targets, 1)
|
||||||
assert.Contains(t, targets[0].Config, targetSecretSegments)
|
assert.Equal(t, targetSecretURL, targets[0].URL)
|
||||||
|
|
||||||
// The body carried the credential, so the access log must still
|
// The body carried the credential, so the access log must still
|
||||||
// not have it: the log records the request line only.
|
// not have it: the log records the request line only.
|
||||||
|
|||||||
@@ -6,73 +6,114 @@ import (
|
|||||||
"net/url"
|
"net/url"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
)
|
)
|
||||||
|
|
||||||
// TestHandleTargetCreate_EveryType adds a target of each type. Each
|
// TestHandleTargetCreate_EveryType adds a target of each type and
|
||||||
// submission carries a url: only the http and slack types store one.
|
// checks the settings it stores. Each submission carries a url and an
|
||||||
|
// expiry: only the http and slack types store the url, only the
|
||||||
|
// database type the expiry, and every setting a type does not use
|
||||||
|
// stays empty.
|
||||||
func TestHandleTargetCreate_EveryType(t *testing.T) {
|
func TestHandleTargetCreate_EveryType(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
env := setupSourceTest(t)
|
env := setupSourceTest(t)
|
||||||
|
|
||||||
// fields is the rest of each submission, as a query string.
|
// fields is the rest of each submission, as a query string, and
|
||||||
|
// want holds the settings stored.
|
||||||
cases := []struct {
|
cases := []struct {
|
||||||
targetType database.TargetType
|
targetType database.TargetType
|
||||||
fields string
|
fields string
|
||||||
wantConfig string
|
want database.Target
|
||||||
wantAttempts int
|
wantAttempts int
|
||||||
}{
|
}{
|
||||||
{
|
{
|
||||||
database.TargetTypeHTTP, "timeout=12&max_attempts=3",
|
database.TargetTypeHTTP,
|
||||||
`{"url":"` + editOriginalURL + `","timeout":12}`, 3,
|
"timeout=12&max_attempts=3&forward_query=on",
|
||||||
|
database.Target{
|
||||||
|
URL: editOriginalURL,
|
||||||
|
Headers: map[string]string{},
|
||||||
|
TimeoutSeconds: 12,
|
||||||
|
ForwardQuery: true,
|
||||||
|
}, 3,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
database.TargetTypeSlack, "max_attempts=4",
|
database.TargetTypeSlack, "max_attempts=4",
|
||||||
`{"webhookUrl":"` + editOriginalURL + `"}`, 4,
|
database.Target{URL: editOriginalURL}, 4,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
database.TargetTypeDatabase, "expiry=720h",
|
database.TargetTypeDatabase, "rotation=daily",
|
||||||
`{"expiry":"720h"}`, 1,
|
database.Target{
|
||||||
|
ArchiveExpiry: 720 * time.Hour,
|
||||||
|
ArchiveRotation: rotationDaily,
|
||||||
|
}, 1,
|
||||||
},
|
},
|
||||||
{database.TargetTypeLog, "", "", 1},
|
{database.TargetTypeLog, "", database.Target{}, 1},
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, tc := range cases {
|
for _, tc := range cases {
|
||||||
t.Run(string(tc.targetType), func(t *testing.T) {
|
t.Run(string(tc.targetType), func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
webhook := seedWebhookWithRetention(t, env.db, 30)
|
_, got := addTargetThroughForm(
|
||||||
|
t, env, tc.targetType,
|
||||||
form, err := url.ParseQuery(tc.fields)
|
"url="+editOriginalURL+"&expiry=720h&"+tc.fields,
|
||||||
require.NoError(t, err)
|
|
||||||
form.Set("name", "every-type")
|
|
||||||
form.Set("type", string(tc.targetType))
|
|
||||||
form.Set("url", editOriginalURL)
|
|
||||||
|
|
||||||
w := serveTarget(
|
|
||||||
env, http.MethodPost,
|
|
||||||
"/hook/"+webhook.ID+"/targets", form,
|
|
||||||
)
|
)
|
||||||
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
assert.Equal(t, tc.targetType, got.Type)
|
||||||
|
assert.Equal(t, tc.wantAttempts, got.MaxAttempts)
|
||||||
targets := targetsForWebhook(t, env.db, webhook.ID)
|
assert.Equal(t, tc.want, targetSettingsOf(got))
|
||||||
require.Len(t, targets, 1)
|
|
||||||
assert.Equal(t, tc.targetType, targets[0].Type)
|
|
||||||
assert.Equal(t, tc.wantAttempts, targets[0].MaxAttempts)
|
|
||||||
|
|
||||||
if tc.wantConfig == "" {
|
|
||||||
assert.Empty(t, targets[0].Config)
|
|
||||||
} else {
|
|
||||||
assert.JSONEq(t, tc.wantConfig, targets[0].Config)
|
|
||||||
}
|
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// addTargetThroughForm adds a target of targetType named every-type to
|
||||||
|
// a new webhook through the add target form, with the rest of the form
|
||||||
|
// given as a query string, and returns the webhook and the target the
|
||||||
|
// form stored.
|
||||||
|
func addTargetThroughForm(
|
||||||
|
t *testing.T,
|
||||||
|
env *sourceTestEnv,
|
||||||
|
targetType database.TargetType,
|
||||||
|
fields string,
|
||||||
|
) (database.Webhook, database.Target) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
webhook := seedWebhookWithRetention(t, env.db, 30)
|
||||||
|
|
||||||
|
form, err := url.ParseQuery(fields)
|
||||||
|
require.NoError(t, err)
|
||||||
|
form.Set("name", "every-type")
|
||||||
|
form.Set("type", string(targetType))
|
||||||
|
|
||||||
|
w := serveTarget(
|
||||||
|
env, http.MethodPost, "/hook/"+webhook.ID+"/targets", form,
|
||||||
|
)
|
||||||
|
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
||||||
|
|
||||||
|
targets := targetsForWebhook(t, env.db, webhook.ID)
|
||||||
|
require.Len(t, targets, 1)
|
||||||
|
|
||||||
|
return webhook, targets[0]
|
||||||
|
}
|
||||||
|
|
||||||
|
// targetSettingsOf returns a target's settings, URL to
|
||||||
|
// ArchiveRotation, and nothing else, for comparing with the settings a
|
||||||
|
// test expects.
|
||||||
|
func targetSettingsOf(target database.Target) database.Target {
|
||||||
|
return database.Target{
|
||||||
|
URL: target.URL,
|
||||||
|
Headers: target.Headers,
|
||||||
|
TimeoutSeconds: target.TimeoutSeconds,
|
||||||
|
ForwardQuery: target.ForwardQuery,
|
||||||
|
ArchiveExpiry: target.ArchiveExpiry,
|
||||||
|
ArchiveRotation: target.ArchiveRotation,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestHandleTargetCreate_RefusedFormComesBack refuses a target of each
|
// TestHandleTargetCreate_RefusedFormComesBack refuses a target of each
|
||||||
// type and checks that the webhook page comes back with the add target
|
// type and checks that the webhook page comes back with the add target
|
||||||
// form open on that type, the values entered, and the reason.
|
// form open on that type, the values entered, and the reason.
|
||||||
|
|||||||
@@ -25,22 +25,15 @@ const (
|
|||||||
tmplKeyMaxTimeout = "MaxTimeout"
|
tmplKeyMaxTimeout = "MaxTimeout"
|
||||||
)
|
)
|
||||||
|
|
||||||
// configUnreadableMessage is shown when a target's stored
|
|
||||||
// configuration does not parse. It says plainly that saving replaces
|
|
||||||
// the stored value rather than preserving it, because the form
|
|
||||||
// cannot pre-fill what it could not read.
|
|
||||||
const configUnreadableMessage = "The stored configuration for this " +
|
|
||||||
"target could not be read. Enter the values below; saving " +
|
|
||||||
"replaces the stored configuration."
|
|
||||||
|
|
||||||
// targetEditView is the display model for the target edit page: the
|
// targetEditView is the display model for the target edit page: the
|
||||||
// target's row fields as stored. The values the form shows, the
|
// target's row fields as stored. The values the form shows, the
|
||||||
// UNMASKED configuration among them, come separately, as a
|
// UNMASKED URL and headers among them, come separately, as a
|
||||||
// targetFormInput.
|
// targetFormInput.
|
||||||
//
|
//
|
||||||
// It deliberately omits database.Target's raw Config blob: the form
|
// It deliberately omits database.Target's settings: the form renders
|
||||||
// renders named fields, and giving the template the blob as well
|
// them from the targetFormInput, and giving the template the target's
|
||||||
// would put an unreviewed second path to the credential on the page.
|
// URL and headers as well would put an unreviewed second path to the
|
||||||
|
// credential on the page.
|
||||||
type targetEditView struct {
|
type targetEditView struct {
|
||||||
ID string
|
ID string
|
||||||
Name string
|
Name string
|
||||||
@@ -48,13 +41,16 @@ type targetEditView struct {
|
|||||||
Active bool
|
Active bool
|
||||||
}
|
}
|
||||||
|
|
||||||
// HandleTargetEdit shows the form to edit a target.
|
// HandleTargetEdit shows the form to edit a target, filled with the
|
||||||
|
// target's settings as stored.
|
||||||
//
|
//
|
||||||
// This page is the one place the full destination URL and header
|
// This page is the one place the full destination URL and header
|
||||||
// values are shown. It is reachable only through the
|
// values are shown: an operator cannot correct a value they cannot
|
||||||
// /hook/{sourceID} route group, which supplies RequireAuth and
|
// see. It is reachable only through the /hook/{sourceID} route
|
||||||
// NoCache, and only for a target of a webhook the session's user
|
// group, which supplies RequireAuth and NoCache, and only for a
|
||||||
// owns; masking (delivery.TargetView) is unchanged everywhere else.
|
// target of a webhook the session's user owns; masking
|
||||||
|
// (delivery.TargetView) is unchanged everywhere else. Do not fill a
|
||||||
|
// form like this one on any other page.
|
||||||
func (h *Handlers) HandleTargetEdit() http.HandlerFunc {
|
func (h *Handlers) HandleTargetEdit() http.HandlerFunc {
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
webhook, target, ok := h.ownedTarget(w, r)
|
webhook, target, ok := h.ownedTarget(w, r)
|
||||||
@@ -62,34 +58,19 @@ func (h *Handlers) HandleTargetEdit() http.HandlerFunc {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
cfg, err := delivery.NewTargetConfigForm(target)
|
|
||||||
msg := ""
|
|
||||||
|
|
||||||
if err != nil {
|
|
||||||
// The error carries the parse failure, never the
|
|
||||||
// blob, so it is safe to log against the target id.
|
|
||||||
h.log.Warn(
|
|
||||||
"stored target config could not be read for editing",
|
|
||||||
"target_id", target.ID,
|
|
||||||
"error", err,
|
|
||||||
)
|
|
||||||
|
|
||||||
msg = configUnreadableMessage
|
|
||||||
}
|
|
||||||
|
|
||||||
form := targetFormInput{
|
form := targetFormInput{
|
||||||
Name: target.Name,
|
Name: target.Name,
|
||||||
URL: cfg.URL,
|
URL: target.URL,
|
||||||
Headers: cfg.Headers,
|
Headers: delivery.FormatTargetHeaders(target.Headers),
|
||||||
Timeout: cfg.Timeout,
|
Timeout: delivery.FormatTargetTimeout(target.TimeoutSeconds),
|
||||||
ForwardQuery: cfg.ForwardQuery,
|
ForwardQuery: target.ForwardQuery,
|
||||||
MaxAttempts: strconv.Itoa(target.MaxAttempts),
|
MaxAttempts: strconv.Itoa(target.MaxAttempts),
|
||||||
Expiry: cfg.Expiry,
|
Expiry: formatArchiveExpiry(target.ArchiveExpiry),
|
||||||
Rotation: cfg.Rotation,
|
Rotation: target.ArchiveRotation,
|
||||||
}
|
}
|
||||||
|
|
||||||
h.renderTargetEdit(
|
h.renderTargetEdit(
|
||||||
w, r, webhook, target, form, msg, http.StatusOK,
|
w, r, webhook, target, form, "", http.StatusOK,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -125,11 +106,11 @@ func (h *Handlers) HandleTargetEditSubmit() http.HandlerFunc {
|
|||||||
// does, so an edited destination is SSRF-validated exactly as a new
|
// does, so an edited destination is SSRF-validated exactly as a new
|
||||||
// one is.
|
// one is.
|
||||||
//
|
//
|
||||||
// The target's type is not editable. Each type stores a different
|
// The target's type is not editable. Each type uses different
|
||||||
// configuration shape and its delivery history is recorded against
|
// settings and its delivery history is recorded against the target
|
||||||
// the target row, so changing the type of an existing target is
|
// row, so changing the type of an existing target is really the
|
||||||
// really the creation of a different one. The stored type decides
|
// creation of a different one. The stored type decides which fields
|
||||||
// which fields the form offers and which builder runs.
|
// the form offers and which settings are set.
|
||||||
func (h *Handlers) applyTargetEdit(
|
func (h *Handlers) applyTargetEdit(
|
||||||
w http.ResponseWriter,
|
w http.ResponseWriter,
|
||||||
r *http.Request,
|
r *http.Request,
|
||||||
@@ -142,13 +123,7 @@ func (h *Handlers) applyTargetEdit(
|
|||||||
// as stored, for the page shown again when the save is refused.
|
// as stored, for the page shown again when the save is refused.
|
||||||
edited := *target
|
edited := *target
|
||||||
|
|
||||||
errMsg, err := h.setTargetFromForm(r.Context(), &edited, in)
|
errMsg := h.setTargetFromForm(r.Context(), &edited, in)
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to encode target config", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
if errMsg != "" {
|
if errMsg != "" {
|
||||||
h.renderTargetEdit(
|
h.renderTargetEdit(
|
||||||
w, r, webhook, target, in, errMsg, http.StatusBadRequest,
|
w, r, webhook, target, in, errMsg, http.StatusBadRequest,
|
||||||
@@ -160,7 +135,7 @@ func (h *Handlers) applyTargetEdit(
|
|||||||
// A new name renames the archive file before it is saved (see
|
// A new name renames the archive file before it is saved (see
|
||||||
// archive.Writers.Rename). If either step fails, it goes back to
|
// archive.Writers.Rename). If either step fails, it goes back to
|
||||||
// the name that is still stored.
|
// the name that is still stored.
|
||||||
err = h.renameTargetArchive(
|
err := h.renameTargetArchive(
|
||||||
target, webhook.Name, target.Name, edited.Name,
|
target, webhook.Name, target.Name, edited.Name,
|
||||||
)
|
)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
|
|||||||
@@ -2,19 +2,18 @@ package handlers_test
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"net/url"
|
"net/url"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
"github.com/go-chi/chi"
|
"github.com/go-chi/chi"
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"gorm.io/gorm/clause"
|
"gorm.io/gorm/clause"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
"sneak.berlin/go/webhooker/internal/delivery"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// The destinations the target edit tests configure. Both are literal
|
// The destinations the target edit tests configure. Both are literal
|
||||||
@@ -150,27 +149,6 @@ func storedTarget(
|
|||||||
return target
|
return target
|
||||||
}
|
}
|
||||||
|
|
||||||
// storedHTTPConfig reloads a target and parses its stored HTTP
|
|
||||||
// configuration.
|
|
||||||
func storedHTTPConfig(
|
|
||||||
t *testing.T,
|
|
||||||
env *sourceTestEnv,
|
|
||||||
targetID string,
|
|
||||||
) delivery.HTTPTargetConfig {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
var cfg delivery.HTTPTargetConfig
|
|
||||||
|
|
||||||
require.NoError(
|
|
||||||
t,
|
|
||||||
json.Unmarshal(
|
|
||||||
[]byte(storedTarget(t, env, targetID).Config), &cfg,
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
return cfg
|
|
||||||
}
|
|
||||||
|
|
||||||
// editForm is the fully populated edit submission for an HTTP
|
// editForm is the fully populated edit submission for an HTTP
|
||||||
// target.
|
// target.
|
||||||
func editForm(targetURL, headers, timeout string) url.Values {
|
func editForm(targetURL, headers, timeout string) url.Values {
|
||||||
@@ -197,12 +175,9 @@ func submitTargetEdit(
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestHandleTargetCreate_Configuration covers the half of the gap
|
// TestHandleTargetCreate_Configuration proves the add target form
|
||||||
// that is not about editing at all: HTTPTargetConfig has carried
|
// stores an HTTP target's headers and timeout, and leaves them empty
|
||||||
// Headers and Timeout, and the delivery path has honoured them, but
|
// when the form leaves them empty.
|
||||||
// the create form wrote {"url":...} and nothing else, so a
|
|
||||||
// destination needing an Authorization header could not be
|
|
||||||
// configured through the UI at all.
|
|
||||||
func TestHandleTargetCreate_Configuration(t *testing.T) {
|
func TestHandleTargetCreate_Configuration(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -213,9 +188,9 @@ func TestHandleTargetCreate_Configuration(t *testing.T) {
|
|||||||
assertCreateStoresHeadersAndTimeout(t, env)
|
assertCreateStoresHeadersAndTimeout(t, env)
|
||||||
})
|
})
|
||||||
|
|
||||||
t.Run("without them keeps a url-only config", func(t *testing.T) {
|
t.Run("without them stores only the url", func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
assertCreateKeepsURLOnlyConfig(t, env)
|
assertCreateStoresOnlyTheURL(t, env)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -228,33 +203,31 @@ func assertCreateStoresHeadersAndTimeout(
|
|||||||
t, env, editAuthHeader+"\nX-Tenant: acme\n", "12",
|
t, env, editAuthHeader+"\nX-Tenant: acme\n", "12",
|
||||||
)
|
)
|
||||||
|
|
||||||
cfg := storedHTTPConfig(t, env, target.ID)
|
stored := storedTarget(t, env, target.ID)
|
||||||
|
|
||||||
assert.Equal(t, editOriginalURL, cfg.URL)
|
assert.Equal(t, editOriginalURL, stored.URL)
|
||||||
assert.Equal(t, 12, cfg.Timeout)
|
assert.Equal(t, 12, stored.TimeoutSeconds)
|
||||||
assert.Equal(
|
assert.Equal(
|
||||||
t,
|
t,
|
||||||
map[string]string{
|
map[string]string{
|
||||||
"Authorization": "Bearer " + editBearerSecret,
|
"Authorization": "Bearer " + editBearerSecret,
|
||||||
"X-Tenant": "acme",
|
"X-Tenant": "acme",
|
||||||
},
|
},
|
||||||
cfg.Headers,
|
stored.Headers,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Without the new fields the stored shape must be the same
|
func assertCreateStoresOnlyTheURL(
|
||||||
// {"url":...} the create form wrote before they existed, so no
|
|
||||||
// existing target's configuration is rewritten by this change.
|
|
||||||
func assertCreateKeepsURLOnlyConfig(
|
|
||||||
t *testing.T, env *sourceTestEnv,
|
t *testing.T, env *sourceTestEnv,
|
||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
_, target := seedHTTPTarget(t, env, "", "")
|
_, target := seedHTTPTarget(t, env, "", "")
|
||||||
|
|
||||||
assert.JSONEq(
|
assert.Equal(t, editOriginalURL, target.URL)
|
||||||
t, `{"url":"`+editOriginalURL+`"}`, target.Config,
|
assert.Empty(t, target.Headers)
|
||||||
)
|
assert.Zero(t, target.TimeoutSeconds)
|
||||||
|
assert.False(t, target.ForwardQuery)
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestHandleTargetEditSubmit_Saves is the round trip the issue asks
|
// TestHandleTargetEditSubmit_Saves is the round trip the issue asks
|
||||||
@@ -299,13 +272,8 @@ func assertEditChangesDestination(
|
|||||||
)
|
)
|
||||||
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
||||||
|
|
||||||
assert.Equal(
|
|
||||||
t,
|
|
||||||
editReplacedURL,
|
|
||||||
storedHTTPConfig(t, env, target.ID).URL,
|
|
||||||
)
|
|
||||||
|
|
||||||
reloaded := storedTarget(t, env, target.ID)
|
reloaded := storedTarget(t, env, target.ID)
|
||||||
|
assert.Equal(t, editReplacedURL, reloaded.URL)
|
||||||
assert.Equal(t, "edited-name", reloaded.Name)
|
assert.Equal(t, "edited-name", reloaded.Name)
|
||||||
assert.Equal(t, 5, reloaded.MaxAttempts)
|
assert.Equal(t, 5, reloaded.MaxAttempts)
|
||||||
assert.Equal(
|
assert.Equal(
|
||||||
@@ -333,16 +301,16 @@ func assertEditRoundTripsHeadersAndTimeout(
|
|||||||
)
|
)
|
||||||
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
||||||
|
|
||||||
cfg := storedHTTPConfig(t, env, target.ID)
|
stored := storedTarget(t, env, target.ID)
|
||||||
|
|
||||||
assert.Equal(t, 21, cfg.Timeout)
|
assert.Equal(t, 21, stored.TimeoutSeconds)
|
||||||
assert.Equal(
|
assert.Equal(
|
||||||
t,
|
t,
|
||||||
map[string]string{
|
map[string]string{
|
||||||
"Authorization": "Bearer rotated",
|
"Authorization": "Bearer rotated",
|
||||||
"X-Trace": "on",
|
"X-Trace": "on",
|
||||||
},
|
},
|
||||||
cfg.Headers,
|
stored.Headers,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -362,10 +330,10 @@ func assertEditClearingRemovesThem(
|
|||||||
)
|
)
|
||||||
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
||||||
|
|
||||||
cfg := storedHTTPConfig(t, env, target.ID)
|
stored := storedTarget(t, env, target.ID)
|
||||||
|
|
||||||
assert.Empty(t, cfg.Headers)
|
assert.Empty(t, stored.Headers)
|
||||||
assert.Zero(t, cfg.Timeout)
|
assert.Zero(t, stored.TimeoutSeconds)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Delivery attempts are offered only by the forms for target types
|
// Delivery attempts are offered only by the forms for target types
|
||||||
@@ -442,6 +410,90 @@ func TestHandleTargetEdit_CallsTheDatabaseTypeArchive(t *testing.T) {
|
|||||||
assert.Contains(t, page, `class="label">Archive rotation</label>`)
|
assert.Contains(t, page, `class="label">Archive rotation</label>`)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestHandleTargetEdit_EveryType edits a target of each type: the edit
|
||||||
|
// form starts on the settings the add target form stored, and saving
|
||||||
|
// it stores the settings submitted.
|
||||||
|
func TestHandleTargetEdit_EveryType(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
env := setupSourceTest(t)
|
||||||
|
|
||||||
|
// created and edited are the add target form's and the edit
|
||||||
|
// form's fields, as query strings; shown is what the edit form
|
||||||
|
// must show of created; want is the settings edited stores.
|
||||||
|
cases := []struct {
|
||||||
|
targetType database.TargetType
|
||||||
|
created, edited string
|
||||||
|
shown []string
|
||||||
|
want database.Target
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
database.TargetTypeHTTP,
|
||||||
|
"url=" + editOriginalURL +
|
||||||
|
"&headers=X-Tenant:+acme&timeout=7&forward_query=on",
|
||||||
|
"url=" + editReplacedURL + "&timeout=9",
|
||||||
|
[]string{
|
||||||
|
`value="` + editOriginalURL + `"`,
|
||||||
|
"X-Tenant: acme",
|
||||||
|
`value="7"`,
|
||||||
|
`name="forward_query" value="on" checked`,
|
||||||
|
},
|
||||||
|
database.Target{
|
||||||
|
URL: editReplacedURL,
|
||||||
|
Headers: map[string]string{},
|
||||||
|
TimeoutSeconds: 9,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
database.TargetTypeSlack,
|
||||||
|
"url=" + editOriginalURL,
|
||||||
|
"url=" + editReplacedURL,
|
||||||
|
[]string{`value="` + editOriginalURL + `"`},
|
||||||
|
database.Target{URL: editReplacedURL},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
database.TargetTypeDatabase,
|
||||||
|
"expiry=720h&rotation=daily",
|
||||||
|
"expiry=24h&rotation=hourly",
|
||||||
|
[]string{
|
||||||
|
`<option value="720h" selected>`,
|
||||||
|
`<option value="daily" selected>`,
|
||||||
|
},
|
||||||
|
database.Target{
|
||||||
|
ArchiveExpiry: 24 * time.Hour,
|
||||||
|
ArchiveRotation: rotationHourly,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{database.TargetTypeLog, "", "", nil, database.Target{}},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range cases {
|
||||||
|
t.Run(string(tc.targetType), func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
webhook, target := addTargetThroughForm(
|
||||||
|
t, env, tc.targetType, tc.created,
|
||||||
|
)
|
||||||
|
|
||||||
|
page := targetEditPage(t, env, webhook.ID, target.ID)
|
||||||
|
for _, shown := range tc.shown {
|
||||||
|
assert.Contains(t, page, shown)
|
||||||
|
}
|
||||||
|
|
||||||
|
form, err := url.ParseQuery(tc.edited)
|
||||||
|
require.NoError(t, err)
|
||||||
|
form.Set("name", "edited")
|
||||||
|
|
||||||
|
w := submitTargetEdit(env, webhook.ID, target.ID, form)
|
||||||
|
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
||||||
|
|
||||||
|
got := storedTarget(t, env, target.ID)
|
||||||
|
assert.Equal(t, "edited", got.Name)
|
||||||
|
assert.Equal(t, tc.want, targetSettingsOf(got))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestHandleTarget_ForwardQuery covers the HTTP target's setting that
|
// TestHandleTarget_ForwardQuery covers the HTTP target's setting that
|
||||||
// passes each event's query string on to it: the add target form
|
// passes each event's query string on to it: the add target form
|
||||||
// stores it checked, the edit form starts with it checked and turns it
|
// stores it checked, the edit form starts with it checked and turns it
|
||||||
@@ -467,7 +519,7 @@ func TestHandleTarget_ForwardQuery(t *testing.T) {
|
|||||||
|
|
||||||
targets := targetsForWebhook(t, env.db, webhook.ID)
|
targets := targetsForWebhook(t, env.db, webhook.ID)
|
||||||
require.Len(t, targets, 1)
|
require.Len(t, targets, 1)
|
||||||
assert.True(t, storedHTTPConfig(t, env, targets[0].ID).ForwardQuery)
|
assert.True(t, storedTarget(t, env, targets[0].ID).ForwardQuery)
|
||||||
|
|
||||||
w = serveTarget(
|
w = serveTarget(
|
||||||
env, http.MethodGet, targetsPath+"/"+targets[0].ID+"/edit", nil,
|
env, http.MethodGet, targetsPath+"/"+targets[0].ID+"/edit", nil,
|
||||||
@@ -479,7 +531,7 @@ func TestHandleTarget_ForwardQuery(t *testing.T) {
|
|||||||
|
|
||||||
w = submitTargetEdit(env, webhook.ID, targets[0].ID, edit)
|
w = submitTargetEdit(env, webhook.ID, targets[0].ID, edit)
|
||||||
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
||||||
assert.False(t, storedHTTPConfig(t, env, targets[0].ID).ForwardQuery)
|
assert.False(t, storedTarget(t, env, targets[0].ID).ForwardQuery)
|
||||||
|
|
||||||
edit.Set("url", editBlockedURL)
|
edit.Set("url", editBlockedURL)
|
||||||
edit.Set("forward_query", "on")
|
edit.Set("forward_query", "on")
|
||||||
@@ -568,7 +620,7 @@ func assertEditRejectsBlockedDestination(
|
|||||||
assert.Contains(t, w.Body.String(), "Invalid target URL")
|
assert.Contains(t, w.Body.String(), "Invalid target URL")
|
||||||
assert.Equal(
|
assert.Equal(
|
||||||
t, editOriginalURL,
|
t, editOriginalURL,
|
||||||
storedHTTPConfig(t, env, target.ID).URL,
|
storedTarget(t, env, target.ID).URL,
|
||||||
"a rejected edit must leave the stored config alone",
|
"a rejected edit must leave the stored config alone",
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
@@ -597,9 +649,9 @@ func assertEditIgnoresQueryString(
|
|||||||
|
|
||||||
assert.Equal(t, http.StatusBadRequest, w.Code)
|
assert.Equal(t, http.StatusBadRequest, w.Code)
|
||||||
|
|
||||||
cfg := storedHTTPConfig(t, env, target.ID)
|
stored := storedTarget(t, env, target.ID)
|
||||||
assert.Equal(t, editOriginalURL, cfg.URL)
|
assert.Equal(t, editOriginalURL, stored.URL)
|
||||||
assert.Empty(t, cfg.Headers)
|
assert.Empty(t, stored.Headers)
|
||||||
}
|
}
|
||||||
|
|
||||||
func assertEditRejectsHeaders(
|
func assertEditRejectsHeaders(
|
||||||
@@ -617,7 +669,7 @@ func assertEditRejectsHeaders(
|
|||||||
assert.Equal(t, http.StatusBadRequest, w.Code)
|
assert.Equal(t, http.StatusBadRequest, w.Code)
|
||||||
assert.Contains(t, w.Body.String(), "Invalid headers")
|
assert.Contains(t, w.Body.String(), "Invalid headers")
|
||||||
assert.Empty(
|
assert.Empty(
|
||||||
t, storedHTTPConfig(t, env, target.ID).Headers,
|
t, storedTarget(t, env, target.ID).Headers,
|
||||||
"a rejected header must not be stored",
|
"a rejected header must not be stored",
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
@@ -637,7 +689,7 @@ func assertEditRejectsTimeout(
|
|||||||
assert.Equal(t, http.StatusBadRequest, w.Code)
|
assert.Equal(t, http.StatusBadRequest, w.Code)
|
||||||
assert.Contains(t, w.Body.String(), "Invalid timeout")
|
assert.Contains(t, w.Body.String(), "Invalid timeout")
|
||||||
assert.Equal(
|
assert.Equal(
|
||||||
t, 9, storedHTTPConfig(t, env, target.ID).Timeout,
|
t, 9, storedTarget(t, env, target.ID).TimeoutSeconds,
|
||||||
"a rejected timeout must leave the stored one alone",
|
"a rejected timeout must leave the stored one alone",
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
@@ -757,7 +809,7 @@ func assertTargetOfAnotherWebhook404s(
|
|||||||
|
|
||||||
assert.Equal(
|
assert.Equal(
|
||||||
t, editOriginalURL,
|
t, editOriginalURL,
|
||||||
storedHTTPConfig(t, env, target.ID).URL,
|
storedTarget(t, env, target.ID).URL,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -778,7 +830,7 @@ func assertWebhookOfAnotherUser404s(
|
|||||||
|
|
||||||
target := seedConfiguredTarget(
|
target := seedConfiguredTarget(
|
||||||
t, env.db, other.ID, database.TargetTypeHTTP,
|
t, env.db, other.ID, database.TargetTypeHTTP,
|
||||||
`{"url":"`+editOriginalURL+`"}`,
|
database.Target{URL: editOriginalURL},
|
||||||
)
|
)
|
||||||
|
|
||||||
w := serveTarget(
|
w := serveTarget(
|
||||||
|
|||||||
@@ -243,10 +243,7 @@ func readTargetDeliveries(
|
|||||||
func (h *Handlers) archiveFileView(
|
func (h *Handlers) archiveFileView(
|
||||||
webhook *database.Webhook, target *database.Target, now time.Time,
|
webhook *database.Webhook, target *database.Target, now time.Time,
|
||||||
) *ArchiveFileView {
|
) *ArchiveFileView {
|
||||||
current, err := archive.PathAt(h.dbMgr, webhook, target, now)
|
current := archive.PathAt(h.dbMgr, webhook, target, now)
|
||||||
if err != nil {
|
|
||||||
return h.archiveUnreadable(&ArchiveFileView{}, target, err)
|
|
||||||
}
|
|
||||||
|
|
||||||
view := &ArchiveFileView{Name: filepath.Base(current)}
|
view := &ArchiveFileView{Name: filepath.Base(current)}
|
||||||
|
|
||||||
|
|||||||
@@ -60,7 +60,5 @@ func TestHandleTargetToggle_DoesNotUndoAnEdit(t *testing.T) {
|
|||||||
assert.False(t, stored.Active)
|
assert.False(t, stored.Active)
|
||||||
assert.Equal(t, "edited-name", stored.Name)
|
assert.Equal(t, "edited-name", stored.Name)
|
||||||
assert.Equal(t, 5, stored.MaxAttempts)
|
assert.Equal(t, 5, stored.MaxAttempts)
|
||||||
assert.Equal(
|
assert.Equal(t, editReplacedURL, stored.URL)
|
||||||
t, editReplacedURL, storedHTTPConfig(t, env, tgt.ID).URL,
|
|
||||||
)
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -425,21 +425,26 @@ func buildDeliveryTasks(
|
|||||||
}
|
}
|
||||||
|
|
||||||
tasks = append(tasks, delivery.Task{
|
tasks = append(tasks, delivery.Task{
|
||||||
DeliveryID: dlv.ID,
|
DeliveryID: dlv.ID,
|
||||||
EventID: event.ID,
|
EventID: event.ID,
|
||||||
WebhookID: event.WebhookID,
|
WebhookID: event.WebhookID,
|
||||||
EntrypointID: event.EntrypointID,
|
EntrypointID: event.EntrypointID,
|
||||||
TargetID: targets[i].ID,
|
TargetID: targets[i].ID,
|
||||||
TargetName: targets[i].Name,
|
TargetName: targets[i].Name,
|
||||||
TargetType: targets[i].Type,
|
TargetType: targets[i].Type,
|
||||||
TargetConfig: targets[i].Config,
|
TargetURL: targets[i].URL,
|
||||||
MaxAttempts: targets[i].MaxAttempts,
|
TargetHeaders: targets[i].Headers,
|
||||||
Method: event.Method,
|
TimeoutSeconds: targets[i].TimeoutSeconds,
|
||||||
RawQuery: event.RawQuery,
|
ForwardQuery: targets[i].ForwardQuery,
|
||||||
Headers: event.Headers,
|
ArchiveExpiry: targets[i].ArchiveExpiry,
|
||||||
ContentType: event.ContentType,
|
ArchiveRotation: targets[i].ArchiveRotation,
|
||||||
Body: bodyPtr,
|
MaxAttempts: targets[i].MaxAttempts,
|
||||||
AttemptNum: 1,
|
Method: event.Method,
|
||||||
|
RawQuery: event.RawQuery,
|
||||||
|
Headers: event.Headers,
|
||||||
|
ContentType: event.ContentType,
|
||||||
|
Body: bodyPtr,
|
||||||
|
AttemptNum: 1,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -115,13 +115,7 @@ func (h *Handlers) HandleSourceCreateSubmit() http.HandlerFunc {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
targets, errMsg, err := h.newWebhookTargets(r.Context(), in)
|
targets, errMsg := h.newWebhookTargets(r.Context(), in)
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to encode target config", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
if errMsg != "" {
|
if errMsg != "" {
|
||||||
refuse(errMsg)
|
refuse(errMsg)
|
||||||
|
|
||||||
@@ -147,7 +141,7 @@ func (h *Handlers) HandleSourceCreateSubmit() http.HandlerFunc {
|
|||||||
func (h *Handlers) newWebhookTargets(
|
func (h *Handlers) newWebhookTargets(
|
||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
in sourceFormInput,
|
in sourceFormInput,
|
||||||
) ([]*database.Target, string, error) {
|
) ([]*database.Target, string) {
|
||||||
var requested []targetFormInput
|
var requested []targetFormInput
|
||||||
|
|
||||||
if in.HTTPURL != "" {
|
if in.HTTPURL != "" {
|
||||||
@@ -170,15 +164,15 @@ func (h *Handlers) newWebhookTargets(
|
|||||||
targets := make([]*database.Target, 0, len(requested))
|
targets := make([]*database.Target, 0, len(requested))
|
||||||
|
|
||||||
for _, form := range requested {
|
for _, form := range requested {
|
||||||
target, errMsg, err := h.newTarget(ctx, "", form)
|
target, errMsg := h.newTarget(ctx, "", form)
|
||||||
if err != nil || errMsg != "" {
|
if errMsg != "" {
|
||||||
return nil, errMsg, err
|
return nil, errMsg
|
||||||
}
|
}
|
||||||
|
|
||||||
targets = append(targets, target)
|
targets = append(targets, target)
|
||||||
}
|
}
|
||||||
|
|
||||||
return targets, "", nil
|
return targets, ""
|
||||||
}
|
}
|
||||||
|
|
||||||
// createWebhookWithEntrypoint creates a webhook, its default
|
// createWebhookWithEntrypoint creates a webhook, its default
|
||||||
|
|||||||
@@ -109,7 +109,7 @@ func (h *Handlers) renderSourceDetail(
|
|||||||
data := map[string]any{
|
data := map[string]any{
|
||||||
tmplKeyWebhook: &webhook,
|
tmplKeyWebhook: &webhook,
|
||||||
// Targets are projected to a display-safe view: a
|
// Targets are projected to a display-safe view: a
|
||||||
// target's stored config blob holds a credential, and it
|
// target's URL and headers hold a credential, and they
|
||||||
// must never reach a template.
|
// must never reach a template.
|
||||||
"Entrypoints": entrypointViews,
|
"Entrypoints": entrypointViews,
|
||||||
"Targets": h.targetRows(&webhook, targets),
|
"Targets": h.targetRows(&webhook, targets),
|
||||||
|
|||||||
@@ -153,7 +153,7 @@ func seedStatsHistory(
|
|||||||
|
|
||||||
first := seedConfiguredTarget(
|
first := seedConfiguredTarget(
|
||||||
t, db, wh.ID, database.TargetTypeHTTP,
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
`{"url":"`+replayTargetURL+`"}`,
|
database.Target{URL: replayTargetURL},
|
||||||
)
|
)
|
||||||
second := seedTarget(t, db, wh.ID, database.TargetTypeLog)
|
second := seedTarget(t, db, wh.ID, database.TargetTypeLog)
|
||||||
|
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ import (
|
|||||||
"net/netip"
|
"net/netip"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
"github.com/go-chi/chi"
|
"github.com/go-chi/chi"
|
||||||
"github.com/google/uuid"
|
"github.com/google/uuid"
|
||||||
@@ -18,6 +19,7 @@ import (
|
|||||||
"gorm.io/gorm/clause"
|
"gorm.io/gorm/clause"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
"sneak.berlin/go/webhooker/internal/handlers"
|
"sneak.berlin/go/webhooker/internal/handlers"
|
||||||
"sneak.berlin/go/webhooker/internal/middleware"
|
"sneak.berlin/go/webhooker/internal/middleware"
|
||||||
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
|
"sneak.berlin/go/webhooker/internal/middleware/middlewaretest"
|
||||||
@@ -206,3 +208,71 @@ func receivedLine(t *testing.T, buf *bytes.Buffer) map[string]any {
|
|||||||
|
|
||||||
return found[0]
|
return found[0]
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// targetWithEverySetting returns target settings with all six set, the
|
||||||
|
// archive ones included, so a task that drops one of them differs from
|
||||||
|
// the target.
|
||||||
|
func targetWithEverySetting() database.Target {
|
||||||
|
return database.Target{
|
||||||
|
URL: replayTargetURL,
|
||||||
|
Headers: map[string]string{"X-Api-Key": "k"},
|
||||||
|
TimeoutSeconds: 7,
|
||||||
|
ForwardQuery: true,
|
||||||
|
ArchiveExpiry: time.Hour,
|
||||||
|
ArchiveRotation: "daily",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// assertTaskCarriesTargetSettings proves a delivery task carries every
|
||||||
|
// one of the target's settings. They are copied into the task one field
|
||||||
|
// at a time, and a field left out is sent empty without any error.
|
||||||
|
func assertTaskCarriesTargetSettings(
|
||||||
|
t *testing.T, task delivery.Task, target *database.Target,
|
||||||
|
) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
assert.Equal(t, target.ID, task.TargetID)
|
||||||
|
assert.Equal(t, target.URL, task.TargetURL)
|
||||||
|
assert.Equal(t, target.Headers, task.TargetHeaders)
|
||||||
|
assert.Equal(t, target.TimeoutSeconds, task.TimeoutSeconds)
|
||||||
|
assert.Equal(t, target.ForwardQuery, task.ForwardQuery)
|
||||||
|
assert.Equal(t, target.ArchiveExpiry, task.ArchiveExpiry)
|
||||||
|
assert.Equal(t, target.ArchiveRotation, task.ArchiveRotation)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHandleWebhook_TaskCarriesEveryTargetSetting proves the task a
|
||||||
|
// received event hands the delivery engine carries every one of the
|
||||||
|
// target's settings.
|
||||||
|
func TestHandleWebhook_TaskCarriesEveryTargetSetting(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
db *database.Database
|
||||||
|
notif *recordingNotifier
|
||||||
|
)
|
||||||
|
|
||||||
|
app := newTestApp(t, &h, &db, ¬if)
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
|
webhook := seedWebhook(t, db)
|
||||||
|
entrypoint := seedEntrypoint(t, db, webhook.ID)
|
||||||
|
target := seedConfiguredTarget(
|
||||||
|
t, db, webhook.ID, database.TargetTypeHTTP,
|
||||||
|
targetWithEverySetting(),
|
||||||
|
)
|
||||||
|
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
h.HandleWebhook().ServeHTTP(w, postRequest(
|
||||||
|
"/h/"+entrypoint.Path, nil,
|
||||||
|
map[string]string{"uuid": entrypoint.Path},
|
||||||
|
))
|
||||||
|
|
||||||
|
require.Equal(t, http.StatusOK, w.Code)
|
||||||
|
|
||||||
|
tasks := notif.Tasks()
|
||||||
|
require.Len(t, tasks, 1)
|
||||||
|
assertTaskCarriesTargetSettings(t, tasks[0], target)
|
||||||
|
}
|
||||||
|
|||||||
@@ -1183,16 +1183,26 @@ func checkNewWebhookTargets(
|
|||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
// Each value the pruning choice submits, after an empty one that
|
// Each value the pruning choice submits and the expiry it stores,
|
||||||
// stands for archive left off.
|
// after an empty one that stands for archive left off.
|
||||||
expiries := []string{
|
expiries := []struct {
|
||||||
"", "never", "1h", "12h", "24h", "720h", "2160h", "8760h",
|
value string
|
||||||
|
stored time.Duration
|
||||||
|
}{
|
||||||
|
{"", 0},
|
||||||
|
{"never", 0},
|
||||||
|
{"1h", time.Hour},
|
||||||
|
{"12h", 12 * time.Hour},
|
||||||
|
{"24h", 24 * time.Hour},
|
||||||
|
{"720h", 720 * time.Hour},
|
||||||
|
{"2160h", 2160 * time.Hour},
|
||||||
|
{"8760h", 8760 * time.Hour},
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, httpURL := range []string{"", publicTargetURL} {
|
for _, httpURL := range []string{"", publicTargetURL} {
|
||||||
for _, expiry := range expiries {
|
for _, expiry := range expiries {
|
||||||
name := "url=" + httpURL + " archive=" + expiry
|
name := "url=" + httpURL + " archive=" + expiry.value
|
||||||
want := map[database.TargetType]string{}
|
want := map[database.TargetType]database.Target{}
|
||||||
|
|
||||||
require.NoError(t, chromedp.Run(
|
require.NoError(t, chromedp.Run(
|
||||||
ctx,
|
ctx,
|
||||||
@@ -1205,10 +1215,12 @@ func checkNewWebhookTargets(
|
|||||||
"#http_url", httpURL, chromedp.ByQuery,
|
"#http_url", httpURL, chromedp.ByQuery,
|
||||||
)))
|
)))
|
||||||
|
|
||||||
want[database.TargetTypeHTTP] = `{"url":"` + httpURL + `"}`
|
want[database.TargetTypeHTTP] = database.Target{
|
||||||
|
URL: httpURL, Headers: map[string]string{},
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if expiry != "" {
|
if expiry.value != "" {
|
||||||
// The choice showing moves Create down, so it is
|
// The choice showing moves Create down, so it is
|
||||||
// waited for before Create is clicked.
|
// waited for before Create is clicked.
|
||||||
click(ctx, t, archiveBox)
|
click(ctx, t, archiveBox)
|
||||||
@@ -1216,29 +1228,30 @@ func checkNewWebhookTargets(
|
|||||||
"%s: checking archive does not show the pruning choice",
|
"%s: checking archive does not show the pruning choice",
|
||||||
name)
|
name)
|
||||||
require.NoError(t, chromedp.Run(ctx, chromedp.SetValue(
|
require.NoError(t, chromedp.Run(ctx, chromedp.SetValue(
|
||||||
pruningChoice, expiry, chromedp.BySearch,
|
pruningChoice, expiry.value, chromedp.BySearch,
|
||||||
)))
|
)))
|
||||||
|
|
||||||
// The rotation choice is left on none.
|
// The rotation choice is left on none.
|
||||||
want[database.TargetTypeDatabase] = `{"expiry":"` + expiry +
|
want[database.TargetTypeDatabase] = database.Target{
|
||||||
`","rotation":"none"}`
|
ArchiveExpiry: expiry.stored, ArchiveRotation: "none",
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
clickAndLoad(ctx, t, createButton)
|
clickAndLoad(ctx, t, createButton)
|
||||||
require.Truef(t, shown(ctx, `//h1[text()="`+name+`"]`),
|
require.Truef(t, shown(ctx, `//h1[text()="`+name+`"]`),
|
||||||
"%s: the new webhook's page does not open", name)
|
"%s: the new webhook's page does not open", name)
|
||||||
|
|
||||||
assert.Equalf(t, want, targetConfigs(t, env, name),
|
assert.Equalf(t, want, targetSettings(t, env, name),
|
||||||
"%s: the webhook does not have the targets asked for", name)
|
"%s: the webhook does not have the targets asked for", name)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// targetConfigs reads the targets of the webhook named name, and
|
// targetSettings reads the targets of the webhook named name, and
|
||||||
// returns each one's stored configuration by its type.
|
// returns each one's stored settings by its type.
|
||||||
func targetConfigs(
|
func targetSettings(
|
||||||
t *testing.T, env *testEnv, name string,
|
t *testing.T, env *testEnv, name string,
|
||||||
) map[database.TargetType]string {
|
) map[database.TargetType]database.Target {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
var (
|
var (
|
||||||
@@ -1251,12 +1264,19 @@ func targetConfigs(
|
|||||||
require.NoError(t, env.db.DB().
|
require.NoError(t, env.db.DB().
|
||||||
Where("webhook_id = ?", webhook.ID).Find(&targets).Error)
|
Where("webhook_id = ?", webhook.ID).Find(&targets).Error)
|
||||||
|
|
||||||
configs := map[database.TargetType]string{}
|
settings := map[database.TargetType]database.Target{}
|
||||||
for _, target := range targets {
|
for _, target := range targets {
|
||||||
configs[target.Type] = target.Config
|
settings[target.Type] = database.Target{
|
||||||
|
URL: target.URL,
|
||||||
|
Headers: target.Headers,
|
||||||
|
TimeoutSeconds: target.TimeoutSeconds,
|
||||||
|
ForwardQuery: target.ForwardQuery,
|
||||||
|
ArchiveExpiry: target.ArchiveExpiry,
|
||||||
|
ArchiveRotation: target.ArchiveRotation,
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return configs
|
return settings
|
||||||
}
|
}
|
||||||
|
|
||||||
// checkRefusedNewWebhook submits the new webhook page with archive
|
// checkRefusedNewWebhook submits the new webhook page with archive
|
||||||
|
|||||||
@@ -311,7 +311,7 @@ func (s *Server) setupSourceRoutes() {
|
|||||||
r.Post("/targets", s.h.HandleTargetCreate())
|
r.Post("/targets", s.h.HandleTargetCreate())
|
||||||
// The edit form is the one page that renders a target's
|
// The edit form is the one page that renders a target's
|
||||||
// destination URL and header values in full; see
|
// destination URL and header values in full; see
|
||||||
// delivery.TargetConfigForm. It belongs to this group for
|
// Handlers.HandleTargetEdit. It belongs to this group for
|
||||||
// its RequireAuth and NoCache, which are what keep that
|
// its RequireAuth and NoCache, which are what keep that
|
||||||
// exception from reaching an unauthenticated request or a
|
// exception from reaching an unauthenticated request or a
|
||||||
// shared cache.
|
// shared cache.
|
||||||
|
|||||||
@@ -414,7 +414,7 @@ func (e *testEnv) seedTarget(
|
|||||||
Name: "routed-target",
|
Name: "routed-target",
|
||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Active: true,
|
Active: true,
|
||||||
Config: `{"url":"http://93.184.216.34/hook"}`,
|
URL: "http://93.184.216.34/hook",
|
||||||
MaxAttempts: 1,
|
MaxAttempts: 1,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user