Compare commits
2
Commits
78cda941b4
...
46399d7d7b
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
46399d7d7b | ||
|
|
ff24638ba4 |
No files matched your search
@@ -1127,7 +1127,7 @@ unconditionally against whatever files it finds:
|
|||||||
- the main database on connect — `Setting`, `User`, `APIKey`, `Webhook`,
|
- the main database on connect — `Setting`, `User`, `APIKey`, `Webhook`,
|
||||||
`Entrypoint`, `Target`
|
`Entrypoint`, `Target`
|
||||||
- each event database when it is lazily opened — `Event`, `Delivery`,
|
- each event database when it is lazily opened — `Event`, `Delivery`,
|
||||||
`DeliveryResult`, `EventTotals`, `TargetTotals`
|
`DeliveryResult`, `EventTotals`, `TargetTotals`, `EntrypointTotals`
|
||||||
- each archive database on every open and reopen
|
- each archive database on every open and reopen
|
||||||
|
|
||||||
There is no schema version table, no migration ledger, and no down
|
There is no schema version table, no migration ledger, and no down
|
||||||
@@ -1531,6 +1531,9 @@ tier** (event ingestion, delivery, and logging).
|
|||||||
│ ┌──────────────┐ (one row per target: running counts │
|
│ ┌──────────────┐ (one row per target: running counts │
|
||||||
│ │ TargetTotals │ of its deliveries) │
|
│ │ TargetTotals │ of its deliveries) │
|
||||||
│ └──────────────┘ │
|
│ └──────────────┘ │
|
||||||
|
│ ┌──────────────────┐ (one row per entrypoint: when the │
|
||||||
|
│ │ EntrypointTotals │ last event arrived on its URL) │
|
||||||
|
│ └──────────────────┘ │
|
||||||
└─────────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -1577,6 +1580,13 @@ more entrypoints (receiver URLs) and one or more targets (delivery
|
|||||||
destinations) into a logical unit. A user creates a webhook to set up
|
destinations) into a logical unit. A user creates a webhook to set up
|
||||||
event routing.
|
event routing.
|
||||||
|
|
||||||
|
The new webhook form can also give the webhook its first targets: an
|
||||||
|
optional HTTP target URL creates an `http` target named `HTTP`, and the
|
||||||
|
archive checkbox creates a `database` target named `Archive` whose
|
||||||
|
`expiry` is the pruning chosen beside it (never, 1h, 12h, 24h, 30d, 90d
|
||||||
|
or 365d). Both are validated as on the add target form, and the webhook
|
||||||
|
and its targets are created together or not at all.
|
||||||
|
|
||||||
| Field | Type | Description |
|
| Field | Type | Description |
|
||||||
| ---------------- | ------- | ----------- |
|
| ---------------- | ------- | ----------- |
|
||||||
| `id` | UUID | Primary key |
|
| `id` | UUID | Primary key |
|
||||||
@@ -1644,6 +1654,11 @@ different event sources that all feed into the same processing pipeline
|
|||||||
(e.g., one entrypoint for GitHub, another for Stripe, both routing to
|
(e.g., one entrypoint for GitHub, another for Stripe, both routing to
|
||||||
the same targets).
|
the same targets).
|
||||||
|
|
||||||
|
The webhook page shows, for each entrypoint, when the last event arrived
|
||||||
|
on its URL, which retention leaves in place, or "never" if none ever has,
|
||||||
|
and how many events arrived on it within the webhook's retention period.
|
||||||
|
A resubmitted event did not arrive on the URL and counts in neither.
|
||||||
|
|
||||||
#### Target
|
#### Target
|
||||||
|
|
||||||
A delivery destination for events. Each target defines where and how
|
A delivery destination for events. Each target defines where and how
|
||||||
@@ -1853,10 +1868,11 @@ retries) is individually logged for full observability.
|
|||||||
|
|
||||||
**Relations:** Belongs to Delivery.
|
**Relations:** Belongs to Delivery.
|
||||||
|
|
||||||
#### EventTotals and TargetTotals
|
#### EventTotals, TargetTotals and EntrypointTotals
|
||||||
|
|
||||||
Running counts in each event database, read by the statistics pane at the
|
Running counts in each event database, read by the statistics pane at the
|
||||||
top of the webhook page and by the webhook list. `EventTotals` is one row:
|
top of the webhook page and by the webhook list, and each entrypoint's last
|
||||||
|
event, read by the webhook page's entrypoint list. `EventTotals` is one row:
|
||||||
|
|
||||||
| Field | Type | Description |
|
| Field | Type | Description |
|
||||||
| ---------------- | --------- | ----------- |
|
| ---------------- | --------- | ----------- |
|
||||||
@@ -1875,18 +1891,26 @@ top of the webhook page and by the webhook list. `EventTotals` is one row:
|
|||||||
| `deliveries_removed` | integer | Its deliveries retention has deleted |
|
| `deliveries_removed` | integer | Its deliveries retention has deleted |
|
||||||
| `failed_removed` | integer | Its failed deliveries retention has deleted |
|
| `failed_removed` | integer | Its failed deliveries retention has deleted |
|
||||||
|
|
||||||
Each count changes in the transaction that writes or deletes the rows it
|
`EntrypointTotals` is one row per entrypoint, created by the first event
|
||||||
counts. The pane's lifetime events are `events`, and its lifetime
|
that arrives on its URL:
|
||||||
deliveries and failures are `deliveries` and `failed` summed over the
|
|
||||||
targets; each figure within retention is the same less what retention
|
| Field | Type | Description |
|
||||||
removed, so neither needs the rows themselves. Its last event is
|
| --------------- | --------- | ----------- |
|
||||||
`last_event_at`, written in the transaction that stores the event, so it
|
| `entrypoint_id` | UUID | The entrypoint (primary key) |
|
||||||
still shows once retention has removed every event. Its last-10-minutes and
|
| `last_event_at` | timestamp | When the newest event arrived on its URL; a resubmitted event leaves it as it is, and so does retention |
|
||||||
last-24-hours figures are counted from the `events` and `deliveries`
|
|
||||||
indexes over just that window, the deliveries in one query grouped by
|
Each count changes in the transaction that writes or deletes the rows it counts,
|
||||||
target. Its failure percentage for a window is the deliveries that became
|
and each `last_event_at` in the transaction that stores the event. The pane's
|
||||||
`failed` in it out of all that became `delivered` or `failed` in it, and
|
lifetime events are `events`, and its lifetime deliveries and failures are
|
||||||
a dash when none did.
|
`deliveries` and `failed` summed over the targets; each figure within retention
|
||||||
|
is the same less what retention removed, so neither needs the rows themselves.
|
||||||
|
Its last event is `last_event_at` in `EventTotals`, so it still shows once
|
||||||
|
retention has removed every event; each entrypoint's last event, from
|
||||||
|
`EntrypointTotals`, does too. Its last-10-minutes and last-24-hours figures are
|
||||||
|
counted from the `events` and `deliveries` indexes over just that window, the
|
||||||
|
deliveries in one query grouped by target. Its failure percentage for a window
|
||||||
|
is the deliveries that became `failed` in it out of all that became `delivered`
|
||||||
|
or `failed` in it, and a dash when none did.
|
||||||
|
|
||||||
The webhook list at `/hooks` shows three of the pane's figures for each
|
The webhook list at `/hooks` shows three of the pane's figures for each
|
||||||
webhook: its events within retention and its last event, both from
|
webhook: its events within retention and its last event, both from
|
||||||
@@ -1908,26 +1932,31 @@ tags, so `AutoMigrate` creates them on a fresh database:
|
|||||||
| `delivery_results` | `delivery_id`, `deleted_at` | The event log, which loads the attempts of a page's deliveries, and retention, which deletes the attempts of expired events |
|
| `delivery_results` | `delivery_id`, `deleted_at` | The event log, which loads the attempts of a page's deliveries, and retention, which deletes the attempts of expired events |
|
||||||
| `events` | `deleted_at`, `created_at` | The webhook page's statistics, which count recent events |
|
| `events` | `deleted_at`, `created_at` | The webhook page's statistics, which count recent events |
|
||||||
| `events` | `resubmitted_from_id`, `deleted_at` | The event log, which counts the events resubmitted from each event on a page |
|
| `events` | `resubmitted_from_id`, `deleted_at` | The event log, which counts the events resubmitted from each event on a page |
|
||||||
|
| `events` | `entrypoint_id`, `deleted_at`, `resubmitted_from_id`, `created_at` | The webhook page's entrypoint list, which counts the events that arrived on each entrypoint's URL within the retention period |
|
||||||
| `events` | `created_at` | Retention, which selects expired events by age |
|
| `events` | `created_at` | Retention, which selects expired events by age |
|
||||||
|
|
||||||
GORM's soft delete adds `deleted_at IS NULL` to these queries; retention
|
GORM's soft delete adds `deleted_at IS NULL` to these queries; retention leaves
|
||||||
leaves it out. SQLite keeps no statistics on these tables, and without them it
|
it out. SQLite keeps no statistics on these tables, and without them it rates
|
||||||
rates the `deleted_at` index, which every live row matches, above an index on
|
the `deleted_at` index, which every live row matches, above an index on a column
|
||||||
a column matched against several values or compared with a range. So every
|
matched against several values or compared with a range. So every index but the
|
||||||
index but the last also covers `deleted_at`. It comes second in the `event_id`
|
last also covers `deleted_at`. It comes second in the `event_id` and
|
||||||
and `delivery_id` indexes, so that retention can use them without it. The event
|
`delivery_id` indexes, so that retention can use them without it. The event
|
||||||
log's count, the one query on the `resubmitted_from_id` index, always carries
|
log's count, the one query on the `resubmitted_from_id` index, always carries
|
||||||
`deleted_at IS NULL` and uses both columns. In the statistics' `events` index
|
`deleted_at IS NULL` and uses both columns. The entrypoint list's count, the one
|
||||||
`deleted_at` comes first, because they compare `created_at` with a range (`>=`)
|
query on the `entrypoint_id` index, uses all four, `resubmitted_from_id IS NULL`
|
||||||
and SQLite narrows by a range only on the last column it uses.
|
leaving out resubmitted copies and `created_at` last because it compares it with
|
||||||
|
a range (`>=`). In the statistics' `events` index `deleted_at` comes first,
|
||||||
|
because they compare `created_at` with a range (`>=`) and SQLite narrows by a
|
||||||
|
range only on the last column it uses.
|
||||||
|
|
||||||
#### Common Fields
|
#### Common Fields
|
||||||
|
|
||||||
Every entity except `Setting`, `EventTotals` and `TargetTotals` includes
|
Every entity except `Setting`, `EventTotals`, `TargetTotals` and
|
||||||
these fields from `BaseModel`. `Setting` is a bare key-value row with no
|
`EntrypointTotals` includes these fields from `BaseModel`. `Setting` is a bare
|
||||||
`id`, no timestamps and no soft delete, and the two totals tables hold
|
key-value row with no `id`, no timestamps and no soft delete. Of the three
|
||||||
counts, plus `last_event_at` in `event_totals`, keyed by a numeric `id`
|
totals tables, `event_totals` holds counts and `last_event_at`, keyed by a
|
||||||
and by `target_id`:
|
numeric `id`; `target_totals` holds counts, keyed by `target_id`; and
|
||||||
|
`entrypoint_totals` holds `last_event_at`, keyed by `entrypoint_id`:
|
||||||
|
|
||||||
| Field | Type | Description |
|
| Field | Type | Description |
|
||||||
| ------------ | --------- | ----------- |
|
| ------------ | --------- | ----------- |
|
||||||
@@ -1969,8 +1998,9 @@ encryption key is generated and stored, and an `admin` user is created.
|
|||||||
- **Events** — captured incoming webhook payloads
|
- **Events** — captured incoming webhook payloads
|
||||||
- **Deliveries** — event-to-target pairings and their status
|
- **Deliveries** — event-to-target pairings and their status
|
||||||
- **DeliveryResults** — individual delivery attempt logs
|
- **DeliveryResults** — individual delivery attempt logs
|
||||||
- **EventTotals** and **TargetTotals** — running counts of the above,
|
- **EventTotals**, **TargetTotals** and **EntrypointTotals** — running
|
||||||
the deliveries per target, kept through retention
|
counts of the above, the deliveries per target, and each entrypoint's
|
||||||
|
last event, kept through retention
|
||||||
|
|
||||||
Per-webhook databases are created automatically when a webhook is
|
Per-webhook databases are created automatically when a webhook is
|
||||||
created. They are managed by the `WebhookDBManager` component, which
|
created. They are managed by the `WebhookDBManager` component, which
|
||||||
@@ -3035,7 +3065,7 @@ webhooker/
|
|||||||
│ │ ├── model_event.go # Event entity (per-webhook DB)
|
│ │ ├── model_event.go # Event entity (per-webhook DB)
|
||||||
│ │ ├── model_delivery.go # Delivery entity (per-webhook DB)
|
│ │ ├── model_delivery.go # Delivery entity (per-webhook DB)
|
||||||
│ │ ├── model_delivery_result.go # DeliveryResult entity (per-webhook DB)
|
│ │ ├── model_delivery_result.go # DeliveryResult entity (per-webhook DB)
|
||||||
│ │ ├── model_totals.go # EventTotals and TargetTotals (per-webhook DB)
|
│ │ ├── model_totals.go # EventTotals, TargetTotals and EntrypointTotals (per-webhook DB)
|
||||||
│ │ ├── model_apikey.go # APIKey entity
|
│ │ ├── model_apikey.go # APIKey entity
|
||||||
│ │ ├── password.go # Argon2id hashing and verification
|
│ │ ├── password.go # Argon2id hashing and verification
|
||||||
│ │ ├── retention.go # Retention reaper (per-webhook event expiry)
|
│ │ ├── retention.go # Retention reaper (per-webhook event expiry)
|
||||||
@@ -3311,9 +3341,9 @@ check, see [The login endpoint](#the-login-endpoint).
|
|||||||
`ENTRYPOINT` script, which sets the data directory's owner and mode
|
`ENTRYPOINT` script, which sets the data directory's owner and mode
|
||||||
before the app starts; the image's health check; and `docker exec`,
|
before the app starts; the image's health check; and `docker exec`,
|
||||||
unless given `--user`
|
unless given `--user`
|
||||||
- GORM soft deletes on every entity that carries `BaseModel`, which is
|
- GORM soft deletes on every entity that carries `BaseModel`, which is all of
|
||||||
all of them but `Setting`, `EventTotals` and `TargetTotals` (data
|
them but `Setting`, `EventTotals`, `TargetTotals` and `EntrypointTotals`
|
||||||
preserved for audit)
|
(data preserved for audit)
|
||||||
|
|
||||||
### Shutdown
|
### Shutdown
|
||||||
|
|
||||||
|
|||||||
@@ -233,6 +233,43 @@ func TestResubmitCountUsesItsIndex(t *testing.T) {
|
|||||||
"(resubmitted_from_id=? AND deleted_at=?)")
|
"(resubmitted_from_id=? AND deleted_at=?)")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestEntrypointEventsUseTheirIndex does the same for the webhook
|
||||||
|
// page's count, for each entrypoint, of the events that arrived on its
|
||||||
|
// URL since the retention cutoff (addEntrypointEvents in the
|
||||||
|
// handlers), which must come from the index alone. It passes 25
|
||||||
|
// entrypoints, as TestResubmitCountUsesItsIndex passes 25 events.
|
||||||
|
func TestEntrypointEventsUseTheirIndex(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
mgr, lc := setupTestWebhookDBManager(t)
|
||||||
|
ctx := context.Background()
|
||||||
|
require.NoError(t, lc.Start(ctx))
|
||||||
|
|
||||||
|
defer func() { require.NoError(t, lc.Stop(ctx)) }()
|
||||||
|
|
||||||
|
db, err := mgr.GetDB(uuid.New().String())
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
dry := db.Session(&gorm.Session{DryRun: true})
|
||||||
|
|
||||||
|
entrypoints := make([]string, 25)
|
||||||
|
for i := range entrypoints {
|
||||||
|
entrypoints[i] = uuid.New().String()
|
||||||
|
}
|
||||||
|
|
||||||
|
var rows []struct{ Events int }
|
||||||
|
|
||||||
|
assertPlanUses(t, db, dry.Model(&database.Event{}).
|
||||||
|
Select("entrypoint_id, count(*) AS events").
|
||||||
|
Where("entrypoint_id IN ? AND resubmitted_from_id IS NULL",
|
||||||
|
entrypoints).
|
||||||
|
Where("created_at >= ?", time.Now()).
|
||||||
|
Group("entrypoint_id").Find(&rows),
|
||||||
|
"COVERING INDEX idx_events_entrypoint_id "+
|
||||||
|
"(entrypoint_id=? AND deleted_at=? AND "+
|
||||||
|
"resubmitted_from_id=? AND created_at>?)")
|
||||||
|
}
|
||||||
|
|
||||||
// assertPlanUses asserts that SQLite's plan for a statement GORM built
|
// assertPlanUses asserts that SQLite's plan for a statement GORM built
|
||||||
// in a dry run, run with the same SQL and arguments GORM would send,
|
// in a dry run, run with the same SQL and arguments GORM would send,
|
||||||
// names each of the given indexes.
|
// names each of the given indexes.
|
||||||
|
|||||||
@@ -20,12 +20,15 @@ type Event struct {
|
|||||||
// has no deleted_at condition and uses the index on created_at
|
// has no deleted_at condition and uses the index on created_at
|
||||||
// alone. The other tables keep the unindexed BaseModel created_at.
|
// alone. The other tables keep the unindexed BaseModel created_at.
|
||||||
// DeletedAt is also the second column of the resubmitted_from_id
|
// DeletedAt is also the second column of the resubmitted_from_id
|
||||||
// index, for the reason DeliveryResult gives.
|
// index, for the reason DeliveryResult gives. The entrypoint_id
|
||||||
CreatedAt time.Time `gorm:"index;index:idx_events_deleted_at_created_at,priority:2" json:"createdAt"`
|
// index, for the webhook page's entrypoint list, has it second too,
|
||||||
DeletedAt gorm.DeletedAt `gorm:"index:idx_events_deleted_at_created_at,priority:1;index:idx_events_resubmitted_from_id,priority:2" json:"deletedAt,omitzero"`
|
// resubmitted_from_id third, and created_at last, which the list
|
||||||
|
// compares with a range.
|
||||||
|
CreatedAt time.Time `gorm:"index;index:idx_events_deleted_at_created_at,priority:2;index:idx_events_entrypoint_id,priority:4" json:"createdAt"`
|
||||||
|
DeletedAt gorm.DeletedAt `gorm:"index:idx_events_deleted_at_created_at,priority:1;index:idx_events_resubmitted_from_id,priority:2;index:idx_events_entrypoint_id,priority:2" json:"deletedAt,omitzero"`
|
||||||
|
|
||||||
WebhookID string `gorm:"type:uuid;not null" json:"webhookId"`
|
WebhookID string `gorm:"type:uuid;not null" json:"webhookId"`
|
||||||
EntrypointID string `gorm:"type:uuid;not null" json:"entrypointId"`
|
EntrypointID string `gorm:"type:uuid;not null;index:idx_events_entrypoint_id,priority:1" json:"entrypointId"`
|
||||||
|
|
||||||
// Request data
|
// Request data
|
||||||
Method string `gorm:"not null" json:"method"`
|
Method string `gorm:"not null" json:"method"`
|
||||||
@@ -44,7 +47,7 @@ type Event struct {
|
|||||||
// existed. It is not a foreign key: the source event can be
|
// existed. It is not a foreign key: the source event can be
|
||||||
// reaped by retention while its copies remain, and the id is
|
// reaped by retention while its copies remain, and the id is
|
||||||
// kept as the record of where the copy came from either way.
|
// kept as the record of where the copy came from either way.
|
||||||
ResubmittedFromID *string `gorm:"type:uuid;index:idx_events_resubmitted_from_id,priority:1" json:"resubmittedFromId,omitempty"`
|
ResubmittedFromID *string `gorm:"type:uuid;index:idx_events_resubmitted_from_id,priority:1;index:idx_events_entrypoint_id,priority:3" json:"resubmittedFromId,omitempty"`
|
||||||
|
|
||||||
// Relations. No model marshals the record it belongs to, so
|
// Relations. No model marshals the record it belongs to, so
|
||||||
// Webhook and Entrypoint are left out of the JSON.
|
// Webhook and Entrypoint are left out of the JSON.
|
||||||
|
|||||||
@@ -52,6 +52,21 @@ func (TargetTotals) TableName() string {
|
|||||||
return "target_totals"
|
return "target_totals"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// EntrypointTotals is one row per entrypoint, created by the first
|
||||||
|
// event that arrives on its URL: when the newest such event arrived,
|
||||||
|
// which retention leaves as it is. A resubmitted copy did not arrive
|
||||||
|
// on the URL and does not change it.
|
||||||
|
type EntrypointTotals struct {
|
||||||
|
EntrypointID string `gorm:"type:uuid;primaryKey"`
|
||||||
|
|
||||||
|
LastEventAt time.Time `gorm:"not null"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// TableName names the table AddEntrypointTotals updates.
|
||||||
|
func (EntrypointTotals) TableName() string {
|
||||||
|
return "entrypoint_totals"
|
||||||
|
}
|
||||||
|
|
||||||
// AddEventTotals adds each count in add to the webhook's event totals,
|
// AddEventTotals adds each count in add to the webhook's event totals,
|
||||||
// and records add.LastEventAt as when the newest event arrived if it is
|
// and records add.LastEventAt as when the newest event arrived if it is
|
||||||
// set. Call it on the transaction that writes or deletes the events it
|
// set. Call it on the transaction that writes or deletes the events it
|
||||||
@@ -97,3 +112,25 @@ func AddTargetTotals(tx *gorm.DB, add TargetTotals) error {
|
|||||||
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// AddEntrypointTotals records add.LastEventAt as when the newest event
|
||||||
|
// arrived on the URL of the entrypoint add.EntrypointID names, creating
|
||||||
|
// its row the first time. Call it on the transaction that stores the
|
||||||
|
// event.
|
||||||
|
func AddEntrypointTotals(tx *gorm.DB, add EntrypointTotals) error {
|
||||||
|
err := tx.Exec(
|
||||||
|
`INSERT INTO entrypoint_totals (entrypoint_id, last_event_at)
|
||||||
|
VALUES (?, ?)
|
||||||
|
ON CONFLICT (entrypoint_id) DO UPDATE SET
|
||||||
|
last_event_at = excluded.last_event_at`,
|
||||||
|
add.EntrypointID, add.LastEventAt,
|
||||||
|
).Error
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf(
|
||||||
|
"adding to totals of entrypoint %s: %w",
|
||||||
|
add.EntrypointID, err,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -111,6 +111,13 @@ func (w *Webhook) RetainsForever() bool {
|
|||||||
return retainsForever(w.RetentionDays)
|
return retainsForever(w.RetentionDays)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// RetentionCutoff returns the time before which this webhook's events
|
||||||
|
// have expired, as the reaper computes it, and false when the webhook
|
||||||
|
// retains them forever.
|
||||||
|
func (w *Webhook) RetentionCutoff(now time.Time) (time.Time, bool) {
|
||||||
|
return retentionCutoff(now, w.RetentionDays)
|
||||||
|
}
|
||||||
|
|
||||||
// RetentionLabel returns the webhook's retention policy as display
|
// RetentionLabel returns the webhook's retention policy as display
|
||||||
// text, so that no template has to know about the sentinel value.
|
// text, so that no template has to know about the sentinel value.
|
||||||
func (w *Webhook) RetentionLabel() string {
|
func (w *Webhook) RetentionLabel() string {
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ package database
|
|||||||
// Migrate runs database migrations for the main application database.
|
// Migrate runs database migrations for the main application database.
|
||||||
// Only configuration-tier models are stored in the main database.
|
// Only configuration-tier models are stored in the main database.
|
||||||
// Event-tier models (Event, Delivery, DeliveryResult, EventTotals,
|
// Event-tier models (Event, Delivery, DeliveryResult, EventTotals,
|
||||||
// TargetTotals) live in
|
// TargetTotals, EntrypointTotals) live in
|
||||||
// per-webhook dedicated databases managed by WebhookDBManager.
|
// per-webhook dedicated databases managed by WebhookDBManager.
|
||||||
func (d *Database) Migrate() error {
|
func (d *Database) Migrate() error {
|
||||||
return d.db.AutoMigrate(
|
return d.db.AutoMigrate(
|
||||||
|
|||||||
@@ -49,7 +49,7 @@ var ErrSidecarNotRemoved = errors.New(
|
|||||||
// WebhookDBManager manages per-webhook SQLite database files
|
// WebhookDBManager manages per-webhook SQLite database files
|
||||||
// for event storage. Each webhook gets its own dedicated
|
// for event storage. Each webhook gets its own dedicated
|
||||||
// database containing Events, Deliveries, DeliveryResults and the
|
// database containing Events, Deliveries, DeliveryResults and the
|
||||||
// running totals of them (EventTotals, TargetTotals).
|
// running totals of them (EventTotals, TargetTotals, EntrypointTotals).
|
||||||
// Database connections are opened lazily and cached.
|
// Database connections are opened lazily and cached.
|
||||||
type WebhookDBManager struct {
|
type WebhookDBManager struct {
|
||||||
dataDir string
|
dataDir string
|
||||||
@@ -381,7 +381,7 @@ func (m *WebhookDBManager) openDB(
|
|||||||
// Run migrations for event-tier models only
|
// Run migrations for event-tier models only
|
||||||
err = db.AutoMigrate(
|
err = db.AutoMigrate(
|
||||||
&Event{}, &Delivery{}, &DeliveryResult{},
|
&Event{}, &Delivery{}, &DeliveryResult{},
|
||||||
&EventTotals{}, &TargetTotals{},
|
&EventTotals{}, &TargetTotals{}, &EntrypointTotals{},
|
||||||
)
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
_ = sqlDB.Close()
|
_ = sqlDB.Close()
|
||||||
|
|||||||
@@ -1,6 +1,11 @@
|
|||||||
package handlers
|
package handlers
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"fmt"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/dustin/go-humanize"
|
||||||
|
"gorm.io/gorm"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -11,6 +16,21 @@ type EntrypointView struct {
|
|||||||
Path string
|
Path string
|
||||||
Description string
|
Description string
|
||||||
Active bool
|
Active bool
|
||||||
|
|
||||||
|
// Events is how many events arrived on the entrypoint's URL within
|
||||||
|
// the webhook's retention period. LastEvent is when the newest
|
||||||
|
// event ever to arrive on it did, relative, and LastEventUTC the
|
||||||
|
// full time; both are empty when none ever did.
|
||||||
|
Events int64
|
||||||
|
LastEvent string
|
||||||
|
LastEventUTC string
|
||||||
|
}
|
||||||
|
|
||||||
|
// entrypointEvents is one entrypoint's count read by
|
||||||
|
// addEntrypointEvents.
|
||||||
|
type entrypointEvents struct {
|
||||||
|
EntrypointID string
|
||||||
|
Events int64
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewEntrypointViews projects entrypoints for rendering.
|
// NewEntrypointViews projects entrypoints for rendering.
|
||||||
@@ -32,3 +52,60 @@ func NewEntrypointViews(
|
|||||||
|
|
||||||
return views
|
return views
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// addEntrypointEvents fills in each view's event figures from the
|
||||||
|
// webhook's event database: when the last event arrived on its URL,
|
||||||
|
// from its EntrypointTotals row, and how many events arrived on it
|
||||||
|
// since the webhook's retention cutoff, counted in one query over the
|
||||||
|
// events' entrypoint_id index. Resubmitted copies did not arrive on
|
||||||
|
// the URL and are left out of both.
|
||||||
|
func addEntrypointEvents(
|
||||||
|
webhookDB *gorm.DB,
|
||||||
|
webhook *database.Webhook,
|
||||||
|
views []EntrypointView,
|
||||||
|
now time.Time,
|
||||||
|
) error {
|
||||||
|
ids := make([]string, len(views))
|
||||||
|
byID := make(map[string]*EntrypointView, len(views))
|
||||||
|
|
||||||
|
for i := range views {
|
||||||
|
ids[i] = views[i].ID
|
||||||
|
byID[views[i].ID] = &views[i]
|
||||||
|
}
|
||||||
|
|
||||||
|
var totals []database.EntrypointTotals
|
||||||
|
|
||||||
|
err := webhookDB.Where("entrypoint_id IN ?", ids).Find(&totals).Error
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("reading entrypoint totals: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
query := webhookDB.Model(&database.Event{}).
|
||||||
|
Select("entrypoint_id, count(*) AS events").
|
||||||
|
Where("entrypoint_id IN ? AND resubmitted_from_id IS NULL", ids)
|
||||||
|
|
||||||
|
cutoff, finite := webhook.RetentionCutoff(now)
|
||||||
|
if finite {
|
||||||
|
query = query.Where("created_at >= ?", cutoff)
|
||||||
|
}
|
||||||
|
|
||||||
|
var counts []entrypointEvents
|
||||||
|
|
||||||
|
err = query.Group("entrypoint_id").Find(&counts).Error
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("counting events by entrypoint: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, row := range totals {
|
||||||
|
view := byID[row.EntrypointID]
|
||||||
|
view.LastEvent = humanize.Time(row.LastEventAt)
|
||||||
|
view.LastEventUTC =
|
||||||
|
row.LastEventAt.UTC().Format(time.DateTime) + " UTC"
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, row := range counts {
|
||||||
|
byID[row.EntrypointID].Events = row.Events
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,197 @@
|
|||||||
|
package handlers_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/google/uuid"
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/handlers"
|
||||||
|
"sneak.berlin/go/webhooker/internal/logger"
|
||||||
|
"sneak.berlin/go/webhooker/internal/session"
|
||||||
|
)
|
||||||
|
|
||||||
|
// entrypointRow returns the part of a rendered webhook page from an
|
||||||
|
// entrypoint's URL to the next entrypoint's, which holds its figures.
|
||||||
|
func entrypointRow(t *testing.T, page, entrypointID string) string {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
_, row, found := strings.Cut(page, `id="entrypoint-url-`+entrypointID+`"`)
|
||||||
|
require.True(t, found)
|
||||||
|
|
||||||
|
row, _, _ = strings.Cut(row, `id="entrypoint-url-`)
|
||||||
|
|
||||||
|
return row
|
||||||
|
}
|
||||||
|
|
||||||
|
// lastEventShown matches an entrypoint row's last event arriving at at.
|
||||||
|
func lastEventShown(at time.Time) string {
|
||||||
|
return `Last Event:</span>\s*<span title="` +
|
||||||
|
at.UTC().Format(time.DateTime) + ` UTC">[^<]+</span>`
|
||||||
|
}
|
||||||
|
|
||||||
|
// eventsShown matches an entrypoint row's count of n events.
|
||||||
|
func eventsShown(n int) string {
|
||||||
|
return `Events Within Retention:</span>\s*<span>` +
|
||||||
|
strconv.Itoa(n) + `</span>`
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHandleSourceDetail_ShowsEntrypointEvents proves each entrypoint
|
||||||
|
// on the webhook page shows its own figures: how many events arrived
|
||||||
|
// through it within the webhook's retention period, leaving out one
|
||||||
|
// older than that, and when the newest arrived, or "never" for an
|
||||||
|
// entrypoint with none.
|
||||||
|
func TestHandleSourceDetail_ShowsEntrypointEvents(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
sess *session.Session
|
||||||
|
db *database.Database
|
||||||
|
dbMgr *database.WebhookDBManager
|
||||||
|
)
|
||||||
|
|
||||||
|
app := newTestApp(t, &h, &sess, &db, &dbMgr)
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
|
wh := &database.Webhook{
|
||||||
|
UserID: deleteTestUserID, Name: "figures", RetentionDays: 7,
|
||||||
|
}
|
||||||
|
require.NoError(t, db.DB().Omit(clause.Associations).Create(wh).Error)
|
||||||
|
|
||||||
|
webhookDB, err := dbMgr.GetDB(wh.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
entrypoint := func() *database.Entrypoint {
|
||||||
|
ep := &database.Entrypoint{
|
||||||
|
WebhookID: wh.ID, Path: uuid.New().String(), Active: true,
|
||||||
|
}
|
||||||
|
require.NoError(t,
|
||||||
|
db.DB().Omit(clause.Associations).Create(ep).Error)
|
||||||
|
|
||||||
|
return ep
|
||||||
|
}
|
||||||
|
|
||||||
|
// event stores an event that arrived on ep's URL age ago and
|
||||||
|
// records it as ep's last event, as the receiver does.
|
||||||
|
event := func(ep *database.Entrypoint, age time.Duration) time.Time {
|
||||||
|
e := &database.Event{
|
||||||
|
WebhookID: wh.ID,
|
||||||
|
EntrypointID: ep.ID,
|
||||||
|
Method: http.MethodPost,
|
||||||
|
}
|
||||||
|
e.CreatedAt = time.Now().Add(-age)
|
||||||
|
require.NoError(t,
|
||||||
|
webhookDB.Omit(clause.Associations).Create(e).Error)
|
||||||
|
require.NoError(t, database.AddEntrypointTotals(webhookDB,
|
||||||
|
database.EntrypointTotals{
|
||||||
|
EntrypointID: ep.ID, LastEventAt: e.CreatedAt,
|
||||||
|
}))
|
||||||
|
|
||||||
|
return e.CreatedAt
|
||||||
|
}
|
||||||
|
|
||||||
|
busy, quiet, unused := entrypoint(), entrypoint(), entrypoint()
|
||||||
|
|
||||||
|
event(busy, 8*24*time.Hour) // older than the 7 days kept
|
||||||
|
event(busy, 3*time.Hour)
|
||||||
|
busyLast := event(busy, time.Hour)
|
||||||
|
quietLast := event(quiet, 2*24*time.Hour)
|
||||||
|
|
||||||
|
body := renderSourceDetailPage(t, h, sess, wh.ID)
|
||||||
|
|
||||||
|
assert.Regexp(t, lastEventShown(busyLast), entrypointRow(t, body, busy.ID))
|
||||||
|
assert.Regexp(t, eventsShown(2), entrypointRow(t, body, busy.ID))
|
||||||
|
assert.Regexp(t, lastEventShown(quietLast), entrypointRow(t, body, quiet.ID))
|
||||||
|
assert.Regexp(t, eventsShown(1), entrypointRow(t, body, quiet.ID))
|
||||||
|
assert.Regexp(t, `Last Event:</span>\s*<span>never</span>`,
|
||||||
|
entrypointRow(t, body, unused.ID))
|
||||||
|
assert.Regexp(t, eventsShown(0), entrypointRow(t, body, unused.ID))
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHandleSourceDetail_EntrypointLastEventSurvivesRetention checks
|
||||||
|
// that once retention has removed every event that arrived on an
|
||||||
|
// entrypoint's URL, the entrypoint still shows when the last one
|
||||||
|
// arrived rather than "never".
|
||||||
|
func TestHandleSourceDetail_EntrypointLastEventSurvivesRetention(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
sess *session.Session
|
||||||
|
db *database.Database
|
||||||
|
dbMgr *database.WebhookDBManager
|
||||||
|
log *logger.Logger
|
||||||
|
)
|
||||||
|
|
||||||
|
app := newTestApp(t, &h, &sess, &db, &dbMgr, &log)
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
|
wh := &database.Webhook{
|
||||||
|
UserID: deleteTestUserID, Name: "swept", RetentionDays: 1,
|
||||||
|
}
|
||||||
|
require.NoError(t, db.DB().Omit(clause.Associations).Create(wh).Error)
|
||||||
|
|
||||||
|
ep := seedEntrypoint(t, db, wh.ID)
|
||||||
|
webhookDB, events := receiveEvents(t, h, dbMgr, wh.ID, ep.Path, 1)
|
||||||
|
arrived := events[0].CreatedAt
|
||||||
|
|
||||||
|
statsAge(t, webhookDB, events[0].ID, time.Now().Add(-50*time.Hour))
|
||||||
|
statsPrune(t, db, dbMgr, log, webhookDB)
|
||||||
|
require.Empty(t, listEvents(t, webhookDB))
|
||||||
|
|
||||||
|
row := entrypointRow(t, renderSourceDetailPage(t, h, sess, wh.ID), ep.ID)
|
||||||
|
assert.Regexp(t, lastEventShown(arrived), row)
|
||||||
|
assert.Regexp(t, eventsShown(0), row)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHandleSourceDetail_ResubmitLeavesEntrypointFigures checks that a
|
||||||
|
// resubmitted copy, which did not arrive on the entrypoint's URL,
|
||||||
|
// changes neither the entrypoint's last event nor its count.
|
||||||
|
func TestHandleSourceDetail_ResubmitLeavesEntrypointFigures(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
sess *session.Session
|
||||||
|
db *database.Database
|
||||||
|
dbMgr *database.WebhookDBManager
|
||||||
|
)
|
||||||
|
|
||||||
|
app := newTestApp(t, &h, &sess, &db, &dbMgr)
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
|
wh := seedWebhook(t, db)
|
||||||
|
ep := seedEntrypoint(t, db, wh.ID)
|
||||||
|
webhookDB, events := receiveEvents(t, h, dbMgr, wh.ID, ep.Path, 1)
|
||||||
|
arrived := events[0].CreatedAt
|
||||||
|
|
||||||
|
require.Equal(t, http.StatusSeeOther,
|
||||||
|
postResubmit(t, h, sess, wh.ID, events[0].ID).Code)
|
||||||
|
require.Len(t, listEvents(t, webhookDB), 2)
|
||||||
|
|
||||||
|
var totals database.EntrypointTotals
|
||||||
|
|
||||||
|
require.NoError(t, webhookDB.Take(&totals).Error)
|
||||||
|
assert.True(t, arrived.Equal(totals.LastEventAt))
|
||||||
|
|
||||||
|
row := entrypointRow(t, renderSourceDetailPage(t, h, sess, wh.ID), ep.ID)
|
||||||
|
assert.Regexp(t, lastEventShown(arrived), row)
|
||||||
|
assert.Regexp(t, eventsShown(1), row)
|
||||||
|
}
|
||||||
@@ -0,0 +1,207 @@
|
|||||||
|
package handlers_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"html"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/url"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
)
|
||||||
|
|
||||||
|
// submitCreateForm posts the new webhook form and returns the
|
||||||
|
// recorder.
|
||||||
|
func submitCreateForm(
|
||||||
|
env *sourceTestEnv, form url.Values,
|
||||||
|
) *httptest.ResponseRecorder {
|
||||||
|
req := formRequest("/hooks/new", env.cookies, form, nil)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
|
env.handlers.HandleSourceCreateSubmit().ServeHTTP(w, req)
|
||||||
|
|
||||||
|
return w
|
||||||
|
}
|
||||||
|
|
||||||
|
// assertNothingCreated checks that the main database holds no webhook,
|
||||||
|
// entrypoint or target.
|
||||||
|
func assertNothingCreated(t *testing.T, db *database.Database) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
for _, model := range []any{
|
||||||
|
&database.Webhook{}, &database.Entrypoint{}, &database.Target{},
|
||||||
|
} {
|
||||||
|
var count int64
|
||||||
|
|
||||||
|
require.NoError(t, db.DB().Model(model).Count(&count).Error)
|
||||||
|
assert.Zerof(t, count, "%T rows were created", model)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHandleSourceCreateSubmit_CreatesRequestedTargets submits the new
|
||||||
|
// webhook form with the HTTP target URL filled in or empty, and with
|
||||||
|
// the archive checkbox off or on with each pruning choice. The webhook
|
||||||
|
// gets an HTTP target only for a URL and a database target only for a
|
||||||
|
// checked archive. The pruning choice is always submitted, as the
|
||||||
|
// browser submits it while it is hidden, and is ignored when archive
|
||||||
|
// is off.
|
||||||
|
func TestHandleSourceCreateSubmit_CreatesRequestedTargets(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
env := setupSourceTest(t)
|
||||||
|
|
||||||
|
// Each value the archive pruning choice submits, after an empty
|
||||||
|
// one that stands for the archive checkbox left off.
|
||||||
|
expiries := []string{
|
||||||
|
"", "never", "1h", "12h", "24h", "720h", "2160h", "8760h",
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, httpURL := range []string{"", editOriginalURL} {
|
||||||
|
for _, expiry := range expiries {
|
||||||
|
name := "url=" + httpURL + " archive=" + expiry
|
||||||
|
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
form := url.Values{}
|
||||||
|
form.Set("name", name)
|
||||||
|
form.Set("http_url", httpURL)
|
||||||
|
form.Set("archive_expiry", "720h")
|
||||||
|
|
||||||
|
if expiry != "" {
|
||||||
|
form.Set("archive", "on")
|
||||||
|
form.Set("archive_expiry", expiry)
|
||||||
|
}
|
||||||
|
|
||||||
|
w := submitCreateForm(env, form)
|
||||||
|
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
||||||
|
|
||||||
|
var webhook database.Webhook
|
||||||
|
|
||||||
|
require.NoError(t, env.db.DB().
|
||||||
|
Where("name = ?", name).First(&webhook).Error)
|
||||||
|
|
||||||
|
byType := map[database.TargetType]database.Target{}
|
||||||
|
for _, target := range targetsForWebhook(t, env.db, webhook.ID) {
|
||||||
|
byType[target.Type] = target
|
||||||
|
}
|
||||||
|
|
||||||
|
wantCount := 0
|
||||||
|
|
||||||
|
if httpURL != "" {
|
||||||
|
wantCount++
|
||||||
|
|
||||||
|
assert.Equal(t, "HTTP", byType[database.TargetTypeHTTP].Name)
|
||||||
|
assert.JSONEq(t, `{"url":"`+httpURL+`"}`,
|
||||||
|
byType[database.TargetTypeHTTP].Config)
|
||||||
|
}
|
||||||
|
|
||||||
|
if expiry != "" {
|
||||||
|
wantCount++
|
||||||
|
|
||||||
|
assert.Equal(t, "Archive",
|
||||||
|
byType[database.TargetTypeDatabase].Name)
|
||||||
|
assert.JSONEq(t, `{"expiry":"`+expiry+`"}`,
|
||||||
|
byType[database.TargetTypeDatabase].Config)
|
||||||
|
}
|
||||||
|
|
||||||
|
assert.Len(t, byType, wantCount)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHandleSourceCreateSubmit_RefusedFormKeepsEveryValue refuses the
|
||||||
|
// new webhook form for an invalid HTTP target URL and for an invalid
|
||||||
|
// retention, each with archive on. Nothing is created, and the form
|
||||||
|
// comes back with the reason and every value entered: name,
|
||||||
|
// description, retention, URL, the checked archive box and the pruning
|
||||||
|
// choice.
|
||||||
|
func TestHandleSourceCreateSubmit_RefusedFormKeepsEveryValue(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const badURL = "Invalid target URL"
|
||||||
|
|
||||||
|
cases := []struct {
|
||||||
|
name string
|
||||||
|
retention string
|
||||||
|
httpURL string
|
||||||
|
reason string
|
||||||
|
}{
|
||||||
|
{"blocked url", "7", editBlockedURL, badURL},
|
||||||
|
{"unsupported scheme", "7", "ftp://93.184.216.34/hook", badURL},
|
||||||
|
{"bad retention", "-5", editOriginalURL, "Retention must be"},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range cases {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
env := setupSourceTest(t)
|
||||||
|
|
||||||
|
form := url.Values{}
|
||||||
|
form.Set("name", "kept name")
|
||||||
|
form.Set("description", "kept description")
|
||||||
|
form.Set("retention_days", tc.retention)
|
||||||
|
form.Set("http_url", tc.httpURL)
|
||||||
|
form.Set("archive", "on")
|
||||||
|
form.Set("archive_expiry", "2160h")
|
||||||
|
|
||||||
|
w := submitCreateForm(env, form)
|
||||||
|
require.Equal(t, http.StatusBadRequest, w.Code)
|
||||||
|
|
||||||
|
page := w.Body.String()
|
||||||
|
assert.Contains(t, page, tc.reason)
|
||||||
|
assert.Contains(t, page, `value="kept name"`)
|
||||||
|
assert.Contains(t, page, `>kept description</textarea>`)
|
||||||
|
assert.Contains(t, page, `value="`+tc.retention+`"`)
|
||||||
|
assert.Contains(t, page,
|
||||||
|
`value="`+html.EscapeString(tc.httpURL)+`"`)
|
||||||
|
assert.Contains(t, page, `name="archive" value="on" checked`)
|
||||||
|
assert.Contains(t, page, `x-data="collapsible" data-open`)
|
||||||
|
assert.Contains(t, page, `<option value="2160h" selected>`)
|
||||||
|
|
||||||
|
assertNothingCreated(t, env.db)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// errInjectedTargetCreate is the failure a test makes the insert of a
|
||||||
|
// target report.
|
||||||
|
var errInjectedTargetCreate = errors.New("injected target create failure")
|
||||||
|
|
||||||
|
// TestHandleSourceCreateSubmit_FailedTargetInsertCreatesNothing makes
|
||||||
|
// inserting a target fail after the webhook and its entrypoint were
|
||||||
|
// inserted, and checks that neither is left behind.
|
||||||
|
func TestHandleSourceCreateSubmit_FailedTargetInsertCreatesNothing(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
env := setupSourceTest(t)
|
||||||
|
|
||||||
|
require.NoError(t, env.db.DB().Callback().Create().
|
||||||
|
Before("gorm:create").
|
||||||
|
Register("test:fail_target_create", func(tx *gorm.DB) {
|
||||||
|
if tx.Statement.Table == "targets" {
|
||||||
|
_ = tx.AddError(errInjectedTargetCreate)
|
||||||
|
}
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
|
||||||
|
form := url.Values{}
|
||||||
|
form.Set("name", "rolled back")
|
||||||
|
form.Set("archive", "on")
|
||||||
|
form.Set("archive_expiry", "never")
|
||||||
|
|
||||||
|
w := submitCreateForm(env, form)
|
||||||
|
require.Equal(t, http.StatusInternalServerError, w.Code)
|
||||||
|
|
||||||
|
assertNothingCreated(t, env.db)
|
||||||
|
}
|
||||||
@@ -276,27 +276,41 @@ func (h *Handlers) HandleSourceCreate() http.HandlerFunc {
|
|||||||
return func(w http.ResponseWriter, r *http.Request) {
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
h.renderTemplate(
|
h.renderTemplate(
|
||||||
w, r, "sources_new.html",
|
w, r, "sources_new.html",
|
||||||
newSourceFormData("", "", ""),
|
newSourceFormData("", sourceFormInput{
|
||||||
|
RetentionDays: strconv.Itoa(
|
||||||
|
database.DefaultRetentionDays,
|
||||||
|
),
|
||||||
|
}),
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// sourceFormInput carries the raw values of the new webhook form. A
|
||||||
|
// refused submission is shown again from it, so every value entered
|
||||||
|
// comes back, retention included.
|
||||||
|
type sourceFormInput struct {
|
||||||
|
Name string
|
||||||
|
Description string
|
||||||
|
RetentionDays string
|
||||||
|
// HTTPURL, when not empty, asks for an HTTP target with this
|
||||||
|
// destination.
|
||||||
|
HTTPURL string
|
||||||
|
// Archive asks for a database (archive) target, whose rows expire
|
||||||
|
// after ArchiveExpiry.
|
||||||
|
Archive bool
|
||||||
|
ArchiveExpiry string
|
||||||
|
}
|
||||||
|
|
||||||
// newSourceFormData builds the template data for the webhook creation
|
// newSourceFormData builds the template data for the webhook creation
|
||||||
// form.
|
// form. It carries the retention default, which the form's help text
|
||||||
//
|
// names, from database.DefaultRetentionDays rather than a hardcoded
|
||||||
// It carries the retention default so the pre-filled value comes from
|
// copy of the same policy.
|
||||||
// database.DefaultRetentionDays rather than being a third hardcoded
|
|
||||||
// copy of the same policy, and it carries the submitted name and
|
|
||||||
// description so that re-rendering the form after a validation failure
|
|
||||||
// gives the user their input back instead of a blank form. The edit
|
|
||||||
// form already behaves that way; create now matches it.
|
|
||||||
func newSourceFormData(
|
func newSourceFormData(
|
||||||
errMsg, name, description string,
|
errMsg string, in sourceFormInput,
|
||||||
) map[string]any {
|
) map[string]any {
|
||||||
return map[string]any{
|
return map[string]any{
|
||||||
tmplKeyError: errMsg,
|
tmplKeyError: errMsg,
|
||||||
"Name": name,
|
"Form": in,
|
||||||
"Description": description,
|
|
||||||
"DefaultRetentionDays": database.DefaultRetentionDays,
|
"DefaultRetentionDays": database.DefaultRetentionDays,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -323,57 +337,112 @@ func (h *Handlers) HandleSourceCreateSubmit() http.HandlerFunc {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
name := r.PostFormValue("name")
|
in := sourceFormInput{
|
||||||
description := r.PostFormValue("description")
|
Name: r.PostFormValue("name"),
|
||||||
retentionStr := r.PostFormValue("retention_days")
|
Description: r.PostFormValue("description"),
|
||||||
|
RetentionDays: r.PostFormValue("retention_days"),
|
||||||
|
HTTPURL: r.PostFormValue("http_url"),
|
||||||
|
Archive: r.PostFormValue("archive") != "",
|
||||||
|
ArchiveExpiry: r.PostFormValue("archive_expiry"),
|
||||||
|
}
|
||||||
|
|
||||||
if name == "" {
|
refuse := func(errMsg string) {
|
||||||
h.renderTemplateStatus(
|
h.renderTemplateStatus(
|
||||||
w, r, "sources_new.html",
|
w, r, "sources_new.html",
|
||||||
newSourceFormData(
|
newSourceFormData(errMsg, in),
|
||||||
"Name is required", name, description,
|
|
||||||
),
|
|
||||||
http.StatusBadRequest,
|
http.StatusBadRequest,
|
||||||
)
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if in.Name == "" {
|
||||||
|
refuse("Name is required")
|
||||||
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
retentionDays, errMsg := parseRetentionDays(
|
retentionDays, errMsg := parseRetentionDays(
|
||||||
retentionStr, database.DefaultRetentionDays,
|
in.RetentionDays, database.DefaultRetentionDays,
|
||||||
)
|
)
|
||||||
if errMsg != "" {
|
if errMsg != "" {
|
||||||
h.renderTemplateStatus(
|
refuse(errMsg)
|
||||||
w, r, "sources_new.html",
|
|
||||||
newSourceFormData(errMsg, name, description),
|
|
||||||
http.StatusBadRequest,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
h.createWebhookWithEntrypoint(
|
targets, errMsg, err := h.newWebhookTargets(r.Context(), in)
|
||||||
w, r, userID, name, description, retentionDays,
|
if err != nil {
|
||||||
)
|
h.serverError(w, r, "failed to encode target config", err)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if errMsg != "" {
|
||||||
|
refuse(errMsg)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
h.createWebhookWithEntrypoint(w, r, &database.Webhook{
|
||||||
|
UserID: userID,
|
||||||
|
Name: in.Name,
|
||||||
|
Description: in.Description,
|
||||||
|
RetentionDays: retentionDays,
|
||||||
|
}, targets)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// createWebhookWithEntrypoint creates a webhook and its default
|
// newWebhookTargets validates the targets the new webhook form asks
|
||||||
// entrypoint in a transaction.
|
// for and returns the rows to create with the webhook, or the message
|
||||||
|
// the form shows for the first one it refuses. A filled-in HTTP URL
|
||||||
|
// asks for an HTTP target named "HTTP", and the archive checkbox for a
|
||||||
|
// database target named "Archive". Each goes through newTarget, as on
|
||||||
|
// the webhook page's add target form. The rows have no WebhookID yet:
|
||||||
|
// the webhook has no ID until it is created.
|
||||||
|
func (h *Handlers) newWebhookTargets(
|
||||||
|
ctx context.Context,
|
||||||
|
in sourceFormInput,
|
||||||
|
) ([]*database.Target, string, error) {
|
||||||
|
var requested []targetFormInput
|
||||||
|
|
||||||
|
if in.HTTPURL != "" {
|
||||||
|
requested = append(requested, targetFormInput{
|
||||||
|
Name: "HTTP",
|
||||||
|
Type: database.TargetTypeHTTP,
|
||||||
|
URL: in.HTTPURL,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
if in.Archive {
|
||||||
|
requested = append(requested, targetFormInput{
|
||||||
|
Name: "Archive",
|
||||||
|
Type: database.TargetTypeDatabase,
|
||||||
|
Expiry: in.ArchiveExpiry,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
targets := make([]*database.Target, 0, len(requested))
|
||||||
|
|
||||||
|
for _, form := range requested {
|
||||||
|
target, errMsg, err := h.newTarget(ctx, "", form)
|
||||||
|
if err != nil || errMsg != "" {
|
||||||
|
return nil, errMsg, err
|
||||||
|
}
|
||||||
|
|
||||||
|
targets = append(targets, target)
|
||||||
|
}
|
||||||
|
|
||||||
|
return targets, "", nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// createWebhookWithEntrypoint creates a webhook, its default
|
||||||
|
// entrypoint and the given targets in a transaction.
|
||||||
func (h *Handlers) createWebhookWithEntrypoint(
|
func (h *Handlers) createWebhookWithEntrypoint(
|
||||||
w http.ResponseWriter,
|
w http.ResponseWriter,
|
||||||
r *http.Request,
|
r *http.Request,
|
||||||
userID, name, description string,
|
webhook *database.Webhook,
|
||||||
retentionDays int,
|
targets []*database.Target,
|
||||||
) {
|
) {
|
||||||
webhook := &database.Webhook{
|
err := h.commitWebhook(webhook, targets)
|
||||||
UserID: userID,
|
|
||||||
Name: name,
|
|
||||||
Description: description,
|
|
||||||
RetentionDays: retentionDays,
|
|
||||||
}
|
|
||||||
|
|
||||||
err := h.commitWebhook(webhook)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
h.serverError(w, r, "failed to create webhook", err)
|
h.serverError(w, r, "failed to create webhook", err)
|
||||||
|
|
||||||
@@ -390,7 +459,7 @@ func (h *Handlers) createWebhookWithEntrypoint(
|
|||||||
|
|
||||||
h.log.Info("webhook created",
|
h.log.Info("webhook created",
|
||||||
"webhook_id", webhook.ID,
|
"webhook_id", webhook.ID,
|
||||||
"name", name, "user_id", userID,
|
"name", webhook.Name, "user_id", webhook.UserID,
|
||||||
)
|
)
|
||||||
|
|
||||||
http.Redirect(
|
http.Redirect(
|
||||||
@@ -399,10 +468,12 @@ func (h *Handlers) createWebhookWithEntrypoint(
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// commitWebhook creates a webhook and default entrypoint in
|
// commitWebhook creates a webhook, its default entrypoint and the
|
||||||
// a transaction. Returns an error on failure (rolls back).
|
// given targets in a transaction. Returns an error on failure (rolls
|
||||||
|
// back).
|
||||||
func (h *Handlers) commitWebhook(
|
func (h *Handlers) commitWebhook(
|
||||||
webhook *database.Webhook,
|
webhook *database.Webhook,
|
||||||
|
targets []*database.Target,
|
||||||
) error {
|
) error {
|
||||||
tx := h.db.DB().Begin()
|
tx := h.db.DB().Begin()
|
||||||
if tx.Error != nil {
|
if tx.Error != nil {
|
||||||
@@ -430,6 +501,17 @@ func (h *Handlers) commitWebhook(
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
for _, target := range targets {
|
||||||
|
target.WebhookID = webhook.ID
|
||||||
|
|
||||||
|
err = tx.Create(target).Error
|
||||||
|
if err != nil {
|
||||||
|
tx.Rollback()
|
||||||
|
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
return tx.Commit().Error
|
return tx.Commit().Error
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -485,6 +567,8 @@ func (h *Handlers) renderSourceDetail(
|
|||||||
"webhook_id = ?", webhook.ID,
|
"webhook_id = ?", webhook.ID,
|
||||||
).Find(&targets)
|
).Find(&targets)
|
||||||
|
|
||||||
|
entrypointViews := NewEntrypointViews(entrypoints)
|
||||||
|
|
||||||
var events []RecentEventView
|
var events []RecentEventView
|
||||||
|
|
||||||
if h.dbMgr.DBExists(webhook.ID) {
|
if h.dbMgr.DBExists(webhook.ID) {
|
||||||
@@ -503,6 +587,15 @@ func (h *Handlers) renderSourceDetail(
|
|||||||
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
|
err = addEntrypointEvents(
|
||||||
|
webhookDB, &webhook, entrypointViews, time.Now(),
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
h.serverError(w, r, "failed to count entrypoint events", err)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
scheme := "http"
|
scheme := "http"
|
||||||
@@ -523,7 +616,7 @@ func (h *Handlers) renderSourceDetail(
|
|||||||
// Targets are projected to a display-safe view: a
|
// Targets are projected to a display-safe view: a
|
||||||
// target's stored config blob holds a credential, and it
|
// target's stored config blob holds a credential, and it
|
||||||
// must never reach a template.
|
// must never reach a template.
|
||||||
"Entrypoints": NewEntrypointViews(entrypoints),
|
"Entrypoints": entrypointViews,
|
||||||
"Targets": h.targetRows(&webhook, targets),
|
"Targets": h.targetRows(&webhook, targets),
|
||||||
"Events": events,
|
"Events": events,
|
||||||
"BaseURL": baseURL,
|
"BaseURL": baseURL,
|
||||||
|
|||||||
@@ -196,8 +196,6 @@ func TestCreateFormRetentionCopyMatchesBehaviour(t *testing.T) {
|
|||||||
t.Cleanup(app.RequireStop)
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
body := renderPage(t, h, sess, "sources_new.html", map[string]any{
|
body := renderPage(t, h, sess, "sources_new.html", map[string]any{
|
||||||
"Name": "",
|
|
||||||
"Description": "",
|
|
||||||
"DefaultRetentionDays": database.DefaultRetentionDays,
|
"DefaultRetentionDays": database.DefaultRetentionDays,
|
||||||
dataKeyError: "",
|
dataKeyError: "",
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -326,6 +326,19 @@ func (h *Handlers) createAndFanOut(
|
|||||||
return nil, nil, err
|
return nil, nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A resubmitted copy did not arrive on its entrypoint's URL, so it
|
||||||
|
// leaves the entrypoint's last event as it is.
|
||||||
|
if src.ResubmittedFromID == nil {
|
||||||
|
err = database.AddEntrypointTotals(tx, database.EntrypointTotals{
|
||||||
|
EntrypointID: event.EntrypointID, LastEventAt: event.CreatedAt,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
tx.Rollback()
|
||||||
|
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
err = tx.Commit().Error
|
err = tx.Commit().Error
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, nil, fmt.Errorf(
|
return nil, nil, fmt.Errorf(
|
||||||
|
|||||||
@@ -122,6 +122,9 @@ func TestAlpineRunsUnderTheSecurityPolicy(t *testing.T) {
|
|||||||
checkEntrypointEdit(ctx, t, page, page+"/events")
|
checkEntrypointEdit(ctx, t, page, page+"/events")
|
||||||
checkRecentEvents(ctx, t, page)
|
checkRecentEvents(ctx, t, page)
|
||||||
checkEventLog(ctx, t, page+"/events", event.ID, target.Name)
|
checkEventLog(ctx, t, page+"/events", event.ID, target.Name)
|
||||||
|
checkArchiveChoice(ctx, t, srv.URL+"/hooks/new", page)
|
||||||
|
checkNewWebhookTargets(ctx, t, env, srv.URL+"/hooks/new")
|
||||||
|
checkRefusedNewWebhook(ctx, t, srv.URL+"/hooks/new")
|
||||||
checkMobileMenu(ctx, t, page)
|
checkMobileMenu(ctx, t, page)
|
||||||
|
|
||||||
assert.Empty(t, problems(), "the browser reported problems")
|
assert.Empty(t, problems(), "the browser reported problems")
|
||||||
@@ -642,6 +645,199 @@ func checkEventLog(
|
|||||||
"clicking the event again does not collapse it")
|
"clicking the event again does not collapse it")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The parts of the new webhook page the checks below find and click.
|
||||||
|
const (
|
||||||
|
archiveBox = `//input[@name="archive"]`
|
||||||
|
archiveIsOn = `document.querySelector('input[name="archive"]').checked`
|
||||||
|
pruningChoice = `//select[@name="archive_expiry"]`
|
||||||
|
createButton = `//button[text()="Create Webhook"]`
|
||||||
|
)
|
||||||
|
|
||||||
|
// checkArchiveChoice loads the new webhook page and checks that the
|
||||||
|
// archive pruning choice stays hidden until the archive box is checked
|
||||||
|
// and hides again when it is unchecked; and that after checking it,
|
||||||
|
// opening the page at elsewhere and going back, the page again shows
|
||||||
|
// the box unchecked and the choice hidden.
|
||||||
|
func checkArchiveChoice(
|
||||||
|
ctx context.Context, t *testing.T, url, elsewhere string,
|
||||||
|
) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
require.NoError(t, chromedp.Run(ctx, loadPage(url)))
|
||||||
|
|
||||||
|
assert.True(t, hidden(ctx, pruningChoice),
|
||||||
|
"the pruning choice shows before archive is checked")
|
||||||
|
|
||||||
|
click(ctx, t, archiveBox)
|
||||||
|
assert.True(t, shown(ctx, pruningChoice),
|
||||||
|
"checking archive does not show the pruning choice")
|
||||||
|
|
||||||
|
click(ctx, t, archiveBox)
|
||||||
|
assert.True(t, hidden(ctx, pruningChoice),
|
||||||
|
"unchecking archive does not hide the pruning choice")
|
||||||
|
|
||||||
|
var (
|
||||||
|
loaded string
|
||||||
|
checked bool
|
||||||
|
)
|
||||||
|
|
||||||
|
click(ctx, t, archiveBox)
|
||||||
|
require.NoError(t, chromedp.Run(
|
||||||
|
ctx,
|
||||||
|
loadPage(elsewhere),
|
||||||
|
chromedp.NavigateBack(),
|
||||||
|
chromedp.WaitNotPresent("[x-cloak]", chromedp.ByQuery),
|
||||||
|
chromedp.Evaluate(
|
||||||
|
`performance.getEntriesByType("navigation")[0].type`, &loaded,
|
||||||
|
),
|
||||||
|
chromedp.Evaluate(archiveIsOn, &checked),
|
||||||
|
))
|
||||||
|
require.Equal(
|
||||||
|
t, "back_forward", loaded,
|
||||||
|
"going back, the browser did not load the page again",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.False(t, checked, "going back leaves archive checked")
|
||||||
|
assert.True(t, hidden(ctx, pruningChoice),
|
||||||
|
"going back shows the pruning choice")
|
||||||
|
}
|
||||||
|
|
||||||
|
// checkNewWebhookTargets submits the new webhook page with the HTTP
|
||||||
|
// target URL filled in or empty, and with archive left off or checked
|
||||||
|
// with each pruning choice, and checks that each webhook is created
|
||||||
|
// with exactly the targets asked for.
|
||||||
|
func checkNewWebhookTargets(
|
||||||
|
ctx context.Context, t *testing.T, env *testEnv, url string,
|
||||||
|
) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
// Each value the pruning choice submits, after an empty one that
|
||||||
|
// stands for archive left off.
|
||||||
|
expiries := []string{
|
||||||
|
"", "never", "1h", "12h", "24h", "720h", "2160h", "8760h",
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, httpURL := range []string{"", publicTargetURL} {
|
||||||
|
for _, expiry := range expiries {
|
||||||
|
name := "url=" + httpURL + " archive=" + expiry
|
||||||
|
want := map[database.TargetType]string{}
|
||||||
|
|
||||||
|
require.NoError(t, chromedp.Run(
|
||||||
|
ctx,
|
||||||
|
loadPage(url),
|
||||||
|
chromedp.SetValue("#name", name, chromedp.ByQuery),
|
||||||
|
))
|
||||||
|
|
||||||
|
if httpURL != "" {
|
||||||
|
require.NoError(t, chromedp.Run(ctx, chromedp.SetValue(
|
||||||
|
"#http_url", httpURL, chromedp.ByQuery,
|
||||||
|
)))
|
||||||
|
|
||||||
|
want[database.TargetTypeHTTP] = `{"url":"` + httpURL + `"}`
|
||||||
|
}
|
||||||
|
|
||||||
|
if expiry != "" {
|
||||||
|
// The choice showing moves Create down, so it is
|
||||||
|
// waited for before Create is clicked.
|
||||||
|
click(ctx, t, archiveBox)
|
||||||
|
require.Truef(t, shown(ctx, pruningChoice),
|
||||||
|
"%s: checking archive does not show the pruning choice",
|
||||||
|
name)
|
||||||
|
require.NoError(t, chromedp.Run(ctx, chromedp.SetValue(
|
||||||
|
pruningChoice, expiry, chromedp.BySearch,
|
||||||
|
)))
|
||||||
|
|
||||||
|
want[database.TargetTypeDatabase] = `{"expiry":"` + expiry + `"}`
|
||||||
|
}
|
||||||
|
|
||||||
|
click(ctx, t, createButton)
|
||||||
|
require.Truef(t, shown(ctx, `//h1[text()="`+name+`"]`),
|
||||||
|
"%s: the new webhook's page does not open", name)
|
||||||
|
|
||||||
|
assert.Equalf(t, want, targetConfigs(t, env, name),
|
||||||
|
"%s: the webhook does not have the targets asked for", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// targetConfigs reads the targets of the webhook named name, and
|
||||||
|
// returns each one's stored configuration by its type.
|
||||||
|
func targetConfigs(
|
||||||
|
t *testing.T, env *testEnv, name string,
|
||||||
|
) map[database.TargetType]string {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var (
|
||||||
|
webhook database.Webhook
|
||||||
|
targets []database.Target
|
||||||
|
)
|
||||||
|
|
||||||
|
require.NoError(t, env.db.DB().
|
||||||
|
Where("name = ?", name).First(&webhook).Error)
|
||||||
|
require.NoError(t, env.db.DB().
|
||||||
|
Where("webhook_id = ?", webhook.ID).Find(&targets).Error)
|
||||||
|
|
||||||
|
configs := map[database.TargetType]string{}
|
||||||
|
for _, target := range targets {
|
||||||
|
configs[target.Type] = target.Config
|
||||||
|
}
|
||||||
|
|
||||||
|
return configs
|
||||||
|
}
|
||||||
|
|
||||||
|
// checkRefusedNewWebhook submits the new webhook page with archive
|
||||||
|
// checked and an HTTP target URL the server refuses, a loopback
|
||||||
|
// destination, and checks that the page comes back with the reason and
|
||||||
|
// every value entered, archive still checked and its pruning choice
|
||||||
|
// showing.
|
||||||
|
func checkRefusedNewWebhook(ctx context.Context, t *testing.T, url string) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
const refusedURL = "http://127.0.0.1/hook"
|
||||||
|
|
||||||
|
require.NoError(t, chromedp.Run(
|
||||||
|
ctx,
|
||||||
|
loadPage(url),
|
||||||
|
chromedp.SetValue("#name", "refused", chromedp.ByQuery),
|
||||||
|
chromedp.SetValue("#description", "kept", chromedp.ByQuery),
|
||||||
|
chromedp.SetValue("#retention_days", "7", chromedp.ByQuery),
|
||||||
|
chromedp.SetValue("#http_url", refusedURL, chromedp.ByQuery),
|
||||||
|
))
|
||||||
|
click(ctx, t, archiveBox)
|
||||||
|
require.True(t, shown(ctx, pruningChoice),
|
||||||
|
"checking archive does not show the pruning choice")
|
||||||
|
require.NoError(t, chromedp.Run(ctx, chromedp.SetValue(
|
||||||
|
pruningChoice, "2160h", chromedp.BySearch,
|
||||||
|
)))
|
||||||
|
click(ctx, t, createButton)
|
||||||
|
|
||||||
|
assert.True(t, shown(ctx, `//div[@class="alert-error"]`),
|
||||||
|
"a refused webhook does not show the reason")
|
||||||
|
|
||||||
|
var (
|
||||||
|
name, description, retention, typed, expiry string
|
||||||
|
checked bool
|
||||||
|
)
|
||||||
|
|
||||||
|
require.NoError(t, chromedp.Run(
|
||||||
|
ctx,
|
||||||
|
chromedp.Value("#name", &name, chromedp.ByQuery),
|
||||||
|
chromedp.Value("#description", &description, chromedp.ByQuery),
|
||||||
|
chromedp.Value("#retention_days", &retention, chromedp.ByQuery),
|
||||||
|
chromedp.Value("#http_url", &typed, chromedp.ByQuery),
|
||||||
|
chromedp.Value("#archive_expiry", &expiry, chromedp.ByQuery),
|
||||||
|
chromedp.Evaluate(archiveIsOn, &checked),
|
||||||
|
))
|
||||||
|
|
||||||
|
assert.Equal(t, "refused", name, "the name entered is lost")
|
||||||
|
assert.Equal(t, "kept", description, "the description entered is lost")
|
||||||
|
assert.Equal(t, "7", retention, "the retention entered is lost")
|
||||||
|
assert.Equal(t, refusedURL, typed, "the url entered is lost")
|
||||||
|
assert.True(t, checked, "archive is no longer checked")
|
||||||
|
assert.True(t, shown(ctx, pruningChoice), "the pruning choice is hidden")
|
||||||
|
assert.Equal(t, "2160h", expiry, "the pruning chosen is lost")
|
||||||
|
}
|
||||||
|
|
||||||
// checkMobileMenu loads a page in a phone-sized window and checks that
|
// checkMobileMenu loads a page in a phone-sized window and checks that
|
||||||
// the menu button opens and closes the mobile menu.
|
// the menu button opens and closes the mobile menu.
|
||||||
func checkMobileMenu(ctx context.Context, t *testing.T, url string) {
|
func checkMobileMenu(ctx context.Context, t *testing.T, url string) {
|
||||||
|
|||||||
+3
-2
@@ -71,8 +71,9 @@ document.addEventListener("alpine:init", function () {
|
|||||||
|
|
||||||
// Something a click shows and hides: the mobile menu, an add form,
|
// Something a click shows and hides: the mobile menu, an add form,
|
||||||
// an entrypoint's edit form, an event in the event log or in the
|
// an entrypoint's edit form, an event in the event log or in the
|
||||||
// recent events, a delivery's attempts. It starts hidden, or shown
|
// recent events, a delivery's attempts, the new webhook page's
|
||||||
// when its element has the data-open attribute.
|
// archive pruning choice. It starts hidden, or shown when its
|
||||||
|
// element has the data-open attribute.
|
||||||
window.Alpine.data("collapsible", function () {
|
window.Alpine.data("collapsible", function () {
|
||||||
return {
|
return {
|
||||||
open: false,
|
open: false,
|
||||||
|
|||||||
@@ -97,6 +97,18 @@
|
|||||||
</div>
|
</div>
|
||||||
<!-- The URL above is the entrypoint's credential:
|
<!-- The URL above is the entrypoint's credential:
|
||||||
anyone holding it can submit events. -->
|
anyone holding it can submit events. -->
|
||||||
|
<div class="text-xs text-gray-500 mt-1">
|
||||||
|
<span class="font-medium text-gray-700">Last Event:</span>
|
||||||
|
{{if .LastEvent}}
|
||||||
|
<span title="{{.LastEventUTC}}">{{.LastEvent}}</span>
|
||||||
|
{{else}}
|
||||||
|
<span>never</span>
|
||||||
|
{{end}}
|
||||||
|
</div>
|
||||||
|
<div class="text-xs text-gray-500 mt-1">
|
||||||
|
<span class="font-medium text-gray-700">Events Within Retention:</span>
|
||||||
|
<span>{{.Events}}</span>
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
{{else}}
|
{{else}}
|
||||||
<div class="p-4 text-sm text-gray-500">No entrypoints configured.</div>
|
<div class="p-4 text-sm text-gray-500">No entrypoints configured.</div>
|
||||||
|
|||||||
@@ -18,20 +18,50 @@
|
|||||||
<input type="hidden" name="csrf_token" value="{{.CSRFToken}}">
|
<input type="hidden" name="csrf_token" value="{{.CSRFToken}}">
|
||||||
<div class="form-group">
|
<div class="form-group">
|
||||||
<label for="name" class="label">Name</label>
|
<label for="name" class="label">Name</label>
|
||||||
<input type="text" id="name" name="name" value="{{.Name}}" required autofocus placeholder="My Webhook" class="input">
|
<input type="text" id="name" name="name" value="{{.Form.Name}}" required autofocus placeholder="My Webhook" class="input">
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div class="form-group">
|
<div class="form-group">
|
||||||
<label for="description" class="label">Description</label>
|
<label for="description" class="label">Description</label>
|
||||||
<textarea id="description" name="description" rows="3" placeholder="Optional description" class="input">{{.Description}}</textarea>
|
<textarea id="description" name="description" rows="3" placeholder="Optional description" class="input">{{.Form.Description}}</textarea>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div class="form-group">
|
<div class="form-group">
|
||||||
<label for="retention_days" class="label">Retention (days)</label>
|
<label for="retention_days" class="label">Retention (days)</label>
|
||||||
<input type="number" id="retention_days" name="retention_days" value="{{.DefaultRetentionDays}}" min="0" class="input">
|
<input type="number" id="retention_days" name="retention_days" value="{{.Form.RetentionDays}}" min="0" class="input">
|
||||||
<p class="text-xs text-gray-500 mt-1">A periodic cleanup permanently deletes events older than this, along with their delivery records. Enter 0 to retain events forever; leave blank to use the default of {{.DefaultRetentionDays}} days.</p>
|
<p class="text-xs text-gray-500 mt-1">A periodic cleanup permanently deletes events older than this, along with their delivery records. Enter 0 to retain events forever; leave blank to use the default of {{.DefaultRetentionDays}} days.</p>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<div class="form-group">
|
||||||
|
<label for="http_url" class="label">HTTP target URL</label>
|
||||||
|
<input type="url" id="http_url" name="http_url" value="{{.Form.HTTPURL}}" placeholder="https://example.com/webhook" class="input">
|
||||||
|
<p class="text-xs text-gray-500 mt-1">Optional. When filled in, the webhook is created with an HTTP target that delivers each event to this URL.</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- The checkbox shows the pruning choice while checked. With
|
||||||
|
autocomplete="off", going back to the page does not
|
||||||
|
check the box again with the choice hidden. -->
|
||||||
|
<div class="form-group" x-data="collapsible"{{if .Form.Archive}} data-open{{end}}>
|
||||||
|
<label class="flex items-center gap-2 text-sm font-medium text-gray-700">
|
||||||
|
<input type="checkbox" name="archive" value="on"{{if .Form.Archive}} checked{{end}} autocomplete="off" @change="toggle" class="h-4 w-4">
|
||||||
|
Archive
|
||||||
|
</label>
|
||||||
|
<p class="text-xs text-gray-500 mt-1">When checked, the webhook is created with a database target that keeps a copy of every event.</p>
|
||||||
|
<div x-show="open" x-cloak class="mt-3">
|
||||||
|
<label for="archive_expiry" class="label">Archive pruning</label>
|
||||||
|
<select id="archive_expiry" name="archive_expiry" class="input">
|
||||||
|
<option value="never"{{if eq .Form.ArchiveExpiry "never"}} selected{{end}}>never</option>
|
||||||
|
<option value="1h"{{if eq .Form.ArchiveExpiry "1h"}} selected{{end}}>1h</option>
|
||||||
|
<option value="12h"{{if eq .Form.ArchiveExpiry "12h"}} selected{{end}}>12h</option>
|
||||||
|
<option value="24h"{{if eq .Form.ArchiveExpiry "24h"}} selected{{end}}>24h</option>
|
||||||
|
<option value="720h"{{if eq .Form.ArchiveExpiry "720h"}} selected{{end}}>30d</option>
|
||||||
|
<option value="2160h"{{if eq .Form.ArchiveExpiry "2160h"}} selected{{end}}>90d</option>
|
||||||
|
<option value="8760h"{{if eq .Form.ArchiveExpiry "8760h"}} selected{{end}}>365d</option>
|
||||||
|
</select>
|
||||||
|
<p class="text-xs text-gray-500 mt-1">Archived events older than this are deleted from the archive; never keeps them all.</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
<div class="flex gap-3">
|
<div class="flex gap-3">
|
||||||
<button type="submit" class="btn-primary">Create Webhook</button>
|
<button type="submit" class="btn-primary">Create Webhook</button>
|
||||||
<a href="/hooks" class="btn-secondary">Cancel</a>
|
<a href="/hooks" class="btn-secondary">Cancel</a>
|
||||||
|
|||||||
Reference in new issue
Block a user