Store and show an event's query string, and pass it on to an HTTP target when set (closes #312)
check / check (push) Successful in 6m32s

The receiver keeps the query string of the request it received on the event, in a new `raw_query` column of the per-webhook `events` table; a resubmitted copy carries its original's. The event log and the event's page show it with the request, the event log leaving out one over 32 KiB as it does request headers. The archive and log targets carry it. An HTTP target gets a `forwardQuery` setting, off by default, on both target forms and in the target list: on, each delivery, replays and resubmits included, appends the query string to the target URL, joined with `&` to one it already has.

Model: opus-5-5
This commit is contained in:
2026-10-04 00:18:34 +00:00
parent a8fc0c5d32
commit ba19430b87
32 changed files with 537 additions and 69 deletions
+34 -18
View File
@@ -1638,11 +1638,19 @@ URL, custom headers, timeout settings).
**`http` target configuration:** **`http` target configuration:**
| Key | Type | Description | | Key | Type | Description |
| --------- | ------------- | -------------------------------------------------------------------------------------- | | -------------- | ------------- | ----------------------------------------------------------------------------------------- |
| `url` | string | Destination the event is POSTed to | | `url` | string | Destination the event is POSTed to |
| `headers` | object | Extra request headers, applied last so they win over the event's own forwarded headers | | `headers` | object | Extra request headers, applied last so they win over the event's own forwarded headers |
| `timeout` | integer (sec) | Per-target request timeout; unset (or 0) uses the shared 30-second client timeout | | `timeout` | integer (sec) | Per-target request timeout; unset (or 0) uses the shared 30-second client timeout |
| `forwardQuery` | boolean | Pass the query string each event arrived with on to the target; unset (or false) does not |
`forwardQuery` is off by default, and the target URL is then sent exactly as
configured. On, each delivery appends the event's query string to the target
URL, joined with `&` when the URL already has a query string of its own; a
replayed delivery and a resubmitted event's deliveries do the same. Both target
forms offer it as "Pass the query string on to this target", and the target list
shows it when it is on.
`timeout` is capped at **300 seconds**, and the form rejects anything above it `timeout` is capped at **300 seconds**, and the form rejects anything above it
rather than substituting the cap. A delivery attempt holds one of the bounded rather than substituting the cap. A delivery attempt holds one of the bounded
@@ -1704,6 +1712,7 @@ auditing, for replay, and for resubmission.
| `webhook_id` | UUID | Foreign key → Webhook | | `webhook_id` | UUID | Foreign key → Webhook |
| `entrypoint_id` | UUID | Foreign key → Entrypoint | | `entrypoint_id` | UUID | Foreign key → Entrypoint |
| `method` | string | HTTP method of the captured request. Always `POST`: the receiver answers every other method with 405 before an Event is created | | `method` | string | HTTP method of the captured request. Always `POST`: the receiver answers every other method with 405 before an Event is created |
| `raw_query` | text | The query string of the captured request, as sent, without the leading `?`; empty when there was none. A resubmitted copy carries its original's |
| `headers` | JSON | Complete request headers | | `headers` | JSON | Complete request headers |
| `body` | text | Raw request body | | `body` | text | Raw request body |
| `content_type` | string | Content-Type header value | | `content_type` | string | Content-Type header value |
@@ -1712,9 +1721,15 @@ auditing, for replay, and for resubmission.
**Relations:** Belongs to Webhook. Belongs to Entrypoint. Has many Deliveries. **Relations:** Belongs to Webhook. Belongs to Entrypoint. Has many Deliveries.
When a request arrives at an entrypoint, the full request (method, headers, When a request arrives at an entrypoint, the full request (method, query string,
body) is captured as an Event. The event is then queued for delivery to every headers, body) is captured as an Event. The event is then queued for delivery to
active target configured on the parent webhook. every active target configured on the parent webhook.
The event log and the event's own page show the query string with the rest of
the request. The event log leaves out one larger than 32 KiB, as it does request
headers, and links to the event's page, which shows it whole. The `database` and
`log` targets carry it with the rest of the event. An `http` target receives it
only when its `forwardQuery` setting is on.
#### Delivery #### Delivery
@@ -1760,14 +1775,14 @@ the webhook's currently active targets.
**Resubmit.** Replay recovers one delivery; **resubmit** re-injects one EVENT. **Resubmit.** Replay recovers one delivery; **resubmit** re-injects one EVENT.
The event log offers a per-event **Resubmit** action that stores a NEW event The event log offers a per-event **Resubmit** action that stores a NEW event
copying the stored one's `method`, `headers`, `body` and `content_type` copying the stored one's `method`, `raw_query`, `headers`, `body` and
verbatim, then fans it out to the webhook's currently **active** targets — `content_type` verbatim, then fans it out to the webhook's currently **active**
resolved fresh by the same query the receiver uses, so a target created long targets — resolved fresh by the same query the receiver uses, so a target
after the original event arrived receives it. That is the difference that created long after the original event arrived receives it. That is the
matters: a target added to test a backend under development has no prior difference that matters: a target added to test a backend under development has
delivery, so there is nothing to replay to it, while a resubmit reaches it like no prior delivery, so there is nothing to replay to it, while a resubmit reaches
any other active target. Inactive targets are skipped, exactly as the receiver it like any other active target. Inactive targets are skipped, exactly as the
skips them. receiver skips them.
The new event is a first-class event in the log with its own deliveries, not a The new event is a first-class event in the log with its own deliveries, not a
marker on the one it came from, and the original's deliveries are left marker on the one it came from, and the original's deliveries are left
@@ -2438,7 +2453,8 @@ in front of them, so a query on a fixed 200 URL would otherwise buy the same
amplification as an invented path. Nothing debuggable is lost: the only query amplification as an invented path. Nothing debuggable is lost: the only query
parameters this service reads are the sign-in page's `next`, the page to return parameters this service reads are the sign-in page's `next`, the page to return
to, `notice`, which names the line a page shows after an action, and the event to, `notice`, which names the line a page shows after an action, and the event
log's `show`, which picks the events it lists. log's `show`, which picks the events it lists. A query string sent to an
entrypoint is not lost either: the event stores it, and the event log shows it.
Client-supplied request content does not leave the host by the other route Client-supplied request content does not leave the host by the other route
either. The Sentry SDK attaches the request to every event it captures, either. The Sentry SDK attaches the request to every event it captures,
@@ -2901,7 +2917,7 @@ page that was asked for.
| `POST` | `/hook/{id}/edit` | Edit webhook submission | | `POST` | `/hook/{id}/edit` | Edit webhook submission |
| `POST` | `/hook/{id}/delete` | Delete webhook | | `POST` | `/hook/{id}/delete` | Delete webhook |
| `GET` | `/hook/{id}/events` | Full Event Log. `?show=failed` lists only the events with a failed delivery, and `?show=pending` only those with a delivery pending or retrying | | `GET` | `/hook/{id}/events` | Full Event Log. `?show=failed` lists only the events with a failed delivery, and `?show=pending` only those with a delivery pending or retrying |
| `GET` | `/hook/{id}/events/{eventID}` | One event's own page: its details, the entrypoint it arrived at (for a resubmitted copy, the one the request it copies arrived at), its request headers, its whole body and every delivery of it | | `GET` | `/hook/{id}/events/{eventID}` | One event's own page: its details, the entrypoint it arrived at (for a resubmitted copy, the one the request it copies arrived at), its query string, its request headers, its whole body and every delivery of it |
| `GET` | `/hook/{id}/events/{eventID}/body` | Download an event's stored body. The pages show a body as text, cut at 32 KiB in the recent events and the event log, and leave a binary one out, so this is the only route that serves the stored bytes; it is offered wherever a body is cut or binary | | `GET` | `/hook/{id}/events/{eventID}/body` | Download an event's stored body. The pages show a body as text, cut at 32 KiB in the recent events and the event log, and leave a binary one out, so this is the only route that serves the stored bytes; it is offered wherever a body is cut or binary |
| `POST` | `/hook/{id}/deliveries/{deliveryID}/replay` | Replay a finished delivery: creates a new delivery for the same event against the target's current configuration (30 per minute per bucket, then `429`) | | `POST` | `/hook/{id}/deliveries/{deliveryID}/replay` | Replay a finished delivery: creates a new delivery for the same event against the target's current configuration (30 per minute per bucket, then `429`) |
| `POST` | `/hook/{id}/events/{eventID}/resubmit` | Resubmit a stored event: creates a new event copying it and fans that out to every currently active target (30 per minute per bucket, then `429`) | | `POST` | `/hook/{id}/events/{eventID}/resubmit` | Resubmit a stored event: creates a new event copying it and fans that out to every currently active target (30 per minute per bucket, then `429`) |
+3 -1
View File
@@ -30,8 +30,10 @@ type Event struct {
WebhookID string `gorm:"type:uuid;not null" json:"webhookId"` WebhookID string `gorm:"type:uuid;not null" json:"webhookId"`
EntrypointID string `gorm:"type:uuid;not null;index:idx_events_entrypoint_id,priority:1" json:"entrypointId"` EntrypointID string `gorm:"type:uuid;not null;index:idx_events_entrypoint_id,priority:1" json:"entrypointId"`
// Request data // Request data. RawQuery is the receiving request's query string
// as sent, without the leading "?".
Method string `gorm:"not null" json:"method"` Method string `gorm:"not null" json:"method"`
RawQuery string `gorm:"type:text" json:"rawQuery"`
Headers string `gorm:"type:text" json:"headers"` // JSON Headers string `gorm:"type:text" json:"headers"` // JSON
Body string `gorm:"type:text" json:"body"` Body string `gorm:"type:text" json:"body"`
ContentType string `json:"contentType"` ContentType string `json:"contentType"`
+3
View File
@@ -110,6 +110,7 @@ type Task struct {
MaxRetries int MaxRetries int
Method string Method string
RawQuery string
Headers string Headers string
ContentType string ContentType string
Body *string Body *string
@@ -1752,6 +1753,7 @@ func buildEventFromTask(task *Task) database.Event {
event := database.Event{ event := database.Event{
EntrypointID: task.EntrypointID, EntrypointID: task.EntrypointID,
Method: task.Method, Method: task.Method,
RawQuery: task.RawQuery,
Headers: task.Headers, Headers: task.Headers,
ContentType: task.ContentType, ContentType: task.ContentType,
} }
@@ -2102,6 +2104,7 @@ func buildRecoveryTask(
TargetConfig: target.Config, TargetConfig: target.Config,
MaxRetries: target.MaxRetries, MaxRetries: target.MaxRetries,
Method: event.Method, Method: event.Method,
RawQuery: event.RawQuery,
Headers: event.Headers, Headers: event.Headers,
ContentType: event.ContentType, ContentType: event.ContentType,
Body: bodyPtr, Body: bodyPtr,
@@ -673,6 +673,11 @@ func TestRecoverPendingDeliveries(t *testing.T) {
t, s.WebhookDB, s.WebhookID, targetID, 3, t, s.WebhookDB, s.WebhookID, targetID, 3,
) )
// A recovered delivery still carries its event's query string.
require.NoError(t, s.WebhookDB.Model(&database.Event{}).
Where("webhook_id = ?", s.WebhookID).
Update("raw_query", eventQuery).Error)
s.Engine.ExportRecoverPendingDeliveries( s.Engine.ExportRecoverPendingDeliveries(
context.Background(), s.WebhookDB, context.Background(), s.WebhookDB,
s.WebhookID, s.WebhookID,
@@ -687,6 +692,8 @@ func TestRecoverPendingDeliveries(t *testing.T) {
database.TargetTypeLog, database.TargetTypeLog,
task.TargetType, task.TargetType,
) )
assert.Equal(t, eventQuery, task.RawQuery)
case <-time.After(2 * time.Second): case <-time.After(2 * time.Second):
t.Fatalf("expected task %d", i) t.Fatalf("expected task %d", i)
} }
+6
View File
@@ -11,6 +11,7 @@ import (
"net/http/httptest" "net/http/httptest"
"os" "os"
"path/filepath" "path/filepath"
"strconv"
"strings" "strings"
"sync" "sync"
"sync/atomic" "sync/atomic"
@@ -1990,6 +1991,10 @@ func assertLogLineComplete(
"log line must contain the full request headers", "log line must contain the full request headers",
) )
assert.Contains(t, out, "raw_query="+strconv.Quote(event.RawQuery),
"log line must contain the query string",
)
assert.Contains(t, out, event.EntrypointID, assert.Contains(t, out, event.EntrypointID,
"log line must contain the entrypoint id", "log line must contain the entrypoint id",
) )
@@ -2012,6 +2017,7 @@ func TestDeliverLog_LogsFullContent(t *testing.T) {
event := seedEvent( event := seedEvent(
t, db, `{"log-body-marker":"abc123"}`, t, db, `{"log-body-marker":"abc123"}`,
) )
event.RawQuery = eventQuery
dlv := seedDelivery( dlv := seedDelivery(
t, db, event.ID, uuid.New().String(), t, db, event.ID, uuid.New().String(),
+7 -3
View File
@@ -39,6 +39,9 @@ type TargetConfigForm struct {
// Timeout is the HTTP target's per-request timeout in seconds, // Timeout is the HTTP target's per-request timeout in seconds,
// empty when unset. // empty when unset.
Timeout string Timeout string
// ForwardQuery is the HTTP target's setting that passes each
// event's query string on to it.
ForwardQuery bool
// Expiry is the database (archive) target's row expiry. // Expiry is the database (archive) target's row expiry.
Expiry string Expiry string
// Rotation is the database (archive) target's rotation. // Rotation is the database (archive) target's rotation.
@@ -64,9 +67,10 @@ func NewTargetConfigForm(
} }
return TargetConfigForm{ return TargetConfigForm{
URL: cfg.URL, URL: cfg.URL,
Headers: FormatTargetHeaders(cfg.Headers), Headers: FormatTargetHeaders(cfg.Headers),
Timeout: FormatTargetTimeout(cfg.Timeout), Timeout: FormatTargetTimeout(cfg.Timeout),
ForwardQuery: cfg.ForwardQuery,
}, nil }, nil
case database.TargetTypeSlack: case database.TargetTypeSlack:
cfg, err := parseSlackConfig(t.Config) cfg, err := parseSlackConfig(t.Config)
+7
View File
@@ -171,6 +171,13 @@ func httpConfigFields(t *database.Target) []ConfigField {
}) })
} }
if cfg.ForwardQuery {
fields = append(fields, ConfigField{
Label: "Query string",
Value: "passed on to this target",
})
}
fields = append(fields, maxRetriesField(t)) fields = append(fields, maxRetriesField(t))
return fields return fields
+3 -1
View File
@@ -223,7 +223,8 @@ func TestNewTargetViews_HTTP(t *testing.T) {
Type: database.TargetTypeHTTP, Type: database.TargetTypeHTTP,
Config: `{"url":"` + viewExampleHook + `",` + Config: `{"url":"` + viewExampleHook + `",` +
`"timeout":30,` + `"timeout":30,` +
`"headers":{"Authorization":"Bearer sekrit"}}`, `"headers":{"Authorization":"Bearer sekrit"},` +
`"forwardQuery":true}`,
MaxRetries: 5, MaxRetries: 5,
}) })
@@ -235,6 +236,7 @@ func TestNewTargetViews_HTTP(t *testing.T) {
"Destination URL": viewMaskedOrigin, "Destination URL": viewMaskedOrigin,
"Timeout": "30s", "Timeout": "30s",
"Headers": "1 configured", "Headers": "1 configured",
"Query string": "passed on to this target",
viewMaxRetries: "5", viewMaxRetries: "5",
}, },
fields, fields,
+1
View File
@@ -184,6 +184,7 @@ func (t *databaseTarget) archive(d *database.Delivery) error {
WebhookID: webhookID, WebhookID: webhookID,
EntrypointID: d.Event.EntrypointID, EntrypointID: d.Event.EntrypointID,
Method: d.Event.Method, Method: d.Event.Method,
RawQuery: d.Event.RawQuery,
Headers: d.Event.Headers, Headers: d.Event.Headers,
Body: d.Event.Body, Body: d.Event.Body,
ContentType: d.Event.ContentType, ContentType: d.Event.ContentType,
@@ -101,6 +101,7 @@ type archivedEvent struct {
WebhookID string WebhookID string
EntrypointID string EntrypointID string
Method string Method string
RawQuery string
Headers string Headers string
Body string Body string
ContentType string ContentType string
@@ -360,6 +360,7 @@ func writeRow(w io.Writer, ev *archivedEvent, period string) error {
"webhook_id": ev.WebhookID, "webhook_id": ev.WebhookID,
"entrypoint_id": ev.EntrypointID, "entrypoint_id": ev.EntrypointID,
"method": ev.Method, "method": ev.Method,
"raw_query": ev.RawQuery,
"headers": ev.Headers, "headers": ev.Headers,
"body": ev.Body, "body": ev.Body,
"content_type": ev.ContentType, "content_type": ev.ContentType,
@@ -166,6 +166,7 @@ func TestArchiveExport_MatchesStoredRows(t *testing.T) {
WebhookID: exportWebhookID, WebhookID: exportWebhookID,
EntrypointID: "ep-1", EntrypointID: "ep-1",
Method: "POST", Method: "POST",
RawQuery: eventQuery,
Headers: `{"X-Test":["yes"]}`, Headers: `{"X-Test":["yes"]}`,
Body: body, Body: body,
ContentType: testContentType, ContentType: testContentType,
@@ -215,12 +216,13 @@ func assertExportedRow(
assert.Equal(t, row.WebhookID, ev["webhook_id"]) assert.Equal(t, row.WebhookID, ev["webhook_id"])
assert.Equal(t, row.EntrypointID, ev["entrypoint_id"]) assert.Equal(t, row.EntrypointID, ev["entrypoint_id"])
assert.Equal(t, row.Method, ev["method"]) assert.Equal(t, row.Method, ev["method"])
assert.Equal(t, row.RawQuery, ev["raw_query"])
assert.Equal(t, row.Headers, ev["headers"]) assert.Equal(t, row.Headers, ev["headers"])
assert.Equal(t, row.ContentType, ev["content_type"]) assert.Equal(t, row.ContentType, ev["content_type"])
if row.Body != binaryBody { if row.Body != binaryBody {
assert.Equal(t, row.Body, ev["body"]) assert.Equal(t, row.Body, ev["body"])
assert.Len(t, ev, 9, "the nine columns and nothing else: %v", ev) assert.Len(t, ev, 10, "the ten columns and nothing else: %v", ev)
return return
} }
@@ -229,7 +231,7 @@ func assertExportedRow(
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, binaryBody, string(body)) assert.Equal(t, binaryBody, string(body))
assert.Equal(t, "base64", ev["body_encoding"]) assert.Equal(t, "base64", ev["body_encoding"])
assert.Len(t, ev, 10, "the nine columns and body_encoding: %v", ev) assert.Len(t, ev, 11, "the ten columns and body_encoding: %v", ev)
} }
// TestArchiveExport_Empty proves an archive with nothing in it exports // TestArchiveExport_Empty proves an archive with nothing in it exports
@@ -85,6 +85,7 @@ func TestDeliverDatabase_ArchivesEvent(t *testing.T) {
webhookDB := testWebhookDB(t) webhookDB := testWebhookDB(t)
event := seedEvent(t, webhookDB, `{"archived":true}`) event := seedEvent(t, webhookDB, `{"archived":true}`)
event.RawQuery = eventQuery
d := seedDatabaseTargetDelivery(t, webhookDB, event, tgt) d := seedDatabaseTargetDelivery(t, webhookDB, event, tgt)
env.eng.ExportDeliverDatabase(webhookDB, d) env.eng.ExportDeliverDatabase(webhookDB, d)
@@ -113,6 +114,7 @@ func TestDeliverDatabase_ArchivesEvent(t *testing.T) {
assert.Equal(t, event.ID, rows[0].EventID) assert.Equal(t, event.ID, rows[0].EventID)
assert.Equal(t, event.WebhookID, rows[0].WebhookID) assert.Equal(t, event.WebhookID, rows[0].WebhookID)
assert.Equal(t, event.Method, rows[0].Method) assert.Equal(t, event.Method, rows[0].Method)
assert.Equal(t, eventQuery, rows[0].RawQuery)
assert.JSONEq(t, `{"archived":true}`, rows[0].Body) assert.JSONEq(t, `{"archived":true}`, rows[0].Body)
} }
+23
View File
@@ -8,6 +8,7 @@ import (
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
"net/url"
"sort" "sort"
"sync" "sync"
"time" "time"
@@ -32,6 +33,11 @@ type HTTPTargetConfig struct {
URL string `json:"url"` URL string `json:"url"`
Headers map[string]string `json:"headers,omitempty"` Headers map[string]string `json:"headers,omitempty"`
Timeout int `json:"timeout,omitempty"` Timeout int `json:"timeout,omitempty"`
// ForwardQuery passes each event's query string on to the target,
// appended to URL. Off, the target URL is sent exactly as
// configured.
ForwardQuery bool `json:"forwardQuery,omitempty"`
} }
// httpCore holds the retry, backoff, and circuit-breaker // httpCore holds the retry, backoff, and circuit-breaker
@@ -444,6 +450,10 @@ func (t *httpTarget) doHTTPRequest(
) )
} }
if cfg.ForwardQuery {
appendQuery(req.URL, event.RawQuery)
}
originScoped := applyRequestHeaders( originScoped := applyRequestHeaders(
req, event, cfg, t.eng.userAgent(), req, event, cfg, t.eng.userAgent(),
) )
@@ -474,6 +484,19 @@ func (t *httpTarget) doHTTPRequest(
return resp.StatusCode, string(body), dur, nil return resp.StatusCode, string(body), dur, nil
} }
// appendQuery adds an event's query string to a delivery's URL, joined
// with "&" to any query string the target URL already has.
func appendQuery(u *url.URL, rawQuery string) {
switch {
case rawQuery == "":
return
case u.RawQuery == "":
u.RawQuery = rawQuery
default:
u.RawQuery += "&" + rawQuery
}
}
// clientForRequest returns the client for one delivery attempt. // clientForRequest returns the client for one delivery attempt.
// originScoped is the header set applyRequestHeaders built for that // originScoped is the header set applyRequestHeaders built for that
// attempt; a request with neither a per-target timeout nor an // attempt; a request with neither a per-target timeout nor an
+172
View File
@@ -0,0 +1,172 @@
package delivery_test
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
)
// eventQuery is the query string the events in these tests arrived
// with.
const eventQuery = "a=1&b=2"
// httpTargetConfig is the stored configuration of an HTTP target at
// targetURL.
func httpTargetConfig(
t *testing.T, targetURL string, forwardQuery bool,
) string {
t.Helper()
cfg, err := json.Marshal(delivery.HTTPTargetConfig{
URL: targetURL, ForwardQuery: forwardQuery,
})
require.NoError(t, err)
return string(cfg)
}
// deliverWithQuery sends one event that arrived with eventQuery to an
// HTTP target configured with cfg, through the path a received event's
// delivery takes, and returns the attempt it recorded.
func deliverWithQuery(t *testing.T, cfg string) database.DeliveryResult {
t.Helper()
s := newISetup(t)
event := iSeedEvent(t, s.WebhookDB, s.WebhookID, "{}")
d := iSeedDelivery(
t, s.WebhookDB, event.ID, uuid.NewString(),
database.DeliveryStatusPending,
)
task := iTask(
d, event, s.WebhookID, d.TargetID, "query", cfg, 0, 1, &event.Body,
)
task.RawQuery = eventQuery
s.Engine.ExportProcessNewTask(context.TODO(), &task)
var result database.DeliveryResult
require.NoError(t, s.WebhookDB.Where(
"delivery_id = ?", d.ID,
).First(&result).Error)
return result
}
// TestDeliverHTTP_ForwardQuery proves the URL a delivery is sent to:
// with the target's setting off, the target URL exactly as configured;
// with it on, the event's query string appended, joined with "&" to a
// query string the target URL already has.
func TestDeliverHTTP_ForwardQuery(t *testing.T) {
t.Parallel()
// The target URL's path, without and with a query string of its
// own.
const (
plain = "/in"
withQuery = "/in?key=k"
)
tests := map[string]struct {
path string
forwardQuery bool
want string
}{
"off": {
path: plain, want: plain,
},
"off, the target URL has a query string": {
path: withQuery, want: withQuery,
},
"on": {
path: plain, forwardQuery: true, want: plain + "?" + eventQuery,
},
"on, the target URL has a query string": {
path: withQuery, forwardQuery: true,
want: withQuery + "&" + eventQuery,
},
}
for name, tc := range tests {
t.Run(name, func(t *testing.T) {
t.Parallel()
received := make(chan string, 1)
ts := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
received <- r.RequestURI
w.WriteHeader(http.StatusOK)
},
))
t.Cleanup(ts.Close)
result := deliverWithQuery(t, httpTargetConfig(
t, ts.URL+tc.path, tc.forwardQuery,
))
assert.True(t, result.Success)
require.Len(t, received, 1)
assert.Equal(t, tc.want, <-received)
})
}
}
// TestDeliverHTTP_ForwardedQueryKeepsTheTargetURLMasked proves the
// credential in a target URL's own query string stays masked once the
// event's query string is appended to it: in a response or error that
// echoes the URL the target was sent, as the event log's Redactor shows
// it, and in the error a failed connection stores.
func TestDeliverHTTP_ForwardedQueryKeepsTheTargetURLMasked(t *testing.T) {
t.Parallel()
const secret = "s3cr3t"
received := make(chan string, 1)
ts := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
received <- r.RequestURI
w.WriteHeader(http.StatusBadRequest)
},
))
t.Cleanup(ts.Close)
target := &database.Target{
Type: database.TargetTypeHTTP,
Config: httpTargetConfig(t, ts.URL+"/in?token="+secret, true),
}
deliverWithQuery(t, target.Config)
require.Len(t, received, 1)
sent := <-received
require.Equal(t, "/in?token="+secret+"&"+eventQuery, sent)
redactor := delivery.NewRedactor(target)
for _, echoed := range []string{sent, ts.URL + sent} {
shown := redactor.Redact("rejected " + echoed)
assert.NotContains(t, shown, secret, echoed)
assert.Contains(t, shown, delivery.RedactionMarker, echoed)
}
// Nothing listens on port 1.
failed := deliverWithQuery(t, httpTargetConfig(
t, "http://127.0.0.1:1/in?token="+secret, true,
))
require.NotEmpty(t, failed.Error)
assert.NotContains(t, failed.Error, secret)
assert.NotContains(t, failed.Error, eventQuery)
}
+4 -3
View File
@@ -9,9 +9,9 @@ import (
) )
// logTarget is a fire-and-forget target that logs the entire // logTarget is a fire-and-forget target that logs the entire
// inbound webhook — the full request body and headers, plus // inbound webhook — the full request body, query string and
// the method, content type, and the webhook and entrypoint // headers, plus the method, content type, and the webhook and
// ids — then records a single successful attempt. // entrypoint ids — then records a single successful attempt.
// //
// This is the one log call in the service that deliberately writes // This is the one log call in the service that deliberately writes
// unbounded client-chosen bytes, so it is the one exception to the // unbounded client-chosen bytes, so it is the one exception to the
@@ -46,6 +46,7 @@ func (t *logTarget) Deliver(
"webhook_id", d.Event.WebhookID, "webhook_id", d.Event.WebhookID,
"entrypoint_id", d.Event.EntrypointID, "entrypoint_id", d.Event.EntrypointID,
"method", d.Event.Method, "method", d.Event.Method,
"raw_query", d.Event.RawQuery,
"content_type", d.Event.ContentType, "content_type", d.Event.ContentType,
"headers", d.Event.Headers, "headers", d.Event.Headers,
"body", d.Event.Body, "body", d.Event.Body,
+1
View File
@@ -310,6 +310,7 @@ func createReplayDelivery(
TargetConfig: target.Config, TargetConfig: target.Config,
MaxRetries: target.MaxRetries, MaxRetries: target.MaxRetries,
Method: event.Method, Method: event.Method,
RawQuery: event.RawQuery,
Headers: event.Headers, Headers: event.Headers,
ContentType: event.ContentType, ContentType: event.ContentType,
Body: replayBody(event.Body), Body: replayBody(event.Body),
@@ -26,6 +26,9 @@ const paramDeliveryID = "deliveryID"
// dispatches to it: the notifier is recorded, not run. // dispatches to it: the notifier is recorded, not run.
const replayTargetURL = "http://93.184.216.34/hook" const replayTargetURL = "http://93.184.216.34/hook"
// replayEventQuery is the query string a seeded event arrived with.
const replayEventQuery = "a=1&b=2"
// seedFailedDelivery records an event, a terminally failed delivery of // seedFailedDelivery records an event, a terminally failed delivery of
// it to the given target, and the attempt that failed. // it to the given target, and the attempt that failed.
func seedFailedDelivery( func seedFailedDelivery(
@@ -42,6 +45,7 @@ func seedFailedDelivery(
WebhookID: webhookID, WebhookID: webhookID,
EntrypointID: "entrypoint-" + webhookID, EntrypointID: "entrypoint-" + webhookID,
Method: http.MethodPost, Method: http.MethodPost,
RawQuery: replayEventQuery,
Headers: `{"X-Test":["yes"]}`, Headers: `{"X-Test":["yes"]}`,
Body: `{"replay":"me"}`, Body: `{"replay":"me"}`,
ContentType: contentTypeJSON, ContentType: contentTypeJSON,
@@ -296,6 +300,10 @@ func assertReplayTask(
"replay must use the target's current configuration", "replay must use the target's current configuration",
) )
assert.Equal(t, event.Method, task.Method) assert.Equal(t, event.Method, task.Method)
assert.Equal(
t, replayEventQuery, task.RawQuery,
"replay re-sends the stored query string",
)
assert.Equal(t, event.Headers, task.Headers) assert.Equal(t, event.Headers, task.Headers)
assert.Equal(t, event.ContentType, task.ContentType) assert.Equal(t, event.ContentType, task.ContentType)
assert.Equal(t, 1, task.AttemptNum) assert.Equal(t, 1, task.AttemptNum)
+2 -1
View File
@@ -324,7 +324,8 @@ func loadEventLogRows(
var rows []eventLogRow var rows []eventLogRow
err = eventsWithStatus(webhookDB, webhookID, statuses).Select( err = eventsWithStatus(webhookDB, webhookID, statuses).Select(
eventLogColumns, maxRenderedBodyBytes, maxRenderedBodyBytes, eventLogColumns,
maxRenderedBodyBytes, maxRenderedBodyBytes, maxRenderedBodyBytes,
).Order("created_at DESC").Limit(recentEventLimit).Find(&rows).Error ).Order("created_at DESC").Limit(recentEventLimit).Find(&rows).Error
return rows, totalEvents, err return rows, totalEvents, err
+29 -7
View File
@@ -17,19 +17,23 @@ import (
// bytes rather than characters, so the cap bounds the page in // bytes rather than characters, so the cap bounds the page in
// bytes whatever the payload's encoding. Cutting in SQLite // bytes whatever the payload's encoding. Cutting in SQLite
// rather than in Go is the point of the projection — an // rather than in Go is the point of the projection — an
// oversized body or set of request headers never becomes a Go // oversized body, query string or set of request headers never
// string at all. // becomes a Go string at all.
const eventLogColumns = "id, created_at, method, content_type, " + const eventLogColumns = "id, created_at, method, content_type, " +
"resubmitted_from_id, entrypoint_id, " + "resubmitted_from_id, entrypoint_id, " +
"substr(cast(raw_query as blob), 1, ?) AS raw_query, " +
"length(cast(raw_query as blob)) AS raw_query_bytes, " +
"substr(cast(headers as blob), 1, ?) AS headers, " + "substr(cast(headers as blob), 1, ?) AS headers, " +
"length(cast(headers as blob)) AS headers_bytes, " + "length(cast(headers as blob)) AS headers_bytes, " +
"substr(cast(body as blob), 1, ?) AS body, " + "substr(cast(body as blob), 1, ?) AS body, " +
"length(cast(body as blob)) AS body_bytes" "length(cast(body as blob)) AS body_bytes"
// eventColumns is eventLogColumns for the event's own page, which // eventColumns is eventLogColumns for the event's own page, which
// shows the whole body and every request header. // shows the whole body, the whole query string and every request
// header.
const eventColumns = "id, created_at, method, content_type, " + const eventColumns = "id, created_at, method, content_type, " +
"resubmitted_from_id, entrypoint_id, headers, " + "resubmitted_from_id, entrypoint_id, raw_query, " +
"length(cast(raw_query as blob)) AS raw_query_bytes, headers, " +
"length(cast(headers as blob)) AS headers_bytes, " + "length(cast(headers as blob)) AS headers_bytes, " +
"cast(body as blob) AS body, " + "cast(body as blob) AS body, " +
"length(cast(body as blob)) AS body_bytes" "length(cast(body as blob)) AS body_bytes"
@@ -57,6 +61,13 @@ type EventLogView struct {
// entrypoint's secret. // entrypoint's secret.
Entrypoint string Entrypoint string
// RawQuery is the query string the event arrived with.
// RawQueryCut reports one left out, RawQuery then empty, because
// it holds more than maxRenderedBodyBytes; only the event log
// leaves it out.
RawQuery string
RawQueryCut bool
// Headers is the event's request headers as text, one // Headers is the event's request headers as text, one
// "Name: value" line per value, sorted by name. HeadersCut // "Name: value" line per value, sorted by name. HeadersCut
// reports headers left out because they hold more than // reports headers left out because they hold more than
@@ -85,9 +96,9 @@ func (v EventLogView) ResubmittedFrom() bool {
} }
// eventLogRow is one row of the event log projection, or of // eventLogRow is one row of the event log projection, or of
// eventColumns. In the event log its headers and body columns // eventColumns. In the event log its query string, headers and
// arrive already cut to the cap by SQLite, each with its true // body columns arrive already cut to the cap by SQLite, each with
// size beside it. // its true size beside it.
type eventLogRow struct { type eventLogRow struct {
ID string ID string
CreatedAt time.Time CreatedAt time.Time
@@ -95,6 +106,8 @@ type eventLogRow struct {
ContentType string ContentType string
ResubmittedFromID *string ResubmittedFromID *string
EntrypointID string EntrypointID string
RawQuery string
RawQueryBytes int64
Headers string Headers string
HeadersBytes int64 HeadersBytes int64
Body []byte Body []byte
@@ -114,6 +127,13 @@ func (r *eventLogRow) view(
headers, fit := requestHeaderLines(r.Headers, maxHeaderBytes) headers, fit := requestHeaderLines(r.Headers, maxHeaderBytes)
rawQuery := r.RawQuery
rawQueryCut := r.RawQueryBytes > int64(len(rawQuery))
if rawQueryCut {
rawQuery = ""
}
return EventLogView{ return EventLogView{
ID: r.ID, ID: r.ID,
Method: r.Method, Method: r.Method,
@@ -123,6 +143,8 @@ func (r *eventLogRow) view(
Body: newBodyView( Body: newBodyView(
"/hook/"+webhookID+"/events/"+r.ID, r.Body, r.BodyBytes, "/hook/"+webhookID+"/events/"+r.ID, r.Body, r.BodyBytes,
), ),
RawQuery: rawQuery,
RawQueryCut: rawQueryCut,
Headers: strings.Join(headers, "\n"), Headers: strings.Join(headers, "\n"),
HeadersCut: !fit || r.HeadersBytes > int64(len(r.Headers)), HeadersCut: !fit || r.HeadersBytes > int64(len(r.Headers)),
ResubmittedFromID: from, ResubmittedFromID: from,
+71
View File
@@ -1,13 +1,16 @@
package handlers_test package handlers_test
import ( import (
"context"
"encoding/json" "encoding/json"
"net/http" "net/http"
"net/http/httptest"
"slices" "slices"
"strings" "strings"
"testing" "testing"
"time" "time"
"github.com/go-chi/chi"
"github.com/google/uuid" "github.com/google/uuid"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
@@ -116,6 +119,7 @@ func TestEventRequest_EachEventShowsItsOwnEntrypointAndHeaders(
t.Helper() t.Helper()
assert.Contains(t, page, arrivedAt("Billing sender")) assert.Contains(t, page, arrivedAt("Billing sender"))
assert.Contains(t, page, "No query string.")
assert.Contains(t, page, headerBox( assert.Contains(t, page, headerBox(
"Accept: */*", "Accept: */*",
"User-Agent: shop/1 build\t7", "User-Agent: shop/1 build\t7",
@@ -331,3 +335,70 @@ func TestEventRequest_ManyShortHeaderLines(t *testing.T) {
}) })
} }
} }
// TestHandleWebhook_StoresAndShowsTheQueryString posts to an
// entrypoint's URL with a query string and proves the event stores it
// as sent, and shows it escaped in the event log and on its own page,
// in a box like the one the request headers show in.
func TestHandleWebhook_StoresAndShowsTheQueryString(t *testing.T) {
t.Parallel()
f := newRecentEventsFixture(t)
ep := seedEntrypoint(t, f.db, f.webhook.ID)
req := httptest.NewRequestWithContext(
context.Background(), http.MethodPost,
"/h/"+ep.Path+"?a=1&b=2", strings.NewReader("{}"),
)
rctx := chi.NewRouteContext()
rctx.URLParams.Add("uuid", ep.Path)
req = req.WithContext(context.WithValue(
req.Context(), chi.RouteCtxKey, rctx,
))
w := httptest.NewRecorder()
f.h.HandleWebhook().ServeHTTP(w, req)
require.Equal(t, http.StatusOK, w.Code)
var stored database.Event
require.NoError(t, f.webhookDB.First(&stored).Error)
assert.Equal(t, "a=1&b=2", stored.RawQuery)
page := renderSourceLogsPage(t, f.h, f.sess, f.webhook.ID)
assert.Contains(t, page, headerBox("a=1&amp;b=2"))
w = serveEventPage(t, f.h, f.sess, f.webhook.ID, stored.ID)
require.Equal(t, http.StatusOK, w.Code)
assert.Contains(t, w.Body.String(), headerBox("a=1&amp;b=2"))
}
// TestEventRequest_QueryStringOverTheLimit proves the event log leaves
// out a query string that holds more than it shows of a body, and links
// to the event's own page, which shows it whole.
func TestEventRequest_QueryStringOverTheLimit(t *testing.T) {
t.Parallel()
f := newRecentEventsFixture(t)
ep := f.entrypoint(t, "Billing sender")
event := f.eventAt(t, ep, `{}`, time.Now())
query := "q=" + strings.Repeat("x", bodyCap)
require.NoError(t, f.webhookDB.Model(event).Update(
"raw_query", query,
).Error)
page := renderSourceLogsPage(t, f.h, f.sess, f.webhook.ID)
assert.Contains(t, page, `<a href="/hook/`+f.webhook.ID+`/events/`+
event.ID+`" class="btn-small">Show the query string</a>`)
assert.NotContains(t, page, "q=x")
assert.Less(t, len(page), 4*bodyCap)
w := serveEventPage(t, f.h, f.sess, f.webhook.ID, event.ID)
require.Equal(t, http.StatusOK, w.Code)
assert.Contains(t, w.Body.String(), headerBox(query))
assert.NotContains(t, w.Body.String(), "Show the query string")
}
+3 -1
View File
@@ -30,6 +30,7 @@ type resubmitSource struct {
ID string ID string
EntrypointID string EntrypointID string
Method string Method string
RawQuery string
Headers string Headers string
ContentType string ContentType string
Body []byte Body []byte
@@ -39,7 +40,7 @@ type resubmitSource struct {
// The cast to blob is what makes the driver hand back the stored bytes // The cast to blob is what makes the driver hand back the stored bytes
// rather than a string conversion, the same reason eventBodyQuery // rather than a string conversion, the same reason eventBodyQuery
// casts. // casts.
const resubmitColumns = "id, entrypoint_id, method, headers, " + const resubmitColumns = "id, entrypoint_id, method, raw_query, headers, " +
"content_type, cast(body as blob) AS body" "content_type, cast(body as blob) AS body"
// HandleEventResubmit re-injects a stored event as a new undelivered // HandleEventResubmit re-injects a stored event as a new undelivered
@@ -193,6 +194,7 @@ func (h *Handlers) queueResubmit(
WebhookID: webhook.ID, WebhookID: webhook.ID,
EntrypointID: src.EntrypointID, EntrypointID: src.EntrypointID,
Method: src.Method, Method: src.Method,
RawQuery: src.RawQuery,
HeadersJSON: src.Headers, HeadersJSON: src.Headers,
ContentType: src.ContentType, ContentType: src.ContentType,
Body: src.Body, Body: src.Body,
+10 -3
View File
@@ -22,9 +22,13 @@ import (
// dispatches to it: the notifier is recorded, not run. // dispatches to it: the notifier is recorded, not run.
const resubmitTargetURL = "http://93.184.216.34/hook" const resubmitTargetURL = "http://93.184.216.34/hook"
// resubmitEventHeaders is the stored header JSON a seeded event // resubmitEventHeaders and resubmitEventQuery are the stored header
// carries, so a test can prove the copy takes it verbatim. // JSON and query string a seeded event carries, so a test can prove the
const resubmitEventHeaders = `{"X-Test":["yes"],"X-Trace":["abc"]}` // copy takes them verbatim.
const (
resubmitEventHeaders = `{"X-Test":["yes"],"X-Trace":["abc"]}`
resubmitEventQuery = "a=1&b=2"
)
// seedStoredEvent records one event in a webhook's own database with // seedStoredEvent records one event in a webhook's own database with
// no deliveries at all, which is the state a captured event is in when // no deliveries at all, which is the state a captured event is in when
@@ -43,6 +47,7 @@ func seedStoredEvent(
WebhookID: webhookID, WebhookID: webhookID,
EntrypointID: "entrypoint-" + webhookID, EntrypointID: "entrypoint-" + webhookID,
Method: http.MethodPost, Method: http.MethodPost,
RawQuery: resubmitEventQuery,
Headers: resubmitEventHeaders, Headers: resubmitEventHeaders,
Body: body, Body: body,
ContentType: contentTypeJSON, ContentType: contentTypeJSON,
@@ -202,6 +207,7 @@ func assertEventCopy(
t.Helper() t.Helper()
assert.Equal(t, original.Method, fresh.Method) assert.Equal(t, original.Method, fresh.Method)
assert.Equal(t, resubmitEventQuery, fresh.RawQuery)
assert.Equal(t, original.Headers, fresh.Headers) assert.Equal(t, original.Headers, fresh.Headers)
assert.Equal(t, original.Body, fresh.Body) assert.Equal(t, original.Body, fresh.Body)
assert.Equal(t, int64(len(original.Body)), fresh.BodyBytes) assert.Equal(t, int64(len(original.Body)), fresh.BodyBytes)
@@ -236,6 +242,7 @@ func assertResubmitTask(
assert.Equal(t, target.ID, task.TargetID) assert.Equal(t, target.ID, task.TargetID)
assert.Equal(t, target.Type, task.TargetType) assert.Equal(t, target.Type, task.TargetType)
assert.Equal(t, fresh.Method, task.Method) assert.Equal(t, fresh.Method, task.Method)
assert.Equal(t, fresh.RawQuery, task.RawQuery)
assert.Equal(t, fresh.Headers, task.Headers) assert.Equal(t, fresh.Headers, task.Headers)
assert.Equal(t, fresh.ContentType, task.ContentType) assert.Equal(t, fresh.ContentType, task.ContentType)
assert.Equal(t, 1, task.AttemptNum) assert.Equal(t, 1, task.AttemptNum)
+18 -13
View File
@@ -173,6 +173,9 @@ type targetFormInput struct {
Headers string Headers string
// Timeout is an HTTP target's per-request timeout in seconds. // Timeout is an HTTP target's per-request timeout in seconds.
Timeout string Timeout string
// ForwardQuery is an HTTP target's checkbox that passes each
// event's query string on to it.
ForwardQuery bool
// MaxRetries is an HTTP or Slack target's max_retries. // MaxRetries is an HTTP or Slack target's max_retries.
MaxRetries string MaxRetries string
// Expiry is a database (archive) target's row expiry. // Expiry is a database (archive) target's row expiry.
@@ -195,14 +198,15 @@ type targetFormInput struct {
// tokens. // tokens.
func targetFormInputFrom(r *http.Request) targetFormInput { func targetFormInputFrom(r *http.Request) targetFormInput {
return targetFormInput{ return targetFormInput{
Name: r.PostFormValue("name"), Name: r.PostFormValue("name"),
Type: database.TargetType(r.PostFormValue("type")), Type: database.TargetType(r.PostFormValue("type")),
URL: r.PostFormValue("url"), URL: r.PostFormValue("url"),
Headers: r.PostFormValue("headers"), Headers: r.PostFormValue("headers"),
Timeout: r.PostFormValue("timeout"), Timeout: r.PostFormValue("timeout"),
MaxRetries: r.PostFormValue("max_retries"), ForwardQuery: r.PostFormValue("forward_query") != "",
Expiry: r.PostFormValue("expiry"), MaxRetries: r.PostFormValue("max_retries"),
Rotation: r.PostFormValue("rotation"), Expiry: r.PostFormValue("expiry"),
Rotation: r.PostFormValue("rotation"),
} }
} }
@@ -232,8 +236,8 @@ func (h *Handlers) buildTargetConfig(
} }
// buildHTTPTargetConfig builds config JSON for an HTTP target: an // buildHTTPTargetConfig builds config JSON for an HTTP target: an
// SSRF-validated destination plus the optional headers and timeout // SSRF-validated destination plus the optional headers, timeout and
// the delivery path honours. // query string setting the delivery path honours.
func (h *Handlers) buildHTTPTargetConfig( func (h *Handlers) buildHTTPTargetConfig(
ctx context.Context, ctx context.Context,
in targetFormInput, in targetFormInput,
@@ -256,9 +260,10 @@ func (h *Handlers) buildHTTPTargetConfig(
} }
configJSON, err := marshalTargetConfig(delivery.HTTPTargetConfig{ configJSON, err := marshalTargetConfig(delivery.HTTPTargetConfig{
URL: in.URL, URL: in.URL,
Headers: headers, Headers: headers,
Timeout: timeout, Timeout: timeout,
ForwardQuery: in.ForwardQuery,
}) })
return configJSON, "", err return configJSON, "", err
+8 -7
View File
@@ -77,13 +77,14 @@ func (h *Handlers) HandleTargetEdit() http.HandlerFunc {
} }
form := targetFormInput{ form := targetFormInput{
Name: target.Name, Name: target.Name,
URL: cfg.URL, URL: cfg.URL,
Headers: cfg.Headers, Headers: cfg.Headers,
Timeout: cfg.Timeout, Timeout: cfg.Timeout,
MaxRetries: strconv.Itoa(target.MaxRetries), ForwardQuery: cfg.ForwardQuery,
Expiry: cfg.Expiry, MaxRetries: strconv.Itoa(target.MaxRetries),
Rotation: cfg.Rotation, Expiry: cfg.Expiry,
Rotation: cfg.Rotation,
} }
h.renderTargetEdit( h.renderTargetEdit(
+53
View File
@@ -442,6 +442,59 @@ func TestHandleTargetEdit_CallsTheDatabaseTypeArchive(t *testing.T) {
assert.Contains(t, page, `class="label">Archive rotation</label>`) assert.Contains(t, page, `class="label">Archive rotation</label>`)
} }
// TestHandleTarget_ForwardQuery covers the HTTP target's setting that
// passes each event's query string on to it: the add target form
// stores it checked, the edit form starts with it checked and turns it
// off when saved unchecked, and both forms come back with it checked
// when refused.
func TestHandleTarget_ForwardQuery(t *testing.T) {
t.Parallel()
const checkbox = `name="forward_query" value="on" checked`
env := setupSourceTest(t)
webhook := seedWebhookWithRetention(t, env.db, 30)
targetsPath := "/hook/" + webhook.ID + "/targets"
form := url.Values{}
form.Set("name", "forwarding")
form.Set("type", string(database.TargetTypeHTTP))
form.Set("url", editOriginalURL)
form.Set("forward_query", "on")
w := serveTarget(env, http.MethodPost, targetsPath, form)
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
targets := targetsForWebhook(t, env.db, webhook.ID)
require.Len(t, targets, 1)
assert.True(t, storedHTTPConfig(t, env, targets[0].ID).ForwardQuery)
w = serveTarget(
env, http.MethodGet, targetsPath+"/"+targets[0].ID+"/edit", nil,
)
require.Equal(t, http.StatusOK, w.Code)
assert.Contains(t, w.Body.String(), checkbox)
edit := editForm(editOriginalURL, "", "")
w = submitTargetEdit(env, webhook.ID, targets[0].ID, edit)
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
assert.False(t, storedHTTPConfig(t, env, targets[0].ID).ForwardQuery)
edit.Set("url", editBlockedURL)
edit.Set("forward_query", "on")
w = submitTargetEdit(env, webhook.ID, targets[0].ID, edit)
require.Equal(t, http.StatusBadRequest, w.Code)
assert.Contains(t, w.Body.String(), checkbox)
form.Set("url", editBlockedURL)
w = serveTarget(env, http.MethodPost, targetsPath, form)
require.Equal(t, http.StatusBadRequest, w.Code)
assert.Contains(t, w.Body.String(), "data-forward-query")
}
// TestHandleTargetEditSubmit_Rejects covers every submission that // TestHandleTargetEditSubmit_Rejects covers every submission that
// must not reach storage. // must not reach storage.
// //
+4
View File
@@ -230,6 +230,7 @@ type eventSource struct {
WebhookID string WebhookID string
EntrypointID string EntrypointID string
Method string Method string
RawQuery string
HeadersJSON string HeadersJSON string
ContentType string ContentType string
Body []byte Body []byte
@@ -245,6 +246,7 @@ func (s eventSource) event() *database.Event {
WebhookID: s.WebhookID, WebhookID: s.WebhookID,
EntrypointID: s.EntrypointID, EntrypointID: s.EntrypointID,
Method: s.Method, Method: s.Method,
RawQuery: s.RawQuery,
Headers: s.HeadersJSON, Headers: s.HeadersJSON,
Body: string(s.Body), Body: string(s.Body),
BodyBytes: int64(len(s.Body)), BodyBytes: int64(len(s.Body)),
@@ -264,6 +266,7 @@ func requestEventSource(
WebhookID: entrypoint.WebhookID, WebhookID: entrypoint.WebhookID,
EntrypointID: entrypoint.ID, EntrypointID: entrypoint.ID,
Method: r.Method, Method: r.Method,
RawQuery: r.URL.RawQuery,
HeadersJSON: string(headersJSON), HeadersJSON: string(headersJSON),
ContentType: r.Header.Get("Content-Type"), ContentType: r.Header.Get("Content-Type"),
Body: body, Body: body,
@@ -441,6 +444,7 @@ func buildDeliveryTasks(
TargetConfig: targets[i].Config, TargetConfig: targets[i].Config,
MaxRetries: targets[i].MaxRetries, MaxRetries: targets[i].MaxRetries,
Method: event.Method, Method: event.Method,
RawQuery: event.RawQuery,
Headers: event.Headers, Headers: event.Headers,
ContentType: event.ContentType, ContentType: event.ContentType,
Body: bodyPtr, Body: bodyPtr,
+19 -4
View File
@@ -507,9 +507,10 @@ func checkRefusedTarget(ctx context.Context, t *testing.T, url string) {
t.Helper() t.Helper()
const ( const (
refusedURL = "http://127.0.0.1/hook" refusedURL = "http://127.0.0.1/hook"
urlField = `form[action$="/targets"] input[name="url"]` urlField = `form[action$="/targets"] input[name="url"]`
reason = `//div[@class="alert-error"]` forwardQuery = `form[action$="/targets"] input[name="forward_query"]`
reason = `//div[@class="alert-error"]`
) )
require.NoError(t, chromedp.Run(ctx, loadPage(url))) require.NoError(t, chromedp.Run(ctx, loadPage(url)))
@@ -519,6 +520,7 @@ func checkRefusedTarget(ctx context.Context, t *testing.T, url string) {
ctx, ctx,
chromedp.SetValue(targetName, "refused", chromedp.ByQuery), chromedp.SetValue(targetName, "refused", chromedp.ByQuery),
chromedp.SetValue(urlField, refusedURL, chromedp.ByQuery), chromedp.SetValue(urlField, refusedURL, chromedp.ByQuery),
chromedp.Click(forwardQuery, chromedp.ByQuery),
)) ))
click(ctx, t, saveButton) click(ctx, t, saveButton)
@@ -526,18 +528,26 @@ func checkRefusedTarget(ctx context.Context, t *testing.T, url string) {
assert.True(t, shown(ctx, reason), assert.True(t, shown(ctx, reason),
"a refused target does not show the reason") "a refused target does not show the reason")
var name, typed string var (
name, typed string
checked bool
)
require.NoError(t, chromedp.Run( require.NoError(t, chromedp.Run(
ctx, ctx,
chromedp.Value(targetName, &name, chromedp.ByQuery), chromedp.Value(targetName, &name, chromedp.ByQuery),
chromedp.Value(urlField, &typed, chromedp.ByQuery), chromedp.Value(urlField, &typed, chromedp.ByQuery),
chromedp.JavascriptAttribute(
forwardQuery, "checked", &checked, chromedp.ByQuery,
),
)) ))
assert.Equal(t, "refused", name, assert.Equal(t, "refused", name,
"a refused target does not keep the name entered") "a refused target does not keep the name entered")
assert.Equal(t, refusedURL, typed, assert.Equal(t, refusedURL, typed,
"a refused target does not keep the url entered") "a refused target does not keep the url entered")
assert.True(t, checked,
"a refused target does not keep the query string setting checked")
assert.True(t, shown(ctx, targetName), assert.True(t, shown(ctx, targetName),
"a refused target does not come back with the form open") "a refused target does not come back with the form open")
assert.True(t, hidden(ctx, typeSelect), assert.True(t, hidden(ctx, typeSelect),
@@ -553,10 +563,15 @@ func checkRefusedTarget(ctx context.Context, t *testing.T, url string) {
ctx, ctx,
chromedp.Value(targetName, &name, chromedp.ByQuery), chromedp.Value(targetName, &name, chromedp.ByQuery),
chromedp.Value(urlField, &typed, chromedp.ByQuery), chromedp.Value(urlField, &typed, chromedp.ByQuery),
chromedp.JavascriptAttribute(
forwardQuery, "checked", &checked, chromedp.ByQuery,
),
)) ))
assert.Empty(t, name, "after Cancel, the next Add keeps the name entered") assert.Empty(t, name, "after Cancel, the next Add keeps the name entered")
assert.Empty(t, typed, "after Cancel, the next Add keeps the url entered") assert.Empty(t, typed, "after Cancel, the next Add keeps the url entered")
assert.False(t, checked,
"after Cancel, the next Add keeps the query string setting checked")
} }
// checkTargetDeliveries loads a webhook page and checks that the row of // checkTargetDeliveries loads a webhook page and checks that the row of
+4
View File
@@ -138,6 +138,7 @@ document.addEventListener("alpine:init", function () {
url: "", url: "",
headers: "", headers: "",
timeout: "", timeout: "",
forwardQuery: false,
maxRetries: "", maxRetries: "",
expiry: "", expiry: "",
rotation: "", rotation: "",
@@ -150,6 +151,8 @@ document.addEventListener("alpine:init", function () {
this.url = refused.destination; this.url = refused.destination;
this.headers = refused.headers; this.headers = refused.headers;
this.timeout = refused.timeout; this.timeout = refused.timeout;
this.forwardQuery =
this.$root.hasAttribute("data-forward-query");
this.maxRetries = refused.maxRetries; this.maxRetries = refused.maxRetries;
this.expiry = refused.expiry; this.expiry = refused.expiry;
this.rotation = refused.rotation; this.rotation = refused.rotation;
@@ -169,6 +172,7 @@ document.addEventListener("alpine:init", function () {
this.url = ""; this.url = "";
this.headers = ""; this.headers = "";
this.timeout = ""; this.timeout = "";
this.forwardQuery = false;
this.maxRetries = ""; this.maxRetries = "";
this.expiry = ""; this.expiry = "";
this.rotation = ""; this.rotation = "";
+13 -5
View File
@@ -1,15 +1,23 @@
{{define "event_request"}} {{define "event_request"}}
<!-- The entrypoint an event arrived at and its request headers, as <!-- The entrypoint an event arrived at, its query string and its
handlers.EventLogView carries them: the same in the event log and request headers, as handlers.EventLogView carries them: the same in
the event's own page. The entrypoint's URL is never shown. A the event log and the event's own page. The entrypoint's URL is
resubmitted copy, even a copy of a copy, did not arrive at an never shown. A resubmitted copy, even a copy of a copy, did not
entrypoint; the request it copies did. --> arrive at an entrypoint; the request it copies did. -->
<div class="space-y-2 text-xs"> <div class="space-y-2 text-xs">
{{if .ResubmittedFrom}} {{if .ResubmittedFrom}}
<p class="text-gray-500">The request it copies arrived at <span class="text-gray-900 wrap-anywhere">{{.Entrypoint}}</span></p> <p class="text-gray-500">The request it copies arrived at <span class="text-gray-900 wrap-anywhere">{{.Entrypoint}}</span></p>
{{else}} {{else}}
<p class="text-gray-500">Arrived at <span class="text-gray-900 wrap-anywhere">{{.Entrypoint}}</span></p> <p class="text-gray-500">Arrived at <span class="text-gray-900 wrap-anywhere">{{.Entrypoint}}</span></p>
{{end}} {{end}}
{{if .RawQueryCut}}
<p class="text-gray-500">The query string is larger than the event log shows. <a href="{{.Body.EventURL}}" class="btn-small">Show the query string</a></p>
{{else if .RawQuery}}
<p class="text-gray-500">Query string</p>
<pre class="rounded-md border border-gray-200 bg-white p-2 text-xs text-gray-700 overflow-x-auto whitespace-pre-wrap break-all">{{.RawQuery}}</pre>
{{else}}
<p class="text-gray-500">No query string.</p>
{{end}}
{{if .HeadersCut}} {{if .HeadersCut}}
<p class="text-gray-500">The request headers are larger than the event log shows. <a href="{{.Body.EventURL}}" class="btn-small">Show the request headers</a></p> <p class="text-gray-500">The request headers are larger than the event log shows. <a href="{{.Body.EventURL}}" class="btn-small">Show the request headers</a></p>
{{else if .Headers}} {{else if .Headers}}
+8
View File
@@ -135,6 +135,7 @@
data-destination="{{.TargetForm.URL}}" data-destination="{{.TargetForm.URL}}"
data-headers="{{.TargetForm.Headers}}" data-headers="{{.TargetForm.Headers}}"
data-timeout="{{.TargetForm.Timeout}}" data-timeout="{{.TargetForm.Timeout}}"
{{if .TargetForm.ForwardQuery}}data-forward-query{{end}}
data-max-retries="{{.TargetForm.MaxRetries}}" data-max-retries="{{.TargetForm.MaxRetries}}"
data-expiry="{{.TargetForm.Expiry}}" data-expiry="{{.TargetForm.Expiry}}"
data-rotation="{{.TargetForm.Rotation}}"> data-rotation="{{.TargetForm.Rotation}}">
@@ -183,6 +184,13 @@
<label class="text-sm text-gray-700">Timeout (seconds, blank = default):</label> <label class="text-sm text-gray-700">Timeout (seconds, blank = default):</label>
<input type="number" name="timeout" :value="timeout" min="0" max="300" class="input text-sm w-24"> <input type="number" name="timeout" :value="timeout" min="0" max="300" class="input text-sm w-24">
</div> </div>
<div>
<label class="flex items-center gap-2 text-sm text-gray-700">
<input type="checkbox" name="forward_query" value="on" :checked="forwardQuery" class="h-4 w-4">
Pass the query string on to this target
</label>
<p class="text-xs text-gray-500 mt-1">Appends the query string each event arrived with to the URL above, after any query string the URL already has.</p>
</div>
<div> <div>
<div class="flex gap-2 items-center"> <div class="flex gap-2 items-center">
<label class="text-sm text-gray-700">Delivery attempts:</label> <label class="text-sm text-gray-700">Delivery attempts:</label>
+8
View File
@@ -47,6 +47,14 @@
<input type="number" id="timeout" name="timeout" value="{{.TargetForm.Timeout}}" min="0" max="{{.MaxTimeout}}" class="input"> <input type="number" id="timeout" name="timeout" value="{{.TargetForm.Timeout}}" min="0" max="{{.MaxTimeout}}" class="input">
<p class="text-xs text-gray-500 mt-1">Per-request timeout, at most {{.MaxTimeout}} seconds. Leave blank to use the default.</p> <p class="text-xs text-gray-500 mt-1">Per-request timeout, at most {{.MaxTimeout}} seconds. Leave blank to use the default.</p>
</div> </div>
<div class="form-group">
<label class="flex items-center gap-2 text-sm font-medium text-gray-700">
<input type="checkbox" id="forward_query" name="forward_query" value="on"{{if .TargetForm.ForwardQuery}} checked{{end}} class="h-4 w-4">
Pass the query string on to this target
</label>
<p class="text-xs text-gray-500 mt-1">Appends the query string each event arrived with to the destination URL, after any query string the URL already has.</p>
</div>
{{end}} {{end}}
{{if eq .Target.Type "slack"}} {{if eq .Target.Type "slack"}}