Compare commits

2 Commits

Author SHA1 Message Date
03c8e4616c Render delivery attempt detail in the event log (closes #202)
All checks were successful
check / check (push) Successful in 3m35s
Expanding a delivery on the event log page now shows each recorded
attempt: attempt number, outcome, status code, duration, error and
response body. Previously a failure rendered as "target: failed" and
diagnosing it meant opening the per-webhook SQLite file by hand.

The response body is cut by SQLite via substr over a blob cast, the
same projection the event body uses, so an oversized stored response
never becomes a Go string. The page reports the cut with a marker.

Response bodies and errors are remote content, so both go through a
new delivery.Redactor that strips the target's own destination URL,
path, query and userinfo, plus the values of credential-shaped
request headers, before rendering. Target configuration keeps
reaching the template only as a TargetView.

A body that reaches the cap is treated as cut whether or not SQLite
is what cut it. The delivery engine stops reading a response at its
own cap, which is the same number of bytes this page renders, and
the row it writes records that cut length as the whole length, so
nothing in the row separates a response that ended at the cap from
one severed there. Such a body goes through RedactCut, which drops
any tail that is a proper prefix of a secret: the remote chooses the
padding in front of a credential it echoes, so it chooses where the
cut falls inside that credential. Its marker says the response
reached the recording limit rather than quoting a total the row does
not know.

Redactors are built from an unscoped target load. Deleting a target
only soft deletes the row while its deliveries survive, and a scoped
load would leave exactly those deliveries rendering unredacted. The
views the page lists stay scoped.

Attempt loading is chunked so the IN clause cannot exceed SQLite's
bound-parameter limit, and a chunk that fails fails the page rather
than rendering the deliveries it covered as never having run. The
page renders at most 20 attempts per delivery, counting what it
leaves out.

static/css/tailwind.css is regenerated with tailwindcss for the
utility classes the new markup uses.
2026-08-20 06:17:12 +00:00
3b0ed826bc Add per-delivery replay to the event log (closes #203) (#240)
All checks were successful
check / check (push) Successful in 3m21s
There was no redelivery path anywhere: once a delivery exhausted
max_retries it was failed permanently, even though the event body is
durably stored. Storing an event and being unable to re-send it defeats
the reason it is stored, and the ordinary case is a destination that was
down longer than the backoff ladder.

Adds POST /source/{sourceID}/deliveries/{deliveryID}/replay, inside the
authenticated group so it inherits MaxBodySize, CSRF, NoCache and
RequireAuth. Replay creates a NEW pending delivery against the target's
CURRENT config and hands it to the engine through the same notifier the
receiver uses, so it runs the normal path with the retry ladder, the
SSRF-guarded transport and the circuit breaker. The original delivery's
rows are never touched, and the stored event body is re-sent, never the
recorded response.

Replay is refused, with a distinct message, for a non-terminal delivery, a
deleted target, a deactivated target, and when an earlier replay of the
same event and target is still in flight. Bounded by a per-client rate
limit and by that in-flight check.

The new delivery row is written with Omit(clause.Associations) and with
neither Event nor Target populated, so it cannot upsert a targets row into
the per-webhook event database (#206).

Counted by webhooker_delivery_replays_total on the existing target_type
label. A replay also moves the ordinary attempt, outcome and duration
series, because it is a real delivery.
2026-08-20 08:11:35 +02:00
20 changed files with 2854 additions and 49 deletions

View File

@@ -1108,6 +1108,23 @@ DeliveryResults.
succeeded). succeeded).
- **`failed`** — All retry attempts exhausted without success. - **`failed`** — All retry attempts exhausted without success.
**Replay.** A `delivered` or `failed` delivery is finished as far as
the engine is concerned, but the event is still stored, so the event
log offers a per-delivery **Replay** action for it. Replay creates a
NEW `pending` delivery for the same event and target and hands it to
the engine on the ordinary path — same retries, same SSRF guard, same
circuit breaker as a first attempt. It never touches the delivery it
repeats: that row's status, timestamps and recorded attempts stand as
the record of what happened.
What is re-sent is the stored event body, against the target's
configuration **as it stands now** — the point of a replay is to
deliver where the destination has since been fixed. A target that has
been deleted or deactivated therefore refuses the replay with a
message on the event log rather than delivering from stale
configuration, and a replay is refused while an earlier one for the
same event and target is still pending or retrying.
#### DeliveryResult #### DeliveryResult
The result of a single delivery attempt. Every attempt (including The result of a single delivery attempt. Every attempt (including
@@ -1439,6 +1456,7 @@ arriving and being stored, they are just not getting anywhere.
| `webhooker_deliveries_succeeded_total` | counter | Deliveries that reached `delivered` | | `webhooker_deliveries_succeeded_total` | counter | Deliveries that reached `delivered` |
| `webhooker_deliveries_failed_total` | counter | Deliveries that failed terminally and will not be retried | | `webhooker_deliveries_failed_total` | counter | Deliveries that failed terminally and will not be retried |
| `webhooker_delivery_retries_total` | counter | Deliveries put back into `retrying` | | `webhooker_delivery_retries_total` | counter | Deliveries put back into `retrying` |
| `webhooker_delivery_replays_total` | counter | Deliveries an operator replayed from the event log. A replay runs the ordinary engine path, so it also moves the attempt, outcome and duration series; this is the only one that separates it from ordinary traffic |
| `webhooker_delivery_duration_seconds` | histogram | Wall time of a single dispatched delivery attempt, the same duration the attempt's `DeliveryResult` records | | `webhooker_delivery_duration_seconds` | histogram | Wall time of a single dispatched delivery attempt, the same duration the attempt's `DeliveryResult` records |
| `webhooker_deliveries_pending` | gauge | Deliveries currently in `pending` | | `webhooker_deliveries_pending` | gauge | Deliveries currently in `pending` |
| `webhooker_deliveries_retrying` | gauge | Deliveries currently in `retrying` | | `webhooker_deliveries_retrying` | gauge | Deliveries currently in `retrying` |
@@ -2036,6 +2054,7 @@ abuse limit later; they are tracked as future work.
| `POST` | `/source/{id}/edit` | Edit webhook submission | | `POST` | `/source/{id}/edit` | Edit webhook submission |
| `POST` | `/source/{id}/delete` | Delete webhook | | `POST` | `/source/{id}/delete` | Delete webhook |
| `GET` | `/source/{id}/logs` | Webhook event logs | | `GET` | `/source/{id}/logs` | Webhook event logs |
| `POST` | `/source/{id}/deliveries/{deliveryID}/replay` | Replay a finished delivery: creates a new delivery for the same event against the target's current configuration (30 per minute per bucket, then `429`) |
| `POST` | `/source/{id}/entrypoints` | Add entrypoint to webhook | | `POST` | `/source/{id}/entrypoints` | Add entrypoint to webhook |
| `POST` | `/source/{id}/entrypoints/{entrypointID}/delete` | Delete an entrypoint | | `POST` | `/source/{id}/entrypoints/{entrypointID}/delete` | Delete an entrypoint |
| `POST` | `/source/{id}/entrypoints/{entrypointID}/toggle` | Enable or disable an entrypoint | | `POST` | `/source/{id}/entrypoints/{entrypointID}/toggle` | Enable or disable an entrypoint |
@@ -2125,6 +2144,7 @@ webhooker/
│ ├── handlers/ │ ├── handlers/
│ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering │ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering
│ │ ├── auth.go # Login, logout handlers │ │ ├── auth.go # Login, logout handlers
│ │ ├── delivery_replay.go # Per-delivery replay: new delivery, current target config
│ │ ├── entrypoint_view.go # Masked entrypoint view for templates │ │ ├── entrypoint_view.go # Masked entrypoint view for templates
│ │ ├── event_log_view.go # Event log projection, byte-capped in SQL │ │ ├── event_log_view.go # Event log projection, byte-capped in SQL
│ │ ├── healthcheck.go # Health check handler │ │ ├── healthcheck.go # Health check handler

View File

@@ -11,6 +11,23 @@ const (
DeliveryStatusRetrying DeliveryStatus = "retrying" DeliveryStatusRetrying DeliveryStatus = "retrying"
) )
// Terminal reports whether a delivery in this status has finished, so
// the delivery engine will make no further attempt of its own.
//
// It is what decides which deliveries the event log offers to replay:
// a pending or retrying delivery is still the engine's, and replaying
// one would race it.
func (s DeliveryStatus) Terminal() bool {
switch s {
case DeliveryStatusDelivered, DeliveryStatusFailed:
return true
case DeliveryStatusPending, DeliveryStatusRetrying:
return false
default:
return false
}
}
// Delivery represents a delivery attempt for an event to a target // Delivery represents a delivery attempt for an event to a target
type Delivery struct { type Delivery struct {
BaseModel BaseModel

View File

@@ -1201,6 +1201,83 @@ func TestDeliverHTTP_TargetTimeout(t *testing.T) {
iAssertResultFailed(t, db, del.ID) iAssertResultFailed(t, db, del.ID)
} }
// TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog pins the size
// this engine stores for an oversized response, because the
// event log's redaction is written against it: the row holds
// exactly maxBodyLog bytes and records nothing about how much
// more the remote sent, so a credential echoed across that
// boundary reaches the database already severed and no reader
// of the row can tell the cut happened.
func TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog(
t *testing.T,
) {
t.Parallel()
// Padded so the cut falls five bytes before the end of the
// echoed webhook URL.
const (
severedTail = 5
overshoot = 100000
)
sent := strings.Repeat(
"A",
delivery.ExportMaxBodyLog-len(slackWebhookURL)+
severedTail,
) + slackWebhookURL + strings.Repeat("Z", overshoot)
s := newISetup(t)
ts := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusBadGateway)
_, _ = io.WriteString(w, sent)
},
))
defer ts.Close()
cfgJSON := iHTTPConfig(ts.URL)
event := iSeedEvent(
t, s.WebhookDB, s.WebhookID, `{"cut":"test"}`,
)
targetID := uuid.New().String()
del := iSeedDelivery(
t, s.WebhookDB, event.ID, targetID,
database.DeliveryStatusPending,
)
bodyStr := event.Body
task := iTask(
del, event, s.WebhookID, targetID,
"cut-target", cfgJSON, 0, 1, &bodyStr,
)
s.Engine.ExportProcessNewTask(context.TODO(), &task)
results := iResults(t, s.WebhookDB, del.ID)
require.Len(t, results, 1)
stored := results[0].ResponseBody
assert.Len(
t, stored, delivery.ExportMaxBodyLog,
"an oversized response is stored at exactly the cap",
)
assert.Equal(
t, sent[:delivery.ExportMaxBodyLog], stored,
)
assert.NotContains(
t, stored, slackWebhookURL,
"the echoed URL is severed by the cut",
)
assert.Contains(
t, stored, "T00000000",
"the severed prefix still carries the credential",
)
}
// iSeedEventAndDelivery creates event + delivery // iSeedEventAndDelivery creates event + delivery
// for standalone tests. // for standalone tests.
func iSeedEventAndDelivery( func iSeedEventAndDelivery(

View File

@@ -25,6 +25,12 @@ const (
ExportRetryChannelSize = retryChannelSize ExportRetryChannelSize = retryChannelSize
ExportDefaultFailureThreshold = defaultFailureThreshold ExportDefaultFailureThreshold = defaultFailureThreshold
ExportDefaultCooldown = defaultCooldown ExportDefaultCooldown = defaultCooldown
// ExportMaxBodyLog is the cap the engine applies to a
// recorded response body. The event log's handling of a cut
// response is written against this number, so a test has to
// be able to name it.
ExportMaxBodyLog = maxBodyLog
) )
// ExportIsBlockedIP exposes isBlockedIP for testing. // ExportIsBlockedIP exposes isBlockedIP for testing.

View File

@@ -0,0 +1,272 @@
package delivery
import (
"net/url"
"slices"
"strings"
"sneak.berlin/go/webhooker/internal/database"
)
// RedactionMarker stands in for a target credential found in
// text the target's remote peer chose.
const RedactionMarker = "(redacted)"
// Redactor removes one target's own credential material from
// text that target's remote peer chose: a delivery response
// body, or a delivery error stored before the delivery path
// learned to mask the URLs it embeds.
//
// It removes byte-identical echoes of strings taken from the
// target's stored configuration, and nothing else. Anything
// the remote re-encodes survives: JSON "\/" escaping (what
// PHP's json_encode emits by default), percent-encoding, HTML
// entities, and an echo of only part of a path. It cannot
// remove a secret the remote invented.
//
// The zero Redactor removes nothing, which is what a caller
// holding no target for a delivery gets.
type Redactor struct {
secrets []string
}
// NewRedactor builds the redactor for one target.
func NewRedactor(t *database.Target) Redactor {
// Drop empty strings here rather than at the site that
// produced one. strings.ReplaceAll with an empty old string
// inserts the marker at every byte boundary, so a single
// empty secret destroys every body and error the target
// renders; filtering at the collection point means no field
// added to targetSecrets later can reintroduce that.
// url.Parse("https://@example.com/in") is the known
// producer: a non-nil User whose String is "".
secrets := slices.DeleteFunc(
targetSecrets(t),
func(s string) bool { return s == "" },
)
// Longest first, so replacing a secret that is contained
// in a longer one cannot leave a fragment of the longer
// one behind. Configured headers arrive in map order, so
// the sort is also what makes the result deterministic.
slices.SortFunc(secrets, func(a, b string) int {
if d := len(b) - len(a); d != 0 {
return d
}
return strings.Compare(a, b)
})
return Redactor{secrets: secrets}
}
// Redact replaces every occurrence of the target's credential
// material in s.
func (r Redactor) Redact(s string) string {
if s == "" {
return s
}
for _, secret := range r.secrets {
s = strings.ReplaceAll(s, secret, RedactionMarker)
}
return s
}
// RedactCut redacts s, which its caller has already cut to a
// byte budget, and additionally drops any tail of s that is a
// proper prefix of a secret.
//
// The cut lands wherever the remote's padding puts it, so the
// remote chooses where inside the credential it falls. The
// severed prefix left behind equals no secret, so plain
// Redact would render it verbatim.
func (r Redactor) RedactCut(s string) string {
s = r.Redact(s)
if n := r.secretPrefixSuffix(s); n > 0 {
return s[:len(s)-n] + RedactionMarker
}
return s
}
// secretPrefixSuffix returns the length of the longest suffix
// of s that is a proper prefix of one of the secrets, or 0
// when there is none.
func (r Redactor) secretPrefixSuffix(s string) int {
longest := 0
for _, secret := range r.secrets {
// Proper prefixes only: a whole secret at the tail was
// already replaced by Redact.
n := min(len(secret)-1, len(s))
for ; n > longest; n-- {
if strings.HasSuffix(s, secret[:n]) {
longest = n
break
}
}
}
return longest
}
// targetSecrets returns the credential-bearing strings a
// target's configuration carries.
//
// The destination URL contributes. Its path, query and
// userinfo are the credential for both target types that have
// one — an incoming-webhook URL is a bearer token, which is
// why MaskURL elides exactly those parts — and they are the
// material this service actually sends, so a remote that
// echoes the request back echoes them.
//
// Configured request headers contribute their values, but
// only for the credential-shaped names isCredentialHeaderName
// picks out. That is the same class-based rule applied to
// URLs: an echoed Accept or User-Agent still renders, an
// echoed Authorization does not.
func targetSecrets(t *database.Target) []string {
if t == nil {
return nil
}
switch t.Type {
case database.TargetTypeSlack:
cfg, err := parseSlackConfig(t.Config)
if err != nil {
return nil
}
return urlSecrets(cfg.WebhookURL)
case database.TargetTypeHTTP:
cfg, err := parseHTTPConfig(t.Config)
if err != nil {
return nil
}
return append(
urlSecrets(cfg.URL),
headerSecrets(cfg.Headers)...,
)
case database.TargetTypeDatabase, database.TargetTypeLog:
// Neither has a destination URL, so neither has
// anything to redact.
return nil
default:
return nil
}
}
// urlSecrets returns the substrings of a destination URL that
// must not survive into a rendered page: the whole URL, the
// parts of it MaskURL elides, and any userinfo.
//
// No length floor is applied to the path, and none to the
// userinfo. A short path or a four-byte username is treated as
// a credential exactly like a long one, because the field takes
// an arbitrary URL and no part of it can be assumed non-secret —
// the same rule MaskURL applies. headerSecrets does carry a
// floor, and the difference is deliberate: a header is picked
// out by a name-shaped guess and its value may be ordinary
// text, whereas a URL's path and userinfo are credential
// material by position.
func urlSecrets(raw string) []string {
raw = strings.TrimSpace(raw)
if raw == "" {
return nil
}
secrets := []string{raw}
parsed, err := url.Parse(raw)
if err != nil {
return secrets
}
if parsed.Path != "" && parsed.Path != "/" {
requestURI := parsed.RequestURI()
secrets = append(secrets, requestURI)
if escaped := parsed.EscapedPath(); escaped != requestURI {
secrets = append(secrets, escaped)
}
}
if parsed.User != nil {
secrets = append(secrets, parsed.User.String())
if pw, ok := parsed.User.Password(); ok && pw != "" {
secrets = append(secrets, pw)
}
}
return secrets
}
// minHeaderSecretBytes is the shortest header value treated as
// a credential. Unlike a URL path, a header value can be a
// couple of bytes long, and redacting those would scatter the
// marker through ordinary response text for no gain.
const minHeaderSecretBytes = 4
// headerSecrets returns the values of the configured headers
// whose names are credential-shaped.
func headerSecrets(headers map[string]string) []string {
var secrets []string
for name, value := range headers {
value = strings.TrimSpace(value)
if len(value) < minHeaderSecretBytes {
continue
}
if isCredentialHeaderName(name) {
secrets = append(secrets, value)
}
}
return secrets
}
// isCredentialHeaderName classifies a header by its name. The
// value is never inspected, so the rule is the same
// class-based one MaskURL applies to a destination URL.
//
// The fragments are short on purpose, and match anywhere in
// the name, so abbreviations an operator might use are covered
// too: X-Sig, X-Pass, X-HMAC. That over-matches — a header
// named X-Design contains "sig" — and over-matching is the
// safe direction here: the cost is a marker where an echoed
// header value would have rendered.
func isCredentialHeaderName(name string) bool {
name = strings.ToLower(strings.TrimSpace(name))
// Names that carry a credential by definition.
switch name {
case "authorization", "proxy-authorization", "cookie":
return true
}
// What operators call their own credential headers:
// X-Api-Key, X-Hub-Signature, X-Auth-Token.
for _, fragment := range []string{
"auth",
"credential",
"hmac",
"key",
"pass",
"secret",
"sig",
"token",
} {
if strings.Contains(name, fragment) {
return true
}
}
return false
}

View File

@@ -0,0 +1,284 @@
package delivery_test
import (
"net/url"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
)
// The secret path segments of a Slack incoming webhook URL.
const (
redactSecretPath = "/services/T11111111/B11111111/" +
"YYYYYYYYYYYYYYYYYYYYYYYY"
redactWebhookURL = "https://hooks.slack.com" +
redactSecretPath
)
func TestRedactor_RemovesSlackWebhookURL(t *testing.T) {
t.Parallel()
r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeSlack,
Config: `{"webhookUrl":"` + redactWebhookURL + `"}`,
})
got := r.Redact("no_service for " + redactWebhookURL)
assert.NotContains(t, got, redactSecretPath)
assert.NotContains(t, got, "T11111111")
// One marker, not a marker with the host left in front of
// it: the whole URL is replaced before the path it
// contains, which is what sorting the secrets longest
// first buys.
assert.Equal(
t,
"no_service for "+delivery.RedactionMarker,
got,
)
}
// TestRedactor_RemovesSecretSeveredByACut covers the input the
// redactor exists for: text cut to a byte budget with the
// credential straddling the cut. The remote chooses the
// padding, so it chooses where the cut lands inside the
// credential, and the severed prefix that remains equals no
// secret.
func TestRedactor_RemovesSecretSeveredByACut(t *testing.T) {
t.Parallel()
r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeSlack,
Config: `{"webhookUrl":"` + redactWebhookURL + `"}`,
})
// Every cut position inside the credential, not just a
// convenient one.
for n := 1; n < len(redactWebhookURL); n++ {
severed := redactWebhookURL[:n]
cut := "padding " + severed
got := r.RedactCut(cut)
assert.Equal(
t,
"padding "+delivery.RedactionMarker,
got,
"cut after %d bytes of the credential", n,
)
}
}
// TestRedactor_RedactsCredentialShapedHeaderValues pins the
// class-based header rule: a header whose name says credential
// has its value redacted, and a routine header does not, so
// ordinary response content survives.
func TestRedactor_RedactsCredentialShapedHeaderValues(
t *testing.T,
) {
t.Parallel()
r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeHTTP,
Config: `{"url":"https://example.com/in",` +
`"headers":{` +
`"Authorization":"Bearer AAAAAAAAAAAA",` +
`"Cookie":"session=BBBBBBBBBBBB",` +
`"X-Api-Key":"CCCCCCCCCCCC",` +
`"X-Hub-Signature":"sha256=DDDDDDDDDDDD",` +
`"X-Sig":"EEEEEEEEEEEE",` +
`"X-Pass":"FFFFFFFFFFFF",` +
`"X-HMAC":"GGGGGGGGGGGG",` +
`"X-Credential":"HHHHHHHHHHHH",` +
`"Accept":"application/json",` +
`"User-Agent":"webhooker/1.0"}}`,
})
for _, secret := range []string{
"Bearer AAAAAAAAAAAA",
"session=BBBBBBBBBBBB",
"CCCCCCCCCCCC",
"sha256=DDDDDDDDDDDD",
// Abbreviated names an operator might use.
"EEEEEEEEEEEE",
"FFFFFFFFFFFF",
"GGGGGGGGGGGG",
"HHHHHHHHHHHH",
} {
got := r.Redact("echo: " + secret)
assert.Equal(
t,
"echo: "+delivery.RedactionMarker,
got,
secret,
)
}
const routine = "Accept: application/json, " +
"User-Agent: webhooker/1.0"
assert.Equal(t, routine, r.Redact(routine))
}
// TestRedactor_IgnoresVeryShortHeaderValues pins the floor
// under a header value. Redacting a two-byte value would put
// the marker through every response that happens to contain
// those bytes.
func TestRedactor_IgnoresVeryShortHeaderValues(t *testing.T) {
t.Parallel()
r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeHTTP,
Config: `{"url":"https://example.com/in",` +
`"headers":{"X-Api-Key":"ab"}}`,
})
const response = "rabbit"
assert.Equal(t, response, r.Redact(response))
}
// TestRedactor_RemovesBarePath covers a remote that echoes
// only the request path rather than the whole URL. The path
// segments are the credential on their own.
func TestRedactor_RemovesBarePath(t *testing.T) {
t.Parallel()
r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeSlack,
Config: `{"webhookUrl":"` + redactWebhookURL + `"}`,
})
got := r.Redact("POST " + redactSecretPath + " 404")
assert.NotContains(t, got, redactSecretPath)
assert.Equal(
t,
"POST "+delivery.RedactionMarker+" 404",
got,
)
}
// TestRedactor_RemovesHTTPURLQueryAndUserinfo covers the HTTP
// target, whose destination is an arbitrary URL: the query
// string and the userinfo carry credentials as readily as the
// path does.
func TestRedactor_RemovesHTTPURLQueryAndUserinfo(t *testing.T) {
t.Parallel()
// Assembled rather than written out, so the literal is
// not itself a credential-shaped string.
dest := url.URL{
Scheme: "https",
User: url.UserPassword("user", "hunter2"),
Host: "example.com",
Path: "/in",
RawQuery: "token=s3cr3t",
}
raw := dest.String()
r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeHTTP,
Config: `{"url":"` + raw + `"}`,
})
for _, echoed := range []string{
raw,
"/in?token=s3cr3t",
"hunter2",
} {
got := r.Redact("rejected: " + echoed)
assert.NotContains(t, got, "s3cr3t", echoed)
assert.NotContains(t, got, "hunter2", echoed)
assert.Contains(
t, got, delivery.RedactionMarker, echoed,
)
}
}
// TestRedactor_LeavesUnrelatedTextAlone pins that the
// redactor matches literally: it does not guess at what a
// secret looks like, so ordinary response content survives.
func TestRedactor_LeavesUnrelatedTextAlone(t *testing.T) {
t.Parallel()
const response = "ok=false error=channel_not_found"
r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeSlack,
Config: `{"webhookUrl":"` + redactWebhookURL + `"}`,
})
assert.Equal(t, response, r.Redact(response))
}
// TestRedactor_EmptyUserinfoDoesNotShredTheBody covers a
// destination URL written with a bare "@" and no userinfo:
// url.Parse returns a non-nil User whose String is empty. An
// empty secret in the list would make strings.ReplaceAll
// insert the marker at every byte boundary, destroying every
// body and error string the target renders.
func TestRedactor_EmptyUserinfoDoesNotShredTheBody(t *testing.T) {
t.Parallel()
const dest = "https://@example.com/in"
// The premise: this URL really does parse to a non-nil
// User contributing an empty string.
parsed, err := url.Parse(dest)
require.NoError(t, err)
require.NotNil(t, parsed.User)
require.Empty(t, parsed.User.String())
r := delivery.NewRedactor(&database.Target{
Type: database.TargetTypeHTTP,
Config: `{"url":"` + dest + `"}`,
})
const body = "ok=false error=channel_not_found"
assert.Equal(t, body, r.Redact(body))
assert.Equal(t, body, r.RedactCut(body))
// The real credential material still goes, so filtering the
// empty string out did not disarm the redactor.
assert.Equal(
t,
"POST "+delivery.RedactionMarker,
r.Redact("POST "+dest),
)
}
// TestRedactor_ZeroValueAndConfiglessTargets pins that a
// caller with no target, an unparseable config, or a target
// type with no destination URL gets a redactor that changes
// nothing rather than one that panics.
func TestRedactor_ZeroValueAndConfiglessTargets(t *testing.T) {
t.Parallel()
const text = "some response body"
var zero delivery.Redactor
assert.Equal(t, text, zero.Redact(text))
assert.Equal(t, text, delivery.NewRedactor(nil).Redact(text))
for _, tgt := range []database.Target{
{Type: database.TargetTypeLog},
{Type: database.TargetTypeDatabase},
{Type: database.TargetTypeSlack, Config: "not json"},
{Type: database.TargetTypeHTTP, Config: ""},
} {
assert.Equal(
t, text,
delivery.NewRedactor(&tgt).Redact(text),
tgt.Type,
)
}
}

View File

@@ -0,0 +1,378 @@
package handlers
import (
"net/http"
"strconv"
"github.com/go-chi/chi"
"gorm.io/gorm"
"gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
)
// replayOutcomeParam is the query parameter the replay POST redirects
// with and the event log page reads its banner from.
const replayOutcomeParam = "replay"
// replayOutcomeCode is the outcome of a replay POST. The redirect
// carries one of these fixed codes rather than a message, so nothing a
// client submits can reach the rendered page through it.
type replayOutcomeCode string
const (
// replayQueued reports that a new delivery was created and handed
// to the delivery engine.
replayQueued replayOutcomeCode = "queued"
// replayTargetDeleted reports a target that once existed and has
// since been deleted. Deletes are soft and deliveries carry no
// foreign key to the target row, so the history survives its
// target and this is the ordinary case for an old event.
replayTargetDeleted replayOutcomeCode = "target-deleted"
// replayTargetMissing reports a target id that names no row at
// all, deleted or otherwise.
replayTargetMissing replayOutcomeCode = "target-missing"
// replayTargetInactive reports a target the operator has
// deactivated. A deactivated target receives no new deliveries, so
// a replay to it would be a delivery they switched off.
replayTargetInactive replayOutcomeCode = "target-inactive"
// replayNotTerminal reports a delivery the engine has not finished
// with.
replayNotTerminal replayOutcomeCode = "not-terminal"
// replayInFlight reports that an earlier replay of this event to
// this target is still running.
replayInFlight replayOutcomeCode = "in-flight"
)
// replayOutcome returns the banner the event log page shows for an
// outcome code, and whether the replay was queued. An unrecognised
// code yields no banner.
func replayOutcome(code string) (string, bool) {
switch replayOutcomeCode(code) {
case replayQueued:
return "Replay queued: a new delivery was created against " +
"the target's current configuration.", true
case replayTargetDeleted:
return "Not replayed: the target this delivery was for has " +
"been deleted. Recreate the target, then replay.", false
case replayTargetMissing:
return "Not replayed: the target this delivery was for no " +
"longer exists.", false
case replayTargetInactive:
return "Not replayed: the target this delivery was for is " +
"deactivated. Activate it, then replay.", false
case replayNotTerminal:
return "Not replayed: this delivery has not finished yet.",
false
case replayInFlight:
return "Not replayed: a delivery of this event to this " +
"target is already in flight.", false
default:
return "", false
}
}
// HandleDeliveryReplay re-sends a finished delivery's event to its
// target.
//
// A replay never touches the delivery it repeats. It creates a NEW
// pending delivery row for the same event and target and hands it to
// the delivery engine through the same Notifier the receiver uses, so
// the original's status, attempts and timestamps stand as the record
// of what actually happened, and the replay is retried, SSRF-guarded
// and circuit-broken exactly as a first attempt is.
//
// What is re-sent is the stored EVENT body, never the response the
// original delivery received.
//
// The target's configuration is read now rather than as it stood when
// the original ran: a replay exists to deliver where the operator
// currently wants the event to go. That is also why a deleted target
// is refused rather than delivered to from stale configuration.
func (h *Handlers) HandleDeliveryReplay() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
webhook, ok := h.ownedWebhook(w, r)
if !ok {
return
}
// The body size cap is enforced by the MaxBodySize
// middleware, which runs before CSRF parses the form.
err := r.ParseForm()
if err != nil {
http.Error(
w, "Bad request", http.StatusBadRequest,
)
return
}
h.replayDelivery(w, r, webhook)
}
}
// replayDelivery performs the replay for a webhook the caller has
// already established the session's user owns.
func (h *Handlers) replayDelivery(
w http.ResponseWriter,
r *http.Request,
webhook database.Webhook,
) {
if !h.dbMgr.DBExists(webhook.ID) {
http.NotFound(w, r)
return
}
webhookDB, err := h.dbMgr.GetDB(webhook.ID)
if err != nil {
h.serverError(w, "failed to get webhook database", err)
return
}
original, ok := h.loadReplaySource(w, r, webhookDB)
if !ok {
return
}
if !original.Status.Terminal() {
h.finishReplay(w, r, webhook, replayNotTerminal)
return
}
target, code := h.replayTarget(webhook.ID, original.TargetID)
if target == nil {
h.finishReplay(w, r, webhook, code)
return
}
h.queueReplay(w, r, webhookDB, webhook, original, target)
}
// loadReplaySource loads the delivery to be replayed, selecting only
// the columns the replay needs so no association is populated. A
// delivery id that names no row in this webhook's database is a 404.
func (h *Handlers) loadReplaySource(
w http.ResponseWriter,
r *http.Request,
webhookDB *gorm.DB,
) (*database.Delivery, bool) {
var original database.Delivery
err := webhookDB.
Select("id", "event_id", "target_id", "status").
First(
&original, "id = ?", chi.URLParam(r, "deliveryID"),
).Error
if err != nil {
http.NotFound(w, r)
return nil, false
}
return &original, true
}
// queueReplay writes the new delivery and hands it to the engine.
func (h *Handlers) queueReplay(
w http.ResponseWriter,
r *http.Request,
webhookDB *gorm.DB,
webhook database.Webhook,
original *database.Delivery,
target *database.Target,
) {
inFlight, err := countInFlightDeliveries(
webhookDB, original.EventID, target.ID,
)
if err != nil {
h.serverError(
w, "failed to count in-flight deliveries", err,
)
return
}
if inFlight > 0 {
h.finishReplay(w, r, webhook, replayInFlight)
return
}
var event database.Event
err = webhookDB.
First(&event, "id = ?", original.EventID).Error
if err != nil {
h.serverError(w, "failed to load event for replay", err)
return
}
task, err := createReplayDelivery(
webhookDB, webhook.ID, &event, target,
)
if err != nil {
h.serverError(
w, "failed to create replay delivery", err,
)
return
}
h.mtr.DeliveryReplayed(target.Type)
h.notifier.Notify([]delivery.Task{task})
h.log.Info(
"delivery replay queued",
"webhook_id", webhook.ID,
"event_id", event.ID,
"target_id", target.ID,
"replayed_delivery_id", original.ID,
"delivery_id", task.DeliveryID,
)
h.finishReplay(w, r, webhook, replayQueued)
}
// replayTarget loads the delivery's target as it stands now.
//
// The load is Unscoped so that a soft-deleted row is still found:
// deletes are soft and a delivery carries no foreign key to its
// target, so a target's history outlives it, and without the deleted
// row there is no way to tell "you deleted this target" from "this id
// never named anything". A nil target means the replay is refused,
// with the returned code saying why.
func (h *Handlers) replayTarget(
webhookID, targetID string,
) (*database.Target, replayOutcomeCode) {
var target database.Target
err := h.db.DB().Unscoped().Where(
"id = ? AND webhook_id = ?", targetID, webhookID,
).First(&target).Error
if err != nil {
return nil, replayTargetMissing
}
if target.DeletedAt.Valid {
return nil, replayTargetDeleted
}
if !target.Active {
return nil, replayTargetInactive
}
return &target, replayQueued
}
// countInFlightDeliveries reports how many deliveries of this event to
// this target the engine has not finished.
//
// It is the replay-storm guard: a replay is refused while an earlier
// one is still pending or retrying, so a held-down button or a scripted
// loop cannot stack copies of work already queued. It is a check and
// not a lock, so two simultaneous POSTs can still both pass it; the
// per-client rate limit on the route is what bounds that.
func countInFlightDeliveries(
webhookDB *gorm.DB, eventID, targetID string,
) (int64, error) {
var count int64
err := webhookDB.Model(&database.Delivery{}).Where(
"event_id = ? AND target_id = ? AND status IN ?",
eventID, targetID,
[]database.DeliveryStatus{
database.DeliveryStatusPending,
database.DeliveryStatusRetrying,
},
).Count(&count).Error
return count, err
}
// createReplayDelivery writes the new pending delivery row and returns
// the task that carries it to the delivery engine.
//
// The row is written with associations omitted, and neither Event nor
// Target is populated on it: GORM's SaveBeforeAssociations would
// otherwise upsert the whole target row — plaintext config, which for a
// Slack target is the credential — into the per-webhook event database.
// See https://git.eeqj.de/sneak/webhooker/issues/206.
func createReplayDelivery(
webhookDB *gorm.DB,
webhookID string,
event *database.Event,
target *database.Target,
) (delivery.Task, error) {
dlv := &database.Delivery{
EventID: event.ID,
TargetID: target.ID,
Status: database.DeliveryStatusPending,
}
err := webhookDB.Omit(clause.Associations).Create(dlv).Error
if err != nil {
return delivery.Task{}, err
}
return delivery.Task{
DeliveryID: dlv.ID,
EventID: event.ID,
WebhookID: webhookID,
EntrypointID: event.EntrypointID,
TargetID: target.ID,
TargetName: target.Name,
TargetType: target.Type,
TargetConfig: target.Config,
MaxRetries: target.MaxRetries,
Method: event.Method,
Headers: event.Headers,
ContentType: event.ContentType,
Body: replayBody(event.Body),
AttemptNum: 1,
}, nil
}
// replayBody returns the stored event body for a replay task to carry
// inline, or nil when it is large enough that the engine should fetch
// it from the per-webhook database instead.
func replayBody(body string) *string {
if len(body) >= delivery.MaxInlineBodySize {
return nil
}
return &body
}
// finishReplay redirects back to the event log the replay was
// triggered from, carrying the outcome code the page turns into a
// banner and the page number the form submitted.
func (h *Handlers) finishReplay(
w http.ResponseWriter,
r *http.Request,
webhook database.Webhook,
code replayOutcomeCode,
) {
dest := "/source/" + webhook.ID + "/logs?" +
replayOutcomeParam + "=" + string(code)
// The page is read from the form rather than the query string:
// this is a POST, and its query string is what logs and Referer
// headers record.
if page := parseNonNegativeInt(
r.PostFormValue("page"),
); page > 1 {
dest += "&page=" + strconv.Itoa(page)
}
http.Redirect(w, r, dest, http.StatusSeeOther)
}

View File

@@ -0,0 +1,526 @@
package handlers_test
import (
"net/http"
"net/http/httptest"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm"
"gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/session"
)
// paramDeliveryID is the chi URL parameter name the replay handler
// reads.
const paramDeliveryID = "deliveryID"
// replayTargetURL is a public destination, so a target configured with
// it is one the SSRF guard would accept. Nothing in these tests
// dispatches to it: the notifier is recorded, not run.
const replayTargetURL = "http://93.184.216.34/hook"
// seedFailedDelivery records an event, a terminally failed delivery of
// it to the given target, and the attempt that failed.
func seedFailedDelivery(
t *testing.T,
dbMgr *database.WebhookDBManager,
webhookID, targetID string,
) (*database.Event, *database.Delivery) {
t.Helper()
webhookDB, err := dbMgr.GetDB(webhookID)
require.NoError(t, err)
event := &database.Event{
WebhookID: webhookID,
EntrypointID: "entrypoint-" + webhookID,
Method: http.MethodPost,
Headers: `{"X-Test":["yes"]}`,
Body: `{"replay":"me"}`,
ContentType: contentTypeJSON,
}
require.NoError(t, webhookDB.Omit(
clause.Associations,
).Create(event).Error)
dlv := &database.Delivery{
EventID: event.ID,
TargetID: targetID,
Status: database.DeliveryStatusFailed,
}
require.NoError(t, webhookDB.Omit(
clause.Associations,
).Create(dlv).Error)
result := &database.DeliveryResult{
DeliveryID: dlv.ID,
AttemptNum: 1,
Success: false,
StatusCode: http.StatusBadGateway,
Error: "connection refused",
}
require.NoError(t, webhookDB.Omit(
clause.Associations,
).Create(result).Error)
return event, dlv
}
// loadDelivery reads a delivery back out of a webhook's database.
func loadDelivery(
t *testing.T, webhookDB *gorm.DB, deliveryID string,
) database.Delivery {
t.Helper()
var dlv database.Delivery
require.NoError(
t,
webhookDB.First(&dlv, "id = ?", deliveryID).Error,
)
return dlv
}
// listDeliveries reads every delivery of an event.
func listDeliveries(
t *testing.T, webhookDB *gorm.DB, eventID string,
) []database.Delivery {
t.Helper()
var deliveries []database.Delivery
require.NoError(t, webhookDB.Where(
"event_id = ?", eventID,
).Find(&deliveries).Error)
return deliveries
}
// theOtherDelivery returns the one delivery in the slice that is not
// excludeID. Identity is used rather than an ordering because the rows
// are minted milliseconds apart and their ids are random.
func theOtherDelivery(
t *testing.T,
deliveries []database.Delivery,
excludeID string,
) database.Delivery {
t.Helper()
var found []database.Delivery
for _, d := range deliveries {
if d.ID != excludeID {
found = append(found, d)
}
}
require.Len(t, found, 1)
return found[0]
}
// postReplay runs the real replay handler for one delivery.
func postReplay(
t *testing.T,
h *handlers.Handlers,
sess *session.Session,
webhookID, deliveryID string,
) *httptest.ResponseRecorder {
t.Helper()
req := postRequest(
"/source/"+webhookID+"/deliveries/"+
deliveryID+"/replay",
authenticatedCookies(
t, sess, deleteTestUserID, deleteTestUsername,
),
map[string]string{
paramSourceID: webhookID,
paramDeliveryID: deliveryID,
},
)
w := httptest.NewRecorder()
h.HandleDeliveryReplay().ServeHTTP(w, req)
return w
}
// TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal is the
// core requirement: replaying a failed delivery succeeds, appends a
// new delivery, and leaves the original row and its recorded attempt
// exactly as they were.
//
// It also pins the two things a replay would be wrong to get from the
// original: the task carries the target's CURRENT configuration, which
// this test changes between the failure and the replay, and it carries
// the stored EVENT body rather than anything the failed attempt
// received back.
func TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal(
t *testing.T,
) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
notif *recordingNotifier
)
app := newTestApp(t, &h, &sess, &db, &dbMgr, &notif)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+replayTargetURL+`"}`,
)
event, original := seedFailedDelivery(
t, dbMgr, wh.ID, tgt.ID,
)
webhookDB, err := dbMgr.GetDB(wh.ID)
require.NoError(t, err)
before := loadDelivery(t, webhookDB, original.ID)
// The operator fixes the destination, which is the whole reason
// to replay. The replay must use this, not the config the
// original delivery ran against.
const fixedConfig = `{"url":"http://93.184.216.34/fixed"}`
require.NoError(t, db.DB().Model(&database.Target{}).
Where("id = ?", tgt.ID).
Update("config", fixedConfig).Error)
w := postReplay(t, h, sess, wh.ID, original.ID)
require.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal(
t,
"/source/"+wh.ID+"/logs?replay=queued",
w.Header().Get("Location"),
)
deliveries := listDeliveries(t, webhookDB, event.ID)
require.Len(
t, deliveries, 2,
"replay must append a delivery, not reuse one",
)
replayed := theOtherDelivery(t, deliveries, original.ID)
assert.Equal(t, tgt.ID, replayed.TargetID)
assert.Equal(t, event.ID, replayed.EventID)
assert.Equal(
t, database.DeliveryStatusPending, replayed.Status,
)
assertDeliveryUntouched(t, webhookDB, before)
tasks := notif.Tasks()
require.Len(t, tasks, 1)
assertReplayTask(
t, tasks[0], wh.ID, event, tgt, replayed.ID, fixedConfig,
)
assertNoLeakedTarget(t, webhookDB)
}
// assertDeliveryUntouched proves a delivery row is exactly as it was
// read before: same terminal status, same timestamps, and the same
// recorded attempts.
func assertDeliveryUntouched(
t *testing.T,
webhookDB *gorm.DB,
before database.Delivery,
) {
t.Helper()
after := loadDelivery(t, webhookDB, before.ID)
assert.Equal(
t, before.Status, after.Status,
"replay must not resurrect the original delivery",
)
assert.Equal(t, before.UpdatedAt, after.UpdatedAt)
assert.Equal(t, before.CreatedAt, after.CreatedAt)
var attempts int64
require.NoError(t, webhookDB.
Model(&database.DeliveryResult{}).
Where("delivery_id = ?", before.ID).
Count(&attempts).Error)
assert.Equal(
t, int64(1), attempts,
"the original delivery's attempt history must stand",
)
}
// assertReplayTask proves the task handed to the delivery engine is
// the one the receiver would build for this event and this target, and
// that it carries wantConfig — the target's configuration as it stands
// now rather than as the original delivery ran against it.
func assertReplayTask(
t *testing.T,
task delivery.Task,
webhookID string,
event *database.Event,
target *database.Target,
wantDeliveryID, wantConfig string,
) {
t.Helper()
assert.Equal(t, wantDeliveryID, task.DeliveryID)
assert.Equal(t, event.ID, task.EventID)
assert.Equal(t, webhookID, task.WebhookID)
assert.Equal(t, event.EntrypointID, task.EntrypointID)
assert.Equal(t, target.ID, task.TargetID)
assert.Equal(t, target.Type, task.TargetType)
assert.JSONEq(
t, wantConfig, task.TargetConfig,
"replay must use the target's current configuration",
)
assert.Equal(t, event.Method, task.Method)
assert.Equal(t, event.Headers, task.Headers)
assert.Equal(t, event.ContentType, task.ContentType)
assert.Equal(t, 1, task.AttemptNum)
require.NotNil(t, task.Body)
assert.Equal(
t, event.Body, *task.Body,
"replay re-sends the stored event body",
)
}
// assertNoLeakedTarget proves the per-webhook database holds no target
// rows. AutoMigrate creates the table there because Delivery declares
// the relation, so it is a ROW that signals a leak: an association
// write would have upserted the whole target, plaintext config and
// all, into the event database. See
// https://git.eeqj.de/sneak/webhooker/issues/206.
func assertNoLeakedTarget(t *testing.T, webhookDB *gorm.DB) {
t.Helper()
var leaked int64
require.NoError(t, webhookDB.Unscoped().
Model(&database.Target{}).Count(&leaked).Error)
assert.Zero(
t, leaked,
"replay must not write the target into the event database",
)
}
// TestHandleDeliveryReplay_RefusesDeletedTarget proves the required
// refusal: a target deleted since the delivery ran is reported as
// deleted rather than erroring, and nothing is created or queued.
func TestHandleDeliveryReplay_RefusesDeletedTarget(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
notif *recordingNotifier
)
app := newTestApp(t, &h, &sess, &db, &dbMgr, &notif)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+replayTargetURL+`"}`,
)
event, original := seedFailedDelivery(
t, dbMgr, wh.ID, tgt.ID,
)
// Deletes are soft, so the delivery history outlives the target.
require.NoError(t, db.DB().Where(
"id = ?", tgt.ID,
).Delete(&database.Target{}).Error)
w := postReplay(t, h, sess, wh.ID, original.ID)
require.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal(
t,
"/source/"+wh.ID+"/logs?replay=target-deleted",
w.Header().Get("Location"),
)
webhookDB, err := dbMgr.GetDB(wh.ID)
require.NoError(t, err)
assert.Len(
t, listDeliveries(t, webhookDB, event.ID), 1,
"a refused replay must create no delivery",
)
assert.Empty(
t, notif.Tasks(),
"a refused replay must queue nothing",
)
// The refusal is specific, which is why the target is looked up
// including soft-deleted rows: an id that never named a target
// is a different outcome, and a different message, from one the
// operator deleted.
_, orphan := seedFailedDelivery(
t, dbMgr, wh.ID, "target-that-never-existed",
)
missing := postReplay(t, h, sess, wh.ID, orphan.ID)
require.Equal(t, http.StatusSeeOther, missing.Code)
assert.Equal(
t,
"/source/"+wh.ID+"/logs?replay=target-missing",
missing.Header().Get("Location"),
)
}
// TestHandleDeliveryReplay_RefusesWhileEarlierReplayInFlight proves
// the replay-storm guard: a second replay of the same event to the
// same target is refused while the first is still queued, so repeated
// submissions cannot stack copies of work the engine has not done.
func TestHandleDeliveryReplay_RefusesWhileEarlierReplayInFlight(
t *testing.T,
) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
notif *recordingNotifier
)
app := newTestApp(t, &h, &sess, &db, &dbMgr, &notif)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+replayTargetURL+`"}`,
)
event, original := seedFailedDelivery(
t, dbMgr, wh.ID, tgt.ID,
)
first := postReplay(t, h, sess, wh.ID, original.ID)
require.Equal(t, http.StatusSeeOther, first.Code)
require.Equal(
t,
"/source/"+wh.ID+"/logs?replay=queued",
first.Header().Get("Location"),
)
second := postReplay(t, h, sess, wh.ID, original.ID)
require.Equal(t, http.StatusSeeOther, second.Code)
assert.Equal(
t,
"/source/"+wh.ID+"/logs?replay=in-flight",
second.Header().Get("Location"),
)
webhookDB, err := dbMgr.GetDB(wh.ID)
require.NoError(t, err)
assert.Len(
t, listDeliveries(t, webhookDB, event.ID), 2,
"the refused second replay must add nothing",
)
assert.Len(
t, notif.Tasks(), 1,
"only the first replay reaches the delivery engine",
)
// A delivery the engine has not finished is not replayable
// either, which is the same rule seen from the other side.
queued := theOtherDelivery(
t, listDeliveries(t, webhookDB, event.ID), original.ID,
)
pending := postReplay(t, h, sess, wh.ID, queued.ID)
require.Equal(t, http.StatusSeeOther, pending.Code)
assert.Equal(
t,
"/source/"+wh.ID+"/logs?replay=not-terminal",
pending.Header().Get("Location"),
)
}
// TestHandleSourceLogs_RendersReplayControlAndBanner proves the action
// reaches the page it belongs on: a finished delivery renders a POST
// form carrying a CSRF token, and the outcome code a refusal redirects
// with becomes a readable message.
func TestHandleSourceLogs_RendersReplayControlAndBanner(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeHTTP,
`{"url":"`+replayTargetURL+`"}`,
)
_, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
body := renderSourceLogsPage(t, h, sess, wh.ID)
assert.Contains(
t, body,
`action="/source/`+wh.ID+`/deliveries/`+
original.ID+`/replay"`,
)
assert.Contains(t, body, `method="POST"`)
assert.Contains(t, body, `name="csrf_token"`)
assert.Contains(t, body, ">Replay<")
refused := renderSourceLogsPageWithQuery(
t, h, sess, wh.ID, "?replay=target-deleted",
)
assert.Contains(t, refused, "alert-error")
assert.Contains(t, refused, "has been deleted")
// An outcome code nobody issued renders no banner at all.
unknown := renderSourceLogsPageWithQuery(
t, h, sess, wh.ID, "?replay=made-up",
)
assert.NotContains(t, unknown, "alert-error")
assert.NotContains(t, unknown, "alert-success")
assert.NotContains(t, unknown, "made-up")
}

View File

@@ -0,0 +1,169 @@
package handlers
import (
"sneak.berlin/go/webhooker/internal/delivery"
)
// maxRenderedResponseBytes caps how many bytes of one stored
// delivery response body reach the event log page.
//
// The bound is enforced here and in SQL, because this page's
// memory profile must not depend on a constant in another
// package staying where it is, and because rows predating the
// delivery engine's own cap or restored from an archive are
// not covered by it at all.
//
// It happens to equal that engine cap, so a row written by the
// current engine reaches this one exactly and is never cut
// twice. Nothing here may assume the two differ: see view.
const maxRenderedResponseBytes = 4096
// deliveryResultColumns is the delivery attempt projection.
// The casts to blob are load-bearing for the same reason they
// are in eventLogColumns: they make substr and length count
// bytes rather than characters, and they make SQLite do the
// cut, so an oversized stored response never becomes a Go
// string at all.
const deliveryResultColumns = "delivery_id, attempt_num, success, " +
"status_code, error, duration, " +
"substr(cast(response_body as blob), 1, ?) AS response_body, " +
"length(cast(response_body as blob)) AS response_bytes"
// DeliveryResultView is the display-safe projection of one
// delivery attempt for the event log page. It carries a
// capped response body plus the true stored size, so the page
// can mark a response as truncated without holding the whole
// thing.
//
// Both Error and ResponseBody have been through the target's
// Redactor. The engine already masks the URL out of the
// errors it stores, so for errors this is a second line
// covering rows written before it did; for response bodies it
// is the only line, and its reach is what
// delivery.Redactor documents.
type DeliveryResultView struct {
AttemptNum int
Success bool
// StatusCode is 0 when the attempt never got a response,
// which is why the page asks HasStatusCode rather than
// printing the number.
StatusCode int
// Error is the stored failure message, redacted.
Error string
// DurationMS is how long the attempt took.
DurationMS int64
// ResponseBody holds at most maxRenderedResponseBytes
// bytes of the stored response, redacted. It is remote
// content and must only ever be rendered escaped.
ResponseBody string
// ResponseBytes is the size of the stored response body,
// before the cut and before redaction. It is what the
// remote sent only when ResponseSizeKnown is set.
ResponseBytes int64
// ResponseShownBytes is how much of that the page is
// showing. It is the size of the cut, taken before
// redaction, so the truncation marker reports what SQLite
// returned rather than how much the marker substitution
// then changed the length.
ResponseShownBytes int
// ResponseTruncated reports that the body shown may be
// incomplete, so the page owes the reader a marker. Every
// body that reaches the cap counts, because one the
// delivery engine cut at its own equal cap is
// indistinguishable from a complete one.
ResponseTruncated bool
// ResponseSizeKnown reports that ResponseBytes is the whole
// response the remote sent, so the marker may quote it. It
// is false for a body that only reaches the cap, where how
// much came after it was never recorded.
ResponseSizeKnown bool
}
// HasStatusCode reports whether the attempt got as far as an
// HTTP response. A transport failure stores no status code,
// and rendering that as "0" would read as a real status.
func (v DeliveryResultView) HasStatusCode() bool {
return v.StatusCode != 0
}
// deliveryResultRow is one row of the delivery attempt
// projection. Its response body arrives already cut to the
// cap by SQLite, with the true size beside it.
type deliveryResultRow struct {
DeliveryID string
AttemptNum int
Success bool
StatusCode int
Error string
Duration int64
ResponseBody []byte
ResponseBytes int64
}
// view projects a loaded row for rendering, stripping the
// target's own credential out of the two fields a remote peer
// gets to influence.
func (r *deliveryResultRow) view(
redactor delivery.Redactor,
) DeliveryResultView {
body := r.ResponseBody
// Two different cuts can have shortened this body, and the
// row records only one of them. SQLite cuts here, whenever
// the stored value is larger than the cap. The delivery
// engine cut earlier, whenever the remote sent more than
// its own maxBodyLog — which is this same number, so such a
// row stores the cut length as its whole length and nothing
// in it separates a response that ended at the cap from one
// severed there.
//
// So a body that reaches the cap is treated as cut either
// way. Gating on ResponseBytes alone would assume the two
// caps differ, and they do not: under the current engine
// that gate never opens.
cut := r.ResponseBytes > int64(len(body)) ||
len(body) >= maxRenderedResponseBytes
// The row holds more than the page shows only in the first
// of those cases. In the second the stored row is all there
// is, and its size is a floor rather than the true one.
sizeKnown := r.ResponseBytes > int64(len(body))
// Only a cut response can have been left mid-sequence,
// exactly as with an event body.
if cut {
body = trimPartialRune(body)
}
// A cut body goes through RedactCut: the remote controls
// the padding ahead of a credential it echoes, so it
// controls where the cut falls inside that credential, and
// the severed prefix left behind matches no secret whole.
rendered := string(body)
if cut {
rendered = redactor.RedactCut(rendered)
} else {
rendered = redactor.Redact(rendered)
}
return DeliveryResultView{
AttemptNum: r.AttemptNum,
Success: r.Success,
StatusCode: r.StatusCode,
Error: redactor.Redact(r.Error),
DurationMS: r.Duration,
ResponseBody: rendered,
ResponseBytes: r.ResponseBytes,
ResponseShownBytes: len(body),
ResponseTruncated: cut,
ResponseSizeKnown: sizeKnown,
}
}

View File

@@ -0,0 +1,515 @@
package handlers_test
import (
"net/http"
"net/http/httptest"
"strconv"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/session"
)
// responseCap is the number of response bytes the event log
// page is allowed to render for one delivery attempt.
const responseCap = handlers.MaxRenderedResponseBytesForTest
// failedAttempt describes the failed delivery every test in
// this file seeds. The values are distinctive so that finding
// them in the rendered page cannot be a coincidence.
const (
attemptStatusCode = 502
attemptDurationMS = 1234
attemptNumber = 3
attemptError = "upstream returned 502 Bad Gateway"
)
// seedFailedDeliveryWithResponse records an event, a failed
// delivery against targetID, and one delivery result carrying
// the given response body. It returns the delivery.
//
// Distinct from seedFailedDelivery in delivery_replay_test.go,
// which seeds an attempt with no response body and returns the
// event as well; these tests need the recorded response.
func seedFailedDeliveryWithResponse(
t *testing.T,
dbMgr *database.WebhookDBManager,
webhookID, targetID, responseBody string,
) *database.Delivery {
t.Helper()
webhookDB, err := dbMgr.GetDB(webhookID)
require.NoError(t, err)
event := &database.Event{
WebhookID: webhookID,
Method: http.MethodPost,
Body: `{"test":true}`,
ContentType: contentTypeJSON,
}
require.NoError(t, webhookDB.Omit(
clause.Associations,
).Create(event).Error)
dlv := &database.Delivery{
EventID: event.ID,
TargetID: targetID,
Status: database.DeliveryStatusFailed,
}
require.NoError(t, webhookDB.Omit(
clause.Associations,
).Create(dlv).Error)
result := &database.DeliveryResult{
DeliveryID: dlv.ID,
AttemptNum: attemptNumber,
Success: false,
StatusCode: attemptStatusCode,
ResponseBody: responseBody,
Error: attemptError,
Duration: attemptDurationMS,
}
require.NoError(t, webhookDB.Omit(
clause.Associations,
).Create(result).Error)
return dlv
}
// seedFailureAndRender seeds a failed delivery against a
// target of the given type and config, and returns the
// rendered event log page.
func seedFailureAndRender(
t *testing.T,
targetType database.TargetType,
config, responseBody string,
) string {
t.Helper()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
tgt := seedConfiguredTarget(
t, db, wh.ID, targetType, config,
)
seedFailedDeliveryWithResponse(
t, dbMgr, wh.ID, tgt.ID, responseBody,
)
return renderSourceLogsPage(t, h, sess, wh.ID)
}
// TestHandleSourceLogs_RendersFailedAttempt is the regression
// test for the reported gap: a failed delivery used to render
// as the status word alone, so diagnosing it meant opening the
// per-webhook SQLite file by hand.
func TestHandleSourceLogs_RendersFailedAttempt(t *testing.T) {
t.Parallel()
body := seedFailureAndRender(
t,
database.TargetTypeHTTP,
`{"url":"https://example.com/hook/abc"}`,
"upstream exploded",
)
assert.Contains(
t, body, strconv.Itoa(attemptStatusCode),
"the attempt's status code must reach the page",
)
assert.Contains(
t, body, attemptError,
"the attempt's error must reach the page",
)
assert.Contains(
t, body, strconv.Itoa(attemptDurationMS),
"the attempt's duration must reach the page",
)
assert.Contains(
t, body, "Attempt "+strconv.Itoa(attemptNumber),
"the attempt number must reach the page",
)
assert.Contains(
t, body, "upstream exploded",
"the attempt's response body must reach the page",
)
}
// TestHandleSourceLogs_EscapesResponseBody proves the
// response body is treated as the untrusted remote content it
// is. The remote chooses these bytes and the page is rendered
// inside the operator's authenticated origin, where the
// application's own CSP allows inline script from 'self'.
func TestHandleSourceLogs_EscapesResponseBody(t *testing.T) {
t.Parallel()
const payload = `<script>alert("xss")</script>`
body := seedFailureAndRender(
t,
database.TargetTypeHTTP,
`{"url":"https://example.com/hook/abc"}`,
payload,
)
assert.NotContains(t, body, payload)
assert.NotContains(t, body, "<script>alert")
assert.Contains(t, body, "alert")
}
// TestHandleSourceLogs_RedactsCredentialEchoedInResponse
// covers the case that makes rendering a response body a
// disclosure question at all: the remote echoes back the
// credential the request carried, and the page would then put
// it on the operator's screen.
func TestHandleSourceLogs_RedactsCredentialEchoedInResponse(
t *testing.T,
) {
t.Parallel()
body := seedFailureAndRender(
t,
database.TargetTypeSlack,
`{"webhookUrl":"`+slackWebhookURL+`"}`,
"no_service: "+slackWebhookURL,
)
assert.NotContains(t, body, slackSecretPath)
assert.NotContains(t, body, "T00000000")
assert.NotContains(t, body, "B00000000")
assert.Contains(t, body, delivery.RedactionMarker)
// The rest of the response is still shown, or the
// redaction would have cost the operator the diagnosis.
assert.Contains(t, body, "no_service")
}
// TestHandleSourceLogs_RedactsCredentialEchoedInError covers
// the same disclosure through the error field. The delivery
// engine masks the URL out of the errors it stores, so this
// holds the read path to the rows written before it did.
func TestHandleSourceLogs_RedactsCredentialEchoedInError(
t *testing.T,
) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
tgt := seedConfiguredTarget(
t, db, wh.ID,
database.TargetTypeSlack,
`{"webhookUrl":"`+slackWebhookURL+`"}`,
)
dlv := seedFailedDeliveryWithResponse(t, dbMgr, wh.ID, tgt.ID, "")
webhookDB, err := dbMgr.GetDB(wh.ID)
require.NoError(t, err)
// An unmasked transport error, exactly as Go's HTTP
// client renders one.
require.NoError(t, webhookDB.Model(
&database.DeliveryResult{},
).Where(
"delivery_id = ?", dlv.ID,
).Update(
"error",
`Post "`+slackWebhookURL+`": dial tcp: i/o timeout`,
).Error)
body := renderSourceLogsPage(t, h, sess, wh.ID)
assert.NotContains(t, body, slackSecretPath)
assert.Contains(t, body, delivery.RedactionMarker)
assert.Contains(t, body, "i/o timeout")
}
// severedPadding is the filler that puts the end of an echoed
// webhook URL five bytes past a cut at the response cap, so
// the cut leaves the workspace ID, the bot ID and all but the
// last few token characters behind.
func severedPadding() string {
const severedTail = 5
return strings.Repeat(
"A", responseCap-len(slackWebhookURL)+severedTail,
)
}
// TestHandleSourceLogs_RedactsCredentialSeveredByTheEngineCut
// is the regression test for a redactor gated on the SQL cut
// alone. The delivery engine stops reading a response at its
// own cap, which is the same number of bytes this page
// renders, so a row the engine cut is byte-for-byte
// indistinguishable from a complete response and that gate
// never opened on anything the engine writes.
//
// The seeded body is what the engine stores for any remote
// that sends at least that much: exactly responseCap bytes,
// ending in a severed webhook URL.
// TestDeliverHTTP_CutsStoredResponseAtMaxBodyLog in
// internal/delivery pins that this is the size it produces.
func TestHandleSourceLogs_RedactsCredentialSeveredByTheEngineCut(
t *testing.T,
) {
t.Parallel()
sent := severedPadding() + slackWebhookURL +
strings.Repeat("Z", 128)
stored := sent[:responseCap]
require.Len(
t, stored, responseCap,
"the engine stores exactly the cap, never more",
)
require.Contains(
t, stored, "T00000000",
"the severed credential must be in what is seeded",
)
body := seedFailureAndRender(
t,
database.TargetTypeSlack,
`{"webhookUrl":"`+slackWebhookURL+`"}`,
stored,
)
assert.NotContains(t, body, "T00000000")
assert.NotContains(t, body, "B00000000")
assert.Contains(t, body, delivery.RedactionMarker)
assert.Contains(
t, body, "reached the recording limit",
"a body the engine cut must not be shown as complete",
)
}
// TestHandleSourceLogs_RedactsCredentialSeveredBySQLCut covers
// the same severing for a row larger than the cap, which is
// SQLite's cut rather than the engine's. The current engine
// writes no such row; rows predating its cap or restored from
// an archive are not bounded by it, which is why the page cuts
// again in SQL and has to redact that cut too.
func TestHandleSourceLogs_RedactsCredentialSeveredBySQLCut(
t *testing.T,
) {
t.Parallel()
stored := severedPadding() + slackWebhookURL +
strings.Repeat("Z", 128)
require.Greater(
t, len(stored), responseCap,
"the stored body must exceed the cap or nothing is cut",
)
body := seedFailureAndRender(
t,
database.TargetTypeSlack,
`{"webhookUrl":"`+slackWebhookURL+`"}`,
stored,
)
assert.NotContains(t, body, "T00000000")
assert.NotContains(t, body, "B00000000")
assert.NotContains(
t, body, slackWebhookURL[:len(slackWebhookURL)-10],
)
assert.Contains(t, body, delivery.RedactionMarker)
}
// TestHandleSourceLogs_RedactsForSoftDeletedTarget covers a
// target an operator has deleted. The row is only soft deleted
// and its deliveries survive in the per-webhook database, so
// its redactor has to survive with it or every response body
// it ever recorded renders unredacted.
func TestHandleSourceLogs_RedactsForSoftDeletedTarget(
t *testing.T,
) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
tgt := seedConfiguredTarget(
t, db, wh.ID,
database.TargetTypeSlack,
`{"webhookUrl":"`+slackWebhookURL+`"}`,
)
seedFailedDeliveryWithResponse(
t, dbMgr, wh.ID, tgt.ID,
"no_service: "+slackWebhookURL,
)
require.NoError(t, db.DB().Delete(tgt).Error)
body := renderSourceLogsPage(t, h, sess, wh.ID)
assert.NotContains(t, body, slackSecretPath)
assert.NotContains(t, body, "T00000000")
assert.Contains(t, body, delivery.RedactionMarker)
assert.Contains(t, body, "no_service")
}
// TestHandleSourceLogs_BoundsRenderedAttempts pins the ceiling
// on how many of one delivery's attempts reach the page, and
// that what it drops is counted rather than hidden.
func TestHandleSourceLogs_BoundsRenderedAttempts(t *testing.T) {
t.Parallel()
const extraAttempts = 7
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeLog, "",
)
dlv := seedFailedDeliveryWithResponse(t, dbMgr, wh.ID, tgt.ID, "")
webhookDB, err := dbMgr.GetDB(wh.ID)
require.NoError(t, err)
total := handlers.MaxRenderedAttemptsForTest + extraAttempts
// seedFailedDeliveryWithResponse already recorded one attempt.
for i := range total - 1 {
require.NoError(t, webhookDB.Omit(
clause.Associations,
).Create(&database.DeliveryResult{
DeliveryID: dlv.ID,
AttemptNum: attemptNumber + 1 + i,
Error: attemptError,
}).Error)
}
views := h.LoadEventLogViewsForTest(
httptest.NewRecorder(), *wh, 1,
)
require.Len(t, views, 1)
require.Len(t, views[0].Deliveries, 1)
dv := views[0].Deliveries[0]
assert.Equal(t, total, dv.AttemptCount)
assert.Len(
t, dv.Results, handlers.MaxRenderedAttemptsForTest,
)
assert.Equal(t, extraAttempts, dv.AttemptsOmitted)
page := renderSourceLogsPage(t, h, sess, wh.ID)
assert.Contains(t, page, "attempts omitted")
assert.Contains(
t, page, strconv.Itoa(total)+" attempts",
"the header must count every recorded attempt",
)
}
// TestHandleSourceLogs_BoundsOversizeResponse proves the
// rendered page is bounded by the response cap rather than by
// the stored response size. The cut happens in SQLite, so the
// oversized value never becomes a Go string; this asserts the
// observable consequence, that neither the page nor the
// projection carries the tail.
func TestHandleSourceLogs_BoundsOversizeResponse(t *testing.T) {
t.Parallel()
const tail = "QQRESPONSETAILQQ"
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
tgt := seedConfiguredTarget(
t, db, wh.ID, database.TargetTypeLog, "",
)
stored := strings.Repeat("A", responseCap*4) + tail
seedFailedDeliveryWithResponse(t, dbMgr, wh.ID, tgt.ID, stored)
views := h.LoadEventLogViewsForTest(
httptest.NewRecorder(), *wh, 1,
)
require.Len(t, views, 1)
require.Len(t, views[0].Deliveries, 1)
require.Len(t, views[0].Deliveries[0].Results, 1)
attempt := views[0].Deliveries[0].Results[0]
assert.LessOrEqual(
t, len(attempt.ResponseBody), responseCap,
)
assert.Equal(
t, int64(len(stored)), attempt.ResponseBytes,
)
assert.True(t, attempt.ResponseTruncated)
page := renderSourceLogsPage(t, h, sess, wh.ID)
assert.NotContains(t, page, tail)
assert.Contains(
t, page, "Response truncated for display",
)
}

View File

@@ -19,6 +19,14 @@ func (s *Handlers) SetLogForTest(log *slog.Logger) {
// to the handlers_test package. // to the handlers_test package.
const MaxRenderedBodyBytesForTest = maxRenderedBodyBytes const MaxRenderedBodyBytesForTest = maxRenderedBodyBytes
// MaxRenderedResponseBytesForTest exposes the event log's
// delivery response cap to the handlers_test package.
const MaxRenderedResponseBytesForTest = maxRenderedResponseBytes
// MaxRenderedAttemptsForTest exposes the event log's
// per-delivery attempt ceiling to the handlers_test package.
const MaxRenderedAttemptsForTest = maxRenderedAttempts
// DummyVerificationsForTest reports how many equivalent-cost // DummyVerificationsForTest reports how many equivalent-cost
// verifications were charged for usernames that do not exist. It // verifications were charged for usernames that do not exist. It
// lets a test prove the anti-enumeration path ran without timing // lets a test prove the anti-enumeration path ran without timing
@@ -43,7 +51,9 @@ func (s *Handlers) LoadEventLogViewsForTest(
webhook database.Webhook, webhook database.Webhook,
page int, page int,
) []EventLogView { ) []EventLogView {
views, _ := s.loadEventsWithDeliveries(w, webhook, nil, page) views, _, _ := s.loadEventsWithDeliveries(
w, webhook, nil, page,
)
return views return views
} }

View File

@@ -24,9 +24,32 @@ import (
"sneak.berlin/go/webhooker/internal/session" "sneak.berlin/go/webhooker/internal/session"
) )
type noopNotifier struct{} // recordingNotifier is a delivery.Notifier that records the tasks it
// was handed, so a test can prove a handler queued the delivery it
// claims to have queued — and, on the refusal paths, that it queued
// nothing.
type recordingNotifier struct {
mu sync.Mutex
tasks []delivery.Task
}
func (n *noopNotifier) Notify([]delivery.Task) {} func (n *recordingNotifier) Notify(tasks []delivery.Task) {
n.mu.Lock()
defer n.mu.Unlock()
n.tasks = append(n.tasks, tasks...)
}
// Tasks returns a copy of the recorded tasks.
func (n *recordingNotifier) Tasks() []delivery.Task {
n.mu.Lock()
defer n.mu.Unlock()
out := make([]delivery.Task, len(n.tasks))
copy(out, n.tasks)
return out
}
// recordingEvictor is a delivery.WebhookEvictor that records // recordingEvictor is a delivery.WebhookEvictor that records
// the webhook ids it was asked to evict, so a test can prove // the webhook ids it was asked to evict, so a test can prove
@@ -74,8 +97,11 @@ func newTestApp(
database.NewWebhookDBManager, database.NewWebhookDBManager,
healthcheck.New, healthcheck.New,
session.New, session.New,
func() delivery.Notifier { func() *recordingNotifier {
return &noopNotifier{} return &recordingNotifier{}
},
func(n *recordingNotifier) delivery.Notifier {
return n
}, },
func() *recordingEvictor { func() *recordingEvictor {
return &recordingEvictor{} return &recordingEvictor{}

View File

@@ -15,6 +15,11 @@ import (
"sneak.berlin/go/webhooker/internal/session" "sneak.berlin/go/webhooker/internal/session"
) )
// contentTypeJSON is the content type the seeded events in this
// package carry. Shared across the seed helpers so the literal
// appears once.
const contentTypeJSON = "application/json"
// seedDeliveredEvent records an event and a delivery for it in // seedDeliveredEvent records an event and a delivery for it in
// the webhook's own database, so the log page has a delivery // the webhook's own database, so the log page has a delivery
// to render against the target. // to render against the target.
@@ -32,7 +37,7 @@ func seedDeliveredEvent(
WebhookID: webhookID, WebhookID: webhookID,
Method: http.MethodPost, Method: http.MethodPost,
Body: `{"test":true}`, Body: `{"test":true}`,
ContentType: "application/json", ContentType: contentTypeJSON,
} }
require.NoError(t, webhookDB.Omit( require.NoError(t, webhookDB.Omit(
@@ -60,10 +65,26 @@ func renderSourceLogsPage(
) string { ) string {
t.Helper() t.Helper()
return renderSourceLogsPageWithQuery(
t, h, sess, webhookID, "",
)
}
// renderSourceLogsPageWithQuery is renderSourceLogsPage over a
// caller-supplied query string, for the page state a redirect back to
// the log carries in one.
func renderSourceLogsPageWithQuery(
t *testing.T,
h *handlers.Handlers,
sess *session.Session,
webhookID, query string,
) string {
t.Helper()
req := httptest.NewRequestWithContext( req := httptest.NewRequestWithContext(
context.Background(), context.Background(),
http.MethodGet, http.MethodGet,
"/source/"+webhookID+"/logs", "/source/"+webhookID+"/logs"+query,
nil, nil,
) )

View File

@@ -4,11 +4,13 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"net/http" "net/http"
"slices"
"strconv" "strconv"
"strings" "strings"
"github.com/go-chi/chi" "github.com/go-chi/chi"
"github.com/google/uuid" "github.com/google/uuid"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database" "sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery" "sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/signature" "sneak.berlin/go/webhooker/internal/signature"
@@ -101,6 +103,32 @@ type DeliveryView struct {
ID string ID string
Status database.DeliveryStatus Status database.DeliveryStatus
Target delivery.TargetView Target delivery.TargetView
// Results is this delivery's attempts in attempt order,
// bounded by maxRenderedAttempts. Without them a failure
// renders as the status word alone and says nothing about
// why.
Results []DeliveryResultView
// AttemptCount is how many attempts were recorded, which
// is more than len(Results) once the middle was dropped.
AttemptCount int
// AttemptsOmitted is how many attempts were dropped from
// the middle of Results. The page must show it, or the
// bound would hide history rather than fold it.
AttemptsOmitted int
}
// eventLogTarget is what the event log needs to know about
// one target: the display-safe view its template renders, and
// the redactor that keeps that target's own credential out of
// the text its remote peer chose. The two are kept together
// so a caller cannot pick up one without the other, and apart
// from TargetView so the secrets never reach a template.
type eventLogTarget struct {
View delivery.TargetView
Redactor delivery.Redactor
} }
// HandleSourceList shows a list of user's webhooks. // HandleSourceList shows a list of user's webhooks.
@@ -769,21 +797,42 @@ func (h *Handlers) HandleSourceLogs() http.HandlerFunc {
return return
} }
targets := h.loadTargetMap(webhook.ID) targets, err := h.loadTargetMap(webhook.ID)
if err != nil {
// Without the map every delivery renders through a
// zero redactor, so failing the page is the only
// safe answer.
h.serverError(w, "failed to load targets", err)
return
}
page := h.parsePage(r) page := h.parsePage(r)
evts, total := h.loadEventsWithDeliveries( evts, total, ok := h.loadEventsWithDeliveries(
w, webhook, targets, page, w, webhook, targets, page,
) )
if !ok {
return
}
totalPages := int(total) / paginationPerPage totalPages := int(total) / paginationPerPage
if int(total)%paginationPerPage != 0 { if int(total)%paginationPerPage != 0 {
totalPages++ totalPages++
} }
// The banner a replay POST redirected back with. The
// message comes from a fixed set keyed by the outcome
// code, never from the query string itself.
replayMsg, replayOK := replayOutcome(
r.URL.Query().Get(replayOutcomeParam),
)
data := map[string]any{ data := map[string]any{
tmplKeyWebhook: &webhook, tmplKeyWebhook: &webhook,
"Events": evts, "Events": evts,
"ReplayMessage": replayMsg,
"ReplayQueued": replayOK,
"Page": page, "Page": page,
"TotalPages": totalPages, "TotalPages": totalPages,
"TotalEvents": total, "TotalEvents": total,
@@ -798,29 +847,54 @@ func (h *Handlers) HandleSourceLogs() http.HandlerFunc {
} }
// loadTargetMap loads targets into a map of display-safe // loadTargetMap loads targets into a map of display-safe
// views keyed by target ID. The projection happens here so // views keyed by target ID, each paired with its redactor.
// that no caller can hand a raw target, configuration blob // The projection happens here so that no caller can hand a
// and all, to a template. // raw target, configuration blob and all, to a template: the
// raw rows do not leave this function.
//
// The load is Unscoped because deleting a target only soft
// deletes the row while its deliveries survive in the
// per-webhook database: a scoped load leaves those deliveries
// with a zero redactor, which renders their response bodies
// unredacted. Only the redactor half of the map is built from
// deleted rows. The view half, which is what the page lists,
// stays scoped.
func (h *Handlers) loadTargetMap( func (h *Handlers) loadTargetMap(
webhookID string, webhookID string,
) map[string]delivery.TargetView { ) (map[string]eventLogTarget, error) {
var targets []database.Target var targets []database.Target
h.db.DB().Where( err := h.db.DB().Unscoped().Where(
"webhook_id = ?", webhookID, "webhook_id = ?", webhookID,
).Find(&targets) ).Find(&targets).Error
if err != nil {
views := delivery.NewTargetViews(targets) return nil, err
targetMap := make(
map[string]delivery.TargetView, len(views),
)
for _, v := range views {
targetMap[v.ID] = v
} }
return targetMap targetMap := make(
map[string]eventLogTarget, len(targets),
)
live := make([]database.Target, 0, len(targets))
for i := range targets {
targetMap[targets[i].ID] = eventLogTarget{
Redactor: delivery.NewRedactor(&targets[i]),
}
if !targets[i].DeletedAt.Valid {
live = append(live, targets[i])
}
}
// The views come from NewTargetViews rather than being
// rebuilt here, so the masking rules stay in one place.
for _, v := range delivery.NewTargetViews(live) {
entry := targetMap[v.ID]
entry.View = v
targetMap[v.ID] = entry
}
return targetMap, nil
} }
// parsePage extracts a page number from the query string. // parsePage extracts a page number from the query string.
@@ -841,18 +915,22 @@ func (h *Handlers) parsePage(r *http.Request) int {
// deliveries from the per-webhook database. Events come back // deliveries from the per-webhook database. Events come back
// as capped projections rather than database.Event rows: see // as capped projections rather than database.Event rows: see
// eventLogColumns for why the cut happens in SQL. // eventLogColumns for why the cut happens in SQL.
//
// The bool reports whether the load succeeded. It is false
// once this has answered the request with an error, and the
// caller must then render nothing further.
func (h *Handlers) loadEventsWithDeliveries( func (h *Handlers) loadEventsWithDeliveries(
w http.ResponseWriter, w http.ResponseWriter,
webhook database.Webhook, webhook database.Webhook,
targetMap map[string]delivery.TargetView, targetMap map[string]eventLogTarget,
page int, page int,
) ([]EventLogView, int64) { ) ([]EventLogView, int64, bool) {
var totalEvents int64 var totalEvents int64
var result []EventLogView var result []EventLogView
if !h.dbMgr.DBExists(webhook.ID) { if !h.dbMgr.DBExists(webhook.ID) {
return result, totalEvents return result, totalEvents, true
} }
webhookDB, err := h.dbMgr.GetDB(webhook.ID) webhookDB, err := h.dbMgr.GetDB(webhook.ID)
@@ -861,7 +939,7 @@ func (h *Handlers) loadEventsWithDeliveries(
w, "failed to get webhook database", err, w, "failed to get webhook database", err,
) )
return nil, 0 return nil, 0, false
} }
webhookDB.Model(&database.Event{}).Where( webhookDB.Model(&database.Event{}).Where(
@@ -881,43 +959,170 @@ func (h *Handlers) loadEventsWithDeliveries(
).Find(&rows) ).Find(&rows)
result = make([]EventLogView, len(rows)) result = make([]EventLogView, len(rows))
eventDeliveries := make([][]database.Delivery, len(rows))
var deliveryIDs []string
for i := range rows { for i := range rows {
result[i] = rows[i].view() result[i] = rows[i].view()
var deliveries []database.Delivery
webhookDB.Where( webhookDB.Where(
"event_id = ?", rows[i].ID, "event_id = ?", rows[i].ID,
).Find(&deliveries) ).Find(&eventDeliveries[i])
for j := range eventDeliveries[i] {
deliveryIDs = append(
deliveryIDs, eventDeliveries[i][j].ID,
)
}
}
attempts, err := h.loadDeliveryResults(
webhookDB, deliveryIDs,
)
if err != nil {
h.serverError(
w, "failed to load delivery attempts", err,
)
return nil, 0, false
}
for i := range rows {
result[i].Deliveries = newDeliveryViews( result[i].Deliveries = newDeliveryViews(
deliveries, targetMap, eventDeliveries[i], targetMap, attempts,
) )
} }
return result, totalEvents return result, totalEvents, true
}
// deliveryIDChunkSize bounds how many delivery IDs go into one
// IN clause. SQLite refuses a statement carrying more than
// SQLITE_MAX_VARIABLE_NUMBER (32766) bound parameters, and a
// page holds one delivery per target per event, so a webhook
// with enough targets would turn the whole query into an error
// and the page into zero attempts.
const deliveryIDChunkSize = 500
// loadDeliveryResults loads the recorded attempts for the
// page's deliveries, keyed by delivery ID.
//
// Each response body is cut by SQLite rather than in Go, for
// the reason deliveryResultColumns gives. How many attempts a
// delivery has is the target's MaxRetries, which the
// authenticated operator sets; how many of them reach the page
// is bounded again by maxRenderedAttempts.
func (h *Handlers) loadDeliveryResults(
webhookDB *gorm.DB,
deliveryIDs []string,
) (map[string][]deliveryResultRow, error) {
byDelivery := make(map[string][]deliveryResultRow)
for chunk := range slices.Chunk(
deliveryIDs, deliveryIDChunkSize,
) {
var rows []deliveryResultRow
err := webhookDB.Model(
&database.DeliveryResult{},
).Select(
deliveryResultColumns, maxRenderedResponseBytes,
).Where(
"delivery_id IN ?", chunk,
).Order("attempt_num ASC").Find(&rows).Error
if err != nil {
// Returning what was loaded so far renders the
// deliveries in the failed chunk as never having run,
// which is indistinguishable from ones that really
// never ran. The page fails instead.
return nil, err
}
for i := range rows {
byDelivery[rows[i].DeliveryID] = append(
byDelivery[rows[i].DeliveryID], rows[i],
)
}
}
return byDelivery, nil
} }
// newDeliveryViews projects deliveries for rendering, // newDeliveryViews projects deliveries for rendering,
// resolving each one's target to its display-safe view. // resolving each one's target to its display-safe view and
// each one's attempts through that target's redactor.
func newDeliveryViews( func newDeliveryViews(
deliveries []database.Delivery, deliveries []database.Delivery,
targetMap map[string]delivery.TargetView, targetMap map[string]eventLogTarget,
attempts map[string][]deliveryResultRow,
) []DeliveryView { ) []DeliveryView {
views := make([]DeliveryView, len(deliveries)) views := make([]DeliveryView, len(deliveries))
for i := range deliveries { for i := range deliveries {
target := targetMap[deliveries[i].TargetID]
rows := attempts[deliveries[i].ID]
results, omitted := renderedAttempts(
rows, target.Redactor,
)
views[i] = DeliveryView{ views[i] = DeliveryView{
ID: deliveries[i].ID, ID: deliveries[i].ID,
Status: deliveries[i].Status, Status: deliveries[i].Status,
Target: targetMap[deliveries[i].TargetID], Target: target.View,
Results: results,
AttemptCount: len(rows),
AttemptsOmitted: omitted,
} }
} }
return views return views
} }
// maxRenderedAttempts bounds how many of one delivery's
// attempts the page renders. Past it the middle is dropped and
// counted, keeping the first attempts and the last ones: how
// the delivery started failing and how it ended are what a
// reader needs, and the count says plainly that the rest was
// dropped rather than never recorded.
const (
renderedAttemptsHead = 10
renderedAttemptsTail = 10
maxRenderedAttempts = renderedAttemptsHead +
renderedAttemptsTail
)
// renderedAttempts projects a delivery's attempts through the
// target's redactor, at most maxRenderedAttempts of them, and
// reports how many it dropped.
func renderedAttempts(
rows []deliveryResultRow,
redactor delivery.Redactor,
) ([]DeliveryResultView, int) {
omitted := 0
if len(rows) > maxRenderedAttempts {
omitted = len(rows) - maxRenderedAttempts
kept := make(
[]deliveryResultRow, 0, maxRenderedAttempts,
)
kept = append(kept, rows[:renderedAttemptsHead]...)
kept = append(
kept, rows[len(rows)-renderedAttemptsTail:]...,
)
rows = kept
}
views := make([]DeliveryResultView, len(rows))
for i := range rows {
views[i] = rows[i].view(redactor)
}
return views, omitted
}
// HandleEntrypointCreate handles adding a new entrypoint. // HandleEntrypointCreate handles adding a new entrypoint.
func (h *Handlers) HandleEntrypointCreate() http.HandlerFunc { func (h *Handlers) HandleEntrypointCreate() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) { return func(w http.ResponseWriter, r *http.Request) {

View File

@@ -82,6 +82,7 @@ type Set struct {
deliveriesSucceeded *prometheus.CounterVec deliveriesSucceeded *prometheus.CounterVec
deliveriesFailed *prometheus.CounterVec deliveriesFailed *prometheus.CounterVec
deliveryRetries *prometheus.CounterVec deliveryRetries *prometheus.CounterVec
deliveryReplays *prometheus.CounterVec
deliveryDuration *prometheus.HistogramVec deliveryDuration *prometheus.HistogramVec
deliveriesPending *prometheus.GaugeVec deliveriesPending *prometheus.GaugeVec
deliveriesRetrying *prometheus.GaugeVec deliveriesRetrying *prometheus.GaugeVec
@@ -149,6 +150,22 @@ func (s *Set) ObserveDeliveryDuration(
Observe(d.Seconds()) Observe(d.Seconds())
} }
// DeliveryReplayed counts one delivery an operator replayed from the
// event log.
//
// A replay runs the ordinary engine path, so it already moves the
// attempt, outcome and duration series exactly as a first delivery
// does — deliberately, since a replay is a real delivery and hiding it
// from those would misreport the pipeline. This counter is the one
// place the two are distinguishable, and it carries the existing
// target-type label rather than adding a replay dimension to every
// other series.
func (s *Set) DeliveryReplayed(t database.TargetType) {
s.deliveryReplays.
WithLabelValues(normalizeTargetType(t)).
Inc()
}
// DeliveryStatusChanged counts a delivery's transition into a new // DeliveryStatusChanged counts a delivery's transition into a new
// status. The mapping from status to counter lives here, next to the // status. The mapping from status to counter lives here, next to the
// collectors, so the engine has a single call for every transition it // collectors, so the engine has a single call for every transition it
@@ -271,6 +288,16 @@ func (s *Set) registerCounters(factory promauto.Factory) {
}, },
[]string{targetTypeLabel}, []string{targetTypeLabel},
) )
s.deliveryReplays = factory.NewCounterVec(
prometheus.CounterOpts{
Namespace: namespace,
Name: "delivery_replays_total",
Help: "Deliveries an operator replayed from the " +
"event log, by target type.",
},
[]string{targetTypeLabel},
)
} }
func (s *Set) registerGauges(factory promauto.Factory) { func (s *Set) registerGauges(factory promauto.Factory) {
@@ -322,6 +349,7 @@ func (s *Set) initSeries() {
s.deliveriesSucceeded.WithLabelValues(label) s.deliveriesSucceeded.WithLabelValues(label)
s.deliveriesFailed.WithLabelValues(label) s.deliveriesFailed.WithLabelValues(label)
s.deliveryRetries.WithLabelValues(label) s.deliveryRetries.WithLabelValues(label)
s.deliveryReplays.WithLabelValues(label)
s.deliveriesPending.WithLabelValues(label) s.deliveriesPending.WithLabelValues(label)
s.deliveriesRetrying.WithLabelValues(label) s.deliveriesRetrying.WithLabelValues(label)
s.circuitBreakersOpen.WithLabelValues(label) s.circuitBreakersOpen.WithLabelValues(label)

View File

@@ -34,6 +34,16 @@ const (
// password change rate limit. // password change rate limit.
passwordChangeRateInterval = 1 * time.Minute passwordChangeRateInterval = 1 * time.Minute
// replayRateLimit is the maximum number of delivery replays one
// client may queue per interval. Each replay puts a delivery on
// the engine's queue, so without a ceiling one operator holding
// the button down — or scripting it — queues unbounded outbound
// work. It sits far above any rate a person clicks at.
replayRateLimit = 30
// replayRateInterval is the time window for the replay limit.
replayRateInterval = 1 * time.Minute
// receiverRateInterval is the time window for the webhook // receiverRateInterval is the time window for the webhook
// receiver rate limit. The configured limit is expressed in // receiver rate limit. The configured limit is expressed in
// requests per minute. // requests per minute.
@@ -290,6 +300,21 @@ func (m *Middleware) PasswordChangeRateLimit() func(http.Handler) http.Handler {
) )
} }
// ReplayRateLimit returns middleware that enforces per-IP rate
// limiting on delivery replays.
//
// Like the password-change limit it is spent on arrival, which is safe
// for the same reason: RequireAuth runs ahead of it, so only a request
// already carrying a valid session can reach the bucket.
func (m *Middleware) ReplayRateLimit() func(http.Handler) http.Handler {
return m.postRateLimit(
replayRateLimit,
replayRateInterval,
"delivery replay rate limit exceeded",
"Too many replays. Please try again later.",
)
}
// postRateLimit builds middleware that enforces a per-IP rate // postRateLimit builds middleware that enforces a per-IP rate
// limit on POST requests only; all other methods pass through // limit on POST requests only; all other methods pass through
// unaffected. Requests over the limit receive a 429 with the // unaffected. Requests over the limit receive a 429 with the

View File

@@ -201,6 +201,18 @@ func (s *Server) setupSourceRoutes() {
"/logs/{eventID}/body", "/logs/{eventID}/body",
s.h.HandleEventBodyDownload(), s.h.HandleEventBodyDownload(),
) )
// Replay is the one page action that queues outbound work:
// it creates a delivery from a stored event and hands it to
// the delivery engine. The rate limit is what bounds a
// held-down button or a scripted loop; the handler
// separately refuses a replay while an earlier one for the
// same event and target is still in flight. POST only, so
// the action cannot be taken by a link, a prefetch or an
// image tag.
r.With(s.mw.ReplayRateLimit()).Post(
"/deliveries/{deliveryID}/replay",
s.h.HandleDeliveryReplay(),
)
r.Post( r.Post(
"/entrypoints", "/entrypoints",
s.h.HandleEntrypointCreate(), s.h.HandleEntrypointCreate(),

View File

@@ -310,6 +310,75 @@ func (e *testEnv) seedEvent(
return event return event
} }
// seedTarget creates an active HTTP target for a webhook.
func (e *testEnv) seedTarget(
t *testing.T,
webhookID string,
) *database.Target {
t.Helper()
tgt := &database.Target{
WebhookID: webhookID,
Name: "routed-target",
Type: database.TargetTypeHTTP,
Active: true,
Config: `{"url":"http://93.184.216.34/hook"}`,
}
require.NoError(
t,
e.db.DB().Omit(clause.Associations).Create(tgt).Error,
)
return tgt
}
// seedFailedDelivery records a terminally failed delivery of an event
// to a target in the webhook's own database.
func (e *testEnv) seedFailedDelivery(
t *testing.T,
webhookID, eventID, targetID string,
) *database.Delivery {
t.Helper()
webhookDB, err := e.dbMgr.GetDB(webhookID)
require.NoError(t, err)
dlv := &database.Delivery{
EventID: eventID,
TargetID: targetID,
Status: database.DeliveryStatusFailed,
}
require.NoError(
t,
webhookDB.Omit(clause.Associations).Create(dlv).Error,
)
return dlv
}
// countDeliveries reports how many deliveries a webhook's database
// holds.
func (e *testEnv) countDeliveries(
t *testing.T, webhookID string,
) int64 {
t.Helper()
webhookDB, err := e.dbMgr.GetDB(webhookID)
require.NoError(t, err)
var count int64
require.NoError(
t,
webhookDB.Model(&database.Delivery{}).
Count(&count).Error,
)
return count
}
// storedHash reads the current password hash for a username. // storedHash reads the current password hash for a username.
func (e *testEnv) storedHash(t *testing.T, username string) string { func (e *testEnv) storedHash(t *testing.T, username string) string {
t.Helper() t.Helper()
@@ -674,6 +743,85 @@ func TestSourceLogsBody_OtherUser404s(t *testing.T) {
assert.Equal(t, "/pages/login", anon.Header().Get("Location")) assert.Equal(t, "/pages/login", anon.Header().Get("Location"))
} }
// TestDeliveryReplay_PostOnlyAndCSRFProtected walks the replay action
// through the production router rather than a forged route context,
// which is the only way to prove what the route group actually gives
// it: a GET cannot trigger a replay, an unauthenticated request never
// reaches the handler, a POST without the token is refused by CSRF,
// and the form the template emits — token and action URL both — works
// as rendered.
func TestDeliveryReplay_PostOnlyAndCSRFProtected(t *testing.T) {
t.Parallel()
env := newTestEnv(t)
userID, _ := env.seedUser(t, "replayer", "somepassword")
cookies := env.authCookies(t, userID, "replayer")
wh := env.seedWebhook(t, userID)
tgt := env.seedTarget(t, wh.ID)
evt := env.seedEvent(t, wh.ID, `{"replay":"me"}`)
dlv := env.seedFailedDelivery(t, wh.ID, evt.ID, tgt.ID)
path := "/source/" + wh.ID + "/deliveries/" + dlv.ID +
"/replay"
assert.Equal(
t, http.StatusMethodNotAllowed,
env.get(path, cookies).Code,
"a replay must not be reachable by GET",
)
assert.Equal(
t, http.StatusForbidden,
env.post(path, url.Values{}, cookies).Code,
"a replay POST without a CSRF token must be refused",
)
anon := env.post(path, url.Values{}, nil)
assert.Equal(t, http.StatusForbidden, anon.Code)
require.Equal(
t, int64(1), env.countDeliveries(t, wh.ID),
"no refused request may have created a delivery",
)
// The token and the action URL both come out of the rendered
// page, so a typo in either the route pattern or the template
// fails here.
logsPath := "/source/" + wh.ID + "/logs"
token, cookies := env.csrfFrom(t, logsPath, cookies)
page := env.get(logsPath, cookies)
require.Equal(t, http.StatusOK, page.Code)
action := regexp.MustCompile(
`action="(/source/[^"]+/replay)"`,
).FindStringSubmatch(page.Body.String())
require.Len(
t, action, 2,
"a finished delivery should render a replay form",
)
form := url.Values{}
form.Set("csrf_token", token)
w := env.post(
html.UnescapeString(action[1]), form, cookies,
)
require.Equal(t, http.StatusSeeOther, w.Code)
assert.Equal(
t, logsPath+"?replay=queued",
w.Header().Get("Location"),
)
assert.Equal(
t, int64(2), env.countDeliveries(t, wh.ID),
"the replay appends a delivery",
)
}
// metricsConfig is a Config differing from the routing default only // metricsConfig is a Config differing from the routing default only
// in the two /metrics credentials. // in the two /metrics credentials.
func metricsConfig( func metricsConfig(

File diff suppressed because one or more lines are too long

View File

@@ -12,6 +12,10 @@
</div> </div>
</div> </div>
{{if .ReplayMessage}}
<div class="{{if .ReplayQueued}}alert-success{{else}}alert-error{{end}}">{{.ReplayMessage}}</div>
{{end}}
<div class="card"> <div class="card">
<div class="divide-y divide-gray-100"> <div class="divide-y divide-gray-100">
{{range .Events}} {{range .Events}}
@@ -40,6 +44,68 @@
{{if .BodyTruncated}} {{if .BodyTruncated}}
<p class="mt-2 text-xs text-gray-500">Body truncated for display: showing {{.BodyShownBytes}} of {{.BodyBytes}} bytes. The stored body is unchanged &mdash; <a href="/source/{{$.Webhook.ID}}/logs/{{.ID}}/body" class="text-primary-600 hover:text-primary-700 underline">download the full body</a>.</p> <p class="mt-2 text-xs text-gray-500">Body truncated for display: showing {{.BodyShownBytes}} of {{.BodyBytes}} bytes. The stored body is unchanged &mdash; <a href="/source/{{$.Webhook.ID}}/logs/{{.ID}}/body" class="text-primary-600 hover:text-primary-700 underline">download the full body</a>.</p>
{{end}} {{end}}
{{if .Deliveries}}
<div class="mt-4 border-t border-gray-200 pt-3">
<h3 class="text-xs font-medium uppercase tracking-wide text-gray-500">Deliveries</h3>
<div class="mt-2 divide-y divide-gray-200">
{{range .Deliveries}}
<div class="py-2" x-data="{ attempts: false }">
<div class="flex items-center justify-between cursor-pointer" @click="attempts = !attempts">
<div class="flex items-center gap-3">
<span class="text-sm text-gray-700">{{.Target.Name}}</span>
<span class="text-xs {{if eq .Status "delivered"}}text-green-600{{else if eq .Status "failed"}}text-red-600{{else if eq .Status "retrying"}}text-yellow-600{{else}}text-gray-400{{end}}">{{.Status}}</span>
</div>
<div class="flex items-center gap-3">
{{if .Status.Terminal}}
<form method="POST" action="/source/{{$.Webhook.ID}}/deliveries/{{.ID}}/replay" class="inline" @click.stop>
<input type="hidden" name="csrf_token" value="{{$.CSRFToken}}">
<input type="hidden" name="page" value="{{$.Page}}">
<button type="submit" class="text-xs text-primary-600 hover:text-primary-700" title="Send this event to the target again">Replay</button>
</form>
{{end}}
<span class="text-xs text-gray-400">{{.AttemptCount}} attempt{{if ne .AttemptCount 1}}s{{end}}</span>
<svg class="w-3 h-3 text-gray-400 transition-transform" :class="{ 'rotate-180': attempts }" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M19 9l-7 7-7-7"/>
</svg>
</div>
</div>
<div x-show="attempts" x-cloak class="mt-2 space-y-2">
{{if .AttemptsOmitted}}
<p class="text-xs text-gray-500">{{.AttemptsOmitted}} attempt{{if ne .AttemptsOmitted 1}}s{{end}} omitted between the first and last shown.</p>
{{end}}
{{range .Results}}
<div class="rounded-md bg-white border border-gray-200 p-2">
<div class="flex flex-wrap items-center gap-3 text-xs">
<span class="text-gray-500">Attempt {{.AttemptNum}}</span>
<span class="{{if .Success}}text-green-600{{else}}text-red-600{{end}}">{{if .Success}}success{{else}}failure{{end}}</span>
<span class="text-gray-500">Status: {{if .HasStatusCode}}{{.StatusCode}}{{else}}&mdash; (no response){{end}}</span>
<span class="text-gray-500">Duration: {{.DurationMS}} ms</span>
</div>
{{if .Error}}
<p class="mt-2 text-xs text-red-700 break-all">Error: {{.Error}}</p>
{{end}}
{{if .ResponseBody}}
<pre class="mt-2 text-xs text-gray-700 overflow-x-auto whitespace-pre-wrap break-all">{{.ResponseBody}}</pre>
{{end}}
{{if .ResponseTruncated}}
{{if .ResponseSizeKnown}}
<p class="mt-1 text-xs text-gray-500">Response truncated for display: showing {{.ResponseShownBytes}} of {{.ResponseBytes}} bytes.</p>
{{else}}
<p class="mt-1 text-xs text-gray-500">Showing {{.ResponseShownBytes}} of the {{.ResponseBytes}} recorded bytes. The response reached the recording limit, so the remote may have sent more that was never stored.</p>
{{end}}
{{end}}
</div>
{{else}}
<p class="text-xs text-gray-500">No attempts recorded yet.</p>
{{end}}
</div>
</div>
{{end}}
</div>
</div>
{{end}}
</div> </div>
</div> </div>
{{else}} {{else}}