Compare commits
4 Commits
issue-115-
...
ea92c616c2
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ea92c616c2 | ||
| 543005c0c2 | |||
| 9bfd033a29 | |||
| fd6397154a |
@@ -93,6 +93,7 @@ TTY detection, and security headers are always applied.
|
|||||||
| `METRICS_USERNAME` | Basic auth username for `/metrics` | `""` |
|
| `METRICS_USERNAME` | Basic auth username for `/metrics` | `""` |
|
||||||
| `METRICS_PASSWORD` | Basic auth password for `/metrics` | `""` |
|
| `METRICS_PASSWORD` | Basic auth password for `/metrics` | `""` |
|
||||||
| `SENTRY_DSN` | Sentry error reporting DSN | `""` |
|
| `SENTRY_DSN` | Sentry error reporting DSN | `""` |
|
||||||
|
| `RETENTION_SWEEP_INTERVAL` | How often the retention reaper and archive sweeper run (Go duration) | `1h` |
|
||||||
| `SESSION_IDLE_TIMEOUT` | Idle session timeout (Go duration) | `24h` |
|
| `SESSION_IDLE_TIMEOUT` | Idle session timeout (Go duration) | `24h` |
|
||||||
| `RECEIVER_RATE_LIMIT` | Receiver requests/minute per IP per entrypoint | `120` |
|
| `RECEIVER_RATE_LIMIT` | Receiver requests/minute per IP per entrypoint | `120` |
|
||||||
| `TRUSTED_PROXIES` | CIDRs whose forwarded headers are trusted | `""` (none) |
|
| `TRUSTED_PROXIES` | CIDRs whose forwarded headers are trusted | `""` (none) |
|
||||||
@@ -260,9 +261,10 @@ webhooker solves this by acting as a durable intermediary:
|
|||||||
targets simultaneously. This enables patterns like forwarding a
|
targets simultaneously. This enables patterns like forwarding a
|
||||||
GitHub webhook to both a deployment service and a Slack channel.
|
GitHub webhook to both a deployment service and a Slack channel.
|
||||||
|
|
||||||
5. **Replay** — Stored events can be manually redelivered for debugging
|
5. **Replay** (not yet implemented) — Every received event is stored in
|
||||||
or testing, without requiring the original sender to fire the webhook
|
full, which is what manual redelivery for debugging or testing will
|
||||||
again.
|
be built on. No redelivery exists today, in the web UI or the API;
|
||||||
|
see [TODO.md](TODO.md).
|
||||||
|
|
||||||
### Use Cases
|
### Use Cases
|
||||||
|
|
||||||
@@ -272,6 +274,7 @@ webhooker solves this by acting as a durable intermediary:
|
|||||||
size, and delivery performance
|
size, and delivery performance
|
||||||
- **Debugging** and introspection of webhook payloads in the web UI
|
- **Debugging** and introspection of webhook payloads in the web UI
|
||||||
- **Replay** of webhook events for application testing and development
|
- **Replay** of webhook events for application testing and development
|
||||||
|
(planned; not yet implemented)
|
||||||
- **Fan-out** delivery of a single webhook to multiple downstream
|
- **Fan-out** delivery of a single webhook to multiple downstream
|
||||||
targets
|
targets
|
||||||
- **High-availability ingestion** for delivery to less reliable backend
|
- **High-availability ingestion** for delivery to less reliable backend
|
||||||
|
|||||||
90
TODO.md
90
TODO.md
@@ -1,35 +1,94 @@
|
|||||||
# Workflow
|
# Workflow
|
||||||
|
|
||||||
* branch (from `main`)
|
One issue per unit of work, one branch and one PR per issue:
|
||||||
* do the work in Next Step
|
|
||||||
* move Next Step to the top of Completed Steps
|
* ensure a tracked issue exists with a definition of done
|
||||||
* move the top item of Future Steps into Next Step
|
* branch from `next` (never from `main`)
|
||||||
* commit (`TODO.md` changes in the same commit as the work)
|
* do the work; open a PR based on `next` (never on `main`)
|
||||||
* merge to `main` if the branch is not protected, otherwise open a PR
|
* pass an independent review, then the manager squash-merges into `next`
|
||||||
* push
|
* push; nothing stays local-only
|
||||||
|
|
||||||
|
`next` is the branch for the next milestone and must stay green and
|
||||||
|
mergeable to `main` without notice. One `next` -> `main` PR accumulates
|
||||||
|
the milestone; releases are cut from `main` separately.
|
||||||
|
|
||||||
|
Issue branches do NOT touch this file — the manager maintains it on
|
||||||
|
`next`. Every branch editing `TODO.md` conflicts with every other
|
||||||
|
(#112).
|
||||||
|
|
||||||
# Status
|
# Status
|
||||||
|
|
||||||
pre-1.0. No git tags exist. main (4f5ecb1) is a working webhook proxy
|
pre-1.0. No git tags exist. `main` (4f5ecb1) is a working webhook proxy
|
||||||
with auth, CSRF/SSRF protections, login rate limiting, Slack target,
|
with auth, CSRF/SSRF protections, login rate limiting, Slack target,
|
||||||
event retention (#63), the database archiving target (#43), the admin
|
event retention (#63), the database archiving target (#43), the admin
|
||||||
password change flow (#65), policy compliance (#6), pinned lint tooling
|
password change flow (#65), policy compliance (#6), pinned lint tooling
|
||||||
(#55), and fail-loud configuration parsing (#80). Note: TODO.md was
|
(#55), and fail-loud configuration parsing (#80).
|
||||||
deliberately deleted from this repo in f9a9569 (2026-03-01, #6); its
|
|
||||||
content was folded into the README TODO section, which this draft
|
`next` (9bfd033) holds the completed 1.0.0 milestone: every issue in it
|
||||||
reconstructs as of 2026-07-06.
|
is closed, and it is verified green by cache-defeated container runs
|
||||||
|
rather than by the CI badge, which can pass without executing anything
|
||||||
|
(#119). Note: TODO.md was deliberately deleted from this repo in f9a9569
|
||||||
|
(2026-03-01, #6); its content was folded into the README TODO section,
|
||||||
|
which this draft reconstructs as of 2026-07-06.
|
||||||
|
|
||||||
# Next Step
|
# Next Step
|
||||||
|
|
||||||
Manual event redelivery from the web UI (replay is a core promised
|
Tag 1.0.0 from `main` once the milestone PR merges, then repair the CI
|
||||||
capability in the README rationale).
|
gate (#119) before the next cycle's work lands — a gate that can report
|
||||||
|
success without running is the one thing every other guarantee here
|
||||||
|
rests on.
|
||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-08-12 Bound the `X-Forwarded-For` scan's allocation to the hop
|
||||||
|
cap: the reverse walk cuts entries with `strings.LastIndexByte`
|
||||||
|
instead of joining and splitting, so a 1 MB header allocates 16 bytes
|
||||||
|
rather than 1.6 MB per request on the unauthenticated receiver.
|
||||||
|
Semantics proven unchanged by differential testing against the
|
||||||
|
previous implementation (#133)
|
||||||
|
- 2026-08-12 Cap the `X-Forwarded-For` hop walk at 64 entries, so an
|
||||||
|
attacker-supplied chain cannot burn unbounded CPU in the rate-limit
|
||||||
|
key function; running off the end falls back to the peer address
|
||||||
|
(#124)
|
||||||
|
- 2026-08-12 Gate forwarded-header trust behind a `TRUSTED_PROXIES` CIDR
|
||||||
|
list: all three rate limiters key on the connection's own address
|
||||||
|
unless the direct peer is a configured proxy, in which case
|
||||||
|
`X-Forwarded-For` is walked right to left for the first non-proxy hop.
|
||||||
|
Default trusts nothing, and a set-but-unparseable value aborts
|
||||||
|
startup. Before this, any client could mint a fresh bucket or drain
|
||||||
|
another's by rotating a spoofed header (#88)
|
||||||
- 2026-08-11 Web UI cleanup: nav terminology unified on Webhooks, the
|
- 2026-08-11 Web UI cleanup: nav terminology unified on Webhooks, the
|
||||||
Profile settings placeholder removed, a progressive-enhancement copy
|
Profile settings placeholder removed, a progressive-enhancement copy
|
||||||
button for the entrypoint URL, and retention form copy that states the
|
button for the entrypoint URL, and retention form copy that states the
|
||||||
actual policy (deletion by the reaper, 0 retains forever) (#57)
|
actual policy (deletion by the reaper, 0 retains forever) (#57)
|
||||||
|
- 2026-08-11 Mask the webhook credential in delivery errors and logs:
|
||||||
|
Go embeds the request URL in `*url.Error`, so every transport failure
|
||||||
|
persisted the full Slack webhook URL into the per-webhook event
|
||||||
|
database via `DeliveryResult.Error`, a field a future REST API would
|
||||||
|
have served. `maskURLError` drops path, query and userinfo while
|
||||||
|
preserving the wrapped cause, so `errors.Is`/`As` and `Timeout()`
|
||||||
|
still work and DNS, TLS and timeout failures still read differently
|
||||||
|
(#118)
|
||||||
|
- 2026-08-11 Rate-limit the public webhook receiver endpoint
|
||||||
|
(`RECEIVER_RATE_LIMIT`, default 120/min), keyed on client IP plus
|
||||||
|
entrypoint path so one entrypoint cannot exhaust another's budget;
|
||||||
|
over-limit requests get 429 with `Retry-After`. It was the one
|
||||||
|
unauthenticated, internet-facing endpoint with no limit at all (#64)
|
||||||
|
- 2026-08-11 Enforce the body size limit before CSRF parses the form:
|
||||||
|
`MaxBodySize` is now first in all four form-parsing route groups, so
|
||||||
|
an oversized request is rejected with 413 instead of being read in
|
||||||
|
full by the CSRF middleware before any cap applied (#90)
|
||||||
|
- 2026-08-11 Mask target config on the source detail page, which
|
||||||
|
rendered the stored blob verbatim and so exposed the Slack
|
||||||
|
incoming-webhook URL — a bearer credential that cannot be revoked
|
||||||
|
per-holder. Config reaches the template only as a `TargetView` of
|
||||||
|
labelled fields, and header values are rendered as a count (#113)
|
||||||
|
- 2026-08-11 Allow `retention_days` of 0 to mean retain forever, via a
|
||||||
|
sentinel written in `BeforeSave` so the GORM column default cannot
|
||||||
|
win the race. Also bounds the reaper's cutoff arithmetic: day counts
|
||||||
|
above 106751 overflowed `time.Duration` and wrapped the cutoff into
|
||||||
|
the future, where every row matched and the sweep deleted everything
|
||||||
|
(#79)
|
||||||
- 2026-08-09 Inactivity-based session timeout: sliding idle expiry
|
- 2026-08-09 Inactivity-based session timeout: sliding idle expiry
|
||||||
(`SESSION_IDLE_TIMEOUT`, default `24h`) refreshed on authenticated
|
(`SESSION_IDLE_TIMEOUT`, default `24h`) refreshed on authenticated
|
||||||
requests, with the 7-day absolute cap kept as an independent
|
requests, with the 7-day absolute cap kept as an independent
|
||||||
@@ -84,6 +143,9 @@ capability in the README rationale).
|
|||||||
|
|
||||||
# Future Steps
|
# Future Steps
|
||||||
|
|
||||||
|
- Manual event redelivery from the web UI — the "Replay" capability the
|
||||||
|
README describes as planned. No redelivery code exists anywhere in the
|
||||||
|
tree; events are stored in full, which is all it would be built on
|
||||||
- Delivery status and retry management UI
|
- Delivery status and retry management UI
|
||||||
- Per-webhook rate limiting in the receiver handler (per-webhook config
|
- Per-webhook rate limiting in the receiver handler (per-webhook config
|
||||||
plus handler enforcement; global limits must not apply to receiver
|
plus handler enforcement; global limits must not apply to receiver
|
||||||
|
|||||||
@@ -106,12 +106,6 @@ func slackConfigFields(configJSON string) []ConfigField {
|
|||||||
// and its retry settings. Header values are not shown — they
|
// and its retry settings. Header values are not shown — they
|
||||||
// routinely carry authorization tokens — only how many are
|
// routinely carry authorization tokens — only how many are
|
||||||
// configured.
|
// configured.
|
||||||
//
|
|
||||||
// The destination is masked to scheme and host by the same
|
|
||||||
// rule the Slack target uses. An HTTP target's destination is
|
|
||||||
// commonly a Slack, Discord or Teams incoming-webhook endpoint
|
|
||||||
// whose path segments are the credential, and the field takes
|
|
||||||
// an arbitrary URL, so no segment can be assumed non-secret.
|
|
||||||
func httpConfigFields(t *database.Target) []ConfigField {
|
func httpConfigFields(t *database.Target) []ConfigField {
|
||||||
cfg, err := parseHTTPConfig(t.Config)
|
cfg, err := parseHTTPConfig(t.Config)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -120,7 +114,7 @@ func httpConfigFields(t *database.Target) []ConfigField {
|
|||||||
|
|
||||||
fields := []ConfigField{{
|
fields := []ConfigField{{
|
||||||
Label: "Destination URL",
|
Label: "Destination URL",
|
||||||
Value: MaskURL(cfg.URL),
|
Value: cfg.URL,
|
||||||
}}
|
}}
|
||||||
|
|
||||||
if cfg.Timeout > 0 {
|
if cfg.Timeout > 0 {
|
||||||
|
|||||||
@@ -19,7 +19,6 @@ const (
|
|||||||
|
|
||||||
viewExampleOrigin = "https://example.com"
|
viewExampleOrigin = "https://example.com"
|
||||||
viewExampleHook = viewExampleOrigin + "/hook"
|
viewExampleHook = viewExampleOrigin + "/hook"
|
||||||
viewMaskedOrigin = viewExampleOrigin + "/..."
|
|
||||||
viewUnavailable = "(unavailable)"
|
viewUnavailable = "(unavailable)"
|
||||||
viewExpiryNever = "never"
|
viewExpiryNever = "never"
|
||||||
)
|
)
|
||||||
@@ -163,7 +162,7 @@ func TestNewTargetViews_HTTP(t *testing.T) {
|
|||||||
assert.Equal(
|
assert.Equal(
|
||||||
t,
|
t,
|
||||||
map[string]string{
|
map[string]string{
|
||||||
"Destination URL": viewMaskedOrigin,
|
"Destination URL": viewExampleHook,
|
||||||
"Timeout": "30s",
|
"Timeout": "30s",
|
||||||
"Headers": "1 configured",
|
"Headers": "1 configured",
|
||||||
"Max Retries": "5",
|
"Max Retries": "5",
|
||||||
@@ -189,41 +188,13 @@ func TestNewTargetViews_HTTPFireAndForget(t *testing.T) {
|
|||||||
assert.Equal(
|
assert.Equal(
|
||||||
t,
|
t,
|
||||||
map[string]string{
|
map[string]string{
|
||||||
"Destination URL": viewMaskedOrigin,
|
"Destination URL": viewExampleHook,
|
||||||
"Max Retries": "0 (fire-and-forget)",
|
"Max Retries": "0 (fire-and-forget)",
|
||||||
},
|
},
|
||||||
fieldMap(view.Config),
|
fieldMap(view.Config),
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestNewTargetViews_HTTPMasksDestinationURL proves the rule
|
|
||||||
// holds for the http target too: an http destination is
|
|
||||||
// routinely an incoming-webhook endpoint whose path segments
|
|
||||||
// are the credential, so none of them is shown.
|
|
||||||
func TestNewTargetViews_HTTPMasksDestinationURL(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
view := viewFor(t, database.Target{
|
|
||||||
Type: database.TargetTypeHTTP,
|
|
||||||
Config: `{"url":"` + slackWebhookURL + `"}`,
|
|
||||||
})
|
|
||||||
|
|
||||||
fields := fieldMap(view.Config)
|
|
||||||
|
|
||||||
assert.Equal(
|
|
||||||
t,
|
|
||||||
"https://hooks.slack.com/...",
|
|
||||||
fields["Destination URL"],
|
|
||||||
)
|
|
||||||
|
|
||||||
for _, v := range fields {
|
|
||||||
assert.NotContains(t, v, slackSecretPath)
|
|
||||||
assert.NotContains(t, v, "T00000000")
|
|
||||||
assert.NotContains(t, v, "B00000000")
|
|
||||||
assert.NotContains(t, v, "XXXXXXXXXXXXXXXXXXXXXXXX")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestNewTargetViews_Database(t *testing.T) {
|
func TestNewTargetViews_Database(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -131,47 +131,6 @@ func TestHandleSourceDetail_MasksSlackWebhookURL(t *testing.T) {
|
|||||||
assert.Contains(t, body, "https://hooks.slack.com/...")
|
assert.Contains(t, body, "https://hooks.slack.com/...")
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestHandleSourceDetail_MasksHTTPDestinationURL is the
|
|
||||||
// regression test for the same leak reached through the http
|
|
||||||
// target: its destination is routinely an incoming-webhook
|
|
||||||
// endpoint whose path segments are the credential, so the
|
|
||||||
// rendered page must not contain them.
|
|
||||||
func TestHandleSourceDetail_MasksHTTPDestinationURL(
|
|
||||||
t *testing.T,
|
|
||||||
) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
var (
|
|
||||||
h *handlers.Handlers
|
|
||||||
sess *session.Session
|
|
||||||
db *database.Database
|
|
||||||
)
|
|
||||||
|
|
||||||
app := newTestApp(t, &h, &sess, &db)
|
|
||||||
app.RequireStart()
|
|
||||||
|
|
||||||
t.Cleanup(app.RequireStop)
|
|
||||||
|
|
||||||
wh := seedWebhook(t, db)
|
|
||||||
seedConfiguredTarget(
|
|
||||||
t, db, wh.ID,
|
|
||||||
database.TargetTypeHTTP,
|
|
||||||
`{"url":"`+slackWebhookURL+`"}`,
|
|
||||||
)
|
|
||||||
|
|
||||||
body := renderSourceDetailPage(t, h, sess, wh.ID)
|
|
||||||
|
|
||||||
assert.NotContains(t, body, slackSecretPath)
|
|
||||||
assert.NotContains(t, body, "T00000000")
|
|
||||||
assert.NotContains(t, body, "B00000000")
|
|
||||||
assert.NotContains(
|
|
||||||
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
|
|
||||||
)
|
|
||||||
|
|
||||||
assert.Contains(t, body, "Destination URL")
|
|
||||||
assert.Contains(t, body, "https://hooks.slack.com/...")
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestHandleSourceDetail_RendersNamedTargetFields proves the
|
// TestHandleSourceDetail_RendersNamedTargetFields proves the
|
||||||
// other target types render labelled fields rather than the
|
// other target types render labelled fields rather than the
|
||||||
// stored blob.
|
// stored blob.
|
||||||
@@ -213,7 +172,7 @@ func TestHandleSourceDetail_RendersNamedTargetFields(
|
|||||||
body := renderSourceDetailPage(t, h, sess, wh.ID)
|
body := renderSourceDetailPage(t, h, sess, wh.ID)
|
||||||
|
|
||||||
assert.Contains(t, body, "Destination URL")
|
assert.Contains(t, body, "Destination URL")
|
||||||
assert.Contains(t, body, "https://example.com/...")
|
assert.Contains(t, body, "https://example.com/hook")
|
||||||
assert.Contains(t, body, "Timeout")
|
assert.Contains(t, body, "Timeout")
|
||||||
assert.Contains(t, body, "1 configured")
|
assert.Contains(t, body, "1 configured")
|
||||||
assert.NotContains(t, body, "sekrit")
|
assert.NotContains(t, body, "sekrit")
|
||||||
|
|||||||
@@ -25,6 +25,11 @@ func IPFromHostPort(hp string) string {
|
|||||||
return ipFromHostPort(hp)
|
return ipFromHostPort(hp)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ClientKeyForTest exposes clientKey for testing.
|
||||||
|
func ClientKeyForTest(m *Middleware, r *http.Request) string {
|
||||||
|
return m.clientKey(r)
|
||||||
|
}
|
||||||
|
|
||||||
// IsClientTLS exposes isClientTLS for testing.
|
// IsClientTLS exposes isClientTLS for testing.
|
||||||
func IsClientTLS(r *http.Request) bool {
|
func IsClientTLS(r *http.Request) bool {
|
||||||
return isClientTLS(r)
|
return isClientTLS(r)
|
||||||
|
|||||||
@@ -32,6 +32,13 @@ const (
|
|||||||
// receiver rate limit. The configured limit is expressed in
|
// receiver rate limit. The configured limit is expressed in
|
||||||
// requests per minute.
|
// requests per minute.
|
||||||
receiverRateInterval = 1 * time.Minute
|
receiverRateInterval = 1 * time.Minute
|
||||||
|
|
||||||
|
// maxForwardedHops bounds how many X-Forwarded-For entries the
|
||||||
|
// chain walk examines. Real chains are one to three hops, but a
|
||||||
|
// client can pad the header up to MaxHeaderBytes, so without a
|
||||||
|
// bound every request pays a walk proportional to whatever the
|
||||||
|
// client sent.
|
||||||
|
maxForwardedHops = 64
|
||||||
)
|
)
|
||||||
|
|
||||||
// normalizeAddr strips the IPv4-in-IPv6 wrapper and any zone from
|
// normalizeAddr strips the IPv4-in-IPv6 wrapper and any zone from
|
||||||
@@ -70,14 +77,36 @@ func (m *Middleware) isTrustedProxy(addr netip.Addr) bool {
|
|||||||
// a trusted proxy is the client. A hop that cannot be read as a bare
|
// a trusted proxy is the client. A hop that cannot be read as a bare
|
||||||
// address ends the walk: past it the chain is not the shape assumed
|
// address ends the walk: past it the chain is not the shape assumed
|
||||||
// here, so the caller falls back to the peer address.
|
// here, so the caller falls back to the peer address.
|
||||||
|
//
|
||||||
|
// Only the last maxForwardedHops entries are examined. A longer chain
|
||||||
|
// is padding, and running out of hops falls back to the peer address
|
||||||
|
// the same way an unreadable hop does.
|
||||||
|
//
|
||||||
|
// The entries are cut off the right end of each header value in place
|
||||||
|
// rather than split out of it: the receiver is unauthenticated and a
|
||||||
|
// client can pad the header up to MaxHeaderBytes, so splitting would
|
||||||
|
// allocate in proportion to the padding (about 8 MB for a 1 MB
|
||||||
|
// header) before the cap could discard any of it. Multiple header
|
||||||
|
// values are walked in reverse for the same reason, since joining
|
||||||
|
// them copies the whole chain.
|
||||||
func (m *Middleware) forwardedClientAddr(
|
func (m *Middleware) forwardedClientAddr(
|
||||||
r *http.Request,
|
r *http.Request,
|
||||||
) (netip.Addr, bool) {
|
) (netip.Addr, bool) {
|
||||||
hops := strings.Split(
|
seen := 0
|
||||||
strings.Join(r.Header.Values("X-Forwarded-For"), ","), ",",
|
|
||||||
)
|
for _, value := range slices.Backward(
|
||||||
|
r.Header.Values("X-Forwarded-For"),
|
||||||
|
) {
|
||||||
|
for last := false; !last && seen < maxForwardedHops; seen++ {
|
||||||
|
hop := value
|
||||||
|
|
||||||
|
comma := strings.LastIndexByte(value, ',')
|
||||||
|
if comma < 0 {
|
||||||
|
last = true
|
||||||
|
} else {
|
||||||
|
hop, value = value[comma+1:], value[:comma]
|
||||||
|
}
|
||||||
|
|
||||||
for _, hop := range slices.Backward(hops) {
|
|
||||||
hop = strings.TrimSpace(hop)
|
hop = strings.TrimSpace(hop)
|
||||||
if hop == "" {
|
if hop == "" {
|
||||||
continue
|
continue
|
||||||
@@ -92,6 +121,7 @@ func (m *Middleware) forwardedClientAddr(
|
|||||||
return addr, true
|
return addr, true
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
return netip.Addr{}, false
|
return netip.Addr{}, false
|
||||||
}
|
}
|
||||||
@@ -113,8 +143,10 @@ func (m *Middleware) clientKey(r *http.Request) string {
|
|||||||
peer, err := netip.ParseAddr(ipFromHostPort(r.RemoteAddr))
|
peer, err := netip.ParseAddr(ipFromHostPort(r.RemoteAddr))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
// Not an address we can reason about; key on the raw
|
// Not an address we can reason about; key on the raw
|
||||||
// value rather than collapsing such peers into one
|
// value, the most specific identity left. On a
|
||||||
// shared bucket.
|
// Unix-socket listener every peer carries the same
|
||||||
|
// RemoteAddr and so shares one bucket, which is the
|
||||||
|
// fail-closed direction.
|
||||||
return r.RemoteAddr
|
return r.RemoteAddr
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -8,7 +8,10 @@ import (
|
|||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"os"
|
"os"
|
||||||
|
"runtime"
|
||||||
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
@@ -568,6 +571,105 @@ func TestRateLimitKey_ChainWalkSkipsClientPrepended(t *testing.T) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestRateLimitKey_LongChainCapsWalkAndFallsBackToPeer covers the
|
||||||
|
// hop-walk cap. A client behind the trusted proxy can pad
|
||||||
|
// X-Forwarded-For with tens of thousands of trusted-looking hops,
|
||||||
|
// which costs a walk proportional to the padding and, once the walk
|
||||||
|
// runs off the left end of the chain, reaches the entry the client
|
||||||
|
// put there. Capping the walk stops both: the key falls back to the
|
||||||
|
// peer address, so rotating the head of the chain mints no bucket,
|
||||||
|
// and the run does not scale with the chain length.
|
||||||
|
func TestRateLimitKey_LongChainCapsWalkAndFallsBackToPeer(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
// 50k hops is roughly 0.9 MB, within the default
|
||||||
|
// MaxHeaderBytes.
|
||||||
|
const hops = 50000
|
||||||
|
|
||||||
|
padding := strings.Repeat(", 10.0.0.2", hops-1)
|
||||||
|
|
||||||
|
start := time.Now()
|
||||||
|
|
||||||
|
assertSharedBucket(
|
||||||
|
t, trustedProxies("10.0.0.0/8"), "10.0.0.1:44444",
|
||||||
|
func(i int) map[string]string {
|
||||||
|
return map[string]string{
|
||||||
|
headerXFF: fmt.Sprintf("9.9.9.%d%s", i+1, padding),
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"a padded X-Forwarded-For chain must fall back to the "+
|
||||||
|
"peer address, not reach the client-controlled entry "+
|
||||||
|
"at the head of the chain",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.Less(
|
||||||
|
t, time.Since(start), 2*time.Second,
|
||||||
|
"the capped walk must not scale with the chain length",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestRateLimitKey_LongChainAllocationIsBounded is the allocation
|
||||||
|
// half of the hop cap. Capping the walk still left every request
|
||||||
|
// paying for the whole header the client sent, because the chain was
|
||||||
|
// split before it was capped: about 8 MB of []string for the 1 MB a
|
||||||
|
// default MaxHeaderBytes allows, on the unauthenticated receiver.
|
||||||
|
//
|
||||||
|
// Bytes are the measurement, not allocation count: strings.Split of a
|
||||||
|
// 1 MB chain is a single allocation, so testing.AllocsPerRun scores
|
||||||
|
// it as cheap. The test is deliberately sequential — it reads
|
||||||
|
// process-wide counters, and Go runs this package's parallel tests
|
||||||
|
// only after the sequential ones finish.
|
||||||
|
//
|
||||||
|
//nolint:paralleltest // reads process-wide allocation counters
|
||||||
|
func TestRateLimitKey_LongChainAllocationIsBounded(t *testing.T) {
|
||||||
|
// 100k hops of ", 10.0.0.2" is roughly 1 MB.
|
||||||
|
const (
|
||||||
|
hops = 100000
|
||||||
|
iterations = 50
|
||||||
|
maxBytesPerCall = 4096
|
||||||
|
)
|
||||||
|
|
||||||
|
m := rateLimitMiddleware(t, &config.Config{
|
||||||
|
TrustedProxies: trustedProxies("10.0.0.0/8"),
|
||||||
|
})
|
||||||
|
|
||||||
|
req := httptest.NewRequestWithContext(
|
||||||
|
context.Background(), http.MethodPost, loginPath, nil,
|
||||||
|
)
|
||||||
|
req.RemoteAddr = "10.0.0.1:44444"
|
||||||
|
req.Header.Set(
|
||||||
|
headerXFF, "9.9.9.9"+strings.Repeat(", 10.0.0.2", hops),
|
||||||
|
)
|
||||||
|
|
||||||
|
var before, after runtime.MemStats
|
||||||
|
|
||||||
|
var key string
|
||||||
|
|
||||||
|
runtime.ReadMemStats(&before)
|
||||||
|
|
||||||
|
for range iterations {
|
||||||
|
key = middleware.ClientKeyForTest(m, req)
|
||||||
|
}
|
||||||
|
|
||||||
|
runtime.ReadMemStats(&after)
|
||||||
|
|
||||||
|
perCall := (after.TotalAlloc - before.TotalAlloc) / iterations
|
||||||
|
|
||||||
|
assert.Less(
|
||||||
|
t, perCall, uint64(maxBytesPerCall),
|
||||||
|
"a %d-byte X-Forwarded-For must not allocate in proportion "+
|
||||||
|
"to its length, but cost %d bytes per call",
|
||||||
|
len(req.Header.Get(headerXFF)), perCall,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.Equal(
|
||||||
|
t, "10.0.0.1", key,
|
||||||
|
"the padded chain must still fall back to the peer address",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
// TestReceiverRateLimit_IgnoresForwardedFromUntrustedPeer proves
|
// TestReceiverRateLimit_IgnoresForwardedFromUntrustedPeer proves
|
||||||
// the receiver limiter uses the same gated key function as the
|
// the receiver limiter uses the same gated key function as the
|
||||||
// POST limiters.
|
// POST limiters.
|
||||||
|
|||||||
@@ -1,5 +1,4 @@
|
|||||||
// Webhooker client-side JavaScript
|
// Webhooker client-side JavaScript
|
||||||
console.log("Webhooker loaded");
|
|
||||||
|
|
||||||
// Copy-to-clipboard, as progressive enhancement.
|
// Copy-to-clipboard, as progressive enhancement.
|
||||||
//
|
//
|
||||||
|
|||||||
Reference in New Issue
Block a user