Compare commits

3 Commits

Author SHA1 Message Date
76a6518282 Add an egress CIDR allowlist to the SSRF guard (closes #204)
All checks were successful
check / check (push) Successful in 3m28s
The SSRF blocklist had no escape hatch, so the thing webhooker is
mostly for — taking a public webhook and forwarding it to something
on your own network — could not be configured at all. Every private
address, Docker sibling and loopback service was permanently
unreachable as a delivery destination.

ALLOWED_EGRESS_CIDRS (default empty) names blocks that delivery
targets may reach despite the default blocklist. It is an allowlist
and only ever adds destinations: there is no boolean, and no value
disables SSRF protection wholesale. Empty, the guard behaves exactly
as before.

A fixed set of addresses is refused before the allowlist is
consulted, so no supplied CIDR opens one — not the exact address,
not a supernet, not 0.0.0.0/0 or ::/0. It is the two link-local
blocks (169.254.0.0/16, fe80::/10) plus host routes for the cloud
metadata endpoints that sit outside them: AWS's IPv6 IMDS at
fd00:ec2::254, which lives in ordinary ULA space, and Alibaba's
100.100.100.200, which lives in CGNAT. Allowlisting fd00::/8 or
100.64.0.0/10 (Tailscale's range) is an ordinary thing for an
operator to do and must not reopen instance-credential theft. The
IPv4-compatible (::a9fe:a9fe) and NAT64 (64:ff9b::a9fe:a9fe)
spellings of 169.254.169.254 are listed too, because To4() does not
normalise them into the link-local block the way it does the
IPv4-mapped form. Reaching any of these is credential theft rather
than delivery to an internal service.

The policy now lives in one function, Guard.checkIP, which both
target-creation validation and the delivery dialer call. The two
paths previously decided separately, which is how they came to
disagree about a destination. The guard is built once from config
and injected via fx into both the handlers and the delivery engine,
so there is a single instance and a single answer.

A set-but-unparseable value aborts startup naming the variable,
reusing the existing envPrefixList parser. A non-empty list is
logged at startup with the blocks spelled out, not counted, so the
hole is visible in the log of any deployment that has one.

Tests: an allowlisted loopback CIDR both validates and delivers to a
live server (and the same URL still fails without the allowlist); a
private address outside the listed block stays refused on both
paths; every unconditionally blocked address stays refused on both
paths under an allowlist that covers it, and the set itself is
pinned entry by entry; public addresses are unaffected either way;
and config coverage for parsing, startup abort, and the warning's
contents.
2026-08-20 04:55:00 +00:00
a13e5b7ded Shut down the app when the listener fails (closes #200) (#218)
All checks were successful
check / check (push) Successful in 3m48s
2026-08-20 06:42:36 +02:00
bb30b3ad64 Fail loudly on half-set metrics auth credentials (closes #205) (#216)
Some checks failed
check / check (push) Superseded by a newer commit; never tested
2026-08-20 06:30:23 +02:00
20 changed files with 1549 additions and 93 deletions

119
README.md
View File

@@ -107,13 +107,105 @@ TTY detection, and security headers are always applied.
| `DATA_DIR` | Directory for all SQLite databases | `/var/lib/webhooker` |
| `DEBUG` | Enable debug logging | `false` |
| `MAINTENANCE_MODE` | Report `maintenanceMode: true` in the healthcheck JSON. It does not change how any request is served — no maintenance page exists | `false` |
| `METRICS_USERNAME` | Basic auth username for `/metrics` | `""` |
| `METRICS_PASSWORD` | Basic auth password for `/metrics` | `""` |
| `METRICS_USERNAME` | Basic auth username for `/metrics`. Must be set together with `METRICS_PASSWORD`; one without the other fails startup | `""` |
| `METRICS_PASSWORD` | Basic auth password for `/metrics`. Must be set together with `METRICS_USERNAME`; one without the other fails startup | `""` |
| `SENTRY_DSN` | Sentry error reporting DSN | `""` |
| `RETENTION_SWEEP_INTERVAL` | How often the retention reaper and archive sweeper run (Go duration, must be positive) | `1h` |
| `SESSION_IDLE_TIMEOUT` | Idle session timeout (Go duration) | `24h` |
| `RECEIVER_RATE_LIMIT` | Receiver requests/minute per IP per entrypoint (10x that per IP across the route) | `120` |
| `TRUSTED_PROXIES` | CIDRs whose forwarded headers are trusted (unset: all clients behind a proxy share one rate-limit bucket; a correct login password is never throttled either way) | `""` (none) |
| `ALLOWED_EGRESS_CIDRS` | CIDRs that delivery targets may reach despite the SSRF blocklist. Read [Allowing egress to your own network](#allowing-egress-to-your-own-network) before setting it | `""` (none) |
#### Allowing egress to your own network
By default every delivery target must resolve to a public address.
The private and reserved ranges — RFC 1918, loopback, CGNAT,
link-local and the rest — are refused, which stops a target from being
used to make webhooker probe the network it sits in.
That default is also inconvenient for the thing webhooker is mostly
for: taking a public webhook and forwarding it to something on your own
network. A container on the same Docker network, a box on `10.x`, a
service on `127.0.0.1` — all refused, until you name them.
`ALLOWED_EGRESS_CIDRS` is a comma-separated list of CIDR blocks (a bare
address such as `10.0.0.7` is accepted and treated as a single host),
for example `10.0.0.0/8, 172.17.0.0/16`. Addresses inside those blocks
become valid delivery destinations. Everything outside them keeps the
default answer, so this only ever adds destinations — it never removes
any, and it cannot narrow what was already reachable.
**The risk, plainly.** Each block you list is a network that anyone who
can create a delivery target can now make this process issue requests
into, and read the response body back out of via the delivery log. That
is server-side request forgery, deliberately enabled and scoped by you.
A webhooker admin account is therefore as trusted as the narrowest
thing on those networks: an unauthenticated admin panel, a database
listening without a password, or an internal API that trusts its
network position is reachable through it. List the smallest blocks that
cover the destinations you actually deliver to — prefer
`10.1.2.3/32` over `10.0.0.0/8` — and never list a block wider than the
network you are willing to expose.
Listing `0.0.0.0/0` or `::/0` opens **every** other private and
reserved range at once — loopback, RFC 1918, CGNAT, ULA, the lot. It is
a functional off switch for everything except the addresses listed as
unconditionally blocked below, and it makes any delivery target a probe
into your entire network and this host's own loopback services. Do not
list it.
Two things this setting cannot do:
- **It cannot turn the guard off.** There is no boolean, and no value
that disables SSRF protection wholesale. The guard is always on and
the list is always an allowlist; an empty list (the default) means
every private and reserved range stays refused. Note that
`0.0.0.0/0` gets you most of the way there anyway, per above.
- **It cannot open link-local or a known cloud metadata endpoint.**
These stay blocked no matter what you list, including when you list
them outright or list a supernet such as `0.0.0.0/0`, `::/0`,
`fd00::/8` or `100.64.0.0/10`:
| Blocked unconditionally | What it is |
| ----------------------- | ---------- |
| `169.254.0.0/16` | IPv4 link-local, carrying `169.254.169.254` (AWS, GCP, Azure and others) |
| `fe80::/10` | IPv6 link-local |
| `fd00:ec2::254/128` | AWS IPv6 IMDS, a host route inside ULA space |
| `100.100.100.200/32` | Alibaba Cloud metadata, a host route inside CGNAT |
| `::a9fe:a9fe/128` | `169.254.169.254` as an IPv4-compatible IPv6 address |
| `64:ff9b::a9fe:a9fe/128` | `169.254.169.254` behind the NAT64 well-known prefix |
The IPv4-mapped form `::ffff:169.254.169.254` is covered by the
`169.254.0.0/16` entry. Reaching any of these is credential theft
rather than delivery to an internal service. The two host routes are
single addresses, so blocking them costs you nothing else on the ULA
or CGNAT networks around them. This list is not exhaustive of every
cloud's metadata address — if yours is not here, do not allowlist the
block that contains it.
The list is applied at one place in the code, which both target
creation and delivery consult, so a URL that the target form accepts is
one that delivery will actually attempt — the two cannot disagree.
Delivery re-resolves and re-checks the destination at dial time, so a
hostname that resolves to an allowed address during validation and a
different one later (DNS rebinding) is still refused unless the new
address is also allowed.
A set but unparseable value aborts startup. When the list is non-empty
webhooker logs it at startup, blocks and all, so the hole is visible in
the log of any deployment that has one.
#### Metrics credentials
`METRICS_USERNAME` and `METRICS_PASSWORD` are set together or not at
all. With both set, `/metrics` is served behind basic auth. With
neither set, the route is not registered and returns 404. With one set
and the other empty or unset, the process refuses to start and exits
non-zero with an error naming both variables — mounting the endpoint
on the username alone would publish it behind a password that is the
empty string, and quietly withholding it would deny an endpoint that
was asked for. The `hasMetricsAuth` field in the startup log and the
existence of the route are the same value, so they cannot disagree.
#### Trusted proxies
@@ -226,8 +318,9 @@ additionally be a number in the range 165535,
`RECEIVER_RATE_LIMIT` must be at least 1,
`RETENTION_SWEEP_INTERVAL` must be greater than zero (it is a ticker
period, so `0s` or a negative value would crash the reaper after
startup), and every entry in `TRUSTED_PROXIES` must be a CIDR block or
a bare IP address. `SESSION_IDLE_TIMEOUT` is the exception: a
startup), and every entry in `TRUSTED_PROXIES` and
`ALLOWED_EGRESS_CIDRS` must be a CIDR block or a bare IP address.
`SESSION_IDLE_TIMEOUT` is the exception: a
non-positive value there means idle expiry is disabled, not invalid.
Boolean variables (`DEBUG`, `MAINTENANCE_MODE`) accept exactly the
@@ -1682,7 +1775,7 @@ abuse limit later; they are tracked as future work.
| Method | Path | Description |
| ------ | ---------- | ----------- |
| `GET` | `/metrics` | Prometheus metrics, behind basic auth. The route is registered only when `METRICS_USERNAME` is set; otherwise it does not exist and returns 404 |
| `GET` | `/metrics` | Prometheus metrics, behind basic auth. The route is registered only when `METRICS_USERNAME` and `METRICS_PASSWORD` are both set; with neither set it does not exist and returns 404, and with only one set the process refuses to start |
#### API (Planned)
@@ -1837,7 +1930,8 @@ Applied to all routes in this order:
Permissions-Policy)
3. **Logging** — Structured request logging (method, URL, status,
latency, remote IP, user agent, request ID)
4. **Metrics** — Prometheus HTTP metrics (if `METRICS_USERNAME` is set)
4. **Metrics** — Prometheus HTTP metrics (if `METRICS_USERNAME` and
`METRICS_PASSWORD` are both set)
5. **CORS** — Cross-origin resource sharing headers
6. **Timeout** — 60-second request timeout
7. **Recoverer** — Panic recovery: one `ERROR` record through
@@ -1869,8 +1963,9 @@ being read and without reaching CSRF, the route group's remaining
middleware, or the handler. It is not rejected before *any* other
middleware, though: the global entries listed above all run first, so
such a request is still logged and given the security headers — and
counted in the metrics, on a deployment where `METRICS_USERNAME` is
set and the Metrics middleware is therefore registered at all. The
counted in the metrics, on a deployment where the `/metrics`
credentials are set and the Metrics middleware is therefore registered
at all. The
rejection itself is logged at `WARN` with the method, path and
declared length. A chunked request, or
one that lies about its length, is hard-capped by
@@ -1920,7 +2015,13 @@ check, see [The login endpoint](#the-login-endpoint).
ranges (RFC 1918, loopback, link-local, cloud metadata) are blocked
both at target creation time (URL validation) and at delivery time
(custom HTTP transport with SSRF-safe dialer that validates resolved
IPs before connecting, preventing DNS rebinding attacks)
IPs before connecting, preventing DNS rebinding attacks). Both paths
route through a single decision function, so they cannot disagree
about a destination. An operator can permit specific blocks with
[`ALLOWED_EGRESS_CIDRS`](#allowing-egress-to-your-own-network); the
guard cannot be switched off, and link-local plus the known cloud
metadata endpoints stay blocked whatever is listed — though listing
`0.0.0.0/0` or `::/0` does open every other private range
- **Login limiting is inverted, deliberately.** The login `POST` has
no pre-emptive rate limiter in front of it. Credentials are
verified first and only a _failed_ attempt spends budget, so a

View File

@@ -75,6 +75,10 @@ func newApp() *fx.App {
session.New,
handlers.New,
middleware.New,
// The one SSRF guard both target-creation validation
// and the delivery dialer consult, so they cannot
// disagree about a destination.
delivery.NewGuard,
delivery.New,
delivery.NewArchiveSweeper,
// Wire *delivery.Engine as delivery.Notifier so the

View File

@@ -71,6 +71,16 @@ var ErrInvalidPort = errors.New("invalid port")
// nor a bare IP address.
var ErrInvalidCIDR = errors.New("invalid CIDR")
// ErrIncompleteMetricsAuth is returned when exactly one of
// METRICS_USERNAME and METRICS_PASSWORD carries a value. Neither
// fallback is acceptable: serving /metrics on the username alone
// publishes an endpoint whose password is the empty string, and
// silently leaving it unmounted withholds an endpoint the operator
// asked for. Half-set is a configuration error, so startup fails.
var ErrIncompleteMetricsAuth = errors.New(
"incomplete metrics credentials",
)
//nolint:revive // ConfigParams is a standard fx naming convention.
type ConfigParams struct {
fx.In
@@ -114,6 +124,17 @@ type Config struct {
// clients.
TrustedProxies []netip.Prefix
// AllowedEgressCIDRs is the set of networks a delivery target
// may reach even though the SSRF guard's default blocklist
// covers them. It is empty unless ALLOWED_EGRESS_CIDRS is set,
// and empty means every private/reserved range stays refused.
//
// This only ever adds destinations to what the guard would
// otherwise refuse. The guard itself is always on: there is no
// setting that disables SSRF protection, and link-local stays
// blocked no matter what is listed here.
AllowedEgressCIDRs []netip.Prefix
params *ConfigParams
log *slog.Logger
}
@@ -128,6 +149,21 @@ func (c *Config) IsProd() bool {
return c.Environment == EnvironmentProd
}
// MetricsAuthEnabled reports whether /metrics is served behind basic
// auth. It is the only answer to that question in the codebase: the
// route mount, the Prometheus recording middleware and the startup
// log's hasMetricsAuth field all read this one method, so the log
// cannot report auth as off while the route is mounted.
//
// It requires both credentials rather than the username alone.
// loadFromEnv already rejects a half-set pair, but a Config built in
// code bypasses that, and the failure mode this guards is an endpoint
// mounted with a credential map whose only password is the empty
// string.
func (c *Config) MetricsAuthEnabled() bool {
return c.MetricsUsername != "" && c.MetricsPassword != ""
}
// envString returns the value of the named environment variable,
// or an empty string if not set.
func envString(key string) string {
@@ -329,6 +365,30 @@ func envPrefixList(key string) ([]netip.Prefix, error) {
return prefixes, nil
}
// resolveMetricsAuth reads the /metrics basic-auth credentials and
// rejects a half-set pair, naming both variables either way. The
// error carries neither value: the password is a secret.
func resolveMetricsAuth() (string, string, error) {
username := envString("METRICS_USERNAME")
password := envString("METRICS_PASSWORD")
if (username == "") == (password == "") {
return username, password, nil
}
set, empty := "METRICS_USERNAME", "METRICS_PASSWORD"
if username == "" {
set, empty = empty, set
}
return "", "", fmt.Errorf(
"%w: %s is set but %s is empty; METRICS_USERNAME and "+
"METRICS_PASSWORD must both be set to serve /metrics, "+
"or both be empty to leave it unmounted",
ErrIncompleteMetricsAuth, set, empty,
)
}
// resolveEnvironment reads WEBHOOKER_ENVIRONMENT, defaulting to
// dev, and rejects unrecognised values.
func resolveEnvironment() (string, error) {
@@ -406,22 +466,71 @@ func loadFromEnv() (*Config, error) {
return nil, err
}
allowedEgressCIDRs, err := envPrefixList("ALLOWED_EGRESS_CIDRS")
if err != nil {
return nil, err
}
metricsUsername, metricsPassword, err := resolveMetricsAuth()
if err != nil {
return nil, err
}
return &Config{
DataDir: envString("DATA_DIR"),
Debug: debug,
MaintenanceMode: maintenanceMode,
Environment: environment,
MetricsUsername: envString("METRICS_USERNAME"),
MetricsPassword: envString("METRICS_PASSWORD"),
MetricsUsername: metricsUsername,
MetricsPassword: metricsPassword,
Port: port,
SentryDSN: envString("SENTRY_DSN"),
RetentionSweepInterval: retentionSweepInterval,
SessionIdleTimeout: sessionIdleTimeout,
ReceiverRateLimit: receiverRateLimit,
TrustedProxies: trustedProxies,
AllowedEgressCIDRs: allowedEgressCIDRs,
}, nil
}
// PrefixStrings renders a prefix list as its CIDR strings, for
// logging a list an operator has to be able to read back.
func PrefixStrings(prefixes []netip.Prefix) []string {
out := make([]string, 0, len(prefixes))
for _, prefix := range prefixes {
out = append(out, prefix.String())
}
return out
}
// warnEgressAllowlist logs the effective ALLOWED_EGRESS_CIDRS
// whenever it is non-empty.
//
// It prints the blocks themselves rather than a count, because
// this is the one setting that lets a delivery target reach the
// host's own network: an operator reading the startup log has to
// be able to see exactly which hole is open. Silence means the
// list is empty and the SSRF guard is refusing every
// private/reserved range, which is the default.
func (c *Config) warnEgressAllowlist(log *slog.Logger) {
if len(c.AllowedEgressCIDRs) == 0 {
return
}
log.Warn(
"ALLOWED_EGRESS_CIDRS lets delivery targets reach these "+
"otherwise-blocked private/reserved networks. Anyone "+
"who can create a delivery target can now make this "+
"process issue requests into them, and read back the "+
"response. Link-local (cloud instance metadata) stays "+
"blocked regardless.",
"allowedEgressCIDRs",
strings.Join(PrefixStrings(c.AllowedEgressCIDRs), ","),
)
}
// warnSharedRateLimitBucket logs a startup warning whenever
// TRUSTED_PROXIES is empty, in any environment.
//
@@ -511,12 +620,13 @@ func New(lc fx.Lifecycle, params ConfigParams) (*Config, error) {
"sessionIdleTimeout", s.SessionIdleTimeout.String(),
"receiverRateLimit", s.ReceiverRateLimit,
"trustedProxies", len(s.TrustedProxies),
"allowedEgressCIDRs", len(s.AllowedEgressCIDRs),
"hasSentryDSN", s.SentryDSN != "",
"hasMetricsAuth",
s.MetricsUsername != "" && s.MetricsPassword != "",
"hasMetricsAuth", s.MetricsAuthEnabled(),
)
s.warnSharedRateLimitBucket(log)
s.warnEgressAllowlist(log)
return s, nil
}

View File

@@ -26,6 +26,12 @@ const (
// cidrPrivateV4 is the sample trusted-proxy block the
// TRUSTED_PROXIES cases are built from.
cidrPrivateV4 = "10.0.0.0/8"
// metricsAuthValue is the sample METRICS_PASSWORD the metrics
// credential cases are built from. It is asserted absent from
// the startup error, so it must not be a substring of either
// variable name that error prints.
metricsAuthValue = "s3cret"
)
func TestEnvironmentConfig(t *testing.T) {
@@ -627,6 +633,183 @@ func testTrustedProxiesSuccess(
assert.Equal(t, expected, got)
}
// TestAllowedEgressCIDRs covers ALLOWED_EGRESS_CIDRS, the escape
// hatch that lets a self-hosted deployment forward to its own
// network. Unset it must stay empty, so the SSRF guard keeps
// refusing every private/reserved range; a set-but-unparseable
// value must abort startup naming the variable rather than
// silently running with a list the operator did not write.
func TestAllowedEgressCIDRs(t *testing.T) {
tests := []struct {
name string
set bool
value string
expected []string
expectError bool
}{
{
name: caseUnsetUsesDefault,
set: false,
expected: []string{},
},
{
name: "empty value yields empty list",
set: true,
value: "",
expected: []string{},
},
{
name: caseValidValueParsed,
set: true,
value: cidrPrivateV4,
expected: []string{cidrPrivateV4},
},
{
name: "multiple blocks with whitespace",
set: true,
value: " 10.0.0.0/8 , 127.0.0.0/8 ",
expected: []string{cidrPrivateV4, "127.0.0.0/8"},
},
{
name: "bare address becomes a single host",
set: true,
value: "172.17.0.5",
expected: []string{"172.17.0.5/32"},
},
{
name: caseUnparseableFails,
set: true,
value: cidrPrivateV4 + ",not-an-address",
expectError: true,
},
{
name: "out-of-range prefix length fails startup",
set: true,
value: "10.0.0.0/33",
expectError: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests.
t.Setenv("WEBHOOKER_ENVIRONMENT", "dev")
if tt.set {
t.Setenv("ALLOWED_EGRESS_CIDRS", tt.value)
} else {
require.NoError(
t, os.Unsetenv("ALLOWED_EGRESS_CIDRS"),
)
}
if tt.expectError {
expectStartupErrorFor(
t, "ALLOWED_EGRESS_CIDRS", config.ErrInvalidCIDR,
)
} else {
testAllowedEgressCIDRsSuccess(t, tt.expected)
}
})
}
}
func testAllowedEgressCIDRsSuccess(
t *testing.T,
expected []string,
) {
t.Helper()
var cfg *config.Config
app := fxtest.New(
t,
fx.Provide(
globals.New,
logger.New,
config.New,
),
fx.Populate(&cfg),
)
require.NoError(t, app.Err())
app.RequireStart()
defer app.RequireStop()
assert.Equal(
t, expected, config.PrefixStrings(cfg.AllowedEgressCIDRs),
)
}
// TestEgressAllowlistWarning covers the startup log that shows an
// operator the hole ALLOWED_EGRESS_CIDRS opened. It must stay
// silent on the default (empty) list and, when set, print the
// blocks themselves rather than a count.
func TestEgressAllowlistWarning(t *testing.T) {
tests := []struct {
name string
allowed string
expectWarning bool
}{
{
name: "empty allowlist is quiet",
expectWarning: false,
},
{
name: "non-empty allowlist warns",
allowed: "10.0.0.0/8,127.0.0.0/8",
expectWarning: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests.
t.Setenv("WEBHOOKER_ENVIRONMENT", config.EnvironmentDev)
if tt.allowed == "" {
require.NoError(
t, os.Unsetenv("ALLOWED_EGRESS_CIDRS"),
)
} else {
t.Setenv("ALLOWED_EGRESS_CIDRS", tt.allowed)
}
var buf bytes.Buffer
log := slog.New(slog.NewJSONHandler(
&buf, &slog.HandlerOptions{
Level: slog.LevelDebug,
},
))
require.NoError(
t, config.WarnEgressAllowlistForTest(log),
)
if !tt.expectWarning {
assert.Empty(t, buf.String())
return
}
logged := buf.String()
assert.Contains(t, logged, `"level":"WARN"`)
assert.Contains(t, logged, "ALLOWED_EGRESS_CIDRS")
// The blocks themselves, not a count: the operator has
// to be able to read back which networks are open.
assert.Contains(t, logged, "10.0.0.0/8")
assert.Contains(t, logged, "127.0.0.0/8")
// The warning must keep saying what stays shut.
assert.Contains(t, logged, "Link-local")
})
}
}
// TestSharedRateLimitBucketWarning covers the startup warning that
// tells an operator a deployment behind a reverse proxy shares one
// rate-limit bucket between every client, which turns the receiver
@@ -726,3 +909,168 @@ func TestSharedRateLimitBucketWarning(t *testing.T) {
})
}
}
// metricsEnv describes what one subtest below puts in the
// environment for a single METRICS_ variable. A variable that is
// set to the empty string and one that is not set at all are
// distinct inputs here, because the reported bug arrived through
// the first of them.
type metricsEnv struct {
set bool
value string
}
// unset leaves the variable out of the environment entirely.
func unset() metricsEnv {
return metricsEnv{set: false, value: ""}
}
// setTo sets the variable, including to the empty string.
func setTo(value string) metricsEnv {
return metricsEnv{set: true, value: value}
}
// metricsAuthCase is one row of the table in TestMetricsAuthConfig,
// named so the table can live in its own function and keep the test
// itself short.
type metricsAuthCase struct {
name string
username metricsEnv
password metricsEnv
expectError bool
expectAuth bool
}
// metricsAuthCases enumerates every combination of the two
// credentials, counting "set to the empty string" and "not set at
// all" as separate inputs on each side.
func metricsAuthCases() []metricsAuthCase {
return []metricsAuthCase{
{
name: "both unset leaves metrics unmounted",
username: unset(),
password: unset(),
},
{
name: "both empty leaves metrics unmounted",
username: setTo(""),
password: setTo(""),
},
{
name: "both set enables metrics auth",
username: setTo("metrics"),
password: setTo(metricsAuthValue),
expectAuth: true,
},
{
name: "username with unset password fails",
username: setTo("metrics"),
password: unset(),
expectError: true,
},
{
name: "username with empty password fails",
username: setTo("metrics"),
password: setTo(""),
expectError: true,
},
{
name: "password with unset username fails",
username: unset(),
password: setTo(metricsAuthValue),
expectError: true,
},
{
name: "password with empty username fails",
username: setTo(""),
password: setTo(metricsAuthValue),
expectError: true,
},
}
}
// TestMetricsAuthConfig covers every combination of METRICS_USERNAME
// and METRICS_PASSWORD. Either both carry a value, in which case
// /metrics is served behind basic auth, or neither does, in which
// case the route is never mounted. One without the other is a
// startup error rather than a fallback: mounting on the username
// alone published /metrics behind a credential map that accepted an
// empty password, which is the defect this test exists to pin. See
// https://git.eeqj.de/sneak/webhooker/issues/205.
func TestMetricsAuthConfig(t *testing.T) {
for _, tt := range metricsAuthCases() {
t.Run(tt.name, func(t *testing.T) {
// Cannot use t.Parallel() here because t.Setenv
// is incompatible with parallel subtests.
if tt.username.set {
t.Setenv("METRICS_USERNAME", tt.username.value)
} else {
require.NoError(
t, os.Unsetenv("METRICS_USERNAME"),
)
}
if tt.password.set {
t.Setenv("METRICS_PASSWORD", tt.password.value)
} else {
require.NoError(
t, os.Unsetenv("METRICS_PASSWORD"),
)
}
if tt.expectError {
assertMetricsAuthRejected(t)
return
}
assertMetricsAuthAccepted(t, tt.expectAuth)
})
}
}
// assertMetricsAuthRejected requires that fx refused to build the
// graph, that the failure is ErrIncompleteMetricsAuth, and that the
// operator is told both variable names — the point of failing here
// rather than degrading is that the message says what to fix.
func assertMetricsAuthRejected(t *testing.T) {
t.Helper()
var cfg *config.Config
app := fx.New(
fx.NopLogger,
fx.Provide(globals.New, logger.New, config.New),
fx.Populate(&cfg),
)
err := app.Err()
require.Error(t, err)
require.ErrorIs(t, err, config.ErrIncompleteMetricsAuth)
assert.Contains(t, err.Error(), "METRICS_USERNAME")
assert.Contains(t, err.Error(), "METRICS_PASSWORD")
// The password is a secret and must not reach a startup error.
assert.NotContains(t, err.Error(), metricsAuthValue)
}
// assertMetricsAuthAccepted requires that startup succeeded and that
// MetricsAuthEnabled — the single value the /metrics mount and the
// startup log both read — reports what the environment asked for.
func assertMetricsAuthAccepted(t *testing.T, expectAuth bool) {
t.Helper()
var cfg *config.Config
app := fxtest.New(
t,
fx.Provide(globals.New, logger.New, config.New),
fx.Populate(&cfg),
)
require.NoError(t, app.Err())
app.RequireStart()
defer app.RequireStop()
assert.Equal(t, expectAuth, cfg.MetricsAuthEnabled())
}

View File

@@ -21,6 +21,21 @@ func WarnSharedRateLimitBucketForTest(log *slog.Logger) error {
return nil
}
// WarnEgressAllowlistForTest loads a Config from the current
// environment and emits its egress-allowlist startup warning to
// log, so a test can assert both that the warning fires only when
// the list is non-empty and that it names the blocks it opened.
func WarnEgressAllowlistForTest(log *slog.Logger) error {
c, err := loadFromEnv()
if err != nil {
return err
}
c.warnEgressAllowlist(log)
return nil
}
// EnvBoolForTest exposes envBool.
func EnvBoolForTest(key string, defaultValue bool) (bool, error) {
return envBool(key, defaultValue)

View File

@@ -19,7 +19,8 @@ func newSSRFTestEngine() *delivery.Engine {
client := &http.Client{
Timeout: 30 * time.Second,
Transport: delivery.NewSSRFSafeTransport(),
Transport: delivery.NewTestGuard().
NewSSRFSafeTransport(),
}
return delivery.NewTestEngine(log, client, 1)
@@ -36,8 +37,8 @@ func TestClientForConfig_TimeoutKeepsSSRFGuard(t *testing.T) {
engine := newSSRFTestEngine()
blocked := []string{
"http://127.0.0.1/hook",
"http://169.254.169.254/latest/meta-data/",
loopbackHookURL,
metadataURL,
"http://[fe80::1]/hook",
}

View File

@@ -120,6 +120,7 @@ type EngineParams struct {
DB *database.Database
DBManager *database.WebhookDBManager
Logger *logger.Logger
SSRFGuard *Guard
}
// Engine processes queued deliveries in the background
@@ -168,7 +169,7 @@ func New(
e.initTargets(&http.Client{
Timeout: httpClientTimeout,
Transport: NewSSRFSafeTransport(),
Transport: params.SSRFGuard.NewSSRFSafeTransport(),
})
e.registerHooks(lc)

View File

@@ -5,6 +5,7 @@ import (
"log/slog"
"net"
"net/http"
"net/netip"
"time"
"go.uber.org/fx"
@@ -31,6 +32,26 @@ func ExportIsBlockedIP(ip net.IP) bool {
return isBlockedIP(ip)
}
// NewTestGuard builds an SSRF Guard from an explicit egress
// allowlist, without going through config. Passing no prefixes
// yields the default guard, which blocks every private/reserved
// range.
func NewTestGuard(allowed ...netip.Prefix) *Guard {
return &Guard{allowed: allowed}
}
// ExportCheckIP exposes the guard's single decision point, so a
// test can assert the policy both the validator and the dialer
// inherit without needing a live destination.
func (g *Guard) ExportCheckIP(ip net.IP) error {
return g.checkIP(ip)
}
// ExportAlwaysBlockedNetworks exposes alwaysBlockedNetworks.
func ExportAlwaysBlockedNetworks() []*net.IPNet {
return alwaysBlockedNetworks
}
// ExportBlockedNetworks exposes blockedNetworks.
func ExportBlockedNetworks() []*net.IPNet {
return blockedNetworks

View File

@@ -6,8 +6,11 @@ import (
"fmt"
"net"
"net/http"
"net/netip"
"net/url"
"time"
"sneak.berlin/go/webhooker/internal/config"
)
const (
@@ -25,20 +28,41 @@ var (
errBlockedIP = errors.New(
"blocked private/reserved IP range",
)
errBlockedMetadata = errors.New(
"blocked link-local or cloud instance metadata " +
"address: ALLOWED_EGRESS_CIDRS cannot open it",
)
errInvalidScheme = errors.New(
"only http and https are allowed",
)
)
// blockedNetworks contains all private/reserved IP ranges
// that should be blocked to prevent SSRF attacks.
// that should be blocked to prevent SSRF attacks. An operator
// can permit specific blocks out of this set with
// ALLOWED_EGRESS_CIDRS; see Guard.
//
//nolint:gochecknoglobals // package-level network list is appropriate here
var blockedNetworks []*net.IPNet
// alwaysBlockedNetworks are the ranges no configuration can
// open: the link-local blocks and the cloud instance metadata
// endpoints that live outside them. Reaching one is credential
// theft rather than delivery to an internal service, so a
// supplied CIDR that covers such an address still leaves it
// blocked.
//
// Some of these are also in blockedNetworks and this list is
// what makes them unconditional; the metadata endpoints outside
// the link-local range are host routes, so blocking them costs
// an operator nothing else on the surrounding network.
//
//nolint:gochecknoglobals // package-level network list is appropriate here
var alwaysBlockedNetworks []*net.IPNet
//nolint:gochecknoinits // init is the idiomatic way to parse CIDRs once at startup
func init() {
cidrs := []string{
blockedNetworks = mustParseCIDRs([]string{
"127.0.0.0/8",
"10.0.0.0/8",
"172.16.0.0/12",
@@ -56,8 +80,45 @@ func init() {
"::1/128",
"fc00::/7",
"fe80::/10",
})
// Every entry is named. The set must not grow or shrink
// without a matching change to
// TestAlwaysBlockedNetworks_PinnedSet.
//
// The IPv4-mapped form ::ffff:169.254.169.254 needs no
// entry: net.IPNet.Contains normalises it via To4() before
// comparing, so 169.254.0.0/16 already matches it. To4()
// does not normalise the IPv4-compatible or NAT64 forms,
// which is why those are listed separately.
alwaysBlockedNetworks = mustParseCIDRs([]string{
// IPv4 link-local, carrying the 169.254.169.254
// metadata service used by AWS, GCP, Azure and others.
"169.254.0.0/16",
// IPv6 link-local, its IPv6 counterpart.
"fe80::/10",
// AWS IPv6 IMDS. It sits in fc00::/7, so an operator
// allowlisting their own ULA block (fd00::/8 is an
// ordinary entry) would otherwise reopen it.
"fd00:ec2::254/128",
// Alibaba Cloud metadata. It sits in CGNAT
// 100.64.0.0/10, which Tailscale also uses, so an
// operator allowlisting a Tailscale peer's range would
// otherwise reopen it.
"100.100.100.200/32",
// 169.254.169.254 as an IPv4-compatible IPv6 address.
"::a9fe:a9fe/128",
// 169.254.169.254 behind the NAT64 well-known prefix.
"64:ff9b::a9fe:a9fe/128",
})
}
// mustParseCIDRs parses a list of CIDR literals, panicking on a
// bad one. The inputs are compile-time constants, so a failure
// is a programming error rather than a runtime condition.
func mustParseCIDRs(cidrs []string) []*net.IPNet {
networks := make([]*net.IPNet, 0, len(cidrs))
for _, cidr := range cidrs {
_, network, err := net.ParseCIDR(cidr)
if err != nil {
@@ -67,16 +128,15 @@ func init() {
))
}
blockedNetworks = append(
blockedNetworks, network,
)
}
networks = append(networks, network)
}
// isBlockedIP checks whether an IP address falls within
// any blocked private/reserved network range.
func isBlockedIP(ip net.IP) bool {
for _, network := range blockedNetworks {
return networks
}
// matchesAny reports whether ip falls inside any of networks.
func matchesAny(networks []*net.IPNet, ip net.IP) bool {
for _, network := range networks {
if network.Contains(ip) {
return true
}
@@ -85,9 +145,40 @@ func isBlockedIP(ip net.IP) bool {
return false
}
// isBlockedIP checks whether an IP address falls within
// any blocked private/reserved network range, before any
// operator allowlist is considered.
func isBlockedIP(ip net.IP) bool {
return matchesAny(blockedNetworks, ip)
}
// Guard makes every SSRF decision in the process.
//
// It holds the operator's ALLOWED_EGRESS_CIDRS allowlist and
// applies it in exactly one place, checkIP, which both the
// target-creation validator (ValidateTargetURL) and the delivery
// dialer call. Routing both through the same function is the
// point: when the two paths decided separately they drifted and
// disagreed, which is what made a target creatable but
// undeliverable.
//
// The guard is always on. The allowlist only ever adds specific
// networks to what the default blocklist refuses, and no
// configuration turns the guard off wholesale.
type Guard struct {
// allowed is the operator's ALLOWED_EGRESS_CIDRS. Empty
// (the default) means the default blocklist stands as-is.
allowed []netip.Prefix
}
// NewGuard builds the process-wide SSRF guard from configuration.
func NewGuard(cfg *config.Config) *Guard {
return &Guard{allowed: cfg.AllowedEgressCIDRs}
}
// ValidateTargetURL checks that an HTTP delivery target
// URL is safe from SSRF attacks.
func ValidateTargetURL(
func (g *Guard) ValidateTargetURL(
ctx context.Context, targetURL string,
) error {
parsed, err := url.Parse(targetURL)
@@ -111,36 +202,79 @@ func ValidateTargetURL(
}
if ip := net.ParseIP(host); ip != nil {
return checkBlockedIP(ip)
return g.checkIP(ip)
}
return validateHostname(ctx, host)
return g.validateHostname(ctx, host)
}
func validateScheme(scheme string) error {
if scheme != "http" && scheme != "https" {
// NewSSRFSafeTransport creates an http.Transport with a
// custom DialContext that refuses connections to any address
// this guard blocks. It resolves and checks at dial time, so a
// name that passed validation but now answers with a blocked
// address (DNS rebinding) is still refused.
func (g *Guard) NewSSRFSafeTransport() *http.Transport {
return &http.Transport{
DialContext: g.ssrfDialContext,
}
}
// allows reports whether ip falls inside the operator's
// configured egress allowlist.
func (g *Guard) allows(ip net.IP) bool {
if len(g.allowed) == 0 {
return false
}
addr, ok := netip.AddrFromSlice(ip)
if !ok {
return false
}
// Config unmaps every parsed prefix, so an IPv4-mapped
// address has to be unmapped too or it would never match.
addr = addr.Unmap()
for _, prefix := range g.allowed {
if prefix.Contains(addr) {
return true
}
}
return false
}
// checkIP is the single point at which SSRF policy is decided.
//
// The order is the policy:
//
// 1. alwaysBlockedNetworks is refused before the allowlist is
// consulted, so no configured CIDR reaches link-local or a
// cloud instance metadata endpoint.
// 2. The allowlist is consulted next, so a listed private
// network becomes reachable.
// 3. Everything else keeps the default blocklist's answer.
func (g *Guard) checkIP(ip net.IP) error {
if matchesAny(alwaysBlockedNetworks, ip) {
return fmt.Errorf(
"unsupported URL scheme %q: %w",
scheme, errInvalidScheme,
"target IP %s: %w", ip, errBlockedMetadata,
)
}
if g.allows(ip) {
return nil
}
func checkBlockedIP(ip net.IP) error {
if isBlockedIP(ip) {
return fmt.Errorf(
"target IP %s is in a blocked "+
"private/reserved range: %w",
ip, errBlockedIP,
"target IP %s: %w", ip, errBlockedIP,
)
}
return nil
}
func validateHostname(
func (g *Guard) validateHostname(
ctx context.Context, host string,
) error {
dnsCtx, cancel := context.WithTimeout(
@@ -165,11 +299,11 @@ func validateHostname(
}
for _, ipAddr := range ips {
if isBlockedIP(ipAddr.IP) {
err = g.checkIP(ipAddr.IP)
if err != nil {
return fmt.Errorf(
"hostname %q resolves to blocked "+
"IP %s: %w",
host, ipAddr.IP, errBlockedIP,
"hostname %q resolves to a blocked address: %w",
host, err,
)
}
}
@@ -177,16 +311,7 @@ func validateHostname(
return nil
}
// NewSSRFSafeTransport creates an http.Transport with a
// custom DialContext that blocks connections to
// private/reserved IP addresses.
func NewSSRFSafeTransport() *http.Transport {
return &http.Transport{
DialContext: ssrfDialContext,
}
}
func ssrfDialContext(
func (g *Guard) ssrfDialContext(
ctx context.Context,
network, addr string,
) (net.Conn, error) {
@@ -209,11 +334,11 @@ func ssrfDialContext(
}
for _, ipAddr := range ips {
if isBlockedIP(ipAddr.IP) {
err = g.checkIP(ipAddr.IP)
if err != nil {
return nil, fmt.Errorf(
"ssrf: connection to %s (%s) "+
"blocked: %w",
host, ipAddr.IP, errBlockedIP,
"ssrf: connection to %s blocked: %w",
host, err,
)
}
}
@@ -225,3 +350,14 @@ func ssrfDialContext(
net.JoinHostPort(ips[0].IP.String(), port),
)
}
func validateScheme(scheme string) error {
if scheme != "http" && scheme != "https" {
return fmt.Errorf(
"unsupported URL scheme %q: %w",
scheme, errInvalidScheme,
)
}
return nil
}

View File

@@ -0,0 +1,445 @@
package delivery_test
import (
"context"
"net"
"net/http"
"net/http/httptest"
"net/netip"
"net/url"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/delivery"
)
// Addresses the SSRF tests in this package share.
const (
// metadataIP is the cloud instance metadata address, and
// metadataURL an endpoint on it. The guard must never reach
// either, whatever an operator lists.
metadataIP = "169.254.169.254"
metadataURL = "http://" + metadataIP + "/latest/meta-data/"
// loopbackHookURL is a target on this host: blocked by
// default, reachable only once an operator allowlists
// loopback.
loopbackHookURL = "http://127.0.0.1/hook"
// publicIP is an ordinary public address, which the guard
// permits with or without an allowlist.
publicIP = "93.184.216.34"
// allowAllIPv4 and allowAllIPv6 are the widest allowlist
// entries expressible: the whole internet, in each family.
// Nothing unconditionally blocked may be reachable under
// them.
allowAllIPv4 = "0.0.0.0/0"
allowAllIPv6 = "::/0"
)
// TestGuardAllowlist_PermittedCIDRDelivers proves the escape
// hatch actually works end to end: with 127.0.0.0/8 allowed, the
// guard's own transport connects to a loopback server and gets a
// response back. The default guard, given the identical URL,
// refuses it — so the delivery succeeds because of the allowlist
// and nothing else.
func TestGuardAllowlist_PermittedCIDRDelivers(t *testing.T) {
t.Parallel()
srv := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNoContent)
},
))
t.Cleanup(srv.Close)
// httptest listens on loopback, which the default blocklist
// covers: exactly the "forward to a service on this host"
// case the allowlist exists for.
requireLoopback(t, srv.URL)
guard := delivery.NewTestGuard(
netip.MustParsePrefix("127.0.0.0/8"),
)
require.NoError(t,
guard.ValidateTargetURL(context.Background(), srv.URL),
"an allowlisted loopback target must pass validation",
)
client := &http.Client{
Timeout: 5 * time.Second,
Transport: guard.NewSSRFSafeTransport(),
}
req, err := http.NewRequestWithContext(
context.Background(), http.MethodPost, srv.URL, nil,
)
require.NoError(t, err)
resp, err := client.Do(req)
require.NoError(t,
err, "an allowlisted loopback target must be deliverable",
)
defer func() { _ = resp.Body.Close() }()
assert.Equal(t, http.StatusNoContent, resp.StatusCode)
// The same URL through the default guard must still fail, or
// this test would pass without the allowlist doing anything.
assert.Error(t,
delivery.NewTestGuard().ValidateTargetURL(
context.Background(), srv.URL,
),
"without the allowlist the same target must be refused",
)
}
// TestGuardAllowlist_UnlistedPrivateStillRefused proves the
// allowlist grants only what it names. A guard that opens one
// private block must keep refusing every other one, at both the
// validation and the delivery entry point.
func TestGuardAllowlist_UnlistedPrivateStillRefused(t *testing.T) {
t.Parallel()
// Only 10.1.0.0/16 is open — a narrow block inside a much
// wider private range, so the test can tell "permits the
// listed block" from "permits anything private".
guard := delivery.NewTestGuard(
netip.MustParsePrefix("10.1.0.0/16"),
)
refused := []string{
"http://192.168.1.10/hook",
"http://172.16.0.1/hook",
loopbackHookURL,
"http://[fc00::1]/hook",
"http://100.64.0.1/hook",
// Private, adjacent to the allowed block, outside it.
"http://10.2.0.1/hook",
}
for _, target := range refused {
t.Run(target, func(t *testing.T) {
t.Parallel()
err := guard.ValidateTargetURL(
context.Background(), target,
)
require.Error(t,
err, "%s is not allowlisted and must be refused",
target,
)
assert.Contains(t, err.Error(), "blocked")
assertDialRefused(t, guard, target)
})
}
// The block that is listed must in fact be permitted, so the
// refusals above are selective rather than a guard that
// ignores its allowlist entirely.
assert.NoError(t,
guard.ValidateTargetURL(
context.Background(), "http://10.1.2.3/hook",
),
"the allowlisted block must be permitted",
)
}
// TestGuardAllowlist_MetadataAlwaysRefused is the load-bearing
// case: cloud instance metadata endpoints are credential theft
// rather than delivery to an internal service, so no allowlist
// reaches one. Every guard below names a CIDR that covers its
// target — including 0.0.0.0/0, ::/0, and the ordinary ULA and
// CGNAT blocks an operator would really list — and the address
// must stay refused anyway, on both the validation and the
// delivery path.
func TestGuardAllowlist_MetadataAlwaysRefused(t *testing.T) {
t.Parallel()
for _, tt := range metadataAlwaysRefusedCases() {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
guard := delivery.NewTestGuard(
netip.MustParsePrefix(tt.allow),
)
err := guard.ValidateTargetURL(
context.Background(), tt.target,
)
require.Error(t,
err,
"%s must stay blocked even though %s covers it",
tt.target, tt.allow,
)
assert.Contains(t,
err.Error(),
"ALLOWED_EGRESS_CIDRS cannot open it",
"the refusal must say why it cannot be opened",
)
assertDialRefused(t, guard, tt.target)
})
}
}
// metadataAlwaysRefusedCase is one (allowlist, target) pair that
// must be refused: allow covers target, and target must stay
// blocked regardless.
type metadataAlwaysRefusedCase struct {
name string
allow string
target string
}
// metadataAlwaysRefusedCases enumerates every unconditionally
// blocked address together with an allowlist entry that would
// otherwise reach it.
func metadataAlwaysRefusedCases() []metadataAlwaysRefusedCase {
return []metadataAlwaysRefusedCase{
{
name: "exact metadata host",
allow: "169.254.169.254/32",
target: metadataURL,
},
{
name: "whole link-local block",
allow: "169.254.0.0/16",
target: metadataURL,
},
{
name: "supernet covering link-local",
allow: "169.0.0.0/8",
target: metadataURL,
},
{
name: "the entire IPv4 internet",
allow: allowAllIPv4,
target: metadataURL,
},
{
name: "other link-local address",
allow: allowAllIPv4,
target: "http://169.254.1.1/",
},
{
name: "IPv6 link-local",
allow: allowAllIPv6,
target: "http://[fe80::1]/",
},
{
// fd00::/8 is an ordinary block for an operator to
// allowlist, and AWS's IPv6 IMDS sits inside it.
name: "AWS IPv6 IMDS under an allowlisted ULA block",
allow: "fd00::/8",
target: "http://[fd00:ec2::254]/latest/meta-data/",
},
{
// Tailscale uses 100.64.0.0/10, so an operator
// forwarding to a Tailscale peer lists exactly this.
name: "Alibaba metadata under allowlisted CGNAT",
allow: "100.64.0.0/10",
target: "http://100.100.100.200/latest/meta-data/",
},
{
// To4() does not normalise the IPv4-compatible form,
// so this needs its own always-blocked entry.
name: "IPv4-compatible IPv6 form of the metadata IP",
allow: allowAllIPv6,
target: "http://[::a9fe:a9fe]/latest/meta-data/",
},
{
// Nor the NAT64 well-known prefix form.
name: "NAT64 form of the metadata IP",
allow: allowAllIPv6,
target: "http://[64:ff9b::a9fe:a9fe]/latest/meta-data/",
},
{
// Already refused before this change: IPNet.Contains
// calls To4() first, so the mapped form matches
// 169.254.0.0/16. Pinned so it cannot regress.
name: "IPv4-mapped IPv6 form of the metadata IP",
allow: allowAllIPv6,
target: "http://[::ffff:169.254.169.254]/latest/meta-data/",
},
}
}
// TestGuardAllowlist_PublicUnaffected asserts the allowlist does
// not narrow anything: public addresses were reachable before it
// existed and stay reachable, whether or not a list is set.
func TestGuardAllowlist_PublicUnaffected(t *testing.T) {
t.Parallel()
guards := map[string]*delivery.Guard{
"default": delivery.NewTestGuard(),
"with allowlist": delivery.NewTestGuard(
netip.MustParsePrefix("10.0.0.0/8"),
),
}
for name, guard := range guards {
t.Run(name, func(t *testing.T) {
t.Parallel()
assert.NoError(t,
guard.ValidateTargetURL(
context.Background(),
"http://"+publicIP+"/webhook",
),
)
})
}
}
// TestGuardCheckIP_BothPathsShareOneDecision asserts that the
// validator and the dialer are not two policies that happen to
// agree: both are defined in terms of checkIP, so the exported
// decision function is the whole answer for a given address.
func TestGuardCheckIP_BothPathsShareOneDecision(t *testing.T) {
t.Parallel()
guard := delivery.NewTestGuard(
netip.MustParsePrefix("10.0.0.0/8"),
)
tests := []struct {
ip string
allowed bool
}{
{"10.1.2.3", true},
{publicIP, true},
{"192.168.1.1", false},
{"127.0.0.1", false},
{metadataIP, false},
}
for _, tt := range tests {
t.Run(tt.ip, func(t *testing.T) {
t.Parallel()
ip := net.ParseIP(tt.ip)
require.NotNil(t, ip)
decision := guard.ExportCheckIP(ip)
validation := guard.ValidateTargetURL(
context.Background(), "http://"+hostFor(tt.ip)+"/x",
)
if tt.allowed {
require.NoError(t, decision)
require.NoError(t, validation)
return
}
require.Error(t, decision)
require.Error(t, validation,
"validation must refuse what checkIP refuses",
)
})
}
}
// TestAlwaysBlockedNetworks_PinnedSet pins the unconditional set
// exactly, so it cannot quietly grow or shrink.
//
// It stays deliberately small. Everything else in the default
// blocklist is an operator's own network and must remain
// openable, or the escape hatch would not work — which is why
// the metadata endpoints outside the link-local range are host
// routes rather than the blocks that contain them.
func TestAlwaysBlockedNetworks_PinnedSet(t *testing.T) {
t.Parallel()
nets := delivery.ExportAlwaysBlockedNetworks()
got := make([]string, 0, len(nets))
for _, n := range nets {
got = append(got, n.String())
}
want := []string{
// IPv4 link-local: the 169.254.169.254 metadata
// service on AWS, GCP, Azure and others.
"169.254.0.0/16",
// IPv6 link-local.
"fe80::/10",
// AWS IPv6 IMDS, inside the ULA space an operator may
// legitimately allowlist.
"fd00:ec2::254/128",
// Alibaba Cloud metadata, inside CGNAT.
"100.100.100.200/32",
// 169.254.169.254 as an IPv4-compatible IPv6 address.
"::a9fe:a9fe/128",
// 169.254.169.254 behind the NAT64 well-known prefix.
"64:ff9b::a9fe:a9fe/128",
}
assert.Equal(t, want, got)
}
// requireLoopback fails the test unless rawURL's host is a
// loopback address, so the allowlist test cannot silently stop
// exercising a blocked range.
func requireLoopback(t *testing.T, rawURL string) {
t.Helper()
parsed, err := url.Parse(rawURL)
require.NoError(t, err)
ip := net.ParseIP(parsed.Hostname())
require.NotNil(t, ip, "test server host must be an IP literal")
require.True(t, ip.IsLoopback(),
"test server must listen on loopback, got %s", ip,
)
}
// assertDialRefused asserts the guard's transport refuses to
// connect to target, which is the delivery-time half of the
// policy. It never reaches the network: the guard checks the
// resolved address before dialling.
func assertDialRefused(
t *testing.T, guard *delivery.Guard, target string,
) {
t.Helper()
client := &http.Client{
Timeout: 5 * time.Second,
Transport: guard.NewSSRFSafeTransport(),
}
req, err := http.NewRequestWithContext(
context.Background(), http.MethodPost, target, nil,
)
require.NoError(t, err)
resp, err := client.Do(req)
if resp != nil {
_ = resp.Body.Close()
}
require.Error(t, err,
"delivery to %s must be refused by the dialer", target,
)
assert.Contains(t, err.Error(), "blocked",
"the refusal must come from the SSRF guard",
)
}
// hostFor renders an IP as it appears in a URL host, bracketing
// IPv6 literals.
func hostFor(ip string) string {
if net.ParseIP(ip).To4() == nil {
return "[" + ip + "]"
}
return ip
}

View File

@@ -31,10 +31,10 @@ func TestIsBlockedIP_PrivateRanges(t *testing.T) {
{"192.168.0.1", "192.168.0.1", true},
{"192.168.255.255", "192.168.255.255", true},
{"169.254.0.1", "169.254.0.1", true},
{"169.254.169.254", "169.254.169.254", true},
{metadataIP, metadataIP, true},
{"8.8.8.8", "8.8.8.8", false},
{"1.1.1.1", "1.1.1.1", false},
{"93.184.216.34", "93.184.216.34", false},
{publicIP, publicIP, false},
{"::1", "::1", true},
{"fd00::1", "fd00::1", true},
{"fc00::1", "fc00::1", true},
@@ -72,12 +72,12 @@ func TestValidateTargetURL_Blocked(t *testing.T) {
t.Parallel()
blockedURLs := []string{
"http://127.0.0.1/hook",
loopbackHookURL,
"http://127.0.0.1:8080/hook",
"https://10.0.0.1/hook",
"http://192.168.1.1/webhook",
"http://172.16.0.1/api",
"http://169.254.169.254/latest/meta-data/",
metadataURL,
"http://[::1]/hook",
"http://[fc00::1]/hook",
"http://[fe80::1]/hook",
@@ -88,7 +88,7 @@ func TestValidateTargetURL_Blocked(t *testing.T) {
t.Run(u, func(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
err := delivery.NewTestGuard().ValidateTargetURL(
context.Background(), u,
)
@@ -112,7 +112,7 @@ func TestValidateTargetURL_Allowed(t *testing.T) {
t.Run(u, func(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
err := delivery.NewTestGuard().ValidateTargetURL(
context.Background(), u,
)
@@ -126,7 +126,7 @@ func TestValidateTargetURL_Allowed(t *testing.T) {
func TestValidateTargetURL_InvalidScheme(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
err := delivery.NewTestGuard().ValidateTargetURL(
context.Background(), "ftp://example.com/hook",
)
@@ -140,7 +140,7 @@ func TestValidateTargetURL_InvalidScheme(t *testing.T) {
func TestValidateTargetURL_EmptyHost(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
err := delivery.NewTestGuard().ValidateTargetURL(
context.Background(), "http:///path",
)
@@ -150,7 +150,7 @@ func TestValidateTargetURL_EmptyHost(t *testing.T) {
func TestValidateTargetURL_InvalidURL(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
err := delivery.NewTestGuard().ValidateTargetURL(
context.Background(), "://invalid",
)

View File

@@ -185,7 +185,7 @@ func TestDoHTTPRequest_TransportErrorMasksURL(t *testing.T) {
func TestValidateTargetURL_UnparsableURLIsMasked(t *testing.T) {
t.Parallel()
err := delivery.ValidateTargetURL(
err := delivery.NewTestGuard().ValidateTargetURL(
context.TODO(),
"https://hooks.slack.com"+maskSecretPath+"\n",
)

View File

@@ -59,6 +59,7 @@ type HandlersParams struct {
Middleware *middleware.Middleware
Notifier delivery.Notifier
Evictor delivery.WebhookEvictor
SSRFGuard *delivery.Guard
}
// Handlers provides HTTP handler methods for all application
@@ -75,6 +76,11 @@ type Handlers struct {
evictor delivery.WebhookEvictor
templates map[string]*template.Template
// ssrf validates submitted target URLs. It is the same guard
// the delivery engine dials through, so a URL accepted here
// is one delivery will actually attempt.
ssrf *delivery.Guard
// dummyVerifications counts the equivalent-cost verifications
// charged for usernames that do not exist. It exists so a test
// can prove that path runs without measuring wall-clock time.
@@ -114,6 +120,7 @@ func New(
s.mw = params.Middleware
s.notifier = params.Notifier
s.evictor = params.Evictor
s.ssrf = params.SSRFGuard
// Parse all page templates once at startup
s.templates = map[string]*template.Template{

View File

@@ -84,6 +84,7 @@ func newTestApp(
return r
},
middleware.New,
delivery.NewGuard,
handlers.New,
),
fx.Populate(targets...),

View File

@@ -1160,7 +1160,7 @@ func (h *Handlers) buildURLTargetConfig(
return "", errMissingURL
}
err := delivery.ValidateTargetURL(
err := h.ssrf.ValidateTargetURL(
r.Context(), targetURL,
)
if err != nil {

View File

@@ -43,10 +43,7 @@ func (s *Server) serveUntilShutdown() {
err := s.httpServer.ListenAndServe()
if err != nil && !errors.Is(err, http.ErrServerClosed) {
s.log.Error("listen error", "error", err)
if s.cancelFunc != nil {
s.cancelFunc()
}
s.shutdownOnListenFailure()
}
}

View File

@@ -0,0 +1,95 @@
package server_test
import (
"context"
"net"
"testing"
"time"
"github.com/stretchr/testify/require"
"go.uber.org/fx"
"sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/server"
)
// listenFailureDeadline is how long the app gets to give up after a
// listen it cannot satisfy. The defect this pins left the process
// reporting RUNNING for 183 seconds with nothing bound; a bind error
// is known instantly, so anything past a moment here is that defect
// back.
const listenFailureDeadline = 2 * time.Second
// lifecycleTimeout bounds the app's start and stop sequences so a
// wedged hook fails the test instead of hanging it.
const lifecycleTimeout = 15 * time.Second
// TestListenFailure_ShutsDownTheApp pins that a listener the server
// cannot bind terminates the application with a non-zero status.
//
// The fx OnStart hook returns as soon as the serving goroutine is
// spawned, so a bind failure is discovered after fx has already
// reported RUNNING. Nothing else in the graph observes it, and the
// process used to stay alive with no listener: down, but indis-
// tinguishable from healthy to systemd's Restart=on-failure and to
// Docker's restart policies, which is the state this test exists to
// keep from returning.
//
// The port is occupied by a listener this test holds open, on a
// kernel-chosen port, so the failure is the real EADDRINUSE the
// operator hits when a second instance starts. Loopback is enough to
// collide with the server's wildcard bind: a listening socket on a
// specific address blocks the wildcard from claiming the same port.
func TestListenFailure_ShutsDownTheApp(t *testing.T) {
t.Parallel()
var listenCfg net.ListenConfig
occupied, err := listenCfg.Listen(
t.Context(), "tcp", "127.0.0.1:0",
)
require.NoError(t, err)
t.Cleanup(func() { _ = occupied.Close() })
addr, ok := occupied.Addr().(*net.TCPAddr)
require.True(t, ok, "listener is not TCP")
// The collaborators come from the wired graph rather than stubs,
// so the Server under test is the one that ships. Only the port
// is test-specific.
env := newTestEnv(t)
env.cfg.Port = addr.Port
app := fx.New(
fx.NopLogger,
fx.Supply(env.log, env.cfg, env.mw, env.hnd),
fx.Provide(globals.New, server.New),
fx.Invoke(func(*server.Server) {}),
)
startCtx, cancelStart := context.WithTimeout(
context.Background(), lifecycleTimeout,
)
defer cancelStart()
require.NoError(t, app.Start(startCtx))
select {
case sig := <-app.Wait():
require.Equal(
t, server.ListenFailureExitCode, sig.ExitCode,
"listen failure must exit non-zero",
)
case <-time.After(listenFailureDeadline):
t.Fatal("listen failure left the app running")
}
// The stop sequence still has to complete: the fix must reach
// shutdown through fx rather than around it.
stopCtx, cancelStop := context.WithTimeout(
context.Background(), lifecycleTimeout,
)
defer cancelStop()
require.NoError(t, app.Stop(stopCtx))
}

View File

@@ -55,8 +55,11 @@ func (s *Server) setupGlobalMiddleware() {
s.router.Use(s.mw.SecurityHeaders())
s.router.Use(s.mw.Logging())
// Metrics middleware (only if credentials are configured)
if s.params.Config.MetricsUsername != "" {
// Metrics recording middleware, registered only when the
// endpoint that exposes what it records is served. The
// condition is the same MetricsAuthEnabled the /metrics mount
// in setupRoutes reads.
if s.params.Config.MetricsAuthEnabled() {
s.router.Use(s.mw.Metrics())
}
@@ -103,8 +106,14 @@ func (s *Server) setupRoutes() {
s.h.HandleHealthCheck(),
)
// set up authenticated /metrics route:
if s.params.Config.MetricsUsername != "" {
// Authenticated /metrics route. The condition is
// Config.MetricsAuthEnabled and never the username alone: a
// username with an empty password would otherwise mount the
// endpoint behind a credential map that accepts an empty
// password. Config rejects that combination at startup, and
// this reads the same value the startup log reports, so the
// two cannot disagree about whether the route exists.
if s.params.Config.MetricsAuthEnabled() {
s.router.Group(func(r chi.Router) {
r.Use(s.mw.MetricsAuth())
r.Get(

View File

@@ -34,6 +34,13 @@ import (
// the CSRF middleware executed.
const csrfCookieName = "_gorilla_csrf"
const (
// metricsUser and metricsAuthValue are the /metrics basic-auth
// credentials the metrics routing tests below configure.
metricsUser = "metrics"
metricsAuthValue = "s3cret"
)
type noopNotifier struct{}
func (n *noopNotifier) Notify([]delivery.Task) {}
@@ -69,9 +76,23 @@ type testEnv struct {
func newTestEnv(t *testing.T) *testEnv {
t.Helper()
return newTestEnvWithConfig(t, &config.Config{
DataDir: t.TempDir(),
Environment: config.EnvironmentDev,
})
}
// newTestEnvWithConfig is newTestEnv over a caller-supplied Config,
// for the routes whose existence the configuration decides. The same
// pointer reaches the router and every middleware, so a test cannot
// accidentally configure one and not the other.
func newTestEnvWithConfig(
t *testing.T, cfg *config.Config,
) *testEnv {
t.Helper()
var (
log *logger.Logger
cfg *config.Config
mw *middleware.Middleware
hnd *handlers.Handlers
sess *session.Session
@@ -84,12 +105,7 @@ func newTestEnv(t *testing.T) *testEnv {
fx.Provide(
globals.New,
logger.New,
func() *config.Config {
return &config.Config{
DataDir: t.TempDir(),
Environment: config.EnvironmentDev,
}
},
func() *config.Config { return cfg },
database.New,
database.NewWebhookDBManager,
healthcheck.New,
@@ -97,9 +113,10 @@ func newTestEnv(t *testing.T) *testEnv {
func() delivery.Notifier { return &noopNotifier{} },
func() delivery.WebhookEvictor { return &noopEvictor{} },
middleware.New,
delivery.NewGuard,
handlers.New,
),
fx.Populate(&log, &cfg, &mw, &hnd, &sess, &db, &dbMgr),
fx.Populate(&log, &mw, &hnd, &sess, &db, &dbMgr),
)
app.RequireStart()
t.Cleanup(app.RequireStop)
@@ -657,3 +674,119 @@ func TestSourceLogsBody_OtherUser404s(t *testing.T) {
assert.Equal(t, http.StatusSeeOther, anon.Code)
assert.Equal(t, "/pages/login", anon.Header().Get("Location"))
}
// metricsConfig is a Config differing from the routing default only
// in the two /metrics credentials.
func metricsConfig(
t *testing.T, username, password string,
) *config.Config {
t.Helper()
return &config.Config{
DataDir: t.TempDir(),
Environment: config.EnvironmentDev,
MetricsUsername: username,
MetricsPassword: password,
}
}
// metricsRequest asks the real router for /metrics with the given
// basic-auth credentials, or with no Authorization header when
// username is empty.
func (e *testEnv) metricsRequest(
username, password string,
) *httptest.ResponseRecorder {
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/metrics", nil,
)
if username != "" {
req.SetBasicAuth(username, password)
}
w := httptest.NewRecorder()
e.router.ServeHTTP(w, req)
return w
}
// TestMetricsRouteUnmountedWithoutCredentials pins that with neither
// credential configured the route does not exist, which is the
// documented behaviour and the only valid way for /metrics to be
// absent.
func TestMetricsRouteUnmountedWithoutCredentials(t *testing.T) {
t.Parallel()
env := newTestEnvWithConfig(t, metricsConfig(t, "", ""))
assert.Equal(
t, http.StatusNotFound,
env.metricsRequest("", "").Code,
)
}
// TestMetricsRouteRequiresCredentials pins that with both credentials
// configured the route exists and every request that does not carry
// the configured pair is refused — including the empty password that
// a half-set configuration used to make sufficient.
func TestMetricsRouteRequiresCredentials(t *testing.T) {
t.Parallel()
env := newTestEnvWithConfig(
t, metricsConfig(t, metricsUser, metricsAuthValue),
)
assert.Equal(
t, http.StatusUnauthorized,
env.metricsRequest("", "").Code,
"no credentials must not reach the metrics handler",
)
assert.Equal(
t, http.StatusUnauthorized,
env.metricsRequest(metricsUser, "").Code,
"an empty password must not reach the metrics handler",
)
assert.Equal(
t, http.StatusUnauthorized,
env.metricsRequest(metricsUser, "wrong").Code,
)
ok := env.metricsRequest(metricsUser, metricsAuthValue)
assert.Equal(t, http.StatusOK, ok.Code)
assert.Contains(t, ok.Body.String(), "go_goroutines")
}
// TestMetricsRouteUnmountedOnHalfSetConfig pins the defect from
// https://git.eeqj.de/sneak/webhooker/issues/205 at the routing
// layer. Config rejects a half-set pair at startup, so this Config
// cannot be reached from the environment; the assertion is that the
// route tree does not publish an endpoint accepting an empty
// password even when handed one anyway, because the mount and the
// startup log's hasMetricsAuth read the same value.
func TestMetricsRouteUnmountedOnHalfSetConfig(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
name string
username string
password string
}{
{name: "username only", username: metricsUser},
{name: "password only", password: metricsAuthValue},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
cfg := metricsConfig(t, tc.username, tc.password)
env := newTestEnvWithConfig(t, cfg)
assert.False(t, cfg.MetricsAuthEnabled())
assert.Equal(
t, http.StatusNotFound,
env.metricsRequest(
tc.username, tc.password,
).Code,
)
})
}
}

View File

@@ -50,6 +50,13 @@ const (
minSentryFlush = 250 * time.Millisecond
)
// ListenFailureExitCode is the status the process exits with when the
// HTTP listener cannot be established, or dies for a reason other
// than a requested shutdown. It must stay non-zero: systemd
// `Restart=on-failure` and Docker's restart policies key off it, and a
// zero exit would read as a deliberate stop.
const ListenFailureExitCode = 1
// SentryFlushBudget reports how long the Sentry flush may run when
// remaining is the time left on the fx stop context after the HTTP
// drain. sentry.Flush takes a bare duration and honours no context,
@@ -75,13 +82,13 @@ type ServerParams struct {
Config *config.Config
Middleware *middleware.Middleware
Handlers *handlers.Handlers
Shutdowner fx.Shutdowner
}
// Server is the main HTTP server that wires up routes and manages
// graceful shutdown.
type Server struct {
startupTime time.Time
exitCode int
sentryEnabled bool
log *slog.Logger
cancelFunc context.CancelFunc
@@ -159,7 +166,12 @@ func (s *Server) enableSentry() {
s.sentryEnabled = true
}
func (s *Server) serve() int {
// serve installs the signal watcher, starts the listener and blocks
// until the server's context is cancelled. The process exit status is
// fx's to decide — from a signal, or from the code
// shutdownOnListenFailure hands the Shutdowner — so this reports
// nothing back to its caller.
func (s *Server) serve() {
ctx, cancelFunc := context.WithCancel(context.Background())
s.cancelFunc = cancelFunc
@@ -185,7 +197,30 @@ func (s *Server) serve() int {
<-ctx.Done()
// Shutdown is handled by the fx OnStop hook (cleanShutdown).
// Do not call cleanShutdown() here to avoid double invocation.
return s.exitCode
}
// shutdownOnListenFailure ends the application after the HTTP
// listener failed. The fx OnStart hook returns as soon as the serving
// goroutine is spawned, so nothing downstream of it ever learns that
// the listen failed: fx reports RUNNING and the process sits alive
// with nothing bound, which is invisible to systemd and Docker
// restart policies. Asking the Shutdowner to stop the app with a
// non-zero code is what turns that into a visible failure.
//
// The context cancel that follows only unwinds serve()'s own wait.
// The shutdown itself runs through fx's normal stop sequence, so the
// clean-shutdown drain in cleanShutdown is reached unchanged.
func (s *Server) shutdownOnListenFailure() {
err := s.params.Shutdowner.Shutdown(
fx.ExitCode(ListenFailureExitCode),
)
if err != nil {
s.log.Error("shutdown request failed", "error", err)
}
if s.cancelFunc != nil {
s.cancelFunc()
}
}
func (s *Server) cleanupForExit() {
@@ -193,9 +228,6 @@ func (s *Server) cleanupForExit() {
}
func (s *Server) cleanShutdown(ctx context.Context) {
// initiate clean shutdown
s.exitCode = 0
ctxShutdown, shutdownCancel := context.WithTimeout(
ctx, ShutdownTimeout,
)