Compare commits
3 Commits
a0e4e32e3e
...
4884581fc5
| Author | SHA1 | Date | |
|---|---|---|---|
| 4884581fc5 | |||
| 76725cffc4 | |||
| 977fe87588 |
319
README.md
319
README.md
@@ -113,7 +113,7 @@ TTY detection, and security headers are always applied.
|
||||
| `RETENTION_SWEEP_INTERVAL` | How often the retention reaper and archive sweeper run (Go duration, must be positive) | `1h` |
|
||||
| `SESSION_IDLE_TIMEOUT` | Idle session timeout (Go duration) | `24h` |
|
||||
| `RECEIVER_RATE_LIMIT` | Receiver requests/minute per IP per entrypoint (10x that per IP across the route) | `120` |
|
||||
| `TRUSTED_PROXIES` | CIDRs whose forwarded headers are trusted (unset: all clients behind a proxy share one rate-limit bucket) | `""` (none) |
|
||||
| `TRUSTED_PROXIES` | CIDRs whose forwarded headers are trusted (unset: all clients behind a proxy share one rate-limit bucket; a correct login password is never throttled either way) | `""` (none) |
|
||||
|
||||
#### Trusted proxies
|
||||
|
||||
@@ -136,13 +136,15 @@ That default is safe against forged headers, but leaving it unset in
|
||||
production has a cost you must know about. Production runs behind a
|
||||
TLS-terminating reverse proxy, so with `TRUSTED_PROXIES` unset every
|
||||
request keys on the proxy's own address and all clients share a single
|
||||
bucket per limit. For the login and password-change limits that is a
|
||||
denial of service anyone can perform: a steady five POSTs per minute
|
||||
from any address on the internet keeps the shared login bucket full,
|
||||
and the operator's own login then returns HTTP 429 for as long as the
|
||||
trickle continues. There is no second administrative path and no
|
||||
bypass. Restarting the service clears the in-memory buckets, but a
|
||||
sustained trickle re-locks them immediately.
|
||||
bucket per limit. The receiver limits become service-wide ceilings,
|
||||
and the login endpoint's failure counting collapses onto one key, so a
|
||||
stranger's wrong passwords throttle every other client's wrong
|
||||
passwords.
|
||||
|
||||
What it cannot do is lock the operator out. The login endpoint
|
||||
verifies credentials **before** it consults any limit and charges only
|
||||
failures, so a correct password is never throttled no matter how full
|
||||
the bucket is. See [Rate Limiting](#rate-limiting).
|
||||
|
||||
The remedy is to set `TRUSTED_PROXIES` to your reverse proxy's
|
||||
address, which restores per-client buckets. webhooker logs a warning
|
||||
@@ -382,11 +384,12 @@ It uses:
|
||||
- **[gorilla/csrf](https://github.com/gorilla/csrf)** for CSRF
|
||||
protection (cookie-based double-submit tokens)
|
||||
- **[go-chi/httprate](https://github.com/go-chi/httprate)** for
|
||||
sliding-window rate limiting of the login, password-change and
|
||||
webhook receiver endpoints. The bucket is per client IP only when
|
||||
sliding-window rate limiting of the password-change and webhook
|
||||
receiver endpoints. The bucket is per client IP only when
|
||||
`TRUSTED_PROXIES` names the reverse proxy; unset, every client
|
||||
behind that proxy shares one bucket per limit (see
|
||||
[Rate Limiting](#rate-limiting))
|
||||
behind that proxy shares one bucket per limit. The login endpoint
|
||||
counts failed attempts itself instead, so that a correct password is
|
||||
never throttled (see [Rate Limiting](#rate-limiting))
|
||||
- **[Prometheus](https://prometheus.io)** for metrics, served at
|
||||
`/metrics` behind basic auth
|
||||
- **[Sentry](https://sentry.io)** for optional error reporting
|
||||
@@ -991,15 +994,16 @@ costs; log volume it caps rather than eliminates. A path that names no
|
||||
entrypoint is recorded by the handler at `DEBUG`, and the aggregate
|
||||
limiter logs its own rejections at `DEBUG` and without the path, so
|
||||
neither appears at all under the default level. The per-entrypoint
|
||||
limiter is the loud one: it still logs every rejection at `WARN` with
|
||||
the request path, which on this route is attacker-controlled text. A
|
||||
client hammering a single invented path is served `RECEIVER_RATE_LIMIT`
|
||||
limiter is the loud one: it logs every rejection at `WARN` with the
|
||||
request path, which on this route is attacker-controlled text. A client
|
||||
hammering a single invented path is served `RECEIVER_RATE_LIMIT`
|
||||
requests and has the rest of its aggregate budget rejected there, so
|
||||
the aggregate limit is what bounds those `WARN` lines — to under ten
|
||||
times `RECEIVER_RATE_LIMIT` per minute per client IP, 1080 at the
|
||||
defaults, where before it there was no bound at all. The access log is
|
||||
bounded by neither limit: every request is recorded once at `INFO`,
|
||||
served or rejected alike.
|
||||
the aggregate limit is what bounds the _number_ of those `WARN` lines —
|
||||
to under ten times `RECEIVER_RATE_LIMIT` per minute per client IP, 1080
|
||||
at the defaults, where before it there was no bound at all. Their
|
||||
_width_ is bounded by the field budgets below, the same ones the access
|
||||
log spends. The access log is bounded by neither limit: every request
|
||||
is recorded once at `INFO`, served or rejected alike.
|
||||
|
||||
What the access log does bound is the _content_ of those lines. A 3xx
|
||||
or 4xx response logs the chi route pattern — `/webhook/{uuid}`,
|
||||
@@ -1022,6 +1026,53 @@ buy the same amplification as an invented path. Nothing debuggable is
|
||||
lost: `page`, on the authenticated pagination links, is the only query
|
||||
parameter this service reads.
|
||||
|
||||
Client-supplied request content does not leave the host by the other
|
||||
route either. The Sentry SDK attaches the request to every event it
|
||||
captures, independently of the access log, and `SendDefaultPII=false`
|
||||
does not cover all of what it copies: the raw query string and the
|
||||
first 10 KiB of the request body are both taken unconditionally, the
|
||||
body precisely because these handlers call `ParseForm`. A `BeforeSend`
|
||||
hook therefore replaces the query string and the body with
|
||||
`(redacted)`, drops cookies and the remote-address environment, and
|
||||
reduces the headers to a fixed allowlist — `Accept`, `Content-Length`,
|
||||
`Content-Type`, `Host`, `Origin`, `Referer`, `User-Agent` and
|
||||
`X-Request-Id`.
|
||||
|
||||
The body is replaced on every route rather than filtered by route, and
|
||||
that is a choice rather than a limitation: the route is reachable from
|
||||
the hook. `sentryhttp`'s recover path puts the request on the context
|
||||
it hands to `RecoverWithContext`, and the SDK carries that context
|
||||
through to `BeforeSend` as `hint.Context`, so
|
||||
`hint.Context.Value(sentry.RequestContextKey)` yields the live request
|
||||
and chi's `RoutePattern()` yields the matched pattern off it. There
|
||||
are two reasons to redact unconditionally anyway. Nothing debuggable
|
||||
is lost:
|
||||
every handler reads its fields with `PostFormValue`, so the body is
|
||||
exactly where the credentials are — the target destination URL, the
|
||||
login password, both password-change fields — and the one route whose
|
||||
body is genuine signal is the receiver, whose body is already stored
|
||||
on the event and served from the UI, so a tracker is not where anyone
|
||||
reads it. And an unconditional rule cannot leak on a route somebody
|
||||
forgets to add to it, which a route-conditional one can.
|
||||
|
||||
The headers are an allowlist for that second reason: the SDK's own
|
||||
filter removes four names and passes everything else, which would ship
|
||||
`X-CSRF-Token` and the shared secrets senders put on the receiver
|
||||
route. What survives still names the failing route — scheme, host,
|
||||
path, method — and `X-Request-Id` ties the event to the local access
|
||||
log line that holds the rest. Nothing dropped is needed for the
|
||||
likeliest use, debugging a CSRF rejection. Its three inputs are the
|
||||
TLS decision, `Origin` and `Referer`; the latter two are kept, and the
|
||||
first is already in the retained URL, because the SDK derives that
|
||||
URL's scheme from `r.TLS != nil || r.Header.Get("X-Forwarded-Proto") == "https"` —
|
||||
byte for byte the predicate `internal/middleware/csrf.go` uses to
|
||||
choose between the `csrf.Secure(true)` and `csrf.Secure(false)`
|
||||
handlers. So dropping `X-Forwarded-Proto` costs nothing. The dropped
|
||||
provider headers (`X-GitHub-Event`, `X-Gitlab-Event` and the like) are
|
||||
real signal but are recorded locally on the event, and
|
||||
`Sentry-Trace`/`Baggage` are already reflected in the event's trace
|
||||
context.
|
||||
|
||||
The remaining client-supplied fields are truncated rather than dropped,
|
||||
each to a fixed budget: 512 bytes for `url`, `useragent` and `referer`,
|
||||
128 for `request_id` (chi passes an inbound `X-Request-Id` header
|
||||
@@ -1063,6 +1114,73 @@ that the rate is not bounded by the limits above on every route:
|
||||
`/.well-known/healthcheck` and `/s/*` sit behind no limiter, so there
|
||||
the multiplier is whatever the deployment will serve.
|
||||
|
||||
**The same ceiling covers every other line the service writes through
|
||||
`slog` that carries text an unauthenticated client supplies.** The
|
||||
access log is not the only line a client can put its own text into, and
|
||||
a budget that held for one line and not the others would be worse than
|
||||
no stated budget at all. Every `slog` call an unauthenticated request
|
||||
can reach spends the same per-field budget through `internal/logfield`,
|
||||
and each carries strictly fewer client-supplied fields than the access
|
||||
log does, so none of them can be wider than it:
|
||||
|
||||
| Log line | Level | Client-chosen value | Reachable unauthenticated |
|
||||
| ------------------------------------------ | ------- | ------------------- | ----------------------------------------- |
|
||||
| `request body exceeds limit` (413) | `WARN` | path, method | yes — `MaxBodySize` precedes `RequireAuth` |
|
||||
| `csrf: token validation failed` (403) | `WARN` | path, method | yes — `CSRF` precedes `RequireAuth` |
|
||||
| `... rate limit exceeded` (429) | `WARN` | path | yes, on the receiver |
|
||||
| `auth middleware: unauthenticated request` | `DEBUG` | path, method | yes, by definition |
|
||||
| `entrypoint not found` | `DEBUG` | entrypoint UUID | yes, on the receiver |
|
||||
| `user not found` / `invalid password` | `DEBUG` | username | yes, on the login form |
|
||||
|
||||
`DEBUG` being off by default is not a bound. An operator turning it on
|
||||
to diagnose a flood must not thereby hand the flood an unbounded write,
|
||||
so those lines are capped too.
|
||||
|
||||
`internal/middleware/logbound_test.go` and
|
||||
`internal/handlers/logbound_test.go` drive 8 KB of client-chosen text at
|
||||
each of these, through both handlers and through every character the
|
||||
handlers escape, and hold each line to the 2,560-byte ceiling — and the
|
||||
whole flood's output to what that ceiling allows, which is the property
|
||||
an operator actually cares about.
|
||||
`internal/logfield/logfield_test.go` measures the per-rune charge
|
||||
against what the handlers really emit, over roughly 3,000 code points on
|
||||
each, so an undercharged rune fails a test rather than quietly
|
||||
falsifying the ceiling.
|
||||
|
||||
What that ceiling does **not** cover, stated here so the figure is not
|
||||
read as more than it is:
|
||||
|
||||
- **Lines carrying an authenticated operator's own input**, which are
|
||||
not truncated at all. `webhook created` logs the submitted `name`
|
||||
verbatim and `target URL blocked by SSRF protection` logs the target
|
||||
host (both `internal/handlers/source_management.go`), as do the
|
||||
`target_name` lines in `internal/delivery/engine.go` and
|
||||
`internal/delivery/target_http.go`. The only bound on any of them is
|
||||
the 1 MB form body cap, so a 100 KB `name` writes a single line of
|
||||
roughly 600 KB — measured. This is deliberate: every one of these
|
||||
requires an authenticated operator on a service with no
|
||||
self-registration, and truncating the operator's own configuration
|
||||
echoed back would cost debuggability against no adversary. It does
|
||||
mean the 2,560-byte figure sizes unauthenticated traffic, not the
|
||||
operator's own administrative requests.
|
||||
- The **`log` delivery target**, which writes the whole inbound event —
|
||||
headers and body — to the log. This one is deliberate: capping it
|
||||
would defeat the target, since emitting the payload is the delivery.
|
||||
It costs nothing unless an authenticated operator creates a target of
|
||||
that type on a specific webhook, and each line it writes is bounded
|
||||
per event by the 1 MB receiver body cap. Adding one is a decision to
|
||||
spend log volume on that webhook's payloads.
|
||||
- **GORM's default logger**, which prints the fully interpolated SQL to
|
||||
stdout on every record-not-found — including the client-chosen path
|
||||
on `/webhook/{uuid}` and the submitted username on the login form.
|
||||
This one is not deliberate and not yet fixed; it does not go through
|
||||
`internal/logger` at all, so no level the operator sets and no budget
|
||||
above applies to it. Tracked at
|
||||
<https://git.eeqj.de/sneak/webhooker/issues/178>. Until it is fixed,
|
||||
an unauthenticated flood can still write text of its own choosing and
|
||||
its own length to the operator's stdout, and the ceiling above
|
||||
describes only the `slog` half of the picture.
|
||||
|
||||
Every limiter here — receiver, login, and password change — identifies
|
||||
the client the same way, through one shared key function: the
|
||||
connection's own address, unless the peer is listed in
|
||||
@@ -1091,14 +1209,117 @@ opposite directions:
|
||||
and all entrypoints, where the per-entrypoint limit's capacity still
|
||||
grows with the number of entrypoints. Any deployment with more than a
|
||||
handful of busy entrypoints must set `TRUSTED_PROXIES`.
|
||||
- For the **login and password-change** limits it costs availability of
|
||||
the only administrative path, which is not safe at all. Five POSTs
|
||||
per minute from any address on the internet keeps the single shared
|
||||
login bucket full, and the operator's own login returns HTTP 429 for
|
||||
as long as that trickle continues. A restart clears the in-memory
|
||||
buckets and a resumed trickle re-locks them. Production deployments
|
||||
must set `TRUSTED_PROXIES`; webhooker warns at startup whenever it is
|
||||
empty, in any environment.
|
||||
- For the **login and password-change** limits it costs precision, not
|
||||
availability. Login failures from every client land in one counter,
|
||||
so a stranger's wrong passwords make the operator's own wrong
|
||||
passwords answer `429` sooner; the operator's _correct_ password is
|
||||
never affected, because it is never counted. Production deployments
|
||||
should still set `TRUSTED_PROXIES`; webhooker warns at startup
|
||||
whenever it is empty, in any environment.
|
||||
|
||||
#### The login endpoint
|
||||
|
||||
The login `POST` is the one endpoint with no pre-emptive limiter in
|
||||
front of it, and that is deliberate. A limiter that spends budget on
|
||||
arrival is a lockout in this deployment shape: sharing one bucket, a
|
||||
stranger sending five POSTs a minute — about 0.08 requests per second,
|
||||
from anywhere — keeps it permanently full, and the operator has no
|
||||
second administrative path. So the handler inverts the order:
|
||||
|
||||
1. **Credentials are verified first, and only a failed attempt spends
|
||||
budget.** A correct password is never rate-limited, whatever the
|
||||
counters hold. This is what guarantees the admin UI stays
|
||||
reachable.
|
||||
2. **Failures are counted per (client bucket, submitted username)**,
|
||||
five per minute, after which further _failures_ from that pair are
|
||||
answered `429` with a `Retry-After`. That `429` is a label on the
|
||||
response, not a gate in front of the work: the credential check has
|
||||
already run by the time the counter is consulted, so a throttled
|
||||
client's guess is still evaluated. See the guessing rate below. A
|
||||
successful login clears the counter, so mistyping a few times and
|
||||
then getting it right leaves you unthrottled. Because the submitted
|
||||
username is attacker-controlled, at most 1024 username counters and
|
||||
1024 fallback address counters are tracked; past the first cap
|
||||
failures fall back to the address counter, and past both they are
|
||||
answered as throttled without being recorded. Total tracked state
|
||||
is under half a megabyte and does not grow with the number of
|
||||
usernames an attacker invents.
|
||||
3. **Concurrent password verifications are capped at two, and the
|
||||
queue for them at 16.** Verifying before counting means every login
|
||||
request costs an Argon2id hash, and Argon2id here is 64 MB per
|
||||
hash — two slots is a 128 MB ceiling on password hashing. Every
|
||||
endpoint that hashes a password takes a slot, including the
|
||||
password-change endpoint, which holds one across both the
|
||||
verification and the new hash. A request that waits five seconds
|
||||
without getting a slot is answered `503 Service Unavailable` and no
|
||||
hash is computed for it. The wait alone does not bound memory, only
|
||||
how long one request holds some, so the number of waiters is capped
|
||||
as well. Size the queue from what a parked waiter actually retains,
|
||||
not from the 1 MB body cap: that caps the raw body read, while the
|
||||
body-cap, CSRF and form-parsing middleware all run before the
|
||||
guard, so a waiter holds its parsed form plus its request header
|
||||
block for the whole wait. Measured on the pinned Go 1.26.1
|
||||
toolchain, as the heap delta with 64 waiters parked in the handler,
|
||||
an ordinary two-field login form retains ~0 MB, a 1 MB urlencoded
|
||||
body at Go's 10,000-parameter parse cap retains 2.82 MB (3.09 MB
|
||||
with `%41` escapes), and the ~0.9 MB of headers the 1 MB header cap
|
||||
allows takes it to **4.18 MB** — the retained parse and the headers
|
||||
dominate, not the raw body. So the cap is 16 waiters: 16 x 4.18 MB
|
||||
is about 67 MB of committed queue memory, and two slots drain a
|
||||
full 16-deep queue in roughly 0.6 s, far inside the five-second
|
||||
deadline. A request arriving past the cap is shed with `503`
|
||||
immediately instead of joining the queue. **Peak commitment for the
|
||||
endpoint is therefore about 203 MB**: 128 MB of Argon2id, plus the
|
||||
18 requests holding a parsed form — 16 queued and the 2 being
|
||||
hashed — at about 75 MB. That 203 MB is _live_ commitment, not
|
||||
resident size: the Go collector lets the heap reach roughly twice
|
||||
the live set before collecting, with transient parse garbage on top
|
||||
of it. The independent review of this endpoint fired 18 adversarial
|
||||
requests at an idle guard and measured a peak `HeapAlloc` of
|
||||
392 MB. **Provision on the order of 400 MB**, not for the 203 MB
|
||||
itemised here and not for the hashing budget alone.
|
||||
|
||||
An unknown username is verified against a dummy hash rather than
|
||||
rejected early, so a nonexistent account costs the same time as a real
|
||||
one and the response cannot be used to enumerate usernames.
|
||||
|
||||
**This raises online guessing throughput by about 300x, and that is
|
||||
the trade.** Because the credential check always precedes the counter,
|
||||
what bounds online brute force is the semaphore, not the failure
|
||||
counter. Two slots at the cost of one Argon2id verification is on the
|
||||
order of **27 guesses per second, about 2.3 million per day**, against
|
||||
5 per minute under the pre-emptive limiter this replaced. Treat that
|
||||
figure as a lower bound rather than a ceiling: it was measured with
|
||||
Go's race detector enabled, so real hardware verifies faster and
|
||||
guesses faster. Choose the admin password to survive millions of
|
||||
online guesses per day — a long random passphrase, not a memorable
|
||||
one. Rate-limiting `POST /pages/login` at the reverse proxy, where the
|
||||
real client address is visible, is the way to put a cheaper bound back
|
||||
on top.
|
||||
|
||||
The residual exposure is a bounded, self-clearing loss of login
|
||||
**availability** — not merely of latency. A flood can keep both
|
||||
verification slots busy, and a request that neither gets a slot within
|
||||
five seconds nor finds room in the queue is answered `503`. Above
|
||||
roughly 27 requests per second the operator is not served slowly, it
|
||||
is shed: its chance per attempt is about the ratio of service rate to
|
||||
flood rate, so at 400 requests per second it is roughly one attempt in
|
||||
fourteen. A sufficiently determined flood still denies login for as
|
||||
long as it runs.
|
||||
|
||||
What changed is the price and the aftermath. Denying login used to
|
||||
cost an attacker 0.08 requests per second from anywhere; it now costs
|
||||
30 or more sustained, about 400 times as much. Nothing accumulates
|
||||
while the flood runs, nothing needs resetting when it stops, and the
|
||||
operator's correct password succeeds on the first attempt afterwards.
|
||||
Restarting the service is **not** a remedy: a restart clears the
|
||||
failure counters, which are not what is saturated, and the flood
|
||||
re-fills both verification slots on its first two requests. The
|
||||
remedies are to block the source at the reverse proxy, or to
|
||||
rate-limit `POST /pages/login` there — the one place a limit can be
|
||||
applied without reintroducing the lockout, because the proxy sees the
|
||||
real client address. Setting `TRUSTED_PROXIES` does not stop the
|
||||
saturation, but it makes the source visible in the failure logs.
|
||||
|
||||
Finer-grained per-webhook rate limits (configured in the web UI and
|
||||
enforced in the webhook handler) can layer on top of this env-level
|
||||
@@ -1119,8 +1340,8 @@ abuse limit later; they are tracked as future work.
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | --------------- | ----------- |
|
||||
| `GET` | `/pages/login` | Login page (not rate limited; the limiter applies to POST only) |
|
||||
| `POST` | `/pages/login` | Login form submission (5 per minute per bucket, then 429) |
|
||||
| `GET` | `/pages/login` | Login page (not rate limited) |
|
||||
| `POST` | `/pages/login` | Login form submission. Credentials are verified before any limit is consulted, so a correct password is never throttled; 5 FAILED attempts per minute per bucket per submitted username, then `429`. `503` if no verification slot frees up within 5s, or immediately if 16 requests are already queued for one (see [Rate Limiting](#rate-limiting)) |
|
||||
| `POST` | `/pages/logout` | Logout (destroys session) |
|
||||
|
||||
#### Authenticated Endpoints
|
||||
@@ -1128,7 +1349,7 @@ abuse limit later; they are tracked as future work.
|
||||
| Method | Path | Description |
|
||||
| ------ | ------------------------ | ----------- |
|
||||
| `GET` | `/user/{username}` | User profile page |
|
||||
| `POST` | `/user/{username}/password` | Change the user's password (5 per minute per bucket, then 429) |
|
||||
| `POST` | `/user/{username}/password` | Change the user's password (5 per minute per bucket, then `429`; `503` if no verification slot frees up within 5s, or immediately if 16 requests are already queued for one) |
|
||||
| `GET` | `/sources` | List user's webhooks |
|
||||
| `GET` | `/sources/new` | Create webhook form |
|
||||
| `POST` | `/sources/new` | Create webhook submission |
|
||||
@@ -1230,6 +1451,7 @@ webhooker/
|
||||
│ │ ├── middleware.go # Logging, CORS, Auth, Metrics, MetricsAuth, SecurityHeaders, MaxBodySize
|
||||
│ │ ├── csrf.go # CSRF protection middleware (gorilla/csrf)
|
||||
│ │ ├── ratelimit.go # Per-IP rate limiting middleware (go-chi/httprate)
|
||||
│ │ ├── loginguard.go # Login failure counters and the Argon2id verification semaphore
|
||||
│ │ └── testing.go # NewForTest: Middleware without the fx lifecycle
|
||||
│ ├── server/
|
||||
│ │ ├── server.go # Server struct, fx lifecycle, signal handling
|
||||
@@ -1327,9 +1549,11 @@ one that lies about its length, is hard-capped by
|
||||
Those same four route groups then apply **CSRF** and **NoCache**
|
||||
(`Cache-Control: no-store`, `Pragma: no-cache`), and every group except
|
||||
`/pages` applies **RequireAuth**. The rate limiters are per-route
|
||||
rather than global: **LoginRateLimit** on `/pages/login`,
|
||||
**PasswordChangeRateLimit** on `/user/{username}/password`, and
|
||||
**ReceiverRateLimit** on `/webhook/{uuid}`.
|
||||
rather than global: **PasswordChangeRateLimit** on
|
||||
`/user/{username}/password` and **ReceiverRateLimit** on
|
||||
`/webhook/{uuid}`. There is deliberately none on `/pages/login` — that
|
||||
endpoint counts failures inside the handler, after the credential
|
||||
check, see [The login endpoint](#the-login-endpoint).
|
||||
|
||||
### Authentication
|
||||
|
||||
@@ -1367,14 +1591,27 @@ rather than global: **LoginRateLimit** on `/pages/login`,
|
||||
both at target creation time (URL validation) and at delivery time
|
||||
(custom HTTP transport with SSRF-safe dialer that validates resolved
|
||||
IPs before connecting, preventing DNS rebinding attacks)
|
||||
- **Login rate limiting** via [go-chi/httprate](https://github.com/go-chi/httprate):
|
||||
sliding-window rate limiter on the login endpoint, 5 POST attempts
|
||||
per minute per bucket, to slow brute-force attacks. GET requests to
|
||||
the login page are not limited. The password-change endpoint carries
|
||||
the same 5-per-minute limit. The bucket is per client IP only when
|
||||
- **Login limiting is inverted, deliberately.** The login `POST` has
|
||||
no pre-emptive rate limiter in front of it. Credentials are
|
||||
verified first and only a _failed_ attempt spends budget, so a
|
||||
correct password is never throttled and no flood of wrong ones can
|
||||
deny the operator the only administrative path. Failures are
|
||||
counted per (bucket, submitted username), five per minute, after
|
||||
which further failures are answered `429` with a `Retry-After`.
|
||||
What bounds brute force is not that counter but the cap of two
|
||||
concurrent Argon2id verifications: a throttled client's guess is
|
||||
still evaluated, so roughly 27 guesses a second get through and the
|
||||
admin password has to carry that load (see
|
||||
[The login endpoint](#the-login-endpoint)). `GET` requests to the
|
||||
login page are not limited
|
||||
- **Password-change rate limiting** via [go-chi/httprate](https://github.com/go-chi/httprate):
|
||||
sliding-window rate limiter, 5 POST attempts per minute per bucket.
|
||||
It runs behind session auth, so only a client already holding a
|
||||
valid session reaches it, and an operator throttled out of changing
|
||||
a password can still log in. The bucket is per client IP only when
|
||||
`TRUSTED_PROXIES` names the reverse proxy; unset, every client
|
||||
shares one bucket and the login becomes remotely deniable (see
|
||||
[Rate Limiting](#rate-limiting)). webhooker warns at startup
|
||||
shares one bucket, which costs precision rather than availability
|
||||
(see [Rate Limiting](#rate-limiting)). webhooker warns at startup
|
||||
whenever `TRUSTED_PROXIES` is empty
|
||||
- Prometheus metrics behind basic auth
|
||||
- Static assets embedded in binary (no filesystem access needed at
|
||||
|
||||
@@ -430,10 +430,14 @@ func loadFromEnv() (*Config, error) {
|
||||
// what is in front of the process, which this code cannot observe:
|
||||
// with nothing in front, the peer is the client and the limits are
|
||||
// per-client as intended; behind a reverse proxy the peer is the proxy
|
||||
// for every request, so all clients share one bucket per limiter. The
|
||||
// login limiter's bucket is the dangerous one: any remote client can
|
||||
// keep it full, which denies the only administrative login to everyone
|
||||
// until the process restarts.
|
||||
// for every request, so all clients share one bucket per limiter.
|
||||
//
|
||||
// The login endpoint no longer spends budget on arrival — it verifies
|
||||
// credentials first and charges only failures — so a shared bucket
|
||||
// cannot deny the operator a correct password. What it does collapse
|
||||
// is the failure counting: one client's wrong passwords throttle
|
||||
// everyone else's wrong passwords, and the receiver's limits become
|
||||
// service-wide ceilings.
|
||||
//
|
||||
// The warning is deliberately not gated on WEBHOOKER_ENVIRONMENT. That
|
||||
// variable defaults to dev, so gating on it would silence the warning
|
||||
@@ -454,11 +458,11 @@ func (c *Config) warnSharedRateLimitBucket(log *slog.Logger) {
|
||||
"this process that is the client itself and the limits "+
|
||||
"are per-client as intended. Behind a reverse proxy the "+
|
||||
"peer is the proxy on every request, so all clients "+
|
||||
"share one bucket per limit and any remote client can "+
|
||||
"keep the login limit full, denying the admin login — "+
|
||||
"the only administrative path — until restart. If "+
|
||||
"anything proxies to this process, set TRUSTED_PROXIES "+
|
||||
"to its address.",
|
||||
"share one bucket per limit: the receiver limits become "+
|
||||
"service-wide ceilings, and one client's failed logins "+
|
||||
"throttle every other client's failed logins — a "+
|
||||
"correct password still gets in. If anything proxies to "+
|
||||
"this process, set TRUSTED_PROXIES to its address.",
|
||||
"environment", c.Environment,
|
||||
"trustedProxies", len(c.TrustedProxies),
|
||||
)
|
||||
|
||||
@@ -629,8 +629,9 @@ func testTrustedProxiesSuccess(
|
||||
|
||||
// TestSharedRateLimitBucketWarning covers the startup warning that
|
||||
// tells an operator a deployment behind a reverse proxy shares one
|
||||
// rate-limit bucket between every client, which makes the admin login
|
||||
// remotely deniable. It must fire whenever TRUSTED_PROXIES is empty,
|
||||
// rate-limit bucket between every client, which turns the receiver
|
||||
// limits into service-wide ceilings and collapses login failure
|
||||
// counting. It must fire whenever TRUSTED_PROXIES is empty,
|
||||
// in any environment: WEBHOOKER_ENVIRONMENT defaults to dev, so gating
|
||||
// on it would silence the warning for exactly the operator who never
|
||||
// configured the deployment. It stays quiet once proxies are named.
|
||||
@@ -707,7 +708,15 @@ func TestSharedRateLimitBucketWarning(t *testing.T) {
|
||||
assert.Contains(t, logged, `"level":"WARN"`)
|
||||
assert.Contains(t, logged, "TRUSTED_PROXIES")
|
||||
assert.Contains(t, logged, "share one bucket")
|
||||
assert.Contains(t, logged, "denying the admin login")
|
||||
assert.Contains(
|
||||
t, logged, "throttle every other client's failed logins",
|
||||
)
|
||||
// The warning must not claim a lockout the login
|
||||
// endpoint no longer permits: credentials are verified
|
||||
// before any budget is spent.
|
||||
assert.Contains(
|
||||
t, logged, "a correct password still gets in",
|
||||
)
|
||||
// The text must stay accurate for a developer with
|
||||
// nothing in front of the process, where an empty
|
||||
// list costs nothing.
|
||||
|
||||
@@ -65,3 +65,9 @@ func (r *RetentionReaper) ExportWedgeLoop(
|
||||
func (r *RetentionReaper) ExportSetInterval(d time.Duration) {
|
||||
r.interval = d
|
||||
}
|
||||
|
||||
// DummyPasswordHashForTest exposes the encoded hash that unknown
|
||||
// usernames are verified against.
|
||||
func DummyPasswordHashForTest() string {
|
||||
return dummyPasswordHash()
|
||||
}
|
||||
|
||||
@@ -2,12 +2,16 @@ package database
|
||||
|
||||
import "time"
|
||||
|
||||
// APIKey represents an API key for a user
|
||||
// APIKey represents an API key for a user.
|
||||
//
|
||||
// Key is a bearer credential, so it is never marshalled with the
|
||||
// model. A creation handler that has to show it once returns it in its
|
||||
// own response type.
|
||||
type APIKey struct {
|
||||
BaseModel
|
||||
|
||||
UserID string `gorm:"type:uuid;not null" json:"userId"`
|
||||
Key string `gorm:"uniqueIndex;not null" json:"key"`
|
||||
Key string `gorm:"uniqueIndex;not null" json:"-"`
|
||||
Description string `json:"description"`
|
||||
LastUsedAt *time.Time `json:"lastUsedAt,omitempty"`
|
||||
|
||||
|
||||
107
internal/database/model_secrets_test.go
Normal file
107
internal/database/model_secrets_test.go
Normal file
@@ -0,0 +1,107 @@
|
||||
package database_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"sneak.berlin/go/webhooker/internal/database"
|
||||
)
|
||||
|
||||
// keptField is a non-secret value planted alongside each secret, so
|
||||
// the assertions below cannot pass by the model marshalling to nothing.
|
||||
const keptField = "keepme"
|
||||
|
||||
// marshalModel encodes a model the way a future JSON handler would.
|
||||
func marshalModel(t *testing.T, v any) string {
|
||||
t.Helper()
|
||||
|
||||
encoded, err := json.Marshal(v)
|
||||
require.NoError(t, err)
|
||||
|
||||
return string(encoded)
|
||||
}
|
||||
|
||||
// TestModelsDoNotMarshalTheirSecrets pins the barrier for the JSON
|
||||
// path. The /api/v1 route group exists and is empty; delivery's
|
||||
// TargetView masks the credential for the HTML path only, so without
|
||||
// these tags the first handler that marshals a model serialises the
|
||||
// secret with it. Each field below is a live credential:
|
||||
//
|
||||
// - Target.Config holds an incoming-webhook URL whose path segments
|
||||
// are the bearer token.
|
||||
// - APIKey.Key is a bearer token outright.
|
||||
// - Setting.Value holds the session encryption key.
|
||||
// - User.Password holds the Argon2 hash, and was already tagged.
|
||||
func TestModelsDoNotMarshalTheirSecrets(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const marker = "QQMODELMARKERQQ"
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
model any
|
||||
}{
|
||||
{
|
||||
name: "target config",
|
||||
model: database.Target{
|
||||
Name: keptField,
|
||||
Type: database.TargetTypeSlack,
|
||||
Config: `{"webhookUrl":"https://h/s/` + marker + `"}`,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "api key",
|
||||
model: database.APIKey{
|
||||
Description: keptField,
|
||||
Key: marker,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "setting value",
|
||||
model: database.Setting{
|
||||
Key: keptField,
|
||||
Value: marker,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "user password hash",
|
||||
model: database.User{
|
||||
Username: keptField,
|
||||
Password: marker,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
encoded := marshalModel(t, tc.model)
|
||||
|
||||
assert.NotContains(t, encoded, marker)
|
||||
assert.Contains(t, encoded, keptField)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestWebhookMarshalsNoTargetConfig covers the nested case: a webhook
|
||||
// marshalled with its targets preloaded must not carry the credential
|
||||
// through the association either.
|
||||
func TestWebhookMarshalsNoTargetConfig(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const marker = "QQNESTEDMARKERQQ"
|
||||
|
||||
encoded := marshalModel(t, database.Webhook{
|
||||
Name: keptField,
|
||||
Targets: []database.Target{{
|
||||
Name: "slack",
|
||||
Config: `{"webhookUrl":"https://h/s/` + marker + `"}`,
|
||||
}},
|
||||
})
|
||||
|
||||
assert.NotContains(t, encoded, marker)
|
||||
assert.Contains(t, encoded, keptField)
|
||||
}
|
||||
@@ -4,5 +4,8 @@ package database
|
||||
// Used for auto-generated values like the session encryption key.
|
||||
type Setting struct {
|
||||
Key string `gorm:"primaryKey" json:"key"`
|
||||
Value string `gorm:"type:text;not null" json:"value"`
|
||||
|
||||
// Value holds the session encryption key, so it is never
|
||||
// marshalled with the model.
|
||||
Value string `gorm:"type:text;not null" json:"-"`
|
||||
}
|
||||
|
||||
@@ -20,8 +20,14 @@ type Target struct {
|
||||
Type TargetType `gorm:"not null" json:"type"`
|
||||
Active bool `gorm:"default:true" json:"active"`
|
||||
|
||||
// Configuration fields (JSON stored based on type)
|
||||
Config string `gorm:"type:text" json:"config"` // JSON configuration
|
||||
// Configuration fields (JSON stored based on type).
|
||||
//
|
||||
// json:"-" because the blob holds the target's credential — a
|
||||
// Slack incoming-webhook URL, or an http destination whose path
|
||||
// segments are the secret. delivery.TargetView is the masking
|
||||
// barrier for the HTML path; this tag is the barrier for any
|
||||
// handler that marshals the model itself.
|
||||
Config string `gorm:"type:text" json:"-"` // JSON configuration
|
||||
|
||||
// For HTTP targets (max_retries=0 means fire-and-forget,
|
||||
// >0 enables retries with backoff)
|
||||
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
"fmt"
|
||||
"math/big"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"golang.org/x/crypto/argon2"
|
||||
)
|
||||
@@ -29,6 +30,10 @@ const hashParts = 6
|
||||
// triggers per-character-class complexity enforcement.
|
||||
const minPasswordComplexityLen = 4
|
||||
|
||||
// dummyPasswordLen is the length of the throwaway password behind
|
||||
// dummyPasswordHash.
|
||||
const dummyPasswordLen = 32
|
||||
|
||||
// Sentinel errors returned by decodeHash.
|
||||
var (
|
||||
errInvalidHashFormat = errors.New("invalid hash format")
|
||||
@@ -122,6 +127,38 @@ func VerifyPassword(
|
||||
return subtle.ConstantTimeCompare(hash, otherHash) == 1, nil
|
||||
}
|
||||
|
||||
// dummyPasswordHash is an encoded Argon2id hash of a random
|
||||
// password, computed once on first use. Nothing can match it: the
|
||||
// password it encodes is discarded as soon as it is hashed. It is
|
||||
// process-wide because building it per request would add a second
|
||||
// 64 MB Argon2id pass to every login for an unknown username.
|
||||
//
|
||||
//nolint:gochecknoglobals // computed once, see above
|
||||
var dummyPasswordHash = sync.OnceValue(func() string {
|
||||
password, err := GenerateRandomPassword(dummyPasswordLen)
|
||||
if err != nil {
|
||||
panic(fmt.Sprintf("generating the dummy password: %v", err))
|
||||
}
|
||||
|
||||
hash, err := HashPassword(password)
|
||||
if err != nil {
|
||||
panic(fmt.Sprintf("hashing the dummy password: %v", err))
|
||||
}
|
||||
|
||||
return hash
|
||||
})
|
||||
|
||||
// VerifyDummyPassword performs a credential verification that cannot
|
||||
// succeed, at the same cost as a real one.
|
||||
//
|
||||
// Login must charge an unknown username the same work as a known
|
||||
// one. Returning early for an account that does not exist answers in
|
||||
// microseconds where a real account takes tens of milliseconds, which
|
||||
// is a username oracle any client can read off the response time.
|
||||
func VerifyDummyPassword(password string) {
|
||||
_, _ = VerifyPassword(password, dummyPasswordHash())
|
||||
}
|
||||
|
||||
// decodeHash extracts parameters, salt, and hash from an
|
||||
// encoded hash string.
|
||||
func decodeHash(
|
||||
|
||||
@@ -191,3 +191,41 @@ func TestHashPasswordUniqueness(t *testing.T) {
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// TestVerifyDummyPassword_DoesRealWork covers the anti-enumeration
|
||||
// path. Login charges an unknown username a verification against a
|
||||
// dummy hash so that a nonexistent account is not answered in
|
||||
// microseconds where a real one takes tens of milliseconds. That only
|
||||
// works if the dummy hash is a real, decodable Argon2id hash: a
|
||||
// malformed one would make VerifyPassword fail on the decode and
|
||||
// return before hashing anything.
|
||||
func TestVerifyDummyPassword_DoesRealWork(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// Runs the OnceValue that builds the dummy hash, so a panic in
|
||||
// it surfaces here rather than on a live login.
|
||||
database.VerifyDummyPassword("whatever was submitted")
|
||||
|
||||
dummy := database.DummyPasswordHashForTest()
|
||||
|
||||
// A hash the verifier cannot decode would make VerifyPassword
|
||||
// return on the decode error, before hashing anything — the
|
||||
// timing oracle this path exists to close.
|
||||
valid, err := database.VerifyPassword("whatever", dummy)
|
||||
if err != nil {
|
||||
t.Fatalf(
|
||||
"the dummy hash must decode like a real one: %v", err,
|
||||
)
|
||||
}
|
||||
|
||||
if valid {
|
||||
t.Error("nothing may authenticate against the dummy hash")
|
||||
}
|
||||
|
||||
if !strings.HasPrefix(dummy, "$argon2id$") {
|
||||
t.Errorf(
|
||||
"the dummy hash must use the same algorithm as real "+
|
||||
"hashes, got %q", dummy,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -11,6 +11,17 @@ import (
|
||||
// inbound webhook — the full request body and headers, plus
|
||||
// the method, content type, and the webhook and entrypoint
|
||||
// ids — then records a single successful attempt.
|
||||
//
|
||||
// This is the one log call in the service that deliberately writes
|
||||
// unbounded client-chosen bytes, so it is the one exception to the
|
||||
// per-field budgets in internal/logfield and to the ceiling stated on
|
||||
// middleware.MaxAccessLogLineBytes. Capping here would defeat the
|
||||
// target: emitting the payload IS the delivery. It costs nothing by
|
||||
// default — an authenticated operator has to create a target of this
|
||||
// type on a specific webhook before a single line is written — and the
|
||||
// bytes it writes are bounded per event by maxWebhookBodySize (1 MB).
|
||||
// An operator who adds one is choosing to spend log volume on the
|
||||
// payloads that webhook receives.
|
||||
type logTarget struct {
|
||||
eng *Engine
|
||||
}
|
||||
|
||||
@@ -2,8 +2,10 @@ package handlers
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strconv"
|
||||
|
||||
"sneak.berlin/go/webhooker/internal/database"
|
||||
"sneak.berlin/go/webhooker/internal/logfield"
|
||||
)
|
||||
|
||||
// HandleLoginPage returns a handler for the login page (GET)
|
||||
@@ -39,8 +41,10 @@ func (h *Handlers) HandleLoginSubmit() http.HandlerFunc {
|
||||
return
|
||||
}
|
||||
|
||||
username := r.FormValue("username")
|
||||
password := r.FormValue("password")
|
||||
// PostFormValue, not FormValue: the credential must come
|
||||
// from the body, never from the query string.
|
||||
username := r.PostFormValue("username")
|
||||
password := r.PostFormValue("password")
|
||||
|
||||
// Validate input
|
||||
if username == "" || password == "" {
|
||||
@@ -67,7 +71,9 @@ func (h *Handlers) HandleLoginSubmit() http.HandlerFunc {
|
||||
|
||||
h.log.Info(
|
||||
"user logged in",
|
||||
"username", username,
|
||||
"username", logfield.Truncate(
|
||||
username, logfield.MaxBytes,
|
||||
),
|
||||
"user_id", user.ID,
|
||||
)
|
||||
|
||||
@@ -93,6 +99,16 @@ func (h *Handlers) renderLoginError(
|
||||
|
||||
// authenticateUser looks up and verifies a user's credentials.
|
||||
// On failure it writes an HTTP response and returns an error.
|
||||
//
|
||||
// The credential check runs BEFORE any rate-limit budget is
|
||||
// consulted, and only a failed check spends budget. That is what
|
||||
// keeps the single administrative path reachable: behind the reverse
|
||||
// proxy this deployment requires, with TRUSTED_PROXIES unset, every
|
||||
// client shares one bucket, so a limiter spent on arrival lets any
|
||||
// stranger deny the operator's own correct password indefinitely.
|
||||
//
|
||||
// Verifying first means every login POST costs an Argon2id hash, so
|
||||
// the work is taken under a bounded number of verification slots.
|
||||
func (h *Handlers) authenticateUser(
|
||||
w http.ResponseWriter,
|
||||
r *http.Request,
|
||||
@@ -100,16 +116,47 @@ func (h *Handlers) authenticateUser(
|
||||
) (database.User, error) {
|
||||
var user database.User
|
||||
|
||||
release, ok := h.mw.BeginPasswordVerification(r.Context())
|
||||
if !ok {
|
||||
h.log.Warn(
|
||||
"password verification capacity exhausted",
|
||||
"path", r.URL.Path,
|
||||
)
|
||||
h.renderLoginError(
|
||||
w, r,
|
||||
"The server is busy verifying credentials. "+
|
||||
"Please try again.",
|
||||
http.StatusServiceUnavailable,
|
||||
)
|
||||
|
||||
return user, errVerificationBusy
|
||||
}
|
||||
|
||||
defer release()
|
||||
|
||||
err := h.db.DB().Where(
|
||||
"username = ?", username,
|
||||
).First(&user).Error
|
||||
if err != nil {
|
||||
h.log.Debug("user not found", "username", username)
|
||||
h.renderLoginError(
|
||||
w, r,
|
||||
"Invalid username or password",
|
||||
http.StatusUnauthorized,
|
||||
// A username that does not exist is charged the same work
|
||||
// as one that does. Skipping the hash here would answer in
|
||||
// microseconds where a real account takes tens of
|
||||
// milliseconds, handing every client a username oracle.
|
||||
h.dummyVerifications.Add(1)
|
||||
database.VerifyDummyPassword(password)
|
||||
|
||||
// Login is unauthenticated, and the submitted username is
|
||||
// a form field the client fills to any length the 1 MB
|
||||
// body cap allows. On this branch it matched no row, so
|
||||
// nothing else bounds it. The rate limiter caps how often
|
||||
// the line is written, not how wide it is.
|
||||
h.log.Debug(
|
||||
"user not found",
|
||||
"username", logfield.Truncate(
|
||||
username, logfield.MaxBytes,
|
||||
),
|
||||
)
|
||||
h.rejectLogin(w, r, username)
|
||||
|
||||
return user, err
|
||||
}
|
||||
@@ -126,17 +173,60 @@ func (h *Handlers) authenticateUser(
|
||||
}
|
||||
|
||||
if !valid {
|
||||
h.log.Debug("invalid password", "username", username)
|
||||
// Reached only once the username matched a stored row, so
|
||||
// it is bounded by the operator's own data. Capped anyway,
|
||||
// so that every username this unauthenticated endpoint
|
||||
// logs is capped and no reader has to work out which
|
||||
// branch narrowed it.
|
||||
h.log.Debug(
|
||||
"invalid password",
|
||||
"username", logfield.Truncate(
|
||||
username, logfield.MaxBytes,
|
||||
),
|
||||
)
|
||||
h.rejectLogin(w, r, username)
|
||||
|
||||
return user, errInvalidPassword
|
||||
}
|
||||
|
||||
// The password was correct, so forgive whatever failures this
|
||||
// client accumulated: an operator who mistypes a few times and
|
||||
// then gets it right must not stay throttled afterwards.
|
||||
h.mw.ForgiveLoginFailures(r, username)
|
||||
|
||||
return user, nil
|
||||
}
|
||||
|
||||
// rejectLogin counts one failed credential verification and answers
|
||||
// it: 401 while this client still has failure budget against the
|
||||
// submitted username, 429 with a Retry-After once it is spent.
|
||||
//
|
||||
// The 429 throttles wrong passwords only. A correct one never
|
||||
// reaches here, so no amount of failure — from this client or any
|
||||
// other sharing its bucket — can keep the operator out.
|
||||
func (h *Handlers) rejectLogin(
|
||||
w http.ResponseWriter,
|
||||
r *http.Request,
|
||||
username string,
|
||||
) {
|
||||
if !h.mw.RecordLoginFailure(r, username) {
|
||||
h.renderLoginError(
|
||||
w, r,
|
||||
"Invalid username or password",
|
||||
http.StatusUnauthorized,
|
||||
)
|
||||
|
||||
return user, errInvalidPassword
|
||||
return
|
||||
}
|
||||
|
||||
return user, nil
|
||||
w.Header().Set("Retry-After", strconv.Itoa(int(
|
||||
h.mw.LoginFailureInterval().Seconds(),
|
||||
)))
|
||||
h.renderLoginError(
|
||||
w, r,
|
||||
"Too many failed login attempts. Please try again later.",
|
||||
http.StatusTooManyRequests,
|
||||
)
|
||||
}
|
||||
|
||||
// createAuthenticatedSession regenerates the session and stores
|
||||
|
||||
455
internal/handlers/auth_test.go
Normal file
455
internal/handlers/auth_test.go
Normal file
@@ -0,0 +1,455 @@
|
||||
package handlers_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"sneak.berlin/go/webhooker/internal/database"
|
||||
"sneak.berlin/go/webhooker/internal/handlers"
|
||||
"sneak.berlin/go/webhooker/internal/session"
|
||||
)
|
||||
|
||||
const (
|
||||
// operatorUser and operatorPassword are the single admin account
|
||||
// these tests defend.
|
||||
operatorUser = "admin"
|
||||
operatorPassword = "correct horse battery staple"
|
||||
|
||||
// sharedProxyPeer is the whole point of this file. Production is
|
||||
// required to run behind a TLS-terminating reverse proxy, and
|
||||
// TRUSTED_PROXIES defaults to empty, so every client — attacker
|
||||
// and operator alike — reaches the process from the proxy's
|
||||
// address and shares one rate-limit bucket. Both parties in
|
||||
// these tests therefore use the same RemoteAddr.
|
||||
sharedProxyPeer = "10.0.0.1:44444"
|
||||
|
||||
// loginFailureLimit is the failure budget one client has against
|
||||
// one submitted username. Restated here rather than imported
|
||||
// from the middleware package, so that changing the production
|
||||
// limit fails these tests instead of silently moving with them.
|
||||
loginFailureLimit = 5
|
||||
)
|
||||
|
||||
// seedOperator gives the bootstrapped admin account a password these
|
||||
// tests know. The account itself is created at startup with a random
|
||||
// password, which is exactly why its username is predictable to an
|
||||
// attacker and why keying failures by username alone does not fix
|
||||
// this issue.
|
||||
func seedOperator(t *testing.T, db *database.Database) {
|
||||
t.Helper()
|
||||
|
||||
hash, err := database.HashPassword(operatorPassword)
|
||||
require.NoError(t, err)
|
||||
|
||||
result := db.DB().Model(&database.User{}).
|
||||
Where("username = ?", operatorUser).
|
||||
Update("password", hash)
|
||||
|
||||
require.NoError(t, result.Error)
|
||||
require.EqualValues(
|
||||
t, 1, result.RowsAffected,
|
||||
"the bootstrap admin account must exist",
|
||||
)
|
||||
}
|
||||
|
||||
// loginPost builds a login form POST arriving from peer.
|
||||
func loginPost(peer, username, password string) *http.Request {
|
||||
form := url.Values{}
|
||||
form.Set("username", username)
|
||||
form.Set("password", password)
|
||||
|
||||
req := httptest.NewRequestWithContext(
|
||||
context.Background(),
|
||||
http.MethodPost,
|
||||
"/pages/login",
|
||||
strings.NewReader(form.Encode()),
|
||||
)
|
||||
req.Header.Set(
|
||||
"Content-Type", "application/x-www-form-urlencoded",
|
||||
)
|
||||
req.RemoteAddr = peer
|
||||
|
||||
return req
|
||||
}
|
||||
|
||||
// submitLogin drives one login POST through the handler.
|
||||
func submitLogin(
|
||||
h *handlers.Handlers, peer, username, password string,
|
||||
) *httptest.ResponseRecorder {
|
||||
w := httptest.NewRecorder()
|
||||
h.HandleLoginSubmit().ServeHTTP(w, loginPost(
|
||||
peer, username, password,
|
||||
))
|
||||
|
||||
return w
|
||||
}
|
||||
|
||||
// floodFailures sends attempts wrong-password logins for username
|
||||
// from peer, which is what an attacker does.
|
||||
func floodFailures(
|
||||
t *testing.T,
|
||||
h *handlers.Handlers,
|
||||
peer, username string,
|
||||
attempts int,
|
||||
) {
|
||||
t.Helper()
|
||||
|
||||
for i := range attempts {
|
||||
w := submitLogin(h, peer, username, fmt.Sprintf("guess-%d", i))
|
||||
require.NotEqual(
|
||||
t, http.StatusSeeOther, w.Code,
|
||||
"attempt %d must not authenticate", i,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// TestLogin_StrangersFloodCannotLockOutTheOperator is the
|
||||
// done-criterion of https://git.eeqj.de/sneak/webhooker/issues/150.
|
||||
//
|
||||
// The attacker and the operator share one rate-limit bucket, because
|
||||
// behind the mandated reverse proxy with TRUSTED_PROXIES unset every
|
||||
// client keys on the proxy's address. The attacker floods the
|
||||
// operator's own username — a single-admin product has a predictable
|
||||
// one — far past the failure limit. The operator must still be able
|
||||
// to log in with the correct password.
|
||||
//
|
||||
// This fails if credentials stop being verified ahead of the limiter.
|
||||
func TestLogin_StrangersFloodCannotLockOutTheOperator(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
db *database.Database
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &db)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
seedOperator(t, db)
|
||||
|
||||
// Well past the limit, and from the same bucket the operator
|
||||
// will arrive in.
|
||||
floodFailures(
|
||||
t, h, sharedProxyPeer, operatorUser,
|
||||
loginFailureLimit*2,
|
||||
)
|
||||
|
||||
w := submitLogin(
|
||||
h, sharedProxyPeer, operatorUser, operatorPassword,
|
||||
)
|
||||
|
||||
assert.Equal(
|
||||
t, http.StatusSeeOther, w.Code,
|
||||
"a correct password must never be throttled: the operator "+
|
||||
"has no second administrative path",
|
||||
)
|
||||
assert.Equal(t, "/", w.Header().Get("Location"))
|
||||
}
|
||||
|
||||
// TestLogin_StrangersFloodCannotDenyAnotherAccount is the
|
||||
// cross-account half: flooding one username must not spend another
|
||||
// account's budget, even from the same shared bucket.
|
||||
func TestLogin_StrangersFloodCannotDenyAnotherAccount(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
db *database.Database
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &db)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
seedOperator(t, db)
|
||||
|
||||
floodFailures(
|
||||
t, h, sharedProxyPeer, "someone-else",
|
||||
loginFailureLimit*2,
|
||||
)
|
||||
|
||||
w := submitLogin(h, sharedProxyPeer, operatorUser, "wrong")
|
||||
|
||||
assert.Equal(
|
||||
t, http.StatusUnauthorized, w.Code,
|
||||
"a flood against one username must not spend another "+
|
||||
"account's failure budget",
|
||||
)
|
||||
}
|
||||
|
||||
// TestLogin_RepeatedWrongPasswordsAreThrottled is the brute-force
|
||||
// half. Verifying before counting must not remove the throttle:
|
||||
// repeated wrong passwords for one username from one client key run
|
||||
// out of budget and are answered 429 with a Retry-After.
|
||||
func TestLogin_RepeatedWrongPasswordsAreThrottled(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
db *database.Database
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &db)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
seedOperator(t, db)
|
||||
|
||||
for i := range loginFailureLimit - 1 {
|
||||
w := submitLogin(
|
||||
h, sharedProxyPeer, operatorUser,
|
||||
fmt.Sprintf("guess-%d", i),
|
||||
)
|
||||
assert.Equal(
|
||||
t, http.StatusUnauthorized, w.Code,
|
||||
"attempt %d is still inside the budget", i,
|
||||
)
|
||||
}
|
||||
|
||||
w := submitLogin(h, sharedProxyPeer, operatorUser, "guess-last")
|
||||
assert.Equal(
|
||||
t, http.StatusTooManyRequests, w.Code,
|
||||
"wrong passwords must still run out of budget",
|
||||
)
|
||||
assert.NotEmpty(
|
||||
t, w.Header().Get("Retry-After"),
|
||||
"a throttled login must say when to come back",
|
||||
)
|
||||
}
|
||||
|
||||
// TestLogin_SuccessForgivesEarlierMistakes covers the operator who
|
||||
// mistypes several times and then gets it right: the successful
|
||||
// attempt clears the counter, so the next mistake is answered 401
|
||||
// rather than 429.
|
||||
func TestLogin_SuccessForgivesEarlierMistakes(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
db *database.Database
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &db)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
seedOperator(t, db)
|
||||
|
||||
floodFailures(
|
||||
t, h, sharedProxyPeer, operatorUser,
|
||||
loginFailureLimit,
|
||||
)
|
||||
|
||||
require.Equal(
|
||||
t, http.StatusSeeOther,
|
||||
submitLogin(
|
||||
h, sharedProxyPeer, operatorUser, operatorPassword,
|
||||
).Code,
|
||||
)
|
||||
|
||||
w := submitLogin(h, sharedProxyPeer, operatorUser, "typo")
|
||||
assert.Equal(
|
||||
t, http.StatusUnauthorized, w.Code,
|
||||
"a success must forgive the failures before it",
|
||||
)
|
||||
}
|
||||
|
||||
// TestLogin_UnknownUsernameCostsTheSameVerification is the
|
||||
// username-enumeration guard. Verifying credentials before the
|
||||
// limiter means response time is observable per attempt, so an
|
||||
// unknown username must be charged an equivalent-cost verification
|
||||
// against a dummy hash rather than returning early.
|
||||
//
|
||||
// The assertion is on the code path, not on wall-clock time: timing
|
||||
// assertions are flaky, and what actually has to hold is that the
|
||||
// hash is computed.
|
||||
func TestLogin_UnknownUsernameCostsTheSameVerification(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
db *database.Database
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &db)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
seedOperator(t, db)
|
||||
|
||||
require.Zero(t, h.DummyVerificationsForTest())
|
||||
|
||||
// A username that exists, with the wrong password: a real
|
||||
// Argon2id verification runs, and no dummy is needed.
|
||||
require.Equal(
|
||||
t, http.StatusUnauthorized,
|
||||
submitLogin(h, sharedProxyPeer, operatorUser, "wrong").Code,
|
||||
)
|
||||
assert.Zero(
|
||||
t, h.DummyVerificationsForTest(),
|
||||
"a known username verifies against its own hash",
|
||||
)
|
||||
|
||||
// A username that does not exist: indistinguishable response,
|
||||
// and the equivalent-cost verification must have run.
|
||||
require.Equal(
|
||||
t, http.StatusUnauthorized,
|
||||
submitLogin(h, sharedProxyPeer, "nosuchuser", "wrong").Code,
|
||||
)
|
||||
assert.Equal(
|
||||
t, uint64(1), h.DummyVerificationsForTest(),
|
||||
"an unknown username must still pay for a hash, or the "+
|
||||
"response time says whether the account exists",
|
||||
)
|
||||
}
|
||||
|
||||
// TestLogin_ConcurrentLoginsAreAllAnswered covers the login path
|
||||
// under the verification bound. The bound itself is pinned in the
|
||||
// middleware package; what matters here is that funnelling every
|
||||
// login through two slots does not lose or wedge a request — each one
|
||||
// is answered, whether it got a slot or was shed with 503.
|
||||
func TestLogin_ConcurrentLoginsAreAllAnswered(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const workers = 4
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
db *database.Database
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &db)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
seedOperator(t, db)
|
||||
|
||||
var (
|
||||
wg sync.WaitGroup
|
||||
mu sync.Mutex
|
||||
answers = map[int]int{}
|
||||
)
|
||||
|
||||
for i := range workers {
|
||||
wg.Go(func() {
|
||||
w := submitLogin(
|
||||
h, fmt.Sprintf("203.0.113.%d:5000", i),
|
||||
operatorUser, fmt.Sprintf("guess-%d", i),
|
||||
)
|
||||
|
||||
mu.Lock()
|
||||
answers[w.Code]++
|
||||
mu.Unlock()
|
||||
})
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
|
||||
assert.Zero(
|
||||
t, answers[http.StatusInternalServerError],
|
||||
"concurrent logins must not error",
|
||||
)
|
||||
assert.Equal(
|
||||
t, workers,
|
||||
answers[http.StatusUnauthorized]+
|
||||
answers[http.StatusTooManyRequests]+
|
||||
answers[http.StatusServiceUnavailable],
|
||||
"every concurrent login must be answered, whether it got "+
|
||||
"a verification slot or was shed with 503",
|
||||
)
|
||||
}
|
||||
|
||||
// TestLogin_MissingCredentialsRejectedBeforeAnyHash pins that the
|
||||
// empty-field check still runs ahead of the verification slot, so a
|
||||
// client sending nothing cannot occupy one.
|
||||
func TestLogin_MissingCredentialsRejectedBeforeAnyHash(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
db *database.Database
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &db)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
seedOperator(t, db)
|
||||
|
||||
w := submitLogin(h, sharedProxyPeer, "", "")
|
||||
|
||||
assert.Equal(t, http.StatusBadRequest, w.Code)
|
||||
assert.Zero(
|
||||
t, h.DummyVerificationsForTest(),
|
||||
"an empty submission must not cost a hash",
|
||||
)
|
||||
}
|
||||
|
||||
// TestLogin_SuccessCreatesSession is the control for the tests above:
|
||||
// the success path they assert on really does authenticate.
|
||||
func TestLogin_SuccessCreatesSession(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
db *database.Database
|
||||
sess *session.Session
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &db, &sess)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
seedOperator(t, db)
|
||||
|
||||
w := submitLogin(
|
||||
h, sharedProxyPeer, operatorUser, operatorPassword,
|
||||
)
|
||||
|
||||
require.Equal(t, http.StatusSeeOther, w.Code)
|
||||
require.NotEmpty(
|
||||
t, w.Result().Cookies(), "a session cookie must be issued",
|
||||
)
|
||||
|
||||
next := httptest.NewRequestWithContext(
|
||||
context.Background(), http.MethodGet, "/", nil,
|
||||
)
|
||||
|
||||
// Login regenerates the session, so the response carries two
|
||||
// Set-Cookie headers under the same name: one expiring the
|
||||
// pre-login cookie and one issuing the new one. A browser keeps
|
||||
// only the second, so replay only the one that is not an
|
||||
// expiry.
|
||||
for _, c := range w.Result().Cookies() {
|
||||
if c.MaxAge >= 0 {
|
||||
next.AddCookie(c)
|
||||
}
|
||||
}
|
||||
|
||||
s, err := sess.Get(next)
|
||||
require.NoError(t, err)
|
||||
assert.True(
|
||||
t, sess.IsAuthenticated(s),
|
||||
"the issued cookie must carry an authenticated session",
|
||||
)
|
||||
}
|
||||
@@ -2,15 +2,31 @@ package handlers
|
||||
|
||||
import (
|
||||
"html/template"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
|
||||
"sneak.berlin/go/webhooker/internal/database"
|
||||
)
|
||||
|
||||
// SetLogForTest replaces the handler's logger, so the handlers_test
|
||||
// package can assert on what a log line actually contains rather than
|
||||
// on what it is meant to contain.
|
||||
func (s *Handlers) SetLogForTest(log *slog.Logger) {
|
||||
s.log = log
|
||||
}
|
||||
|
||||
// MaxRenderedBodyBytesForTest exposes the event log's body cap
|
||||
// to the handlers_test package.
|
||||
const MaxRenderedBodyBytesForTest = maxRenderedBodyBytes
|
||||
|
||||
// DummyVerificationsForTest reports how many equivalent-cost
|
||||
// verifications were charged for usernames that do not exist. It
|
||||
// lets a test prove the anti-enumeration path ran without timing
|
||||
// anything.
|
||||
func (s *Handlers) DummyVerificationsForTest() uint64 {
|
||||
return s.dummyVerifications.Load()
|
||||
}
|
||||
|
||||
// TrimPartialRuneForTest exposes trimPartialRune for use in the
|
||||
// handlers_test package.
|
||||
func TrimPartialRuneForTest(b []byte) []byte {
|
||||
|
||||
@@ -10,6 +10,7 @@ import (
|
||||
"html/template"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"sync/atomic"
|
||||
|
||||
"go.uber.org/fx"
|
||||
"sneak.berlin/go/webhooker/internal/database"
|
||||
@@ -39,6 +40,12 @@ const (
|
||||
// errInvalidPassword is returned when a password does not match.
|
||||
var errInvalidPassword = errors.New("invalid password")
|
||||
|
||||
// errVerificationBusy is returned when no password-verification slot
|
||||
// became free before the wait elapsed, so no password was verified.
|
||||
var errVerificationBusy = errors.New(
|
||||
"password verification capacity exhausted",
|
||||
)
|
||||
|
||||
//nolint:revive // HandlersParams is a standard fx naming convention.
|
||||
type HandlersParams struct {
|
||||
fx.In
|
||||
@@ -49,6 +56,7 @@ type HandlersParams struct {
|
||||
WebhookDBMgr *database.WebhookDBManager
|
||||
Healthcheck *healthcheck.Healthcheck
|
||||
Session *session.Session
|
||||
Middleware *middleware.Middleware
|
||||
Notifier delivery.Notifier
|
||||
Evictor delivery.WebhookEvictor
|
||||
}
|
||||
@@ -62,9 +70,15 @@ type Handlers struct {
|
||||
db *database.Database
|
||||
dbMgr *database.WebhookDBManager
|
||||
session *session.Session
|
||||
mw *middleware.Middleware
|
||||
notifier delivery.Notifier
|
||||
evictor delivery.WebhookEvictor
|
||||
templates map[string]*template.Template
|
||||
|
||||
// dummyVerifications counts the equivalent-cost verifications
|
||||
// charged for usernames that do not exist. It exists so a test
|
||||
// can prove that path runs without measuring wall-clock time.
|
||||
dummyVerifications atomic.Uint64
|
||||
}
|
||||
|
||||
// parsePageTemplate parses a page-specific template set from the
|
||||
@@ -97,6 +111,7 @@ func New(
|
||||
s.db = params.Database
|
||||
s.dbMgr = params.WebhookDBMgr
|
||||
s.session = params.Session
|
||||
s.mw = params.Middleware
|
||||
s.notifier = params.Notifier
|
||||
s.evictor = params.Evictor
|
||||
|
||||
|
||||
@@ -20,6 +20,7 @@ import (
|
||||
"sneak.berlin/go/webhooker/internal/handlers"
|
||||
"sneak.berlin/go/webhooker/internal/healthcheck"
|
||||
"sneak.berlin/go/webhooker/internal/logger"
|
||||
"sneak.berlin/go/webhooker/internal/middleware"
|
||||
"sneak.berlin/go/webhooker/internal/session"
|
||||
)
|
||||
|
||||
@@ -82,6 +83,7 @@ func newTestApp(
|
||||
func(r *recordingEvictor) delivery.WebhookEvictor {
|
||||
return r
|
||||
},
|
||||
middleware.New,
|
||||
handlers.New,
|
||||
),
|
||||
fx.Populate(targets...),
|
||||
|
||||
451
internal/handlers/logbound_test.go
Normal file
451
internal/handlers/logbound_test.go
Normal file
@@ -0,0 +1,451 @@
|
||||
package handlers_test
|
||||
|
||||
// The handler-side half of the log-field audit. Two slog calls in
|
||||
// this package reach a value an UNAUTHENTICATED client picks outright
|
||||
// and of a length it picks outright:
|
||||
//
|
||||
// - the unknown-entrypoint DEBUG line on /webhook/{uuid}, whose
|
||||
// path segment matched no stored entrypoint and so is bounded by
|
||||
// nothing;
|
||||
// - the failed-login DEBUG lines, whose username is a form field.
|
||||
//
|
||||
// Both are at DEBUG, which is off in production by default. That is
|
||||
// not a bound: an operator turning DEBUG on to diagnose a flood must
|
||||
// not thereby hand the flood an unbounded write. Both spend the same
|
||||
// internal/logfield budget as the access log, and both are held here
|
||||
// to middleware.MaxAccessLogLineBytes.
|
||||
//
|
||||
// The two login lines past the username lookup — "invalid password"
|
||||
// and "user logged in" — carry the same cap without needing it, since
|
||||
// by then the value is a stored row rather than the client's. They are
|
||||
// pinned here too, so the caps cannot be dropped silently.
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/go-chi/chi"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"sneak.berlin/go/webhooker/internal/database"
|
||||
"sneak.berlin/go/webhooker/internal/handlers"
|
||||
"sneak.berlin/go/webhooker/internal/middleware"
|
||||
)
|
||||
|
||||
// floodRequests is the number of distinct invented values each flood
|
||||
// drives through the call site under test.
|
||||
const floodRequests = 32
|
||||
|
||||
// oversizedFillBytes is the length of the single client-chosen value
|
||||
// used to show that line size does not track input size.
|
||||
const oversizedFillBytes = 8192
|
||||
|
||||
// attackerMarker and tailMarker sit at the END of every oversized
|
||||
// value, past every budget. Their absence from the log is what
|
||||
// proves the value was cut rather than merely being short.
|
||||
const (
|
||||
attackerMarker = "QQATTACKERTEXTQQ"
|
||||
tailMarker = "QQTRUNCATEDTAILQQ"
|
||||
)
|
||||
|
||||
// escapeFills are the characters the log handlers escape, so a value
|
||||
// built out of them costs more on the line than it did on the wire. A
|
||||
// budget counted in raw bytes passes the plain case and fails these.
|
||||
//
|
||||
// U+1000C is unassigned, hence non-printable, and strconv.Quote
|
||||
// spells it as a ten-byte \UXXXXXXXX while the JSON handler passes
|
||||
// its four UTF-8 bytes through; only the text shape of these tests
|
||||
// reaches that charge.
|
||||
func escapeFills() map[string]string {
|
||||
return map[string]string{
|
||||
"plain": "x",
|
||||
"quote": `"`,
|
||||
"backslash": `\`,
|
||||
"tab": "\t",
|
||||
"newline": "\n",
|
||||
// A C0 control neither handler has a short escape for, so
|
||||
// each one costs six bytes on the line against the single
|
||||
// byte it cost to send: the widest multiplier a client can
|
||||
// drive, and the case a raw-byte budget breaks on first.
|
||||
//
|
||||
// This fill is load-bearing, not decoration. Budgeting raw
|
||||
// bytes instead of encoded is caught by this fill alone,
|
||||
// and only under the JSON handler, at 3,072 bytes against
|
||||
// the 2,560 ceiling. Drop it and that mutation passes.
|
||||
"control": "\x01",
|
||||
"astral": "\U0001000C",
|
||||
}
|
||||
}
|
||||
|
||||
// logHandlers are the two handlers internal/logger can install.
|
||||
func logHandlers() map[string]func(
|
||||
io.Writer, *slog.HandlerOptions,
|
||||
) slog.Handler {
|
||||
return map[string]func(
|
||||
io.Writer, *slog.HandlerOptions,
|
||||
) slog.Handler{
|
||||
"json": func(
|
||||
w io.Writer, o *slog.HandlerOptions,
|
||||
) slog.Handler {
|
||||
return slog.NewJSONHandler(w, o)
|
||||
},
|
||||
"text": func(
|
||||
w io.Writer, o *slog.HandlerOptions,
|
||||
) slog.Handler {
|
||||
return slog.NewTextHandler(w, o)
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// oversizedFill builds an 8 KB client-chosen value out of
|
||||
// repetitions of ch, with both markers at its far end.
|
||||
func oversizedFill(ch string) string {
|
||||
return "x" + strings.Repeat(ch, oversizedFillBytes) +
|
||||
attackerMarker + tailMarker
|
||||
}
|
||||
|
||||
// capturingHandlersWithDB is capturingHandlers plus the database, for
|
||||
// the call sites that write their line only once the client's value
|
||||
// matched a stored row.
|
||||
func capturingHandlersWithDB(
|
||||
t *testing.T,
|
||||
newHandler func(io.Writer, *slog.HandlerOptions) slog.Handler,
|
||||
) (*handlers.Handlers, *database.Database, *bytes.Buffer) {
|
||||
t.Helper()
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
db *database.Database
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &db)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
buf := new(bytes.Buffer)
|
||||
h.SetLogForTest(slog.New(newHandler(
|
||||
buf, &slog.HandlerOptions{Level: slog.LevelDebug},
|
||||
)))
|
||||
|
||||
return h, db, buf
|
||||
}
|
||||
|
||||
// capturingHandlers builds a Handlers whose log is captured into the
|
||||
// returned buffer at DEBUG through the named handler.
|
||||
func capturingHandlers(
|
||||
t *testing.T,
|
||||
newHandler func(io.Writer, *slog.HandlerOptions) slog.Handler,
|
||||
) (*handlers.Handlers, *bytes.Buffer) {
|
||||
t.Helper()
|
||||
|
||||
var h *handlers.Handlers
|
||||
|
||||
app := newTestApp(t, &h)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
buf := new(bytes.Buffer)
|
||||
h.SetLogForTest(slog.New(newHandler(
|
||||
buf, &slog.HandlerOptions{Level: slog.LevelDebug},
|
||||
)))
|
||||
|
||||
return h, buf
|
||||
}
|
||||
|
||||
// logLines splits the captured buffer into non-empty lines, holding
|
||||
// each to bound bytes.
|
||||
func logLines(t *testing.T, buf *bytes.Buffer, bound int) []string {
|
||||
t.Helper()
|
||||
|
||||
var lines []string
|
||||
|
||||
for line := range strings.SplitSeq(
|
||||
strings.TrimSpace(buf.String()), "\n",
|
||||
) {
|
||||
if line == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
require.LessOrEqual(
|
||||
t, len(line), bound,
|
||||
"log line exceeded its bound: %s", line,
|
||||
)
|
||||
|
||||
lines = append(lines, line)
|
||||
}
|
||||
|
||||
return lines
|
||||
}
|
||||
|
||||
// assertNoClientText fails if the far end of the client-chosen input
|
||||
// survived into the log.
|
||||
func assertNoClientText(t *testing.T, buf *bytes.Buffer) {
|
||||
t.Helper()
|
||||
|
||||
assert.NotContains(
|
||||
t, buf.String(), attackerMarker,
|
||||
"log carried attacker-chosen text",
|
||||
)
|
||||
assert.NotContains(
|
||||
t, buf.String(), tailMarker,
|
||||
"log carried the tail of the attacker-chosen text",
|
||||
)
|
||||
}
|
||||
|
||||
// receiverRouter mounts the real receiver handler at the production
|
||||
// route pattern.
|
||||
func receiverRouter(h *handlers.Handlers) *chi.Mux {
|
||||
router := chi.NewRouter()
|
||||
router.Post("/webhook/{uuid}", h.HandleWebhook())
|
||||
|
||||
return router
|
||||
}
|
||||
|
||||
// postReceiver sends one POST at /webhook/<segment>.
|
||||
//
|
||||
// RawPath is cleared after parsing so chi routes on the decoded path
|
||||
// and the handler sees the raw bytes rather than their percent-escaped
|
||||
// spelling. That is the harder case for the budget: the escaped
|
||||
// spelling is plain ASCII, which costs one byte per byte, while the
|
||||
// decoded bytes are what the log handler has to escape.
|
||||
func postReceiver(
|
||||
t *testing.T, router *chi.Mux, segment string,
|
||||
) int {
|
||||
t.Helper()
|
||||
|
||||
req := httptest.NewRequestWithContext(
|
||||
context.Background(),
|
||||
http.MethodPost,
|
||||
"/webhook/"+url.PathEscape(segment),
|
||||
strings.NewReader(""),
|
||||
)
|
||||
req.URL.RawPath = ""
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
router.ServeHTTP(w, req)
|
||||
|
||||
return w.Code
|
||||
}
|
||||
|
||||
// postLogin submits the login form with the given username and a
|
||||
// non-empty password.
|
||||
func postLogin(
|
||||
t *testing.T, h *handlers.Handlers, username string,
|
||||
) int {
|
||||
t.Helper()
|
||||
|
||||
return postLoginWithPassword(t, h, username, "not-the-password")
|
||||
}
|
||||
|
||||
// postLoginWithPassword submits the login form with both credentials
|
||||
// chosen by the caller, so a test can reach the branches past the
|
||||
// username lookup.
|
||||
func postLoginWithPassword(
|
||||
t *testing.T, h *handlers.Handlers, username, password string,
|
||||
) int {
|
||||
t.Helper()
|
||||
|
||||
form := url.Values{
|
||||
"username": {username},
|
||||
"password": {password},
|
||||
}
|
||||
|
||||
req := httptest.NewRequestWithContext(
|
||||
context.Background(),
|
||||
http.MethodPost,
|
||||
"/pages/login",
|
||||
strings.NewReader(form.Encode()),
|
||||
)
|
||||
req.Header.Set(
|
||||
"Content-Type", "application/x-www-form-urlencoded",
|
||||
)
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
h.HandleLoginSubmit().ServeHTTP(w, req)
|
||||
|
||||
return w.Code
|
||||
}
|
||||
|
||||
// TestUnknownEntrypoint_LogLineDoesNotTrackPathSize drives 8 KB of
|
||||
// client-chosen path at the unauthenticated receiver's
|
||||
// unknown-entrypoint DEBUG line and holds it to the same ceiling the
|
||||
// access log states.
|
||||
func TestUnknownEntrypoint_LogLineDoesNotTrackPathSize(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for handlerName, newHandler := range logHandlers() {
|
||||
for fillName, fill := range escapeFills() {
|
||||
t.Run(handlerName+"/"+fillName, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
h, buf := capturingHandlers(t, newHandler)
|
||||
router := receiverRouter(h)
|
||||
|
||||
for i := range floodRequests {
|
||||
assert.Equal(
|
||||
t,
|
||||
http.StatusNotFound,
|
||||
postReceiver(
|
||||
t, router,
|
||||
oversizedFill(fill)+
|
||||
strings.Repeat("y", i),
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
lines := logLines(
|
||||
t, buf, middleware.MaxAccessLogLineBytes,
|
||||
)
|
||||
require.Len(t, lines, floodRequests)
|
||||
assertNoClientText(t, buf)
|
||||
|
||||
assertBoundedFlood(t, buf.Len())
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestFailedLogin_LogLineDoesNotTrackUsernameSize drives 8 KB of
|
||||
// client-chosen username at the unauthenticated login endpoint's
|
||||
// DEBUG line and holds it to the same ceiling.
|
||||
func TestFailedLogin_LogLineDoesNotTrackUsernameSize(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for handlerName, newHandler := range logHandlers() {
|
||||
for fillName, fill := range escapeFills() {
|
||||
t.Run(handlerName+"/"+fillName, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
h, buf := capturingHandlers(t, newHandler)
|
||||
|
||||
for i := range floodRequests {
|
||||
assert.Equal(
|
||||
t,
|
||||
http.StatusUnauthorized,
|
||||
postLogin(
|
||||
t, h,
|
||||
oversizedFill(fill)+
|
||||
strings.Repeat("y", i),
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
lines := logLines(
|
||||
t, buf, middleware.MaxAccessLogLineBytes,
|
||||
)
|
||||
require.Len(t, lines, floodRequests)
|
||||
assertNoClientText(t, buf)
|
||||
|
||||
assertBoundedFlood(t, buf.Len())
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// storedUserPassword is the password held by the oversize accounts
|
||||
// the test below creates.
|
||||
const storedUserPassword = "correct-horse-battery-staple"
|
||||
|
||||
// storedFillBytes is the raw length of the client-chosen value in
|
||||
// those accounts' usernames. It is well past the 512-byte field
|
||||
// budget, so the line is still truncated, but short enough that the
|
||||
// session cookie a successful login writes stays inside
|
||||
// securecookie's 4 KB limit: the cookie is written BEFORE the
|
||||
// "user logged in" line, so an 8 KB username answers 500 and never
|
||||
// reaches it.
|
||||
const storedFillBytes = 1024
|
||||
|
||||
// storedFill builds a username fill of storedFillBytes raw bytes out
|
||||
// of repetitions of ch, with both markers at its far end.
|
||||
func storedFill(ch string) string {
|
||||
return "x" + strings.Repeat(ch, storedFillBytes/len(ch)) +
|
||||
attackerMarker + tailMarker
|
||||
}
|
||||
|
||||
// TestStoredUsername_LogLinesDoNotTrackUsernameSize pins the two
|
||||
// login lines that are reached only AFTER the username matched a
|
||||
// stored row: "invalid password" and "user logged in". Neither
|
||||
// strictly needs its cap — the value is the operator's own data by
|
||||
// then, not the client's — but both carry one so that every username
|
||||
// this unauthenticated endpoint logs is capped, and an unasserted cap
|
||||
// is one a later edit removes for free.
|
||||
//
|
||||
// One app per handler with the accounts created inside it, and no
|
||||
// parallelism below that level: every account costs an Argon2id hash
|
||||
// and every attempt costs a verification.
|
||||
func TestStoredUsername_LogLinesDoNotTrackUsernameSize(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for handlerName, newHandler := range logHandlers() {
|
||||
t.Run(handlerName, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
h, db, buf := capturingHandlersWithDB(t, newHandler)
|
||||
|
||||
hash, err := database.HashPassword(storedUserPassword)
|
||||
require.NoError(t, err)
|
||||
|
||||
fills := escapeFills()
|
||||
for fillName, fill := range fills {
|
||||
username := storedFill(fill) + fillName
|
||||
|
||||
require.NoError(t, db.DB().Create(&database.User{
|
||||
Username: username,
|
||||
Password: hash,
|
||||
}).Error)
|
||||
|
||||
// Matched the row, wrong secret: "invalid
|
||||
// password".
|
||||
assert.Equal(
|
||||
t, http.StatusUnauthorized,
|
||||
postLoginWithPassword(
|
||||
t, h, username, "not-the-password",
|
||||
),
|
||||
)
|
||||
|
||||
// Matched the row, right secret: "user logged
|
||||
// in".
|
||||
assert.Equal(
|
||||
t, http.StatusSeeOther,
|
||||
postLoginWithPassword(
|
||||
t, h, username, storedUserPassword,
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
lines := logLines(
|
||||
t, buf, middleware.MaxAccessLogLineBytes,
|
||||
)
|
||||
require.Len(t, lines, 2*len(fills))
|
||||
assertNoClientText(t, buf)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// assertBoundedFlood holds the whole flood's log output to what the
|
||||
// stated per-line ceiling allows. The flood sent
|
||||
// floodRequests * oversizedFillBytes bytes of client-chosen text;
|
||||
// this is the assertion that the log did not grow with it.
|
||||
func assertBoundedFlood(t *testing.T, got int) {
|
||||
t.Helper()
|
||||
|
||||
sent := floodRequests * oversizedFillBytes
|
||||
|
||||
require.Less(
|
||||
t, got, sent/2,
|
||||
"log volume tracked the size of the flood's input",
|
||||
)
|
||||
require.LessOrEqual(
|
||||
t, got,
|
||||
floodRequests*middleware.MaxAccessLogLineBytes,
|
||||
)
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
|
||||
"github.com/go-chi/chi"
|
||||
@@ -42,11 +43,14 @@ func (h *Handlers) HandlePasswordChange() http.HandlerFunc {
|
||||
}
|
||||
|
||||
successMessage, errorMessage, handled := h.applyPasswordChange(
|
||||
r.Context(),
|
||||
w,
|
||||
sessionUsername,
|
||||
r.FormValue("current_password"),
|
||||
r.FormValue("new_password"),
|
||||
r.FormValue("confirm_password"),
|
||||
// PostFormValue, not FormValue: the credential must
|
||||
// come from the body, never from the query string.
|
||||
r.PostFormValue("current_password"),
|
||||
r.PostFormValue("new_password"),
|
||||
r.PostFormValue("confirm_password"),
|
||||
)
|
||||
if !handled {
|
||||
return
|
||||
@@ -66,9 +70,30 @@ func (h *Handlers) HandlePasswordChange() http.HandlerFunc {
|
||||
// 500 response itself and returns handled=false, signalling the caller
|
||||
// to stop without re-rendering the page.
|
||||
func (h *Handlers) applyPasswordChange(
|
||||
ctx context.Context,
|
||||
w http.ResponseWriter,
|
||||
username, currentPassword, newPassword, confirmPassword string,
|
||||
) (string, string, bool) {
|
||||
// This endpoint verifies one password and hashes another, at
|
||||
// 64 MB each, so it takes a slot from the same bound the login
|
||||
// endpoint uses. The bound is per hash, not per endpoint: leaving
|
||||
// this path outside it would leave a hole in it. The slot is held
|
||||
// across both hashes.
|
||||
release, ok := h.mw.BeginPasswordVerification(ctx)
|
||||
if !ok {
|
||||
h.log.Warn("password verification capacity exhausted")
|
||||
http.Error(
|
||||
w,
|
||||
"The server is busy verifying credentials. "+
|
||||
"Please try again.",
|
||||
http.StatusServiceUnavailable,
|
||||
)
|
||||
|
||||
return "", "", false
|
||||
}
|
||||
|
||||
defer release()
|
||||
|
||||
// Load the user row so we can verify the current password and
|
||||
// persist the new hash.
|
||||
var user database.User
|
||||
|
||||
@@ -227,9 +227,9 @@ func (h *Handlers) HandleSourceCreateSubmit() http.HandlerFunc {
|
||||
return
|
||||
}
|
||||
|
||||
name := r.FormValue("name")
|
||||
description := r.FormValue("description")
|
||||
retentionStr := r.FormValue("retention_days")
|
||||
name := r.PostFormValue("name")
|
||||
description := r.PostFormValue("description")
|
||||
retentionStr := r.PostFormValue("retention_days")
|
||||
|
||||
if name == "" {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
@@ -509,7 +509,7 @@ func (h *Handlers) applyWebhookEdit(
|
||||
) {
|
||||
// The body size cap is enforced by the MaxBodySize middleware,
|
||||
// which runs before CSRF parses the form.
|
||||
name := r.FormValue("name")
|
||||
name := r.PostFormValue("name")
|
||||
if name == "" {
|
||||
data := map[string]any{
|
||||
tmplKeyWebhook: webhook,
|
||||
@@ -523,12 +523,12 @@ func (h *Handlers) applyWebhookEdit(
|
||||
}
|
||||
|
||||
webhook.Name = name
|
||||
webhook.Description = r.FormValue("description")
|
||||
webhook.Description = r.PostFormValue("description")
|
||||
|
||||
// An empty field falls back to the stored value, so submitting the
|
||||
// form without touching retention leaves the policy alone.
|
||||
retentionDays, retErr := parseRetentionDays(
|
||||
r.FormValue("retention_days"), webhook.RetentionDays,
|
||||
r.PostFormValue("retention_days"), webhook.RetentionDays,
|
||||
)
|
||||
if retErr != nil {
|
||||
data := map[string]any{
|
||||
@@ -950,7 +950,7 @@ func (h *Handlers) HandleEntrypointCreate() http.HandlerFunc {
|
||||
return
|
||||
}
|
||||
|
||||
description := r.FormValue("description")
|
||||
description := r.PostFormValue("description")
|
||||
|
||||
entrypoint := &database.Entrypoint{
|
||||
WebhookID: webhook.ID,
|
||||
@@ -1020,11 +1020,18 @@ func (h *Handlers) processTargetCreate(
|
||||
) {
|
||||
// The body size cap is enforced by the MaxBodySize middleware,
|
||||
// which runs before CSRF parses the form.
|
||||
name := r.FormValue("name")
|
||||
targetType := database.TargetType(r.FormValue("type"))
|
||||
targetURL := r.FormValue("url")
|
||||
maxRetriesStr := r.FormValue("max_retries")
|
||||
expiry := r.FormValue("expiry")
|
||||
//
|
||||
// Every field here is read with PostFormValue, not FormValue.
|
||||
// FormValue falls back to the query string, which would let
|
||||
// `POST /source/{id}/targets?url=https://hooks.slack.com/...`
|
||||
// configure a target from a value the request line carries — and
|
||||
// the request line, unlike the body, is what logs, proxies,
|
||||
// Referer headers and error trackers record.
|
||||
name := r.PostFormValue("name")
|
||||
targetType := database.TargetType(r.PostFormValue("type"))
|
||||
targetURL := r.PostFormValue("url")
|
||||
maxRetriesStr := r.PostFormValue("max_retries")
|
||||
expiry := r.PostFormValue("expiry")
|
||||
|
||||
if name == "" {
|
||||
http.Error(
|
||||
|
||||
206
internal/handlers/target_create_query_test.go
Normal file
206
internal/handlers/target_create_query_test.go
Normal file
@@ -0,0 +1,206 @@
|
||||
package handlers_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/go-chi/chi"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"sneak.berlin/go/webhooker/internal/config"
|
||||
"sneak.berlin/go/webhooker/internal/database"
|
||||
"sneak.berlin/go/webhooker/internal/middleware"
|
||||
)
|
||||
|
||||
// targetSecretSegments are the path segments of an incoming-webhook
|
||||
// URL. For Slack, Discord and Teams the path IS the bearer credential,
|
||||
// so this string must not reach storage or the access log by way of
|
||||
// the request line.
|
||||
const targetSecretSegments = "T00000000/B00000000/QQTARGETSECRETQQ"
|
||||
|
||||
// targetSecretURL is a destination whose secret lives in its path. It
|
||||
// uses a literal public address rather than a hostname so the SSRF
|
||||
// check resolves nothing: with a hostname, a sandbox without DNS would
|
||||
// reject the URL for the wrong reason and the test would pass even
|
||||
// with the defect reintroduced.
|
||||
const targetSecretURL = "https://93.184.216.34/services/" +
|
||||
targetSecretSegments
|
||||
|
||||
// targetsForWebhook returns every target stored against a webhook.
|
||||
func targetsForWebhook(
|
||||
t *testing.T,
|
||||
db *database.Database,
|
||||
webhookID string,
|
||||
) []database.Target {
|
||||
t.Helper()
|
||||
|
||||
var targets []database.Target
|
||||
|
||||
require.NoError(
|
||||
t,
|
||||
db.DB().Where("webhook_id = ?", webhookID).
|
||||
Find(&targets).Error,
|
||||
)
|
||||
|
||||
return targets
|
||||
}
|
||||
|
||||
// postTargetCreate drives HandleTargetCreate through the production
|
||||
// access-log middleware and a chi route, so the logged url field is
|
||||
// produced exactly as it ships, and returns the recorder plus the
|
||||
// captured log.
|
||||
func postTargetCreate(
|
||||
t *testing.T,
|
||||
env *sourceTestEnv,
|
||||
webhookID string,
|
||||
query string,
|
||||
form url.Values,
|
||||
) (*httptest.ResponseRecorder, string) {
|
||||
t.Helper()
|
||||
|
||||
logBuf := new(bytes.Buffer)
|
||||
mw := middleware.NewForTest(
|
||||
slog.New(slog.NewJSONHandler(
|
||||
logBuf, &slog.HandlerOptions{Level: slog.LevelInfo},
|
||||
)),
|
||||
&config.Config{Environment: config.EnvironmentDev},
|
||||
nil,
|
||||
)
|
||||
|
||||
router := chi.NewRouter()
|
||||
router.Use(mw.Logging())
|
||||
router.Post(
|
||||
"/source/{sourceID}/targets",
|
||||
env.handlers.HandleTargetCreate(),
|
||||
)
|
||||
|
||||
target := "/source/" + webhookID + "/targets"
|
||||
if query != "" {
|
||||
target += "?" + query
|
||||
}
|
||||
|
||||
body := ""
|
||||
if form != nil {
|
||||
body = form.Encode()
|
||||
}
|
||||
|
||||
req := httptest.NewRequestWithContext(
|
||||
context.Background(),
|
||||
http.MethodPost,
|
||||
target,
|
||||
strings.NewReader(body),
|
||||
)
|
||||
req.Header.Set(
|
||||
"Content-Type", "application/x-www-form-urlencoded",
|
||||
)
|
||||
|
||||
for _, c := range env.cookies {
|
||||
req.AddCookie(c)
|
||||
}
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
router.ServeHTTP(w, req)
|
||||
|
||||
return w, logBuf.String()
|
||||
}
|
||||
|
||||
// TestHandleTargetCreate_QueryStringURLDoesNotConfigureATarget is the
|
||||
// regression test for the ingress leak. r.FormValue falls back to the
|
||||
// query string when a field is absent from the POST body, so
|
||||
//
|
||||
// POST /source/{id}/targets?url=https://hooks.slack.com/services/...
|
||||
//
|
||||
// with an empty url field used to create a working target from a value
|
||||
// carried on the request line — where logs, proxies, Referer headers
|
||||
// and error trackers record it. The handler reads the body only, so
|
||||
// the request is rejected for a missing URL and stores nothing.
|
||||
//
|
||||
// name and type are sent in the BODY on purpose: the request has to
|
||||
// get past those two validations for the assertion to be about the url
|
||||
// read specifically.
|
||||
func TestHandleTargetCreate_QueryStringURLDoesNotConfigureATarget(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
env := setupSourceTest(t)
|
||||
webhook := seedWebhookWithRetention(t, env.db, 30)
|
||||
|
||||
body := url.Values{}
|
||||
body.Set("name", "leaky")
|
||||
body.Set("type", string(database.TargetTypeSlack))
|
||||
|
||||
w, logged := postTargetCreate(
|
||||
t, env, webhook.ID,
|
||||
"url="+url.QueryEscape(targetSecretURL),
|
||||
body,
|
||||
)
|
||||
|
||||
assert.Equal(t, http.StatusBadRequest, w.Code)
|
||||
|
||||
targets := targetsForWebhook(t, env.db, webhook.ID)
|
||||
assert.Empty(
|
||||
t, targets,
|
||||
"a query-string value must not populate a target config",
|
||||
)
|
||||
|
||||
assert.NotContains(t, logged, targetSecretSegments)
|
||||
assert.NotContains(t, logged, "93.184.216.34")
|
||||
assert.NotEmpty(t, logged, "the access log line must still be written")
|
||||
}
|
||||
|
||||
// TestHandleTargetCreate_BodyURLStillCreatesTheTarget is the positive
|
||||
// control for the test above: the rejection has to come from where the
|
||||
// value was read, not from the handler being broken.
|
||||
func TestHandleTargetCreate_BodyURLStillCreatesTheTarget(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
env := setupSourceTest(t)
|
||||
webhook := seedWebhookWithRetention(t, env.db, 30)
|
||||
|
||||
body := url.Values{}
|
||||
body.Set("name", "legit")
|
||||
body.Set("type", string(database.TargetTypeSlack))
|
||||
body.Set("url", targetSecretURL)
|
||||
|
||||
w, logged := postTargetCreate(t, env, webhook.ID, "", body)
|
||||
|
||||
assert.Equal(t, http.StatusSeeOther, w.Code)
|
||||
|
||||
targets := targetsForWebhook(t, env.db, webhook.ID)
|
||||
require.Len(t, targets, 1)
|
||||
assert.Contains(t, targets[0].Config, targetSecretSegments)
|
||||
|
||||
// The body carried the credential, so the access log must still
|
||||
// not have it: the log records the request line only.
|
||||
assert.NotContains(t, logged, targetSecretSegments)
|
||||
}
|
||||
|
||||
// TestHandleTargetCreate_QueryStringCannotSupplyNameOrType covers the
|
||||
// rest of the converted reads on this handler in one request: with an
|
||||
// empty body, nothing the query carries is visible to it.
|
||||
func TestHandleTargetCreate_QueryStringCannotSupplyNameOrType(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
env := setupSourceTest(t)
|
||||
webhook := seedWebhookWithRetention(t, env.db, 30)
|
||||
|
||||
w, _ := postTargetCreate(
|
||||
t, env, webhook.ID,
|
||||
"name=leaky&type=slack&max_retries=9&expiry=30d&url="+
|
||||
url.QueryEscape(targetSecretURL),
|
||||
url.Values{},
|
||||
)
|
||||
|
||||
assert.Equal(t, http.StatusBadRequest, w.Code)
|
||||
assert.Contains(t, w.Body.String(), "Name is required")
|
||||
assert.Empty(t, targetsForWebhook(t, env.db, webhook.ID))
|
||||
}
|
||||
@@ -9,6 +9,7 @@ import (
|
||||
"gorm.io/gorm"
|
||||
"sneak.berlin/go/webhooker/internal/database"
|
||||
"sneak.berlin/go/webhooker/internal/delivery"
|
||||
"sneak.berlin/go/webhooker/internal/logfield"
|
||||
)
|
||||
|
||||
const (
|
||||
@@ -125,9 +126,16 @@ func (h *Handlers) lookupEntrypoint(
|
||||
"path = ?", entrypointUUID,
|
||||
).First(&entrypoint)
|
||||
if result.Error != nil {
|
||||
// The receiver is unauthenticated and /webhook/{uuid}
|
||||
// matches any single segment, so this value is entirely
|
||||
// client-chosen on exactly the branch where the lookup
|
||||
// failed. DEBUG is off by default; the cap is what keeps
|
||||
// turning it on from restoring an unbounded write.
|
||||
h.log.Debug(
|
||||
"entrypoint not found",
|
||||
"path", entrypointUUID,
|
||||
"path", logfield.Truncate(
|
||||
entrypointUUID, logfield.MaxBytes,
|
||||
),
|
||||
)
|
||||
http.NotFound(w, r)
|
||||
|
||||
|
||||
143
internal/logfield/logfield.go
Normal file
143
internal/logfield/logfield.go
Normal file
@@ -0,0 +1,143 @@
|
||||
// Package logfield bounds the client-supplied values this service
|
||||
// writes into its logs.
|
||||
//
|
||||
// Any log field whose content a client picks is spent against a budget
|
||||
// here, in ENCODED bytes rather than in the bytes the client sent, so
|
||||
// that escaping cannot multiply a field past its nominal size. One
|
||||
// budget and one implementation serves the access log in
|
||||
// internal/middleware and every other slog call that reaches a
|
||||
// client-chosen path, header or form value; a second, ad-hoc
|
||||
// truncation somewhere else in the tree is the thing this package
|
||||
// exists to prevent.
|
||||
package logfield
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
const (
|
||||
// MaxBytes is the default budget for a log field whose value the
|
||||
// client supplies outright: a URL, a path, a header, a form value.
|
||||
// The budget is spent in ENCODED bytes (see Truncate), so 512 still
|
||||
// holds a real browser's User-Agent whole — those are plain ASCII,
|
||||
// which encodes one byte for one — while a value built from
|
||||
// characters the encoder escapes keeps a shorter prefix. That is
|
||||
// the intended trade: 500 quotation marks are not a debugging
|
||||
// asset.
|
||||
MaxBytes = 512
|
||||
|
||||
// TruncationMarker is appended to any field that was cut, so a
|
||||
// short value and a truncated one cannot be confused. It is charged
|
||||
// on top of the budget, not inside it.
|
||||
TruncationMarker = "[truncated]"
|
||||
)
|
||||
|
||||
// EncodedBytes is what r costs on the line once the log handler has
|
||||
// escaped it, taking the worse of the two handlers internal/logger
|
||||
// configures.
|
||||
//
|
||||
// slog's JSON handler escapes quote, backslash, newline, carriage
|
||||
// return and tab to two bytes each, and every other C0 control plus
|
||||
// LINE SEPARATOR and PARAGRAPH SEPARATOR to a six-byte \u escape; it
|
||||
// passes every other rune through as its own UTF-8. Its text handler
|
||||
// quotes with strconv.Quote, which spells a non-printable rune below
|
||||
// U+10000 as \uXXXX but one at or above U+10000 as \UXXXXXXXX — ten
|
||||
// bytes, not six. The text handler is therefore the worse of the two
|
||||
// for every non-printable rune, and by four bytes apiece for the
|
||||
// 955,086 unassigned, private-use and format code points on planes 1
|
||||
// to 16.
|
||||
//
|
||||
// Charging ten there is what makes the stated line ceilings hold for
|
||||
// the tty handler as well: U+1000C encodes as F0 90 80 8C, every byte
|
||||
// >= 0x80, which httpguts.ValidHeaderFieldValue accepts and
|
||||
// net/textproto does not strip, so a header can be filled with them.
|
||||
//
|
||||
// Both handlers pass printable runes through as their own UTF-8, so
|
||||
// unicode.IsPrint separates the escaped cases from the plain ones for
|
||||
// either handler.
|
||||
func EncodedBytes(r rune) int {
|
||||
const (
|
||||
// A backslash and the character itself.
|
||||
shortEscapeBytes = 2
|
||||
// \uXXXX, which is also the width of \u00XX.
|
||||
escapedRuneBytes = 6
|
||||
// \UXXXXXXXX, strconv.Quote's spelling of a non-printable
|
||||
// rune outside the basic multilingual plane.
|
||||
escapedAstralRuneBytes = 10
|
||||
// The first code point strconv.Quote spells with \U.
|
||||
firstAstralRune = 0x10000
|
||||
)
|
||||
|
||||
switch {
|
||||
case r == '"' || r == '\\' || r == '\n' || r == '\r' || r == '\t':
|
||||
return shortEscapeBytes
|
||||
case !unicode.IsPrint(r) && r >= firstAstralRune:
|
||||
return escapedAstralRuneBytes
|
||||
case !unicode.IsPrint(r):
|
||||
return escapedRuneBytes
|
||||
default:
|
||||
return utf8.RuneLen(r)
|
||||
}
|
||||
}
|
||||
|
||||
// Truncate caps s at maxBytes of ENCODED output, marking the value
|
||||
// when it cuts.
|
||||
//
|
||||
// Budgeting raw bytes would not bound the line. Escaping only ever
|
||||
// grows a value, so a raw budget spent on characters the encoder
|
||||
// escapes buys a field several times its nominal size — and the line
|
||||
// is the thing an operator is told to multiply by their request rate.
|
||||
// Charging each rune what it will actually cost is what makes a stated
|
||||
// ceiling true rather than merely larger. The visible consequence is
|
||||
// that an escape-heavy value keeps a shorter prefix than a plain one,
|
||||
// which is the correct trade.
|
||||
//
|
||||
// The result is always valid UTF-8. A cut on a byte boundary can split
|
||||
// a multi-byte rune, and a header can carry bytes that were never
|
||||
// valid UTF-8 to begin with; both are dropped rather than kept, since
|
||||
// an encoder would otherwise spend six bytes replacing each one.
|
||||
func Truncate(s string, maxBytes int) string {
|
||||
// No rune encodes to fewer bytes than it occupies, so nothing past
|
||||
// maxBytes raw can fit the budget. Slicing first bounds the scan
|
||||
// below to the budget rather than to the size of the header the
|
||||
// client sent.
|
||||
window, cut := s, false
|
||||
if len(window) > maxBytes {
|
||||
window, cut = window[:maxBytes], true
|
||||
}
|
||||
|
||||
var (
|
||||
kept strings.Builder
|
||||
spent int
|
||||
)
|
||||
|
||||
for i := 0; i < len(window); {
|
||||
r, size := utf8.DecodeRuneInString(window[i:])
|
||||
if r == utf8.RuneError && size == 1 {
|
||||
i += size
|
||||
|
||||
continue
|
||||
}
|
||||
|
||||
cost := EncodedBytes(r)
|
||||
if spent+cost > maxBytes {
|
||||
cut = true
|
||||
|
||||
break
|
||||
}
|
||||
|
||||
spent += cost
|
||||
|
||||
kept.WriteString(window[i : i+size])
|
||||
|
||||
i += size
|
||||
}
|
||||
|
||||
if !cut {
|
||||
return kept.String()
|
||||
}
|
||||
|
||||
return kept.String() + TruncationMarker
|
||||
}
|
||||
221
internal/logfield/logfield_test.go
Normal file
221
internal/logfield/logfield_test.go
Normal file
@@ -0,0 +1,221 @@
|
||||
package logfield_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"io"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"testing"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"sneak.berlin/go/webhooker/internal/logfield"
|
||||
)
|
||||
|
||||
// budget is the field budget these tests spend. Small enough that a
|
||||
// cut is unambiguous, large enough to hold several runes of every
|
||||
// width.
|
||||
const budget = 64
|
||||
|
||||
// sampleRunes is how many runes wide the values in the charge test
|
||||
// are. The handlers add a constant per field — a pair of quotes when
|
||||
// the value needs quoting — so the per-rune charge is only visible
|
||||
// once it is amortised over a run of them.
|
||||
const sampleRunes = 64
|
||||
|
||||
// quotingSlack is that constant: the pair of quotes a handler adds to
|
||||
// a value that needs them and omits from one that does not.
|
||||
const quotingSlack = 2
|
||||
|
||||
// newHandlers are the two handlers internal/logger can install. Time
|
||||
// is dropped so a line's width is a function of its value alone —
|
||||
// RFC3339Nano trims trailing zeros, so two consecutive timestamps do
|
||||
// not render to the same number of bytes.
|
||||
func newHandlers() map[string]func(io.Writer) slog.Handler {
|
||||
opts := &slog.HandlerOptions{
|
||||
Level: slog.LevelDebug,
|
||||
ReplaceAttr: func(_ []string, a slog.Attr) slog.Attr {
|
||||
if a.Key == slog.TimeKey {
|
||||
return slog.Attr{}
|
||||
}
|
||||
|
||||
return a
|
||||
},
|
||||
}
|
||||
|
||||
return map[string]func(io.Writer) slog.Handler{
|
||||
"json": func(w io.Writer) slog.Handler {
|
||||
return slog.NewJSONHandler(w, opts)
|
||||
},
|
||||
"text": func(w io.Writer) slog.Handler {
|
||||
return slog.NewTextHandler(w, opts)
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// renderedWidth is the number of bytes a handler writes for a line
|
||||
// carrying value in a single attribute.
|
||||
func renderedWidth(
|
||||
newHandler func(io.Writer) slog.Handler,
|
||||
value string,
|
||||
) int {
|
||||
buf := new(bytes.Buffer)
|
||||
slog.New(newHandler(buf)).Info("m", "v", value)
|
||||
|
||||
return buf.Len()
|
||||
}
|
||||
|
||||
// chargeTestRunes is the set of code points the charge test measures:
|
||||
// every rune in the first two planes' worth of the BMP that the
|
||||
// handlers are most likely to treat specially, the separators that
|
||||
// only slog's JSON handler escapes, and a stratified sample across
|
||||
// the rest of Unicode so the astral charge is exercised on more than
|
||||
// one hand-picked rune.
|
||||
func chargeTestRunes() []rune {
|
||||
const (
|
||||
denseCeiling = 0x800
|
||||
stride = 1021
|
||||
surrogateLo = 0xD800
|
||||
surrogateHi = 0xDFFF
|
||||
)
|
||||
|
||||
var runes []rune
|
||||
|
||||
keep := func(r rune) {
|
||||
if r >= surrogateLo && r <= surrogateHi {
|
||||
return
|
||||
}
|
||||
|
||||
runes = append(runes, r)
|
||||
}
|
||||
|
||||
for r := range rune(denseCeiling) {
|
||||
keep(r)
|
||||
}
|
||||
|
||||
for _, r := range []rune{
|
||||
0x2028, 0x2029, 0x200B, 0x4E00, 0xE000, 0xFFFD,
|
||||
0x1000C, 0x1F600, 0xE0001, 0x10FFFF,
|
||||
} {
|
||||
keep(r)
|
||||
}
|
||||
|
||||
for r := rune(denseCeiling); r <= utf8.MaxRune; r += stride {
|
||||
keep(r)
|
||||
}
|
||||
|
||||
return runes
|
||||
}
|
||||
|
||||
// TestEncodedBytes_ChargesAtLeastWhatTheHandlersEmit is the property
|
||||
// the whole capping scheme rests on: a rune may not cost more on the
|
||||
// line than the budget was charged for it. An undercharged rune is
|
||||
// how a stated ceiling becomes false without any test noticing, so
|
||||
// the charge is measured against what the handlers actually write
|
||||
// rather than against the escaping rules as read.
|
||||
func TestEncodedBytes_ChargesAtLeastWhatTheHandlersEmit(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for name, newHandler := range newHandlers() {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// 'a' is a printable ASCII rune, charged exactly one
|
||||
// byte, so it is the zero point the other runes are
|
||||
// measured against.
|
||||
base := renderedWidth(
|
||||
newHandler, strings.Repeat("a", sampleRunes),
|
||||
)
|
||||
|
||||
for _, r := range chargeTestRunes() {
|
||||
got := renderedWidth(
|
||||
newHandler,
|
||||
strings.Repeat(string(r), sampleRunes),
|
||||
)
|
||||
charged := sampleRunes *
|
||||
(logfield.EncodedBytes(r) - 1)
|
||||
|
||||
require.LessOrEqual(
|
||||
t, got-base, charged+quotingSlack,
|
||||
"U+%04X costs more on the line than "+
|
||||
"EncodedBytes charges for it",
|
||||
r,
|
||||
)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestTruncate_SpendsNoMoreThanTheBudget holds the result to the
|
||||
// budget in ENCODED bytes, which is the unit the budget is stated in.
|
||||
// A raw-byte cap passes the ASCII case here and fails every other
|
||||
// one.
|
||||
func TestTruncate_SpendsNoMoreThanTheBudget(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for name, fill := range map[string]string{
|
||||
"plain": "x",
|
||||
"quote": `"`,
|
||||
"backslash": `\`,
|
||||
"tab": "\t",
|
||||
"newline": "\n",
|
||||
"control": "\x01",
|
||||
"astral": "\U0001000C",
|
||||
// U+4E00, a printable multi-byte rune, charged its three
|
||||
// UTF-8 bytes rather than an escape. Spelled numerically
|
||||
// because gosmopolitan rejects Han in a string literal.
|
||||
"cjk": string(rune(0x4E00)),
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
got := logfield.Truncate(
|
||||
strings.Repeat(fill, budget*8), budget,
|
||||
)
|
||||
|
||||
require.True(
|
||||
t, strings.HasSuffix(
|
||||
got, logfield.TruncationMarker,
|
||||
),
|
||||
"an oversized value must be marked as cut",
|
||||
)
|
||||
|
||||
spent := 0
|
||||
for _, r := range strings.TrimSuffix(
|
||||
got, logfield.TruncationMarker,
|
||||
) {
|
||||
spent += logfield.EncodedBytes(r)
|
||||
}
|
||||
|
||||
assert.LessOrEqual(t, spent, budget)
|
||||
assert.True(t, utf8.ValidString(got))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestTruncate_LeavesShortValuesAlone keeps the marker meaningful: a
|
||||
// value that fits comes back byte for byte, so a marked value is
|
||||
// always a cut one.
|
||||
func TestTruncate_LeavesShortValuesAlone(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, s := range []string{
|
||||
"", "GET", "/source/abc/edit", "Mozilla/5.0 (X11)",
|
||||
} {
|
||||
assert.Equal(t, s, logfield.Truncate(s, budget))
|
||||
}
|
||||
}
|
||||
|
||||
// TestTruncate_DropsInvalidUTF8 covers the bytes a header can carry
|
||||
// that were never valid UTF-8. Keeping them would make the encoder
|
||||
// spend six bytes apiece replacing them, which is exactly the
|
||||
// amplification the budget exists to prevent.
|
||||
func TestTruncate_DropsInvalidUTF8(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
got := logfield.Truncate("a\xffb\xfe\xfec", budget)
|
||||
|
||||
assert.Equal(t, "abc", got)
|
||||
assert.True(t, utf8.ValidString(got))
|
||||
}
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"net/http"
|
||||
|
||||
"github.com/gorilla/csrf"
|
||||
"sneak.berlin/go/webhooker/internal/logfield"
|
||||
)
|
||||
|
||||
// CSRFToken retrieves the CSRF token from the request context.
|
||||
@@ -42,9 +43,22 @@ func isClientTLS(r *http.Request) bool {
|
||||
// csrf.Secure option is set at creation time, not per-request.
|
||||
func (m *Middleware) CSRF() func(http.Handler) http.Handler {
|
||||
csrfErrorHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
// CSRF is registered ahead of RequireAuth on every route
|
||||
// group that uses it, so this WARN is reachable by an
|
||||
// unauthenticated client: a POST with no token to
|
||||
// /source/<any length of any text>/edit lands here. The
|
||||
// method and path are capped against the same budgets as
|
||||
// the access log. remote_addr is set by net/http from the
|
||||
// accepted connection rather than by the client, and
|
||||
// csrf.FailureReason returns one of gorilla/csrf's own
|
||||
// fixed error values, so neither is client-sized.
|
||||
m.log.Warn("csrf: token validation failed",
|
||||
"method", r.Method,
|
||||
"path", r.URL.Path,
|
||||
"method", logfield.Truncate(
|
||||
r.Method, maxLogMethodBytes,
|
||||
),
|
||||
"path", logfield.Truncate(
|
||||
r.URL.Path, logfield.MaxBytes,
|
||||
),
|
||||
"remote_addr", r.RemoteAddr,
|
||||
"reason", csrf.FailureReason(r),
|
||||
)
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"time"
|
||||
)
|
||||
|
||||
// NewLoggingResponseWriterForTest wraps newLoggingResponseWriter
|
||||
@@ -35,9 +37,79 @@ func IsClientTLS(r *http.Request) bool {
|
||||
return isClientTLS(r)
|
||||
}
|
||||
|
||||
// LoginRateLimitConst exposes the loginRateLimit constant.
|
||||
// LoginRateLimitConst exposes the loginRateLimit constant: the
|
||||
// number of FAILED login attempts one client may make against one
|
||||
// submitted username per interval.
|
||||
const LoginRateLimitConst = loginRateLimit
|
||||
|
||||
// LoginFailureMaxKeysConst exposes the cap on each of the login
|
||||
// guard's key sets.
|
||||
const LoginFailureMaxKeysConst = loginFailureMaxKeys
|
||||
|
||||
// PasswordVerifyConcurrencyConst exposes the bound on concurrent
|
||||
// Argon2id verifications.
|
||||
const PasswordVerifyConcurrencyConst = passwordVerifyConcurrency
|
||||
|
||||
// PasswordVerifyMaxWaitersConst exposes the bound on how many
|
||||
// requests may queue for a verification slot.
|
||||
const PasswordVerifyMaxWaitersConst = passwordVerifyMaxWaiters
|
||||
|
||||
// LoginGuard is the login failure counter and verification
|
||||
// semaphore, exposed for direct testing.
|
||||
type LoginGuard = loginGuard
|
||||
|
||||
// NewLoginGuardForTest builds a guard with test-sized parameters.
|
||||
func NewLoginGuardForTest(
|
||||
limit int,
|
||||
interval time.Duration,
|
||||
maxKeys, concurrency, maxWaiters int,
|
||||
wait time.Duration,
|
||||
) *LoginGuard {
|
||||
return newLoginGuard(
|
||||
limit, interval, maxKeys, concurrency, maxWaiters, wait,
|
||||
)
|
||||
}
|
||||
|
||||
// QueuedWaitersForTest reports how many requests are currently
|
||||
// queued for a verification slot.
|
||||
func (g *LoginGuard) QueuedWaitersForTest() int {
|
||||
return len(g.queue)
|
||||
}
|
||||
|
||||
// SetNowForTest replaces the guard's clock.
|
||||
func (g *LoginGuard) SetNowForTest(now func() time.Time) {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
|
||||
g.now = now
|
||||
}
|
||||
|
||||
// FailForTest exposes fail.
|
||||
func (g *LoginGuard) FailForTest(clientKey, username string) bool {
|
||||
return g.fail(clientKey, username)
|
||||
}
|
||||
|
||||
// SucceedForTest exposes succeed.
|
||||
func (g *LoginGuard) SucceedForTest(clientKey, username string) {
|
||||
g.succeed(clientKey, username)
|
||||
}
|
||||
|
||||
// AcquireForTest exposes acquire.
|
||||
func (g *LoginGuard) AcquireForTest(
|
||||
ctx context.Context,
|
||||
) (func(), bool) {
|
||||
return g.acquire(ctx)
|
||||
}
|
||||
|
||||
// TrackedKeysForTest reports how many failure counters the guard
|
||||
// holds, per-username and per-address respectively.
|
||||
func (g *LoginGuard) TrackedKeysForTest() (int, int) {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
|
||||
return len(g.byUser), len(g.byAddr)
|
||||
}
|
||||
|
||||
// PasswordChangeRateLimitConst exposes the
|
||||
// passwordChangeRateLimit constant.
|
||||
const PasswordChangeRateLimitConst = passwordChangeRateLimit
|
||||
|
||||
480
internal/middleware/logbound_test.go
Normal file
480
internal/middleware/logbound_test.go
Normal file
@@ -0,0 +1,480 @@
|
||||
package middleware_test
|
||||
|
||||
// This file covers the log lines OUTSIDE the access log that carry a
|
||||
// client-chosen value. accesslog_test.go bounds the one INFO line the
|
||||
// Logging middleware writes; these are the separate slog calls that
|
||||
// were never in that sweep and so never got the budget:
|
||||
//
|
||||
// - MaxBodySize's 413 rejection, at WARN, registered ahead of
|
||||
// RequireAuth and therefore reachable unauthenticated at a URL of
|
||||
// the client's choosing.
|
||||
// - CSRF's 403 rejection, at WARN, also registered ahead of
|
||||
// RequireAuth.
|
||||
// - The rate limiters' 429 rejection, at WARN, on the
|
||||
// unauthenticated receiver among others.
|
||||
// - RequireAuth's own unauthenticated-request line, at DEBUG.
|
||||
//
|
||||
// Every case here holds the ENCODED line to
|
||||
// middleware.MaxAccessLogLineBytes, under both handlers
|
||||
// internal/logger can install, against 8 KB of client-chosen text
|
||||
// built out of the characters those handlers escape. A budget spent
|
||||
// in raw bytes passes the plain-ASCII cases and fails the rest.
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"sneak.berlin/go/webhooker/internal/config"
|
||||
"sneak.berlin/go/webhooker/internal/middleware"
|
||||
)
|
||||
|
||||
// bodyLimitBytes is the MaxBodySize cap these tests install. Any
|
||||
// declared Content-Length above it takes the 413 branch.
|
||||
const bodyLimitBytes = 1024
|
||||
|
||||
// declaredBodyBytes is the Content-Length an oversize request
|
||||
// declares. Nothing is actually sent: the 413 branch fires off the
|
||||
// declaration alone, which is what makes the attack free.
|
||||
const declaredBodyBytes = bodyLimitBytes * 2
|
||||
|
||||
// receiverLimitPerMinute is the per-entrypoint receiver limit these
|
||||
// tests install. The aggregate limiter sits at ten times this, so a
|
||||
// flood stays under it and the rejections come from the
|
||||
// per-entrypoint limiter, which is the one that logs the path.
|
||||
const receiverLimitPerMinute = 8
|
||||
|
||||
// escapeFills are the characters a client can put in a request that
|
||||
// the log handlers then escape, coming out wider than they went in.
|
||||
// A budget counted in raw bytes lets any of them buy a field several
|
||||
// times its nominal size.
|
||||
//
|
||||
// U+1000C is the case the JSON handler alone does not reach: it is
|
||||
// unassigned, so it is non-printable, and strconv.Quote spells a
|
||||
// non-printable rune at or above U+10000 as a ten-byte \UXXXXXXXX
|
||||
// while the JSON handler passes its four UTF-8 bytes through. Only
|
||||
// the text-handler shape of these tests holds that charge honest.
|
||||
func escapeFills() map[string]string {
|
||||
return map[string]string{
|
||||
"plain": "x",
|
||||
"quote": `"`,
|
||||
"backslash": `\`,
|
||||
"tab": "\t",
|
||||
"newline": "\n",
|
||||
// A C0 control neither handler has a short escape for, so
|
||||
// each one costs six bytes on the line against the single
|
||||
// byte it cost to send. This is the widest multiplier a
|
||||
// client can drive, and the case a raw-byte budget breaks
|
||||
// on first.
|
||||
//
|
||||
// This fill is load-bearing, not decoration. Budgeting raw
|
||||
// bytes instead of encoded is caught by this fill alone,
|
||||
// and only under the JSON handler, at 3,072 bytes against
|
||||
// the 2,560 ceiling. Drop it and that mutation passes.
|
||||
"control": "\x01",
|
||||
"astral": "\U0001000C",
|
||||
}
|
||||
}
|
||||
|
||||
// logHandlers are the two handlers internal/logger can install: the
|
||||
// JSON one, and the text one it selects when stderr is a tty. They do
|
||||
// not escape alike, and MaxAccessLogLineBytes is quoted unqualified,
|
||||
// so every case runs through both.
|
||||
func logHandlers() map[string]func(
|
||||
io.Writer, *slog.HandlerOptions,
|
||||
) slog.Handler {
|
||||
return map[string]func(
|
||||
io.Writer, *slog.HandlerOptions,
|
||||
) slog.Handler{
|
||||
"json": func(
|
||||
w io.Writer, o *slog.HandlerOptions,
|
||||
) slog.Handler {
|
||||
return slog.NewJSONHandler(w, o)
|
||||
},
|
||||
"text": func(
|
||||
w io.Writer, o *slog.HandlerOptions,
|
||||
) slog.Handler {
|
||||
return slog.NewTextHandler(w, o)
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// oversizedPathSegment builds an 8 KB client-chosen path segment out
|
||||
// of repetitions of ch, percent-encoded so it survives URL parsing
|
||||
// into r.URL.Path the way it would arriving off a socket.
|
||||
//
|
||||
// Both markers sit at the END, past every budget, so their absence
|
||||
// from the log is what proves the value was cut rather than merely
|
||||
// being short. The leading 'x' keeps the segment non-empty for fills
|
||||
// that a parser might otherwise fold away.
|
||||
func oversizedPathSegment(ch string) string {
|
||||
return url.PathEscape(
|
||||
"x" + strings.Repeat(ch, oversizedSegmentBytes) +
|
||||
attackerMarker + tailMarker,
|
||||
)
|
||||
}
|
||||
|
||||
// capturingLogger returns a logger at DEBUG writing into the returned
|
||||
// buffer through the named handler.
|
||||
func capturingLogger(
|
||||
newHandler func(io.Writer, *slog.HandlerOptions) slog.Handler,
|
||||
) (*slog.Logger, *bytes.Buffer) {
|
||||
buf := new(bytes.Buffer)
|
||||
opts := &slog.HandlerOptions{Level: slog.LevelDebug}
|
||||
|
||||
return slog.New(newHandler(buf, opts)), buf
|
||||
}
|
||||
|
||||
// capturingBoundMiddleware builds a Middleware with a real session
|
||||
// manager (CSRF needs its key, RequireAuth needs its store) whose log
|
||||
// is captured at DEBUG.
|
||||
func capturingBoundMiddleware(
|
||||
t *testing.T,
|
||||
newHandler func(io.Writer, *slog.HandlerOptions) slog.Handler,
|
||||
) (*middleware.Middleware, *bytes.Buffer) {
|
||||
t.Helper()
|
||||
|
||||
log, buf := capturingLogger(newHandler)
|
||||
|
||||
cfg := &config.Config{
|
||||
Environment: config.EnvironmentDev,
|
||||
ReceiverRateLimit: receiverLimitPerMinute,
|
||||
}
|
||||
|
||||
sess := newTestSessionManager(cfg, log, nil)
|
||||
|
||||
return middleware.NewForTest(log, cfg, sess), buf
|
||||
}
|
||||
|
||||
// unreachable is a next-handler that fails the test if the middleware
|
||||
// under test let the request through. Every site here rejects.
|
||||
func unreachable(t *testing.T) http.Handler {
|
||||
t.Helper()
|
||||
|
||||
return http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
|
||||
assert.Fail(t, "rejected request reached the next handler")
|
||||
})
|
||||
}
|
||||
|
||||
// logSite is one non-access-log call site that logs a client-chosen
|
||||
// path. drive sends requests at it that all take the rejecting
|
||||
// branch; linesPerRequest is how many log lines one such request
|
||||
// produces there.
|
||||
type logSite struct {
|
||||
// build wraps the site's middleware around a handler that must
|
||||
// not be reached.
|
||||
build func(
|
||||
t *testing.T, m *middleware.Middleware,
|
||||
) http.Handler
|
||||
// send issues one request for the given client-chosen path and
|
||||
// returns the status. Some sites need a warm-up request before
|
||||
// they reject, which send performs itself.
|
||||
send func(h http.Handler, path string) int
|
||||
// wantStatus is the status the rejecting branch answers with.
|
||||
wantStatus int
|
||||
}
|
||||
|
||||
// postOversize sends a POST whose declared Content-Length exceeds the
|
||||
// body limit without sending a body, which is the whole cost of the
|
||||
// attack on the MaxBodySize branch.
|
||||
func postOversize(h http.Handler, path string) int {
|
||||
req := httptest.NewRequestWithContext(
|
||||
context.Background(), http.MethodPost, path, nil,
|
||||
)
|
||||
req.ContentLength = declaredBodyBytes
|
||||
req.Header.Set(
|
||||
"Content-Type", "application/x-www-form-urlencoded",
|
||||
)
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, req)
|
||||
|
||||
return w.Code
|
||||
}
|
||||
|
||||
// postNoToken sends a POST carrying no CSRF token and no session
|
||||
// cookie, which is what an unauthenticated client sends.
|
||||
func postNoToken(h http.Handler, path string) int {
|
||||
req := httptest.NewRequestWithContext(
|
||||
context.Background(), http.MethodPost, path,
|
||||
strings.NewReader(""),
|
||||
)
|
||||
req.Header.Set(
|
||||
"Content-Type", "application/x-www-form-urlencoded",
|
||||
)
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, req)
|
||||
|
||||
return w.Code
|
||||
}
|
||||
|
||||
// getNoSession sends a GET with no session cookie.
|
||||
func getNoSession(h http.Handler, path string) int {
|
||||
req := httptest.NewRequestWithContext(
|
||||
context.Background(), http.MethodGet, path, nil,
|
||||
)
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, req)
|
||||
|
||||
return w.Code
|
||||
}
|
||||
|
||||
// logSites enumerates the call sites under test.
|
||||
func logSites() map[string]logSite {
|
||||
return map[string]logSite{
|
||||
// The site this file exists for: WARN, on by default, and
|
||||
// registered ahead of RequireAuth.
|
||||
"maxbodysize 413": {
|
||||
build: func(
|
||||
t *testing.T, m *middleware.Middleware,
|
||||
) http.Handler {
|
||||
t.Helper()
|
||||
|
||||
return m.MaxBodySize(bodyLimitBytes)(
|
||||
unreachable(t),
|
||||
)
|
||||
},
|
||||
send: postOversize,
|
||||
wantStatus: http.StatusRequestEntityTooLarge,
|
||||
},
|
||||
// Also ahead of RequireAuth, also WARN.
|
||||
"csrf 403": {
|
||||
build: func(
|
||||
t *testing.T, m *middleware.Middleware,
|
||||
) http.Handler {
|
||||
t.Helper()
|
||||
|
||||
return m.CSRF()(unreachable(t))
|
||||
},
|
||||
send: postNoToken,
|
||||
wantStatus: http.StatusForbidden,
|
||||
},
|
||||
// The per-entrypoint receiver limiter, unauthenticated. Its
|
||||
// bucket is keyed on the path, so the first request through a
|
||||
// fresh path is served and only the ones after it are
|
||||
// rejected; sendUntilLimited absorbs that.
|
||||
"receiver rate limit 429": {
|
||||
build: func(
|
||||
t *testing.T, m *middleware.Middleware,
|
||||
) http.Handler {
|
||||
t.Helper()
|
||||
|
||||
return m.ReceiverRateLimit()(okHandler())
|
||||
},
|
||||
send: sendUntilLimited,
|
||||
wantStatus: http.StatusTooManyRequests,
|
||||
},
|
||||
// RequireAuth's own line. DEBUG is off in production by
|
||||
// default, but turning it on to diagnose a flood must not
|
||||
// restore an unbounded write.
|
||||
"requireauth redirect": {
|
||||
build: func(
|
||||
t *testing.T, m *middleware.Middleware,
|
||||
) http.Handler {
|
||||
t.Helper()
|
||||
|
||||
return m.RequireAuth()(unreachable(t))
|
||||
},
|
||||
send: getNoSession,
|
||||
wantStatus: http.StatusSeeOther,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// sendUntilLimited drives the per-entrypoint receiver limiter past
|
||||
// its allowance on one path and returns the status of the rejected
|
||||
// request. Every request before the last is served, and only the last
|
||||
// one logs.
|
||||
func sendUntilLimited(h http.Handler, path string) int {
|
||||
code := http.StatusOK
|
||||
|
||||
for range receiverLimitPerMinute + 1 {
|
||||
req := httptest.NewRequestWithContext(
|
||||
context.Background(), http.MethodPost, path, nil,
|
||||
)
|
||||
req.RemoteAddr = "203.0.113.7:5555"
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, req)
|
||||
code = w.Code
|
||||
}
|
||||
|
||||
return code
|
||||
}
|
||||
|
||||
// logLines splits the captured buffer into non-empty lines, holding
|
||||
// each to bound bytes.
|
||||
func logLines(t *testing.T, buf *bytes.Buffer, bound int) []string {
|
||||
t.Helper()
|
||||
|
||||
var lines []string
|
||||
|
||||
for line := range strings.SplitSeq(
|
||||
strings.TrimSpace(buf.String()), "\n",
|
||||
) {
|
||||
if line == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
require.LessOrEqual(
|
||||
t, len(line), bound,
|
||||
"log line exceeded its bound: %s", line,
|
||||
)
|
||||
|
||||
lines = append(lines, line)
|
||||
}
|
||||
|
||||
return lines
|
||||
}
|
||||
|
||||
// assertNoClientText fails if any marker from the far end of the
|
||||
// client-chosen input survived into the log. Their absence is what
|
||||
// distinguishes a real cut from a value that merely happened to be
|
||||
// short.
|
||||
func assertNoClientText(t *testing.T, buf *bytes.Buffer) {
|
||||
t.Helper()
|
||||
|
||||
assert.NotContains(
|
||||
t, buf.String(), attackerMarker,
|
||||
"log carried attacker-chosen text",
|
||||
)
|
||||
assert.NotContains(
|
||||
t, buf.String(), tailMarker,
|
||||
"log carried the tail of the attacker-chosen text",
|
||||
)
|
||||
}
|
||||
|
||||
// TestLogLines_ClientChosenPathDoesNotSizeTheLine points 8 KB of
|
||||
// client-chosen path at each non-access-log call site that logs one,
|
||||
// through both handlers and through every character those handlers
|
||||
// escape, and holds the resulting line to MaxAccessLogLineBytes.
|
||||
//
|
||||
// Removing any one of the logfield.Truncate calls at those sites
|
||||
// fails this test: the line grows to roughly the size of the input,
|
||||
// or to several times it on the escaping fills.
|
||||
func TestLogLines_ClientChosenPathDoesNotSizeTheLine(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for siteName, site := range logSites() {
|
||||
for handlerName, newHandler := range logHandlers() {
|
||||
for fillName, fill := range escapeFills() {
|
||||
name := siteName + "/" + handlerName + "/" + fillName
|
||||
|
||||
t.Run(name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
m, buf := capturingBoundMiddleware(
|
||||
t, newHandler,
|
||||
)
|
||||
|
||||
path := "/source/" +
|
||||
oversizedPathSegment(fill) + "/edit"
|
||||
|
||||
assert.Equal(
|
||||
t,
|
||||
site.wantStatus,
|
||||
site.send(site.build(t, m), path),
|
||||
)
|
||||
|
||||
lines := logLines(
|
||||
t, buf,
|
||||
middleware.MaxAccessLogLineBytes,
|
||||
)
|
||||
require.NotEmpty(
|
||||
t, lines,
|
||||
"the site under test logged nothing, "+
|
||||
"so the bound proves nothing",
|
||||
)
|
||||
assertNoClientText(t, buf)
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestMaxBodySize_FloodOfOversizePathsDoesNotGrowTheLog is the
|
||||
// flood shape from the issue: an unauthenticated client posting
|
||||
// oversize declarations at invented 8 KB paths, as fast as it likes.
|
||||
//
|
||||
// It asserts the property directly rather than by proxy — the bytes
|
||||
// the flood writes to the operator's log do not track the bytes the
|
||||
// flood sent. The same flood at a one-character path is the control:
|
||||
// 8 KB of extra input per request buys at most the field budget, not
|
||||
// 8 KB of log.
|
||||
func TestMaxBodySize_FloodOfOversizePathsDoesNotGrowTheLog(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
for handlerName, newHandler := range logHandlers() {
|
||||
for fillName, fill := range escapeFills() {
|
||||
t.Run(handlerName+"/"+fillName, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
flood := func(segment func(i int) string) int {
|
||||
m, buf := capturingBoundMiddleware(
|
||||
t, newHandler,
|
||||
)
|
||||
h := m.MaxBodySize(bodyLimitBytes)(
|
||||
unreachable(t),
|
||||
)
|
||||
|
||||
for i := range floodRequests {
|
||||
assert.Equal(
|
||||
t,
|
||||
http.StatusRequestEntityTooLarge,
|
||||
postOversize(
|
||||
h,
|
||||
"/source/"+segment(i)+"/edit",
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
lines := logLines(
|
||||
t, buf,
|
||||
middleware.MaxAccessLogLineBytes,
|
||||
)
|
||||
require.Len(t, lines, floodRequests)
|
||||
assertNoClientText(t, buf)
|
||||
|
||||
return buf.Len()
|
||||
}
|
||||
|
||||
sent := oversizedSegmentBytes * floodRequests
|
||||
|
||||
oversize := flood(func(i int) string {
|
||||
return oversizedPathSegment(fill) +
|
||||
strings.Repeat("y", i)
|
||||
})
|
||||
control := flood(func(i int) string {
|
||||
return "a" + strings.Repeat("y", i)
|
||||
})
|
||||
|
||||
// The whole point: 8 KB per request of extra
|
||||
// client-chosen input bought a bounded amount of
|
||||
// log, not a proportional amount.
|
||||
assert.Less(
|
||||
t, oversize-control, sent/2,
|
||||
"log volume tracked the size of the flood's "+
|
||||
"input",
|
||||
)
|
||||
assert.LessOrEqual(
|
||||
t,
|
||||
oversize,
|
||||
floodRequests*
|
||||
middleware.MaxAccessLogLineBytes,
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
370
internal/middleware/loginguard.go
Normal file
370
internal/middleware/loginguard.go
Normal file
@@ -0,0 +1,370 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
const (
|
||||
// loginFailureMaxKeys bounds how many distinct failure counters
|
||||
// each of the guard's two key sets holds. The submitted username
|
||||
// is part of a key, so the key set is attacker-influenced and
|
||||
// needs a hard cap or the limiter becomes the memory
|
||||
// amplification surface it exists to protect.
|
||||
//
|
||||
// A single-admin deployment has a handful of legitimate (client,
|
||||
// username) pairs, so 1024 is three orders of magnitude of
|
||||
// headroom before a real operator can be pushed onto the
|
||||
// fallback. It costs little: a counter is a ~64-byte key string,
|
||||
// a 32-byte window and map overhead, call it 170 bytes, so both
|
||||
// key sets full is 2 * 1024 * 170 bytes, under 0.4 MB.
|
||||
loginFailureMaxKeys = 1024
|
||||
|
||||
// passwordVerifyConcurrency bounds how many Argon2id
|
||||
// verifications may run at once across every password-verifying
|
||||
// endpoint. Because credentials are now verified before any
|
||||
// limiter budget is spent, an attacker can force one hash per
|
||||
// request, and each hash allocates argon2Memory — 64 MB. Two
|
||||
// slots commit at most 128 MB to password hashing, which fits
|
||||
// inside the smallest container this service is realistically
|
||||
// given alongside its own working set; four would commit 256 MB
|
||||
// and crowd it. A single-admin product needs no concurrent
|
||||
// logins at all, so the second slot exists only so that one
|
||||
// stalled request does not serialise the endpoint.
|
||||
passwordVerifyConcurrency = 2
|
||||
|
||||
// passwordVerifyWait is how long a request waits for a
|
||||
// verification slot before it is answered 503. Slots are handed
|
||||
// out in arrival order, so a legitimate request queues behind
|
||||
// the requests already waiting rather than behind the flood as a
|
||||
// whole. The wait is well inside the 60s request timeout.
|
||||
passwordVerifyWait = 5 * time.Second
|
||||
|
||||
// passwordVerifyMaxWaiters bounds how many requests may be
|
||||
// queued for a slot at once. Past it, acquire sheds immediately
|
||||
// with 503 instead of joining the queue.
|
||||
//
|
||||
// The wait bounds how long one request occupies memory; this
|
||||
// bounds how many do so at the same time, and without it the
|
||||
// 128 MB hashing budget above is the smaller half of the real
|
||||
// footprint. At the 400 req/s a saturation attack can offer, an
|
||||
// unbounded queue would park ~2000 requests for the full five
|
||||
// seconds.
|
||||
//
|
||||
// A waiter costs far more than maxFormBodySize suggests: that
|
||||
// caps the raw body read, not what the parse retains. MaxBodySize,
|
||||
// CSRF and ParseForm all run before acquire, so a parked waiter
|
||||
// holds r.Form plus r.PostForm plus its header block for the
|
||||
// whole wait. Measured on the pinned go1.26.1 toolchain, as the
|
||||
// HeapAlloc delta across two GCs with 64 waiters parked in the
|
||||
// handler: an ordinary two-field login form retains ~0 MB, but a
|
||||
// 1 MB urlencoded body at Go's 10,000-parameter parse cap retains
|
||||
// 2.82 MB (3.09 MB with %41 escapes), and adding the ~0.9 MB of
|
||||
// headers httpMaxHeaderBytes allows takes it to 4.18 MB. The
|
||||
// retained parse and the header block dominate; the raw body does
|
||||
// not.
|
||||
//
|
||||
// Arithmetic, from the measured 4.18 MB worst case: 16 waiters
|
||||
// commit ~67 MB of queue memory, and peak commitment for the
|
||||
// endpoint is 128 MB of Argon2id plus the 18 requests that retain
|
||||
// a parsed form — 16 queued and the 2 being hashed — at
|
||||
// 18 * 4.18 MB, so ~75 MB: about 203 MB in all. Cross-check
|
||||
// against the deadline: two slots at the ~27 verifications/s
|
||||
// measured on a review host (with the race detector on, so the
|
||||
// real rate is higher) drain a full 16-deep queue in about 0.6 s,
|
||||
// far inside passwordVerifyWait.
|
||||
//
|
||||
// Those 203 MB are live bytes, not resident bytes: the Go
|
||||
// collector lets the heap reach roughly twice the live set before
|
||||
// collecting, with transient parse garbage on top. The review
|
||||
// measured a peak HeapAlloc of 392 MB against this guard under 18
|
||||
// adversarial requests, so provision on the order of 400 MB rather
|
||||
// than 203 MB.
|
||||
passwordVerifyMaxWaiters = 16
|
||||
|
||||
// failureKeyHashBytes is how much of the username digest goes
|
||||
// into a failure key. 64 bits over at most loginFailureMaxKeys
|
||||
// live keys makes a collision negligible, and a collision would
|
||||
// only merge two usernames' failure counters, which throttles
|
||||
// sooner rather than later.
|
||||
failureKeyHashBytes = 8
|
||||
)
|
||||
|
||||
// failureWindow counts failed credential verifications for one
|
||||
// bucket, and records when that count lapses.
|
||||
type failureWindow struct {
|
||||
count int
|
||||
resetAt time.Time
|
||||
}
|
||||
|
||||
// loginGuard is what replaced the pre-emptive rate limiter on the
|
||||
// login POST.
|
||||
//
|
||||
// A limiter that spends budget on arrival cannot protect a
|
||||
// single-admin product: behind the reverse proxy the deployment
|
||||
// requires, with TRUSTED_PROXIES unset, every client keys on the
|
||||
// proxy, so a stranger trickling five POSTs a minute keeps the one
|
||||
// bucket full and the operator's own correct password is answered 429
|
||||
// forever. There is no second administrative path.
|
||||
//
|
||||
// So budget is spent only by a FAILED verification. A correct
|
||||
// password is never throttled, whatever the counters say, which is
|
||||
// the only shape that guarantees the operator can get in. Two
|
||||
// consequences follow and are handled here:
|
||||
//
|
||||
// - Every login request now costs an Argon2id hash, so the number
|
||||
// running concurrently is bounded by slots. Without that bound
|
||||
// this trades an admin lockout for memory exhaustion, which is
|
||||
// strictly worse.
|
||||
// - Counting per (client, username) makes the key set
|
||||
// attacker-influenced, so both key sets are capped. Beyond the
|
||||
// per-username cap, failures fall back to a counter keyed on the
|
||||
// client alone; beyond that cap too, a failure is answered as
|
||||
// throttled without being recorded, since refusing to answer a
|
||||
// wrong password costs the operator nothing.
|
||||
type loginGuard struct {
|
||||
mu sync.Mutex
|
||||
byUser map[string]*failureWindow
|
||||
byAddr map[string]*failureWindow
|
||||
|
||||
slots chan struct{}
|
||||
|
||||
// queue holds one token per request waiting for a slot. A token
|
||||
// is taken non-blockingly, so a request that finds it full is
|
||||
// shed rather than queued, and is given up as soon as the wait
|
||||
// ends however it ends.
|
||||
queue chan struct{}
|
||||
|
||||
limit int
|
||||
interval time.Duration
|
||||
maxKeys int
|
||||
wait time.Duration
|
||||
|
||||
// now is time.Now outside tests.
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
// newLoginGuard builds a guard with the given failure limit per
|
||||
// interval, key-set cap, verification concurrency, queue depth and
|
||||
// slot wait.
|
||||
func newLoginGuard(
|
||||
limit int,
|
||||
interval time.Duration,
|
||||
maxKeys, concurrency, maxWaiters int,
|
||||
wait time.Duration,
|
||||
) *loginGuard {
|
||||
return &loginGuard{
|
||||
byUser: make(map[string]*failureWindow),
|
||||
byAddr: make(map[string]*failureWindow),
|
||||
slots: make(chan struct{}, concurrency),
|
||||
queue: make(chan struct{}, maxWaiters),
|
||||
limit: limit,
|
||||
interval: interval,
|
||||
maxKeys: maxKeys,
|
||||
wait: wait,
|
||||
now: time.Now,
|
||||
}
|
||||
}
|
||||
|
||||
// acquire reserves a verification slot, waiting up to the guard's
|
||||
// wait for one. It reports false when the queue of waiters is
|
||||
// already full, when no slot became available in time, or when the
|
||||
// request was cancelled first; the caller must then answer 503
|
||||
// without verifying anything. The returned function releases the
|
||||
// slot and must be called exactly once.
|
||||
func (g *loginGuard) acquire(ctx context.Context) (func(), bool) {
|
||||
// Shedding past the queue depth is what keeps waiting memory
|
||||
// bounded; the wait alone only bounds how long one waiter holds
|
||||
// its parsed form, not how many hold one at once.
|
||||
select {
|
||||
case g.queue <- struct{}{}:
|
||||
default:
|
||||
return nil, false
|
||||
}
|
||||
|
||||
// Held only for the wait. A request that gets a slot gives its
|
||||
// queue token back before it starts hashing, so the depth is a
|
||||
// bound on waiters rather than on requests in the handler.
|
||||
defer func() { <-g.queue }()
|
||||
|
||||
timer := time.NewTimer(g.wait)
|
||||
defer timer.Stop()
|
||||
|
||||
// The blocking send is deliberate: a receive on a full buffered
|
||||
// channel hands the slot straight to the head of the send queue,
|
||||
// so slots go out in arrival order and a later arrival cannot
|
||||
// barge past a request already waiting.
|
||||
select {
|
||||
case g.slots <- struct{}{}:
|
||||
return func() { <-g.slots }, true
|
||||
case <-timer.C:
|
||||
return nil, false
|
||||
case <-ctx.Done():
|
||||
return nil, false
|
||||
}
|
||||
}
|
||||
|
||||
// fail records one failed credential verification by clientKey
|
||||
// against username, and reports whether this client has now spent
|
||||
// its failure budget and should be answered 429.
|
||||
func (g *loginGuard) fail(clientKey, username string) bool {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
|
||||
now := g.now()
|
||||
|
||||
window := g.window(
|
||||
g.byUser, userFailureKey(clientKey, username), now,
|
||||
)
|
||||
if window == nil {
|
||||
window = g.window(g.byAddr, clientKey, now)
|
||||
}
|
||||
|
||||
if window == nil {
|
||||
// Both key sets are full and neither already tracks this
|
||||
// client, so nothing can be counted without unbounded
|
||||
// growth. Answering the failure as throttled is the safe
|
||||
// direction: it never touches a correct password.
|
||||
return true
|
||||
}
|
||||
|
||||
window.count++
|
||||
|
||||
return window.count >= g.limit
|
||||
}
|
||||
|
||||
// succeed forgives clientKey's failures against username. A correct
|
||||
// password clears the counters, so an operator who mistypes several
|
||||
// times and then gets it right is not throttled afterwards.
|
||||
func (g *loginGuard) succeed(clientKey, username string) {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
|
||||
delete(g.byUser, userFailureKey(clientKey, username))
|
||||
delete(g.byAddr, clientKey)
|
||||
}
|
||||
|
||||
// window returns the live counter for key in set, resetting a lapsed
|
||||
// one and creating a missing one when the cap allows. It returns nil
|
||||
// only when key is absent and set is full even after lapsed entries
|
||||
// are swept.
|
||||
func (g *loginGuard) window(
|
||||
set map[string]*failureWindow,
|
||||
key string,
|
||||
now time.Time,
|
||||
) *failureWindow {
|
||||
window, ok := set[key]
|
||||
if ok {
|
||||
if !now.Before(window.resetAt) {
|
||||
window.count = 0
|
||||
window.resetAt = now.Add(g.interval)
|
||||
}
|
||||
|
||||
return window
|
||||
}
|
||||
|
||||
if len(set) >= g.maxKeys {
|
||||
sweepLapsed(set, now)
|
||||
}
|
||||
|
||||
if len(set) >= g.maxKeys {
|
||||
return nil
|
||||
}
|
||||
|
||||
window = &failureWindow{resetAt: now.Add(g.interval)}
|
||||
set[key] = window
|
||||
|
||||
return window
|
||||
}
|
||||
|
||||
// sweepLapsed drops counters whose interval has elapsed.
|
||||
func sweepLapsed(set map[string]*failureWindow, now time.Time) {
|
||||
for key, window := range set {
|
||||
if !now.Before(window.resetAt) {
|
||||
delete(set, key)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// userFailureKey identifies one (client, submitted username) pair.
|
||||
// The username is hashed rather than embedded: a submitted username
|
||||
// is attacker-controlled text of attacker-chosen length, and hashing
|
||||
// makes every key the same size whatever was sent.
|
||||
func userFailureKey(clientKey, username string) string {
|
||||
sum := sha256.Sum256([]byte(username))
|
||||
|
||||
return clientKey + "|" +
|
||||
hex.EncodeToString(sum[:failureKeyHashBytes])
|
||||
}
|
||||
|
||||
// guard returns the middleware's login guard, building it on first
|
||||
// use so that every construction path — fx and the test constructor
|
||||
// alike — gets one.
|
||||
func (m *Middleware) guard() *loginGuard {
|
||||
m.loginGuardOnce.Do(func() {
|
||||
m.loginGuard = newLoginGuard(
|
||||
loginRateLimit,
|
||||
loginRateInterval,
|
||||
loginFailureMaxKeys,
|
||||
passwordVerifyConcurrency,
|
||||
passwordVerifyMaxWaiters,
|
||||
passwordVerifyWait,
|
||||
)
|
||||
})
|
||||
|
||||
return m.loginGuard
|
||||
}
|
||||
|
||||
// BeginPasswordVerification reserves one of the bounded Argon2id
|
||||
// verification slots. It reports false when the queue of waiting
|
||||
// requests is already at passwordVerifyMaxWaiters, or when no slot
|
||||
// became free within passwordVerifyWait; in either case the caller
|
||||
// must answer 503 and must not verify a password. The returned
|
||||
// function releases the slot and must be called exactly once.
|
||||
//
|
||||
// Every endpoint that hashes a password on request must go through
|
||||
// this, or the bound has a hole: the memory is committed per hash,
|
||||
// not per endpoint.
|
||||
func (m *Middleware) BeginPasswordVerification(
|
||||
ctx context.Context,
|
||||
) (func(), bool) {
|
||||
return m.guard().acquire(ctx)
|
||||
}
|
||||
|
||||
// RecordLoginFailure counts a failed credential verification for the
|
||||
// request's client against the submitted username, and reports
|
||||
// whether the response should be 429 rather than 401.
|
||||
func (m *Middleware) RecordLoginFailure(
|
||||
r *http.Request,
|
||||
username string,
|
||||
) bool {
|
||||
throttled := m.guard().fail(m.clientKey(r), username)
|
||||
if throttled {
|
||||
m.log.Warn(
|
||||
"login failure limit exceeded", "path", r.URL.Path,
|
||||
)
|
||||
}
|
||||
|
||||
return throttled
|
||||
}
|
||||
|
||||
// ForgiveLoginFailures clears the failure counters for the request's
|
||||
// client and the submitted username after a successful
|
||||
// authentication.
|
||||
func (m *Middleware) ForgiveLoginFailures(
|
||||
r *http.Request,
|
||||
username string,
|
||||
) {
|
||||
m.guard().succeed(m.clientKey(r), username)
|
||||
}
|
||||
|
||||
// LoginFailureInterval is how long a spent login failure budget
|
||||
// takes to refill, which is what a throttled login answers as
|
||||
// Retry-After.
|
||||
func (m *Middleware) LoginFailureInterval() time.Duration {
|
||||
return m.guard().interval
|
||||
}
|
||||
511
internal/middleware/loginguard_test.go
Normal file
511
internal/middleware/loginguard_test.go
Normal file
@@ -0,0 +1,511 @@
|
||||
package middleware_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"sneak.berlin/go/webhooker/internal/database"
|
||||
"sneak.berlin/go/webhooker/internal/middleware"
|
||||
)
|
||||
|
||||
// mib converts the Argon2id memory parameter, which is in KiB, to MB.
|
||||
const mib = 1024
|
||||
|
||||
const (
|
||||
// guardInterval is the failure window these tests use. It is
|
||||
// long enough that nothing lapses mid-test on its own; tests
|
||||
// that need a lapse drive the clock instead.
|
||||
guardInterval = time.Minute
|
||||
|
||||
// guardWait is the slot wait for tests that expect to get a
|
||||
// slot. Tests that expect to be refused set their own.
|
||||
guardWait = 2 * time.Second
|
||||
|
||||
guardClient = "198.51.100.7"
|
||||
guardUser = "admin"
|
||||
)
|
||||
|
||||
// newGuard builds a guard with production-shaped defaults and the
|
||||
// given key-set cap and verification concurrency.
|
||||
func newGuard(maxKeys, concurrency int) *middleware.LoginGuard {
|
||||
return middleware.NewLoginGuardForTest(
|
||||
middleware.LoginRateLimitConst,
|
||||
guardInterval,
|
||||
maxKeys,
|
||||
concurrency,
|
||||
middleware.PasswordVerifyMaxWaitersConst,
|
||||
guardWait,
|
||||
)
|
||||
}
|
||||
|
||||
// TestLoginGuard_ThrottlesRepeatedFailures is the brute-force half:
|
||||
// wrong passwords for one username from one client key still run out
|
||||
// of budget and are answered 429.
|
||||
func TestLoginGuard_ThrottlesRepeatedFailures(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
g := newGuard(middleware.LoginFailureMaxKeysConst, 1)
|
||||
|
||||
for i := range middleware.LoginRateLimitConst - 1 {
|
||||
assert.False(
|
||||
t, g.FailForTest(guardClient, guardUser),
|
||||
"failure %d is still inside the budget", i,
|
||||
)
|
||||
}
|
||||
|
||||
assert.True(
|
||||
t, g.FailForTest(guardClient, guardUser),
|
||||
"the last failure of the budget must throttle",
|
||||
)
|
||||
assert.True(
|
||||
t, g.FailForTest(guardClient, guardUser),
|
||||
"failures past the budget must stay throttled",
|
||||
)
|
||||
}
|
||||
|
||||
// TestLoginGuard_SuccessForgivesFailures pins the forgiveness rule:
|
||||
// an operator who mistypes several times and then gets it right must
|
||||
// not be left throttled.
|
||||
func TestLoginGuard_SuccessForgivesFailures(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
g := newGuard(middleware.LoginFailureMaxKeysConst, 1)
|
||||
|
||||
for range middleware.LoginRateLimitConst {
|
||||
g.FailForTest(guardClient, guardUser)
|
||||
}
|
||||
|
||||
g.SucceedForTest(guardClient, guardUser)
|
||||
|
||||
assert.False(
|
||||
t, g.FailForTest(guardClient, guardUser),
|
||||
"a success must reset the counter, so the next mistake "+
|
||||
"starts a fresh budget",
|
||||
)
|
||||
}
|
||||
|
||||
// TestLoginGuard_FailuresAreKeyedPerUsername proves the second half
|
||||
// of the keying: one username's spent budget does not throttle
|
||||
// another's from the same client.
|
||||
func TestLoginGuard_FailuresAreKeyedPerUsername(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
g := newGuard(middleware.LoginFailureMaxKeysConst, 1)
|
||||
|
||||
for range middleware.LoginRateLimitConst {
|
||||
g.FailForTest(guardClient, guardUser)
|
||||
}
|
||||
|
||||
assert.True(t, g.FailForTest(guardClient, guardUser))
|
||||
assert.False(
|
||||
t, g.FailForTest(guardClient, "someone-else"),
|
||||
"a different submitted username must have its own budget",
|
||||
)
|
||||
}
|
||||
|
||||
// TestLoginGuard_WindowLapses covers the interval: a counter that has
|
||||
// gone quiet for the whole window starts again from zero.
|
||||
func TestLoginGuard_WindowLapses(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
g := newGuard(middleware.LoginFailureMaxKeysConst, 1)
|
||||
|
||||
var now atomic.Int64
|
||||
|
||||
now.Store(time.Now().UnixNano())
|
||||
g.SetNowForTest(func() time.Time {
|
||||
return time.Unix(0, now.Load())
|
||||
})
|
||||
|
||||
for range middleware.LoginRateLimitConst {
|
||||
g.FailForTest(guardClient, guardUser)
|
||||
}
|
||||
|
||||
assert.True(t, g.FailForTest(guardClient, guardUser))
|
||||
|
||||
now.Add(int64(guardInterval) + 1)
|
||||
|
||||
assert.False(
|
||||
t, g.FailForTest(guardClient, guardUser),
|
||||
"a lapsed window must start a fresh budget",
|
||||
)
|
||||
}
|
||||
|
||||
// TestLoginGuard_UsernameKeySetIsBounded is the memory bound. The
|
||||
// submitted username is attacker-controlled, so an attacker rotating
|
||||
// usernames must not be able to grow the guard without limit: past
|
||||
// the cap, tracking falls back to a counter keyed on the client
|
||||
// address alone.
|
||||
func TestLoginGuard_UsernameKeySetIsBounded(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const (
|
||||
maxKeys = 8
|
||||
attempts = 500
|
||||
)
|
||||
|
||||
g := newGuard(maxKeys, 1)
|
||||
|
||||
for i := range attempts {
|
||||
g.FailForTest(guardClient, fmt.Sprintf("user-%d", i))
|
||||
}
|
||||
|
||||
byUser, byAddr := g.TrackedKeysForTest()
|
||||
|
||||
assert.LessOrEqual(
|
||||
t, byUser, maxKeys,
|
||||
"the per-username key set must not grow past its cap",
|
||||
)
|
||||
assert.LessOrEqual(
|
||||
t, byAddr, maxKeys,
|
||||
"the fallback key set must not grow past its cap either",
|
||||
)
|
||||
assert.Positive(
|
||||
t, byAddr,
|
||||
"past the cap, failures must fall back to the address "+
|
||||
"bucket rather than being dropped",
|
||||
)
|
||||
assert.Less(
|
||||
t, byUser+byAddr, attempts,
|
||||
"memory must not grow with the number of distinct "+
|
||||
"usernames submitted",
|
||||
)
|
||||
}
|
||||
|
||||
// TestLoginGuard_BeyondBothCapsStaysThrottled covers the hard stop.
|
||||
// When both key sets are full of live counters and the client is in
|
||||
// neither, there is nothing to count without unbounded growth, so the
|
||||
// failure is answered as throttled. That costs the operator nothing:
|
||||
// a correct password never reaches this path.
|
||||
func TestLoginGuard_BeyondBothCapsStaysThrottled(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const maxKeys = 4
|
||||
|
||||
g := newGuard(maxKeys, 1)
|
||||
|
||||
// Fill the per-username set from one client, then fill the
|
||||
// address set from distinct clients.
|
||||
for i := range maxKeys {
|
||||
g.FailForTest(guardClient, fmt.Sprintf("user-%d", i))
|
||||
}
|
||||
|
||||
for i := range maxKeys {
|
||||
g.FailForTest(fmt.Sprintf("203.0.113.%d", i), "whoever")
|
||||
}
|
||||
|
||||
assert.True(
|
||||
t, g.FailForTest("203.0.113.200", "brand-new"),
|
||||
"a client that fits in neither full key set must be "+
|
||||
"answered as throttled rather than tracked",
|
||||
)
|
||||
|
||||
byUser, byAddr := g.TrackedKeysForTest()
|
||||
assert.LessOrEqual(t, byUser, maxKeys)
|
||||
assert.LessOrEqual(t, byAddr, maxKeys)
|
||||
}
|
||||
|
||||
// TestLoginGuard_SemaphoreBoundsConcurrentVerifications is the memory
|
||||
// bound on the hashing itself. Verifying credentials before spending
|
||||
// limiter budget means an attacker can force one Argon2id hash per
|
||||
// request, and each allocates 64 MB; without this bound the fix for
|
||||
// an admin lockout would be a memory-exhaustion DoS instead.
|
||||
func TestLoginGuard_SemaphoreBoundsConcurrentVerifications(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
const (
|
||||
concurrency = 2
|
||||
workers = 12
|
||||
)
|
||||
|
||||
g := newGuard(middleware.LoginFailureMaxKeysConst, concurrency)
|
||||
|
||||
var (
|
||||
mu sync.Mutex
|
||||
inside int
|
||||
highest int
|
||||
wg sync.WaitGroup
|
||||
)
|
||||
|
||||
for range workers {
|
||||
wg.Go(func() {
|
||||
release, ok := g.AcquireForTest(context.Background())
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
|
||||
defer release()
|
||||
|
||||
mu.Lock()
|
||||
|
||||
inside++
|
||||
if inside > highest {
|
||||
highest = inside
|
||||
}
|
||||
|
||||
mu.Unlock()
|
||||
|
||||
// Hold the slot long enough that the other workers are
|
||||
// certainly contending for it.
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
|
||||
mu.Lock()
|
||||
inside--
|
||||
mu.Unlock()
|
||||
})
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
|
||||
assert.Equal(
|
||||
t, concurrency, highest,
|
||||
"no more than %d verifications may run at once", concurrency,
|
||||
)
|
||||
}
|
||||
|
||||
// TestLoginGuard_SaturatedSemaphoreRefusesRatherThanQueueing pins
|
||||
// what happens when every slot is taken for longer than the wait: the
|
||||
// request is refused, so the caller answers 503 without allocating
|
||||
// another 64 MB hash.
|
||||
func TestLoginGuard_SaturatedSemaphoreRefusesRatherThanQueueing(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
g := middleware.NewLoginGuardForTest(
|
||||
middleware.LoginRateLimitConst,
|
||||
guardInterval,
|
||||
middleware.LoginFailureMaxKeysConst,
|
||||
1,
|
||||
middleware.PasswordVerifyMaxWaitersConst,
|
||||
10*time.Millisecond,
|
||||
)
|
||||
|
||||
release, ok := g.AcquireForTest(context.Background())
|
||||
require.True(t, ok, "the first acquire must get the only slot")
|
||||
|
||||
_, ok = g.AcquireForTest(context.Background())
|
||||
assert.False(
|
||||
t, ok,
|
||||
"with the only slot held, a second request must be refused "+
|
||||
"rather than wait indefinitely",
|
||||
)
|
||||
|
||||
release()
|
||||
|
||||
release, ok = g.AcquireForTest(context.Background())
|
||||
assert.True(
|
||||
t, ok, "the slot must be reusable once released",
|
||||
)
|
||||
|
||||
release()
|
||||
}
|
||||
|
||||
// TestLoginGuard_AcquireHonoursCancellation proves a client that
|
||||
// disconnects while queued frees its place immediately instead of
|
||||
// holding it for the full wait.
|
||||
func TestLoginGuard_AcquireHonoursCancellation(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
g := newGuard(middleware.LoginFailureMaxKeysConst, 1)
|
||||
|
||||
release, ok := g.AcquireForTest(context.Background())
|
||||
require.True(t, ok)
|
||||
|
||||
defer release()
|
||||
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
|
||||
_, ok = g.AcquireForTest(ctx)
|
||||
assert.False(
|
||||
t, ok, "a cancelled request must not wait for a slot",
|
||||
)
|
||||
}
|
||||
|
||||
// TestPasswordVerifyConcurrency_MatchesMemoryBudget pins the
|
||||
// concurrency constant to the arithmetic behind it: the number of
|
||||
// slots is the hashing budget divided by what one Argon2id hash
|
||||
// actually costs.
|
||||
//
|
||||
// The per-hash figure is read out of the shipped password
|
||||
// parameters rather than copied here. A guard that asserts a literal
|
||||
// against a literal cannot see the thing it guards: raising
|
||||
// argon2Memory would leave it green while the real ceiling doubled.
|
||||
func TestPasswordVerifyConcurrency_MatchesMemoryBudget(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// Memory is the real argon2Memory, in KiB.
|
||||
perHashMB := int(database.DefaultPasswordConfig().Memory) / mib
|
||||
|
||||
require.Positive(
|
||||
t, perHashMB,
|
||||
"the Argon2id memory parameter must be readable in MB",
|
||||
)
|
||||
|
||||
// The memory this service commits to password hashing.
|
||||
const budgetMB = 128
|
||||
|
||||
assert.Equal(
|
||||
t,
|
||||
middleware.PasswordVerifyConcurrencyConst,
|
||||
budgetMB/perHashMB,
|
||||
"the verification concurrency must be the %d MB hashing "+
|
||||
"budget divided by the %d MB one Argon2id hash costs; "+
|
||||
"if the Argon2id parameters changed, the slot count "+
|
||||
"must change with them",
|
||||
budgetMB, perHashMB,
|
||||
)
|
||||
}
|
||||
|
||||
// TestLoginGuard_ShedsPastTheQueueCap pins the memory bound on
|
||||
// waiting, as distinct from the bound on hashing. A waiter arrives
|
||||
// with its form already parsed, and the retained parse plus its
|
||||
// header block cost several MB — far more than maxFormBodySize
|
||||
// suggests, since that caps only the raw body read — so an unbounded
|
||||
// queue would hold that much per waiting request for the whole wait;
|
||||
// past the cap the guard must refuse instantly rather than grow.
|
||||
func TestLoginGuard_ShedsPastTheQueueCap(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const (
|
||||
maxWaiters = 2
|
||||
|
||||
// Long enough that a queued waiter never times out on its
|
||||
// own, so anything the test observes leaving the queue left
|
||||
// because it was shed.
|
||||
neverElapses = time.Minute
|
||||
|
||||
// The probe carries its own deadline, so a guard that queues
|
||||
// the probe instead of shedding it fails on the elapsed time
|
||||
// rather than hanging until the package test timeout.
|
||||
probeWait = 200 * time.Millisecond
|
||||
|
||||
// Shedding takes no measurable time; queueing takes the whole
|
||||
// probeWait. Anything under half of it is unambiguous.
|
||||
shedFast = probeWait / 2
|
||||
)
|
||||
|
||||
g := middleware.NewLoginGuardForTest(
|
||||
middleware.LoginRateLimitConst,
|
||||
guardInterval,
|
||||
middleware.LoginFailureMaxKeysConst,
|
||||
1,
|
||||
maxWaiters,
|
||||
neverElapses,
|
||||
)
|
||||
|
||||
// Occupy the only slot, so everything after this queues.
|
||||
release, ok := g.AcquireForTest(context.Background())
|
||||
require.True(t, ok)
|
||||
|
||||
defer release()
|
||||
defer fillQueue(t, g, maxWaiters)()
|
||||
|
||||
got := probeQueueCap(g, probeWait)
|
||||
|
||||
require.NotNil(
|
||||
t, got,
|
||||
"a request arriving past the queue cap is still waiting to "+
|
||||
"be queued; it must have been shed",
|
||||
)
|
||||
assert.False(
|
||||
t, got.ok,
|
||||
"a request arriving past the queue cap must be shed",
|
||||
)
|
||||
assert.Less(
|
||||
t, got.elapsed, shedFast,
|
||||
"shedding must be immediate; waiting for a place in the "+
|
||||
"queue is the memory growth this bounds",
|
||||
)
|
||||
assert.Equal(
|
||||
t, maxWaiters, g.QueuedWaitersForTest(),
|
||||
"a shed request must not have grown the queue",
|
||||
)
|
||||
}
|
||||
|
||||
// fillQueue starts n waiters on g and returns once all of them are
|
||||
// queued for a slot. The returned function releases them and waits
|
||||
// for them to exit.
|
||||
func fillQueue(
|
||||
t *testing.T,
|
||||
g *middleware.LoginGuard,
|
||||
n int,
|
||||
) func() {
|
||||
t.Helper()
|
||||
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
|
||||
var wg sync.WaitGroup
|
||||
|
||||
for range n {
|
||||
wg.Go(func() {
|
||||
done, got := g.AcquireForTest(ctx)
|
||||
if got {
|
||||
done()
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
require.Eventually(
|
||||
t,
|
||||
func() bool { return g.QueuedWaitersForTest() == n },
|
||||
time.Second, time.Millisecond,
|
||||
"the waiters must reach the queue before the cap is tested",
|
||||
)
|
||||
|
||||
return func() {
|
||||
cancel()
|
||||
wg.Wait()
|
||||
}
|
||||
}
|
||||
|
||||
// probeResult is what the queue-cap probe reports: whether it got a
|
||||
// slot, and how long it took to find out.
|
||||
type probeResult struct {
|
||||
ok bool
|
||||
elapsed time.Duration
|
||||
}
|
||||
|
||||
// probeQueueCap acquires from another goroutine and reports the
|
||||
// result, or nil if the call was still blocked after wait.
|
||||
//
|
||||
// It runs off the test goroutine deliberately. Joining a full queue
|
||||
// is not cancellable by context — refusing to join is the property
|
||||
// under test — so a guard that fails this would otherwise hang the
|
||||
// package until the test timeout instead of failing here.
|
||||
func probeQueueCap(
|
||||
g *middleware.LoginGuard,
|
||||
wait time.Duration,
|
||||
) *probeResult {
|
||||
probed := make(chan probeResult, 1)
|
||||
|
||||
go func() {
|
||||
start := time.Now()
|
||||
|
||||
release, ok := g.AcquireForTest(context.Background())
|
||||
if ok {
|
||||
release()
|
||||
}
|
||||
|
||||
probed <- probeResult{ok: ok, elapsed: time.Since(start)}
|
||||
}()
|
||||
|
||||
select {
|
||||
case result := <-probed:
|
||||
return &result
|
||||
case <-time.After(wait):
|
||||
return nil
|
||||
}
|
||||
}
|
||||
@@ -6,10 +6,8 @@ import (
|
||||
"log/slog"
|
||||
"net"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
|
||||
basicauth "github.com/99designs/basicauth-go"
|
||||
"github.com/go-chi/chi"
|
||||
@@ -21,6 +19,7 @@ import (
|
||||
"go.uber.org/fx"
|
||||
"sneak.berlin/go/webhooker/internal/config"
|
||||
"sneak.berlin/go/webhooker/internal/globals"
|
||||
"sneak.berlin/go/webhooker/internal/logfield"
|
||||
"sneak.berlin/go/webhooker/internal/logger"
|
||||
"sneak.berlin/go/webhooker/internal/session"
|
||||
)
|
||||
@@ -43,16 +42,6 @@ const (
|
||||
// pick the size of the line it writes.
|
||||
redactedQuery = "?(redacted)"
|
||||
|
||||
// maxLogFieldBytes bounds each access log field whose value the
|
||||
// client supplies outright: the URL, the User-Agent and the
|
||||
// Referer. The budget is spent in ENCODED bytes (see
|
||||
// truncateLogField), so 512 still holds a real browser's User-Agent
|
||||
// whole — those are plain ASCII, which encodes one byte for one —
|
||||
// while a value built from characters the encoder escapes keeps a
|
||||
// shorter prefix. That is the intended trade: 500 quotation marks
|
||||
// are not a debugging asset.
|
||||
maxLogFieldBytes = 512
|
||||
|
||||
// maxLogRequestIDBytes bounds the request id, which is also
|
||||
// client-supplied: chi's RequestID middleware passes an inbound
|
||||
// X-Request-Id header through verbatim. Its generated form is an
|
||||
@@ -65,15 +54,10 @@ const (
|
||||
// is half this.
|
||||
maxLogMethodBytes = 32
|
||||
|
||||
// truncationMarker is appended to any field the access log cut, so
|
||||
// a short value and a truncated one cannot be confused. It is
|
||||
// charged on top of the budget, not inside it.
|
||||
truncationMarker = "[truncated]"
|
||||
|
||||
// MaxAccessLogLineBytes is the ceiling on one JSON access log line,
|
||||
// and the number an operator multiplies by the request rate to size
|
||||
// log storage. It is not an observation of a sample: it is the sum
|
||||
// of the budgets above, each of which truncateLogField enforces in
|
||||
// of the budgets above, each of which logfield.Truncate enforces in
|
||||
// ENCODED bytes, plus the part of the line no client can influence.
|
||||
//
|
||||
// url, useragent, referer 3*(512+11) = 1569
|
||||
@@ -90,13 +74,45 @@ const (
|
||||
// than sitting on the arithmetic.
|
||||
//
|
||||
// The tty text handler in internal/logger is covered by the same
|
||||
// figure. encodedLogFieldBytes charges every rune at least what
|
||||
// figure. logfield.EncodedBytes charges every rune at least what
|
||||
// the wider of the two handlers emits for it — including the ten
|
||||
// bytes strconv.Quote spends on a non-printable rune at or above
|
||||
// U+10000, which is four more than the JSON handler ever spends —
|
||||
// so each budget bounds the encoded field under either handler.
|
||||
// The text handler's fixed portion is 286, the smaller of the two,
|
||||
// which puts its worst case at 2037.
|
||||
//
|
||||
// It is also the ceiling on every OTHER line this service writes
|
||||
// THROUGH SLOG that carries text an UNAUTHENTICATED client
|
||||
// supplies. Those lines — the MaxBodySize rejection, the CSRF
|
||||
// rejection, the rate-limit rejection, the unauthenticated-request
|
||||
// and unknown-entrypoint DEBUG lines, and the failed-login DEBUG
|
||||
// lines — spend the same per-field budgets, and each carries
|
||||
// strictly fewer client-supplied fields than the access log does,
|
||||
// so none of them can reach a width the access log cannot. That is
|
||||
// asserted directly, per line and under both handlers, rather than
|
||||
// left to the reasoning: see logbound_test.go in this package and
|
||||
// in internal/handlers.
|
||||
//
|
||||
// What it does NOT cover, so that the figure above is not read as
|
||||
// more than it is:
|
||||
//
|
||||
// - Lines carrying an AUTHENTICATED operator's own input, which
|
||||
// are not truncated at all: the webhook name on "webhook
|
||||
// created" and the target host on "target URL blocked by SSRF
|
||||
// protection" (both internal/handlers/source_management.go),
|
||||
// and target_name in internal/delivery/engine.go and
|
||||
// target_http.go. Each is bounded only by the 1 MB form body
|
||||
// cap, so a 100 KB name writes one line of roughly 600 KB.
|
||||
// Deliberate: truncating the operator's own configuration
|
||||
// echoed back costs debuggability against no adversary.
|
||||
// - The "log" delivery target, which exists to write the whole
|
||||
// inbound event to the log. Deliberate; see
|
||||
// internal/delivery/target_log.go.
|
||||
// - GORM's default logger, which prints the interpolated SQL to
|
||||
// stdout on a record-not-found and so is unbounded on the
|
||||
// receiver and login lookups. NOT deliberate; filed as
|
||||
// https://git.eeqj.de/sneak/webhooker/issues/178.
|
||||
MaxAccessLogLineBytes = 2560
|
||||
)
|
||||
|
||||
@@ -116,6 +132,12 @@ type Middleware struct {
|
||||
log *slog.Logger
|
||||
params *MiddlewareParams
|
||||
session *session.Session
|
||||
|
||||
// loginGuard counts failed credential verifications and bounds
|
||||
// concurrent password hashing. It is built on first use so that
|
||||
// every construction path gets one; see guard().
|
||||
loginGuardOnce sync.Once
|
||||
loginGuard *loginGuard
|
||||
}
|
||||
|
||||
// New creates a Middleware from the provided fx parameters.
|
||||
@@ -167,114 +189,6 @@ func (lrw *loggingResponseWriter) WriteHeader(code int) {
|
||||
lrw.ResponseWriter.WriteHeader(code)
|
||||
}
|
||||
|
||||
// encodedLogFieldBytes is what r costs on the line once the log
|
||||
// handler has escaped it, taking the worse of the two handlers
|
||||
// internal/logger configures.
|
||||
//
|
||||
// slog's JSON handler escapes quote, backslash, newline, carriage
|
||||
// return and tab to two bytes each, and every other C0 control plus
|
||||
// LINE SEPARATOR and PARAGRAPH SEPARATOR to a six-byte \u escape; it
|
||||
// passes every other rune through as its own UTF-8. Its text handler
|
||||
// quotes with strconv.Quote, which spells a non-printable rune below
|
||||
// U+10000 as \uXXXX but one at or above U+10000 as \UXXXXXXXX — ten
|
||||
// bytes, not six. The text handler is therefore the worse of the two
|
||||
// for every non-printable rune, and by four bytes apiece for the
|
||||
// 955,086 unassigned, private-use and format code points on planes 1
|
||||
// to 16.
|
||||
//
|
||||
// Charging ten there is what makes MaxAccessLogLineBytes hold for the
|
||||
// tty handler as well: U+1000C encodes as F0 90 80 8C, every byte
|
||||
// >= 0x80, which httpguts.ValidHeaderFieldValue accepts and
|
||||
// net/textproto does not strip, so a header can be filled with them.
|
||||
//
|
||||
// Both handlers pass printable runes through as their own UTF-8, so
|
||||
// unicode.IsPrint separates the escaped cases from the plain ones for
|
||||
// either handler.
|
||||
func encodedLogFieldBytes(r rune) int {
|
||||
const (
|
||||
// A backslash and the character itself.
|
||||
shortEscapeBytes = 2
|
||||
// \uXXXX, which is also the width of \u00XX.
|
||||
escapedRuneBytes = 6
|
||||
// \UXXXXXXXX, strconv.Quote's spelling of a non-printable
|
||||
// rune outside the basic multilingual plane.
|
||||
escapedAstralRuneBytes = 10
|
||||
// The first code point strconv.Quote spells with \U.
|
||||
firstAstralRune = 0x10000
|
||||
)
|
||||
|
||||
switch {
|
||||
case r == '"' || r == '\\' || r == '\n' || r == '\r' || r == '\t':
|
||||
return shortEscapeBytes
|
||||
case !unicode.IsPrint(r) && r >= firstAstralRune:
|
||||
return escapedAstralRuneBytes
|
||||
case !unicode.IsPrint(r):
|
||||
return escapedRuneBytes
|
||||
default:
|
||||
return utf8.RuneLen(r)
|
||||
}
|
||||
}
|
||||
|
||||
// truncateLogField caps s at maxBytes of ENCODED output, marking the
|
||||
// value when it cuts.
|
||||
//
|
||||
// Budgeting raw bytes would not bound the line. Escaping only ever
|
||||
// grows a value, so a raw budget spent on characters the encoder
|
||||
// escapes buys a field several times its nominal size — and the line
|
||||
// is the thing an operator is told to multiply by their request rate.
|
||||
// Charging each rune what it will actually cost is what makes
|
||||
// MaxAccessLogLineBytes true rather than merely larger. The visible
|
||||
// consequence is that an escape-heavy value keeps a shorter prefix
|
||||
// than a plain one, which is the correct trade.
|
||||
//
|
||||
// The result is always valid UTF-8. A cut on a byte boundary can split
|
||||
// a multi-byte rune, and a header can carry bytes that were never
|
||||
// valid UTF-8 to begin with; both are dropped rather than kept, since
|
||||
// an encoder would otherwise spend six bytes replacing each one.
|
||||
func truncateLogField(s string, maxBytes int) string {
|
||||
// No rune encodes to fewer bytes than it occupies, so nothing past
|
||||
// maxBytes raw can fit the budget. Slicing first bounds the scan
|
||||
// below to the budget rather than to the size of the header the
|
||||
// client sent.
|
||||
window, cut := s, false
|
||||
if len(window) > maxBytes {
|
||||
window, cut = window[:maxBytes], true
|
||||
}
|
||||
|
||||
var (
|
||||
kept strings.Builder
|
||||
spent int
|
||||
)
|
||||
|
||||
for i := 0; i < len(window); {
|
||||
r, size := utf8.DecodeRuneInString(window[i:])
|
||||
if r == utf8.RuneError && size == 1 {
|
||||
i += size
|
||||
|
||||
continue
|
||||
}
|
||||
|
||||
cost := encodedLogFieldBytes(r)
|
||||
if spent+cost > maxBytes {
|
||||
cut = true
|
||||
|
||||
break
|
||||
}
|
||||
|
||||
spent += cost
|
||||
|
||||
kept.WriteString(window[i : i+size])
|
||||
|
||||
i += size
|
||||
}
|
||||
|
||||
if !cut {
|
||||
return kept.String()
|
||||
}
|
||||
|
||||
return kept.String() + truncationMarker
|
||||
}
|
||||
|
||||
// concreteLogURL renders the request's own URL for the access log
|
||||
// branches that keep it, with the query string replaced by a fixed
|
||||
// marker.
|
||||
@@ -368,21 +282,21 @@ func (s *Middleware) Logging() func(http.Handler) http.Handler {
|
||||
// line does not track the size of the request.
|
||||
s.log.Info("http request",
|
||||
"request_start", start,
|
||||
"method", truncateLogField(
|
||||
"method", logfield.Truncate(
|
||||
r.Method, maxLogMethodBytes,
|
||||
),
|
||||
"url", truncateLogField(
|
||||
"url", logfield.Truncate(
|
||||
accessLogURL(r, lrw.statusCode),
|
||||
maxLogFieldBytes,
|
||||
logfield.MaxBytes,
|
||||
),
|
||||
"useragent", truncateLogField(
|
||||
r.UserAgent(), maxLogFieldBytes,
|
||||
"useragent", logfield.Truncate(
|
||||
r.UserAgent(), logfield.MaxBytes,
|
||||
),
|
||||
"request_id", truncateLogField(
|
||||
"request_id", logfield.Truncate(
|
||||
requestID, maxLogRequestIDBytes,
|
||||
),
|
||||
"referer", truncateLogField(
|
||||
r.Referer(), maxLogFieldBytes,
|
||||
"referer", logfield.Truncate(
|
||||
r.Referer(), logfield.MaxBytes,
|
||||
),
|
||||
"proto", r.Proto,
|
||||
"remoteIP", ipFromHostPort(r.RemoteAddr),
|
||||
@@ -450,10 +364,21 @@ func (s *Middleware) RequireAuth() func(http.Handler) http.Handler {
|
||||
// session lands here and is sent back to the login
|
||||
// page.
|
||||
if !s.session.IsAuthenticated(sess) {
|
||||
// This is the unauthenticated branch, so both
|
||||
// fields are entirely client-chosen and neither
|
||||
// is bounded by anything the router did. DEBUG
|
||||
// is off by default, but turning it on to
|
||||
// diagnose a problem must not hand a client an
|
||||
// unbounded write into the log, so the same
|
||||
// budgets apply here as in the access log.
|
||||
s.log.Debug(
|
||||
"auth middleware: unauthenticated request",
|
||||
"path", r.URL.Path,
|
||||
"method", r.Method,
|
||||
"path", logfield.Truncate(
|
||||
r.URL.Path, logfield.MaxBytes,
|
||||
),
|
||||
"method", logfield.Truncate(
|
||||
r.Method, maxLogMethodBytes,
|
||||
),
|
||||
)
|
||||
http.Redirect(
|
||||
w, r, "/pages/login", http.StatusSeeOther,
|
||||
@@ -613,10 +538,26 @@ func (s *Middleware) MaxBodySize(
|
||||
}
|
||||
|
||||
if r.ContentLength > maxBytes {
|
||||
// This runs ahead of RequireAuth (see
|
||||
// setupUserRoutes and friends in
|
||||
// internal/server/routes.go), so an
|
||||
// unauthenticated client reaches it with a path
|
||||
// of its own choosing and its own length —
|
||||
// POST /source/<8 KB>/edit with an oversize
|
||||
// declared Content-Length costs nothing to
|
||||
// send. At WARN, on by default, that is a
|
||||
// write into the operator's log sized by the
|
||||
// attacker unless the path is capped. Same
|
||||
// budgets as the access log, so this line
|
||||
// cannot be wider than that one.
|
||||
s.log.Warn(
|
||||
"request body exceeds limit",
|
||||
"method", r.Method,
|
||||
"path", r.URL.Path,
|
||||
"method", logfield.Truncate(
|
||||
r.Method, maxLogMethodBytes,
|
||||
),
|
||||
"path", logfield.Truncate(
|
||||
r.URL.Path, logfield.MaxBytes,
|
||||
),
|
||||
"content_length", r.ContentLength,
|
||||
"limit", maxBytes,
|
||||
)
|
||||
|
||||
@@ -57,7 +57,21 @@ func testMiddlewareWithSessionClock(
|
||||
SessionIdleTimeout: idleTimeout,
|
||||
}
|
||||
|
||||
// Create a real session manager with a known key
|
||||
sessManager := newTestSessionManager(cfg, log, clock)
|
||||
|
||||
m := middleware.NewForTest(log, cfg, sessManager)
|
||||
|
||||
return m, sessManager, clock
|
||||
}
|
||||
|
||||
// newTestSessionManager builds the real session.Session the
|
||||
// middleware tests run against: an in-memory cookie store with a
|
||||
// known key, and optionally a manually advanced clock.
|
||||
func newTestSessionManager(
|
||||
cfg *config.Config,
|
||||
log *slog.Logger,
|
||||
clock *fakeClock,
|
||||
) *session.Session {
|
||||
key := make([]byte, testKeySize)
|
||||
|
||||
for i := range key {
|
||||
@@ -79,11 +93,7 @@ func testMiddlewareWithSessionClock(
|
||||
now = clock.Now
|
||||
}
|
||||
|
||||
sessManager := session.NewForTest(store, cfg, log, key, now)
|
||||
|
||||
m := middleware.NewForTest(log, cfg, sessManager)
|
||||
|
||||
return m, sessManager, clock
|
||||
return session.NewForTest(store, cfg, log, key, now)
|
||||
}
|
||||
|
||||
// fakeClock is a manually advanced clock, so session expiry can be
|
||||
|
||||
@@ -9,14 +9,19 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/go-chi/httprate"
|
||||
"sneak.berlin/go/webhooker/internal/logfield"
|
||||
)
|
||||
|
||||
const (
|
||||
// loginRateLimit is the maximum number of login attempts
|
||||
// per interval.
|
||||
// loginRateLimit is the maximum number of FAILED login attempts
|
||||
// one client may make against one submitted username per
|
||||
// interval before further failures are answered 429. Successful
|
||||
// attempts are never counted and never throttled — see
|
||||
// loginGuard.
|
||||
loginRateLimit = 5
|
||||
|
||||
// loginRateInterval is the time window for the rate limit.
|
||||
// loginRateInterval is the time window for the login failure
|
||||
// limit.
|
||||
loginRateInterval = 1 * time.Minute
|
||||
|
||||
// passwordChangeRateLimit is the maximum number of password
|
||||
@@ -216,16 +221,27 @@ func (m *Middleware) clientKey(r *http.Request) string {
|
||||
return bucketKey(peer)
|
||||
}
|
||||
|
||||
// tooManyRequests returns the 429 handler used by the login,
|
||||
// tooManyRequests returns the 429 handler used by the
|
||||
// password-change and per-entrypoint receiver limiters: it logs the
|
||||
// rejection with logMessage and answers with responseMessage.
|
||||
// httprate adds the Retry-After header (RFC 6585). The aggregate
|
||||
// receiver limiter uses floodTooManyRequests instead.
|
||||
//
|
||||
// The path is capped against the same budget as the access log's url
|
||||
// field. The per-entrypoint receiver limiter is unauthenticated and
|
||||
// its path is a client-chosen segment of client-chosen length, so at
|
||||
// WARN an uncapped path would let a sender pick the size of the line
|
||||
// it writes — the same defect the access log capping closed.
|
||||
func (m *Middleware) tooManyRequests(
|
||||
logMessage, responseMessage string,
|
||||
) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
m.log.Warn(logMessage, "path", r.URL.Path)
|
||||
m.log.Warn(
|
||||
logMessage,
|
||||
"path", logfield.Truncate(
|
||||
r.URL.Path, logfield.MaxBytes,
|
||||
),
|
||||
)
|
||||
http.Error(w, responseMessage, http.StatusTooManyRequests)
|
||||
}
|
||||
}
|
||||
@@ -255,26 +271,15 @@ func (m *Middleware) floodTooManyRequests(
|
||||
}
|
||||
}
|
||||
|
||||
// LoginRateLimit returns middleware that enforces per-IP rate
|
||||
// limiting on login attempts using go-chi/httprate. Only POST
|
||||
// requests are rate-limited; GET requests (rendering the login
|
||||
// form) pass through unaffected. When the rate limit is exceeded,
|
||||
// a 429 Too Many Requests response is returned. Clients are
|
||||
// identified by rateLimitKey.
|
||||
func (m *Middleware) LoginRateLimit() func(http.Handler) http.Handler {
|
||||
return m.postRateLimit(
|
||||
loginRateLimit,
|
||||
loginRateInterval,
|
||||
"login rate limit exceeded",
|
||||
"Too many login attempts. Please try again later.",
|
||||
)
|
||||
}
|
||||
|
||||
// PasswordChangeRateLimit returns middleware that enforces
|
||||
// per-IP rate limiting on password change attempts. The change
|
||||
// endpoint verifies the current password, so without a limit a
|
||||
// stolen session could be used to brute-force it; the limit
|
||||
// matches the login endpoint's.
|
||||
// stolen session could be used to brute-force it.
|
||||
//
|
||||
// Unlike the login POST this limit is still spent on arrival, which
|
||||
// is safe here: RequireAuth runs ahead of it, so only a request
|
||||
// already carrying a valid session can reach the bucket, and an
|
||||
// operator locked out of changing a password can still log in.
|
||||
func (m *Middleware) PasswordChangeRateLimit() func(http.Handler) http.Handler {
|
||||
return m.postRateLimit(
|
||||
passwordChangeRateLimit,
|
||||
|
||||
@@ -20,14 +20,14 @@ import (
|
||||
"sneak.berlin/go/webhooker/internal/middleware"
|
||||
)
|
||||
|
||||
func TestLoginRateLimit_AllowsGET(t *testing.T) {
|
||||
func TestPostRateLimit_AllowsGET(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
m, _ := testMiddleware(t, config.EnvironmentDev)
|
||||
|
||||
var callCount int
|
||||
|
||||
handler := m.LoginRateLimit()(http.HandlerFunc(
|
||||
handler := m.PasswordChangeRateLimit()(http.HandlerFunc(
|
||||
func(w http.ResponseWriter, _ *http.Request) {
|
||||
callCount++
|
||||
|
||||
@@ -39,7 +39,7 @@ func TestLoginRateLimit_AllowsGET(t *testing.T) {
|
||||
for i := range 20 {
|
||||
req := httptest.NewRequestWithContext(
|
||||
context.Background(),
|
||||
http.MethodGet, "/pages/login", nil,
|
||||
http.MethodGet, "/user/admin/password", nil,
|
||||
)
|
||||
req.RemoteAddr = "192.168.1.1:12345"
|
||||
|
||||
@@ -110,20 +110,6 @@ func runPostLimitTest(
|
||||
assert.Equal(t, limit, callCount)
|
||||
}
|
||||
|
||||
func TestLoginRateLimit_LimitsPOST(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
m, _ := testMiddleware(t, config.EnvironmentDev)
|
||||
|
||||
runPostLimitTest(
|
||||
t,
|
||||
m.LoginRateLimit(),
|
||||
middleware.LoginRateLimitConst,
|
||||
"/pages/login",
|
||||
"10.0.0.1:12345",
|
||||
)
|
||||
}
|
||||
|
||||
func TestPasswordChangeRateLimit_LimitsPOST(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -138,19 +124,19 @@ func TestPasswordChangeRateLimit_LimitsPOST(t *testing.T) {
|
||||
)
|
||||
}
|
||||
|
||||
func TestLoginRateLimit_IndependentPerIP(t *testing.T) {
|
||||
func TestPostRateLimit_IndependentPerIP(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
m, _ := testMiddleware(t, config.EnvironmentDev)
|
||||
|
||||
handler := m.LoginRateLimit()(http.HandlerFunc(
|
||||
handler := m.PasswordChangeRateLimit()(http.HandlerFunc(
|
||||
func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
},
|
||||
))
|
||||
|
||||
// Exhaust limit for IP1
|
||||
for range middleware.LoginRateLimitConst {
|
||||
for range middleware.PasswordChangeRateLimitConst {
|
||||
req := httptest.NewRequestWithContext(
|
||||
context.Background(),
|
||||
http.MethodPost, "/pages/login", nil,
|
||||
@@ -367,7 +353,14 @@ func TestReceiverRateLimit_CountsEveryMethod(t *testing.T) {
|
||||
}
|
||||
|
||||
const (
|
||||
loginPath = "/pages/login"
|
||||
// limitedPath is the endpoint these tests drive the shared POST
|
||||
// rate limiter through. It is the password-change path: since
|
||||
// the login POST verifies credentials before spending any
|
||||
// budget, the password-change limiter is the only pre-emptive
|
||||
// POST limiter left, and it is what pins the shared key
|
||||
// function's behaviour here.
|
||||
limitedPath = "/user/admin/password"
|
||||
|
||||
headerXFF = "X-Forwarded-For"
|
||||
headerReal = "X-Real-IP"
|
||||
headerTrue = "True-Client-IP"
|
||||
@@ -415,10 +408,10 @@ func assertSharedBucket(
|
||||
m := rateLimitMiddleware(
|
||||
t, &config.Config{TrustedProxies: proxies},
|
||||
)
|
||||
handler := m.LoginRateLimit()(okHandler())
|
||||
handler := m.PasswordChangeRateLimit()(okHandler())
|
||||
|
||||
for i := range middleware.LoginRateLimitConst {
|
||||
w := postWithHeaders(handler, peer, loginPath, headers(i))
|
||||
for i := range middleware.PasswordChangeRateLimitConst {
|
||||
w := postWithHeaders(handler, peer, limitedPath, headers(i))
|
||||
assert.Equal(
|
||||
t, http.StatusOK, w.Code,
|
||||
"request %d should pass", i,
|
||||
@@ -426,8 +419,8 @@ func assertSharedBucket(
|
||||
}
|
||||
|
||||
w := postWithHeaders(
|
||||
handler, peer, loginPath,
|
||||
headers(middleware.LoginRateLimitConst),
|
||||
handler, peer, limitedPath,
|
||||
headers(middleware.PasswordChangeRateLimitConst),
|
||||
)
|
||||
assert.Equal(t, http.StatusTooManyRequests, w.Code, msg)
|
||||
}
|
||||
@@ -549,24 +542,24 @@ func TestRateLimitKey_ForwardedHonouredFromTrustedPeer(
|
||||
m := rateLimitMiddleware(t, &config.Config{
|
||||
TrustedProxies: trustedProxies(trustedProxyCIDR),
|
||||
})
|
||||
handler := m.LoginRateLimit()(okHandler())
|
||||
handler := m.PasswordChangeRateLimit()(okHandler())
|
||||
|
||||
const peer = trustedPeer
|
||||
|
||||
first := map[string]string{headerXFF: clientIPv4}
|
||||
|
||||
for range middleware.LoginRateLimitConst {
|
||||
postWithHeaders(handler, peer, loginPath, first)
|
||||
for range middleware.PasswordChangeRateLimitConst {
|
||||
postWithHeaders(handler, peer, limitedPath, first)
|
||||
}
|
||||
|
||||
w := postWithHeaders(handler, peer, loginPath, first)
|
||||
w := postWithHeaders(handler, peer, limitedPath, first)
|
||||
assert.Equal(
|
||||
t, http.StatusTooManyRequests, w.Code,
|
||||
"the forwarded client's own bucket must fill up",
|
||||
)
|
||||
|
||||
w = postWithHeaders(
|
||||
handler, peer, loginPath,
|
||||
handler, peer, limitedPath,
|
||||
map[string]string{headerXFF: clientIPv4Alt},
|
||||
)
|
||||
assert.Equal(
|
||||
@@ -662,7 +655,7 @@ func TestRateLimitKey_LongChainAllocationIsBounded(t *testing.T) {
|
||||
})
|
||||
|
||||
req := httptest.NewRequestWithContext(
|
||||
context.Background(), http.MethodPost, loginPath, nil,
|
||||
context.Background(), http.MethodPost, limitedPath, nil,
|
||||
)
|
||||
req.RemoteAddr = trustedPeer
|
||||
req.Header.Set(
|
||||
@@ -869,7 +862,7 @@ func clientKeyFor(
|
||||
t.Helper()
|
||||
|
||||
req := httptest.NewRequestWithContext(
|
||||
context.Background(), http.MethodPost, loginPath, nil,
|
||||
context.Background(), http.MethodPost, limitedPath, nil,
|
||||
)
|
||||
req.RemoteAddr = remoteAddr
|
||||
|
||||
@@ -1019,23 +1012,23 @@ func TestRateLimitKey_UnparseablePeerKeepsDistinctBuckets(
|
||||
)
|
||||
}
|
||||
|
||||
// TestLoginRateLimit_IPv6SharesBucketWithinSlash64 is the behavioural
|
||||
// TestPostRateLimit_IPv6SharesBucketWithinSlash64 is the behavioural
|
||||
// half, and the regression test for the bypass itself: a client that
|
||||
// rotates source addresses inside its own routed /64 must stay in one
|
||||
// bucket. Reverting the masking makes this test fail, because each
|
||||
// rotated address would mint a fresh bucket and nothing would be
|
||||
// rejected.
|
||||
func TestLoginRateLimit_IPv6SharesBucketWithinSlash64(t *testing.T) {
|
||||
func TestPostRateLimit_IPv6SharesBucketWithinSlash64(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
m := rateLimitMiddleware(t, &config.Config{})
|
||||
handler := m.LoginRateLimit()(okHandler())
|
||||
handler := m.PasswordChangeRateLimit()(okHandler())
|
||||
|
||||
for i := range middleware.LoginRateLimitConst {
|
||||
for i := range middleware.PasswordChangeRateLimitConst {
|
||||
w := postWithHeaders(
|
||||
handler,
|
||||
fmt.Sprintf("[2001:db8:1:2::%d]:44444", i+1),
|
||||
loginPath, nil,
|
||||
limitedPath, nil,
|
||||
)
|
||||
assert.Equal(
|
||||
t, http.StatusOK, w.Code, "request %d should pass", i,
|
||||
@@ -1043,7 +1036,7 @@ func TestLoginRateLimit_IPv6SharesBucketWithinSlash64(t *testing.T) {
|
||||
}
|
||||
|
||||
w := postWithHeaders(
|
||||
handler, "[2001:db8:1:2::ffff]:44444", loginPath, nil,
|
||||
handler, "[2001:db8:1:2::ffff]:44444", limitedPath, nil,
|
||||
)
|
||||
assert.Equal(
|
||||
t, http.StatusTooManyRequests, w.Code,
|
||||
@@ -1052,23 +1045,23 @@ func TestLoginRateLimit_IPv6SharesBucketWithinSlash64(t *testing.T) {
|
||||
)
|
||||
}
|
||||
|
||||
// TestLoginRateLimit_IPv6IndependentAcrossSlash64 is the other side
|
||||
// TestPostRateLimit_IPv6IndependentAcrossSlash64 is the other side
|
||||
// of the trade: bucketing by /64 must not merge separate allocations,
|
||||
// so a client in a different /64 keeps its own limit.
|
||||
func TestLoginRateLimit_IPv6IndependentAcrossSlash64(t *testing.T) {
|
||||
func TestPostRateLimit_IPv6IndependentAcrossSlash64(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
m := rateLimitMiddleware(t, &config.Config{})
|
||||
handler := m.LoginRateLimit()(okHandler())
|
||||
handler := m.PasswordChangeRateLimit()(okHandler())
|
||||
|
||||
for range middleware.LoginRateLimitConst + 1 {
|
||||
for range middleware.PasswordChangeRateLimitConst + 1 {
|
||||
postWithHeaders(
|
||||
handler, "[2001:db8:1:2::1]:44444", loginPath, nil,
|
||||
handler, "[2001:db8:1:2::1]:44444", limitedPath, nil,
|
||||
)
|
||||
}
|
||||
|
||||
w := postWithHeaders(
|
||||
handler, "[2001:db8:1:3::1]:44444", loginPath, nil,
|
||||
handler, "[2001:db8:1:3::1]:44444", limitedPath, nil,
|
||||
)
|
||||
assert.Equal(
|
||||
t, http.StatusOK, w.Code,
|
||||
@@ -1076,23 +1069,23 @@ func TestLoginRateLimit_IPv6IndependentAcrossSlash64(t *testing.T) {
|
||||
)
|
||||
}
|
||||
|
||||
// TestLoginRateLimit_IPv4IndependentPerAddress guards against the
|
||||
// TestPostRateLimit_IPv4IndependentPerAddress guards against the
|
||||
// masking leaking into IPv4: two addresses one apart must still hold
|
||||
// separate buckets.
|
||||
func TestLoginRateLimit_IPv4IndependentPerAddress(t *testing.T) {
|
||||
func TestPostRateLimit_IPv4IndependentPerAddress(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
m := rateLimitMiddleware(t, &config.Config{})
|
||||
handler := m.LoginRateLimit()(okHandler())
|
||||
handler := m.PasswordChangeRateLimit()(okHandler())
|
||||
|
||||
for range middleware.LoginRateLimitConst + 1 {
|
||||
for range middleware.PasswordChangeRateLimitConst + 1 {
|
||||
postWithHeaders(
|
||||
handler, clientIPv4+":44444", loginPath, nil,
|
||||
handler, clientIPv4+":44444", limitedPath, nil,
|
||||
)
|
||||
}
|
||||
|
||||
w := postWithHeaders(
|
||||
handler, clientIPv4Alt+":44444", loginPath, nil,
|
||||
handler, clientIPv4Alt+":44444", limitedPath, nil,
|
||||
)
|
||||
assert.Equal(
|
||||
t, http.StatusOK, w.Code,
|
||||
@@ -1112,7 +1105,7 @@ func forwardedKeyFor(
|
||||
t.Helper()
|
||||
|
||||
req := httptest.NewRequestWithContext(
|
||||
context.Background(), http.MethodPost, loginPath, nil,
|
||||
context.Background(), http.MethodPost, limitedPath, nil,
|
||||
)
|
||||
req.RemoteAddr = trustedPeer
|
||||
req.Header.Set(headerXFF, forwarded)
|
||||
@@ -1177,11 +1170,77 @@ func TestRateLimitKey_ForwardedIPv6BucketsByPrefix(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestLoginRateLimit_ForwardedIPv6SharesBucketWithinSlash64 is the
|
||||
// TestRateLimitKey_TrustedPeerUnusableForwardedMasksPeer covers the
|
||||
// third bucketKey call site: the peer IS a trusted proxy, but the
|
||||
// forwarded chain cannot name a client, so the key falls back to the
|
||||
// peer address — and that fallback owes the same /64 masking every
|
||||
// other key gets.
|
||||
//
|
||||
// Every existing test of this fallback uses an IPv4 proxy, where
|
||||
// bucketKey is the identity function, so replacing the call with
|
||||
// peer.String() leaves the whole suite green. Only operator-listed
|
||||
// addresses reach this line and the fallback is fail-closed, so this
|
||||
// pins behaviour rather than fixing a defect.
|
||||
func TestRateLimitKey_TrustedPeerUnusableForwardedMasksPeer(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
const (
|
||||
proxyCIDR = "2001:db8:ffff::/48"
|
||||
proxyPeer = "[2001:db8:ffff:1::5]:44444"
|
||||
wantKey = "2001:db8:ffff:1::/64"
|
||||
)
|
||||
|
||||
m := rateLimitMiddleware(t, &config.Config{
|
||||
TrustedProxies: trustedProxies(proxyCIDR),
|
||||
})
|
||||
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
forwarded string
|
||||
about string
|
||||
}{{
|
||||
name: "absent",
|
||||
about: "no X-Forwarded-For at all falls back to the peer",
|
||||
}, {
|
||||
name: "unreadable-hop",
|
||||
forwarded: "unknown",
|
||||
about: "a hop that is not a bare address ends the walk " +
|
||||
"and falls back to the peer",
|
||||
}, {
|
||||
name: "all-hops-trusted",
|
||||
forwarded: "2001:db8:ffff:2::9",
|
||||
about: "a chain naming only trusted proxies names no " +
|
||||
"client, so the peer is used",
|
||||
}} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
req := httptest.NewRequestWithContext(
|
||||
context.Background(),
|
||||
http.MethodPost, limitedPath, nil,
|
||||
)
|
||||
req.RemoteAddr = proxyPeer
|
||||
|
||||
if tc.forwarded != "" {
|
||||
req.Header.Set(headerXFF, tc.forwarded)
|
||||
}
|
||||
|
||||
assert.Equal(
|
||||
t, wantKey,
|
||||
middleware.ClientKeyForTest(m, req),
|
||||
"%s, masked to its /64", tc.about,
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestPostRateLimit_ForwardedIPv6SharesBucketWithinSlash64 is the
|
||||
// behavioural half on the production path: behind a trusted proxy, a
|
||||
// client rotating source addresses inside its own routed /64 must
|
||||
// stay in one bucket.
|
||||
func TestLoginRateLimit_ForwardedIPv6SharesBucketWithinSlash64(
|
||||
func TestPostRateLimit_ForwardedIPv6SharesBucketWithinSlash64(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
@@ -1198,10 +1257,10 @@ func TestLoginRateLimit_ForwardedIPv6SharesBucketWithinSlash64(
|
||||
)
|
||||
}
|
||||
|
||||
// TestLoginRateLimit_ForwardedIPv6IndependentAcrossSlash64 is the
|
||||
// TestPostRateLimit_ForwardedIPv6IndependentAcrossSlash64 is the
|
||||
// other side of that trade on the same path: bucketing by /64 must
|
||||
// not merge two allocations reaching the proxy.
|
||||
func TestLoginRateLimit_ForwardedIPv6IndependentAcrossSlash64(
|
||||
func TestPostRateLimit_ForwardedIPv6IndependentAcrossSlash64(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
@@ -1209,15 +1268,15 @@ func TestLoginRateLimit_ForwardedIPv6IndependentAcrossSlash64(
|
||||
m := rateLimitMiddleware(t, &config.Config{
|
||||
TrustedProxies: trustedProxies(trustedProxyCIDR),
|
||||
})
|
||||
handler := m.LoginRateLimit()(okHandler())
|
||||
handler := m.PasswordChangeRateLimit()(okHandler())
|
||||
|
||||
spent := map[string]string{headerXFF: clientIPv6}
|
||||
for range middleware.LoginRateLimitConst + 1 {
|
||||
postWithHeaders(handler, trustedPeer, loginPath, spent)
|
||||
for range middleware.PasswordChangeRateLimitConst + 1 {
|
||||
postWithHeaders(handler, trustedPeer, limitedPath, spent)
|
||||
}
|
||||
|
||||
w := postWithHeaders(
|
||||
handler, trustedPeer, loginPath,
|
||||
handler, trustedPeer, limitedPath,
|
||||
map[string]string{headerXFF: clientIPv6Other},
|
||||
)
|
||||
assert.Equal(
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"log/slog"
|
||||
"net/http"
|
||||
|
||||
"github.com/getsentry/sentry-go"
|
||||
"sneak.berlin/go/webhooker/internal/config"
|
||||
"sneak.berlin/go/webhooker/internal/handlers"
|
||||
"sneak.berlin/go/webhooker/internal/middleware"
|
||||
@@ -13,6 +14,25 @@ import (
|
||||
// build requests that sit exactly at, below, and above it.
|
||||
const MaxFormBodySizeForTest = maxFormBodySize
|
||||
|
||||
// ScrubSentryRequestForTest exposes the BeforeSend hook that
|
||||
// enableSentry installs, so a test can assert on what it leaves in an
|
||||
// event without standing up a Sentry client.
|
||||
func ScrubSentryRequestForTest(
|
||||
event *sentry.Event,
|
||||
hint *sentry.EventHint,
|
||||
) *sentry.Event {
|
||||
return scrubSentryRequest(event, hint)
|
||||
}
|
||||
|
||||
// SentryClientOptionsForTest exposes the exact options enableSentry
|
||||
// initialises the SDK with, so a test can capture events through the
|
||||
// production hook wiring rather than a hand-built equivalent.
|
||||
func SentryClientOptionsForTest(
|
||||
dsn, release string,
|
||||
) sentry.ClientOptions {
|
||||
return sentryClientOptions(dsn, release)
|
||||
}
|
||||
|
||||
// NewRouterForTest builds the real route tree via SetupRoutes with
|
||||
// the supplied middleware and handlers, bypassing the fx lifecycle
|
||||
// and the HTTP listener. Tests use it so that route-group middleware
|
||||
|
||||
@@ -14,6 +14,28 @@ import (
|
||||
// maxFormBodySize is the maximum allowed request body size (in
|
||||
// bytes) for form POST endpoints. 1 MB is generous for any form
|
||||
// submission while preventing abuse from oversized payloads.
|
||||
//
|
||||
// Every route group below installs MaxBodySize(maxFormBodySize) as
|
||||
// its FIRST middleware, ahead of both CSRF and RequireAuth. Both
|
||||
// orderings are deliberate.
|
||||
//
|
||||
// Ahead of CSRF because gorilla/csrf parses the form. The cap has to
|
||||
// be installed before anything reads the body, or the parse runs
|
||||
// under net/http's 10 MB default instead of this one.
|
||||
//
|
||||
// Ahead of RequireAuth because an oversize body should be refused
|
||||
// before the request buys a cookie decrypt, a session load and the
|
||||
// database read behind it. Rejecting first is the cheaper failure,
|
||||
// and it is the ordering that keeps an unauthenticated flood from
|
||||
// choosing how much session work the process does.
|
||||
//
|
||||
// What that ordering costs: the 413 branch is reachable
|
||||
// unauthenticated, at a URL of the client's choosing and of the
|
||||
// client's chosen length. So is the CSRF rejection, which sits in
|
||||
// front of RequireAuth for the same reason. Both log that path, so
|
||||
// both cap it — see the log calls in Middleware.MaxBodySize and
|
||||
// Middleware.CSRF, which spend the same per-field budget as the
|
||||
// access log.
|
||||
const maxFormBodySize int64 = 1 * 1024 * 1024 // 1 MB
|
||||
|
||||
// requestTimeout is the maximum time allowed for a single HTTP
|
||||
@@ -90,17 +112,20 @@ func (s *Server) setupRoutes() {
|
||||
|
||||
func (s *Server) setupPageRoutes() {
|
||||
s.router.Route("/pages", func(r chi.Router) {
|
||||
// MaxBodySize must precede CSRF: gorilla/csrf parses the
|
||||
// form, so the cap has to be installed before it runs.
|
||||
// MaxBodySize precedes CSRF and RequireAuth deliberately;
|
||||
// see maxFormBodySize for why, and for what it costs.
|
||||
r.Use(s.mw.MaxBodySize(maxFormBodySize))
|
||||
r.Use(s.mw.CSRF())
|
||||
r.Use(s.mw.NoCache())
|
||||
|
||||
r.Group(func(r chi.Router) {
|
||||
r.Use(s.mw.LoginRateLimit())
|
||||
// The login POST carries no pre-emptive rate limiter. Behind
|
||||
// the reverse proxy production requires, with TRUSTED_PROXIES
|
||||
// unset, every client shares one bucket, so a limiter spent
|
||||
// on arrival lets any stranger deny the operator the only
|
||||
// administrative path. The handler verifies credentials first
|
||||
// and charges only failures; see Handlers.authenticateUser.
|
||||
r.Get("/login", s.h.HandleLoginPage())
|
||||
r.Post("/login", s.h.HandleLoginSubmit())
|
||||
})
|
||||
|
||||
r.Post("/logout", s.h.HandleLogout())
|
||||
})
|
||||
@@ -108,8 +133,8 @@ func (s *Server) setupPageRoutes() {
|
||||
|
||||
func (s *Server) setupUserRoutes() {
|
||||
s.router.Route("/user/{username}", func(r chi.Router) {
|
||||
// MaxBodySize must precede CSRF: gorilla/csrf parses the
|
||||
// form, so the cap has to be installed before it runs.
|
||||
// MaxBodySize precedes CSRF and RequireAuth deliberately;
|
||||
// see maxFormBodySize for why, and for what it costs.
|
||||
r.Use(s.mw.MaxBodySize(maxFormBodySize))
|
||||
r.Use(s.mw.CSRF())
|
||||
r.Use(s.mw.NoCache())
|
||||
@@ -123,8 +148,8 @@ func (s *Server) setupUserRoutes() {
|
||||
|
||||
func (s *Server) setupSourceRoutes() {
|
||||
s.router.Route("/sources", func(r chi.Router) {
|
||||
// MaxBodySize must precede CSRF: gorilla/csrf parses the
|
||||
// form, so the cap has to be installed before it runs.
|
||||
// MaxBodySize precedes CSRF and RequireAuth deliberately;
|
||||
// see maxFormBodySize for why, and for what it costs.
|
||||
r.Use(s.mw.MaxBodySize(maxFormBodySize))
|
||||
r.Use(s.mw.CSRF())
|
||||
r.Use(s.mw.NoCache())
|
||||
@@ -135,8 +160,8 @@ func (s *Server) setupSourceRoutes() {
|
||||
})
|
||||
|
||||
s.router.Route("/source/{sourceID}", func(r chi.Router) {
|
||||
// MaxBodySize must precede CSRF: gorilla/csrf parses the
|
||||
// form, so the cap has to be installed before it runs.
|
||||
// MaxBodySize precedes CSRF and RequireAuth deliberately;
|
||||
// see maxFormBodySize for why, and for what it costs.
|
||||
r.Use(s.mw.MaxBodySize(maxFormBodySize))
|
||||
r.Use(s.mw.CSRF())
|
||||
r.Use(s.mw.NoCache())
|
||||
|
||||
@@ -420,6 +420,78 @@ func TestPagesLogin_UnderLimit_ValidToken_ReachesHandler(
|
||||
)
|
||||
}
|
||||
|
||||
// TestPagesLogin_CorrectPasswordSurvivesASpentBudget pins the
|
||||
// routing half of the fix, which every other login test misses by
|
||||
// driving the handler directly: no pre-emptive limiter sits in front
|
||||
// of POST /pages/login on the real route tree.
|
||||
//
|
||||
// A limiter registered there would answer the last request 429
|
||||
// however correct its password is, because the wrong passwords
|
||||
// before it have already spent the bucket — which is the lockout
|
||||
// this endpoint exists to not have. CSRF and the body cap still run,
|
||||
// since every request here carries a harvested token.
|
||||
func TestPagesLogin_CorrectPasswordSurvivesASpentBudget(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
const (
|
||||
username = "operator"
|
||||
password = "correct-horse-battery-staple"
|
||||
)
|
||||
|
||||
env := newTestEnv(t)
|
||||
env.seedUser(t, username, password)
|
||||
|
||||
submit := func(t *testing.T, pw string) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
|
||||
token, cookies := env.csrfFrom(t, "/pages/login", nil)
|
||||
|
||||
form := url.Values{}
|
||||
form.Set("csrf_token", token)
|
||||
form.Set("username", username)
|
||||
form.Set("password", pw)
|
||||
|
||||
return env.post("/pages/login", form, cookies)
|
||||
}
|
||||
|
||||
// Spend the failure budget against this username. The exact
|
||||
// limit belongs to the middleware; this waits for the throttle
|
||||
// to appear rather than restating it, under a ceiling well
|
||||
// above it so a broken limiter fails the test instead of
|
||||
// looping.
|
||||
const maxAttempts = 20
|
||||
|
||||
spent := false
|
||||
|
||||
for range maxAttempts {
|
||||
code := submit(t, "wrong").Code
|
||||
if code == http.StatusTooManyRequests {
|
||||
spent = true
|
||||
|
||||
break
|
||||
}
|
||||
|
||||
require.Equal(
|
||||
t, http.StatusUnauthorized, code,
|
||||
"a wrong password must be rejected, not accepted",
|
||||
)
|
||||
}
|
||||
|
||||
require.True(
|
||||
t, spent,
|
||||
"repeated wrong passwords must eventually be throttled",
|
||||
)
|
||||
|
||||
assert.Equal(
|
||||
t, http.StatusSeeOther, submit(t, password).Code,
|
||||
"a correct password must be accepted on the routed "+
|
||||
"endpoint even with the failure budget spent: the "+
|
||||
"operator has no second administrative path",
|
||||
)
|
||||
}
|
||||
|
||||
// --- /user/{username} group ---
|
||||
|
||||
// TestPasswordChange_OversizeBody_RejectedAndPasswordUnchanged
|
||||
|
||||
117
internal/server/sentry.go
Normal file
117
internal/server/sentry.go
Normal file
@@ -0,0 +1,117 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
"github.com/getsentry/sentry-go"
|
||||
)
|
||||
|
||||
// sentryRedacted stands in for a withheld field on every event shipped
|
||||
// to Sentry. It is a marker rather than an empty string so a reader
|
||||
// can tell a suppressed value from an absent one.
|
||||
const sentryRedacted = "(redacted)"
|
||||
|
||||
// sentryClientOptions builds the options the SDK is initialised with.
|
||||
// It is its own function so a test can stand up a client wired exactly
|
||||
// as production is, with only the transport swapped.
|
||||
func sentryClientOptions(dsn, release string) sentry.ClientOptions {
|
||||
return sentry.ClientOptions{
|
||||
Dsn: dsn,
|
||||
Release: release,
|
||||
// Both hooks, because the SDK runs one for error events
|
||||
// and the other for transactions.
|
||||
BeforeSend: scrubSentryRequest,
|
||||
BeforeSendTransaction: scrubSentryRequest,
|
||||
}
|
||||
}
|
||||
|
||||
// scrubSentryRequest strips client-supplied content from an event's
|
||||
// request context before it leaves the process.
|
||||
//
|
||||
// sentryhttp attaches the whole *http.Request to the scope
|
||||
// (sentryhttp.go:113), and Scope.ApplyToEvent fills the event's
|
||||
// Request from it inside prepareEvent, which runs before this hook.
|
||||
// Two of the fields it fills are copied with no SendDefaultPII guard:
|
||||
//
|
||||
// - QueryString, verbatim from r.URL.RawQuery.
|
||||
// - Data, the first 10 KiB of the request body, teed off r.Body by
|
||||
// SetRequest and filled precisely because the handlers call
|
||||
// ParseForm.
|
||||
//
|
||||
// Since every form field in this service is read with PostFormValue,
|
||||
// the body is the only place a credential is submitted: a target's
|
||||
// destination URL, whose path segments are the bearer token, plus the
|
||||
// login password and both password-change fields. None of that may
|
||||
// reach a third-party service.
|
||||
//
|
||||
// This hook is a floor, not a default: the fields it clears stay
|
||||
// cleared even if SendDefaultPII is ever turned on.
|
||||
func scrubSentryRequest(
|
||||
event *sentry.Event,
|
||||
_ *sentry.EventHint,
|
||||
) *sentry.Event {
|
||||
if event == nil || event.Request == nil {
|
||||
return event
|
||||
}
|
||||
|
||||
req := event.Request
|
||||
|
||||
if req.QueryString != "" {
|
||||
req.QueryString = sentryRedacted
|
||||
}
|
||||
|
||||
if req.Data != "" {
|
||||
req.Data = sentryRedacted
|
||||
}
|
||||
|
||||
req.Cookies = ""
|
||||
req.Env = nil
|
||||
req.Headers = keptSentryHeaders(req.Headers)
|
||||
|
||||
return event
|
||||
}
|
||||
|
||||
// keptSentryHeaders returns the subset of headers an event may carry
|
||||
// off-host. Dropping by allowlist rather than by blocklist is what
|
||||
// makes an unrecognised header safe: the SDK's own filter removes four
|
||||
// names and passes everything else, so X-Csrf-Token — which
|
||||
// gorilla/csrf accepts in place of the form field — and the shared
|
||||
// secrets senders put on the receiver route (X-Gitlab-Token and the
|
||||
// per-provider signature headers) would otherwise ship verbatim.
|
||||
func keptSentryHeaders(headers map[string]string) map[string]string {
|
||||
if len(headers) == 0 {
|
||||
return headers
|
||||
}
|
||||
|
||||
kept := make(map[string]string, len(headers))
|
||||
|
||||
for name, value := range headers {
|
||||
if sentryKeepsHeader(name) {
|
||||
kept[name] = value
|
||||
}
|
||||
}
|
||||
|
||||
return kept
|
||||
}
|
||||
|
||||
// sentryKeepsHeader reports whether a request header is routing or
|
||||
// content metadata rather than client-chosen payload. Referer is kept
|
||||
// on the reasoning that it is browser-set, that this service emits
|
||||
// only ?page= in its own links, and that Referrer-Policy is set to
|
||||
// strict-origin-when-cross-origin. X-Request-Id ties the event to the
|
||||
// local access log line, which holds the rest of the detail.
|
||||
func sentryKeepsHeader(name string) bool {
|
||||
switch http.CanonicalHeaderKey(name) {
|
||||
case "Accept",
|
||||
"Content-Length",
|
||||
"Content-Type",
|
||||
"Host",
|
||||
"Origin",
|
||||
"Referer",
|
||||
"User-Agent",
|
||||
"X-Request-Id":
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
227
internal/server/sentry_test.go
Normal file
227
internal/server/sentry_test.go
Normal file
@@ -0,0 +1,227 @@
|
||||
package server_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/getsentry/sentry-go"
|
||||
sentryhttp "github.com/getsentry/sentry-go/http"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"sneak.berlin/go/webhooker/internal/server"
|
||||
)
|
||||
|
||||
// The three markers below are the credentials a captured event could
|
||||
// carry off-host, one per field of sentry.Request that the SDK fills
|
||||
// from the request without a SendDefaultPII guard.
|
||||
const (
|
||||
// sentryBodyMarker is submitted as a form value. Since every
|
||||
// handler reads its fields with PostFormValue, the body is the
|
||||
// only place a password or a target URL is ever supplied.
|
||||
sentryBodyMarker = "QQSENTRYBODYMARKERQQ"
|
||||
|
||||
// sentryQueryMarker rides the request line.
|
||||
sentryQueryMarker = "T00000000/B00000000/QQSENTRYQUERYMARKERQQ"
|
||||
|
||||
// sentryHeaderMarker rides X-Csrf-Token, which gorilla/csrf
|
||||
// accepts in place of the form field.
|
||||
sentryHeaderMarker = "QQSENTRYHEADERMARKERQQ"
|
||||
)
|
||||
|
||||
// sentryKeptUserAgent is a non-secret header value planted so the
|
||||
// assertions below cannot pass by the event carrying no headers at
|
||||
// all.
|
||||
const sentryKeptUserAgent = "webhooker-test-agent"
|
||||
|
||||
// captureTransport records events instead of shipping them, so a test
|
||||
// sees exactly the payload the SDK would have put on the wire.
|
||||
type captureTransport struct {
|
||||
mu sync.Mutex
|
||||
events []*sentry.Event
|
||||
}
|
||||
|
||||
func (c *captureTransport) Configure(sentry.ClientOptions) {}
|
||||
|
||||
func (c *captureTransport) Flush(time.Duration) bool { return true }
|
||||
|
||||
func (c *captureTransport) SendEvent(event *sentry.Event) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
|
||||
c.events = append(c.events, event)
|
||||
}
|
||||
|
||||
// captureThroughSentryHTTP panics inside a form handler wrapped in the
|
||||
// real sentryhttp middleware and returns the event the SDK produced.
|
||||
//
|
||||
// This is the only construction path on which Request.Data appears:
|
||||
// sentryhttp calls Scope.SetRequest, which tees r.Body into a 10 KiB
|
||||
// buffer, ParseForm drains the tee, and Scope.ApplyToEvent copies the
|
||||
// buffer into the event inside prepareEvent — before BeforeSend runs.
|
||||
// A hand-built sentry.NewRequest never reads the body and so cannot
|
||||
// regress-test any of it.
|
||||
//
|
||||
// scrub selects whether the production BeforeSend hooks are installed,
|
||||
// so the same path shows both what the SDK collects and what survives.
|
||||
func captureThroughSentryHTTP(t *testing.T, scrub bool) *sentry.Event {
|
||||
t.Helper()
|
||||
|
||||
transport := &captureTransport{}
|
||||
|
||||
opts := server.SentryClientOptionsForTest(
|
||||
"https://public@sentry.invalid/1", "webhooker-test",
|
||||
)
|
||||
opts.Transport = transport
|
||||
|
||||
if !scrub {
|
||||
opts.BeforeSend = nil
|
||||
opts.BeforeSendTransaction = nil
|
||||
}
|
||||
|
||||
client, err := sentry.NewClient(opts)
|
||||
require.NoError(t, err)
|
||||
|
||||
handler := sentryhttp.New(sentryhttp.Options{}).Handle(
|
||||
http.HandlerFunc(func(_ http.ResponseWriter, r *http.Request) {
|
||||
// This call is what drains the tee and fills the
|
||||
// buffer. Its success is asserted by the unscrubbed
|
||||
// case below, which sees the body in the event.
|
||||
_ = r.ParseForm()
|
||||
|
||||
panic("boom")
|
||||
}),
|
||||
)
|
||||
|
||||
handler.ServeHTTP(
|
||||
httptest.NewRecorder(),
|
||||
sentryLoginRequest(client),
|
||||
)
|
||||
|
||||
require.Len(t, transport.events, 1)
|
||||
|
||||
return transport.events[0]
|
||||
}
|
||||
|
||||
// sentryLoginRequest builds the password POST the capture above drives,
|
||||
// with a credential planted in the body, the query and a header.
|
||||
func sentryLoginRequest(client *sentry.Client) *http.Request {
|
||||
form := url.Values{}
|
||||
form.Set("username", "admin")
|
||||
form.Set("password", sentryBodyMarker)
|
||||
|
||||
req := httptest.NewRequestWithContext(
|
||||
sentry.SetHubOnContext(
|
||||
context.Background(),
|
||||
sentry.NewHub(client, sentry.NewScope()),
|
||||
),
|
||||
http.MethodPost,
|
||||
"/pages/login?url=https://hooks.slack.com/services/"+
|
||||
sentryQueryMarker,
|
||||
strings.NewReader(form.Encode()),
|
||||
)
|
||||
|
||||
req.Header.Set(
|
||||
"Content-Type", "application/x-www-form-urlencoded",
|
||||
)
|
||||
req.Header.Set("X-Csrf-Token", sentryHeaderMarker)
|
||||
req.Header.Set("User-Agent", sentryKeptUserAgent)
|
||||
|
||||
return req
|
||||
}
|
||||
|
||||
// marshalEvent encodes an event the way the transport does.
|
||||
func marshalEvent(t *testing.T, event *sentry.Event) string {
|
||||
t.Helper()
|
||||
|
||||
encoded, err := json.Marshal(event)
|
||||
require.NoError(t, err)
|
||||
|
||||
return string(encoded)
|
||||
}
|
||||
|
||||
// TestSentryScrub_SDKCollectsTheRequestUnscrubbed pins the premise the
|
||||
// hook exists for. Without it the SDK ships the whole POST body, the
|
||||
// raw query and the CSRF header, none of which SendDefaultPII=false
|
||||
// suppresses.
|
||||
func TestSentryScrub_SDKCollectsTheRequestUnscrubbed(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
event := captureThroughSentryHTTP(t, false)
|
||||
require.NotNil(t, event.Request)
|
||||
|
||||
assert.Contains(
|
||||
t, event.Request.Data, sentryBodyMarker,
|
||||
"the SDK is expected to collect the POST body; if it no "+
|
||||
"longer does, the scrub hook's premise changed",
|
||||
)
|
||||
assert.Contains(t, event.Request.QueryString, sentryQueryMarker)
|
||||
assert.Contains(
|
||||
t, marshalEvent(t, event), sentryHeaderMarker,
|
||||
)
|
||||
}
|
||||
|
||||
// TestSentryScrub_RedactsTheCapturedRequest is the regression test: no
|
||||
// byte of any planted credential may survive into the marshalled event
|
||||
// that leaves the process.
|
||||
func TestSentryScrub_RedactsTheCapturedRequest(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
event := captureThroughSentryHTTP(t, true)
|
||||
require.NotNil(t, event.Request)
|
||||
|
||||
encoded := marshalEvent(t, event)
|
||||
|
||||
assert.NotContains(t, encoded, sentryBodyMarker)
|
||||
assert.NotContains(t, encoded, sentryQueryMarker)
|
||||
assert.NotContains(t, encoded, sentryHeaderMarker)
|
||||
assert.NotContains(t, encoded, "hooks.slack.com")
|
||||
|
||||
assert.Equal(t, "(redacted)", event.Request.Data)
|
||||
assert.Equal(t, "(redacted)", event.Request.QueryString)
|
||||
assert.Empty(t, event.Request.Cookies)
|
||||
assert.Empty(t, event.Request.Env)
|
||||
}
|
||||
|
||||
// TestSentryScrub_KeepsTheRoutingContext checks the hook does not cost
|
||||
// the debugging signal: the route, the method and the metadata headers
|
||||
// still identify what failed.
|
||||
func TestSentryScrub_KeepsTheRoutingContext(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
event := captureThroughSentryHTTP(t, true)
|
||||
require.NotNil(t, event.Request)
|
||||
|
||||
assert.Contains(t, event.Request.URL, "/pages/login")
|
||||
assert.Equal(t, http.MethodPost, event.Request.Method)
|
||||
assert.Equal(
|
||||
t,
|
||||
sentryKeptUserAgent,
|
||||
event.Request.Headers["User-Agent"],
|
||||
)
|
||||
assert.Equal(
|
||||
t,
|
||||
"application/x-www-form-urlencoded",
|
||||
event.Request.Headers["Content-Type"],
|
||||
)
|
||||
}
|
||||
|
||||
// TestSentryScrub_ToleratesEventsWithoutARequest covers the events the
|
||||
// hook sees outside an HTTP handler, where no request is attached.
|
||||
func TestSentryScrub_ToleratesEventsWithoutARequest(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
scrubbed := server.ScrubSentryRequestForTest(
|
||||
sentry.NewEvent(), nil,
|
||||
)
|
||||
|
||||
require.NotNil(t, scrubbed)
|
||||
assert.Nil(t, scrubbed.Request)
|
||||
assert.Nil(t, server.ScrubSentryRequestForTest(nil, nil))
|
||||
}
|
||||
@@ -141,14 +141,14 @@ func (s *Server) enableSentry() {
|
||||
return
|
||||
}
|
||||
|
||||
err := sentry.Init(sentry.ClientOptions{
|
||||
Dsn: s.params.Config.SentryDSN,
|
||||
Release: fmt.Sprintf(
|
||||
err := sentry.Init(sentryClientOptions(
|
||||
s.params.Config.SentryDSN,
|
||||
fmt.Sprintf(
|
||||
"%s-%s",
|
||||
s.params.Globals.Appname,
|
||||
s.params.Globals.Version,
|
||||
),
|
||||
})
|
||||
))
|
||||
if err != nil {
|
||||
s.log.Error("sentry init failure", "error", err)
|
||||
// Don't use fatal since we still want the service to run
|
||||
|
||||
Reference in New Issue
Block a user