With TRUSTED_PROXIES empty behind the reverse proxy production is
required to run behind, every login POST keyed on the proxy's address
and shared one 5/minute bucket. A stranger sending five POSTs a
minute -- 0.08 requests per second, from anywhere -- kept that bucket
permanently full, and the operator's own correct password was answered
429 indefinitely with no second administrative path.
The login POST no longer has a pre-emptive limiter. The handler
verifies credentials first and spends budget only on a FAILED attempt,
so a correct password is never throttled whatever the counters hold.
Three things follow, and are implemented together because the first is
unsafe without the other two:
- Failures are counted per (client bucket, submitted username), five
per minute, after which further failures get 429 with a Retry-After.
A successful login clears the counter, so mistyping and then
succeeding does not leave the operator throttled.
- Both key sets are capped at 1024 entries. The submitted username is
attacker-controlled, so past the first cap failures fall back to a
counter keyed on the client alone, and past both caps a failure is
answered as throttled without being recorded. Tracked state stays
under half a megabyte and does not grow with invented usernames.
- Concurrent Argon2id verifications are capped at two, a 128 MB
ceiling at 64 MB per hash, and the queue for those slots is capped
at 16 waiters. Every password-hashing endpoint takes a slot,
including the password-change endpoint, which holds one across both
its hashes. A request that waits five seconds without a slot is
answered 503, and one that arrives with the queue already full is
shed with 503 immediately rather than joining it. Bounding the wait
alone would not bound memory, and the queue depth is sized from what
a parked waiter measurably retains rather than from the 1 MB body
cap, which bounds only the raw body read. The body-cap, CSRF and
form-parsing middleware all run before the guard, so a waiter holds
its parsed form plus its request header block for the whole wait.
Measured on the pinned go1.26.1 toolchain as the HeapAlloc delta
across two GCs with 64 waiters parked in the handler: an ordinary
two-field login form retains ~0 MB, a 1 MB urlencoded body at Go's
10,000-parameter parse cap retains 2.82 MB (3.09 MB with %41
escapes), and the ~0.9 MB of headers the 1 MB header cap allows
takes it to 4.18 MB. The retained parse and the header block
dominate, not the raw body. So 16 waiters: 16 x 4.18 MB is about
67 MB of committed queue memory, and two slots drain a full 16-deep
queue in about 0.6 s, far inside the deadline. Peak commitment for
the endpoint is about 203 MB — 128 MB of Argon2id plus the 18
requests holding a parsed form, 16 queued and the 2 being hashed, at
about 75 MB. That 203 MB is live commitment, not resident size: the
Go collector lets the heap reach roughly twice the live set before
collecting, and the review measured a peak HeapAlloc of 392 MB under
18 adversarial requests, so the README says to provision on the
order of 400 MB.
An unknown username is verified against a dummy hash instead of
returning early, so a nonexistent account costs the same time as a
real one and the response cannot be used to enumerate usernames.
The password-change limiter is unchanged: RequireAuth runs ahead of
it, so only a request already carrying a valid session reaches its
bucket.
Two consequences are documented rather than fixed, because they follow
from the shape the issue asks for. Online guessing throughput rises
from 5 a minute to roughly 27 a second, about 2.3 million a day: the
credential check always precedes the counter, so the 429 is a label on
the response rather than a gate in front of the hash, and what bounds
brute force is the semaphore. And under a sustained flood the residual
exposure is a loss of login availability, not merely of latency --
above about 27 requests a second most attempts are shed with 503, so a
determined flood still denies login for as long as it runs. It costs
roughly 400x more to run, nothing accumulates, and the first attempt
after it stops succeeds. Restarting the service does not help: the
counters a restart clears are not what is saturated.
Also adds the missing test for the third bucketKey call site, where
the peer is a trusted proxy but the forwarded chain names no client.
Every existing test of that fallback uses an IPv4 proxy, where
bucketKey is the identity function, so dropping the /64 masking there
left the suite green.
README and the TRUSTED_PROXIES startup warning updated: a shared
bucket now costs precision, not the availability of the admin path.
The delivery engine worker pool and the retention reaper both rooted their
goroutines in the fx OnStart hook context, which fx cancels 15s into startup.
Both now use context.WithCancel(context.Background()), bounded by OnStop.
Enforces each webhook's `RetentionDays` so per-webhook SQLite files no longer grow without bound.
## Reaper
New `RetentionReaper` in `internal/database/retention.go`. A background ticker runs each sweep: it lists all webhooks from the main DB and, for each webhook with a positive `RetentionDays`, opens its per-webhook DB via `WebhookDBManager.GetDB` and deletes every `Event` (and its dependent `Delivery` and `DeliveryResult` rows) whose `CreatedAt` is older than `RetentionDays` days.
- Deletions run in foreign-key-safe order: delivery results, then deliveries, then events.
- Deletes are unscoped (hard deletes) so rows are physically removed and disk is reclaimed, rather than GORM soft-deleting them.
- `RetentionDays <= 0` means retain forever; those webhooks are skipped.
- Webhooks whose per-webhook DB does not yet exist are skipped.
## Config
`internal/config/config.go` gains `RetentionSweepInterval` (env `RETENTION_SWEEP_INTERVAL`, parsed as a Go duration, default `1h`) via a new `envDuration` helper, following the existing env-helper conventions.
## Wiring
`cmd/webhooker/main.go` registers `database.NewRetentionReaper` as an fx provider and forces its construction in `fx.Invoke`. The reaper starts its sweep loop on an fx `OnStart` hook and stops cleanly on `OnStop` via context cancellation, matching the existing lifecycle components.
## Test
`internal/database/retention_test.go` seeds an old event chain (event + delivery + result, 40 days old) and a recent one (1 day old) in a real per-webhook DB and asserts a single sweep removes only the expired chain while keeping the recent one. A second test forces a non-positive `RetentionDays` and asserts an ancient event is retained.
Note: the `Webhook.RetentionDays` column carries `gorm:"default:30"`, so a `0` passed to a GORM `Create` is replaced by the default; the test forces the value with an explicit column update to exercise the retain-forever path. No model changes were made.
Validated with `docker build .` (fmt-check, lint, test, build) exit 0.
Closes#63
Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #78
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
