With TRUSTED_PROXIES empty behind the reverse proxy production is
required to run behind, every login POST keyed on the proxy's address
and shared one 5/minute bucket. A stranger sending five POSTs a
minute -- 0.08 requests per second, from anywhere -- kept that bucket
permanently full, and the operator's own correct password was answered
429 indefinitely with no second administrative path.
The login POST no longer has a pre-emptive limiter. The handler
verifies credentials first and spends budget only on a FAILED attempt,
so a correct password is never throttled whatever the counters hold.
Three things follow, and are implemented together because the first is
unsafe without the other two:
- Failures are counted per (client bucket, submitted username), five
per minute, after which further failures get 429 with a Retry-After.
A successful login clears the counter, so mistyping and then
succeeding does not leave the operator throttled.
- Both key sets are capped at 1024 entries. The submitted username is
attacker-controlled, so past the first cap failures fall back to a
counter keyed on the client alone, and past both caps a failure is
answered as throttled without being recorded. Tracked state stays
under half a megabyte and does not grow with invented usernames.
- Concurrent Argon2id verifications are capped at two, a 128 MB
ceiling at 64 MB per hash, and the queue for those slots is capped
at 16 waiters. Every password-hashing endpoint takes a slot,
including the password-change endpoint, which holds one across both
its hashes. A request that waits five seconds without a slot is
answered 503, and one that arrives with the queue already full is
shed with 503 immediately rather than joining it. Bounding the wait
alone would not bound memory, and the queue depth is sized from what
a parked waiter measurably retains rather than from the 1 MB body
cap, which bounds only the raw body read. The body-cap, CSRF and
form-parsing middleware all run before the guard, so a waiter holds
its parsed form plus its request header block for the whole wait.
Measured on the pinned go1.26.1 toolchain as the HeapAlloc delta
across two GCs with 64 waiters parked in the handler: an ordinary
two-field login form retains ~0 MB, a 1 MB urlencoded body at Go's
10,000-parameter parse cap retains 2.82 MB (3.09 MB with %41
escapes), and the ~0.9 MB of headers the 1 MB header cap allows
takes it to 4.18 MB. The retained parse and the header block
dominate, not the raw body. So 16 waiters: 16 x 4.18 MB is about
67 MB of committed queue memory, and two slots drain a full 16-deep
queue in about 0.6 s, far inside the deadline. Peak commitment for
the endpoint is about 203 MB — 128 MB of Argon2id plus the 18
requests holding a parsed form, 16 queued and the 2 being hashed, at
about 75 MB. That 203 MB is live commitment, not resident size: the
Go collector lets the heap reach roughly twice the live set before
collecting, and the review measured a peak HeapAlloc of 392 MB under
18 adversarial requests, so the README says to provision on the
order of 400 MB.
An unknown username is verified against a dummy hash instead of
returning early, so a nonexistent account costs the same time as a
real one and the response cannot be used to enumerate usernames.
The password-change limiter is unchanged: RequireAuth runs ahead of
it, so only a request already carrying a valid session reaches its
bucket.
Two consequences are documented rather than fixed, because they follow
from the shape the issue asks for. Online guessing throughput rises
from 5 a minute to roughly 27 a second, about 2.3 million a day: the
credential check always precedes the counter, so the 429 is a label on
the response rather than a gate in front of the hash, and what bounds
brute force is the semaphore. And under a sustained flood the residual
exposure is a loss of login availability, not merely of latency --
above about 27 requests a second most attempts are shed with 503, so a
determined flood still denies login for as long as it runs. It costs
roughly 400x more to run, nothing accumulates, and the first attempt
after it stops succeeds. Restarting the service does not help: the
counters a restart clears are not what is saturated.
Also adds the missing test for the third bucketKey call site, where
the peer is a trusted proxy but the forwarded chain names no client.
Every existing test of that fallback uses an IPv4 proxy, where
bucketKey is the identity function, so dropping the /64 masking there
left the suite green.
README and the TRUSTED_PROXIES startup warning updated: a shared
bucket now costs precision, not the availability of the admin path.
Publishing this README would have shipped false statements about the
product. Corrects the eight items on the issue plus everything a full
sweep turned up: the Slack circuit-breaker scope, a nonexistent WAL, the
wrong config key for slack targets, six undocumented routes, the
conditional /metrics registration, wrong retention bands, wrong shutdown
mechanism, and a Quick Start that led a new contributor into a red
build.
The lockout warning now fires whenever TRUSTED_PROXIES is empty rather
than only in production, since the variable it was gated on defaults to
dev. Rate-limit keying, the limits and the TRUSTED_PROXIES default are
untouched — those belong to #150.
What /s/* actually serves was settled empirically rather than by
reading: all five of GET/HEAD/POST/PUT/DELETE return 200, pinned by
TestStaticServesEveryMethod. Restricting it is filed separately.
Independently reviewed after three prior rounds. The reviewer
re-derived all fifteen claim-table rows against the code, including
every row a previous revision had marked "correct, left alone" and got
wrong, and found zero false; then verified every route method-by-method,
all twelve environment variables, all nine entity tables, and the
package tree against git ls-files. The Quick Start was confirmed by
running it in a fresh clone.
With no trusted proxies configured, every client behind the reverse proxy production requires shares one rate-limit bucket per limit, so five POSTs per minute from anywhere holds the login limit full and denies the admin login until restart. The default is still correct; it was the consequence that was invisible. Startup now warns, and the README no longer claims the login limit is per-IP unconditionally.
A non-positive value reached time.NewTicker in the retention reaper and the archive sweeper, panicking both goroutines after startup had already reported success. envPositiveDuration now rejects it in loadFromEnv, matching how PORT and RECEIVER_RATE_LIMIT fail. SESSION_IDLE_TIMEOUT keeps treating non-positive as disabled, which is guarded at every use site.
All three rate limiters (receiver, login, password change) now key on the connection's own address unless the direct peer is inside the new TRUSTED_PROXIES CIDR list, in which case X-Forwarded-For is walked right to left for the first non-proxy hop. Default is the empty list, which trusts nothing. A set-but-unparseable value aborts startup.
The receiver was the one unauthenticated, internet-facing endpoint with no
rate limit, so a misbehaving or hostile sender could flood a webhook
without bound. RECEIVER_RATE_LIMIT (default 120/min) now caps it, keyed on
client IP plus entrypoint path so one entrypoint cannot exhaust another's
budget. Over-limit requests get 429 with Retry-After.
The limiter deliberately does not reuse postRateLimit: that helper is
POST-only and keys on IP alone, whereas the receiver must count every
method. A test locks that property in.
Config parsing follows the fail-loudly idiom: a set-but-unparseable or
non-positive value aborts startup rather than falling back to the default.
Known limitation, tracked in #88: the key still trusts forwarded headers
unconditionally, so the limit is evadable by rotating X-Forwarded-For until
trusted-proxy gating lands.
Sessions now carry a server-enforced idle deadline (SESSION_IDLE_TIMEOUT,
default 24h) alongside the 7-day absolute cap, refreshed on authenticated
activity. Activity never extends the absolute cap.
Enforces each webhook's `RetentionDays` so per-webhook SQLite files no longer grow without bound.
## Reaper
New `RetentionReaper` in `internal/database/retention.go`. A background ticker runs each sweep: it lists all webhooks from the main DB and, for each webhook with a positive `RetentionDays`, opens its per-webhook DB via `WebhookDBManager.GetDB` and deletes every `Event` (and its dependent `Delivery` and `DeliveryResult` rows) whose `CreatedAt` is older than `RetentionDays` days.
- Deletions run in foreign-key-safe order: delivery results, then deliveries, then events.
- Deletes are unscoped (hard deletes) so rows are physically removed and disk is reclaimed, rather than GORM soft-deleting them.
- `RetentionDays <= 0` means retain forever; those webhooks are skipped.
- Webhooks whose per-webhook DB does not yet exist are skipped.
## Config
`internal/config/config.go` gains `RetentionSweepInterval` (env `RETENTION_SWEEP_INTERVAL`, parsed as a Go duration, default `1h`) via a new `envDuration` helper, following the existing env-helper conventions.
## Wiring
`cmd/webhooker/main.go` registers `database.NewRetentionReaper` as an fx provider and forces its construction in `fx.Invoke`. The reaper starts its sweep loop on an fx `OnStart` hook and stops cleanly on `OnStop` via context cancellation, matching the existing lifecycle components.
## Test
`internal/database/retention_test.go` seeds an old event chain (event + delivery + result, 40 days old) and a recent one (1 day old) in a real per-webhook DB and asserts a single sweep removes only the expired chain while keeping the recent one. A second test forces a non-positive `RetentionDays` and asserts an ancient event is retained.
Note: the `Webhook.RetentionDays` column carries `gorm:"default:30"`, so a `0` passed to a GORM `Create` is replaced by the default; the test forces the value with an explicit column update to exercise the retain-forever path. No model changes were made.
Validated with `docker build .` (fmt-check, lint, test, build) exit 0.
Closes#63
Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #78
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
Closes [issue #50](#50)
## Summary
Refactors the Dockerfile to use a separate lint stage with a pinned golangci-lint Docker image, following the pattern used by [sneak/pixa](https://git.eeqj.de/sneak/pixa). This replaces the previous approach of installing golangci-lint via curl in the builder stage.
## Changes
### Dockerfile
- **New `lint` stage** using `golangci/golangci-lint:v2.11.3` (Debian-based, pinned by sha256 digest) as a separate build stage
- **Builder stage** depends on lint via `COPY --from=lint /src/go.sum /dev/null` — build won't proceed unless linting passes
- **Go bumped** from 1.24 to 1.26.1 (`golang:1.26.1-bookworm`, pinned by sha256)
- **golangci-lint bumped** from v1.64.8 to v2.11.3
- All three Docker images (golangci-lint, golang, alpine) pinned by sha256 digest
- Debian-based golangci-lint image used (not Alpine) because mattn/go-sqlite3 CGO does not compile on musl (off64_t)
### Linter Config (.golangci.yml)
- Migrated from v1 to v2 format (`version: "2"` added)
- Removed linters no longer available in v2: `gofmt` (handled by `make fmt-check`), `gosimple` (merged into `staticcheck`), `typecheck` (always-on in v2)
- Same set of linters enabled — no rules weakened
### Code Fixes (all lint issues from v2 upgrade)
- Added package comments to all packages
- Added doc comments to all exported types, functions, and methods
- Fixed unchecked errors flagged by `errcheck` (sqlDB.Close, os.Setenv in tests, resp.Body.Close, fmt.Fprint)
- Fixed unused parameters flagged by `revive` (renamed to `_`)
- Fixed `gosec` G120 warnings: added `http.MaxBytesReader` before `r.ParseForm()` calls
- Fixed `staticcheck` QF1012: replaced `WriteString(fmt.Sprintf(...))` with `fmt.Fprintf`
- Fixed `staticcheck` QF1003: converted if/else chain to tagged switch
- Renamed `DeliveryTask` → `Task` to avoid package stutter (`delivery.Task` instead of `delivery.DeliveryTask`)
- Renamed shadowed builtin `max` parameter to `upperBound` in `cryptoRandInt`
- Used `t.Setenv` instead of `os.Setenv` in tests (auto-restores)
### README.md
- Updated version requirements: Go 1.26+, golangci-lint v2.11+
- Updated Dockerfile description in project structure
## Verification
`docker build .` passes cleanly — formatting check, linting, all tests, and build all succeed.
Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #55
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
Closes#45.
## Problem
1. The README didn't clearly explain what `WEBHOOKER_ENVIRONMENT=dev` vs `prod` actually changes.
2. The dev-mode default for `DATA_DIR` was `./data` — a relative path whose meaning depends on the working directory. There's no reason to use a relative path even in development.
## Changes
### Code (`internal/config/config.go`)
- Replace the dev default `DATA_DIR` from `./data` to `$XDG_DATA_HOME/webhooker` (falling back to `$HOME/.local/share/webhooker`). This follows the XDG Base Directory Specification and ensures the data directory is always an absolute path regardless of the working directory.
- Add `devDataDir()` helper that resolves the XDG path, with a `/tmp/webhooker` last-resort fallback if `$HOME` can't be determined.
### Tests (`internal/config/config_test.go`)
- `TestDevDataDir`: verifies XDG_DATA_HOME is respected, HOME fallback works, and the result is always absolute.
- `TestDevDefaultDataDirIsAbsolute`: integration test that creates a full Config via fx and asserts the dev default DataDir is absolute.
### README
- Add a table documenting exactly what `dev` vs `prod` changes: DATA_DIR default, CORS policy, and session cookie Secure flag.
- Clarify that log format and security headers are independent of the environment setting.
- Update the DATA_DIR default in the configuration variable table.
Co-authored-by: clawbot <clawbot@eeqj.de>
Co-authored-by: user <user@Mac.lan guest wan>
Reviewed-on: #46
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
DBURL → DATA_DIR consolidation:
- Remove DBURL env var entirely; main DB now lives at {DATA_DIR}/webhooker.db
- database.go constructs DB path from config.DataDir, ensures dir exists
- Update DATA_DIR prod default from /data/events to /data
- Update all tests to use DataDir instead of DBURL
- Update Dockerfile: /data (not /data/events) for all SQLite databases
- Update README configuration table, Docker examples, architecture docs
Dead code removal:
- Remove unused IndexResponse struct (handlers/index.go)
- Remove unused TemplateData struct (handlers/handlers.go)
Stale comment cleanup:
- Remove TODO in server.go (DB cleanup handled by fx lifecycle)
- Fix nolint:golint → nolint:revive on ServerParams for consistency
- Clean up verbose middleware/routing comments in routes.go
- Fix TODO fan-out description (worker pool, not goroutine-per-target)
.gitignore fixes:
- Add data/ directory to gitignore
- Remove stale config.yaml entry (env-only config since rework)
Remove the entire pkg/config package (Viper-based YAML config file
loader) and simplify internal/config to read all settings directly from
environment variables via os.Getenv(). This eliminates the spurious
"Failed to load config" log messages that appeared when no config.yaml
file was present.
- Delete pkg/config/ (YAML loader, resolver, manager, tests)
- Delete configs/config.yaml.example
- Simplify internal/config helper functions to use os.Getenv() with
defaults instead of falling back to pkgconfig
- Update tests to set env vars directly instead of creating in-memory
YAML config files via afero
- Remove afero, cloud.google.com/*, aws-sdk-go dependencies from go.mod
- Update README: document env-var-only configuration, remove YAML/Viper
references
- Keep godotenv/autoload for .env file convenience in local development
closes #27
Remove SESSION_KEY env var requirement. On first startup, a
cryptographically secure 32-byte key is generated and stored in a new
settings table. Subsequent startups load the key from the database.
- Add Setting model (key-value table) for application config
- Add Database.GetOrCreateSessionKey() method
- Session manager initializes in OnStart after database is connected
- Remove DevSessionKey constant and SESSION_KEY env var handling
- Remove prod validation requiring SESSION_KEY
- Update README: config table, Docker instructions, security notes
- Update config.yaml.example
- Update all tests to remove SessionKey references
Addresses owner feedback on issue #15.
Remove DevAdminUsername and DevAdminPassword fields from the Config
struct and their loading code. These fields were never referenced
anywhere else in the codebase.
Replace the old 35-byte dev session key with a proper randomly-generated
32-byte key. Also ensure dev mode actually falls back to DevSessionKey
when SESSION_KEY is not set in the environment, rather than leaving
SessionKey empty and failing at session creation.
Update tests to remove the old key references.
## Summary
This PR brings the webhooker repo into full REPO_POLICIES compliance, addressing both [issue #1](#1) and [issue #2](#2).
## Changes
### New files
- **`cmd/webhooker/main.go`** — The missing application entry point. Uses Uber fx to wire together all internal packages (config, database, logger, server, handlers, middleware, healthcheck, globals, session). Minimal glue code.
- **`REPO_POLICIES.md`** — Fetched from authoritative source (`sneak/prompts`)
- **`.editorconfig`** — Fetched from authoritative source
- **`.dockerignore`** — Sensible Go project exclusions
- **`.gitea/workflows/check.yml`** — CI workflow that runs `docker build .` on push to any branch (Gitea Actions format, actions/checkout pinned by sha256)
- **`configs/config.yaml.example`** — Moved from root `config.yaml`
### Modified files
- **`Makefile`** — Complete rewrite with all REPO_POLICIES required targets: `test`, `lint`, `fmt`, `fmt-check`, `check`, `build`, `hooks`, `docker`, `clean`, plus `dev`, `run`, `deps`
- **`Dockerfile`** — Complete rewrite:
- Builder: `golang:1.24` (Debian-based, pinned by `sha256:d2d2bc1c84f7...`). Debian needed because `gorm.io/driver/sqlite` pulls `mattn/go-sqlite3` (CGO) which fails on Alpine musl.
- golangci-lint v1.64.8 installed from GitHub release archive with sha256 verification (v1.x because `.golangci.yml` uses v1 config format)
- Runs `make check` (fmt-check + lint + test + build) as build step
- Final stage: `alpine:3.21` (pinned by `sha256:c3f8e73fdb79...`) with non-root user, healthcheck, port 8080
- **`README.md`** — Rewritten with all required REPO_POLICIES sections: description line with name/purpose/category/license/author, Getting Started, Rationale, Design, TODO (integrated from TODO.md), License, Author
- **`.gitignore`** — Fixed `webhooker` pattern to `/webhooker` (was blocking `cmd/webhooker/`), added `config.yaml` to prevent committing runtime config with secrets
- **`static/static.go`** — Removed `vendor` from embed directive (directory was empty/missing)
- **`internal/database/database_test.go`** — Fixed to use in-memory config via `afero.MemMapFs` instead of depending on `config.yaml` on disk. Test is now properly isolated.
- **`go.mod`/`go.sum`** — `go mod tidy`
### Removed files
- **`TODO.md`** — Content integrated into README.md TODO section
- **`config.yaml`** — Moved to `configs/config.yaml.example`
## Verification
- `docker build .` passes (lint ✅, test ✅, build ✅)
- All existing tests pass with no modifications to assertions or test logic
- `.golangci.yml` untouched
closes #1
closes #2
Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #6
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
