sneak/webhooker
1
0
Fork 0
Code Issues 34 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Compare commits

base: sneak:9ac8ff28ecea675984045733ac0496602551d695
Branches Tags
sneak:main sneak:next sneak:feat/receiver-rate-limit sneak:issue-70-log-target
...
compare: sneak:92f3a016e1aefca458e2d950de11259f88bd8d7a
Branches Tags
sneak:next sneak:main sneak:feat/receiver-rate-limit sneak:issue-70-log-target

4 Commits
9ac8ff28ec ... 92f3a016e1

Author SHA1 Message Date
sneak
92f3a016e1 Correct release-blocking README and startup-warning inaccuracies (closes #151)
All checks were successful
check / check (push) Successful in 2m59s
Details 
The empty-TRUSTED_PROXIES warning was gated on IsProd(), but
WEBHOOKER_ENVIRONMENT defaults to dev, so an internet-exposed
deployment whose operator never set it got no warning at all — the
exact operator error the warning exists to catch. It now fires whenever
the list is empty, in any environment, and its text is accurate both
behind a reverse proxy (shared buckets, remotely deniable admin login)
and with nothing in front of the process (harmless). The startup
configuration summary also now logs sessionIdleTimeout, the one value
where a valid setting silently disables a security control.

The README documented a two-stage Docker build on golang:1.24 running
"make check" (the tree has three stages: a golangci-lint lint stage
running fmt-check and lint, a golang:1.26.1-bookworm builder running
test and build, then the Alpine runtime), advertised the public
receiver as accepting all methods (it answers 405 to everything but
POST), claimed unqualified per-IP login rate limiting, and left the
session-expiry prose orphaned inside the trusted-proxy subsection.

The rest of the README was swept against the code rather than only the
reported lines: every documented route checked method-by-method against
internal/server/routes.go (adding the password-change, entrypoint and
target routes that were missing), every environment variable checked
against internal/config/config.go (MAINTENANCE_MODE serves no
maintenance page — it only sets a healthcheck field), the fx wiring,
package tree, prerequisites and dev commands brought back in line with
the tree.

Statements the sweep had waved through, each re-derived from the code
this time:

- Circuit breakers are not HTTP-only. target_slack.go embeds httpCore
  and hands its own MaxRetries to the same retry path, so a slack
  target with max_retries > 0 gets a breaker on the same defaults.
- No WAL exists. Both DSNs are file:%s?cache=shared&mode=rwc and no
  journal_mode pragma is issued anywhere, so every database runs on
  SQLite's rollback journal.
- Slack config is keyed webhookUrl, not webhook_url; the underscored
  spelling is only the error message. An operator copying the README
  wrote config that was silently ignored.
- Setting carries no BaseModel, so neither the common-field list nor
  "soft deletes on all entities" held for it.
- DeliveryTask names no type; it is delivery.Task.
- script/lint runs golangci-lint on the host. Only script/cibuild and
  script/docker involve Docker; #109 tracks changing that, and until it
  lands the README says what the tree does.
- An entrypoint's path column holds a bare UUID. The /webhook/ prefix
  is route only and never stored.
- max_queue_size is stored and displayed but consulted by nothing;
  queue depth is the two fixed 10,000-entry channels.
- Inline event bodies are cut at "< 16 KiB", not "≤".
- retention_days 0 belongs to the retain-forever band — BeforeSave
  rewrites it to the sentinel — not the finite one, whose floor is 1.
  A negative value is a 400 in parseRetentionDays.
- The receiver's per-client limit keys on the request path
  (httprate.KeyByEndpoint), not on the entrypoint; the paragraph eight
  lines below already said so and the two contradicted each other.
- Shutdown goes through lifecycle.WaitForShutdown, which is bounded by
  fx's stop context and can return with goroutines still running,
  rather than a bare WaitGroup.Wait().
- Nine of the Makefile's fifteen targets shim script/; build, run, dev,
  deps, clean and css are inline.
- Nine entities are documented and nine model files exist, not eight.
- The metrics middleware is only registered when METRICS_USERNAME is
  set, so an over-limit request is not always counted.

Two gaps closed while checking: the target-type list omitted slack
entirely, and the package tree omitted event_log_view.go and the three
testing.go files, which are ordinary compiled sources exporting
NewTestDatabase and NewForTest rather than _test.go scaffolding.

TestStaticServesEveryMethod settles what /s/* actually answers: chi's
Mount registers every method and http.FileServer special-cases only
HEAD, so POST, PUT and DELETE to an asset are served the file. The
README claimed GET and HEAD.

TODO.md drops the unsupported half of its CI claim, keeping the
cache-defeated container runs, and splits the landed password change
away from the unimplemented reset flow.

Rebasing onto current next moved the tree under three of these
sections, so they were re-derived against the rebased tree rather
than the one the branch was cut from: alpine.min.js is no longer
committed but fetched and hash-verified by script/fetch-assets, so the
package tree entry for it was wrong and vendor.sha256 was absent; the
builder stage runs that script before make test; the Makefile is now
ten shims out of sixteen targets, and fmt-check was missing from the
development command list. The Quick Start was also wrong in a way that
costs a new contributor a red build: it said make deps, which only
runs go mod download and tidy, and make check then fails two tests on
the absent asset. It now says make bootstrap, which fetches it.
 2026-08-17 21:23:02 +00:00
clawbot
c378690977 Fetch and verify Alpine at build time instead of committing it (closes #145)
All checks were successful
check / check (push) Successful in 2m58s
Details 
static/js/alpine.min.js was a committed minified bundle, which
REPO_POLICIES forbids, referenced by no content hash at all. A minified
blob is unreviewable, which is the shape a supply-chain compromise
takes.

script/fetch-assets now downloads Alpine 3.14.9 from the npm registry
and verifies sha256 on both the tarball and the extracted file, and
static/vendor_test.go re-hashes the bytes go:embed actually placed in
the binary. The shipped bytes are byte-identical to the blob that was
committed, so the served asset does not change.

Independently reviewed. Five negative controls reproduced by the
reviewer: flipped expected hash, repointed URL, post-fetch tampering,
asset absent, and manifest inconsistencies — each fails closed with
static/js/ left clean. Registry hashes confirmed against the pins, and
the runtime image was built, run and curled to confirm the asset is
still served and the login page still loads it.

Known gap, filed separately: static/static.go embeds the js directory
rather than named files, so a missing fetched asset is not a compile
error on ungated local build paths. Every gated path fails loudly, so
the release artifact is unaffected.
 2026-08-17 23:12:17 +02:00
clawbot
279effb4c2 Bound the event log's rendered bodies in the query (closes #135)
All checks were successful
check / check (push) Successful in 3m0s
Details 
The event log rendered stored bodies untruncated. Since buffered
rendering landed (#123) that became resident memory per concurrent
viewer, up to tens of MB, driven by payloads unauthenticated clients
supply to the public receiver.

Bound in the query rather than the template, via
substr(cast(body as blob), 1, ?) plus length(cast(body as blob)), so an
oversized body never becomes a Go string at all. Adds an EventLogView
projection carrying the true byte count, and trims a partial UTF-8 tail
without rewriting bodies that are merely invalid UTF-8.

Independently reviewed. The generated SQL was dumped under GORM DryRun
to confirm the cap is a bound parameter, both casts are present, and no
other path selects the full column; soft-delete scope, ordering and
pagination are unchanged.

Correction to the PR body: its quoted mutation output was produced by
removing the bound from eventLogColumns, not by raising the cap to
1&lt;&lt;30 as the text claimed. The reviewer reproduced the real
mutation and confirmed the tests do catch removal of the bound.

Follow-up #157 restores in-app retrieval of bodies above the cap.
 2026-08-17 22:57:08 +02:00
clawbot
9ae19159a3 Mask the http target's destination URL in the UI (closes #115)
Some checks failed
check / check (push) Superseded by a newer commit; never tested
The http target's destination URL can itself be a bearer credential, and
the source detail page rendered it in full. Render it through the
existing MaskURL instead, matching the rule already applied to slack
targets.

Independently reviewed: mutation-verified (reverting to the raw value
fails the absence assertions, not merely the masked-form ones), MaskURL
probed against userinfo, query, fragment, port, IPv6 literal and
non-http schemes, and every sibling path that surfaces target data
re-walked and found clean.
 2026-08-17 22:50:26 +02:00
24 changed files with 1173 additions and 174 deletions
Expand all files Collapse all files

5
.dockerignore
View File

@@ -3,6 +3,11 @@
# stage of the Dockerfile. # stage of the Dockerfile.
.git/ .git/
bin/ bin/
# Third-party browser assets are fetched and hash-verified inside the build by
# script/fetch-assets. Excluding any host copy keeps a developer's working tree
# from supplying the bytes that get shipped. The script and its
# static/vendor.sha256 manifest stay in the context.
static/js/alpine.min.js
*.md *.md
LICENSE LICENSE
.editorconfig .editorconfig

5
.gitignore vendored
View File

@@ -45,3 +45,8 @@ temp/
# CI cache barrier, written into the build context by the check workflow # CI cache barrier, written into the build context by the check workflow
.ci-fingerprint .ci-fingerprint
# Third-party browser assets, fetched and hash-verified by
# script/fetch-assets against static/vendor.sha256. Not committed:
# REPO_POLICIES.md forbids minified bundles in version control.
/static/js/alpine.min.js

10
Dockerfile
View File

@@ -32,7 +32,7 @@ FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a349228
# Depend on lint stage passing # Depend on lint stage passing
COPY --from=lint /src/go.sum /dev/null COPY --from=lint /src/go.sum /dev/null
RUN apt-get update && apt-get install -y --no-install-recommends make && rm -rf /var/lib/apt/lists/* RUN apt-get update && apt-get install -y --no-install-recommends make curl ca-certificates && rm -rf /var/lib/apt/lists/*
WORKDIR /build WORKDIR /build
@@ -44,6 +44,14 @@ RUN go mod download
# the lint stage above. # the lint stage above.
COPY . . COPY . .
# Fetch the third-party browser assets the UI serves. They are not committed
# (REPO_POLICIES.md forbids minified bundles in version control) and
# .dockerignore keeps any host copy out of the build context, so this step is
# the only way they enter the image. Each download is checked against a
# hardcoded sha256 and the build fails on mismatch; make test re-checks the
# hashes against the bytes go:embed actually put in the binary.
RUN script/fetch-assets
# Run tests and build # Run tests and build
RUN make test RUN make test
RUN make build RUN make build

5
Makefile
View File

@@ -1,4 +1,4 @@
.PHONY: bootstrap setup test lint fmt fmt-check check build run dev deps docker clean hooks css .PHONY: bootstrap setup assets test lint fmt fmt-check check build run dev deps docker clean hooks css
# Default target # Default target
.DEFAULT_GOAL := check .DEFAULT_GOAL := check
@@ -9,6 +9,9 @@ bootstrap:
setup: setup:
@script/setup @script/setup
assets:
@script/fetch-assets
test: test:
@script/test @script/test

377
README.md
View File

@@ -11,9 +11,14 @@ with retry support, logging, and observability. Category: infrastructure
### Prerequisites ### Prerequisites
- Go 1.26+ - Go 1.26.1+ (the version in `go.mod`)
- golangci-lint v2.11+ - golangci-lint v2.12.2 (the version pinned in `script/bootstrap` and
- Docker (for containerized deployment) in the `Dockerfile`'s lint stage; `make bootstrap` installs it)
- Docker (for containerized deployment, and for the lint and test
stages of the CI gate)
- `curl`, used by `script/fetch-assets` to download the third-party
browser assets, which are not committed (`make bootstrap` installs
it if missing)
### Quick Start ### Quick Start
@@ -22,14 +27,18 @@ with retry support, logging, and observability. Category: infrastructure
git clone https://git.eeqj.de/sneak/webhooker.git git clone https://git.eeqj.de/sneak/webhooker.git
cd webhooker cd webhooker
# Install Go dependencies # Install Go dependencies, the pinned linter, and the third-party
make deps # browser assets. `make deps` alone is not enough: it only runs
# go mod download/tidy, and the checks below need the fetched assets.
make bootstrap
# Run all checks (format, lint, test, build) # Run all checks (test, lint, format check)
make check make check
# Run in development mode (uses SQLite in current directory) # Run in development mode. DATA_DIR defaults to /var/lib/webhooker in
make dev # every environment, so set it (in .env or the shell) to a writable
# directory when running from a clone.
DATA_DIR=./data make dev
# Build Docker image # Build Docker image
make docker make docker
@@ -40,14 +49,20 @@ make docker
```bash ```bash
make bootstrap # Install all dependencies (idempotent) make bootstrap # Install all dependencies (idempotent)
make setup # Bootstrap + install git pre-commit hook make setup # Bootstrap + install git pre-commit hook
make assets # Fetch + verify third-party browser assets
make fmt # Format code (gofmt + goimports) make fmt # Format code (gofmt + goimports)
make fmt-check # Fail if gofmt would change anything (writes nothing)
make lint # Run golangci-lint make lint # Run golangci-lint
make test # Run tests with race detection make test # Run tests with race detection
make check # test + lint + fmt-check (CI gate) make check # test + lint + fmt-check (CI gate)
make build # Build binary to bin/webhooker make build # Build binary to bin/webhooker
make run # build, then run ./bin/webhooker
make dev # go run ./cmd/webhooker make dev # go run ./cmd/webhooker
make deps # go mod download + go mod tidy
make docker # Build Docker image make docker # Build Docker image
make hooks # Install git pre-commit hook that runs script/precommit make hooks # Install git pre-commit hook that runs script/precommit
make css # Regenerate static/css/tailwind.css (needs tailwindcss)
make clean # Remove bin/
``` ```
### Configuration ### Configuration
@@ -89,7 +104,7 @@ TTY detection, and security headers are always applied.
| `PORT` | HTTP listen port | `8080` | | `PORT` | HTTP listen port | `8080` |
| `DATA_DIR` | Directory for all SQLite databases | `/var/lib/webhooker` | | `DATA_DIR` | Directory for all SQLite databases | `/var/lib/webhooker` |
| `DEBUG` | Enable debug logging | `false` | | `DEBUG` | Enable debug logging | `false` |
| `MAINTENANCE_MODE` | Serve the maintenance page | `false` | | `MAINTENANCE_MODE` | Report `maintenanceMode: true` in the healthcheck JSON. It does not change how any request is served — no maintenance page exists | `false` |
| `METRICS_USERNAME` | Basic auth username for `/metrics` | `""` | | `METRICS_USERNAME` | Basic auth username for `/metrics` | `""` |
| `METRICS_PASSWORD` | Basic auth password for `/metrics` | `""` | | `METRICS_PASSWORD` | Basic auth password for `/metrics` | `""` |
| `SENTRY_DSN` | Sentry error reporting DSN | `""` | | `SENTRY_DSN` | Sentry error reporting DSN | `""` |
@@ -129,8 +144,13 @@ sustained trickle re-locks them immediately.
The remedy is to set `TRUSTED_PROXIES` to your reverse proxy's The remedy is to set `TRUSTED_PROXIES` to your reverse proxy's
address, which restores per-client buckets. webhooker logs a warning address, which restores per-client buckets. webhooker logs a warning
at startup when `WEBHOOKER_ENVIRONMENT=prod` and `TRUSTED_PROXIES` is at startup whenever `TRUSTED_PROXIES` is empty, in every environment —
empty. See [Rate Limiting](#rate-limiting) for what each limit shares. not only when `WEBHOOKER_ENVIRONMENT=prod`, because that variable
defaults to `dev` and an operator who never set it is precisely the
one at risk. The warning is informational when nothing proxies to the
process: with no proxy in front, the peer address is the client's own
and the buckets are already per-client. See
[Rate Limiting](#rate-limiting) for what each limit shares.
`X-Real-IP` and `True-Client-IP` are **never** read, from any peer. `X-Real-IP` and `True-Client-IP` are **never** read, from any peer.
Reverse proxies append to `X-Forwarded-For` but forward other client Reverse proxies append to `X-Forwarded-For` but forward other client
@@ -162,6 +182,8 @@ Two operator requirements follow:
makes all three limits, including the unauthenticated webhook makes all three limits, including the unauthenticated webhook
receiver, silently bypassable by every client in the block. receiver, silently bypassable by every client in the block.
#### Sessions
Sessions are bounded by two independent clocks, and end at whichever Sessions are bounded by two independent clocks, and end at whichever
one runs out first: one runs out first:
@@ -231,22 +253,27 @@ docker run -d \
The container runs as a non-root user (`webhooker`, UID 1000), exposes The container runs as a non-root user (`webhooker`, UID 1000), exposes
port 8080, and includes a health check against port 8080, and includes a health check against
`/.well-known/healthcheck`. The `/var/lib/webhooker` volume holds all `/.well-known/healthcheck`. The `/var/lib/webhooker` volume holds all
SQLite databases: the main application database (`webhooker.db`) and SQLite databases: the main application database (`webhooker.db`), the
the per-webhook event databases (`events-{uuid}.db`). Mount this as a per-webhook event databases (`events-{uuid}.db`), and any archive
persistent volume to preserve data across container restarts. databases written by `database` targets (`archive-{uuid}.db`). Mount
this as a persistent volume to preserve data across container
restarts.
## Entrypoints ## Entrypoints
This repository adheres to the This repository adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all) [Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard: normalized scripts in `script/` are the entrypoints for the standard: normalized scripts in `script/` are the entrypoints for the
development workflow, and the Makefile targets are thin shims that call development workflow. Ten of the Makefile's sixteen targets are thin
them. We provide: shims that call them; `build`, `run`, `dev`, `deps`, `clean` and `css`
are inline commands with no script behind them. We provide:
- `script/bootstrap` — install all dependencies (idempotent) - `script/bootstrap` — install all dependencies (idempotent)
- `script/setup` — make a fresh clone ready for development - `script/setup` — make a fresh clone ready for development
(bootstrap, then install-precommit) (bootstrap, then install-precommit)
- `script/projectname` — output the project name ("webhooker") - `script/projectname` — output the project name ("webhooker")
- `script/fetch-assets` — download the third-party browser assets into
`static/`, verifying each against its pinned sha256
- `script/test` — run the test suite - `script/test` — run the test suite
- `script/lint` — run golangci-lint - `script/lint` — run golangci-lint
- `script/fmt` — format all code (writes) - `script/fmt` — format all code (writes)
@@ -260,6 +287,27 @@ them. We provide:
- `script/install-precommit` — install the git pre-commit hook that - `script/install-precommit` — install the git pre-commit hook that
runs `script/precommit` runs `script/precommit`
## Third-party browser assets
The web UI serves one third-party script, Alpine.js. It is **not** committed:
a minified bundle in the tree is unreviewable, and `REPO_POLICIES.md` bars
both committed build artifacts and unpinned external references.
Instead `script/fetch-assets` downloads it from a pinned URL, checks the
download against a hardcoded sha256, and installs it under `static/`. The
sha256 of every installed asset is recorded in `static/vendor.sha256`, and
`static/vendor_test.go` re-hashes the bytes `go:embed` put in the binary
against that manifest — so the pin is enforced on what actually ships, not
merely written down. Any mismatch fails the build.
`make bootstrap` runs the fetch for local development, and the Dockerfile
runs it in the build stage; `.gitignore` and `.dockerignore` keep the
artifact out of both the repo and the build context.
To move to a new version: update the version, URL, and tarball sha256 in
`script/fetch-assets` and the asset sha256 in `static/vendor.sha256`, then
run `make assets && make check`.
## Rationale ## Rationale
Webhook integrations between services are inherently fragile. The Webhook integrations between services are inherently fragile. The
@@ -330,7 +378,11 @@ It uses:
- **[gorilla/csrf](https://github.com/gorilla/csrf)** for CSRF - **[gorilla/csrf](https://github.com/gorilla/csrf)** for CSRF
protection (cookie-based double-submit tokens) protection (cookie-based double-submit tokens)
- **[go-chi/httprate](https://github.com/go-chi/httprate)** for - **[go-chi/httprate](https://github.com/go-chi/httprate)** for
per-IP login rate limiting (sliding window counter) sliding-window rate limiting of the login, password-change and
webhook receiver endpoints. The bucket is per client IP only when
`TRUSTED_PROXIES` names the reverse proxy; unset, every client
behind that proxy shares one bucket per limit (see
[Rate Limiting](#rate-limiting))
- **[Prometheus](https://prometheus.io)** for metrics, served at - **[Prometheus](https://prometheus.io)** for metrics, served at
`/metrics` behind basic auth `/metrics` behind basic auth
- **[Sentry](https://sentry.io)** for optional error reporting - **[Sentry](https://sentry.io)** for optional error reporting
@@ -348,7 +400,7 @@ The codebase uses consistent naming throughout (rename completed in
### Data Model ### Data Model
webhooker's data model has eight entities organized into two tiers: the webhooker's data model has nine entities organized into two tiers: the
**application tier** (user and webhook configuration) and the **event **application tier** (user and webhook configuration) and the **event
tier** (event ingestion, delivery, and logging). tier** (event ingestion, delivery, and logging).
@@ -440,9 +492,25 @@ days (`database.RetentionForeverDays`). The retention reaper recognises
that sentinel and skips the webhook entirely, and the web UI displays that sentinel and skips the webhook entirely, and the web UI displays
such a webhook's retention as "forever" rather than as a day count. such a webhook's retention as "forever" rather than as a day count.
A *finite* retention is capped at `database.MaxFiniteRetentionDays` Submitted `retention_days` values therefore fall into three bands, not
(106751 days, about 292 years), and a larger one is rejected with a two:
400. The cap is not arbitrary: the reaper computes its cutoff as a
- `1` up to `database.MaxFiniteRetentionDays` (106751 days, about 292
years) is accepted as a finite retention.
- Above that ceiling but below the retain-forever sentinel of 365000
(`database.RetentionForeverDays`) is rejected with a 400. This is the
band the cap exists for.
- `0`, and `365000` or above, are accepted and mean retain forever,
collapsing to the sentinel — `0` in `Webhook.BeforeSave`, the large
values in `parseRetentionDays`. The large values are not out of
range: the edit form pre-fills the sentinel for a retain-forever
webhook, so submitting that form back unchanged has to keep meaning
"forever".
A negative value is in none of the three: `parseRetentionDays` rejects
it with a 400 before `BeforeSave` ever sees it.
The cap is not arbitrary: the reaper computes its cutoff as a
`time.Duration`, an int64 nanosecond count, and a longer period `time.Duration`, an int64 nanosecond count, and a longer period
overflows it. An overflowed cutoff lands in the future, where it overflows it. An overflowed cutoff lands in the future, where it
matches every row, so the sweep would delete every event the webhook matches every row, so the sweep would delete every event the webhook
@@ -460,7 +528,7 @@ the full request and creates an Event.
| -------------- | ------- | ----------- | | -------------- | ------- | ----------- |
| `id` | UUID | Primary key | | `id` | UUID | Primary key |
| `webhook_id` | UUID | Foreign key → Webhook | | `webhook_id` | UUID | Foreign key → Webhook |
| `path` | string | Unique URL path (UUID-based, e.g. `/webhook/{uuid}`) | | `path` | string | Unique bare UUID, generated at creation. The `/webhook/` prefix is route only and is not stored: the receiver matches this column against the raw `{uuid}` path segment |
| `description` | string | Optional description | | `description` | string | Optional description |
| `active` | boolean | Whether this entrypoint accepts events (default: true) | | `active` | boolean | Whether this entrypoint accepts events (default: true) |
@@ -484,8 +552,8 @@ events should be forwarded.
| `type` | TargetType | One of: `http`, `slack`, `database`, `log` | | `type` | TargetType | One of: `http`, `slack`, `database`, `log` |
| `active` | boolean | Whether deliveries are enabled (default: true) | | `active` | boolean | Whether deliveries are enabled (default: true) |
| `config` | JSON text | Type-specific configuration | | `config` | JSON text | Type-specific configuration |
| `max_retries` | integer | Maximum retry attempts for HTTP targets (0 = fire-and-forget, >0 = retries with backoff) | | `max_retries` | integer | Maximum retry attempts for `http` and `slack` targets (0 = fire-and-forget, >0 = retries with backoff and a circuit breaker). Ignored by `database` and `log` targets |
| `max_queue_size` | integer | Maximum queued deliveries (for HTTP targets with retries) | | `max_queue_size` | integer | Stored and shown on the target's detail view, but not enforced anywhere yet: nothing in the delivery engine consults it. Queue depth is set by the two fixed 10,000-entry channels |
**Relations:** Belongs to Webhook. Has many Deliveries. **Relations:** Belongs to Webhook. Has many Deliveries.
@@ -498,6 +566,11 @@ events should be forwarded.
greater than 0, failed deliveries are retried with exponential backoff greater than 0, failed deliveries are retried with exponential backoff
up to `max_retries` attempts, protected by a per-target circuit up to `max_retries` attempts, protected by a per-target circuit
breaker. breaker.
- **`slack`** — Post the event as a formatted message to a
Slack-compatible incoming webhook URL (`webhookUrl` in `config`). It
is built on the same HTTP core as `http` and honours `max_retries`
identically, circuit breaker included. See the Slack target section
under "Per-Webhook Event Databases" for the message format.
- **`database`** — Archive the full event as a row into a separate - **`database`** — Archive the full event as a row into a separate
per-webhook archive database (`archive-{webhookID}.db`) for long-term per-webhook archive database (`archive-{webhookID}.db`) for long-term
retention, with an optional creation-validated expiry (default: keep retention, with an optional creation-validated expiry (default: keep
@@ -534,7 +607,7 @@ data for auditing and for the planned replay capability.
| `id` | UUID | Primary key | | `id` | UUID | Primary key |
| `webhook_id` | UUID | Foreign key → Webhook | | `webhook_id` | UUID | Foreign key → Webhook |
| `entrypoint_id` | UUID | Foreign key → Entrypoint | | `entrypoint_id` | UUID | Foreign key → Entrypoint |
| `method` | string | HTTP method (POST, PUT, etc.) | | `method` | string | HTTP method of the captured request. Always `POST`: the receiver answers every other method with 405 before an Event is created |
| `headers` | JSON | Complete request headers | | `headers` | JSON | Complete request headers |
| `body` | text | Raw request body | | `body` | text | Raw request body |
| `content_type` | string | Content-Type header value | | `content_type` | string | Content-Type header value |
@@ -589,7 +662,9 @@ retries) is individually logged for full observability.
#### Common Fields #### Common Fields
All entities include these fields from `BaseModel`: Every entity except `Setting` includes these fields from `BaseModel`.
`Setting` is a bare key-value row with no `id`, no timestamps and no
soft delete:
| Field | Type | Description | | Field | Type | Description |
| ------------ | --------- | ----------- | | ------------ | --------- | ----------- |
@@ -635,7 +710,7 @@ handles connection pooling, lazy opening, migrations, and cleanup.
This separation provides: This separation provides:
- **Isolation** — a high-volume webhook won't cause lock contention or - **Isolation** — a high-volume webhook won't cause lock contention or
WAL bloat affecting the main application or other webhooks. journal growth affecting the main application or other webhooks.
- **Independent lifecycle** — event databases can be independently - **Independent lifecycle** — event databases can be independently
backed up, archived, rotated, or size-limited without impacting the backed up, archived, rotated, or size-limited without impacting the
application. application.
@@ -645,9 +720,12 @@ This separation provides:
- **Per-webhook retention** — the `retention_days` field on each webhook - **Per-webhook retention** — the `retention_days` field on each webhook
controls automatic cleanup of old events in that webhook's database controls automatic cleanup of old events in that webhook's database
only, or disables cleanup entirely when set to `0` (retain forever). only, or disables cleanup entirely when set to `0` (retain forever).
- **Performance** — each webhook's database has its own WAL, its own - **Performance** — each webhook's database has its own page cache and
page cache, and its own lock, so concurrent event ingestion across its own lock, so concurrent event ingestion across webhooks won't
webhooks won't contend. contend. No write-ahead log is involved: both DSNs are
`file:{path}?cache=shared&mode=rwc` and no `journal_mode` pragma is
ever issued, so every database runs on SQLite's default rollback
journal.
The **database target type** builds on this architecture to provide The **database target type** builds on this architecture to provide
long-term archiving, separate from the per-webhook event database (which long-term archiving, separate from the per-webhook event database (which
@@ -703,8 +781,9 @@ and other compatible services). Each message includes event metadata
pretty-printed in a code block. JSON payloads are automatically pretty-printed in a code block. JSON payloads are automatically
formatted with indentation for readability; non-JSON payloads are shown formatted with indentation for readability; non-JSON payloads are shown
as raw text. Large payloads are truncated to keep messages reasonable. as raw text. Large payloads are truncated to keep messages reasonable.
Config stores `webhook_url` — the Slack/Mattermost incoming webhook Config stores `webhookUrl` — the Slack/Mattermost incoming webhook
endpoint. endpoint. That is the JSON key; the error text for a missing one reads
`webhook_url is required`, which is the message, not the key.
The database uses the The database uses the
[modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) driver at [modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite) driver at
@@ -726,8 +805,9 @@ External Service
1. Look up Entrypoint by UUID 1. Look up Entrypoint by UUID
2. Capture full request as Event 2. Capture full request as Event
3. Create Delivery records for each active Target 3. Create Delivery records for each active Target
4. Build self-contained DeliveryTask structs 4. Build self-contained delivery.Task structs
(target config + event data inline for ≤16KB) (target config + event data inline for
bodies < 16 KiB)
5. Notify Engine via channel (no DB read needed) 5. Notify Engine via channel (no DB read needed)
│ │
▼ ▼
@@ -762,7 +842,7 @@ at any time, preventing goroutine explosions regardless of queue depth.
a delivery channel (new tasks from the webhook handler) and a retry a delivery channel (new tasks from the webhook handler) and a retry
channel (tasks from backoff timers). Both are buffered to 10,000. channel (tasks from backoff timers). Both are buffered to 10,000.
- **Fan-out via channel, not goroutines:** When an event arrives with - **Fan-out via channel, not goroutines:** When an event arrives with
multiple targets, each `DeliveryTask` is sent to the delivery channel. multiple targets, each `delivery.Task` is sent to the delivery channel.
Workers pick them up and process them — no goroutine-per-target. Workers pick them up and process them — no goroutine-per-target.
- **Worker goroutines:** A fixed number of worker goroutines select from - **Worker goroutines:** A fixed number of worker goroutines select from
both channels. Each worker processes one task at a time, then picks up both channels. Each worker processes one task at a time, then picks up
@@ -786,7 +866,12 @@ This means:
- **Independent results** — each worker records its own delivery result - **Independent results** — each worker records its own delivery result
in the per-webhook database without coordination. in the per-webhook database without coordination.
- **Graceful shutdown** — cancel the context, workers finish their - **Graceful shutdown** — cancel the context, workers finish their
current task and exit. `WaitGroup.Wait()` ensures clean shutdown. current task and exit. The stop hook waits for the pool via
`lifecycle.WaitForShutdown`, which bounds that wait by fx's stop
timeout rather than blocking forever on a wedged worker. On timeout
it logs at `ERROR` and returns an error, and the goroutines that
did not finish are still running — an unclean shutdown is reported
rather than hidden.
**Recovery paths:** **Recovery paths:**
@@ -814,12 +899,13 @@ remains stored in the per-webhook event database, there is no way to
redeliver it: manual redelivery is planned, not implemented (see redeliver it: manual redelivery is planned, not implemented (see
[TODO.md](TODO.md)). [TODO.md](TODO.md)).
### Circuit Breaker (HTTP Targets with Retries) ### Circuit Breaker (HTTP and Slack Targets with Retries)
HTTP targets with `max_retries` > 0 are protected by a **per-target circuit breaker** that `http` and `slack` targets with `max_retries` > 0 are protected by a
prevents hammering a down target with repeated failed delivery attempts. **per-target circuit breaker** that prevents hammering a down target
The circuit breaker is in-memory only and resets on restart (which is with repeated failed delivery attempts. The circuit breaker is
fine — startup recovery rescans the database anyway). in-memory only and resets on restart (which is fine — startup recovery
rescans the database anyway).
**States:** **States:**
@@ -855,10 +941,12 @@ fine — startup recovery rescans the database anyway).
- **Failure threshold:** 5 consecutive failures before opening - **Failure threshold:** 5 consecutive failures before opening
- **Cooldown:** 30 seconds in open state before probing - **Cooldown:** 30 seconds in open state before probing
**Scope:** Circuit breakers only apply to **HTTP targets with **Scope:** Circuit breakers apply to **`http` and `slack` targets with
`max_retries` > 0**. Fire-and-forget HTTP targets (`max_retries` == 0), `max_retries` > 0**. The Slack target is built on the same HTTP core
Slack targets, database targets (local operations), and log and hands its own `max_retries` to the same retry path, so it gets a
targets (stdout) do not use circuit breakers. breaker with the same 5-failure / 30-second defaults. Fire-and-forget
targets of either type (`max_retries` == 0), database targets (local
operations), and log targets (stdout) do not use circuit breakers.
When a circuit is open and a new delivery arrives, the engine marks the When a circuit is open and a new delivery arrives, the engine marks the
delivery as `retrying` and schedules a retry timer for after the delivery as `retrying` and schedules a retry timer for after the
@@ -874,9 +962,11 @@ unpredictable rates, and blanket limits shared with other routes would
cause legitimate deliveries to be dropped. cause legitimate deliveries to be dropped.
The receiver instead has its own dedicated abuse limit, scoped to the The receiver instead has its own dedicated abuse limit, scoped to the
`/webhook/{uuid}` route only and keyed per client IP per entrypoint: one `/webhook/{uuid}` route only and keyed per client IP per request path
misbehaving sender is throttled without affecting other senders of the (`httprate.KeyByEndpoint`): one misbehaving sender is throttled without
same entrypoint or the same sender's other entrypoints. The limit is affecting other senders of the same entrypoint or the same sender's
other entrypoints. Keying on the path rather than on the entrypoint
matters — see the aggregate limit below. The limit is
`RECEIVER_RATE_LIMIT` requests per minute (default 120, generous for `RECEIVER_RATE_LIMIT` requests per minute (default 120, generous for
legitimate webhook senders). Requests over the limit receive HTTP 429 legitimate webhook senders). Requests over the limit receive HTTP 429
with a `Retry-After` header. A set-but-invalid `RECEIVER_RATE_LIMIT` with a `Retry-After` header. A set-but-invalid `RECEIVER_RATE_LIMIT`
@@ -934,8 +1024,8 @@ opposite directions:
login bucket full, and the operator's own login returns HTTP 429 for login bucket full, and the operator's own login returns HTTP 429 for
as long as that trickle continues. A restart clears the in-memory as long as that trickle continues. A restart clears the in-memory
buckets and a resumed trickle re-locks them. Production deployments buckets and a resumed trickle re-locks them. Production deployments
must set `TRUSTED_PROXIES`; webhooker warns at startup when it is must set `TRUSTED_PROXIES`; webhooker warns at startup whenever it is
empty in `prod`. empty, in any environment.
Finer-grained per-webhook rate limits (configured in the web UI and Finer-grained per-webhook rate limits (configured in the web UI and
enforced in the webhook handler) can layer on top of this env-level enforced in the webhook handler) can layer on top of this env-level
@@ -947,17 +1037,17 @@ abuse limit later; they are tracked as future work.
| Method | Path | Description | | Method | Path | Description |
| ------ | --------------------------- | ----------- | | ------ | --------------------------- | ----------- |
| `GET` | `/` | Root redirect (authenticated → `/sources`, unauthenticated → `/pages/login`) | | `GET` | `/` | Root redirect, 303 (authenticated → `/sources`, unauthenticated → `/pages/login`) |
| `GET` | `/.well-known/healthcheck` | Health check (JSON: status, uptime, version) | | `GET` | `/.well-known/healthcheck` | Health check (JSON: `status`, `now`, `uptimeSeconds`, `uptimeHuman`, `version`, `appname`, `maintenanceMode`) |
| `GET` | `/s/*` | Static file serving (embedded CSS, JS) | | any | `/s/*` | Static file serving (embedded CSS, JS). Mounted for every method, not just `GET`/`HEAD`: chi's `Mount` registers all methods and `http.FileServer` special-cases only `HEAD` (by omitting the body), so a `POST` or `DELETE` to an asset is answered `200` with the file. Pinned by `TestStaticServesEveryMethod` |
| `ANY` | `/webhook/{uuid}` | Webhook receiver endpoint (accepts all methods) | | `POST` | `/webhook/{uuid}` | Webhook receiver endpoint. `POST` only — every other method is answered `405 Method Not Allowed` with `Allow: POST`. Rate limited (see [Rate Limiting](#rate-limiting)) |
#### Authentication Endpoints #### Authentication Endpoints
| Method | Path | Description | | Method | Path | Description |
| ------ | --------------- | ----------- | | ------ | --------------- | ----------- |
| `GET` | `/pages/login` | Login page | | `GET` | `/pages/login` | Login page (not rate limited; the limiter applies to POST only) |
| `POST` | `/pages/login` | Login form submission | | `POST` | `/pages/login` | Login form submission (5 per minute per bucket, then 429) |
| `POST` | `/pages/logout` | Logout (destroys session) | | `POST` | `/pages/logout` | Logout (destroys session) |
#### Authenticated Endpoints #### Authenticated Endpoints
@@ -965,6 +1055,7 @@ abuse limit later; they are tracked as future work.
| Method | Path | Description | | Method | Path | Description |
| ------ | ------------------------ | ----------- | | ------ | ------------------------ | ----------- |
| `GET` | `/user/{username}` | User profile page | | `GET` | `/user/{username}` | User profile page |
| `POST` | `/user/{username}/password` | Change the user's password (5 per minute per bucket, then 429) |
| `GET` | `/sources` | List user's webhooks | | `GET` | `/sources` | List user's webhooks |
| `GET` | `/sources/new` | Create webhook form | | `GET` | `/sources/new` | Create webhook form |
| `POST` | `/sources/new` | Create webhook submission | | `POST` | `/sources/new` | Create webhook submission |
@@ -974,13 +1065,17 @@ abuse limit later; they are tracked as future work.
| `POST` | `/source/{id}/delete` | Delete webhook | | `POST` | `/source/{id}/delete` | Delete webhook |
| `GET` | `/source/{id}/logs` | Webhook event logs | | `GET` | `/source/{id}/logs` | Webhook event logs |
| `POST` | `/source/{id}/entrypoints` | Add entrypoint to webhook | | `POST` | `/source/{id}/entrypoints` | Add entrypoint to webhook |
| `POST` | `/source/{id}/entrypoints/{entrypointID}/delete` | Delete an entrypoint |
| `POST` | `/source/{id}/entrypoints/{entrypointID}/toggle` | Enable or disable an entrypoint |
| `POST` | `/source/{id}/targets` | Add target to webhook | | `POST` | `/source/{id}/targets` | Add target to webhook |
| `POST` | `/source/{id}/targets/{targetID}/delete` | Delete a target |
| `POST` | `/source/{id}/targets/{targetID}/toggle` | Enable or disable a target |
#### Infrastructure Endpoints #### Infrastructure Endpoints
| Method | Path | Description | | Method | Path | Description |
| ------ | ---------- | ----------- | | ------ | ---------- | ----------- |
| `GET` | `/metrics` | Prometheus metrics (requires basic auth) | | `GET` | `/metrics` | Prometheus metrics, behind basic auth. The route is registered only when `METRICS_USERNAME` is set; otherwise it does not exist and returns 404 |
#### API (Planned) #### API (Planned)
@@ -994,8 +1089,10 @@ abuse limit later; they are tracked as future work.
| `GET` | `/api/v1/webhooks/{id}/events` | List events for webhook | | `GET` | `/api/v1/webhooks/{id}/events` | List events for webhook |
| `POST` | `/api/v1/events/{id}/redeliver`| Redeliver an event | | `POST` | `/api/v1/events/{id}/redeliver`| Redeliver an event |
API authentication will use API keys passed via `Authorization: Bearer None of these exist yet. `/api/v1` is mounted with no routes, so every
<key>` header. path under it returns 404 today. API authentication will use API keys
passed via `Authorization: Bearer <key>` header; no Bearer middleware
is implemented either.
### Package Layout ### Package Layout
@@ -1023,16 +1120,30 @@ webhooker/
│ │ ├── model_delivery_result.go # DeliveryResult entity (per-webhook DB) │ │ ├── model_delivery_result.go # DeliveryResult entity (per-webhook DB)
│ │ ├── model_apikey.go # APIKey entity │ │ ├── model_apikey.go # APIKey entity
│ │ ├── password.go # Argon2id hashing and verification │ │ ├── password.go # Argon2id hashing and verification
│ │ ├── retention.go # Retention reaper (per-webhook event expiry)
│ │ ├── testing.go # NewTestDatabase: wrapper for tests, no fx lifecycle
│ │ └── webhook_db_manager.go # Per-webhook DB lifecycle manager │ │ └── webhook_db_manager.go # Per-webhook DB lifecycle manager
│ ├── globals/ │ ├── globals/
│ │ └── globals.go # Build-time variables (appname, version, arch) │ │ └── globals.go # Build-time variables (appname, version, arch)
│ ├── delivery/ │ ├── delivery/
│ │ ├── engine.go # Event-driven delivery engine (channel + timer based) │ │ ├── engine.go # Event-driven delivery engine (channel + timer based)
│ │ ├── circuit_breaker.go # Per-target circuit breaker for HTTP targets with retries │ │ ├── circuit_breaker.go # Per-target circuit breaker for http/slack targets with retries
│ │ ├── target.go # Target interface, Task, Scheduler
│ │ ├── target_http.go # HTTP target (retries, circuit breaker)
│ │ ├── target_slack.go # Slack/Mattermost incoming-webhook target
│ │ ├── target_database.go # Database archive target
│ │ ├── target_database_archive.go # Archive file lifecycle and pruning
│ │ ├── target_log.go # Log target (stdout)
│ │ ├── target_config_view.go # Masked target config for templates
│ │ ├── archive_sweeper.go # Periodic pruning of idle archives
│ │ ├── url_mask.go # Strips credentials from *url.Error
│ │ └── ssrf.go # SSRF prevention (IP validation, safe HTTP transport) │ │ └── ssrf.go # SSRF prevention (IP validation, safe HTTP transport)
│ ├── lifecycle/
│ │ └── lifecycle.go # Shared fx start/stop hook helpers
│ ├── handlers/ │ ├── handlers/
│ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering │ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering
│ │ ├── auth.go # Login, logout handlers │ │ ├── auth.go # Login, logout handlers
│ │ ├── event_log_view.go # Event log projection, byte-capped in SQL
│ │ ├── healthcheck.go # Health check handler │ │ ├── healthcheck.go # Health check handler
│ │ ├── index.go # Index page handler │ │ ├── index.go # Index page handler
│ │ ├── profile.go # User profile handler │ │ ├── profile.go # User profile handler
@@ -1045,20 +1156,27 @@ webhooker/
│ ├── middleware/ │ ├── middleware/
│ │ ├── middleware.go # Logging, CORS, Auth, Metrics, MetricsAuth, SecurityHeaders, MaxBodySize │ │ ├── middleware.go # Logging, CORS, Auth, Metrics, MetricsAuth, SecurityHeaders, MaxBodySize
│ │ ├── csrf.go # CSRF protection middleware (gorilla/csrf) │ │ ├── csrf.go # CSRF protection middleware (gorilla/csrf)
│ │ └── ratelimit.go # Per-IP rate limiting middleware (go-chi/httprate) │ │ ├── ratelimit.go # Per-IP rate limiting middleware (go-chi/httprate)
│ │ └── testing.go # NewForTest: Middleware without the fx lifecycle
│ ├── server/ │ ├── server/
│ │ ├── server.go # Server struct, fx lifecycle, signal handling │ │ ├── server.go # Server struct, fx lifecycle, signal handling
│ │ ├── http.go # HTTP server setup with timeouts │ │ ├── http.go # HTTP server setup with timeouts
│ │ └── routes.go # All route definitions │ │ └── routes.go # All route definitions
│ └── session/ │ └── session/
│ └── session.go # Cookie-based session management │ ├── session.go # Cookie-based session management
│ └── testing.go # NewForTest: Session without the fx lifecycle
├── static/ ├── static/
│ ├── static.go # //go:embed directive │ ├── static.go # //go:embed directive
│ ├── css/style.css # Custom stylesheet (system font stack, card effects, layout) │ ├── css/input.css # Tailwind input, source for tailwind.css (make css)
│ └── js/app.js # Client-side JavaScript (minimal bootstrap) │ ├── css/tailwind.css # Generated stylesheet the pages load
├── templates/ # Go HTML templates (base, index, login, etc.) │ ├── css/style.css # Older hand-written stylesheet, no longer loaded
├── Dockerfile # Multi-stage: lint, build+test, then Alpine runtime │ ├── js/app.js # Progressive-enhancement copy-to-clipboard
├── Makefile # fmt, lint, test, check, build, docker targets │ ├── js/alpine.min.js # Alpine.js, fetched by script/fetch-assets, not committed
│ └── vendor.sha256 # Pinned hashes the fetched assets are verified against
├── templates/ # Go HTML templates (base, login, sources, etc.)
├── script/ # Scripts to Rule Them All entrypoints
├── Dockerfile # Three stages: lint, test+build, Alpine runtime
├── Makefile # 10 of 16 targets shim script/; 6 are inline
├── go.mod / go.sum ├── go.mod / go.sum
└── .golangci.yml # Linter configuration └── .golangci.yml # Linter configuration
``` ```
@@ -1074,21 +1192,27 @@ Components are wired via Uber fx in this order:
user seed user seed
5. `database.NewWebhookDBManager` — Per-webhook event database 5. `database.NewWebhookDBManager` — Per-webhook event database
lifecycle manager lifecycle manager
6. `healthcheck.New` — Health check service 6. `database.NewRetentionReaper` — Per-webhook event retention sweep
7. `session.New` — Cookie-based session manager (key from database) 7. `healthcheck.New` — Health check service
8. `handlers.New` — HTTP handlers 8. `session.New` — Cookie-based session manager (key from database)
9. `middleware.New` — HTTP middleware 9. `handlers.New` — HTTP handlers
10. `delivery.New` — Event-driven delivery engine 10. `middleware.New` — HTTP middleware
11. `delivery.Engine` → `handlers.DeliveryNotifier` — interface bridge 11. `delivery.New` — Event-driven delivery engine
12. `server.New` — HTTP server and router 12. `delivery.NewArchiveSweeper` — Periodic pruning of idle archives
13. `delivery.Engine` → `delivery.Notifier` — interface bridge
14. `delivery.Engine` → `delivery.WebhookEvictor` — interface bridge so
deleting a webhook releases its archive writer
15. `server.New` — HTTP server and router
The server starts via `fx.Invoke(func(*server.Server, *delivery.Engine) The server starts via `fx.Invoke(func(*server.Server, *delivery.Engine,
{})` which triggers the fx lifecycle hooks in dependency order. The *database.RetentionReaper, *delivery.ArchiveSweeper) {})`, which
`DeliveryNotifier` interface allows the webhook handler to send triggers the fx lifecycle hooks in dependency order. The
self-contained `DeliveryTask` slices to the engine without a direct `delivery.Notifier` interface allows the webhook handler to send
self-contained `delivery.Task` slices to the engine without a direct
package dependency. Each task carries all target config and event data package dependency. Each task carries all target config and event data
inline (for bodies ≤16KB), so the engine can deliver without reading inline (for bodies under 16 KiB, `delivery.MaxInlineBodySize`), so the
from any database — it only writes to record results. engine can deliver without reading from any database — it only writes
to record results.
### Middleware Stack ### Middleware Stack
@@ -1114,16 +1238,32 @@ CSRF middleware in every one of those route groups, because
gorilla/csrf parses the form; if the cap were installed after it, form gorilla/csrf parses the form; if the cap were installed after it, form
parsing would run under net/http's 10 MB default and the 1 MB limit parsing would run under net/http's 10 MB default and the 1 MB limit
would never apply. A request that declares a `Content-Length` over the would never apply. A request that declares a `Content-Length` over the
limit is answered with `413 Request Entity Too Large` before any other limit is answered with `413 Request Entity Too Large` without its body
middleware or handler runs; a chunked request, or one that lies about being read and without reaching CSRF, the route group's remaining
its length, is hard-capped by `http.MaxBytesReader` and fails middleware, or the handler. It is not rejected before *any* other
downstream at form-parse time. middleware, though: the global entries listed above all run first, so
such a request is still logged and given the security headers — and
counted in the metrics, on a deployment where `METRICS_USERNAME` is
set and the Metrics middleware is therefore registered at all. The
rejection itself is logged at `WARN` with the method, path and
declared length. A chunked request, or
one that lies about its length, is hard-capped by
`http.MaxBytesReader` and fails downstream at form-parse time.
Those same four route groups then apply **CSRF** and **NoCache**
(`Cache-Control: no-store`, `Pragma: no-cache`), and every group except
`/pages` applies **RequireAuth**. The rate limiters are per-route
rather than global: **LoginRateLimit** on `/pages/login`,
**PasswordChangeRateLimit** on `/user/{username}/password`, and
**ReceiverRateLimit** on `/webhook/{uuid}`.
### Authentication ### Authentication
- **Web UI:** Cookie-based sessions using gorilla/sessions with - **Web UI:** Cookie-based sessions using gorilla/sessions with
encrypted cookies. Sessions are configured with HttpOnly, SameSite encrypted cookies. Sessions are configured with HttpOnly, SameSite
Lax, and Secure (in production). Session lifetime is 7 days. Lax, and Secure (in production). Absolute session lifetime is 7 days,
with a sliding idle timeout on top of it (see
[Sessions](#sessions)).
- **API (planned):** API key authentication via `Authorization: Bearer` - **API (planned):** API key authentication via `Authorization: Bearer`
header. API keys are stored per-user with usage tracking header. API keys are stored per-user with usage tracking
(`last_used_at`). (`last_used_at`).
@@ -1155,33 +1295,53 @@ downstream at form-parse time.
IPs before connecting, preventing DNS rebinding attacks) IPs before connecting, preventing DNS rebinding attacks)
- **Login rate limiting** via [go-chi/httprate](https://github.com/go-chi/httprate): - **Login rate limiting** via [go-chi/httprate](https://github.com/go-chi/httprate):
sliding-window rate limiter on the login endpoint, 5 POST attempts sliding-window rate limiter on the login endpoint, 5 POST attempts
per minute per bucket, to slow brute-force attacks. The bucket is per per minute per bucket, to slow brute-force attacks. GET requests to
client IP only when `TRUSTED_PROXIES` names the reverse proxy; the login page are not limited. The password-change endpoint carries
unset, every client shares one bucket and the login becomes remotely the same 5-per-minute limit. The bucket is per client IP only when
deniable (see [Rate Limiting](#rate-limiting)) `TRUSTED_PROXIES` names the reverse proxy; unset, every client
shares one bucket and the login becomes remotely deniable (see
[Rate Limiting](#rate-limiting)). webhooker warns at startup
whenever `TRUSTED_PROXIES` is empty
- Prometheus metrics behind basic auth - Prometheus metrics behind basic auth
- Static assets embedded in binary (no filesystem access needed at - Static assets embedded in binary (no filesystem access needed at
runtime) runtime)
- Container runs as non-root user (UID 1000) - Container runs as non-root user (UID 1000)
- GORM soft deletes on all entities (data preserved for audit) - GORM soft deletes on every entity that carries `BaseModel`, which is
all of them but `Setting` (data preserved for audit)
### Docker ### Docker
The Dockerfile uses a multi-stage build: The Dockerfile uses a three-stage build. Each stage is pinned by
digest, and the two check stages are separate images so the linter's
version is fixed independently of the compiler's:
1. **Builder stage** (Debian-based `golang:1.24`) — installs 1. **Lint stage** (`golangci/golangci-lint:v2.12.2`, Debian-based) —
golangci-lint, downloads dependencies, copies source, runs `make installs `make`, downloads dependencies, copies the source, and runs
check` (format verification, linting, tests, compilation). `make fmt-check` then `make lint`.
2. **Runtime stage** (`alpine:3.21`) — copies the binary, creates the 2. **Builder stage** (`golang:1.26.1-bookworm`) — depends on the lint
`/var/lib/webhooker` directory for all SQLite databases, runs as stage passing (it copies a file from it), runs `script/fetch-assets`
non-root user, exposes port 8080, includes a health check. to download and verify the third-party browser assets, then runs
`make test` and `make build`, and finally rebuilds the binary with
`CGO_ENABLED=1` and static linking so it runs on musl.
3. **Runtime stage** (`alpine:3.21`) — copies the static binary,
creates the `/var/lib/webhooker` directory for all SQLite databases,
runs as the non-root `webhooker` user (UID 1000), exposes port 8080,
and includes a health check against `/.well-known/healthcheck`.
The builder uses Debian rather than Alpine because GORM's SQLite Both check stages use Debian rather than Alpine because
dialect pulls in CGO-dependent headers at compile time. The runtime `gorm.io/driver/sqlite` pulls in `mattn/go-sqlite3`, which needs CGO
binary is statically linked and runs on Alpine. and does not compile against musl. Only the final binary is statically
linked, which is what lets it run on the Alpine runtime image.
`docker build .` is the CI gate — if it passes, the code is formatted, `script/cibuild` — `docker build .` — is the CI gate: the four check
linted, tested, and compiled. targets run inside the image, so a build that succeeds is a repo that
is formatted, linted, tested and compiled. Only `script/cibuild` and
`script/docker` involve Docker. `script/lint`, and therefore
`make lint` and `make check`, run whatever `golangci-lint` is on the
host, which can be a different version from the pinned one — so the
container is the authoritative lint result
([issue #109](https://git.eeqj.de/sneak/webhooker/issues/109) tracks
routing local linting through it as well).
#### CI gate honesty #### CI gate honesty
@@ -1197,16 +1357,17 @@ the hash of the last commit that touched the build context, so:
`make fmt-check`, `make lint`, `make test`, and `make build`. A run `make fmt-check`, `make lint`, `make test`, and `make build`. A run
that reports success ran them. that reports success ran them.
- A docs-only commit leaves the fingerprint unchanged — `.dockerignore` - A docs-only commit leaves the fingerprint unchanged — `.dockerignore`
excludes `*.md` and `LICENSE` from the context anyway — so the image excludes `*.md`, `LICENSE` and `.editorconfig` from the context
replays from cache and costs seconds. anyway — so the image replays from cache and costs seconds.
The module download layer sits above `COPY . .` and stays cached either The module download layer sits above `COPY . .` and stays cached either
way. way.
The workflow's first step covers a second way the gate lied: Gitea A separate workflow step, run before the fingerprint is written, covers
cancels an in-flight run when a newer commit lands on the same branch a second way the gate lied: Gitea cancels an in-flight run when a newer
and records that cancellation as a `failure` status, marking a commit commit lands on the same branch and records that cancellation as a
red that was never tested. Cancellation is unconditional server-side for `failure` status, marking a commit red that was never tested.
Cancellation is unconditional server-side for
push events, so the superseding run rewrites the exact push events, so the superseding run rewrites the exact
`Has been cancelled` status to `skipped`. Genuine failures are never `Has been cancelled` status to `skipped`. Genuine failures are never
touched. touched.

22
TODO.md
View File

@@ -25,13 +25,17 @@ password change flow (#65), policy compliance (#6), pinned lint tooling
(#55), and fail-loud configuration parsing (#80). (#55), and fail-loud configuration parsing (#80).
`next` holds the completed 1.0.0 milestone: every issue in it is closed, `next` holds the completed 1.0.0 milestone: every issue in it is closed,
and it is verified green both by CI and by cache-defeated container and it is verified green by cache-defeated container runs
runs. The two were only made to mean the same thing this cycle — before (`docker build --no-cache-filter=lint --no-cache-filter=builder`). The
#119, a warm layer cache let the gate report success without executing CI status is not independently claimed here: a superseded run is
anything, and replayed the previous build's console log so the lie recorded as `skipped` and still rolls up green, so a commit status on
looked like a real run. Note: TODO.md was deliberately deleted from this `next` does not by itself evidence an executed check (#152). Before
repo in f9a9569 (2026-03-01, #6); its content was folded into the README #119, a warm layer cache also let the gate report success without
TODO section, which this draft reconstructs as of 2026-07-06. executing anything, and replayed the previous build's console log so
the lie looked like a real run. Note: `TODO.md` was deliberately
deleted from this repo in f9a9569 (2026-03-01, #6); its content was
folded into the README TODO section, which this draft reconstructs as
of 2026-07-06.
# Next Step # Next Step
@@ -190,7 +194,9 @@ rate-limit keys should bucket by `/64`).
- OpenAPI specification - OpenAPI specification
- Analytics dashboard: success rates, response times, volume - Analytics dashboard: success rates, response times, volume
- A remember-me option at login - A remember-me option at login
- Password change and reset flow - Password reset flow for a forgotten password. The authenticated
password *change* flow already landed on `main` (#65); reset does not
exist
- Later, nice to have - Later, nice to have
- email delivery target type - email delivery target type
- SNS and S3 delivery targets - SNS and S3 delivery targets

44
internal/config/config.go
View File

@@ -422,33 +422,43 @@ func loadFromEnv() (*Config, error) {
}, nil }, nil
} }
// warnSharedRateLimitBucket logs a startup warning when a production // warnSharedRateLimitBucket logs a startup warning whenever
// deployment leaves TRUSTED_PROXIES empty. // TRUSTED_PROXIES is empty, in any environment.
// //
// With no trusted proxies every rate limiter keys on the connecting // With no trusted proxies every rate limiter keys on the connecting
// peer's address. A production deployment is required to run behind a // peer's address. Whether that is harmless or dangerous depends on
// TLS-terminating reverse proxy, and the peer is then that proxy for // what is in front of the process, which this code cannot observe:
// every request, so all clients share one bucket per limiter. The // with nothing in front, the peer is the client and the limits are
// per-client as intended; behind a reverse proxy the peer is the proxy
// for every request, so all clients share one bucket per limiter. The
// login limiter's bucket is the dangerous one: any remote client can // login limiter's bucket is the dangerous one: any remote client can
// keep it full, which denies the only administrative login to // keep it full, which denies the only administrative login to everyone
// everyone until the process restarts. // until the process restarts.
//
// The warning is deliberately not gated on WEBHOOKER_ENVIRONMENT. That
// variable defaults to dev, so gating on it would silence the warning
// for exactly the operator who forgot to configure the deployment —
// the case it exists to catch.
// //
// The default of trusting nobody is deliberate — trusting forwarded // The default of trusting nobody is deliberate — trusting forwarded
// headers from arbitrary peers lets any client choose its own bucket — // headers from arbitrary peers lets any client choose its own bucket —
// so this warns rather than failing startup or changing the key. // so this warns rather than failing startup or changing the key.
func (c *Config) warnSharedRateLimitBucket(log *slog.Logger) { func (c *Config) warnSharedRateLimitBucket(log *slog.Logger) {
if !c.IsProd() || len(c.TrustedProxies) > 0 { if len(c.TrustedProxies) > 0 {
return return
} }
log.Warn( log.Warn(
"TRUSTED_PROXIES is empty: rate limits key on the "+ "TRUSTED_PROXIES is empty: every rate limit keys on the "+
"connecting peer, so behind the reverse proxy a "+ "connecting peer's address. With nothing proxying to "+
"production deployment runs behind, every client "+ "this process that is the client itself and the limits "+
"shares one bucket per limit. Any remote client can "+ "are per-client as intended. Behind a reverse proxy the "+
"then keep the login limit full and deny the admin "+ "peer is the proxy on every request, so all clients "+
"login, the only administrative path, until restart. "+ "share one bucket per limit and any remote client can "+
"Set TRUSTED_PROXIES to your reverse proxy's address.", "keep the login limit full, denying the admin login — "+
"the only administrative path — until restart. If "+
"anything proxies to this process, set TRUSTED_PROXIES "+
"to its address.",
"environment", c.Environment, "environment", c.Environment,
"trustedProxies", len(c.TrustedProxies), "trustedProxies", len(c.TrustedProxies),
) )
@@ -491,6 +501,10 @@ func New(lc fx.Lifecycle, params ConfigParams) (*Config, error) {
"maintenanceMode", s.MaintenanceMode, "maintenanceMode", s.MaintenanceMode,
"dataDir", s.DataDir, "dataDir", s.DataDir,
"retentionSweepInterval", s.RetentionSweepInterval.String(), "retentionSweepInterval", s.RetentionSweepInterval.String(),
// Logged because a perfectly valid non-positive value here
// disables idle expiry entirely, and that is worth showing
// back to the operator.
"sessionIdleTimeout", s.SessionIdleTimeout.String(),
"receiverRateLimit", s.ReceiverRateLimit, "receiverRateLimit", s.ReceiverRateLimit,
"trustedProxies", len(s.TrustedProxies), "trustedProxies", len(s.TrustedProxies),
"hasSentryDSN", s.SentryDSN != "", "hasSentryDSN", s.SentryDSN != "",

37
internal/config/config_test.go
View File

@@ -628,10 +628,12 @@ func testTrustedProxiesSuccess(
} }
// TestSharedRateLimitBucketWarning covers the startup warning that // TestSharedRateLimitBucketWarning covers the startup warning that
// tells an operator their production deployment shares one rate-limit // tells an operator a deployment behind a reverse proxy shares one
// bucket between every client, which makes the admin login remotely // rate-limit bucket between every client, which makes the admin login
// deniable. It must fire when TRUSTED_PROXIES is empty in production // remotely deniable. It must fire whenever TRUSTED_PROXIES is empty,
// and stay quiet otherwise. // in any environment: WEBHOOKER_ENVIRONMENT defaults to dev, so gating
// on it would silence the warning for exactly the operator who never
// configured the deployment. It stays quiet once proxies are named.
func TestSharedRateLimitBucketWarning(t *testing.T) { func TestSharedRateLimitBucketWarning(t *testing.T) {
tests := []struct { tests := []struct {
name string name string
@@ -651,12 +653,19 @@ func TestSharedRateLimitBucketWarning(t *testing.T) {
expectWarning: false, expectWarning: false,
}, },
{ {
// Development is not required to run behind a // The default environment. An internet-exposed
// reverse proxy, so the shared bucket the warning // deployment whose operator never set
// describes is not the expected shape there. // WEBHOOKER_ENVIRONMENT lands here and has exactly
name: "dev without trusted proxies is quiet", // the exposure the warning announces.
name: "dev without trusted proxies warns",
environment: config.EnvironmentDev, environment: config.EnvironmentDev,
expectWarning: false, expectWarning: true,
},
{
name: "dev with trusted proxies is quiet",
environment: config.EnvironmentDev,
trustedProxies: cidrPrivateV4,
expectWarning: false,
}, },
} }
@@ -697,8 +706,14 @@ func TestSharedRateLimitBucketWarning(t *testing.T) {
assert.Contains(t, logged, `"level":"WARN"`) assert.Contains(t, logged, `"level":"WARN"`)
assert.Contains(t, logged, "TRUSTED_PROXIES") assert.Contains(t, logged, "TRUSTED_PROXIES")
assert.Contains(t, logged, "shares one bucket") assert.Contains(t, logged, "share one bucket")
assert.Contains(t, logged, "deny the admin login") assert.Contains(t, logged, "denying the admin login")
// The text must stay accurate for a developer with
// nothing in front of the process, where an empty
// list costs nothing.
assert.Contains(
t, logged, "nothing proxying to this process",
)
}) })
} }
} }

8
internal/delivery/target_config_view.go
View File

@@ -106,6 +106,12 @@ func slackConfigFields(configJSON string) []ConfigField {
// and its retry settings. Header values are not shown — they // and its retry settings. Header values are not shown — they
// routinely carry authorization tokens — only how many are // routinely carry authorization tokens — only how many are
// configured. // configured.
//
// The destination is masked to scheme and host by the same
// rule the Slack target uses. An HTTP target's destination is
// commonly a Slack, Discord or Teams incoming-webhook endpoint
// whose path segments are the credential, and the field takes
// an arbitrary URL, so no segment can be assumed non-secret.
func httpConfigFields(t *database.Target) []ConfigField { func httpConfigFields(t *database.Target) []ConfigField {
cfg, err := parseHTTPConfig(t.Config) cfg, err := parseHTTPConfig(t.Config)
if err != nil { if err != nil {
@@ -114,7 +120,7 @@ func httpConfigFields(t *database.Target) []ConfigField {
fields := []ConfigField{{ fields := []ConfigField{{
Label: "Destination URL", Label: "Destination URL",
Value: cfg.URL, Value: MaskURL(cfg.URL),
}} }}
if cfg.Timeout > 0 { if cfg.Timeout > 0 {

33
internal/delivery/target_config_view_test.go
View File

@@ -19,6 +19,7 @@ const (
viewExampleOrigin = "https://example.com" viewExampleOrigin = "https://example.com"
viewExampleHook = viewExampleOrigin + "/hook" viewExampleHook = viewExampleOrigin + "/hook"
viewMaskedOrigin = viewExampleOrigin + "/..."
viewUnavailable = "(unavailable)" viewUnavailable = "(unavailable)"
viewExpiryNever = "never" viewExpiryNever = "never"
) )
@@ -162,7 +163,7 @@ func TestNewTargetViews_HTTP(t *testing.T) {
assert.Equal( assert.Equal(
t, t,
map[string]string{ map[string]string{
"Destination URL": viewExampleHook, "Destination URL": viewMaskedOrigin,
"Timeout": "30s", "Timeout": "30s",
"Headers": "1 configured", "Headers": "1 configured",
"Max Retries": "5", "Max Retries": "5",
@@ -188,13 +189,41 @@ func TestNewTargetViews_HTTPFireAndForget(t *testing.T) {
assert.Equal( assert.Equal(
t, t,
map[string]string{ map[string]string{
"Destination URL": viewExampleHook, "Destination URL": viewMaskedOrigin,
"Max Retries": "0 (fire-and-forget)", "Max Retries": "0 (fire-and-forget)",
}, },
fieldMap(view.Config), fieldMap(view.Config),
) )
} }
// TestNewTargetViews_HTTPMasksDestinationURL proves the rule
// holds for the http target too: an http destination is
// routinely an incoming-webhook endpoint whose path segments
// are the credential, so none of them is shown.
func TestNewTargetViews_HTTPMasksDestinationURL(t *testing.T) {
t.Parallel()
view := viewFor(t, database.Target{
Type: database.TargetTypeHTTP,
Config: `{"url":"` + slackWebhookURL + `"}`,
})
fields := fieldMap(view.Config)
assert.Equal(
t,
"https://hooks.slack.com/...",
fields["Destination URL"],
)
for _, v := range fields {
assert.NotContains(t, v, slackSecretPath)
assert.NotContains(t, v, "T00000000")
assert.NotContains(t, v, "B00000000")
assert.NotContains(t, v, "XXXXXXXXXXXXXXXXXXXXXXXX")
}
}
func TestNewTargetViews_Database(t *testing.T) { func TestNewTargetViews_Database(t *testing.T) {
t.Parallel() t.Parallel()

120
internal/handlers/event_log_view.go Normal file
View File

@@ -0,0 +1,120 @@
package handlers
import (
"time"
"unicode/utf8"
)
// maxRenderedBodyBytes caps how many bytes of a stored event
// body reach the event log page. Bodies come from the
// unauthenticated receiver under the 1 MB ingest cap and
// renderTemplate buffers a whole page before writing it, so
// an uncapped page of paginationPerPage events is tens of
// megabytes of resident memory per concurrent viewer.
const maxRenderedBodyBytes = 8192
// eventLogColumns is the event log's projection. The casts to
// blob are load-bearing: they make substr and length count
// bytes rather than characters, so the cap bounds the page in
// bytes whatever the payload's encoding. Cutting in SQLite
// rather than in Go is the point of the projection — an
// oversized body never becomes a Go string at all.
const eventLogColumns = "id, created_at, method, content_type, " +
"substr(cast(body as blob), 1, ?) AS body, " +
"length(cast(body as blob)) AS body_bytes"
// EventLogView is the display-safe projection of an event for
// the event log page, alongside DeliveryView and TargetView.
// It carries a capped body plus the true stored size, so the
// page can mark a body as truncated without ever holding the
// whole thing.
type EventLogView struct {
ID string
CreatedAt time.Time
Method string
ContentType string
// Body holds at most maxRenderedBodyBytes bytes of the
// stored body.
Body string
// BodyBytes is the true size of the stored body.
BodyBytes int64
// BodyTruncated reports that the stored body was larger
// than the cap, so the page owes the reader a marker.
BodyTruncated bool
Deliveries []DeliveryView
}
// BodyShownBytes is how many body bytes the page is actually
// rendering, which the truncation marker reports beside the
// true size.
func (v EventLogView) BodyShownBytes() int {
return len(v.Body)
}
// eventLogRow is one row of the event log projection. Its
// body column arrives already cut to the cap by SQLite, with
// the true size beside it.
type eventLogRow struct {
ID string
CreatedAt time.Time
Method string
ContentType string
Body []byte
BodyBytes int64
}
// view projects a loaded row for rendering.
func (r *eventLogRow) view() EventLogView {
body := r.Body
truncated := r.BodyBytes > int64(len(body))
// Only a cut body can have been left mid-sequence by
// this query. A whole body is passed through exactly as
// stored, however malformed.
if truncated {
body = trimPartialRune(body)
}
return EventLogView{
ID: r.ID,
CreatedAt: r.CreatedAt,
Method: r.Method,
ContentType: r.ContentType,
Body: string(body),
BodyBytes: r.BodyBytes,
BodyTruncated: truncated,
}
}
// trimPartialRune drops a trailing UTF-8 sequence that the
// byte-wise cut left incomplete, so a multi-byte rune severed
// at the cap does not surface as a mojibake tail.
//
// Bytes that are merely invalid UTF-8 are left exactly as
// stored: this service receives binary payloads, and rewriting
// them would misreport what was delivered. The distinction is
// utf8.FullRune's — it reports a complete sequence for an
// invalid encoding too, since that decodes to a width-1 error
// rune, so only a valid prefix still waiting for its
// continuation bytes is removed. A tail with no rune start in
// its last utf8.UTFMax bytes cannot be an incomplete sequence
// either, and is likewise left alone.
func trimPartialRune(b []byte) []byte {
for i := len(b) - 1; i >= 0 && len(b)-i <= utf8.UTFMax; i-- {
if !utf8.RuneStart(b[i]) {
continue
}
if utf8.FullRune(b[i:]) {
return b
}
return b[:i]
}
return b
}

258
internal/handlers/event_log_view_test.go Normal file
View File

@@ -0,0 +1,258 @@
package handlers_test
import (
"net/http"
"net/http/httptest"
"strconv"
"strings"
"testing"
"unicode/utf8"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/session"
)
// bodyCap is the number of body bytes the event log page is
// allowed to render for one event.
const bodyCap = handlers.MaxRenderedBodyBytesForTest
// snowman is a three-byte rune, so a body of them straddles the
// byte-wise cut: bodyCap is not a multiple of three.
const snowman = "☃"
// seedEventWithBody records one event with the given body in the
// webhook's own database.
func seedEventWithBody(
t *testing.T,
dbMgr *database.WebhookDBManager,
webhookID string,
body string,
) {
t.Helper()
webhookDB, err := dbMgr.GetDB(webhookID)
require.NoError(t, err)
event := &database.Event{
WebhookID: webhookID,
Method: http.MethodPost,
Body: body,
ContentType: "application/octet-stream",
}
require.NoError(t, webhookDB.Omit(
clause.Associations,
).Create(event).Error)
}
// seedAndProject stores one body and returns the projection the
// event log page would be handed for it.
func seedAndProject(
t *testing.T,
body string,
) handlers.EventLogView {
t.Helper()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
seedEventWithBody(t, dbMgr, wh.ID, body)
views := h.LoadEventLogViewsForTest(
httptest.NewRecorder(), *wh, 1,
)
require.Len(t, views, 1)
return views[0]
}
// TestHandleSourceLogs_BoundsOversizeBody proves the rendered
// page is bounded by the cap rather than by the stored payload:
// the body here is 64 times the cap, and the ingest path would
// accept twice as much again.
func TestHandleSourceLogs_BoundsOversizeBody(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
const (
sentinel = "TAIL-SENTINEL-1f4a9c"
storedBytes = 512 * 1024
)
wh := seedWebhook(t, db)
seedEventWithBody(
t, dbMgr, wh.ID,
strings.Repeat("A", storedBytes-len(sentinel))+sentinel,
)
page := renderSourceLogsPage(t, h, sess, wh.ID)
// Nothing past the cap reaches the page, and the whole page
// stays far below the stored body it is reporting on.
assert.NotContains(t, page, sentinel)
assert.Less(t, len(page), 4*bodyCap)
// The marker states the true stored size, not the cut one.
assert.Contains(
t, page,
"showing "+strconv.Itoa(bodyCap)+
" of "+strconv.Itoa(storedBytes)+" bytes",
)
}
// TestHandleSourceLogs_SmallBodyRendersWhole guards the other
// side of the cap: a body under it is shown in full and carries
// no truncation marker.
func TestHandleSourceLogs_SmallBodyRendersWhole(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
seedEventWithBody(t, dbMgr, wh.ID, `{"kept":"whole"}`)
page := renderSourceLogsPage(t, h, sess, wh.ID)
assert.Contains(t, page, "&#34;kept&#34;")
assert.NotContains(t, page, "Body truncated for display")
}
// TestEventLogView_CutMidRune proves a multi-byte rune severed
// by the byte-wise cut is dropped rather than surfaced as a
// mojibake tail.
func TestEventLogView_CutMidRune(t *testing.T) {
t.Parallel()
body := strings.Repeat(snowman, 4096)
view := seedAndProject(t, body)
// bodyCap bytes hold bodyCap/3 whole snowmen and two bytes
// of the next one; those two are dropped.
whole := bodyCap / len(snowman)
assert.True(t, view.BodyTruncated)
assert.Equal(t, int64(len(body)), view.BodyBytes)
assert.Equal(t, strings.Repeat(snowman, whole), view.Body)
assert.True(t, utf8.ValidString(view.Body))
assert.LessOrEqual(t, len(view.Body), bodyCap)
}
// TestEventLogView_BinaryBodyLeftAsStored proves a binary
// payload is passed through byte for byte. Its tail is invalid
// UTF-8 however the cut falls, so repairing it would misreport
// what the sender delivered.
func TestEventLogView_BinaryBodyLeftAsStored(t *testing.T) {
t.Parallel()
raw := make([]byte, bodyCap+808)
for i := range raw {
// 0x80..0xBF: continuation bytes, never a rune start.
raw[i] = 0x80 | byte(i%0x40)
}
view := seedAndProject(t, string(raw))
assert.True(t, view.BodyTruncated)
assert.Equal(t, int64(len(raw)), view.BodyBytes)
assert.Equal(t, string(raw[:bodyCap]), view.Body)
assert.False(t, utf8.ValidString(view.Body))
}
// TestTrimPartialRune covers the distinction the cut repair
// turns on: an incomplete but valid sequence is dropped, while
// bytes that are merely invalid UTF-8 are left alone.
func TestTrimPartialRune(t *testing.T) {
t.Parallel()
cases := []struct {
name string
in []byte
want []byte
}{{
name: "complete ascii",
in: []byte("abc"),
want: []byte("abc"),
}, {
name: "complete multibyte",
in: []byte("ab" + snowman),
want: []byte("ab" + snowman),
}, {
name: "two byte rune cut",
in: []byte{'a', 0xC3},
want: []byte{'a'},
}, {
name: "three byte rune cut after one",
in: []byte{'a', 0xE2},
want: []byte{'a'},
}, {
name: "three byte rune cut after two",
in: []byte{'a', 0xE2, 0x98},
want: []byte{'a'},
}, {
name: "four byte rune cut",
in: []byte{'a', 0xF0, 0x9F, 0x92}, // U+1F4A9 cut
want: []byte{'a'},
}, {
name: "invalid start byte kept",
in: []byte{'a', 0xFF},
want: []byte{'a', 0xFF},
}, {
name: "orphan continuation bytes kept",
in: []byte{0x80, 0x81, 0x82, 0x83, 0x84},
want: []byte{0x80, 0x81, 0x82, 0x83, 0x84},
}, {
name: "truncated sequence followed by junk kept",
in: []byte{0xE2, 0x98, 0xFF},
want: []byte{0xE2, 0x98, 0xFF},
}, {
name: "empty",
in: []byte{},
want: []byte{},
}}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
assert.Equal(
t, tc.want,
handlers.TrimPartialRuneForTest(tc.in),
)
})
}
}

27
internal/handlers/export_test.go
View File

@@ -3,8 +3,35 @@ package handlers
import ( import (
"html/template" "html/template"
"net/http" "net/http"
"sneak.berlin/go/webhooker/internal/database"
) )
// MaxRenderedBodyBytesForTest exposes the event log's body cap
// to the handlers_test package.
const MaxRenderedBodyBytesForTest = maxRenderedBodyBytes
// TrimPartialRuneForTest exposes trimPartialRune for use in the
// handlers_test package.
func TrimPartialRuneForTest(b []byte) []byte {
return trimPartialRune(b)
}
// LoadEventLogViewsForTest exposes loadEventsWithDeliveries for
// use in the handlers_test package. Assertions on the projected
// body need the bytes as loaded: html/template rewrites invalid
// UTF-8 on the way out, so the rendered page cannot show whether
// a binary body survived the projection intact.
func (s *Handlers) LoadEventLogViewsForTest(
w http.ResponseWriter,
webhook database.Webhook,
page int,
) []EventLogView {
views, _ := s.loadEventsWithDeliveries(w, webhook, nil, page)
return views
}
// AddTemplateForTest registers a template under a page name so that // AddTemplateForTest registers a template under a page name so that
// the handlers_test package can drive the render path with a // the handlers_test package can drive the render path with a
// template of its own. // template of its own.

6
internal/handlers/handlers.go
View File

@@ -229,8 +229,10 @@ func (s *Handlers) renderTemplate(
// the response only once rendering has fully succeeded. Executing // the response only once rendering has fully succeeded. Executing
// straight into the ResponseWriter commits a partial body and a 200 // straight into the ResponseWriter commits a partial body and a 200
// status before a mid-render error can be reported, leaving no way // status before a mid-render error can be reported, leaving no way
// to serve a 500. These pages are small, so holding one in memory is // to serve a 500. Buffering makes a page's rendered size resident
// the right trade. // memory per concurrent viewer, so every page owes it a bound: the
// event log caps each stored body at maxRenderedBodyBytes for exactly
// this reason.
func (s *Handlers) executeTemplate( func (s *Handlers) executeTemplate(
w http.ResponseWriter, w http.ResponseWriter,
tmpl *template.Template, tmpl *template.Template,

43
internal/handlers/source_detail_test.go
View File

@@ -131,6 +131,47 @@ func TestHandleSourceDetail_MasksSlackWebhookURL(t *testing.T) {
assert.Contains(t, body, "https://hooks.slack.com/...") assert.Contains(t, body, "https://hooks.slack.com/...")
} }
// TestHandleSourceDetail_MasksHTTPDestinationURL is the
// regression test for the same leak reached through the http
// target: its destination is routinely an incoming-webhook
// endpoint whose path segments are the credential, so the
// rendered page must not contain them.
func TestHandleSourceDetail_MasksHTTPDestinationURL(
t *testing.T,
) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
)
app := newTestApp(t, &h, &sess, &db)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
seedConfiguredTarget(
t, db, wh.ID,
database.TargetTypeHTTP,
`{"url":"`+slackWebhookURL+`"}`,
)
body := renderSourceDetailPage(t, h, sess, wh.ID)
assert.NotContains(t, body, slackSecretPath)
assert.NotContains(t, body, "T00000000")
assert.NotContains(t, body, "B00000000")
assert.NotContains(
t, body, "XXXXXXXXXXXXXXXXXXXXXXXX",
)
assert.Contains(t, body, "Destination URL")
assert.Contains(t, body, "https://hooks.slack.com/...")
}
// TestHandleSourceDetail_RendersNamedTargetFields proves the // TestHandleSourceDetail_RendersNamedTargetFields proves the
// other target types render labelled fields rather than the // other target types render labelled fields rather than the
// stored blob. // stored blob.
@@ -172,7 +213,7 @@ func TestHandleSourceDetail_RendersNamedTargetFields(
body := renderSourceDetailPage(t, h, sess, wh.ID) body := renderSourceDetailPage(t, h, sess, wh.ID)
assert.Contains(t, body, "Destination URL") assert.Contains(t, body, "Destination URL")
assert.Contains(t, body, "https://example.com/hook") assert.Contains(t, body, "https://example.com/...")
assert.Contains(t, body, "Timeout") assert.Contains(t, body, "Timeout")
assert.Contains(t, body, "1 configured") assert.Contains(t, body, "1 configured")
assert.NotContains(t, body, "sekrit") assert.NotContains(t, body, "sekrit")

31
internal/handlers/source_management.go
View File

@@ -92,13 +92,6 @@ func parseRetentionDays(raw string, fallback int) (int, error) {
return v, nil return v, nil
} }
// EventWithDeliveries holds an event and its deliveries.
type EventWithDeliveries struct {
database.Event
Deliveries []DeliveryView
}
// DeliveryView is the display-safe projection of a delivery // DeliveryView is the display-safe projection of a delivery
// for the event log page. Its target is a TargetView, so the // for the event log page. Its target is a TargetView, so the
// stored configuration blob — which holds the target's // stored configuration blob — which holds the target's
@@ -815,16 +808,18 @@ func (h *Handlers) parsePage(r *http.Request) int {
} }
// loadEventsWithDeliveries loads paginated events and their // loadEventsWithDeliveries loads paginated events and their
// deliveries from the per-webhook database. // deliveries from the per-webhook database. Events come back
// as capped projections rather than database.Event rows: see
// eventLogColumns for why the cut happens in SQL.
func (h *Handlers) loadEventsWithDeliveries( func (h *Handlers) loadEventsWithDeliveries(
w http.ResponseWriter, w http.ResponseWriter,
webhook database.Webhook, webhook database.Webhook,
targetMap map[string]delivery.TargetView, targetMap map[string]delivery.TargetView,
page int, page int,
) ([]EventWithDeliveries, int64) { ) ([]EventLogView, int64) {
var totalEvents int64 var totalEvents int64
var result []EventWithDeliveries var result []EventLogView
if !h.dbMgr.DBExists(webhook.ID) { if !h.dbMgr.DBExists(webhook.ID) {
return result, totalEvents return result, totalEvents
@@ -845,23 +840,25 @@ func (h *Handlers) loadEventsWithDeliveries(
offset := (page - 1) * paginationPerPage offset := (page - 1) * paginationPerPage
var events []database.Event var rows []eventLogRow
webhookDB.Where( webhookDB.Model(&database.Event{}).Select(
eventLogColumns, maxRenderedBodyBytes,
).Where(
"webhook_id = ?", webhook.ID, "webhook_id = ?", webhook.ID,
).Order("created_at DESC").Offset(offset).Limit( ).Order("created_at DESC").Offset(offset).Limit(
paginationPerPage, paginationPerPage,
).Find(&events) ).Find(&rows)
result = make([]EventWithDeliveries, len(events)) result = make([]EventLogView, len(rows))
for i := range events { for i := range rows {
result[i].Event = events[i] result[i] = rows[i].view()
var deliveries []database.Delivery var deliveries []database.Delivery
webhookDB.Where( webhookDB.Where(
"event_id = ?", events[i].ID, "event_id = ?", rows[i].ID,
).Find(&deliveries) ).Find(&deliveries)
result[i].Deliveries = newDeliveryViews( result[i].Deliveries = newDeliveryViews(

51
internal/server/routes_test.go
View File

@@ -24,6 +24,7 @@ import (
"sneak.berlin/go/webhooker/internal/middleware" "sneak.berlin/go/webhooker/internal/middleware"
"sneak.berlin/go/webhooker/internal/server" "sneak.berlin/go/webhooker/internal/server"
"sneak.berlin/go/webhooker/internal/session" "sneak.berlin/go/webhooker/internal/session"
"sneak.berlin/go/webhooker/static"
) )
// csrfCookieName is the cookie gorilla/csrf issues when it runs. Its // csrfCookieName is the cookie gorilla/csrf issues when it runs. Its
@@ -246,6 +247,56 @@ func (e *testEnv) storedHash(t *testing.T, username string) string {
return user.Password return user.Password
} }
// --- /s static group ---
// TestStaticServesEveryMethod pins what the static mount actually
// answers. chi's Mount registers the handler for all methods and
// http.FileServer only special-cases HEAD (by suppressing the body),
// so a POST or a DELETE to an asset is served the file rather than
// refused. The README documents this; the test is what keeps the two
// from drifting.
func TestStaticServesEveryMethod(t *testing.T) {
t.Parallel()
env := newTestEnv(t)
body, err := static.Static.ReadFile("js/app.js")
require.NoError(t, err)
require.NotEmpty(t, body)
for _, method := range []string{
http.MethodGet,
http.MethodHead,
http.MethodPost,
http.MethodPut,
http.MethodDelete,
} {
t.Run(method, func(t *testing.T) {
t.Parallel()
req := httptest.NewRequestWithContext(
context.Background(), method,
"/s/js/app.js", nil,
)
w := httptest.NewRecorder()
env.router.ServeHTTP(w, req)
assert.Equal(t, http.StatusOK, w.Code,
"static mount answers every method")
if method == http.MethodHead {
assert.Empty(t, w.Body.Bytes(),
"HEAD must not carry a body")
return
}
assert.Equal(t, body, w.Body.Bytes(),
"the asset itself is returned")
})
}
}
// --- /pages group --- // --- /pages group ---
// TestPagesLogin_OversizeBody_RejectedBeforeCSRF proves the cap runs // TestPagesLogin_OversizeBody_RejectedBeforeCSRF proves the cap runs

50
internal/server/static_assets_test.go Normal file
View File

@@ -0,0 +1,50 @@
package server_test
import (
"net/http"
"regexp"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/templates"
)
// TestBaseTemplateScriptsAreServed walks every /s/ script the base
// template loads on each page and fetches it through the real router.
// Alpine.js is fetched at build time rather than committed, so nothing
// in the repo guarantees it is present: this is the check that the page
// still gets the JavaScript it asks for.
func TestBaseTemplateScriptsAreServed(t *testing.T) {
t.Parallel()
// scriptSrc matches the src of every <script> tag pointing at the
// /s/ static mount.
scriptSrc := regexp.MustCompile(`<script[^>]+src="(/s/[^"]+)"`)
base, err := templates.Templates.ReadFile("base.html")
require.NoError(t, err)
matches := scriptSrc.FindAllStringSubmatch(string(base), -1)
require.NotEmpty(t, matches, "base.html should load scripts from /s/")
env := newTestEnv(t)
for _, m := range matches {
src := m[1]
t.Run(src, func(t *testing.T) {
t.Parallel()
w := env.get(src, nil)
require.Equalf(
t, http.StatusOK, w.Code,
"base.html loads %s but the server does not serve it", src,
)
assert.NotEmptyf(
t, w.Body.Bytes(), "%s is served but empty", src,
)
})
}
}

8
script/bootstrap
View File

@@ -5,7 +5,8 @@
# or apk (detected in that order); assumes NOTHING is present (not git, # or apk (detected in that order); assumes NOTHING is present (not git,
# make, or go). golangci-lint is packaged in nix, brew, and apk; on apt # make, or go). golangci-lint is packaged in nix, brew, and apk; on apt
# it is installed from a hash-verified GitHub release archive (never # it is installed from a hash-verified GitHub release archive (never
# curl | sh). # curl | sh). Finishes by running script/fetch-assets, which installs the
# hash-pinned third-party browser assets the repo does not commit.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
@@ -115,6 +116,11 @@ main() {
go mod download go mod download
# Third-party browser assets are not committed; fetch and verify them
# so a fresh clone can build and test.
if missing curl; then pkg_install curl curl curl curl; fi
"$ROOT/script/fetch-assets"
echo "bootstrap complete" echo "bootstrap complete"
} }

104
script/fetch-assets Executable file
View File

@@ -0,0 +1,104 @@
#!/bin/sh
# script/fetch-assets: download the third-party browser assets the web UI
# ships and install them under static/. Minified bundles are not committed
# (REPO_POLICIES.md: no build artifacts in version control), so the build
# fetches them here. Every download is verified against a hardcoded sha256
# before it is installed, and any mismatch aborts. Idempotent: an asset
# already present with its pinned hash is left alone.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# The sha256 of each installed asset lives in static/vendor.sha256, in
# sha256sum(1) format, with paths relative to static/. That file is the
# single source of truth: this script verifies against it, and
# static/vendor_test.go asserts the bytes embedded into the binary match
# it, so the hash cannot rot into a value nothing checks.
MANIFEST="static/vendor.sha256"
# Alpine.js 3.14.9, 2026-08-17. Fetched from registry.npmjs.org, the
# publisher of record; the jsDelivr and unpkg copies are mirrors of this
# same tarball. dist/cdn.min.js is the browser build Alpine publishes for
# a <script> tag.
ALPINE_VERSION="3.14.9"
ALPINE_URL="https://registry.npmjs.org/alpinejs/-/alpinejs-${ALPINE_VERSION}.tgz"
# sha256 of alpinejs-3.14.9.tgz
ALPINE_TARBALL_SHA256="97dad7c0c81e659cfc8e7700055da9770f8186187cb9a8a76efb57e00d5ce52a"
ALPINE_MEMBER="package/dist/cdn.min.js"
ALPINE_DEST="js/alpine.min.js"
sha256_of() {
if command -v sha256sum >/dev/null 2>&1; then
sha256sum "$1" | cut -d' ' -f1
else
shasum -a 256 "$1" | cut -d' ' -f1
fi
}
# expected_sha256 <path-relative-to-static>
expected_sha256() {
awk -v want="$1" '$2 == want { print $1; found = 1 }
END { if (!found) exit 1 }' "$ROOT/$MANIFEST"
}
# verify <file> <expected-sha256> <what>
verify() {
actual="$(sha256_of "$1")"
if [ "$actual" != "$2" ]; then
echo "fetch-assets: sha256 mismatch for $3" >&2
echo " expected: $2" >&2
echo " actual: $actual" >&2
exit 1
fi
}
# up_to_date <path-relative-to-static> <expected-sha256>
up_to_date() {
[ -f "$ROOT/static/$1" ] || return 1
[ "$(sha256_of "$ROOT/static/$1")" = "$2" ]
}
fetch_alpine() {
want="$(expected_sha256 "$ALPINE_DEST")"
if up_to_date "$ALPINE_DEST" "$want"; then
echo "fetch-assets: static/$ALPINE_DEST already at $want"
return 0
fi
echo "fetch-assets: fetching Alpine.js $ALPINE_VERSION from $ALPINE_URL"
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT INT TERM
curl -fsSL -o "$tmp/alpine.tgz" "$ALPINE_URL"
verify "$tmp/alpine.tgz" "$ALPINE_TARBALL_SHA256" "alpinejs-${ALPINE_VERSION}.tgz"
tar -xzOf "$tmp/alpine.tgz" "$ALPINE_MEMBER" >"$tmp/alpine.min.js"
verify "$tmp/alpine.min.js" "$want" "$ALPINE_MEMBER from alpinejs-${ALPINE_VERSION}.tgz"
mkdir -p "$(dirname "$ROOT/static/$ALPINE_DEST")"
cp "$tmp/alpine.min.js" "$ROOT/static/$ALPINE_DEST"
rm -rf "$tmp"
trap - EXIT INT TERM
echo "fetch-assets: installed static/$ALPINE_DEST ($want)"
}
# Re-check every manifest entry against what is now on disk, so an entry
# no script installs fails loudly instead of passing silently.
verify_manifest() {
while read -r want path; do
case "$want" in '' | '#'*) continue ;; esac
if [ ! -f "$ROOT/static/$path" ]; then
echo "fetch-assets: $MANIFEST lists static/$path, which is missing" >&2
exit 1
fi
verify "$ROOT/static/$path" "$want" "static/$path"
done <"$ROOT/$MANIFEST"
}
main() {
cd "$ROOT"
fetch_alpine
verify_manifest
echo "fetch-assets: all assets in $MANIFEST verified"
}
main "$@"

5
static/js/alpine.min.js vendored
View File

File diff suppressed because one or more lines are too long

1
static/vendor.sha256 Normal file
View File

@@ -0,0 +1 @@
3ed1eed252488921df65e363d6715deb04d7f92aaedb9e52199fdf73cb1e0ad3 js/alpine.min.js

92
static/vendor_test.go Normal file
View File

@@ -0,0 +1,92 @@
package static_test
import (
"bufio"
"crypto/sha256"
"encoding/hex"
"os"
"strings"
"testing"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/static"
)
const manifestPath = "vendor.sha256"
// fetchHint is appended to every failure here: the assets the manifest
// covers are fetched by the build, not committed, so a fresh clone that
// has not run script/fetch-assets fails this test and should be told why.
const fetchHint = "run `script/fetch-assets` (or `make assets`) to install " +
"the pinned third-party assets"
// TestVendoredAssetsMatchManifest asserts that every asset listed in
// static/vendor.sha256 is embedded in the binary with exactly the pinned
// bytes. script/fetch-assets verifies the same hashes at download time;
// this test verifies them again on what actually ships, so a build that
// skipped, cached, or subverted the fetch cannot produce a binary serving
// unpinned third-party JavaScript.
func TestVendoredAssetsMatchManifest(t *testing.T) {
t.Parallel()
entries := readManifest(t)
require.NotEmpty(t, entries, "%s lists no assets", manifestPath)
for path, want := range entries {
t.Run(path, func(t *testing.T) {
t.Parallel()
data, err := static.Static.ReadFile(path)
require.NoErrorf(
t, err,
"%s is listed in %s but is not embedded; %s",
path, manifestPath, fetchHint,
)
sum := sha256.Sum256(data)
got := hex.EncodeToString(sum[:])
require.Equalf(
t, want, got,
"embedded %s does not match its pinned sha256 in %s; %s",
path, manifestPath, fetchHint,
)
})
}
}
// readManifest parses static/vendor.sha256, which is in sha256sum(1)
// format with paths relative to static/.
func readManifest(t *testing.T) map[string]string {
t.Helper()
f, err := os.Open(manifestPath)
require.NoError(t, err, "opening %s", manifestPath)
defer func() { require.NoError(t, f.Close()) }()
entries := make(map[string]string)
scanner := bufio.NewScanner(f)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" || strings.HasPrefix(line, "#") {
continue
}
fields := strings.Fields(line)
require.Lenf(
t, fields, 2,
"%s: malformed entry %q, want \"<sha256> <path>\"",
manifestPath, line,
)
sum, path := fields[0], fields[1]
require.Lenf(t, sum, 64, "%s: %q is not a sha256", manifestPath, sum)
entries[path] = sum
}
require.NoError(t, scanner.Err(), "reading %s", manifestPath)
return entries
}

3
templates/source_logs.html
View File

@@ -37,6 +37,9 @@
<div x-show="open" x-cloak class="mt-3 p-3 bg-gray-50 rounded-md"> <div x-show="open" x-cloak class="mt-3 p-3 bg-gray-50 rounded-md">
<pre class="text-xs text-gray-700 overflow-x-auto whitespace-pre-wrap break-all">{{.Body}}</pre> <pre class="text-xs text-gray-700 overflow-x-auto whitespace-pre-wrap break-all">{{.Body}}</pre>
{{if .BodyTruncated}}
<p class="mt-2 text-xs text-gray-500">Body truncated for display: showing {{.BodyShownBytes}} of {{.BodyBytes}} bytes. The stored body is unchanged.</p>
{{end}}
</div> </div>
</div> </div>
{{else}} {{else}}