Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3502fac119 |
+5
-4
@@ -1,14 +1,15 @@
|
|||||||
|
# .git is deliberately NOT excluded: the build derives the version it stamps
|
||||||
|
# into the binary from it (script/version). Nor is any tracked file: git in
|
||||||
|
# the build would see it as deleted and mark the version -dirty. Only
|
||||||
|
# untracked files belong here.
|
||||||
|
#
|
||||||
# .ci-fingerprint is deliberately NOT excluded: it is the CI cache barrier
|
# .ci-fingerprint is deliberately NOT excluded: it is the CI cache barrier
|
||||||
# that keeps the check stages from replaying a cached pass. See the lint
|
# that keeps the check stages from replaying a cached pass. See the lint
|
||||||
# stage of the Dockerfile.
|
# stage of the Dockerfile.
|
||||||
.git/
|
|
||||||
bin/
|
bin/
|
||||||
# Extracted from 3p/ by `make assets` inside the build; a host copy is not
|
# Extracted from 3p/ by `make assets` inside the build; a host copy is not
|
||||||
# needed. The tarball in 3p/ must stay in the context.
|
# needed. The tarball in 3p/ must stay in the context.
|
||||||
static/js/alpine.min.js
|
static/js/alpine.min.js
|
||||||
*.md
|
|
||||||
LICENSE
|
|
||||||
.editorconfig
|
|
||||||
.env
|
.env
|
||||||
.env.*
|
.env.*
|
||||||
*.db
|
*.db
|
||||||
|
|||||||
@@ -28,12 +28,11 @@ jobs:
|
|||||||
run: script/ci-mark-superseded
|
run: script/ci-mark-superseded
|
||||||
|
|
||||||
- name: Fingerprint the build context
|
- name: Fingerprint the build context
|
||||||
# `.dockerignore` keeps docs out of the build context, so a docs-only
|
# Every commit that changes more than docs writes a new fingerprint
|
||||||
# commit legitimately replays the whole image from cache and stays
|
# into the context, which invalidates the `COPY . .` layer of both
|
||||||
# cheap. Every other commit writes a new fingerprint into the context,
|
# check stages: a commit that was never linted, formatted-checked,
|
||||||
# which invalidates the `COPY . .` layer of both check stages: a
|
# tested and built cannot report success from cache. Docs-only
|
||||||
# commit that was never linted, formatted-checked, tested and built
|
# commits rebuild too, since the context also carries `.git`.
|
||||||
# cannot report success from cache.
|
|
||||||
run: |
|
run: |
|
||||||
set -eu
|
set -eu
|
||||||
fp="$(git log -1 --format=%H -- . ':!*.md' ':!LICENSE' ':!.editorconfig')"
|
fp="$(git log -1 --format=%H -- . ':!*.md' ':!LICENSE' ':!.editorconfig')"
|
||||||
|
|||||||
+15
-7
@@ -38,8 +38,8 @@ FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a349228
|
|||||||
COPY --from=lint /src/go.sum /dev/null
|
COPY --from=lint /src/go.sum /dev/null
|
||||||
|
|
||||||
# jq is a runtime dependency of script/ci-mark-superseded, which the test
|
# jq is a runtime dependency of script/ci-mark-superseded, which the test
|
||||||
# suite executes.
|
# suite executes. git is what script/version derives the version with.
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends make curl ca-certificates jq && rm -rf /var/lib/apt/lists/*
|
RUN apt-get update && apt-get install -y --no-install-recommends make curl ca-certificates jq git && rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
WORKDIR /build
|
WORKDIR /build
|
||||||
|
|
||||||
@@ -55,14 +55,22 @@ COPY . .
|
|||||||
# from its tarball in 3p/.
|
# from its tarball in 3p/.
|
||||||
RUN make test
|
RUN make test
|
||||||
|
|
||||||
# Version stamped into the binary. .dockerignore excludes .git/, so
|
# Version stamped into the binary: the VERSION build arg when one is
|
||||||
# nothing in this stage can derive it: script/docker resolves it on the
|
# given, otherwise what script/version derives from the .git the build
|
||||||
# host and passes it in. The default is what a bare `docker build .`
|
# context carries, so any `docker build .` of a clone stamps its commit.
|
||||||
# with no --build-arg gets, and it names no tag the tree may not be at.
|
# With neither, as from a source tarball, it is "unknown".
|
||||||
#
|
#
|
||||||
# Declared here, below the test step, so a changed version does not
|
# Declared here, below the test step, so a changed version does not
|
||||||
# invalidate its cached layer.
|
# invalidate its cached layer.
|
||||||
ARG VERSION=unknown
|
ARG VERSION
|
||||||
|
|
||||||
|
# A context that carries .git must not stamp "unknown": that means git is
|
||||||
|
# missing here or refused to read the checkout, and the image could not be
|
||||||
|
# traced back to its commit.
|
||||||
|
RUN if [ -d .git ] && [ "$(make version VERSION="$VERSION")" = unknown ]; then \
|
||||||
|
echo "version is unknown although the build context carries .git" >&2; \
|
||||||
|
exit 1; \
|
||||||
|
fi
|
||||||
|
|
||||||
RUN make build VERSION="$VERSION"
|
RUN make build VERSION="$VERSION"
|
||||||
|
|
||||||
|
|||||||
@@ -4,12 +4,12 @@
|
|||||||
.DEFAULT_GOAL := check
|
.DEFAULT_GOAL := check
|
||||||
|
|
||||||
# Version stamped into the binary. Derived from git by script/version;
|
# Version stamped into the binary. Derived from git by script/version;
|
||||||
# override it (`make build VERSION=v1.2.3`) where git metadata is
|
# override it (`make build VERSION=v1.2.3`) to stamp a given value, which is
|
||||||
# unavailable, which is how the Dockerfile passes its build arg in.
|
# how the Dockerfile passes its build arg in.
|
||||||
VERSION ?= $(shell script/version)
|
VERSION ?= $(shell script/version)
|
||||||
|
|
||||||
# An empty override (`make build VERSION=`, or a `--build-arg VERSION=`
|
# An empty override (`make build VERSION=`, or the Dockerfile's `make build
|
||||||
# landing on the Dockerfile's `make build VERSION="$VERSION"`) means unset,
|
# VERSION="$VERSION"` when no VERSION build arg was given) means unset,
|
||||||
# exactly as it does in script/version -- stamping "" would leave the binary
|
# exactly as it does in script/version -- stamping "" would leave the binary
|
||||||
# reporting no version and the footer back on its "dev" fallback. `override`
|
# reporting no version and the footer back on its "dev" fallback. `override`
|
||||||
# is required: a plain assignment loses to the command-line definition it
|
# is required: a plain assignment loses to the command-line definition it
|
||||||
|
|||||||
@@ -142,7 +142,7 @@ TTY detection, and security headers are always applied.
|
|||||||
| `RETENTION_SWEEP_INTERVAL` | How often the retention reaper and archive sweeper run (Go duration, must be positive) | `1h` |
|
| `RETENTION_SWEEP_INTERVAL` | How often the retention reaper and archive sweeper run (Go duration, must be positive) | `1h` |
|
||||||
| `SESSION_IDLE_TIMEOUT` | Idle session timeout (Go duration) | `24h` |
|
| `SESSION_IDLE_TIMEOUT` | Idle session timeout (Go duration) | `24h` |
|
||||||
| `RECEIVER_RATE_LIMIT` | Receiver requests/minute per IP per entrypoint (10x that per IP across the route) | `120` |
|
| `RECEIVER_RATE_LIMIT` | Receiver requests/minute per IP per entrypoint (10x that per IP across the route) | `120` |
|
||||||
| `TRUSTED_PROXIES` | CIDRs whose forwarded headers are trusted. A set value replaces the default. If any client can reach webhooker, or the proxy in front of it, from an RFC 1918 source address, set it to the proxy's address alone. See [Trusted proxies](#trusted-proxies) | `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16` (RFC 1918) |
|
| `TRUSTED_PROXIES` | CIDRs whose forwarded headers are trusted (unset: all clients behind a proxy share one rate-limit bucket; a correct login password is never throttled either way) | `""` (none) |
|
||||||
| `ALLOWED_EGRESS_CIDRS` | CIDRs that delivery targets may reach despite the SSRF blocklist. Read [Allowing egress to your own network](#allowing-egress-to-your-own-network) before setting it | `""` (none) |
|
| `ALLOWED_EGRESS_CIDRS` | CIDRs that delivery targets may reach despite the SSRF blocklist. Read [Allowing egress to your own network](#allowing-egress-to-your-own-network) before setting it | `""` (none) |
|
||||||
|
|
||||||
#### Allowing egress to your own network
|
#### Allowing egress to your own network
|
||||||
@@ -157,21 +157,6 @@ public cloud metadata addresses: currently only `168.63.129.16`, Azure's
|
|||||||
WireServer, which serves an Azure VM its credentials. Because it is a
|
WireServer, which serves an Azure VM its credentials. Because it is a
|
||||||
public address, listing it in `ALLOWED_EGRESS_CIDRS` reopens it.
|
public address, listing it in `ALLOWED_EGRESS_CIDRS` reopens it.
|
||||||
|
|
||||||
That is all the default blocklist covers: the IPv4 private and reserved
|
|
||||||
ranges; of IPv6, only loopback (`::1`), unique local addresses
|
|
||||||
(`fc00::/7`) and link-local addresses (`fe80::/10`); and certain public
|
|
||||||
addresses. A public address belongs on the default blocklist only if it
|
|
||||||
hands credentials, user data or bootstrap material to whatever can reach
|
|
||||||
it, without the caller presenting anything. A provider's other public
|
|
||||||
addresses are not refused. IBM Cloud, for example, serves its package
|
|
||||||
mirrors, time servers and object storage on `161.26.0.0/16`, and the
|
|
||||||
private endpoints of its own cloud services on `166.8.0.0/14`. Neither
|
|
||||||
range hands out credentials that way: the token service among those
|
|
||||||
endpoints issues a token only in exchange for something the caller
|
|
||||||
presents, such as an API key. Reaching these services can be a
|
|
||||||
legitimate delivery, and every cloud has some, so a partial list would
|
|
||||||
promise coverage it does not give.
|
|
||||||
|
|
||||||
That default is also inconvenient for the thing webhooker is mostly
|
That default is also inconvenient for the thing webhooker is mostly
|
||||||
for: taking a public webhook and forwarding it to something on your own
|
for: taking a public webhook and forwarding it to something on your own
|
||||||
network. A container on the same Docker network, a box on `10.x`, a
|
network. A container on the same Docker network, a box on `10.x`, a
|
||||||
@@ -389,37 +374,41 @@ unlocked.
|
|||||||
`TRUSTED_PROXIES` is a comma-separated list of CIDR blocks (a bare
|
`TRUSTED_PROXIES` is a comma-separated list of CIDR blocks (a bare
|
||||||
address such as `192.168.1.7` is accepted and treated as a single
|
address such as `192.168.1.7` is accepted and treated as a single
|
||||||
host), for example `192.168.1.7, 2001:db8::5`. It decides whose
|
host), for example `192.168.1.7, 2001:db8::5`. It decides whose
|
||||||
`X-Forwarded-For` header the rate limiters believe, so it should cover
|
`X-Forwarded-For` header the rate limiters believe, so it should name
|
||||||
the addresses of your reverse proxies.
|
the addresses of your reverse proxies and nothing else.
|
||||||
|
|
||||||
`X-Forwarded-For` is honoured **only** when the connecting peer is
|
`X-Forwarded-For` is honoured **only** when the connecting peer is
|
||||||
inside one of these blocks; for every other peer the client identity is
|
inside one of these blocks; for every other peer the client identity is
|
||||||
the connection's own address and the header is ignored. Unset (or
|
the connection's own address and the header is ignored. The default is
|
||||||
empty), the list is the RFC 1918 private ranges: `10.0.0.0/8`,
|
the empty list, which trusts nobody — anything else would let any
|
||||||
`172.16.0.0/12` and `192.168.0.0/16`. A set value replaces the default
|
client pick its own rate limit bucket, minting a fresh one per request
|
||||||
entirely. A set but unparseable value aborts startup.
|
or draining someone else's. Set it to the address of your reverse
|
||||||
|
proxy, and to nothing wider. A set but unparseable value aborts
|
||||||
|
startup.
|
||||||
|
|
||||||
If any client can reach webhooker, or the proxy in front of it, from an
|
That default is safe against forged headers, but leaving it unset in
|
||||||
RFC 1918 source address (directly, or through anything that can
|
production has a cost you must know about. Production runs behind a
|
||||||
rewrite source addresses, such as NAT or a published container port),
|
TLS-terminating reverse proxy, so with `TRUSTED_PROXIES` unset every
|
||||||
set `TRUSTED_PROXIES` to the proxy's address alone, or every rate
|
request keys on the proxy's own address and all clients share a single
|
||||||
limit, the webhook receiver's included, can be bypassed by those
|
bucket per limit. The receiver limits become service-wide ceilings,
|
||||||
clients. The address to set is the `remoteIP` field of the
|
and the login endpoint's failure counting collapses onto one key, so a
|
||||||
`http request` log line for a request that came through the proxy.
|
stranger's wrong passwords throttle every other client's wrong
|
||||||
|
passwords.
|
||||||
Behind a proxy the list does not cover, every request keys on the
|
|
||||||
proxy's own address and all clients share a single bucket per limit.
|
|
||||||
The receiver limits become service-wide ceilings, and the login
|
|
||||||
endpoint's failure counting collapses onto one key, so a stranger's
|
|
||||||
wrong passwords throttle every other client's wrong passwords. Set
|
|
||||||
`TRUSTED_PROXIES` to that proxy's address to restore per-client
|
|
||||||
buckets.
|
|
||||||
|
|
||||||
What it cannot do is lock the operator out. The login endpoint
|
What it cannot do is lock the operator out. The login endpoint
|
||||||
verifies credentials **before** it consults any limit and charges only
|
verifies credentials **before** it consults any limit and charges only
|
||||||
failures, so a correct password is never throttled no matter how full
|
failures, so a correct password is never throttled no matter how full
|
||||||
the bucket is. See [Rate Limiting](#rate-limiting).
|
the bucket is. See [Rate Limiting](#rate-limiting).
|
||||||
|
|
||||||
|
The remedy is to set `TRUSTED_PROXIES` to your reverse proxy's
|
||||||
|
address, which restores per-client buckets. webhooker logs a warning
|
||||||
|
at startup whenever `TRUSTED_PROXIES` is empty, in every environment,
|
||||||
|
because behind a proxy every client shares one bucket in `dev` and
|
||||||
|
`prod` alike. The warning is informational when nothing proxies to the
|
||||||
|
process: with no proxy in front, the peer address is the client's own
|
||||||
|
and the buckets are already per-client. See
|
||||||
|
[Rate Limiting](#rate-limiting) for what each limit shares.
|
||||||
|
|
||||||
`X-Real-IP` and `True-Client-IP` are **never** read, from any peer.
|
`X-Real-IP` and `True-Client-IP` are **never** read, from any peer.
|
||||||
Reverse proxies append to `X-Forwarded-For` but forward other client
|
Reverse proxies append to `X-Forwarded-For` but forward other client
|
||||||
headers verbatim, so a single-valued header is client-controlled even
|
headers verbatim, so a single-valued header is client-controlled even
|
||||||
@@ -435,10 +424,20 @@ instead, since past such an entry the chain is not the shape assumed
|
|||||||
here. The peer address is likewise used when the header is absent or
|
here. The peer address is likewise used when the header is absent or
|
||||||
every hop in it is a trusted proxy.
|
every hop in it is a trusted proxy.
|
||||||
|
|
||||||
Your proxy must therefore **append** the peer address to
|
Two operator requirements follow:
|
||||||
`X-Forwarded-For` (nginx `$proxy_add_x_forwarded_for`, HAProxy
|
|
||||||
`option forwardfor`, Caddy and AWS ALB by default), and must append a
|
- Your proxy must **append** the peer address to `X-Forwarded-For`
|
||||||
bare address with no port.
|
(nginx `$proxy_add_x_forwarded_for`, HAProxy `option forwardfor`,
|
||||||
|
Caddy and AWS ALB by default), and must append a bare address with
|
||||||
|
no port.
|
||||||
|
- List proxy hosts **only**. Any address inside `TRUSTED_PROXIES`
|
||||||
|
chooses its own rate-limit key: its `X-Forwarded-For` is walked, so
|
||||||
|
it can name a different address on every request to get a fresh
|
||||||
|
bucket each time, or name another client's address to drain that
|
||||||
|
client's bucket. Never list a block that also covers clients — a
|
||||||
|
broad `10.0.0.0/8` on a network where clients live in the same range
|
||||||
|
makes all three limits, including the unauthenticated webhook
|
||||||
|
receiver, silently bypassable by every client in the block.
|
||||||
|
|
||||||
#### Sessions
|
#### Sessions
|
||||||
|
|
||||||
@@ -737,15 +736,10 @@ repository's `Dockerfile` and runs it. The app needs:
|
|||||||
- **Volume:** one host directory mounted at `/var/lib/webhooker`.
|
- **Volume:** one host directory mounted at `/var/lib/webhooker`.
|
||||||
- **Environment variables:**
|
- **Environment variables:**
|
||||||
- `WEBHOOKER_ENVIRONMENT=prod`
|
- `WEBHOOKER_ENVIRONMENT=prod`
|
||||||
- `TRUSTED_PROXIES`: unset, it is the RFC 1918 ranges. Set it to
|
- `TRUSTED_PROXIES`: your reverse proxy's address on that Docker
|
||||||
your reverse proxy's address alone if that address is outside
|
network. The `remoteIP` field of the `http request` log line for a
|
||||||
those ranges, or if any client can reach webhooker, or the proxy,
|
request that came through the proxy shows it; the health check's
|
||||||
from an RFC 1918 source address (directly, or through anything
|
own lines show `::1`. See [Trusted proxies](#trusted-proxies).
|
||||||
that can rewrite source addresses, such as NAT or a published
|
|
||||||
container port). The `remoteIP` field of the `http request` log
|
|
||||||
line for a request that came through the proxy shows that
|
|
||||||
address; the health check's own lines show `::1`. See
|
|
||||||
[Trusted proxies](#trusted-proxies).
|
|
||||||
- Leave `BIND_ADDRESS` and `DATA_DIR` unset: the image sets
|
- Leave `BIND_ADDRESS` and `DATA_DIR` unset: the image sets
|
||||||
`BIND_ADDRESS` to `0.0.0.0`, and `DATA_DIR` defaults to
|
`BIND_ADDRESS` to `0.0.0.0`, and `DATA_DIR` defaults to
|
||||||
`/var/lib/webhooker`.
|
`/var/lib/webhooker`.
|
||||||
@@ -808,16 +802,12 @@ reports.
|
|||||||
behind a proxy means the `X-Forwarded-Proto` header. The block below
|
behind a proxy means the `X-Forwarded-Proto` header. The block below
|
||||||
sets it; without it every request is read as plaintext and cookies
|
sets it; without it every request is read as plaintext and cookies
|
||||||
ship without `Secure`. See [Configuration](#configuration).
|
ship without `Secure`. See [Configuration](#configuration).
|
||||||
3. **Make sure `TRUSTED_PROXIES` covers the proxy's address.** For a
|
3. **Set `TRUSTED_PROXIES` to the proxy's address.** Unset, every rate
|
||||||
proxy it does not cover, every rate limiter keys on the proxy, so
|
limiter keys on the connecting peer, which behind a proxy is the
|
||||||
all clients share one bucket per limit. Unset, the list is the RFC
|
proxy on every request: all clients collapse into one global bucket
|
||||||
1918 ranges, which do not cover a proxy that reaches the binary
|
per limit and the receiver's per-IP limits become service-wide
|
||||||
itself over loopback (the binary bound to `127.0.0.1`). With the
|
ceilings. See [Trusted proxies](#trusted-proxies). List the proxy
|
||||||
image, the address to check is the `remoteIP` field of the
|
and nothing else.
|
||||||
`http request` log line for a request that came through the proxy.
|
|
||||||
If any client can reach webhooker, or the proxy, from an RFC 1918
|
|
||||||
source address, set the list to the proxy's address alone. See
|
|
||||||
[Trusted proxies](#trusted-proxies).
|
|
||||||
4. **Send `Host` as `$http_host`, not `$host`.** `$host` strips the
|
4. **Send `Host` as `$http_host`, not `$host`.** `$host` strips the
|
||||||
port. webhooker's Origin/Referer check compares against the host it
|
port. webhooker's Origin/Referer check compares against the host it
|
||||||
was given, so on any port other than 443 `$host` makes every form
|
was given, so on any port other than 443 `$host` makes every form
|
||||||
@@ -1133,13 +1123,21 @@ build itself.
|
|||||||
| Uncommitted changes | the above with a `-dirty` suffix |
|
| Uncommitted changes | the above with a `-dirty` suffix |
|
||||||
| No git metadata | `unknown` |
|
| No git metadata | `unknown` |
|
||||||
|
|
||||||
`unknown` is what a source tarball or a `docker build .` with no
|
The image derives it the same way, from the `.git` that the build
|
||||||
`--build-arg VERSION=...` reports. `.dockerignore` excludes `.git/`, so
|
context carries, so any `docker build .` of a clone stamps the commit it
|
||||||
the build context carries no git metadata and the image cannot derive
|
was built from; a shallow clone of one branch has no tags and stamps the
|
||||||
the version itself: `script/docker` (and so `make docker`) resolves it
|
short SHA. `.dockerignore` must therefore leave out neither `.git` nor
|
||||||
on the host and passes it in as the `VERSION` build arg. A build that
|
any tracked file, which git in the build would see as deleted, marking
|
||||||
reports `unknown` is a build nobody told what it was; it is not a
|
the version `-dirty`. A `VERSION` build arg (`--build-arg VERSION=...`)
|
||||||
failure, but it cannot be traced back to a commit.
|
takes precedence; `script/docker` (and so `make docker`) passes the one
|
||||||
|
`script/version` resolves on the host. The image build fails if its
|
||||||
|
context carries `.git` and the version still comes out `unknown`, which
|
||||||
|
means git in the build could not read the checkout.
|
||||||
|
|
||||||
|
`unknown` is what a source tarball, or a `docker build` with no `.git`
|
||||||
|
in its context and no `VERSION` build arg, reports. A build that reports
|
||||||
|
`unknown` is a build nobody told what it was; it is not a failure, but
|
||||||
|
it cannot be traced back to a commit.
|
||||||
|
|
||||||
`make version` prints what the current checkout would stamp, and
|
`make version` prints what the current checkout would stamp, and
|
||||||
`make build VERSION=v1.2.3` overrides it. An empty override — from
|
`make build VERSION=v1.2.3` overrides it. An empty override — from
|
||||||
@@ -1370,11 +1368,10 @@ It uses:
|
|||||||
- **[go-chi/httprate](https://github.com/go-chi/httprate)** for
|
- **[go-chi/httprate](https://github.com/go-chi/httprate)** for
|
||||||
sliding-window rate limiting of the password-change and webhook
|
sliding-window rate limiting of the password-change and webhook
|
||||||
receiver endpoints. The bucket is per client IP only when
|
receiver endpoints. The bucket is per client IP only when
|
||||||
`TRUSTED_PROXIES` covers the reverse proxy (by default it covers the
|
`TRUSTED_PROXIES` names the reverse proxy; unset, every client
|
||||||
RFC 1918 private ranges); otherwise every client behind that proxy
|
behind that proxy shares one bucket per limit. The login endpoint
|
||||||
shares one bucket per limit. The login endpoint counts failed
|
counts failed attempts itself instead, so that a correct password is
|
||||||
attempts itself instead, so that a correct password is never
|
never throttled (see [Rate Limiting](#rate-limiting))
|
||||||
throttled (see [Rate Limiting](#rate-limiting))
|
|
||||||
- **[Prometheus](https://prometheus.io)** for metrics, served at
|
- **[Prometheus](https://prometheus.io)** for metrics, served at
|
||||||
`/metrics` behind basic auth
|
`/metrics` behind basic auth
|
||||||
- **[Sentry](https://sentry.io)** for optional error reporting
|
- **[Sentry](https://sentry.io)** for optional error reporting
|
||||||
@@ -2545,44 +2542,47 @@ the tree is checked out: four checkouts have reported 3,959, 3,961,
|
|||||||
client-supplied field was cut, and that the shipped chain's stack
|
client-supplied field was cut, and that the shipped chain's stack
|
||||||
arrived uncut — never the numbers.
|
arrived uncut — never the numbers.
|
||||||
|
|
||||||
Every limiter here — receiver, login, password change, delivery replay
|
Every limiter here — receiver, login, and password change — identifies
|
||||||
and event resubmit — identifies the client the same way, through one
|
the client the same way, through one shared key function: the
|
||||||
shared key function: the connection's own address, unless the peer is
|
connection's own address, unless the peer is listed in
|
||||||
inside `TRUSTED_PROXIES`, in which case the forwarded client address is
|
`TRUSTED_PROXIES`, in which case the forwarded client address is used
|
||||||
used instead. That address becomes a bucket by family: IPv4 keys on
|
instead. That address becomes a bucket by family: IPv4 keys on the full
|
||||||
the full address, IPv6 on its `/64` prefix. A routed `/64` is the normal
|
address, IPv6 on its `/64` prefix. A routed `/64` is the normal
|
||||||
residential and mobile IPv6 allocation, so keying IPv6 per address would
|
residential and mobile IPv6 allocation, so keying IPv6 per address would
|
||||||
let one subscriber rotate source addresses and mint a fresh bucket per
|
let one subscriber rotate source addresses and mint a fresh bucket per
|
||||||
request, evading these limits at the network layer without spoofing
|
request, evading these limits at the network layer without spoofing
|
||||||
anything; the cost is that distinct clients inside one `/64` share a
|
anything; the cost is that distinct clients inside one `/64` share a
|
||||||
bucket. IPv4-mapped addresses (`::ffff:1.2.3.4`) key as the IPv4 address
|
bucket. IPv4-mapped addresses (`::ffff:1.2.3.4`) key as the IPv4 address
|
||||||
they carry. See [Trusted proxies](#trusted-proxies). When that variable
|
they carry. See [Trusted proxies](#trusted-proxies). Deployed without that
|
||||||
does not cover the reverse proxy, a client behind it shares one bucket
|
variable set, a client behind a reverse proxy shares one bucket with
|
||||||
with every other client behind the same proxy. Set `TRUSTED_PROXIES` to
|
every other client behind the same proxy. Set `TRUSTED_PROXIES` to the
|
||||||
the proxy's address to get per-client limits back. What the shared bucket
|
proxy's address to get per-client limits back. What the shared bucket
|
||||||
costs is not the same for every limiter, and the two cases pull in
|
costs is not the same for every limiter, and the two cases pull in
|
||||||
opposite directions:
|
opposite directions:
|
||||||
|
|
||||||
- For the **receiver** limits it costs throughput, which is the safe
|
- For the **receiver** limits it costs throughput, which is the safe
|
||||||
direction to be wrong in: sharing can only make a limit bind sooner,
|
direction to be wrong in: sharing can only make a limit bind sooner,
|
||||||
never let a sender past it. It matters more for the aggregate limit
|
never let a sender past it. It matters more for the aggregate limit
|
||||||
than for the per-entrypoint one: with every request keyed on the
|
than for the per-entrypoint one: with `TRUSTED_PROXIES` unset behind
|
||||||
proxy, the aggregate limit becomes a service-wide ceiling of 1200
|
the reverse proxy a production deployment is required to run behind,
|
||||||
requests per minute across all senders and all entrypoints, where the
|
every request keys on the proxy, so the aggregate limit becomes a
|
||||||
per-entrypoint limit's capacity still grows with the number of
|
service-wide ceiling of 1200 requests per minute across all senders
|
||||||
entrypoints.
|
and all entrypoints, where the per-entrypoint limit's capacity still
|
||||||
|
grows with the number of entrypoints. Any deployment with more than a
|
||||||
|
handful of busy entrypoints must set `TRUSTED_PROXIES`.
|
||||||
- For the **login and password-change** limits it costs precision, not
|
- For the **login and password-change** limits it costs precision, not
|
||||||
availability. Login failures from every client land in one counter,
|
availability. Login failures from every client land in one counter,
|
||||||
so a stranger's wrong passwords make the operator's own wrong
|
so a stranger's wrong passwords make the operator's own wrong
|
||||||
passwords answer `429` sooner; the operator's _correct_ password is
|
passwords answer `429` sooner; the operator's _correct_ password is
|
||||||
never affected, because it is never counted.
|
never affected, because it is never counted. Production deployments
|
||||||
|
should still set `TRUSTED_PROXIES`; webhooker warns at startup
|
||||||
|
whenever it is empty, in any environment.
|
||||||
|
|
||||||
#### The login endpoint
|
#### The login endpoint
|
||||||
|
|
||||||
The login `POST` is the one endpoint with no pre-emptive limiter in
|
The login `POST` is the one endpoint with no pre-emptive limiter in
|
||||||
front of it, and that is deliberate. A limiter that spends budget on
|
front of it, and that is deliberate. A limiter that spends budget on
|
||||||
arrival is a lockout wherever clients share one bucket, as they do
|
arrival is a lockout in this deployment shape: sharing one bucket, a
|
||||||
behind a reverse proxy that `TRUSTED_PROXIES` does not cover: a
|
|
||||||
stranger sending five POSTs a minute — about 0.08 requests per second,
|
stranger sending five POSTs a minute — about 0.08 requests per second,
|
||||||
from anywhere — keeps it permanently full, and the operator has no
|
from anywhere — keeps it permanently full, and the operator has no
|
||||||
second administrative path. So the handler inverts the order:
|
second administrative path. So the handler inverts the order:
|
||||||
@@ -2679,10 +2679,8 @@ re-fills both verification slots on its first two requests. The
|
|||||||
remedies are to block the source at the reverse proxy, or to
|
remedies are to block the source at the reverse proxy, or to
|
||||||
rate-limit `POST /pages/login` there — the one place a limit can be
|
rate-limit `POST /pages/login` there — the one place a limit can be
|
||||||
applied without reintroducing the lockout, because the proxy sees the
|
applied without reintroducing the lockout, because the proxy sees the
|
||||||
real client address. `TRUSTED_PROXIES` does not stop the saturation.
|
real client address. Setting `TRUSTED_PROXIES` does not stop the
|
||||||
The flood's source is in the proxy's access log: webhooker's own logs
|
saturation, but it makes the source visible in the failure logs.
|
||||||
record the proxy's address, not the client's (see
|
|
||||||
[Deployment behind a reverse proxy](#deployment-behind-a-reverse-proxy)).
|
|
||||||
|
|
||||||
Finer-grained per-webhook rate limits (configured in the web UI and
|
Finer-grained per-webhook rate limits (configured in the web UI and
|
||||||
enforced in the webhook handler) can layer on top of this env-level
|
enforced in the webhook handler) can layer on top of this env-level
|
||||||
@@ -2883,15 +2881,13 @@ Components are wired via Uber fx in this order:
|
|||||||
7. `healthcheck.New` — Health check service
|
7. `healthcheck.New` — Health check service
|
||||||
8. `session.New` — Cookie-based session manager (key from database)
|
8. `session.New` — Cookie-based session manager (key from database)
|
||||||
9. `handlers.New` — HTTP handlers
|
9. `handlers.New` — HTTP handlers
|
||||||
10. `metrics.NewRegistry` — The registry `/metrics` serves
|
10. `middleware.New` — HTTP middleware
|
||||||
11. `metrics.New` — The delivery collectors, registered on that registry
|
11. `delivery.New` — Event-driven delivery engine
|
||||||
12. `middleware.New` — HTTP middleware
|
12. `delivery.NewArchiveSweeper` — Periodic pruning of idle archives
|
||||||
13. `delivery.New` — Event-driven delivery engine
|
13. `delivery.Engine` → `delivery.Notifier` — interface bridge
|
||||||
14. `delivery.NewArchiveSweeper` — Periodic pruning of idle archives
|
14. `delivery.Engine` → `delivery.WebhookEvictor` — interface bridge so
|
||||||
15. `delivery.Engine` → `delivery.Notifier` — interface bridge
|
|
||||||
16. `delivery.Engine` → `delivery.WebhookEvictor` — interface bridge so
|
|
||||||
deleting a webhook releases its archive writer
|
deleting a webhook releases its archive writer
|
||||||
17. `server.New` — HTTP server and router
|
15. `server.New` — HTTP server and router
|
||||||
|
|
||||||
The server starts via `fx.Invoke(func(*server.Server, *delivery.Engine,
|
The server starts via `fx.Invoke(func(*server.Server, *delivery.Engine,
|
||||||
*database.RetentionReaper, *delivery.ArchiveSweeper) {})`, which
|
*database.RetentionReaper, *delivery.ArchiveSweeper) {})`, which
|
||||||
@@ -3042,9 +3038,10 @@ check, see [The login endpoint](#the-login-endpoint).
|
|||||||
It runs behind session auth, so only a client already holding a
|
It runs behind session auth, so only a client already holding a
|
||||||
valid session reaches it, and an operator throttled out of changing
|
valid session reaches it, and an operator throttled out of changing
|
||||||
a password can still log in. The bucket is per client IP only when
|
a password can still log in. The bucket is per client IP only when
|
||||||
`TRUSTED_PROXIES` covers the reverse proxy; otherwise every client
|
`TRUSTED_PROXIES` names the reverse proxy; unset, every client
|
||||||
shares one bucket, which costs precision rather than availability
|
shares one bucket, which costs precision rather than availability
|
||||||
(see [Rate Limiting](#rate-limiting))
|
(see [Rate Limiting](#rate-limiting)). webhooker warns at startup
|
||||||
|
whenever `TRUSTED_PROXIES` is empty
|
||||||
- Prometheus metrics behind basic auth
|
- Prometheus metrics behind basic auth
|
||||||
- Static assets embedded in binary (no filesystem access needed at
|
- Static assets embedded in binary (no filesystem access needed at
|
||||||
runtime)
|
runtime)
|
||||||
@@ -3176,8 +3173,9 @@ version is fixed independently of the compiler's:
|
|||||||
rebuilds the binary with `CGO_ENABLED=1` and static linking so it
|
rebuilds the binary with `CGO_ENABLED=1` and static linking so it
|
||||||
runs on musl. Both builds go through `make build`, the relink adding
|
runs on musl. Both builds go through `make build`, the relink adding
|
||||||
its `-extldflags` via `GO_LDFLAGS`, so neither can drop the `-X` that
|
its `-extldflags` via `GO_LDFLAGS`, so neither can drop the `-X` that
|
||||||
stamps the version. The version arrives as the `VERSION` build arg,
|
stamps the version. The version is the `VERSION` build arg if one is
|
||||||
since the context has no `.git` (see
|
given, otherwise derived from the `.git` in the context, and the
|
||||||
|
stage fails if a context with `.git` would stamp `unknown` (see
|
||||||
[Version stamping](#version-stamping)).
|
[Version stamping](#version-stamping)).
|
||||||
3. **Runtime stage** (`alpine:3.21`) — copies the static binary and
|
3. **Runtime stage** (`alpine:3.21`) — copies the static binary and
|
||||||
`deploy/docker-entrypoint.sh`, creates the `/var/lib/webhooker`
|
`deploy/docker-entrypoint.sh`, creates the `/var/lib/webhooker`
|
||||||
@@ -3209,16 +3207,17 @@ A layer cache lets `docker build .` exit 0 in seconds with the lint and
|
|||||||
test stages replayed rather than executed, which would make a green
|
test stages replayed rather than executed, which would make a green
|
||||||
check meaningless. The `check` workflow therefore writes
|
check meaningless. The `check` workflow therefore writes
|
||||||
`.ci-fingerprint` into the build context before building. Its value is
|
`.ci-fingerprint` into the build context before building. Its value is
|
||||||
the hash of the last commit that touched the build context, so:
|
the hash of the last commit that touched anything other than `*.md`,
|
||||||
|
`LICENSE` and `.editorconfig`, so:
|
||||||
|
|
||||||
- Any commit that changes code (including a squash merge whose tree
|
- Any commit that changes code (including a squash merge whose tree
|
||||||
matches an already-built branch) gets a new fingerprint, invalidates
|
matches an already-built branch) gets a new fingerprint, invalidates
|
||||||
the `COPY . .` layer of both check stages, and really runs
|
the `COPY . .` layer of both check stages, and really runs
|
||||||
`make fmt-check`, `golangci-lint`, `make test`, and `make build`. A
|
`make fmt-check`, `golangci-lint`, `make test`, and `make build`. A
|
||||||
run that reports success ran them.
|
run that reports success ran them.
|
||||||
- A docs-only commit leaves the fingerprint unchanged — `.dockerignore`
|
- A docs-only commit leaves the fingerprint unchanged, but it still
|
||||||
excludes `*.md`, `LICENSE` and `.editorconfig` from the context
|
rebuilds in full: the context also carries `.git`, which changes with
|
||||||
anyway — so the image replays from cache and costs seconds.
|
every commit (see [Version stamping](#version-stamping)).
|
||||||
|
|
||||||
The module download layer sits above `COPY . .` and stays cached either
|
The module download layer sits above `COPY . .` and stays cached either
|
||||||
way.
|
way.
|
||||||
|
|||||||
@@ -16,7 +16,6 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/handlers"
|
"sneak.berlin/go/webhooker/internal/handlers"
|
||||||
"sneak.berlin/go/webhooker/internal/healthcheck"
|
"sneak.berlin/go/webhooker/internal/healthcheck"
|
||||||
"sneak.berlin/go/webhooker/internal/logger"
|
"sneak.berlin/go/webhooker/internal/logger"
|
||||||
"sneak.berlin/go/webhooker/internal/metrics"
|
|
||||||
"sneak.berlin/go/webhooker/internal/middleware"
|
"sneak.berlin/go/webhooker/internal/middleware"
|
||||||
"sneak.berlin/go/webhooker/internal/resetpw"
|
"sneak.berlin/go/webhooker/internal/resetpw"
|
||||||
"sneak.berlin/go/webhooker/internal/server"
|
"sneak.berlin/go/webhooker/internal/server"
|
||||||
@@ -178,10 +177,6 @@ func newApp() *fx.App {
|
|||||||
healthcheck.New,
|
healthcheck.New,
|
||||||
session.New,
|
session.New,
|
||||||
handlers.New,
|
handlers.New,
|
||||||
// The registry /metrics serves, and the delivery
|
|
||||||
// collectors registered on it.
|
|
||||||
metrics.NewRegistry,
|
|
||||||
metrics.New,
|
|
||||||
middleware.New,
|
middleware.New,
|
||||||
// The one SSRF guard both target-creation validation
|
// The one SSRF guard both target-creation validation
|
||||||
// and the delivery dialer consult, so they cannot
|
// and the delivery dialer consult, so they cannot
|
||||||
|
|||||||
+60
-22
@@ -75,11 +75,6 @@ const (
|
|||||||
// internet-exposed endpoint.
|
// internet-exposed endpoint.
|
||||||
defaultReceiverRateLimit = 120
|
defaultReceiverRateLimit = 120
|
||||||
|
|
||||||
// defaultTrustedProxies is TRUSTED_PROXIES when it is unset: the
|
|
||||||
// RFC 1918 private ranges, which a reverse proxy reaching the
|
|
||||||
// process over a Docker network or a private LAN connects from.
|
|
||||||
defaultTrustedProxies = "10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"
|
|
||||||
|
|
||||||
// maxPort is the highest valid TCP port number. The lower
|
// maxPort is the highest valid TCP port number. The lower
|
||||||
// bound (at least 1) is enforced by envPositiveInt.
|
// bound (at least 1) is enforced by envPositiveInt.
|
||||||
maxPort = 65535
|
maxPort = 65535
|
||||||
@@ -177,14 +172,13 @@ type Config struct {
|
|||||||
|
|
||||||
// TrustedProxies is the set of networks whose members are
|
// TrustedProxies is the set of networks whose members are
|
||||||
// allowed to speak for the client with X-Forwarded-For, the
|
// allowed to speak for the client with X-Forwarded-For, the
|
||||||
// only forwarded header read. Unless TRUSTED_PROXIES is set it
|
// only forwarded header read. It is empty unless
|
||||||
// is the RFC 1918 private ranges (defaultTrustedProxies); a set
|
// TRUSTED_PROXIES is set, and empty means no peer is
|
||||||
// value replaces them. If any client can reach the process, or
|
// trusted: forwarded headers are then ignored entirely and
|
||||||
// the proxy in front of it, from an RFC 1918 source address
|
// clients are identified by the connection's own address.
|
||||||
// (directly, or through anything that can rewrite source
|
// Members can choose their own rate-limit key, so this must
|
||||||
// addresses, such as NAT or a published container port), it
|
// name proxy hosts only, never a block that also covers
|
||||||
// must be set to the proxy's address alone, or every rate limit
|
// clients.
|
||||||
// can be bypassed by those clients.
|
|
||||||
TrustedProxies []netip.Prefix
|
TrustedProxies []netip.Prefix
|
||||||
|
|
||||||
// AllowedEgressCIDRs is the set of networks a delivery target
|
// AllowedEgressCIDRs is the set of networks a delivery target
|
||||||
@@ -466,15 +460,14 @@ func parseCIDR(entry string) (netip.Prefix, error) {
|
|||||||
|
|
||||||
// envPrefixList returns the value of the named environment variable
|
// envPrefixList returns the value of the named environment variable
|
||||||
// parsed as a comma-separated list of CIDR blocks (bare addresses
|
// parsed as a comma-separated list of CIDR blocks (bare addresses
|
||||||
// allowed). An unset, empty, or blank value is read as defaultValue
|
// allowed). An unset, empty, or blank value yields an empty list. A
|
||||||
// instead. A set value containing an unparseable entry is a hard
|
// set value containing an unparseable entry is a hard error naming
|
||||||
// error naming the key and the bad entry, so startup fails loudly
|
// the key and the bad entry, so startup fails loudly rather than
|
||||||
// rather than silently running with a list the operator did not
|
// silently running with a list the operator did not intend.
|
||||||
// intend.
|
func envPrefixList(key string) ([]netip.Prefix, error) {
|
||||||
func envPrefixList(key, defaultValue string) ([]netip.Prefix, error) {
|
|
||||||
v := strings.TrimSpace(os.Getenv(key))
|
v := strings.TrimSpace(os.Getenv(key))
|
||||||
if v == "" {
|
if v == "" {
|
||||||
v = defaultValue
|
return nil, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
var prefixes []netip.Prefix
|
var prefixes []netip.Prefix
|
||||||
@@ -688,12 +681,12 @@ func loadFromEnv() (*Config, error) {
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
trustedProxies, err := envPrefixList("TRUSTED_PROXIES", defaultTrustedProxies)
|
trustedProxies, err := envPrefixList("TRUSTED_PROXIES")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
allowedEgressCIDRs, err := envPrefixList("ALLOWED_EGRESS_CIDRS", "")
|
allowedEgressCIDRs, err := envPrefixList("ALLOWED_EGRESS_CIDRS")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -767,6 +760,50 @@ func (c *Config) warnEgressAllowlist(log *slog.Logger) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// warnSharedRateLimitBucket logs a startup warning whenever
|
||||||
|
// TRUSTED_PROXIES is empty, in any environment.
|
||||||
|
//
|
||||||
|
// With no trusted proxies every rate limiter keys on the connecting
|
||||||
|
// peer's address. Whether that is harmless or dangerous depends on
|
||||||
|
// what is in front of the process, which this code cannot observe:
|
||||||
|
// with nothing in front, the peer is the client and the limits are
|
||||||
|
// per-client as intended; behind a reverse proxy the peer is the proxy
|
||||||
|
// for every request, so all clients share one bucket per limiter.
|
||||||
|
//
|
||||||
|
// The login endpoint no longer spends budget on arrival — it verifies
|
||||||
|
// credentials first and charges only failures — so a shared bucket
|
||||||
|
// cannot deny the operator a correct password. What it does collapse
|
||||||
|
// is the failure counting: one client's wrong passwords throttle
|
||||||
|
// everyone else's wrong passwords, and the receiver's limits become
|
||||||
|
// service-wide ceilings.
|
||||||
|
//
|
||||||
|
// The warning is deliberately not gated on WEBHOOKER_ENVIRONMENT:
|
||||||
|
// behind a proxy every client shares one bucket in dev and prod alike.
|
||||||
|
//
|
||||||
|
// The default of trusting nobody is deliberate — trusting forwarded
|
||||||
|
// headers from arbitrary peers lets any client choose its own bucket —
|
||||||
|
// so this warns rather than failing startup or changing the key.
|
||||||
|
func (c *Config) warnSharedRateLimitBucket(log *slog.Logger) {
|
||||||
|
if len(c.TrustedProxies) > 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
log.Warn(
|
||||||
|
"TRUSTED_PROXIES is empty: every rate limit keys on the "+
|
||||||
|
"connecting peer's address. With nothing proxying to "+
|
||||||
|
"this process that is the client itself and the limits "+
|
||||||
|
"are per-client as intended. Behind a reverse proxy the "+
|
||||||
|
"peer is the proxy on every request, so all clients "+
|
||||||
|
"share one bucket per limit: the receiver limits become "+
|
||||||
|
"service-wide ceilings, and one client's failed logins "+
|
||||||
|
"throttle every other client's failed logins — a "+
|
||||||
|
"correct password still gets in. If anything proxies to "+
|
||||||
|
"this process, set TRUSTED_PROXIES to its address.",
|
||||||
|
"environment", c.Environment,
|
||||||
|
"trustedProxies", len(c.TrustedProxies),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
// New creates a Config by reading environment variables.
|
// New creates a Config by reading environment variables.
|
||||||
//
|
//
|
||||||
//nolint:revive // lc parameter is required by fx even if unused.
|
//nolint:revive // lc parameter is required by fx even if unused.
|
||||||
@@ -812,6 +849,7 @@ func New(lc fx.Lifecycle, params ConfigParams) (*Config, error) {
|
|||||||
"hasMetricsAuth", s.MetricsAuthEnabled(),
|
"hasMetricsAuth", s.MetricsAuthEnabled(),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
s.warnSharedRateLimitBucket(log)
|
||||||
s.warnEgressAllowlist(log)
|
s.warnEgressAllowlist(log)
|
||||||
|
|
||||||
return s, nil
|
return s, nil
|
||||||
|
|||||||
+101
-14
@@ -551,11 +551,6 @@ func testReceiverRateLimitSuccess(
|
|||||||
}
|
}
|
||||||
|
|
||||||
func TestTrustedProxies(t *testing.T) {
|
func TestTrustedProxies(t *testing.T) {
|
||||||
// Unset, the RFC 1918 private ranges are trusted, so a reverse
|
|
||||||
// proxy on a Docker network or a private LAN is covered without
|
|
||||||
// configuration.
|
|
||||||
defaultProxies := []string{cidrPrivateV4, "172.16.0.0/12", "192.168.0.0/16"}
|
|
||||||
|
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
name string
|
name string
|
||||||
set bool
|
set bool
|
||||||
@@ -564,21 +559,18 @@ func TestTrustedProxies(t *testing.T) {
|
|||||||
expected []string
|
expected []string
|
||||||
}{
|
}{
|
||||||
{
|
{
|
||||||
|
// The default must be "trust nobody": an empty list
|
||||||
|
// means forwarded headers are ignored, never that
|
||||||
|
// every peer may speak for the client.
|
||||||
name: caseUnsetUsesDefault,
|
name: caseUnsetUsesDefault,
|
||||||
set: false,
|
set: false,
|
||||||
expected: defaultProxies,
|
expected: []string{},
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "blank value uses default",
|
name: "blank value trusts nothing",
|
||||||
set: true,
|
set: true,
|
||||||
value: " ",
|
value: " ",
|
||||||
expected: defaultProxies,
|
expected: []string{},
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "set value replaces the default entirely",
|
|
||||||
set: true,
|
|
||||||
value: "203.0.113.7",
|
|
||||||
expected: []string{"203.0.113.7/32"},
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: caseValidValueParsed,
|
name: caseValidValueParsed,
|
||||||
@@ -853,6 +845,101 @@ func TestEgressAllowlistWarning(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestSharedRateLimitBucketWarning covers the startup warning that
|
||||||
|
// tells an operator a deployment behind a reverse proxy shares one
|
||||||
|
// rate-limit bucket between every client, which turns the receiver
|
||||||
|
// limits into service-wide ceilings and collapses login failure
|
||||||
|
// counting. It must fire whenever TRUSTED_PROXIES is empty, in any
|
||||||
|
// environment, because behind a proxy every client shares one bucket
|
||||||
|
// in dev and prod alike. It stays quiet once proxies are named.
|
||||||
|
func TestSharedRateLimitBucketWarning(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
environment string
|
||||||
|
trustedProxies string
|
||||||
|
expectWarning bool
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "prod without trusted proxies warns",
|
||||||
|
environment: config.EnvironmentProd,
|
||||||
|
expectWarning: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "prod with trusted proxies is quiet",
|
||||||
|
environment: config.EnvironmentProd,
|
||||||
|
trustedProxies: cidrPrivateV4,
|
||||||
|
expectWarning: false,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "dev without trusted proxies warns",
|
||||||
|
environment: config.EnvironmentDev,
|
||||||
|
expectWarning: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "dev with trusted proxies is quiet",
|
||||||
|
environment: config.EnvironmentDev,
|
||||||
|
trustedProxies: cidrPrivateV4,
|
||||||
|
expectWarning: false,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
// Cannot use t.Parallel() here because t.Setenv
|
||||||
|
// is incompatible with parallel subtests.
|
||||||
|
t.Setenv("WEBHOOKER_ENVIRONMENT", tt.environment)
|
||||||
|
|
||||||
|
if tt.trustedProxies == "" {
|
||||||
|
require.NoError(
|
||||||
|
t, os.Unsetenv("TRUSTED_PROXIES"),
|
||||||
|
)
|
||||||
|
} else {
|
||||||
|
t.Setenv("TRUSTED_PROXIES", tt.trustedProxies)
|
||||||
|
}
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
|
||||||
|
log := slog.New(slog.NewJSONHandler(
|
||||||
|
&buf, &slog.HandlerOptions{
|
||||||
|
Level: slog.LevelDebug,
|
||||||
|
},
|
||||||
|
))
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t,
|
||||||
|
config.WarnSharedRateLimitBucketForTest(log),
|
||||||
|
)
|
||||||
|
|
||||||
|
if !tt.expectWarning {
|
||||||
|
assert.Empty(t, buf.String())
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
logged := buf.String()
|
||||||
|
|
||||||
|
assert.Contains(t, logged, `"level":"WARN"`)
|
||||||
|
assert.Contains(t, logged, "TRUSTED_PROXIES")
|
||||||
|
assert.Contains(t, logged, "share one bucket")
|
||||||
|
assert.Contains(
|
||||||
|
t, logged, "throttle every other client's failed logins",
|
||||||
|
)
|
||||||
|
// The warning must not claim a lockout the login
|
||||||
|
// endpoint no longer permits: credentials are verified
|
||||||
|
// before any budget is spent.
|
||||||
|
assert.Contains(
|
||||||
|
t, logged, "a correct password still gets in",
|
||||||
|
)
|
||||||
|
// The text must stay accurate for a developer with
|
||||||
|
// nothing in front of the process, where an empty
|
||||||
|
// list costs nothing.
|
||||||
|
assert.Contains(
|
||||||
|
t, logged, "nothing proxying to this process",
|
||||||
|
)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// metricsEnv describes what one subtest below puts in the
|
// metricsEnv describes what one subtest below puts in the
|
||||||
// environment for a single METRICS_ variable. A variable that is
|
// environment for a single METRICS_ variable. A variable that is
|
||||||
// set to the empty string and one that is not set at all are
|
// set to the empty string and one that is not set at all are
|
||||||
|
|||||||
@@ -6,6 +6,21 @@ import "log/slog"
|
|||||||
// the external config_test package so each helper can be covered by
|
// the external config_test package so each helper can be covered by
|
||||||
// its own table-driven test without weakening the package API.
|
// its own table-driven test without weakening the package API.
|
||||||
|
|
||||||
|
// WarnSharedRateLimitBucketForTest loads a Config from the current
|
||||||
|
// environment and emits its startup warnings to log. The real logger
|
||||||
|
// writes to stdout, so this lets the warning's firing condition be
|
||||||
|
// asserted against a handler the test controls.
|
||||||
|
func WarnSharedRateLimitBucketForTest(log *slog.Logger) error {
|
||||||
|
c, err := loadFromEnv()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
c.warnSharedRateLimitBucket(log)
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
// WarnEgressAllowlistForTest loads a Config from the current
|
// WarnEgressAllowlistForTest loads a Config from the current
|
||||||
// environment and emits its egress-allowlist startup warning to
|
// environment and emits its egress-allowlist startup warning to
|
||||||
// log, so a test can assert both that the warning fires only when
|
// log, so a test can assert both that the warning fires only when
|
||||||
|
|||||||
@@ -148,7 +148,6 @@ type EngineParams struct {
|
|||||||
DBManager *database.WebhookDBManager
|
DBManager *database.WebhookDBManager
|
||||||
Logger *logger.Logger
|
Logger *logger.Logger
|
||||||
SSRFGuard *Guard
|
SSRFGuard *Guard
|
||||||
Metrics *metrics.Set
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Engine processes queued deliveries in the background
|
// Engine processes queued deliveries in the background
|
||||||
@@ -168,10 +167,10 @@ type Engine struct {
|
|||||||
retryCh chan Task
|
retryCh chan Task
|
||||||
workers int
|
workers int
|
||||||
|
|
||||||
// mtr is the delivery metric set. Production wires the one
|
// mtr is the delivery metric set. Production wires the
|
||||||
// registered on the registry /metrics serves; a test can
|
// process-wide one; a test can substitute a set registered on
|
||||||
// substitute a set registered on a registry it holds, so it can
|
// a private registry so its assertions are not disturbed by
|
||||||
// gather what its own deliveries recorded.
|
// deliveries other tests are making at the same time.
|
||||||
mtr *metrics.Set
|
mtr *metrics.Set
|
||||||
|
|
||||||
// targets maps each target type to its implementation.
|
// targets maps each target type to its implementation.
|
||||||
@@ -205,7 +204,7 @@ func New(
|
|||||||
deliveryCh: make(chan Task, deliveryChannelSize),
|
deliveryCh: make(chan Task, deliveryChannelSize),
|
||||||
retryCh: make(chan Task, retryChannelSize),
|
retryCh: make(chan Task, retryChannelSize),
|
||||||
workers: defaultWorkers,
|
workers: defaultWorkers,
|
||||||
mtr: params.Metrics,
|
mtr: metrics.Default(),
|
||||||
}
|
}
|
||||||
|
|
||||||
e.initTargets(&http.Client{
|
e.initTargets(&http.Client{
|
||||||
|
|||||||
@@ -9,7 +9,6 @@ import (
|
|||||||
"net/url"
|
"net/url"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/prometheus/client_golang/prometheus"
|
|
||||||
"go.uber.org/fx"
|
"go.uber.org/fx"
|
||||||
"gorm.io/gorm"
|
"gorm.io/gorm"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
@@ -390,7 +389,7 @@ func NewTestEngine(
|
|||||||
deliveryCh: make(chan Task, deliveryChannelSize),
|
deliveryCh: make(chan Task, deliveryChannelSize),
|
||||||
retryCh: make(chan Task, retryChannelSize),
|
retryCh: make(chan Task, retryChannelSize),
|
||||||
workers: workers,
|
workers: workers,
|
||||||
mtr: metrics.New(prometheus.NewRegistry()),
|
mtr: metrics.Default(),
|
||||||
}
|
}
|
||||||
e.initTargets(client)
|
e.initTargets(client)
|
||||||
|
|
||||||
@@ -405,7 +404,7 @@ func NewTestEngineSmallRetry(
|
|||||||
e := &Engine{
|
e := &Engine{
|
||||||
log: log,
|
log: log,
|
||||||
retryCh: make(chan Task, 1),
|
retryCh: make(chan Task, 1),
|
||||||
mtr: metrics.New(prometheus.NewRegistry()),
|
mtr: metrics.Default(),
|
||||||
}
|
}
|
||||||
e.initTargets(nil)
|
e.initTargets(nil)
|
||||||
|
|
||||||
@@ -428,7 +427,7 @@ func NewTestEngineWithDB(
|
|||||||
deliveryCh: make(chan Task, deliveryChannelSize),
|
deliveryCh: make(chan Task, deliveryChannelSize),
|
||||||
retryCh: make(chan Task, retryChannelSize),
|
retryCh: make(chan Task, retryChannelSize),
|
||||||
workers: workers,
|
workers: workers,
|
||||||
mtr: metrics.New(prometheus.NewRegistry()),
|
mtr: metrics.Default(),
|
||||||
}
|
}
|
||||||
e.initTargets(client)
|
e.initTargets(client)
|
||||||
|
|
||||||
@@ -436,7 +435,8 @@ func NewTestEngineWithDB(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// ExportSetMetrics substitutes the engine's metric set, so a test can
|
// ExportSetMetrics substitutes the engine's metric set, so a test can
|
||||||
// assert on collectors registered on a registry it holds.
|
// assert on collectors registered on a private registry instead of
|
||||||
|
// the process-wide ones every other test is also moving.
|
||||||
func (e *Engine) ExportSetMetrics(mtr *metrics.Set) {
|
func (e *Engine) ExportSetMetrics(mtr *metrics.Set) {
|
||||||
e.mtr = mtr
|
e.mtr = mtr
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -35,8 +35,9 @@ const (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// mIsolate gives the setup's engine a metric set registered on a
|
// mIsolate gives the setup's engine a metric set registered on a
|
||||||
// registry this test holds, so its exact assertions can gather from
|
// private registry. The process-wide collectors are moved by every
|
||||||
// it.
|
// other delivery test running in parallel, so exact assertions are
|
||||||
|
// only possible against a registry this test owns.
|
||||||
func mIsolate(
|
func mIsolate(
|
||||||
t *testing.T, s iSetup,
|
t *testing.T, s iSetup,
|
||||||
) *prometheus.Registry {
|
) *prometheus.Registry {
|
||||||
|
|||||||
@@ -43,13 +43,6 @@ var (
|
|||||||
// permit specific blocks out of this set with
|
// permit specific blocks out of this set with
|
||||||
// ALLOWED_EGRESS_CIDRS; see Guard.
|
// ALLOWED_EGRESS_CIDRS; see Guard.
|
||||||
//
|
//
|
||||||
// A public address belongs on the default blocklist only if it
|
|
||||||
// hands credentials, user data or bootstrap material to whatever
|
|
||||||
// can reach it, without the caller presenting anything. A
|
|
||||||
// provider's other public addresses are not refused, since
|
|
||||||
// reaching them can be legitimate and no list of them could be
|
|
||||||
// complete.
|
|
||||||
//
|
|
||||||
//nolint:gochecknoglobals // package-level network list is appropriate here
|
//nolint:gochecknoglobals // package-level network list is appropriate here
|
||||||
var blockedNetworks []*net.IPNet
|
var blockedNetworks []*net.IPNet
|
||||||
|
|
||||||
|
|||||||
@@ -103,10 +103,9 @@ func (h *Handlers) renderLoginError(
|
|||||||
// The credential check runs BEFORE any rate-limit budget is
|
// The credential check runs BEFORE any rate-limit budget is
|
||||||
// consulted, and only a failed check spends budget. That is what
|
// consulted, and only a failed check spends budget. That is what
|
||||||
// keeps the single administrative path reachable: behind the reverse
|
// keeps the single administrative path reachable: behind the reverse
|
||||||
// proxy this deployment requires, when TRUSTED_PROXIES does not cover
|
// proxy this deployment requires, with TRUSTED_PROXIES unset, every
|
||||||
// it, every client shares one bucket, so a limiter spent on arrival
|
// client shares one bucket, so a limiter spent on arrival lets any
|
||||||
// lets any stranger deny the operator's own correct password
|
// stranger deny the operator's own correct password indefinitely.
|
||||||
// indefinitely.
|
|
||||||
//
|
//
|
||||||
// Verifying first means every login POST costs an Argon2id hash, so
|
// Verifying first means every login POST costs an Argon2id hash, so
|
||||||
// the work is taken under a bounded number of verification slots.
|
// the work is taken under a bounded number of verification slots.
|
||||||
|
|||||||
@@ -25,7 +25,7 @@ const (
|
|||||||
|
|
||||||
// sharedProxyPeer is the whole point of this file. Production is
|
// sharedProxyPeer is the whole point of this file. Production is
|
||||||
// required to run behind a TLS-terminating reverse proxy, and
|
// required to run behind a TLS-terminating reverse proxy, and
|
||||||
// when TRUSTED_PROXIES does not cover it every client — attacker
|
// TRUSTED_PROXIES defaults to empty, so every client — attacker
|
||||||
// and operator alike — reaches the process from the proxy's
|
// and operator alike — reaches the process from the proxy's
|
||||||
// address and shares one rate-limit bucket. Both parties in
|
// address and shares one rate-limit bucket. Both parties in
|
||||||
// these tests therefore use the same RemoteAddr.
|
// these tests therefore use the same RemoteAddr.
|
||||||
@@ -115,11 +115,11 @@ func floodFailures(
|
|||||||
// done-criterion of https://git.eeqj.de/sneak/webhooker/issues/150.
|
// done-criterion of https://git.eeqj.de/sneak/webhooker/issues/150.
|
||||||
//
|
//
|
||||||
// The attacker and the operator share one rate-limit bucket, because
|
// The attacker and the operator share one rate-limit bucket, because
|
||||||
// behind the mandated reverse proxy, when TRUSTED_PROXIES does not
|
// behind the mandated reverse proxy with TRUSTED_PROXIES unset every
|
||||||
// cover it, every client keys on the proxy's address. The attacker
|
// client keys on the proxy's address. The attacker floods the
|
||||||
// floods the operator's own username — a single-admin product has a
|
// operator's own username — a single-admin product has a predictable
|
||||||
// predictable one — far past the failure limit. The operator must
|
// one — far past the failure limit. The operator must still be able
|
||||||
// still be able to log in with the correct password.
|
// to log in with the correct password.
|
||||||
//
|
//
|
||||||
// This fails if credentials stop being verified ahead of the limiter.
|
// This fails if credentials stop being verified ahead of the limiter.
|
||||||
func TestLogin_StrangersFloodCannotLockOutTheOperator(t *testing.T) {
|
func TestLogin_StrangersFloodCannotLockOutTheOperator(t *testing.T) {
|
||||||
|
|||||||
@@ -12,7 +12,6 @@ import (
|
|||||||
"net/http"
|
"net/http"
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
|
|
||||||
"github.com/prometheus/client_golang/prometheus"
|
|
||||||
"go.uber.org/fx"
|
"go.uber.org/fx"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
"sneak.berlin/go/webhooker/internal/delivery"
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
@@ -62,8 +61,6 @@ type HandlersParams struct {
|
|||||||
Notifier delivery.Notifier
|
Notifier delivery.Notifier
|
||||||
Evictor delivery.WebhookEvictor
|
Evictor delivery.WebhookEvictor
|
||||||
SSRFGuard *delivery.Guard
|
SSRFGuard *delivery.Guard
|
||||||
Metrics *metrics.Set
|
|
||||||
Registry *prometheus.Registry
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Handlers provides HTTP handler methods for all application
|
// Handlers provides HTTP handler methods for all application
|
||||||
@@ -125,7 +122,7 @@ func New(
|
|||||||
s.mw = params.Middleware
|
s.mw = params.Middleware
|
||||||
s.notifier = params.Notifier
|
s.notifier = params.Notifier
|
||||||
s.evictor = params.Evictor
|
s.evictor = params.Evictor
|
||||||
s.mtr = params.Metrics
|
s.mtr = metrics.Default()
|
||||||
s.ssrf = params.SSRFGuard
|
s.ssrf = params.SSRFGuard
|
||||||
|
|
||||||
// Parse all page templates once at startup
|
// Parse all page templates once at startup
|
||||||
|
|||||||
@@ -20,7 +20,6 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/handlers"
|
"sneak.berlin/go/webhooker/internal/handlers"
|
||||||
"sneak.berlin/go/webhooker/internal/healthcheck"
|
"sneak.berlin/go/webhooker/internal/healthcheck"
|
||||||
"sneak.berlin/go/webhooker/internal/logger"
|
"sneak.berlin/go/webhooker/internal/logger"
|
||||||
"sneak.berlin/go/webhooker/internal/metrics"
|
|
||||||
"sneak.berlin/go/webhooker/internal/middleware"
|
"sneak.berlin/go/webhooker/internal/middleware"
|
||||||
"sneak.berlin/go/webhooker/internal/session"
|
"sneak.berlin/go/webhooker/internal/session"
|
||||||
)
|
)
|
||||||
@@ -110,8 +109,6 @@ func newTestApp(
|
|||||||
func(r *recordingEvictor) delivery.WebhookEvictor {
|
func(r *recordingEvictor) delivery.WebhookEvictor {
|
||||||
return r
|
return r
|
||||||
},
|
},
|
||||||
metrics.NewRegistry,
|
|
||||||
metrics.New,
|
|
||||||
middleware.New,
|
middleware.New,
|
||||||
delivery.NewGuard,
|
delivery.NewGuard,
|
||||||
handlers.New,
|
handlers.New,
|
||||||
|
|||||||
@@ -1,21 +0,0 @@
|
|||||||
package handlers
|
|
||||||
|
|
||||||
import (
|
|
||||||
"net/http"
|
|
||||||
|
|
||||||
"github.com/prometheus/client_golang/prometheus/promhttp"
|
|
||||||
)
|
|
||||||
|
|
||||||
// HandleMetrics returns the Prometheus scrape handler for the
|
|
||||||
// registry built by metrics.NewRegistry, which the HTTP, delivery, Go
|
|
||||||
// runtime and process collectors register on. It is what
|
|
||||||
// promhttp.Handler builds for the global default registry, including
|
|
||||||
// the promhttp_metric_handler_* series that count scrapes, pointed at
|
|
||||||
// that registry instead.
|
|
||||||
func (s *Handlers) HandleMetrics() http.HandlerFunc {
|
|
||||||
reg := s.params.Registry
|
|
||||||
|
|
||||||
return promhttp.InstrumentMetricHandler(
|
|
||||||
reg, promhttp.HandlerFor(reg, promhttp.HandlerOpts{}),
|
|
||||||
).ServeHTTP
|
|
||||||
}
|
|
||||||
+20
-27
@@ -3,18 +3,17 @@
|
|||||||
// deliveries are attempted, how they end, how long they take, how
|
// deliveries are attempted, how they end, how long they take, how
|
||||||
// deep the queues are, and how many circuit breakers are open.
|
// deep the queues are, and how many circuit breakers are open.
|
||||||
//
|
//
|
||||||
// It also builds the registry the authenticated /metrics route
|
// The inbound HTTP metrics come from the go-http-metrics recorder in
|
||||||
// serves. In production, these collectors, the inbound HTTP metrics
|
// internal/middleware and land on prometheus.DefaultRegisterer. These
|
||||||
// recorded in internal/middleware, and the Go runtime and process
|
// collectors register there too, so both surfaces are gathered by the
|
||||||
// collectors all register on that one registry, never on Prometheus's
|
// one promhttp handler mounted on the authenticated /metrics route.
|
||||||
// global default.
|
|
||||||
package metrics
|
package metrics
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/prometheus/client_golang/prometheus"
|
"github.com/prometheus/client_golang/prometheus"
|
||||||
"github.com/prometheus/client_golang/prometheus/collectors"
|
|
||||||
"github.com/prometheus/client_golang/prometheus/promauto"
|
"github.com/prometheus/client_golang/prometheus/promauto"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
)
|
)
|
||||||
@@ -58,31 +57,25 @@ var knownTargetTypes = []database.TargetType{
|
|||||||
database.TargetTypeSlack,
|
database.TargetTypeSlack,
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewRegistry returns the registry /metrics serves, carrying the Go
|
// defaultSet is the process-wide metric set, registered on the same
|
||||||
// runtime and process collectors that Prometheus's global default
|
// registry the HTTP middleware and the /metrics handler already use.
|
||||||
// registry carries, so the go_* and process_* series stay in the
|
// It is built on first use rather than in an init so that a test
|
||||||
// scrape.
|
// binary that never touches metrics never registers them.
|
||||||
//
|
//
|
||||||
// A registry of its own, rather than the global default, is what lets
|
//nolint:gochecknoglobals // one process-wide registration, by design
|
||||||
// two dependency graphs in one process — two tests, say — each
|
var defaultSet = sync.OnceValue(func() *Set {
|
||||||
// register their collectors without the second registration
|
return New(prometheus.DefaultRegisterer)
|
||||||
// panicking.
|
})
|
||||||
func NewRegistry() *prometheus.Registry {
|
|
||||||
reg := prometheus.NewRegistry()
|
|
||||||
reg.MustRegister(
|
|
||||||
collectors.NewGoCollector(),
|
|
||||||
collectors.NewProcessCollector(
|
|
||||||
collectors.ProcessCollectorOpts{},
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
return reg
|
// Default returns the process-wide metric set.
|
||||||
|
func Default() *Set {
|
||||||
|
return defaultSet()
|
||||||
}
|
}
|
||||||
|
|
||||||
// Set is one registered group of webhooker's delivery collectors.
|
// Set is one registered group of webhooker's delivery collectors.
|
||||||
// Production builds one on the registry /metrics serves; tests build
|
// Production uses the single Default set; tests build their own
|
||||||
// one on a registry of their own so they can gather what their own
|
// against a private registry so assertions are not disturbed by
|
||||||
// deliveries recorded.
|
// deliveries other tests are making concurrently.
|
||||||
type Set struct {
|
type Set struct {
|
||||||
eventsReceived prometheus.Counter
|
eventsReceived prometheus.Counter
|
||||||
deliveryAttempts *prometheus.CounterVec
|
deliveryAttempts *prometheus.CounterVec
|
||||||
@@ -100,7 +93,7 @@ type Set struct {
|
|||||||
// New registers a full set of delivery collectors on reg and returns
|
// New registers a full set of delivery collectors on reg and returns
|
||||||
// it. It panics if reg already holds them, which is the intended
|
// it. It panics if reg already holds them, which is the intended
|
||||||
// behaviour for a duplicate registration.
|
// behaviour for a duplicate registration.
|
||||||
func New(reg *prometheus.Registry) *Set {
|
func New(reg prometheus.Registerer) *Set {
|
||||||
factory := promauto.With(reg)
|
factory := promauto.With(reg)
|
||||||
|
|
||||||
s := &Set{
|
s := &Set{
|
||||||
|
|||||||
@@ -10,7 +10,8 @@ import (
|
|||||||
|
|
||||||
// MetricsMiddlewareForTest builds the metrics recording middleware
|
// MetricsMiddlewareForTest builds the metrics recording middleware
|
||||||
// against a caller-supplied recorder, so a test can gather from its
|
// against a caller-supplied recorder, so a test can gather from its
|
||||||
// own Prometheus registry without building a whole Middleware.
|
// own Prometheus registry rather than the process-wide default one
|
||||||
|
// that Middleware.Metrics uses.
|
||||||
func MetricsMiddlewareForTest(
|
func MetricsMiddlewareForTest(
|
||||||
rec httpmetrics.Recorder,
|
rec httpmetrics.Recorder,
|
||||||
) func(http.Handler) http.Handler {
|
) func(http.Handler) http.Handler {
|
||||||
|
|||||||
@@ -108,10 +108,10 @@ type failureWindow struct {
|
|||||||
//
|
//
|
||||||
// A limiter that spends budget on arrival cannot protect a
|
// A limiter that spends budget on arrival cannot protect a
|
||||||
// single-admin product: behind the reverse proxy the deployment
|
// single-admin product: behind the reverse proxy the deployment
|
||||||
// requires, when TRUSTED_PROXIES does not cover it, every client
|
// requires, with TRUSTED_PROXIES unset, every client keys on the
|
||||||
// keys on the proxy, so a stranger trickling five POSTs a minute
|
// proxy, so a stranger trickling five POSTs a minute keeps the one
|
||||||
// keeps the one bucket full and the operator's own correct password
|
// bucket full and the operator's own correct password is answered 429
|
||||||
// is answered 429 forever. There is no second administrative path.
|
// forever. There is no second administrative path.
|
||||||
//
|
//
|
||||||
// So budget is spent only by a FAILED verification. A correct
|
// So budget is spent only by a FAILED verification. A correct
|
||||||
// password is never throttled, whatever the counters say, which is
|
// password is never throttled, whatever the counters say, which is
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
|
|
||||||
"github.com/go-chi/chi"
|
"github.com/go-chi/chi"
|
||||||
httpmetrics "github.com/slok/go-http-metrics/metrics"
|
httpmetrics "github.com/slok/go-http-metrics/metrics"
|
||||||
|
prommetrics "github.com/slok/go-http-metrics/metrics/prometheus"
|
||||||
ghmm "github.com/slok/go-http-metrics/middleware"
|
ghmm "github.com/slok/go-http-metrics/middleware"
|
||||||
"github.com/slok/go-http-metrics/middleware/std"
|
"github.com/slok/go-http-metrics/middleware/std"
|
||||||
)
|
)
|
||||||
@@ -150,17 +151,17 @@ func (r boundedLabelRecorder) AddInflightRequests(
|
|||||||
|
|
||||||
var _ httpmetrics.Recorder = boundedLabelRecorder{}
|
var _ httpmetrics.Recorder = boundedLabelRecorder{}
|
||||||
|
|
||||||
// Metrics returns middleware that records Prometheus HTTP metrics
|
// Metrics returns middleware that records Prometheus HTTP metrics on
|
||||||
// with the Middleware's one recorder, which New builds on the registry
|
// the default registry, which is the one the /metrics route gathers.
|
||||||
// the /metrics route serves and NewForTest on a registry of its own.
|
|
||||||
// Every call reuses that recorder, so any number of routers can
|
|
||||||
// install it.
|
|
||||||
func (s *Middleware) Metrics() func(http.Handler) http.Handler {
|
func (s *Middleware) Metrics() func(http.Handler) http.Handler {
|
||||||
return metricsMiddleware(s.metricsRecorder)
|
return metricsMiddleware(
|
||||||
|
prommetrics.NewRecorder(prommetrics.Config{}),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// metricsMiddleware builds the recording middleware against a given
|
// metricsMiddleware builds the recording middleware against a given
|
||||||
// recorder, so tests can gather from a registry of their own.
|
// recorder, so tests can gather from a registry of their own instead
|
||||||
|
// of the process-wide default.
|
||||||
func metricsMiddleware(
|
func metricsMiddleware(
|
||||||
rec httpmetrics.Recorder,
|
rec httpmetrics.Recorder,
|
||||||
) func(http.Handler) http.Handler {
|
) func(http.Handler) http.Handler {
|
||||||
|
|||||||
@@ -57,8 +57,9 @@ const (
|
|||||||
// Server.setupWebhookRoutes inside it. That ordering is the whole
|
// Server.setupWebhookRoutes inside it. That ordering is the whole
|
||||||
// defect, so a test that flattens it would prove nothing.
|
// defect, so a test that flattens it would prove nothing.
|
||||||
//
|
//
|
||||||
// The recorder writes to a registry of the test's own, so each test
|
// The recorder writes to a registry of the test's own rather than the
|
||||||
// observes only its own traffic.
|
// process-wide default one, so each test observes only its own
|
||||||
|
// traffic.
|
||||||
func metricsTestRouter(
|
func metricsTestRouter(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
receiverLimit int,
|
receiverLimit int,
|
||||||
@@ -454,29 +455,3 @@ func TestMetrics_StatusAndSizeStillRecorded(t *testing.T) {
|
|||||||
"the interceptor must still count written bytes",
|
"the interceptor must still count written bytes",
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestMetrics_WorksOnNewForTestMiddleware pins that a Middleware built
|
|
||||||
// by NewForTest has a recorder of its own: its Metrics() serves a
|
|
||||||
// request instead of panicking, and a second one does not collide
|
|
||||||
// with the first.
|
|
||||||
func TestMetrics_WorksOnNewForTestMiddleware(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
log := slog.New(slog.DiscardHandler)
|
|
||||||
cfg := &config.Config{Environment: "prod"}
|
|
||||||
ok := http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
|
||||||
_, _ = w.Write([]byte(okBody))
|
|
||||||
})
|
|
||||||
|
|
||||||
for range 2 {
|
|
||||||
h := middleware.NewForTest(log, cfg, nil).Metrics()(ok)
|
|
||||||
|
|
||||||
req := httptest.NewRequestWithContext(
|
|
||||||
t.Context(), http.MethodGet, okRoute, nil,
|
|
||||||
)
|
|
||||||
w := httptest.NewRecorder()
|
|
||||||
h.ServeHTTP(w, req)
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, w.Code)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -13,9 +13,6 @@ import (
|
|||||||
"github.com/go-chi/chi"
|
"github.com/go-chi/chi"
|
||||||
"github.com/go-chi/chi/middleware"
|
"github.com/go-chi/chi/middleware"
|
||||||
"github.com/go-chi/cors"
|
"github.com/go-chi/cors"
|
||||||
"github.com/prometheus/client_golang/prometheus"
|
|
||||||
httpmetrics "github.com/slok/go-http-metrics/metrics"
|
|
||||||
prommetrics "github.com/slok/go-http-metrics/metrics/prometheus"
|
|
||||||
"go.uber.org/fx"
|
"go.uber.org/fx"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
"sneak.berlin/go/webhooker/internal/globals"
|
"sneak.berlin/go/webhooker/internal/globals"
|
||||||
@@ -151,11 +148,10 @@ const (
|
|||||||
type MiddlewareParams struct {
|
type MiddlewareParams struct {
|
||||||
fx.In
|
fx.In
|
||||||
|
|
||||||
Logger *logger.Logger
|
Logger *logger.Logger
|
||||||
Globals *globals.Globals
|
Globals *globals.Globals
|
||||||
Config *config.Config
|
Config *config.Config
|
||||||
Session *session.Session
|
Session *session.Session
|
||||||
Registry *prometheus.Registry
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Middleware provides HTTP middleware for logging, CORS, auth, and
|
// Middleware provides HTTP middleware for logging, CORS, auth, and
|
||||||
@@ -165,14 +161,6 @@ type Middleware struct {
|
|||||||
params *MiddlewareParams
|
params *MiddlewareParams
|
||||||
session *session.Session
|
session *session.Session
|
||||||
|
|
||||||
// metricsRecorder records the inbound HTTP metrics. New builds
|
|
||||||
// it on the registry /metrics serves, NewForTest on a registry
|
|
||||||
// of its own. Either way it is built once per Middleware and
|
|
||||||
// Metrics reuses it, because building it registers its
|
|
||||||
// collectors, and a second registration on the same registry
|
|
||||||
// panics.
|
|
||||||
metricsRecorder httpmetrics.Recorder
|
|
||||||
|
|
||||||
// loginGuard counts failed credential verifications and bounds
|
// loginGuard counts failed credential verifications and bounds
|
||||||
// concurrent password hashing. It is built on first use so that
|
// concurrent password hashing. It is built on first use so that
|
||||||
// every construction path gets one; see guard().
|
// every construction path gets one; see guard().
|
||||||
@@ -191,9 +179,6 @@ func New(
|
|||||||
s.params = ¶ms
|
s.params = ¶ms
|
||||||
s.log = params.Logger.Get()
|
s.log = params.Logger.Get()
|
||||||
s.session = params.Session
|
s.session = params.Session
|
||||||
s.metricsRecorder = prommetrics.NewRecorder(
|
|
||||||
prommetrics.Config{Registry: params.Registry},
|
|
||||||
)
|
|
||||||
|
|
||||||
return s, nil
|
return s, nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -123,8 +123,9 @@ func bucketKey(addr netip.Addr) string {
|
|||||||
return prefix.String()
|
return prefix.String()
|
||||||
}
|
}
|
||||||
|
|
||||||
// isTrustedProxy reports whether addr belongs to a network in
|
// isTrustedProxy reports whether addr belongs to a network the
|
||||||
// TRUSTED_PROXIES, which by default is the RFC 1918 private ranges.
|
// operator listed in TRUSTED_PROXIES. The list is empty by default,
|
||||||
|
// so by default nothing is trusted.
|
||||||
func (m *Middleware) isTrustedProxy(addr netip.Addr) bool {
|
func (m *Middleware) isTrustedProxy(addr netip.Addr) bool {
|
||||||
for _, prefix := range m.params.Config.TrustedProxies {
|
for _, prefix := range m.params.Config.TrustedProxies {
|
||||||
if prefix.Contains(addr) {
|
if prefix.Contains(addr) {
|
||||||
|
|||||||
@@ -384,8 +384,8 @@ const (
|
|||||||
// trustedProxyCIDR is the proxy network the forwarded-path
|
// trustedProxyCIDR is the proxy network the forwarded-path
|
||||||
// tests configure, and trustedPeer an address inside it. A
|
// tests configure, and trustedPeer an address inside it. A
|
||||||
// production deployment is required to run behind a reverse
|
// production deployment is required to run behind a reverse
|
||||||
// proxy that TRUSTED_PROXIES covers, either by the default or by
|
// proxy with TRUSTED_PROXIES set, so this is the shape the
|
||||||
// a set value, so this is the shape the bucketing has to hold in.
|
// bucketing has to hold in.
|
||||||
trustedProxyCIDR = "10.0.0.0/8"
|
trustedProxyCIDR = "10.0.0.0/8"
|
||||||
trustedPeer = "10.0.0.1:44444"
|
trustedPeer = "10.0.0.1:44444"
|
||||||
)
|
)
|
||||||
@@ -426,8 +426,8 @@ func assertSharedBucket(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// TestRateLimitKey_SpoofedForwardedFromUntrustedPeer is the test
|
// TestRateLimitKey_SpoofedForwardedFromUntrustedPeer is the test
|
||||||
// this gating exists for: from a peer that is not a trusted
|
// this gating exists for: with no trusted proxies configured (the
|
||||||
// proxy, a client that rotates a forwarded header on every
|
// default), a client that rotates a forwarded header on every
|
||||||
// request must stay in one bucket. If forwarded headers were
|
// request must stay in one bucket. If forwarded headers were
|
||||||
// trusted unconditionally, each spoofed value would mint a fresh
|
// trusted unconditionally, each spoofed value would mint a fresh
|
||||||
// bucket and the limit would stop no one.
|
// bucket and the limit would stop no one.
|
||||||
@@ -1097,9 +1097,8 @@ func TestPostRateLimit_IPv4IndependentPerAddress(t *testing.T) {
|
|||||||
// that arrives from trustedPeer — a configured trusted proxy — and
|
// that arrives from trustedPeer — a configured trusted proxy — and
|
||||||
// names forwarded as its client in X-Forwarded-For. That is the
|
// names forwarded as its client in X-Forwarded-For. That is the
|
||||||
// production path: a deployment is required to run behind a reverse
|
// production path: a deployment is required to run behind a reverse
|
||||||
// proxy that TRUSTED_PROXIES covers, either by the default or by a
|
// proxy with TRUSTED_PROXIES set, so the forwarded address, not the
|
||||||
// set value, so the forwarded address, not the peer, is what the
|
// peer, is what the limiters bucket on there.
|
||||||
// limiters bucket on there.
|
|
||||||
func forwardedKeyFor(
|
func forwardedKeyFor(
|
||||||
t *testing.T, m *middleware.Middleware, forwarded string,
|
t *testing.T, m *middleware.Middleware, forwarded string,
|
||||||
) string {
|
) string {
|
||||||
@@ -1179,9 +1178,9 @@ func TestRateLimitKey_ForwardedIPv6BucketsByPrefix(t *testing.T) {
|
|||||||
//
|
//
|
||||||
// Every existing test of this fallback uses an IPv4 proxy, where
|
// Every existing test of this fallback uses an IPv4 proxy, where
|
||||||
// bucketKey is the identity function, so replacing the call with
|
// bucketKey is the identity function, so replacing the call with
|
||||||
// peer.String() leaves the whole suite green. Only addresses inside
|
// peer.String() leaves the whole suite green. Only operator-listed
|
||||||
// TRUSTED_PROXIES reach this line and the fallback is fail-closed, so
|
// addresses reach this line and the fallback is fail-closed, so this
|
||||||
// this pins behaviour rather than fixing a defect.
|
// pins behaviour rather than fixing a defect.
|
||||||
func TestRateLimitKey_TrustedPeerUnusableForwardedMasksPeer(
|
func TestRateLimitKey_TrustedPeerUnusableForwardedMasksPeer(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
) {
|
) {
|
||||||
|
|||||||
@@ -3,17 +3,12 @@ package middleware
|
|||||||
import (
|
import (
|
||||||
"log/slog"
|
"log/slog"
|
||||||
|
|
||||||
"github.com/prometheus/client_golang/prometheus"
|
|
||||||
prommetrics "github.com/slok/go-http-metrics/metrics/prometheus"
|
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
"sneak.berlin/go/webhooker/internal/session"
|
"sneak.berlin/go/webhooker/internal/session"
|
||||||
)
|
)
|
||||||
|
|
||||||
// NewForTest creates a Middleware with the minimum dependencies
|
// NewForTest creates a Middleware with the minimum dependencies
|
||||||
// needed for testing. This bypasses the fx lifecycle.
|
// needed for testing. This bypasses the fx lifecycle.
|
||||||
//
|
|
||||||
// Its metrics recorder writes to a fresh registry of its own, so
|
|
||||||
// Metrics() works on it and two of them never collide.
|
|
||||||
func NewForTest(
|
func NewForTest(
|
||||||
log *slog.Logger,
|
log *slog.Logger,
|
||||||
cfg *config.Config,
|
cfg *config.Config,
|
||||||
@@ -25,8 +20,5 @@ func NewForTest(
|
|||||||
Config: cfg,
|
Config: cfg,
|
||||||
},
|
},
|
||||||
session: sess,
|
session: sess,
|
||||||
metricsRecorder: prommetrics.NewRecorder(
|
|
||||||
prommetrics.Config{Registry: prometheus.NewRegistry()},
|
|
||||||
),
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -24,7 +24,6 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/handlers"
|
"sneak.berlin/go/webhooker/internal/handlers"
|
||||||
"sneak.berlin/go/webhooker/internal/healthcheck"
|
"sneak.berlin/go/webhooker/internal/healthcheck"
|
||||||
"sneak.berlin/go/webhooker/internal/logger"
|
"sneak.berlin/go/webhooker/internal/logger"
|
||||||
"sneak.berlin/go/webhooker/internal/metrics"
|
|
||||||
"sneak.berlin/go/webhooker/internal/middleware"
|
"sneak.berlin/go/webhooker/internal/middleware"
|
||||||
"sneak.berlin/go/webhooker/internal/resetpw"
|
"sneak.berlin/go/webhooker/internal/resetpw"
|
||||||
"sneak.berlin/go/webhooker/internal/session"
|
"sneak.berlin/go/webhooker/internal/session"
|
||||||
@@ -164,8 +163,6 @@ func newServerApp(
|
|||||||
session.New,
|
session.New,
|
||||||
func() delivery.Notifier { return &noopNotifier{} },
|
func() delivery.Notifier { return &noopNotifier{} },
|
||||||
func() delivery.WebhookEvictor { return &noopEvictor{} },
|
func() delivery.WebhookEvictor { return &noopEvictor{} },
|
||||||
metrics.NewRegistry,
|
|
||||||
metrics.New,
|
|
||||||
middleware.New,
|
middleware.New,
|
||||||
delivery.NewGuard,
|
delivery.NewGuard,
|
||||||
handlers.New,
|
handlers.New,
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
sentryhttp "github.com/getsentry/sentry-go/http"
|
sentryhttp "github.com/getsentry/sentry-go/http"
|
||||||
"github.com/go-chi/chi"
|
"github.com/go-chi/chi"
|
||||||
"github.com/go-chi/chi/middleware"
|
"github.com/go-chi/chi/middleware"
|
||||||
|
"github.com/prometheus/client_golang/prometheus/promhttp"
|
||||||
"sneak.berlin/go/webhooker/static"
|
"sneak.berlin/go/webhooker/static"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -129,7 +130,12 @@ func (s *Server) setupRoutes() {
|
|||||||
if s.params.Config.MetricsAuthEnabled() {
|
if s.params.Config.MetricsAuthEnabled() {
|
||||||
s.router.Group(func(r chi.Router) {
|
s.router.Group(func(r chi.Router) {
|
||||||
r.Use(s.mw.MetricsAuth())
|
r.Use(s.mw.MetricsAuth())
|
||||||
r.Get("/metrics", s.h.HandleMetrics())
|
r.Get(
|
||||||
|
"/metrics",
|
||||||
|
http.HandlerFunc(
|
||||||
|
promhttp.Handler().ServeHTTP,
|
||||||
|
),
|
||||||
|
)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -148,12 +154,11 @@ func (s *Server) setupPageRoutes() {
|
|||||||
r.Use(s.mw.NoCache())
|
r.Use(s.mw.NoCache())
|
||||||
|
|
||||||
// The login POST carries no pre-emptive rate limiter. Behind
|
// The login POST carries no pre-emptive rate limiter. Behind
|
||||||
// the reverse proxy production requires, when TRUSTED_PROXIES
|
// the reverse proxy production requires, with TRUSTED_PROXIES
|
||||||
// does not cover it, every client shares one bucket, so a
|
// unset, every client shares one bucket, so a limiter spent
|
||||||
// limiter spent on arrival lets any stranger deny the operator
|
// on arrival lets any stranger deny the operator the only
|
||||||
// the only administrative path. The handler verifies
|
// administrative path. The handler verifies credentials first
|
||||||
// credentials first and charges only failures; see
|
// and charges only failures; see Handlers.authenticateUser.
|
||||||
// Handlers.authenticateUser.
|
|
||||||
r.Get("/login", s.h.HandleLoginPage())
|
r.Get("/login", s.h.HandleLoginPage())
|
||||||
r.Post("/login", s.h.HandleLoginSubmit())
|
r.Post("/login", s.h.HandleLoginSubmit())
|
||||||
|
|
||||||
|
|||||||
@@ -24,7 +24,6 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/handlers"
|
"sneak.berlin/go/webhooker/internal/handlers"
|
||||||
"sneak.berlin/go/webhooker/internal/healthcheck"
|
"sneak.berlin/go/webhooker/internal/healthcheck"
|
||||||
"sneak.berlin/go/webhooker/internal/logger"
|
"sneak.berlin/go/webhooker/internal/logger"
|
||||||
"sneak.berlin/go/webhooker/internal/metrics"
|
|
||||||
"sneak.berlin/go/webhooker/internal/middleware"
|
"sneak.berlin/go/webhooker/internal/middleware"
|
||||||
"sneak.berlin/go/webhooker/internal/server"
|
"sneak.berlin/go/webhooker/internal/server"
|
||||||
"sneak.berlin/go/webhooker/internal/session"
|
"sneak.berlin/go/webhooker/internal/session"
|
||||||
@@ -114,8 +113,6 @@ func newTestEnvWithConfig(
|
|||||||
session.New,
|
session.New,
|
||||||
func() delivery.Notifier { return &noopNotifier{} },
|
func() delivery.Notifier { return &noopNotifier{} },
|
||||||
func() delivery.WebhookEvictor { return &noopEvictor{} },
|
func() delivery.WebhookEvictor { return &noopEvictor{} },
|
||||||
metrics.NewRegistry,
|
|
||||||
metrics.New,
|
|
||||||
middleware.New,
|
middleware.New,
|
||||||
delivery.NewGuard,
|
delivery.NewGuard,
|
||||||
handlers.New,
|
handlers.New,
|
||||||
@@ -1030,46 +1027,3 @@ func TestMetricsRouteUnmountedOnHalfSetConfig(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestTwoMetricsRoutersInOneProcess pins
|
|
||||||
// https://git.eeqj.de/sneak/webhooker/issues/227: a second
|
|
||||||
// metrics-enabled router in one process used to panic, because the
|
|
||||||
// HTTP metrics registered on Prometheus's global default registry.
|
|
||||||
// Two routers are built over separate dependency graphs and a third
|
|
||||||
// over the first graph again, and each must still serve the HTTP,
|
|
||||||
// delivery, Go runtime and process series, and the series counting
|
|
||||||
// scrapes of /metrics itself.
|
|
||||||
func TestTwoMetricsRoutersInOneProcess(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
first := newTestEnvWithConfig(
|
|
||||||
t, metricsConfig(t, metricsUser, metricsAuthValue),
|
|
||||||
)
|
|
||||||
second := newTestEnvWithConfig(
|
|
||||||
t, metricsConfig(t, metricsUser, metricsAuthValue),
|
|
||||||
)
|
|
||||||
third := &testEnv{
|
|
||||||
router: server.NewRouterForTest(
|
|
||||||
first.log.Get(), first.cfg, first.mw, first.hnd,
|
|
||||||
),
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, env := range []*testEnv{first, second, third} {
|
|
||||||
env.get("/", nil)
|
|
||||||
|
|
||||||
scrape := env.metricsRequest(metricsUser, metricsAuthValue)
|
|
||||||
require.Equal(t, http.StatusOK, scrape.Code)
|
|
||||||
|
|
||||||
for _, series := range []string{
|
|
||||||
"http_request_duration_seconds",
|
|
||||||
"http_response_size_bytes",
|
|
||||||
"http_requests_inflight",
|
|
||||||
"webhooker_events_received_total",
|
|
||||||
"go_goroutines",
|
|
||||||
"process_start_time_seconds",
|
|
||||||
"promhttp_metric_handler_requests_total",
|
|
||||||
} {
|
|
||||||
assert.Contains(t, scrape.Body.String(), series)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -115,8 +115,8 @@ func TestVersion_EnclosingRepositoryIsNotUsed(t *testing.T) {
|
|||||||
require.Equal(t, unknown, runScript(t, inner, nil))
|
require.Equal(t, unknown, runScript(t, inner, nil))
|
||||||
}
|
}
|
||||||
|
|
||||||
// The Docker build has no git metadata, so the version arrives as an
|
// An explicit VERSION, such as the Dockerfile's build arg, wins over
|
||||||
// environment override. It wins over anything derivable.
|
// anything derivable.
|
||||||
func TestVersion_EnvironmentOverrideWins(t *testing.T) {
|
func TestVersion_EnvironmentOverrideWins(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -128,8 +128,8 @@ func TestVersion_EnvironmentOverrideWins(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// An empty VERSION is treated as unset rather than stamping an empty
|
// An empty VERSION is treated as unset rather than stamping an empty
|
||||||
// string: the Dockerfile's build arg has a non-empty default, but a
|
// string: a caller exporting VERSION= must not produce a binary
|
||||||
// caller exporting VERSION= must not produce a binary reporting "".
|
// reporting "".
|
||||||
func TestVersion_EmptyOverrideFallsBackToGit(t *testing.T) {
|
func TestVersion_EmptyOverrideFallsBackToGit(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -168,8 +168,8 @@ func TestMakefile_BuildComposesVersionAndExtraFlags(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// A caller can define VERSION as the empty string -- `make build
|
// A caller can define VERSION as the empty string -- `make build
|
||||||
// VERSION=`, or a `--build-arg VERSION=` reaching the Dockerfile's `make
|
// VERSION=`, or the Dockerfile's `make build VERSION="$VERSION"` when no
|
||||||
// build VERSION="$VERSION"`. script/version's own guard does not cover
|
// VERSION build arg was given. script/version's own guard does not cover
|
||||||
// that: the value never passes through the script. Stamping "" would
|
// that: the value never passes through the script. Stamping "" would
|
||||||
// leave the binary reporting no version and the footer on "dev", which
|
// leave the binary reporting no version and the footer on "dev", which
|
||||||
// is the defect this package exists for.
|
// is the defect this package exists for.
|
||||||
@@ -231,7 +231,7 @@ func TestDockerfile_BuildsThroughTheMakeTarget(t *testing.T) {
|
|||||||
|
|
||||||
require.NotContains(t, dockerfile, "go build",
|
require.NotContains(t, dockerfile, "go build",
|
||||||
"a raw go build bypasses the Makefile's -X flag")
|
"a raw go build bypasses the Makefile's -X flag")
|
||||||
require.Contains(t, dockerfile, "ARG VERSION=")
|
require.Contains(t, dockerfile, "ARG VERSION")
|
||||||
require.Contains(t, dockerfile,
|
require.Contains(t, dockerfile,
|
||||||
`make build VERSION="$VERSION" GO_LDFLAGS='-extldflags "-static"'`)
|
`make build VERSION="$VERSION" GO_LDFLAGS='-extldflags "-static"'`)
|
||||||
}
|
}
|
||||||
|
|||||||
+3
-3
@@ -2,9 +2,9 @@
|
|||||||
# script/docker: build the Docker image tagged with the project name.
|
# script/docker: build the Docker image tagged with the project name.
|
||||||
# The tag comes from script/projectname.
|
# The tag comes from script/projectname.
|
||||||
#
|
#
|
||||||
# .dockerignore excludes .git/, so the builder stage cannot derive the
|
# The version script/version resolves here goes in as the VERSION build
|
||||||
# version itself. It is resolved here, where the checkout is, and passed
|
# arg, which takes precedence over what the build would derive from the
|
||||||
# in as a build arg; without it the image would stamp itself "unknown".
|
# .git in its context.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
|
|||||||
+5
-7
@@ -7,18 +7,16 @@
|
|||||||
#
|
#
|
||||||
# Order of precedence:
|
# Order of precedence:
|
||||||
#
|
#
|
||||||
# 1. $VERSION, if set and non-empty. This is how the value reaches a
|
# 1. $VERSION, if set and non-empty: an explicit value, such as the
|
||||||
# build that cannot derive it: .dockerignore excludes .git/, so the
|
# Dockerfile's VERSION build arg.
|
||||||
# builder stage has no git metadata and the Dockerfile takes the
|
|
||||||
# value as a build arg instead.
|
|
||||||
# 2. `git describe --tags --always --dirty` against this checkout. At
|
# 2. `git describe --tags --always --dirty` against this checkout. At
|
||||||
# a clean tagged commit that is exactly the tag; otherwise it
|
# a clean tagged commit that is exactly the tag; otherwise it
|
||||||
# carries the short SHA, the commit distance when a tag is
|
# carries the short SHA, the commit distance when a tag is
|
||||||
# reachable, and a -dirty suffix for uncommitted changes.
|
# reachable, and a -dirty suffix for uncommitted changes.
|
||||||
# 3. "unknown", for a tree with no git metadata and no $VERSION -- a
|
# 3. "unknown", for a tree with no git metadata and no $VERSION -- a
|
||||||
# source tarball, or `docker build .` with no --build-arg. That
|
# source tarball, or a `docker build` with no .git in its context
|
||||||
# case must not fail the build and must not name a tag the tree may
|
# and no VERSION build arg. That case must not fail the build and
|
||||||
# not be at, so it names nothing.
|
# must not name a tag the tree may not be at, so it names nothing.
|
||||||
#
|
#
|
||||||
# The git step insists the enclosing repository is this checkout, not
|
# The git step insists the enclosing repository is this checkout, not
|
||||||
# merely some repository above it: an unpacked tarball sitting inside an
|
# merely some repository above it: an unpacked tarball sitting inside an
|
||||||
|
|||||||
Reference in New Issue
Block a user