Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3699f37b76 | ||
|
|
b78abdc9da | ||
|
|
c23ffbac65 | ||
|
|
2ac4d4d793 |
@@ -158,19 +158,20 @@ WireServer, which serves an Azure VM its credentials. Because it is a
|
||||
public address, listing it in `ALLOWED_EGRESS_CIDRS` reopens it.
|
||||
|
||||
That is all the default blocklist covers: the IPv4 private and reserved
|
||||
ranges; of IPv6, only loopback (`::1`), unique local addresses
|
||||
(`fc00::/7`) and link-local addresses (`fe80::/10`); and certain public
|
||||
addresses. A public address belongs on the default blocklist only if it
|
||||
hands credentials, user data or bootstrap material to whatever can reach
|
||||
it, without the caller presenting anything. A provider's other public
|
||||
addresses are not refused. IBM Cloud, for example, serves its package
|
||||
mirrors, time servers and object storage on `161.26.0.0/16`, and the
|
||||
private endpoints of its own cloud services on `166.8.0.0/14`. Neither
|
||||
range hands out credentials that way: the token service among those
|
||||
endpoints issues a token only in exchange for something the caller
|
||||
presents, such as an API key. Reaching these services can be a
|
||||
legitimate delivery, and every cloud has some, so a partial list would
|
||||
promise coverage it does not give.
|
||||
ranges; of IPv6, only loopback (`::1`), the unspecified address (`::`),
|
||||
unique local addresses (`fc00::/7`), link-local addresses (`fe80::/10`),
|
||||
multicast (`ff00::/8`) and documentation space (`2001:db8::/32`); and
|
||||
certain public addresses. A public address belongs on the default
|
||||
blocklist only if it hands credentials, user data or bootstrap material
|
||||
to whatever can reach it, without the caller presenting anything. A
|
||||
provider's other public addresses are not refused. IBM Cloud, for
|
||||
example, serves its package mirrors, time servers and object storage on
|
||||
`161.26.0.0/16`, and the private endpoints of its own cloud services on
|
||||
`166.8.0.0/14`. Neither range hands out credentials that way: the token
|
||||
service among those endpoints issues a token only in exchange for
|
||||
something the caller presents, such as an API key. Reaching these
|
||||
services can be a legitimate delivery, and every cloud has some, so a
|
||||
partial list would promise coverage it does not give.
|
||||
|
||||
That default is also inconvenient for the thing webhooker is mostly
|
||||
for: taking a public webhook and forwarding it to something on your own
|
||||
@@ -210,16 +211,16 @@ Two things this setting cannot do:
|
||||
the list is always an allowlist; an empty list (the default) means
|
||||
every private and reserved range stays refused. Note that
|
||||
`0.0.0.0/0` gets you most of the way there anyway, per above.
|
||||
- **It cannot open link-local, or a cloud metadata endpoint at a
|
||||
non-public address that discloses credentials or user data.** An
|
||||
address is on the list below when it is not a public address and both
|
||||
of these hold: the provider fixes it, so it cannot collide with
|
||||
anything you run; and reaching it hands out credentials, user data or
|
||||
bootstrap material. Those stay blocked no matter what you list,
|
||||
including when you list them outright or list a supernet such as
|
||||
`0.0.0.0/0`, `::/0`, `fd00::/8` or `100.64.0.0/10`. Treat this as best
|
||||
effort rather than a guarantee — it is a hand-maintained list and the
|
||||
caveat below the table applies:
|
||||
- **It cannot open link-local, the unspecified addresses, or a cloud
|
||||
metadata endpoint at a non-public address that discloses credentials
|
||||
or user data.** A metadata address is on the list below when it is not
|
||||
a public address and both of these hold: the provider fixes it, so it
|
||||
cannot collide with anything you run; and reaching it hands out
|
||||
credentials, user data or bootstrap material. Those stay blocked no
|
||||
matter what you list, including when you list them outright or list a
|
||||
supernet such as `0.0.0.0/0`, `::/0`, `fd00::/8` or `100.64.0.0/10`.
|
||||
Treat this as best effort rather than a guarantee — it is a
|
||||
hand-maintained list and the caveat below the table applies:
|
||||
|
||||
| Blocked unconditionally | What it is |
|
||||
| ----------------------- | ---------- |
|
||||
@@ -233,14 +234,25 @@ Two things this setting cannot do:
|
||||
| `fd00:a9fe:a9fe::1/128` | Linode/Akamai metadata over IPv6 |
|
||||
| `100.100.100.200/32` | Alibaba Cloud metadata, inside CGNAT |
|
||||
| `192.0.0.192/32` | Oracle Cloud Classic metadata |
|
||||
| `0.0.0.0/32` | IPv4 unspecified address, which reaches this host's loopback on Linux |
|
||||
| `::/128` | IPv6 unspecified address, which reaches this host's loopback on Linux |
|
||||
| `::a9fe:a9fe/128` | `169.254.169.254` as an IPv4-compatible IPv6 address |
|
||||
| `64:ff9b::a9fe:a9fe/128` | `169.254.169.254` behind the NAT64 well-known prefix |
|
||||
|
||||
The IPv4-mapped form `::ffff:169.254.169.254` is covered by the
|
||||
`169.254.0.0/16` entry. Reaching any of these is credential or
|
||||
user-data theft rather than delivery to an internal service. Every
|
||||
entry outside the two link-local blocks is a single address, so
|
||||
blocking it costs you nothing else on the network around it.
|
||||
`169.254.0.0/16` entry. Reaching any of these but the two unspecified
|
||||
addresses is credential or user-data theft rather than delivery to an
|
||||
internal service. Every entry outside the two link-local blocks is a
|
||||
single address, so blocking it costs you nothing else on the network
|
||||
around it.
|
||||
|
||||
The unspecified addresses `0.0.0.0` and `::` hand out nothing
|
||||
themselves, but no host can have either, and on Linux a connection to
|
||||
one reaches this host's own loopback. They are listed so that an
|
||||
allowlist reaches loopback only through an entry that covers a loopback
|
||||
address, such as `127.0.0.0/8`, `::1` or `0.0.0.0/0`, never through one
|
||||
that covers only `0.0.0.0` or `::`; `0.0.0.0/8`, for example, does not
|
||||
open loopback.
|
||||
|
||||
The six ULA entries, all inside `fd00::/8`, are why this matters in
|
||||
practice: `fd00::/8` is an ordinary block to allowlist for your own
|
||||
@@ -1768,7 +1780,7 @@ retries) is individually logged for full observability.
|
||||
#### EventTotals and TargetTotals
|
||||
|
||||
Running counts in each event database, read by the statistics pane at the
|
||||
top of the webhook page and by the webhook list. `EventTotals` is one row:
|
||||
top of the webhook page. `EventTotals` is one row:
|
||||
|
||||
| Field | Type | Description |
|
||||
| ---------------- | --------- | ----------- |
|
||||
@@ -1800,14 +1812,6 @@ target. Its failure percentage for a window is the deliveries that became
|
||||
`failed` in it out of all that became `delivered` or `failed` in it, and
|
||||
a dash when none did.
|
||||
|
||||
The webhook list at `/hooks` shows three of the pane's figures for each
|
||||
webhook: its events within retention and its last event, both from
|
||||
`EventTotals`, and its deliveries that failed in the last 24 hours,
|
||||
counted with the pane's query. It opens each webhook's event database once
|
||||
(the handle stays open) and runs those two reads there, so its cost grows
|
||||
with the number of webhooks and, for each, with the deliveries that
|
||||
finished in the last 24 hours, never with the events stored.
|
||||
|
||||
#### Event-tier indexes
|
||||
|
||||
These indexes on the per-webhook event databases are declared in the model
|
||||
@@ -1815,7 +1819,7 @@ tags, so `AutoMigrate` creates them on a fresh database:
|
||||
|
||||
| Table | Columns | Serves |
|
||||
| ------------------ | --------------------------- | ------ |
|
||||
| `deliveries` | `status`, `deleted_at`, `finished_at`, `target_id` | Startup recovery, the retry and pending sweeps every 60 seconds and the queue-depth sampler every 30 seconds, which select deliveries by status, and the webhook page's statistics and the webhook list, which count each target's deliveries by status and when they finished |
|
||||
| `deliveries` | `status`, `deleted_at`, `finished_at`, `target_id` | Startup recovery, the retry and pending sweeps every 60 seconds and the queue-depth sampler every 30 seconds, which select deliveries by status, and the webhook page's statistics, which count each target's deliveries by status and when they finished |
|
||||
| `deliveries` | `event_id`, `deleted_at` | The event log, which loads each event's deliveries, and retention, which counts and deletes the deliveries of expired events |
|
||||
| `delivery_results` | `delivery_id`, `deleted_at` | The event log, which loads the attempts of a page's deliveries, and retention, which deletes the attempts of expired events |
|
||||
| `events` | `deleted_at`, `created_at` | The webhook page's statistics, which count recent events |
|
||||
@@ -3101,7 +3105,8 @@ check, see [The login endpoint](#the-login-endpoint).
|
||||
route through a single decision function, so they cannot disagree
|
||||
about a destination. An operator can permit specific blocks with
|
||||
[`ALLOWED_EGRESS_CIDRS`](#allowing-egress-to-your-own-network); the
|
||||
guard cannot be switched off, and link-local plus a
|
||||
guard cannot be switched off, and link-local, the unspecified
|
||||
addresses `0.0.0.0` and `::`, and a
|
||||
[pinned set](#allowing-egress-to-your-own-network) of known cloud
|
||||
metadata endpoints — several of which are ULAs outside link-local —
|
||||
stay blocked whatever is listed, though listing `0.0.0.0/0` or
|
||||
|
||||
@@ -40,12 +40,6 @@ duplicate. That is deliberate — the alternative is a silent lost
|
||||
delivery — and the README says so under Rationale. It is not a defect
|
||||
to re-file.
|
||||
|
||||
One caveat on reading a green check: a docs-only commit deliberately
|
||||
replays from the layer cache
|
||||
(https://git.eeqj.de/sneak/webhooker/issues/119), so a green status on
|
||||
such a commit evidences a replay rather than an executed run. A code
|
||||
commit invalidates the `COPY` layer and genuinely executes.
|
||||
|
||||
# Next Step
|
||||
|
||||
Clear the rest of the open 1.0.0 milestone
|
||||
|
||||
@@ -196,10 +196,11 @@ type Config struct {
|
||||
// otherwise refuse. The guard itself is always on: there is no
|
||||
// setting that disables SSRF protection, and delivery's
|
||||
// alwaysBlockedNetworks stays blocked no matter what is listed
|
||||
// here. That set is link-local plus the cloud metadata
|
||||
// endpoints outside it that disclose credentials or user data
|
||||
// at a provider-fixed, non-public address; it is not
|
||||
// exhaustive of every cloud's metadata address. See
|
||||
// here. That set is link-local, the unspecified addresses
|
||||
// 0.0.0.0 and ::, and the cloud metadata endpoints outside
|
||||
// link-local that disclose credentials or user data at a
|
||||
// provider-fixed, non-public address; it is not exhaustive of
|
||||
// every cloud's metadata address. See
|
||||
// alwaysBlockedNetworks for the authoritative list and the
|
||||
// criterion it is built from.
|
||||
AllowedEgressCIDRs []netip.Prefix
|
||||
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"go.uber.org/fx"
|
||||
"gorm.io/gorm"
|
||||
"sneak.berlin/go/webhooker/internal/database"
|
||||
"sneak.berlin/go/webhooker/internal/globals"
|
||||
"sneak.berlin/go/webhooker/internal/lifecycle"
|
||||
"sneak.berlin/go/webhooker/internal/logger"
|
||||
"sneak.berlin/go/webhooker/internal/metrics"
|
||||
@@ -146,6 +147,7 @@ type EngineParams struct {
|
||||
|
||||
DB *database.Database
|
||||
DBManager *database.WebhookDBManager
|
||||
Globals *globals.Globals
|
||||
Logger *logger.Logger
|
||||
SSRFGuard *Guard
|
||||
Metrics *metrics.Set
|
||||
@@ -168,6 +170,10 @@ type Engine struct {
|
||||
retryCh chan Task
|
||||
workers int
|
||||
|
||||
// version is the running build's version, the one the web UI
|
||||
// footer shows. userAgent puts it on every outbound request.
|
||||
version string
|
||||
|
||||
// mtr is the delivery metric set. Production wires the one
|
||||
// registered on the registry /metrics serves; a test can
|
||||
// substitute a set registered on a registry it holds, so it can
|
||||
@@ -205,6 +211,7 @@ func New(
|
||||
deliveryCh: make(chan Task, deliveryChannelSize),
|
||||
retryCh: make(chan Task, retryChannelSize),
|
||||
workers: defaultWorkers,
|
||||
version: params.Globals.Version,
|
||||
mtr: params.Metrics,
|
||||
}
|
||||
|
||||
@@ -301,6 +308,13 @@ func (e *Engine) ScheduleRetry(
|
||||
})
|
||||
}
|
||||
|
||||
// userAgent is the User-Agent header of every http and slack
|
||||
// delivery request: the program name and the running build's
|
||||
// version.
|
||||
func (e *Engine) userAgent() string {
|
||||
return "webhooker/" + e.version
|
||||
}
|
||||
|
||||
// registerHooks wires the engine's start and stop into the fx
|
||||
// lifecycle. The start hook's context is deliberately ignored
|
||||
// (see start for why the worker pool must not inherit it); the
|
||||
|
||||
@@ -1247,11 +1247,6 @@ func TestDoHTTPRequest_ForwardsHeaders(t *testing.T) {
|
||||
testContentType,
|
||||
receivedHeaders.Get("Content-Type"),
|
||||
)
|
||||
|
||||
assert.Equal(t,
|
||||
"webhooker/1.0",
|
||||
receivedHeaders.Get("User-Agent"),
|
||||
)
|
||||
}
|
||||
|
||||
// The event's stored inbound headers carry the same Content-Type the
|
||||
@@ -1320,6 +1315,7 @@ func TestApplyRequestHeaders_SendsOneContentType(t *testing.T) {
|
||||
ContentType: tc.event,
|
||||
},
|
||||
cfg,
|
||||
"webhooker/dev",
|
||||
)
|
||||
|
||||
assert.Equal(t,
|
||||
|
||||
@@ -83,8 +83,9 @@ func ExportApplyRequestHeaders(
|
||||
req *http.Request,
|
||||
event *database.Event,
|
||||
cfg *HTTPTargetConfig,
|
||||
userAgent string,
|
||||
) []string {
|
||||
return applyRequestHeaders(req, event, cfg)
|
||||
return applyRequestHeaders(req, event, cfg, userAgent)
|
||||
}
|
||||
|
||||
// ExportTruncate exposes truncate for testing.
|
||||
|
||||
@@ -375,6 +375,7 @@ func TestApplyRequestHeaders_ReportsOriginScopedNames(t *testing.T) {
|
||||
"Content-Type": testContentType,
|
||||
},
|
||||
},
|
||||
"webhooker/dev",
|
||||
)
|
||||
|
||||
assert.Equal(t,
|
||||
|
||||
+53
-10
@@ -37,8 +37,8 @@ var (
|
||||
"blocked cloud metadata address",
|
||||
)
|
||||
errBlockedMetadata = errors.New(
|
||||
"blocked link-local or cloud instance metadata " +
|
||||
"address: ALLOWED_EGRESS_CIDRS cannot open it",
|
||||
"blocked link-local, cloud instance metadata or " +
|
||||
"unspecified address: ALLOWED_EGRESS_CIDRS cannot open it",
|
||||
)
|
||||
errInvalidScheme = errors.New(
|
||||
"only http and https are allowed",
|
||||
@@ -72,14 +72,16 @@ var blockedNetworks []*net.IPNet
|
||||
var blockedPublicNetworks []*net.IPNet
|
||||
|
||||
// alwaysBlockedNetworks are the ranges no configuration can
|
||||
// open: the link-local blocks and the cloud instance metadata
|
||||
// endpoints that live outside them. Reaching one is credential
|
||||
// or user-data theft rather than delivery to an internal
|
||||
// service, so a supplied CIDR that covers such an address still
|
||||
// leaves it blocked.
|
||||
// open: the link-local blocks, the cloud instance metadata
|
||||
// endpoints that live outside them, and the unspecified
|
||||
// addresses. Reaching a metadata endpoint is credential or
|
||||
// user-data theft rather than delivery to an internal service,
|
||||
// so a supplied CIDR that covers such an address still leaves it
|
||||
// blocked.
|
||||
//
|
||||
// Inclusion criterion — an address belongs here only if BOTH
|
||||
// hold, and every entry below satisfies both:
|
||||
// hold, and every entry below but the unspecified addresses
|
||||
// satisfies both:
|
||||
//
|
||||
// 1. It is a fixed address assigned by the provider, or a
|
||||
// range reserved by IANA — never one the operator chose.
|
||||
@@ -112,6 +114,15 @@ var blockedPublicNetworks []*net.IPNet
|
||||
// This is a criterion, not an enumeration of every metadata
|
||||
// address in existence.
|
||||
//
|
||||
// The unspecified addresses 0.0.0.0 and :: fail (2) and are here
|
||||
// anyway. No host can have either, and on Linux a connection to
|
||||
// one reaches this host's own loopback. Listing them means an
|
||||
// allowlist reaches loopback only through an entry that covers a
|
||||
// loopback address (127.0.0.0/8, ::1/128, 0.0.0.0/0), never
|
||||
// through one that covers only 0.0.0.0 or :: (0.0.0.0/8, for
|
||||
// example). Nothing else lives at either address, so refusing
|
||||
// them costs nothing.
|
||||
//
|
||||
// Every entry is either already in blockedNetworks — this list is
|
||||
// what makes it unconditional — or an alternate encoding of
|
||||
// 169.254.169.254 that Contains does not match against
|
||||
@@ -131,23 +142,46 @@ var alwaysBlockedNetworks []*net.IPNet
|
||||
//nolint:gochecknoinits // init is the idiomatic way to parse CIDRs once at startup
|
||||
func init() {
|
||||
blockedNetworks = mustParseCIDRs([]string{
|
||||
// IPv4 loopback.
|
||||
"127.0.0.0/8",
|
||||
// RFC 1918 private network.
|
||||
"10.0.0.0/8",
|
||||
// RFC 1918 private network.
|
||||
"172.16.0.0/12",
|
||||
// RFC 1918 private network.
|
||||
"192.168.0.0/16",
|
||||
// IPv4 link-local.
|
||||
"169.254.0.0/16",
|
||||
// "This network", holding the IPv4 unspecified address 0.0.0.0.
|
||||
"0.0.0.0/8",
|
||||
// Carrier-grade NAT shared address space.
|
||||
"100.64.0.0/10",
|
||||
// IETF protocol assignments.
|
||||
"192.0.0.0/24",
|
||||
// IPv4 documentation (TEST-NET-1).
|
||||
"192.0.2.0/24",
|
||||
// Benchmarking.
|
||||
"198.18.0.0/15",
|
||||
// IPv4 documentation (TEST-NET-2).
|
||||
"198.51.100.0/24",
|
||||
// IPv4 documentation (TEST-NET-3).
|
||||
"203.0.113.0/24",
|
||||
// IPv4 multicast.
|
||||
"224.0.0.0/4",
|
||||
// Reserved, including the broadcast address.
|
||||
"240.0.0.0/4",
|
||||
// IPv6 loopback.
|
||||
"::1/128",
|
||||
// IPv6 unspecified address.
|
||||
"::/128",
|
||||
// IPv6 unique local addresses.
|
||||
"fc00::/7",
|
||||
// IPv6 link-local.
|
||||
"fe80::/10",
|
||||
// IPv6 multicast.
|
||||
"ff00::/8",
|
||||
// IPv6 documentation.
|
||||
"2001:db8::/32",
|
||||
})
|
||||
|
||||
blockedPublicNetworks = mustParseCIDRs([]string{
|
||||
@@ -207,6 +241,14 @@ func init() {
|
||||
// allowlist from opening it.
|
||||
"192.0.0.192/32",
|
||||
|
||||
// The unspecified addresses, each of which reaches this
|
||||
// host's loopback on Linux.
|
||||
//
|
||||
// IPv4 unspecified address, inside the blocked 0.0.0.0/8.
|
||||
"0.0.0.0/32",
|
||||
// IPv6 unspecified address.
|
||||
"::/128",
|
||||
|
||||
// 169.254.169.254 as an IPv4-compatible IPv6 address.
|
||||
"::a9fe:a9fe/128",
|
||||
// 169.254.169.254 behind the NAT64 well-known prefix.
|
||||
@@ -343,8 +385,9 @@ func (g *Guard) allows(ip net.IP) bool {
|
||||
// The order is the policy:
|
||||
//
|
||||
// 1. alwaysBlockedNetworks is refused before the allowlist is
|
||||
// consulted, so no configured CIDR reaches link-local or a
|
||||
// cloud metadata endpoint at a non-public address.
|
||||
// consulted, so no configured CIDR reaches link-local, a
|
||||
// cloud metadata endpoint at a non-public address, or an
|
||||
// unspecified address.
|
||||
// 2. The allowlist is consulted next, so a listed private
|
||||
// network, or a listed public address on the default
|
||||
// blocklist, becomes reachable.
|
||||
|
||||
@@ -168,12 +168,13 @@ func TestGuardAllowlist_UnlistedPrivateStillRefused(t *testing.T) {
|
||||
|
||||
// TestGuardAllowlist_MetadataAlwaysRefused is the load-bearing
|
||||
// case: cloud instance metadata endpoints are credential theft
|
||||
// rather than delivery to an internal service, so no allowlist
|
||||
// reaches one. Every guard below names a CIDR that covers its
|
||||
// target — including 0.0.0.0/0, ::/0, and the ordinary ULA and
|
||||
// CGNAT blocks an operator would really list — and the address
|
||||
// must stay refused anyway, on both the validation and the
|
||||
// delivery path.
|
||||
// rather than delivery to an internal service, and the
|
||||
// unspecified addresses 0.0.0.0 and :: reach this host's loopback
|
||||
// on Linux, so no allowlist reaches any of them. Every guard
|
||||
// below names a CIDR that covers its target — including
|
||||
// 0.0.0.0/0, ::/0, and the ordinary ULA and CGNAT blocks an
|
||||
// operator would really list — and the address must stay
|
||||
// refused anyway, on both the validation and the delivery path.
|
||||
func TestGuardAllowlist_MetadataAlwaysRefused(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -219,15 +220,17 @@ type metadataAlwaysRefusedCase struct {
|
||||
}
|
||||
|
||||
// metadataAlwaysRefusedCases enumerates every unconditionally
|
||||
// blocked address together with an allowlist entry that would
|
||||
// otherwise reach it. Split by family of address only to stay
|
||||
// under the function-length limit.
|
||||
// blocked address (link-local, the cloud metadata endpoints and
|
||||
// the unspecified addresses) together with an allowlist entry
|
||||
// that would otherwise reach it. Split by family of address only
|
||||
// to stay under the function-length limit.
|
||||
func metadataAlwaysRefusedCases() []metadataAlwaysRefusedCase {
|
||||
cases := linkLocalRefusedCases()
|
||||
cases = append(cases, ulaMetadataRefusedCases()...)
|
||||
cases = append(cases, ipv4MetadataRefusedCases()...)
|
||||
cases = append(cases, encodedMetadataRefusedCases()...)
|
||||
|
||||
return append(cases, encodedMetadataRefusedCases()...)
|
||||
return append(cases, unspecifiedRefusedCases()...)
|
||||
}
|
||||
|
||||
// linkLocalRefusedCases covers the link-local blocks, including
|
||||
@@ -367,6 +370,23 @@ func encodedMetadataRefusedCases() []metadataAlwaysRefusedCase {
|
||||
}
|
||||
}
|
||||
|
||||
// unspecifiedRefusedCases covers the unspecified addresses, each
|
||||
// of which reaches this host's loopback on Linux.
|
||||
func unspecifiedRefusedCases() []metadataAlwaysRefusedCase {
|
||||
return []metadataAlwaysRefusedCase{
|
||||
{
|
||||
name: "IPv4 unspecified address under 0.0.0.0/0",
|
||||
allow: allowAllIPv4,
|
||||
target: "http://0.0.0.0:8080/hook",
|
||||
},
|
||||
{
|
||||
name: "IPv6 unspecified address under ::/0",
|
||||
allow: allowAllIPv6,
|
||||
target: "http://[::]:8080/hook",
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// TestGuardAllowlist_PublicUnaffected asserts the allowlist does
|
||||
// not narrow anything: public addresses were reachable before it
|
||||
// existed and stay reachable, whether or not a list is set.
|
||||
@@ -524,6 +544,10 @@ func TestAlwaysBlockedNetworks_PinnedSet(t *testing.T) {
|
||||
// Oracle Cloud Classic metadata, inside the blocked
|
||||
// 192.0.0.0/24.
|
||||
"192.0.0.192/32",
|
||||
// The IPv4 and IPv6 unspecified addresses, each of
|
||||
// which reaches this host's loopback on Linux.
|
||||
"0.0.0.0/32",
|
||||
"::/128",
|
||||
// 169.254.169.254 as an IPv4-compatible IPv6 address.
|
||||
"::a9fe:a9fe/128",
|
||||
// 169.254.169.254 behind the NAT64 well-known prefix.
|
||||
@@ -556,7 +580,8 @@ func TestDefaultBlocklist_PinnedSet(t *testing.T) {
|
||||
{cidr: "172.16.0.0/12", reopenable: true},
|
||||
{cidr: "192.168.0.0/16", reopenable: true},
|
||||
{cidr: linkLocalIPv4, reopenable: false},
|
||||
{cidr: "0.0.0.0/8", reopenable: true},
|
||||
// Its first address, 0.0.0.0, is in the unconditional set.
|
||||
{cidr: "0.0.0.0/8", reopenable: false},
|
||||
{cidr: "100.64.0.0/10", reopenable: true},
|
||||
{cidr: "192.0.0.0/24", reopenable: true},
|
||||
{cidr: "192.0.2.0/24", reopenable: true},
|
||||
@@ -566,8 +591,11 @@ func TestDefaultBlocklist_PinnedSet(t *testing.T) {
|
||||
{cidr: "224.0.0.0/4", reopenable: true},
|
||||
{cidr: "240.0.0.0/4", reopenable: true},
|
||||
{cidr: "::1/128", reopenable: true},
|
||||
{cidr: "::/128", reopenable: false},
|
||||
{cidr: "fc00::/7", reopenable: true},
|
||||
{cidr: "fe80::/10", reopenable: false},
|
||||
{cidr: "ff00::/8", reopenable: true},
|
||||
{cidr: "2001:db8::/32", reopenable: true},
|
||||
{cidr: "168.63.129.16/32", public: true, reopenable: true},
|
||||
}
|
||||
|
||||
|
||||
@@ -101,6 +101,42 @@ func TestValidateTargetURL_Blocked(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestDefaultGuard_RefusesUnspecifiedMulticastAndDocumentation
|
||||
// covers the unspecified addresses and the IPv6 multicast and
|
||||
// documentation ranges: with no allowlist set, each is refused
|
||||
// both when a target is created and when a delivery dials it.
|
||||
func TestDefaultGuard_RefusesUnspecifiedMulticastAndDocumentation(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
guard := delivery.NewTestGuard()
|
||||
|
||||
targets := []string{
|
||||
// The unspecified addresses. On Linux a connection to
|
||||
// either reaches this host's loopback.
|
||||
"http://0.0.0.0:8080/hook",
|
||||
"http://[::]:8080/hook",
|
||||
// IPv6 multicast, all nodes.
|
||||
"http://[ff02::1]/hook",
|
||||
// IPv6 documentation.
|
||||
"http://[2001:db8::1]/hook",
|
||||
}
|
||||
|
||||
for _, target := range targets {
|
||||
t.Run(target, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
require.Error(t,
|
||||
guard.ValidateTargetURL(context.Background(), target),
|
||||
"%s must be refused at target creation", target,
|
||||
)
|
||||
|
||||
assertDialRefused(t, guard, target)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateTargetURL_Allowed(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
@@ -442,7 +442,9 @@ func (t *httpTarget) doHTTPRequest(
|
||||
)
|
||||
}
|
||||
|
||||
originScoped := applyRequestHeaders(req, event, cfg)
|
||||
originScoped := applyRequestHeaders(
|
||||
req, event, cfg, t.eng.userAgent(),
|
||||
)
|
||||
|
||||
client := t.clientForRequest(cfg, originScoped)
|
||||
|
||||
@@ -562,10 +564,13 @@ func isForwardableHeader(name string) bool {
|
||||
// Content-Type goes out once: a Content-Type configured on the target
|
||||
// wins, otherwise the event's ContentType, otherwise none. The inbound
|
||||
// Content-Type in the event's headers is never forwarded.
|
||||
//
|
||||
// userAgent is set last, over any configured or inbound User-Agent.
|
||||
func applyRequestHeaders(
|
||||
req *http.Request,
|
||||
event *database.Event,
|
||||
cfg *HTTPTargetConfig,
|
||||
userAgent string,
|
||||
) []string {
|
||||
if event.ContentType != "" {
|
||||
req.Header.Set(
|
||||
@@ -580,7 +585,7 @@ func applyRequestHeaders(
|
||||
originScoped[http.CanonicalHeaderKey(k)] = struct{}{}
|
||||
}
|
||||
|
||||
req.Header.Set("User-Agent", "webhooker/1.0")
|
||||
req.Header.Set("User-Agent", userAgent)
|
||||
|
||||
// A Content-Type configured on the target describes the body
|
||||
// being sent rather than the sender. A 307/308 preserves the
|
||||
|
||||
@@ -136,7 +136,7 @@ func (t *slackTarget) attempt(
|
||||
}
|
||||
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("User-Agent", "webhooker/1.0")
|
||||
req.Header.Set("User-Agent", t.eng.userAgent())
|
||||
|
||||
resp, doErr := executeHTTPRequest(t.client, req)
|
||||
durationMs := time.Since(start).Milliseconds()
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
package delivery_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/netip"
|
||||
"testing"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/prometheus/client_golang/prometheus"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"go.uber.org/fx/fxtest"
|
||||
"sneak.berlin/go/webhooker/internal/database"
|
||||
"sneak.berlin/go/webhooker/internal/delivery"
|
||||
"sneak.berlin/go/webhooker/internal/globals"
|
||||
"sneak.berlin/go/webhooker/internal/logger"
|
||||
"sneak.berlin/go/webhooker/internal/metrics"
|
||||
)
|
||||
|
||||
// Both the http and the slack target send webhooker/ and the version
|
||||
// in Globals, the value the web UI footer shows. A User-Agent
|
||||
// configured on the target or carried in by the sender does not
|
||||
// replace it.
|
||||
func TestUserAgent_IsTheBuildVersion(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const want = "webhooker/1.2.3-test"
|
||||
|
||||
userAgents := make(chan string, 1)
|
||||
|
||||
ts := httptest.NewServer(http.HandlerFunc(
|
||||
func(w http.ResponseWriter, r *http.Request) {
|
||||
userAgents <- r.Header.Get("User-Agent")
|
||||
|
||||
w.WriteHeader(http.StatusOK)
|
||||
},
|
||||
))
|
||||
defer ts.Close()
|
||||
|
||||
g := &globals.Globals{Version: "1.2.3-test"}
|
||||
lc := fxtest.NewLifecycle(t)
|
||||
|
||||
log, err := logger.New(lc, logger.LoggerParams{Globals: g})
|
||||
require.NoError(t, err)
|
||||
|
||||
e := delivery.New(lc, delivery.EngineParams{
|
||||
Globals: g,
|
||||
Logger: log,
|
||||
// httptest listens on loopback, which the default guard
|
||||
// refuses.
|
||||
SSRFGuard: delivery.NewTestGuard(
|
||||
netip.MustParsePrefix("127.0.0.0/8"),
|
||||
),
|
||||
Metrics: metrics.New(prometheus.NewRegistry()),
|
||||
})
|
||||
|
||||
statusCode, _, _, err := e.ExportDoHTTPRequest(
|
||||
context.Background(),
|
||||
&delivery.HTTPTargetConfig{
|
||||
URL: ts.URL,
|
||||
Headers: map[string]string{"User-Agent": "configured/1"},
|
||||
},
|
||||
&database.Event{Headers: `{"User-Agent":["curl/8"]}`},
|
||||
)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, http.StatusOK, statusCode)
|
||||
require.Len(t, userAgents, 1, "the http target sent no request")
|
||||
assert.Equal(t, want, <-userAgents, "http target")
|
||||
|
||||
db := testWebhookDB(t)
|
||||
targetID := uuid.New().String()
|
||||
|
||||
slackCfg, err := json.Marshal(
|
||||
delivery.SlackTargetConfig{WebhookURL: ts.URL},
|
||||
)
|
||||
require.NoError(t, err)
|
||||
|
||||
event := seedEvent(t, db, `{"action":"test"}`)
|
||||
dlv := seedDelivery(
|
||||
t, db, event.ID, targetID, database.DeliveryStatusPending,
|
||||
)
|
||||
|
||||
e.ExportDeliverSlack(context.Background(), db, buildSlackDelivery(
|
||||
dlv, event, targetID, "test-slack", string(slackCfg),
|
||||
))
|
||||
require.Len(t, userAgents, 1, "the slack target sent no request")
|
||||
assert.Equal(t, want, <-userAgents, "slack target")
|
||||
}
|
||||
@@ -241,3 +241,37 @@ func TestHandleSourceDetail_RendersNamedTargetFields(
|
||||
assert.Contains(t, body, "(unavailable)")
|
||||
assert.NotContains(t, body, "beak")
|
||||
}
|
||||
|
||||
// TestHandleSourceDetail_FitsWideAndNarrowWindows pins the webhook
|
||||
// page's maximum width at 108rem (1728 px), half again the 72rem of
|
||||
// max-w-6xl that the webhook list and the event log use, so an
|
||||
// entrypoint URL fits on one line in a 1920-pixel window; and the
|
||||
// wrapping of its title row, so the buttons beside the title do not
|
||||
// push a phone-width window into scrolling sideways.
|
||||
func TestHandleSourceDetail_FitsWideAndNarrowWindows(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
sess *session.Session
|
||||
db *database.Database
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &sess, &db)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
wh := seedWebhook(t, db)
|
||||
|
||||
body := renderSourceDetailPage(t, h, sess, wh.ID)
|
||||
|
||||
assert.Contains(
|
||||
t, body,
|
||||
`<div class="mx-auto px-6 py-8" style="max-width: 108rem"`,
|
||||
)
|
||||
assert.Contains(
|
||||
t, body,
|
||||
`<div class="flex flex-wrap justify-between items-center gap-2 mt-2">`,
|
||||
)
|
||||
}
|
||||
|
||||
@@ -1,355 +0,0 @@
|
||||
package handlers_test
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
"sneak.berlin/go/webhooker/internal/database"
|
||||
"sneak.berlin/go/webhooker/internal/handlers"
|
||||
"sneak.berlin/go/webhooker/internal/logger"
|
||||
"sneak.berlin/go/webhooker/internal/session"
|
||||
)
|
||||
|
||||
// failedHighlight is how the list marks a number of failed deliveries
|
||||
// that is not zero.
|
||||
const failedHighlight = `class="font-medium text-red-600"`
|
||||
|
||||
// listWebhook adds a webhook with the given name, owned by the test
|
||||
// user.
|
||||
func listWebhook(
|
||||
t *testing.T, db *database.Database, name string,
|
||||
) *database.Webhook {
|
||||
t.Helper()
|
||||
|
||||
wh := &database.Webhook{UserID: deleteTestUserID, Name: name}
|
||||
require.NoError(t, db.DB().Omit(clause.Associations).Create(wh).Error)
|
||||
|
||||
return wh
|
||||
}
|
||||
|
||||
// renderWebhookList runs the real webhook list handler as the test user
|
||||
// and returns the rendered page.
|
||||
func renderWebhookList(
|
||||
t *testing.T, h *handlers.Handlers, sess *session.Session,
|
||||
) string {
|
||||
t.Helper()
|
||||
|
||||
cookies := authenticatedCookies(
|
||||
t, sess, deleteTestUserID, deleteTestUsername,
|
||||
)
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
h.HandleSourceList().ServeHTTP(
|
||||
w, getRequest(t, "/hooks", cookies, nil),
|
||||
)
|
||||
require.Equal(t, http.StatusOK, w.Code)
|
||||
|
||||
return w.Body.String()
|
||||
}
|
||||
|
||||
// listCard returns one webhook's entry in a rendered webhook list, its
|
||||
// markup as rendered and its text with the markup taken out and each
|
||||
// run of space made one space.
|
||||
func listCard(t *testing.T, page, webhookID string) (string, string) {
|
||||
t.Helper()
|
||||
|
||||
_, card, found := strings.Cut(page, `href="/hook/`+webhookID+`"`)
|
||||
require.True(t, found, "the list has no entry for %s", webhookID)
|
||||
|
||||
card, _, _ = strings.Cut(card, "</a>")
|
||||
text := regexp.MustCompile(`<[^>]*>`).ReplaceAllString(card, " ")
|
||||
|
||||
return card, strings.Join(strings.Fields(text), " ")
|
||||
}
|
||||
|
||||
// receiveEvents posts the given number of events to an entrypoint
|
||||
// through the real receiver, and returns the webhook's event database
|
||||
// and its events, oldest first.
|
||||
func receiveEvents(
|
||||
t *testing.T,
|
||||
h *handlers.Handlers,
|
||||
dbMgr *database.WebhookDBManager,
|
||||
webhookID, path string,
|
||||
count int,
|
||||
) (*gorm.DB, []database.Event) {
|
||||
t.Helper()
|
||||
|
||||
router := receiverRouter(h)
|
||||
|
||||
for range count {
|
||||
require.Equal(t, http.StatusOK, postReceiver(t, router, path))
|
||||
}
|
||||
|
||||
webhookDB, err := dbMgr.GetDB(webhookID)
|
||||
require.NoError(t, err)
|
||||
|
||||
events := listEvents(t, webhookDB)
|
||||
require.Len(t, events, count)
|
||||
|
||||
return webhookDB, events
|
||||
}
|
||||
|
||||
// seedFailingWebhook adds a webhook with two entrypoints, one inactive,
|
||||
// and four targets, one inactive. Three events each reach the three
|
||||
// active targets. Two deliveries failed in the last 24 hours, one 30
|
||||
// hours ago, and one was delivered. It returns the webhook and its
|
||||
// newest event.
|
||||
func seedFailingWebhook(
|
||||
t *testing.T,
|
||||
h *handlers.Handlers,
|
||||
db *database.Database,
|
||||
dbMgr *database.WebhookDBManager,
|
||||
) (*database.Webhook, database.Event) {
|
||||
t.Helper()
|
||||
|
||||
wh := listWebhook(t, db, "failing")
|
||||
path := statsEntrypoint(t, db, wh.ID, true)
|
||||
|
||||
statsEntrypoint(t, db, wh.ID, false)
|
||||
|
||||
first := seedTarget(t, db, wh.ID, database.TargetTypeLog)
|
||||
second := seedTarget(t, db, wh.ID, database.TargetTypeLog)
|
||||
|
||||
seedTarget(t, db, wh.ID, database.TargetTypeLog)
|
||||
|
||||
inactive := seedTarget(t, db, wh.ID, database.TargetTypeLog)
|
||||
require.NoError(t, db.DB().Model(inactive).
|
||||
Update("active", false).Error)
|
||||
|
||||
webhookDB, events := receiveEvents(t, h, dbMgr, wh.ID, path, 3)
|
||||
now := time.Now()
|
||||
|
||||
statsFinish(t, webhookDB,
|
||||
statsDelivery(t, webhookDB, events[0].ID, first.ID),
|
||||
database.DeliveryStatusFailed, now.Add(-30*time.Hour))
|
||||
statsFinish(t, webhookDB,
|
||||
statsDelivery(t, webhookDB, events[1].ID, first.ID),
|
||||
database.DeliveryStatusFailed, now.Add(-time.Hour))
|
||||
statsFinish(t, webhookDB,
|
||||
statsDelivery(t, webhookDB, events[2].ID, first.ID),
|
||||
database.DeliveryStatusFailed, now.Add(-time.Minute))
|
||||
statsFinish(t, webhookDB,
|
||||
statsDelivery(t, webhookDB, events[2].ID, second.ID),
|
||||
database.DeliveryStatusDelivered, now.Add(-time.Minute))
|
||||
|
||||
return wh, events[2]
|
||||
}
|
||||
|
||||
// seedHealthyWebhook adds a webhook with one entrypoint and one target,
|
||||
// both active, and two events, both delivered. It returns the webhook
|
||||
// and its newest event.
|
||||
func seedHealthyWebhook(
|
||||
t *testing.T,
|
||||
h *handlers.Handlers,
|
||||
db *database.Database,
|
||||
dbMgr *database.WebhookDBManager,
|
||||
) (*database.Webhook, database.Event) {
|
||||
t.Helper()
|
||||
|
||||
wh := listWebhook(t, db, "healthy")
|
||||
path := statsEntrypoint(t, db, wh.ID, true)
|
||||
target := seedTarget(t, db, wh.ID, database.TargetTypeLog)
|
||||
|
||||
webhookDB, events := receiveEvents(t, h, dbMgr, wh.ID, path, 2)
|
||||
|
||||
for _, ev := range events {
|
||||
statsFinish(t, webhookDB,
|
||||
statsDelivery(t, webhookDB, ev.ID, target.ID),
|
||||
database.DeliveryStatusDelivered, time.Now())
|
||||
}
|
||||
|
||||
return wh, events[1]
|
||||
}
|
||||
|
||||
// lastEventText is how the list shows the arrival of an event.
|
||||
func lastEventText(ev database.Event) string {
|
||||
return ev.CreatedAt.UTC().Format("2006-01-02 15:04:05 UTC")
|
||||
}
|
||||
|
||||
// TestSourceList_ShowsActivityOfEachWebhook checks the figures the list
|
||||
// shows for a webhook with recent failures, a healthy one, a new one
|
||||
// that has received no event, and one without an event database.
|
||||
func TestSourceList_ShowsActivityOfEachWebhook(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
sess *session.Session
|
||||
db *database.Database
|
||||
dbMgr *database.WebhookDBManager
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &sess, &db, &dbMgr)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
failing, failingNewest := seedFailingWebhook(t, h, db, dbMgr)
|
||||
healthy, healthyNewest := seedHealthyWebhook(t, h, db, dbMgr)
|
||||
|
||||
// Creating a webhook creates its event database.
|
||||
fresh := listWebhook(t, db, "fresh")
|
||||
require.NoError(t, dbMgr.CreateDB(fresh.ID))
|
||||
|
||||
quiet := listWebhook(t, db, "quiet")
|
||||
|
||||
page := renderWebhookList(t, h, sess)
|
||||
|
||||
card, text := listCard(t, page, failing.ID)
|
||||
assert.Contains(t, text, "2 entrypoints, 1 inactive "+
|
||||
"4 targets, 1 inactive "+
|
||||
"3 events within retention "+
|
||||
"Last event "+lastEventText(failingNewest)+" "+
|
||||
"2 failed deliveries in the last 24 hours")
|
||||
assert.Contains(t, card,
|
||||
failedHighlight+">2 failed deliveries in the last 24 hours<")
|
||||
|
||||
card, text = listCard(t, page, healthy.ID)
|
||||
assert.Contains(t, text, "1 entrypoint "+
|
||||
"1 target "+
|
||||
"2 events within retention "+
|
||||
"Last event "+lastEventText(healthyNewest)+" "+
|
||||
"0 failed deliveries in the last 24 hours")
|
||||
assert.NotContains(t, text, "inactive")
|
||||
assert.NotContains(t, card, failedHighlight)
|
||||
|
||||
card, text = listCard(t, page, fresh.ID)
|
||||
assert.Contains(t, text, "0 entrypoints "+
|
||||
"0 targets "+
|
||||
"0 events within retention "+
|
||||
"No events yet "+
|
||||
"0 failed deliveries in the last 24 hours")
|
||||
assert.NotContains(t, card, failedHighlight)
|
||||
|
||||
card, text = listCard(t, page, quiet.ID)
|
||||
assert.Contains(t, text, "0 entrypoints "+
|
||||
"0 targets "+
|
||||
"0 events within retention "+
|
||||
"No events yet "+
|
||||
"0 failed deliveries in the last 24 hours")
|
||||
assert.NotContains(t, card, failedHighlight)
|
||||
assert.False(t, dbMgr.DBExists(quiet.ID),
|
||||
"showing the list must not create an event database")
|
||||
}
|
||||
|
||||
// TestSourceList_CountsOnlyEventsWithinRetention checks that once
|
||||
// retention has removed one of a webhook's three events, the list
|
||||
// counts the two still stored.
|
||||
func TestSourceList_CountsOnlyEventsWithinRetention(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
sess *session.Session
|
||||
db *database.Database
|
||||
dbMgr *database.WebhookDBManager
|
||||
log *logger.Logger
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &sess, &db, &dbMgr, &log)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
wh := &database.Webhook{
|
||||
UserID: deleteTestUserID, Name: "pruned", RetentionDays: 14,
|
||||
}
|
||||
require.NoError(t, db.DB().Omit(clause.Associations).Create(wh).Error)
|
||||
|
||||
path := statsEntrypoint(t, db, wh.ID, true)
|
||||
webhookDB, events := receiveEvents(t, h, dbMgr, wh.ID, path, 3)
|
||||
|
||||
statsAge(t, webhookDB, events[0].ID, time.Now().Add(-15*24*time.Hour))
|
||||
statsPrune(t, db, dbMgr, log, webhookDB)
|
||||
require.Len(t, listEvents(t, webhookDB), 2)
|
||||
|
||||
_, text := listCard(t, renderWebhookList(t, h, sess), wh.ID)
|
||||
assert.Contains(t, text,
|
||||
"1 entrypoint 0 targets 2 events within retention")
|
||||
}
|
||||
|
||||
// TestSourceList_LastEventSurvivesPruningEveryEvent checks that once
|
||||
// retention has removed every event of a webhook, the list still shows
|
||||
// when the last one arrived rather than "No events yet".
|
||||
func TestSourceList_LastEventSurvivesPruningEveryEvent(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
sess *session.Session
|
||||
db *database.Database
|
||||
dbMgr *database.WebhookDBManager
|
||||
log *logger.Logger
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &sess, &db, &dbMgr, &log)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
wh := &database.Webhook{
|
||||
UserID: deleteTestUserID, Name: "emptied", RetentionDays: 1,
|
||||
}
|
||||
require.NoError(t, db.DB().Omit(clause.Associations).Create(wh).Error)
|
||||
|
||||
path := statsEntrypoint(t, db, wh.ID, true)
|
||||
webhookDB, events := receiveEvents(t, h, dbMgr, wh.ID, path, 1)
|
||||
|
||||
statsAge(t, webhookDB, events[0].ID, time.Now().Add(-50*time.Hour))
|
||||
statsPrune(t, db, dbMgr, log, webhookDB)
|
||||
require.Empty(t, listEvents(t, webhookDB))
|
||||
|
||||
_, text := listCard(t, renderWebhookList(t, h, sess), wh.ID)
|
||||
assert.Contains(t, text,
|
||||
"0 events within retention "+
|
||||
"Last event "+lastEventText(events[0]))
|
||||
assert.NotContains(t, text, "No events yet")
|
||||
}
|
||||
|
||||
// TestSourceList_UnreadableEventDatabase checks that a webhook whose
|
||||
// event database cannot be read says so in its entry instead of
|
||||
// showing zeros, and that the rest of the list is still shown.
|
||||
func TestSourceList_UnreadableEventDatabase(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
sess *session.Session
|
||||
db *database.Database
|
||||
dbMgr *database.WebhookDBManager
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &sess, &db, &dbMgr)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
broken := listWebhook(t, db, "broken")
|
||||
statsEntrypoint(t, db, broken.ID, true)
|
||||
|
||||
brokenDB, err := dbMgr.GetDB(broken.ID)
|
||||
require.NoError(t, err)
|
||||
require.NoError(t,
|
||||
brokenDB.Migrator().DropTable(&database.EventTotals{}))
|
||||
|
||||
quiet := listWebhook(t, db, "quiet")
|
||||
|
||||
page := renderWebhookList(t, h, sess)
|
||||
|
||||
_, text := listCard(t, page, broken.ID)
|
||||
assert.Contains(t, text,
|
||||
"1 entrypoint 0 targets The event figures could not be read.")
|
||||
assert.NotContains(t, text, "events")
|
||||
assert.NotContains(t, text, "failed")
|
||||
|
||||
_, text = listCard(t, page, quiet.ID)
|
||||
assert.Contains(t, text, "No events yet")
|
||||
}
|
||||
@@ -3,12 +3,10 @@ package handlers
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/go-chi/chi"
|
||||
"github.com/google/uuid"
|
||||
@@ -22,20 +20,9 @@ import (
|
||||
type WebhookListItem struct {
|
||||
database.Webhook
|
||||
|
||||
EntrypointCount int
|
||||
InactiveEntrypointCount int
|
||||
TargetCount int
|
||||
InactiveTargetCount int
|
||||
|
||||
// EventCount is how many events the webhook holds, LastEventAt
|
||||
// when the newest arrived (nil before the first), and
|
||||
// FailedLast24Hours how many of its deliveries failed in the last
|
||||
// 24 hours. When the webhook's event database could not be read,
|
||||
// EventsUnreadable is set and these three are not known.
|
||||
EventCount int64
|
||||
LastEventAt *time.Time
|
||||
FailedLast24Hours int64
|
||||
EventsUnreadable bool
|
||||
EntrypointCount int64
|
||||
TargetCount int64
|
||||
EventCount int64
|
||||
}
|
||||
|
||||
// errMissingURL signals that a required URL was not provided.
|
||||
@@ -167,12 +154,7 @@ func (h *Handlers) HandleSourceList() http.HandlerFunc {
|
||||
return
|
||||
}
|
||||
|
||||
items, err := h.buildWebhookListItems(webhooks)
|
||||
if err != nil {
|
||||
h.serverError(w, r, "failed to list webhooks", err)
|
||||
|
||||
return
|
||||
}
|
||||
items := h.buildWebhookListItems(webhooks)
|
||||
|
||||
data := map[string]any{
|
||||
"Webhooks": items,
|
||||
@@ -182,115 +164,36 @@ func (h *Handlers) HandleSourceList() http.HandlerFunc {
|
||||
}
|
||||
}
|
||||
|
||||
// buildWebhookListItems builds the list's entry for each webhook. It
|
||||
// fails when the main database cannot be read. A webhook whose event
|
||||
// database cannot be read is marked on its own entry, and the error is
|
||||
// logged.
|
||||
// buildWebhookListItems builds list items with counts.
|
||||
func (h *Handlers) buildWebhookListItems(
|
||||
webhooks []database.Webhook,
|
||||
) ([]WebhookListItem, error) {
|
||||
) []WebhookListItem {
|
||||
items := make([]WebhookListItem, len(webhooks))
|
||||
since := time.Now().Add(-longWindow)
|
||||
|
||||
for i := range webhooks {
|
||||
item := &items[i]
|
||||
item.Webhook = webhooks[i]
|
||||
items[i].Webhook = webhooks[i]
|
||||
|
||||
var err error
|
||||
h.db.DB().Model(&database.Entrypoint{}).Where(
|
||||
"webhook_id = ?", webhooks[i].ID,
|
||||
).Count(&items[i].EntrypointCount)
|
||||
|
||||
item.EntrypointCount, item.InactiveEntrypointCount, err =
|
||||
h.countWithInactive(&database.Entrypoint{}, item.ID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
h.db.DB().Model(&database.Target{}).Where(
|
||||
"webhook_id = ?", webhooks[i].ID,
|
||||
).Count(&items[i].TargetCount)
|
||||
|
||||
item.TargetCount, item.InactiveTargetCount, err =
|
||||
h.countWithInactive(&database.Target{}, item.ID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Opening an event database that does not exist would create
|
||||
// it, and it would hold nothing to count.
|
||||
if !h.dbMgr.DBExists(item.ID) {
|
||||
continue
|
||||
}
|
||||
|
||||
err = h.readListEventFigures(item, since)
|
||||
if err != nil {
|
||||
h.log.Error(
|
||||
"failed to read webhook list figures",
|
||||
"webhook_id", item.ID,
|
||||
"error", err,
|
||||
if h.dbMgr.DBExists(webhooks[i].ID) {
|
||||
webhookDB, err := h.dbMgr.GetDB(
|
||||
webhooks[i].ID,
|
||||
)
|
||||
|
||||
item.EventsUnreadable = true
|
||||
if err == nil {
|
||||
webhookDB.Model(
|
||||
&database.Event{},
|
||||
).Count(&items[i].EventCount)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return items, nil
|
||||
}
|
||||
|
||||
// countWithInactive returns how many entrypoints or targets, as model
|
||||
// says, a webhook has, and how many of them are inactive.
|
||||
func (h *Handlers) countWithInactive(
|
||||
model any, webhookID string,
|
||||
) (int, int, error) {
|
||||
var active []bool
|
||||
|
||||
err := h.db.DB().Model(model).
|
||||
Where("webhook_id = ?", webhookID).
|
||||
Pluck("active", &active).Error
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf(
|
||||
"reading active flags of webhook %s: %w", webhookID, err,
|
||||
)
|
||||
}
|
||||
|
||||
inactive := 0
|
||||
|
||||
for _, a := range active {
|
||||
if !a {
|
||||
inactive++
|
||||
}
|
||||
}
|
||||
|
||||
return len(active), inactive, nil
|
||||
}
|
||||
|
||||
// readListEventFigures fills in the figures the list shows from the
|
||||
// webhook's event database, with the statistics pane's own queries:
|
||||
// the event count and last arrival from the event totals row, and the
|
||||
// deliveries that failed since the given time from the deliveries'
|
||||
// status index.
|
||||
func (h *Handlers) readListEventFigures(
|
||||
item *WebhookListItem, since time.Time,
|
||||
) error {
|
||||
webhookDB, err := h.dbMgr.GetDB(item.ID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
var totals database.EventTotals
|
||||
|
||||
err = webhookDB.Take(&totals).Error
|
||||
if err != nil {
|
||||
return fmt.Errorf("reading event totals: %w", err)
|
||||
}
|
||||
|
||||
item.EventCount = totals.Events - totals.EventsRemoved
|
||||
item.LastEventAt = totals.LastEventAt
|
||||
|
||||
byTarget, err := finishedByTarget(webhookDB, since)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for _, f := range byTarget {
|
||||
item.FailedLast24Hours += f.Failed
|
||||
}
|
||||
|
||||
return nil
|
||||
return items
|
||||
}
|
||||
|
||||
// HandleSourceCreate shows the form to create a new webhook.
|
||||
@@ -1666,10 +1569,11 @@ func (h *Handlers) validateTargetURL(
|
||||
msg := "Invalid target URL: " + err.Error()
|
||||
|
||||
// Only a private or reserved address's refusal says how
|
||||
// to allow it. Metadata refusals never do: link-local and
|
||||
// the other unconditional metadata addresses cannot be
|
||||
// opened, and the default blocklist's public addresses,
|
||||
// which listing does open, hand out credentials.
|
||||
// to allow it. Other refusals never do: link-local, the
|
||||
// unspecified addresses and the other unconditional
|
||||
// metadata addresses cannot be opened, and the default
|
||||
// blocklist's public addresses, which listing does open,
|
||||
// hand out credentials.
|
||||
if errors.Is(err, delivery.ErrBlockedPrivateOrReservedIP) {
|
||||
msg += ". Private and reserved addresses are refused " +
|
||||
"by default; the server's ALLOWED_EGRESS_CIDRS " +
|
||||
|
||||
@@ -3,10 +3,14 @@
|
||||
{{define "title"}}{{.Webhook.Name}} - Webhooker{{end}}
|
||||
|
||||
{{define "content"}}
|
||||
<div class="max-w-6xl mx-auto px-6 py-8" x-data="{ showAddEntrypoint: false, showAddTarget: false }">
|
||||
<!-- 108rem, half again the 72rem (max-w-6xl) of the webhook list, the
|
||||
event log, the navbar and the footer, so an entrypoint URL fits on
|
||||
one line. An inline style, because the committed tailwind.css has
|
||||
no class this wide. -->
|
||||
<div class="mx-auto px-6 py-8" style="max-width: 108rem" x-data="{ showAddEntrypoint: false, showAddTarget: false }">
|
||||
<div class="mb-6">
|
||||
<a href="/hooks" class="text-sm text-primary-600 hover:text-primary-700">← Back to webhooks</a>
|
||||
<div class="flex justify-between items-center mt-2">
|
||||
<div class="flex flex-wrap justify-between items-center gap-2 mt-2">
|
||||
<div>
|
||||
<h1 class="text-2xl font-medium text-gray-900">{{.Webhook.Name}}</h1>
|
||||
{{if .Webhook.Description}}
|
||||
|
||||
@@ -27,16 +27,10 @@
|
||||
</div>
|
||||
<span class="badge-info">Retention: {{.RetentionLabel}}</span>
|
||||
</div>
|
||||
<div class="flex flex-wrap gap-6 mt-4 text-sm text-gray-500">
|
||||
<span>{{.EntrypointCount}} entrypoint{{if ne .EntrypointCount 1}}s{{end}}{{if .InactiveEntrypointCount}}, {{.InactiveEntrypointCount}} inactive{{end}}</span>
|
||||
<span>{{.TargetCount}} target{{if ne .TargetCount 1}}s{{end}}{{if .InactiveTargetCount}}, {{.InactiveTargetCount}} inactive{{end}}</span>
|
||||
{{if .EventsUnreadable}}
|
||||
<span class="text-red-600">The event figures could not be read.</span>
|
||||
{{else}}
|
||||
<span>{{.EventCount}} event{{if ne .EventCount 1}}s{{end}} within retention</span>
|
||||
<span>{{with .LastEventAt}}Last event {{.UTC.Format "2006-01-02 15:04:05 UTC"}}{{else}}No events yet{{end}}</span>
|
||||
<span class="{{if .FailedLast24Hours}}font-medium text-red-600{{end}}">{{.FailedLast24Hours}} failed deliver{{if eq .FailedLast24Hours 1}}y{{else}}ies{{end}} in the last 24 hours</span>
|
||||
{{end}}
|
||||
<div class="flex gap-6 mt-4 text-sm text-gray-500">
|
||||
<span>{{.EntrypointCount}} entrypoint{{if ne .EntrypointCount 1}}s{{end}}</span>
|
||||
<span>{{.TargetCount}} target{{if ne .TargetCount 1}}s{{end}}</span>
|
||||
<span>{{.EventCount}} event{{if ne .EventCount 1}}s{{end}}</span>
|
||||
</div>
|
||||
</a>
|
||||
{{end}}
|
||||
|
||||
Reference in New Issue
Block a user