Compare commits
6 Commits
f9362101e6
...
15ada44902
| Author | SHA1 | Date | |
|---|---|---|---|
| 15ada44902 | |||
| 3b0ed826bc | |||
| 9969694a47 | |||
| fcead5d401 | |||
| ac782f4c5a | |||
| 89b2dadd48 |
333
README.md
333
README.md
@@ -163,14 +163,19 @@ Two things this setting cannot do:
|
|||||||
the list is always an allowlist; an empty list (the default) means
|
the list is always an allowlist; an empty list (the default) means
|
||||||
every private and reserved range stays refused. Note that
|
every private and reserved range stays refused. Note that
|
||||||
`0.0.0.0/0` gets you most of the way there anyway, per above.
|
`0.0.0.0/0` gets you most of the way there anyway, per above.
|
||||||
- **It cannot open link-local or a known cloud metadata endpoint.**
|
- **It cannot open link-local, or a cloud metadata endpoint that
|
||||||
These stay blocked no matter what you list, including when you list
|
discloses credentials or user data.** An address is on the list below
|
||||||
them outright or list a supernet such as `0.0.0.0/0`, `::/0`,
|
when both of these hold: the provider fixes it, so it cannot collide
|
||||||
`fd00::/8` or `100.64.0.0/10`:
|
with anything you run; and reaching it hands out credentials, user
|
||||||
|
data or bootstrap material. Those stay blocked no matter what you
|
||||||
|
list, including when you list them outright or list a supernet such
|
||||||
|
as `0.0.0.0/0`, `::/0`, `fd00::/8` or `100.64.0.0/10`. Treat this as
|
||||||
|
best effort rather than a guarantee — it is a hand-maintained list
|
||||||
|
and the caveat below the table applies:
|
||||||
|
|
||||||
| Blocked unconditionally | What it is |
|
| Blocked unconditionally | What it is |
|
||||||
| ----------------------- | ---------- |
|
| ----------------------- | ---------- |
|
||||||
| `169.254.0.0/16` | IPv4 link-local, carrying `169.254.169.254` (AWS, Azure, Alibaba, DigitalOcean, Hetzner, OpenStack and others) |
|
| `169.254.0.0/16` | IPv4 link-local, carrying `169.254.169.254` (AWS, Azure, DigitalOcean, Hetzner, OpenStack and others — not Alibaba, which uses `100.100.100.200` below) |
|
||||||
| `fe80::/10` | IPv6 link-local |
|
| `fe80::/10` | IPv6 link-local |
|
||||||
| `fd00:ec2::254/128` | AWS IPv6 IMDS |
|
| `fd00:ec2::254/128` | AWS IPv6 IMDS |
|
||||||
| `fd00:ec2::23/128` | AWS EKS Pod Identity Agent |
|
| `fd00:ec2::23/128` | AWS EKS Pod Identity Agent |
|
||||||
@@ -186,18 +191,20 @@ Two things this setting cannot do:
|
|||||||
| `64:ff9b::a9fe:a9fe/128` | `169.254.169.254` behind the NAT64 well-known prefix |
|
| `64:ff9b::a9fe:a9fe/128` | `169.254.169.254` behind the NAT64 well-known prefix |
|
||||||
|
|
||||||
The IPv4-mapped form `::ffff:169.254.169.254` is covered by the
|
The IPv4-mapped form `::ffff:169.254.169.254` is covered by the
|
||||||
`169.254.0.0/16` entry. Reaching any of these is credential theft
|
`169.254.0.0/16` entry. Reaching any of these is credential or
|
||||||
rather than delivery to an internal service. Every entry outside the
|
user-data theft rather than delivery to an internal service. Every
|
||||||
two link-local blocks is a single address, so blocking it costs you
|
entry outside the two link-local blocks is a single address, so
|
||||||
nothing else on the network around it.
|
blocking it costs you nothing else on the network around it.
|
||||||
|
|
||||||
The six `fd00::/8` and `fd20::/8` entries are why this matters in
|
The six ULA entries, all inside `fd00::/8`, are why this matters in
|
||||||
practice: `fd00::/8` is an ordinary block to allowlist for your own
|
practice: `fd00::/8` is an ordinary block to allowlist for your own
|
||||||
IPv6 network, and without those host routes that one line would hand
|
IPv6 network, and without those host routes that one line would hand
|
||||||
out cloud credentials on six providers at once. Several of them are
|
out cloud credentials on five providers at once. There is only one
|
||||||
described as "link-local" — or even "localhost" — in their own
|
`/8` involved — `fd20:ce::254` masks into `fd00::/8` as well — and
|
||||||
vendor's documentation, but they are ULAs and `fe80::/10` does not
|
the six endpoints are five providers because AWS appears twice, IMDS
|
||||||
cover them.
|
and EKS Pod Identity. Several of them are described as "link-local" —
|
||||||
|
or even "localhost" — in their own vendor's documentation, but they
|
||||||
|
are ULAs and `fe80::/10` does not cover them.
|
||||||
|
|
||||||
`168.63.129.16` and `147.75.207.243` are ordinary public addresses
|
`168.63.129.16` and `147.75.207.243` are ordinary public addresses
|
||||||
rather than reserved ones, so they are the only two entries here that
|
rather than reserved ones, so they are the only two entries here that
|
||||||
@@ -390,9 +397,84 @@ On first startup, webhooker automatically generates a cryptographically
|
|||||||
secure session encryption key and stores it in the database. This key
|
secure session encryption key and stores it in the database. This key
|
||||||
persists across restarts — no manual key management is needed.
|
persists across restarts — no manual key management is needed.
|
||||||
|
|
||||||
On first startup, webhooker creates an `admin` user
|
#### The admin account
|
||||||
with a randomly generated password and logs it to stdout. This password
|
|
||||||
is only displayed once.
|
On first startup — a `DATA_DIR` with no accounts in it — webhooker
|
||||||
|
creates an `admin` user with a randomly generated password and prints
|
||||||
|
it to standard output as a ruled banner:
|
||||||
|
|
||||||
|
```
|
||||||
|
========================================================================
|
||||||
|
WEBHOOKER FIRST BOOT: an admin account has been created.
|
||||||
|
|
||||||
|
username: admin
|
||||||
|
password: 3xamPl3-p4ssw0rd
|
||||||
|
|
||||||
|
Save this password now: it is shown only here, and only once.
|
||||||
|
If it is lost, run `webhooker resetpw admin` on a stopped deployment.
|
||||||
|
========================================================================
|
||||||
|
```
|
||||||
|
|
||||||
|
It is a banner rather than a log line because that is the only time it
|
||||||
|
is ever shown: as one `INFO` record it sat among the roughly 45 fx
|
||||||
|
`PROVIDE`/`RUN`/`HOOK` lines a boot writes, and under `docker run -d`
|
||||||
|
it is one line in a log subject to rotation. The database stores only
|
||||||
|
its Argon2id hash. There is no second account and no forgot-password
|
||||||
|
flow, so the banner and the reset command below are the only two ways
|
||||||
|
in.
|
||||||
|
|
||||||
|
#### Recovering a lost admin password
|
||||||
|
|
||||||
|
`webhooker resetpw` sets an existing account's password from the
|
||||||
|
command line:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Generate a new password and print it.
|
||||||
|
DATA_DIR=/var/lib/webhooker webhooker resetpw -generate admin
|
||||||
|
|
||||||
|
# Or supply one on standard input (minimum 8 characters).
|
||||||
|
printf '%s' "$NEW_PASSWORD" | \
|
||||||
|
DATA_DIR=/var/lib/webhooker webhooker resetpw admin
|
||||||
|
```
|
||||||
|
|
||||||
|
In a container it is the same binary, which the image sets as `CMD`
|
||||||
|
rather than `ENTRYPOINT`, so the whole command has to be given:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run --rm -v webhooker-data:/var/lib/webhooker \
|
||||||
|
webhooker /app/webhooker resetpw -generate admin
|
||||||
|
```
|
||||||
|
|
||||||
|
Stop the service first — with the volume still attached to a running
|
||||||
|
container, the command refuses.
|
||||||
|
|
||||||
|
The password is never taken as a command-line argument: on Linux argv
|
||||||
|
is readable through `/proc` by every account on the host for as long as
|
||||||
|
the process lives. Standard input is echoed when it is a terminal — the
|
||||||
|
prompt says so — so `-generate` or a pipe is preferable on a shared
|
||||||
|
machine.
|
||||||
|
|
||||||
|
What it will not do:
|
||||||
|
|
||||||
|
- **Run against a live deployment.** It takes the same exclusive
|
||||||
|
`DATA_DIR` lock the server does (see
|
||||||
|
[Single-instance lock](#single-instance-lock)) and refuses while a
|
||||||
|
running instance holds it, naming the directory and exiting non-zero.
|
||||||
|
A running process keeps serving every session that authenticated with
|
||||||
|
the old password, so a reset underneath it would report a change the
|
||||||
|
service does not honour.
|
||||||
|
- **Create anything.** A `DATA_DIR` that does not exist, or that holds
|
||||||
|
no `webhooker.db`, is an error rather than a new empty deployment —
|
||||||
|
a mistyped path must not be built out and then reported as a success.
|
||||||
|
- **Create an account.** A username that does not exist is an error.
|
||||||
|
`resetpw` changes an existing account's password and nothing else.
|
||||||
|
|
||||||
|
`DATA_DIR` selects the deployment exactly as it does for the server. A
|
||||||
|
password that changes on disk takes effect at the next login; sessions
|
||||||
|
that are already authenticated are unaffected either way.
|
||||||
|
|
||||||
|
Changing a password you still know needs none of this — use
|
||||||
|
`POST /user/{username}/password` in the web UI.
|
||||||
|
|
||||||
#### What `DEBUG=true` exposes
|
#### What `DEBUG=true` exposes
|
||||||
|
|
||||||
@@ -429,11 +511,14 @@ What it does **not** put in the log:
|
|||||||
What is in the log regardless of `DEBUG`, and is not a debug-logging
|
What is in the log regardless of `DEBUG`, and is not a debug-logging
|
||||||
decision:
|
decision:
|
||||||
|
|
||||||
- **The initial `admin` password**, in the clear, once, at `INFO`, on
|
- **The initial `admin` password**, in the clear, once, on the first
|
||||||
the first boot that creates the account. That line is the only place
|
boot that creates the account — as the banner described under
|
||||||
|
[The admin account](#the-admin-account), written straight to standard
|
||||||
|
output rather than through the logger. That banner is the only place
|
||||||
it is ever shown; the database stores the hash. A first boot's output
|
it is ever shown; the database stores the hash. A first boot's output
|
||||||
is not safe to paste anywhere until that account's password has been
|
is not safe to paste anywhere until that account's password has been
|
||||||
changed.
|
changed. The same applies to `webhooker resetpw -generate`, which
|
||||||
|
prints the password it generated in the same form.
|
||||||
- **An authenticated operator's own configuration**, echoed back
|
- **An authenticated operator's own configuration**, echoed back
|
||||||
untruncated — webhook names, target hostnames. See the logging
|
untruncated — webhook names, target hostnames. See the logging
|
||||||
section under Security for the full list and for the per-line size
|
section under Security for the full list and for the per-line size
|
||||||
@@ -610,17 +695,134 @@ backups at rest and restrict who can read them.
|
|||||||
- `events-{uuid}.db` and `archive-{uuid}.db` hold the **full payload
|
- `events-{uuid}.db` and `archive-{uuid}.db` hold the **full payload
|
||||||
body and headers** of every event as received, including whatever the
|
body and headers** of every event as received, including whatever the
|
||||||
sending service put in them — tokens, signatures, personal data.
|
sending service put in them — tokens, signatures, personal data.
|
||||||
- Until
|
- Event databases written before
|
||||||
[issue #206](https://git.eeqj.de/sneak/webhooker/issues/206) is fixed,
|
[issue #206](https://git.eeqj.de/sneak/webhooker/issues/206) was fixed
|
||||||
the event databases **also contain target credentials**: a GORM
|
**also contain target credentials**: a GORM association upsert on the
|
||||||
association upsert on the delivery and retry write path copies
|
delivery and retry write path copied `targets` rows, `config`
|
||||||
`targets` rows, `config` included, into the per-webhook database. For
|
included, into the per-webhook database. For a Slack target the
|
||||||
a Slack target the `webhookUrl` *is* the bearer credential, and an
|
`webhookUrl` *is* the bearer credential, and an `http` target's URL
|
||||||
`http` target's URL can embed userinfo. Handing someone an
|
can embed userinfo. This version never writes those rows; the first
|
||||||
`events-*.db` today hands them live delivery destinations.
|
time it opens such a file it deletes them and vacuums the file, which
|
||||||
|
removes the credential bytes rather than only unlinking the rows.
|
||||||
|
Deleting alone would not: the bytes stay readable in the file's free
|
||||||
|
pages until it is rewritten. The sweep is recorded in the file's
|
||||||
|
`user_version` only once the vacuum returns, so a sweep that fails or
|
||||||
|
is interrupted fails the open and is retried on the next one, and a
|
||||||
|
file this version has opened without error holds no leaked rows and
|
||||||
|
no recoverable bytes from them. On upgrade this rewrites each
|
||||||
|
existing `events-{uuid}.db` once, on its first open. Two cases still
|
||||||
|
hand over live delivery destinations: a backup taken from an older
|
||||||
|
build, and a backup of a file this version has not yet opened
|
||||||
|
successfully. Copies already made stay affected — the sweep only
|
||||||
|
rewrites the file it opens, and freed blocks may persist in
|
||||||
|
filesystem snapshots and on the underlying storage. Rotate any target
|
||||||
|
credential that was in a backup you cannot account for.
|
||||||
- `webhooker.db` stores target config **unencrypted**, tracked at
|
- `webhooker.db` stores target config **unencrypted**, tracked at
|
||||||
[issue #212](https://git.eeqj.de/sneak/webhooker/issues/212), next to
|
[issue #212](https://git.eeqj.de/sneak/webhooker/issues/212), next to
|
||||||
the session encryption key and the Argon2id password hashes.
|
the session encryption key and the Argon2id password hashes. It also
|
||||||
|
holds each entrypoint's inbound signature secret in the clear, for
|
||||||
|
the reason given under
|
||||||
|
[Inbound Signature Verification](#inbound-signature-verification):
|
||||||
|
HMAC verification needs the key itself, so it cannot be hashed.
|
||||||
|
|
||||||
|
## Inbound Signature Verification
|
||||||
|
|
||||||
|
A receiver URL is a bare v4 UUID in a path. That is unguessable, but it
|
||||||
|
is not a credential: anyone who learns it — from a browser history, a
|
||||||
|
proxy log, a screenshot, a copy-pasted support ticket — can post events
|
||||||
|
that webhooker stores and forwards, and the inbound headers are passed
|
||||||
|
on to your targets almost verbatim, so they also choose what the
|
||||||
|
downstream service sees. Verification is how an entrypoint stops
|
||||||
|
accepting anything that reaches its URL.
|
||||||
|
|
||||||
|
It is optional and configured per entrypoint. An entrypoint with no
|
||||||
|
scheme selected is not verified, which is what every entrypoint was
|
||||||
|
before this existed and what every entrypoint remains after an
|
||||||
|
upgrade — enabling verification is always a deliberate act, and no
|
||||||
|
existing deployment is locked out of its own receivers by installing a
|
||||||
|
new version.
|
||||||
|
|
||||||
|
When a scheme **is** selected, a request whose signature is missing,
|
||||||
|
malformed or wrong is answered `401` and **nothing is stored**: no
|
||||||
|
event row, no delivery row, no delivery attempt. Rejection happens
|
||||||
|
after the body is read (the signature covers it) and before the first
|
||||||
|
write.
|
||||||
|
|
||||||
|
### Supported schemes
|
||||||
|
|
||||||
|
| Scheme | Header | Check |
|
||||||
|
| -------- | --------------------- | ----- |
|
||||||
|
| `github` | `X-Hub-Signature-256` | HMAC-SHA256 of the raw request body under the shared secret, hex-encoded, prefixed `sha256=` |
|
||||||
|
| `gitlab` | `X-Gitlab-Token` | The header is the shared secret itself, compared as-is |
|
||||||
|
|
||||||
|
Both comparisons run in constant time (`hmac.Equal`). The HMAC is
|
||||||
|
computed over the request body exactly as received, before any parsing,
|
||||||
|
and under the same 1 MB body cap every other request obeys — an
|
||||||
|
unsigned sender cannot make webhooker buffer more than a signed one.
|
||||||
|
|
||||||
|
GitHub's older SHA-1 `X-Hub-Signature` is **not** accepted. Neither is
|
||||||
|
a GitHub digest sent without its `sha256=` prefix.
|
||||||
|
|
||||||
|
### Configuring a sender
|
||||||
|
|
||||||
|
The secret is a value you choose and enter in two places: at the sender
|
||||||
|
and in webhooker. webhooker never generates or displays one, so there
|
||||||
|
is no stored credential the UI can be made to reveal.
|
||||||
|
|
||||||
|
1. Generate a secret, e.g. `openssl rand -hex 32`.
|
||||||
|
2. In webhooker, open the webhook's page, find the entrypoint, and
|
||||||
|
click **Configure** (or **Rotate**, if it already has one). Select
|
||||||
|
the scheme and paste the secret. Surrounding whitespace is stripped,
|
||||||
|
so a value pasted with a trailing space still works; a secret whose
|
||||||
|
own first or last character is a space cannot be stored.
|
||||||
|
3. At the sender:
|
||||||
|
- **GitHub** — repository (or organization) → Settings → Webhooks →
|
||||||
|
the hook → **Secret**. GitHub then signs every delivery with
|
||||||
|
`X-Hub-Signature-256`.
|
||||||
|
- **GitLab** — project → Settings → Webhooks → the hook → **Secret
|
||||||
|
token**. GitLab sends it verbatim as `X-Gitlab-Token`.
|
||||||
|
|
||||||
|
**Rotation** is the same form: submit the new secret. Deliveries signed
|
||||||
|
with the old secret are rejected from that moment, so change it at the
|
||||||
|
sender in the same sitting. Selecting **None** removes verification and
|
||||||
|
deletes the stored secret with it.
|
||||||
|
|
||||||
|
The page shows which scheme an entrypoint uses and which header it
|
||||||
|
reads, never the secret. The value is stored in the clear — HMAC
|
||||||
|
verification needs the key itself, and a hash of it cannot recompute a
|
||||||
|
sender's digest — so it is handled like the other credentials
|
||||||
|
webhooker holds: excluded from JSON, kept out of templates by a
|
||||||
|
projection (`handlers.EntrypointView`), and absent from every log line,
|
||||||
|
including the ones written when verification fails.
|
||||||
|
|
||||||
|
### The credential is not stored or forwarded
|
||||||
|
|
||||||
|
Under the `gitlab` scheme the signature header **is** the secret. An
|
||||||
|
accepted request's headers are persisted on the event and forwarded to
|
||||||
|
every delivery target, so `X-Gitlab-Token` is removed from that copy
|
||||||
|
before the event is written — otherwise every target operator, every
|
||||||
|
backup and everyone with read access to `events-*.db` would hold the
|
||||||
|
value needed to forge signed requests to the entrypoint it protects.
|
||||||
|
The sender's other headers are untouched, and the request the receiver
|
||||||
|
itself verifies against is not modified.
|
||||||
|
|
||||||
|
The stripping is driven by the scheme's own description rather than by
|
||||||
|
a header name, and a scheme is stripped unless it declares that its
|
||||||
|
header carries a digest. `github` declares it: `X-Hub-Signature-256` is
|
||||||
|
an HMAC over the body, from which the key cannot be recovered, so it is
|
||||||
|
stored and forwarded intact. A scheme added later is stripped by
|
||||||
|
default.
|
||||||
|
|
||||||
|
### When configuration is broken
|
||||||
|
|
||||||
|
An entrypoint whose stored scheme this build does not recognise, or
|
||||||
|
which has one half of the scheme/secret pair and not the other, is
|
||||||
|
answered `500` and stores nothing. It is not treated as unverified. The
|
||||||
|
UI cannot create such a row — it rejects an unknown scheme with a `400`
|
||||||
|
— so this covers a hand-edited database or a downgrade to a build that
|
||||||
|
predates a scheme. Failing closed is the point: an entrypoint the
|
||||||
|
operator believes is protected must never quietly go back to accepting
|
||||||
|
anything.
|
||||||
|
|
||||||
## Entrypoints
|
## Entrypoints
|
||||||
|
|
||||||
@@ -829,7 +1031,10 @@ A registered user of the webhooker service.
|
|||||||
Passwords are hashed with Argon2id using secure defaults (64 MB memory,
|
Passwords are hashed with Argon2id using secure defaults (64 MB memory,
|
||||||
1 iteration, 4 threads, 32-byte key, 16-byte salt). On first startup,
|
1 iteration, 4 threads, 32-byte key, 16-byte salt). On first startup,
|
||||||
an `admin` user is created with a randomly generated 16-character
|
an `admin` user is created with a randomly generated 16-character
|
||||||
password logged to stdout.
|
password printed once to stdout; `webhooker resetpw` sets it again if
|
||||||
|
it is lost (see [The admin account](#the-admin-account)). Every one of
|
||||||
|
those paths hashes through the same `internal/database` code, so the
|
||||||
|
parameters cannot drift between them.
|
||||||
|
|
||||||
#### Webhook
|
#### Webhook
|
||||||
|
|
||||||
@@ -897,9 +1102,15 @@ the full request and creates an Event.
|
|||||||
| `path` | string | Unique bare UUID, generated at creation. The `/webhook/` prefix is route only and is not stored: the receiver matches this column against the raw `{uuid}` path segment |
|
| `path` | string | Unique bare UUID, generated at creation. The `/webhook/` prefix is route only and is not stored: the receiver matches this column against the raw `{uuid}` path segment |
|
||||||
| `description` | string | Optional description |
|
| `description` | string | Optional description |
|
||||||
| `active` | boolean | Whether this entrypoint accepts events (default: true) |
|
| `active` | boolean | Whether this entrypoint accepts events (default: true) |
|
||||||
|
| `signature_scheme` | string | How inbound requests are authenticated: `github`, `gitlab`, or empty for no verification (default: empty). See [Inbound Signature Verification](#inbound-signature-verification) |
|
||||||
|
| `signature_secret` | string | The secret shared with the sender, stored in the clear because HMAC verification needs the key itself. Never marshalled to JSON, never rendered, never logged. Empty when no scheme is set |
|
||||||
|
|
||||||
**Relations:** Belongs to Webhook.
|
**Relations:** Belongs to Webhook.
|
||||||
|
|
||||||
|
Both signature columns arrive through `AutoMigrate` with an empty
|
||||||
|
default, so every entrypoint written before they existed migrates to
|
||||||
|
"not configured" and keeps accepting the traffic it already accepted.
|
||||||
|
|
||||||
A webhook can have multiple entrypoints. This allows separate URLs for
|
A webhook can have multiple entrypoints. This allows separate URLs for
|
||||||
different event sources that all feed into the same processing pipeline
|
different event sources that all feed into the same processing pipeline
|
||||||
(e.g., one entrypoint for GitHub, another for Stripe, both routing to
|
(e.g., one entrypoint for GitHub, another for Stripe, both routing to
|
||||||
@@ -1008,6 +1219,23 @@ DeliveryResults.
|
|||||||
succeeded).
|
succeeded).
|
||||||
- **`failed`** — All retry attempts exhausted without success.
|
- **`failed`** — All retry attempts exhausted without success.
|
||||||
|
|
||||||
|
**Replay.** A `delivered` or `failed` delivery is finished as far as
|
||||||
|
the engine is concerned, but the event is still stored, so the event
|
||||||
|
log offers a per-delivery **Replay** action for it. Replay creates a
|
||||||
|
NEW `pending` delivery for the same event and target and hands it to
|
||||||
|
the engine on the ordinary path — same retries, same SSRF guard, same
|
||||||
|
circuit breaker as a first attempt. It never touches the delivery it
|
||||||
|
repeats: that row's status, timestamps and recorded attempts stand as
|
||||||
|
the record of what happened.
|
||||||
|
|
||||||
|
What is re-sent is the stored event body, against the target's
|
||||||
|
configuration **as it stands now** — the point of a replay is to
|
||||||
|
deliver where the destination has since been fixed. A target that has
|
||||||
|
been deleted or deactivated therefore refuses the replay with a
|
||||||
|
message on the event log rather than delivering from stale
|
||||||
|
configuration, and a replay is refused while an earlier one for the
|
||||||
|
same event and target is still pending or retrying.
|
||||||
|
|
||||||
#### DeliveryResult
|
#### DeliveryResult
|
||||||
|
|
||||||
The result of a single delivery attempt. Every attempt (including
|
The result of a single delivery attempt. Every attempt (including
|
||||||
@@ -1169,12 +1397,16 @@ External Service
|
|||||||
└─────────────┘ └──────────────┘ └──────┬───────┘
|
└─────────────┘ └──────────────┘ └──────┬───────┘
|
||||||
│
|
│
|
||||||
1. Look up Entrypoint by UUID
|
1. Look up Entrypoint by UUID
|
||||||
2. Capture full request as Event
|
2. Read the body under the 1 MB cap
|
||||||
3. Create Delivery records for each active Target
|
3. Verify the signature, if the entrypoint has
|
||||||
4. Build self-contained delivery.Task structs
|
one configured — 401 and no writes if it
|
||||||
|
fails (see Inbound Signature Verification)
|
||||||
|
4. Capture full request as Event
|
||||||
|
5. Create Delivery records for each active Target
|
||||||
|
6. Build self-contained delivery.Task structs
|
||||||
(target config + event data inline for
|
(target config + event data inline for
|
||||||
bodies < 16 KiB)
|
bodies < 16 KiB)
|
||||||
5. Notify Engine via channel (no DB read needed)
|
7. Notify Engine via channel (no DB read needed)
|
||||||
│
|
│
|
||||||
▼
|
▼
|
||||||
┌──────────────┐
|
┌──────────────┐
|
||||||
@@ -1335,6 +1567,7 @@ arriving and being stored, they are just not getting anywhere.
|
|||||||
| `webhooker_deliveries_succeeded_total` | counter | Deliveries that reached `delivered` |
|
| `webhooker_deliveries_succeeded_total` | counter | Deliveries that reached `delivered` |
|
||||||
| `webhooker_deliveries_failed_total` | counter | Deliveries that failed terminally and will not be retried |
|
| `webhooker_deliveries_failed_total` | counter | Deliveries that failed terminally and will not be retried |
|
||||||
| `webhooker_delivery_retries_total` | counter | Deliveries put back into `retrying` |
|
| `webhooker_delivery_retries_total` | counter | Deliveries put back into `retrying` |
|
||||||
|
| `webhooker_delivery_replays_total` | counter | Deliveries an operator replayed from the event log. A replay runs the ordinary engine path, so it also moves the attempt, outcome and duration series; this is the only one that separates it from ordinary traffic |
|
||||||
| `webhooker_delivery_duration_seconds` | histogram | Wall time of a single dispatched delivery attempt, the same duration the attempt's `DeliveryResult` records |
|
| `webhooker_delivery_duration_seconds` | histogram | Wall time of a single dispatched delivery attempt, the same duration the attempt's `DeliveryResult` records |
|
||||||
| `webhooker_deliveries_pending` | gauge | Deliveries currently in `pending` |
|
| `webhooker_deliveries_pending` | gauge | Deliveries currently in `pending` |
|
||||||
| `webhooker_deliveries_retrying` | gauge | Deliveries currently in `retrying` |
|
| `webhooker_deliveries_retrying` | gauge | Deliveries currently in `retrying` |
|
||||||
@@ -1932,9 +2165,11 @@ abuse limit later; they are tracked as future work.
|
|||||||
| `POST` | `/source/{id}/edit` | Edit webhook submission |
|
| `POST` | `/source/{id}/edit` | Edit webhook submission |
|
||||||
| `POST` | `/source/{id}/delete` | Delete webhook |
|
| `POST` | `/source/{id}/delete` | Delete webhook |
|
||||||
| `GET` | `/source/{id}/logs` | Webhook event logs |
|
| `GET` | `/source/{id}/logs` | Webhook event logs |
|
||||||
|
| `POST` | `/source/{id}/deliveries/{deliveryID}/replay` | Replay a finished delivery: creates a new delivery for the same event against the target's current configuration (30 per minute per bucket, then `429`) |
|
||||||
| `POST` | `/source/{id}/entrypoints` | Add entrypoint to webhook |
|
| `POST` | `/source/{id}/entrypoints` | Add entrypoint to webhook |
|
||||||
| `POST` | `/source/{id}/entrypoints/{entrypointID}/delete` | Delete an entrypoint |
|
| `POST` | `/source/{id}/entrypoints/{entrypointID}/delete` | Delete an entrypoint |
|
||||||
| `POST` | `/source/{id}/entrypoints/{entrypointID}/toggle` | Enable or disable an entrypoint |
|
| `POST` | `/source/{id}/entrypoints/{entrypointID}/toggle` | Enable or disable an entrypoint |
|
||||||
|
| `POST` | `/source/{id}/entrypoints/{entrypointID}/secret` | Set, rotate or remove the entrypoint's inbound signature scheme and secret (see [Inbound Signature Verification](#inbound-signature-verification)) |
|
||||||
| `POST` | `/source/{id}/targets` | Add target to webhook |
|
| `POST` | `/source/{id}/targets` | Add target to webhook |
|
||||||
| `POST` | `/source/{id}/targets/{targetID}/delete` | Delete a target |
|
| `POST` | `/source/{id}/targets/{targetID}/delete` | Delete a target |
|
||||||
| `POST` | `/source/{id}/targets/{targetID}/toggle` | Enable or disable a target |
|
| `POST` | `/source/{id}/targets/{targetID}/toggle` | Enable or disable a target |
|
||||||
@@ -1970,8 +2205,12 @@ imports. The entry point is `cmd/webhooker/main.go`.
|
|||||||
```
|
```
|
||||||
webhooker/
|
webhooker/
|
||||||
├── cmd/webhooker/
|
├── cmd/webhooker/
|
||||||
│ └── main.go # Entry point: sets globals, locks DATA_DIR, wires fx
|
│ └── main.go # Entry point: subcommand dispatch; no args locks DATA_DIR and wires fx
|
||||||
├── internal/
|
├── internal/
|
||||||
|
│ ├── banner/
|
||||||
|
│ │ └── banner.go # Ruled block for the one credential shown in the clear
|
||||||
|
│ ├── resetpw/
|
||||||
|
│ │ └── resetpw.go # `webhooker resetpw`: set an account's password, stopped deployments only
|
||||||
│ ├── config/
|
│ ├── config/
|
||||||
│ │ └── config.go # Configuration loading from environment variables
|
│ │ └── config.go # Configuration loading from environment variables
|
||||||
│ ├── database/
|
│ ├── database/
|
||||||
@@ -1981,7 +2220,7 @@ webhooker/
|
|||||||
│ │ ├── model_setting.go # Setting entity (key-value app config)
|
│ │ ├── model_setting.go # Setting entity (key-value app config)
|
||||||
│ │ ├── model_user.go # User entity
|
│ │ ├── model_user.go # User entity
|
||||||
│ │ ├── model_webhook.go # Webhook entity
|
│ │ ├── model_webhook.go # Webhook entity
|
||||||
│ │ ├── model_entrypoint.go # Entrypoint entity
|
│ │ ├── model_entrypoint.go # Entrypoint entity and SignatureScheme enum
|
||||||
│ │ ├── model_target.go # Target entity and TargetType enum
|
│ │ ├── model_target.go # Target entity and TargetType enum
|
||||||
│ │ ├── model_event.go # Event entity (per-webhook DB)
|
│ │ ├── model_event.go # Event entity (per-webhook DB)
|
||||||
│ │ ├── model_delivery.go # Delivery entity (per-webhook DB)
|
│ │ ├── model_delivery.go # Delivery entity (per-webhook DB)
|
||||||
@@ -2016,6 +2255,8 @@ webhooker/
|
|||||||
│ ├── handlers/
|
│ ├── handlers/
|
||||||
│ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering
|
│ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering
|
||||||
│ │ ├── auth.go # Login, logout handlers
|
│ │ ├── auth.go # Login, logout handlers
|
||||||
|
│ │ ├── delivery_replay.go # Per-delivery replay: new delivery, current target config
|
||||||
|
│ │ ├── entrypoint_view.go # Masked entrypoint view for templates
|
||||||
│ │ ├── event_log_view.go # Event log projection, byte-capped in SQL
|
│ │ ├── event_log_view.go # Event log projection, byte-capped in SQL
|
||||||
│ │ ├── healthcheck.go # Health check handler
|
│ │ ├── healthcheck.go # Health check handler
|
||||||
│ │ ├── index.go # Index page handler
|
│ │ ├── index.go # Index page handler
|
||||||
@@ -2040,9 +2281,11 @@ webhooker/
|
|||||||
│ │ ├── server.go # Server struct, fx lifecycle, signal handling
|
│ │ ├── server.go # Server struct, fx lifecycle, signal handling
|
||||||
│ │ ├── http.go # HTTP server setup with timeouts
|
│ │ ├── http.go # HTTP server setup with timeouts
|
||||||
│ │ └── routes.go # All route definitions
|
│ │ └── routes.go # All route definitions
|
||||||
│ └── session/
|
│ ├── session/
|
||||||
│ ├── session.go # Cookie-based session management
|
│ │ ├── session.go # Cookie-based session management
|
||||||
│ └── testing.go # NewForTest: Session without the fx lifecycle
|
│ │ └── testing.go # NewForTest: Session without the fx lifecycle
|
||||||
|
│ └── signature/
|
||||||
|
│ └── signature.go # Inbound signature verification (GitHub, GitLab)
|
||||||
├── static/
|
├── static/
|
||||||
│ ├── static.go # //go:embed directive
|
│ ├── static.go # //go:embed directive
|
||||||
│ ├── css/input.css # Tailwind input, source for tailwind.css (make css)
|
│ ├── css/input.css # Tailwind input, source for tailwind.css (make css)
|
||||||
@@ -2164,6 +2407,9 @@ check, see [The login endpoint](#the-login-endpoint).
|
|||||||
header. API keys are stored per-user with usage tracking
|
header. API keys are stored per-user with usage tracking
|
||||||
(`last_used_at`).
|
(`last_used_at`).
|
||||||
- **Metrics:** Basic authentication protecting the `/metrics` endpoint.
|
- **Metrics:** Basic authentication protecting the `/metrics` endpoint.
|
||||||
|
- **Recovery:** `webhooker resetpw <username>` on a stopped deployment
|
||||||
|
is the only way back into an account whose password was lost (see
|
||||||
|
[Recovering a lost admin password](#recovering-a-lost-admin-password)).
|
||||||
|
|
||||||
### Security
|
### Security
|
||||||
|
|
||||||
@@ -2184,6 +2430,15 @@ check, see [The login endpoint](#the-login-endpoint).
|
|||||||
`/api` (stateless API). The middleware auto-detects TLS status
|
`/api` (stateless API). The middleware auto-detects TLS status
|
||||||
per-request (via `r.TLS` and `X-Forwarded-Proto`) to set appropriate
|
per-request (via `r.TLS` and `X-Forwarded-Proto`) to set appropriate
|
||||||
cookie security flags and Origin/Referer validation mode
|
cookie security flags and Origin/Referer validation mode
|
||||||
|
- **Optional inbound signature verification** per entrypoint (GitHub
|
||||||
|
`X-Hub-Signature-256`, GitLab `X-Gitlab-Token`). Off by default and
|
||||||
|
off after an upgrade, so behaviour is unchanged until an operator
|
||||||
|
turns it on. Where it is on, an unsigned or wrongly signed request
|
||||||
|
is `401` and is not persisted, and a configuration the receiver
|
||||||
|
cannot apply fails closed rather than reverting to unverified. The
|
||||||
|
comparison is constant time and the secret never reaches a template,
|
||||||
|
a JSON response or a log line (see
|
||||||
|
[Inbound Signature Verification](#inbound-signature-verification))
|
||||||
- **SSRF prevention** for HTTP delivery targets: private/reserved IP
|
- **SSRF prevention** for HTTP delivery targets: private/reserved IP
|
||||||
ranges (RFC 1918, loopback, link-local, cloud metadata) are blocked
|
ranges (RFC 1918, loopback, link-local, cloud metadata) are blocked
|
||||||
both at target creation time (URL validation) and at delivery time
|
both at target creation time (URL validation) and at delivery time
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/healthcheck"
|
"sneak.berlin/go/webhooker/internal/healthcheck"
|
||||||
"sneak.berlin/go/webhooker/internal/logger"
|
"sneak.berlin/go/webhooker/internal/logger"
|
||||||
"sneak.berlin/go/webhooker/internal/middleware"
|
"sneak.berlin/go/webhooker/internal/middleware"
|
||||||
|
"sneak.berlin/go/webhooker/internal/resetpw"
|
||||||
"sneak.berlin/go/webhooker/internal/server"
|
"sneak.berlin/go/webhooker/internal/server"
|
||||||
"sneak.berlin/go/webhooker/internal/session"
|
"sneak.berlin/go/webhooker/internal/session"
|
||||||
)
|
)
|
||||||
@@ -48,6 +49,11 @@ import (
|
|||||||
// and can still consume the whole budget on their own.
|
// and can still consume the whole budget on their own.
|
||||||
const stopTimeout = 5 * time.Second
|
const stopTimeout = 5 * time.Second
|
||||||
|
|
||||||
|
// exitUsage is the status for a command line this binary cannot make
|
||||||
|
// sense of, kept distinct from the 1 a refusal exits with so that a
|
||||||
|
// caller can tell "called wrong" from "declined".
|
||||||
|
const exitUsage = 2
|
||||||
|
|
||||||
// Build-time variables set via -ldflags.
|
// Build-time variables set via -ldflags.
|
||||||
//
|
//
|
||||||
//nolint:gochecknoglobals // Build-time variables injected by the linker.
|
//nolint:gochecknoglobals // Build-time variables injected by the linker.
|
||||||
@@ -60,7 +66,54 @@ func main() {
|
|||||||
globals.Appname = appname
|
globals.Appname = appname
|
||||||
globals.Version = version
|
globals.Version = version
|
||||||
|
|
||||||
os.Exit(run(os.Stderr))
|
os.Exit(dispatch(os.Args[1:], os.Stdin, os.Stdout, os.Stderr))
|
||||||
|
}
|
||||||
|
|
||||||
|
// dispatch routes the command line to a subcommand.
|
||||||
|
//
|
||||||
|
// No arguments runs the server, which is what the image's CMD and
|
||||||
|
// every existing deployment invoke; that path is unchanged, including
|
||||||
|
// where the DATA_DIR lock is taken relative to building the fx graph
|
||||||
|
// and how fx propagates a non-zero exit itself.
|
||||||
|
func dispatch(
|
||||||
|
args []string,
|
||||||
|
stdin io.Reader,
|
||||||
|
stdout, stderr io.Writer,
|
||||||
|
) int {
|
||||||
|
if len(args) == 0 {
|
||||||
|
return run(stderr)
|
||||||
|
}
|
||||||
|
|
||||||
|
switch args[0] {
|
||||||
|
case resetpw.Name:
|
||||||
|
return resetpw.Run(args[1:], stdin, stdout, stderr)
|
||||||
|
case "help", "-h", "-help", "--help":
|
||||||
|
usage(stdout)
|
||||||
|
|
||||||
|
return 0
|
||||||
|
default:
|
||||||
|
_, _ = fmt.Fprintf(
|
||||||
|
stderr, "%s: unknown subcommand %q\n", appname, args[0],
|
||||||
|
)
|
||||||
|
usage(stderr)
|
||||||
|
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// usage lists what the binary can be asked to do.
|
||||||
|
func usage(w io.Writer) {
|
||||||
|
_, _ = fmt.Fprintf(w, `usage: %s [subcommand]
|
||||||
|
|
||||||
|
With no subcommand, runs the webhooker server.
|
||||||
|
|
||||||
|
Subcommands:
|
||||||
|
%s [-generate] <username>
|
||||||
|
Set an existing account's password on a stopped deployment.
|
||||||
|
Recovers an admin account whose bootstrap password was lost.
|
||||||
|
help
|
||||||
|
Print this message.
|
||||||
|
`, appname, resetpw.Name)
|
||||||
}
|
}
|
||||||
|
|
||||||
// run takes the exclusive DATA_DIR lock, then runs the application
|
// run takes the exclusive DATA_DIR lock, then runs the application
|
||||||
|
|||||||
@@ -2,12 +2,14 @@ package main
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"bytes"
|
"bytes"
|
||||||
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"sneak.berlin/go/webhooker/internal/datadir"
|
"sneak.berlin/go/webhooker/internal/datadir"
|
||||||
|
"sneak.berlin/go/webhooker/internal/resetpw"
|
||||||
"sneak.berlin/go/webhooker/internal/server"
|
"sneak.berlin/go/webhooker/internal/server"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -68,6 +70,65 @@ func TestRunRefusesLockedDataDir(t *testing.T) {
|
|||||||
assert.Contains(t, stderr.String(), "another instance")
|
assert.Contains(t, stderr.String(), "another instance")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestDispatch_NoArgumentsRunsTheServer pins the routing of a bare
|
||||||
|
// invocation, which is what the image's CMD and every deployment use.
|
||||||
|
// Adding subcommands must not move the server off the empty argument
|
||||||
|
// list, and must not move the DATA_DIR lock: this asserts the refusal
|
||||||
|
// arrives with no fx graph built, exactly as run does on its own.
|
||||||
|
func TestDispatch_NoArgumentsRunsTheServer(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
t.Setenv("DATA_DIR", dir)
|
||||||
|
|
||||||
|
lock, err := datadir.Acquire(dir)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
defer func() { _ = lock.Release() }()
|
||||||
|
|
||||||
|
var stdout, stderr bytes.Buffer
|
||||||
|
|
||||||
|
code := dispatch(nil, strings.NewReader(""), &stdout, &stderr)
|
||||||
|
|
||||||
|
require.Equal(t, 1, code)
|
||||||
|
assert.Contains(t, stderr.String(), "another instance")
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestDispatch_UnknownSubcommand keeps a mistyped subcommand from
|
||||||
|
// starting a server. Anything else would have `webhooker resetpww`
|
||||||
|
// silently take the DATA_DIR lock and serve.
|
||||||
|
func TestDispatch_UnknownSubcommand(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var stdout, stderr bytes.Buffer
|
||||||
|
|
||||||
|
code := dispatch(
|
||||||
|
[]string{"resetpww", "admin"},
|
||||||
|
strings.NewReader(""), &stdout, &stderr,
|
||||||
|
)
|
||||||
|
|
||||||
|
require.Equal(t, 2, code)
|
||||||
|
assert.Contains(t, stderr.String(), "unknown subcommand")
|
||||||
|
assert.Contains(
|
||||||
|
t, stderr.String(), resetpw.Name,
|
||||||
|
"the usage must name the subcommand that does exist",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestDispatch_Help answers on standard output with a zero status, so
|
||||||
|
// `webhooker help` is usable in a pipe.
|
||||||
|
func TestDispatch_Help(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var stdout, stderr bytes.Buffer
|
||||||
|
|
||||||
|
code := dispatch(
|
||||||
|
[]string{"help"}, strings.NewReader(""), &stdout, &stderr,
|
||||||
|
)
|
||||||
|
|
||||||
|
require.Equal(t, 0, code)
|
||||||
|
assert.Empty(t, stderr.String())
|
||||||
|
assert.Contains(t, stdout.String(), resetpw.Name)
|
||||||
|
}
|
||||||
|
|
||||||
// tailHeadroom is the slack the fx stop budget must keep beyond the
|
// tailHeadroom is the slack the fx stop budget must keep beyond the
|
||||||
// server stop hook. The hooks that run after the server — the
|
// server stop hook. The hooks that run after the server — the
|
||||||
// delivery engine, the healthcheck, the webhook DB manager and the
|
// delivery engine, the healthcheck, the webhook DB manager and the
|
||||||
|
|||||||
47
internal/banner/banner.go
Normal file
47
internal/banner/banner.go
Normal file
@@ -0,0 +1,47 @@
|
|||||||
|
// Package banner renders the operator-facing blocks that carry a
|
||||||
|
// plaintext credential.
|
||||||
|
//
|
||||||
|
// A generated password printed as one more structured log line is lost:
|
||||||
|
// a boot writes roughly 45 fx PROVIDE/RUN/HOOK lines around it, and
|
||||||
|
// under `docker run -d` it is one line in a log subject to rotation. A
|
||||||
|
// credential that is shown exactly once has to be findable by eye when
|
||||||
|
// an operator scrolls back, so it is written as a ruled block rather
|
||||||
|
// than as a log record.
|
||||||
|
//
|
||||||
|
// It is deliberately not a log line: it goes straight to the writer the
|
||||||
|
// caller names — standard output for both the first-boot account and
|
||||||
|
// the `resetpw` subcommand — so it is neither levelled, filtered, nor
|
||||||
|
// rendered as JSON by whichever handler internal/logger installed.
|
||||||
|
package banner
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ruleWidth is the length of the horizontal rules, chosen to fit an
|
||||||
|
// 80-column terminal without wrapping.
|
||||||
|
const ruleWidth = 72
|
||||||
|
|
||||||
|
// Credentials writes a ruled block naming an account and its plaintext
|
||||||
|
// password. headline says which event produced it, and note says what
|
||||||
|
// the operator must do about it; both are written verbatim, so a
|
||||||
|
// multi-line note must already be wrapped.
|
||||||
|
func Credentials(
|
||||||
|
w io.Writer,
|
||||||
|
headline, username, password, note string,
|
||||||
|
) error {
|
||||||
|
rule := strings.Repeat("=", ruleWidth)
|
||||||
|
|
||||||
|
_, err := fmt.Fprintf(
|
||||||
|
w,
|
||||||
|
"\n%s\n%s\n\n username: %s\n password: %s\n\n%s\n%s\n\n",
|
||||||
|
rule, headline, username, password, note, rule,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("writing credentials banner: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
59
internal/banner/banner_test.go
Normal file
59
internal/banner/banner_test.go
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
package banner_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"sneak.berlin/go/webhooker/internal/banner"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestCredentials_IsFindableByEye pins the properties that make the
|
||||||
|
// block worth having: rules above and below it, the two fields on
|
||||||
|
// their own lines, and blank lines separating it from whatever the
|
||||||
|
// surrounding log wrote.
|
||||||
|
func TestCredentials_IsFindableByEye(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var out bytes.Buffer
|
||||||
|
|
||||||
|
require.NoError(t, banner.Credentials(
|
||||||
|
&out, "HEADLINE", "admin", "s3cret", "NOTE",
|
||||||
|
))
|
||||||
|
|
||||||
|
got := out.String()
|
||||||
|
lines := strings.Split(strings.Trim(got, "\n"), "\n")
|
||||||
|
|
||||||
|
require.GreaterOrEqual(t, len(lines), 3)
|
||||||
|
assert.Equal(t, lines[0], lines[len(lines)-1], "rules must match")
|
||||||
|
assert.Greater(
|
||||||
|
t, len(lines[0]), 40, "the rule must be visible at a glance",
|
||||||
|
)
|
||||||
|
assert.Equal(t, strings.Repeat("=", len(lines[0])), lines[0])
|
||||||
|
|
||||||
|
assert.Contains(t, got, "\n username: admin\n")
|
||||||
|
assert.Contains(t, got, "\n password: s3cret\n")
|
||||||
|
assert.Contains(t, got, "HEADLINE")
|
||||||
|
assert.Contains(t, got, "NOTE")
|
||||||
|
assert.True(t, strings.HasPrefix(got, "\n"))
|
||||||
|
}
|
||||||
|
|
||||||
|
// failingWriter reports the write error a banner must not swallow: it
|
||||||
|
// is the one copy of a password that will never be shown again.
|
||||||
|
type failingWriter struct{}
|
||||||
|
|
||||||
|
func (failingWriter) Write([]byte) (int, error) {
|
||||||
|
return 0, assert.AnError
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCredentials_ReportsAWriteFailure(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
err := banner.Credentials(
|
||||||
|
failingWriter{}, "HEADLINE", "admin", "s3cret", "NOTE",
|
||||||
|
)
|
||||||
|
|
||||||
|
require.ErrorIs(t, err, assert.AnError)
|
||||||
|
}
|
||||||
@@ -136,9 +136,12 @@ type Config struct {
|
|||||||
// This only ever adds destinations to what the guard would
|
// This only ever adds destinations to what the guard would
|
||||||
// otherwise refuse. The guard itself is always on: there is no
|
// otherwise refuse. The guard itself is always on: there is no
|
||||||
// setting that disables SSRF protection, and delivery's
|
// setting that disables SSRF protection, and delivery's
|
||||||
// alwaysBlockedNetworks — link-local plus every known cloud
|
// alwaysBlockedNetworks stays blocked no matter what is listed
|
||||||
// metadata endpoint outside it — stays blocked no matter what
|
// here. That set is link-local plus the cloud metadata
|
||||||
// is listed here.
|
// endpoints outside it that disclose credentials or user data
|
||||||
|
// at a provider-fixed address; it is not exhaustive of every
|
||||||
|
// cloud's metadata address. See alwaysBlockedNetworks for the
|
||||||
|
// authoritative list and the criterion it is built from.
|
||||||
AllowedEgressCIDRs []netip.Prefix
|
AllowedEgressCIDRs []netip.Prefix
|
||||||
|
|
||||||
params *ConfigParams
|
params *ConfigParams
|
||||||
|
|||||||
85
internal/database/bootstrap_banner_test.go
Normal file
85
internal/database/bootstrap_banner_test.go
Normal file
@@ -0,0 +1,85 @@
|
|||||||
|
package database_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
)
|
||||||
|
|
||||||
|
// passwordField is the banner line carrying the plaintext.
|
||||||
|
const passwordField = "password: "
|
||||||
|
|
||||||
|
// bannerPassword returns the password the banner printed.
|
||||||
|
func bannerPassword(t *testing.T, out string) string {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
for line := range strings.SplitSeq(out, "\n") {
|
||||||
|
_, value, found := strings.Cut(line, passwordField)
|
||||||
|
if found {
|
||||||
|
return strings.TrimSpace(value)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Fatalf("no %q line in the banner:\n%s", passwordField, out)
|
||||||
|
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestFirstBoot_PrintsTheAdminPasswordAsABanner is the bootstrap half
|
||||||
|
// of https://git.eeqj.de/sneak/webhooker/issues/208.
|
||||||
|
//
|
||||||
|
// The password is shown exactly once, and it used to be shown as one
|
||||||
|
// slog record among the roughly 45 fx PROVIDE/RUN/HOOK lines a boot
|
||||||
|
// writes — which is how deployments lost it and, with no reset path,
|
||||||
|
// locked themselves out. It must be emitted as a block an operator can
|
||||||
|
// find by eye, it must carry the plaintext that actually opens the
|
||||||
|
// account, and it must name the command that recovers it.
|
||||||
|
func TestFirstBoot_PrintsTheAdminPasswordAsABanner(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
db, lc := setupTestDB(t)
|
||||||
|
|
||||||
|
var out bytes.Buffer
|
||||||
|
|
||||||
|
db.ExportSetBannerOut(&out)
|
||||||
|
|
||||||
|
ctx := context.Background()
|
||||||
|
require.NoError(t, lc.Start(ctx))
|
||||||
|
|
||||||
|
defer func() { require.NoError(t, lc.Stop(ctx)) }()
|
||||||
|
|
||||||
|
printed := out.String()
|
||||||
|
|
||||||
|
require.Contains(
|
||||||
|
t, printed, strings.Repeat("=", 20),
|
||||||
|
"the banner must be ruled off, not read as one more log line",
|
||||||
|
)
|
||||||
|
require.Contains(t, printed, "username: admin")
|
||||||
|
assert.Contains(
|
||||||
|
t, printed, "resetpw",
|
||||||
|
"the banner must name the command that recovers the account",
|
||||||
|
)
|
||||||
|
|
||||||
|
password := bannerPassword(t, printed)
|
||||||
|
require.NotEmpty(t, password)
|
||||||
|
|
||||||
|
// The printed plaintext must be the one that opens the account:
|
||||||
|
// a banner showing a different string would be worse than none.
|
||||||
|
var user database.User
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t,
|
||||||
|
db.DB().Where("username = ?", "admin").First(&user).Error,
|
||||||
|
)
|
||||||
|
|
||||||
|
ok, err := database.VerifyPassword(password, user.Password)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.True(
|
||||||
|
t, ok, "the printed password must open the seeded account",
|
||||||
|
)
|
||||||
|
}
|
||||||
@@ -8,6 +8,7 @@ import (
|
|||||||
"encoding/base64"
|
"encoding/base64"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"io"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
@@ -16,6 +17,7 @@ import (
|
|||||||
"gorm.io/driver/sqlite"
|
"gorm.io/driver/sqlite"
|
||||||
"gorm.io/gorm"
|
"gorm.io/gorm"
|
||||||
_ "modernc.org/sqlite" // Pure Go SQLite driver
|
_ "modernc.org/sqlite" // Pure Go SQLite driver
|
||||||
|
"sneak.berlin/go/webhooker/internal/banner"
|
||||||
"sneak.berlin/go/webhooker/internal/config"
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
"sneak.berlin/go/webhooker/internal/gormlog"
|
"sneak.berlin/go/webhooker/internal/gormlog"
|
||||||
"sneak.berlin/go/webhooker/internal/logger"
|
"sneak.berlin/go/webhooker/internal/logger"
|
||||||
@@ -27,6 +29,20 @@ const (
|
|||||||
sessionKeyLen = 32
|
sessionKeyLen = 32
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// MainDBFileName is the main application database inside DATA_DIR. It
|
||||||
|
// is exported so that an entry point acting on a data directory
|
||||||
|
// outside the fx graph can test for a deployment's existence without
|
||||||
|
// spelling the name a second time.
|
||||||
|
const MainDBFileName = "webhooker.db"
|
||||||
|
|
||||||
|
// BootstrapPasswordNote is what the first-boot banner tells the
|
||||||
|
// operator to do about the password it just printed. It names the
|
||||||
|
// recovery command, because the moment that line scrolls away is
|
||||||
|
// exactly when the operator needs to know one exists.
|
||||||
|
const BootstrapPasswordNote = "Save this password now: it is shown " +
|
||||||
|
"only here, and only once.\nIf it is lost, run `webhooker " +
|
||||||
|
"resetpw admin` on a stopped deployment."
|
||||||
|
|
||||||
//nolint:revive // DatabaseParams is a standard fx naming convention.
|
//nolint:revive // DatabaseParams is a standard fx naming convention.
|
||||||
type DatabaseParams struct {
|
type DatabaseParams struct {
|
||||||
fx.In
|
fx.In
|
||||||
@@ -40,6 +56,39 @@ type Database struct {
|
|||||||
db *gorm.DB
|
db *gorm.DB
|
||||||
log *slog.Logger
|
log *slog.Logger
|
||||||
params *DatabaseParams
|
params *DatabaseParams
|
||||||
|
|
||||||
|
// bannerOut receives the first-boot credentials banner. Nil means
|
||||||
|
// os.Stdout, resolved at write time rather than at construction so
|
||||||
|
// that a caller which redirects the variable still captures it.
|
||||||
|
bannerOut io.Writer
|
||||||
|
}
|
||||||
|
|
||||||
|
// Open connects to the main database in dataDir and migrates it,
|
||||||
|
// without the fx lifecycle and without seeding an admin account.
|
||||||
|
//
|
||||||
|
// It is for entry points that act on an existing deployment's data
|
||||||
|
// directory from outside the server graph — `webhooker resetpw`. Such a
|
||||||
|
// caller must already hold the DATA_DIR lock (see internal/datadir),
|
||||||
|
// and must Close the result.
|
||||||
|
//
|
||||||
|
// It does not create the admin account: seeding belongs to a server
|
||||||
|
// start, and a maintenance command that silently invented an account
|
||||||
|
// would answer "no such user" by creating one.
|
||||||
|
func Open(dataDir string, log *slog.Logger) (*Database, error) {
|
||||||
|
d := &Database{log: log}
|
||||||
|
|
||||||
|
err := d.connectTo(dataDir)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
return d, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close closes the underlying connection. It is the exported form of
|
||||||
|
// the fx stop hook, for callers that built the Database with Open.
|
||||||
|
func (d *Database) Close() error {
|
||||||
|
return d.close()
|
||||||
}
|
}
|
||||||
|
|
||||||
// New creates a Database that connects on fx start and disconnects on stop.
|
// New creates a Database that connects on fx start and disconnects on stop.
|
||||||
@@ -122,10 +171,22 @@ func (d *Database) GetOrCreateSessionKey() (string, error) {
|
|||||||
return encoded, nil
|
return encoded, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// connect opens the configured data directory and, this being a
|
||||||
|
// server start, seeds the admin account when the deployment has none.
|
||||||
func (d *Database) connect() error {
|
func (d *Database) connect() error {
|
||||||
// Ensure the data directory exists before opening the database.
|
err := d.connectTo(d.params.Config.DataDir)
|
||||||
dataDir := d.params.Config.DataDir
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
return d.ensureAdminUser()
|
||||||
|
}
|
||||||
|
|
||||||
|
// connectTo opens and migrates the main database in dataDir. It seeds
|
||||||
|
// nothing: whether an empty deployment gets an admin account is the
|
||||||
|
// caller's decision.
|
||||||
|
func (d *Database) connectTo(dataDir string) error {
|
||||||
|
// Ensure the data directory exists before opening the database.
|
||||||
err := os.MkdirAll(dataDir, dataDirPerm)
|
err := os.MkdirAll(dataDir, dataDirPerm)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf(
|
return fmt.Errorf(
|
||||||
@@ -136,7 +197,7 @@ func (d *Database) connect() error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Construct the main application database path inside DATA_DIR.
|
// Construct the main application database path inside DATA_DIR.
|
||||||
dbPath := filepath.Join(dataDir, "webhooker.db")
|
dbPath := filepath.Join(dataDir, MainDBFileName)
|
||||||
dbURL := fmt.Sprintf(
|
dbURL := fmt.Sprintf(
|
||||||
"file:%s?cache=shared&mode=rwc",
|
"file:%s?cache=shared&mode=rwc",
|
||||||
dbPath,
|
dbPath,
|
||||||
@@ -190,10 +251,16 @@ func (d *Database) migrate() error {
|
|||||||
|
|
||||||
d.log.Info("database migrations completed")
|
d.log.Info("database migrations completed")
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ensureAdminUser creates the bootstrap admin account when the
|
||||||
|
// deployment has no users at all.
|
||||||
|
func (d *Database) ensureAdminUser() error {
|
||||||
// Check if admin user exists
|
// Check if admin user exists
|
||||||
var userCount int64
|
var userCount int64
|
||||||
|
|
||||||
err = d.db.Model(&User{}).Count(&userCount).Error
|
err := d.db.Model(&User{}).Count(&userCount).Error
|
||||||
if err != nil {
|
if err != nil {
|
||||||
d.log.Error(
|
d.log.Error(
|
||||||
"failed to count users",
|
"failed to count users",
|
||||||
@@ -253,16 +320,46 @@ func (d *Database) createAdminUser() error {
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
d.log.Info("admin user created",
|
// The plaintext leaves this process here and nowhere else. It is
|
||||||
"username", "admin",
|
// deliberately not a log field: as one INFO record among the fx
|
||||||
"password", password,
|
// graph's own output it read as one more startup line, which is
|
||||||
"message",
|
// how deployments lost it. See internal/banner.
|
||||||
"SAVE THIS PASSWORD - it will not be shown again!",
|
err = banner.Credentials(
|
||||||
|
d.banner(),
|
||||||
|
"WEBHOOKER FIRST BOOT: an admin account has been created.",
|
||||||
|
adminUser.Username,
|
||||||
|
password,
|
||||||
|
BootstrapPasswordNote,
|
||||||
)
|
)
|
||||||
|
if err != nil {
|
||||||
|
// Fail the start. The account is already committed, so the
|
||||||
|
// next boot seeds nothing and prints nothing: continuing here
|
||||||
|
// would hand the operator a running service whose only
|
||||||
|
// password was never shown. `webhooker resetpw` recovers it.
|
||||||
|
d.log.Error(
|
||||||
|
"failed to print the admin credentials banner",
|
||||||
|
"error", err,
|
||||||
|
)
|
||||||
|
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
d.log.Info("admin user created", "username", adminUser.Username)
|
||||||
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// banner returns where the credentials banner is written. os.Stdout is
|
||||||
|
// resolved here rather than stored, so that a test which redirects the
|
||||||
|
// variable captures the banner.
|
||||||
|
func (d *Database) banner() io.Writer {
|
||||||
|
if d.bannerOut != nil {
|
||||||
|
return d.bannerOut
|
||||||
|
}
|
||||||
|
|
||||||
|
return os.Stdout
|
||||||
|
}
|
||||||
|
|
||||||
func (d *Database) close() error {
|
func (d *Database) close() error {
|
||||||
if d.db != nil {
|
if d.db != nil {
|
||||||
sqlDB, err := d.db.DB()
|
sqlDB, err := d.db.DB()
|
||||||
|
|||||||
159
internal/database/event_db_isolation.go
Normal file
159
internal/database/event_db_isolation.go
Normal file
@@ -0,0 +1,159 @@
|
|||||||
|
package database
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
|
||||||
|
"gorm.io/gorm"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
|
)
|
||||||
|
|
||||||
|
// omitAssociationsCallback is the name the association guard is
|
||||||
|
// registered under on a per-webhook database's create and update
|
||||||
|
// callback chains.
|
||||||
|
const omitAssociationsCallback = "webhooker:omit_associations"
|
||||||
|
|
||||||
|
// omitAssociations makes every create and update issued against a
|
||||||
|
// per-webhook database skip GORM's automatic association save.
|
||||||
|
//
|
||||||
|
// A per-webhook database holds the event tier only, but Delivery
|
||||||
|
// declares belongs-to Event and Target and the delivery engine fills
|
||||||
|
// both in memory before writing. Without this guard GORM upserts
|
||||||
|
// those parent rows here on the delivery and retry write paths,
|
||||||
|
// copying targets.config, which holds destination URLs and bearer
|
||||||
|
// credentials, into the file most likely to be backed up or handed
|
||||||
|
// to someone else. Registering the guard on the connection covers
|
||||||
|
// every write path, including writes inside a transaction and write
|
||||||
|
// paths added later. Every event-tier row this file holds is written
|
||||||
|
// explicitly, so nothing depends on the automatic save.
|
||||||
|
func omitAssociations(db *gorm.DB) error {
|
||||||
|
omit := func(tx *gorm.DB) {
|
||||||
|
tx.Statement.Omits = append(
|
||||||
|
tx.Statement.Omits, clause.Associations,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
err := db.Callback().Create().
|
||||||
|
Before("gorm:save_before_associations").
|
||||||
|
Register(omitAssociationsCallback, omit)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf(
|
||||||
|
"registering create association guard: %w", err,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
err = db.Callback().Update().
|
||||||
|
Before("gorm:save_before_associations").
|
||||||
|
Register(omitAssociationsCallback, omit)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf(
|
||||||
|
"registering update association guard: %w", err,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// eventDBSweptVersion is the PRAGMA user_version purgeTargetRows
|
||||||
|
// stamps into a per-webhook database once it has removed any leaked
|
||||||
|
// target rows *and* the VACUUM that removes their bytes has returned.
|
||||||
|
// Nothing else in the tree uses user_version, so 0 means "not swept
|
||||||
|
// by this build".
|
||||||
|
//
|
||||||
|
// The stamp, not the DELETE, is what records that a file is done. A
|
||||||
|
// DELETE commits on its own, so a sweep that is interrupted or whose
|
||||||
|
// VACUUM fails leaves a file whose rows are gone but whose credential
|
||||||
|
// bytes are still in the free pages -- indistinguishable, by row
|
||||||
|
// count, from a file that never leaked. Both leave the stamp unset,
|
||||||
|
// so the next open sweeps again.
|
||||||
|
const eventDBSweptVersion = 1
|
||||||
|
|
||||||
|
// purgeTargetRows deletes target rows that an earlier build's
|
||||||
|
// association upsert wrote into a per-webhook database, and rewrites
|
||||||
|
// the file so their bytes are gone with them. AutoMigrate creates a
|
||||||
|
// targets table in every one of these files because Delivery declares
|
||||||
|
// a belongs-to Target, but nothing in the event tier may put rows in
|
||||||
|
// it. The rows it did put there are junk, not history: they carry an
|
||||||
|
// empty webhook_id, and delivery rows resolve their target against
|
||||||
|
// the main database, so nothing here refers to them.
|
||||||
|
//
|
||||||
|
// The DELETE only unlinks the rows: modernc.org/sqlite leaves
|
||||||
|
// secure_delete at SQLite's default of off, so the credential bytes
|
||||||
|
// stay readable in the file's free pages and a backup of a swept file
|
||||||
|
// would still hand them over. VACUUM rewrites the file without them.
|
||||||
|
//
|
||||||
|
// This runs before every migration and is gated on
|
||||||
|
// eventDBSweptVersion, so a file pays for the rewrite once, on the
|
||||||
|
// first open that finds it unstamped, and every open after that is a
|
||||||
|
// PRAGMA read. A file this build created is stamped before its
|
||||||
|
// targets table exists, so it never vacuums at all. A failure here
|
||||||
|
// fails the open with the stamp left unset, so the sweep is retried
|
||||||
|
// rather than skipped -- a webhook whose file cannot be swept stays
|
||||||
|
// unusable instead of quietly serving from a file that still holds
|
||||||
|
// recoverable credentials.
|
||||||
|
func purgeTargetRows(
|
||||||
|
db *gorm.DB, log *slog.Logger, webhookID string,
|
||||||
|
) error {
|
||||||
|
var version int
|
||||||
|
|
||||||
|
// Row().Scan, not (*gorm.DB).Scan: see internal/gormlog.
|
||||||
|
err := db.Raw("PRAGMA user_version").Row().Scan(&version)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf(
|
||||||
|
"reading sweep marker of webhook database %s: %w",
|
||||||
|
webhookID, err,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if version >= eventDBSweptVersion {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var purged int64
|
||||||
|
|
||||||
|
if db.Migrator().HasTable("targets") {
|
||||||
|
res := db.Exec("DELETE FROM targets")
|
||||||
|
if res.Error != nil {
|
||||||
|
return fmt.Errorf(
|
||||||
|
"purging target rows from webhook database %s: %w",
|
||||||
|
webhookID, res.Error,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
purged = res.RowsAffected
|
||||||
|
|
||||||
|
// Unconditional: a zero row count here does not mean there is
|
||||||
|
// nothing to remove, only that no *live* row is left. See
|
||||||
|
// eventDBSweptVersion.
|
||||||
|
err = db.Exec("VACUUM").Error
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf(
|
||||||
|
"purged %d leaked target rows from webhook database "+
|
||||||
|
"%s but vacuuming it failed, so the deleted "+
|
||||||
|
"target credentials are still recoverable from "+
|
||||||
|
"the file; it stays marked unswept and the next "+
|
||||||
|
"open retries: %w",
|
||||||
|
purged, webhookID, err,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
err = db.Exec(fmt.Sprintf(
|
||||||
|
"PRAGMA user_version = %d", eventDBSweptVersion,
|
||||||
|
)).Error
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf(
|
||||||
|
"marking webhook database %s swept: %w", webhookID, err,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if purged > 0 {
|
||||||
|
log.Warn(
|
||||||
|
"purged leaked target rows from per-webhook database",
|
||||||
|
"webhook_id", webhookID,
|
||||||
|
"rows", purged,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
438
internal/database/event_db_isolation_test.go
Normal file
438
internal/database/event_db_isolation_test.go
Normal file
@@ -0,0 +1,438 @@
|
|||||||
|
package database_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"database/sql"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/google/uuid"
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
_ "modernc.org/sqlite"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
)
|
||||||
|
|
||||||
|
// testDataDirPerm is the mode the test data directory is created
|
||||||
|
// with.
|
||||||
|
const testDataDirPerm = 0o750
|
||||||
|
|
||||||
|
// eventDBDataDir returns a data directory that a WebhookDBManager
|
||||||
|
// can be pointed at.
|
||||||
|
func eventDBDataDir(t *testing.T) string {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
dir := filepath.Join(t.TempDir(), "events")
|
||||||
|
require.NoError(t, os.MkdirAll(dir, testDataDirPerm))
|
||||||
|
|
||||||
|
return dir
|
||||||
|
}
|
||||||
|
|
||||||
|
// openRawEventDB opens the per-webhook database file directly,
|
||||||
|
// without the manager, so a test can put a file on disk in a state
|
||||||
|
// the manager has to cope with, or inspect one afterwards.
|
||||||
|
func openRawEventDB(
|
||||||
|
t *testing.T, dataDir, webhookID string,
|
||||||
|
) *sql.DB {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
path := filepath.Join(
|
||||||
|
dataDir, fmt.Sprintf("events-%s.db", webhookID),
|
||||||
|
)
|
||||||
|
|
||||||
|
sqlDB, err := sql.Open(
|
||||||
|
"sqlite",
|
||||||
|
fmt.Sprintf("file:%s?mode=rwc", path),
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
t.Cleanup(func() { _ = sqlDB.Close() })
|
||||||
|
|
||||||
|
return sqlDB
|
||||||
|
}
|
||||||
|
|
||||||
|
// eventDBFileBytes reads a per-webhook database file off disk, so a
|
||||||
|
// test can assert on what the file itself still holds rather than on
|
||||||
|
// what a query returns.
|
||||||
|
func eventDBFileBytes(t *testing.T, dataDir, webhookID string) []byte {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
//nolint:gosec // reads a file the test just created under t.TempDir()
|
||||||
|
raw, err := os.ReadFile(filepath.Join(
|
||||||
|
dataDir, fmt.Sprintf("events-%s.db", webhookID),
|
||||||
|
))
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
return raw
|
||||||
|
}
|
||||||
|
|
||||||
|
// eventDBUserVersion returns the PRAGMA user_version of a per-webhook
|
||||||
|
// database file, which is the marker purgeTargetRows stamps once it
|
||||||
|
// has swept and vacuumed.
|
||||||
|
func eventDBUserVersion(t *testing.T, sqlDB *sql.DB) int {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var version int
|
||||||
|
|
||||||
|
require.NoError(t, sqlDB.QueryRowContext(
|
||||||
|
t.Context(), "PRAGMA user_version",
|
||||||
|
).Scan(&version))
|
||||||
|
|
||||||
|
return version
|
||||||
|
}
|
||||||
|
|
||||||
|
// clearEventDBSweptMarker resets the sweep marker to 0, which is what
|
||||||
|
// a file written by a build without the sweep looks like. Tests that
|
||||||
|
// seed a leaked row have to create the file through the manager to
|
||||||
|
// get the real targets table shape, and that stamps it.
|
||||||
|
func clearEventDBSweptMarker(t *testing.T, sqlDB *sql.DB) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
_, err := sqlDB.ExecContext(t.Context(), "PRAGMA user_version = 0")
|
||||||
|
require.NoError(t, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// countTargetRows returns the number of rows in the targets table of
|
||||||
|
// a per-webhook database file, or -1 if the table does not exist.
|
||||||
|
func countTargetRows(t *testing.T, sqlDB *sql.DB) int {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var tables int
|
||||||
|
|
||||||
|
require.NoError(t, sqlDB.QueryRowContext(
|
||||||
|
t.Context(),
|
||||||
|
"SELECT count(*) FROM sqlite_master "+
|
||||||
|
"WHERE type = 'table' AND name = 'targets'",
|
||||||
|
).Scan(&tables))
|
||||||
|
|
||||||
|
if tables == 0 {
|
||||||
|
return -1
|
||||||
|
}
|
||||||
|
|
||||||
|
var rows int
|
||||||
|
|
||||||
|
require.NoError(t, sqlDB.QueryRowContext(
|
||||||
|
t.Context(), "SELECT count(*) FROM targets",
|
||||||
|
).Scan(&rows))
|
||||||
|
|
||||||
|
return rows
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestOpenPurgesLeakedTargetRows covers the sweep for event
|
||||||
|
// databases written by a build that let GORM upsert target rows
|
||||||
|
// into them: opening the database clears them, and opening it again
|
||||||
|
// is a no-op.
|
||||||
|
func TestOpenPurgesLeakedTargetRows(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
dataDir := eventDBDataDir(t)
|
||||||
|
webhookID := uuid.New().String()
|
||||||
|
|
||||||
|
// Create the file the way the application does, so the targets
|
||||||
|
// table has exactly the shape AutoMigrate gives it, then write
|
||||||
|
// a leaked row into it the way the association upsert did.
|
||||||
|
initial := database.NewTestWebhookDBManager(dataDir)
|
||||||
|
|
||||||
|
_, err := initial.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.NoError(t, initial.CloseAll())
|
||||||
|
|
||||||
|
seed := openRawEventDB(t, dataDir, webhookID)
|
||||||
|
|
||||||
|
_, err = seed.ExecContext(
|
||||||
|
t.Context(),
|
||||||
|
"INSERT INTO targets "+
|
||||||
|
"(id, webhook_id, name, type, config) "+
|
||||||
|
"VALUES (?, '', ?, ?, ?)",
|
||||||
|
uuid.New().String(),
|
||||||
|
"leaked-target",
|
||||||
|
"slack",
|
||||||
|
`{"webhookUrl":"https://hooks.example/T000/B000/secret"}`,
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Equal(t, 1, countTargetRows(t, seed))
|
||||||
|
clearEventDBSweptMarker(t, seed)
|
||||||
|
require.NoError(t, seed.Close())
|
||||||
|
|
||||||
|
mgr := database.NewTestWebhookDBManager(dataDir)
|
||||||
|
|
||||||
|
_, err = mgr.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.NoError(t, mgr.CloseAll())
|
||||||
|
|
||||||
|
check := openRawEventDB(t, dataDir, webhookID)
|
||||||
|
assert.Zero(t, countTargetRows(t, check))
|
||||||
|
assert.Equal(
|
||||||
|
t, 1, eventDBUserVersion(t, check),
|
||||||
|
"a completed sweep must mark the file so later opens skip it",
|
||||||
|
)
|
||||||
|
require.NoError(t, check.Close())
|
||||||
|
|
||||||
|
// Idempotent: a second open leaves it at zero and does not
|
||||||
|
// error.
|
||||||
|
again := database.NewTestWebhookDBManager(dataDir)
|
||||||
|
|
||||||
|
_, err = again.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.NoError(t, again.CloseAll())
|
||||||
|
|
||||||
|
recheck := openRawEventDB(t, dataDir, webhookID)
|
||||||
|
assert.Zero(t, countTargetRows(t, recheck))
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestOpenPurgeRemovesCredentialBytes covers the sweep at the level
|
||||||
|
// that matters for a backup handed to someone else: the leaked
|
||||||
|
// credential must be gone from the raw bytes of the file, not merely
|
||||||
|
// unreachable by query. A bare DELETE unlinks the row and leaves the
|
||||||
|
// bytes readable in the free pages, so this fails without the VACUUM
|
||||||
|
// in purgeTargetRows.
|
||||||
|
func TestOpenPurgeRemovesCredentialBytes(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
dataDir := eventDBDataDir(t)
|
||||||
|
webhookID := uuid.New().String()
|
||||||
|
credential := "T00000000/B00000000/" + uuid.New().String()
|
||||||
|
|
||||||
|
initial := database.NewTestWebhookDBManager(dataDir)
|
||||||
|
|
||||||
|
_, err := initial.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.NoError(t, initial.CloseAll())
|
||||||
|
|
||||||
|
seed := openRawEventDB(t, dataDir, webhookID)
|
||||||
|
|
||||||
|
_, err = seed.ExecContext(
|
||||||
|
t.Context(),
|
||||||
|
"INSERT INTO targets "+
|
||||||
|
"(id, webhook_id, name, type, config) "+
|
||||||
|
"VALUES (?, '', ?, ?, ?)",
|
||||||
|
uuid.New().String(),
|
||||||
|
"leaked-target",
|
||||||
|
"slack",
|
||||||
|
fmt.Sprintf(
|
||||||
|
`{"webhookUrl":"https://hooks.example/%s"}`, credential,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
clearEventDBSweptMarker(t, seed)
|
||||||
|
require.NoError(t, seed.Close())
|
||||||
|
|
||||||
|
// The seed has to be in the file for its absence later to mean
|
||||||
|
// anything.
|
||||||
|
require.True(
|
||||||
|
t,
|
||||||
|
bytes.Contains(
|
||||||
|
eventDBFileBytes(t, dataDir, webhookID),
|
||||||
|
[]byte(credential),
|
||||||
|
),
|
||||||
|
"seeded credential is not in the file, so this test proves nothing",
|
||||||
|
)
|
||||||
|
|
||||||
|
mgr := database.NewTestWebhookDBManager(dataDir)
|
||||||
|
|
||||||
|
_, err = mgr.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.NoError(t, mgr.CloseAll())
|
||||||
|
|
||||||
|
assert.NotContains(
|
||||||
|
t,
|
||||||
|
string(eventDBFileBytes(t, dataDir, webhookID)),
|
||||||
|
credential,
|
||||||
|
"leaked credential is still recoverable from the raw file",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestOpenRevacuumsAfterIncompleteSweep covers the case a row count
|
||||||
|
// cannot see: the rows are already deleted but the file was never
|
||||||
|
// vacuumed, because an earlier sweep died between the two or its
|
||||||
|
// VACUUM failed. The credential bytes are still recoverable, and the
|
||||||
|
// unset marker is the only thing that says so, so the next open must
|
||||||
|
// vacuum rather than conclude from the empty table that there is
|
||||||
|
// nothing to do.
|
||||||
|
func TestOpenRevacuumsAfterIncompleteSweep(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
dataDir := eventDBDataDir(t)
|
||||||
|
webhookID := uuid.New().String()
|
||||||
|
credential := "T00000000/B00000000/" + uuid.New().String()
|
||||||
|
|
||||||
|
initial := database.NewTestWebhookDBManager(dataDir)
|
||||||
|
|
||||||
|
_, err := initial.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.NoError(t, initial.CloseAll())
|
||||||
|
|
||||||
|
seed := openRawEventDB(t, dataDir, webhookID)
|
||||||
|
|
||||||
|
_, err = seed.ExecContext(
|
||||||
|
t.Context(),
|
||||||
|
"INSERT INTO targets "+
|
||||||
|
"(id, webhook_id, name, type, config) "+
|
||||||
|
"VALUES (?, '', ?, ?, ?)",
|
||||||
|
uuid.New().String(),
|
||||||
|
"leaked-target",
|
||||||
|
"slack",
|
||||||
|
fmt.Sprintf(
|
||||||
|
`{"webhookUrl":"https://hooks.example/%s"}`, credential,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
// Exactly the state an interrupted sweep leaves: rows gone,
|
||||||
|
// marker unset, bytes still in the free pages.
|
||||||
|
_, err = seed.ExecContext(t.Context(), "DELETE FROM targets")
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Zero(t, countTargetRows(t, seed))
|
||||||
|
clearEventDBSweptMarker(t, seed)
|
||||||
|
require.NoError(t, seed.Close())
|
||||||
|
|
||||||
|
require.True(
|
||||||
|
t,
|
||||||
|
bytes.Contains(
|
||||||
|
eventDBFileBytes(t, dataDir, webhookID),
|
||||||
|
[]byte(credential),
|
||||||
|
),
|
||||||
|
"the deleted row's bytes must still be in the file, or this "+
|
||||||
|
"test proves nothing",
|
||||||
|
)
|
||||||
|
|
||||||
|
mgr := database.NewTestWebhookDBManager(dataDir)
|
||||||
|
|
||||||
|
_, err = mgr.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.NoError(t, mgr.CloseAll())
|
||||||
|
|
||||||
|
assert.NotContains(
|
||||||
|
t,
|
||||||
|
string(eventDBFileBytes(t, dataDir, webhookID)),
|
||||||
|
credential,
|
||||||
|
"an interrupted sweep was not retried, so the credential is "+
|
||||||
|
"still recoverable from the raw file",
|
||||||
|
)
|
||||||
|
|
||||||
|
check := openRawEventDB(t, dataDir, webhookID)
|
||||||
|
assert.Equal(t, 1, eventDBUserVersion(t, check))
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestOpenSkipsSweptDatabase covers the other half of the marker: a
|
||||||
|
// file this build created is marked without ever being vacuumed, and
|
||||||
|
// a marked file is not swept again.
|
||||||
|
func TestOpenSkipsSweptDatabase(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
dataDir := eventDBDataDir(t)
|
||||||
|
webhookID := uuid.New().String()
|
||||||
|
|
||||||
|
mgr := database.NewTestWebhookDBManager(dataDir)
|
||||||
|
|
||||||
|
_, err := mgr.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.NoError(t, mgr.CloseAll())
|
||||||
|
|
||||||
|
marked := openRawEventDB(t, dataDir, webhookID)
|
||||||
|
assert.Equal(t, 1, eventDBUserVersion(t, marked))
|
||||||
|
|
||||||
|
// A marked file is left alone, so a row written into it survives
|
||||||
|
// a reopen. Nothing writes target rows any more; this stands in
|
||||||
|
// for the sweep having run.
|
||||||
|
_, err = marked.ExecContext(
|
||||||
|
t.Context(),
|
||||||
|
"INSERT INTO targets "+
|
||||||
|
"(id, webhook_id, name, type, config) "+
|
||||||
|
"VALUES (?, '', ?, ?, ?)",
|
||||||
|
uuid.New().String(), "sentinel", "slack", `{}`,
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.NoError(t, marked.Close())
|
||||||
|
|
||||||
|
again := database.NewTestWebhookDBManager(dataDir)
|
||||||
|
|
||||||
|
_, err = again.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.NoError(t, again.CloseAll())
|
||||||
|
|
||||||
|
check := openRawEventDB(t, dataDir, webhookID)
|
||||||
|
assert.Equal(
|
||||||
|
t, 1, countTargetRows(t, check),
|
||||||
|
"a marked file must not be swept again",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestOpenSucceedsWithoutTargetsTable covers an existing event
|
||||||
|
// database that never grew a targets table. The sweep must not fail
|
||||||
|
// startup on it.
|
||||||
|
func TestOpenSucceedsWithoutTargetsTable(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
dataDir := eventDBDataDir(t)
|
||||||
|
webhookID := uuid.New().String()
|
||||||
|
|
||||||
|
seed := openRawEventDB(t, dataDir, webhookID)
|
||||||
|
|
||||||
|
_, err := seed.ExecContext(
|
||||||
|
t.Context(),
|
||||||
|
"CREATE TABLE events (id text PRIMARY KEY)",
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.NoError(t, seed.Close())
|
||||||
|
|
||||||
|
mgr := database.NewTestWebhookDBManager(dataDir)
|
||||||
|
|
||||||
|
db, err := mgr.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.NotNil(t, db)
|
||||||
|
require.NoError(t, mgr.CloseAll())
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestEventDBCreateOmitsAssociations covers the connection-level
|
||||||
|
// guard directly: a Delivery carrying its Event and Target in
|
||||||
|
// memory, written through the manager's handle, must store only the
|
||||||
|
// delivery row.
|
||||||
|
func TestEventDBCreateOmitsAssociations(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
dataDir := eventDBDataDir(t)
|
||||||
|
webhookID := uuid.New().String()
|
||||||
|
|
||||||
|
mgr := database.NewTestWebhookDBManager(dataDir)
|
||||||
|
|
||||||
|
db, err := mgr.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
target := database.Target{
|
||||||
|
WebhookID: webhookID,
|
||||||
|
Name: "leaky-target",
|
||||||
|
Type: database.TargetTypeSlack,
|
||||||
|
Config: `{"webhookUrl":"https://hooks.example/secret"}`,
|
||||||
|
}
|
||||||
|
target.ID = uuid.New().String()
|
||||||
|
|
||||||
|
event := database.Event{
|
||||||
|
WebhookID: webhookID,
|
||||||
|
EntrypointID: uuid.New().String(),
|
||||||
|
Method: "POST",
|
||||||
|
Headers: `{}`,
|
||||||
|
Body: `{}`,
|
||||||
|
}
|
||||||
|
event.ID = uuid.New().String()
|
||||||
|
|
||||||
|
d := &database.Delivery{
|
||||||
|
EventID: event.ID,
|
||||||
|
TargetID: target.ID,
|
||||||
|
Status: database.DeliveryStatusPending,
|
||||||
|
Event: event,
|
||||||
|
Target: target,
|
||||||
|
}
|
||||||
|
d.ID = uuid.New().String()
|
||||||
|
|
||||||
|
require.NoError(t, db.Create(d).Error)
|
||||||
|
require.NoError(t, db.Model(d).
|
||||||
|
Update("status", database.DeliveryStatusDelivered).
|
||||||
|
Error)
|
||||||
|
require.NoError(t, mgr.CloseAll())
|
||||||
|
|
||||||
|
check := openRawEventDB(t, dataDir, webhookID)
|
||||||
|
assert.Zero(t, countTargetRows(t, check))
|
||||||
|
}
|
||||||
@@ -2,6 +2,7 @@ package database
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"io"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"os"
|
"os"
|
||||||
"time"
|
"time"
|
||||||
@@ -66,6 +67,13 @@ func (r *RetentionReaper) ExportSetInterval(d time.Duration) {
|
|||||||
r.interval = d
|
r.interval = d
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ExportSetBannerOut redirects the first-boot credentials banner, so a
|
||||||
|
// test can read what the operator would have seen. It must be called
|
||||||
|
// before the fx start hook runs, which is where the account is seeded.
|
||||||
|
func (d *Database) ExportSetBannerOut(w io.Writer) {
|
||||||
|
d.bannerOut = w
|
||||||
|
}
|
||||||
|
|
||||||
// DummyPasswordHashForTest exposes the encoded hash that unknown
|
// DummyPasswordHashForTest exposes the encoded hash that unknown
|
||||||
// usernames are verified against.
|
// usernames are verified against.
|
||||||
func DummyPasswordHashForTest() string {
|
func DummyPasswordHashForTest() string {
|
||||||
|
|||||||
85
internal/database/migration_entrypoint_test.go
Normal file
85
internal/database/migration_entrypoint_test.go
Normal file
@@ -0,0 +1,85 @@
|
|||||||
|
package database_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/signature"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestEntrypointSignatureColumnsMigrateToUnconfigured pins the
|
||||||
|
// upgrade path for a deployment that already has entrypoints.
|
||||||
|
//
|
||||||
|
// The signature columns arrive through GORM's AutoMigrate, so every
|
||||||
|
// row written before they existed acquires them with no value. That
|
||||||
|
// has to land on "not configured", because the alternative is an
|
||||||
|
// upgrade that rejects the traffic the operator was already
|
||||||
|
// receiving — a self-inflicted outage on a receiver whose senders
|
||||||
|
// cannot be told to start signing.
|
||||||
|
//
|
||||||
|
// The legacy schema is reproduced by dropping the columns from a
|
||||||
|
// migrated database and writing a row through the old shape, so the
|
||||||
|
// row really predates them rather than merely being blank.
|
||||||
|
func TestEntrypointSignatureColumnsMigrateToUnconfigured(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
db, lc := setupTestDB(t)
|
||||||
|
lc.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(lc.RequireStop)
|
||||||
|
|
||||||
|
for _, column := range []string{
|
||||||
|
"signature_scheme", "signature_secret",
|
||||||
|
} {
|
||||||
|
require.NoError(
|
||||||
|
t,
|
||||||
|
db.DB().Exec(
|
||||||
|
"ALTER TABLE entrypoints DROP COLUMN "+column,
|
||||||
|
).Error,
|
||||||
|
"dropping %s to reproduce the pre-upgrade schema",
|
||||||
|
column,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
const legacyID = "legacy-entrypoint"
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t,
|
||||||
|
db.DB().Exec(
|
||||||
|
`INSERT INTO entrypoints
|
||||||
|
(id, created_at, updated_at, webhook_id, path,
|
||||||
|
description, active)
|
||||||
|
VALUES (?, ?, ?, ?, ?, ?, ?)`,
|
||||||
|
legacyID, "2026-01-01 00:00:00", "2026-01-01 00:00:00",
|
||||||
|
"legacy-webhook", "legacy-path", "predates signatures",
|
||||||
|
true,
|
||||||
|
).Error,
|
||||||
|
)
|
||||||
|
|
||||||
|
// The upgrade.
|
||||||
|
require.NoError(t, db.Migrate())
|
||||||
|
|
||||||
|
var ep database.Entrypoint
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t,
|
||||||
|
db.DB().Where("id = ?", legacyID).First(&ep).Error,
|
||||||
|
"the migrated row must still load; a NULL landing in a "+
|
||||||
|
"string column would fail here",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.Equal(t, database.SignatureSchemeNone, ep.SignatureScheme)
|
||||||
|
assert.Empty(t, ep.SignatureSecret)
|
||||||
|
assert.False(t, ep.SignatureConfigured())
|
||||||
|
assert.True(t, ep.Active, "the row's other columns survive")
|
||||||
|
|
||||||
|
// The behaviour that actually matters: an unsigned request to
|
||||||
|
// this entrypoint is still accepted.
|
||||||
|
assert.NoError(
|
||||||
|
t,
|
||||||
|
signature.Verify(&ep, http.Header{}, []byte(`{"a":1}`)),
|
||||||
|
)
|
||||||
|
}
|
||||||
@@ -11,6 +11,23 @@ const (
|
|||||||
DeliveryStatusRetrying DeliveryStatus = "retrying"
|
DeliveryStatusRetrying DeliveryStatus = "retrying"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// Terminal reports whether a delivery in this status has finished, so
|
||||||
|
// the delivery engine will make no further attempt of its own.
|
||||||
|
//
|
||||||
|
// It is what decides which deliveries the event log offers to replay:
|
||||||
|
// a pending or retrying delivery is still the engine's, and replaying
|
||||||
|
// one would race it.
|
||||||
|
func (s DeliveryStatus) Terminal() bool {
|
||||||
|
switch s {
|
||||||
|
case DeliveryStatusDelivered, DeliveryStatusFailed:
|
||||||
|
return true
|
||||||
|
case DeliveryStatusPending, DeliveryStatusRetrying:
|
||||||
|
return false
|
||||||
|
default:
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Delivery represents a delivery attempt for an event to a target
|
// Delivery represents a delivery attempt for an event to a target
|
||||||
type Delivery struct {
|
type Delivery struct {
|
||||||
BaseModel
|
BaseModel
|
||||||
|
|||||||
@@ -1,5 +1,22 @@
|
|||||||
package database
|
package database
|
||||||
|
|
||||||
|
// SignatureScheme names the way an entrypoint authenticates inbound
|
||||||
|
// requests. A scheme fixes both the header the signature arrives in
|
||||||
|
// and the algorithm used to check it, so an operator cannot pair one
|
||||||
|
// sender's header with another sender's comparison.
|
||||||
|
type SignatureScheme string
|
||||||
|
|
||||||
|
// Signature scheme values. The empty scheme means the entrypoint
|
||||||
|
// performs no inbound verification: it is the default, and it is the
|
||||||
|
// state every entrypoint created before this column existed migrates
|
||||||
|
// to, so an existing deployment keeps accepting the requests it
|
||||||
|
// accepted before.
|
||||||
|
const (
|
||||||
|
SignatureSchemeNone SignatureScheme = ""
|
||||||
|
SignatureSchemeGitHub SignatureScheme = "github"
|
||||||
|
SignatureSchemeGitLab SignatureScheme = "gitlab"
|
||||||
|
)
|
||||||
|
|
||||||
// Entrypoint represents an inbound URL endpoint that feeds into a webhook
|
// Entrypoint represents an inbound URL endpoint that feeds into a webhook
|
||||||
type Entrypoint struct {
|
type Entrypoint struct {
|
||||||
BaseModel
|
BaseModel
|
||||||
@@ -12,6 +29,43 @@ type Entrypoint struct {
|
|||||||
Description string `json:"description"`
|
Description string `json:"description"`
|
||||||
Active bool `gorm:"default:true" json:"active"`
|
Active bool `gorm:"default:true" json:"active"`
|
||||||
|
|
||||||
|
// SignatureScheme selects how inbound requests to this
|
||||||
|
// entrypoint are authenticated. Empty means unauthenticated,
|
||||||
|
// which is what a UUID-only entrypoint has always been.
|
||||||
|
SignatureScheme SignatureScheme `gorm:"default:''" json:"signatureScheme"`
|
||||||
|
|
||||||
|
// SignatureSecret is the secret shared with the sender.
|
||||||
|
//
|
||||||
|
// It is stored in the clear because HMAC verification needs the
|
||||||
|
// key itself: a hash of it cannot recompute the sender's digest.
|
||||||
|
// It is therefore a live credential, and json:"-" keeps it out of
|
||||||
|
// any handler that marshals the model, the way APIKey.Key and
|
||||||
|
// Target.Config are kept out. handlers.EntrypointView is the
|
||||||
|
// matching barrier for the HTML path.
|
||||||
|
SignatureSecret string `gorm:"default:''" json:"-"`
|
||||||
|
|
||||||
// Relations
|
// Relations
|
||||||
Webhook Webhook `json:"webhook,omitzero"`
|
Webhook Webhook `json:"webhook,omitzero"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// SignatureConfigured reports whether this entrypoint verifies
|
||||||
|
// inbound requests. Both halves must be present: a scheme without a
|
||||||
|
// secret, or a secret without a scheme, is a broken configuration
|
||||||
|
// rather than a configured one, and signature.Verify fails those
|
||||||
|
// closed rather than treating them as "off".
|
||||||
|
func (e *Entrypoint) SignatureConfigured() bool {
|
||||||
|
return e.SignatureScheme != SignatureSchemeNone &&
|
||||||
|
e.SignatureSecret != ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// SignatureHalfConfigured reports whether exactly one half of the
|
||||||
|
// scheme/secret pair is present. The receiver refuses such a row on
|
||||||
|
// every request, so the UI must not describe it as unverified. It
|
||||||
|
// reports the state without exposing the secret, which is why it
|
||||||
|
// lives here rather than in the display projection.
|
||||||
|
func (e *Entrypoint) SignatureHalfConfigured() bool {
|
||||||
|
hasScheme := e.SignatureScheme != SignatureSchemeNone
|
||||||
|
hasSecret := e.SignatureSecret != ""
|
||||||
|
|
||||||
|
return hasScheme != hasSecret
|
||||||
|
}
|
||||||
|
|||||||
@@ -34,6 +34,8 @@ func marshalModel(t *testing.T, v any) string {
|
|||||||
// - APIKey.Key is a bearer token outright.
|
// - APIKey.Key is a bearer token outright.
|
||||||
// - Setting.Value holds the session encryption key.
|
// - Setting.Value holds the session encryption key.
|
||||||
// - User.Password holds the Argon2 hash, and was already tagged.
|
// - User.Password holds the Argon2 hash, and was already tagged.
|
||||||
|
// - Entrypoint.SignatureSecret is the secret its senders sign with,
|
||||||
|
// stored in the clear because HMAC verification needs the key.
|
||||||
func TestModelsDoNotMarshalTheirSecrets(t *testing.T) {
|
func TestModelsDoNotMarshalTheirSecrets(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -72,6 +74,14 @@ func TestModelsDoNotMarshalTheirSecrets(t *testing.T) {
|
|||||||
Password: marker,
|
Password: marker,
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
name: "entrypoint signature secret",
|
||||||
|
model: database.Entrypoint{
|
||||||
|
Description: keptField,
|
||||||
|
SignatureScheme: database.SignatureSchemeGitHub,
|
||||||
|
SignatureSecret: marker,
|
||||||
|
},
|
||||||
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, tc := range cases {
|
for _, tc := range cases {
|
||||||
@@ -105,3 +115,24 @@ func TestWebhookMarshalsNoTargetConfig(t *testing.T) {
|
|||||||
assert.NotContains(t, encoded, marker)
|
assert.NotContains(t, encoded, marker)
|
||||||
assert.Contains(t, encoded, keptField)
|
assert.Contains(t, encoded, keptField)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestWebhookMarshalsNoEntrypointSecret covers the same nested case
|
||||||
|
// for the entrypoint's inbound signature secret, which reaches a
|
||||||
|
// marshalled webhook through the Entrypoints association.
|
||||||
|
func TestWebhookMarshalsNoEntrypointSecret(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const marker = "QQENTRYPOINTMARKERQQ"
|
||||||
|
|
||||||
|
encoded := marshalModel(t, database.Webhook{
|
||||||
|
Name: keptField,
|
||||||
|
Entrypoints: []database.Entrypoint{{
|
||||||
|
Path: "some-uuid",
|
||||||
|
SignatureScheme: database.SignatureSchemeGitLab,
|
||||||
|
SignatureSecret: marker,
|
||||||
|
}},
|
||||||
|
})
|
||||||
|
|
||||||
|
assert.NotContains(t, encoded, marker)
|
||||||
|
assert.Contains(t, encoded, keptField)
|
||||||
|
}
|
||||||
|
|||||||
@@ -24,11 +24,24 @@ func NewTestDatabase(db *gorm.DB) *Database {
|
|||||||
// NewTestWebhookDBManager creates a WebhookDBManager backed by the given
|
// NewTestWebhookDBManager creates a WebhookDBManager backed by the given
|
||||||
// data directory. Intended for use in tests without the fx lifecycle.
|
// data directory. Intended for use in tests without the fx lifecycle.
|
||||||
func NewTestWebhookDBManager(dataDir string) *WebhookDBManager {
|
func NewTestWebhookDBManager(dataDir string) *WebhookDBManager {
|
||||||
return &WebhookDBManager{
|
return NewTestWebhookDBManagerWithLogger(
|
||||||
dataDir: dataDir,
|
dataDir,
|
||||||
log: slog.New(slog.NewTextHandler(
|
slog.New(slog.NewTextHandler(
|
||||||
os.Stderr,
|
os.Stderr,
|
||||||
&slog.HandlerOptions{Level: slog.LevelDebug},
|
&slog.HandlerOptions{Level: slog.LevelDebug},
|
||||||
)),
|
)),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewTestWebhookDBManagerWithLogger is NewTestWebhookDBManager with the
|
||||||
|
// logger supplied by the caller. The per-webhook databases this manager
|
||||||
|
// opens hand that logger to gormlog, so a test that needs to see the SQL
|
||||||
|
// the service emits can capture it.
|
||||||
|
func NewTestWebhookDBManagerWithLogger(
|
||||||
|
dataDir string, log *slog.Logger,
|
||||||
|
) *WebhookDBManager {
|
||||||
|
return &WebhookDBManager{
|
||||||
|
dataDir: dataDir,
|
||||||
|
log: log,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -262,6 +262,25 @@ func (m *WebhookDBManager) openDB(
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Keep main-database rows out of this file. See
|
||||||
|
// event_db_isolation.go.
|
||||||
|
err = omitAssociations(db)
|
||||||
|
if err != nil {
|
||||||
|
_ = sqlDB.Close()
|
||||||
|
|
||||||
|
return nil, fmt.Errorf(
|
||||||
|
"guarding webhook database %s: %w",
|
||||||
|
webhookID, err,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
err = purgeTargetRows(db, m.log, webhookID)
|
||||||
|
if err != nil {
|
||||||
|
_ = sqlDB.Close()
|
||||||
|
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
// Run migrations for event-tier models only
|
// Run migrations for event-tier models only
|
||||||
err = db.AutoMigrate(
|
err = db.AutoMigrate(
|
||||||
&Event{}, &Delivery{}, &DeliveryResult{},
|
&Event{}, &Delivery{}, &DeliveryResult{},
|
||||||
|
|||||||
157
internal/delivery/event_db_isolation_test.go
Normal file
157
internal/delivery/event_db_isolation_test.go
Normal file
@@ -0,0 +1,157 @@
|
|||||||
|
package delivery_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/google/uuid"
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
_ "modernc.org/sqlite"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
)
|
||||||
|
|
||||||
|
// assertNoTargetRows opens the per-webhook database file directly,
|
||||||
|
// outside GORM, and fails if its targets table holds any rows.
|
||||||
|
// Target config is the credential for slack and http targets, and
|
||||||
|
// event databases are the files that get backed up and handed
|
||||||
|
// around.
|
||||||
|
func assertNoTargetRows(t *testing.T, dbPath string) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
sqlDB, err := sql.Open(
|
||||||
|
"sqlite", fmt.Sprintf("file:%s?mode=ro", dbPath),
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
defer func() { _ = sqlDB.Close() }()
|
||||||
|
|
||||||
|
var tables int
|
||||||
|
|
||||||
|
require.NoError(t, sqlDB.QueryRowContext(
|
||||||
|
t.Context(),
|
||||||
|
"SELECT count(*) FROM sqlite_master "+
|
||||||
|
"WHERE type = 'table' AND name = 'targets'",
|
||||||
|
).Scan(&tables))
|
||||||
|
|
||||||
|
if tables == 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
var rows int
|
||||||
|
|
||||||
|
require.NoError(t, sqlDB.QueryRowContext(
|
||||||
|
t.Context(), "SELECT count(*) FROM targets",
|
||||||
|
).Scan(&rows))
|
||||||
|
|
||||||
|
assert.Zero(
|
||||||
|
t, rows,
|
||||||
|
"per-webhook event database must hold no target rows",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestEventDBHoldsNoTargetRows drives a delivery and then a retry
|
||||||
|
// through the real engine write paths and asserts neither leaves a
|
||||||
|
// target row behind in events-*.db.
|
||||||
|
func TestEventDBHoldsNoTargetRows(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s := newISetup(t)
|
||||||
|
|
||||||
|
ts := httptest.NewServer(http.HandlerFunc(
|
||||||
|
func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
},
|
||||||
|
))
|
||||||
|
defer ts.Close()
|
||||||
|
|
||||||
|
cfg := iHTTPConfig(ts.URL)
|
||||||
|
targetID := uuid.New().String()
|
||||||
|
dbPath := s.DBMgr.DBPath(s.WebhookID)
|
||||||
|
|
||||||
|
event := iSeedEvent(
|
||||||
|
t, s.WebhookDB, s.WebhookID, `{"leak":"none"}`,
|
||||||
|
)
|
||||||
|
body := event.Body
|
||||||
|
|
||||||
|
// A new delivery.
|
||||||
|
d := iSeedDelivery(
|
||||||
|
t, s.WebhookDB, event.ID, targetID,
|
||||||
|
database.DeliveryStatusPending,
|
||||||
|
)
|
||||||
|
task := iTask(
|
||||||
|
d, event, s.WebhookID, targetID,
|
||||||
|
"leaky-target", cfg, 5, 1, &body,
|
||||||
|
)
|
||||||
|
|
||||||
|
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
||||||
|
|
||||||
|
iAssertStatus(
|
||||||
|
t, s.WebhookDB, d.ID,
|
||||||
|
database.DeliveryStatusDelivered,
|
||||||
|
)
|
||||||
|
assertNoTargetRows(t, dbPath)
|
||||||
|
|
||||||
|
// A retry.
|
||||||
|
rd := iSeedDelivery(
|
||||||
|
t, s.WebhookDB, event.ID, targetID,
|
||||||
|
database.DeliveryStatusRetrying,
|
||||||
|
)
|
||||||
|
rTask := iTask(
|
||||||
|
rd, event, s.WebhookID, targetID,
|
||||||
|
"leaky-target", cfg, 5, 2, &body,
|
||||||
|
)
|
||||||
|
|
||||||
|
s.Engine.ExportProcessRetryTask(context.TODO(), &rTask)
|
||||||
|
|
||||||
|
iAssertStatus(
|
||||||
|
t, s.WebhookDB, rd.ID,
|
||||||
|
database.DeliveryStatusDelivered,
|
||||||
|
)
|
||||||
|
assertNoTargetRows(t, dbPath)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestEventDBHoldsNoTargetRowsOnFailedDelivery covers the failure
|
||||||
|
// write path, which updates the delivery to failed and records a
|
||||||
|
// result, rather than the success path above.
|
||||||
|
func TestEventDBHoldsNoTargetRowsOnFailedDelivery(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s := newISetup(t)
|
||||||
|
|
||||||
|
ts := httptest.NewServer(http.HandlerFunc(
|
||||||
|
func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
w.WriteHeader(http.StatusInternalServerError)
|
||||||
|
},
|
||||||
|
))
|
||||||
|
defer ts.Close()
|
||||||
|
|
||||||
|
cfg := iHTTPConfig(ts.URL)
|
||||||
|
targetID := uuid.New().String()
|
||||||
|
|
||||||
|
event := iSeedEvent(
|
||||||
|
t, s.WebhookDB, s.WebhookID, `{"leak":"none"}`,
|
||||||
|
)
|
||||||
|
body := event.Body
|
||||||
|
|
||||||
|
d := iSeedDelivery(
|
||||||
|
t, s.WebhookDB, event.ID, targetID,
|
||||||
|
database.DeliveryStatusPending,
|
||||||
|
)
|
||||||
|
task := iTask(
|
||||||
|
d, event, s.WebhookID, targetID,
|
||||||
|
"leaky-target", cfg, 0, 1, &body,
|
||||||
|
)
|
||||||
|
|
||||||
|
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
||||||
|
|
||||||
|
iAssertStatus(
|
||||||
|
t, s.WebhookDB, d.ID,
|
||||||
|
database.DeliveryStatusFailed,
|
||||||
|
)
|
||||||
|
assertNoTargetRows(t, s.DBMgr.DBPath(s.WebhookID))
|
||||||
|
}
|
||||||
@@ -63,6 +63,16 @@ func ExportIsForwardableHeader(name string) bool {
|
|||||||
return isForwardableHeader(name)
|
return isForwardableHeader(name)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ExportApplyRequestHeaders exposes applyRequestHeaders, so a test
|
||||||
|
// can inspect the header set an outbound delivery actually carries.
|
||||||
|
func ExportApplyRequestHeaders(
|
||||||
|
req *http.Request,
|
||||||
|
event *database.Event,
|
||||||
|
cfg *HTTPTargetConfig,
|
||||||
|
) {
|
||||||
|
applyRequestHeaders(req, event, cfg)
|
||||||
|
}
|
||||||
|
|
||||||
// ExportTruncate exposes truncate for testing.
|
// ExportTruncate exposes truncate for testing.
|
||||||
func ExportTruncate(s string, maxLen int) string {
|
func ExportTruncate(s string, maxLen int) string {
|
||||||
return truncate(s, maxLen)
|
return truncate(s, maxLen)
|
||||||
|
|||||||
@@ -95,6 +95,8 @@ func (e *Engine) sampleQueueDepths(ctx context.Context) {
|
|||||||
// targetTypesByID maps every configured target id to its type. The
|
// targetTypesByID maps every configured target id to its type. The
|
||||||
// deliveries live in the per-webhook databases but carry only a
|
// deliveries live in the per-webhook databases but carry only a
|
||||||
// target id, so the type label has to come from the main database.
|
// target id, so the type label has to come from the main database.
|
||||||
|
//
|
||||||
|
// Find rather than Scan: see sampleWebhookQueueDepths.
|
||||||
func (e *Engine) targetTypesByID() (
|
func (e *Engine) targetTypesByID() (
|
||||||
map[string]database.TargetType, error,
|
map[string]database.TargetType, error,
|
||||||
) {
|
) {
|
||||||
@@ -106,7 +108,7 @@ func (e *Engine) targetTypesByID() (
|
|||||||
err := e.database.DB().
|
err := e.database.DB().
|
||||||
Model(&database.Target{}).
|
Model(&database.Target{}).
|
||||||
Select("id", "type").
|
Select("id", "type").
|
||||||
Scan(&rows).Error
|
Find(&rows).Error
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("loading targets: %w", err)
|
return nil, fmt.Errorf("loading targets: %w", err)
|
||||||
}
|
}
|
||||||
@@ -128,6 +130,13 @@ func (e *Engine) targetTypesByID() (
|
|||||||
// folds that into the unknown series rather than dropping it: a
|
// folds that into the unknown series rather than dropping it: a
|
||||||
// backlog stuck behind a deleted target is a backlog that still needs
|
// backlog stuck behind a deleted target is a backlog that still needs
|
||||||
// to be alertable.
|
// to be alertable.
|
||||||
|
//
|
||||||
|
// The aggregate is read with Find, not Scan. (*gorm.DB).Scan swaps
|
||||||
|
// GORM's own trace recorder in for the logging adapter, and that
|
||||||
|
// recorder does not implement gorm.ParamsFilter, so the statement
|
||||||
|
// reaches the log with its bound values interpolated — here, the
|
||||||
|
// status list. Find goes through the normal query callback, which is
|
||||||
|
// filtered. See internal/gormlog and its scan_guard_test.go.
|
||||||
func (e *Engine) sampleWebhookQueueDepths(
|
func (e *Engine) sampleWebhookQueueDepths(
|
||||||
webhookID string,
|
webhookID string,
|
||||||
types map[string]database.TargetType,
|
types map[string]database.TargetType,
|
||||||
@@ -158,7 +167,7 @@ func (e *Engine) sampleWebhookQueueDepths(
|
|||||||
database.DeliveryStatusRetrying,
|
database.DeliveryStatusRetrying,
|
||||||
}).
|
}).
|
||||||
Group("target_id, status").
|
Group("target_id, status").
|
||||||
Scan(&rows).Error
|
Find(&rows).Error
|
||||||
if err != nil {
|
if err != nil {
|
||||||
e.log.Error(
|
e.log.Error(
|
||||||
"queue depth sample: "+
|
"queue depth sample: "+
|
||||||
|
|||||||
179
internal/delivery/queue_depth_gormlog_test.go
Normal file
179
internal/delivery/queue_depth_gormlog_test.go
Normal file
@@ -0,0 +1,179 @@
|
|||||||
|
package delivery_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
"net/http"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/google/uuid"
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"gorm.io/driver/sqlite"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
|
"sneak.berlin/go/webhooker/internal/gormlog"
|
||||||
|
)
|
||||||
|
|
||||||
|
// qdAggregateMarker identifies the queue-depth aggregate in the
|
||||||
|
// captured SQL. It is the one statement in this test that binds
|
||||||
|
// anything, and the raw count() expression appears in no other.
|
||||||
|
const qdAggregateMarker = "count(*)"
|
||||||
|
|
||||||
|
// qdSyncBuf collects log output from whichever goroutine GORM writes
|
||||||
|
// on.
|
||||||
|
type qdSyncBuf struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
b bytes.Buffer
|
||||||
|
}
|
||||||
|
|
||||||
|
func (q *qdSyncBuf) Write(p []byte) (int, error) {
|
||||||
|
q.mu.Lock()
|
||||||
|
defer q.mu.Unlock()
|
||||||
|
|
||||||
|
return q.b.Write(p)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (q *qdSyncBuf) String() string {
|
||||||
|
q.mu.Lock()
|
||||||
|
defer q.mu.Unlock()
|
||||||
|
|
||||||
|
return q.b.String()
|
||||||
|
}
|
||||||
|
|
||||||
|
// qdMainDB opens a main database whose GORM logger is the service's
|
||||||
|
// adapter, writing through log.
|
||||||
|
func qdMainDB(t *testing.T, log *slog.Logger) *gorm.DB {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
dsn := fmt.Sprintf(
|
||||||
|
"file:%s?cache=shared&mode=rwc",
|
||||||
|
filepath.Join(t.TempDir(), "main-gormlog.db"),
|
||||||
|
)
|
||||||
|
|
||||||
|
sqlDB, err := sql.Open("sqlite", dsn)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
t.Cleanup(func() { _ = sqlDB.Close() })
|
||||||
|
|
||||||
|
db, err := gorm.Open(
|
||||||
|
sqlite.Dialector{Conn: sqlDB},
|
||||||
|
&gorm.Config{Logger: gormlog.New(log)},
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
require.NoError(t, db.AutoMigrate(
|
||||||
|
&database.Webhook{},
|
||||||
|
&database.Target{},
|
||||||
|
))
|
||||||
|
|
||||||
|
return db
|
||||||
|
}
|
||||||
|
|
||||||
|
// qdLinesContaining returns every captured line carrying marker.
|
||||||
|
func qdLinesContaining(out, marker string) []string {
|
||||||
|
var found []string
|
||||||
|
|
||||||
|
for line := range strings.SplitSeq(out, "\n") {
|
||||||
|
if strings.Contains(line, marker) {
|
||||||
|
found = append(found, line)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return found
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestQueueDepthSample_LogsNoBoundValue holds the queue-depth sampler
|
||||||
|
// to the values-off property internal/gormlog exists to provide.
|
||||||
|
//
|
||||||
|
// The aggregate binds the delivery status list. Read with
|
||||||
|
// (*gorm.DB).Scan it was logged with those values interpolated, because
|
||||||
|
// Scan records the statement through GORM's own traceRecorder, which
|
||||||
|
// does not implement gorm.ParamsFilter. Read with Find it goes through
|
||||||
|
// the normal query callback and the adapter's filter applies. Restore
|
||||||
|
// the Scan call in queue_depth.go and this fails on the status literals
|
||||||
|
// below; scan_guard_test.go catches the same regression statically.
|
||||||
|
func TestQueueDepthSample_LogsNoBoundValue(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
buf := &qdSyncBuf{}
|
||||||
|
log := slog.New(slog.NewTextHandler(
|
||||||
|
buf, &slog.HandlerOptions{Level: slog.LevelDebug},
|
||||||
|
))
|
||||||
|
|
||||||
|
mainDB := qdMainDB(t, log)
|
||||||
|
dbMgr := database.NewTestWebhookDBManagerWithLogger(
|
||||||
|
t.TempDir(), log,
|
||||||
|
)
|
||||||
|
|
||||||
|
webhookID := uuid.New().String()
|
||||||
|
webhookDB := iSeedWebhookDB(t, dbMgr, webhookID)
|
||||||
|
|
||||||
|
iCreateWebhook(t, mainDB, webhookID, "queue-depth-gormlog")
|
||||||
|
|
||||||
|
targetID := uuid.New().String()
|
||||||
|
|
||||||
|
iCreateTarget(t, mainDB, targetID, webhookID,
|
||||||
|
"queue-depth-gormlog-target", database.TargetTypeHTTP,
|
||||||
|
iHTTPConfig("https://example.com/hook"), 3,
|
||||||
|
)
|
||||||
|
|
||||||
|
event := iSeedEvent(
|
||||||
|
t, webhookDB, webhookID, `{"queued":true}`,
|
||||||
|
)
|
||||||
|
|
||||||
|
iSeedDelivery(
|
||||||
|
t, webhookDB, event.ID, targetID,
|
||||||
|
database.DeliveryStatusPending,
|
||||||
|
)
|
||||||
|
iSeedDelivery(
|
||||||
|
t, webhookDB, event.ID, targetID,
|
||||||
|
database.DeliveryStatusRetrying,
|
||||||
|
)
|
||||||
|
|
||||||
|
engine := delivery.NewTestEngineWithDB(
|
||||||
|
database.NewTestDatabase(mainDB),
|
||||||
|
dbMgr,
|
||||||
|
log,
|
||||||
|
&http.Client{Timeout: 5 * time.Second},
|
||||||
|
2,
|
||||||
|
)
|
||||||
|
|
||||||
|
engine.ExportSampleQueueDepths(context.Background())
|
||||||
|
|
||||||
|
out := buf.String()
|
||||||
|
|
||||||
|
lines := qdLinesContaining(out, qdAggregateMarker)
|
||||||
|
require.NotEmpty(
|
||||||
|
t, lines,
|
||||||
|
"the queue-depth aggregate was never logged, so the "+
|
||||||
|
"assertions below are vacuous",
|
||||||
|
)
|
||||||
|
|
||||||
|
for _, line := range lines {
|
||||||
|
assert.Contains(
|
||||||
|
t, line, "?",
|
||||||
|
"the aggregate was logged without its placeholders: %s",
|
||||||
|
line,
|
||||||
|
)
|
||||||
|
|
||||||
|
for _, status := range []database.DeliveryStatus{
|
||||||
|
database.DeliveryStatusPending,
|
||||||
|
database.DeliveryStatusRetrying,
|
||||||
|
} {
|
||||||
|
assert.NotContains(
|
||||||
|
t, line, string(status),
|
||||||
|
"a bound status value was interpolated into the "+
|
||||||
|
"logged statement: %s", line,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -48,16 +48,42 @@ var blockedNetworks []*net.IPNet
|
|||||||
// alwaysBlockedNetworks are the ranges no configuration can
|
// alwaysBlockedNetworks are the ranges no configuration can
|
||||||
// open: the link-local blocks and the cloud instance metadata
|
// open: the link-local blocks and the cloud instance metadata
|
||||||
// endpoints that live outside them. Reaching one is credential
|
// endpoints that live outside them. Reaching one is credential
|
||||||
// theft rather than delivery to an internal service, so a
|
// or user-data theft rather than delivery to an internal
|
||||||
// supplied CIDR that covers such an address still leaves it
|
// service, so a supplied CIDR that covers such an address still
|
||||||
// blocked.
|
// leaves it blocked.
|
||||||
|
//
|
||||||
|
// Inclusion criterion — an address belongs here only if BOTH
|
||||||
|
// hold, and every entry below satisfies both:
|
||||||
|
//
|
||||||
|
// 1. It is a fixed address assigned by the provider, or a
|
||||||
|
// range reserved by IANA — never one the operator chose.
|
||||||
|
// That is what makes a host route free: it cannot collide
|
||||||
|
// with anything the operator runs.
|
||||||
|
// 2. Reaching it discloses credentials, or user data or
|
||||||
|
// bootstrap material — something granting onward access, or
|
||||||
|
// not cheaply rotated.
|
||||||
|
//
|
||||||
|
// Both halves are load-bearing, so use them to refuse a
|
||||||
|
// candidate and say why. An endpoint disclosing only the
|
||||||
|
// operator's own inventory (instance id, region, disks, NICs)
|
||||||
|
// fails (2): letting a delivery target reach the operator's own
|
||||||
|
// infrastructure is the feature ALLOWED_EGRESS_CIDRS exists to
|
||||||
|
// provide. But (2) is not "IAM credentials only" either —
|
||||||
|
// fd00:42::42 serves /user_data and /conf rather than tokens,
|
||||||
|
// and user data routinely carries bootstrap secrets. An address
|
||||||
|
// stays out if it fails (1) however well it clears (2): a host
|
||||||
|
// route inside a block operators really assign from, such as
|
||||||
|
// 10.0.0.0/8, can collide with a real internal service and
|
||||||
|
// forfeits the justification in (1).
|
||||||
|
//
|
||||||
|
// This is a criterion, not an enumeration of every metadata
|
||||||
|
// address in existence.
|
||||||
//
|
//
|
||||||
// Some of these are also in blockedNetworks and this list is
|
// Some of these are also in blockedNetworks and this list is
|
||||||
// what makes them unconditional; two are public unicast and are
|
// what makes them unconditional; two are public unicast and are
|
||||||
// reachable by default without it. Every metadata endpoint
|
// reachable by default without it. Every entry outside the
|
||||||
// outside the link-local range is a /32 or /128 host route, so
|
// link-local blocks is a /32 or /128 host route, so blocking it
|
||||||
// blocking it costs an operator nothing else on the surrounding
|
// costs an operator nothing else on the surrounding network.
|
||||||
// network.
|
|
||||||
//
|
//
|
||||||
// Derive membership from the address, never from the vendor's
|
// Derive membership from the address, never from the vendor's
|
||||||
// prose. Several providers call these endpoints "link-local" or
|
// prose. Several providers call these endpoints "link-local" or
|
||||||
@@ -101,8 +127,9 @@ func init() {
|
|||||||
// which is why those are listed separately.
|
// which is why those are listed separately.
|
||||||
alwaysBlockedNetworks = mustParseCIDRs([]string{
|
alwaysBlockedNetworks = mustParseCIDRs([]string{
|
||||||
// IPv4 link-local, carrying the 169.254.169.254
|
// IPv4 link-local, carrying the 169.254.169.254
|
||||||
// metadata service used by AWS, Azure, Alibaba,
|
// metadata service used by AWS, Azure, DigitalOcean,
|
||||||
// DigitalOcean, Hetzner, OpenStack and others.
|
// Hetzner, OpenStack and others. Not Alibaba, which uses
|
||||||
|
// 100.100.100.200 below exclusively.
|
||||||
"169.254.0.0/16",
|
"169.254.0.0/16",
|
||||||
// IPv6 link-local, its IPv6 counterpart.
|
// IPv6 link-local, its IPv6 counterpart.
|
||||||
"fe80::/10",
|
"fe80::/10",
|
||||||
|
|||||||
@@ -470,7 +470,7 @@ func TestAlwaysBlockedNetworks_PinnedSet(t *testing.T) {
|
|||||||
|
|
||||||
want := []string{
|
want := []string{
|
||||||
// IPv4 link-local: the 169.254.169.254 metadata
|
// IPv4 link-local: the 169.254.169.254 metadata
|
||||||
// service on AWS, Azure, Alibaba and others.
|
// service on AWS, Azure and others.
|
||||||
"169.254.0.0/16",
|
"169.254.0.0/16",
|
||||||
// IPv6 link-local.
|
// IPv6 link-local.
|
||||||
"fe80::/10",
|
"fe80::/10",
|
||||||
|
|||||||
142
internal/delivery/target_http_secret_test.go
Normal file
142
internal/delivery/target_http_secret_test.go
Normal file
@@ -0,0 +1,142 @@
|
|||||||
|
package delivery_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"net/http"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
|
"sneak.berlin/go/webhooker/internal/signature"
|
||||||
|
)
|
||||||
|
|
||||||
|
// gitlabDeliverySecret is the shared secret the entrypoint in these
|
||||||
|
// tests is configured with. No outbound request may contain it.
|
||||||
|
const gitlabDeliverySecret = "QQDELIVERYSECRETQQ"
|
||||||
|
|
||||||
|
// receivedEventHeaders builds the Event.Headers value the receiver
|
||||||
|
// stores for an inbound request, by running the request's headers
|
||||||
|
// through the same sanitizer the receive path uses. Going through
|
||||||
|
// signature.SanitizeHeaders rather than a literal is the point of
|
||||||
|
// the test: it joins the two egresses at the field they share, so a
|
||||||
|
// regression at either end shows up here.
|
||||||
|
func receivedEventHeaders(
|
||||||
|
t *testing.T,
|
||||||
|
scheme database.SignatureScheme,
|
||||||
|
inbound http.Header,
|
||||||
|
) string {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
ep := &database.Entrypoint{
|
||||||
|
SignatureScheme: scheme,
|
||||||
|
SignatureSecret: gitlabDeliverySecret,
|
||||||
|
}
|
||||||
|
|
||||||
|
encoded, err := json.Marshal(
|
||||||
|
signature.SanitizeHeaders(ep, inbound),
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
return string(encoded)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestApplyRequestHeadersDropsInboundCredential proves a delivery to
|
||||||
|
// an HTTP target does not carry the GitLab shared secret.
|
||||||
|
//
|
||||||
|
// isForwardableHeader is a blocklist of hop-by-hop names, so it
|
||||||
|
// forwards X-Gitlab-Token like any other header; what keeps the
|
||||||
|
// secret out of the outbound request is that the receiver never
|
||||||
|
// stored it. Handing a target operator the token would hand them the
|
||||||
|
// ability to forge requests to the entrypoint it authenticates,
|
||||||
|
// which is the one control the receiver has.
|
||||||
|
func TestApplyRequestHeadersDropsInboundCredential(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
inbound := http.Header{}
|
||||||
|
inbound.Set(signature.HeaderGitLab, gitlabDeliverySecret)
|
||||||
|
inbound.Set("X-Gitlab-Event", "Push Hook")
|
||||||
|
|
||||||
|
event := &database.Event{
|
||||||
|
Headers: receivedEventHeaders(
|
||||||
|
t, database.SignatureSchemeGitLab, inbound,
|
||||||
|
),
|
||||||
|
ContentType: "application/json",
|
||||||
|
}
|
||||||
|
|
||||||
|
req, err := http.NewRequestWithContext(
|
||||||
|
context.Background(),
|
||||||
|
http.MethodPost,
|
||||||
|
"https://target.example.com/hook",
|
||||||
|
http.NoBody,
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
delivery.ExportApplyRequestHeaders(
|
||||||
|
req, event, &delivery.HTTPTargetConfig{},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.Empty(
|
||||||
|
t,
|
||||||
|
req.Header.Values(signature.HeaderGitLab),
|
||||||
|
"the shared secret header must not reach a target",
|
||||||
|
)
|
||||||
|
|
||||||
|
// Header.Values canonicalises, so a differently-cased spelling
|
||||||
|
// would be caught above; this catches the value arriving under
|
||||||
|
// some other name.
|
||||||
|
for name, values := range req.Header {
|
||||||
|
for _, v := range values {
|
||||||
|
assert.NotContains(
|
||||||
|
t, v, gitlabDeliverySecret,
|
||||||
|
"secret present in outbound header %s", name,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The rest of the sender's headers still arrive. A fix that
|
||||||
|
// dropped everything would pass the assertions above while
|
||||||
|
// breaking delivery.
|
||||||
|
assert.Equal(
|
||||||
|
t,
|
||||||
|
"Push Hook",
|
||||||
|
req.Header.Get("X-Gitlab-Event"),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestApplyRequestHeadersKeepsGitHubDigest proves the stripping is
|
||||||
|
// scoped to headers that carry the secret itself. GitHub's
|
||||||
|
// X-Hub-Signature-256 is an HMAC over the body, so a target can be
|
||||||
|
// shown it without being handed the key.
|
||||||
|
func TestApplyRequestHeadersKeepsGitHubDigest(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const digest = "sha256=deadbeef"
|
||||||
|
|
||||||
|
inbound := http.Header{}
|
||||||
|
inbound.Set(signature.HeaderGitHub, digest)
|
||||||
|
|
||||||
|
event := &database.Event{
|
||||||
|
Headers: receivedEventHeaders(
|
||||||
|
t, database.SignatureSchemeGitHub, inbound,
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
req, err := http.NewRequestWithContext(
|
||||||
|
context.Background(),
|
||||||
|
http.MethodPost,
|
||||||
|
"https://target.example.com/hook",
|
||||||
|
http.NoBody,
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
delivery.ExportApplyRequestHeaders(
|
||||||
|
req, event, &delivery.HTTPTargetConfig{},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.Equal(
|
||||||
|
t, digest, req.Header.Get(signature.HeaderGitHub),
|
||||||
|
)
|
||||||
|
}
|
||||||
378
internal/handlers/delivery_replay.go
Normal file
378
internal/handlers/delivery_replay.go
Normal file
@@ -0,0 +1,378 @@
|
|||||||
|
package handlers
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"strconv"
|
||||||
|
|
||||||
|
"github.com/go-chi/chi"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
|
)
|
||||||
|
|
||||||
|
// replayOutcomeParam is the query parameter the replay POST redirects
|
||||||
|
// with and the event log page reads its banner from.
|
||||||
|
const replayOutcomeParam = "replay"
|
||||||
|
|
||||||
|
// replayOutcomeCode is the outcome of a replay POST. The redirect
|
||||||
|
// carries one of these fixed codes rather than a message, so nothing a
|
||||||
|
// client submits can reach the rendered page through it.
|
||||||
|
type replayOutcomeCode string
|
||||||
|
|
||||||
|
const (
|
||||||
|
// replayQueued reports that a new delivery was created and handed
|
||||||
|
// to the delivery engine.
|
||||||
|
replayQueued replayOutcomeCode = "queued"
|
||||||
|
|
||||||
|
// replayTargetDeleted reports a target that once existed and has
|
||||||
|
// since been deleted. Deletes are soft and deliveries carry no
|
||||||
|
// foreign key to the target row, so the history survives its
|
||||||
|
// target and this is the ordinary case for an old event.
|
||||||
|
replayTargetDeleted replayOutcomeCode = "target-deleted"
|
||||||
|
|
||||||
|
// replayTargetMissing reports a target id that names no row at
|
||||||
|
// all, deleted or otherwise.
|
||||||
|
replayTargetMissing replayOutcomeCode = "target-missing"
|
||||||
|
|
||||||
|
// replayTargetInactive reports a target the operator has
|
||||||
|
// deactivated. A deactivated target receives no new deliveries, so
|
||||||
|
// a replay to it would be a delivery they switched off.
|
||||||
|
replayTargetInactive replayOutcomeCode = "target-inactive"
|
||||||
|
|
||||||
|
// replayNotTerminal reports a delivery the engine has not finished
|
||||||
|
// with.
|
||||||
|
replayNotTerminal replayOutcomeCode = "not-terminal"
|
||||||
|
|
||||||
|
// replayInFlight reports that an earlier replay of this event to
|
||||||
|
// this target is still running.
|
||||||
|
replayInFlight replayOutcomeCode = "in-flight"
|
||||||
|
)
|
||||||
|
|
||||||
|
// replayOutcome returns the banner the event log page shows for an
|
||||||
|
// outcome code, and whether the replay was queued. An unrecognised
|
||||||
|
// code yields no banner.
|
||||||
|
func replayOutcome(code string) (string, bool) {
|
||||||
|
switch replayOutcomeCode(code) {
|
||||||
|
case replayQueued:
|
||||||
|
return "Replay queued: a new delivery was created against " +
|
||||||
|
"the target's current configuration.", true
|
||||||
|
case replayTargetDeleted:
|
||||||
|
return "Not replayed: the target this delivery was for has " +
|
||||||
|
"been deleted. Recreate the target, then replay.", false
|
||||||
|
case replayTargetMissing:
|
||||||
|
return "Not replayed: the target this delivery was for no " +
|
||||||
|
"longer exists.", false
|
||||||
|
case replayTargetInactive:
|
||||||
|
return "Not replayed: the target this delivery was for is " +
|
||||||
|
"deactivated. Activate it, then replay.", false
|
||||||
|
case replayNotTerminal:
|
||||||
|
return "Not replayed: this delivery has not finished yet.",
|
||||||
|
false
|
||||||
|
case replayInFlight:
|
||||||
|
return "Not replayed: a delivery of this event to this " +
|
||||||
|
"target is already in flight.", false
|
||||||
|
default:
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// HandleDeliveryReplay re-sends a finished delivery's event to its
|
||||||
|
// target.
|
||||||
|
//
|
||||||
|
// A replay never touches the delivery it repeats. It creates a NEW
|
||||||
|
// pending delivery row for the same event and target and hands it to
|
||||||
|
// the delivery engine through the same Notifier the receiver uses, so
|
||||||
|
// the original's status, attempts and timestamps stand as the record
|
||||||
|
// of what actually happened, and the replay is retried, SSRF-guarded
|
||||||
|
// and circuit-broken exactly as a first attempt is.
|
||||||
|
//
|
||||||
|
// What is re-sent is the stored EVENT body, never the response the
|
||||||
|
// original delivery received.
|
||||||
|
//
|
||||||
|
// The target's configuration is read now rather than as it stood when
|
||||||
|
// the original ran: a replay exists to deliver where the operator
|
||||||
|
// currently wants the event to go. That is also why a deleted target
|
||||||
|
// is refused rather than delivered to from stale configuration.
|
||||||
|
func (h *Handlers) HandleDeliveryReplay() http.HandlerFunc {
|
||||||
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
webhook, ok := h.ownedWebhook(w, r)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// The body size cap is enforced by the MaxBodySize
|
||||||
|
// middleware, which runs before CSRF parses the form.
|
||||||
|
err := r.ParseForm()
|
||||||
|
if err != nil {
|
||||||
|
http.Error(
|
||||||
|
w, "Bad request", http.StatusBadRequest,
|
||||||
|
)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
h.replayDelivery(w, r, webhook)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// replayDelivery performs the replay for a webhook the caller has
|
||||||
|
// already established the session's user owns.
|
||||||
|
func (h *Handlers) replayDelivery(
|
||||||
|
w http.ResponseWriter,
|
||||||
|
r *http.Request,
|
||||||
|
webhook database.Webhook,
|
||||||
|
) {
|
||||||
|
if !h.dbMgr.DBExists(webhook.ID) {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
webhookDB, err := h.dbMgr.GetDB(webhook.ID)
|
||||||
|
if err != nil {
|
||||||
|
h.serverError(w, "failed to get webhook database", err)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
original, ok := h.loadReplaySource(w, r, webhookDB)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if !original.Status.Terminal() {
|
||||||
|
h.finishReplay(w, r, webhook, replayNotTerminal)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
target, code := h.replayTarget(webhook.ID, original.TargetID)
|
||||||
|
if target == nil {
|
||||||
|
h.finishReplay(w, r, webhook, code)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
h.queueReplay(w, r, webhookDB, webhook, original, target)
|
||||||
|
}
|
||||||
|
|
||||||
|
// loadReplaySource loads the delivery to be replayed, selecting only
|
||||||
|
// the columns the replay needs so no association is populated. A
|
||||||
|
// delivery id that names no row in this webhook's database is a 404.
|
||||||
|
func (h *Handlers) loadReplaySource(
|
||||||
|
w http.ResponseWriter,
|
||||||
|
r *http.Request,
|
||||||
|
webhookDB *gorm.DB,
|
||||||
|
) (*database.Delivery, bool) {
|
||||||
|
var original database.Delivery
|
||||||
|
|
||||||
|
err := webhookDB.
|
||||||
|
Select("id", "event_id", "target_id", "status").
|
||||||
|
First(
|
||||||
|
&original, "id = ?", chi.URLParam(r, "deliveryID"),
|
||||||
|
).Error
|
||||||
|
if err != nil {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
|
||||||
|
return &original, true
|
||||||
|
}
|
||||||
|
|
||||||
|
// queueReplay writes the new delivery and hands it to the engine.
|
||||||
|
func (h *Handlers) queueReplay(
|
||||||
|
w http.ResponseWriter,
|
||||||
|
r *http.Request,
|
||||||
|
webhookDB *gorm.DB,
|
||||||
|
webhook database.Webhook,
|
||||||
|
original *database.Delivery,
|
||||||
|
target *database.Target,
|
||||||
|
) {
|
||||||
|
inFlight, err := countInFlightDeliveries(
|
||||||
|
webhookDB, original.EventID, target.ID,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
h.serverError(
|
||||||
|
w, "failed to count in-flight deliveries", err,
|
||||||
|
)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if inFlight > 0 {
|
||||||
|
h.finishReplay(w, r, webhook, replayInFlight)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
var event database.Event
|
||||||
|
|
||||||
|
err = webhookDB.
|
||||||
|
First(&event, "id = ?", original.EventID).Error
|
||||||
|
if err != nil {
|
||||||
|
h.serverError(w, "failed to load event for replay", err)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
task, err := createReplayDelivery(
|
||||||
|
webhookDB, webhook.ID, &event, target,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
h.serverError(
|
||||||
|
w, "failed to create replay delivery", err,
|
||||||
|
)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
h.mtr.DeliveryReplayed(target.Type)
|
||||||
|
h.notifier.Notify([]delivery.Task{task})
|
||||||
|
|
||||||
|
h.log.Info(
|
||||||
|
"delivery replay queued",
|
||||||
|
"webhook_id", webhook.ID,
|
||||||
|
"event_id", event.ID,
|
||||||
|
"target_id", target.ID,
|
||||||
|
"replayed_delivery_id", original.ID,
|
||||||
|
"delivery_id", task.DeliveryID,
|
||||||
|
)
|
||||||
|
|
||||||
|
h.finishReplay(w, r, webhook, replayQueued)
|
||||||
|
}
|
||||||
|
|
||||||
|
// replayTarget loads the delivery's target as it stands now.
|
||||||
|
//
|
||||||
|
// The load is Unscoped so that a soft-deleted row is still found:
|
||||||
|
// deletes are soft and a delivery carries no foreign key to its
|
||||||
|
// target, so a target's history outlives it, and without the deleted
|
||||||
|
// row there is no way to tell "you deleted this target" from "this id
|
||||||
|
// never named anything". A nil target means the replay is refused,
|
||||||
|
// with the returned code saying why.
|
||||||
|
func (h *Handlers) replayTarget(
|
||||||
|
webhookID, targetID string,
|
||||||
|
) (*database.Target, replayOutcomeCode) {
|
||||||
|
var target database.Target
|
||||||
|
|
||||||
|
err := h.db.DB().Unscoped().Where(
|
||||||
|
"id = ? AND webhook_id = ?", targetID, webhookID,
|
||||||
|
).First(&target).Error
|
||||||
|
if err != nil {
|
||||||
|
return nil, replayTargetMissing
|
||||||
|
}
|
||||||
|
|
||||||
|
if target.DeletedAt.Valid {
|
||||||
|
return nil, replayTargetDeleted
|
||||||
|
}
|
||||||
|
|
||||||
|
if !target.Active {
|
||||||
|
return nil, replayTargetInactive
|
||||||
|
}
|
||||||
|
|
||||||
|
return &target, replayQueued
|
||||||
|
}
|
||||||
|
|
||||||
|
// countInFlightDeliveries reports how many deliveries of this event to
|
||||||
|
// this target the engine has not finished.
|
||||||
|
//
|
||||||
|
// It is the replay-storm guard: a replay is refused while an earlier
|
||||||
|
// one is still pending or retrying, so a held-down button or a scripted
|
||||||
|
// loop cannot stack copies of work already queued. It is a check and
|
||||||
|
// not a lock, so two simultaneous POSTs can still both pass it; the
|
||||||
|
// per-client rate limit on the route is what bounds that.
|
||||||
|
func countInFlightDeliveries(
|
||||||
|
webhookDB *gorm.DB, eventID, targetID string,
|
||||||
|
) (int64, error) {
|
||||||
|
var count int64
|
||||||
|
|
||||||
|
err := webhookDB.Model(&database.Delivery{}).Where(
|
||||||
|
"event_id = ? AND target_id = ? AND status IN ?",
|
||||||
|
eventID, targetID,
|
||||||
|
[]database.DeliveryStatus{
|
||||||
|
database.DeliveryStatusPending,
|
||||||
|
database.DeliveryStatusRetrying,
|
||||||
|
},
|
||||||
|
).Count(&count).Error
|
||||||
|
|
||||||
|
return count, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// createReplayDelivery writes the new pending delivery row and returns
|
||||||
|
// the task that carries it to the delivery engine.
|
||||||
|
//
|
||||||
|
// The row is written with associations omitted, and neither Event nor
|
||||||
|
// Target is populated on it: GORM's SaveBeforeAssociations would
|
||||||
|
// otherwise upsert the whole target row — plaintext config, which for a
|
||||||
|
// Slack target is the credential — into the per-webhook event database.
|
||||||
|
// See https://git.eeqj.de/sneak/webhooker/issues/206.
|
||||||
|
func createReplayDelivery(
|
||||||
|
webhookDB *gorm.DB,
|
||||||
|
webhookID string,
|
||||||
|
event *database.Event,
|
||||||
|
target *database.Target,
|
||||||
|
) (delivery.Task, error) {
|
||||||
|
dlv := &database.Delivery{
|
||||||
|
EventID: event.ID,
|
||||||
|
TargetID: target.ID,
|
||||||
|
Status: database.DeliveryStatusPending,
|
||||||
|
}
|
||||||
|
|
||||||
|
err := webhookDB.Omit(clause.Associations).Create(dlv).Error
|
||||||
|
if err != nil {
|
||||||
|
return delivery.Task{}, err
|
||||||
|
}
|
||||||
|
|
||||||
|
return delivery.Task{
|
||||||
|
DeliveryID: dlv.ID,
|
||||||
|
EventID: event.ID,
|
||||||
|
WebhookID: webhookID,
|
||||||
|
EntrypointID: event.EntrypointID,
|
||||||
|
TargetID: target.ID,
|
||||||
|
TargetName: target.Name,
|
||||||
|
TargetType: target.Type,
|
||||||
|
TargetConfig: target.Config,
|
||||||
|
MaxRetries: target.MaxRetries,
|
||||||
|
Method: event.Method,
|
||||||
|
Headers: event.Headers,
|
||||||
|
ContentType: event.ContentType,
|
||||||
|
Body: replayBody(event.Body),
|
||||||
|
AttemptNum: 1,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// replayBody returns the stored event body for a replay task to carry
|
||||||
|
// inline, or nil when it is large enough that the engine should fetch
|
||||||
|
// it from the per-webhook database instead.
|
||||||
|
func replayBody(body string) *string {
|
||||||
|
if len(body) >= delivery.MaxInlineBodySize {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return &body
|
||||||
|
}
|
||||||
|
|
||||||
|
// finishReplay redirects back to the event log the replay was
|
||||||
|
// triggered from, carrying the outcome code the page turns into a
|
||||||
|
// banner and the page number the form submitted.
|
||||||
|
func (h *Handlers) finishReplay(
|
||||||
|
w http.ResponseWriter,
|
||||||
|
r *http.Request,
|
||||||
|
webhook database.Webhook,
|
||||||
|
code replayOutcomeCode,
|
||||||
|
) {
|
||||||
|
dest := "/source/" + webhook.ID + "/logs?" +
|
||||||
|
replayOutcomeParam + "=" + string(code)
|
||||||
|
|
||||||
|
// The page is read from the form rather than the query string:
|
||||||
|
// this is a POST, and its query string is what logs and Referer
|
||||||
|
// headers record.
|
||||||
|
if page := parseNonNegativeInt(
|
||||||
|
r.PostFormValue("page"),
|
||||||
|
); page > 1 {
|
||||||
|
dest += "&page=" + strconv.Itoa(page)
|
||||||
|
}
|
||||||
|
|
||||||
|
http.Redirect(w, r, dest, http.StatusSeeOther)
|
||||||
|
}
|
||||||
526
internal/handlers/delivery_replay_test.go
Normal file
526
internal/handlers/delivery_replay_test.go
Normal file
@@ -0,0 +1,526 @@
|
|||||||
|
package handlers_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
|
"sneak.berlin/go/webhooker/internal/handlers"
|
||||||
|
"sneak.berlin/go/webhooker/internal/session"
|
||||||
|
)
|
||||||
|
|
||||||
|
// paramDeliveryID is the chi URL parameter name the replay handler
|
||||||
|
// reads.
|
||||||
|
const paramDeliveryID = "deliveryID"
|
||||||
|
|
||||||
|
// replayTargetURL is a public destination, so a target configured with
|
||||||
|
// it is one the SSRF guard would accept. Nothing in these tests
|
||||||
|
// dispatches to it: the notifier is recorded, not run.
|
||||||
|
const replayTargetURL = "http://93.184.216.34/hook"
|
||||||
|
|
||||||
|
// seedFailedDelivery records an event, a terminally failed delivery of
|
||||||
|
// it to the given target, and the attempt that failed.
|
||||||
|
func seedFailedDelivery(
|
||||||
|
t *testing.T,
|
||||||
|
dbMgr *database.WebhookDBManager,
|
||||||
|
webhookID, targetID string,
|
||||||
|
) (*database.Event, *database.Delivery) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
webhookDB, err := dbMgr.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
event := &database.Event{
|
||||||
|
WebhookID: webhookID,
|
||||||
|
EntrypointID: "entrypoint-" + webhookID,
|
||||||
|
Method: http.MethodPost,
|
||||||
|
Headers: `{"X-Test":["yes"]}`,
|
||||||
|
Body: `{"replay":"me"}`,
|
||||||
|
ContentType: "application/json",
|
||||||
|
}
|
||||||
|
|
||||||
|
require.NoError(t, webhookDB.Omit(
|
||||||
|
clause.Associations,
|
||||||
|
).Create(event).Error)
|
||||||
|
|
||||||
|
dlv := &database.Delivery{
|
||||||
|
EventID: event.ID,
|
||||||
|
TargetID: targetID,
|
||||||
|
Status: database.DeliveryStatusFailed,
|
||||||
|
}
|
||||||
|
|
||||||
|
require.NoError(t, webhookDB.Omit(
|
||||||
|
clause.Associations,
|
||||||
|
).Create(dlv).Error)
|
||||||
|
|
||||||
|
result := &database.DeliveryResult{
|
||||||
|
DeliveryID: dlv.ID,
|
||||||
|
AttemptNum: 1,
|
||||||
|
Success: false,
|
||||||
|
StatusCode: http.StatusBadGateway,
|
||||||
|
Error: "connection refused",
|
||||||
|
}
|
||||||
|
|
||||||
|
require.NoError(t, webhookDB.Omit(
|
||||||
|
clause.Associations,
|
||||||
|
).Create(result).Error)
|
||||||
|
|
||||||
|
return event, dlv
|
||||||
|
}
|
||||||
|
|
||||||
|
// loadDelivery reads a delivery back out of a webhook's database.
|
||||||
|
func loadDelivery(
|
||||||
|
t *testing.T, webhookDB *gorm.DB, deliveryID string,
|
||||||
|
) database.Delivery {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var dlv database.Delivery
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t,
|
||||||
|
webhookDB.First(&dlv, "id = ?", deliveryID).Error,
|
||||||
|
)
|
||||||
|
|
||||||
|
return dlv
|
||||||
|
}
|
||||||
|
|
||||||
|
// listDeliveries reads every delivery of an event.
|
||||||
|
func listDeliveries(
|
||||||
|
t *testing.T, webhookDB *gorm.DB, eventID string,
|
||||||
|
) []database.Delivery {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var deliveries []database.Delivery
|
||||||
|
|
||||||
|
require.NoError(t, webhookDB.Where(
|
||||||
|
"event_id = ?", eventID,
|
||||||
|
).Find(&deliveries).Error)
|
||||||
|
|
||||||
|
return deliveries
|
||||||
|
}
|
||||||
|
|
||||||
|
// theOtherDelivery returns the one delivery in the slice that is not
|
||||||
|
// excludeID. Identity is used rather than an ordering because the rows
|
||||||
|
// are minted milliseconds apart and their ids are random.
|
||||||
|
func theOtherDelivery(
|
||||||
|
t *testing.T,
|
||||||
|
deliveries []database.Delivery,
|
||||||
|
excludeID string,
|
||||||
|
) database.Delivery {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var found []database.Delivery
|
||||||
|
|
||||||
|
for _, d := range deliveries {
|
||||||
|
if d.ID != excludeID {
|
||||||
|
found = append(found, d)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
require.Len(t, found, 1)
|
||||||
|
|
||||||
|
return found[0]
|
||||||
|
}
|
||||||
|
|
||||||
|
// postReplay runs the real replay handler for one delivery.
|
||||||
|
func postReplay(
|
||||||
|
t *testing.T,
|
||||||
|
h *handlers.Handlers,
|
||||||
|
sess *session.Session,
|
||||||
|
webhookID, deliveryID string,
|
||||||
|
) *httptest.ResponseRecorder {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
req := postRequest(
|
||||||
|
"/source/"+webhookID+"/deliveries/"+
|
||||||
|
deliveryID+"/replay",
|
||||||
|
authenticatedCookies(
|
||||||
|
t, sess, deleteTestUserID, deleteTestUsername,
|
||||||
|
),
|
||||||
|
map[string]string{
|
||||||
|
paramSourceID: webhookID,
|
||||||
|
paramDeliveryID: deliveryID,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
h.HandleDeliveryReplay().ServeHTTP(w, req)
|
||||||
|
|
||||||
|
return w
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal is the
|
||||||
|
// core requirement: replaying a failed delivery succeeds, appends a
|
||||||
|
// new delivery, and leaves the original row and its recorded attempt
|
||||||
|
// exactly as they were.
|
||||||
|
//
|
||||||
|
// It also pins the two things a replay would be wrong to get from the
|
||||||
|
// original: the task carries the target's CURRENT configuration, which
|
||||||
|
// this test changes between the failure and the replay, and it carries
|
||||||
|
// the stored EVENT body rather than anything the failed attempt
|
||||||
|
// received back.
|
||||||
|
func TestHandleDeliveryReplay_AppendsDeliveryAndLeavesOriginal(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
sess *session.Session
|
||||||
|
db *database.Database
|
||||||
|
dbMgr *database.WebhookDBManager
|
||||||
|
notif *recordingNotifier
|
||||||
|
)
|
||||||
|
|
||||||
|
app := newTestApp(t, &h, &sess, &db, &dbMgr, ¬if)
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
|
wh := seedWebhook(t, db)
|
||||||
|
tgt := seedConfiguredTarget(
|
||||||
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
|
`{"url":"`+replayTargetURL+`"}`,
|
||||||
|
)
|
||||||
|
|
||||||
|
event, original := seedFailedDelivery(
|
||||||
|
t, dbMgr, wh.ID, tgt.ID,
|
||||||
|
)
|
||||||
|
|
||||||
|
webhookDB, err := dbMgr.GetDB(wh.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
before := loadDelivery(t, webhookDB, original.ID)
|
||||||
|
|
||||||
|
// The operator fixes the destination, which is the whole reason
|
||||||
|
// to replay. The replay must use this, not the config the
|
||||||
|
// original delivery ran against.
|
||||||
|
const fixedConfig = `{"url":"http://93.184.216.34/fixed"}`
|
||||||
|
|
||||||
|
require.NoError(t, db.DB().Model(&database.Target{}).
|
||||||
|
Where("id = ?", tgt.ID).
|
||||||
|
Update("config", fixedConfig).Error)
|
||||||
|
|
||||||
|
w := postReplay(t, h, sess, wh.ID, original.ID)
|
||||||
|
|
||||||
|
require.Equal(t, http.StatusSeeOther, w.Code)
|
||||||
|
assert.Equal(
|
||||||
|
t,
|
||||||
|
"/source/"+wh.ID+"/logs?replay=queued",
|
||||||
|
w.Header().Get("Location"),
|
||||||
|
)
|
||||||
|
|
||||||
|
deliveries := listDeliveries(t, webhookDB, event.ID)
|
||||||
|
require.Len(
|
||||||
|
t, deliveries, 2,
|
||||||
|
"replay must append a delivery, not reuse one",
|
||||||
|
)
|
||||||
|
|
||||||
|
replayed := theOtherDelivery(t, deliveries, original.ID)
|
||||||
|
assert.Equal(t, tgt.ID, replayed.TargetID)
|
||||||
|
assert.Equal(t, event.ID, replayed.EventID)
|
||||||
|
assert.Equal(
|
||||||
|
t, database.DeliveryStatusPending, replayed.Status,
|
||||||
|
)
|
||||||
|
|
||||||
|
assertDeliveryUntouched(t, webhookDB, before)
|
||||||
|
|
||||||
|
tasks := notif.Tasks()
|
||||||
|
require.Len(t, tasks, 1)
|
||||||
|
assertReplayTask(
|
||||||
|
t, tasks[0], wh.ID, event, tgt, replayed.ID, fixedConfig,
|
||||||
|
)
|
||||||
|
assertNoLeakedTarget(t, webhookDB)
|
||||||
|
}
|
||||||
|
|
||||||
|
// assertDeliveryUntouched proves a delivery row is exactly as it was
|
||||||
|
// read before: same terminal status, same timestamps, and the same
|
||||||
|
// recorded attempts.
|
||||||
|
func assertDeliveryUntouched(
|
||||||
|
t *testing.T,
|
||||||
|
webhookDB *gorm.DB,
|
||||||
|
before database.Delivery,
|
||||||
|
) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
after := loadDelivery(t, webhookDB, before.ID)
|
||||||
|
assert.Equal(
|
||||||
|
t, before.Status, after.Status,
|
||||||
|
"replay must not resurrect the original delivery",
|
||||||
|
)
|
||||||
|
assert.Equal(t, before.UpdatedAt, after.UpdatedAt)
|
||||||
|
assert.Equal(t, before.CreatedAt, after.CreatedAt)
|
||||||
|
|
||||||
|
var attempts int64
|
||||||
|
|
||||||
|
require.NoError(t, webhookDB.
|
||||||
|
Model(&database.DeliveryResult{}).
|
||||||
|
Where("delivery_id = ?", before.ID).
|
||||||
|
Count(&attempts).Error)
|
||||||
|
assert.Equal(
|
||||||
|
t, int64(1), attempts,
|
||||||
|
"the original delivery's attempt history must stand",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// assertReplayTask proves the task handed to the delivery engine is
|
||||||
|
// the one the receiver would build for this event and this target, and
|
||||||
|
// that it carries wantConfig — the target's configuration as it stands
|
||||||
|
// now rather than as the original delivery ran against it.
|
||||||
|
func assertReplayTask(
|
||||||
|
t *testing.T,
|
||||||
|
task delivery.Task,
|
||||||
|
webhookID string,
|
||||||
|
event *database.Event,
|
||||||
|
target *database.Target,
|
||||||
|
wantDeliveryID, wantConfig string,
|
||||||
|
) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
assert.Equal(t, wantDeliveryID, task.DeliveryID)
|
||||||
|
assert.Equal(t, event.ID, task.EventID)
|
||||||
|
assert.Equal(t, webhookID, task.WebhookID)
|
||||||
|
assert.Equal(t, event.EntrypointID, task.EntrypointID)
|
||||||
|
assert.Equal(t, target.ID, task.TargetID)
|
||||||
|
assert.Equal(t, target.Type, task.TargetType)
|
||||||
|
assert.JSONEq(
|
||||||
|
t, wantConfig, task.TargetConfig,
|
||||||
|
"replay must use the target's current configuration",
|
||||||
|
)
|
||||||
|
assert.Equal(t, event.Method, task.Method)
|
||||||
|
assert.Equal(t, event.Headers, task.Headers)
|
||||||
|
assert.Equal(t, event.ContentType, task.ContentType)
|
||||||
|
assert.Equal(t, 1, task.AttemptNum)
|
||||||
|
require.NotNil(t, task.Body)
|
||||||
|
assert.Equal(
|
||||||
|
t, event.Body, *task.Body,
|
||||||
|
"replay re-sends the stored event body",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// assertNoLeakedTarget proves the per-webhook database holds no target
|
||||||
|
// rows. AutoMigrate creates the table there because Delivery declares
|
||||||
|
// the relation, so it is a ROW that signals a leak: an association
|
||||||
|
// write would have upserted the whole target, plaintext config and
|
||||||
|
// all, into the event database. See
|
||||||
|
// https://git.eeqj.de/sneak/webhooker/issues/206.
|
||||||
|
func assertNoLeakedTarget(t *testing.T, webhookDB *gorm.DB) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var leaked int64
|
||||||
|
|
||||||
|
require.NoError(t, webhookDB.Unscoped().
|
||||||
|
Model(&database.Target{}).Count(&leaked).Error)
|
||||||
|
assert.Zero(
|
||||||
|
t, leaked,
|
||||||
|
"replay must not write the target into the event database",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHandleDeliveryReplay_RefusesDeletedTarget proves the required
|
||||||
|
// refusal: a target deleted since the delivery ran is reported as
|
||||||
|
// deleted rather than erroring, and nothing is created or queued.
|
||||||
|
func TestHandleDeliveryReplay_RefusesDeletedTarget(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
sess *session.Session
|
||||||
|
db *database.Database
|
||||||
|
dbMgr *database.WebhookDBManager
|
||||||
|
notif *recordingNotifier
|
||||||
|
)
|
||||||
|
|
||||||
|
app := newTestApp(t, &h, &sess, &db, &dbMgr, ¬if)
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
|
wh := seedWebhook(t, db)
|
||||||
|
tgt := seedConfiguredTarget(
|
||||||
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
|
`{"url":"`+replayTargetURL+`"}`,
|
||||||
|
)
|
||||||
|
|
||||||
|
event, original := seedFailedDelivery(
|
||||||
|
t, dbMgr, wh.ID, tgt.ID,
|
||||||
|
)
|
||||||
|
|
||||||
|
// Deletes are soft, so the delivery history outlives the target.
|
||||||
|
require.NoError(t, db.DB().Where(
|
||||||
|
"id = ?", tgt.ID,
|
||||||
|
).Delete(&database.Target{}).Error)
|
||||||
|
|
||||||
|
w := postReplay(t, h, sess, wh.ID, original.ID)
|
||||||
|
|
||||||
|
require.Equal(t, http.StatusSeeOther, w.Code)
|
||||||
|
assert.Equal(
|
||||||
|
t,
|
||||||
|
"/source/"+wh.ID+"/logs?replay=target-deleted",
|
||||||
|
w.Header().Get("Location"),
|
||||||
|
)
|
||||||
|
|
||||||
|
webhookDB, err := dbMgr.GetDB(wh.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
assert.Len(
|
||||||
|
t, listDeliveries(t, webhookDB, event.ID), 1,
|
||||||
|
"a refused replay must create no delivery",
|
||||||
|
)
|
||||||
|
assert.Empty(
|
||||||
|
t, notif.Tasks(),
|
||||||
|
"a refused replay must queue nothing",
|
||||||
|
)
|
||||||
|
|
||||||
|
// The refusal is specific, which is why the target is looked up
|
||||||
|
// including soft-deleted rows: an id that never named a target
|
||||||
|
// is a different outcome, and a different message, from one the
|
||||||
|
// operator deleted.
|
||||||
|
_, orphan := seedFailedDelivery(
|
||||||
|
t, dbMgr, wh.ID, "target-that-never-existed",
|
||||||
|
)
|
||||||
|
|
||||||
|
missing := postReplay(t, h, sess, wh.ID, orphan.ID)
|
||||||
|
require.Equal(t, http.StatusSeeOther, missing.Code)
|
||||||
|
assert.Equal(
|
||||||
|
t,
|
||||||
|
"/source/"+wh.ID+"/logs?replay=target-missing",
|
||||||
|
missing.Header().Get("Location"),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHandleDeliveryReplay_RefusesWhileEarlierReplayInFlight proves
|
||||||
|
// the replay-storm guard: a second replay of the same event to the
|
||||||
|
// same target is refused while the first is still queued, so repeated
|
||||||
|
// submissions cannot stack copies of work the engine has not done.
|
||||||
|
func TestHandleDeliveryReplay_RefusesWhileEarlierReplayInFlight(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
sess *session.Session
|
||||||
|
db *database.Database
|
||||||
|
dbMgr *database.WebhookDBManager
|
||||||
|
notif *recordingNotifier
|
||||||
|
)
|
||||||
|
|
||||||
|
app := newTestApp(t, &h, &sess, &db, &dbMgr, ¬if)
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
|
wh := seedWebhook(t, db)
|
||||||
|
tgt := seedConfiguredTarget(
|
||||||
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
|
`{"url":"`+replayTargetURL+`"}`,
|
||||||
|
)
|
||||||
|
|
||||||
|
event, original := seedFailedDelivery(
|
||||||
|
t, dbMgr, wh.ID, tgt.ID,
|
||||||
|
)
|
||||||
|
|
||||||
|
first := postReplay(t, h, sess, wh.ID, original.ID)
|
||||||
|
require.Equal(t, http.StatusSeeOther, first.Code)
|
||||||
|
require.Equal(
|
||||||
|
t,
|
||||||
|
"/source/"+wh.ID+"/logs?replay=queued",
|
||||||
|
first.Header().Get("Location"),
|
||||||
|
)
|
||||||
|
|
||||||
|
second := postReplay(t, h, sess, wh.ID, original.ID)
|
||||||
|
require.Equal(t, http.StatusSeeOther, second.Code)
|
||||||
|
assert.Equal(
|
||||||
|
t,
|
||||||
|
"/source/"+wh.ID+"/logs?replay=in-flight",
|
||||||
|
second.Header().Get("Location"),
|
||||||
|
)
|
||||||
|
|
||||||
|
webhookDB, err := dbMgr.GetDB(wh.ID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
assert.Len(
|
||||||
|
t, listDeliveries(t, webhookDB, event.ID), 2,
|
||||||
|
"the refused second replay must add nothing",
|
||||||
|
)
|
||||||
|
assert.Len(
|
||||||
|
t, notif.Tasks(), 1,
|
||||||
|
"only the first replay reaches the delivery engine",
|
||||||
|
)
|
||||||
|
|
||||||
|
// A delivery the engine has not finished is not replayable
|
||||||
|
// either, which is the same rule seen from the other side.
|
||||||
|
queued := theOtherDelivery(
|
||||||
|
t, listDeliveries(t, webhookDB, event.ID), original.ID,
|
||||||
|
)
|
||||||
|
|
||||||
|
pending := postReplay(t, h, sess, wh.ID, queued.ID)
|
||||||
|
require.Equal(t, http.StatusSeeOther, pending.Code)
|
||||||
|
assert.Equal(
|
||||||
|
t,
|
||||||
|
"/source/"+wh.ID+"/logs?replay=not-terminal",
|
||||||
|
pending.Header().Get("Location"),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHandleSourceLogs_RendersReplayControlAndBanner proves the action
|
||||||
|
// reaches the page it belongs on: a finished delivery renders a POST
|
||||||
|
// form carrying a CSRF token, and the outcome code a refusal redirects
|
||||||
|
// with becomes a readable message.
|
||||||
|
func TestHandleSourceLogs_RendersReplayControlAndBanner(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
sess *session.Session
|
||||||
|
db *database.Database
|
||||||
|
dbMgr *database.WebhookDBManager
|
||||||
|
)
|
||||||
|
|
||||||
|
app := newTestApp(t, &h, &sess, &db, &dbMgr)
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
|
wh := seedWebhook(t, db)
|
||||||
|
tgt := seedConfiguredTarget(
|
||||||
|
t, db, wh.ID, database.TargetTypeHTTP,
|
||||||
|
`{"url":"`+replayTargetURL+`"}`,
|
||||||
|
)
|
||||||
|
|
||||||
|
_, original := seedFailedDelivery(t, dbMgr, wh.ID, tgt.ID)
|
||||||
|
|
||||||
|
body := renderSourceLogsPage(t, h, sess, wh.ID)
|
||||||
|
|
||||||
|
assert.Contains(
|
||||||
|
t, body,
|
||||||
|
`action="/source/`+wh.ID+`/deliveries/`+
|
||||||
|
original.ID+`/replay"`,
|
||||||
|
)
|
||||||
|
assert.Contains(t, body, `method="POST"`)
|
||||||
|
assert.Contains(t, body, `name="csrf_token"`)
|
||||||
|
assert.Contains(t, body, ">Replay<")
|
||||||
|
|
||||||
|
refused := renderSourceLogsPageWithQuery(
|
||||||
|
t, h, sess, wh.ID, "?replay=target-deleted",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.Contains(t, refused, "alert-error")
|
||||||
|
assert.Contains(t, refused, "has been deleted")
|
||||||
|
|
||||||
|
// An outcome code nobody issued renders no banner at all.
|
||||||
|
unknown := renderSourceLogsPageWithQuery(
|
||||||
|
t, h, sess, wh.ID, "?replay=made-up",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.NotContains(t, unknown, "alert-error")
|
||||||
|
assert.NotContains(t, unknown, "alert-success")
|
||||||
|
assert.NotContains(t, unknown, "made-up")
|
||||||
|
}
|
||||||
342
internal/handlers/entrypoint_secret_test.go
Normal file
342
internal/handlers/entrypoint_secret_test.go
Normal file
@@ -0,0 +1,342 @@
|
|||||||
|
package handlers_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/url"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/handlers"
|
||||||
|
"sneak.berlin/go/webhooker/internal/session"
|
||||||
|
)
|
||||||
|
|
||||||
|
// submitEntrypointSecret posts the signature configuration form for
|
||||||
|
// an entrypoint and returns the recorder.
|
||||||
|
func submitEntrypointSecret(
|
||||||
|
t *testing.T,
|
||||||
|
h *handlers.Handlers,
|
||||||
|
cookies []*http.Cookie,
|
||||||
|
webhookID, entrypointID, scheme, secret string,
|
||||||
|
) *httptest.ResponseRecorder {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
form := url.Values{}
|
||||||
|
form.Set("signature_scheme", scheme)
|
||||||
|
form.Set("secret", secret)
|
||||||
|
|
||||||
|
req := formRequest(
|
||||||
|
"/source/"+webhookID+"/entrypoints/"+
|
||||||
|
entrypointID+"/secret",
|
||||||
|
cookies,
|
||||||
|
form,
|
||||||
|
map[string]string{
|
||||||
|
paramSourceID: webhookID,
|
||||||
|
entrypointIDParam: entrypointID,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
|
h.HandleEntrypointSecret().ServeHTTP(w, req)
|
||||||
|
|
||||||
|
return w
|
||||||
|
}
|
||||||
|
|
||||||
|
// reloadEntrypoint reads an entrypoint back from the database,
|
||||||
|
// including the columns the model keeps out of JSON.
|
||||||
|
func reloadEntrypoint(
|
||||||
|
t *testing.T,
|
||||||
|
db *database.Database,
|
||||||
|
id string,
|
||||||
|
) database.Entrypoint {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var ep database.Entrypoint
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t, db.DB().Where("id = ?", id).First(&ep).Error,
|
||||||
|
)
|
||||||
|
|
||||||
|
return ep
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestEntrypointSecretSetRotateAndRemove walks the whole lifecycle
|
||||||
|
// the UI has to support: turning verification on, rotating the secret
|
||||||
|
// to a new value, and turning it back off.
|
||||||
|
func TestEntrypointSecretSetRotateAndRemove(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
sess *session.Session
|
||||||
|
db *database.Database
|
||||||
|
)
|
||||||
|
|
||||||
|
app := newTestApp(t, &h, &sess, &db)
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
|
cookies := authenticatedCookies(
|
||||||
|
t, sess, deleteTestUserID, deleteTestUsername,
|
||||||
|
)
|
||||||
|
wh := seedWebhook(t, db)
|
||||||
|
ep := seedSignedEntrypoint(
|
||||||
|
t, db, wh.ID, database.SignatureSchemeNone, "",
|
||||||
|
)
|
||||||
|
|
||||||
|
// Set.
|
||||||
|
w := submitEntrypointSecret(
|
||||||
|
t, h, cookies, wh.ID, ep.ID, "github", inboundSecret,
|
||||||
|
)
|
||||||
|
require.Equal(t, http.StatusSeeOther, w.Code)
|
||||||
|
|
||||||
|
stored := reloadEntrypoint(t, db, ep.ID)
|
||||||
|
assert.Equal(
|
||||||
|
t, database.SignatureSchemeGitHub, stored.SignatureScheme,
|
||||||
|
)
|
||||||
|
assert.Equal(t, inboundSecret, stored.SignatureSecret)
|
||||||
|
assert.True(t, stored.SignatureConfigured())
|
||||||
|
|
||||||
|
// Rotate: a new secret and a different scheme in one submission.
|
||||||
|
// The new value is submitted with surrounding whitespace, the way
|
||||||
|
// a secret pasted out of a password manager arrives; storing that
|
||||||
|
// verbatim would make every later request fail verification with
|
||||||
|
// nothing visible on either side to explain it.
|
||||||
|
const rotated = "QQROTATEDSECRETQQ"
|
||||||
|
|
||||||
|
w = submitEntrypointSecret(
|
||||||
|
t, h, cookies, wh.ID, ep.ID, "gitlab", " "+rotated+"\t",
|
||||||
|
)
|
||||||
|
require.Equal(t, http.StatusSeeOther, w.Code)
|
||||||
|
|
||||||
|
stored = reloadEntrypoint(t, db, ep.ID)
|
||||||
|
assert.Equal(
|
||||||
|
t, database.SignatureSchemeGitLab, stored.SignatureScheme,
|
||||||
|
)
|
||||||
|
assert.Equal(t, rotated, stored.SignatureSecret)
|
||||||
|
|
||||||
|
// Remove. The secret has to go with the scheme: a stored
|
||||||
|
// credential nothing reads is one more copy to leak.
|
||||||
|
w = submitEntrypointSecret(t, h, cookies, wh.ID, ep.ID, "", "")
|
||||||
|
require.Equal(t, http.StatusSeeOther, w.Code)
|
||||||
|
|
||||||
|
stored = reloadEntrypoint(t, db, ep.ID)
|
||||||
|
assert.Equal(
|
||||||
|
t, database.SignatureSchemeNone, stored.SignatureScheme,
|
||||||
|
)
|
||||||
|
assert.Empty(t, stored.SignatureSecret)
|
||||||
|
assert.False(t, stored.SignatureConfigured())
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestEntrypointSecretRejectsBadInput proves the form cannot create a
|
||||||
|
// row the receiver would later have to refuse. Both rejections leave
|
||||||
|
// the stored configuration untouched rather than half-applied.
|
||||||
|
func TestEntrypointSecretRejectsBadInput(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
cases := []struct {
|
||||||
|
name string
|
||||||
|
scheme string
|
||||||
|
secret string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "unsupported scheme",
|
||||||
|
scheme: "stripe",
|
||||||
|
secret: inboundSecret,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "scheme with no secret",
|
||||||
|
scheme: "github",
|
||||||
|
secret: "",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
// Whitespace is stripped, so a secret of spaces is an
|
||||||
|
// empty one.
|
||||||
|
name: "scheme with blank secret",
|
||||||
|
scheme: "github",
|
||||||
|
secret: " ",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
sess *session.Session
|
||||||
|
db *database.Database
|
||||||
|
)
|
||||||
|
|
||||||
|
app := newTestApp(t, &h, &sess, &db)
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
|
cookies := authenticatedCookies(
|
||||||
|
t, sess, deleteTestUserID, deleteTestUsername,
|
||||||
|
)
|
||||||
|
|
||||||
|
for _, tc := range cases {
|
||||||
|
wh := seedWebhook(t, db)
|
||||||
|
ep := seedSignedEntrypoint(
|
||||||
|
t, db, wh.ID,
|
||||||
|
database.SignatureSchemeGitLab, inboundSecret,
|
||||||
|
)
|
||||||
|
|
||||||
|
w := submitEntrypointSecret(
|
||||||
|
t, h, cookies, wh.ID, ep.ID, tc.scheme, tc.secret,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.Equal(
|
||||||
|
t, http.StatusBadRequest, w.Code, "case %s", tc.name,
|
||||||
|
)
|
||||||
|
|
||||||
|
stored := reloadEntrypoint(t, db, ep.ID)
|
||||||
|
assert.Equal(
|
||||||
|
t,
|
||||||
|
database.SignatureSchemeGitLab,
|
||||||
|
stored.SignatureScheme,
|
||||||
|
"case %s", tc.name,
|
||||||
|
)
|
||||||
|
assert.Equal(
|
||||||
|
t, inboundSecret, stored.SignatureSecret,
|
||||||
|
"case %s", tc.name,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestEntrypointSecretRequiresOwnership proves the configuration
|
||||||
|
// endpoint is bound by the same ownership check as the rest of the
|
||||||
|
// webhook's pages: another user's entrypoint is a 404, and the secret
|
||||||
|
// is not touched.
|
||||||
|
func TestEntrypointSecretRequiresOwnership(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
sess *session.Session
|
||||||
|
db *database.Database
|
||||||
|
)
|
||||||
|
|
||||||
|
app := newTestApp(t, &h, &sess, &db)
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
|
wh := seedWebhook(t, db)
|
||||||
|
ep := seedSignedEntrypoint(
|
||||||
|
t, db, wh.ID,
|
||||||
|
database.SignatureSchemeGitLab, inboundSecret,
|
||||||
|
)
|
||||||
|
|
||||||
|
stranger := authenticatedCookies(
|
||||||
|
t, sess, "someone-else", "someoneelse",
|
||||||
|
)
|
||||||
|
|
||||||
|
w := submitEntrypointSecret(
|
||||||
|
t, h, stranger, wh.ID, ep.ID, "github", "hijacked",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.Equal(t, http.StatusNotFound, w.Code)
|
||||||
|
assert.Equal(
|
||||||
|
t,
|
||||||
|
inboundSecret,
|
||||||
|
reloadEntrypoint(t, db, ep.ID).SignatureSecret,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHandleSourceDetail_MasksEntrypointSecret is the regression test
|
||||||
|
// for the credential on the entrypoint: the page has to say that
|
||||||
|
// verification is configured and which header carries it, without the
|
||||||
|
// secret itself ever reaching the rendered HTML.
|
||||||
|
func TestHandleSourceDetail_MasksEntrypointSecret(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
sess *session.Session
|
||||||
|
db *database.Database
|
||||||
|
)
|
||||||
|
|
||||||
|
app := newTestApp(t, &h, &sess, &db)
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
|
wh := seedWebhook(t, db)
|
||||||
|
seedSignedEntrypoint(
|
||||||
|
t, db, wh.ID,
|
||||||
|
database.SignatureSchemeGitHub, inboundSecret,
|
||||||
|
)
|
||||||
|
|
||||||
|
body := renderSourceDetailPage(t, h, sess, wh.ID)
|
||||||
|
|
||||||
|
assert.NotContains(t, body, inboundSecret)
|
||||||
|
assert.Contains(t, body, "GitHub")
|
||||||
|
assert.Contains(t, body, "X-Hub-Signature-256")
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestEntrypointViewsDropTheSecret pins the projection itself, so the
|
||||||
|
// barrier survives a template rewrite that stops rendering the field
|
||||||
|
// the page test above looks at.
|
||||||
|
func TestEntrypointViewsDropTheSecret(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
views := handlers.NewEntrypointViews([]database.Entrypoint{
|
||||||
|
{
|
||||||
|
Path: "p1",
|
||||||
|
Active: true,
|
||||||
|
SignatureScheme: database.SignatureSchemeGitHub,
|
||||||
|
SignatureSecret: inboundSecret,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Path: "p2",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
// Half a configuration. The receiver 500s every request
|
||||||
|
// to this row, so the UI must not call it unverified.
|
||||||
|
Path: "p2a",
|
||||||
|
SignatureScheme: database.SignatureSchemeGitLab,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
// The other half.
|
||||||
|
Path: "p2b",
|
||||||
|
SignatureSecret: inboundSecret,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
// A scheme this build does not know: described as
|
||||||
|
// unavailable, never echoed back.
|
||||||
|
Path: "p3",
|
||||||
|
SignatureScheme: database.SignatureScheme("stripe"),
|
||||||
|
SignatureSecret: inboundSecret,
|
||||||
|
},
|
||||||
|
})
|
||||||
|
|
||||||
|
require.Len(t, views, 5)
|
||||||
|
|
||||||
|
assert.True(t, views[0].Configured)
|
||||||
|
assert.Equal(t, "GitHub", views[0].SchemeLabel)
|
||||||
|
assert.Equal(t, "X-Hub-Signature-256", views[0].SchemeHeader)
|
||||||
|
|
||||||
|
assert.False(t, views[1].Configured)
|
||||||
|
assert.Equal(t, "not verified", views[1].SchemeLabel)
|
||||||
|
assert.Empty(t, views[1].SchemeHeader)
|
||||||
|
|
||||||
|
for _, v := range []handlers.EntrypointView{views[2], views[3]} {
|
||||||
|
assert.False(t, v.Configured)
|
||||||
|
assert.Equal(t, "misconfigured", v.SchemeLabel)
|
||||||
|
assert.Empty(t, v.SchemeHeader)
|
||||||
|
}
|
||||||
|
|
||||||
|
assert.True(t, views[4].Configured)
|
||||||
|
assert.Equal(t, "(unavailable)", views[4].SchemeLabel)
|
||||||
|
|
||||||
|
// The struct has no field that could carry the secret, so this
|
||||||
|
// fails to compile rather than fails at runtime if one is added
|
||||||
|
// and populated. The assertion covers the labels it derives.
|
||||||
|
for _, v := range views {
|
||||||
|
assert.NotContains(t, v.SchemeLabel, inboundSecret)
|
||||||
|
assert.NotContains(t, v.SchemeHeader, inboundSecret)
|
||||||
|
assert.NotContains(t, string(v.Scheme), inboundSecret)
|
||||||
|
}
|
||||||
|
}
|
||||||
91
internal/handlers/entrypoint_view.go
Normal file
91
internal/handlers/entrypoint_view.go
Normal file
@@ -0,0 +1,91 @@
|
|||||||
|
package handlers
|
||||||
|
|
||||||
|
import (
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/signature"
|
||||||
|
)
|
||||||
|
|
||||||
|
// signatureUnavailable is what an entrypoint's scheme renders as when
|
||||||
|
// the stored value is not one this build supports. The stored string
|
||||||
|
// is never echoed as a fallback: it is operator-supplied and the row
|
||||||
|
// is already in a state the receiver refuses, so the UI says so
|
||||||
|
// rather than inventing a description for it.
|
||||||
|
const signatureUnavailable = "(unavailable)"
|
||||||
|
|
||||||
|
// signatureNotVerified is the label for an entrypoint that performs
|
||||||
|
// no inbound verification.
|
||||||
|
const signatureNotVerified = "not verified"
|
||||||
|
|
||||||
|
// signatureMisconfigured is the label for a row holding one half of
|
||||||
|
// the scheme/secret pair. The receiver answers every request to such
|
||||||
|
// an entrypoint 500, so calling it "not verified" would describe a
|
||||||
|
// receiver that is refusing everything as one that is accepting
|
||||||
|
// everything. The form cannot create the state; a hand-edited
|
||||||
|
// database or a downgrade past a scheme can.
|
||||||
|
const signatureMisconfigured = "misconfigured"
|
||||||
|
|
||||||
|
// EntrypointView is the display-safe projection of an entrypoint for
|
||||||
|
// the UI. It deliberately has no secret field, so no template —
|
||||||
|
// present or future — can render the shared secret, in the same way
|
||||||
|
// delivery.TargetView keeps a target's stored credential away from
|
||||||
|
// one.
|
||||||
|
type EntrypointView struct {
|
||||||
|
ID string
|
||||||
|
Path string
|
||||||
|
Description string
|
||||||
|
Active bool
|
||||||
|
|
||||||
|
// Configured reports whether inbound requests to this entrypoint
|
||||||
|
// are verified.
|
||||||
|
Configured bool
|
||||||
|
|
||||||
|
// Scheme is the stored scheme, carried so the form can preselect
|
||||||
|
// it. It names an algorithm, not a secret.
|
||||||
|
Scheme database.SignatureScheme
|
||||||
|
|
||||||
|
// SchemeLabel and SchemeHeader describe the configured scheme for
|
||||||
|
// display: the sender's name, and the header its signature
|
||||||
|
// arrives in.
|
||||||
|
SchemeLabel string
|
||||||
|
SchemeHeader string
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewEntrypointViews projects entrypoints for rendering, dropping the
|
||||||
|
// shared secret on the way.
|
||||||
|
func NewEntrypointViews(
|
||||||
|
entrypoints []database.Entrypoint,
|
||||||
|
) []EntrypointView {
|
||||||
|
views := make([]EntrypointView, 0, len(entrypoints))
|
||||||
|
|
||||||
|
for i := range entrypoints {
|
||||||
|
e := &entrypoints[i]
|
||||||
|
|
||||||
|
view := EntrypointView{
|
||||||
|
ID: e.ID,
|
||||||
|
Path: e.Path,
|
||||||
|
Description: e.Description,
|
||||||
|
Active: e.Active,
|
||||||
|
Configured: e.SignatureConfigured(),
|
||||||
|
Scheme: e.SignatureScheme,
|
||||||
|
SchemeLabel: signatureNotVerified,
|
||||||
|
SchemeHeader: "",
|
||||||
|
}
|
||||||
|
|
||||||
|
switch {
|
||||||
|
case view.Configured:
|
||||||
|
view.SchemeLabel = signatureUnavailable
|
||||||
|
|
||||||
|
info, ok := signature.Info(e.SignatureScheme)
|
||||||
|
if ok {
|
||||||
|
view.SchemeLabel = info.Label
|
||||||
|
view.SchemeHeader = info.Header
|
||||||
|
}
|
||||||
|
case e.SignatureHalfConfigured():
|
||||||
|
view.SchemeLabel = signatureMisconfigured
|
||||||
|
}
|
||||||
|
|
||||||
|
views = append(views, view)
|
||||||
|
}
|
||||||
|
|
||||||
|
return views
|
||||||
|
}
|
||||||
@@ -24,9 +24,32 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/session"
|
"sneak.berlin/go/webhooker/internal/session"
|
||||||
)
|
)
|
||||||
|
|
||||||
type noopNotifier struct{}
|
// recordingNotifier is a delivery.Notifier that records the tasks it
|
||||||
|
// was handed, so a test can prove a handler queued the delivery it
|
||||||
|
// claims to have queued — and, on the refusal paths, that it queued
|
||||||
|
// nothing.
|
||||||
|
type recordingNotifier struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
tasks []delivery.Task
|
||||||
|
}
|
||||||
|
|
||||||
func (n *noopNotifier) Notify([]delivery.Task) {}
|
func (n *recordingNotifier) Notify(tasks []delivery.Task) {
|
||||||
|
n.mu.Lock()
|
||||||
|
defer n.mu.Unlock()
|
||||||
|
|
||||||
|
n.tasks = append(n.tasks, tasks...)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Tasks returns a copy of the recorded tasks.
|
||||||
|
func (n *recordingNotifier) Tasks() []delivery.Task {
|
||||||
|
n.mu.Lock()
|
||||||
|
defer n.mu.Unlock()
|
||||||
|
|
||||||
|
out := make([]delivery.Task, len(n.tasks))
|
||||||
|
copy(out, n.tasks)
|
||||||
|
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
// recordingEvictor is a delivery.WebhookEvictor that records
|
// recordingEvictor is a delivery.WebhookEvictor that records
|
||||||
// the webhook ids it was asked to evict, so a test can prove
|
// the webhook ids it was asked to evict, so a test can prove
|
||||||
@@ -74,8 +97,11 @@ func newTestApp(
|
|||||||
database.NewWebhookDBManager,
|
database.NewWebhookDBManager,
|
||||||
healthcheck.New,
|
healthcheck.New,
|
||||||
session.New,
|
session.New,
|
||||||
func() delivery.Notifier {
|
func() *recordingNotifier {
|
||||||
return &noopNotifier{}
|
return &recordingNotifier{}
|
||||||
|
},
|
||||||
|
func(n *recordingNotifier) delivery.Notifier {
|
||||||
|
return n
|
||||||
},
|
},
|
||||||
func() *recordingEvictor {
|
func() *recordingEvictor {
|
||||||
return &recordingEvictor{}
|
return &recordingEvictor{}
|
||||||
|
|||||||
@@ -60,10 +60,26 @@ func renderSourceLogsPage(
|
|||||||
) string {
|
) string {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
|
return renderSourceLogsPageWithQuery(
|
||||||
|
t, h, sess, webhookID, "",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// renderSourceLogsPageWithQuery is renderSourceLogsPage over a
|
||||||
|
// caller-supplied query string, for the page state a redirect back to
|
||||||
|
// the log carries in one.
|
||||||
|
func renderSourceLogsPageWithQuery(
|
||||||
|
t *testing.T,
|
||||||
|
h *handlers.Handlers,
|
||||||
|
sess *session.Session,
|
||||||
|
webhookID, query string,
|
||||||
|
) string {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
req := httptest.NewRequestWithContext(
|
req := httptest.NewRequestWithContext(
|
||||||
context.Background(),
|
context.Background(),
|
||||||
http.MethodGet,
|
http.MethodGet,
|
||||||
"/source/"+webhookID+"/logs",
|
"/source/"+webhookID+"/logs"+query,
|
||||||
nil,
|
nil,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ import (
|
|||||||
"github.com/google/uuid"
|
"github.com/google/uuid"
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
"sneak.berlin/go/webhooker/internal/delivery"
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
|
"sneak.berlin/go/webhooker/internal/signature"
|
||||||
)
|
)
|
||||||
|
|
||||||
// WebhookListItem holds data for the webhook list view.
|
// WebhookListItem holds data for the webhook list view.
|
||||||
@@ -414,13 +415,16 @@ func (h *Handlers) renderSourceDetail(
|
|||||||
// receivers; html/template cannot address a value stored in a map.
|
// receivers; html/template cannot address a value stored in a map.
|
||||||
data := map[string]any{
|
data := map[string]any{
|
||||||
tmplKeyWebhook: &webhook,
|
tmplKeyWebhook: &webhook,
|
||||||
"Entrypoints": entrypoints,
|
// Entrypoints and targets are both projected to
|
||||||
// Targets are projected to a display-safe view: the
|
// display-safe views: an entrypoint carries the shared
|
||||||
// stored config blob holds credentials and must never
|
// secret its senders sign with and a target's stored
|
||||||
|
// config blob holds a credential, and neither must ever
|
||||||
// reach a template.
|
// reach a template.
|
||||||
"Targets": delivery.NewTargetViews(targets),
|
"Entrypoints": NewEntrypointViews(entrypoints),
|
||||||
"Events": events,
|
"Targets": delivery.NewTargetViews(targets),
|
||||||
"BaseURL": scheme + "://" + host,
|
"SignatureSchemes": signature.Schemes(),
|
||||||
|
"Events": events,
|
||||||
|
"BaseURL": scheme + "://" + host,
|
||||||
}
|
}
|
||||||
|
|
||||||
h.renderTemplate(w, r, "source_detail.html", data)
|
h.renderTemplate(w, r, "source_detail.html", data)
|
||||||
@@ -777,16 +781,25 @@ func (h *Handlers) HandleSourceLogs() http.HandlerFunc {
|
|||||||
totalPages++
|
totalPages++
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The banner a replay POST redirected back with. The
|
||||||
|
// message comes from a fixed set keyed by the outcome
|
||||||
|
// code, never from the query string itself.
|
||||||
|
replayMsg, replayOK := replayOutcome(
|
||||||
|
r.URL.Query().Get(replayOutcomeParam),
|
||||||
|
)
|
||||||
|
|
||||||
data := map[string]any{
|
data := map[string]any{
|
||||||
tmplKeyWebhook: &webhook,
|
tmplKeyWebhook: &webhook,
|
||||||
"Events": evts,
|
"Events": evts,
|
||||||
"Page": page,
|
"ReplayMessage": replayMsg,
|
||||||
"TotalPages": totalPages,
|
"ReplayQueued": replayOK,
|
||||||
"TotalEvents": total,
|
"Page": page,
|
||||||
"HasPrev": page > 1,
|
"TotalPages": totalPages,
|
||||||
"HasNext": page < totalPages,
|
"TotalEvents": total,
|
||||||
"PrevPage": page - 1,
|
"HasPrev": page > 1,
|
||||||
"NextPage": page + 1,
|
"HasNext": page < totalPages,
|
||||||
|
"PrevPage": page - 1,
|
||||||
|
"NextPage": page + 1,
|
||||||
}
|
}
|
||||||
|
|
||||||
h.renderTemplate(w, r, "source_logs.html", data)
|
h.renderTemplate(w, r, "source_logs.html", data)
|
||||||
@@ -972,6 +985,145 @@ func (h *Handlers) HandleEntrypointCreate() http.HandlerFunc {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// HandleEntrypointSecret sets, rotates or removes the shared secret
|
||||||
|
// an entrypoint verifies inbound requests with.
|
||||||
|
//
|
||||||
|
// Setting and rotating are the same operation: the form always takes
|
||||||
|
// the secret afresh and the stored value is never sent to the browser
|
||||||
|
// to be edited, so there is no path by which the page can display a
|
||||||
|
// credential it holds. Rotation is therefore "submit the new secret",
|
||||||
|
// and the operator already has that value — both supported senders
|
||||||
|
// require them to enter the same string on the sender's side, so
|
||||||
|
// there is no generated value for webhooker to reveal once.
|
||||||
|
func (h *Handlers) HandleEntrypointSecret() http.HandlerFunc {
|
||||||
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
webhook, ok := h.ownedWebhook(w, r)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// The body size cap is enforced by the MaxBodySize
|
||||||
|
// middleware, which runs before CSRF parses the form.
|
||||||
|
err := r.ParseForm()
|
||||||
|
if err != nil {
|
||||||
|
http.Error(
|
||||||
|
w, "Bad request", http.StatusBadRequest,
|
||||||
|
)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
var entrypoint database.Entrypoint
|
||||||
|
|
||||||
|
err = h.db.DB().Where(
|
||||||
|
"id = ? AND webhook_id = ?",
|
||||||
|
chi.URLParam(r, "entrypointID"), webhook.ID,
|
||||||
|
).First(&entrypoint).Error
|
||||||
|
if err != nil {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
h.applyEntrypointSecret(w, r, &entrypoint)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// applyEntrypointSecret validates the submitted scheme and secret and
|
||||||
|
// stores them.
|
||||||
|
//
|
||||||
|
// A scheme this build does not support is a 400, never a stored value
|
||||||
|
// the receiver would later have to interpret: the receiver fails such
|
||||||
|
// a row closed, so letting one be created would take the entrypoint
|
||||||
|
// offline through a form that reported success.
|
||||||
|
func (h *Handlers) applyEntrypointSecret(
|
||||||
|
w http.ResponseWriter,
|
||||||
|
r *http.Request,
|
||||||
|
entrypoint *database.Entrypoint,
|
||||||
|
) {
|
||||||
|
// PostFormValue, not FormValue: a credential must come from the
|
||||||
|
// body. FormValue falls back to the query string, and the request
|
||||||
|
// line — unlike the body — is what logs, proxies, Referer headers
|
||||||
|
// and error trackers record.
|
||||||
|
scheme := database.SignatureScheme(
|
||||||
|
r.PostFormValue("signature_scheme"),
|
||||||
|
)
|
||||||
|
|
||||||
|
// Surrounding whitespace is stripped, because a secret pasted from
|
||||||
|
// a password manager routinely carries some and the resulting
|
||||||
|
// mismatch is undiagnosable from the sender's side. A secret whose
|
||||||
|
// own first or last character is a space cannot be stored; the
|
||||||
|
// README says so.
|
||||||
|
secret := strings.TrimSpace(r.PostFormValue("secret"))
|
||||||
|
|
||||||
|
if !signature.Supported(scheme) {
|
||||||
|
http.Error(
|
||||||
|
w, "Invalid signature scheme",
|
||||||
|
http.StatusBadRequest,
|
||||||
|
)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if scheme == database.SignatureSchemeNone {
|
||||||
|
// Turning verification off drops the secret with it: a stored
|
||||||
|
// credential nothing reads is one more copy to leak, and
|
||||||
|
// Verify refuses that pairing in any case.
|
||||||
|
secret = ""
|
||||||
|
} else if secret == "" {
|
||||||
|
http.Error(
|
||||||
|
w,
|
||||||
|
"A shared secret is required for this signature scheme.",
|
||||||
|
http.StatusBadRequest,
|
||||||
|
)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
h.storeEntrypointSecret(w, r, entrypoint, scheme, secret)
|
||||||
|
}
|
||||||
|
|
||||||
|
// storeEntrypointSecret writes a validated scheme and secret to an
|
||||||
|
// entrypoint and returns the operator to the webhook page.
|
||||||
|
func (h *Handlers) storeEntrypointSecret(
|
||||||
|
w http.ResponseWriter,
|
||||||
|
r *http.Request,
|
||||||
|
entrypoint *database.Entrypoint,
|
||||||
|
scheme database.SignatureScheme,
|
||||||
|
secret string,
|
||||||
|
) {
|
||||||
|
// Updates with a map rather than a struct: a struct update skips
|
||||||
|
// zero values, and the empty pair is exactly what has to be
|
||||||
|
// written when verification is being turned off.
|
||||||
|
err := h.db.DB().Model(entrypoint).Updates(map[string]any{
|
||||||
|
"signature_scheme": scheme,
|
||||||
|
"signature_secret": secret,
|
||||||
|
}).Error
|
||||||
|
if err != nil {
|
||||||
|
// The error is logged by serverError; GORM's error text
|
||||||
|
// carries the statement, not the bound values, so the secret
|
||||||
|
// does not travel with it.
|
||||||
|
h.serverError(
|
||||||
|
w, "failed to update entrypoint signature", err,
|
||||||
|
)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
h.log.Info(
|
||||||
|
"entrypoint signature configuration updated",
|
||||||
|
"entrypoint_id", entrypoint.ID,
|
||||||
|
"webhook_id", entrypoint.WebhookID,
|
||||||
|
"scheme", string(scheme),
|
||||||
|
)
|
||||||
|
|
||||||
|
http.Redirect(
|
||||||
|
w, r,
|
||||||
|
"/source/"+entrypoint.WebhookID,
|
||||||
|
http.StatusSeeOther,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
// HandleTargetCreate handles adding a new target to a webhook.
|
// HandleTargetCreate handles adding a new target to a webhook.
|
||||||
func (h *Handlers) HandleTargetCreate() http.HandlerFunc {
|
func (h *Handlers) HandleTargetCreate() http.HandlerFunc {
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/delivery"
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
"sneak.berlin/go/webhooker/internal/handlers"
|
"sneak.berlin/go/webhooker/internal/handlers"
|
||||||
"sneak.berlin/go/webhooker/internal/session"
|
"sneak.berlin/go/webhooker/internal/session"
|
||||||
|
"sneak.berlin/go/webhooker/internal/signature"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Template data keys the page templates read. The handlers package has
|
// Template data keys the page templates read. The handlers package has
|
||||||
@@ -268,12 +269,16 @@ func TestEntrypointCopyButtonIsProgressiveEnhancement(t *testing.T) {
|
|||||||
|
|
||||||
body := renderPage(t, h, sess, "source_detail.html", map[string]any{
|
body := renderPage(t, h, sess, "source_detail.html", map[string]any{
|
||||||
dataKeyWebhook: webhook,
|
dataKeyWebhook: webhook,
|
||||||
"Entrypoints": []database.Entrypoint{entrypoint},
|
// The handler passes projected views, never raw rows — an
|
||||||
// The handler passes delivery.NewTargetViews(targets), never
|
// entrypoint carries its shared secret and a target its
|
||||||
// raw targets, so the test data has to have that same shape.
|
// stored credential — so the test data has that same shape.
|
||||||
"Targets": delivery.NewTargetViews(nil),
|
"Entrypoints": handlers.NewEntrypointViews(
|
||||||
"Events": []database.Event{},
|
[]database.Entrypoint{entrypoint},
|
||||||
"BaseURL": "https://hooks.example.com",
|
),
|
||||||
|
"Targets": delivery.NewTargetViews(nil),
|
||||||
|
"SignatureSchemes": signature.Schemes(),
|
||||||
|
"Events": []database.Event{},
|
||||||
|
"BaseURL": "https://hooks.example.com",
|
||||||
})
|
})
|
||||||
|
|
||||||
assert.Contains(
|
assert.Contains(
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ package handlers
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
"io"
|
"io"
|
||||||
"net/http"
|
"net/http"
|
||||||
|
|
||||||
@@ -10,6 +11,7 @@ import (
|
|||||||
"sneak.berlin/go/webhooker/internal/database"
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
"sneak.berlin/go/webhooker/internal/delivery"
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
"sneak.berlin/go/webhooker/internal/logfield"
|
"sneak.berlin/go/webhooker/internal/logfield"
|
||||||
|
"sneak.berlin/go/webhooker/internal/signature"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -69,8 +71,8 @@ func (h *Handlers) HandleWebhook() http.HandlerFunc {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// processWebhookRequest reads the body, serializes headers,
|
// processWebhookRequest reads the body, verifies the sender,
|
||||||
// loads targets, and delivers the event.
|
// serializes headers, loads targets, and delivers the event.
|
||||||
func (h *Handlers) processWebhookRequest(
|
func (h *Handlers) processWebhookRequest(
|
||||||
w http.ResponseWriter,
|
w http.ResponseWriter,
|
||||||
r *http.Request,
|
r *http.Request,
|
||||||
@@ -81,7 +83,26 @@ func (h *Handlers) processWebhookRequest(
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
headersJSON, err := json.Marshal(r.Header)
|
// Before anything is written. An unverified request must leave no
|
||||||
|
// event row, no delivery row and no delivery task behind, so this
|
||||||
|
// sits above every write rather than inside the transaction that
|
||||||
|
// performs them. It has to sit below the body read because the
|
||||||
|
// signature is computed over the body; readWebhookBody is what
|
||||||
|
// bounds that read, so an unauthenticated sender still cannot make
|
||||||
|
// the process hold more than the 1 MB cap.
|
||||||
|
if !h.verifyInboundSignature(w, entrypoint, r.Header, body) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// These headers are about to be stored verbatim and handed to
|
||||||
|
// every delivery target, so the scheme's credential comes out
|
||||||
|
// first. Under GitLab's scheme the header is the shared secret
|
||||||
|
// itself, and leaving it in would hand the ability to forge
|
||||||
|
// signed requests to exactly the parties the signature is meant
|
||||||
|
// to exclude.
|
||||||
|
headersJSON, err := json.Marshal(
|
||||||
|
signature.SanitizeHeaders(&entrypoint, r.Header),
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
h.serverError(w, "failed to serialize headers", err)
|
h.serverError(w, "failed to serialize headers", err)
|
||||||
|
|
||||||
@@ -100,6 +121,63 @@ func (h *Handlers) processWebhookRequest(
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// verifyInboundSignature authenticates the request against the
|
||||||
|
// entrypoint's configured secret, reporting false once it has written
|
||||||
|
// the response.
|
||||||
|
//
|
||||||
|
// An entrypoint with no secret configured is not checked and this
|
||||||
|
// returns true, which is the unchanged behaviour every existing
|
||||||
|
// entrypoint keeps.
|
||||||
|
//
|
||||||
|
// A configuration that cannot be applied — an unknown scheme, or one
|
||||||
|
// half of the pair missing — is a 500, not a 401: the request may well
|
||||||
|
// be authentic, and calling it unauthorized would tell a legitimate
|
||||||
|
// sender to go fix its own signing. Either way it is refused. Failing
|
||||||
|
// open here would mean an entrypoint the operator has protected
|
||||||
|
// quietly accepting anything.
|
||||||
|
func (h *Handlers) verifyInboundSignature(
|
||||||
|
w http.ResponseWriter,
|
||||||
|
entrypoint database.Entrypoint,
|
||||||
|
header http.Header,
|
||||||
|
body []byte,
|
||||||
|
) bool {
|
||||||
|
err := signature.Verify(&entrypoint, header, body)
|
||||||
|
if err == nil {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
if errors.Is(err, signature.ErrConfig) {
|
||||||
|
h.log.Error(
|
||||||
|
"entrypoint signature configuration cannot be applied",
|
||||||
|
"entrypoint_id", entrypoint.ID,
|
||||||
|
"webhook_id", entrypoint.WebhookID,
|
||||||
|
"error", err,
|
||||||
|
)
|
||||||
|
http.Error(
|
||||||
|
w, "Internal server error",
|
||||||
|
http.StatusInternalServerError,
|
||||||
|
)
|
||||||
|
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// Every field here is bounded and none is client-chosen: the ids
|
||||||
|
// are ours, the scheme is one of a fixed set, and the error is a
|
||||||
|
// static string carrying no part of the secret or of what the
|
||||||
|
// client presented. Reaching this line also requires a real
|
||||||
|
// entrypoint UUID, so it is not a line a stranger can drive.
|
||||||
|
h.log.Warn(
|
||||||
|
"inbound signature verification failed",
|
||||||
|
"entrypoint_id", entrypoint.ID,
|
||||||
|
"webhook_id", entrypoint.WebhookID,
|
||||||
|
"scheme", string(entrypoint.SignatureScheme),
|
||||||
|
"error", err,
|
||||||
|
)
|
||||||
|
http.Error(w, "Unauthorized", http.StatusUnauthorized)
|
||||||
|
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
// loadActiveTargets returns all active targets for a webhook.
|
// loadActiveTargets returns all active targets for a webhook.
|
||||||
func (h *Handlers) loadActiveTargets(
|
func (h *Handlers) loadActiveTargets(
|
||||||
webhookID string,
|
webhookID string,
|
||||||
|
|||||||
468
internal/handlers/webhook_signature_test.go
Normal file
468
internal/handlers/webhook_signature_test.go
Normal file
@@ -0,0 +1,468 @@
|
|||||||
|
package handlers_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"crypto/hmac"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"log/slog"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/go-chi/chi"
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"gorm.io/gorm/clause"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/handlers"
|
||||||
|
"sneak.berlin/go/webhooker/internal/signature"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
// inboundSecret is the shared secret the signed-receiver tests
|
||||||
|
// configure on their entrypoint. It doubles as a marker: no log
|
||||||
|
// line and no rendered page may contain it.
|
||||||
|
inboundSecret = "QQINBOUNDSECRETQQ"
|
||||||
|
|
||||||
|
// inboundBody is the payload the sender signs.
|
||||||
|
inboundBody = `{"zen":"Non-blocking is better than blocking."}`
|
||||||
|
|
||||||
|
// entrypointIDParam is the chi URL parameter naming an entrypoint.
|
||||||
|
entrypointIDParam = "entrypointID"
|
||||||
|
)
|
||||||
|
|
||||||
|
// hubSignature returns the X-Hub-Signature-256 value a GitHub sender
|
||||||
|
// holding secret would send for inboundBody.
|
||||||
|
func hubSignature(secret string) string {
|
||||||
|
mac := hmac.New(sha256.New, []byte(secret))
|
||||||
|
_, _ = mac.Write([]byte(inboundBody))
|
||||||
|
|
||||||
|
return "sha256=" + hex.EncodeToString(mac.Sum(nil))
|
||||||
|
}
|
||||||
|
|
||||||
|
// seedSignedEntrypoint inserts an active entrypoint for a webhook
|
||||||
|
// with the given signature configuration and returns it.
|
||||||
|
func seedSignedEntrypoint(
|
||||||
|
t *testing.T,
|
||||||
|
db *database.Database,
|
||||||
|
webhookID string,
|
||||||
|
scheme database.SignatureScheme,
|
||||||
|
secret string,
|
||||||
|
) *database.Entrypoint {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
ep := &database.Entrypoint{
|
||||||
|
WebhookID: webhookID,
|
||||||
|
Path: "path-" + webhookID,
|
||||||
|
Description: "signed",
|
||||||
|
Active: true,
|
||||||
|
SignatureScheme: scheme,
|
||||||
|
SignatureSecret: secret,
|
||||||
|
}
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t,
|
||||||
|
db.DB().Omit(clause.Associations).Create(ep).Error,
|
||||||
|
)
|
||||||
|
|
||||||
|
return ep
|
||||||
|
}
|
||||||
|
|
||||||
|
// postToEntrypoint drives the real receiver handler at an
|
||||||
|
// entrypoint's path with one optional header set.
|
||||||
|
func postToEntrypoint(
|
||||||
|
t *testing.T,
|
||||||
|
h *handlers.Handlers,
|
||||||
|
path, body, headerName, headerValue string,
|
||||||
|
) *httptest.ResponseRecorder {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
req := httptest.NewRequestWithContext(
|
||||||
|
context.Background(),
|
||||||
|
http.MethodPost,
|
||||||
|
"/webhook/"+path,
|
||||||
|
strings.NewReader(body),
|
||||||
|
)
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
|
||||||
|
if headerName != "" {
|
||||||
|
req.Header.Set(headerName, headerValue)
|
||||||
|
}
|
||||||
|
|
||||||
|
rctx := chi.NewRouteContext()
|
||||||
|
rctx.URLParams.Add("uuid", path)
|
||||||
|
|
||||||
|
req = req.WithContext(
|
||||||
|
context.WithValue(req.Context(), chi.RouteCtxKey, rctx),
|
||||||
|
)
|
||||||
|
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
h.HandleWebhook().ServeHTTP(w, req)
|
||||||
|
|
||||||
|
return w
|
||||||
|
}
|
||||||
|
|
||||||
|
// storedEvents counts the event rows a webhook's per-webhook database
|
||||||
|
// holds. A database that was never opened holds none, which is the
|
||||||
|
// state a rejected request has to leave behind.
|
||||||
|
func storedEvents(
|
||||||
|
t *testing.T,
|
||||||
|
mgr *database.WebhookDBManager,
|
||||||
|
webhookID string,
|
||||||
|
) int64 {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
if !mgr.DBExists(webhookID) {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
db, err := mgr.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
var count int64
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t,
|
||||||
|
db.Model(&database.Event{}).
|
||||||
|
Where("webhook_id = ?", webhookID).
|
||||||
|
Count(&count).Error,
|
||||||
|
)
|
||||||
|
|
||||||
|
return count
|
||||||
|
}
|
||||||
|
|
||||||
|
// storedEventHeaders reads back the Headers column of the single
|
||||||
|
// event row a webhook's per-webhook database holds.
|
||||||
|
//
|
||||||
|
// It reads the database rather than an in-memory struct on purpose:
|
||||||
|
// what matters is what an operator, a backup or the reaper's archive
|
||||||
|
// would find on disk, not what the handler passed around.
|
||||||
|
func storedEventHeaders(
|
||||||
|
t *testing.T,
|
||||||
|
mgr *database.WebhookDBManager,
|
||||||
|
webhookID string,
|
||||||
|
) string {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
require.True(t, mgr.DBExists(webhookID))
|
||||||
|
|
||||||
|
db, err := mgr.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
var events []database.Event
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t,
|
||||||
|
db.Where("webhook_id = ?", webhookID).
|
||||||
|
Find(&events).Error,
|
||||||
|
)
|
||||||
|
require.Len(t, events, 1)
|
||||||
|
|
||||||
|
return events[0].Headers
|
||||||
|
}
|
||||||
|
|
||||||
|
// signedReceiverCase is one inbound request against an entrypoint
|
||||||
|
// with a given stored signature configuration.
|
||||||
|
type signedReceiverCase struct {
|
||||||
|
name string
|
||||||
|
scheme database.SignatureScheme
|
||||||
|
secret string
|
||||||
|
headerName string
|
||||||
|
headerValue string
|
||||||
|
body string
|
||||||
|
wantStatus int
|
||||||
|
}
|
||||||
|
|
||||||
|
// signedReceiverCases covers each supported scheme with a valid
|
||||||
|
// signature, an invalid one and none at all, plus the two states that
|
||||||
|
// are not "a client got it wrong": an entrypoint with nothing
|
||||||
|
// configured, and one whose stored configuration cannot be applied.
|
||||||
|
func signedReceiverCases() []signedReceiverCase {
|
||||||
|
return append(
|
||||||
|
schemeReceiverCases(), unverifiedReceiverCases()...,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// schemeReceiverCases covers the two supported schemes.
|
||||||
|
func schemeReceiverCases() []signedReceiverCase {
|
||||||
|
return []signedReceiverCase{
|
||||||
|
{
|
||||||
|
name: "github valid",
|
||||||
|
scheme: database.SignatureSchemeGitHub,
|
||||||
|
secret: inboundSecret,
|
||||||
|
headerName: signature.HeaderGitHub,
|
||||||
|
headerValue: hubSignature(inboundSecret),
|
||||||
|
body: inboundBody,
|
||||||
|
wantStatus: http.StatusOK,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "github wrong secret",
|
||||||
|
scheme: database.SignatureSchemeGitHub,
|
||||||
|
secret: inboundSecret,
|
||||||
|
headerName: signature.HeaderGitHub,
|
||||||
|
headerValue: hubSignature("wrong"),
|
||||||
|
body: inboundBody,
|
||||||
|
wantStatus: http.StatusUnauthorized,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
// A digest that was valid for a different body: the
|
||||||
|
// check is over the bytes as received.
|
||||||
|
name: "github body tampered",
|
||||||
|
scheme: database.SignatureSchemeGitHub,
|
||||||
|
secret: inboundSecret,
|
||||||
|
headerName: signature.HeaderGitHub,
|
||||||
|
headerValue: hubSignature(inboundSecret),
|
||||||
|
body: inboundBody + " ",
|
||||||
|
wantStatus: http.StatusUnauthorized,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "github unsigned",
|
||||||
|
scheme: database.SignatureSchemeGitHub,
|
||||||
|
secret: inboundSecret,
|
||||||
|
body: inboundBody,
|
||||||
|
wantStatus: http.StatusUnauthorized,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "gitlab valid",
|
||||||
|
scheme: database.SignatureSchemeGitLab,
|
||||||
|
secret: inboundSecret,
|
||||||
|
headerName: signature.HeaderGitLab,
|
||||||
|
headerValue: inboundSecret,
|
||||||
|
body: inboundBody,
|
||||||
|
wantStatus: http.StatusOK,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "gitlab wrong token",
|
||||||
|
scheme: database.SignatureSchemeGitLab,
|
||||||
|
secret: inboundSecret,
|
||||||
|
headerName: signature.HeaderGitLab,
|
||||||
|
headerValue: "wrong",
|
||||||
|
body: inboundBody,
|
||||||
|
wantStatus: http.StatusUnauthorized,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "gitlab unsigned",
|
||||||
|
scheme: database.SignatureSchemeGitLab,
|
||||||
|
secret: inboundSecret,
|
||||||
|
body: inboundBody,
|
||||||
|
wantStatus: http.StatusUnauthorized,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// unverifiedReceiverCases covers the two entrypoint states that are
|
||||||
|
// not about a client getting its signature wrong: nothing configured
|
||||||
|
// at all, and a configuration the receiver cannot apply.
|
||||||
|
func unverifiedReceiverCases() []signedReceiverCase {
|
||||||
|
return []signedReceiverCase{
|
||||||
|
{
|
||||||
|
// The pass-through case. An entrypoint with nothing
|
||||||
|
// configured is what every deployment already has, and
|
||||||
|
// it must keep accepting unsigned requests so that an
|
||||||
|
// upgrade does not lock an operator out of their own
|
||||||
|
// receivers.
|
||||||
|
name: "unconfigured accepts unsigned",
|
||||||
|
scheme: database.SignatureSchemeNone,
|
||||||
|
body: inboundBody,
|
||||||
|
wantStatus: http.StatusOK,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
// A stray signature header changes nothing when nothing
|
||||||
|
// is configured to check it.
|
||||||
|
name: "unconfigured ignores a stray header",
|
||||||
|
scheme: database.SignatureSchemeNone,
|
||||||
|
headerName: signature.HeaderGitHub,
|
||||||
|
headerValue: "sha256=deadbeef",
|
||||||
|
body: inboundBody,
|
||||||
|
wantStatus: http.StatusOK,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
// A scheme this build cannot apply, reachable only by
|
||||||
|
// editing the database: refused, not waved through as
|
||||||
|
// unverified.
|
||||||
|
name: "unknown scheme fails closed",
|
||||||
|
scheme: database.SignatureScheme("stripe"),
|
||||||
|
secret: inboundSecret,
|
||||||
|
headerName: signature.HeaderGitHub,
|
||||||
|
headerValue: hubSignature(inboundSecret),
|
||||||
|
body: inboundBody,
|
||||||
|
wantStatus: http.StatusInternalServerError,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestReceiverVerifiesConfiguredEntrypoints is the load-bearing test
|
||||||
|
// for the feature: for each supported scheme a correctly signed
|
||||||
|
// request is accepted and stored, and an incorrectly signed or
|
||||||
|
// unsigned one is answered 401 having stored nothing.
|
||||||
|
//
|
||||||
|
// The event count is the half that matters most. A rejection that
|
||||||
|
// still wrote a row would leave the receiver a place for a stranger
|
||||||
|
// who knows a URL to deposit content, which is exactly what the
|
||||||
|
// signature is there to prevent.
|
||||||
|
//
|
||||||
|
// The cases share one application and take a webhook each, rather
|
||||||
|
// than each standing up its own: every newTestApp seeds an admin user
|
||||||
|
// and so pays an Argon2id hash at 64 MB, and this package's test
|
||||||
|
// budget is not large enough to spend one per table row.
|
||||||
|
func TestReceiverVerifiesConfiguredEntrypoints(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
db *database.Database
|
||||||
|
mgr *database.WebhookDBManager
|
||||||
|
)
|
||||||
|
|
||||||
|
app := newTestApp(t, &h, &db, &mgr)
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
|
for _, tc := range signedReceiverCases() {
|
||||||
|
wh := seedWebhook(t, db)
|
||||||
|
ep := seedSignedEntrypoint(
|
||||||
|
t, db, wh.ID, tc.scheme, tc.secret,
|
||||||
|
)
|
||||||
|
|
||||||
|
w := postToEntrypoint(
|
||||||
|
t, h, ep.Path, tc.body,
|
||||||
|
tc.headerName, tc.headerValue,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.Equal(t, tc.wantStatus, w.Code, "case %s", tc.name)
|
||||||
|
|
||||||
|
want := int64(0)
|
||||||
|
if tc.wantStatus == http.StatusOK {
|
||||||
|
want = 1
|
||||||
|
}
|
||||||
|
|
||||||
|
assert.Equal(
|
||||||
|
t, want, storedEvents(t, mgr, wh.ID),
|
||||||
|
"case %s: stored event rows after a %d response",
|
||||||
|
tc.name, w.Code,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestReceiverLogsNoSecret proves the rejection path does not write
|
||||||
|
// the shared secret, or what the client presented, into the log. A
|
||||||
|
// GitLab token arrives as the credential itself, so echoing the
|
||||||
|
// header value would put a live secret in the log of every deployment
|
||||||
|
// whose sender is briefly misconfigured.
|
||||||
|
func TestReceiverLogsNoSecret(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const presented = "QQPRESENTEDVALUEQQ"
|
||||||
|
|
||||||
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
db *database.Database
|
||||||
|
)
|
||||||
|
|
||||||
|
app := newTestApp(t, &h, &db)
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
|
||||||
|
h.SetLogForTest(slog.New(slog.NewJSONHandler(&buf, nil)))
|
||||||
|
|
||||||
|
wh := seedWebhook(t, db)
|
||||||
|
ep := seedSignedEntrypoint(
|
||||||
|
t, db, wh.ID,
|
||||||
|
database.SignatureSchemeGitLab, inboundSecret,
|
||||||
|
)
|
||||||
|
|
||||||
|
w := postToEntrypoint(
|
||||||
|
t, h, ep.Path, inboundBody,
|
||||||
|
signature.HeaderGitLab, presented,
|
||||||
|
)
|
||||||
|
require.Equal(t, http.StatusUnauthorized, w.Code)
|
||||||
|
|
||||||
|
// The rejection is recorded at all — a silent 401 leaves an
|
||||||
|
// operator no way to see a sender failing to authenticate.
|
||||||
|
assert.Contains(t, buf.String(), "verification failed")
|
||||||
|
assert.NotContains(t, buf.String(), inboundSecret)
|
||||||
|
assert.NotContains(t, buf.String(), presented)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestReceiverDoesNotStoreInboundCredential proves an accepted
|
||||||
|
// request leaves no copy of the shared secret in the event store.
|
||||||
|
//
|
||||||
|
// GitLab's X-Gitlab-Token is the credential itself, not a digest
|
||||||
|
// over the request. Stored headers are read back by the UI, copied
|
||||||
|
// into every backup and archive, and handed verbatim to every
|
||||||
|
// delivery target, so a stored token is the entrypoint's only
|
||||||
|
// authentication control disclosed to precisely the parties it
|
||||||
|
// exists to exclude.
|
||||||
|
//
|
||||||
|
// The two cases share one application: every newTestApp seeds an
|
||||||
|
// admin user and pays an Argon2id hash at 64 MB, and this package's
|
||||||
|
// test budget does not stretch to one per case.
|
||||||
|
func TestReceiverDoesNotStoreInboundCredential(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var (
|
||||||
|
h *handlers.Handlers
|
||||||
|
db *database.Database
|
||||||
|
mgr *database.WebhookDBManager
|
||||||
|
)
|
||||||
|
|
||||||
|
app := newTestApp(t, &h, &db, &mgr)
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
|
gitlab := seedWebhook(t, db)
|
||||||
|
gitlabEP := seedSignedEntrypoint(
|
||||||
|
t, db, gitlab.ID,
|
||||||
|
database.SignatureSchemeGitLab, inboundSecret,
|
||||||
|
)
|
||||||
|
|
||||||
|
w := postToEntrypoint(
|
||||||
|
t, h, gitlabEP.Path, inboundBody,
|
||||||
|
signature.HeaderGitLab, inboundSecret,
|
||||||
|
)
|
||||||
|
require.Equal(t, http.StatusOK, w.Code)
|
||||||
|
|
||||||
|
stored := storedEventHeaders(t, mgr, gitlab.ID)
|
||||||
|
|
||||||
|
assert.NotContains(
|
||||||
|
t, stored, inboundSecret,
|
||||||
|
"the shared secret must not be persisted",
|
||||||
|
)
|
||||||
|
assert.NotContains(
|
||||||
|
t, stored, signature.HeaderGitLab,
|
||||||
|
"the credential header must not be persisted at all",
|
||||||
|
)
|
||||||
|
|
||||||
|
// Everything else the sender set is still there. A fix that
|
||||||
|
// stored no headers would satisfy the assertions above while
|
||||||
|
// discarding the record the receiver exists to keep.
|
||||||
|
assert.Contains(t, stored, "Content-Type")
|
||||||
|
|
||||||
|
// A GitHub digest is an HMAC over the body, so the key cannot be
|
||||||
|
// recovered from it and it stays: the stripping is scoped to
|
||||||
|
// what actually carries the secret.
|
||||||
|
github := seedWebhook(t, db)
|
||||||
|
githubEP := seedSignedEntrypoint(
|
||||||
|
t, db, github.ID,
|
||||||
|
database.SignatureSchemeGitHub, inboundSecret,
|
||||||
|
)
|
||||||
|
|
||||||
|
w = postToEntrypoint(
|
||||||
|
t, h, githubEP.Path, inboundBody,
|
||||||
|
signature.HeaderGitHub, hubSignature(inboundSecret),
|
||||||
|
)
|
||||||
|
require.Equal(t, http.StatusOK, w.Code)
|
||||||
|
|
||||||
|
stored = storedEventHeaders(t, mgr, github.ID)
|
||||||
|
|
||||||
|
assert.Contains(t, stored, signature.HeaderGitHub)
|
||||||
|
assert.NotContains(t, stored, inboundSecret)
|
||||||
|
}
|
||||||
@@ -82,6 +82,7 @@ type Set struct {
|
|||||||
deliveriesSucceeded *prometheus.CounterVec
|
deliveriesSucceeded *prometheus.CounterVec
|
||||||
deliveriesFailed *prometheus.CounterVec
|
deliveriesFailed *prometheus.CounterVec
|
||||||
deliveryRetries *prometheus.CounterVec
|
deliveryRetries *prometheus.CounterVec
|
||||||
|
deliveryReplays *prometheus.CounterVec
|
||||||
deliveryDuration *prometheus.HistogramVec
|
deliveryDuration *prometheus.HistogramVec
|
||||||
deliveriesPending *prometheus.GaugeVec
|
deliveriesPending *prometheus.GaugeVec
|
||||||
deliveriesRetrying *prometheus.GaugeVec
|
deliveriesRetrying *prometheus.GaugeVec
|
||||||
@@ -149,6 +150,22 @@ func (s *Set) ObserveDeliveryDuration(
|
|||||||
Observe(d.Seconds())
|
Observe(d.Seconds())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// DeliveryReplayed counts one delivery an operator replayed from the
|
||||||
|
// event log.
|
||||||
|
//
|
||||||
|
// A replay runs the ordinary engine path, so it already moves the
|
||||||
|
// attempt, outcome and duration series exactly as a first delivery
|
||||||
|
// does — deliberately, since a replay is a real delivery and hiding it
|
||||||
|
// from those would misreport the pipeline. This counter is the one
|
||||||
|
// place the two are distinguishable, and it carries the existing
|
||||||
|
// target-type label rather than adding a replay dimension to every
|
||||||
|
// other series.
|
||||||
|
func (s *Set) DeliveryReplayed(t database.TargetType) {
|
||||||
|
s.deliveryReplays.
|
||||||
|
WithLabelValues(normalizeTargetType(t)).
|
||||||
|
Inc()
|
||||||
|
}
|
||||||
|
|
||||||
// DeliveryStatusChanged counts a delivery's transition into a new
|
// DeliveryStatusChanged counts a delivery's transition into a new
|
||||||
// status. The mapping from status to counter lives here, next to the
|
// status. The mapping from status to counter lives here, next to the
|
||||||
// collectors, so the engine has a single call for every transition it
|
// collectors, so the engine has a single call for every transition it
|
||||||
@@ -271,6 +288,16 @@ func (s *Set) registerCounters(factory promauto.Factory) {
|
|||||||
},
|
},
|
||||||
[]string{targetTypeLabel},
|
[]string{targetTypeLabel},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
s.deliveryReplays = factory.NewCounterVec(
|
||||||
|
prometheus.CounterOpts{
|
||||||
|
Namespace: namespace,
|
||||||
|
Name: "delivery_replays_total",
|
||||||
|
Help: "Deliveries an operator replayed from the " +
|
||||||
|
"event log, by target type.",
|
||||||
|
},
|
||||||
|
[]string{targetTypeLabel},
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (s *Set) registerGauges(factory promauto.Factory) {
|
func (s *Set) registerGauges(factory promauto.Factory) {
|
||||||
@@ -322,6 +349,7 @@ func (s *Set) initSeries() {
|
|||||||
s.deliveriesSucceeded.WithLabelValues(label)
|
s.deliveriesSucceeded.WithLabelValues(label)
|
||||||
s.deliveriesFailed.WithLabelValues(label)
|
s.deliveriesFailed.WithLabelValues(label)
|
||||||
s.deliveryRetries.WithLabelValues(label)
|
s.deliveryRetries.WithLabelValues(label)
|
||||||
|
s.deliveryReplays.WithLabelValues(label)
|
||||||
s.deliveriesPending.WithLabelValues(label)
|
s.deliveriesPending.WithLabelValues(label)
|
||||||
s.deliveriesRetrying.WithLabelValues(label)
|
s.deliveriesRetrying.WithLabelValues(label)
|
||||||
s.circuitBreakersOpen.WithLabelValues(label)
|
s.circuitBreakersOpen.WithLabelValues(label)
|
||||||
|
|||||||
@@ -34,6 +34,16 @@ const (
|
|||||||
// password change rate limit.
|
// password change rate limit.
|
||||||
passwordChangeRateInterval = 1 * time.Minute
|
passwordChangeRateInterval = 1 * time.Minute
|
||||||
|
|
||||||
|
// replayRateLimit is the maximum number of delivery replays one
|
||||||
|
// client may queue per interval. Each replay puts a delivery on
|
||||||
|
// the engine's queue, so without a ceiling one operator holding
|
||||||
|
// the button down — or scripting it — queues unbounded outbound
|
||||||
|
// work. It sits far above any rate a person clicks at.
|
||||||
|
replayRateLimit = 30
|
||||||
|
|
||||||
|
// replayRateInterval is the time window for the replay limit.
|
||||||
|
replayRateInterval = 1 * time.Minute
|
||||||
|
|
||||||
// receiverRateInterval is the time window for the webhook
|
// receiverRateInterval is the time window for the webhook
|
||||||
// receiver rate limit. The configured limit is expressed in
|
// receiver rate limit. The configured limit is expressed in
|
||||||
// requests per minute.
|
// requests per minute.
|
||||||
@@ -290,6 +300,21 @@ func (m *Middleware) PasswordChangeRateLimit() func(http.Handler) http.Handler {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ReplayRateLimit returns middleware that enforces per-IP rate
|
||||||
|
// limiting on delivery replays.
|
||||||
|
//
|
||||||
|
// Like the password-change limit it is spent on arrival, which is safe
|
||||||
|
// for the same reason: RequireAuth runs ahead of it, so only a request
|
||||||
|
// already carrying a valid session can reach the bucket.
|
||||||
|
func (m *Middleware) ReplayRateLimit() func(http.Handler) http.Handler {
|
||||||
|
return m.postRateLimit(
|
||||||
|
replayRateLimit,
|
||||||
|
replayRateInterval,
|
||||||
|
"delivery replay rate limit exceeded",
|
||||||
|
"Too many replays. Please try again later.",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
// postRateLimit builds middleware that enforces a per-IP rate
|
// postRateLimit builds middleware that enforces a per-IP rate
|
||||||
// limit on POST requests only; all other methods pass through
|
// limit on POST requests only; all other methods pass through
|
||||||
// unaffected. Requests over the limit receive a 429 with the
|
// unaffected. Requests over the limit receive a 429 with the
|
||||||
|
|||||||
472
internal/resetpw/resetpw.go
Normal file
472
internal/resetpw/resetpw.go
Normal file
@@ -0,0 +1,472 @@
|
|||||||
|
// Package resetpw implements the `webhooker resetpw` subcommand,
|
||||||
|
// which sets an existing account's password from the command line.
|
||||||
|
//
|
||||||
|
// It exists because the bootstrap password is shown exactly once. If it
|
||||||
|
// is lost — the boot's output rotated away, the terminal closed — the
|
||||||
|
// deployment has no other way in: there is no second account, no
|
||||||
|
// forgot-password flow, and no environment override. The only recovery
|
||||||
|
// before this command was deleting the row from webhooker.db with a
|
||||||
|
// SQLite client so the next start would re-seed.
|
||||||
|
//
|
||||||
|
// It operates on a stopped deployment only. The password is read from
|
||||||
|
// standard input or generated, never taken from argv, and it reuses the
|
||||||
|
// service's own Argon2id hashing rather than reimplementing it.
|
||||||
|
package resetpw
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"errors"
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"io/fs"
|
||||||
|
"log/slog"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gorm.io/gorm"
|
||||||
|
"sneak.berlin/go/webhooker/internal/banner"
|
||||||
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/datadir"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Name is the subcommand's name on the command line.
|
||||||
|
const Name = "resetpw"
|
||||||
|
|
||||||
|
const (
|
||||||
|
// generatedPasswordLen is the length of a -generate password. It
|
||||||
|
// is longer than the 16 characters the first boot generates: this
|
||||||
|
// one is typed or pasted once by an operator recovering a
|
||||||
|
// deployment, not carried around.
|
||||||
|
generatedPasswordLen = 24
|
||||||
|
|
||||||
|
// minPasswordLen is the shortest password accepted on standard
|
||||||
|
// input. It is a floor against a stray keystroke or a truncated
|
||||||
|
// pipe silently becoming the account's credential, not a password
|
||||||
|
// policy.
|
||||||
|
minPasswordLen = 8
|
||||||
|
)
|
||||||
|
|
||||||
|
// Exit statuses. Usage errors are distinguished from operational ones
|
||||||
|
// so that a script can tell "you called it wrong" from "it refused".
|
||||||
|
const (
|
||||||
|
exitOK = 0
|
||||||
|
exitFailure = 1
|
||||||
|
exitUsage = 2
|
||||||
|
)
|
||||||
|
|
||||||
|
// Sentinel errors, exported so a test can assert on the refusal rather
|
||||||
|
// than on the wording of a message.
|
||||||
|
var (
|
||||||
|
// ErrNoDataDir reports that DATA_DIR names nothing, or names
|
||||||
|
// something that is not a directory.
|
||||||
|
ErrNoDataDir = errors.New("no data directory")
|
||||||
|
|
||||||
|
// ErrNoDatabase reports that the data directory holds no main
|
||||||
|
// database, so there is no deployment to reset a password in.
|
||||||
|
ErrNoDatabase = errors.New("no webhooker database")
|
||||||
|
|
||||||
|
// ErrLiveInstance reports that a running webhooker holds the data
|
||||||
|
// directory.
|
||||||
|
ErrLiveInstance = errors.New(
|
||||||
|
"data directory is held by a running webhooker",
|
||||||
|
)
|
||||||
|
|
||||||
|
// ErrNoSuchUser reports that the named account does not exist.
|
||||||
|
ErrNoSuchUser = errors.New("no such user")
|
||||||
|
|
||||||
|
// ErrEmptyPassword reports that standard input carried nothing.
|
||||||
|
ErrEmptyPassword = errors.New("empty password")
|
||||||
|
|
||||||
|
// ErrPasswordTooShort reports a password below minPasswordLen.
|
||||||
|
ErrPasswordTooShort = errors.New("password too short")
|
||||||
|
|
||||||
|
// ErrNotUpdated reports that the update matched no row, which
|
||||||
|
// means the account disappeared between the lookup and the write.
|
||||||
|
ErrNotUpdated = errors.New("password was not updated")
|
||||||
|
)
|
||||||
|
|
||||||
|
// usage describes the subcommand. It is written to the same stream as
|
||||||
|
// the error that provoked it.
|
||||||
|
func usage(w io.Writer, flags *flag.FlagSet) {
|
||||||
|
_, _ = fmt.Fprintf(w, `usage: webhooker %s [-generate] <username>
|
||||||
|
|
||||||
|
Set an existing account's password. The deployment must be stopped:
|
||||||
|
webhooker %s takes the same exclusive DATA_DIR lock the server does
|
||||||
|
and refuses to run while a live instance holds it.
|
||||||
|
|
||||||
|
The password is read as one line from standard input, or generated
|
||||||
|
with -generate. It is never taken as a command-line argument, which
|
||||||
|
on Linux would publish it in /proc to every account on the host.
|
||||||
|
|
||||||
|
DATA_DIR selects the deployment exactly as it does for the server
|
||||||
|
(default %s). The directory and its database must already exist;
|
||||||
|
nothing is created.
|
||||||
|
|
||||||
|
Flags:
|
||||||
|
`, Name, Name, config.DefaultDataDir)
|
||||||
|
|
||||||
|
flags.PrintDefaults()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Run executes the subcommand and returns the process exit status.
|
||||||
|
func Run(
|
||||||
|
args []string,
|
||||||
|
stdin io.Reader,
|
||||||
|
stdout, stderr io.Writer,
|
||||||
|
) int {
|
||||||
|
flags := flag.NewFlagSet("webhooker "+Name, flag.ContinueOnError)
|
||||||
|
flags.SetOutput(stderr)
|
||||||
|
|
||||||
|
generate := flags.Bool(
|
||||||
|
"generate", false,
|
||||||
|
"generate a random password instead of reading one from "+
|
||||||
|
"standard input, and print it",
|
||||||
|
)
|
||||||
|
|
||||||
|
flags.Usage = func() { usage(stderr, flags) }
|
||||||
|
|
||||||
|
err := flags.Parse(args)
|
||||||
|
if err != nil {
|
||||||
|
// flag has already reported the error and printed the usage.
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
|
||||||
|
if flags.NArg() != 1 {
|
||||||
|
_, _ = fmt.Fprintf(
|
||||||
|
stderr,
|
||||||
|
"webhooker %s: exactly one username is required\n",
|
||||||
|
Name,
|
||||||
|
)
|
||||||
|
usage(stderr, flags)
|
||||||
|
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
|
||||||
|
err = reset(flags.Arg(0), *generate, stdin, stdout, stderr)
|
||||||
|
if err != nil {
|
||||||
|
_, _ = fmt.Fprintf(stderr, "webhooker %s: %v\n", Name, err)
|
||||||
|
|
||||||
|
return exitFailure
|
||||||
|
}
|
||||||
|
|
||||||
|
return exitOK
|
||||||
|
}
|
||||||
|
|
||||||
|
// reset performs the whole operation against the configured data
|
||||||
|
// directory.
|
||||||
|
func reset(
|
||||||
|
username string,
|
||||||
|
generate bool,
|
||||||
|
stdin io.Reader,
|
||||||
|
stdout, stderr io.Writer,
|
||||||
|
) error {
|
||||||
|
dir := config.DataDir()
|
||||||
|
|
||||||
|
err := checkDataDir(dir)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
lock, err := acquire(dir)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// The kernel drops the lock when this process exits, whatever
|
||||||
|
// happens below; releasing explicitly is what makes a long-running
|
||||||
|
// caller — a test — see it freed. A release error tells the
|
||||||
|
// operator nothing they can act on.
|
||||||
|
defer func() { _ = lock.Release() }()
|
||||||
|
|
||||||
|
db, err := database.Open(dir, cliLogger(stderr))
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
password, err := setPassword(db, username, generate, stdin, stderr)
|
||||||
|
|
||||||
|
closeErr := db.Close()
|
||||||
|
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
if closeErr != nil {
|
||||||
|
return fmt.Errorf("closing the database: %w", closeErr)
|
||||||
|
}
|
||||||
|
|
||||||
|
return report(stdout, stderr, dir, username, password, generate)
|
||||||
|
}
|
||||||
|
|
||||||
|
// report tells the operator what happened.
|
||||||
|
//
|
||||||
|
// A generated password is printed in the same banner the first boot
|
||||||
|
// uses, because this is the only time it is ever shown. A password the
|
||||||
|
// operator supplied is not echoed back: they already have it, and
|
||||||
|
// writing it to standard output a second time would put it in another
|
||||||
|
// log for no gain.
|
||||||
|
func report(
|
||||||
|
stdout, stderr io.Writer,
|
||||||
|
dir, username, password string,
|
||||||
|
generate bool,
|
||||||
|
) error {
|
||||||
|
if generate {
|
||||||
|
write := func(w io.Writer) error {
|
||||||
|
return banner.Credentials(
|
||||||
|
w,
|
||||||
|
"WEBHOOKER PASSWORD RESET: this account's new "+
|
||||||
|
"password is",
|
||||||
|
username,
|
||||||
|
password,
|
||||||
|
"Save this password now: it is shown only here.\n"+
|
||||||
|
"The database stores only its Argon2id hash.",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
err := write(stdout)
|
||||||
|
if err == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// The password is already stored. Standard output failing
|
||||||
|
// here is the difference between a recovered deployment and
|
||||||
|
// one locked out behind a password nobody has ever seen, so
|
||||||
|
// try the other stream before giving up.
|
||||||
|
if write(stderr) == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
_, err := fmt.Fprintf(
|
||||||
|
stdout,
|
||||||
|
"password updated for user %q in %s\n", username, dir,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("writing the result: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// acquire takes the DATA_DIR lock, translating the contended case into
|
||||||
|
// the refusal this command owes the operator.
|
||||||
|
//
|
||||||
|
// Resetting a password underneath a live instance would not corrupt
|
||||||
|
// anything, but the running process keeps serving every session that
|
||||||
|
// authenticated with the old one, so the operator would be told the
|
||||||
|
// password changed while the deployment still behaved as though it had
|
||||||
|
// not. Refusing is also what the lock is for.
|
||||||
|
func acquire(dir string) (*datadir.Lock, error) {
|
||||||
|
lock, err := datadir.Acquire(dir)
|
||||||
|
if err == nil {
|
||||||
|
return lock, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
if errors.Is(err, datadir.ErrLocked) {
|
||||||
|
return nil, fmt.Errorf(
|
||||||
|
"%w: %s. Stop it and run this again: a password reset "+
|
||||||
|
"does not reach a running process, whose existing "+
|
||||||
|
"sessions stay authenticated",
|
||||||
|
ErrLiveInstance, dir,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// checkDataDir refuses to act on a path that does not already hold a
|
||||||
|
// deployment.
|
||||||
|
//
|
||||||
|
// This runs before datadir.Acquire on purpose. Acquire calls
|
||||||
|
// os.MkdirAll, so a mistyped DATA_DIR would otherwise be built out —
|
||||||
|
// the directory tree, the lock file, and then an empty migrated
|
||||||
|
// database — and the command would report success against a deployment
|
||||||
|
// that does not exist while the real one stayed locked out. Nothing
|
||||||
|
// here creates anything.
|
||||||
|
func checkDataDir(dir string) error {
|
||||||
|
info, err := os.Stat(dir)
|
||||||
|
|
||||||
|
switch {
|
||||||
|
case errors.Is(err, fs.ErrNotExist):
|
||||||
|
return fmt.Errorf(
|
||||||
|
"%w: %s (DATA_DIR). This acts on an existing "+
|
||||||
|
"deployment and creates nothing",
|
||||||
|
ErrNoDataDir, dir,
|
||||||
|
)
|
||||||
|
case err != nil:
|
||||||
|
return fmt.Errorf("checking data directory %s: %w", dir, err)
|
||||||
|
case !info.IsDir():
|
||||||
|
return fmt.Errorf(
|
||||||
|
"%w: %s (DATA_DIR) is not a directory", ErrNoDataDir, dir,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
dbPath := filepath.Join(dir, database.MainDBFileName)
|
||||||
|
|
||||||
|
_, err = os.Stat(dbPath)
|
||||||
|
|
||||||
|
switch {
|
||||||
|
case errors.Is(err, fs.ErrNotExist):
|
||||||
|
return fmt.Errorf(
|
||||||
|
"%w: %s does not exist. The admin account is created by "+
|
||||||
|
"the first server start",
|
||||||
|
ErrNoDatabase, dbPath,
|
||||||
|
)
|
||||||
|
case err != nil:
|
||||||
|
return fmt.Errorf("checking %s: %w", dbPath, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// setPassword looks the account up, obtains the new password, and
|
||||||
|
// writes its hash.
|
||||||
|
//
|
||||||
|
// The order matters: the account is resolved before an operator is
|
||||||
|
// asked to type anything, and the hash is computed in full before the
|
||||||
|
// single UPDATE that stores it. A failure at any step therefore leaves
|
||||||
|
// the stored credential exactly as it was — there is no half-written
|
||||||
|
// state to recover from.
|
||||||
|
func setPassword(
|
||||||
|
db *database.Database,
|
||||||
|
username string,
|
||||||
|
generate bool,
|
||||||
|
stdin io.Reader,
|
||||||
|
stderr io.Writer,
|
||||||
|
) (string, error) {
|
||||||
|
var user database.User
|
||||||
|
|
||||||
|
err := db.DB().Where("username = ?", username).First(&user).Error
|
||||||
|
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||||
|
return "", fmt.Errorf(
|
||||||
|
"%w: %q. This changes an existing account's password "+
|
||||||
|
"and never creates an account",
|
||||||
|
ErrNoSuchUser, username,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("looking up user %q: %w", username, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
password, err := newPassword(generate, stdin, stderr)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
|
||||||
|
hash, err := database.HashPassword(password)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("hashing the new password: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
result := db.DB().Model(&database.User{}).
|
||||||
|
Where("id = ?", user.ID).
|
||||||
|
Update("password", hash)
|
||||||
|
if result.Error != nil {
|
||||||
|
return "", fmt.Errorf(
|
||||||
|
"updating user %q: %w", username, result.Error,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if result.RowsAffected != 1 {
|
||||||
|
return "", fmt.Errorf(
|
||||||
|
"%w: %q matched %d rows",
|
||||||
|
ErrNotUpdated, username, result.RowsAffected,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
return password, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// newPassword returns the password to store: generated, or read from
|
||||||
|
// standard input.
|
||||||
|
func newPassword(
|
||||||
|
generate bool,
|
||||||
|
stdin io.Reader,
|
||||||
|
stderr io.Writer,
|
||||||
|
) (string, error) {
|
||||||
|
if generate {
|
||||||
|
password, err := database.GenerateRandomPassword(
|
||||||
|
generatedPasswordLen,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("generating a password: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return password, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return readPassword(stdin, stderr)
|
||||||
|
}
|
||||||
|
|
||||||
|
// readPassword reads the new password as one line from standard input,
|
||||||
|
// minus its line ending.
|
||||||
|
//
|
||||||
|
// Standard input rather than an argument: on Linux argv is readable
|
||||||
|
// through /proc by every account on the host for as long as the process
|
||||||
|
// lives, and a password typed as an argument lands in shell history
|
||||||
|
// besides.
|
||||||
|
//
|
||||||
|
// When standard input is a terminal the input is echoed — no attempt is
|
||||||
|
// made to put the terminal into no-echo mode — so the prompt says so
|
||||||
|
// rather than letting an operator assume otherwise.
|
||||||
|
func readPassword(stdin io.Reader, stderr io.Writer) (string, error) {
|
||||||
|
if f, ok := stdin.(*os.File); ok && isTerminal(f) {
|
||||||
|
_, _ = fmt.Fprintf(
|
||||||
|
stderr,
|
||||||
|
"New password (echoed as you type), then Enter: ",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
line, err := bufio.NewReader(stdin).ReadString('\n')
|
||||||
|
if err != nil && !errors.Is(err, io.EOF) {
|
||||||
|
return "", fmt.Errorf(
|
||||||
|
"reading the password from standard input: %w", err,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
password := strings.TrimRight(line, "\r\n")
|
||||||
|
|
||||||
|
if password == "" {
|
||||||
|
return "", fmt.Errorf(
|
||||||
|
"%w: standard input carried no password. Pipe one in, "+
|
||||||
|
"or pass -generate",
|
||||||
|
ErrEmptyPassword,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(password) < minPasswordLen {
|
||||||
|
return "", fmt.Errorf(
|
||||||
|
"%w: %d bytes, minimum %d",
|
||||||
|
ErrPasswordTooShort, len(password), minPasswordLen,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
return password, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// isTerminal reports whether f is a character device, which is as much
|
||||||
|
// as this needs to know to decide whether to prompt.
|
||||||
|
func isTerminal(f *os.File) bool {
|
||||||
|
info, err := f.Stat()
|
||||||
|
if err != nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
return info.Mode()&os.ModeCharDevice != 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// cliLogger builds the logger the database layer writes through while
|
||||||
|
// this command runs. It is deliberately quiet: connecting and migrating
|
||||||
|
// are steps the operator did not ask about, and the one thing they need
|
||||||
|
// to see is the outcome on standard output.
|
||||||
|
func cliLogger(stderr io.Writer) *slog.Logger {
|
||||||
|
return slog.New(slog.NewTextHandler(stderr, &slog.HandlerOptions{
|
||||||
|
Level: slog.LevelWarn,
|
||||||
|
}))
|
||||||
|
}
|
||||||
444
internal/resetpw/resetpw_test.go
Normal file
444
internal/resetpw/resetpw_test.go
Normal file
@@ -0,0 +1,444 @@
|
|||||||
|
package resetpw_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"log/slog"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/url"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"go.uber.org/fx"
|
||||||
|
"go.uber.org/fx/fxtest"
|
||||||
|
"sneak.berlin/go/webhooker/internal/config"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/datadir"
|
||||||
|
"sneak.berlin/go/webhooker/internal/delivery"
|
||||||
|
"sneak.berlin/go/webhooker/internal/globals"
|
||||||
|
"sneak.berlin/go/webhooker/internal/handlers"
|
||||||
|
"sneak.berlin/go/webhooker/internal/healthcheck"
|
||||||
|
"sneak.berlin/go/webhooker/internal/logger"
|
||||||
|
"sneak.berlin/go/webhooker/internal/middleware"
|
||||||
|
"sneak.berlin/go/webhooker/internal/resetpw"
|
||||||
|
"sneak.berlin/go/webhooker/internal/session"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
// operatorUser is the account these tests recover.
|
||||||
|
operatorUser = "admin"
|
||||||
|
|
||||||
|
// newPassword is what the operator sets it to.
|
||||||
|
newPassword = "correct horse battery staple"
|
||||||
|
|
||||||
|
// placeholderHash stands in for the stored credential nobody
|
||||||
|
// knows any more — the lost bootstrap password. Nothing here
|
||||||
|
// verifies against it; what matters is whether it is still there
|
||||||
|
// after a refusal, or replaced after a reset.
|
||||||
|
placeholderHash = "$argon2id$lost"
|
||||||
|
|
||||||
|
// exitOK and exitFailure are the statuses Run returns.
|
||||||
|
exitOK = 0
|
||||||
|
exitFailure = 1
|
||||||
|
)
|
||||||
|
|
||||||
|
// testLogger is quiet unless something goes wrong.
|
||||||
|
func testLogger() *slog.Logger {
|
||||||
|
return slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{
|
||||||
|
Level: slog.LevelWarn,
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
|
||||||
|
// newDeployment builds a data directory holding a migrated database
|
||||||
|
// with one account whose password is unknown, and points DATA_DIR at
|
||||||
|
// it. It deliberately does not boot the server graph: seeding through
|
||||||
|
// it would spend an Argon2id hash on a password no test can use.
|
||||||
|
func newDeployment(t *testing.T) string {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
dir := t.TempDir()
|
||||||
|
t.Setenv("DATA_DIR", dir)
|
||||||
|
|
||||||
|
db, err := database.Open(dir, testLogger())
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
require.NoError(t, db.DB().Create(&database.User{
|
||||||
|
Username: operatorUser,
|
||||||
|
Password: placeholderHash,
|
||||||
|
}).Error)
|
||||||
|
|
||||||
|
require.NoError(t, db.Close())
|
||||||
|
|
||||||
|
return dir
|
||||||
|
}
|
||||||
|
|
||||||
|
// storedHash reads the account's stored credential back.
|
||||||
|
func storedHash(t *testing.T, dir string) string {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
db, err := database.Open(dir, testLogger())
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
defer func() { require.NoError(t, db.Close()) }()
|
||||||
|
|
||||||
|
var user database.User
|
||||||
|
|
||||||
|
require.NoError(t, db.DB().
|
||||||
|
Where("username = ?", operatorUser).
|
||||||
|
First(&user).Error)
|
||||||
|
|
||||||
|
return user.Password
|
||||||
|
}
|
||||||
|
|
||||||
|
// bannerPassword returns the plaintext a credentials banner printed.
|
||||||
|
func bannerPassword(t *testing.T, out string) string {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
for line := range strings.SplitSeq(out, "\n") {
|
||||||
|
_, value, found := strings.Cut(line, "password: ")
|
||||||
|
if found {
|
||||||
|
return strings.TrimSpace(value)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Fatalf("no password line in:\n%s", out)
|
||||||
|
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// run drives the subcommand with the given standard input and returns
|
||||||
|
// its status alongside what it wrote.
|
||||||
|
func run(
|
||||||
|
t *testing.T, stdin string, args ...string,
|
||||||
|
) (int, string, string) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var stdout, stderr bytes.Buffer
|
||||||
|
|
||||||
|
code := resetpw.Run(
|
||||||
|
args, strings.NewReader(stdin), &stdout, &stderr,
|
||||||
|
)
|
||||||
|
|
||||||
|
return code, stdout.String(), stderr.String()
|
||||||
|
}
|
||||||
|
|
||||||
|
type noopNotifier struct{}
|
||||||
|
|
||||||
|
func (n *noopNotifier) Notify([]delivery.Task) {}
|
||||||
|
|
||||||
|
type noopEvictor struct{}
|
||||||
|
|
||||||
|
func (n *noopEvictor) EvictWebhook(string) {}
|
||||||
|
|
||||||
|
// newServerApp starts the real login path against dir: the handlers,
|
||||||
|
// the middleware that bounds password verification, the session store
|
||||||
|
// and the database, exactly as internal/handlers builds them.
|
||||||
|
//
|
||||||
|
// One application per test function, not per case: every start that
|
||||||
|
// finds no account seeds one at 64 MB of Argon2id, and this package's
|
||||||
|
// budget is not the place to spend that repeatedly.
|
||||||
|
func newServerApp(
|
||||||
|
t *testing.T, dir string,
|
||||||
|
) (*handlers.Handlers, *fxtest.App) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var h *handlers.Handlers
|
||||||
|
|
||||||
|
app := fxtest.New(
|
||||||
|
t,
|
||||||
|
fx.Provide(
|
||||||
|
globals.New,
|
||||||
|
logger.New,
|
||||||
|
func() *config.Config {
|
||||||
|
return &config.Config{DataDir: dir}
|
||||||
|
},
|
||||||
|
database.New,
|
||||||
|
database.NewWebhookDBManager,
|
||||||
|
healthcheck.New,
|
||||||
|
session.New,
|
||||||
|
func() delivery.Notifier { return &noopNotifier{} },
|
||||||
|
func() delivery.WebhookEvictor { return &noopEvictor{} },
|
||||||
|
middleware.New,
|
||||||
|
delivery.NewGuard,
|
||||||
|
handlers.New,
|
||||||
|
),
|
||||||
|
fx.Populate(&h),
|
||||||
|
)
|
||||||
|
|
||||||
|
app.RequireStart()
|
||||||
|
|
||||||
|
return h, app
|
||||||
|
}
|
||||||
|
|
||||||
|
// submitLogin drives one login form POST through the real handler.
|
||||||
|
func submitLogin(
|
||||||
|
h *handlers.Handlers, username, password string,
|
||||||
|
) *httptest.ResponseRecorder {
|
||||||
|
form := url.Values{}
|
||||||
|
form.Set("username", username)
|
||||||
|
form.Set("password", password)
|
||||||
|
|
||||||
|
req := httptest.NewRequestWithContext(
|
||||||
|
context.Background(),
|
||||||
|
http.MethodPost,
|
||||||
|
"/pages/login",
|
||||||
|
strings.NewReader(form.Encode()),
|
||||||
|
)
|
||||||
|
req.Header.Set(
|
||||||
|
"Content-Type", "application/x-www-form-urlencoded",
|
||||||
|
)
|
||||||
|
req.RemoteAddr = "10.0.0.1:44444"
|
||||||
|
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
h.HandleLoginSubmit().ServeHTTP(w, req)
|
||||||
|
|
||||||
|
return w
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestResetThenLogin is the definition of done of
|
||||||
|
// https://git.eeqj.de/sneak/webhooker/issues/208: an operator who lost
|
||||||
|
// the one-time bootstrap password sets a new one from the command line
|
||||||
|
// and logs in with it.
|
||||||
|
//
|
||||||
|
// The login is the real one — the form POST through
|
||||||
|
// handlers.HandleLoginSubmit, which looks the account up and verifies
|
||||||
|
// the stored Argon2id hash — so a reset that wrote a hash the login
|
||||||
|
// path cannot verify fails here rather than passing a re-implementation
|
||||||
|
// of the check.
|
||||||
|
//
|
||||||
|
//nolint:paralleltest // newDeployment sets DATA_DIR with t.Setenv.
|
||||||
|
func TestResetThenLogin(t *testing.T) {
|
||||||
|
dir := newDeployment(t)
|
||||||
|
|
||||||
|
code, stdout, stderr := run(t, newPassword+"\n", operatorUser)
|
||||||
|
require.Equal(t, exitOK, code, "stderr: %s", stderr)
|
||||||
|
|
||||||
|
assert.NotContains(
|
||||||
|
t, stdout, newPassword,
|
||||||
|
"a password the operator supplied must not be echoed back",
|
||||||
|
)
|
||||||
|
assert.Contains(t, stdout, operatorUser)
|
||||||
|
|
||||||
|
h, app := newServerApp(t, dir)
|
||||||
|
|
||||||
|
defer app.RequireStop()
|
||||||
|
|
||||||
|
got := submitLogin(h, operatorUser, newPassword)
|
||||||
|
require.Equal(
|
||||||
|
t, http.StatusSeeOther, got.Code,
|
||||||
|
"the new password must log in",
|
||||||
|
)
|
||||||
|
|
||||||
|
got = submitLogin(h, operatorUser, "not-"+newPassword)
|
||||||
|
require.NotEqual(
|
||||||
|
t, http.StatusSeeOther, got.Code,
|
||||||
|
"the reset must not make every password work",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestGeneratedPasswordIsPrintedAndWorks covers -generate, the mode an
|
||||||
|
// operator recovering a deployment actually reaches for. The generated
|
||||||
|
// password is shown once, in the banner, and must be the one stored.
|
||||||
|
//
|
||||||
|
//nolint:paralleltest // newDeployment sets DATA_DIR with t.Setenv.
|
||||||
|
func TestGeneratedPasswordIsPrintedAndWorks(t *testing.T) {
|
||||||
|
dir := newDeployment(t)
|
||||||
|
|
||||||
|
code, stdout, stderr := run(t, "", "-generate", operatorUser)
|
||||||
|
require.Equal(t, exitOK, code, "stderr: %s", stderr)
|
||||||
|
|
||||||
|
require.Contains(
|
||||||
|
t, stdout, strings.Repeat("=", 20),
|
||||||
|
"a generated password must be printed as a banner",
|
||||||
|
)
|
||||||
|
|
||||||
|
password := bannerPassword(t, stdout)
|
||||||
|
|
||||||
|
ok, err := database.VerifyPassword(password, storedHash(t, dir))
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.True(
|
||||||
|
t, ok, "the printed password must open the account",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// failingWriter is a standard output that cannot be written to.
|
||||||
|
type failingWriter struct{}
|
||||||
|
|
||||||
|
func (failingWriter) Write([]byte) (int, error) {
|
||||||
|
return 0, assert.AnError
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestGeneratedPasswordSurvivesAFailedStdout covers the one outcome
|
||||||
|
// worse than an error: the password is already stored, so a banner
|
||||||
|
// that cannot be written to standard output must go to standard error
|
||||||
|
// rather than leaving the deployment behind a password nobody has seen.
|
||||||
|
//
|
||||||
|
//nolint:paralleltest // newDeployment sets DATA_DIR with t.Setenv.
|
||||||
|
func TestGeneratedPasswordSurvivesAFailedStdout(t *testing.T) {
|
||||||
|
dir := newDeployment(t)
|
||||||
|
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
|
||||||
|
code := resetpw.Run(
|
||||||
|
[]string{"-generate", operatorUser},
|
||||||
|
strings.NewReader(""), failingWriter{}, &stderr,
|
||||||
|
)
|
||||||
|
|
||||||
|
require.Equal(t, exitOK, code)
|
||||||
|
|
||||||
|
password := bannerPassword(t, stderr.String())
|
||||||
|
|
||||||
|
ok, err := database.VerifyPassword(password, storedHash(t, dir))
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.True(t, ok)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestRefusesLiveInstance pins the refusal the issue requires. The
|
||||||
|
// running deployment keeps serving the sessions that authenticated
|
||||||
|
// with the old password, so a reset underneath it would report a
|
||||||
|
// change the service does not honour.
|
||||||
|
//
|
||||||
|
//nolint:paralleltest // newDeployment sets DATA_DIR with t.Setenv.
|
||||||
|
func TestRefusesLiveInstance(t *testing.T) {
|
||||||
|
dir := newDeployment(t)
|
||||||
|
|
||||||
|
lock, err := datadir.Acquire(dir)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
defer func() { require.NoError(t, lock.Release()) }()
|
||||||
|
|
||||||
|
code, _, stderr := run(t, newPassword+"\n", operatorUser)
|
||||||
|
|
||||||
|
require.Equal(t, exitFailure, code)
|
||||||
|
assert.Contains(t, stderr, dir, "the refusal must name DATA_DIR")
|
||||||
|
assert.Contains(t, stderr, "running webhooker")
|
||||||
|
assert.Equal(
|
||||||
|
t, placeholderHash, storedHash(t, dir),
|
||||||
|
"a refused reset must not touch the stored credential",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestMissingDataDirCreatesNothing pins the side effect that must not
|
||||||
|
// happen. datadir.Acquire calls os.MkdirAll, so reaching it with a
|
||||||
|
// mistyped DATA_DIR would build the directory, take a lock in it and
|
||||||
|
// migrate an empty database there — reporting success against a
|
||||||
|
// deployment that does not exist.
|
||||||
|
func TestMissingDataDirCreatesNothing(t *testing.T) {
|
||||||
|
dir := filepath.Join(t.TempDir(), "typo", "webhooker")
|
||||||
|
t.Setenv("DATA_DIR", dir)
|
||||||
|
|
||||||
|
code, _, stderr := run(t, newPassword+"\n", operatorUser)
|
||||||
|
|
||||||
|
require.Equal(t, exitFailure, code)
|
||||||
|
assert.Contains(t, stderr, dir)
|
||||||
|
|
||||||
|
_, err := os.Stat(dir)
|
||||||
|
assert.ErrorIs(
|
||||||
|
t, err, os.ErrNotExist,
|
||||||
|
"a mistyped DATA_DIR must not be created",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestMissingDatabaseCreatesNothing covers the directory that exists
|
||||||
|
// but holds no deployment: an empty volume, or the wrong one. Nothing
|
||||||
|
// may be written there either, lock file included.
|
||||||
|
func TestMissingDatabaseCreatesNothing(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
t.Setenv("DATA_DIR", dir)
|
||||||
|
|
||||||
|
code, _, stderr := run(t, newPassword+"\n", operatorUser)
|
||||||
|
|
||||||
|
require.Equal(t, exitFailure, code)
|
||||||
|
assert.Contains(t, stderr, database.MainDBFileName)
|
||||||
|
|
||||||
|
entries, err := os.ReadDir(dir)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Empty(
|
||||||
|
t, entries,
|
||||||
|
"nothing may be created in a directory holding no database",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestUnknownUserFails states the decision: resetpw changes an
|
||||||
|
// existing account's password and never creates an account. A typo in
|
||||||
|
// the username must say so rather than quietly adding a second user.
|
||||||
|
//
|
||||||
|
//nolint:paralleltest // newDeployment sets DATA_DIR with t.Setenv.
|
||||||
|
func TestUnknownUserFails(t *testing.T) {
|
||||||
|
dir := newDeployment(t)
|
||||||
|
|
||||||
|
code, _, stderr := run(t, newPassword+"\n", "amdin")
|
||||||
|
|
||||||
|
require.Equal(t, exitFailure, code)
|
||||||
|
assert.Contains(t, stderr, "amdin")
|
||||||
|
assert.Equal(t, placeholderHash, storedHash(t, dir))
|
||||||
|
|
||||||
|
var count int64
|
||||||
|
|
||||||
|
db, err := database.Open(dir, testLogger())
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
defer func() { require.NoError(t, db.Close()) }()
|
||||||
|
|
||||||
|
require.NoError(t, db.DB().Model(&database.User{}).
|
||||||
|
Count(&count).Error)
|
||||||
|
assert.EqualValues(
|
||||||
|
t, 1, count, "no account may have been created",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestRejectsUnusablePasswords covers what standard input can carry by
|
||||||
|
// accident: nothing at all, and a stray keystroke. Either would
|
||||||
|
// otherwise become the account's only credential.
|
||||||
|
//
|
||||||
|
//nolint:paralleltest // newDeployment sets DATA_DIR with t.Setenv.
|
||||||
|
func TestRejectsUnusablePasswords(t *testing.T) {
|
||||||
|
dir := newDeployment(t)
|
||||||
|
|
||||||
|
for name, stdin := range map[string]string{
|
||||||
|
"empty": "",
|
||||||
|
"newline": "\n",
|
||||||
|
"short": "hunter2\n",
|
||||||
|
} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
code, _, stderr := run(t, stdin, operatorUser)
|
||||||
|
|
||||||
|
require.Equal(t, exitFailure, code)
|
||||||
|
assert.NotEmpty(t, stderr)
|
||||||
|
assert.Equal(
|
||||||
|
t, placeholderHash, storedHash(t, dir),
|
||||||
|
"a rejected password must not be stored",
|
||||||
|
)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestUsageErrors pins the statuses a caller can script against: 2 for
|
||||||
|
// being called wrong, which is not the same as a refusal.
|
||||||
|
func TestUsageErrors(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for name, args := range map[string][]string{
|
||||||
|
"no username": {},
|
||||||
|
"two usernames": {"admin", "root"},
|
||||||
|
"unknown flag": {"-force", "admin"},
|
||||||
|
} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
var stdout, stderr bytes.Buffer
|
||||||
|
|
||||||
|
code := resetpw.Run(
|
||||||
|
args, strings.NewReader(""), &stdout, &stderr,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.Equal(t, 2, code)
|
||||||
|
assert.NotEmpty(t, stderr)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -201,6 +201,18 @@ func (s *Server) setupSourceRoutes() {
|
|||||||
"/logs/{eventID}/body",
|
"/logs/{eventID}/body",
|
||||||
s.h.HandleEventBodyDownload(),
|
s.h.HandleEventBodyDownload(),
|
||||||
)
|
)
|
||||||
|
// Replay is the one page action that queues outbound work:
|
||||||
|
// it creates a delivery from a stored event and hands it to
|
||||||
|
// the delivery engine. The rate limit is what bounds a
|
||||||
|
// held-down button or a scripted loop; the handler
|
||||||
|
// separately refuses a replay while an earlier one for the
|
||||||
|
// same event and target is still in flight. POST only, so
|
||||||
|
// the action cannot be taken by a link, a prefetch or an
|
||||||
|
// image tag.
|
||||||
|
r.With(s.mw.ReplayRateLimit()).Post(
|
||||||
|
"/deliveries/{deliveryID}/replay",
|
||||||
|
s.h.HandleDeliveryReplay(),
|
||||||
|
)
|
||||||
r.Post(
|
r.Post(
|
||||||
"/entrypoints",
|
"/entrypoints",
|
||||||
s.h.HandleEntrypointCreate(),
|
s.h.HandleEntrypointCreate(),
|
||||||
@@ -213,6 +225,10 @@ func (s *Server) setupSourceRoutes() {
|
|||||||
"/entrypoints/{entrypointID}/toggle",
|
"/entrypoints/{entrypointID}/toggle",
|
||||||
s.h.HandleEntrypointToggle(),
|
s.h.HandleEntrypointToggle(),
|
||||||
)
|
)
|
||||||
|
r.Post(
|
||||||
|
"/entrypoints/{entrypointID}/secret",
|
||||||
|
s.h.HandleEntrypointSecret(),
|
||||||
|
)
|
||||||
r.Post("/targets", s.h.HandleTargetCreate())
|
r.Post("/targets", s.h.HandleTargetCreate())
|
||||||
// The edit form is the one page that renders a target's
|
// The edit form is the one page that renders a target's
|
||||||
// destination URL and header values in full; see
|
// destination URL and header values in full; see
|
||||||
|
|||||||
@@ -311,6 +311,75 @@ func (e *testEnv) seedEvent(
|
|||||||
return event
|
return event
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// seedTarget creates an active HTTP target for a webhook.
|
||||||
|
func (e *testEnv) seedTarget(
|
||||||
|
t *testing.T,
|
||||||
|
webhookID string,
|
||||||
|
) *database.Target {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
tgt := &database.Target{
|
||||||
|
WebhookID: webhookID,
|
||||||
|
Name: "routed-target",
|
||||||
|
Type: database.TargetTypeHTTP,
|
||||||
|
Active: true,
|
||||||
|
Config: `{"url":"http://93.184.216.34/hook"}`,
|
||||||
|
}
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t,
|
||||||
|
e.db.DB().Omit(clause.Associations).Create(tgt).Error,
|
||||||
|
)
|
||||||
|
|
||||||
|
return tgt
|
||||||
|
}
|
||||||
|
|
||||||
|
// seedFailedDelivery records a terminally failed delivery of an event
|
||||||
|
// to a target in the webhook's own database.
|
||||||
|
func (e *testEnv) seedFailedDelivery(
|
||||||
|
t *testing.T,
|
||||||
|
webhookID, eventID, targetID string,
|
||||||
|
) *database.Delivery {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
webhookDB, err := e.dbMgr.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
dlv := &database.Delivery{
|
||||||
|
EventID: eventID,
|
||||||
|
TargetID: targetID,
|
||||||
|
Status: database.DeliveryStatusFailed,
|
||||||
|
}
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t,
|
||||||
|
webhookDB.Omit(clause.Associations).Create(dlv).Error,
|
||||||
|
)
|
||||||
|
|
||||||
|
return dlv
|
||||||
|
}
|
||||||
|
|
||||||
|
// countDeliveries reports how many deliveries a webhook's database
|
||||||
|
// holds.
|
||||||
|
func (e *testEnv) countDeliveries(
|
||||||
|
t *testing.T, webhookID string,
|
||||||
|
) int64 {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
webhookDB, err := e.dbMgr.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
var count int64
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t,
|
||||||
|
webhookDB.Model(&database.Delivery{}).
|
||||||
|
Count(&count).Error,
|
||||||
|
)
|
||||||
|
|
||||||
|
return count
|
||||||
|
}
|
||||||
|
|
||||||
// storedHash reads the current password hash for a username.
|
// storedHash reads the current password hash for a username.
|
||||||
func (e *testEnv) storedHash(t *testing.T, username string) string {
|
func (e *testEnv) storedHash(t *testing.T, username string) string {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
@@ -675,6 +744,85 @@ func TestSourceLogsBody_OtherUser404s(t *testing.T) {
|
|||||||
assert.Equal(t, "/pages/login", anon.Header().Get("Location"))
|
assert.Equal(t, "/pages/login", anon.Header().Get("Location"))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestDeliveryReplay_PostOnlyAndCSRFProtected walks the replay action
|
||||||
|
// through the production router rather than a forged route context,
|
||||||
|
// which is the only way to prove what the route group actually gives
|
||||||
|
// it: a GET cannot trigger a replay, an unauthenticated request never
|
||||||
|
// reaches the handler, a POST without the token is refused by CSRF,
|
||||||
|
// and the form the template emits — token and action URL both — works
|
||||||
|
// as rendered.
|
||||||
|
func TestDeliveryReplay_PostOnlyAndCSRFProtected(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
env := newTestEnv(t)
|
||||||
|
|
||||||
|
userID, _ := env.seedUser(t, "replayer", "somepassword")
|
||||||
|
cookies := env.authCookies(t, userID, "replayer")
|
||||||
|
|
||||||
|
wh := env.seedWebhook(t, userID)
|
||||||
|
tgt := env.seedTarget(t, wh.ID)
|
||||||
|
evt := env.seedEvent(t, wh.ID, `{"replay":"me"}`)
|
||||||
|
dlv := env.seedFailedDelivery(t, wh.ID, evt.ID, tgt.ID)
|
||||||
|
|
||||||
|
path := "/source/" + wh.ID + "/deliveries/" + dlv.ID +
|
||||||
|
"/replay"
|
||||||
|
|
||||||
|
assert.Equal(
|
||||||
|
t, http.StatusMethodNotAllowed,
|
||||||
|
env.get(path, cookies).Code,
|
||||||
|
"a replay must not be reachable by GET",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.Equal(
|
||||||
|
t, http.StatusForbidden,
|
||||||
|
env.post(path, url.Values{}, cookies).Code,
|
||||||
|
"a replay POST without a CSRF token must be refused",
|
||||||
|
)
|
||||||
|
|
||||||
|
anon := env.post(path, url.Values{}, nil)
|
||||||
|
assert.Equal(t, http.StatusForbidden, anon.Code)
|
||||||
|
|
||||||
|
require.Equal(
|
||||||
|
t, int64(1), env.countDeliveries(t, wh.ID),
|
||||||
|
"no refused request may have created a delivery",
|
||||||
|
)
|
||||||
|
|
||||||
|
// The token and the action URL both come out of the rendered
|
||||||
|
// page, so a typo in either the route pattern or the template
|
||||||
|
// fails here.
|
||||||
|
logsPath := "/source/" + wh.ID + "/logs"
|
||||||
|
|
||||||
|
token, cookies := env.csrfFrom(t, logsPath, cookies)
|
||||||
|
|
||||||
|
page := env.get(logsPath, cookies)
|
||||||
|
require.Equal(t, http.StatusOK, page.Code)
|
||||||
|
|
||||||
|
action := regexp.MustCompile(
|
||||||
|
`action="(/source/[^"]+/replay)"`,
|
||||||
|
).FindStringSubmatch(page.Body.String())
|
||||||
|
require.Len(
|
||||||
|
t, action, 2,
|
||||||
|
"a finished delivery should render a replay form",
|
||||||
|
)
|
||||||
|
|
||||||
|
form := url.Values{}
|
||||||
|
form.Set("csrf_token", token)
|
||||||
|
|
||||||
|
w := env.post(
|
||||||
|
html.UnescapeString(action[1]), form, cookies,
|
||||||
|
)
|
||||||
|
|
||||||
|
require.Equal(t, http.StatusSeeOther, w.Code)
|
||||||
|
assert.Equal(
|
||||||
|
t, logsPath+"?replay=queued",
|
||||||
|
w.Header().Get("Location"),
|
||||||
|
)
|
||||||
|
assert.Equal(
|
||||||
|
t, int64(2), env.countDeliveries(t, wh.ID),
|
||||||
|
"the replay appends a delivery",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
// metricsConfig is a Config differing from the routing default only
|
// metricsConfig is a Config differing from the routing default only
|
||||||
// in the two /metrics credentials.
|
// in the two /metrics credentials.
|
||||||
func metricsConfig(
|
func metricsConfig(
|
||||||
|
|||||||
283
internal/signature/signature.go
Normal file
283
internal/signature/signature.go
Normal file
@@ -0,0 +1,283 @@
|
|||||||
|
// Package signature verifies that an inbound webhook request really
|
||||||
|
// came from the sender an entrypoint was configured for.
|
||||||
|
//
|
||||||
|
// Verification is optional and per entrypoint. An entrypoint with no
|
||||||
|
// scheme configured is not verified at all, which is what every
|
||||||
|
// entrypoint was before this package existed. An entrypoint whose
|
||||||
|
// configuration is present but incoherent is failed closed, never
|
||||||
|
// treated as unverified: the whole point of the feature is that
|
||||||
|
// turning it on cannot silently turn itself back off.
|
||||||
|
package signature
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/hmac"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Header names each supported scheme reads its signature from.
|
||||||
|
const (
|
||||||
|
// HeaderGitHub is GitHub's HMAC-SHA256 signature header. GitHub
|
||||||
|
// also sends the older SHA-1 X-Hub-Signature; it is not accepted.
|
||||||
|
HeaderGitHub = "X-Hub-Signature-256"
|
||||||
|
|
||||||
|
// HeaderGitLab is GitLab's plain shared-token header.
|
||||||
|
HeaderGitLab = "X-Gitlab-Token"
|
||||||
|
)
|
||||||
|
|
||||||
|
// githubPrefix is the algorithm label GitHub puts in front of the hex
|
||||||
|
// digest. It is required, not optional: accepting a bare digest too
|
||||||
|
// would mean accepting a spelling no supported sender produces.
|
||||||
|
const githubPrefix = "sha256="
|
||||||
|
|
||||||
|
// ErrConfig marks a failure caused by the entrypoint's stored
|
||||||
|
// configuration rather than by the request. A caller must fail these
|
||||||
|
// closed — refuse the request — because the alternative is an
|
||||||
|
// entrypoint the operator believes is verified silently accepting
|
||||||
|
// anything.
|
||||||
|
var ErrConfig = errors.New("entrypoint signature configuration invalid")
|
||||||
|
|
||||||
|
// ErrUnauthorized marks a request that failed verification. A caller
|
||||||
|
// answers these 401.
|
||||||
|
var ErrUnauthorized = errors.New("inbound signature verification failed")
|
||||||
|
|
||||||
|
// Configuration failures. None of these carry any part of the secret.
|
||||||
|
var (
|
||||||
|
errSchemeUnknown = fmt.Errorf(
|
||||||
|
"%w: unsupported scheme", ErrConfig,
|
||||||
|
)
|
||||||
|
errSecretMissing = fmt.Errorf(
|
||||||
|
"%w: scheme set with no secret", ErrConfig,
|
||||||
|
)
|
||||||
|
errSchemeMissing = fmt.Errorf(
|
||||||
|
"%w: secret set with no scheme", ErrConfig,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
// Request failures. These are logged, so none of them carries the
|
||||||
|
// value the client sent: under the GitLab scheme that value is a
|
||||||
|
// guess at the token, and under either scheme a misconfigured sender
|
||||||
|
// could be presenting the real one.
|
||||||
|
var (
|
||||||
|
errHeaderMissing = fmt.Errorf(
|
||||||
|
"%w: signature header absent", ErrUnauthorized,
|
||||||
|
)
|
||||||
|
errHeaderMalformed = fmt.Errorf(
|
||||||
|
"%w: signature header malformed", ErrUnauthorized,
|
||||||
|
)
|
||||||
|
errSignatureMismatch = fmt.Errorf(
|
||||||
|
"%w: signature does not match", ErrUnauthorized,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
// SchemeInfo describes one supported scheme for the UI.
|
||||||
|
type SchemeInfo struct {
|
||||||
|
Scheme database.SignatureScheme
|
||||||
|
Label string
|
||||||
|
Header string
|
||||||
|
|
||||||
|
// HeaderIsDigest reports that Header carries a value derived from
|
||||||
|
// the request rather than the shared secret itself, and so may be
|
||||||
|
// kept when the request is stored and forwarded.
|
||||||
|
//
|
||||||
|
// The polarity is deliberate: false — the zero value — means the
|
||||||
|
// header is the credential and must be stripped. A scheme added
|
||||||
|
// later is therefore stripped unless whoever adds it positively
|
||||||
|
// declares the header safe to keep.
|
||||||
|
HeaderIsDigest bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// Schemes returns the supported schemes in the order the UI offers
|
||||||
|
// them. It returns a fresh slice per call so no caller can edit the
|
||||||
|
// set out from under another.
|
||||||
|
func Schemes() []SchemeInfo {
|
||||||
|
return []SchemeInfo{
|
||||||
|
{
|
||||||
|
Scheme: database.SignatureSchemeGitHub,
|
||||||
|
Label: "GitHub",
|
||||||
|
Header: HeaderGitHub,
|
||||||
|
// An HMAC over the body, not the key. Keeping it lets an
|
||||||
|
// operator see what the sender sent.
|
||||||
|
HeaderIsDigest: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Scheme: database.SignatureSchemeGitLab,
|
||||||
|
Label: "GitLab",
|
||||||
|
Header: HeaderGitLab,
|
||||||
|
// X-Gitlab-Token is the shared secret in plaintext.
|
||||||
|
HeaderIsDigest: false,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Info returns the description of a supported scheme. It reports
|
||||||
|
// false for the empty scheme and for anything unrecognised, which is
|
||||||
|
// what a row hand-edited in the database could hold.
|
||||||
|
func Info(scheme database.SignatureScheme) (SchemeInfo, bool) {
|
||||||
|
for _, s := range Schemes() {
|
||||||
|
if s.Scheme == scheme {
|
||||||
|
return s, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return SchemeInfo{}, false
|
||||||
|
}
|
||||||
|
|
||||||
|
// Supported reports whether a scheme may be stored on an entrypoint.
|
||||||
|
// The empty scheme is supported: it means no verification.
|
||||||
|
func Supported(scheme database.SignatureScheme) bool {
|
||||||
|
if scheme == database.SignatureSchemeNone {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
_, ok := Info(scheme)
|
||||||
|
|
||||||
|
return ok
|
||||||
|
}
|
||||||
|
|
||||||
|
// SanitizeHeaders returns a copy of an accepted request's headers
|
||||||
|
// with the entrypoint's credential removed.
|
||||||
|
//
|
||||||
|
// Under a scheme whose header is the shared secret itself — GitLab's
|
||||||
|
// X-Gitlab-Token — every downstream use of the inbound headers is a
|
||||||
|
// disclosure of the credential: they are persisted verbatim in the
|
||||||
|
// per-webhook event store and forwarded to every delivery target, so
|
||||||
|
// a target operator or anyone who reads the event database could
|
||||||
|
// forge signed requests to the very entrypoint the secret protects.
|
||||||
|
// Stripping happens here, once, above the first write, rather than
|
||||||
|
// at each egress, so a new consumer of Event.Headers cannot reopen
|
||||||
|
// the leak by forgetting to filter.
|
||||||
|
//
|
||||||
|
// header is never modified; the caller's request keeps its headers
|
||||||
|
// intact for anything that still needs the original.
|
||||||
|
//
|
||||||
|
// An entrypoint with no scheme, or one whose stored scheme this
|
||||||
|
// build does not know, is returned unchanged: there is no configured
|
||||||
|
// credential to remove, and the unknown case is refused by Verify
|
||||||
|
// before a request reaches storage.
|
||||||
|
func SanitizeHeaders(
|
||||||
|
entrypoint *database.Entrypoint,
|
||||||
|
header http.Header,
|
||||||
|
) http.Header {
|
||||||
|
clone := header.Clone()
|
||||||
|
if clone == nil {
|
||||||
|
return header
|
||||||
|
}
|
||||||
|
|
||||||
|
info, ok := Info(entrypoint.SignatureScheme)
|
||||||
|
if !ok || info.HeaderIsDigest {
|
||||||
|
return clone
|
||||||
|
}
|
||||||
|
|
||||||
|
clone.Del(info.Header)
|
||||||
|
|
||||||
|
return clone
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verify checks an inbound request against an entrypoint's
|
||||||
|
// configuration and returns nil when the request may be accepted.
|
||||||
|
//
|
||||||
|
// body must be the raw bytes exactly as received, before any parsing
|
||||||
|
// or normalisation: the sender computed its digest over those bytes,
|
||||||
|
// so anything that re-encodes them produces a different digest and a
|
||||||
|
// spurious rejection. The caller is also responsible for bounding
|
||||||
|
// that read; this package hashes what it is handed.
|
||||||
|
//
|
||||||
|
// Every non-nil error is either ErrConfig or ErrUnauthorized, so a
|
||||||
|
// caller can tell "the server is misconfigured" from "the client did
|
||||||
|
// not authenticate" with errors.Is.
|
||||||
|
func Verify(
|
||||||
|
entrypoint *database.Entrypoint,
|
||||||
|
header http.Header,
|
||||||
|
body []byte,
|
||||||
|
) error {
|
||||||
|
scheme := entrypoint.SignatureScheme
|
||||||
|
secret := entrypoint.SignatureSecret
|
||||||
|
|
||||||
|
if scheme == database.SignatureSchemeNone {
|
||||||
|
// A secret with no scheme names no header and no algorithm,
|
||||||
|
// so there is nothing to check it with. Accepting the request
|
||||||
|
// would make a half-applied configuration indistinguishable
|
||||||
|
// from no configuration at all.
|
||||||
|
if secret != "" {
|
||||||
|
return errSchemeMissing
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
if secret == "" {
|
||||||
|
return errSecretMissing
|
||||||
|
}
|
||||||
|
|
||||||
|
switch scheme {
|
||||||
|
case database.SignatureSchemeGitHub:
|
||||||
|
return verifyGitHub(secret, header.Get(HeaderGitHub), body)
|
||||||
|
case database.SignatureSchemeGitLab:
|
||||||
|
return verifyGitLab(secret, header.Get(HeaderGitLab))
|
||||||
|
case database.SignatureSchemeNone:
|
||||||
|
// Handled above; restated so the switch stays exhaustive and
|
||||||
|
// adding a scheme has to be decided here.
|
||||||
|
return nil
|
||||||
|
default:
|
||||||
|
return errSchemeUnknown
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// verifyGitHub checks a GitHub-style X-Hub-Signature-256: the string
|
||||||
|
// "sha256=" followed by the hex HMAC-SHA256 of the raw body under the
|
||||||
|
// shared secret.
|
||||||
|
func verifyGitHub(secret, provided string, body []byte) error {
|
||||||
|
if provided == "" {
|
||||||
|
return errHeaderMissing
|
||||||
|
}
|
||||||
|
|
||||||
|
encoded, ok := strings.CutPrefix(provided, githubPrefix)
|
||||||
|
if !ok {
|
||||||
|
return errHeaderMalformed
|
||||||
|
}
|
||||||
|
|
||||||
|
got, err := hex.DecodeString(encoded)
|
||||||
|
if err != nil {
|
||||||
|
return errHeaderMalformed
|
||||||
|
}
|
||||||
|
|
||||||
|
mac := hmac.New(sha256.New, []byte(secret))
|
||||||
|
|
||||||
|
// hash.Hash.Write is documented never to return an error.
|
||||||
|
_, _ = mac.Write(body)
|
||||||
|
|
||||||
|
// hmac.Equal, never ==: string comparison stops at the first
|
||||||
|
// differing byte, which tells a client how much of a forged
|
||||||
|
// digest it got right and turns forgery into a per-byte search.
|
||||||
|
if !hmac.Equal(mac.Sum(nil), got) {
|
||||||
|
return errSignatureMismatch
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// verifyGitLab checks a GitLab-style X-Gitlab-Token, which is the
|
||||||
|
// shared secret itself rather than a digest over the body.
|
||||||
|
//
|
||||||
|
// The comparison is constant time in the same way as the HMAC one.
|
||||||
|
// hmac.Equal returns early for unequal lengths, so the length of the
|
||||||
|
// token is not hidden; its contents are, and length alone does not
|
||||||
|
// let a client search for the value.
|
||||||
|
func verifyGitLab(secret, provided string) error {
|
||||||
|
if provided == "" {
|
||||||
|
return errHeaderMissing
|
||||||
|
}
|
||||||
|
|
||||||
|
if !hmac.Equal([]byte(provided), []byte(secret)) {
|
||||||
|
return errSignatureMismatch
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
340
internal/signature/signature_test.go
Normal file
340
internal/signature/signature_test.go
Normal file
@@ -0,0 +1,340 @@
|
|||||||
|
package signature_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/hmac"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"net/http"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
"sneak.berlin/go/webhooker/internal/signature"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
// testSharedKey is the shared secret under test. It is not named
|
||||||
|
// "secret": gosec reads a credential-shaped name bound to a
|
||||||
|
// high-entropy literal as a leaked credential, which is the right
|
||||||
|
// rule and the wrong finding here.
|
||||||
|
testSharedKey = "s3kr1t-shared-value"
|
||||||
|
testBody = `{"action":"opened","number":1}`
|
||||||
|
)
|
||||||
|
|
||||||
|
// githubSignature returns the X-Hub-Signature-256 value GitHub would
|
||||||
|
// send for testBody signed with secret.
|
||||||
|
func githubSignature(secret string) string {
|
||||||
|
mac := hmac.New(sha256.New, []byte(secret))
|
||||||
|
_, _ = mac.Write([]byte(testBody))
|
||||||
|
|
||||||
|
return "sha256=" + hex.EncodeToString(mac.Sum(nil))
|
||||||
|
}
|
||||||
|
|
||||||
|
// headerWith builds a request header carrying one value.
|
||||||
|
func headerWith(name, value string) http.Header {
|
||||||
|
h := http.Header{}
|
||||||
|
if name != "" {
|
||||||
|
h.Set(name, value)
|
||||||
|
}
|
||||||
|
|
||||||
|
return h
|
||||||
|
}
|
||||||
|
|
||||||
|
// entrypoint builds an entrypoint with a signature configuration.
|
||||||
|
func entrypoint(
|
||||||
|
scheme database.SignatureScheme, secret string,
|
||||||
|
) *database.Entrypoint {
|
||||||
|
return &database.Entrypoint{
|
||||||
|
SignatureScheme: scheme,
|
||||||
|
SignatureSecret: secret,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestVerifyUnconfiguredAcceptsAnything pins the pass-through case:
|
||||||
|
// an entrypoint with no scheme is the entrypoint every deployment
|
||||||
|
// already has, and it must keep accepting requests that carry no
|
||||||
|
// signature at all.
|
||||||
|
func TestVerifyUnconfiguredAcceptsAnything(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
ep := entrypoint(database.SignatureSchemeNone, "")
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t, signature.Verify(ep, http.Header{}, []byte(testBody)),
|
||||||
|
)
|
||||||
|
require.NoError(
|
||||||
|
t,
|
||||||
|
signature.Verify(
|
||||||
|
ep,
|
||||||
|
headerWith(signature.HeaderGitHub, "sha256=deadbeef"),
|
||||||
|
[]byte(testBody),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// githubCase is one inbound request against a GitHub-scheme
|
||||||
|
// entrypoint.
|
||||||
|
type githubCase struct {
|
||||||
|
name string
|
||||||
|
header string
|
||||||
|
value string
|
||||||
|
body string
|
||||||
|
want error
|
||||||
|
}
|
||||||
|
|
||||||
|
// githubCases enumerates the shapes a GitHub signature can arrive in.
|
||||||
|
func githubCases() []githubCase {
|
||||||
|
valid := githubSignature(testSharedKey)
|
||||||
|
|
||||||
|
return []githubCase{
|
||||||
|
{
|
||||||
|
name: "valid",
|
||||||
|
header: signature.HeaderGitHub,
|
||||||
|
value: valid,
|
||||||
|
body: testBody,
|
||||||
|
want: nil,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "absent header",
|
||||||
|
header: "",
|
||||||
|
body: testBody,
|
||||||
|
want: signature.ErrUnauthorized,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "wrong secret",
|
||||||
|
header: signature.HeaderGitHub,
|
||||||
|
value: githubSignature("not-the-shared-value"),
|
||||||
|
body: testBody,
|
||||||
|
want: signature.ErrUnauthorized,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
// The digest is valid for a different body: the check
|
||||||
|
// has to be over the bytes actually received.
|
||||||
|
name: "body altered in flight",
|
||||||
|
header: signature.HeaderGitHub,
|
||||||
|
value: valid,
|
||||||
|
body: testBody + " ",
|
||||||
|
want: signature.ErrUnauthorized,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "missing algorithm prefix",
|
||||||
|
header: signature.HeaderGitHub,
|
||||||
|
value: valid[len("sha256="):],
|
||||||
|
body: testBody,
|
||||||
|
want: signature.ErrUnauthorized,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "not hex",
|
||||||
|
header: signature.HeaderGitHub,
|
||||||
|
value: "sha256=zzzz",
|
||||||
|
body: testBody,
|
||||||
|
want: signature.ErrUnauthorized,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "empty digest",
|
||||||
|
header: signature.HeaderGitHub,
|
||||||
|
value: "sha256=",
|
||||||
|
body: testBody,
|
||||||
|
want: signature.ErrUnauthorized,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
// GitLab's header does not authenticate a GitHub
|
||||||
|
// entrypoint, even holding the right secret.
|
||||||
|
name: "wrong header for the scheme",
|
||||||
|
header: signature.HeaderGitLab,
|
||||||
|
value: testSharedKey,
|
||||||
|
body: testBody,
|
||||||
|
want: signature.ErrUnauthorized,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestVerifyGitHub(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for _, tc := range githubCases() {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
err := signature.Verify(
|
||||||
|
entrypoint(
|
||||||
|
database.SignatureSchemeGitHub, testSharedKey,
|
||||||
|
),
|
||||||
|
headerWith(tc.header, tc.value),
|
||||||
|
[]byte(tc.body),
|
||||||
|
)
|
||||||
|
|
||||||
|
if tc.want == nil {
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
require.ErrorIs(t, err, tc.want)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestVerifyGitLab(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
cases := []struct {
|
||||||
|
name string
|
||||||
|
header string
|
||||||
|
value string
|
||||||
|
want error
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "valid",
|
||||||
|
header: signature.HeaderGitLab,
|
||||||
|
value: testSharedKey,
|
||||||
|
want: nil,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "absent header",
|
||||||
|
header: "",
|
||||||
|
want: signature.ErrUnauthorized,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "wrong token",
|
||||||
|
header: signature.HeaderGitLab,
|
||||||
|
value: "not-the-shared-value",
|
||||||
|
want: signature.ErrUnauthorized,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "token prefix only",
|
||||||
|
header: signature.HeaderGitLab,
|
||||||
|
value: testSharedKey[:5],
|
||||||
|
want: signature.ErrUnauthorized,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "wrong header for the scheme",
|
||||||
|
header: signature.HeaderGitHub,
|
||||||
|
value: githubSignature(testSharedKey),
|
||||||
|
want: signature.ErrUnauthorized,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range cases {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
err := signature.Verify(
|
||||||
|
entrypoint(
|
||||||
|
database.SignatureSchemeGitLab, testSharedKey,
|
||||||
|
),
|
||||||
|
headerWith(tc.header, tc.value),
|
||||||
|
[]byte(testBody),
|
||||||
|
)
|
||||||
|
|
||||||
|
if tc.want == nil {
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
require.ErrorIs(t, err, tc.want)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestVerifyBrokenConfigurationFailsClosed covers the rows a caller
|
||||||
|
// must refuse rather than wave through. Each is a state an operator
|
||||||
|
// could only reach outside the UI, and each one would otherwise be
|
||||||
|
// indistinguishable from "verification is off".
|
||||||
|
func TestVerifyBrokenConfigurationFailsClosed(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
cases := []struct {
|
||||||
|
name string
|
||||||
|
scheme database.SignatureScheme
|
||||||
|
secret string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "unknown scheme",
|
||||||
|
scheme: database.SignatureScheme("stripe"),
|
||||||
|
secret: testSharedKey,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "scheme without secret",
|
||||||
|
scheme: database.SignatureSchemeGitHub,
|
||||||
|
secret: "",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "secret without scheme",
|
||||||
|
scheme: database.SignatureSchemeNone,
|
||||||
|
secret: testSharedKey,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range cases {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
err := signature.Verify(
|
||||||
|
entrypoint(tc.scheme, tc.secret),
|
||||||
|
headerWith(
|
||||||
|
signature.HeaderGitHub,
|
||||||
|
githubSignature(testSharedKey),
|
||||||
|
),
|
||||||
|
[]byte(testBody),
|
||||||
|
)
|
||||||
|
|
||||||
|
require.ErrorIs(t, err, signature.ErrConfig)
|
||||||
|
assert.NotErrorIs(t, err, signature.ErrUnauthorized)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestErrorsCarryNoSecret proves the strings that reach the log hold
|
||||||
|
// no part of the shared secret or of what the client presented.
|
||||||
|
func TestErrorsCarryNoSecret(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const presented = "QQPRESENTEDTOKENQQ"
|
||||||
|
|
||||||
|
for _, scheme := range []database.SignatureScheme{
|
||||||
|
database.SignatureSchemeGitHub,
|
||||||
|
database.SignatureSchemeGitLab,
|
||||||
|
} {
|
||||||
|
for _, header := range []string{
|
||||||
|
signature.HeaderGitHub, signature.HeaderGitLab,
|
||||||
|
} {
|
||||||
|
err := signature.Verify(
|
||||||
|
entrypoint(scheme, testSharedKey),
|
||||||
|
headerWith(header, presented),
|
||||||
|
[]byte(testBody),
|
||||||
|
)
|
||||||
|
require.Error(t, err)
|
||||||
|
assert.NotContains(t, err.Error(), testSharedKey)
|
||||||
|
assert.NotContains(t, err.Error(), presented)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSchemeMetadata(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
assert.True(t, signature.Supported(database.SignatureSchemeNone))
|
||||||
|
assert.True(t, signature.Supported(database.SignatureSchemeGitHub))
|
||||||
|
assert.True(t, signature.Supported(database.SignatureSchemeGitLab))
|
||||||
|
assert.False(
|
||||||
|
t, signature.Supported(database.SignatureScheme("stripe")),
|
||||||
|
)
|
||||||
|
|
||||||
|
// The empty scheme describes no sender, so it has no info even
|
||||||
|
// though it is a storable value.
|
||||||
|
_, ok := signature.Info(database.SignatureSchemeNone)
|
||||||
|
assert.False(t, ok)
|
||||||
|
|
||||||
|
info, ok := signature.Info(database.SignatureSchemeGitHub)
|
||||||
|
require.True(t, ok)
|
||||||
|
assert.Equal(t, "GitHub", info.Label)
|
||||||
|
assert.Equal(t, signature.HeaderGitHub, info.Header)
|
||||||
|
|
||||||
|
info, ok = signature.Info(database.SignatureSchemeGitLab)
|
||||||
|
require.True(t, ok)
|
||||||
|
assert.Equal(t, "GitLab", info.Label)
|
||||||
|
assert.Equal(t, signature.HeaderGitLab, info.Header)
|
||||||
|
}
|
||||||
@@ -48,7 +48,7 @@
|
|||||||
|
|
||||||
<div class="divide-y divide-gray-100">
|
<div class="divide-y divide-gray-100">
|
||||||
{{range .Entrypoints}}
|
{{range .Entrypoints}}
|
||||||
<div class="p-4">
|
<div class="p-4" x-data="{ showSecret: false }">
|
||||||
<div class="flex items-center justify-between mb-1">
|
<div class="flex items-center justify-between mb-1">
|
||||||
<span class="text-sm font-medium text-gray-900">{{if .Description}}{{.Description}}{{else}}Entrypoint{{end}}</span>
|
<span class="text-sm font-medium text-gray-900">{{if .Description}}{{.Description}}{{else}}Entrypoint{{end}}</span>
|
||||||
<div class="flex items-center gap-2">
|
<div class="flex items-center gap-2">
|
||||||
@@ -75,6 +75,38 @@
|
|||||||
script the URL above stays selectable. -->
|
script the URL above stays selectable. -->
|
||||||
<button type="button" hidden data-copy-target="entrypoint-url-{{.ID}}" class="text-xs text-gray-500 hover:text-primary-600">Copy</button>
|
<button type="button" hidden data-copy-target="entrypoint-url-{{.ID}}" class="text-xs text-gray-500 hover:text-primary-600">Copy</button>
|
||||||
</div>
|
</div>
|
||||||
|
<div class="flex items-center gap-2 mt-2">
|
||||||
|
<span class="text-xs text-gray-500">
|
||||||
|
Signature: {{.SchemeLabel}}{{if .SchemeHeader}} ({{.SchemeHeader}}){{end}}
|
||||||
|
</span>
|
||||||
|
<button type="button" @click="showSecret = !showSecret" class="text-xs text-gray-500 hover:text-primary-600">
|
||||||
|
{{if .Configured}}Rotate{{else}}Configure{{end}}
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
<!-- The stored secret is never sent to the browser: the
|
||||||
|
form takes a new one every time, so setting and
|
||||||
|
rotating are the same submission. -->
|
||||||
|
<div x-show="showSecret" x-cloak class="mt-2">
|
||||||
|
<form method="POST" action="/source/{{$.Webhook.ID}}/entrypoints/{{.ID}}/secret" class="flex gap-2">
|
||||||
|
<input type="hidden" name="csrf_token" value="{{$.CSRFToken}}">
|
||||||
|
<select name="signature_scheme" class="input text-sm w-28">
|
||||||
|
<!-- Selection follows the stored scheme, not
|
||||||
|
whether the pair is complete: a row with a
|
||||||
|
scheme and no secret would otherwise mark
|
||||||
|
both this option and its own selected. -->
|
||||||
|
<option value="" {{if not .Scheme}}selected{{end}}>None</option>
|
||||||
|
{{$current := .Scheme}}
|
||||||
|
{{range $.SignatureSchemes}}
|
||||||
|
<option value="{{.Scheme}}" {{if eq .Scheme $current}}selected{{end}}>{{.Label}}</option>
|
||||||
|
{{end}}
|
||||||
|
</select>
|
||||||
|
<input type="password" name="secret" autocomplete="new-password" placeholder="Shared secret" class="input text-sm flex-1">
|
||||||
|
<button type="submit" class="btn-primary text-sm">Save</button>
|
||||||
|
</form>
|
||||||
|
<p class="text-xs text-gray-500 mt-1">
|
||||||
|
Enter the same secret you configured at the sender. Selecting None removes verification.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
{{else}}
|
{{else}}
|
||||||
<div class="p-4 text-sm text-gray-500">No entrypoints configured.</div>
|
<div class="p-4 text-sm text-gray-500">No entrypoints configured.</div>
|
||||||
|
|||||||
@@ -12,6 +12,10 @@
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
{{if .ReplayMessage}}
|
||||||
|
<div class="{{if .ReplayQueued}}alert-success{{else}}alert-error{{end}}">{{.ReplayMessage}}</div>
|
||||||
|
{{end}}
|
||||||
|
|
||||||
<div class="card">
|
<div class="card">
|
||||||
<div class="divide-y divide-gray-100">
|
<div class="divide-y divide-gray-100">
|
||||||
{{range .Events}}
|
{{range .Events}}
|
||||||
@@ -40,6 +44,23 @@
|
|||||||
{{if .BodyTruncated}}
|
{{if .BodyTruncated}}
|
||||||
<p class="mt-2 text-xs text-gray-500">Body truncated for display: showing {{.BodyShownBytes}} of {{.BodyBytes}} bytes. The stored body is unchanged — <a href="/source/{{$.Webhook.ID}}/logs/{{.ID}}/body" class="text-primary-600 hover:text-primary-700 underline">download the full body</a>.</p>
|
<p class="mt-2 text-xs text-gray-500">Body truncated for display: showing {{.BodyShownBytes}} of {{.BodyBytes}} bytes. The stored body is unchanged — <a href="/source/{{$.Webhook.ID}}/logs/{{.ID}}/body" class="text-primary-600 hover:text-primary-700 underline">download the full body</a>.</p>
|
||||||
{{end}}
|
{{end}}
|
||||||
|
|
||||||
|
{{if .Deliveries}}
|
||||||
|
<div class="mt-3 pt-4 border-t border-gray-200">
|
||||||
|
{{range .Deliveries}}
|
||||||
|
<div class="flex items-center justify-between py-2 text-xs text-gray-700">
|
||||||
|
<span><span class="font-medium">{{.Target.Name}}</span>: {{.Status}}</span>
|
||||||
|
{{if .Status.Terminal}}
|
||||||
|
<form method="POST" action="/source/{{$.Webhook.ID}}/deliveries/{{.ID}}/replay" class="inline">
|
||||||
|
<input type="hidden" name="csrf_token" value="{{$.CSRFToken}}">
|
||||||
|
<input type="hidden" name="page" value="{{$.Page}}">
|
||||||
|
<button type="submit" class="text-xs text-primary-600 hover:text-primary-700" title="Send this event to the target again">Replay</button>
|
||||||
|
</form>
|
||||||
|
{{end}}
|
||||||
|
</div>
|
||||||
|
{{end}}
|
||||||
|
</div>
|
||||||
|
{{end}}
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
{{else}}
|
{{else}}
|
||||||
|
|||||||
Reference in New Issue
Block a user