Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
251cb3d3d3 |
@@ -7,13 +7,6 @@ services, durably stores them, and delivers them to configured targets
|
|||||||
with retry support, logging, and observability. Category: infrastructure
|
with retry support, logging, and observability. Category: infrastructure
|
||||||
/ web service. License: MIT.
|
/ web service. License: MIT.
|
||||||
|
|
||||||
Each entrypoint is a version 4 UUID served at `/webhook/{uuid}`, and
|
|
||||||
that UUID is the entrypoint's only credential. webhooker does not use
|
|
||||||
shared secrets, HMAC signatures or token headers on the receiver, and
|
|
||||||
will not add them — read
|
|
||||||
[The entrypoint URL is the authentication secret](#the-entrypoint-url-is-the-authentication-secret)
|
|
||||||
before deploying one.
|
|
||||||
|
|
||||||
## Getting Started
|
## Getting Started
|
||||||
|
|
||||||
### Prerequisites
|
### Prerequisites
|
||||||
@@ -1156,38 +1149,14 @@ backups at rest and restrict who can read them.
|
|||||||
|
|
||||||
## The entrypoint URL is the authentication secret
|
## The entrypoint URL is the authentication secret
|
||||||
|
|
||||||
**The entrypoint UUID is the credential, and it is the only one.**
|
The receiver verifies nothing about an inbound request. The UUID in an
|
||||||
webhooker mints a version 4 UUID per entrypoint and serves it at
|
entrypoint's URL is its credential: anyone who holds that URL can
|
||||||
`/webhook/{uuid}`. Possession of that URL is the authentication:
|
submit events to it, and the receiver checks nothing else about the
|
||||||
anyone who holds it can submit events to the entrypoint, and the
|
sender. Treat an entrypoint URL the way you would treat an API token.
|
||||||
receiver verifies nothing else about the sender.
|
|
||||||
|
|
||||||
There is no shared secret, no HMAC signature, no bearer token and no
|
There is no way to rotate the UUID in place. To retire one, delete the
|
||||||
second factor on the receiver, and none will be added. This was
|
entrypoint (or deactivate it, which answers `410`) and create a new
|
||||||
considered and rejected; the implementation that existed was removed
|
one, then point the sender at the new URL.
|
||||||
in [PR #279](https://git.eeqj.de/sneak/webhooker/pulls/279), closing
|
|
||||||
[issue #67](https://git.eeqj.de/sneak/webhooker/issues/67) and
|
|
||||||
[issue #241](https://git.eeqj.de/sneak/webhooker/issues/241). A
|
|
||||||
proposal to reintroduce any of them — including as "defence in depth"
|
|
||||||
alongside the UUID — is answered by this section. Inbound signature
|
|
||||||
headers a sender sends anyway (`X-Hub-Signature` and its
|
|
||||||
per-provider equivalents) are stored and forwarded as ordinary
|
|
||||||
headers; nothing checks them.
|
|
||||||
|
|
||||||
What that means for an operator:
|
|
||||||
|
|
||||||
- **The URL is a capability, so treat it as a secret.** Keep it out of
|
|
||||||
logs, ticket bodies, chat messages and screenshots. Anyone who reads
|
|
||||||
it anywhere can post events as that sender.
|
|
||||||
- **Rotating means minting a new entrypoint, not changing a key.**
|
|
||||||
There is no way to rotate the UUID in place. To retire one, delete
|
|
||||||
the entrypoint (or deactivate it, which answers `410`) and create a
|
|
||||||
new one, then point the sender at the new URL.
|
|
||||||
- **A sender that cannot be given a secret URL is a constraint on that
|
|
||||||
integration, not a reason to change this.** If a service only
|
|
||||||
supports signed payloads to a well-known URL, raise it as its own
|
|
||||||
problem — pick a different integration path, or accept that it
|
|
||||||
cannot be used. It is not grounds to reintroduce shared secrets.
|
|
||||||
|
|
||||||
## Entrypoints
|
## Entrypoints
|
||||||
|
|
||||||
@@ -1700,22 +1669,6 @@ retries) is individually logged for full observability.
|
|||||||
|
|
||||||
**Relations:** Belongs to Delivery.
|
**Relations:** Belongs to Delivery.
|
||||||
|
|
||||||
#### Event-tier indexes
|
|
||||||
|
|
||||||
Beyond the primary keys, the per-webhook event databases carry secondary
|
|
||||||
indexes on the columns the background work reads by, each created by
|
|
||||||
`AutoMigrate` on a fresh and on an existing database:
|
|
||||||
|
|
||||||
| Column | Serves |
|
|
||||||
| ------------------------------ | ------ |
|
|
||||||
| `deliveries.status` | The recovery and sweep queries that select deliveries by status once a minute |
|
|
||||||
| `deliveries.event_id` | Loading a page of the event log, which reads deliveries by event |
|
|
||||||
| `delivery_results.delivery_id` | Loading a page of the event log, which reads results by delivery |
|
|
||||||
| `events.created_at` | Retention, which deletes events by age |
|
|
||||||
|
|
||||||
The `events.resubmitted_from_id` column is also indexed, to resolve the
|
|
||||||
resubmit relationship both ways in the event log.
|
|
||||||
|
|
||||||
#### Common Fields
|
#### Common Fields
|
||||||
|
|
||||||
Every entity except `Setting` includes these fields from `BaseModel`.
|
Every entity except `Setting` includes these fields from `BaseModel`.
|
||||||
@@ -2914,10 +2867,6 @@ check, see [The login endpoint](#the-login-endpoint).
|
|||||||
|
|
||||||
### Authentication
|
### Authentication
|
||||||
|
|
||||||
- **Webhook receiver:** the entrypoint UUID in the URL, and nothing
|
|
||||||
else. No shared secret, no HMAC signature, no token header, and none
|
|
||||||
will be added — see
|
|
||||||
[The entrypoint URL is the authentication secret](#the-entrypoint-url-is-the-authentication-secret).
|
|
||||||
- **Web UI:** Cookie-based sessions using gorilla/sessions with
|
- **Web UI:** Cookie-based sessions using gorilla/sessions with
|
||||||
encrypted cookies. Sessions are configured with HttpOnly, SameSite
|
encrypted cookies. Sessions are configured with HttpOnly, SameSite
|
||||||
Lax, and Secure whenever the request is on TLS — the flag follows the
|
Lax, and Secure whenever the request is on TLS — the flag follows the
|
||||||
@@ -2957,8 +2906,7 @@ check, see [The login endpoint](#the-login-endpoint).
|
|||||||
mode
|
mode
|
||||||
- **The entrypoint URL is the receiver's only credential.** Nothing
|
- **The entrypoint URL is the receiver's only credential.** Nothing
|
||||||
about an inbound request is verified; possession of the UUID
|
about an inbound request is verified; possession of the UUID
|
||||||
authorises submission, and no shared secret or signature check will
|
authorises submission (see
|
||||||
be added alongside it (see
|
|
||||||
[The entrypoint URL is the authentication secret](#the-entrypoint-url-is-the-authentication-secret))
|
[The entrypoint URL is the authentication secret](#the-entrypoint-url-is-the-authentication-secret))
|
||||||
- **SSRF prevention** for HTTP delivery targets: private/reserved IP
|
- **SSRF prevention** for HTTP delivery targets: private/reserved IP
|
||||||
ranges (RFC 1918, loopback, link-local, cloud metadata) are blocked
|
ranges (RFC 1918, loopback, link-local, cloud metadata) are blocked
|
||||||
|
|||||||
@@ -1,72 +0,0 @@
|
|||||||
package database_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"github.com/google/uuid"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
|
||||||
)
|
|
||||||
|
|
||||||
// indexedColumn names a secondary index by the model and struct field
|
|
||||||
// GORM derives the index name from.
|
|
||||||
type indexedColumn struct {
|
|
||||||
model any
|
|
||||||
field string
|
|
||||||
}
|
|
||||||
|
|
||||||
// eventTierIndexes are the columns the background work reads by: the
|
|
||||||
// recovery and sweep queries (status), the event log (event_id and
|
|
||||||
// delivery_id) and retention (created_at).
|
|
||||||
var eventTierIndexes = []indexedColumn{
|
|
||||||
{&database.Delivery{}, "Status"},
|
|
||||||
{&database.Delivery{}, "EventID"},
|
|
||||||
{&database.DeliveryResult{}, "DeliveryID"},
|
|
||||||
{&database.Event{}, "CreatedAt"},
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestWebhookDBManager_OpenAddsEventTierIndexes verifies that opening a
|
|
||||||
// per-webhook database that predates these indexes creates them, so the
|
|
||||||
// queries above stop scanning whole tables. It stands in for an older
|
|
||||||
// database file by dropping the indexes AutoMigrate just created, then
|
|
||||||
// reopening the same file.
|
|
||||||
func TestWebhookDBManager_OpenAddsEventTierIndexes(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
mgr, lc := setupTestWebhookDBManager(t)
|
|
||||||
ctx := context.Background()
|
|
||||||
require.NoError(t, lc.Start(ctx))
|
|
||||||
|
|
||||||
defer func() { require.NoError(t, lc.Stop(ctx)) }()
|
|
||||||
|
|
||||||
webhookID := uuid.New().String()
|
|
||||||
|
|
||||||
db, err := mgr.GetDB(webhookID)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
// A fresh database has them.
|
|
||||||
for _, ix := range eventTierIndexes {
|
|
||||||
require.True(t, db.Migrator().HasIndex(ix.model, ix.field))
|
|
||||||
}
|
|
||||||
|
|
||||||
// Stand in for a database file created before the indexes existed.
|
|
||||||
for _, ix := range eventTierIndexes {
|
|
||||||
require.NoError(t, db.Migrator().DropIndex(ix.model, ix.field))
|
|
||||||
require.False(t, db.Migrator().HasIndex(ix.model, ix.field))
|
|
||||||
}
|
|
||||||
|
|
||||||
// Drop the cached connection so the next open reopens the file and
|
|
||||||
// runs AutoMigrate against it, as a restart would.
|
|
||||||
require.NoError(t, mgr.CloseAll())
|
|
||||||
|
|
||||||
db, err = mgr.GetDB(webhookID)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
for _, ix := range eventTierIndexes {
|
|
||||||
assert.True(t, db.Migrator().HasIndex(ix.model, ix.field),
|
|
||||||
"opening the existing database should create the index on %s",
|
|
||||||
ix.field)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -32,9 +32,9 @@ func (s DeliveryStatus) Terminal() bool {
|
|||||||
type Delivery struct {
|
type Delivery struct {
|
||||||
BaseModel
|
BaseModel
|
||||||
|
|
||||||
EventID string `gorm:"type:uuid;not null;index" json:"eventId"`
|
EventID string `gorm:"type:uuid;not null" json:"eventId"`
|
||||||
TargetID string `gorm:"type:uuid;not null" json:"targetId"`
|
TargetID string `gorm:"type:uuid;not null" json:"targetId"`
|
||||||
Status DeliveryStatus `gorm:"not null;default:'pending';index" json:"status"`
|
Status DeliveryStatus `gorm:"not null;default:'pending'" json:"status"`
|
||||||
|
|
||||||
// Relations
|
// Relations
|
||||||
Event Event `json:"event,omitzero"`
|
Event Event `json:"event,omitzero"`
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ package database
|
|||||||
type DeliveryResult struct {
|
type DeliveryResult struct {
|
||||||
BaseModel
|
BaseModel
|
||||||
|
|
||||||
DeliveryID string `gorm:"type:uuid;not null;index" json:"deliveryId"`
|
DeliveryID string `gorm:"type:uuid;not null" json:"deliveryId"`
|
||||||
AttemptNum int `gorm:"not null" json:"attemptNum"`
|
AttemptNum int `gorm:"not null" json:"attemptNum"`
|
||||||
Success bool `json:"success"`
|
Success bool `json:"success"`
|
||||||
StatusCode int `json:"statusCode,omitempty"`
|
StatusCode int `json:"statusCode,omitempty"`
|
||||||
|
|||||||
@@ -1,17 +1,9 @@
|
|||||||
package database
|
package database
|
||||||
|
|
||||||
import "time"
|
|
||||||
|
|
||||||
// Event represents a captured webhook event
|
// Event represents a captured webhook event
|
||||||
type Event struct {
|
type Event struct {
|
||||||
BaseModel
|
BaseModel
|
||||||
|
|
||||||
// CreatedAt overrides BaseModel.CreatedAt only to add an index:
|
|
||||||
// retention deletes events by age, so events.created_at is queried
|
|
||||||
// on every sweep. The other tables keep the unindexed BaseModel
|
|
||||||
// field.
|
|
||||||
CreatedAt time.Time `gorm:"index" json:"createdAt"`
|
|
||||||
|
|
||||||
WebhookID string `gorm:"type:uuid;not null" json:"webhookId"`
|
WebhookID string `gorm:"type:uuid;not null" json:"webhookId"`
|
||||||
EntrypointID string `gorm:"type:uuid;not null" json:"entrypointId"`
|
EntrypointID string `gorm:"type:uuid;not null" json:"entrypointId"`
|
||||||
|
|
||||||
|
|||||||
@@ -170,8 +170,7 @@ func TestDelivery_CrossOriginRedirectDropsOriginScopedHeaders(
|
|||||||
// Stripping must not fire within the configured origin, or every
|
// Stripping must not fire within the configured origin, or every
|
||||||
// destination that redirects its own path would lose its
|
// destination that redirects its own path would lose its
|
||||||
// credential and start answering 401 — and would lose the inbound
|
// credential and start answering 401 — and would lose the inbound
|
||||||
// signature header the target endpoint verifies. webhooker's own
|
// signature the receiver verifies.
|
||||||
// receiver verifies no signature; it only forwards the header.
|
|
||||||
func TestDelivery_SameOriginRedirectKeepsOriginScopedHeaders(
|
func TestDelivery_SameOriginRedirectKeepsOriginScopedHeaders(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
) {
|
) {
|
||||||
|
|||||||
Reference in New Issue
Block a user