Compare commits
6
Commits
8b5e3b734f
...
1ac8102243
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1ac8102243 | ||
|
|
0945831442 | ||
|
|
f82b730c31 | ||
|
|
1a1fee0874 | ||
|
|
290925f184 | ||
|
|
73353bc8e5 |
@@ -1327,15 +1327,17 @@ under the real policy and checks that: both add forms stay hidden until Add is
|
|||||||
clicked; choosing Slack in the add target form leaves the HTTP fields out of
|
clicked; choosing Slack in the add target form leaves the HTTP fields out of
|
||||||
what it submits, also after leaving the page and going back to it, when the
|
what it submits, also after leaving the page and going back to it, when the
|
||||||
browser restores the choice; the Copy button beside an entrypoint URL reads
|
browser restores the choice; the Copy button beside an entrypoint URL reads
|
||||||
"Copied" once clicked; an event expands and collapses, and so do a delivery's
|
"Copied" once clicked; an entrypoint's Edit button shows its edit form in place
|
||||||
attempts inside it; and at phone width the menu button opens and closes the
|
of its description and hides until the form closes, Cancel hides the form and
|
||||||
mobile menu. It also fails if the browser reports a console warning or error,
|
drops what was typed, and Save changes the description; an event expands and
|
||||||
an uncaught exception, or anything the policy refused. `make check` and the
|
collapses, and so do a delivery's attempts inside it; and at phone width the
|
||||||
image build lint it but do not run it, and `make test` leaves it out (its file
|
menu button opens and closes the mobile menu. It also fails if the browser
|
||||||
is built only with the `browser` build tag). Run it with `make test-browser`
|
reports a console warning or error, an uncaught exception, or anything the
|
||||||
after changing `templates/` or `static/js/`: that builds `Dockerfile.browser`,
|
policy refused. `make check` and the image build lint it but do not run it, and
|
||||||
which runs the test in a digest-pinned headless browser image, so the host
|
`make test` leaves it out (its file is built only with the `browser` build tag).
|
||||||
needs no browser.
|
Run it with `make test-browser` after changing `templates/` or `static/js/`:
|
||||||
|
that builds `Dockerfile.browser`, which runs the test in a digest-pinned
|
||||||
|
headless browser image, so the host needs no browser.
|
||||||
|
|
||||||
The package's tarball is committed as `3p/alpinejs-csp-3.14.9.tgz`, byte for
|
The package's tarball is committed as `3p/alpinejs-csp-3.14.9.tgz`, byte for
|
||||||
byte as the npm registry publishes it. It is a dependency, not this repo's build
|
byte as the npm registry publishes it. It is a dependency, not this repo's build
|
||||||
@@ -2893,6 +2895,7 @@ returns to the page that was asked for.
|
|||||||
| `POST` | `/hook/{id}/deliveries/{deliveryID}/replay` | Replay a finished delivery: creates a new delivery for the same event against the target's current configuration (30 per minute per bucket, then `429`) |
|
| `POST` | `/hook/{id}/deliveries/{deliveryID}/replay` | Replay a finished delivery: creates a new delivery for the same event against the target's current configuration (30 per minute per bucket, then `429`) |
|
||||||
| `POST` | `/hook/{id}/events/{eventID}/resubmit` | Resubmit a stored event: creates a new event copying it and fans that out to every currently active target (30 per minute per bucket, then `429`) |
|
| `POST` | `/hook/{id}/events/{eventID}/resubmit` | Resubmit a stored event: creates a new event copying it and fans that out to every currently active target (30 per minute per bucket, then `429`) |
|
||||||
| `POST` | `/hook/{id}/entrypoints` | Add entrypoint to webhook |
|
| `POST` | `/hook/{id}/entrypoints` | Add entrypoint to webhook |
|
||||||
|
| `POST` | `/hook/{id}/entrypoints/{entrypointID}/edit` | Change an entrypoint's description; its URL stays the same |
|
||||||
| `POST` | `/hook/{id}/entrypoints/{entrypointID}/delete` | Delete an entrypoint |
|
| `POST` | `/hook/{id}/entrypoints/{entrypointID}/delete` | Delete an entrypoint |
|
||||||
| `POST` | `/hook/{id}/entrypoints/{entrypointID}/toggle` | Enable or disable an entrypoint |
|
| `POST` | `/hook/{id}/entrypoints/{entrypointID}/toggle` | Enable or disable an entrypoint |
|
||||||
| `POST` | `/hook/{id}/targets` | Add target to webhook |
|
| `POST` | `/hook/{id}/targets` | Add target to webhook |
|
||||||
@@ -3245,9 +3248,9 @@ each hook. The order, read off the fx stop-hook log:
|
|||||||
|
|
||||||
1. `ArchiveSweeper`
|
1. `ArchiveSweeper`
|
||||||
2. `RetentionReaper`
|
2. `RetentionReaper`
|
||||||
3. `server` — the HTTP drain, bounded separately by
|
3. `server` — the HTTP drain, bounded by `server.ShutdownTimeout`
|
||||||
`server.ShutdownTimeout` (**3 seconds**), then a Sentry flush if
|
(**3 seconds**) and by what the hooks before it left, then a Sentry
|
||||||
`SENTRY_DSN` is set
|
flush if `SENTRY_DSN` is set
|
||||||
4. `delivery.Engine` — waits for its workers, then closes the archive
|
4. `delivery.Engine` — waits for its workers, then closes the archive
|
||||||
databases
|
databases
|
||||||
5. `healthcheck`
|
5. `healthcheck`
|
||||||
@@ -3267,23 +3270,30 @@ exhaust the sequence budget at the instant it finished, and every
|
|||||||
later hook — the delivery engine, the healthcheck, the webhook DB
|
later hook — the delivery engine, the healthcheck, the webhook DB
|
||||||
manager and the database close — would be skipped in exactly the
|
manager and the database close — would be skipped in exactly the
|
||||||
case where the drain mattered. 3 seconds leaves 2 seconds
|
case where the drain mattered. 3 seconds leaves 2 seconds
|
||||||
(`server.TailHookReserve`) for the tail, which is far more than the
|
(`server.TailHookReserve`) for the tail. The reserve is that
|
||||||
microseconds it needs.
|
remainder, not a figure sized to the tail, which takes about a
|
||||||
|
millisecond.
|
||||||
|
|
||||||
That reserve belongs to the tail hooks, not to the server hook, and
|
That reserve belongs to the tail hooks, not to the server hook, and
|
||||||
the Sentry flush is what could take it: it runs after the drain
|
the server hook could take it in two ways. The hooks before it may
|
||||||
**inside the same hook**, and `sentry.Flush` takes a bare duration
|
already have spent part of the budget, so a full 3-second drain
|
||||||
and honours no context, so an unreachable Sentry endpoint would add
|
would come out of the reserve; the drain is therefore also bounded
|
||||||
its own timeout on top of a full-length drain and consume the whole
|
by whatever is left on the stop context minus the reserve. And the
|
||||||
sequence budget by itself. It is therefore clamped to whatever is
|
Sentry flush runs after the drain **inside the same hook**, and
|
||||||
left on the stop context minus the reserve, and skipped when that
|
`sentry.Flush` takes a bare duration and honours no context, so an
|
||||||
leaves too little to be worth attempting — so a full-length drain
|
unreachable Sentry endpoint would add its own timeout on top of a
|
||||||
means Sentry events are dropped rather than the database close being
|
full-length drain and consume the whole sequence budget by itself.
|
||||||
skipped.
|
It is clamped the same way, and skipped when that leaves too little
|
||||||
|
to be worth attempting — so a full-length drain means Sentry events
|
||||||
|
are dropped rather than the database close being skipped.
|
||||||
|
|
||||||
This does not make the database close unconditional: a wedged
|
This does not make the database close unconditional. A slow
|
||||||
`ArchiveSweeper` or `RetentionReaper` still runs first and can
|
`ArchiveSweeper` or `RetentionReaper` is enough to cut the shutdown
|
||||||
consume the whole budget on its own.
|
short, not only one that consumes the whole budget: what they spend
|
||||||
|
comes out of the drain first, so after 2 seconds of theirs a request
|
||||||
|
still in flight gets 1 second to finish, and after 3 it gets none.
|
||||||
|
Past 3 seconds they spend the reserve itself, and one that takes the
|
||||||
|
whole budget skips every hook after it, the database close included.
|
||||||
|
|
||||||
The value is chosen to sit inside the container stop grace period.
|
The value is chosen to sit inside the container stop grace period.
|
||||||
Docker's default `docker stop` grace is 10 seconds and the Dockerfile
|
Docker's default `docker stop` grace is 10 seconds and the Dockerfile
|
||||||
|
|||||||
@@ -38,17 +38,19 @@ import (
|
|||||||
// hook that used the whole budget would exhaust it at that instant,
|
// hook that used the whole budget would exhaust it at that instant,
|
||||||
// and fx would skip every hook after the server — the delivery
|
// and fx would skip every hook after the server — the delivery
|
||||||
// engine, the healthcheck, the webhook DB manager and the database
|
// engine, the healthcheck, the webhook DB manager and the database
|
||||||
// close. That hook is the 3s HTTP drain plus the Sentry flush that
|
// close. That hook is the HTTP drain plus the Sentry flush that
|
||||||
// follows it in the same hook, so the flush is clamped to the stop
|
// follows it in the same hook, and each is clamped to the stop
|
||||||
// context's remaining time less server.TailHookReserve rather than
|
// context's remaining time less server.TailHookReserve rather than
|
||||||
// running for its own fixed 2s; the reserve is what the tail hooks
|
// running for its own fixed 3s and 2s; the reserve is what the tail
|
||||||
// live on, and they are microsecond-scale in normal operation.
|
// hooks live on, and they are microsecond-scale in normal operation.
|
||||||
// TestStopTimeout_LeavesHeadroomForTailHooks pins the arithmetic
|
// TestStopTimeout_LeavesHeadroomForTailHooks pins the arithmetic
|
||||||
// across every drain length.
|
// across every drain length and every amount of budget the hooks
|
||||||
|
// before the server may already have spent.
|
||||||
//
|
//
|
||||||
// This does not make the database close unconditional: the
|
// This does not make the database close unconditional: the
|
||||||
// ArchiveSweeper and RetentionReaper hooks run before the server
|
// ArchiveSweeper and RetentionReaper hooks run before the server.
|
||||||
// and can still consume the whole budget on their own.
|
// What they spend comes out of the drain first, but past 3s it comes
|
||||||
|
// out of the reserve, and they can consume the whole budget.
|
||||||
const stopTimeout = 5 * time.Second
|
const stopTimeout = 5 * time.Second
|
||||||
|
|
||||||
// exitUsage is the status for a command line this binary cannot make
|
// exitUsage is the status for a command line this binary cannot make
|
||||||
|
|||||||
@@ -252,22 +252,40 @@ const tailHeadroom = 2 * time.Second
|
|||||||
// can produce, since a shorter drain leaves the flush more room and
|
// can produce, since a shorter drain leaves the flush more room and
|
||||||
// the worst case is not necessarily at either extreme.
|
// the worst case is not necessarily at either extreme.
|
||||||
//
|
//
|
||||||
// Shrinking either budget, or unbounding the flush again, must fail
|
// Nor does the hook start on a full budget: the ArchiveSweeper and
|
||||||
// here rather than silently recreating a hook that swallows the
|
// RetentionReaper hooks run before it, and whatever they spent is
|
||||||
// whole sequence.
|
// gone. The outer sweep walks every amount they can spend. Once they
|
||||||
|
// have eaten into the headroom themselves, the hook must spend
|
||||||
|
// nothing of what is left. A drain that starts on the full budget
|
||||||
|
// must still get all of ShutdownTimeout, so a smaller stopTimeout
|
||||||
|
// cannot silently shorten every drain.
|
||||||
|
//
|
||||||
|
// Shrinking either budget, or unbounding the drain or the flush
|
||||||
|
// again, must fail here rather than silently recreating a hook that
|
||||||
|
// swallows the whole sequence.
|
||||||
func TestStopTimeout_LeavesHeadroomForTailHooks(t *testing.T) {
|
func TestStopTimeout_LeavesHeadroomForTailHooks(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
require.Less(t, server.ShutdownTimeout, stopTimeout)
|
require.Less(t, server.ShutdownTimeout, stopTimeout)
|
||||||
|
require.Equal(
|
||||||
|
t, server.ShutdownTimeout, server.DrainBudget(stopTimeout),
|
||||||
|
"a drain that starts on the full stop budget is cut short",
|
||||||
|
)
|
||||||
|
|
||||||
const step = 10 * time.Millisecond
|
const step = 10 * time.Millisecond
|
||||||
|
|
||||||
for drain := time.Duration(0); drain <= server.ShutdownTimeout; drain += step {
|
for spent := time.Duration(0); spent <= stopTimeout; spent += step {
|
||||||
hook := drain + server.SentryFlushBudget(stopTimeout-drain)
|
remaining := stopTimeout - spent
|
||||||
|
longest := max(server.DrainBudget(remaining), 0)
|
||||||
|
|
||||||
require.LessOrEqual(
|
for drain := time.Duration(0); drain <= longest; drain += step {
|
||||||
t, hook+tailHeadroom, stopTimeout,
|
hook := drain + server.SentryFlushBudget(remaining-drain)
|
||||||
"a %s drain leaves the tail hooks short", drain,
|
|
||||||
)
|
require.GreaterOrEqual(
|
||||||
|
t, remaining-hook, min(remaining, tailHeadroom),
|
||||||
|
"a %s drain after %s of earlier hooks leaves "+
|
||||||
|
"the tail hooks short", drain, spent,
|
||||||
|
)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1425,6 +1425,32 @@ func TestDeliverHTTP_InvalidConfig(t *testing.T) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestDeliverHTTP_InvalidConfigUnrecordedStaysPending: a delivery is
|
||||||
|
// failed for an invalid config only once the reason is recorded.
|
||||||
|
// Unrecorded, it stays pending, where the sweep finds it again.
|
||||||
|
func TestDeliverHTTP_InvalidConfigUnrecordedStaysPending(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
db := testWebhookDB(t)
|
||||||
|
e := testEngine(t, 1)
|
||||||
|
|
||||||
|
event, del := iSeedEventAndDelivery(
|
||||||
|
t, db, `{"config":"invalid"}`, "",
|
||||||
|
)
|
||||||
|
|
||||||
|
task, d := iHTTPTaskAndDelivery(
|
||||||
|
event, del, "bad-config", `not-json`, 0, 1,
|
||||||
|
)
|
||||||
|
|
||||||
|
require.NoError(t, db.Exec("drop table delivery_results").Error)
|
||||||
|
|
||||||
|
e.ExportDeliverHTTP(context.TODO(), db, d, task)
|
||||||
|
|
||||||
|
iAssertStatus(t, db, del.ID,
|
||||||
|
database.DeliveryStatusPending,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
// --- Notify batching ---
|
// --- Notify batching ---
|
||||||
|
|
||||||
func TestNotify_MultipleTasks(t *testing.T) {
|
func TestNotify_MultipleTasks(t *testing.T) {
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"io"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
@@ -1056,6 +1057,21 @@ func TestParseHTTPConfig_MissingURL(t *testing.T) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestParseHTTPConfig_Undecodable(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
e := testEngine(t, 1)
|
||||||
|
|
||||||
|
_, err := e.ExportParseHTTPConfig(
|
||||||
|
`{"url":"https://example.com/hook","timeout":"soon"}`,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.Error(t, err,
|
||||||
|
"config that does not decode should return error, "+
|
||||||
|
"even when the part that did names a URL",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
func TestScheduleRetry_SendsToRetryChannel(
|
func TestScheduleRetry_SendsToRetryChannel(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
) {
|
) {
|
||||||
@@ -1241,6 +1257,33 @@ func TestDoHTTPRequest_ForwardsHeaders(t *testing.T) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A response that ends before the length it announced is an error, not
|
||||||
|
// a short body.
|
||||||
|
func TestDoHTTPRequest_CutShortResponseIsAnError(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
ts := httptest.NewServer(
|
||||||
|
http.HandlerFunc(
|
||||||
|
func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
w.Header().Set("Content-Length", "100")
|
||||||
|
_, _ = w.Write([]byte("cut short"))
|
||||||
|
},
|
||||||
|
),
|
||||||
|
)
|
||||||
|
defer ts.Close()
|
||||||
|
|
||||||
|
e := testEngine(t, 1)
|
||||||
|
|
||||||
|
_, body, _, err := e.ExportDoHTTPRequest(
|
||||||
|
context.TODO(),
|
||||||
|
&delivery.HTTPTargetConfig{URL: ts.URL},
|
||||||
|
&database.Event{},
|
||||||
|
)
|
||||||
|
|
||||||
|
require.ErrorIs(t, err, io.ErrUnexpectedEOF)
|
||||||
|
assert.Empty(t, body)
|
||||||
|
}
|
||||||
|
|
||||||
// The event's stored inbound headers carry the same Content-Type the
|
// The event's stored inbound headers carry the same Content-Type the
|
||||||
// receiver saved as the event's ContentType, so a delivery could send
|
// receiver saved as the event's ContentType, so a delivery could send
|
||||||
// it twice. It must go out exactly once, with a Content-Type configured
|
// it twice. It must go out exactly once, with a Content-Type configured
|
||||||
@@ -1317,6 +1360,34 @@ func TestApplyRequestHeaders_SendsOneContentType(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Stored inbound headers that do not decode forward nothing, not the
|
||||||
|
// part of them that happened to decode.
|
||||||
|
func TestApplyRequestHeaders_UndecodableInboundForwardsNothing(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
req, err := http.NewRequestWithContext(
|
||||||
|
context.Background(),
|
||||||
|
http.MethodPost,
|
||||||
|
"https://target.example.com/hook",
|
||||||
|
http.NoBody,
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
names := delivery.ExportApplyRequestHeaders(
|
||||||
|
req,
|
||||||
|
&database.Event{
|
||||||
|
Headers: `{"X-Custom":["value1"],"X-Broken":"not a list"}`,
|
||||||
|
},
|
||||||
|
&delivery.HTTPTargetConfig{},
|
||||||
|
"webhooker/dev",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.Empty(t, names)
|
||||||
|
assert.Empty(t, req.Header.Get("X-Custom"))
|
||||||
|
}
|
||||||
|
|
||||||
func TestProcessDelivery_RoutesToCorrectHandler(
|
func TestProcessDelivery_RoutesToCorrectHandler(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
) {
|
) {
|
||||||
|
|||||||
@@ -376,3 +376,97 @@ func TestFailedResultWriteLeavesDeliveryRecoverable(
|
|||||||
database.DeliveryStatusPending,
|
database.DeliveryStatusPending,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestFailedResultWriteWithRetriesLeavesDeliveryRecoverable is the same
|
||||||
|
// rule for a target with retries: whatever the receiver answered, the
|
||||||
|
// delivery stays pending and no retry is scheduled. The circuit breaker
|
||||||
|
// still learns the answer, because it describes the target's health,
|
||||||
|
// not the database's.
|
||||||
|
func TestFailedResultWriteWithRetriesLeavesDeliveryRecoverable(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
// The "send succeeded" case starts with the breaker tripped open,
|
||||||
|
// so the delivery goes out as its probe and only a recorded
|
||||||
|
// success closes it again.
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
answer int
|
||||||
|
tripped bool
|
||||||
|
wantBreaker delivery.CircuitState
|
||||||
|
}{
|
||||||
|
{"send succeeded", http.StatusOK, true, delivery.CircuitClosed},
|
||||||
|
{"send failed", http.StatusBadGateway, false, delivery.CircuitOpen},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range tests {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s := newISetup(t)
|
||||||
|
targetID := uuid.New().String()
|
||||||
|
|
||||||
|
ts := httptest.NewServer(http.HandlerFunc(
|
||||||
|
func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
w.WriteHeader(tc.answer)
|
||||||
|
},
|
||||||
|
))
|
||||||
|
defer ts.Close()
|
||||||
|
|
||||||
|
event := iSeedEvent(
|
||||||
|
t, s.WebhookDB, s.WebhookID, `{"unwritable":true}`,
|
||||||
|
)
|
||||||
|
|
||||||
|
d := iSeedDelivery(
|
||||||
|
t, s.WebhookDB, event.ID, targetID,
|
||||||
|
database.DeliveryStatusPending,
|
||||||
|
)
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t,
|
||||||
|
s.WebhookDB.Exec("drop table delivery_results").Error,
|
||||||
|
)
|
||||||
|
|
||||||
|
// A single failure opens this breaker, and with no
|
||||||
|
// cooldown an open breaker lets the next delivery
|
||||||
|
// through as a probe.
|
||||||
|
cb := delivery.NewTestCircuitBreaker(1, 0)
|
||||||
|
if tc.tripped {
|
||||||
|
cb.RecordFailure()
|
||||||
|
}
|
||||||
|
|
||||||
|
s.Engine.ExportSetCircuitBreaker(targetID, cb)
|
||||||
|
|
||||||
|
full := &database.Delivery{
|
||||||
|
EventID: event.ID,
|
||||||
|
TargetID: targetID,
|
||||||
|
Status: database.DeliveryStatusPending,
|
||||||
|
Event: event,
|
||||||
|
Target: database.Target{
|
||||||
|
Name: "unwritable",
|
||||||
|
Type: database.TargetTypeHTTP,
|
||||||
|
Config: iHTTPConfig(ts.URL),
|
||||||
|
MaxRetries: 3,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
full.ID = d.ID
|
||||||
|
|
||||||
|
sched := &recordingScheduler{}
|
||||||
|
|
||||||
|
s.Engine.ExportDeliverHTTPWithScheduler(
|
||||||
|
context.Background(), s.WebhookDB, full,
|
||||||
|
&delivery.Task{
|
||||||
|
DeliveryID: d.ID,
|
||||||
|
TargetID: targetID,
|
||||||
|
AttemptNum: 1,
|
||||||
|
},
|
||||||
|
sched,
|
||||||
|
)
|
||||||
|
|
||||||
|
iAssertStatus(t, s.WebhookDB, d.ID, database.DeliveryStatusPending)
|
||||||
|
assert.Empty(t, sched.delays, "no retry may be scheduled")
|
||||||
|
assert.Equal(t, tc.wantBreaker, cb.State())
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -179,6 +179,27 @@ func TestDoHTTPRequest_TransportErrorMasksURL(t *testing.T) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestDoHTTPRequest_UnparsableURLIsMasked is the same for an HTTP
|
||||||
|
// target URL that no request can be built from.
|
||||||
|
func TestDoHTTPRequest_UnparsableURLIsMasked(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
e := testEngine(t, 1)
|
||||||
|
|
||||||
|
statusCode, _, _, reqErr := e.ExportDoHTTPRequest(
|
||||||
|
context.TODO(),
|
||||||
|
&delivery.HTTPTargetConfig{
|
||||||
|
URL: "https://hooks.example.com" + maskSecretPath + "\n",
|
||||||
|
},
|
||||||
|
&database.Event{},
|
||||||
|
)
|
||||||
|
require.Error(t, reqErr)
|
||||||
|
assert.Zero(t, statusCode)
|
||||||
|
|
||||||
|
assertNoCredential(t, reqErr.Error())
|
||||||
|
assert.Contains(t, reqErr.Error(), "invalid control character")
|
||||||
|
}
|
||||||
|
|
||||||
// TestValidateTargetURL_UnparsableURLIsMasked proves the SSRF
|
// TestValidateTargetURL_UnparsableURLIsMasked proves the SSRF
|
||||||
// validator's error does not carry the submitted URL, which
|
// validator's error does not carry the submitted URL, which
|
||||||
// the handler both logs and shows.
|
// the handler both logs and shows.
|
||||||
|
|||||||
@@ -14,18 +14,16 @@ import (
|
|||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
)
|
)
|
||||||
|
|
||||||
// minNonTestFiles guards the walk below against passing because it
|
// isRowProducer reports whether name is GORM's Row or database/sql's
|
||||||
// found nothing to look at. The tree held 60 non-test .go files when
|
// QueryRow or QueryRowContext, which return a *sql.Row whose Scan is
|
||||||
// this was written.
|
// database/sql's and not (*gorm.DB).Scan. GORM's Rows is not listed:
|
||||||
const minNonTestFiles = 40
|
// it also returns an error, so Scan is never called on its result
|
||||||
|
// directly. It matches the method name only and resolves no types, so
|
||||||
// isRowProducer reports whether name is a method that returns a
|
// a repo-local method with one of these names that returns *gorm.DB
|
||||||
// database/sql row handle. GORM's Row and Rows return *sql.Row and
|
// gets past it: Scan on that method's result is not reported.
|
||||||
// *sql.Rows, so Scan on the result of one of them is database/sql's
|
|
||||||
// Scan and never (*gorm.DB).Scan.
|
|
||||||
func isRowProducer(name string) bool {
|
func isRowProducer(name string) bool {
|
||||||
switch name {
|
switch name {
|
||||||
case "Row", "Rows", "QueryRow", "QueryRowContext":
|
case "Row", "QueryRow", "QueryRowContext":
|
||||||
return true
|
return true
|
||||||
default:
|
default:
|
||||||
return false
|
return false
|
||||||
@@ -50,9 +48,14 @@ func receiverIsRowHandle(x ast.Expr) bool {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// unguardedScans returns the position of every Scan call in file whose
|
// unguardedScans returns the position of every Scan call in file whose
|
||||||
// receiver is not a row handle. It fails closed: a receiver it cannot
|
// receiver is not a call to a row producer. It fails closed: any other
|
||||||
// resolve syntactically — a local variable, a struct field — is
|
// receiver — a local variable, a struct field, a call to any other
|
||||||
// reported rather than assumed safe.
|
// method — is reported rather than assumed safe.
|
||||||
|
//
|
||||||
|
// It sees only calls written x.Scan(...). A method value, f := db.Scan
|
||||||
|
// followed by f(&v), is out of scope: Scan is never the called
|
||||||
|
// expression there, and nobody writes a query that way by accident,
|
||||||
|
// which is the mistake this check exists to catch.
|
||||||
func unguardedScans(
|
func unguardedScans(
|
||||||
fset *token.FileSet, file *ast.File,
|
fset *token.FileSet, file *ast.File,
|
||||||
) []token.Position {
|
) []token.Position {
|
||||||
@@ -111,15 +114,15 @@ func skipDir(name string) bool {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// walkNonTestGo parses every non-test .go file under root and returns
|
// walkNonTestGo parses every non-test .go file under root. It returns
|
||||||
// how many it parsed along with every unguarded Scan it found.
|
// the directories, relative to root, it parsed a file in, along with
|
||||||
func walkNonTestGo(t *testing.T, root string) (int, []string) {
|
// every unguarded Scan it found.
|
||||||
|
func walkNonTestGo(t *testing.T, root string) (map[string]bool, []string) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
var (
|
walked := map[string]bool{}
|
||||||
parsed int
|
|
||||||
hits []string
|
var hits []string
|
||||||
)
|
|
||||||
|
|
||||||
fset := token.NewFileSet()
|
fset := token.NewFileSet()
|
||||||
|
|
||||||
@@ -147,7 +150,12 @@ func walkNonTestGo(t *testing.T, root string) (int, []string) {
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
parsed++
|
dir, err := filepath.Rel(root, filepath.Dir(path))
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
walked[dir] = true
|
||||||
|
|
||||||
for _, pos := range unguardedScans(fset, file) {
|
for _, pos := range unguardedScans(fset, file) {
|
||||||
hits = append(hits, relPosition(root, pos))
|
hits = append(hits, relPosition(root, pos))
|
||||||
@@ -157,7 +165,7 @@ func walkNonTestGo(t *testing.T, root string) (int, []string) {
|
|||||||
},
|
},
|
||||||
))
|
))
|
||||||
|
|
||||||
return parsed, hits
|
return walked, hits
|
||||||
}
|
}
|
||||||
|
|
||||||
// isNonTestGo reports whether a file name is Go source this check
|
// isNonTestGo reports whether a file name is Go source this check
|
||||||
@@ -189,19 +197,39 @@ func relPosition(root string, pos token.Position) string {
|
|||||||
// logged with its values interpolated. The package comment states the
|
// logged with its values interpolated. The package comment states the
|
||||||
// limit; this fails when someone adds a call site anyway.
|
// limit; this fails when someone adds a call site anyway.
|
||||||
//
|
//
|
||||||
// The current tree has one caller, internal/database/database_test.go,
|
// Test files are not governed: what a test binds is fixture data.
|
||||||
// which this check does not govern: it is test-only and its SELECT 1
|
|
||||||
// binds nothing.
|
|
||||||
func TestGormScanIsNeverCalledOutsideTests(t *testing.T) {
|
func TestGormScanIsNeverCalledOutsideTests(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
parsed, offenders := walkNonTestGo(t, moduleRoot(t))
|
root := moduleRoot(t)
|
||||||
|
walked, offenders := walkNonTestGo(t, root)
|
||||||
|
|
||||||
|
// The module's packages are static, templates, and every directory
|
||||||
|
// directly under cmd and internal. Each holds non-test code, so one
|
||||||
|
// the walk parsed nothing in was skipped, and a Scan there would
|
||||||
|
// pass unseen.
|
||||||
|
packages := []string{"static", "templates"}
|
||||||
|
|
||||||
|
for _, parent := range []string{"cmd", "internal"} {
|
||||||
|
entries, err := os.ReadDir(filepath.Join(root, parent))
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
for _, entry := range entries {
|
||||||
|
if !entry.IsDir() {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
packages = append(packages, filepath.Join(parent, entry.Name()))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, dir := range packages {
|
||||||
|
require.True(
|
||||||
|
t, walked[dir],
|
||||||
|
"the walk parsed no non-test .go file in %s", dir,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
require.GreaterOrEqual(
|
|
||||||
t, parsed, minNonTestFiles,
|
|
||||||
"parsed %d non-test .go files, so this check found "+
|
|
||||||
"nothing to look at", parsed,
|
|
||||||
)
|
|
||||||
require.Empty(
|
require.Empty(
|
||||||
t, offenders,
|
t, offenders,
|
||||||
"Scan called on a receiver this check cannot show is a "+
|
"Scan called on a receiver this check cannot show is a "+
|
||||||
@@ -222,18 +250,51 @@ type scanGuardCase struct {
|
|||||||
want int
|
want int
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// scanGuardCases covers each receiver form unguardedScans names, plus
|
||||||
|
// each row producer isRowProducer lets through. Each body is valid Go
|
||||||
|
// inside plantedFile.
|
||||||
func scanGuardCases() []scanGuardCase {
|
func scanGuardCases() []scanGuardCase {
|
||||||
return []scanGuardCase{
|
return []scanGuardCase{
|
||||||
{"gorm chain", `db.DB().Raw("SELECT 1").Scan(&v)`, 1},
|
{"local variable", "q := gdb.Raw(\"SELECT 1\")\n\tq.Scan(&v)", 1},
|
||||||
{"gorm receiver", `gdb.Scan(&v)`, 1},
|
{"struct field", `s.db.Scan(&v)`, 1},
|
||||||
{"gorm via variable", "q := gdb.Raw(\"x\")\nq.Scan(&v)", 1},
|
{"gorm chain", `gdb.Raw("SELECT 1").Scan(&v)`, 1},
|
||||||
{"gorm model chain", `gdb.Model(&x).Scan(&v)`, 1},
|
{
|
||||||
{"sql row", `gdb.Raw("SELECT 1").Row().Scan(&v)`, 0},
|
"sql rows in a variable",
|
||||||
{"sql rows", `gdb.Raw("SELECT 1").Rows().Scan(&v)`, 0},
|
"rows, _ := gdb.Raw(\"SELECT 1\").Rows()\n\trows.Scan(&v)",
|
||||||
|
1,
|
||||||
|
},
|
||||||
|
{"gorm Row", `gdb.Raw("SELECT 1").Row().Scan(&v)`, 0},
|
||||||
|
{"sql QueryRow", `sqlDB.QueryRow("SELECT 1").Scan(&v)`, 0},
|
||||||
|
{
|
||||||
|
"sql QueryRowContext",
|
||||||
|
`sqlDB.QueryRowContext(ctx, "SELECT 1").Scan(&v)`,
|
||||||
|
0,
|
||||||
|
},
|
||||||
{"unrelated call", `gdb.Find(&v)`, 0},
|
{"unrelated call", `gdb.Find(&v)`, 0},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// plantedFile wraps one case body in a function that declares every
|
||||||
|
// name the bodies use, so each body is the Go it stands for. The result
|
||||||
|
// is parsed, never compiled.
|
||||||
|
const plantedFile = `package p
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
|
||||||
|
"gorm.io/gorm"
|
||||||
|
)
|
||||||
|
|
||||||
|
type store struct{ db *gorm.DB }
|
||||||
|
|
||||||
|
func f(ctx context.Context, gdb *gorm.DB, sqlDB *sql.DB, s store) {
|
||||||
|
var v int
|
||||||
|
|
||||||
|
%s
|
||||||
|
}
|
||||||
|
`
|
||||||
|
|
||||||
// TestScanGuard_ReportsPlantedCalls proves the check fires. Without it
|
// TestScanGuard_ReportsPlantedCalls proves the check fires. Without it
|
||||||
// a detector that matched nothing would satisfy the walk above no
|
// a detector that matched nothing would satisfy the walk above no
|
||||||
// matter what the tree contained.
|
// matter what the tree contained.
|
||||||
@@ -245,9 +306,7 @@ func TestScanGuard_ReportsPlantedCalls(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
fset := token.NewFileSet()
|
fset := token.NewFileSet()
|
||||||
src := fmt.Sprintf(
|
src := fmt.Sprintf(plantedFile, tc.body)
|
||||||
"package p\n\nfunc f() {\n\t%s\n}\n", tc.body,
|
|
||||||
)
|
|
||||||
|
|
||||||
file, err := parser.ParseFile(
|
file, err := parser.ParseFile(
|
||||||
fset, tc.name+".go", src, 0,
|
fset, tc.name+".go", src, 0,
|
||||||
|
|||||||
@@ -0,0 +1,95 @@
|
|||||||
|
package handlers_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/url"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/go-chi/chi"
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"gorm.io/gorm"
|
||||||
|
"sneak.berlin/go/webhooker/internal/database"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestHandleEntrypointToggle_DoesNotUndoAnEdit proves that a toggle
|
||||||
|
// which loaded the entrypoint before an edit of its description was
|
||||||
|
// saved does not write the old description back over the edit. The
|
||||||
|
// edit is submitted from a callback on the toggle's own read of the
|
||||||
|
// entrypoint, so it is saved after that read and before the toggle
|
||||||
|
// writes.
|
||||||
|
func TestHandleEntrypointToggle_DoesNotUndoAnEdit(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
env := setupSourceTest(t)
|
||||||
|
wh := seedWebhookWithRetention(t, env.db, 30)
|
||||||
|
ep := seedEntrypoint(t, env.db, wh.ID)
|
||||||
|
require.True(t, ep.Active)
|
||||||
|
|
||||||
|
router := chi.NewRouter()
|
||||||
|
router.Post(
|
||||||
|
"/hook/{sourceID}/entrypoints/{entrypointID}/edit",
|
||||||
|
env.handlers.HandleEntrypointEdit(),
|
||||||
|
)
|
||||||
|
router.Post(
|
||||||
|
"/hook/{sourceID}/entrypoints/{entrypointID}/toggle",
|
||||||
|
env.handlers.HandleEntrypointToggle(),
|
||||||
|
)
|
||||||
|
|
||||||
|
// post submits one of the entrypoint's forms as the test user and
|
||||||
|
// returns the response's status code.
|
||||||
|
post := func(action string, form url.Values) int {
|
||||||
|
req := httptest.NewRequestWithContext(
|
||||||
|
context.Background(), http.MethodPost,
|
||||||
|
"/hook/"+wh.ID+"/entrypoints/"+ep.ID+"/"+action,
|
||||||
|
strings.NewReader(form.Encode()),
|
||||||
|
)
|
||||||
|
req.Header.Set(
|
||||||
|
"Content-Type", "application/x-www-form-urlencoded",
|
||||||
|
)
|
||||||
|
|
||||||
|
for _, c := range env.cookies {
|
||||||
|
req.AddCookie(c)
|
||||||
|
}
|
||||||
|
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
router.ServeHTTP(w, req)
|
||||||
|
|
||||||
|
return w.Code
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
edited bool
|
||||||
|
editCode int
|
||||||
|
)
|
||||||
|
|
||||||
|
require.NoError(t, env.db.DB().Callback().Query().
|
||||||
|
After("gorm:query").
|
||||||
|
Register("test:edit_after_toggle_read", func(tx *gorm.DB) {
|
||||||
|
// Only the first read of an entrypoint, the toggle's,
|
||||||
|
// submits the edit.
|
||||||
|
if tx.Statement.Table != "entrypoints" || edited {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
edited = true
|
||||||
|
editCode = post(
|
||||||
|
"edit", url.Values{"description": {"Billing sender"}},
|
||||||
|
)
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
|
||||||
|
require.Equal(t, http.StatusSeeOther, post("toggle", nil))
|
||||||
|
require.Equal(t, http.StatusSeeOther, editCode)
|
||||||
|
|
||||||
|
var stored database.Entrypoint
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t, env.db.DB().First(&stored, "id = ?", ep.ID).Error,
|
||||||
|
)
|
||||||
|
assert.False(t, stored.Active)
|
||||||
|
assert.Equal(t, "Billing sender", stored.Description)
|
||||||
|
}
|
||||||
@@ -15,10 +15,12 @@ import (
|
|||||||
// eventBodyQuery reads one event's stored body as bytes. The cast
|
// eventBodyQuery reads one event's stored body as bytes. The cast
|
||||||
// to blob is what makes the driver hand back the stored bytes
|
// to blob is what makes the driver hand back the stored bytes
|
||||||
// rather than a string conversion, so Content-Length taken from
|
// rather than a string conversion, so Content-Length taken from
|
||||||
// the result matches what goes on the wire. The soft-delete
|
// the result matches what goes on the wire. The retention reaper
|
||||||
// predicate is spelled out because Raw bypasses GORM's default
|
// deletes event rows outright, so a reaped event is simply gone
|
||||||
// scope, and it is what stops a reaped event still being
|
// and the query finds no row. The deleted_at predicate repeats
|
||||||
// downloadable.
|
// the soft-delete scope GORM adds to its own queries, which Raw
|
||||||
|
// bypasses; nothing soft-deletes an event, so today it excludes
|
||||||
|
// nothing.
|
||||||
const eventBodyQuery = "SELECT cast(body as blob) " +
|
const eventBodyQuery = "SELECT cast(body as blob) " +
|
||||||
"FROM events WHERE id = ? AND webhook_id = ? AND deleted_at IS NULL"
|
"FROM events WHERE id = ? AND webhook_id = ? AND deleted_at IS NULL"
|
||||||
|
|
||||||
|
|||||||
@@ -405,10 +405,11 @@ func TestHandleEventBodyDownload_UnknownEvent404s(t *testing.T) {
|
|||||||
// route. The body is read in one query before any header is
|
// route. The body is read in one query before any header is
|
||||||
// written, so a reaped event cannot produce a partial download:
|
// written, so a reaped event cannot produce a partial download:
|
||||||
// it is a clean 404 with no Content-Length and no
|
// it is a clean 404 with no Content-Length and no
|
||||||
// Content-Disposition. Both removals the codebase performs are
|
// Content-Disposition. The reaper deletes event rows outright,
|
||||||
// covered — the reaper hard-deletes, and a soft-deleted row is
|
// which is the "hard deleted" case. The "soft deleted" case
|
||||||
// excluded by the query's own deleted_at predicate rather than
|
// covers a row no code produces today: it only pins the query's
|
||||||
// by GORM's default scope, which Raw bypasses.
|
// own deleted_at predicate, the soft-delete condition Raw would
|
||||||
|
// otherwise skip.
|
||||||
func TestHandleEventBodyDownload_ReapedEvent404s(t *testing.T) {
|
func TestHandleEventBodyDownload_ReapedEvent404s(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -145,8 +145,9 @@ func (h *Handlers) resubmitEvent(
|
|||||||
// per-webhook database files — a sibling webhook's event is not in the
|
// per-webhook database files — a sibling webhook's event is not in the
|
||||||
// database being queried at all — and is there so the scoping survives
|
// database being queried at all — and is there so the scoping survives
|
||||||
// any future change that puts more than one webhook's events in one
|
// any future change that puts more than one webhook's events in one
|
||||||
// file. Going through Model applies GORM's soft-delete scope, which is
|
// file. A reaped event is not found because the retention reaper
|
||||||
// what stops a reaped event being resubmitted.
|
// deletes its row outright rather than marking it deleted; see
|
||||||
|
// deleteEvents in internal/database/retention.go.
|
||||||
func loadResubmitSource(
|
func loadResubmitSource(
|
||||||
webhookDB *gorm.DB,
|
webhookDB *gorm.DB,
|
||||||
webhookID, eventID string,
|
webhookID, eventID string,
|
||||||
|
|||||||
@@ -20,6 +20,7 @@ const (
|
|||||||
webhookSaved noticeCode = "webhook-saved"
|
webhookSaved noticeCode = "webhook-saved"
|
||||||
webhookDeleted noticeCode = "webhook-deleted"
|
webhookDeleted noticeCode = "webhook-deleted"
|
||||||
entrypointAdded noticeCode = "entrypoint-added"
|
entrypointAdded noticeCode = "entrypoint-added"
|
||||||
|
entrypointSaved noticeCode = "entrypoint-saved"
|
||||||
entrypointDeleted noticeCode = "entrypoint-deleted"
|
entrypointDeleted noticeCode = "entrypoint-deleted"
|
||||||
entrypointActivated noticeCode = "entrypoint-activated"
|
entrypointActivated noticeCode = "entrypoint-activated"
|
||||||
entrypointDeactivated noticeCode = "entrypoint-deactivated"
|
entrypointDeactivated noticeCode = "entrypoint-deactivated"
|
||||||
@@ -48,6 +49,7 @@ func noticeFor(r *http.Request) *notice {
|
|||||||
webhookSaved: {Text: "Webhook saved."},
|
webhookSaved: {Text: "Webhook saved."},
|
||||||
webhookDeleted: {Text: "Webhook deleted."},
|
webhookDeleted: {Text: "Webhook deleted."},
|
||||||
entrypointAdded: {Text: "Entrypoint added."},
|
entrypointAdded: {Text: "Entrypoint added."},
|
||||||
|
entrypointSaved: {Text: "Entrypoint description saved."},
|
||||||
entrypointDeleted: {Text: "Entrypoint deleted."},
|
entrypointDeleted: {Text: "Entrypoint deleted."},
|
||||||
entrypointActivated: {Text: "Entrypoint activated."},
|
entrypointActivated: {Text: "Entrypoint activated."},
|
||||||
entrypointDeactivated: {Text: "Entrypoint deactivated."},
|
entrypointDeactivated: {Text: "Entrypoint deactivated."},
|
||||||
|
|||||||
@@ -86,12 +86,13 @@ var errInjectedDelete = errors.New("injected delete failure")
|
|||||||
// save of an existing row.
|
// save of an existing row.
|
||||||
var errInjectedSave = errors.New("injected save failure")
|
var errInjectedSave = errors.New("injected save failure")
|
||||||
|
|
||||||
// seedEntrypoint inserts an entrypoint for a webhook.
|
// seedEntrypoint inserts an active entrypoint for a webhook and
|
||||||
|
// returns it.
|
||||||
func seedEntrypoint(
|
func seedEntrypoint(
|
||||||
t *testing.T,
|
t *testing.T,
|
||||||
db *database.Database,
|
db *database.Database,
|
||||||
webhookID string,
|
webhookID string,
|
||||||
) {
|
) *database.Entrypoint {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
ep := &database.Entrypoint{
|
ep := &database.Entrypoint{
|
||||||
@@ -104,6 +105,8 @@ func seedEntrypoint(
|
|||||||
t,
|
t,
|
||||||
db.DB().Omit(clause.Associations).Create(ep).Error,
|
db.DB().Omit(clause.Associations).Create(ep).Error,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
return ep
|
||||||
}
|
}
|
||||||
|
|
||||||
// countRows counts the live (not soft-deleted) rows of a model
|
// countRows counts the live (not soft-deleted) rows of a model
|
||||||
|
|||||||
@@ -1396,6 +1396,70 @@ func (h *Handlers) HandleEntrypointCreate() http.HandlerFunc {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// HandleEntrypointEdit handles changing an entrypoint's description.
|
||||||
|
// It writes only the description column, so the entrypoint keeps its
|
||||||
|
// URL, and an activate or deactivate saved since the page was shown
|
||||||
|
// is not undone.
|
||||||
|
func (h *Handlers) HandleEntrypointEdit() http.HandlerFunc {
|
||||||
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
userID, ok := h.getUserID(r)
|
||||||
|
if !ok {
|
||||||
|
http.Redirect(
|
||||||
|
w, r, "/pages/login", http.StatusSeeOther,
|
||||||
|
)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
sourceID := chi.URLParam(r, "sourceID")
|
||||||
|
entrypointID := chi.URLParam(r, "entrypointID")
|
||||||
|
|
||||||
|
var webhook database.Webhook
|
||||||
|
|
||||||
|
err := h.db.DB().Where(
|
||||||
|
"id = ? AND user_id = ?", sourceID, userID,
|
||||||
|
).First(&webhook).Error
|
||||||
|
if err != nil {
|
||||||
|
h.renderError(w, r, http.StatusNotFound)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// The body size cap is enforced by the MaxBodySize
|
||||||
|
// middleware, which runs before CSRF parses the form.
|
||||||
|
err = r.ParseForm()
|
||||||
|
if err != nil {
|
||||||
|
h.renderError(w, r, http.StatusBadRequest)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
result := h.db.DB().Model(&database.Entrypoint{}).Where(
|
||||||
|
"id = ? AND webhook_id = ?", entrypointID, webhook.ID,
|
||||||
|
).Update("description", r.PostFormValue("description"))
|
||||||
|
if result.Error != nil {
|
||||||
|
h.serverError(
|
||||||
|
w, r, "failed to edit entrypoint", result.Error,
|
||||||
|
)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// The id came from the URL and may name another webhook's
|
||||||
|
// entrypoint, which this webhook does not have.
|
||||||
|
if result.RowsAffected == 0 {
|
||||||
|
h.renderError(w, r, http.StatusNotFound)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
http.Redirect(
|
||||||
|
w, r, withNotice("/hook/"+webhook.ID, entrypointSaved),
|
||||||
|
http.StatusSeeOther,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// HandleTargetCreate handles adding a new target to a webhook.
|
// HandleTargetCreate handles adding a new target to a webhook.
|
||||||
func (h *Handlers) HandleTargetCreate() http.HandlerFunc {
|
func (h *Handlers) HandleTargetCreate() http.HandlerFunc {
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
@@ -1877,9 +1941,13 @@ func (h *Handlers) HandleEntrypointToggle() http.HandlerFunc {
|
|||||||
return false, err
|
return false, err
|
||||||
}
|
}
|
||||||
|
|
||||||
ep.Active = !ep.Active
|
// Only the active column: saving the whole row would
|
||||||
|
// write back the description read above over an edit
|
||||||
|
// saved since.
|
||||||
|
active := !ep.Active
|
||||||
|
|
||||||
return ep.Active, h.db.DB().Save(&ep).Error
|
return active, h.db.DB().Model(&ep).
|
||||||
|
Update("active", active).Error
|
||||||
},
|
},
|
||||||
"failed to toggle entrypoint",
|
"failed to toggle entrypoint",
|
||||||
entrypointActivated, entrypointDeactivated,
|
entrypointActivated, entrypointDeactivated,
|
||||||
|
|||||||
@@ -272,10 +272,12 @@ func requestEventSource(
|
|||||||
|
|
||||||
// createAndFanOut writes the event and one pending delivery per target,
|
// createAndFanOut writes the event and one pending delivery per target,
|
||||||
// and adds them to the webhook's running totals, in a single
|
// and adds them to the webhook's running totals, in a single
|
||||||
// transaction, then hands the tasks to the delivery engine. It is the
|
// transaction, then hands the tasks to the delivery engine. Every
|
||||||
// only path by which an event and its deliveries are created, so a
|
// event is created here, received or resubmitted, so a resubmitted
|
||||||
// resubmitted event is retried, SSRF-guarded and circuit-broken
|
// event is retried, SSRF-guarded and circuit-broken exactly as a
|
||||||
// exactly as a received one is.
|
// received one is. Per-delivery replay is the one other path that
|
||||||
|
// creates a delivery: it adds one to an existing event without
|
||||||
|
// coming through here.
|
||||||
//
|
//
|
||||||
// The tasks are returned as well as queued, so a caller can report how
|
// The tasks are returned as well as queued, so a caller can report how
|
||||||
// many targets the event went to.
|
// many targets the event went to.
|
||||||
|
|||||||
@@ -86,6 +86,7 @@ func TestAlpineRunsUnderTheSecurityPolicy(t *testing.T) {
|
|||||||
checkAddForms(ctx, t, page)
|
checkAddForms(ctx, t, page)
|
||||||
checkTargetType(ctx, t, page+"/events")
|
checkTargetType(ctx, t, page+"/events")
|
||||||
checkCopy(ctx, t, page)
|
checkCopy(ctx, t, page)
|
||||||
|
checkEntrypointEdit(ctx, t, page)
|
||||||
checkEventLog(ctx, t, page+"/events", event.ID, target.Name)
|
checkEventLog(ctx, t, page+"/events", event.ID, target.Name)
|
||||||
checkMobileMenu(ctx, t, page)
|
checkMobileMenu(ctx, t, page)
|
||||||
|
|
||||||
@@ -363,6 +364,62 @@ func checkCopy(ctx context.Context, t *testing.T, url string) {
|
|||||||
`clicking Copy does not show "Copied"`)
|
`clicking Copy does not show "Copied"`)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// checkEntrypointEdit loads a webhook page whose entrypoint has no
|
||||||
|
// description, and checks that Edit shows the edit form in place of
|
||||||
|
// the description and hides until the form closes, so the form always
|
||||||
|
// opens on the saved description; that Cancel hides it and drops what
|
||||||
|
// was typed; and that Save changes the description the page shows.
|
||||||
|
func checkEntrypointEdit(ctx context.Context, t *testing.T, url string) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
const (
|
||||||
|
editForm = `form[action$="/edit"]`
|
||||||
|
input = editForm + ` input[name="description"]`
|
||||||
|
description = `//span[text()="Entrypoint"]`
|
||||||
|
edit = `//button[text()="Edit"]`
|
||||||
|
)
|
||||||
|
|
||||||
|
require.NoError(t, chromedp.Run(ctx, loadPage(url)))
|
||||||
|
|
||||||
|
assert.True(t, hidden(ctx, editForm),
|
||||||
|
"the edit form shows before Edit is clicked")
|
||||||
|
|
||||||
|
click(ctx, t, edit)
|
||||||
|
assert.True(t, shown(ctx, editForm),
|
||||||
|
"clicking Edit does not show the edit form")
|
||||||
|
assert.True(t, hidden(ctx, description),
|
||||||
|
"the description stays shown beside the edit form")
|
||||||
|
assert.True(t, hidden(ctx, edit),
|
||||||
|
"Edit stays shown while the edit form is open")
|
||||||
|
|
||||||
|
require.NoError(t, chromedp.Run(
|
||||||
|
ctx, chromedp.SendKeys(input, "draft", chromedp.ByQuery),
|
||||||
|
))
|
||||||
|
click(ctx, t, `//button[text()="Cancel"]`)
|
||||||
|
assert.True(t, hidden(ctx, editForm),
|
||||||
|
"clicking Cancel does not hide the edit form")
|
||||||
|
assert.True(t, shown(ctx, description),
|
||||||
|
"clicking Cancel does not show the description again")
|
||||||
|
assert.True(t, shown(ctx, edit),
|
||||||
|
"clicking Cancel does not show Edit again")
|
||||||
|
|
||||||
|
var typed string
|
||||||
|
|
||||||
|
click(ctx, t, edit)
|
||||||
|
require.NoError(t, chromedp.Run(
|
||||||
|
ctx, chromedp.Value(input, &typed, chromedp.ByQuery),
|
||||||
|
))
|
||||||
|
assert.Empty(t, typed, "Cancel keeps what was typed")
|
||||||
|
|
||||||
|
require.NoError(t, chromedp.Run(
|
||||||
|
ctx, chromedp.SendKeys(input, "Billing sender", chromedp.ByQuery),
|
||||||
|
))
|
||||||
|
click(ctx, t, `//button[text()="Save"]`)
|
||||||
|
|
||||||
|
assert.True(t, shown(ctx, `//span[text()="Billing sender"]`),
|
||||||
|
"saving the edit form does not change the description")
|
||||||
|
}
|
||||||
|
|
||||||
// checkEventLog loads the event log and checks that clicking an event's
|
// checkEventLog loads the event log and checks that clicking an event's
|
||||||
// row expands it, that in there clicking its delivery shows the
|
// row expands it, that in there clicking its delivery shows the
|
||||||
// delivery's attempts and clicking again hides them, and that clicking
|
// delivery's attempts and clicking again hides them, and that clicking
|
||||||
|
|||||||
@@ -1,6 +1,8 @@
|
|||||||
package server
|
package server
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
|
"log/slog"
|
||||||
"net/http"
|
"net/http"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
@@ -37,6 +39,14 @@ func SentryClientOptionsForTest(
|
|||||||
return sentryClientOptions(dsn, release)
|
return sentryClientOptions(dsn, release)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// CleanShutdownForTest runs the server's stop hook, cleanShutdown,
|
||||||
|
// against hs: a server the test started itself, so it can hold a
|
||||||
|
// request open across the drain. Sentry is off.
|
||||||
|
func CleanShutdownForTest(ctx context.Context, hs *http.Server) {
|
||||||
|
s := &Server{log: slog.New(slog.DiscardHandler), httpServer: hs}
|
||||||
|
s.cleanShutdown(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
// newServerForTest builds a Server through New, as the application
|
// newServerForTest builds a Server through New, as the application
|
||||||
// does, on a lifecycle that is never started: the hooks New adds to
|
// does, on a lifecycle that is never started: the hooks New adds to
|
||||||
// it never run, so nothing listens.
|
// it never run, so nothing listens.
|
||||||
|
|||||||
@@ -0,0 +1,215 @@
|
|||||||
|
package server_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"net/url"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
)
|
||||||
|
|
||||||
|
// maxResubmits bounds the requests the tests below send to the
|
||||||
|
// resubmit route. The route's rate limit belongs to the middleware;
|
||||||
|
// this only has to sit well above it, so that a route without the
|
||||||
|
// limiter fails its test instead of looping.
|
||||||
|
const maxResubmits = 100
|
||||||
|
|
||||||
|
// resubmitPath is the resubmit route for one stored event.
|
||||||
|
func resubmitPath(webhookID, eventID string) string {
|
||||||
|
return "/hook/" + webhookID + "/events/" + eventID + "/resubmit"
|
||||||
|
}
|
||||||
|
|
||||||
|
// csrfForm is a resubmit form carrying the given CSRF token.
|
||||||
|
func csrfForm(token string) url.Values {
|
||||||
|
form := url.Values{}
|
||||||
|
form.Set("csrf_token", token)
|
||||||
|
|
||||||
|
return form
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestEventResubmit_SignedOutRequestsNeverReachTheRateLimit pins
|
||||||
|
// RequireAuth on the resubmit route. The handler also turns away a
|
||||||
|
// request without a session, with the same redirect, so a refusal
|
||||||
|
// alone would pass without RequireAuth. What RequireAuth adds is that
|
||||||
|
// it refuses such a request before the route's rate limit, so a
|
||||||
|
// signed-out client cannot spend the budget a signed-in user
|
||||||
|
// resubmits from. Each request carries a CSRF token valid for its own
|
||||||
|
// cookie, so CSRF lets it through to RequireAuth.
|
||||||
|
func TestEventResubmit_SignedOutRequestsNeverReachTheRateLimit(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
env := newTestEnv(t)
|
||||||
|
|
||||||
|
userID, _ := env.seedUser(t, "resubmitter", "somepassword")
|
||||||
|
wh := env.seedWebhook(t, userID)
|
||||||
|
evt := env.seedEvent(t, wh.ID, `{"resubmit":"me"}`)
|
||||||
|
path := resubmitPath(wh.ID, evt.ID)
|
||||||
|
logsPath := "/hook/" + wh.ID + "/events"
|
||||||
|
|
||||||
|
token, signedOut := env.csrfFrom(t, "/pages/login", nil)
|
||||||
|
|
||||||
|
for i := range maxResubmits {
|
||||||
|
w := env.post(path, csrfForm(token), signedOut)
|
||||||
|
require.Equal(t, http.StatusSeeOther, w.Code, "request %d", i)
|
||||||
|
require.Equal(
|
||||||
|
t, "/pages/login", w.Header().Get("Location"),
|
||||||
|
"request %d", i,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
require.Equal(
|
||||||
|
t, int64(1), env.countEvents(t, wh.ID),
|
||||||
|
"a signed-out request must store nothing",
|
||||||
|
)
|
||||||
|
|
||||||
|
token, cookies := env.csrfFrom(
|
||||||
|
t, logsPath, env.authCookies(t, userID, "resubmitter"),
|
||||||
|
)
|
||||||
|
|
||||||
|
env.requireNotice(
|
||||||
|
t, env.post(path, csrfForm(token), cookies),
|
||||||
|
logsPath, "resubmit-no-targets",
|
||||||
|
"this source has no active targets", cookies,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestEventResubmit_RefusedWithoutAValidCSRFToken pins CSRF on the
|
||||||
|
// resubmit route: a signed-in user's POST is refused with 403, and
|
||||||
|
// stores nothing, unless it carries the token issued to that user's
|
||||||
|
// own browser.
|
||||||
|
func TestEventResubmit_RefusedWithoutAValidCSRFToken(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
env := newTestEnv(t)
|
||||||
|
|
||||||
|
userID, _ := env.seedUser(t, "resubmitter", "somepassword")
|
||||||
|
wh := env.seedWebhook(t, userID)
|
||||||
|
evt := env.seedEvent(t, wh.ID, `{"resubmit":"me"}`)
|
||||||
|
path := resubmitPath(wh.ID, evt.ID)
|
||||||
|
logsPath := "/hook/" + wh.ID + "/events"
|
||||||
|
|
||||||
|
token, cookies := env.csrfFrom(
|
||||||
|
t, logsPath, env.authCookies(t, userID, "resubmitter"),
|
||||||
|
)
|
||||||
|
otherBrowsers, _ := env.csrfFrom(t, "/pages/login", nil)
|
||||||
|
|
||||||
|
for name, form := range map[string]url.Values{
|
||||||
|
"no token": {},
|
||||||
|
"a malformed token": csrfForm("not-a-token"),
|
||||||
|
"another browser's token": csrfForm(otherBrowsers),
|
||||||
|
} {
|
||||||
|
assert.Equal(
|
||||||
|
t, http.StatusForbidden,
|
||||||
|
env.post(path, form, cookies).Code, name,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
assert.Equal(
|
||||||
|
t, int64(1), env.countEvents(t, wh.ID),
|
||||||
|
"a refused request must store nothing",
|
||||||
|
)
|
||||||
|
|
||||||
|
// The same request with the user's own token goes through, so the
|
||||||
|
// refusals above were the token's doing.
|
||||||
|
env.requireNotice(
|
||||||
|
t, env.post(path, csrfForm(token), cookies),
|
||||||
|
logsPath, "resubmit-no-targets",
|
||||||
|
"this source has no active targets", cookies,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestEventResubmit_AnotherWebhooksEvent404s pins, on the route as
|
||||||
|
// registered, that a signed-in user gets 404, and nothing is stored,
|
||||||
|
// for an event of a webhook another user owns, which the handler's
|
||||||
|
// ownership check refuses, and for another webhook's event posted
|
||||||
|
// under a webhook the user does own, which the event lookup refuses.
|
||||||
|
// The user's own event, posted the same way, is accepted, so the
|
||||||
|
// second 404 comes from the lookup and not from a route that never
|
||||||
|
// passed the event ID to the handler.
|
||||||
|
func TestEventResubmit_AnotherWebhooksEvent404s(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
env := newTestEnv(t)
|
||||||
|
|
||||||
|
ownerID, _ := env.seedUser(t, "owner", "somepassword")
|
||||||
|
owners := env.seedWebhook(t, ownerID)
|
||||||
|
ownersEvent := env.seedEvent(t, owners.ID, `{"owner":"only"}`)
|
||||||
|
|
||||||
|
intruderID, _ := env.seedUser(t, "intruder", "somepassword")
|
||||||
|
intruders := env.seedWebhook(t, intruderID)
|
||||||
|
intrudersEvent := env.seedEvent(
|
||||||
|
t, intruders.ID, `{"intruder":"own"}`,
|
||||||
|
)
|
||||||
|
intrudersLogs := "/hook/" + intruders.ID + "/events"
|
||||||
|
|
||||||
|
token, cookies := env.csrfFrom(
|
||||||
|
t, intrudersLogs, env.authCookies(t, intruderID, "intruder"),
|
||||||
|
)
|
||||||
|
|
||||||
|
for name, path := range map[string]string{
|
||||||
|
"another user's webhook": resubmitPath(
|
||||||
|
owners.ID, ownersEvent.ID,
|
||||||
|
),
|
||||||
|
"another webhook's event": resubmitPath(
|
||||||
|
intruders.ID, ownersEvent.ID,
|
||||||
|
),
|
||||||
|
} {
|
||||||
|
w := env.post(path, csrfForm(token), cookies)
|
||||||
|
assert.Equal(t, http.StatusNotFound, w.Code, name)
|
||||||
|
}
|
||||||
|
|
||||||
|
assert.Equal(t, int64(1), env.countEvents(t, owners.ID))
|
||||||
|
assert.Equal(t, int64(1), env.countEvents(t, intruders.ID))
|
||||||
|
|
||||||
|
env.requireNotice(
|
||||||
|
t,
|
||||||
|
env.post(
|
||||||
|
resubmitPath(intruders.ID, intrudersEvent.ID),
|
||||||
|
csrfForm(token), cookies,
|
||||||
|
),
|
||||||
|
intrudersLogs, "resubmit-no-targets",
|
||||||
|
"this source has no active targets", cookies,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestEventResubmit_RateLimited pins the rate limit on the resubmit
|
||||||
|
// route: a signed-in user's resubmits are accepted until the budget
|
||||||
|
// is spent, and then refused with 429.
|
||||||
|
func TestEventResubmit_RateLimited(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
env := newTestEnv(t)
|
||||||
|
|
||||||
|
userID, _ := env.seedUser(t, "resubmitter", "somepassword")
|
||||||
|
wh := env.seedWebhook(t, userID)
|
||||||
|
evt := env.seedEvent(t, wh.ID, `{"resubmit":"me"}`)
|
||||||
|
path := resubmitPath(wh.ID, evt.ID)
|
||||||
|
|
||||||
|
token, cookies := env.csrfFrom(
|
||||||
|
t, "/hook/"+wh.ID+"/events",
|
||||||
|
env.authCookies(t, userID, "resubmitter"),
|
||||||
|
)
|
||||||
|
|
||||||
|
limited := false
|
||||||
|
|
||||||
|
for range maxResubmits {
|
||||||
|
code := env.post(path, csrfForm(token), cookies).Code
|
||||||
|
if code == http.StatusTooManyRequests {
|
||||||
|
limited = true
|
||||||
|
|
||||||
|
break
|
||||||
|
}
|
||||||
|
|
||||||
|
require.Equal(
|
||||||
|
t, http.StatusSeeOther, code,
|
||||||
|
"a resubmit within the budget must be accepted",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
assert.True(
|
||||||
|
t, limited, "repeated resubmits must eventually be refused",
|
||||||
|
)
|
||||||
|
}
|
||||||
@@ -289,6 +289,10 @@ func (s *Server) setupSourceRoutes() {
|
|||||||
"/entrypoints",
|
"/entrypoints",
|
||||||
s.h.HandleEntrypointCreate(),
|
s.h.HandleEntrypointCreate(),
|
||||||
)
|
)
|
||||||
|
r.Post(
|
||||||
|
"/entrypoints/{entrypointID}/edit",
|
||||||
|
s.h.HandleEntrypointEdit(),
|
||||||
|
)
|
||||||
r.Post(
|
r.Post(
|
||||||
"/entrypoints/{entrypointID}/delete",
|
"/entrypoints/{entrypointID}/delete",
|
||||||
s.h.HandleEntrypointDelete(),
|
s.h.HandleEntrypointDelete(),
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
|
"github.com/google/uuid"
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"go.uber.org/fx"
|
"go.uber.org/fx"
|
||||||
@@ -398,6 +399,44 @@ func (e *testEnv) seedTarget(
|
|||||||
return tgt
|
return tgt
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// seedEntrypoint creates an active entrypoint for a webhook.
|
||||||
|
func (e *testEnv) seedEntrypoint(
|
||||||
|
t *testing.T,
|
||||||
|
webhookID string,
|
||||||
|
) *database.Entrypoint {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
ep := &database.Entrypoint{
|
||||||
|
WebhookID: webhookID,
|
||||||
|
Path: uuid.New().String(),
|
||||||
|
Description: "Default entrypoint",
|
||||||
|
Active: true,
|
||||||
|
}
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t,
|
||||||
|
e.db.DB().Omit(clause.Associations).Create(ep).Error,
|
||||||
|
)
|
||||||
|
|
||||||
|
return ep
|
||||||
|
}
|
||||||
|
|
||||||
|
// storedEntrypoint reloads an entrypoint row.
|
||||||
|
func (e *testEnv) storedEntrypoint(
|
||||||
|
t *testing.T,
|
||||||
|
entrypointID string,
|
||||||
|
) database.Entrypoint {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var ep database.Entrypoint
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t, e.db.DB().First(&ep, "id = ?", entrypointID).Error,
|
||||||
|
)
|
||||||
|
|
||||||
|
return ep
|
||||||
|
}
|
||||||
|
|
||||||
// seedFailedDelivery records a terminally failed delivery of an event
|
// seedFailedDelivery records a terminally failed delivery of an event
|
||||||
// to a target in the webhook's own database.
|
// to a target in the webhook's own database.
|
||||||
func (e *testEnv) seedFailedDelivery(
|
func (e *testEnv) seedFailedDelivery(
|
||||||
@@ -444,6 +483,23 @@ func (e *testEnv) countDeliveries(
|
|||||||
return count
|
return count
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// countEvents reports how many events a webhook's database holds.
|
||||||
|
func (e *testEnv) countEvents(t *testing.T, webhookID string) int64 {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
webhookDB, err := e.dbMgr.GetDB(webhookID)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
var count int64
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t,
|
||||||
|
webhookDB.Model(&database.Event{}).Count(&count).Error,
|
||||||
|
)
|
||||||
|
|
||||||
|
return count
|
||||||
|
}
|
||||||
|
|
||||||
// storedHash reads the current password hash for a username.
|
// storedHash reads the current password hash for a username.
|
||||||
func (e *testEnv) storedHash(t *testing.T, username string) string {
|
func (e *testEnv) storedHash(t *testing.T, username string) string {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
@@ -1070,6 +1126,119 @@ func TestHook_EntrypointActions(t *testing.T) {
|
|||||||
assert.Zero(t, left, "the delete should remove the entrypoint")
|
assert.Zero(t, left, "the delete should remove the entrypoint")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestHook_EntrypointEdit changes an entrypoint's description with the
|
||||||
|
// edit form on the webhook page, then empties it. The entrypoint keeps
|
||||||
|
// its URL, and with no description it shows as "Entrypoint". Without
|
||||||
|
// the CSRF token, or without a session, the edit is refused.
|
||||||
|
func TestHook_EntrypointEdit(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
env := newTestEnv(t)
|
||||||
|
|
||||||
|
userID, _ := env.seedUser(t, "epeditor", "somepassword")
|
||||||
|
cookies := env.authCookies(t, userID, "epeditor")
|
||||||
|
wh := env.seedWebhook(t, userID)
|
||||||
|
ep := env.seedEntrypoint(t, wh.ID)
|
||||||
|
page := "/hook/" + wh.ID
|
||||||
|
|
||||||
|
token, cookies := env.csrfFrom(t, page, cookies)
|
||||||
|
action := env.urlFrom(
|
||||||
|
t, page, `action="(/hook/[^/"]+/entrypoints/[^/"]+/edit)"`,
|
||||||
|
cookies,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.Equal(
|
||||||
|
t, http.StatusForbidden,
|
||||||
|
env.post(action, entrypointEditForm("", "no token"), cookies).Code,
|
||||||
|
"an edit without a CSRF token must be refused",
|
||||||
|
)
|
||||||
|
|
||||||
|
// The request without a session carries a valid CSRF token from
|
||||||
|
// the login page, so only the session check can refuse it.
|
||||||
|
anonToken, anon := env.csrfFrom(t, "/pages/login", nil)
|
||||||
|
refused := env.post(
|
||||||
|
action, entrypointEditForm(anonToken, "no session"), anon,
|
||||||
|
)
|
||||||
|
assert.Equal(t, http.StatusSeeOther, refused.Code)
|
||||||
|
assert.Equal(
|
||||||
|
t, "/pages/login", refused.Header().Get("Location"),
|
||||||
|
"an edit without a session must be refused",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert.Equal(
|
||||||
|
t, ep.Description, env.storedEntrypoint(t, ep.ID).Description,
|
||||||
|
"a refused edit must not change the description",
|
||||||
|
)
|
||||||
|
|
||||||
|
// edit submits the form with description and requires the
|
||||||
|
// redirect back to the webhook page with the notice.
|
||||||
|
edit := func(description string) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
w := env.post(
|
||||||
|
action, entrypointEditForm(token, description), cookies,
|
||||||
|
)
|
||||||
|
env.requireNotice(t, w, page, "entrypoint-saved",
|
||||||
|
"Entrypoint description saved.", cookies)
|
||||||
|
}
|
||||||
|
|
||||||
|
edit("Billing sender")
|
||||||
|
|
||||||
|
stored := env.storedEntrypoint(t, ep.ID)
|
||||||
|
assert.Equal(t, "Billing sender", stored.Description)
|
||||||
|
assert.Equal(t, ep.Path, stored.Path,
|
||||||
|
"the edit must keep the entrypoint's URL")
|
||||||
|
|
||||||
|
body := env.get(page, cookies).Body.String()
|
||||||
|
assert.Contains(t, body, ">Billing sender</span>")
|
||||||
|
assert.Contains(t, body, "/h/"+ep.Path+"</code>")
|
||||||
|
|
||||||
|
edit("")
|
||||||
|
|
||||||
|
assert.Empty(t, env.storedEntrypoint(t, ep.ID).Description)
|
||||||
|
assert.Contains(t, env.get(page, cookies).Body.String(),
|
||||||
|
">Entrypoint</span>", "no description shows as Entrypoint")
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHook_EntrypointEdit_OtherUser404s has another logged-in user, with
|
||||||
|
// a CSRF token of their own, try to edit an entrypoint: through the
|
||||||
|
// owner's webhook, and through a webhook of their own. Both are 404s
|
||||||
|
// and the description stays as it was.
|
||||||
|
func TestHook_EntrypointEdit_OtherUser404s(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
env := newTestEnv(t)
|
||||||
|
|
||||||
|
ownerID, _ := env.seedUser(t, "epowner", "somepassword")
|
||||||
|
wh := env.seedWebhook(t, ownerID)
|
||||||
|
ep := env.seedEntrypoint(t, wh.ID)
|
||||||
|
|
||||||
|
otherID, _ := env.seedUser(t, "epother", "somepassword")
|
||||||
|
other := env.authCookies(t, otherID, "epother")
|
||||||
|
theirs := env.seedWebhook(t, otherID)
|
||||||
|
token, other := env.csrfFrom(t, "/hook/"+theirs.ID, other)
|
||||||
|
|
||||||
|
for _, webhookID := range []string{wh.ID, theirs.ID} {
|
||||||
|
path := "/hook/" + webhookID + "/entrypoints/" + ep.ID + "/edit"
|
||||||
|
|
||||||
|
w := env.post(path, entrypointEditForm(token, "not theirs"), other)
|
||||||
|
assert.Equal(t, http.StatusNotFound, w.Code, path)
|
||||||
|
}
|
||||||
|
|
||||||
|
assert.Equal(
|
||||||
|
t, ep.Description, env.storedEntrypoint(t, ep.ID).Description,
|
||||||
|
"another user must not change the description",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// entrypointEditForm fills in the webhook page's entrypoint edit form.
|
||||||
|
func entrypointEditForm(token, description string) url.Values {
|
||||||
|
return url.Values{
|
||||||
|
"csrf_token": {token},
|
||||||
|
"description": {description},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestHook_TargetActions adds a target with the form on the webhook
|
// TestHook_TargetActions adds a target with the form on the webhook
|
||||||
// page, follows its Edit link to the target edit form and submits
|
// page, follows its Edit link to the target edit form and submits
|
||||||
// it, then deactivates, activates and deletes it, every URL and token
|
// it, then deactivates, activates and deletes it, every URL and token
|
||||||
|
|||||||
@@ -39,6 +39,12 @@ const (
|
|||||||
// refuses to spend, leaving it for the hooks that run after the
|
// refuses to spend, leaving it for the hooks that run after the
|
||||||
// server: the delivery engine, the healthcheck, the webhook DB
|
// server: the delivery engine, the healthcheck, the webhook DB
|
||||||
// manager and the database close.
|
// manager and the database close.
|
||||||
|
//
|
||||||
|
// Its value is not tuned to those hooks, which take about a
|
||||||
|
// millisecond between them. It is what the 5s fx stop timeout in
|
||||||
|
// cmd/webhooker leaves after a full ShutdownTimeout drain, so a
|
||||||
|
// drain that starts on a full budget still gets all of
|
||||||
|
// ShutdownTimeout.
|
||||||
TailHookReserve = 2 * time.Second
|
TailHookReserve = 2 * time.Second
|
||||||
|
|
||||||
// sentryFlushTimeout is the longest wait for Sentry to flush
|
// sentryFlushTimeout is the longest wait for Sentry to flush
|
||||||
@@ -59,6 +65,16 @@ const (
|
|||||||
// key off it, and a zero exit would read as a deliberate stop.
|
// key off it, and a zero exit would read as a deliberate stop.
|
||||||
const StartupFailureExitCode = 1
|
const StartupFailureExitCode = 1
|
||||||
|
|
||||||
|
// DrainBudget reports how long the HTTP drain may wait for in-flight
|
||||||
|
// requests when remaining is the time left on the fx stop context as
|
||||||
|
// the server's stop hook starts. The hooks before the server can
|
||||||
|
// already have spent part of the budget, so the drain takes its time
|
||||||
|
// out of what they left, never out of TailHookReserve. Zero or less
|
||||||
|
// means no wait at all.
|
||||||
|
func DrainBudget(remaining time.Duration) time.Duration {
|
||||||
|
return min(ShutdownTimeout, remaining-TailHookReserve)
|
||||||
|
}
|
||||||
|
|
||||||
// SentryFlushBudget reports how long the Sentry flush may run when
|
// SentryFlushBudget reports how long the Sentry flush may run when
|
||||||
// remaining is the time left on the fx stop context after the HTTP
|
// remaining is the time left on the fx stop context after the HTTP
|
||||||
// drain. sentry.Flush takes a bare duration and honours no context,
|
// drain. sentry.Flush takes a bare duration and honours no context,
|
||||||
@@ -261,10 +277,17 @@ func (s *Server) cleanupForExit() {
|
|||||||
s.log.Info("cleaning up")
|
s.log.Info("cleaning up")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// cleanShutdown drains the HTTP server and flushes Sentry inside what
|
||||||
|
// is left of the fx stop budget. A context carrying no deadline — a
|
||||||
|
// caller outside the fx lifecycle — gets the full ShutdownTimeout.
|
||||||
func (s *Server) cleanShutdown(ctx context.Context) {
|
func (s *Server) cleanShutdown(ctx context.Context) {
|
||||||
ctxShutdown, shutdownCancel := context.WithTimeout(
|
drain := ShutdownTimeout
|
||||||
ctx, ShutdownTimeout,
|
|
||||||
)
|
if deadline, ok := ctx.Deadline(); ok {
|
||||||
|
drain = DrainBudget(time.Until(deadline))
|
||||||
|
}
|
||||||
|
|
||||||
|
ctxShutdown, shutdownCancel := context.WithTimeout(ctx, drain)
|
||||||
defer shutdownCancel()
|
defer shutdownCancel()
|
||||||
|
|
||||||
err := s.httpServer.Shutdown(ctxShutdown)
|
err := s.httpServer.Shutdown(ctxShutdown)
|
||||||
|
|||||||
@@ -1,13 +1,148 @@
|
|||||||
package server_test
|
package server_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
"testing"
|
"testing"
|
||||||
|
"testing/synctest"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
"sneak.berlin/go/webhooker/internal/server"
|
"sneak.berlin/go/webhooker/internal/server"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// TestDrainBudget covers the clamp that keeps the HTTP drain from
|
||||||
|
// spending the tail hooks' share of the fx stop budget when the hooks
|
||||||
|
// before the server have already used part of it.
|
||||||
|
func TestDrainBudget(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
remaining time.Duration
|
||||||
|
want time.Duration
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "only the reserve is left",
|
||||||
|
remaining: server.TailHookReserve,
|
||||||
|
want: 0,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "earlier hooks spent part of the budget",
|
||||||
|
remaining: server.TailHookReserve + time.Second,
|
||||||
|
want: time.Second,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "capped at the nominal timeout",
|
||||||
|
remaining: time.Hour,
|
||||||
|
want: server.ShutdownTimeout,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
require.Equal(t, tt.want, server.DrainBudget(tt.remaining))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestCleanShutdown_LeavesTailHookReserve stops the server with a
|
||||||
|
// request still in flight, after the hooks before it have spent all
|
||||||
|
// of the stop budget but TailHookReserve. The drain must give up at
|
||||||
|
// once rather than wait for the request: what is left belongs to the
|
||||||
|
// hooks after the server, the database close among them. A drain
|
||||||
|
// bounded only by ShutdownTimeout waits until the stop context
|
||||||
|
// expires, and fx then skips those hooks.
|
||||||
|
//
|
||||||
|
// The test runs in a synctest bubble, whose clock moves only while
|
||||||
|
// every goroutine in it is blocked, so a drain that gives up at once
|
||||||
|
// leaves the stop context unexpired however slow the host is. The
|
||||||
|
// request travels over net.Pipe because a goroutine waiting on a
|
||||||
|
// real socket would stop that clock from moving at all.
|
||||||
|
func TestCleanShutdown_LeavesTailHookReserve(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
synctest.Test(t, func(t *testing.T) {
|
||||||
|
entered := make(chan struct{})
|
||||||
|
release := make(chan struct{})
|
||||||
|
|
||||||
|
hs := &http.Server{
|
||||||
|
Handler: http.HandlerFunc(
|
||||||
|
func(http.ResponseWriter, *http.Request) {
|
||||||
|
close(entered)
|
||||||
|
<-release
|
||||||
|
},
|
||||||
|
),
|
||||||
|
ReadHeaderTimeout: time.Second,
|
||||||
|
}
|
||||||
|
|
||||||
|
srvConn, cliConn := net.Pipe()
|
||||||
|
|
||||||
|
listener := pipeListener{
|
||||||
|
conns: make(chan net.Conn, 1),
|
||||||
|
closed: make(chan struct{}),
|
||||||
|
}
|
||||||
|
listener.conns <- srvConn
|
||||||
|
|
||||||
|
go func() { _ = hs.Serve(listener) }()
|
||||||
|
|
||||||
|
// Cleanups run last first: the handler returns, then closing
|
||||||
|
// the client end ends the server's write of the response.
|
||||||
|
t.Cleanup(func() { _ = cliConn.Close() })
|
||||||
|
t.Cleanup(func() { close(release) })
|
||||||
|
|
||||||
|
_, err := cliConn.Write(
|
||||||
|
[]byte("GET / HTTP/1.1\r\nHost: webhooker.test\r\n\r\n"),
|
||||||
|
)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
<-entered
|
||||||
|
|
||||||
|
stopCtx, cancel := context.WithTimeout(
|
||||||
|
t.Context(), server.TailHookReserve,
|
||||||
|
)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
server.CleanShutdownForTest(stopCtx, hs)
|
||||||
|
|
||||||
|
require.NoError(
|
||||||
|
t, stopCtx.Err(), "the drain spent the tail hooks' reserve",
|
||||||
|
)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// pipeListener is the net.Listener http.Server.Serve needs to serve
|
||||||
|
// the server end of a net.Pipe: Accept returns that one connection,
|
||||||
|
// then waits until Close, as a real listener with no more clients
|
||||||
|
// does.
|
||||||
|
type pipeListener struct {
|
||||||
|
conns chan net.Conn
|
||||||
|
closed chan struct{}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (l pipeListener) Accept() (net.Conn, error) {
|
||||||
|
select {
|
||||||
|
case conn := <-l.conns:
|
||||||
|
return conn, nil
|
||||||
|
case <-l.closed:
|
||||||
|
return nil, net.ErrClosed
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (l pipeListener) Close() error {
|
||||||
|
close(l.closed)
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Addr is never called by http.Server.Serve.
|
||||||
|
func (pipeListener) Addr() net.Addr {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
// TestSentryFlushBudget covers the clamp that keeps the Sentry flush
|
// TestSentryFlushBudget covers the clamp that keeps the Sentry flush
|
||||||
// from spending the tail hooks' share of the fx stop budget.
|
// from spending the tail hooks' share of the fx stop budget.
|
||||||
// sentry.Flush ignores the stop context, so without the clamp a
|
// sentry.Flush ignores the stop context, so without the clamp a
|
||||||
|
|||||||
+2
-1
@@ -70,7 +70,8 @@ document.addEventListener("alpine:init", function () {
|
|||||||
"use strict";
|
"use strict";
|
||||||
|
|
||||||
// Something a click shows and hides: the mobile menu, an add form,
|
// Something a click shows and hides: the mobile menu, an add form,
|
||||||
// an event in the event log, a delivery's attempts.
|
// an entrypoint's edit form, an event in the event log, a delivery's
|
||||||
|
// attempts.
|
||||||
window.Alpine.data("collapsible", function () {
|
window.Alpine.data("collapsible", function () {
|
||||||
return {
|
return {
|
||||||
open: false,
|
open: false,
|
||||||
|
|||||||
@@ -54,15 +54,26 @@
|
|||||||
|
|
||||||
<div class="divide-y divide-gray-100">
|
<div class="divide-y divide-gray-100">
|
||||||
{{range .Entrypoints}}
|
{{range .Entrypoints}}
|
||||||
<div class="p-4">
|
<div class="p-4" x-data="collapsible">
|
||||||
<div class="flex flex-wrap items-center justify-between gap-2 mb-1">
|
<div class="flex flex-wrap items-center justify-between gap-2 mb-1">
|
||||||
<span class="text-sm font-medium text-gray-900">{{if .Description}}{{.Description}}{{else}}Entrypoint{{end}}</span>
|
<span x-show="closed" class="text-sm font-medium text-gray-900">{{if .Description}}{{.Description}}{{else}}Entrypoint{{end}}</span>
|
||||||
|
<!-- Edit shows this form in place of the
|
||||||
|
description and hides until it closes, so
|
||||||
|
the form always opens on the saved
|
||||||
|
description. Cancel resets what was typed. -->
|
||||||
|
<form x-show="open" x-cloak method="POST" action="/hook/{{$.Webhook.ID}}/entrypoints/{{.ID}}/edit" class="flex w-full gap-2">
|
||||||
|
<input type="hidden" name="csrf_token" value="{{$.CSRFToken}}">
|
||||||
|
<input type="text" name="description" value="{{.Description}}" placeholder="Description (optional)" class="input text-sm flex-1">
|
||||||
|
<button type="submit" class="btn-primary text-sm">Save</button>
|
||||||
|
<button type="reset" @click="toggle" class="btn-secondary text-sm">Cancel</button>
|
||||||
|
</form>
|
||||||
<div class="flex flex-wrap items-center gap-2">
|
<div class="flex flex-wrap items-center gap-2">
|
||||||
{{if .Active}}
|
{{if .Active}}
|
||||||
<span class="badge-success">Active</span>
|
<span class="badge-success">Active</span>
|
||||||
{{else}}
|
{{else}}
|
||||||
<span class="badge-error">Inactive</span>
|
<span class="badge-error">Inactive</span>
|
||||||
{{end}}
|
{{end}}
|
||||||
|
<button type="button" x-show="closed" @click="toggle" class="btn-small" title="Edit">Edit</button>
|
||||||
<form method="POST" action="/hook/{{$.Webhook.ID}}/entrypoints/{{.ID}}/toggle" class="inline">
|
<form method="POST" action="/hook/{{$.Webhook.ID}}/entrypoints/{{.ID}}/toggle" class="inline">
|
||||||
<input type="hidden" name="csrf_token" value="{{$.CSRFToken}}">
|
<input type="hidden" name="csrf_token" value="{{$.CSRFToken}}">
|
||||||
<button type="submit" class="btn-small" title="{{if .Active}}Deactivate{{else}}Activate{{end}}">
|
<button type="submit" class="btn-small" title="{{if .Active}}Deactivate{{else}}Activate{{end}}">
|
||||||
|
|||||||
Reference in New Issue
Block a user