1 Commits
Author SHA1 Message Date
clawbot 12cc9228c3 Run the tests with cgo off and without the race detector (closes #538)
check / check (push) Canceled after 0s
sneak ruled the race detector out
(#538 (comment)), and on
Linux it was the only reason the tests needed cgo. The test phase and the
browser test binary now build with CGO_ENABLED=0 and without -race, so no
test binary compiles mattn's C SQLite. The test phase keeps the builder
stage's Debian Go image. Its comments, the README and the code comments that
said the suite runs under -race now say what is true, and the README notes
the exception to the vendored REPO_POLICIES.md.

Model: opus-5-5
2026-10-21 14:32:39 +00:00
19 changed files with 3560 additions and 881 deletions
+3 -1
View File
@@ -72,8 +72,10 @@
**/*.sublime-*
# This repository's own host-built artifacts: the binary `make build`
# writes.
# writes, and the Alpine.js file `make assets` extracts from 3p/ (the
# build extracts its own).
/bin
/static/js/alpine.min.js
# SQLite databases, which hold the session key and webhook payloads, at
# any depth.
+4
View File
@@ -77,3 +77,7 @@ data/
# Temporary files
tmp/
temp/
# Alpine.js, extracted by `make assets` from its tarball in 3p/, which is
# what is committed.
/static/js/alpine.min.js
-1
View File
@@ -1,3 +1,2 @@
node_modules/
yarn.lock
static/js/alpine.min.js
Binary file not shown.
+11 -2
View File
@@ -92,6 +92,10 @@ COPY . .
# fmt-check` is the markdown-check stage above.
RUN if [ -n "$(gofmt -s -l .)" ]; then echo "gofmt needed on:"; gofmt -s -l .; exit 1; fi
# static/static.go embeds the Alpine.js file this extracts from 3p/; without
# it the static package does not compile and cannot be linted.
RUN script/assets
# The golangci-lint steps run with --network=none. `golangci-lint config
# verify` is documented as fetching its JSON schema over HTTPS; this pinned
# image resolves the schema without any network, and --network=none enforces
@@ -126,6 +130,9 @@ RUN go mod download
COPY . .
# static/static.go embeds the Alpine.js file this extracts from 3p/.
RUN script/assets
# -timeout applies to each package on its own, so 90s has only to clear the
# slowest one. -p 4 -parallel 8 keep the run under 2 GB of memory: at most
# four test binaries build or run at once, each with at most eight parallel
@@ -204,8 +211,10 @@ RUN version="$(make version VERSION="$VERSION")"; \
esac; \
fi
# The Makefile's build target builds with cgo off, so the binary is
# statically linked and runs on the Alpine runtime image below.
# Builds through the Makefile's build target, which runs script/assets
# (Alpine.js, extracted from its tarball in 3p/) first. The target builds
# with cgo off, so the binary is statically linked and runs on the Alpine
# runtime image below as it is.
RUN make build VERSION="$VERSION"
# Runtime stage
+1 -1
View File
@@ -17,7 +17,7 @@ COPY . .
# The test binary embeds the templates and static files, so the browser
# stage needs nothing else. It is built with cgo off and -p 4 keeps the
# compile's memory down, both as in the test phase of Dockerfile.
RUN CGO_ENABLED=0 go test -c -p 4 -tags browser -o /browser.test ./internal/server
RUN make assets && CGO_ENABLED=0 go test -c -p 4 -tags browser -o /browser.test ./internal/server
# chromedp/headless-shell:151.0.7922.109 (Debian trixie), 2026-08-11. The
# browser is on PATH as headless-shell, where the test's browser library
+6 -3
View File
@@ -1,4 +1,4 @@
.PHONY: bootstrap setup test test-browser lint fmt fmt-check check build run dev deps docker clean hooks css css-check version
.PHONY: bootstrap setup assets test test-browser lint fmt fmt-check check build run dev deps docker clean hooks css css-check version
# Default target
.DEFAULT_GOAL := check
@@ -22,6 +22,9 @@ bootstrap:
setup:
@script/setup
assets:
@script/assets
test:
@script/test
@@ -44,13 +47,13 @@ check:
# REPO_POLICIES.md does: the binary carries no build paths, symbol table or
# debug information, and on Linux, the image's included, it is statically
# linked.
build:
build: assets
CGO_ENABLED=0 go build -trimpath -ldflags '-s -w -X main.version=$(VERSION)' -o bin/webhooker ./cmd/webhooker
run: build
./bin/webhooker
dev:
dev: assets
go run ./cmd/webhooker
deps:
+3440 -639
View File
File diff suppressed because it is too large Load Diff
+46 -51
View File
@@ -1,77 +1,72 @@
# Workflow
One issue per unit of work, and one branch and one PR per issue:
One issue per unit of work, one branch and one PR per issue:
- make sure a tracked issue exists with a definition of done
- branch from `next`, never from `main`
- do the work and push the branch
- open a PR with base `next`, never `main`
- pass an independent review
- the repository manager squash-merges it into `next`
- ensure a tracked issue exists with a definition of done
- branch from `next` (never from `main`)
- do the work; open a PR based on `next` (never on `main`)
- pass an independent review, then the manager squash-merges into `next`
- push; nothing stays local-only
The repository manager is the role that merges each issue's PR into `next`.
Nothing stays local-only: every commit is pushed.
`next` is the branch for the next milestone and must stay green and mergeable to
`main` without notice. One `next` -> `main` PR accumulates the milestone;
releases are cut from `main` separately.
`next` is the branch for the next milestone. It must stay green and mergeable
into `main` at all times. A milestone's work reaches `main` through one PR from
`next`. The owner, @sneak, merges that PR. Cutting a release from `main` is a
separate step after that.
The tracker (https://git.eeqj.de/sneak/webhooker/issues) records what is done
and what is under way. This file keeps no list of either.
The tracker (https://git.eeqj.de/sneak/webhooker/issues) is the record of what
is done and what is under way. This file keeps no list of either.
# Status
What is still open for 1.0.0 is in its milestone
(https://git.eeqj.de/sneak/webhooker/milestone/9). `v1.0.0` is not tagged. Only
@sneak tags it.
(https://git.eeqj.de/sneak/webhooker/milestone/9). `v1.0.0` is not tagged, and
only @sneak tags it.
Delivery is at least once by design. When webhooker cannot record the outcome of
a delivery, it sends that delivery again. The target can then receive the same
event twice. The alternative is a lost delivery that nothing reports. The README
says so. Such a duplicate is not a defect to re-file.
Delivery is at-least-once by design, not by accident: a send whose result row
does not land is attempted again, so a receiver can see a duplicate. That is
deliberate — the alternative is a silent lost delivery — and the README says so
under Rationale. It is not a defect to re-file.
# Next Step
@sneak runs webhooker in limited production, then tags `v1.0.0`. Merging `next`
into `main` is a separate act. It waits on neither the production run nor the
tag.
into `main` is a separate act from tagging and waits on neither of those: `next`
is kept mergeable at all times, which is the point of the branch.
# Future Steps
- Per-webhook rate limits, set in the web UI and enforced at the entrypoint.
Today `RECEIVER_RATE_LIMIT` is the only rate-limit setting. It is how many
requests per minute one client address may send to one entrypoint. The same
address is also capped at ten times that across all entrypoints together.
- API key authentication for programmatic access. The `APIKey` model and its
`api_keys` table exist. Nothing creates a key and nothing checks one.
- REST API v1. `/api/v1` is mounted with no routes today.
- CRUD for webhooks, entrypoints and targets
- Per-webhook rate limits, set in the web UI and enforced in the receiver
handler, on top of the receiver's existing limits, which `RECEIVER_RATE_LIMIT`
sets for the whole server (see Rate Limiting in the README)
- API key authentication for programmatic access (the `APIKey` model exists, but
nothing creates a key and no middleware checks one)
- REST API v1
- CRUD for webhooks, entrypoints, targets
- event viewing and filtering endpoints
- event redelivery endpoint
- OpenAPI specification
- Analytics in the web UI: response times summarised per webhook or per target,
and figures for all webhooks together. Today the webhook page shows event,
delivery and failure counts and failure percentages. The target list counts
each target's deliveries. The event log shows every attempt's duration.
`/metrics` carries server-wide counts and delivery durations.
- A remember-me option at login. Today every session survives a browser restart.
It lasts up to 7 days. It ends sooner after `SESSION_IDLE_TIMEOUT` (default 24
hours) without use.
- Password reset over the web for a forgotten password. Today a signed-in user
can change their password. A lost password is reset from the command line with
`webhooker resetpw` (https://git.eeqj.de/sneak/webhooker/issues/208).
and figures for all webhooks together. The webhook page already shows event,
delivery and failure counts and failure percentages, the target list counts
each target's deliveries, the event log and each event's page show every
attempt's duration, and `/metrics` carries server-wide counts and delivery
durations
- A remember-me option at login. Every session already survives a browser
restart: it lasts up to 7 days, and ends sooner after `SESSION_IDLE_TIMEOUT`
(default 24 hours) without activity
- Password reset flow for a forgotten password over the web. The authenticated
password _change_ flow already landed, and a lost password is recoverable from
the console with `webhooker resetpw`
(https://git.eeqj.de/sneak/webhooker/issues/208)
- Later, nice to have
- email delivery target type
- SNS and S3 delivery targets
- data transformations, such as a configurable Slack message format. The
Slack target sends one fixed format today.
- JSONL file delivery with periodic S3 upload. Today an archive target
writes events to SQLite files. An archive target can rotate those files
hourly, daily or monthly. The webhook page offers them as one gzipped JSON
download.
- data transformations (e.g. a configurable Slack message format; the Slack
target sends one fixed format today)
- JSONL file delivery with periodic S3 upload (the `database` target already
archives events to SQLite files, rotated hourly, daily or monthly if set,
and the webhook page offers them as one gzipped JSON download)
- webhook event search, and filtering beyond the event log's failed and
pending views
- multi-user with role-based access control. Today each webhook belongs to
one account. Only the account made at first boot exists. There is no way
to add another.
- multi-user with role-based access control (each webhook already belongs to
one account, but there is only the account made at first boot and no way
to add another)
+1 -1
View File
@@ -5,7 +5,7 @@
// REPO_POLICIES.md states that a linter can check: const for everything,
// let only for a variable that is reassigned, never var.
export default [
// Alpine.js, committed as published; not ours to lint.
// Alpine.js, extracted from 3p/ by make assets; not ours to lint.
{ ignores: ["static/js/alpine.min.js"] },
{
// The pages load static/js/app.js as a classic script, not a module.
-6
View File
@@ -96,12 +96,6 @@ func (s *Handlers) WebhookStatsForTest(webhookID string) *WebhookStats {
return s.loadWebhookStats(webhookID, entrypoints, targets)
}
// RelativeTimeForTest exposes relativeTime for use in the
// handlers_test package.
func RelativeTimeForTest(at, now time.Time) string {
return relativeTime(at, now)
}
// FinishedByTargetForTest exposes finishedByTarget for use in the
// handlers_test package.
func FinishedByTargetForTest(
+6 -55
View File
@@ -15,9 +15,6 @@ const (
longWindow = 24 * time.Hour
)
// day is the largest unit relativeTime writes.
const day = 24 * time.Hour
// percent turns a fraction into a percentage.
const percent = 100
@@ -38,12 +35,8 @@ type WebhookStats struct {
InProgress int64
// LastEventAt is when the newest event arrived, or nil when none
// has. Retention does not change it. LastEvent is how long ago
// that was, as relativeTime writes it, and LastEventUTC the full
// time; both are empty when no event has arrived.
LastEventAt *time.Time
LastEvent string
LastEventUTC string
// has. Retention does not change it.
LastEventAt *time.Time
Last10Minutes RecentWindow
Last24Hours RecentWindow
@@ -88,42 +81,6 @@ func (w RecentWindow) FailurePercent() string {
)
}
// relativeTime is how long before now the time at was, in whole
// seconds, written as its two largest units out of d, h, m and s:
// "59s", "1m1s", "23h59m", "3d4h". A second unit of zero is left off,
// as in "5m". A time after now is "0s".
func relativeTime(at, now time.Time) string {
elapsed := max(now.Sub(at), 0)
days := int64(elapsed / day)
hours := int64(elapsed % day / time.Hour)
minutes := int64(elapsed % time.Hour / time.Minute)
seconds := int64(elapsed % time.Minute / time.Second)
switch {
case days > 0:
return twoUnits(days, "d", hours, "h")
case hours > 0:
return twoUnits(hours, "h", minutes, "m")
case minutes > 0:
return twoUnits(minutes, "m", seconds, "s")
default:
return fmt.Sprintf("%ds", seconds)
}
}
// twoUnits writes a count of a larger unit followed by a count of the
// next smaller one, leaving the smaller off when it is zero.
func twoUnits(
larger int64, largerUnit string, smaller int64, smallerUnit string,
) string {
if smaller == 0 {
return fmt.Sprintf("%d%s", larger, largerUnit)
}
return fmt.Sprintf("%d%s%d%s", larger, largerUnit, smaller, smallerUnit)
}
// loadWebhookStats gathers the figures for the statistics pane from the
// webhook's entrypoints and targets, as the page has already loaded
// them, and from its event database. It returns nil, and logs why, when
@@ -182,7 +139,7 @@ func (h *Handlers) loadWebhookStats(
func readEventStats(
db *gorm.DB, now time.Time, stats *WebhookStats,
) error {
err := readTotals(db, now, stats)
err := readTotals(db, stats)
if err != nil {
return err
}
@@ -212,9 +169,9 @@ func readEventStats(
}
// readTotals fills in the lifetime and within-retention figures, and
// when the last event arrived and how long before now that was, from
// the running totals: the events' row, and the targets' rows summed.
func readTotals(db *gorm.DB, now time.Time, stats *WebhookStats) error {
// when the last event arrived, from the running totals: the events'
// row, and the targets' rows summed.
func readTotals(db *gorm.DB, stats *WebhookStats) error {
var events database.EventTotals
err := db.Take(&events).Error
@@ -233,12 +190,6 @@ func readTotals(db *gorm.DB, now time.Time, stats *WebhookStats) error {
stats.WithinRetention.Events = events.Events - events.EventsRemoved
stats.LastEventAt = events.LastEventAt
if events.LastEventAt != nil {
stats.LastEvent = relativeTime(*events.LastEventAt, now)
stats.LastEventUTC =
events.LastEventAt.UTC().Format(time.DateTime) + " UTC"
}
for _, t := range targets {
stats.Lifetime.Deliveries += t.Deliveries
stats.Lifetime.Failures += t.Failed
+17 -62
View File
@@ -265,13 +265,6 @@ func statsPane(t *testing.T, page string) string {
return strings.Join(strings.Fields(pane), " ")
}
// statsLastEventShown matches the pane's last event for an event that
// arrived at at: a figure with the full time in UTC as its tooltip.
func statsLastEventShown(at time.Time) string {
return `Last event</span>\s*<span [^>]*title="` +
at.UTC().Format(time.DateTime) + ` UTC">[^<]+</span>`
}
// assertStatsTargets checks, for the history seedStatsHistory builds,
// each target's totals and its deliveries finished in the last 24
// hours. The first target has ten deliveries and the replay, the
@@ -307,17 +300,14 @@ func assertStatsPaneAfterPrune(
) {
t.Helper()
page := renderSourceDetailPage(t, h, sess, hist.webhook.ID)
pane := statsPane(t, page)
pane := statsPane(t, renderSourceDetailPage(t, h, sess, hist.webhook.ID))
lastEvent := hist.newest.CreatedAt.UTC().Format("2006-01-02 15:04:05 UTC")
// The newest event arrived moments ago, so the last event is a
// few seconds ago.
assert.Regexp(t, `Entrypoints 12 \(11 active\) `+
`Targets 6 \(2 active\) `+
`Deliveries in progress 13 `+
`Last event \d+s `+
`Retention 14 days`, pane)
assert.Regexp(t, statsLastEventShown(hist.newest.CreatedAt), page)
assert.Contains(t, pane, "Entrypoints 12 (11 active) "+
"Targets 6 (2 active) "+
"Deliveries in progress 13 "+
"Last event "+lastEvent+" "+
"Retention 14 days")
assert.Contains(t, pane, "Lifetime Within retention "+
"Events 10 9 "+
"Deliveries 21 18 "+
@@ -450,14 +440,15 @@ func TestWebhookStats_LastEventSurvivesPruningEveryEvent(t *testing.T) {
require.NotNil(t, stats.LastEventAt)
assert.True(t, arrived.Equal(*stats.LastEventAt))
assert.Regexp(t, statsLastEventShown(arrived),
renderSourceDetailPage(t, h, sess, wh.ID))
pane := statsPane(t, renderSourceDetailPage(t, h, sess, wh.ID))
assert.Contains(t, pane,
"Last event "+arrived.UTC().Format("2006-01-02 15:04:05 UTC"))
}
// TestWebhookStats_LastEventInUTC checks that the tooltip of the pane's
// last event gives the time in UTC, as the event list does, when the
// time was stored in another zone, as it is on a host whose local time
// is not UTC.
// TestWebhookStats_LastEventInUTC checks that the pane shows when the
// last event arrived in UTC, as the event list does, when the time was
// stored in another zone, as it is on a host whose local time is not
// UTC.
func TestWebhookStats_LastEventInUTC(t *testing.T) {
t.Parallel()
@@ -483,9 +474,8 @@ func TestWebhookStats_LastEventInUTC(t *testing.T) {
require.NoError(t, database.AddEventTotals(webhookDB,
database.EventTotals{Events: 1, LastEventAt: &arrived}))
assert.Regexp(t,
`Last event</span>\s*<span [^>]*title="2026-03-05 03:30:00 UTC">`,
renderSourceDetailPage(t, h, sess, wh.ID))
pane := statsPane(t, renderSourceDetailPage(t, h, sess, wh.ID))
assert.Contains(t, pane, "Last event 2026-03-05 03:30:00 UTC")
}
// TestWebhookStats_PaneShowsRetentionPeriod checks that the statistics
@@ -551,11 +541,8 @@ func TestWebhookStats_WebhookWithNoEvents(t *testing.T) {
assert.Equal(t, &handlers.WebhookStats{}, h.WebhookStatsForTest(wh.ID))
assert.Equal(t, "—", handlers.RecentWindow{}.FailurePercent())
page := renderSourceDetailPage(t, h, sess, wh.ID)
pane := statsPane(t, page)
pane := statsPane(t, renderSourceDetailPage(t, h, sess, wh.ID))
assert.Contains(t, pane, "Last event none")
assert.Regexp(t, `Last event</span>\s*<span class="[^"]*">none</span>`,
page, "a last event of none has a tooltip")
assert.Contains(t, pane, "Failure percentage — —")
assert.False(t, dbMgr.DBExists(wh.ID))
}
@@ -580,35 +567,3 @@ func TestRecentWindow_FailurePercent(t *testing.T) {
assert.Equal(t, tt.want, tt.window.FailurePercent(), tt.window)
}
}
// TestRelativeTime pins how the pane writes the time since the last
// event, at each boundary between its units.
func TestRelativeTime(t *testing.T) {
t.Parallel()
now := time.Date(2026, time.October, 26, 12, 0, 0, 0, time.UTC)
tests := []struct {
ago time.Duration
want string
}{
{0, "0s"},
{59900 * time.Millisecond, "59s"},
{60 * time.Second, "1m"},
{61 * time.Second, "1m1s"},
{3599 * time.Second, "59m59s"},
{3600 * time.Second, "1h"},
{3660 * time.Second, "1h1m"},
{86399 * time.Second, "23h59m"},
{86400 * time.Second, "1d"},
{90000 * time.Second, "1d1h"},
{400*24*time.Hour + 3*time.Hour, "400d3h"},
// An event time ahead of the server clock.
{-90 * time.Second, "0s"},
}
for _, tt := range tests {
assert.Equal(t, tt.want,
handlers.RelativeTimeForTest(now.Add(-tt.ago), now), tt.ago)
}
}
+1 -38
View File
@@ -61,8 +61,7 @@ const (
// event log in a headless browser, served by the real router and so
// under the real Content-Security-Policy, and checks that the pages'
// Alpine.js directives, the copy control and the delete prompts work,
// that a target's row shows its delivery counts, and that the
// Statistics block shows how long ago the last event arrived.
// and that a target's row shows its delivery counts.
func TestAlpineRunsUnderTheSecurityPolicy(t *testing.T) {
t.Parallel()
@@ -98,7 +97,6 @@ func TestAlpineRunsUnderTheSecurityPolicy(t *testing.T) {
checkCopy(ctx, t, page)
checkEntrypointEdit(ctx, t, page, page+"/events")
checkRecentEvents(ctx, t, page)
checkStatsLastEvent(ctx, t, env, webhook.ID, page)
checkArchiveChoice(ctx, t, srv.URL+"/hooks/new", page)
checkNewWebhookTargets(ctx, t, env, srv.URL+"/hooks/new")
checkRefusedNewWebhook(ctx, t, srv.URL+"/hooks/new")
@@ -833,41 +831,6 @@ func checkRecentEvents(ctx context.Context, t *testing.T, url string) {
"the event's own page does not show its body")
}
// checkStatsLastEvent records that the last event of the webhook
// webhookID arrived 3 days, 4 hours and 30 minutes ago, loads its page,
// and checks that the Statistics block shows the last event as "3d4h"
// with the full time in UTC as its tooltip.
func checkStatsLastEvent(
ctx context.Context, t *testing.T, env *testEnv, webhookID, url string,
) {
t.Helper()
webhookDB, err := env.dbMgr.GetDB(webhookID)
require.NoError(t, err)
ago := 3*24*time.Hour + 4*time.Hour + 30*time.Minute
arrived := time.Now().Add(-ago)
require.NoError(t, database.AddEventTotals(webhookDB,
database.EventTotals{LastEventAt: &arrived}))
figure := `//span[text()="Last event"]` +
`/following-sibling::span[text()="3d4h"]`
var (
title string
found bool
)
require.NoError(t, chromedp.Run(ctx, loadPage(url)))
require.True(t, shown(ctx, figure),
`the Statistics block does not show the last event as "3d4h"`)
require.NoError(t, chromedp.Run(ctx, chromedp.AttributeValue(
figure, "title", &title, &found, chromedp.BySearch,
)))
assert.Equal(t, arrived.UTC().Format(time.DateTime)+" UTC", title,
"the last event's tooltip is not the full time in UTC")
}
// checkEventLog loads the event log and checks that of its events only
// the newest, eventID, starts expanded: its row says so and its caret is
// turned up, and the log's last event, lastEventID, starts collapsed. In
+3
View File
@@ -13,6 +13,9 @@ import (
// TestBaseTemplateScriptsAreServed walks every /s/ script the base
// template loads on each page and fetches it through the real router.
// Alpine.js is extracted from its tarball in 3p/ at build time, so the
// file is not in the tree: this is the check that the page still gets
// the JavaScript it asks for.
func TestBaseTemplateScriptsAreServed(t *testing.T) {
t.Parallel()
Executable
+17
View File
@@ -0,0 +1,17 @@
#!/bin/sh
# script/assets: extract Alpine.js from its npm package tarball, committed
# in 3p/, to static/js/alpine.min.js, where go:embed reads it. The package
# is @alpinejs/csp, Alpine's build for pages whose Content-Security-Policy
# forbids eval. The extracted file is not committed. make build, make dev
# and the Dockerfile's lint, test and build stages run this first.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
tar -xzOf 3p/alpinejs-csp-3.14.9.tgz package/dist/cdn.min.js \
>static/js/alpine.min.js
}
main "$@"
-11
View File
File diff suppressed because one or more lines are too long
+3 -5
View File
@@ -6,11 +6,9 @@ import (
)
// Static holds the CSS and JavaScript files the web UI's pages load. They
// are named one by one, so a missing file fails the build instead of
// leaving the pages without it.
//
// js/alpine.min.js is Alpine.js, committed unchanged. It is
// package/dist/cdn.min.js from the npm package @alpinejs/csp 3.14.9.
// are named one by one so that a missing js/alpine.min.js, which make
// assets extracts and git does not track, fails the build instead of
// leaving the pages without Alpine.js.
//
//go:embed css/tailwind.css css/style.css js/app.js js/alpine.min.js
var Static embed.FS
+1 -5
View File
@@ -22,11 +22,7 @@
</div>
<div>
<span class="text-gray-500">Last event</span>
{{if .LastEvent}}
<span class="font-medium text-gray-900" title="{{.LastEventUTC}}">{{.LastEvent}}</span>
{{else}}
<span class="font-medium text-gray-900">none</span>
{{end}}
<span class="font-medium text-gray-900">{{with .LastEventAt}}{{.UTC.Format "2006-01-02 15:04:05 UTC"}}{{else}}none{{end}}</span>
</div>
<div>
<span class="text-gray-500">Retention</span>